Version: v3.x

Upgrading to project config v2

What has happened?

A new revision (2024.06.06) of the DDN CLI has been released. With this release, local project directory structures created by CLI versions <= 2024.06.05 are now out-of-date and deprecated.

This page contains some information on what has changed and instructions to migrate your existing local projects. However, for users to most efficiently familiarize themselves with the new CLI and project structure, we recommend setting up a new project from scratch by following the new getting started guide.

A newer CLI version has been released since!

A newer revision (v1.x.x) of the DDN CLI has been released. See the upgrade guide here.

If your local project was created by CLI versions <= 2024.06.05, you will need to update your local project directory using this current guide first before upgrading to the above revision.

What has changed?

The project config version defined in hasura.yaml is bumped from v1 to v2.
Local dev workflows using Docker have been introduced.
The DDN CLI commands. See details on the new commands here.
The local project directory structure.
SupergraphManifest and ConnectorManifest objects are now called Supergraph and Connector respectively.
The source field for Relationship objects has been deprecated in favour of the sourceType field. Though Relationship objects with the source field will continue to be supported.

We strongly recommend you go through our new getting started guide to learn about all the changes and experience the new workflow to develop your API with DDN.

No changes to existing DDN projects

Note that this a CLI only change and does not impact Hasura DDN projects. You can continue using existing DDN projects and builds and also be able to create new builds using the new CLI revision (with the new local project structure).

Migrate an existing local project

Below are steps to convert a local project created with the DDN CLI versions <= v2024.06.05 to the new project structure.

Step 1: Get the new CLI

You can download the CLI binary below. Please follow the instructions for your system.

macOS and Linux
Windows

Simply run the installer script in your terminal:

curl -L https://graphql-engine-cdn.hasura.io/ddn/cli/v2/get.sh | bash

Download the latest cli-ddn-windows-amd64.exe binary and run it.

curl -L https://graphql-engine-cdn.hasura.io/ddn/cli/v2/latest/cli-ddn-windows-amd64.exe -o ddn.exe

Unrecognized application warning

In Windows, if you get an "Unrecognized application" warning, click "Run anyway".

Additionally, you'll need to add this executable to your system's PATH:

Step 1: Locate the and rename the executable: Find the full path to the directory containing the executable. For example, if the executable is located in C:\Program Files\cli-ddn-windows-amd64.exe, note this path. Also, rename it to ddn for ease when writing commands later on.

Step 2: Open System Properties:

Press Win + X and select System.
Click on Advanced system settings on the left side.
In the System Properties window, click on the Environment Variables button.

Step 3: Edit the PATH Variable:

In the Environment Variables window, find the System variables section.
Scroll down and select the Path variable, then click Edit.
In the Edit Environment Variable window, click New and paste the path to the directory containing your executable (e.g., C:\Program Files\ddn.exe).

Step 4: Save Your Changes:

Click OK to close the Edit Environment Variable window.
Click OK to close the Environment Variables window.
Click OK to close the System Properties window.

Step 5: Verify the Changes: Open a new Command Prompt or PowerShell window and type the following.

From any directory, run:

ddn

Also, update your Hasura VS Code extension to v0.6.x if not done automatically.

Need the previous CLI revision?

Note that the above will overwrite your current DDN CLI revision. You can re-install the previous revision by replacing /ddn/cli/v2/ with /ddn/cli/v1/ in the above download command.

Step 2: Set up a fresh local project

Step 2.1: Initialize a new supergraph

Run:

ddn supergraph init --dir <new-proj-dir>

Set the default supergraph config file to the CLI context to avoid having to repeat it in future commands. This file contains the Supergraph object similar to the SupergraphManifest object in your existing project directory.

cd <new-proj-dir> && ddn context set supergraph ./supergraph.local.yaml

Step 2.2: Copy over supergraph global objects

Copy the files containing the supergraph global objects CompatibilityConfig, GraphqlConfig and AuthConfig from your existing project directory to the supergraph_globals directory in the new project directory. These should typically be in the supergraph directory in your existing project directory.

Note that these objects are already created in the supergraph_globals directory of your new project directory. The AuthConfig object created here is to be used for local development. To avoid overwriting it, while copying your auth-config.hml file from the existing project directory rename it to auth-config.cloud.hml instead. The graphql-config.hml and compatibility-config.hml files on the other hand should be overwritten.

Step 3: Add your subgraphs

For each subgraph in your project:

Follow the below steps:

Step 3.1: Initialize the subgraph

Run:

ddn subgraph init <subgraph-name>

Step 3.2: Add an env file for deploying to DDN

A .env.<subgraph-name> file is created in the <subgraph-name> directory to be used during local development.

Create a new file .env.cloud.<subgraph-name> alongside it to be used for deployments to DDN. You can leave this file empty for now.

Step 4: Set up your data connectors

Case 1:

For self-deployed data connectors in your subgraphs, i.e. ConnectorManifest of type endpoints:

You can skip this step.

Case 2:

For each data connector deployed on Hasura DDN in your subgraphs, i.e. ConnectorManifest of type cloud:

Follow the steps below:

Step 4.1: Initialize data connector

Run:

ddn connector init <connector-name> --subgraph <subgraph-name> --hub-connector <connector-type>

Ensure you use the same connector name and type from your existing ConnectorManifest.

Step 4.2: Create connector configuration files for deploying to DDN

Step 4.2.1: Create a connector.cloud.yaml

Each connector utilizes a connector.yaml file for local development. This file contains the Connector object similar to the ConnectorManifest object in your existing project.

To deploy your connector to DDN create a connector.cloud.yaml file alongside the connector's local connector.yaml. You can simply copy the contents of the connector.yaml file and update the envFile key's value to .env.cloud which we'll create now.

For example, <subgraph-name>/connector/<connector-name>/connector.cloud.yaml
kind: Connector
version: v1
definition:
  name: <connector-name>
  subgraph: <subgraph-name>
  source: hasura/<connector-type>:<version>
  context: .
  envFile: .env.cloud

Step 4.2.2: Create a .env.cloud for the connector

Create an environment variable file .env.cloud alongside the .env.local file for the connector. You can leave this file empty for now.

Step 4.3: Add data connector configuration

Case 1: For hasura/postgres connectors

Step 4.3.1: Set database connection string

Add a new key CONNECTION_URI to the file <subgraph_name>/connector/<connector_name>/.env.cloud and set your database connection string as the value.

For example:

CONNECTION_URI=<your_pg_url>

Step 4.3.2: Introspect the database schema

Run:

ddn connector introspect --connector <subgraph-name>/connector/<connector-name>/connector.cloud.yaml

Case 2: For hasura/nodejs connectors

Step 4.3.1: Copy over functions

Copy the file(s) containing your TS functions from your existing project directory to the <subgraph-name>/connector/<connector-name> directory in the new project directory. This file should typically be the functions.ts file in the <subgraph-name>/<connector-name>/connector directory in your existing project directory.

For each data connector:

Follow the below steps:

Copy the files containing metadata objects related to the connector from your existing project directory to the <subgraph-name>/metadata directory in the new project directory (the directory might need to be created). These files should typically be in the <subgraph-name>/<connector-name> directory in your existing project directory and be suffixed as .hml.

the file with the DataConnectorLink object. Typically <connector-name>.hml
the file with the data connector type objects. Typically <connector-name>-types.hml
the files with Models, Commands, Relationships, etc. Typically in models/commands directory.

Within the <subgraph-name>/metadata directory you can choose to arrange these files however you like.

Step 5.2: Update DataConnectorLink object

In the DataConnectorLink object replace the url section with:

url:
    readWriteUrls:
      read:
        valueFromEnv: <CONNECTOR_NAME>_READ_URL
      write:
        valueFromEnv: <CONNECTOR_NAME>_WRITE_URL

We will set the values of these env vars in the following steps.

Step 5.3: (Optional) Update Relationship objects

As mentioned earlier in the what-has-changed section, the source field for Relationship objects has been deprecated in favour of the sourceType field. Though Relationship objects with the source field will continue to be supported, it is recommended to update them.

To update the relationship objects, rename the source key in your objects to sourceType.

For example:

kind: Relationship
version: v1
definition:
  name: <rel-name>
  sourceType: <source-object-type>
  target: ...

Step 6: Deploy to a DDN project

Step 6.1: Set the DDN project to deploy to

Run the following command to set your DDN project to the CLI context to avoid having to repeat it in future commands. You can find your project name in the hasura.yaml file in your existing project directory.

ddn context set project <project-name>

Step 6.2: Deploy the data connectors

For each data connector deployed on Hasura DDN in your subgraphs, i.e. ConnectorManifest of type cloud:

Run:

ddn connector build create --connector <subgraph-name>/connector/<connector-name>/connector.cloud.yaml

On deployment completion, the read and write URLs for your deployed connector will be returned as a response. You will need these in the next step.

Step 6.3: Set connector endpoints in env files

For each subgraph:

Update the file <subgraph-name>/.env.cloud.<subgraph-name> with the read and write URLs of the data connectors in the subgraph.

For a self-deployed data connector in your subgraph, i.e. ConnectorManifest of type endpoints, use the connector URL provided in the existing ConnectorManifest as both the read and write URLs.

For data connectors deployed on Hasura DDN in your subgraph, use the connector URLs returned in the previous deploy step.

For example:

<subgraph-name>/.env.cloud.<subgraph-name>:
<CONNECTOR_NAME_1>_READ_URL=<connector1-read-url>
<CONNECTOR_NAME_1>_WRITE_URL=<connector1-write-url>
<CONNECTOR_NAME_2>_READ_URL=<connector2-read-url>
<CONNECTOR_NAME_2>_WRITE_URL=<connector2-write-url>

Step 6.4: Deploy the supergraph

Step 6.4.1: Create a supergraph config file for deploying to DDN

In the root of the new project directory, create a new file supergraph.cloud.yaml and copy the contents of the existing supergraph.yaml file to it.

Update the following values in the new file:

./supergraph_globals/auth-config.hml -> To the name of the AuthConfig file copied earlier. e.g. ./supergraph_globals/auth-config.cloud.hml
For each subgraph: envFile: <subgraph-name>/.env.<subgraph-name> -> To the name of the cloud env files created earlier. e.g. envFile: <subgraph-name>/.env.cloud.<subgraph-name>

For example:

supergraph.cloud.yaml
kind: Supergraph
version: v1
definition:
  supergraph_globals:
    generator:
      rootPath: ./supergraph_globals
    envFile: ./supergraph_globals/.env.supergraph_globals
    includePaths:
      - ./supergraph_globals/auth-config.cloud.hml
      - ./supergraph_globals/compatibility-config.hml
      - ./supergraph_globals/graphql-config.hml
  subgraphs:
    <subgraph-name>:
      generator:
        rootPath: <subgraph-name>/metadata
      envFile: <subgraph-name>/.env.cloud.<subgraph-name>
      includePaths:
        - <subgraph-name>/metadata

Step 6.4.2: Build and deploy your supergraph

Run:

ddn supergraph build create --supergraph ./supergraph.cloud.yaml

On build completion, the build version, API endpoint and console URLs will be returned as response.

Step 6.5: Verify migration

You can now head to your project console using the console URL returned in the previous step and verify the API generated with the above build is the same as what you had earlier.

Need help?

If you need help migrating your project or have any other questions please reach out to us on our Discord.

Legacy project structure

See the legacy project structure before this update below.

File type	Description
`hasura.yaml`	The main configuration file.
`*.env.yaml`	A file that stores environment variables.
`*.supergraph.yaml`	Build manifest file(s) for a project.
`*.hml`	Hasura metadata files for a project.
`supergraph`	A directory, containing all the necessary configuration and metadata files, for the supergraph-level scope of a project.
`<SUBGRAPH_DIRECTORY>`	A directory, containing all the necessary configuration and metadata files, for a corresponding subgraph within a project.

hasura.yaml

This is the entrypoint to a Hasura project.

version: v1
project: <PROJECT_NAME>
subgraphs:
  app:
    path: ./app
defaultSupergraphManifest: base

The version section is used to specify the version of the configuration file. The project field is used to specify the project name.

The hasura.yaml file also contains a subgraphs section. This section is used to specify the various subgraphs associated with the project.

*.env.yaml

This file is used to store environment variables. The CLI generates a base.env.yaml for you by default. You can add environment variables by adding key-value pairs under a particular subgraph:

supergraph: {}
subgraphs:
  app:
    APP_CONNECTOR_CONNECTION_URI: "CONNECTION_URI"

And then referencing them in your metadata using valueFromEnv:

kind: ConnectorManifest
version: v1
spec:
  supergraphManifests:
    - base
definition:
  name: app_connector
  type: cloud
  connector:
    type: hub
    name: <CONNECTOR_NAME>
  deployments:
    - context: .
      env:
        CONNECTION_URI:
          valueFromEnv: APP_CONNECTOR_CONNECTION_URI

Accessing environment variables

In the example above, our supergraphManifests array references our base.supergraph.hml file, which contains an envfile key that maps to our base.env.yaml. Thus, any environment variables present in this YAML file will then be accessible in our HML.

If these files contain sensitive information, you should add them to your gitignore so as to avoid accidentally committing them to version control.

Supergraph manifests

Supergraph manifests tell Hasura DDN how to construct your supergraph. A manifest will contain information such as which subgraphs to include and which resources to use for the build.

kind: SupergraphManifest
version: v1
definition:
  name: base
  envfile: base.env.yaml
  subgraphs:
    - app

Supergraph

The supergraph directory contains the supergraph configuration files. The three included files are:

├── auth-config.hml
├── compatibility-config.hml
└── graphql-config.hml

Organizing files

The contents of these files include the necessary metadata objects to build your supergraph. We've separated them out into files organized by their purpose, but you can organize them anyway you like so long as the metadata objects within them are all included.

Each of these files' contents are used to configure the supergraph.

The auth-config.hml file contains the authentication configuration for the supergraph, allowing you to utilize either JWTs or webhooks.

The compatibility-config.hml file contains the compatibility configuration — using a version number — for the supergraph.

The graphql-config.hml file contains the GraphQL configuration for the supergraph, which allows you to customize the available query and mutation capabilities along with the schema.

Subgraphs

Each subgraph is listed as a top-level directory in the root of the project. The CLI will init your project with an app subgraph.

Each subgraph directory will contain the following folder structure when a new connector manifest is added for the subgraph:

└── <CONNECTOR_NAME>
│   ├── <CONNECTOR_NAME>-types.hml
│   ├── <CONNECTOR_NAME>.hml
│   ├── connector
│   │   ├── <CONNECTOR_NAME>.build.hml
│   │   ├── configuration.json
│   │   └── schema.json
│   └── models

File/Folder	Description
`<CONNECTOR_NAME>`	The directory containing the build artifacts for the connector.
`<CONNECTOR_NAME>-types.html`	The metadata file for the connector's types.
`<CONNECTOR_NAME>.hml`	The metadata file for the connector based on the introspection of the data source.
`<CONNECTOR_NAME>/configuration.json`	The configuration file for the connector.
`<CONNECTOR_NAME>/schema.json`	The schema file for the connector.
`/connector/<CONNECTOR_NAME>.build.hml`	The build manifest for the connector.

Upgrading to project config v2

What has happened?​

What has changed?​

Migrate an existing local project​

Step 1: Get the new CLI​

Step 2: Set up a fresh local project​

Step 2.1: Initialize a new supergraph​

Step 2.2: Copy over supergraph global objects​

Step 3: Add your subgraphs​

Step 3.1: Initialize the subgraph​

Step 3.2: Add an env file for deploying to DDN​

Step 4: Set up your data connectors​

Step 4.1: Initialize data connector​

Step 4.2: Create connector configuration files for deploying to DDN​

Step 4.2.1: Create a connector.cloud.yaml​

Step 4.2.2: Create a .env.cloud for the connector​

Step 4.3: Add data connector configuration​

Case 1: For hasura/postgres connectors​

Step 4.3.1: Set database connection string​

Step 4.3.2: Introspect the database schema​

Case 2: For hasura/nodejs connectors​

Step 4.3.1: Copy over functions​

Step 5: Bring over connector related metadata​

Step 5.1: Copy over connector related metadata files​

Step 5.2: Update DataConnectorLink object​

Step 5.3: (Optional) Update Relationship objects​

Step 6: Deploy to a DDN project​

Step 6.1: Set the DDN project to deploy to​

Step 6.2: Deploy the data connectors​

Step 6.3: Set connector endpoints in env files​

Step 6.4: Deploy the supergraph​

Step 6.4.1: Create a supergraph config file for deploying to DDN​

Step 6.4.2: Build and deploy your supergraph​

Step 6.5: Verify migration​

Need help?​

Legacy project structure​

hasura.yaml​

*.env.yaml​

Supergraph manifests​

Supergraph​

Subgraphs​

Was this helpful?

What has happened?

What has changed?

Migrate an existing local project

Step 1: Get the new CLI

Step 2: Set up a fresh local project

Step 2.1: Initialize a new supergraph

Step 2.2: Copy over supergraph global objects

Step 3: Add your subgraphs

Step 3.1: Initialize the subgraph

Step 3.2: Add an env file for deploying to DDN

Step 4: Set up your data connectors

Step 4.1: Initialize data connector

Step 4.2: Create connector configuration files for deploying to DDN

Step 4.2.1: Create a connector.cloud.yaml

Step 4.2.2: Create a .env.cloud for the connector

Step 4.3: Add data connector configuration

Case 1: For hasura/postgres connectors

Step 4.3.1: Set database connection string

Step 4.3.2: Introspect the database schema

Case 2: For hasura/nodejs connectors

Step 4.3.1: Copy over functions

Step 5: Bring over connector related metadata

Step 5.1: Copy over connector related metadata files

Step 5.2: Update DataConnectorLink object

Step 5.3: (Optional) Update Relationship objects

Step 6: Deploy to a DDN project

Step 6.1: Set the DDN project to deploy to

Step 6.2: Deploy the data connectors

Step 6.3: Set connector endpoints in env files

Step 6.4: Deploy the supergraph

Step 6.4.1: Create a supergraph config file for deploying to DDN

Step 6.4.2: Build and deploy your supergraph

Step 6.5: Verify migration

Need help?

Legacy project structure

hasura.yaml

*.env.yaml

Supergraph manifests

Supergraph

Subgraphs