Upgrading to supergraph config v2

What has happened?

A new revision (v1.x.x) of the DDN CLI has been released with some changes to the project directory structure and CLI commands. With this release, local project directory structures created by CLI versions <= 2024.07.09 are now deprecated.

This page contains some information on what has changed and instructions to migrate your existing local projects.

A newer CLI version has been released since!

A newer revision (v2.x.x) of the DDN CLI has been released. See the upgrade guide here.

If your local project was created by CLI versions < v2.0.0, you will need to update your local project directory using this current guide first before upgrading to the above revision.

What has changed?

• The project directory structure
  • See details on the updated directory structure here.
  • The Supergraph object is upgraded from v1 to v2.
  • A Subgraph object is introduced to define how to build a subgraph.
  • The AuthConfig, GraphqlConfig and CompatibilityConfig metadata objects are now not defined as supergraph globals but can instead be defined in any subgraph of the supergraph. Note that these are still supergraph-level objects and only one of these objects can be defined across the supergraph, but they can now be defined in any of the subgraphs based on convenience. By default, these are now added to a new globals subgraph.
• The --subgraph flag in DDN CLI commands now accepts a path to a subgraph config file as opposed to the subgraph name.
• The DDN CLI now uses SemVer instead of CalVer for versioning.
• The otel-collector version is now updated to the latest version and the otel-collector-config.yaml is now updated accordingly.

No changes to existing DDN projects

Note that this is a CLI-only change and does not impact Hasura DDN projects. You can continue using existing DDN projects and builds and also be able to create new builds on them with both the new and old CLI revisions.

Though we strongly recommend updating to the latest version of the CLI and project structure by following the steps below.

Migrate an existing local project

Below are steps to convert a local project created with the DDN CLI versions <= v2024.07.09 to the new project structure.

Step 1: Get the new CLI

You can download the CLI binary below. Please follow the instructions for your system.

macOS and Linux

Simply run the installer script in your terminal:

curl -L https://graphql-engine-cdn.hasura.io/ddn/cli/v3/get.sh | bash

Windows

Download the latest cli-ddn-windows-amd64.exe binary and run it.

curl -L https://graphql-engine-cdn.hasura.io/ddn/cli/v3/latest/cli-ddn-windows-amd64.exe -o ddn.exe

Unrecognized application warning

In Windows, if you get an "Unrecognized application" warning, click "Run anyway".

Additionally, you'll need to add this executable to your system's PATH:

Step 1: Locate the and rename the executable: Find the full path to the directory containing the executable. For example, if the executable is located in C:\Program Files\cli-ddn-windows-amd64.exe, note this path. Also, rename it to ddn for ease when writing commands later on.

Step 2: Open System Properties:

• Press Win + X and select System.
• Click on Advanced system settings on the left side.
• In the System Properties window, click on the Environment Variables button.

Step 3: Edit the PATH Variable:

• In the Environment Variables window, find the System variables section.
• Scroll down and select the Path variable, then click Edit.
• In the Edit Environment Variable window, click New and paste the path to the directory containing your executable (e.g., C:\Program Files\ddn.exe).

Step 4: Save Your Changes:

• Click OK to close the Edit Environment Variable window.
• Click OK to close the Environment Variables window.
• Click OK to close the System Properties window.

Step 5: Verify the Changes: Open a new Command Prompt or PowerShell window and type the following.

From any directory, run:

ddn

Also, update your Hasura VS Code extension to v1.x.x if not done automatically.

Need the previous CLI revision?

Note that the above will overwrite your current DDN CLI revision. You can re-install the previous revision by replacing /ddn/cli/v3/ with /ddn/cli/v2/ in the above download command.

Step 2: Upgrade existing Supergraph config v1 objects

You can update your existing Supergraph config v1 objects in your local project by running the following in your project directory:

ddn codemod upgrade-supergraph-config-v1-to-v2 --dir <project-dir-path>

This will detect any existing Supergraph v1 objects in your project directory and initialize a wizard to update them to the v2 format and make any other related changes, such as initializing the newly introduced Subgraph config objects and setting up the new globals subgraph.

Step 3: Create new globals subgraph on the DDN project

Run the following command to create a new globals subgraph on your DDN project:

ddn project subgraph create globals --project <project-name>

Step 4: Verify the migration

Create a new supergraph build using:

ddn supergraph build create --supergraph ./supergraph.cloud.yaml

On build completion, the build version, API endpoint, and console URLs will be returned as in the response.

You can now head to your project console using the console URL returned in the previous step and verify the API generated with the above build is the same as what you had earlier.

Step 5: Update the OpenTelemetry exporter config

Replace the contents of the file otel-collector-config.yaml (present at the root of the project by default) with the following:

exporters:
  otlp:
    endpoint: https://gateway.otlp.hasura.io:443
    headers:
      Authorization: pat ${env:HASURA_DDN_PAT}
processors:
  batch: {}
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
service:
  pipelines:
    logs:
      exporters:
        - otlp
      processors:
        - batch
      receivers:
        - otlp
    metrics:
      exporters:
        - otlp
      processors:
        - batch
      receivers:
        - otlp
    traces:
      exporters:
        - otlp
      processors:
        - batch
      receivers:
        - otlp

You can now pin the version of otel-collector to a specific version (say 0.104.0) to ensure breaking changes to the OpenTelemetry collector config does not affect your local dev by setting the field $.services.otel-collector.image in the docker-compose.hasura.yaml file to otel/opentelemetry-collector:0.104.0.

Need help?

If you need help migrating your project or have any other questions please reach out to us on our Discord.

Legacy project structure

See the legacy project structure before this update below.

.
├── .devcontainer
│  └── devcontainer.json
├── .hasura
│  └── context.yaml
├── .vscode
│  ├── extensions.json
│  ├── launch.json
│  └── tasks.json
├── engine
│  ├── .env.engine
│  ├── auth_config.json
│  ├── metadata.json
│  └── open_dd.json
├── globals
│  ├── .env.globals.cloud
│  ├── .env.globals.local
│  ├── subgraph.cloud.yaml
│  ├── subgraph.local.yaml
│  ├── auth-config.cloud.hml
│  ├── auth-config.local.hml
│  ├── compatibility-config.hml
│  └── graphql-config.hml
├── app
│  ├── .env.app.cloud
│  ├── .env.app.local
│  └── subgraph.yaml
├── .gitattributes
├── compose.yaml
├── hasura.yaml
├── otel-collector-config.yaml
├── supergraph.cloud.yaml
└── supergraph.local.yaml

File type	Description
hasura.yaml	The file that denotes the root of a Hasura project.
.env.*	Files that store environment variables.
supergraph.*.yaml	Files that describe how to build the supergraph.
subgraph.*.yaml	Files that describe how to build a subgraph.
connector.*.yaml	Files that describe how to build a data connector.
*.hml	Hasura metadata files for a project.
.hasura/context.yaml	File that stores certain default values of a project.
compose.yaml	Docker Compose files for local development.
engine	Files mounted to the engine container for local development.
globals	Directory containing files for the globals subgraph.
otel-collector-config.yaml	Configuration for OpenTelemetry Collector used for seeing traces during local development.

hasura.yaml

hasura.yaml appears at the root of a Hasura project.

For example:

version: v2

• version specifies the version of the project directory structure.

.env.*

This file is used to store environment variables. The file should be in the format:

ENV1=val1
ENV2=val2

These files are referenced inside:

• Docker Compose files: Docker Compose files use *.env files to specify environment variables needed for containers.
• Subgraph files: Subgraph files use *.env files to specify environment variables. This is needed by globals and other subgraphs. Each subgraph can have its own .env file.
• Connector files (such as connector.yaml): Connector files use *.env files to specify environment variables needed for building the connector.

supergraph.*.yaml

These config files tell Hasura DDN how to construct your supergraph. It will contain information such as which subgraph config files to use for building the supergraph

For example:

kind: Supergraph
version: v2
definition:
  subgraphs:
    - globals/subgraph.cloud.yaml
    - app/subgraph.yaml

• version is the version of the supergraph object
• subgraphs specifies paths to all the subgraph config files. These are then read recursively to construct the metadata for the supergraph.

By default, the CLI generates two files - supergraph.cloud.yaml and supergraph.local.yaml but any number of supergraph config files can be created and referenced in CLI commands.

subgraph.*.yaml

These config files tell Hasura DDN how to construct your subgraph. It will contain information such as which metadata resources to use for the build.

For example:

kind: Subgraph
version: v1
definition:
  name: app
  generator:
    rootPath: .
  includePaths:
    - metadata

• version is the version of the subgraph object
• includePaths specifies the directories and files where metadata for the subgraph can be found. If a directory is specified, all the *.hml files inside the directory and its subdirectories will be used to construct the metadata.
• generator.rootPath specifies the directory into which any new files will be generated.

By default, the CLI generates a file called subgraph.yaml for a new subgraph but any number of subgraph config files can be created and referenced in CLI commands.

connector.*.yaml

These config files tell Hasura DDN how to build your connector. It will contain information such as the type of connector and the location to the context files needed to build the connector.

For example:

kind: Connector
version: v1
definition:
  name: mypg
  source: hasura/postgres:v0.7.0
  context: .
  envFile: .env.local

• version is the version of the connector object
• name is a name given to the connector
• source is the connector to use specific to the data source
• context specifies the connector directory
• envFile specifies the connector specific environment variables to use for introspecting and building the connector. If you are deploying your connector on DDN cloud, you also need to specify subgraph. Value of this field is name of the subgraph you want to deploy your connector to.

By default, the CLI generates two files - connector.cloud.yaml and connector.local.yaml but any number of connector config files can be created and referenced in CLI commands.

.hasura/context.yaml

This specifies the default DDN project and supergraph file path. The default values are used by all commands that accept --supergraph flag, --subgraph flag and --project flag. The flags can be used to override the default values.

context:
  project: emerging-stag-9129
  supergraph: ../supergraph.cloud.yaml
  subgraph: ../app/subgraph.yaml

This file is configured by the ddn context set command.

Engine

The engine directory contains the files required for Hasura v3 engine container. This directory has the following structure:

├── .env.engine
├── auth_config.json
├── metadata.json
└── open_dd.json

The .env.engine file specifies environment variables required by the Hasura v3 engine container.

The auth_config.json, metadata.json and open_dd.json are generated as a result of ddn supergraph build local command and do not need to be edited by the user.

Globals

The globals directory contains the files for the globals subgraph which is generated by default to hold the supergraph-level metadata objects, i.e. AuthConfig, GraphqlConfig and CompatibilityConfig.

For example:

├── .env.globals.cloud
├── .env.globals.local
├── auth-config.cloud.hml
├── auth-config.local.hml
├── compatibility-config.hml
├── graphql-config.hml
├── subgraph.cloud.yaml
└── subgraph.local.yaml

• auth-config.cloud.hml and auth-config.local.hml files contain the AuthConfig object which define the authentication configuration for the supergraph for cloud and local deployments respectively.
• compatibility-config.hml file contains the compatibility date configuration for the supergraph.
• graphql-config.hml file contains the GraphQL configuration for the supergraph, which allows you to customize the available query and mutation capabilities along with the schema.
• .env.globals.cloud and .env.globals.local files contain all the environment variables, if any, which are required by the globals subgraph's metadata objects for cloud and local deployments respectively.