Upgrading to supergraph config v2
What has happened?
A new revision (v1.x.x) of the DDN CLI has been released with some changes to the project directory structure and CLI commands. With this release, local project directory structures created by CLI versions <= 2024.07.09 are now deprecated.
This page contains some information on what has changed and instructions to migrate your existing local projects.
What has changed?
- The project directory structure
- See details on the updated directory structure here.
- The
Supergraph
object is upgraded fromv1
tov2
. - A
Subgraph
object is introduced to define how to build a subgraph. - The
AuthConfig
,GraphqlConfig
andCompatibilityConfig
metadata objects are now not defined as supergraph globals but can instead be defined in any subgraph of the supergraph. Note that these are still supergraph-level objects and only one of these objects can be defined across the supergraph, but they can now be defined in any of the subgraphs based on convenience. By default, these are now added to a new globals subgraph.
- The
--subgraph
flag in DDN CLI commands now accepts a path to a subgraph config file as opposed to the subgraph name. - The DDN CLI now uses SemVer instead of CalVer for versioning.
- The
otel-collector
version is now updated to the latest version and theotel-collector-config.yaml
is now updated accordingly.
Note that this is a CLI-only change and does not impact Hasura DDN projects. You can continue using existing DDN projects and builds and also be able to create new builds on them with both the new and old CLI revisions.
Though we strongly recommend updating to the latest version of the CLI and project structure by following the steps below.
Migrate an existing local project
Below are steps to convert a local project created with the DDN CLI versions <= v2024.07.09
to the new project
structure.
Step 1: Get the new CLI
You can download the CLI binary below. Please follow the instructions for your system.
- macOS and Linux
- Windows
Simply run the installer script in your terminal:
curl -L https://graphql-engine-cdn.hasura.io/ddn/cli/v3/get.sh | bash
Download the latest cli-ddn-windows-amd64.exe
binary and run it.
curl -L https://graphql-engine-cdn.hasura.io/ddn/cli/v3/latest/cli-ddn-windows-amd64.exe -o ddn.exe
In Windows, if you get an "Unrecognized application" warning, click "Run anyway".
Additionally, you'll need to add this executable to your system's PATH:
Step 1: Locate the and rename the executable: Find the full path to the directory containing the executable. For
example, if the executable is located in C:\Program Files\cli-ddn-windows-amd64.exe
, note this path. Also, rename it
to ddn
for ease when writing commands later on.
Step 2: Open System Properties:
- Press
Win + X
and select System. - Click on Advanced system settings on the left side.
- In the System Properties window, click on the Environment Variables button.
Step 3: Edit the PATH Variable:
- In the Environment Variables window, find the System variables section.
- Scroll down and select the Path variable, then click Edit.
- In the Edit Environment Variable window, click New and paste the path to the directory containing your executable
(e.g.,
C:\Program Files\ddn.exe
).
Step 4: Save Your Changes:
- Click OK to close the Edit Environment Variable window.
- Click OK to close the Environment Variables window.
- Click OK to close the System Properties window.
Step 5: Verify the Changes: Open a new Command Prompt or PowerShell window and type the following.
ddn
Also, update your Hasura VS Code extension to v1.x.x
if not done automatically.
Note that the above will overwrite your current DDN CLI revision. You can re-install the previous revision by replacing
/ddn/cli/v3/
with /ddn/cli/v2/
in the above download command.
Step 2: Upgrade existing Supergraph config v1 objects
You can update your existing Supergraph config v1 objects in your local project by running the following in your project directory:
ddn codemod upgrade-supergraph-config-v1-to-v2 --dir <project-dir-path>
This will detect any existing Supergraph v1 objects in your project directory and initialize a wizard to update them to the v2 format and make any other related changes, such as initializing the newly introduced Subgraph config objects and setting up the new globals subgraph.
Step 3: Create new globals subgraph on the DDN project
Run the following command to create a new globals subgraph on your DDN project:
ddn project subgraph create globals --project <project-name>
Step 4: Verify the migration
Create a new supergraph build using:
ddn supergraph build create --supergraph ./supergraph.cloud.yaml
On build completion, the build version, API endpoint, and console URLs will be returned as in the response.
You can now head to your project console using the console URL returned in the previous step and verify the API generated with the above build is the same as what you had earlier.
Step 5: Update the OpenTelemetry exporter config
Replace the contents of the file otel-collector-config.yaml
(present at the root of the project by default) with the
following:
exporters:
otlp:
endpoint: https://gateway.otlp.hasura.io:443
headers:
Authorization: pat ${env:HASURA_DDN_PAT}
processors:
batch: {}
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
service:
pipelines:
logs:
exporters:
- otlp
processors:
- batch
receivers:
- otlp
metrics:
exporters:
- otlp
processors:
- batch
receivers:
- otlp
traces:
exporters:
- otlp
processors:
- batch
receivers:
- otlp
You can now pin the version of otel-collector
to a specific version (say 0.104.0
) to ensure breaking changes to the
OpenTelemetry collector config does not affect your local dev by setting the field $.services.otel-collector.image
in
the docker-compose.hasura.yaml
file to otel/opentelemetry-collector:0.104.0
.
Need help?
If you need help migrating your project or have any other questions please reach out to us on our Discord.
Legacy project structure
See the legacy project structure before this update below.
.
├── .devcontainer
│ └── devcontainer.json
├── .hasura
│ └── context.yaml
├── .vscode
│ ├── extensions.json
│ ├── launch.json
│ └── tasks.json
├── engine
│ ├── .env.engine
│ ├── auth_config.json
│ ├── metadata.json
│ └── open_dd.json
├── globals
│ ├── .env.globals.cloud
│ ├── .env.globals.local
│ ├── subgraph.cloud.yaml
│ ├── subgraph.local.yaml
│ ├── auth-config.cloud.hml
│ ├── auth-config.local.hml
│ ├── compatibility-config.hml
│ └── graphql-config.hml
├── app
│ ├── .env.app.cloud
│ ├── .env.app.local
│ └── subgraph.yaml
├── .gitattributes
├── compose.yaml
├── hasura.yaml
├── otel-collector-config.yaml
├── supergraph.cloud.yaml
└── supergraph.local.yaml
File type | Description |
---|---|
hasura.yaml | The file that denotes the root of a Hasura project. |
.env.* | Files that store environment variables. |
supergraph.*.yaml | Files that describe how to build the supergraph. |
subgraph.*.yaml | Files that describe how to build a subgraph. |
connector.*.yaml | Files that describe how to build a data connector. |
*.hml | Hasura metadata files for a project. |
.hasura/context.yaml | File that stores certain default values of a project. |
compose.yaml | Docker Compose files for local development. |
engine | Files mounted to the engine container for local development. |
globals | Directory containing files for the globals subgraph. |
otel-collector-config.yaml | Configuration for OpenTelemetry Collector used for seeing traces during local development. |
hasura.yaml
hasura.yaml
appears at the root of a Hasura project.
For example:
version: v2
version
specifies the version of the project directory structure.
.env.*
This file is used to store environment variables. The file should be in the format:
ENV1=val1
ENV2=val2
These files are referenced inside:
- Docker Compose files: Docker Compose files use *.env files to specify environment variables needed for containers.
- Subgraph files: Subgraph files use *.env files to specify environment variables. This is needed by
globals
and other subgraphs. Each subgraph can have its own .env file. - Connector files (such as
connector.yaml
): Connector files use *.env files to specify environment variables needed for building the connector.
supergraph.*.yaml
These config files tell Hasura DDN how to construct your supergraph. It will contain information such as which subgraph config files to use for building the supergraph
For example:
kind: Supergraph
version: v2
definition:
subgraphs:
- globals/subgraph.cloud.yaml
- app/subgraph.yaml
version
is the version of the supergraph objectsubgraphs
specifies paths to all the subgraph config files. These are then read recursively to construct the metadata for the supergraph.
By default, the CLI generates two files - supergraph.cloud.yaml
and supergraph.local.yaml
but any number of
supergraph config files can be created and referenced in CLI commands.
subgraph.*.yaml
These config files tell Hasura DDN how to construct your subgraph. It will contain information such as which metadata resources to use for the build.
For example:
kind: Subgraph
version: v1
definition:
name: app
generator:
rootPath: .
includePaths:
- metadata
version
is the version of the subgraph objectincludePaths
specifies the directories and files where metadata for the subgraph can be found. If a directory is specified, all the *.hml files inside the directory and its subdirectories will be used to construct the metadata.generator.rootPath
specifies the directory into which any new files will be generated.
By default, the CLI generates a file called subgraph.yaml
for a new subgraph but any number of subgraph config files
can be created and referenced in CLI commands.
connector.*.yaml
These config files tell Hasura DDN how to build your connector. It will contain information such as the type of connector and the location to the context files needed to build the connector.
For example:
kind: Connector
version: v1
definition:
name: mypg
source: hasura/postgres:v0.7.0
context: .
envFile: .env.local
version
is the version of the connector objectname
is a name given to the connectorsource
is the connector to use specific to the data sourcecontext
specifies the connector directoryenvFile
specifies the connector specific environment variables to use for introspecting and building the connector. If you are deploying your connector on DDN cloud, you also need to specifysubgraph
. Value of this field is name of the subgraph you want to deploy your connector to.
By default, the CLI generates two files - connector.cloud.yaml
and connector.local.yaml
but any number of connector
config files can be created and referenced in CLI commands.
.hasura/context.yaml
This specifies the default DDN project and supergraph file path. The default values are used by all commands that accept
--supergraph
flag, --subgraph
flag and --project
flag. The flags can be used to override the default values.
context:
project: emerging-stag-9129
supergraph: ../supergraph.cloud.yaml
subgraph: ../app/subgraph.yaml
This file is configured by the ddn context set
command.
Engine
The engine
directory contains the files required for Hasura v3 engine container. This directory has the following
structure:
├── .env.engine
├── auth_config.json
├── metadata.json
└── open_dd.json
The .env.engine
file specifies environment variables required by the Hasura v3 engine container.
The auth_config.json
, metadata.json
and open_dd.json
are generated as a result of ddn supergraph build local
command and do not need to be edited by the user.
Globals
The globals
directory contains the files for the globals subgraph which is generated by default to hold the
supergraph-level metadata objects, i.e. AuthConfig
, GraphqlConfig
and CompatibilityConfig
.
For example:
├── .env.globals.cloud
├── .env.globals.local
├── auth-config.cloud.hml
├── auth-config.local.hml
├── compatibility-config.hml
├── graphql-config.hml
├── subgraph.cloud.yaml
└── subgraph.local.yaml
auth-config.cloud.hml
andauth-config.local.hml
files contain theAuthConfig
object which define the authentication configuration for the supergraph for cloud and local deployments respectively.compatibility-config.hml
file contains the compatibility date configuration for the supergraph.graphql-config.hml
file contains the GraphQL configuration for the supergraph, which allows you to customize the available query and mutation capabilities along with the schema..env.globals.cloud
and.env.globals.local
files contain all the environment variables, if any, which are required by the globals subgraph's metadata objects for cloud and local deployments respectively.