Project directory structure

Introduction

Hasura DDN projects are configured via metadata and config files which are managed in a project directory. Metadata can be organized however you wish, but the CLI will scaffold a few directories and files by default.

Getting started

Follow the steps in the Quickstart to set up a project.

Sample project directory structure

The following is a sample directory structure of a local project:

.
├── hasura.yaml
├── supergraph.yaml
├── compose.yaml
├── .env
├── .env.cloud
├── otel-collector-config.yaml
├── engine
│   ├── .env.engine
│   └── build
│       ├── auth_config.json
│       ├── metadata.json
│       └── open_dd.json
├── .hasura
│   └── context.yaml
├── app
│   ├── metadata
│   └── subgraph.yaml
├── globals
│   ├── subgraph.yaml
│   └── metadata
│     ├── auth-config.hml
│     ├── compatibility-config.hml
│     └── graphql-config.hml
├── .devcontainer
│   └── devcontainer.json
├── .gitattributes
├── .gitignore
└── .vscode
    ├── extensions.json
    ├── launch.json
    └── tasks.json

Types of files and directories

The configuration relies on the following types of files:

File type	Description
hasura.yaml	The file that denotes the root of a Hasura project.
.env.*	Files that store environment variables.
supergraph.*.yaml	Config files that describe how to build the supergraph.
subgraph.*.yaml	Config files that describe how to build a subgraph.
connector.*.yaml	Config files that describe how to build a data connector.
*.hml	Files containing Hasura metadata objects of the supergraph.
context.yaml	Config file that stores project context configuration.
compose.yaml	Docker Compose files for local development.
otel-collector-config.yaml	File containing config for the OpenTelemetry Collector.

The configuration contains the following directories by default:

Directory	Description
engine	Directory containing files required for running the engine container.
globals	Directory containing files for the globals subgraph.
app	Directory containing files for the app subgraph.

hasura.yaml

hasura.yaml appears at the root of a Hasura project.

For example:

hasura.yaml

version: v3

• version specifies the version of the project directory structure.

Env files

These files are used to store environment variables. They are typically of the format .env.*.

The file structure should be in the format:

.env

ENV1=abcd
ENV2="abcd"

# remember to escape special characters
# ENV3 = ab$c"d
ENV3="ab\$c\"d"

When a new project directory is initialized with the DDN CLI, it creates a single .env file by default. This file will store all the environment variables required for local development.

ddn project init command will initialize a single .env.cloud file which will be used to store all environment variables required by the DDN project.

All .env files in the project root directory are ignored by default by the .gitignore file generated by the CLI.

It is also possible to split the single .env file into different .env files required for:

• Docker Compose: Docker Compose files can use .env files to specify environment variables needed for containers.
• Subgraph config files (such as subgraph.yaml): Each subgraph config can have its own .env file which will be used to specify environment variables required for building the subgraph.
• Connector config files (such as connector.yaml): Each connector config can have its own .env file which will be used to specify environment variables needed for building the connector.

Other than this, .env files can also be passed into various DDN commands using --env-file flag.

Supergraph config files

These files contain Supergraph configs which tell Hasura DDN how to construct your supergraph. They are typically of the format supergraph.*.yaml.

It will contain information such as which subgraph config files to use for building the supergraph

For example:

supergraph.yaml

kind: Supergraph
version: v2
definition:
  subgraphs:
    - globals/subgraph.yaml
    - app/subgraph.yaml

• version is the version of the supergraph object
• subgraphs specifies paths to all the subgraph config files. These are then read recursively to construct the metadata for the supergraph.

By default, the CLI generates a single supergraph file - supergraph.yaml but any number of supergraph config files can be created and referenced in CLI commands.

Subgraph config files

These files contain Subgraph configs which tell Hasura DDN how to construct your subgraph. They are typically of the format subgraph.*.yaml.

It will contain information such as which metadata resources to use for the build. It also defines how metadata is generated for the subgraph by the CLI.

For example:

app/subgraph.yaml

kind: Subgraph
version: v2
definition:
  name: app
  generator:
    rootPath: .
  includePaths:
    - metadata
  envMapping:
    POSTGRES_READ_URL:
      fromEnv: APP_POSTGRES_READ_URL
    POSTGRES_WRITE_URL:
      fromEnv: APP_POSTGRES_READ_URL
  connectors:
    - path: connector/mypg/connector.yaml
      connectorLinkName: mypg

• version is the version of the subgraph object
• includePaths specifies the directories and files where metadata for the subgraph can be found. If a directory is specified, all the *.hml files inside the directory and its subdirectories will be used to construct the metadata.
• generator.rootPath specifies the directory into which any new files will be generated.
• envMapping declares the environment variables needed by the subgraph and where to read it from. For example,

  POSTGRES_WRITE_URL:
     fromEnv: APP_POSTGRES_WRITE_URL
  This means that some metadata in the subgraph requires an env var called POSTGRES_WRITE_URL. The value of this has to be read from an env var called APP_POSTGRES_WRITE_URL from any of the .env.* files passed as input. Only env vars declared in the envMapping are passed into the subgraph metadata.
• connectors lists all the connectors that are to be built when this subgraph is built. It contains path to the connector configuration and the name of the corresponding connector link.

By default, the CLI generates a single file called subgraph.yaml for a new subgraph but any number of subgraph config files can be created and referenced in CLI commands.

Connector config files

These files contain Connector configs which tell Hasura DDN how to build your connector. They are typically of the format connector.*.yaml.

It will contain information such as the type of connector and the location to the context files needed to build the connector.

For example:

app/connector/mypg/connector.yaml

kind: Connector
version: v2
definition:
  name: mypg
  source: hasura/postgres:v1.0.1
  context: .
  envMapping:
    CONNECTION_URI:
      fromEnv: APP_MYPG_CONNECTION_URI

• version is the version of the connector object
• name is a name given to the connector
• source is the connector to use specific to the data source
• context specifies the connector directory
• envMapping declares the environment variables needed by the connector and where to read it from. It works the same way as it does for subgraphs. Only env vars declared in the envMapping are passed into the connector.

By default, the CLI generates a single connector.yaml file for every new connector that is added but any number of connector config files can be created and referenced in CLI commands.

HML files

Files of the format *.hml contain the metadata objects used to build the supergraph.

Context config file

This file, located at .hasura/context.yaml, specifies the different contexts and custom scripts set up for the project.

Contexts allow setting default values for certain flags that are often needed in CLI commands. By default, a context called default is created and set as the current context. Values in the context can be configured using the ddn context set of CLI commands.

Scripts allow executing complex command sequences with simple memorable script names using the ddn run CLI command. By default, a script called docker-start is added to the context config to allow starting the Hasura v3 engine via Docker Compose and the engine compose file.

For example:

.hasura/context.yaml

kind: Context
version: v3
definition:
  current: default
  contexts:
    default:
      project: emerging-stag-9129
      supergraph: ../supergraph.yaml
      subgraph: ../app/subgraph.yaml
      localEnvFile: ../.env
      cloudEnvFile: ../.env.cloud
  scripts:
    docker-start:
      bash: HASURA_DDN_PAT=$(ddn auth print-pat) docker compose --env-file .env up --build --pull always -d
      powershell: $Env:HASURA_DDN_PAT = ddn auth print-pat; docker compose --env-file .env up --build --pull always -d

• contexts section contains the contexts and their key-values.
• scripts section contains the defined custom scripts.

Docker Compose files

Docker Compose files are generated to run your engine and connectors locally for development. They are typically of the format compose.yaml.

When a new project directory is initialized with the DDN CLI, it creates a compose.yaml file by default to run the Hasura engine locally. This Compose file contains an engine service to run the engine along with an otel-collector service used to collect traces from the engine.

Every time a connector is initialized in a project, a compose.yaml files is generated for it to run the connector locally. By default, this compose-yaml is also added to the engine Compose file via the include directive to allow running the engine and connector together by just using the engine Compose file.

OTEL config file

This config file, typically otel-collector-config.yaml, contains the configuration for OpenTelemetry Collector used for seeing traces during local development.

It is used by the otel-collector service defined in the engine Compose file.

Engine directory

The engine directory contains the files mounted to the Hasura v3 engine container during local development. This directory has the following structure:

.
├── .env.engine
└── build
    ├── auth_config.json
    ├── metadata.json
    └── open_dd.json

The .env.engine file specifies environment variables required by the Hasura v3 engine container.

The auth_config.json, metadata.json and open_dd.json are generated as a result of ddn supergraph build local command and do not need to be edited by the user.

All files inside build directory are ignored by default by the .gitignore file generated by the CLI.

Globals subgraph directory

The globals directory contains the files for the globals subgraph which is generated by default to hold the supergraph-level metadata objects, i.e. AuthConfig, GraphqlConfig and CompatibilityConfig.

For example:

.
├── subgraph.yaml
└── metadata
    ├── auth-config.hml
    ├── compatibility-config.hml
    └── graphql-config.hml

• auth-config.hml contain the AuthConfig object which define the authentication configuration for the supergraph.
• compatibility-config.hml file contains the compatibility date configuration for the supergraph.
• graphql-config.hml file contains the GraphQL configuration for the supergraph, which allows you to customize the available query and mutation capabilities along with the schema.
• subgraph.yaml contains the config that describes how to build the globals subgraph.

App subgraph directory

The app directory contains the metadata and connector related files for the app subgraph which is generated by default. The name app is just based on convention and can be chosen to be anything.

For example:

.
├── metadata
├── connector
└── subgraph.yaml

• metadata directory contains the hml files containing metadata objects of the subgraph.
• connector directory contains the connector related files of the subgraph.
• subgraph.yaml contains the config that describes how to build and generate metadata for the subgraph.

A similar subgraph directory is created for every subgraph in the supergraph.