Native Queries

Introduction

Native Queries allow you to run custom queries on your MongoDB database. This allows you to run queries that are not supported by Hasura's GraphQL engine. These queries are defined as aggregation pipelines which is the query format that is used by MongoDB's aggregate command.

Managing native queries

Native Queries can be managed either via the DDN CLI, or by editing connector configuration files directly.

To connect a query to your data graph the native query configuration must declare the type of data that it produces, and the types of any parameters. The major difference between using the CLI vs configuring by hand is how these type declarations are managed.

1. CLI (BETA)

The CLI will type-check your aggregation pipeline, and will automatically produce type declarations for you. But this functionality is experimental - there are aggregation pipeline features that the CLI does not recognize yet. The CLI will not be able to produce configurations for pipelines that use those features. See supported pipeline features.

2. Directly in configuration files

When writing configuration directly you may use any aggregation pipeline features, and you have complete control over how types are declared. But you must write all type declarations yourself.

Switching between CLI and configuration files

You can freely switch between configuring with the CLI or writing configuration files directly. For example you can manage one native query with the CLI, and configure another directly; or you can edit CLI-generated configuration files. The recommendation is to try using the CLI first, and if it doesn't work or does not produce the correct configuration then write configuration directly.

Manage native queries with the DDN CLI

Connector version requirement

Managing native queries using the CLI requires version 1.5.0 or later of the MongoDB Connector. If you have an earlier version please skip to the Write native query configurations directly section.

This is a BETA feature - it is a work in progress, and will not work for all cases. It is safe to experiment with since it is limited to managing native query configuration files, and does not lock you into anything.

Create a native query

Create a file with a .json extension that contains the aggregation pipeline for you query. For example this pipeline outputs frequency counts for words appearing in movie titles in a given year:

title_word_frequency.json

[
  {
    "$match": {
      "year": { "$eq": "{{ year }}" }
    }
  },
  {
    "$replaceWith": {
      "title_words": { "$split": ["$title", " "] }
    }
  },
  { "$unwind": { "path": "$title_words" } },
  {
    "$group": {
      "_id": "$title_words",
      "count": { "$count": {} }
    }
  }
]

In your supergraph directory run a command like this using the path to the pipeline file as an argument,

$ ddn connector plugin \
  --connector app/connector/my_connector/connector.yaml \
  -- native-query create title_word_frequency.json \
  --collection movies

The name of the native query is based on the input JSON file name, in this case title_word_frequency. You can override this with the --name flag.

If your aggregation pipeline uses a collection as an input specify it with the --collection flag. Your connector must already be configured with the schema for the collection. Type inference will be based on the configured schema.

If you are updating an existing native query add the --force flag to overwrite the existing configuration.

You should see output like this:

Wrote native query configuration to your-project/connector/native_queries/title_word_frequency.json

input collection: movies
representation: collection

## parameters

year: int!

## result type

{
  _id: string!,
  count: int!
}

You may keep the pipeline file that you wrote if you think that you might want to edit it to update the native query later, or you may delete it.

The final step is to update your metadata to track the new native query:

$ ddn connector-link update my_connector --add-all-resources

Native Query Parameters

Parameters / arguments for your native query are specified using the placeholder syntax, "{{ parameter_name }}" inside a string. The example above includes a parameter called year using this syntax.

Avoid code injection

When placeholders are substituted, arguments are not sanitized. It is up to you as the author of a native query to construct the query to avoid potential code injection. See Security Best Practices.

Despite the placeholder appearing in a string, if the only content of the string is the placeholder then the placeholder will be replaced with an argument value at query time which may be any type, including complex types such as arrays or objects. In the example above the type of the year parameter is int! so the string "{{ year }}" will be replaced with a non-null int value.

The type checker attempts to infer types for parameters. If it is not able to do so you can add a type annotation with a bar (|) following the parameter name. For example, "{{ year | int! }}". Type annotations use a GraphQL-style type expression syntax so for example a non-nullable list of nullable double values would be [double]!. But instead of GraphQL types these annotations use BSON types using the lower-case alias names here. In addition to BSON types you can also reference object types using names specified in your connector configuration schema, and there is a special type, extendedJSON that is the "any" type - values of that type can hold any data.

The same parameter may occur any number of times in a pipeline. If you use a type annotation you may apply it to any one occurrence of the parameter, or to all of them.

List, inspect, and delete native queries

List names of configured native queries with,

$ ddn connector plugin \
  --connector app/connector/my_connector/connector.yaml \
  -- native-query list

Show information about a native including its pipeline and type declarations with,

$ ddn connector plugin \
  --connector app/connector/my_connector/connector.yaml \
  -- native-query show <native-query-name>

Delete a native query with,

$ ddn connector plugin \
  --connector app/connector/my_connector/connector.yaml \
  -- native-query delete <native-query-name>

After deleting you will need to update your metadata configuration as well.

Write native query configurations directly

Native Queries can be defined by

1. Adding a native_queries directory if one doesn't already exist in your connector configuration directory
2. Adding a .json file following the syntax laid out in the following sections.

Native Queries can take arguments using the placeholder syntax, "{{argument_name}}". Arguments must be specified along with their type.

Avoid code injection

When placeholders are substituted, arguments are not sanitized. It is up to you as the author of a native query to construct the query to avoid potential code injection. See Security Best Practices.

Here's an example of simple "hello" Native Query:

Configuration

{
  "name": "hello",
  "representation": "function",
  "description": "Basic test of native queries",
  "arguments": {
    "name": { "type": { "scalar": "string" } }
  },
  "resultDocumentType": "Hello",
  "objectTypes": {
    "Hello": {
      "fields": {
        "__value": { "type": { "scalar": "string" } }
      }
    }
  },
  "pipeline": [
    {
      "$documents": [
        {
          "__value": { "$literal": "{{ name }}" }
        }
      ]
    }
  ]
}

This will create a query called "hello" which takes a single argument called "name" of type string. The query will return an object with the key "__value" and the value of the argument "hello".

Valid Native Query syntax

Check out our page on writing valid Hasura DDN Native Operations syntax.

To add the native query to your supergraph you need to update the metadata with a command like this one,

$ ddn connector-link update <connector-name> --add-all-resources

Usage

With the example above, you can then use the query in your GraphQL API like this:

query MyQuery {
  hello(name: "world") {
    __value
  }
}