Regression Testing

Regression tests ensure continued support for operations required by your front-end apps or users, i.e., validating changes to the GraphQL schema (schema integrity) against these operations to ensure that there are no breaking changes or regressions in your GraphQL API.

You should configure your production instance to run regression test suites since changes to the underlying Postgres schema and/or the Hasura configuration could potentially lead to unwanted regressions in your schema. This is crucial for iterating on new features or removing existing features.

Create a test suite

On the Hasura Cloud console, under the Pro/Monitoring tab, head to Regression Tests at the end. Select the actions that are typically accessed from the front-end client. In our slack model, we want to add the users and channel queries we have tried previously. Click on Add to test suite. Once this is added, we can change a schema to verify if a regression has been caught.

Open the Hasura Console via the CLI. (http://localhost:9695). Now let's make a schema change to the channel table to rename the name column to channel_name.

Then run the test suite through the Run Tests tab. The channel query should now fail.

With the following error field "name" not found in type: 'channel'.

These tests ensure that breaking changes are only pushed to production with the relevant updates to the client.

Note: We can select the test suite from any other cloud project to run on the current project. This is useful when the test suite configured on the production instance works on the staging environment post the schema change.