Announcing Hasura Data Delivery Network Alpha

We are excited to announce the alpha version of Hasura Data Delivery Network (DDN) and sharing key features of our new platform, building on the advancements introduced in Hasura version 3.

In speaking with developers building APIs and microservices, we realize the community is faced with significant challenges. Hasura DDN is being built to address these problems and simplify the lives of software developers and architects everywhere.

Here’s what we heard most frequently in our conversations:

1. Developers want to create APIs in their preferred programming language
Adopting a new language involves a steep learning curve and requires adapting to different tools. As a result, in creating a new microservice or API, developers often opt for the programming language they’re most comfortable with – a preference driven by concerns about performance, reliability, backward compatibility, and integration, especially for crucial applications. Also, a new language and framework usually means more training, adding overhead to the business.

Not just a black box – Hasura DDN is a low-code API development platform, fully language agnostic, so developers are free to augment Hasura's capabilities and compose business logic in the programming languages they’ve already invested in.

2. Developers want to create complex and feature-rich APIs – fast.
Creating an API server that aligns with business needs and user expectations involves addressing various requirements –  supporting different API styles (such as Relay, GraphQL queries, subscriptions, pagination, and sorting) and formats like REST and gRPC. Backend developers are navigating complexity, designing for scalability, defining user experiences, testing and monitoring, and maintaining the codebase – and it’s exceptionally time-consuming.

Hasura DDN enables developers to create complex API features in minutes, rather than the days or months it would have taken to build manually. The capabilities for creating feature-rich APIs extend across all of our data connectors. Our platform is also extensible – our GraphQL capabilities extend to other API protocols, like gRPC and REST, with our open metadata specification.

3. Developers want the ability to quickly test and iterate.
Building a feature-rich microservice that optimally utilizes the data source demands significant expertise and effort to make inferences from the available information. Another challenge is implementing continuous integration (CI) to handle frequent code changes and database schema adjustments. This process involves writing extensive boilerplate code, managing dependencies, and coordinating with other developers.

Hasura DDN offers a free and intuitive developer environment with robust tooling, seamlessly combining local and cloud development for efficient API design, testing, iteration, and maintenance. The platform also provides code generation and reusable components for rapid development, allowing full flexibility for customization.

On the path to achieving the vision of Hasura being a "CDN"-like layer for real-time (transactional), analytical, and streaming data, the following are features found in the Hasura DDN alpha release.

New console geared for rapid onboarding and troubleshooting

We have introduced major changes to the console in Hasura DDN. We have added features that help in onboarding developers in your team on your data domain. That way, you’re able to collaborate and go to market faster.

As the number of models and relationships increases, so does the complexity of understanding and making sense of it. That’s why we are rolling out tools on the console for rich visualization of the data domains and their interactions. On top of that, users can now drill down into each of the models and commands and browse through further details, such as model-level analytics, fields, permissions, and relationship details.

A large part of going to production successfully is to quickly debug and troubleshoot errors and issues. We have enhanced our monitoring capabilities in Hasura DDN, providing users the ability to get detailed traces and spans of the request as it traverses from the Hasura server all the way to the data source.

That way, with all the context and information easily available to them, users quickly detect any latency issues while locating the root cause of the error in real time.

Hasura DDN also features a new database administration tool that’s ideal for exploring and editing data in PostgreSQL for now and going forward in most of the other supported databases and it’s geared to scale to massive workloads.

Code-first declarative API authoring and iteration

One of the main pain points of our developer community over the past few years has revolved around metadata authoring, management, and deployment. We’ve heard complaints from users on the time a metadata change takes to get built and rolled out, mainly because of Hasura cold-start issues. On top of that, the hard coupling of Hasura to upstream services (such as the database or remote API servers) led to Hasura not being available unless upstream services were available. And lastly, being dependent on a metadata database implied that users have another thing to run and maintain in production.

In Hasura DDN, we have introduced a build and runtime separation – a pattern that is analogous to compiler-based programming languages. The DDN now has a serverless runtime built-in with per-request tenancy. Each metadata change will now produce an atomic build and an associated artifact that is loaded to Hasura production instantly.

This means users no longer experience cold starts and can achieve zero downtime rollouts without any restarts or downtimes. Each atomic build represents an immutable API that can be incrementally tested and applied to a project so that it is made available to API consumers. Hasura DDN is also no longer tied to the upstream services and introspection is not decoupled from runtime. As a result, there is no dependence on upstream services and users can now safely go to production multiple times in a day or even an hour.

In Hasura DDN, users can interact with Hasura in a code-first way. To aid in declarative authoring and editing of metadata, we are introducing some potent metadata management tools such as a YAML-based metadata file type HML – a brand new language server protocol (shipped with VScode extension) to provide language features like autocomplete and code actions.

Users also have access to a brand-new CLI that offers useful scaffolding and helps provide logical separations of the metadata via folders and files. You no longer need to browse a large metadata file and can instead navigate a file structure that makes the most sense for your team. The new CLI will stitch the metadata together and build the API from different folders and files.

Lastly, to leverage the convenience of DDN cloud’s out-of-the-box features, yet easily prototype using local resources like a local database or a local authentication server, the CLI now provides secure connect options in the form of a local daemon and a cloud-based tunnel server system.

This would mean that you can develop your APIs to do all local development and testing scenarios while still being deployed on Hasura Cloud, making it even faster to go to production from the development stages.

The experience is designed to be highly secure ensuring only those involved in the user's projects have access. It also offers a seamless experience, aiming to replicate the feel of true local development as closely as possible.

Developers accustomed to using other web frameworks in a code-first approach can now quickly adapt and efficiently develop in code with DDN.

Feature-rich data domain modeling

Hasura users have often wanted more control over how the schema is generated, what names are used, and how types are generated. As we have expanded to other types of data sources, such as NoSQL databases and API servers, we realized our users have been spending time to understand the Hasura metadata structure. Furthermore, the metadata specification was predominantly centered on SQL and GraphQL, which limited its adaptability to other protocols such as gRPC.

In Hasura DDN, we’ve made significant changes to our metadata structure with the release of the new OpenDataDomain specification that provides fixed and a small set of types such as Types, Models, Commands, Permissions, and Relationships to compose and author all forms of APIs encompassing complex business logic.

As a result, users can employ contract-first or schema-first types of development practices while developing APIs with Hasura, freeing up time to focus more on domain design than how the APIs are actually implemented.

The domain-led design approach using Models and Commands could be of great benefit to separate the command (write) and query (read) responsibilities in APIs. This pattern can be beneficial in reflecting the distinct needs and optimizations required for data modification versus data retrieval, and help users design, develop, and deploy scalable command and read graphs in production.

Similarly, Permissions type can help set bounded contexts and define specific boundaries for different parts of the domain, while Relationships type can aid in dealing with interconnected contexts or bounded contexts and facilitating communication and integration between these contexts.

With the new metadata specification, users have a single ubiquitous language across data sources and domains that can be shared by domain experts, developers, and stakeholders. Furthermore, API generation now becomes protocol agnostic as the same terms and concepts from this language could be used for GraphQL, Relay, REST, gRPC, and so on, ensuring consistency and clarity across the domain and the API.

Given the strict and typed metadata contract, there’s a higher correctness guarantee and more opportunities for Hasura to automate query planning and caching across future domains.

Embed business logic using TypeScript

Our users have asked for more flexibility and feature support when it comes to implementing custom business logic in Hasura. While the Actions functionality provided a lot of support in this area, it was limited and also led to a learning curve for most users. There was also the burden of hosting functions in a different environment and bringing that to Hasura to implement a simple but core piece of business logic for the API.

In Hasura DDN alpha, users can code in TypeScript, host the TypeScript functions in Hasura Cloud, and embed that function-backed business logic into the Hasura API instantly. This allows users to use the tooling and libraries of their choice while leveraging the instant production-grade API server creation capabilities of Hasura. It also enables highly requested features, such as data transformations with mutations, and provides new and innovative ways to do pre-mutation input validations and integrate with APIs on the Hasura platform.

As part of alpha, users can leverage the new TypeScript Deno Connector, write their functions in index.ts, use the Hasura LSP to automatically generate Command based metadata, and instantly build to start using the API and a functional command graph.

Native extensibility with data connectors

Hasura DDN is a completely modular and extensible platform. Users can bring in any data connectors that follow the new Hasura native data connector specification to leverage the type of functionalities that Hasura currently offers with native databases such as PostgreSQL. With the recent release of the Hasura Connector Hub, all native data connectors are open source and free to use with Hasura DDN.

We have two types of data connectors:

1. Verified and Hasura-hosted connectors
These connectors are deployed in multi-region and have stricter performance and availability SLAs. Currently, in alpha, we support PostgreSQL and its flavors in this category.

2. Community-developed connectors
These connectors need to be separately deployed before building Hasura metadata, either in the Hasura Cloud infrastructure or in your own environment. Currently in alpha, we support ClickHouse, SendGrid, and TypeScript (Deno).

To help in fast iteration and less overhead, we have a CLI plugin to help deploy community-developed connectors on Hasura Cloud. After deployment, you can use the connector URL to embed in Hasura metadata as a data source for a Model.

We also have extensive resources to aid in developing your own connectors with the NDC learn course and SDKs in TypeScript and Rust.

Take a sneak peek into the future of data delivery – building supergraphs at the API layer – and contribute to shaping our product vision. Help us build the best, most useful product for your needs.

Be among the first to experience Hasura DDN. Check out the following resources:

1. Docs on getting started: https://hasura.io/docs/3.0/
2. Register here to access the playback recording of our November 30 Community Call, which featured powerful Hasura DDN demos and the product roadmap.
3. Connect with other alpha users and our product development team on Discord. We would love to hear from you! You can also reach out to us via email at [email protected].
4. Try out our Learn course here, where you can follow a tutorial and get started today!

As we push forward in our mission to revolutionize microservice development and move towards the release of the beta version of Hasura DDN in early 2024, we're eager to iron out any rough edges and improve the experience based on invaluable feedback from our alpha users.

The beta roadmap is razor-focused on improving and tightening the developer experience as we acknowledge our product is still evolving. Your participation will be crucial in identifying areas for refinement and we urge you to stay engaged, share your thoughts, and help us ensure the beta launch delivers the best developer experience possible.

Areas of focus in the coming weeks:

• Metadata authoring enhancements: LSP code generation and auto-complete enhancements. Making it easier to build permissions and add relationships to models and commands. Ease of use for debugging metadata errors at the time of authoring. Improvements to the CLI watch command for rapid iteration loops.
• Avoiding API breaking changes: Build validations with dry run mode and compile time guarantees. Improved error handling at build time. Enhanced operational health dashboard and metrics for advanced error handling, insights, troubleshooting, and production monitoring of APIs across builds.
• Integrating business logic DX: Improvements to the TypeScript SDK and connector for accelerating connector and business logic development with fast iteration loops. Ease of use for importing REST and GraphQL.
• Team collaboration: Enhancements to the multi-repo workflow for building a supergraph across multiple data domains owned by different teams.
• API features: Complete the API feature set including mutations and other query features.
• V3-engine open source release: Making the API server behind DDN open source.Migration: Streamlined migration utilities for current v2 users to transition to DDN.

We greatly value your feedback, and our commitment to crafting an exceptional developer experience is deeply rooted in the insights and suggestions we receive from our community.

Thank you for being the driving force behind Hasura DDN's evolution.

Let’s continue this transformative journey together. We encourage you to share your thoughts, queries, or feedback about the alpha launch.

Get access to Hasura DDN alpha and get started today!