Building a Digital Twin with MQTT and Hasura Streaming Subscriptions

In this blog we will be talking about how to build a digital twin of a toy car with an android phone (to leverage sensors),  MQTT broker and Hasura GraphQL Engine.

We will go over,

• Building a digital twin of a toy car attached with a gyroscope sensor through an android device
• Setting up the device to stream real time sensor data to an MQTT Broker.
• Receiving data from the MQTT broker and sending it as a GraphQL mutation to Hasura GraphQL engine.
• Consume live data on the ReactJS application with Hasura GraphQL subscriptions
• Rendering the real time gyroscope sensor data as a 3D car model using ThreeJS

A digital twin is a virtual representation of a real-world physical system or product (a physical twin) that serves as the digital counterpart of it for practical purposes, such as system simulation, integration, testing, monitoring, and maintenance.

Digital Twins are created in a way that it can receive sensor data in real time from the physical counterpart. This allows the twin to simulate the physical object in real time to learn about the characteristics of the physical twin. There are systems that also have controllers to remotely manage and maintain the system in real time.
It is also used to remotely debug large machinery, with the help of mixed reality gears- this ease out the need of skilled technicians to travel across the globe and save time and cost.

Digital Twins are mainly used in

• Manufacturing
• Automobiles
• Power Generation
• Healthcare
• Industry 4.0
• Town planning
• Remote Debugging

The basic requirements to build a digital twin are

• Physical object (a toy car in this case)
• Sensors to collect data (will be using an android device) & Controllers
• Connectivity to the central system (Internet)
• Server that manages the data flow and control events (Hasura GraphQL Engine & MQTT Broker)
• Virtual Representation (a static web application, using ThreeJS to render 3D in this case)

Note : The repository is available here and the commands mentioned in the steps are configured in the repository.

As shown above, we can have a toy car and an android phone attached to it.

The android device attached to the device will be able to collect real time sensor data (Gyroscope in this case) and send it to the cloud.

Considering most of the real world IoT systems use low footprint transfer protocols to send data from device to server communication, we will be using MQTT as the preferred protocol here.

We can also use GraphQL over HTTP from this point itself, but considering the current systems and integrations this demo will talk about consuming and ingesting data from end devices to MQTT Broker and then to a Postgres DB through a Hasura instance.

To simplify the end device implementation, we will be using the android application  MQTT Simulator to collect and send rotational data to an MQTT broker.

MQTT follows a topic based pub-sub pattern, and it requires a broker to handle this.

For the demo purpose, let’s use HiveMQ public MQTT broker so that we don’t need any setup here.
If you use the above application, the data will be streaming to digital_twin/android/<provided_device_id> by default and this can be listened easily by subscribing to the same topic or digital_twin/android/#. # here will make sure any topic name that starts with the pattern will be subscribed.

At this point, we have successfully received the data from the device to the internet/network, we now need a way to store this data and stream the data to the clients who require live data from the device.

Now we need a place to store and access the data in a secure way, to do this we will be using Hasura GraphQL Engine backed with a postgres DB.

• Deploy to Hasura

• Open Hasura Console, head to Data tab and connect a Postgres DB if not done already.
• Head to Data->SQL tab to create a table.

CREATE TABLE "public"."device_data"("id" serial NOT NULL, "data" jsonb NOT NULL, "timestamp" timestamptz NOT NULL, "device_id" text NOT NULL, PRIMARY KEY ("id") );

• Track the table device_data so that it can be accessed through GraphQL APIs.

We need a NodeJS runtime to consume data from the MQTT runtime and send it to Hasura. I have a small NodeJS script that does the following,

• Subscribe to the MQTT broker with the same channel name used by the Android MQTT Simulator
• Parse the MQTT payload and transform the data

client.on("message", function (topic, message) {
 // message is Buffer, convert this to generate GraphQL mutation from the message
 const decoded = getMutation(message.toString(), topic);
 sendToHasura(decoded, CONSTANTS.HASURA_HOST);
});

• GraphQL mutations (as shown below) on Hasura GraphQL Engine to store it on the DB.

mutation AddDeviceData($data: jsonb!, $device_id: String!, $timestamp: timestamptz!) {
 insert_device_data_one(object: {data: $data, device_id: $device_id, timestamp: $timestamp}) {
   id
 }
}

At this point we successfully managed to get the data to the postgres instance and the data flow will looks like the following

To start this service,

1. clone the project from https://github.com/soorajshankar/digital-twin-graphql
2. Change .env file with your project configuration
3. Run the command `yarn connect`  or `npm run connect`

This command will connect to MQTT broker and start subscribing to the channel mentioned in the env file.

You could also use the public MQTT broker (HiveMQ) used in the repository for testing purpose.

From this part we will be discussing the ReactJS application and the main functionalities.

When you connect a postgres table on Hasura, you automatically get a GraphQL Subscription API to listen to the live changes. We will be using the latest streaming subscription feature from Hasura to stream the live data.

In this case, the query looks like the following:

subscription GetDeviceData($timestamp: timestamptz) {
   device_data_stream(
     batch_size: 1
     cursor: { initial_value: { timestamp: $timestamp }, ordering: ASC }
     where: { device_id: { _eq: "android" } }
   ) {
     data
     device_id
     id
     timestamp
   }
 }

Here is how the streaming subscription work in general:

Visualizing data is very important in the connected world as it can save a lot of time to understand, make decisions and act on it. The concept of Digital Twin plays a great role in simplifying this task to an extent. Both human and simulated environments can process the data much faster when it is virtualized in a digital environment.
To do this we will be using ThreeJS model to visualize the toy car, the car model orientation will refresh frequently with the latest data stream.

Repo: https://github.com/soorajshankar/digital-twin-graphql

Even though we built a digital twin of a toy car, this can be made useful for variety of use cases like

• Connected cars
• Remote Monitoring : Windmills, Smart Factories, Industry 4.0 and other Industrial IoT systems
• Remote Debugging with AR goggles and connected data.

Also if you have an existing data stream/ queue, it can easily be connected to Hasura GraphQL to set up a secure and easy way to store and access realtime data until the end consumer.