Why Saigar chose GraphQL & Hasura to build a crowdsourced intelligence platform
Every year thousands of families are struck with tragedy as their loved ones go missing. The recent explosion in the volume of publicly available data has given rise to a new category of intelligence tools to help deal with this problem - commonly referred to as OSINT (Open Source Intelligence). OSINT refers to the collection, processing and analysis of publicly available data that has been determined to be of intelligence value.
Sources of OSINT include the surface, deep and dark web, social media, news media, academic sources, and government records. A typical use case would be to leverage these sources of information to crowdsource new leads on active missing persons cases.
Saigar Technologies has built Saigar CE, the world’s first dedicated OSINT collection platform for crowdsourced intelligence initiatives. It contains tools to crowdsource OSINT from multiple sources and combine them together in a structured format for sharing with intelligence stakeholders such as law enforcement.
Since Saigar CE’s launch in June 2019, the platform has been used to facilitate 6 crowdsourced OSINT events at various industry conferences such as DEFCON and BSidesLV, and other virtual crowdsourced OSINT events.
The platform
The Saigar CE platform empowers users to work collaboratively across the entire OSINT process. The typical OSINT lifecycle looks like this:
Intelligence data is collected, tagged and submitted.
Next, the intelligence is processed .i.e. it is archived as accepted or rejected. It can also be modified with new intel as and when data comes in.
Case creation, user management and general administration of crowdsourced initiatives can be done by "Admin" users. These admins can also export intel packages for the stakeholders involved in the case.
Saigar CE also provides a system for the different users involved in each case to get feedback so that they can continuously improve their performance.
Saigar’s requirements
Saigar’s primary KPIs were:
Feature velocity: As they were building a product for a new market, they would have to iterate on the platform quickly based on community feedback and push those changes quickly for future events.
Out-of-the-box support for realtime data: The nature of these events calls for realtime data. Intelligence Administrators need to process submissions as they come in and contestants need to receive feedback on their submissions in a timely manner.
Security: User accounts on the platform have different privilege levels and should only have the ability to view and modify data for their access level
Scalable from day 1: They wanted a backend that was scalable and performant by default. They wanted to focus on providing the maximum value for their customers i.e. building powerful features, and did not want to spend time on backend implementation and worrying about performance issues.
Evaluating the Stack - Choosing GraphQL
With several years experience in fullstack development working with various products, the Saigar Team knew right from the start that they wanted to use GraphQL. However, they wanted to avoid using a solution that would require them to build custom JWT middleware to validate Auth0 JWT Tokens and then handle RBAC for each GraphQL API call. They also wanted to avoid having to write custom resolvers for what are essentially CRUD operations, increasing development time and efforts.
Evaluating and using Hasura
After their initial trial, the team felt that Hasura suited their needs perfectly!
It enabled them to create the schema and interact with the data quickly. They did not have to worry about backend implementation, including deploying, scaling, and securing the backend server.
The platform required Role Based Access control rather cumbersome to do with a custom GraphQL server implementation, but was incredibly easy to do with Hasura. Not having to worry about concerns such as the RBAC implementation and the security layer on the backend had a major impact.
In total, from the time they started development to their first launch, it took them 4 months with very limited dev resources!
Going Forward
Saigar will be exhibiting Saigar CE at more industry related events, putting it into the hands of potential users to get their thoughts and feedback. For more information and to follow their progress, you can check out their website, and follow them on Twitter.
This article is based on interviews with Peter Vicherek, the Co-Founder & CTO of Saigar Technologies. Over his career he has worked with multiple startups to develop and bring new products to market. Peter is also the author of multiple open source security tools and has spoken at numerous cyber security conferences, including BSides Toronto and DEFCON Toronto. With a passion for crowdsourcing technology, Peter enjoys working in the space to tackle complex social problems. Follow Peter on Twitter here.
