Using Redpanda to build a real-time security IoT platform

In today's world, securing millions of embedded Internet of things (IoT) devices is a monumental challenge. At Exein, we ensure the safety and security of all kinds of devices, ranging from small embedded systems running real-time operating systems (e.g., FreeRTOS) to larger, more capable IoT devices running a Linux kernel.

Exein Runtime is our enterprise solution that enables businesses from all industries to monitor and manage their fleet of devices in real time. Each device runs Pulsar, our open-source observability framework powered by Rust and eBPF, a powerful technology for efficient network and system performance analysis. Pulsar performs detection and response on the edge and emits threats and events to the central streaming platform for operators and analysts to action.

In this blog post, we'll discuss the challenges we faced in building the Exein Runtime, how we overcame them using Redpanda, and the benefits we gained from our partnership with them.

The challenges of real-time streaming in an IoT ecosystem

Coming from a background in Application Security, I’m accustomed to mature security frameworks (e.g. OWASP SAMM) tailored to identify common denominators across cloud environments—usually running common target architectures with relatively capable computing.

Working in embedded IoT systems is another story. The ecosystem is vast and diverse with a wide range of target architectures such as x86_64, ARM, RISC-V, and FreeRTOS, and compute availability ranging from small embedded devices all the way to powerful Linux systems.

Each and every one of these devices tells its own story through events, and it’s our job to ensure that these events are sent to a central streaming platform for customers to monitor in near real time. This raised a number of challenges that we needed to overcome.

• Event producers (devices) and event consumers (customer dashboards) are asymmetric, with tens of thousands of agents potentially tied to a single user.
• We need to allow producers and consumers to evolve independently with minimal coupling.
• We require a solution that can handle tens of thousands of events per second with low latency while ensuring data integrity.
• Updating devices is difficult and often not guaranteed, so we need to factor in backward compatibility.

Choosing the right stream processing pattern

During the initial design of the platform, we looked at a number of stream processing patterns. These ranged from more common such as CRUD or MVC, to more niche patterns such as Event Sourcing or Command & Query Responsibility Segregation (CQRS); selecting the right pattern would have a compounding effect throughout the development of the platform.

Looking at our requirements we have chosen to adopt the CQRS pattern for our platform. In Martin Fowler’s words, “The main benefit is handling high-performance applications. CQRS allows you to separate the load from reads and writes allowing you to scale each independently. If your application sees a big disparity between reads and writes this is very handy.”

To date, this has proven to be a good decision that fits our requirements nicely. We can scale our services independently, operate under varying loads, and develop our query models without affecting any deployed devices. Overall, a huge win!

Let’s take a brief look at how we applied this pattern using Redpanda.

Powering our IoT platform with Redpanda

Taking a 30,000-foot view of the Exein Runtime platform, we have our fleet of devices running Pulsar, emitting a large volume of events to our application proxy, which then sends them to Redpanda.

On the consumer side, we have a database sink that consumes events from the Redpanda queue and serializes them into their database representation, which is synthesized in real time to more elaborate views.

Redpanda is a critical service in our infrastructure, acting as a load regulator between agents with spiky traffic and users requiring a live feed of events with soft real-time performance. Using Redpanda, we achieved several significant benefits:

• Simplified deployment and maintenance: Redpanda streamlined our deployment process and made maintenance easier, thanks to its single-binary and configuration simplicity.
• Improved performance: Redpanda's high-throughput and low-latency design allowed us to handle over 39,000 events/per second, improving the overall performance of our platform.
• Seamless scalability: As our customer’s fleets grew, the Redpanda team allowed us to scale our infrastructure with ease and without compromising on performance or reliability thanks to the CQRS pattern and scalability of Redpanda’s cloud-managed solution.
• Production operation ease: Collaborating with the Redpanda team allowed them to become a logical extension of our Engineering team, ensuring that clusters are up-to-date and operating nominally at all times.

Winning with Redpanda: faster, simpler streaming at scale

Integrating Redpanda into our CQRS architecture has not only improved the performance and reliability of our platform but also made our lives easier by simplifying deployment and maintenance processes!

As we continue to expand our platform and user base, we plan to leverage more of Redpanda's advanced features, such as its support for geo-replication and Kubernetes integration, to further enhance the resilience and scalability of our system. We also aim to contribute to the Redpanda Community by sharing the experiences and improvements we make along the way.

To learn more about Exein and its open-source project, Pulsar, check Exein’s website and GitHub repository. To learn more about Redpanda, browse the Redpanda blog for tutorials and on how to easily integrate with Redpanda. You can also take Redpanda for a test drive!

If you get stuck, have a question, or want to chat with their engineers and fellow Redpanda users, join the Redpanda Community on Slack.

‍