Redpanda Hackathon winner: Sovereign Structure for enterprise AI

The Redpanda AI Integration and Innovation Hackathon ran from September 30th to October 28th — a new challenge inviting developers worldwide to flex their creativity with Redpanda Data Transforms, Redpanda Connect, AWS, and other cutting-edge technologies. 

The focus wasn’t just on technical brilliance but also on the simplicity of how each solution could help people and solve real-world problems. Whether improving efficiency, automating tedious tasks, or creating entirely new experiences, the hackathon judges were determined to crown a meaningful and impactful project as the winner. 

Deservingly, the winner was founder Dan Goodman and his project: Sovereign Structure. In this post, we hand it over to Dan to introduce himself and his project, which proposes a simple solution for enterprises to tap into AI — without everyone tapping into their data.

Get to know Dan

I’m the co-founder of Tangia — a toolset for streamers to engage and monetize their audiences. We help thin the glass between streamers and chat by roping the viewers into the content creation. I’m also a chronic distributed systems and database nerd who loves to build, which is how we got here.

I’ve always tried to learn everything there is to know about distributed systems and infrastructure. Back when I was relatively new to data streaming technologies, I dug into traditional queues like RabbitMQ and newer methods like Apache Kafka^® — which I found to be a colossal headache to actually operate (and have faith in a production deployment).

I was already familiar with ScyllaDB and I stumbled on a post about how Redpanda also used the Seastar architecture. I was impressed by how easy it was to bootstrap a cluster, and once I saw the Redpanda HTTP Proxy, I was hooked (because a common HTTP interface is awesome). I’ve been tinkering with Redpanda since, so when the Redpanda AI Hackathon popped up, I jumped at the opportunity.

The project: Sovereign Structure

The inspiration for this project was the application of AI for enterprises. AI is clearly sweeping the tech space, but the baseline resource requirements of AI solutions make it a unique challenge for builders.

To run the most capable large language models, you need significant hardware (with machines costing upwards of $40/hr) and novel code/services to run it. This makes managed models attractive, but it also makes it harder to serve enterprise customers who want to run solutions on their own infrastructure. Being able to easily connect models to existing data streams with only a few lines of YAML on top of proven systems in your own infrastructure is basically a cheat code to winning in enterprise sales against the competition that can't.

Enterprises want AI solutions but don't want their data to leave their infrastructure. They want support, but they don't want to open access to their data (which is becoming a more significant moat in the "intelligence age").

Sovereign Structure just scratches the surface as a solution and demonstrates that a solution is not only possible, but simple.

In a nutshell, Sovereign Structure reliably turns unstructured data into JSON-schema conformant structured data using LLMs that run directly in Redpanda, giving you the choice of model, and never requiring your data to leave your infrastructure.

Here’s a video walkthrough of how it works.

‍

To dig into the details, check out the Sovereign Structure GitHub repo.

Tips for the next Hackathon winner

Redpanda really is wildly easier to use than similar solutions in both the streaming and inference space. This, combined with great support in the Redpanda Community on Slack, allowed me to rapidly iterate on this project.

At the risk of helping someone else beat me at the next Redpanda Hackathon, I have some tried and true advice: listen and communicate well. It doesn't matter how cool something you built is, what matters is whether you can communicate value. That tidbit can be applied far beyond hackathons, too.

If you want to chat about distributed systems or learn more about Sovereign Stucture, find me on LinkedIn or in the Redpanda Community. To check out the other Hackathon projects that landed second and third place, watch Redpanda's winner announcement video.