How poolside uses Redpanda BYOC to stay ahead in the competitive world of AI

Challenge

Choosing the right building blocks to automate pipelines

poolside is building the world’s most capable AI-driven software development tools to enhance developers' efficiency and capabilities. The company trains large language models (LLMs) to understand and execute coding tasks, improving and streamlining software development processes.

Initially, poolside chose the Apache Kafka^® ecosystem to connect the real-time data pipelines needed to train AI models. However, Kafka isn’t optimized for composable architecture, the cloud, edge computing, or the high throughput and massive processing requirements of LLMs and AI applications.

“We wanted the power of the Kafka API without the limitations of an aging platform optimized for obsolete hardware,” says a founding poolside engineer.

Why Redpanda

Owning all the data and infrastructure without the management

Redpanda BYOC offered the “best of both worlds,” where poolside owns the infrastructure and stays in control over their data, while Redpanda handles day-to-day cluster management. In brief, poolside chose Redpanda due to its:

• Reliable, 24/7 support
• Complete compatibility with Kafka APIs
• BYOC for fully managed services without the heavy lifting
• Easy-to-use UI to monitor and control data movement
• Complete visibility into compute and network resources
• Significantly lower total costs compared to Kafka

The poolside team deployed Redpanda BYOC clusters on its on-prem cloud, offering GDPR compliance and full data sovereignty, ensuring the data remains inside the company’s virtual private cloud (VPC) while being accessible to poolside’s customers.

Redpanda also connects poolside’s data pipelines across different teams, enabling the company’s engineers to sample data, catch and correct mistakes, and seamlessly integrate different streaming jobs, making it easier to process data.

Results

Moving faster without reinventing the wheel

With Redpanda in its stack, poolside can iterate faster, avoid delays and missed deadlines, and launch new features sooner to stay highly competitive. With AWS powering its infrastructure, poolside now uses Redpanda to train its ML models at massive scale and speed. It used to take weeks for these training jobs to complete. With Redpanda, it takes a day.

Speed and flexibility are essential when training LLMs. As poolside relies on Redpanda’s streaming capabilities to easily integrate and update external data sources, the company can improve the accuracy and relevance of its AI-generated output using retrieval-augmented generation (RAG). Everything plays together so well that they can move more data faster.

poolside’s engineers also have full visibility and control while offloading the time-consuming work of cluster management to Redpanda. “Everybody knows where the data is and how it flows from one place to another," says the poolside engineer. "Redpanda does many little things that add up to a great workflow.”

As Redpanda automates some of the most challenging and time-consuming aspects of building streaming data pipelines, poolside engineers have more time to bring new product ideas to life.