LiveRamp modernizes its privacy-centric data infrastructure with BYOC

Challenge

Modernizing a batch processing infrastructure

LiveRamp, a data collaboration platform, helps companies build enduring brand and business value with technology solutions for customer intelligence, identity enhancement, cross-screen measurement, media networks, and more.

LiveRamp was traditionally reliant on batch processing systems like Apache Hadoop and Apache Spark™ but needed to adapt to the industry’s growing demand for real-time solutions. The organization’s legacy systems were built around encrypted batch files, making it challenging to meet the latency and privacy needs of time-sensitive issues, such as cart abandonment and ad suppression.

So, LiveRamp was looking to modernize its data infrastructure from traditional batch systems to streaming data systems. However, the privacy-focused AdTech environment meant LiveRamp needed a solution that could support both real-time processing and strict data privacy requirements.

This meant finding a scalable streaming platform that could be deployed across major cloud providers, allow for data sovereignty, and reduce complexity and costs while also improving performance.

Why Redpanda

Privacy and performance in a fully managed solution

Aside from steep data privacy and performance requirements, LiveRamp wanted to enable faster development and more efficient collaboration, which meant finding a partner with a robust streaming data ecosystem around its APIs. Ultimately, LiveRamp chose Redpanda as its partner due to the following reasons:

• All-in-one platform compatible with Apache Kafka^®  APIs
• Operational simplicity and cost efficiency
• Flexible deployment options across major clouds
• Privacy-centric BYOC model for secure operations
• Low-latency, high throughput data processing
• Collaborative support for specific infrastructure needs

During LiveRamp’s benchmarking, the team verified that Redpanda was not only capable of meeting their incredibly high performance requirements — but surpassing them by a significant margin. “We were blown away by the results,” says Abhishek Jain, Enterprise Platform Architect at LiveRamp

Abhishek also emphasizes the value of Redpanda BYOC since keeping data within LiveRamp’s network boundaries was a key requirement, but they also wanted a fully managed experience. “With BYOC, LiveRamp’s clusters remain in our own Virtual Private Cloud (VPC), while Redpanda manages the provisioning, monitoring, and maintenance via their secure agent. Data never leaves our VPC,” Abhishek explains.

Results

Less complexity, lower costs, happier engineers

With Redpanda at the core of LiveRamp’s event-driven architecture, the company has eliminated the complexity of its legacy batch system, boosting engineer productivity and enhancing the maintainability of its simplified codebase. LiveRamp has also significantly lowered its infrastructure costs and even its carbon footprint due to Redpanda’s hardware-efficient design.

As LiveRamp continues to help the AdTech ecosystem transition to addressable audiences without third-party cookies or mobile identifiers, Redpanda remains a key partner in syncing LiveRamp's vast offline reference graph, supporting billions of daily syncs from thousands of data sources.

With Redpanda’s continued success in everything from collecting pixel traffic for mobile applications to improving application monitoring tooling, LiveRamp is now expanding its use of Redpanda across the organization to support all incoming and outgoing data across more than 500 different platform integrations.

In addition, LiveRamp was glad to have a close partner that listened to feedback and tailored solutions. For example, both teams collaborated to build a Terraform wrapper layer to automate the deployment of Redpanda in LiveRamp’s environment — a workaround that was later integrated into the product itself.

Looking ahead, LiveRamp plans to simplify operations further with Redpanda Data Transforms (Wasm) to read data, prepare messages, and make API calls faster and easier. As the industry continues to change, LiveRamp is excited to have a reliable partner throughout its batch-to-streaming journey to help pioneer the real-time evolution of AdTech.