Akamai sharpens threat intelligence and futureproofs streaming pipelines

Challenge

Escalating costs and latency in critical data pipelines

Protecting the edge at scale demands enormous data capabilities. Every second, millions of digital experiences — from streaming videos and websites to apps and software — are powered by Akamai. As the steward of one of the world’s largest Content Delivery Networks (CDNs), Akamai quietly makes the internet faster, more reliable, and inherently secure through an edge-computing network spanning over 4,100 locations in more than 135 countries.

“30% of the internet bandwidth is going through Akamai networks,” says Idan Harel, Infrastructure Team Lead in Akamai’s Cloud Security Intelligence (CSI).

Akamai’s Cloud Security Intelligence (CSI) division ingests massive volumes of security event data from its global infrastructure, with its largest feed handling over three million events per second. These data streams fuel threat detection, content-rich analytics, and real-time enforcement at the edge.

The CSI team’s existing platform used Confluent Cloud and object storage, which began to present a growing number of challenges:

• Escalating costs of Confluent Cloud as data volumes exploded
• High latency caused by batch processing via object storage
• The existing solution had object storage scaling issues with Linode

“We had a big data platform that processed events from our security products and made them accessible to other security systems. But our Kafka just referenced files stored in object storage. Latency was an issue, and the costs were growing,” explained Yaniv Kunda, Senior Software Architect at Akamai.

Why Redpanda

Cost and resource efficiency, with private cloud compatibility

Looking to simplify its architecture, reduce infrastructure spending, and future-proof its data pipeline, Akamai discovered Redpanda.

“Confluent Cloud had become one of our top third-party expenses,” said Yaniv. “When we ran the numbers, Redpanda came in at about 55% lower costs, and that includes both cloud and self-hosted alternatives.”

The team initially adopted Redpanda as a Kafka-compatible, self-hosted replacement for Confluent Cloud. But after validating its performance and ease of operation, they realized they could aim higher.

“Once we saw how resource-efficient Redpanda was, we realized we had headroom to grow. That opened the door to rethinking our whole pipeline.”

Previously, stages in the processing flow were separated by object storage. With Redpanda’s low-latency capabilities, they began shifting more data directly into Kafka topics and moving toward a true, real-time streaming architecture.

To support the transformation, Akamai deployed Redpanda Self-Managed Enterprise, powering two of its most critical application security systems: Cloud Security Intelligence (CSI) and Client-Side Protection and Compliance (CPC). While the initial deployment was on Azure, Akamai set a strategic goal to bring more workloads in-house to Linode, its recently acquired cloud platform.

Results

55% cost reduction and accelerated development cycles

Akamai’s shift to Redpanda optimized performance and resource efficiency, further advancing real-time insights and cybersecurity at the edge for its customers. The impact on Akamai’s team was undeniable, with key advantages including:

• Cost reduction: 55% savings compared to Confluent Cloud and self-hosted alternatives
• Low latency: enables near real-time processing across global data streams
• Scalability: replaced object storage limitations with a cleaner, more efficient architecture
• Developer agility: Redpanda Console gives developers instant insights into their topics
• Strategic alignment: runs on Akamai-owned Linode infrastructure, giving them full visibility and control
• Smooth upgrades: despite running on Akamai’s private Linode cloud, three major upgrades were performed with zero downtime

Redpanda’s team was instrumental in ensuring a smooth deployment. Despite running on Akamai’s private Linode cloud, three major upgrades were performed with zero downtime.

“Self-hosting can be painful, especially on a private cloud,” said Yaniv. “But Redpanda’s support team stood by us the entire way, even when the issues weren’t theirs. They helped us establish everything we needed for a stable deployment.”

In addition to stability, Redpanda delivered what the DevOps team at Akamai needed: greater control and fewer restraints. Previously, running a cloud-based service meant they couldn’t fully tune or customize the platform. With Redpanda, they regained ownership of their environment.

“With SaaS, you’re always working within someone else’s constraints. Running Redpanda ourselves let us fit the platform to our exact needs and scale on our terms,” Idan explains. Meanwhile, all existing DevOps tooling, such as Terraform scripts, automation, and user/topic management processes, continued working unchanged.

“We didn’t have to rewrite a thing. That was a huge advantage.”

Based on this success, Akamai’s CSI division is already planning its next architectural move. Today, much of their long-term event data is stored in flat files accessed with Jupyter-based tools. Now with Redpanda’s support for Apache Iceberg™, they plan to shift toward a modern, open data lakehouse format with lower operational overhead and more flexible querying.

Furthermore, as Akamai infuses AI-powered features into its threat intelligence products, Redpanda serves as a reliable streaming backbone, delivering the low-latency data pipelines that AI models need to perform in real time.