Jump Trading drives faster insights at scale to stay ahead in global financial markets

Challenge

Reliable performance for data-intensive workloads

Jump Trading is a proprietary trading firm specializing in algorithmic and high-frequency trading. The firm’s success relies heavily on real-time insights, which demand a streaming data infrastructure that supports Jump Trading’s data-intensive, mission-critical workflows.

After evaluating numerous leading messaging systems over the past 20 years for their log files and telemetry use cases, Jump Trading found each platform lacking in some combination of:

• Consistent performance, like tail latency and tolerance of slow consumers
• Reliable message delivery
• Cost-effectiveness and compatibility with a wide range of tools
• Access to low-level metrics to debug performance issues, particularly in cloud deployments

As Jump Trading grew, so did its workload. With the demand to process millions of messages per second across thousands of nodes, Jump Trading needed a new streaming data platform capable of scaling and supporting its real-time trading infrastructure.

Why Redpanda

Simplicity, performance, and open-source compatibility

Jump Trading’s search started with Apache Kafka^® but was hesitant about having a Java application in such a mission-critical role. So, Jump Trading looked to Redpanda due to its Kafka-compatible protocol, which integrates easily with the existing ecosystem while avoiding dependencies on the Java Virtual Machine (JVM).

Redpanda, written in C++, offered several benefits aligned with Jump’s requirements:

• Simplified installation, operation, and upgrades
• Prometheus support for performance monitoring
• Stable, low-latency messaging for predictable performance
• Kafka compatibility for easy integration with open-source tools
• Cost-effective data retention with Amazon S3-compatible tiered storage

“When selecting a streaming data platform, the choice for Jump was obvious,” said Alex Davies, CTO at Jump Trading. “For starters, we needed assurance we’d have zero data loss. We also needed something that would meet our high throughput requirements. That put Redpanda in a class of its own.”

Furthermore, Redpanda’s straightforward architecture promised a better experience for Jump Trading’s engineers. According to Alex, “The choice became a no-brainer.”

Results

A high-performing backbone for real-time data infrastructure

Jump Trading began with a self-managed deployment of Redpanda on bare metal, then added containerized deployments (on Podman and K8s). Today, the firm has expanded into fully managed, dedicated cloud instances, where Redpanda handles mission-critical workloads and reliably processes billions of messages daily.

One of Jump Trading’s largest, most complex workloads is its telemetry pipeline — which publishes market data, system measurements, and other information that needs to be consumed and processed in real time. With Redpanda, that data pipeline is fully supported and highly efficient with incredibly low latencies: 50 milliseconds for p95 and 150 milliseconds for p99.

“Jump Trading is a leader in global financial markets, and our streaming data infrastructure can make or break our business. With Redpanda, it’s not just meeting our requirements, but taking us to the next level.” Alex says.

Redpanda’s simplicity and alignment with open-source standards have also streamlined the learning process for engineers, enabling faster setup and reducing maintenance complexity. Built-in monitoring tools, like the Prometheus exporter, provide Jump Trading’s team with deep insights into system performance, further optimizing resource usage.

“By eliminating the Java dependencies in Kafka, Redpanda gave us less complexity and superior performance on like-for-like hardware,” Alex says.

After two years of successfully running in production, Redpanda has established itself as a reliable, high-performing backbone for Jump Trading’s real-time data infrastructure, helping the firm maintain a competitive edge as a leader in global financial markets.