Redpanda Serverless: the simplest ingredient for developing real-time applications

Real-time data is no longer a nice-to-have, but a must-have when creating relevant and engaging user experiences. Most industries today have grown accustomed to consuming instant updates, so if you’re a frontend developer looking to break into real-time app development, you’ll need to master the flow of real-time data.

As a developer advocate at Redpanda, my job is to help enable developers to leverage streaming data in their applications. Part of that involves introducing developers to better technologies and showcasing them in practical and fun use cases.

So, in this post, I’ll demonstrate how I used three modern technologies — Redpanda Serverless, Pusher, and Vercel — to create a compelling real-time frontend application, which I hope will spark your own ideas for how you can implement this powerful trio in your world.

The cupcake conundrum

Imagine a bustling cupcake business in NYC. To reel in customers in such a competitive market, they’ll need to make their location readily visible to nearby cupcake fans. They’ll also need to engage their customers via immediate feedback to build trust, enhance the overall user experience, and drive repeat business and customer loyalty.

However, developing real-time applications has been traditionally difficult as they are designed to respond to user inputs rather than continuously listen for and process incoming data streams. The latter requires a robust and complex infrastructure to manage persistent connections and handle high volumes of data with minimal latency.

For the visual learners, here’s a quick video explaining the use case:

Selecting the right technologies

I chose Redpanda Serverless as the streaming data platform since traditional streaming data solutions, like Apache Kafka®, can be complex and resource-intensive, making it a massive hurdle for teams with limited time and resources.

In contrast, Redpanda Serverless…

• Eliminates infrastructure overhead: It manages the backbone of streaming data, allowing me to focus on application logic.
• Simplifies scalability: Effortlessly scales with my application's needs, accommodating spikes in data without manual intervention.
• Reduces time to market: With a setup time of seconds, it speeds up development for quicker iterations and feedback.
• Pay as you grow: It adapts to my usage, ensuring costs align with my actual data processing needs, which is ideal for startups and small projects.

This takes care of the complex infrastructure for dealing with high volumes of data and the low latency that’s expected of real-time applications.

Now, I need to establish a single, long-lived connection between the browser and the server, typically done through WebSocket, which sets up a full-duplex communication channel over an HTTP connection. This allows the server to push updates to the browser client without needing periodic requests.

However, Vercel doesn't support WebSocket, so I needed an alternative solution. Here's where Pusher pops up. Pusher lets me create real-time channels between the server and client, simplifying the complexity associated with directly using WebSocket.

When deploying real-time frontend applications, Vercel stands out for its seamless Git repository integration that makes deployments easy. With a push of a button, changes are automatically updated and I can get website statistics and data from other solutions (like databases) when needed.

Preparing the application

In my application, mapview.js acts as a Vercel serverless function, which plays the most important role by consuming data from the topic I created in Redpanda Serverless and then updating the inventory status.

Before using Pusher to relay these updates to the front end, Serverless maps the store IDs in store_nyc.csv to their physical locations and then adds the location information (latitude and longitude) that the client needs to render.‍


await consumer.run({
      eachMessage: async ({ topic, partition, message }) => {
        const messageData = JSON.parse(message.value.toString());
        const location = storeLocations[messageData.store];
        const { store, ...rest } = messageData;
        for (let store in inventory) {
            inventory[store].latest = false;
        }

        inventory[messageData.store] = { ...rest, ...location, latest: true };
        try {
          pusher.trigger("my-channel",channelId, JSON.stringify(inventory));
        } catch (error) {
          console.error('Error:', error);
        }

      },
    })

Note: Vercel serverless functions have a maximum duration limit, which varies depending on your subscription plan. So, I set the MAX_BLOCK_TIME to five seconds. The Pro plan allows up to 300 seconds of execution for a better user experience.

  await new Promise(resolve => setTimeout(resolve, MAX_BLOCK_TIME) );

On the front end, index.html presents the real-time map using the LeafletJS libraries and inventory updates, giving the end users a dynamic and interactive experience.‍


channel.bind('cupcake-inv', function(data) {
   var inventory = data;
   tableBody.innerHTML = '';
   for (var store in inventory) {
    var storeData = inventory[store];
    if (markers[store]) {
       markers[store].setLatLng([storeData.lat, storeData.lng])
                        .setPopupContent(`<b>${storeData.store}</b><br>Blueberry: ${storeData.blueberry}<br>Strawberry: ${storeData.strawberry}`);
    } else {
       markers[store] = L.marker([storeData.lat, storeData.lng]).addTo(map)
                        .bindPopup(`<b>${storeData.store}</b><br>Blueberry: ${storeData.blueberry}<br>Strawberry: ${storeData.strawberry}`);
    }

It also generates a unique session ID per session to create channels in Pusher, so each session will have its unique channel to receive updates.‍


channel.bind(uniqueChannelId, function(data) {
      var inventory = data;
      for (var store in inventory) {
            var storeData = inventory[store];
……
document.addEventListener('DOMContentLoaded', () => {
            fetch(`/api/mapview?channelId=${encodeURIComponent(uniqueChannelId)}`)

The recipe: real-time cupcake updates with Redpanda Serverless, Vercel, and Pusher

It’s time to start cooking! Here's a step-by-step breakdown of how I brought this vision to life, which you can follow. If you want to skip ahead, you can find all the code in this GitHub repository.

Step 1. Set up Redpanda Serverless

1. Sign up and create the cluster: After signing up, click the Create Cluster button and select
2. a region close to your workload, ensuring low latency for your data.
3. Create the user and set permissions: Under the Security tab, create a new user and set the necessary permissions.
4. Create the topic: Create a topic called inv-count that’s dedicated to tracking cupcake stock updates.

Step 2. Integrate Pusher for real-time updates

Register the application: After creating an app within Pusher, copy the application credentials, including the app_id, key, secret, and cluster information, and store them to use in your application.

Step 3. Deploy with Vercel

1. Integrate with GitHub: Push the updated codebase to a GitHub repository, ensuring your changes are version-controlled and ready for deployment.
2. Import and set up the project in Vercel: Navigate to Vercel and import the project by selecting the “cupcakefanatic” repository. Specify cupcake-pusher as the root directory for the deployment.
3. Configure the environment: Enter the project-specific environment variables.

With that, I can establish a seamless real-time connection between the server and clients, enhancing the store’s online presence and user engagement — without the heavy lifting traditionally associated with real-time streaming data.

Below is a screenshot of the resulting real-time data in our cupcake app.

With the winning combination of Redpanda Serverless, Pusher, and Vercel, I easily created a dynamic, responsive application that keeps customers informed and engaged with live inventory updates.

The proof is in the pudding: Redpanda Serverless simplifies real-time app development!

For front-end developers looking to develop real-time applications, Redpanda Serverless is a compelling solution. It eliminates the technical overhead associated with traditional streaming data services, enabling developers to focus on creating engaging and interactive user experiences.

To start streaming data in seconds with Redpanda Serverless, sign up for a free trial and try it yourself! If you have questions, ask me in the Redpanda Community on Slack.

‍