Getting started with Redpanda using PHP

In this blog, I’m [Nick Chiu] going to show you how to set up Redpanda and use it with PHP.
We will build a very simple producer and consumer example in a Docker setup.
This is an introduction to Redpanda, so if you already have experience with it
or similar streaming platforms, you can check out some of our posts on more advanced topics,
like this one on consistency, or this one on autotuning.

As you may have heard by now, Redpanda is Kafka® API-compatible,
which means that although Redpanda is relatively new, you can leverage the countless client libraries
created for Kafka® (if you find something that is not supported, reach out to our team on our Slack community.
In this case we will use the PHP extension simple_kafka_client.

Prepare Docker setup

First create a Dockerfile for our PHP container:

FROM php:8.0-cli-alpine3.13

# Install packages
RUN apk --no-cache add bash gcc g++ make autoconf && \
    apk add librdkafka librdkafka-dev \
                    --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/community && \
    pecl install simple_kafka_client && \
    docker-php-ext-enable simple_kafka_client

Then let's create our docker-compose.yml like follows:

version: '3.7'
services:
  php:
    build:
      context: ./
    tty: true
    working_dir: /src
    volumes:
      - ./src:/src
  redpanda:
    entrypoint:
      - /usr/bin/rpk
      - redpanda
      - start
      - --smp
      - '1'
      - --reserve-memory
      - 0M
      - --overprovisioned
      - --node-id
      - '0'
      - --kafka-addr
      - PLAINTEXT://0.0.0.0:29097,OUTSIDE://0.0.0.0:9097
      - --advertise-kafka-addr
      - PLAINTEXT://redpanda:29097,OUTSIDE://redpanda:9097
      - --check=false
    image: vectorized/redpanda:v21.4.12
    ports:
      - 9097:9097
      - 29097:29097

Create the PHP code examples

Next we create the src folder with our examples.
This folder will be mounted into our Docker setup.

mkdir src;cd src

Then let's create our producer.php in the src folder with the following content:

<?php

declare(strict_types=1);

use SimpleKafkaClient\Configuration;
use SimpleKafkaClient\Message;
use SimpleKafkaClient\Producer;

$conf = new Configuration();
$conf->set('metadata.broker.list', 'redpanda:9097');
$conf->set('compression.codec', 'zstd');

$producer = new Producer($conf);
$topic = $producer->getTopicHandle('php-test-topic');

for ($i = 0; $i < 10; ++$i) {
    $topic->producev(
        RD_KAFKA_PARTITION_UA,
        RD_KAFKA_MSG_F_BLOCK, // will block produce if queue is full
        sprintf('test message-%d',$i),
        sprintf('test-key-%d', $i),
        [
            'some' => sprintf('header value %d', $i)
        ]
    );

    $producer->poll(0);
}

$result = $producer->flush(20000);

echo sprintf('Produced %d messages', $i) . PHP_EOL;

if (RD_KAFKA_RESP_ERR_NO_ERROR !== $result) {
    echo 'Was not able to shutdown within 20s. Messages might be lost!' . PHP_EOL;
}

Next let's create our consumer.php in the src folder with the following content:

<?php

declare(strict_types=1);

use SimpleKafkaClient\Configuration;
use SimpleKafkaClient\Consumer;

$conf = new Configuration();
$conf->set('group.id', 'php-consumer');
$conf->set('metadata.broker.list', 'redpanda:9097');
$conf->set('auto.offset.reset', 'earliest');
$conf->set('enable.partition.eof', 'true');

$consumer = new Consumer($conf);
$consumer->subscribe(['php-test-topic']);

while (true) {
    $message = $consumer->consume(20000);

    if (RD_KAFKA_RESP_ERR__PARTITION_EOF === $message->err) {
        echo 'Reached end of partition, shutting down' . PHP_EOL;
        break;
    } else if (RD_KAFKA_RESP_ERR__TIMED_OUT === $message->err) {
        echo 'Timed out without receiving a new message, waiting for more messages...' . PHP_EOL;
        continue;
    } else if (RD_KAFKA_RESP_ERR_NO_ERROR !== $message->err) {
        echo kafka_err2str($message->err) . PHP_EOL;
        continue;
    }

    echo sprintf(
            'Read message with key:%s payload:%s topic:%s partition:%d offset:%d',
            $message->key,
            $message->payload,
            $message->topic_name,
            $message->partition,
            $message->offset
        ) . PHP_EOL;
}

Running the examples

Now let's put it all together and run our examples:

docker-compose up -d
docker-compose exec php php producer.php
docker-compose exec php php consumer.php

Wrapping up

As you can see, it is a breeze to setup PHP integration with Redpanda.
Try it yourself, there are endless use cases - what we built here was just the simplest of examples.

By Special Guest, Nick Chiu @thr33One

Join our Redpanda Slack Community.