Search Icon
Search Icon

Kubernetes Quick Start Guide with external connectivity

The Kubernetes Quick Start guide describes how to quickly get up and running with a Kubernetes cluster. Those instructions only provide access to the cluster from within the Kuberenetes network.

Here we’ll show you an example of how to set up Kubernetes in Google GKE, Amazon EKS, or Digital Ocean so you can work with Redpanda from outside of the Kubernetes network.

Let’s get started…

Create a Kubernetes cluster

Create a 3-node cluster on the platform of your choice:

Use the EKS Getting Started guide to set up EKS. When you finish, you’ll have eksctl installed so that you can create and delete clusters in EKS. Then, create an EKS cluster with:

eksctl create cluster \
--name redpanda \
--nodegroup-name standard-workers \
--node-type m5.xlarge \
--nodes 3 \
--nodes-min 1 \
--nodes-max 4

It will take about 10-15 minutes for the process to finish.

Kubectl context

Most cloud utility tools will automatically change your kubectl config file.
To check if you’re in the correct context, run the command:

kubectl config current-context

For Digital Ocean for example, the output will look similar to this:

do-nyc1-redpanda

If you’re running multiple clusters or if the config file wasn’t set up automatically, look for more information in the Kubernetes documentation.

Prepare TLS certificate infrastructure

The Redpanda cluster uses cert-manager to create TLS certificates for communication between the cluster nodes.

We’ll use Helm to install cert-manager:

helm repo add jetstack https://charts.jetstack.io && \
helm repo update && \
helm install \
cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.2.0 \
--set installCRDs=true

Install the Redpanda operator and cluster

  1. Just to simplify the commands, create a variable to hold the latest version number:

    export VERSION=$(curl -s https://api.github.com/repos/vectorizedio/redpanda/releases/latest | jq -r .tag_name)

    Note - You can find information about the versions of the operator in the list of operator releases.
    We’re using jq to help us. If you don’t have it installed run this command:

    sudo apt-get update && \
    sudo apt-get install jq
  2. Install the latest redpanda operator:

kubectl apply -k https://github.com/vectorizedio/redpanda/src/go/k8s/config/crd?ref=$VERSION && \
helm repo add redpanda https://charts.vectorized.io/ && \
helm repo update && \
helm install \
--namespace redpanda-system \
--create-namespace redpanda-operator \
--version $VERSION \
redpanda/redpanda-operator
  1. Install a cluster with external connectivity:

    kubectl apply -f https://raw.githubusercontent.com/vectorizedio/redpanda/$VERSION/src/go/k8s/config/samples/external_connectivity.yaml
  2. Get the addresses of the brokers:

    kubectl get clusters external-connectivity -o=jsonpath='{.status.nodes.external}'

    The broker addresses are shown in the command output, for example:

    ["34.121.167.159:30249","34.71.125.54:30249","35.184.221.5:30249"]

    If you don’t get any response for this command, please check if the pods are healthy and are running with no errors.

    Commands like these will help you better understand what’s happening:

    kubectl describe statefulset external-connectivity
    kubectl describe pods external-connectivity-0
  1. Configure security access

When you run eksctl it automatically creates a lot of resources for you (dedicated VPC, new Security Group and etc). Because of that, you have to enter your security configurations and open the ports that external-connectivity uses in order to follow the next steps.

The easiest way to do that is to:

a. Get the ports that you need to open with the command that you ran in step 4.

b. Go to your Security Group configurations and check the newly created rule for your cluster.

c. Open TCP traffic to the ports.

If you don’t know how to do it, refer to the AWS guide for configuring VPCs and Security Groups.

Verify the connection

  1. From a remote machine that has rpk installed, get information about the cluster:

    rpk --brokers 34.121.167.159:30249,34.71.125.54:30249,35.184.221.5:30249 \
    cluster info

    Note - Check if you’re using the correct address and ports. Otherwise you may run into errors like:

    unable to create topics [chat-rooms]: invalid large response size 1213486160 > limit 104857600
  2. Create a topic in your Redpanda cluster:

    rpk --brokers 34.121.167.159:30249,34.71.125.54:30249,35.184.221.5:30249 \
    topic create chat-rooms -p 5
  3. Show the list of topics:

    rpk --brokers 34.121.167.159:30249,34.71.125.54:30249,35.184.221.5:30249 \
    topic list

Now you know how to set up a Kubernetes cluster in a cloud and access it from a remote machine.

Next steps

  • Check out our in-depth explanation of Kubernetes connectivity.
  • Contact us in our Slack community so we can work together to implement your Kubernetes use cases.