Security should be at the heart of the design of any software project, and Redpanda is no exception. Here we’ll dive into authorization, one of the baked-in mechanisms that helps you make your Redpanda cluster more secure. If you want to know about authentication with mTLS and SASL, check out our docs on security. But if you want a demonstration of ACLs, follow along.
Note: System security requires attention to many domains, such as network configuration and organization-wide roles management. Here we only address the security of client access to data. Make sure to review all of your other security requirements on a regular basis.
Access control lists (ACLs) make sure your data is protected by granular, least-privilege policies. By following this guide, you will get to know how to set up Redpanda with ACLs, both on a conceptual level and on a technical level, through hands-on examples.
While authentication is about making sure that whatever client connects to your cluster is verified as a trusted client, authorization is about making sure that each client has access to exactly the data it should. ACLs are the main mechanism supported by Redpanda to manage permissions. Our implementation of ACLs is Kafka-compatible, too!
At a high level, ACLs specify what authenticated users can or cannot do.
ACLs are all about who from where can or cannot do what action to what object.
In ACL terminology, the entities accessing the resources are called principals.
Principals are users (
User:) that are allowed or denied access to resources.
You can also specify from which hosts the principals are allowed or denied.
For more granular ACLs, you can allow or deny principals access to specific resources, such as specific topics, and for specific operations, such as “read”, “write”, or “describe”. These ACLs really lock down access to your data so you can rest assured that no one is getting unauthorized access.
The fields and values you can use are:
|Principal for which access will be granted or denied. (repeatable)||User name, consumer group name|
|Host (from where)||--allow-host|
|Host to which permissions will be granted or denied. (repeatable)||Host IP address|
|Cluster (resource type)||--cluster||Whether to grant ACLs to the cluster.||Cluster name|
|Group (resource type)||--group||Group to grant ACLs for. (repeatable)||Group name|
|Operation (action)||--operation||Operation that the principal will be allowed or denied. Can be passed many times.|
|Resource Pattern Type Matching||--resource-pattern-type||Pattern to use when matching resource names (literal or prefixed. Default "literal"). If literal, it matches literal foo. If prefix matches foobar, for example.|
|Topic (resource type)||--topic||Topic to grant ACLs for. (repeatable)||Topic name|
|Transactional id (resource type)||--transactional-id||Transactional IDs to grant ACLs for. (repeatable)||Transactional id used|
In addition to other Kafka-compatible tools, ACLs are managed with the Redpanda rpk utility with the commands:
- Creating ACLs -
rpk acl create
- Listing ACLs -
rpk acl list
- Deleting ACLs -
rpk acl delete
- Create an ACL allowing a user to perform all operations from all hosts to a topic named “pings”:
rpk acl create \--allow-principal 'User:Charlie' \--allow-host '*' \--operation all \--topic pings
rpk confirms that you created the ACL:
PRINCIPAL HOST RESOURCE-TYPE RESOURCE-NAME RESOURCE-PATTERN-TYPE OPERATION PERMISSION ERRORUser:Charlie * TOPIC pings LITERAL ALL ALLOW
You can also use a single command to create multiple ACLs, including ACLs that deny certain principals access to a resource and allow other principals access to the same resource:
rpk acl create \--cluster \--allow-host 126.96.36.199 \--deny-host 188.8.131.52 \--deny-principal 'User:david' \--allow-principal 'User:Alex' \--allow-principal 'User:Ben' \--operation 'all'
rpk confirms that you created multiple ACLs:
PRINCIPAL HOST RESOURCE-TYPE RESOURCE-NAME RESOURCE-PATTERN-TYPE OPERATION PERMISSION ERRORUser:Ben 184.108.40.206 CLUSTER kafka-cluster LITERAL ALL ALLOWUser:david 220.127.116.11 CLUSTER kafka-cluster LITERAL ALL ALLOWUser:david 18.104.22.168 CLUSTER kafka-cluster LITERAL ALL DENY
Now let’s see what the ACLs look like for a specific user:
rpk acl list --allow-principal 'User:david' --deny-principal 'User:david'
This list of ACLs is shown with all of their parameters:
PRINCIPAL HOST RESOURCE-TYPE RESOURCE-NAME RESOURCE-PATTERN-TYPE OPERATION PERMISSION ERRORUser:david 22.214.171.124 CLUSTER kafka-cluster LITERAL ALL DENYUser:david 126.96.36.199 CLUSTER kafka-cluster LITERAL ALL ALLOW
Notice that you can choose which flags to use in specific situations.
Of course, you can list ACLs with other filters, like host, permission and resource.
You can check a list of possible flags with
rpk acl list -h
If you want to see them all execute
rpk acl list.
rpk acl delete allows you to delete ACLs.
It’s important to note, however, that wildcard values such as
any for operations and permissions, as well as
* for hosts and resources may result in the deletion filters matching more than one ACL.
Delete all ACLs for a specific user targeting a specific resource:rpk acl delete --allow-principal 'User:david' --deny-principal 'User:david' --cluster
Redpanda will ask you for confirmation? Confirm deletion of the above matching ACLs? (Y/n)
All of the related ACLs are listed as deleted:DELETIONS=========PRINCIPAL HOST RESOURCE-TYPE RESOURCE-NAME RESOURCE-PATTERN-TYPE OPERATION PERMISSION ERRORUser:david 188.8.131.52 CLUSTER kafka-cluster LITERAL ALL DENY
Delete all ACLs granting a principal permissions to all operations from a host:rpk acl delete \--allow-principal 'User:Ben' \--allow-host 184.108.40.206 \--operation all
This is just a small example of what we can do with ACLs.
If you want to dive deeper, check our in-depth guide about ACLs.
We are moving fast to keep up with the demand for Redpanda, the intelligent data API, and security with ACLs is only one of the many features that we’re working on.
Join our Slack community to stay up on the latest developments.