How BYOC fits into your cloud governance framework

In the cloud-first era where compliance is king, organizations are increasingly favoring cloud offerings that allow them to keep their data contained in their own environment, while also offloading provisioning, monitoring, and maintenance to free up their teams.

Enter Bring Your Own Cloud (BYOC). BYOC is Redpanda Cloud’s fully managed offering where Redpanda clusters are automatically provisioned within the customer's cloud account, and continuously operated by Redpanda engineers to meet a strict availability SLA. With this privacy-first approach, sensitive data and credentials never leave the customer’s environment, giving them the convenience of a fully managed cloud and the confidence to meet their data sovereignty and privacy requirements.

When you combine the lower total cost of ownership (TCO) of SaaS and the security, reliability, and performance of Redpanda—it’s clear why the cloud services industry is shifting toward the BYOC model. However, when larger customers adopt BYOC, some are perplexed by the idea of integrating a third-party, fully managed Redpanda cluster into their cloud governance framework.

So, this post explains how BYOC fits into the cloud governance puzzle. We also explain how you can run BYOC clusters in your cloud—without losing control, compliance, or data sovereignty.

A dive into cloud governance frameworks

Since cloud governance setups can vary, let’s begin with a rundown of the different cloud governance frameworks.

Cloud governance frameworks define and codify change management workflows, ownership, policies, and architectural conventions across multiple dimensions, such as:

• Identity and Access Management (IAM)
• Network
• Compute
• Storage
• Cost management and attribution
• Security
• Cloud organizational structure: resource hierarchy and landing zones

Major cloud vendors, like AWS and Azure, provide tooling and best practices to help organizations bootstrap, enforce, and maintain their cloud governance models. As of this writing, Google Cloud’s professional services organization also maintains a GitHub repository that provides good guidelines and tooling for GCP.

Now, let’s zoom in on a few pillar concepts in cloud governance.

Landing zones

Landing zones (LZ) are a fundamental building block of a cloud governance strategy. The reasoning behind landing zones are well explained in this post by JV Roig. In short, LZs provide a smooth cloud adoption experience through fully integrated cloud accounts, aligned with organizational policies and configured with architectural foundations, such as CI/CD workflows to author and deploy infrastructure changes. Feel free to dive deeper into how AWS, GCP, and Azure approach landing zones.

At Redpanda, we use AWS Control Tower and AWS Control Tower Account Factory for Terraform to set up landing zones in AWS. We use Enterprise Scale for our Azure cloud and we're implementing Google's Fabric FAST framework in GCP.

Architecture

As a best practice, cloud providers recommend a hub-and-spoke network topology and a multi-account governance structure. Mainly because this architectural style provides the most robust security isolation and autonomy for teams to be productive in the cloud. However, working with this structure also requires a cultural shift for traditional Security and Network teams to ensure the focus is on enablement rather than gatekeeping.

In hub-and-spoke architectures, the Security, Network, and Reliability teams typically own core and shared infrastructure services; providing CI/CD pipelines, tooling, and guidance for the rest of the organization to integrate instead of doing the integration work for teams. Adopting this mindset allows core infrastructure teams to scale their functions without becoming an organizational bottleneck as internal cloud adoption grows.

In more technical terms, hub-and-spoke allows individual teams to own and manage their VPCs (spokes) with the network and security teams' enablement and oversight. To connect to shared and core infrastructure services (hubs), Private Links, VPC Peerings, and Transit Gateways are typically used, with unidirectional Private Links being the most secure choice.

The guidance from every major cloud provider regarding connectivity can be found in each link for AWS, GCP, and Azure.

Now that you understand the foundations of cloud governance, let’s introduce BYOC to the mix.

How Redpanda BYOC complements cloud governance

The Redpanda Cloud data plane is where your Redpanda cluster resides within Redpanda Cloud. It was designed from the ground up to be self-contained and smoothly deployed in cloud governance models that implement hub-and-spoke and multi-account (in AWS), multi-project (in GCP), or multi-subscription (in Azure) structures.

Customers who own and manage the cloud account with BYOC deployments have access to all BYOC cloud resources and configurations. This allows customers to audit all actions, set perimeter IAM guardrails, do proactive security and compliance checks, and so on.

Given that BYOC is a fully managed offering with a strict availability SLA commitment, we automatically provision, and continuously monitor, test, and enhance the entire data plane stack on behalf of our customers—without access to customer data.

While technically possible, we don’t recommend deploying fully managed BYOC clusters in cloud accounts shared with internal customer teams, especially if the data stored in Redpanda is highly confidential.

At Redpanda, performance, security, and reliability are first-class product features, not afterthoughts or SOC2 checkboxes. Subsequently, we proactively invest and prioritize them over functional features.

Shared responsibility model

Shared responsibility models have become the industry's way of clarifying expectations around cloud resource ownership. It harmonizes the relationship between cloud providers and their customers, given that the architectural boundaries of some cloud services can be fuzzy.

Since BYOC is a fully managed offering running in customers' cloud accounts, a small set of responsibilities are shared. However, we strive to ensure the user experience is and remains a smooth SaaS experience.

The following table shows the shared responsibility model across our current cloud offerings.

In this post, we’ll only focus on the responsibilities that we get asked about most often. (Of course, we can always elaborate on the others during your BYOC onboarding process.)

[CTA_MODULE]

With that, let’s dive into the responsibilities.

Cost management and attribution

Redpanda enables customers to easily tag all the cloud resources that make up a BYOC cluster. Customers are responsible for tracking cloud spending and cost attribution through their standardized tags. By default, we tag cloud resources with the following tags: redpanda_id, redpanda_org, and network_id.

A convenient property of BYOC clusters is that they use the same certified data plane we run in our Dedicated offering. So, any cost optimization improvement automatically reflects on BYOC.

Identity and access management (IAM)

Following best practices, Redpanda Cloud enforces the principle of least privilege, as well as segments access needs per workload type (Workload Identity) and cluster ID, and writes the code to grant those permissions.

Customers are responsible for reviewing and applying permissions using their private cloud credentials by running the rpk BYOC plugin. The rpk plugin contains all the IAM policies, roles, and service accounts required for a BYOC cluster to function correctly at all times. Customers are also responsible for ensuring the provisioned IAM resources are always present and correctly configured by periodically re-running the plugin.

Virtual private cloud (VPC)

VPCs are designed, codified, and tested by Redpanda—including subnets, firewall rules, and some routing rules. Similarly to IAM resources, customers are responsible for applying VPC resources through the rpk BYOC plugin and preventing drift by periodically re-running the rpk command.

Customers are also responsible for enabling VPC flow logs if their security or compliance needs require it, as well as monitoring and alerting their security team on any suspicious network activity.

For customers using cloud governance models where all security and network resources are centrally managed, we now offer customer-managed VPC, which allows you to deploy BYOC clusters on existing and customer-managed VPCs and IAM resources. You can read more about this new deployment option and other updates in our release announcement.

Access controls and audit

Redpanda defines IAM access controls and segments access according to workload type. Customers review and apply them through the Redpanda CLI (rpk). Customers are also responsible for managing peripheral controls, such as access to the cloud account. As well as VPC connections, such as Private Links, VPC peerings, and object storage buckets. Redpanda is responsible for managing, auditing, and monitoring access within the BYOC stack.

As far as auditing, customers are responsible for collecting, aggregating, auditing, and alerting suspicious cloud provider API events through their SIEM and logging infrastructure. Customers are also responsible for coordinating with Redpanda any preventive or mitigating actions since these can affect the correct functioning of BYOC clusters.

Operational trust boundary

The operational trust boundary is the area where Redpanda requests customers’ full trust to carry out the operations required to meet the “fully managed promise” and the SLA commitment. That said, and following cloud governance best practices, we have primarily defined one operational trust boundary:

AWS account, GCP project, or Azure subscription. Created and managed by the customer and dedicated solely to Redpanda Cloud clusters.

Some practical reasons are the following:

1. Cloud IAM implementations don’t always fully support restricting actions to particular cloud resources by ID, name, or tags. Since our cloud agents are managing the lifecycle of every resource we provision, it would be an unacceptable risk to grant permissions with a wide scope of action inside a shared cloud account and network.
2. To meet our availability SLA, we have to resolve most issues without human intervention. One of the things we do to meet this requirement is continuous drift correction, which requires IAM permissions to manage the cloud resources' full lifecycle at all times, not only during initial provisioning time.
3. A small percentage of situations will require human intervention from Redpanda engineers. When this happens, we temporarily use a zero-trust break-glass mechanism to enter the environment and mitigate the issue. Break-glass access is done only within the context of our incident response process and follows our customers' security and compliance regulations. The trust boundary strongly isolates this access from the rest of the customer's organizational cloud resources.

These responsibilities only scratch the surface of all the ways we ensure our customers can confidently own all the data they produce—without sacrificing the convenience of managed cloud services.

Complement your cloud governance with BYOC

Fully managed BYOC offers a popular SaaS-like experience while giving customers complete control over their data and the freedom from operations and maintenance. This pioneering approach to cloud has proven highly valuable for data privacy use cases where security, compliance, and reliability are must-haves.

In most situations, the best way organizations can prepare for BYOC is to model their cloud governance using a hub-and-spoke and multi-account structure—also recommended by cloud providers as a best practice.

Although, as individual organizations’ governance, compliance, and infrastructure policies vary, we’re continuously committed to offering the greatest level of deployment flexibility in the market with Redpanda Cloud. In line with that mission, we now offer customer-managed VPC deployments for BYOC, where customers can use BYOC with their existing VPC environments in this deployment type, including shared VPCs in GCP.

Want to see it in action for yourself? Sign up for a free trial of Redpanda Cloud! If you have questions about BYOC or streaming data, ask our team in the Redpanda Community on Slack.