More than one cluster

Almost every company starts with one Kubernetes cluster. Then a second one for staging. Then one in another region. Then a team needs its own for compliance reasons. Then there's the edge, the data science team, the acquisition... And suddenly someone asks "how many clusters do we have?" and nobody knows the exact answer.

Multi-cluster isn't a decision most teams make once. It's something that happens. The question is whether you manage it on purpose.

Why one cluster isn't enough

There are good reasons to run more than one:

• Environment isolation. Production shouldn't share a cluster with experiments.
• Blast radius. A bad upgrade, a misconfigured admission webhook or a control plane problem only affects one cluster.
• Regions and latency. Users in Europe and Asia are better served by clusters near them.
• Compliance and data residency. Some data must stay in a specific country or a specially controlled environment.
• Team autonomy. Some teams need cluster-level permissions or special configurations.
• Scale limits. Very large clusters hit practical limits on nodes, pods and API load.

And some less good reasons: every team getting its own cluster just because it was easy to click one together.

Every cluster you add is another thing to upgrade, secure, monitor and pay for. Make each one earn its place.

Managing a fleet

Once you have more than a few clusters, doing things by hand stops working. You need a fleet approach.

GitOps is the foundation. Argo CD or Flux deploy the same baseline to every cluster from Git: monitoring agents, policies, ingress or gateway controllers, cert-manager. Argo CD ApplicationSets with the cluster generator are especially useful: define an application once, and it's deployed to every cluster that matches a label.

Cluster lifecycle tools create and upgrade the clusters themselves:

• Cluster API manages Kubernetes clusters with Kubernetes resources, across many providers.
• Terraform or OpenTofu with managed services (EKS, AKS, GKE) is the most common approach.
• Rancher, Open Cluster Management and cloud-specific fleet tools (like GKE fleets) add a central management layer.

Standardisation is what makes it scale. Clusters should be cattle, not pets: created from the same templates, with the same baseline, so any engineer can work on any cluster.

Connecting clusters

Sometimes services in different clusters need to talk to each other. Options:

• Through public or internal load balancers: simple, works everywhere, but each service needs to be exposed explicitly.
• Cilium Cluster Mesh: connects pod networks across clusters, with global services that load-balance across them.
• Istio multi-cluster: service mesh across clusters, with mTLS and traffic policies.
• Submariner and the Multi-Cluster Services API: Kubernetes-native ways to export services from one cluster to others.

My advice: avoid cross-cluster dependencies when you can. Each cluster serving a complete slice of the system (for example a region) is much simpler than services constantly calling across clusters.

Identity and secrets everywhere

With many clusters, access control gets tricky:

• Single sign-on for humans, with the same groups and roles mapped in every cluster. No shared kubeconfigs.
• Workload identity so pods authenticate to cloud services without static credentials.
• A central secret store (Vault, OpenBao, or your cloud's secret manager) with External Secrets Operator in each cluster, instead of copying secrets around.
• Policy everywhere: the same Kyverno or Gatekeeper policies deployed to every cluster through GitOps.

Observability across the fleet

Each cluster sending metrics, logs and traces to a central place is essential. Label everything with the cluster name and region. A dashboard showing all clusters side by side, with versions and health, quickly answers "is it just this cluster, or all of them?"

Also track versions: which Kubernetes version and which add-on versions each cluster runs. Version drift across a fleet is a classic source of "it works in cluster A but not B".

So, how many clusters should you have?

As few as your requirements allow, and as many as your isolation, region and compliance needs demand. Whatever the number, manage them as a fleet from day one: GitOps for everything, the same baseline everywhere, and central observability.

My take: multi-cluster is manageable when every cluster looks the same. The trouble starts the day one of them becomes special.

— Desly