What does each team's Kubernetes cost?

On a cloud bill, Kubernetes looks simple: you pay for nodes. Twenty VMs, a load balancer, some disks. But who's actually using those nodes? Ten teams, fifty services, a few hundred pods, all sharing the same machines. When finance asks "how much does the search team's infrastructure cost?", the cloud bill can't answer.

That's the Kubernetes cost visibility problem. Here's how to solve it, and how to reduce the bill once you can see it.

Splitting the bill

To allocate cluster costs to teams, you need to know how much of each node each pod uses or reserves, and what that node costs.

OpenCost is a CNCF project that does exactly this. It combines:

• Pod resource requests and actual usage (CPU, memory, GPU, storage).
• Prices from your cloud provider's billing API (including discounts and Spot pricing) or custom prices for on-premises.
• Kubernetes metadata: namespaces, labels, deployments.

The result: cost per namespace, per label (like team or app), per workload, per hour.

Kubecost (now part of IBM) builds on OpenCost with more features: savings recommendations, budgets, alerts, multi-cluster views and a polished UI. Cloud providers also have their own Kubernetes cost features, like split cost allocation for EKS in AWS, and cost allocation in GKE and AKS.

helm install opencost opencost/opencost --namespace opencost --create-namespace
kubectl port-forward -n opencost svc/opencost 9090

A few minutes later, you see costs per namespace.

The "idle" problem

One thing that surprises teams the first time: a big chunk of the cost belongs to nobody. That's idle capacity: parts of nodes that no pod requested.

If your nodes are only 50% requested on average, half the cluster is idle. You decide how to handle it: share it out proportionally, assign it to the platform team, or (best) reduce it.

You can't fix what you can't see. Once teams see their Kubernetes cost, half the optimisation happens on its own.

Labels make or break it

Cost allocation is only as good as your labels. Agree on a few standard labels for every workload:

• team or owner
• app or service
• environment
• cost-center (if finance needs it)

Enforce them with a policy (Kyverno or Gatekeeper), so new workloads can't be deployed without them.

Packing nodes tightly

Once you see the waste, reducing it usually comes down to utilisation: getting more useful work out of each node.

• Right-size requests. Over-requested pods reserve capacity they never use. VPA recommendations and Goldilocks help find realistic values.
• Consolidation. Karpenter continuously looks for underused nodes and moves pods so it can remove them or replace them with smaller, cheaper ones. The Cluster Autoscaler also scales down underused nodes, more conservatively.
• Better instance choices. Mixed instance types, newer generations and ARM-based nodes can lower cost per CPU.
• Bin-packing scheduling. The default scheduler spreads pods out. Scheduler configuration that favours fuller nodes, or Karpenter's consolidation, packs them tighter.
• PodDisruptionBudgets so consolidation can move pods safely without causing outages.

Spot on Kubernetes

Spot instances can cut node costs dramatically. Kubernetes is a great fit because it reschedules pods automatically when a node disappears.

Good practice:

• A separate Spot node pool (or Karpenter NodePool) for fault-tolerant workloads.
• Diversify instance types, so you're not dependent on one Spot pool.
• Handle interruption notices (Karpenter and the AWS Node Termination Handler do this) to drain nodes gracefully.
• Multiple replicas and PDBs for anything running on Spot.
• Keep stateful and critical singleton workloads on on-demand nodes.

Showback or chargeback

What do you do with the numbers?

• Showback: show each team its costs. No money moves, but visibility changes behaviour. Most companies start here.
• Chargeback: actually charge teams' budgets for their usage. More accountability, more arguments about fairness, especially for shared and idle costs.

I'd always start with showback: a weekly report or dashboard per team, with trends and the biggest opportunities. It's surprising how quickly teams fix their own waste when the number has their name on it.

So, where do I start?

Install OpenCost, make sure every workload has a team label, and share a cost-per-team dashboard. Then look at idle capacity, right-size the biggest over-requesters, and enable consolidation. Spot comes next, for the workloads that can handle it.

My take: Kubernetes makes sharing infrastructure easy, and sharing costs invisible. Make them visible, and teams will help you bring them down.

— Desly