The average Kubernetes cluster runs at somewhere between 20% and 40% actual resource utilization, sixty cents or more of every infrastructure dollar going toward capacity that sits idle. Many teams are still running on resource requests copied from a dev environment years ago and never revisited.
This guide covers resource allocation fundamentals, rightsizing mechanics, how Horizontal Pod Autoscaler (HPA), Vertical Pod Autoscaler (VPA), and the Cluster Autoscaler work together, scheduling and bin packing, the monitoring tooling needed for visibility, and a practical set of best practices for sustained optimization.
Understanding Resource Allocation and Configuration in Kubernetes
Before we can optimize anything, we need to understand the fundamental mechanics of how Kubernetes manages resources. Get this foundation wrong and every optimization effort you build on top of it will be flawed.
Resource Requests and Limits Fundamentals
Every container can have two resource specifications: resource requests and resource limits.
Resource requests define the minimum guaranteed resources a container needs. The scheduler uses requests as its primary input for placement, filtering out nodes without enough allocatable resources, then scoring what remains. Without requests, the scheduler is flying blind.
Resource limits cap the maximum CPU and memory a container can consume. Hitting the CPU limit throttles it; exceeding the memory limit triggers an OOM kill. Limits stop any single container from monopolizing shared node resources.
Requests and limits together determine a pod's Quality of Service (QoS) class:
Misconfigured requests and limits are one of the most common sources of cluster instability and wasted spend: underestimate them and pods get scheduled onto nodes that can't support load under pressure; overestimate them and the scheduler reserves capacity that never gets used. A practical heuristic from production experience is to set requests around the p50 to p75 of observed usage and limits around the p95 to p99 range.
CPU Requests, Throttling, and Memory Pressure
The scheduler sums the CPU requests already on a node against its allocatable CPU. A pod won't be scheduled on a node that can't satisfy its request, regardless of current actual usage.
CPU throttling happens when a container tries to use more CPU than its limit allows. The Linux kernel's CFS (Completely Fair Scheduler) throttles it once it exceeds the limit within a scheduling period, reducing throughput and raising latency. According to Google Cloud, a CPU utilization-to-limit ratio of 0.8 or higher signals a container is being throttled and its limit needs raising.
Memory is non-compressible, so Kubernetes can't throttle it, exceeding the memory limit gets a container killed outright (OOMKilled, visible in pod events and restart counts). Prevention comes down to setting memory limits with appropriate headroom based on actual observed usage, not whatever was in the dev manifest.
At the node level, watch for MemoryPressure (low available memory), DiskPressure (disk approaching capacity), and PIDPressure (approaching the process ID limit), all signal something upstream has gone wrong with resource allocation.
Resource Quotas and Allocation Policies
Resource quotas enforce boundaries on total CPU and memory requests and limits across a namespace, preventing any team from consuming a disproportionate share, once a namespace hits its quota, new pods simply won't create. LimitRange objects complement quotas by setting default requests and limits for containers that don't specify their own, important because containers without requests land in BestEffort class and get evicted first.
Quotas are a governance tool, not an optimization tool. They enforce guardrails and prevent sprawl in multi-tenant clusters; they don't right-size anything on their own.
Rightsizing Workloads and Eliminating Resource Wastage
With the fundamentals in place, let's talk about the biggest lever for cost optimization in most production clusters: rightsizing.
Overprovisioning and Underprovisioning Challenges
Overprovisioning is the most common problem in mature clusters: workloads allocated far more CPU and memory than they consume. Nothing breaks, so nobody notices, and the cluster just runs on mostly-idle nodes, usually from specs copied out of dev with inflated safety margins, requests set once at deployment and never revisited, or "just in case" limits with no visibility tooling to catch the gap. Every unit requested but unused is capacity unavailable to other pods, forcing the Cluster Autoscaler to provision nodes that shouldn't be necessary. According to CAST AI's Kubernetes optimization report, most clusters have significant room for rightsizing that translates directly into node count reduction.
Underprovisioning is more immediately painful: requests set too low cause throttling, OOM kills, and degraded performance, and the symptoms look like application bugs rather than infrastructure problems. Both directions are the same failure, resource wastage, just pointed opposite ways.
Workload Analysis and Resource Pattern Identification
You can't rightsize what you haven't measured. Collect CPU and memory usage over two to four weeks using Prometheus or kubectl top, long enough to capture weekly cycles, batch jobs, and recurring spikes. Historical metrics collected over this window are what VPA and other rightsizing tools use to generate recommendations.
Bursty workloads (web servers, APIs, event processors) need requests sized for normal load but limits sized for peak load. Steady-state workloads (background workers, pipelines, cron jobs) are easier to rightsize because p50 and p95 sit close together. Track the gap between requested and actual usage, a pod requesting 2 CPU cores but using 0.3 is a 1.7-core gap, pure waste, and multiplying that across dozens or hundreds of pods is where most cloud bills are actually going.
Rightsizing Strategies and Resource Tuning
Start with the highest-cost workloads first, ranked by requested resource cost (requests × node cost), and attack the top 20% of wasted spend before touching anything else. Reduce requests gradually rather than all at once, watching for pressure signals (throttling, OOM kills, latency) and validating before moving to the next workload. Canary rollouts before broad deployment keep rightsizing from becoming a reliability incident.
Be careful with CPU limits specifically, setting them too low is a common cause of performance degradation that looks like an application bug; some teams omit CPU limits entirely on certain workloads, keeping only memory limits, reasoning throttling does more damage than the noisy-neighbor risk. Stateful workloads (databases, message brokers, StatefulSets) need more caution than stateless ones, their caching behavior needs consistent headroom, so lean conservative on request reductions there. Resource tuning has no end date, what was accurate six months ago may be wrong today.
Autoscaling Mechanisms for Dynamic Resource Optimization
Horizontal Pod Autoscaler (HPA)
The horizontal pod autoscaler scales pod replica count based on observed metrics, most commonly CPU, but also memory or custom and external metrics via the metrics API:
desiredReplicas = ceil(currentReplicas * (currentMetricValue / desiredMetricValue))
The part teams get wrong: HPA's CPU calculation is a ratio of actual usage to what the pod requested, not to node capacity. Inflated requests make HPA think utilization is low, so it won't scale up when it should.
Set target utilization in the 60-70% range rather than 80%+ for room to scale before users feel it, configure stabilization windows to prevent oscillation, and test under realistic load before trusting it in production. HPA has limitations: it reacts to metrics with inherent delay and can't handle extremely rapid spikes, where pre-warming or predictive scaling are better complements.
Vertical Pod Autoscaler (VPA)
Where HPA changes how many pods run, VPA changes how much each pod requests, automated rightsizing. VPA operates in three modes:
VPA generates recommendations from historical usage percentiles, and these are useful even without Auto mode.
The critical gotcha is the VPA + HPA conflict: both autoscalers should never act on the same CPU or memory signal for the same workload, or they oscillate, VPA raises requests, HPA reads the changed ratio and scales down, VPA reacts again. The safer pattern is VPA managing requests while HPA scales on an independent signal like queue depth. For teams starting out: run VPA in recommendation-only mode for four to eight weeks before ever enabling Auto mode.
Cluster Autoscaler and Node-Level Scaling
The cluster autoscaler watches for Pending pods that can't be scheduled and provisions nodes to fit them, then removes nodes when they're underutilized. It uses resource requests to evaluate whether a node can fit pending pods, so inflated requests provision more nodes than necessary, and deflated requests make nodes look removable when real usage says otherwise.
For scale-down, configure a utilization threshold (below which a node becomes a candidate), a delay (how long it must stay underutilized first), and an unneeded-time buffer before actual removal. Pod disruption budgets are the safety mechanism, blocking node removal that would violate availability requirements for critical workloads.
The most effective setup runs all three together: HPA for demand-driven pod scaling, VPA for accurate per-pod requests, Cluster Autoscaler matching node count to the real workload footprint.
Pod Scheduling, Bin Packing, and Node Utilization
Autoscaling handles dynamic changes, but the quality of your baseline scheduling has a huge impact on cluster utilization and cost.
Scheduler Behavior and Resource-Aware Pod Placement
The scheduler works in two phases: filtering, which eliminates nodes that can't satisfy a pod's requests, taints, tolerations, or affinity rules, and scoring, which ranks what's left.
Node capacity (total hardware) and allocatable resources (capacity minus what's reserved for the kubelet, system processes, and eviction thresholds) aren't the same thing, and the scheduler only works with the latter, a 16-core node might have only 14.5 allocatable cores, and ignoring that gap causes scheduling surprises. Taints, tolerations, and affinity rules give fine-grained placement control but, overused, fragment the scheduling space and make bin packing harder, treat them as precision tools, not defaults.
Bin Packing and Pod Density Optimization
Bin packing schedules pods onto the fewest possible nodes: pods are items, nodes are bins, minimize the bin count. By default the scheduler uses LeastAllocated scoring, which spreads pods for resilience. For cost, the opposite, MostAllocated via the NodeResourcesFit plugin, packs pods onto already-fuller nodes before opening new ones. Bin packing maximizes density but concentrates blast radius; spreading improves resilience at the cost of more nodes. A hybrid, bin packing for stateless workloads, spreading for stateful or critical ones, works for most environments.
Matching node instance types to a workload's CPU-to-memory ratio matters too: CPU-heavy, memory-light workloads exhaust CPU capacity while memory sits unused, forcing new nodes open regardless.
Node Utilization and Capacity Planning
Two numbers matter: allocated capacity (the sum of requests on a node) and actual utilization (real consumption). The gap between them is the optimization opportunity, 80% allocated against 30% actual utilization is a major rightsizing target.
Capacity planning should be forward-looking: project growth from historical patterns rather than reacting after hitting limits. Running nodes at 90%+ allocation looks efficient but leaves no headroom for spikes, and provisioning takes time. A practical target for most production clusters is 65-75% node utilization.
Resource Contention and Its Impact on Cluster Performance
Resource contention is the noisy-neighbor problem: multiple pods on one node competing for CPU or memory beyond what's available, degrading everything on that node at once. Detect it top-down: pod-level CPU usage relative to limits (kubectl top pods), the throttle ratio (≥0.8 is a strong signal), restart counts for OOM kills, then node-level MemoryPressure and CPU saturation if pod metrics look fine.
Mitigation comes back to QoS: set requests equal to limits (Guaranteed) for workloads that must not be evicted, Burstable with sensible limits for everything else. In multi-tenant environments, contention in shared namespaces is a governance problem, quotas, priority classes, and LimitRange policies are what keep one team from degrading everyone else.
Monitoring, Visibility, and Optimization Tooling
You can't optimize what you can't see. Visibility is the operational foundation that makes everything else in this guide actionable.
Metrics Server and Core Resource Visibility
The Metrics Server is the lightweight resource metrics API built into Kubernetes, collecting real-time CPU and memory data from each node's kubelet. It powers kubectl top and is the data source HPA and VPA depend on, but it holds no historical data, just a real-time snapshot, so it can't support the trend analysis serious optimization work needs.
Monitoring Tools and Observability Platforms
The standard stack is Prometheus for metrics collection, Grafana for dashboards, and Kubecost or OpenCost for cost allocation. Prometheus slices utilization data at pod, namespace, and node levels; Grafana surfaces patterns invisible in raw metrics; Kubecost translates utilization into dollar amounts, which is what actually builds the business case for optimization.
Alerts worth configuring: sustained CPU throttling (>20% over 15 minutes), any production OOM kill, node MemoryPressure or DiskPressure, low node utilization, and restart-count spikes. Datadog and Dynatrace are mature managed alternatives for teams that don't want to run their own stack.
Optimization Recommendations and Automated Insights
VPA's Off mode is the most accessible starting point, container-level recommendations from real historical usage with zero disruption risk. Goldilocks wraps those recommendations in a cluster-wide dashboard, and Kubecost attaches a dollar figure to every resize recommendation, which is what actually gets them prioritized. The more mature pattern integrates these into GitOps: recommendations open pull requests with proposed changes instead of requiring manual manifest edits, and the normal review-and-merge flow handles rollout.
Resource Efficiency Reporting and Cluster Utilization Metrics
Efficiency metrics matter more than raw utilization because they account for what was requested, not just what hardware exists: request utilization ratio (actual ÷ requested; 30% average means requesting 3x actual need), limit utilization ratio (actual ÷ limit; high values signal throttling or OOM risk), and cluster-wide utilization (the aggregate across namespaces and node pools). An average request utilization of 25-35% signals a major rightsizing opportunity.
For multi-tenant clusters, chargeback (assigning costs to the teams that generated them) and showback (the same visibility, no financial transfer) work as accountability mechanisms, teams that see what their workloads cost tend to optimize them.
Best Practices and Strategies for Sustained Kubernetes Resource Optimization
Always define resource requests for every container, no exceptions. Containers without them land in BestEffort class and get evicted first.
Don't set requests equal to limits for every workload. That creates Guaranteed QoS pods that can't burst, sounds safe but causes unnecessary throttling on workloads that could otherwise absorb occasional spikes with slack capacity. Reserve Guaranteed for workloads where protected resources genuinely matter.
Implement namespace-level resource quotas everywhere, not just multi-tenant clusters, they're organizational guardrails: visible capacity, accountable consumption, even in single-tenant environments.
Review resource configurations on a cadence: every two to four weeks while initially tuning a workload, monthly once it stabilizes. These practices are consistently recommended across production Kubernetes environments.
On cost specifically: spot or preemptible instances deliver real savings for interruption-tolerant workloads, batch jobs, background workers, dev/staging, paired with autoscaling for fast reprovisioning, while critical services stay on-demand. Consolidating underutilized nodes cuts licensing, monitoring, and kubelet overhead proportionally, and a 20-percentage-point improvement in request utilization typically translates into a meaningful drop in node count and spend.
For capacity planning, combine historical patterns with growth projections rather than reacting after hitting limits, and schedule periodic audits, overprovisioning drift accumulates quietly as deployments change and nobody revisits the configuration.
Resource optimization is not a one-time project. Workloads change, and what was optimized six months ago may be wrong today. Clusters stay efficient when there's a feedback loop, monitoring surfaces opportunities, pipelines act on them, and developers and platform engineers work the loop together rather than in isolation.
Slash Your Cloud Costs with Rackspace Spot
Once you've optimized your Kubernetes resource configurations as described throughout this guide, the next lever for cost reduction is the underlying infrastructure you're running on. That's where Rackspace Spot comes in.
Rackspace Spot is a managed Kubernetes offering built on spot market infrastructure. Instead of paying fixed on-demand rates for worker nodes, Rackspace Spot uses an open market auction to price compute capacity, with potential savings of up to 90% versus typical fixed-rate instances. For teams that have done the rightsizing work and are running well-optimized workloads, moving eligible workloads to spot capacity is a natural next step that compounds the infrastructure efficiency gains you've already made.
The platform includes tools that align well with the optimization strategies in this guide:
- MCP server: Rackspace Spot provides an MCP server that helps with workload placement and optimization decisions
- Web Dashboard: visibility into your cluster's resource utilization and cost allocation
- spotctl CLI and Terraform Provider: infrastructure-as-code workflows for teams managing clusters through GitOps pipelines
- Managed Kubernetes: turnkey cluster deployment so you spend time on optimization, not cluster operations
- Open-market auction pricing: compute capacity priced through real-time bidding instead of fixed rates, with bids starting as low as $0.001 per hour
Rackspace Spot supports hybrid clusters that mix spot and on-demand nodes in the same environment. This means you can run stateless, fault-tolerant workloads on spot capacity for maximum cost savings while keeping critical services on stable on-demand instances. Rackspace highlights this as the right pattern for cost efficiency without sacrificing reliability on critical workloads.
If you're serious about combining Kubernetes resource optimization with infrastructure cost savings, visit spot.rackspace.com to see how the platform can complement the optimization work you're already doing.
Frequently asked questions
What's the difference between resource requests and resource limits in Kubernetes?
Resource requests define the minimum guaranteed resources a container needs and are used by the scheduler for pod scheduling decisions. Resource limits cap the maximum resources a container can consume. Exceeding the CPU limit causes throttling; exceeding the memory limit causes an OOM kill. Requests affect scheduling placement; limits affect runtime enforcement.
Can I run HPA and VPA together on the same workload?
Yes, but with important caveats. The critical rule is not to let both autoscalers act on the same CPU or memory utilization signal for the same workload, or they'll oscillate against each other. The safer pattern is to use VPA for managing resource requests and HPA driven by an independent custom metric (like queue depth or requests per second) so the two autoscalers aren't interfering with each other's signals.
How often should we review and adjust our Kubernetes resource configurations?
When you're initially tuning a workload, review every two to four weeks to catch throttling, OOM, or utilization signals quickly. Once configurations have stabilized, a monthly review cadence is appropriate for most production workloads. Build periodic resource audits into your operational calendar to prevent gradual overprovisioning drift from accumulating unnoticed over time.
What's the biggest mistake teams make with Kubernetes resource optimization?
The most common mistake is the set-and-forget pattern: configuring resource requests and limits at initial deployment and never revisiting them as the workload evolves. Traffic patterns change, application code changes, and the deployment environment changes, but the resource configuration stays the same. The result is gradual drift toward either significant overprovisioning (wasted spend) or underprovisioning (reliability incidents). Treating resource optimization as an ongoing operational discipline rather than a one-time configuration task is the mindset shift that separates mature platform teams from everyone else.