Kubernetes rightsizing, from metrics to AI-assisted action

A technical guide to reclaiming CPU and memory using runtime evidence, recommendation models, policy, and AI.

The Technical Guide to Kubernetes Rightsizing with supporting resource review materials

342 pages · 57,000+ words · 50+ original illustrations · 4 runtime guides

The Technical Guide to Kubernetes Rightsizing shows which evidence to trust, what your application runtime changes, how recommendations are calculated, and which savings can survive contact with production.

By downloading, your email will be shared with CloudBolt. This sponsor may contact you about their products and services. You can unsubscribe from their communications at any time.
Sponsor privacy policies

You have probably seen this before:

  • You apply the memory recommendation and the pod is immediately OOMKilled. The new value did not leave enough room for the pod to start.
  • The p95 says CPU is low, but reducing the request increases throttling or latency. Samples can miss the short CPU spikes that determine user-facing latency.
  • The report is useful, but applying hundreds of recommendations manually is not. A useful report can still leave thousands of repositories, owners, exceptions, and approvals unresolved.
  • You rightsize the requests and suddenly the HPA no longer behaves as expected. Changing a CPU request also changes an HPA that scales on CPU utilization.

The cluster looks overprovisioned, so you start correcting requests and limits. But every value is connected to scheduling, Linux resource controls, application runtimes, autoscaling, and node capacity. Without a clear mental model of those interactions, one apparently safe change can create a failure somewhere else.

What's inside the book

Chapter 1

How Requests and Limits Actually Work

Average CPU is low. Why is the pod still being throttled?

This chapter follows requests and limits into the Kubernetes scheduler and Linux cgroups to show what is accounted, weighted, throttled, reclaimed, evicted, or killed.

  • Requests as scheduling commitments and CPU weights
  • CPU quota periods and throttling
  • Memory limits, eviction, and Kubernetes QoS

Chapter 2

Why Container Memory Is Not Application Memory

The application released 128MiB. Why did container memory stay flat?

Separate live objects, committed heap, RSS, cache, stacks, buffers, and native memory. Reproduce the cases where runtime and container measurements tell different stories.

  • Application memory versus container memory
  • Garbage collection, allocation, and retained pages
  • Runtime OOM errors versus containers marked OOMKilled

Chapter 3

Collecting Container, Runtime, and Kubernetes Metrics

The graph looks convincing. Did it mix two revisions and miss the spike that mattered?

Build a review record from container, runtime, application, and Kubernetes evidence. Preserve the observation window, revision, queries, and missing data that make a conclusion reproducible.

  • Container, runtime, application, and Kubernetes signals
  • Sampling gaps, stale series, restarts, and mixed revisions
  • Evidence provenance another engineer can challenge

Chapter 4

From Metrics to a Safe Recommendation

The recommendation says 516Mi. The pod needs more than that just to start.

Take KRR and VPA apart. See how lookback windows, percentiles, maxima, safety margins, decay, confidence, and OOM adjustments turn history into different candidate values.

  • KRR strategies and Prometheus samples
  • VPA targets, bounds, decay, and confidence
  • The application context neither model can infer

Chapter 5

Rightsizing at Scale Is a Policy Problem

The report has 2,100 recommendations. Which five deserve a pull request?

Move from one candidate to fleet-scale policy. Select material changes, preserve workload exceptions, find the source manifest, assign authority, reject stale work, and prove that released requests can become removable infrastructure.

  • Generation, selection, execution, and lifecycle policy
  • Ownership, evidence, exceptions, approval, and GitOps
  • Request opportunity, reclaimable capacity, and realizable savings

Chapter 6

Can AI Optimize Kubernetes for You?

The AI can open and merge the pull request. Who proved it should?

Use AI to assemble context, explain missing evidence, draft policy, and carry approved work across systems without replacing deterministic checks, explicit authority, or rollback conditions.

  • Purpose-built models for candidate calculation
  • LLMs for evidence synthesis and policy generation
  • Stale-change checks, production authority, and rollback signals

The Kubernetes limit is the same. The application inside it is not.

Each guide starts with a production sizing example, then tests the runtime controls and failure modes that a container-level recommendation cannot see.

Java and the JVM

The JVM heap is only part of a Java container’s memory. Thread stacks, direct buffers, metaspace, code cache, and native libraries sit outside it, while CPU limits also change garbage collection. A recommendation based only on container usage can leave the JVM without enough room.

Node.js

V8’s JavaScript heap is only part of a Node.js process. Buffers, external memory, workers, and native libraries sit outside it, while CPU limits affect the event loop and background work. Container usage alone cannot show which part is the real constraint.

Go

Go’s runtime needs to know the pod’s resource boundaries. GOMEMLIMIT and GOMAXPROCS shape garbage collection and concurrency, but defaults may follow the node instead of the container. Usage alone cannot show when runtime settings are causing the pressure.

Python

Python worker count is part of the resource budget. Each process has its own interpreter and memory, while native libraries can create extra threads outside the GIL. Container-level sizing can overcommit both memory and CPU before the application shows why.

Yasmin Rajabi
“Automation is essential, but trust decides how much you can rely on it.”

Yasmin Rajabi has worked across infrastructure automation at Puppet, Kubernetes optimization at StormForge, and cloud management at CloudBolt. Her foreword frames the distance between producing a recommendation and applying it with confidence.

Yasmin Rajabi · COO, CloudBolt Software

Questions before downloading

What this book will and will not give you.

Does the book give me exact CPU and memory values?

It shows how to calculate candidates from production history and load tests, then adjust them for runtime behavior, application outcomes, workload policy, and rollout risk. No fixed value can do that for every workload.

Do I need to use KRR, VPA, or Prometheus?

No. The book uses them to make recommendation and evidence mechanics concrete. The same questions apply to managed metrics backends and commercial optimization products.

Is this mainly a book about AI?

AI frames the opening question and the final operating model. Most of the book builds the Kubernetes, Linux, runtime, evidence, recommendation, and policy foundation an AI system needs before receiving production authority.

Is it useful if we already use an optimizer?

Yes. It gives you a model for interrogating recommendations, identifying missing context, setting workload policy, deciding approval and rollout modes, and checking whether reported savings are realizable.

Before you remove the headroom, understand what it is protecting.

Get the complete guide to Kubernetes resource mechanics, runtime behavior, recommendation models, fleet policy, realizable savings, and AI-assisted operations.

By downloading, your email will be shared with CloudBolt. This sponsor may contact you about their products and services. You can unsubscribe from their communications at any time.
Sponsor privacy policies

This series is made possible by

Diamond

Silver