Deployment
A Deployment is the canonical Kubernetes API object for managing stateless, replica-symmetric workloads. It is the
apps/v1resource that almost every user-facing service in the cluster — HTTP servers, gRPC backends, batch workers without persistent state, frontends — is shipped as. The user writes aDeploymentdescribing a Pod template and a desired replica count; the Deployment controller (a control loop insidekube-controller-manager) reacts to spec changes by creating a fresh ReplicaSet carrying the new Pod template, then ramping the new ReplicaSet up and the old one down according to a configurable rollout strategy, all while preserving previous ReplicaSets for rollback (kubernetes.io — Deployment). The Deployment is therefore not a primitive — it is a meta-controller: it does not directly manage Pods, but manages ReplicaSets which in turn manage Pods. This indirection is exactly what makes rolling updates, pause/resume, and N-revision history possible. The mental model to internalize: a Deployment is a history of ReplicaSets, parameterized by a strategy that controls how the cluster transitions between them.
Mental Model
flowchart TB USER[User: kubectl apply<br/>spec.template changed] DEPLOY[Deployment<br/>spec.replicas=5<br/>strategy.RollingUpdate<br/>maxSurge=25%<br/>maxUnavailable=25%] RS_OLD["ReplicaSet rev-3<br/>(old template)<br/>5 → 4 → 2 → 0"] RS_NEW["ReplicaSet rev-4<br/>(new template)<br/>0 → 2 → 4 → 5"] PODS_OLD[Old Pods<br/>v1.0] PODS_NEW[New Pods<br/>v2.0] READY[Readiness Probe<br/>gates rollout pace] DEADLINE[progressDeadlineSeconds<br/>default 600s<br/>ProgressDeadlineExceeded] HISTORY[Revision History<br/>revisionHistoryLimit=10<br/>retained for undo] USER --> DEPLOY DEPLOY -- "creates / scales" --> RS_NEW DEPLOY -- "scales down" --> RS_OLD RS_OLD --> PODS_OLD RS_NEW --> PODS_NEW PODS_NEW --> READY READY -. "Ready=True before<br/>next surge" .-> DEPLOY DEPLOY -. "no progress in deadline" .-> DEADLINE RS_OLD -. "kept around" .-> HISTORY
What this diagram shows. A user edits the Deployment’s spec.template (a new container image, an env var change, anything inside the Pod template). The Deployment controller hashes the new template, finds no matching existing ReplicaSet, and creates a fresh one (rev-4). It then gradually increases the new ReplicaSet’s spec.replicas and decreases the old ReplicaSet’s spec.replicas according to two knobs — maxSurge (how many Pods above desired may exist transiently) and maxUnavailable (how many below desired may exist transiently). The pace is gated by readiness probes: a newly created Pod must report Ready=True before the controller scales the new ReplicaSet up another step. If no progress happens within progressDeadlineSeconds (default 600), a ProgressDeadlineExceeded condition appears on the Deployment status and kubectl rollout status exits non-zero. Old ReplicaSets stick around (count bounded by revisionHistoryLimit, default 10) so kubectl rollout undo can flip back to a prior revision by reversing the same ramp. The insight to extract is that every state in the picture is durable in etcd — Deployment, both ReplicaSets, all Pods — so the controller can crash and resume mid-rollout without losing position, which is the level-triggered guarantee from Kubernetes Control Loop Pattern applied to rollouts.
Mechanical Walk-through
What the Deployment controller actually watches and does
The Deployment controller lives in kube-controller-manager and runs the canonical reconcile loop (see Kubernetes Control Loop Pattern). It watches three resource types:
- Deployments — its primary resource. Any change to a Deployment’s
specenqueues that Deployment’s key for reconciliation. - ReplicaSets — owned by Deployments via
metadata.ownerReferences. Any change to a ReplicaSet status (replicas became Ready, etc.) enqueues the owning Deployment. - Pods — indirectly, via the watch on owned ReplicaSets.
On reconcile, the controller computes:
- The hash of
spec.template. The current Pod template’s hash, used as the suffix on the managed ReplicaSet’s name (mydeploy-7c8f5b9d4where7c8f5b9d4is the truncated hash). This is the canonical identity of a revision — two Deployments with identical Pod templates would map to ReplicaSets with identical hashes. - The set of existing ReplicaSets owned by this Deployment, by selector. One of them, possibly, has a matching template hash — that’s the “current” RS. The rest are “old” RSs.
- The strategy’s intended next state. For RollingUpdate: how many new-RS Pods can we add right now (bounded by
maxSurge), and how many old-RS Pods can we drop right now (bounded bymaxUnavailable)?
The controller then issues Scale subresource patches against the relevant ReplicaSets — never directly creating or deleting Pods. ReplicaSet controllers handle Pod creation/deletion in their own reconcile loops. This split is the architectural reason Deployments are reliable: the Deployment controller is just shifting replica counts between objects; the ReplicaSet controllers do the actual Pod work. See ReplicaSet for that layer.
RollingUpdate strategy
The default strategy. Two knobs (kubernetes.io):
maxSurge(default25%). The maximum number of Pods that can exist above the desired replica count during the rollout. May be an integer or a percentage ofspec.replicas. Percentage values are rounded up.maxUnavailable(default25%). The maximum number of Pods that can be unavailable (not Ready, or being deleted) during the rollout. Integer or percentage, rounded down.
For a Deployment with replicas: 10 and the defaults, the controller targets at most 13 Pods total (10 + 25% surge, rounded up) and at least 8 ready Pods (10 − 25% unavailable, rounded down) at any moment of the rollout. Concretely the controller does this:
- Scale the new RS up by
maxSurge(here, +3 → 3 new Pods). - Wait for those Pods to become Ready (readiness probe passes; see Pod Probes).
- Scale the old RS down by
maxSurgeworth of replicas (−3 → 7 old Pods). - Repeat until the new RS has 10 replicas and the old has 0.
If maxUnavailable: 0 is set, the controller is forbidden from killing any old Pod before a replacement new Pod is Ready — surge-only rollout, zero-downtime guarantee. If maxSurge: 0 is set, the controller is forbidden from adding any new Pod before an old one is killed — drain-first rollout, useful when total cluster capacity is tight. Both cannot be 0 simultaneously (the controller would deadlock; the API server rejects this configuration).
Recreate strategy
The alternative (kubernetes.io). On a template change:
- The controller scales the old RS to 0, waiting for all old Pods to terminate.
- Then it scales the new RS up to
spec.replicas.
This is explicitly non-zero-downtime. It is the right choice when the workload cannot tolerate two versions running simultaneously — for example, a single-writer batch job that mounts a ReadWriteOnce volume (see Access Modes (PV)), or a legacy app whose old/new versions corrupt shared state if both run. Recreate is also the right choice for very fast iteration in development, where the downtime is preferable to managing parallel versions.
Readiness gates the rollout pace
maxSurge and maxUnavailable define the budget; readiness probes drive the pace. The Deployment controller treats a Pod as “available” only when:
- It has been Ready (readiness probe passing) for
spec.minReadySecondscontinuously (default 0). TheminReadySecondsfield adds extra waiting on top of readiness — useful when “ready” doesn’t mean “warmed up” (cache priming, JIT, connection pool fill). - It is not in the process of being deleted.
This is why a misconfigured readiness probe is the most common reason a rollout stalls: if the new Pods never report Ready, the controller never proceeds with scaling down the old RS, and the rollout sits at the surge cap indefinitely. Cross-reference Liveness Probe Misuse and Kubernetes Liveness Readiness Startup Probes.
Progress deadline
spec.progressDeadlineSeconds (default 600, i.e., 10 minutes per kubernetes.io) bounds how long the controller will wait without observed progress before flagging the rollout as failed. “Progress” means any of: the Deployment creates a new RS, the new RS gains a Ready Pod, the old RS loses a Pod. If progressDeadlineSeconds elapses with none of these events, the Deployment’s .status.conditions array gains a Progressing condition with status: False and reason: ProgressDeadlineExceeded. kubectl rollout status then exits non-zero — which is what CI/CD pipelines (Argo CD, Flux, the Jenkins K8s plugin) consume to decide “this deploy failed.”
Critically: a ProgressDeadlineExceeded does not trigger an automatic rollback. The Deployment continues to try the rollout (it’s still level-triggered). The user/CI must explicitly invoke kubectl rollout undo or update the Deployment’s spec back to a working configuration. This is a frequent misconception — see Failure Modes below.
Revision history and rollback
Every distinct spec.template ever applied to the Deployment becomes a ReplicaSet kept around for rollback. The bound is spec.revisionHistoryLimit (default 10). Older RSs beyond the limit are deleted (their Pods, if any, are deleted by the RS controller’s cascading delete via owner references — see Owner References and Garbage Collection).
kubectl rollout history deployment/<name> lists the retained revisions. kubectl rollout undo deployment/<name> rolls back to the immediately preceding revision; --to-revision=N targets a specific one. Mechanically, “undo” works by setting the Deployment’s spec.template back to the template of the chosen prior RS — which triggers a fresh rollout, in the opposite direction, using the same strategy. There is no special “rollback mode”; it’s just another rollout, which means it is also subject to readiness gating and progress deadlines.
Pause and resume
kubectl rollout pause deployment/<name> sets .spec.paused: true. The controller continues to reconcile but does not progress the rollout — it simply maintains the current mid-rollout split between old and new ReplicaSets. This is the canonical way to make several changes to a Deployment (image bump + env var change + resource request change) without triggering three separate rollouts. After all edits, kubectl rollout resume clears the flag and the rollout proceeds.
Two non-obvious points about pause:
- A paused rollout is not equivalent to a rolled-back rollout. Both
pausedandunpausedDeployments still hold the new template; the difference is only whether new RS scaling proceeds. Aborting a problematic rollout requireskubectl rollout undo, notkubectl rollout pause. - Scaling commands still work while paused.
kubectl scale deploymentchangesspec.replicasand the controller honors it. It scales the current RS (whichever one has the largest replica count at the time of pause), which can be subtle if you paused mid-rollout with both old and new RSs holding Pods.
Configuration / API Surface
A complete Deployment with line-by-line commentary:
apiVersion: apps/v1 # the stable Deployment API group/version
kind: Deployment
metadata:
name: web # Deployment name; managed RSs are name-<template-hash>
namespace: shop
labels:
app: web
spec:
replicas: 6 # desired Pod count; HPA may override this
revisionHistoryLimit: 10 # keep 10 old ReplicaSets for rollback (default 10)
progressDeadlineSeconds: 600 # 10 min; triggers ProgressDeadlineExceeded if no progress
minReadySeconds: 30 # Pod must be Ready for 30s before counted available
paused: false # default; set true to halt rollout mid-flight
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 25% # up to +25% Pods above replicas during rollout
maxUnavailable: 25% # up to 25% Pods unavailable during rollout
selector:
matchLabels:
app: web # MUST match template.metadata.labels;
# MUST be immutable post-create (a tightness K8s enforces)
template:
metadata:
labels:
app: web # at minimum, the selector's labels
spec:
terminationGracePeriodSeconds: 30 # SIGTERM → wait → SIGKILL
containers:
- name: web
image: registry.example.com/web:v2.7.1
ports:
- containerPort: 8080
name: http
readinessProbe: # gates rollout pace — most important probe for Deployments
httpGet:
path: /healthz
port: http
initialDelaySeconds: 5
periodSeconds: 10
livenessProbe: # restart unhealthy containers; NOT involved in rollout pace
httpGet:
path: /healthz
port: http
initialDelaySeconds: 30
periodSeconds: 30
resources:
requests: { cpu: "100m", memory: "128Mi" }
limits: { cpu: "500m", memory: "256Mi" }Notable surface points:
spec.selectoris immutable. Once the Deployment is created withselector: {matchLabels: {app: web}}, you cannot change those labels. Trying to do so returns aselector does not match template labels422. This is because the Deployment uses the selector to find its owned ReplicaSets — mutating it mid-life would orphan or misadopt RSs.spec.templateis the only mutable cause of rollout. Changingspec.replicastriggers a scale, not a rollout. Changingspec.strategychanges how future rollouts work but doesn’t trigger one. Only changing something insidespec.template(image, env, command, resources, labels-on-template, etc.) creates a new RS and starts a rollout.- The
pod-template-hashlabel. The Deployment controller injects apod-template-hash=<value>label into each ReplicaSet’s selector and Pod template. This is what letsselector: {matchLabels: {app: web}}work even though three different ReplicaSets all have Pods withapp=web— they additionally have distinctpod-template-hashvalues that the controller uses for disambiguation.
The relevant kubectl rollout subcommands:
kubectl rollout status deployment/web # blocks until rollout finishes; non-zero if it fails
kubectl rollout history deployment/web # list retained revisions
kubectl rollout history deployment/web --revision=3 # show the spec of revision 3
kubectl rollout undo deployment/web # roll back to previous revision
kubectl rollout undo deployment/web --to-revision=2
kubectl rollout pause deployment/web # halt mid-rollout
kubectl rollout resume deployment/web # continue after pause
kubectl rollout restart deployment/web # force a rollout with no template change
# (annotates template with kubectl.kubernetes.io/restartedAt)kubectl rollout restart is worth special mention: it works by adding/updating a kubectl.kubernetes.io/restartedAt annotation on the Pod template, which is a template change — therefore a new template hash, therefore a new RS, therefore a fresh rollout. This is the standard way to force every Pod to be replaced (e.g., to pick up a new ConfigMap that the app reads on startup) without having to actually change the spec.
Failure Modes
-
Stuck rollout, all new Pods unready. The new RS scales up to
maxSurgePods, but none of them ever reach Ready. The rollout sits there, eventually hittingProgressDeadlineExceeded. Almost always a misconfigured readiness probe (wrong path, wrong port, port unbound), a startup bug in the new image (crashloop), or a missing dependency (DB unreachable, missing Secret). Diagnose withkubectl describe pod,kubectl logs,kubectl describe deploymentfor the conditions list. -
ProgressDeadlineExceededdoes not auto-rollback. Many users assume a failed rollout reverts. It does not. The Deployment will keep retrying forever (level-triggered: the new RS still exists in spec). The user/CI mustkubectl rollout undoor fix-forward. Tools like Argo Rollouts add automatic rollback on top of native Deployments; stock Deployments don’t have it. -
maxUnavailable=0 with a tight node budget. Setting
maxUnavailable: 0requires the cluster to have headroom for an extramaxSurgeworth of Pods simultaneously. On a cluster sized exactly to the desired replica count, the new Pods can’t be scheduled, the new RS stays Pending, and the rollout deadlocks. Diagnose viakubectl get podsshowing Pending status andkubectl describe podshowingFailedScheduling. The fix is either to lowermaxSurgeto a value the cluster can absorb, or to scale the cluster (see Cluster Autoscaler, Karpenter). -
Concurrent edits during pause. A paused Deployment that’s mid-rollout with the old RS at 4 Pods and the new RS at 2 Pods has 6 Pods total of two different versions. A user running
kubectl scale deployment web --replicas=10while paused will see the controller scale only one of the two RSs (typically the larger one) to 10, leaving the other at 2 — for a total of 12. Confusing if not expected. The remedy is to resume before scaling, or scale and let the resume normalize. -
Hash collisions on the pod-template-hash. Theoretically possible but practically vanishing: the controller uses FNV-1a → base32 → 10-char suffix, with collision avoidance via
.status.collisionCount(incremented on collision; mixed into the hash on retry). If you ever see this in the wild, it’s almost certainly a bug — file an issue. -
Selector and template labels drift. Editing a Deployment to change
spec.selector.matchLabelsis rejected. But editingspec.template.metadata.labelsto no longer contain the selector’s labels is silently accepted — and immediately fails because the new RS Pods don’t match the selector. Symptom:kubectl get rsshows the new RS with 0 ready Pods despite Pods existing. Fix: keep selector labels and template labels in sync. -
Rollout works in dev, fails in prod due to PodDisruptionBudget conflicts. During an automated rollout, the controller respects Pod Disruption Budget only for voluntary evictions during node drains — not for rolling updates. But if your PDB and
maxUnavailablesettings are inconsistent (PDB allows 0 disruptions, Deployment’smaxUnavailable: 50%), you’ll see node drains block forever during cluster upgrades while Deployment rollouts proceed normally. Diagnose: compare PDBminAvailable/maxUnavailableto Deployment strategy. -
revisionHistoryLimit: 0 disables undo. Setting
revisionHistoryLimit: 0causes old RSs to be deleted immediately.kubectl rollout undofails withno rollout history found. Set to ≥1 if rollback is desired (default 10 is fine for almost everyone). -
spec.strategy.typechange mid-rollout. Switching from RollingUpdate to Recreate while a RollingUpdate is in progress doesn’t terminate the in-flight rollout cleanly — the new strategy applies on the next template change, but the current rollout finishes under the old rules. Surprising but documented. -
kubectl rollout restartdoesn’t restart the cluster autoscaler’s view. Forcing a Pod replacement viarollout restartadds a template annotation, creating a new RS with a new pod-template-hash. If you have an HPA in place tuned to the old RS’s metrics endpoint, you may briefly see scale events from the HPA reacting to the rolling transition. Usually self-correcting; occasionally surprising.
Alternatives and When to Choose Them
- StatefulSet — choose when Pods need stable identities and stable storage. Databases, leader-elected clusters, Kafka brokers, etcd. Deployments are explicitly the wrong tool for these: their Pods are interchangeable and ephemeral, with no identity guarantees.
- DaemonSet — choose when the workload is per-node (log shipper, monitoring agent, CNI plugin). Deployment doesn’t model node-locality; DaemonSet does.
- Job / CronJob — choose when the workload is run-to-completion rather than long-running. Deployments restart their Pods on exit (the default
restartPolicy: Alwaysis enforced in the template); Jobs let Pods exit successfully. - Argo Rollouts — choose when you want advanced progressive delivery (canary with metric-driven analysis, automatic rollback, blue-green with traffic-shifting Service swap). Replaces Deployment with the
RolloutCRD. Worth it once you’re past simple rolling updates. - Knative Serving — choose for request-driven autoscale-to-zero stateless services. Builds on top of Deployments but adds revision routing and KPA scaling. Heavier; use only if scale-to-zero is essential.
- Manual ReplicaSet + Service swap (Blue/Green) — choose if you want atomic cutover semantics: spin up a second ReplicaSet at full capacity, then swap a Service’s selector to point at it. Deployments don’t support this natively (their cutover is gradual); Argo Rollouts or hand-rolled YAML are the usual approach. See Blue-Green Deployment on Kubernetes.
Production Notes
- Spotify’s Backstage uses Helm-managed Deployments for almost all stateless plugins, with HPA on top. The standard pattern: Deployment + Service + HPA + PodDisruptionBudget, generated by a Helm chart per microservice. Spotify’s published “Golden Path” docs treat Deployment as the only default workload type developers should reach for, with platform team approval required for StatefulSet/DaemonSet.
- Shopify’s chargeback platform uses Deployments for everything except their MySQL/Vitess cluster (StatefulSet) and node-level monitoring (DaemonSet). Their internal SLO is “no Deployment rollout should take longer than 10 minutes” — they tune
progressDeadlineSecondsto enforce this. - k8s.af incidents involving Deployments (Kubernetes Failure Stories) cluster around: (a) misconfigured readiness probes causing rollouts to stall and downstream services to keep hitting old replicas while clients lose their connections; (b)
revisionHistoryLimitset too high (e.g., 100), accumulating thousands of empty ReplicaSets that bloat etcd and the API server’s watch cache; (c)maxSurge: 0+maxUnavailable: 100%(“kill everything first then ramp up”) used by accident, causing total service outages. - The HPA + Deployment combo. A Deployment’s
spec.replicasis also the target of the Horizontal Pod Autoscaler’s scale subresource. When an HPA is attached, the HPA ownsreplicasand the Deployment owns the template. The user should not editreplicasdirectly when an HPA is in place — it’ll just be overwritten on the next HPA reconcile. - GitOps with Deployments. ArgoCD and Flux reconcile Deployments by applying their full spec from Git. A surprising consequence: if you
kubectl edita Deployment’sreplicasdirectly while ArgoCD is managing it, ArgoCD will flip it back. The remedy is either ahelm.sh/resource-policy: keepannotation, theignoreDifferencesArgoCD field, or — better — letting the HPA manage replicas and pinning everything else in Git.
See Also
- ReplicaSet — the resource Deployment manages; the actual Pod replication primitive
- Pod — what a ReplicaSet manages
- Pod Probes / Kubernetes Liveness Readiness Startup Probes — readiness drives rollout pace
- Rolling Update Strategy — the strategy as a pattern (cross-vault, not just Deployments)
- Recreate Strategy — the other option
- Blue-Green Deployment on Kubernetes — alternative cutover model
- Canary Deployment on Kubernetes — alternative progressive-delivery model
- Argo Rollouts — the advanced replacement
- Horizontal Pod Autoscaler — what scales the Deployment dynamically
- Pod Disruption Budget — the voluntary-disruption budget that interacts with rollouts during node drains
- Kubernetes Control Loop Pattern — the reconciliation model the Deployment controller is an instance of
- Owner References and Garbage Collection — how Deployment→RS→Pod cascade-deletes work
- Kubernetes MOC — umbrella index