Deployment

A Deployment is the canonical Kubernetes API object for managing stateless, replica-symmetric workloads. It is the apps/v1 resource that almost every user-facing service in the cluster — HTTP servers, gRPC backends, batch workers without persistent state, frontends — is shipped as. The user writes a Deployment describing a Pod template and a desired replica count; the Deployment controller (a control loop inside kube-controller-manager) reacts to spec changes by creating a fresh ReplicaSet carrying the new Pod template, then ramping the new ReplicaSet up and the old one down according to a configurable rollout strategy, all while preserving previous ReplicaSets for rollback (kubernetes.io — Deployment). The Deployment is therefore not a primitive — it is a meta-controller: it does not directly manage Pods, but manages ReplicaSets which in turn manage Pods. This indirection is exactly what makes rolling updates, pause/resume, and N-revision history possible. The mental model to internalize: a Deployment is a history of ReplicaSets, parameterized by a strategy that controls how the cluster transitions between them.

Mental Model

flowchart TB
    USER[User: kubectl apply<br/>spec.template changed]
    DEPLOY[Deployment<br/>spec.replicas=5<br/>strategy.RollingUpdate<br/>maxSurge=25%<br/>maxUnavailable=25%]
    RS_OLD["ReplicaSet rev-3<br/>(old template)<br/>5 → 4 → 2 → 0"]
    RS_NEW["ReplicaSet rev-4<br/>(new template)<br/>0 → 2 → 4 → 5"]
    PODS_OLD[Old Pods<br/>v1.0]
    PODS_NEW[New Pods<br/>v2.0]
    READY[Readiness Probe<br/>gates rollout pace]
    DEADLINE[progressDeadlineSeconds<br/>default 600s<br/>ProgressDeadlineExceeded]
    HISTORY[Revision History<br/>revisionHistoryLimit=10<br/>retained for undo]

    USER --> DEPLOY
    DEPLOY -- "creates / scales" --> RS_NEW
    DEPLOY -- "scales down" --> RS_OLD
    RS_OLD --> PODS_OLD
    RS_NEW --> PODS_NEW
    PODS_NEW --> READY
    READY -. "Ready=True before<br/>next surge" .-> DEPLOY
    DEPLOY -. "no progress in deadline" .-> DEADLINE
    RS_OLD -. "kept around" .-> HISTORY

What this diagram shows. A user edits the Deployment’s spec.template (a new container image, an env var change, anything inside the Pod template). The Deployment controller hashes the new template, finds no matching existing ReplicaSet, and creates a fresh one (rev-4). It then gradually increases the new ReplicaSet’s spec.replicas and decreases the old ReplicaSet’s spec.replicas according to two knobs — maxSurge (how many Pods above desired may exist transiently) and maxUnavailable (how many below desired may exist transiently). The pace is gated by readiness probes: a newly created Pod must report Ready=True before the controller scales the new ReplicaSet up another step. If no progress happens within progressDeadlineSeconds (default 600), a ProgressDeadlineExceeded condition appears on the Deployment status and kubectl rollout status exits non-zero. Old ReplicaSets stick around (count bounded by revisionHistoryLimit, default 10) so kubectl rollout undo can flip back to a prior revision by reversing the same ramp. The insight to extract is that every state in the picture is durable in etcd — Deployment, both ReplicaSets, all Pods — so the controller can crash and resume mid-rollout without losing position, which is the level-triggered guarantee from Kubernetes Control Loop Pattern applied to rollouts.

Mechanical Walk-through

What the Deployment controller actually watches and does

The Deployment controller lives in kube-controller-manager and runs the canonical reconcile loop (see Kubernetes Control Loop Pattern). It watches three resource types:

  1. Deployments — its primary resource. Any change to a Deployment’s spec enqueues that Deployment’s key for reconciliation.
  2. ReplicaSetsowned by Deployments via metadata.ownerReferences. Any change to a ReplicaSet status (replicas became Ready, etc.) enqueues the owning Deployment.
  3. Pods — indirectly, via the watch on owned ReplicaSets.

On reconcile, the controller computes:

  • The hash of spec.template. The current Pod template’s hash, used as the suffix on the managed ReplicaSet’s name (mydeploy-7c8f5b9d4 where 7c8f5b9d4 is the truncated hash). This is the canonical identity of a revision — two Deployments with identical Pod templates would map to ReplicaSets with identical hashes.
  • The set of existing ReplicaSets owned by this Deployment, by selector. One of them, possibly, has a matching template hash — that’s the “current” RS. The rest are “old” RSs.
  • The strategy’s intended next state. For RollingUpdate: how many new-RS Pods can we add right now (bounded by maxSurge), and how many old-RS Pods can we drop right now (bounded by maxUnavailable)?

The controller then issues Scale subresource patches against the relevant ReplicaSets — never directly creating or deleting Pods. ReplicaSet controllers handle Pod creation/deletion in their own reconcile loops. This split is the architectural reason Deployments are reliable: the Deployment controller is just shifting replica counts between objects; the ReplicaSet controllers do the actual Pod work. See ReplicaSet for that layer.

RollingUpdate strategy

The default strategy. Two knobs (kubernetes.io):

  • maxSurge (default 25%). The maximum number of Pods that can exist above the desired replica count during the rollout. May be an integer or a percentage of spec.replicas. Percentage values are rounded up.
  • maxUnavailable (default 25%). The maximum number of Pods that can be unavailable (not Ready, or being deleted) during the rollout. Integer or percentage, rounded down.

For a Deployment with replicas: 10 and the defaults, the controller targets at most 13 Pods total (10 + 25% surge, rounded up) and at least 8 ready Pods (10 − 25% unavailable, rounded down) at any moment of the rollout. Concretely the controller does this:

  1. Scale the new RS up by maxSurge (here, +3 → 3 new Pods).
  2. Wait for those Pods to become Ready (readiness probe passes; see Pod Probes).
  3. Scale the old RS down by maxSurge worth of replicas (−3 → 7 old Pods).
  4. Repeat until the new RS has 10 replicas and the old has 0.

If maxUnavailable: 0 is set, the controller is forbidden from killing any old Pod before a replacement new Pod is Ready — surge-only rollout, zero-downtime guarantee. If maxSurge: 0 is set, the controller is forbidden from adding any new Pod before an old one is killed — drain-first rollout, useful when total cluster capacity is tight. Both cannot be 0 simultaneously (the controller would deadlock; the API server rejects this configuration).

Recreate strategy

The alternative (kubernetes.io). On a template change:

  1. The controller scales the old RS to 0, waiting for all old Pods to terminate.
  2. Then it scales the new RS up to spec.replicas.

This is explicitly non-zero-downtime. It is the right choice when the workload cannot tolerate two versions running simultaneously — for example, a single-writer batch job that mounts a ReadWriteOnce volume (see Access Modes (PV)), or a legacy app whose old/new versions corrupt shared state if both run. Recreate is also the right choice for very fast iteration in development, where the downtime is preferable to managing parallel versions.

Readiness gates the rollout pace

maxSurge and maxUnavailable define the budget; readiness probes drive the pace. The Deployment controller treats a Pod as “available” only when:

  1. It has been Ready (readiness probe passing) for spec.minReadySeconds continuously (default 0). The minReadySeconds field adds extra waiting on top of readiness — useful when “ready” doesn’t mean “warmed up” (cache priming, JIT, connection pool fill).
  2. It is not in the process of being deleted.

This is why a misconfigured readiness probe is the most common reason a rollout stalls: if the new Pods never report Ready, the controller never proceeds with scaling down the old RS, and the rollout sits at the surge cap indefinitely. Cross-reference Liveness Probe Misuse and Kubernetes Liveness Readiness Startup Probes.

Progress deadline

spec.progressDeadlineSeconds (default 600, i.e., 10 minutes per kubernetes.io) bounds how long the controller will wait without observed progress before flagging the rollout as failed. “Progress” means any of: the Deployment creates a new RS, the new RS gains a Ready Pod, the old RS loses a Pod. If progressDeadlineSeconds elapses with none of these events, the Deployment’s .status.conditions array gains a Progressing condition with status: False and reason: ProgressDeadlineExceeded. kubectl rollout status then exits non-zero — which is what CI/CD pipelines (Argo CD, Flux, the Jenkins K8s plugin) consume to decide “this deploy failed.”

Critically: a ProgressDeadlineExceeded does not trigger an automatic rollback. The Deployment continues to try the rollout (it’s still level-triggered). The user/CI must explicitly invoke kubectl rollout undo or update the Deployment’s spec back to a working configuration. This is a frequent misconception — see Failure Modes below.

Revision history and rollback

Every distinct spec.template ever applied to the Deployment becomes a ReplicaSet kept around for rollback. The bound is spec.revisionHistoryLimit (default 10). Older RSs beyond the limit are deleted (their Pods, if any, are deleted by the RS controller’s cascading delete via owner references — see Owner References and Garbage Collection).

kubectl rollout history deployment/<name> lists the retained revisions. kubectl rollout undo deployment/<name> rolls back to the immediately preceding revision; --to-revision=N targets a specific one. Mechanically, “undo” works by setting the Deployment’s spec.template back to the template of the chosen prior RS — which triggers a fresh rollout, in the opposite direction, using the same strategy. There is no special “rollback mode”; it’s just another rollout, which means it is also subject to readiness gating and progress deadlines.

Pause and resume

kubectl rollout pause deployment/<name> sets .spec.paused: true. The controller continues to reconcile but does not progress the rollout — it simply maintains the current mid-rollout split between old and new ReplicaSets. This is the canonical way to make several changes to a Deployment (image bump + env var change + resource request change) without triggering three separate rollouts. After all edits, kubectl rollout resume clears the flag and the rollout proceeds.

Two non-obvious points about pause:

  1. A paused rollout is not equivalent to a rolled-back rollout. Both paused and unpaused Deployments still hold the new template; the difference is only whether new RS scaling proceeds. Aborting a problematic rollout requires kubectl rollout undo, not kubectl rollout pause.
  2. Scaling commands still work while paused. kubectl scale deployment changes spec.replicas and the controller honors it. It scales the current RS (whichever one has the largest replica count at the time of pause), which can be subtle if you paused mid-rollout with both old and new RSs holding Pods.

Configuration / API Surface

A complete Deployment with line-by-line commentary:

apiVersion: apps/v1                            # the stable Deployment API group/version
kind: Deployment
metadata:
  name: web                                    # Deployment name; managed RSs are name-<template-hash>
  namespace: shop
  labels:
    app: web
spec:
  replicas: 6                                  # desired Pod count; HPA may override this
  revisionHistoryLimit: 10                     # keep 10 old ReplicaSets for rollback (default 10)
  progressDeadlineSeconds: 600                 # 10 min; triggers ProgressDeadlineExceeded if no progress
  minReadySeconds: 30                          # Pod must be Ready for 30s before counted available
  paused: false                                # default; set true to halt rollout mid-flight
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%                            # up to +25% Pods above replicas during rollout
      maxUnavailable: 25%                      # up to 25% Pods unavailable during rollout
  selector:
    matchLabels:
      app: web                                 # MUST match template.metadata.labels;
                                               # MUST be immutable post-create (a tightness K8s enforces)
  template:
    metadata:
      labels:
        app: web                               # at minimum, the selector's labels
    spec:
      terminationGracePeriodSeconds: 30        # SIGTERM → wait → SIGKILL
      containers:
        - name: web
          image: registry.example.com/web:v2.7.1
          ports:
            - containerPort: 8080
              name: http
          readinessProbe:                      # gates rollout pace — most important probe for Deployments
            httpGet:
              path: /healthz
              port: http
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:                       # restart unhealthy containers; NOT involved in rollout pace
            httpGet:
              path: /healthz
              port: http
            initialDelaySeconds: 30
            periodSeconds: 30
          resources:
            requests: { cpu: "100m", memory: "128Mi" }
            limits:   { cpu: "500m", memory: "256Mi" }

Notable surface points:

  • spec.selector is immutable. Once the Deployment is created with selector: {matchLabels: {app: web}}, you cannot change those labels. Trying to do so returns a selector does not match template labels 422. This is because the Deployment uses the selector to find its owned ReplicaSets — mutating it mid-life would orphan or misadopt RSs.
  • spec.template is the only mutable cause of rollout. Changing spec.replicas triggers a scale, not a rollout. Changing spec.strategy changes how future rollouts work but doesn’t trigger one. Only changing something inside spec.template (image, env, command, resources, labels-on-template, etc.) creates a new RS and starts a rollout.
  • The pod-template-hash label. The Deployment controller injects a pod-template-hash=<value> label into each ReplicaSet’s selector and Pod template. This is what lets selector: {matchLabels: {app: web}} work even though three different ReplicaSets all have Pods with app=web — they additionally have distinct pod-template-hash values that the controller uses for disambiguation.

The relevant kubectl rollout subcommands:

kubectl rollout status   deployment/web              # blocks until rollout finishes; non-zero if it fails
kubectl rollout history  deployment/web              # list retained revisions
kubectl rollout history  deployment/web --revision=3 # show the spec of revision 3
kubectl rollout undo     deployment/web              # roll back to previous revision
kubectl rollout undo     deployment/web --to-revision=2
kubectl rollout pause    deployment/web              # halt mid-rollout
kubectl rollout resume   deployment/web              # continue after pause
kubectl rollout restart  deployment/web              # force a rollout with no template change
                                                      # (annotates template with kubectl.kubernetes.io/restartedAt)

kubectl rollout restart is worth special mention: it works by adding/updating a kubectl.kubernetes.io/restartedAt annotation on the Pod template, which is a template change — therefore a new template hash, therefore a new RS, therefore a fresh rollout. This is the standard way to force every Pod to be replaced (e.g., to pick up a new ConfigMap that the app reads on startup) without having to actually change the spec.

Failure Modes

  1. Stuck rollout, all new Pods unready. The new RS scales up to maxSurge Pods, but none of them ever reach Ready. The rollout sits there, eventually hitting ProgressDeadlineExceeded. Almost always a misconfigured readiness probe (wrong path, wrong port, port unbound), a startup bug in the new image (crashloop), or a missing dependency (DB unreachable, missing Secret). Diagnose with kubectl describe pod, kubectl logs, kubectl describe deployment for the conditions list.

  2. ProgressDeadlineExceeded does not auto-rollback. Many users assume a failed rollout reverts. It does not. The Deployment will keep retrying forever (level-triggered: the new RS still exists in spec). The user/CI must kubectl rollout undo or fix-forward. Tools like Argo Rollouts add automatic rollback on top of native Deployments; stock Deployments don’t have it.

  3. maxUnavailable=0 with a tight node budget. Setting maxUnavailable: 0 requires the cluster to have headroom for an extra maxSurge worth of Pods simultaneously. On a cluster sized exactly to the desired replica count, the new Pods can’t be scheduled, the new RS stays Pending, and the rollout deadlocks. Diagnose via kubectl get pods showing Pending status and kubectl describe pod showing FailedScheduling. The fix is either to lower maxSurge to a value the cluster can absorb, or to scale the cluster (see Cluster Autoscaler, Karpenter).

  4. Concurrent edits during pause. A paused Deployment that’s mid-rollout with the old RS at 4 Pods and the new RS at 2 Pods has 6 Pods total of two different versions. A user running kubectl scale deployment web --replicas=10 while paused will see the controller scale only one of the two RSs (typically the larger one) to 10, leaving the other at 2 — for a total of 12. Confusing if not expected. The remedy is to resume before scaling, or scale and let the resume normalize.

  5. Hash collisions on the pod-template-hash. Theoretically possible but practically vanishing: the controller uses FNV-1a → base32 → 10-char suffix, with collision avoidance via .status.collisionCount (incremented on collision; mixed into the hash on retry). If you ever see this in the wild, it’s almost certainly a bug — file an issue.

  6. Selector and template labels drift. Editing a Deployment to change spec.selector.matchLabels is rejected. But editing spec.template.metadata.labels to no longer contain the selector’s labels is silently accepted — and immediately fails because the new RS Pods don’t match the selector. Symptom: kubectl get rs shows the new RS with 0 ready Pods despite Pods existing. Fix: keep selector labels and template labels in sync.

  7. Rollout works in dev, fails in prod due to PodDisruptionBudget conflicts. During an automated rollout, the controller respects Pod Disruption Budget only for voluntary evictions during node drains — not for rolling updates. But if your PDB and maxUnavailable settings are inconsistent (PDB allows 0 disruptions, Deployment’s maxUnavailable: 50%), you’ll see node drains block forever during cluster upgrades while Deployment rollouts proceed normally. Diagnose: compare PDB minAvailable/maxUnavailable to Deployment strategy.

  8. revisionHistoryLimit: 0 disables undo. Setting revisionHistoryLimit: 0 causes old RSs to be deleted immediately. kubectl rollout undo fails with no rollout history found. Set to ≥1 if rollback is desired (default 10 is fine for almost everyone).

  9. spec.strategy.type change mid-rollout. Switching from RollingUpdate to Recreate while a RollingUpdate is in progress doesn’t terminate the in-flight rollout cleanly — the new strategy applies on the next template change, but the current rollout finishes under the old rules. Surprising but documented.

  10. kubectl rollout restart doesn’t restart the cluster autoscaler’s view. Forcing a Pod replacement via rollout restart adds a template annotation, creating a new RS with a new pod-template-hash. If you have an HPA in place tuned to the old RS’s metrics endpoint, you may briefly see scale events from the HPA reacting to the rolling transition. Usually self-correcting; occasionally surprising.

Alternatives and When to Choose Them

  • StatefulSet — choose when Pods need stable identities and stable storage. Databases, leader-elected clusters, Kafka brokers, etcd. Deployments are explicitly the wrong tool for these: their Pods are interchangeable and ephemeral, with no identity guarantees.
  • DaemonSet — choose when the workload is per-node (log shipper, monitoring agent, CNI plugin). Deployment doesn’t model node-locality; DaemonSet does.
  • Job / CronJob — choose when the workload is run-to-completion rather than long-running. Deployments restart their Pods on exit (the default restartPolicy: Always is enforced in the template); Jobs let Pods exit successfully.
  • Argo Rollouts — choose when you want advanced progressive delivery (canary with metric-driven analysis, automatic rollback, blue-green with traffic-shifting Service swap). Replaces Deployment with the Rollout CRD. Worth it once you’re past simple rolling updates.
  • Knative Serving — choose for request-driven autoscale-to-zero stateless services. Builds on top of Deployments but adds revision routing and KPA scaling. Heavier; use only if scale-to-zero is essential.
  • Manual ReplicaSet + Service swap (Blue/Green) — choose if you want atomic cutover semantics: spin up a second ReplicaSet at full capacity, then swap a Service’s selector to point at it. Deployments don’t support this natively (their cutover is gradual); Argo Rollouts or hand-rolled YAML are the usual approach. See Blue-Green Deployment on Kubernetes.

Production Notes

  • Spotify’s Backstage uses Helm-managed Deployments for almost all stateless plugins, with HPA on top. The standard pattern: Deployment + Service + HPA + PodDisruptionBudget, generated by a Helm chart per microservice. Spotify’s published “Golden Path” docs treat Deployment as the only default workload type developers should reach for, with platform team approval required for StatefulSet/DaemonSet.
  • Shopify’s chargeback platform uses Deployments for everything except their MySQL/Vitess cluster (StatefulSet) and node-level monitoring (DaemonSet). Their internal SLO is “no Deployment rollout should take longer than 10 minutes” — they tune progressDeadlineSeconds to enforce this.
  • k8s.af incidents involving Deployments (Kubernetes Failure Stories) cluster around: (a) misconfigured readiness probes causing rollouts to stall and downstream services to keep hitting old replicas while clients lose their connections; (b) revisionHistoryLimit set too high (e.g., 100), accumulating thousands of empty ReplicaSets that bloat etcd and the API server’s watch cache; (c) maxSurge: 0 + maxUnavailable: 100% (“kill everything first then ramp up”) used by accident, causing total service outages.
  • The HPA + Deployment combo. A Deployment’s spec.replicas is also the target of the Horizontal Pod Autoscaler’s scale subresource. When an HPA is attached, the HPA owns replicas and the Deployment owns the template. The user should not edit replicas directly when an HPA is in place — it’ll just be overwritten on the next HPA reconcile.
  • GitOps with Deployments. ArgoCD and Flux reconcile Deployments by applying their full spec from Git. A surprising consequence: if you kubectl edit a Deployment’s replicas directly while ArgoCD is managing it, ArgoCD will flip it back. The remedy is either a helm.sh/resource-policy: keep annotation, the ignoreDifferences ArgoCD field, or — better — letting the HPA manage replicas and pinning everything else in Git.

See Also