Topology Spread Constraints

Topology spread constraints declare, on the Pod, how evenly a set of Pods should be distributed across a topology domain — zones, nodes, racks, any node-label dimension. Where Pod Affinity and Anti-Affinity expresses binary “co-locate” or “never co-locate” rules, spec.topologySpreadConstraints expresses a graded skew tolerance: “no domain may hold more than maxSkew Pods above the least-loaded domain.” It is the modern, scalable replacement for using podAntiAffinity to spread replicas — anti-affinity for spreading is O(Pods²) for the scheduler to evaluate and can only express “at most one per domain,” while topology spread is cheaper and lets you say “evenly, ±2.” The feature went alpha in v1.16, beta in v1.18, and GA in v1.19 (Introducing PodTopologySpread, Topology Spread Constraints). The scheduler enforces it via the PodTopologySpread Filter + Score plugin.

Mental Model

The scheduler counts, for the matching Pod set, how many Pods sit in each topology domain (each distinct value of topologyKey), finds the global minimum count, and rejects (or penalizes) any node whose domain would exceed minimum + maxSkew once this Pod lands.

flowchart TB
    subgraph BEFORE["Before scheduling Pod X — labelSelector: app=web"]
        ZA["zone-a: 3 web Pods"]
        ZB["zone-b: 2 web Pods"]
        ZC["zone-c: 1 web Pod  ← global minimum"]
    end
    X["Pod X (app=web)<br/>maxSkew: 1, topologyKey: zone<br/>whenUnsatisfiable: DoNotSchedule"]
    X --> CHECK{"Place in which zone?"}
    CHECK -->|"zone-a → 4: skew 4-1=3 > 1"| RA["REJECTED by Filter"]
    CHECK -->|"zone-b → 3: skew 3-1=2 > 1"| RB["REJECTED by Filter"]
    CHECK -->|"zone-c → 2: skew 2-1=1 ≤ 1"| OK["FEASIBLE — bind here"]

What this diagram shows. Three zones hold 3, 2, and 1 matching Pods; the global minimum is 1. With maxSkew: 1, the only zone where adding Pod X keeps the spread within tolerance is zone-c (which becomes 2 — a skew of 1 over the minimum). zone-a and zone-b are filtered out entirely because DoNotSchedule makes the constraint hard. The insight to extract: topology spread does not compute a “perfect” balanced placement up front — it is evaluated one Pod at a time during that Pod’s scheduling cycle, and the skew bound is checked relative to the cluster state at that instant. Spreading therefore emerges from each Pod individually being pushed toward under-loaded domains, not from a global optimizer.

Mechanical Walk-through

The required fields

Every constraint in topologySpreadConstraints[] carries:

  • maxSkew (required, > 0) — the maximum permitted difference between the Pod count in any domain and the global minimum. With whenUnsatisfiable: DoNotSchedule it is a hard ceiling; with ScheduleAnyway it tells the Score plugin to prefer nodes that reduce skew.
  • topologyKey (required) — the node-label key that defines a domain. Each distinct value of that label is one domain. topology.kubernetes.io/zone spreads across availability zones; kubernetes.io/hostname spreads across individual nodes.
  • whenUnsatisfiable (required) — DoNotSchedule (hard: Filter rejects nodes that would violate maxSkew) or ScheduleAnyway (soft: Score penalty only — the Pod still schedules even if it worsens skew).
  • labelSelector — which Pods are counted per domain. Critically, this is the set the skew is computed over; it normally matches the Pod’s own labels so a Deployment’s replicas are spread relative to each other.

The optional, fine-grained fields

Later releases added precision knobs:

  • minDomains — the minimum number of domains the spread should assume exist. If fewer eligible domains are present than minDomains, the global minimum is treated as 0, which forces the scheduler to keep filling new domains rather than packing into the few it sees. Only valid with DoNotSchedule. Introduced alpha in v1.24 (KEP-3022), promoted to beta in v1.27 under the MinDomainsInPodTopologySpread feature gate (default-on since v1.28), and GA since v1.30 (Fine-grained Pod topology spread features to beta, Topology Spread Constraints).
  • nodeAffinityPolicyHonor or Ignore: whether the Pod’s own nodeAffinity/nodeSelector should restrict which domains count as eligible when computing skew. Alpha in v1.25 (KEP-3094), beta in v1.27 (gate NodeInclusionPolicyInPodTopologySpread, default-on), GA in v1.33. (The default when the field is unset is Honor — node affinity is respected.)
  • nodeTaintsPolicyHonor or Ignore: whether nodes carrying taints the Pod does not tolerate are excluded from the eligible-domain set. Same KEP-3094 lifecycle as nodeAffinityPolicy: alpha v1.25, beta v1.27, GA in v1.33 (shared gate NodeInclusionPolicyInPodTopologySpread). The default when unset is Ignore, so you must opt in to Honor to get the behavior below. Setting this to Honor prevents the classic bug where the scheduler counts a tainted, unreachable zone as a valid spread target.
  • matchLabelKeys — a list of Pod label keys whose values are looked up from the incoming Pod and merged into labelSelector. The canonical use is a controller-managed revision label (e.g., pod-template-hash): it lets each Deployment rollout spread independently of the prior rollout’s still-terminating Pods. Introduced for topology spread under KEP-3243, it reached beta and default-on in v1.27 (gate MatchLabelKeysInPodTopologySpread) and remains beta as of the v1.34 docs (Respect Pod topology spread after rolling upgrades). A behavioral subtlety: before v1.34 the merge was implicit (the scheduler did it); since v1.34 kube-apiserver explicitly merges the matchLabelKeys values into labelSelector at admission time — gated by MatchLabelKeysInPodTopologySpreadSelectorMerge, which can be disabled to revert to the pre-v1.34 behavior (Topology Spread Constraints). The identically-named field on Pod affinity graduated on its own separate timeline — do not assume one’s status from the other.

A consistency rule worth knowing

You may declare at most one constraint per unique (topologyKey, whenUnsatisfiable) pair. You can have two constraints with the same topologyKey if they differ in whenUnsatisfiable (one hard, one soft) — and multiple constraints with different topology keys are common (spread across zones and across nodes). All constraints are ANDed: a node must satisfy every one of them.

Cluster-level default constraints

Most workload authors never write a topologySpreadConstraints block. The scheduler config (KubeSchedulerConfiguration) lets a cluster operator set default constraints in the PodTopologySpread plugin’s pluginConfig, applied to any Pod that does not declare its own:

pluginConfig:
- name: PodTopologySpread
  args:
    defaultConstraints:
    - maxSkew: 3
      topologyKey: "topology.kubernetes.io/zone"
      whenUnsatisfiable: ScheduleAnyway
    defaultingType: List   # use the explicit list above

defaultingType: System instead uses Kubernetes’ built-in default (a soft spread across zones and nodes). This is how a platform team enforces “everything spreads across zones” without touching a single application manifest.

Configuration / API Surface

A Deployment whose Pods spread hard across zones and soft across nodes:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 9
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      topologySpreadConstraints:
      - maxSkew: 1                                  # at most 1 Pod above the least-loaded zone
        topologyKey: topology.kubernetes.io/zone    # domain = availability zone
        whenUnsatisfiable: DoNotSchedule            # HARD — refuse to skew zones
        labelSelector:
          matchLabels:
            app: web                                # count peer replicas
        minDomains: 3                               # assume >= 3 zones; force filling all 3
        nodeTaintsPolicy: Honor                     # ignore tainted/unreachable zones
        matchLabelKeys:
        - pod-template-hash                         # spread THIS rollout independently
      - maxSkew: 2                                  # nodes may differ by up to 2
        topologyKey: kubernetes.io/hostname         # domain = individual node
        whenUnsatisfiable: ScheduleAnyway           # SOFT — prefer, don't require
        labelSelector:
          matchLabels:
            app: web
      containers:
      - name: web
        image: web:v5

Line-by-line. First constraint is the HA-critical one: maxSkew: 1 over zone with DoNotSchedule forces 9 replicas into a 3-3-3 zone layout; minDomains: 3 guarantees the scheduler does not pack everything into two zones if the third is briefly empty; nodeTaintsPolicy: Honor stops a cordoned zone from being counted as a valid spread target (without it, the scheduler might “spread” into a zone no node can actually accept the Pod); matchLabelKeys: [pod-template-hash] ensures a rolling update’s new Pods spread relative to each other, not contaminated by the old ReplicaSet’s terminating Pods. Second constraint softly discourages piling many replicas on one node — ScheduleAnyway means a node shortage will not block the rollout, it merely nudges. The two constraints are ANDed.

Failure Modes

Rollout wedged at DoNotSchedule. A hard maxSkew plus too few domains (e.g., a 2-zone cluster, maxSkew: 1, an odd replica count, all zones equally loaded) leaves no feasible node — the new replica is Pending forever. Symptom: FailedScheduling … didn't match pod topology spread constraints. Fix: raise maxSkew, switch to ScheduleAnyway, or add capacity.

“Spreading” into a dead zone. Without nodeTaintsPolicy: Honor, the scheduler counts a zone whose nodes are all tainted/NotReady toward the domain set; the global minimum drops and the scheduler refuses to place Pods in the healthy zones to “preserve” the dead one. This produces Pending Pods amid free capacity. Set nodeTaintsPolicy: Honor.

Old + new ReplicaSet skew during rollout. Without matchLabelKeys, a labelSelector: app=web counts both the old and new ReplicaSet’s Pods. Mid-rollout the counts are lopsided and the new Pods scatter oddly. matchLabelKeys: [pod-template-hash] scopes the count to the current revision.

Scheduler latency at scale. PodTopologySpread is, after NodeResourcesFit, the most expensive Filter+Score plugin — every decision iterates over peer Pods in the matching topology. Airbnb’s 2000+-node benchmarks identified it as the dominant cost. Mitigations: prefer ScheduleAnyway over DoNotSchedule, use coarse keys (zone, not host), and use minDomains sparingly.

Misunderstanding maxSkew as “max per domain.” maxSkew is the spread relative to the global minimum, not an absolute cap. With minimum 100 and maxSkew: 5, a domain may legally hold 105 Pods. To cap absolute counts, use a different mechanism (capacity, quota).

Alternatives and When to Choose Them

  • Pod Affinity and Anti-AffinitypodAntiAffinity with topologyKey was the original way to spread replicas. It can only express “at most one per domain” (or a soft preference), is O(Pods²) for the scheduler, and degrades badly past a few hundred Pods. Topology spread is the modern replacement for the spreading use case; reserve anti-affinity for genuine “these two specific workloads must never share a node” rules. Use affinity (not spread) for co-location.
  • Node Affinity / Node Selector — constrain which nodes are candidates at all; orthogonal to spread, frequently combined (spread within the GPU pool).
  • Descheduler — topology spread is enforced only at scheduling time; it does not rebalance after a node failure repopulates a zone unevenly. The out-of-tree Descheduler has a RemovePodsViolatingTopologySpreadConstraint strategy that evicts skewed Pods so the scheduler re-places them.
  • Doing nothing — for a small Deployment on a single-zone cluster, spread constraints add no value. They earn their keep at HA scale across ≥3 zones.

Production Notes

  • Cluster-level defaults are the leverage point. Mature platform teams set defaultConstraints in the scheduler config so every workload spreads across zones by default — application teams opt out, not in. This is how organizations get fleet-wide zonal HA without auditing thousands of manifests.
  • Combine hard zone spread with soft node spread, exactly as in the example: a hard zone constraint protects against an entire-AZ outage (the expensive failure), while a soft node constraint merely discourages hot nodes (a cheap, recoverable inconvenience). Making the node constraint hard is the most common cause of wedged rollouts.
  • The Descheduler is the missing half. Topology spread keeps placement balanced as Pods are created; after a zone outage and recovery, the cluster can be badly skewed with no scheduler event to fix it. Pair topology spread with the Descheduler’s topology-spread strategy for true steady-state balance.
  • EKS/GKE/AKS all label nodes with topology.kubernetes.io/zone out of the box, so topologyKey: topology.kubernetes.io/zone works without extra labeling on every managed cluster.
  • StatefulSets benefit too — spreading database replicas across zones is a textbook use — but verify the whenUnsatisfiable setting does not deadlock the ordered StatefulSet rollout when a zone is short on capacity.

See Also