Node Pressure Conditions

Node pressure conditions are the MemoryPressure, DiskPressure, and PIDPressure booleans that the kubelet writes onto its own Node object’s .status.conditions array when a node runs short of a resource (Kubernetes — Node-pressure Eviction). Each condition is derived from a set of eviction signals — measured statistics like memory.available or nodefs.inodesFree — compared against operator-configured thresholds. A condition becoming True has two consequences: the kube-scheduler stops placing new Pods on the node (the kubelet automatically applies a matching taint), and the kubelet itself begins reclaiming resources and, if that is not enough, evicting Pods in QoS order. Node pressure conditions are therefore the public, observable surface of the same machinery described mechanically in Pod Eviction — this note covers what the conditions are and how they are computed; the eviction-ordering algorithm lives in that sibling note.

Mental Model

The kubelet runs an eviction manager on a ticker (default housekeeping interval ~10 s). On each tick it gathers eviction signals from cgroup statistics, the cAdvisor-derived summary API, and the container runtime’s image-filesystem stats, then compares each signal against the configured thresholds. The mapping is fixed:

Node Condition	Triggering eviction signal(s)
MemoryPressure	memory.available
DiskPressure	nodefs.available, nodefs.inodesFree, imagefs.available, imagefs.inodesFree (and the newer containerfs.* split)
PIDPressure	pid.available

Note the asymmetry: one DiskPressure condition covers four (or six) disk signals. A node can be under disk pressure because its root/nodefs filesystem is low on bytes or inodes, or because the dedicated image filesystem (imagefs, used by the runtime to store pulled image layers and writable container layers) is low — all of them flip the single DiskPressure bit. Memory and PID each have exactly one signal.

flowchart TD
    subgraph kubelet["kubelet — eviction manager (ticks ~10s)"]
        SIG["Collect eviction signals<br/>memory.available · nodefs.available<br/>nodefs.inodesFree · imagefs.available<br/>imagefs.inodesFree · pid.available"]
        CMP{"signal &lt; threshold?"}
        SOFT["soft threshold crossed<br/>→ wait eviction-soft-grace-period"]
        HARD["hard threshold crossed<br/>→ act immediately (0s grace)"]
        COND["Set Node condition True<br/>MemoryPressure / DiskPressure / PIDPressure"]
        TAINT["Apply taint<br/>node.kubernetes.io/&lt;x&gt;-pressure"]
        RECLAIM["Reclaim node-level resources<br/>(dead containers, unused images, logs)"]
        EVICT["Still over threshold?<br/>→ evict Pods in QoS order"]
    end
    SCHED["kube-scheduler"]
    SIG --> CMP
    CMP -->|soft| SOFT --> COND
    CMP -->|hard| HARD --> COND
    COND --> TAINT
    TAINT -.->|"NoSchedule: no new Pods land here"| SCHED
    COND --> RECLAIM --> EVICT

The diagram shows the single decision loop. The insight to extract: a node condition is not just a status flag — it is simultaneously a signal to the scheduler (via the auto-applied taint) and a trigger for the kubelet’s own reclaim/evict actions. The condition couples the cluster-wide placement decision and the node-local survival decision through one boolean.

Mechanical Walk-through

The eviction signals

The signals the kubelet monitors (Node-pressure Eviction):

• memory.available — node allocatable memory minus the working set of all Pods. Computed as node.status.capacity[memory] - node.stats.memory.workingSet. Working set, not RSS — it excludes reclaimable page cache, which is why a node can show “low free memory” in free -m while memory.available is still healthy.
• nodefs.available / nodefs.inodesFree — free bytes and free inodes on the node filesystem: the volume the kubelet uses for local storage (emptyDir volumes, the kubelet’s working directory, and — if there is no separate imagefs — also container writable layers).
• imagefs.available / imagefs.inodesFree — free bytes/inodes on the image filesystem, an optional separate volume the container runtime uses for image layers and writable container layers. If the runtime does not use a separate filesystem, imagefs equals nodefs.
• containerfs.available / containerfs.inodesFree — a more recent split distinguishing the writable-container-layer filesystem from the image-layer filesystem; relevant when the runtime stores them separately.
• pid.available — free process IDs: node.stats.rlimit.maxpid - node.stats.rlimit.curproc. PID exhaustion is rare but catastrophic — once a node cannot fork(), nothing new can start, including recovery tooling.

Soft vs hard thresholds

The kubelet supports two threshold sets per signal, configured via flags or the KubeletConfiguration file:

• Hard thresholds (--eviction-hard / evictionHard) — when crossed, the kubelet evicts Pods immediately with a zero-second grace period. There is no eviction-soft-grace-period, no respect for the Pod’s terminationGracePeriodSeconds, and no respect for Pod Disruption Budget. This is the “node is about to fall over” emergency brake. Defaults: memory.available<100Mi, nodefs.available<10%, nodefs.inodesFree<5%, imagefs.available<15%, imagefs.inodesFree<5%.
• Soft thresholds (--eviction-soft / evictionSoft) — when crossed, the kubelet does not act until the signal stays below the threshold for the entire associated --eviction-soft-grace-period. A soft eviction also honours --eviction-max-pod-grace-period (a cap on how long termination may take). Soft thresholds have no defaults — they exist only if you configure them. They give a buffer for transient spikes and let Pods exit gracefully.

A soft threshold should be set more conservatively (a higher available floor) than the hard threshold for the same signal, so the soft alarm fires first and gives the grace period a chance to resolve the pressure before the hard threshold’s emergency eviction kicks in.

The automatic taint and the scheduler

When a pressure condition becomes True, the kubelet applies a corresponding taint to the node — node.kubernetes.io/memory-pressure, node.kubernetes.io/disk-pressure, or node.kubernetes.io/pid-pressure (see Taints and Tolerations). The default scheduler adds a NoSchedule toleration only for BestEffort Pods against memory-pressure (so a memory-starved node can still receive Pods that would not make memory worse), but for disk-pressure and pid-pressure the taint stops all new Pods. This is the mechanism by which a pressured node is automatically removed from the scheduler’s candidate set — the kubelet does not message the scheduler directly; it just taints itself and the scheduler’s existing TaintToleration filter does the rest.

Reclaim before evict

Before evicting any Pod, the kubelet first tries to reclaim node-level resources — resources not attributed to any Pod:

• For imagefs / disk pressure: garbage-collect unused container images (subject to imageMinimumGCAge, imageGCHighThresholdPercent, imageGCLowThresholdPercent) and remove dead containers.
• For nodefs pressure: delete dead Pods and their containers, and (if imagefs is not separate) also remove unused images.

Only if reclaiming node-level resources fails to bring the signal back above the threshold does the kubelet proceed to evict running Pods, in the QoS-driven order detailed in Pod Eviction: BestEffort first, then Burstable Pods exceeding their requests, then Guaranteed Pods only as a last resort. There is no node-level reclaim available for memory or PID pressure — the kubelet goes straight from condition to eviction.

Flapping prevention — the transition period

A node whose signal oscillates around a threshold would otherwise flip its condition (and taint) True/False every tick, causing the scheduler to repeatedly add and remove the node from its candidate set. The flag --eviction-pressure-transition-period (default 5m) suppresses this: once a pressure condition is set, the kubelet will not clear it back to False until the signal has stayed above the threshold for the entire transition period. The cost of this stability is that a node which has genuinely recovered still rejects new Pods for up to five minutes after recovery — a deliberate trade of placement latency for control-loop stability.

Configuration / API Surface

Eviction is configured in the KubeletConfiguration object (the modern path; the equivalent --eviction-* flags are deprecated but still work):

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
 
# HARD thresholds — immediate eviction, 0s grace. Values shown are the defaults.
evictionHard:
  memory.available:  "100Mi"   # evict immediately when working-set headroom < 100Mi
  nodefs.available:  "10%"     # < 10% free bytes on the node filesystem
  nodefs.inodesFree: "5%"      # < 5% free inodes — inode exhaustion is as fatal as byte exhaustion
  imagefs.available: "15%"     # < 15% free bytes on the image filesystem
  imagefs.inodesFree: "5%"     # < 5% free inodes on the image filesystem
 
# SOFT thresholds — no defaults; act only after the grace period elapses.
evictionSoft:
  memory.available:  "300Mi"   # alarm earlier than the 100Mi hard floor
  nodefs.available:  "15%"
evictionSoftGracePeriod:
  memory.available:  "1m30s"   # signal must stay below 300Mi for 90s before eviction
  nodefs.available:  "2m"
 
evictionMaxPodGracePeriod: 60  # cap on per-Pod termination time during a SOFT eviction
evictionPressureTransitionPeriod: 5m   # min time a condition must be False-able before clearing
mergeDefaultEvictionSettings: true     # keep built-in hard defaults when adding custom thresholds
 
# Resources carved out for the OS and kubelet/runtime — NOT visible to Pods.
# Reserving headroom is the primary way to keep memory.available from ever reaching 100Mi.
systemReserved:
  cpu: "500m"
  memory: "1Gi"
kubeReserved:
  cpu: "500m"
  memory: "1Gi"

Line-by-line. evictionHard lists the five default signals; overriding any of them without mergeDefaultEvictionSettings: true silently drops the others — a classic foot-gun that leaves a node with, say, no inode protection. evictionSoft plus evictionSoftGracePeriod together define the grace-buffered alarms; the grace period is keyed per signal. evictionMaxPodGracePeriod caps termination time on soft evictions so a Pod with a 30-minute terminationGracePeriodSeconds cannot stall reclaim. evictionPressureTransitionPeriod is the flapping damper. systemReserved/kubeReserved shrink the node’s allocatable capacity so that the OS and kubelet always have memory — this is upstream of eviction: good reservation values mean the eviction thresholds are rarely reached at all.

Inspect conditions and taints directly:

kubectl describe node ip-10-0-3-7 | grep -A6 Conditions
#   MemoryPressure   False   ... kubelet has sufficient memory available
#   DiskPressure     False   ... kubelet has no disk pressure
#   PIDPressure      False   ... kubelet has sufficient PID available
#   Ready            True    ... kubelet is posting ready status
 
kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.taints}{"\n"}{end}'
# a tainted node shows node.kubernetes.io/disk-pressure:NoSchedule

Failure Modes

Inode exhaustion with bytes to spare. A node with 60% free disk but a workload that creates millions of tiny files (a logging bug, a runaway cache) crosses nodefs.inodesFree<5%. df -h looks fine; df -i tells the truth. DiskPressure flips True and Pods are evicted on a node that appears, by the usual metric, healthy. Diagnose with df -i on the node.

imagefs filling from image churn. A node that pulls many large images (frequent deploys, many distinct images) fills imagefs. The kubelet’s first reclaim action — garbage-collecting unused images — fixes this, but only down to imageGCLowThresholdPercent; if live Pods reference enough images to keep usage above the high threshold, the kubelet cannot reclaim and proceeds to evict. Symptom: DiskPressure plus ImageGCFailed events.

Eviction storm walking the cluster. A BestEffort Pod with no memory limit balloons, the node hits memory.available<100Mi, the kubelet evicts it — and the controller immediately reschedules it onto another node, which then tips into pressure too. The storm walks across the fleet. Root cause is missing requests/limits collapsing QoS Classes; the fix is realistic requests plus LimitRange defaults. Documented in Pod Eviction and kubelet production notes.

Condition stuck True after recovery. Operators see DiskPressure: True on a node whose disk is clearly fine again and panic. Often it is just the evictionPressureTransitionPeriod (5 m) not yet elapsed — the condition is correctly lagging recovery. Wait the transition period before treating it as a real fault.

Memory pressure not firing despite OOM kills. memory.available uses working set, and the kernel’s per-cgroup OOM killer can kill a container that breaches its own limit before the node-level signal crosses its threshold. So a container can be OOM-killed (reason: OOMKilled) with the node never showing MemoryPressure — node pressure and per-container OOM are two distinct mechanisms. See cgroups Integration.

Alternatives and When to Choose Them

• Node-pressure eviction vs API-initiated eviction. Node-pressure eviction (this note) is involuntary, kubelet-driven, and ignores Pod Disruption Budget. API-initiated eviction (the Eviction API behind kubectl drain) is voluntary, respects PDBs and graceful termination, and is used for node maintenance. Do not conflate them: a PDB protects you from drains, not from a node running out of memory.
• Node-pressure eviction vs scheduler preemption. Pod Priority and Preemption evicts Pods to make room for a higher-priority pending Pod — a placement decision made by the scheduler. Node-pressure eviction evicts Pods to keep a node alive — a survival decision made by the kubelet. Different actor, different trigger.
• Eviction vs resource reservation. systemReserved/kubeReserved (plus --enforce-node-allocatable) are preventive: shrink allocatable so the node never reaches the eviction threshold. Eviction is reactive. A well-run cluster relies mostly on reservation and treats eviction as a rare backstop.

Production Notes

• Always reserve. Production nodes should set systemReserved and kubeReserved (commonly ~1 CPU / 1–2 Gi memory on mid-size nodes). Without reservation, Pods can be scheduled to use 100% of node memory and the kubelet itself gets starved — at which point PLEG stalls and the node goes NotReady, a worse outcome than a clean eviction. Spotify and Shopify post-mortems both trace eviction storms to zero reservation plus zero Pod requests.
• Monitor the signals, not just the conditions. By the time MemoryPressure is True, eviction is imminent. Alert on the underlying kubelet_* and node-exporter metrics (node_memory_MemAvailable_bytes, node_filesystem_avail_bytes, node_filesystem_files_free) trending toward the thresholds, not on the boolean condition.
• Set soft thresholds. Stock clusters ship only hard thresholds, so the first sign of pressure is an immediate zero-grace eviction. Adding soft thresholds with a 1–2 minute grace period converts many would-be hard evictions into graceful terminations and gives alerting a head start.
• Separate imagefs. Putting image layers on a dedicated filesystem isolates image churn from nodefs, so a deploy-storm of large images cannot evict Pods by filling the same volume their emptyDir volumes live on.
• PID limits. PID exhaustion is rare but lethal and recovery-hostile. Set podPidsLimit on multi-tenant nodes so one fork-bombing container cannot reach pid.available for the whole node.

See Also

• Pod Eviction — the eviction algorithm and full QoS-ordering rules; the mechanical sibling of this note
• kubelet — the agent that runs the eviction manager and writes these conditions
• Kubernetes Node — the Node resource whose .status.conditions array these populate
• QoS Classes — Guaranteed / Burstable / BestEffort; the eviction-priority classification
• Taints and Tolerations — the node.kubernetes.io/*-pressure taints the kubelet auto-applies
• Resource Requests and Limits — requests/limits drive QoS, which drives eviction order
• cgroups Integration — per-container OOM kill, distinct from node-pressure eviction
• Pod Priority and Preemption — scheduler-driven eviction, contrasted above
• Pod Disruption Budget — voluntary-disruption protection that node-pressure eviction ignores
• Kubernetes MOC — parent MOC (§11 Multi-tenancy and Resource Management)