vault

Kubernetes Volume

17 min read

Kubernetes Volume

A Kubernetes Volume is the abstraction by which a Pod gives its containers access to named, mountable storage that has a defined lifetime relative to the Pod (kubernetes.io/docs/concepts/storage/volumes/). It exists to solve two distinct problems: (1) share data across containers in the same Pod (containers in a Pod don’t share filesystems by default — each one starts from its own image’s filesystem layers), and (2) survive container restarts within a Pod (when a container crashes and is restarted by kubelet, its container-filesystem changes are lost; a Volume mounted into it persists). A Pod declares Volumes in spec.volumes[] (each with a name and a type identifying where storage comes from) and each container declares volumeMounts[] indicating which named Volumes to mount where in its filesystem. Volumes split sharply into two lifetime classes: Pod-lifetime (emptyDir, configMap, secret, downwardAPI, projected, ephemeral, hostPath) — created with the Pod, destroyed with it — and persistent (via persistentVolumeClaim) where the underlying storage outlives the Pod and is reattached when the Pod is rescheduled. The Volume types themselves further split into the legacy in-tree volume plugins (compiled into the kubelet binary: awsElasticBlockStore, gcePersistentDisk, azureDisk, vsphereVolume, cephfs, rbd, nfs, etc.) — most of which have been migrated to or replaced by CSI drivers via the CSI Migration project and the in-tree drivers progressively removed from the Kubernetes codebase from v1.26 through v1.31 (kubernetes.io/blog — Kubernetes Removals in v1.31) — and the modern out-of-tree drivers invoked via the csi Volume type. This note covers Volume semantics, the in-Pod API surface, the Pod-lifetime types in depth, the bridge to persistent storage, and the state of the in-tree → CSI migration as of mid-2026. See PersistentVolume, PersistentVolumeClaim, and Container Storage Interface for the persistent-storage half of the story, and Projected Volume for the canonical token+config aggregation pattern.

Mental Model

flowchart TB
    subgraph POD["Pod (one network namespace, one IPC namespace, one set of Volumes)"]
        subgraph PSPEC["spec.volumes (declared once per Pod)"]
            VOL1[volumes:<br/>- name: config<br/>  configMap:<br/>    name: app-config]
            VOL2[volumes:<br/>- name: scratch<br/>  emptyDir: medium: Memory]
            VOL3[volumes:<br/>- name: data<br/>  persistentVolumeClaim:<br/>    claimName: app-data]
        end
        subgraph C1["Container A"]
            CA_MOUNT1[/etc/config -> 'config']
            CA_MOUNT2[/tmp/cache -> 'scratch']
            CA_MOUNT3[/var/data  -> 'data']
        end
        subgraph C2["Container B (init or sidecar)"]
            CB_MOUNT1[/etc/config -> 'config']
            CB_MOUNT2[/tmp/cache -> 'scratch']
        end
    end
    subgraph LIFE["Volume lifetime"]
        EPHEMERAL["Pod-lifetime:<br/>configMap, secret,<br/>downwardAPI, projected,<br/>emptyDir, ephemeral,<br/>hostPath"]
        PERSISTENT["Beyond Pod:<br/>persistentVolumeClaim<br/>(backed by PV / CSI driver)"]
    end
    VOL1 -. "deleted on Pod terminate" .- EPHEMERAL
    VOL2 -. "deleted on Pod terminate" .- EPHEMERAL
    VOL3 -. "PV remains, can rebind" .- PERSISTENT

What this diagram shows. A Pod with three Volumes declared at spec.volumes[]: a ConfigMap-backed config, an in-memory tmpfs scratch scratch, and a PVC-backed data. Both containers mount config at /etc/config (so they read the same ConfigMap files), both mount scratch at /tmp/cache (so they share a workspace), but only container A mounts the persistent data at /var/data. The Volume declarations live on the Pod spec; the mounts live on each container spec — so Volumes can be partially shared. Underneath, the lifetime is what determines whether the Volume survives the Pod: configMap/emptyDir/secret/downwardAPI/projected/ephemeral/hostPath are torn down when the Pod terminates; persistentVolumeClaim references a PersistentVolume managed independently by the cluster, so the underlying block device or network filesystem survives Pod termination and can rebind when the Pod is rescheduled. The insight to extract: Volumes are the join point between Pods and storage, but they’re declared at Pod scope, not container scope — every container in the Pod can mount the same Volume from a different path, which is the entire reason patterns like sidecar log-shippers and init-container setup-helpers exist.

Mechanical Walk-through

The API surface: spec.volumes[] and volumeMounts[]

Every Volume in a Pod is declared in spec.volumes[] with a name and a type-specific source:

spec:
  volumes:
    - name: <volume-name>                      # arbitrary, unique within the Pod
      <type>:                                  # exactly one of: configMap, secret,
        <type-specific fields>                 # emptyDir, hostPath, persistentVolumeClaim,
                                               # csi, projected, downwardAPI, nfs, ...

Each container then opts into individual Volumes via volumeMounts:

spec:
  containers:
    - name: app
      volumeMounts:
        - name: <volume-name>                  # must match a Pod-level volumes entry
          mountPath: /etc/app                  # path inside the container
          readOnly: true                       # optional, defaults to false
          subPath: app.conf                    # optional: mount a single file/subdir
          subPathExpr: $(POD_NAME)/logs        # optional: with env-var expansion

Crucial constraint: the Volume must be declared at the Pod level even if only one container uses it. There’s no per-container Volume declaration. This is what enables the “shared scratch space” pattern for sidecars and init containers.

Pod-lifetime Volume types

emptyDir — ephemeral scratch

The simplest Pod-lifetime Volume:

volumes:
  - name: scratch
    emptyDir: {}                               # empty object = disk-backed

Created when the Pod is assigned to a node, deleted when the Pod is removed. Survives container restarts within the Pod (so a crashing container’s scratch data survives the restart) but not Pod re-scheduling.

The medium field controls backing:

  • (default, omitted) — disk-backed, sized from node’s ephemeral storage budget.
  • Memory — tmpfs-backed; lives in RAM, counts against the Pod’s memory limit.
volumes:
  - name: ramscratch
    emptyDir:
      medium: Memory                           # tmpfs; super fast, counts against memory limit
      sizeLimit: 1Gi                           # kubelet evicts the Pod if exceeded

Use cases: temporary scratch for image-processing pipelines, sidecar-to-main shared workspace, checkpoint files for long computations, sidecar log shipping (main writes to emptyDir, sidecar reads and forwards).

configMap and secret

Inject ConfigMap or Secret data into the container as files:

volumes:
  - name: config
    configMap:
      name: app-config                         # ConfigMap to read
      items:                                   # optional: select specific keys
        - key: log_level
          path: log_level.conf                 # file path inside the mount
      defaultMode: 0644                        # file permissions

Each key in the ConfigMap becomes a file at <mountPath>/<key> (or the explicit path if specified). Updates to the ConfigMap propagate to the file after a short delay (kubelet’s sync period, ~60s) — unless subPath is used, in which case the file is a snapshot at Pod-create time.

Secrets work identically but with secret: and (optionally) per-file mode overrides and optional: true for “OK if missing.” Secrets are stored in tmpfs on the node to avoid hitting disk.

downwardAPI — Pod metadata as files

Expose Pod or container fields as files inside the container:

volumes:
  - name: podinfo
    downwardAPI:
      items:
        - path: namespace                      # file: /etc/podinfo/namespace
          fieldRef:
            fieldPath: metadata.namespace
        - path: pod_name
          fieldRef:
            fieldPath: metadata.name
        - path: cpu_limit                      # resource fields are supported
          resourceFieldRef:
            containerName: app
            resource: limits.cpu

Use case: feeding the Pod’s own identity (namespace, name, node, IP) and resource limits into the running process — useful for things like service discovery clients (“register with my Pod IP”) and JVM tuning (“set heap to 75% of my memory limit”).

projected — combine multiple sources into one mount

A projected Volume aggregates secrets, configMaps, downwardAPI, and ServiceAccount tokens into a single mount path:

volumes:
  - name: combined
    projected:
      sources:
        - configMap:
            name: app-config
        - secret:
            name: app-tls
        - downwardAPI:
            items:
              - path: namespace
                fieldRef: { fieldPath: metadata.namespace }
        - serviceAccountToken:
            path: token
            audience: vault                    # for workload-identity flows
            expirationSeconds: 3600

This is the modern way to assemble per-Pod credential bundles. Notably, the projected ServiceAccountToken source is the foundation of all modern Kubernetes ServiceAccount token usage — short-lived, audience-bound JWTs replacing the legacy long-lived auto-mounted Secret tokens. See Projected Volume for the deep dive.

ephemeral — PVC-style ephemeral storage

The newest Pod-lifetime Volume type (GA since 1.23). Lets a Pod claim a fresh PVC that lives only for the Pod’s lifetime:

volumes:
  - name: data
    ephemeral:
      volumeClaimTemplate:
        metadata:
          labels:
            type: pipeline-scratch
        spec:
          accessModes: ["ReadWriteOnce"]
          storageClassName: gp3
          resources:
            requests:
              storage: 100Gi

The kubelet creates a PVC named <pod-name>-<volume-name> at Pod-create, the storage class’s CSI driver dynamically provisions a PV, and both PVC and PV are deleted when the Pod is deleted. Use case: large ephemeral scratch where emptyDir would consume too much node disk; per-Pod build artifacts; CI runners.

hostPath — mount node filesystem into Pod

volumes:
  - name: docker-socket
    hostPath:
      path: /var/run/docker.sock               # path on the node
      type: Socket                             # or DirectoryOrCreate, FileOrCreate, etc.

Security-sensitive — mounting / from the host gives the Pod root-equivalent access to everything on the node. Use only for system-level Pods (CNI agents, log shippers, monitoring daemons) and forbid in user-facing namespaces (the Pod Security Admission Restricted profile blocks it). Cluster-aware schedulers should also be considered: a hostPath-using Pod that’s re-scheduled to a different node sees that node’s filesystem, which may be a different state.

Persistent Volumes — PVC as a Volume source

The bridge from Pod to durable storage:

volumes:
  - name: data
    persistentVolumeClaim:
      claimName: app-data                      # references a PVC in the Pod's namespace
      readOnly: false

The PVC (PersistentVolumeClaim) is a namespaced request for storage that binds to a cluster-scoped PersistentVolume. The Volume’s lifetime is the Pod’s, but the PV survives and can rebind when the Pod is rescheduled (especially with StatefulSet, where each replica’s PVC is sticky). Covered in depth in PersistentVolume, PersistentVolumeClaim, and StorageClass.

csi — the catch-all for modern storage

volumes:
  - name: encrypted-data
    csi:
      driver: secrets-store.csi.k8s.io
      readOnly: true
      volumeAttributes:
        secretProviderClass: app-vault-secrets

The csi Volume type embeds a CSI driver invocation directly into the Pod spec — no PVC needed. Most CSI usage goes through PVC + StorageClass + dynamic provisioning, but a few drivers (notably the Secrets Store CSI Driver and some FUSE-based file mounts) are designed for in-line csi: use because there’s no underlying durable PV to manage. See Container Storage Interface.

The in-tree volume plugin removal story

Historically, Kubernetes shipped a hard-coded set of “in-tree” volume plugins inside the kubelet binary itself: awsElasticBlockStore, azureDisk, azureFile, gcePersistentDisk, vsphereVolume, cinder (OpenStack), rbd (Ceph), cephfs, glusterfs, portworxVolume, and others. Each was tightly coupled to a specific storage backend, vendored into the Kubernetes release cycle, and impossible to update independently — every CVE in vSphere’s storage layer required a kubelet patch.

The CSI Migration project (KEP-625, github.com/kubernetes/enhancements) replaces these in-tree plugins with calls to out-of-tree CSI drivers while preserving the in-tree API shape for backward compatibility. The core CSI Migration feature reached GA in Kubernetes v1.25 (kubernetes.io/blog — In-Tree to CSI Migration Status Update).

After CSI Migration GA, the in-tree drivers themselves have been progressively removed from the Kubernetes codebase:

DriverDeprecatedRemoved
glusterfs1.251.26
cinder (OpenStack)1.111.26
awsElasticBlockStore1.191.27
azureDisk1.191.27
gcePersistentDisk1.171.28
vsphereVolume1.191.30
azureFile1.211.30
cephfs1.281.31
rbd (Ceph RBD)1.281.31
portworxVolume1.25not yet removed (CSI migration on by default since 1.31)

The vsphereVolume and azureFile removal versions were the previously uncertain rows; both are now confirmed against the kubernetes/website source: “The vsphereVolume in-tree storage driver was deprecated in the Kubernetes v1.19 release and then removed entirely in the v1.30 release,” and “The azureFile in-tree storage driver was deprecated in the Kubernetes v1.21 release and then removed entirely in the v1.30 release.” portworxVolume is the important correction: it was not removed in 1.31. It was deprecated in v1.25, but as of current Kubernetes (May 2026) the in-tree type is still present and “all operations for the in-tree Portworx volumes are redirected to the pxd.portworx.com Container Storage Interface (CSI) Driver by default” (Kubernetes — Volumes) — that is, migration-on-by-default, not removal. This is the canonical end-state of CSI migration before a driver is finally deleted from the tree.

What this means in practice: as of K8s 1.31 (released August 2024) and the versions since, the major cloud and Ceph in-tree volume types are gone from the kubelet. There are two distinct end-states to keep straight. For a type that has only reached migration-on-by-default but is still in the tree (e.g. portworxVolume), the API object still parses and a Pod referencing it works — the kubelet simply redirects the mount through the corresponding CSI driver transparently. For a type that has been removed (e.g. awsElasticBlockStore since 1.27, vsphereVolume/azureFile since 1.30, cephfs/rbd since 1.31), the in-tree handler no longer exists: on a cluster running at or past the removal version the kubelet cannot mount such a volume at all, and the corresponding CSI driver (e.g. ebs.csi.aws.com) must be installed and the workloads migrated to PVC-based access. Operators upgrading from older K8s versions must therefore ensure the corresponding CSI driver is installed before upgrading the cluster past the in-tree driver’s removal version — otherwise volumes that mounted fine yesterday fail to mount after the upgrade.

The CSIMigration feature gate, once a per-driver knob (CSIMigrationAWS, CSIMigrationGCE, etc.), is now largely a no-op since migration is GA — all the per-driver gates have been removed.

Volume mounting semantics: subPath, readOnly, and propagation

  • subPath: mount a single file or subdirectory from a Volume rather than the whole thing. Useful for mounting one ConfigMap key as /etc/nginx/nginx.conf without obliterating the rest of /etc/nginx. Critically: subPath mounts are snapshots — ConfigMap/Secret updates do not propagate to subPath mounts.
  • subPathExpr: like subPath but with environment-variable expansion ($(POD_NAME)/logs). Lets multiple Pod replicas write to distinct subdirectories of the same Volume.
  • readOnly: enforces read-only mount semantics. Belt-and-suspenders defense; some Volume types (configMap, secret, downwardAPI) are read-only at the API layer regardless.
  • mountPropagation: HostToContainer / Bidirectional / None. Controls whether mounts created inside the container appear on the host (Bidirectional) or in other containers (HostToContainer). Used by some CSI drivers; rarely needed in application code.

Configuration / API Surface

A representative Pod with five Volume types:

apiVersion: v1
kind: Pod
metadata:
  name: realistic-app
  namespace: payments
spec:
  serviceAccountName: payments-sa
  containers:
    - name: app
      image: registry.example.com/payments:v2
      env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
      volumeMounts:
        - name: config                        # ConfigMap-backed (R/O)
          mountPath: /etc/app
          readOnly: true
        - name: tls                           # Secret-backed (R/O)
          mountPath: /etc/tls
          readOnly: true
        - name: cache                         # In-memory tmpfs scratch
          mountPath: /var/cache/app
        - name: data                          # Persistent (PVC)
          mountPath: /var/data
          subPathExpr: $(POD_NAME)            # each replica writes to its own subdir
        - name: token-bundle                  # Projected: combine token + CA
          mountPath: /var/run/secrets/payments
          readOnly: true
  volumes:
    - name: config
      configMap:
        name: app-config
        defaultMode: 0644
    - name: tls
      secret:
        secretName: app-tls
        defaultMode: 0400                     # tighter perms for TLS keys
    - name: cache
      emptyDir:
        medium: Memory
        sizeLimit: 256Mi                      # tmpfs counts against memory limit
    - name: data
      persistentVolumeClaim:
        claimName: payments-data              # bound to a PV provisioned by gp3 CSI
    - name: token-bundle
      projected:
        sources:
          - serviceAccountToken:
              path: token
              audience: vault.example.com    # audience-bound JWT for workload identity
              expirationSeconds: 3600
          - configMap:
              name: vault-ca                  # ConfigMap holding the Vault CA cert
              items:
                - key: ca.crt
                  path: ca.crt

Observe inside the container after start:

$ ls /etc/app/                       # ConfigMap files
app.conf log_level.conf
 
$ ls -la /etc/tls/                   # Secret files; tmpfs-backed
total 4
-r-------- 1 root root  ... tls.crt
-r-------- 1 root root  ... tls.key
 
$ mount | grep -E "/var/cache|/var/data"
tmpfs on /var/cache/app type tmpfs (rw,relatime,size=262144k)
/dev/nvme1n1 on /var/data type ext4 (rw,relatime)                # PV-backed EBS gp3
 
$ ls /var/run/secrets/payments/      # Projected — multi-source
token ca.crt

Failure Modes

  1. subPath mount doesn’t update on ConfigMap change. A widely-encountered gotcha: changing the ConfigMap doesn’t propagate when subPath is used to mount a single key. Symptom: app keeps using stale config after kubectl edit configmap. Fix: mount the whole ConfigMap directory instead, or trigger a rolling restart of the Pods on config change (the standard pattern via Argo Rollouts or kubectl rollout restart).

  2. emptyDir: medium: Memory evicts the Pod. tmpfs usage counts against the Pod’s memory limit. Symptom: Pod OOMKilled when scratch usage grows. Fix: set sizeLimit to cap the tmpfs, or use disk-backed emptyDir.

  3. hostPath Pod sees different filesystem on rescheduling. When a hostPath-using Pod is re-scheduled to a different node, the mount path is the new node’s filesystem — typically empty or different. Fix: pin the Pod to a node via nodeAffinity; better, use local PV or proper PVC.

  4. Legacy awsElasticBlockStore Pod won’t start on K8s 1.27+. The in-tree driver was removed. Symptom: Pod stuck Pending with “no driver found.” Fix: install ebs-csi-driver, create a StorageClass referencing it, migrate to PVC-based access.

  5. Projected ServiceAccountToken stale. Bound tokens expire (default 1 hour); the kubelet refreshes them, but apps that read the token once at startup miss rotations. Fix: re-read the token periodically; SDK clients in Go, Python, Java do this automatically when using the standard K8s client libraries.

  6. csi Volume references a driver that’s not installed cluster-wide. Symptom: kubelet error “driver not registered.” Fix: deploy the CSI driver DaemonSet on all nodes (csi-node) plus the controller Deployment (csi-controller).

  7. Pod stuck in ContainerCreating with MountVolume.SetUp failed. Generic failure mode covering many causes: storage backend unreachable, PVC unbound, permissions wrong on hostPath, ConfigMap doesn’t exist. Diagnostic: kubectl describe pod reveals the underlying mount error; kubectl logs -n kube-system <csi-controller-pod> for CSI cases.

  8. subPath to a directory that doesn’t exist. Some Volume types create subPath dirs on demand, others don’t. configMap/secret/downwardAPI projections create the file; hostPath/PVC may or may not depending on type. Fix: pre-create the subdirectory in an init container.

Alternatives and When to Choose Them

  • No Volume at all. Simplest case: container reads/writes only to its own ephemeral filesystem. Fine for stateless workers, lambdas-on-K8s patterns.
  • hostPath for node-agent Pods. When the Pod is a node agent (CNI plugin, log shipper, monitoring), hostPath is the correct way to access host state. Forbidden in user-facing namespaces by PSS Restricted/Baseline.
  • emptyDir for shared scratch between containers in the same Pod. The canonical sidecar pattern: main container writes logs to /var/log/app, sidecar log-shipper reads from the same emptyDir mounted at /logs.
  • ephemeral (PVC template) instead of emptyDir for large scratch. Better when scratch exceeds node ephemeral-storage limits — moves bytes to network-attached block storage.
  • projected over multiple separate Volumes. When mounting more than one secret/configMap to the same path, projected is the right abstraction. Replaces the older “many Volumes mounted side-by-side” pattern.
  • persistentVolumeClaim for any data that must survive Pod restart. The non-negotiable choice for any stateful workload.

Production Notes

  • The CSI Migration story is largely complete by K8s 1.31. New clusters should not see in-tree drivers in any practical sense; install CSI drivers as the explicit storage layer and use PVC + StorageClass + CSI for all real storage. The migration KEP is one of the most successful examples of a multi-year Kubernetes refactor — over 1.3 million lines of in-tree code removed across 6 minor versions.
  • projected ServiceAccountToken is the right way to plumb workload identity. Manual long-lived Secret-mounted SA tokens are a security antipattern; the auto-mount default for ServiceAccounts has been migrated to projected tokens in modern clusters. Cross-link ServiceAccount, Workload Identity (Kubernetes).
  • Don’t OOM the kubelet via emptyDir: medium: Memory. Tmpfs counts against the Pod’s memory limit, but if the limit is unset (BestEffort QoS) it counts against the node’s overall memory pressure. Aggressive tmpfs use in BestEffort Pods has crashed nodes in production. Always set sizeLimit.
  • ConfigMap/Secret mount semantics (no subPath) propagate updates. This is useful: you can update config without rolling Pods. But it can also be a footgun — the application must handle file changes (re-read on demand, or use inotify). Many applications don’t, and the answer is still “restart on change.”
  • local PV is not the same as hostPath. local is a real PV type with node-affinity and proper lifecycle; hostPath is the legacy stuff-things-into-the-host approach. Always prefer local for “I need to access node-attached SSD/NVMe storage.”
  • Generic Ephemeral Volume (ephemeral Volume type) is underused. Most workloads still reach for emptyDir reflexively, even when the scratch volume should really be network-attached. CI runners and batch jobs in particular benefit from ephemeral over emptyDir.

See Also

Graph View