vault

CRI-O

14 min read

CRI-O

CRI-O is Red Hat’s minimalist Kubernetes-only container runtime, designed from day one to implement only the CRI for the kubelet and nothing else. It was born in the Kubernetes incubator in 2016 as Red Hat’s reaction to Docker’s then-monolithic engine, accepted to CNCF at incubating in April 2019, and promoted to graduated status on July 19 2023 (CNCF announcement). Its defining characteristic is what it doesn’t ship: no Docker-style CLI (crio is admin-only, not user-facing; crictl is the K8s-aligned CLI everyone uses), no swarm, no compose, no general-purpose container API. CRI-O exists solely to make the kubelet’s CRI calls happen. The largest deployment is Red Hat OpenShift Container Platform 4, where CRI-O has been the only supported runtime since the 2019 4.0 release (Red Hat blog announcement). This note covers CRI-O’s architecture (crio daemon → conmon monitor → runc/crun), the OCI-spec lineage, and the comparison to containerd.

Mental Model

flowchart TB
    KUBELET[kubelet]
    subgraph "Node"
        CRIO[crio daemon<br/>CRI gRPC server<br/>unix:///run/crio/crio.sock]
        STORE[Image + container storage<br/>containers/storage library<br/>/var/lib/containers]
    end
    subgraph "Per-container processes"
        CONMON1[conmon<br/>monitors container 1]
        CONMON2[conmon<br/>monitors container 2]
        OCI1[runc / crun<br/>container 1 init]
        OCI2[runc / crun<br/>container 2 init]
    end
    CNI[CNI plugin]

    KUBELET -- "CRI gRPC" --> CRIO
    CRIO -- "fork+exec, double-fork detach" --> CONMON1
    CRIO -- "fork+exec, double-fork detach" --> CONMON2
    CRIO -- "configure netns" --> CNI
    CRIO --> STORE
    CONMON1 -- "runc create / start" --> OCI1
    CONMON2 -- "runc create / start" --> OCI2

What this diagram shows. Like containerd, CRI-O has a per-node daemon (crio) and a per-container monitor process — but the monitor is conmon, a small C program shared with Podman/Buildah (containers/conmon), rather than containerd’s containerd-shim-runc-v2. conmon is a deliberately minimal monitor: it double-forks to detach from crio, holds the container PID1’s pty, demultiplexes stdout/stderr to log files, serves attach clients via a UNIX socket, detects OOM kills via cgroup events, and reports the exit code back to crio when the container dies. The OCI runtime under conmon is crun by default since CRI-O 1.31 (the upstream package and static-binary bundles ship crun-default — see What’s new in CRI-O 1.31); older releases and many downstream distributions still default to runc. Insight: CRI-O’s architecture is intentionally smaller than containerd’s — fewer layers, fewer plugins, no native non-CRI API — and that minimalism is the whole product differentiation.

Design Philosophy — “Only What Kubernetes Needs”

The argument Red Hat made for CRI-O at its 2016 inception was that containerd in 2016 was still entangled with Docker (the company), still exposed a native API meant for non-K8s use, and was therefore both bigger than the K8s integration needed and politically tied to a vendor whose interests were not fully aligned with the CNCF’s. CRI-O would be: open-governed from day one (under the Kubernetes umbrella, then CNCF); CRI-only (so the kubelet was the only legitimate client); and a vehicle for the Open Container Initiative (OCI) specifications — Image, Runtime, and Distribution — without any Docker-specific extensions.

That philosophy maps to concrete architectural choices:

  • No native CLI for users. crictl (the K8s-supplied CRI client) works against CRI-O the same way it works against containerd; CRI-O does not provide a containerd-style ctr equivalent. If you want to run containers outside K8s, use Podman (which shares the containers/storage and containers/image libraries with CRI-O), not CRI-O directly.
  • No plugin model for the API. containerd’s CRI is “one plugin among many” inside the containerd daemon. CRI-O’s CRI is the whole daemon. There is no other API surface to forget about, lock down, or worry about.
  • OCI all the way down. Images are OCI Image Spec artefacts (containers/image pulls them; the Distribution Spec is the registry wire format); containers are spawned via the OCI Runtime Spec’s config.json bundle; the runtime under conmon is runc (the reference OCI runtime) or any other OCI-conformant runtime. The CRI-O daemon’s responsibility is to translate CRI requests into OCI artefacts and to invoke conmon+runc for each container.
  • Version lockstep with Kubernetes. Per the CRI-O README, “CRI-O follows the Kubernetes release cycles with respect to its minor versions (1.x.y)” and adheres to the Kubernetes n-2 release-skew policy for feature graduation, deprecation, and removal. In practice this means CRI-O 1.28 is paired with K8s 1.28, 1.29 with 1.29, and so on; the active CRI-O release branch (v1.x) tracks the corresponding Kubernetes release branch. This is unusual (containerd has its own version stream) and is the most operationally-visible consequence of the “K8s-only” focus.

Mechanical Walk-through — A Pod Start in CRI-O

When the kubelet calls RunPodSandbox:

  1. crio allocates a Pod sandbox ID; pulls (if missing) the pause image declared in [crio.image] pause_image; creates a network namespace; invokes the CNI plugin chain to wire the namespace.
  2. crio writes an OCI bundle for the pause container under /run/containers/storage/.../userdata/, including the config.json that pins the namespaces and cgroups.
  3. crio forks a conmon process: conmon --runtime /usr/bin/runc --bundle <path> --container-id <id> --pidfile <path>. The conmon double-forks to detach from crio (so crio can be restarted without killing containers), then execs runc create followed by runc start. The pause container’s init process is now running inside the sandbox’s namespaces; runc exits; conmon stays as the parent.

When the kubelet then calls CreateContainer + StartContainer for a real container:

  1. crio pulls the container image into the containers/image storage at /var/lib/containers/storage/. The storage backend is overlay by default (containers/storage’s overlay driver, separate from but compatible with containerd’s overlayfs snapshotter).
  2. crio writes a new OCI bundle that mounts the container image’s layers as the rootfs, joins the sandbox’s network/IPC namespaces, and includes the seccomp/SELinux/AppArmor profiles requested by the Pod spec.
  3. crio forks another conmon, which forks runc start and supervises the container’s lifecycle. conmon writes stdout/stderr to /var/log/pods/... files (where the kubelet reads them for kubectl logs).

Container exit: conmon notices the container’s init process exited, writes the exit code to a sidecar file, sends a UNIX-domain-socket notification to crio, and itself exits cleanly. crio then reports ContainerStatus with the exit code on the next CRI poll.

The salient differences from containerd: there is no ttrpc layer (CRI-O’s daemon → conmon communication is via UNIX signals + sidecar files), no shim v2 protocol (conmon is its own protocol), and no namespace abstraction like containerd’s k8s.io (CRI-O only has one tenant: the kubelet).

conmon — The Container Monitor

conmon (containers/conmon on GitHub) is a single-purpose C program shared between CRI-O and Podman. Its responsibilities are tightly scoped:

  • Double-fork to daemonize. This is what lets crio (the parent that forks conmon) restart without killing the container. The container’s PID 1 (the init process inside its namespace) is parented to conmon; conmon is parented to PID 1 of the host (after double-forking detaches from crio).
  • Hold the container’s pty. TTY-attached containers need someone to keep the pty’s master end open even when no kubectl attach is connected.
  • Demultiplex stdout/stderr. conmon reads the container’s two streams and writes them to log files in the format <timestamp> stdout F <line> / <timestamp> stderr F <line> (the F means “full line”; P is used for partial lines that didn’t end with newline). This is the format kubectl logs parses.
  • Detect OOM. conmon subscribes to the container’s cgroup OOM event file. When the kernel OOM-kills the container, conmon notices and records OOM status, so the kubelet sees terminated.reason: OOMKilled.
  • Serve attach clients. When the kubelet proxies a kubectl exec or kubectl attach, the request lands on a UNIX socket conmon listens on; conmon shuttles bytes between the socket and the container’s pty.

The CRI-O community has been developing conmon-rs, a Rust rewrite that re-architects the monitor as pod-level rather than container-level: instead of forking one conmon per container, a single conmon-rs process listens on a UNIX-domain socket for create-container and exec requests, sharing an event loop across all containers it monitors. This addresses the criticism that the C conmon’s one-process-per-container model adds PID-table and memory overhead at high Pod density. Per What’s new in CRI-O 1.31, CRI-O 1.31 added support for conmon-rs > v0.6.5 as a drop-in replacement for the C conmon; as of the 1.31 release it is offered as a supported alternative rather than the default, with the C conmon remaining the out-of-the-box monitor.

runc vs crun — The OCI Runtime Choice

CRI-O dispatches to an OCI-conformant runtime. The historical default was runc (Go, the reference implementation under the OCI). The 2018-era alternative crun (Red Hat introduction to crun) is a C reimplementation that is shorter, faster to start, and lower-memory than runc. Per What’s new in CRI-O 1.31, the CRI-O upstream packages and static binary bundles now ship with crun as the default runtime, with the maintainers citing “overall better performance and lower memory footprint than runc” and explicitly calling out edge and WebAssembly workloads as motivations. Downstream distributions and OpenShift moved more cautiously: OpenShift 4.18 release notes make crun the default for new containers in OpenShift 4.18 (which uses Kubernetes 1.31); OpenShift 4.17 and earlier defaulted to runc, and clusters upgrading from 4.17.z to 4.18 retain their existing runc default unless explicitly changed. So as of mid-2026 the answer to “which runtime is the default?” is “crun in fresh installs of CRI-O 1.31+, runc in older clusters and earlier OpenShift releases” — verify on the specific release.

Whichever runtime is configured, the contract is the same: read an OCI config.json bundle, set up the namespaces and cgroups as specified, exec the container’s entry point inside them. Both runc and crun are OCI Runtime Spec implementations; switching between them is a default_runtime config change and a restart of crio. Other runtimes plug in the same way: Kata Containers as runtime_type = "vm" for hypervisor-isolated Pods; gVisor as runtime_type = "oci" pointing at runsc.

Configuration / API Surface

CRI-O’s primary config is /etc/crio/crio.conf (TOML), with drop-ins in /etc/crio/crio.conf.d/*.conf:

# /etc/crio/crio.conf
[crio]
root = "/var/lib/containers/storage"           # containers/storage root
runroot = "/run/containers/storage"            # ephemeral state
storage_driver = "overlay"                     # overlay | vfs | btrfs | zfs | aufs (legacy)
 
[crio.api]
listen = "/var/run/crio/crio.sock"             # CRI gRPC socket
stream_address = "127.0.0.1"                   # exec/attach streaming server bind
stream_port = "0"                              # 0 = ephemeral port
 
[crio.runtime]
default_runtime = "runc"                       # or "crun" in newer installs
selinux = true                                 # enforce SELinux per-container labels
seccomp_profile = "/etc/crio/seccomp.json"
apparmor_profile = "crio-default"
cgroup_manager = "systemd"                     # MUST match kubelet cgroupDriver
default_capabilities = [                       # CAP_* set granted by default; drop bound by Pod spec
    "CHOWN", "DAC_OVERRIDE", "FSETID", "FOWNER",
    "SETGID", "SETUID", "SETPCAP", "NET_BIND_SERVICE",
    "KILL",
]
pids_limit = -1                                # cap on PIDs per container; -1 = unlimited
 
  [crio.runtime.runtimes.runc]
  runtime_path = "/usr/bin/runc"
  runtime_type = "oci"
  runtime_root = "/run/runc"
 
  [crio.runtime.runtimes.crun]
  runtime_path = "/usr/bin/crun"
  runtime_type = "oci"
 
  [crio.runtime.runtimes.kata]
  runtime_path = "/usr/bin/kata-runtime"
  runtime_type = "vm"
 
[crio.image]
pause_image = "registry.k8s.io/pause:3.9"
pause_image_auth_file = "/etc/crio/auth.json"  # registry credentials for the pause image pull
default_transport = "docker://"
signature_policy = "/etc/containers/policy.json"  # required cosign / GPG signature policy
 
[crio.network]
network_dir = "/etc/cni/net.d"                 # CNI conflist directory
plugin_dirs = ["/opt/cni/bin"]

crictl works the same as on containerd; only crictl.yaml needs to point at the right socket:

# /etc/crictl.yaml
runtime-endpoint: unix:///run/crio/crio.sock
image-endpoint: unix:///run/crio/crio.sock
timeout: 10
debug: false

For diagnostics, crio itself exposes a small operator CLI (crio --help) and a Prometheus metrics endpoint on port 9090. Day-to-day, you operate CRI-O through crictl and through the kubelet’s behaviour, not through a CRI-O-specific UX.

Failure Modes

conmon orphans. A crashed crio daemon during a teardown can leave a conmon process whose container has exited but whose crio parent never collected the exit status. Symptom: ps auxf shows conmon processes with no children and no crio parent. Modern CRI-O reaps these on restart; on older versions they accumulate until reboot. Mitigation: monitor the conmon PID count via node-exporter; alert if it exceeds the count of running Pods.

SELinux denials on Red Hat hosts. CRI-O’s default selinux = true will deny operations the Pod’s spec doesn’t anticipate (e.g. mounting hostPath volumes from labels other than container_file_t). Symptoms appear in audit.log as type=AVC denials and as the Pod failing with permission denied on file access. Diagnosis: ausearch -m AVC -ts recent. The fix is either to label the host path correctly (semanage fcontext) or to apply a Pod-level SELinux context. This is a frequent migration surprise when moving workloads from a containerd-on-Debian cluster to a CRI-O-on-RHEL cluster.

Pause-image pull failure. CRI-O has a single pause image (vs. containerd’s per-sandbox approach to the same thing). If that image is unpullable on a fresh node (registry credentials missing, network policy blocking, registry down), no Pod can start. Symptom: every Pod stuck in ContainerCreating with failed to pull pause image. Mitigation: pre-bake the pause image into the node image; configure pause_image_auth_file with mirror credentials.

Storage driver mismatch on host upgrade. Switching the host’s filesystem (e.g. from xfs+overlay to btrfs) without changing storage_driver produces inscrutable mount errors at next container start. Always match storage_driver to the underlying filesystem.

Version-skew failures with kubelet. Because CRI-O’s minor versions track K8s minor versions, running CRI-O 1.28 with kubelet 1.30 is unsupported — the CRI v1 API may have field changes the older CRI-O doesn’t understand. Symptom: kubelet logs unknown field warnings on every CRI call. Fix: keep CRI-O and kubelet within one minor version of each other.

Cgroup manager mismatch. Same hazard as on containerd: cgroup_manager = "cgroupfs" in crio.conf paired with cgroupDriver: systemd in the kubelet config produces nodes where memory limits aren’t enforced. Both must be the same.

Alternatives and When to Choose Them

containerd is the primary alternative and the bigger ecosystem. The trade space:

AxisCRI-Ocontainerd
ScopeK8s-onlyK8s + Docker Engine + standalone use (ctr)
Daemon size & surfaceSmaller, fewer pluginsLarger, plugin-rich
Per-container monitorconmon (or conmon-rs)containerd-shim-runc-v2 (ttrpc)
Image librarycontainers/image, containers/storage (Podman shared)bespoke (containerd content store)
OS focusRHEL/CentOS/Fedora (Red Hat ecosystem)Distro-agnostic
Default inOpenShift 4.xEKS, GKE, AKS, kubeadm, k3s
K8s minor couplingStrict (CRI-O 1.X ↔ K8s 1.X)Loose (separate version stream)
Native CLINone (use crictl)ctr
CNCF maturityGraduated 2023Graduated 2019

Choose CRI-O when: you’re on OpenShift (no choice), you want the smallest possible runtime surface, your security/compliance posture rewards “less code = less attack surface” arguments, or your fleet is Red Hat-aligned and you benefit from the shared library stack with Podman/Buildah/Skopeo.

Choose containerd when: you’re on a major managed K8s service (it’s the default), you have non-K8s container workloads to standardise around, or you need niche features (devmapper snapshotter, lazy-loading snapshotters like SOCI/stargz) that CRI-O doesn’t ship.

cri-dockerd is rarely chosen new; only used to keep Docker Engine alive on K8s nodes for legacy tooling. Avoid for greenfield.

Production Notes

  • OpenShift 4 at large enterprises. Banks, telcos, US federal agencies, and other organisations running OpenShift in regulated environments effectively run CRI-O in production at very large scale. Red Hat’s “OpenShift on the bare metal” deployments at the major US banks reportedly run thousands of CRI-O nodes; the Air Force’s Platform One uses OpenShift / CRI-O for parts of its software supply chain. This is the largest single CRI-O user base in the world.
  • The kubelet ↔ CRI-O ↔ kubelet ↔ containerd interfaces are interchangeable from the kubelet’s perspective. This is a deliberate consequence of CRI: nothing in the kubelet differs between the two, including the gRPC protocol version negotiation, the streaming server URL handshake, or the way Pod sandbox state is reported. Migrating between them is “drain node, change runtime endpoint, restart kubelet” — well exercised in OpenShift-to-vanilla-K8s and reverse migrations.
  • conmon-rs adoption (CNCF What’s new in CRI-O 1.31, 2024) is the single most important recent CRI-O initiative; it shifts conmon from “one process per container” to a shared event-loop that scales to hundreds of containers per node without the OS PID overhead.
  • Performance. The CRI-O 1.31 release commentary frames the runtime-default change to crun as a performance win — faster container start times and lower per-container memory than runc — though the upstream blog post does not publish head-to-head Pod-start numbers against containerd. In practice the gap between the two CRI implementations is small and almost never the deciding factor in node sizing; image pull and CNI plugin setup dominate Pod-start latency.

Uncertain

Comparative Pod-start latency and node-memory benchmarks between CRI-O and containerd at equivalent configuration (same OCI runtime, same image, same kernel) are not consistently published in primary sources. The text above describes the qualitative consensus from the CRI-O 1.31 announcement; precise numbers vary by workload and kernel version. To resolve: cite a peer-reviewed benchmark such as a SIG-Node performance test report rather than vendor blogs.

See Also

Graph View

Backlinks