vault

Containers vs Virtual Machines

14 min read

Containers vs Virtual Machines

Containers and virtual machines both promise “isolated execution,” but they draw the isolation boundary in fundamentally different places, and that one difference cascades into everything else. A container shares the host kernel: the workload’s system calls hit the same kernel the host runs, and the boundary is enforced inside that kernel by namespaces and cgroups at the system-call layer. A virtual machine runs a separate guest kernel on emulated or virtualized hardware behind a hypervisor: the workload’s system calls hit the guest’s own kernel, and the only thing crossing into the host is a narrow stream of privileged hardware operations the hypervisor mediates. The container’s attack surface is therefore the entire kernel system-call ABI — on x86-64 in Linux 6.12 LTS that is on the order of 460-odd system calls (syscall_64.tbl, v6.12, highest number 462 = mseal). The VM’s attack surface is the much narrower hypercall/virtio device interface — a handful of emulated devices (Firecracker design). In exchange, the container is dramatically lighter: no second kernel to boot, no duplicated memory, startup in milliseconds. Everything below is the elaboration of that single trade — boundary placement against weight — and the hybrid sandboxes (gVisor, Kata/Firecracker) that try to buy some of the VM’s safety at closer to a container’s cost.

Mental Model — Where Is the Boundary?

The clarifying question is not “how isolated is it?” but “where does the workload’s system call go, and what is on the other side?” In a container, the system call goes straight to the host kernel; the isolation is a set of checks the host kernel performs on itself (does this PID namespace let you see that process? does this cgroup have CPU budget left?). In a VM, the system call goes to a guest kernel running in a virtual CPU; that guest kernel is itself a normal Linux/Windows kernel, fully unaware it is virtualized for most operations, and only when it touches privileged hardware (set a page table, do device I/O) does control trap down to the hypervisor on the host.

flowchart TB
  subgraph HOST["Single physical host (one CPU, one set of RAM)"]
    HK["HOST KERNEL"]
    subgraph CONT["Container model"]
      CW["Container workload"]
      CW -->|"~460 host syscalls<br/>(full kernel ABI)"| HK
      NS["namespaces + cgroups<br/>check INSIDE host kernel"]
      HK --- NS
    end
    subgraph VM["Virtual-machine model"]
      VMW["VM workload"]
      GK["GUEST KERNEL<br/>(separate copy)"]
      VMW -->|"syscalls hit the GUEST"| GK
      HV["Hypervisor / VMM<br/>(KVM + virtio)"]
      GK -->|"narrow: vCPU traps + virtio<br/>(~handful of devices)"| HV
      HV --> HK
    end
  end

Two boundary placements on one host. What it shows: in the container model (left) the workload’s system calls reach the host kernel directly and isolation is enforced by namespace/cgroup checks within that kernel; in the VM model (right) the workload’s system calls are absorbed by a separate guest kernel, and only a thin trickle of privileged operations (virtual-CPU traps plus a few virtio devices) reaches the host via the hypervisor. The insight to take: the container exposes the whole host-kernel system-call ABI as attack surface (a kernel bug reachable from any of ~460 syscalls can escape), whereas the VM exposes only the hypervisor’s narrow device/hypercall interface — far less code, far fewer reachable bugs. That smaller surface is the entire security argument for VMs, and it is paid for with a second kernel’s worth of memory and boot time.

The Container Boundary — Shared Kernel, Syscall-Level Checks

A container is, mechanically, just a host process whose view and budget have been restricted (see Namespaces and cgroups as Container Building Blocks and What Is a Linux Container). Its threads run on the host CPU in user mode and make ordinary system calls into the host kernel. There is no second kernel anywhere. When a containerized process calls openat(), clone(), or io_uring_setup(), that call enters the exact same entry_SYSCALL_64 path the host’s own daemons use; the namespace and cgroup checks happen along that path inside the one shared kernel.

This is why the container’s kernel attack surface is the full system-call ABI. Every system call the kernel implements is reachable from inside the container (unless a seccomp filter removes it), and a memory-safety bug or logic flaw in any of those handlers is a potential container escape — a bug in the kernel the container shares is a bug in the container’s security boundary. On x86-64 in Linux 6.12, the syscall table runs from number 0 up to 462 (mseal), with the 64-bit-specific tail including entries like 461 (lsm_list_modules) — on the order of 460-plus distinct system calls in the reachable surface (syscall_64.tbl, v6.12). That number grows every release as new syscalls are added; each addition widens the surface. By contrast, the namespace machinery that does the isolating is comparatively tiny — eight CLONE_NEW* flags (namespaces(7); flag values in v6.12 sched.h) — but those flags gate visibility, not the size of the syscall surface behind them.

The upside of sharing the kernel is exactly what makes containers light. There is no guest kernel to boot, so startup is the cost of clone() + exec() — single-digit to low-tens of milliseconds. There is no guest kernel resident in RAM, no duplicated page cache, no second copy of kernel data structures — so memory overhead per container is essentially the workload’s own footprint plus a thin slice of metadata. Density is correspondingly high: hundreds to thousands of containers per host are routine, because they are just processes. And because the kernel’s page cache and CPU scheduler are shared, identical pages (shared image layers via OverlayFS) are deduplicated for free and idle containers cost almost nothing.

The VM Boundary — Separate Guest Kernel Behind a Hypervisor

A virtual machine inverts the picture. A hypervisor (on Linux, the Kernel-based Virtual Machine, KVM, exposed to userspace through a Virtual Machine Monitor (VMM) such as QEMU or Firecracker) uses CPU virtualization extensions (Intel VT-x / AMD-V) to create a virtual CPU. The guest runs a complete, separate kernel on that virtual CPU. The guest kernel handles the workload’s system calls itself — they never reach the host. Only when the guest performs an operation that the hardware traps (accessing a model-specific register, programming a device, taking certain page faults) does control “VM-exit” down to KVM, and from there possibly up to the VMM.

The security consequence is the mirror image of the container’s: the host-facing attack surface is not the system-call ABI but the device/hypercall interface, and that interface is deliberately tiny. Firecracker, Amazon’s minimalist VMM built for serverless, is the cleanest illustration. It gives each guest only “a network device, a block I/O device, a Programmable Interval Timer, the KVM clock, a serial console, and a partial keyboard” — a handful of emulated devices, chosen explicitly to “minimize the attack surface” (AWS Firecracker announcement). The richer devices use virtio, a paravirtualized interface where the guest cooperates with the host through shared ring buffers rather than the host emulating real hardware register-by-register (Firecracker design). For an attacker in the guest to escape, they must find a bug in that narrow virtio/device surface plus KVM’s vCPU handling — a far smaller and far more scrutinized body of code than the entire kernel syscall ABI. Firecracker further wraps the VMM in a jailer (which sets up restricted resources, drops privileges, then execs Firecracker) and a default seccomp filter limiting the host syscalls the VMM itself may make (Firecracker design) — defense in depth around an already-narrow boundary.

The cost is the second kernel. A VM must boot a guest kernel and a guest userspace, which classically meant seconds of startup and tens to hundreds of MiB of overhead per VM just for the guest OS. Firecracker’s entire reason for existing is to shrink that: it boots a microVM “in as little as 125 ms” and consumes “about 5 MiB of memory per microVM” of VMM overhead (AWS announcement), sustaining a mutation rate of “5 microVMs per host core per second” (Firecracker design). Even so, each microVM still runs a real guest kernel — the 5 MiB is the VMM’s overhead, not the guest kernel and userspace RAM, which is additional. A plain container has neither cost because it has no guest kernel at all.

The Trade-off, Made Concrete

DimensionContainer (shared kernel)Virtual machine (separate kernel)
Host attack surfaceFull host-kernel syscall ABI (~460 syscalls on x86-64, v6.12)Narrow hypercall + virtio/device interface (a handful of devices)
KernelShares the host’sRuns its own guest kernel
StartupMilliseconds (clone+exec)~125 ms (Firecracker microVM) to seconds (full VM)
Per-instance memory overheadWorkload footprint + thin metadataGuest kernel + userspace + VMM (~5 MiB VMM for Firecracker, plus the guest OS)
Density per hostHundreds–thousandsLower (each carries a guest OS)
Resource sharingShared page cache, dedup of common layersLargely private per guest

The container–VM trade-off in one table. The insight: moving the boundary down to the hardware level (VM) shrinks the host attack surface by orders of magnitude but pays for it in a second kernel’s weight; keeping the boundary at the syscall level (container) is featherweight but exposes the whole kernel. There is no free lunch — which is precisely why hybrid designs exist. The syscall count is pinned to x86-64 in v6.12 and grows each release; treat it as “hundreds, and increasing,” not a fixed constant.

The Hybrids — Buying VM-Grade Safety at Near-Container Cost

The container/VM dichotomy is a spectrum, and two influential sandboxes sit between the poles by moving the syscall boundary somewhere new. Both target the same goal: shrink the host-kernel attack surface without paying full-VM weight.

gVisor keeps the lightweight feel of a container but interposes a user-space kernel between the workload and the host. Its core component, the Sentry, “can be thought of as an application kernel” that “implements all the kernel functionality needed by the application, including: system calls, signal delivery, memory management and page faulting logic, the threading model, and more” (gVisor docs). The container’s system calls are intercepted and redirected to the Sentry, which services them itself — “the Sentry does not pass system calls through to the host kernel” (gVisor docs). The Sentry does make a small, fixed set of host syscalls to do its own work, but it cannot “open new files, create new sockets or do many other interesting things on the host”; file access is brokered through a separate Gofer process, and its host syscall set is “explicitly enumerated and controlled” by its own seccomp filter (gVisor security model). The security argument is defense in depth: “no system call is passed through directly to the host”, and the Sentry’s independent re-implementation is unlikely to share the host kernel’s exact bugs (gVisor security model). How the interception physically happens is the platform layer: the default since mid-2023 is systrap, which traps syscalls via seccomp so “the kernel send[s] SIGSYS to the triggering thread, which hands over control to gVisor”; alternatives are a KVM platform (Sentry acts as guest OS and VMM) and a legacy ptrace platform (PTRACE_SYSEMU, high overhead, being removed) (gVisor platforms). The cost is real: gVisor’s own docs note “reduced application compatibility and higher per-system call overhead” (gVisor docs) — the price of re-implementing the kernel in user space.

Uncertain

Verify: the exact count of host syscalls the gVisor Sentry’s seccomp filter permits (a frequently cited “~50–70” figure). Reason: gVisor’s public security and platform docs (fetched 2026-06-14) describe the host syscall set qualitatively — “explicitly enumerated and controlled,” limited to fd duplication/closing, synchronization, timers, signals, and a virtual-ethernet socket — but give no integer in the fetched pages. To resolve: read the seccomp filter source in the gVisor repo (runsc/boot/filter) at a pinned release tag. The qualitative claim (Sentry makes far fewer host syscalls than the ~460 a native container can) is well-supported; the precise number is not pinned here. uncertain

Kata Containers takes the opposite tack: keep the real boundary of a VM, but make it look and feel like a container. Kata wraps each OCI container (or Kubernetes pod) in a “lightweight virtual machine” so the workload runs against a guest kernel, giving “stronger workload isolation using hardware virtualization technology” while still feeling like a standard container (Kata architecture). It can drive several hypervisors — QEMU, Cloud Hypervisor, Firecracker, and its own Dragonball — and a kata-agent runs inside the guest as “the supervisor for managing the containers and the workload”, talking to the host runtime over a VSOCK channel. The host-side piece is a containerd-shim-kata-v2 binary, a shimv2-compatible runtime that “runs a single instance of the runtime binary (for any number of containers)”, so Kubernetes/CRI-O can launch a Kata pod exactly as they would a runc pod — only now the pod is a microVM (Kata architecture). The isolation is the genuine VM boundary (nested: workload inside container inside guest VM inside host), at microVM weight when paired with Firecracker.

The neat symmetry: gVisor pulls the syscall boundary up into user space without a guest kernel; Kata pushes the whole container down behind a real guest kernel using a minimal VMM. Both shrink the host-kernel attack surface relative to a native container; both pay overhead a native container avoids.

Failure Modes and Misunderstandings

“Containers are insecure, VMs are secure” — too coarse. A well-confined container (dropped capabilities, a tight seccomp allowlist, a user namespace, an LSM profile) closes most of the syscall surface and is far safer than a naive one. Conversely a VM with a fat emulated-device set and a buggy VMM can be escaped. The accurate statement is architectural: the VM’s host-facing surface is intrinsically narrower, so it has a higher security ceiling — but a poorly configured VM can still lose, and a well-configured container can be quite strong. The default container surface, however, is the full kernel ABI, which is why multi-tenant platforms reach for VMs or hybrids.

“A container is a lightweight VM.” No — there is no VM, no guest kernel, no hypervisor in a plain container. It is a host process with restricted visibility and budget. Calling it a “lightweight VM” hides the one fact that matters: it shares your kernel. Marketing that blurs this has caused real security misjudgements.

“VMs and containers are mutually exclusive.” They compose. The overwhelmingly common production pattern is containers running inside VMs: a Kubernetes node is usually a VM, and the pods on it are containers. You get the VM boundary between tenants/nodes and the container’s density and packaging within a node — and Kata makes that boundary per-pod.

“The syscall count is a fixed number.” It is version- and architecture-specific and monotonically grows. The 462 ceiling here is x86-64 in v6.12; ARM64 differs, and later kernels add more. The takeaway is the order of magnitude (hundreds, expanding), not a constant.

Choosing Between Them

Reach for a plain container when the workloads are trusted relative to each other (your own services, one team, one trust domain) and you want maximum density, fastest startup, and simplest packaging. Reach for a VM boundary (or a hybrid) when you run untrusted or mutually-distrusting code on shared hardware — multi-tenant serverless (AWS Lambda/Fargate use Firecracker), CI runners executing arbitrary user code, or any setting where a host-kernel escape is catastrophic. Within “stronger isolation,” choose gVisor when you want a container-like deployment and can tolerate syscall-compatibility gaps and per-syscall overhead, and Kata + Firecracker when you want a true hardware boundary at near-container weight and full kernel compatibility (the guest is a real kernel). The decision is always the same trade restated: how much of the host kernel are you willing to expose, and how much weight will you pay to expose less. See The Container Isolation Spectrum for the full continuum.

See Also

Graph View

Backlinks