vault

KSM and Memory Overcommit for VMs

16 min read

KSM and Memory Overcommit for VMs

Memory overcommit for virtual machines is the practice of provisioning the sum of all guests’ configured RAM to exceed the host’s physical RAM — Σ(guest RAM) > host RAM — on the bet that guests rarely use all their memory at once. Because under KVM (the Kernel-based Virtual Machine) a guest’s RAM is just ordinary host anonymous memory backing a VMM process (see Guest Physical Memory and Memory Slots), it is reclaimable, swappable, and dedup-able like any other host memory. Three mechanisms make overcommit safe and dense: lazy allocation (a guest page consumes host RAM only when first touched, via demand paging), virtio-balloon (a paravirtual device that lets the host reclaim idle guest RAM by asking the guest to “inflate” a balloon of pages it pledges not to use), and Kernel Samepage Merging (KSM), which scans guest RAM marked MADV_MERGEABLE and collapses byte-identical pages across VMs into one shared, copy-on-write page. KSM was, in fact, born for this: it “was originally developed for use with KVM … to fit more virtual machines into physical memory, by sharing the data common between them” (ksm.rst, v6.12). This note owns the overcommit-for-VMs angle and pins to the Linux 6.12 / 6.18 long-term-support (LTS) kernels; the KSM internals (the stable and unstable red-black trees, the scan algorithm) live in the sibling note Kernel Samepage Merging, and the generic accounting (vm.overcommit_memory, Committed_AS, CommitLimit) lives in Memory Overcommit and Accounting.

Overcommit is a density-vs-risk dial. Pushed too far it ends in host out-of-memory: when too many guests touch too many pages at once, the host has no RAM to satisfy the page faults, swap thrashes, and the OOM killer may shoot a VMM process — killing an entire guest. KSM and ballooning widen the safe band of overcommit, but neither removes the fundamental gamble.

Mental Model — Three Levers on One Pool

Picture the host’s physical RAM as a single shared pool, and every guest as a tenant who reserved a certain amount but occupies only part of it. The three mechanisms each attack a different kind of slack:

flowchart TB
  POOL["Host physical RAM<br/>(the real, finite pool)"]
  subgraph LEVERS["Three levers that pack more guests into the pool"]
    LAZY["Lazy allocation<br/>guest page → host page<br/>only on first touch"]
    BALLOON["virtio-balloon<br/>host reclaims idle guest RAM<br/>(guest inflates balloon)"]
    KSM["KSM<br/>merge byte-identical pages<br/>ACROSS guests into one COW page"]
  end
  G1["Guest A<br/>(reserved 8G, touched 3G)"]
  G2["Guest B<br/>(reserved 8G, touched 3G)"]
  G3["Guest C<br/>(reserved 8G, touched 3G)"]
  G1 --> LEVERS
  G2 --> LEVERS
  G3 --> LEVERS
  LEVERS --> POOL
  RISK["Risk: if guests touch their<br/>reservations simultaneously →<br/>host OOM, OOM-killer shoots a VMM"]
  POOL -.->|"overcommit too far"| RISK

Figure: the three overcommit levers and the shared risk. What it shows: lazy allocation exploits reserved-but-untouched memory, ballooning exploits touched-but-now-idle memory, and KSM exploits touched-and-identical memory (zeroed pages, duplicate guest kernels, shared libraries). All three feed back into one finite physical pool. The insight to take: each lever harvests a different form of slack, so they compose — but they all draw against the same pool, and overcommitting past the point where guests’ simultaneous working sets exceed RAM is a bet that can be lost catastrophically.

Mechanical Walk-through

Lever one: lazy allocation (why a 64 GiB guest does not eat 64 GiB)

When a VMM mmaps an anonymous region for guest RAM and registers it as a KVM memory slot, no physical pages are allocated yet. The first time the guest writes to a guest-physical page, the second-dimension page-table walk (NPT) finds no mapping, the CPU raises a fault that surfaces to the host as an ordinary anonymous-memory page fault, and only then does the host allocate a real page. A freshly-booted guest with 64 GiB configured may have touched only a couple of gigabytes, so it consumes only that. This is plain Linux demand paging applied to guest RAM — the same mechanism that lets malloc() overcommit (see Memory Overcommit and Accounting). It is the foundation overcommit rests on: without it, every guest would pin its full reservation immediately and no overcommit would be possible.

Lever two: virtio-balloon — reclaiming touched-but-idle RAM

Lazy allocation only helps with never-touched pages. Once a guest has dirtied a page, the host page stays allocated even after the guest stops needing it — the host cannot tell the page is now idle. virtio-balloon is a paravirtual device that gives the host a cooperative channel to reclaim such pages. Per the OASIS virtio 1.3 specification, “the device asks for a certain amount of memory, and the driver supplies it (or withdraws it, if the device has more than it asks for)” (virtio 1.3 balloon spec).

The mechanism: the device exposes a config field num_pages (the host’s target balloon size) and actual (how much the guest has actually given). To inflate the balloon, “the driver constructs an array of addresses of unused memory pages. These addresses are divided by 4096 and the descriptor describing the resulting 32-bit array is added to the inflateq.” In plain terms: the guest kernel’s balloon driver allocates ordinary guest pages, removes them from guest use, and hands their guest-page-frame-numbers (PFNs) to the host. The host, seeing those pages are now off-limits to the guest, frees the corresponding host physical pages (e.g. via madvise(MADV_DONTNEED) on the backing region) and hands that physical RAM to other guests — “achieving memory overcommit and reuse” (rwmj 2010). To deflate, the host lowers num_pages; the guest may then re-use balloon pages it had surrendered, submitting them on the deflateq.

Two feature bits matter for safety. VIRTIO_BALLOON_F_MUST_TELL_HOST means “host has to be told before pages from the balloon are used” — without it, deflation is merely advisory. VIRTIO_BALLOON_F_DEFLATE_ON_OOM “permits emergency balloon deflation when guests face out-of-memory conditions,” so a guest under memory pressure can claw pages back rather than OOM-kill its own processes. VIRTIO_BALLOON_F_STATS_VQ adds a statistics virtqueue so the host can read guest memory pressure and size the balloon intelligently. VIRTIO_BALLOON_F_FREE_PAGE_HINT lets the guest proactively tell the host which pages are currently free (useful to skip transferring them during migration). The hard rule the guest must obey: once a page is in the balloon, the guest “cannot access pages after submitting them via inflate operations” until it has been deflated — touching a ballooned page is a guest bug.

Uncertain

Verify: that the traditional balloon (this device, using 4 KiB-PFN arrays on inflateq/deflateq) is the one in current production use versus newer alternatives like virtio-mem (which adds/removes memory in larger blocks and is now often preferred for dense overcommit). Reason: the OASIS 1.3 spec describes the traditional balloon authoritatively, but the relative production prevalence of balloon vs virtio-mem on 6.12/6.18-era stacks was not verified against a primary deployment source in this task. To resolve: consult current QEMU/libvirt and cloud-hypervisor docs on virtio-mem vs virtio-balloon recommendations. uncertain

Lever three: KSM — merging touched-and-identical RAM across VMs

Even idle reclamation leaves a large class of slack untouched: distinct pages in different guests that happen to hold identical bytes. Run twenty VMs of the same Linux distribution and they share enormous quantities of identical content — the zeroed pages of freshly-allocated buffers, identical guest-kernel text, the same libc/glibc pages loaded read-only in every guest, identical container base images. KSM harvests this. A background kernel thread, ksmd, scans anonymous memory that has been opted in, finds byte-for-byte identical pages, and replaces them with “a single write-protected page (which is automatically copied if a process later wants to update its content)” (ksm.rst, v6.12). The copy-on-write break makes the merge transparent: the instant any owner writes, that writer silently gets a private copy, so correctness is never affected — only timing (which is the side-channel hazard below).

The crucial detail for VMs is the cross-VM scope. KSM does not care which process owns a page; if guest A and guest B each have a page of the same zeroed buffer, KSM merges both into one host physical page shared between the two VMM processes. That is how N identical guests can collectively cost far less than N× the per-guest footprint.

How a VMM opts guest RAM into KSM

KSM is off by default and opt-in twice over: the daemon must be running (/sys/kernel/mm/ksm/run set to 1) and the memory must be marked mergeable. The classic per-region opt-in is madvise(addr, length, MADV_MERGEABLE), which QEMU can apply to its guest-RAM mappings. Linux 6.4 added a coarser, process-wide opt-in that is friendlier for VMM and container runtimes: the prctl options PR_SET_MEMORY_MERGE (= 67) and PR_GET_MEMORY_MERGE (= 68) (prctl.h, v6.12). Calling prctl(PR_SET_MEMORY_MERGE, 1) marks all of a process’s eligible anonymous memory (and that of future allocations) as mergeable without enumerating individual regions — a single call to enroll an entire VMM. KSM only ever merges anonymous pages, never page-cache (file-backed) pages, because identical file pages are already shared through the page cache; there is nothing for KSM to add there.

Reading whether it is working: the sysfs knobs and metrics

The control and accounting surface lives under /sys/kernel/mm/ksm/ (ksm.rst, v6.12). The operational knobs that an overcommit operator tunes:

  • run — 0 stop but keep already-merged pages, 1 run, 2 stop and un-merge everything.
  • pages_to_scan — pages examined per scan cycle (default 100).
  • sleep_millisecs — milliseconds ksmd sleeps between cycles (default 20). Together these two cap KSM’s CPU cost: more pages/cycle and less sleep = faster dedup but more CPU.
  • merge_across_nodes — 1 (default) merges across NUMA nodes; 0 confines merges to a single node, accepting less sharing in exchange for lower memory-access latency for guests.
  • use_zero_pages — when 1, empty pages merge with the kernel zero page rather than with each other.
  • max_page_sharing — cap on how many owners may share one KSM page (minimum 2); bounds the cost of the reverse-mapping walk when that page is finally written.
  • advisor_mode — none (default) or scan-time, an auto-tuner that adjusts pages_to_scan to hit a target scan time (advisor_target_scan_time, default 200 s) within a CPU budget (advisor_max_cpu, default 70%).

The metrics that tell you whether overcommit is paying off:

  • pages_shared — the number of distinct KSM “master” pages.
  • pages_sharing — the number of additional mappings collapsed onto those masters; this is the savings. “A high ratio of pages_sharing to pages_shared indicates good sharing, but a high ratio of pages_unshared to pages_sharing indicates wasted effort.”
  • pages_unshared — pages that are candidates but found no duplicate (pure overhead: they cost rmap bookkeeping for no gain).
  • pages_volatile — pages changing too fast to merge.
  • general_profit — KSM’s net win in bytes, computed roughly as ksm_saved_pages * sizeof(page) - (all_rmap_items) * sizeof(rmap_item). When use_zero_pages is on, “the sum of pages_sharing + ksm_zero_pages represents the actual number of pages saved.” general_profit is the single number that answers “is KSM helping or hurting this host?” — if negative, the rmap bookkeeping costs more memory than the dedup saves, and KSM should be turned off.

Configuration / Worked Example

A density-oriented host running many similar Linux guests, tuning KSM and observing the result:

# Enable the KSM daemon and tune it for aggressive dedup of guest RAM.
echo 1     > /sys/kernel/mm/ksm/run            # start ksmd
echo 1000  > /sys/kernel/mm/ksm/pages_to_scan  # scan 1000 pages/cycle (10x default)
echo 20    > /sys/kernel/mm/ksm/sleep_millisecs # 20 ms between cycles
echo 1     > /sys/kernel/mm/ksm/use_zero_pages  # collapse zeroed guest pages
 
# Observe the payoff after VMs have booted and settled.
PS=$(cat /sys/kernel/mm/ksm/pages_sharing)
PSH=$(cat /sys/kernel/mm/ksm/pages_shared)
echo "saved ~$(( PS * 4096 / 1024 / 1024 )) MiB; share ratio = pages_sharing/pages_shared"
cat /sys/kernel/mm/ksm/general_profit   # net bytes saved; if negative, disable KSM

Line-by-line: writing 1 to run starts ksmd; raising pages_to_scan to 1000 makes it dedup faster at the cost of more CPU (each cycle hashes and tree-walks ten times as many pages); use_zero_pages is especially valuable for VMs because freshly-allocated guest buffers are full of zeros. The savings estimate multiplies pages_sharing (the number of eliminated duplicate mappings) by the 4 KiB page size. The final general_profit read is the go/no-go: a negative value means the per-page rmap_item overhead exceeds the dedup win on this workload, and KSM is a net loss.

A VMM can alternatively opt its whole address space in programmatically:

#include <sys/prctl.h>
/* Enroll the entire VMM process's anonymous memory into KSM, no per-region madvise. */
if (prctl(PR_SET_MEMORY_MERGE, 1, 0, 0, 0) != 0)
        perror("PR_SET_MEMORY_MERGE");        /* needs CONFIG_KSM and Linux >= 6.4 */

This single call (option value 67) marks all current and future eligible anonymous regions mergeable — far simpler than tracking every guest-RAM mmap and calling madvise(MADV_MERGEABLE) on each (prctl.h, v6.12).

Failure Modes and the Cross-VM Side-Channel

The dedup timing side-channel. KSM’s transparency hides a leak. Writing to a merged page is measurably slower than writing to a non-merged page, because the merged page is write-protected and the first write must take a fault and perform a copy-on-write break, whereas an unmerged page is written directly. The VUSEC “Dedup Est Machina” work (IEEE Symposium on Security and Privacy, 2016) weaponized exactly this: an attacker writes a page with guessed contents, then times a write to it — a slow write means the page was deduplicated, which means that exact content already existed elsewhere on the system (VUSEC, S&P 2016). Because KSM merges across VMs and processes, this leaks information across the VM isolation boundary: a malicious guest can probe whether a victim guest holds a specific page (a known library version, a secret-derived value, an ASLR-relevant pointer). The paper combined this primitive with Rowhammer to escalate from information disclosure to memory corruption from JavaScript. The fallout was concrete: Microsoft assigned CVE-2016-3272 and disabled memory deduplication by default on Windows 10. The lesson for Linux operators: KSM’s whole value (cross-tenant sharing) is also its whole danger — never enable cross-VM KSM between mutually-distrusting tenants. On multi-tenant clouds KSM is therefore typically restricted to merging within a single trust domain, or disabled outright. (KSM internals and the merge mechanics that create this write-protected page are detailed in Kernel Samepage Merging.)

Un-merging can trigger OOM. Setting run to 2, or calling madvise(MADV_UNMERGEABLE), breaks every merge — each shared page must become N private copies. The kernel doc warns this “may suddenly require more memory than is available - possibly failing with EAGAIN, but more probably arousing the Out-Of-Memory killer.” On an overcommitted host this is a real hazard: the savings KSM provided are exactly the memory you suddenly need back.

Overcommitting past the working-set sum. The deepest failure is policy, not mechanism. KSM and ballooning reclaim idle and duplicate memory; they cannot conjure RAM when guests’ active, distinct working sets sum above physical RAM. When that happens the host swaps heavily (guest performance collapses) and may OOM-kill a VMM — destroying an entire guest. Worse, KSM and ballooning encourage aggressive overcommit by hiding slack, so a host that looks comfortably packed can fall off a cliff when a correlated event (every guest’s nightly cron, a traffic spike) makes working sets balloon simultaneously. Overcommit ratios must be set against the correlated peak of working sets, not their average.

KSM CPU cost on the wrong workload. Hosts with little inter-guest duplication (heterogeneous guests, mostly-unique data) get high pages_unshared and pages_volatile and low pages_sharing — KSM burns CPU scanning and hashing for negligible savings, and general_profit goes negative. KSM is a similar-guests optimization; on dissimilar guests it should be off.

Alternatives and When to Choose Them

  • KSM (same-page merging). Best when guests are similar (same OS image, same libraries, lots of zeroed/duplicate content) and mutually trusted. CPU cost for memory savings. Avoid across distrusting tenants (side-channel).
  • virtio-balloon. Best for reclaiming idle memory from cooperative guests and dynamically rebalancing RAM between guests. Requires a guest driver and guest cooperation; a hostile or buggy guest can refuse to inflate. VIRTIO_BALLOON_F_DEFLATE_ON_OOM mitigates guest-side OOM.
  • virtio-mem. A newer paravirtual device that plugs/unplugs guest memory in larger blocks; often preferred over the traditional balloon for large-scale dynamic resizing and hot-plug semantics (see uncertainty flag above). Cross-link virtio Device Model.
  • Transparent huge pages (THP) and lazy allocation. Foundational and always-on; lazy allocation enables overcommit at all, THP improves NPT walk efficiency for guests (see Huge Pages and NUMA for Guests). Note THP and KSM interact: KSM operates at base-page granularity and can fragment huge pages.
  • No overcommit (RAM = Σ guest RAM). The conservative choice for latency-critical or untrusted-tenant workloads where the OOM risk and side-channel are unacceptable. You pay in density.

Production Notes

KSM’s KVM origin is not incidental — it is the canonical use case and the reason use_zero_pages and cross-process merging exist. Cloud Hypervisor’s memory-management guide and Firecracker’s ballooning docs both treat balloon-based reclamation as a first-class density tool for serverless/microVM fleets, where packing many short-lived guests onto one host is the whole economic model (Cloud Hypervisor memory techniques; Firecracker ballooning). The standard production posture is: enable lazy allocation (free, always), enable ballooning for elastic rebalancing, and enable KSM only within a single tenant’s VMs (or not at all on adversarial multi-tenant hosts) because of the dedup side-channel. The interaction with Confidential Computing and the Trust Boundary is decisive: SEV-SNP/TDX guests have hardware-encrypted RAM, so identical plaintext produces different ciphertext per guest — KSM finds nothing to merge, and ballooning must be re-architected around encrypted pages. Confidential VMs therefore largely forfeit KSM-based density, which is part of their cost. Overcommit policy must also account for the OOM killer and the generic accounting machinery in Memory Overcommit and Accounting, which is what ultimately pays the debt when the overcommit bet is lost.

See Also

Graph View

Backlinks