vault

Lightweight Kubernetes Distributions

12 min read

Lightweight Kubernetes Distributions

A lightweight Kubernetes distribution is a packaging of Kubernetes optimized for minimal resource footprint and operational simplicity — typically a single binary that bundles the control plane, kubelet, a container runtime, a CNI plugin, and a datastore, so a working cluster can run in a few hundred megabytes of RAM on a Raspberry Pi or a branch-office server (k3s — Architecture). The three dominant examples are k3s (originated at Rancher, now SUSE; a CNCF Sandbox project), k0s (Mirantis-sponsored, a CNCF Sandbox project under the Linux Foundation), and MicroK8s (Canonical). They exist because the standard kubeadm-bootstrapped stack — multiple control-plane processes, a separate etcd, a separate runtime, a separately-installed CNI — is heavy: too heavy for edge devices, IoT gateways, single-node deployments, and CI runners. Lightweight distributions are still certified-conformant Kubernetes (your manifests run unchanged), but they make trade-offs — bundling components, simplifying the datastore, and removing legacy or alpha cruft — that the full upstream stack does not. They are distinct from Local Kubernetes Distributions (kind, minikube), which target the developer laptop rather than production edge.

Mental Model

flowchart TB
    subgraph FULL["Standard kubeadm cluster — many parts"]
        F1["kube-apiserver"]
        F2["etcd (separate)"]
        F3["kube-scheduler"]
        F4["kube-controller-manager"]
        F5["kubelet"]
        F6["containerd (separate install)"]
        F7["CNI plugin (separate install)"]
    end
    subgraph LIGHT["k3s — ONE binary, <100 MB"]
        L1["control plane + kubelet + kube-proxy<br/>+ containerd + flannel CNI<br/>+ datastore (SQLite default)"]
    end
    FULL -.->|"collapse + simplify"| LIGHT

    subgraph USE["Where lightweight distros win"]
        U1["Edge / IoT gateways"]
        U2["Single-node / branch office"]
        U3["CI runners / ephemeral clusters"]
        U4["ARM devices (Raspberry Pi)"]
    end

What this shows. The defining move of a lightweight distribution is collapse: the seven-or-more separately-installed components of a standard cluster become one binary. k3s additionally substitutes — etcd is replaced by SQLite for single-node clusters via the kine shim — and subtracts — legacy and alpha features the edge doesn’t need are removed. The payoff is the lower box: edge, single-node, CI, and ARM deployments where the full stack simply doesn’t fit.

Mechanical Walk-through

k3s — the dominant lightweight distribution

k3s (originated at Rancher, now SUSE; the project name is a play on “K8s” — half the letters, “five less than K8s”) is a single Go binary under ~100 MB (the 2026 builds are roughly 85 MB) that bundles essentially everything a cluster needs (k3s — Architecture, k3s — GitHub):

  • The control-plane components — kube-apiserver, kube-scheduler, kube-controller-manager — run as goroutines inside the single k3s process, not as separate Static Pods.
  • containerd is embedded as the container runtime (k3s ships it; you don’t install it separately).
  • Flannel is the bundled default CNI.
  • A CoreDNS, a Traefik ingress controller, a service-load-balancer (klipper-lb), and a local-path storage provisioner are bundled add-ons.
  • The datastore defaults to SQLite (a single file on disk) for single-server clusters, swappable for embedded etcd, or external MySQL/PostgreSQL, all through a shim called kine.

A k3s server node runs the control plane plus a kubelet; a k3s agent node runs only the kubelet and runtime. Bringing up a cluster is one binary and one command — curl -sfL https://get.k3s.io | sh - — which is why k3s dominates edge, IoT, and CI use cases.

kine — the etcd shim

The component that lets k3s run on SQLite, MySQL, or PostgreSQL is kine (“kine is not etcd”). The kube-apiserver speaks only the etcd API; it has no notion of SQL. kine is a translation layer that implements the etcd v3 gRPC API on top of a relational datastore (kine — GitHub). The API server thinks it is talking to etcd; kine maps each etcd operation onto SQL against SQLite/MySQL/Postgres. This is the single most important architectural fact about k3s: it does not change Kubernetes, it swaps the storage backend transparently. The trade-off — discussed under Failure Modes — is that kine-over-SQLite is not a distributed consensus store, so single-server k3s has no HA story without switching to embedded etcd or an external HA SQL cluster.

k3s CNCF status

k3s was accepted into the CNCF as a Sandbox project on 19 August 2020 (CNCF — k3s, CNCF TOC PR #447). As of 2026-05-24 it remains at the Sandbox tier — the CNCF project page still records only the August 2020 Sandbox acceptance with no advancement to Incubating (CNCF — k3s, verified 2026-05-24). Sandbox is the CNCF’s entry tier (experimental/early-stage), below Incubating and Graduated; a long-standing Sandbox project is not unusual, and Sandbox status is about CNCF governance maturity, not about production-readiness — k3s is widely run in production and is a CNCF-certified conformant Kubernetes distribution regardless of its maturity tier.

k0s — Mirantis’s “zero friction” distribution

k0s is a single, self-contained binary with the design goal of “zero friction” — “distributed as a single binary with zero host OS dependencies besides the kernel,” and zero required configuration to start (k0s — Documentation). Mirantis offers commercial support and sponsors the project, but k0s is developed as an open-source project under the Linux Foundation umbrella, and — a correction worth flagging against older write-ups that describe it purely as a “Mirantis distribution” — k0s was accepted into the CNCF Sandbox in early 2025 and has since formally submitted its Incubation application (CNCF — k0s in 2025, as of 2026-05-24). So both k3s and k0s are now CNCF Sandbox projects.

Architecturally k0s differs from k3s in two notable ways: k0s runs the control-plane components as regular processes (managed by k0s’s supervisor) and, importantly, does not strip Kubernetes features the way k3s removes legacy/alpha code — k0s is “Certified and 100% upstream Kubernetes,” aiming to be vanilla upstream in a convenient single binary rather than an opinionated, slimmed-down variant. For its datastore, k0s defaults to embedded etcd for multi-node clusters and SQLite for single-node clusters, and also supports MySQL and PostgreSQL via kine (k0s — Documentation); it ships Kube-Router as the default CNI, with Calico as a preconfigured alternative. The split is roughly: k3s optimizes for the smallest possible footprint and edge focus; k0s optimizes for “full, unstripped Kubernetes with zero install friction.”

MicroK8s — Canonical’s snap-packaged distribution

MicroK8s (Canonical) is distributed as a snap package — snap install microk8s — which gives it transactional install/upgrade/rollback and tight Ubuntu integration (MicroK8s — Documentation). Its defining feature is an add-on system: a minimal core cluster, with DNS, storage, ingress, the dashboard, GPU support, and an observability stack enabled on demand via microk8s enable <addon>. MicroK8s supports single-node and (via Dqlite, a distributed SQLite) multi-node HA. Its strongest fit is Ubuntu-based environments and developers already in the snap ecosystem.

Why lightweight distributions exist

The full kubeadm stack assumes a server — multiple gigabytes of RAM, a multi-core machine, a separately-managed etcd, a separately-installed runtime and CNI. That assumption breaks for:

  • Edge and IoT — gateways, retail-store servers, factory-floor devices with constrained CPU/RAM and intermittent connectivity.
  • Single-node — many real deployments need exactly one node; the full HA control plane is pure overhead.
  • CI runners — ephemeral clusters spun up per pipeline run; startup speed and small footprint matter more than features.
  • ARM hardware — Raspberry Pi clusters, ARM edge devices; lightweight distros ship ARM64/ARMv7 builds first-class.

The trade-offs

Lightweight distributions are not free wins:

  • Removed features. k3s explicitly removes legacy, alpha, and non-default in-tree features (old cloud providers, deprecated APIs) — usually irrelevant at the edge, occasionally a surprise if you depended on one. k0s deliberately does not do this.
  • Datastore fidelity. k3s’s default SQLite-via-kine is a single file, not a distributed consensus store. For HA you must move to embedded etcd or external HA SQL — the simplicity of “SQLite by default” is genuinely simpler but genuinely not HA.
  • Bundled-component lock-in. k3s bundles Flannel and Traefik; using Calico/Cilium or a different ingress means disabling the bundled component, a minor friction point.

Configuration / API Surface

# --- k3s: single-server install (control plane on SQLite) ---
curl -sfL https://get.k3s.io | sh -
# Installs the k3s binary, a systemd unit, and starts a one-node cluster
# in well under a minute. Kubeconfig lands at /etc/rancher/k3s/k3s.yaml.
 
# --- k3s: HA server with EMBEDDED etcd (replaces SQLite) ---
# First server initialises the embedded-etcd cluster:
curl -sfL https://get.k3s.io | sh -s - server \
  --cluster-init                         # use embedded etcd instead of SQLite
 
# Additional servers join the etcd quorum:
curl -sfL https://get.k3s.io | sh -s - server \
  --server https://<first-server>:6443 \
  --token <node-token>
 
# --- k3s: agent (worker) node ---
curl -sfL https://get.k3s.io | sh -s - agent \
  --server https://<server>:6443 --token <node-token>
 
# --- k3s: external-datastore HA (kine -> PostgreSQL) ---
curl -sfL https://get.k3s.io | sh -s - server \
  --datastore-endpoint="postgres://user:pw@db-host:5432/k3s"
 
# --- k0s: single binary, controller + worker ---
k0s install controller --single        # install as a systemd service
k0s start
 
# --- MicroK8s: snap install + on-demand add-ons ---
sudo snap install microk8s --classic
microk8s enable dns ingress hostpath-storage   # opt into add-ons as needed

Line-by-line. The first k3s line is the defining ergonomic — one piped command yields a running cluster on SQLite in seconds; this is what makes k3s ubiquitous in CI and edge. --cluster-init is the single flag that flips k3s from “SQLite, no HA” to “embedded etcd, HA-capable” — it is the most consequential k3s configuration decision. --datastore-endpoint is the kine shim made visible: point k3s at a Postgres URL and the API server transparently stores cluster state in SQL. The MicroK8s enable line shows its philosophy — start minimal, opt into capability — versus k3s’s “bundle sensible defaults, disable what you don’t want.”

Failure Modes

k3s single-server has no HA. Default k3s with SQLite is a single point of failure: lose the server node and the cluster is gone (agents keep running their last-known Pods briefly, then drift). Teams that need HA must use --cluster-init (embedded etcd) or an external HA datastore. “k3s is easy” can quietly mean “k3s is a single-node toy” if HA was never configured.

SQLite write contention at scale. kine-over-SQLite handles small edge clusters fine, but a single SQLite file becomes a write bottleneck as object count and churn grow. Symptom: rising API-server latency, etcd-API timeouts surfaced through kine. The fix is moving to embedded etcd or external Postgres before the cluster grows past SQLite’s comfort zone.

Bundled-component surprises. k3s ships Traefik as the ingress controller and Flannel as the CNI. A team expecting nginx-ingress, or needing NetworkPolicy (which Flannel doesn’t enforce), gets unexpected behavior until they disable the bundled component and install their own.

Edge connectivity loss. Lightweight distributions are deployed at the edge precisely where the network is unreliable. An agent node that loses connectivity to the server keeps its Pods running but cannot receive new work or report status — and on reconnect must reconcile. For genuinely disconnected operation, KubeEdge’s eventual-connectivity model is the better fit than a plain k3s agent.

Removed feature you depended on. A workload that relied on a legacy in-tree provider or a deprecated API removed from k3s fails on k3s but works on upstream. The fix is k0s (which doesn’t strip features) or upstream Kubernetes; the prevention is checking k3s’s removed-feature list before migrating.

Alternatives and When to Choose Them

Local Kubernetes Distributions (kind, minikube, k3d). The sibling category for the developer laptop and CI, not production edge. k3d in particular is “k3s-in-Docker” — it uses k3s as its engine but targets local development. Choose local distros for the dev loop; choose lightweight distros for production edge/single-node.

KubeEdge. Purpose-built for the edge specifically — it extends a central Kubernetes cluster to manage edge nodes that are eventually connected, with edge-side autonomy during disconnection. Choose KubeEdge when edge nodes are genuinely intermittently-connected and need to operate autonomously; choose k3s when the edge site is a small-but-connected cluster.

Full kubeadm / Managed Kubernetes Services. When the deployment is a real server fleet or a cloud cluster, the full stack’s features and the managed control plane are worth their weight. Lightweight distributions are an answer to a constrained-resource problem, not a general replacement.

k3s vs k0s vs MicroK8s. Choose k3s for the smallest footprint, the strongest edge/IoT ecosystem, and the simplest single-command install. Choose k0s when you want unstripped, vanilla Kubernetes in a single binary with zero host dependencies. Choose MicroK8s in Ubuntu/snap-centric environments where transactional snap upgrades and the add-on model fit the operating model.

Production Notes

k3s’s adoption in edge and IoT is its standout production story — retail chains, telco edge sites, and industrial deployments run k3s on thousands of small nodes precisely because one binary, low RAM, and a one-line install scale operationally in a way the full kubeadm stack does not. Its CI use is equally widespread: k3d (k3s-in-Docker) spins up disposable clusters per pipeline run in seconds.

The kine architecture deserves emphasis as a general lesson, not just a k3s detail: because the kube-apiserver depends only on the etcd API, not on etcd itself, that API is a clean seam. kine exploits it to run Kubernetes on SQL; the same seam is what makes the etcd storage layer swappable in principle. k3s turned a theoretical pluggability into a shipped product feature.

The 2026 k3s release line tracks upstream Kubernetes closely — the January 2026 release blog covers Kubernetes v1.35 (k3s — v1.35 release) — which is the reassurance edge operators need: a lightweight distribution that nonetheless stays current with upstream and remains conformance-certified, so manifests are portable back to a full cluster.

See Also

Graph View

Backlinks