vault

Ambient Mesh

12 min read

Ambient Mesh

Ambient mesh is Istio’s sidecarless data-plane architecture — the alternative to injecting a per-Pod Envoy sidecar into every workload. Announced as an experimental “sidecarless” mode in September 2022 (by Google and Solo.io), promoted to beta in Istio 1.22, and reaching General Availability in Istio 1.24 on 7 November 2024 when the Istio Technical Oversight Committee marked ztunnel, waypoints, and the ambient APIs Stable (istio.io, cncf.io). Ambient’s defining idea is to split the mesh data plane into two independently adoptable layers: a secure overlay (L4) layer — ztunnel, a per-node DaemonSet agent that transparently provides mutual TLS, L4 authorization, and telemetry for every Pod on its node; and an optional L7 layer — waypoint proxies, namespace- or service-account-scoped Envoy deployments you add only where you need HTTP routing, L7 authorization, traffic splitting, or retries. The motivation is the sidecar tax: a sidecar per Pod multiplies memory, CPU, startup-ordering bugs, and Job-completion problems by every Pod in the fleet (Sidecar Sprawl Anti-Pattern). Ambient removes the per-Pod proxy entirely for L4 and amortizes L7 cost across a shared proxy. This note covers the two-layer architecture, the HBONE tunnel protocol, ztunnel and waypoint mechanics, incremental adoption, the security trade-off of a shared node proxy, and GA status.

Mental Model

flowchart TB
    subgraph N1["Node 1"]
        subgraph PA["Pod A (no sidecar)"]
            APPA[app container only]
        end
        ZT1["ztunnel<br/>per-node DaemonSet (Rust)<br/>L4 mTLS · L4 authz · telemetry"]
        APPA -- "redirected by CNI" --> ZT1
    end
    subgraph N2["Node 2"]
        subgraph PB["Pod B (no sidecar)"]
            APPB[app container only]
        end
        ZT2["ztunnel<br/>per-node DaemonSet"]
        ZT2 -- "redirected" --> APPB
    end
    WP["Waypoint proxy<br/>per-namespace / per-SA Envoy<br/>L7 routing · L7 authz · retries<br/>(deployed only where needed)"]
    ISTIOD["istiod control plane<br/>xDS + CA"]

    ZT1 == "HBONE tunnel<br/>(mTLS over HTTP/2 CONNECT, port 15008)" ==> ZT2
    ZT1 -. "if namespace has a waypoint:<br/>route L4→waypoint→dest" .-> WP
    WP == HBONE ==> ZT2
    ISTIOD -. xDS .-> ZT1
    ISTIOD -. xDS .-> ZT2
    ISTIOD -. xDS .-> WP

What this diagram shows and the insight to extract. Pods A and B run no sidecar — just the application container. On each node a ztunnel DaemonSet Pod handles all mesh L4 work for every Pod on that node. The node CNI redirects each Pod’s traffic into the local ztunnel. ztunnel-to-ztunnel traffic rides an HBONE tunnel — mutual TLS carried inside an HTTP/2 CONNECT stream on port 15008 — so every hop is encrypted and identity-authenticated without any per-Pod proxy. The waypoint proxy (a normal Envoy, deployed per namespace or per service account, only when L7 features are wanted) is inserted into the path: ztunnel routes traffic destined for a waypointed service through the waypoint, which applies HTTP routing, L7 authorization, traffic splitting, and retries. The insight: L4 security is now “ambient” — always on, free of per-Pod cost — while L7 is pay-as-you-go, added surgically. The two layers are independently adoptable: a team can get zero-trust mTLS for an entire namespace by labelling it, with zero Pod restarts and zero L7 proxies, and only later add a waypoint for the one service that needs canary routing.

Mechanical Walk-through

Layer 1 — the secure L4 overlay (ztunnel)

ztunnel (“zero-trust tunnel”) is a purpose-built, Rust-written, per-node proxy deployed as a DaemonSet (istio.io ambient architecture). It is intentionally minimal — it does only L4: mutual TLS, L4 authorization policy, and L4 telemetry. It deliberately does not parse HTTP. This narrow scope is a security choice: a smaller proxy has a smaller attack surface, and an L4-only proxy never buffers or interprets application payloads.

When a meshed Pod on a node sends traffic, the node’s CNI redirects it to the local ztunnel. ztunnel looks up the destination, establishes (or reuses) an HBONE tunnel to the destination node’s ztunnel, and forwards. The destination ztunnel terminates the tunnel and delivers plaintext to the destination Pod. Both ztunnels present the SPIFFE workload identity of the Pods they front (see mTLS in Service Mesh) — identity is per-workload even though the proxy is per-node, because ztunnel multiplexes many workloads’ identities.

HBONE — the tunnel protocol

HBONE (“HTTP-Based Overlay Network Environment”) is ambient’s transport. It wraps the original connection inside an HTTP/2 CONNECT stream protected by mTLS, on a fixed port (15008). Using HTTP/2 CONNECT (a standard mechanism) means the tunnel is a well-understood, multiplexable, firewall-friendly construct rather than a bespoke protocol. Many workloads’ connections multiplex over shared HBONE tunnels between a pair of nodes.

Layer 2 — the L7 layer (waypoint proxies)

A waypoint proxy is a standard Envoy deployment that provides Istio’s full L7 feature set: HTTP routing (VirtualService), L7 AuthorizationPolicy, traffic splitting/canary, retries, timeouts, fault injection, and L7 telemetry (istio.io waypoint). Unlike a sidecar, a waypoint is not per Pod — it is deployed per namespace (default) or per service account, and you create one only for the services that need L7 features. A waypoint is a normal Deployment (created via the Gateway API Gateway resource with the istio-waypoint class) — so it can be scaled, scheduled, and HA’d independently of the workloads it serves.

When a namespace has a waypoint, ztunnel routes traffic to that namespace’s services through the waypoint: source-ztunnel → waypoint → destination-ztunnel → Pod. Namespaces without a waypoint get L4-only treatment and never pay the L7 cost.

Incremental adoption

This is ambient’s headline operational property. The adoption path:

  1. Install ambient — istiod, ztunnel DaemonSet, the CNI node-agent. No workload changes.
  2. Add a namespace to the mesh — label it istio.io/dataplane-mode: ambient. Every Pod in it immediately gets mTLS, L4 authz, and telemetry — with no Pod restart. (A sidecar mesh requires restarting every Pod to inject the sidecar.)
  3. Add a waypoint — only for namespaces/services that need L7. Create a Gateway of class istio-waypoint. Still no workload restart.

L4 first, L7 surgically, no restarts — versus the sidecar model’s all-or-nothing per-Pod injection.

What ambient eliminates

The GA announcement quantified it: one organization removed sidecars and the SPIRE agent DaemonSet and cut ~45% of running containers (istio.io GA blog). The per-Pod sidecar problems that vanish:

  • Resource tax — memory/CPU per Pod, multiplied fleet-wide.
  • Startup ordering — the sidecar-race-with-the-app problem (app connects before sidecar is ready) is gone; there is no sidecar to race.
  • Job/CronJob completion — a sidecar that never exits keeps a Job Pod “running” forever; a classic sidecar foot-gun, eliminated.
  • Lifecycle coupling — upgrading the mesh no longer means restarting every workload Pod to re-inject.

Configuration / API Surface

Enrolling a namespace into the L4 layer

apiVersion: v1
kind: Namespace
metadata:
  name: shop
  labels:
    istio.io/dataplane-mode: ambient        # (1) L4 overlay ON for every Pod here
  1. This single label enrolls every Pod in shop into the ztunnel L4 overlay — mTLS, L4 authz, telemetry — with no Pod restart. This is the entire L4 onboarding step.

Adding a waypoint (L7 layer) for one namespace

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: shop-waypoint
  namespace: shop
  labels:
    istio.io/waypoint-for: service          # (2) waypoint handles traffic TO services in this ns
spec:
  gatewayClassName: istio-waypoint          # (3) the special class that makes this a waypoint
  listeners:
  - name: mesh
    port: 15008                             # (4) HBONE port
    protocol: HBONE
  1. istio.io/waypoint-for: service scopes the waypoint to traffic destined for services in the namespace (other scopes: workload, all, none).
  2. gatewayClassName: istio-waypoint is what turns an ordinary Gateway API Gateway into a waypoint-proxy Deployment — ambient reuses Gateway API rather than inventing a CRD.
  3. The waypoint listens for HBONE-tunnelled traffic on 15008.

An L7 policy that requires the waypoint

apiVersion: networking.istio.io/v1
kind: VirtualService                        # (5) L7 routing — only takes effect WITH a waypoint
metadata:
  name: cart-canary
  namespace: shop
spec:
  hosts: [cart]
  http:
  - route:
    - destination: { host: cart, subset: v1 }
      weight: 90
    - destination: { host: cart, subset: v2 }
      weight: 10
  1. This 90/10 canary is an L7 feature — it is enforced by the waypoint. If shop has no waypoint, this VirtualService has no effect: ztunnel alone cannot split HTTP traffic. This is the concrete meaning of “L7 is pay-as-you-go” — you must deploy a waypoint to get it.

Failure Modes

  1. ztunnel as a per-node blast radius. ztunnel handles every meshed Pod on its node. A ztunnel crash, OOM, or bug disrupts mesh traffic for all of them — a wider blast radius than a single Pod’s sidecar. Diagnostic: ztunnel DaemonSet Pod health, istioctl ztunnel-config. Mitigation: ztunnel is deliberately small/Rust for stability; size node memory for it.
  2. L7 policy silently inert without a waypoint. A VirtualService or L7 AuthorizationPolicy applied to a namespace with no waypoint simply does nothing — no error, no traffic effect. Diagnostic: check the namespace has a waypoint Gateway; istioctl analyze. This is the most common ambient surprise.
  3. CNI redirection failure. Ambient relies on the Istio CNI node-agent to redirect Pod traffic to ztunnel. If the CNI agent fails on a node, Pods there fall out of the mesh (no mTLS) or lose connectivity. Diagnostic: Istio CNI DaemonSet logs.
  4. Waypoint as a single point of failure / bottleneck. A per-namespace waypoint handling all L7 traffic for that namespace can be overwhelmed or be a SPOF if run single-replica. Mitigation: scale the waypoint Deployment; it is HA’d like any Deployment.
  5. Mixed sidecar + ambient transition bugs. During migration a mesh may have some namespaces on sidecars and some on ambient; traffic crossing the boundary has edge cases. Mitigation: Istio supports interop, but migrate namespace-by-namespace and test boundaries.
  6. HBONE port (15008) blocked. A restrictive NetworkPolicy or firewall blocking 15008 between nodes breaks all ambient traffic. Diagnostic: connectivity tests on 15008.

Alternatives and When to Choose Them

  • Istio sidecar mode — the classic per-Pod Envoy. Choose it when you need strong per-Pod isolation as a security boundary (a compromised app container cannot reach the sidecar’s keys; a compromised ztunnel exposes every Pod on the node), or for features not yet on ambient parity. The cost is the full sidecar tax.
  • Cilium Service Mesh — also sidecarless, but architecturally different: Cilium pushes L3/L4 (routing, LB, policy) into eBPF in the kernel and uses a per-node Envoy for L7, whereas ambient uses a userspace Rust ztunnel for L4 and per-namespace Envoy waypoints for L7. Choose Cilium when you want the mesh fused with the CNI and kernel-level L4; choose ambient when you want Istio’s mature L7 API surface and CRD ecosystem. See Cilium Service Mesh for the contrast.
  • Linkerd — Linkerd remains a per-Pod sidecar mesh (its Rust micro-proxy is already tiny, ~10–20 MB, so the tax is smaller). Choose Linkerd for operational simplicity and a minimal feature surface; choose ambient to eliminate the per-Pod proxy entirely.
  • No mesh — for a small fleet, library-based resiliency and manual TLS may be cheaper than running ztunnel + istiod. Mesh value (and ambient’s tax reduction) compounds with fleet size.

Production Notes

  • GA in Istio 1.24, 7 Nov 2024 (istio.io) — ztunnel, waypoints, and the ambient APIs marked Stable by the Istio TOC after ~26 months of development with contributions from Solo.io, Google, Microsoft, Intel, IBM, Red Hat, and others. Beta was Istio 1.22.
  • Adoption signal — the ztunnel image passed 1 million Docker Hub pulls by GA; the cited ~45% container reduction (sidecars + SPIRE agents removed) is the headline operational win.
  • Waypoint = Gateway API — ambient deliberately reuses Gateway API for waypoint definition rather than inventing a CRD; an Envoy Gateway can serve as a unified ingress gateway and ambient waypoint (cncf.io 2025).
  • The security trade-off is real and debated. A per-Pod sidecar gives a clean isolation boundary — compromising an app container does not compromise the sidecar. ztunnel is shared per node, so a ztunnel compromise affects every meshed Pod on that node. Ambient mitigates by keeping ztunnel small and L4-only (no payload parsing), but teams with strict per-workload isolation requirements may still prefer sidecars. This is the central ambient trade-off.
  • Multi-cluster ambient — beta in Istio 1.29 (Feb 2026). Ambient multi-network multicluster support was promoted from alpha to beta in Istio 1.29.0, released 18 February 2026 (istio.io blog). The 1.29 release closed the central telemetry gap of the alpha — previously, peer source/destination labels showed as unknown across networks because ztunnels lacked a way to exchange workload identity metadata over multi-network HBONE; 1.29 enriches HBONE with W3C Baggage headers to carry peer metadata cross-network, gated by the AMBIENT_ENABLE_BAGGAGE feature flag. Ingress gateways and waypoints can now route directly to remote-cluster workloads. Single-network multicluster ambient remains in alpha as of 1.29. The east-west gateway has an endpoint-stickiness limitation that affects both ambient and sidecar modes, signaling ongoing parity work rather than full equivalence. Istio 1.30 (released 18 May 2026) is primarily a security release on top of these multicluster changes. So the practical state as of 2026-05-30: single-cluster ambient is GA (since 1.24); multi-network multicluster ambient is beta (since 1.29); single-network multicluster ambient is alpha.
  • L7 feature parity with sidecar — not fully closed. As of 1.30, ambient supports the core Istio L7 surface (VirtualService routing, AuthorizationPolicy, retries, timeouts, fault injection, traffic splitting) via the waypoint, but a small set of advanced sidecar-mode features still lag. Teams adopting ambient should verify their specific advanced features against the ambient docs for the Istio version in use.

See Also

Graph View

Backlinks