vault

KEDA

11 min read

KEDA

KEDA — Kubernetes Event-Driven Autoscaling — is a CNCF graduated project (accepted 12 March 2020, incubating 18 August 2021, graduated 22 August 2023 — CNCF — KEDA) that extends Kubernetes autoscaling in two specific directions the built-in Horizontal Pod Autoscaler cannot reach. First, it adds external event sources as scaling triggers: instead of scaling on CPU and memory, KEDA can scale a Deployment on Kafka consumer lag, RabbitMQ queue depth, AWS SQS message count, a Prometheus query result, a cron schedule, Azure Service Bus, Redis list length, NATS, and ~70 other sources (KEDA — Scalers). Second — and uniquely among Kubernetes autoscalers — it supports scale-to-zero: when there is no work, KEDA scales a workload down to zero replicas, and back up to one when work arrives. KEDA does not replace the HPA; it wraps and feeds it. KEDA owns the 0↔1 transition (the activation problem, which the HPA structurally cannot solve) and delegates the 1↔N transition (the scaling problem) to a standard HPA that KEDA itself creates and configures. The mental shorthand is “HPA on steroids”: the HPA’s proportional-control math, unchanged, now driven by event metrics it could never previously see.

Mental Model

The HPA has two structural gaps. It can only consume metrics already exposed through the Kubernetes metrics APIs (so external queue depth is invisible to it without an adapter), and it cannot scale to or from zero — at zero replicas there are no Pods, hence no resource metrics, hence nothing for the HPA’s proportional formula to act on. KEDA plugs both gaps. It runs an operator that translates a user-friendly ScaledObject CRD into a managed HPA, and it runs a metrics adapter that registers as an external.metrics.k8s.io provider so the HPA can read event-source metrics as if they were native.

flowchart TB
    SRC[(External event source:<br/>Kafka, SQS, RabbitMQ,<br/>Prometheus, cron, ...)]
    SO[ScaledObject CR<br/>target workload + triggers]
    OP[KEDA Operator] -->|"creates & owns"| HPA[HPA<br/>auto-generated]
    SO --> OP
    OP -->|"polls scaler<br/>for activation"| SRC
    OP -. "0 → 1 and 1 → 0<br/>(activation)" .-> WL[Deployment /<br/>StatefulSet]
    ADAPTER[KEDA Metrics Adapter<br/>external.metrics.k8s.io] -->|"queries scaler"| SRC
    HPA -->|"reads metric via<br/>external metrics API"| ADAPTER
    HPA -. "1 → N and N → 1<br/>(scaling)" .-> WL

What this diagram shows. A ScaledObject names a target workload and one or more triggers. The KEDA operator does two jobs: it polls the event source to decide whether any work exists (the activation decision, which drives 0↔1), and it creates a standard HPA for the 1↔N range. The KEDA metrics adapter is a separate component registered with the API aggregation layer as an external.metrics.k8s.io server; the HPA reads the event metric through the adapter exactly as it would read any external metric. The insight to extract: KEDA splits autoscaling at the boundary at zero. Below one replica, only KEDA’s operator can act (the HPA is structurally blind). At one replica and above, the ordinary HPA does the proportional math — KEDA’s only contribution there is feeding it the metric.

Mechanical Walk-through

Activation vs. scaling

This split is the conceptual core. KEDA distinguishes:

  • Activation — the decision to go from 0 → 1 (start the first Pod) or 1 → 0 (stop the last Pod). This is KEDA’s job. The KEDA operator polls each scaler; if a scaler reports any work above its activationThreshold, the operator scales the workload to one replica. Once at zero, the HPA has nothing to act on, so KEDA must own this transition.
  • Scaling — the decision to go from 1 → N and back to 1. This is delegated to a standard HPA that KEDA generates. The HPA reads the event metric through KEDA’s metrics adapter and applies its normal proportional-control formula. KEDA does not reimplement the HPA’s math; it reuses it.

When a ScaledObject is created, KEDA’s operator creates a matching HPA behind the scenes (named keda-hpa-<scaledobject-name>). Users should never edit that HPA directly — it is owned and reconciled by KEDA.

Scalers — the trigger ecosystem

A scaler is a connector that knows how to query one external system for a metric. KEDA ships ~70 built-in scalers (KEDA — Scalers), spanning:

  • Message queues / streams — Apache Kafka (consumer lag), RabbitMQ (queue length), AWS SQS, Azure Service Bus, Google Pub/Sub, NATS JetStream, Redis lists/streams.
  • Metrics systems — Prometheus (an arbitrary PromQL query), Datadog, New Relic, Azure Monitor, AWS CloudWatch.
  • Schedules — Cron (scale up during business hours, to zero overnight).
  • Databases / storage — PostgreSQL, MySQL, MongoDB, Elasticsearch, Azure Blob Storage.
  • HTTP — the KEDA HTTP add-on scales on incoming request rate (the closest analog to serverless request-driven autoscaling).

Each scaler has a type and a metadata block. A workload can list multiple triggers; KEDA scales to satisfy the most demanding of them.

ScaledObject vs. ScaledJob

KEDA exposes two CRDs:

  • ScaledObject — wraps a long-running workload (Deployment or StatefulSet, or any custom resource with a /scale subresource). It scales the replica count, including to/from zero.
  • ScaledJob — for batch work: instead of scaling replicas of a long-lived Pod, it creates a new Kubernetes Job per unit of work (e.g., one Job per SQS message batch). The Job runs to completion and exits. This suits work where each item is a discrete, finite task rather than a stream serviced by a persistent consumer.

The metrics adapter and the aggregation layer

The HPA cannot query Kafka or SQS directly — it only speaks the Kubernetes metrics APIs. KEDA’s metrics adapter bridges this: it registers an APIService for external.metrics.k8s.io with the API Aggregation Layer, so the kube-apiserver proxies external-metric requests to it. When the HPA asks “what is the current value of the kafka external metric?”, the request is routed to KEDA’s adapter, which invokes the relevant scaler, queries Kafka for consumer lag, and returns the number. From the HPA’s perspective it is consuming an ordinary external metric — see Metrics Pipeline for Autoscaling for the three metrics APIs.

Configuration / API Surface

A ScaledObject scaling a Deployment on Kafka consumer lag, with scale-to-zero:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-consumer
  namespace: orders
spec:
  scaleTargetRef:
    name: order-consumer            # the Deployment to scale (same namespace)
  minReplicaCount: 0                # scale-to-zero: idle → 0 Pods
  maxReplicaCount: 30
  cooldownPeriod: 300               # wait 5 min of no activity before going 1 → 0
  pollingInterval: 15               # operator polls the scaler every 15 s
  idleReplicaCount: 0               # optional: distinct idle count (must be < minReplicaCount logic)
  advanced:
    horizontalPodAutoscalerConfig:
      behavior:                     # passed straight through to the generated HPA
        scaleDown:
          stabilizationWindowSeconds: 120
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: kafka.orders.svc:9092
        consumerGroup: order-consumers
        topic: orders
        lagThreshold: "50"          # target: ~50 lagging messages per replica
        activationLagThreshold: "1" # 0 → 1 as soon as lag exceeds 1

Line-by-line:

  • scaleTargetRef — names the workload; KEDA creates the HPA against it.
  • minReplicaCount: 0 — the headline feature: when idle, the Deployment drops to zero Pods. A plain HPA cannot do this (its minReplicas floor is 1).
  • cooldownPeriod: 300 — after the last activity, KEDA waits 5 minutes before the 1 → 0 transition, so brief lulls don’t cause cold-start churn.
  • pollingInterval: 15 — how often the operator polls scalers for the activation decision (distinct from how often the HPA reads the metric).
  • lagThreshold: "50" — the HPA’s target value: KEDA aims for ~50 lagging Kafka messages per replica, so 500 lag → 10 replicas.
  • activationLagThreshold: "1" — the activation threshold owned by KEDA: any lag above 1 wakes the workload from zero.
  • advanced.horizontalPodAutoscalerConfig.behavior — KEDA passes HPA behavior tuning straight through to the generated HPA, so HPA-level scale-down stabilization still applies in the 1↔N range.

Failure Modes

Workload never wakes from zero. Usually the activationThreshold is set too high, or the scaler cannot reach the event source (bad bootstrapServers, missing credentials in the referenced TriggerAuthentication). Check the KEDA operator logs and the ScaledObject status conditions.

Cold-start latency on the 0 → 1 transition. At zero replicas, the first request must wait for a Pod to schedule, pull an image, and pass readiness — seconds to minutes. Scale-to-zero trades steady-state cost for first-request latency; for latency-critical paths, set minReplicaCount: 1 and accept the idle cost, or pre-warm.

Editing the generated HPA directly. The keda-hpa-* HPA is reconciled by KEDA; manual edits are reverted on the next reconcile. All HPA tuning must go through the ScaledObject’s advanced.horizontalPodAutoscalerConfig.

HPA + KEDA fighting another autoscaler. A workload must not be targeted by both a hand-written HPA and a KEDA ScaledObject — two controllers writing the same /scale subresource oscillate. KEDA’s HPA must be the only one.

Metrics adapter unavailable. If the KEDA metrics adapter Pod is down, the HPA loses its external metric and stops scaling in the 1↔N range (it typically holds the last replica count). The adapter is a single point of dependence for the scaling half.

Scaler rate-limiting the event source. A short pollingInterval against an external API (e.g., CloudWatch, Datadog) can hit the provider’s rate limits or incur cost. Widen pollingInterval for expensive sources.

Alternatives and When to Choose Them

  • Horizontal Pod Autoscaler alone — sufficient when CPU/memory (or a custom metric tied to a K8s object) is the right scaling signal and the workload never needs to reach zero. Choose plain HPA for steady, CPU-bound services.
  • KEDA — choose when the scaling signal lives in an external system (queue depth, stream lag, cron) and/or the workload should scale to zero when idle (event-driven consumers, batch processors, dev environments). KEDA is the standard way to make a Kafka/SQS consumer fleet right-sized.
  • Knative Serving — request-driven autoscaling with scale-to-zero for HTTP workloads, with its own routing and revision model. Choose Knative for a full serverless platform on Kubernetes; choose KEDA when you want to keep ordinary Deployments and only add event-driven scaling. (KEDA’s HTTP add-on narrows the gap for the HTTP case.)
  • prometheus-adapter + HPA — exposes Prometheus metrics as custom/external metrics for the HPA without KEDA’s operator, but cannot scale to zero and lacks the 70-scaler ecosystem. Choose only if Prometheus is the sole source and zero-scale is not needed.
  • Vertical Pod Autoscaler — orthogonal: it resizes Pods rather than counting them. Composes with KEDA on different axes (with care).

Production Notes

  • KEDA is add-on, not core — it is installed (Helm chart or operator) and runs two Deployments in its own namespace: the operator and the metrics apiserver/adapter. AKS offers KEDA as a managed add-on (Microsoft Learn — KEDA on AKS).
  • Scale-to-zero is the most-cited reason teams adopt KEDA: event-driven consumer fleets (order processors, image-resize workers, ETL stages) spend most of their time idle; dropping to zero replaces a permanently-running fleet with one that costs nothing between bursts.
  • Pair KEDA with the node layer. KEDA scaling consumers from 0 to 30 produces 30 Pending Pods; those Pods then drive Cluster Autoscaler or Karpenter to add nodes. KEDA scales Pods, not machines — the two layers compose.
  • TriggerAuthentication / ClusterTriggerAuthentication are companion CRDs that hold scaler credentials (Kafka SASL, SQS IAM, Prometheus tokens) separately from the ScaledObject, so secrets are not inlined into the scaling spec.

Version and scaler count (as of 2026-05)

KEDA’s CNCF graduated status and milestone dates (accepted 12 Mar 2020, incubating 18 Aug 2021, graduated 22 Aug 2023) are confirmed against the CNCF project page. The current stable release is 2.19.0 (released February 2026); KEDA ships a new minor roughly every four months, supporting the latest two minors, so 2.17 is already end-of-life (endoflife.date/keda, kedacore/keda releases). The exact scaler count is a moving target — the project advertises “70+” built-in scalers and the 2.19 scalers index lists on the order of 70–80; treat any precise figure as version-dependent and check the index for your installed version.

See Also

Graph View

Backlinks