vault

Amazon EKS

11 min read

Amazon EKS

Amazon Elastic Kubernetes Service (EKS) is AWS’s managed Kubernetes offering (Amazon EKS docs). AWS runs the control plane — the API server, etcd, scheduler, controller-manager — across multiple Availability Zones on AWS-operated infrastructure you never see, with a published 99.95% control-plane SLA; you choose the Kubernetes version, AWS executes the upgrade. EKS charges a flat per-cluster-hour fee of 0.60/hour (0.50 extended-support premium) once a cluster’s minor version exits standard support and enters extended support, plus the worker compute (EKS Pricing). EKS’s defining characteristic is deep AWS integration: worker nodes are EC2 instances or Fargate tasks, the default CNI is the AWS VPC CNI (which gives Pods real VPC IP addresses), workload-to-AWS-service identity flows through IAM via IRSA and EKS Pod Identity, and LoadBalancer/Ingress objects provision real Elastic Load Balancers. EKS also reaches beyond the AWS cloud — EKS Anywhere runs the same distribution on-prem, and EKS Distro is the open-source build of the Kubernetes that EKS itself runs. Karpenter, now the leading node-autoscaler across the industry, was born inside AWS for EKS.

Mental Model

flowchart TB
    subgraph AWSMANAGED["AWS-managed (invisible to you)"]
        CP["EKS control plane<br/>API server · etcd · scheduler · CM<br/>multi-AZ · 99.95% SLA"]
    end
    EP["Cluster API endpoint + kubeconfig"]
    subgraph YOURVPC["Your VPC — your responsibility"]
        subgraph COMPUTE["Worker compute — pick one or mix"]
            MNG["Managed Node Groups<br/>(AWS-lifecycle EC2 ASGs)"]
            SELF["Self-managed nodes<br/>(your own ASG/AMI)"]
            FG["Fargate profiles<br/>(serverless Pods, per-Pod billing)"]
            AUTO["EKS Auto Mode<br/>(fully-managed, Karpenter-based)"]
        end
        CNI["AWS VPC CNI — Pods get real VPC IPs"]
        IAM["IRSA / EKS Pod Identity → AWS IAM"]
        LBC["AWS Load Balancer Controller → ELB/ALB/NLB"]
    end
    CP --- EP --- COMPUTE
    COMPUTE --- CNI --- IAM --- LBC

What this shows. The control plane is wholly AWS’s, multi-AZ, behind one endpoint. Inside your VPC sits everything you own — and the salient point is the menu of worker-compute modes (managed node groups, self-managed nodes, Fargate, Auto Mode) that you mix as needed. The three integration boxes — VPC CNI, IAM identity, Load Balancer Controller — are what make EKS feel like AWS’s Kubernetes rather than generic Kubernetes. The insight to extract: EKS’s value and its lock-in are the same thing — the seamless wiring into VPC networking, IAM, and ELB.

Mechanical Walk-through

The managed control plane. When you create an EKS cluster, AWS stands up a control plane on its own infrastructure, spread across at least two (typically three) AZs (EKS Clusters). You get an API server endpoint (public, private, or both) and a kubeconfig. Control-plane scaling and etcd backup are automatic and invisible. Version upgrades are customer-initiated: you select the next minor version and AWS performs the control-plane upgrade; you then separately upgrade your worker nodes and add-ons. AWS supports each Kubernetes minor version for a standard window and applies extended support (at additional cost) beyond it before forcing an upgrade.

Worker-node options — the four modes.

  1. Managed node groups — AWS provisions and lifecycle-manages a group of EC2 worker nodes for you: it creates the EC2 Auto Scaling group, uses an EKS-optimized AMI, and performs graceful, drain-aware rolling updates when you upgrade. You choose instance types and min/max counts (or attach the Cluster Autoscaler / Karpenter).
  2. Self-managed nodes — you build the Auto Scaling group and AMI yourself, you own the bootstrap script and OS patching. Maximum control (custom AMIs, custom kernels); maximum work.
  3. Fargate profiles — serverless Pods. A Fargate profile is a selector (namespace + labels); matching Pods run on AWS-provisioned, single-Pod micro-VMs with per-Pod billing (you pay for the vCPU/memory the Pod requests, for the time it runs). No node to manage. Constraints: no DaemonSets, no privileged Pods, no hostPath, limited to certain Pod sizes.
  4. EKS Auto Mode — AWS’s fully-managed compute mode (EKS Auto Mode, AWS Auto Mode docs). AWS automatically provisions and right-sizes EC2 instances in response to Pod demand, manages the node OS and patching, and additionally takes over compute autoscaling, Pod and Service networking with policy enforcement, Elastic Load Balancing integration, cluster DNS, block storage (EBS CSI), and GPU support as core managed components rather than as add-ons. Internally it uses Karpenter (exposed through managed NodePool and NodeClass objects you can extend with custom — never edit the default — pools/classes), and the nodes themselves run a locked-down Bottlerocket AMI variant with SELinux enforcement, a read-only root filesystem, no SSH or SSM access, and a maximum lifetime of 21 days after which Auto Mode automatically replaces the node (Auto Mode docs). You can reduce that ceiling but not raise it. Because the EKS Pod Identity Agent is built in as a core component, Auto Mode clusters do not need the Pod Identity Agent installed as an add-on. Auto Mode sits between Fargate (per-Pod, no nodes at all) and managed node groups (you still size the group): nodes exist, but their entire lifecycle is AWS’s — and beginning 22 April 2026, those Auto Mode-launched EC2 instances are hidden by default from EC2 console and list APIs to signal that they are AWS-managed resources, not customer-owned VMs (New EKS Auto Mode features blog). Recent 2026 additions include enhanced logging, custom certificate-authority bundles via certificateBundles on the NodeClass, IPv4-egress translation for IPv6 clusters, and GPU on-demand-capacity-reservation support (enhanced logging announcement).

Networking — the AWS VPC CNI. EKS’s default CNI assigns each Pod a secondary IP address from an Elastic Network Interface (ENI) on the worker node — meaning every Pod has a real, routable VPC IP, reachable directly from anything in the VPC, no overlay. The trade-off is that the number of Pods per node is capped by the instance type’s ENI/IP limits (mitigated by prefix-delegation mode). EKS also supports alternative CNIs (Cilium, Calico).

Workload identity to AWS — IRSA and EKS Pod Identity. A Pod that needs to call S3 or DynamoDB needs AWS credentials, and EKS provides two mechanisms (Workload Identity (Kubernetes)):

  • IRSA (IAM Roles for Service Accounts) — the original. EKS exposes an OIDC identity provider for the cluster; a ServiceAccount is annotated with an IAM role ARN; the Pod’s projected service-account token is exchanged (via OIDC) for AWS credentials for that role. Works on EKS, EKS Anywhere, and self-managed clusters.
  • EKS Pod Identity — the newer, AWS-recommended mechanism announced in November 2023 (EKS Pod Identity docs, launch blog). An EKS Pod Identity Agent (a DaemonSet add-on on managed/self-managed node groups, built-in as a core component on EKS Auto Mode so no separate install is needed) intercepts the Pod’s credential request and calls the EKS Auth API. It removes the per-cluster OIDC-provider setup, lets one IAM role be reused across many clusters without editing trust policies, and injects session tags (cluster, namespace, ServiceAccount, Pod) for ABAC. It is EKS-cloud-only and, because the agent is a DaemonSet on standard node groups, requires EC2/Karpenter nodes — Fargate Pods must still use IRSA.

EKS add-ons. EKS can manage the lifecycle of curated add-ons (the VPC CNI, CoreDNS, kube-proxy, EBS/EFS CSI drivers, the Pod Identity Agent) as first-class addon resources — AWS versions and upgrades them. AWS recommends Pod Identity associations to grant add-ons their IAM permissions.

Beyond the cloud — EKS Anywhere and EKS Distro. EKS Distro is the open-source Kubernetes distribution AWS builds and uses for EKS — the same patched binaries, published for anyone to run. EKS Anywhere is a deployment option built on EKS Distro for running EKS-consistent clusters on-premises (on bare metal, vSphere, etc.), optionally connected back to an AWS Region for IAM/IRSA/observability integration.

Configuration / API Surface

Creating an EKS cluster with eksctl and a managed node group:

# eksctl ClusterConfig
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: prod
  region: us-east-1
  version: "1.30"                       # the control-plane K8s version AWS runs
iam:
  withOIDC: true                        # provision the OIDC provider — enables IRSA
managedNodeGroups:
  - name: ng-general
    instanceType: m6i.large
    minSize: 3
    maxSize: 10                         # bounds for autoscaling
    volumeSize: 50
addons:
  - name: vpc-cni                       # AWS manages this add-on's lifecycle
  - name: eks-pod-identity-agent        # required for EKS Pod Identity

Associating an IAM role to a workload via EKS Pod Identity:

# Bind the ServiceAccount 'app-sa' in namespace 'prod' to an IAM role — no OIDC trust policy edits.
aws eks create-pod-identity-association \
  --cluster-name prod \
  --namespace prod \
  --service-account app-sa \
  --role-arn arn:aws:iam::111122223333:role/app-s3-access

Line-by-line: version is the control-plane version AWS upgrades on request. iam.withOIDC: true provisions the cluster’s OIDC identity provider — the prerequisite for IRSA (Pod Identity does not need it). managedNodeGroups declares an AWS-lifecycle-managed EC2 group with autoscaling bounds. The addons list hands add-on lifecycle to AWS. The create-pod-identity-association call is the modern way to grant a Pod an IAM role — note there is no trust-policy editing, the key operational win over IRSA.

Failure Modes

ENI/IP exhaustion under the VPC CNI. Because every Pod consumes a VPC IP, a small instance type runs out of Pod capacity quickly, and a busy cluster can exhaust the subnet’s IP space entirely — new Pods stay Pending. Mitigations: VPC CNI prefix-delegation mode (more IPs per ENI), larger instance types, dedicated large Pod subnets.

Forgetting Fargate’s IRSA-only constraint. Migrating workloads to EKS Pod Identity and assuming Fargate Pods come along — they do not; the Pod Identity Agent DaemonSet cannot run on Fargate. Fargate Pods silently keep needing IRSA. Symptom: Fargate Pods lose AWS access after an “IRSA-to-Pod-Identity” migration.

Control-plane version EOL forcing an upgrade. A cluster left on an old minor version eventually exits standard support; AWS charges extended-support fees, then forces the upgrade. Teams that don’t track the support calendar get surprise upgrades or surprise bills.

aws-auth ConfigMap drift (legacy). Historically, mapping IAM identities to Kubernetes users went through the aws-auth ConfigMap — a single, easily-corrupted, un-version-controlled object; a bad edit could lock everyone out of the cluster. The modern replacement is EKS Access Entries (first-class IAM-to-Kubernetes mapping). New clusters should use Access Entries, not aws-auth.

Auto Mode constraints surprise. EKS Auto Mode’s managed NodePool/NodeClass and 21-day node rotation mean nodes are ephemeral by design — workloads that assume long-lived nodes or pin to specific node identities can be disrupted by routine node replacement.

Alternatives and When to Choose Them

Google GKE. Frequently cited as the more mature managed offering, with GKE Dataplane V2 and the well-regarded Autopilot mode. Choose GKE if you are on Google Cloud or value Autopilot’s maturity; choose EKS if you are on AWS — the VPC/IAM/ELB integration is the deciding factor.

Azure AKS. Microsoft’s equivalent; choose if you are on Azure and want Entra ID (Azure AD) integration.

Self-managed Kubernetes on EC2 (Self-Managed Kubernetes). Run your own control plane on EC2 with kubeadm. Choose only for deep control-plane customization or to avoid the per-cluster fee at extreme scale — rarely worth it.

EKS compute modes relative to each other. Within EKS: choose Fargate for spiky, bursty, isolation-sensitive workloads where per-Pod billing and zero node management win; choose managed node groups when you need DaemonSets, GPUs, or steady-state cost efficiency; choose EKS Auto Mode when you want the operational simplicity of Fargate but with real nodes (DaemonSets, broader instance selection); choose self-managed nodes only when a custom AMI/kernel forces it. Karpenter can drive node provisioning under managed/self-managed modes for fast, bin-packing-aware scaling.

Production Notes

  • Karpenter is AWS’s gift to the ecosystem. Karpenter was built inside AWS for EKS as a faster, more flexible replacement for the Cluster Autoscaler — it provisions right-sized instances directly in response to unschedulable Pods, is instance-type- and bin-packing-aware, and has since gone multi-cloud. EKS Auto Mode uses Karpenter internally.
  • The AWS Load Balancer Controller is near-mandatory. It turns Ingress objects into ALBs and LoadBalancer Services into NLBs, with rich annotation-driven configuration. Most production EKS clusters install it.
  • VPC CNI’s “real Pod IPs” is a double-edged sword. Direct VPC routing simplifies connectivity to RDS, ElastiCache, and other VPC resources — but IP exhaustion is a real, recurring production incident. Plan subnet sizing and prefix delegation up front.
  • Migrate off aws-auth to Access Entries. The aws-auth ConfigMap has locked countless teams out of their own clusters. Access Entries make IAM-to-Kubernetes mapping a first-class, auditable API.
  • Prefer EKS Pod Identity for new workloads on EC2/Auto Mode nodes. It eliminates the per-cluster OIDC-provider sprawl and IAM trust-policy-size limits that bite IRSA at fleet scale — but remember Fargate still needs IRSA.

See Also

Graph View

Backlinks