Autoscaling Groups and Managed Instance Groups

An Auto Scaling Group (ASG) — Google’s Managed Instance Group (MIG), Azure’s Virtual Machine Scale Set (VMSS) — is the cloud primitive that maintains a fleet of identical virtual machines against a declared desired count, launching replacements when instances die and adding or removing instances as load changes. It is a control loop: you declare a template plus a min/desired/max, and the service continuously drives reality toward that target — terminating an unhealthy instance and launching a fresh one to keep the number fixed, and adjusting the desired count according to scaling policies. AWS states the core plainly: the group launches enough instances to meet desired capacity and “maintains this number of instances by performing periodic health checks”, terminating an unhealthy instance and launching a replacement (AWS ASG docs). The same shape appears in GCP MIGs (GCP docs) and Azure VMSS (Azure docs).

This note teaches the cloud primitive — the maintain-desired-count machine and its knobs. The operational decision discipline — which metric to scale on, how to pick targets, how to avoid flapping — lives in Autoscaling in Practice under the Site Reliability Engineering MOC and is cross-linked, not duplicated. The container-level analogue of this loop (scaling pods, not VMs) is the Horizontal Pod Autoscaler in Kubernetes MOC; this note is the VM-fleet layer beneath it.

Mental Model — A Thermostat for a Fleet of Servers

The right mental image is a thermostat: you set a target, and a controller continuously nudges the system toward it. AWS uses exactly this analogy for target-tracking — you pick a value and the group “does the rest” (AWS dynamic scaling docs). The group has three numbers that bound it:

• min — the floor; it never scales below this.
• desired — the number it currently wants (scaling policies move this).
• max — the ceiling; it never scales above this.

Everything the group does is an attempt to make actual healthy instance count == desired capacity, with desired clamped into [min, max].

stateDiagram-v2
    [*] --> Launching: create group<br/>desired = N
    Launching --> Steady: N healthy instances running

    Steady --> Replacing: health check FAILS<br/>on an instance
    Replacing --> Steady: terminate unhealthy +<br/>launch replacement<br/>(count restored to desired)

    Steady --> ScalingOut: scaling policy raises<br/>desired capacity
    ScalingOut --> Cooldown: launch new instances<br/>(clamped ≤ max)
    Cooldown --> Steady: warm-up / cooldown elapses

    Steady --> ScalingIn: scaling policy lowers<br/>desired capacity
    ScalingIn --> Cooldown: terminate instances<br/>(clamped ≥ min)

    Steady --> Refreshing: instance refresh /<br/>rolling update triggered
    Refreshing --> Steady: replace instances in batches<br/>with new template

What it shows and the insight to take: there is only one job — keep healthy count equal to desired — reached through three kinds of transition: self-healing (replace a failed instance), scaling (a policy changes desired, clamped into [min, max]), and updating (roll the whole fleet onto a new template). The min/max bounds are the guardrails that keep a runaway metric or a bad policy from scaling to zero or to bankruptcy. Every feature in the rest of this note is one of these three transitions.

Mechanical Walk-through

The Launch Template / Instance Template

The fleet is identical because every instance is stamped from one blueprint. On AWS this is a launch template (the successor to the older launch configuration): it pins the Amazon Machine Image (AMI), instance type, key pair, security groups, IAM instance profile, and user-data (AWS ASG docs). On GCP it is an instance template defining machine type, image, and network settings, and it is required for a MIG (GCP docs). On Azure the VMSS model is created from a single base OS image so that “all VM instances are created from the same base OS image and configuration” (Azure docs). Templates are typically immutable and versioned — you don’t edit running instances, you cut a new template version and roll the fleet onto it (see Mutable versus Immutable Infrastructure). This is the whole reason the “identical fleet” abstraction holds.

Maintaining Desired Count and Health-Based Replacement

The group launches enough instances to hit desired capacity, then runs periodic health checks. If an instance fails, the group terminates it and launches a replacement to restore the count (AWS ASG docs). The health signal can come from several sources: the cloud’s own instance status (is the VM/hypervisor healthy), a load-balancer health check (is the application answering on its port/path), or a custom signal. GCP calls the recreate-on-failure behaviour autohealing, and stresses that an application-based health check verifies the app is responding, not merely that the VM booted (GCP docs) — a critical distinction, because a VM can be “running” while the process inside is hung. Azure VMSS calls it automatic instance repair, gated on an application health probe.

Uncertain

Verify: the precise health-check grace period / unhealthy-threshold defaults for each provider (e.g., how many consecutive failures before replacement, and the initial grace window). Reason: these numeric defaults were not fetched per-provider in this note and differ across ASG/MIG/VMSS and across health-check types. To resolve: read each provider’s health-check reference at write time. The mechanism (fail → terminate → relaunch) is well-sourced; the exact thresholds are point-in-time (as of 2026-07-24). #uncertain

Distribution Across Availability Zones

A fleet exists partly for high availability, so the group spreads instances across failure domains. AWS distributes desired capacity across the Availability Zones you specify and rebalances to keep them even after scaling actions (AWS ASG docs). GCP offers regional MIGs that spread instances across multiple zones in a region (up to 2,000 VMs) versus zonal MIGs in a single zone (up to 1,000 VMs) (GCP docs). Azure VMSS spreads VMs across Availability Zones or fault domains, and Microsoft is explicit that a scale set alone doesn’t protect against a datacenter failure — you must spread across zones for that (Azure docs). See Regions Availability Zones and Edge Locations.

Load-Balancer Integration

An autoscaling fleet is almost always fronted by a managed load balancer, and the two are wired together: as the group launches instances it registers them with the load balancer’s target pool/backend, and as it terminates them it deregisters them; the load balancer’s health check can double as the group’s health signal. AWS registers ASG instances with Elastic Load Balancing target groups (using IP or instance targets); Azure VMSS integrates with the Azure Load Balancer (L4) and Application Gateway (L7) (Azure docs); GCP MIGs act as backends for its load balancers and can even autoscale on serving capacity. This closes the loop: traffic only reaches healthy, registered instances.

Scaling Policies — How Desired Capacity Moves

The group’s desired count changes by one of four methods. AWS’s taxonomy is the clearest and maps closely onto the other clouds (AWS scaling docs):

Policy	How it decides	Best for	Analogy
Manual	You set desired directly	One-off changes, testing	Turning the dial by hand
Scheduled	Change desired at a set time/recurrence	Known daily/weekly patterns (business hours)	A programmable timer
Target tracking	Hold a metric at a target value; the service computes the count	The common case — CPU% or requests/target	A thermostat
Step scaling	Add/remove N based on how far a CloudWatch alarm is breached	Fine-grained, breach-size-aware response	Graduated dimmer
Simple scaling	One fixed adjustment per alarm, with a cooldown	Legacy / simple cases	On-off switch with a rest
Predictive	Forecast load from history; pre-scale ahead of it	Recurring, forecastable demand	Pre-heating the oven

Target tracking is the recommended default. You choose a metric that moves inversely with capacity — double the fleet and the metric roughly halves — so the data can drive proportional scaling (AWS dynamic scaling docs). Average CPU utilization or average request-count-per-target are the canonical choices. The service manages the underlying CloudWatch alarms for you; you just name a metric and a target. The metric is an aggregate across all instances — two instances at 60% and 40% CPU present as 50% — and when the alarm breaches, the group recomputes desired up or down (AWS dynamic scaling docs).

Step scaling reacts to how badly a threshold is breached: a small breach adds one instance, a large breach adds several, via step adjustments (AWS dynamic scaling docs). Simple scaling makes a single adjustment and then waits out a cooldown before acting again. Predictive scaling uses historical data to forecast demand and provision capacity in advance of the predicted spike — valuable for daily cycles where reactive scaling would always lag the ramp.

A subtle but important rule: when multiple policies fire at once, AWS takes the policy that yields the largest capacity for both scale-out and scale-in, to avoid removing too many instances (AWS dynamic scaling docs). And whatever a policy computes, the result is always clamped into [min, max] — a policy that “adds 3” to a group one below its max of 3 adds only 1 (AWS dynamic scaling docs).

Cooldowns and Warm-Up

Scaling too eagerly causes flapping — launching then terminating in a tight oscillation. Two dampers exist. A cooldown (simple scaling) pauses further scaling activity after one action so its effect can register. A warm-up / default instance warmup tells the group how long a new instance takes to boot and become useful, so its not-yet-warm metrics don’t trigger further scale-out (AWS scaling docs). The decision of how to tune these to avoid oscillation is the SRE concern in Autoscaling in Practice.

flowchart LR
    M["CloudWatch / Cloud Monitoring<br/>metric (avg CPU%, req/target)"] --> ALARM{"metric vs<br/>target/threshold?"}
    ALARM -- "above" --> OUT["compute higher desired<br/>→ launch instances"]
    ALARM -- "at target" --> HOLD["hold — no change"]
    ALARM -- "below" --> IN["compute lower desired<br/>→ terminate instances"]
    OUT --> CLAMP["clamp into [min, max]"]
    IN --> CLAMP
    CLAMP --> WARM["cooldown / warm-up<br/>(dampen flapping)"]
    WARM --> M

What it shows and the insight to take: target tracking is a closed feedback loop — measure, compare to target, adjust, clamp, wait, remeasure. The clamp and the cooldown/warm-up are the two safety elements; remove them and the loop either runs to a bound or oscillates. This is why “just autoscale on CPU” is naive — the tuning of target, bounds, and dampers is the real work.

Instance Refresh and Rolling Replacement — Immutable Updates

The fleet’s identity is defined by its template, so updating the fleet means replacing every instance with one built from the new template — you never mutate a running instance. This is the operational realization of Mutable versus Immutable Infrastructure.

• AWS Instance Refresh rolls out a new launch-template version (new AMI, user-data, or instance type) by replacing instances in batches (AWS docs). You bound disruption with a minimum healthy percentage (how much capacity must stay up during the roll) and a maximum healthy percentage, add checkpoints to pause between batches for verification, and can roll back if health degrades. “Skip matching” avoids replacing instances already on the desired config.
• GCP MIG rolling updates use a declarative updater API: you specify the target end-state and Compute Engine orchestrates it (GCP docs). maxSurge sets how many extra instances above target to create during the roll (faster, costs more); maxUnavailable sets how many can be down at once. A proactive update replaces instances automatically; an opportunistic one applies only on the next resize/recreate. Running two template versions at once enables canary — send a fraction to the new version, watch, then complete (GCP docs).
• Azure VMSS rolling upgrades replace instances in batches gated on the application health probe, with configurable batch size and pause between batches.

sequenceDiagram
    participant Op as Operator
    participant G as Autoscaling Group
    participant LB as Load Balancer
    Op->>G: new template version (new image)
    Note over G: batch 1 (respect min-healthy %)
    G->>LB: deregister old instances (batch 1)
    G->>G: terminate old · launch new-template instances
    LB-->>G: new instances pass health check
    G->>LB: register new instances (batch 1)
    Note over G: checkpoint — verify, then next batch
    G->>G: repeat for remaining batches
    Note over G: canary option — hold at N% on new version, observe, then finish

What it shows and the insight to take: a rolling replacement is a choreographed drain-and-replace that keeps the service up throughout by never taking more than the allowed fraction offline and only registering new instances once they’re healthy. The min-healthy / maxUnavailable / maxSurge knobs are the dial between speed and safety; checkpoints and canary add human/automated verification gates. This is how you ship a new OS image to a live fleet without downtime.

Failure Modes and Common Misunderstandings

• Health check checks the VM, not the app. If the group’s health signal is the hypervisor status, a hung application on a “running” VM is never replaced. Use an application/load-balancer health check (GCP explicitly distinguishes app-based health checks) so a wedged process triggers autohealing.
• Flapping from an over-eager policy. Too-tight thresholds plus no cooldown/warm-up cause launch-terminate oscillation, churning cost and destabilizing the service. Tune warm-up to real boot time and prefer target tracking. (Decision detail: Autoscaling in Practice.)
• max too low throttles a real spike; min too low risks a cold fleet. The clamp is absolute — a legitimate surge is capped at max, and desired can’t drop below min. Set both deliberately.
• Autohealing loops on a broken image. If a new template’s image crashes on boot, the group will terminate and relaunch endlessly, each replacement failing health checks. Roll forward carefully with checkpoints and be ready to roll back.
• Stateful workloads on a fleet built for cattle. ASGs/MIGs/VMSS assume interchangeable, disposable instances. Local disk is lost on replacement. Use stateful MIGs / persistent volumes and external state, or don’t put databases here. (GCP offers stateful MIGs that preserve instance names, disks, and metadata across updates (GCP docs).)
• Scale-in terminates the wrong instance. Default termination picks may kill an instance mid-request. Use connection draining/deregistration delay on the load balancer and lifecycle hooks to drain gracefully.

Alternatives and When to Choose Them

An autoscaling VM fleet is the right primitive when you run your own instances (a specific OS, GPU/driver stack, or a non-containerized app) and want HA plus elasticity. If your unit of deployment is a container, the rung above — Containers as a Service (Fargate, Cloud Run, ACI, Container Apps) — gives you the same elasticity without managing instances or images at all, often with scale-to-zero that a VM fleet can’t cheaply match. If you run many containers with an orchestrator, Managed Kubernetes Services scales pods (Horizontal Pod Autoscaler) and nodes (cluster autoscaler / Karpenter) instead — and in fact a managed-K8s node group is frequently implemented on top of an ASG/MIG/VMSS. For workloads that are purely event-triggered and short, FaaS removes the fleet entirely. The VM-fleet primitive wins when you need control of the machine itself; the container and function rungs win when you want to stop thinking about machines.

Production Notes

• Immutable, template-driven fleets are the backbone of blue-green and rolling deploys. Baking an AMI/image and rolling the fleet with instance refresh (or a MIG canary) is how teams ship without SSH-ing into servers — cross-link Mutable versus Immutable Infrastructure and CI/CD delivery.
• Spot/preemptible capacity inside the fleet is a major cost lever: mixed-instances ASGs and VMSS Spot let the group absorb interruptions by relaunching to maintain desired count (AWS ASG docs) — pairs with Spot and Preemptible Instances and FinOps and Cloud Cost Optimization.
• Predictive + reactive together is common for daily-cyclic traffic: predictive scaling pre-provisions ahead of the known morning ramp while target tracking handles the unpredictable remainder (AWS predictive scaling docs).
• Kubernetes clusters ride on this primitive. A managed-K8s node pool is usually a MIG/ASG/VMSS under the hood; understanding the VM-fleet loop explains a lot of cluster-autoscaler behaviour (Managed Kubernetes Services).

See Also

• Autoscaling in Practice — the SRE decision discipline: which metric, target selection, avoiding flapping (owns the “how to decide”)
• Containers as a Service — the container rung above; elasticity without managing instances
• Managed Kubernetes Services — pod + node autoscaling; node groups are built on this primitive
• Virtual Machine Instances and Instance Families — the instances a fleet is made of
• Mutable versus Immutable Infrastructure — why fleets are rebuilt from templates, not patched in place
• Cloud Load Balancers — the front door a fleet registers with; the health-check source
• Regions Availability Zones and Edge Locations — the failure domains a fleet spreads across for HA
• Spot and Preemptible Instances — interruptible capacity inside a mixed fleet
• Instance Sizing and Right-Sizing — choosing the instance shape a fleet stamps out
• Cloud Architecture MOC — parent map (§2 Compute)