vault

Borg Omega and Kubernetes Lineage

26 min read

Borg Omega and Kubernetes Lineage

The intellectual genealogy of Kubernetes is documented by its own designers in a 2016 retrospective: Burns, Grant, Oppenheimer, Brewer, and Wilkes, “Borg, Omega, and Kubernetes,” ACM Queue 14(1), January–February 2016 (queue.acm.org/detail.cfm?id=2898444; reprinted in Communications of the ACM 59(5), May 2016). The article describes three generations of container-management systems built at Google — Borg (in production since approximately 2003–2004), Omega (begun around 2013 as a research successor), and Kubernetes (announced June 2014 as the open-source distillation of the two) — and the lessons each carried forward. The companion EuroSys 2015 paper “Large-scale cluster management at Google with Borg” (Verma et al. 2015) is the canonical Borg description. Together these two sources are required reading for anyone who wants to defend Kubernetes’ design choices rather than merely use the result. This note distills the lineage: what each system was, what each kept from its predecessor, what each abandoned, and which design instincts coalesced into the K8s we know in 2026. The framing repeatedly invoked by the authors is that “though widespread interest in software containers is a relatively recent phenomenon, at Google we have been managing Linux containers at scale for more than ten years and built three different container-management systems in that time” (Burns et al. 2016).

Mental Model

flowchart LR
    subgraph "Borg (2003-2004 → present)"
        B[Jobs · Tasks · Allocs<br/>BCL config language<br/>Monolithic scheduler<br/>Cells of 10K+ machines]
    end
    subgraph "Omega (2013 → research)"
        O[Optimistic concurrency<br/>Shared Paxos store<br/>Parallel verticals<br/>Per-team schedulers]
    end
    subgraph "Kubernetes (2014 → open source)"
        K[Pods · Services · Labels<br/>Declarative spec/status<br/>API-first extensibility<br/>YAML/JSON manifests]
    end
    B -- "allocs → Pods<br/>BNS naming → Services<br/>IP-per-task → IP-per-Pod<br/>declarative reconciliation" --> K
    B -- "monolithic scheduler<br/>+ rigid Job/Task hierarchy<br/>too inflexible at scale" --> O
    O -- "shared-state schedulers<br/>+ optimistic concurrency<br/>+ API-as-source-of-truth" --> K
    B -- "BCL config language<br/>+ Google-internal coupling<br/>+ job-as-primary-unit<br/>DROPPED" --x K

What this diagram shows. Borg, Omega, and Kubernetes form a lineage in which the predecessor’s primitives are inherited but the predecessor’s constraints are deliberately discarded. From Borg, Kubernetes inherited the resource-allocation construct (allocs → Pods), the service-naming abstraction (Borg Name Service → Services), the IP-per-workload networking model, and the principle of declarative reconciliation between desired and observed state. From Omega, Kubernetes inherited the design of a shared persistent store read and written by independent control-plane components via optimistic concurrency control — a generalization of the scheduler-vertical pattern Omega had pioneered. What Kubernetes deliberately dropped was Borg’s job-as-primary-unit hierarchy, Borg’s monolithic scheduler, Borg’s BCL configuration language, and the deep coupling to Google-internal infrastructure (Chubby, BNS, Stubby, etc.) that made Borg un-externalizable. The insight to extract is that Kubernetes is not a clean-room rewrite — it is a third generation refinement that explicitly preserves what worked and discards what proved to be operational or social mistakes.

Borg (2003–present): The Original

Borg is Google’s internal cluster manager, in continuous production use since approximately 2003–2004 (Kubernetes blog — Borg: The Predecessor to Kubernetes notes that Google has been “running containerized workloads in production for more than a decade” as of April 2015). The fullest public description is the EuroSys 2015 paper by Verma et al. (research.google — Large-scale cluster management at Google with Borg), which describes Borg as “a cluster manager that runs hundreds of thousands of jobs, from many thousands of different applications, across a number of clusters each with up to tens of thousands of machines.” The architecture is the model from which both Omega and Kubernetes descended.

A Borg installation is organized into cells. A cell is the unit of administrative isolation — typically several thousand to tens of thousands of machines in a single failure domain, with its own Borgmaster (the control plane) and Borglet (the per-machine agent). Within a cell, the resource model is hierarchical:

  • A Job is a named collection of identical replicas of a service or batch workload — a single web frontend, a single MapReduce execution, a single Bigtable tablet server tier. Jobs are the unit of configuration and the addressable handle for operations.
  • A Task is an individual replica of a Job — Job myservice with replicas: 100 is composed of tasks myservice.0 through myservice.99. Tasks are the unit of scheduling: the Borg scheduler decides which task runs on which machine.
  • An Alloc (“allocation”) is a reserved set of resources on a single machine into which one or more tasks can be placed. Allocs allow co-located tasks — a primary web server and a sidecar log shipper, or a primary cache and a sidecar metric exporter — to share a machine’s resources without each task negotiating independently with the scheduler. The article phrases the lineage directly: “the outermost container is called a resource allocation, or alloc; in Kubernetes, it is called a pod” (Burns et al. 2016, p. 79). Popular alloc usage patterns are “running a web server that generates logs alongside a lightweight log collection process that ships the log to a cluster filesystem (not unlike fluentd or logstash); running a web server that serves data from a disk directory that is populated by a process that reads data from a cluster filesystem and prepares/stages it for the web server (not unlike a Content Management System); and running user-defined processing functions alongside a storage shard” (kubernetes.io 2015). One notable difference the article calls out: “Borg also allows top-level application containers to run outside allocs; this has been a source of much inconvenience, so Kubernetes regularizes things and always runs an application container inside a top-level pod, even if the pod contains a single container” — i.e. K8s deliberately tightened the alloc/Pod boundary into an unconditional invariant. Allocs are the direct conceptual ancestor of the Kubernetes Pod.

Configuration in Borg is written in BCL (Borg Configuration Language), a domain-specific declarative language descended from GCL (Google Configuration Language), itself a successor to Lisp-based configuration languages used internally at Google for decades. BCL is feature-rich (variables, conditionals, computed defaults, inheritance) and tightly coupled to the rest of Google’s internal infrastructure — it is essentially impossible to externalize without rebuilding several decades of dependencies. This single design choice — make the config language deeply expressive but proprietary — is one of the principal reasons Kubernetes opted for plain YAML/JSON manifests instead.

The Borg scheduler is monolithic: a single Borgmaster process arbitrates all scheduling decisions for the cell, using preemption, priorities, and quotas to keep the cell utilized. The scheduler supports two large job classes: production jobs (latency-sensitive, high-priority) and batch jobs (throughput-oriented, low-priority), with the batch tier consuming resources reclaimed from the production tier’s unused headroom. This bin-packing across priority classes is the principal mechanism by which Google achieves the high utilization figures (50–70% steady-state CPU usage, vs. typical industry 5–25%) that Borg is famous for. The Kubernetes scheduler (kube-scheduler) inherits the basic filter-and-score architecture but exposes far less of the priority machinery (see Pod Priority and Preemption) by default — the priority surface in K8s is a deliberately scaled-down version.

What Borg got right — what every successor preserved — was the declarative-with-reconciliation operating model: users declare desired state via BCL manifests, and a set of cooperating controllers continuously reconcile the cell’s observed state to match. The Burns et al. retrospective phrases it as having taught the team that “a centralized point of management … combined with a declarative API to manage resources, was a great fit for the kind of scale at which Google operates” (Burns et al. 2016). What Borg got wrong in ways the successors deliberately addressed: the rigid Job/Task hierarchy made cross-cutting concerns (canary tracking, version rollout grouping) awkward; the monolithic scheduler made experimentation with new scheduling policies expensive; the BCL coupling made Borg’s design un-externalizable.

Omega (2013): The Research Successor

Omega was begun around 2013 inside Google with the explicit goal of addressing Borg’s structural limitations without abandoning the operational properties that made Borg successful (Burns et al. 2016). The article phrases it as: “Omega, an offspring of Borg, was driven by a desire to improve the software engineering of the Borg ecosystem. It applied many of the patterns that had proved successful in Borg, but was built from the ground up to have a more consistent, principled architecture.” Omega never reached the same production deployment as Borg — it was primarily a research and engineering vehicle — but several of its ideas migrated back into Borg over time and every one of its core ideas migrated into Kubernetes.

The single most consequential Omega contribution is shared persistent state read and written by independent control-plane components via optimistic concurrency control. Where Borg has a monolithic Borgmaster that holds the cell state in memory and serializes mutations, Omega stores cell state in a shared (Paxos-backed) store that multiple parallel schedulers read and write directly. When two schedulers attempt conflicting placements, the loser detects the conflict via a version-mismatch on the store and retries — the optimistic concurrency pattern from database literature, applied to cluster scheduling. This eliminates the head-of-line blocking that a single scheduler creates and lets each team operate its own scheduler vertical tuned to its workload class (batch, latency-sensitive, ML training) without contending for a global lock.

The mechanism is direct ancestor to Kubernetes’ design: etcd is Omega’s shared store; the API server is the gateway to it; every controller (scheduler, replication, autoscaler, ingress, …) reads from and writes to etcd via the API server using resourceVersion-based optimistic concurrency. When two controllers attempt conflicting updates to the same object, the loser receives a Conflict (409) response and retries — the same pattern Omega introduced. The Kubernetes API server’s resourceVersion field is literally the optimistic-concurrency token, and the entire watch/list/update protocol is the Omega pattern with a REST surface bolted on. See Resource Versioning and Optimistic Concurrency for the K8s-specific elaboration. The article’s own framing of how this descended: “Like Omega, Kubernetes has at its core a shared persistent store, with components watching for changes to relevant objects. In contrast to Omega, which exposes the store directly to trusted control-plane components, state in Kubernetes is accessed exclusively through a domain-specific REST API that applies higher-level versioning, validation, semantics, and policy, in support of a more diverse array of clients” (Burns et al. 2016, p. 75–76). Omega’s mistake — letting trusted clients write the store directly — turns out to be the single biggest reason it could not absorb the ecosystem of untrusted clients that Kubernetes was designed to serve, and the REST-API-as-gatekeeper choice is the explicit course-correction.

Omega also pioneered the idea of the cluster state as the primary abstraction, rather than the scheduler as the primary abstraction. In Borg, the Borgmaster was the central process and its in-memory state was the source of truth. In Omega, the shared store is the source of truth and every component — schedulers, replication controllers, monitoring, anything else — is a peer client reading and writing it. This inversion is the structural basis of Kubernetes’ extensibility model: new controllers do not need to extend the scheduler — they just need read/write access to the API server, which is what makes Custom Resource Definitions and the Operator pattern (see Operator Pattern) work.

Kubernetes (2014): The Open-Source Distillation

Kubernetes was conceived in late 2013 / early 2014 by Joe Beda, Brendan Burns, and Craig McLuckie at Google, with early contributions from Ville Aikas, Tim Hockin, Dawn Chen, Brian Grant, and Daniel Smith (the latter three are co-authors of the Borg, Omega, and Kubernetes retrospective). The first public commit landed on 6 June 2014 (10 Years of Kubernetes); v1.0 shipped on 21 July 2015; the project was formally donated to the newly-formed CNCF on 10 March 2016 (CNCF — Kubernetes is ten years old). The internal project codename was Project Seven — a Star Trek reference to Seven of Nine, the former Borg drone severed from the collective and “rehabilitated” — preserved in the seven-spoked ship’s wheel logo (GeekWire). See Kubernetes for the surface-level umbrella note.

The retrospective frames Kubernetes’ design intent in explicit contrast to its predecessors: “Kubernetes was developed with a stronger focus on the experience of developers writing applications that run in a cluster: its main design goal is to make it easy to deploy and manage complex distributed systems, while still benefiting from the improved utilization that containers enable” (Burns et al. 2016). Three specific decisions distinguish K8s from Borg:

What Kubernetes kept from Borg

  1. The alloc → Pod abstraction. A Kubernetes Pod is structurally identical to a Borg alloc: one or more containers co-located on a machine, sharing a network namespace and (optionally) storage volumes. The Kubernetes blog post on Borg (kubernetes.io) makes this lineage explicit. See Pod for the K8s elaboration.
  2. Service naming and discovery. Borg has BNS (Borg Name Service) — a cluster-internal DNS-like resolver that maps job names to task locations. Kubernetes’ Service abstraction (Service (Kubernetes)) is the open-source descendant: a stable virtual IP and DNS name fronting a dynamic set of Pods. The Kubernetes blog post phrases the inheritance directly: “K8s adopted Borg’s cluster services approach for service naming, dynamic load balancing, and automatic pod discovery across rescheduling events.”
  3. IP-per-workload networking. Borg gives every task its own IP — Kubernetes gives every Pod its own IP. The “no NAT between Pods” rule of Kubernetes Networking Model is the direct descendant of Borg’s flat-network design.
  4. Declarative reconciliation as the operating mode. The user declares desired state; controllers continuously reconcile actual state to match. This is the Borg pattern in its pure form, elevated to the foundational principle of the platform. See Kubernetes Control Loop Pattern and Desired State vs Observed State.
  5. Resource quotas and admission control. Borg has explicit quota and admission tiers ensuring that production jobs always win over batch jobs. Kubernetes inherits this via ResourceQuota, LimitRange, Pod Priority and Preemption, and the QoS Classes derived from request/limit ratios. The K8s surface is less elaborate than Borg’s but the conceptual lineage is direct.

What Kubernetes kept from Omega

  1. Shared persistent store as the source of truth. etcd plays the role of Omega’s Paxos-backed store; the API server is the gateway through which every other component interacts with it. The Burns et al. retrospective phrases the lesson as “the API server should be the centralized point of contact” — Omega proved this works.
  2. Optimistic concurrency over shared state. Every Kubernetes object carries a metadata.resourceVersion field; every update is a CAS (compare-and-swap) against that field; conflicting updates return HTTP 409 and force the client to refresh-and-retry. This is Omega’s pattern in REST clothing. See Resource Versioning and Optimistic Concurrency.
  3. Independent controllers as the unit of behavior. Where Borg had a centralized Borgmaster handling everything, Omega split control logic into multiple independent components that share state. Kubernetes takes this to its logical conclusion: kube-controller-manager runs dozens of independent control loops; cloud-controller-manager runs cloud-specific loops; users add their own via CRDs and operators. No single component is responsible for “the schedule” — the cluster’s behavior emerges from the interaction of many small controllers, each implementing the Kubernetes Control Loop Pattern.
  4. API-first extensibility. Omega exposed cluster state via a typed API; Kubernetes extends this to the principle that the entire platform is the API surface, and extending the platform means extending the API. This is what makes the Custom Resource Definition mechanism and the entire Operator ecosystem possible.

What Kubernetes added beyond Borg and Omega

  1. Labels and selectors replacing the rigid Job/Task hierarchy. This is the most consequential K8s-specific innovation. Borg organizes work as Job → Task — flat tasks under named jobs, with no flexibility for cross-cutting groupings. Kubernetes replaces this with arbitrary key/value labels attached to any object plus selectors that pick out sets of objects matching label expressions (Kubernetes Labels and Selectors). The Kubernetes blog post on Borg phrases the win: “While the Job is a very useful abstraction, it can be limiting. … In Kubernetes you use labels to identify groups of pods. The ‘job:’ is equivalent to a Borg Job, but a label can also be used to identify the service, tier, release type (production, staging, test), and in general, any arbitrary subset of pods.” This single change — from hard-coded hierarchy to many-to-many label-driven grouping — is what makes Deployments managing ReplicaSets managing Pods, canary tracks coexisting with stable tracks, and Services fronting Pod sets across multiple Deployments all uniformly expressible.
  2. A neutrally-owned, declarative manifest format (YAML/JSON). Borg’s BCL is feature-rich but Google-proprietary; Kubernetes uses plain YAML or JSON with a JSON Schema-derived OpenAPI spec for validation. This is less powerful than BCL — there is no inheritance, no first-class variables, no conditionals — and the consequence has been the rise of templating layers (Helm, Kustomize, Jsonnet) to add the expressiveness back. The trade is intentional: a portable, language-agnostic config format trumps a more expressive but proprietary one for a platform whose entire value proposition is portability across organizations.
  3. Pods as the atomic scheduling unit and the unit of network identity. Borg’s allocs were a resource-reservation construct primarily; Kubernetes elevates the Pod to the atomic unit of scheduling, network identity (one IP per Pod), and lifecycle management. A Pod gets one IP that its containers share; a Pod is the smallest thing the scheduler places; a Pod is the smallest thing that fails and is replaced. This consolidation of concerns into a single resource is one of the things that makes Kubernetes’ mental model simpler than Borg’s.
  4. An open community and a non-Google governance model. Borg is and always has been Google-internal. Kubernetes was donated to the CNCF on 10 March 2016 specifically so that no single company controls the project. The Steering Committee is elected; the SIG (Special Interest Group) structure distributes design authority; the deprecation policy and the KEP (Kubernetes Enhancement Proposal) process are externally documented. The Burns et al. retrospective acknowledges this as a design decision, not just a governance one: “Open-sourcing Kubernetes was an explicit decision to let it grow beyond what a single company could maintain.”

What Kubernetes deliberately dropped

  1. BCL (Borg Configuration Language). Replaced by plain YAML/JSON, with templating layers built on top. The decision priced portability over expressiveness.
  2. The Job → Task hierarchy as the primary identity model. Replaced by labels and selectors over Pods. Workload kinds (Deployment, StatefulSet, DaemonSet, Job, CronJob) are now controllers over labeled Pods, not first-class identity hierarchies.
  3. Google-internal infrastructure coupling. Borg depends on Chubby (lock service), BNS (naming), Stubby (RPC), Google Production Environment (machine management), and dozens of other internal services. Kubernetes depends on etcd (open-source consensus store), DNS (CoreDNS), gRPC (open-source RPC) — every piece is independently externalizable.
  4. The monolithic scheduler. Replaced by a single default scheduler that is itself pluggable (the Scheduling Framework) and explicitly designed to coexist with custom schedulers per workload class — the Omega vertical pattern made operationally accessible.
  5. Priority-class-driven preemption as the central mechanism. Borg uses preemption aggressively to keep cells utilized. Kubernetes supports priority and preemption (Pod Priority and Preemption) but it is opt-in and used much less aggressively by default — production teams typically run with one or two priority tiers, not Borg’s elaborate priority taxonomy.

Lessons the Burns et al. Retrospective Distills

A common secondary-source claim is that the Borg, Omega, and Kubernetes article enumerates “seven lessons” — verifying against the published PDF (archive copy) shows this is a mischaracterization. The article does not present a numbered lessons list. Instead, the “Things to Avoid” section names four anti-patterns the Borg/Omega experience taught the team, plus two open problems (“Configuration” and “Dependency management”) explicitly flagged as unsolved. The Conclusions section is brief and contains no separate enumerated lessons. The actual section structure, quoted verbatim from the PDF:

  1. “Don’t make the container system manage port numbers.” “All containers running on a Borg machine share the host’s IP address, so Borg assigns the containers unique port numbers… This means that traditional networking services such as the DNS (Domain Name System) have to be replaced by home-brew versions; service clients do not know the port number assigned to the service a priori and have to be told; port numbers cannot be embedded in URLs, requiring name-based redirection mechanisms…” (Burns et al. 2016, p. 83–84). The fix: “Learning from our experiences with Borg, we decided that Kubernetes would allocate an IP address per pod, thus aligning network identity (IP address) with application identity.” See Pod Networking and Kubernetes Networking Model.
  2. “Don’t just number containers: give them labels.” Borg used a Job → Task[i] integer-indexed vector. The article explicitly catalogs the rigidity costs: holes when middle tasks exit, unhelpful slot-reuse semantics on restart, restart-order interactions with sharded applications causing data unavailability, and the lack of any clean place to put application metadata like role=frontend or stage=canary. Kubernetes’ answer: “a key/value pair that contains information that helps identify the object… Sets of objects are defined by label selectors (e.g., stageproduction && rolefrontend). Sets can overlap, and an object can be in multiple sets, so labels are inherently more flexible than explicit lists of objects or simple static properties.” See Kubernetes Labels and Selectors.
  3. “Be careful with ownership.” In Borg, tasks are owned by jobs — “Creating a job creates its tasks; those tasks are forever associated with that particular job, and deleting the job deletes the tasks.” The problem the authors highlight: “because there is only one grouping mechanism, it needs to handle all use cases.” Kubernetes’ looser label-selector-based ownership allows orphan/adopt semantics — a misbehaving Pod can be removed from a Service’s selection by editing its labels, taking it out of rotation while leaving it running for debugging, with the controller spinning up a replacement. The flexibility has a cost: “multiple controllers might think they have jurisdiction over a single pod. It is important to prevent such conflicts through appropriate configuration choices.”
  4. “Don’t expose raw state.” This is the article’s most architecturally consequential lesson. “The Borgmaster is a monolithic component that knows the semantics of every API operation… In contrast, Omega has no centralized component except the store, which simply holds passive state information and enforces optimistic concurrency control: all logic and semantics are pushed into the clients of the store, which directly read and write the store contents.” Both extremes have problems: monolithic Borg is hard to extend; raw-store Omega makes it hard to enforce system-wide invariants when every component bypasses any central validation. “Kubernetes picks a middle ground… by forcing all store accesses through a centralized API server that hides the details of the store implementation and provides services for object validation, defaulting, and versioning.” See Aggregated API Server and Custom Resource Definition.

The two open problems the article explicitly flags as unsolved (and which remain interesting reading for anyone who thinks the platform is “done”):

  • Configuration. The article’s frank conclusion: “To cope with these kinds of requirements, configuration-management systems tend to invent a domain-specific configuration language that (eventually) becomes Turing complete… The result is the kind of inscrutable ‘configuration is code’ that people were trying to avoid by eliminating hard-coded parameters in the application’s source code.” The recommended approach: “accept this need, embrace the inevitability of programmatic configuration, and maintain a clean separation between computation and data. The language to represent the data should be a simple, data-only format such as JSON or YAML, and programmatic modification of this data should be done in a real programming language.” The Helm/Kustomize/Jsonnet ecosystem is the post-publication answer to this open problem.
  • Dependency management. “If an application has dependencies on other applications, wouldn’t it be nice if those dependencies (and any transitive dependencies they may have) were automatically instantiated by the cluster-management system?” The article calls this an open challenge; the Operator Pattern and Service Broker / Open Application Model attempts since 2016 are partial answers.

Three additional positive insights the article foregrounds — these are themes that recur throughout rather than enumerated lessons:

  • The application-oriented shift. “Our load balancers don’t balance traffic across machines; they balance across application instances. Logs are keyed by application, not machine, so they can easily be collected and aggregated across instances.” See Kubernetes Object Model.
  • The reconciliation loop as universal pattern. “The idea of a reconciliation controller loop is shared throughout Borg, Omega, and Kubernetes to improve the resiliency of a system: it compares a desired state (e.g., how many pods should match a label-selector query) against the observed state (the number of such pods that it can find), and takes actions to converge the observed and desired states.” See Kubernetes Control Loop Pattern and Desired State vs Observed State.
  • Uniform API shape. “Every Kubernetes object has three basic fields in its description: ObjectMetadata, Specification (or Spec), and Status.” The uniformity makes generic tools, dynamic API extension, and operator-style controllers possible. See Kubernetes Object Model.

Mechanical Walk-through: A Specific Inheritance

To make the lineage concrete, consider the sequence of decisions that turn a Borg job into a Kubernetes Deployment:

In Borg, a user submits a BCL file declaring job myservice { task_count: 100; binary: ...; ... }. The Borgmaster ingests the file, assigns each of the 100 tasks to a machine, monitors them, and restarts failures. The job has a stable identity (myservice); each task has an identity (myservice.42) tied to its position in the job’s task array. To deploy a new binary, the user runs borg-update myservice which sequentially restarts each task with the new binary, optionally with a configured concurrency and rollback strategy.

In Kubernetes, the user creates a Deployment manifest declaring replicas: 100 with a Pod template specifying the container image. The flow:

  • The Deployment controller observes the manifest and creates a ReplicaSet with replicas: 100 and the same Pod template plus a hash label pod-template-hash: <hash>.
  • The ReplicaSet controller observes the ReplicaSet and creates 100 Pods, each labeled app=myservice, pod-template-hash=<hash>.
  • Each Pod is scheduled to a node and started by the kubelet.
  • To deploy a new image, the user applys an updated manifest. The Deployment controller creates a new ReplicaSet with a different pod-template-hash. The Deployment then orchestrates a rolling update by adjusting the two ReplicaSets’ replicas counts: scale the new ReplicaSet up by maxSurge, scale the old one down by maxUnavailable, wait for new Pods to pass readiness probes, repeat until the new ReplicaSet has 100 and the old has 0.

The key conceptual differences:

  • Identity is per-Pod, not per-position. There is no myservice.42 in K8s — Pods get generated names (myservice-<rs-hash>-<random>) and may be replaced with new ones at any time. Pods are mortal and interchangeable.
  • Grouping is by label. “All Pods belonging to myservice” is app=myservice; “all current-version Pods” is app=myservice, pod-template-hash=<new-hash>; “all old-version Pods” is app=myservice, pod-template-hash=<old-hash>. The selector is the abstraction; the controllers maintain the right number of Pods matching each selector.
  • The rollout is controller-driven, not user-driven. The user updates the Deployment manifest and the Deployment controller computes the necessary ReplicaSet-level scaling actions. The user never touches Pods or ReplicaSets directly. The lineage runs straight from Borg’s borg-update but the implementation is now decomposed into independent controllers.

Production Notes

  • The “Borg paper” effect. Verma et al.’s 2015 EuroSys paper is one of the most cited cluster-management papers in the field. It is required reading at every major hyperscaler’s platform team. Anecdotally, the paper drove a wave of “let’s adopt Borg-style ideas” projects at Facebook (Tupperware, now Twine), Twitter (Aurora on Mesos), Microsoft (Service Fabric), and elsewhere — many of which subsequently migrated to Kubernetes as the open-source equivalent matured.
  • The Burns et al. 2016 retrospective is the design document Kubernetes never had. Joe Beda has repeatedly cited the ACM Queue article in conference talks and in Kubernetes: Up and Running (Hightower et al. 2017) as the place to look for why K8s is shaped the way it is.
  • Omega never reached the scale of Borg in production, but its ideas have continuously trickled back into Borg over time. Internally at Google, the line between Borg and Omega has blurred; externally, both feed into Kubernetes.
  • Kubernetes is not the only Borg-influenced open-source orchestrator. HashiCorp’s Nomad explicitly draws on Borg’s two-level scheduler design (priorities, preemption, batch + service); Twitter’s Aurora on Mesos was Borg-influenced; Netflix’s Titus was Borg-influenced. Kubernetes’ dominance is the result of the CNCF ecosystem effect, not of any monopoly on Borg’s ideas.

Failure Modes (of Misunderstanding the Lineage)

  1. “Kubernetes is just open-source Borg.” It is not. Borg and Kubernetes share intellectual ancestors but have very different operating models in practice — Borg’s monolithic scheduler, BCL config, and Google-internal coupling make it un-runnable outside Google, and the workload-mix patterns (Borg’s heavy batch tier, K8s’ service-heavy mix) are quite different. Calling K8s “open-source Borg” obscures the specific decisions the K8s authors made not to inherit.
  2. “Kubernetes was a clean-room rewrite.” It was not. The retrospective is explicit that K8s was third generation — it inherits primitives, lessons, and design instincts from Borg and Omega even where the code is wholly new.
  3. “The Borg paper tells me how Kubernetes works.” The Borg paper describes Borg circa 2014. Kubernetes evolved away from many Borg patterns (e.g., the Job/Task hierarchy → labels), introduced things Borg lacks (CRDs, operators, the federation/multi-cluster story), and operates at different workload mixes. The paper is background, not a manual.
  4. “Omega never shipped.” Omega did ship inside Google, just not at Borg’s scale. The “research” framing in some secondary sources understates its production impact. The Burns et al. retrospective is careful to say Omega was “driven by a desire to improve the software engineering of the Borg ecosystem” — i.e., it was an engineering project with research properties, not pure research.

See Also

Graph View

Backlinks