vault

Kubebuilder

11 min read

Kubebuilder

Kubebuilder is the SIG-API-Machinery scaffolding framework for building Kubernetes controllers and operators in Go on top of the controller-runtime library (book.kubebuilder.io). It is not a runtime component — nothing called “Kubebuilder” runs in a cluster. It is a project generator plus a code-generation toolchain plus a canonical book: a CLI (kubebuilder) that scaffolds a complete operator repository (Go module layout, a PROJECT metadata file, typed API structs, reconciler stubs, Kustomize manifests, a Makefile, a Dockerfile), a set of marker comments (// +kubebuilder:...) that drive controller-gen to emit Custom Resource Definition (CRD) YAML and Go boilerplate, and the Kubebuilder Book, which is the de facto reference for operator authoring. Every Go operator you have ever used — cert-manager, the Prometheus Operator, CNPG, Argo’s controllers — is either built directly with Kubebuilder or with Operator SDK’s Go path, which itself wraps Kubebuilder. Understanding Kubebuilder is understanding the mechanical side of the Operator Pattern: it is the tooling that turns “I have operational knowledge to encode” into a deployable controller without writing the ~90 % of a controller that is identical across every operator.

Mental Model

The clean way to hold Kubebuilder in your head is as three layers that you progressively own less of as you go up:

  1. controller-runtime (bottom) — a Go library you do not edit. It supplies the Manager, the informer-backed cached client, the workqueue, the leader-election machinery, the webhook server. This is the engine that runs the Kubernetes Control Loop Pattern.
  2. controller-gen (middle) — a code generator you do not edit but do invoke (via make). It reads marker comments in your Go source and emits artifacts: CRD YAML, zz_generated.deepcopy.go, RBAC Role YAML, webhook configuration YAML.
  3. Your code (top) — the only thing you actually write: the Go structs that define your custom resource’s Spec/Status, and the body of the Reconcile function.

Kubebuilder the CLI exists to wire those three layers together: it scaffolds the directory layout, generates the glue between your structs and the manager, and gives you a Makefile that knows how to call controller-gen and go build.

flowchart TD
    subgraph AUTHOR["What you write"]
        TYPES["api/v1/&lt;kind&gt;_types.go<br/>Spec + Status structs<br/>+kubebuilder markers"]
        RECON["internal/controller/&lt;kind&gt;_controller.go<br/>Reconcile function body"]
    end
    subgraph GEN["controller-gen (make generate / make manifests)"]
        DEEPCOPY["zz_generated.deepcopy.go<br/>DeepCopy() methods"]
        CRDYAML["config/crd/bases/*.yaml<br/>CRD OpenAPI schema"]
        RBACYAML["config/rbac/role.yaml<br/>ClusterRole"]
        WEBHOOKYAML["config/webhook/*.yaml"]
    end
    subgraph RUNTIME["controller-runtime (you don't edit)"]
        MGR["Manager<br/>cached client + workqueue<br/>+ leader election"]
    end
    CLUSTER[("kube-apiserver + etcd")]

    TYPES -- "markers parsed by" --> GEN
    RECON --> MGR
    DEEPCOPY --> MGR
    GEN --> MGR
    CRDYAML -- "make install" --> CLUSTER
    RBACYAML -- "make deploy" --> CLUSTER
    MGR -- "watch / reconcile" --> CLUSTER

The diagram shows the division of labour. The insight to extract: you only ever touch the two boxes in “What you write” — the type definitions and the reconcile body. Everything else (DeepCopy methods, the CRD YAML, the RBAC, the manager wiring) is generated from marker comments. Kubebuilder’s value proposition is that the single source of truth for “what does this CRD look like” is your Go struct, annotated with markers; the YAML is downstream of it, never hand-edited.

Mechanical Walk-through

kubebuilder init — scaffold the project

You run kubebuilder init --domain=example.com --repo=github.com/myorg/myproject in an empty directory. This produces:

  • PROJECT — a YAML metadata file that records the project’s domain, repo, layout (which plugin generated it, e.g. go.kubebuilder.io/v4), projectName, and an initially-empty resources: list. The CLI and its plugins read this file on every subsequent command to know how to scaffold consistently (book.kubebuilder.io — Project config). It is the project’s “self-description”; do not hand-edit it.
  • go.mod — the Go module, with sigs.k8s.io/controller-runtime pinned.
  • cmd/main.go — the entry point. It constructs a ctrl.Manager, registers health/readiness endpoints, sets up leader election, and (later) calls each controller’s SetupWithManager.
  • config/ — a tree of Kustomize bases and overlays: config/default (the deployable composition), config/manager (the operator Deployment), config/rbac, config/crd, config/prometheus, config/webhook.
  • Makefile — the workflow driver (see below).
  • Dockerfile — a multi-stage build that compiles the manager binary and packages it into a distroless image.
  • .golangci.yml, hack/, test/ — linting config and an envtest-based test harness.

At this point there are no API types and no controllers — init only lays the foundation.

kubebuilder create api — add a resource + its controller

You run kubebuilder create api --group cache --version v1alpha1 --kind Memcached. The CLI prompts whether to scaffold a resource (the API types) and a controller (the reconciler). Saying yes to both produces:

  • api/v1alpha1/memcached_types.go — a Go file with two structs: MemcachedSpec (the desired state — what the user fills in) and MemcachedStatus (the observed state — what the controller reports), plus the wrapper Memcached and list MemcachedList. This file is where you write the actual schema, as plain Go fields decorated with marker comments.
  • internal/controller/memcached_controller.go — a reconciler stub: a MemcachedReconciler struct with an empty Reconcile method and a SetupWithManager method. You fill in the Reconcile body with your operational logic. See Reconcile Function Patterns.
  • An update to PROJECT — the new resource is appended to resources:.
  • An update to cmd/main.go — a call to (&MemcachedReconciler{...}).SetupWithManager(mgr).

Marker comments — the heart of the system

A marker is a single-line Go comment beginning with // +, optionally with arguments (book.kubebuilder.io — Markers). controller-gen parses them. There are three broad families:

  • CRD / validation markers — // +kubebuilder:validation:Minimum=1, // +kubebuilder:validation:MaxItems=10, // +kubebuilder:default=3, // +kubebuilder:validation:Enum=Blue;Green;Red. These become OpenAPI v3 schema constraints in the generated CRD. A // +kubebuilder:validation:XValidation:rule="..." marker emits a CEL rule (see CEL in Kubernetes).
  • Object markers — // +kubebuilder:object:root=true marks a struct as a top-level API type (so controller-gen object emits its DeepCopyObject); // +kubebuilder:subresource:status enables the /status subresource; // +kubebuilder:subresource:scale enables /scale (so the resource works with the Horizontal Pod Autoscaler); // +kubebuilder:printcolumn:... adds columns to kubectl get output.
  • RBAC markers — // +kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete placed above the Reconcile function. controller-gen collects every such marker in the codebase and emits a single ClusterRole covering exactly the permissions the controller’s code claims. This is RBAC-as-code: the permission set is derived from the source, not maintained separately.

The development workflow

The Makefile encodes the canonical loop (book.kubebuilder.io — Getting Started):

kubebuilder init --domain=example.com --repo=github.com/myorg/myproject
kubebuilder create api --group cache --version v1alpha1 --kind Memcached
# ... edit api/v1alpha1/memcached_types.go  and  internal/controller/memcached_controller.go ...
make generate    # controller-gen object: regenerate zz_generated.deepcopy.go
make manifests   # controller-gen crd/rbac/webhook: regenerate config/ YAML
make install     # kubectl apply the CRDs into the current-context cluster
make run         # go run ./cmd/main.go  — runs the manager LOCALLY against the cluster
# OR, for in-cluster:
make docker-build docker-push IMG=<registry>/memcached:tag
make deploy IMG=<registry>/memcached:tag   # kustomize build config/default | kubectl apply

The crucial habit: after every edit to a _types.go file, run make generate && make manifests. The generated DeepCopy methods and the CRD YAML are downstream artifacts; forgetting to regenerate them is the single most common Kubebuilder mistake (see Failure Modes). make run is the fast dev loop — it runs the controller binary on your laptop, talking to a remote cluster’s API server, so you get edit-compile-test in seconds without building an image.

Configuration / API Surface — an annotated _types.go

package v1alpha1
 
import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 
// MemcachedSpec defines the DESIRED state of Memcached.
type MemcachedSpec struct {
    // Size is the number of memcached replicas to run.
    // +kubebuilder:validation:Minimum=1          // generated CRD rejects size < 1
    // +kubebuilder:validation:Maximum=10         // ...and size > 10, at apiserver admission
    // +kubebuilder:default=3                     // if the user omits it, the apiserver defaults it to 3
    Size int32 `json:"size"`
 
    // ContainerPort is the port memcached listens on.
    // +optional                                  // marks the field non-required in the CRD schema
    ContainerPort int32 `json:"containerPort,omitempty"`
}
 
// MemcachedStatus defines the OBSERVED state of Memcached.
type MemcachedStatus struct {
    // Conditions follows the standard metav1.Condition convention.
    // +optional
    Conditions []metav1.Condition `json:"conditions,omitempty"`
}
 
// +kubebuilder:object:root=true                  // this is a top-level Kind: emit DeepCopyObject
// +kubebuilder:subresource:status                // split .status into its own /status subresource
// +kubebuilder:printcolumn:name="Size",type=integer,JSONPath=`.spec.size`
 
// Memcached is the Schema for the memcacheds API.
type Memcached struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`
    Spec   MemcachedSpec   `json:"spec,omitempty"`
    Status MemcachedStatus `json:"status,omitempty"`
}
 
// +kubebuilder:object:root=true
 
// MemcachedList contains a list of Memcached.
type MemcachedList struct {
    metav1.TypeMeta `json:",inline"`
    metav1.ListMeta `json:"metadata,omitempty"`
    Items           []Memcached `json:"items"`
}

Line-by-line: the +kubebuilder:validation:* markers above Size become minimum/maximum in the CRD’s OpenAPI schema — the apiserver will reject a bad Memcached object at admission time, before your controller ever sees it. +kubebuilder:default=3 becomes a schema default the apiserver applies server-side. +optional and the omitempty JSON tag mark ContainerPort non-required. +kubebuilder:object:root=true tells controller-gen object to generate DeepCopyObject() so the struct satisfies runtime.Object. +kubebuilder:subresource:status is load-bearing: it makes .status a separate REST endpoint so that controller status writes do not bump metadata.generation (the convention from the Kubernetes Control Loop Pattern). The takeaway: the CRD is a projection of these structs. You never write CRD YAML; you write annotated Go and run make manifests.

Failure Modes

  • Stale generated artifacts. Editing _types.go and forgetting make generate && make manifests means the cluster’s CRD schema and the in-binary DeepCopy methods disagree with your structs. Symptoms: the apiserver rejects fields your code expects, or the controller panics in deepcopy. Fix: always regenerate; consider a go generate / CI check.
  • RBAC marker drift. The controller calls an API the +kubebuilder:rbac markers do not cover. The generated ClusterRole lacks the verb, and the controller gets Forbidden errors at runtime. Symptom: forbidden in reconcile logs. Fix: add the marker, make manifests, make deploy.
  • make run vs make deploy confusion. make run runs the controller on your laptop with your kubeconfig identity — RBAC problems are masked because you are usually cluster-admin locally. The controller can pass make run and fail make deploy because in-cluster it uses the (correctly RBAC-limited) ServiceAccount. Always test make deploy before declaring done.
  • API version skew across kubebuilder CLI upgrades. A newer kubebuilder CLI uses a newer layout plugin (go.kubebuilder.io/v4); regenerating an old project can rewrite scaffolding incompatibly. Pin the CLI version; the PROJECT file records the layout so kubebuilder knows which plugin to use.
  • Webhook certificate plumbing. Scaffolded webhooks need a serving cert; the default config/ assumes cert-manager is installed. Forgetting it leaves the webhook server with no TLS cert and the apiserver unable to call it — every create of the resource fails. See MutatingAdmissionWebhook.

Alternatives and When to Choose Them

  • Operator SDK (Go flavor) — Red Hat’s framework. Its Go path wraps Kubebuilder; the two have largely converged, and a Kubebuilder project is essentially an Operator-SDK Go project minus the OLM bundle scaffolding. Choose Operator SDK over bare Kubebuilder when you want the non-Go flavors (Ansible, Helm) or first-class Operator Lifecycle Manager bundle/CSV generation.
  • Bare controller-runtime — skip the scaffolding entirely and use the library directly. Reasonable for a one-controller project where the generated config/ tree is overkill, or when embedding a controller inside a larger application. You lose marker-driven CRD/RBAC generation.
  • client-go informers + workqueue by hand — the pre-controller-runtime way (how the built-in kube-controller-manager controllers are written). Maximum control, maximum boilerplate. Almost never the right choice for new operators.
  • Metacontroller / KUDO / shell-operator — declarative or scripting-based operator frameworks. Choose when the reconcile logic is simple enough to express without compiling Go; lower ceiling, much lower floor.
  • Aggregated API Server — a different extension axis entirely: if you need custom storage or REST semantics rather than a CRD, Kubebuilder is the wrong tool. CRD + controller (Kubebuilder’s model) covers ~95 % of cases.

Production Notes

  • The Kubebuilder Book is the canonical operator-authoring reference. Its CronJob tutorial is the standard onboarding path; the “Markers” and “controller-gen CLI” reference pages are the day-to-day lookups.
  • envtest (scaffolded under test/) spins up a real kube-apiserver + etcd binary pair locally — no node, no kubelet — so controller logic can be integration-tested in CI without a full cluster. This is the standard testing approach for every serious operator.
  • Distroless images by default. The scaffolded Dockerfile builds onto gcr.io/distroless/static — no shell, minimal attack surface. A deliberate security default.
  • The go.kubebuilder.io/v4 layout is the current default plugin layout; it places controllers under internal/controller/ (not the older controllers/) so they cannot be imported by external modules.
  • cert-manager, the Prometheus Operator, KubeVirt, CNPG, Crossplane providers — the bulk of the CNCF operator landscape is Kubebuilder- or Operator-SDK-Go-scaffolded. Reading their api/ and internal/controller/ trees is the fastest way to learn idiomatic operator design.

See Also

Graph View

Backlinks