Pipeline as Code

Pipeline as code is the practice of expressing an entire CI/CD pipeline — its triggers, stages, jobs, steps, dependencies, and environment configuration — as one or more version-controlled text files that live in the same repository as the application code, rather than as settings clicked together in a web UI. GitHub Actions stores YAML workflows in .github/workflows (GitHub, About workflows); GitLab CI reads a .gitlab-ci.yml from the repository root (GitLab, CI/CD pipelines); Jenkins reads a Jenkinsfile committed to source control (Jenkins, Pipeline); Tekton expresses pipelines as Kubernetes Custom Resource Definition (CRD) objects (Tekton, Concepts). The core claim is simple and load-bearing: because the pipeline definition is a file under version control, it is reviewable, versioned, diffable, revertible, branchable, and reproducible in exactly the same way the source code is — which is the entire difference between a pipeline you can reason about and one you cannot.

This note owns the concept and its mechanics — the file-in-repo model, why it beats click-configured jobs, and the reuse mechanisms (reusable/templated/composite pipelines) that keep large pipeline definitions from rotting. The concrete engines are cross-linked, not re-taught: GitHub Actions, GitLab CI CD, and the DAG execution model in Pipeline DAGs Stages and Gates. Deployment strategy (canary/blue-green) belongs to Site Reliability Engineering MOC; pipeline security (OIDC, secrets) to DevSecOps and Supply Chain Security MOC; environment provisioning to Infrastructure as Code MOC.

Mental Model — The Pipeline Lives Next to the Code It Builds

The mental shift is to stop thinking of the pipeline as configuration of a build server and start thinking of it as part of the application. A commit that changes src/ and a commit that changes .github/workflows/ci.yml are the same kind of event: a versioned change to the repository, reviewed in a pull request, tied to a commit SHA, and revertible. The pipeline evolves with the branch it lives on — a feature branch can carry a modified pipeline and test it before it merges, and a revert of a bad pipeline change is a git revert like any other.

flowchart TD
    subgraph REPO["Git repository (one versioned tree)"]
        SRC["Application source<br/>src/ · tests/"]
        PIPE[".github/workflows/ci.yml<br/>OR .gitlab-ci.yml<br/>OR Jenkinsfile<br/>OR Tekton CRDs"]
    end
    REPO -->|"push / PR event"| ENGINE["CI/CD engine reads<br/>the pipeline file<br/>at that commit"]
    ENGINE --> W["Workflow / Pipeline"]
    W --> J1["Job: build"]
    W --> J2["Job: test"]
    W --> J3["Job: deploy"]
    J1 --> S1["step: checkout"]
    J1 --> S2["step: compile"]
    J2 --> S3["step: run tests"]
    J3 --> S4["step: promote artifact"]
    S1 -.->|"runs on"| RUN["Runner / agent / pod"]

What this shows and the insight to take: the pipeline file is inside the same tree as the source, so the same commit that introduces a feature can introduce the CI steps that test it, and the engine always executes the pipeline as it existed at the commit being built. The insight: pipeline-as-code makes the build definition a function of the commit, not a mutable global setting — which is why a build from six months ago can be reproduced exactly, and why a pipeline change is subject to the same review and rollback guarantees as a code change.

The Structural Vocabulary — Workflow → Job → Step → Action

Every pipeline-as-code system decomposes the same way, though names differ. Using GitHub Actions’ terms as the reference model (GitHub, About workflows):

• A workflow is “a configurable automated process that will run one or more jobs,” defined as a YAML file in .github/workflows and triggered by repository events (configured under the on key), manual dispatch, or a schedule.
• A job is a unit of work that executes on a runner machine and comprises multiple steps. Jobs run in parallel by default and can declare dependencies on each other.
• A step is an individual task that “either executes a script that you define or runs an action.”
• An action is a reusable extension — a packaged unit of behavior invoked from a step.
• A runner is the machine that executes a job, GitHub-hosted or self-hosted.

GitLab uses stages (sequential groups) of jobs (parallel within a stage), where “stages run in sequence, while the jobs in a stage run in parallel” and a needs: keyword collapses strict stage ordering into a Directed Acyclic Graph (DAG) for speed (GitLab, CI/CD pipelines). Jenkins uses stages (e.g. “Build”, “Test”, “Deploy”) of steps (Jenkins, Pipeline). Tekton uses Pipelines composed of Tasks composed of Steps, each Step a container (Tekton, Concepts). The vocabulary maps cleanly across all of them:

Concept	GitHub Actions	GitLab CI	Jenkins	Tekton
File in repo	.github/workflows/*.yml	.gitlab-ci.yml	Jenkinsfile	*.yaml CRDs
Top-level unit	Workflow	Pipeline	Pipeline	Pipeline (CRD)
Grouping	Job	Stage → Job	Stage	Task
Smallest unit	Step	Job’s script line	Step	Step (container)
Reusable unit	Action / reusable workflow	include / extends	Shared Library	Task (catalog)
Executor	Runner	Runner (executor)	Agent	Pod

The needs: DAG and the runner-execution model are owned in depth by Pipeline DAGs Stages and Gates and Runners Agents and Executors respectively — this note stays on the “it’s a file in the repo” dimension.

Why Config-in-Repo Beats Click-Configured Jobs

The historical alternative — configuring jobs through a build server’s web UI (classic Jenkins “freestyle” jobs, old Travis/Bamboo screens) — fails on every axis that matters, and the failures are worth stating precisely because they are the entire justification for the practice.

Reviewable. A pipeline-as-code change arrives as a diff in a pull request and goes through the same review as source. Jenkins lists this as a headline benefit: pipeline-as-code enables “Code review/iteration on the Pipeline (along with the remaining source code)” (Jenkins, Pipeline). A UI change, by contrast, is applied by one person clicking Save, with no reviewer and no diff — a silent, unaudited mutation of production-critical infrastructure.

Versioned and auditable. The file is in Git history, so every pipeline change has an author, a timestamp, a message, and a SHA; Jenkins calls this the “Audit trail for the Pipeline” (Jenkins). “Who changed the deploy step and when, and why?” is answered by git blame. In a UI-configured system it is answered by “nobody knows.”

Revertible. A bad pipeline change is undone with git revert — the same mechanism as the broken-build revert. A UI misconfiguration has no revert; you must remember and re-enter the previous settings.

Single source of truth. Jenkins names this explicitly: a “Single source of truth for the Pipeline, which can be viewed and edited by multiple members of the project” (Jenkins). The pipeline is not locked inside one server’s database; it is in the repo, clonable and portable.

Reproducible and commit-pinned. Because the engine runs the pipeline as it existed at the built commit, an old build reproduces with its contemporary pipeline — not whatever the UI happens to say today. This is the same guarantee that makes build-once meaningful: the how of the build is versioned alongside the what.

Branchable and testable. A pipeline change can be developed on a branch and validated by the pipeline itself before it merges. Jenkins’ multibranch model “automatically creates a Pipeline build process for all branches and pull requests” (Jenkins), so a PR that edits the Jenkinsfile runs its own edited pipeline — you see the change work before you adopt it.

flowchart LR
    subgraph CLICK["Click-configured (UI)"]
        direction TB
        U1["Edit in web UI"] --> U2["Click Save"]
        U2 --> U3["Silent mutation<br/>no diff · no review<br/>no history · no revert"]
    end
    subgraph CODE["Pipeline as code"]
        direction TB
        C1["Edit pipeline file"] --> C2["Open PR"]
        C2 --> C3["Reviewed diff · CI-tested<br/>SHA-pinned · git revert<br/>branch-scoped"]
    end
    U3 -.->|"the gap"| C3

What this shows and the insight to take: the two columns are the same intent (change the pipeline) with radically different guarantees. The insight is that pipeline-as-code doesn’t add capability to the pipeline — it adds governance: review, audit, rollback, and reproducibility that a UI-mutated pipeline structurally cannot have. That governance is the whole point.

A Worked Example — Line-by-Line

A minimal but realistic GitHub Actions workflow, annotated:

name: CI                          # human-readable workflow name, shown in the UI
on:                               # the EVENT triggers — when this workflow runs
  push:
    branches: [main]              # run on every push to main (post-merge verification)
  pull_request:                   # AND on every PR update (the merge gate — see PR gates note)
 
jobs:
  build-and-test:                 # a JOB id; jobs run in parallel unless they 'needs:' each other
    runs-on: ubuntu-latest        # the RUNNER: an ephemeral GitHub-hosted Ubuntu VM
    steps:
      - uses: actions/checkout@v4  # STEP 1: invoke a reusable ACTION to check out the commit
      - uses: actions/setup-go@v5  # STEP 2: another action — installs a pinned Go toolchain
        with:
          go-version: '1.24'       # 'with:' passes INPUTS to the action
      - run: go build ./...        # STEP 3: 'run:' executes a shell SCRIPT instead of an action
      - run: go test -race ./...   # STEP 4: the actual test gate; a non-zero exit fails the job

Reading it as the vocabulary above: the on: block is the event binding; build-and-test is a job pinned to a runner (runs-on); each - uses: or - run: is a step that either invokes a reusable action (actions/checkout@v4 — note the @v4 version pin) or runs an inline script. The @v4 pin is itself a version-control discipline: the action is a dependency, and pinning it (ideally to a commit SHA for security, per DevSecOps and Supply Chain Security MOC) keeps the build reproducible. Every element of the run is declared in the file — nothing is hidden in server state.

Reuse — Reusable, Templated, and Composite Pipelines

The moment an organization has more than a handful of repositories, copy-pasting the same 200-line workflow into each is unsustainable: a fix must be applied N times and drifts immediately. Every mature system therefore provides a reuse mechanism, and understanding them is what separates a toy pipeline from a maintainable one.

GitHub Actions — reusable workflows and composite actions. A reusable workflow is “a YAML-formatted file, very similar to any other workflow file,” distinguished by including workflow_call in its on: triggers (GitHub, Reuse workflows). A caller invokes it at the job level with uses: — either {owner}/{repo}/.github/workflows/{file}@{ref} for a shared repo or ./.github/workflows/{file} locally — passing data via with: (inputs) and secrets: (or secrets: inherit). Nesting is capped at ten levels. This lets an organization define a single “build-and-publish” workflow once and call it from every repository:

# caller workflow in any repo
jobs:
  ship:
    uses: my-org/ci-workflows/.github/workflows/build.yml@v2   # reusable workflow, version-pinned
    with:
      language: go                                             # input to the reusable workflow
    secrets: inherit                                           # pass all caller secrets down

A composite action is the finer-grained sibling — it bundles several steps into one reusable step-level unit, whereas a reusable workflow bundles whole jobs.

GitLab CI — include and extends. GitLab’s primary reuse keyword is include, which pulls external YAML into the pipeline: include:local (same repo), include:project (another GitLab project), include:remote (an HTTP(S) URL), and include:template (a built-in template like Auto-DevOps.gitlab-ci.yml) (GitLab, Includes). Included files deep-merge, with “the last included file taking precedence,” and extends plus YAML anchors let one job inherit and override another’s definition. A central platform team can thus own a canonical include:project template that dozens of application repos pull in and lightly customize — the fix-once, apply-everywhere property that copy-paste lacks.

# .gitlab-ci.yml in an application repo
include:
  - project: 'platform/ci-templates'      # pull the org's canonical pipeline...
    file: '/build.yml'
    ref: v3                               # ...pinned to a version
stages: [build, test, deploy]

Jenkins — Shared Libraries. Jenkins factors common Jenkinsfile logic into Shared Libraries — versioned Groovy code in a separate repo, loaded via @Library, so a Jenkinsfile can call standardBuild() instead of inlining the steps (Jenkins, Pipeline — extensibility via shared libraries).

Tekton — the Task catalog. Because Tekton Tasks are Kubernetes CRDs, a Task (e.g. git-clone, buildah) is an addressable, reusable object; the Tekton Catalog / Artifact Hub distributes them, and a Pipeline references existing Tasks by name rather than redefining their steps (Tekton, Concepts).

flowchart TD
    CENTRAL["Central pipeline definition<br/>(reusable workflow / include template /<br/>shared library / Tekton Task)"]
    CENTRAL --> R1["Repo A pipeline<br/>calls it + local inputs"]
    CENTRAL --> R2["Repo B pipeline<br/>calls it + local inputs"]
    CENTRAL --> R3["Repo C pipeline<br/>calls it + local inputs"]
    FIX["Fix / policy change<br/>made ONCE"] --> CENTRAL
    FIX -.->|"propagates to all callers<br/>on their next run"| R1
    FIX -.-> R2
    FIX -.-> R3

What this shows and the insight to take: reuse inverts the maintenance cost — a change to the central definition propagates to every caller instead of requiring N edits. The insight is that reuse is what makes pipeline-as-code scale: without it, “config in the repo” degrades into “the same config copy-pasted into 50 repos, all subtly different,” which is drift by another name. Version-pinning the reference (@v2, ref: v3) is essential — otherwise a central change silently breaks every caller at once.

Beyond YAML — HCL, Starlark, and Dynamically Generated Pipelines

Pipeline-as-code does not require YAML; the “code” can be a real programming language, which matters when pipelines need logic that declarative YAML expresses awkwardly. Bazel and some CI systems use Starlark (a Python dialect) so the pipeline is computed by a program. Buildkite dynamic pipelines go further: a script “written in any language (Bash, Python, Ruby, Node.js, Go, …) generates pipeline steps as YAML or JSON output,” which is uploaded to the running build via buildkite-agent pipeline upload (Buildkite, Dynamic pipelines). The generator can “inspect repository structure, read build context, make conditional decisions based on branch or changed files,” and emit exactly the steps that run — so the pipeline adapts per build without hardcoding every case. This is the far end of the spectrum: the pipeline is not a static file but a program that outputs the pipeline, still version-controlled in the repo.

Uncertain

Verify: the specific claim that Starlark is used as a CI pipeline definition language (as opposed to Bazel BUILD/.bzl build files). Reason: Starlark’s canonical role is Bazel build configuration; its use as a CI pipeline DSL (e.g. in some internal or hosted systems) is asserted here from general knowledge and the MOC’s framing, not pinned to a fetched primary for a named CI product. To resolve: cite a specific product’s docs that use Starlark for pipeline definition, or narrow the claim to “build configuration” only. #uncertain

Common Misunderstandings and Failure Modes

• “Pipeline-as-code means the pipeline is trivially portable between vendors.” No — the file is portable in the sense of being in Git, but the schema is vendor-specific. A .gitlab-ci.yml does not run on GitHub Actions; a Jenkinsfile is Groovy. Migrating vendors means rewriting the pipeline, even though both are “as code.” The portability is of governance (review/version/revert), not of syntax.
• “YAML is declarative, so pipelines can’t have logic bugs.” They very much can — conditional if:/rules: expressions, matrix expansions, and needs: graphs are logic, and a wrong condition silently skips a test gate. The DAG semantics are owned by Pipeline DAGs Stages and Gates; the point here is that pipeline code needs review because it has logic.
• “Put everything in one giant workflow file.” A monolithic pipeline file becomes as unmaintainable as monolithic code. The reuse mechanisms above exist precisely to decompose it — but unpinned reuse (@main instead of @v2) reintroduces instability, because a central change can break every caller without warning. Pin your reused pipelines like you pin dependencies.
• “The pipeline file and the app must always share a repo.” Usually yes (that co-location is the reproducibility win), but reusable/included pipelines deliberately live in separate platform repos. The invariant that matters is that every pipeline definition is version-controlled somewhere reviewable — not that it sits beside the app.

Production Notes

Pipeline-as-code is now the default across every major CI/CD platform, and the trajectory is uniform: classic Jenkins freestyle (UI) jobs gave way to Jenkinsfile; the entire GitHub Actions / GitLab CI generation was born file-first; and Kubernetes-native systems (Tekton, Argo Workflows) express pipelines as CRDs so they inherit kubectl apply, GitOps reconciliation, and RBAC for free. The load-bearing practice in a real organization is centralized, version-pinned reusable pipelines owned by a platform team, with application repos calling them and supplying only their local inputs — this is what keeps a fleet of hundreds of pipelines consistent and patchable from one place. For interviews and design reviews, the crisp framing is: pipeline-as-code is not about the file format — it is about subjecting the build/deploy process to the same review, versioning, rollback, and reproducibility guarantees as the application it ships. Everything downstream — DAG execution, runners, Matrix Builds — is a feature of that versioned file.

See Also

• Pipeline DAGs Stages and Gates — how the jobs declared in the pipeline file actually execute: stages vs needs: DAG, gates and conditional execution
• GitHub Actions — the concrete YAML-workflow engine used as the reference model above (jobs/steps/actions, reusable workflows, Marketplace)
• GitLab CI CD — .gitlab-ci.yml, include/extends reuse, the needs: DAG, integrated runners
• Jenkins — the Jenkinsfile, declarative vs scripted syntax, shared libraries, the controller/agent model
• Tekton — Kubernetes-native pipeline-as-code: Tasks/Pipelines as CRDs executed as pods
• Runners Agents and Executors — the machines that execute the jobs a pipeline file declares
• Stop the Line and the Broken Build Rule — the revert discipline that pipeline-as-code makes a plain git revert
• Build Once Promote the Artifact — why a commit-pinned pipeline reinforces build reproducibility
• Continuous Integration and Delivery MOC — parent map (see §5, Pipeline Machinery)