vault

Image Signing with Sigstore

10 min read

Image Signing with Sigstore

A container image is just bytes pulled from a registry by tag — and a tag is mutable, so “the image our cluster runs” and “the image our build pipeline produced” are not the same thing unless something cryptographically ties them together. Sigstore is the CNCF project that provides that tie (Sigstore docs). It comprises three components: cosign (the CLI that signs and verifies container images and other artifacts), Fulcio (a certificate authority that issues short-lived signing certificates bound to an OIDC identity), and Rekor (a public, append-only, tamper-evident transparency log of signing events). Together they enable keyless signing: no long-lived private key to store, leak, or rotate — the identity (a GitHub Actions workflow, a user’s Google account) is the root of trust. The signature lives in the OCI registry next to the image, and a Kubernetes admission policy can reject Pods whose images are not signed by a trusted identity, turning Sigstore into a supply-chain enforcement point.

Mental Model

Traditional code signing has a chronic problem: the private signing key. It must be generated, stored securely, distributed to whoever signs, rotated periodically, and revoked if compromised. Most organizations do this badly — keys end up on laptops, in CI environment variables, in shared vaults. The key is the identity, and keys leak.

Sigstore’s central insight is to invert this: don’t make the key the identity — make the identity the identity. In keyless signing, cosign generates an ephemeral keypair in memory, uses it once, and destroys the private key seconds later. There is no key to store because there is no persistent key. What proves “who signed this” is not key custody but an OIDC token — the same kind of token that proves you are logged into Google or that a workflow is running in a specific GitHub repository. Fulcio binds the ephemeral key to that verified OIDC identity in a short-lived certificate. Rekor records the event in a public log so that even after the ephemeral key is gone, the signature remains verifiable forever.

sequenceDiagram
    autonumber
    participant CI as Signer<br/>(CI job / developer)
    participant COS as cosign
    participant IDP as OIDC Provider<br/>(GitHub / Google / ...)
    participant FUL as Fulcio<br/>(certificate authority)
    participant REK as Rekor<br/>(transparency log)
    participant REG as OCI Registry

    COS->>COS: generate ephemeral keypair (in memory)
    COS->>IDP: obtain OIDC identity token
    COS->>FUL: ephemeral public key + OIDC token
    FUL->>FUL: verify token, bind key to identity
    FUL-->>COS: short-lived cert (~10 min)
    COS->>COS: sign image digest with ephemeral key
    COS->>REK: submit {digest, signature, cert}
    REK-->>COS: inclusion proof + log entry
    COS->>REG: push signature + cert next to image
    COS->>COS: destroy ephemeral private key

The keyless signing flow. Insight to extract: the private key exists only for the few seconds between steps 1 and 7, then is destroyed (step 10). Verifiability survives because Rekor (step 8) permanently records the digest, signature, and certificate. The certificate’s ~10-minute lifetime is irrelevant after signing — Rekor’s timestamped entry proves the signature was made while the cert was valid.

Mechanical Walk-through

cosign — sign and verify

cosign is the user-facing CLI. cosign sign <image>@<digest> runs the flow above; cosign verify <image> --certificate-identity=... --certificate-oidc-issuer=... checks it. Cosign always operates on the image digest (the content-addressed SHA-256 of the manifest), never the mutable tag — signing a tag would be meaningless because the tag can be repointed.

Fulcio — the keyless CA

Fulcio is a certificate authority with one job: take an ephemeral public key plus an OIDC identity token, verify the token against the issuer, and return a short-lived X.509 certificate (≈10 minutes) whose subject is the OIDC identity. Because the cert is so short-lived, there is no revocation problem — it expires before it could be abused. The identity in the cert is the durable fact: e.g. https://github.com/myorg/myrepo/.github/workflows/release.yml@refs/heads/main for a GitHub Actions signer, or an email for a human signer.

Rekor — the transparency log

Rekor is an append-only, cryptographically verifiable (Merkle-tree) transparency log, conceptually the same design as Certificate Transparency for TLS. Every signing event — the artifact digest, the signature, and the Fulcio certificate — is recorded as a Rekor entry with a timestamp and an inclusion proof. Rekor solves the “short-lived cert” puzzle: the cert expires in 10 minutes, but Rekor’s tamper-evident, timestamped entry proves the signature was created while the cert was valid. The public Rekor instance is monitored and offers a published availability SLO. Rekor also makes signing auditable — anyone can ask “what has identity X signed?”

TUF — distributing the root of trust

Verifiers need to trust Fulcio’s root CA and Rekor’s public key. TUF (The Update Framework) securely distributes this root of trust to cosign clients, so a verifier can be bootstrapped without manually trusting keys.

Where the signature lives

The signature and certificate are stored in the OCI registry alongside the image — as a separate OCI artifact whose tag is derived from the image digest (the sha256-<digest>.sig convention; newer setups use the OCI 1.1 referrers API). No separate signature store is needed; pulling the image and pulling its signature both hit the same registry.

Attestations and SLSA provenance

Beyond a bare signature (“identity X vouches for this image”), cosign can attach attestations — signed statements about the image. The most important is SLSA provenance (SLSA): a signed document describing how the image was built — which source commit, which builder, which parameters. A verifier can then assert not just “this is signed” but “this was built from main of our repo by our trusted CI builder.” SLSA defines escalating levels of build-integrity guarantee; signed provenance is how a consumer checks them. Attestations are themselves signed via the same Fulcio/Rekor machinery and stored alongside the image.

Verification at admission time

Signing is worthless without enforcement. In Kubernetes, a policy controller verifies image signatures during the API Server Request Flow admission phase and rejects Pods whose images fail verification. Three common enforcers:

  • Kyverno verifyImages — a verifyImages rule names trusted identities (OIDC issuer + subject); a Pod with an unsigned or wrongly-signed image is denied.
  • OPA Gatekeeper — can verify signatures via templates that call cosign’s verification logic.
  • sigstore-policy-controller — Sigstore’s own purpose-built admission controller (ClusterImagePolicy CRD), the most direct integration.

The verifier checks: does the signature match the image digest, was it made by an allowed identity, and is there a corresponding Rekor entry? All three must hold.

Configuration / API Surface

Keyless signing of a built image, then a Kyverno policy that enforces it:

# --- Sign (typically inside a CI job; OIDC token is the CI workflow identity) ---
# COSIGN_EXPERIMENTAL keyless mode is the default in modern cosign.
cosign sign \
  --yes \                                          # non-interactive (CI)
  registry.example.com/app@sha256:abc123...        # ALWAYS sign the digest, not a tag
 
# cosign: obtains OIDC token -> Fulcio issues a ~10-min cert ->
#         signs digest -> uploads entry to Rekor -> pushes .sig to the registry
 
# --- Verify (locally or in a pipeline gate) ---
cosign verify \
  --certificate-identity-regexp="https://github.com/myorg/.*" \   # allowed signer identity
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \  # allowed issuer
  registry.example.com/app@sha256:abc123...
# --- Enforce at admission with Kyverno: reject unsigned images ---
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-signed-images
spec:
  validationFailureAction: Enforce                 # reject, do not just report
  rules:
    - name: verify-app-images
      match:
        any:
          - resources:
              kinds: ["Pod"]
      verifyImages:
        - imageReferences:
            - "registry.example.com/app*"           # which images this rule covers
          attestors:
            - entries:
                - keyless:                          # keyless verification block
                    issuer: "https://token.actions.githubusercontent.com"   # required OIDC issuer
                    subject: "https://github.com/myorg/*"                   # required signer identity
                    rekor:
                      url: "https://rekor.sigstore.dev"                     # transparency log to check

Line-by-line: cosign sign operates on the digest (@sha256:...) — signing a tag is meaningless. The keyless flow needs no --key because the identity is the key. In cosign verify, --certificate-identity-regexp and --certificate-oidc-issuer are the trust assertions — “I only trust signatures from my org’s GitHub workflows.” In the Kyverno policy, verifyImages.attestors.entries.keyless declares the same trust constraints; validationFailureAction: Enforce makes a verification failure block the Pod. Without an admission policy, signatures exist but nothing checks them.

Failure Modes

  • Signed but never verified. The most common real failure: teams sign images in CI and never deploy an admission policy to check the signatures. Signing without enforcement provides zero security — it is the verification gate that matters.
  • Tag vs digest confusion. Signing or verifying a tag is unsound — the tag is mutable. Only digest-pinned references (@sha256:...) are cryptographically meaningful. A policy that verifies images but workloads that reference :latest is a false sense of security.
  • Rekor/Fulcio availability dependency. Keyless signing depends on the public Fulcio and Rekor instances (or a self-hosted equivalent). If they are unreachable, signing fails; verification that requires a live Rekor check also fails. For air-gapped or high-availability needs, organizations self-host these or use offline verification with bundled inclusion proofs.
  • Over-broad trust constraints. --certificate-identity-regexp="https://github.com/.*" trusts every GitHub repository on earth — an attacker need only sign from any public GitHub Actions workflow. Identity constraints must be scoped tightly to your org and ideally your specific workflow file.
  • Provenance ≠ vulnerability-free. A signature proves who built it and an attestation proves how — neither proves the image is free of CVEs. Signing complements, does not replace, image scanning.
  • Key compromise in key-based mode. Cosign also supports traditional key-based signing (--key); if you use it, you reinherit the long-lived-key custody problem keyless was designed to eliminate.

Alternatives and When to Choose Them

  • Notation / Notary v2 — the CNCF Notary project’s signing tool, used by Kyverno’s verifyImages as an alternative to cosign. Notary v2 is more oriented to a traditional PKI/CA trust model (named trust policies, registry-published certs) and is favored where keyless OIDC signing does not fit (regulated environments wanting explicit CA control). Choose cosign/keyless for low operational overhead and CI-native identity; choose Notation where an existing enterprise PKI is the trust anchor.
  • GPG / detached signatures — the pre-Sigstore approach: sign artifacts with a long-lived GPG key. Reinherits every key-management problem; no transparency log, no admission-time tooling. Largely superseded for container images.
  • Docker Content Trust (Notary v1) — the original Docker image-signing system; effectively deprecated in favor of Notary v2 / Sigstore.

Production Notes

  • CNCF Graduated. Sigstore is a CNCF Graduated project — a strong production-maturity signal. It is used by major package registries (npm, PyPI, Maven Central), Kubernetes’ own release artifacts, the Distroless images, GitHub Actions, and many Linux distributions.
  • The gold-standard pattern: sign in CI using the CI workflow’s OIDC identity (no secrets to manage), attach SLSA provenance as an attestation, and enforce at admission with a tightly-scoped identity constraint. The signing key never exists outside a few seconds of CI runtime.
  • Self-hosting Fulcio + Rekor + a TUF root is supported and is the route for air-gapped clusters; the public instances are convenient but introduce an external dependency.
  • Signature verification at admission adds a registry round-trip (and optionally a Rekor lookup) to Pod creation — a small but non-zero latency cost on the API Server Request Flow, and an availability dependency to account for.

See Also

Graph View

Backlinks