Skip to main content

Release process

A release builds the api, gateway and session-worker images and pushes them to immutable ECR repositories. Releases are maintainer actions: the owner approves a tag, and a deploy is a separate step in the hostinger repo. Source: .github/workflows/docker-release.yml.

Release history​

TagWhatNotes
v0.1.0api, gateway, worker imagesBuilt by CI run 43298582379 for the throwaway test; OIDC trust worked for the repo. The first mirror attempt also left an index-wrapped v0.1.0 in the postgres and temporal repos (immutable, unused)
v0.1.1mirrors of postgres and temporalCI run 43317947163, FROM-only build after the imagetools create index problem
v0.2.0api, gateway, worker imagesOn commit 8d7a615, run 43744340182. The first attempt pushed the worker and failed the api and gateway with no space left on device; only the two failed jobs were re-run. This is what runs on the core host
v0.2.1 (candidate)would carry GATEWAY_CONTROL_BIND_CIDR (bead .93)The owner declined the tag on 2026-11-16: wait until more of the quality work is done, so one release carries both. Nothing newer than v0.2.0 is released

Digests of v0.2.0 (from the 2026-11-15 dossier): api sha256:ed98edd3...5bc26, gateway sha256:f3a96f34...d799, worker sha256:fe84adf4...2726 (abbreviated in the source).

Triggers​

TriggerInputsRuns
Push of a tag matching v*.*.*nonebuild for api, gateway, worker
workflow_dispatchimage_tag (required, an unused tag), build_apps (default true), postgres_ref and temporal_ref (digest-pinned, default empty = do not mirror)build if build_apps; mirror if either ref is non-empty

Commands:

# Release
git tag v0.1.0 && git push origin v0.1.0
# Verify the build path with a throwaway tag (ECR is immutable: a failed tag is burned)
gh workflow run docker-release -f image_tag=check-1
# Mirror pinned upstream images
gh workflow run docker-release -f image_tag=v0.1.0 -f build_apps=false \
-f postgres_ref='postgres:16@sha256:<digest>' -f temporal_ref='temporalio/auto-setup:1.23.1.1@sha256:<digest>'

The mirror tag must be vX.Y.Z so the ECR lifecycle v prefix rule keeps it.

Build job​

Permissions are contents: read and id-token: write. Runs on the ci-runners group (self-hosted macOS ARM64), timeout 30 minutes, fail-fast: false, matrix service: [api, gateway, worker]. Actions are pinned by commit SHA.

StepRule and reason
Tag on maingit merge-base --is-ancestor "$GITHUB_SHA" origin/main, on tag pushes only. A tag pushed from a feature branch would otherwise publish untested code under a version number. Manual dispatch lets the operator pick the ref, so the guard does not run there
Preflight disk guardPrints uptime, df -h /, docker system df and docker or buildkit processes by name only (never command lines). Fails with less than 20 GB free on the runner; prune Docker (docker buildx prune -af; docker image prune -af) and re-run when free disk is below 20 GB
AuthGitHub OIDC only, no static keys and no other secrets. The role is scoped to push and pull on portfolio/* repositories
Tagstype=ref,event=tag plus type=raw,value=<image_tag> for dispatch; latest=false from the first commit because immutable repos break the second release on an automatic latest
Platformlinux/amd64 explicitly (the runner is ARM64, the deploy host pulls amd64)
Provenance and SBOMboth off: they turn the push into a multi-manifest index some pullers reject
Dockerfileapps/<service>/Dockerfile, context the repo root; apps/worker builds its default (last) stage, the session worker. The factory, exec and quality-exec targets are not released here
CacheGitHub cache, scope=<service>, mode=max

Immutable tags​

The ECR repositories are immutable. A failed push burns the tag, so:

  • Verify the build path with a throwaway tag (check-1) before a real vX.Y.Z.
  • Never retype a digest. Extract it from the run log and compare programmatically (a mistyped digest was caught that way).
  • A full runner disk can burn a tag halfway: that is why the 20 GB preflight exists.
  • Mirror dry-runs: check the manifest type before the real tag. docker buildx imagetools create always wraps even one manifest in an OCI index, which immutable ECR keeps forever.

Mirror job​

Runs on dispatch only, timeout-minutes: 15, matrix image: [postgres, temporal]. For each non-empty ref:

  1. The ref must match ^[a-z0-9./_-]+(:[A-Za-z0-9._-]+)?@sha256:[0-9a-f]{64}$, otherwise ref must be pinned by digest.
  2. The tag must match ^v[0-9]+\.[0-9]+\.[0-9]+$, otherwise mirror tag must be vX.Y.Z.
  3. A FROM-only build of the digest-pinned upstream pushes portfolio/agentforge-<image>:<tag>: same layers, one linux/amd64 manifest, no attestations.
  4. The job inspects the pushed manifest with docker buildx imagetools inspect --raw and fails with mirror must be one single-platform manifest if it finds a manifests array.

From image to running service​

  • The catalog image: string must equal the pushed tag exactly, for example <registry>/portfolio/agentforge-api:v0.2.0.
  • Migrations are never run at boot. If a release adds a migration, run node dist/migrate.js in the api container after the apply (see Persistence and backups). Releases that carry migrations 0006_quality and 0007_quality_submissions ship the quality code dormant, and the migration is still a manual step.
  • The drift gate fails the whole fleet's applies while a merged entry has no live app, so merge right before the apply and keep the revert path ready.
  • Policy changes for the gateway are image changes: a new image tag and a re-pin (ADR 0003).
  • Rollback is re-pointing the catalog entry to the previous immutable tag, since tags are never overwritten. There is no documented rollback of a migration (Unverified).

Release checklist​

  1. CI on the tip commit is green on main. If CI is queued or failed on load, re-run failed jobs when the machine is quiet; a timing flake is not a release blocker, a real regression is.
  2. Owner approves the tag (releases are owner-gated; the .93 tag was declined 2026-11-16).
  3. Free disk on the runner at least 20 GB (the workflow checks).
  4. Optionally dispatch image_tag=check-N first.
  5. git tag vX.Y.Z && git push origin vX.Y.Z, with the commit on main.
  6. Compare the three pushed digests with the run log.
  7. Hostinger PR with the exact tags, owner go, apply per service, migrate, smoke.

The repository defines no release automation beyond this workflow.

Not covered​

  • The exec image target is built per laptop and is not in the release workflow.
  • The quality image is built by scripts/quality-image.test.sh locally (keyed by lockfile hash) and run by digest only on an exec host that does not exist yet.
  • Provenance and SBOM attestations are deliberately off.

Sources​

  • .github/workflows/docker-release.yml
  • .github/workflows/ci.yml (runner group, disk snapshots)
  • docs/sessions/2026-11-15-app03-deploy-and-smoke/SUMMARY.md, LESSONS.md
  • docs/sessions/2026-11-13-agentforge-app02-test-and-worker-fleet/SUMMARY.md, LESSONS.md
  • docs/sessions/2026-11-16-codecrucible-build/FOLLOWUPS.md, README.md
  • docs/sessions/2026-11-16-codecrucible-and-gateway-cidr/README.md
  • docs/p2-findings.md ("App-02 test deployment")
  • docs/deploy-app02-test.md (images and order sections)
  • apps/api/Dockerfile, apps/gateway/Dockerfile, apps/worker/Dockerfile