Skip to main content

Operations overview

This section describes how AgentForge is built, tested, released, deployed, backed up and watched. It is written from the repository runbooks, the architecture decision records and the session dossiers. Where a step has never run against real infrastructure, the page says so ("Unverified") instead of presenting it as fact.

What is deployed today​

Only the core path of AgentForge runs in production. Everything else is built and tested locally and ships dormant or is not built.

CapabilityStateEvidence
REST API (apps/api), session pathDeployed on the core host, release v0.2.0Two authenticated smoke runs completed a real turn (cost 0.00037995 USD and 0.0003812 USD)
Gateway (apps/gateway)Deployed on the core host, release v0.2.0, publishes no portSame smoke runs; ADR 0005
Core session worker (apps/worker, default target)Deployed on the core host, also polls the exec queueSame smoke runs
Postgres 16Deployed on a persistent named volume, nightly pg_dump to off-host storageRestart proof and restore drill, 2026-11-16
TemporalShared cluster on a separate host, namespace agentforge-prod, tailnet ACL onlyADR 0003 (dossier 2026-11-13)
Fleet exec workers (laptops, VPSes)Built and tested locally; cannot reach the gateway yetADR 0005, bead .93, docs/design-gateway-fleet-exposure.md
Gateway GATEWAY_CONTROL_BIND_CIDRCode on main, not released, not deployedADR 0008
Factory (software factory)Built and tested locally, not deployeddocs/deploy-factory.md
CodeCrucible (quality)Built locally, ships dormant (QUALITY_ENABLED unset everywhere)docs/deploy-quality.md, dossier 2026-11-16 second half

The names app-01, app-02, app-03 and app-04 in the repository documents are labels of hosts in a separate infrastructure repository (hostinger). This site calls them by role. See Deployment topology.

Lifecycle of a change​

Each arrow has a rule behind it:

StepRulePage
CIRuns on a self-hosted macOS ARM64 runner group ci-runners; timing tests are sensitive to loadTesting and CI
TagThe tag commit must be an ancestor of origin/mainRelease process
ReleaseECR tags are immutable; a failed push burns the tag; the workflow refuses to start below 20 GB free diskRelease process
Secrets1Password is upstream, SSM parameters are delivered by the hostinger apps roleSecrets flow
ApplyOne service per apply, with an owner go per serviceDeploy runbooks
MigrationNever at boot; run node dist/migrate.js once per deployPersistence and backups

Who does what​

Rules that appear in every runbook:

  • An agent never prints a secret value. Output filters print names, counts and statuses only.
  • A live change (apply, restart, teardown, release tag) needs the owner's go, given in the session that performs it.
  • The hostinger repo owns the catalog and the apply. This repo does not apply infrastructure changes.
  • "Exit 0" is not state. The apps role reports changed=0 even when it deployed. Prove a deploy with the drift gate and docker service ps.

Where the runbooks live​

TopicPageRepository source
Hosts, networks, ports, imagesDeployment topologydocs/sessions/2026-11-15-app03-deploy-and-smoke, ADRs 0004 to 0006
Per-service deploy orderDeploy runbooksdocs/deploy-*.md
SecretsSecrets flowsecrets.manifest.yml, scripts/deploy/make-app03-secrets.sh
Postgres and backupsPersistence and backupsADR 0006, docs/p2-findings.md
Tests and CITesting and CI.github/workflows/ci.yml, turbo.json, scripts/*.test.sh
Logs, alerts, kill switchObservability and alertspackages/agent-core, apps/api/src
ReleasesRelease process.github/workflows/docker-release.yml
Dossier indexSession dossiersdocs/sessions/*/README.md

Stale or contradicting statements in the repository docs​

The code and the later records win. These statements are out of date and the runbooks below use the corrected version.

StatementWhereCorrected by
"Every step below is UNVERIFIED. Nothing here has been run against a Coolify host"docs/deploy-api-coolify.md, docs/deploy-gateway-coolify.md, docs/deploy-worker-coolify.mdProduction runs on CapRover (ADR 0004); the api, gateway and worker were deployed and smoke-tested. The Coolify-specific checks (proxy hop count, SSE through the proxy, restore drill) remain unverified
ADR 0006 "not yet applied"docs/sessions/2026-11-15-app03-deploy-and-smoke/adr/0006-postgres-persistent-volume.mdVolume applied and restart proof passed on 2026-11-16
Manifest header "for the app-02 throwaway test"secrets.manifest.ymlThe service entries describe the app-03 layout (gateway-routed, six items)
docs/deploy-app02-test.md runbook stepsthat fileThe file marks itself OUTDATED; the test was torn down
GLOSSARY terms marked "planned" for sessions, gateway and factoryGLOSSARY.mdThese are built (see Glossary)

Sources​

  • docs/sessions/2026-11-15-app03-deploy-and-smoke/README.md, SUMMARY.md, ARCHITECTURE.md
  • docs/sessions/2026-11-16-codecrucible-and-gateway-cidr/README.md, SUMMARY.md
  • docs/sessions/2026-11-16-codecrucible-build/README.md
  • docs/sessions/2026-11-15-app03-deploy-and-smoke/adr/0004-app03-dedicated-agentforge-host.md
  • docs/sessions/2026-11-15-app03-deploy-and-smoke/adr/0005-gateway-publishes-no-port-at-launch.md
  • docs/sessions/2026-11-15-app03-deploy-and-smoke/adr/0006-postgres-persistent-volume.md
  • docs/sessions/2026-11-16-codecrucible-and-gateway-cidr/adr/0008-gateway-control-bind-cidr.md
  • docs/deploy-api-coolify.md, docs/deploy-gateway-coolify.md, docs/deploy-worker-coolify.md, docs/deploy-app02-test.md
  • docs/p2-findings.md ("App-03 production deploy and smoke test", "App-03 Postgres persistent volume")
  • secrets.manifest.yml
  • GLOSSARY.md