Skip to main content

Deployment topology

This page lists the hosts by role, the CapRover and swarm layout on the core host, the networks, the published ports and why each is published or not, and the ECR images.

The source documents call hosts app-01 to app-04. Those are labels from the infrastructure repository (hostinger). Addresses, tailnet names and domains are deliberately not reproduced here.

Hosts by role​

RoleSource labelStateWhat runs there
Core hostapp-03LiveA second, independent single-node CapRover (own catalog file, own drift gate arguments): Postgres, gateway, core session worker, API
Temporal hostapp-01Live, shared with four other productsThe shared Temporal cluster (temporalio/auto-setup:1.23.1.1), namespace agentforge-prod with 168 h retention. No TLS or auth; tailnet ACL only
Throwaway test hostapp-02Torn down 2026-11-13The earlier throwaway session-path test. Crowded production host; AgentForge was removed
Spare core hostapp-04Not builtA further 4 vCPU host that "may take AgentForge later by re-pointing the same entries" (ADR 0004)
Exec host (sandbox host)noneNot builtNeeded for CodeCrucible and any non-local sandbox work: Linux, Docker, verified runsc, a tailnet grant to Temporal only (precondition PRE-1)
Laptop and VPS fleet workersnoneBuilt, cannot reach the gateway yetmacOS OrbStack or Linux containers running apps/worker target exec

Why a dedicated core host (ADR 0004): app-02 was crowded, and sharing a host with a public-facing product was rejected for blast radius because AgentForge holds the model key and runs model-driven code. Decision date: 2026-11-14/15, accepted by the owner. Rejected alternatives: deploy on app-02 as first planned, compose plus ansible outside the catalog (a second deploy method), and sharing app-03 with another product.

CapRover and swarm layout on the core host​

All four services are CapRover apps of type worker (the API is a worker-type app with a published port and no domain), one instance each (instance_count: 1). Service names resolve on the private overlay by Swarm service name, for example agentforge-postgres:5432, unless an opt-in network_aliases feature is used (hostinger draft PR #80 at the time of the 2026-11-13 findings; whether it merged is unverified).

ServiceImage repoEntryReplicasPort(s)Published
agentforge-postgresportfolio/agentforge-postgres (mirror of postgres:16 by digest)upstream image15432no
agentforge-gatewayportfolio/agentforge-gatewaynode dist/server.js1 (state is in memory)data 8787, control 8788none at launch (ADR 0005)
agentforge-workerportfolio/agentforge-workernode dist/main.js (default Dockerfile stage)1none (liveness by process only)no
agentforge-apiportfolio/agentforge-apinode dist/server.js1 (rate limits and semaphore are in memory)container 3000host port 3100, tailnet reachable

Replica rules, from the runbooks:

  • API: replicas must stay at 1. The rate limits, the auth-failure throttle, MAX_CONCURRENCY and the in-flight abort set live in process memory. A second replica doubles every limit. Spend caps are in Postgres and stay correct.
  • Gateway: replicas must stay at 1. Registrations, job tokens, subject tokens, rate windows and the per-call settleCall guards are in memory. A restart drops them all and fails closed (401 until callers re-register; unsettled debits stay at the worst case).
  • Worker: replicas 1 on the core host. Turn concurrency is WORKER_CONCURRENCY (default 2).

Stop grace periods the runbooks ask for:

ServiceMinimum stop graceDerivation
API10 sSHUTDOWN_GRACE_MS (default 8000) plus 2 s
Gateway12 sGATEWAY_SHUTDOWN_GRACE_MS (default 10000) plus 2 s
Worker8 sWORKER_SHUTDOWN_GRACE_MS (default 5000) plus ABORT_GRACE_MS (1 s) plus 2 s

Container hardening common to the images: non-root user (uid 10001 for the swarm secret match, see Secrets flow), PID 1 is tini, /app read-only, and a Docker HEALTHCHECK. The API healthcheck fetches http://127.0.0.1:$PORT/health every 10 s (timeout 3 s, start period 20 s). Worker images use a process liveness check every 30 s.

Networks​

  • agentforge-net is an encrypted, attachable overlay the apps role creates if missing. An --internal network has to be created by hand first (owner step).
  • The networks: list in the catalog is the complete attachment list written to TaskTemplate.Networks. Services that nginx must reach list captain-overlay-network plus the private net; private services list only the private net. The API keeps both in the design; the live attachment of the API is not recorded in the dossiers (Unverified).
  • The gateway is on agentforge-net only. This is a precondition for GATEWAY_CONTROL_BIND_ANY=private-network-only: the control listener binds 0.0.0.0 (the image has no default for GATEWAY_CONTROL_HOST), which is safe only when the gateway sits on the private overlay alone.
  • The core host is a single swarm node, so isolation (not encryption) is the control between services on it.

Published ports and why​

PortServicePublishedReason
3000 (host 3100)APIyes, tailnet reachable, no public domainClients reach the API directly over the tailnet. TLS and a public domain are not set up
8787Gateway data listenerno at launchA published swarm port joins the shared ingress overlay, which would expose 8788 to every other published service (ADR 0005)
8788Gateway control listenerneverControl tokens are the second barrier, not the first
5432PostgresnoReached by service name on agentforge-net
7233Temporal (remote)on the Temporal host, tailnet onlyA ufw rule allows it on the tailscale interface only

The gateway decision in detail:

The fix path (design option A, docs/design-gateway-fleet-exposure.md), all steps still open except step 1:

  1. Gateway setting GATEWAY_CONTROL_BIND_CIDR: coded on main (commit 1da8928), not released.
  2. Hostinger pins the agentforge-net subnet to a known value.
  3. Catalog PR: re-add published_ports for 8787 on the gateway only; drop GATEWAY_CONTROL_BIND_ANY and GATEWAY_CONTROL_HOST from the SSM set; add GATEWAY_CONTROL_BIND_CIDR.
  4. Tailnet ACL grant from the worker tag to the core host tag on tcp:8787, plus the ufw tailnet route allow for 8787 (the owner's action).
  5. Fleet worker GATEWAY_URL as plain http over the tailnet, with that exact host in GATEWAY_HTTP_ALLOWED_HOSTS.

The release tag that carries step 1 was declined by the owner on 2026-11-16 (wait until more of the quality work is done).

Tailnet access rules​

SourceDestinationPortState
Worker tag (tag:agentforge-worker)Temporal host7233ACL applied by the owner 2026-11-13, one grant, no wildcard rules. Not read back by an agent
Worker tagCore host8787Not granted; waits for the CIDR design
Application-host containersTemporal host7233Measured open from the throwaway host on 2026-11-13 (host, captain-overlay container, default-bridge container): no ufw, ACL or route change was needed. The core host reached Temporal in the smoke runs

Temporal 1.23 OSS cannot restrict by task queue. A fleet host that reaches 7233 can poll any queue of any namespace and fail its tasks (risk R-5). The ACL is the only control until Temporal auth is enabled. The worker tag can therefore reach all namespaces of the shared cluster; payloads stay protected by the application codec (TEMPORAL_PAYLOAD_KEYS). This is ADR 0003 (dossier 2026-11-13).

ECR images​

RepositoryContentPushed byTags that exist
portfolio/agentforge-apiapps/api/Dockerfiledocker-release on a tag pushv0.1.0, v0.2.0
portfolio/agentforge-gatewayapps/gateway/Dockerfile, baked /policies, empty /registrysamev0.1.0, v0.2.0
portfolio/agentforge-workerapps/worker/Dockerfile, default stage (session worker)samev0.1.0, v0.2.0
portfolio/agentforge-postgresmirror of postgres:16 pinned by digestdocker-release manual dispatch, mirror jobv0.1.1 (plus an unused index-wrapped v0.1.0)
portfolio/agentforge-temporalmirror of temporalio/auto-setup:1.23.1.1samev0.1.1 (plus an unused index-wrapped v0.1.0); not used on the core host, which uses the shared cluster

Not released through the workflow: the Dockerfile targets exec (fleet exec worker), factory, quality-exec, the sandbox image apps/sandbox-runner and the quality image apps/worker/quality-image. The exec image is built locally on each laptop (docker build -f apps/worker/Dockerfile --target exec). Builds are linux/amd64, no provenance, no SBOM, no latest tag.

The ECR repositories are immutable and have a lifecycle policy with a v prefix rule. The registry account and region are not reproduced here.

Release v0.2.0 ran as workflow run 43744340182. Its first attempt pushed the worker and failed on the api and gateway with no space left on device; only the two failed jobs were re-run. That incident produced the 20 GB preflight guard described in Release process.

Host-level facts that matter​

  • Docker publishes on 127.0.0.1 only on a sandbox host: without gateway_mode_ipv4=isolated, a port published on 0.0.0.0 is reachable from an --internal network (docs/deploy-sandbox.md, probe 0l).
  • The gateway on a sandbox host runs with net.ipv4.ip_forward=0 and net.ipv4.conf.all.forwarding=0. The production deploy has to reproduce both sysctls and check them (cat /proc/sys/net/ipv4/ip_forward /proc/sys/net/ipv4/conf/all/forwarding prints 0 0). The driver does not set them.
  • The captain-overlay-network is shared by about 65 production apps and is unencrypted. This is why the production requirement is a dedicated overlay.

Unverified​

  • Tailnet-inbound access to a published port on the core host (a stale ufw route-deny blocked it on the test, and Docker's anti-spoof rule blocked a shared overlay on the throwaway host).
  • Interface naming and ordering inside a swarm task with several networks.
  • Whether CapRover accepts a pinned agentforge-net subnet.
  • MagicDNS inside OrbStack containers (use the tailnet address if it fails).
  • runsc anywhere (spike S3), and any real exec host.

Sources​

  • docs/sessions/2026-11-15-app03-deploy-and-smoke/ARCHITECTURE.md, README.md, SUMMARY.md, LESSONS.md
  • docs/sessions/2026-11-15-app03-deploy-and-smoke/adr/0004-app03-dedicated-agentforge-host.md, 0005-gateway-publishes-no-port-at-launch.md, 0006-postgres-persistent-volume.md
  • docs/sessions/2026-11-13-agentforge-app02-test-and-worker-fleet/adr/0003-temporal-acl-only-and-baked-gateway-policy.md
  • docs/design-gateway-fleet-exposure.md
  • docs/deploy-app02-test.md (services, images, networks sections)
  • docs/deploy-api-coolify.md, docs/deploy-gateway-coolify.md, docs/deploy-worker-coolify.md, docs/deploy-worker-fleet.md, docs/deploy-sandbox.md
  • docs/p2-findings.md ("App-02 test deployment", "Production deploy: answers from the fleet sessions", "Temporal route app-02 -> app-01: measured", "App-03 production deploy and smoke test")
  • apps/api/Dockerfile, apps/gateway/Dockerfile, apps/worker/Dockerfile
  • .github/workflows/docker-release.yml
  • secrets.manifest.yml