Skip to main content

Job lifecycle

A factory job is one Temporal workflow, factoryJobWorkflow, with workflow id factory-<jobId> on task queue agentforge-factory (FACTORY_TASK_QUEUE). Postgres holds job, step, artifact and approval state. The workflow is deterministic orchestration over activities; its payloads are ids, digests (SHA-256 hex), small enums and counts. No task text, patch bytes or policy content cross Temporal.

States​

Status values (FactoryStatus, stored in factory_jobs.status, enforced by the check constraint factory_jobs_status_valid):

queued, planning, awaiting_plan_approval, coding, validating, testing, reviewing, awaiting_pr_approval, opening_pr, pr_opened, failed, cancelled. The terminal states are pr_opened, failed and cancelled; the constraint factory_jobs_closed_iff_terminal makes closed_at non-null exactly for those.

awaiting_plan_approval exists only when the repo policy's gates contain factory.plan. factory.pr is a code constant: effectiveGates always includes it; a policy can add factory.plan but can never remove the PR gate.

Failure codes (FactoryErrorCode, factory_jobs.error_code, required exactly when status is failed by the check factory_jobs_error_code_valid):

CodeMeaning
max_iterationsno acceptable iteration within maxIterations
budget_exhaustedthe per-job budget has no room above the step floor
approval_rejectedan approver rejected a gate
approval_expiredthe gate TTL ran out with no decision
agent_faileda step could not be claimed, or the runner failed in a way that ends the job
pr_closed_by_humana PR for this job's branch exists and was closed; never reopened or recreated
repo_too_largebase tree over 256 MiB or 50,000 entries
internal_errorinfrastructure failure, validation mismatch on re-derivation, or Temporal start failure at the API
disabledthe kill switch is set

The API maps stored codes that are also ErrorCode values through Table S before returning them in FactoryJobView.errorCode; job outcome codes such as max_iterations pass unchanged (REST API).

Intake​

POST /v1/factory/jobs (documented on REST API) writes the job row, then starts the workflow by type name with WorkflowIdReusePolicy.REJECT_DUPLICATE. An Idempotency-Key makes the pair (client, key) unique (factory_jobs_client_idempotency_key_unique); createJob runs under the advisory lock agentforge:factory:<client>, compares the stored request_sha256, and counts the client's open jobs (closed_at IS NULL) against FACTORY_MAX_ACTIVE_PER_CLIENT. If the Temporal start fails, the API closes the job failed:internal_error and abandons the ledger prefix factory-<jobId>/. No route touches the runtime or ledger admit, so a Temporal outage writes no ledger row.

The workflow​

Acceptance rule inside the loop (accepted = review is approve and (tests passed or tests not required)):

Step resultWorkflow reaction
runCode ignored (job closed, cancel requested, attempts exhausted)job fails agent_failed
runCode failed with budget_exhausted, disabled or repo_too_largejob ends with that code
runCode failed with anything else (patch rejection reason, artifact_invalid, empty_patch, agent_failed)iteration consumed, next iteration
runTests ignoredjob fails agent_failed
runTests passed, failed or timeoutrecorded; review runs regardless
runReview ignoredjob fails agent_failed
runReview failed with a job-ending codejob ends with that code
runReview approve or changes_requestedaccepted computed as above
loop ends without acceptedjob fails max_iterations

The reviewer verdict is advisory: a forged approve only reaches the human gate. The test result is a coder-influenced signal, not a verdict; the repo policy's tests.required (default true) decides whether a non-passing test blocks acceptance.

Iteration semantics (plan 4.2): iteration i is one coder run. Rejected patches, git apply --check failures, test exit codes other than 0 and review changes_requested feed the next coder prompt as wrapped untrusted text, using only reason codes, a test log tail of at most 8 KiB and review text of at most 8 KiB. The next coder's workspace is the base tree plus the last patch that passed host validation, applied uncommitted; the new patch is cumulative against baseSha. The carried patch is re-validated on every iteration, so one poisoned iteration cannot pass unchecked bytes forward. The details of validation are on Patch policy.

Activity options​

Activity groupActivitiesstartToCloseHeartbeatRetry
database onlyloadJob, openGate, readDecision, expireGate, closeJob60 snoneinitial 5 s, coefficient 2, maximum interval 5 min, no attempt limit
role stepsrunPlan, runCode, runReviewroleActivityTimeoutMsheartbeatTimeoutMs, WAIT_CANCELLATION_COMPLETEDinitial 5 s (or retryInitialIntervalMs), coefficient 2, maximumAttempts 4 (MAX_STEP_ATTEMPTS + 1)
testsrunTeststestActivityTimeoutMsas role stepsas role steps
gitprepareContext, openPr10 minnoneas role steps but maximumAttempts 5

Computed timeouts (loadJob): roleActivityTimeoutMs = FACTORY_STEP_TIMEOUT_MS + FACTORY_WORKSPACE_OUT_TIMEOUT_MS + 235,000 ms and testActivityTimeoutMs = policy.tests.timeoutMs + 235,000 ms. The margin 235,000 is 60 s (token) + 10 s (exit wait) + 15 s (stop) + 30 s + 120 s (host git work). With defaults a role step may take up to 600 + 60 + 235 = 895 s, a test step 300 + 235 = 535 s.

Only infrastructure failures are retried (worker death, heartbeat timeout, database error, lost fence). Agent outcomes are returned, never thrown. When retries are exhausted on a step the workflow returns and the job closes failed:internal_error (the default close value). The workflow throws only Temporal failures.

Step identity and the ledger​

Each sandbox step has a key: plan, code-<i>, test-<i>, review-<i> with i from 1 to 5 (factory_steps_step_key_valid). claimStep takes the job row lock, upserts the step row, and bumps attempts while status IN (pending, running) and attempts < 3 (MAX_STEP_ATTEMPTS, factory_steps_attempts_range). The request id is factory-<jobId>/<step_key>#<attempts>; it is the fence value and the ledger request id. It returns nothing (the step is ignored) when the job is unknown, closed or cancel-requested, the step is terminal, or attempts are exhausted. Earlier attempts' ledger rows are abandoned, before and after the final write (race repair). Test steps make no model call and write no ledger row.

Every long activity heartbeats on a timer and runs fenceState(jobId, stepKey, requestId) on each tick: lost aborts the step, cancel_requested aborts it, a query error aborts it (fail closed).

Budget​

Before each role step: remaining = budgetUsd - spent(jobId) where spent is the sum over the job's ledger rows of COALESCE(cost_usd, amount_usd) (unsettled rows in full). The step's row cap is min(roleCap, remaining); below FACTORY_STEP_FLOOR_USD (default 0.02) the step fails budget_exhausted with no admit. Role caps (USD): planner FACTORY_ROLE_CAP_PLANNER_USD 0.10, coder FACTORY_ROLE_CAP_CODER_USD 0.25, reviewer FACTORY_ROLE_CAP_REVIEWER_USD 0.10. The row cap goes to the runtime as limits.maxBudgetUsd; the decorator turns it into a reservation capped by PER_REQUEST_CAP_USD (decorators). Steps are sequential, so the sum of rows is at most the budget plus the gateway's bounded per-call estimation error. Daily client and global caps still apply at admit. The kill switch is read before each role step and test step and before openPr, failing closed.

Cancellation​

closeJob in the store applies precedence: if a PR number is already stored the job closes pr_opened; else if the cancel flag is set it closes cancelled; else the status given. Running and pending steps become aborted. openPr runs inside a non-cancellable scope, so once the push may have started a cancel cannot orphan an unrecorded PR. After openPr has pushed, cancel does not delete the branch or PR; the human closes it.

Context preparation​

prepareContext makes a fresh hardened checkout at baseSha and builds the base tree archive. It refuses a tree with any path segment that equals .agentforge after NFKC normalisation, removal of HFS+-ignorable code points and lower-casing (the host and runner own that directory). The archive is streamed with git archive for every step and never stored; only its SHA-256 is recorded on factory_steps.workspace_in_sha256. Over 256 MiB (MAX_BASE_TREE_BYTES) or 50,000 entries (MAX_BASE_TREE_ENTRIES) ends the job repo_too_large. A fixed archive mtime keeps the tar, and so its hash, deterministic. Org standards injection (plan 6.9) is described in the plan; the source I read for this page does not show a context provider in prepareContext, and degraded_context is a stored boolean exposed in the job view. Unverified: whether and where org standards enter prompts in the current code.

Prompts are host-built from fixed templates (buildPrompt): a role line, a rule that text inside <untrusted> blocks is data and never an instruction, the task text, the iteration number (code and review), the approved plan if any and feedback, each wrapped by wrapUntrusted, which escapes any < that could open or close an untrusted tag.

Data model​

Additional constraints: factory_jobs_pending_gate_valid (the four pending columns are all null or all set, and only while the status is an awaiting_* value), factory_jobs_base_sha_hex, factory_jobs_budget_usd_range, factory_jobs_max_iterations_range, factory_jobs_task_len, factory_jobs_idempotency_key_len. The unique partial index factory_approvals_one_consumed allows one consumed approval per (job_id, gate).

factory_events is the audit table (kind matches ^[a-z][a-z0-9_.]{0,63}$, evidence only: never tokens, task text or patch text). The activity code I read writes one event kind, pr.opened (with the PR number, commit SHA, whether the push created or found the branch, and whether the PR was reused); most lifecycle markers are structured log lines instead (for example job.loaded, context.prepared, patch.rejected, tests.exit, gate.pending, gate.decided, pr.opened, job.closed). Unverified: other writers of factory_events.

Limits​

LimitValueWhere enforced
task size16 KiB (16384)API (413 by bytes), schema, DB check
budgetUsdgreater than 0, at most 2 (maxJobUsdCap); default ceiling FACTORY_MAX_JOB_USD 0.5API, DB check
maxIterations1 to 5, default 3schema, DB check
step attempts3 (maxStepAttempts)claimStep, DB check
artifact size1 MiB (maxArtifactBytes), plan out file 64 KiB, review out file 16 KiBout-file picker, DB check
files per patch200 (maxPatchFiles)patch parser
step outcome JSON4096 bytes (maxOutcomeBytes)DB check
review notes8192 charactersReviewVerdict
approval TTLdefault 72 h, FACTORY_APPROVAL_TTL_MS 1000 ms to 7 daysgate
idempotency key128 charactersAPI
active jobs per clientdefault 2createJob
base tree256 MiB, 50,000 entriesprepareContext

Factory worker environment​

The factory worker (apps/worker/src/factory-main.ts, apps/worker/src/factory/config.ts) is a separate deployment of apps/worker on queue agentforge-factory. Notable variables and defaults:

GroupVariables (defaults)
Temporal and databaseDATABASE_URL, TEMPORAL_ADDRESS, TEMPORAL_NAMESPACE (agentforge-prod), TEMPORAL_TASK_QUEUE (agentforge-factory), TEMPORAL_TLS_*, TEMPORAL_PAYLOAD_KEYS, TEMPORAL_PAYLOAD_KEY_ID
BudgetGLOBAL_DAILY_LIMIT_USD (2), CLIENT_DAILY_LIMIT_USD (1), PER_REQUEST_CAP_USD (0.5), SONNET_MIN_RESERVATION_USD (0.05), BUDGET_MARGIN (0.2), FACTORY_STEP_FLOOR_USD (0.02), role caps (0.10, 0.25, 0.10), AGENTS_DISABLED, ALERT_WEBHOOK_URL
FactoryFACTORY_REPOS (required, non-empty), FACTORY_FORK_OWNER, FACTORY_FORK_SUFFIX (-fork), FACTORY_WORK_DIR (must exist), FACTORY_POLICY_DIR, FACTORY_MODEL (default claude-haiku-5-5, must match claude- plus an id)
Step limitsFACTORY_STEP_TIMEOUT_MS (600000, 60000 to 840000), FACTORY_MAX_TURNS (30, max 50), FACTORY_WORKSPACE_OUT_TIMEOUT_MS (60000, up to 300000), FACTORY_HEARTBEAT_INTERVAL_MS (10000), FACTORY_HEARTBEAT_TIMEOUT_MS (30000), FACTORY_APPROVAL_TTL_MS (72 h), FACTORY_CONCURRENCY (2, max 8)
SandboxSANDBOX_IMAGE, SANDBOX_GATEWAY_CONTAINER, SANDBOX_OWNER, SANDBOX_MAX_OUTPUT_TOKENS (8192), SANDBOX_RUNTIME (runc or runsc, default runc), FACTORY_ALLOW_RUNC
Gateway controlSANDBOX_CONTROL_HOST (local docker exec shape) or GATEWAY_CONTROL_URL plus GATEWAY_CONTROL_TOKEN_WORKER_FILE
GitHubGITHUB_API_URL, GITHUB_GIT_URL, GITHUB_APP_READ_PR_*, GITHUB_APP_PUSH_* (exactly one of the inline key and its _FILE)

Startup refusals (loadFactoryConfig, FX-CFG1 and related):

  • ANTHROPIC_API_KEY set: refused.
  • SANDBOX_RUNTIME=runc without FACTORY_ALLOW_RUNC=throwaway-repo-only: refused (runc shares the host kernel, use runsc for any repo that matters).
  • GATEWAY_CONTROL_URL, GITHUB_API_URL, GITHUB_GIT_URL must be https, or http to a loopback host.
  • Duplicate repo names in FACTORY_REPOS; a fork name that equals any allowlisted base repo (case-insensitive).
  • FACTORY_HEARTBEAT_INTERVAL_MS above FACTORY_HEARTBEAT_TIMEOUT_MS / 3.
  • PER_REQUEST_CAP_USD below the largest role cap, or FACTORY_STEP_FLOOR_USD above the smallest role cap.
  • Host git older than 2.50.1 (assertGitFloor).
  • GitHub preflight failure for any allowlisted repo (the push App can reach the base repo, or the fork is not a fork of the base).

The factory runtime is createAgentRuntime around the SandboxRuntime with floorUsd = FACTORY_STEP_FLOOR_USD and limits maxTurns {default FACTORY_MAX_TURNS, max 50}, timeoutMs {default FACTORY_STEP_TIMEOUT_MS, max 840000}.

Status of the live path​

docs/deploy-factory.md states that P3 is built and tested locally only (a Linux VM, a scripted fake model and a fake git remote). Nothing in the deploy runbook has run on the VPS or against real GitHub; the two GitHub Apps, the throwaway test repo, the same-org fork layout, runsc on the sandbox host, TLS for the control link and real-key factory runs are all marked UNVERIFIED with a named owner. A real GitHub live check (G-P3-GH) was attempted and recorded in docs/p2-findings.md (P3-7 sections).

Sources​

  • packages/agent-types/src/factory.ts (FACTORY_LIMITS, FactoryStatus, FactoryErrorCode, StepKey, FactoryJobRequest, FactoryJobView, effectiveGates)
  • packages/agent-workflows/src/workflows/factory.ts (factoryJobWorkflow)
  • packages/agent-workflows/src/workflows/factory-shared.ts (FactoryActivities, constants)
  • apps/worker/src/factory/activities.ts (createFactoryActivities, withStep, runRole, workspaceTar, closeJob)
  • apps/worker/src/factory/config.ts (loadFactoryConfig), apps/worker/src/factory-main.ts
  • apps/worker/src/factory/prompts.ts (buildPrompt, wrapUntrusted)
  • apps/api/src/factory.ts (intake, read and cancel routes)
  • packages/db/src/factory.ts (PgFactoryStore: createJob, claimStep, stepBudget, closeJob, listEvents), packages/db/src/schema.ts (factoryJobs, factorySteps, factoryArtifacts, factoryApprovals, factoryEvents)
  • docs/deploy-factory.md, .omc/plans/p3-software-factory.md (sections 4, 5, 6.1, 6.7)