CodeCrucible: the mutation-evidence quality gate
CodeCrucible is AgentForge's deterministic, model-free quality analyzer. It measures one package of one repository at one
commit and records, per function: cyclomatic and cognitive complexity, coverage, CRAP, and a mutation score from
StrykerJS runs. It then gates a pushed change against the protected branch ("no new debt")
and, in a verify run, checks that every kill the change claims is real.
The defining idea is mutation evidence: a coverage number alone can be raised by tests without assertions, so CodeCrucible credits only mutants that a test actually kills, and it re-kills the important ones itself in a fresh container before it believes them.
Everything on this section's pages is built and tested locally only (OrbStack, runc, a local git mirror). Nothing
has run on a VPS, against the shared Temporal cluster or against GitHub. QUALITY_ENABLED is unset everywhere, so the
REST routes answer 404 and the core worker registers no quality workflow or activity. Migrations 0006_quality.sql and
0007_quality_submissions.sql exist but have run nowhere. See status and limits.
Why it exists
The source note (summarised in the plan .omc/plans/codecrucible-mutant-hunter.md, section 1) asks for an analyzer whose
findings an agent must act on, with these properties:
| Requirement | How v1 meets it |
|---|---|
| Two separate gates, not one composite score | CRAP per function (C^2 * (1 - cov)^3 + C) and mutation score per function, both in evaluateGate |
| Mutation engine is StrykerJS | Stryker 10.1.1 with the Vitest runner, inside the quality image |
| Baseline plus "no new debt" | quality_baselines rows written only by runs on the protected branch tip; deltas against them |
| Deterministic, model-independent | packages/quality-core is pure; no model call happens anywhere, so no gateway budget is touched |
| Guard-rails: the agent never edits rules, never weakens tests, killed mutants stay killed | Operator-only policy, static checks, executed-identity check, materialized re-kill, controls (anti-gaming) |
Scope cut (v1)
ADR 0007 (docs/sessions/2026-11-16-codecrucible-and-gateway-cidr/adr/0007-codecrucible-v1-shape.md) fixes the v1 shape:
"measure, gate, and verify a pushed change over REST". v1 does not close the agent loop: a person or an existing agent
session reads the survivors, pushes a fix, then calls verify.
| In v1 | Later (plan section 10) |
|---|---|
packages/quality-core: complexity, CRAP, report schema, stable mutant ids, gate, tamper scan | Composite score, Tree-sitter, other languages |
| Deep run: mutation testing of one package, chunked per file, plus one coverage run per chunk for CRAP | A "fast" level as its own run type; exhaustive schedules |
qualityAssessmentWorkflow, qualityVerifyWorkflow with re-kill and controls | Factory child workflow; survivors in the coder prompt |
| Postgres tables, baseline, "no new debt" | Exclusions for equivalent mutants with approval; artifacts table; dashboards |
REST /v1/quality/* in apps/api | MCP read facade (quality.get_report, get_function, get_survivors) |
| Database-free exec worker on a sandbox host | More than one package per run; continue-as-new beyond 20 chunks |
Dogfood on packages/agent-core | apps/worker, apps/gateway, packages/db |
Alternatives rejected by ADR 0007: a separate always-on service on its own host; a custom Tree-sitter mutation engine; MCP-first exposure.
Design principles
From plan section 2:
- The analyzer is deterministic and holds no model key. The agent never computes or stores a score.
- Untrusted code (repository tests) runs only in the sandbox. The process that drives Docker and reads sandbox output
holds no database credential, no push key, no gateway control token and no
agentforge-prodTemporal payload key. - Everything that defines the measurement (rules, config, thresholds, scope, the mutant list, the cache) is produced by the host from the operator policy and the base commit, never by the test container.
- No new always-on service on the core host. One small database-free worker on a sandbox host is the price of principle 2.
- Measure before building, and ship the smallest thing one team can verify.
Architecture
The plan calls this option "B-prime": a pure library, core activities inside the existing core worker, and a database-free exec worker on a dedicated sandbox host. It reuses the FL-3 worker split of the session runtime (see worker fleet).
The trust boundaries in that picture:
The two run types
| Run | Started by | What it does | Verdict |
|---|---|---|---|
Assessment (qualityAssessmentWorkflow) | POST /v1/quality/runs | Measures the head commit of one package, gates it against the stored baseline, writes a new baseline when the head is the protected tip | passed:remediation or failed (assessments always judge as remediation) |
Verify (qualityVerifyWorkflow) | POST /v1/quality/verify | Runs two child assessments (base and head), then steps 1 to 9 of plan 4.3: stability, executed identities, characterization, deltas, solo checks, materialized re-kills, controls, CRAP credit | passed:remediation, passed:feature or failed |
Details: assessment workflow, verify workflow.
Component map
| Piece | Location | Holds |
|---|---|---|
| Analysis library | packages/quality-core | Nothing (pure; imports only typescript, zod, node:crypto, node:zlib) |
| Wire types and limits | packages/agent-types/src/quality.ts, QualityPolicy in packages/agent-types/src/factory.ts, HTTP schemas in packages/agent-types/src/http.ts | Schemas only |
| Workflows | packages/agent-workflows/src/workflows/quality.ts, quality-shared.ts | Workflow code, queue names, QUALITY_RUN_LIMITS |
| Core activities | apps/worker/src/quality/core-activities.ts, apps/worker/src/quality/verify.ts | DATABASE_URL (through the core worker) |
| Exec worker | apps/worker/src/quality-exec-main.ts, apps/worker/src/quality/exec-activities.ts, exec-config.ts | Docker socket, repo mirror, quality payload key |
| Quality image | apps/worker/quality-image/ (Dockerfile, analyze.mjs, run.mjs, tests.mjs, common.mjs) | Stryker 10.1.1, Vitest 5.1.0, repo dependencies, base dist |
| Persistence | packages/db/src/quality.ts, migrations 0006_quality.sql, 0007_quality_submissions.sql | quality_runs, quality_functions, quality_mutants, quality_baselines, quality_chunk_cache, quality_submissions |
| REST | apps/api/src/quality.ts | QUALITY_ENABLED, QUALITY_CLIENTS, limits |
Pages in this section
| Page | Content |
|---|---|
| Library | quality-core: function discovery, qualified names, complexity, CRAP, coverage, report schema, trusted-list join, stable ids, gate evaluator, static checks, control rewrites, analysis out file |
| Assessment workflow | qualityAssessmentWorkflow end to end: queues, admission, chunking, cache, history budget, partial and exec_lost |
| Verify workflow | qualityVerifyWorkflow steps 1 to 9, intent semantics, credit candidates, verdicts |
| Exec worker and image | Exec activities, workspace, sandbox argv, image contents, analyze.mjs, run.mjs, tests.mjs, config refusals |
| Anti-gaming | Threat model, named gaming paths with defences and the tests that prove them |
| Data model | ER diagram and every constraint of the quality tables |
| REST and limits | Routes, admission, pending cap, daily limits, queued view, error codes |
| Status and limits | What is built, what is not, measured numbers, stated limits, open beads |
Related pages elsewhere: sandbox, patch policy (the
.agentforge/quality/ path is a forbidden patch path), deploy runbooks,
decisions.
Sources
.omc/plans/codecrucible-mutant-hunter.md(git-ignored plan; sections 1, 2, 3, 10 and the ADR draft)docs/sessions/2026-11-16-codecrucible-and-gateway-cidr/adr/0007-codecrucible-v1-shape.mddocs/sessions/2026-11-16-codecrucible-build/README.md,SUMMARY.mddocs/deploy-quality.md(Roles, state line)packages/quality-core/README.mdapps/api/src/app.ts(QUALITY_ENABLEDregistration),apps/worker/src/worker.ts(qualityWorker)packages/agent-workflows/src/workflows/quality.ts,quality-shared.ts