Skip to main content

Status, measured numbers and limits

State as recorded on 2026-11-16 (session dossier docs/sessions/2026-11-16-codecrucible-build/): CodeCrucible is built on main from the pure library up to the REST routes, both workflows, the exec worker, the quality image and the verify steps, all tested locally. Nothing is deployed or enabled. QUALITY_ENABLED is unset everywhere, so the code ships dormant. Everything left is owner-gated.

Built versus not built​

AreaBeadState
quality-core: discovery, complexityCC-1.1Built
Coverage reader, CRAPCC-1.2 (.96 real v8 report)Built
Report schema, join, stable ids, aggregateCC-1.3Built
Gate evaluator, QualityPolicyCC-1.4 (.98)Built
Static checks, 16 AC9 fixturesCC-1.5 (.99)Built
Quality image, analyze.mjs, run.mjsCC-2.1 (.100)Built; image built locally only
Out-file support, golden argvCC-2.2 (.101)Built
Schema, store, migration 0006CC-2.3 (.102)Built; migration not run anywhere
Exec worker and activitiesCC-2.4 (.103)Built; never run against a real Temporal server or from the quality-exec Docker target
qualityAssessmentWorkflow, core activitiesCC-2.5 (.104)Built; tested with fake activities and the Temporal test environment
REST routes and configCC-2.6 (.105)Built
qualityVerifyWorkflow steps 1 to 4, 8, 9, intentCC-3.1 (.106)Built
Solo check, re-kill, controls, AC10 fixturesCC-3.2 (.107)Built
Pending-submission cap, migration 0007.111Built
Real Temporal starter in the API.112Built
Deploy runbookCC-4.1 (.108, part)Written (docs/deploy-quality.md)
Secrets manifest entries, change requests, baseline run, hand-made verifyCC-4.1 (.108, rest)Not built (needs the host decision)
CI enforcement: submit step, scheduled quality-verify status jobCC-4.2 (.109)Not built
Verify activities registered in the exec entry pointnoneNot built: quality-exec-main.ts registers only prepareWorkspace, analyze, runChunk, cleanup
Alerts (global daily limit at 80 percent, any control failure) through ALERT_WEBHOOK_URLplan section 3Not built (runbook: "UNVERIFIED: not wired yet")
review finding for a 30 percent fall in trusted mutantsplan 4.4Not built
Line-range chunks (split of one file)plan section 3Not built: one chunk per file
Factory loop, exclusions, MCP facade, fast level, continue-as-new beyond 20 chunksplan section 10Later, not planned for v1

About 95 files and 21,000 lines were added across these beads (SUMMARY). CI for the last commits was queued or failed on load-related timeouts of the shared runner at wrap time, so local verification is the evidence, not CI.

Preconditions (owner decisions)​

IdNeeded forState
PRE-0 Temporal access: an authorizer with a claim mapper, a dedicated cluster, or a written sign-off. A separate namespace does not stop one client from touching others on the shared clusterAny exec worker on the tailnetOpen; deferred 2026-11-16 until a host exists
PRE-1 Sandbox host: Linux, Docker, runsc verified, tailnet grant to Temporal onlySpike S3, any non-local exec hostOpen
PRE-2 Postgres volume on the core hostPhase 4Done on the current core host (bead .91, 2026-11-16)
PRE-3 Read-only GitHub App keyAny repository fetched from GitHubOpen

Until PRE-0 and PRE-1 are settled only local development is supported: a Temporal dev server, QUALITY_RUNTIME=runc, QUALITY_ALLOW_RUNC=own-repo-only, a local mirror.

The rollout order is in deploy runbooks and docs/deploy-quality.md.

Measured numbers​

Spikes (Phase 0, docs/p2-findings.md)​

Laptop (Apple M3 Pro, 12 cores, 36 GB), Node 24.21.0, Stryker core and vitest-runner 10.1.1, Vitest 5.1.0, TypeScript 6.0.3 (type checks disabled). Host measurements, not the sandbox argv.

SpikeMeasurementResult
S1 (hard gate)src/budget.ts per-file run96 mutants, dry run 5 s for 77 tests, total 55 s; 82 killed, 12 survived, 2 no coverage; killedBy on 82 of 82
S1trusted list run (host-only config, one empty test)same 96 tuples in 6 s, set equality with the real run
S1whole-package list run8 files, 417 mutants, 22 s; Stryker ids differ between package and one-file runs, tuples agree
S1materialized re-killkilling test alone exit 1; unmutated 0; comment control 0; code control 0
S2full packages/agent-core, concurrency 2417 mutants, dry run 5 s, full run 4 min 59 s; 327 killed, 68 survived, 10 no coverage, 3 timeout, 9 runtime error; score 79.1 percent; about 0.7 s wall time per mutant
S4stability20 of 20 unmutated runs passed; 3 full runs: 0 of 417 mutants changed status; scores 79.1, 79.1, 79.1
S5offline image prototype2.06 GB, cold build 1 min 54 s, 77 of 77 tests under --network none
S3runsc on a sandbox hostNot run (needs PRE-1)

Built code (OrbStack, runc, machine under heavy load)​

CheckResult
Quality image1.79 GB; build 70 to 276 s
Analysis35 to 92 s; 417 trusted mutants in 8 chunks
One chunk (src/budget.ts)62 to 114 s; 96 mutants: 82 killed, 12 survived, 2 no coverage; 100 percent of tuples join the trusted list
Exec activities end to end (scripts/quality-exec.test.sh)122 s; cleanup idempotent; nothing left behind
Verify steps 5 to 7 (scripts/quality-verify.test.sh)195 s; solo 3 of 3; re-kill confirmed with the new test, not_reproduced with the base tests; controls pass
Workflow history of 20 chunks147 events (limit 150)

Not measured: runsc anywhere, a full-package mutation run at concurrency 2 inside the sandbox, a real tailnet exec host, a real GitHub fetch, and the end-to-end baseline and verify of packages/agent-core (the dogfood).

Stated limits​

LimitSource
Report forgery from test code is mitigated by re-kill and controls, not preventedrunbook
Coverage is untrusted and capped by the mutation resultsrunbook, capCoverage
Equivalent mutants are not excluded in v1runbook, plan 4.5
!(a >= b) control differs from a < b for NaN or undefined operandsquality-core README
A retained Stryker timeout with no killers costs creditFOLLOWUPS
A pending submission can leak a slot past its TTL, or when a workflow ends unadmitted (no execution timeout)ADR 0009, FOLLOWUPS
readTrees does not heartbeatFOLLOWUPS
Daily limits count a run wholly on its start UTC daysandboxSecondsToday shortcut
Limits are checked at admission and once before the chunks, not per chunkADR 0009
Rules 1, 2 and 4 of the static checks are tripwiresquality-core README
Cognitive complexity is a SonarSource subsetbead .110
One package per run, at most 20 chunks, one chunk per fileplan, buildAnalysis
runc is allowed only with QUALITY_ALLOW_RUNC=own-repo-only against a local Temporal dev server (retire at PRE-1, before any other repo, or 30 days after phase 2 starts)plan risk table
Verify is voluntary until CC-4.2 makes it a required statusplan risk table

Hard numbers at a glance​

LimitValue
Chunks per assessment20
Chunk time including dry run840 s (policy default 600 s)
Analysis container540 s
Sandbox time per assessment / per verify3,600 s / 7,200 s
Daily sandbox seconds per client / global14,400 / 28,800
Active top-level runs1
Pending submissions per client, TTL4, 86,400 s
Chunk report (gzip)128 KiB
Analysis out file8 MiB raw, 1 MiB gzip
Raw out file from a container8 MiB
Mutants per report5,000
Snippet in Postgres80 characters
killed_by in Postgres4,096 bytes
Findings in Postgres64 KiB
Credit candidates per verify50
Stability and solo runs3 and 3
Chunk cache2,000 rows, 30-day TTL
Survivors page200
Workflow history150 events, 3 MiB
Container2 CPUs, 2 GiB, 512 pids, --network none, 1 GiB workspace

Open items​

From docs/sessions/2026-11-16-codecrucible-build/FOLLOWUPS.md and the README:

  • Local dogfood (owner decision): hold until 2026-11-17 after 10:00 local, then a baseline and one hand-made verify of packages/agent-core end to end (Temporal dev server, core and exec workers with runc, local mirror, real API), with numbers into docs/p2-findings.md.
  • CC-4.1 remainder (.108), CC-4.2 (.109), .110 (cognitive subset; turbo boundary tag for quality-core undecided).
  • Re-check CI for the last commits; earlier runs failed on load timeouts.
  • Observed while writing these pages (code wins over docs): the verify activities are not registered in quality-exec-main.ts; the core worker has no separate quality namespace while the API and exec worker can use one; docs/deploy-quality.md lists a QUALITY_EXEC_TASK_QUEUE variable that exec-config.ts does not read (the queue is the constant agentforge-quality-exec).

Sources​

  • docs/sessions/2026-11-16-codecrucible-build/README.md, SUMMARY.md, FOLLOWUPS.md
  • docs/sessions/2026-11-16-codecrucible-build/adr/0009-quality-admission-and-pending-submissions.md
  • docs/deploy-quality.md (Preconditions, Local evidence, Rollout order, Operations and rollback)
  • docs/p2-findings.md (CodeCrucible Phase 0 spikes; CodeCrucible S5)
  • .omc/plans/codecrucible-mutant-hunter.md (sections 3, 6, 7, 9, 10)
  • packages/quality-core/README.md
  • apps/worker/src/quality-exec-main.ts, apps/worker/src/quality/exec-config.ts, apps/worker/src/worker.ts
  • packages/agent-workflows/src/workflows/quality-shared.ts (QUALITY_RUN_LIMITS), packages/agent-types/src/quality.ts (QUALITY_LIMITS), packages/db/src/quality.ts (sandboxSecondsToday)