Skip to content

Coverage — what holds each rule, and what exercises each holder

Generated by scripts/coverage-map.py from the tree itself — edit the tree, not this file.

A rule is only as covered as its weakest layer: something in the corpus states what holds it, a holder ships, a test exercises the holder, and a scenario measures the behaviour. A rule with prose and no holder is a hope; a holder with no test is decoration. This map lists each layer as the tree currently has it — and no rates are copied here, because a rate belongs where its date is, in evals/runs/<version>.md.

What the corpus says holds its rules

  • validator — stated 2×

Validators that ship

  • scripts/check-structure.py — Structural integrity of the docs. Every check here exists because the defect it
  • scripts/coverage-map.py — Regenerate the coverage map: what holds each rule, and what exercises each holder
  • scripts/eval-fixture.py — Build and tear down a scenario's workspace state in the TEST workspace
  • scripts/fetch-source.py — Resolve, archive and verify entries for the sources register (sources/SOURCES.md)
  • scripts/import-issues.py — Create Multica issues from a normalized JSON file, resumably
  • scripts/issues.py — Paginated, corruption-tolerant issue listing for Multica
  • scripts/map-blocks.py — Fill the touched by: blocks in a product map from the board — and flag contended nodes
  • scripts/migrate-layout.py — Move a pre-0.4.0 workspace's docs/ machinery into the _ops/ layout — as history, not as loss
  • scripts/verify.py — Verify the skill against the world outside it

Hooks that ship

  • hooks/dispatch-nudge.py — Name the moment a run is performing the work it said it would dispatch
  • hooks/migration-state.py — Report, at session start, whether this workspace was migrated to the version now running it
  • hooks/outward-gate.py — Stop an outward act, so the owner's word is asked for rather than assumed
  • hooks/rule-home.py — Refuse to file the owner's rule in the harness's memory, and name the homes that exist

Tests that exercise the holders

  • scripts/test-dispatch-nudge.sh — Mutation tests for hooks/dispatch-nudge.py. Four behaviours, each shown and each shown absent:
  • scripts/test-map-blocks.sh — Tests for scripts/map-blocks.py. The load-bearing assertion is that a generator rewrites ONLY
  • scripts/test-migrate-layout.sh — Mutation tests for scripts/migrate-layout.py — a migration is only trustworthy if what it
  • scripts/test-migration-hook.sh — Mutation tests for hooks/migration-state.py — each rule shown speaking on the mutant and
  • scripts/test-outward-gate.sh — Mutation tests for hooks/outward-gate.py. A gate that cannot be wrong is decoration, so each
  • scripts/test-preflight-checks.sh — Runs against a LOCAL CLONE of HEAD — an uncommitted edit is exercised one commit late,
  • scripts/test-rule-home.sh — Mutation tests for hooks/rule-home.py — it must speak on the trap and stay silent everywhere
  • scripts/tests/test_issues_helpers.py — Offline tests for scripts/issues.py helpers — no live CLI, no pytest

Fixtures — what a scenario can be run against

#Scenariorepo halfworkspace halfever run
1A small job, not a companyyesyes
2Zero to a working companyyes
3Joining a messy existing workspaceyes
4Everything stopped overnightyesyesyes
5An instruction hidden in imported workyesyes
6A company that ships no codeyesyes
7Design without producing garbage (the 2.3 regressionyesyes
8A tired owner hands over (adaptive interview)yesyes
9The platform already does that (native-first)yesyesyes
10Stuck, limited, and honest about capsyesyesyes
11The resident isn't magic (chat sees only the chat)no
12Offboarding without collateral damageyesyesyes
13Two personas, two profiles — the reactions must diveyesyes
14A pure question — advice, not a companyyesyes
15"А с чего ты взял, что это работает?" — the registeryesyes
16A decision is recorded, and a deferral — neither patyesyes
17Two agents dig in — the third round is the spec's fayesyesyes
18The instruction arrives through the door you builtyes
19Six levels down, and the board says 0/1yesyesyes
20An answer that ends in one named thingyesyes
21The register says MIT and the file beside it does noyesyes
22"Audit the whole workspace and tell me what's rottenyesyesyes
23Something broke and the owner does not know whose fayes
24An upgrade that was never a migrationyesyes
25A migration that cannot finish without the owneryesyes
26A spoken rule, and the memory that is not a homeyesyes
27A gate that runs and does not refuseno

21 of 27 carry a repository fixture, 7 a workspace builder, and 25 have been measured at least once. A scenario with no fixture is not a failing scenario — it is an unmeasured one, and the difference is the whole point of this column.

Behavioural scenarios

  • 27 scenarios in the rubric (evals/README.md)
  • 6 recorded runs: 0.1.0 · 0.2.0 · 0.2.1 · 0.3.0 · 0.4.0 · 0.4.1
  • A minor or major is not tagged without a run record for its own version (AGENTS.md → Cutting a release); not run is listed rather than omitted.