feat(ship): the measure loop decides by the bar's exit codes

Step 4 branches on MEETS/MEETS-qualified, needs-classify (with the
qualifying and fix-round classes), EXTEND once and BELOW; the rest of the
loop prose is tightened to offset most of the growth. ship's union size
cap moves from 1.495 to 1.4975 (measured 281,058 / 187,706 = 1.4973).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

ship/sections/measure.md

@@ -28,26 +28,35 @@ completed at the cap) and one failure capture, then continue the triage.
    before any paid trial. A red that is infrastructure (provider outage, runner
    loss) may skip measurement: `ship-measure skip --case <id> --reason "<why>"`
    labels it `unmeasured` in the PR body. It never counts as a pass.
-2. **Show the plan.** Print `ship-measure table` (trials, pass bar, panels and
-   budget per kind, from the `ship_measure_*` gstack config keys) and the
-   estimate: per-trial cost from pass-rate history times trials, or "estimate unknown".
+2. **Show the plan.** Print `ship-measure table` (trials, bar and budget per
+   kind, from the `ship_measure_*` gstack config keys) and the estimate:
+   per-trial cost from pass-rate history times trials, or "estimate unknown".
 3. **Measure.** `ship-measure measure --case <id> --round baseline [--cost-per-trial <usd>]`
-   runs the case alone on the current head at its kind's count, in parallel. It
-   uses the project's documented single-case eval command,
-   `--command '<command> {case}'`, found the way Step 4 finds test commands
-   (CLAUDE.md/AGENTS.md, then package scripts). gstack's default needs no
-   `--command`: `scripts/test-paid-shards.ts --tier <tier> --case <id>`, and it
-   accepts standalone judge ids. With no documented single-case command, ask
-   once and record the answer in the report. Exit 2 means it needs approval:
-   ask once with the reason it printed, then rerun with `--approved`.
-4. **Fix at the cause.** Root-cause the failed trials from their captures in
+   runs the case alone on the current head at its kind's count, in parallel,
+   through the project's documented single-case eval command,
+   `--command '<command> {case}'` (found the way Step 4 finds test commands;
+   gstack needs none and accepts judge ids). With no documented command, ask
+   once and record the answer in the report. Exit 2 needs approval: ask once
+   with its reason, then rerun with `--approved`.
+4. **Decide by the exit code** (MEETS 9/10, MEETS-qualified 8/10 with every
+   red qualifying, EXTEND 7/10; behavior 11/12, 10/12, 9/12):
+   - 0: MEETS or MEETS-qualified; done.
+   - 5: needs-classify. Record each listed red from its capture:
+     `ship-measure classify --case <id> --round <r> --trials <2,5> --class <class> --evidence "<path:line>"`.
+     Qualifying: `provider` (only on the trial's own provider error),
+     `judge-noise`, `model-miss` (cite the line). Fix round: `timeout`,
+     `hang`, `regression`, `fixable`. `unclassified`: reviewed, no evidence.
+   - 6: EXTEND. `ship-measure extend --case <id> --round <r>` once, on the
+     unchanged tree; never a third batch.
+   - 1: BELOW, a fix round.
+5. **Fix at the cause.** Root-cause the failed trials from their captures in
    `.context/ship-measure/<case>/<round>/tNN/` and change the cause.
-5. **Re-measure.** `ship-measure measure --case <id> --round round-<n> --fix "<cause and change>"`.
-   Exit 0 is at or above target. Exit 1 is below target: return to step 4. Call
-   a case fixed only with a named causal change and every failure in the closing
-   measurement explained; report "observed k/n after fix at <cause>", never a proven rate.
-6. **Stop.** Exit 3 is a named red (repair-round limit or the per-case budget
-   exhausted): stop, do not push, and report `ship-measure report` with every trial.
+6. **Re-measure.** `ship-measure measure --case <id> --round round-<n> --fix "<cause and change>"`,
+   then item 4. Call a case fixed only with a named causal change and every
+   failure in the closing measurement explained;
+   report "observed k/n after fix at <cause>", never a proven rate.
+7. **Stop.** Exit 3 is a named red (round limit, budget, or two void
+   batches): stop, do not push, and report `ship-measure report` with every trial.
 
 **Gate once.** When every red case measures at or above target, run the full
 gate once (Step 6 item 2). When the project's CI runs that gate on push (gstack:

ship/sections/measure.md.tmpl

@@ -26,26 +26,35 @@ completed at the cap) and one failure capture, then continue the triage.
    before any paid trial. A red that is infrastructure (provider outage, runner
    loss) may skip measurement: `ship-measure skip --case <id> --reason "<why>"`
    labels it `unmeasured` in the PR body. It never counts as a pass.
-2. **Show the plan.** Print `ship-measure table` (trials, pass bar, panels and
-   budget per kind, from the `ship_measure_*` gstack config keys) and the
-   estimate: per-trial cost from pass-rate history times trials, or "estimate unknown".
+2. **Show the plan.** Print `ship-measure table` (trials, bar and budget per
+   kind, from the `ship_measure_*` gstack config keys) and the estimate:
+   per-trial cost from pass-rate history times trials, or "estimate unknown".
 3. **Measure.** `ship-measure measure --case <id> --round baseline [--cost-per-trial <usd>]`
-   runs the case alone on the current head at its kind's count, in parallel. It
-   uses the project's documented single-case eval command,
-   `--command '<command> {case}'`, found the way Step 4 finds test commands
-   (CLAUDE.md/AGENTS.md, then package scripts). gstack's default needs no
-   `--command`: `scripts/test-paid-shards.ts --tier <tier> --case <id>`, and it
-   accepts standalone judge ids. With no documented single-case command, ask
-   once and record the answer in the report. Exit 2 means it needs approval:
-   ask once with the reason it printed, then rerun with `--approved`.
-4. **Fix at the cause.** Root-cause the failed trials from their captures in
+   runs the case alone on the current head at its kind's count, in parallel,
+   through the project's documented single-case eval command,
+   `--command '<command> {case}'` (found the way Step 4 finds test commands;
+   gstack needs none and accepts judge ids). With no documented command, ask
+   once and record the answer in the report. Exit 2 needs approval: ask once
+   with its reason, then rerun with `--approved`.
+4. **Decide by the exit code** (MEETS 9/10, MEETS-qualified 8/10 with every
+   red qualifying, EXTEND 7/10; behavior 11/12, 10/12, 9/12):
+   - 0: MEETS or MEETS-qualified; done.
+   - 5: needs-classify. Record each listed red from its capture:
+     `ship-measure classify --case <id> --round <r> --trials <2,5> --class <class> --evidence "<path:line>"`.
+     Qualifying: `provider` (only on the trial's own provider error),
+     `judge-noise`, `model-miss` (cite the line). Fix round: `timeout`,
+     `hang`, `regression`, `fixable`. `unclassified`: reviewed, no evidence.
+   - 6: EXTEND. `ship-measure extend --case <id> --round <r>` once, on the
+     unchanged tree; never a third batch.
+   - 1: BELOW, a fix round.
+5. **Fix at the cause.** Root-cause the failed trials from their captures in
    `.context/ship-measure/<case>/<round>/tNN/` and change the cause.
-5. **Re-measure.** `ship-measure measure --case <id> --round round-<n> --fix "<cause and change>"`.
-   Exit 0 is at or above target. Exit 1 is below target: return to step 4. Call
-   a case fixed only with a named causal change and every failure in the closing
-   measurement explained; report "observed k/n after fix at <cause>", never a proven rate.
-6. **Stop.** Exit 3 is a named red (repair-round limit or the per-case budget
-   exhausted): stop, do not push, and report `ship-measure report` with every trial.
+6. **Re-measure.** `ship-measure measure --case <id> --round round-<n> --fix "<cause and change>"`,
+   then item 4. Call a case fixed only with a named causal change and every
+   failure in the closing measurement explained;
+   report "observed k/n after fix at <cause>", never a proven rate.
+7. **Stop.** Exit 3 is a named red (round limit, budget, or two void
+   batches): stop, do not push, and report `ship-measure report` with every trial.
 
 **Gate once.** When every red case measures at or above target, run the full
 gate once (Step 6 item 2). When the project's CI runs that gate on push (gstack:

test/helpers/carve-guards.ts

@@ -166,7 +166,7 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
     // wave's headline capability) grows the union to 1.195x. Deliberate:
     // the section is on-demand (loads only for Apple store targets), so
     // per-invocation cost for non-iOS ships is one manifest line.
-    maxSizeRatio: 1.495, // Shared advisory identity/dedup + critical-severity validation: 248,065 union bytes / 187,706 baseline = 1.3216 (2026-09-17). + test value bar in the lazy Step 7 section (value cards, weak paths, gate table, base control, machine checks; ~13.6KB): measured 1.396 (2026-09-29). + W1 guarded state-root resolution (`eval gstack-paths; : "${GSTACK_STATE_ROOT:?…}"`) in the Context Recovery preamble, the eureka log and each state-writing bash block; measured 1.401 (2026-09-30). + the shared QA review step's plan-check timing rule (plan checks and their revalidation run on --timeout-ms after smoke expiry); measured 1.4022 (2026-09-30). + the shared TEST_BOOTSTRAP owned-change undo rules (s03 H1-H3, M1) and the value-bar coverage wording (s03 M2/M3); measured 1.4044 (2026-10-02). + severe fix wave ship-review lane: G1 NO_VERSION dispatch, title and CHANGELOG arms (versionless repos ship without inventing a version), F1 parent-side plan binding before the audit dispatch (binding, docs/designs candidates, exact not-run line) and D3's data-not-instructions line; +2,126 union bytes, measured 1.4135 (2026-10-03). + INV-1 consumer lines (outside review `unverified`/`unavailable` is missing coverage in the readiness note and PR body); measured 1.4148 (2026-10-03). + v1.91.19.0 severe fix wave: verdict-form outside-review callers with sandbox preflight and first-use notice (INV-1/B1/Q2), P0/P1 gate and unverified wording, bound-plan discovery (F1), visible LEARNINGS failures (B5), carved external pointers; measured 267,752 / 187,706 = 1.4264 (2026-10-03). + v1.91.18.0 (#3023) merged into the v1.91.19.0 wave; measured 267,874 / 187,706 = 1.4271 (2026-10-04). + Oct 6 fix wave: quota_exhausted preflight branch (#3051) and quoted-heredoc issue template (#3046); measured 268,199 / 187,706 = 1.4288 (2026-10-06). + Step 10's inline no-PR check before the Greptile dispatch (#3020); measured 268,655 / 187,706 = 1.4313 (2026-10-06). + Oct 6 fix wave A3/CEO-12: free text reaches commands only through agent-written mktemp files (mktemp block, write rule, path rebuild and refusal at each posting, prompt and brief site); measured 275,422 / 187,706 = 1.4673 (2026-10-06). Combined Oct 6 wave head (B free-text files + D measure loop + A rework): measured 280,363 / 187,706 = 1.4936 (2026-10-06). + gate-once push rule (no second paid gate after a CI push); measured 280,573 / 187,706 = 1.4947 (2026-10-06).
+    maxSizeRatio: 1.4975, // Shared advisory identity/dedup + critical-severity validation: 248,065 union bytes / 187,706 baseline = 1.3216 (2026-09-17). + test value bar in the lazy Step 7 section (value cards, weak paths, gate table, base control, machine checks; ~13.6KB): measured 1.396 (2026-09-29). + W1 guarded state-root resolution (`eval gstack-paths; : "${GSTACK_STATE_ROOT:?…}"`) in the Context Recovery preamble, the eureka log and each state-writing bash block; measured 1.401 (2026-09-30). + the shared QA review step's plan-check timing rule (plan checks and their revalidation run on --timeout-ms after smoke expiry); measured 1.4022 (2026-09-30). + the shared TEST_BOOTSTRAP owned-change undo rules (s03 H1-H3, M1) and the value-bar coverage wording (s03 M2/M3); measured 1.4044 (2026-10-02). + severe fix wave ship-review lane: G1 NO_VERSION dispatch, title and CHANGELOG arms (versionless repos ship without inventing a version), F1 parent-side plan binding before the audit dispatch (binding, docs/designs candidates, exact not-run line) and D3's data-not-instructions line; +2,126 union bytes, measured 1.4135 (2026-10-03). + INV-1 consumer lines (outside review `unverified`/`unavailable` is missing coverage in the readiness note and PR body); measured 1.4148 (2026-10-03). + v1.91.19.0 severe fix wave: verdict-form outside-review callers with sandbox preflight and first-use notice (INV-1/B1/Q2), P0/P1 gate and unverified wording, bound-plan discovery (F1), visible LEARNINGS failures (B5), carved external pointers; measured 267,752 / 187,706 = 1.4264 (2026-10-03). + v1.91.18.0 (#3023) merged into the v1.91.19.0 wave; measured 267,874 / 187,706 = 1.4271 (2026-10-04). + Oct 6 fix wave: quota_exhausted preflight branch (#3051) and quoted-heredoc issue template (#3046); measured 268,199 / 187,706 = 1.4288 (2026-10-06). + Step 10's inline no-PR check before the Greptile dispatch (#3020); measured 268,655 / 187,706 = 1.4313 (2026-10-06). + Oct 6 fix wave A3/CEO-12: free text reaches commands only through agent-written mktemp files (mktemp block, write rule, path rebuild and refusal at each posting, prompt and brief site); measured 275,422 / 187,706 = 1.4673 (2026-10-06). Combined Oct 6 wave head (B free-text files + D measure loop + A rework): measured 280,363 / 187,706 = 1.4936 (2026-10-06). + gate-once push rule (no second paid gate after a CI push); measured 280,573 / 187,706 = 1.4947 (2026-10-06). + the measurement bar's decision step (exit-code branches for MEETS, MEETS-qualified, needs-classify with the qualifying and fix-round classes, EXTEND once; the measure prose was tightened to offset most of it); measured 281,058 / 187,706 = 1.4973 (2026-10-06).
   },
   'plan-ceo-review': {
     skill: 'plan-ceo-review',

test/ship-measure-loop-skill.test.ts

@@ -1,7 +1,7 @@
 /**
  * A6 acceptance: the generated ship skill runs the measure-then-fix loop on a
- * red paid eval (classify, measure alone, fix at the cause, re-measure, gate
- * once) and measures a failed free shard before calling it pre-existing, and
+ * red paid eval (classify, measure alone, decide by the measurement bar, fix
+ * at the cause, re-measure, gate once) and measures a failed free shard before calling it pre-existing, and
  * it states the "diagnostic, not verdict" rule. Every ship-measure command and
  * flag the skill names exists in scripts/ship-measure.ts.
  */
@@ -25,8 +25,8 @@ describe('the generated ship skill runs the loop (A6 acceptance)', () => {
     expect(skeleton).toContain('| `sections/measure.md` |');
   });
 
-  test('the loop steps run in order: classify, plan, measure, fix, re-measure, stop, then gate once', () => {
-    const steps = ['1. **Classify.**', '2. **Show the plan.**', '3. **Measure.**', '4. **Fix at the cause.**', '5. **Re-measure.**', '6. **Stop.**', '**Gate once.**'];
+  test('the loop steps run in order: classify, plan, measure, decide, fix, re-measure, stop, then gate once', () => {
+    const steps = ['1. **Classify.**', '2. **Show the plan.**', '3. **Measure.**', '4. **Decide by the exit code**', '5. **Fix at the cause.**', '6. **Re-measure.**', '7. **Stop.**', '**Gate once.**'];
     const positions = steps.map(step => loop.indexOf(step));
     expect(positions.every(p => p >= 0)).toBe(true);
     expect([...positions].sort((a, b) => a - b)).toEqual(positions);
@@ -36,6 +36,17 @@ describe('the generated ship skill runs the loop (A6 acceptance)', () => {
     expect(loop).toContain('do not also start the gate command');
   });
 
+  test('the decision branches on every exit code the runner returns, with the bar and its rules', () => {
+    const decideStep = loop.slice(loop.indexOf('4. **Decide by the exit code**'), loop.indexOf('5. **Fix at the cause.**'));
+    for (const code of ['- 0: MEETS or MEETS-qualified', '- 5: needs-classify', '- 6: EXTEND', '- 1: BELOW']) expect(decideStep).toContain(code);
+    for (const cls of ['provider', 'judge-noise', 'model-miss', 'timeout', 'hang', 'regression', 'fixable', 'unclassified']) expect(decideStep).toContain(`\`${cls}\``);
+    expect(decideStep).toMatch(/MEETS 9\/10, MEETS-qualified 8\/10[\s\S]*EXTEND 7\/10; behavior 11\/12, 10\/12, 9\/12/);
+    expect(decideStep).toMatch(/never a third batch/i);
+    expect(loop.slice(loop.indexOf('7. **Stop.**'))).toContain('Exit 3 is a named red');
+    const source = fs.readFileSync(path.join(ROOT, 'scripts/lib/measure-bar.ts'), 'utf8');
+    for (const cls of ['provider', 'judge-noise', 'model-miss', 'timeout', 'hang', 'regression', 'fixable', 'unclassified']) expect(source).toContain(`'${cls}'`);
+  });
+
   test('the diagnostic-not-verdict rule, the honest-fix rule and the unmeasured label are stated', () => {
     expect(loop).toContain('Every rerun here is **diagnostic, not a verdict**: it never changes the recorded\nverdict of the run that failed');
     expect(loop).toContain('Never raise a budget, lower a\nthreshold, add a verdict retry or skip a case to get a pass.');