From 93d2d43c2c78b7580505e3846fcb66687236837f Mon Sep 17 00:00:00 2001 From: garrytan Date: Sat, 3 Oct 2026 01:26:13 +0000 Subject: [PATCH] test(W7c): shrink gen-skill-docs to structural and meaning checks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit W7c conversion of the sentence/caps pins in test/gen-skill-docs.test.ts (t01 F2-F9, F18, F19; t08 F12; t10 F4; t02 #5 duplicate STOP literal). t01 F1 ('Every commit must map to at least one bullet point') was already gone on the base. Pins (grep -cE 'toContain|toMatch|indexOf|includes') 1446 -> 1360; tests 457 -> 457. Untouched: the FRESH/STALE freshness check, the routing soft-policy negative, skill-start/end script pins, the SESSION_KIND spawned line, and the old-wording negative pins (delete-then-append, 0D/0F, gate "if applicable"). Machine markers stay byte-exact: NO UNRESOLVED DECISIONS, `**UNRESOLVED DECISIONS:**`, `Approval readiness: PASS`, `D — : ...`, `## EXIT PLAN MODE GATE (BLOCKING)` anchors, `**Gate outcome: Blocked**`, helper commands/flags (gstack-office-hours-review prepare/check/finalize --design, --out-dir, --report, --unreviewed), helper stop reasons, analytics commands. Consent, stop, approval, decision-cycle and navigation lines (incl. the Eng W7c-gate lines at the old 2362-2370) are kept as short case-insensitive checks. Removed assertion groups (invariant -> remaining protection): - qa-only 'Never fix bugs' + 'NEVER fix anything' (t01 F8): report-only skill -> kept free: /never fix/i in Additional Rules + frontmatter no Edit/Glob/Grep. - qa 'WTF' token (t08 F12): fix-loop self-check -> kept free: /every 5 fixes/i. - TEST_COVERAGE_AUDIT sharedPhrases sentences and the three Test-step sentences (t01 F4): read source before tracing/diagramming in plan and ship -> kept free: order 'concrete source and test files' < Trace data flow < Diagram the execution < Quality scoring rubric in both, star/GAP markers, the plan/diff mode split, the Test review subheading structure test; gate judges 'plan-eng-review/SKILL.md sections', 'ship/SKILL.md workflow'. - 'REGRESSION RULE' / 'IRON RULE' labels (t01 F4): regression coverage is required -> kept free: REGRESSION RULE heading, /without regression coverage, that coverage is a critical requirement/i (plan), /regression test is written immediately/i (ship), plus the existing regression-contract tests. - '**STOP for each pending decision.**' (t01 F7, retired) -> kept free: 'Wait for its answer before applying that remedy' (control C1). - F19 rule sentences: step numbers dropped; kept free /prevent duplicate bumps, pushes and PRs, never verification/i, /native result never supplies outside coverage/i, /required-probe gate, never silently waive/i, /never offer a conversion/i, /actual counts, never estimates/i (deduped 2 -> 1). - SPEC_REVIEW_LOOP (t01 F3, about 74 sentence pins): - office-hours dispatch sentences (Before EACH dispatch, parent Read does not deliver, string unchanged) -> kept free: prepare --design/--out-dir, order mktemp < prepare < round-N.prompt.md < `dispatch`, /all preceding valid round files in order/i, /read the entire prepared prompt/i. - reviewer prompt sentences -> kept free: document/verdict path and schema wiring, mode sections ('The Assignment', 'What I noticed...'), dimension order, short checks for coaching protection, no invented customer answers, minor findings included, identical JSON, classify every prior finding, open questions vs committed behavior. Dropped: 'Could an engineer implement this...' ban (positive design-and-coaching check carries it), absence-is-not- confirmation, unsupported-claims and labeling sentences -> gate E2E office-hours-spec-review; prior-status validation in office-hours-review.test.ts. - convergence evidence sentences, 'Recording concerns does not mark them fixed', Disposition, observation/attempt counting, 'unavailable score is null', 'missing or invalid verdict is ... never PASS', 'Preserve the failed output' -> office-hours-review.test.ts (convergence, finalize, invalid artifact vs --unreviewed, null quality_score) + gate E2E office-hours-spec-review; kept free: helper stop-reason order and round 3, /before fixing any findings or dispatching again/i, /on a stop, do not fix again or re-dispatch/i, /do not edit that generated section/i, finalize before /existing user approval/i, --unreviewed wiring, /do not fabricate JSON/i, /not an approval gate/i. - CEO spec-review sentences (Read Agent's tool definition, omit field, consume completed review, Keep both consistent, duplicated 'both complete labeled texts'/'all five dimensions'/'Preserve the failure'/'citing the owning input'/'## Reviewer Concerns', 'Read these documents' ban, strategy-only and narrow-decision sentences) -> CEO is periodic-only, so kept free: one short check per invariant (wait tool, no-wait resume, no advance while waiting, three launches, stop after third, stop the loop, missing score, null grade, SCORE 1-10, not persisted, mkdir/append stop, save-failure stop, ITERATIONS, FOUND/FIXED/REMAINING order, both inputs in full, partial-input refusal, summary-is-not-the-plan, 1-10 score, PASS rule, cite input) plus the existing order/count/command checks. - Design approval reconciliation (t01 F2, about 79 pins): - Pass 5 sentences and 'AskUserQuestion once per issue. Do NOT batch. Recommend + WHY.' -> kept free: /stop\W+AskUserQuestion once per issue/i located after four short reuse/new-evidence/first-decision checks; batching -> auq-format-always-loaded.test.ts PER_SKILL_RULES ('One issue = one AskUserQuestion call'). - duplicate 'Never group distinct issues.' pins (2 -> 1) and the rest of the Design gate 0 sentences -> kept free: 0 < 1 order, short checks per approval line, refresh < Read-back gate < rerun order; gate judge 'plan-design-review/SKILL.md passes'. - CEO readiness/gate/0D sentences -> kept free: one short check per approval line, readiness order, `Approval readiness: PASS`, gate order Read-only < Verify readiness < 1. Read, route order, /stop for the actual answer/i in step 4, `D — ` header, row fields; admin-route sentences -> ceo-workflow-clarity.test.ts kept checks. Dropped: 'No report or completion log is needed' (order check carries it), 'Save or present the updated plan', 'checked row IDs...' (marker kept), strategy-only/narrow-decision sentences (default kept), cross-model/integration sentences reduced to external-only and native-fallback checks. - Eng readiness/procedure sentences -> kept free: short approval checks, gate numbering/order, AUQ call < /stop until the actual answer arrives/i < Record the answer, recovery order; 'separate it and rebuild this comparison' -> auq-format-always-loaded.test.ts PER_SKILL_RULES. - Exit gate 'contract violation' sentence (t01 F9, design/devex/codex) -> kept free: heading anchor + /do not call ExitPlanMode/i inside the gate. - Unresolved-decisions gate (t01 F5): '(MANDATORY', 'Never omit'/'never omitted', 'FINAL non-whitespace line is the unresolved-decisions', 'FAILS the gate', 'DROP the current skill' -> kept free: NO UNRESOLVED DECISIONS, /final\s+non-whitespace\s+line/i, `**UNRESOLVED DECISIONS:**`, /missing\s+status [^.]*fails/i in every gate skill and the resolver, /avoids double-counting/i, the old soft-wording negative pin. - delete-then-append 'never mid-file' / 'Do NOT replace the section in place' (t01 F6) -> kept free: 'delete-then-append flow' + /append the new report at EOF/i; old-wording negatives kept. - design-html '**Never include by default (AI slop blacklist):**' (t01 F18) -> kept free: /AI slop blacklist/i, catalog id comment, lib/design-catalog.ts. - CODEX_BOUNDARY 'Do NOT read or execute any' casing (t10 F4) -> kept free: /do not read or execute any/i in every codex-calling skill. - CEO 0D order anchor '**STOP for the actual answer...**' (one of the 8) -> kept free: /stop for the actual answer/i at the same position. Mutation pass (scratch worktree, generated SKILL.md/sections plus spec-review.ts and lib/office-hours-review.ts; the freshness test is excluded with -t because any mutation makes it STALE): 43/43 as expected. 25 kept-step, marker, safety, numeric and retirement-control mutations go red; 18 rewordings of dropped sentences and lowered-register STOP/NEVER/FAILS lines stay green. Log: .context/mutation-gen-skill-docs.md. Runtime (Bun 1.4.0, wall, 3 runs): before 4.42/4.49/4.70s (median 4.49s), after 4.42/4.53/4.63s (median 4.53s). --- test/gen-skill-docs.test.ts | 509 +++++++++++++++--------------------- 1 file changed, 211 insertions(+), 298 deletions(-) diff --git a/test/gen-skill-docs.test.ts b/test/gen-skill-docs.test.ts index ed0034cb7..19afa4261 100644 --- a/test/gen-skill-docs.test.ts +++ b/test/gen-skill-docs.test.ts @@ -31,6 +31,12 @@ function readShipUnion(): string { return readSkillUnion('ship'); } +function ordered(text: string, anchors: string[]): void { + const positions = anchors.map(anchor => text.indexOf(anchor)); + expect(positions.every(position => position >= 0), `missing one of ${JSON.stringify(anchors)}`).toBe(true); + expect(positions).toEqual([...positions].sort((a, b) => a - b)); +} + function readExternalSkillUnion(output: string, hostSubdir: string, skill: string): string { const dir = path.join(output, hostSubdir, 'skills', `gstack-${skill}`); const sections = path.join(dir, 'sections'); @@ -635,8 +641,7 @@ describe('gen-skill-docs', () => { test('qa-only has no-fix guardrails', () => { const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8'); - expect(qaOnlyContent).toContain('Never fix bugs'); - expect(qaOnlyContent).toContain('NEVER fix anything'); + expect(extractMarkdownSection(qaOnlyContent, '## Additional Rules (qa-only specific)')).toMatch(/never fix/i); // Should not have Edit, Glob, or Grep in allowed-tools. // Scope to frontmatter (between the first two --- lines) — the body can // legitimately mention these tool names in prose (e.g., Claude model @@ -661,7 +666,7 @@ describe('gen-skill-docs', () => { expect(qaContent).toContain('Phase 8'); expect(qaContent).toContain('Fix Loop'); expect(qaContent).toContain('Triage'); - expect(qaContent).toContain('WTF'); + expect(qaContent).toMatch(/every 5 fixes/i); }); }); @@ -915,30 +920,10 @@ describe('TEST_COVERAGE_AUDIT placeholders', () => { // Review mode delegates test coverage to the Testing specialist subagent (Review Army) const normalizedPlanSkill = planSkill.replace(/\s+/g, ' '); const normalizedShipSkill = shipSkill.replace(/\s+/g, ' '); - const sharedPhrases = [ - 'Trace data flow', - 'Diagram the execution', - 'concrete source and test files', - 'dedicated tool call before drawing the diagram', - 'use separate calls for', - 'context. Base the diagram on that read', - 'start Test review output with the coverage diagram', - 'In full plan reviews, put it inside the normal Test review section', - 'Required outputs keep the final terminal report order', - 'Avoid bare `[ ]` or `[x]` in diagrams', - 'keep user-flow markers off code-path rows', - 'Quality scoring rubric', - '★★★', - '★★', - 'GAP', - ]; - for (const phrase of sharedPhrases) { - expect(normalizedPlanSkill).toContain(phrase); - expect(normalizedShipSkill).toContain(phrase); + for (const skill of [normalizedPlanSkill, normalizedShipSkill]) { + ordered(skill, ['concrete source and test files', '**Trace data flow.**', '**Diagram the execution.**', 'Quality scoring rubric']); + for (const marker of ['★★★', '★★', 'GAP']) expect(skill).toContain(marker); } - expect(normalizedPlanSkill).toContain('For every target, run these five Test steps inside Section 3, after Scope Challenge and the Architecture/Code Quality reviews'); - expect(normalizedPlanSkill).toContain('Within Test step 1, read concrete source/tests before tracing or diagramming; Test step 2 adds user flows'); - expect(normalizedShipSkill).toContain('Finish this source read before tracing data flow in audit item 2 below; map user flows afterward'); // Plan mode traces the plan, not a git diff expect(planSkill).toContain('Trace every codepath in the plan'); expect(planSkill).not.toContain('git diff origin'); @@ -964,9 +949,10 @@ describe('TEST_COVERAGE_AUDIT placeholders', () => { test('plan and ship modes include regression rule', () => { // Review mode delegates to Testing specialist for (const skill of [planSkill, shipSkill]) { - expect(skill).toContain('REGRESSION RULE'); - expect(skill).toContain('IRON RULE'); + expect(skill).toMatch(/^#{3,4} REGRESSION RULE/m); } + expect(planSkill).toMatch(/without regression coverage, that coverage is a critical requirement/i); + expect(shipSkill).toMatch(/regression test is written immediately/i); }); test('all five plan audit subheadings stay inside Test review while ship keeps its existing depth', () => { @@ -999,7 +985,6 @@ describe('TEST_COVERAGE_AUDIT placeholders', () => { expect(action).toContain('Collect the requirements for each GAP and the LLM/eval scope above'); expect(action).toContain('Carry forward required proof of approved behavior'); expect(action).toContain('Mark new contracts and optional depth choices pending until the decision gate below resolves them'); - expect(action).toContain('**STOP for each pending decision.**'); expect(action).toContain('Wait for its answer before applying that remedy'); expect(action).not.toContain('report the findings and their dispositions and continue'); expect(action).not.toContain('For each GAP identified in the diagram, add a test requirement'); @@ -1165,7 +1150,7 @@ describe('TEST_COVERAGE_AUDIT placeholders', () => { expect(shipSkill).toContain('**Route:**'); expect(shipSkill.replace(/\s+/g, ' ')).toContain('integrate (1–3) → test and review (4–11.5) → prepare the release (12–15) → verify frozen content (16) → push and publish (17–21)'); expect(shipSkill.replace(/\s+/g, ' ')).toContain('Every new invocation repeats Steps 1–16, including both reviews and the docs audit'); - expect(shipSkill.replace(/\s+/g, ' ')).toContain('Steps 12, 17 and 19 prevent duplicate bumps, pushes and PRs, never verification'); + expect(shipSkill.replace(/\s+/g, ' ')).toMatch(/prevent duplicate bumps, pushes and PRs, never verification/i); }); }); @@ -1264,7 +1249,7 @@ describe('PLAN_FILE_REVIEW_REPORT resolver', () => { expect(report).toContain('`' + field + '`'); } expect(outside).toContain(host.name === 'codex' ? 'claude auth login' : 'codex login'); - expect(outside).toContain('A native result never supplies outside coverage'); + expect(outside).toMatch(/native result never supplies outside coverage/i); } }); @@ -1393,7 +1378,7 @@ describe('PLAN_VERIFICATION_EXEC placeholder', () => { test('blocks unavailable required checks instead of silently skipping them', () => { const flat = shipSkill.replace(/\s+/g, ' '); expect(flat).toContain('Noninteractive runs return blocked'); - expect(flat).toContain("Send failed, blocked or unrun checks through Step 9's required-probe gate, never silently waive them"); + expect(flat).toMatch(/required-probe gate, never silently waive/i); expect(flat).toContain('Missing/unreadable assets block required QA'); expect(flat).toContain('explicitly accept each named probe\'s concrete risk'); expect(flat).toContain('Keep actual outcomes and incomplete flags; VERIFY_RESULT stays fail'); @@ -1547,118 +1532,82 @@ describe('SPEC_REVIEW_LOOP resolver', () => { for (const host of Object.keys(HOST_PATHS)) { const output = render('office-hours', host); expect(output).toContain('gstack-office-hours-review prepare --design'); - expect(output).toContain('Before EACH dispatch'); - expect(output).toContain('all preceding valid round files in order'); - expect(output).toContain('string unchanged as the prompt'); - expect(output).toContain('Read the entire prepared prompt'); - expect(output).toContain('A parent Read does not deliver the file to the reviewer'); - const prompt = renderOfficeHoursReviewerPrompt({ document: '/tmp/design.md', verdictPath: '/tmp/round-1.json' }); - expect(prompt).toContain('design and coaching document, produced before engineering'); - expect(prompt).toContain("startup-mode 'The Assignment'"); - expect(prompt).toContain("both modes' 'What I noticed about how you think'"); - expect(prompt).toContain('evaluate their evidence and usefulness'); - expect(prompt).toContain('do not remove them merely because they are coaching content'); - expect(prompt).toContain('Unknown customer facts may remain explicit Open Questions or assignments; do not invent answers'); - expect(prompt).toContain('unsupported claims, contradictions, safety/correctness risks'); - expect(prompt).toContain('missing behavior needed by the approach the document actually commits to'); - expect(prompt).toContain('a contradiction or a required behavior an open question does not resolve it'); - expect(prompt).toContain('clear enough for user approval and the next engineering review'); - expect(prompt).toContain('Are open discovery questions distinguished from committed behavior?'); - expect(prompt).toContain('Flag ambiguous or missing behavior in the chosen approach'); - expect(prompt).not.toContain('Could an engineer implement this without asking questions?'); - expect(prompt).toContain('return that identical JSON as your entire response'); - expect(prompt).toContain('include every unresolved problem and necessary remedy, including minor findings'); - expect(prompt).toContain('classify EVERY preceding finding as resolved, persisting, or unverified'); - expect(prompt).toContain('Absence from the new findings list is not confirmation'); - expect(prompt).toContain('specific document decision/behavior proving the status'); - expect(prompt).toContain('Never invent customer answers'); - expect(prompt).toContain('"version": 1'); - expect(prompt).toContain('"prior": []'); + expect(output).toContain('--out-dir ""'); + ordered(output, ['mktemp -d ".review.XXXXXX"', 'prepare --design', '`round-N.prompt.md`', '`dispatch`']); + expect(output).toMatch(/all preceding valid round files in order/i); + expect(output).toMatch(/read the entire prepared prompt/i); } + const prompt = renderOfficeHoursReviewerPrompt({ document: '/tmp/design.md', verdictPath: '/tmp/round-1.json' }); + expect(prompt).toContain('Document: /tmp/design.md'); + expect(prompt).toContain('Verdict: /tmp/round-1.json'); + expect(prompt).toContain('"document": "/tmp/design.md"'); + expect(prompt).toMatch(/design and coaching document/i); + for (const section of ["'The Assignment'", "'What I noticed about how you think'"]) expect(prompt).toContain(section); + expect(prompt).toMatch(/merely because they are coaching content/i); + expect(prompt).toMatch(/never invent customer answers/i); + expect(prompt).toMatch(/including minor findings/i); + expect(prompt).toMatch(/identical JSON/i); + expect(prompt).toMatch(/classify every preceding finding/i); + ordered(prompt, ['1. **Completeness**', '2. **Consistency**', '3. **Clarity**', '4. **Scope**', '5. **Feasibility**']); + expect(prompt).toMatch(/distinguished from committed behavior/i); + expect(prompt).toContain('"version": 1'); + expect(prompt).toContain('"prior": []'); }); test('office-hours checks stop conditions before edits and preserves concerns mechanically', () => { const output = render('office-hours'); const step2 = output.slice(output.indexOf('**Step 2:'), output.indexOf('**Step 3:')); - expect(step2).toContain('BEFORE fixing any findings or dispatching again'); - expect(step2).toContain('gstack-office-hours-review check'); - expect(step2).toContain('PASS: no unresolved findings'); - expect(step2).toContain('CONVERGENCE: the reviewer explicitly marked a prior obligation persisting'); - expect(step2).toContain('concrete prior/current finding pair and document evidence'); - expect(step2).toContain('Shared topic labels or new refinements alone are insufficient'); - expect(step2).toContain('MAX_ITERATIONS: round 3 completed; stop'); - expect(step2).toContain('CONTINUE: fix the listed findings'); - expect(step2).toContain('On a stop, do not fix again or re-dispatch'); - expect(step2).toContain('finalizer before approval'); - expect(step2).toContain('gstack-office-hours-review finalize --design'); - expect(step2).toContain('Recording concerns does not mark them fixed'); - expect(step2).toContain('Do not edit that generated section'); - expect(step2).toContain('Then proceed to Step 3 and the existing user approval'); - expect(step2.indexOf('gstack-office-hours-review check')).toBeLessThan(step2.indexOf('CONTINUE: fix')); + expect(step2).toMatch(/before fixing any findings or dispatching again/i); + ordered(step2, ['gstack-office-hours-review check', '- PASS:', '- CONVERGENCE:', '- MAX_ITERATIONS: round 3', '- CONTINUE:', 'gstack-office-hours-review finalize --design']); + expect(step2).toMatch(/on a stop, do not fix again or re-dispatch/i); + expect(step2).toMatch(/do not edit that generated section/i); + expect(step2.slice(step2.indexOf('finalize --design'))).toMatch(/existing user approval/i); }); test('office-hours finalizes the report after writing it and labels derivable metrics', () => { const output = render('office-hours'); - expect(output).toContain('with `--report ""` after the report exists'); - expect(output).toContain('Disposition mechanically'); - expect(output).toContain('Do not\nsummarize or replace that section afterward'); - expect(output).toContain('Preserve the Assignment, coaching, approval, and Handoff'); - expect(output).toContain('Confirmed resolutions require explicit later reviewer evidence'); - expect(output).toContain('finding observations across rounds'); - expect(output).toContain('attempted fix\nrounds are counted separately'); - expect(output).toContain('An unavailable score is null, never invented'); - expect(output).not.toContain('M issues caught and fixed'); + expect(output).toContain('--report ""'); + expect(output).toMatch(/after the report exists/i); + expect(output).toMatch(/do not\s+summarize or replace that section/i); + expect(output).toMatch(/preserve the assignment, coaching, approval/i); + expect(output).toMatch(/explicit later reviewer evidence/i); }); test('office-hours errors stay explicit and preserve preceding valid evidence', () => { const output = render('office-hours'); - expect(output).toContain('A missing or invalid verdict is an explicit review failure, never PASS'); - expect(output).toContain('Preserve the\nfailed output and its error'); - expect(output).toContain('--unreviewed'); - expect(output).toContain('preceding valid round files'); - expect(output).toContain('Do not fabricate JSON or hide a completed verdict'); - expect(output).toContain('quality bonus, not an approval gate'); + expect(output).toMatch(/--unreviewed ""`\s+and only the\s+preceding valid round files/); + expect(output).toMatch(/do not fabricate JSON/i); + expect(output).toMatch(/not an approval gate/i); }); test('CEO keeps its loop limits, scoring, failure handling, and reporting', () => { const ceo = render('plan-ceo-review'); - const report = ceo.split('**Step 3:')[1]!; - expect(report).toContain('For an unavailable review or missing/invalid grade, use JSON `null`'); - expect(ceo).toContain('implementable without follow-up questions'); + const report = ceo.split('**Step 3:')[1]!.replace(/\s+/g, ' '); + expect(report).toMatch(/use JSON `null`/i); + expect(ceo).toMatch(/implementable without follow-up questions/i); expect(ceo).not.toContain('design and coaching document'); expect(ceo).not.toContain('gstack-office-hours-review'); - const dispatch = ceo.slice(ceo.indexOf('**Step 1:'), ceo.indexOf('**Step 2:')); - expect(dispatch).toContain("Read Agent's tool definition"); - expect(dispatch).toContain('Set `run_in_background: false` if that field is available; omit it otherwise'); - expect(dispatch).toContain('If the result contains a completed review, consume it'); - expect(dispatch).toContain("If it returns a pending task, use the host's wait tool"); - expect(dispatch).toContain('With no wait tool, end this response and resume on its completion notification'); - expect(dispatch).toContain('While waiting, do not advance, edit either input or launch another reviewer'); - expect(dispatch).toContain('both complete labeled texts'); - expect(dispatch).toContain('all five dimensions'); - const fixes = ceo.slice(ceo.indexOf('**Step 2:'), ceo.indexOf('**Step 3:')); - const stages = ['use 0D for new or reopened choices', 'amend the working plan', - 're-dispatch with both updated inputs and the same instructions'].map(text => fixes.indexOf(text)); - expect(stages.every(index => index >= 0)).toBe(true); - expect(stages).toEqual([...stages].sort((a, b) => a - b)); - expect(fixes).toContain('Keep both consistent'); - expect(fixes).toContain('Make at most three reviewer launches'); - expect(fixes).toContain('Stop after the third review, or when consecutive reviews repeat the same unresolved issues'); - expect(fixes).toContain('If launch or review fails, times out, or cannot review both complete inputs, stop the loop'); - expect(fixes).toContain('Preserve the failure and all prior findings'); - expect(fixes).toContain('Continue to Step 3 to record the unavailable outcome; a successful reviewer result is not required'); - expect(fixes).toContain('A missing score alone does not require another review'); + const dispatch = ceo.slice(ceo.indexOf('**Step 1:'), ceo.indexOf('**Step 2:')).replace(/\s+/g, ' '); + expect(dispatch).toContain('`run_in_background: false`'); + expect(dispatch).toMatch(/use the host's wait tool/i); + expect(dispatch).toMatch(/end this response and resume on its completion/i); + expect(dispatch).toMatch(/while waiting, do not advance/i); + const fixes = ceo.slice(ceo.indexOf('**Step 2:'), ceo.indexOf('**Step 3:')).replace(/\s+/g, ' '); + ordered(fixes, ['use 0D', 'amend the working plan', 're-dispatch']); + expect(fixes).toMatch(/at most three reviewer launches/i); + expect(fixes).toMatch(/stop after the third review/i); + expect(fixes).toMatch(/stop the loop/i); + expect(fixes).toMatch(/successful reviewer result is not required/i); + expect(fixes).toMatch(/missing score alone does not require another review/i); expect(ceo).toContain('Spec review unavailable — presenting unreviewed doc.'); - expect(report).toContain('List unresolved issues under "## Reviewer Concerns"'); - expect(report).toContain('citing the owning input'); - expect(report).toContain("SCORE is the latest attempt's reported 1–10 grade after reviewing both full inputs"); - expect(report).toContain('Label earlier grades "prior review score"'); - expect(report).toContain('reviewer-confirmed fixes'); - expect(report).toContain('Use actual counts, never estimates'); - expect(report.replace(/\s+/g, ' ')).toContain('When writing is forbidden, show the actual fields as not persisted and continue without writing'); - expect(report).toContain('failed mkdir or append stops the review'); - expect(report.replace(/\s+/g, ' ')).toContain('Recording the **0H spec-review metrics** is required when writing is permitted, even if the reviewer failed'); - expect(report.replace(/\s+/g, ' ')).toContain('If the reviewer fails, report that limit and continue after recording the outcome; if a required save fails, stop before claiming completion'); + expect(report).toContain('"## Reviewer Concerns"'); + expect(report).toMatch(/SCORE is the latest attempt's reported 1–10 grade/i); + expect(report).toContain('"prior review score"'); + expect(report).toMatch(/as not persisted/i); + expect(report).toMatch(/failed mkdir or append stops the review/i); + expect(report).toContain('**0H spec-review metrics**'); + expect(report).toMatch(/required when writing is permitted/i); + expect(report).toMatch(/if a required save fails, stop before claiming completion/i); expect(report).toContain('eval "$(~/.claude/skills/gstack/bin/gstack-paths)"; : "${GSTACK_STATE_ROOT:?gstack-paths failed; reinstall with ./setup or /gstack-upgrade}"'); expect(report).toContain('mkdir -p "$GSTACK_STATE_ROOT/analytics" || exit 1'); expect(report).toContain('>> "$GSTACK_STATE_ROOT/analytics/spec-review.jsonl" || exit 1'); @@ -1667,42 +1616,40 @@ describe('SPEC_REVIEW_LOOP resolver', () => { test('CEO spec report separates findings, confirmed fixes, and unresolved concerns', () => { const report = render('plan-ceo-review').split('**Step 3: Report and persist metrics**')[1]!.replace(/\s+/g, ' '); - expect(report).toContain('ITERATIONS counts actual reviewer launches'); - expect(report).toContain('FOUND, FIXED and REMAINING count reported issues, reviewer-confirmed fixes and reported unresolved issues'); - expect(report).toContain('Use actual counts, never estimates'); + expect(report).toMatch(/ITERATIONS counts actual reviewer launches/i); + ordered(report, ['FOUND, FIXED and REMAINING', 'reviewer-confirmed fixes']); + expect(report).toMatch(/actual counts, never estimates/i); expect(report).not.toContain('M issues caught and fixed'); - expect(report).toContain('List unresolved issues under "## Reviewer Concerns"'); }); test('CEO spec review receives the full source plan as well as the scope artifact on every host', () => { for (const host of Object.keys(HOST_PATHS)) { const output = render('plan-ceo-review', host).replace(/\s+/g, ' '); - expect(output).toContain('Both saved absolute paths, or both complete labeled texts if either input is not persisted: CEO scope summary and current amended working plan'); - expect(output).toContain('Read both inputs in full'); - expect(output).toContain('Evaluate them together on all five dimensions'); - expect(output).toContain('Flag contradictions, unsupported accepted expansions and required behavior missing from both'); - expect(output).toContain('report that failure instead of grading partial input'); + expect(output).toMatch(/CEO scope summary and current amended working plan/i); + expect(output).toMatch(/read both inputs in full/i); + expect(output).toMatch(/all five dimensions/i); + expect(output).toMatch(/unsupported accepted expansions/i); + expect(output).toMatch(/instead of grading partial input/i); } const source = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md.tmpl'), 'utf8'); expect(source).toContain('## Plan under review\n{working plan path, or'); const template = source.replace(/\s+/g, ' '); - expect(template).toContain('Prepare the full amended working plan and a separate, consistent CEO scope summary'); - expect(template).toContain('the summary cannot serve as the plan'); - expect(template).toContain('**Save or present both inputs under the storage policy.**'); + expect(template).toMatch(/the summary cannot serve as the plan/i); + expect(template).toMatch(/save or present both inputs/i); }); test('CEO shares both inputs after spec review and owns unresolved concerns in its scope document', () => { const template = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md.tmpl'), 'utf8'); const persist = (template.split('### 0H.')[1]?.split('### 0I.')[0] ?? '').replace(/\s+/g, ' '); const review = persist.indexOf('{{SPEC_REVIEW_LOOP}}'); - const sharing = persist.indexOf('present both inputs for final scope-document approval'); + const sharing = persist.search(/present both inputs for final scope-document approval/i); expect(review).toBeGreaterThanOrEqual(0); expect(sharing).toBeGreaterThan(review); const output = render('plan-ceo-review'); - const processing = output.split('**Step 2:')[1]!.split('**Step 3:')[0]!; + const processing = output.split('**Step 2:')[1]!.split('**Step 3:')[0]!.replace(/\s+/g, ' '); const report = output.split('**Step 3:')[1] ?? ''; - expect(processing).toContain('consecutive reviews repeat the same unresolved issues'); - expect(processing).toContain('Preserve the failure and all prior findings'); + expect(processing).toMatch(/consecutive reviews repeat the same unresolved issues/i); + expect(processing).toMatch(/preserve the failure and all prior findings/i); expect(report).toContain('CEO summary'); expect(report).toContain('Reviewer Concerns'); expect(report).toMatch(/owning\s+input/); @@ -1710,15 +1657,13 @@ describe('SPEC_REVIEW_LOOP resolver', () => { test('CEO reviewer receives one complete prompt without duplicate scoring instructions', () => { const output = render('plan-ceo-review'); - const dispatch = output.split('**Step 1: Dispatch reviewer subagent**')[1]?.split('**Step 2:')[0] ?? ''; - expect(dispatch.match(/Read both inputs in full/g)).toHaveLength(1); - expect(dispatch).not.toContain('Read these documents and review them'); - expect(dispatch.match(/quality score/g)).toHaveLength(1); - expect(dispatch).toContain('A quality score (1-10) across all dimensions'); - expect(dispatch).toContain('Overall PASS only if all dimensions pass'); - expect(dispatch).toContain('For each dimension, PASS or numbered issues with suggested fixes'); - expect(dispatch).toContain('Cite input and requirement for each finding'); - expect(dispatch).toContain('report that failure instead of grading'); + const dispatch = (output.split('**Step 1: Dispatch reviewer subagent**')[1]?.split('**Step 2:')[0] ?? '').replace(/\s+/g, ' '); + expect(dispatch.match(/Read both inputs in full/gi)).toHaveLength(1); + expect(dispatch.match(/quality score/gi)).toHaveLength(1); + expect(dispatch).toMatch(/quality score \(1-10\)/i); + expect(dispatch).toMatch(/overall PASS only if all dimensions pass/i); + expect(dispatch).toMatch(/PASS or numbered issues/i); + expect(dispatch).toMatch(/cite input and requirement/i); }); test('contains all 5 review dimensions', () => { @@ -1844,13 +1789,13 @@ describe('Codex filesystem boundary', () => { 'office-hours', // second opinion resolver ]; - const BOUNDARY_MARKER = 'Do NOT read or execute any'; + const BOUNDARY_MARKER = /do not read or execute any/i; test('boundary instruction appears in all skills that call codex', () => { for (const skill of CODEX_CALLING_SKILLS) { // Union: ship's codex call lives in sections/adversarial.md after the carve. const content = readSkillUnion(skill); - expect(content).toContain(BOUNDARY_MARKER); + expect(content).toMatch(BOUNDARY_MARKER); } }); @@ -1872,7 +1817,7 @@ describe('Codex filesystem boundary', () => { // prompts. Carved: the adversarial step lives in sections/adversarial.md. const reviewContent = readSkillUnion('review'); // Boundary should appear near codex exec invocations - const boundaryIdx = reviewContent.indexOf(BOUNDARY_MARKER); + const boundaryIdx = reviewContent.search(BOUNDARY_MARKER); const codexExecIdx = reviewContent.indexOf('codex exec'); // Both must exist and boundary must come before a codex exec call expect(boundaryIdx).toBeGreaterThan(-1); @@ -1889,7 +1834,7 @@ describe('Codex filesystem boundary', () => { expect(boundarySection).not.toContain('~/.claude/'); expect(boundarySection).not.toContain('.agents/skills'); expect(boundarySection).toContain('skills/gstack'); - expect(boundarySection).toContain(BOUNDARY_MARKER); + expect(boundarySection).toMatch(BOUNDARY_MARKER); }); }); @@ -2312,7 +2257,7 @@ describe('DESIGN_HARD_RULES resolver', () => { test('design-html blacklist lines carry catalog ids', () => { const content = fs.readFileSync(path.join(ROOT, 'design-html', 'SKILL.md'), 'utf-8'); - expect(content).toContain('**Never include by default (AI slop blacklist):**'); + expect(content).toMatch(/AI slop blacklist/i); expect(content).toContain('Purple/blue gradients as default '); expect(content).toContain('lib/design-catalog.ts'); }); @@ -2354,163 +2299,135 @@ describe('Design approval reconciliation', () => { test('token alignment reuses the approved outcome while preserving first decisions and new tradeoffs', () => { const section = fs.readFileSync(path.join(ROOT, 'plan-design-review/sections/review-sections.md'), 'utf8'); const pass = extractMarkdownSection(section, '### Pass 5: Design System Alignment'); - const stop = pass.indexOf('**STOP.**'); - expect(stop).toBeGreaterThan(0); - const beforeQuestion = pass.slice(0, stop); - expect(beforeQuestion).toContain('check whether an earlier pass already approved that outcome'); - expect(beforeQuestion).toContain('apply the established tokens and update every stale gap/reference under that decision'); - expect(beforeQuestion).toContain('changing the plan location or spelling out the same fix is not a new issue'); - expect(beforeQuestion).toContain('Ask again only if new evidence exposes an unresolved requirement or tradeoff, and name it'); - expect(beforeQuestion).toContain('An unapproved violation still needs its first individual decision'); - expect(pass.slice(stop)).toContain('AskUserQuestion once per issue. Do NOT batch. Recommend + WHY.'); + const ask = pass.search(/stop\W+AskUserQuestion once per issue/i); + expect(ask).toBeGreaterThan(0); + const beforeQuestion = pass.slice(0, ask); + expect(beforeQuestion).toMatch(/already approved that outcome/i); + expect(beforeQuestion).toMatch(/not a new issue/i); + expect(beforeQuestion).toMatch(/ask again only if new evidence/i); + expect(beforeQuestion).toMatch(/unapproved violation still needs its first individual decision/i); }); test('loaded review section distinguishes individual issue decisions from navigation', () => { - const section = fs.readFileSync(path.join(ROOT, 'plan-design-review/sections/review-sections.md'), 'utf8'); - expect(section).toContain('Complete one decision cycle per unresolved finding'); - expect(section).toContain('Scope, focus, setup, and next-step choices approve no remedies.'); - expect(section).toContain('Never use the final next-step AskUserQuestion to satisfy the issue-approval loop.'); - expect(section).toContain('With no unresolved findings, no issue question is required.'); + const section = fs.readFileSync(path.join(ROOT, 'plan-design-review/sections/review-sections.md'), 'utf8').replace(/\s+/g, ' '); + expect(section).toMatch(/one decision cycle per unresolved finding/i); + expect(section).toMatch(/next-step choices approve no remedies/i); + expect(section).toMatch(/never use the final next-step AskUserQuestion to satisfy the issue-approval loop/i); + expect(section).toMatch(/with no unresolved findings, no issue question is required/i); }); test('Design blocking Exit checklist reconciles approvals and refreshes stale output after a decision', () => { const main = fs.readFileSync(path.join(ROOT, 'plan-design-review/SKILL.md'), 'utf8'); const check = extractMarkdownSection(main, '## Section self-check'); - expect(check).toContain('Before summaries, review logs or next-step menus, run approval check 0 below.'); - const gate = extractMarkdownSection(main, '## EXIT PLAN MODE GATE (BLOCKING)'); - expect(gate.indexOf('0. Approvals:')).toBeGreaterThanOrEqual(0); - expect(gate.indexOf('0. Approvals:')).toBeLessThan(gate.indexOf('1. Read the plan file')); - expect(gate).toContain("each issue's remedy needs its own AskUserQuestion call and answer."); - expect(gate).toContain("Never group distinct issues."); - expect(gate).toContain('Never group distinct issues. DESIGN.md tokens and navigation are not approval.'); - expect(gate).toContain('Honor prior exact decisions and preamble-authorized per-issue auto-decisions;'); - expect(gate).toContain('preamble-authorized'); - expect(gate).toContain('record why. Deferrals remain unresolved.'); - expect(gate).toContain('If missing, reset drafts to pending, ask and wait.'); - expect(gate).toContain('refresh the plan and report, pass the Read-back gate, then update the review'); - expect(gate).toContain('log and rerun this gate.'); - expect(gate).toContain('after your most recent write to it'); + expect(check).toMatch(/run approval check 0/i); + const gate = extractMarkdownSection(main, '## EXIT PLAN MODE GATE (BLOCKING)').replace(/\s+/g, ' '); + ordered(gate, ['0. Approvals:', '1. Read the plan file']); + const approvals = gate.slice(gate.indexOf('0. Approvals:'), gate.indexOf('1. Read the plan file')); + expect(approvals).toMatch(/own AskUserQuestion call and answer/i); + expect(approvals).toMatch(/never group distinct issues/i); + expect(approvals).toMatch(/navigation are not approval/i); + expect(approvals).toContain('preamble-authorized'); + expect(approvals).toMatch(/deferrals remain unresolved/i); + expect(approvals).toMatch(/reset drafts to pending, ask and wait/i); + ordered(approvals, ['refresh the plan and report', 'Read-back gate', 'rerun this gate']); + expect(gate).toMatch(/after your most recent write/i); }); test('CEO readiness binds actual approvals before its read-only exit verification', () => { const main = fs.readFileSync(path.join(ROOT, 'plan-ceo-review/SKILL.md'), 'utf8'); - const check = extractMarkdownSection(main, '## Section self-check'); + const check = extractMarkdownSection(main, '## Section self-check').replace(/\s+/g, ' '); const gate = extractMarkdownSection(main, '## EXIT PLAN MODE GATE (BLOCKING)').replace(/\s+/g, ' '); const section = fs.readFileSync(path.join(ROOT, 'plan-ceo-review/sections/review-sections.md'), 'utf8'); const readiness = extractMarkdownSection(section, '## Approval readiness').replace(/\s+/g, ' '); expect(section.indexOf('## Approval readiness')).toBeLessThan(section.indexOf('## Required Outputs')); - expect(readiness).toContain('No report or completion log is needed to run this check'); - expect(readiness).toContain('For each approved remedy:'); - expect(readiness).toContain('its actual answer, exact prior approval or preamble-authorized per-issue auto-decision'); - expect(readiness).toContain('Setup, mode and navigation are not remedy approvals'); - expect(readiness).toContain('an approach approves only its explicit commitments and their directly required tests'); - expect(readiness).toContain("the plan applies only that answer's scope"); - expect(readiness).toContain('Independent remedies and additional verification choices need their own rows and answers'); - expect(readiness).toContain('Keep declined, deferred and unanswered changes out of accepted work'); - expect(readiness).toContain('An approved delivery-scope deferral is settled'); - expect(readiness).toContain('Deferring a needed policy or remedy decision leaves that choice unresolved; show it in the final report'); - expect(readiness).toContain('If a draft lacks approval, mark it pending and use 0D; repeat this check after its answer'); - expect(readiness).toContain('At the end of the six-column decision ledger, record `Approval readiness: PASS`'); - expect(readiness).toContain('the checked row IDs and their actual answer or approval references'); - expect(readiness).toContain('Save or present the updated plan under Step 0'); - expect(readiness).toContain('A substantive change invalidates this result; navigation alone does not'); - const normalizedMain = main.replace(/\s+/g, ' '); - expect(normalizedMain).toContain('Default to implementation-ready'); - expect(normalizedMain).toContain('Use strategy-only only when the user asks for'); - expect(normalizedMain).toContain('Use one narrow decision only when the user names a single choice'); - expect(gate).toContain('Verify `Approval readiness: PASS` against current row IDs and answer references'); - expect(gate).toContain('Read-only verification'); - expect(gate.indexOf('Verify `Approval readiness: PASS`')).toBeLessThan(gate.indexOf('1. Read the plan file')); - expect(gate).toContain('If stale because a choice changed'); - expect(gate.replace(/\s+/g, ' ')).toContain('return to 0D for that choice only; then repeat readiness, affected outputs, report Read-back, Review Log and dashboard'); - expect(gate).toContain('Failed checks use **Gate outcome: Blocked**'); + expect(readiness).toMatch(/exact prior approval or preamble-authorized/i); + expect(readiness).toMatch(/navigation are not remedy approvals/i); + expect(readiness).toMatch(/approves only its explicit commitments/i); + expect(readiness).toMatch(/applies only that answer's scope/i); + expect(readiness).toMatch(/independent remedies[^.]*need their own rows and answers/i); + expect(readiness).toMatch(/keep declined, deferred and unanswered changes out of accepted work/i); + expect(readiness).toMatch(/delivery-scope deferral is settled/i); + expect(readiness).toMatch(/leaves that choice unresolved/i); + expect(readiness).toMatch(/mark it pending and use 0D/i); + expect(readiness).toContain('`Approval readiness: PASS`'); + expect(readiness).toMatch(/navigation alone does not/i); + expect(main.replace(/\s+/g, ' ')).toMatch(/default to implementation-ready/i); + ordered(gate, ['Read-only verification', 'Verify `Approval readiness: PASS`', '1. Read the plan file']); + expect(gate).toMatch(/return to 0D for that choice only; then repeat readiness/i); + expect(gate).toContain('**Gate outcome: Blocked**'); expect(main).toContain('**Pass with log-only gaps:**'); - expect(gate).toContain('end without success telemetry, ExitPlanMode or the queued handoff'); - expect(check.replace(/\s+/g, ' ')).toContain('Confirm you Read `sections/review-sections.md` and executed Sections 1–10'); - expect(check.replace(/\s+/g, ' ')).toContain("Section 11's findings or no-UI skip, required outputs and report from that file"); - expect(check).toContain('stop, Read and redo the review'); + expect(gate).toMatch(/end without success telemetry, ExitPlanMode or the queued handoff/i); + expect(check).toContain('`sections/review-sections.md`'); + expect(check).toMatch(/stop, Read and redo the review/i); const decisions = main.slice(main.indexOf('### 0D.'), main.indexOf('### 0E.')).replace(/\s+/g, ' '); - expect(decisions).toMatch(/Ask one row per call with that object unchanged, without recomposing/); - expect(decisions).toContain("**Choose the question's route first:**"); - expect(decisions).toContain('**Admin question:** mode, setup, navigation, document approval or promotion'); - const admin = decisions.split('- **Admin question:**')[1]!.split('- **Plan decision:**')[0]!; - expect(admin).toContain('Use its listed menu and the preamble question transport, then wait and record the answer'); - expect(admin).toContain('Skip steps 1–4; this approves no plan changes'); - expect(decisions).toContain('**Plan decision:** review-depth expansion, scope additions/cuts, approach choices, TODOs, specs and review/outside findings'); - expect(decisions).toContain('Start at step 1. Reuse exact prior approvals; run steps 2–4 only when a new answer is needed, even for one option'); - expect(decisions).toContain('If an admin answer requests a plan change, use the Plan decision route for that change'); - expect(decisions).toContain('Compare its question, header, labels and full descriptions literally with the verified fields'); + ordered(decisions, ["**Choose the question's route first:**", '**Admin question:**', '**Plan decision:**']); + expect(decisions).toMatch(/ask one row per call/i); + expect(decisions).toMatch(/reuse exact prior approvals/i); + expect(decisions).toMatch(/compare its question, header, labels and full descriptions literally/i); expect(decisions).toContain('`D — : `'); const row = decisions.slice(decisions.indexOf('**Pre-question checkpoint:**'), decisions.indexOf('Effort/risk must')); - expect(row).toMatch(/Find exactly one (?:ledger )?row/); - for (const field of ['ID', 'step 2', 'owner', 'Current/Proposed', 'Status', 'Exact approval and scope']) expect(row).toContain(field); - expect(decisions).toContain('**STOP for the actual answer, even for a lone option.**'); - expect(decisions).toMatch(/amend only (?:what it authorizes|authorized work)/); - expect(decisions).toContain('Reuse exact approvals. Reopen only for contradictions, changed assumptions or user instructions, never speculation or reviewer agreement'); - expect(decisions).toMatch(/reopen only|requires reopening it/i); - expect(decisions).toContain('Record findings even after resolution'); - expect(decisions).toContain('say "No issues, moving on." only with none'); + expect(row).toMatch(/find exactly one (?:ledger )?row/i); + for (const field of ['ID', 'owner', 'Current/Proposed', 'Status', 'Exact approval and scope']) expect(row).toContain(field); + expect(decisions.slice(decisions.indexOf('**4. Ask'))).toMatch(/stop for the actual answer/i); + expect(decisions).toMatch(/amend only (?:what it authorizes|authorized work)/i); + expect(decisions).toMatch(/reopen only for contradictions, changed assumptions or user instructions/i); + expect(decisions).toMatch(/record findings even after resolution/i); + expect(decisions).toContain('"No issues, moving on."'); const outside = extractMarkdownSection(section, '**Cross-model tension:**').replace(/\s+/g, ' '); - expect(outside).toContain('compare reviews only if an external reviewer completed'); - expect(outside).toContain('For a same-harness/native fallback, skip this comparison'); - expect(outside).toContain('do not write a CROSS-MODEL line'); + expect(outside).toMatch(/only if an external reviewer completed/i); + expect(outside).toContain('CROSS-MODEL'); const findings = extractMarkdownSection(section, '**Integrate reviewer findings:**').replace(/\s+/g, ' '); - expect(findings).toContain('either an external reviewer or the bounded native fallback completed with a valid report'); - expect(findings).toContain('Native fallback findings count as findings from the current harness'); - expect(findings).toContain('Apply Outside Voice Integration Rule to every finding'); + expect(findings).toMatch(/bounded native fallback/i); + expect(findings).toContain('Outside Voice Integration Rule'); }); test('Eng cannot exit with unasked findings listed only in an unresolved-decisions report', () => { const main = fs.readFileSync(path.join(ROOT, 'plan-eng-review/SKILL.md'), 'utf8'); - const check = extractMarkdownSection(main, '## Section self-check'); const rawGate = extractMarkdownSection(main, '## EXIT PLAN MODE GATE (BLOCKING)'); const gate = rawGate.replace(/\s+/g, ' '); const section = fs.readFileSync(path.join(ROOT, 'plan-eng-review/sections/review-sections.md'), 'utf8'); const readiness = extractMarkdownSection(section, '## Approval readiness'); const normalizedReadiness = readiness.replace(/\s+/g, ' '); expect(section.indexOf('## Approval readiness')).toBeLessThan(section.indexOf('## Required outputs')); - expect(normalizedReadiness).toContain('Only the ledger is needed here; completion outputs and logs come next'); - expect(normalizedReadiness).toContain('At the end of `## Decision ledger`, record `Approval readiness: PASS`'); - expect(normalizedReadiness).toContain('checked IDs and actual answer references'); - expect(normalizedReadiness).toContain('A substantive change invalidates this result; navigation alone does not'); + expect(normalizedReadiness).toContain('`Approval readiness: PASS`'); + expect(normalizedReadiness).toMatch(/navigation alone does not/i); expect(readiness).not.toMatch(/^ {3}\S/m); - expect(gate).toContain('Confirm Approval readiness passed for the current decisions'); - expect(gate).toContain('read-only verification, not a new approval or output-writing step'); + expect(gate).toMatch(/confirm approval readiness passed/i); + expect(gate).toMatch(/read-only verification, not a new approval/i); const approvalParagraph = rawGate.slice(rawGate.indexOf('Confirm Approval readiness'), rawGate.indexOf('Verify all five checks')); expect(approvalParagraph).not.toMatch(/^ {3}\S/m); expect([...rawGate.matchAll(/^([1-5])\. /gm)].map(match => match[1])).toEqual(['1', '2', '3', '4', '5']); - expect(gate.indexOf('Confirm Approval readiness')).toBeLessThan(gate.indexOf('1. Read the report file')); - expect(normalizedReadiness).toContain('check the ledger against every accepted remedy'); - expect(normalizedReadiness).toContain('Each must cite its own actual answer, exact prior approval or authorized auto-decision'); - expect(normalizedReadiness).toContain('setup, mode, approach and navigation do not count'); - expect(normalizedReadiness).toContain('Deferrals remain unresolved'); - expect(normalizedReadiness).toContain('Carry forward an exact approved regression contract'); - expect(normalizedReadiness).toContain('Otherwise, its behavior and assertions need one dedicated decision'); - expect(readiness).not.toContain('REGRESSION test is already authorized'); - expect(normalizedReadiness).toContain('If approval is missing, mark that draft pending, resolve the choice through Decision procedure and repeat this check'); - expect(normalizedReadiness).toContain('Continue to Required outputs, preserving unresolved decisions in the report'); - // Readiness verifies evidence; the one procedure owns asking and applying. - // The procedure contains a fenced H2 ledger example; use its real next step. + ordered(gate, ['Confirm Approval readiness', '1. Read the report file']); + expect(normalizedReadiness).toMatch(/every accepted remedy/i); + expect(normalizedReadiness).toMatch(/its own actual answer, exact prior approval or authorized auto-decision/i); + expect(normalizedReadiness).toMatch(/navigation do not count/i); + expect(normalizedReadiness).toMatch(/deferrals remain unresolved/i); + expect(normalizedReadiness).toMatch(/carry forward an exact approved regression contract/i); + expect(normalizedReadiness).toMatch(/one dedicated decision/i); + expect(normalizedReadiness).toMatch(/mark that draft pending/i); + expect(normalizedReadiness).toMatch(/preserving unresolved decisions in the report/i); const decisionStart = section.indexOf('\n## Decision procedure\n'); const decisionEnd = section.indexOf('\n## Scope Challenge\n', decisionStart); expect(decisionStart).toBeGreaterThan(0); expect(decisionEnd).toBeGreaterThan(decisionStart); const decisions = section.slice(decisionStart, decisionEnd).replace(/\s+/g, ' '); expect(decisions).toContain('AskUserQuestion({ questions: [currentDecision] })'); - expect(decisions).toContain('one question object for one choice; other IDs wait'); - expect(decisions).toContain("If you discover another independent choice, separate it and rebuild this comparison before saving or sending the question"); - expect(decisions.indexOf("**STOP until the actual answer arrives.**")).toBeLessThan(decisions.indexOf('### Record the answer')); - expect(decisions).toContain("For the next choice, use the updated working plan and answer"); - expect(gate).toContain('report the stale verification and stop'); - expect(gate).toContain('Resume under **Recovery routing → Late change or missing work**'); + expect(decisions).toMatch(/one question object for one choice/i); + const ask = decisions.indexOf('AskUserQuestion({ questions: [currentDecision] })'); + const stop = decisions.search(/stop until the actual answer arrives/i); + expect(stop).toBeGreaterThan(ask); + expect(stop).toBeLessThan(decisions.indexOf('### Record the answer')); + expect(decisions).toMatch(/for the next choice, use the updated working plan and answer/i); + expect(gate).toMatch(/report the stale verification and stop/i); + expect(gate).toContain('**Recovery routing → Late change or missing work**'); const recovery = readSkillUnion('plan-eng-review').split('**Late change or missing work:**')[1]!.split('**Blocked outcome:**')[0]!.replace(/\s+/g, ' '); - expect(recovery).toContain('new or reopened choices use Decision procedure'); - expect(recovery).toContain('Repeat Approval readiness, then Required outputs steps 1–4 for changed outputs before choosing navigation again'); - expect(gate).toContain('all six columns: Review / Trigger / Why / Runs / Status / Findings'); - expect(gate).toContain('follow **Blocked outcome**'); + expect(recovery).toMatch(/reopened choices use Decision procedure/i); + ordered(recovery, ['Repeat Approval readiness', 'Required outputs', 'navigation']); + expect(gate).toContain('Review / Trigger / Why / Runs / Status / Findings'); + expect(gate).toContain('**Blocked outcome**'); const report = extractMarkdownSection(section, '### Write to the report file'); - expect(report).toContain('Then follow **Blocked outcome** in the entrypoint.'); - expect(report).toContain('report the error and follow **Blocked outcome** before Review Log or decision logging'); + expect(report).toContain('**Blocked outcome**'); + expect(report.replace(/\s+/g, ' ')).toMatch(/follow \*\*Blocked outcome\*\* before Review Log or decision logging/i); }); test('approval entry does not alter other review Exit checklists', () => { @@ -2523,13 +2440,13 @@ describe('Design approval reconciliation', () => { }); test('task and decision reporting require approval while preserving unanswered findings', () => { - const section = fs.readFileSync(path.join(ROOT, 'plan-design-review/sections/review-sections.md'), 'utf8'); - const reconcile = section.indexOf('Before synthesizing tasks or the completion summary'); + const section = fs.readFileSync(path.join(ROOT, 'plan-design-review/sections/review-sections.md'), 'utf8').replace(/\s+/g, ' '); + const reconcile = section.search(/before synthesizing tasks or the completion summary/i); expect(reconcile).toBeGreaterThan(0); expect(reconcile).toBeLessThan(section.indexOf('## Implementation Tasks')); - expect(section).toContain('Export only agreed implementation work; retain unapproved remedies as pending findings.'); - expect(section).toContain('Count only individually approved new decisions'); - expect(section).toContain('including a finding not yet asked'); + expect(section).toMatch(/retain unapproved remedies as pending findings/i); + expect(section).toMatch(/count only individually approved new decisions/i); + expect(section).toMatch(/including a finding not yet asked/i); }); }); @@ -2669,7 +2586,7 @@ describe('DESIGN_MD_CHECK resolver and open DESIGN.md adoption', () => { const dr = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8'); expect(dr).toContain('gstack-design-md.ts check DESIGN.md'); expect(dr).toContain('gstack-design-md.ts tokens DESIGN.md'); - expect(dr).toContain('never offer a conversion here'); + expect(dr).toMatch(/never offer a conversion/i); expect(dr).not.toContain('mark legacy-keep'); const dh = fs.readFileSync(path.join(ROOT, 'design-html', 'SKILL.md'), 'utf-8'); expect(dh).toContain('# gstack: design-md-format=spec'); @@ -2849,7 +2766,7 @@ describe('Codex generation (--host codex)', () => { const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8'); // Outside prompts may explicitly forbid reading Claude's skill directory. // Every executable/runtime path must still use the selected host root. - const withoutBoundary = content.replace(/^.*IMPORTANT: Do NOT read or execute[^\n]*$/gm, ''); + const withoutBoundary = content.replace(/^.*IMPORTANT: do not read or execute[^\n]*$/gim, ''); expect(withoutBoundary).not.toContain('~/.claude/'); } }); @@ -4499,7 +4416,8 @@ describe('plan-mode-info resolver (handshake-replacement)', () => { const constructIdx = content.indexOf('Build one `currentDecision`', approachIdx); const saveIdx = content.indexOf('**Pre-question checkpoint:**', constructIdx); const presentIdx = content.indexOf('Ask one row per call', saveIdx); - const stopIdx = content.indexOf('**STOP for the actual answer, even for a lone option.**', presentIdx); + const stopOffset = content.slice(presentIdx).search(/stop for the actual answer/i); + const stopIdx = stopOffset < 0 ? -1 : presentIdx + stopOffset; const modeIdx = content.indexOf('### 0E. Mode Selection'); const preludeIdx = content.indexOf('### 0F'); const positions = [approachIdx, constructIdx, saveIdx, presentIdx, stopIdx, modeIdx, preludeIdx]; @@ -4577,8 +4495,7 @@ describe('GSTACK REVIEW REPORT delete-then-append flow', () => { // The new (correct) instruction must be present. expect(content).toContain('delete-then-append flow'); - expect(content).toContain('never mid-file'); - expect(content).toContain('Do NOT replace the section in place'); + expect(content).toMatch(/append the new report at EOF/i); // The old contradictory bullets must be gone. The signature phrase // from the buggy prompt was 'replace it entirely using the Edit tool' @@ -4591,8 +4508,7 @@ describe('GSTACK REVIEW REPORT delete-then-append flow', () => { test('plan-file review report resolver renders the rewritten flow', () => { const src = renderPlanReportAndGate(); expect(src).toContain('delete-then-append flow'); - expect(src).toContain('never mid-file'); - expect(src).toContain('Do NOT replace the section in place'); + expect(src).toMatch(/append the new report at EOF/i); // Old contradictory bullets are gone from the source resolver. expect(src).not.toContain('replace it** entirely using the Edit tool'); expect(src).not.toContain('If it was found mid-file, move it'); @@ -4696,16 +4612,18 @@ describe('EXIT PLAN MODE GATE placement', () => { const gate = extractMarkdownSection(md, '## EXIT PLAN MODE GATE (BLOCKING)').replace(/\s+/g, ' '); expect(gate).toContain('Run this final verification for every review target, in every host mode'); expect(gate).toContain('If any check fails, follow **Blocked outcome** without success telemetry or ExitPlanMode'); - } else expect(md, `${skill}/SKILL.md gate body`).toContain(skill === 'plan-ceo-review' - ? 'Failed checks use **Gate outcome: Blocked**' - : 'Failing this gate and calling ExitPlanMode anyway is a contract violation'); + } else if (skill === 'plan-ceo-review') { + expect(md, `${skill}/SKILL.md gate body`).toContain('Failed checks use **Gate outcome: Blocked**'); + } else { + expect(extractMarkdownSection(md, '## EXIT PLAN MODE GATE (BLOCKING)'), `${skill}/SKILL.md gate body`).toMatch(/do not call ExitPlanMode/i); + } } }); test('codex/SKILL.md contains gate (mid-file per D5; Step 2B/2C follow)', () => { const codex = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8'); expect(codex).toContain('## EXIT PLAN MODE GATE (BLOCKING)'); - expect(codex).toContain('Failing this gate and calling ExitPlanMode anyway is a contract violation'); + expect(extractMarkdownSection(codex, '## EXIT PLAN MODE GATE (BLOCKING)')).toMatch(/do not call ExitPlanMode/i); }); }); @@ -4800,11 +4718,8 @@ describe('GSTACK REVIEW REPORT mandatory unresolved-decisions status', () => { test(`${skill}: report mandates the unresolved-decisions status as final content`, () => { const content = readSkillUnion(skill); expect(content).toContain('NO UNRESOLVED DECISIONS'); - // The "never omit / always final" contract must be present, not just the phrase. - expect(content).toContain('Unresolved-decisions status (MANDATORY'); - expect(content).toMatch(skill === 'plan-ceo-review' ? /Never omit this status/ : /never omitted/); // \s+ tolerates prose line-wraps within "final non-whitespace line". - expect(content).toMatch(/final\s+non-whitespace\s+line/); + expect(content).toMatch(/final\s+non-whitespace\s+line/i); }); } @@ -4815,29 +4730,27 @@ describe('GSTACK REVIEW REPORT mandatory unresolved-decisions status', () => { expect(md).toContain('NO UNRESOLVED DECISIONS'); if (['plan-ceo-review', 'plan-eng-review'].includes(skill)) { const gate = md.split('## EXIT PLAN MODE GATE (BLOCKING)')[1]!.replace(/\s+/g, ' '); - expect(gate).toContain('final non-whitespace line is the exact unbolded `NO UNRESOLVED DECISIONS`'); - expect(gate).toContain('or the last bullet under `**UNRESOLVED DECISIONS:**`'); - expect(gate).toMatch(/A bolded sentinel, missing status or (?:any )?trailing prose fails this check/); + expect(gate).toMatch(/final\s+non-whitespace\s+line is the exact unbolded `NO UNRESOLVED DECISIONS`/i); + expect(gate).toContain('`**UNRESOLVED DECISIONS:**`'); + expect(gate).toMatch(/missing\s+status[^.]*fails/i); expect(gate).toContain(skill === 'plan-eng-review' ? 'If any check fails, follow **Blocked outcome** without success telemetry or ExitPlanMode' : 'Failed checks use **Gate outcome: Blocked**'); } else { - expect(md).toContain('FINAL non-whitespace line is the unresolved-decisions'); - expect(md).toContain('FAILS the gate'); + const gate = extractMarkdownSection(md, '## EXIT PLAN MODE GATE (BLOCKING)'); + expect(gate).toMatch(/final\s+non-whitespace\s+line/i); + expect(gate).toContain('`**UNRESOLVED DECISIONS:**`'); + expect(gate).toMatch(/missing\s+status[^.]*fails/i); } }); } test('plan-file review report and exit gate resolvers render the mandatory block + blocking gate', () => { const src = renderPlanReportAndGate(); - // Report resolver: mandatory, never-omitted, exact sentinel, anti-double-count algorithm. - expect(src).toContain('Unresolved-decisions status (MANDATORY'); expect(src).toContain('NO UNRESOLVED DECISIONS'); - expect(src).toContain('avoids double-counting'); - expect(src).toContain('DROP the current skill'); - // Gate resolver: the blocking final-line check with no "if applicable" escape. - expect(src).toContain('FINAL non-whitespace line is the unresolved-decisions'); - expect(src).toContain('FAILS the gate'); + expect(src).toMatch(/avoids double-counting/i); + expect(src).toMatch(/final\s+non-whitespace\s+line/i); + expect(src).toMatch(/missing\s+status[^.]*fails/i); // The old soft wording must be gone from the gate. expect(src).not.toContain('absorbs CODEX / CROSS-MODEL / UNRESOLVED lines if applicable'); });