test(W7c): shrink skill-ceo-section-ordering to structural and meaning checks

0 Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-11-13 03:48:06 +01:00
W7c conversion of test/skill-ceo-section-ordering.test.ts: t06 #1-#6, t05 F5
(cross-reference: the remaining bold-STOP pins), t13 F3 / W7.md CRITICAL RULE
anchor. #1 (ASK_QUESTIONS_HEADING anchor) was already on the base branch; #5
now slices the blocked chat return to the next "\n## " heading, so removing or
renaming `## Review Log` cannot widen or break the window.
Pins (grep -cE 'toContain|toMatch|indexOf|includes'): 647 -> 388 lines;
toContain( 535 -> 136; expect() calls per run 1657 -> 1399. The carve block,
section-capture fixtures and validateCeoReviewCompletion tests are unchanged
(t06 keep list) except two exact-duplicate 0D pins.
CEO invariants count as periodic-only (plan-ceo-review-benefits retires), so
every invariant below keeps a free structural or case-insensitive check.
Gate-tier judge that also grades this prose: 'plan-ceo-review/SKILL.md modes'.

New helpers: ordered() (exact labels/headings/placeholders/table cells as
strings, meaning anchors as /i RegExp) and between() (fails loudly on a missing
end anchor instead of silently running to EOF).

Removed assertion groups (invariant -> remaining protection):
- t06 #2 / t05 F5: '**STOP for the actual answer, even for a lone option.**'
  (x6 exact pins + 3 split/indexOf anchors) and '**STOP for the answer**'
  (x3): wait for the actual answer before recording -> kept free: one
  /stop for the actual answer/i per slice (template 0D in dispatch order
  between the ask-one-row check and the answer save; every generated host
  carrier between **4. Ask and **Post-answer checkpoint:**), /stop for the
  answer/i in 0E step 3 (template and generated), plus the Pre-question <
  4. Ask < Post-answer checkpoint order and the 2-checkpoint count. Dispatch
  slices now end at **Post-answer checkpoint:**.
- t06 #4 'Recommendation is ALWAYS present': recommendation label contract ->
  kept free: '(recommended)' and 'AUTO_DECIDE depends on it' in generated
  SKILL.md; exactly-one-(recommended) rule in 0D's currentDecision table.
- t06 #6 and other bans of retired phrasings ('If the section has findings,
  you MUST call AskUserQuestion', 'Otherwise, use AskUserQuestion for each
  finding', 'After each section, pause and wait for feedback', 'Save pending
  rows before comparisons', 'using the Step 0E mode-handoff format', 'Save a
  chat-only plan', 'quality bonus, not a gate', 'If no file is in scope, skip
  this section', 'always write', 'For every question', 'for architecture
  choices', 'These approaches differ in coverage (...)', 'Finish with
  ExitPlanMode', 'Before summaries, review logs or next-step menus...', "put the
  complete comparison in the question's brief", 'Use this exact route:',
  'citing its ID', /\blog (?:with|that ID)\b/): same invariants -> kept free:
  the positive checks (one **Decision gate.** per section wired to Analyze →
  Resolve → Apply; pending-rows-before-comparisons order; **Otherwise:** in
  the 0H order; report not persisted + Read-back gate; write-when-permitted
  incl. zero tasks; Outcome routing count 1; record-provenance-after-handoff
  order; literal dispatch-object comparison). 'Do NOT proceed ... 0C-bis'
  becomes not.toContain('0C-bis') on 0D. Short stale-token bans stay
  (<approved 0D approach>, reference | commitment | current value, ___
  methods, retry once, telemetry has logged, NUMBER + LETTER / "3A", the
  Forbidden-storage regex, removed menu option, unbound placeholders).
- t06 #3, magic counts: literal 11 sections / toHaveLength(7) closing steps ->
  kept free: section numbers 1..N derived from '## Review Sections (N
  sections' and checked against the ### Section headings; closing-step count
  derived from the 7 route labels.
- Decision cycle / storage / comparison sentences (Start at step 1, run steps
  2–4 only when needed, mode-change pointer, storage-policy save sentence,
  Copy the grid, Validate every field, Show unchanged/shared/pending repeated
  x4, other rows stay fixed or pending x5, never prewrite x3, Build one
  currentDecision x3, Ask one row per call x6, Save the answer reference x5,
  Keep independent changes separate x4): full decision protocol in order ->
  kept free: label order 1. Check < 2. Record < 3. Compare < grid <
  Pre-question < Read-back < 4. Ask < Post-answer (template + every host),
  one /i check each for pending-rows-before-comparisons, save-before-ask,
  amend only authorized work, failed save stops, decline-all stop, show
  unchanged/shared/pending, independent changes separate.
  Admin/plan routes also kept in ceo-workflow-clarity.test.ts.
- Handoff/provenance sentences (After selection, send brief chat, Keep rows
  separate, logging-row repeats, Section 1 current-scope sentences): mode
  handoff applies before questions and logs provenance after -> kept free:
  /before tools or further questions/i, handoff < Other selections <
  provenance < selecting-a-mode order, the two exact output-format lines with
  <rows or none> / <Application and rationale>, auto_decided true/false
  fields, 'No new approach decision was needed' (quoted chat output). Section
  1 dispositions -> ceo-workflow-clarity's Section 1 test.
- Scope/mode-selection sentences (REDUCTION, HOLD and SELECTIVE's HOLD
  checks; For both expansion modes...; Skip/Defer destination repeats;
  present both inputs for final approval; 0E counts changed files; Count all
  deliverables; >15/greenfield repeats; SELECTIVE balances...): scope
  decisions per item with exact destinations -> kept free: prescribed menu
  label orders (A Add/B Defer/C Skip; A Defer/B Keep; TODO A/B/C; depth
  A/B), Defer -> TODOS.md + NOT in scope, Skip -> no TODO, /ask separately/i,
  /all four 0D steps for each unanswered addition or deferral/i, routing rows
  and >15 / 8-file / 2-class numbers (once each; ceo-workflow-clarity keeps
  the reuse-count checks), carve block's document-approval order.
- TODO menu (per plan-review-calibration t04 F11 retirement, which names this
  file as the remaining free protection): kept free in the TODOS.md-updates
  slice: /only unanswered TODO proposals reach this menu/i, /do not ask again
  about an item already deferred, skipped or kept/i, /one per question/i.
- Resolver prose (spec review, plan report, tasks emit, exit gate, codex plan
  review, brain cache refresh, approval readiness, design clause): each
  sentence pin -> one short /i check per invariant on every host (five
  dimensions, at most three launches, required save stops, destination-file
  Read/Write order, zero tasks, unknown files JSONL format, ExitPlanMode only
  after checks, forbidden history, outcome routing and skip routes, native
  fallback never outside coverage); exact strings kept only for machine/output
  tokens (NO UNRESOLVED DECISIONS, '- + N unresolved from prior reviews',
  STATUS/SOURCE/OUTSIDE_STATUS assignments, CROSS-MODEL, OUTSIDE COVERAGE,
  'N findings; R resolved; U unresolved', quoted report phrases, Gate outcome:
  Blocked, Read-back gate). Dropped: 'If both counts are zero', 'For
  **Outside Review**, use ...', 'Enter after either an external reviewer...',
  'identify the causal mechanism', 'A credible material risk...', "check the
  assembled set's capacity", 'including findings that needed only factual
  correction', 'show unsaved fields and errors', Strategy-only tasks sentence,
  verified-report-can-pass sentence -> neighbouring kept /i checks in the
  same slice plus the gate-tier 'plan-ceo-review/SKILL.md modes' judge.
- Review-section prose (continuity requirement lists, test-review and outside
  rule duplicates, strategy-only diagram sentences, Review facts repeats,
  Stage 3 publishes it, Count a reopened choice once, Keep them before the
  terminal report repeats, EXIT gate verifies..., no report or log needed
  yet, then run telemetry as the last review operation): per-section gate and
  ledger continuity -> kept free: Analyze < Resolve < Apply, Resolve paths
  via check-scope < correct < record order, /resolve critical risks now/i,
  /review only; do not change code/i, ledger header + status tokens, one /i
  per approval/stop line; reopened-once lives in ceo-workflow-clarity;
  terminal order kept as cache refresh < {{BRAIN_CACHE_REFRESH}} < Telemetry
  < finished < Call ExitPlanMode < handoff.
- Duplicate pins removed outright (same string asserted 2-6 times across
  tests, incl. carve-block 'Ask one row per call' and bold-STOP pins): each
  keeps its single remaining check named above.
Cosmetic pins dropped without controls: bold/caps (**STOP**, ALWAYS, MUST),
double compactProse wrappers.

Mutation pass (scratch worktree; templates, generated SKILL.md and
review-dashboard resolver): 39/39 as expected; 28 kept-step/marker/
placeholder/safety/consent/numeric mutations go red (incl. the three TODO-menu
lines), 11 rewordings of dropped sentences, the CRITICAL RULE heading rename,
re-adding the retired #6 sentence, and lowered-register STOP lines stay green.
Log: .context/mutation-skill-ceo-section-ordering.md.
Runtime (Bun 1.4.0, 3 runs): before 2.05/1.84/1.77 s (median 1.84),
after 1.87/1.80/2.08 s (median 1.87); dominated by runGeneration and the
capture subprocess fixtures.
This commit is contained in:
garrytan committed 2026-10-03 01:18:06 +00:00
1 parent d0fc479073
commit 6a83ba86c3
1 file changed
+407 -705
test/skill-ceo-section-ordering.test.ts
+407 -705
View File
File diff suppressed because it is too large. Load diff