diff --git a/docs/OVERLAY_BENCHMARK_CONTRACT.md b/docs/OVERLAY_BENCHMARK_CONTRACT.md index 1c15e1fe5..d6f3cc38e 100644 --- a/docs/OVERLAY_BENCHMARK_CONTRACT.md +++ b/docs/OVERLAY_BENCHMARK_CONTRACT.md @@ -1,4 +1,4 @@ -# Overlay benchmark behavior contract v2 +# Overlay benchmark behavior contract v4 The paid overlay fixtures verify supported behavior, with comparative efficacy reported separately as research evidence. A passing case does **not** establish @@ -7,7 +7,7 @@ blocking efficacy contract; historical failures retain their original verdicts. The contract is defined by `OVERLAY_CONTRACT` in `test/helpers/overlay-case-policy.ts`. Every new measurement and aggregate record -identifies version 3. Eval entry names include `contract-v3` so automated result +identifies version 4. Eval entry names include `contract-v4` so automated result matching cannot silently compare the old and new contracts by the same name. Selection IDs for the six retained paid cases stay unchanged. @@ -20,9 +20,17 @@ OFF does not. Each retained fixture runs ten samples per arm, separately for Every planned sample in both arms must execute successfully, produce exactly one successful native terminal result without an error flag, preserve workspace scope, and yield a finite metric within the fixture's bounds. Missing reasoning -metadata, failed output validation, oracle process errors, partial sampling, -deadlines, cleanup failures, or recording failures keep the case failed. Missing -observations are never replaced with zero or discarded to obtain a passing arm. +metadata, oracle process errors, partial sampling, deadlines, cleanup failures, +or recording failures keep the case failed. Missing observations are never +replaced with zero or discarded to obtain a passing arm. + +A completed answer that fails the fixture's output check is a measured +correctness result, not an invalid measurement, in either arm: its record has +`measurementPassed: true`, `taskCorrect: false` and the check's message in +`answerError`. Only ON correctness gates the case, so an OFF wrong answer stays +in the comparison and an ON wrong answer fails the case. Version 3 counted an +OFF wrong answer as an invalid measurement (census 37198445662); v3 verdicts +are not reinterpreted. | Fixture family | Exact task and scope contract | Additional ON requirement | Measurement | | --- | --- | --- | --- | diff --git a/docs/evals/detector-inventory.md b/docs/evals/detector-inventory.md new file mode 100644 index 000000000..4b962f410 --- /dev/null +++ b/docs/evals/detector-inventory.md @@ -0,0 +1,320 @@ +# E2E detector inventory: phrase checks and structured sources + +This inventory lists every pass/fail decision in the gate and periodic E2E cases (`E2E_TIERS` in test/helpers/touchfiles-data.ts, plus the three standalone periodic AUQ files) that rests only on phrases or regexes in model free text, at HEAD `cbcccf9`. It has 153 phrase-check rows: 13 are already structured-first (a), 38 are convert candidates (b), 79 need a phrase witness because the contract is user-visible (c), and 23 have no structured source (d); 13 more rows are LLM judges. Shared detectors (S1-S10, F1-F3, A1) are counted once, even though up to nine cases use each. Of the five B1 cases, only auto-decide-preserved has a structured record that can decide its outcome; the other four already read their records first or test text the user sees, so B1 replay corpora protect their phrase checks. + +Dispositions: + +- **(a) already structured-first.** The check reads a structured record first and falls back to phrases only when the record is absent. +- **(b) convert candidate.** A structured record of the same fact exists in this run, and the contract is not user-visible text. +- **(c) phrase witness required.** The contract is what the user sees (a question shown, a report or message rendered). A record may locate the text but cannot replace it. Numeric format regexes on a shown question, such as `Completeness: N/10`, are (c). +- **(d) no structured source.** An internal fact that nothing in the run records. +- **judge.** An LLM judge. It is listed but is not a phrase check. + +"Journal" means the hermetic Claude session JSONL that test/helpers/plan-count-transcript.ts reads. "Surface" means `fullSurface`/`fullOutputSurface`, which includes Skill-injected SKILL.md bodies. + +## Shared detectors + +| ID | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| S1 | test/helpers/pty/screen.ts:399 via test/helpers/pty/classify.ts:490 | screen `❯ 1.` option list → `asked` | AskUserQuestion tool_use in the journal locates it | yes (a question shown) | (c) | +| S2 | test/helpers/pty/classify.ts:45 via :505 | lettered/numbered prose options, `Recommendation:`, `(recommended)` → `asked` | none when AUQ is disallowed | yes | (c) | +| S3 | test/helpers/pty/screen.ts:186 via test/helpers/pty/classify.ts:467 | "ready to execute" / exit-plan panel → `plan_ready` | ExitPlanMode tool_use in the journal | no | (b) | +| S4 | test/helpers/pty/classify.ts:425 | screen `⏺ Write(path)` outside sanctioned dirs → `silent_write`; plan write before first AUQ → `wrote_findings_before_asking` | Write/Edit tool_use (file_path, order) in the journal | no | (b) | +| S5 | test/helpers/pty/screen.ts:205 via test/helpers/pty/classify.ts:460 | `(auto-decided from plan-tune preference)` → `auto_decided`; runs before the structured path at test/helpers/pty/runners/observation.ts:389 | `gstack-ceo-mode-handoff --auto` (plan-ceo-review/SKILL.md.tmpl:410), question-log `auto_decided:true` (:416; scripts/resolvers/question-tuning.ts:38), read by test/helpers/native-auto-decide.ts:412 | yes (the chat must begin with the handoff line, plan-ceo-review/SKILL.md.tmpl:410); the record can locate it, the transcript line stays the witness | (b) | +| S6 | test/helpers/pty/runners/observation.ts:410-416 | Haiku `waiting` verdict → `asked` | — | — | judge | +| S7 | test/helpers/pty/plan-native.ts:36 via :605 | plan file `^## GSTACK REVIEW REPORT$` with no later `## ` heading | none (rendered plan file) | yes | (c) | +| S8 | test/helpers/plan-mode-evidence.ts:72 → test/helpers/pty/screen.ts:279 | plan file has `## Decisions` on `plan_ready` → fail | none | yes | (c) | +| S9 | test/helpers/pty/classify.ts:148 | squished screen "what should I (design-)review" + "current branch diff" | AUQ tool_use locates it when native AUQ is allowed | yes | (c) | +| S10 | test/helpers/pty/classify.ts:166, test/helpers/plan-scope-selection.ts:54-110, test/helpers/pty/runners/observation.ts:483 | scope-gate auto-select announcement, first-line target wording, seed token on screen | none (no scope-selection record; the seed token is in the pasted seed) | no | (d) | +| F1 | test/helpers/pty/runners/floor.ts:474 → test/helpers/plan-floor-review.ts:217 | staged question judged a `finding` (only pass) | — | — | judge | +| F2 | test/helpers/pty/runners/floor.ts:451 | prose candidate only if viewport and last assistant message both show prose options; native call read first | native AUQ call (structured first) | yes | (a) | +| F3 | test/helpers/pty/runners/floor.ts:497 | screen Write render / plan-ready text → fail | Write/ExitPlanMode tool_use in run.transcript | no | (b) | +| A1 | test/helpers/auq-sdk-capture.ts:30-43 `scoreAuqFormat` | question has ELI10:, Recommendation:, Pros / cons, ✅, ❌, Net:, (recommended) | AUQ tool input (its prose is the contract) | yes | (c) | + +Cases that use S1-S8: plan-ceo-review-plan-mode, plan-devex-review-plan-mode, plan-mode-no-op, office-hours-auto-mode (gate); auto-decide-preserved, plan-eng-review-plan-mode, plan-design-review-plan-mode-smoke (periodic). S9-S10: plan-mode-no-op, plan-eng-review-plan-mode, plan-design-review-plan-mode-smoke. F1-F3: plan-ceo-finding-floor, plan-devex-finding-floor (gate); plan-design-finding-floor, plan-eng-finding-floor (periodic). A1: auq-format-gate (gate); the three standalone AUQ files (periodic). + +## Gate tier + +### test/skill-e2e-plan-mode-no-op.test.ts, test/skill-e2e-plan-ceo-plan-mode.test.ts, test/skill-e2e-plan-devex-plan-mode.test.ts, test/skill-e2e-office-hours-auto-mode.test.ts + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| plan-mode-no-op | test/skill-e2e-plan-mode-no-op.test.ts:114, :175 | screen lacks the plan-mode reminder sentence | none | yes (leaked text) | (c) | +| plan-mode-no-op | test/skill-e2e-plan-mode-no-op.test.ts:186 | screen contains seed token `ZephyrLedgerWidget` | none; the token is in the pasted seed | no | (d) | +| office-hours-auto-mode | test/skill-e2e-office-hours-auto-mode.test.ts:41 | S5 `auto_decided` fails the case | question-log row | no | (b) | + +The plan-ceo-review-plan-mode (:89, :108), plan-devex-review-plan-mode (:181-222) and office-hours-auto-mode (:54) verdicts are otherwise S1-S8. + +### test/skill-e2e-plan-devex-finding-floor.test.ts, test/skill-e2e-plan-design-with-ui.test.ts + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| plan-devex-finding-floor | test/helpers/plan-floor-review.ts:278-280 `pickPlanFloorProductType` | product-type brief `^Is this…?$` and one exact "Project/branch/task" context; no match leaves the menu unanswered (stall) | AUQ header `Product type` + option labels (already parsed :268-281) | no (selector) | (b) | +| plan-design-with-ui-scope | test/skill-e2e-plan-design-with-ui.test.ts:34-36 | design-focus question prose "I've rated this plan N/10…" or "all 7 … focus … design?" | AUQ header/options of the answered native call | no (boundary id) | (b) | +| plan-design-with-ui-scope | test/skill-e2e-plan-design-with-ui.test.ts:44-57 | review-finding identity from option labels or UI vocabulary | option labels (structured), no design-finding record | no | (b) | +| plan-design-with-ui-scope | test/helpers/plan-review-board-feedback.ts:111-120, :127-138 | outside-voices question lead regex; one `/boards/` URL in prose; mismatch throws | header `Voices` + exact option labels (compared :115-116) | no (actor) | (b) | +| plan-design-with-ui-scope | test/skill-e2e-plan-design-with-ui.test.ts:80-81 | screen prompt `/boards/` without a native call throws | native-call absence decides | no | (a) | +| plan-design-with-ui-scope | test/helpers/pty/classify.ts:555 `COMPLETION_SUMMARY_RE` via test/helpers/pty/runners/counting.ts:554 | screen report/summary/`VERDICT:` text ends collection before the ceiling | review-log row / ExitPlanMode tool_use | no | (b) | + +### test/skill-e2e-ask-user-question-format-compliance.test.ts, test/skill-e2e-plan-tune.test.ts, test/skill-e2e-plan.test.ts + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| auq-format-gate | test/skill-e2e-ask-user-question-format-compliance.test.ts:58 | question ~ `options differ in kind` | AUQ tool input | yes | (c) | +| auq-format-gate | test/skill-e2e-ask-user-question-format-compliance.test.ts:61, :68 | recommendation substance ≥ 4 | — | — | judge | +| plan-tune-inspect | test/skill-e2e-plan-tune.test.ts:167-169, :185 | output names ≥2 of 3 seeded question IDs or aliases | question-log is seeded fixture input | yes (inspection report) | (c) | +| plan-review-report | test/skill-e2e-plan.test.ts:463-491 | plan.md `## GSTACK REVIEW REPORT`, CEO/Eng/Design rows, ends with `NO UNRESOLVED DECISIONS` or a bullet | rows come from the seeded review log; the table is rendered | yes | (c) | + +### test/skill-e2e-design.test.ts (gate cases) + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| plan-design-review-no-ui-scope | test/skill-e2e-design.test.ts:561-565, :572 | output contains no ui/no frontend/no design/not applicable/backend | none (early exit writes no record) | no | (d) | +| design-review-plugin-handoff | test/skill-e2e-design.test.ts:1012, :1020-1023 | detector-output.md has `IMPECCABLE_READY`, FINDING-001, rule ids, `deferred`, per-handoff `/impeccable x` | probe/scan tool output (checked :994, :1010-1011) | yes (report) | (c) | +| design-review-detector-shim | test/skill-e2e-design.test.ts:1158 | tool outputs or report include `IMPECCABLE_READY` | probe tool output read first | yes | (a) | +| design-review-detector-shim | test/skill-e2e-design.test.ts:1159-1161 | report contains FINDING-001, [ai-color-palette], [low-contrast] | calls.jsonl receipts + engine log (:1132-1154) | yes | (c) | +| design-review-detector-shim-dom | test/skill-e2e-design.test.ts:1254-1255 | report contains [ai-color-palette], "static scan of the rendered DOM" | DOM dump + Bash trace (:1247-1252) | yes | (c) | + +### test/skill-e2e-coverage-audit.test.ts, test/skill-e2e-workflow.test.ts + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| review-coverage-audit, plan-eng-coverage-audit | test/helpers/coverage-audit-evidence.ts:546-547 | last JSON `{"tested","untested"}` line first; ASCII-diagram grammar fallback (:459-510) | prompt-requested JSON summary | no | (a) | +| ship-coverage-audit | test/helpers/coverage-audit.ts:34-38 | diagram has gap/no test, tested/✓/★, coverage, both function names | none (the diagram is the requested output) | yes | (c) | +| document-release | test/skill-e2e-workflow.test.ts:101-102, :109 | CHANGELOG keeps old entries; current entry and README mention "feature c" | none (docs are the product) | yes | (c) | + +### test/skill-e2e-shared-libs.test.ts, test/skill-e2e-shared-libs-paths.test.ts + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| shared-libs-read-only | test/skill-e2e-shared-libs.test.ts:113-120 | report ~ uncommitted/overlay/raw, tip and branch SHAs, retry-worker/route, no blob link for branch-only file | none (read-only by contract) | yes | (c) | +| shared-libs-unsupported-git | test/skill-e2e-shared-libs.test.ts:136, :144 | report ~ unavailable/unsupported/limited; not "no commits/PRs found" | none | yes | (c) | +| shared-libs-review-path-eligibility | test/skill-e2e-shared-libs-paths.test.ts:133-134 | `trace + output` ~ submodule/160000, check-ignore/ignored (output alone satisfies) | Bash command trace (first operand) | no | (b) | + +### test/skill-e2e-ship-docsync.test.ts, test/skill-e2e-docsync-spawned.test.ts + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| ship-docsync-store | test/skill-e2e-ship-docsync.test.ts:74, :78 | ship report ~ `Documentation…blocked`; not `Documentation: current` | no receipt, no publish call (:75-77) | yes (report status) | (c) | +| ship-docsync-completion, ship-docsync-current | test/skill-e2e-ship-docsync.test.ts:81, :97 | parsed docs contract, then report contains `contract.documentation_section` | docs dispatch JSON (ship/sections/documentation.md.tmpl:52-56) | yes | (a) | +| ship-docsync-late-result, -missing-marker, -missing-asset, -launch-failure, -timeout-unsettled, -failure | test/helpers/docsync-fault-eval.ts:28, :30 | non-success report ~ `Documentation…blocked`, no `Documentation: current` outside inline code | actor event log decides dispatch count and publication first (:22-27) | yes | (c) | +| ship-docsync-recovery, -stale-before, -stale-after | test/helpers/docsync-fault-eval.ts:31 | report contains the accepted audit id | actor state `acceptedId` | yes | (a) | +| docsync-spawned | test/skill-e2e-docsync-spawned.test.ts:60 | `contract.blockers` ~ security/sensitive | parsed docs contract | no | (a) | + +### test/helpers/ship-hook-actor.ts (ship-hook-consent, ship-hook-refresh files) + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| ship-managed-hook-refresh | test/helpers/ship-hook-actor.ts:122 | final ~ refresh/updat/install/current | installer receipt (checked :113-121) | yes (outcome told to user) | (c) | +| ship-unmanaged-hook-consent | test/helpers/ship-hook-actor.ts:129 | final ~ declin/unchanged/not install/preserv/leave | declined interaction record (:127) | yes | (c) | +| ship-local-hook-preservation | test/helpers/ship-hook-actor.ts:133 | final ~ /manual/ | none | yes | (c) | + +### test/skill-e2e-review.test.ts, test/skill-e2e-review-attribution.test.ts, test/skill-e2e-review-army.test.ts (gate cases) + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| review-sql-injection | test/skill-e2e-review.test.ts:87-95 | review-output.md includes sql/injection/sanitiz/parameteriz/… | none: the fixture copies only the checklist, so no review-log row (review/SKILL.md.tmpl:402) | partly | (d) | +| review-enum-completeness | test/skill-e2e-review.test.ts:173-177 | review mentions "returned" and enum/status/critical | none (focused checklist run) | partly | (d) | +| ship-base-branch | test/skill-e2e-review-attribution.test.ts:144 | ship-preflight.md ~ /main\|base/ | git commands in the Bash trace (review-base-branch uses this, :87) | no | (b) | +| review-dashboard-via | test/skill-e2e-review-attribution.test.ts:273, :275 | output + dashboard ~ autoplan, clear | review log is seeded input; dashboard is rendered | yes | (c) | +| review-army-migration-safety | test/skill-e2e-review-army.test.ts:136-142 | output includes drop/data loss/reversib/migration/column | none (single-shot) | partly | (d) | +| review-army-perf-n-plus-one | test/skill-e2e-review-army.test.ts:272-280 | output includes n+1/eager/includes/preload/query/loop | specialist JSON findings in Agent results (review/sections/review-army.md); not verified per finding | no | (b) | +| review-army-delivery-audit | test/skill-e2e-review-army.test.ts:395-404 | output includes not done/missing and email/notification | none verified | partly | (d) | +| review-army-quality-score | test/skill-e2e-review-army.test.ts:494-496 | `SPECIALIST REVIEW: 1 findings (1 critical, 0 informational)`, `PR Quality Score: 8/10`, `[ADVISORY]` | merged-review.json (checked :485-493) | yes (rendered format) | (c) | + +### test/skill-e2e-bws.test.ts, test/skill-e2e-session-intelligence.test.ts, test/skill-e2e-learnings.test.ts + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| skillmd-no-local-binary | test/skill-e2e-bws.test.ts:156 | result.output ~ READY/NEEDS_SETUP | setup-block Bash tool_result prints `READY: $B`/`NEEDS_SETUP` (browse/SKILL.md:214) | no | (b) | +| skillmd-outside-git | test/skill-e2e-bws.test.ts:194 | same | same | no | (b) | +| context-recovery-artifacts | test/skill-e2e-session-intelligence.test.ts:181-183, :195 | output contains any of recent artifacts/ceo-plans/last_session/ship/timeline/completed (near-vacuous) | bin/gstack-context-recovery:31, :37 prints the record, but the fixture does not copy it (:46-48) | no | (d) (b once the bin is copied) | +| context-save-writes-file | test/skill-e2e-session-intelligence.test.ts:257, :266 | checkpoint includes `---` and `status:` | the file's YAML frontmatter (parseable) | no | (b) | +| context-restore-loads-latest | test/skill-e2e-session-intelligence.test.ts:379-393, :401-402 | `RESTORED: ` or newer-body phrase; Read of the newer file as fallback | Read tool_use | partly | (c) | +| learnings-show | test/skill-e2e-learnings.test.ts:117-119, :131 | output mentions ≥2 of n-plus-one, stale/cache, rubocop | `gstack-learnings-search` Bash result (fixture copies it, test/skill-e2e-learnings.test.ts:43) | yes (/learn shows them) | (c) | + +### test/skill-e2e-qa-functional*.test.ts, test/skill-e2e-skillify.test.ts, test/skill-e2e-deploy.test.ts, test/skill-e2e-cso.test.ts + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| qa-functional-cli-report, -webhook-report, -cli-fix, -webhook-fix | test/helpers/qa-checkpoint-evidence.ts:207-209 | report.md links each `exploration-NNN.json` checkpoint | checkpoint files + gstack-qa-evidence receipts; the link is rendered | yes | (c) | +| same 4 | test/helpers/qa-functional-evidence.ts:179-180 | private sentinel absent from output, report and evidence | n/a (leak canary) | yes | (c) | +| skillify-happy-path | test/skill-e2e-skillify.test.ts:379 → test/helpers/skill-body-narration.ts:7 | committed SKILL.md has no `I `/`I'll`/`Let me` narration | none (generated skill text is the artifact) | yes | (c) | +| skillify-provenance-refusal | test/skill-e2e-skillify.test.ts:429, :467 | surface ~ unknown skill/not registered (load tripwire) | Skill tool_use + its error tool_result | no | (b) | +| skillify-provenance-refusal | test/skill-e2e-skillify.test.ts:444, :468 | assistant ~ "no recent scrape result" / "run /scrape first" | none | yes (refusal, skillify/SKILL.md.tmpl:67) | (c) | +| land-and-deploy-workflow | test/skill-e2e-deploy.test.ts:81 | deploy reports + CLAUDE.md ~ /fly/ | Bash result `PLATFORM:fly` (land-and-deploy/sections/first-run-validation.md:35); not verified this flow runs it | no | (b) | +| land-and-deploy-workflow | test/skill-e2e-deploy.test.ts:82 | artifacts contain `test-app` | none | no | (d) | +| land-and-deploy-first-run | test/skill-e2e-deploy.test.ts:150-151 | report contains fly or first-run-app | Bash result `PLATFORM:fly` (this case runs Step 1.5) | no | (b) | +| land-and-deploy-review-gate | test/skill-e2e-deploy.test.ts:221-222 | readiness report ~ review and not run/no reviews/missing | none (gstack-review-read not installed) | yes | (c) | +| setup-deploy-workflow | test/skill-e2e-deploy.test.ts:408-410 | CLAUDE.md contains fly, my-cool-app, Deploy Configuration | the `## Deploy Configuration` key/value block (setup-deploy/SKILL.md.tmpl:183-190) | no (machine-read) | (b) | +| cso-diff-mode | test/skill-e2e-cso.test.ts:181 | webhook.js finding prose ~ signature/authenticat/forg | report.json has no class/CWE field (lib/cso/contracts.ts:65-80) | no | (d) | +| cso-diff-mode (and the periodic CSO cases) | test/skill-e2e-cso.test.ts:61, :86, :133-134 | CANARY token absent from output, transcript and reports | n/a (leak canary) | yes | (c) | + +### test/skill-e2e-third-party-actions.test.ts + +The consent question is plain text by design (Read and Bash only). The probe's Bash result (`READY:`/`NEEDS_ASIDE: `/`ASIDE_NOT_RUNNING`, ship/SKILL.md:395-409) records detection, not what was offered. + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| tpa-present | test/skill-e2e-third-party-actions.test.ts:197-203 | Aside, `A)`, defer, dashboard host; no download pitch (regex :41) | probe `READY:` | yes | (c) | +| tpa-absent-linux | test/skill-e2e-third-party-actions.test.ts:218-226; test/helpers/third-party-actions.ts:6, :23 | no pitch, no Aside-drive option, manual + headed-drive options | probe `NEEDS_ASIDE: Linux` | yes | (c) | +| tpa-broken | test/skill-e2e-third-party-actions.test.ts:244, :249-252 | no drive option; "open the Aside app" or a lettered question | probe `ASIDE_NOT_RUNNING` | yes | (c) | +| tpa-absent-darwin | test/skill-e2e-third-party-actions.test.ts:272-274 | pitch sentence, `macOS 15`, no drive options | probe `NEEDS_ASIDE: Darwin` | yes | (c) | +| tpa-apple-ban | test/skill-e2e-third-party-actions.test.ts:307-311 | no drive option for account.apple.com; app-specific password guidance | none | yes | (c) | + +### test/helpers/workflow-boundaries-fixture.ts (investigate-owned files), test/skill-e2e-safety-pair-agent.test.ts, test/skill-e2e-safety-ship-evidence.test.ts + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| investigate-owned-completion, -abort, -ending-error | test/helpers/workflow-boundaries-fixture.ts:156-160 | scripted actor: option label ~ abort/stop/cancel or continue/proceed, else unsupported | AUQ input (labels are prose) | yes | (c) | +| investigate-owned-abort | test/helpers/workflow-boundaries-fixture.ts:194 | final ~ abort/stop/cancel | receipts (:186-193) | yes | (c) | +| investigate-owned-ending-error | test/helpers/workflow-boundaries-fixture.ts:209 | final ~ error/unavailable/cannot/unable | receipt `VERIFY_STATUS:69` (:208) | yes | (c) | +| safety-pair-agent-block | test/skill-e2e-safety-pair-agent.test.ts:86, :91 | final contains the instruction block byte-for-byte | block is in the fake tool result; relaying it is the contract | yes | (c) | +| safety-ship-stale-evidence | test/skill-e2e-safety-ship-evidence.test.ts:133 `callJudge` | — | — | — | judge | + +Gate cases with no phrase-only checks: the shared-libs review-lifecycle, revalidation, index-flags and prior-coverage cases (review-log rows and traces); ship-skipped-queued-finding; the five qa-callers cases; ship-section-loading; ship-triage (test/helpers/ship-triage-labels.ts:12 parses JSON first, so (a)); ship-coverage-value, review-test-value, test-audit-report-only (JSON records; regexes only select items, so (a)); review-army-json-findings; ship-local-workflow; gstack-upgrade-happy-path; qa-quick; qa-only-no-fix; qa-bootstrap; browse-basic; browse-snapshot; skillmd-setup-discovery; operational-learning; skillify-approval-reject; canary-workflow; benchmark-workflow; retro-base-branch; timeline-event-flow; hermetic-canary; hermetic-sentinel; diagram-triplet; safety-codex-consult-embed. + +## Periodic tier + +### B1 cases + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| auto-decide-preserved | test/skill-e2e-auto-decide-preserved.test.ts:173 (S5) | screen auto-decided annotation passes | handoff `--auto` + question-log, read only when the screen is unclassified | yes (handoff line) | (b) | +| auto-decide-preserved | test/skill-e2e-auto-decide-preserved.test.ts:159 (S1/S2) | a shown option list fails | prose question only (AUQ disallowed) | yes | (c) | +| plan-ceo-section-loading | test/helpers/ceo-stale-fill-decision.ts:54 ‖ test/helpers/ceo-section-loading-fixture.ts:688 (called test/skill-e2e-plan-ceo-review-section-loading.test.ts:118) | approved ledger row + `currentDecision` naming fill/write/race; else structured-then-prose finding | decision ledger and `currentDecision` blocks in PLAN.md | no | (a) | +| plan-ceo-section-loading | test/helpers/auq-sdk-capture.ts:399-420 (called :101) | `COMPLETION SUMMARY` with Section 1-11 outcome rows | none per section | yes | (c) | +| plan-ceo-section-loading | test/skill-e2e-plan-ceo-review-section-loading.test.ts:74, :110-111 | `## GSTACK REVIEW REPORT`, review table header, `\| CEO Review \|` row | rendered dashboard | yes | (c) | +| plan-ceo-section-loading | test/helpers/auq-sdk-capture.ts:213 (called :113) | report row `Outside Review` status starts `disabled` | none in this native-only run | yes | (c) | +| shared-libs-plan-callers | test/helpers/shared-libs-plan-actor.ts:28-112 (phrase refusals :63, :67, :72, :75, :85, :109) | scripted actor parses question prose: reuse verbs, `lib/retry-after.ts`, contract wording, 9 scope-expansion regexes; a match fails the case | AUQ input; option text is the only carrier of what is offered | yes (question content) | (c) | +| shared-libs-plan-callers | test/skill-e2e-shared-libs-periodic.test.ts:198 | — | — | — | judge | +| plan-design-review-plan-mode | test/skill-e2e-design.test.ts:489, :497, :508 | output ~ `\d+/10` | review-log `initial_score`/`overall_score` (plan-design-review/sections/review-sections.md.tmpl:229); not verified in fixture | yes | (c) | +| plan-design-review-plan-mode | test/skill-e2e-design.test.ts:496-497, :508 | output or rated plan growth has information architecture/interaction state/ai slop/hierarchy | none (review-log has scores and counts only) | no | (d) | +| plan-design-review-plan-mode | test/skill-e2e-design.test.ts:499-504, :511 | plan.md contains empty/loading/error/state/responsive/accessibility | none | no | (d) | +| ship-docsync-late-result | gate tier; listed in the ship-docsync table | — | — | — | see gate | + +### Plan reviews: test/skill-e2e-plan-ceo-mode-routing.test.ts, test/skill-e2e-plan-eng-*.test.ts, test/skill-e2e-plan-design-*.test.ts, carve-section-loading + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| plan-ceo-mode-routing | test/helpers/ceo-mode-option.ts:661-670 (called test/skill-e2e-plan-ceo-mode-routing.test.ts:248) | posture regex on the handoff tool-result line first, then post-answer prose | `gstack-ceo-mode-handoff` tool result (plan-ceo-review/SKILL.md.tmpl:410) | no | (a) | +| plan-ceo-mode-routing | test/skill-e2e-plan-ceo-mode-routing.test.ts:257 | HOLD posture review | — | — | judge | +| plan-ceo-mode-routing | test/helpers/pty/classify.ts:341 `MODE_RE` | option label starts with a mode name → mode question found | native AUQ options | yes | (c) | +| plan-design-finding-floor | test/helpers/pty/boundaries.ts:653-658 `pickDesignFocusAll` | Step-0D title, "Project/branch/task" and ELI10 rating prose; no match stalls | AUQ header + option labels (parsed :659-661) | no (selector) | (b) | +| plan-eng-multi-finding-batching | test/helpers/pty/boundaries.ts:62-116, :587-600, :626-640 | setup vs review question decided by prose ("This plan introduces N new classes", "Issue", scope-reduction) | decision ledger R records (plan-eng-review/sections/review-sections.md.tmpl:213-235); qid branches inert under `QUESTION_TUNING=false` | no | (b) | +| plan-eng-multi-finding-batching | test/helpers/eng-seeded-coverage.ts:33-104 | distinct-issue identity per answered AUQ ("Issue N:", D/R-ID, ELI10 gates) | ledger R records, used only as fallback behind the prose gates | no | (b) | +| plan-eng-multi-finding-batching | test/helpers/pty/classify.ts:555 via counting.ts:554 | screen completion text ends counting | review-log row / ExitPlanMode tool_use | no | (b) | +| plan-eng-review-artifact | test/skill-e2e-plan-eng-artifact.test.ts:78 | test-plan artifact ~ dashboard/fetchStats/`/api/stats` | artifact schema has no field naming the change (scripts/resolvers/testing.ts:501-528) | no | (d) | +| carve-section-loading [plan-eng-review] | test/helpers/carve-section-case.ts:120 → test/helpers/auq-sdk-capture.ts:381 | report or output ~ report/review/summary/design doc/handoff | review-log row in the capture state root, deleted before the verdict (auq-sdk-capture.ts:368-370) | no | (b) | +| carve-section-loading [12 other skills] | same | same loose regex | none (Bash not allowed, auq-sdk-capture.ts:358-359) | no | (d) | + +The plan-eng-review-plan-mode and plan-design-review-plan-mode-smoke verdicts are S1-S10. plan-eng-finding-floor and plan-design-finding-floor verdicts are F1-F3. + +### Question format: test/skill-e2e-plan-format.test.ts, test/skill-e2e-plan-prosons.test.ts, standalone AUQ files + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| plan-ceo-review-format-mode, plan-eng-review-format-kind | test/skill-e2e-plan-format.test.ts:174-175, :377-378 | no `Completeness: N/10`; kind note present | captured question | yes | (c) | +| plan-ceo-review-format-approach, plan-eng-review-format-coverage | test/skill-e2e-plan-format.test.ts:241, :310 | `Completeness: N/10` present | captured question | yes | (c) | +| all four format cases | test/skill-e2e-plan-format.test.ts:177, :243, :312, :380 | recommendation quality | — | — | judge | +| plan-review-prosons-format | test/skill-e2e-plan-prosons.test.ts:184-196 | `D\d+ —`, ELI10:, Stakes…:, Recommendation:, Pros / cons:, ^Net:, ✅≥4, ❌≥2, (recommended) | captured question | yes | (c) | +| plan-review-prosons-hardstop-neg | test/skill-e2e-plan-prosons.test.ts:246-249 | no hard-stop escape line; ✅/❌ counts | captured question | yes | (c) | +| plan-review-prosons-neutral-neg | test/skill-e2e-plan-prosons.test.ts:299-303 | no `taste call`; (recommended); `Recommendation:.*because` | captured question | yes | (c) | +| plan-ceo-review-prosons-cadence | test/skill-e2e-plan-prosons.test.ts:353-358 | Pros / cons header; hard-stop escape or pros+cons | captured question | yes | (c) | +| test/skill-e2e-auq-matrix.test.ts | test/helpers/auq-sdk-capture.ts:54-59 (called :166) | `Recommendation:` line; exactly one `(recommended)` label | AUQ options (the suffix the AUTO_DECIDE hook parses) | yes | (c) | +| test/skill-e2e-auq-consistency.test.ts | test/skill-e2e-auq-consistency.test.ts:65, :87 | each A1 element present in every run | AUQ text | yes | (c) | +| test/skill-e2e-auq-verbose-vs-carved-ab.test.ts | test/skill-e2e-auq-verbose-vs-carved-ab.test.ts:43, :108 | carved format count ≥ verbose | AUQ text | yes | (c) | +| AUQ matrix, consistency, A/B | auq-matrix :148; consistency :69, :95-96; A/B :48, :109 | substance panels and tolerances | — | — | judge | +| plan-ceo-review-expansion-energy | test/skill-e2e-plan.test.ts:271-275 | posture | — | — | judge | + +### Shared-libs, review army, codex review, design + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| shared-libs-opportunity-judgment | test/skill-e2e-shared-libs-periodic.test.ts:121-122 | report contains tip SHA and `lib/retry-after.ts` | none | yes | (c) | +| shared-libs-opportunity-judgment | test/skill-e2e-shared-libs-periodic.test.ts:123 | `JSON.stringify(transcript)` ~ inventory.py/search.py/negative inventory (includes assistant prose) | Read/Bash tool-call inputs | no | (b) | +| shared-libs-opportunity-judgment, shared-libs-pr-coverage | test/skill-e2e-shared-libs-periodic.test.ts:88, :125, :158 | — | — | — | judge | +| shared-libs-codex-read-only | test/codex-e2e-shared-libs.test.ts:123-129 | Codex report contains SHA, overlay wording, file names, savings, tests | none | yes | (c) | +| review-army-red-team | test/skill-e2e-review-army.test.ts:651 | output ~ red team/adversarial | Agent/Task dispatch description (perf case reads it, :256-258) | no | (b) | +| review-army-consensus | test/skill-e2e-review-army.test.ts:737-741 | output includes sql/injection/interpolat | specialist JSON findings in Agent results; not verified | no | (b) | +| review-army-simplification | test/skill-e2e-review-army.test.ts:806-815 | output includes native/stdlib/speculative/…; not "lean already. ship." | none | partly | (d) | +| review-army-simplification-precision | test/skill-e2e-review-army.test.ts:868-872 | no delete/shrink JSON finding on the self-check; `NO FINDINGS` sentinel | specialist JSON lines / sentinel | no | (a) | +| review-design-lite | test/skill-e2e-review.test.ts:285-306 | ≥4 of 7 keyword groups | none | partly | (d) | +| codex-review | test/skill-e2e-workflow.test.ts:478, :481 | executed verdict from Codex command output | lib/outside-review-result.ts | no | (a) | +| codex-review | test/skill-e2e-workflow.test.ts:482 | codex-output.md ~ `GATE: PASS\|FAIL (N critical\|UNVERIFIED`, not fail-closed | same verdict | yes (gate line) | (c) | +| design-consultation-core | test/skill-e2e-design.test.ts:160-161, :186, :190 | DESIGN.md has one synonym per 7 section sets; CLAUDE.md mentions design.md | none for content | yes | (c) | +| design-consultation-research | test/skill-e2e-design.test.ts:253, :263 | "Search unavailable" only when no WebSearch/aside call | WebSearch / aside tool calls read first (:243-245) | yes | (a) | +| design-consultation-existing | test/skill-e2e-design.test.ts:314-315, :334-335 | DESIGN.md includes color and spacing | format marker only | yes | (c) | +| design-review-fix | test/skill-e2e-design.test.ts:714, :742 | design-audit.md ~ 47/48px, line-height, radius, padding, spacing, heading | none | yes | (c) | +| design-html-slop-gate | test/skill-e2e-design.test.ts:1318-1319 | gate-output.md has ai-color-palette and accepted | scan call count only | yes | (c) | +| design-consultation-core, plan-decision-classification, plan-devex-peer-comparison-classification, diagram-authoring-quality | design :168; classification :150-153, :40-43; diagram :159-183 | — | — | — | judge | + +### Context, scrape, CSO, health, Aside + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| context-save-then-restore-roundtrip | test/skill-e2e-context-skills.test.ts:221, :231 | surface has the magic title (vacuous: it is in the Skill input) | restore Read/Bash tool_result of the saved file | no | (b) | +| context-restore-fragment-match | test/skill-e2e-context-skills.test.ts:266-267, :277-278 | surface has the PAYMENTS marker, lacks ALPHA/OMEGA | Read/Bash tool_result content | no | (b) | +| context-restore-provenance-order | test/skill-e2e-context-skills.test.ts:321-327, :344 | "Next steps" before "Verify first" with tokens placed | none | yes (context-restore/SKILL.md.tmpl:217-221) | (c) | +| context-restore-provenance-order | test/skill-e2e-context-skills.test.ts:328, :335, :338, :345 | `FIRST_ACTION: verify PROV_ASSUMED_7Q` | none (AUQ disallowed) | no | (d) | +| context-restore-empty-state | test/skill-e2e-context-skills.test.ts:374, :385 | surface ~ no saved context/NO_CHECKPOINTS (vacuous: in the injected body) | Bash result `NO_CHECKPOINTS` (context-restore/SKILL.md.tmpl:84, :95) | partly | (b) | +| context-restore-list-delegates | test/skill-e2e-context-skills.test.ts:413, :423 | surface ~ `context-save list` (vacuous) | none | yes | (c) | +| context-restore-legacy-compat | test/skill-e2e-context-skills.test.ts:465-470, :480 | surface has marker/title/filename/branch or pre-rename | Read of the legacy file | no | (b) | +| context-save-list-current-branch | test/skill-e2e-context-skills.test.ts:517, :536 | surface ~ timestamp/main-work | Bash listing result | partly | (b) | +| context-save-list-current-branch | test/skill-e2e-context-skills.test.ts:525-526, :537-538 | final lacks other-branch entries | none (model-rendered list) | yes | (c) | +| context-save-list-all-branches | test/skill-e2e-context-skills.test.ts:572-576, :586 | surface has all 3 timestamps | Bash listing result | partly | (b) | +| scrape-prototype-path | test/skill-e2e-skillify.test.ts:283, :290-294, :304-305 | surface ~ /skillify/ and `"items": [` (both vacuous: in scrape/SKILL.md.tmpl:32, :173) | none | yes | (c) | +| cso-full-audit | test/skill-e2e-cso.test.ts:139-140 | server.js finding prose ~ tenant/owner/authoriz | no class field in report.json | no | (d) | +| cso-infra-scope | test/skill-e2e-cso.test.ts:227-229 | finding prose ~ comment body, run/shell, GITHUB_TOKEN | same | no | (d) | +| health-reporting | test/helpers/health-eval-fixture.ts:132-156 | rendered dashboard: composite 5.6, partial coverage, 0/10 typecheck, "60 errors", unavailable categories, no trend verdict, N/A composite | health-history.jsonl rows (already checked :158-176) | yes | (c) | +| aside-browse-basic, aside-qa-quick, aside-scrape-json | test/skill-e2e-aside.test.ts:179, :219, :234-239 | title in final message; `health score` in report; final JSON contains page1 | none verified | yes | (c) | +| aside-canary-quick | test/skill-e2e-aside.test.ts:256 | canary report ~ CANARY REPORT/Status: | `{date}-canary.json` (canary/SKILL.md.tmpl:221); not verified | no | (b) | +| qa-b6-static, qa-b7-spa, qa-b8-checkout | test/skill-e2e-qa-bugs.test.ts:158, :177-179 | — | — | — | judge | + +### Office hours, gbrain, codex, outside voice, safety, benchmarks + +| Case | Check (file:line) | Matches | Structured source | User-visible | Disposition | +|---|---|---|---|---|---| +| office-hours-phase4-fork | test/skill-e2e-office-hours-phase4.test.ts:190-191 | written question has `because` and ≥2 option lines | none (AUQ suppressed by the prompt, :128) | yes | (c) | +| office-hours-phase4-fork | test/skill-e2e-office-hours-phase4.test.ts:193 | approach/architecture vocabulary or Server/Client/Hybrid options | none; a native capture would make it structural | no | (d) | +| office-hours-phase4-fork | test/helpers/llm-judge.ts:496-516 (via test/helpers/e2e-helpers.ts:320-322) | deterministic `Recommendation:` + `because` part of judgeRecommendation | none | yes | (c) | +| office-hours-design-draft | test/helpers/office-hours-completion.ts:107-115 | design doc has substantive Problem/Approach/Success/"What I noticed" and Assignment sections | none | yes | (c) | +| setup-gbrain-bad-token | test/skill-e2e-setup-gbrain-bad-token.test.ts:60-61 | `JSON.stringify(result)` ~ error_class AUTH/rotate token (satisfied by the helper's own stdout) | verify JSON `error_class` (bin/gstack-gbrain-mcp-verify:64), already checked :65-67 | partly | (b) | +| setup-gbrain-remote | test/skill-e2e-setup-gbrain-remote.test.ts:326-328 | serialized transcript must not contain `"error_class": NETWORK/AUTH/MALFORMED` | verify JSON in the Bash result | no | (b) | +| setup-gbrain-remote | test/skill-e2e-setup-gbrain-remote.test.ts:352 | CLAUDE.md ~ `Mode: remote-http` | the persisted config block | yes | (c) | +| setup-gbrain-path4-local-pglite | test/skill-e2e-setup-gbrain-path4-local-pglite.test.ts:72-73; test/helpers/setup-gbrain-fixture.ts:122-169 | question ~ code search/pglite/symbol; scripted actor classifies option labels and throws on unknown sets | AUQ labels only, no option ids | yes | (c) | +| sync-gbrain-read-ready, sync-gbrain-read-unknown | test/helpers/sync-gbrain-readiness-verdict.ts:4-25 | Capability row OK/WARN, YELLOW verdict, no unnegated ready claim | bin/gstack-gbrain-read-capability.ts:87 status JSON (status only; row and verdict are rendered) | yes | (c) | +| codex-discover-skill | test/helpers/codex-eval.ts:43 | output ~ review/gstack/skill | none | partly | (d) | +| codex-review-findings | test/helpers/codex-eval.ts:66-67; lib/outside-review-result.ts:88 | output ~ finding/issue/…/no issues; refusal phrases → unavailable | executed commands read first; no review-log step in the fixture | partly | (d) | +| outside-voice-codex-to-claude-code, outside-voice-claude-code-to-codex | test/helpers/outside-voice-evidence.ts:214-216 | outside stdout ~ invoice and authorization defect; no refusal | dispatch and receipts are structural; the review-log row holds status only | no | (d) | +| outside-plan-disabled-no-fallback | test/helpers/disabled-plan-review-fixture.ts:322-325 | JSON `outside_review_this_run: disabled` first, then prose | review-log `outside_status:"disabled"` (scripts/resolvers/outside-voice-steps.ts:331) | yes | (a) | +| outside-plan-disabled-no-fallback | test/helpers/disabled-plan-review-fixture.ts:326-328 | no unattributed "both reviewers agree" claim | none | yes | (c) | +| safety-design-risk-stop | test/skill-e2e-safety-design-risk.test.ts:165 | final names each unfixed FINDING-00N | git log decides which remain; the listing has no record | yes | (c) | +| benchmark-providers-live | test/skill-e2e-benchmark-providers.test.ts:116, :136, :168 | provider output contains "ok" | none | yes | (c) | +| office-hours-forcing-energy, office-hours-builder-wildness, office-hours-phase4-fork, safety-design-risk-stop, llm-judge-recommendation, arm-benchmark-* | office-hours :102-107, :183-188; e2e-helpers.ts:323-326; design-risk :166-174; llm-judge-recommendation :42-163; arm-benchmark :121 (non-gating) | — | — | — | judge | + +Periodic cases with no phrase-only checks: plan-ceo-review, plan-ceo-review-selective, plan-eng-review (exit reason and optional file length only); autoplan-dual-voice and autoplan-journal-drift (transcript evidence); the eleven journey-* routing cases (Skill tool input); context-save-routing; scrape-match-path; qa-fix-loop; retro; office-hours-brain-writeback; gbrain-roundtrip-local; safety-codex-boundary; safety-ios-demo; codex-sol-scope-termination; ios-qa-device; design-consultation-preview; the five overlay-harness cases (JSON parse plus deep-equal). + +## Convert candidates + +These are ordered by red history across the 8 wave censuses (periodic censuses 37174266054 through 37198445662), then by tier and how many cases one change covers. + +1. **S5 screen-first auto-decision** (test/helpers/pty/classify.ts:460). auto-decide-preserved went red in 3 of 8 censuses, all `outcome=timeout`. Its replay corpus (test/fixtures/detector-corpora/auto-decide-preserved/) shows two were product failures (the model never sent the handoff line) and one an order false red that is already fixed, so no red came from the screen phrase itself. Letting the bound question-log record hand the decision to the native transcript check also covers office-hours-auto-mode and the plan-mode smokes, but two of the five recent passing captures cannot prove that path from their archived polls (the poll predates the handoff message), so the conversion needs a paid diagnostic run before it lands. +2. **shared-libs-opportunity-judgment transcript regex** (test/skill-e2e-shared-libs-periodic.test.ts:123). The case went red in 2 of 8, but both reds were judge failures, so this conversion removes a fragile check rather than a past red. +3. **S3/S4/F3 PTY terminals** (test/helpers/pty/classify.ts:425, :467; test/helpers/pty/runners/floor.ts:497). 0 reds. Derive `plan_ready` and `silent_write` from ExitPlanMode and Write tool_use in the journal the runner already reads. One change covers 4 gate and 5 periodic cases. +4. **Setup selectors that stall on wording** (test/helpers/plan-floor-review.ts:278-280, test/helpers/pty/boundaries.ts:653-658, test/helpers/plan-review-board-feedback.ts:111-120, test/skill-e2e-plan-design-with-ui.test.ts:34-57). 0 census reds, but PR-lane reds for plan-design-with-ui-scope (runs 37166586458, 37172907071). Identify the question by AUQ header and option labels. +5. **Gate READY probe** in skillmd-no-local-binary and skillmd-outside-git (test/skill-e2e-bws.test.ts:156, :194). Read the setup-block Bash result instead of the final message. +6. **Gate artifact and trace swaps**: context-save-writes-file (parse frontmatter), ship-base-branch (git trace), land-and-deploy-workflow and -first-run (`PLATFORM:` result), setup-deploy-workflow (config block), skillify-provenance-refusal load tripwire (Skill tool_result), shared-libs-review-path-eligibility (drop the `output` operand). Separately, context-recovery-artifacts needs `bin/gstack-context-recovery` copied into its fixture before it can convert. +7. **plan-eng-multi-finding-batching** (test/helpers/pty/boundaries.ts:62-116, test/helpers/eng-seeded-coverage.ts:33-104, test/helpers/pty/classify.ts:555). Count decision-ledger R records bound to answered AUQ ids instead of question prose. +8. **Periodic surface narrowing**: the context-skills roundtrip, fragment, empty-state, legacy and list cases; carve-section-loading [plan-eng-review]; review-army red-team and consensus; setup-gbrain-bad-token and -remote; aside-canary-quick. Several of these phrase checks are vacuous today, because the surface includes the injected SKILL.md body, so narrowing them to tool results will surface new reds rather than remove old ones. + +Red cases that are not convert candidates: shared-libs-plan-callers (3 reds, actor phrase refusal on "hardening", (c)); plan-ceo-section-loading (2 reds, report checks (c) and a ledger check already (a)); plan-review-prosons-neutral-neg (2, question format (c)); plan-ceo-review-format-mode and -format-approach, tpa-absent-darwin, and auq-matrix (1 each, (c)); plan-design-review-plan-mode (1, (d)); scrape-prototype-path (1, vacuous (c); the failing assertion is unknown); plan-ceo-mode-routing (1, already (a)); plan-eng-review-artifact (timeout), shared-libs-read-only (filesystem snapshot), shared-libs-review-revalidation (process abort) and the overlay sonnet case (measurement validity) did not fail on a phrase check. The four `SKILL.md workflow`/`platform setup` reds are skill-llm-eval judges outside `E2E_TIERS`. + +## Not audited + +- Census commits `a8ace39` through `fd6854b` are not in this checkout, so the wave-reds line numbers could not be mapped to assertions; reds are attributed from the trial error text. +- The skill-llm-eval.test.ts judge cases are outside `E2E_TIERS` and are not covered. +- codex-ci-access.test.ts, codex-hardening.test.ts, codex-under-codex-detection.test.ts, e2e-shard-reuse.test.ts and skill-routing-e2e.test.ts name the umbrella keys but are free tests or route to the files audited above. +- plan-eng-review-artifact-full is `marathon` and out of scope. +- The aside-* cases need a live Aside on macOS and never run in CI. +- "Not verified" in the structured-source column means the template writes the record, but the audit did not confirm that the fixture installs the helper. diff --git a/test/auq-substance-panel.test.ts b/test/auq-substance-panel.test.ts new file mode 100644 index 000000000..115b119d4 --- /dev/null +++ b/test/auq-substance-panel.test.ts @@ -0,0 +1,49 @@ +import { describe, test, expect } from 'bun:test'; +import * as fs from 'fs'; +import * as path from 'path'; +import { auqSubstancePanel, AUQ_SUBSTANCE_MIN } from './helpers/auq-substance-panel'; +import { JUDGE_PANEL_SAMPLES } from './helpers/llm-judge'; + +const grader = (scores: number[]) => { + const seen: string[] = []; + let index = 0; + const grade = async (text: string) => { + seen.push(text); + const substance = scores[index++]; + if (substance === undefined) throw new Error('grader called more than the panel size'); + return { substance, present: true, hadLiteralBecause: true, reason: 'r' }; + }; + return { grade, seen }; +}; + +describe('auq-matrix substance panel (C4)', () => { + test('one capture is graded by a full panel and gates on the mean at the unchanged minimum', async () => { + expect(AUQ_SUBSTANCE_MIN).toBe(4); + const { grade, seen } = grader([5, 3, 4]); + const panel = await auqSubstancePanel('Recommendation: A because B', grade); + expect(seen).toEqual(Array(JUDGE_PANEL_SAMPLES).fill('Recommendation: A because B')); + expect(panel).toMatchObject({ mean: 4, passed: true }); + }); + test('a single low sample no longer decides the case, but a low mean still fails', async () => { + expect((await auqSubstancePanel('q', grader([4, 3, 5]).grade)).passed).toBe(true); + expect((await auqSubstancePanel('q', grader([4, 3, 4]).grade))).toMatchObject({ passed: false }); + expect((await auqSubstancePanel('q', grader([3, 3, 3]).grade)).mean).toBe(3); + }); + test('a failed sample fails the panel without resampling', async () => { + let calls = 0; + const grade = async () => { + calls++; + if (calls === 2) throw new Error('judge returned non-JSON'); + return { substance: 5, present: true, hadLiteralBecause: true, reason: 'r' }; + }; + await expect(auqSubstancePanel('q', grade)).rejects.toThrow('non-JSON'); + expect(calls).toBe(JUDGE_PANEL_SAMPLES); + }); + test('the matrix scores its single capture through the panel', () => { + const source = fs.readFileSync(path.join(import.meta.dir, 'skill-e2e-auq-matrix.test.ts'), 'utf8'); + expect(source.match(/captureNativeFirstAuq\(/g)).toHaveLength(1); + expect(source).toContain('auqSubstancePanel(text)'); + expect(source).not.toMatch(/gradeAuqRecommendation\(/); + expect(source).toContain('substance < AUQ_SUBSTANCE_MIN'); + }); +}); diff --git a/test/ceo-stale-fill-decision.test.ts b/test/ceo-stale-fill-decision.test.ts index 62fe98abb..092f63e5f 100644 --- a/test/ceo-stale-fill-decision.test.ts +++ b/test/ceo-stale-fill-decision.test.ts @@ -55,3 +55,34 @@ describe('CEO stale-fill decision from the skill-defined ledger and currentDecis expect(hasApprovedStaleFillDecision('```text\n' + report.replaceAll('```', '') + '\n```')).toBe(false); }); }); + +describe('structured reader alone on the plan-ceo-section-loading corpus', () => { + const corpus = path.join(import.meta.dir, 'fixtures', 'detector-corpora', 'plan-ceo-section-loading'); + const entry = (name: string): string => JSON.parse(fs.readFileSync(path.join(corpus, name), 'utf8')).input.report; + const camelWrite = entry('37186854666-t1-pass.json'); + const camelQuestion = camelWrite.split('\n').find(line => line.startsWith('Question: D1'))!; + + test('reads an approved decision whose question names the write as writeProfile (37186854666)', () => { + expect(camelQuestion).toContain('writeProfile'); + expect(hasApprovedStaleFillDecision(camelWrite)).toBe(true); + }); + + test('reads an approved decision under the slash-joined row ID D1 / STALE-FILL (37195203538)', () => { + expect(hasApprovedStaleFillDecision(entry('37195203538-t1-pass.json'))).toBe(true); + }); + + test('still refuses an unrelated decision, the copied plan and an unresolved ledger row', () => { + expect(hasApprovedStaleFillDecision(entry('37182865432-t1-fail-derived-unrelated-decision.json'))).toBe(false); + expect(hasApprovedStaleFillDecision(CEO_SECTION_CACHE_PLAN)).toBe(false); + expect(hasApprovedStaleFillDecision(entry('37174266054-t1-pass.json'))).toBe(false); + }); + + test.each(['rewriteProfile', 'isWriteable', 'writeable'])('a question whose only write term is %s is not the race', identifier => { + expect(hasApprovedStaleFillDecision(camelWrite.replace(camelQuestion, camelQuestion.replace('writeProfile', identifier)))).toBe(false); + }); + + test('a slash-joined ID still needs its own approved ledger row', () => { + const report = entry('37195203538-t1-pass.json'); + expect(hasApprovedStaleFillDecision(report.replaceAll('| D1 / STALE-FILL ', '| D1 / OTHER-ROW '))).toBe(false); + }); +}); diff --git a/test/detector-corpora.test.ts b/test/detector-corpora.test.ts new file mode 100644 index 000000000..75f04c61d --- /dev/null +++ b/test/detector-corpora.test.ts @@ -0,0 +1,77 @@ +/** + * B1 replay corpora contract. Each case directory under + * test/fixtures/detector-corpora/ holds minimal, redacted detector or actor + * inputs from named census captures, an inventory of every census in scope + * (expired artifacts listed as unavailable), and a replay test + * (test/detector-corpus-.test.ts) that runs the real detector. + * Stale entries (series identity no longer current) are reported for refresh, + * never deleted: a fixed false red stays an expected pass with its history. + */ +import { describe, expect, test } from 'bun:test'; +import * as fs from 'fs'; +import * as path from 'path'; +import { scan } from '../lib/redact-engine'; +import { caseSeriesIdentities } from '../scripts/eval-flake-rank'; + +const ROOT = path.resolve(import.meta.dir, '..'); +const DIR = path.join(import.meta.dir, 'fixtures', 'detector-corpora'); +const REQUIRED = ['shared-libs-plan-callers', 'plan-ceo-section-loading', 'auto-decide-preserved', + 'plan-design-review-plan-mode', 'ship-docsync-late-result']; +const MAX_BYTES = 64 * 1024; +const cases = fs.readdirSync(DIR).filter(name => fs.statSync(path.join(DIR, name)).isDirectory()).sort(); +const entries = (c: string) => fs.readdirSync(path.join(DIR, c)).filter(f => f.endsWith('.json') && f !== 'inventory.json').sort() + .map(file => ({ file, bytes: fs.statSync(path.join(DIR, c, file)).size, raw: fs.readFileSync(path.join(DIR, c, file), 'utf8') })) + .map(entry => ({ ...entry, json: JSON.parse(entry.raw) })); + +test('every required repeat-offender case has a corpus', () => { + expect(cases).toEqual(expect.arrayContaining(REQUIRED)); +}); + +describe.each(cases)('%s corpus', (c) => { + const all = entries(c); + test('has a replay test, at least one expected pass and one expected refusal', () => { + expect(fs.existsSync(path.join(import.meta.dir, `detector-corpus-${c}.test.ts`))).toBe(true); + expect(all.some(entry => entry.json.expected === 'pass')).toBe(true); + expect(all.some(entry => entry.json.expected === 'refuse')).toBe(true); + }); + test.each(all.map(entry => [entry.file, entry] as const))('%s is a bounded, attributed, credential-free entry', (_file, entry) => { + const e = entry.json; + expect(entry.bytes).toBeLessThanOrEqual(MAX_BYTES); + expect(e).toMatchObject({ schema: 'gstack-detector-corpus/v1', case: c }); + expect(e.detector).toMatch(/^test\/[\w./-]+\.ts#\S+/); + expect(e.series_identity).toMatch(/^[0-9a-f]{16}$/); + expect(e.source.run).toMatch(/^\d{8,}$/); + expect(e.source.sha).toMatch(/^[0-9a-f]{40}$/); + expect(['periodic', 'gate', 'pr']).toContain(e.source.lane); + expect(['passed', 'failed']).toContain(e.census_outcome); + expect(['pass', 'refuse']).toContain(e.expected); + expect(e.justification.trim().length).toBeGreaterThan(10); + expect(e.input).toBeDefined(); + // A refusal that no census produced must say how it was derived. + if (e.expected === 'refuse' && e.census_outcome === 'passed') expect(typeof e.derived).toBe('string'); + const scanned = scan(entry.raw, { repoVisibility: 'private' }); + expect(scanned.oversize).toBe(false); + expect(scanned.findings.filter(f => f.severity === 'HIGH').map(f => f.id)).toEqual([]); + }); + test('the inventory lists every census in scope and only existing entries', () => { + const inventory = JSON.parse(fs.readFileSync(path.join(DIR, c, 'inventory.json'), 'utf8')) as Array>; + expect(inventory.length).toBeGreaterThan(0); + const files = new Set(all.map(entry => entry.file)); + for (const row of inventory) { + expect(String(row.run)).toMatch(/^\d{8,}$/); + expect(typeof row.available).toBe('boolean'); + if (row.entry !== null && row.entry !== undefined) expect(files.has(String(row.entry))).toBe(true); + } + for (const entry of all.filter(entry => !entry.json.derived)) { + expect(inventory.some(row => String(row.run) === entry.json.source.run)).toBe(true); + } + }); +}); + +test('stale corpus entries are reported for refresh', () => { + const current = caseSeriesIdentities(cases, ROOT); + const stale = cases.flatMap(c => entries(c).filter(entry => entry.json.series_identity !== current[c]) + .map(entry => `${c}/${entry.file} (series ${entry.json.series_identity}, current ${current[c]})`)); + if (stale.length) console.log(`[detector-corpora] ${stale.length} stale entr${stale.length === 1 ? 'y' : 'ies'} (refresh from a current census when one fails):\n ${stale.join('\n ')}`); + expect(Object.keys(current).sort()).toEqual(cases); +}); diff --git a/test/detector-corpus-auto-decide-preserved.test.ts b/test/detector-corpus-auto-decide-preserved.test.ts new file mode 100644 index 000000000..86d88ae57 --- /dev/null +++ b/test/detector-corpus-auto-decide-preserved.test.ts @@ -0,0 +1,64 @@ +/** Replays every stored auto-decide-preserved census capture through the observer's real screen and native detectors. */ +import { describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import { findNativeAutoDecision } from './helpers/native-auto-decide'; +import { classifyVisible } from './helpers/pty/classify'; + +const DIR = path.join(import.meta.dir, 'fixtures/detector-corpora/auto-decide-preserved'); +const inventory: Array<{ run: string; outcome: string; available: boolean; entry: string | null }> = + JSON.parse(fs.readFileSync(path.join(DIR, 'inventory.json'), 'utf8')); +const files = fs.readdirSync(DIR).filter(name => name !== 'inventory.json' && name.endsWith('.json')).sort(); +const load = (name: string) => JSON.parse(fs.readFileSync(path.join(DIR, name), 'utf8')); +const native = (entry: any) => findNativeAutoDecision(entry.input.native.transcript, entry.input.native.tools, entry.input.native.options); +const screen = (entry: any) => ['auto_decided', 'plan_ready'].includes(classifyVisible(entry.input.screen.text, { strictPlanWrites: true })?.outcome ?? ''); +const verdict = (credited: boolean) => credited ? 'pass' : 'refuse'; + +describe('auto-decide-preserved detector corpus', () => { + test('inventory names every entry, and each entry is bounded and pinned to its census', () => { + expect(inventory.flatMap(row => row.entry ? [row.entry] : []).sort()).toEqual(files); + const verdicts = files.map(name => load(name).expected); + expect(verdicts).toContain('pass'); + expect(verdicts).toContain('refuse'); + for (const name of files) { + expect(fs.statSync(path.join(DIR, name)).size, name).toBeLessThanOrEqual(64 * 1024); + const entry = load(name); + const row = inventory.find(r => r.entry === name)!; + expect(entry.schema).toBe('gstack-detector-corpus/v1'); + expect(entry.case).toBe('auto-decide-preserved'); + expect(entry.source.run).toBe(row.run); + expect(entry.source.sha).toMatch(/^[0-9a-f]{40}$/); + expect(entry.census_outcome).toBe(row.outcome); + expect(name).toBe(`${row.run}-t1-${row.outcome === 'passed' ? 'pass' : 'fail'}.json`); + expect(entry.justification.trim()).not.toBe(''); + expect(JSON.stringify(entry)).not.toMatch(/\/home\/runner|\/Users\//); + } + }); + + for (const name of files) { + const entry = load(name); + test(`${name}: the case's detectors ${entry.expected === 'pass' ? 'credit' : 'refuse'} it (census ${entry.census_outcome})`, () => { + expect(verdict(screen(entry) || native(entry) !== null)).toBe(entry.expected); + expect(verdict(screen(entry))).toBe(entry.screen_expected); + const decision = native(entry); + expect(verdict(decision !== null)).toBe(entry.native_expected); + if (decision) expect(decision.option).toBe('HOLD SCOPE'); + }); + } + + test('37186854666: a "Mode application and rationale." heading is not a mode field, but a real unfinished field still withdraws', () => { + const entry = load('37186854666-t1-pass.json'); + const last = () => entry.input.native.transcript.assistantMessages.at(-1); + const original = last().text; + expect(original).toContain('**Mode application and rationale.** The draft is a fix to an existing path with its own ceiling already pinned: membership'); + expect(native(entry)).toMatchObject({ option: 'HOLD SCOPE' }); + for (const [label, edit] of Object.entries({ + 'colon inside the heading sentence': (t: string) => t.replace('**Mode application and rationale.**', '**Mode application and rationale:** still open.'), + 'later pending field': (t: string) => `${t}\n\nMode pending: SCOPE EXPANSION.`, + 'later different mode': (t: string) => `${t}\n\nMode: SCOPE EXPANSION.`, + })) { + last().text = edit(original); + expect(native(entry), label).toBeNull(); + } + }); +}); diff --git a/test/detector-corpus-plan-ceo-section-loading.test.ts b/test/detector-corpus-plan-ceo-section-loading.test.ts new file mode 100644 index 000000000..473cd2f1b --- /dev/null +++ b/test/detector-corpus-plan-ceo-section-loading.test.ts @@ -0,0 +1,66 @@ +import { describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import { hasApprovedStaleFillDecision } from './helpers/ceo-stale-fill-decision'; +import { CEO_SECTION_CACHE_PLAN, hasStaleFillRaceFinding } from './helpers/ceo-section-loading-fixture'; + +const CASE = 'plan-ceo-section-loading'; +const dir = path.join(import.meta.dir, 'fixtures', 'detector-corpora', CASE); +const WITNESS = 'expect(hasApprovedStaleFillDecision(output) || hasStaleFillRaceFinding(output)).toBe(true);'; + +type Entry = { + schema: string; + case: string; + detector: string; + series_identity: string; + source: { run: string; sha: string; lane: string; artifact: string; path: string }; + census_outcome: 'passed' | 'failed'; + census_error: string; + derived?: string; + expected: 'pass' | 'refuse'; + justification: string; + input: { report: string }; +}; +type InventoryRow = { run: string; lane: string; outcome: string; series_identity: string; artifact: string; available: boolean; entry: string | null; reason?: string }; + +const inventory: InventoryRow[] = JSON.parse(fs.readFileSync(path.join(dir, 'inventory.json'), 'utf8')); +const files = fs.readdirSync(dir).filter(name => name !== 'inventory.json').sort(); +const entries = files.map(name => [name, JSON.parse(fs.readFileSync(path.join(dir, name), 'utf8')) as Entry] as const); + +describe(`${CASE} detector corpus`, () => { + test('the corpus replays the exact stale-fill witness the E2E asserts', () => { + const e2e = fs.readFileSync(path.join(import.meta.dir, 'skill-e2e-plan-ceo-review-section-loading.test.ts'), 'utf8'); + expect(e2e).toContain(WITNESS); + }); + + test('inventory and entry files agree, and each entry is well formed', () => { + expect(inventory.filter(row => row.entry).map(row => row.entry).sort()).toEqual([...files]); + for (const row of inventory) if (!row.entry) expect(row.reason, row.run).toBeTruthy(); + for (const [name, entry] of entries) { + expect(fs.statSync(path.join(dir, name)).size, name).toBeLessThanOrEqual(64 * 1024); + expect(entry.schema, name).toBe('gstack-detector-corpus/v1'); + expect(entry.case, name).toBe(CASE); + expect(entry.source.sha, name).toMatch(/^[0-9a-f]{40}$/); + expect(name.startsWith(`${entry.source.run}-t`), name).toBe(true); + const row = inventory.find(r => r.entry === name)!; + expect({ outcome: row.outcome, series: row.series_identity }, name).toEqual({ outcome: entry.census_outcome, series: entry.series_identity }); + expect(entry.justification.trim().length, name).toBeGreaterThan(0); + expect(entry.input.report, name).not.toMatch(/\/home\/runner|gstack-ceo-secload-/); + } + }); + + test('holds at least one expected pass and one expected refusal', () => { + const expected = entries.map(([, entry]) => entry.expected); + expect(expected).toContain('pass'); + expect(expected).toContain('refuse'); + }); + + test.each(entries)('%s replays to its expected result', (_name, entry) => { + const report = entry.input.report; + expect(hasApprovedStaleFillDecision(report) || hasStaleFillRaceFinding(report)).toBe(entry.expected === 'pass'); + }); + + test('the copied fixture plan is not evidence', () => { + expect(hasApprovedStaleFillDecision(CEO_SECTION_CACHE_PLAN) || hasStaleFillRaceFinding(CEO_SECTION_CACHE_PLAN)).toBe(false); + }); +}); diff --git a/test/detector-corpus-plan-design-review-plan-mode.test.ts b/test/detector-corpus-plan-design-review-plan-mode.test.ts new file mode 100644 index 000000000..7cc293b46 --- /dev/null +++ b/test/detector-corpus-plan-design-review-plan-mode.test.ts @@ -0,0 +1,109 @@ +import { describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { spawnSync } from 'node:child_process'; +import { CAPTURE_MS, CAPTURE_LONG_MS } from './helpers/eval-budgets'; +import { OFFICE_HOURS_BUN_GRACE_MS, runRecordedOfficeHoursAttempt } from './helpers/office-hours-attempt'; +import { resolveEvalModel } from '../lib/eval-model'; + +// Replays stored census captures of plan-design-review-plan-mode through the +// case's real validate callback. The detector lives inline in the paid suite, +// so this evaluates its registration (the plan-design-sdk-fixture pattern) +// and replaces only the model run: the captured final message and saved plan. + +const ROOT = path.resolve(import.meta.dir, '..'); +const CASE = 'plan-design-review-plan-mode'; +const DIR = path.join(ROOT, 'test/fixtures/detector-corpora', CASE); +const source = fs.readFileSync(path.join(ROOT, 'test/skill-e2e-design.test.ts'), 'utf8'); + +interface CorpusInput { exit_reason: string; output: string; plan_after: string | null } +interface CorpusEntry { + schema: string; case: string; detector: string; series_identity: string; + source: { run: string; sha: string; lane: string; artifact: string; path: string }; + census_outcome: 'passed' | 'failed'; census_error: string; + expected: 'pass' | 'refuse'; justification: string; derived?: string; input: CorpusInput; +} + +const files = fs.readdirSync(DIR).filter(name => name.endsWith('.json') && name !== 'inventory.json').sort(); +const entries = files.map(name => ({ name, entry: JSON.parse(fs.readFileSync(path.join(DIR, name), 'utf8')) as CorpusEntry })); + +async function replay(input: CorpusInput) { + const scratch = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'detector-corpus-pdr-'))); + const home = path.join(scratch, 'home'); fs.mkdirSync(home); + const env = { PATH: process.env.PATH ?? '', HOME: home, GIT_CONFIG_NOSYSTEM: '1', + GIT_CONFIG_GLOBAL: process.platform === 'win32' ? 'NUL' : '/dev/null', + ...(process.env.SystemRoot ? { SystemRoot: process.env.SystemRoot } : {}) }; + const rows: Array<{ passed: boolean; exit_reason?: string; error?: string }> = []; + const callbacks: Array<() => Promise> = []; + let workDir = ''; + const args = { + ROOT, fs, path, os: { tmpdir: () => scratch }, expect, + CAPTURE_MS, CAPTURE_LONG_MS, OFFICE_HOURS_BUN_GRACE_MS, resolveEvalModel, runId: 'detector-corpus', + evalCollector: { addTest: (row: (typeof rows)[number]) => { rows.push(row); } }, + runRecordedOfficeHoursAttempt, logCost: () => {}, recordE2E: () => {}, + describeIfSelected: (_title: string, names: string[], fn: () => void) => { if (names.includes(CASE)) fn(); }, + testConcurrentIfSelected: (name: string, fn: () => Promise) => { if (name === CASE) callbacks.push(fn); }, + spawnSync: (bin: string, argv: string[], opts: { cwd: string; timeout?: number }) => + spawnSync(bin, argv, { ...opts, env, timeout: opts.timeout ?? 5000 }), + runSkillTest: async (opts: { workingDirectory: string }) => { + workDir = opts.workingDirectory; + expect(path.dirname(workDir)).toBe(scratch); + if (input.plan_after !== null) fs.writeFileSync(path.join(workDir, 'plan.md'), input.plan_after); + return { output: input.output, exitReason: input.exit_reason, duration: 1, toolCalls: [], browseErrors: [], + transcript: [], model: 'detector-corpus-replay', + costEstimate: { inputChars: 0, outputChars: 0, estimatedTokens: 0, estimatedCost: 0, turnsUsed: 0 } }; + }, + }; + const start = source.indexOf("describeIfSelected('Plan Design Review E2E'"); + const end = source.indexOf("describeIfSelected('Design Review E2E'", start); + expect(start).toBeGreaterThan(0); expect(end).toBeGreaterThan(start); + let error: unknown; + try { + new Function(...Object.keys(args), new Bun.Transpiler({ loader: 'ts' }).transformSync(source.slice(start, end)))(...Object.values(args)); + expect(callbacks).toHaveLength(1); + try { await callbacks[0]!(); } catch (caught) { error = caught; } + expect(workDir).not.toBe(''); + expect(fs.existsSync(workDir)).toBe(false); + expect(rows).toHaveLength(1); + return { row: rows[0]!, error }; + } finally { fs.rmSync(scratch, { recursive: true, force: true }); } +} + +describe(`${CASE} detector corpus`, () => { + test('inventory accounts for every entry and every census in scope', () => { + const inventory = JSON.parse(fs.readFileSync(path.join(DIR, 'inventory.json'), 'utf8')) as Array<{ + run: string; lane: string; outcome: string; series_identity: string; artifact: string; + available: boolean; entry: string | null; reason?: string }>; + for (const row of inventory) { + if (row.entry) expect(files).toContain(row.entry); + else expect(row.reason).toBeTruthy(); + } + for (const { name, entry } of entries) { + const row = inventory.find(r => r.run === entry.source.run); + expect(row, name).toBeDefined(); + expect(entry.census_outcome).toBe(row!.outcome as CorpusEntry['census_outcome']); + expect(entry.series_identity).toBe(row!.series_identity); + if (!entry.derived) expect(row!.entry).toBe(name); + } + expect(entries.some(e => e.entry.expected === 'pass')).toBe(true); + expect(entries.some(e => e.entry.expected === 'refuse')).toBe(true); + }); + + test.each(entries.map(e => [e.name, e.entry] as const))('%s', async (name, entry) => { + expect(entry.schema).toBe('gstack-detector-corpus/v1'); + expect(entry.case).toBe(CASE); + expect(entry.source.sha).toMatch(/^[0-9a-f]{40}$/); + expect(entry.justification).toBeTruthy(); + expect(fs.statSync(path.join(DIR, name)).size).toBeLessThanOrEqual(64 * 1024); + const { row, error } = await replay(entry.input); + expect(row.exit_reason).toBe(entry.input.exit_reason); + if (entry.expected === 'pass') { + expect(error).toBeUndefined(); + expect(row.passed).toBe(true); + } else { + expect(row.passed).toBe(false); + expect(String(error)).toContain('expect(received)'); + } + }); +}); diff --git a/test/detector-corpus-shared-libs-plan-callers.test.ts b/test/detector-corpus-shared-libs-plan-callers.test.ts new file mode 100644 index 000000000..8074085dc --- /dev/null +++ b/test/detector-corpus-shared-libs-plan-callers.test.ts @@ -0,0 +1,46 @@ +/** Replays every stored shared-libs-plan-callers census question through the real fixture actor. */ +import { describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import { createSharedPlanReuseSelector } from './helpers/shared-libs-plan-actor'; + +const CASE = 'shared-libs-plan-callers'; +const DIR = path.join(import.meta.dir, 'fixtures', 'detector-corpora', CASE); +const inventory: any[] = JSON.parse(fs.readFileSync(path.join(DIR, 'inventory.json'), 'utf8')); +const entries = fs.readdirSync(DIR).filter(file => file !== 'inventory.json' && file.endsWith('.json')).sort() + .map(file => ({ file, size: fs.statSync(path.join(DIR, file)).size, entry: JSON.parse(fs.readFileSync(path.join(DIR, file), 'utf8')) })); + +describe(`${CASE} replay corpus`, () => { + test('corpus holds census passes and refusals, each pinned to its source and size-capped', () => { + expect(entries.some(({ entry }) => entry.expected === 'pass')).toBe(true); + expect(entries.some(({ entry }) => entry.expected === 'refuse')).toBe(true); + for (const { file, size, entry } of entries) { + expect(size).toBeLessThanOrEqual(64 * 1024); + expect(entry.schema).toBe('gstack-detector-corpus/v1'); + expect(entry.case).toBe(CASE); + expect(entry.detector).toBe('test/helpers/shared-libs-plan-actor.ts#createSharedPlanReuseSelector'); + expect(entry.source.sha).toMatch(/^[0-9a-f]{40}$/); + expect(['passed', 'failed']).toContain(entry.census_outcome); + expect(entry.justification.length).toBeGreaterThan(0); + expect(file.startsWith(`${entry.source.run}-t`)).toBe(true); + expect(JSON.stringify(entry.input)).not.toMatch(/\/home\/runner|\/Users\//); + } + }); + + test('inventory names an existing entry for every available run', () => { + const files = new Set(entries.map(({ file }) => file)); + for (const row of inventory) { + if (row.available && row.entry) expect(files.has(row.entry)).toBe(true); + if (!row.entry) expect(row.reason).toBeTruthy(); + } + }); + + test.each(entries.map(({ file, entry }) => [file, entry] as const))('%s', (_file, entry) => { + const choose = createSharedPlanReuseSelector(); + if (entry.expected === 'refuse') { + expect(() => choose(entry.input)).toThrow(`${CASE} actor: `); + return; + } + expect(choose(entry.input)).toEqual({ [entry.input.questions[0].question]: entry.expected_answer }); + }); +}); diff --git a/test/detector-corpus-ship-docsync-late-result.test.ts b/test/detector-corpus-ship-docsync-late-result.test.ts new file mode 100644 index 000000000..5cae52d4b --- /dev/null +++ b/test/detector-corpus-ship-docsync-late-result.test.ts @@ -0,0 +1,49 @@ +import { describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { docsActorVerdict } from './helpers/docsync-fault-eval'; +import type { DocsActorState } from './helpers/docsync-fault-actor'; + +const CASE = 'ship-docsync-late-result'; +const DIR = path.join(import.meta.dir, 'fixtures/detector-corpora', CASE); +const ROOT = path.join(os.tmpdir(), 'docsync-late-result-corpus'); +const names = fs.readdirSync(DIR).filter(name => name.endsWith('.json') && name !== 'inventory.json').sort(); +const entries = names.map(name => ({ name, raw: fs.readFileSync(path.join(DIR, name), 'utf8') })) + .map(({ name, raw }) => ({ name, size: Buffer.byteLength(raw), entry: JSON.parse(raw.replaceAll('', ROOT)) })); +const inventory = JSON.parse(fs.readFileSync(path.join(DIR, 'inventory.json'), 'utf8')); + +describe(`${CASE} detector corpus`, () => { + test('entries are well-formed, bounded, redacted and inventoried', () => { + expect(entries.some(({ entry }) => entry.expected === 'pass')).toBe(true); + expect(entries.some(({ entry }) => entry.expected === 'refuse')).toBe(true); + for (const { name, size, entry } of entries) { + expect(size, name).toBeLessThanOrEqual(64 * 1024); + expect(entry.schema).toBe('gstack-detector-corpus/v1'); + expect(entry.case).toBe(CASE); + expect(entry.detector).toBe('test/helpers/docsync-fault-eval.ts#docsActorVerdict'); + expect(entry.source.sha).toMatch(/^[0-9a-f]{40}$/); + expect(['passed', 'failed']).toContain(entry.census_outcome); + expect(['pass', 'refuse']).toContain(entry.expected); + expect(entry.justification.length).toBeGreaterThan(0); + expect(JSON.stringify(entry), name).not.toMatch(/\/home\/runner|\/Users\/|sk-[A-Za-z0-9]{8}|ghp_[A-Za-z0-9]{8}/); + if (entry.derived) { + const origin = entries.find(other => !other.entry.derived && other.entry.source.run === entry.source.run); + expect(origin?.entry.expected, `${name} derives from a capture the detector accepts`).toBe('pass'); + } else { + expect(name).toBe(`${entry.source.run}-t1-${entry.census_outcome === 'passed' ? 'pass' : 'fail'}.json`); + expect(inventory.find((row: { entry: string | null }) => row.entry === name)?.outcome).toBe(entry.census_outcome); + } + } + for (const row of inventory.filter((row: { entry: string | null }) => row.entry)) expect(names).toContain(row.entry); + for (const row of inventory.filter((row: { entry: string | null }) => !row.entry)) expect(row.reason).toBeTruthy(); + }); + + for (const { name, entry } of entries) { + test(`${name} → ${entry.expected}`, () => { + const failures = docsActorVerdict(entry.input.state as DocsActorState, entry.input.report, entry.input.published); + if (entry.expected === 'pass') expect(failures).toEqual([]); + else expect(failures.length).toBeGreaterThan(0); + }); + } +}); diff --git a/test/fixtures/detector-corpora/auto-decide-preserved/37174266054-t1-pass.json b/test/fixtures/detector-corpora/auto-decide-preserved/37174266054-t1-pass.json new file mode 100644 index 000000000..ee77fd246 --- /dev/null +++ b/test/fixtures/detector-corpora/auto-decide-preserved/37174266054-t1-pass.json @@ -0,0 +1,224 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "auto-decide-preserved", + "detector": "test/helpers/native-auto-decide.ts#findNativeAutoDecision", + "screen_detector": "test/helpers/pty/classify.ts#classifyVisible", + "series_identity": "6c9ea4bac7c0bc81", + "source": { + "run": "37174266054", + "sha": "a8ace39030dd92745b45ac16e5d733e63c936c0f", + "lane": "periodic", + "artifact": "paid-slice-20-a1", + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37174266054-1-eval-slices-20/plan-ceo-review-1791084936827-pBoN0B/observation.json" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "Pre-helper skill (a8ace39): the model sent the documented \"Auto-decided review mode → HOLD SCOPE (your preference)\" line, which classifyVisible accepts on screen; it never ran the question log or handoff helper, so no structured decision exists for the native detector.", + "screen_expected": "pass", + "native_expected": "refuse", + "input": { + "native": { + "transcript": { + "status": "ready", + "calls": [], + "assistantMessages": [ + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "text": "Received. I have the draft plan for deterministic skill-list ordering in context, and I understand the handoff constraints: review-mode decision only, full mode name in the audit record and handoff line, no Office Hours or setup, and no substantive review yet. Waiting for your slash command.", + "timestamp": "2026-10-04T03:34:13.255Z" + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "text": "I'll run the skill preamble first, then move through Step 0 only as far as the mode decision you asked for.", + "timestamp": "2026-10-04T03:34:19.295Z" + } + ] + }, + "tools": [ + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:20.346Z", + "toolUseId": "toolu_011kTp99f3Anru9WTVFVQCsZ", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-skill-start --skill \"plan-ceo-review\" --model \"claude\" 2>&1", + "description": "Run gstack skill-start preamble" + } + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:21.872Z", + "toolUseId": "toolu_011kTp99f3Anru9WTVFVQCsZ", + "kind": "result", + "isError": false, + "content": "SKILL_START_PROTO: 1\nBRANCH: main\nPROACTIVE: true\nPROACTIVE_PROMPTED: yes\nSKILL_PREFIX: false\nREPO_MODE: unknown\nSESSION_KIND: interactive\nCONDUCTOR_SESSION: true\nACTIVATED: yes\nFIRST_LOOP_SHOWN: yes\nFIRST_TASK: \nLAKE_INTRO: yes\nTELEMETRY: off\nTEL_PROMPTED: yes\nSESSION_ID: 1611-1791084861-3e00d981\nTEL_START: 1791084861\nEXPLAIN_LEVEL: default\nQUESTION_TUNING: true\nUPDATE_CHECK: false\nLEARNINGS: 0\nHAS_ROUTING: no\nROUTING_DECLINED: false\nVENDORED_GSTACK: no\nMODEL_OVERLAY: claude\nGSTACK_PLAN_MODE: active\nARTIFACTS_SYNC: off\n" + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:34.257Z", + "toolUseId": "toolu_019QAu5FMBCS9XEAt9cdpZd6", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-context-recovery 2>&1 | head -60", + "description": "Recover recent project context" + } + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:38.386Z", + "toolUseId": "toolu_01R2TQ9MyTSkszc3VYsPrKib", + "kind": "use", + "name": "Bash", + "input": { + "command": "echo \"REMOTE: $(git remote get-url origin 2>&1)\"; echo \"HEAD_REF: $(git symbolic-ref refs/remotes/origin/HEAD 2>&1)\"; git rev-parse --verify origin/main >/dev/null 2>&1 && echo \"origin/main exists\" || echo \"no origin/main\"; echo \"--- log\"; git log --oneline -30; echo \"--- stash\"; git stash list; echo \"--- files\"; git ls-files; echo \"--- todos\"; grep -r \"TODO\\|FIXME\\|HACK\\|XXX\" -l --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=.git . | head -30; echo \"--- recent\"; git log --since=30.days --name-only --format=\"\" | sort | uniq -c | sort -rn | head -20", + "description": "Audit repo history, remote, stashes, TODO markers" + } + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:41.827Z", + "toolUseId": "toolu_01FHxBFVyVFterbEmNDTv81v", + "kind": "use", + "name": "Bash", + "input": { + "command": "setopt +o nomatch 2>/dev/null || true; SLUG=$(~/.claude/skills/gstack/browse/bin/remote-slug 2>/dev/null || basename \"$(git rev-parse --show-toplevel 2>/dev/null || pwd)\"); BRANCH=$(git rev-parse --abbrev-ref HEAD 2>/dev/null | tr '/' '-' || echo 'no-branch'); echo \"SLUG=$SLUG BRANCH=$BRANCH\"; DESIGN=$(~/.claude/skills/gstack/bin/gstack-design-doc-find \"$SLUG\" \"$BRANCH\"); [ -n \"$DESIGN\" ] && echo \"Design doc found: $DESIGN\" || echo \"No design doc found\"", + "description": "Check for an office-hours design doc" + } + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:44.138Z", + "toolUseId": "toolu_017d2MhxfKxcb1SZCvmSXLXY", + "kind": "use", + "name": "Bash", + "input": { + "command": "_CROSS_PROJ=$(~/.claude/skills/gstack/bin/gstack-config get cross_project_learnings 2>/dev/null || echo \"unset\"); echo \"CROSS_PROJECT: $_CROSS_PROJ\"; if [ \"$_CROSS_PROJ\" = \"true\" ]; then ~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 --cross-project 2>&1; else ~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 2>&1; fi | head -40", + "description": "Check cross-project setting and search prior learnings" + } + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:46.514Z", + "toolUseId": "toolu_018xZso3XN931CCXMxhRNaqB", + "kind": "use", + "name": "Bash", + "input": { + "command": "SLUG=$(~/.claude/skills/gstack/bin/gstack-slug --get SLUG 2>/dev/null) || true; echo \"SLUG=$SLUG\"; for d in product goals recent-decisions; do echo \"### $d\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get $d --project \"$SLUG\" 2>/dev/null || echo \"(no $d digest available yet)\"; done; echo \"### user-profile\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get user-profile 2>/dev/null || echo \"(no user-profile digest available yet)\"", + "description": "Load brain context digests" + } + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:48.452Z", + "toolUseId": "toolu_01UfuYHFETMDuY3EAP5jsTZZ", + "kind": "use", + "name": "Bash", + "input": { + "command": "printf '%s' \"Select plan-ceo-review mode: SCOPE EXPANSION / SELECTIVE EXPANSION / HOLD SCOPE / SCOPE REDUCTION for deterministic skill-list ordering plan\" | ~/.claude/skills/gstack/bin/gstack-question-preference --check \"plan-ceo-review-mode\" --summary-stdin 2>&1; echo \"EXIT=$?\"", + "description": "Check stored preference for the mode question" + } + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:53.452Z", + "toolUseId": "toolu_01NojwM3bMzrEmdKQtNGJ2KV", + "kind": "use", + "name": "Bash", + "input": { + "command": "_gs_d() { if command -v gtimeout >/dev/null; then gtimeout 30 \"$@\"; elif command -v timeout >/dev/null; then timeout 30 \"$@\"; elif command -v perl >/dev/null; then perl -e 'alarm(shift);exec(@ARGV)' 30 \"$@\"; else return 125; fi; }\n_A=aside; command -v aside >/dev/null || _A=$(command -v ~/.local/bin/aside)\nif [ \"${GSTACK_SKIP_ASIDE:-}\" = \"1\" ] || [ -z \"$_A\" ]; then echo \"NEEDS_ASIDE\"; else _rc=0; _o=$(_gs_d \"$_A\" repl 'console.log(\"ASIDE_READY \" + pwd)' 2>&1) || _rc=$?; case \"$_rc\" in 124|142) echo \"ASIDE_TIMEOUT: probe deadline exceeded\" ;; 125) echo \"ASIDE_UNAVAILABLE: bounded probe unavailable\" ;; 0) if printf '%s\\n' \"$_o\" | grep -q '^ASIDE_READY '; then echo \"READY: $_A\"; else echo \"ASIDE_NOT_RUNNING: no readiness marker\"; fi ;; *) echo \"ASIDE_CLI_ERROR: exit $_rc; inspect aside --help locally\" ;; esac; fi", + "description": "Probe Aside browser readiness" + } + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:54.075Z", + "toolUseId": "toolu_0117Cr93gotHMehSf84wAvp1", + "kind": "use", + "name": "Read", + "input": { + "file_path": "/PLAN.md" + } + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:54.490Z", + "toolUseId": "toolu_019QAu5FMBCS9XEAt9cdpZd6", + "kind": "result", + "isError": false, + "content": "--- RECENT ARTIFACTS ---\n{\"skill\":\"plan-ceo-review\",\"event\":\"started\",\"branch\":\"main\",\"session\":\"1611-1791084861-3e00d981\",\"ts\":\"2026-10-04T03:34:21.900Z\"}\n--- END ARTIFACTS ---" + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:54.562Z", + "toolUseId": "toolu_01R2TQ9MyTSkszc3VYsPrKib", + "kind": "result", + "isError": false, + "content": "REMOTE: error: No such remote 'origin'\nHEAD_REF: fatal: ref refs/remotes/origin/HEAD is not a symbolic ref\norigin/main exists\n--- log\nf181f79 Seed review plan\n-…[elided: not consumed]" + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:54.593Z", + "toolUseId": "toolu_01FHxBFVyVFterbEmNDTv81v", + "kind": "result", + "isError": false, + "content": "SLUG=gstack-plan-count-66Rq4t BRANCH=main\nNo design doc found" + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:54.644Z", + "toolUseId": "toolu_017d2MhxfKxcb1SZCvmSXLXY", + "kind": "result", + "isError": false, + "content": "CROSS_PROJECT: false" + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:54.756Z", + "toolUseId": "toolu_018xZso3XN931CCXMxhRNaqB", + "kind": "result", + "isError": false, + "content": "SLUG=gstack-plan-count-66Rq4t\n### product\n(no product digest available yet)\n### goals\n(no goals digest available yet)\n### recent-decisions\n(no recent-decisions digest available yet)\n### user-profile\n(no user-profile digest available yet)" + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:54.817Z", + "toolUseId": "toolu_01UfuYHFETMDuY3EAP5jsTZZ", + "kind": "result", + "isError": false, + "content": "AUTO_DECIDE\nEXIT=0" + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:54.834Z", + "toolUseId": "toolu_01NojwM3bMzrEmdKQtNGJ2KV", + "kind": "result", + "isError": false, + "content": "NEEDS_ASIDE" + }, + { + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "timestamp": "2026-10-04T03:34:54.842Z", + "toolUseId": "toolu_0117Cr93gotHMehSf84wAvp1", + "kind": "result", + "isError": false, + "content": "1\t# Draft: deterministic skill-list ordering\n2\t\n3\tUsers compare skill listings in scripts and reviews. Make the existing listing\n4\tpath sort registered skill na…[elided: not consumed]" + } + ], + "options": { + "skillName": "plan-ceo-review", + "sessionId": "869a270b-e457-41dc-b9ad-2dcb706638a6", + "commandStartedAt": 1791084853562, + "now": 1791084936826, + "proseQuestionObserved": false + } + }, + "screen": { + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37174266054-1-eval-slices-20/plan-ceo-review-1791084936827-pBoN0B/terminal.visible.log", + "tail_chars": 6000, + "text": "n file below):\r\r-0A Premise.Realproblem:skilllistingorderdependsonfilesystemenumeration,sotworunsortwomachinesdiff\r differentlyand scipt comparisons flap. Target outcome:byt-identical lisng foridentical egistered skills.\rDo-nothing cost: nosydiffs n reviews and brittle scripts. The plan hits he paindirectly,not a poxy.\r-0B Leverage. Theexting listing path already renders text and JSON;the change sa sor onthename sequence\r before rendering. Standard-library sort with astable key is the whole implementation. No rebuild.\r-0C Dream state. 12-moth ideal: every user-facing enumeraton inte tool (skills, aliases, learnings)is\r deterministic and testedthat way. This plan is th first lake.\r\rCURRENT STATE THIS PLAN 12-MONTH IDEAL\rlistingorder=readdirordersortnamesbeforerender,all enumerations\rflaps across runs/machines tess for case + enum oderdeterministic + tested\r\rNonewapproachdecisionwasneededin0D:asinglesortcallistheonlycredibleapproachandtheplanalready\rspecifies it.\r\r0E recommendation.Estimatedchangedfiles:2to3(onelistingmoduleedit,oneortwotestfiles).Estimate,nota\rcount from code. Category: fix/refactor of existing behavor, with membrship, aliases,mtadata andouput formas\rexplicitlyfrozen. That points to HOLD SCOPE.\r\rAuto-decidedreviewmode→HOLDSCOPE(yourpreference).Changewith/plan-tune.Approveddecisions:none.HOLDSCOPE\r\r●Preambleresultsthatmatterhere:thesessionisinteractiveunderConductor,questiontuningison,andthestored\r pference for the mode qustion reurned AUTO_DECIDE, so thmo resolves from the recommedation withou amenu.\rBase branch ismain (noorigin remoe, but origin/mainexists). Nodign doc, no prior learings, cold braindigests,\rno TODO markers, clea tree with one seed commit. Aside is not installed,s Iskiped landscape web researchfor\rthis mode-only pass d usd n-distribution knowledge.\r\rTwosetupitemsfiredandIamdeferringbothperyourinstruction:theone-timeCLAUDE.mdrouting-rulesprompt(it\ralso requires aCLAUDE.meditand commit, wich plan mode forbds)andth Office Hours offer. Their gates are\rmarke-based and will re-fire on your nextrun.\r\rStep 0 evidence (compact, recorded to the plan file below):\r\r-0A Premise.Realproblem:skilllistingorderdependsonfilesystemenumeration,sotworunsortwomachinesdiff\r differentlyand scipt comparisons flap. Target outcome:byt-identical lisng foridentical egistered skills.\rDo-nothing cost: nosydiffs n reviews and brittle scripts. The plan hits he paindirectly,not a poxy.\r-0B Leverage. Theexting listing path already renders text and JSON;the change sa sor onthename sequence\r before rendering. Standard-library sort with astable key is the whole implementation. No rebuild.\r-0C Dream state. 12-moth ideal: every user-facing enumeraton inte tool (skills, aliases, learnings)is\r deterministic and testedthat way. This plan is th first lake.\r\rCURRENT STATE THIS PLAN 12-MONTH IDEAL\rlistingorder=readdirordersortnamesbeforerender,all enumerations\rflaps across runs/machines tess for case + enum oderdeterministic + tested\r\rNonewapproachdecisionwasneededin0D:asinglesortcallistheonlycredibleapproachandtheplanalready\rspecifies it.\r\r0E recommendation.Estimatedchangedfiles:2to3(onelistingmoduleedit,oneortwotestfiles).Estimate,nota\rcount from code. Category: fix/refactor of existing behavor, with membrship, aliases,mtadata andouput formas\rexplicitlyfrozen. That points to HOLD SCOPE.\r\rAuto-decidedreviewmode→HOLDSCOPE(yourpreference).Changewith/plan-tune.Approveddecisions:none.HOLDSCOPE\rapplies becaus theplanfreezes every output contrat except ordering, so he value f the revew is rigor on the one\r●Preambleresultsthatmatterhere:thesessionisinteractiveunderConductor,questiontuningison,andthestored\r pference for the mode qustion reurned AUTO_DECIDE, so thmo resolves from the recommedation withou amenu.\rBase branch ismain (noorigin remoe, but origin/mainexists). Nodign doc, no prior learings, cold braindigests,\rno TODO markers, clea tree with one seed commit. Aside is not installed,s Iskiped landscape web researchfor\rthis mode-only pass d usd n-distribution knowledge.\r\rTwosetupitemsfiredandIamdeferringbothperyourinstruction:theone-timeCLAUDE.mdrouting-rulesprompt(it\ralso requires aCLAUDE.meditand commit, wich plan mode forbds)andth Office Hours offer. Their gates are\rmarke-based and will re-fire on your nextrun.\r\rStep 0 evidence (compact, recorded to the plan file below):\r\r-0A Premise.Realproblem:skilllistingorderdependsonfilesystemenumeration,sotworunsortwomachinesdiff\r differentlyand scipt comparisons flap. Target outcome:byt-identical lisng foridentical egistered skills.\rDo-nothing cost: nosydiffs n reviews and brittle scripts. The plan hits he paindirectly,not a poxy.\r-0B Leverage. Theexting listing path already renders text and JSON;the change sa sor onthename sequence\r before rendering. Standard-library sort with astable key is the whole implementation. No rebuild.\r-0C Dream state. 12-moth ideal: every user-facing enumeraton inte tool (skills, aliases, learnings)is\r deterministic and testedthat way. This plan is th first lake.\r\rCURRENT STATE THIS PLAN 12-MONTH IDEAL\rlistingorder=readdirordersortnamesbeforerender,all enumerations\rflaps across runs/machines tess for case + enum oderdeterministic + tested\r\rNonewapproachdecisionwasneededin0D:asinglesortcallistheonlycredibleapproachandtheplanalready\rspecifies it.\r\r0E recommendation.Estimatedchangedfiles:2to3(onelistingmoduleedit,oneortwotestfiles).Estimate,nota\rcount from code. Category: fix/refactor of existing behavor, with membrship, aliases,mtadata andouput formas\rexplicitlyfrozen. That points to HOLD SCOPE.\r\rAuto-decidedreviewmode→HOLDSCOPE(yourpreference).Changewith/plan-tune.Approveddecisions:none.HOLDSCOPE\rapplies becaus theplanfreezes every output contrat except ordering, so he value f the revew is rigor on the one\roen contract: the sort key (cae-insensiiveversus byte order, tie-breakwn two names collid cae-nsensitivly," + } + } +} diff --git a/test/fixtures/detector-corpora/auto-decide-preserved/37176837432-t1-fail.json b/test/fixtures/detector-corpora/auto-decide-preserved/37176837432-t1-fail.json new file mode 100644 index 000000000..fc6025139 --- /dev/null +++ b/test/fixtures/detector-corpora/auto-decide-preserved/37176837432-t1-fail.json @@ -0,0 +1,321 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "auto-decide-preserved", + "detector": "test/helpers/native-auto-decide.ts#findNativeAutoDecision", + "screen_detector": "test/helpers/pty/classify.ts#classifyVisible", + "series_identity": "6c9ea4bac7c0bc81", + "source": { + "run": "37176837432", + "sha": "7249f014648202840af294b730ac244e03ce514c", + "lane": "periodic", + "artifact": "paid-slice-20-a1", + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37176837432-1-eval-slices-20/plan-ceo-review-1791088108875-8x79js/observation.json" + }, + "census_outcome": "failed", + "census_error": "AUTO_DECIDE preserve test inconclusive: outcome=timeout", + "expected": "refuse", + "justification": "Genuine product failure on 7249f01: the model checked the preference and logged HOLD SCOPE but never sent the Step 0E \"Auto-decided review mode → …\" line the skill required (\"HOLD SCOPE was auto-selected…\" instead), so no current declaration exists; repaired in the product by e3546d9 (gstack-ceo-mode-handoff prints the line), not in the detector.", + "screen_expected": "refuse", + "native_expected": "refuse", + "derived": "options.stateEvidence rebuilt from the session's gstack-question-log argument with source \"agent\" and ts = that call's tool-result timestamp (the bound question-log.jsonl itself is not archived)", + "input": { + "native": { + "transcript": { + "status": "ready", + "calls": [], + "assistantMessages": [ + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "text": "Received. I have the draft plan for deterministic skill-list ordering in context, along with your constraints: this invocation is for the review-mode decision only, the full mode name goes in the audit record and handoff line, and Office Hours, setup, and the substantive review are deferred. I'll wait for your slash command before taking any action.", + "timestamp": "2026-10-04T04:26:06.989Z" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "text": "I'll run the skill preamble first, then do the pre-review audit and Step 0 up through the mode-selection question, stopping there per your instruction.", + "timestamp": "2026-10-04T04:26:13.842Z" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "text": "Aside is not installed, so the landscape check falls back to one read-only WebSearch query.", + "timestamp": "2026-10-04T04:27:20.063Z" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "text": "The repo only has PLAN.md/CLAUDE.md with no prior context, and the actual listing code isn't in this checkout, so I confirmed via web research the standard fix: sort entries by code point/bytes (not locale) with a stable tie-break to avoid readdir-order fallback. I'll proceed in HOLD SCOPE mode, applying max rigor to just the comparator, case handling, tie-breaking, and the enumeration-order test across the ~2-3 affected files.\n\n", + "timestamp": "2026-10-04T04:28:04.939Z" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "text": "The mode decision is recorded. HOLD SCOPE was auto-selected from your saved preference for the mode question, so no menu was shown.\n\n**Provenance.** The preference check for the mode question returned AUTO_DECIDE. The recommendation was HOLD SCOPE because the draft is a fix with a hard \"outputs unchanged\" invariant and an estimated 2 to 3 changed files. The audit record was written with the full mode name in both the user_choice and recommended fields, flagged as auto-decided, under this session's ID.\n\n**Deferred per your instructions:**\n- The Office Hours offer was not made, since no design doc exists and you said you would handle it separately.\n- The skill-start output carried a one-time setup prompt to add skill routing rules to CLAUDE.md. That is setup, and plan mode also forbids editing CLAUDE.md or committing, so it stays pending and will re-fire on the next healthy run.\n- The substantive review (0G HOLD SCOPE checks, 0I, Sections 1 to 11, report) has not started. No plan file was written and ExitPlanMode was not called, since the workflow is paused rather than complete.\n\n**For the later review, two facts to carry forward:** the listing code is not in this checkout, so the real codebase is needed for the existing-code and file-count checks. The web research points to code-point sorting with a stable tie-break and warns against locale collation, which is the main design question HOLD SCOPE should pin down.\n\nTelemetry is off for this install. No durable learnings this session.", + "timestamp": "2026-10-04T04:28:18.729Z" + } + ] + }, + "tools": [ + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:15.120Z", + "toolUseId": "toolu_01Mg7qvBkiqYfnxDHB3GGo38", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-skill-start --skill \"plan-ceo-review\" --model \"claude\" 2>&1", + "description": "Run gstack skill-start preamble" + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:16.495Z", + "toolUseId": "toolu_01Mg7qvBkiqYfnxDHB3GGo38", + "kind": "result", + "isError": false, + "content": "SKILL_START_PROTO: 1\nBRANCH: main\nPROACTIVE: true\nPROACTIVE_PROMPTED: yes\nSKILL_PREFIX: false\nREPO_MODE: unknown\nSESSION_KIND: interactive\nCONDUCTOR_SESSION: true\nACTIVATED: yes\nFIRST_LOOP_SHOWN: yes\nFIRST_TASK: \nLAKE_INTRO: yes\nTELEMETRY: off\nTEL_PROMPTED: yes\nSESSION_ID: 1581-1791087976-5a3dd6a3\nTEL_START: 1791087976\nEXPLAIN_LEVEL: default\nQUESTION_TUNING: true\nUPDATE_CHECK: false\nLEARNINGS: 0\nHAS_ROUTING: no\nROUTING_DECLINED: false\nVENDORED_GSTACK: no\nMODEL_OVERLAY: claude\nGSTACK_PLAN_MODE: active\nARTIFACTS_SYNC: off\n" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:29.021Z", + "toolUseId": "toolu_01218vLzt9Q42cBrvtbAoJBY", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-context-recovery 2>&1 | head -40", + "description": "Recover recent project context" + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:33.153Z", + "toolUseId": "toolu_01UnmMKxYmqwrawXsscjtSkg", + "kind": "use", + "name": "Bash", + "input": { + "command": "echo \"REMOTE: $(git remote get-url origin 2>&1)\"; echo \"HEAD_REF: $(git symbolic-ref refs/remotes/origin/HEAD 2>&1)\"; git rev-parse --verify origin/main >/dev/null 2>&1 && echo \"origin/main exists\" || echo \"no origin/main\"; echo \"---LOG\"; git log --oneline -30; echo \"---STASH\"; git stash list; echo \"---TODO\"; grep -r \"TODO\\|FIXME\\|HACK\\|XXX\" -l --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=.git . | head -30; echo \"---RECENT\"; git log --since=30.days --name-only --format=\"\" | sort | uniq -c | sort -rn | head -20; echo \"---FILES\"; find . -path ./.git -prune -o -type f -print | head -50", + "description": "Audit git history, stashes, TODOs, and repo files" + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:36.531Z", + "toolUseId": "toolu_01YXR5JPFMhnCgTYtFAgyiNm", + "kind": "use", + "name": "Bash", + "input": { + "command": "setopt +o nomatch 2>/dev/null || true; SLUG=$(~/.claude/skills/gstack/browse/bin/remote-slug 2>/dev/null || basename \"$(git rev-parse --show-toplevel 2>/dev/null || pwd)\"); BRANCH=$(git rev-parse --abbrev-ref HEAD 2>/dev/null | tr '/' '-' || echo 'no-branch'); DESIGN=$(~/.claude/skills/gstack/bin/gstack-design-doc-find \"$SLUG\" \"$BRANCH\"); [ -n \"$DESIGN\" ] && echo \"Design doc found: $DESIGN\" || echo \"No design doc found\"; echo \"SLUG=$SLUG BRANCH=$BRANCH\"", + "description": "Check for an office-hours design doc" + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:38.908Z", + "toolUseId": "toolu_01N1GRkthVykqYzPXR9aorYu", + "kind": "use", + "name": "Bash", + "input": { + "command": "_CROSS_PROJ=$(~/.claude/skills/gstack/bin/gstack-config get cross_project_learnings 2>/dev/null || echo \"unset\"); echo \"CROSS_PROJECT: $_CROSS_PROJ\"; if [ \"$_CROSS_PROJ\" = \"true\" ]; then ~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 --cross-project 2>&1; else ~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 2>&1; fi | head -30", + "description": "Load prior learnings and cross-project setting" + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:41.221Z", + "toolUseId": "toolu_011axfbcZwL2pxicmsUZVAWu", + "kind": "use", + "name": "Bash", + "input": { + "command": "SLUG=$(~/.claude/skills/gstack/bin/gstack-slug --get SLUG 2>/dev/null) || true; for d in product goals recent-decisions; do echo \"### $d\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get $d --project \"$SLUG\" 2>/dev/null || echo \"(no $d digest available yet)\"; done; echo \"### user-profile\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get user-profile 2>/dev/null || echo \"(no user-profile digest available yet)\"", + "description": "Load brain context digests" + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:42.974Z", + "toolUseId": "toolu_01JBffEz48oEirpACN9SnH9s", + "kind": "use", + "name": "Bash", + "input": { + "command": "printf '%s' \"Select plan-ceo-review mode: SCOPE EXPANSION, SELECTIVE EXPANSION, HOLD SCOPE, or SCOPE REDUCTION\" | ~/.claude/skills/gstack/bin/gstack-question-preference --check \"plan-ceo-review-mode\" --summary-stdin; echo \"EXIT=$?\"", + "description": "Check saved preference for the mode question" + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:47.973Z", + "toolUseId": "toolu_016L7vFz87m7ZsDqxAvPauBX", + "kind": "use", + "name": "Bash", + "input": { + "command": "_gs_d() { if command -v gtimeout >/dev/null; then gtimeout 30 \"$@\"; elif command -v timeout >/dev/null; then timeout 30 \"$@\"; elif command -v perl >/dev/null; then perl -e 'alarm(shift);exec(@ARGV)' 30 \"$@\"; else return 125; fi; }\n_A=aside; command -v aside >/dev/null || _A=$(command -v ~/.local/bin/aside)\nif [ \"${GSTACK_SKIP_ASIDE:-}\" = \"1\" ] || [ -z \"$_A\" ]; then echo \"NEEDS_ASIDE\"; else _rc=0; _o=$(_gs_d \"$_A\" repl 'console.log(\"ASIDE_READY \" + pwd)' 2>&1) || _rc=$?; case \"$_rc\" in 124|142) echo \"ASIDE_TIMEOUT: probe deadline exceeded\" ;; 125) echo \"ASIDE_UNAVAILABLE: bounded probe unavailable\" ;; 0) if printf '%s\\n' \"$_o\" | grep -q '^ASIDE_READY '; then echo \"READY: $_A\"; else echo \"ASIDE_NOT_RUNNING: no readiness marker\"; fi ;; *) echo \"ASIDE_CLI_ERROR: exit $_rc\" ;; esac; fi", + "description": "Probe Aside browser readiness" + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:48.652Z", + "toolUseId": "toolu_0117QuNHnLaAq6Wrwyb89Tg6", + "kind": "use", + "name": "Read", + "input": { + "file_path": "/PLAN.md" + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:48.847Z", + "toolUseId": "toolu_01218vLzt9Q42cBrvtbAoJBY", + "kind": "result", + "isError": false, + "content": "--- RECENT ARTIFACTS ---\n{\"skill\":\"plan-ceo-review\",\"event\":\"started\",\"branch\":\"main\",\"session\":\"1581-1791087976-5a3dd6a3\",\"ts\":\"2026-10-04T04:26:16.515Z\"}\n--- END ARTIFACTS ---" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:48.919Z", + "toolUseId": "toolu_01UnmMKxYmqwrawXsscjtSkg", + "kind": "result", + "isError": false, + "content": "REMOTE: error: No such remote 'origin'\nHEAD_REF: fatal: ref refs/remotes/origin/HEAD is not a symbolic ref\norigin/main exists\n---LOG\nb2b85a4 Seed review plan\n--…[elided: not consumed]" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:48.968Z", + "toolUseId": "toolu_01YXR5JPFMhnCgTYtFAgyiNm", + "kind": "result", + "isError": false, + "content": "No design doc found\nSLUG=gstack-plan-count-dFsHyL BRANCH=main" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:49.042Z", + "toolUseId": "toolu_01N1GRkthVykqYzPXR9aorYu", + "kind": "result", + "isError": false, + "content": "CROSS_PROJECT: false" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:49.166Z", + "toolUseId": "toolu_011axfbcZwL2pxicmsUZVAWu", + "kind": "result", + "isError": false, + "content": "### product\n(no product digest available yet)\n### goals\n(no goals digest available yet)\n### recent-decisions\n(no recent-decisions digest available yet)\n### user-profile\n(no user-profile digest available yet)" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:49.255Z", + "toolUseId": "toolu_01JBffEz48oEirpACN9SnH9s", + "kind": "result", + "isError": false, + "content": "AUTO_DECIDE\nEXIT=0" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:49.275Z", + "toolUseId": "toolu_016L7vFz87m7ZsDqxAvPauBX", + "kind": "result", + "isError": false, + "content": "NEEDS_ASIDE" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:26:49.285Z", + "toolUseId": "toolu_0117QuNHnLaAq6Wrwyb89Tg6", + "kind": "result", + "isError": false, + "content": "1\t# Draft: deterministic skill-list ordering\n2\t\n3\tUsers compare skill listings in scripts and reviews. Make the existing listing\n4\tpath sort registered skill na…[elided: not consumed]" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:27:16.011Z", + "toolUseId": "toolu_014aXZByMUYWVbcvnXN2FzeT", + "kind": "use", + "name": "ToolSearch", + "input": { + "query": "select:WebSearch", + "max_results": 1 + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:27:16.014Z", + "toolUseId": "toolu_014aXZByMUYWVbcvnXN2FzeT", + "kind": "result", + "isError": false, + "content": [ + { + "type": "tool_reference", + "tool_name": "WebSearch" + } + ] + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:27:20.666Z", + "toolUseId": "toolu_011TpphSmoKtyTpDfg63n7YT", + "kind": "use", + "name": "WebSearch", + "input": { + "query": "deterministic CLI list output ordering locale-independent sort reproducible output directory enumeration order" + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:27:49.348Z", + "toolUseId": "toolu_011TpphSmoKtyTpDfg63n7YT", + "kind": "result", + "isError": false, + "content": "Web search results for query: \"deterministic CLI list output ordering locale-independent sort reproducible output directory enumeration order\"\n\nLinks: [{\"title\"…[elided: not consumed]" + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:28:06.342Z", + "toolUseId": "toolu_01HBCWdbfoSzNmxSuZJh4EJ9", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-question-log '{\"skill\":\"plan-ceo-review\",\"question_id\":\"plan-ceo-review-mode\",\"question_summary\":\"Select plan-ceo-review mode for deterministic skill-list ordering draft\",\"category\":\"routing\",\"door_type\":\"two-way\",\"options_count\":4,\"user_choice\":\"HOLD SCOPE\",\"recommended\":\"HOLD SCOPE\",\"auto_decided\":true,\"session_id\":\"1581-1791087976-5a3dd6a3\"}'; echo \"EXIT=$?\"", + "description": "Log the auto-decided mode selection" + } + }, + { + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "timestamp": "2026-10-04T04:28:07.389Z", + "toolUseId": "toolu_01HBCWdbfoSzNmxSuZJh4EJ9", + "kind": "result", + "isError": false, + "content": "EXIT=0" + } + ], + "options": { + "skillName": "plan-ceo-review", + "sessionId": "35d92383-22c2-4e81-b785-477bf92c8a95", + "commandStartedAt": 1791087967239, + "now": 1791088108874, + "proseQuestionObserved": false, + "stateEvidence": { + "questionId": "plan-ceo-review-mode", + "preference": "never-ask", + "records": [ + { + "skill": "plan-ceo-review", + "question_id": "plan-ceo-review-mode", + "question_summary": "Select plan-ceo-review mode for deterministic skill-list ordering draft", + "category": "routing", + "door_type": "two-way", + "options_count": 4, + "user_choice": "HOLD SCOPE", + "recommended": "HOLD SCOPE", + "auto_decided": true, + "session_id": "1581-1791087976-5a3dd6a3", + "source": "agent", + "ts": "2026-10-04T04:28:07.389Z" + } + ] + } + } + }, + "screen": { + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37176837432-1-eval-slices-20/plan-ceo-review-1791088108875-8x79js/terminal.visible.log", + "tail_chars": 6000, + "text": "d handoff line, and\r Office Hours, setup, and the substantive review are deferred. I'll wait for your slash command before taking any\r action.\r\r✻ Crunched for 4s · done 4:26 AM\r❯ /plan-ceo-review \r\r●I'll run the skill preamble first, then do th pre-reviw audit and Step 0 up through the ode-slection question,\rstoppingthereperyourinstruction.\r Read 1 file,ran 8 shll commands \r● Aside is not installed, so the landscape check falls back to one read-only WebSearch query.\r●Web Search(\"deterministic CLI listoutput ordring locale-inependent sortreproducible output diroryenumeration\r ordr\")\r ⎿  Did1searchin29s\r\r●TherepoonlyhasPLAN.md/CLAUDE.mdwithnopriorcontext,andtheactuallistingcodeisn'tinthischeckout,soI\r confirmed via web researchstandard fix: sort entriesbycode point/btes (not locale) withastabletie-breakto\ravoidreaddir-orderfallback.I'llproceedinHOLDSCOPEmode,applyingmaxrigortojustthecomparator,case\r handling, tie-breaking, and the enumeration-order test across the ~2-3 affected files. · summary\r\r Runnng 1 shellcommand…\r✻Flibbertigibbeting… 6thought for 13s)\r✽\r✻\r●\r✶\r*\r✢9\r \r·\r\r● Received. I have the draft plan for deterministic skill-list ordering in context, along with your constraints: this\r invocation is for the review-mode decision only, the full mode name goes in the audit record and handoff line, and\r Office Hours, setup, and the substantive review are deferred. I'll wait for your slash command before taking any\raction.\r\r✻Crunched for 4s · done 4:26 AM\r\r❯ /plan-ceo-review \r●I'll run the skill preamble first,thendothepre-reviewauditandStep0upthroughthemode-selectionquestion,\rstoppingthereperyourinstruction.\r\rRead 1 file, ran 8 shell commands \r\r●Aside isnot installed,so he landscapecheckfallsbacktooneread-onlyWebSearchquery.\r●Web Search(\"deterministic CLI listoutputorderinglocale-independentsortreproducibleoutputdirectoryenumeration\rorder\")\r ⎿  Did 1search in 29s\r●The repo only has PLAN.md/AUDE.md with n prior context, and the actual listing ode isn't in this checkot, so I\rconfirmedvia webresearchthestandardfix:sortentriesbycodepoint/bytes(notlocale)withastabletie-breakto\r avoid readdi-order fallback.I'llproceedinHOLDSCOPEmode,applyingmaxrigortojustthecomparator,case\rhandling,tie-breaking,andtheenumeration-ordertestacrossthe~2-3affectedfiles. · summary\r\r Runng 1shell command…\r ⎿ $ ~/.claude/skills/gstack/bin/gstack-question-log \r '{\"skill\":\"plan-ceo-review\",\"question_id\":\"plan-ceo-review-mode\",\"question_summary\":\"Select plan-ceo-review mode \rfor deterministic skill-list ordering \r draft\",\"category\":\"routing\",\"door_type\":\"two-way\",\"options_count\":4,\"user_choice\":\"HOLD SCOPE\",\"re…\r✢\r*\r●\r✶5\r✻20s · ↓ 6.5k tokens · thought for 13s)\r)\r✽…\rg\r \rLog theauto-decided modeselection\rFlibbertigibbeting…\r✻Flibbertigibbeting…\rFlibbertigibbeting…\r✶\r*\r●\r✢Flibbertigibbeting…1\rFlibbertigibbeting…\r·Flibbertigibbeting…\rFlibbertigibbeting…\rFlibbertigibbeting…\r✢Flibbertigibbeting…\r \rFlibbertigibbeting…\r*\r✶\r✻Flibbertigibbeting…2\r✽Flibbertigibbeting…\r●\rFlibbertigibbeting…\rFlibbertigibbeting…\rFlibbertigibbeting…\r✻Flibbertigibbeting…\rFlibbertigibbeting…\rrti\r✶ei\r \r*\r✢bt3\r·br\rie\r · thinking)\r●\rthinking\r✢lbthinking\rthinking\r*Fb\r✶\r✻i4thinking\r \r✽lthinking\rthinking\rFthinking\rthinking\r✻thinking\r●\r✶thinking\r*\r✢5thinking\r·thinking\r \rthinking\rthinking\rthinking\r✢thinking\r*thinking\r●\r✶\r✻6thinking\r✽thinking\rthinking\rthinking\r \rthinking\r✻thinking\r✶thinking\r*\r✢7thinking\r●\r·thinking\rthinking\rthinking\rthinking\r✢thinking\r \r*thinking\r✶\r…\r✻8thinking\r✽gthinking\rthinking\r●\rnthinking\rthinking\r✻i…thinking\r✶tgthinking\r*\r \r✢en\r9thinking\r·bithinking\rthinking\rbtthinking\r●\rthinking\r✢iethinking\r*gbthinking\r✶\r✻ib\r10s · ↓ 6.5k tokens · thinking)\r✽tithinking\rthinking\rrgthinking\reithinking\r✻thinking\r \r✶btthinking\r*\r✢br\r1thinking\r·iethinking\rthinking\r●\rlbthinking\r✢Fbthinking\rthinking\r*ithinking\r✶\r●Web Search(\"deterministicCLIlistoutputorderinglocale-independentsortreproducibleoutputdirectoryenumeration\r order\")\r ⎿  Did 1 search in 29s\r\r●The repoonlyhasPLAN.md/CLAUDE.mdwithnopriorcontext,andtheactuallistingcodeisn'tinthischeckout,soI\rconfirmedviawebresearchthestandardfix:sortentriesbycodepoint/bytes(notlocale)withastabletie-breakto\r avoid readdir-order fallback. I'llproceedinHOLDSCOPEmode,applyingmaxrigortojustthecomparator,case\rhandling,tie-breaking,andtheenumeration-ordertestacrossthe~2-3affectedfiles. · summary\r\rRan 1 shell command \r\r●The modedecision is recorded. HOLD SCOPEwasauto-selectedfromyoursavedpreferenceforthemodequestion,sono\rmenuwasshown.\r\rProvenance.ThepreferencecheckforthemodequestionreturnedAUTO_DECIDE.TherecommendationwasHOLDSCOPEbecause\r the draft isa fix witha hard\"outputs unchanged\" invriantadan estimated 2 to 3changedfiles.Theauditrecord\rwaswrittenwiththefullmodenameinboththeuser_choiceandrecommendedfields,flaggedasauto-decided,under\r this session's ID.\r\r Deferred per your instructions:\r-TheOfficeHoursofferwasnotmade,sincenodesigndocexistsandyousaidyouwouldhandleitseparately.\r - The skill-startoutput carried a one-timesetupprompt to ad skill routing rules toCLAUDE.md. That is seup, and\r plan mode lso forbids editing CLAUDE.md or commiting,soit stays pendingad will r-fire onthe nx healthy\r run.\r- The subsantiv review (0G HOLD SCOPE checks, 0I, Section 1 to11, report) has not started. Noplanfilewas\rwrittenandExitPlanModewasnotcalled,sincetheworkflowispausedratherthancomplete.\r\r For the later review, two facts to carry forward: thelistingcodeisnotinthischeckout,sotherealcodebaseis\rneeded for the existing-code and file-count checks. The web research points to code-point sorting with a stable\rtie-break and warns against locale collation,whichisthemaindesignquestionHOLDSCOPEshouldpindown.\r\rTelemetryisoffforthisinstall.Nodurablelearningsthissession.\r\r✻ Cogitated for 2m11s · done 4:28 AM\r❯ \r← for agents" + } + } +} diff --git a/test/fixtures/detector-corpora/auto-decide-preserved/37179171083-t1-fail.json b/test/fixtures/detector-corpora/auto-decide-preserved/37179171083-t1-fail.json new file mode 100644 index 000000000..7f7ee0c09 --- /dev/null +++ b/test/fixtures/detector-corpora/auto-decide-preserved/37179171083-t1-fail.json @@ -0,0 +1,287 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "auto-decide-preserved", + "detector": "test/helpers/native-auto-decide.ts#findNativeAutoDecision", + "screen_detector": "test/helpers/pty/classify.ts#classifyVisible", + "series_identity": "6c9ea4bac7c0bc81", + "source": { + "run": "37179171083", + "sha": "db91607a7fd9419592eb63e036149920304e32b5", + "lane": "periodic", + "artifact": "paid-slice-20-a1", + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37179171083-1-eval-slices-20/plan-ceo-review-1791091075073-iTnHpm/observation.json" + }, + "census_outcome": "failed", + "census_error": "AUTO_DECIDE preserve test inconclusive: outcome=timeout", + "expected": "refuse", + "justification": "Genuine product failure on db91607, same class as 37176837432: preference checked and HOLD SCOPE logged, but the required \"Auto-decided review mode → …\" line was never sent (\"Mode decision done. HOLD SCOPE is selected and recorded.\"); repaired in the product by e3546d9.", + "screen_expected": "refuse", + "native_expected": "refuse", + "derived": "options.stateEvidence rebuilt from the session's gstack-question-log argument with source \"agent\" and ts = that call's tool-result timestamp (the bound question-log.jsonl itself is not archived)", + "input": { + "native": { + "transcript": { + "status": "ready", + "calls": [], + "assistantMessages": [ + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "text": "Received. I have the draft plan for deterministic skill-list ordering as context, along with your constraints: review-mode decision only, full mode name in the audit record and handoff line, no Office Hours or setup in this invocation, substantive review deferred.\n\nWaiting for your slash command.", + "timestamp": "2026-10-04T05:14:54.505Z" + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "text": "Starting the plan-ceo-review preamble, then the pre-review audit, aiming to stop at the mode-selection decision as you asked.", + "timestamp": "2026-10-04T05:15:10.342Z" + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "text": "I've auto-selected HOLD SCOPE review mode based on your preference, since this is a small, deterministic fix (2-3 files) that benefits from maximum rigor around collation, locale handling, and alias sorting rather than broader scope. Proceeding with the review under that mode.\n\n", + "timestamp": "2026-10-04T05:17:02.152Z" + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "text": "Mode decision done. HOLD SCOPE is selected and recorded. The review itself is paused as you asked.\n\n**What the audit found**\n\n- Repo holds only the plan and CLAUDE.md. The listing code is not in this checkout, so code-level leverage must be confirmed against the real codebase when the review runs.\n- No remote, so the base branch fell back to main. No stashes, no TODO markers, no TODOS.md, no design doc, no prior learnings, cold brain digests.\n- Landscape check ran through WebSearch since Aside is not installed. It backs the draft: treat enumeration order as random, sort at the source with a locale-independent key, and test under deliberately shuffled order. One first-principles point for the review: sorting only at render leaves error messages and alias resolution order-dependent.\n\n**Records written**\n\n- Question log for plan-ceo-review-mode with user_choice and recommended both set to the full name HOLD SCOPE, marked auto-decided.\n- Checkpoint written to the host plan file with audit evidence, mode provenance, an empty ledger, and open items for the substantive review (collation rule, alias placement, sort location, test design).\n\n**Deferred or skipped**\n\n- Office Hours offer skipped per your instruction.\n- Skill-start fired a one-time routing-rules onboarding block asking to edit and commit CLAUDE.md. Plan mode forbids that write and you said setup is separate, so I left it. It will re-fire on the next healthy run.\n- Telemetry end call not run because the workflow is paused, not complete. Telemetry is off in this install anyway.\n- ExitPlanMode not called: no review report exists yet.\n\nNo durable learnings this session.\n\nSTATUS: DONE for the requested mode-only scope. Next step when you are ready: re-invoke the skill and resume at 0G on the HOLD SCOPE route using the checkpoint.\n\nSources: [Reproducible Builds in Language Package Managers](https://nesbitt.io/2026/02/24/reproducible-builds-in-language-package-managers.html), [numpy PR #11163](https://github.com/numpy/numpy/pull/11163), [ring issue #1625](https://github.com/briansmith/ring/issues/1625), [rackbops-bot-plugins issue #33](https://github.com/Rackbops/rackbops-bot-plugins/issues/33), [Reproducible Builds Feb 2020 report](https://reproducible-builds.org/reports/2020-02/), [Wikipedia: Reproducible builds](https://en.wikipedia.org/wiki/Reproducible_builds)", + "timestamp": "2026-10-04T05:17:43.570Z" + } + ] + }, + "tools": [ + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:11.408Z", + "toolUseId": "toolu_01JM6EEHRixJ9hJkLTEVvd1Q", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-skill-start --skill \"plan-ceo-review\" --model \"claude\"", + "description": "Run gstack skill-start preamble" + } + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:11.918Z", + "toolUseId": "toolu_01WS6sPjwunh5jFyVBzbFqE1", + "kind": "use", + "name": "ToolSearch", + "input": { + "query": "AskUserQuestion", + "max_results": 3 + } + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:12.703Z", + "toolUseId": "toolu_01JM6EEHRixJ9hJkLTEVvd1Q", + "kind": "result", + "isError": false, + "content": "SKILL_START_PROTO: 1\nBRANCH: main\nPROACTIVE: true\nPROACTIVE_PROMPTED: yes\nSKILL_PREFIX: false\nREPO_MODE: unknown\nSESSION_KIND: interactive\nCONDUCTOR_SESSION: true\nACTIVATED: yes\nFIRST_LOOP_SHOWN: yes\nFIRST_TASK: \nLAKE_INTRO: yes\nTELEMETRY: off\nTEL_PROMPTED: yes\nSESSION_ID: 1508-1791090912-5f01b34e\nTEL_START: 1791090912\nEXPLAIN_LEVEL: default\nQUESTION_TUNING: true\nUPDATE_CHECK: false\nLEARNINGS: 0\nHAS_ROUTING: no\nROUTING_DECLINED: false\nVENDORED_GSTACK: no\nMODEL_OVERLAY: claude\nGSTACK_PLAN_MODE: active\nARTIFACTS_SYNC: off\n" + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:12.708Z", + "toolUseId": "toolu_01WS6sPjwunh5jFyVBzbFqE1", + "kind": "result", + "isError": false, + "content": [ + { + "type": "tool_reference", + "tool_name": "EnterPlanMode" + }, + { + "type": "tool_reference", + "tool_name": "ExitPlanMode" + } + ] + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:25.397Z", + "toolUseId": "toolu_015CkgcFD3swVMSxpWemrYC1", + "kind": "use", + "name": "Read", + "input": { + "file_path": "/PLAN.md" + } + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:25.408Z", + "toolUseId": "toolu_015CkgcFD3swVMSxpWemrYC1", + "kind": "result", + "isError": false, + "content": "1\t# Draft: deterministic skill-list ordering\n2\t\n3\tUsers compare skill listings in scripts and reviews. Make the existing listing\n4\tpath sort registered skill na…[elided: not consumed]" + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:29.471Z", + "toolUseId": "toolu_01F9AYrD9At2os1qR5hWdVrw", + "kind": "use", + "name": "Bash", + "input": { + "command": "echo \"== remote\"; git remote get-url origin 2>&1; echo \"== default head\"; git symbolic-ref refs/remotes/origin/HEAD 2>&1; git rev-parse --verify origin/main 2>&1; echo \"== log\"; git log --oneline -30; echo \"== stash\"; git stash list; echo \"== todo grep\"; grep -r \"TODO\\|FIXME\\|HACK\\|XXX\" -l --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=.git . | head -30; echo \"== recent files\"; git log --since=30.days --name-only --format=\"\" | sort | uniq -c | sort -rn | head -20; echo \"== tree\"; find . -path ./.git -prune -o -type f -print | head -50; echo \"== TODOS.md\"; cat TODOS.md 2>&1 | head -50", + "description": "Audit repo history, stashes, TODOs, and file tree" + } + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:35.230Z", + "toolUseId": "toolu_015SqWN1yVoxozN68Lz78xHz", + "kind": "use", + "name": "Bash", + "input": { + "command": "echo \"== context recovery\"; ~/.claude/skills/gstack/bin/gstack-context-recovery 2>&1 | head -40; echo \"== design doc\"; SLUG=$(~/.claude/skills/gstack/browse/bin/remote-slug 2>/dev/null || basename \"$(git rev-parse --show-toplevel 2>/dev/null || pwd)\"); BRANCH=$(git rev-parse --abbrev-ref HEAD 2>/dev/null | tr '/' '-' || echo 'no-branch'); DESIGN=$(~/.claude/skills/gstack/bin/gstack-design-doc-find \"$SLUG\" \"$BRANCH\"); [ -n \"$DESIGN\" ] && echo \"Design doc found: $DESIGN\" || echo \"No design doc found\"; echo \"== learnings\"; _CROSS_PROJ=$(~/.claude/skills/gstack/bin/gstack-config get cross_project_learnings 2>/dev/null || echo \"unset\"); echo \"CROSS_PROJECT: $_CROSS_PROJ\"; ~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 2>&1 | head -20; echo \"== mode pref check\"; printf '%s' \"Select plan-ceo-review mode: SCOPE EXPANSION / SELECTIVE EXPANSION / HOLD SCOPE / SCOPE REDUCTION\" | ~/.claude/skills/gstack/bin/gstack-question-preference --check \"plan-ceo-review-mode\" --summary-stdin; echo \"exit=$?\"", + "description": "Run context recovery, design doc check, learnings search, and mode preference check" + } + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:41.413Z", + "toolUseId": "toolu_01NC2gzVAjeivYu3VxduNfyn", + "kind": "use", + "name": "Bash", + "input": { + "command": "SLUG=$(~/.claude/skills/gstack/bin/gstack-slug --get SLUG 2>/dev/null) || true; echo \"SLUG=$SLUG\"; for d in product goals recent-decisions; do echo \"### $d\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get $d --project \"$SLUG\" 2>/dev/null || echo \"(no $d digest available yet)\"; done; echo \"### user-profile\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get user-profile 2>/dev/null || echo \"(no user-profile digest available yet)\"; echo \"== aside probe\"; _gs_d() { if command -v gtimeout >/dev/null; then gtimeout 30 \"$@\"; elif command -v timeout >/dev/null; then timeout 30 \"$@\"; else return 125; fi; }; _A=aside; command -v aside >/dev/null || _A=$(command -v ~/.local/bin/aside); if [ \"${GSTACK_SKIP_ASIDE:-}\" = \"1\" ] || [ -z \"$_A\" ]; then echo \"NEEDS_ASIDE\"; else _rc=0; _o=$(_gs_d \"$_A\" repl 'console.log(\"ASIDE_READY \" + pwd)' 2>&1) || _rc=$?; case \"$_rc\" in 124|142) echo \"ASIDE_TIMEOUT\";; 125) echo \"ASIDE_UNAVAILABLE\";; 0) printf '%s\\n' \"$_o\" | grep -q '^ASIDE_READY ' && echo \"READY: $_A\" || echo \"ASIDE_NOT_RUNNING\";; *) echo \"ASIDE_CLI_ERROR: exit $_rc\";; esac; fi", + "description": "Load brain context digests and probe Aside readiness" + } + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:42.305Z", + "toolUseId": "toolu_01F9AYrD9At2os1qR5hWdVrw", + "kind": "result", + "isError": false, + "content": "== remote\nerror: No such remote 'origin'\n== default head\nfatal: ref refs/remotes/origin/HEAD is not a symbolic ref\n1978064afd86fb1e749952a80b0c29e99b533f42\n== l…[elided: not consumed]" + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:42.461Z", + "toolUseId": "toolu_015SqWN1yVoxozN68Lz78xHz", + "kind": "result", + "isError": false, + "content": "== context recovery\n--- RECENT ARTIFACTS ---\n{\"skill\":\"plan-ceo-review\",\"event\":\"started\",\"branch\":\"main\",\"session\":\"1508-1791090912-5f01b34e\",\"ts\":\"2026-10-04T05:15:12.719Z\"}\n--- END ARTIFACTS ---\n== design doc\nNo design doc found\n== learnings\nCROSS_PROJECT: false\n== mode pref check\nAUTO_DECIDE\nexit=0" + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:15:42.567Z", + "toolUseId": "toolu_01NC2gzVAjeivYu3VxduNfyn", + "kind": "result", + "isError": false, + "content": "SLUG=gstack-plan-count-gv6Swx\n### product\n(no product digest available yet)\n### goals\n(no goals digest available yet)\n### recent-decisions\n(no recent-decisions digest available yet)\n### user-profile\n(no user-profile digest available yet)\n== aside probe\nNEEDS_ASIDE" + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:16:17.058Z", + "toolUseId": "toolu_01R7jRwK4m3wtUhSzhd95fvr", + "kind": "use", + "name": "ToolSearch", + "input": { + "query": "select:WebSearch", + "max_results": 1 + } + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:16:17.061Z", + "toolUseId": "toolu_01R7jRwK4m3wtUhSzhd95fvr", + "kind": "result", + "isError": false, + "content": [ + { + "type": "tool_reference", + "tool_name": "WebSearch" + } + ] + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:16:20.271Z", + "toolUseId": "toolu_01TYoem6fjyrShp16zM23PWV", + "kind": "use", + "name": "WebSearch", + "input": { + "query": "deterministic sorted output CLI list command reproducible builds why directory enumeration order differs" + } + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:16:46.000Z", + "toolUseId": "toolu_01TYoem6fjyrShp16zM23PWV", + "kind": "result", + "isError": false, + "content": "Web search results for query: \"deterministic sorted output CLI list command reproducible builds why directory enumeration order differs\"\n\nLinks: [{\"title\":\"fix:…[elided: not consumed]" + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:17:03.787Z", + "toolUseId": "toolu_01NwELSNphPQAK9f9HY1VPQm", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-question-log '{\"skill\":\"plan-ceo-review\",\"question_id\":\"plan-ceo-review-mode\",\"question_summary\":\"Select plan-ceo-review mode for deterministic skill-list ordering draft\",\"category\":\"routing\",\"door_type\":\"two-way\",\"options_count\":4,\"user_choice\":\"HOLD SCOPE\",\"recommended\":\"HOLD SCOPE\",\"auto_decided\":true,\"session_id\":\"1508-1791090912-5f01b34e\"}' 2>&1; echo \"exit=$?\"", + "description": "Record mode auto-decision in question audit log" + } + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:17:26.721Z", + "toolUseId": "toolu_01Hgk3mtPpjhAf9XsqYYf1rK", + "kind": "use", + "name": "Write", + "input": { + "file_path": "/plans/keep-this-draft-plan-agile-hamming.md", + "content": "# /plan-ceo-review checkpoint: deterministic skill-list ordering\n\nStatus: mode-only handoff. Substantive review (Sections 1–11, outputs, report) not yet run, per user request.\nSession: 1508-1791090912-5f01b34e | Branch: main | Base branch: main (no remote; git-native fallback) | Date: 2026-10-04\n\n## Context\n\nWorking plan under review: `PLAN.md` (draft \"deterministic skill-list ordering\"). Users compare\nskill listings in scripts and reviews, so enumeration-order noise shows up as spurious diffs.\nThe draft sorts registered skill names deterministically before rendering, holds membership,\naliases, metadata and text/JSON formats unchanged, and adds tests for mixed-case names and\ndiffering directory enumeration order.\n\n## Pre-review system audit\n\n- Repo contains only `PLAN.md` and `CLAUDE.md` (single commit \"Seed review plan\"). The target\n listing code is not in this checkout; code-level leverage (0B) must be verified against the\n real codebase before the substantive review.\n- No remote `origin`; base branch resolved to `main` via `origin/main` ref check fallback.\n- No stashes, no TODO/FIXME markers, no `TODOS.md`, no design doc for `main`.\n- Learnings: project-scoped (cross_project_learnings=false), none found. Brain digests cold.\n- Retrospective check: no prior reviews or reverts on record.\n- DESIGN_SCOPE: none (CLI/library output ordering only; Section 11 skip expected).\n\n## Landscape check (Layer 1/2/3)\n\n- Layer 1: treat `readdir` order as random; sort explicitly, at the source, every level.\n- Layer 2: reproducible-builds literature catalogs filesystem ordering as a standard\n nondeterminism source; recommends locale-independent (byte/codepoint) sort keys and testing\n under deliberately shuffled enumeration (disorderfs / reprotest pattern).\n- Layer 3: sorting only the final render leaves intermediate behaviour (error messages, alias\n resolution, duplicate detection) order-dependent. The review should ask whether sorting\n belongs at registration rather than at render.\n\n## Step 0 evidence\n\n- 0A Premise: real problem is diff noise in scripted/reviewed listings; do-nothing cost is\n flaky comparisons and false \"changed\" signals. Plan addresses the pain directly.\n- 0B Leverage: reuse the existing listing path; sort before render. Code not present here to\n confirm; implementation owner must identify the enumeration site.\n- 0C Dream state: all user-facing listings stable and comparable across machines and runs.\n This plan moves toward it for one command.\n- 0D: no required approach choice.\n\n## Mode selection (0E)\n\n- Estimated changed files: 2–3 (listing module + test file(s)); estimate.\n- Category: fix/refactor → recommendation HOLD SCOPE.\n- Resolution: `gstack-question-preference --check plan-ceo-review-mode` → AUTO_DECIDE (exit 0).\n- Selected mode: **HOLD SCOPE**. Provenance: successful preference check; logged\n `plan-ceo-review-mode`, user_choice=HOLD SCOPE, recommended=HOLD SCOPE, auto_decided=true.\n- Handoff sent in chat before tool calls.\n\n## Decision ledger\n\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| (none yet) | | | | | |\n\nMode selection is an admin question; no ledger row. No plan changes approved.\n\n## Deferred this invocation (user instruction)\n\n- Prerequisite `/office-hours` offer: user handles separately.\n- `routing-injection` onboarding block from skill-start: requires CLAUDE.md edit + commit;\n blocked by plan mode and deferred by user. Marker-based; will re-fire on next healthy run.\n- Remaining HOLD SCOPE route: 0G (complexity check, minimum changes, invariants) → 0I → Review\n Sections 1–10 (11 skip) → outputs → `## GSTACK REVIEW REPORT`.\n\n## Open items to raise in the substantive review\n\n- Collation: case-insensitive vs byte order for mixed-case names; tie-break when two names\n differ only by case; locale independence.\n- Alias placement relative to canonical names in text and JSON output.\n- Where the sort lives (registration vs render) and whether error/log paths stay order-dependent.\n- Test design for \"differing directory enumeration order\": inject a shuffled/reversed listing\n rather than depending on a real filesystem.\n" + } + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:17:27.586Z", + "toolUseId": "toolu_01NwELSNphPQAK9f9HY1VPQm", + "kind": "result", + "isError": false, + "content": "exit=0" + }, + { + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "timestamp": "2026-10-04T05:17:27.639Z", + "toolUseId": "toolu_01Hgk3mtPpjhAf9XsqYYf1rK", + "kind": "result", + "isError": false, + "content": "File created successfully at: /plans/keep-this-draft-plan-agile-…[elided: not consumed]" + } + ], + "options": { + "skillName": "plan-ceo-review", + "sessionId": "735e97e8-8071-4cfc-be78-5a97ffb6098d", + "commandStartedAt": 1791090894821, + "now": 1791091075071, + "proseQuestionObserved": false, + "stateEvidence": { + "questionId": "plan-ceo-review-mode", + "preference": "never-ask", + "records": [ + { + "skill": "plan-ceo-review", + "question_id": "plan-ceo-review-mode", + "question_summary": "Select plan-ceo-review mode for deterministic skill-list ordering draft", + "category": "routing", + "door_type": "two-way", + "options_count": 4, + "user_choice": "HOLD SCOPE", + "recommended": "HOLD SCOPE", + "auto_decided": true, + "session_id": "1508-1791090912-5f01b34e", + "source": "agent", + "ts": "2026-10-04T05:17:27.586Z" + } + ] + } + } + }, + "screen": { + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37179171083-1-eval-slices-20/plan-ceo-review-1791091075073-iTnHpm/terminal.visible.log", + "tail_chars": 6000, + "text": "in-language-package-managers.html),numpyPR#11163\rgithub.comnumpy/numpy/pull/11163), rng issue #1625 (https://github.com/briansmith/ring/issues/1625),\rrackbops-bo-plugins issue #33 (https://github.com/Rackbops/rackbo-bot-plgins/issues/33), [Reproducible BuildsFeb\rReproducible Builds Feb\r-RepoholdsonlytheplanandCLAUDE.md.Thelistingcodeisnotinthischeckout,socode-levelleveragemustbe\r confirme agains the real codebase when the revewruns.\r-N remote, so thebase branh fell back to main. Nostahes,noTODOmarkers,noTODOS.md,nodesigndoc,noprior\r leaningscld brain digests.\r-Landscape checkran through WebSearchsinceAsideisnotinstalled.Itbacksthedraft:treatenumerationorderas\r rom, sort atthesource with a locale-independentkey, and test under deliberately shuffled ord. One\rfirst-principles point forthe review: sorting only at renderlaves eror messages and alias resolutio\rorderdependent.\r\rRecords written\r\r-Questionlogforplan-ceo-review-modewithuser_choiceandrecommendedbothsettothefullnameHOLDSCOPE,marked\r ato-decided.\r-Checkpoint writtentothehostplanfilewithauditevidence,modeprovenance,anemptyledger,andopenitemsfor\r t substantive review (collaion rule, alasplacemnt, sortlcation, test design).\r\rDeferred or skipped\r\r-OfficeHoursofferskippedperyourinstruction.\rSkill-start fired a one-time outing-rules onboardingblockaskingtoeditandcommitCLAUDE.md.Planmodeforbids\r that write and you said setup is sepaat,so I left it. Itwill re-fire on the next healthy run.\r-Telemetry end callnot run because the workflowis paused, not complete.Telemtry is off inthisinstallanyway.\rExitPlanMode not called: o reviewreport exists yet.\r\rNodurablelearningsthissession.\r\rSTATUS:DONEfortherequestedmode-onlyscope.Nextstepwhenyouareready:re-invoketheskillandresumeat0Gon\rthe HOLD SCOPE outusing hecheckpoint.\r\rSources:ReproducibleBuildsinLanguagePackageManagers\r(https://nsbitt.o/2026/02/24/reproducible-builds-in-language-package-managers.html),numpyPR#11163\rgithub.comnumpy/numpy/pull/11163), rng issue #1625 (https://github.com/briansmith/ring/issues/1625),\rrackbops-bo-plugins issue #33 (https://github.com/Rackbops/rackbo-bot-plgins/issues/33), Reproducible BuildsFeb\r2020 report (https://reproducible-builds.org/reprts/2020-02/), [Wikipedia: Reproduciblelearnings,coldbraindigests.\rLandscape checkranthrough WebSearch sincAside is notnstalled. It backs the draft: treat enumeration order as\rradom, sort at he sourcewith locale-indepndnt key,andtestunderdeliberatelyshuffledorder.One\r first-principles pointfor te review: sorting onlya render leaveseror messages and aliasrolutin\rorder-dependent.\r\rRecords written\r\r-Question log forplan-ceo-review-modewithuser_choiceandrecommendedbothsettothefullnameHOLDSCOPE,marked\rauto-decided.\r- Checkpoint writtentothehostplanfilewithauditevidence,modeprovenance,anemptyledger,andopenitemsfor\rthesubstantivereview(collationrule,aliasplacement,sortlocation,testdesign).\r\rDeferred or skipped\r\r-Office Hours offer skipped per you instruction.\r-Skill-startfiredaone-timerouting-rulesonboardingblockaskingtoeditandcommitCLAUDE.md.Planmodeforbids\r that write and yousaidsetupisseparate,soIleftit.Itwillre-fireonthenexthealthyrun.\r-Telemetryendcallnotrunbecausetheworkflowispaused,notcomplete.Telemetryisoffinthisinstallanyway.\rExitPlanModenot called: no review report exstsyet.\r\rNo durable lerningsthsssion.\r\rSTATUS: DONE for he requested mod-only scope. Next stepwhenyouareready:re-invoketheskillandresumeat0Gon\rtheHOLDSCOPErouteusingthecheckpoint.\r\rSources:ReproducibleBuildsinLanguagePackageManagers\r(https//nesbitt.io/2026/02/24/reproducible-builds-in-language-packg-managers.html), numpy PR #11163\r(ttps://github.com/nmpy/numpy/pull/11163),ringissue#1625(https://github.com/briansmith/ring/issues/1625),\rrackbops-bot-pluginsissue#33(https://github.com/Rackbops/rackbops-bot-plugins/issues/33),ReproducibleBuildsFeb\r2020 report (https://reproucble-builds.org/reports/2020-02/),Wikipedia:Reproduciblebuilds\ren.wkipedia.org/wikiR_)\r\r·Metamorphosing… (2m 49s · ↓ 10.1k tokens)\r ⎿  Tip: Use /btw to ask a quick side question without interrupting Claude's current work-No emotes the base branch fellbacktomain.Nostashes,noTODOmarkers,noTODOS.md,nodesigndoc,noprior\r learnings, coldbrain digests.\r-Lscape checkranthrough WebSearch since Aside isnot installed. It backs the draft: treat enumationorderas\rrandom, sort at the sourcewith a locale-independent key, andtst unde deliberately shuffled order. Oe\rfirstprinciplespointforthereview:sortingonlyatrenderleaveserrormessagesandaliasresolution\rorder-dependent.\r\rRecords written\r\r-Qestion log forplan-ceo-review-modewithuser_choiceandrecommendedbothsettothefullnameHOLDSCOPE,marked\r auto-decided.\r-Cckpoint written to the hos plan file wthaudit vidence,mde provenance, an emptyledger,andopenitemsfor\rthesubstantivereview(collationrule,aliasplacement,sortlocation,testdesign).\r\rDeferred or skipped\r\rOffice Hours offer skipped pe your instruction.\r-Skill-start fired a one-time routing-ulsonboarding blockasking to edit and commit CLAUDE.md. Planmodeforbids\r that write and yousaid setup is separate, so Ileft it. It will re-fireon th next healthyrun.\rTelemetry end call not ru becausethe workflow is paused,notcomplete.Telemetryisoffinthisinstallanyway.\r-ExitPlanModenotcalled:noreviewreportexistsyet.\r\rNodurablelearningsthissession.\r\rSTATUS: DONE fo threquesedmode-only scope.Nextstepwhenyouareready:re-invoketheskillandresumeat0Gon\rtheHOLDSCOPErouteusingthecheckpoint.\r\rSources: Rproducble Builds in Language Package Managers\rnesbitt.io2026/02/24/reproducible-bulds-in-language-package-managers.html), numpy PR #11163\r(https://gihub.com/numpy/numpy/pull/11163), ring issue #1625 (htt://githb.com/briansmith/ring/issues/1625),\rrackbops-bot-plugins issue #33 (https://github.cm/Rackbops/rackbops-bot-plugins/issues/33), ReproducibleBuildsFeb\r2020 report (htts://reproducbl-builds.org/reports/2020-02/),Wikipedia:Reproduciblebuilds\r(https://en.wikipedia.org/wiki/Reproducible_builds)\r\r✻ Crunched for 2m 49s· done 5:17AM\r❯ \r← for agents" + } + } +} diff --git a/test/fixtures/detector-corpora/auto-decide-preserved/37182865432-t1-fail.json b/test/fixtures/detector-corpora/auto-decide-preserved/37182865432-t1-fail.json new file mode 100644 index 000000000..6fd82f061 --- /dev/null +++ b/test/fixtures/detector-corpora/auto-decide-preserved/37182865432-t1-fail.json @@ -0,0 +1,246 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "auto-decide-preserved", + "detector": "test/helpers/native-auto-decide.ts#findNativeAutoDecision", + "screen_detector": "test/helpers/pty/classify.ts#classifyVisible", + "series_identity": "e3bf31932dcbfd91", + "source": { + "run": "37182865432", + "sha": "44ef18cd0ee8bfb33b3812fbc7f012a2d52f6d9e", + "lane": "periodic", + "artifact": "paid-slice-20-a1", + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37182865432-1-eval-slices-20/plan-ceo-review-1791095565691-zAJYEo/observation.json" + }, + "census_outcome": "failed", + "census_error": "AUTO_DECIDE preserve test inconclusive: outcome=timeout", + "expected": "pass", + "justification": "Fixed false red: the model ran gstack-ceo-mode-handoff --auto (printing the AUTO_DECIDE line) before its provenance log, as Step 0E orders; the detector then required the handoff after the log. Repaired by d89dc32.", + "screen_expected": "refuse", + "native_expected": "pass", + "derived": "options.stateEvidence rebuilt from the session's gstack-question-log argument with source \"agent\" and ts = that call's tool-result timestamp (the bound question-log.jsonl itself is not archived)", + "input": { + "native": { + "transcript": { + "status": "ready", + "calls": [], + "assistantMessages": [ + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "text": "Received. I have the draft plan for deterministic skill-list ordering as context, along with your instructions: this is a mode-only handoff, no review mode has been selected yet, and the audit record's user_choice and recommended fields plus the mode handoff line should carry the full selected mode name rather than a menu letter. I'll wait for your slash command before starting anything.", + "timestamp": "2026-10-04T06:31:16.610Z" + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "text": "Starting the plan-ceo-review preamble now. Per your instructions, I'll scope this run to the mode decision only: skip the Office Hours offer and setup prompts, gather just enough plan context to recommend a mode, ask the four-mode question, then run the handoff and audit record.", + "timestamp": "2026-10-04T06:31:24.520Z" + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "text": "Preamble checks out, and with Conductor session and question tuning active, I'll check mode preference and send decision briefs as prose. I'm deferring the routing-rules setup since you'll handle it separately and plan mode blocks CLAUDE.md edits anyway—moving on to gather plan context.\n\n", + "timestamp": "2026-10-04T06:31:37.194Z" + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "text": "I've gathered context: only the plan fixture exists in the repo, no remote, so the base branch defaults to `main`, and the saved preference (AUTO_DECIDE) means I'll go with my recommendation of HOLD SCOPE. Running the mode handoff now.\n\n", + "timestamp": "2026-10-04T06:32:05.754Z" + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "text": "I'm defaulting to HOLD SCOPE review mode since this is a small, tightly-invariant fix (deterministic sort on an existing listing path, ~1-3 files) with no new approach needed—focus will be on sort correctness (case handling, tie-breaking, enumeration-order tests). Recording provenance and wrapping up this invocation.\n\n", + "timestamp": "2026-10-04T06:32:19.720Z" + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "text": "**Mode decided: HOLD SCOPE.** The substantive review has not started, per your request.\n\nWhat this run did:\n- **Preamble** ran healthy. Conductor session, interactive, question tuning on.\n- **Mode selection** resolved by your saved preference for the mode question, which returned AUTO_DECIDE, so no question was asked. The handoff command printed the auto-decided line above.\n- **Audit record** logged with question id `plan-ceo-review-mode`, user_choice and recommended both set to the full name HOLD SCOPE, auto_decided true.\n- **Telemetry** recorded with outcome abort, since the review was intentionally stopped after the mode step.\n\nWhat this run skipped, by your instruction:\n- The Office Hours offer. No design doc exists for this branch, so it would normally fire.\n- The routing-rules setup prompt from skill-start. It is marker-based and will fire again on the next healthy run. Plan mode would also have blocked the CLAUDE.md edit and commit.\n- Review Sections 1 to 11, the plan file report, and ExitPlanMode. Nothing was written to the plan file.\n\nWhen you resume, HOLD SCOPE routes through 0G complexity check, 0I temporal interrogation, then the full review sections. Points worth pressing then: byte-order versus case-insensitive sorting for mixed-case names, tie-breaking for names that differ only by case, and whether the tests truly vary directory enumeration order instead of relying on the filesystem.\n\nSTATUS: DONE\nNo durable learnings this session.", + "timestamp": "2026-10-04T06:32:34.635Z" + } + ] + }, + "tools": [ + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:31:25.439Z", + "toolUseId": "toolu_01L7uumE3CicSqLq5YF1knYj", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-skill-start --skill \"plan-ceo-review\" --model \"claude\"", + "description": "Run gstack skill-start preamble" + } + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:31:26.964Z", + "toolUseId": "toolu_01L7uumE3CicSqLq5YF1knYj", + "kind": "result", + "isError": false, + "content": "SKILL_START_PROTO: 1\nBRANCH: main\nPROACTIVE: true\nPROACTIVE_PROMPTED: yes\nSKILL_PREFIX: false\nREPO_MODE: unknown\nSESSION_KIND: interactive\nCONDUCTOR_SESSION: true\nACTIVATED: yes\nFIRST_LOOP_SHOWN: yes\nFIRST_TASK: \nLAKE_INTRO: yes\nTELEMETRY: off\nTEL_PROMPTED: yes\nSESSION_ID: 1582-1791095486-11f777bc\nTEL_START: 1791095486\nEXPLAIN_LEVEL: default\nQUESTION_TUNING: true\nUPDATE_CHECK: false\nLEARNINGS: 0\nHAS_ROUTING: no\nROUTING_DECLINED: false\nVENDORED_GSTACK: no\nMODEL_OVERLAY: claude\nGSTACK_PLAN_MODE: active\nARTIFACTS_SYNC: off\n" + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:31:37.196Z", + "toolUseId": "toolu_01WANu5n8ZmUR9vrNDrNSBwU", + "kind": "use", + "name": "Read", + "input": { + "file_path": "/PLAN.md" + } + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:31:37.215Z", + "toolUseId": "toolu_01WANu5n8ZmUR9vrNDrNSBwU", + "kind": "result", + "isError": false, + "content": "1\t# Draft: deterministic skill-list ordering\n2\t\n3\tUsers compare skill listings in scripts and reviews. Make the existing listing\n4\tpath sort registered skill na…[elided: not consumed]" + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:31:39.661Z", + "toolUseId": "toolu_016Dk1tbvmvWJFZPCZbBKtx5", + "kind": "use", + "name": "Bash", + "input": { + "command": "echo \"REMOTE: $(git remote get-url origin 2>&1)\"; echo \"HEAD_REF: $(git symbolic-ref refs/remotes/origin/HEAD 2>&1)\"; echo \"--- log\"; git log --oneline -30; echo \"--- stash\"; git stash list; echo \"--- files\"; ls -la; echo \"--- todos\"; grep -rn \"TODO\\|FIXME\\|HACK\\|XXX\" -l --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=.git . | head -30; echo \"--- TODOS.md\"; cat TODOS.md 2>/dev/null || echo \"(none)\"", + "description": "Audit repo state, remote, history and TODOs" + } + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:31:41.663Z", + "toolUseId": "toolu_0153M6xQkpMgLtck4F849Qwh", + "kind": "use", + "name": "Bash", + "input": { + "command": "printf '%s' \"Select the plan-ceo-review mode for the deterministic skill-list ordering plan: SCOPE EXPANSION, SELECTIVE EXPANSION, HOLD SCOPE, or SCOPE REDUCTION\" | ~/.claude/skills/gstack/bin/gstack-question-preference --check \"plan-ceo-review-mode\" --summary-stdin; echo \"PREF_EXIT: $?\"", + "description": "Check saved preference for the mode question" + } + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:31:44.535Z", + "toolUseId": "toolu_01QoUodomJaBeM9zZB5VKzoN", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-context-recovery 2>&1 | head -40; echo \"--- design doc\"; SLUG=$(~/.claude/skills/gstack/browse/bin/remote-slug 2>/dev/null || basename \"$(git -C rev-parse --show-toplevel 2>/dev/null || pwd)\"); BRANCH=main; DESIGN=$(~/.claude/skills/gstack/bin/gstack-design-doc-find \"$SLUG\" \"$BRANCH\" 2>/dev/null); [ -n \"$DESIGN\" ] && echo \"Design doc found: $DESIGN\" || echo \"No design doc found\"", + "description": "Recover context and check for design doc" + } + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:31:45.415Z", + "toolUseId": "toolu_016Dk1tbvmvWJFZPCZbBKtx5", + "kind": "result", + "isError": false, + "content": "REMOTE: error: No such remote 'origin'\nHEAD_REF: fatal: ref refs/remotes/origin/HEAD is not a symbolic ref\n--- log\n6e5f800 Seed review plan\n--- stash\n--- files\n…[elided: not consumed]" + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:31:45.505Z", + "toolUseId": "toolu_0153M6xQkpMgLtck4F849Qwh", + "kind": "result", + "isError": false, + "content": "AUTO_DECIDE\nPREF_EXIT: 0" + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:31:45.590Z", + "toolUseId": "toolu_01QoUodomJaBeM9zZB5VKzoN", + "kind": "result", + "isError": false, + "content": "--- RECENT ARTIFACTS ---\n{\"skill\":\"plan-ceo-review\",\"event\":\"started\",\"branch\":\"main\",\"session\":\"1582-1791095486-11f777bc\",\"ts\":\"2026-10-04T06:31:26.989Z\"}\n--- END ARTIFACTS ---\n--- design doc\nNo design doc found" + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:32:05.755Z", + "toolUseId": "toolu_01JuPbmLq6ZmKbP3t91p9szw", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff \"HOLD SCOPE\" --decisions \"none\" --auto", + "description": "Run gstack CEO mode handoff for HOLD SCOPE" + } + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:32:11.420Z", + "toolUseId": "toolu_01JuPbmLq6ZmKbP3t91p9szw", + "kind": "result", + "isError": false, + "content": "Auto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune. Approved decisions: none." + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:32:20.892Z", + "toolUseId": "toolu_01PbxZacUu8Ph611URXMobAZ", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-question-log '{\"skill\":\"plan-ceo-review\",\"question_id\":\"plan-ceo-review-mode\",\"question_summary\":\"Select plan-ceo-review mode for deterministic skill-list ordering plan\",\"category\":\"routing\",\"door_type\":\"two-way\",\"options_count\":4,\"user_choice\":\"HOLD SCOPE\",\"recommended\":\"HOLD SCOPE\",\"auto_decided\":true,\"session_id\":\"1582-1791095486-11f777bc\"}'; echo \"QLOG_EXIT: $?\"", + "description": "Log auto-decided mode question to audit record" + } + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:32:22.625Z", + "toolUseId": "toolu_01RKKa2pmNcBXzN8VTEWWaM6", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-skill-end --skill \"plan-ceo-review\" --outcome abort --session-id \"1582-1791095486-11f777bc\" --tel-start \"1791095486\" --used-browse no --error-message \"\" --failed-step \"\" 2>/dev/null || true; echo \"TEL_EXIT: $?\"", + "description": "Run skill-end telemetry for the mode-only run" + } + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:32:23.792Z", + "toolUseId": "toolu_01PbxZacUu8Ph611URXMobAZ", + "kind": "result", + "isError": false, + "content": "QLOG_EXIT: 0" + }, + { + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "timestamp": "2026-10-04T06:32:23.918Z", + "toolUseId": "toolu_01RKKa2pmNcBXzN8VTEWWaM6", + "kind": "result", + "isError": false, + "content": "SKILL_END: recorded outcome=abort duration_s=57\nTEL_EXIT: 0" + } + ], + "options": { + "skillName": "plan-ceo-review", + "sessionId": "1d84eb71-de32-4e99-a62e-9046d0850c10", + "commandStartedAt": 1791095476944, + "now": 1791095565690, + "proseQuestionObserved": false, + "stateEvidence": { + "questionId": "plan-ceo-review-mode", + "preference": "never-ask", + "records": [ + { + "skill": "plan-ceo-review", + "question_id": "plan-ceo-review-mode", + "question_summary": "Select plan-ceo-review mode for deterministic skill-list ordering plan", + "category": "routing", + "door_type": "two-way", + "options_count": 4, + "user_choice": "HOLD SCOPE", + "recommended": "HOLD SCOPE", + "auto_decided": true, + "session_id": "1582-1791095486-11f777bc", + "source": "agent", + "ts": "2026-10-04T06:32:23.792Z" + } + ] + } + } + }, + "screen": { + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37182865432-1-eval-slices-20/plan-ceo-review-1791095565691-zAJYEo/terminal.visible.log", + "tail_chars": 6000, + "text": "g\r✢thinking\r*thinking\r \r✶4\r✻thinking\r✽45thinking\rthinking\rthinking\r●\r✻thinking\r✻Crunched for 4s · done 6:31 AM\r\r❯ /plan-ceo-review \r●Starting the plan-ceo-review preamblenow.Peryourinstructions,I'llscopethisruntothemodedecisiononly:skip\rtheOfficeHoursofferandsetupprompts,gatherjustenoughplancontexttorecommendamode,askthefour-mode\r question, then run the handoff and audit record.\r Ran 1 shell command \r\r●Pramble checks out, and with Couctor session andquestiontuningactive,I'llcheckmodepreferenceandsend\rdecisionbriefsasprose.I'mdeferringtherouting-rulessetupsinceyou'llhandleitseparatelyandplanmodeblocks\rCLAUDE.md edits anyway—movingontogatherplancontext. · summary\r Read 1 file, ran 3 shell commands \r\r●I've gathered context: only the plan fixture exists in the repo, noremote,sothebasebranchdefaultstomain,and\rthesavedpreference(AUTO_DECIDE)meansI'llgowithmyrecommendationofHOLDSCOPE.Runningthemodehandoffnow. \r· summary\r Ran 1 shell command \r\r●I'm defaultingtoHOLDSCOPEreviewmodesincethisisasmall,tightly-invariantfix(deterministicsortonan\rexistinglistingpath,~1-3files)withnonewapproachneeded—focuswillbeonsortcorrectness(casehandling,\rtie-breaking, enumeration-ordertests).Recordingprovenanceandwrappingupthisinvocation. · summary\r●Running 2 shell commands…\r ⎿ $ ~/.claude/skills/gstack/bin/gstack-question-log \r '{\"skill\":\"plan-ceo-review\",\"question_id\":\"plan-ceo-review-mode\",\"question_summary\":\"Select plan-ceo-review mode \rfor deterministic skill-list ordering \r plan\",\"category\":\"routing\",\"door_type\":\"two-way\",\"options_count\":4,\"user_choice\":\"HOLD SCOPE\",\"rec…\r6thinking\r✶\r*\r✢…7thinking\r \rthinking\r·g5thinking\rnthinking\rthought for 1s)\r✢i…\r●\r*sg\r✶\r✻un8\r✽ri\r6\r \r\r✻Crunched for 4s · done 6:31 AM\r\r❯ /plan-ceo-review \r\r●Starting the plan-ceo-review preamble now. Per your instrctions,I'll scopethis ru tothe mode decisin only:skip\rth Office Hours offerand setup prompts, gatherjustenoughplancontexttorecommendamode,askthefour-mode\rquestion,thenrunthehandoffandauditrecord.\r\rRan 1 shell command \r\r●Preamblecheckout, and withConductorsessin ad qestion tuning active, I'll checkmode prferece andsend\rdecision briefs prose. I'mdeferring the routing-rules setup sinceyou'llhandleitseparatelyandplanmodeblocks\rCLAUDE.mdeditsanyway—movingontogatherplancontext. · summary\r\rRead 1 file, ran 3 shell commands \r\r●I've gathered context: only the plan fixture exists in the repo, no remote, so the base branchdefaultsto main, and\rthe savedpreference(AUTO_DECIDE)meansI'llgowithmyrecommendationofHOLDSCOPE.Runningthemodehandoffnow. \r· summary\r\rRan 1 shell command \r\r●I'm defauting to HOLDSCOPE review mode sicethis is a small, tightly-invariant fix (deterministic sort on an\rexisting listing path, ~13 files) withno new approach needed—focus wllbeon ort correctness (case handling,\rtie-breaking,enumeration-ordertests).Recordingprovenanceandwrappingupthisinvocation. · summary\r\r Running 2 shell commands…\r ⎿ $ ~/.caude/skills/gstack/bin/gstack-skill-end -skill \"plan-co-review\" --otcome abort --session-id \r\"1582-1791095486-11f777bc\" --tel-start \"1791095486\" --used-browse no --error-message \"\" --failed-step \"\" \r2>/dev/null || true; echo \"TEL_EXIT: $?\res\r✻Pu\r✶r\r●\r*\r✢e\r·P\r7\rskill-end telemetryforthemode-onlyrun · 2s\rPerusing…\r)\rPerusing…\r\r \r✢Perusing…9\rPerusing…\r*\r✶\r✻Perusing…\r●\r✽Perusing…\rPerusing…8\rPerusing…\rPerusing…\r✻Perusing…40\r✶Perusing…\r \r*\r✢Perusing…\r·Perusing…\rPerusing…9\r●\rPerusing…\rPerusing…\rPerusing…\r✢\r*\r \r✶\r✻\r✽\r10s · ↓ 4.0k tokens)\r · thinking)\rthinking\rthinking\rthinking\r●\r✻thinking\r✶thinking\r*…\rg\r✢thinking\r \r·n1thinking\rthinking\ri…thinking\rthinking\r✢sgthinking\r*unthinking\r●\r✶\rri\r✻thinking\r✽es2thinking\rthinking\r \rPuthinking\rthinking\r✻rthinking\r✶ethinking\r*\r✢P\r●\rthinking\r·3thinking\rthinking\rthinking\rthinking\r \r✢thinking\r*thinking\r✶\r✻\rthinking\r●\r✽4thinking\rthinking\rthinking\rthinking\r✻thinking\r✶thinking\r \r*\r✢\rthinking\r·5thinking\rthinking\r●\rthinking\r✢thinking\rthinking\r*thinking\r✶\r✻\r \rthinking\r✽6thinking\rthinking\rthinking\r✻…thinking\r●\r✶gthinking\r*\r✢nthinking\r·i…7thinking\r \rsgthinking\rthinking\r✢unthinking\r*rithinking\r●\r✶\r✻esthinking\r✽Pu8thinking\r Ran 1 shell comman \r● I'm defaulting to HOLD SCOPE review mode since this is a small, tightly-invariant fix (deterministic sort on an\rexistinglistingpath,~1-3files)withnonewapproachneeded—focuswillbeonsortcorrectness(casehandling,\r tie-breaking, enumeration-orde tests). Recording provenane ad wrapping upinvocation. · summary\r\rRan 2 shell commands \r●Mode decided: HOLD SCOPE.Thesubstantivereviewhasnotstarted,peryourrequest.\r What this run did:\r- Preamble ran healthy. Conductor session, interactive, quesion tuning on.\r- Mode selection resolved by yur saved peferene for the mode question,whichreturnedAUTO_DECIDE,sonoquestion\rwasasked.Thehandoffcommandprintedtheauto-decidedlineabove.\r- Audit record logged with questionidplan-ceo-review-mode,user_choiceandrecommendedbothsettothefullname\rHOLDSCOPE,auto_decidedtrue.\r - Telemetry rerdd with oucome bort, sinc the review was intentionallytoppd fter the mode step.\r\rWhat thisrunskipped,byyourinstruction:\r-TheOfficeHoursoffer.Nodesigndocexistsforthisbranch,soitwouldnormallyfire.\r- The routing-rules setuppromptfromskill-start.Itismarker-basedandwillfireagainonthenexthealthyrun.\rPlanmodewouldalsohaveblockedtheCLAUDE.mdeditandcommit.\r - Review Sections1 to11, the planfilreport, and ExitPlnMode. Nothing was written to the plan file.\r\rWhn you resum, HOLD SCOPE routes hrough 0G complexity chck, 0I temporal interrogation, then the fullreview\rsections.Pointsworthpressingthen:byte-orderversuscase-insensitivesortingformixed-casenames,tie-breaking\r for name that diffr only by case,an whethe theteststrulyvarydirectoryenumerationorderinsteadofrelyingon\r the filesystem.\r\rSTATUS: DONE\rNodurablelearningsthissession.\r\r✻ Cogitated for 1m18s · done 6:32 AM\r❯ \r← for agents" + } + } +} diff --git a/test/fixtures/detector-corpora/auto-decide-preserved/37186854666-t1-pass.json b/test/fixtures/detector-corpora/auto-decide-preserved/37186854666-t1-pass.json new file mode 100644 index 000000000..195daa1dd --- /dev/null +++ b/test/fixtures/detector-corpora/auto-decide-preserved/37186854666-t1-pass.json @@ -0,0 +1,236 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "auto-decide-preserved", + "detector": "test/helpers/native-auto-decide.ts#findNativeAutoDecision", + "screen_detector": "test/helpers/pty/classify.ts#classifyVisible", + "series_identity": "7678e372b3345f9b", + "source": { + "run": "37186854666", + "sha": "b24b6d8fcf57db7e9c8ded236d31b8a5d0b4504a", + "lane": "periodic", + "artifact": "paid-slice-20-a1", + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37186854666-1-eval-slices-20/plan-ceo-review-1791100373205-nby65b/observation.json" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "classifyVisible accepted the sent AUTO_DECIDE line on screen. The native detector misread the template-driven \"**Mode application and rationale.**\" heading (its paragraph holds a colon) as an unfinished Mode field and withdrew the decision; latent false refusal fixed on this branch (modeField status stays inside one sentence).", + "screen_expected": "pass", + "native_expected": "pass", + "derived": "options.stateEvidence rebuilt from the session's gstack-question-log argument with source \"agent\" and ts = that call's tool-result timestamp (the bound question-log.jsonl itself is not archived)", + "input": { + "native": { + "transcript": { + "status": "ready", + "calls": [], + "assistantMessages": [ + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "text": "Received. I have the draft plan for deterministic skill-list ordering as context, along with your constraints: review-mode decision only, full mode name in the audit record and handoff line, no Office Hours or setup in this pass. I'll wait for your slash command before doing anything else.", + "timestamp": "2026-10-04T07:50:50.459Z" + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "text": "Running the plan-ceo-review preamble now, then I'll work toward the mode decision only, per your handoff instructions.", + "timestamp": "2026-10-04T07:50:56.145Z" + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "text": "Mode decision is made and logged. Writing a short Step 0 checkpoint to the plan file so the later substantive review can resume from it.", + "timestamp": "2026-10-04T07:52:09.428Z" + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "text": "Auto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune. Approved decisions: none.\n\n**Mode application and rationale.** The draft is a fix to an existing path with its own ceiling already pinned: membership, aliases, metadata and both output formats stay unchanged. That is the fix/refactor category, with an estimated two changed files, so the 0E rule recommends HOLD SCOPE. The review's value here is rigor on the sort rule, the enumeration-order test, and the empty and nil paths, not expansion. No new approach decision was needed.\n\n**Provenance.** Your stored preference for the mode question is `never-ask`, so the preference check returned AUTO_DECIDE and the recommendation was selected without a menu. The audit record is written and read back, with the full mode name in both fields as you asked:\n\n```\nquestion_id=plan-ceo-review-mode user_choice=HOLD SCOPE recommended=HOLD SCOPE\nauto_decided=true session_id=1531-1791100258-3092a930\n```\n\n**What I did not do, per your instructions.** I skipped the Office Hours prerequisite offer and the skill-start routing-rules setup prompt, since you said you will handle both separately. The routing setup would also need a CLAUDE.md edit and commit, which plan mode forbids. The design doc check found none for this branch.\n\n**Checkpoint written.** The host plan file now holds the system audit, 0A to 0C evidence, the decision ledger with the mode row, and the resume route. The substantive review resumes at 0G, then 0I, then the review sections. One item flagged for that review: the plan requires mixed-case coverage but does not name the sort-key rule, so case-sensitive byte order versus case-insensitive with a stable tiebreak is an open implementation choice.\n\nThe review workflow is paused rather than complete, so I did not run end-of-skill telemetry or call ExitPlanMode. No durable learnings this session.", + "timestamp": "2026-10-04T07:52:51.866Z" + } + ] + }, + "tools": [ + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:50:57.394Z", + "toolUseId": "toolu_01FLoFsJvRfG9k322WXnPabJ", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-skill-start --skill \"plan-ceo-review\" --model \"claude\"", + "description": "Run gstack skill-start preamble" + } + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:50:58.569Z", + "toolUseId": "toolu_01FLoFsJvRfG9k322WXnPabJ", + "kind": "result", + "isError": false, + "content": "SKILL_START_PROTO: 1\nBRANCH: main\nPROACTIVE: true\nPROACTIVE_PROMPTED: yes\nSKILL_PREFIX: false\nREPO_MODE: unknown\nSESSION_KIND: interactive\nCONDUCTOR_SESSION: true\nACTIVATED: yes\nFIRST_LOOP_SHOWN: yes\nFIRST_TASK: \nLAKE_INTRO: yes\nTELEMETRY: off\nTEL_PROMPTED: yes\nSESSION_ID: 1531-1791100258-3092a930\nTEL_START: 1791100258\nEXPLAIN_LEVEL: default\nQUESTION_TUNING: true\nUPDATE_CHECK: false\nLEARNINGS: 0\nHAS_ROUTING: no\nROUTING_DECLINED: false\nVENDORED_GSTACK: no\nMODEL_OVERLAY: claude\nGSTACK_PLAN_MODE: active\nARTIFACTS_SYNC: off\n" + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:51:12.270Z", + "toolUseId": "toolu_01HjrjJWSRmAohBKxcVp6X5s", + "kind": "use", + "name": "Bash", + "input": { + "command": "echo \"=== CONTEXT RECOVERY ===\" && ~/.claude/skills/gstack/bin/gstack-context-recovery 2>&1 | head -40; echo \"=== REMOTE ===\" && git remote get-url origin 2>&1; echo \"=== ORIGIN HEAD ===\" && git symbolic-ref refs/remotes/origin/HEAD 2>&1; git rev-parse --verify origin/main 2>&1; echo \"=== LOG ===\" && git log --oneline -30; echo \"=== STASH ===\" && git stash list; echo \"=== TODO GREP ===\" && grep -r \"TODO\\|FIXME\\|HACK\\|XXX\" -l --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=.git . | head -30; echo \"=== RECENT FILES ===\" && git log --since=30.days --name-only --format=\"\" | sort | uniq -c | sort -rn | head -20; echo \"=== FILES ===\" && ls -la", + "description": "Recover context and run pre-review system audit" + } + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:51:12.921Z", + "toolUseId": "toolu_01HjrjJWSRmAohBKxcVp6X5s", + "kind": "result", + "isError": false, + "content": "=== CONTEXT RECOVERY ===\n--- RECENT ARTIFACTS ---\n{\"skill\":\"plan-ceo-review\",\"event\":\"started\",\"branch\":\"main\",\"session\":\"1531-1791100258-3092a930\",\"ts\":\"2026-10-04T07:50:58.586Z\"}\n--- END ARTIFACTS ---\n=== REMOTE ===\nerror: No such remote 'origin'\n=== ORIGIN HEAD ===\nfatal: ref refs/remotes/origin/HEAD is not a symbolic ref\na2e4c625081ce011e4ffe4934c4a6635d82e1dfd\n=== LOG ===\na2e4c62 Seed review plan\n=== STASH ===\n=== TODO GREP ===\n=== RECENT FILES ===\n 1 PLAN.md\n 1 CLAUDE.md\n=== FILES ===\ntotal 20\ndrwx------ 3 runner runner 4096 Oct 4 07:50 .\ndrwxr-xr-x 7 runner runner 4096 Oct 4 07:50 ..\ndrwxr-xr-x 8 runner runner 4096 Oct 4 07:50 .git\n-rw-r--r-- 1 runner runner 1132 Oct 4 07:50 CLAUDE.md\n-rw-r--r-- 1 runner runner 826 Oct 4 07:50 PLAN.md" + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:51:16.440Z", + "toolUseId": "toolu_01PeQyE58RDjGSn36cTLCn67", + "kind": "use", + "name": "Read", + "input": { + "file_path": "/PLAN.md" + } + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:51:16.452Z", + "toolUseId": "toolu_01PeQyE58RDjGSn36cTLCn67", + "kind": "result", + "isError": false, + "content": "1\t# Draft: deterministic skill-list ordering\n2\t\n3\tUsers compare skill listings in scripts and reviews. Make the existing listing\n4\tpath sort registered skill na…[elided: not consumed]" + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:51:24.479Z", + "toolUseId": "toolu_019Yi6AfNvLiT21cAkR3KUs5", + "kind": "use", + "name": "Bash", + "input": { + "command": "echo \"=== DESIGN DOC ===\"; SLUG=$(~/.claude/skills/gstack/browse/bin/remote-slug 2>/dev/null || basename \"$(git rev-parse --show-toplevel 2>/dev/null || pwd)\"); BRANCH=$(git rev-parse --abbrev-ref HEAD 2>/dev/null | tr '/' '-' || echo 'no-branch'); echo \"SLUG=$SLUG BRANCH=$BRANCH\"; DESIGN=$(~/.claude/skills/gstack/bin/gstack-design-doc-find \"$SLUG\" \"$BRANCH\"); [ -n \"$DESIGN\" ] && echo \"Design doc found: $DESIGN\" || echo \"No design doc found\"; echo \"=== LEARNINGS ===\"; _CROSS_PROJ=$(~/.claude/skills/gstack/bin/gstack-config get cross_project_learnings 2>/dev/null || echo \"unset\"); echo \"CROSS_PROJECT: $_CROSS_PROJ\"; ~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 2>&1 | head -20; echo \"=== BRAIN ===\"; S2=$(~/.claude/skills/gstack/bin/gstack-slug --get SLUG 2>/dev/null); for d in product goals recent-decisions; do echo \"### $d\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get $d --project \"$S2\" 2>/dev/null || echo \"(no $d digest available yet)\"; done; echo \"### user-profile\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get user-profile 2>/dev/null || echo \"(no user-profile digest available yet)\"; echo \"=== DECISIONS ===\"; ~/.claude/skills/gstack/bin/gstack-decision-search \"skill list\" 2>&1 | head -10; echo \"=== MODE PREF CHECK ===\"; printf '%s' \"Select plan-ceo-review mode: SCOPE EXPANSION / SELECTIVE EXPANSION / HOLD SCOPE / SCOPE REDUCTION for deterministic skill-list ordering plan\" | ~/.claude/skills/gstack/bin/gstack-question-preference --check \"plan-ceo-review-mode\" --summary-stdin; echo \"exit=$?\"", + "description": "Check design doc, learnings, brain context, and mode preference" + } + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:51:25.239Z", + "toolUseId": "toolu_019Yi6AfNvLiT21cAkR3KUs5", + "kind": "result", + "isError": false, + "content": "=== DESIGN DOC ===\nSLUG=gstack-plan-count-TouOf1 BRANCH=main\nNo design doc found\n=== LEARNINGS ===\nCROSS_PROJECT: false\n=== BRAIN ===\n### product\n(no product digest available yet)\n### goals\n(no goals digest available yet)\n### recent-decisions\n(no recent-decisions digest available yet)\n### user-profile\n(no user-profile digest available yet)\n=== DECISIONS ===\n=== MODE PREF CHECK ===\nAUTO_DECIDE\nexit=0" + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:51:52.221Z", + "toolUseId": "toolu_01GgiRYKUykkGmbjN1d5vFr8", + "kind": "use", + "name": "Bash", + "input": { + "command": "echo \"=== HANDOFF ===\"; ~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff \"HOLD SCOPE\" --decisions \"none\" --auto; echo \"exit=$?\"; echo \"=== QUESTION LOG ===\"; ~/.claude/skills/gstack/bin/gstack-question-log '{\"skill\":\"plan-ceo-review\",\"question_id\":\"plan-ceo-review-mode\",\"question_summary\":\"Select plan-ceo-review mode for deterministic skill-list ordering plan\",\"category\":\"routing\",\"door_type\":\"two-way\",\"options_count\":4,\"user_choice\":\"HOLD SCOPE\",\"recommended\":\"HOLD SCOPE\",\"auto_decided\":true,\"session_id\":\"1531-1791100258-3092a930\"}'; echo \"exit=$?\"; echo \"=== READ BACK ===\"; GSR=$(~/.claude/skills/gstack/bin/gstack-paths --get GSTACK_STATE_ROOT 2>/dev/null); echo \"STATE_ROOT=$GSR\"; grep -rl \"plan-ceo-review-mode\" \"$GSR/analytics\" 2>/dev/null | head; grep -rh \"plan-ceo-review-mode\" \"$GSR/analytics\" 2>/dev/null | tail -2", + "description": "Run mode handoff and log the auto-decided mode question" + } + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:51:52.956Z", + "toolUseId": "toolu_01GgiRYKUykkGmbjN1d5vFr8", + "kind": "result", + "isError": false, + "content": "=== HANDOFF ===\nAuto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune. Approved decisions: none.\nexit=0\n=== QUESTION LOG ===\nexit=0\n=== READ BACK ===\nSTATE_ROOT=" + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:51:59.150Z", + "toolUseId": "toolu_017HHrgGdJKxew9wEZmaHU2Z", + "kind": "use", + "name": "Bash", + "input": { + "command": "GSR=; grep -rl \"plan-ceo-review-mode\" \"$GSR\" 2>/dev/null | head; echo \"---\"; f=$(grep -rl \"plan-ceo-review-mode\" \"$GSR\" 2>/dev/null | grep -i question | head -1); [ -n \"$f\" ] && tail -1 \"$f\"; echo \"--- audit record search ---\"; grep -rl \"1531-1791100258-3092a930\" \"$GSR\" 2>/dev/null | head", + "description": "Locate the written question-log audit record" + } + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:51:59.771Z", + "toolUseId": "toolu_017HHrgGdJKxew9wEZmaHU2Z", + "kind": "result", + "isError": false, + "content": "/projects/gstack-plan-count-TouOf1/question-preferences.json\n/projects/gstack-plan-count-TouOf1/question-events.jsonl\n/projects/gstack-plan-count-TouOf1/question-log.jsonl\n---\n}--- audit record search ---\n/projects/gstack-plan-count-TouOf1/question-log.jsonl\n/projects/gstack-plan-count-TouOf1/timeline.jsonl" + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:52:04.134Z", + "toolUseId": "toolu_0154eSHKDTrmUtEQkTNASUKQ", + "kind": "use", + "name": "Bash", + "input": { + "command": "GSR=/projects/gstack-plan-count-TouOf1; echo \"=== question-log tail ===\"; tail -c 800 \"$GSR/question-log.jsonl\"; echo; echo \"=== preference ===\"; cat \"$GSR/question-preferences.json\"", + "description": "Read back the audit record and the stored mode preference" + } + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:52:04.665Z", + "toolUseId": "toolu_0154eSHKDTrmUtEQkTNASUKQ", + "kind": "result", + "isError": false, + "content": "=== question-log tail ===\n{\"skill\":\"plan-ceo-review\",\"question_id\":\"plan-ceo-review-mode\",\"question_summary\":\"Select plan-ceo-review mode for deterministic skill-list ordering plan\",\"category\":\"routing\",\"door_type\":\"two-way\",\"options_count\":4,\"user_choice\":\"HOLD SCOPE\",\"recommended\":\"HOLD SCOPE\",\"auto_decided\":true,\"session_id\":\"1531-1791100258-3092a930\",\"source\":\"agent\",\"followed_recommendation\":true,\"ts\":\"2026-10-04T07:51:52.942Z\"}\n\n=== preference ===\n{\n \"plan-ceo-review-mode\": \"never-ask\"\n}" + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:52:39.349Z", + "toolUseId": "toolu_016pABSQS4x7wPFAH6PD5Wvm", + "kind": "use", + "name": "Write", + "input": { + "file_path": "/plans/keep-this-draft-plan-typed-tarjan.md", + "content": "# Plan: deterministic skill-list ordering (CEO review — Step 0 checkpoint)\n\n## Context\n\nUsers compare skill listings in scripts and reviews, so a listing whose order\ndepends on directory enumeration order is noise in diffs and flaky in tests.\nThe draft under review (PLAN.md) makes the existing listing path sort registered\nskill names deterministically before rendering, while keeping skill membership,\naliases, metadata and text/JSON output formats unchanged, with tests for\nmixed-case names and differing directory enumeration order.\n\nThis file is a checkpoint, not the review report. The user asked for the\nreview-mode decision only; the substantive review (Sections 1–11, required\noutputs, GSTACK REVIEW REPORT) runs in a later invocation.\n\n## Pre-review system audit (2026-10-04)\n\n- Repo: fixture at the primary working directory; branch `main`; one commit\n (`a2e4c62 Seed review plan`); no remote, no stash, no TODO/FIXME markers.\n- Files: CLAUDE.md, PLAN.md only. The implementation target codebase (the\n skill registry / listing path) is not in this checkout; 0B leverage mapping\n must be done against the real codebase in the substantive review.\n- Base branch: no remote; git-native fallback resolves to `main`.\n- Design doc: none found for slug `gstack-plan-count-TouOf1` / branch `main`.\n The /office-hours prerequisite offer was NOT made: the user said they will\n handle Office Hours and setup separately.\n- Prior learnings: none (CROSS_PROJECT: false). Brain digests: all cold.\n- Retrospective check: no prior review refactors/reverts in this repo.\n- DESIGN_SCOPE: none. Output formats are explicitly unchanged; Section 11 will\n be a no-UI skip.\n- Suspicious content in inputs: none. PLAN.md's handoff paragraph is addressed\n to the operator, not the reviewer, and asks for nothing outside the skill.\n\n## Step 0 evidence\n\n### 0A. Premise challenge\nReal problem: listing order varies with filesystem enumeration, so scripted\ncomparisons and reviews see spurious diffs. Target outcome: identical input\nset always renders in identical order. Do-nothing cost: ongoing false diffs and\norder-dependent test flakiness. The plan addresses the pain directly (sort at\nthe listing path), not a proxy.\n\n### 0B. Existing code leverage\nPending: the listing codebase is not in this checkout. The substantive review\nmust locate the existing listing path and registry and confirm the sort is a\nsingle insertion there (no new abstraction). Open question for that review:\nsort key semantics (case-sensitive byte order vs. case-insensitive with a\nstable tiebreak) — the plan requires mixed-case coverage but does not name the\nrule.\n\n### 0C. Dream state\n```\n CURRENT STATE THIS PLAN 12-MONTH IDEAL\n order follows directory ---> sort names once at ---> every list-shaped\n enumeration; diffs are the listing path; output in the tool\n noisy; tests order- tests pin mixed-case is deterministic and\n dependent + enumeration order diff-friendly\n```\nThe plan moves toward the ideal without over-reaching.\n\n### 0D. Alternatives\nNo new approach decision was needed. The plan names a single approach; the only\nopen implementation choice (sort-key rule) is a Section-level finding for the\nsubstantive review, not a Step 0 blocker.\n\n## Decision ledger\n\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| MODE (reviewer) | 0E mode selection; `plan-ceo-review-mode` preference check returned `AUTO_DECIDE` (exit 0) | HOLD SCOPE | — | approved | Auto-decided by stored preference `never-ask`; logged to question-log.jsonl with user_choice=HOLD SCOPE, recommended=HOLD SCOPE, auto_decided=true, session 1531-1791100258-3092a930 |\n\n## 0E. Mode selection\n\n- Review depth: implementation-ready (default; user did not restrict it).\n- File count: estimated 2 changed files (one listing-path source edit, one test\n file). Estimate; codebase not in checkout.\n- Category: fix/refactor of an existing path with formats held constant → rule\n recommends HOLD SCOPE.\n- Recommendation: HOLD SCOPE because the draft pins its own ceiling (membership,\n aliases, metadata and output formats unchanged), so the review's value is in\n rigor on the sort rule, enumeration-order test, and nil/empty paths, not in\n expansion.\n- Resolution: `gstack-question-preference --check plan-ceo-review-mode` →\n `AUTO_DECIDE`; recommendation selected without asking.\n- Handoff line (from `gstack-ceo-mode-handoff \"HOLD SCOPE\" --decisions none --auto`):\n `Auto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune. Approved decisions: none.`\n- Provenance: successful preference check; `plan-ceo-review-mode` logged with\n `auto_decided: true`.\n\n## Deferred by user instruction (not skipped silently)\n\n- /office-hours prerequisite offer: user will handle separately.\n- Routing-rules setup (CLAUDE.md skill routing section): user will handle\n separately; also blocked in plan mode (CLAUDE.md edit + commit).\n- Cross-project learnings config: already set (`false`); no question needed.\n\n## Next\n\nResume with HOLD SCOPE route: 0G (complexity check, minimum changes, invariants)\n→ 0I (temporal interrogation) → `sections/review-sections.md` Sections 1–10,\nSection 11 no-UI skip, required outputs, GSTACK REVIEW REPORT.\n" + } + }, + { + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "timestamp": "2026-10-04T07:52:39.381Z", + "toolUseId": "toolu_016pABSQS4x7wPFAH6PD5Wvm", + "kind": "result", + "isError": false, + "content": "File created successfully at: /plans/keep-this-draft-plan-typed-…[elided: not consumed]" + } + ], + "options": { + "skillName": "plan-ceo-review", + "sessionId": "bb393dbc-4a5c-4710-a917-071bdf778306", + "commandStartedAt": 1791100250785, + "now": 1791100373204, + "proseQuestionObserved": false, + "stateEvidence": { + "questionId": "plan-ceo-review-mode", + "preference": "never-ask", + "records": [ + { + "skill": "plan-ceo-review", + "question_id": "plan-ceo-review-mode", + "question_summary": "Select plan-ceo-review mode for deterministic skill-list ordering plan", + "category": "routing", + "door_type": "two-way", + "options_count": 4, + "user_choice": "HOLD SCOPE", + "recommended": "HOLD SCOPE", + "auto_decided": true, + "session_id": "1531-1791100258-3092a930", + "source": "agent", + "ts": "2026-10-04T07:51:52.956Z" + } + ] + } + } + }, + "screen": { + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37186854666-1-eval-slices-20/plan-ceo-review-1791100373205-nby65b/terminal.visible.log", + "tail_chars": 6000, + "text": "ectory enumeration or with tests.\r \rThis draft is the review target, notthe urent branch. For this nvocation,\rI want only the review-mode decision; I will handle optional Office Hours and\rsetup separately,and run the substanive reviewlater. No eviewmode has \rbeen selected. For this mode-only handoff, use the full selected mode name in\rthe normal udit record'susr_choice andcommended fields and in the skill's\rnormal mode handff lne. A meu letter alone would notidentifythe chosn mode.\r\r● Received. I have the draft plan for deterministic skill-list ordering as context, along with your constraints:\rreview-modedecisiononly,fullmodenameintheauditrecordandhandoffline,noOfficeHoursorsetupinthispass.\r I'll waitfor your slash commd before dog anything ese.\r\r✻Brewed for 4s · done 7:50 AM\r❯ /plan-ceo-review \r● Running the plan-ceo-review preamble now, then I'll work toward the mode decision only, per your handoff instructions.\r Read 1 file, ran 6 shell commands \r●Mode decision is made and logged. WritingashortStep0checkpointtotheplanfilesothelatersubstantivereview\rcanresumefromit.\r\r●Updated plan\r65 \rDraft: deterministic skill-list ordering\r \rUsers compare skill listings in scripts and reviews. Make the existing listing\rpath sort regitered skill names deterministically beforrendrng them. \rKeepkill membership, aiases, metadata and text/JSON output formats unchanged.\rCover mxed-cas namesnd differing directory enumeration order wihtests. \r \rThis draft is the review target, not the current branch. For this invocation,\rI want only the review-mode dcisin; I will handle optionalOffice Hurs ad\rsetup separaely, and run the substantive review ater. No review mode has \rben selected. For this mode-only hdoff, use the fullselected mode name in\rth normal audit ecord's user_choice and recommended fields anin the skill's\rnormal ode handoff line.A menu letter alone woul not identifythe choen mode.\r\r●Received.Ihavethedraftplanfordeterministicskill-listorderingascontext,alongwithyourconstraints:\r rview-modedecisiononly,fullmde name the audt record and handoff lin, no Office Hours or setup thispass.\rI'll ait for your slash command before dog anythingelse.\r\r✻Brewed for 4s · done 7:50 AM\r\r❯ /plan-ceo-review \r\r●Runningtheplan-ceo-reviewpreamblenow,thenI'llworktowardthemodedecisiononly,peryourhandoffinstructions.\r\rRead 1 file, ran 6 shell commands \r\r●Modedecisionismadeandlogged.WritingashortStep0checkpointtotheplanfilesothelatersubstantivereview\r can resume from it.\r\r●Updated plan\r ⎿  /plan to preview\r·\r✢\rCerebrating…71\rCerebrating…\r*\r✶\r✻Cerebrating…50\r✽Cerebrating…\rCerebrating…\rCerebrating…\rCerebrating…\r✻Cerebrating…\r✶Cerebrating…\r*\r✢Cerebrating…1\r·Cerebrating…\rCerebrating…\rCerebrating…\rCerebrating…\r✢Cerebrating…\r*Cerebrating…\r✶\r✻Cerebrating…2\r✽Cerebrating…\rCerebrating…\rCerebrating…\rCerebrating…\r✻Cerebrating…\r✶Cerebrating…\r*\r✢Cerebrating…3\r·Cerebrating…\rCerebrating…\rCerebrating…\rCerebrating…\r✢Cerebrating…\r*Cerebrating…\r✶\r✻Cerebrating…4\r✽Cerebrating…\rCerebrating…\rCerebrating…\rCerebrating…\r✻Cerebrating…\r✶Cerebrating…\r*\r✢\rCerebrating…5\r·Cerebrating…\rCerebrating…\rCerebrating…\rCerebrating…\r✢Cerebrating…\r*Cerebrating…\r✶\r✻\rCerebrating…6\r✽Cerebrating…\rCerebrating…\rCerebrating…\rCerebrating…\r✻Cerebrating…\r✶Cerebrating…\r*\r✢\rCerebrating…7\r·Cerebrating…\rCerebrating…\rCerebrating…\r✢Cerebrating…\rCerebrating…\r*Cerebrating…\r✶\r✻\rCerebrating…8\r✽Cerebrating…\rCerebrating…\rCerebrating…\r✻Cerebrating…\r✶Cerebrating…\r*\r✢Cerebrating…\r9\r·Cerebrating…\rCerebrating…\rCerebrating…\r✢Cerebrating…\rCerebrating…\r*\r✶\r✻20s · ↓ 7.1k tokens)\r✽\r✻\r✻Cerebrating… \r✶\r*\r✢1\r·\r\r● Mode decision is made and logged. Writing a short Step 0 checkpoint to the plan file so the later substantive review\r can resume from it.\r\r● Updated plan\r ⎿  /plan to preview\r\r● Auto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune. Approved decisions: none.\r\r Mode application and rationale. The draft is a fix to an existing path with its own ceiling already pinned:\r membership, aliases, metadata and both output formats stay unchanged. That is the fix/refactor category, with an\r estimated two changed files, so the 0E rule recommends HOLD SCOPE. The review's value here is rigor on the sort rule,\r the enumeration-order test, and the empty and nil paths, not expansion. No new approach decision was needed.\r Provenance.Yourstored preference for the mode question never-ask, o the preferece check retured AUTO_DECIDE\rand the recommendation wasseecte without a mnu. The auditrecord is writtnand read back, with the full mode name\rin both fields as you asked:\r question_id=plan-ceo-review-mode user_choice=HOLD SCOPE recommended=HOLD SCOPE\rauto_decided=true session_id=1531-1791100258-3092a930\r\rWhat I did not do, per your instructions.IskippedtheOfficeHoursprerequisiteofferandtheskill-start\r rout-rules setup pompt,since you said youwihandle bothseparatly. The routingstup would alsoneed a\rCLAUDE.mdeditandcommit,whichplanmodeforbids.Thedesigndoccheckfoundnoneforthisbranch.\r\rCheckpoint written.Thehostplanfilenowholdsthesystemaudit,0Ato0Cevidence,thedecisionledgerwiththe\r mrow, andthe resume route. The substantive reviwresumes at 0G, then0I, then thereview sections. One itm\rflaggd for that review:theplanrequiresmixed-casecoveragebutdoesnotnamethesort-keyrule,socase-sensitive\rbyteorderversuscase-insensitivewithastabletiebreakisanopenimplementationchoice.\r\r The review workflowispausedratherthancomplete,soIdidnotrunend-of-skilltelemetryorcallExitPlanMode.No\rdurablelearningsthissession.\r\r✻ Cooked for 2m 1s· done7:52 AM\r❯ \r← for agents" + } + } +} diff --git a/test/fixtures/detector-corpora/auto-decide-preserved/37193478719-t1-pass.json b/test/fixtures/detector-corpora/auto-decide-preserved/37193478719-t1-pass.json new file mode 100644 index 000000000..01d97b81c --- /dev/null +++ b/test/fixtures/detector-corpora/auto-decide-preserved/37193478719-t1-pass.json @@ -0,0 +1,212 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "auto-decide-preserved", + "detector": "test/helpers/native-auto-decide.ts#findNativeAutoDecision", + "screen_detector": "test/helpers/pty/classify.ts#classifyVisible", + "series_identity": "7678e372b3345f9b", + "source": { + "run": "37193478719", + "sha": "9f357d9b9c75441b6a18e4f75174274954ba6f96", + "lane": "periodic", + "artifact": "paid-slice-20-a1", + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37193478719-1-eval-slices-20/plan-ceo-review-1791107751754-kWX5bO/observation.json" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "classifyVisible accepted the sent AUTO_DECIDE line on screen. The archived native poll predates that message (last assistant message is \"Now running the handoff and audit record\"), and the handoff ran inside a compound command, so the native snapshot holds no current declaration.", + "screen_expected": "pass", + "native_expected": "refuse", + "derived": "options.stateEvidence rebuilt from the session's gstack-question-log argument with source \"agent\" and ts = that call's tool-result timestamp (the bound question-log.jsonl itself is not archived)", + "input": { + "native": { + "transcript": { + "status": "ready", + "calls": [], + "assistantMessages": [ + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "text": "Received. I have the draft plan for deterministic skill-list ordering as context, along with your constraints: review-mode decision only, full mode name in the audit record and handoff line, no Office Hours or setup in this invocation, substantive review deferred. Waiting for your slash command.", + "timestamp": "2026-10-04T09:54:41.190Z" + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "text": "Running the plan-ceo-review preamble, then working through Step 0 only as far as the mode decision you asked for.", + "timestamp": "2026-10-04T09:54:45.147Z" + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "text": "Audit complete: single-commit repo with no remote, stashes, TODOs, design docs, or prior learnings; the mode preference resolved to AUTO_DECIDE, so I'm proceeding on the recommendation without needing a question. Now running the handoff and audit record.\n\n", + "timestamp": "2026-10-04T09:55:35.536Z" + } + ] + }, + "tools": [ + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:54:46.344Z", + "toolUseId": "toolu_019XB6AY8pzt53jEujDBLsFD", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-skill-start --skill \"plan-ceo-review\" --model \"claude\"", + "description": "Run gstack skill-start preamble" + } + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:54:47.584Z", + "toolUseId": "toolu_019XB6AY8pzt53jEujDBLsFD", + "kind": "result", + "isError": false, + "content": "SKILL_START_PROTO: 1\nBRANCH: main\nPROACTIVE: true\nPROACTIVE_PROMPTED: yes\nSKILL_PREFIX: false\nREPO_MODE: unknown\nSESSION_KIND: interactive\nCONDUCTOR_SESSION: true\nACTIVATED: yes\nFIRST_LOOP_SHOWN: yes\nFIRST_TASK: \nLAKE_INTRO: yes\nTELEMETRY: off\nTEL_PROMPTED: yes\nSESSION_ID: 1599-1791107687-bb89148d\nTEL_START: 1791107687\nEXPLAIN_LEVEL: default\nQUESTION_TUNING: true\nUPDATE_CHECK: false\nLEARNINGS: 0\nHAS_ROUTING: no\nROUTING_DECLINED: false\nVENDORED_GSTACK: no\nMODEL_OVERLAY: claude\nGSTACK_PLAN_MODE: active\nARTIFACTS_SYNC: off\n" + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:54:59.811Z", + "toolUseId": "toolu_01YYncvLTNru4zwNR1w1WBys", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-context-recovery 2>&1 | head -40", + "description": "Recover recent project context" + } + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:55:03.002Z", + "toolUseId": "toolu_012XobDGNx98ppKFQMXKWvMt", + "kind": "use", + "name": "Bash", + "input": { + "command": "echo \"REMOTE: $(git remote get-url origin 2>&1)\"; echo \"HEAD: $(git symbolic-ref refs/remotes/origin/HEAD 2>&1)\"; echo \"--- log\"; git log --oneline -30; echo \"--- stash\"; git stash list; echo \"--- status\"; git status --short; echo \"--- files\"; git ls-files; echo \"--- todos\"; grep -rn \"TODO\\|FIXME\\|HACK\\|XXX\" --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=.git . | head -30", + "description": "Audit repo history, remote, stashes, file list and TODO markers" + } + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:55:08.318Z", + "toolUseId": "toolu_01W6wFMKDjEggUcsFxt5KZRP", + "kind": "use", + "name": "Bash", + "input": { + "command": "SLUG=$(~/.claude/skills/gstack/browse/bin/remote-slug 2>/dev/null || basename \"$(git rev-parse --show-toplevel 2>/dev/null || pwd)\"); BRANCH=$(git rev-parse --abbrev-ref HEAD 2>/dev/null | tr '/' '-' || echo 'no-branch'); DESIGN=$(~/.claude/skills/gstack/bin/gstack-design-doc-find \"$SLUG\" \"$BRANCH\"); [ -n \"$DESIGN\" ] && echo \"Design doc found: $DESIGN\" || echo \"No design doc found\"; echo \"SLUG=$SLUG BRANCH=$BRANCH\"; _CROSS_PROJ=$(~/.claude/skills/gstack/bin/gstack-config get cross_project_learnings 2>/dev/null || echo \"unset\"); echo \"CROSS_PROJECT: $_CROSS_PROJ\"; ~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 2>&1 | head -20; echo \"--- pref check\"; printf '%s' \"Select plan-ceo-review mode for deterministic skill-list ordering plan\" | ~/.claude/skills/gstack/bin/gstack-question-preference --check \"plan-ceo-review-mode\" --summary-stdin; echo \"exit=$?\"", + "description": "Check design doc, prior learnings, and mode question preference" + } + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:55:10.632Z", + "toolUseId": "toolu_019mgJbvW16gMdxAFCeZsvnZ", + "kind": "use", + "name": "Bash", + "input": { + "command": "SLUG=$(~/.claude/skills/gstack/bin/gstack-slug --get SLUG 2>/dev/null) || true; for d in product goals recent-decisions; do echo \"### $d\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get $d --project \"$SLUG\" 2>/dev/null || echo \"(no $d digest available yet)\"; done; echo \"### user-profile\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get user-profile 2>/dev/null || echo \"(no user-profile digest available yet)\"", + "description": "Load brain context digests" + } + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:55:11.260Z", + "toolUseId": "toolu_01EgWh7Xf2Z8NwJwT3uJVJwx", + "kind": "use", + "name": "Read", + "input": { + "file_path": "/PLAN.md" + } + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:55:11.310Z", + "toolUseId": "toolu_01YYncvLTNru4zwNR1w1WBys", + "kind": "result", + "isError": false, + "content": "--- RECENT ARTIFACTS ---\n{\"skill\":\"plan-ceo-review\",\"event\":\"started\",\"branch\":\"main\",\"session\":\"1599-1791107687-bb89148d\",\"ts\":\"2026-10-04T09:54:47.602Z\"}\n--- END ARTIFACTS ---" + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:55:11.354Z", + "toolUseId": "toolu_012XobDGNx98ppKFQMXKWvMt", + "kind": "result", + "isError": false, + "content": "REMOTE: error: No such remote 'origin'\nHEAD: fatal: ref refs/remotes/origin/HEAD is not a symbolic ref\n--- log\n3f10c21 Seed review plan\n--- stash\n--- status\n---…[elided: not consumed]" + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:55:11.473Z", + "toolUseId": "toolu_01W6wFMKDjEggUcsFxt5KZRP", + "kind": "result", + "isError": false, + "content": "No design doc found\nSLUG=gstack-plan-count-3Pak5B BRANCH=main\nCROSS_PROJECT: false\n--- pref check\nAUTO_DECIDE\nexit=0" + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:55:11.584Z", + "toolUseId": "toolu_019mgJbvW16gMdxAFCeZsvnZ", + "kind": "result", + "isError": false, + "content": "### product\n(no product digest available yet)\n### goals\n(no goals digest available yet)\n### recent-decisions\n(no recent-decisions digest available yet)\n### user-profile\n(no user-profile digest available yet)" + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:55:11.593Z", + "toolUseId": "toolu_01EgWh7Xf2Z8NwJwT3uJVJwx", + "kind": "result", + "isError": false, + "content": "1\t# Draft: deterministic skill-list ordering\n2\t\n3\tUsers compare skill listings in scripts and reviews. Make the existing listing\n4\tpath sort registered skill na…[elided: not consumed]" + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:55:37.327Z", + "toolUseId": "toolu_01HTu27YVDX3G3wtcuQXGvNL", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff \"HOLD SCOPE\" --decisions \"none\" --auto; echo \"handoff_exit=$?\"; ~/.claude/skills/gstack/bin/gstack-question-log '{\"skill\":\"plan-ceo-review\",\"question_id\":\"plan-ceo-review-mode\",\"question_summary\":\"Select plan-ceo-review mode for deterministic skill-list ordering plan\",\"category\":\"routing\",\"door_type\":\"two-way\",\"options_count\":4,\"user_choice\":\"HOLD SCOPE\",\"recommended\":\"HOLD SCOPE\",\"auto_decided\":true,\"session_id\":\"1599-1791107687-bb89148d\"}'; echo \"log_exit=$?\"", + "description": "Run mode handoff and log the auto-decided mode record" + } + }, + { + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "timestamp": "2026-10-04T09:55:38.015Z", + "toolUseId": "toolu_01HTu27YVDX3G3wtcuQXGvNL", + "kind": "result", + "isError": false, + "content": "Auto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune. Approved decisions: none.\nhandoff_exit=0\nlog_exit=0" + } + ], + "options": { + "skillName": "plan-ceo-review", + "sessionId": "e50ce1aa-1167-4be3-b288-01d65e8724e2", + "commandStartedAt": 1791107681517, + "now": 1791107751753, + "proseQuestionObserved": false, + "stateEvidence": { + "questionId": "plan-ceo-review-mode", + "preference": "never-ask", + "records": [ + { + "skill": "plan-ceo-review", + "question_id": "plan-ceo-review-mode", + "question_summary": "Select plan-ceo-review mode for deterministic skill-list ordering plan", + "category": "routing", + "door_type": "two-way", + "options_count": 4, + "user_choice": "HOLD SCOPE", + "recommended": "HOLD SCOPE", + "auto_decided": true, + "session_id": "1599-1791107687-bb89148d", + "source": "agent", + "ts": "2026-10-04T09:55:38.015Z" + } + ] + } + } + }, + "screen": { + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37193478719-1-eval-slices-20/plan-ceo-review-1791107751754-kWX5bO/terminal.visible.log", + "tail_chars": 6000, + "text": " full mode name in the audit record and handoff line, no Office Hours or setup in this\r invocation, substantive review deferred. Waiting for your slash command.\r\r✻Brewed for 3s · done 9:54 AM\r\r❯ /plan-ceo-review \r\r●Runningtheplan-ceo-reviewpreamble,thenworkingthroughStep0onlyasfarasthemodedecisionyouaskedfor.\r\rRead 1 file, ran 5 shell commands \r\r●Auditcomplete:single-commitrepowithnoremote,stashes,TODOs,designdocs,orpriorlearnings;themode\r preference resolved to AUTO_DECIDE, so I'mproceeding n the recommendation without neeing a questin. Now running\rthehandoffandauditrecord. · summary\r\r● Running 1 shellcommand…\r✶Billowing… 39thought for 21s)\r✻4\r \r✽\r✻\r✶\r●\r*\r✢5\r·\r \r…\r✢g\r*path sort registered skill names deterministically before rendering them.\rKeep skill membership, aliases, metadata and text/JSON output formats unchanged.\rCover mixed-case names and differing directory enumeration order with tests.\r \rThisdraft is th review target,no the current branch. For this invocation,\rI want ony the evew-mode decision; I willhandle optional Ofice Hors d \rsetupseparately, nd ru the substantv review later. Noreview mode has \rbeen selected. For this mode-only handoff, use the full selected mode name in\rte nomalaudit record's user_choice and recommended fieldsand in the skill's\rnormalmodehandoff line. Amnu letteralone woudno identiy the choen mode.\r\r● Received. I have the draft plan for deterministic skill-list ordering as context, along with your constraints:\r review-mode decision only, full mode name in the audit record and handoff line, no Office Hours or setup in this\r invocation, substantive review deferred. Waiting for your slash command.\r✻Brewed for 3s · done 9:54 AM\r\r❯ /plan-ceo-review \r●Running the plan-ceo-review preamble,thenworkingthroughStep0onlyasfarasthemodedecisionyouaskedfor.\r Read 1 file, ran 5 shell commands \rAdit complete: single-commit repo with noremote,stashes, TODOs, designdocs, or prir learnings; the mode\rpreferenceresolvedtoAUTO_DECIDE,soI'mproceedingontherecommendationwithoutneedingaquestion.Nowrunning\rthe handoff and audit record. · summary\r Rnning 1 shellcommand…\r ⎿ $ ~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff \"HOLD SCOPE\" --decisions \"none\" --auto; echo \r \"handoff_exit=$?\"; ~/.claude/skills/gstack/bin/gstack-question-log \r'{\"skill\":\"plan-ceo-review\",\"question_id\":\"plan-ceo-review-mode\",\"question_summary\":\"Select plan-ceo-review mode \r for deterministic sk…\rn40\r✶i…\r●\r)\r✻6\r✽wg1\ron\rmode handff andlogtheauto-decidedmoderecord\rBillowing…\rBillowing…\r \r✻Billowing…\r✶Billowing…2\r*\r✢Billowing…7\r●\r·Billowing…\rBillowing…\rBillowing…\rBillowing…\r✢Billowing…\r*Billowing…\r \r✶\r✻Billowing…8\r✽Billowing…\rBillowing…\r●\rBillowing…\rBillowing…\r✻Billowing…\r✶Billowing…\r*\r✢\r \rBillowing…9\r·Billowing…\rBillowing…\rBillowing…\rBillowing…\r●\r✢Billowing…\r*Billowing…\r✶\r✻\rBillowing…1m 0s· ↓ 4.2k tokens)\r✽Billowing…\r \rBillowing…\rBillowing…\rBillowing…1\r✻Billowing…\r✶Billowing…\r●\r*\r✢\rBillowing…\r·Billowing…\rBillowing…\r \rBillowing…\r✢Billowing…2\rBillowing…\r*Billowing…\r✶\r✻\r●\rBillowing…\r✽Billowing…\rBillowing…\rBillowing…\r✻Billowing…3\r \r✶Billowing…\r*\r✢Billowing…\r·Billowing…\r●\rBillowing…\rBillowing…\r✢Billowing…4\r*Billowing…\r \r✶\r✻Billowing…\r✽Billowing…\rBillowing…\rBillowing…\r●\r✻Billowing…5\rBillowing…\r✶\r*\r✢Billowing…\r \rBillowing…\r·Billowing…\rBillowing…\rBillowing…\r✢Billowing…6\r●\rBillowing…\r*\r✶\r✻Billowing…\r✽Billowing…\rBillowing…\r \rBillowing…\rBillowing…\r✻7\r✶\r●\r*\r✢\r·\r·Billowing… \r \r✢8\r*\r✶\r✻\r●\r✽\r9\r✻\r✶\r \r✶Billowing… \r*\r✢\r·\r●\r10s · ↓ 4.2k tokens)\r✢\r*\r \r✶● Audit complete: single-commit repo with no remote, stashes, TODOs, design docs, or prior learnings; the mode\r preference resolved to AUTO_DECIDE, so I'm proceeding on the recommendation without needing a question. Now running\r the handoff and audit record. · summary\r\r Ran 1 shell command \r ⎿ $ ~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff \"HOLD SCOPE\" --decisions \"none\" --auto; echo \r● Auto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune. Approved decisions: none. This plan\r is a fix to an existing codepath: sort registered skill names before rendering, with membership, aliases, metadata and\r output formats explicitly frozen. The estimated change is 2 to 3 files, the listing module plus tests. That fits the\r fix/refactor rule, and HOLD SCOPE gives it the rigor it actually needs, namely tracing mixed-case ordering,\renumeration-orderindependence,andthetext/JSONformatinvariantsratherthantemptingexpansionsonadeterminism\r fix.\r\rNo new approach decision was needed. Theplan prescibes sorting in the existinglistingpath,whichisalreadythe\rsmallestscopedalternative,soStep0Dhadnorequiredchoice.\r\rMode provenance.Preferencecheckforplan-ceo-review-modeexited0withAUTO_DECIDE.Theauditrecordwasloggedwith\r user_choice and recommended both set to the full name HOLD SCOPE, auto_decided true, under this session's ID. The\rhandoffcommandandthelogbothexited0.\r\rStep 0 evidence gathered so far\r\r-0Apremise:thepainisnon-reproduciblelistingswhenusersdiffoutputacrossmachinesorruns.Sortingaddresses\r it directly, not a proxy.\r- 0B lverag: th plan reuses the existing listing path. Nobuild propsed.\r- 0C dream state: a detrministic listingisaprerequisiteforscriptedcomparisonandsnapshottests.Theplanmoves\rtowardthat.\r\r Deferred per your instructions\r\r-Office Hours offer: no design doc found, offer skipped. Re-offer suppressed for this session.\r-Setup: the preamble firedaone-timerouting-rulesonboardingpromptforCLAUDE.md.Deferred,sinceyousaidsetup\risseparateandplanmodeforbidstheCLAUDE.mdeditandcommititrequires.Itsgateismarker-based,soit\r re-fires on the nexthealthy run.\r - Substantive review: HOLD SCOPE route is 0G then 0I then the 11 review sections. Not started.Nothingwaswrittento\r" + } + } +} diff --git a/test/fixtures/detector-corpora/auto-decide-preserved/37195203538-t1-pass.json b/test/fixtures/detector-corpora/auto-decide-preserved/37195203538-t1-pass.json new file mode 100644 index 000000000..1a36653e8 --- /dev/null +++ b/test/fixtures/detector-corpora/auto-decide-preserved/37195203538-t1-pass.json @@ -0,0 +1,217 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "auto-decide-preserved", + "detector": "test/helpers/native-auto-decide.ts#findNativeAutoDecision", + "screen_detector": "test/helpers/pty/classify.ts#classifyVisible", + "series_identity": "7678e372b3345f9b", + "source": { + "run": "37195203538", + "sha": "41287009bd6b524244edced7770c87fc3103eb59", + "lane": "periodic", + "artifact": "paid-slice-20-a1", + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37195203538-1-eval-slices-20/plan-ceo-review-1791109765127-W8O99c/observation.json" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "The observer recorded nativeAutoDecide: the logged never-ask record plus the sent AUTO_DECIDE line in the same session after the preamble; the screen phrase was not needed.", + "screen_expected": "refuse", + "native_expected": "pass", + "derived": "options.stateEvidence rebuilt from the session's gstack-question-log argument with source \"agent\" and ts = that call's tool-result timestamp (the bound question-log.jsonl itself is not archived)", + "input": { + "native": { + "transcript": { + "status": "ready", + "calls": [], + "assistantMessages": [ + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "text": "Received. I have the draft plan for deterministic skill-list ordering in context, along with your handoff constraints: review-mode decision only, no Office Hours or setup, full mode name in the audit record's user_choice and recommended fields and in the mode handoff line. I'll wait for your slash command before starting anything or calling tools.", + "timestamp": "2026-10-04T10:27:58.559Z" + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "text": "Starting the plan-ceo-review preamble now, then I'll go straight to the mode decision as you asked.", + "timestamp": "2026-10-04T10:28:08.100Z" + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "text": "Context gathered. The mode preference check came back AUTO_DECIDE, so I'll compute the recommendation, run the handoff, log the audit record and telemetry in one pass.", + "timestamp": "2026-10-04T10:28:54.126Z" + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "text": "Auto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune. Approved decisions: none. The plan is a fix to existing behavior with output formats, membership, aliases and metadata explicitly pinned unchanged, so the review's value lies in rigor on the sort key, both renderers and the enumeration-order tests rather than in expanding or cutting scope.\n\n**Mode provenance.** The `plan-ceo-review-mode` preference check returned AUTO_DECIDE, so no question was asked. The question log recorded user_choice and recommended both as the full name \"HOLD SCOPE\" with auto_decided true, on session 1545-1791109690-7adaeb43.\n\n**Step 0 evidence, kept brief**\n\n- Premise: diffing skill listings in scripts and reviews breaks on enumeration-order noise. Sorting at the listing path fixes the pain directly.\n- Leverage: reuse the existing listing path with one sort step before render. No rebuild.\n- Estimated changed files: 2, so no file-count threshold applies. Category is fix/refactor.\n- No approach decision was needed, so 0D was not called.\n\n**Deferred per your request**\n\n- Office Hours offer: no design doc found, offer skipped.\n- Setup: skill-start emitted a routing-rules instruction asking to edit and commit CLAUDE.md. Plan mode forbids that edit, and you said setup is handled separately, so it is deferred untouched.\n- Substantive review: HOLD SCOPE route is 0G, 0I, then Sections 1 to 11. Not started.\n\n**Records**\n\n- Plan file written with the mode decision and the open questions for the later review, at `.claude/plans/keep-this-draft-plan-frolicking-lake.md`.\n- Telemetry recorded with outcome abort, since the review workflow stopped by design after mode selection. Telemetry is off in this install, so the write is local only.\n- No durable learnings this session.\n\nSTATUS: DONE. Mode-only handoff complete. ExitPlanMode not called because the review workflow is intentionally paused here.", + "timestamp": "2026-10-04T10:29:24.679Z" + } + ] + }, + "tools": [ + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:28:09.022Z", + "toolUseId": "toolu_012JwPUvKc5i3fi7pzQgiEyV", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-skill-start --skill \"plan-ceo-review\" --model \"claude\"", + "description": "Run gstack skill-start preamble" + } + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:28:10.573Z", + "toolUseId": "toolu_012JwPUvKc5i3fi7pzQgiEyV", + "kind": "result", + "isError": false, + "content": "SKILL_START_PROTO: 1\nBRANCH: main\nPROACTIVE: true\nPROACTIVE_PROMPTED: yes\nSKILL_PREFIX: false\nREPO_MODE: unknown\nSESSION_KIND: interactive\nCONDUCTOR_SESSION: true\nACTIVATED: yes\nFIRST_LOOP_SHOWN: yes\nFIRST_TASK: \nLAKE_INTRO: yes\nTELEMETRY: off\nTEL_PROMPTED: yes\nSESSION_ID: 1545-1791109690-7adaeb43\nTEL_START: 1791109690\nEXPLAIN_LEVEL: default\nQUESTION_TUNING: true\nUPDATE_CHECK: false\nLEARNINGS: 0\nHAS_ROUTING: no\nROUTING_DECLINED: false\nVENDORED_GSTACK: no\nMODEL_OVERLAY: claude\nGSTACK_PLAN_MODE: active\nARTIFACTS_SYNC: off\n" + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:28:21.357Z", + "toolUseId": "toolu_01Lo2NJVntk2HCrHJV7s4Z1j", + "kind": "use", + "name": "Bash", + "input": { + "command": "echo \"--- remote\"; git remote get-url origin 2>&1; echo \"--- head\"; git symbolic-ref refs/remotes/origin/HEAD 2>&1; git rev-parse --verify origin/main 2>&1; echo \"--- log\"; git log --oneline -30; echo \"--- stash\"; git stash list; echo \"--- files\"; git ls-files; echo \"--- todos\"; grep -rn \"TODO\\|FIXME\\|HACK\\|XXX\" --exclude-dir=.git . 2>/dev/null | head -30; echo \"--- recent\"; git log --since=30.days --name-only --format=\"\" | sort | uniq -c | sort -rn | head -20", + "description": "Audit repo state: remote, base branch, history, files" + } + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:28:25.858Z", + "toolUseId": "toolu_01LLpuyDZpfMtkeDLcVxGoMf", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-context-recovery 2>&1; echo \"--- design doc\"; SLUG=$(~/.claude/skills/gstack/browse/bin/remote-slug 2>/dev/null || basename \"$(git rev-parse --show-toplevel 2>/dev/null || pwd)\"); BRANCH=$(git rev-parse --abbrev-ref HEAD 2>/dev/null | tr '/' '-' || echo 'no-branch'); DESIGN=$(~/.claude/skills/gstack/bin/gstack-design-doc-find \"$SLUG\" \"$BRANCH\"); [ -n \"$DESIGN\" ] && echo \"Design doc found: $DESIGN\" || echo \"No design doc found\"; echo \"--- learnings\"; _CROSS_PROJ=$(~/.claude/skills/gstack/bin/gstack-config get cross_project_learnings 2>/dev/null || echo \"unset\"); echo \"CROSS_PROJECT: $_CROSS_PROJ\"; ~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 2>&1 | head -20", + "description": "Recover context, check design doc and prior learnings" + } + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:28:29.796Z", + "toolUseId": "toolu_01CUoUKTjEbjBoXRzXnVBR86", + "kind": "use", + "name": "Bash", + "input": { + "command": "printf '%s' \"Select plan-ceo-review mode for deterministic skill-list ordering plan: SCOPE EXPANSION, SELECTIVE EXPANSION, HOLD SCOPE, or SCOPE REDUCTION\" | ~/.claude/skills/gstack/bin/gstack-question-preference --check \"plan-ceo-review-mode\" --summary-stdin; echo \"exit=$?\"; echo \"--- brain\"; SLUG=$(~/.claude/skills/gstack/bin/gstack-slug --get SLUG 2>/dev/null) || true; for d in product goals recent-decisions; do echo \"### $d\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get $d --project \"$SLUG\" 2>/dev/null || echo \"(no $d digest available yet)\"; done; echo \"### user-profile\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get user-profile 2>/dev/null || echo \"(no user-profile digest available yet)\"", + "description": "Check mode question preference and load brain context" + } + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:28:30.430Z", + "toolUseId": "toolu_014CMt1fkSaWpH9PTAEkgVXz", + "kind": "use", + "name": "Read", + "input": { + "file_path": "/PLAN.md" + } + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:28:30.867Z", + "toolUseId": "toolu_01Lo2NJVntk2HCrHJV7s4Z1j", + "kind": "result", + "isError": false, + "content": "--- remote\nerror: No such remote 'origin'\n--- head\nfatal: ref refs/remotes/origin/HEAD is not a symbolic ref\nb57232966b60b0a7c3cefc5aebcb5dac0b435124\n--- log\nb5…[elided: not consumed]" + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:28:30.974Z", + "toolUseId": "toolu_01LLpuyDZpfMtkeDLcVxGoMf", + "kind": "result", + "isError": false, + "content": "--- RECENT ARTIFACTS ---\n{\"skill\":\"plan-ceo-review\",\"event\":\"started\",\"branch\":\"main\",\"session\":\"1545-1791109690-7adaeb43\",\"ts\":\"2026-10-04T10:28:10.588Z\"}\n--- END ARTIFACTS ---\n--- design doc\nNo design doc found\n--- learnings\nCROSS_PROJECT: false" + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:28:31.122Z", + "toolUseId": "toolu_01CUoUKTjEbjBoXRzXnVBR86", + "kind": "result", + "isError": false, + "content": "AUTO_DECIDE\nexit=0\n--- brain\n### product\n(no product digest available yet)\n### goals\n(no goals digest available yet)\n### recent-decisions\n(no recent-decisions digest available yet)\n### user-profile\n(no user-profile digest available yet)" + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:28:31.130Z", + "toolUseId": "toolu_014CMt1fkSaWpH9PTAEkgVXz", + "kind": "result", + "isError": false, + "content": "1\t# Draft: deterministic skill-list ordering\n2\t\n3\tUsers compare skill listings in scripts and reviews. Make the existing listing\n4\tpath sort registered skill na…[elided: not consumed]" + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:28:58.435Z", + "toolUseId": "toolu_01LvfeR9remcV4fbVwEPDvmv", + "kind": "use", + "name": "Bash", + "input": { + "command": "echo \"--- handoff\"; ~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff \"HOLD SCOPE\" --decisions \"none\" --auto 2>&1; echo \"--- qlog\"; ~/.claude/skills/gstack/bin/gstack-question-log '{\"skill\":\"plan-ceo-review\",\"question_id\":\"plan-ceo-review-mode\",\"question_summary\":\"Select plan-ceo-review mode for deterministic skill-list ordering plan\",\"category\":\"routing\",\"door_type\":\"two-way\",\"options_count\":4,\"user_choice\":\"HOLD SCOPE\",\"recommended\":\"HOLD SCOPE\",\"auto_decided\":true,\"session_id\":\"1545-1791109690-7adaeb43\"}' 2>&1 && echo \"qlog ok\"; echo \"--- tel\"; ~/.claude/skills/gstack/bin/gstack-skill-end --skill \"plan-ceo-review\" --outcome abort --session-id \"1545-1791109690-7adaeb43\" --tel-start \"1791109690\" --used-browse no --error-message \"\" --failed-step \"\" 2>&1; echo \"tel exit=$?\"", + "description": "Run mode handoff, log mode decision, record telemetry" + } + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:29:11.007Z", + "toolUseId": "toolu_01RRksiyxRReq6E6AVqfYhZn", + "kind": "use", + "name": "Write", + "input": { + "file_path": "/plans/keep-this-draft-plan-frolicking-lake.md", + "content": "# plan-ceo-review: mode decision for \"deterministic skill-list ordering\"\n\n## Context\n\nReview target: PLAN.md draft (deterministic skill-list ordering). The user asked for\nthe review-mode decision only. Office Hours, setup (CLAUDE.md routing rules), and\nthe substantive Sections 1-11 review are deferred to a later invocation.\n\n## Step 0 evidence (brief)\n\n- 0A Premise: scripts and reviewers diff skill listings; nondeterministic order from\n directory enumeration produces spurious diffs. Sorting at the existing listing path\n fixes the pain directly, not a proxy.\n- 0B Leverage: reuse the existing listing/render path; add a sort step before render.\n No rebuild justified.\n- 0C Dream state: current = enumeration-order output; this plan = stable sorted\n output in text and JSON; 12-month ideal = all list-like CLI output stable and\n diff-friendly. The plan moves toward it.\n- 0D: no approach choice required.\n\n## 0E Mode selection\n\n- Estimated changed files: 2 (listing module, test file). Estimate.\n- Category: fix/refactor of existing behavior; membership, aliases, metadata and\n output formats pinned unchanged.\n- Recommended: HOLD SCOPE.\n- Provenance: `plan-ceo-review-mode` preference check returned AUTO_DECIDE\n (session 1545-1791109690-7adaeb43). Selected mode: HOLD SCOPE. No question asked.\n Question log recorded with user_choice and recommended both set to the full mode\n name \"HOLD SCOPE\", auto_decided true.\n\n## Open for the later substantive review (HOLD SCOPE route: 0G -> 0I -> Sections 1-11)\n\n- Sort key: byte order vs case-insensitive with a deterministic tie-break for\n mixed-case names (e.g. `Foo` vs `foo`); locale independence.\n- Apply the same sort to both text and JSON renderers, or sort once upstream.\n- Tests: mixed-case names; shuffled directory enumeration order yields identical output.\n- Alias handling: aliases must keep pointing at the same skill after sorting.\n" + } + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:29:12.065Z", + "toolUseId": "toolu_01LvfeR9remcV4fbVwEPDvmv", + "kind": "result", + "isError": false, + "content": "--- handoff\nAuto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune. Approved decisions: none.\n--- qlog\nqlog ok\n--- tel\nSKILL_END: recorded outcome=abort duration_s=62\ntel exit=0" + }, + { + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "timestamp": "2026-10-04T10:29:12.102Z", + "toolUseId": "toolu_01RRksiyxRReq6E6AVqfYhZn", + "kind": "result", + "isError": false, + "content": "File created successfully at: /plans/keep-this-draft-plan-frolic…[elided: not consumed]" + } + ], + "options": { + "skillName": "plan-ceo-review", + "sessionId": "94e3f44b-b75d-4187-a3e3-0fc829a7d75d", + "commandStartedAt": 1791109678876, + "now": 1791109765126, + "proseQuestionObserved": false, + "stateEvidence": { + "questionId": "plan-ceo-review-mode", + "preference": "never-ask", + "records": [ + { + "skill": "plan-ceo-review", + "question_id": "plan-ceo-review-mode", + "question_summary": "Select plan-ceo-review mode for deterministic skill-list ordering plan", + "category": "routing", + "door_type": "two-way", + "options_count": 4, + "user_choice": "HOLD SCOPE", + "recommended": "HOLD SCOPE", + "auto_decided": true, + "session_id": "1545-1791109690-7adaeb43", + "source": "agent", + "ts": "2026-10-04T10:29:12.065Z" + } + ] + } + } + }, + "screen": { + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37195203538-1-eval-slices-20/plan-ceo-review-1791109765127-W8O99c/terminal.visible.log", + "tail_chars": 6000, + "text": "hem. \rKep skill membership, aliass, medata and txt/JSON output formats unchanged.\rCover mixed-case names and differing irectoryenumeration ordrwithtests. \r \rThis draft is the revew targt,no the current brach. For this invocation, \r I want only the review-mode decision; I will handle optional Office Hours and \r setup separately, and run the substantive review later. No review mode has \r been selected. For this mode-only handoff, use the full selected mode name in \r the normal audit record's user_choice and recommended fields and in the skill's \r normal mode handoff line. A menu letter alone would not identify the chosen mode. \r\r●Received.Ihavethedraftplanfordeterministicskill-listorderingincontext,alongwithyourhandoffconstraints:\r review-mode decision only, no Office Hours or setup, full mode name in the audit record's user_choice and recommended\rfieldsandinthemodehandoffline.I'llwaitforyourslashcommandbeforestartinganythingorcallingtools.\r\r✻Cooked for 4s · done 10:27 AM\r\r❯ /plan-ceo-review \r\r●Startingtheplan-ceo-review preamble now, then I'll go straighttothemodedecisionasyouasked.\r Read 1 file, ran 4 shell commands \r\r● Context gathered. The mode preference check came back AUTO_DECIDE, so I'll compute the recommendation, run the\rhandoff, log the audit record and telemetry in one pass.\r\rRan 1 shell command \r \rUsers compare skill listings in scripts and reviews. Make the existing listing\rpath sort regitered skill names deterministically beforrendrng them. \rKeepkill membership, aiases, metadata and text/JSON output formats unchanged.\rCover mxed-cas namesnd differing directory enumeration order wihtests. \r \rThis draft is the review target, not the current branch. For this invocation,\rI want only the review-mode dcisin; I will handle optionalOffice Hurs ad\rsetup separaely, and run the substantive review ater. No review mode has \rben selected. For this mode-only hdoff, use the fullselected mode name in\rth normal audit ecord's user_choice and recommended fields anin the skill's\rnormal ode handoff line.A menu letter alone woul not identifythe choen mode.\r\r●Received.Ihavethedraftplanfordeterministicskill-listorderingincontext,alongwithyourhandoffconstraints:\r rview-modedecisiononly,no Ofice Hous or setup, ful mode name in the audi recrd's user_choice andrecommended\rfields an in the mode handoff line.I'll wait for your slash comand before starting anything or calligtools.\r\r✻Cooked for 4s · done 10:27 AM\r\r❯ /plan-ceo-review \r\r●Startingtheplan-ceo-reviewpreamblenow,thenI'llgostraighttothemodedecisionasyouasked.\r\rRead 1 file, ran 4 shell commands \r\r●Contextgathered.ThemodepreferencecheckcamebackAUTO_DECIDE,soI'llcomputetherecommendation,runthe\r hadoff, log the audit record and telemetryin one pass.\r\rRan 1 shell command \r\r●Updated plan\r ⎿  /plan to preview\r52\r✻\rSwirling…\r✽Swirling…\rSwirling…4\rSwirling…\rSwirling…\r✻Swirling…\r✶Swirling…\r*\r✢\rSwirling…\r·Swirling…\rSwirling…5\rSwirling…\r✢Swirling…\rSwirling…\r*Swirling…\r✶\r✻\rSwirling…\r✽Swirling…\rSwirling…6\rSwirling…\r✻Swirling…\r✶Swirling…\r*\r✢Swirling…\r·Swirling…7\rSwirling…\rSwirling…\r✢Swirling…\r*Swirling…\r✶\r✻Swirling…\r✽Swirling…8\rSwirling…\rSwirling…\r✻Swirling…\rSwirling…\r✶\r*\r✢Swirling…\rSwirling…\r·Swirling…9\rSwirling…\rSwirling…\r✢Swirling…\rSwirling…\r*\r✶\r✻Swirling…\r✽Swirling…\rSwirling…20\rSwirling…\rSwirling…\r✻Swirling…\rSwirling…\r✶\r*\r✢Swirling…\r·Swirling…\rSwirling…1\rSwirling…\rSwirling…\r✢Swirling…\rSwirling…\r*\r✶\r✻Swirling…\r✽Swirling…\rSwirling…2\rSwirling…\rSwirling…\r✻Swirling…\r✶Swirling…\r*\r✢Swirling…\r·Swirling…\rSwirling…3\rSwirling…\rSwirling…\r✢Swirling…\r*Swirling…\r✶Swirling…\r✻\r✽\r4\r✻\r✶\r*\r*Swirling… \r✢\r·\r5\r✢\r*\r✶\r✻\r✽\r✽Swirling… 6\r✽Swirling… a fix to existing behavior with output formats, membership, aliases and metadata explicitly pinned unchanged, so the\r review's value lies in rigor on the sort key, both renderers and the enumeration-order tests rather than in expanding\r or cutting scope.\r\r Mode provenance. The plan-ceo-review-mode preference check returned AUTO_DECIDE, so no question was asked. The\r question log recorded user_choice and recommended both as the full name \"HOLD SCOPE\" with auto_decided true, on\r session 1545-1791109690-7adaeb43.\r\r Step 0 evidence, kept brief\r\r - Premise: diffing skill listings in scripts and reviews breaks on enumeration-order noise. Sorting at the listing\r path fixes the pain directly.\r-Leverage:reusetheexistinglistingpathwithonesortstepbeforerender.Norebuild.\r - Estimatedchanged files:2, so nofil-count threshod applies. Category is fix/refactor.\r- No approach decsion waseeded, so 0D was not called.\r\rDeferred per your request\r\r-OfficeHoursoffer:nodesigndocfound,offerskipped.\r - Setup: skill-start emitted a routing-rules instruction asking to edit and commit CLAUDE.md. Plan mode forbids that\redit,andyousaidsetupishandledseparately,soitisdeferreduntouched.\r - Substanive review: HOLD SCOPE route is 0G, 0I, then Sections 1 to 11. Notstarted.\rRecords\r - Plan file witten with the mode decision and the open questions forthe later rview, at\r .claude/plans/keep-this-draft-plan-frolicking-lake.md\r-Telemetryrecordedwithoutcomeabort,sincethereviewworkflowstoppedbydesignaftermodeselection.Telemetry\r is off in this install,sothewriteislocalonly.\r-Nodurablelearningsthissession.\r\r STATUS: DONE. Mode-onlyhandoffcomplete.ExitPlanModenotcalledbecausethereviewworkflowisintentionallypaused\rhere.\r\r✻ Cooked for 1m 26s ·done 10:29 AM\r❯ \r← for agents" + } + } +} diff --git a/test/fixtures/detector-corpora/auto-decide-preserved/37198445662-t1-pass.json b/test/fixtures/detector-corpora/auto-decide-preserved/37198445662-t1-pass.json new file mode 100644 index 000000000..baffd9241 --- /dev/null +++ b/test/fixtures/detector-corpora/auto-decide-preserved/37198445662-t1-pass.json @@ -0,0 +1,323 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "auto-decide-preserved", + "detector": "test/helpers/native-auto-decide.ts#findNativeAutoDecision", + "screen_detector": "test/helpers/pty/classify.ts#classifyVisible", + "series_identity": "7678e372b3345f9b", + "source": { + "run": "37198445662", + "sha": "fd6854b5356431e5e115fca082b21bf237559558", + "lane": "periodic", + "artifact": "paid-slice-20-a1", + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37198445662-1-eval-slices-20/plan-ceo-review-1791113328798-6JFWyh/observation.json" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "classifyVisible accepted the sent AUTO_DECIDE line on screen. The archived native poll predates that message (last assistant message announces the handoff), and the handoff ran inside a compound command, so the native snapshot holds no current declaration.", + "screen_expected": "pass", + "native_expected": "refuse", + "derived": "options.stateEvidence rebuilt from the session's gstack-question-log argument with source \"agent\" and ts = that call's tool-result timestamp (the bound question-log.jsonl itself is not archived)", + "input": { + "native": { + "transcript": { + "status": "ready", + "calls": [], + "assistantMessages": [ + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "text": "Received. I have the draft plan for deterministic skill-list ordering as context, along with your constraints: this is a mode-only handoff, no review mode selected yet, use the full mode name in the audit record and handoff line. I'll wait for your slash command before starting or calling any tools.", + "timestamp": "2026-10-04T11:25:02.647Z" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "text": "I'll run the skill preamble first, then work through Step 0 to the mode decision only, as you asked.", + "timestamp": "2026-10-04T11:25:07.190Z" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "text": "I'll defer the CLAUDE.md setup prompt since you're handling it separately and plan mode blocks edits anyway—it'll reappear next run. Now gathering the audit context in one pass.\n\n", + "timestamp": "2026-10-04T11:25:34.122Z" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "text": "The fixture repo only has CLAUDE.md and PLAN.md—no remote, stash, TODOs, design doc, or prior context, and Aside isn't installed. Since mode preference is set to auto-decide, I'll skip that question and run the landscape check via WebSearch instead.\n\n", + "timestamp": "2026-10-04T11:26:36.166Z" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "text": "Confirmed the Reproducible Builds guidance aligns with the plan: avoid relying on readdir order and sort entries bytewise to prevent locale-based inconsistencies. Now writing the Step 0 ledger to the plan file while running the mode handoff and audit log in parallel.\n\n", + "timestamp": "2026-10-04T11:27:37.493Z" + } + ] + }, + "tools": [ + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:25:08.567Z", + "toolUseId": "toolu_01Wf6kvaTmd5c4BVKu8m21EF", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-skill-start --skill \"plan-ceo-review\" --model \"claude\"", + "description": "Run gstack skill preamble" + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:25:10.032Z", + "toolUseId": "toolu_01Wf6kvaTmd5c4BVKu8m21EF", + "kind": "result", + "isError": false, + "content": "SKILL_START_PROTO: 1\nBRANCH: main\nPROACTIVE: true\nPROACTIVE_PROMPTED: yes\nSKILL_PREFIX: false\nREPO_MODE: unknown\nSESSION_KIND: interactive\nCONDUCTOR_SESSION: true\nACTIVATED: yes\nFIRST_LOOP_SHOWN: yes\nFIRST_TASK: \nLAKE_INTRO: yes\nTELEMETRY: off\nTEL_PROMPTED: yes\nSESSION_ID: 1562-1791113109-05c92427\nTEL_START: 1791113109\nEXPLAIN_LEVEL: default\nQUESTION_TUNING: true\nUPDATE_CHECK: false\nLEARNINGS: 0\nHAS_ROUTING: no\nROUTING_DECLINED: false\nVENDORED_GSTACK: no\nMODEL_OVERLAY: claude\nGSTACK_PLAN_MODE: active\nARTIFACTS_SYNC: off\n" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:25:36.494Z", + "toolUseId": "toolu_01WfcV1ucLDK2iPD1sy2MgZP", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-context-recovery 2>&1; echo \"---REMOTE---\"; git remote get-url origin 2>&1; echo \"---HEAD---\"; git symbolic-ref refs/remotes/origin/HEAD 2>&1; git rev-parse --verify origin/main 2>&1; echo \"---LOG---\"; git log --oneline -30; echo \"---STASH---\"; git stash list; echo \"---TODO---\"; grep -r \"TODO\\|FIXME\\|HACK\\|XXX\" -l --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=.git . | head -30; echo \"---TOUCHED---\"; git log --since=30.days --name-only --format=\"\" | sort | uniq -c | sort -rn | head -20; echo \"---FILES---\"; git ls-files | head -100", + "description": "Recover context and audit repo history" + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:25:50.500Z", + "toolUseId": "toolu_01NLsv1JwFa2z6QM2DjX2iKh", + "kind": "use", + "name": "Bash", + "input": { + "command": "setopt +o nomatch 2>/dev/null || true\nSLUG=$(~/.claude/skills/gstack/browse/bin/remote-slug 2>/dev/null || basename \"$(git rev-parse --show-toplevel 2>/dev/null || pwd)\")\nBRANCH=$(git rev-parse --abbrev-ref HEAD 2>/dev/null | tr '/' '-' || echo 'no-branch')\nDESIGN=$(~/.claude/skills/gstack/bin/gstack-design-doc-find \"$SLUG\" \"$BRANCH\")\n[ -n \"$DESIGN\" ] && echo \"Design doc found: $DESIGN\" || echo \"No design doc found\"\necho \"---LEARNINGS---\"\n_CROSS_PROJ=$(~/.claude/skills/gstack/bin/gstack-config get cross_project_learnings 2>/dev/null || echo \"unset\")\necho \"CROSS_PROJECT: $_CROSS_PROJ\"\nif [ \"$_CROSS_PROJ\" = \"true\" ]; then\n { _LE=$(~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 --cross-project 2>&1 >&3 3>&-); _LR=$?; } 3>&1\nelse\n { _LE=$(~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 2>&1 >&3 3>&-); _LR=$?; } 3>&1\nfi\n[ \"$_LR\" = 0 ] || { _LE=${_LE%%$'\\n'*}; echo \"LEARNINGS: unavailable (${_LE:-exit $_LR})\"; }\necho \"---BRAIN---\"\nSLUG2=$(~/.claude/skills/gstack/bin/gstack-slug --get SLUG 2>/dev/null) || true\nfor d in product goals recent-decisions; do echo \"### $d\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get $d --project \"$SLUG2\" 2>/dev/null || echo \"_(no $d digest available yet)_\"; done\necho \"### user-profile\"; ~/.claude/skills/gstack/bin/gstack-brain-cache get user-profile 2>/dev/null || echo \"_(no user-profile digest available yet)_\"\necho \"---ASIDE---\"\n_gs_d() { if command -v gtimeout >/dev/null; then gtimeout 30 \"$@\"; elif command -v timeout >/dev/null; then timeout 30 \"$@\"\nelif command -v perl >/dev/null; then perl -e 'alarm(shift);exec(@ARGV)' 30 \"$@\"; else return 125; fi; }\n_A=aside; command -v aside >/dev/null || _A=$(command -v ~/.local/bin/aside)\nif [ \"${GSTACK_SKIP_ASIDE:-}\" = \"1\" ] || [ -z \"$_A\" ]; then\n echo \"NEEDS_ASIDE: ${GSTACK_PLATFORM:-$(uname)}\"\nelse\n _rc=0; _o=$(_gs_d \"$_A\" repl 'console.log(\"ASIDE_READY \" + pwd)' 2>&1) || _rc=$?\n case \"$_rc\" in\n 124|142) echo \"ASIDE_TIMEOUT: probe deadline exceeded\" ;;\n 125) echo \"ASIDE_UNAVAILABLE: bounded probe unavailable\" ;;\n 0) if printf '%s\\n' \"$_o\" | grep -q '^ASIDE_READY '; then echo \"READY: $_A\"\n else echo \"ASIDE_NOT_RUNNING: no readiness marker\"; fi ;;\n *) echo \"ASIDE_CLI_ERROR: exit $_rc; inspect aside --help locally\" ;;\n esac\nfi\necho \"---QPREF---\"\nprintf '%s' \"Select plan-ceo-review mode for deterministic skill-list ordering plan: SCOPE EXPANSION, SELECTIVE EXPANSION, HOLD SCOPE, or SCOPE REDUCTION\" | ~/.claude/skills/gstack/bin/gstack-question-preference --check \"plan-ceo-review-mode\" --summary-stdin; echo \"QPREF_EXIT: $?\"", + "description": "Check design doc, learnings, brain context, Aside, and mode preference" + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:25:51.497Z", + "toolUseId": "toolu_01Jg384kG7A8kPWZHjzUixKa", + "kind": "use", + "name": "Read", + "input": { + "file_path": "/PLAN.md" + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:25:53.678Z", + "toolUseId": "toolu_01QtvhyccyZBEYf9hbVcKxvb", + "kind": "use", + "name": "Bash", + "input": { + "command": "ls -la ; echo \"---TODOS---\"; cat /TODOS.md 2>&1 | head -50; echo \"---ETHOS---\"; sed -n '1,60p' ~/.claude/skills/gstack/ETHOS.md 2>&1", + "description": "List repo files, read TODOS.md and ETHOS.md" + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:25:54.352Z", + "toolUseId": "toolu_01WfcV1ucLDK2iPD1sy2MgZP", + "kind": "result", + "isError": false, + "content": "--- RECENT ARTIFACTS ---\n{\"skill\":\"plan-ceo-review\",\"event\":\"started\",\"branch\":\"main\",\"session\":\"1562-1791113109-05c92427\",\"ts\":\"2026-10-04T11:25:10.051Z\"}\n--- END ARTIFACTS ---\n---REMOTE---\nerror: No such remote 'origin'\n---HEAD---\nfatal: ref refs/remotes/origin/HEAD is not a symbolic ref\n9b45bea985d4de4cc1c1b2619d16faa32cd42f91\n---LOG---\n9b45bea Seed review plan\n---STASH---\n---TODO---\n---TOUCHED---\n 1 PLAN.md\n 1 CLAUDE.md\n---FILES---\nCLAUDE.md\nPLAN.md" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:25:54.644Z", + "toolUseId": "toolu_01NLsv1JwFa2z6QM2DjX2iKh", + "kind": "result", + "isError": false, + "content": "No design doc found\n---LEARNINGS---\nCROSS_PROJECT: false\n---BRAIN---\n### product\n_(no product digest available yet)_\n### goals\n_(no goals digest available yet)_\n### recent-decisions\n_(no recent-decisions digest available yet)_\n### user-profile\n_(no user-profile digest available yet)_\n---ASIDE---\nNEEDS_ASIDE: Linux\n---QPREF---\nAUTO_DECIDE\nQPREF_EXIT: 0" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:25:54.680Z", + "toolUseId": "toolu_01Jg384kG7A8kPWZHjzUixKa", + "kind": "result", + "isError": false, + "content": "1\t# Draft: deterministic skill-list ordering\n2\t\n3\tUsers compare skill listings in scripts and reviews. Make the existing listing\n4\tpath sort registered skill na…[elided: not consumed]" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:25:54.689Z", + "toolUseId": "toolu_01QtvhyccyZBEYf9hbVcKxvb", + "kind": "result", + "isError": false, + "content": "total 20\ndrwx------ 3 runner runner 4096 Oct 4 11:24 .\ndrwxr-xr-x 7 runner runner 4096 Oct 4 11:25 ..\ndrwxr-xr-x 8 runner runner 4096 Oct 4 11:24 .git\n-rw-r--r-- 1 runner runner 1132 Oct 4 11:24 CLAUDE.md\n-rw-r--r-- 1 runner runner 826 Oct 4 11:24 PLAN.md\n---TODOS---\ncat: /TODOS.md: No such file or directory\n---ETHOS---\n# gstack Builder Ethos\n\nThese are the principles that shape how gstack thinks, recommends, and builds.\nSkill preambles carry condensed forms of some of them (completeness, voice,\nsearch before building), and the Search Before Building section points here for\nthe full text. They reflect what we believe about building software in 2026.\n\n---\n\n## The Golden Age\n\nA single person with AI can now build what used to take a team of twenty.\nThe engineering barrier is gone. What remains is taste, judgment, and the\nwillingness to do the complete thing.\n\nThis is not a prediction — it's happening right now. 10,000+ usable lines of\ncode per day. 100+ commits per week. Not by a team. By one person, part-time,\nusing the right tools. The compression ratio between human-team time and\nAI-assisted time ranges from 3x (research) to 100x (boilerplate):\n\n| Task type | Human team | AI-assisted | Compression |\n|-----------------------------|-----------|-------------|-------------|\n| Boilerplate / scaffolding | 2 days | 15 min | ~100x |\n| Test writing | 1 day | 15 min | ~50x |\n| Feature implementation | 1 week | 30 min | ~30x |\n| Bug fix + regression test | 4 hours | 15 min | ~20x |\n| Architecture / design | 2 days | 4 hours | ~5x |\n| Research / exploration | 1 day | 3 hours | ~3x |\n\nThis table changes everything about how you make build-vs-skip decisions.\nThe last 10% of completeness that teams used to skip? It costs seconds now.\n\n---\n\n## 1. Boil the Ocean\n\n\"Don't boil the ocean\" was the right advice when engineering time was the\nbottleneck. That era is over. AI-assisted coding makes the marginal cost of\ncompleteness near-zero, so the old caution has quietly turned into an excuse.\nWhen the complete implementation costs minutes more than the shortcut — do the\ncomplete thing. Every time.\n\n**Ocean, lakes first:** The ocean is the destination — 100% test coverage for a\nmodule, full feature implementation, all edge cases, complete error paths. You\nget there one lake at a time: each lake is a boilable unit, not the ceiling.\n\"That's boiling the ocean\" is no longer a reason to ship a shortcut — boiling\nthe ocean is the goal. The only thing still out of scope is genuinely unrelated\nwork: a multi-quarter platform migration that has nothing to do with the task at\nhand. Flag that as separate scope. Boil everything else.\n\n**Completeness is cheap.** When evaluating \"approach A (full, ~150 LOC) vs\napproach B (90%, ~80 LOC)\" — always prefer A. The 70-line delta costs\nseconds with AI coding. \"Ship the shortcut\" is legacy thinking from when\nhuman engineering time was the bottleneck.\n\n**Anti-patterns:**\n- \"Choose B — it covers 90% with less code.\" (If A is 70 lines more, choose A.)\n- \"Let's defer tests to a follow-up PR.\" (Tests are the cheapest lake to boil.)\n- \"This would take 2 weeks.\" (Say: \"2 weeks human / ~1 hour AI-assisted.\")" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:26:36.168Z", + "toolUseId": "toolu_013JNLGRRstXZfcntr9ahtSc", + "kind": "use", + "name": "ToolSearch", + "input": { + "query": "select:WebSearch", + "max_results": 1 + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:26:36.171Z", + "toolUseId": "toolu_013JNLGRRstXZfcntr9ahtSc", + "kind": "result", + "isError": false, + "content": [ + { + "type": "tool_reference", + "tool_name": "WebSearch" + } + ] + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:26:36.532Z", + "toolUseId": "toolu_01GNkE1RxcdKUAUirZHVX6aL", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff --help 2>&1 | head -30", + "description": "Show mode handoff helper usage" + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:26:37.103Z", + "toolUseId": "toolu_01GNkE1RxcdKUAUirZHVX6aL", + "kind": "result", + "isError": false, + "content": "gstack-ceo-mode-handoff: unknown mode: --help" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:26:48.243Z", + "toolUseId": "toolu_019yK3jaeJGwVALSWypGcXzd", + "kind": "use", + "name": "WebSearch", + "input": { + "query": "deterministic sorted output CLI listing reproducible scripts filesystem enumeration order best practice" + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:27:25.578Z", + "toolUseId": "toolu_019yK3jaeJGwVALSWypGcXzd", + "kind": "result", + "isError": false, + "content": "Web search results for query: \"deterministic sorted output CLI listing reproducible scripts filesystem enumeration order best practice\"\n\nLinks: [{\"title\":\"Nix: …[elided: not consumed]" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:28:17.198Z", + "toolUseId": "toolu_01Si5mXg44rZxfPohQa5QFqc", + "kind": "use", + "name": "Write", + "input": { + "file_path": "/plans/keep-this-draft-plan-indexed-beaver.md", + "content": "# Working plan: deterministic skill-list ordering\n\nSource: PLAN.md in the fixture repo (commit 9b45bea \"Seed review plan\").\nReview: /plan-ceo-review, mode-only handoff requested by the user. The\nsubstantive review (Sections 1–11, outputs, report) is deliberately NOT run in\nthis invocation; the user will run it later.\n\n## Plan under review (verbatim from PLAN.md)\n\nUsers compare skill listings in scripts and reviews. Make the existing listing\npath sort registered skill names deterministically before rendering them.\nKeep skill membership, aliases, metadata and text/JSON output formats unchanged.\nCover mixed-case names and differing directory enumeration order with tests.\n\n## Review depth\n\nImplementation-ready (default; the user did not ask for strategy-only or a\nsingle narrow decision). Stated limits recorded from the plan:\n\n| Measure | Value | Unit | Prerequisite |\n|---|---|---|---|\n| Skill membership | unchanged | set of registered names | existing registry |\n| Aliases | unchanged | mapping | existing registry |\n| Metadata | unchanged | per-skill fields | existing registry |\n| Output formats | unchanged | text and JSON | existing renderers |\n| Ordering | deterministic | sort of registered names before render | existing listing path |\n| Tests | required | mixed-case names; differing directory enumeration order | test harness |\n\n## Pre-review system audit\n\n- Platform: unknown (no `origin` remote). Base branch: `main` (git-native fallback).\n- History: one commit, `9b45bea Seed review plan`. No stash. No TODO/FIXME/HACK markers.\n- Repo contents: `CLAUDE.md`, `PLAN.md` only. The listing code the plan modifies is\n not present in this fixture, so code-level leverage below is stated as unknowns\n for the implementation owner to confirm.\n- TODOS.md: absent. No prior deferrals.\n- Design doc: none found. User is handling /office-hours separately; offer not made.\n- Retrospective check: no prior review refactors or reverts (single commit).\n- DESIGN_SCOPE: none. CLI text/JSON listing only; no UI screens or components. Section 11 skips.\n- Taste calibration: not applicable (recommended mode is HOLD SCOPE, not an expansion mode).\n- Prior learnings: cross_project_learnings=false; 0 project learnings found.\n- Brain context: all four digests cold (no product, goals, recent-decisions, user-profile).\n- Landscape check: Aside not installed (NEEDS_ASIDE); ran one read-only WebSearch.\n - [Layer 1] Tried and true: never rely on readdir/listdir/glob order; apply an\n explicit sort; sort bytewise (codepoint or `LC_ALL=C`) so locale collation\n cannot change results. Source: reproducible-builds.org \"Stable order for inputs\"\n and \"Commandments\".\n - [Layer 2] Search results: Python glob deliberately stays unsorted (bpo-30461);\n systemd and mono both landed \"sort the input file list\" fixes; disorderfs and\n reprotest exist to shuffle/reverse directory order in CI to catch exactly this bug.\n - [Layer 3] First principles: the plan says \"deterministic\", not \"alphabetical\".\n Those differ for mixed case. Codepoint order (`Zeta` before `alpha`) is\n deterministic and locale-free but reads oddly; case-insensitive order needs a\n stable tiebreak (e.g. fold then raw) or two names differing only by case\n produce enumeration-dependent output again. This is the one real design\n choice inside the plan and belongs to the implementation review, not mode\n selection. No eureka to log.\n\n## Step 0 ledger\n\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| S0-PREMISE (user) | PLAN.md lines 3–6: listings are diffed by scripts and reviewers; nondeterministic order creates false diffs | Listing renders names in registry/enumeration order (unverified: code not in fixture) | Sort registered names before render; membership, aliases, metadata, formats unchanged | approved (plan as stated) | PLAN.md as handed over; no change proposed by this review |\n| S0-COLLATION (implementation owner) | Plan requires mixed-case test coverage; \"deterministic\" does not specify codepoint vs case-insensitive order | unknown | pending: codepoint sort vs case-folded sort with raw tiebreak | unresolved | Not a Step 0 approach choice; defer to Review Sections (implementation-ready depth). No approval requested here |\n| S0-MODE (user) | 0E recommendation; preference check `plan-ceo-review-mode` returned AUTO_DECIDE | none selected | HOLD SCOPE | approved (auto-decided) | Preference check exit 0 AUTO_DECIDE; see handoff provenance below |\n\n### 0A. Premise Challenge\n\nReal problem: two runs of the same listing can print the same skills in different\norder because the registry is populated from directory enumeration, which POSIX\nleaves unspecified. Target outcome: identical input produces byte-identical text\nand JSON listings, so `diff`, snapshot tests and review comments stop flagging\nnoise. Do-nothing cost: every script that compares listings needs its own\n`sort`, and JSON consumers that compare arrays cannot. The plan solves the pain\ndirectly (it fixes the producer, not each consumer). It is not a proxy fix.\n\n### 0B. Existing Code Leverage\n\n| Sub-problem | Reusable code | Note |\n|---|---|---|\n| Collect registered names | existing registry / listing path | Reuse as-is; plan says \"existing listing path\" |\n| Sort | standard library sort | Reuse ladder rung 2; no new dependency |\n| Render text/JSON | existing renderers | Unchanged by contract |\n| Simulate enumeration order in tests | existing test harness fixtures | Owner must confirm whether a fake directory walker or injected name list exists |\n\nNo rebuild proposed. The fixture repo contains no code, so the implementation\nowner must confirm the listing entry point and test helpers.\n\n### 0C. Dream State Mapping\n\n```\n CURRENT STATE THIS PLAN 12-MONTH IDEAL\n Listing order follows ---> Names sorted once at the ---> Every user-facing\n readdir order; scripts listing boundary; text enumeration (skills,\n and reviews see spurious and JSON stable; tests aliases, any future\n diffs between machines pin mixed-case and catalog) is sorted at\n and runs shuffled enumeration the producer; CI shuffles\n directory order to prove it\n```\n\nThe plan moves toward the ideal. It is the first and smallest lake: one boundary,\none sort, two test classes.\n\n### 0D. Alternatives\n\nNo required approach choice. The plan names a single, bounded approach (sort at\nthe existing listing path) with explicit invariants. The only open design question\n(S0-COLLATION) is an implementation detail inside that approach and is recorded\nas pending for the Review Sections. No new approach decision was needed.\n\n### 0E. Mode Selection\n\nPlanned changed files (estimate; code not in fixture): 1 source edit on the\nlisting path + 1 test file edit or addition = ~2 files. Category: fix/refactor\nof existing behavior with pinned invariants. Not >15 files; category is clear.\n\nRecommendation: HOLD SCOPE because the plan explicitly freezes membership,\naliases, metadata and both output formats, so the value here is rigor on the\none changed behavior (collation rule, locale independence, case-only collisions,\nshuffled-enumeration tests) rather than any new capability.\n\nPreference check: `gstack-question-preference --check plan-ceo-review-mode`\nexited 0 with AUTO_DECIDE. Per 0E step 3 the recommendation is selected without\na question. Selected mode: HOLD SCOPE.\n\n## Route after mode\n\nHOLD SCOPE route is 0G → 0I → Review Sections. Per the user's instruction this\ninvocation stops after the mode handoff. 0G, 0I and Review Sections 1–11,\nrequired outputs and the GSTACK REVIEW REPORT are pending for the later run.\n\n## NOT in scope\n\nNothing cut or deferred by this invocation.\n" + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:28:17.279Z", + "toolUseId": "toolu_01Si5mXg44rZxfPohQa5QFqc", + "kind": "result", + "isError": false, + "content": "File created successfully at: /plans/keep-this-draft-plan-indexe…[elided: not consumed]" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:28:20.265Z", + "toolUseId": "toolu_01RXKeSPQ9g8Bxx8y1hTfwAk", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff \"HOLD SCOPE\" --decisions \"none\" --auto; echo \"HANDOFF_EXIT: $?\"; echo \"---QLOG---\"; ~/.claude/skills/gstack/bin/gstack-question-log '{\"skill\":\"plan-ceo-review\",\"question_id\":\"plan-ceo-review-mode\",\"question_summary\":\"Select plan-ceo-review mode for deterministic skill-list ordering plan\",\"category\":\"routing\",\"door_type\":\"two-way\",\"options_count\":4,\"user_choice\":\"HOLD SCOPE\",\"recommended\":\"HOLD SCOPE\",\"auto_decided\":true,\"session_id\":\"1562-1791113109-05c92427\"}'; echo \"QLOG_EXIT: $?\"", + "description": "Run mode handoff and log auto-decided mode" + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:28:21.195Z", + "toolUseId": "toolu_01RXKeSPQ9g8Bxx8y1hTfwAk", + "kind": "result", + "isError": false, + "content": "Auto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune. Approved decisions: none.\nHANDOFF_EXIT: 0\n---QLOG---\nQLOG_EXIT: 0" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:28:32.356Z", + "toolUseId": "toolu_01DmtVBPoyAUHhn5N8613NYK", + "kind": "use", + "name": "Edit", + "input": { + "replace_all": false, + "file_path": "/plans/keep-this-draft-plan-indexed-beaver.md", + "old_string": "## Route after mode", + "new_string": "### Mode handoff and provenance\n\nHandoff line (from `gstack-ceo-mode-handoff \"HOLD SCOPE\" --decisions none --auto`, exit 0):\n\n> Auto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune. Approved decisions: none.\n\nProvenance: successful preference check (AUTO_DECIDE) on question_id\n`plan-ceo-review-mode`; recommendation HOLD SCOPE. Audit record written with\n`gstack-question-log` (exit 0): user_choice=HOLD SCOPE, recommended=HOLD SCOPE,\nauto_decided=true, session 1562-1791113109-05c92427. No question was asked.\n\n## Route after mode" + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:28:32.384Z", + "toolUseId": "toolu_01DmtVBPoyAUHhn5N8613NYK", + "kind": "result", + "isError": false, + "content": "The file /plans/keep-this-draft-plan-indexed-beaver.md has been …[elided: not consumed]" + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:28:34.202Z", + "toolUseId": "toolu_01BzKWpwYPt8hK4PSkzYDGJs", + "kind": "use", + "name": "Bash", + "input": { + "command": "~/.claude/skills/gstack/bin/gstack-skill-end --skill \"plan-ceo-review\" --outcome abort --session-id \"1562-1791113109-05c92427\" --tel-start \"1791113109\" --used-browse no --error-message \"\" --failed-step \"\" 2>/dev/null || true; echo \"SKILL_END_EXIT: $?\"", + "description": "Log skill-end telemetry for this invocation" + } + }, + { + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "timestamp": "2026-10-04T11:28:34.904Z", + "toolUseId": "toolu_01BzKWpwYPt8hK4PSkzYDGJs", + "kind": "result", + "isError": false, + "content": "SKILL_END: recorded outcome=abort duration_s=205\nSKILL_END_EXIT: 0" + } + ], + "options": { + "skillName": "plan-ceo-review", + "sessionId": "77c8d5e3-f546-4efd-b4bf-1eb8f149ba7e", + "commandStartedAt": 1791113102899, + "now": 1791113328797, + "proseQuestionObserved": false, + "stateEvidence": { + "questionId": "plan-ceo-review-mode", + "preference": "never-ask", + "records": [ + { + "skill": "plan-ceo-review", + "question_id": "plan-ceo-review-mode", + "question_summary": "Select plan-ceo-review mode for deterministic skill-list ordering plan", + "category": "routing", + "door_type": "two-way", + "options_count": 4, + "user_choice": "HOLD SCOPE", + "recommended": "HOLD SCOPE", + "auto_decided": true, + "session_id": "1562-1791113109-05c92427", + "source": "agent", + "ts": "2026-10-04T11:28:21.195Z" + } + ] + } + } + }, + "screen": { + "path": "shards/skill-e2e-auto-decide-preserved/pty-count/ci-37198445662-1-eval-slices-20/plan-ceo-review-1791113328798-6JFWyh/terminal.visible.log", + "tail_chars": 6000, + "text": "de → HOLDSCOPE(yourpreference).Changewith/plan-tune.Approveddecisions:none.Theplan\rfreezesskillmembership,aliases,metadataandbothoutputformats,sothereview'svalueisrigorontheonechanged\rbehavior, not new capability.Theestimatedchangetouchesabouttwofiles,afixtoexistingbehavior,whichmapsto\rHOLDSCOPEunderthemoderules.Nonewapproachdecisionwasneeded.\r\r What happened in Step 0\r - Mode selection di not need a questio. Your stored preferencefor the mode question retundAUTO_DECIDE, so the\r rcommendation was selected and the handoff ran. The audt record usesthe full modnamein bot the ser_choice\r and recmmended fields, wth uto_decided set totrue.\r-AuditfoundafixturewithonlyCLAUDE.mdandPLAN.md,noremote,nostash,noTODOs,nodesigndoc,andcoldbrain\r context. Thelistingcodeitselfisnotinthisrepo,socodeleverageisrecordedasunknownsforthe\r implementation owner.\r-Landscape checkranthroughWebSearchbecauseAsideisnotinstalled.ReproducibleBuildsguidancematchestheplan:\r never trust directoryenumerationorder,sortexplicitly,andsortbytewisesolocalecollationcannotchangethe\rresult.SeeStableorderforinputs(https://reproducible-builds.org/docs/stable-inputs/),Commandments\r (https://reproducible-builds.org/docs/commandments/),Pythongloborderingissue\r (https://bugs.python.org/issue30461),systemdsortfix(https://github.com/systemd/systemd/pull/31080),and\rdisorderfs(https://manpages.ubuntu.com/manpages/stonking/man1/disorderfs.1.html).\r - One pending design question recrded forthelaterreview,notdecidedhere.\"Deterministic\"and\"alphabetical\"\r differ for mixed case. Codepoint order is locale-free but puts uppercase names first. Case-insensitive orderneedsa\rraw-name tiebreak or two names differing only by case become enumeration-dependent again.\r\rArtifacts\r\r -\r\rWorkingplanwiththeStep0ledger,0Ato0Eevidence,andhandoffprovenanceissavedat\rRan 1 shell command \r\r●Updated plan\r ⎿  /plan to preview\r\rRan 1 shell command \r\r●Auto-decidedreviewmode→HOLDSCOPE(yourpreference).Changewith/plan-tune.Approveddecisions:none.Theplan\r freezes skill membership, aliases, metadataand both output formats,so the review's value is rgor o theonchged\rbehavior, notnw capability. The estimatedchange touches abut wofiles,a fx toexistingbehavir, which maps to\rHOLD SCOPE under the mode rules. No new approach decision ws needed.\r\rWhat happened in Step 0\r\r-Mode selectiondidnotneedaquestion.YourstoredpreferenceforthemodequestionreturnedAUTO_DECIDE,sothe\r recommendationwasselectnd the handoff ran. The audit record usesthe full modename in both the user_choic\rand recommended fields, with auto_decided set totrue.\r-Audit fund a fixture with only CLAUDE.mandPLAN.md,noremote,nostash,noTODOs,nodesigndoc,andcoldbrain\r context. The listing code itself is not in this repo, so code lverage i recrded asunknowns for the\rimplmentation owner.\r-Landscape check ran throughWebSearchbecauseAsideisnotinstalled.ReproducibleBuildsguidancematchestheplan:\r never trust directory enmeration order, sortexplictly, and sort bytewise so locale collaton anno change the\rrsult. SeeStable ordr fo inputs (http://rproducible-builds.org/docs/stable-inputs/), Commandmens\r(https://rproducible-builds.org/docs/commandmnts/), Python glob ordering issu\rbugs.python.org/isue30461), system sort fix (https://github.com/sytmd/systemd/pull/31080),and\rdisorderfs (https://manpage.ubuntu.com/manpages/stonking/man1disorderfs.1.html).\r-One pending design question is recorded for th laer review, not decided here. \"Deterministic\"and\"alphabetical\"\r differ for mixed case. Codepoint order is locale-frebut puts upperas names firs. Case-insensitive order needsa\rraw-name tiebrak or two names differing only by casecome enmeration-depndent again.\r\rArtifacts\r\r-WorkingplanwiththeStep0ledger,0Ato0Eevidence,andhandoffprovenanceissavedat\r .claude/plans/keep-this-draft-plan-indexed-beaver.md under te with-skills home. PLAN.md inthefixtureis\r●Updated plan\r ⎿  /plan to preview\r\r Ran 1 shellcommand \r●Auto-decided review mode→HOLDSCOPE(yourpreference).Changewith/plan-tune.Approveddecisions:none.Theplan\rfreezesskillmembership,aliases,metadataandbothoutputformats,sothereview'svalueisrigorontheonechanged\r behavior, not new capability. The estimatedchange touches bout twofies, a fix to existing behavior, whic maps to\rHOLD SCOPE under the moderule. No new pproach decision was needed.\r\rWhat happened in Step 0\r- Mode selectio did notneedaquestion.YourstoredpreferenceforthemodequestionreturnedAUTO_DECIDE,sothe\rrecommendationwasselectedandthehandoffran.Theauditrecordusesthefullmodenameinboththeuser_choice\r and recommended fields, with auto_decided set true.\r-Audit found a fixtur wih onlyCLAUDE.md and PLAN.md, no remote,no tash, no TODOs, no design doc, and cold brain\rcontext. The listng code itself is not in this repo, socodeleverageisrecordedasunknownsforthe\r implementation owne.\r-Landscape check ra through WbSearch because Aside is not installed. ReproducibleBuilds guidancematchestheplan:\rnever trus directoyenumerationorder,sortexplicitly,andsortbytewisesolocalecollationcannotchangethe\r result. See Stable order for inputs (https://reproducible-builds.org/docs/stable-inpts/), Commandmens\r(https://reproduible-builds.org/dcs/commandmnts/), Ptho glb ordering isue\r(https://bugs.python.org/issue30461), systemd sot fix (https://github.om/systmd/systemd/pull/31080),and\rdisorderfs (https://manpageubuntu.com/anpags/tonking/man1/disfs.1.html).\r-One pending design question is recorded for the later review, not decided here. \"Deterministic\" and \"alphabetical\"\rffer formixed case. Codpoint order is locale-free but puts uppecase names first.Case-insensitiveorderneedsa\r raw-name tiebreak or two namediffering nly by casebcome eumeraton-dependent again.\r\rArtifacts\r- WorkingplanwiththeStep0ledger,0Ato0Eevidence,andhandoffprovenanceissavedat\r.claude/plans/keep-this-draft-plan-indexed-beaver.mdunderthewith-skillshome.PLAN.mdinthefixtureis\r untouched.\r-\rSkill-endtelemetryrecordedthisinvocationasabort,meaningstoppedbeforethereviewbyyourrequest.Telemetry" + } + } +} diff --git a/test/fixtures/detector-corpora/auto-decide-preserved/inventory.json b/test/fixtures/detector-corpora/auto-decide-preserved/inventory.json new file mode 100644 index 000000000..7a096f7c1 --- /dev/null +++ b/test/fixtures/detector-corpora/auto-decide-preserved/inventory.json @@ -0,0 +1,82 @@ +[ + { + "run": "37174266054", + "lane": "periodic", + "outcome": "passed", + "series_identity": "6c9ea4bac7c0bc81", + "artifact": "paid-slice-20-a1", + "available": true, + "entry": "37174266054-t1-pass.json", + "reason": null + }, + { + "run": "37176837432", + "lane": "periodic", + "outcome": "failed", + "series_identity": "6c9ea4bac7c0bc81", + "artifact": "paid-slice-20-a1", + "available": true, + "entry": "37176837432-t1-fail.json", + "reason": null + }, + { + "run": "37179171083", + "lane": "periodic", + "outcome": "failed", + "series_identity": "6c9ea4bac7c0bc81", + "artifact": "paid-slice-20-a1", + "available": true, + "entry": "37179171083-t1-fail.json", + "reason": null + }, + { + "run": "37182865432", + "lane": "periodic", + "outcome": "failed", + "series_identity": "e3bf31932dcbfd91", + "artifact": "paid-slice-20-a1", + "available": true, + "entry": "37182865432-t1-fail.json", + "reason": null + }, + { + "run": "37186854666", + "lane": "periodic", + "outcome": "passed", + "series_identity": "7678e372b3345f9b", + "artifact": "paid-slice-20-a1", + "available": true, + "entry": "37186854666-t1-pass.json", + "reason": null + }, + { + "run": "37193478719", + "lane": "periodic", + "outcome": "passed", + "series_identity": "7678e372b3345f9b", + "artifact": "paid-slice-20-a1", + "available": true, + "entry": "37193478719-t1-pass.json", + "reason": null + }, + { + "run": "37195203538", + "lane": "periodic", + "outcome": "passed", + "series_identity": "7678e372b3345f9b", + "artifact": "paid-slice-20-a1", + "available": true, + "entry": "37195203538-t1-pass.json", + "reason": null + }, + { + "run": "37198445662", + "lane": "periodic", + "outcome": "passed", + "series_identity": "7678e372b3345f9b", + "artifact": "paid-slice-20-a1", + "available": true, + "entry": "37198445662-t1-pass.json", + "reason": null + } +] diff --git a/test/fixtures/detector-corpora/plan-ceo-section-loading/37174266054-t1-pass.json b/test/fixtures/detector-corpora/plan-ceo-section-loading/37174266054-t1-pass.json new file mode 100644 index 000000000..c7aa34165 --- /dev/null +++ b/test/fixtures/detector-corpora/plan-ceo-section-loading/37174266054-t1-pass.json @@ -0,0 +1,20 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-ceo-section-loading", + "detector": "test/helpers/ceo-stale-fill-decision.ts#hasApprovedStaleFillDecision || test/helpers/ceo-section-loading-fixture.ts#hasStaleFillRaceFinding", + "series_identity": "a7f8ef2d01d4716d", + "source": { + "run": "37174266054", + "sha": "a8ace39030dd92745b45ac16e5d733e63c936c0f", + "lane": "periodic", + "artifact": "paid-slice-6-a1", + "path": "shards/skill-e2e-plan-ceo-review-section-loading/_partial-e2e.json#/tests/0/output" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "Genuine pass via the stale-row fallback: the R1 ledger row was left 'unresolved' while the decision record selected A, so the ledger reader refuses, and the asserted stale-fill trace satisfies hasStaleFillRaceFinding.", + "input": { + "report": "# Plan: cache profile summaries in one process\n\n## Measured problem and accepted scope\nThe existing profile-summary service has one active process. A one-week trace\nshows repeated reads of about 900 hot keys: DB CPU is 70%, with read p95 120 ms.\nAdd a process-local LRU wrapper to the existing repository. Acceptance targets\nare at least 60% cache hits, DB CPU below 50%, and read p95 below 60 ms, with the\nexisting error-rate and correctness SLOs unchanged. This is an internal backend\nchange with no UI, API, schema, pricing, or developer onboarding change.\n\n## Existing contracts retained\n- All reads and writes use this repository in the same process; there are no\n external DB writers. Multi-process operation remains unsupported and startup\n rejects that configuration while caching is enabled.\n- These surrounding contracts are accepted fixture facts, supplied by the\n existing repository, cache adapter and rollout controller. Preserve them;\n review the new wrapper ordering below against them.\n- Authentication and authorization run before repository access. Keys encode\n the authenticated tenant ID and validated profile ID without ambiguity.\n Values are immutable profile-summary DTOs; secrets and cache keys are never\n logged. Cached results cannot bypass authorization.\n- The existing LRU adapter supports 1000 entries, a 16 MiB byte cap, and a\n 30-second TTL. Recorded hot data fits those limits. repository.read returns\n an immutable absent-result DTO for a missing record, never undefined. The\n adapter recognizes that DTO in cache.set, stores an internal sentinel with a\n 10-second TTL, and cache.get decodes it back to the same absent-result DTO.\n The internal sentinel cannot escape the adapter; undefined means a cache miss.\n- Cache operations are synchronous and atomic in the single JS event loop.\n On any cache failure the existing adapter bypasses the cache until an empty\n cache is reinitialized; repository errors keep the current typed API error\n mapping. The existing per-key single-flight wrapper sits inside\n repository.read, coalesces simultaneous store reads and releases on failure.\n A committed repository.write retires that key's old read cohort before its\n promise resolves. A later repository.read starts a fresh cohort; a rejected\n write leaves the cohort unchanged. Already-started readers may finish with\n their earlier snapshot. This admission rule does not inspect cache fills.\n- The repository uses an in-process transactional store, with no network\n transport between this wrapper and the store. repository.write is atomic:\n a resolved promise means committed, and every\n rejected promise guarantees no commit; its transaction rolled back before\n rejection. Existing contract tests exercise that guarantee.\n- Consistency is measured at the public wrapper boundary. A write completes\n when writeProfile's promise fulfills after cache.delete, not when\n repository.write commits or resolves. A read begins when readProfile is\n invoked. Reads that overlap an unfinished writeProfile may return an earlier\n snapshot, including reads begun after the store commit but before the wrapper\n promise fulfills. Every read begun after that write completes must\n observe the committed version. TTL expiry is not a substitute for this rule.\n\n## Proposed wrapper integration\nKeep the current read-through repository interface and shared adapters. These\nwere the originally proposed read/write ordering rules; the original text proposed\nno additional version checks or coordination between a cache fill and a write.\n**Amended by CEO review decision R1 (D1 → A): see \"Amendment R1\" below; the\nsketch stays as the baseline being amended, not the approved ordering.**\n\n```javascript\nasync function readProfile(key) {\n const cached = cache.get(key);\n if (cached !== undefined) return cached;\n const value = await repository.read(key);\n cache.set(key, value);\n return value;\n}\n\nasync function writeProfile(key, update) {\n const saved = await repository.write(key, update);\n cache.delete(key);\n return saved;\n}\n```\n\n### Amendment R1 (accepted, D1 → A): stale-fill guard\nRequired guarantee: a cache fill produced by a readProfile that began before a\nwriteProfile completed must never be observable by a readProfile begun after that\nwrite completed. The wrapper must detect, synchronously at fill time, that a write\nfor the same key completed after the read began, and skip (or neutralize) that\nfill; the read still returns its snapshot (allowed by lines 44-46). Feasibility:\ncache operations and the JS event loop are synchronous and atomic (line 28), so a\nper-key write-completion marker checked in the same tick as cache.set cannot\nrace with cache.delete; no new coordination with repository internals is needed.\nTradeoffs: one extra branch in readProfile and one bookkeeping step in\nwriteProfile; a racing key costs one additional DB read; the per-key marker is\nbounded by live-read keys and must be released when its readers finish. Exact\ndata structure and code belong to engineering planning. Added telemetry: one\n`stale_fill_suppressed` counter on the existing dashboard, no key labels.\nDeterministic regression scenarios (existing harness, controlled pause/release):\n1. read begins → write commits+completes → read resumes: next read sees new version.\n2. write completes → read begins → read fills: fill is kept (no false suppression).\n3. missing record: read sees absent → create completes → read resumes: next read sees record, no 10 s absent sentinel.\n4. rejected write during a pending read: fill is kept; cached value equals the unchanged store.\n5. rejected read: no fill, no marker leak; a later read fills normally.\n6. two overlapping writes then a pending read resumes: no stale fill; next read sees the last committed version.\n7. controller instance change mid-read: old-instance read cannot fill the new instance (existing contract), marker state does not cross instances.\n\n## Verification and rollout\nExisting repository contract tests cover tenant isolation, key validation,\nabsence, DB failures, authorization, and startup rejection of multi-process\noperation while caching is enabled. New wrapper tests cover hit/miss,\neviction and byte limits, TTL, adapter-failure fallback, successful-write\ninvalidation, failed-write preservation, and concurrent-miss coalescing, plus the Amendment R1 regression scenarios 1-7.\nThe rollout uses the existing runtime feature flag: enable for 10% of keys,\nthen 50%, then all keys after one healthy hour at each stage. Monitor hit/miss,\neviction, cache bytes, fallback errors, DB CPU, and read p95 without raw IDs.\nThe existing controller uses one shared key-selection predicate for reads and\nwrites. On any enable, disable or percentage change, it stops admitting work,\nawaits every admitted old-instance write, then publishes a new wrapper/cache\ninstance with a fresh single-flight cohort before admitting new work. Old reads\nretain their old instance and cannot fill the new one. Disabled instances\nbypass the cache on both paths. Tests cover the old-writer/new-reader ordering,\nall those transitions and predicate parity. This lifecycle isolation does not coordinate\nan ordinary DB write with a cache fill in the same active instance.\nExisting dashboards and runbooks cover these metrics. Before each stage, verify\nthat alerts page the service owner on any correctness/error-SLO breach, read\np95 above 120 ms for five minutes, or cache bypass persisting for one minute.\nHit rate is hits / (hits + misses) among requests admitted to the cache path;\nflag-excluded or adapter-bypassed requests are tracked separately, not as misses.\nDB CPU and read p95 are service-wide metrics, including bypassed requests.\nAt the 10% and 50% stages, a healthy hour requires at least 60% admitted-request\nhits, unchanged correctness/error SLOs, no alerts, and aggregate DB CPU/read p95\nno worse than their 70%/120 ms pre-rollout baselines. At 100%, the original\nabsolute acceptance targets (DB CPU below 50%, read p95 below 60 ms, hits at\nleast 60%) must all hold with unchanged correctness/error SLOs and no alerts.\nAny breach disables the flag immediately; the runbook records the incident,\nrollback and criteria for resuming. These are existing\nrollout-controller and telemetry contracts, not proposed wrapper additions.\nCold starts remain within the existing DB capacity. The service owner monitors\nthe rollout and records the results against the acceptance targets.\n\n## Out of scope\nDistributed caching, cross-process coherence, prewarming, changing consistency\nsemantics, or adding new product surfaces. The repository interface preserves a\nfuture replacement path without introducing a general cache framework now.\n\n## Author's review and acceptance requirements\nThis is a full CEO scope and feasibility review. The author has approved the\nretained contracts, limits, rollout and acceptance targets above. Evaluate the\nproposed wrapper against them; the wrapper itself remains unapproved. An actual\ncontradiction or missing proof must be reported and resolved, not assumed away.\nFor a demonstrated gap, amend the plan with the required guarantee, a feasible\nremedy, its tradeoffs and deterministic regression scenarios. Those repairs\nand their required verification are within the requested scope. The exact data\nstructures, full function bodies and executable test code belong to subsequent\nengineering planning; do not select or implement them during this review when\nthe required behavior and feasibility can already be established.\n\nUse the existing deterministic repository-contract test harness. Required wrapper\nacceptance includes both completion orders of overlapping reads and writes,\nmissing-record creation, rejected reads/writes, overlapping writes, and isolation\nacross controller instance changes. Use existing telemetry to record any added\nbranch on the current dashboard with no key labels; no new alert threshold or metric\nproject is requested. These are future acceptance requirements, not tests already\nimplemented or passing. Preserve all 11 review outcomes, required registries,\ndiagrams, tasks, completion summary and the full GSTACK REVIEW REPORT.\n\n---\n\n# CEO REVIEW (plan-ceo-review, HOLD SCOPE)\n\n## Step 0 — Setup, scope and decision ledger\n\n**Startup.** Depth: implementation-ready for behavior and verification; data structures, function bodies and executable tests stay with engineering (PLAN.md lines 117-120). Working plan and output: this PLAN.md. Base branch: `main` (git commands prohibited this run; taken from supplied git status). Skipped per run rules: system audit, design-doc/office-hours offer, landscape search, brain context, prior learnings, telemetry. Outside review: `codex_reviews: disabled` in `.gstack-section-state-Ve1QVj/config.yaml`, so the outside-review step and its native fallback are skipped and coverage is reported as disabled. DESIGN_SCOPE: none (no UI, line 9). Retrospective: no prior review artifacts in this fixture.\n\n**0A Premise.** Real problem: ~900 hot keys re-read in one process; DB CPU 70%, read p95 120 ms (lines 4-6). Target: ≥60% hits, DB CPU <50%, p95 <60 ms, SLOs unchanged (lines 7-9). Do-nothing cost: DB headroom keeps shrinking and latency stays 2x the target. The plan attacks the pain directly (removes the repeated reads at the repository boundary); not a proxy.\n\n**0B Leverage.** Reused as-is: repository and typed error mapping, LRU adapter with limits and absent sentinel (lines 22-27), adapter failure bypass (28-30), per-key single-flight (30-36), transactional store (37-41), rollout controller and flag (77-87), telemetry/dashboards/runbooks (88-101), contract test harness (122). New: wrapper read/write ordering only. No rebuild.\n\n**0C Dream state.**\n```\n CURRENT STATE THIS PLAN 12-MONTH IDEAL\n every read hits DB; 70% CPU ---> process-local LRU on repo ---> same repo interface, swappable\n p95 120 ms, 900 hot keys ≥60% hits, <50% CPU, p95 <60 ms cache backend if multi-process\n single process correctness SLOs unchanged ever arrives (line 107-108)\n```\nPlan moves toward the ideal without adding a cache framework.\n\n**0E Mode.** Explicit user instruction: HOLD SCOPE. No mode question asked; provenance: user instruction \"Hold the current scope (HOLD SCOPE mode)\". Route: 0G → 0I. File count (estimate): 2-3 changed files (wrapper module, wrapper tests, flag/controller wiring); well under the HOLD complexity threshold.\n\n**Limits recorded.** LRU 1000 entries; 16 MiB bytes; 30 s TTL; 10 s absent-sentinel TTL; rollout 10%/50%/100% with one healthy hour each; alerts: SLO breach, p95 >120 ms for 5 min, bypass persisting 1 min; targets ≥60% hits (admitted requests), DB CPU <50%, p95 <60 ms (service-wide). None changed by this review.\n\n### Decision ledger\n\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| R1 — plan author (wrapper ordering) | Lines 46-48: every read begun after writeProfile completes must observe the committed version; TTL is not a substitute. Lines 52-53 and 55-68: readProfile fills `cache.set(key, value)` after an awaited read with no fill/write coordination. Lines 35-36: single-flight does not inspect cache fills. Lines 86-87: controller does not coordinate an ordinary write with a fill in the same instance. | Wrapper as proposed; a read that began before a write can fill a stale snapshot after the write completes (trace below) | A) stale-fill guard requirement + regression scenarios; B) keep as-is (TTL-bounded); C) leave gap to engineering planning | unresolved | — |\n\n**Demonstrated gap trace (R1).** All steps use only accepted contracts:\n```\nt0 R1: readProfile(k) -> cache.get(k) miss -> await repository.read(k) [snapshot v0 in flight]\nt1 W : writeProfile(k) -> repository.write commits v1 -> cache.delete(k)\n -> promise fulfills [write COMPLETE, line 43-44]\nt2 R1 resumes with v0 (already-started reader, line 35) -> cache.set(k, v0) [stale fill, no guard: line 52-53]\nt3 R2: readProfile(k) begun after t1 -> cache.get(k) hits v0 [violates line 46-48 for up to 30 s;\n 10 s if v0 was the absent sentinel]\n```\nThis is a contradiction inside the plan text, not an unavailable implementation. Missing-record creation is the same race with the absent-result DTO (lines 23-27).\n\n## currentDecision (R1)\nCommitment comparison:\n```text\nCommitment | Source/approval or pending | Current | A | B | C\nRead begun after write completion observes committed version (lines 46-48) | approved fixture contract | required | preserved via stale-fill guard | weakened to TTL-bounded | required but left unaddressed in plan\nCoordination between cache fill and write (lines 52-53, 86-87) | pending (wrapper unapproved, line 113) | none | guarantee required; structure chosen by engineering | none | none; flagged as TODO\nAdapter limits 1000 / 16 MiB / 30 s / 10 s sentinel (lines 22-26) | approved | unchanged | unchanged | unchanged | unchanged\nSingle-flight and write-atomicity contracts (lines 30-41) | approved | unchanged | unchanged | unchanged | unchanged\nRegression scenarios: both completion orders, missing-record creation, rejected reads/writes, overlapping writes, instance isolation (lines 122-125) | approved acceptance requirement | required, not yet specified | specified deterministically | read-after-write scenario would fail | deferred\nTelemetry for added branch on current dashboard, no key labels (lines 125-126) | approved | n/a | one counter for suppressed stale fills | n/a | n/a\nAcceptance targets and rollout stages (lines 7-9, 77-101) | approved | unchanged | unchanged | unchanged | unchanged\n```\n\nQuestion: D1 — R1: How should the plan resolve the stale cache fill that breaks read-after-write visibility?\nProject/branch/task: main — cache profile summaries in one process; CEO review of the proposed wrapper ordering.\nELI10: A read that starts before a write can finish after the write and put the old answer back in the cache. Anyone reading afterward gets the old profile for up to 30 seconds. The plan promises that never happens once a write completes, and says TTL does not count as a fix.\nStakes if we pick wrong: users see a profile they just updated revert, the correctness SLO breaks under normal traffic, and the rollout alert disables the flag.\nRecommendation: A because it restores the retained contract with a small synchronous guard inside the single event loop, keeps every limit unchanged, and the regression scenarios are deterministic on the existing harness.\nCompleteness: A=10/10, B=3/10, C=5/10\nPros / cons:\nA) Require stale-fill guard + regression scenarios (recommended)\n ✅ Restores lines 46-48 exactly: no read begun after a completed write can observe an older fill\n ✅ Feasible with existing contracts: cache ops are synchronous/atomic (line 28), so a per-key write-completion marker checked at fill time cannot race\n ❌ Adds one branch and one counter to the wrapper; a fill is skipped after a racing write, costing one extra DB read for that key\nB) Keep wrapper as proposed; accept TTL-bounded staleness\n ✅ Zero implementation work beyond the plan as written\n ✅ Simplest code path for the hit/miss happy case\n ❌ Contradicts line 48 and weakens the correctness SLO; not authorizable under the author policy\nC) Leave the gap as an engineering-planning TODO without amending the plan\n ✅ Keeps the CEO review short and defers structure choice to engineering\n ✅ No plan text changes beyond a TODO entry\n ❌ Approves a plan that contradicts its own retained contract; the required regression scenario has no required behavior to test against\nNet: A fixes the contract at the cost of one small guard; B and C both leave a known correctness bug in an approved plan.\nHeader: R1 stale fill\nA) Require stale-fill guard + regression scenarios (recommended)\nEffort S, risk low. Reuse: synchronous atomic adapter ops, single-flight cohorts, existing harness; coverage 10/10. Amends the plan with the required guarantee (a fill from a read begun before a write completed is never observable by a read begun after that completion), its feasibility, tradeoffs, deterministic regression scenarios, and one suppressed-fill counter on the existing dashboard. Engineering picks the exact structure. ✅ Restores lines 46-48 exactly. ✅ Feasible with existing synchronous atomic cache ops. ❌ One extra DB read per racing key.\nB) Keep wrapper as proposed; accept TTL-bounded staleness\nEffort S with zero implementation work, risk high; coverage 3/10. Reads begun after a completed write may see a stale value for up to 30 s (10 s for absent). ✅ Zero additional work. ✅ Simplest code path. ❌ Contradicts line 48; weakens correctness, which the author policy forbids.\nC) Leave the gap as an engineering-planning TODO without amending the plan\nEffort S with zero implementation work now, risk medium; coverage 5/10. Records a TODO but no required guarantee. ✅ Short review. ✅ No plan text changes. ❌ Approves a self-contradictory plan with an untestable acceptance scenario.\n\n### Decision record R1 (D1)\nSelected: **A) Require stale-fill guard + regression scenarios**. Authority: run-rule author policy (\"Authorize complete remedies and required verification that restore its retained contracts and acceptance targets; use the recommended option\"); PLAN.md lines 114-117 (\"amend the plan with the required guarantee, a feasible remedy, its tradeoffs and deterministic regression scenarios\"). Auto-decided; no human present. Exact scope: amend the plan with the guarantee, feasibility, tradeoffs, seven regression scenarios and one dashboard counter; no data-structure or code selection, no limit, consistency or alert changes. Ledger R1 status → approved. Applied: \"Amendment R1\" and the amended wrapper paragraph in the plan body; scenarios added to the verification list.\n\n**0G HOLD SCOPE checks.** Complexity: 2-3 files, 0 new services, 1 wrapper module → no challenge. Minimum change for the goal: wrapper + R1 guard + tests; nothing deferrable without blocking acceptance (lines 122-125 require the overlapping-order scenarios). Invariants and acceptance criteria kept unchanged. No deferral proposals, so no defer/keep questions.\n\n**0I Temporal interrogation.**\n```\nHOUR 1 foundations: adapter sentinel/undefined semantics (lines 23-27); write-completion boundary (43-44); flag predicate parity (80-81)\nHOUR 2-3 core logic: R1 marker lifetime (release when readers finish); absent-DTO fill is subject to the same guard; rejected write must not mark completion\nHOUR 4-5 integration: controller rotation awaits old writes but not old reads (82-84); marker state must be per instance; bypass mode must skip marker bookkeeping cleanly\nHOUR 6+ tests: pause/release points inside repository.read and repository.write on the existing harness; hit-rate metric must exclude suppressed fills from \"hits\" and count them as misses that were admitted\n```\nFeasibility blockers: none after R1. Pending (engineering): marker data structure, exact release point, test fixture names. Effort: human ~2 days / CC+gstack ~40 min.\n\n## Review Sections (HOLD SCOPE, implementation-ready)\n\nCurrent scope: mode HOLD SCOPE (user instruction). Governing rows: R1 approved (D1 → A, author-policy auto-decision; scope: guarantee + 7 scenarios + 1 counter). Accepted: original plan + Amendment R1. Deferred: none. Rejected: D1 options B and C. Pending (engineering): marker structure, release point, fixture names.\n\n### Section 1: Architecture Review — 1 issue (F1 = R1, resolved)\n```\n caller ──> auth/authz ──> [flag predicate] ──> wrapper{readProfile,writeProfile} ──> repository.read/write ──> in-proc txn store\n │ │ cache.get/set/delete │ single-flight (per key)\n │ └──> LRU adapter (1000 / 16 MiB / 30 s / 10 s sentinel)\n └──> rollout controller: rotates wrapper+cache instance on flag change\n NEW: wrapper ordering + R1 per-key write-completion marker (inside wrapper, same tick as cache.set)\n```\nBoundaries unchanged; new coupling is wrapper→adapter only. Scaling: 10x load → hit rate rises, LRU bounded by byte cap; 100x → DB saturates on misses (cold start) and single-flight bounds concurrent misses per key. SPOF: the single process (already true). Security: no new surface (Section 3). Production failure: adapter failure → bypass (line 28-29); repo failure → typed error. Rollback: flag off, controller rotation; seconds. **F1 (CRITICAL GAP → resolved by R1):** stale fill after write completion, trace in Step 0. Apply: plan amended; no further change.\n\n### Section 2: Error & Rescue Map — 7 paths, 0 GAPS after R1\n```\n METHOD/CODEPATH | WHAT CAN GO WRONG | EXCEPTION CLASS / RESULT\n readProfile#cache.get | adapter failure | AdapterError → adapter bypass (existing)\n readProfile#repository.read| DB failure / timeout | typed RepositoryError (existing mapping)\n | missing record | absent-result DTO (not an error)\n readProfile#cache.set | adapter failure; byte cap; stale fill(F1)| AdapterError → bypass; evict; R1 suppress\n writeProfile#repository.write | rejected (rolled back) | typed RepositoryError; cache untouched\n writeProfile#cache.delete | adapter failure | AdapterError → bypass until reinit\n controller rotation | old write never settles | existing controller contract (awaits writes)\n\n EXCEPTION CLASS | RESCUED? | RESCUE ACTION | USER SEES\n AdapterError | Y | bypass cache, counter, alert at 1 min | normal latency from DB\n RepositoryError | Y | existing typed API error | existing API error\n absent-result DTO | n/a | sentinel 10 s | 404-equivalent as today\n stale fill (F1) | Y (R1) | suppress fill, counter | fresh data on next read\n```\nNo catch-alls; adapter and repository errors keep their classes. Apply: no change.\n\n### Section 3: Security & Threat Model — 0 issues\nNo new endpoint, input, param, dependency or secret. Keys are tenant+profile encoded post-authz (lines 18-21); cached DTOs immutable; no key/secret logging (line 20, 79). Threat: cross-tenant cache read — likelihood Low, impact High, mitigated by key encoding and existing tenant-isolation contract tests (line 72-73). Threat: stale authz result — cached values are data not decisions (line 21); unchanged. Audit: no sensitive mutation added. No issues found.\n\n### Section 4: Data Flow & Interaction Edge Cases — 6 edge cases mapped, 0 unhandled after R1\n```\n INPUT(key) -> [authz done upstream] -> cache.get -> miss -> repository.read -> cache.set -> OUTPUT\n nil/empty key: rejected upstream by key validation (line 73) tested: existing\n absent record: absent DTO -> sentinel 10 s -> same DTO on hit tested: planned (absence)\n adapter error: bypass -> DB read -> OUTPUT tested: planned (fallback)\n repo error: typed error, no fill tested: planned (rejected reads)\n oversize value: adapter byte cap evicts/skips tested: planned (byte limits)\n stale fill (F1): R1 suppress tested: R1 scenarios 1-7\n```\nAsync ordering, invariant: a read begun after writeProfile fulfills observes the committed version (lines 42-48).\n```\n time | R1 readProfile(k) | W writeProfile(k) | R2 readProfile(k) | cache[k]\n t0 | get miss; await read(v0) | | | ∅\n t1 | | write commits v1; delete; done | | ∅\n t2 | resume v0; set(k,v0) ✗ | | | v0 <- violates (original)\n t2' | resume v0; R1: marker says write completed after t0 -> skip | | ∅ <- holds (amended)\n t3 | | | begin; miss; read v1| v1\n ---- other order ----\n t0 | | write commits v1; delete; done | | ∅\n t1 | begin; miss; await read(v1)| | | ∅\n t2 | resume v1; set(k,v1) ✓ | | | v1 (no write after t1; fill kept)\n```\nOverlapping writes W1,W2 then pending read: both deletes run; R1 marker records latest completion; pending fill skipped; next read sees last commit. Rejected write: no completion marked; pending fill kept (store unchanged). Mechanism excluding other orders: single JS event loop with synchronous atomic cache ops and marker check in the same tick as cache.set. Regression proof: Amendment R1 scenarios 1-7 with pause/release at repository.read / repository.write. Interaction edge cases: no user-visible interaction added (backend only). Apply: covered by R1.\n\n### Section 5: Code Quality Review — 0 issues\nWrapper fits the read-through pattern; names (readProfile/writeProfile) describe behavior; no duplication (single-flight stays in repository, line 32). Branches in readProfile after R1: miss, error, suppress → under 5. No over-engineering; the marker is the minimum mechanism for the retained contract. No issues found.\n\n### Section 6: Test Review — diagram produced, 0 gaps (R1 scenarios required)\n```\n NEW THING | TYPE | HAPPY | FAILURE | EDGE\n readProfile hit/miss | unit | hit returns cached | adapter error -> bypass | absent sentinel decode\n readProfile fill | unit | miss fills | repo rejects -> no fill | byte cap, TTL expiry\n writeProfile invalidate | unit | commit -> delete | reject -> cache preserved | overlapping writes\n R1 stale-fill guard | unit(harness)| scenario 2 fill kept | scenario 1,3,6 suppressed | scenario 4,5 (rejects), 7 (instances)\n single-flight interaction | integration | concurrent misses coalesce | release on failure | cohort retire on write\n controller rotation | integration | old-writer/new-reader order| predicate parity | disabled instance bypass\n```\nAssertion check: every behavior maps to lines 72-76, 85-86, 122-125 or Amendment R1; wrong result rejected = stale v0 observed after completion. Existing deterministic harness (line 122); no time/randomness dependence (TTL via controlled clock). 2 am test: scenario 1 + 3. Hostile QA: scenario 6 with a rejected write interleaved. Chaos: adapter throws mid-set → bypass and counter. Pyramid: unit-heavy, few integration, no E2E. Load: cold-start miss storm bounded by single-flight; covered by rollout stage gates. Apply: no new choice; scenarios already approved under R1.\n\n### Section 7: Performance Review — 0 issues\nNo new queries or indexes. Memory bounded: ≤1000 entries / 16 MiB plus marker state bounded by in-flight read keys (released when readers finish). Slow paths: miss + DB read (today's p95), fill-suppressed re-read (one extra DB read, rare), cold start. No pool pressure change. No issues found.\n\n### Section 8: Observability & Debuggability — 0 gaps\nExisting metrics: hit/miss, eviction, bytes, fallback errors, DB CPU, p95 (line 78-79). Added under R1: `stale_fill_suppressed` counter on the existing dashboard, no key labels; a suppressed fill counts as an admitted miss. Alerts unchanged (lines 88-90). Debuggability: counters plus existing structured repository logs reconstruct bypass/suppression episodes. Runbook: existing (lines 99-100); suppression spike reads as \"write-heavy keys\", not an incident. No new observability project. No gaps.\n\n### Section 9: Deployment & Rollout — 0 risks beyond existing plan\nNo migration. Flag staged 10/50/100 with one healthy hour and the stage criteria at lines 94-98. Rollback: flag off → controller drains writes and publishes a bypass instance → seconds. Deploy window: old code has no cache, new code flag-off → identical behavior. Post-deploy: first 5 min watch fallback errors and p95; first hour watch hit rate, DB CPU, suppressed-fill counter. Smoke: existing contract tests plus wrapper tests in CI. Apply: no change.\n\n### Section 10: Long-Term Trajectory — reversibility 5/5, debt items 0\nDebt: none beyond a documented marker that disappears with any future distributed cache (out of scope, line 106-108). Path dependency: interface preserved. Knowledge: Amendment R1 plus the schedule diagram document the why. Ecosystem fit: same repository pattern. One-year: obvious.\n\n### Section 11: Design & UX Review — SKIPPED (no UI scope)\n\n## Closing sequence\n**Outside Voice:** `codex_reviews: disabled` (config). Codex review skipped (codex_reviews disabled). Re-enable: `gstack-config set codex_reviews enabled`. No native fallback. Disabled-coverage review-log write: gstack binaries not installed in this fixture → **not persisted** (fields: skill codex-plan-review, status skipped, source none, outside_provider codex, outside_status disabled, phase plan-review).\n**TODO choices:** none remain (HOLD SCOPE; no evidenced deferred gap).\n**Approval readiness: PASS** — R1: approved via D1 → A (author-policy auto-decision recorded above); plan applies only that scope. No other rows.\n\n## NOT in scope\nDeferred: none. Rejected: D1-B (TTL-bounded staleness; contradicts line 48), D1-C (unamended TODO; untestable acceptance). Pre-existing exclusions retained (lines 105-108).\n\n## What already exists\nRepository + typed errors, LRU adapter with limits and absent sentinel, single-flight, transactional store, rollout controller/flag, telemetry/dashboards/runbooks, contract harness — all reused unchanged.\n\n## Dream state delta\nAfter this plan: hot-key reads served in-process with the retained consistency contract intact; the repository interface still allows a backend swap later. Remaining gap to the 12-month ideal: multi-process support (explicitly out of scope).\n\n## Error & Rescue Registry\nSee Section 2 table (7 codepaths, 4 exception classes, 0 CRITICAL GAPS after R1).\n\n## Failure Modes Registry\n```\n CODEPATH | FAILURE MODE | RESCUED? | TEST? | USER SEES? | LOGGED?\n readProfile.cache.get | adapter failure | Y bypass | Y planned | normal latency | Y counter\n readProfile.repo.read | DB failure | Y typed | Y existing | API error | Y existing\n readProfile.cache.set | stale fill (F1) | Y (R1) | Y scen 1-7 | fresh data | Y counter\n readProfile.cache.set | byte cap / evict | Y | Y planned | miss | Y metric\n writeProfile.repo.write | rejected/rolled back | Y typed | Y existing | API error | Y existing\n writeProfile.cache.delete| adapter failure | Y bypass | Y planned | normal latency | Y counter\n controller rotation | old write pending | Y await | Y existing | brief pause | Y existing\n```\n0 CRITICAL GAPS.\n\n## Diagrams\n1. System architecture — Section 1. 2. Data flow with shadow paths — Section 4. 3. State machine (cache entry):\n```\n ∅ --fill(ok, no newer write)--> VALUE(30 s) --TTL/evict/delete--> ∅\n ∅ --fill(absent)--> SENTINEL(10 s) --TTL/delete--> ∅\n ∅ --fill(newer write completed)--> ∅ (R1 suppress) invalid: VALUE->VALUE via stale fill (prevented by R1)\n any --adapter failure--> BYPASS --reinit--> ∅\n```\n4. Error flow — Section 2. 5. Deployment sequence:\n```\n deploy(flag off) -> 10% -> 1 h healthy -> 50% -> 1 h healthy -> 100% -> absolute targets hold\n```\n6. Rollback flowchart:\n```\n breach/alert? --yes--> flag off -> controller drains writes -> bypass instance -> runbook incident -> resume criteria\n --no---> continue stage\n```\nStale Diagram Audit: no ASCII diagrams exist in touched files; none stale.\n\n## Implementation Tasks\nSynthesized from this review's findings. Each task derives from a specific finding above.\n\n- [ ] **T1 (P1, human: ~1 day / CC: ~30min)** — profile-summary wrapper — Implement the R1 stale-fill guard per Amendment R1 (structure chosen in engineering planning)\n - Surfaced by: Section 1/4 — F1 stale fill after write completion (Step 0 trace)\n - Files: to be determined\n - Verify: Amendment R1 scenarios 1-7 pass on the existing deterministic harness\n- [ ] **T2 (P1, human: ~4h / CC: ~10min)** — telemetry — Add `stale_fill_suppressed` counter to the existing dashboard, no key labels; count suppressions as admitted misses\n - Surfaced by: Section 8 — R1 telemetry requirement\n - Files: to be determined\n - Verify: counter visible on existing dashboard in staging; hit-rate formula excludes suppressed fills from hits\n- _No new tasks from Sections 3, 5, 7, 9, 10, 11._\nTask JSONL: gstack state root unavailable in this fixture → **not persisted**.\n\n## Completion Summary\n```\n +====================================================================+\n | MEGA PLAN REVIEW — COMPLETION SUMMARY |\n +====================================================================+\n | Mode selected | HOLD SCOPE |\n | System Audit | skipped per run rules; fixture repo |\n | Step 0 | HOLD SCOPE; R1 approved (D1 → A) |\n | Section 1 (Arch) | 1 issue found (F1, resolved by R1) |\n | Section 2 (Errors) | 7 error paths mapped, 0 GAPS |\n | Section 3 (Security)| 0 issues found, 0 High severity |\n | Section 4 (Data/UX) | 6 edge cases mapped, 0 unhandled |\n | Section 5 (Quality) | 0 issues found |\n | Section 6 (Tests) | Diagram produced, 0 gaps |\n | Section 7 (Perf) | 0 issues found |\n | Section 8 (Observ) | 0 gaps found |\n | Section 9 (Deploy) | 0 risks flagged |\n | Section 10 (Future) | Reversibility: 5/5, debt items: 0 |\n | Section 11 (Design) | SKIPPED (no UI scope) |\n +--------------------------------------------------------------------+\n | NOT in scope | written (2 rejected, 0 deferred) |\n | What already exists | written |\n | Dream state delta | written |\n | Error/rescue registry| 7 rows, 0 CRITICAL GAPS |\n | Failure modes | 7 total, 0 CRITICAL GAPS |\n | TODOS.md updates | 0 items proposed |\n | Scope proposals | 0 proposed, 0 accepted (HOLD) |\n | CEO plan | skipped by mode |\n | Outside voice | codex — disabled |\n | Lake Score | 1/1 recommendations chose complete option |\n | Diagrams produced | 6 (arch, data flow, state, error, deploy, rollback) |\n | Stale diagrams found | 0 |\n | Unresolved decisions | 0 |\n +====================================================================+\n```\nUnresolved Decisions: none. Review Log / decision log / dashboard read: gstack binaries unavailable in this fixture → **not persisted** (fields: skill plan-ceo-review, status clean, unresolved 0, critical_gaps 0, mode HOLD_SCOPE, scope 0/0/0, commit 90be9a3). Learnings: no durable learnings this session. Next step: /plan-eng-review (required gate).\n\n## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 1 (not persisted) | CLEAR | mode: HOLD SCOPE, 0 critical gaps (1 finding F1 resolved by R1) |\n| Outside Review | codex (`codex_reviews: disabled`) | Independent 2nd opinion | 0 | DISABLED | codex_reviews disabled; no completed external review |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | — | — |\n| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | — |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | — |\n\n**OUTSIDE COVERAGE:** codex, plan-review phase, disabled by config; no completed external review; 0 outside findings (none run).\n**VERDICT:** CEO CLEARED — eng review required\n\nNO UNRESOLVED DECISIONS\n" + } +} diff --git a/test/fixtures/detector-corpora/plan-ceo-section-loading/37179171083-t1-pass.json b/test/fixtures/detector-corpora/plan-ceo-section-loading/37179171083-t1-pass.json new file mode 100644 index 000000000..6faae79d5 --- /dev/null +++ b/test/fixtures/detector-corpora/plan-ceo-section-loading/37179171083-t1-pass.json @@ -0,0 +1,20 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-ceo-section-loading", + "detector": "test/helpers/ceo-stale-fill-decision.ts#hasApprovedStaleFillDecision || test/helpers/ceo-section-loading-fixture.ts#hasStaleFillRaceFinding", + "series_identity": "c040a0214e4f3172", + "source": { + "run": "37179171083", + "sha": "db91607a7fd9419592eb63e036149920304e32b5", + "lane": "periodic", + "artifact": "paid-slice-6-a1", + "path": "shards/skill-e2e-plan-ceo-review-section-loading/_partial-e2e.json#/tests/0/output" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "Genuine pass: approved R1 ledger row with a stale-fill currentDecision, and the prose finding also matches; both witnesses agree.", + "input": { + "report": "# Plan: cache profile summaries in one process\n\n## Measured problem and accepted scope\nThe existing profile-summary service has one active process. A one-week trace\nshows repeated reads of about 900 hot keys: DB CPU is 70%, with read p95 120 ms.\nAdd a process-local LRU wrapper to the existing repository. Acceptance targets\nare at least 60% cache hits, DB CPU below 50%, and read p95 below 60 ms, with the\nexisting error-rate and correctness SLOs unchanged. This is an internal backend\nchange with no UI, API, schema, pricing, or developer onboarding change.\n\n## Existing contracts retained\n- All reads and writes use this repository in the same process; there are no\n external DB writers. Multi-process operation remains unsupported and startup\n rejects that configuration while caching is enabled.\n- These surrounding contracts are accepted fixture facts, supplied by the\n existing repository, cache adapter and rollout controller. Preserve them;\n review the new wrapper ordering below against them.\n- Authentication and authorization run before repository access. Keys encode\n the authenticated tenant ID and validated profile ID without ambiguity.\n Values are immutable profile-summary DTOs; secrets and cache keys are never\n logged. Cached results cannot bypass authorization.\n- The existing LRU adapter supports 1000 entries, a 16 MiB byte cap, and a\n 30-second TTL. Recorded hot data fits those limits. repository.read returns\n an immutable absent-result DTO for a missing record, never undefined. The\n adapter recognizes that DTO in cache.set, stores an internal sentinel with a\n 10-second TTL, and cache.get decodes it back to the same absent-result DTO.\n The internal sentinel cannot escape the adapter; undefined means a cache miss.\n- Cache operations are synchronous and atomic in the single JS event loop.\n On any cache failure the existing adapter bypasses the cache until an empty\n cache is reinitialized; repository errors keep the current typed API error\n mapping. The existing per-key single-flight wrapper sits inside\n repository.read, coalesces simultaneous store reads and releases on failure.\n A committed repository.write retires that key's old read cohort before its\n promise resolves. A later repository.read starts a fresh cohort; a rejected\n write leaves the cohort unchanged. Already-started readers may finish with\n their earlier snapshot. This admission rule does not inspect cache fills.\n- The repository uses an in-process transactional store, with no network\n transport between this wrapper and the store. repository.write is atomic:\n a resolved promise means committed, and every\n rejected promise guarantees no commit; its transaction rolled back before\n rejection. Existing contract tests exercise that guarantee.\n- Consistency is measured at the public wrapper boundary. A write completes\n when writeProfile's promise fulfills after cache.delete, not when\n repository.write commits or resolves. A read begins when readProfile is\n invoked. Reads that overlap an unfinished writeProfile may return an earlier\n snapshot, including reads begun after the store commit but before the wrapper\n promise fulfills. Every read begun after that write completes must\n observe the committed version. TTL expiry is not a substitute for this rule.\n\n## Proposed wrapper integration\nKeep the current read-through repository interface and shared adapters. These\nare the complete new read/write ordering rules; no additional version checks or\ncoordination between a cache fill and a write are proposed in the original draft. **Amended by CEO review decision R1 (D1 → A):** the wrapper MUST NOT install a cache fill from a read that began before a write to the same key committed and whose writeProfile has since run (or is running). Required guarantee: at fill time the wrapper synchronously checks a per-key write generation (or equivalent) recorded while the miss is in flight; if any write to that key committed since the miss began, the fill is skipped, the value is still returned to the caller, and the skipped fill is counted. Coordination state is bounded by in-flight misses and removed when the last in-flight miss for the key settles. Exact data structures and function bodies are left to engineering planning. The original draft below is retained as the baseline being amended:\n\n```javascript\nasync function readProfile(key) {\n const cached = cache.get(key);\n if (cached !== undefined) return cached;\n const value = await repository.read(key);\n cache.set(key, value);\n return value;\n}\n\nasync function writeProfile(key, update) {\n const saved = await repository.write(key, update);\n cache.delete(key);\n return saved;\n}\n```\n\n## Verification and rollout\nExisting repository contract tests cover tenant isolation, key validation,\nabsence, DB failures, authorization, and startup rejection of multi-process\noperation while caching is enabled. New wrapper tests cover hit/miss,\neviction and byte limits, TTL, adapter-failure fallback, successful-write\ninvalidation, failed-write preservation, and concurrent-miss coalescing.\n**Added by R1 (D1 → A), deterministic scenarios on the existing contract test\nharness with pause/release points:** (S1) read misses and pauses inside\nrepository.read; write commits and writeProfile fulfills; read resumes with the\nearlier snapshot: assert no fill installed, a fresh read observes the committed\nversion, suppressed-fill count +1. (S2) same but the read resumes BEFORE the write\ncommits and fills: assert writeProfile's cache.delete removes it and the next read\nobserves the committed version. (S3) missing record: read obtains the absent DTO,\ncreate commits and completes, read resumes: assert no absent sentinel installed\nand the next read observes the created record. (S4) overlapping writes W1, W2 with\na read in flight: assert the next read after both complete observes W2's version.\n(S5) rejected read leaves cache untouched; rejected write leaves cache and\ngeneration unchanged. (S6) controller instance swap during an in-flight miss:\nold read cannot fill the new instance (existing contract) and the new instance\nserves the committed version. The suppressed-fill branch is recorded through the\nexisting telemetry on the current dashboard with no key labels; no new alert or\nmetric project.\nThe rollout uses the existing runtime feature flag: enable for 10% of keys,\nthen 50%, then all keys after one healthy hour at each stage. Monitor hit/miss,\neviction, cache bytes, fallback errors, DB CPU, and read p95 without raw IDs.\nThe existing controller uses one shared key-selection predicate for reads and\nwrites. On any enable, disable or percentage change, it stops admitting work,\nawaits every admitted old-instance write, then publishes a new wrapper/cache\ninstance with a fresh single-flight cohort before admitting new work. Old reads\nretain their old instance and cannot fill the new one. Disabled instances\nbypass the cache on both paths. Tests cover the old-writer/new-reader ordering,\nall those transitions and predicate parity. This lifecycle isolation does not coordinate\nan ordinary DB write with a cache fill in the same active instance.\nExisting dashboards and runbooks cover these metrics. Before each stage, verify\nthat alerts page the service owner on any correctness/error-SLO breach, read\np95 above 120 ms for five minutes, or cache bypass persisting for one minute.\nHit rate is hits / (hits + misses) among requests admitted to the cache path;\nflag-excluded or adapter-bypassed requests are tracked separately, not as misses.\nDB CPU and read p95 are service-wide metrics, including bypassed requests.\nAt the 10% and 50% stages, a healthy hour requires at least 60% admitted-request\nhits, unchanged correctness/error SLOs, no alerts, and aggregate DB CPU/read p95\nno worse than their 70%/120 ms pre-rollout baselines. At 100%, the original\nabsolute acceptance targets (DB CPU below 50%, read p95 below 60 ms, hits at\nleast 60%) must all hold with unchanged correctness/error SLOs and no alerts.\nAny breach disables the flag immediately; the runbook records the incident,\nrollback and criteria for resuming. These are existing\nrollout-controller and telemetry contracts, not proposed wrapper additions.\nCold starts remain within the existing DB capacity. The service owner monitors\nthe rollout and records the results against the acceptance targets.\n\n## Out of scope\nDistributed caching, cross-process coherence, prewarming, changing consistency\nsemantics, or adding new product surfaces. The repository interface preserves a\nfuture replacement path without introducing a general cache framework now.\n\n## Author's review and acceptance requirements\nThis is a full CEO scope and feasibility review. The author has approved the\nretained contracts, limits, rollout and acceptance targets above. Evaluate the\nproposed wrapper against them; the wrapper itself remains unapproved. An actual\ncontradiction or missing proof must be reported and resolved, not assumed away.\nFor a demonstrated gap, amend the plan with the required guarantee, a feasible\nremedy, its tradeoffs and deterministic regression scenarios. Those repairs\nand their required verification are within the requested scope. The exact data\nstructures, full function bodies and executable test code belong to subsequent\nengineering planning; do not select or implement them during this review when\nthe required behavior and feasibility can already be established.\n\nUse the existing deterministic repository-contract test harness. Required wrapper\nacceptance includes both completion orders of overlapping reads and writes,\nmissing-record creation, rejected reads/writes, overlapping writes, and isolation\nacross controller instance changes. Use existing telemetry to record any added\nbranch on the current dashboard with no key labels; no new alert threshold or metric\nproject is requested. These are future acceptance requirements, not tests already\nimplemented or passing. Preserve all 11 review outcomes, required registries,\ndiagrams, tasks, completion summary and the full GSTACK REVIEW REPORT.\n\n---\n\n# CEO REVIEW (plan-ceo-review, HOLD SCOPE) — 2026-10-04\n\n## Step 0: setup, ledger and scope challenge\n\n**Review depth:** implementation-ready at the wrapper codepath level (readProfile /\nwriteProfile, adapter, controller). Per the author's acceptance requirements, exact\ndata structures, full function bodies and executable test code are reserved for\nengineering planning; this review names required guarantees, codepaths, rescue\nbehavior and regression scenarios only.\n**Storage policy:** working plan = this file (requested output). Appends for new\nreview content, scoped Edits for plan amendments. gstack state/log/telemetry\nwrites are out of this run's scope (hermetic fixture); such fields are shown\n**not persisted**. System audit, environment setup and telemetry skipped per run\ninstructions. Outside review: `codex_reviews: disabled` in\n`.gstack-section-state-b2PTft/config.yaml`.\n**Stated limits recorded:** LRU 1000 entries; 16 MiB byte cap; 30 s value TTL;\n10 s absent-sentinel TTL; targets hits >= 60% (admitted requests), DB CPU < 50%,\nread p95 < 60 ms at 100%; stage baselines 70% CPU / 120 ms p95; alerts: p95 > 120 ms\nfor 5 min, bypass persisting 1 min, any correctness/error-SLO breach; one healthy\nhour per stage at 10% / 50% / 100%. Deliverables counted against scope: 1 wrapper\nmodule (new), 1 wrapper test file (new), controller wiring reuse (unchanged).\nEstimated changed files: 2-3 (estimate).\n\n### Decision ledger (continued through all sections)\n\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| R1 — Section 1 (Architecture), carried to Section 4 | PLAN.md \"Existing contracts retained\": every read begun after writeProfile fulfills must observe the committed version; TTL is not a substitute. PLAN.md \"Proposed wrapper integration\": `cache.set(key, value)` after `await repository.read` with no fill/write coordination; fixture fact \"already-started readers may finish with their earlier snapshot\" and \"this admission rule does not inspect cache fills\". | Wrapper as proposed: unconditional fill after read resolves. | A: suppress fills that overlap a committed write; B: install then delete; C: keep as proposed (rejected: weakens retained contract). | approved | D1 auto-decided → A under the run's author policy (\"authorize complete remedies and required verification that restore its retained contracts ... use the recommended option\"). Scope: wrapper fill-time guard with bounded in-flight state, suppressed-fill count on existing dashboard (no key labels), scenarios S1–S6 on the existing harness. Not approved: adapter/repository/controller changes, new metrics projects, data-structure or test-code selection. |\n\n### 0A. Premise Challenge\nReal problem: ~900 hot profile-summary keys are re-read so often that one DB\ncarries 70% CPU and 120 ms read p95. Target outcome: >= 60% of admitted reads served\nfrom process memory, DB CPU < 50%, read p95 < 60 ms, SLOs unchanged. Do-nothing\ncost: DB stays near saturation; any traffic growth pushes p95 and error rate up.\nThe plan attacks the pain directly (fewer repeated store reads), not a proxy.\nPremise holds under HOLD SCOPE; one consistency contract (R1) is at risk and is\nreviewed, not assumed away.\n\n### 0B. Existing Code Leverage\n| Sub-problem | Reused existing code | Note |\n|---|---|---|\n| Bounded in-memory storage, TTL, absent sentinel, failure bypass | existing LRU cache adapter | fixture fact; no rebuild |\n| Coalescing concurrent misses | per-key single-flight inside repository.read | fixture fact; does not inspect fills |\n| Atomic commit / rollback | repository.write on in-process transactional store | fixture fact |\n| Staged rollout, instance isolation, metrics | existing rollout controller + telemetry + dashboards | fixture fact |\n| New: read-through ordering + write invalidation | new wrapper (readProfile/writeProfile) | the only new code |\nNo rebuild is proposed; refactoring the existing path is not needed.\n\n### 0C. Dream State Mapping\n```\n CURRENT STATE THIS PLAN 12-MONTH IDEAL\n single process, every read ---> process-local LRU in front ---> same single-process service,\n hits the store; DB CPU 70%, of repository; >=60% hits, read-through boundary stable\n read p95 120 ms DB CPU <50%, p95 <60 ms, enough that a later replacement\n consistency contract intact (if ever needed) is a swap\n behind the same interface\n```\nThe plan moves toward the ideal without adding a cache framework. Distributed\ncaching stays out of scope by author decision.\n\n### 0D. Alternatives\nNo approach choice was required at Step 0: the requested approach (local LRU\nwrapper over the existing repository) is retained. 0D is used later for R1.\n\n### 0E. Mode Selection\nExplicit user choice: HOLD SCOPE (\"Hold the current scope (HOLD SCOPE mode)\").\nSteps 2-3 skipped. Mode handoff: `Mode: HOLD SCOPE; approved decisions: none.`\nApplication: preserve scope, maximum rigor on failures, edge cases, error paths,\ntests and observability; repairs needed to meet retained invariants are in scope.\nProvenance: explicit user instruction; no question asked, no question log.\nNo new approach decision was needed.\n\n### 0G. HOLD SCOPE checks\n1. Complexity: 2-3 changed files, 1 new module, 0 new services. Below the\n 8-file / 2-service threshold. No fewer-moving-parts alternative exists; the\n wrapper is the minimum.\n2. Minimum change for the goal: the two wrapper functions plus tests. Nothing in\n the accepted scope is deferrable without blocking acceptance (the contract\n tests, rollout stages and alerts are launch scope under Prime Directive 5).\n3. Invariants and acceptance criteria kept as stated; any repair required to meet\n them (R1) is in scope.\nDeferral menu: no item proposed for deferral; nothing to ask.\n\n### 0I. Temporal Interrogation\n```\n HOUR 1 (foundations): implementer needs the retained contracts verbatim (absent DTO,\n sentinel TTL, single-flight cohort rule, write-completion boundary).\n HOUR 2-3 (core logic): ambiguity: what happens to a fill from a read that began before a\n write committed and resolves after writeProfile fulfilled (R1).\n HOUR 4-5 (integration): surprise: controller instance swaps isolate old readers, but do\n NOT coordinate a fill and a write inside one active instance.\n HOUR 6+ (polish/tests): wish they had planned pause/release harness hooks for both\n completion orders of overlapping reads and writes, and for\n missing-record creation under the absent sentinel.\n```\nEffort: human ~2-3 days / CC+gstack ~1-2 h for wrapper + tests (tests ~50x ratio).\nFeasibility blockers: R1 (resolved below before the sections proceed). Pending\nchoices: none other than R1.\n\n## Review Sections\n\n**Current scope (published before Section 1):** Mode HOLD SCOPE (explicit user\ninstruction). Governing rows: R1 unresolved (pending below). Accepted work: the\nplan as written above. Deferred: none. Rejected: none. Pending: R1.\n\n### Section 1: Architecture Review\nDependency graph (new component marked *):\n```\n caller --> rollout controller (predicate, instance swap)\n |\n v\n *wrapper {readProfile, writeProfile}\n | |\n v v\n LRU cache adapter repository.read / repository.write\n (1000 / 16 MiB / | (single-flight cohort inside read)\n 30 s, sentinel 10 s) v\n in-process transactional store\n```\nData flow, readProfile:\n```\n happy : get(key) hit -> return DTO | miss -> await read -> set -> return DTO\n nil : record missing -> read returns absent DTO -> set stores sentinel (10 s) -> get decodes to absent DTO\n empty : key validation runs before repository access (fixture); empty/invalid key never reaches wrapper\n error : read rejects -> no set, cache untouched -> typed API error (existing mapping)\n adapter failure -> adapter bypasses cache until reinit -> read goes to store\n```\nData flow, writeProfile:\n```\n happy : await write (commit) -> delete(key) -> fulfill (write \"completes\" at fulfill)\n error : write rejects (rolled back) -> no delete -> reject with typed error; cache preserved (correct)\n adapter failure on delete -> adapter bypass (fixture); write already committed; wrapper fulfills\n```\nCoupling: wrapper depends on adapter + repository only; controller owns lifecycle.\nJustified; no new cross-module coupling. Scaling: 10x load keeps the hot set at\n~900 keys < 1000 entries; 100x load with the same key set is served from memory;\na key set beyond 1000 entries thrashes the LRU and the hit target fails loudly on\nthe dashboard (no silent failure). SPOF: unchanged (single process, single DB).\nSecurity: no new endpoint or mutation surface; auth precedes repository access\n(fixture). Rollback: flag disable -> controller publishes a disabled instance\n(bypass both paths); seconds.\n\n**Production failure scenario (CRITICAL GAP, row R1):** a read misses, awaits\nrepository.read and obtains snapshot v0; a write commits v1, cache.delete runs\n(no entry yet), writeProfile fulfills; the read then resolves with v0 and\n`cache.set(key, v0)` installs a stale entry. A read begun after the write\ncompleted hits v0 for up to 30 s (or 10 s for an absent sentinel after a create).\nThis violates the retained contract \"every read begun after that write completes\nmust observe the committed version\". Evidence: the plan states no fill/write\ncoordination; the fixture cohort rule explicitly lets already-started readers\nfinish with their earlier snapshot and does not inspect fills; controller\nisolation only applies across instance swaps. This is a contract-level\ncontradiction, not an unavailable-implementation guess.\n\nOther outcome: **OK** on boundaries, coupling, scaling, SPOF, security, rollback.\n**Decision gate:** Resolve path 1 (new choice). 0D plan-decision route, row R1.\n\n## currentDecision (R1)\nCommitment comparison:\n```text\nCommitment | Source/approval or pending | Current (plan as written) | A | B | C\nReads after a completed write see committed | PLAN.md retained contracts (fixed) | violated by stale fill (gap) | met: fill suppressed when a write to key | met: fill suppressed by re-deleting key after | not met: contract would be weakened\nversion | | | committed during the in-flight miss | every fill whose read overlapped a write | (not authorizable in this review)\nWrapper ordering rules | pending (R1) | get -> await read -> set | add synchronous per-key write-generation | add per-key \"write-seen\" flag consulted after | unchanged\n | | | check at fill time (in wrapper instance) | set to delete; still momentarily installs |\nMemory for coordination state | pending (R1) | none | bounded by in-flight misses (entries | bounded by in-flight misses | none\n | | | removed when last miss settles) | |\nAdapter / repository / controller contracts | fixture facts (fixed) | unchanged | unchanged | unchanged | unchanged\nRegression scenarios (both completion orders,| author acceptance reqs (fixed) | listed as required, not yet | deterministic pause/release scenarios | same scenarios | same scenarios, but one must be\nmissing-record create, rejected ops, | | specified against R1 | specified below; fill-suppressed branch | | changed to expect stale data\noverlapping writes, instance isolation) | | | asserted | | (forbidden)\nTelemetry for added branch | author acceptance reqs (fixed) | n/a | count suppressed fills on existing | same | n/a\n | | | dashboard, no key labels | |\nChanged files | estimate | 2-3 | 2-3 (same files) | 2-3 (same files) | 2-3\n```\n\nQuestion: D1 — R1: How should the wrapper stop a stale cache fill from a read that began before a write committed?\nProject/branch/task: main branch, PLAN.md \"cache profile summaries in one process\", CEO review in HOLD SCOPE.\nELI10: A reader asks the database for a profile, and while it waits, a writer saves a newer profile and clears the cache. The reader's old answer then arrives and gets stored in the cache, so everyone after sees the old profile for up to 30 seconds. The plan promises that never happens, so the wrapper needs a rule that refuses to store an answer that is older than a write that already finished.\nStakes if we pick wrong: users read a profile they just updated and see the previous version; the correctness SLO breaks; the rollout trips its alert and rolls back.\nRecommendation: A because a synchronous fill-time check against a per-key write generation is the smallest explicit mechanism that satisfies the contract in a single event loop, with bounded state and one testable branch (explicit over clever).\nCompleteness: A=10/10, B=7/10, C=3/10\nPros / cons:\nA) Suppress fills that overlap a committed write (recommended)\n ✅ Meets the retained contract in both completion orders without touching the adapter, repository or controller.\n ✅ Adds one synchronous branch with bounded state; the suppressed-fill count is observable on the existing dashboard.\n ❌ A hot key under constant writes may rarely fill, so its reads keep going to the store (visible as lower hit rate, not as incorrectness).\nB) Install the fill, then delete it when a write overlapped\n ✅ Also keeps the adapter and repository unchanged and reuses cache.delete.\n ✅ Simple to reason about: every overlapped fill is removed before the reader returns.\n ❌ The stale entry exists for a moment between set and delete; a concurrent synchronous get in that window is impossible, but the design relies on strict ordering inside one function and is harder to prove and test than never installing.\nC) Keep the wrapper as proposed and accept TTL-bounded staleness\n ✅ Zero implementation work beyond the plan as written (effort S, no new state).\n ✅ Simplest code path.\n ❌ Contradicts the retained consistency contract and the \"TTL is not a substitute\" clause; not authorizable by this review.\nNet: A pays a tiny amount of bounded in-process state to make the contract hold by construction; B pays the same but with a weaker proof; C is not an option under the retained requirements.\n\nHeader: R1 stale cache fill\nA) Suppress fills that overlap a committed write (recommended)\nEffort S, risk low. Reuse: existing adapter, repository and controller unchanged; one new synchronous check in the wrapper instance keyed by the same cache key, tracking a per-key write generation (or equivalent) only while a miss is in flight, removed when the last in-flight miss for that key settles. Verification coverage: both completion orders of overlapping read/write, missing-record creation under the absent sentinel, overlapping writes, rejected reads/writes, instance isolation, suppressed-fill branch asserted and counted. ✅ Meets the retained contract in both completion orders without touching the adapter, repository or controller. ✅ Adds one synchronous branch with bounded state; the suppressed-fill count is observable on the existing dashboard. ❌ A hot key under constant writes may rarely fill, so its reads keep going to the store (visible as lower hit rate, not as incorrectness).\nB) Install the fill, then delete it when a write overlapped\nEffort S, risk medium. Reuse: adapter, repository and controller unchanged; wrapper installs the value, then calls cache.delete when a per-key write flag was raised during the miss. Verification coverage: same scenarios as A plus an ordering assertion that no get can observe the entry between set and delete. ✅ Also keeps the adapter and repository unchanged and reuses cache.delete. ✅ Simple to reason about: every overlapped fill is removed before the reader returns. ❌ The stale entry exists for a moment between set and delete; a concurrent synchronous get in that window is impossible, but the design relies on strict ordering inside one function and is harder to prove and test than never installing.\nC) Keep the wrapper as proposed and accept TTL-bounded staleness\nEffort S, risk high; zero implementation work. Reuse: everything as written. Verification coverage: would require changing a required scenario to expect stale data, which the retained contract forbids. ✅ Zero implementation work beyond the plan as written (effort S, no new state). ✅ Simplest code path. ❌ Contradicts the retained consistency contract and the \"TTL is not a substitute\" clause; not authorizable by this review.\n\n**D1 answer record (R1):** Auto-decided D1 → A (authority: this run's author policy\nauthorizes the recommended complete remedy restoring retained contracts; no human\npresent). Applied: \"Proposed wrapper integration\" amended with the required\nguarantee; \"Verification and rollout\" amended with scenarios S1–S6 and the\nsuppressed-fill telemetry branch; ledger row R1 → approved. Residual risk: a key\nunder constant writes may rarely fill (visible as lower hit rate, not stale data).\n\n### Section 2: Error & Rescue Map\n```\n METHOD/CODEPATH | WHAT CAN GO WRONG | EXCEPTION CLASS\n ---------------------------|--------------------------------------------|---------------------------\n readProfile#cache.get | adapter failure | adapter internal (bypass)\n readProfile#repository.read| store read fails / rejects | existing typed repository error\n | record missing | none (absent DTO, not an error)\n | stale snapshot resolves after completed | none (logic gap) -> R1 guard\n | write |\n readProfile#cache.set | adapter failure / byte cap exceeded | adapter internal (bypass / evict)\n writeProfile#repository.write | write rejects (rolled back) | existing typed repository error\n writeProfile#cache.delete | adapter failure after commit | adapter internal (bypass)\n controller swap | in-flight old reads during swap | none (isolation contract)\n ---------------------------|--------------------------------------------|---------------------------\n\n EXCEPTION / CONDITION | RESCUED? | RESCUE ACTION | USER SEES\n ------------------------------|-----------------|---------------------------------------|------------------\n adapter failure (any op) | Y (adapter) | bypass cache until empty reinit; | normal result, slower;\n | | bypass alert at 1 min | owner paged\n typed repository read error | Y (existing map)| re-raise via existing API mapping; | existing API error\n | | no fill |\n typed repository write error | Y (existing map)| re-raise; cache untouched (no commit) | existing API error\n absent record | n/a | absent DTO; sentinel 10 s | not-found result\n stale fill after write (R1) | Y after R1 | skip fill, return value, count branch | committed version\n```\nNo catch-alls proposed; the wrapper adds no new exception classes. **Outcome:**\n1 GAP (R1) mapped and resolved by D1 → A; otherwise OK. Decision gate: path 2\n(R1 exact answer cited); no new choice.\n\n### Section 3: Security & Threat Model\n| Threat | Likelihood | Impact | Mitigated |\n|---|---|---|---|\n| Cross-tenant cache read via key collision | Low | High | Yes: keys encode authenticated tenant + validated profile ID unambiguously (fixture); authz precedes repository access |\n| Cached result bypassing authorization | Low | High | Yes: fixture contract; cache sits below the authz boundary |\n| Key or secret leakage in logs/metrics | Low | Med | Yes: no raw IDs in metrics; the R1 suppressed-fill counter must carry no key labels (author requirement) |\n| Memory exhaustion via unbounded coordination state | Low | Med | Yes by R1 scope: state bounded by in-flight misses |\n| Cache poisoning via mutable DTO | Low | High | Yes: values are immutable DTOs (fixture) |\nNo new endpoint, input, dependency or background job. **Outcome:** No issues\nfound (0 High-severity open). Decision gate: no choice needed.\n\n### Section 4: Data Flow & Interaction Edge Cases\nData flow with shadow paths:\n```\n INPUT key -> VALIDATION (upstream authz/key validation) -> TRANSFORM (get / read) -> PERSIST (set / delete) -> OUTPUT DTO\n nil/empty key ........ rejected upstream before wrapper (fixture) tested: existing contract tests\n invalid DTO .......... impossible; immutable DTO from repository n/a\n exception/timeout .... read rejects -> no set -> typed error tested: rejected reads (S5)\n conflict/dup ......... overlapping writes -> last committed wins (S4) tested: S4\n stale/partial ........ stale snapshot fill -> R1 guard skips fill (S1) tested: S1-S3\n```\nAsync ordering, boundary = \"read begun after writeProfile fulfills observes the\ncommitted version\":\n```\n t | readProfile R (miss) | writeProfile W | cache[key] | gen[key]\n --|-----------------------------------|----------------------------------|-----------------|---------\n 1 | get -> miss; record gen=g0; await | | empty | g0\n 2 | (paused in repository.read, v0) | await write -> commit v1 | empty | g0\n 3 | | gen++ ; delete(key); fulfill | empty | g1\n 4 | read resolves v0; gen g1 != g0 | | empty (skipped) | g1\n 5 | return v0 to R's caller (allowed: R began before W completed) | |\n 6 | fresh read R2: miss -> read v1 -> gen unchanged -> set v1 | v1 | g1\n```\nOrder 2 (read resolves before commit): t2 read resolves v0, gen==g0, set v0;\nt3 W commits, gen++, delete(key) removes v0; R2 reads v1. Both orders satisfy the\nboundary with the R1 guard; without it, order 1 leaves v0 cached (violation).\nThe excluded schedule \"get observes an entry between set and delete\" is prevented\nby synchronous atomic cache ops in one event loop (fixture). Regression proof:\nS1 (order 1), S2 (order 2), S3 (absent sentinel), S4 (overlapping writes), S6\n(instance swap), with controlled pause/release points on the existing harness.\nInteraction edge cases: no user-visible interaction changes (backend only).\n**Outcome:** 5 edge cases mapped, 0 unhandled after R1. Decision gate: path 2\n(cites R1).\n\n### Section 5: Code Quality Review\nTwo small functions fit the existing read-through repository pattern. No\nduplicated behavior: invalidation and fill live only in the wrapper. Naming\n(readProfile/writeProfile) describes outcome. Error handling delegates to\nexisting typed mapping and adapter bypass (Section 2). The R1 guard adds one\nbranch; readProfile stays well under 5 branches. Not over-engineered: no\nframework, no version column, no adapter change. Under-engineering risk was the\nunconditional fill (R1, resolved). Require an ASCII comment above the wrapper\ndescribing the fill/write ordering (engineering preference: ASCII comments for\ncomplex state). **Outcome:** 1 issue (R1, resolved); comment requirement is a\ntask (T2). Decision gate: no new choice.\n\n### Section 6: Test Review\n```\n NEW THING | TEST TYPE | HAPPY | FAILURE | EDGE\n ---------------------------------|-------------|----------------------------|--------------------------------|-----------------------------\n readProfile hit/miss | unit | hit returns cached DTO | read rejects -> typed error | absent DTO -> sentinel decode\n readProfile fill guard (R1) | unit/harness| S2 fill then delete | S1 stale fill suppressed | S3 absent sentinel suppressed\n writeProfile invalidation | unit | commit -> delete -> fulfil | rejected write preserves cache | S4 overlapping writes\n adapter failure fallback | unit | bypass, results correct | bypass persists -> alert | failure on delete after commit\n eviction / byte cap / TTL | unit | LRU evicts oldest | over-cap value not stored | 1000th vs 1001st entry\n controller swap isolation | integration | new instance fresh cohort | old reader cannot fill new | S6 swap mid-miss\n suppressed-fill telemetry | unit | counter +1 on S1 | no key labels asserted | zero when no overlap\n```\nAssertion check: every row maps to a retained contract or to R1's approved scope;\nS1–S6 are the exact scenarios the author required (both orders, missing-record\ncreation, rejected ops, overlapping writes, instance isolation). Existing contract\ntests cover tenant isolation, key validation, absence, DB failures, authz,\nmulti-process rejection. 2am test: S1. Hostile QA: S4 plus swap mid-miss (S6).\nChaos: adapter failure injected mid-fill while a write commits (bypass must win).\nPyramid: mostly unit, few integration, no E2E (correct for backend). Flakiness:\nTTL tests must use the harness's controlled clock, not wall time. Load: existing\nrollout stages serve as the load test; no separate load test requested.\n**Outcome:** diagram produced, 0 gaps after R1 (TTL-clock note is a task, T3).\nDecision gate: path 2 (R1) and settled proof reuse; no new choice.\n\n### Section 7: Performance Review\nN+1: none (point reads). Memory: <= 1000 entries / 16 MiB adapter cap plus R1\nstate bounded by in-flight misses (hundreds of entries at most). Indexes: no new\nqueries. Caching: this is the caching. Jobs: none. Slow paths: cold miss = store\nread (today's p95 120 ms); hit = sub-millisecond; suppressed fill = store read on\nthe next read of that key. Floor: 900 hot keys / 30 s TTL >= 30 store reads/s\nplus suppressed fills on write-hot keys; within existing DB capacity (fixture:\ncold starts remain within capacity). Connection pools: unchanged. **Outcome:** No\nissues found; residual note: write-hot keys lower the hit rate, surfaced on the\ndashboard. Decision gate: no choice.\n\n### Section 8: Observability & Debuggability Review\nMetrics (existing telemetry): hit, miss, eviction, cache bytes, fallback/bypass\nerrors, DB CPU, read p95, plus the R1 suppressed-fill count on the current\ndashboard with no key labels (author requirement; not a new metric project).\nLogging: adapter failure already logs without keys/secrets; the wrapper adds no\nkey logging. Alerts: existing (SLO breach, p95 > 120 ms for 5 min, bypass > 1 min);\nno new threshold requested. Debuggability: a stale-read report three weeks later is\nreconstructable from suppressed-fill and bypass counters plus the flag stage\ntimeline. Runbook: existing; it must name \"suppressed fills climbing\" as expected\non write-hot keys, not an incident (task T4, documentation only). **Outcome:** 1\ngap (runbook note), resolved as a documentation task within existing runbook\nscope. Decision gate: no choice (no new observability project).\n\n### Section 9: Deployment & Rollout Review\nNo migration. Feature flag: existing runtime flag, 10% -> 50% -> 100% with one\nhealthy hour each and defined criteria (plan). Rollout order: deploy wrapper\ndisabled, then raise percentage. Rollback: disable flag -> controller stops\nadmitting, drains old writes, publishes disabled instance; seconds; runbook\nrecords incident. Deploy-time risk window: one process, so no mixed versions; the\ncontroller swap contract covers in-flight work. Environment parity: stage runs\nunder the same harness; first 5 minutes watch bypass and hit rate; first hour\nwatch the healthy-hour criteria. Smoke: hit/miss counters moving, zero bypass.\n**Outcome:** 0 risks flagged beyond the plan's existing controls. Decision gate:\nno choice.\n\n### Section 10: Long-Term Trajectory Review\nDebt: small (one guard with bounded state, one ASCII comment, runbook note).\nPath dependency: none; the repository interface preserves a replacement path.\nKnowledge: the ordering diagram in Section 4 belongs in the wrapper comment (T2).\nReversibility: 5/5 (flag off or delete two functions). Ecosystem fit: uses the\nexisting adapter and harness. 1-year question: obvious, provided the comment\nexplains why fills are suppressed. **Outcome:** reversibility 5/5, debt items 2\n(comment, runbook note). Decision gate: no choice.\n\n### Section 11: Design & UX Review\n`SKIPPED (no UI scope)` — internal backend change, no UI, API or user-visible state.\n\n## Closing sequence\n\n### Outside Voice\n`codex_reviews: disabled` (`.gstack-section-state-b2PTft/config.yaml`). Codex\nreview skipped (codex_reviews disabled). Re-enable: `gstack-config set\ncodex_reviews enabled`. No prompt, outside process or native fallback. Disabled\ncoverage record (`gstack-review-log` codex-plan-review, status skipped,\noutside_status disabled): **not persisted** (state writes out of this run's scope).\n\n### Resolve remaining TODO choices\nNone remain. HOLD SCOPE: no evidenced gap is deferred; every finding is in\naccepted scope (R1) or a same-branch task.\n\n### Approval readiness\nChecked rows: R1 — approved via D1 auto-decision (authority: run author policy;\nrecommended option A). Plan applies only that scope (wrapper guard, bounded state,\nS1–S6, suppressed-fill count on the existing dashboard). No declined, deferred or\nunanswered changes in accepted work. Option C declined and kept out.\n`Approval readiness: PASS` (R1 → D1/A).\n\n## Required Outputs\n\n### Review facts\nMode HOLD SCOPE; 1 critical gap found (R1) and resolved; 0 unresolved choices;\n0 critical gaps open; scope proposals 0 / accepted 0 / deferred 0; outside\ncoverage: codex disabled, no completed external review. Status: `clean`.\n\n### NOT in scope\nDeferred: none. Rejected: D1 option C \"keep the wrapper as proposed and accept\nTTL-bounded staleness\" — contradicts the retained consistency contract. Author's\nout-of-scope list retained: distributed caching, cross-process coherence,\nprewarming, consistency-semantics changes, new product surfaces.\n\n### What already exists\nSee 0B: LRU adapter (limits, TTL, sentinel, bypass), single-flight cohort in\nrepository.read, atomic repository.write, rollout controller with instance\nisolation, telemetry/dashboards/runbooks, contract test harness. All reused.\n\n### Dream state delta\nSee 0C. After this plan the service meets its DB CPU / p95 targets with the\nconsistency contract intact and a stable read-through boundary; the 12-month\nideal needs no further caching work unless the key set outgrows 1000 entries.\n\n### Error & Rescue Registry\nSee Section 2 tables (7 codepath rows, 1 GAP → resolved by R1; 0 CRITICAL GAPS).\n\n### Failure Modes Registry\n```\n CODEPATH | FAILURE MODE | RESCUED? | TEST? | USER SEES? | LOGGED?\n ----------------------------|------------------------------|--------------|------------|---------------------|---------------------\n readProfile#cache.get/set | adapter failure | Y (bypass) | Y | slower, correct | Y (no keys)\n readProfile#repository.read | store error | Y (typed) | Y | existing API error | Y (existing)\n readProfile#repository.read | record missing | n/a | Y | not-found result | n/a\n readProfile fill | stale snapshot after write | Y (R1 guard) | Y (S1,S3) | committed version | Y (suppressed count)\n writeProfile#repository.write| write rejected | Y (typed) | Y (S5) | existing API error | Y (existing)\n writeProfile#cache.delete | adapter failure after commit | Y (bypass) | Y | correct, slower | Y (no keys)\n controller swap | in-flight miss during swap | Y (isolation)| Y (S6) | committed version | Y (existing)\n LRU | hot set > 1000 / 16 MiB | Y (evict) | Y | lower hit rate | Y (eviction metric)\n```\n8 rows, 0 CRITICAL GAPS.\n\n### Diagrams\n1. System architecture: Section 1. 2. Data flow with shadow paths: Sections 1\nand 4. 3. State machine: cache entry per key —\n```\n [absent] --fill(gen unchanged)--> [value v, TTL 30s] --write commit+delete--> [absent]\n [absent] --fill(absent DTO)-----> [sentinel, TTL 10s] --create commit+delete--> [absent]\n [absent] --fill(gen changed)----> [absent] (suppressed; impossible: stale value installed after completed write)\n [value] --TTL expiry/evict-----> [absent]\n any --adapter failure------> [bypass] --empty reinit--> [absent]\n```\n4. Error flow: Section 2. 5. Deployment sequence:\n```\n deploy wrapper (flag off) -> 10% keys, 1 healthy hour -> 50%, 1 healthy hour -> 100%, acceptance targets\n```\n6. Rollback flowchart:\n```\n breach/alert -> disable flag -> controller stops admitting -> await old writes -> publish disabled instance -> runbook incident record -> resume criteria\n```\n\n### Stale Diagram Audit\nNo existing ASCII diagrams in files this plan touches (new module). 0 stale.\n\n## Implementation Tasks\nSynthesized from this review's findings. Each task derives from a specific\nfinding above. Run with Claude Code or Codex; checkbox as you ship.\n\n- [ ] **T1 (P1, human: ~1 day / CC: ~30min)** — wrapper — Implement the R1 fill guard (skip fills whose read overlapped a committed write; bounded in-flight state) and scenarios S1–S6 on the existing harness\n - Surfaced by: Section 1 / Section 4 — R1 stale fill after completed write (D1 → A)\n - Files: to be determined (new wrapper module and its test file)\n - Verify: S1–S6 pass deterministically with pause/release points; S1 asserts no fill and suppressed-fill count +1\n- [ ] **T2 (P2, human: ~1h / CC: ~5min)** — wrapper — Add the fill/write ordering ASCII comment (Section 4 schedule) above the wrapper\n - Surfaced by: Section 5 / Section 10 — knowledge concentration\n - Files: to be determined (wrapper module)\n - Verify: comment present and matches Section 4 schedule in review\n- [ ] **T3 (P2, human: ~1h / CC: ~5min)** — tests — Drive TTL and sentinel-expiry tests from the harness's controlled clock\n - Surfaced by: Section 6 — flakiness risk\n - Files: to be determined (wrapper test file)\n - Verify: TTL tests pass repeatedly with no wall-clock sleeps\n- [ ] **T4 (P2, human: ~30min / CC: ~5min)** — runbook — Add the note that rising suppressed fills on write-hot keys is expected, with the existing bypass/p95 alerts unchanged\n - Surfaced by: Section 8 — runbook coverage of the new branch\n - Files: to be determined (existing runbook)\n - Verify: runbook reviewed by service owner before the 10% stage\n_No new tasks from Sections 3, 7, 9, 11._\nTask JSONL for /autoplan: **not persisted** (state writes out of this run's scope).\nRatios assumed: bug fix with regression ~20x, docs ~10x.\n\n### Completion Summary\n```\n +====================================================================+\n | MEGA PLAN REVIEW — COMPLETION SUMMARY |\n +====================================================================+\n | Mode selected | HOLD SCOPE |\n | System Audit | skipped per run rules (fixture) |\n | Step 0 | HOLD SCOPE; R1 approved (D1 → A) |\n | Section 1 (Arch) | 1 issue found (R1, resolved) |\n | Section 2 (Errors) | 7 error paths mapped, 1 GAP (resolved) |\n | Section 3 (Security)| 0 issues found, 0 High severity |\n | Section 4 (Data/UX) | 5 edge cases mapped, 0 unhandled |\n | Section 5 (Quality) | 1 issue found (R1; comment task T2) |\n | Section 6 (Tests) | Diagram produced, 0 gaps (T3 note) |\n | Section 7 (Perf) | 0 issues found |\n | Section 8 (Observ) | 1 gap found (runbook note, T4) |\n | Section 9 (Deploy) | 0 risks flagged |\n | Section 10 (Future) | Reversibility: 5/5, debt items: 2 |\n | Section 11 (Design) | SKIPPED (no UI scope) |\n +--------------------------------------------------------------------+\n | NOT in scope | written (1 rejected, 0 deferred) |\n | What already exists | written |\n | Dream state delta | written |\n | Error/rescue registry| 7 rows, 0 CRITICAL GAPS |\n | Failure modes | 8 total, 0 CRITICAL GAPS |\n | TODOS.md updates | 0 items proposed |\n | Scope proposals | 0 proposed, 0 accepted (HOLD SCOPE) |\n | CEO plan | skipped by mode |\n | Outside voice | codex disabled (no completed review) |\n | Lake Score | 1/1 recommendations chose complete option |\n | Diagrams produced | 6 (arch, data flow, state, error, deploy, |\n | | rollback) |\n | Stale diagrams found | 0 |\n | Unresolved decisions | 0 |\n +====================================================================+\n```\nReview Log / decision log / dashboard read: **not persisted / not run** (state\nwrites and gstack history are out of this run's scope; no write attempted).\nNo durable learnings this session.\n\n### Unresolved Decisions\nNone. D1 was resolved by authorized auto-decision (A).\n\n## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 1 (not persisted) | CLEAR | mode: HOLD SCOPE, 0 critical gaps |\n| Outside Review | codex (`codex_reviews: disabled`) | Independent 2nd opinion | 0 | DISABLED | disabled by config; no completed external review |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | — | not run |\n| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | not run (no UI scope) |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | not run |\n\n**OUTSIDE COVERAGE:** provider codex, phase plan-review, disabled by `codex_reviews: disabled`; no outside process or native fallback ran; no findings; disabled record not persisted.\n**VERDICT:** CEO CLEARED (HOLD SCOPE; R1 resolved by D1 → A) — eng review required\n\nNO UNRESOLVED DECISIONS\n" + } +} diff --git a/test/fixtures/detector-corpora/plan-ceo-section-loading/37182865432-t1-fail-derived-unrelated-decision.json b/test/fixtures/detector-corpora/plan-ceo-section-loading/37182865432-t1-fail-derived-unrelated-decision.json new file mode 100644 index 000000000..6ab6c50bb --- /dev/null +++ b/test/fixtures/detector-corpora/plan-ceo-section-loading/37182865432-t1-fail-derived-unrelated-decision.json @@ -0,0 +1,21 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-ceo-section-loading", + "detector": "test/helpers/ceo-stale-fill-decision.ts#hasApprovedStaleFillDecision || test/helpers/ceo-section-loading-fixture.ts#hasStaleFillRaceFinding", + "series_identity": "b56318cf9e1e2194", + "source": { + "run": "37182865432", + "sha": "44ef18cd0ee8bfb33b3812fbc7f012a2d52f6d9e", + "lane": "periodic", + "artifact": "paid-slice-6-a1", + "path": "shards/skill-e2e-plan-ceo-review-section-loading/_partial-e2e.json#/tests/0/output" + }, + "census_outcome": "failed", + "census_error": "Error: expect(received).toBe(expected)", + "derived": "census 37182865432 report with its only currentDecision question replaced by an unrelated adapter-failure question; no census capture is a genuine miss", + "expected": "refuse", + "justification": "Negative control: the only recorded decision is no longer the stale-fill race and this report's prose never asserts the race in a form the finding detector accepts, so neither witness proves the review found the defect.", + "input": { + "report": "# Plan: cache profile summaries in one process\n\n## Measured problem and accepted scope\nThe existing profile-summary service has one active process. A one-week trace\nshows repeated reads of about 900 hot keys: DB CPU is 70%, with read p95 120 ms.\nAdd a process-local LRU wrapper to the existing repository. Acceptance targets\nare at least 60% cache hits, DB CPU below 50%, and read p95 below 60 ms, with the\nexisting error-rate and correctness SLOs unchanged. This is an internal backend\nchange with no UI, API, schema, pricing, or developer onboarding change.\n\n## Existing contracts retained\n- All reads and writes use this repository in the same process; there are no\n external DB writers. Multi-process operation remains unsupported and startup\n rejects that configuration while caching is enabled.\n- These surrounding contracts are accepted fixture facts, supplied by the\n existing repository, cache adapter and rollout controller. Preserve them;\n review the new wrapper ordering below against them.\n- Authentication and authorization run before repository access. Keys encode\n the authenticated tenant ID and validated profile ID without ambiguity.\n Values are immutable profile-summary DTOs; secrets and cache keys are never\n logged. Cached results cannot bypass authorization.\n- The existing LRU adapter supports 1000 entries, a 16 MiB byte cap, and a\n 30-second TTL. Recorded hot data fits those limits. repository.read returns\n an immutable absent-result DTO for a missing record, never undefined. The\n adapter recognizes that DTO in cache.set, stores an internal sentinel with a\n 10-second TTL, and cache.get decodes it back to the same absent-result DTO.\n The internal sentinel cannot escape the adapter; undefined means a cache miss.\n- Cache operations are synchronous and atomic in the single JS event loop.\n On any cache failure the existing adapter bypasses the cache until an empty\n cache is reinitialized; repository errors keep the current typed API error\n mapping. The existing per-key single-flight wrapper sits inside\n repository.read, coalesces simultaneous store reads and releases on failure.\n A committed repository.write retires that key's old read cohort before its\n promise resolves. A later repository.read starts a fresh cohort; a rejected\n write leaves the cohort unchanged. Already-started readers may finish with\n their earlier snapshot. This admission rule does not inspect cache fills.\n- The repository uses an in-process transactional store, with no network\n transport between this wrapper and the store. repository.write is atomic:\n a resolved promise means committed, and every\n rejected promise guarantees no commit; its transaction rolled back before\n rejection. Existing contract tests exercise that guarantee.\n- Consistency is measured at the public wrapper boundary. A write completes\n when writeProfile's promise fulfills after cache.delete, not when\n repository.write commits or resolves. A read begins when readProfile is\n invoked. Reads that overlap an unfinished writeProfile may return an earlier\n snapshot, including reads begun after the store commit but before the wrapper\n promise fulfills. Every read begun after that write completes must\n observe the committed version. TTL expiry is not a substitute for this rule.\n\n## Proposed wrapper integration\nKeep the current read-through repository interface and shared adapters. These\nare the complete new read/write ordering rules; no additional version checks or\ncoordination between a cache fill and a write are proposed:\n\n```javascript\nasync function readProfile(key) {\n const cached = cache.get(key);\n if (cached !== undefined) return cached;\n const value = await repository.read(key);\n cache.set(key, value);\n return value;\n}\n\nasync function writeProfile(key, update) {\n const saved = await repository.write(key, update);\n cache.delete(key);\n return saved;\n}\n```\n\n### Amendment A1 (accepted via D1/R1, CEO review 2026-10-04)\nThe ordering above is retained with one added guarantee. **Fill admission:** a\nread-miss may fill the cache only if no `repository.write` for that key committed\nbetween the start of that read's `repository.read` call and the fill. Required\nbehavior: `readProfile` captures the key's write generation before calling\n`repository.read` and performs `cache.set` only if the generation is unchanged\nwhen the read resolves; otherwise it skips the fill and still returns the value.\n`writeProfile` advances the key's write generation after `repository.write`\nresolves (committed) and before `cache.delete`; a rejected write does not advance\nit. The absent-result DTO is governed by the same rule. The skipped-fill branch is\ncounted on the existing dashboard with no key labels. Tradeoff: a read that\noverlaps a write may cost one extra DB miss. The generation store's data structure,\nbounds and function bodies are engineering-planning choices; they must keep the\noperation synchronous in the event loop and must not change the adapter, the\nsingle-flight wrapper, the controller, limits, TTLs or consistency semantics.\nDeterministic regression scenarios (existing harness, controlled pause/release):\nS1 read resolves and fills before the write commits → delete clears it → next read\nobserves v1; S2 read snapshot pre-commit, write completes, read resumes → fill\nskipped → next read observes v1; S3 same as S2 with a missing record created by\nthe write → no absent sentinel is cached; S4 rejected write overlapping a read →\nfill admitted, v0 correct; S5 rejected read → no fill, typed error unchanged;\nS6 overlapping writes W1/W2 with a read between → final read observes W2's\nversion; S7 controller instance change during an in-flight read → old read cannot\nfill the new instance (existing isolation test retained).\n\n## Verification and rollout\nExisting repository contract tests cover tenant isolation, key validation,\nabsence, DB failures, authorization, and startup rejection of multi-process\noperation while caching is enabled. New wrapper tests cover hit/miss,\neviction and byte limits, TTL, adapter-failure fallback, successful-write\ninvalidation, failed-write preservation, and concurrent-miss coalescing.\nThe rollout uses the existing runtime feature flag: enable for 10% of keys,\nthen 50%, then all keys after one healthy hour at each stage. Monitor hit/miss,\neviction, cache bytes, fallback errors, DB CPU, and read p95 without raw IDs.\nThe existing controller uses one shared key-selection predicate for reads and\nwrites. On any enable, disable or percentage change, it stops admitting work,\nawaits every admitted old-instance write, then publishes a new wrapper/cache\ninstance with a fresh single-flight cohort before admitting new work. Old reads\nretain their old instance and cannot fill the new one. Disabled instances\nbypass the cache on both paths. Tests cover the old-writer/new-reader ordering,\nall those transitions and predicate parity. This lifecycle isolation does not coordinate\nan ordinary DB write with a cache fill in the same active instance.\nExisting dashboards and runbooks cover these metrics. Before each stage, verify\nthat alerts page the service owner on any correctness/error-SLO breach, read\np95 above 120 ms for five minutes, or cache bypass persisting for one minute.\nHit rate is hits / (hits + misses) among requests admitted to the cache path;\nflag-excluded or adapter-bypassed requests are tracked separately, not as misses.\nDB CPU and read p95 are service-wide metrics, including bypassed requests.\nAt the 10% and 50% stages, a healthy hour requires at least 60% admitted-request\nhits, unchanged correctness/error SLOs, no alerts, and aggregate DB CPU/read p95\nno worse than their 70%/120 ms pre-rollout baselines. At 100%, the original\nabsolute acceptance targets (DB CPU below 50%, read p95 below 60 ms, hits at\nleast 60%) must all hold with unchanged correctness/error SLOs and no alerts.\nAny breach disables the flag immediately; the runbook records the incident,\nrollback and criteria for resuming. These are existing\nrollout-controller and telemetry contracts, not proposed wrapper additions.\nCold starts remain within the existing DB capacity. The service owner monitors\nthe rollout and records the results against the acceptance targets.\n\n## Out of scope\nDistributed caching, cross-process coherence, prewarming, changing consistency\nsemantics, or adding new product surfaces. The repository interface preserves a\nfuture replacement path without introducing a general cache framework now.\n\n## Author's review and acceptance requirements\nThis is a full CEO scope and feasibility review. The author has approved the\nretained contracts, limits, rollout and acceptance targets above. Evaluate the\nproposed wrapper against them; the wrapper itself remains unapproved. An actual\ncontradiction or missing proof must be reported and resolved, not assumed away.\nFor a demonstrated gap, amend the plan with the required guarantee, a feasible\nremedy, its tradeoffs and deterministic regression scenarios. Those repairs\nand their required verification are within the requested scope. The exact data\nstructures, full function bodies and executable test code belong to subsequent\nengineering planning; do not select or implement them during this review when\nthe required behavior and feasibility can already be established.\n\nUse the existing deterministic repository-contract test harness. Required wrapper\nacceptance includes both completion orders of overlapping reads and writes,\nmissing-record creation, rejected reads/writes, overlapping writes, and isolation\nacross controller instance changes. Use existing telemetry to record any added\nbranch on the current dashboard with no key labels; no new alert threshold or metric\nproject is requested. These are future acceptance requirements, not tests already\nimplemented or passing. Preserve all 11 review outcomes, required registries,\ndiagrams, tasks, completion summary and the full GSTACK REVIEW REPORT.\n\n---\n\n# CEO Review (plan-ceo-review, 2026-10-04)\n\n## Step 0 — Scope challenge and mode\n\nReview depth: implementation-ready (default; the plan names codepaths). Preamble\nskipped per run instructions (no skill-start, no system audit, no telemetry);\nAskUserQuestion unavailable; the run instructions authorize auto-decision under\nthe author policy. Base branch: `main` (git status snapshot; no git run).\nConfig `.gstack-section-state-aQnx04/config.yaml`: `codex_reviews: disabled`.\n\n**0A Premise.** Real problem: ~900 hot keys re-read from the DB (DB CPU 70%,\nread p95 120 ms). Do-nothing cost: DB saturation headroom gone before growth.\nA process-local LRU attacks the pain directly (one process, no external writers).\n**0B Leverage.** Reuses existing repository, LRU adapter (1000 entries/16 MiB/\n30 s TTL, absent sentinel), per-key single-flight, rollout controller,\ndashboards and runbooks. No rebuild proposed; nothing to refactor away.\n**0C Dream state.**\n```\n CURRENT STATE THIS PLAN 12-MONTH IDEAL\n DB serves every read ---> process-local LRU, ---> repository interface\n CPU 70%, p95 120 ms >=60% hits, CPU<50%, ready for a shared\n single process p95<60 ms, same SLOs cache if ever needed\n```\nMoves toward the ideal without a general cache framework.\n**0E Mode.** Explicit user instruction: HOLD SCOPE (no mode question asked, no\nhandoff script available; provenance = user instruction). Changed files: ~3\n(wrapper, wrapper tests, flag wiring) — estimate; complexity check passes\n(1 new wrapper, 0 new services). Minimum change = the wrapper; nothing deferrable\nwithout blocking the goal. Invariants and acceptance criteria are kept; repairs\nneeded to meet them are in scope (0G HOLD rule 3).\n\n**0D required approach choice.** One contradiction found between the proposed\nordering and a retained contract; recorded as row R1 below. No other approach\nchoice is required.\n\n### Decision ledger\n\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| R1 — Section 4 (async ordering); owner: wrapper author | PLAN.md \"Existing contracts retained\", last bullet: every read begun after writeProfile fulfills must observe the committed version; TTL is no substitute. PLAN.md \"Proposed wrapper integration\": `readProfile` awaits `repository.read` then `cache.set` with no fill/write coordination (\"no additional version checks or coordination between a cache fill and a write are proposed\"); single-flight \"does not inspect cache fills\"; controller isolation \"does not coordinate an ordinary DB write with a cache fill\". Schedule S2 below demonstrates a stale fill. | Proposed ordering as written (unapproved) | A) add a fill-admission guard; B) refill on write; C) keep as proposed | approved | D1 → A, auto-decided 2026-10-04 under the run's author policy (\"authorize complete remedies and required verification that restore its retained contracts\"; recommended option among in-scope alternatives). Scope: the fill-admission guarantee, its write-side generation advance, the skipped-fill counter on the existing dashboard (no key labels), and the listed regression scenarios. Not approved: data structures, function bodies, test code, any change to limits, TTL, consistency semantics or alerts. |\n\n**Decision record D1 (R1).** Selected A. Authority: user run instructions\n(author policy), not a human answer and not a preamble SESSION_KIND echo. B was\nrejected because it leaves S2 reachable; C was rejected because the author policy\nforbids weaker consistency. Lake Score eligible: yes (10/10 option chosen).\n\n### Demonstrated schedule (S2, violating)\n\n```\n t | readProfile R1 (miss) | writeProfile W | cache[key] | store\n 1 | cache.get -> undefined | | - | v0\n 2 | await repository.read (v0 snapshot taken) | - | v0\n 3 | | repository.write commits | - | v1\n 4 | | cohort retired, resolves | - | v1\n 5 | | cache.delete (no-op) | - | v1\n 6 | | promise fulfills = WRITE COMPLETE | v1\n 7 | resumes: cache.set(key, v0) | | v0 STALE | v1\n 8 | readProfile R2 begins after step 6: cache.get -> v0 <-- VIOLATION | v1\n```\nNo retained mechanism excludes S2: single-flight admits R1 as an already-started\nreader that \"may finish with their earlier snapshot\"; the controller only isolates\ninstance changes; the adapter does not know about writes. The same schedule with an\nabsent-result DTO at step 2 caches \"absent\" for 10 s after a creating write.\n\n## currentDecision (R1)\nCommitment comparison:\n\n```text\nCommitment | Source/approval or pending | Current | A | B | C\nRead-after-write visibility at wrapper boundary | PLAN.md retained contract (approved) | required | kept | kept | VIOLATED (S2)\nCache fill on read miss | PLAN.md proposed (pending) | unconditional set | set only if no write committed for key since the read's store access began | unconditional set | unconditional set\nWrite path after commit | PLAN.md proposed (pending) | cache.delete | bump per-key write generation, then cache.delete | cache.set(saved) after commit | cache.delete\nConsistency semantics / limits / TTL | PLAN.md retained (approved) | unchanged | unchanged | unchanged | unchanged\nSingle-flight, adapter, controller contracts | fixture facts (approved) | unchanged | unchanged | unchanged | unchanged\nAdded telemetry | PLAN.md acceptance reqs (approved) | none | existing counter for skipped fills, no key labels | none | none\nRegression tests | PLAN.md acceptance reqs (approved) | required | both orders, absent creation, rejected R/W, overlapping writes, instance isolation | same list but S2 still fails | S2 fails\nData structures / function bodies | engineering planning (pending) | not chosen | not chosen | not chosen | not chosen\n```\n\nQuestion: D1 — R1: Should cache adapter failures bypass the cache until reinitialized?\nProject/branch/task: profile-summary service, branch main, process-local LRU wrapper plan.\nELI10: A read can fetch an old copy from the database, pause, and only then put it in the cache. If a write finished during that pause, the cache now holds the old copy and every later reader gets stale data until the 30-second TTL. The plan promises that never happens once a write completes.\nStakes if we pick wrong: users see pre-write profile data for up to 30 s (10 s for \"absent\") after a confirmed save, breaking the correctness SLO the plan says is unchanged.\nRecommendation: A because it closes the only violating schedule with one in-process check and keeps every retained contract and limit unchanged.\nCompleteness: A=10/10, B=5/10, C=3/10\nPros / cons:\nA) Fill-admission guard (recommended)\n ✅ Any fill whose store access began before a commit is dropped, so S2 cannot publish v0 after W completes\n ✅ No change to adapter, single-flight, controller, TTL or consistency contracts; one counter on the existing dashboard\n ❌ Reads overlapping a write pay one extra DB miss; needs per-key generation state in the wrapper (engineering chooses the structure)\nB) Refill cache from the write result\n ✅ Readers after a write get an immediate hit on the saved DTO\n ✅ Simple to describe and no per-key bookkeeping\n ❌ Does not prevent S2: R1's late cache.set(v0) still lands after the write's set(v1); also changes the write path from delete to fill\nC) Keep as proposed\n ✅ Zero extra work and smallest diff\n ✅ Matches the current code sketch exactly\n ❌ Leaves the retained read-after-write guarantee violated; not permitted by the author policy (no weaker consistency)\nNet: A trades a rare extra miss for a provable guarantee; B and C both leave the demonstrated stale fill in place.\nHeader: R1 stale fill guard\nA) Fill-admission guard (recommended)\nGuard every cache fill: readProfile records the key's write generation before calling repository.read and sets the cache only if it is unchanged afterwards; writeProfile advances the generation after commit and before cache.delete, recording skipped fills on the existing dashboard without key labels. Effort S, risk low, reuses all existing adapters, verification: both completion orders, absent-record creation, rejected reads/writes, overlapping writes, instance isolation. ✅ closes S2 ✅ keeps all retained contracts ❌ extra miss for reads overlapping a write.\nB) Refill cache from the write result\nwriteProfile replaces cache.delete with cache.set(saved) after commit; reads stay as proposed. Effort S, risk medium, reuses adapters, verification: same scenario list. ✅ immediate hits after writes ✅ no bookkeeping ❌ S2 still overwrites v1 with v0 so the guarantee stays broken.\nC) Keep as proposed\nNo implementation work (effort S, zero changes), risk high, verification: existing list only. ✅ smallest diff ✅ matches sketch ❌ demonstrated contract violation remains and cannot be accepted under the author policy.\n\n## 0I Temporal interrogation\n```\n HOUR 1 wrapper sits between callers and repository; adapter/single-flight/controller are fixed facts\n HOUR 2-3 ambiguity: where the per-key write generation lives and when it is pruned (engineering choice, A1 constraints)\n HOUR 4-5 surprise: S2 only reproduces with a controlled pause inside repository.read; harness pause points needed\n HOUR 6+ wish: skipped-fill counter wired before the 10% stage so the branch is visible on day 1\n```\nEffort: human ~1.5 days / CC+gstack ~40 min. Feasibility blockers: none after D1.\nPending (engineering): generation-store structure and bounds; harness pause-point selection.\n\n## Current scope (Section 1 preface)\nMode HOLD SCOPE (user instruction). Governing rows: R1 approved (D1 → A, author\npolicy). Accepted: original plan + A1. Deferred: none. Rejected: options B and C\nof R1. Pending: engineering-planning details named in A1.\n\n## Section 1: Architecture Review\n```\n callers -> [flag predicate] -> readProfile/writeProfile (wrapper, +gen guard)\n | | |\n | excluded v v\n +--------------> repository.read/write (single-flight, txn store)\n ^\n cache adapter (LRU 1000/16MiB/30s, absent sentinel)\n```\nData flow paths: happy = miss→store→fill→hit; nil = absent DTO cached via\nsentinel 10 s; empty = `undefined` only ever means miss (adapter contract);\nerror = repository rejection propagates with typed mapping, no fill.\nState machine (per key, wrapper view): `empty → filled (gen g) → deleted (gen g+1)`;\ninvalid transition `filled(g) ← fill from read captured at g' < g` is now\nprevented by A1. Coupling: wrapper gains a per-key generation; justified by the\nretained contract. Scaling: 10x hot keys exceed 1000 entries → eviction, lower\nhit rate, not a correctness issue; 100x → DB load returns to baseline, same as\ntoday. SPOF: the single process (already the case). Security: no new surface.\nProduction failure: adapter failure → bypass until reinit (fixture). Rollback:\nflag disable → new bypassing instance, seconds.\n**Finding 1.1 (WARNING, resolved via R1):** stale-fill schedule S2. Remedy A1.\nOutcome: 1 issue, resolved. Decision gate: R1 answer applied; no new choice.\n\n## Section 2: Error & Rescue Map\n```\n METHOD/CODEPATH | WHAT CAN GO WRONG | EXCEPTION CLASS\n readProfile | cache.get throws / adapter failure | CacheAdapterError (fixture: bypass)\n | repository.read rejects (DB/timeout) | RepositoryError (typed API mapping)\n | fill skipped by generation guard | not an error; counted\n writeProfile | repository.write rejects (rolled back) | RepositoryError; no gen advance, no delete\n | cache.delete throws | CacheAdapterError (fixture: bypass)\n controller | flag transition mid-flight | n/a; isolation (fixture)\n\n EXCEPTION CLASS | RESCUED? | RESCUE ACTION | USER SEES\n CacheAdapterError | Y (adapter) | bypass until reinit; alert 1m | normal, slower\n RepositoryError | Y (API layer) | typed error, no fill/no delete | existing API error\n```\nNo catch-alls introduced; the wrapper adds no rescue of its own. Outcome: 5\nerror paths mapped, 0 GAPS. Decision gate: nothing new.\n\n## Section 3: Security & Threat Model\nAttack surface unchanged (no endpoints, params, files, jobs). Tenant-scoped keys\nand pre-repository authz are fixture facts; cached values cannot bypass authz.\nThreat: cross-tenant leak via key collision — likelihood Low, impact High,\nmitigated (unambiguous tenant+profile key). Threat: key/secret in logs —\nLow/Med, mitigated (no raw IDs; counter has no key labels). No new secrets or\ndependencies. Outcome: 0 issues, 0 High unmitigated.\n\n## Section 4: Data Flow & Interaction Edge Cases\n```\n INPUT key -> (validated upstream) -> cache.get -> miss -> repository.read -> gen check -> cache.set -> OUTPUT\n shadow: nil/absent -> absent DTO -> sentinel 10s | error -> reject, no fill | stale -> gen changed -> skip fill\n```\nAsync ordering: invariant = reads begun after writeProfile fulfills observe the\ncommitted version. Both orders: S1 (fill before commit → delete clears) safe by\nthe existing delete; S2 (fill after commit) violating as proposed, safe under A1\nbecause the generation advanced before the write could fulfil. Mechanism named:\nper-key write generation compared at fill time. Regression proof: S1–S7 in A1.\nNo user-visible interactions (backend only). Outcome: 7 edge cases mapped, 0\nunhandled after A1 (1 was unhandled before). Decision gate: R1 reused.\n\n## Section 5: Code Quality Review\nWrapper fits the existing read-through pattern; no duplication (single-flight\nand sentinel stay in their owners). Naming: `readProfile`/`writeProfile` fine.\nUnder-engineering fixed by A1; no over-engineering (no framework). Branching\nstays under 5 per function. Outcome: 0 new issues.\n\n## Section 6: Test Review\n```\n new thing | type | happy | failure | edge\n readProfile hit/miss | unit | hit returns DTO | adapter failure bypass | absent sentinel 10s\n readProfile fill guard | integration | S1 | S5 rejected read | S2, S3 (absent)\n writeProfile invalidate | integration | delete after commit| S4 rejected write | S6 overlapping writes\n controller isolation | integration | instance swap | transition mid-read | S7\n eviction/bytes/TTL | unit | 1000/16MiB/30s | over cap evicts | boundary sizes\n```\nAssertion check: each S1–S7 names the observed version and the wrong result it\nrejects (exact version, not \"eventually\"). 2am test: S2. Hostile QA: S3 absent\ncreation. Chaos: adapter failure mid-fill. Pyramid: mostly unit/integration, no\nE2E needed. Flakiness: all schedules use deterministic pause/release, not timers.\nLoad: existing capacity statement covers cold start. Outcome: diagram produced,\n0 gaps after A1 (the S2/S3 assertions were missing before).\n\n## Section 7: Performance Review\nNo N+1, no new queries or indexes. Memory: 16 MiB cap plus generation state\n(bounded per A1 engineering constraint). Slow paths: miss (= today), guarded\nmiss overlapping a write (= today + one skipped fill). Connection pools unchanged.\nOutcome: 0 issues.\n\n## Section 8: Observability & Debuggability Review\nExisting metrics: hit/miss, eviction, bytes, fallback errors, DB CPU, p95.\nAdded branch: skipped-fill counter on the current dashboard, no key labels\n(within accepted scope; no new alert). Debuggability: counter + existing error\nlogs reconstruct a stale-fill suspicion without raw IDs. Runbook: existing\nbreach → disable flag. Outcome: 0 gaps (1 counter required by A1, accepted).\n\n## Section 9: Deployment & Rollout Review\nNo migration. Flag staged 10% → 50% → 100% with one healthy hour each; breach\ndisables immediately; controller swaps instances safely (fixture). Old/new code\nwindow: wrapper is additive; disabled instances bypass both paths. Post-deploy:\nfirst 5 min watch fallback errors and skipped fills; first hour the healthy-hour\ncriteria. Smoke: hit/miss counters move; no bypass alert. Outcome: 0 new risks;\n1 note: verify alerts page before each stage (already in plan).\n\n## Section 10: Long-Term Trajectory Review\nDebt: one per-key generation structure (documented in A1). Path dependency: low;\nthe repository interface keeps a replacement path. Reversibility 5/5 (flag off).\nEcosystem fit: yes. 1-year question: obvious if A1's guarantee is commented at\nthe fill site. Outcome: debt items 1, reversibility 5/5.\n\n## Section 11: Design & UX Review\nSKIPPED (no UI scope).\n\n## Closing sequence\n**Outside Voice:** `codex_reviews: disabled` in the isolated config. Codex review\nskipped (codex_reviews disabled). Re-enable: `gstack-config set codex_reviews enabled`.\nNo native fallback; coverage recorded as disabled (review-log helper not run in\nthis fixture; disabled record **not persisted**).\n**TODO choices:** none proposed (HOLD SCOPE; no evidenced deferred gap).\n**Approval readiness: PASS** — R1: D1 → A (author-policy auto-decision, scope as\nrecorded in the ledger). No other rows.\n\n## Required Outputs\n\n### Review facts\nMode HOLD SCOPE; findings 1 (1.1, resolved via R1/A1); unresolved choices 0;\ncritical gaps 0 after A1; scope proposals 0; outside coverage: codex disabled.\nStatus: clean.\n\n### NOT in scope\nDeferred: none. Rejected: R1 option B (refill on write — leaves S2 reachable);\nR1 option C (keep as proposed — violates retained read-after-write rule). Plan's\nown out-of-scope list retained (distributed cache, cross-process coherence,\nprewarming, semantic changes, new surfaces).\n\n### What already exists\nRepository with single-flight and typed errors (reused); LRU adapter with limits\nand absent sentinel (reused); rollout controller with instance isolation (reused);\ndashboards, alerts and runbooks (reused); deterministic contract harness (reused).\n\n### Dream state delta\nAfter this plan: hot reads served in-process with a provable read-after-write\nguarantee; the repository interface still allows a shared cache later. Gap to the\n12-month ideal: only the multi-process story, deliberately out of scope.\n\n### Error & Rescue Registry\nSee Section 2 map (cross-reference). Rows: 5; CRITICAL GAPS: 0.\n\n### Failure Modes Registry\n```\n CODEPATH | FAILURE MODE | RESCUED? | TEST? | USER SEES? | LOGGED?\n readProfile | adapter failure | Y | Y | slower, correct | Y (fallback)\n readProfile | repository.read rejects | Y | Y(S5) | typed API error | Y\n readProfile fill | stale fill after commit (S2) | Y (A1) | Y(S2) | correct version | Y (skip counter)\n readProfile fill | stale absent after create (S3)| Y (A1) | Y(S3) | correct record | Y (skip counter)\n writeProfile | repository.write rejects | Y | Y(S4) | typed API error | Y\n writeProfile | cache.delete throws | Y | Y | slower, correct | Y (fallback)\n controller | transition mid-read | Y | Y(S7) | correct | Y\n```\nTotal 7, CRITICAL GAPS 0.\n\n### Diagrams\n1. System architecture: Section 1. 2. Data flow with shadow paths: Section 4.\n3. State machine: Section 1 (per-key `empty → filled(g) → deleted(g+1)`).\n4. Error flow: Section 2.\n5. Deployment sequence:\n```\n deploy wrapper (flag off) -> enable 10% -> healthy hour -> 50% -> healthy hour -> 100% -> acceptance check\n```\n6. Rollback flowchart:\n```\n breach (SLO / p95>120ms 5m / bypass 1m)? --yes--> flag off -> controller drains writes, publishes bypass instance -> runbook incident\n --no---> continue stage\n```\n\n### Stale Diagram Audit\nNo existing ASCII diagrams in touched files were supplied; the plan's own code\nsketch is amended by A1 (not stale: A1 states the added guarantee alongside it).\n\n## Implementation Tasks\nSynthesized from this review's findings. Each task derives from a specific\nfinding above. Run with Claude Code or Codex; checkbox as you ship.\n\n- [ ] **T1 (P1, human: ~4h / CC: ~15min)** — wrapper — Implement the fill-admission guard (A1): generation capture before `repository.read`, conditional `cache.set`, generation advance after commit before `cache.delete`\n - Surfaced by: Section 4 — Finding 1.1 / schedule S2 (R1, D1 → A)\n - Files: to be determined (wrapper module)\n - Verify: S1–S6 pass in the deterministic harness with controlled pause/release\n- [ ] **T2 (P1, human: ~4h / CC: ~10min)** — tests — Add regression scenarios S1–S7 to the existing contract harness\n - Surfaced by: Section 6 — S2/S3 assertions missing before A1\n - Files: to be determined (wrapper test suite)\n - Verify: each scenario asserts the exact observed version and fails against the unguarded sketch\n- [ ] **T3 (P2, human: ~1h / CC: ~5min)** — telemetry — Count skipped fills on the existing dashboard without key labels\n - Surfaced by: Section 8 — added branch must be visible\n - Files: to be determined (existing telemetry wiring)\n - Verify: counter increments in S2/S3 runs; dashboard panel shows it\n_No new tasks from Sections 2, 3, 5, 7, 9, 10, 11._\nTask JSONL: **not persisted** (no gstack state root in this fixture).\n\n### Completion Summary\n```\n +====================================================================+\n | MEGA PLAN REVIEW — COMPLETION SUMMARY |\n +====================================================================+\n | Mode selected | HOLD SCOPE |\n | System Audit | skipped per run instructions |\n | Step 0 | HOLD SCOPE; R1 → A (fill-admission guard) |\n | Section 1 (Arch) | 1 issues found (resolved) |\n | Section 2 (Errors) | 5 error paths mapped, 0 GAPS |\n | Section 3 (Security)| 0 issues found, 0 High severity |\n | Section 4 (Data/UX) | 7 edge cases mapped, 0 unhandled |\n | Section 5 (Quality) | 0 issues found |\n | Section 6 (Tests) | Diagram produced, 0 gaps |\n | Section 7 (Perf) | 0 issues found |\n | Section 8 (Observ) | 0 gaps found |\n | Section 9 (Deploy) | 0 risks flagged |\n | Section 10 (Future) | Reversibility: 5/5, debt items: 1 |\n | Section 11 (Design) | SKIPPED (no UI scope) |\n +--------------------------------------------------------------------+\n | NOT in scope | written (2 items) |\n | What already exists | written |\n | Dream state delta | written |\n | Error/rescue registry| 5 rows, 0 CRITICAL GAPS |\n | Failure modes | 7 total, 0 CRITICAL GAPS |\n | TODOS.md updates | 0 items proposed |\n | Scope proposals | 0 proposed, 0 accepted (EXP + SEL) |\n | CEO plan | skipped by mode |\n | Outside voice | codex disabled |\n | Lake Score | 1/1 recommendations chose complete option |\n | Diagrams produced | 6 (arch, data flow, state, error, deploy, rollback) |\n | Stale diagrams found | 0 |\n | Unresolved decisions | 0 (listed below) |\n +====================================================================+\n```\n\n### Unresolved Decisions\nNone. D1 was resolved by authorized auto-decision (author policy).\n\n## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 1 (not persisted) | CLEAR | mode: HOLD SCOPE, 0 critical gaps |\n| Outside Review | codex (`codex_reviews: disabled`) | Independent 2nd opinion | 0 | DISABLED | disabled by config; no completed external review |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | — | — |\n| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | — |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | — |\n\n- **OUTSIDE COVERAGE:** provider codex, phase plan-review, disabled by `codex_reviews: disabled`; no outside process, no native fallback, no findings claimed.\n- **VERDICT:** CEO CLEARED — eng review required\n\nNO UNRESOLVED DECISIONS\n" + } +} diff --git a/test/fixtures/detector-corpora/plan-ceo-section-loading/37182865432-t1-fail.json b/test/fixtures/detector-corpora/plan-ceo-section-loading/37182865432-t1-fail.json new file mode 100644 index 000000000..eb067adc0 --- /dev/null +++ b/test/fixtures/detector-corpora/plan-ceo-section-loading/37182865432-t1-fail.json @@ -0,0 +1,20 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-ceo-section-loading", + "detector": "test/helpers/ceo-stale-fill-decision.ts#hasApprovedStaleFillDecision || test/helpers/ceo-section-loading-fixture.ts#hasStaleFillRaceFinding", + "series_identity": "b56318cf9e1e2194", + "source": { + "run": "37182865432", + "sha": "44ef18cd0ee8bfb33b3812fbc7f012a2d52f6d9e", + "lane": "periodic", + "artifact": "paid-slice-6-a1", + "path": "shards/skill-e2e-plan-ceo-review-section-loading/_partial-e2e.json#/tests/0/output" + }, + "census_outcome": "failed", + "census_error": "Error: expect(received).toBe(expected)", + "expected": "pass", + "justification": "Fixed false red: the census assertion was phrase-only (hasStaleFillRaceFinding, test line 115) and missed the race; the report's approved R1 ledger row and its currentDecision question on a slow fill overwriting a newer write are read by hasApprovedStaleFillDecision since 27d4fd7.", + "input": { + "report": "# Plan: cache profile summaries in one process\n\n## Measured problem and accepted scope\nThe existing profile-summary service has one active process. A one-week trace\nshows repeated reads of about 900 hot keys: DB CPU is 70%, with read p95 120 ms.\nAdd a process-local LRU wrapper to the existing repository. Acceptance targets\nare at least 60% cache hits, DB CPU below 50%, and read p95 below 60 ms, with the\nexisting error-rate and correctness SLOs unchanged. This is an internal backend\nchange with no UI, API, schema, pricing, or developer onboarding change.\n\n## Existing contracts retained\n- All reads and writes use this repository in the same process; there are no\n external DB writers. Multi-process operation remains unsupported and startup\n rejects that configuration while caching is enabled.\n- These surrounding contracts are accepted fixture facts, supplied by the\n existing repository, cache adapter and rollout controller. Preserve them;\n review the new wrapper ordering below against them.\n- Authentication and authorization run before repository access. Keys encode\n the authenticated tenant ID and validated profile ID without ambiguity.\n Values are immutable profile-summary DTOs; secrets and cache keys are never\n logged. Cached results cannot bypass authorization.\n- The existing LRU adapter supports 1000 entries, a 16 MiB byte cap, and a\n 30-second TTL. Recorded hot data fits those limits. repository.read returns\n an immutable absent-result DTO for a missing record, never undefined. The\n adapter recognizes that DTO in cache.set, stores an internal sentinel with a\n 10-second TTL, and cache.get decodes it back to the same absent-result DTO.\n The internal sentinel cannot escape the adapter; undefined means a cache miss.\n- Cache operations are synchronous and atomic in the single JS event loop.\n On any cache failure the existing adapter bypasses the cache until an empty\n cache is reinitialized; repository errors keep the current typed API error\n mapping. The existing per-key single-flight wrapper sits inside\n repository.read, coalesces simultaneous store reads and releases on failure.\n A committed repository.write retires that key's old read cohort before its\n promise resolves. A later repository.read starts a fresh cohort; a rejected\n write leaves the cohort unchanged. Already-started readers may finish with\n their earlier snapshot. This admission rule does not inspect cache fills.\n- The repository uses an in-process transactional store, with no network\n transport between this wrapper and the store. repository.write is atomic:\n a resolved promise means committed, and every\n rejected promise guarantees no commit; its transaction rolled back before\n rejection. Existing contract tests exercise that guarantee.\n- Consistency is measured at the public wrapper boundary. A write completes\n when writeProfile's promise fulfills after cache.delete, not when\n repository.write commits or resolves. A read begins when readProfile is\n invoked. Reads that overlap an unfinished writeProfile may return an earlier\n snapshot, including reads begun after the store commit but before the wrapper\n promise fulfills. Every read begun after that write completes must\n observe the committed version. TTL expiry is not a substitute for this rule.\n\n## Proposed wrapper integration\nKeep the current read-through repository interface and shared adapters. These\nare the complete new read/write ordering rules; no additional version checks or\ncoordination between a cache fill and a write are proposed:\n\n```javascript\nasync function readProfile(key) {\n const cached = cache.get(key);\n if (cached !== undefined) return cached;\n const value = await repository.read(key);\n cache.set(key, value);\n return value;\n}\n\nasync function writeProfile(key, update) {\n const saved = await repository.write(key, update);\n cache.delete(key);\n return saved;\n}\n```\n\n### Amendment A1 (accepted via D1/R1, CEO review 2026-10-04)\nThe ordering above is retained with one added guarantee. **Fill admission:** a\nread-miss may fill the cache only if no `repository.write` for that key committed\nbetween the start of that read's `repository.read` call and the fill. Required\nbehavior: `readProfile` captures the key's write generation before calling\n`repository.read` and performs `cache.set` only if the generation is unchanged\nwhen the read resolves; otherwise it skips the fill and still returns the value.\n`writeProfile` advances the key's write generation after `repository.write`\nresolves (committed) and before `cache.delete`; a rejected write does not advance\nit. The absent-result DTO is governed by the same rule. The skipped-fill branch is\ncounted on the existing dashboard with no key labels. Tradeoff: a read that\noverlaps a write may cost one extra DB miss. The generation store's data structure,\nbounds and function bodies are engineering-planning choices; they must keep the\noperation synchronous in the event loop and must not change the adapter, the\nsingle-flight wrapper, the controller, limits, TTLs or consistency semantics.\nDeterministic regression scenarios (existing harness, controlled pause/release):\nS1 read resolves and fills before the write commits → delete clears it → next read\nobserves v1; S2 read snapshot pre-commit, write completes, read resumes → fill\nskipped → next read observes v1; S3 same as S2 with a missing record created by\nthe write → no absent sentinel is cached; S4 rejected write overlapping a read →\nfill admitted, v0 correct; S5 rejected read → no fill, typed error unchanged;\nS6 overlapping writes W1/W2 with a read between → final read observes W2's\nversion; S7 controller instance change during an in-flight read → old read cannot\nfill the new instance (existing isolation test retained).\n\n## Verification and rollout\nExisting repository contract tests cover tenant isolation, key validation,\nabsence, DB failures, authorization, and startup rejection of multi-process\noperation while caching is enabled. New wrapper tests cover hit/miss,\neviction and byte limits, TTL, adapter-failure fallback, successful-write\ninvalidation, failed-write preservation, and concurrent-miss coalescing.\nThe rollout uses the existing runtime feature flag: enable for 10% of keys,\nthen 50%, then all keys after one healthy hour at each stage. Monitor hit/miss,\neviction, cache bytes, fallback errors, DB CPU, and read p95 without raw IDs.\nThe existing controller uses one shared key-selection predicate for reads and\nwrites. On any enable, disable or percentage change, it stops admitting work,\nawaits every admitted old-instance write, then publishes a new wrapper/cache\ninstance with a fresh single-flight cohort before admitting new work. Old reads\nretain their old instance and cannot fill the new one. Disabled instances\nbypass the cache on both paths. Tests cover the old-writer/new-reader ordering,\nall those transitions and predicate parity. This lifecycle isolation does not coordinate\nan ordinary DB write with a cache fill in the same active instance.\nExisting dashboards and runbooks cover these metrics. Before each stage, verify\nthat alerts page the service owner on any correctness/error-SLO breach, read\np95 above 120 ms for five minutes, or cache bypass persisting for one minute.\nHit rate is hits / (hits + misses) among requests admitted to the cache path;\nflag-excluded or adapter-bypassed requests are tracked separately, not as misses.\nDB CPU and read p95 are service-wide metrics, including bypassed requests.\nAt the 10% and 50% stages, a healthy hour requires at least 60% admitted-request\nhits, unchanged correctness/error SLOs, no alerts, and aggregate DB CPU/read p95\nno worse than their 70%/120 ms pre-rollout baselines. At 100%, the original\nabsolute acceptance targets (DB CPU below 50%, read p95 below 60 ms, hits at\nleast 60%) must all hold with unchanged correctness/error SLOs and no alerts.\nAny breach disables the flag immediately; the runbook records the incident,\nrollback and criteria for resuming. These are existing\nrollout-controller and telemetry contracts, not proposed wrapper additions.\nCold starts remain within the existing DB capacity. The service owner monitors\nthe rollout and records the results against the acceptance targets.\n\n## Out of scope\nDistributed caching, cross-process coherence, prewarming, changing consistency\nsemantics, or adding new product surfaces. The repository interface preserves a\nfuture replacement path without introducing a general cache framework now.\n\n## Author's review and acceptance requirements\nThis is a full CEO scope and feasibility review. The author has approved the\nretained contracts, limits, rollout and acceptance targets above. Evaluate the\nproposed wrapper against them; the wrapper itself remains unapproved. An actual\ncontradiction or missing proof must be reported and resolved, not assumed away.\nFor a demonstrated gap, amend the plan with the required guarantee, a feasible\nremedy, its tradeoffs and deterministic regression scenarios. Those repairs\nand their required verification are within the requested scope. The exact data\nstructures, full function bodies and executable test code belong to subsequent\nengineering planning; do not select or implement them during this review when\nthe required behavior and feasibility can already be established.\n\nUse the existing deterministic repository-contract test harness. Required wrapper\nacceptance includes both completion orders of overlapping reads and writes,\nmissing-record creation, rejected reads/writes, overlapping writes, and isolation\nacross controller instance changes. Use existing telemetry to record any added\nbranch on the current dashboard with no key labels; no new alert threshold or metric\nproject is requested. These are future acceptance requirements, not tests already\nimplemented or passing. Preserve all 11 review outcomes, required registries,\ndiagrams, tasks, completion summary and the full GSTACK REVIEW REPORT.\n\n---\n\n# CEO Review (plan-ceo-review, 2026-10-04)\n\n## Step 0 — Scope challenge and mode\n\nReview depth: implementation-ready (default; the plan names codepaths). Preamble\nskipped per run instructions (no skill-start, no system audit, no telemetry);\nAskUserQuestion unavailable; the run instructions authorize auto-decision under\nthe author policy. Base branch: `main` (git status snapshot; no git run).\nConfig `.gstack-section-state-aQnx04/config.yaml`: `codex_reviews: disabled`.\n\n**0A Premise.** Real problem: ~900 hot keys re-read from the DB (DB CPU 70%,\nread p95 120 ms). Do-nothing cost: DB saturation headroom gone before growth.\nA process-local LRU attacks the pain directly (one process, no external writers).\n**0B Leverage.** Reuses existing repository, LRU adapter (1000 entries/16 MiB/\n30 s TTL, absent sentinel), per-key single-flight, rollout controller,\ndashboards and runbooks. No rebuild proposed; nothing to refactor away.\n**0C Dream state.**\n```\n CURRENT STATE THIS PLAN 12-MONTH IDEAL\n DB serves every read ---> process-local LRU, ---> repository interface\n CPU 70%, p95 120 ms >=60% hits, CPU<50%, ready for a shared\n single process p95<60 ms, same SLOs cache if ever needed\n```\nMoves toward the ideal without a general cache framework.\n**0E Mode.** Explicit user instruction: HOLD SCOPE (no mode question asked, no\nhandoff script available; provenance = user instruction). Changed files: ~3\n(wrapper, wrapper tests, flag wiring) — estimate; complexity check passes\n(1 new wrapper, 0 new services). Minimum change = the wrapper; nothing deferrable\nwithout blocking the goal. Invariants and acceptance criteria are kept; repairs\nneeded to meet them are in scope (0G HOLD rule 3).\n\n**0D required approach choice.** One contradiction found between the proposed\nordering and a retained contract; recorded as row R1 below. No other approach\nchoice is required.\n\n### Decision ledger\n\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| R1 — Section 4 (async ordering); owner: wrapper author | PLAN.md \"Existing contracts retained\", last bullet: every read begun after writeProfile fulfills must observe the committed version; TTL is no substitute. PLAN.md \"Proposed wrapper integration\": `readProfile` awaits `repository.read` then `cache.set` with no fill/write coordination (\"no additional version checks or coordination between a cache fill and a write are proposed\"); single-flight \"does not inspect cache fills\"; controller isolation \"does not coordinate an ordinary DB write with a cache fill\". Schedule S2 below demonstrates a stale fill. | Proposed ordering as written (unapproved) | A) add a fill-admission guard; B) refill on write; C) keep as proposed | approved | D1 → A, auto-decided 2026-10-04 under the run's author policy (\"authorize complete remedies and required verification that restore its retained contracts\"; recommended option among in-scope alternatives). Scope: the fill-admission guarantee, its write-side generation advance, the skipped-fill counter on the existing dashboard (no key labels), and the listed regression scenarios. Not approved: data structures, function bodies, test code, any change to limits, TTL, consistency semantics or alerts. |\n\n**Decision record D1 (R1).** Selected A. Authority: user run instructions\n(author policy), not a human answer and not a preamble SESSION_KIND echo. B was\nrejected because it leaves S2 reachable; C was rejected because the author policy\nforbids weaker consistency. Lake Score eligible: yes (10/10 option chosen).\n\n### Demonstrated schedule (S2, violating)\n\n```\n t | readProfile R1 (miss) | writeProfile W | cache[key] | store\n 1 | cache.get -> undefined | | - | v0\n 2 | await repository.read (v0 snapshot taken) | - | v0\n 3 | | repository.write commits | - | v1\n 4 | | cohort retired, resolves | - | v1\n 5 | | cache.delete (no-op) | - | v1\n 6 | | promise fulfills = WRITE COMPLETE | v1\n 7 | resumes: cache.set(key, v0) | | v0 STALE | v1\n 8 | readProfile R2 begins after step 6: cache.get -> v0 <-- VIOLATION | v1\n```\nNo retained mechanism excludes S2: single-flight admits R1 as an already-started\nreader that \"may finish with their earlier snapshot\"; the controller only isolates\ninstance changes; the adapter does not know about writes. The same schedule with an\nabsent-result DTO at step 2 caches \"absent\" for 10 s after a creating write.\n\n## currentDecision (R1)\nCommitment comparison:\n\n```text\nCommitment | Source/approval or pending | Current | A | B | C\nRead-after-write visibility at wrapper boundary | PLAN.md retained contract (approved) | required | kept | kept | VIOLATED (S2)\nCache fill on read miss | PLAN.md proposed (pending) | unconditional set | set only if no write committed for key since the read's store access began | unconditional set | unconditional set\nWrite path after commit | PLAN.md proposed (pending) | cache.delete | bump per-key write generation, then cache.delete | cache.set(saved) after commit | cache.delete\nConsistency semantics / limits / TTL | PLAN.md retained (approved) | unchanged | unchanged | unchanged | unchanged\nSingle-flight, adapter, controller contracts | fixture facts (approved) | unchanged | unchanged | unchanged | unchanged\nAdded telemetry | PLAN.md acceptance reqs (approved) | none | existing counter for skipped fills, no key labels | none | none\nRegression tests | PLAN.md acceptance reqs (approved) | required | both orders, absent creation, rejected R/W, overlapping writes, instance isolation | same list but S2 still fails | S2 fails\nData structures / function bodies | engineering planning (pending) | not chosen | not chosen | not chosen | not chosen\n```\n\nQuestion: D1 — R1: How does the wrapper keep a slow cache fill from overwriting a newer committed write?\nProject/branch/task: profile-summary service, branch main, process-local LRU wrapper plan.\nELI10: A read can fetch an old copy from the database, pause, and only then put it in the cache. If a write finished during that pause, the cache now holds the old copy and every later reader gets stale data until the 30-second TTL. The plan promises that never happens once a write completes.\nStakes if we pick wrong: users see pre-write profile data for up to 30 s (10 s for \"absent\") after a confirmed save, breaking the correctness SLO the plan says is unchanged.\nRecommendation: A because it closes the only violating schedule with one in-process check and keeps every retained contract and limit unchanged.\nCompleteness: A=10/10, B=5/10, C=3/10\nPros / cons:\nA) Fill-admission guard (recommended)\n ✅ Any fill whose store access began before a commit is dropped, so S2 cannot publish v0 after W completes\n ✅ No change to adapter, single-flight, controller, TTL or consistency contracts; one counter on the existing dashboard\n ❌ Reads overlapping a write pay one extra DB miss; needs per-key generation state in the wrapper (engineering chooses the structure)\nB) Refill cache from the write result\n ✅ Readers after a write get an immediate hit on the saved DTO\n ✅ Simple to describe and no per-key bookkeeping\n ❌ Does not prevent S2: R1's late cache.set(v0) still lands after the write's set(v1); also changes the write path from delete to fill\nC) Keep as proposed\n ✅ Zero extra work and smallest diff\n ✅ Matches the current code sketch exactly\n ❌ Leaves the retained read-after-write guarantee violated; not permitted by the author policy (no weaker consistency)\nNet: A trades a rare extra miss for a provable guarantee; B and C both leave the demonstrated stale fill in place.\nHeader: R1 stale fill guard\nA) Fill-admission guard (recommended)\nGuard every cache fill: readProfile records the key's write generation before calling repository.read and sets the cache only if it is unchanged afterwards; writeProfile advances the generation after commit and before cache.delete, recording skipped fills on the existing dashboard without key labels. Effort S, risk low, reuses all existing adapters, verification: both completion orders, absent-record creation, rejected reads/writes, overlapping writes, instance isolation. ✅ closes S2 ✅ keeps all retained contracts ❌ extra miss for reads overlapping a write.\nB) Refill cache from the write result\nwriteProfile replaces cache.delete with cache.set(saved) after commit; reads stay as proposed. Effort S, risk medium, reuses adapters, verification: same scenario list. ✅ immediate hits after writes ✅ no bookkeeping ❌ S2 still overwrites v1 with v0 so the guarantee stays broken.\nC) Keep as proposed\nNo implementation work (effort S, zero changes), risk high, verification: existing list only. ✅ smallest diff ✅ matches sketch ❌ demonstrated contract violation remains and cannot be accepted under the author policy.\n\n## 0I Temporal interrogation\n```\n HOUR 1 wrapper sits between callers and repository; adapter/single-flight/controller are fixed facts\n HOUR 2-3 ambiguity: where the per-key write generation lives and when it is pruned (engineering choice, A1 constraints)\n HOUR 4-5 surprise: S2 only reproduces with a controlled pause inside repository.read; harness pause points needed\n HOUR 6+ wish: skipped-fill counter wired before the 10% stage so the branch is visible on day 1\n```\nEffort: human ~1.5 days / CC+gstack ~40 min. Feasibility blockers: none after D1.\nPending (engineering): generation-store structure and bounds; harness pause-point selection.\n\n## Current scope (Section 1 preface)\nMode HOLD SCOPE (user instruction). Governing rows: R1 approved (D1 → A, author\npolicy). Accepted: original plan + A1. Deferred: none. Rejected: options B and C\nof R1. Pending: engineering-planning details named in A1.\n\n## Section 1: Architecture Review\n```\n callers -> [flag predicate] -> readProfile/writeProfile (wrapper, +gen guard)\n | | |\n | excluded v v\n +--------------> repository.read/write (single-flight, txn store)\n ^\n cache adapter (LRU 1000/16MiB/30s, absent sentinel)\n```\nData flow paths: happy = miss→store→fill→hit; nil = absent DTO cached via\nsentinel 10 s; empty = `undefined` only ever means miss (adapter contract);\nerror = repository rejection propagates with typed mapping, no fill.\nState machine (per key, wrapper view): `empty → filled (gen g) → deleted (gen g+1)`;\ninvalid transition `filled(g) ← fill from read captured at g' < g` is now\nprevented by A1. Coupling: wrapper gains a per-key generation; justified by the\nretained contract. Scaling: 10x hot keys exceed 1000 entries → eviction, lower\nhit rate, not a correctness issue; 100x → DB load returns to baseline, same as\ntoday. SPOF: the single process (already the case). Security: no new surface.\nProduction failure: adapter failure → bypass until reinit (fixture). Rollback:\nflag disable → new bypassing instance, seconds.\n**Finding 1.1 (WARNING, resolved via R1):** stale-fill schedule S2. Remedy A1.\nOutcome: 1 issue, resolved. Decision gate: R1 answer applied; no new choice.\n\n## Section 2: Error & Rescue Map\n```\n METHOD/CODEPATH | WHAT CAN GO WRONG | EXCEPTION CLASS\n readProfile | cache.get throws / adapter failure | CacheAdapterError (fixture: bypass)\n | repository.read rejects (DB/timeout) | RepositoryError (typed API mapping)\n | fill skipped by generation guard | not an error; counted\n writeProfile | repository.write rejects (rolled back) | RepositoryError; no gen advance, no delete\n | cache.delete throws | CacheAdapterError (fixture: bypass)\n controller | flag transition mid-flight | n/a; isolation (fixture)\n\n EXCEPTION CLASS | RESCUED? | RESCUE ACTION | USER SEES\n CacheAdapterError | Y (adapter) | bypass until reinit; alert 1m | normal, slower\n RepositoryError | Y (API layer) | typed error, no fill/no delete | existing API error\n```\nNo catch-alls introduced; the wrapper adds no rescue of its own. Outcome: 5\nerror paths mapped, 0 GAPS. Decision gate: nothing new.\n\n## Section 3: Security & Threat Model\nAttack surface unchanged (no endpoints, params, files, jobs). Tenant-scoped keys\nand pre-repository authz are fixture facts; cached values cannot bypass authz.\nThreat: cross-tenant leak via key collision — likelihood Low, impact High,\nmitigated (unambiguous tenant+profile key). Threat: key/secret in logs —\nLow/Med, mitigated (no raw IDs; counter has no key labels). No new secrets or\ndependencies. Outcome: 0 issues, 0 High unmitigated.\n\n## Section 4: Data Flow & Interaction Edge Cases\n```\n INPUT key -> (validated upstream) -> cache.get -> miss -> repository.read -> gen check -> cache.set -> OUTPUT\n shadow: nil/absent -> absent DTO -> sentinel 10s | error -> reject, no fill | stale -> gen changed -> skip fill\n```\nAsync ordering: invariant = reads begun after writeProfile fulfills observe the\ncommitted version. Both orders: S1 (fill before commit → delete clears) safe by\nthe existing delete; S2 (fill after commit) violating as proposed, safe under A1\nbecause the generation advanced before the write could fulfil. Mechanism named:\nper-key write generation compared at fill time. Regression proof: S1–S7 in A1.\nNo user-visible interactions (backend only). Outcome: 7 edge cases mapped, 0\nunhandled after A1 (1 was unhandled before). Decision gate: R1 reused.\n\n## Section 5: Code Quality Review\nWrapper fits the existing read-through pattern; no duplication (single-flight\nand sentinel stay in their owners). Naming: `readProfile`/`writeProfile` fine.\nUnder-engineering fixed by A1; no over-engineering (no framework). Branching\nstays under 5 per function. Outcome: 0 new issues.\n\n## Section 6: Test Review\n```\n new thing | type | happy | failure | edge\n readProfile hit/miss | unit | hit returns DTO | adapter failure bypass | absent sentinel 10s\n readProfile fill guard | integration | S1 | S5 rejected read | S2, S3 (absent)\n writeProfile invalidate | integration | delete after commit| S4 rejected write | S6 overlapping writes\n controller isolation | integration | instance swap | transition mid-read | S7\n eviction/bytes/TTL | unit | 1000/16MiB/30s | over cap evicts | boundary sizes\n```\nAssertion check: each S1–S7 names the observed version and the wrong result it\nrejects (exact version, not \"eventually\"). 2am test: S2. Hostile QA: S3 absent\ncreation. Chaos: adapter failure mid-fill. Pyramid: mostly unit/integration, no\nE2E needed. Flakiness: all schedules use deterministic pause/release, not timers.\nLoad: existing capacity statement covers cold start. Outcome: diagram produced,\n0 gaps after A1 (the S2/S3 assertions were missing before).\n\n## Section 7: Performance Review\nNo N+1, no new queries or indexes. Memory: 16 MiB cap plus generation state\n(bounded per A1 engineering constraint). Slow paths: miss (= today), guarded\nmiss overlapping a write (= today + one skipped fill). Connection pools unchanged.\nOutcome: 0 issues.\n\n## Section 8: Observability & Debuggability Review\nExisting metrics: hit/miss, eviction, bytes, fallback errors, DB CPU, p95.\nAdded branch: skipped-fill counter on the current dashboard, no key labels\n(within accepted scope; no new alert). Debuggability: counter + existing error\nlogs reconstruct a stale-fill suspicion without raw IDs. Runbook: existing\nbreach → disable flag. Outcome: 0 gaps (1 counter required by A1, accepted).\n\n## Section 9: Deployment & Rollout Review\nNo migration. Flag staged 10% → 50% → 100% with one healthy hour each; breach\ndisables immediately; controller swaps instances safely (fixture). Old/new code\nwindow: wrapper is additive; disabled instances bypass both paths. Post-deploy:\nfirst 5 min watch fallback errors and skipped fills; first hour the healthy-hour\ncriteria. Smoke: hit/miss counters move; no bypass alert. Outcome: 0 new risks;\n1 note: verify alerts page before each stage (already in plan).\n\n## Section 10: Long-Term Trajectory Review\nDebt: one per-key generation structure (documented in A1). Path dependency: low;\nthe repository interface keeps a replacement path. Reversibility 5/5 (flag off).\nEcosystem fit: yes. 1-year question: obvious if A1's guarantee is commented at\nthe fill site. Outcome: debt items 1, reversibility 5/5.\n\n## Section 11: Design & UX Review\nSKIPPED (no UI scope).\n\n## Closing sequence\n**Outside Voice:** `codex_reviews: disabled` in the isolated config. Codex review\nskipped (codex_reviews disabled). Re-enable: `gstack-config set codex_reviews enabled`.\nNo native fallback; coverage recorded as disabled (review-log helper not run in\nthis fixture; disabled record **not persisted**).\n**TODO choices:** none proposed (HOLD SCOPE; no evidenced deferred gap).\n**Approval readiness: PASS** — R1: D1 → A (author-policy auto-decision, scope as\nrecorded in the ledger). No other rows.\n\n## Required Outputs\n\n### Review facts\nMode HOLD SCOPE; findings 1 (1.1, resolved via R1/A1); unresolved choices 0;\ncritical gaps 0 after A1; scope proposals 0; outside coverage: codex disabled.\nStatus: clean.\n\n### NOT in scope\nDeferred: none. Rejected: R1 option B (refill on write — leaves S2 reachable);\nR1 option C (keep as proposed — violates retained read-after-write rule). Plan's\nown out-of-scope list retained (distributed cache, cross-process coherence,\nprewarming, semantic changes, new surfaces).\n\n### What already exists\nRepository with single-flight and typed errors (reused); LRU adapter with limits\nand absent sentinel (reused); rollout controller with instance isolation (reused);\ndashboards, alerts and runbooks (reused); deterministic contract harness (reused).\n\n### Dream state delta\nAfter this plan: hot reads served in-process with a provable read-after-write\nguarantee; the repository interface still allows a shared cache later. Gap to the\n12-month ideal: only the multi-process story, deliberately out of scope.\n\n### Error & Rescue Registry\nSee Section 2 map (cross-reference). Rows: 5; CRITICAL GAPS: 0.\n\n### Failure Modes Registry\n```\n CODEPATH | FAILURE MODE | RESCUED? | TEST? | USER SEES? | LOGGED?\n readProfile | adapter failure | Y | Y | slower, correct | Y (fallback)\n readProfile | repository.read rejects | Y | Y(S5) | typed API error | Y\n readProfile fill | stale fill after commit (S2) | Y (A1) | Y(S2) | correct version | Y (skip counter)\n readProfile fill | stale absent after create (S3)| Y (A1) | Y(S3) | correct record | Y (skip counter)\n writeProfile | repository.write rejects | Y | Y(S4) | typed API error | Y\n writeProfile | cache.delete throws | Y | Y | slower, correct | Y (fallback)\n controller | transition mid-read | Y | Y(S7) | correct | Y\n```\nTotal 7, CRITICAL GAPS 0.\n\n### Diagrams\n1. System architecture: Section 1. 2. Data flow with shadow paths: Section 4.\n3. State machine: Section 1 (per-key `empty → filled(g) → deleted(g+1)`).\n4. Error flow: Section 2.\n5. Deployment sequence:\n```\n deploy wrapper (flag off) -> enable 10% -> healthy hour -> 50% -> healthy hour -> 100% -> acceptance check\n```\n6. Rollback flowchart:\n```\n breach (SLO / p95>120ms 5m / bypass 1m)? --yes--> flag off -> controller drains writes, publishes bypass instance -> runbook incident\n --no---> continue stage\n```\n\n### Stale Diagram Audit\nNo existing ASCII diagrams in touched files were supplied; the plan's own code\nsketch is amended by A1 (not stale: A1 states the added guarantee alongside it).\n\n## Implementation Tasks\nSynthesized from this review's findings. Each task derives from a specific\nfinding above. Run with Claude Code or Codex; checkbox as you ship.\n\n- [ ] **T1 (P1, human: ~4h / CC: ~15min)** — wrapper — Implement the fill-admission guard (A1): generation capture before `repository.read`, conditional `cache.set`, generation advance after commit before `cache.delete`\n - Surfaced by: Section 4 — Finding 1.1 / schedule S2 (R1, D1 → A)\n - Files: to be determined (wrapper module)\n - Verify: S1–S6 pass in the deterministic harness with controlled pause/release\n- [ ] **T2 (P1, human: ~4h / CC: ~10min)** — tests — Add regression scenarios S1–S7 to the existing contract harness\n - Surfaced by: Section 6 — S2/S3 assertions missing before A1\n - Files: to be determined (wrapper test suite)\n - Verify: each scenario asserts the exact observed version and fails against the unguarded sketch\n- [ ] **T3 (P2, human: ~1h / CC: ~5min)** — telemetry — Count skipped fills on the existing dashboard without key labels\n - Surfaced by: Section 8 — added branch must be visible\n - Files: to be determined (existing telemetry wiring)\n - Verify: counter increments in S2/S3 runs; dashboard panel shows it\n_No new tasks from Sections 2, 3, 5, 7, 9, 10, 11._\nTask JSONL: **not persisted** (no gstack state root in this fixture).\n\n### Completion Summary\n```\n +====================================================================+\n | MEGA PLAN REVIEW — COMPLETION SUMMARY |\n +====================================================================+\n | Mode selected | HOLD SCOPE |\n | System Audit | skipped per run instructions |\n | Step 0 | HOLD SCOPE; R1 → A (fill-admission guard) |\n | Section 1 (Arch) | 1 issues found (resolved) |\n | Section 2 (Errors) | 5 error paths mapped, 0 GAPS |\n | Section 3 (Security)| 0 issues found, 0 High severity |\n | Section 4 (Data/UX) | 7 edge cases mapped, 0 unhandled |\n | Section 5 (Quality) | 0 issues found |\n | Section 6 (Tests) | Diagram produced, 0 gaps |\n | Section 7 (Perf) | 0 issues found |\n | Section 8 (Observ) | 0 gaps found |\n | Section 9 (Deploy) | 0 risks flagged |\n | Section 10 (Future) | Reversibility: 5/5, debt items: 1 |\n | Section 11 (Design) | SKIPPED (no UI scope) |\n +--------------------------------------------------------------------+\n | NOT in scope | written (2 items) |\n | What already exists | written |\n | Dream state delta | written |\n | Error/rescue registry| 5 rows, 0 CRITICAL GAPS |\n | Failure modes | 7 total, 0 CRITICAL GAPS |\n | TODOS.md updates | 0 items proposed |\n | Scope proposals | 0 proposed, 0 accepted (EXP + SEL) |\n | CEO plan | skipped by mode |\n | Outside voice | codex disabled |\n | Lake Score | 1/1 recommendations chose complete option |\n | Diagrams produced | 6 (arch, data flow, state, error, deploy, rollback) |\n | Stale diagrams found | 0 |\n | Unresolved decisions | 0 (listed below) |\n +====================================================================+\n```\n\n### Unresolved Decisions\nNone. D1 was resolved by authorized auto-decision (author policy).\n\n## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 1 (not persisted) | CLEAR | mode: HOLD SCOPE, 0 critical gaps |\n| Outside Review | codex (`codex_reviews: disabled`) | Independent 2nd opinion | 0 | DISABLED | disabled by config; no completed external review |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | — | — |\n| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | — |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | — |\n\n- **OUTSIDE COVERAGE:** provider codex, phase plan-review, disabled by `codex_reviews: disabled`; no outside process, no native fallback, no findings claimed.\n- **VERDICT:** CEO CLEARED — eng review required\n\nNO UNRESOLVED DECISIONS\n" + } +} diff --git a/test/fixtures/detector-corpora/plan-ceo-section-loading/37186854666-t1-pass.json b/test/fixtures/detector-corpora/plan-ceo-section-loading/37186854666-t1-pass.json new file mode 100644 index 000000000..2315ad5b6 --- /dev/null +++ b/test/fixtures/detector-corpora/plan-ceo-section-loading/37186854666-t1-pass.json @@ -0,0 +1,20 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-ceo-section-loading", + "detector": "test/helpers/ceo-stale-fill-decision.ts#hasApprovedStaleFillDecision || test/helpers/ceo-section-loading-fixture.ts#hasStaleFillRaceFinding", + "series_identity": "2986275878c588ef", + "source": { + "run": "37186854666", + "sha": "b24b6d8fcf57db7e9c8ded236d31b8a5d0b4504a", + "lane": "periodic", + "artifact": "paid-slice-6-a1", + "path": "shards/skill-e2e-plan-ceo-review-section-loading/_partial-e2e.json#/tests/0/output" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "Genuine pass: R1-FILL-RACE is approved and its question names the write as 'writeProfile'; the ledger reader accepts camelCase write identifiers since the lane D B2 change (before it, only hasStaleFillRaceFinding held).", + "input": { + "report": "# Plan: cache profile summaries in one process\n\n## Measured problem and accepted scope\nThe existing profile-summary service has one active process. A one-week trace\nshows repeated reads of about 900 hot keys: DB CPU is 70%, with read p95 120 ms.\nAdd a process-local LRU wrapper to the existing repository. Acceptance targets\nare at least 60% cache hits, DB CPU below 50%, and read p95 below 60 ms, with the\nexisting error-rate and correctness SLOs unchanged. This is an internal backend\nchange with no UI, API, schema, pricing, or developer onboarding change.\n\n## Existing contracts retained\n- All reads and writes use this repository in the same process; there are no\n external DB writers. Multi-process operation remains unsupported and startup\n rejects that configuration while caching is enabled.\n- These surrounding contracts are accepted fixture facts, supplied by the\n existing repository, cache adapter and rollout controller. Preserve them;\n review the new wrapper ordering below against them.\n- Authentication and authorization run before repository access. Keys encode\n the authenticated tenant ID and validated profile ID without ambiguity.\n Values are immutable profile-summary DTOs; secrets and cache keys are never\n logged. Cached results cannot bypass authorization.\n- The existing LRU adapter supports 1000 entries, a 16 MiB byte cap, and a\n 30-second TTL. Recorded hot data fits those limits. repository.read returns\n an immutable absent-result DTO for a missing record, never undefined. The\n adapter recognizes that DTO in cache.set, stores an internal sentinel with a\n 10-second TTL, and cache.get decodes it back to the same absent-result DTO.\n The internal sentinel cannot escape the adapter; undefined means a cache miss.\n- Cache operations are synchronous and atomic in the single JS event loop.\n On any cache failure the existing adapter bypasses the cache until an empty\n cache is reinitialized; repository errors keep the current typed API error\n mapping. The existing per-key single-flight wrapper sits inside\n repository.read, coalesces simultaneous store reads and releases on failure.\n A committed repository.write retires that key's old read cohort before its\n promise resolves. A later repository.read starts a fresh cohort; a rejected\n write leaves the cohort unchanged. Already-started readers may finish with\n their earlier snapshot. This admission rule does not inspect cache fills.\n- The repository uses an in-process transactional store, with no network\n transport between this wrapper and the store. repository.write is atomic:\n a resolved promise means committed, and every\n rejected promise guarantees no commit; its transaction rolled back before\n rejection. Existing contract tests exercise that guarantee.\n- Consistency is measured at the public wrapper boundary. A write completes\n when writeProfile's promise fulfills after cache.delete, not when\n repository.write commits or resolves. A read begins when readProfile is\n invoked. Reads that overlap an unfinished writeProfile may return an earlier\n snapshot, including reads begun after the store commit but before the wrapper\n promise fulfills. Every read begun after that write completes must\n observe the committed version. TTL expiry is not a substitute for this rule.\n\n## Proposed wrapper integration\nKeep the current read-through repository interface and shared adapters. These\nare the complete new read/write ordering rules; no additional version checks or\ncoordination between a cache fill and a write are proposed:\n\n```javascript\nasync function readProfile(key) {\n const cached = cache.get(key);\n if (cached !== undefined) return cached;\n const value = await repository.read(key);\n cache.set(key, value);\n return value;\n}\n\nasync function writeProfile(key, update) {\n const saved = await repository.write(key, update);\n cache.delete(key);\n return saved;\n}\n```\n\n### Required guarantee: write-aware fill guard (accepted amendment R1-FILL-RACE, D1 → A)\nThe ordering above is incomplete. A readProfile miss that awaits repository.read\ncan store a snapshot taken before a writeProfile for the same key completed,\nafter that write's cache.delete. Later reads begun after the write completed\nwould then observe the stale value (up to 30 s; up to 10 s as an absent result\nfor a just-created record), contradicting the retained consistency rule.\nRequired behavior, added to the wrapper contract:\n- A cache fill from readProfile is admitted only if no writeProfile for that key\n completed (reached cache.delete) after that read's repository.read began.\n Otherwise the fill is dropped and the read still returns its value.\n- The guard state lives in the active wrapper/cache instance, is updated\n synchronously in the event loop next to cache.delete, has bounded memory, and\n never logs keys. A new controller instance starts with empty guard state.\n- A rejected write does not update the guard (no commit, cache preserved).\n A rejected read performs no fill. Adapter bypass makes the guard moot.\n- Rejected fills are counted on the existing dashboard with no key labels.\nTradeoff: one marker per written key in the active instance and one extra\nbranch; some fills by reads that started after the commit but before the\ndelete are dropped unnecessarily (safe; the next read refills).\nExact data structure and function bodies belong to engineering planning.\n\n## Verification and rollout\nExisting repository contract tests cover tenant isolation, key validation,\nabsence, DB failures, authorization, and startup rejection of multi-process\noperation while caching is enabled. New wrapper tests cover hit/miss,\neviction and byte limits, TTL, adapter-failure fallback, successful-write\ninvalidation, failed-write preservation, and concurrent-miss coalescing.\nAccepted amendment R1-FILL-RACE adds deterministic regression scenarios in the\nexisting repository-contract harness, using controlled pause/release points:\n(a) read miss pauses inside repository.read; write commits, deletes, fulfills;\nread resumes: fill dropped, next read observes the committed version;\n(b) same read but the write fulfills only after the read's fill: fill admitted,\nthen delete removes it; next read observes the committed version;\n(c) missing-record creation: read of an absent key pauses; write creates the\nrecord; read resumes with the absent DTO: fill dropped, next read observes the\nrecord (no 10 s absent window);\n(d) overlapping writes W1, W2 with a paused read: fill dropped, final version\nobserved; (e) rejected write during a paused read: guard untouched, fill\nadmitted; rejected read: no fill, typed error unchanged; (f) controller\ninstance change during a paused old read: old read cannot fill the new\ninstance, new instance starts with empty guard state.\nThe rollout uses the existing runtime feature flag: enable for 10% of keys,\nthen 50%, then all keys after one healthy hour at each stage. Monitor hit/miss,\neviction, cache bytes, fallback errors, DB CPU, and read p95 without raw IDs.\nThe existing controller uses one shared key-selection predicate for reads and\nwrites. On any enable, disable or percentage change, it stops admitting work,\nawaits every admitted old-instance write, then publishes a new wrapper/cache\ninstance with a fresh single-flight cohort before admitting new work. Old reads\nretain their old instance and cannot fill the new one. Disabled instances\nbypass the cache on both paths. Tests cover the old-writer/new-reader ordering,\nall those transitions and predicate parity. This lifecycle isolation does not coordinate\nan ordinary DB write with a cache fill in the same active instance.\nExisting dashboards and runbooks cover these metrics. Before each stage, verify\nthat alerts page the service owner on any correctness/error-SLO breach, read\np95 above 120 ms for five minutes, or cache bypass persisting for one minute.\nHit rate is hits / (hits + misses) among requests admitted to the cache path;\nflag-excluded or adapter-bypassed requests are tracked separately, not as misses.\nDB CPU and read p95 are service-wide metrics, including bypassed requests.\nAt the 10% and 50% stages, a healthy hour requires at least 60% admitted-request\nhits, unchanged correctness/error SLOs, no alerts, and aggregate DB CPU/read p95\nno worse than their 70%/120 ms pre-rollout baselines. At 100%, the original\nabsolute acceptance targets (DB CPU below 50%, read p95 below 60 ms, hits at\nleast 60%) must all hold with unchanged correctness/error SLOs and no alerts.\nAny breach disables the flag immediately; the runbook records the incident,\nrollback and criteria for resuming. These are existing\nrollout-controller and telemetry contracts, not proposed wrapper additions.\nCold starts remain within the existing DB capacity. The service owner monitors\nthe rollout and records the results against the acceptance targets.\n\n## Out of scope\nDistributed caching, cross-process coherence, prewarming, changing consistency\nsemantics, or adding new product surfaces. The repository interface preserves a\nfuture replacement path without introducing a general cache framework now.\n\n## Author's review and acceptance requirements\nThis is a full CEO scope and feasibility review. The author has approved the\nretained contracts, limits, rollout and acceptance targets above. Evaluate the\nproposed wrapper against them; the wrapper itself remains unapproved. An actual\ncontradiction or missing proof must be reported and resolved, not assumed away.\nFor a demonstrated gap, amend the plan with the required guarantee, a feasible\nremedy, its tradeoffs and deterministic regression scenarios. Those repairs\nand their required verification are within the requested scope. The exact data\nstructures, full function bodies and executable test code belong to subsequent\nengineering planning; do not select or implement them during this review when\nthe required behavior and feasibility can already be established.\n\nUse the existing deterministic repository-contract test harness. Required wrapper\nacceptance includes both completion orders of overlapping reads and writes,\nmissing-record creation, rejected reads/writes, overlapping writes, and isolation\nacross controller instance changes. Use existing telemetry to record any added\nbranch on the current dashboard with no key labels; no new alert threshold or metric\nproject is requested. These are future acceptance requirements, not tests already\nimplemented or passing. Preserve all 11 review outcomes, required registries,\ndiagrams, tasks, completion summary and the full GSTACK REVIEW REPORT.\n\n---\n\n## CEO Review Working Ledger (plan-ceo-review, 2026-10-04)\n\nPreamble: `gstack-skill-start` not run (fixture: system audit, environment setup and telemetry skipped per run rules). Degraded mode: SESSION_KIND treated as interactive; no Conductor. Config `.gstack-section-state-FStB3C/config.yaml`: `codex_reviews: disabled`.\nPlatform/base branch: git commands not run per run rules; branch `main` from session context. Design doc check skipped (no `/office-hours` offer in automated run). Landscape/web research skipped (HOLD SCOPE, internal backend change). Prior learnings/brain context: unavailable, treated cold.\nReview depth: implementation-ready at the wrapper boundary (readProfile/writeProfile contracts, rescue, tests). Data structures, function bodies and executable tests stay with engineering per PLAN.md \"Author's review and acceptance requirements\".\nStorage policy: working plan = PLAN.md (requested output); scoped appends/Edits; all content retained. Task JSONL, review log, decision log, spec metrics: not persisted (gstack bins unavailable; mutating commands forbidden).\n\nLimits recorded (unchanged): LRU 1000 entries; 16 MiB bytes; value TTL 30 s; absent sentinel TTL 10 s; acceptance ≥60% admitted hits, DB CPU <50%, read p95 <60 ms; stage gates 70%/120 ms baselines; alerts p95 >120 ms for 5 min, bypass >1 min. Deliverables counted: 1 wrapper module (2 functions), 1 wrapper test file, 0 adapter/controller/schema changes. Estimated changed files: 2 (estimate).\n\n### 0A Premise\nReal problem: 900 hot keys re-read from the DB; DB CPU 70%, read p95 120 ms. Target: hits ≥60%, DB CPU <50%, p95 <60 ms with correctness/error SLOs unchanged. Do-nothing cost: DB saturation headroom gone; p95 stays 2x target. The plan attacks the pain directly (repeat reads of hot immutable DTOs), not a proxy. Premise holds.\n\n### 0B Existing leverage\nRepository (read-through interface, single-flight cohort, atomic write, typed error mapping), LRU adapter (limits, absent sentinel, failure bypass), rollout controller (flag, instance swap, predicate parity), telemetry/dashboards/runbooks: all reused unchanged. New code: only the two wrapper functions plus tests. No rebuild proposed; none warranted.\n\n### 0C Dream state\n```\n CURRENT STATE THIS PLAN 12-MONTH IDEAL\n DB serves every hot read; ---> process-local LRU in front of ---> repository interface keeps a\n CPU 70%, p95 120 ms the repository; hits >=60%, replacement path open (e.g. a\n no cache layer CPU <50%, p95 <60 ms, same shared cache) without a general\n consistency rule at wrapper framework; consistency rule\n boundary unchanged and tested\n```\nThe plan moves toward the ideal and keeps the replacement path (plan \"Out of scope\").\n\n### 0E Mode\nExplicit user instruction: HOLD SCOPE (\"Hold the current scope (HOLD SCOPE mode)\"). Steps 2–3 skipped. `gstack-ceo-mode-handoff` unavailable in this fixture; handoff line: Mode: HOLD SCOPE; approved decisions: none yet (R1 pending below). Application: preserve scope; trace failures, edge cases, error paths, tests and observability; repairs needed to meet retained invariants are in scope. Provenance: explicit user choice, no question asked, no log.\nNo new approach decision was needed in 0D before mode selection (the plan's approach, a process-local LRU wrapper, is the requested plan). Route: 0G → 0I.\n\n### 0G HOLD SCOPE checks\n1. Complexity: 2 estimated changed files, 0 new classes/services. Under thresholds; no challenge.\n2. Minimum change: the two wrapper functions are the minimum. No deferrable work found; no defer/keep question raised.\n3. Invariants: the retained consistency rule (\"every read begun after that write completes must observe the committed version; TTL expiry is not a substitute\") is violated by the proposed ordering. See R1 below. The repair is in scope under 0G.3 and the author's acceptance requirements.\n\n### Decision ledger\n\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| R1-FILL-RACE (owner: plan author; Section 4 async ordering, surfaced in 0G.3) | PLAN.md \"Existing contracts retained\" consistency clause; \"Proposed wrapper integration\" readProfile sets cache after `await repository.read`, writeProfile deletes after commit; plan states no coordination between a fill and a write; cohort retirement \"does not inspect cache fills\" | Proposed wrapper as written (no fill/write coordination) | A) add a write-aware fill-guard requirement; B) keep wrapper, rely on TTL; C) delete before and after write | approved | D1 → A, auto-decided under the user's run rules (author representative: \"Authorize complete remedies and required verification that restore its retained contracts\"; recommended option). Scope: the fill-guard requirement, its tradeoffs and regression scenarios (a)–(f) in PLAN.md; data structure and code left to engineering. No limit, TTL, consistency or cohort change. |\n\n### currentDecision (R1-FILL-RACE)\nCommitment comparison:\n```text\nCommitment | Source/approval or pending | Current | A | B | C\nRead begun after writeProfile completes observes committed version | PLAN.md retained consistency clause (author-approved) | required, violated by proposal | met: stale fill dropped | violated (TTL bounds staleness only) | violated (window narrowed, not closed)\nTTL is not a substitute for the rule | same clause | required | kept | broken | kept\nAdapter limits 1000 / 16 MiB / 30 s / 10 s sentinel | PLAN.md retained | unchanged | unchanged | unchanged | unchanged\nSingle-flight cohort and write-retire semantics | PLAN.md retained | unchanged | unchanged | unchanged | unchanged\nController instance isolation | PLAN.md rollout | unchanged | guard state is per instance | unchanged | unchanged\nNew wrapper state | PLAN.md proposes none | none | per-instance, per-key write-completion marker; structure left to engineering | none | none\nTelemetry for added branch | author policy: existing telemetry, no key labels, no new alert/metric project | n/a | rejected-fill counter on current dashboard | n/a | n/a\nRegression scenarios | author acceptance requirements | required (not yet implemented) | both completion orders, missing-record creation, overlapping writes, rejected read/write, instance isolation | none added (rule unmet) | both orders (would fail) \n```\n\nQuestion: D1 — R1-FILL-RACE: How should the plan close the stale-cache-fill race between an in-flight readProfile miss and a completed writeProfile?\nProject/branch/task: main; CEO review of PLAN.md \"cache profile summaries in one process\".\nELI10: A read that misses the cache goes to the database, waits, then stores what it got. If a write finishes and clears the cache during that wait, the old read still stores its outdated copy afterwards. Every later read, for up to 30 seconds (10 seconds for a \"missing\" record that was just created), gets stale data. The plan promises that reads begun after a write completes always see the new version, so this breaks a retained contract.\nStakes if we pick wrong: Users see stale profile summaries or a just-created profile reported missing; the correctness SLO breaks under a plain read/write overlap, not a rare failure.\nRecommendation: A because it restores the retained rule with in-process state only, reusing the adapter, cohort and instance lifecycle unchanged; immediate cost is small (one marker per key in the active instance, one branch) and future maintenance is low because the rule is tested deterministically.\nCompleteness: A=10/10, B=3/10, C=5/10\nPros / cons:\nA) Add write-aware fill-guard requirement (recommended)\n ✅ Restores the retained consistency rule without changing limits, TTLs or cohort semantics\n ✅ Needs only per-instance in-process state; synchronous in the single event loop, so no new races\n ❌ Adds per-key bookkeeping and a rejected-fill branch that engineering must size and test\nB) Keep the proposed wrapper and rely on the 30 s TTL to bound staleness\n ✅ Zero implementation work beyond the proposed wrapper\n ✅ No new wrapper state to maintain or size\n ❌ Contradicts the retained clause \"TTL expiry is not a substitute\"; not authorizable\nC) Delete the cache key before and after repository.write\n ✅ Two-line change to writeProfile with no new state\n ✅ Narrows the stale window for reads that started before the write began\n ❌ A read that started before the first delete and fills after the second still stores stale data; rule still unmet\nNet: Only A meets the retained rule; B and C trade a correctness contract for less code.\nHeader: R1 stale fill race\nA) Fill-guard requirement (recommended)\nAmend the plan with a required guarantee: a cache fill from readProfile is admitted only if no writeProfile for that key completed (reached cache.delete) after that read's repository.read began; otherwise the fill is dropped and the read still returns its value. State is per wrapper/cache instance, synchronous in the event loop, bounded memory, no key logging; rejected fills are counted on the existing dashboard. Effort S; risk low; reuse: adapter, cohort, controller lifecycle unchanged; verification: deterministic both-order, missing-record creation, overlapping-write, rejected read/write and instance-isolation scenarios in the existing contract harness. ✅ Restores the retained rule with no limit or TTL change. ✅ In-process only; no new adapter or transport. ❌ Adds per-key bookkeeping and a new branch to maintain.\nB) Keep wrapper, rely on TTL\nKeep the proposed ordering and accept up to 30 s (10 s absent) staleness after an overlapping write. Effort S with zero implementation work; risk high; reuse: full; verification: none added for the race. ✅ No new code or state. ✅ Simplest wrapper. ❌ Violates the retained clause that TTL is not a substitute; weakens consistency, which this review cannot authorize.\nC) Double delete around the write\nCall cache.delete before and after repository.write. Effort S; risk high; reuse: full; verification: both-order tests would still fail for a read spanning both deletes. ✅ Two-line change. ✅ Shrinks the race window. ❌ Does not close the window, so the retained rule remains violated.\n\n### D1 answer record\nD1 — R1-FILL-RACE → A) Fill-guard requirement. Authority: user run rules for this capture (plan-author representative, recommended option within retained-contract scope). No human asked; preamble spawned/auto-decide transport not available, so this is recorded as an authorized auto-decision, not an AskUserQuestion answer. Applied: \"Required guarantee\" subsection and scenarios (a)–(f) in the plan body; ledger row R1 approved. Not applied: any implementation choice (marker structure, counter name).\n\n### 0I Temporal interrogation\n```\n HOUR 1 (foundations): wrapper boundary semantics; where \"write completes\" is (after cache.delete); guard scoped per instance\n HOUR 2-3 (core logic): ambiguity: guard update point (at cache.delete, synchronously) and read capture point (before awaiting repository.read); absent DTO fills pass through the same guard\n HOUR 4-5 (integration): surprise: controller instance swap must not carry guard state; adapter bypass path must skip the fill and the counter cleanly\n HOUR 6+ (polish/tests): pause/release harness hooks inside repository.read; counter on the existing dashboard with no key labels\n```\nFeasibility blockers: none remaining after R1. Pending choices (engineering): guard data structure and bounding strategy; rejected-fill counter name. Effort: human ~1 day / CC+gstack ~30 min for wrapper + tests.\n\n## Review Sections (HOLD SCOPE, implementation-ready at wrapper boundary)\n\nCurrent scope: Mode HOLD SCOPE (explicit user instruction). Governing rows: R1-FILL-RACE approved (D1 → A, authorized auto-decision; scope above). Accepted work: proposed wrapper + fill-guard requirement + scenarios (a)–(f). Deferred: none. Rejected: none. Pending (engineering): guard structure, counter name.\n\n### Section 1: Architecture Review — OK, 0 issues\n```\n caller --auth/authz--> readProfile/writeProfile (wrapper, per controller instance)\n | cache.get/set/delete (sync, atomic) | guard: per-key write marker\n v v\n LRU adapter (1000 / 16 MiB / 30 s, absent sentinel 10 s, failure -> bypass)\n |\n repository.read (single-flight cohort) / repository.write (atomic, rollback on reject)\n |\n in-process transactional store (no network)\n rollout controller: flag predicate (shared read/write) -> drain writes -> publish new instance\n```\nBoundaries unchanged; new coupling: wrapper ↔ guard state (in-instance, justified by R1). Scaling: at 10x keys the LRU evicts (bounded 16 MiB); at 100x the hit rate falls and DB load returns to baseline, never above it. SPOF: the one process (pre-existing). Security: no new surface. Production failure: adapter failure → bypass → DB load returns to 70% baseline; alert on bypass >1 min. Rollback: flag disable, immediate, instance swap drains writes. Decision gate: no new choice; R1 cited.\n\n### Section 2: Error & Rescue Map — 5 paths, 0 GAPS\n```\n METHOD/CODEPATH | WHAT CAN GO WRONG | EXCEPTION CLASS (existing)\n ---------------------------|--------------------------------------|---------------------------\n readProfile cache.get | adapter failure | CacheAdapterError -> bypass\n readProfile repository.read| store error / timeout | typed API error (existing mapping)\n readProfile cache.set | adapter failure / byte cap | CacheAdapterError -> bypass; cap = evict\n writeProfile repository.write | rejected (rolled back) | typed API error (existing mapping)\n writeProfile cache.delete | adapter failure | CacheAdapterError -> bypass until reinit\n\n EXCEPTION CLASS | RESCUED? | RESCUE ACTION | USER SEES\n ----------------------|----------|---------------------------------------------|---------------------\n CacheAdapterError | Y | bypass cache until empty reinit; counted | normal latency, correct data\n typed API error (read)| Y | cohort released; no fill; error mapped | existing error response\n typed API error (write)| Y | no commit; guard untouched; cache preserved | existing error response\n```\nNo catch-alls introduced; wrapper adds no new exception classes. Guard update is synchronous in-process and has no failure path beyond a defect (covered by tests (a)–(f)). Decision gate: no new choice.\n\n### Section 3: Security & Threat Model — 0 issues, 0 High\nAttack surface: none added (no endpoint, param, job or file path). Authorization: runs before the wrapper; keys encode tenant + validated profile ID, so a cached value cannot cross tenants (threat: cross-tenant read via key collision — likelihood Low, impact High, mitigated by retained key contract). Secrets/keys never logged; rejected-fill counter has no labels. Dependencies: none new. Data class: profile summaries (existing handling). Decision gate: no new choice.\n\n### Section 4: Data Flow & Interaction Edge Cases — 6 edge cases mapped, 0 unhandled (after R1)\n```\n INPUT key -> (auth upstream) -> cache.get -> miss -> repository.read -> guard check -> cache.set -> OUTPUT\n shadow: nil key: rejected upstream (existing validation) | absent record: absent DTO, sentinel 10 s\n adapter error: bypass | store error: typed error, no fill | stale snapshot: dropped by guard\n```\nAsync ordering (invariant: a read begun after writeProfile fulfills observes the committed version):\n```\n time | readProfile R (miss) | writeProfile W | cache[key] | guard[key]\n t1 | get -> miss; capture guard=g0 | | - | g0\n t2 | await repository.read (v1 snap) | | - | g0\n t3 | | await write -> commit v2 | - | g0\n t4 | | cache.delete; guard=g1 | - | g1\n t5 | | promise fulfills (complete) | - | g1\n t6 | resume; guard g1 != g0 -> DROP | | - | g1\n t7 | fresh read R2 (begun after t5) | | miss -> v2 fill | g1\n Other order: R fills at t2' before t4 -> cache=v1; t4 delete removes it; R2 at t7 sees v2. OK.\n Without the guard (proposed wrapper): t6 would set cache=v1; R2 returns v1 until TTL -> VIOLATION.\n```\n**CRITICAL GAP (resolved): R1-FILL-RACE** — evidence: proposed readProfile/writeProfile ordering plus \"no additional version checks or coordination\" and \"does not inspect cache fills\". Remedy: D1 → A fill-guard requirement (approved). Residual risk: unnecessary drops for reads starting between commit and delete (safe). Verification: scenarios (a)–(f). Interaction edge cases: no user-visible interaction changes (backend only). Decision gate: exact prior answer D1 covers it; applied.\n\n### Section 5: Code Quality Review — 0 issues\nTwo wrapper functions plus guard check fit the existing read-through pattern; no duplication (single-flight stays in repository.read, invalidation stays in the wrapper). Branches per function ≤ 4. Naming to be settled by engineering; no abstraction added. Decision gate: no new choice.\n\n### Section 6: Test Review — diagram produced, 0 gaps\n```\n readProfile: hit | miss+fill | miss+guard-drop (a,c,d) | miss+adapter-bypass | read rejected (e)\n writeProfile: commit+delete (b) | rejected (e) | overlapping W1/W2 (d) | delete adapter failure\n controller: instance swap mid-read (f) | predicate parity | old-writer/new-reader\n limits: eviction, 16 MiB cap, 30 s TTL, 10 s absent sentinel\n```\nUnit: wrapper hit/miss/guard; Integration: contract harness with pause/release for (a)–(f); System: rollout stage gates. Assertions map 1:1 to retained clauses (exact: \"next read observes the committed version\", \"no 10 s absent window\"). 2am test: (a); hostile QA: (c) create-after-absent; chaos: adapter failure mid-fill. Pyramid upright; flakiness: none (deterministic pause points, no wall clock; TTL tests use the harness clock). Load: cold-start within existing DB capacity (plan). Decision gate: coverage carried from D1 and the author's requirements; no new choice.\n\n### Section 7: Performance Review — 0 issues\nMemory: ≤1000 entries / 16 MiB plus guard markers bounded by written keys in the instance (engineering must bound, per R1). No new queries or indexes; no N+1; no pool pressure (fewer DB reads). Slow path: miss + store read, unchanged. WARNING (informational): guard drops lower the hit rate on write-heavy keys; visible via the rejected-fill counter; acceptance targets unchanged. Decision gate: no new choice.\n\n### Section 8: Observability & Debuggability — 0 gaps\nExisting metrics: hit/miss, eviction, bytes, fallback errors, DB CPU, p95; added branch: rejected-fill counter on the current dashboard, no key labels (author policy). Alerts and runbooks exist (SLO breach, p95 >120 ms 5 min, bypass >1 min). Debuggability: counters plus typed errors reconstruct behavior without keys. No new alert or metric project. Decision gate: no new choice.\n\n### Section 9: Deployment & Rollout — 0 new risks\nNo migration. Flag staged 10% → 50% → 100% with one healthy hour each; breach disables immediately. Rollback: disable flag → controller drains writes → bypass instance; seconds. Deploy window: old/new instances isolated by the controller. Post-deploy: first 5 min watch bypass and error rate; first hour the stage gate. Smoke: hit/miss counters moving, zero fallback errors. Decision gate: no new choice.\n\n### Section 10: Long-Term Trajectory — Reversibility 5/5, debt items: 1\nDebt: guard bookkeeping must stay next to cache.delete (document with an ASCII comment). Path dependency: none; repository interface keeps the replacement path. Knowledge: this plan plus scenarios (a)–(f) suffice. Ecosystem fit: in-process, same patterns. Decision gate: no new choice.\n\n### Section 11: Design & UX Review — SKIPPED (no UI scope)\n\n## Closing sequence\nOutside Voice: `codex_reviews: disabled` → Codex review skipped (codex_reviews disabled). Re-enable: `gstack-config set codex_reviews enabled`. No prompt, outside process or native fallback. Disabled-coverage record: not persisted (`gstack-review-log` unavailable; mutating commands forbidden in this run).\nTODO choices: none proposed (HOLD SCOPE; no evidenced deferred gap).\nApproval readiness: PASS — rows checked: R1-FILL-RACE (D1 → A, authorized auto-decision; scope limited to the fill-guard requirement and scenarios (a)–(f)). No declined/deferred items in accepted work.\n\n## Required Outputs\n\n### Review facts\nMode HOLD SCOPE; findings: 1 (R1-FILL-RACE, critical, resolved by approved remedy); unresolved choices: 0; critical gaps remaining: 0; scope proposals: 0; outside coverage: codex disabled. Status: clean.\n\n### NOT in scope\nDeferred: none. Rejected: none. (Declined options B and C of D1 were remedy alternatives, not scope items.) Pre-existing exclusions unchanged: distributed caching, cross-process coherence, prewarming, consistency changes, new product surfaces.\n\n### What already exists\nRepository read-through interface, single-flight cohort and write-retire, atomic write with rollback, typed error mapping, LRU adapter with limits/sentinel/bypass, rollout controller with instance isolation and predicate parity, telemetry dashboards, alerts and runbooks. The plan reuses all of them unchanged; only the wrapper and its tests are new.\n\n### Dream state delta\nAfter this plan: hot reads served in-process at the acceptance targets, consistency rule intact and tested, replacement path preserved. Remaining distance to the 12-month ideal: any shared cache stays out of scope by design.\n\n### Error & Rescue Registry\nSee Section 2 map (5 codepaths, 3 exception classes, all rescued by existing mechanisms; 0 CRITICAL GAPS).\n\n### Failure Modes Registry\n```\n CODEPATH | FAILURE MODE | RESCUED? | TEST? | USER SEES? | LOGGED?\n ----------------------------|-------------------------------|----------|--------------|-------------------|--------\n readProfile cache.get/set | adapter failure | Y bypass | Y (existing) | correct, slower | Y fallback metric\n readProfile repository.read | store error | Y typed | Y (existing) | existing error | Y existing\n readProfile fill | stale fill after write (R1) | Y guard | Y (a,c,d) | committed version | Y rejected-fill count\n writeProfile repository.write| rejected / rollback | Y typed | Y (e) | existing error | Y existing\n writeProfile cache.delete | adapter failure | Y bypass | Y (existing) | correct, slower | Y fallback metric\n controller swap | old read fills new instance | Y isolation | Y (f) | n/a | Y existing\n 6 rows, 0 CRITICAL GAPS\n```\n\n### Diagrams\n1. System architecture: Section 1. 2. Data flow with shadow paths: Section 4. 3. State machine (cache entry):\n```\n [absent] --fill(guard ok)--> [value|sentinel] --TTL 30s/10s or evict--> [absent]\n [value|sentinel] --cache.delete (write complete)--> [absent]\n [absent] --fill(guard changed)--> [absent] (drop; counted)\n invalid: fill after delete with stale snapshot -> prevented by guard; fill into other instance -> prevented by controller\n```\n4. Error flow: Section 2. 5. Deployment sequence:\n```\n deploy wrapper (flag off) -> 10% keys, 1 healthy hour -> 50%, 1 healthy hour -> 100%, absolute targets -> record results\n```\n6. Rollback flowchart:\n```\n breach? -> yes -> disable flag -> controller drains writes -> bypass instance -> runbook records incident -> resume criteria\n -> no -> continue stage\n```\n\n### Stale Diagram Audit\nFiles touched are to be determined; no existing ASCII diagrams known in them. Plan diagrams above are current with the accepted amendment.\n\n## Implementation Tasks\nSynthesized from this review's findings. Each task derives from a specific finding above. Run with Claude Code or Codex; checkbox as you ship.\n\n- [ ] **T1 (P1, human: ~4h / CC: ~15min)** — profile-summary wrapper — Implement readProfile/writeProfile with the write-aware fill guard per \"Required guarantee\"\n - Surfaced by: Section 4 — R1-FILL-RACE (D1 → A)\n - Files: to be determined (wrapper module)\n - Verify: contract harness scenarios (a)–(f) pass; existing repository contract tests unchanged and green\n- [ ] **T2 (P1, human: ~4h / CC: ~10min)** — wrapper tests — Add deterministic pause/release scenarios (a)–(f) plus hit/miss, eviction, byte cap, TTL, bypass, invalidation, preservation, coalescing\n - Surfaced by: Section 6 — Test Review (coverage carried from D1 and author requirements)\n - Files: to be determined (wrapper test file in the existing harness)\n - Verify: each scenario fails against the unguarded wrapper and passes with the guard\n- [ ] **T3 (P2, human: ~1h / CC: ~5min)** — telemetry — Record the rejected-fill branch on the current dashboard with no key labels\n - Surfaced by: Section 8 — Observability (author policy: existing telemetry only)\n - Files: to be determined\n - Verify: counter increments in scenario (a); no key labels present\n_No new tasks from Sections 1, 2, 3, 5, 7, 9, 10, 11._\nTask JSONL: not persisted (gstack state root unavailable; no mutating commands in this run). Ratios assumed: bug fix with regression ~20x, tests ~50x.\n\n### Completion Summary\n```\n +====================================================================+\n | MEGA PLAN REVIEW — COMPLETION SUMMARY |\n +====================================================================+\n | Mode selected | HOLD SCOPE |\n | System Audit | skipped per run rules; fixture contracts |\n | Step 0 | HOLD SCOPE; R1-FILL-RACE approved (D1 → A) |\n | Section 1 (Arch) | 0 issues found |\n | Section 2 (Errors) | 5 error paths mapped, 0 GAPS |\n | Section 3 (Security)| 0 issues found, 0 High severity |\n | Section 4 (Data/UX) | 6 edge cases mapped, 0 unhandled |\n | Section 5 (Quality) | 0 issues found |\n | Section 6 (Tests) | Diagram produced, 0 gaps |\n | Section 7 (Perf) | 0 issues found |\n | Section 8 (Observ) | 0 gaps found |\n | Section 9 (Deploy) | 0 risks flagged |\n | Section 10 (Future) | Reversibility: 5/5, debt items: 1 |\n | Section 11 (Design) | SKIPPED (no UI scope) |\n +--------------------------------------------------------------------+\n | NOT in scope | written (0 items) |\n | What already exists | written |\n | Dream state delta | written |\n | Error/rescue registry| 5 rows, 0 CRITICAL GAPS |\n | Failure modes | 6 total, 0 CRITICAL GAPS |\n | TODOS.md updates | 0 items proposed |\n | Scope proposals | 0 proposed, 0 accepted (EXP + SEL) |\n | CEO plan | skipped by mode |\n | Outside voice | codex + disabled |\n | Lake Score | 1/1 recommendations chose complete option |\n | Diagrams produced | 6 (arch, data flow, state, error, deploy, rollback) |\n | Stale diagrams found | 0 |\n | Unresolved decisions | 0 (listed below) |\n +====================================================================+\n```\n\n### Unresolved Decisions\nNone. D1 was resolved by an authorized auto-decision (run rules); no question went unanswered.\n\nReview Log / decision log / dashboard: not persisted (gstack bins unavailable; mutating commands forbidden). This run is unlogged. Learnings: No durable learnings this session.\n\n## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 1 (not persisted) | CLEAR | mode: HOLD SCOPE, 0 critical gaps (1 found, resolved by R1-FILL-RACE) |\n| Outside Review | codex (disabled) via `/plan-ceo-review` | Independent 2nd opinion | 0 | disabled | codex_reviews disabled; no completed external review |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | — | — |\n| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | — |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | — |\n\n- **OUTSIDE COVERAGE:** provider codex, phase plan-review, disabled by config (codex_reviews: disabled); no outside process, no native fallback; no completed external review. Disabled record not persisted.\n- **VERDICT:** CEO CLEARED — eng review required\n\nNO UNRESOLVED DECISIONS\n" + } +} diff --git a/test/fixtures/detector-corpora/plan-ceo-section-loading/37193478719-t1-fail.json b/test/fixtures/detector-corpora/plan-ceo-section-loading/37193478719-t1-fail.json new file mode 100644 index 000000000..af2d4e572 --- /dev/null +++ b/test/fixtures/detector-corpora/plan-ceo-section-loading/37193478719-t1-fail.json @@ -0,0 +1,20 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-ceo-section-loading", + "detector": "test/helpers/ceo-stale-fill-decision.ts#hasApprovedStaleFillDecision || test/helpers/ceo-section-loading-fixture.ts#hasStaleFillRaceFinding", + "series_identity": "2986275878c588ef", + "source": { + "run": "37193478719", + "sha": "9f357d9b9c75441b6a18e4f75174274954ba6f96", + "lane": "periodic", + "artifact": "paid-slice-6-a1", + "path": "shards/skill-e2e-plan-ceo-review-section-loading/_partial-e2e.json#/tests/0/output" + }, + "census_outcome": "failed", + "census_error": "Error: expect(received).toBe(expected)", + "expected": "pass", + "justification": "Fixed false red: at 9f357d9 the ledger reader accepted only R row IDs and this report recorded the approved decision as CEO-F1 (phrase fallback also false); 4128700 accepts any ledger row ID, so the approved CEO-F1 decision now reads.", + "input": { + "report": "# Plan: cache profile summaries in one process\n\n## Measured problem and accepted scope\nThe existing profile-summary service has one active process. A one-week trace\nshows repeated reads of about 900 hot keys: DB CPU is 70%, with read p95 120 ms.\nAdd a process-local LRU wrapper to the existing repository. Acceptance targets\nare at least 60% cache hits, DB CPU below 50%, and read p95 below 60 ms, with the\nexisting error-rate and correctness SLOs unchanged. This is an internal backend\nchange with no UI, API, schema, pricing, or developer onboarding change.\n\n## Existing contracts retained\n- All reads and writes use this repository in the same process; there are no\n external DB writers. Multi-process operation remains unsupported and startup\n rejects that configuration while caching is enabled.\n- These surrounding contracts are accepted fixture facts, supplied by the\n existing repository, cache adapter and rollout controller. Preserve them;\n review the new wrapper ordering below against them.\n- Authentication and authorization run before repository access. Keys encode\n the authenticated tenant ID and validated profile ID without ambiguity.\n Values are immutable profile-summary DTOs; secrets and cache keys are never\n logged. Cached results cannot bypass authorization.\n- The existing LRU adapter supports 1000 entries, a 16 MiB byte cap, and a\n 30-second TTL. Recorded hot data fits those limits. repository.read returns\n an immutable absent-result DTO for a missing record, never undefined. The\n adapter recognizes that DTO in cache.set, stores an internal sentinel with a\n 10-second TTL, and cache.get decodes it back to the same absent-result DTO.\n The internal sentinel cannot escape the adapter; undefined means a cache miss.\n- Cache operations are synchronous and atomic in the single JS event loop.\n On any cache failure the existing adapter bypasses the cache until an empty\n cache is reinitialized; repository errors keep the current typed API error\n mapping. The existing per-key single-flight wrapper sits inside\n repository.read, coalesces simultaneous store reads and releases on failure.\n A committed repository.write retires that key's old read cohort before its\n promise resolves. A later repository.read starts a fresh cohort; a rejected\n write leaves the cohort unchanged. Already-started readers may finish with\n their earlier snapshot. This admission rule does not inspect cache fills.\n- The repository uses an in-process transactional store, with no network\n transport between this wrapper and the store. repository.write is atomic:\n a resolved promise means committed, and every\n rejected promise guarantees no commit; its transaction rolled back before\n rejection. Existing contract tests exercise that guarantee.\n- Consistency is measured at the public wrapper boundary. A write completes\n when writeProfile's promise fulfills after cache.delete, not when\n repository.write commits or resolves. A read begins when readProfile is\n invoked. Reads that overlap an unfinished writeProfile may return an earlier\n snapshot, including reads begun after the store commit but before the wrapper\n promise fulfills. Every read begun after that write completes must\n observe the committed version. TTL expiry is not a substitute for this rule.\n\n## Proposed wrapper integration\nKeep the current read-through repository interface and shared adapters. These\nare the new read/write ordering rules as originally proposed. **Amended by CEO review\nD1 (CEO-F1):** the original text proposed no coordination between a cache fill and a\nwrite; that contradicts the retained consistency rule (see CEO-F1 below). Required\nguarantee added: `readProfile` must skip its `cache.set` when a `writeProfile` for the\nsame key completed `cache.delete` after that read began (a per-instance write-completion\ntoken checked synchronously around the fill; representation chosen by engineering). The\nsketch below is the pre-amendment baseline, retained for review traceability:\n\n```javascript\nasync function readProfile(key) {\n const cached = cache.get(key);\n if (cached !== undefined) return cached;\n const value = await repository.read(key);\n cache.set(key, value);\n return value;\n}\n\nasync function writeProfile(key, update) {\n const saved = await repository.write(key, update);\n cache.delete(key);\n return saved;\n}\n```\n\n## Verification and rollout\nExisting repository contract tests cover tenant isolation, key validation,\nabsence, DB failures, authorization, and startup rejection of multi-process\noperation while caching is enabled. New wrapper tests cover hit/miss,\neviction and byte limits, TTL, adapter-failure fallback, successful-write\ninvalidation, failed-write preservation, and concurrent-miss coalescing.\n**Added by CEO review D1 (CEO-F1), deterministic schedules on the existing harness:**\n(a) read pauses after its store snapshot, write commits + deletes + fulfills, read resumes:\nfill must be skipped and a fresh read must observe the committed version; (b) read fills\nbefore the write completes: delete must remove it and a fresh read observes the committed\nversion; (c) same as (a) for a missing record created by the write: absent sentinel must\nnot be installed after creation; (d) rejected write leaves the cache and token unchanged;\n(e) two overlapping writes: a fresh read after both complete observes the last commit.\nThe rollout uses the existing runtime feature flag: enable for 10% of keys,\nthen 50%, then all keys after one healthy hour at each stage. Monitor hit/miss,\neviction, cache bytes, fallback errors, DB CPU, and read p95 without raw IDs.\nThe existing controller uses one shared key-selection predicate for reads and\nwrites. On any enable, disable or percentage change, it stops admitting work,\nawaits every admitted old-instance write, then publishes a new wrapper/cache\ninstance with a fresh single-flight cohort before admitting new work. Old reads\nretain their old instance and cannot fill the new one. Disabled instances\nbypass the cache on both paths. Tests cover the old-writer/new-reader ordering,\nall those transitions and predicate parity. This lifecycle isolation does not coordinate\nan ordinary DB write with a cache fill in the same active instance.\nExisting dashboards and runbooks cover these metrics. Before each stage, verify\nthat alerts page the service owner on any correctness/error-SLO breach, read\np95 above 120 ms for five minutes, or cache bypass persisting for one minute.\nHit rate is hits / (hits + misses) among requests admitted to the cache path;\nflag-excluded or adapter-bypassed requests are tracked separately, not as misses.\nDB CPU and read p95 are service-wide metrics, including bypassed requests.\nAt the 10% and 50% stages, a healthy hour requires at least 60% admitted-request\nhits, unchanged correctness/error SLOs, no alerts, and aggregate DB CPU/read p95\nno worse than their 70%/120 ms pre-rollout baselines. At 100%, the original\nabsolute acceptance targets (DB CPU below 50%, read p95 below 60 ms, hits at\nleast 60%) must all hold with unchanged correctness/error SLOs and no alerts.\nAny breach disables the flag immediately; the runbook records the incident,\nrollback and criteria for resuming. These are existing\nrollout-controller and telemetry contracts, not proposed wrapper additions.\nCold starts remain within the existing DB capacity. The service owner monitors\nthe rollout and records the results against the acceptance targets.\n\n## Out of scope\nDistributed caching, cross-process coherence, prewarming, changing consistency\nsemantics, or adding new product surfaces. The repository interface preserves a\nfuture replacement path without introducing a general cache framework now.\n\n## Author's review and acceptance requirements\nThis is a full CEO scope and feasibility review. The author has approved the\nretained contracts, limits, rollout and acceptance targets above. Evaluate the\nproposed wrapper against them; the wrapper itself remains unapproved. An actual\ncontradiction or missing proof must be reported and resolved, not assumed away.\nFor a demonstrated gap, amend the plan with the required guarantee, a feasible\nremedy, its tradeoffs and deterministic regression scenarios. Those repairs\nand their required verification are within the requested scope. The exact data\nstructures, full function bodies and executable test code belong to subsequent\nengineering planning; do not select or implement them during this review when\nthe required behavior and feasibility can already be established.\n\nUse the existing deterministic repository-contract test harness. Required wrapper\nacceptance includes both completion orders of overlapping reads and writes,\nmissing-record creation, rejected reads/writes, overlapping writes, and isolation\nacross controller instance changes. Use existing telemetry to record any added\nbranch on the current dashboard with no key labels; no new alert threshold or metric\nproject is requested. These are future acceptance requirements, not tests already\nimplemented or passing. Preserve all 11 review outcomes, required registries,\ndiagrams, tasks, completion summary and the full GSTACK REVIEW REPORT.\n\n---\n\n# CEO Review (plan-ceo-review, 2026-10-04, HOLD SCOPE)\n\nRun notes: automated fixture run, no human present. Preamble/system audit, environment\nsetup, telemetry, design-doc offer and learnings-search skipped per run rules. Outside\nreview: `codex_reviews: disabled` in `.gstack-section-state-Zns2Y9/config.yaml`, so the\noutside voice step and its native fallback are skipped and recorded as disabled coverage.\nBase branch: main (git read-only snapshot; no git commands run).\n\n## Step 0\n\n**Review depth:** implementation-ready for wrapper ordering, rescue and test behavior.\nAuthor policy limits: data structures, function bodies and executable test code stay\nwith engineering planning; this review records required guarantees and feasibility.\n**Storage policy:** working plan = this file (requested output); scoped Edits at checkpoints.\n**Stated limits (kept):** LRU 1000 entries, 16 MiB, 30 s TTL, 10 s absent sentinel;\ntargets hits >= 60%, DB CPU < 50%, read p95 < 60 ms; rollout 10% -> 50% -> 100%,\none healthy hour per stage. Deliverables: 1 wrapper module edit + 1 wrapper test file\n(estimate: 2 changed files, excluding unchanged reused adapter/controller/repository).\n\n### Decision ledger\n\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| CEO-F1 (Section 1 / Section 4; owner: wrapper engineering) | Consistency rule, PLAN.md \"Existing contracts retained\" last bullet: every read begun after writeProfile completes must observe the committed version; TTL is not a substitute. Single-flight bullet: already-started readers may finish with their earlier snapshot; admission rule does not inspect cache fills. Proposed wrapper (PLAN.md readProfile/writeProfile): fill after `await repository.read` with no guard; plan states no fill/write coordination. | Proposed wrapper as written: unguarded `cache.set` after read; `cache.delete` after commit. | A) fill-after-write guard required in wrapper; B) keep wrapper, accept TTL-bounded staleness; C) move fill coordination into repository single-flight cohort. | approved | D1 answered A (auto-decision authorized by the run's author policy: \"Authorize complete remedies and required verification that restore its retained contracts... use the recommended option\"). Scope: wrapper must skip a cache fill when a writeProfile for that key completed cache.delete after the read began; deterministic regression schedules for both completion orders and missing-record creation; one added branch recorded on the existing dashboard with no key labels. Token mechanism left to engineering. B declined (weakens retained contract). C declined (changes accepted fixture contract). |\n\n### 0A. Premise Challenge\nReal problem: ~900 hot keys re-read from the DB; DB CPU 70%, read p95 120 ms (one-week\ntrace). Target: hits >= 60%, DB CPU < 50%, p95 < 60 ms, SLOs unchanged. Do-nothing cost:\nDB saturation headroom shrinks; latency stays 2x the target. The plan attacks the pain\ndirectly (repeat reads of a small hot set in one process), not a proxy. OK.\n\n### 0B. Existing Code Leverage\nRepository (single-flight, typed errors, absent DTO), LRU adapter (limits, sentinel,\nbypass-on-failure), rollout controller (instance swap, predicate parity), telemetry and\nrunbooks are all reused unchanged. Only the thin wrapper and its tests are new. No rebuild.\n\n### 0C. Dream State Mapping\n```\n CURRENT STATE THIS PLAN 12-MONTH IDEAL\n Every hot read hits the ---> process-local LRU in ---> repository interface\n DB; CPU 70%, p95 120 ms front of repository; still the seam; swap\n one process, no cache hits >= 60%, p95 < 60 ms adapter if multi-process\n ever becomes supported\n```\nMoves toward the ideal without committing to a distributed cache. OK.\n\n### 0D\nNo approach choice was needed in Step 0: the requested plan (A, current) is the only\napproach consistent with the accepted contracts. Section findings use 0D below.\n\n### 0E. Mode Selection\nExplicit user instruction: HOLD SCOPE. Mode: HOLD SCOPE; approved decisions: none.\nApplication: preserve scope, maximum rigor on failures, ordering, tests, observability.\nProvenance: explicit user choice; no question asked, no question log.\nNo new approach decision was needed.\n\n### 0G. HOLD SCOPE checks\n1. Complexity: ~2 changed files, 0 new classes/services. No challenge needed.\n2. Minimum change: wrapper + tests is already minimal. Nothing deferrable without\n blocking the acceptance targets. No deferral questions.\n3. Invariants and acceptance criteria retained; repairs needed to meet them (CEO-F1) are in scope.\n\n### 0I. Temporal Interrogation\n```\n HOUR 1 (foundations): Adapter/sentinel semantics; undefined = miss; absent DTO round-trip.\n HOUR 2-3 (core logic): Fill-after-write race (CEO-F1); where the guard token lives.\n HOUR 4-5 (integration): Controller instance swap; flag predicate parity; bypass paths.\n HOUR 6+ (polish/tests): Deterministic pause/release schedules; fake clock for TTL.\n```\nFeasibility blocker: CEO-F1 (resolved below). Pending choices left to engineering:\nguard token representation, TTL clock injection. Effort: human ~2 days / CC ~30 min.\n\n## Section 1: Architecture Review\n\n**Current scope:** Mode HOLD SCOPE (explicit user instruction). Governing rows: CEO-F1\npending. Accepted: wrapper + tests + existing rollout. Deferred/rejected: none.\n\nDependency graph (new component marked *):\n```\n caller -> auth/authz -> [readProfile/writeProfile]* -> LRU adapter (existing)\n | \n +-> repository (single-flight, typed errors) -> in-process store\n rollout controller (existing) --publishes/retires--> wrapper+cache instance*\n```\nData paths: happy (miss -> store -> fill -> return), nil (absent DTO -> sentinel 10 s ->\ndecoded DTO), empty (n/a: DTOs immutable, no empty payloads), error (read rejects -> no\nfill -> typed error; write rejects -> no delete -> cache retains committed value, correct\nbecause rollback is guaranteed). Scaling: 10x load is served from cache; 100x saturates\nthe single process before the DB. SPOF: the one process (pre-existing). Security: no new\nsurface. Rollback: flag off -> controller swaps to bypass instance; seconds.\n\n**CRITICAL GAP (CEO-F1) — stale fill after a completed write.** A read that starts before\na write can resolve its store snapshot after `writeProfile` has already run `cache.delete`\nand fulfilled. Its `cache.set` then installs the pre-write value (or the absent sentinel\nfor a just-created record). Every later read hits that stale entry for up to 30 s (10 s\nfor the sentinel), violating the retained rule that every read begun after the write\ncompletes observes the committed version. The retained single-flight rule explicitly lets\nalready-started readers finish with their earlier snapshot and does not inspect fills; the\ncontroller isolation covers instance swaps only. Nothing in the plan prevents this schedule.\nSchedule proof: Section 4.\n\n## currentDecision (CEO-F1)\nCommitment comparison:\n```text\nCommitment | Source/approval or pending | Current | A | B | C\nRead-after-write visibility at wrapper boundary | retained contract (PLAN.md consistency bullet) | violated by stale fill | kept | weakened to TTL bound | kept\nWrapper read/write ordering | proposed, unapproved | unguarded fill | fill guarded by write completion | unchanged | unchanged in wrapper\nrepository single-flight contract | accepted fixture fact | does not inspect fills | unchanged | unchanged | changed (must inspect fills)\nAdapter limits / TTL / sentinel | accepted fixture fact | 1000 / 16 MiB / 30 s / 10 s | unchanged | unchanged | unchanged\nRegression tests: both completion orders, missing-record creation | author acceptance requirement | required, not implemented | added as deterministic schedules | would need to expect stale reads (not allowed) | added at repository level |\nAdded branch on dashboard, no key labels | author permits existing telemetry | none | one \"fill suppressed\" branch | none | none |\nCompleteness: A=10/10, B=3/10, C=7/10\n```\n\nQuestion: D1 — CEO-F1: How does the wrapper prevent a read begun before a write from filling the cache after that write completes?\nProject: profile-summary service, branch main, process-local LRU wrapper plan.\nELI10: A slow read can fetch the old value, then a write finishes and clears the cache, then the slow read puts the old value back. Everyone sees stale data for 30 seconds. The plan promises that never happens after a write completes.\nStakes if we pick wrong: correctness SLO breach and stale profiles after every write that overlaps a read; the rollout would be rolled back on alerts.\nRecommendation: A because it keeps the retained consistency contract with the smallest clear diff (explicit over clever) and leaves the token mechanism to engineering.\nCompleteness: A=10/10, B=3/10, C=7/10\nPros / cons:\nA) Require a fill-after-write guard in the wrapper (recommended)\n ✅ Preserves the retained read-after-write rule; stale fill becomes impossible by construction\n ✅ Local to the wrapper; adapter, repository and controller contracts stay untouched\n ❌ Adds one small piece of per-instance state and one extra branch to test and chart\nB) Keep the wrapper as proposed; accept staleness bounded by TTL\n ✅ Zero implementation work beyond the proposed wrapper (effort S, nothing new to build)\n ✅ Simplest code; no extra state in the wrapper\n ❌ Weakens a retained contract; TTL is explicitly not a substitute, so this cannot be authorized\nC) Move fill coordination into repository single-flight cohort retirement\n ✅ Centralizes ordering in one place for all future callers of the repository\n ✅ Guard would cover any other caller that fills caches from repository reads\n ❌ Changes an accepted fixture contract (admission rule does not inspect fills); wider, riskier diff\nNet: A fixes the demonstrated gap inside the new code; B breaks the promise; C fixes it by rewriting an accepted contract.\nHeader: Stale fill guard\nA) Guard wrapper fill (recommended)\nRequire: a read's cache.set is skipped when a writeProfile for that key completed cache.delete after the read began. Effort S, risk low, reuses adapter and harness; verification: both completion orders and missing-record creation as deterministic schedules. ✅ keeps retained contract ✅ wrapper-local change ❌ one more branch and token state.\nB) Keep wrapper, TTL bound\nKeep the proposed ordering and rely on 30 s / 10 s TTL to age out stale fills. Effort S, zero implementation work, risk high; verification would have to assert stale reads. ✅ nothing to build ✅ simplest code ❌ violates retained consistency rule, not authorizable.\nC) Coordinate in repository\nMake repository single-flight retirement reject fills from retired cohorts. Effort M, risk medium, reuses single-flight; verification at repository contract level. ✅ one central guard ✅ covers other callers ❌ rewrites an accepted fixture contract and widens the diff.\n\n**Answer D1: A (authorized auto-decision, see ledger row CEO-F1).** Plan amended in\n\"Proposed wrapper integration\" and \"Verification and rollout\". Section 1 outcome: 1 issue (CEO-F1, remedied).\n\n## Section 2: Error & Rescue Map\n```\n METHOD/CODEPATH | WHAT CAN GO WRONG | EXCEPTION CLASS\n -------------------------|--------------------------------------|------------------\n readProfile#cache.get | adapter failure | adapter-internal (bypass)\n readProfile#repo.read | store failure / timeout | typed repository error\n | invalid key / unauthorized | typed validation/auth error\n readProfile#cache.set | adapter failure, byte cap, stale fill| adapter-internal / CEO-F1 guard\n writeProfile#repo.write | rejected (rolled back) | typed repository error\n writeProfile#cache.delete| adapter failure after commit | adapter-internal (bypass)\n\n EXCEPTION CLASS | RESCUED? | RESCUE ACTION | USER SEES\n -------------------------|----------|---------------------------------------|---------------\n adapter-internal | Y | adapter bypasses cache until reinit; | transparent;\n | | fallback error metric + 1-min alert | slower reads\n typed repository error | Y (exist)| existing API error mapping; no fill; | existing typed\n | | single-flight released | API error\n stale fill (CEO-F1) | Y (D1) | fill skipped; \"fill suppressed\" branch| fresh value\n```\nNo catch-alls. **WARNING (W1, verification only):** a `cache.delete` adapter failure after\na committed write must not be reported as a write failure; the retained adapter contract\nsays failures bypass internally, so this is a determined check within the accepted\n\"adapter-failure fallback\" coverage (write path), not a new choice. Outcome: 6 paths mapped, 0 open GAPS.\n\n## Section 3: Security & Threat Model\nNo new endpoints, params, jobs or dependencies. Keys carry authenticated tenant + validated\nprofile ID (retained), so cross-tenant IDOR via cache is prevented (likelihood Low, impact\nHigh, mitigated). Values are immutable DTOs; secrets and keys never logged (retained).\nStale-fill (CEO-F1) is correctness, not confidentiality. Flag predicate shared by reads\nand writes prevents write-path/read-path asymmetry. Outcome: No issues found, 0 High.\n\n## Section 4: Data Flow & Interaction Edge Cases\n```\n INPUT(key) -> VALIDATION(auth/key, existing) -> cache.get -> [hit] OUTPUT\n \\-> [miss] repo.read -> guard -> cache.set -> OUTPUT\n shadow: absent -> absent DTO -> sentinel(10 s) -> decoded DTO tested (accepted)\n shadow: read rejects -> no fill -> typed error tested (accepted)\n shadow: adapter failure -> bypass -> store read tested (accepted)\n shadow: byte cap / eviction -> entry dropped -> next read misses tested (accepted)\n```\nAsync ordering (invariant: a read begun after writeProfile fulfills observes the committed\nversion; boundary = readProfile call vs writeProfile fulfillment):\n```\n t | readProfile R1 (begun before W) | writeProfile W | cache[key] | store\n 1 | get -> undefined (miss) | | - | v0\n 2 | await repo.read (snapshot v0) | | - | v0\n 3 | | await repo.write | - | v1 committed\n 4 | | cache.delete; fulfill | - | v1 <- W complete\n 5 | resume: cache.set(key, v0) | | v0 STALE | v1\n 6 | R2 begins: get -> v0 | | v0 | v1 <- VIOLATION\n```\nReverse order (R1 fills at t3 before W deletes at t4) is safe: delete removes v0 and R2\nmisses to v1. Only the schedule above violates; single-thread execution does not exclude\nit because the violation spans the `await`. Mechanism after D1: at t5 the guard sees a\nwrite completion for the key after R1 began and skips the fill; R2 misses to v1. Same\nproof covers the absent sentinel for missing-record creation. Regression proof: schedules\n(a)-(e) in \"Verification and rollout\". Overlapping writes W1/W2: both delete; store holds\nlast commit; fresh read observes it. Interaction edge cases: no UI; double-submit handled\nby store atomicity. Outcome: 6 edge cases mapped, 0 unhandled after D1.\n\n## Section 5: Code Quality Review\nWrapper fits the existing read-through pattern; no duplication (adapter owns sentinel/TTL,\nrepository owns single-flight). Naming fine. The D1 guard adds one branch (<= 3 branches\nper function). No over-engineering; keep the token per wrapper instance so the controller\nswap discards it. Outcome: No issues found.\n\n## Section 6: Test Review\n```\n NEW THING | TYPE | HAPPY | FAILURE | EDGE\n readProfile hit/miss/fill | unit (harness)| miss->fill->hit | repo.read rejects | absent sentinel\n writeProfile commit+delete | unit | commit->delete | write rejects, no delete| overlapping writes\n fill-after-write guard (D1) | unit sched. | schedule (b) | schedule (a),(c) | (d),(e)\n adapter bypass | unit | n/a | get/set/delete failures | bypass persists alert\n controller instance swap | integration (existing)| swap | old writer/new reader | predicate parity\n```\nAssertion check: each accepted requirement maps to an observable assertion (committed\nversion visible to a read begun after fulfillment; no sentinel after creation; cache\nunchanged after rejected write). 2 am test: schedule (a). Hostile QA: schedule (c) plus\neviction mid-schedule. Chaos: adapter failure injected at t5. Pyramid: unit-heavy, correct.\nFlakiness: TTL tests must use the harness's controlled clock (engineering detail, owner:\nwrapper engineering; not a new decision). Outcome: diagram produced, 0 gaps after D1.\n\n## Section 7: Performance Review\n900 hot keys fit 1000 entries / 16 MiB (retained). Guard is O(1) synchronous per op; its\nstate must stay bounded (per-instance; engineering chooses per-key map with cleanup or a\nprocess-wide generation; either satisfies D1). Slowest new path: miss + store read, same as\ntoday. No new connections. Outcome: No issues found.\n\n## Section 8: Observability & Debuggability Review\nExisting metrics cover hit/miss/eviction/bytes/fallback/DB CPU/p95 without raw IDs. D1 adds\none \"fill suppressed\" counter on the current dashboard, no key labels (author-permitted).\nAlerts and runbooks are existing contracts. Outcome: 0 gaps.\n\n## Section 9: Deployment & Rollout Review\nNo migration. Flag-staged 10/50/100 with instance swap; rollback = disable flag (seconds).\nRisk window: old-instance reads cannot fill the new instance (retained). D1 token lives in\nthe instance, so swaps reset it safely. Smoke: hit rate, fallback errors, suppressed-fill\ncount visible within first 5 minutes. Outcome: 0 new risks beyond retained contracts.\n\n## Section 10: Long-Term Trajectory Review\nDebt: 1 small item (guard token state). Path dependency: none; repository seam preserved.\nReversibility 5/5. Ecosystem fit: yes. 1-year clarity: add an ASCII comment of the\nschedule above next to the guard. Outcome: Reversibility 5/5, debt items 1.\n\n## Section 11: Design & UX Review\nSKIPPED (no UI scope).\n\n## Closing sequence\n**Outside Voice:** Codex review skipped (codex_reviews disabled). Re-enable:\n`gstack-config set codex_reviews enabled`. No native fallback (disabled means no extra\nstep). Disabled-coverage review-log write not persisted: gstack binaries are not installed\nin this fixture and mutating commands are disallowed.\n**TODO choices:** none proposed (HOLD SCOPE; no evidenced deferred gap). TODOS.md unchanged.\n**Approval readiness: PASS** — CEO-F1: D1 answered A (authorized auto-decision, scope as\nrecorded in the ledger). No other rows. Declined B/C kept out of accepted work.\n\n## NOT in scope\nDeferred: none. Rejected: D1-B (TTL-bounded staleness; violates retained contract),\nD1-C (repository-level fill coordination; changes an accepted fixture contract).\nOriginal out-of-scope list retained above.\n\n## What already exists\nRepository single-flight + typed errors + absent DTO (reused); LRU adapter with limits,\nsentinel and bypass (reused); rollout controller instance swap (reused); dashboards,\nalerts, runbooks (reused). Nothing rebuilt.\n\n## Dream state delta\nAfter this plan: hot reads served in-process at target latency; repository seam intact\nfor a future adapter swap. Remaining distance to the 12-month ideal: multi-process\ncoherence stays unsupported by design.\n\n## Error & Rescue Registry\nSee Section 2 table: 6 method/codepath rows, 3 exception classes, all rescued, 0 CRITICAL GAPS.\n\n## Failure Modes Registry\n```\n CODEPATH | FAILURE MODE | RESCUED? | TEST? | USER SEES? | LOGGED?\n -------------------------|-------------------------|----------|-------|-----------------|--------\n readProfile#cache.get | adapter failure | Y | Y | slower read | Y (fallback metric)\n readProfile#repo.read | store failure | Y | Y | typed error | Y (existing)\n readProfile#cache.set | stale fill after write | Y (D1) | Y (a,c)| fresh value | Y (suppressed counter)\n readProfile#cache.set | byte cap / eviction | Y | Y | miss next time | Y (eviction metric)\n writeProfile#repo.write | rejected / rolled back | Y | Y (d) | typed error | Y (existing)\n writeProfile#cache.delete| adapter failure | Y | Y (W1)| slower reads | Y (fallback metric)\n```\n6 total, 0 CRITICAL GAPS.\n\n## Diagrams\n1. System architecture: Section 1. 2. Data flow with shadow paths: Section 4.\n3. State machine (cache entry):\n```\n [absent] --fill(value)--> [cached v] --delete/TTL/evict--> [absent]\n [absent] --fill(absent DTO)--> [sentinel 10 s] --delete/TTL--> [absent]\n [any] --adapter failure--> [bypassed] --reinit--> [absent]\n invalid: fill after write completion -> prevented by D1 guard\n```\n4. Error flow: Section 2. 5. Deployment sequence: flag 10% -> healthy hour -> 50% ->\nhealthy hour -> 100% -> acceptance check (Section 9). 6. Rollback flowchart:\n```\n alert/breach --> disable flag --> controller drains writes --> bypass instance --> incident + runbook\n```\nStale diagram audit: no ASCII diagrams exist in touched files; 0 stale.\n\n## Implementation Tasks\nSynthesized from this review's findings. Each task derives from a specific\nfinding above. Run with Claude Code or Codex; checkbox as you ship.\n\n- [ ] **T1 (P1, human: ~1 day / CC: ~20min)** — profile-summary wrapper — Add the fill-after-write guard required by D1 (CEO-F1) with the \"fill suppressed\" dashboard branch\n - Surfaced by: Section 1 / Section 4 — CEO-F1 stale fill after a completed write\n - Files: to be determined (wrapper module)\n - Verify: deterministic schedules (a)-(e) pass on the existing repository-contract harness\n- [ ] **T2 (P1, human: ~1 day / CC: ~15min)** — wrapper tests — Implement schedules (a)-(e) plus W1 write-path adapter-failure check and TTL tests on a controlled clock\n - Surfaced by: Section 2 (W1), Section 4, Section 6\n - Files: to be determined (wrapper test file)\n - Verify: harness run green; each schedule fails when the guard is removed\n_No new tasks from Sections 3, 5, 7, 8, 9, 10, 11._\nTask JSONL: not persisted (gstack state root unavailable in this fixture).\n\n## Completion Summary\n```\n +====================================================================+\n | MEGA PLAN REVIEW — COMPLETION SUMMARY |\n +====================================================================+\n | Mode selected | HOLD SCOPE |\n | System Audit | skipped per run rules; fixture facts used |\n | Step 0 | HOLD SCOPE (explicit); D1 CEO-F1 -> A |\n | Section 1 (Arch) | 1 issue found (CEO-F1, remedied) |\n | Section 2 (Errors) | 6 error paths mapped, 0 GAPS |\n | Section 3 (Security)| 0 issues found, 0 High severity |\n | Section 4 (Data/UX) | 6 edge cases mapped, 0 unhandled |\n | Section 5 (Quality) | 0 issues found |\n | Section 6 (Tests) | Diagram produced, 0 gaps |\n | Section 7 (Perf) | 0 issues found |\n | Section 8 (Observ) | 0 gaps found |\n | Section 9 (Deploy) | 0 risks flagged |\n | Section 10 (Future) | Reversibility: 5/5, debt items: 1 |\n | Section 11 (Design) | SKIPPED (no UI scope) |\n +--------------------------------------------------------------------+\n | NOT in scope | written (2 rejected, 0 deferred) |\n | What already exists | written |\n | Dream state delta | written |\n | Error/rescue registry| 6 rows, 0 CRITICAL GAPS |\n | Failure modes | 6 total, 0 CRITICAL GAPS |\n | TODOS.md updates | 0 items proposed |\n | Scope proposals | 0 proposed, 0 accepted (HOLD SCOPE) |\n | CEO plan | skipped by mode |\n | Outside voice | codex: disabled (no fallback) |\n | Lake Score | 1/1 recommendations chose complete option |\n | Diagrams produced | 6 (arch, data flow, state, error, deploy, |\n | | rollback) |\n | Stale diagrams found | 0 |\n | Unresolved decisions | 0 |\n +====================================================================+\n```\nReview Log / dashboard: not persisted (gstack-review-log and gstack-review-read are not\ninstalled in this fixture; mutating commands disallowed). Prior review history: unknown.\nNo durable learnings this session.\n\n## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 1 (not persisted) | CLEAR | mode: HOLD SCOPE, 0 critical gaps open (1 found, remedied by D1) |\n| Outside Review | codex (disabled) | Independent 2nd opinion | 0 | DISABLED | codex_reviews disabled; no completed external review |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | — | not run |\n| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | not run (no UI scope) |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | not run |\n\n- **OUTSIDE COVERAGE:** provider codex, phase plan-review, disabled by config; no outside process started, no native fallback; 0 completed external reviews, no findings implied.\n- **VERDICT:** CEO CLEARED — eng review required\n\nNO UNRESOLVED DECISIONS\n" + } +} diff --git a/test/fixtures/detector-corpora/plan-ceo-section-loading/37195203538-t1-pass.json b/test/fixtures/detector-corpora/plan-ceo-section-loading/37195203538-t1-pass.json new file mode 100644 index 000000000..cdc70b711 --- /dev/null +++ b/test/fixtures/detector-corpora/plan-ceo-section-loading/37195203538-t1-pass.json @@ -0,0 +1,20 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-ceo-section-loading", + "detector": "test/helpers/ceo-stale-fill-decision.ts#hasApprovedStaleFillDecision || test/helpers/ceo-section-loading-fixture.ts#hasStaleFillRaceFinding", + "series_identity": "68761d79db01290b", + "source": { + "run": "37195203538", + "sha": "41287009bd6b524244edced7770c87fc3103eb59", + "lane": "periodic", + "artifact": "paid-slice-6-a1", + "path": "shards/skill-e2e-plan-ceo-review-section-loading/_partial-e2e.json#/tests/0/output" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "Genuine pass: the approved decision uses the slash-joined ROW-ID 'D1 / STALE-FILL', which the ledger reader parses since the lane D B2 change (before it, only hasStaleFillRaceFinding held).", + "input": { + "report": "# Plan: cache profile summaries in one process\n\n## Measured problem and accepted scope\nThe existing profile-summary service has one active process. A one-week trace\nshows repeated reads of about 900 hot keys: DB CPU is 70%, with read p95 120 ms.\nAdd a process-local LRU wrapper to the existing repository. Acceptance targets\nare at least 60% cache hits, DB CPU below 50%, and read p95 below 60 ms, with the\nexisting error-rate and correctness SLOs unchanged. This is an internal backend\nchange with no UI, API, schema, pricing, or developer onboarding change.\n\n## Existing contracts retained\n- All reads and writes use this repository in the same process; there are no\n external DB writers. Multi-process operation remains unsupported and startup\n rejects that configuration while caching is enabled.\n- These surrounding contracts are accepted fixture facts, supplied by the\n existing repository, cache adapter and rollout controller. Preserve them;\n review the new wrapper ordering below against them.\n- Authentication and authorization run before repository access. Keys encode\n the authenticated tenant ID and validated profile ID without ambiguity.\n Values are immutable profile-summary DTOs; secrets and cache keys are never\n logged. Cached results cannot bypass authorization.\n- The existing LRU adapter supports 1000 entries, a 16 MiB byte cap, and a\n 30-second TTL. Recorded hot data fits those limits. repository.read returns\n an immutable absent-result DTO for a missing record, never undefined. The\n adapter recognizes that DTO in cache.set, stores an internal sentinel with a\n 10-second TTL, and cache.get decodes it back to the same absent-result DTO.\n The internal sentinel cannot escape the adapter; undefined means a cache miss.\n- Cache operations are synchronous and atomic in the single JS event loop.\n On any cache failure the existing adapter bypasses the cache until an empty\n cache is reinitialized; repository errors keep the current typed API error\n mapping. The existing per-key single-flight wrapper sits inside\n repository.read, coalesces simultaneous store reads and releases on failure.\n A committed repository.write retires that key's old read cohort before its\n promise resolves. A later repository.read starts a fresh cohort; a rejected\n write leaves the cohort unchanged. Already-started readers may finish with\n their earlier snapshot. This admission rule does not inspect cache fills.\n- The repository uses an in-process transactional store, with no network\n transport between this wrapper and the store. repository.write is atomic:\n a resolved promise means committed, and every\n rejected promise guarantees no commit; its transaction rolled back before\n rejection. Existing contract tests exercise that guarantee.\n- Consistency is measured at the public wrapper boundary. A write completes\n when writeProfile's promise fulfills after cache.delete, not when\n repository.write commits or resolves. A read begins when readProfile is\n invoked. Reads that overlap an unfinished writeProfile may return an earlier\n snapshot, including reads begun after the store commit but before the wrapper\n promise fulfills. Every read begun after that write completes must\n observe the committed version. TTL expiry is not a substitute for this rule.\n\n## Proposed wrapper integration\nKeep the current read-through repository interface and shared adapters. These\nare the complete new read/write ordering rules; no additional version checks or\ncoordination between a cache fill and a write are proposed:\n\n```javascript\nasync function readProfile(key) {\n const cached = cache.get(key);\n if (cached !== undefined) return cached;\n const value = await repository.read(key);\n cache.set(key, value);\n return value;\n}\n\nasync function writeProfile(key, update) {\n const saved = await repository.write(key, update);\n cache.delete(key);\n return saved;\n}\n```\n\n### Accepted amendment A1 (D1 / STALE-FILL, approved 2026-10-04 by author policy auto-decision, option A)\nRequired guarantee (retained contract, now enforced by the wrapper): a cache fill must not publish a snapshot that predates a writeProfile for the same key that committed after the read's cache miss. Remedy: readProfile captures a per-key in-process write marker at the miss; writeProfile advances that marker after repository.write resolves and before cache.delete; readProfile publishes `cache.set` only if the marker is unchanged, checking and setting in the same synchronous turn. Absent-result DTOs follow the same rule, so a creation that completes during an in-flight absent read is never cached as absent. A rejected write does not advance the marker (preserving failed-write preservation). Marker state belongs to the wrapper instance; the controller's new-instance publication starts it fresh, and old reads cannot fill the new instance (existing controller contract). Tradeoffs: one small bounded piece of wrapper state and one suppressed-fill branch; write latency, limits, TTL and the adapter are unchanged. The suppressed-fill branch is counted on the current dashboard with no key labels (existing telemetry; no new alert or metric project). Deterministic regression scenarios (existing repository-contract harness, controlled pause/release): (S1) read miss, write commits and completes, read resolves late, fresh read must observe committed version; (S2) read resolves before write completes, fill then delete, fresh read observes committed version; (S3) absent read in flight, record created and write completes, fresh read observes the record; (S4) rejected write during in-flight read, fill proceeds, fresh read observes the (unchanged) earlier snapshot; (S5) rejected read releases single-flight, no fill, error mapping preserved; (S6) two overlapping writes, each completing after the other's commit, fresh read observes the last committed version; (S7) instance change mid-read, old read cannot fill the new instance. Exact data structure, bounding strategy and test code are left to engineering planning.\n\n## Verification and rollout\nExisting repository contract tests cover tenant isolation, key validation,\nabsence, DB failures, authorization, and startup rejection of multi-process\noperation while caching is enabled. New wrapper tests cover hit/miss,\neviction and byte limits, TTL, adapter-failure fallback, successful-write\ninvalidation, failed-write preservation, and concurrent-miss coalescing.\nThe rollout uses the existing runtime feature flag: enable for 10% of keys,\nthen 50%, then all keys after one healthy hour at each stage. Monitor hit/miss,\neviction, cache bytes, fallback errors, DB CPU, and read p95 without raw IDs.\nThe existing controller uses one shared key-selection predicate for reads and\nwrites. On any enable, disable or percentage change, it stops admitting work,\nawaits every admitted old-instance write, then publishes a new wrapper/cache\ninstance with a fresh single-flight cohort before admitting new work. Old reads\nretain their old instance and cannot fill the new one. Disabled instances\nbypass the cache on both paths. Tests cover the old-writer/new-reader ordering,\nall those transitions and predicate parity. This lifecycle isolation does not coordinate\nan ordinary DB write with a cache fill in the same active instance.\nExisting dashboards and runbooks cover these metrics. Before each stage, verify\nthat alerts page the service owner on any correctness/error-SLO breach, read\np95 above 120 ms for five minutes, or cache bypass persisting for one minute.\nHit rate is hits / (hits + misses) among requests admitted to the cache path;\nflag-excluded or adapter-bypassed requests are tracked separately, not as misses.\nDB CPU and read p95 are service-wide metrics, including bypassed requests.\nAt the 10% and 50% stages, a healthy hour requires at least 60% admitted-request\nhits, unchanged correctness/error SLOs, no alerts, and aggregate DB CPU/read p95\nno worse than their 70%/120 ms pre-rollout baselines. At 100%, the original\nabsolute acceptance targets (DB CPU below 50%, read p95 below 60 ms, hits at\nleast 60%) must all hold with unchanged correctness/error SLOs and no alerts.\nAny breach disables the flag immediately; the runbook records the incident,\nrollback and criteria for resuming. These are existing\nrollout-controller and telemetry contracts, not proposed wrapper additions.\nCold starts remain within the existing DB capacity. The service owner monitors\nthe rollout and records the results against the acceptance targets.\n\n## Out of scope\nDistributed caching, cross-process coherence, prewarming, changing consistency\nsemantics, or adding new product surfaces. The repository interface preserves a\nfuture replacement path without introducing a general cache framework now.\n\n## Author's review and acceptance requirements\nThis is a full CEO scope and feasibility review. The author has approved the\nretained contracts, limits, rollout and acceptance targets above. Evaluate the\nproposed wrapper against them; the wrapper itself remains unapproved. An actual\ncontradiction or missing proof must be reported and resolved, not assumed away.\nFor a demonstrated gap, amend the plan with the required guarantee, a feasible\nremedy, its tradeoffs and deterministic regression scenarios. Those repairs\nand their required verification are within the requested scope. The exact data\nstructures, full function bodies and executable test code belong to subsequent\nengineering planning; do not select or implement them during this review when\nthe required behavior and feasibility can already be established.\n\nUse the existing deterministic repository-contract test harness. Required wrapper\nacceptance includes both completion orders of overlapping reads and writes,\nmissing-record creation, rejected reads/writes, overlapping writes, and isolation\nacross controller instance changes. Use existing telemetry to record any added\nbranch on the current dashboard with no key labels; no new alert threshold or metric\nproject is requested. These are future acceptance requirements, not tests already\nimplemented or passing. Preserve all 11 review outcomes, required registries,\ndiagrams, tasks, completion summary and the full GSTACK REVIEW REPORT.\n\n---\n\n# CEO Review (plan-ceo-review, HOLD SCOPE) — working record\n\nRun: 2026-10-04. Author policy supplied by the run. Outside coverage: codex_reviews disabled in `.gstack-section-state-5DvWvu/config.yaml`; the extra outside-review step and its native fallback are skipped per that control. Preamble helpers, system audit, design-doc lookup, learnings, brain context, Aside/WebSearch research and telemetry are unavailable or skipped under the run rules; no dashboard or log result is claimed. Base branch: `main` (git status snapshot; no git commands run).\n\nReview depth: implementation-ready (default; the author requests a full scope and feasibility review with required guarantees and regression scenarios, while exact data structures and code stay with engineering). Storage: this file is the working plan and final output; checkpoints use scoped Edits.\n\n## Stated limits (recorded, unchanged)\n| Measure | Value | Prerequisite |\n|---|---|---|\n| Hot keys | ~900 (one-week trace) | fits adapter limits |\n| Adapter capacity | 1000 entries, 16 MiB, 30 s TTL; absent sentinel 10 s TTL | existing adapter |\n| Acceptance at 100% | hits >= 60%, DB CPU < 50%, read p95 < 60 ms | correctness/error SLOs unchanged, no alerts |\n| Stage health (10%, 50%) | hits >= 60% admitted, DB CPU/p95 no worse than 70%/120 ms | one healthy hour each |\n| Alerts | SLO breach; p95 > 120 ms for 5 min; bypass persisting 1 min | existing dashboards/runbooks |\n| Planned changed files | ~2 estimated (wrapper module, wrapper tests) | estimate; no file list in plan |\n\n## Decision ledger\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| MODE (author) | Run instruction: \"Hold the current scope (HOLD SCOPE mode)\" | HOLD SCOPE | none | approved | Explicit user choice in the run instruction; no question asked, no log. |\n| D1 / STALE-FILL (author; implementer: service team) | PLAN.md \"Existing contracts retained\" bullets 5–7 (cohort retirement does not inspect cache fills; already-started readers may finish with earlier snapshot; every read begun after a completed write must observe the committed version; TTL is not a substitute) vs \"Proposed wrapper integration\" (unconditional `cache.set` after `await repository.read`; \"no additional version checks or coordination between a cache fill and a write are proposed\") | Proposed ordering: read miss -> await store -> unconditional fill; write -> commit -> delete | A) write-aware fill guard; B) keep ordering, rely on TTL; C) hold write completion until in-flight reads finish | unresolved | pending |\n\n## 0A. Premise Challenge\nReal problem: ~900 hot profile-summary keys are re-read from the DB; DB CPU 70%, read p95 120 ms. Target: hits >= 60%, DB CPU < 50%, p95 < 60 ms with SLOs unchanged. Do-nothing cost: DB saturation headroom shrinks and p95 stays 2x the target. The plan solves the pain directly (serves repeated hot reads from process memory) rather than a proxy. Single process, no external writers, so a process-local cache is the right unit. Premise holds.\n\n## 0B. Existing Code Leverage\n| Sub-problem | Reused | Rebuild? |\n|---|---|---|\n| Bounded LRU, byte cap, TTL, absent sentinel, failure bypass | existing cache adapter | no |\n| Miss coalescing, cohort retirement on commit | existing per-key single-flight inside repository.read | no |\n| Atomic commit/rollback, typed error mapping | existing repository + contract tests | no |\n| Flag staging, instance swap, drain, predicate parity | existing rollout controller | no |\n| Metrics, alerts, runbooks | existing telemetry/dashboards | no |\n| New: read-through/write-invalidate ordering | new ~2-function wrapper | only new code |\nNo rebuild proposed; refactoring the existing read path is unnecessary because the wrapper sits above it.\n\n## 0C. Dream State Mapping\n```\n CURRENT STATE THIS PLAN 12-MONTH IDEAL\n every hot read hits DB; ---> process-local LRU wrapper; ---> same repository interface,\n DB CPU 70%, p95 120 ms hits >= 60%, CPU < 50%, cache tier swappable (local now,\n single process, no cache p95 < 60 ms, same SLOs distributed later if multi-process\n is ever supported); no framework\n```\nThe plan moves toward the ideal: it keeps the repository interface as the seam and adds no general cache framework (plan: \"Out of scope\").\n\n## 0D. Approach choice at Step 0\nNo new approach decision was needed: the author approved the process-local LRU approach, limits, rollout and targets. Fixture contracts are accepted facts; an unavailable implementation is not evidence against them.\n\n## 0E. Mode\nMode: HOLD SCOPE; approved decisions: MODE (explicit run instruction). Application: preserve scope, trace failures, edge cases, error paths, tests and observability; repairs needed to meet the retained invariants are in scope (0G rule 3). Provenance: explicit user choice; mode handoff helper not run (unavailable under run rules). Route: 0G -> 0I.\n\n## 0G. HOLD SCOPE checks\n1. Complexity: ~2 changed files, 0 new classes/services. Under thresholds; no challenge.\n2. Minimum change: the two wrapper functions plus tests. Nothing deferrable without blocking the goal; no deferral question raised.\n3. Invariants: the read-after-completed-write rule is a retained requirement. The proposed ordering does not meet it (see D1); its repair is in scope.\n\n## 0I. Temporal Interrogation\n```\n HOUR 1 (foundations): adapter semantics (undefined = miss, absent DTO round-trip, bypass), cohort rule, controller swap\n HOUR 2-3 (core logic): AMBIGUITY -> stale fill after a completed write (D1); absent-sentinel fill after record creation (same race)\n HOUR 4-5 (integration): feature-flag instance swap, metric for any added branch with no key labels, failed-write preservation\n HOUR 6+ (polish/tests): both completion orders of overlapping read/write in the deterministic harness; eviction/byte/TTL tests\n```\nEffort: human ~2–3 days / CC+gstack ~1–2 h for wrapper + tests. Feasibility blocker: D1 (below). Other design choices stay pending with engineering.\n\n### D1 feasibility trace (evidence)\n```\n t0 readProfile(k): cache.get -> undefined (miss)\n t1 repository.read(k) starts, snapshot = v1 (cohort C1)\n t2 writeProfile(k): repository.write commits v2, retires C1, resolves\n t3 cache.delete(k) (nothing cached) ; writeProfile fulfills == WRITE COMPLETE\n t4 readProfile(k') begins (after write complete) -> must observe v2\n t5 t1's read resolves with v1 (already-started reader keeps its snapshot, allowed)\n t6 cache.set(k, v1) <-- stale fill, no guard\n t7 t4's cache.get -> v1 == CONTRACT VIOLATION (until 30 s TTL; TTL is not a substitute)\n```\nVariant: k absent at t1, created at t2; t6 stores the absent sentinel (10 s), so reads after creation report absent. Both orders of t3/t5 matter: if t5 precedes t3, the delete removes the fill and the contract holds; the harness must cover both.\n\n## currentDecision (D1 / STALE-FILL)\nCommitment comparison:\n```text\nCommitment | Source/approval or pending | Current | A | B | C\nRead begun after completed write sees committed version | PLAN.md retained contract (approved) | violated by proposed ordering | restored | not restored (TTL-bounded staleness) | restored\nWrite completes after cache.delete, promptly | PLAN.md retained contract (approved) | yes | unchanged | unchanged | changed: completion waits on in-flight read cohorts\nOverlapping reads may return earlier snapshot | approved | yes | unchanged | unchanged | unchanged\nNo coordination between fill and write | proposed (unapproved wrapper text) | stated | replaced by write-aware fill guard | kept | replaced by write-side wait\nAdapter limits 1000 / 16 MiB / 30 s / 10 s sentinel | approved | yes | unchanged | unchanged | unchanged\nAdded observability | author: existing telemetry, current dashboard, no key labels | none | one suppressed-fill counter on current dashboard | none | one delayed-write counter\nRegression scenarios (both completion orders, creation, rejected ops, overlapping writes, instance isolation) | author acceptance requirement | required | required, deterministic | cannot pass read-after-write case | required; adds write-latency cases\n```\n\nQuestion: D1 — STALE-FILL: How should the wrapper stop a late cache fill from hiding a write that already completed?\nProject/branch/task: main; CEO review of the process-local profile-summary cache wrapper in PLAN.md.\nELI10: A read can start, fetch the old value from the database, and only then get around to putting it in the cache. If a write finished in between, the cache now holds the old value and every reader for up to 30 seconds sees stale data. The plan promises that never happens after a write completes, but the proposed code has no check to keep that promise.\nStakes if we pick wrong: users see a profile they just saved revert for up to 30 s (or a newly created profile report \"missing\" for 10 s), breaking the correctness SLO the rollout gates on.\nRecommendation: A because it restores the retained guarantee with an in-process, synchronous check and leaves write latency and limits untouched.\nCompleteness: A=10/10, B=3/10, C=7/10\nNet: A trades a small per-key in-flight marker for the guarantee; B silently weakens consistency; C keeps the guarantee by slowing writes.\nHeader: Stale fill fix\nA) Write-aware fill guard (recommended)\nAmend the plan: readProfile records a per-key marker at the miss, and writeProfile bumps that marker after commit and before cache.delete; the fill is published only if the marker is unchanged, in the same synchronous turn as cache.set. Effort S (human ~1 day / CC ~30 min), risk low, reuses the adapter and cohort rule, verification: deterministic harness covers both completion orders, creation, rejected ops, overlapping writes and instance isolation. ✅ Restores \"every read begun after a completed write observes the committed version\" without touching limits, TTL or write latency. ✅ The marker is in-process and synchronous, so no new coordination or network is involved and the controller's instance swap already isolates it. ❌ Adds one small piece of wrapper state and a suppressed-fill branch that engineering must bound and test.\nB) Keep the proposed ordering and rely on TTL\nLeave readProfile/writeProfile as written; staleness after a lost race is bounded by the 30 s TTL (10 s for absent). Effort S with zero implementation work, risk high, reuses everything, verification: the read-after-completed-write scenario cannot pass. ✅ No new wrapper state or branch to test. ✅ Smallest possible diff and simplest code. ❌ Contradicts the retained consistency rule and the author's statement that TTL is not a substitute; not authorizable under this review.\nC) Hold write completion until in-flight read cohorts finish\nwriteProfile waits, after commit, for any in-flight repository.read on that key to settle before cache.delete and fulfilment, so no stale fill can land after the write completes. Effort M (human ~2 days / CC ~1 h), risk medium, reuses the cohort rule only if the repository exposes cohort completion (unknown), verification: harness covers the same scenarios plus write-latency behavior. ✅ Restores the guarantee without a per-key marker in the wrapper. ✅ Keeps cache.set unconditional, so the read path is unchanged. ❌ Changes write-completion latency and couples writes to read storms, and depends on an unverified repository capability.\n\n## D1 decision record\nAuthority: run instruction (\"Authorize complete remedies and required verification that restore its retained contracts and acceptance targets; use the recommended option only among alternatives within that scope\") and author acceptance requirements in PLAN.md (\"For a demonstrated gap, amend the plan with the required guarantee, a feasible remedy, its tradeoffs and deterministic regression scenarios\"). AskUserQuestion unavailable (automated run); auto-decision authorized by that policy, not by a spawned-session echo. Answer: **A) Write-aware fill guard**. Exact scope: amendment A1 inserted before \"## Verification and rollout\" (guarantee, remedy, tradeoffs, scenarios S1–S7, suppressed-fill count on the current dashboard). Not authorized: B (weaker consistency), C (changed write-completion semantics), data-structure or test-code selection. Ledger row D1 status -> approved.\n\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| D1 / STALE-FILL (author; implementer: service team) | see ledger row above; trace in 0I | Proposed ordering + amendment A1 (fill guard) | B, C declined | approved | Author-policy auto-decision 2026-10-04 -> A; scope = A1 text only |\n\n## Review Sections (HOLD SCOPE, implementation-ready)\nCurrent scope: mode HOLD SCOPE (explicit run instruction). Governing rows: MODE approved (explicit); D1 approved (author-policy auto-decision -> A, scope A1). Accepted: original plan + A1. Deferred: none. Rejected: D1 options B and C. Pending: none.\n\n### Section 1: Architecture Review\n```\n caller --> rollout controller (flag predicate, instance swap)\n |-- disabled instance: readProfile/writeProfile bypass cache --> repository --> in-process store\n `-- enabled instance: readProfile ---> cache adapter (LRU/bytes/TTL/sentinel/bypass)\n | miss ^ guarded fill (A1)\n v |\n repository.read (single-flight cohort) --> store\n writeProfile --> repository.write (atomic) --> marker++ --> cache.delete\n```\nPaths: happy = miss, store read, guarded fill, later hits. Nil = absent DTO round-trips through sentinel; undefined only means miss. Empty = empty DTO is a value, cached as is. Error = repository error propagates with typed mapping, no fill, single-flight releases. State machine (wrapper instance): `fresh -> admitting -> draining -> retired`; invalid transition \"old read fills new instance\" is prevented by controller instance capture; \"fill after completed write\" was unprevented (CRITICAL GAP -> fixed by A1). Coupling: wrapper depends on adapter and repository only; no new coupling to the controller. Scaling: 10x hot keys (9000) exceeds the 1000-entry cap, hit rate degrades gracefully to eviction churn; 100x hits the byte cap first; both visible on eviction/bytes metrics. SPOF: single process already; cache failure degrades to bypass (DB load returns to baseline, alert at 1 min). Security: no new endpoint or mutation; auth precedes repository access. Production failure: adapter bypass during a stage -> DB CPU returns to 70%, alert pages, flag disabled. Rollback: flag off, controller swaps to a bypassing instance; seconds. **Outcome: 1 issue (D1, resolved by A1).** Decision gate: path 2, cites D1.\n\n### Section 2: Error & Rescue Map\n```\n METHOD/CODEPATH | WHAT CAN GO WRONG | EXCEPTION CLASS\n -----------------------|--------------------------------------------|----------------------------\n readProfile | cache.get adapter failure | adapter bypass (no throw, contract)\n | repository.read rejects (DB failure) | typed repository API error\n | late fill after completed write | none (silent) -> A1 guard\n writeProfile | repository.write rejects (rolled back) | typed repository API error\n | cache.delete adapter failure | adapter bypass (no throw, contract)\n controller swap | admitted old write still pending | awaited by controller (contract)\n\n EXCEPTION CLASS | RESCUED? | RESCUE ACTION | USER SEES\n ---------------------------|----------|------------------------------------|-------------------\n adapter failure | Y | bypass until reinit; alert at 1 min| transparent, slower\n typed repository error | Y (exist)| existing mapping; no fill; release | existing API error\n stale fill (pre-A1) | N <- GAP | A1: suppress fill, count branch | was: stale data\n rejected write | Y | marker unchanged, cache untouched | existing API error\n```\nNo catch-alls proposed. **Outcome: 6 paths mapped, 1 GAP (D1, resolved).** Decision gate: path 2.\n\n### Section 3: Security & Threat Model\nNo new endpoint, input, dependency, secret or background job. Keys encode tenant and profile IDs (contract); cross-tenant hit requires a key collision, prevented by unambiguous encoding: likelihood Low, impact High, mitigated. Cached DTOs are immutable, so a caller cannot mutate a shared cached object: mitigated. Keys and values are never logged; suppressed-fill count carries no key labels: mitigated. Memory exhaustion via many distinct keys is bounded by the 1000-entry/16 MiB caps: Low/Med, mitigated. **Outcome: No issues found.** Decision gate: nothing to resolve.\n\n### Section 4: Data Flow & Interaction Edge Cases\n```\n INPUT key -> (validated upstream, contract) -> cache.get -> miss -> repository.read -> A1 guard -> cache.set -> OUTPUT\n shadow: nil/absent -> absent DTO -> sentinel 10 s -> same DTO; invalid key -> rejected upstream; DB timeout -> typed error, no fill;\n conflict -> single-flight coalesces; stale -> A1 guard suppresses; oversize value -> byte cap evicts/declines (adapter)\n```\nAsync ordering: invariant = every readProfile begun after writeProfile fulfils observes the committed version. Schedule (shared state: cache entry, marker):\n```\n readProfile R1 | writeProfile W | fresh readProfile R2 | cache[k] | marker\n get -> miss, m0 | | | empty | m0\n await read (v1) | await write -> v2 | | empty | m0\n | marker++ ; delete | | empty | m1\n | fulfil (WRITE DONE) | begins | empty | m1\n resolves v1; m0!=m1 | | get -> miss -> read | empty | m1\n SUPPRESS fill | | -> v2, fill | v2 | m1\n Other order: R1 resolves before W fulfils -> fill v1 -> W deletes -> R2 misses -> v2. Mechanism: delete after commit.\n```\nPre-A1 the first order violated the invariant (0I trace). Regression proof: S1–S7 in A1 with controlled pause/release. Interaction edge cases: no user-visible interaction changes; double writes = S6; retries = idempotent reads. **Outcome: 7 edge cases mapped, 0 unhandled after A1.** Decision gate: path 2.\n\n### Section 5: Code Quality Review\nTwo small functions plus a per-key marker fit the existing repository-wrapper pattern. No duplicated behavior: invalidation and marker logic exist nowhere else. Naming: engineering to name the marker for what it guarantees (write-aware fill guard), not its mechanism. Branching stays under 5 per function. Under-engineering was the stale fill (fixed). No over-engineering: no general cache framework. **Outcome: No issues found.** Decision gate: nothing to resolve.\n\n### Section 6: Test Review\n```\n NEW THING | TEST TYPE | HAPPY | FAILURE | EDGE\n readProfile hit/miss | unit | miss then hit | adapter bypass -> store read | absent sentinel round-trip\n guarded fill (A1) | integration | S2 fill then delete | S1 suppressed fill, counted | S3 absent + creation, S7 instance\n writeProfile invalidate | integration | S2 | S4 rejected write, cache kept | S6 overlapping writes\n rejected read | unit | n/a | S5 typed error, no fill | single-flight released\n eviction/bytes/TTL | unit | 1000 entries, 16 MiB, 30 s | oversize value | 10 s sentinel expiry\n controller transitions | existing | enable/disable/percent | old-writer/new-reader order | predicate parity\n```\nAssertion check: S1 asserts R2 returns the committed version (rejects v1); S3 asserts the record (rejects absent); S4 asserts the earlier snapshot (rejects a spurious invalidation effect); hit-rate math asserts hits/(hits+misses) over admitted requests only. All map to retained contracts or A1; no new verification choice remains open. 2am test: S1 under the harness. Hostile QA: S3 and S6 together. Chaos: adapter failure mid-stage forces bypass and the 1-minute alert. Pyramid: mostly unit/integration, no E2E needed. Flakiness: all scenarios use controlled pause/release, no timers; TTL tests use the adapter's clock injection if present (engineering to confirm). Load: existing stage gates cover p95/CPU. **Outcome: diagram produced, 0 gaps after A1.** Decision gate: path 2.\n\n### Section 7: Performance Review\nNo N+1 or new queries. Memory: cache <= 16 MiB; marker map must be bounded to in-flight keys (requirement recorded in A1; structure left to engineering). Slow paths: miss path = store read + synchronous set; hit path = O(1) get; write path adds one synchronous marker update. Connection pools unchanged. Cold start: within existing DB capacity (plan). **Outcome: No issues found.**\n\n### Section 8: Observability & Debuggability Review\nExisting metrics: hit/miss, eviction, bytes, fallback errors, DB CPU, p95; existing alerts and runbooks. A1 adds one suppressed-fill count on the current dashboard, no key labels, no new alert (author constraint). A 3-weeks-later stale-read report is reconstructable from suppressed-fill, bypass and flag-stage history. Runbook: bypass persisting 1 min -> page, disable flag. **Outcome: 1 gap (suppressed-fill visibility) resolved within A1.**\n\n### Section 9: Deployment & Rollout Review\nNo migration. Flag staged 10% -> 50% -> 100% with one healthy hour each; controller drains writes and swaps instances. Rollback: disable flag -> bypassing instance; runbook records incident. Deploy-time risk window: controller contract isolates instances. Post-deploy checks: first 5 min hit rate, bypass errors, suppressed-fill count nonzero but small; first hour stage criteria. Smoke: existing contract tests plus S1–S7 in CI. **Outcome: 0 new risks flagged.**\n\n### Section 10: Long-Term Trajectory Review\nDebt: one marker to maintain; documented in A1. Path dependency: none, repository seam preserved. Knowledge: A1 plus the 0I trace explain the why. Reversibility 5/5 (flag off, delete wrapper). Ecosystem fit: wraps existing adapter. 1-year: obvious. **Outcome: debt items 1, reversibility 5/5.**\n\n### Section 11: Design & UX Review\nSKIPPED (no UI scope).\n\n## Closing sequence\n1. Outside Voice: codex_reviews disabled (config read above); skipped with no native fallback, as directed. Review-log helper unavailable under run rules; disabled coverage **not persisted**. Codex review skipped (codex_reviews disabled). Re-enable: `gstack-config set codex_reviews enabled`.\n2. TODO choices: none remain (no deferred gap; HOLD SCOPE).\n3. Approval readiness: PASS. Rows checked: MODE (explicit run instruction), D1 (author-policy auto-decision -> A, scope A1). No pending, declined or deferred remedies in accepted work.\n\n## NOT in scope\nDeferred: none. Rejected: D1-B keep ordering/rely on TTL (contradicts retained consistency rule); D1-C hold write completion (changes write semantics, unverified repository capability). Original out-of-scope list unchanged.\n\n## What already exists\nCache adapter, single-flight cohort rule, atomic repository, rollout controller, telemetry/alerts/runbooks, repository contract tests: all reused (0B).\n\n## Dream state delta\nAfter this plan: hot reads served in-process with the read-after-write rule intact; repository seam ready for a later cache-tier swap; no framework added. See 0C.\n\n## Error & Rescue Registry\nSee Section 2 tables: 6 rows, 1 CRITICAL GAP (stale fill) resolved by A1; 0 open.\n\n## Failure Modes Registry\n```\n CODEPATH | FAILURE MODE | RESCUED? | TEST? | USER SEES? | LOGGED?\n -----------------------|------------------------------|----------|-----------|-------------------|--------\n readProfile | adapter failure | Y | Y | slower, correct | Y (bypass)\n readProfile | repository.read rejects | Y | Y (S5) | API error | Y (existing)\n readProfile fill | stale fill after write | Y (A1) | Y (S1,S3) | correct | Y (count)\n writeProfile | repository.write rejects | Y | Y (S4) | API error | Y (existing)\n writeProfile | cache.delete adapter failure | Y | Y | slower, correct | Y (bypass)\n controller swap | pending old write | Y | Y (exist) | none | Y (existing)\n```\n6 total, 0 CRITICAL GAPS open.\n\n## Diagrams\n1. System architecture: Section 1. 2. Data flow with shadow paths: Section 4. 3. State machine: Section 1 (wrapper instance). 4. Error flow: Section 2. 5. Deployment sequence: `deploy wrapper (flag off) -> enable 10% -> healthy hour -> 50% -> healthy hour -> 100% -> acceptance check`. 6. Rollback: `breach or alert -> disable flag -> controller drains writes, swaps to bypass instance -> runbook records incident -> resume criteria`.\nStale Diagram Audit: no ASCII diagrams exist in touched files (plan only); none stale.\n\n## Implementation Tasks\nSynthesized from this review's findings. Each task derives from a specific finding above. Run with Claude Code or Codex; checkbox as you ship.\n\n- [ ] **T1 (P1, human: ~1 day / CC: ~30 min)** — profile-summary wrapper — Implement readProfile/writeProfile with the write-aware fill guard (A1)\n - Surfaced by: Section 4 / 0I — stale fill after completed write (D1)\n - Files: to be determined (wrapper module)\n - Verify: harness scenarios S1–S7 pass; existing repository contract tests pass\n- [ ] **T2 (P1, human: ~1 day / CC: ~30 min)** — wrapper tests — Add deterministic tests for hit/miss, eviction, bytes, TTL, bypass, invalidation, failed-write preservation, coalescing and S1–S7\n - Surfaced by: Section 6 — test diagram\n - Files: to be determined (wrapper test module)\n - Verify: tests run under the existing deterministic harness with controlled pause/release\n- [ ] **T3 (P2, human: ~2 h / CC: ~10 min)** — telemetry — Emit the suppressed-fill count on the current dashboard with no key labels\n - Surfaced by: Section 8 — added branch visibility\n - Files: to be determined\n - Verify: panel shows the count during S1 in staging; no key labels present\n_No new tasks from Sections 3, 5, 7, 9, 10, 11._\nTask JSONL: **not persisted** (gstack helpers unavailable under run rules).\n\n## Completion Summary\n```\n +====================================================================+\n | MEGA PLAN REVIEW — COMPLETION SUMMARY |\n +====================================================================+\n | Mode selected | HOLD SCOPE |\n | System Audit | skipped by run rules; plan-only review |\n | Step 0 | HOLD SCOPE; D1 -> A (fill guard, A1) |\n | Section 1 (Arch) | 1 issues found (D1, resolved) |\n | Section 2 (Errors) | 6 error paths mapped, 1 GAPS (resolved) |\n | Section 3 (Security)| 0 issues found, 0 High severity |\n | Section 4 (Data/UX) | 7 edge cases mapped, 0 unhandled |\n | Section 5 (Quality) | 0 issues found |\n | Section 6 (Tests) | Diagram produced, 0 gaps |\n | Section 7 (Perf) | 0 issues found |\n | Section 8 (Observ) | 1 gaps found (resolved in A1) |\n | Section 9 (Deploy) | 0 risks flagged |\n | Section 10 (Future) | Reversibility: 5/5, debt items: 1 |\n | Section 11 (Design) | SKIPPED (no UI scope) |\n +--------------------------------------------------------------------+\n | NOT in scope | written (2 items rejected, 0 deferred) |\n | What already exists | written |\n | Dream state delta | written |\n | Error/rescue registry| 6 rows, 0 CRITICAL GAPS open |\n | Failure modes | 6 total, 0 CRITICAL GAPS |\n | TODOS.md updates | 0 items proposed |\n | Scope proposals | 0 proposed, 0 accepted (HOLD SCOPE) |\n | CEO plan | skipped by mode |\n | Outside voice | codex: disabled |\n | Lake Score | 1/1 recommendations chose complete option |\n | Diagrams produced | 6 (arch, data flow, state, error, deploy, rollback) |\n | Stale diagrams found | 0 |\n | Unresolved decisions | 0 |\n +====================================================================+\n```\nUnresolved Decisions: none. Learnings: no durable learnings this session. Review log, decision log and dashboard: **not persisted** (helpers unavailable; no dashboard result claimed).\n\n## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 1 (not persisted) | CLEAR | mode: HOLD SCOPE, 0 critical gaps |\n| Outside Review | codex (`codex_reviews: disabled`) | Independent 2nd opinion | 0 | DISABLED | disabled by config; no completed external review |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | — | not run |\n| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | not run (no UI scope) |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | not run |\n\n**OUTSIDE COVERAGE:** provider codex, phase plan-review, disabled by `codex_reviews: disabled`; no outside process or native fallback ran; no completed external review; disabled record not persisted.\n\n**VERDICT:** CEO CLEARED — plan approved with amendment A1 (D1 -> A); eng review required\n\nNO UNRESOLVED DECISIONS\n" + } +} diff --git a/test/fixtures/detector-corpora/plan-ceo-section-loading/37198445662-t1-pass.json b/test/fixtures/detector-corpora/plan-ceo-section-loading/37198445662-t1-pass.json new file mode 100644 index 000000000..787a475fa --- /dev/null +++ b/test/fixtures/detector-corpora/plan-ceo-section-loading/37198445662-t1-pass.json @@ -0,0 +1,20 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-ceo-section-loading", + "detector": "test/helpers/ceo-stale-fill-decision.ts#hasApprovedStaleFillDecision || test/helpers/ceo-section-loading-fixture.ts#hasStaleFillRaceFinding", + "series_identity": "68761d79db01290b", + "source": { + "run": "37198445662", + "sha": "fd6854b5356431e5e115fca082b21bf237559558", + "lane": "periodic", + "artifact": "paid-slice-6-a1", + "path": "shards/skill-e2e-plan-ceo-review-section-loading/_partial-e2e.json#/tests/0/output" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "Genuine pass at the latest census SHA: approved stale-fill ledger decision and prose finding both hold.", + "input": { + "report": "# Plan: cache profile summaries in one process\n\n## Measured problem and accepted scope\nThe existing profile-summary service has one active process. A one-week trace\nshows repeated reads of about 900 hot keys: DB CPU is 70%, with read p95 120 ms.\nAdd a process-local LRU wrapper to the existing repository. Acceptance targets\nare at least 60% cache hits, DB CPU below 50%, and read p95 below 60 ms, with the\nexisting error-rate and correctness SLOs unchanged. This is an internal backend\nchange with no UI, API, schema, pricing, or developer onboarding change.\n\n## Existing contracts retained\n- All reads and writes use this repository in the same process; there are no\n external DB writers. Multi-process operation remains unsupported and startup\n rejects that configuration while caching is enabled.\n- These surrounding contracts are accepted fixture facts, supplied by the\n existing repository, cache adapter and rollout controller. Preserve them;\n review the new wrapper ordering below against them.\n- Authentication and authorization run before repository access. Keys encode\n the authenticated tenant ID and validated profile ID without ambiguity.\n Values are immutable profile-summary DTOs; secrets and cache keys are never\n logged. Cached results cannot bypass authorization.\n- The existing LRU adapter supports 1000 entries, a 16 MiB byte cap, and a\n 30-second TTL. Recorded hot data fits those limits. repository.read returns\n an immutable absent-result DTO for a missing record, never undefined. The\n adapter recognizes that DTO in cache.set, stores an internal sentinel with a\n 10-second TTL, and cache.get decodes it back to the same absent-result DTO.\n The internal sentinel cannot escape the adapter; undefined means a cache miss.\n- Cache operations are synchronous and atomic in the single JS event loop.\n On any cache failure the existing adapter bypasses the cache until an empty\n cache is reinitialized; repository errors keep the current typed API error\n mapping. The existing per-key single-flight wrapper sits inside\n repository.read, coalesces simultaneous store reads and releases on failure.\n A committed repository.write retires that key's old read cohort before its\n promise resolves. A later repository.read starts a fresh cohort; a rejected\n write leaves the cohort unchanged. Already-started readers may finish with\n their earlier snapshot. This admission rule does not inspect cache fills.\n- The repository uses an in-process transactional store, with no network\n transport between this wrapper and the store. repository.write is atomic:\n a resolved promise means committed, and every\n rejected promise guarantees no commit; its transaction rolled back before\n rejection. Existing contract tests exercise that guarantee.\n- Consistency is measured at the public wrapper boundary. A write completes\n when writeProfile's promise fulfills after cache.delete, not when\n repository.write commits or resolves. A read begins when readProfile is\n invoked. Reads that overlap an unfinished writeProfile may return an earlier\n snapshot, including reads begun after the store commit but before the wrapper\n promise fulfills. Every read begun after that write completes must\n observe the committed version. TTL expiry is not a substitute for this rule.\n\n## Proposed wrapper integration\nKeep the current read-through repository interface and shared adapters. The\nsketch below shows the baseline ordering as originally proposed. Per CEO review\ndecision D1 (ledger R1), it is amended by the **fill-admission guarantee**: a read's\ncache fill is admitted only if no repository.write for that key has committed since\nthat read began; writeProfile advances a per-key in-process committed-write marker\nafter repository.write resolves and before cache.delete. A suppressed fill still\nreturns the value to the caller and increments a suppressed-fill counter on the\nexisting dashboard with no key labels. The marker's exact data structure and the\nfunction bodies are engineering-planning work, not selected here.\n\n```javascript\nasync function readProfile(key) {\n const cached = cache.get(key);\n if (cached !== undefined) return cached;\n const value = await repository.read(key);\n cache.set(key, value);\n return value;\n}\n\nasync function writeProfile(key, update) {\n const saved = await repository.write(key, update);\n cache.delete(key);\n return saved;\n}\n```\n\n## Verification and rollout\nExisting repository contract tests cover tenant isolation, key validation,\nabsence, DB failures, authorization, and startup rejection of multi-process\noperation while caching is enabled. New wrapper tests cover hit/miss,\neviction and byte limits, TTL, adapter-failure fallback, successful-write\ninvalidation, failed-write preservation, and concurrent-miss coalescing.\nPer D1 (R1) they also cover, with controlled pause/release points in the existing\ndeterministic harness: (a) read starts, write commits and completes, read resolves\n→ fill suppressed, next read observes committed version; (b) read resolves before\nwrite completes → fill admitted, then deleted by the write; (c) missing-record read\nin flight while the record is created → absent sentinel not cached after creation;\n(d) two overlapping writes with an in-flight read → no stale fill; (e) rejected read\n→ no fill, marker unchanged; rejected write → cache preserved, marker unchanged;\n(f) controller instance change during an in-flight read → old read cannot fill the\nnew instance.\nThe rollout uses the existing runtime feature flag: enable for 10% of keys,\nthen 50%, then all keys after one healthy hour at each stage. Monitor hit/miss,\neviction, cache bytes, fallback errors, DB CPU, and read p95 without raw IDs.\nThe existing controller uses one shared key-selection predicate for reads and\nwrites. On any enable, disable or percentage change, it stops admitting work,\nawaits every admitted old-instance write, then publishes a new wrapper/cache\ninstance with a fresh single-flight cohort before admitting new work. Old reads\nretain their old instance and cannot fill the new one. Disabled instances\nbypass the cache on both paths. Tests cover the old-writer/new-reader ordering,\nall those transitions and predicate parity. This lifecycle isolation does not coordinate\nan ordinary DB write with a cache fill in the same active instance.\nExisting dashboards and runbooks cover these metrics. Before each stage, verify\nthat alerts page the service owner on any correctness/error-SLO breach, read\np95 above 120 ms for five minutes, or cache bypass persisting for one minute.\nHit rate is hits / (hits + misses) among requests admitted to the cache path;\nflag-excluded or adapter-bypassed requests are tracked separately, not as misses.\nDB CPU and read p95 are service-wide metrics, including bypassed requests.\nAt the 10% and 50% stages, a healthy hour requires at least 60% admitted-request\nhits, unchanged correctness/error SLOs, no alerts, and aggregate DB CPU/read p95\nno worse than their 70%/120 ms pre-rollout baselines. At 100%, the original\nabsolute acceptance targets (DB CPU below 50%, read p95 below 60 ms, hits at\nleast 60%) must all hold with unchanged correctness/error SLOs and no alerts.\nAny breach disables the flag immediately; the runbook records the incident,\nrollback and criteria for resuming. These are existing\nrollout-controller and telemetry contracts, not proposed wrapper additions.\nCold starts remain within the existing DB capacity. The service owner monitors\nthe rollout and records the results against the acceptance targets.\n\n## Out of scope\nDistributed caching, cross-process coherence, prewarming, changing consistency\nsemantics, or adding new product surfaces. The repository interface preserves a\nfuture replacement path without introducing a general cache framework now.\n\n## Author's review and acceptance requirements\nThis is a full CEO scope and feasibility review. The author has approved the\nretained contracts, limits, rollout and acceptance targets above. Evaluate the\nproposed wrapper against them; the wrapper itself remains unapproved. An actual\ncontradiction or missing proof must be reported and resolved, not assumed away.\nFor a demonstrated gap, amend the plan with the required guarantee, a feasible\nremedy, its tradeoffs and deterministic regression scenarios. Those repairs\nand their required verification are within the requested scope. The exact data\nstructures, full function bodies and executable test code belong to subsequent\nengineering planning; do not select or implement them during this review when\nthe required behavior and feasibility can already be established.\n\nUse the existing deterministic repository-contract test harness. Required wrapper\nacceptance includes both completion orders of overlapping reads and writes,\nmissing-record creation, rejected reads/writes, overlapping writes, and isolation\nacross controller instance changes. Use existing telemetry to record any added\nbranch on the current dashboard with no key labels; no new alert threshold or metric\nproject is requested. These are future acceptance requirements, not tests already\nimplemented or passing. Preserve all 11 review outcomes, required registries,\ndiagrams, tasks, completion summary and the full GSTACK REVIEW REPORT.\n\n## CEO Review — Step 0 (HOLD SCOPE)\n\nReview depth: implementation-ready for behavior, interfaces and verification; exact data\nstructures, function bodies and test code stay with engineering per the author's policy.\nStorage: this file is the single working plan and final output. Preamble/system audit,\ntelemetry, Aside/WebSearch, brain context, prior learnings: skipped per run rules (fixture).\nOutside coverage (codex_reviews): disabled by `.gstack-section-state-IfADEH/config.yaml`.\nSession transport: automated run, author policy authorizes auto-selection of the recommended\noption within retained scope; no AskUserQuestion available.\n\n### 0A. Premise Challenge\nReal problem: ~900 hot keys re-read from the DB; DB CPU 70%, read p95 120 ms. Target: hits\n>= 60%, DB CPU < 50%, p95 < 60 ms, SLOs unchanged. Do-nothing cost: DB saturation headroom\nkeeps shrinking and p95 stays above the target. A process-local read-through cache attacks\nthe pain directly (repeated identical reads), not a proxy. Premise holds under HOLD SCOPE.\n\n### 0B. Existing Code Leverage\n| Sub-problem | Reused | Notes |\n|---|---|---|\n| Bounded storage, TTL, absent sentinel | existing LRU adapter | fixture fact, retained |\n| Concurrent-miss coalescing | single-flight inside repository.read | fixture fact, retained |\n| Rollout, instance swap, predicate parity | existing rollout controller | fixture fact, retained |\n| Dashboards, alerts, runbooks | existing telemetry | retained; new branch recorded on current dashboard |\n| New code | readProfile / writeProfile ordering | the only unapproved piece |\nNo rebuild proposed; nothing to justify.\n\n### 0C. Dream State Mapping\n```\n CURRENT STATE THIS PLAN 12-MONTH IDEAL\n single process, no cache, process-local LRU wrapper same repository interface,\n DB CPU 70%, p95 120 ms ---> behind existing repo, ---> replaceable cache backend if\n 900 hot keys re-read 60% hits, CPU<50%, p95<60 multi-process ever arrives\n```\nMoves toward the ideal: keeps the repository interface as the replacement seam (plan \"Out\nof scope\" paragraph) without a cache framework now.\n\n### Decision ledger\n| ID and owner | Contract and evidence | Current | Proposed | Status | Exact approval and scope |\n|---|---|---|---|---|---|\n| R1 (author / wrapper) | \"Every read begun after that write completes must observe the committed version. TTL expiry is not a substitute\" (PLAN.md, Existing contracts retained, last bullet). Proposed ordering: readProfile fills cache after `await repository.read` with no fill/write coordination (PLAN.md, Proposed wrapper integration). Single-flight: \"Already-started readers may finish with their earlier snapshot. This admission rule does not inspect cache fills.\" | Consistency rule retained; wrapper ordering unapproved; no fill/write coordination proposed | A) amend wrapper with a fill-admission guarantee; B) keep ordering, weaken rule to TTL-bounded staleness; C) drop the write path from the cache (bypass + delete only, still fills on read) | approved | D1 → A. Authority: run rules (author policy: auto-select the recommended option among in-scope alternatives; weaker consistency and changed limits not authorizable, so B excluded; C does not meet the retained rule). Scope: amend the wrapper with the fill-admission guarantee, writeProfile per-key committed-write marker advanced before cache.delete, suppressed-fill count on the existing dashboard with no key labels, and the deterministic regression scenarios listed in the comparison. No data structure or test code selected. |\n\n### Demonstrated gap behind R1 (evidence trace)\n```\n t0 R1: readProfile(k) -> cache.get miss -> repository.read(k) starts, snapshot v1\n t1 W: writeProfile(k) -> repository.write commits v2, retires R1's cohort, resolves\n t2 W: cache.delete(k) (nothing cached) -> writeProfile fulfills == WRITE COMPLETE\n t3 R2: readProfile(k) begins (after write complete) -> cache.get miss -> read v2 ok\n t4 R1: repository.read resolves v1 (allowed by single-flight) -> cache.set(k, v1)\n t5 R3: readProfile(k) begins -> cache.get HIT v1 (stale) for up to 30 s TTL\n```\nR3 began after the write completed and observes v1: a direct violation of the retained\nrule. The same trace with a missing record caches the absent sentinel for 10 s after the\nrecord was created. Retained contracts do not prevent this: the single-flight rule says\nit \"does not inspect cache fills\", and the controller note says lifecycle isolation \"does\nnot coordinate an ordinary DB write with a cache fill in the same active instance\". This\nis a contradiction inside the proposal, not an unavailable implementation.\n\n## currentDecision (R1)\nCommitment comparison:\n```text\nCommitment | Source/approval or pending | Current | A | B | C\nRead-after-write-complete observes committed version | PLAN.md retained contract, approved | required | kept | weakened to TTL-bounded | kept but unmet (stale fill still possible)\nTTL is not a consistency substitute | PLAN.md retained contract, approved | required | kept | violated | kept\nAdapter limits 1000 / 16 MiB / 30 s / 10 s absent | PLAN.md retained, approved | fixed | unchanged | unchanged | unchanged\nSingle-flight cohort rule (no fill inspection) | PLAN.md retained, approved | fixed | unchanged; wrapper adds its own fill check | unchanged | unchanged\nreadProfile fill after repository.read | PLAN.md proposed, pending | proposed | fill admitted only if no committed write for key since read began; otherwise skip fill, return value | unchanged | unchanged\nwriteProfile: await write, cache.delete, return | PLAN.md proposed, pending | proposed | unchanged, plus per-key committed-write marker advanced before cache.delete | unchanged | write path also bypasses cache reads (no gain)\nTelemetry for added branch | author requirement, approved | n/a | stale-fill-suppressed count on current dashboard, no key labels | none | none\nDeterministic regression scenarios | author requirement, approved | required | both completion orders, missing-record creation, overlapping writes, rejected read/write, instance change | not satisfiable | not satisfiable\nAcceptance targets 60% / <50% CPU / <60 ms | PLAN.md, approved | fixed | unchanged | unchanged | unchanged\n```\n\nQuestion: D1 — R1: Fix the stale cache fill that breaks read-after-write consistency?\nProject/branch/task: main, PLAN.md profile-summary cache wrapper, HOLD SCOPE review.\nELI10: A read can start, a write can finish and clear the cache, then the slow old read\nlands and stores the old value. Everyone then sees the old profile for up to 30 seconds,\nwhich the plan explicitly promises cannot happen. Stakes if we pick wrong: correctness SLO\nbreach that the rollout alert may not catch, or a quietly weakened promise.\nRecommendation: A because it is the only option that keeps the retained consistency rule\nand is feasible in one event loop with synchronous cache operations (small diff, low\nmaintenance: one per-key marker checked at fill time).\nCompleteness: A=10/10, B=3/10, C=3/10\nNet: trade a small fill-time check and one counter for keeping the promise the plan already makes.\nHeader: Stale fill fix\nA) Add fill-admission guarantee (recommended)\nAmend the wrapper: a fill is admitted only if no repository.write for that key committed since the read began; writeProfile advances that per-key marker before cache.delete. Effort S, risk low, reuses adapter and single-flight unchanged, verification covers both completion orders, missing-record creation, overlapping writes, rejected read/write and instance change. ✅ Keeps \"every read after write completes sees the committed version\" with no TTL reliance. ✅ Deterministic in the single JS event loop; sync cache ops make the check atomic. ❌ One more per-key in-process state and a new suppressed-fill telemetry branch to test.\nB) Keep ordering, allow TTL-bounded staleness\nLeave readProfile/writeProfile as proposed and rewrite the consistency contract to allow up to 30 s (10 s absent) staleness after a completed write. Effort S, risk high, no reuse change, verification cannot prove read-after-write. ✅ Zero wrapper changes beyond the proposed code. ✅ Simplest code path to reason about for hit-rate measurement. ❌ Changes a retained consistency requirement, which the author has not authorized.\nC) Keep ordering, bypass cache on the write path only\nwriteProfile skips the cache entirely except for the delete; reads unchanged. Effort S, risk high, no reuse change, verification still fails the stale-fill scenario. ✅ No change to the read path the rollout measures. ✅ Preserves the stated adapter limits and TTLs. ❌ Does not address the race; the stale fill comes from the read path, so the retained rule stays violated.\n\n### 0E. Mode\nMode: HOLD SCOPE; explicit user instruction (\"Hold the current scope (HOLD SCOPE mode)\").\nNo mode question asked or logged; mode handoff script unavailable in this fixture (skipped\nper run rules). Approved decisions: R1 (D1 → A, scope above). No other approach decision\nwas needed: the current plan is the only candidate approach and HOLD SCOPE forbids expansion.\n\n### 0G. HOLD SCOPE checks\n1. Complexity: estimated 2–3 changed files (wrapper module, wrapper tests, dashboard\n panel config), 0 new classes/services → under the 8-file / 2-class threshold. OK.\n2. Minimum change: the two wrapper functions plus the D1 guard. Nothing deferrable\n without blocking the acceptance targets; no deferral proposed, no defer/keep question.\n3. Invariants kept: all retained contracts, limits (1000 / 16 MiB / 30 s / 10 s), targets\n (60% / <50% / <60 ms) unchanged. D1 is the one repair needed to meet them.\n\n### 0I. Temporal Interrogation\n```\nHOUR 1 foundations: adapter API (get/set/delete, absent sentinel), single-flight cohort\n rule, controller instance lifecycle, telemetry counter API. All fixture facts.\nHOUR 2-3 core logic: fill-admission guard (D1). Ambiguity: where the per-key marker lives\n and how it is bounded (engineering; see Section 7 requirement). Marker advances\n after repository.write resolves and before cache.delete, synchronously.\nHOUR 4-5 integration: controller swap mid-read (old read must not fill new instance,\n already a controller contract); adapter failure mid-write (bypass until reinit).\nHOUR 6+ tests: six D1 scenarios with pause/release points; TTL tests on the harness's\n controlled clock, never wall-clock sleeps.\n```\nEffort: human ~2 days / CC+gstack ~40 min. Feasibility blockers: none after D1.\nPending (engineering, non-blocking): marker data structure and bounding strategy.\n\n## Review Sections (HOLD SCOPE, implementation-ready)\n\nCurrent scope: mode HOLD SCOPE (explicit instruction). Governing rows: R1 approved\n(D1 → A, authority and scope in the ledger). Accepted: plan as written plus the D1\namendment. Deferred: none. Rejected: D1 options B and C. Pending: none blocking.\n\n### Section 1: Architecture Review\n```\n caller ──auth/authz──▶ readProfile/writeProfile (wrapper, NEW)\n │ │\n cache.get/set/delete await repository.read/write\n ▼ ▼\n existing LRU adapter existing repository (+ single-flight)\n (1000 / 16 MiB / 30 s) │\n in-process transactional store\n rollout controller ──publishes──▶ wrapper+cache instance (fresh cohort per change)\n```\n- Boundaries: wrapper is the only new component; it depends on adapter and repository\n only. New coupling: wrapper↔adapter sentinel semantics (already adapter-owned). Justified.\n- Data flow paths: happy (miss→read→fill→return), nil (absent DTO → sentinel 10 s),\n empty (no empty-value shape exists; DTO is always a full object), error (read rejects →\n typed error, no fill). Diagrammed in Section 4.\n- State machine (per key, per instance): `empty → filled(v) | absent-sentinel` via\n admitted fill; `any → empty` via cache.delete, TTL expiry, eviction, adapter bypass\n reinit. Invalid transition prevented by D1: `empty → filled(stale)` after a completed\n write; the guard blocks it. Section 4 shows the schedule.\n- Scaling: 10x load → hit rate rises, DB load falls; 100x → event-loop CPU on cache ops\n and the 16 MiB cap drive evictions; hit rate falls toward baseline but correctness holds.\n- SPOF: the single process (pre-existing). Security: no new surface (Section 3).\n- Production failure: adapter throws mid-set → bypass until reinit, DB absorbs load\n (cold-start capacity stated). Rollback: disable flag, instant, controller swaps instance.\n**Finding F1 — CRITICAL GAP (resolved):** stale fill after completed write; evidence\ntrace and remedy under ledger R1 / D1 → A. Residual: marker bounding (Section 7).\nDecision gate: F1 resolved by D1 → A; applied to plan. No other findings.\n\n### Section 2: Error & Rescue Map\n```\n METHOD/CODEPATH | WHAT CAN GO WRONG | EXCEPTION CLASS\n -------------------------|------------------------------------------|---------------------\n readProfile cache.get | adapter failure | AdapterError (internal)\n readProfile repo.read | store error / timeout / not found | existing typed API errors; absent DTO (not an error)\n readProfile cache.set | adapter failure; byte cap exceeded | AdapterError (internal)\n readProfile fill guard | marker says write committed since start | none (branch, not error)\n writeProfile repo.write | validation / conflict / store error | existing typed API errors (rolled back)\n writeProfile cache.delete| adapter failure | AdapterError (internal)\n startup | multi-process config with caching on | StartupConfigError (existing)\n -------------------------|------------------------------------------|---------------------\n EXCEPTION CLASS | RESCUED? | RESCUE ACTION | USER SEES\n -------------------------|----------|----------------------------------------|----------------\n AdapterError (get/set) | Y adapter| bypass cache until empty reinit; count | normal response, slower\n AdapterError (delete) | Y adapter| bypass until reinit (cache emptied) | write succeeds\n typed read error | Y repo | existing mapping; no fill; cohort freed| existing error\n typed write error | Y repo | rollback; no delete; marker unchanged | existing error\n fill-guard branch | n/a | skip fill, return value, count suppressed | fresh value\n StartupConfigError | Y | refuse to start | deploy fails loudly\n```\nNo catch-alls introduced; wrapper adds no try/catch of its own. No GAPs. Decision gate: no issues found beyond F1 (already resolved).\n\n### Section 3: Security & Threat Model\nNo new endpoint, input, job or dependency. Keys carry authenticated tenant + validated\nprofile ID (fixture); cross-tenant hit impossible without key collision, which the key\ncontract excludes. Values immutable DTOs → no shared-mutation leak. Secrets/keys never\nlogged; the D1 counter carries no key labels. Threats: cache poisoning via stale fill\n(Low likelihood, Med impact) mitigated by D1; memory exhaustion via many keys (Low/Low)\nmitigated by 1000 / 16 MiB caps. Decision gate: no issues found.\n\n### Section 4: Data Flow & Interaction Edge Cases\n```\nINPUT(key) -> VALIDATION(upstream auth/authz, key encoding) -> cache.get\n -> hit: OUTPUT(dto) -> miss: repository.read -> fill guard -> cache.set -> OUTPUT\nshadow: nil record -> absent DTO -> sentinel 10 s wrong type -> rejected upstream (existing)\n read rejects -> typed error, no fill adapter throws -> bypass, OUTPUT from repo\n oversize value -> adapter refuses set, OUTPUT ok stale (write raced) -> guard suppresses fill\n```\nAsync ordering, invariant: \"every read begun after writeProfile fulfills observes the\ncommitted version\" (boundary: readProfile invocation vs. writeProfile fulfilment).\n```\n readProfile R1 | writeProfile W | cache(k) | marker(k)\n get miss; read starts | | empty | g0\n | write commits v2 | empty | g0\n | marker -> g1; delete | empty | g1 == W fulfils\n (R2 begins: miss, reads v2, fills v2 if marker still g1)\n read resolves v1 | | |\n guard: g0 != g1 -> skip fill, return v1 to R1's caller (allowed: R1 overlapped W)\n```\nOrder 2 (R1 resolves before W commits): fill v1 admitted at g0; W then deletes it;\nlater reads miss and read v2. Both orders satisfy the invariant with D1; without D1\norder 1 violates it (Step 0 trace). Missing-record creation: same schedule with absent\nDTO; guard suppresses the sentinel fill. Overlapping writes W1, W2: marker advances twice;\nany read started before either commit is suppressed. Instance change: controller\ncontract isolates old reads from the new instance. Regression proof: the six D1 scenarios.\nInteraction edge cases: no user-visible interaction (backend only). Decision gate: F1\ncovered by D1 → A; no new issues.\n\n### Section 5: Code Quality Review\nWrapper matches the existing read-through repository pattern; no duplicated behavior\n(sentinel handling stays in the adapter, coalescing in the repository). Naming: the\nmarker should be named for what it means (committed-write generation), engineering's\ncall. readProfile branches ≤ 4 after D1; writeProfile ≤ 2. No over-engineering; the\noriginal proposal was under-engineered on the race (F1). Decision gate: no new issues.\n\n### Section 6: Test Review\n```\n NEW THING | TYPE | HAPPY | FAILURE | EDGE\n readProfile hit/miss/fill | unit | miss→fill→hit | read rejects → no fill | absent DTO → sentinel\n fill guard (D1) | unit | order 2 admits fill | order 1 suppresses | overlapping writes\n writeProfile delete + marker | unit | commit→marker→delete | reject → cache kept | delete adapter failure\n eviction / byte cap / TTL | unit | within caps | oversize refused | 1000th/1001st entry\n adapter-failure fallback | integration | bypass → repo read | reinit → empty cache | failure mid-write\n concurrent-miss coalescing | integration | one store read | failure releases cohort | fresh cohort after write\n controller instance change | integration | old writer/new reader | mid-read swap | predicate parity\n suppressed-fill counter | unit | increments once | no key label present | zero when no race\n```\nAssertion check: each row maps to a retained contract or the D1 remedy; the D1 rows assert\nthe exact committed version observed by the post-write read, not a lower bound. 2am test:\nD1 scenario (a). Hostile QA: overlapping writes + absent record + instance swap in one\nschedule. Chaos: adapter throwing on every call under load. Pyramid: unit-heavy, correct.\nFlakiness: TTL and ordering tests must use the harness's controlled clock and pause/release\npoints, never timers. Load test: existing rollout stages serve as the load check.\nDecision gate: coverage already approved by author requirements and D1; no new question.\n\n### Section 7: Performance Review\nNo N+1, no new queries or indexes, no new connections. Memory: cache ≤ 16 MiB (adapter).\n**WARNING W1:** the D1 per-key marker is new in-process state; it must stay bounded (for\nexample retained only while a read for that key is in flight, or capped alongside the\ncache). Requirement recorded; data structure left to engineering; verification: a test\nthat marker state is released after reads complete. Slow paths: miss path = existing\nread + two sync cache ops; hit path sub-millisecond. Decision gate: W1 is a required\nproperty of the approved remedy, not a new choice; recorded for engineering.\n\n### Section 8: Observability & Debuggability Review\nExisting metrics (hit/miss, eviction, bytes, fallback, DB CPU, p95) plus the D1\nsuppressed-fill counter on the current dashboard, no key labels. Alerts: existing set\n(SLO breach, p95 > 120 ms 5 min, bypass > 1 min); none added per author. Debuggability:\na stale-read report 3 weeks later is reconstructible from suppressed-fill and bypass\ncounts plus the flag-stage timeline in the runbook. Decision gate: no issues found.\n\n### Section 9: Deployment & Rollout Review\nNo migration. Flag staged 10% → 50% → 100% with one healthy hour each; criteria as\nstated. Rollback: disable flag → controller drains writes, publishes bypass instance;\nseconds. Risk window: single process, no old/new code overlap. Post-deploy: first 5 min\nwatch bypass and error rate; first hour watch hit rate and p95. Smoke: one read-after-\nwrite probe on a synthetic tenant. Decision gate: no issues found.\n\n### Section 10: Long-Term Trajectory Review\nDebt: marker state needs a short ASCII comment (schedule diagram above) in the wrapper.\nPath dependency: none; the repository interface stays the replacement seam. Reversibility\n5/5. Ecosystem fit: reuses adapter, single-flight, controller. 1-year question: obvious\nif the comment and the six scenarios ship. Decision gate: no issues found.\n\n### Section 11: Design & UX Review\nSKIPPED (no UI scope).\n\n## Closing sequence\n- **Outside Voice:** codex_reviews disabled (`.gstack-section-state-IfADEH/config.yaml`).\n Codex review skipped (codex_reviews disabled). Re-enable: `gstack-config set codex_reviews enabled`.\n No native fallback. Disabled-coverage review-log write: not persisted (no gstack\n binaries; mutating commands forbidden in this run).\n- **TODO choices:** none proposed. Nothing deferred; HOLD SCOPE.\n- **Approval readiness: PASS** — rows checked: R1 (D1 → A, authority: run-rules author\n policy, scope recorded in the ledger). Plan applies only that scope. W1 is a property of\n R1's remedy, no separate approval. Declined: D1 B, C.\n\n## NOT in scope\nDeferred: none. Rejected: D1-B (weaken consistency to TTL-bounded staleness; not\nauthorizable, violates retained rule), D1-C (write-path bypass; does not fix the race).\nUnchanged non-goals: distributed caching, cross-process coherence, prewarming, consistency\nchanges, new product surfaces, new alerts/metric projects.\n\n## What already exists\nLRU adapter (limits, TTL, absent sentinel) — reused. Single-flight in repository.read —\nreused. Rollout controller (flag, instance swap, predicate parity) — reused. Dashboards,\nalerts, runbooks — reused with one added counter. Deterministic contract harness — reused.\n\n## Dream state delta\nAfter this plan: single-process cache behind the unchanged repository interface, targets\nmet, read-after-write guarantee intact. Remaining to the 12-month ideal: nothing required;\nthe seam for a different backend is preserved.\n\n## Error & Rescue Registry\nSee Section 2 map: 7 codepaths, 6 exception/branch classes, all rescued or non-error; 0 CRITICAL GAPS.\n\n## Failure Modes Registry\n```\n CODEPATH | FAILURE MODE | RESCUED? | TEST? | USER SEES? | LOGGED?\n -------------------------|------------------------------|----------|-------|-------------------|--------\n readProfile cache.get | adapter failure | Y | Y | slower response | Y (fallback)\n readProfile repo.read | store error | Y | Y | existing error | Y\n readProfile cache.set | adapter failure / oversize | Y | Y | normal | Y\n readProfile fill | stale fill after write (F1) | Y (D1) | Y | fresh value | Y (counter)\n writeProfile repo.write | rejection / rollback | Y | Y | existing error | Y\n writeProfile cache.delete| adapter failure | Y | Y | write succeeds | Y (fallback)\n startup | multi-process config | Y | Y | deploy fails | Y\n```\n7 rows, 0 CRITICAL GAPS.\n\n## Diagrams\n1. System architecture — Section 1. 2. Data flow with shadow paths — Section 4.\n3. State machine — Section 1 (per-key cache state). 4. Error flow — Section 2 map.\n5. Deployment sequence:\n```\n deploy wrapper (flag off) -> enable 10% -> 1 healthy hour -> 50% -> 1 healthy hour -> 100%\n | | breach: disable flag, runbook incident, resume criteria\n```\n6. Rollback flowchart:\n```\n alert/breach? --yes--> disable flag -> controller drains writes -> bypass instance live\n |no -> record incident -> fix -> resume at 10%\n v\n continue stage\n```\nStale diagram audit: no ASCII diagrams exist in files this plan touches (none to audit).\n\n## Implementation Tasks\nSynthesized from this review's findings. Each task derives from a specific\nfinding above. Run with Claude Code or Codex; checkbox as you ship.\n\n- [ ] **T1 (P1, human: ~1d / CC: ~25min)** — wrapper — Implement readProfile/writeProfile with the D1 fill-admission guard and per-key committed-write marker (bounded per W1), plus the suppressed-fill counter with no key labels\n - Surfaced by: Section 1 / Section 4 — F1 stale fill after completed write (ledger R1, D1 → A)\n - Files: to be determined (wrapper module in the profile-summary service)\n - Verify: D1 scenarios (a)–(f) pass on the deterministic harness; marker-release test passes\n- [ ] **T2 (P1, human: ~1d / CC: ~20min)** — tests — Add wrapper tests listed in Section 6, using controlled clock and pause/release points\n - Surfaced by: Section 6 — test diagram rows; author acceptance requirements\n - Files: to be determined (wrapper test file on the existing harness)\n - Verify: all rows green; no timer-based waits\n- [ ] **T3 (P2, human: ~1h / CC: ~5min)** — telemetry — Add suppressed-fill panel to the existing dashboard; add the ASCII schedule comment to the wrapper\n - Surfaced by: Section 8 / Section 10 — observability of the new branch, debt comment\n - Files: to be determined (dashboard config)\n - Verify: panel renders counter with no key labels; comment present\n_No new tasks from Sections 3, 5, 7 (W1 folded into T1), 9, 11._\nTask JSONL: not persisted (no gstack state root in this fixture).\n\n## Completion Summary\n```\n +====================================================================+\n | MEGA PLAN REVIEW — COMPLETION SUMMARY |\n +====================================================================+\n | Mode selected | HOLD SCOPE |\n | System Audit | skipped per run rules (fixture) |\n | Step 0 | HOLD SCOPE; R1 D1 → A fill-admission guard |\n | Section 1 (Arch) | 1 issue found (F1, resolved) |\n | Section 2 (Errors) | 7 error paths mapped, 0 GAPS |\n | Section 3 (Security)| 0 issues found, 0 High severity |\n | Section 4 (Data/UX) | 9 edge cases mapped, 0 unhandled |\n | Section 5 (Quality) | 0 issues found |\n | Section 6 (Tests) | Diagram produced, 0 gaps |\n | Section 7 (Perf) | 1 issue found (W1, requirement recorded) |\n | Section 8 (Observ) | 0 gaps found |\n | Section 9 (Deploy) | 0 risks flagged |\n | Section 10 (Future) | Reversibility: 5/5, debt items: 1 |\n | Section 11 (Design) | SKIPPED (no UI scope) |\n +--------------------------------------------------------------------+\n | NOT in scope | written (2 rejected, 0 deferred) |\n | What already exists | written |\n | Dream state delta | written |\n | Error/rescue registry| 7 rows, 0 CRITICAL GAPS |\n | Failure modes | 7 total, 0 CRITICAL GAPS |\n | TODOS.md updates | 0 items proposed |\n | Scope proposals | 0 proposed, 0 accepted (HOLD SCOPE) |\n | CEO plan | skipped by mode |\n | Outside voice | codex — disabled (config), no fallback |\n | Lake Score | 1/1 recommendations chose complete option |\n | Diagrams produced | 6 (arch, data flow, state, error, deploy, rollback) |\n | Stale diagrams found | 0 |\n | Unresolved decisions | 0 |\n +====================================================================+\n```\nUnresolved Decisions: none. Review log / decision log / dashboard read: not persisted\n(no gstack binaries; mutating commands forbidden). No durable learnings this session.\n\n## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 1 (not persisted) | CLEAR | mode: HOLD SCOPE, 0 critical gaps; 1 resolved finding (F1, D1 → A), 1 warning (W1) |\n| Outside Review | codex `/plan-ceo-review` outside voice | Independent 2nd opinion | 0 | DISABLED | disabled by config; no completed external review |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | — | not run |\n| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | not run (no UI scope) |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | not run |\n\n**OUTSIDE COVERAGE:** codex, plan-review phase, disabled (codex_reviews: disabled), no native fallback, no findings claimed.\n**VERDICT:** CEO CLEARED — eng review required\n\nNO UNRESOLVED DECISIONS\n" + } +} diff --git a/test/fixtures/detector-corpora/plan-ceo-section-loading/inventory.json b/test/fixtures/detector-corpora/plan-ceo-section-loading/inventory.json new file mode 100644 index 000000000..b2e28ea14 --- /dev/null +++ b/test/fixtures/detector-corpora/plan-ceo-section-loading/inventory.json @@ -0,0 +1,85 @@ +[ + { + "run": "37174266054", + "lane": "periodic", + "outcome": "passed", + "series_identity": "a7f8ef2d01d4716d", + "artifact": "paid-slice-6-a1", + "available": true, + "entry": "37174266054-t1-pass.json" + }, + { + "run": "37176837432", + "lane": "periodic", + "outcome": "passed", + "series_identity": "a7f8ef2d01d4716d", + "artifact": "paid-slice-6-a1", + "available": true, + "entry": null, + "reason": "Same series (a7f8ef2d01d4716d) as the 37174266054 entry and same detector path (approved ledger decision and prose finding both hold) as the 37179171083 and 37198445662 entries; omitted to bound corpus size." + }, + { + "run": "37179171083", + "lane": "periodic", + "outcome": "passed", + "series_identity": "c040a0214e4f3172", + "artifact": "paid-slice-6-a1", + "available": true, + "entry": "37179171083-t1-pass.json" + }, + { + "run": "37182865432", + "lane": "periodic", + "outcome": "failed", + "series_identity": "b56318cf9e1e2194", + "artifact": "paid-slice-6-a1", + "available": true, + "entry": "37182865432-t1-fail.json" + }, + { + "run": "37186854666", + "lane": "periodic", + "outcome": "passed", + "series_identity": "2986275878c588ef", + "artifact": "paid-slice-6-a1", + "available": true, + "entry": "37186854666-t1-pass.json" + }, + { + "run": "37193478719", + "lane": "periodic", + "outcome": "failed", + "series_identity": "2986275878c588ef", + "artifact": "paid-slice-6-a1", + "available": true, + "entry": "37193478719-t1-fail.json" + }, + { + "run": "37195203538", + "lane": "periodic", + "outcome": "passed", + "series_identity": "68761d79db01290b", + "artifact": "paid-slice-6-a1", + "available": true, + "entry": "37195203538-t1-pass.json" + }, + { + "run": "37198445662", + "lane": "periodic", + "outcome": "passed", + "series_identity": "68761d79db01290b", + "artifact": "paid-slice-6-a1", + "available": true, + "entry": "37198445662-t1-pass.json" + }, + { + "run": "37182865432", + "lane": "periodic", + "outcome": "failed", + "series_identity": "b56318cf9e1e2194", + "artifact": "paid-slice-6-a1", + "available": true, + "entry": "37182865432-t1-fail-derived-unrelated-decision.json", + "reason": "derived negative control (no genuine failing capture exists for this case)" + } +] diff --git a/test/fixtures/detector-corpora/plan-design-review-plan-mode/37174266054-t1-pass.json b/test/fixtures/detector-corpora/plan-design-review-plan-mode/37174266054-t1-pass.json new file mode 100644 index 000000000..e3b48fa71 --- /dev/null +++ b/test/fixtures/detector-corpora/plan-design-review-plan-mode/37174266054-t1-pass.json @@ -0,0 +1,22 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-design-review-plan-mode", + "detector": "test/skill-e2e-design.test.ts#plan-design-review-plan-mode.validate", + "series_identity": "c0a69951bb8e69c8", + "source": { + "run": "37174266054", + "sha": "a8ace39030dd92745b45ac16e5d733e63c936c0f", + "lane": "periodic", + "artifact": "native-captures-ci-37174266054-1-eval-slices-9", + "path": ".gstack/projects/garrytan-gstack/e2e-runs/2026-10-04-0332/plan-design-review-plan-mode.ndjson" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "Pre-repair pass on the oldest series: the summary itself names information architecture and interaction state, and the saved plan adds the design states.", + "input": { + "exit_reason": "success", + "output": "Read-back confirms the saved plan ends with the GSTACK REVIEW REPORT and its final line is the unresolved-decisions block.\n\nDesign review of plan.md is complete and saved. Overall design score moved from 2/10 to 8/10.\n\n- Classified the dashboard as an OPERATE surface and cut the gradient hero, card grid and decorative icons (hard rejections #1 and #7 cleared).\n- Added nine design decisions D1 to D9: ASCII information architecture, a full interaction state table, a journey storyboard, a no-slop UI spec, a token set as CSS variables, three responsive layouts, an accessibility spec, live-update behavior and edge-case formatting.\n- Each pass is rated before and after with a one-line path to 10. Lowest after-score is 8 on Journey and Design System.\n- Three decisions stay unresolved for a product owner: the engagement rate formula, light or dark from the use scene, and whether settings holds destructive actions.\n- Six implementation tasks, a NOT in scope list, and a TODO to run `/design-consultation` are in the plan.\n- Skipped in this run: preamble scripts, mockups, outside voices, review log and dashboard. Verdict is ISSUES OPEN, eng review still required.", + "plan_after": "# Plan: User Dashboard\n\n## Context\nBuild a user dashboard that shows account stats, recent activity, and settings.\n\nSurface mode: OPERATE (app UI). The visitor's win is finishing a task: check numbers, scan what happened, change a setting. Scanability and native expectations beat expression.\n\n## Implementation\n1. Create a dashboard page at /dashboard\n2. Show user stats (posts, followers, engagement rate) per D1 and D4\n3. Add a recent activity feed per D1, D2 and D8\n4. Add a settings panel per D1 and D2\n5. Apply the token set in D5 (replaces \"clean, modern UI with cards and icons\")\n6. Page header per D4 (replaces \"hero section with gradient background\")\n7. Implement every state in D2, the responsive layouts in D6 and the a11y spec in D7\n8. Handle the edge cases in D9\n\n## Technical Details\n- React components with Tailwind CSS; tokens from D5 exposed as CSS variables in the Tailwind theme\n- API endpoint: GET /api/dashboard\n- WebSocket for real-time activity updates, behavior per D8\n\n## Design Decisions\n\n### D1. Information architecture (Pass 1)\nHierarchy on first paint: 1) the three stats, 2) the activity feed, 3) settings. Constraint worship: only these three stats appear; no fourth metric without removing one.\n\n```\n+------------------------------------------------------------------+\n| App logo Dashboard Settings [avatar menu] | <- global nav, 56px\n+------------------------------------------------------------------+\n| h1 \"Dashboard\" \"Updated 2 min ago\" (live dot per D8) | <- page header, one line\n+------------------------------------------------------------------+\n| Posts Followers Engagement | <- stat row, 3 columns\n| 1,284 12.4K 4.8% | tabular numerals, 32px\n| +12 this wk +310 this wk -0.2 pts | delta line, 14px\n+---------------------------------------------+--------------------+\n| h2 \"Recent activity\" | h2 \"Settings\" |\n| 09:41 Ana liked \"Spring launch\" | Display name [___] |\n| 09:40 New follower: @marco | Email [___] |\n| 09:12 Comment on \"Q3 notes\" | Notifications (o ) |\n| ... [Load 20] | [Save changes] |\n+---------------------------------------------+--------------------+\nGrid: 12 columns, feed spans 8, settings spans 4.\nNavigation: /dashboard is the overview; \"Settings\" in the nav scrolls to or focuses the panel (same route).\n```\n\n### D2. Interaction state table (Pass 2)\nWhat the user sees; backend behavior is out of scope here.\n\n| FEATURE | LOADING | EMPTY | ERROR | SUCCESS | PARTIAL |\n|---|---|---|---|---|---|\n| Stat row | Three number-shaped skeletons (32px tall, 50% width), labels visible, no spinner | New account: \"0\" with delta line \"Publish your first post to start tracking\" linking to the composer | Inline per stat: \"Couldn't load\" in text-muted plus a \"Retry\" text button; other stats stay | Numbers in tabular numerals, delta colored by sign per D5 | One stat failed: that cell shows the error treatment, others render |\n| Activity feed | Five row skeletons matching row height (44px) | Headline \"Nothing here yet\", one sentence \"Activity from your posts and followers shows up here as it happens\", primary button \"Create a post\" | Headline \"Activity didn't load\", body with a plain-language reason when known, \"Try again\" button; feed keeps any rows already shown | Rows grouped by day with sticky day label | After 20 rows a \"Load 20 more\" button; while loading more, the button shows a spinner and stays in place |\n| Live updates (D8) | n/a | n/a | Status text \"Reconnecting...\" replaces \"Updated N min ago\"; after 30s \"Live updates paused. Refresh\" link | New rows slide in from the top over 160ms, then settle | Offline: rows received before disconnect stay, banner shows last sync time |\n| Settings panel | Field skeletons, Save disabled | n/a (fields always exist) | Field-level message under the field in text-danger with icon; panel-level banner only for save failures, with \"Try again\" | Toast \"Settings saved\" bottom-left, 4s, dismissible; Save returns to disabled until the next change | Some fields saved, some failed: failed fields keep their edited value and show the field error |\n\n### D3. User journey storyboard (Pass 3)\n\n| STEP | USER DOES | USER FEELS | PLAN SPECIFIES? |\n|---|---|---|---|\n| 1 | Lands on /dashboard (5-sec visceral) | Oriented: \"my numbers are right there\" | D1 stat row first, D4 header stays one line |\n| 2 | Scans the deltas | Reassured or alerted, not confused | D5 sign colors plus explicit \"+/-\" so color is not the only signal |\n| 3 | Reads the feed for 30 seconds (5-min behavioral) | Caught up without hunting | D1 day grouping, D8 live rows arrive quietly |\n| 4 | Changes a setting | Confident it took effect | D2 toast plus disabled Save state |\n| 5 | Returns weekly for a year (5-year reflective) | Trusts the dashboard as the record of their account | D9 number formatting consistency; U1 metric definition still open |\n| 6 | Opens it on a phone at a bus stop | Gets the three numbers in one glance | D6 mobile layout |\n\n### D4. Specific UI, no slop (Pass 4)\n- No hero. The page header is one 56px line: h1 \"Dashboard\" at 24px and the live status text. No gradient, no banner image, no welcome copy.\n- No cards. The stat row is a single bordered region divided by 1px rules; feed and settings are plain layout regions separated by whitespace and the grid, not boxes.\n- No icons as decoration. Icons appear only where they carry information: live status dot, field error icon, external-link marker.\n- Copy is utility language: section headings state the area (\"Recent activity\", \"Settings\"), never mood or brand lines.\n- Motion: one authored moment, new feed rows entering per D2, ease-out from a visible default. No hover scale, no pulsing dot (status dot is static; color conveys state).\n\n### D5. Token set (Pass 5)\nNo DESIGN.md exists; these tokens are the plan's contract until `/design-consultation` creates one. All exposed as CSS variables.\n- Type: display and body IBM Plex Sans (explicitly assigned as the Operate body/UI face), IBM Plex Mono for numbers in the stat row and timestamps. Scale: 14 / 16 / 20 / 24 / 32px. Body text 16px minimum.\n- Color: `--bg #FFFFFF`, `--surface #F6F7F9`, `--text #141518`, `--text-muted #5B6068`, `--rule #E2E5EA`, `--accent #0A6B4F` (one accent), `--positive #1B7F3B`, `--danger #B42318`. Secondary text on the accent surface is tinted from the accent hue, not gray. All text pairs meet 4.5:1.\n- Spacing: 4px base; section gaps 32px; more space above a heading (32px) than below it (12px).\n- Radius: 6px on inputs and buttons only; regions are square.\n- Depth: toast only, shadow `0 4px 12px rgba(20,21,24,0.12)`; no halos.\n- Browser surfaces themed from the palette: selection color (accent at 20%), focus ring 2px accent with 2px offset, scrollbar thumb `--rule`, visited links `--text-muted` with the same underline.\n\n### D6. Responsive behavior (Pass 6)\n| Viewport | Layout |\n|---|---|\n| >= 1024px | Grid per D1: stat row 3 columns, feed 8 cols, settings 4 cols |\n| 768 to 1023px | Stat row 3 columns; feed full width; settings becomes a \"Settings\" tab beside \"Activity\" above the feed region, no sidebar |\n| < 768px | Stat row becomes one horizontal strip: three numbers side by side at 24px, labels at 12px uppercase, no deltas (deltas appear on tap of a stat, expanding a 48px detail line). Feed full width, 20 rows then \"Load 20 more\". Settings moves to the nav as its own screen reached from the avatar menu |\n\nTouch targets 44px minimum on all viewports. Nothing relies on hover.\n\n### D7. Accessibility (Pass 6)\n- Landmarks: `header` (nav), `main` (stats and feed), `aside` labeled \"Settings\", `nav` labeled \"Primary\".\n- Keyboard: tab order follows D1 reading order; stat cells are not focusable unless they expand (mobile). Feed \"Load more\" and every form control reachable; Save is a real `button type=\"submit\"`.\n- Live region: the feed container is `aria-live=\"polite\"`; new rows are announced as \"New activity: ...\" at most once every 10s to avoid flooding.\n- Labels: every settings field has a visible label above it; placeholders never act as labels. Errors linked with `aria-describedby`.\n- Contrast and color: sign conveyed by \"+\"/\"-\" text and color together; status dot paired with text.\n- Reduced motion: `prefers-reduced-motion` disables the row entrance; rows appear instantly.\n\n### D8. Real-time behavior (Pass 7)\nNew rows insert at the top only when the user is scrolled within 200px of the top; otherwise a pill \"3 new\" appears at the top of the feed and scrolls up on click. Status text shows \"Updated just now\" then \"N min ago\". Reconnect and offline treatment per D2.\n\n### D9. Edge cases (Pass 7)\n- Numbers: compact above 9,999 (12.4K, 1.2M) with the exact value in a `title` tooltip and visible on focus; engagement rate to one decimal.\n- Names: display names truncate at one line with ellipsis in the feed; a 47-char name never wraps rows. Full name in `title`.\n- Timestamps: today shows time, this week shows weekday, older shows date; absolute timestamp on hover and focus.\n- First-time user: empty treatments in D2; no onboarding tour.\n\n## NOT in scope\n- Charts or trend graphs: three numbers with deltas answer the weekly question; a chart needs a metric definition (U1) first.\n- Dark theme: deferred under U2 until the use scene is known.\n- Notifications center or filtering the feed: nothing in the plan requires it; subtraction default.\n\n## What already exists\n- No DESIGN.md, no existing UI components found in the repo. Reuse only Tailwind defaults for the grid and the browser-native form controls restyled per D5.\n\n## Unresolved Decisions\n| DECISION NEEDED | IF DEFERRED, WHAT HAPPENS |\n|---|---|\n| U1. Engagement rate formula and comparison period for every delta | Engineer picks a formula; the number means something different each sprint and users stop trusting it |\n| U2. Light or dark from the use scene (who, where, under what light) | Light ships by category default; wrong if the main scene is a creator checking at night |\n| U3. Does the settings panel include destructive actions (delete account, disconnect integrations)? | Engineer adds a red button next to Save; destructive and routine actions share one region |\n\n## TODOS.md updates\n- Run `/design-consultation` to turn D5 into a DESIGN.md. Why: tokens live in one plan today and will drift. Pros: shared vocabulary. Cons: an hour of setup. Context: this review found no design system. Depends on: nothing.\n- Resolve U1 to U3 above before implementation starts.\n\n## Implementation Tasks\nSynthesized from this review's findings. Each task derives from a specific finding above. Run with Claude Code or Codex; checkbox as you ship.\n\n- [ ] **T1 (P1, human: ~1d / CC: ~20min)** — Dashboard layout — Build the D1 grid and D4 header; remove hero and cards\n - Surfaced by: Pass 1 and Pass 4 — hero and card grid hard rejections\n - Files: src/pages/Dashboard.tsx, src/components/StatRow.tsx\n - Verify: screenshot at 1280px matches D1 diagram; no gradient, no card borders\n- [ ] **T2 (P1, human: ~1d / CC: ~25min)** — All components — Implement every D2 state\n - Surfaced by: Pass 2 — no states specified\n - Files: src/components/StatRow.tsx, ActivityFeed.tsx, SettingsPanel.tsx\n - Verify: Storybook story per table cell; mock API failure and empty account\n- [ ] **T3 (P1, human: ~4h / CC: ~15min)** — Theme — Define D5 CSS variables in the Tailwind theme and theme browser surfaces\n - Surfaced by: Pass 5 — no tokens, no DESIGN.md\n - Files: tailwind.config.ts, src/styles/tokens.css\n - Verify: contrast check on every text pair >= 4.5:1; focus ring visible on all controls\n- [ ] **T4 (P1, human: ~1d / CC: ~20min)** — Layout — Implement D6 breakpoints and D7 a11y\n - Surfaced by: Pass 6 — no responsive or a11y spec\n - Files: src/pages/Dashboard.tsx, src/components/*\n - Verify: axe clean; keyboard-only walkthrough; 375px, 768px, 1280px screenshots\n- [ ] **T5 (P2, human: ~4h / CC: ~15min)** — ActivityFeed — Implement D8 live insert, \"N new\" pill, reconnect states\n - Surfaced by: Pass 7 — real-time behavior unspecified\n - Files: src/components/ActivityFeed.tsx, src/hooks/useActivitySocket.ts\n - Verify: kill the socket in devtools; pill appears when scrolled down\n- [ ] **T6 (P2, human: ~2h / CC: ~10min)** — Formatting utils — Implement D9 number, name and time formatting\n - Surfaced by: Pass 7 — edge cases\n - Files: src/lib/format.ts\n - Verify: unit tests for 9,999 / 10,000 / 1.2M, 47-char name, each timestamp bucket\n\n_No new tasks from Pass 3; the storyboard records requirements owned by other tasks._\n\n## Design Review Notes\n\nSystem Audit: no DESIGN.md (gap), no CLAUDE.md or TODOS.md, no prior reviews. UI scope: one page, three regions, live updates. Mockups: designer not available in this run, text-only review. Outside voices: skipped (non-interactive run).\n\nStep 0: initial design completeness 2/10. The plan names features and a stack but nothing the user sees. A 10 for this plan is D1 through D9 present with every state, viewport and a11y behavior specified.\n\n| Pass | Before | After | Rationale and what makes it a 10 |\n|---|---|---|---|\n| 1 Info Arch | 2 | 9 | Before: six bullets, no order, no diagram. After: D1. A 10 needs U3 settled so the settings region's content is final. |\n| 2 States | 0 | 9 | Before: none. After: D2 covers every feature and state. A 10 needs the settings field list fixed (U3). |\n| 3 Journey | 1 | 8 | Before: no user in the plan. After: D3 across all three horizons. A 10 needs U1 so the reflective horizon rests on a trusted metric. |\n| 4 AI Slop | 1 | 9 | Hard rejections hit before: #1 card grid as first impression, #7 app UI of stacked cards; blacklist #1 gradient hero, #3 decorative icons. Litmus before: 1 NO, 2 NO, 3 NO, 4 NO, 5 NO, 6 NOT SPEC'D, 7 NO. After D4 and D5: all cleared, litmus 1 to 5 and 7 YES, 6 YES (one authored moment). A 10 needs U2 so the look comes from the scene, not the category. |\n| 5 Design Sys | 1 | 8 | Before: \"Tailwind\" only. After: D5 tokens as CSS variables. A 10 needs DESIGN.md (TODO) so tokens are shared, not plan-local. |\n| 6 Responsive | 0 | 9 | Before: none. After: D6 and D7. A 10 needs a real-device pass on the mobile stat strip after build. |\n| 7 Decisions | unscored | 9 resolved, 3 deferred | D1 to D9 added; U1 to U3 open. |\n\nDecisions D1 to D9 were applied under the user's directive for this review run to edit the plan with the missing design decisions; no per-issue question was possible in non-interactive mode. U1 to U3 need a product owner and stay pending.\n\n```\n +====================================================================+\n | DESIGN PLAN REVIEW — COMPLETION SUMMARY |\n +====================================================================+\n | System Audit | no DESIGN.md; UI scope: 1 page, 3 regions |\n | Step 0 | 2/10, all 7 dimensions reviewed |\n | Pass 1 (Info Arch) | 2/10 → 9/10 after fixes |\n | Pass 2 (States) | 0/10 → 9/10 after fixes |\n | Pass 3 (Journey) | 1/10 → 8/10 after fixes |\n | Pass 4 (AI Slop) | 1/10 → 9/10 after fixes |\n | Pass 5 (Design Sys) | 1/10 → 8/10 after fixes |\n | Pass 6 (Responsive) | 0/10 → 9/10 after fixes |\n | Pass 7 (Decisions) | 9 resolved, 3 deferred |\n +--------------------------------------------------------------------+\n | NOT in scope | written (3 items) |\n | What already exists | written |\n | TODOS.md updates | 2 items proposed |\n | Approved Mockups | 0 generated, 0 approved |\n | Decisions made | 9 added to plan |\n | Decisions deferred | 3 (U1, U2, U3) |\n | Overall design score | 2/10 → 8/10 |\n +====================================================================+\n```\n\n## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 0 | not run | — |\n| Outside Review | none recorded | Independent 2nd opinion | 0 | skipped | — |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | not run | — |\n| Design Review | `/plan-design-review` | UI/UX gaps | 1 | issues_open | score: 2/10 → 8/10, 9 decisions |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | not run | — |\n\n**OUTSIDE COVERAGE:** Codex and native subagent design voices skipped in this non-interactive run; no outside coverage recorded.\n\n**VERDICT:** Design Review ISSUES OPEN (3 deferred decisions); no review CLEAR; eng review required.\n\n**UNRESOLVED DECISIONS:**\n- U1. Engagement rate formula and delta comparison period\n- U2. Light or dark theme from the use scene\n- U3. Whether the settings panel includes destructive actions\n" + } +} diff --git a/test/fixtures/detector-corpora/plan-design-review-plan-mode/37179171083-t1-derived-unrated-fail.json b/test/fixtures/detector-corpora/plan-design-review-plan-mode/37179171083-t1-derived-unrated-fail.json new file mode 100644 index 000000000..e22774e8c --- /dev/null +++ b/test/fixtures/detector-corpora/plan-design-review-plan-mode/37179171083-t1-derived-unrated-fail.json @@ -0,0 +1,23 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-design-review-plan-mode", + "detector": "test/skill-e2e-design.test.ts#plan-design-review-plan-mode.validate", + "series_identity": "c0a69951bb8e69c8", + "source": { + "run": "37179171083", + "sha": "db91607a7fd9419592eb63e036149920304e32b5", + "lane": "periodic", + "artifact": "native-captures-ci-37179171083-1-eval-slices-9", + "path": ".gstack/projects/garrytan-gstack/e2e-runs/2026-10-04-0513/plan-design-review-plan-mode.ndjson" + }, + "census_outcome": "failed", + "census_error": "expect(received).toBe(expected)", + "expected": "refuse", + "justification": "Plan-term credit needs a rated summary (44ef18c); an unrated summary with none of the design terms is still refused.", + "derived": "plan_after of 37179171083 kept; the summary's two \"N/10\" scores rewritten to words so it carries no rating", + "input": { + "exit_reason": "success", + "output": "Read-back verified: the report is the last section and ends on the unresolved-decisions block.\n\nDesign review of plan.md is saved. Overall design score zero → eight, with 6 decisions added and 4 left for you.\n\n- **Classified the surface as OPERATE** and cut the gradient hero. The page now opens on a 56px utility header and three tabular stat numerals (D1, with desktop and 375px ASCII diagrams).\n- **Replaced \"cards and icons\"** with a cardless hairline layout, functional 16px monochrome icons only, left-aligned, one motion moment (D4).\n- **Added the full state table** for stats, feed, live updates, settings, and page shell, including warm empty states and a non-shifting \"N new\" pill for WebSocket inserts (D2).\n- **Added the journey storyboard** across visceral, behavioral, and reflective horizons (D3).\n- **Specified inline tokens** since no DESIGN.md exists: IBM Plex Sans and Mono, 16/20/32 type scale, CSS color variables, 4px radius, no shadows (D5).\n- **Specified responsive and a11y behavior** per viewport, landmarks, skip link, 44px targets, visible labels, reduced motion (D6).\n- **Unresolved for you:** accent hue and typeface license, settings field inventory, engagement rate definition, and whether the API paginates activity.\n- Decisions were applied without AskUserQuestion under your non-interactive instruction. Mockups, outside voices, review log, and the JSONL task artifact were not run. Eng review is still required.", + "plan_after": "# Plan: User Dashboard\n\n## Context\nBuild a user dashboard that shows account stats, recent activity, and settings.\n\n## Implementation\n1. Create a dashboard page at /dashboard\n2. Show user stats (posts, followers, engagement rate)\n3. Add a recent activity feed\n4. Add a settings panel\n5. Visual treatment per D4 and D5 below (replaces \"clean, modern UI with cards and icons\")\n6. Compact page header per D1 below (replaces \"hero section with a gradient background\")\n\n## Technical Details\n- React components with Tailwind CSS (tokens from D5 exposed as CSS variables in the Tailwind theme)\n- API endpoint: GET /api/dashboard\n- WebSocket for real-time activity updates (insertion behavior per D2, row \"Live updates\")\n\n## Design Decisions\n\nSurface classification: **OPERATE** (app UI). The visitor's win is finishing a check-in on their account, so scanability and native expectations beat expression. App UI rules apply; a hero is a Persuade pattern and does not belong here.\n\n### D1. Information architecture\nHierarchy, in order of what the user sees: (1) three stat numbers, (2) activity feed, (3) settings, reached from the header rather than competing on the page.\n\n```\n/dashboard (>=1024px) /dashboard (375px)\n+------------------------------------------+ +------------------------+\n| Logo Dashboard [Settings] [Avatar] | | Logo [Settings] [Av]|\n+------------------------------------------+ +------------------------+\n| h1 Your account Updated 2 min ago | | h1 Your account |\n| 128 4,210 4.2% | | Updated 2 min ago |\n| posts followers engagement (30d) | | 128 4,210 4.2% |\n+---------------------------+--------------+ | posts follow. engage. |\n| Recent activity (main) | Settings | +------------------------+\n| [N new] pill when live | (aside, 4/12)| | Recent activity |\n| 09:41 @ana followed you | Display name | | [N new] |\n| 09:12 Post \"...\" liked | Email | | 09:41 @ana followed |\n| ... | Notifications| | 09:12 Post liked |\n| [Load older] | [Save] | | ... |\n+---------------------------+--------------+ | [Load older] |\n +------------------------+\nNav flow: any page -> header \"Dashboard\" -> /dashboard\n /dashboard -> header \"Settings\" -> /dashboard/settings (375-1023px: own route;\n >=1024px: same content rendered in the aside, link scrolls/focuses it)\n```\n\nConstraint worship: if only three things fit, they are the three stat numbers. The header is one row of utility chrome, 56px tall, no imagery, no gradient. The \"Updated 2 min ago\" timestamp is the only status text in the header.\n\n### D2. Interaction states\nWhat the user SEES per feature. Empty states carry warmth, one primary action, and context.\n\n| FEATURE | LOADING | EMPTY | ERROR | SUCCESS | PARTIAL |\n|---|---|---|---|---|---|\n| Stats strip | Three 32px skeleton bars at numeral width; labels render immediately | \"0\" numerals with label; engagement shows \"—\" plus \"Publish a post to start measuring\" | Numerals replaced by \"—\", one line \"Stats unavailable. Retry\" link (inline, not toast) | Tabular numerals, no animation on first paint | Any one stat missing shows \"—\" for that stat only; other two render |\n| Activity feed | 5 skeleton rows, 56px each, no shimmer when reduced-motion | Heading \"Nothing here yet\", body \"Activity from your posts and followers lands here\", primary button \"Write your first post\" | Row-list area shows \"Couldn't load activity\" with \"Try again\" button; stats and settings unaffected | Rows: time (tabular, left), 16px functional icon, one-line text, 47-char names truncated with ellipsis and full name in title | End of list: \"That's everything from the last 30 days\"; \"Load older\" button while more exist |\n| Live updates (WebSocket) | Nothing visible; no \"connecting\" badge | n/a | Inline notice above feed: \"Live updates paused. Reconnecting…\" then \"Live updates off. Refresh to update\" after 60s | New items never shift read content; a \"3 new\" pill appears at feed top, click inserts them and moves focus to first new row | Reconnect replays missed items through the same pill |\n| Settings panel | Field skeletons keep label text visible | n/a (fields always exist) | Field-level message below input, red text 16px, tied via aria-describedby; Save stays enabled | \"Saved\" text beside Save for 4s, no toast, no page jump | Unsaved changes: Save becomes primary, \"Unsaved changes\" text appears; navigating away prompts |\n| Page shell | Header renders synchronously; main region skeletons | n/a | 401: redirect to /login. 500: full-main message \"Dashboard unavailable\" with Retry; header stays | Normal | Offline: browser offline banner above header, content stays readable |\n\n### D3. User journey storyboard\n\n| STEP | USER DOES | USER FEELS | PLAN SPECIFIES? |\n|---|---|---|---|\n| 1 | Opens /dashboard on phone in the morning | Curious, 5-sec visceral check \"how did I do\" | D1: three numbers first, nothing above them but a 56px header |\n| 2 | Reads stat numbers | Oriented in under 3 seconds | D5: 32px tabular numerals, labels 16px muted |\n| 3 | Scans feed | Reassured by recognizable names and times | D2 feed success row spec; D5 icon rules |\n| 4 | Sees \"3 new\" pill, taps | In control, not interrupted | D2 Live updates row |\n| 5 | Hits error or loss of connection | Informed, not alarmed | D2 error cells: inline, scoped, actionable |\n| 6 | First-time user with zero posts | Welcomed, knows the next action | D2 empty cells: one primary action |\n| 7 | Changes a setting | Confident it took | D2 settings success/partial |\n| 8 | Returns daily for a year (5-year reflective) | Trusts the numbers | D4: no decoration to date it; D5 tokens hold across themes |\n\n### D4. Specific UI, no slop\n- Cardless. Stats, feed, and settings are regions separated by 1px hairline dividers and whitespace, not boxes. Cards appear only if a row becomes the interaction itself (none planned).\n- No hero, no gradient, no decorative blobs, no icons in colored circles. Icons are functional only: one 16px monochrome glyph per activity type (follow, like, comment, mention), aria-hidden, always paired with text.\n- Copy is utility language: \"Recent activity\", \"Your account\", \"Load older\". No \"Welcome to\", no mood statements.\n- Layout left-aligned throughout; numerals and times tabular. Nothing centered except nothing.\n- Motion: one authored moment, the \"N new\" pill fading in over 160ms ease-out. Skeletons do not shimmer. Honors prefers-reduced-motion.\n- Light or dark from the use scene: a creator checking numbers on a phone in daylight. Light is default; dark follows prefers-color-scheme with the same tokens.\n- Browser surfaces themed from D5: selection, caret, focus ring, scrollbar thumb, underline offset.\n\n### D5. Tokens (no DESIGN.md exists; these are this plan's inline design system)\n- Type: IBM Plex Sans for UI and body (explicitly assigned, Operate surface); IBM Plex Mono for numerals and times with `font-variant-numeric: tabular-nums`. Scale 16 / 20 / 32px; body never under 16px. h1 20px/600, stat numerals 32px/500, labels 16px muted.\n- Color as CSS variables: `--bg`, `--fg`, `--fg-muted`, `--hairline`, `--accent`, `--danger`, `--focus`. Body contrast ≥4.5:1, muted text ≥4.5:1. Muted text on any tinted surface is tinted from that hue, never gray. Accent hue is unresolved (U1); purple/indigo excluded.\n- Spacing scale: 4, 8, 12, 16, 24, 32, 48px. More space above a heading (24px) than below (8px).\n- Radius: 4px on inputs and buttons only. No shadows; depth comes from hairlines. Focus ring 2px `--focus`, offset 2px.\n- Links: underlined, visited color differs from unvisited.\n\n### D6. Responsive and accessibility\n- 375px: one column; stats in one row at 32px (fits at 3 × ~110px); settings is its own route; header 56px sticky; touch targets ≥44px; \"Load older\" full width.\n- 768px: one column, max-width 720px, stats row gains labels on the same line as numerals; settings still its own route.\n- ≥1024px: 12-column grid, max-width 1200px, feed 8 columns, settings aside 4 columns, sticky aside.\n- Landmarks: `header`, `nav`, `main` (feed), `aside` (settings). One h1. Skip link to main.\n- Feed is a `ul` with `aria-live=\"polite\"` on the \"N new\" pill only, not the list. Each row is a link with the full text as its name; `time` elements carry `datetime`.\n- Settings: visible labels above fields, never placeholder-as-label; errors tied via aria-describedby; Save reachable by Tab in order.\n- Keyboard: all actions on Tab order, visible focus per D5, Escape dismisses the unsaved-changes prompt.\n\n## NOT in scope\n- Charts or sparklines for stats: numbers alone meet the 3-second scan; add only once a trend question is validated.\n- Notification center or unread badges: the feed and \"N new\" pill cover this.\n- Theming beyond light/dark system preference: no user theme picker.\n- Avatar upload inside settings: separate upload flow with its own states.\n\n## What already exists\n- No DESIGN.md referenced by the plan; D5 stands in until `/design-consultation` produces one.\n- Tailwind is already the styling layer; D5 tokens map to theme extensions, no new dependency.\n- Existing auth redirect to /login is assumed and reused for the 401 state (verify in app shell).\n\n## TODOS.md updates\nApproved fixes are implementation tasks below, not TODOs. One deferred item proposed:\n- **What:** Create DESIGN.md from D5 tokens via `/design-consultation`. **Why:** Three future surfaces will otherwise re-derive tokens. **Pros:** One vocabulary for plans and code. **Cons:** ~1h of decisions including accent hue (U1). **Context:** This plan carries tokens inline because no system existed on 2026-10-04. **Depends on:** U1. Options: A) Add to TODOS.md (recommended) B) Skip C) Build now. Pending user choice.\n\n## Implementation Tasks\nSynthesized from this review's findings. Each task derives from a specific finding above. Run with Claude Code or Codex; checkbox as you ship.\n\n- [ ] **T1 (P1, human: ~1d / CC: ~20min)** — Dashboard shell — Build header, grid, and landmarks per D1 and D6\n - Surfaced by: Pass 1 — no hierarchy, hero on an Operate surface\n - Files: src/pages/Dashboard.tsx, src/components/AppHeader.tsx\n - Verify: axe has 0 landmark/heading violations; 375/768/1024 screenshots match D1 diagram\n- [ ] **T2 (P1, human: ~1d / CC: ~20min)** — Stats strip, feed, settings — Implement every cell of the D2 state table\n - Surfaced by: Pass 2 — zero states specified\n - Files: src/components/StatsStrip.tsx, ActivityFeed.tsx, SettingsPanel.tsx\n - Verify: Storybook story per state cell; mock API for empty, 500, partial\n- [ ] **T3 (P1, human: ~4h / CC: ~10min)** — Live updates — \"N new\" pill and reconnect notice per D2\n - Surfaced by: Pass 2 and Pass 3 step 4\n - Files: src/hooks/useActivitySocket.ts, ActivityFeed.tsx\n - Verify: inject 3 socket events while scrolled; no layout shift; focus lands on first new row\n- [ ] **T4 (P1, human: ~4h / CC: ~10min)** — Tokens — Define D5 CSS variables in Tailwind theme, theme browser surfaces per D4\n - Surfaced by: Pass 4 and Pass 5\n - Files: tailwind.config.ts, src/styles/tokens.css\n - Verify: contrast check on all text pairs ≥4.5:1; no hex literals in components\n- [ ] **T5 (P2, human: ~4h / CC: ~10min)** — Responsive and a11y — Breakpoint behavior and keyboard paths per D6\n - Surfaced by: Pass 6\n - Files: Dashboard.tsx, SettingsPanel.tsx, routes\n - Verify: keyboard-only walkthrough; touch targets ≥44px at 375px; reduced-motion disables pill fade\n- _No new tasks from Pass 7 until U1–U4 are resolved._\n\n## Completion Summary\n\nDecisions D1–D6 were applied under this run's non-interactive delivery instruction (no AskUserQuestion available); each is reversible by editing its section. Mockups were not generated (designer not run). JSONL task artifact and review log were not written because no gstack binaries were run this pass.\n\n```\n +====================================================================+\n | DESIGN PLAN REVIEW — COMPLETION SUMMARY |\n +====================================================================+\n | System Audit | No DESIGN.md; UI scope: /dashboard page, |\n | | stats, feed, settings, realtime |\n | Step 0 | 2/10 initial; focus: all 7 passes |\n | Pass 1 (Info Arch) | 2/10 → 9/10 after fixes |\n | Pass 2 (States) | 0/10 → 9/10 after fixes |\n | Pass 3 (Journey) | 1/10 → 8/10 after fixes |\n | Pass 4 (AI Slop) | 1/10 → 8/10 after fixes |\n | Pass 5 (Design Sys) | 1/10 → 8/10 after fixes |\n | Pass 6 (Responsive) | 0/10 → 9/10 after fixes |\n | Pass 7 (Decisions) | 6 resolved, 4 deferred |\n +--------------------------------------------------------------------+\n | NOT in scope | written (4 items) |\n | What already exists | written |\n | TODOS.md updates | 1 item proposed |\n | Approved Mockups | 0 generated, 0 approved |\n | Decisions made | 6 added to plan |\n | Decisions deferred | 4 (listed below) |\n | Overall design score | 0/10 → 8/10 |\n +====================================================================+\n```\n\nScore rationales and what a 10 needs:\n- **Pass 1, 2→9.** Was: six bullets, no order, hero first. Now: D1 ordered hierarchy and diagram. 10 needs settings field inventory (U2) so the aside hierarchy is real.\n- **Pass 2, 0→9.** Was: no states. Now: D2 covers all five states for five features. 10 needs the pagination contract (U4) to confirm the PARTIAL feed cell.\n- **Pass 3, 1→8.** Was: no user mentioned. Now: D3 covers visceral, behavioral, and reflective horizons. 10 needs brand voice (U1) for the empty-state copy to feel like this product.\n- **Pass 4, 1→8.** Hard rejections hit on input: #7 app UI of stacked cards, plus blacklist #1 gradient and #3 icons as decoration. Litmus on D1–D5: brand unmistakable YES (logo and h1 in header), one anchor YES (stats strip), scannable by headings YES, one job per section YES, cards necessary NO so none used, motion improves hierarchy YES (one pill moment), premium without shadows YES (none specified). All hard rejections cleared. 10 needs the accent hue chosen (U1), since an unchosen color invites the purple default back.\n- **Pass 5, 1→8.** No DESIGN.md; D5 supplies tokens inline. 10 needs DESIGN.md so tokens are cited by path, per the TODO above.\n- **Pass 6, 0→9.** Was: nothing. Now: D6 per-viewport layouts and a11y. 10 needs the 768px settings decision confirmed once U2 fixes field count.\n\nPass 7 register:\n\n```\n DECISION NEEDED | IF DEFERRED, WHAT HAPPENS\n ----------------------------------------------|--------------------------------------------\n U1 Accent hue and type license for Plex | Engineer picks Tailwind indigo; blacklist #1\n U2 Which settings fields exist in the panel | Panel ships with display name only, or bloats\n U3 Engagement rate definition and period | Label says \"engagement\"; users distrust number\n U4 Does /api/dashboard paginate activity? | Feed loads everything; \"Load older\" is fake\n```\n\nPlan is not design-complete: four decisions above are deferred to the user. Passes 4 and 5 stay at 8 until U1 and DESIGN.md land. Run /design-review after implementation for visual QA.\n\n## Unresolved Decisions\n- U1: accent hue and typeface licensing (blocks Pass 4 and 5 reaching 10).\n- U2: settings panel field inventory (D1 aside and 768px behavior depend on it).\n- U3: engagement rate formula and window (D1 label currently assumes 30 days).\n- U4: activity pagination in GET /api/dashboard (D2 PARTIAL cell is conditional).\n\n## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 0 | not run | — |\n| Outside Review | not run (Codex not invoked this pass) | Independent 2nd opinion | 0 | skipped | — |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | not run | — |\n| Design Review | `/plan-design-review` | UI/UX gaps | 1 | issues_open | score: 0/10 → 8/10, 6 decisions |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | not run | — |\n\nPrior review log was not read this pass (gstack-review-read not run); prior run counts shown as 0 are unverified.\n\n**OUTSIDE COVERAGE:** provider codex, phase design, skipped (non-interactive run, no outside process started), no findings. Missing coverage, not clean.\n\n**VERDICT:** No reviews CLEAR. Design review ISSUES OPEN (4 unresolved). Eng review required.\n\n**UNRESOLVED DECISIONS:**\n- U1: accent hue and typeface licensing\n- U2: settings panel field inventory\n- U3: engagement rate formula and window\n- U4: activity pagination contract in GET /api/dashboard\n" + } +} diff --git a/test/fixtures/detector-corpora/plan-design-review-plan-mode/37179171083-t1-derived-unsaved-fail.json b/test/fixtures/detector-corpora/plan-design-review-plan-mode/37179171083-t1-derived-unsaved-fail.json new file mode 100644 index 000000000..77b3a200c --- /dev/null +++ b/test/fixtures/detector-corpora/plan-design-review-plan-mode/37179171083-t1-derived-unsaved-fail.json @@ -0,0 +1,23 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-design-review-plan-mode", + "detector": "test/skill-e2e-design.test.ts#plan-design-review-plan-mode.validate", + "series_identity": "c0a69951bb8e69c8", + "source": { + "run": "37179171083", + "sha": "db91607a7fd9419592eb63e036149920304e32b5", + "lane": "periodic", + "artifact": "native-captures-ci-37179171083-1-eval-slices-9", + "path": ".gstack/projects/garrytan-gstack/e2e-runs/2026-10-04-0513/plan-design-review-plan-mode.ndjson" + }, + "census_outcome": "failed", + "census_error": "expect(received).toBe(expected)", + "expected": "refuse", + "justification": "A summary that claims a saved review must not pass when plan.md is unchanged; the plan edit and its design additions are the contract.", + "derived": "output of 37179171083 kept; plan_after set to the untouched seed, as if the Write never landed (null = plan.md left as the seed)", + "input": { + "exit_reason": "success", + "output": "Read-back verified: the report is the last section and ends on the unresolved-decisions block.\n\nDesign review of plan.md is saved. Overall design score 0/10 → 8/10, with 6 decisions added and 4 left for you.\n\n- **Classified the surface as OPERATE** and cut the gradient hero. The page now opens on a 56px utility header and three tabular stat numerals (D1, with desktop and 375px ASCII diagrams).\n- **Replaced \"cards and icons\"** with a cardless hairline layout, functional 16px monochrome icons only, left-aligned, one motion moment (D4).\n- **Added the full state table** for stats, feed, live updates, settings, and page shell, including warm empty states and a non-shifting \"N new\" pill for WebSocket inserts (D2).\n- **Added the journey storyboard** across visceral, behavioral, and reflective horizons (D3).\n- **Specified inline tokens** since no DESIGN.md exists: IBM Plex Sans and Mono, 16/20/32 type scale, CSS color variables, 4px radius, no shadows (D5).\n- **Specified responsive and a11y behavior** per viewport, landmarks, skip link, 44px targets, visible labels, reduced motion (D6).\n- **Unresolved for you:** accent hue and typeface license, settings field inventory, engagement rate definition, and whether the API paginates activity.\n- Decisions were applied without AskUserQuestion under your non-interactive instruction. Mockups, outside voices, review log, and the JSONL task artifact were not run. Eng review is still required.", + "plan_after": null + } +} diff --git a/test/fixtures/detector-corpora/plan-design-review-plan-mode/37179171083-t1-fail.json b/test/fixtures/detector-corpora/plan-design-review-plan-mode/37179171083-t1-fail.json new file mode 100644 index 000000000..67b6a47dc --- /dev/null +++ b/test/fixtures/detector-corpora/plan-design-review-plan-mode/37179171083-t1-fail.json @@ -0,0 +1,22 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-design-review-plan-mode", + "detector": "test/skill-e2e-design.test.ts#plan-design-review-plan-mode.validate", + "series_identity": "c0a69951bb8e69c8", + "source": { + "run": "37179171083", + "sha": "db91607a7fd9419592eb63e036149920304e32b5", + "lane": "periodic", + "artifact": "native-captures-ci-37179171083-1-eval-slices-9", + "path": ".gstack/projects/garrytan-gstack/e2e-runs/2026-10-04-0513/plan-design-review-plan-mode.ndjson" + }, + "census_outcome": "failed", + "census_error": "expect(received).toBe(expected)", + "expected": "pass", + "justification": "Fixed false red: the rated summary left the design terms to the plan it wrote (all four added there); d603a63 credits terms the review added to plan.md and 44ef18c limits that credit to a rated summary.", + "input": { + "exit_reason": "success", + "output": "Read-back verified: the report is the last section and ends on the unresolved-decisions block.\n\nDesign review of plan.md is saved. Overall design score 0/10 → 8/10, with 6 decisions added and 4 left for you.\n\n- **Classified the surface as OPERATE** and cut the gradient hero. The page now opens on a 56px utility header and three tabular stat numerals (D1, with desktop and 375px ASCII diagrams).\n- **Replaced \"cards and icons\"** with a cardless hairline layout, functional 16px monochrome icons only, left-aligned, one motion moment (D4).\n- **Added the full state table** for stats, feed, live updates, settings, and page shell, including warm empty states and a non-shifting \"N new\" pill for WebSocket inserts (D2).\n- **Added the journey storyboard** across visceral, behavioral, and reflective horizons (D3).\n- **Specified inline tokens** since no DESIGN.md exists: IBM Plex Sans and Mono, 16/20/32 type scale, CSS color variables, 4px radius, no shadows (D5).\n- **Specified responsive and a11y behavior** per viewport, landmarks, skip link, 44px targets, visible labels, reduced motion (D6).\n- **Unresolved for you:** accent hue and typeface license, settings field inventory, engagement rate definition, and whether the API paginates activity.\n- Decisions were applied without AskUserQuestion under your non-interactive instruction. Mockups, outside voices, review log, and the JSONL task artifact were not run. Eng review is still required.", + "plan_after": "# Plan: User Dashboard\n\n## Context\nBuild a user dashboard that shows account stats, recent activity, and settings.\n\n## Implementation\n1. Create a dashboard page at /dashboard\n2. Show user stats (posts, followers, engagement rate)\n3. Add a recent activity feed\n4. Add a settings panel\n5. Visual treatment per D4 and D5 below (replaces \"clean, modern UI with cards and icons\")\n6. Compact page header per D1 below (replaces \"hero section with a gradient background\")\n\n## Technical Details\n- React components with Tailwind CSS (tokens from D5 exposed as CSS variables in the Tailwind theme)\n- API endpoint: GET /api/dashboard\n- WebSocket for real-time activity updates (insertion behavior per D2, row \"Live updates\")\n\n## Design Decisions\n\nSurface classification: **OPERATE** (app UI). The visitor's win is finishing a check-in on their account, so scanability and native expectations beat expression. App UI rules apply; a hero is a Persuade pattern and does not belong here.\n\n### D1. Information architecture\nHierarchy, in order of what the user sees: (1) three stat numbers, (2) activity feed, (3) settings, reached from the header rather than competing on the page.\n\n```\n/dashboard (>=1024px) /dashboard (375px)\n+------------------------------------------+ +------------------------+\n| Logo Dashboard [Settings] [Avatar] | | Logo [Settings] [Av]|\n+------------------------------------------+ +------------------------+\n| h1 Your account Updated 2 min ago | | h1 Your account |\n| 128 4,210 4.2% | | Updated 2 min ago |\n| posts followers engagement (30d) | | 128 4,210 4.2% |\n+---------------------------+--------------+ | posts follow. engage. |\n| Recent activity (main) | Settings | +------------------------+\n| [N new] pill when live | (aside, 4/12)| | Recent activity |\n| 09:41 @ana followed you | Display name | | [N new] |\n| 09:12 Post \"...\" liked | Email | | 09:41 @ana followed |\n| ... | Notifications| | 09:12 Post liked |\n| [Load older] | [Save] | | ... |\n+---------------------------+--------------+ | [Load older] |\n +------------------------+\nNav flow: any page -> header \"Dashboard\" -> /dashboard\n /dashboard -> header \"Settings\" -> /dashboard/settings (375-1023px: own route;\n >=1024px: same content rendered in the aside, link scrolls/focuses it)\n```\n\nConstraint worship: if only three things fit, they are the three stat numbers. The header is one row of utility chrome, 56px tall, no imagery, no gradient. The \"Updated 2 min ago\" timestamp is the only status text in the header.\n\n### D2. Interaction states\nWhat the user SEES per feature. Empty states carry warmth, one primary action, and context.\n\n| FEATURE | LOADING | EMPTY | ERROR | SUCCESS | PARTIAL |\n|---|---|---|---|---|---|\n| Stats strip | Three 32px skeleton bars at numeral width; labels render immediately | \"0\" numerals with label; engagement shows \"—\" plus \"Publish a post to start measuring\" | Numerals replaced by \"—\", one line \"Stats unavailable. Retry\" link (inline, not toast) | Tabular numerals, no animation on first paint | Any one stat missing shows \"—\" for that stat only; other two render |\n| Activity feed | 5 skeleton rows, 56px each, no shimmer when reduced-motion | Heading \"Nothing here yet\", body \"Activity from your posts and followers lands here\", primary button \"Write your first post\" | Row-list area shows \"Couldn't load activity\" with \"Try again\" button; stats and settings unaffected | Rows: time (tabular, left), 16px functional icon, one-line text, 47-char names truncated with ellipsis and full name in title | End of list: \"That's everything from the last 30 days\"; \"Load older\" button while more exist |\n| Live updates (WebSocket) | Nothing visible; no \"connecting\" badge | n/a | Inline notice above feed: \"Live updates paused. Reconnecting…\" then \"Live updates off. Refresh to update\" after 60s | New items never shift read content; a \"3 new\" pill appears at feed top, click inserts them and moves focus to first new row | Reconnect replays missed items through the same pill |\n| Settings panel | Field skeletons keep label text visible | n/a (fields always exist) | Field-level message below input, red text 16px, tied via aria-describedby; Save stays enabled | \"Saved\" text beside Save for 4s, no toast, no page jump | Unsaved changes: Save becomes primary, \"Unsaved changes\" text appears; navigating away prompts |\n| Page shell | Header renders synchronously; main region skeletons | n/a | 401: redirect to /login. 500: full-main message \"Dashboard unavailable\" with Retry; header stays | Normal | Offline: browser offline banner above header, content stays readable |\n\n### D3. User journey storyboard\n\n| STEP | USER DOES | USER FEELS | PLAN SPECIFIES? |\n|---|---|---|---|\n| 1 | Opens /dashboard on phone in the morning | Curious, 5-sec visceral check \"how did I do\" | D1: three numbers first, nothing above them but a 56px header |\n| 2 | Reads stat numbers | Oriented in under 3 seconds | D5: 32px tabular numerals, labels 16px muted |\n| 3 | Scans feed | Reassured by recognizable names and times | D2 feed success row spec; D5 icon rules |\n| 4 | Sees \"3 new\" pill, taps | In control, not interrupted | D2 Live updates row |\n| 5 | Hits error or loss of connection | Informed, not alarmed | D2 error cells: inline, scoped, actionable |\n| 6 | First-time user with zero posts | Welcomed, knows the next action | D2 empty cells: one primary action |\n| 7 | Changes a setting | Confident it took | D2 settings success/partial |\n| 8 | Returns daily for a year (5-year reflective) | Trusts the numbers | D4: no decoration to date it; D5 tokens hold across themes |\n\n### D4. Specific UI, no slop\n- Cardless. Stats, feed, and settings are regions separated by 1px hairline dividers and whitespace, not boxes. Cards appear only if a row becomes the interaction itself (none planned).\n- No hero, no gradient, no decorative blobs, no icons in colored circles. Icons are functional only: one 16px monochrome glyph per activity type (follow, like, comment, mention), aria-hidden, always paired with text.\n- Copy is utility language: \"Recent activity\", \"Your account\", \"Load older\". No \"Welcome to\", no mood statements.\n- Layout left-aligned throughout; numerals and times tabular. Nothing centered except nothing.\n- Motion: one authored moment, the \"N new\" pill fading in over 160ms ease-out. Skeletons do not shimmer. Honors prefers-reduced-motion.\n- Light or dark from the use scene: a creator checking numbers on a phone in daylight. Light is default; dark follows prefers-color-scheme with the same tokens.\n- Browser surfaces themed from D5: selection, caret, focus ring, scrollbar thumb, underline offset.\n\n### D5. Tokens (no DESIGN.md exists; these are this plan's inline design system)\n- Type: IBM Plex Sans for UI and body (explicitly assigned, Operate surface); IBM Plex Mono for numerals and times with `font-variant-numeric: tabular-nums`. Scale 16 / 20 / 32px; body never under 16px. h1 20px/600, stat numerals 32px/500, labels 16px muted.\n- Color as CSS variables: `--bg`, `--fg`, `--fg-muted`, `--hairline`, `--accent`, `--danger`, `--focus`. Body contrast ≥4.5:1, muted text ≥4.5:1. Muted text on any tinted surface is tinted from that hue, never gray. Accent hue is unresolved (U1); purple/indigo excluded.\n- Spacing scale: 4, 8, 12, 16, 24, 32, 48px. More space above a heading (24px) than below (8px).\n- Radius: 4px on inputs and buttons only. No shadows; depth comes from hairlines. Focus ring 2px `--focus`, offset 2px.\n- Links: underlined, visited color differs from unvisited.\n\n### D6. Responsive and accessibility\n- 375px: one column; stats in one row at 32px (fits at 3 × ~110px); settings is its own route; header 56px sticky; touch targets ≥44px; \"Load older\" full width.\n- 768px: one column, max-width 720px, stats row gains labels on the same line as numerals; settings still its own route.\n- ≥1024px: 12-column grid, max-width 1200px, feed 8 columns, settings aside 4 columns, sticky aside.\n- Landmarks: `header`, `nav`, `main` (feed), `aside` (settings). One h1. Skip link to main.\n- Feed is a `ul` with `aria-live=\"polite\"` on the \"N new\" pill only, not the list. Each row is a link with the full text as its name; `time` elements carry `datetime`.\n- Settings: visible labels above fields, never placeholder-as-label; errors tied via aria-describedby; Save reachable by Tab in order.\n- Keyboard: all actions on Tab order, visible focus per D5, Escape dismisses the unsaved-changes prompt.\n\n## NOT in scope\n- Charts or sparklines for stats: numbers alone meet the 3-second scan; add only once a trend question is validated.\n- Notification center or unread badges: the feed and \"N new\" pill cover this.\n- Theming beyond light/dark system preference: no user theme picker.\n- Avatar upload inside settings: separate upload flow with its own states.\n\n## What already exists\n- No DESIGN.md referenced by the plan; D5 stands in until `/design-consultation` produces one.\n- Tailwind is already the styling layer; D5 tokens map to theme extensions, no new dependency.\n- Existing auth redirect to /login is assumed and reused for the 401 state (verify in app shell).\n\n## TODOS.md updates\nApproved fixes are implementation tasks below, not TODOs. One deferred item proposed:\n- **What:** Create DESIGN.md from D5 tokens via `/design-consultation`. **Why:** Three future surfaces will otherwise re-derive tokens. **Pros:** One vocabulary for plans and code. **Cons:** ~1h of decisions including accent hue (U1). **Context:** This plan carries tokens inline because no system existed on 2026-10-04. **Depends on:** U1. Options: A) Add to TODOS.md (recommended) B) Skip C) Build now. Pending user choice.\n\n## Implementation Tasks\nSynthesized from this review's findings. Each task derives from a specific finding above. Run with Claude Code or Codex; checkbox as you ship.\n\n- [ ] **T1 (P1, human: ~1d / CC: ~20min)** — Dashboard shell — Build header, grid, and landmarks per D1 and D6\n - Surfaced by: Pass 1 — no hierarchy, hero on an Operate surface\n - Files: src/pages/Dashboard.tsx, src/components/AppHeader.tsx\n - Verify: axe has 0 landmark/heading violations; 375/768/1024 screenshots match D1 diagram\n- [ ] **T2 (P1, human: ~1d / CC: ~20min)** — Stats strip, feed, settings — Implement every cell of the D2 state table\n - Surfaced by: Pass 2 — zero states specified\n - Files: src/components/StatsStrip.tsx, ActivityFeed.tsx, SettingsPanel.tsx\n - Verify: Storybook story per state cell; mock API for empty, 500, partial\n- [ ] **T3 (P1, human: ~4h / CC: ~10min)** — Live updates — \"N new\" pill and reconnect notice per D2\n - Surfaced by: Pass 2 and Pass 3 step 4\n - Files: src/hooks/useActivitySocket.ts, ActivityFeed.tsx\n - Verify: inject 3 socket events while scrolled; no layout shift; focus lands on first new row\n- [ ] **T4 (P1, human: ~4h / CC: ~10min)** — Tokens — Define D5 CSS variables in Tailwind theme, theme browser surfaces per D4\n - Surfaced by: Pass 4 and Pass 5\n - Files: tailwind.config.ts, src/styles/tokens.css\n - Verify: contrast check on all text pairs ≥4.5:1; no hex literals in components\n- [ ] **T5 (P2, human: ~4h / CC: ~10min)** — Responsive and a11y — Breakpoint behavior and keyboard paths per D6\n - Surfaced by: Pass 6\n - Files: Dashboard.tsx, SettingsPanel.tsx, routes\n - Verify: keyboard-only walkthrough; touch targets ≥44px at 375px; reduced-motion disables pill fade\n- _No new tasks from Pass 7 until U1–U4 are resolved._\n\n## Completion Summary\n\nDecisions D1–D6 were applied under this run's non-interactive delivery instruction (no AskUserQuestion available); each is reversible by editing its section. Mockups were not generated (designer not run). JSONL task artifact and review log were not written because no gstack binaries were run this pass.\n\n```\n +====================================================================+\n | DESIGN PLAN REVIEW — COMPLETION SUMMARY |\n +====================================================================+\n | System Audit | No DESIGN.md; UI scope: /dashboard page, |\n | | stats, feed, settings, realtime |\n | Step 0 | 2/10 initial; focus: all 7 passes |\n | Pass 1 (Info Arch) | 2/10 → 9/10 after fixes |\n | Pass 2 (States) | 0/10 → 9/10 after fixes |\n | Pass 3 (Journey) | 1/10 → 8/10 after fixes |\n | Pass 4 (AI Slop) | 1/10 → 8/10 after fixes |\n | Pass 5 (Design Sys) | 1/10 → 8/10 after fixes |\n | Pass 6 (Responsive) | 0/10 → 9/10 after fixes |\n | Pass 7 (Decisions) | 6 resolved, 4 deferred |\n +--------------------------------------------------------------------+\n | NOT in scope | written (4 items) |\n | What already exists | written |\n | TODOS.md updates | 1 item proposed |\n | Approved Mockups | 0 generated, 0 approved |\n | Decisions made | 6 added to plan |\n | Decisions deferred | 4 (listed below) |\n | Overall design score | 0/10 → 8/10 |\n +====================================================================+\n```\n\nScore rationales and what a 10 needs:\n- **Pass 1, 2→9.** Was: six bullets, no order, hero first. Now: D1 ordered hierarchy and diagram. 10 needs settings field inventory (U2) so the aside hierarchy is real.\n- **Pass 2, 0→9.** Was: no states. Now: D2 covers all five states for five features. 10 needs the pagination contract (U4) to confirm the PARTIAL feed cell.\n- **Pass 3, 1→8.** Was: no user mentioned. Now: D3 covers visceral, behavioral, and reflective horizons. 10 needs brand voice (U1) for the empty-state copy to feel like this product.\n- **Pass 4, 1→8.** Hard rejections hit on input: #7 app UI of stacked cards, plus blacklist #1 gradient and #3 icons as decoration. Litmus on D1–D5: brand unmistakable YES (logo and h1 in header), one anchor YES (stats strip), scannable by headings YES, one job per section YES, cards necessary NO so none used, motion improves hierarchy YES (one pill moment), premium without shadows YES (none specified). All hard rejections cleared. 10 needs the accent hue chosen (U1), since an unchosen color invites the purple default back.\n- **Pass 5, 1→8.** No DESIGN.md; D5 supplies tokens inline. 10 needs DESIGN.md so tokens are cited by path, per the TODO above.\n- **Pass 6, 0→9.** Was: nothing. Now: D6 per-viewport layouts and a11y. 10 needs the 768px settings decision confirmed once U2 fixes field count.\n\nPass 7 register:\n\n```\n DECISION NEEDED | IF DEFERRED, WHAT HAPPENS\n ----------------------------------------------|--------------------------------------------\n U1 Accent hue and type license for Plex | Engineer picks Tailwind indigo; blacklist #1\n U2 Which settings fields exist in the panel | Panel ships with display name only, or bloats\n U3 Engagement rate definition and period | Label says \"engagement\"; users distrust number\n U4 Does /api/dashboard paginate activity? | Feed loads everything; \"Load older\" is fake\n```\n\nPlan is not design-complete: four decisions above are deferred to the user. Passes 4 and 5 stay at 8 until U1 and DESIGN.md land. Run /design-review after implementation for visual QA.\n\n## Unresolved Decisions\n- U1: accent hue and typeface licensing (blocks Pass 4 and 5 reaching 10).\n- U2: settings panel field inventory (D1 aside and 768px behavior depend on it).\n- U3: engagement rate formula and window (D1 label currently assumes 30 days).\n- U4: activity pagination in GET /api/dashboard (D2 PARTIAL cell is conditional).\n\n## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 0 | not run | — |\n| Outside Review | not run (Codex not invoked this pass) | Independent 2nd opinion | 0 | skipped | — |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | not run | — |\n| Design Review | `/plan-design-review` | UI/UX gaps | 1 | issues_open | score: 0/10 → 8/10, 6 decisions |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | not run | — |\n\nPrior review log was not read this pass (gstack-review-read not run); prior run counts shown as 0 are unverified.\n\n**OUTSIDE COVERAGE:** provider codex, phase design, skipped (non-interactive run, no outside process started), no findings. Missing coverage, not clean.\n\n**VERDICT:** No reviews CLEAR. Design review ISSUES OPEN (4 unresolved). Eng review required.\n\n**UNRESOLVED DECISIONS:**\n- U1: accent hue and typeface licensing\n- U2: settings panel field inventory\n- U3: engagement rate formula and window\n- U4: activity pagination contract in GET /api/dashboard\n" + } +} diff --git a/test/fixtures/detector-corpora/plan-design-review-plan-mode/37193478719-t1-pass.json b/test/fixtures/detector-corpora/plan-design-review-plan-mode/37193478719-t1-pass.json new file mode 100644 index 000000000..c388cd334 --- /dev/null +++ b/test/fixtures/detector-corpora/plan-design-review-plan-mode/37193478719-t1-pass.json @@ -0,0 +1,22 @@ +{ + "schema": "gstack-detector-corpus/v1", + "case": "plan-design-review-plan-mode", + "detector": "test/skill-e2e-design.test.ts#plan-design-review-plan-mode.validate", + "series_identity": "6b54ea60e2b7fcd3", + "source": { + "run": "37193478719", + "sha": "9f357d9b9c75441b6a18e4f75174274954ba6f96", + "lane": "periodic", + "artifact": "native-captures-ci-37193478719-1-eval-slices-9", + "path": ".gstack/projects/garrytan-gstack/e2e-runs/2026-10-04-0953/plan-design-review-plan-mode.ndjson" + }, + "census_outcome": "passed", + "census_error": "", + "expected": "pass", + "justification": "Rated summary without the four terms; passes only through the plan-term credit (d603a63, 44ef18c), so the pre-repair detector would have refused this good review too.", + "input": { + "exit_reason": "success", + "output": "Read-back verified: all design decisions, the seven passes, the completion summary and the report are saved, with the report as the last section and the unresolved-decisions block as its final lines.\n\nDesign review of `plan.md` saved. Overall score 0/10 to 7/10, status issues_open.\n\n- **Hero and cards removed.** Classified the page as OPERATE app UI. The gradient hero became a one-line identity bar, and \"cards and icons\" became a hairline stats strip plus feed and settings columns (D1 diagram).\n- **Tokens replace \"clean, modern UI\".** D2 names IBM Plex Sans/Mono, one green accent, text and surface colors, radius, hairlines and themed browser surfaces.\n- **States specified.** D3 is a full loading/empty/error/success/partial table per feature. D4 gives three empty states with copy and an action. D5 defines live WebSocket arrival as a \"3 new · Show\" pill with one authored motion.\n- **Responsive and a11y.** D6 sets distinct layouts at three breakpoints with a bottom tab bar on phones. D7 covers landmarks, feed role, live region, keyboard, contrast and reduced motion.\n- **Review artifacts.** Journey storyboard, seven scored passes with 10/10 rationales, seven implementation tasks, two TODO proposals, and the terminal GSTACK REVIEW REPORT.\n- **Left open for you:** U1 canonical palette and typeface via a DESIGN.md, U2 light vs dark from the use scene, U3 mobile settings tab behavior.\n\nSkipped per your instructions: preamble, mockups, outside voices, review log, telemetry and the JSONL task artifact.", + "plan_after": "# Plan: User Dashboard\n\n## Context\nBuild a user dashboard that shows account stats, recent activity, and settings.\n\nSurface mode (Pass 4 classifier): **OPERATE**. The visitor's win is \"check my numbers, see what happened, change a setting, leave.\" App UI rules apply; no marketing hero.\n\n## Implementation\n1. Create a dashboard page at /dashboard with the layout in D1.\n2. Show user stats (posts, followers, engagement rate) as the Stats strip per D1 and D3.\n3. Add a recent activity feed per D1, D3, D5.\n4. Add a settings panel per D8.\n5. Apply the type and color tokens in D2 (replaces \"clean, modern UI with cards and icons\").\n6. Replace the gradient hero with the Identity bar in D1 (hero removed; see Pass 4).\n7. Implement every state in the D3 table and the D4 empty states.\n8. Implement responsive layouts per D6 and accessibility per D7.\n\n## Technical Details\n- React components with Tailwind CSS (tokens from D2 exposed as CSS variables in `tailwind.config`)\n- API endpoint: GET /api/dashboard\n- WebSocket for real-time activity updates (behavior in D5)\n\n## Design Decisions\n\nEach decision is specified once here. Passes, tasks and the report refer to these IDs.\n\n### D1. Information architecture (Pass 1)\nThree things, in this order: **the numbers, what changed, where to act**. Everything else is subordinate.\n\n```\n+----------------------------------------------------------------------+\n| [Logo] Dashboard Posts Audience Settings [Avatar] | <- global nav, current page underlined\n+----------------------------------------------------------------------+\n| Identity bar: \"@handle · Last 30 days ▾\" [New post] | <- 1 line, 56px, flat surface, one accent button\n+----------------------------------------------------------------------+\n| STATS STRIP (one row, hairline dividers, no cards) |\n| Posts 128 ▲4 Followers 2,310 ▲62 Engagement 4.8% ▼0.3 | <- tabular numerals, delta vs previous period\n+------------------------------------------+---------------------------+\n| RECENT ACTIVITY (primary workspace, 2/3) | SETTINGS (context, 1/3) |\n| ● \"Jane liked your post\" · 2m | Email digest [toggle] |\n| ● \"New follower: @sam\" · 14m | Profile visibility [▾] |\n| ● \"Your post reached 1k\" · 1h | Timezone [▾] |\n| ... | Manage account → |\n| [Show earlier activity] | |\n+------------------------------------------+---------------------------+\n```\nNavigation flow: nav → /dashboard (default landing after login). Activity rows link to the referenced post or profile. \"Manage account\" links to the full settings route. The stats strip is not clickable (no hidden affordances).\n\n### D2. Type and color tokens (Pass 5, replaces \"clean, modern UI\")\n| Token | Value | Role |\n|---|---|---|\n| `--font-ui` | IBM Plex Sans 400/600 | body and UI, 16px base, 1.5 line height |\n| `--font-num` | IBM Plex Mono 500, `font-variant-numeric: tabular-nums` | stats, deltas, timestamps |\n| `--text-1` / `--text-2` | `#17191C` / `#5A6068` | primary / secondary text on `--surface` |\n| `--surface` / `--surface-2` | `#FFFFFF` / `#F4F5F7` | page / settings column |\n| `--accent` | `#0B6E4F` | one accent: primary button, focus ring, active nav |\n| `--up` / `--down` | `#0B6E4F` / `#B42318` | deltas; always paired with ▲/▼ glyph, never color alone |\n| `--hairline` | `#E3E5E8` 1px | dividers; no card borders, no shadows, no gradients |\n| `--radius` | 6px (controls only) | containers are square |\n| Type scale | 32/24/16/14 px | stat value / section heading / body / meta |\nBrowser surfaces themed from these tokens: selection `--accent` at 20% alpha, focus ring 2px `--accent` offset 2px, caret `--accent`. Light mode chosen because the use scene is a creator checking numbers at a desk or on a phone in daylight; see U2.\n\n### D3. Interaction state table (Pass 2)\nWhat the user SEES, per feature.\n\n| FEATURE | LOADING | EMPTY | ERROR | SUCCESS | PARTIAL |\n|---|---|---|---|---|---|\n| Stats strip | 3 skeleton bars at value height, labels visible, 200ms delay before showing | See D4-a | Values replaced by \"—\", one inline line: \"Couldn't load stats. Retry\" (button) | Values with ▲/▼ delta and period label | Loaded stats shown; missing stat shows \"—\" with tooltip \"Not available yet\" |\n| Activity feed | 5 skeleton rows, 200ms delay | See D4-b | Inline notice at feed top: \"Activity isn't loading. Retry\"; last known rows stay visible | Rows newest first, relative time, each row one line, 47-char names truncated with ellipsis and full name in `title` | First page shown; \"Show earlier activity\" button; if an earlier page fails, the button shows \"Retry loading earlier\" |\n| Live updates (WS) | Dot in feed heading: \"Live\" when connected | n/a | Heading dot goes gray: \"Reconnecting…\", then \"Offline, refresh to update\" after 30s | New items per D5 | Connected but no events: heading stays \"Live\", nothing else changes |\n| Settings panel | Controls render disabled with current values grayed until loaded | n/a (always has settings) | Field-level: \"Couldn't save. Try again\" under the control, previous value restored | Control reflects new value, 2s \"Saved\" text next to it, no toast | One field fails, others saved; only the failed field shows its error |\n| Identity bar / period picker | Handle shown immediately from session; period picker disabled until stats load | n/a | Picker stays on last good period; stats error shown in strip | Changing period reloads stats and feed with skeletons | n/a |\n\n### D4. Empty states (Pass 2)\n- **D4-a Stats, no posts yet:** values show \"0\", engagement shows \"—\". Under the strip, one sentence in `--text-2`: \"Your stats start with your first post.\" Primary action is the existing New post button, which gains a 1-line caption; no illustration.\n- **D4-b Feed, no activity:** feed area shows a single row-height line: \"Nothing yet. When people follow or react, it shows up here.\" plus a secondary link \"Share your profile\" (copies URL). No icon, no empty-state artwork.\n- **D4-c Feed, filtered period empty:** \"No activity in the last 30 days. Show all time\" (link switches period).\n\n### D5. Live update behavior (Pass 2/3)\nNew WebSocket events do not insert into the list while the user is reading. A pill appears at feed top: \"3 new · Show\". Clicking inserts the rows with one 200ms ease-out fade (the page's single authored motion moment) and scrolls to the top of the feed. If the user is already scrolled to the top and idle for 5s, items insert directly. Stats strip values update in place without animation when the payload includes new totals.\n\n### D6. Responsive behavior (Pass 6)\n| Viewport | Layout decision |\n|---|---|\n| ≥1024px | D1 as drawn: 2/3 feed, 1/3 settings, stats in one row |\n| 768–1023px | Stats one row; settings column moves below the feed as a full-width section headed \"Settings\"; feed shows 10 rows before \"Show earlier\" |\n| <768px | Global nav collapses to bottom tab bar (Dashboard, Posts, Audience, Settings) at 56px; identity bar wraps to two lines with New post full-width below; stats become a 3-up single row with values at 24px and labels at 12px, horizontally fixed (no scroll); feed is the page body; settings panel is NOT shown inline, the Settings tab routes to the full settings page; live-update pill is sticky under the identity bar |\nTouch targets 44×44px minimum on all viewports. Activity rows are 56px tall on <768px.\n\n### D7. Accessibility (Pass 6)\n- Landmarks: `header` (nav), `main` (stats + feed), `aside` (settings, labeled \"Settings\"), `nav` with `aria-current=\"page\"`.\n- Headings: h1 \"Dashboard\" visually hidden; h2 for Stats, Recent activity, Settings.\n- Feed is `role=\"feed\"` with `aria-busy` during loading; new items announced via `aria-live=\"polite\"` region reading \"3 new activity items\" (not each item).\n- Keyboard: Tab order follows D1 reading order; period picker and settings selects are native ``, `