From 5d032ef29912350e76c733c0485d9099f3c0035c Mon Sep 17 00:00:00 2001 From: garrytan Date: Tue, 29 Sep 2026 05:09:36 +0000 Subject: [PATCH] test: delete tests of dead eval code (A) - A1: the retired Eng lexical oracle (evaluateEngSeedCoverage, isEngSeedDecisionAUQ), the completion-handoff detector and the retained corpus had no paid caller since v1.87.6; delete their 26 replay files, ~2.6k helper LOC and fixtures, and the dead blocks in 8 mixed files (live hasNativePlanTerminal / batching assertions stay). - A2: dead viewport approvers in autoplan-artifact-permission and their 11 replay files + fixtures; recorder/launcher cases stay. - A3: never-wired oracles and seeders (autoplan-phase-order, eng-finding-fixture, ceo-paired-fixture, design-ui-scope, plan-skill-completion, pty-current-screen, required-reads, transcript-section-logger); plan-seed-submission now decodes through the production createPtyScreen; section manifests name their actual guard. - A4: zero-reference helper exports, plus execGit and invokeAndObserve found by the reachability pass. - 52 fixtures orphaned by the deletions; touchfile and selection-table entries for every deleted path. --- TODOS.md | 10 +- autoplan/sections/manifest.json | 2 +- browse/sections/manifest.json | 2 +- codex/sections/manifest.json | 2 +- design-html/sections/manifest.json | 2 +- design-shotgun/sections/manifest.json | 2 +- land-and-deploy/sections/manifest.json | 2 +- plan-ceo-review/sections/manifest.json | 2 +- qa/sections/manifest.json | 2 +- review/sections/manifest.json | 2 +- setup-gbrain/sections/manifest.json | 2 +- ship/sections/manifest.json | 2 +- spec/sections/manifest.json | 2 +- test/autoplan-artifact-permission.test.ts | 214 -- test/autoplan-artifact-stall-as.test.ts | 144 - test/autoplan-clipped-suffix-aq.test.ts | 93 +- test/autoplan-command-prefix-au.test.ts | 206 -- test/autoplan-cropped-command-av.test.ts | 116 - test/autoplan-edit-digests-al.test.ts | 95 +- test/autoplan-edit-edges-an.test.ts | 121 - test/autoplan-edit-header-ag.test.ts | 109 - test/autoplan-edit-panel-aj.test.ts | 100 - test/autoplan-edit-prefix-ai.test.ts | 121 - test/autoplan-edit-queue-am.test.ts | 189 -- test/autoplan-pending-artifact.test.ts | 139 - test/autoplan-permission-viewport.test.ts | 334 --- test/autoplan-phase-handoff.test.ts | 2 - test/autoplan-phase-observation.test.ts | 505 ---- test/autoplan-rendered-batch-at.test.ts | 70 - test/autoplan-repeated-header-ak.test.ts | 91 - test/ceo-paired-payment-fixture.test.ts | 118 - test/design-ui-scope.test.ts | 124 - test/eng-before-rewrite-ar.test.ts | 92 - test/eng-blocking-baseline-at.test.ts | 90 - test/eng-count-ad-v2.test.ts | 77 +- test/eng-count-owned-outcomes.test.ts | 119 - test/eng-current-native-seeds.test.ts | 71 - test/eng-declared-regression-ai.test.ts | 197 -- test/eng-declared-suite-ak.test.ts | 120 - test/eng-error-flow-seed.test.ts | 512 ---- test/eng-finding-fixture.test.ts | 105 - test/eng-golden-master-al.test.ts | 115 - test/eng-golden-parity-an.test.ts | 184 -- test/eng-initial-selector-043a.test.ts | 61 - test/eng-legacy-contract-am.test.ts | 64 - test/eng-mandatory-baseline-as.test.ts | 93 - test/eng-native-seed-contract.test.ts | 915 ------ test/eng-next-handoff-ah.test.ts | 317 +- test/eng-owned-explanation.test.ts | 215 -- test/eng-owned-seeds-av.test.ts | 219 -- test/eng-paired-regression-av.test.ts | 71 - test/eng-published-navigation.test.ts | 772 +---- test/eng-regression-pinning-ag.test.ts | 49 - test/eng-required-parity-au.test.ts | 123 - test/eng-resolution-block-position.test.ts | 35 +- test/eng-retained-corpus-au.test.ts | 78 - test/eng-retry-contract-am.test.ts | 45 - test/eng-retry-coverage-as.test.ts | 130 - test/eng-retry-coverage-at.test.ts | 131 - test/eng-scheduled-regression.test.ts | 154 - test/eng-seeded-completion-ai.test.ts | 61 - test/eng-seeded-coverage.test.ts | 1015 +------ test/eng-seeded-packet-ae.test.ts | 125 - test/eng-snapshot-adapter-aj.test.ts | 130 - test/eng-staged-regression-aq.test.ts | 74 - test/eng-task-pause-navigation-f359.test.ts | 112 +- .../autoplan-artifact-permission-ad-v3.json | 87 - test/fixtures/autoplan-artifact-stall-as.json | 1590 ---------- test/fixtures/autoplan-command-prefix-au.json | 940 ------ .../fixtures/autoplan-cropped-command-av.json | 156 - test/fixtures/autoplan-edit-edges-an.json | 120 - test/fixtures/autoplan-edit-header-ag.json | 727 ----- test/fixtures/autoplan-edit-panel-aj.json | 34 - test/fixtures/autoplan-edit-prefix-ai.json | 53 - test/fixtures/autoplan-edit-queue-am.json | 211 -- test/fixtures/autoplan-rendered-batch-at.json | 543 ---- .../fixtures/autoplan-repeated-header-ak.json | 41 - test/fixtures/eng-69193-count-public.json | 332 --- test/fixtures/eng-6aef-count-public.json | 300 -- test/fixtures/eng-a689-count-public.json | 239 -- test/fixtures/eng-before-rewrite-ar.md | 452 --- test/fixtures/eng-blocking-baseline-at.md | 401 --- test/fixtures/eng-cdd-regression-task.json | 9 - test/fixtures/eng-count-c6fc-public.json | 460 --- .../eng-count-owned-outcomes-f359.json | 379 --- test/fixtures/eng-current-choice-cab3.json | 177 -- test/fixtures/eng-current-ledger-seeds.json | 43 - .../eng-current-native-seeds-6714.json | 474 --- test/fixtures/eng-declared-regression-ai.json | 315 -- test/fixtures/eng-declared-suite-ak.json | 152 - test/fixtures/eng-e366-count-public.json | 360 --- .../fixtures/eng-existing-auth/legacy-auth.ts | 37 - test/fixtures/eng-existing-auth/package.json | 8 - test/fixtures/eng-golden-master-al.json | 170 -- test/fixtures/eng-golden-parity-an.json | 150 - test/fixtures/eng-idp-choice-90f.json | 39 - test/fixtures/eng-initial-selector-043a.json | 416 --- test/fixtures/eng-legacy-contract-am.json | 138 - test/fixtures/eng-legacy-declaration-90f.json | 44 - test/fixtures/eng-mandatory-baseline-as.md | 367 --- test/fixtures/eng-native-packets-b955.json | 728 ----- .../fixtures/eng-native-seed-contract-6f.json | 494 --- test/fixtures/eng-neutral-seed-749df.json | 83 - test/fixtures/eng-owned-explanation.json | 50 - test/fixtures/eng-owned-seeds-av.json | 88 - test/fixtures/eng-paired-regression-av.md | 32 - test/fixtures/eng-paired-suite-749df.json | 87 - test/fixtures/eng-regression-pinning-ag.json | 382 --- test/fixtures/eng-required-parity-au.md | 397 --- test/fixtures/eng-retained-corpus-au.md | 459 --- test/fixtures/eng-retry-baseline-as.md | 385 --- test/fixtures/eng-retry-baseline-at.md | 461 --- test/fixtures/eng-retry-contract-am.json | 138 - test/fixtures/eng-retry-coverage-as.json | 299 -- test/fixtures/eng-retry-coverage-at.json | 340 --- test/fixtures/eng-seeded-packet-ae.json | 96 - test/fixtures/eng-snapshot-adapter-aj.json | 168 -- test/fixtures/eng-staged-regression-aq.md | 445 --- test/fixtures/eng-structure-choice-90f.json | 71 - test/fixtures/native-viewport.ts | 15 - test/fixtures/paired-payment/README.md | 39 - .../paired-payment/contract.test.ts.fixture | 87 - test/fixtures/paired-payment/src/payment.ts | 44 - test/fixtures/plan-design-ui-scope.json | 540 ---- test/helpers/auq-sdk-capture.ts | 6 - test/helpers/autoplan-artifact-digest.ts | 88 - test/helpers/autoplan-artifact-permission.ts | 502 ---- test/helpers/autoplan-phase-order.ts | 380 --- test/helpers/captured-paths.ts | 14 - test/helpers/ceo-finding-fixture.ts | 14 - test/helpers/ceo-paired-fixture.ts | 19 - test/helpers/claude-pty-runner.ts | 61 - test/helpers/claude-pty-runner.unit.test.ts | 6 + test/helpers/cso-eval-oracles.ts | 8 +- test/helpers/design-ui-scope.ts | 27 - test/helpers/e2e-helpers.ts | 21 +- test/helpers/eng-completion-handoff.ts | 1481 --------- test/helpers/eng-finding-fixture.ts | 60 - test/helpers/eng-retained-corpus.ts | 67 - test/helpers/eng-seeded-coverage.ts | 2642 ----------------- test/helpers/plan-review-board-feedback.ts | 12 +- test/helpers/plan-review-cases.ts | 27 - test/helpers/plan-skill-completion.ts | 81 - test/helpers/pty-current-screen.ts | 148 - test/helpers/required-reads.ts | 40 - test/helpers/scratch-repo.ts | 12 - test/helpers/touchfiles-data.ts | 108 +- test/helpers/transcript-section-logger.ts | 196 -- test/periodic-fixture-selection.test.ts | 38 +- test/plan-count-fixture.test.ts | 22 +- test/plan-seed-submission.test.ts | 10 +- test/plan-skill-completion.test.ts | 166 -- test/pty-current-screen.test.ts | 239 -- test/required-reads.test.ts | 41 - test/touchfiles.test.ts | 2 +- test/transcript-section-logger.test.ts | 136 - 156 files changed, 107 insertions(+), 32055 deletions(-) delete mode 100644 test/autoplan-artifact-permission.test.ts delete mode 100644 test/autoplan-artifact-stall-as.test.ts delete mode 100644 test/autoplan-command-prefix-au.test.ts delete mode 100644 test/autoplan-cropped-command-av.test.ts delete mode 100644 test/autoplan-edit-edges-an.test.ts delete mode 100644 test/autoplan-edit-header-ag.test.ts delete mode 100644 test/autoplan-edit-panel-aj.test.ts delete mode 100644 test/autoplan-edit-prefix-ai.test.ts delete mode 100644 test/autoplan-edit-queue-am.test.ts delete mode 100644 test/autoplan-permission-viewport.test.ts delete mode 100644 test/autoplan-phase-observation.test.ts delete mode 100644 test/autoplan-rendered-batch-at.test.ts delete mode 100644 test/autoplan-repeated-header-ak.test.ts delete mode 100644 test/ceo-paired-payment-fixture.test.ts delete mode 100644 test/design-ui-scope.test.ts delete mode 100644 test/eng-before-rewrite-ar.test.ts delete mode 100644 test/eng-blocking-baseline-at.test.ts delete mode 100644 test/eng-count-owned-outcomes.test.ts delete mode 100644 test/eng-current-native-seeds.test.ts delete mode 100644 test/eng-declared-regression-ai.test.ts delete mode 100644 test/eng-declared-suite-ak.test.ts delete mode 100644 test/eng-error-flow-seed.test.ts delete mode 100644 test/eng-golden-master-al.test.ts delete mode 100644 test/eng-golden-parity-an.test.ts delete mode 100644 test/eng-initial-selector-043a.test.ts delete mode 100644 test/eng-legacy-contract-am.test.ts delete mode 100644 test/eng-mandatory-baseline-as.test.ts delete mode 100644 test/eng-native-seed-contract.test.ts delete mode 100644 test/eng-owned-explanation.test.ts delete mode 100644 test/eng-owned-seeds-av.test.ts delete mode 100644 test/eng-paired-regression-av.test.ts delete mode 100644 test/eng-regression-pinning-ag.test.ts delete mode 100644 test/eng-required-parity-au.test.ts delete mode 100644 test/eng-retained-corpus-au.test.ts delete mode 100644 test/eng-retry-contract-am.test.ts delete mode 100644 test/eng-retry-coverage-as.test.ts delete mode 100644 test/eng-retry-coverage-at.test.ts delete mode 100644 test/eng-scheduled-regression.test.ts delete mode 100644 test/eng-seeded-packet-ae.test.ts delete mode 100644 test/eng-snapshot-adapter-aj.test.ts delete mode 100644 test/eng-staged-regression-aq.test.ts delete mode 100644 test/fixtures/autoplan-artifact-permission-ad-v3.json delete mode 100644 test/fixtures/autoplan-artifact-stall-as.json delete mode 100644 test/fixtures/autoplan-command-prefix-au.json delete mode 100644 test/fixtures/autoplan-cropped-command-av.json delete mode 100644 test/fixtures/autoplan-edit-edges-an.json delete mode 100644 test/fixtures/autoplan-edit-header-ag.json delete mode 100644 test/fixtures/autoplan-edit-panel-aj.json delete mode 100644 test/fixtures/autoplan-edit-prefix-ai.json delete mode 100644 test/fixtures/autoplan-edit-queue-am.json delete mode 100644 test/fixtures/autoplan-rendered-batch-at.json delete mode 100644 test/fixtures/autoplan-repeated-header-ak.json delete mode 100644 test/fixtures/eng-69193-count-public.json delete mode 100644 test/fixtures/eng-6aef-count-public.json delete mode 100644 test/fixtures/eng-a689-count-public.json delete mode 100644 test/fixtures/eng-before-rewrite-ar.md delete mode 100644 test/fixtures/eng-blocking-baseline-at.md delete mode 100644 test/fixtures/eng-cdd-regression-task.json delete mode 100644 test/fixtures/eng-count-c6fc-public.json delete mode 100644 test/fixtures/eng-count-owned-outcomes-f359.json delete mode 100644 test/fixtures/eng-current-choice-cab3.json delete mode 100644 test/fixtures/eng-current-ledger-seeds.json delete mode 100644 test/fixtures/eng-current-native-seeds-6714.json delete mode 100644 test/fixtures/eng-declared-regression-ai.json delete mode 100644 test/fixtures/eng-declared-suite-ak.json delete mode 100644 test/fixtures/eng-e366-count-public.json delete mode 100644 test/fixtures/eng-existing-auth/legacy-auth.ts delete mode 100644 test/fixtures/eng-existing-auth/package.json delete mode 100644 test/fixtures/eng-golden-master-al.json delete mode 100644 test/fixtures/eng-golden-parity-an.json delete mode 100644 test/fixtures/eng-idp-choice-90f.json delete mode 100644 test/fixtures/eng-initial-selector-043a.json delete mode 100644 test/fixtures/eng-legacy-contract-am.json delete mode 100644 test/fixtures/eng-legacy-declaration-90f.json delete mode 100644 test/fixtures/eng-mandatory-baseline-as.md delete mode 100644 test/fixtures/eng-native-packets-b955.json delete mode 100644 test/fixtures/eng-native-seed-contract-6f.json delete mode 100644 test/fixtures/eng-neutral-seed-749df.json delete mode 100644 test/fixtures/eng-owned-explanation.json delete mode 100644 test/fixtures/eng-owned-seeds-av.json delete mode 100644 test/fixtures/eng-paired-regression-av.md delete mode 100644 test/fixtures/eng-paired-suite-749df.json delete mode 100644 test/fixtures/eng-regression-pinning-ag.json delete mode 100644 test/fixtures/eng-required-parity-au.md delete mode 100644 test/fixtures/eng-retained-corpus-au.md delete mode 100644 test/fixtures/eng-retry-baseline-as.md delete mode 100644 test/fixtures/eng-retry-baseline-at.md delete mode 100644 test/fixtures/eng-retry-contract-am.json delete mode 100644 test/fixtures/eng-retry-coverage-as.json delete mode 100644 test/fixtures/eng-retry-coverage-at.json delete mode 100644 test/fixtures/eng-seeded-packet-ae.json delete mode 100644 test/fixtures/eng-snapshot-adapter-aj.json delete mode 100644 test/fixtures/eng-staged-regression-aq.md delete mode 100644 test/fixtures/eng-structure-choice-90f.json delete mode 100644 test/fixtures/native-viewport.ts delete mode 100644 test/fixtures/paired-payment/README.md delete mode 100644 test/fixtures/paired-payment/contract.test.ts.fixture delete mode 100644 test/fixtures/paired-payment/src/payment.ts delete mode 100644 test/fixtures/plan-design-ui-scope.json delete mode 100644 test/helpers/autoplan-phase-order.ts delete mode 100644 test/helpers/captured-paths.ts delete mode 100644 test/helpers/ceo-paired-fixture.ts delete mode 100644 test/helpers/design-ui-scope.ts delete mode 100644 test/helpers/eng-completion-handoff.ts delete mode 100644 test/helpers/eng-finding-fixture.ts delete mode 100644 test/helpers/eng-retained-corpus.ts delete mode 100644 test/helpers/plan-skill-completion.ts delete mode 100644 test/helpers/pty-current-screen.ts delete mode 100644 test/helpers/required-reads.ts delete mode 100644 test/helpers/transcript-section-logger.ts delete mode 100644 test/plan-skill-completion.test.ts delete mode 100644 test/pty-current-screen.test.ts delete mode 100644 test/required-reads.test.ts delete mode 100644 test/transcript-section-logger.test.ts diff --git a/TODOS.md b/TODOS.md index 9f2dc1740..e44efb428 100644 --- a/TODOS.md +++ b/TODOS.md @@ -961,7 +961,7 @@ coverage fill. Remaining, in rough priority order: semantics-preserving; keep the daemon path for remote callers), plus a namespace hint appended to read-commands.ts:313's error. Effort S. - **P2 — PTY boot-readiness wait.** The PTY tests' Bun.sleep(8000) preludes - and invokeAndObserve's 6s boot_grace_ms are blind waits; a real readiness + are blind waits; a real readiness waitFor needs empirical CLI 2.1.x ready-marker probing in a working terminal environment (this sandbox's PTY probe wedged). Effort S, needs a dev machine. @@ -4102,9 +4102,9 @@ makes live agents start skipping a section. The canary is the only mechanism that catches that, from real usage. **Context:** Deferred from the carve-guard-hardening plan (D5→T2, codex -outside-voice #7). `test/helpers/transcript-section-logger.ts` exists but -is built for deterministic test transcripts + ship action fingerprints, -NOT real-session drift — it needs rework before it can back this. Ship +outside-voice #7). The deterministic `test/helpers/transcript-section-logger.ts` +was deleted in the 2026-09 test audit (no paid or production caller; see +docs/test-audit-2026-09.md); a real-session logger starts from scratch. Ship the deterministic guards first; add this once they've proven useful. The carved-skill set + each skill's `requiredReads` are already declared in `test/helpers/carve-guards.ts`, so the canary reads its expectations @@ -4112,7 +4112,7 @@ from there. **Effort:** M (human ~2d, CC ~4h). -**Depends on:** `transcript-section-logger.ts` real-session-drift rework. +**Depends on:** a real-session section-read logger (none exists today). ### P2: Harden behavioral section-loading test hermeticity diff --git a/autoplan/sections/manifest.json b/autoplan/sections/manifest.json index cd7bb295a..70e43f1f9 100644 --- a/autoplan/sections/manifest.json +++ b/autoplan/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "autoplan", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's phase sequencing (Sequential Execution + the Phase 0 UI/DX scope detection) is the ONLY place that decides WHEN to read a section \u2014 Phase 2 and Phase 2.5 are conditional and their sections must NOT be read when their scope is absent; required-reads live in the E2E fixtures. No machine predicate here \u2014 see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's phase sequencing (Sequential Execution + the Phase 0 UI/DX scope detection) is the ONLY place that decides WHEN to read a section \u2014 Phase 2 and Phase 2.5 are conditional and their sections must NOT be read when their scope is absent; required section reads are checked by test/skill-e2e-autoplan-chain.test.ts (auditAutoplanMethodReads). No machine predicate here \u2014 see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "ceo-phase", diff --git a/browse/sections/manifest.json b/browse/sections/manifest.json index e5f5108aa..f8a906689 100644 --- a/browse/sections/manifest.json +++ b/browse/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "browse", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's prose is the ONLY place that decides WHEN to read a section; required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's prose is the ONLY place that decides WHEN to read a section; required section reads are checked by test/carve-section-loading-browse.test.ts. No machine predicate here — see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "command-list", diff --git a/codex/sections/manifest.json b/codex/sections/manifest.json index dffd32902..e5c5be9b9 100644 --- a/codex/sections/manifest.json +++ b/codex/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "codex", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's Step 1 mode dispatch is the ONLY place that decides WHEN to read a section (the three modes are mutually exclusive — at most one section loads per invocation); required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's Step 1 mode dispatch is the ONLY place that decides WHEN to read a section (the three modes are mutually exclusive — at most one section loads per invocation); required section reads are checked by test/carve-section-loading-codex.test.ts. No machine predicate here — see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "review-mode", diff --git a/design-html/sections/manifest.json b/design-html/sections/manifest.json index 3adf8c19b..c452a68a3 100644 --- a/design-html/sections/manifest.json +++ b/design-html/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "design-html", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required section reads are checked by test/carve-section-loading-design-html.test.ts. No machine predicate here — see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "doctrine", diff --git a/design-shotgun/sections/manifest.json b/design-shotgun/sections/manifest.json index 198220262..70a515964 100644 --- a/design-shotgun/sections/manifest.json +++ b/design-shotgun/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "design-shotgun", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required section reads are checked by test/carve-section-loading-design-shotgun.test.ts. No machine predicate here — see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "doctrine", diff --git a/land-and-deploy/sections/manifest.json b/land-and-deploy/sections/manifest.json index 072a70481..610fb6d8a 100644 --- a/land-and-deploy/sections/manifest.json +++ b/land-and-deploy/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "land-and-deploy", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required section reads are checked by test/carve-section-loading-land-and-deploy.test.ts. No machine predicate here — see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "first-run-validation", diff --git a/plan-ceo-review/sections/manifest.json b/plan-ceo-review/sections/manifest.json index 5ef4425a9..21d5bea4f 100644 --- a/plan-ceo-review/sections/manifest.json +++ b/plan-ceo-review/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "plan-ceo-review", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required section reads are checked by test/skill-e2e-plan-ceo-review-section-loading.test.ts. No machine predicate here — see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "review-sections", diff --git a/qa/sections/manifest.json b/qa/sections/manifest.json index be254b68f..e964910fe 100644 --- a/qa/sections/manifest.json +++ b/qa/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "qa", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required section reads are checked by test/carve-section-loading-qa.test.ts. No machine predicate here — see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "test-bootstrap", diff --git a/review/sections/manifest.json b/review/sections/manifest.json index 26a94ca3a..e6a14bc06 100644 --- a/review/sections/manifest.json +++ b/review/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "review", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required section reads are checked by test/carve-section-loading-review.test.ts. No machine predicate here — see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "plan-completion", diff --git a/setup-gbrain/sections/manifest.json b/setup-gbrain/sections/manifest.json index e9cbf0c65..de5265113 100644 --- a/setup-gbrain/sections/manifest.json +++ b/setup-gbrain/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "setup-gbrain", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's detect (Step 1) + path picker (Step 2) are the ONLY places that decide WHEN a section is read (the install routes are branch-exclusive — at most one init route runs per invocation); required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's detect (Step 1) + path picker (Step 2) are the ONLY places that decide WHEN a section is read (the install routes are branch-exclusive — at most one init route runs per invocation); required section reads are checked by test/carve-section-loading-setup-gbrain.test.ts. No machine predicate here — see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "engine-remediation", diff --git a/ship/sections/manifest.json b/ship/sections/manifest.json index e4394e562..8b088da11 100644 --- a/ship/sections/manifest.json +++ b/ship/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "ship", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required-reads live in the E2E fixtures. No machine predicate here \u2014 see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required section reads are checked by test/skill-e2e-ship-section-loading.test.ts. No machine predicate here \u2014 see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "apple-release", diff --git a/spec/sections/manifest.json b/spec/sections/manifest.json index 9fcb83135..a443bf56e 100644 --- a/spec/sections/manifest.json +++ b/spec/sections/manifest.json @@ -2,7 +2,7 @@ "$schema": "https://gstack.dev/schemas/section-manifest.json", "skill": "spec", "version": 1, - "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.", + "note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required section reads are checked by test/carve-section-loading-spec.test.ts. No machine predicate here — see docs/designs/v2_PLAN.md:663.", "sections": [ { "id": "gate-and-file", diff --git a/test/autoplan-artifact-permission.test.ts b/test/autoplan-artifact-permission.test.ts deleted file mode 100644 index 0d7fcd718..000000000 --- a/test/autoplan-artifact-permission.test.ts +++ /dev/null @@ -1,214 +0,0 @@ -import { afterEach, describe, expect, test } from 'bun:test'; -import * as fs from 'node:fs'; -import * as os from 'node:os'; -import * as path from 'node:path'; -import fixture from './fixtures/autoplan-artifact-permission-ad-v3.json'; -import { autoplanArtifactPermissionInput } from './helpers/autoplan-artifact-permission'; -import { isPermissionDialogVisible } from './helpers/claude-pty-runner'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; -import type { NativePublicToolEvent } from './helpers/plan-count-transcript'; - -const roots: string[] = []; -afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); }); -function replay(relative?: string) { - const root = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-artifact-permission-')); roots.push(root); - const cwd = path.join(root, path.basename(fixture.cwd)); fs.mkdirSync(cwd); - const ownedStateRoot = path.join(root, 'home', '.gstack'); - const original = fixture.events.at(-1)!.input!.file_path; - const file = path.join(ownedStateRoot, 'projects', path.basename(cwd), relative ?? path.relative( - path.join(fixture.stateRoot, 'projects', path.basename(fixture.cwd)), original)); - fs.mkdirSync(path.dirname(file), { recursive: true }); - const publicTools = structuredClone(fixture.events) as NativePublicToolEvent[]; - for (const event of publicTools) if (event.input?.file_path) event.input.file_path = file; - const lastWrite = publicTools.filter(event => event.name === 'Write').at(-1)!; - fs.writeFileSync(file, lastWrite.input!.content as string); - const context = { cwd, ownedStateRoot, commandStartedAt: fixture.commandStartedAt, - now: Date.parse('2026-09-09T20:36:27.729Z'), transcriptStatus: 'ready', publicTools }; - const viewport = fixture.viewport.replaceAll(path.basename(original), path.basename(file)); - return { root, file, context, viewport }; -} -const pick = (r: ReturnType, seen = new Set()) => - autoplanArtifactPermissionInput(r.viewport, r.context, seen); - -describe('owned Autoplan artifact edit permission', () => { - test('captured cropped pane needs its pending identity; shared generic recognition stays unchanged', () => { - const r = replay(); - expect(isPermissionDialogVisible(fixture.viewport)).toBe(false); - expect(pick(r)).toEqual({ input: '1\r', signature: `${fixture.sessionId}:${fixture.events.at(-1)!.toolUseId}`, file: r.file }); - expect(pick(r, new Set([pick(r)!.signature]))).toBeNull(); - }); - - test('a later same-file Edit has a new one-time epoch even when the footer is identical', () => { - const r = replay(); const first = pick(r)!; const edit = r.context.publicTools.at(-1)!; - fs.writeFileSync(r.file, fs.readFileSync(r.file, 'utf8').replace(edit.input!.old_string as string, edit.input!.new_string as string)); - r.context.publicTools.push({ sessionId: fixture.sessionId, toolUseId: edit.toolUseId, kind: 'result', - timestamp: '2026-09-09T20:28:00.000Z', isError: false }); - r.context.publicTools.push({ ...structuredClone(edit), toolUseId: 'next-owned-edit', timestamp: '2026-09-09T20:28:01.000Z' }); - expect(pick(r, new Set([first.signature]))?.signature).toBe(`${fixture.sessionId}:next-owned-edit`); - }); - - test('a queued non-file tool cannot replace or grant the unique current Edit permission', () => { - const r = replay(); - r.context.publicTools.push({ sessionId: fixture.sessionId, toolUseId: 'queued-bash', kind: 'use', - timestamp: '2026-09-09T20:27:41.541Z', name: 'Bash', input: { command: 'echo unrelated queued work' } }); - expect(pick(r)?.input).toBe('1\r'); - r.context.publicTools.push({ ...r.context.publicTools.at(-1)!, toolUseId: 'concurrent-write', name: 'Write', - input: { file_path: r.file, content: 'other mutation' } }); - expect(pick(r)).toBeNull(); - }); - - test('the two source-declared Eng test-plan layouts have the same bounded edit path', () => { - for (const file of ['test-main-eng-review-test-plan-20260909-203000.md', 'test-main-test-plan-20260909-203000.md']) - expect(pick(replay(file))?.input).toBe('1\r'); - }); - - test('requires the exact owned project and known artifact filename; no broad state/home approval', () => { - for (const file of ['../sibling/ceo-plans/2026-09-09-user-dashboard.md', 'config.yaml', 'reviews.jsonl', - 'main-autoplan-restore-20260909-200700.md', 'ceo-plans/archive/2026-09-09-user-dashboard.md', - 'designs/screen-20260909/mockup.md', 'dx-plans/2026-09-09-plan.md', 'arbitrary.md']) - expect(pick(replay(file)), file).toBeNull(); - const r = replay(); - r.context.ownedStateRoot = undefined as any; expect(pick(r)).toBeNull(); - r.context.ownedStateRoot = path.join(r.root, 'caller-GSTACK_HOME'); expect(pick(r)).toBeNull(); - r.context.ownedStateRoot = path.join(r.root, 'home', '.gstack'); - r.context.cwd = path.join(r.root, 'sibling'); expect(pick(r)).toBeNull(); - }); - - test('regular current file and exact requested old/new text are mandatory', () => { - const r = replay(); const before = fs.readFileSync(r.file); - fs.writeFileSync(r.file, 'unrelated current content'); expect(pick(r)).toBeNull(); - fs.writeFileSync(r.file, before); - r.context.publicTools.at(-1)!.input!.new_string = 'unrelated replacement'; expect(pick(r)).toBeNull(); - fs.unlinkSync(r.file); fs.mkdirSync(r.file); expect(pick(r)).toBeNull(); - }); - - test.skipIf(process.platform === 'win32')('rejects symlink escape and symlink aliases within the owned tree', () => { - const r = replay(); const other = path.join(r.root, 'external.md'); - fs.renameSync(r.file, other); fs.symlinkSync(other, r.file); expect(pick(r)).toBeNull(); - fs.unlinkSync(r.file); fs.renameSync(other, r.file); - const directory = path.dirname(r.file); const alias = directory + '-actual'; - fs.renameSync(directory, alias); fs.symlinkSync(alias, directory); expect(pick(r)).toBeNull(); - }); - - test.skipIf(process.platform === 'win32')('trusted temp-parent aliases preserve ownership without permitting a symlink state root', () => { - const r = replay(); const alias = path.join(r.root, 'temp-parent-alias'); - fs.symlinkSync(path.join(r.root, 'home'), alias); - const target = path.join(alias, '.gstack', path.relative(r.context.ownedStateRoot, r.file)); - r.context.ownedStateRoot = path.join(alias, '.gstack'); - for (const event of r.context.publicTools) if (event.input?.file_path) event.input.file_path = target; - r.file = target; - expect(pick(r)?.input).toBe('1\r'); // e.g. macOS /var -> /private/var, above owned root - const stateAlias = path.join(r.root, 'state-alias'); - fs.symlinkSync(r.context.ownedStateRoot, stateAlias); - const other = path.join(stateAlias, path.relative(r.context.ownedStateRoot, r.file)); - r.context.ownedStateRoot = stateAlias; - for (const event of r.context.publicTools) if (event.input?.file_path) event.input.file_path = other; - r.file = other; - expect(pick(r)).toBeNull(); - }); - - test('missing, stale, future, foreign, completed, failed, duplicate and concurrent identities stay closed', () => { - const mutations: Array<(r: ReturnType) => void> = [ - r => { r.context.transcriptStatus = 'error'; }, - r => { r.context.publicTools = []; }, - r => { r.context.commandStartedAt = r.context.now + 1; }, - r => { r.context.commandStartedAt = Date.parse(r.context.publicTools.at(-1)!.timestamp) + 1; }, - r => { r.context.publicTools.at(-1)!.timestamp = '2026-09-10T00:00:00.000Z'; }, - r => { r.context.publicTools.at(-1)!.timestamp = 'invalid'; }, - r => { r.context.publicTools.at(-1)!.sessionId = 'foreign'; }, - r => { r.context.publicTools.at(-1)!.sessionId = ''; }, - r => { r.context.publicTools.at(-1)!.toolUseId = ''; }, - r => { r.context.publicTools.at(-1)!.name = 'Write'; }, - r => { r.context.publicTools.at(-1)!.input!.replace_all = true; }, - r => { r.context.publicTools.push({ ...r.context.publicTools.at(-1)!, kind: 'result', isError: false }); }, - r => { r.context.publicTools.push({ ...r.context.publicTools.at(-1)!, kind: 'result', isError: true }); }, - r => { r.context.publicTools.push(structuredClone(r.context.publicTools.at(-1)!)); }, - r => { r.context.publicTools.splice(-1, 0, { ...structuredClone(r.context.publicTools.at(-1)!), toolUseId: 'other-pending-edit' }); }, - r => { for (const event of r.context.publicTools) if (event.kind === 'result') event.isError = true; }, - r => { for (const event of r.context.publicTools.slice(0, -1)) if (event.input) event.input.file_path = r.file + '-sibling'; }, - r => { r.context.publicTools.reverse(); }, - ]; - for (const mutate of mutations) { const r = replay(); mutate(r); expect(pick(r), mutate.toString()).toBeNull(); } - }); - - test('quotes, examples, unrelated diffs, malformed menus, extra options and broad selection are rejected', () => { - const mutations = [ - (s: string) => 'Example:\n' + s, (s: string) => '```\n' + s + '\n```', - (s: string) => s.split('\n').map(line => '> ' + line).join('\n'), - (s: string) => s.replace('Success target made numeric', 'Unrelated line copied from another plan'), - (s: string) => s.replace('2026-09-09-user-dashboard.md?', 'sibling.md?'), - (s: string) => s.replace('❯ 1. Yes', ' 1. Yes').replace(' 2. Yes', '❯2. Yes'), - (s: string) => s.replace('❯ 1. Yes', '❯ 1. Yes, always allow'), - (s: string) => s.replace(' 3. No', ' 3. No\n 4. Change permission mode'), - (s: string) => s.replace('Esc to cancel · Tab to amend', 'Enter to select'), - (s: string) => s + '\nPlease choose the quoted example above.', - (s: string) => s.slice(s.indexOf(' Do you want')), // no bound diff - ]; - for (const mutate of mutations) { const r = replay(); r.viewport = mutate(r.viewport); expect(pick(r), mutate.toString()).toBeNull(); } - }); - - for (const deletion of [false, true]) test(`native ${deletion ? 'deletion' : 'replacement'} diff rows remain bound to the requested old/new text`, () => { - const r = replay(); const before = 'Old first\nOld second\nContext\n'; - fs.writeFileSync(r.file, before); - r.context.publicTools.filter(event => event.name === 'Write').at(-1)!.input!.content = before; - const edit = r.context.publicTools.at(-1)!; - edit.input!.old_string = 'Old first\nOld second'; - edit.input!.new_string = deletion ? '' : 'New first\nNew second'; - const menu = r.viewport.slice(r.viewport.indexOf(' Do you want')); - // Existing native fixtures include 102-,103-,102+,103+ replacements, - // and deleted-only rows. These small controls are projected, not live panes. - r.viewport = ' 1 -Old first\n 2 -Old second\n' + - (deletion ? '' : ' 1 +New first\n 2 +New second\n') + - ' 3 Context\n' + '╌'.repeat(20) + '\n' + menu; - expect(pick(r)?.input).toBe('1\r'); - r.viewport = r.viewport.replace(' 2 -Old second', ' 2 -Context'); - expect(pick(r)).toBeNull(); // Existing context is not part of the requested deletion. - }); - - // AZ's public line 116 wraps at column five, not the old fixed column four. - // These small panes exercise the same renderer rule without a transcript corpus. - for (const [line, numbered, continuation] of [ - [7, ' 7 ', ' '], [17, ' 17 ', ' '], - [116, ' 116 ', ' '], [1024, ' 1024 ', ' '], - ] as const) test(`wrapped line ${line} binds its own marker column and exact requested bytes`, () => { - const r = replay(), old = 'Old first portion kept together', replacement = 'New first portion kept together'; - const before = Array.from({ length: line - 1 }, (_, n) => `Context ${n}`).concat(old, 'Context tail').join('\n'); - fs.writeFileSync(r.file, before); - r.context.publicTools.filter(event => event.name === 'Write').at(-1)!.input!.content = before; - const edit = r.context.publicTools.at(-1)!; - edit.input!.old_string = old; edit.input!.new_string = replacement; - const menu = r.viewport.slice(r.viewport.indexOf(' Do you want')); - const rows = `${numbered}-Old first portion\n${continuation}- kept together\n` + - `${numbered}+New first portion\n${continuation}+ kept together\n`; - const pane = rows + '╌'.repeat(20) + '\n' + menu; - r.viewport = pane; - expect(pick(r)).toEqual({ input: '1\r', signature: `${edit.sessionId}:${edit.toolUseId}`, file: r.file }); - expect(pick(r, new Set([pick(r)!.signature]))).toBeNull(); - for (const invalid of [ - pane.replaceAll(`\n${continuation}`, `\n${continuation.slice(1)}`), // left-shifted continuation - pane.replaceAll(`\n${continuation}`, `\n ${continuation}`), // right-shifted continuation - pane.replace(`${continuation}- kept`, `${continuation}+ kept`), // different kind - pane.replace(`${numbered}+New`, ` ${numbered}+New`), // mixed complete-row columns - `${continuation}- kept together\n` + pane, // no owning numbered row - pane.replace('New first portion', 'Foreign replacement'), - pane.replace(numbered, ' 0 '), - pane.replace(numbered, ' 01 '), - pane.replace(numbered, ' 9007199254740992 '), - ]) { r.viewport = invalid; expect(pick(r), invalid).toBeNull(); } - }); - - test('an earlier unresolved mutation cannot make the latest completed Edit current', () => { - const r = replay(); const events = r.context.publicTools; const edit = events.at(-1)!; - events.splice(-1, 0, { ...structuredClone(edit), toolUseId: 'earlier-unresolved-edit', - input: { ...edit.input, file_path: r.file + '-other' } }); - events.push({ sessionId: edit.sessionId, toolUseId: edit.toolUseId, kind: 'result', - timestamp: '2026-09-09T20:28:00.000Z', isError: false }); - expect(pick(r)).toBeNull(); - }); - - test('shared artifact permission controls select Eng and Autoplan while the UI fixture stays Autoplan-only', () => { - for (const file of ['test/helpers/autoplan-artifact-permission.ts', 'test/autoplan-artifact-permission.test.ts']) - expect(selectTests([file], E2E_TOUCHFILES).selected.sort()).toEqual(['autoplan-chain-pty', 'plan-eng-finding-count']); - expect(selectTests(['test/fixtures/autoplan-artifact-permission-ad-v3.json'], E2E_TOUCHFILES).selected).toEqual(['autoplan-chain-pty']); - }); -}); diff --git a/test/autoplan-artifact-stall-as.test.ts b/test/autoplan-artifact-stall-as.test.ts deleted file mode 100644 index 803398fe2..000000000 --- a/test/autoplan-artifact-stall-as.test.ts +++ /dev/null @@ -1,144 +0,0 @@ -import { capturedPathRebaser } from './helpers/captured-paths'; -import {expect,test} from 'bun:test'; -import fs from 'node:fs';import os from 'node:os';import path from 'node:path'; -import fixture from './fixtures/autoplan-artifact-stall-as.json'; -import * as permission from './helpers/autoplan-artifact-permission'; -import {readPendingAutoplanArtifact,autoplanArtifactRecorderStatus} from './helpers/autoplan-artifact-recorder'; -import {readPlanCountTranscript,type NativePublicToolEvent} from './helpers/plan-count-transcript'; -import {E2E_TOUCHFILES,selectTests} from './helpers/touchfiles'; - -test('captured path rebasing preserves JSON strings and emits canonical native file paths',()=>{ - const destination=String.raw`C:\a\repo`,source={file:'/captured/plans/plan.md',content:'First\n/captured/notes\nLast'}; - const rebase=capturedPathRebaser([['/captured',destination]]); - const display=destination.split(path.sep).join('/'); - expect(rebase.json(source)).toEqual({file:path.normalize(display+'/plans/plan.md'),content:'First\n'+display+'/notes\nLast'}); - expect(source.file).toBe('/captured/plans/plan.md'); -}); - -test('captured path rebasing preserves malformed and foreign ownership inputs',()=>{ - const destination=path.join(path.parse(process.cwd()).root,'replayed'); - const rebase=capturedPathRebaser([['/captured',destination]]); - for(const suffix of ['../foreign.md','plans/../plan.md','plans//plan.md','plans/./plan.md']){ - expect(rebase.json({file:'/captured/'+suffix}).file).toBe(destination+path.sep+suffix.split('/').join(path.sep)); - } - expect(rebase.json({file:'../foreign.md'}).file).toBe('..'+path.sep+'foreign.md'); - expect(rebase.json({file:'/foreign/plans/../plan.md'}).file).toBe(path.sep+'foreign'+path.sep+'plans'+path.sep+'..'+path.sep+'plan.md'); -}); - -function replay() { - const root=fs.mkdtempSync(path.join(os.tmpdir(),'gstack-ap-stall-')); - const runtimeBefore=path.dirname(path.dirname(fixture.stateRoot)); - const runtime=path.join(root,path.basename(runtimeBefore)),cwd=path.join(root,path.basename(fixture.cwd)); - const rebase=capturedPathRebaser([[runtimeBefore,runtime],[fixture.cwd,cwd]]); - const hook=rebase.json(fixture.hook),stateRoot=rebase.file(fixture.stateRoot),config=rebase.file(fixture.config); - const events=rebase.json(fixture.publicTools) as NativePublicToolEvent[]; - const now=Date.parse(fixture.viewportCapturedAt),startedAt=Date.parse(fixture.commandStartedAt); - const file=hook.pending.file,nativePlan=events.filter(e=>e.kind==='use'&&e.name==='Edit').at(-1)!.input!.file_path as string; - for(const [target,content] of [[file,fixture.before],[nativePlan,fixture.nativePlanBefore]]) { - fs.mkdirSync(path.dirname(target),{recursive:true});fs.writeFileSync(target,content); - const at=new Date(Date.parse(hook.pending.timestamp)-1000);fs.utimesSync(target,at,at); - } - fs.mkdirSync(cwd,{recursive:true});fs.mkdirSync(path.dirname(hook.pending.transcriptPath),{recursive:true}); - const records=events.map(e=>({sessionId:e.sessionId,cwd,isSidechain:false,timestamp:e.timestamp,requestId:e.requestId, - message:{id:e.messageId,role:e.kind==='use'?'assistant':'user',content:e.kind==='use'?[{type:'tool_use',id:e.toolUseId,name:e.name,input:e.input}]:[{type:'tool_result',tool_use_id:e.toolUseId,content:e.content??'',is_error:e.isError}]}})); - fs.writeFileSync(hook.pending.transcriptPath,records.map(r=>JSON.stringify(r)).join('\n')+'\n'); - const hookFile=path.join(root,'hook.json');fs.writeFileSync(hookFile,JSON.stringify(hook)+'\n'); - const publicTools:NativePublicToolEvent[]=[];const transcript=readPlanCountTranscript(config,cwd,e=>publicTools.push(e)); - const pending=readPendingAutoplanArtifact(hookFile,cwd,config,stateRoot,startedAt,publicTools,now,true); - const context={cwd,ownedStateRoot:stateRoot,ownedNativePlansRoot:path.join(config,'plans'),commandStartedAt:startedAt, - now,viewportCapturedAt:now,transcriptStatus:transcript.status,publicTools,pending}; - const viewport=rebase.text(fixture.viewport); - const invoke=(screen=viewport,ctx=context,seen=new Set())=>permission.publishedAutoplanArtifactPermissionInput(screen,ctx,seen); - return {root,hook,hookFile,config,file,nativePlan,context,viewport,invoke,dispose:()=>fs.rmSync(root,{recursive:true,force:true})}; -} -type Replay=ReturnType; -const current=(r:Replay)=>r.context.publicTools.find(e=>e.toolUseId===r.hook.pending.toolUseId&&e.kind==='use')!; -const queued=(r:Replay)=>r.context.publicTools.filter(e=>e.kind==='use'&&e.name==='Edit'&&Date.parse(e.timestamp)>Date.parse(r.hook.pending.timestamp)); -function reject(cases:Array<[string,(r:Replay)=>void]>) { - for(const [name,change] of cases){const r=replay();try{change(r);expect(r.invoke(),name).toBeNull()}finally{r.dispose()}} -} - -test('the retained pending CEO edit remains distinct from later published native-plan edits',()=>{ - const r=replay();try{ - expect(autoplanArtifactRecorderStatus(r.hookFile,r.context.cwd,r.config,r.context.ownedStateRoot)).toEqual({status:'pending'}); - expect(r.context.pending?.toolUseId).toBe(fixture.hook.pending.toolUseId); - expect(queued(r)).toHaveLength(2); - expect(permission.autoplanArtifactPermissionInput(r.viewport,r.context,new Set())).toBeNull(); - expect(permission.pendingAutoplanArtifactPermissionInput(r.viewport,r.context,new Set())).toBeNull(); - expect(r.invoke()).toEqual({input:'1\r',signature:fixture.hook.sessionId+':'+fixture.hook.pending.toolUseId,file:r.file}); - expect(fixture.provenance.retrospectivePass).toBe(false); - expect(r.invoke(r.viewport,r.context,new Set([r.hook.sessionId+':'+r.hook.pending.toolUseId]))).toBeNull(); - expect(r.invoke(r.viewport,r.context,new Set([permission.autoplanArtifactMenuKey(r.viewport)]))).toBeNull(); - }finally{r.dispose()} -}); - -test('a bare current panel and its bound redraw labels represent the same one-time permission',()=>{ - const r=replay();try{ - const title=r.viewport.indexOf('● Update('),panel=r.viewport.indexOf('────────────────'); - expect(r.invoke(r.viewport.slice(title))?.input).toBe('1\r'); - expect(r.invoke(r.viewport.slice(panel))?.input).toBe('1\r'); - }finally{r.dispose()} -}); - -test('only unstarted same-batch publications to the launcher-owned native plans root may wait behind it',()=>{ - reject([ - ['no launcher root',r=>{delete (r.context as any).ownedNativePlansRoot}], - ['foreign launcher root',r=>{r.context.ownedNativePlansRoot=path.join(r.root,'foreign')}], - ['foreign message',r=>{queued(r)[0]!.messageId='msg_other'}], - ['foreign request',r=>{queued(r)[0]!.requestId='req_other'}], - ['foreign session',r=>{queued(r)[0]!.sessionId='other'}], - ['foreign target',r=>{queued(r)[0]!.input!.file_path=r.file+'.other'}], - ['queued Write',r=>{queued(r)[0]!.name='Write'}], - ['replace-all successor',r=>{queued(r)[0]!.input!.replace_all=true}], - ['already started successor',r=>{r.context.pending!.hookSeenIds!.push(queued(r)[0]!.toolUseId)}], - ['successor completion',r=>{const q=queued(r)[0]!;r.context.publicTools.push({kind:'result',sessionId:q.sessionId,toolUseId:q.toolUseId,timestamp:new Date(r.context.now).toISOString(),isError:false})}], - ['successor failure',r=>{const q=queued(r)[0]!;r.context.publicTools.push({kind:'result',sessionId:q.sessionId,toolUseId:q.toolUseId,timestamp:new Date(r.context.now).toISOString(),isError:true})}], - ['successor published after viewport',r=>{queued(r)[0]!.timestamp=new Date(r.context.viewportCapturedAt+1).toISOString()}], - ['missing native plan',r=>{fs.unlinkSync(r.nativePlan)}], - ['native plan changed after current hook',r=>{fs.utimesSync(r.nativePlan,new Date(r.context.now),new Date(r.context.now))}], - ['symlink native plan',r=>{const other=path.join(r.root,'other.md');fs.renameSync(r.nativePlan,other);fs.symlinkSync(other,r.nativePlan)}], - ['successful Read cannot replace native-plan mutation history',r=>{for(const e of r.context.publicTools)if(e.kind==='use'&&e.input?.file_path===r.nativePlan&&Date.parse(e.timestamp){const ids=new Set(r.context.publicTools.filter(e=>e.input?.file_path===r.nativePlan).map(e=>e.toolUseId));for(const e of r.context.publicTools)if(e.kind==='result'&&ids.has(e.toolUseId))e.isError=true}], - ]); -}); - -test('the active hook, current digest, successful owned history and time remain mandatory',()=>{ - reject([ - ['no current hook',r=>{r.context.pending=undefined}],['foreign hook',r=>{r.context.pending!.sessionId='other'}], - ['wrong current ID',r=>{r.context.pending!.toolUseId=queued(r)[0]!.toolUseId}], - ['no digest',r=>{delete r.context.pending!.editDigest}], - ['changed digest',r=>{r.context.pending!.editDigest!.requestSHA256='0'.repeat(64)}], - ['changed replacement',r=>{current(r).input!.new_string+=' changed'}], - ['changed current file',r=>{fs.appendFileSync(r.file,'changed');fs.utimesSync(r.file,new Date(0),new Date(0))}], - ['completed current',r=>{const q=current(r);r.context.publicTools.push({kind:'result',sessionId:q.sessionId,toolUseId:q.toolUseId,timestamp:new Date(r.context.now).toISOString(),isError:false})}], - ['current file newer than hook',r=>{fs.utimesSync(r.file,new Date(r.context.now),new Date(r.context.now))}], - ['pending after viewport',r=>{r.context.pending!.timestamp=new Date(r.context.now+1).toISOString()}], - ['stale hook',r=>{r.context.pending!.timestamp=new Date(r.context.commandStartedAt-1).toISOString()}], - ['unavailable transcript',r=>{r.context.transcriptStatus='missing'}], - ]); - const r=replay();try{ - fs.writeFileSync(r.hookFile+'.invalid','{"reason":"concurrent_pending"}'); - expect(readPendingAutoplanArtifact(r.hookFile,r.context.cwd,r.config,r.context.ownedStateRoot,r.context.commandStartedAt,r.context.publicTools,r.context.now,true)).toBeUndefined(); - }finally{r.dispose()} -}); - -test('completed output and redraw labels cannot hide a foreign, quoted or persistent-permission panel',()=>{ - const changes:Array<[string,(s:string)=>string]>=[ - ['example prefix',s=>'Example:\n'+s],['quoted whole pane',s=>s.split('\n').map(r=>'> '+r).join('\n')], - ['arbitrary output',s=>s.replace('"changed": true','"changed": false')], - ['foreign completed command',s=>s.replace('with-skills/.clau','foreign/.clau')], - ['missing one redraw',s=>s.replace('● Updated plan','')],['extra redraw',s=>s.replace('● Updated plan','● Updated plan\n● Updated plan')], - ['arbitrary redraw prose',s=>s.replace('● Updated plan','● Example plan')], - ['foreign current title',s=>s.replace('Update(~/.gstack/','Update(/foreign/')], - ['foreign displayed project',s=>s.replace('…-207152-jk89F3/skill-home-bOPSw5/.gstack/projects/gstack-autoplan-chain-kVh2Sb','…projects/foreign')], - ['different requested addition',s=>s.replace('## Reviewer Concerns','## An unrelated edit')], - ['wrong menu file',s=>s.replace('user-dashboard.md?','other.md?')], - ['persistent session approval',s=>s.replace('❯ 1. Yes','❯ 2. Yes')],['trailing prose',s=>s+'\nAnother prompt'], - ]; - for(const [name,edit] of changes){const r=replay();try{expect(r.invoke(edit(r.viewport)),name).toBeNull()}finally{r.dispose()}} -}); - -test('only Autoplan discovers the permission regression and its captured fixture',()=>{ - for(const file of ['test/autoplan-artifact-stall-as.test.ts','test/fixtures/autoplan-artifact-stall-as.json']) - expect(selectTests([file],E2E_TOUCHFILES,[]).selected).toEqual(['autoplan-chain-pty']); -}); diff --git a/test/autoplan-clipped-suffix-aq.test.ts b/test/autoplan-clipped-suffix-aq.test.ts index 3ffbfe6a1..ee968baed 100644 --- a/test/autoplan-clipped-suffix-aq.test.ts +++ b/test/autoplan-clipped-suffix-aq.test.ts @@ -1,97 +1,6 @@ -import {test,expect,afterEach} from 'bun:test';import fs from 'node:fs';import os from 'node:os';import path from 'node:path'; -import fixture from './fixtures/autoplan-clipped-suffix-aq.json'; -import {createAutoplanEditDigest,validAutoplanEditDigest,matchesAutoplanDigestRows} from './helpers/autoplan-artifact-digest'; -import {createAutoplanArtifactRecorder,recordAutoplanArtifact,readPendingAutoplanArtifact,autoplanArtifactRecorderStatus} from './helpers/autoplan-artifact-recorder'; -import {pendingAutoplanArtifactPermissionInput,autoplanArtifactMenuKey} from './helpers/autoplan-artifact-permission'; +import {test,expect,afterEach} from 'bun:test'; import {E2E_TOUCHFILES} from './helpers/touchfiles-data'; const cleanup:Array<()=>void>=[];afterEach(()=>{for(const f of cleanup.splice(0))f()}); -function replay(before=fixture.before,removed=fixture.request.old_string,added=fixture.request.new_string){ - const root=fs.mkdtempSync(path.join(os.tmpdir(),'ap-suffix-')),cwd=path.join(root,path.basename(fixture.cwd)),config=path.join(root,'config'),stateRoot=path.join(root,'home/.gstack'); - const file=path.normalize(fixture.hook.pending.file.replace(fixture.stateRoot,stateRoot)),native=path.join(config,'projects/owned',fixture.hook.sessionId+'.jsonl'); - fs.mkdirSync(cwd,{recursive:true});fs.mkdirSync(path.dirname(file),{recursive:true});fs.mkdirSync(path.dirname(native),{recursive:true});fs.writeFileSync(native,'');fs.writeFileSync(file,before);fs.utimesSync(file,new Date(0),new Date(0)); - const recorder=createAutoplanArtifactRecorder(cwd,config,stateRoot);cleanup.push(()=>{recorder.dispose();fs.rmSync(root,{recursive:true,force:true})}); - const event={hook_event_name:'PreToolUse',tool_name:'Edit',session_id:fixture.hook.sessionId,tool_use_id:fixture.hook.pending.toolUseId,cwd,transcript_path:native,tool_input:{file_path:file,old_string:removed,new_string:added}}; - recordAutoplanArtifact(JSON.stringify(event),recorder.file,cwd,config,stateRoot); - const publicTools=structuredClone(fixture.publicTools) as any[];for(const e of publicTools)if(e.input)e.input.file_path=file; - const commandStartedAt=Date.parse(publicTools[0].timestamp)-1; - const pending=readPendingAutoplanArtifact(recorder.file,cwd,config,stateRoot,commandStartedAt,publicTools); - const context={cwd,ownedStateRoot:stateRoot,commandStartedAt,transcriptStatus:'ready',publicTools,pending,now:Date.now()+1000,viewportCapturedAt:Date.now()}; - const invoke=(viewport=fixture.viewport,seen=new Set())=>pendingAutoplanArtifactPermissionInput(viewport,context,seen); - return {root,cwd,config,stateRoot,file,recorder,event,context,invoke}; -} -const menu=fixture.viewport.slice(fixture.viewport.indexOf('╌')); -const panel=(rows:string[])=>rows.join('\n')+'\n'+menu; -test('exact current clipped pane requires new recorded suffix commitments and preserves original request bytes',()=>{ - const r=replay(),digest=r.context.pending!.editDigest!; - expect(digest.beforeSHA256).toBe(fixture.provenance.beforeSHA256);expect(digest.requestSHA256).toBe(fixture.provenance.requestSHA256); - expect(digest.oldLineHashes).toEqual(fixture.hook.pending.editDigest.oldLineHashes);expect(digest.newLineHashes).toEqual(fixture.hook.pending.editDigest.newLineHashes); - expect(digest.clippedAdditions?.status).toBe('complete');expect(r.invoke()?.input).toBe('1\r'); - delete digest.clippedAdditions;expect(r.invoke()).toBeNull();expect(r.invoke(fixture.viewport.split('\n').slice(1).join('\n'))?.input).toBe('1\r'); - expect(fixture.provenance.actualCoverage).toContain('no phase credit'); -}); -test('first, middle and last changed lines support full120-column crops and following context',()=>{ - const lines=Array.from({length:32},(_,i)=>'Line '+i+' '+String.fromCharCode(65+i%26).repeat(180)); - const r=replay('Heading\nAnchor\nAfter one\nAfter two\n','Anchor',lines.join('\n')); - expect(r.context.pending!.editDigest!.clippedAdditions?.status).toBe('complete'); - for(const i of [0,15,31]){ - const row=i+2,tail=lines[i]!.slice(-114),next=i+1{ - for(const change of [ - (s:string)=>s.replace(/^ \+t\./,' +x.'), (s:string)=>s.replace(/^ \+t\./,' +t!'), - (s:string)=>s.replace(/^ \+t\./,' +t.'),(s:string)=>s.replace(/^ \+t\./,' +t.'), - (s:string)=>s.replace(/^ \+t\./,' -t.'),(s:string)=>s.replace(/^ \+t\./,' Source: t.'), - // A forged deletion marker cannot make rejected digest rows use legacy authority. - (s:string)=>s.replace(/^ \+t\./,' -t.').replace(/^ 139 /m,' 140 '), - (s:string)=>s.replace(/^ \+t\./,' -t.').replace('Snapshot consistency','Foreign consistency'), - (s:string)=>s.replace(/^ 139 /m,' 140 '),(s:string)=>s.replace('Snapshot consistency','Foreign consistency'), - (s:string)=>s.replace('authoritative gate','unrequested gate'),(s:string)=>'> source\n'+s, - (s:string)=>s.replace('3. No','3. Maybe'),(s:string)=>s.replace('❯ 1. Yes','❯ 2. Yes'), - (s:string)=>s+'\nUnrelated menu', - ]){const r=replay();expect(r.invoke(change(fixture.viewport))).toBeNull()} -}); -test('wrong digest, file, current native history and previously seen menu remain denied',()=>{ - for(const edit of [ - (r:any)=>{r.context.pending.sessionId='foreign';},(r:any)=>{r.context.pending.editDigest.beforeSHA256='0'.repeat(64);}, - (r:any)=>{r.context.pending.editDigest.clippedAdditions.lines[0].lineHash='0'.repeat(64);}, - (r:any)=>{r.context.publicTools[1].isError=true;},(r:any)=>{r.context.publicTools=[];}, - (r:any)=>{r.context.viewportCapturedAt=Date.parse(r.context.pending.timestamp)-1;}, - (r:any)=>{fs.appendFileSync(r.file,'changed');fs.utimesSync(r.file,new Date(0),new Date(0));}, - (r:any)=>{r.context.pending.file=r.file.replace('user-dashboard','foreign-dashboard');}, - (r:any)=>{r.context.publicTools.push({kind:'use',name:'Edit',sessionId:r.context.pending.sessionId,toolUseId:'queued',timestamp:new Date().toISOString(),input:{file_path:r.file}});}, - ]){const r=replay();edit(r);expect(r.invoke()).toBeNull()} - const r=replay();expect(r.invoke(fixture.viewport,new Set([autoplanArtifactMenuKey(fixture.viewport)]))).toBeNull();expect(r.invoke(fixture.viewport,new Set([r.context.pending!.sessionId+':'+r.context.pending!.toolUseId]))).toBeNull(); -}); -test('suffix commitments are not body persistence and current replay cannot retain stale hashes',()=>{ - const r=replay(),raw=fs.readFileSync(r.recorder.file,'utf8');for(const text of ['old_string','new_string','Preconditions heading','Snapshot consistency'])expect(raw).not.toContain(text); - recordAutoplanArtifact(JSON.stringify(r.event),r.recorder.file,r.cwd,r.config,r.stateRoot);expect(fs.readFileSync(r.recorder.file,'utf8')).toBe(raw); - r.event.tool_input.new_string+='changed';recordAutoplanArtifact(JSON.stringify(r.event),r.recorder.file,r.cwd,r.config,r.stateRoot); - expect(autoplanArtifactRecorderStatus(r.recorder.file,r.cwd,r.config,r.stateRoot)).toEqual({status:'invalid',reason:'conflicting_replay'}); -}); -test('legacy digest replay is harmless and partial-edge requests do not manufacture suffix authority',()=>{ - const r=replay(),state=JSON.parse(fs.readFileSync(r.recorder.file,'utf8'));delete state.pending.editDigest.clippedAdditions; - fs.writeFileSync(r.recorder.file,JSON.stringify(state)+'\n');const raw=fs.readFileSync(r.recorder.file,'utf8');recordAutoplanArtifact(JSON.stringify(r.event),r.recorder.file,r.cwd,r.config,r.stateRoot);expect(fs.readFileSync(r.recorder.file,'utf8')).toBe(raw); - const q=replay('Prefix Anchor suffix\nAfter one\nAfter two\n','Anchor','New');expect(q.context.pending!.editDigest!.clippedAdditions).toBeUndefined(); -}); -test('malformed, sparse, tampered and excessive suffix records fail closed',()=>{ - for(const edit of [ - (c:any)=>{c.version=2;},(c:any)=>{c.extra=true;},(c:any)=>{c.startLine=0;},(c:any)=>{c.lines=Array(2);}, - (c:any)=>{c.lines[0].suffixHashes=Array(2);},(c:any)=>{c.lines[0].suffixHashes=Array(257).fill('0'.repeat(64));}, - (c:any)=>{c.lines[0].nextLineHash='0'.repeat(64);},(c:any)=>{c.lines[0].line++;}, - ]){const r=replay(),d=r.context.pending!.editDigest!;edit(d.clippedAdditions);expect(validAutoplanEditDigest(d)).toBe(false);expect(r.invoke()).toBeNull()} - const r=replay(),c=r.context.pending!.editDigest!.clippedAdditions;if(c?.status!=='complete')throw Error('missing');const target=c.lines.find(x=>x.line===138)!;target.suffixHashes[1]='0'.repeat(64);expect(r.invoke()).toBeNull(); -}); -test('overflow is explicit for every crop while complete-row legacy authority remains intact',()=>{ - const lines=Array.from({length:40},(_,i)=>'Line '+i+' '+String.fromCharCode(65+i%26).repeat(300));const r=replay('Anchor\nAfter one\nAfter two\n','Anchor',lines.join('\n'));const d=r.context.pending!.editDigest!; - expect(d.clippedAdditions).toEqual({version:1,status:'overflow'});expect(validAutoplanEditDigest(d)).toBe(true); - for(const i of [0,20,39]){const n=i+1,next=i+1{ for(const p of ['test/autoplan-clipped-suffix-aq.test.ts','test/fixtures/autoplan-clipped-suffix-aq.json'])expect(Object.entries(E2E_TOUCHFILES).filter(([,files])=>files.includes(p)).map(([owner])=>owner)).toEqual(['autoplan-chain-pty']); }); diff --git a/test/autoplan-command-prefix-au.test.ts b/test/autoplan-command-prefix-au.test.ts deleted file mode 100644 index c730dc064..000000000 --- a/test/autoplan-command-prefix-au.test.ts +++ /dev/null @@ -1,206 +0,0 @@ -import { capturedPathRebaser } from './helpers/captured-paths'; -import { expect, test } from 'bun:test'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import { createHash } from 'node:crypto'; -import fixture from './fixtures/autoplan-command-prefix-au.json'; -import * as permission from './helpers/autoplan-artifact-permission'; -import { readPendingAutoplanArtifact } from './helpers/autoplan-artifact-recorder'; -import { readPlanCountTranscript, type NativePublicToolEvent } from './helpers/plan-count-transcript'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; - -function replay() { - const root = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-ap-command-')); - const old = path.dirname(path.dirname(fixture.stateRoot)); - const runtime = path.join(root, path.basename(old)), cwd = path.join(root, path.basename(fixture.cwd)); - const rebase = capturedPathRebaser([[old,runtime],[fixture.cwd,cwd]]); - const hook = rebase.json(fixture.hook); - const stateRoot = rebase.file(fixture.stateRoot), config = rebase.file(fixture.config), file = hook.pending.file; - const events = rebase.json(fixture.publicTools) as NativePublicToolEvent[]; - fs.mkdirSync(cwd, { recursive: true }); fs.mkdirSync(path.dirname(file), { recursive: true }); - fs.writeFileSync(file, fixture.before, { mode: fixture.targetStat.mode }); - const mtime = Number(BigInt(fixture.targetStat.mtimeNs)) / 1e9; - fs.utimesSync(file, mtime, mtime); - fs.mkdirSync(path.dirname(hook.pending.transcriptPath), { recursive: true }); - const records = events.map(e => ({ sessionId: e.sessionId, cwd, isSidechain: false, timestamp: e.timestamp, - requestId: e.requestId, message: { id: e.messageId, role: e.kind === 'use' ? 'assistant' : 'user', - content: e.kind === 'use' ? [{ type: 'tool_use', id: e.toolUseId, name: e.name, input: e.input }] - : [{ type: 'tool_result', tool_use_id: e.toolUseId, content: e.content, is_error: e.isError }] } })); - fs.writeFileSync(hook.pending.transcriptPath, records.map(r => JSON.stringify(r)).join('\n') + '\n'); - const hookFile = path.join(root, 'hook.json'); fs.writeFileSync(hookFile, JSON.stringify(hook)); - const publicTools: NativePublicToolEvent[] = []; - const transcript = readPlanCountTranscript(config, cwd, e => publicTools.push(e)); - const now = Date.parse(fixture.viewportCapturedAt), commandStartedAt = Date.parse(fixture.commandTimestamp); - const pending = readPendingAutoplanArtifact(hookFile, cwd, config, stateRoot, commandStartedAt, publicTools, now, true); - const context = { cwd, ownedStateRoot: stateRoot, ownedNativePlansRoot: path.join(config, 'plans'), - commandStartedAt, now, viewportCapturedAt: now, transcriptStatus: transcript.status, publicTools, pending }; - return { root, file, context, viewport: rebase.text(fixture.viewport), dispose: () => fs.rmSync(root, { recursive: true, force: true }) }; -} -type Replay = ReturnType; -const pick = (r: Replay, seen = new Set()) => permission.pendingAutoplanArtifactPermissionInput(r.viewport, r.context, seen); -const panel = (viewport: string) => viewport.slice(viewport.search(/^[─╌]{8,}\n {0,3}Edit file/m)); - -// Exact AY public native prefix; only its owned archive path is relocated onto -// this existing digest fixture. The unpublished Bash body is not reconstructed. -function nativeCards(r: Replay): string { - const relative = path.relative(r.context.ownedStateRoot, r.file).split(path.sep).join('/'); - return [ - `● Update(~/.gstack/${relative})`, ' ', '● Updated plan', ' ', '● Updated plan', ' ', - '● Bash(mkdir -p ~/.gstack/analytics', - ` echo '{"skill":"plan-ceo-review","via":"autoplan","ts":"'$(date -u`, - ` +%Y-%m-%dT%H:%M:%SZ)'","iterations":3,"issues_found":56,"issues_…)`, - ' ⎿  Waiting…', '', '', '', - ].join('\n') + panel(r.viewport); -} - -test('native plan redraws and a queued command preserve only the digest-bound pending Edit', () => { - const r = replay(); try { - r.viewport = nativeCards(r); - const granted = pick(r); - expect(granted).toEqual({ input: '1\r', signature: `${r.context.pending!.sessionId}:${r.context.pending!.toolUseId}`, file: r.file }); - expect(permission.autoplanArtifactPermissionInput(r.viewport, r.context, new Set())).toBeNull(); - expect(permission.publishedAutoplanArtifactPermissionInput(r.viewport, r.context, new Set())).toBeNull(); - expect(pick(r, new Set([granted!.signature]))).toBeNull(); - expect(pick(r, new Set([permission.autoplanArtifactMenuKey(r.viewport)]))).toBeNull(); - } finally { r.dispose(); } -}); - -const nativeScreens: Array<[string, (s: string) => string]> = [ - ['foreign Update title', s => s.replace('Update(~/.gstack/', 'Update(/foreign/')], - ['unbound redraw', s => s.replace('● Updated plan', '● Updated another file')], - ['second Update', s => s.replace('● Updated plan', '● Update(/foreign/plan.md)')], - ['second Bash', s => s.replace('● Updated plan', '● Bash(echo another…)')], - ['no native redraw', s => s.replaceAll('● Updated plan', '')], - ['completed command', s => s.replace('Waiting…', 'Done')], - ['missing Waiting marker', s => s.replace(' ⎿  Waiting…', '')], - ['unclosed command card', s => s.replace('"issues_…)', '"issues_…')], - ['unindented command continuation', s => s.replace(' echo ', 'echo ')], - ['competing permission', s => s.replace(' echo ', ' Do you want to proceed? ')], - ['indented native action', s => s.replace(' echo ', ' ● Read ')], - ['indented question', s => s.replace(' echo ', ' ❯ 1. ')], - ['source prefix', s => 'Source:\n' + s], - ['quoted pane', s => s.split('\n').map(row => '> ' + row).join('\n')], - ['code pane', s => '```text\n' + s + '\n```'], - ['second edit panel', s => s + '\n' + panel(s)], - ['foreign active panel', s => s.replace('projects/gstack-autoplan-chain-zmFsqo/', 'projects/foreign/')], - ['foreign menu', s => s.replace('edit to 2026-09-10-user-dashboard.md?', 'edit to other.md?')], - ['persistent approval', s => s.replace('❯ 1. Yes', '❯ 2. Yes')], - ['changed digest-bound addition', s => s.replace('zero before advancing', 'ten before advancing')], -]; -for (const [name, change] of nativeScreens) test(`native batch cards cannot hide another authority: ${name}`, () => { - const r = replay(); try { r.viewport = change(nativeCards(r)); expect(pick(r)).toBeNull(); } finally { r.dispose(); } -}); - -test('the exact public command display preserves only the current unpublished Edit approval', () => { - const r = replay(); try { - expect(r.context.transcriptStatus).toBe('ready'); expect(r.context.publicTools).toHaveLength(2); - expect(r.context.pending?.toolUseId).toBe(fixture.hook.pending.toolUseId); - expect(r.context.publicTools.some(e => e.toolUseId === r.context.pending?.toolUseId)).toBe(false); - expect(createHash('sha256').update(fs.readFileSync(r.file)).digest('hex')).toBe(fixture.provenance.beforeSHA256); - expect(Math.floor(fs.statSync(r.file).mtimeMs)).toBe(Number(BigInt(fixture.targetStat.mtimeNs) / 1_000_000n)); - expect(permission.autoplanArtifactPermissionInput(r.viewport, r.context, new Set())).toBeNull(); - expect(permission.publishedAutoplanArtifactPermissionInput(r.viewport, r.context, new Set())).toBeNull(); - const expected = { input: '1\r', signature: `${fixture.hook.sessionId}:${fixture.hook.pending.toolUseId}`, file: r.file }; - expect(pick(r)).toEqual(expected); - expect(pick(r, new Set([expected.signature]))).toBeNull(); - expect(pick(r, new Set([permission.autoplanArtifactMenuKey(r.viewport)]))).toBeNull(); - r.viewport = panel(r.viewport); expect(pick(r)).toEqual(expected); - expect(fixture.provenance.paidOutcomesReclassified).toBe(false); - } finally { r.dispose(); } -}); - -test('a plain native command description and wrapped display supply no command authority', () => { - for (const prefix of ['● Recording review metrics\n ⎿ $ echo recorded\n\n', - '⏺ Running local diagnostics\n ⎿ $ bun test\n echo finished\n\n']) { - const r = replay(); try { r.viewport = prefix + panel(r.viewport); expect(pick(r)?.input).toBe('1\r'); } - finally { r.dispose(); } - } -}); - -const screens: Array<[string, (s: string) => string]> = [ - ['source introduction', s => 'Source:\n' + s], ['example introduction', s => 'Example:\n' + s], - ['whole quotation', s => s.split('\n').map(line => '> ' + line).join('\n')], - ['whole code block', s => '```text\n' + s + '\n```'], - ['quoted title', s => s.replace('● Appending spec-review metrics', '● "Appending spec-review metrics"')], - ['source title', s => s.replace('● Appending spec-review metrics', '● Source: an example command')], - ['second native action', s => s.replace(' echo logged', '● Another tool\n ⎿ $ echo other')], - ['indented second action', s => s.replace(' echo logged', ' ● Another tool')], - ['Bash confirmation', s => s.replace(' echo logged', ' Do you want to proceed?')], - ['Bash permission', s => s.replace(' echo logged', ' Bash command requires permission')], - ['second question', s => s.replace(' echo logged', ' ❯ 1. Approve this command')], - ['missing command marker', s => s.replace('⎿ $', '⎿ ')], - ['unbound command prose', s => s.replace(' echo logged', 'Unrelated current prose')], - ['second edit panel', s => s + '\n' + panel(s)], - ['foreign panel path', s => s.replace('projects/gstack-autoplan-chain-zmFsqo/', 'projects/another-project/')], - ['basename-only panel', s => s.replace(/^ …[^\n]+$/m, ' 2026-09-10-user-dashboard.md')], - ['foreign menu target', s => s.replace('edit to 2026-09-10-user-dashboard.md?', 'edit to another.md?')], - ['session approval cursor', s => s.replace('❯ 1. Yes', '❯ 2. Yes')], - ['extra current prompt', s => s + '\nChoose another action'], - ['changed added rows', s => s.replace('zero before advancing', 'ten before advancing')], - ['removed-line gap', s => s.replace(' 98 -', ' 100 -')], - ['added-line gap', s => s.replace(' 98 +', ' 100 +')], - ['different reset start', s => s.replace(' 97 +', ' 96 +')], - ['duplicate removed row', s => s.replace(/(^ 98 -[^\n]*\n)/m, '$1$1')], - ['duplicate added row', s => s.replace(/(^ 98 \+[^\n]*\n)/m, '$1$1')], - ['multiple resets', s => s.replace(' 108 5.', s.slice(s.indexOf(' 97 -'), s.indexOf(' 108 5.')) + ' 108 5.')], - ['truncated removed block', s => s.replace(/^ 99 -[^\n]*\n/m, '')], - ['truncated added block', s => s.replace(/^ 107 \+[^\n]*\n/m, '')], - ['missing panel separator', s => s.replace(/^[─]{8,}\n/m, '')], -]; -for (const [name, change] of screens) test(`current panel remains unambiguous: ${name}`, () => { - const r = replay(); try { r.viewport = change(r.viewport); expect(pick(r)).toBeNull(); } finally { r.dispose(); } -}); - -const bindings: Array<[string, (r: Replay) => void]> = [ - ['missing hook', r => { r.context.pending = undefined; }], - ['wrong hook tool', r => { r.context.pending!.tool = 'Write' as 'Edit'; }], - ['foreign hook session', r => { r.context.pending!.sessionId = 'foreign'; }], - ['foreign hook path', r => { r.context.pending!.file += '.other'; }], - ['missing digest', r => { delete r.context.pending!.editDigest; }], - ['invalid digest', r => { r.context.pending!.editDigest!.beforeSHA256 = 'invalid'; }], - ['wrong before digest', r => { r.context.pending!.editDigest!.beforeSHA256 = '0'.repeat(64); }], - ['wrong addition commitments', r => { r.context.pending!.editDigest!.newLineHashes.fill('0'.repeat(64)); }], - ['current file changed', r => { fs.appendFileSync(r.file, '\nchanged'); fs.utimesSync(r.file, new Date(0), new Date(0)); }], - ['file newer than pending', r => { fs.utimesSync(r.file, new Date(r.context.now), new Date(r.context.now)); }], - ['history is Read', r => { r.context.publicTools[0]!.name = 'Read'; }], - ['foreign history file', r => { r.context.publicTools[0]!.input!.file_path = r.file + '.other'; }], - ['failed history', r => { r.context.publicTools[1]!.isError = true; }], - ['unresolved mutation', r => { r.context.publicTools.pop(); }], - ['published pending request', r => { r.context.publicTools.push({ kind: 'use', name: 'Edit', sessionId: r.context.pending!.sessionId, - toolUseId: r.context.pending!.toolUseId, timestamp: r.context.pending!.timestamp, input: { file_path: r.file } }); }], - ['missing transcript', r => { r.context.transcriptStatus = 'missing'; }], - ['future hook', r => { r.context.pending!.timestamp = new Date(r.context.now + 1).toISOString(); }], - ['viewport predates hook', r => { r.context.viewportCapturedAt = Date.parse(r.context.pending!.timestamp) - 1; }], - ['command after hook', r => { r.context.commandStartedAt = Date.parse(r.context.pending!.timestamp) + 1; }], -]; -for (const [name, change] of bindings) test(`pending authorization is retained: ${name}`, () => { - const r = replay(); try { change(r); expect(pick(r)).toBeNull(); } finally { r.dispose(); } -}); -for (const [name, change] of bindings) test(`native cards retain pending authorization: ${name}`, () => { - const r = replay(); try { r.viewport = nativeCards(r); change(r); expect(pick(r)).toBeNull(); } finally { r.dispose(); } -}); -test('only the Autoplan workflow selects this fixture and behavioral regression', () => { - for (const file of ['test/autoplan-command-prefix-au.test.ts', 'test/fixtures/autoplan-command-prefix-au.json']) - expect(selectTests([file], E2E_TOUCHFILES, []).selected).toEqual(['autoplan-chain-pty']); -}); - -test('removed row order is bound to both the current file and pending digest', () => { - const r = replay(); try { - const rows = r.viewport.split('\n'), a = rows.findIndex(row => /^ 97 -/.test(row)), b = rows.findIndex(row => /^ 98 -/.test(row)); - expect(a).toBeGreaterThan(0); expect(b).toBe(a + 1); - const first = rows[a]!.slice(6), second = rows[b]!.slice(6); - rows[a] = rows[a]!.slice(0, 6) + second; rows[b] = rows[b]!.slice(0, 6) + first; - r.viewport = rows.join('\n'); expect(pick(r)).toBeNull(); - } finally { r.dispose(); } -}); - -test('added row order is bound to the complete pending replacement digest', () => { - const r = replay(); try { - const rows = r.viewport.split('\n'), a = rows.findIndex(row => /^ 97 \+/.test(row)), b = rows.findIndex(row => /^ 98 \+/.test(row)); - expect(a).toBeGreaterThan(0); expect(b).toBe(a + 1); - const first = rows[a]!.slice(6), second = rows[b]!.slice(6); - rows[a] = rows[a]!.slice(0, 6) + second; rows[b] = rows[b]!.slice(0, 6) + first; - r.viewport = rows.join('\n'); expect(pick(r)).toBeNull(); - } finally { r.dispose(); } -}); diff --git a/test/autoplan-cropped-command-av.test.ts b/test/autoplan-cropped-command-av.test.ts deleted file mode 100644 index 85645745b..000000000 --- a/test/autoplan-cropped-command-av.test.ts +++ /dev/null @@ -1,116 +0,0 @@ -import {expect,test} from 'bun:test'; -import fs from 'node:fs';import os from 'node:os';import path from 'node:path';import {createHash} from 'node:crypto'; -import fixture from './fixtures/autoplan-cropped-command-av.json'; -import * as permission from './helpers/autoplan-artifact-permission'; -import {E2E_TOUCHFILES,LLM_JUDGE_TOUCHFILES,selectTests} from './helpers/touchfiles'; -type Context=Parameters[1]; -function replay(){ - const root=fs.mkdtempSync(path.join(os.tmpdir(),'gstack-cropped-command-')); - const replace=(s:string)=>s.replaceAll(path.dirname(fixture.context.cwd),root); - const context=JSON.parse(replace(JSON.stringify(fixture.context))) as Context; - const nativePlan=replace(fixture.nativePlan.path),file=context.pending!.file; - for(const [name,body,mtime] of [[file,fixture.before,fixture.beforeMtimeMs],[nativePlan,fixture.nativePlan.text,fixture.nativePlan.mtimeMs]] as const){ - fs.mkdirSync(path.dirname(name),{recursive:true});fs.writeFileSync(name,body);fs.utimesSync(name,mtime/1000,mtime/1000); - } - fs.mkdirSync(context.cwd,{recursive:true}); - return {root,file,nativePlan,context,viewport:replace(fixture.viewport),dispose:()=>fs.rmSync(root,{recursive:true,force:true})}; -} -type Replay=ReturnType; -const invoke=(r:Replay,seen=new Set())=>permission.publishedAutoplanArtifactPermissionInput(r.viewport,r.context,seen); -const current=(r:Replay)=>r.context.publicTools.find(e=>e.kind==='use'&&e.toolUseId===r.context.pending!.toolUseId)!; -const queued=(r:Replay)=>r.context.publicTools.find(e=>e.kind==='use'&&e.name==='Edit'&&e.input?.file_path===r.nativePlan&& - !r.context.publicTools.some(result=>result.kind==='result'&&result.toolUseId===e.toolUseId))!; -const bash=(r:Replay)=>r.context.publicTools.find(e=>e.kind==='use'&&e.name==='Bash')!; -const complete=(r:Replay,e:ReturnType,isError=false)=>r.context.publicTools.push({kind:'result',sessionId:e.sessionId, - toolUseId:e.toolUseId,timestamp:new Date(r.context.now!).toISOString(),isError,content:'completed'}); -const panel=(r:Replay)=>r.viewport.slice(r.viewport.search(/^[─╌]{8,}\n {0,3}Edit file/m)); -const show=(r:Replay,command:string,rows=[command])=>{bash(r).input!.command=command;r.viewport=' ⎿ $ '+rows.join('\n ')+'\n\n'+panel(r)}; - -test('the retained captionless queued command grants only the current digest-bound Edit',()=>{const r=replay();try{ - expect(r.context.publicTools).toHaveLength(7); - expect(createHash('sha256').update(fs.readFileSync(r.file)).digest('hex')).toBe(fixture.beforeSha256); - const expected={input:'1\r',signature:fixture.context.pending.sessionId+':'+fixture.context.pending.toolUseId,file:r.file}; - expect(permission.autoplanArtifactPermissionInput(r.viewport,r.context,new Set())).toBeNull(); - expect(permission.pendingAutoplanArtifactPermissionInput(r.viewport,r.context,new Set())).toBeNull(); - expect(invoke(r)).toEqual(expected); - expect(invoke(r,new Set([expected.signature]))).toBeNull(); - expect(invoke(r,new Set([permission.autoplanArtifactMenuKey(r.viewport)]))).toBeNull(); - r.viewport=panel(r);expect(invoke(r)).toEqual(expected); - expect(fixture.provenance.paidOutcomesReclassified).toBe(false); -}finally{r.dispose()}}); - -for(const [name,change] of [ - ['single row',(r:Replay)=>show(r,bash(r).input!.command)], - ['different soft wrap',(r:Replay)=>{const command=bash(r).input!.command as string;const at=command.indexOf(' && ');show(r,command,[command.slice(0,at),command.slice(at+1)])}], - ['CRLF renderer',(r:Replay)=>{r.viewport=r.viewport.replaceAll('\n','\r\n')}], - ['nonbreaking native gutter',(r:Replay)=>{r.viewport=r.viewport.replace('⎿ $','⎿\u00a0 $')}], - ['quoted argument with literal spaces',(r:Replay)=>show(r,"printf '%s' 'two words'")], - ['soft wrap inside a quoted argument',(r:Replay)=>show(r,"printf '%s' 'two words'",["printf '%s' 'two","words'"])], -] as const)test(`complete public command binding accepts ${name}`,()=>{const r=replay();try{change(r);expect(invoke(r)?.signature).toBe(`${r.context.pending!.sessionId}:${r.context.pending!.toolUseId}`)}finally{r.dispose()}}); - -const identityCases:Array<[string,(r:Replay)=>void]>=[ - ['missing Bash publication',r=>{r.context.publicTools=r.context.publicTools.filter(e=>e!==bash(r))}], - ['foreign Bash message',r=>{bash(r).messageId='msg_foreign'}],['foreign Bash request',r=>{bash(r).requestId='req_foreign'}], - ['foreign Bash session',r=>{bash(r).sessionId='foreign'}],['Bash with no identity',r=>{bash(r).toolUseId=''}], - ['different command',r=>{bash(r).input!.command+=' && echo other'}],['missing command',r=>{delete bash(r).input!.command}], - ['multiline command',r=>{bash(r).input!.command+='\n'}],['control byte in command',r=>{bash(r).input!.command+='\x1b'}], - ['another tool name',r=>{bash(r).name='Read'}],['started command',r=>{r.context.pending!.hookSeenIds!.push(bash(r).toolUseId)}], - ['completed command',r=>complete(r,bash(r))],['failed command',r=>complete(r,bash(r),true)], - ['ambiguous queued commands',r=>{r.context.publicTools.push({...structuredClone(bash(r)),toolUseId:'toolu_duplicate'})}], - ['second unmatched queued command',r=>{r.context.publicTools.push({...structuredClone(bash(r)),toolUseId:'toolu_other',input:{command:'echo other'}})}], - ['command after viewport',r=>{r.context.viewportCapturedAt=Date.parse(bash(r).timestamp)-1}], - ['command before queued Edit',r=>{const e=bash(r),q=queued(r),at=r.context.publicTools.indexOf(q);e.timestamp=current(r).timestamp;r.context.publicTools.pop();r.context.publicTools.splice(at,0,e)}], - ['no queued mutation',r=>{const q=queued(r);r.context.publicTools=r.context.publicTools.filter(e=>e!==q)}], - ['foreign queued mutation path',r=>{queued(r).input!.file_path='/tmp/foreign.md'}], - ['foreign queued message',r=>{queued(r).messageId='msg_foreign'}],['foreign queued request',r=>{queued(r).requestId='req_foreign'}], - ['queued Write',r=>{queued(r).name='Write'}],['queued replace all',r=>{queued(r).input!.replace_all=true}], - ['started queued Edit',r=>{r.context.pending!.hookSeenIds!.push(queued(r).toolUseId)}], - ['completed queued Edit',r=>complete(r,queued(r))], - ['failed native-plan history',r=>{const previous=r.context.publicTools.find(e=>e.kind==='use'&&e.input?.file_path===r.nativePlan&&e!==queued(r))!;r.context.publicTools.find(e=>e.kind==='result'&&e.toolUseId===previous.toolUseId)!.isError=true}], - ['Read is not native-plan mutation history',r=>{r.context.publicTools.find(e=>e.kind==='use'&&e.input?.file_path===r.nativePlan&&e!==queued(r))!.name='Read'}], - ['native plan modified after hook',r=>{fs.utimesSync(r.nativePlan,new Date(r.context.now!),new Date(r.context.now!))}], - ['foreign native-plan root',r=>{r.context.ownedNativePlansRoot=path.join(r.root,'foreign')}], - ['missing hook',r=>{r.context.pending=undefined}],['missing current publication',r=>{const c=current(r);r.context.publicTools=r.context.publicTools.filter(e=>e!==c)}], - ['foreign current message',r=>{current(r).messageId='msg_foreign'}],['foreign current request',r=>{current(r).requestId='req_foreign'}], - ['foreign current session',r=>{current(r).sessionId='foreign'}],['completed current Edit',r=>complete(r,current(r))], - ['current request changed',r=>{current(r).input!.new_string+=' changed'}], - ['missing digest',r=>{delete r.context.pending!.editDigest}],['wrong request digest',r=>{r.context.pending!.editDigest!.requestSHA256='0'.repeat(64)}], - ['wrong before digest',r=>{r.context.pending!.editDigest!.beforeSHA256='0'.repeat(64)}], - ['file changed',r=>{fs.appendFileSync(r.file,'changed');fs.utimesSync(r.file,0,0)}], - ['file modified after hook',r=>{fs.utimesSync(r.file,new Date(r.context.now!),new Date(r.context.now!))}], - ['missing native transcript',r=>{r.context.transcriptStatus='missing'}],['wrong pending identity',r=>{r.context.pending!.toolUseId='toolu_other'}], - ['failed archive history',r=>{r.context.publicTools.find(e=>e.kind==='result')!.isError=true}], - ['command before launched review',r=>{r.context.commandStartedAt=r.context.now!+1}], -]; -for(const[name,change]of identityCases)test(`caption crop retains native authority: ${name}`,()=>{const r=replay();try{change(r);expect(invoke(r)).toBeNull()}finally{r.dispose()}}); - -const displayCases:Array<[string,(r:Replay)=>void]>=[ - ['example introduction',r=>{r.viewport='Example:\n'+r.viewport}],['historical introduction',r=>{r.viewport='Historical screen:\n'+r.viewport}], - ['quoted display',r=>{r.viewport=r.viewport.split('\n').map(line=>'> '+line).join('\n')}], - ['fenced display',r=>{r.viewport='```text\n'+r.viewport+'\n```'}], - ['caption instead of native cropped prefix',r=>{r.viewport='● Approve everything\n'+r.viewport}], - ['missing dollar marker',r=>{r.viewport=r.viewport.replace('⎿ $','⎿ ')}], - ['different command prefix',r=>{r.viewport=r.viewport.replace('mkdir -p','mkdir -m 777 -p')}], - ['truncated command',r=>{r.viewport=r.viewport.replace('&& echo logged','…')}], - ['extra command suffix',r=>{r.viewport=r.viewport.replace('&& echo logged','&& echo logged; echo other')}], - ['missing wrapped row',r=>{r.viewport=r.viewport.split('\n').filter((_,i)=>i!==1).join('\n')}], - ['blank row in command',r=>{r.viewport=r.viewport.replace('\n +%','\n\n +%')}], - ['extra non-command row',r=>{r.viewport=r.viewport.replace('\n \n','\n completed successfully\n')}], - ['second dollar command',r=>{r.viewport=r.viewport.replace('\n \n','\n ⎿ $ echo other\n')}], - ['Bash approval menu',r=>{r.viewport='Bash command permission\nDo you want to run this command?\n'+r.viewport}], - ['duplicate Edit panel',r=>{r.viewport+=panel(r)}], - ['foreign Edit target',r=>{r.viewport=r.viewport.replace('gstack-autoplan-chain-ZdZS9F','gstack-autoplan-chain-foreign')}], - ['altered added diff row',r=>{r.viewport=r.viewport.replace('server clock','attacker clock')}], - ['persistent permission selected',r=>{r.viewport=r.viewport.replace('❯ 1. Yes','❯ 2. Yes')}], - ['trailing unrelated prose',r=>{r.viewport+='\nAnother current request'}], - ['within-row quoted whitespace contradiction',r=>{show(r,"printf '%s' 'two words'");r.viewport=r.viewport.replace('two words','two words')}], - ['within-row unquoted whitespace contradiction',r=>{r.viewport=r.viewport.replace('mkdir -p','mkdir -p')}], -]; -for(const[name,change]of displayCases)test(`caption crop rejects unrelated display: ${name}`,()=>{const r=replay();try{change(r);expect(invoke(r)).toBeNull()}finally{r.dispose()}}); - -test('only Autoplan selects the public fixture and focused regression',()=>{ - for(const file of ['test/autoplan-cropped-command-av.test.ts','test/fixtures/autoplan-cropped-command-av.json']){ - expect(selectTests([file],E2E_TOUCHFILES,[]).selected).toEqual(['autoplan-chain-pty']); - expect(selectTests([file],LLM_JUDGE_TOUCHFILES,[]).selected).toEqual([]); - } -}); diff --git a/test/autoplan-edit-digests-al.test.ts b/test/autoplan-edit-digests-al.test.ts index f87e5927a..e3f4f130d 100644 --- a/test/autoplan-edit-digests-al.test.ts +++ b/test/autoplan-edit-digests-al.test.ts @@ -2,8 +2,7 @@ import {test,expect,afterEach} from 'bun:test'; import fs from 'node:fs';import os from 'node:os';import path from 'node:path';import {spawnSync} from 'node:child_process'; import fixture from './fixtures/autoplan-edit-digests-al.json'; import {createAutoplanArtifactRecorder,recordAutoplanArtifact,readPendingAutoplanArtifact,autoplanArtifactRecorderStatus} from './helpers/autoplan-artifact-recorder'; -import {pendingAutoplanArtifactPermissionInput,autoplanArtifactMenuKey} from './helpers/autoplan-artifact-permission'; -import {createAutoplanEditDigest,validAutoplanEditDigest,autoplanEditLineHash} from './helpers/autoplan-artifact-digest'; +import {createAutoplanEditDigest,validAutoplanEditDigest} from './helpers/autoplan-artifact-digest'; import type {NativePublicToolEvent} from './helpers/plan-count-transcript'; import {E2E_TOUCHFILES,selectTests} from './helpers/touchfiles'; const cleanups:Array<()=>void>=[];afterEach(()=>{for(const cleanup of cleanups.splice(0))cleanup()}); @@ -21,12 +20,7 @@ function replay(record=true) { const context={cwd,ownedStateRoot,commandStartedAt:startedAt,transcriptStatus:'ready',publicTools:history,pending,viewportCapturedAt:Date.now(),now:Date.now()+1000}; return {root,file,config,recorder,event,context,viewport:fixture.viewport}; } -const pick=(r:ReturnType,seen=new Set())=>pendingAutoplanArtifactPermissionInput(r.viewport,r.context,seen); -test('actual added-only three-digit pane rejects without request digests, accepts a separately recorded reconstructed insertion',()=>{ - const r=replay();expect(r.context.pending?.editDigest).toBeDefined();expect(pick(r)?.input).toBe('1\r'); - delete r.context.pending!.editDigest;expect(pick(r)).toBeNull(); - expect(fixture.provenance.reconstruction).toContain('not the original'); -}); + test('hook persists bounded digests from its input, never request or result text',()=>{ const r=replay(false),hook=r.recorder.hooks.PreToolUse[0]!.hooks[0]!; const child=spawnSync('bash',['-c',hook.command],{input:JSON.stringify({...r.event,tool_response:'PRIVATE_RESULT_SENTINEL'}),encoding:'utf8',timeout:6000}); @@ -38,58 +32,11 @@ test('hook persists bounded digests from its input, never request or result text const legacy=structuredClone(state);delete legacy.pending.editDigest.clippedAdditions; expect(Buffer.byteLength(JSON.stringify(legacy))).toBeLessThan(64*1024); }); -test('digests of a different request cannot authorize the displayed additions',()=>{ - const r=replay();r.context.pending!.editDigest=createAutoplanEditDigest(r.file,'Owner: the user.\n','Owner: the user.\nDifferent requested insertion.\n')!;expect(pick(r)).toBeNull(); - r.context.pending!.editDigest=createAutoplanEditDigest(r.file,r.event.tool_input.old_string,r.event.tool_input.new_string)!; - r.context.pending!.editDigest.newLineHashes=r.context.pending!.editDigest.oldLineHashes;expect(pick(r)).toBeNull(); -}); -test('current file hash, native identity, predecessor and single use stay required',()=>{ - const mutations:Array<(r:ReturnType)=>void>=[ - r=>{r.context.pending!.sessionId='foreign';},r=>{r.context.pending!.file=path.join(r.root,'foreign.md');}, - r=>{r.context.pending!.editDigest!.beforeSHA256='0'.repeat(64);}, - r=>{fs.writeFileSync(r.file,fixture.before+'Changed concurrently.');const old=new Date(0);fs.utimesSync(r.file,old,old);}, - r=>{r.context.pending!.timestamp=new Date(r.context.now+1000).toISOString();},r=>{r.context.viewportCapturedAt=Date.parse(r.context.pending!.timestamp)-1;}, - r=>{r.context.commandStartedAt=r.context.now+1;},r=>{r.context.publicTools[1]!.isError=true;},r=>{r.context.publicTools=[];}, - r=>{r.context.publicTools.push({kind:'result',sessionId:r.context.pending!.sessionId,toolUseId:r.context.pending!.toolUseId,timestamp:new Date(r.context.now).toISOString(),isError:false});}, - r=>{r.context.publicTools.push({kind:'use',name:'Write',sessionId:r.context.pending!.sessionId,toolUseId:'successor',timestamp:new Date(r.context.now).toISOString(),input:{file_path:r.file}});}, - ];for(const change of mutations){const r=replay();change(r);expect(pick(r)).toBeNull()} - const r=replay();expect(pick(r,new Set([r.context.pending!.sessionId+':'+r.context.pending!.toolUseId]))).toBeNull();expect(pick(r,new Set([autoplanArtifactMenuKey(r.viewport)]))).toBeNull(); -}); -test('exact three-digit marker column rejects wrong gutters, arbitrary source rows and malformed numbering',()=>{ - for(const change of [ - (s:string)=>s.replace(/^ \+/m,' +'),(s:string)=>s.replace(/^ \+/m,' +'), - (s:string)=>s.replace(/^ \+/m,' Source: '),(s:string)=>'> quoted example\n'+s, - (s:string)=>s.replace(/^ 140 /m,' 0 '),(s:string)=>s.replace(/^ 141 /m,' 139 '), - (s:string)=>s.replace(/^ 140 /m,' 999999999999999999999 '),(s:string)=>s.replace(/^ 140 \+/m,' 140 -'), - (s:string)=>s.replace('3. No','3. Maybe'),(s:string)=>s.replace('❯ 1. Yes','❯ 2. Yes'), - (s:string)=>s.replace('2026-09-10-user-dashboard.md?','foreign.md?'),(s:string)=>s+'\nUnrelated prompt', - ]){const r=replay();r.viewport=change(r.viewport);expect(pick(r)).toBeNull()} -}); -test('four-space continuation is accepted only with the matching two-digit numbered gutter',()=>{ - const r=replay();r.viewport=r.viewport.replace(/^ (1[4][0-9]) /gm,(_,n)=>' '+(Number(n)-130)+' ').replace(/^ ([+ -])/gm,' $1');expect(pick(r)?.input).toBe('1\r'); -}); -test('original or context rows cannot supply insertion authority',()=>{ - const r=replay(),menu=r.viewport.slice(r.viewport.indexOf('╌')); - r.context.pending!.editDigest=createAutoplanEditDigest(r.file,'Owner: the user.\n','Owner: the user.\nNew actual request.\n')!; - r.viewport=' 140 +Owner: the user.\n 141 +Owner: the user.\n'+menu;expect(pick(r)).toBeNull(); - r.viewport=' 140 Owner: the user.\n 141 Owner: the user.\n'+menu;expect(pick(r)).toBeNull(); -}); -test('malformed, sparse and high-volume persisted digest records fail closed',()=>{ - for(const change of [(d:any)=>{d.version=2},(d:any)=>{d.extra='text'},(d:any)=>{d.beforeSHA256='bad'},(d:any)=>{d.newLineHashes=[]},(d:any)=>{d.newLineHashes=Array(513).fill('a'.repeat(64))},(d:any)=>{d.oldLineHashes[0]=null}]){ - const r=replay(),s=JSON.parse(fs.readFileSync(r.recorder.file,'utf8'));change(s.pending.editDigest);fs.writeFileSync(r.recorder.file,JSON.stringify(s));expect(autoplanArtifactRecorderStatus(r.recorder.file,r.context.cwd,r.config,r.context.ownedStateRoot).status).toBe('invalid'); - r.context.pending!.editDigest=s.pending.editDigest;expect(pick(r)).toBeNull(); - } - const r=replay(),sparse={...r.context.pending!.editDigest!,newLineHashes:Array(2)};expect(validAutoplanEditDigest(sparse)).toBe(false); -}); test('unavailable or oversized before/request data yields no new digest authority',()=>{ const r=replay();expect(createAutoplanEditDigest(r.file,'missing original','new')).toBeUndefined();expect(createAutoplanEditDigest(r.file,'Owner: the user.\n','x\n'.repeat(513))).toBeUndefined(); const link=path.join(r.root,'linked');fs.symlinkSync(r.file,link);expect(createAutoplanEditDigest(link,r.event.tool_input.old_string,r.event.tool_input.new_string)).toBeUndefined(); fs.writeFileSync(r.file,'x'.repeat(1024*1024+1));expect(createAutoplanEditDigest(r.file,'x','new')).toBeUndefined();fs.unlinkSync(r.file);expect(createAutoplanEditDigest(r.file,'old','new')).toBeUndefined(); }); -test('normalization joins display wrapping but keeps changed nonwhitespace bytes distinct',()=>{ - expect(autoplanEditLineHash('same body\t')).toBe(autoplanEditLineHash('samebody'));expect(autoplanEditLineHash('same body')).not.toBe(autoplanEditLineHash('different body')); - const r=replay();r.viewport=r.viewport.replace('Toast stacking','Toast stacKING');expect(pick(r)).toBeNull(); -}); test('Eng and Autoplan share the digest helper and regression evidence',()=>{ const owner=E2E_TOUCHFILES['autoplan-chain-pty']!;for(let i=0;i{ - const r=replay(false),before='First original full line\nOld second\nContext\n'; - fs.writeFileSync(r.file,before);const old=new Date(Date.parse(fixture.pending.timestamp)-1000);fs.utimesSync(r.file,old,old); - for(const e of r.context.publicTools)if(e.name==='Write'&&e.input?.file_path===r.file)e.input.content=before; - r.event.tool_input.old_string=c.removed;r.event.tool_input.new_string=c.added; - const child=spawnSync('bash',['-c',r.recorder.hooks.PreToolUse[0]!.hooks[0]!.command],{input:JSON.stringify(r.event),encoding:'utf8',timeout:6000}); - expect(child.status).toBe(0);expect(child.stdout).toBe('');expect(child.stderr).toBe(''); - r.context.pending=readPendingAutoplanArtifact(r.recorder.file,r.context.cwd,r.config,r.context.ownedStateRoot,r.context.commandStartedAt,r.context.publicTools); - r.context.viewportCapturedAt=Date.now();r.context.now=Date.now()+1000; - const menu=r.viewport.slice(r.viewport.indexOf('Do you want to make this edit')); - r.viewport=c.rows.join('\n')+'\n'+'╌'.repeat(20)+'\n'+menu; - expect(validAutoplanEditDigest(r.context.pending?.editDigest)).toBe(true); - const digest=structuredClone(r.context.pending!.editDigest!); - expect(pick(r)?.input).toBe('1\r'); - delete r.context.pending!.editDigest;expect(pick(r)?.input).toBe('1\r'); - r.context.pending!.editDigest={...digest,beforeSHA256:'0'.repeat(64)};expect(pick(r)).toBeNull(); - r.context.pending!.editDigest={...digest,beforeSHA256:'malformed'};expect(pick(r)).toBeNull(); - r.context.pending!.editDigest=digest; - const viewport=r.viewport;r.viewport=r.viewport.replace(/^((?: {0,3}\d+ | {4})-).*$/gm,'$1Foreign unowned deletion');expect(pick(r)).toBeNull();r.viewport=viewport; - // The digest's request ownership remains binding through the legacy crop path. - r.context.pending!.editDigest={...digest,oldLineHashes:[autoplanEditLineHash('Context')]};expect(pick(r)).toBeNull(); - r.context.pending!.editDigest=digest; - if(c.rows.some(row=>/^[ ]*\d+ \+/.test(row))){ - r.viewport=viewport.replace(/^([ ]*\d+ \+).*$/gm,'$1Context');expect(pick(r)).toBeNull();r.viewport=viewport; - } - if(c.name==='leading partial deletion'){ - r.viewport=viewport.replace(' -full line',' -Context');expect(pick(r)).toBeNull();r.viewport=viewport; - } - if(c.name==='leading partial context'){ - r.viewport=viewport.replace(' full line',' +full line');expect(pick(r)).toBeNull();r.viewport=viewport; - } - fs.unlinkSync(r.file);expect(pick(r)).toBeNull(); -}); diff --git a/test/autoplan-edit-edges-an.test.ts b/test/autoplan-edit-edges-an.test.ts deleted file mode 100644 index df5e48ab5..000000000 --- a/test/autoplan-edit-edges-an.test.ts +++ /dev/null @@ -1,121 +0,0 @@ -import { capturedPathRebaser } from './helpers/captured-paths'; -import {expect,test} from 'bun:test'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import fixture from './fixtures/autoplan-edit-edges-an.json'; -import * as permission from './helpers/autoplan-artifact-permission'; -import {readPendingAutoplanArtifact} from './helpers/autoplan-artifact-recorder'; -import {createAutoplanEditDigest} from './helpers/autoplan-artifact-digest'; -import {readPlanCountTranscript,type NativePublicToolEvent} from './helpers/plan-count-transcript'; -import {E2E_TOUCHFILES,selectTests} from './helpers/touchfiles'; - -function setup(changeRecords?:(records:any[])=>void){ - const dir=fs.mkdtempSync(path.join(os.tmpdir(),'gstack-ap-edges-')); - const cwd=path.join(dir,path.basename(fixture.cwd)),config=path.join(dir,'config'); - const stateRoot=path.join(dir,'gstack-hermetic-2546450-gfwm4G/skill-home-k7zGB1/.gstack'); - const rebase=capturedPathRebaser([[fixture.stateRoot,stateRoot],[fixture.cwd,cwd],[fixture.config,config]]); - const hook=rebase.json(fixture.hook); - const file=hook.pending.file,nativeFile=path.join(config,'projects','owned',hook.sessionId+'.jsonl');hook.pending.transcriptPath=nativeFile; - fs.mkdirSync(cwd,{recursive:true});fs.mkdirSync(path.dirname(file),{recursive:true});fs.mkdirSync(path.dirname(nativeFile),{recursive:true}); - fs.writeFileSync(file,fixture.before);fs.utimesSync(file,new Date(fixture.now-1000000),new Date(Date.parse(hook.pending.timestamp)-1000)); - const events=rebase.json(fixture.publicTools) as (NativePublicToolEvent & {messageId?:string;requestId?:string})[]; - const records=events.map(e=>({sessionId:e.sessionId,cwd,isSidechain:false,timestamp:e.timestamp,requestId:e.requestId,message:{id:e.messageId,role:e.kind==='use'?'assistant':'user',content:e.kind==='use'?[{type:'tool_use',id:e.toolUseId,name:e.name,input:e.input}]:[{type:'tool_result',tool_use_id:e.toolUseId,content:'',is_error:e.isError}]}})); - changeRecords?.(records); - fs.writeFileSync(nativeFile,records.map(r=>JSON.stringify(r)).join('\n')+'\n'); - const hookFile=path.join(dir,'hook.json');fs.writeFileSync(hookFile,JSON.stringify(hook)+'\n'); - const publicTools:NativePublicToolEvent[]=[];const native=readPlanCountTranscript(config,cwd,e=>publicTools.push(e)); - const pending=(readPendingAutoplanArtifact as any)(hookFile,cwd,config,stateRoot,fixture.commandStartedAt,publicTools,fixture.now,true); - const context={cwd,ownedStateRoot:stateRoot,commandStartedAt:fixture.commandStartedAt,now:fixture.now,viewportCapturedAt:fixture.now,transcriptStatus:native.status,publicTools,pending}; - const invoke=(screen=fixture.viewport,ctx:any=context,seen=new Set())=>(permission as any).publishedAutoplanArtifactPermissionInput?.(screen,ctx,seen)??null; - return {dir,cwd,config,stateRoot,hook,hookFile,file,nativeFile,publicTools,context,invoke,dispose:()=>fs.rmSync(dir,{recursive:true,force:true})}; -} - - -test('exact published Edit keeps unchanged suffixes in complete native preview rows',()=>{ - const s=setup();try{ - expect(s.context.pending?.toolUseId).toBe(fixture.hook.pending.toolUseId); - expect(permission.autoplanArtifactPermissionInput(fixture.viewport,s.context,new Set())).toBeNull(); - expect(permission.pendingAutoplanArtifactPermissionInput(fixture.viewport,s.context,new Set())).toBeNull(); - expect(s.invoke()).toEqual({input:'1\r',signature:s.hook.sessionId+':'+s.hook.pending.toolUseId,file:s.file}); - }finally{s.dispose()} -}); - -type Replay=ReturnType; -const current=(s:Replay)=>s.context.publicTools.find(e=>e.kind==='use'&&e.toolUseId===fixture.hook.pending.toolUseId)!; -const queued=(s:Replay)=>s.context.publicTools.filter(e=>e.kind==='use'&&e.name==='Edit'&&e.toolUseId!==fixture.hook.pending.toolUseId).at(-1)!; -function panel(s:Replay,rows:string[]){const bar='─'.repeat(120);return `${bar}\n Edit file\n ${s.file}\n${bar}\n${rows.join('\n')}\n${bar}\n Do you want to make this edit to ${path.basename(s.file)}?\n ❯ 1. Yes\n 2. Yes, and switch to accept edits (auto-approve file edits and common file commands) for this session (shift+tab)\n 3. No\n\n Esc to cancel · Tab to amend\n`;} -function request(s:Replay,before:string,old:string,replacement:string){ - fs.writeFileSync(s.file,before);fs.utimesSync(s.file,new Date(0),new Date(Date.parse(s.hook.pending.timestamp)-1000)); - const input=current(s).input!;input.old_string=old;input.new_string=replacement; - s.context.pending!.editDigest=createAutoplanEditDigest(s.file,old,replacement)!; -} - -test('unique request edges reconstruct exact prefix, suffix, newline and file boundaries',()=>{ - const cases:Array<[string,string,string,string,string[]]>=[ - ['both edges','prefix OLD suffix\n','OLD','NEW',[' 1 -prefix OLD suffix',' 1 +prefix NEW suffix']], - ['file start','OLD suffix\n','OLD','NEW',[' 1 -OLD suffix',' 1 +NEW suffix']], - ['file end','prefix OLD','OLD','NEW',[' 1 -prefix OLD',' 1 +prefix NEW']], - ['line start','head\nOLD suffix\n','OLD','NEW',[' 2 -OLD suffix',' 2 +NEW suffix']], - ['multiline edges','prefix first\nsecond suffix\n','first\nsecond','one\ntwo',[' 1 -prefix first',' 2 -second suffix',' 1 +prefix one',' 2 +two suffix']], - ['trailing newline','prefix OLD\nnext\n','OLD\n','NEW\n',[' 1 -prefix OLD',' 1 +prefix NEW',' 2 next']], - ['leading newline','head\nOLD suffix\n','\nOLD','\nNEW',[' 1 head',' 2 -OLD suffix',' 2 +NEW suffix']], - ['insert newline','prefix OLD suffix\n','OLD','NEW\nNEXT',[' 1 -prefix OLD suffix',' 1 +prefix NEW',' 2 +NEXT suffix']], - ['remove middle text','keep token tail\n','token ','',[' 1 -keep token tail',' 1 +keep tail']], - ]; - for(const [name,before,old,replacement,rows] of cases){const s=setup();try{request(s,before,old,replacement);expect(s.invoke(panel(s,rows))?.input,name).toBe('1\r');}finally{s.dispose()}} -}); - -test('viewport edges must be exact unchanged file bytes and cannot come from queued edits',()=>{ - const s=setup();try{ - expect(s.invoke(fixture.viewport.replaceAll('the envelope becomes the response','the envelope leaks a secret'))).toBeNull(); - request(s,'prefix OLD suffix\n','OLD','NEW'); - for(const rows of [ - [' 1 -foreign OLD suffix',' 1 +foreign NEW suffix'], - [' 1 -prefix OLD forged',' 1 +prefix NEW forged'], - [' 1 -prefix OLD suffix',' 1 +prefix UNREQUESTED suffix'], - [' 1 -prefix OLD suffix',' 1 +prefix NEW suffix',' 2 +queued sibling change'], - [' 1 prefix OLD suffix',' 1 +prefix OLD suffix'], - ])expect(s.invoke(panel(s,rows))).toBeNull(); - // A repeated old snippet must not select an arbitrary copy even when the pane matches one. - request(s,'prefix OLD suffix\nanother OLD line\n','OLD','NEW'); - expect(s.context.pending!.editDigest).toBeUndefined(); - expect(s.invoke(panel(s,[' 1 -prefix OLD suffix',' 1 +prefix NEW suffix']))).toBeNull(); - const direct={...s.context,publicTools:s.context.publicTools.filter(e=>e.toolUseId===current(s).toolUseId||e.kind==='result'||e.toolUseId===fixture.publicTools[0]!.toolUseId)}; - expect(permission.autoplanArtifactPermissionInput(panel(s,[' 1 -prefix OLD suffix',' 1 +prefix NEW suffix']),direct,new Set())).toBeNull(); - }finally{s.dispose()} -}); - -test('exact digest, current ownership and batch authority stay mandatory for the actual partial-line pane',()=>{ - const cases:Array<[string,(s:Replay)=>void]>=[ - ['before digest',s=>{s.context.pending!.editDigest.beforeSHA256='0'.repeat(64)}], - ['request digest',s=>{s.context.pending!.editDigest.requestSHA256='0'.repeat(64)}], - ['changed file',s=>{fs.appendFileSync(s.file,'\nChanged');fs.utimesSync(s.file,new Date(0),new Date(0))}], - ['changed request',s=>{current(s).input!.new_string+=' '}], - ['stale hook',s=>{s.context.pending!.timestamp=new Date(fixture.commandStartedAt-1).toISOString()}], - ['stale viewport',s=>{s.context.viewportCapturedAt=Date.parse(s.hook.pending.timestamp)-1}], - ['foreign session',s=>{s.context.pending!.sessionId='foreign'}], - ['foreign file',s=>{current(s).input!.file_path=s.file+'.other'}], - ['foreign queued batch',s=>{queued(s).requestId='req_foreign'}], - ['hooked queued sibling',s=>{s.context.pending!.hookSeenIds!.push(queued(s).toolUseId)}], - ['no successful prior write',s=>{for(const e of s.context.publicTools)if(e.kind==='result')e.isError=true}], - ['completed current request',s=>{s.context.publicTools.push({kind:'result',sessionId:s.hook.sessionId,toolUseId:current(s).toolUseId,timestamp:s.hook.pending.timestamp,isError:false})}], - ]; - for(const [name,change] of cases){const s=setup();try{change(s);expect(s.invoke(),name).toBeNull()}finally{s.dispose()}} - const s=setup();try{ - expect(s.invoke(fixture.viewport,s.context,new Set([s.hook.sessionId+':'+s.hook.pending.toolUseId]))).toBeNull(); - expect(s.invoke(fixture.viewport,s.context,new Set([permission.autoplanArtifactMenuKey(fixture.viewport)]))).toBeNull(); - expect(s.invoke('Source excerpt:\n'+fixture.viewport)).toBeNull(); - expect(s.invoke(fixture.viewport.split('\n').map(row=>'> '+row).join('\n'))).toBeNull(); - expect(s.invoke(fixture.viewport.replace('❯ 1. Yes','❯ 2. Yes'))).toBeNull(); - expect(s.invoke(fixture.viewport.replace('3. No','3. Maybe'))).toBeNull(); - }finally{s.dispose()} -}); - -test('the partial-line fixture and tests register only the Autoplan owner densely',()=>{ - const owner=E2E_TOUCHFILES['autoplan-chain-pty']!; - expect(Object.keys(owner)).toHaveLength(owner.length); - expect(Array.from(owner).every(x=>typeof x==='string')).toBe(true); - for(const file of ['test/autoplan-edit-edges-an.test.ts','test/fixtures/autoplan-edit-edges-an.json']) - expect(selectTests([file],E2E_TOUCHFILES,[]).selected).toEqual(['autoplan-chain-pty']); -}); diff --git a/test/autoplan-edit-header-ag.test.ts b/test/autoplan-edit-header-ag.test.ts deleted file mode 100644 index 99e8d8d53..000000000 --- a/test/autoplan-edit-header-ag.test.ts +++ /dev/null @@ -1,109 +0,0 @@ -import { afterEach, expect, test } from 'bun:test'; -import * as fs from 'node:fs'; -import * as os from 'node:os'; -import * as path from 'node:path'; -import { autoplanArtifactPermissionInput, pendingAutoplanArtifactPermissionInput, autoplanArtifactMenuKey } from './helpers/autoplan-artifact-permission'; -import type { NativePublicToolEvent } from './helpers/plan-count-transcript'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; -import captured from './fixtures/autoplan-edit-header-ag.json'; - -const roots: string[] = []; -afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, {recursive:true,force:true}); }); -function replay() { - const root = fs.mkdtempSync(path.join(os.tmpdir(), 'autoplan-edit-header-')); roots.push(root); - const cwd = path.join(root,path.basename(captured.cwd)); - const ownedStateRoot = path.join(root,'home','.gstack'); - const file = path.normalize(captured.pending.file.replace(captured.ownedStateRoot,ownedStateRoot)); - fs.mkdirSync(cwd,{recursive:true}); fs.mkdirSync(path.dirname(file),{recursive:true}); - fs.writeFileSync(file,captured.before); - const beforeTime = new Date(Date.parse(captured.pending.timestamp)-1000); - fs.utimesSync(file,beforeTime,beforeTime); - const publicTools = structuredClone(captured.events) as NativePublicToolEvent[]; - for (const event of publicTools) if (event.input?.file_path === captured.pending.file) event.input.file_path = file; - const pending = {...captured.pending,file,source:'pre_tool_use' as const,tool:'Edit' as const}; - const context = {cwd,ownedStateRoot,commandStartedAt:Date.parse(publicTools[0]!.timestamp)-1, - now:Date.parse(captured.viewportCapturedAt),viewportCapturedAt:Date.parse(captured.viewportCapturedAt), - transcriptStatus:'ready',publicTools,pending}; - const viewport = captured.viewport.replace(/^ (…[^\n]+)$/m,' …'+file.slice(root.length+1)); - return {root,file,context,viewport}; -} -const pick = (r:ReturnType, seen = new Set()) => - pendingAutoplanArtifactPermissionInput(r.viewport,r.context,seen); - -test('the captured native edit header preserves the current owned hook and diff', () => { - const r = replay(); - expect(r.context.publicTools).toHaveLength(88); - expect(autoplanArtifactPermissionInput(r.viewport,r.context,new Set())).toBeNull(); - expect(pick(r)).toEqual({input:'1\r',signature:captured.pending.sessionId+':'+captured.pending.toolUseId,file:r.file}); -}); - -test('exact absolute paths, full owned suffixes and launcher-owned aliases bind the same one-time request', () => { - for (const absoluteTitle of [false,true]) for (const displayedPath of ['cropped','absolute','relative']) { - const r = replay(); - if (absoluteTitle) r.viewport = r.viewport.replace(/^([●⏺] Update\()[^\n]+(?=\)$)/m,'$1'+r.file); - if (displayedPath === 'absolute') r.viewport = r.viewport.replace(/^ …[^\n]+$/m,' '+r.file); - if (displayedPath === 'relative') r.viewport = r.viewport.replace(/^ …[^\n]+$/m,' …'+path.relative(r.context.ownedStateRoot,r.file)); - const result = pick(r); expect(result?.input).toBe('1\r'); - expect(pick(r,new Set([result!.signature]))).toBeNull(); - expect(pick(r,new Set([autoplanArtifactMenuKey(r.viewport)]))).toBeNull(); - } -}); - -test('a retained header does not permit unrelated, ambiguous or quoted prefix rows', () => { - const changes = [ - (s:string) => s.replace('● Update(', '● Write('), - (s:string) => s.replace(/^● Update\([^\n]+\)/, '● Update(/tmp/foreign.md)'), - (s:string) => s.replace('~/.gstack/projects/', '~/.gstack/../projects/'), - (s:string) => s.replace(/(^ …[^\n]+)dashboard.md/m, '$1other.md'), - (s:string) => s.replace(/^ …[^\n]+$/m, ' …2026-09-10-user-dashboard.md'), - (s:string) => s.replace(/^ …[^\n]+$/m, ' …projects/sibling/ceo-plans/2026-09-10-user-dashboard.md'), - (s:string) => s.replace(' Edit file', ' Read file'), - (s:string) => s.replace(' Edit file', ' Run this first\n Edit file'), - (s:string) => s.replace(' Edit file', ' Edit file\n Edit file'), - (s:string) => 'Example:\n'+s, - (s:string) => '> '+s.replaceAll('\n','\n> '), - (s:string) => '```text\n'+s+'\n```', - (s:string) => s+'\nRun another action.', - (s:string) => s.replace(' ❯ 1. Yes',' ❯ 1. Yes, always allow'), - (s:string) => s.replace('to 2026-09-10-user-dashboard.md?','to sibling.md?'), - ]; - for (const change of changes) { const r=replay(); r.viewport=change(r.viewport); expect(pick(r),change.toString()).toBeNull(); } -}); - -test('framed edits retain stale, wrong-tool, foreign-path and success-history gates', () => { - const changes: Array<(r:ReturnType)=>void> = [ - r=>{r.context.pending.tool='Write' as 'Edit';}, - r=>{r.context.pending.sessionId='foreign';}, - r=>{r.context.pending.file=r.file+'.sibling';}, - r=>{r.context.viewportCapturedAt=Date.parse(r.context.pending.timestamp)-1;}, - r=>{r.context.pending.timestamp=new Date(r.context.now+1000).toISOString();}, - r=>{r.context.publicTools.push({kind:'result',sessionId:r.context.pending.sessionId,toolUseId:r.context.pending.toolUseId,timestamp:new Date(r.context.now).toISOString(),isError:false});}, - r=>{r.context.publicTools.push({kind:'use',sessionId:r.context.pending.sessionId,toolUseId:'unresolved-other',name:'Write',timestamp:new Date(r.context.now).toISOString(),input:{file_path:r.file}});}, - r=>{for(const event of r.context.publicTools) if(event.kind==='result') event.isError=true;}, - r=>{fs.writeFileSync(r.file,'Unrelated replacement content');}, - r=>{fs.utimesSync(r.file,new Date(r.context.now+1000),new Date(r.context.now+1000));}, - ]; - for(const change of changes) { const r=replay();change(r);expect(pick(r),change.toString()).toBeNull(); } -}); - -test('the same header works for fully published synthetic Edit inputs without replacing their comparison', () => { - const r = replay(); - const oldString = captured.before.split('\n')[0]!; - const newString = oldString+' (revised)'; - const lines = r.viewport.split('\n'); - const menu = r.viewport.slice(r.viewport.indexOf(' Do you want')); - r.viewport = lines.slice(0,6).join('\n')+'\n 1 -'+oldString+'\n 1 +'+newString+'\n────────\n'+menu; - r.context.publicTools.push({kind:'use',sessionId:r.context.pending.sessionId,toolUseId:r.context.pending.toolUseId, - name:'Edit',timestamp:r.context.pending.timestamp,input:{file_path:r.file,old_string:oldString,new_string:newString}}); - expect(pendingAutoplanArtifactPermissionInput(r.viewport,r.context,new Set())).toBeNull(); - expect(autoplanArtifactPermissionInput(r.viewport,r.context,new Set())?.input).toBe('1\r'); - r.context.publicTools.at(-1)!.input!.new_string='Different unpublished replacement'; - expect(autoplanArtifactPermissionInput(r.viewport,r.context,new Set())).toBeNull(); -}); - -test('the new native header evidence selects only the existing Autoplan paid case', () => { - for(const file of ['test/autoplan-edit-header-ag.test.ts','test/fixtures/autoplan-edit-header-ag.json']) { - expect(Object.entries(E2E_TOUCHFILES).filter(([,files])=>files.includes(file)).map(([owner])=>owner)).toEqual(['autoplan-chain-pty']); - expect(selectTests([file],E2E_TOUCHFILES,[]).selected).toEqual(['autoplan-chain-pty']); - } -}); diff --git a/test/autoplan-edit-panel-aj.test.ts b/test/autoplan-edit-panel-aj.test.ts deleted file mode 100644 index b2d2d7671..000000000 --- a/test/autoplan-edit-panel-aj.test.ts +++ /dev/null @@ -1,100 +0,0 @@ -import { afterEach, expect, test } from 'bun:test'; -import * as fs from 'node:fs'; -import * as os from 'node:os'; -import * as path from 'node:path'; -import captured from './fixtures/autoplan-edit-panel-aj.json'; -import published from './fixtures/autoplan-edit-prefix-ai.json'; -import { autoplanArtifactPermissionInput, pendingAutoplanArtifactPermissionInput, autoplanArtifactMenuKey } from './helpers/autoplan-artifact-permission'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; -import type { NativePublicToolEvent } from './helpers/plan-count-transcript'; -const roots: string[] = []; -afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); }); -function replay() { - const root = fs.mkdtempSync(path.join(os.tmpdir(), 'autoplan-edit-panel-')); roots.push(root); - const cwd = path.join(root, path.basename(captured.cwd)), ownedStateRoot = path.join(root, 'home', '.gstack'); - const file = path.normalize(captured.pending.file.replace(captured.ownedStateRoot, ownedStateRoot)); - fs.mkdirSync(cwd, { recursive: true }); fs.mkdirSync(path.dirname(file), { recursive: true }); fs.writeFileSync(file, captured.before); - const time = new Date(Date.parse(captured.pending.timestamp) - 1000); fs.utimesSync(file, time, time); - const events = structuredClone(captured.events) as NativePublicToolEvent[]; - for (const event of events) if (event.input?.file_path === captured.pending.file) event.input.file_path = file; - const context = { cwd, ownedStateRoot, commandStartedAt: Date.parse(events[0]!.timestamp) - 1, - now: captured.viewportCapturedAt, viewportCapturedAt: captured.viewportCapturedAt, - pending: { ...captured.pending, source: 'pre_tool_use' as const, tool: 'Edit' as const, file }, transcriptStatus: 'ready', publicTools: events }; - const viewport = captured.viewport.replace(/^ …[^\n]+$/m, ' …' + path.relative(ownedStateRoot, file)); - return { root, file, context, viewport }; -} -const pick = (r: ReturnType, seen = new Set()) => pendingAutoplanArtifactPermissionInput(r.viewport, r.context, seen); - -test('the exact standalone native Edit panel binds the owned current unpublished request', () => { - const r = replay(); - expect(pick(r)).toEqual({ input: '1\r', signature: r.context.pending.sessionId + ':' + r.context.pending.toolUseId, file: r.file }); - expect(autoplanArtifactPermissionInput(r.viewport, r.context, new Set())).toBeNull(); -}); - -test('complete absolute, home alias and full relative suffix paths retain ownership', () => { - for (const displayed of ['absolute', 'alias', 'suffix'] as const) { - const r = replay(), relative = path.relative(r.context.ownedStateRoot, r.file).split(path.sep).join('/'); - const value = displayed === 'absolute' ? r.file : displayed === 'alias' ? '~/.gstack/' + relative : '…' + relative; - r.viewport = r.viewport.replace(/^ …[^\n]+$/m, ' ' + value); expect(pick(r)?.input).toBe('1\r'); - } - const crop = replay(); crop.viewport = crop.viewport.split('\n').slice(4).join('\n'); expect(pick(crop)?.input).toBe('1\r'); -}); - -test('missing, foreign, quoted and ambiguous headers do not authorize the current file', () => { - for (const change of [ - (s: string) => s.replace(/^ …[^\n]+$/m, ' /tmp/foreign.md'), - (s: string) => s.replace(/^ …[^\n]+$/m, ' …' + path.basename(captured.pending.file)), - (s: string) => s.replace(/^ …[^\n]+$/m, ' …projects/sibling/ceo-plans/' + path.basename(captured.pending.file)), - (s: string) => s.replace(' Edit file\n', ''), - (s: string) => s.replace(' Edit file', ' Read file'), - (s: string) => s.split('\n').slice(1).join('\n'), - (s: string) => s.replace(/^─+\n/, '--------\n'), - (s: string) => '> quoted panel\n' + s, - (s: string) => '```text\n' + s + '\n```', - (s: string) => s.split('\n').slice(0, 4).join('\n') + '\n' + s, - (s: string) => '● Update(/tmp/foreign.md)\n\n' + s, - (s: string) => s + '\n' + s, - ]) { const r = replay(); r.viewport = change(r.viewport); expect(pick(r)).toBeNull(); } -}); - -test('current hook, observed time, same-file history and one-time menu remain required', () => { - const once = replay(), granted = pick(once)!; - expect(pick(once, new Set([granted.signature]))).toBeNull(); - expect(pick(once, new Set([autoplanArtifactMenuKey(once.viewport)]))).toBeNull(); - for (const change of [ - (r: ReturnType) => { r.context.pending.sessionId = 'foreign'; }, - (r: ReturnType) => { r.context.pending.file = r.file + '.foreign'; }, - (r: ReturnType) => { r.context.viewportCapturedAt = Date.parse(r.context.pending.timestamp) - 1; }, - (r: ReturnType) => { r.context.publicTools[1]!.isError = true; }, - (r: ReturnType) => { r.context.publicTools.push({ kind: 'result', sessionId: r.context.pending.sessionId, toolUseId: r.context.pending.toolUseId, timestamp: new Date(r.context.now).toISOString(), isError: false }); }, - (r: ReturnType) => { r.context.publicTools.push({ kind: 'use', sessionId: r.context.pending.sessionId, toolUseId: 'newer', timestamp: new Date(r.context.now).toISOString(), name: 'Write', input: { file_path: r.file } }); }, - (r: ReturnType) => { fs.writeFileSync(r.file, 'Foreign content'); }, - (r: ReturnType) => { fs.renameSync(r.file, r.file + '.target'); fs.symlinkSync(r.file + '.target', r.file); }, - (r: ReturnType) => { r.viewport = r.viewport.replace('❯ 1. Yes', '❯ 2. Yes'); }, - (r: ReturnType) => { r.viewport = r.viewport.replace('3. No', '3. Maybe'); }, - (r: ReturnType) => { r.viewport = r.viewport.replace(' 10 ', ' 0 '); }, - ]) { const r = replay(); change(r); expect(pick(r)).toBeNull(); } -}); - -test('published edits retain exact old/new content guards with the standalone presentation', () => { - const r = replay(), events = structuredClone(published.events) as NativePublicToolEvent[]; - const edit = events.find(e => e.kind === 'use' && e.toolUseId === published.pending.toolUseId)!; - const oldFile = edit.input!.file_path; - const file = path.normalize((oldFile as string).replace(published.ownedStateRoot, r.context.ownedStateRoot)); - const cwd = path.join(r.root, path.basename(published.cwd)); fs.mkdirSync(cwd, { recursive: true }); - fs.mkdirSync(path.dirname(file), { recursive: true }); fs.writeFileSync(file, published.before); - for (const event of events) if (event.input?.file_path === oldFile) event.input.file_path = file; - const header = published.viewport.lastIndexOf('\n● Update(') + 1; - const viewport = published.viewport.slice(header).split('\n').slice(2).join('\n').replace(/^ …[^\n]+$/m, ' …' + path.relative(r.context.ownedStateRoot, file)); - const context = { cwd, ownedStateRoot: r.context.ownedStateRoot, commandStartedAt: Date.parse(events[0]!.timestamp) - 1, now: Date.parse(published.viewportCapturedAt), transcriptStatus: 'ready', publicTools: events }; - expect(autoplanArtifactPermissionInput(viewport, context, new Set())?.input).toBe('1\r'); - const original = edit.input!.new_string; edit.input!.new_string = 'Unrelated replacement'; - expect(autoplanArtifactPermissionInput(viewport, context, new Set())).toBeNull(); - edit.input!.new_string = original; edit.input!.old_string = 'Unrelated original'; - expect(autoplanArtifactPermissionInput(viewport, context, new Set())).toBeNull(); -}); - -test('only Autoplan owns the standalone panel regression inputs', () => { - for (const file of ['test/autoplan-edit-panel-aj.test.ts', 'test/fixtures/autoplan-edit-panel-aj.json']) - expect(selectTests([file], E2E_TOUCHFILES, []).selected).toEqual(['autoplan-chain-pty']); -}); diff --git a/test/autoplan-edit-prefix-ai.test.ts b/test/autoplan-edit-prefix-ai.test.ts deleted file mode 100644 index 10b428efc..000000000 --- a/test/autoplan-edit-prefix-ai.test.ts +++ /dev/null @@ -1,121 +0,0 @@ -import { afterEach, expect, test } from 'bun:test'; -import * as fs from 'node:fs'; -import * as os from 'node:os'; -import * as path from 'node:path'; -import captured from './fixtures/autoplan-edit-prefix-ai.json'; -import { autoplanArtifactPermissionInput, pendingAutoplanArtifactPermissionInput, autoplanArtifactMenuKey } from './helpers/autoplan-artifact-permission'; -import type { NativePublicToolEvent } from './helpers/plan-count-transcript'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; - -const roots: string[] = []; -afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); }); -function replay() { - const root = fs.mkdtempSync(path.join(os.tmpdir(), 'autoplan-edit-prefix-')); roots.push(root); - const cwd = path.join(root, path.basename(captured.cwd)), ownedStateRoot = path.join(root, 'home', '.gstack'); - const events = structuredClone(captured.events) as NativePublicToolEvent[]; - const latest = events.find(e => e.kind === 'use' && e.toolUseId === captured.pending.toolUseId)! - const original = latest.input!.file_path as string, file = path.normalize(original.replace(captured.ownedStateRoot, ownedStateRoot)); - fs.mkdirSync(cwd, { recursive: true }); fs.mkdirSync(path.dirname(file), { recursive: true }); fs.writeFileSync(file, captured.before); - const time = new Date(Date.parse(latest.timestamp) - 1000); fs.utimesSync(file, time, time); - for (const event of events) if (event.input?.file_path === original) event.input.file_path = file; - const context = { cwd, ownedStateRoot, commandStartedAt: Date.parse(events[0]!.timestamp) - 1, - now: Date.parse(captured.viewportCapturedAt), viewportCapturedAt: Date.parse(captured.viewportCapturedAt), transcriptStatus: 'ready', publicTools: events }; - const viewport = captured.viewport.replace(/^ …[^\n]+$/m, ' …' + path.relative(ownedStateRoot, file)); - const header = viewport.lastIndexOf('\n● Update(') + 1; - return { root, file, current: latest, context, viewport, prefix: viewport.slice(0, header), panel: viewport.slice(header) }; -} -const pick = (r: ReturnType, seen = new Set()) => autoplanArtifactPermissionInput(r.viewport, r.context, seen); - -test('the exact retained prior diff output does not hide the current published owned edit', () => { - const r = replay(); - expect(r.prefix.split('\n')).toHaveLength(17); - expect(pick(r)).toEqual({ input: '1\r', signature: r.current.sessionId + ':' + r.current.toolUseId, file: r.file }); - r.viewport = r.panel; - expect(pick(r)?.input).toBe('1\r'); -}); - -test('completed diff rows are ignored only before one complete current native panel', () => { - for (const prefix of [' 1 +Previous completed output\n\n', ' +cropped prior row\n 12 +next prior row\n +wrapped row\n\n', ' 1 -Old value\n 1 +New value\n\n']) { - const r = replay(); r.viewport = prefix + r.panel; expect(pick(r)?.input).toBe('1\r'); - } -}); - -test('competing headers, previous panels, misleading prose and quotes remain rejected', () => { - for (const prefix of [ - '● Update(/tmp/foreign.md)\n\n', - '● Update(~/.gstack/projects/gstack-autoplan-chain-9599im/ceo-plans/2026-09-10-user-dashboard.md)\n ⎿ Added 1 line\n\n', - ' Edit file\n /tmp/foreign.md\n────────\n', - 'Example:\n', '> quoted output\n', '```diff\n 1 +quoted\n```\n', - ]) { const r = replay(); r.viewport = prefix + r.viewport; expect(pick(r)).toBeNull(); } - const priorPanel = replay(); priorPanel.viewport = priorPanel.panel + '\n' + priorPanel.panel; expect(pick(priorPanel)).toBeNull(); -}); - -test('malformed completed-output gutters cannot become a panel delimiter', () => { - for (const prefix of [' 1 +wrong indent\n', ' 0 +zero line\n', ' 9007199254740992 +unsafe line\n', ' 11 +row\n +short wrap\n', ' 11 +row\n -wrong kind\n', ' +only a cropped fragment\n']) { - const r = replay(); r.viewport = prefix + r.panel; expect(pick(r)).toBeNull(); - } -}); - -for (const [numbered, continuation] of [ - [' 7 ', ' '], [' 17 ', ' '], - [' 116 ', ' '], [' 1024 ', ' '], -] as const) test(`completed prefix ${numbered.trim()} infers one column before checking cropped and wrapped rows`, () => { - const r = replay(); - const prefix = `${continuation}+leading cropped fragment\n${numbered}+Previous completed\n${continuation}+ output\n\n`; - r.viewport = prefix + r.panel; - expect(pick(r)?.input).toBe('1\r'); - for (const invalid of [ - prefix.replaceAll(continuation + '+', continuation.slice(1) + '+'), - prefix.replaceAll(continuation + '+', ' ' + continuation + '+'), - prefix.replace(continuation + '+ output', continuation + '- output'), - prefix + numbered.replace(/(\d+) /, '$10 ') + '+mixed column\n', - prefix.replace(numbered + '+', ' ' + numbered.trim() + ' +'), - prefix.replace(numbered + '+Previous completed\n', ''), - 'Example:\n' + prefix, - ]) { r.viewport = invalid + r.panel; expect(pick(r), invalid).toBeNull(); } -}); - -test('the complete current header, exact target, menu and requested replacement remain binding', () => { - for (const change of [ - (r: ReturnType) => { r.viewport = r.viewport.replace('● Update(~/.gstack/', '● Update(/foreign/'); }, - (r: ReturnType) => { r.viewport = r.viewport.replace(/^ …[^\n]+$/m, ' …projects/sibling/ceo-plans/2026-09-10-user-dashboard.md'); }, - (r: ReturnType) => { r.viewport = r.viewport.replace(' Edit file', ' Read file'); }, - (r: ReturnType) => { r.viewport = r.viewport.replace('❯ 1. Yes', '❯ 2. Yes'); }, - (r: ReturnType) => { r.viewport = r.viewport.replace('3. No', '3. Maybe'); }, - (r: ReturnType) => { r.viewport += '\nDo another action.'; }, - (r: ReturnType) => { r.current.input!.new_string = 'Unrelated replacement'; }, - (r: ReturnType) => { fs.writeFileSync(r.file, 'Unrelated current file'); }, - ]) { const r = replay(); change(r); expect(pick(r)).toBeNull(); } -}); - -test('seen, completed, foreign or superseded native requests cannot borrow the valid panel', () => { - const once = replay(), granted = pick(once)!; - expect(pick(once, new Set([granted.signature]))).toBeNull(); - for (const change of [ - (r: ReturnType) => { const e = r.current; r.context.publicTools.push({ kind: 'result', sessionId: e.sessionId, toolUseId: e.toolUseId, timestamp: new Date(r.context.now).toISOString(), isError: false }); }, - (r: ReturnType) => { r.current.sessionId = 'foreign'; }, - (r: ReturnType) => { r.current.name = 'Write'; }, - (r: ReturnType) => { r.current.input!.file_path = r.file + '.foreign'; }, - (r: ReturnType) => { r.context.publicTools.find(e => e.kind === 'result')!.isError = true; }, - (r: ReturnType) => { const e = structuredClone(r.current); e.toolUseId = 'newer-edit'; r.context.publicTools.push(e); }, - ]) { const r = replay(); change(r); expect(pick(r)).toBeNull(); } -}); - -test('metadata fallback uses the same panel boundary while published inputs stay authoritative', () => { - const r = replay(), current = r.current; - const pending = { source: 'pre_tool_use' as const, tool: 'Edit' as const, sessionId: current.sessionId, toolUseId: current.toolUseId, timestamp: captured.pending.timestamp, file: r.file }; - expect(pendingAutoplanArtifactPermissionInput(r.viewport, { ...r.context, pending }, new Set())).toBeNull(); - // Synthetic missing-publication projection; actual AI request was published. - r.context.publicTools = r.context.publicTools.filter(e => e.toolUseId !== current.toolUseId); - const context = { ...r.context, pending }; - expect(pendingAutoplanArtifactPermissionInput(r.viewport, context, new Set())?.input).toBe('1\r'); - expect(pendingAutoplanArtifactPermissionInput(r.viewport, context, new Set([autoplanArtifactMenuKey(r.viewport)]))).toBeNull(); - expect(pendingAutoplanArtifactPermissionInput(r.viewport, { ...context, viewportCapturedAt: Date.parse(pending.timestamp) - 1 }, new Set())).toBeNull(); - r.viewport = 'Example:\n' + r.viewport; - expect(pendingAutoplanArtifactPermissionInput(r.viewport, context, new Set())).toBeNull(); -}); - -test('the exact prefix fixture and controls select only Autoplan', () => { - for (const file of ['test/autoplan-edit-prefix-ai.test.ts', 'test/fixtures/autoplan-edit-prefix-ai.json']) - expect(selectTests([file], E2E_TOUCHFILES, []).selected).toEqual(['autoplan-chain-pty']); -}); diff --git a/test/autoplan-edit-queue-am.test.ts b/test/autoplan-edit-queue-am.test.ts deleted file mode 100644 index 2e02a957e..000000000 --- a/test/autoplan-edit-queue-am.test.ts +++ /dev/null @@ -1,189 +0,0 @@ -import { capturedPathRebaser } from './helpers/captured-paths'; -import {expect,test} from 'bun:test'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import fixture from './fixtures/autoplan-edit-queue-am.json'; -import * as permission from './helpers/autoplan-artifact-permission'; -import {readPendingAutoplanArtifact} from './helpers/autoplan-artifact-recorder'; -import {readPlanCountTranscript,type NativePublicToolEvent} from './helpers/plan-count-transcript'; -import {E2E_TOUCHFILES,selectTests} from './helpers/touchfiles'; - -function setup(changeRecords?:(records:any[])=>void){ - const dir=fs.mkdtempSync(path.join(os.tmpdir(),'gstack-ap-queue-')); - const cwd=path.join(dir,path.basename(fixture.cwd)),config=path.join(dir,'config'); - const stateRoot=path.join(dir,'gstack-hermetic-2101964-HvDZyN/skill-home-zgCNxG/.gstack'); - const rebase=capturedPathRebaser([[fixture.stateRoot,stateRoot],[fixture.cwd,cwd],[fixture.config,config]]); - const hook=rebase.json(fixture.hook); - const file=hook.pending.file,nativeFile=path.join(config,'projects','owned',hook.sessionId+'.jsonl');hook.pending.transcriptPath=nativeFile; - fs.mkdirSync(cwd,{recursive:true});fs.mkdirSync(path.dirname(file),{recursive:true});fs.mkdirSync(path.dirname(nativeFile),{recursive:true}); - fs.writeFileSync(file,fixture.before);fs.utimesSync(file,new Date(fixture.now-1000000),new Date(Date.parse(hook.pending.timestamp)-1000)); - const events=rebase.json(fixture.publicTools) as (NativePublicToolEvent & {messageId?:string;requestId?:string})[]; - const records=events.map(e=>({sessionId:e.sessionId,cwd,isSidechain:false,timestamp:e.timestamp,requestId:e.requestId,message:{id:e.messageId,role:e.kind==='use'?'assistant':'user',content:e.kind==='use'?[{type:'tool_use',id:e.toolUseId,name:e.name,input:e.input}]:[{type:'tool_result',tool_use_id:e.toolUseId,content:'',is_error:e.isError}]}})); - changeRecords?.(records); - fs.writeFileSync(nativeFile,records.map(r=>JSON.stringify(r)).join('\n')+'\n'); - const hookFile=path.join(dir,'hook.json');fs.writeFileSync(hookFile,JSON.stringify(hook)+'\n'); - const publicTools:NativePublicToolEvent[]=[];const native=readPlanCountTranscript(config,cwd,e=>publicTools.push(e)); - const pending=(readPendingAutoplanArtifact as any)(hookFile,cwd,config,stateRoot,fixture.commandStartedAt,publicTools,fixture.now,true); - const context={cwd,ownedStateRoot:stateRoot,commandStartedAt:fixture.commandStartedAt,now:fixture.now,viewportCapturedAt:fixture.now,transcriptStatus:native.status,publicTools,pending}; - const invoke=(screen=fixture.viewport,ctx:any=context,seen=new Set())=>(permission as any).publishedAutoplanArtifactPermissionInput?.(screen,ctx,seen)??null; - return {dir,cwd,config,stateRoot,hook,hookFile,file,nativeFile,publicTools,context,invoke,dispose:()=>fs.rmSync(dir,{recursive:true,force:true})}; -} - -test('the actual active hook binds its published request amid later queued edits and both native prefix forms',()=>{ - const s=setup();try{ - expect(permission.autoplanArtifactPermissionInput(fixture.viewport,s.context,new Set())).toBeNull(); - expect(permission.pendingAutoplanArtifactPermissionInput(fixture.viewport,s.context,new Set())).toBeNull(); - expect(s.context.pending?.toolUseId).toBe(fixture.hook.pending.toolUseId); - expect(s.invoke()?.signature).toBe(`${fixture.hook.sessionId}:${fixture.hook.pending.toolUseId}`); - expect(s.invoke()?.file).toBe(s.file); - expect(s.invoke()?.input).toBe('1\r'); - }finally{s.dispose()} -}); - -test('the default metadata-only reader continues excluding a published request',()=>{ - const s=setup();try{expect(readPendingAutoplanArtifact(s.hookFile,s.cwd,s.config,s.stateRoot,fixture.commandStartedAt,s.publicTools,fixture.now)).toBeUndefined();}finally{s.dispose()} -}); - -type Replay=ReturnType; -const current=(s:Replay)=>s.context.publicTools.find(e=>e.kind==='use'&&e.toolUseId===fixture.hook.pending.toolUseId)!; -const queued=(s:Replay)=>s.context.publicTools.find(e=>e.kind==='use'&&e.toolUseId==='toolu_01SYiANcdq3hLqGxEhDQVNJf')!; -function rejects(cases:Array<[string,(s:Replay)=>void]>){ - for(const [name,change] of cases){const s=setup();try{change(s);expect(s.invoke(),name).toBeNull()}finally{s.dispose()}} -} - -test('only exact native message and request identifiers establish queued membership',()=>{ - const s=setup();try{ - expect(current(s).messageId).toBe('msg_011CeuYDnRH9L1Qoom8gBVdc'); - expect(current(s).requestId).toBe('req_011CeuYDk9cAd6Yozh8QnH62'); - expect(queued(s).messageId).toBe(current(s).messageId); - }finally{s.dispose()} - for(const change of [ - (r:any)=>{delete r.message.id},(r:any)=>{delete r.requestId}, - (r:any)=>{r.message.id='quoted msg_example'},(r:any)=>{r.requestId='req_'+ 'a'.repeat(161)}, - ]){const s=setup(records=>{for(const r of records)if(r.message.content[0]?.id===fixture.hook.pending.toolUseId)change(r)});try{ - expect(current(s).messageId).toBeUndefined();expect(current(s).requestId).toBeUndefined();expect(s.invoke()).toBeNull(); - }finally{s.dispose()}} -}); - -test.each(['one native record','equal timestamps'])('ordered later blocks in %s remain queued behind the current hook',shape=>{ - const ids=['toolu_01SYiANcdq3hLqGxEhDQVNJf','toolu_01LgaibBToDfuxGNFBKew9PS','toolu_01VqJFXfD5cfdjiar1gAjpkV']; - const s=setup(records=>{ - const active=records.find(r=>r.message.content[0]?.id===fixture.hook.pending.toolUseId)!; - for(let i=records.length-1;i>=0;i--){const r=records[i];if(!ids.includes(r.message.content[0]?.id))continue; - if(shape==='one native record'){active.message.content.splice(1,0,r.message.content[0]);records.splice(i,1)} - else r.timestamp=active.timestamp; - } - });try{ - const active=current(s),remaining=s.publicTools.filter(e=>e.kind==='use'&&ids.includes(e.toolUseId)); - expect(remaining.map(e=>e.toolUseId)).toEqual(ids); - expect(remaining.every(e=>e.timestamp===active.timestamp&&e.messageId===active.messageId&&e.requestId===active.requestId)).toBe(true); - expect(s.invoke()?.signature).toBe(s.hook.sessionId+':'+s.hook.pending.toolUseId); - // Moving a same-time unresolved block ahead of the current request is not a queued successor. - const earlier=remaining[0]!,events=s.context.publicTools;events.splice(events.indexOf(earlier),1);events.splice(events.indexOf(active),0,earlier); - expect(s.invoke()).toBeNull(); - }finally{s.dispose()} -}); - -test('another batch, session, path, tool, malformed edit or already hooked successor cannot be ignored',()=>{ - rejects([ - ['foreign message',s=>{queued(s).messageId='msg_other'}], - ['foreign request',s=>{queued(s).requestId='req_other'}], - ['missing message',s=>{delete queued(s).messageId}], - ['foreign session',s=>{queued(s).sessionId='foreign'}], - ['foreign file',s=>{queued(s).input!.file_path=s.file+'.other'}], - ['queued Write',s=>{queued(s).name='Write'}], - ['empty old request',s=>{queued(s).input!.old_string=''}], - ['missing replacement',s=>{delete queued(s).input!.new_string}], - ['replace all',s=>{queued(s).input!.replace_all=true}], - ['already hooked',s=>{s.context.pending!.hookSeenIds!.push(queued(s).toolUseId)}], - ['older unresolved',s=>{s.context.publicTools=s.context.publicTools.filter(e=>!(e.kind==='result'&&e.toolUseId==='toolu_01BbKwZ7JFFdm2FLFdcNQXPq'))}], - ]); -}); - -test('current hook identity, completed or failed requests and ordering cannot be overridden',()=>{ - rejects([ - ['foreign pending',s=>{s.context.pending!.sessionId='foreign'}], - ['wrong current hook',s=>{s.context.pending!.toolUseId=queued(s).toolUseId}], - ['missing hook',s=>{s.context.pending=undefined}], - ['missing tombstones',s=>{delete s.context.pending!.hookSeenIds}], - ['duplicate tombstone',s=>{s.context.pending!.hookSeenIds!.push(fixture.hook.pending.toolUseId)}], - ['unseen current',s=>{s.context.pending!.hookSeenIds=[]}], - ['duplicate current',s=>{const at=s.context.publicTools.indexOf(current(s));s.context.publicTools.splice(at,0,structuredClone(current(s)))}], - ['completion',s=>{s.context.publicTools.push({kind:'result',sessionId:s.hook.sessionId,toolUseId:current(s).toolUseId,timestamp:s.hook.pending.timestamp,isError:false})}], - ['failure',s=>{s.context.publicTools.push({kind:'result',sessionId:s.hook.sessionId,toolUseId:current(s).toolUseId,timestamp:s.hook.pending.timestamp,isError:true})}], - ['completed queued',s=>{s.context.publicTools.push({kind:'result',sessionId:s.hook.sessionId,toolUseId:queued(s).toolUseId,timestamp:s.hook.pending.timestamp,isError:false})}], - ['failed queued',s=>{s.context.publicTools.push({kind:'result',sessionId:s.hook.sessionId,toolUseId:queued(s).toolUseId,timestamp:s.hook.pending.timestamp,isError:true})}], - ['late predecessor completion',s=>{s.context.publicTools.at(-1)!.timestamp=new Date(Date.parse(s.hook.pending.timestamp)+1).toISOString()}], - ['no successful predecessor',s=>{for(const e of s.context.publicTools)if(e.kind==='result')e.isError=true}], - ['out of order',s=>{s.context.publicTools.reverse()}], - ['future publication',s=>{queued(s).timestamp=new Date(fixture.now+1).toISOString()}], - ]); -}); - -test('exact digest and current before file are required independently of the visible subset',()=>{ - rejects([ - ['missing digest',s=>{delete s.context.pending!.editDigest}], - ['malformed digest',s=>{s.context.pending!.editDigest.version=2}], - ['different request hash',s=>{s.context.pending!.editDigest.requestSHA256='0'.repeat(64)}], - ['different before hash',s=>{s.context.pending!.editDigest.beforeSHA256='0'.repeat(64)}], - ['different old lines',s=>{s.context.pending!.editDigest.oldLineHashes=['0'.repeat(64)]}], - ['different new lines',s=>{s.context.pending!.editDigest.newLineHashes=['0'.repeat(64)]}], - ['changed old request',s=>{current(s).input!.old_string+=' '}], - ['changed replacement',s=>{current(s).input!.new_string+=' '}], - ['missing current file',s=>{fs.unlinkSync(s.file)}], - ['changed current file with old mtime',s=>{fs.writeFileSync(s.file,fixture.before+'\nChanged.');fs.utimesSync(s.file,new Date(0),new Date(0))}], - ['file updated after hook',s=>{fs.utimesSync(s.file,new Date(fixture.now),new Date(fixture.now))}], - ['stale viewport',s=>{s.context.viewportCapturedAt=Date.parse(s.hook.pending.timestamp)-1}], - ['stale hook',s=>{s.context.pending!.timestamp=new Date(fixture.commandStartedAt-1).toISOString()}], - ['future viewport',s=>{s.context.viewportCapturedAt=fixture.now+1}], - ['unavailable native',s=>{s.context.transcriptStatus='missing'}], - ]); - const s=setup();try{ - expect(s.invoke(fixture.viewport,s.context,new Set([s.hook.sessionId+':'+s.hook.pending.toolUseId]))).toBeNull(); - expect(s.invoke(fixture.viewport,s.context,new Set([permission.autoplanArtifactMenuKey(fixture.viewport)]))).toBeNull(); - }finally{s.dispose()} -}); - -test('invalid, busy, foreign or ambiguous persisted hook state supplies no current authority',()=>{ - for(const change of [ - (s:Replay)=>{fs.writeFileSync(s.hookFile+'.invalid','{"reason":"conflicting_replay"}')}, - (s:Replay)=>{fs.writeFileSync(s.hookFile+'.lock','')}, - (s:Replay)=>{s.hook.pending.transcriptPath=path.join(s.dir,'foreign.jsonl');fs.writeFileSync(s.hookFile,JSON.stringify(s.hook))}, - (s:Replay)=>{s.hook.pending.hookSeenIds=[];fs.writeFileSync(s.hookFile,JSON.stringify(s.hook))}, - ]){const s=setup();try{change(s);expect(readPendingAutoplanArtifact(s.hookFile,s.cwd,s.config,s.stateRoot,fixture.commandStartedAt,s.publicTools,fixture.now,true)).toBeUndefined()}finally{s.dispose()}} -}); - -test('existing prefix forms compose but cannot hide a competing title, source or malformed current panel',()=>{ - const s=setup();try{ - const first=fixture.viewport.indexOf('● Update('),screen=fixture.viewport.slice(first); - const titles=screen.match(/^● Update\([^\n]+\)\n/gm)!; - expect(titles).toHaveLength(4); - expect(s.invoke(screen)?.input).toBe('1\r'); - expect(s.invoke('\n\n'+screen)?.input).toBe('1\r'); - expect(s.invoke(fixture.viewport.replaceAll(titles[0]!,''))).toBeNull(); // A completed prefix still needs its current tool boundary. - expect(s.invoke(screen.slice(screen.indexOf('────────────────')))?.input).toBe('1\r'); - let one=screen;for(let n=0;n<3;n++)one=one.replace(titles[0]!,''); - expect(s.invoke(one.trimStart())?.input).toBe('1\r'); - for(const [name,changed] of [ - ['foreign first title',fixture.viewport.replace(titles[0]!,titles[0]!.replace('user-dashboard.md','foreign.md'))], - ['quoted whole pane',fixture.viewport.split('\n').map(row=>'> '+row).join('\n')], - ['source prefix','Example:\n'+fixture.viewport], - ['arbitrary indented prose',' This is an example.\n'+fixture.viewport], - ['competing completed panel','● Update(/tmp/foreign.md)\n'+fixture.viewport], - ['broken wrap kind',fixture.viewport.replace(/^ \+/m,' -')], - ['foreign displayed path',fixture.viewport.replace('…2101964-HvDZyN','…foreign')], - ['wrong menu target',fixture.viewport.replace('user-dashboard.md?','foreign.md?')], - ['persistent edit mode',fixture.viewport.replace('❯ 1. Yes','❯ 2. Yes')], - ['malformed no',fixture.viewport.replace('3. No','3. Maybe')], - ['changed addition',fixture.viewport.replace(/^( {0,3}\d+ \+).*/m,'$1A different current edit')], - ])expect(s.invoke(changed),name).toBeNull(); - }finally{s.dispose()} -}); - -test('the new queue regression files select only the Autoplan owner with dense registration',()=>{ - const owner=E2E_TOUCHFILES['autoplan-chain-pty']!; - for(let i=0;i{for(const root of roots.splice(0))fs.rmSync(root,{recursive:true,force:true});}); -function replay(relative='ceo-plans/2026-09-09-user-dashboard.md') { - const root=fs.mkdtempSync(path.join(os.tmpdir(),'gstack-pending-artifact-test-'));roots.push(root); - const cwd=path.join(root,path.basename(fixture.cwd)),ownedStateRoot=path.join(root,'home','.gstack'),config=path.join(root,'config'); - fs.mkdirSync(cwd);const file=path.join(ownedStateRoot,'projects',path.basename(cwd),relative); - fs.mkdirSync(path.dirname(file),{recursive:true});fs.writeFileSync(file,fixture.before); - const native=path.join(config,'projects','fixture',fixture.sessionId+'.jsonl');fs.mkdirSync(path.dirname(native),{recursive:true});fs.writeFileSync(native,''); - const publicTools=structuredClone(fixture.events) as NativePublicToolEvent[]; - for(const e of publicTools)if(e.input)e.input.file_path=file; - const recorder=createAutoplanArtifactRecorder(cwd,config,ownedStateRoot); - // Synthetic hook: only its identity/path are retained. Neither this input - // nor the displayed additions are claimed to reproduce the unpublished body. - const event={hook_event_name:'PreToolUse',tool_name:'Edit',session_id:fixture.sessionId,tool_use_id:'synthetic-current-edit', - cwd,transcript_path:native,tool_input:{file_path:file,old_string:'Synthetic old content',new_string:'Synthetic new content',replace_all:false}}; - const record=(change:Record={})=>recordAutoplanArtifact(JSON.stringify({...event,...change}),recorder.file,cwd,config,ownedStateRoot); - record(); - const context={cwd,ownedStateRoot,commandStartedAt:fixture.commandStartedAt,now:Date.now(),viewportCapturedAt:Date.now(), - transcriptStatus:'ready',publicTools,pending:readPendingAutoplanArtifact(recorder.file,cwd,config,ownedStateRoot,fixture.commandStartedAt,publicTools)}; - const screen=fixture.viewport.replaceAll(path.basename(fixture.file),path.basename(file)); - roots.push(path.dirname(recorder.file)); - return {root,file,native,config,recorder,event,record,context,screen}; -} -const pick=(r:ReturnType,seen=new Set())=>pendingAutoplanArtifactPermissionInput(r.screen,r.context,seen); - -test('actual public pane stays blocked without hook identity; synthetic owned metadata enables only one option',()=>{ - const r=replay(); - expect(autoplanArtifactPermissionInput(r.screen,r.context,new Set())).toBeNull(); - expect(pick({...r,context:{...r.context,pending:undefined}})).toBeNull(); - expect(pick(r)).toEqual({input:'1\r',signature:fixture.sessionId+':synthetic-current-edit',file:r.file}); - expect(pick(r,new Set([pick(r)!.signature]))).toBeNull(); - expect(JSON.stringify(r.context.pending)).not.toContain('Synthetic old content'); - expect(r.context.publicTools).toHaveLength(fixture.events.length); -}); - -test('all130 actual published tool events preserve the same metadata-only fallback boundary',()=>{ - const r=replay();r.context.publicTools=structuredClone(fixture.allPublicTools) as NativePublicToolEvent[]; - for(const e of r.context.publicTools)if(e.input?.file_path===fixture.file)e.input.file_path=r.file; - expect(r.context.publicTools).toHaveLength(130); - expect(r.context.publicTools.filter(e=>e.kind==='use' && ['Write','Edit'].includes(e.name??''))).toHaveLength(39); - expect(pick(r)?.input).toBe('1\r'); -}); - -test('completed or published requests and newer identities on an old granted viewport remain closed',()=>{ - const r=replay(),first=pick(r)!; - const seen=new Set([first.signature,autoplanArtifactMenuKey(r.screen)]); - r.record({hook_event_name:'PostToolUse'}); - expect(readPendingAutoplanArtifact(r.recorder.file,r.context.cwd,r.config,r.context.ownedStateRoot,r.context.commandStartedAt,r.context.publicTools)).toBeUndefined(); - r.record({tool_use_id:'newer-request'}); - r.context.now=Date.now();r.context.viewportCapturedAt=r.context.now; - r.context.pending=readPendingAutoplanArtifact(r.recorder.file,r.context.cwd,r.config,r.context.ownedStateRoot,r.context.commandStartedAt,r.context.publicTools); - expect(pick(r,seen)).toBeNull(); - r.context.publicTools.push({kind:'result',sessionId:fixture.sessionId,toolUseId:'newer-request',timestamp:new Date().toISOString(),isError:false}); - expect(pick(r)).toBeNull(); -}); - -test('hook after viewport, invalid clocks, future/stale/foreign IDs and missing success cannot authorize input',()=>{ - const changes:Array<(r:ReturnType)=>void>=[ - r=>{r.context.viewportCapturedAt=Date.parse(r.context.pending!.timestamp)-1;}, - r=>{r.context.now=NaN;},r=>{r.context.now=Infinity;},r=>{r.context.viewportCapturedAt=NaN;}, - r=>{r.context.pending!.timestamp=new Date(r.context.now+10000).toISOString();}, - r=>{r.context.pending!.timestamp=new Date(r.context.commandStartedAt-1).toISOString();}, - r=>{r.context.pending!.sessionId='foreign';},r=>{r.context.pending!.toolUseId='';}, - r=>{r.context.pending!.toolUseId='invalid:id';},r=>{r.context.pending!.file=42 as any;},r=>{r.context.publicTools=[];}, - r=>{r.context.transcriptStatus='error';}, - r=>{for(const e of r.context.publicTools)if(e.kind==='result')e.isError=true;}, - r=>{r.context.publicTools.push({...r.context.publicTools[0]!,toolUseId:'unresolved-concurrent',timestamp:new Date().toISOString()});}, - r=>{r.context.publicTools.push({...r.context.publicTools[0]!,toolUseId:r.context.pending!.toolUseId,timestamp:new Date().toISOString()});}, - r=>{r.context.publicTools.push({...r.context.publicTools.at(-1)!,sessionId:'sibling'});}, - ]; - for(const change of changes){const r=replay();change(r);expect(pick(r),change.toString()).toBeNull();} -}); - -test('changed, foreign and symlink files are rejected; all existing owned artifact layouts stay scoped',()=>{ - for(const relative of ['ceo-plans/2026-09-09-user-dashboard.md','main-test-plan-20260909-220000.md','main-eng-review-test-plan-20260909-220000.md'])expect(pick(replay(relative))?.input).toBe('1\r'); - for(const relative of ['other.md','config.yaml','tasks.jsonl','../sibling/ceo-plans/2026-09-09-user-dashboard.md'])expect(pick(replay(relative))).toBeNull(); - let r=replay();fs.writeFileSync(r.file,'Changed unrelated content');expect(pick(r)).toBeNull(); - r=replay();fs.utimesSync(r.file,new Date(r.context.now+10000),new Date(r.context.now+10000));expect(pick(r)).toBeNull(); - if(process.platform!=='win32'){ - r=replay();const sibling=r.file+'.sibling';fs.renameSync(r.file,sibling);fs.symlinkSync(sibling,r.file);expect(pick(r)).toBeNull(); - } - r=replay();r.context.ownedStateRoot=path.join(r.root,'ambient-home');expect(pick(r)).toBeNull(); -}); - -test('only a complete current native menu and current-file deleted/context rows support pending metadata',()=>{ - const changes=[ - (s:string)=>'Example:\n'+s,(s:string)=>'```\n'+s+'```', - (s:string)=>s.split('\n').map(l=>'> '+l).join('\n'), - (s:string)=>s.replace(' ❯ 1. Yes',' ❯ 1. Yes, always allow'), - (s:string)=>s.replace(' ❯ 1. Yes',' 1. Yes').replace(' 2. Yes',' ❯ 2. Yes'), - (s:string)=>s.replace(' 3. No',' 3. No\n 4. Run a command'), - (s:string)=>s.replace('2026-09-09-user-dashboard.md?','foreign.md?'), - (s:string)=>s.replace('Esc to cancel · Tab to amend','Enter to select'), - (s:string)=>s+'\nPlease run the extra work.', - (s:string)=>s.replace(' -than the latest',' -unrelated cropped text'), - (s:string)=>s.replace(' 50 -- **Retry.**',' 50 -- **Unrelated deletion.**'), - (s:string)=>s.slice(s.indexOf(' Do you want')), - ]; - for(const change of changes){const r=replay();r.screen=change(r.screen);expect(pick(r),change.toString()).toBeNull();} -}); - -test('queued unrelated public tools do not confer permission or block the current owned edit',()=>{ - const r=replay();r.context.publicTools.push({kind:'use',sessionId:fixture.sessionId,toolUseId:'queued-bash',name:'Bash', - timestamp:new Date(r.context.now).toISOString(),input:{command:'echo queued'}}); - expect(pick(r)?.input).toBe('1\r'); - r.context.publicTools.at(-1)!.name='Write';expect(pick(r)).toBeNull(); -}); - -for (const [line, numbered, next, continuation] of [ - [7, ' 7 ', ' 8 ', ' '], [17, ' 17 ', ' 18 ', ' '], - [116, ' 116 ', ' 117 ', ' '], [1024, ' 1024 ', ' 1025 ', ' '], -] as const) test(`legacy pending deletion line ${line} binds leading and wrapped fragments to its numbered column`, () => { - const r = replay(); - expect(r.context.pending?.editDigest).toBeUndefined(); - const before = Array.from({ length: line - 2 }, (_, n) => `Context ${n}`) - .concat('Head before crop tail', 'Old complete row', 'Context').join('\n'); - fs.writeFileSync(r.file, before); - const at = new Date(Date.parse(r.context.pending!.timestamp) - 1); fs.utimesSync(r.file, at, at); - const menu = r.screen.slice(r.screen.indexOf(' Do you want')); - const rows = `${continuation}-tail\n${numbered}-Old complete\n${continuation}- row\n` + - `${numbered}+New complete\n${continuation}+ row\n${next} Context\n`; - const pane = rows + '╌'.repeat(20) + '\n' + menu; - r.screen = pane; - expect(pick(r)?.input).toBe('1\r'); - expect(pick(r, new Set([pick(r)!.signature]))).toBeNull(); - for (const invalid of [ - pane.replaceAll(continuation + '-', continuation.slice(1) + '-'), - pane.replaceAll(continuation + '-', ' ' + continuation + '-'), - pane.replace(continuation + '- row', continuation + '+ row'), - pane.replace(next + ' Context', ' ' + next + ' Context'), - pane.replace('Old complete', 'Unrelated deleted'), - pane.replace(continuation + '-tail', continuation + '-foreign suffix'), - pane.replaceAll(numbered, ' 0 '), - ]) { r.screen = invalid; expect(pick(r), invalid).toBeNull(); } -}); - test.skipIf(process.platform==='win32')('real launcher installs only opt-in owned hooks and removes records on close or early exit',async()=>{ const root=fs.mkdtempSync(path.join(os.tmpdir(),'gstack-artifact-launch-'));roots.push(root); const fake=path.join(root,'fake-claude');fs.writeFileSync(fake,`#!${process.execPath}\n`+String.raw` diff --git a/test/autoplan-permission-viewport.test.ts b/test/autoplan-permission-viewport.test.ts deleted file mode 100644 index f6650d6a4..000000000 --- a/test/autoplan-permission-viewport.test.ts +++ /dev/null @@ -1,334 +0,0 @@ -import { afterEach, beforeEach, expect, test } from 'bun:test'; -import * as fs from 'node:fs'; -import * as os from 'node:os'; -import * as path from 'node:path'; -import { AutoplanFilePermissionViewport, reserveAutoplanFilePermission } from './helpers/autoplan-phase-order'; -import { PtyCurrentScreen } from './helpers/pty-current-screen'; -import { isNumberedOptionListVisible, isPermissionDialogVisible } from './helpers/claude-pty-runner'; -import type { readPlanSkillQuestions, NativePermissionGrant } from './helpers/plan-skill-questions'; - -// Pinned 2.1.263 file renderer layout: full relative subtitle above the diff, -// basename below it, and the settings-specific standing option. Only 1 is sent. -let cwd: string, file: string, screen: PtyCurrentScreen; -let native: ReturnType; -let viewport: AutoplanFilePermissionViewport; -let granted: Set, requests: Map; -let raw = '', lines = 350, extraWidth = 0, repaint = true, displayPath: string; -let resizes: number[], sends: string[], deadlineAt: number; -let operation: 'create' | 'edit' | 'overwrite' = 'edit'; -const card = () => [ - '─'.repeat(120), ` ${{ create: 'Create', edit: 'Edit', overwrite: 'Overwrite' }[operation]} file`, ' ' + displayPath, '╌'.repeat(120), - ...Array.from({ length: lines }, (_, i) => ` ${i + 1} +ordinary proposed plan line ${i + 1}` + 'x'.repeat(extraWidth)), - '╌'.repeat(120), ` Do you want to ${operation === 'edit' ? 'make this edit to' : operation} ${path.basename(file)}?`, - ' ❯ 1. Yes', ' 2. Yes, and allow Claude to edit its own settings for this session', - ' 3. No', '', ' Esc to cancel · Tab to amend', -].join('\r\n'); -const paint = () => { const text = '\x1b[2J\x1b[H' + card(); raw += text; screen.feed(text); }; -const sample = async () => ({ text: (await screen.snapshot()).text, rawEnd: raw.length }); -const reserve = (frame: { text: string }) => reserveAutoplanFilePermission(native, frame.text, - { cwd, planDir: path.join(cwd, '.claude', 'plans'), granted, requests }); -const tick = async () => { - const frame = await sample(); - if (viewport.active && await viewport.advance(native, frame)) return; - try { if (reserve(frame)) sends.push('1\r'); } - catch (error) { if (!await viewport.recover(error, native, frame)) throw error; } -}; -const useOperation = (value: typeof operation) => { - operation = value; - const owner = native.permissionRequests[0]!; - owner.name = value === 'edit' ? 'Edit' : 'Write'; - owner.input = value === 'edit' ? { file_path: file, old_string: 'existing plan', new_string: 'reviewed plan' } - : { file_path: file, content: 'reviewed plan' }; - paint(); -}; -beforeEach(() => { - cwd = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'autoplan-card-free-'))); - file = path.join(cwd, '.claude', 'plans', 'review.md'); - fs.mkdirSync(path.dirname(file), { recursive: true }); fs.writeFileSync(file, 'existing plan'); - raw = ''; lines = 350; extraWidth = 0; repaint = true; operation = 'edit'; displayPath = path.relative(cwd, file); - resizes = []; sends = []; granted = new Set(); requests = new Map(); deadlineAt = Date.now() + 5000; - native = { calls: [], ready: false, pendingExitPlanModeIds: [], pendingBytes: 0, - permissionTools: [], permissionResults: [], permissionRequestCapture: true, - permissionRequests: [{ requestId: 'owned-edit', capturedAtMs: 1, name: 'Edit', cwd, - input: { file_path: file, old_string: 'existing plan', new_string: 'reviewed plan' }, result: 'pending', nativeToolId: null }] }; - screen = new PtyCurrentScreen({ cols: 120, rows: 120 }); - viewport = new AutoplanFilePermissionViewport({ deadlineAt, granted, session: { - mark: () => raw.length, - resizeQuestionViewport: async (rows, deadline) => { - if (Date.now() >= deadline) return null; - await screen.snapshot(); const mark = raw.length; - screen.resize(120, rows); resizes.push(rows); - if (repaint) paint(); return mark; - }, - } }); - paint(); -}); -afterEach(() => { screen.dispose(); fs.rmSync(cwd, { recursive: true, force: true }); }); - -test.each(['create', 'edit', 'overwrite'] as const)('a taller-than120 owned file needs fresh paints, grants once, and restores only after ACK (%s)', async operation => { - useOperation(operation); - const name = operation === 'edit' ? 'Edit' : 'Write'; - expect((await sample()).text).not.toContain(` file\n ${path.join('.claude','plans','review.md')}`); - expect(() => reserve({ text: card().split('\r\n').slice(-120).join('\n') })).toThrow('cannot be bound'); - await tick(); expect(resizes).toEqual([240]); expect(sends).toEqual([]); - await tick(); expect(resizes).toEqual([240, 480]); expect(sends).toEqual([]); - expect((await sample()).text).toContain(` file\n ${path.join('.claude','plans','review.md')}`); - await tick(); await tick(); - expect(sends).toEqual(['1\r']); expect(resizes).toEqual([240, 480]); - Object.assign(native.permissionRequests[0]!, { result: 'completed', nativeToolId: 'actual-edit', nativeResultAtMs: 2 }); - await tick(); expect(resizes).toEqual([240, 480, 120]); expect(viewport.active).toBe(false); - expect([...granted]).toEqual(['request:owned-edit']); expect([...requests.keys()]).toEqual([name + ':' + file]); -}); - -test('captured Autoplan overwrite reaches a controlled full-header repaint before one grant and a controlled native-ID ACK', async () => { - const captured = JSON.parse(fs.readFileSync(path.join(import.meta.dir, 'fixtures', 'autoplan-settings-overwrite.json'), 'utf8')); - // Preserve the actual card and public input; remap only the dead fixture root - // to this owned disposable root. Later header paints and ACK are controlled. - file = path.join(cwd, '.claude/plans', path.basename(captured.pendingRequest.input.file_path)); - fs.writeFileSync(file, 'existing plan'); displayPath = path.relative(cwd, file); operation = 'overwrite'; - native.permissionRequests = [{ ...structuredClone(captured.pendingRequest), cwd, - input: { ...captured.pendingRequest.input, file_path: file } }]; - const literal = '\x1b[2J\x1b[H' + captured.frame.text.replaceAll('\n', '\r\n'); - raw += literal; screen.feed(literal); - const initial = await sample(); - expect(initial.text).toBe(captured.frame.text); - expect(isNumberedOptionListVisible(initial.text)).toBe(true); - expect(isPermissionDialogVisible(initial.text)).toBe(true); - expect(() => reserve(initial)).toThrow('cannot be bound'); - await tick(); expect(resizes).toEqual([240]); expect(sends).toEqual([]); - await tick(); expect(resizes).toEqual([240, 480]); expect(sends).toEqual([]); - await tick(); await tick(); expect(sends).toEqual(['1\r']); - expect([...requests.entries()]).toEqual([['Write:' + file, { requestId: captured.pendingRequest.requestId, operation: 'overwrite' }]]); - expect(native.permissionRequests[0]!.nativeToolId).toBeNull(); - expect(viewport.active).toBe(true); - Object.assign(native.permissionRequests[0]!, { result: 'completed', nativeToolId: 'controlled-write-ack', nativeResultAtMs: captured.pendingRequest.capturedAtMs + 1 }); - await tick(); expect(resizes).toEqual([240, 480, 120]); expect(viewport.active).toBe(false); - expect(sends).toEqual(['1\r']); -}); - -test('a 600-line owned Edit recovers its complete path only after the third fresh paint', async () => { - lines = 600; paint(); await tick(); await tick(); - expect((await sample()).text).not.toContain(' Edit file'); - expect(sends).toEqual([]); expect(granted.size).toBe(0); - await tick(); expect(resizes).toEqual([240, 480, 960]); - expect((await sample()).text).toContain(` Edit file\n ${path.join('.claude','plans','review.md')}`); - await tick(); await tick(); - expect(sends).toEqual(['1\r']); - expect([...granted]).toEqual(['request:owned-edit']); - expect([...requests.entries()]).toEqual([['Edit:' + file, { requestId: 'owned-edit', operation: 'edit' }]]); - Object.assign(native.permissionRequests[0]!, { result: 'completed', nativeToolId: 'large-edit', nativeResultAtMs: 2 }); - await tick(); expect(resizes).toEqual([240, 480, 960, 120]); expect(viewport.active).toBe(false); -}); - -test.each(['create', 'edit', 'overwrite'] as const)('a card still clipped at the finite cap fails with the original identity error and no grant (%s)', async operation => { - useOperation(operation); - lines = 1000; paint(); await tick(); await tick(); await tick(); - await expect(tick()).rejects.toThrow('Visible permission cannot be bound'); - expect(resizes).toEqual([240, 480, 960]); expect(sends).toEqual([]); expect(granted.size).toBe(0); -}); - -test('wrapped physical diff rows recover within the cap without treating logical lines as viewport height', async () => { - lines = 180; extraWidth = 160; paint(); - expect((await screen.snapshot()).lines.some(line => line.wrapped)).toBe(true); - expect((await sample()).text).not.toContain(' Edit file'); - await tick(); expect((await sample()).text).not.toContain(' Edit file'); - await tick(); expect((await sample()).text).toContain(` Edit file\n ${path.join('.claude','plans','review.md')}`); - await tick(); expect(sends).toEqual(['1\r']); expect(resizes).toEqual([240, 480]); - Object.assign(native.permissionRequests[0]!, { result: 'completed', nativeToolId: 'wrapped-edit', nativeResultAtMs: 2 }); - await tick(); expect(resizes).toEqual([240, 480, 120]); -}); - -test('the first learned native tool ID cannot change on a later recovery sample', async () => { - await tick(); native.permissionRequests[0]!.nativeToolId = 'first-known-id'; - await tick(); native.permissionRequests[0]!.nativeToolId = 'other-known-id'; - await expect(tick()).rejects.toThrow('changed ownership or input'); - expect(sends).toEqual([]); expect(resizes).toEqual([240, 480]); -}); - -test.each(['input', 'request', 'cwd', 'operation', 'time', 'native-id'])('repaint cannot transfer authority to changed %s', async kind => { - if (kind === 'native-id') native.permissionRequests[0]!.nativeToolId = 'first-id'; - await tick(); const owner = native.permissionRequests[0]!; - if (kind === 'input') owner.input.new_string = 'different changes'; - if (kind === 'request') owner.requestId = 'different-request'; - if (kind === 'cwd') owner.cwd += '-other'; - if (kind === 'operation') owner.name = 'Write'; - if (kind === 'time') owner.capturedAtMs++; - if (kind === 'native-id') owner.nativeToolId = 'different-id'; - await expect(tick()).rejects.toThrow('changed ownership or input'); - expect(resizes).toEqual([240]); expect(sends).toEqual([]); -}); - -test.each(['request', 'tool'])('a competing %s introduced during recovery remains ambiguous', async kind => { - await tick(); - if (kind === 'request') native.permissionRequests.push({ ...structuredClone(native.permissionRequests[0]!), requestId: 'competing' }); - else native.permissionTools.push({ id: 'competing', name: 'Edit', cwd, input: { file_path: file } }); - await expect(tick()).rejects.toThrow('Ambiguous native permission owner'); - expect(resizes).toEqual([240]); expect(sends).toEqual([]); -}); - -test('an already ambiguous request cannot start recovery', async () => { - native.permissionTools.push({ id: 'competing', name: 'Edit', cwd, input: { file_path: file } }); - await expect(tick()).rejects.toThrow('multiple tools are pending'); - expect(resizes).toEqual([]); expect(sends).toEqual([]); -}); - -test.each(['create', 'edit', 'overwrite'] as const)('explicit full-path mismatch is an error, not another request to enlarge the viewport (%s)', async operation => { - useOperation(operation); - await tick(); lines = 3; displayPath = '.claude/other/review.md'; paint(); - await expect(tick()).rejects.toThrow('cannot be bound'); - expect(resizes).toEqual([240]); expect(sends).toEqual([]); -}); - -test.each(['create', 'edit', 'overwrite'] as const)('a resize without new native output cannot reuse stale text or renew recovery (%s)', async operation => { - useOperation(operation); - repaint = false; await tick(); - for (let i = 0; i < 4; i++) await tick(); - expect(resizes).toEqual([240]); expect(sends).toEqual([]); expect(granted.size).toBe(0); -}); - -test.each(['create', 'overwrite'] as const)('a settings %s card cannot nominate an Edit owner for repaint', async operation => { - useOperation(operation); native.permissionRequests[0]!.name = 'Edit'; - await expect(tick()).rejects.toThrow('cannot be bound'); - expect(resizes).toEqual([]); expect(sends).toEqual([]); expect(granted.size).toBe(0); -}); - -test.each(['error', 'missing-ack'])('a %s completion never restores or grants again', async kind => { - await tick(); await tick(); await tick(); - native.permissionRequests[0]!.result = kind === 'error' ? 'error' : 'completed'; - await expect(tick()).rejects.toThrow(kind === 'error' ? 'returned an error' : 'successful native ACK'); - expect(sends).toEqual(['1\r']); expect(resizes).toEqual([240, 480]); -}); - -test('a complete initial card uses the unchanged grant without a viewport transaction', async () => { - lines = 3; paint(); await tick(); await tick(); - expect(sends).toEqual(['1\r']); expect(resizes).toEqual([]); expect(viewport.active).toBe(false); -}); - -test('existing scope refusal is not a clipping recovery trigger', async () => { - native.permissionRequests[0]!.input.file_path = path.join(path.dirname(cwd), 'outside', 'review.md'); - await expect(tick()).rejects.toThrow('outside its fixture'); - expect(resizes).toEqual([]); expect(sends).toEqual([]); -}); - -test('a different basename cannot start recovery', async () => { - native.permissionRequests[0]!.input.file_path = path.join(path.dirname(file), 'different.md'); - await expect(tick()).rejects.toThrow('cannot be bound'); - expect(resizes).toEqual([]); expect(sends).toEqual([]); -}); - -test('a changed raw barrier cannot start recovery from the previous frame', async () => { - const frame = await sample(); let error: unknown; - try { reserve(frame); } catch (cause) { error = cause; } - raw += 'later native output'; - expect(await viewport.recover(error, native, frame)).toBe(false); - expect(resizes).toEqual([]); expect(sends).toEqual([]); -}); - -test('a recovery deadline causes no viewport mutation or permission input', async () => { - const expired = new AutoplanFilePermissionViewport({ deadlineAt: Date.now() - 1, granted, session: { - mark: () => raw.length, resizeQuestionViewport: async (_rows, deadline) => { - expect(deadline).toBeLessThan(Date.now()); return null; - }, - } }); - const frame = await sample(); let error: unknown; - try { reserve(frame); } catch (cause) { error = cause; } - expect(await expired.recover(error, native, frame)).toBe(true); - expect(expired.inputMark).toBe(-1); expect(resizes).toEqual([]); expect(sends).toEqual([]); -}); - - -const queueBashDuringRepaint = () => { - const owner = native.permissionRequests[0]!; - owner.nativeToolId = 'owned-edit-tool'; - native.permissionTools.push( - { id: owner.nativeToolId, name: 'Edit', cwd, input: structuredClone(owner.input) }, - { id: 'queued-bash', name: 'Bash', cwd, - input: { command: 'printf queued', description: 'Separate queued command' }, bashPermissionRequestId: null }, - ); -}; - -test('a queued Bash during file repaint cannot own or block the exact Edit grant', async () => { - await tick(); expect(resizes).toEqual([240]); - queueBashDuringRepaint(); - await tick(); expect(resizes).toEqual([240, 480]); expect(sends).toEqual([]); - await tick(); await tick(); - expect(sends).toEqual(['1\r']); - expect([...granted]).toEqual(['request:owned-edit']); - expect([...requests.entries()]).toEqual([['Edit:' + file, { requestId: 'owned-edit', operation: 'edit' }]]); - expect(native.permissionTools.find(tool => tool.id === 'queued-bash')?.bashPermissionRequestId).toBeNull(); - Object.assign(native.permissionRequests[0]!, { result: 'completed', nativeResultAtMs: 2 }); - native.permissionTools = native.permissionTools.filter(tool => tool.name === 'Bash'); - await tick(); expect(resizes).toEqual([240, 480, 120]); expect(viewport.active).toBe(false); - expect(sends).toEqual(['1\r']); expect(granted.has('queued-bash')).toBe(false); -}); - -test.each(['request', 'Edit', 'Write'])('queued Bash cannot hide a competing %s owner', async kind => { - await tick(); queueBashDuringRepaint(); - if (kind === 'request') native.permissionRequests.push({ ...structuredClone(native.permissionRequests[0]!), requestId: 'competitor' }); - else native.permissionTools.push({ id: 'competitor', name: kind, cwd, input: { file_path: file } }); - await expect(tick()).rejects.toThrow('Ambiguous native permission owner'); - expect(resizes).toEqual([240]); expect(sends).toEqual([]); expect(granted.size).toBe(0); -}); - -test('queued Bash cannot conceal a change to the pinned file input', async () => { - await tick(); queueBashDuringRepaint(); - native.permissionRequests[0]!.input.new_string = 'changed plan'; - await expect(tick()).rejects.toThrow('changed ownership or input'); - expect(resizes).toEqual([240]); expect(sends).toEqual([]); expect(granted.size).toBe(0); -}); - -test('a Bash permission frame cannot replace the pinned Edit during recovery', async () => { - await tick(); queueBashDuringRepaint(); - const bashFrame = '\x1b[2J\x1b[H' + [ - ' Bash command', ' printf queued', ' Separate queued command', - ' Do you want to proceed?', ' ❯ 1. Yes', ' 2. No', '', ' Esc to cancel', - ].join('\r\n'); - raw += bashFrame; screen.feed(bashFrame); - await expect(tick()).rejects.toThrow('Visible permission cannot be bound'); - expect(resizes).toEqual([240]); expect(sends).toEqual([]); expect(granted.size).toBe(0); -}); - - -test('a Bash queued before the first repaint still leaves one exact captured Edit owner', async () => { - queueBashDuringRepaint(); - await tick(); expect(resizes).toEqual([240]); expect(sends).toEqual([]); - await tick(); expect(resizes).toEqual([240, 480]); expect(sends).toEqual([]); - await tick(); await tick(); - expect(sends).toEqual(['1\r']); expect([...granted]).toEqual(['request:owned-edit']); - expect([...requests.keys()]).toEqual(['Edit:' + file]); - Object.assign(native.permissionRequests[0]!, { result: 'completed', nativeResultAtMs: 2 }); - native.permissionTools = native.permissionTools.filter(tool => tool.name === 'Bash'); - await tick(); expect(resizes).toEqual([240, 480, 120]); expect(viewport.active).toBe(false); - expect(sends).toEqual(['1\r']); expect(granted.has('queued-bash')).toBe(false); -}); - -test.each(['Edit', 'Write'])('initial queued Bash cannot hide a second %s file owner', async name => { - queueBashDuringRepaint(); - native.permissionTools.push({ id: 'competitor', name, cwd, input: { file_path: file } }); - await expect(tick()).rejects.toThrow('multiple tools are pending'); - expect(viewport.active).toBe(false); expect(resizes).toEqual([]); expect(sends).toEqual([]); -}); - -test('initial queued Bash cannot start recovery with two captured file requests', async () => { - queueBashDuringRepaint(); - native.permissionRequests.push({ ...structuredClone(native.permissionRequests[0]!), requestId: 'competitor' }); - await tick(); - expect(viewport.active).toBe(false); expect(resizes).toEqual([]); expect(sends).toEqual([]); expect(granted.size).toBe(0); -}); - -test('initial queued Bash cannot turn an explicit full-path mismatch into clipping', async () => { - queueBashDuringRepaint(); lines = 3; displayPath = '.claude/other/review.md'; paint(); - await expect(tick()).rejects.toThrow('multiple tools are pending'); - expect(viewport.active).toBe(false); expect(resizes).toEqual([]); expect(sends).toEqual([]); expect(granted.size).toBe(0); -}); - -test('an initial Bash permission card cannot start file recovery', async () => { - queueBashDuringRepaint(); - const bashFrame = '\x1b[2J\x1b[H' + [ - ' Bash command', ' printf queued', ' Separate queued command', - ' Do you want to proceed?', ' ❯ 1. Yes', ' 2. No', '', ' Esc to cancel', - ].join('\r\n'); - raw += bashFrame; screen.feed(bashFrame); - await expect(tick()).rejects.toThrow('multiple tools are pending'); - expect(viewport.active).toBe(false); expect(resizes).toEqual([]); expect(sends).toEqual([]); expect(granted.size).toBe(0); -}); diff --git a/test/autoplan-phase-handoff.test.ts b/test/autoplan-phase-handoff.test.ts index a1c61cd93..ccd38eb4e 100644 --- a/test/autoplan-phase-handoff.test.ts +++ b/test/autoplan-phase-handoff.test.ts @@ -8,7 +8,6 @@ import { initializePlan, prepareMethodology, createSnapshot, amendImplementation import { autoplanPhaseCompletions } from './helpers/autoplan-phase-observer'; import { auditAutoplanMethodReads, loadAutoplanMethodologyBinding } from './helpers/autoplan-method-read-audit'; import { readPlanCountTranscript, type NativePublicToolEvent } from './helpers/plan-count-transcript'; -import { readPlanSkillCompletion } from './helpers/plan-skill-completion'; import captured from './fixtures/autoplan-phase-handoff-6714.json'; const ROOT = resolve(import.meta.dir, '..'); @@ -197,7 +196,6 @@ test('captured parent text and a following tool can share a response without end expect(result.hits.map(hit => hit.phase)).toEqual(phases); expect(result.tools).toHaveLength(4); expect(result.hits.every((hit, index) => hit.ts < Date.parse(result.tools[index]!.timestamp))).toBe(true); - expect(readPlanSkillCompletion(root, textEnvelope!.sessionId, 'Phase 3 complete.')).toBeNull(); // Tool arguments, tool results and sidechain text are not parent announcements. expect(read([{ ...rows[1], message: { ...rows[1]!.message, content: [{ type: 'tool_use', id: 'source', name: 'Bash', input: { command: 'echo "Phase 1 complete."' } }] } }]).hits).toEqual([]); diff --git a/test/autoplan-phase-observation.test.ts b/test/autoplan-phase-observation.test.ts deleted file mode 100644 index 79a184459..000000000 --- a/test/autoplan-phase-observation.test.ts +++ /dev/null @@ -1,505 +0,0 @@ -/** Free ordering regressions for the paid autoplan chain's observed markers. */ -import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; -import * as fs from 'node:fs'; -import * as os from 'node:os'; -import * as path from 'node:path'; -import { createHash } from 'node:crypto'; -import { corroboratedAutoplanPhases, observedAutoplanPhases, readAutoplanTranscript, reserveAutoplanFilePermission, retainAutoplanFailure, validateAutoplanPhaseOrder } from './helpers/autoplan-phase-order'; -import { stripAnsi } from './helpers/claude-pty-runner'; -import type { readPlanSkillQuestions, NativePermissionGrant } from './helpers/plan-skill-questions'; - -describe('autoplan file grants stay inside their owned fixture', () => { - let root: string; - let cwd: string; - let planDir: string; - let native: ReturnType; - let granted: Set; - let requests: Map; - const dialog = (file: string) => `Do you want to create ${file}?\n❯ 1. Yes\n 2. Yes, and switch to accept edits (auto-approve file edits and common file commands) for this session\n 3. No\nEsc to cancel`; - const reserve = (file = String(native.permissionRequests[0]?.input.file_path), visible = dialog(file)) => - reserveAutoplanFilePermission(native, visible, { cwd, planDir, granted, requests }); - beforeEach(() => { - root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'autoplan-permission-'))); - cwd = path.join(root, 'project'); - planDir = path.join(root, 'config', 'plans'); - fs.mkdirSync(cwd); - fs.mkdirSync(planDir, { recursive: true }); - granted = new Set(); - requests = new Map(); - native = { calls: [], ready: false, pendingExitPlanModeIds: [], pendingBytes: 0, - permissionTools: [], permissionResults: [], permissionRequestCapture: true, - permissionRequests: [{ requestId: 'owned-write', capturedAtMs: 1, name: 'Write', cwd, - input: { file_path: path.join(cwd, '.gstack', 'projects', 'fixture', 'restore.md') }, result: 'pending' }] }; - }); - afterEach(() => { fs.rmSync(root, { recursive: true, force: true }); }); - - test('reserves a current fixture-owned restore request only once', () => { - expect(reserve()).toBe(true); - expect(reserve()).toBe(false); - expect([...granted]).toEqual(['request:owned-write']); - }); - - test('allows the launch-owned native plan directory', () => { - native.permissionRequests[0]!.input.file_path = path.join(planDir, 'review.md'); - expect(reserve()).toBe(true); - }); - - test.each(['outside', 'sibling-prefix', 'dotdot'])('rejects the %s path before reserving', kind => { - const file = kind === 'outside' ? path.join(root, 'operator-home', '.gstack', 'restore.md') - : kind === 'sibling-prefix' ? cwd + '-other/restore.md' : path.join(cwd, '..', 'restore.md'); - native.permissionRequests[0]!.input.file_path = file; - expect(() => reserve()).toThrow('outside its fixture'); - expect(granted.size).toBe(0); - }); - - test.skipIf(process.platform === 'win32')('rejects a symlink that redirects a fixture path outside', () => { - fs.mkdirSync(path.join(root, 'outside')); - fs.symlinkSync(path.join(root, 'outside'), path.join(cwd, '.gstack'), 'dir'); - expect(() => reserve()).toThrow('symlink'); - expect(granted.size).toBe(0); - }); - - test('rejects a request from another cwd', () => { - native.permissionRequests[0]!.cwd = root; - expect(() => reserve()).toThrow('cwd differs'); - }); - - test.each(['no-capture', 'no-request', 'partial', 'exit', 'question'])('does not grant with %s evidence', kind => { - if (kind === 'no-capture') native.permissionRequestCapture = false; - if (kind === 'no-request') native.permissionRequests = []; - if (kind === 'partial') native.pendingBytes = 1; - if (kind === 'exit') native.ready = true; - if (kind === 'question') native.calls = [{ id: 'question', result: 'pending', questions: [] }]; - expect(reserve(path.join(cwd, 'restore.md'))).toBe(false); - expect(granted.size).toBe(0); - }); - - test('keeps the shared rejection of a different or ambiguous native owner', () => { - expect(() => reserve(path.join(cwd, 'other.md'))).toThrow('bound to its pending'); - native.permissionTools = [{ id: 'other', name: 'Write', cwd, input: { ...native.permissionRequests[0]!.input } }]; - expect(() => reserve()).toThrow('multiple tools are pending'); - expect(granted.size).toBe(0); - }); - - test('an unrelated pending Bash does not own the current file grant', () => { - native.permissionTools = [{ id: 'other', name: 'Bash', input: { command: 'echo other' } }]; - expect(reserve()).toBe(true); - expect(reserve()).toBe(false); - expect([...granted]).toEqual(['request:owned-write']); - expect(native.permissionTools.map(tool => tool.id)).toEqual(['other']); - }); -}); - -describe('autoplan announcements from the owned main transcript', () => { - const sessionId = 'b4a90d12-0134-4ecf-9931-a2d453cc874a'; - const otherSession = '00000000-0000-4000-8000-000000000000'; - let configDir: string; - const row = (content: unknown, extra: Record = {}) => JSON.stringify({ - type: 'assistant', isSidechain: false, sessionId, - message: { role: 'assistant', content }, ...extra, - }) + '\n'; - const text = (value: string) => [{ type: 'text', text: value }]; - const write = (source: string, project = 'fixture', id = sessionId) => { - const file = path.join(configDir, 'projects', project, `${id}.jsonl`); - fs.mkdirSync(path.dirname(file), { recursive: true }); - fs.writeFileSync(file, source); - return file; - }; - beforeEach(() => { configDir = fs.mkdtempSync(path.join(os.tmpdir(), 'autoplan-transcript-')); }); - afterEach(() => { fs.rmSync(configDir, { recursive: true, force: true }); }); - - test('missing transcript stays pending, and an owned config and UUID are required', () => { - expect(readAutoplanTranscript(configDir, sessionId)).toEqual({ file: null, phases: [], completedLines: 0, pendingBytes: 0 }); - expect(() => readAutoplanTranscript(null, sessionId)).toThrow('owned hermetic'); - expect(() => readAutoplanTranscript(configDir, '../other')).toThrow('UUID'); - }); - - test('reads the captured assistant schema and canonical Markdown announcements', () => { - // Same role/content shape and four lines as ship-phase-render-probe-attempt2.json. - const file = write(row(text('**Phase 1 complete.**\n**Phase 2 complete.**\n> **Phase 2.5 complete.**\nPhase 3 complete.'))); - const observation = readAutoplanTranscript(configDir, sessionId); - expect(observation).toEqual({ file, phases: [1, 2, 2.5, 3], completedLines: 1, pendingBytes: 0 }); - const visible = stripAnsi('\x1b[2CPhase\x1b[9G1\x1b[11Gcomplete.\nPhase2complete.\nPhase2.5complete.\nPhase3complete.'); - expect(corroboratedAutoplanPhases(observation.phases, visible)).toEqual([1, 2, 2.5, 3]); - }); - - test('tool inputs/results, thinking, user text, other sessions, and sidechains cannot announce phases', () => { - const marker = '**Phase 3 complete.**'; - write([ - row([{ type: 'tool_use', input: { content: marker } }, { type: 'thinking', thinking: marker }]), - row([{ type: 'tool_result', content: marker }]), - row(text(marker), { type: 'user', message: { role: 'user', content: text(marker) } }), - row(text(marker), { isSidechain: true }), - row(text(marker), { parent_tool_use_id: 'child-call' }), - row(text(marker), { sessionId: otherSession }), - row(text(marker), { message: { role: 'user', content: text(marker) } }), - row(text('**Phase 1 complete.**')), - ].join('')); - expect(readAutoplanTranscript(configDir, sessionId).phases).toEqual([1]); - }); - - test('quoted future markers and fenced or indented code are not announcements', () => { - write(row(text([ - 'I will print **Phase 3 complete.** later.', - '"Phase 3 complete."', - '```markdown', '**Phase 3 complete.**', '```', - '~~~', 'Phase 4 complete.', '~~~', - ' Phase 3 complete.', - '**Phase 1 complete.** Codex: 2 concerns.', - ].join('\n')))); - expect(readAutoplanTranscript(configDir, sessionId).phases).toEqual([1]); - }); - - test('reads only the exact UUID in direct project directories, never subagents or other sessions', () => { - write(row(text('Phase 3 complete.')), 'fixture', otherSession); - write(row(text('Phase 3 complete.')), `fixture/${sessionId}/subagents`); - expect(readAutoplanTranscript(configDir, sessionId).file).toBeNull(); - write(row(text('Phase 1 complete.'))); - expect(readAutoplanTranscript(configDir, sessionId).phases).toEqual([1]); - }); - - test('ambiguous exact-session files fail instead of selecting an arbitrary project', () => { - write(row(text('Phase 1 complete.')), 'one'); - write(row(text('Phase 3 complete.')), 'two'); - expect(() => readAutoplanTranscript(configDir, sessionId)).toThrow('Ambiguous'); - }); - - test.skipIf(process.platform === 'win32')('does not follow project or transcript symlinks', () => { - const external = path.join(configDir, 'outside-projects'); - fs.mkdirSync(external); - fs.writeFileSync(path.join(external, `${sessionId}.jsonl`), row(text('Phase 3 complete.'))); - const projects = path.join(configDir, 'projects'); - fs.mkdirSync(projects); - fs.symlinkSync(external, path.join(projects, 'linked-project'), 'dir'); - expect(readAutoplanTranscript(configDir, sessionId).file).toBeNull(); - fs.mkdirSync(path.join(projects, 'fixture')); - fs.symlinkSync(path.join(external, `${sessionId}.jsonl`), path.join(projects, 'fixture', `${sessionId}.jsonl`)); - expect(() => readAutoplanTranscript(configDir, sessionId)).toThrow('not a regular file'); - }); - - test('partial final JSONL remains pending until its newline is written', () => { - const final = row(text('Phase 3 complete.')); - const split = Math.floor(final.length / 2); - const file = write(row(text('Phase 1 complete.')) + final.slice(0, split)); - expect(readAutoplanTranscript(configDir, sessionId).phases).toEqual([1]); - expect(readAutoplanTranscript(configDir, sessionId).pendingBytes).toBeGreaterThan(0); - fs.appendFileSync(file, final.slice(split, -1)); - expect(readAutoplanTranscript(configDir, sessionId).phases).toEqual([1]); - fs.appendFileSync(file, '\n'); - expect(readAutoplanTranscript(configDir, sessionId).phases).toEqual([1, 3]); - }); - - test('malformed completed JSONL fails with file/line diagnostics without exposing contents', () => { - const file = write(row(text('Phase 1 complete.')) + '{"sensitive-fixture-data":broken}\n'); - expect(() => readAutoplanTranscript(configDir, sessionId)).toThrow(`${file}:2`); - try { readAutoplanTranscript(configDir, sessionId); } catch (error) { - expect(String(error)).not.toContain('sensitive-fixture-data'); - } - }); - - test('first assistant observation order and unknown phase errors are preserved', () => { - write(row(text('Phase 1 complete.\nPhase 2.5 complete.\nPhase 2 complete.\nPhase 1 complete.\nPhase 3 complete.'))); - const phases = readAutoplanTranscript(configDir, sessionId).phases; - expect(phases).toEqual([1, 2.5, 2, 3]); - expect(() => validateAutoplanPhaseOrder(phases)).toThrow('optional Design (2), optional DX (2.5)'); - write(row(text('Phase 1 complete.\nPhase 4 complete.\nPhase 3 complete.'))); - expect(() => validateAutoplanPhaseOrder(readAutoplanTranscript(configDir, sessionId).phases)).toThrow(); - }); - - test('failed chain retains exact owned commands and pending status after native cleanup', () => { - const command = 'printf "Phase 3 complete."; codex exec "Review the design — café"'; - write(row([ - { type: 'thinking', thinking: 'private-reasoning', signature: 'private-signature' }, - { type: 'tool_use', id: 'design-command', name: 'Bash', input: { command, timeout: 600_000 } }, - ])); - write(row([{ type: 'tool_use', id: 'foreign', name: 'Bash', input: { command: 'foreign-command' } }]), 'foreign', otherSession); - const destination = fs.mkdtempSync(path.join(os.tmpdir(), 'autoplan-retained-')); - try { - const saved = retainAutoplanFailure({ configDir, sessionId, evalDir: destination, - observation: { outcome: 'timeout', phases: [1] }, raw: () => '\x1b[2JRunning design command', visible: () => 'Running design command' }); - expect(saved).not.toBeNull(); - fs.rmSync(configDir, { recursive: true, force: true }); - const contents = fs.readFileSync(saved!, 'utf8'); - const record = JSON.parse(contents); - expect(JSON.parse(record.calls[0].inputJson.text)).toEqual({ command, timeout: 600_000 }); - expect(record.calls[0].result).toBe('pending'); - expect(record.pendingIds[0].text).toBe('design-command'); - expect(JSON.parse(record.observation.text)).toEqual({ outcome: 'timeout', phases: [1] }); - expect(contents).not.toContain('private-reasoning'); - expect(contents).not.toContain('private-signature'); - expect(contents).not.toContain('foreign-command'); - expect(fs.statSync(saved!).mode & 0o777).toBe(0o600); - } finally { fs.rmSync(destination, { recursive: true, force: true }); } - }); - - test('diagnostics preserve completed/error tools and mark partial input and native tails explicitly', () => { - const command = 'x'.repeat(40_000); - const calls = Array.from({ length: 20 }, (_, index) => ({ type: 'tool_use', id: `call-${index}`, name: 'Bash', input: { command } })); - write(row(calls) + row([], { type: 'user', message: { role: 'user', content: [ - { type: 'tool_result', tool_use_id: 'call-18', is_error: false }, - { type: 'tool_result', tool_use_id: 'call-19', is_error: true }, - ] } }) + '{"partial":'); - const destination = fs.mkdtempSync(path.join(os.tmpdir(), 'autoplan-retained-')); - try { - const saved = retainAutoplanFailure({ configDir, sessionId, evalDir: destination, - observation: { outcome: 'timeout' }, raw: () => '界'.repeat(70_000), visible: () => 'partial input' }); - const record = JSON.parse(fs.readFileSync(saved!, 'utf8')); - expect(record.pendingBytes).toBeGreaterThan(0); - expect(record.calls).toHaveLength(16); - expect(record.callsOmitted).toBe(4); - expect(record.calls[0].inputJson.truncated).toBe(true); - expect(record.calls.at(-1).result).toBe('error'); - expect(record.calls.at(-2).result).toBe('completed'); - expect(record.pendingCount).toBe(18); - expect(record.rawCodeUnits).toBe(70_000); - expect(record.rawTail.text.length).toBe(65_536); - expect(record.rawTail.omittedPrefixCodeUnits).toBe(4_464); - const before = fs.readFileSync(saved!, 'utf8'); - expect(retainAutoplanFailure({ configDir, sessionId, evalDir: destination, - observation: null, raw: () => '', visible: () => '' })).toBeNull(); - expect(fs.readFileSync(saved!, 'utf8')).toBe(before); - } finally { fs.rmSync(destination, { recursive: true, force: true }); } - }); - - test('diagnostic observation failure cannot replace the test outcome', () => { - expect(retainAutoplanFailure({ configDir, sessionId, observation: { outcome: 'timeout' }, - raw: () => { throw new Error('terminal capture failed'); }, visible: () => '' })).toBeNull(); - }); - - const pendingQuestion = (id = 'pending-question', question = 'D4 — Choose one remedy') => ({ id, result: 'pending' as const, - questions: [{ header: 'Remedy', question, multiSelect: false, - options: [{ label: 'Fix it', description: 'Apply the remedy' }, { label: 'Defer', description: 'Keep current behavior' }] }], - }); - const retainQuestions = (calls = [pendingQuestion()], raw = () => 'PRIVATE_SCREEN') => { - const saved = retainAutoplanFailure({ configDir, sessionId, evalDir: path.join(configDir, 'retained'), - observation: { observedBeforeRetention: true }, raw, visible: () => 'PRIVATE_SCREEN', - counting: { native: { calls, ready: false, pendingExitPlanModeIds: [], pendingBytes: 0, permissionTools: [], - permissionResults: [], permissionRequestCapture: true, permissionRequests: [] }, dialog: 'PRIVATE_SCREEN' } }); - expect(saved).not.toBeNull(); - expect(fs.statSync(saved!).mode & 0o777).toBe(0o600); - expect(fs.statSync(path.dirname(saved!)).mode & 0o777).toBe(0o700); - return JSON.parse(fs.readFileSync(saved!, 'utf8')); - }; - - test.each([false, true])('failure frame retention keeps sampled text separate from later history (long=%s)', long => { - write(row([{ type: 'thinking', thinking: 'PRIVATE_THINKING', signature: 'PRIVATE_SIGNATURE' }])); - const text = long ? '😀'.repeat(35_000) : 'Current permission viewport\n❯ 1. Yes\n 2. No'; - const frame = { text, rawEnd: 1234, observedAtMs: 22, questionSince: 100, viewportInputSince: 110 }; - const saved = retainAutoplanFailure({ configDir, sessionId, evalDir: path.join(configDir, 'retained'), - observation: { observedAtMs: 99 }, raw: () => 'PRIVATE_LATER_RAW_HISTORY', visible: () => 'PRIVATE_LATER_VISIBLE_HISTORY', - counting: { native: null, dialog: text, frame } }); - expect(saved).not.toBeNull(); - const record = JSON.parse(fs.readFileSync(saved!, 'utf8')); - expect(record.counting.decodedFrame).toEqual({ source: 'last-sampled-current-screen', ...frame, - text: text.slice(0, 65_536), codeUnits: text.length, truncated: long, - sha256: createHash('sha256').update(text).digest('hex') }); - expect(JSON.stringify(record)).not.toContain('PRIVATE_'); - expect(fs.statSync(saved!).mode & 0o777).toBe(0o600); - expect(fs.statSync(path.dirname(saved!)).mode & 0o777).toBe(0o700); - }); - - test('failure frame retention keeps fallback history hashed when no decoded sample exists', () => { - write(row([])); - const record = retainQuestions([]); - expect(record.counting.decodedFrame).toBeNull(); - expect(JSON.stringify(record)).not.toContain('PRIVATE_SCREEN'); - }); - - test('pending-question retention includes unfinished invocation structure while excluding foreign and unrelated payloads', () => { - const call = pendingQuestion(); - const input = { questions: call.questions }; - const block = { type: 'tool_use', id: call.id, name: 'AskUserQuestion', input }; - write(row([block], { timestamp: '2026-09-10T00:00:01Z', cwd: '/owned', message: { role: 'assistant', stop_reason: null, content: [block] } }) - + row([{ type: 'thinking', thinking: 'PRIVATE_THINKING' }, { type: 'tool_use', id: 'other', name: 'Bash', input: { command: 'PRIVATE_COMMAND' } }]) - + row([], { sessionId: otherSession, type: 'user', message: { role: 'user', content: [{ type: 'tool_result', tool_use_id: call.id, content: 'PRIVATE_FOREIGN_RESULT' }] } }) - + row([], { isSidechain: true, type: 'user', message: { role: 'user', content: [{ type: 'tool_result', tool_use_id: call.id, content: 'PRIVATE_SIDECHAIN_RESULT' }] } }) - + row([], { type: 'user', message: { role: 'user', content: [{ type: 'tool_result', tool_use_id: 'other', content: 'PRIVATE_UNRELATED_RESULT' }] } })); - const record = retainQuestions(); - const evidence = record.counting.questionEvidence; - expect(evidence.observed[0]).toMatchObject({ observedResult: 'pending', resultAtRetention: 'pending' }); - expect(JSON.parse(evidence.observed[0].questionsJson.text)).toEqual(call.questions); - expect(evidence.nativeBlocks.rows).toHaveLength(1); - expect(evidence.nativeBlocks.rows[0]).toMatchObject({ rowIndex: 0, stopReason: null, timestamp: { text: '2026-09-10T00:00:01Z' }, cwd: { text: '/owned' } }); - expect(JSON.parse(evidence.nativeBlocks.rows[0].blockJson.text)).toEqual(block); - expect(JSON.stringify(record)).not.toContain('PRIVATE_'); - }); - - test.each([false, true])('pending-question retention distinguishes a late matching native result (error=%s)', isError => { - const call = pendingQuestion(); - const block = { type: 'tool_use', id: call.id, name: 'AskUserQuestion', input: { questions: call.questions } }; - const file = write(row([block], { message: { role: 'assistant', stop_reason: 'tool_use', content: [block] } })); - const result = { type: 'tool_result', tool_use_id: call.id, is_error: isError, content: isError ? 'Question failed' : 'Answer: Fix it' }; - const record = retainQuestions([call], () => { - fs.appendFileSync(file, row([], { timestamp: '2026-09-10T00:00:02Z', type: 'user', toolUseResult: { answers: { 'D4 — Choose one remedy': 'Fix it' } }, - message: { role: 'user', content: [result] } })); - return 'PRIVATE_SCREEN'; - }); - const evidence = record.counting.questionEvidence; - expect(evidence.observed[0]).toMatchObject({ observedResult: 'pending', resultAtRetention: isError ? 'error' : 'completed' }); - expect(call.result).toBe('pending'); - expect(evidence.nativeBlocks.rows).toHaveLength(2); - expect(JSON.parse(evidence.nativeBlocks.rows[1].blockJson.text)).toEqual(result); - expect(JSON.parse(evidence.nativeBlocks.rows[1].toolUseResultJson.text)).toEqual({ answers: { 'D4 — Choose one remedy': 'Fix it' } }); - expect(evidence.nativeBlocks.rows[1].timestamp.text).toBe('2026-09-10T00:00:02Z'); - }); - - test('pending-question retention marks per-payload truncation and preserves the native partial-byte boundary', () => { - const call = pendingQuestion('large-question', 'é'.repeat(70_000)); - const block = { type: 'tool_use', id: call.id, name: 'AskUserQuestion', input: { questions: call.questions } }; - write(row([block]) + '{"unfinished":'); - const record = retainQuestions([call]); - expect(record.pendingBytes).toBeGreaterThan(0); - const evidence = record.counting.questionEvidence; - expect(evidence.observed[0].questionsJson).toMatchObject({ truncated: true, codeUnits: JSON.stringify(call.questions).length }); - expect(evidence.observed[0].questionsJson.text.length).toBe(65_536); - expect(evidence.nativeBlocks.rows[0].blockJson.truncated).toBe(true); - expect(evidence.nativeBlocks.rows[0].blockJson.text.length).toBe(65_536); - }); - - test('pending-question retention bounds the selected IDs and native blocks without leaking omitted payloads', () => { - const calls = Array.from({ length: 20 }, (_, i) => pendingQuestion(`question-${i}`, i < 4 ? 'PRIVATE_OMITTED' : `Question ${i}`)); - const blocks = calls.map(call => ({ type: 'tool_use', id: call.id, name: 'AskUserQuestion', input: { questions: call.questions } })); - write(row(blocks) + row(blocks) + row(blocks)); - const evidence = retainQuestions(calls).counting.questionEvidence; - expect(evidence.count).toBe(20); - expect(evidence.omitted).toBe(4); - expect(evidence.observed).toHaveLength(16); - expect(evidence.nativeBlocks.count).toBe(48); - expect(evidence.nativeBlocks.omitted).toBe(16); - expect(evidence.nativeBlocks.rows).toHaveLength(32); - expect(JSON.stringify(evidence)).not.toContain('PRIVATE_OMITTED'); - }); -}); - -describe('rendered corroboration of authoritative assistant announcements', () => { - test('tool-only markers cannot complete the chain', () => { - const visible = 'Bash(printf "Phase 1 complete. Phase 3 complete.")'; - expect(observedAutoplanPhases(visible)).toEqual([1, 3]); - expect(corroboratedAutoplanPhases([], visible)).toEqual([]); - }); - - test('early Eng previews do not establish order or satisfy Eng visibility after CEO', () => { - const preview = 'Read: Phase3complete.\n'; - expect(corroboratedAutoplanPhases([], preview)).toEqual([]); - expect(corroboratedAutoplanPhases([1], preview + 'Phase1complete.')).toEqual([1]); - expect(corroboratedAutoplanPhases([1, 3], preview + 'Phase1complete.')).toEqual([1]); - expect(corroboratedAutoplanPhases([1, 3], preview + 'Phase1complete.\nPhase3complete.')).toEqual([1, 3]); - }); - - test('every announced optional phase must render, and a visible-only optional phase cannot alter order', () => { - expect(corroboratedAutoplanPhases([1, 2, 2.5, 3], 'Phase1complete. Phase3complete.')).toEqual([1]); - expect(corroboratedAutoplanPhases([1, 3], 'Phase2.5complete. Phase1complete. Phase3complete.')).toEqual([1, 3]); - }); - - test('valid-looking tool previews cannot launder a wrong assistant announcement order', () => { - const assistant = [1, 2.5, 2, 3]; - const visible = 'Phase1complete. Phase2complete. Phase2.5complete. Phase3complete.\n' - + 'Phase1complete. Phase2.5complete. Phase2complete. Phase3complete.'; - expect(corroboratedAutoplanPhases(assistant, visible)).toEqual(assistant); - expect(() => validateAutoplanPhaseOrder(assistant)).toThrow(); - }); -}); - -describe('autoplan completion markers from rendered output', () => { - test('reads actual Claude 2.1.257 cursor-positioned output after ANSI stripping', () => { - // Reduced from a real PTY capture; its saved assistant response contains - // all four **Phase N complete.** lines, but the terminal omits the stars. - const raw = '\x1b[2C\x1b[9BPhase\x1b[9G1\x1b[11Gcomplete.\n' - + '\x1b[2C\x1b[1BPhase\x1b[9G2\x1b[11Gcomplete.\n' - + '\x1b[2C\x1b[11BPhase\x1b[9G2.5\x1b[13Gcomplete.\n' - + '\x1b[2C\x1b[12BPhase\x1b[9G3\x1b[11Gcomplete.'; - const visible = stripAnsi(raw); - expect(visible).toBe('Phase1complete.\nPhase2complete.\nPhase2.5complete.\nPhase3complete.'); - expect(observedAutoplanPhases(visible)).toEqual([1, 2, 2.5, 3]); - }); - - test.each([ - 'Phase 1 complete.\nPhase 3 complete.', - '**Phase 1 complete.**\n**Phase 3 complete.**', - '**Phase 1 complete**\n**Phase 3 complete**', - 'Phase1complete. Phase3complete.', - ])('accepts plain, Markdown, and compacted markers: %s', visible => { - expect(observedAutoplanPhases(visible)).toEqual([1, 3]); - }); - - test('keeps decimal DX, duplicates, and actual match order within one poll', () => { - expect(observedAutoplanPhases('Phase2.5complete. Phase 2 complete. Phase2.5complete.')) - .toEqual([2.5, 2, 2.5]); - }); - - test.each([ - 'SubPhase1complete.', - 'pre_Phase 1 complete.', - 'Phase1completed.', - 'Phase 1 completeness.', - 'Phase1complete_more', - 'Phase 1 incomplete.', - 'Phase 3', - 'Phase3 pending completion.', - 'Reply with word Phase, then number 3, then word complete.', - ])('rejects incomplete markers and unrelated words: %s', visible => { - expect(observedAutoplanPhases(visible)).toEqual([]); - }); - - test('retains unknown phases for the order validator to reject', () => { - const phases = observedAutoplanPhases('Phase1complete. Phase4complete. Phase3complete.'); - expect(phases).toEqual([1, 4, 3]); - expect(() => validateAutoplanPhaseOrder(phases)).toThrow(); - }); - - test('extraction does not sort a reversed Design/DX stream into valid order', () => { - const phases = observedAutoplanPhases('Phase1complete. Phase2.5complete. Phase2complete. Phase3complete.'); - expect(phases).toEqual([1, 2.5, 2, 3]); - expect(() => validateAutoplanPhaseOrder(phases)).toThrow('optional Design (2), optional DX (2.5)'); - }); -}); - -describe('autoplan completion order from the observed stream', () => { - test('a correctly ordered same-poll batch passes even when timestamps are identical', () => { - const hits = [1, 2, 2.5, 3].map(phase => ({ phase, ts: 1234 })); - expect(() => validateAutoplanPhaseOrder(hits.map(hit => hit.phase))).not.toThrow(); - }); - - test.each([ - [1, 3], - [1, 2, 3], - [1, 2.5, 3], - ].map(phases => ({ phases })))('optional phases may be absent: %j', ({ phases }) => { - expect(() => validateAutoplanPhaseOrder(phases)).not.toThrow(); - }); - - test.each([ - [], - [1], - [3], - [2, 2.5], - ].map(phases => ({ phases })))('missing required completion fails: %j', ({ phases }) => { - expect(() => validateAutoplanPhaseOrder(phases)).toThrow('requires CEO (1) and Eng (3)'); - }); - - test.each([ - [3, 1], - [2, 1, 3], - [2.5, 1, 3], - ].map(phases => ({ phases })))('inverted required or preceding optional phases fail: %j', ({ phases }) => { - expect(() => validateAutoplanPhaseOrder(phases)).toThrow(); - }); - - test('Design must precede DX when both completed', () => { - expect(() => validateAutoplanPhaseOrder([1, 2.5, 2, 3])).toThrow('optional Design (2), optional DX (2.5)'); - }); - - test.each([ - [1, 3, 2], - [1, 3, 2.5], - ].map(phases => ({ phases })))('Eng cannot precede a later completed phase: %j', ({ phases }) => { - expect(() => validateAutoplanPhaseOrder(phases)).toThrow('Eng (3) must complete last'); - }); - - test.each([ - [1, 2, 2, 3], - [1, 4, 3], - ].map(phases => ({ phases })))('duplicate or unknown first-observation markers fail: %j', ({ phases }) => { - expect(() => validateAutoplanPhaseOrder(phases)).toThrow(); - }); -}); diff --git a/test/autoplan-rendered-batch-at.test.ts b/test/autoplan-rendered-batch-at.test.ts deleted file mode 100644 index 715cac183..000000000 --- a/test/autoplan-rendered-batch-at.test.ts +++ /dev/null @@ -1,70 +0,0 @@ -import { capturedPathRebaser } from './helpers/captured-paths'; -import {expect,test} from 'bun:test'; -import fs from 'node:fs';import os from 'node:os';import path from 'node:path'; -import fixture from './fixtures/autoplan-rendered-batch-at.json'; -import * as permission from './helpers/autoplan-artifact-permission'; -import {readPendingAutoplanArtifact} from './helpers/autoplan-artifact-recorder'; -import {readPlanCountTranscript,type NativePublicToolEvent} from './helpers/plan-count-transcript'; -import {E2E_TOUCHFILES,selectTests} from './helpers/touchfiles'; -function replay(){ - const root=fs.mkdtempSync(path.join(os.tmpdir(),'gstack-ap-batch-')),old=path.dirname(path.dirname(fixture.stateRoot)); - const runtime=path.join(root,path.basename(old)),cwd=path.join(root,path.basename(fixture.cwd)); - const rebase=capturedPathRebaser([[old,runtime],[fixture.cwd,cwd]]); - const hook=rebase.json(fixture.hook),stateRoot=rebase.file(fixture.stateRoot),config=rebase.file(fixture.config),file=hook.pending.file; - const events=rebase.json(fixture.publicTools) as NativePublicToolEvent[]; - const now=Date.parse(fixture.viewportCapturedAt),startedAt=Date.parse(fixture.commandStartedAt); - fs.mkdirSync(path.dirname(file),{recursive:true});fs.writeFileSync(file,fixture.before,{mode:0o644}); - const mtime=Number(BigInt(fixture.targetStat.mtimeNs))/1e9;fs.utimesSync(file,mtime,mtime);fs.mkdirSync(cwd,{recursive:true});fs.mkdirSync(path.dirname(hook.pending.transcriptPath),{recursive:true}); - const records=events.map(e=>({sessionId:e.sessionId,cwd,isSidechain:false,timestamp:e.timestamp,requestId:e.requestId,message:{id:e.messageId,role:e.kind==='use'?'assistant':'user',content:e.kind==='use'?[{type:'tool_use',id:e.toolUseId,name:e.name,input:e.input}]:[{type:'tool_result',tool_use_id:e.toolUseId,content:e.content??'',is_error:e.isError}]}})); - fs.writeFileSync(hook.pending.transcriptPath,records.map(e=>JSON.stringify(e)).join('\n')+'\n');const hookFile=path.join(root,'hook.json');fs.writeFileSync(hookFile,JSON.stringify(hook)); - const publicTools:NativePublicToolEvent[]=[];const transcript=readPlanCountTranscript(config,cwd,e=>publicTools.push(e));const pending=readPendingAutoplanArtifact(hookFile,cwd,config,stateRoot,startedAt,publicTools,now,true); - const context={cwd,ownedStateRoot:stateRoot,ownedNativePlansRoot:path.join(config,'plans'),commandStartedAt:startedAt,now,viewportCapturedAt:now,transcriptStatus:transcript.status,publicTools,pending}; - return {root,file,hook,context,viewport:rebase.text(fixture.viewport),dispose:()=>fs.rmSync(root,{recursive:true,force:true})}; -} -type R=ReturnType; -const invoke=(r:R,seen=new Set())=>permission.publishedAutoplanArtifactPermissionInput(r.viewport,r.context,seen); -const current=(r:R)=>r.context.publicTools.find(e=>e.kind==='use'&&e.toolUseId===r.hook.pending.toolUseId)!; -const queued=(r:R)=>r.context.publicTools.filter(e=>e.kind==='use'&&e.name==='Edit'&&e!==current(r)&&!r.context.publicTools.some(x=>x.kind==='result'&&x.toolUseId===e.toolUseId)); -const waiting=(r:R)=>r.context.publicTools.find(e=>e.kind==='use'&&e.name==='Bash')!; -const previous=(r:R)=>r.context.publicTools.find(e=>e.kind==='use'&&e.toolUseId==='toolu_0199q2iK6Pa1xTqiZGNqq81u')!; -const complete=(r:R,e:NativePublicToolEvent,isError=false)=>r.context.publicTools.push({kind:'result',sessionId:e.sessionId,toolUseId:e.toolUseId,timestamp:new Date(r.context.now).toISOString(),isError}); -const cases:Array<[string,(r:R)=>void]>=[ - ['Read is not publication history',r=>{previous(r).name='Read'}],['foreign history file',r=>{previous(r).input!.file_path=r.file+'.other'}], - ['foreign history message',r=>{previous(r).messageId='msg_foreign'}],['foreign history request',r=>{previous(r).requestId='req_foreign'}], - ['unrelated replacement',r=>{previous(r).input!.new_string='## Clarifications from spec review round 2'}], - ['failed history',r=>{r.context.publicTools.find(e=>e.kind==='result'&&e.toolUseId===previous(r).toolUseId)!.isError=true}], - ['missing history completion',r=>{r.context.publicTools=r.context.publicTools.filter(e=>!(e.kind==='result'&&e.toolUseId===previous(r).toolUseId))}], - ['foreign waiting message',r=>{waiting(r).messageId='msg_foreign'}],['foreign waiting request',r=>{waiting(r).requestId='req_foreign'}],['foreign waiting session',r=>{waiting(r).sessionId='foreign'}], - ['different waiting command',r=>{waiting(r).input!.command='echo different'}],['missing waiting use',r=>{const w=waiting(r);r.context.publicTools=r.context.publicTools.filter(e=>e!==w)}], - ['completed waiting command',r=>{complete(r,waiting(r))}],['failed waiting command',r=>{complete(r,waiting(r),true)}], - ['foreign queued target',r=>{queued(r)[0]!.input!.file_path=r.file+'.other'}],['foreign queued batch',r=>{queued(r)[0]!.messageId='msg_foreign'}],['queued Write',r=>{queued(r)[0]!.name='Write'}], - ['started queued edit',r=>{r.context.pending!.hookSeenIds!.push(queued(r)[0]!.toolUseId)}],['completed queued edit',r=>{complete(r,queued(r)[0]!)}], - ['different active hook',r=>{r.context.pending!.toolUseId=queued(r)[0]!.toolUseId}],['changed current request',r=>{current(r).input!.new_string+=' changed'}], - ['missing digest',r=>{delete r.context.pending!.editDigest}],['changed digest',r=>{r.context.pending!.editDigest!.requestSHA256='0'.repeat(64)}], - ['changed current file',r=>{fs.appendFileSync(r.file,'changed');fs.utimesSync(r.file,new Date(0),new Date(0))}],['file newer than hook',r=>{fs.utimesSync(r.file,new Date(r.context.now),new Date(r.context.now))}], - ['no hook',r=>{r.context.pending=undefined}],['missing transcript',r=>{r.context.transcriptStatus='missing'}],['future command',r=>{r.context.commandStartedAt=r.context.now+1}], - ['foreign current session',r=>{current(r).sessionId='foreign'}],['completed current request',r=>{complete(r,current(r))}], -]; -for(const[name,change]of cases)test(`current native authorization survives renderer normalization: ${name}`,()=>{const r=replay();try{change(r);expect(invoke(r)).toBeNull()}finally{r.dispose()}}); - -test('exact public batch and actual file stat authorize only the pending CEO edit',()=>{const r=replay();try{ - expect(r.context.pending?.toolUseId).toBe(fixture.hook.pending.toolUseId);expect(r.context.publicTools).toHaveLength(10);expect(queued(r)).toHaveLength(2); - expect(fs.statSync(r.file).size).toBe(fixture.targetStat.size);expect(Math.floor(fs.statSync(r.file).mtimeMs)).toBe(Number(BigInt(fixture.targetStat.mtimeNs)/1_000_000n)); - expect(permission.autoplanArtifactPermissionInput(r.viewport,r.context,new Set())).toBeNull();expect(permission.pendingAutoplanArtifactPermissionInput(r.viewport,r.context,new Set())).toBeNull(); - const expected={input:'1\r',signature:fixture.hook.sessionId+':'+fixture.hook.pending.toolUseId,file:r.file};expect(invoke(r)).toEqual(expected); - expect(invoke(r,new Set([expected.signature]))).toBeNull();expect(invoke(r,new Set([permission.autoplanArtifactMenuKey(r.viewport)]))).toBeNull(); - r.viewport=r.viewport.slice(r.viewport.indexOf('────────────────'));expect(invoke(r)).toEqual(expected); - expect(fixture.provenance.paidOutcomesReclassified).toBe(false);expect(fixture.provenance.originalOutcome).toBe('operator-cancelled-incomplete'); -}finally{r.dispose()}}); -const screens:Array<[string,(s:string)=>string]>=[ - ['source example',s=>'Example:\n'+s],['quoted screen',s=>s.split('\n').map(l=>'> '+l).join('\n')], - ['unrelated clipped row',s=>s.replace('e, flag-off landing), endpoint p95 check on staging.','This is unrelated current prose; approve all commands.')],['short clipped row',s=>s.replace(/^.*\n/,' staging.\n')], - ['extra clipped row',s=>s.replace(/^.*\n/,'$& Another unbound prefix row.\n')], - ['extra title',s=>s.replace('● Update(','● Update(~/.gstack/foreign.md)\n\n● Update(')],['missing title',s=>s.replace(/^● Update\([^\n]+\)\n/m,'')],['foreign title',s=>s.replace('● Update(~/.gstack/','● Update(/foreign/')], - ['foreign waiting path',s=>s.replace(/Bash\(cd [^\s]+/,'Bash(cd /other/')],['finished command display',s=>s.replace('Waiting…','Done')], - ['active panel target mismatch',s=>s.replace(' Edit file\n …',' Edit file\n …foreign/')], - ['different addition',s=>s.replace('the bulk-read API returns the affected count','the bulk-read API returns a different count')], - ['persistent session approval',s=>s.replace('❯ 1. Yes','❯ 2. Yes')],['trailing prose',s=>s+'\nAnother active request'], -]; -for(const[name,change]of screens)test(`display evidence remains scoped: ${name}`,()=>{const r=replay();try{r.viewport=change(r.viewport);expect(invoke(r)).toBeNull()}finally{r.dispose()}}); -test('only Autoplan discovers the public fixture and regression',()=>{for(const file of ['test/autoplan-rendered-batch-at.test.ts','test/fixtures/autoplan-rendered-batch-at.json'])expect(selectTests([file],E2E_TOUCHFILES,[]).selected).toEqual(['autoplan-chain-pty'])}); diff --git a/test/autoplan-repeated-header-ak.test.ts b/test/autoplan-repeated-header-ak.test.ts deleted file mode 100644 index 6b2089f85..000000000 --- a/test/autoplan-repeated-header-ak.test.ts +++ /dev/null @@ -1,91 +0,0 @@ -import { afterEach, expect, test } from 'bun:test'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import captured from './fixtures/autoplan-repeated-header-ak.json'; -import published from './fixtures/autoplan-edit-prefix-ai.json'; -import { autoplanArtifactPermissionInput, pendingAutoplanArtifactPermissionInput, autoplanArtifactMenuKey } from './helpers/autoplan-artifact-permission'; -import type { NativePublicToolEvent } from './helpers/plan-count-transcript'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; -const roots: string[] = []; -afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, {recursive:true,force:true}); }); -function replay() { - const root = fs.mkdtempSync(path.join(os.tmpdir(),'ap-repeat-ak-')); roots.push(root); - const cwd = path.join(root,path.basename(captured.cwd)), ownedStateRoot = path.join(root,'home','.gstack'); - const file = path.normalize(captured.pending.file.replace(captured.ownedStateRoot,ownedStateRoot)); - fs.mkdirSync(cwd,{recursive:true}); fs.mkdirSync(path.dirname(file),{recursive:true}); - fs.writeFileSync(file,captured.events[0]!.input!.content!); - const old = new Date(Date.parse(captured.pending.timestamp)-1000); fs.utimesSync(file,old,old); - const events = structuredClone(captured.events) as NativePublicToolEvent[]; - for (const e of events) if (e.input?.file_path===captured.pending.file) e.input.file_path=file; - const context={cwd,ownedStateRoot,commandStartedAt:Date.parse(events[0]!.timestamp)-1,now:captured.viewportCapturedAt, - viewportCapturedAt:captured.viewportCapturedAt,transcriptStatus:'ready',publicTools:events, - pending:{...captured.pending,source:'pre_tool_use' as const,tool:'Edit' as const,file}}; - const viewport=captured.viewport.replace(/^ …[^\n]+$/m,' …'+path.relative(ownedStateRoot,file)); - return {root,file,context,viewport}; -} -const pick=(r:ReturnType,seen=new Set())=>pendingAutoplanArtifactPermissionInput(r.viewport,r.context,seen); - -test('the exact homogeneous repeated native title prefix preserves the current owned Edit',()=>{ - const r=replay(); expect(pick(r)).toEqual({input:'1\r',signature:r.context.pending.sessionId+':'+r.context.pending.toolUseId,file:r.file}); - expect(autoplanArtifactPermissionInput(r.viewport,r.context,new Set())).toBeNull(); -}); -test('two through seven identical owned titles and harmless blank spacing preserve the same panel',()=>{ - for(const count of [2,3,7]) { - const r=replay(); const panel=r.viewport.slice(r.viewport.indexOf('\n────────────────')+1); - const title=r.viewport.split('\n').find(s=>s.startsWith('● Update('))!; - r.viewport=Array(count).fill(title+'\n').join('\n')+'\n'+panel; - expect(pick(r)?.input).toBe('1\r'); - } -}); -test('foreign, mixed, malformed, quoted and competing prefix panels reject',()=>{ - for(const change of [ - (s:string)=>s.replace(/^● Update\([^\n]+\)/m,'● Update(/tmp/foreign.md)'), - (s:string)=>s.replace(/gstack-autoplan-chain-RWuak5/,'sibling-project'), - (s:string)=>s.replace(/^● Update/m,'● Read'), - (s:string)=>s.replace(/^● Update\(([^\n]+)\)/m,'● Update($1) extra command'), - (s:string)=>s.replace(/^● Update/m,'> ● Update'), - (s:string)=>'Example: current edit\n'+s, - (s:string)=>'```text\n'+s+'\n```', - (s:string)=>s.replace(/^● Update/m,'☐ Current task\n● Update'), - (s:string)=>s.replace(/^● Update/m,'Prior file completed\n● Update'), - (s:string)=>s.replace(' Edit file\n',' Read file\n'), - (s:string)=>s.replace(/^ …[^\n]+$/m,' /tmp/foreign.md'), - (s:string)=>s+'\n'+s, - ]) {const r=replay(); r.viewport=change(r.viewport); expect(pick(r)).toBeNull();} -}); -test('owned native epoch, content, successful predecessor and one-time keys remain mandatory',()=>{ - const r=replay(), result=pick(r)!; - expect(pick(r,new Set([result.signature]))).toBeNull(); - expect(pick(r,new Set([autoplanArtifactMenuKey(r.viewport)]))).toBeNull(); - for(const change of [ - (r:ReturnType)=>{r.context.pending.sessionId='foreign';}, - (r:ReturnType)=>{r.context.pending.file=r.file+'.foreign';}, - (r:ReturnType)=>{r.context.viewportCapturedAt=Date.parse(r.context.pending.timestamp)-1;}, - (r:ReturnType)=>{r.context.publicTools[1]!.isError=true;}, - (r:ReturnType)=>{r.context.publicTools.push({kind:'result',sessionId:r.context.pending.sessionId,toolUseId:r.context.pending.toolUseId,timestamp:new Date(r.context.now).toISOString(),isError:false});}, - (r:ReturnType)=>{r.context.publicTools.push({kind:'use',name:'Write',sessionId:r.context.pending.sessionId,toolUseId:'newer',timestamp:new Date(r.context.now).toISOString(),input:{file_path:r.file}});}, - (r:ReturnType)=>{fs.writeFileSync(r.file,'Foreign contents');}, - (r:ReturnType)=>{r.viewport=r.viewport.replace('❯ 1. Yes','❯ 2. Yes');}, - (r:ReturnType)=>{r.viewport=r.viewport.replace('3. No','3. No; run command');}, - (r:ReturnType)=>{r.viewport=r.viewport.replace('Esc to cancel · Tab to amend','');}, - ]) {const r=replay(); change(r); expect(pick(r)).toBeNull();} -}); -test('published Edit still needs exact old and new bytes with repeated titles',()=>{ - const r=replay(),events=structuredClone(published.events) as NativePublicToolEvent[]; - const edit=events.find(e=>e.kind==='use'&&e.toolUseId===published.pending.toolUseId)!; - const originalFile=edit.input!.file_path as string, file=path.normalize(originalFile.replace(published.ownedStateRoot,r.context.ownedStateRoot)); - const cwd=path.join(r.root,path.basename(published.cwd));fs.mkdirSync(cwd,{recursive:true});fs.mkdirSync(path.dirname(file),{recursive:true});fs.writeFileSync(file,published.before); - for(const e of events)if(e.input?.file_path===originalFile)e.input.file_path=file; - const header=published.viewport.lastIndexOf('\n● Update(')+1; - const panel=published.viewport.slice(header).split('\n').slice(2).join('\n').replace(/^ …[^\n]+$/m,' …'+path.relative(r.context.ownedStateRoot,file)); - const title='● Update('+file+')\n\n',viewport=title+title+panel; - const context={cwd,ownedStateRoot:r.context.ownedStateRoot,commandStartedAt:Date.parse(events[0]!.timestamp)-1,now:Date.parse(published.viewportCapturedAt),transcriptStatus:'ready',publicTools:events}; - expect(autoplanArtifactPermissionInput(viewport,context,new Set())?.input).toBe('1\r'); - const before=edit.input!.new_string;edit.input!.new_string='Different replacement';expect(autoplanArtifactPermissionInput(viewport,context,new Set())).toBeNull(); - edit.input!.new_string=before;edit.input!.old_string='Different original';expect(autoplanArtifactPermissionInput(viewport,context,new Set())).toBeNull(); -}); -test('only existing Autoplan owner receives repeated-title regression inputs',()=>{ - for(const file of ['test/autoplan-repeated-header-ak.test.ts','test/fixtures/autoplan-repeated-header-ak.json']) - expect(selectTests([file],E2E_TOUCHFILES,[]).selected).toEqual(['autoplan-chain-pty']); -}); diff --git a/test/ceo-paired-payment-fixture.test.ts b/test/ceo-paired-payment-fixture.test.ts deleted file mode 100644 index f74aa9801..000000000 --- a/test/ceo-paired-payment-fixture.test.ts +++ /dev/null @@ -1,118 +0,0 @@ -import { expect, test } from 'bun:test'; -import * as fs from 'node:fs'; -import * as os from 'node:os'; -import * as path from 'node:path'; -import { seedCeoPairedProject } from './helpers/ceo-paired-fixture'; -import { processPayment, PaymentFailure, ProviderError, type Payment } from './fixtures/paired-payment/src/payment'; - -test('paired review gets runnable existing coverage that leaves both intended gaps open', () => { - const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'paired-payment-fixture-')); - try { - seedCeoPairedProject(dir, '# Add two payment tests\n'); - const file = path.join(dir, 'src/payment.ts'); const original = fs.readFileSync(file, 'utf8'); - // The review's two missing tests are injected only for this free proof, - // never copied into the agent's seeded baseline. - const checks = { - receipt: `test('target receipt: first success returns the correct value without a retry', async () => { - let calls = 0; const waits: number[] = []; - expect(await processPayment(payment, { - chargeOnce: async () => { calls++; return { id: 'charge-42' }; }, - sleep: async ms => { waits.push(ms); }, - })).toEqual({ chargeId: 'charge-42', amount: 1200, currency: 'usd' }); - expect(calls).toBe(1); expect(waits).toEqual([]); - });`, - failure: `test.each(['502', 'timeout'] as const)('target failure: repeated %s stops after one wait and retry', async code => { - const causes = [new ProviderError(code), new ProviderError(code)]; - const requests: Readonly[] = []; const waits: number[] = []; - const result = processPayment(payment, { - chargeOnce: async request => { requests.push(request); throw causes[Math.min(requests.length - 1, 1)]; }, - sleep: async ms => { waits.push(ms); }, - }); - const failure = await result.then(() => { throw new Error('expected rejection'); }, error => error); - expect(failure).toBeInstanceOf(PaymentFailure); - expect(failure).toMatchObject({ key: payment.key, outcomeUnknown: true }); - expect(failure.cause).toBe(causes[1]); - expect(requests).toEqual([payment, payment]); expect(requests[0]).toBe(requests[1]); - expect(waits).toEqual([100]); - });`, - }; - type Target = keyof typeof checks; - const targetFile = path.join(dir, 'target-check.test.ts'); - expect(fs.existsSync(targetFile)).toBe(false); - const run = (source: string, targets: Target[]) => { - fs.writeFileSync(file, source); - fs.writeFileSync(targetFile, `import { expect, test } from 'bun:test'; - import { PaymentFailure, ProviderError, processPayment, type Payment } from './src/payment'; - const payment: Payment = { key: 'order-42', amount: 1200, currency: 'usd' }; - ${targets.map(target => checks[target]).join('\n')}`); - return Bun.spawnSync([process.execPath, 'test', './contract.test.ts', './target-check.test.ts'], { - cwd: dir, timeout: 5000, env: { PATH: process.env.PATH ?? '' }, - }); - }; - const variants = [ - { target: null, source: original }, - { target: 'receipt', source: original.replace('amount: request.amount, currency:', 'amount: request.amount + (attempt === 0 ? 1 : 0), currency:') }, - { target: 'failure', source: original.replace('attempt === 1', 'attempt === 2') }, - ] as const; - expect(new Set(variants.map(variant => variant.source)).size).toBe(3); - // Baseline detects neither target mutant. Each missing contract catches its - // own mutant and leaves the other live; adding both catches both. - for (const targets of [[], ['receipt'], ['failure'], ['receipt', 'failure']] as Target[][]) { - for (const variant of variants) { - const result = run(variant.source, targets); - const output = result.stderr.toString(); - const killed = variant.target !== null && targets.includes(variant.target); - expect(result.exitCode, `${targets.join('+') || 'baseline'} / ${variant.target || 'original'}\n${output}`).toBe(killed ? 1 : 0); - if (killed) { - expect(output).toContain(`(fail) target ${variant.target}:`); - } else { - expect(output).toContain(`${20 + (targets.includes('receipt') ? 1 : 0) + (targets.includes('failure') ? 2 : 0)} pass`); - expect(output).toContain('0 fail'); - } - } - } - // The fixture must enforce its advertised pre-existing contracts without - // closing either of the review's missing first-success/exhaustion tests. - for (const [source, failedTest] of [ - [original.replace('amount: request.amount, currency:', 'amount: request.amount + 1, currency:'), 'recovery after one 502'], - [original.replace('await io.sleep(100);', 'void io.sleep(100);'), 'recovery after one 502'], - [original.replace('outcomeUnknown, error);', 'outcomeUnknown, new ProviderError((error as ProviderError).code));'), 'declined is never retried'], - [original.replace('outcomeUnknown ||= retryable;', 'outcomeUnknown = retryable;'), 'uncertain 502 followed by declined stays unknown'], - [original.replace("error.code === '502' || error.code === 'timeout'", "error.code === '502'"), 'recovery after one timeout'], - [original.replace('throw new PaymentFailure(request.key, outcomeUnknown, cause);', 'throw cause;'), 'rejected backoff after 502'], - ]) { - expect(source).not.toBe(original); - const result = run(source!, []); - expect(result.exitCode, result.stderr.toString()).toBe(1); - expect(result.stderr.toString()).toContain('(fail) ' + failedTest); - } - } finally { fs.rmSync(dir, { recursive: true, force: true }); } -}); - -test('existing payment behavior supports the missing happy and exhausted-retry tests', async () => { - const payment: Payment = { key: 'order-42', amount: 1200, currency: 'usd' }; - let calls = 0; const delays: number[] = []; - const receipt = await processPayment(payment, { - chargeOnce: async () => { calls++; return { id: 'charge-42' }; }, - sleep: async ms => { delays.push(ms); }, - }); - expect(receipt).toEqual({ chargeId: 'charge-42', amount: 1200, currency: 'usd' }); - expect(calls).toBe(1); expect(delays).toEqual([]); - for (const code of ['502', 'timeout'] as const) { - const requests: Readonly[] = []; const waits: number[] = []; const cause = new ProviderError(code); - const outcome = processPayment(payment, { - chargeOnce: async request => { requests.push(request); throw cause; }, - sleep: async ms => { waits.push(ms); }, - }); - await expect(outcome).rejects.toBeInstanceOf(PaymentFailure); - await expect(outcome).rejects.toMatchObject({ key: payment.key, outcomeUnknown: true, cause }); - expect(requests).toEqual([payment, payment]); expect(requests[0]).toBe(requests[1]); - expect(waits).toEqual([100]); - } - const causes = [new ProviderError('timeout'), new ProviderError('auth')]; - let mixedCalls = 0; - await expect(processPayment(payment, { - chargeOnce: async () => { throw causes[mixedCalls++]; }, sleep: async () => {}, - })).rejects.toMatchObject({ key: payment.key, outcomeUnknown: true, cause: causes[1] }); - expect(mixedCalls).toBe(2); -}); diff --git a/test/design-ui-scope.test.ts b/test/design-ui-scope.test.ts deleted file mode 100644 index 37c04a824..000000000 --- a/test/design-ui-scope.test.ts +++ /dev/null @@ -1,124 +0,0 @@ -import { expect, test } from 'bun:test'; -import { nativePlanCallFingerprint } from './helpers/claude-pty-runner'; -import { isDesignUIScopeReview } from './helpers/design-ui-scope'; -import type { NativePlanQuestionCall } from './helpers/plan-count-transcript'; -import captured from './fixtures/plan-design-ui-scope.json'; -import { E2E_TOUCHFILES } from './helpers/touchfiles-data'; - -const calls = captured.calls as NativePlanQuestionCall[]; -const fingerprint = (call: NativePlanQuestionCall) => nativePlanCallFingerprint(call, 0, true); -const recovered = captured.additionalQuestionCaptures[0]!; -const recoveredCall: NativePlanQuestionCall = { - sessionId: 'ui-scope-replay', - toolUseId: 'recovered-question', - questions: [recovered.question], - answered: true, - failed: false, - answers: { [recovered.question.question]: recovered.answer }, - unansweredQuestionIndices: [], -}; - -test('the captured untagged dashboard decision proves UI review, but its setup questions do not', () => { - expect(calls.map(call => isDesignUIScopeReview(fingerprint(call)))).toEqual([false, false, false, true]); - expect(calls[3]!.questions[0]!.question).not.toContain(' { - const replay = captured.additionalCaptures[0]!.calls as NativePlanQuestionCall[]; - expect(replay.map(call => isDesignUIScopeReview(fingerprint(call)))) - .toEqual([false, false, ...Array(10).fill(true)]); -}); - -test('issue and pass separators do not change native design evidence', () => { - for (const issueSeparator of [':', ' —', ' –', ' -']) { - for (const passSeparator of [',', ';', ' —', ' –', ' -', ':', ' (']) { - const call = structuredClone(calls[3]!); - const q = call.questions[0]!; - q.question = q.question.replace('Issue 1:', `Issue 1${issueSeparator}`) - .replace(', Pass 1', `${passSeparator} Pass 1`); - call.answers = { [q.question]: q.options[0]!.label }; - expect(isDesignUIScopeReview(fingerprint(call)), `${issueSeparator} / ${passSeparator}`).toBe(true); - } - } -}); - -test('a recovered UI decision replays with fixture-owned metadata without filename, pass, or leading question verb', () => { - expect(isDesignUIScopeReview(fingerprint(recoveredCall))).toBe(true); - const call = structuredClone(recoveredCall); - const q = call.questions[0]!; - q.question = q.question.replace(/^Project\/branch\/task:[^\n]*\n/m, ''); - call.answers = { [q.question]: q.options[0]!.label }; - expect(isDesignUIScopeReview(fingerprint(call))).toBe(true); -}); - -test('choice identity does not depend on punctuation after the issue letter', () => { - for (const separator of ['', ':', '.', ')', '—', '–', '-']) { - const call = structuredClone(recoveredCall); - const q = call.questions[0]!; - for (const option of q.options) option.label = option.label.replace(/^6([A-Z]) /, `6$1${separator} `); - call.answers = { [q.question]: q.options[0]!.label }; - expect(isDesignUIScopeReview(fingerprint(call)), separator).toBe(true); - } -}); - -test('numbered UI language still requires concrete design choices rather than workflow or another target', () => { - for (const mutate of [ - (q: NativePlanQuestionCall['questions'][number]) => { q.header = 'Scope'; }, - (q: NativePlanQuestionCall['questions'][number]) => { q.question = q.question.replace('dashboard plan on main', 'OTHER.md dashboard plan on main'); }, - (q: NativePlanQuestionCall['questions'][number]) => { q.question = q.question.replace('D10 — Issue 6:', 'Example:'); }, - (q: NativePlanQuestionCall['questions'][number]) => { q.question = q.question.replace('Undo toast?', 'Undo toast.'); }, - (q: NativePlanQuestionCall['questions'][number]) => { q.options[0]!.label = '7A Immediate + Undo toast'; }, - (q: NativePlanQuestionCall['questions'][number]) => { q.options = [{ label: '6A Yes' }, { label: '6B No' }]; }, - (q: NativePlanQuestionCall['questions'][number]) => { q.options[0]!.label = '6A Review the modal later'; }, - (q: NativePlanQuestionCall['questions'][number]) => { q.question = q.question.replace("'Mark all as read' — confirmation modal (as planned) or immediate action with an Undo toast?", 'Which modal should the outside reviewers discuss?'); }, - ]) { - const call = structuredClone(recoveredCall); - const q = call.questions[0]!; - mutate(q); - call.answers = { [q.question]: q.options[0]!.label }; - expect(isDesignUIScopeReview(fingerprint(call))).toBe(false); - } -}); - -test('native ownership and complete offered answers are required for UI evidence', () => { - for (const mutate of [ - (call: NativePlanQuestionCall) => { call.answered = false; }, - (call: NativePlanQuestionCall) => { call.failed = true; }, - (call: NativePlanQuestionCall) => { call.unansweredQuestionIndices = [0]; }, - (call: NativePlanQuestionCall) => { call.answers = {}; }, - (call: NativePlanQuestionCall) => { call.answers = { [call.questions[0]!.question]: 'Unrelated answer' }; }, - (call: NativePlanQuestionCall) => { call.questions[0]!.multiSelect = true; }, - (call: NativePlanQuestionCall) => { call.questions[0]!.options = call.questions[0]!.options.slice(0, 1); }, - ]) { - const call = structuredClone(calls[3]!); - mutate(call); - expect(isDesignUIScopeReview(fingerprint(call))).toBe(false); - } - expect(isDesignUIScopeReview({ ...fingerprint(calls[3]!), signature: 'another-session:another-call' })).toBe(false); -}); - -test('issue-like framing cannot promote setup, examples, another plan, or mismatched choices', () => { - for (const mutate of [ - (q: NativePlanQuestionCall['questions'][number]) => { q.header = 'Outside voices'; }, - (q: NativePlanQuestionCall['questions'][number]) => { q.question = 'Example:\n' + q.question; }, - (q: NativePlanQuestionCall['questions'][number]) => { q.question = q.question.replace('PLAN.md', 'OTHER.md'); }, - (q: NativePlanQuestionCall['questions'][number]) => { q.question = q.question.replace('Pass 1', 'before Pass 1'); }, - (q: NativePlanQuestionCall['questions'][number]) => { q.question = q.question.replace("Which panel is primary, and what's the order?", 'Which review scope should cover the panels?'); }, - (q: NativePlanQuestionCall['questions'][number]) => { q.options[1]!.label = '2B: Another issue'; }, - (q: NativePlanQuestionCall['questions'][number]) => { q.options[1]!.label = '1B: Run outside reviewers'; }, - (q: NativePlanQuestionCall['questions'][number]) => { q.options[1]!.label = q.options[0]!.label; }, - ]) { - const call = structuredClone(calls[3]!); - const q = call.questions[0]!; - mutate(q); - call.answers = { [q.question]: q.options[0]!.label }; - expect(isDesignUIScopeReview(fingerprint(call))).toBe(false); - } -}); - -test('the UI gate owns its classifier, captured evidence, and regression tests', () => { - for (const file of ['test/helpers/design-ui-scope.ts', 'test/design-ui-scope.test.ts', 'test/fixtures/plan-design-ui-scope.json']) { - expect(Object.entries(E2E_TOUCHFILES).filter(([, files]) => files.includes(file)).map(([owner]) => owner)) - .toEqual(['plan-design-with-ui-scope']); - } -}); diff --git a/test/eng-before-rewrite-ar.test.ts b/test/eng-before-rewrite-ar.test.ts deleted file mode 100644 index efb276c87..000000000 --- a/test/eng-before-rewrite-ar.test.ts +++ /dev/null @@ -1,92 +0,0 @@ -import { describe, expect, test } from 'bun:test'; -import { readFileSync } from 'node:fs'; -import { createHash } from 'node:crypto'; -import { evaluateEngSeedCoverage } from './helpers/eng-seeded-coverage'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; - -// Exact acknowledged public report; the paid attempt remains failed. -const report = readFileSync(new URL('./fixtures/eng-before-rewrite-ar.md', import.meta.url), 'utf8'); -const declaration = report.match(/^### REGRESSION RULE \(mandatory, no decision required\)\n[\s\S]*?(?=\n### )/m)![0]; -const task = report.match(/^- \[ \] \*\*T1 .*\n(?: .*(?:\n|$))*/m)![0]; -const compact = '# Current reviewed plan\n\n## Tests (reviewed)\n\n' + declaration + - '\n## Implementation Tasks\n' + task + '\n## GSTACK REVIEW REPORT\n| Eng Review | complete |\n'; -const check = (text: string) => evaluateEngSeedCoverage({ status: 'ready', calls: [], assistantMessages: [] }, text, 0, 1); - -const negative: Array<[string, (text: string) => string]> = [ - ['missing mandatory declaration', s => s.replace(declaration, '')], - ['optional declaration', s => s.replace('mandatory, no decision required', 'optional, decision pending')], - ['historical owner', s => s.replace('## Tests (reviewed)', '## Historical tests')], - ['quoted source ancestor', s => '# Source excerpt\n' + s.replace('# Current reviewed plan\n', '')], - ['source declaration prefix', s => s.replace('`legacyAuthFlow()` is', 'Source excerpt:\n`legacyAuthFlow()` is')], - ['conditional declaration', s => s.replace('`legacyAuthFlow()` is', 'If approved, `legacyAuthFlow()` is')], - ['quoted declaration', s => s.replace(declaration, declaration.split('\n').map(line => '> ' + line).join('\n'))], - ['fenced declaration', s => s.replace(declaration, '```\n' + declaration + '\n```')], - ['literal declaration', s => s.replace(declaration, declaration.replace(/`/g, '').split('\n').map(line => '`' + line + '`').join('\n'))], - ['wrong legacy target', s => s.replaceAll('legacyAuthFlow', 'anotherFlow')], - ['capture after rewrite', s => s.replace('before any rewrite, record', 'after the rewrite, record')], - ['proposed outputs', s => s.replace('the exact output', 'the proposed output')], - ['no required flag-on rerun', s => s.replace('The new path must pass', 'The new path might pass')], - ['different rerun suite', s => s.replace('pass the same suite', 'pass a different suite')], - ['missing baseline task', s => s.replace(task, '')], - ['historical task section', s => s.replace('## Implementation Tasks', '## Historical Implementation Tasks')], - ['conditional task', s => s.replace(task, 'If approved:\n' + task)], - ['source task', s => s.replace(task, 'Source excerpt:\n' + task)], - ['quoted task', s => s.replace(task, task.split('\n').map(line => '> ' + line).join('\n'))], - ['another test file', s => s.replace(' - Files: tests/auth/legacyAuthFlow.regression.test.ts', ' - Files: tests/auth/anotherFlow.regression.test.ts')], - ['missing baseline verification', s => s.replace(' - Verify: suite green on current code; green again with flag on after rewrite', '')], - ['modified baseline', s => s.replace('suite green on current code;', 'suite green on rewritten code;')], - ['missing flag-on verification', s => s.replace('; green again with flag on after rewrite', '')], - ['source verification', s => s.replace(' - Verify:', ' Source:\n - Verify:')], - ['conditional verification', s => s.replace(' - Verify:', ' If approved:\n - Verify:')], - ['assuming verification', s => s.replace(' - Verify:', ' Assuming approval,\n - Verify:')], - ['verification from neighboring task', s => s.replace(' - Verify:', '- [ ] T2 — tests/auth — Another test suite\n - Verify:')], - ['duplicate task identities', s => s.replace(task, task + task)], - ['cancelled task', s => s + '\n## Current assessment\nT1 is withdrawn.\n'], - ['quoted task status', s => s + '\n## Current assessment\nT1 verification is "withdrawn".\n'], - ['cancelled legacy suite', s => s + '\n## Current assessment\nThe legacy regression suite is "withdrawn".\n'], - ['cancelled baseline verification', s => s.replace(task, task + ' Correction: this baseline verification is withdrawn.\n')], - ['quoted baseline status', s => s.replace(task, task + ' Correction: this baseline verification is "withdrawn".\n')], - ['superseded baseline verification', s => s.replace(task, task + ' Correction: this baseline verification is "superseded".\n')], - ['baseline verification no longer current', s => s.replace(task, task + ' Correction: this baseline verification is not current.\n')], - ['verification waits for approval', s => s.replace(' - Verify:', ' Once approved:\n - Verify:')], - ['verification depends on approval', s => s.replace(' - Verify:', ' When approved:\n - Verify:')], - ['verification has approval pending', s => s.replace(' - Verify:', ' Pending approval:\n - Verify:')], - ['bare source owns the following sections', s => 'Source:\n\n' + s.replace('# Current reviewed plan\n', '')], - ['current task withdrawal row', s => s + '\n## Current assessment\n| T1 | Withdrawn |\n'], - ['quoted current task withdrawal value', s => s + '\n## Current assessment\n| T1 | "Withdrawn" |\n'], - ['baseline changes before task', s => s + '\n## Current assessment\nlegacyAuthFlow() is modified before T1.\n'], -]; - -describe('mandatory characterization binds the current baseline and same-file rerun', () => { - test('the exact acknowledged report supplies regression coverage, without inventing native decisions', () => { - expect(createHash('sha256').update(report).digest('hex')).toBe('7e4c56d66f98b9ccea54b7427ec2dbbca89adfc6407f0fedf2017801eafada99'); - expect(check(report)).toMatchObject({regression: 'plan', ok: false, - missing: ['complexity', 'shared-cache', 'swallowed-errors', 'sequential-idp']}); - expect(check(compact).regression).toBe('plan'); - }); - - test('formatting and current task numbering do not affect the obligation', () => { - expect(check(compact.replaceAll('T1', 'T21')).regression).toBe('plan'); - expect(check(compact.replace(/\n(?=[a-z])/g, ' ')).regression).toBe('plan'); - expect(check(compact.replaceAll('tests/auth', 'test/login')).regression).toBe('plan'); - }); - - test('quoted history and a separate suite cannot cancel the current legacy obligation', () => { - expect(check(compact + '\n## History\n"T1 is withdrawn."\n').regression).toBe('plan'); - expect(check(compact + '\n## Payment regression suite\nThe regression suite is withdrawn.\n').regression).toBe('plan'); - expect(check(compact + '\n## Historical task status\n| T1 | Withdrawn |\n').regression).toBe('plan'); - expect(check(compact + '\n## Current task status\n| T9 | Withdrawn |\n').regression).toBe('plan'); - expect(check('Source:\n\n' + compact).regression).toBe('plan'); - }); - - test.each(negative)('%s supplies no mandatory legacy baseline', (_, change) => { - const altered = change(compact); - expect(altered).not.toBe(compact); - expect(check(altered).regression).toBeUndefined(); - }); - - test('new artifacts select only the existing Eng owner', () => { - for (const file of ['test/eng-before-rewrite-ar.test.ts', 'test/fixtures/eng-before-rewrite-ar.md']) - expect(selectTests([file], E2E_TOUCHFILES, []).selected).toEqual(['plan-eng-finding-count']); - }); -}); diff --git a/test/eng-blocking-baseline-at.test.ts b/test/eng-blocking-baseline-at.test.ts deleted file mode 100644 index bbdacbe17..000000000 --- a/test/eng-blocking-baseline-at.test.ts +++ /dev/null @@ -1,90 +0,0 @@ -import { expect, test } from 'bun:test'; -import { readFileSync } from 'node:fs'; -import { createHash } from 'node:crypto'; -import { evaluateEngSeedCoverage } from './helpers/eng-seeded-coverage'; -import { E2E_TOUCHFILES } from './helpers/touchfiles-data'; -const report = readFileSync(new URL('./fixtures/eng-blocking-baseline-at.md', import.meta.url), 'utf8'); -const declaration = report.match(/^### REGRESSION[^\n]+\n[\s\S]*?(?=\n### )/m)![0]; -const ordered = report.match(/^## Implementation steps \(ordered\)\n[\s\S]*?(?=\n## )/m)![0]; -const task = report.match(/^- \[ \] \*\*T1 .*\n(?: .*(?:\n|$))*/m)![0]; -const compact = '# Current reviewed plan\n\n## Tests\n\n' + declaration + '\n' + ordered + '\n## Implementation Tasks\n' + task; -const check = (text: string) => evaluateEngSeedCoverage({ status: 'ready', calls: [], assistantMessages: [] }, text, 0, 1); - -test('the exact acknowledged report supplies a mandatory current-code baseline', () => { - expect(createHash('sha256').update(report).digest('hex')).toBe('0b1c69727fc89ae972bc023cc5677b90708507356084c65dc18b979906521d98'); - expect(check(report)).toMatchObject({ regression: 'plan', ok: false, - missing: ['complexity', 'shared-cache', 'swallowed-errors', 'sequential-idp'] }); - expect(check(compact).regression).toBe('plan'); -}); - -test('task IDs, paths and markup can change while the same baseline stays required', () => { - for (const value of [compact.replaceAll('T1', 'T31'), compact.replaceAll('tests/auth/legacyAuthFlow.regression', 'test/login/prior-behavior.test.ts'), - compact.replace(/[`*]/g, ''), compact.replace('capture the current behavior', 'record the current behavior')]) { - expect(check(value).regression).toBe('plan'); - } -}); - -const negatives: Array<[string, (text: string) => string]> = [ - ['missing declaration', text => text.replace(declaration, '')], - ['optional heading', text => text.replace('CRITICAL, mandatory under', 'CRITICAL, optional under')], - ['historical owner', text => text.replace('## Tests', '## Historical Tests')], - ['source ancestor', text => '# Source excerpt\n' + text.replace('# Current reviewed plan\n', '')], - ['bare source owner', text => 'Source:\n\n' + text.replace('# Current reviewed plan\n', '')], - ['quoted declaration', text => text.replace(declaration, declaration.split('\n').map(line => '> ' + line).join('\n'))], - ['fenced declaration', text => text.replace(declaration, '```\n' + declaration + '\n```')], - ['inline literal declaration', text => text.replace(declaration, declaration.replace(/`/g, '').split('\n').map(line => '`' + line + '`').join('\n'))], - ['conditional requirement', text => text.replace('**T1 is a blocking requirement:**', 'If approved, **T1 is a blocking requirement:**')], - ['proposed baseline', text => text.replace('capture the current behavior', 'capture the proposed behavior')], - ['capture after change', text => text.replace('before any rewrite, capture', 'after the rewrite, capture')], - ['wrong legacy target', text => text.replaceAll('legacyAuthFlow', 'otherAuthFlow')], - ['missing accepted tokens', text => text.replace('every accepted token shape, ', '')], - ['missing rejected tokens', text => text.replace('every rejected token shape, ', '')], - ['missing errors', text => text.replace('every error response, ', '')], - ['parity targets unrelated module', text => text.replace('against the `AuthBroker` path', 'against the `OtherBroker` path')], - ['parity permits differences', text => text.replace('must produce identical outcomes', 'may produce different outcomes')], - ['missing ordered baseline', text => text.replace(ordered, '')], - ['historical ordering', text => text.replace('## Implementation steps (ordered)', '## Historical implementation steps (ordered)')], - ['wrong ordered task', text => text.replace('1. **T1** Characterization', '1. **T99** Characterization')], - ['changed first', text => text.replace('Green on current code before anything else changes.', 'Green on changed code after everything else changes.')], - ['parallel baseline', text => text.replace('Green on current code before anything else changes.', 'Run in parallel with the rewrite.')], - ['new-path-only baseline', text => text.replace('Green on current code before anything else changes.', 'Green on AuthBroker after rewriting legacy code.')], - ['missing task', text => text.replace(task, '')], - ['historical task owner', text => text.replace('## Implementation Tasks', '## Historical Implementation Tasks')], - ['wrong owned task', text => text.replace(task, task.replace('**T1 ', '**T99 '))], - ['duplicate task', text => text.replace(task, task + task)], - ['missing verification', text => text.replace(' - Verify: suite green on current code; later green on both flag states', '')], - ['post-rewrite verification only', text => text.replace('suite green on current code; later green on both flag states', 'suite green only after the rewrite')], - ['neighboring verification', text => text.replace(' - Verify:', '- [ ] T99 — tests — Another suite\n - Verify:')], - ['missing task files', text => text.replace(/^ - Files: .+$/m, '')], - ...['Source:', 'If approved:', 'Assuming approval,', 'Provided approval,', 'Once approved:', 'When approved:', 'Pending approval:'].flatMap(prefix => [ - [`conditional task ${prefix}`, (text: string) => text.replace(task, prefix + '\n' + task)], - [`conditional verification ${prefix}`, (text: string) => text.replace(' - Verify:', ' ' + prefix + '\n - Verify:')], - ] as Array<[string, (text: string) => string]>), - ...['withdrawn', 'declined', 'optional', 'superseded', 'not current', 'no longer current'].flatMap(status => [ - [`current T1 ${status}`, (text: string) => text + `\n## Current assessment\nT1 baseline requirement is ${status}.\n`], - [`quoted T1 ${status}`, (text: string) => text + `\n## Current assessment\nT1 baseline requirement is "${status}".\n`], - ] as Array<[string, (text: string) => string]>), - ['status table', text => text + '\n## Current assessment\n| T1 | Withdrawn |\n'], - ['baseline changed first correction', text => text + '\n## Current assessment\nlegacyAuthFlow() is rewritten before T1.\n'], -]; - -test.each(negatives)('%s supplies no required legacy baseline', (_, change) => { - const altered = change(compact); - expect(altered).not.toBe(compact); - expect(check(altered).regression).toBeUndefined(); -}); - -test('quoted history and another suite cannot withdraw this required baseline', () => { - for (const tail of ['\n## History\n"T1 baseline requirement is withdrawn."', '\n## History\n> T1 baseline requirement is withdrawn.', - '\n## Historical task status\n| T1 | Withdrawn |', '\n## Current assessment\n| T9 | Withdrawn |', - '\n## Payment regression suite\nThe regression suite is withdrawn.', '\n## Current assessment\nIf T1 is withdrawn, reopen the decision.']) { - expect(check(compact + tail).regression).toBe('plan'); - } -}); - -test('the regression and exact report select only the Eng finding-count workflow', () => { - for (const file of ['test/eng-blocking-baseline-at.test.ts', 'test/fixtures/eng-blocking-baseline-at.md']) { - expect(Object.entries(E2E_TOUCHFILES).filter(([, paths]) => paths.includes(file)).map(([name]) => name)) - .toEqual(['plan-eng-finding-count']); - } -}); diff --git a/test/eng-count-ad-v2.test.ts b/test/eng-count-ad-v2.test.ts index 887d6dffe..45fade61d 100644 --- a/test/eng-count-ad-v2.test.ts +++ b/test/eng-count-ad-v2.test.ts @@ -2,17 +2,12 @@ import { describe, expect, test } from 'bun:test'; import captured from './fixtures/eng-count-ad-v2.json'; import { engFirstReviewAUQ, engSetupAUQ, engStep0Boundary, nativePlanCallFingerprint, planCountQuestionPhase } from './helpers/claude-pty-runner'; import type { NativePlanQuestionCall } from './helpers/plan-count-transcript'; -import { isEngCompletionHandoff } from './helpers/eng-completion-handoff'; import { E2E_TOUCHFILES, matchGlob } from './helpers/touchfiles'; - const firstCalls = captured.cases.first.calls as NativePlanQuestionCall[]; const retryCalls = captured.cases.retry.calls as NativePlanQuestionCall[]; -const catalog = captured.reviewedTasks.lines.join('\n'); const issue = () => structuredClone(retryCalls[3]!); -const handoff = () => structuredClone(firstCalls.at(-1)!); const fp = (call: NativePlanQuestionCall) => nativePlanCallFingerprint(call, 0, true); const isFirst = (call: NativePlanQuestionCall) => engFirstReviewAUQ(fp(call)); -const isHandoff = (call: NativePlanQuestionCall, plan = catalog) => isEngCompletionHandoff(fp(call), plan); function setupPacket(): NativePlanQuestionCall { const c = issue(); c.questions = [ @@ -32,8 +27,7 @@ function census(calls: NativePlanQuestionCall[]) { let reviewStarted = false; const counts = { setup: 0, review: 0, administrative: 0 }; const phases = calls.map(call => { - const phase = planCountQuestionPhase(fp(call), reviewStarted, engStep0Boundary, engFirstReviewAUQ, engSetupAUQ, - current => isEngCompletionHandoff(current, catalog)); + const phase = planCountQuestionPhase(fp(call), reviewStarted, engStep0Boundary, engFirstReviewAUQ, engSetupAUQ); reviewStarted = phase.reviewStarted; counts[phase.administrative ? 'administrative' : phase.preReview ? 'setup' : 'review']++; return phase; @@ -78,17 +72,6 @@ describe('Eng AD v2 completed native count evidence', () => { expect(engStep0Boundary(fp(c))).toBe(false); }); - test('first attempt retains seven substantive decisions and separates the completed D9 handoff', () => { - const { counts, phases } = census(firstCalls); - expect(counts).toEqual({ setup: 4, review: 7, administrative: 1 }); - expect(phases.slice(4, 11).every(p => !p.preReview && !p.administrative)).toBe(true); - expect(firstCalls[9]!.questions[0]!.question).toContain('TODO 1'); - expect(firstCalls[10]!.questions[0]!.question).toContain('TODO 2'); - expect(phases[11]!.administrative).toBe('completion-handoff'); - expect(captured.cases.first.actual.outcome).toBe('ceiling_reached'); - expect(captured.cases.first.actual.reviewCount).toBe(8); - }); - test('retry ordinary Issue identity starts review without qids, retaining its later TODO', () => { const { counts, phases } = census(retryCalls); expect(counts).toEqual({ setup: 3, review: 6, administrative: 0 }); @@ -98,16 +81,6 @@ describe('Eng AD v2 completed native count evidence', () => { expect(captured.cases.retry.actual.reviewCount).toBe(0); }); - test('the prior successful plan Write already contains the exact referenced task and regression step', () => { - expect(captured.reviewedTasks.isError).toBe(false); - expect(Date.parse(captured.reviewedTasks.replyAt)).toBeLessThan(Date.parse(handoff().answeredAt!)); - expect(captured.reviewedTasks.lines).toHaveLength(10); - expect(captured.reviewedTasks.lines[2]).toContain('Record regression characterization fixtures before any change'); - expect(isHandoff(handoff())).toBe(true); - // The menu's Tasks JSONL claim is not independently verified by this fixture. - expect(captured.provenance.privateThinkingInspected).toBe(false); - }); - test('ordinary issue presentation can vary while completed identity and section number remain bound', () => { for (const title of ['Issue 1', 'Finding 1.2 (D17)', 'D42 — Issue 1']) { const call = changeQuestion(issue(), s => s.replace('Issue 1 (D4)', title).replace('AuthCache', 'SessionCache')); @@ -157,9 +130,9 @@ describe('Eng AD v2 completed native count evidence', () => { expect(isFirst(c)).toBe(false); }); - test('new first-finding and handoff paths require exact completed native identity and answer', () => { - for (const factory of [issue, handoff]) { - const classify = factory === issue ? isFirst : isHandoff; + test('new first-finding path requires exact completed native identity and answer', () => { + for (const factory of [issue]) { + const classify = isFirst; for (const mutate of [ (c: NativePlanQuestionCall) => { c.answered = false; }, (c: NativePlanQuestionCall) => { c.failed = true; }, @@ -177,7 +150,7 @@ describe('Eng AD v2 completed native count evidence', () => { (c: NativePlanQuestionCall) => { c.answers = { [c.questions[0]!.question]: 'Unoffered' }; }, (c: NativePlanQuestionCall) => { c.answers!.foreign = 'Foreign'; }, ]) { const c = factory(); mutate(c); expect(classify(c)).toBe(false); } - const classifyFp = factory === issue ? engFirstReviewAUQ : (f: ReturnType) => isEngCompletionHandoff(f, catalog); + const classifyFp = engFirstReviewAUQ; expect(classifyFp({ ...fp(factory()), signature: 'foreign:call' })).toBe(false); expect(classifyFp({ ...fp(factory()), nativeCall: undefined })).toBe(false); expect(classifyFp({ ...fp(factory()), nativeQuestionIndex: 1 })).toBe(false); @@ -186,50 +159,10 @@ describe('Eng AD v2 completed native count evidence', () => { } }); - test('closed handoff accepts either offered action and order, but cannot start or satisfy a review', () => { - const call = handoff(); call.questions[0]!.options.reverse(); - for (const option of call.questions[0]!.options) { - call.answers = { [call.questions[0]!.question]: option.label }; - expect(isHandoff(call)).toBe(true); - } - expect(census([call]).counts).toEqual({ setup: 0, review: 0, administrative: 1 }); - expect(census([call]).phases[0]!.reviewStarted).toBe(false); - }); - - test('new task references or a missing, contradicted, or incomplete reviewed catalog remain substantive', () => { - for (const plan of ['', catalog.replace('**T10 ', '**T11 '), catalog + '\n' + captured.reviewedTasks.lines[2], - catalog.replace('Record regression', 'Do not record regression'), catalog.replace('Record regression', 'Discuss regression')]) { - expect(isHandoff(handoff(), plan)).toBe(false); - } - for (const change of [ - (s: string) => s.replace('T1–T10', 'T1–T11'), - (s: string) => s.replace('T1–T10', 'T2–T10'), - (s: string) => s.replace('record T3', 'record T4'), - (s: string) => s + ' Also add a new migration before shipping.', - (s: string) => s.replace('implement T1–T10', 'approve and implement T1–T10'), - ]) { const c = handoff(); c.questions[0]!.options[0]!.description = change(c.questions[0]!.options[0]!.description); expect(isHandoff(c)).toBe(false); } - }); - - test('conditional closure, extra decisions, appended new work and quoted navigation are never discounted', () => { - for (const change of [ - (s: string) => s.replace('Eng Review is CLEAR', 'Eng Review will be CLEAR after fixing the race'), - (s: string) => s.replace('Eng Review is CLEAR', 'Eng Review is not CLEAR'), - (s: string) => s.replace('What next?', 'What next? Also approve deleting the migration?'), - (s: string) => s + '\nCreate another cache before the next review.', - (s: string) => '> ' + s, - (s: string) => '```text\n' + s + '\n```', - ]) expect(isHandoff(changeQuestion(handoff(), change))).toBe(false); - const c = handoff(); c.questions[0]!.options[1]!.description += ' Remove the CI gate first.'; expect(isHandoff(c)).toBe(false); - const label = handoff(); label.questions[0]!.options[0]!.label += ' and rewrite auth'; - label.answers = { [label.questions[0]!.question]: label.questions[0]!.options[0]!.label }; expect(isHandoff(label)).toBe(false); - const header = handoff(); header.questions[0]!.header = 'Issue 9'; expect(isHandoff(header)).toBe(false); - }); - test('new evidence selects precisely its affected existing paid workflows', () => { const selected = (path: string) => Object.entries(E2E_TOUCHFILES).filter(([, patterns]) => patterns.some(p => matchGlob(path, p))).map(([name]) => name).sort(); for (const path of ['test/eng-count-ad-v2.test.ts', 'test/fixtures/eng-count-ad-v2.json']) { expect(selected(path)).toEqual(['plan-eng-finding-count', 'plan-eng-multi-finding-batching']); } - expect(selected('test/helpers/eng-completion-handoff.ts')).toEqual(['plan-eng-finding-count']); }); }); diff --git a/test/eng-count-owned-outcomes.test.ts b/test/eng-count-owned-outcomes.test.ts deleted file mode 100644 index 6b6c0bd3c..000000000 --- a/test/eng-count-owned-outcomes.test.ts +++ /dev/null @@ -1,119 +0,0 @@ -// Exact acknowledged public decisions and owned plan excerpts from the cancelled f359 run. -// These free controls diagnose detectors; they do not credit the paid attempt. -import { expect, test } from 'bun:test'; -import captured from './fixtures/eng-count-owned-outcomes-f359.json'; -import { evaluateEngSeedCoverage } from './helpers/eng-seeded-coverage'; -const result = (calls: any[] = [], plan = '') => evaluateEngSeedCoverage({status:'ready',calls,assistantMessages:[],planReadyRequests:[]}, plan, captured.startedAt, captured.finishedAt); -const regression = (plan: string) => result([],plan).regression; -const replace = (s: string, from: string, to: string) => {expect(s).toContain(from); return s.replace(from,to);}; -const decision = (mutate: (q:any,c:any)=>void = () => {}) => {const c=structuredClone(captured.calls[7]!); const q=c.questions[0]!; mutate(q,c); c.answers={[q.question]: q.options[0]!.label}; return c;}; -test('captured decisions independently cover four seeds without administrative or duplicate credit', () => { - const seeds = [[3,'sequential-idp'],[4,'complexity'],[5,'shared-cache'],[7,'swallowed-errors']] as const; - for(const [index,seed] of seeds) expect(Object.keys(result([captured.calls[index]!]).decisions)).toEqual([seed]); - for(const index of [0,1,2,6,8,9,10]) expect(result([captured.calls[index]!]).decisions).toEqual({}); -}); -test('captured required corpus binds capture before rewrite and identical replay to one approved decision',()=>{expect(regression(captured.plan)).toBe('plan');}); -for (const [name, mutate] of Object.entries({ - 'missing function':(q:any)=>{q.question=q.question.replaceAll('validateAndDispatch()', 'otherFunction()');}, - 'foreign source':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','OTHER.md');}, - 'archived source':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','archive/PLAN.md');}, - 'quoted explanation':(q:any)=>{q.question=q.question.replace(/^ELI10: (.+)$/m,'ELI10: "$1"');}, - 'historical explanation':(q:any)=>{q.question=q.question.replace('ELI10: ','ELI10: Historical example: ');}, - 'conditional explanation':(q:any)=>{q.question=q.question.replace('ELI10: ','ELI10: If approved, ');}, - 'withdrawn question':(q:any)=>{q.question+='\nThis decision is withdrawn.';}, - 'reopened question':(q:any)=>{q.question+='\nThis decision is reopened.';}, - 'no current swallow defect':(q:any)=>{q.question=q.question.replace(/quietly eating/g,'correctly propagating');}, - 'no named steps':(q:any)=>{q.options[0].description=replace(q.options[0].description,'named steps','unrelated helpers');}, - 'no error boundary':(q:any)=>{q.options[0].description=replace(q.options[0].description,'one top-level boundary','several independent handlers');}, - 'partial mapping':(q:any)=>{q.options[0].description=replace(q.options[0].description,'each error class','some error classes');}, - 'no explicit outcomes':(q:any)=>{q.options[0].description=replace(q.options[0].description,'an explicit outcome','a log entry');}, - 'no legacy oracle':(q:any)=>{q.options[0].description=replace(q.options[0].description,'legacyAuthFlow()', 'otherAuthFlow()');}, - 'borrowed remedy':(q:any)=>{q.question+='\nNet: '+q.options[0].description;q.options[0].description='Discuss the next steps.';}, - 'split remedy across options':(q:any)=>{const [a,b]=q.options[0].description.split(';');q.options[0].description=a;q.options[1].description=b;}, - 'quoted option':(q:any)=>{q.options[0].description='"'+q.options[0].description+'"';}, - 'option withdrawn':(q:any)=>{q.options[0].description+='\nThis remedy is withdrawn.';}, - 'option conditional':(q:any)=>{q.options[0].description='If approved, '+q.options[0].description;}, - 'imperative mapping veto':(q:any)=>{q.options[0].description+='\nDo not map each error class.';}, - 'declarative mapping veto':(q:any)=>{q.options[0].description=replace(q.options[0].description,'boundary maps','boundary does not map');}, - 'negated legacy match':(q:any)=>{q.options[0].description=replace(q.options[0].description,'that matches','that never matches');}, -})) { - test('owned outcome mapping rejects '+name,()=>{expect(result([decision(mutate)]).decisions).toEqual({});}); -} -for(const [name,mutate] of Object.entries({ - 'pending':(c:any)=>{c.answered=false;},'failed':(c:any)=>{c.failed=true;},'late ACK':(c:any)=>{c.answeredAt=new Date(captured.finishedAt+1).toISOString();},'foreign ACK':(c:any)=>{c.answers={'another question':'A'};}, -}))test('outcome mapping preserves '+name+' control',()=>{const c=decision();mutate(c);expect(result([c]).decisions).toEqual({});}); -const planChanges: Recordstring> = { - 'missing source record':s=>s.replace(/### R4:[\s\S]*?(?=## Implementation Tasks)/,''), - 'foreign plan source':s=>s.replaceAll('PLAN.md','OTHER.md'), - 'archived record':s=>s.replace('## Decision ledger','## Historical decision ledger'), - 'source-owned record':s=>s.replace('### R4:','Source example:\n\n### R4:'), - 'source-owned task section':s=>s.replace('## Implementation Tasks','Source example:\n\n## Implementation Tasks'), - 'missing baseline task':s=>s.replace(/- \[ \] \*\*T1 [\s\S]*?(?=- \[ \] \*\*T6 )/,''), - 'missing replay task':s=>s.replace(/- \[ \] \*\*T6 [\s\S]*/,''), - 'foreign replay file':s=>replace(s,'tests/auth/legacyCharacterization.test.* (target switch)','tests/auth/other.test.* (target switch)'), - 'duplicate source decision':s=>replace(s,'(D9 → A)\n - Files: tests/auth/legacyCharacterization.test.* (target switch)','(D9 → A; D10 → A)\n - Files: tests/auth/legacyCharacterization.test.* (target switch)'), - 'wrong selected source':s=>s.replaceAll('(D9 → A)','(D9 → B)'), - 'different actual answer':s=>replace(s,'**A — Full characterization suite**','**B — Reduced characterization matrix**'), - 'ambiguous record state':s=>replace(s,'State: approved\n','State: rejected\n'), - 'duplicate accepted scope':s=>s.replace(/^(Accepted scope: .+)$/m,'$1\n$1'), - 'duplicate record':s=>s.replace('## Implementation Tasks',s.slice(s.indexOf('### R4:'),s.indexOf('## Implementation Tasks'))+'\n## Implementation Tasks'), - 'duplicate task':s=>s+ '\n'+s.slice(s.indexOf('- [ ] **T6 ')), - 'no full inventory':s=>s.replace(/^\| Input matrix \|.*$/m,''), - 'late task baseline':s=>replace(s,'doubles) before any rewrite','doubles) after any rewrite'), - 'negated task baseline':s=>replace(s,'doubles) before any rewrite','doubles) not before any rewrite'), - 'baseline does not pass':s=>replace(s,'Verify: suite green against legacy','Verify: suite not green against legacy'), - 'partial baseline corpus':s=>replace(s,'every matrix row present','some matrix rows present'), - 'partial replay outcomes':s=>replace(s,'identical outcomes on every row','identical outcomes on some rows'), - 'negated replay outcomes':s=>replace(s,'Verify: identical outcomes on every row','Verify: not identical outcomes on every row'), - 'foreign replay target':s=>replace(s,'suite against `AuthBroker` + `SessionMint`;','suite against `OtherBroker` + `SessionMint`;'), - 'no deletion parity gate':s=>replace(s,'only when identical','whenever convenient'), - 'selected option lacks capture':s=>replace(s,"Capture legacyAuthFlow()'s observable behavior",'Discuss the observable behavior'), - 'scope late baseline':s=>replace(s,'Accepted scope: before any rewrite','Accepted scope: after any rewrite'), - 'scope different corpus':s=>replace(s,'Replay the identical suite','Replay a different suite'), - 'scope conditional':s=>replace(s,'Accepted scope: before','Accepted scope: If approved, before'), - 'quoted whole report':s=>s.split('\n').map(l=>'> '+l).join('\n'), - 'fenced whole report':s=>'```\n'+s+'\n```', - 'withdrawn T1':s=>s+'\n## Current status\nT1 is withdrawn.\n', - 'withdrawn T6':s=>s+'\n## Current status\nT6 is withdrawn.\n', - 'withdrawn R4':s=>s+'\n## Current status\nR4 is withdrawn.\n', - 'changed expected outcomes':s=>s+'\n## Current status\nChange T1 assertions to match the new behavior.\n', - 'modified before capture':s=>s+'\n## Current status\nlegacyAuthFlow() is rewritten before T1.\n', -}; -for(const [name,change] of Object.entries(planChanges))test('owned corpus rejects '+name,()=>{const s=change(captured.plan);expect(s).not.toBe(captured.plan);expect(regression(s)).toBeUndefined();}); -for(const [name,change] of Object.entries({ - 'renamed task IDs':(s:string)=>s.replaceAll('T1','T13').replaceAll('T6','T18'), - 'renamed corpus file':(s:string)=>s.replaceAll('tests/auth/legacyCharacterization.test.*','spec/previousBehavior.test.ts'), - 'renamed R/D IDs':(s:string)=>s.replaceAll('R4','R23').replaceAll('D9','D17'), - 'quoted withdrawn status':(s:string)=>s+'\n## Current status\n"T1 is withdrawn."\n', - 'historical withdrawn status':(s:string)=>s+'\n## History\nT1 is withdrawn.\n', -}))test('owned corpus supports '+name,()=>{expect(regression(change(captured.plan))).toBe('plan');}); -for(const veto of ['The boundary does not map each error class.', 'The rewrite will not preserve the captured behavior.', 'The boundary will not match legacyAuthFlow() outcomes.']) test('a later current outcome veto overrides the earlier remedy: '+veto,()=>{ - expect(result([decision(q=>{q.options[0].description+='\n'+veto;})]).decisions).toEqual({}); - expect(Object.keys(result([decision(q=>{q.options[0].description+='\nPrior note: "'+veto+'"';})]).decisions)).toEqual(['swallowed-errors']); -}); -for(const [name,change] of Object.entries({ - 'inconsistent inventory count':(s:string)=>replace(s,'15 rows listed in R4 grid','14 rows listed in R4 grid'), - 'foreign inventory owner':(s:string)=>replace(s,'15 rows listed in R4 grid','15 rows listed in R9 grid'), - 'withdrawn replay verification':(s:string)=>s+'\n## Current status\nT6 verification is optional.\n', - 'mismatched selected label':(s:string)=>replace(s,'**A — Full characterization suite**','**A — Reduced characterization matrix**'), - 'duplicate baseline verification':(s:string)=>s.replace(/^( - Verify: suite green.*)$/m,'$1\n$1'), - 'conditional task':(s:string)=>replace(s,'— Capture `legacyAuthFlow()`','— If approved, capture `legacyAuthFlow()`'), - 'borrowed record under example ancestor':(s:string)=>replace(s,'## Decision ledger','## Copied example\n### Decision ledger'), -}))test('owned corpus rejects '+name,()=>{expect(regression(change(captured.plan))).toBeUndefined();}); - -for(const [name,change] of Object.entries({ - 'selected capture refused':(s:string)=>replace(s,"Capture legacyAuthFlow()'s observable behavior","Do not capture legacyAuthFlow()'s observable behavior"), - 'selected recording refused':(s:string)=>replace(s,"Capture legacyAuthFlow()'s observable behavior","Never record legacyAuthFlow()'s observable behavior"), - 'selected capture conditional':(s:string)=>replace(s,"Capture legacyAuthFlow()'s observable behavior","If approved, capture legacyAuthFlow()'s observable behavior"), - 'scope replay refused':(s:string)=>replace(s,'Replay the identical suite','Do not replay the identical suite'), - 'task replay refused':(s:string)=>replace(s,'only when identical\n','only when identical; do not replay the characterization suite\n'), - 'task replay prohibited':(s:string)=>replace(s,'only when identical\n','only when identical; never replay the characterization suite\n'), -}))test('owned corpus rejects current action veto: '+name,()=>{expect(regression(change(captured.plan))).toBeUndefined();}); -test('owned corpus keeps a quoted replay veto distinct from the required task',()=>{ - const plan=replace(captured.plan,'only when identical\n','only when identical; prior note: "do not replay the characterization suite"\n'); - expect(regression(plan)).toBe('plan'); -}); -for(const [name,change] of Object.entries({ - 'negated accepted baseline':(s:string)=>replace(s,'Accepted scope: before any rewrite','Accepted scope: not before any rewrite'), - 'duplicate selected grid column':(s:string)=>replace(s,'| Choice | Current | A | B | C |','| Choice | Current | A | A | C |'), -}))test('owned corpus rejects '+name,()=>{expect(regression(change(captured.plan))).toBeUndefined();}); diff --git a/test/eng-current-native-seeds.test.ts b/test/eng-current-native-seeds.test.ts deleted file mode 100644 index 592a4ceb2..000000000 --- a/test/eng-current-native-seeds.test.ts +++ /dev/null @@ -1,71 +0,0 @@ -import { expect, test } from 'bun:test'; -import captured from './fixtures/eng-current-native-seeds-6714.json'; -import { evaluateEngSeedCoverage } from './helpers/eng-seeded-coverage'; -import type { NativePlanQuestionCall } from './helpers/plan-count-transcript'; -const calls = () => structuredClone(captured.calls) as NativePlanQuestionCall[]; -const cases = [[3, 'complexity'], [4, 'shared-cache'], [8, 'swallowed-errors'], [12, 'sequential-idp']] as const; -const evaluate = (calls: NativePlanQuestionCall[]) => evaluateEngSeedCoverage({ status: 'ready', calls, assistantMessages: [], planReadyRequests: [] }, '', captured.startedAt, captured.finishedAt); -test('the four actual completed decisions independently cover their own seeds', () => { - for (const [index, seed] of cases) expect(Object.keys(evaluate([calls()[index]!]).decisions)).toEqual([seed]); -}); -test('the cancelled attempt has four decision witnesses but no fabricated final report or regression evidence', () => { - const input = calls(), before = JSON.stringify(input), result = evaluate(input); - expect(Object.keys(result.decisions).sort()).toEqual(cases.map(([, seed]) => seed).sort()); - expect(new Set(Object.values(result.decisions)).size).toBe(4); - expect(result.ok).toBe(false); - expect(result.problems).toContain('mandatory legacy regression coverage absent'); - expect(result.problems).toContain('final review report absent or empty'); - expect(JSON.stringify(input)).toBe(before); -}); -const edit = (index: number, mutate: (q: NativePlanQuestionCall['questions'][number]) => void) => { - const call = calls()[index]!, q = call.questions[0]!; mutate(q); call.answers = { [q.question]: q.options[0]!.label }; return call; -}; -for (const [name, mutate] of Object.entries({ - 'quoted source': (q: any) => { q.question = q.question.replace(/^Project\/branch\/task: (.+)$/m, 'Project/branch/task: "$1"'); }, - 'historical source': (q: any) => { q.question = q.question.replace('Project/branch/task: ', 'Project/branch/task: Historical example: '); }, - 'foreign plan': (q: any) => { q.question = q.question.replaceAll('PLAN.md', 'OTHER.md'); }, - 'quoted explanation': (q: any) => { q.question = q.question.replace(/^ELI10: (.+)$/m, 'ELI10: "$1"'); }, - 'conditional explanation': (q: any) => { q.question = q.question.replace('ELI10: ', 'ELI10: If approved, '); }, - 'withdrawn decision': (q: any) => { q.question += '\nThis decision is withdrawn.'; }, - 'withdrawn selected remedy': (q: any) => { q.options[0].description += '\nThis remedy is withdrawn.'; }, - 'borrowed repair in Net': (q: any) => { q.question += '\nNet: ' + q.options[0].description; q.options[0].description = 'Discuss the next steps.'; }, -})) test('new explained classes reject ' + name, () => { - for (const [index] of cases.slice(0, 3)) expect(evaluate([edit(index, mutate)]).decisions).toEqual({}); -}); -test('inventory counts, one backing store, injection ownership and propagated errors remain required', () => { - const changes = [ - [3, (q: any) => { q.question = q.question.replace('plan adds five new units', 'plan adds six new units'); }], - [3, (q: any) => { q.options[0].description = q.options[0].description.replace('3 new classes', '4 new classes'); }], - [3, (q: any) => { q.options[0].description = q.options[0].description.replace('One token layer', 'Two token layers'); }], - [4, (q: any) => { q.options[0].description = q.options[0].description.replace('passed to both services', 'passed to a different service'); }], - [4, (q: any) => { q.options[0].description = q.options[0].description.replace('tests pass a fresh one', 'tests share the existing one'); }], - [8, (q: any) => { q.options[0].description = q.options[0].description.replace('every error logged and propagated', 'some errors logged and propagated'); }], - [8, (q: any) => { q.options[0].description = q.options[0].description.replace('every error logged and propagated', 'every error logged and swallowed'); }], - ] as const; - for (const [index, change] of changes) expect(evaluate([edit(index, change)]).decisions).toEqual({}); -}); -test('owned R status and header must agree with the native decision', () => { - for (const [index, id] of [[4, 'R1'], [8, 'R5']] as const) { - expect(evaluate([edit(index, q => { q.header = 'R99'; })]).decisions).toEqual({}); - expect(evaluate([edit(index, q => { q.question += `\n${id} is withdrawn.`; })]).decisions).toEqual({}); - } -}); -test('pending, failed and out-of-window native decisions cannot cover seeds', () => { - for (const [index] of cases) for (const mutate of [ - (c: NativePlanQuestionCall) => { c.answered = false; }, - (c: NativePlanQuestionCall) => { c.failed = true; }, - (c: NativePlanQuestionCall) => { c.answeredAt = new Date(captured.finishedAt + 1).toISOString(); }, - (c: NativePlanQuestionCall) => { c.answers = { [c.questions[0]!.question]: 'not offered' }; }, - ]) { const call = calls()[index]!; mutate(call); expect(evaluate([call]).decisions).toEqual({}); } -}); - -test('current owned source filenames cannot be borrowed from suffixes or another directory', () => { - for (const [index] of cases.slice(0, 3)) for (const file of ['OTHER-PLAN.md', 'archive/PLAN.md', '../PLAN.md']) - expect(evaluate([edit(index, q => { q.question = q.question.replaceAll('PLAN.md', file); })]).decisions).toEqual({}); -}); -test('local cancellation of each offered repair overrides earlier positive details', () => { - for (const [index, veto] of [[3, 'Do not fold TokenStore or RequestPolicy.'], [4, 'Do not inject AuthCache.'], [8, 'Never propagate errors.']] as const) { - expect(evaluate([edit(index, q => { q.options[0]!.description += '\nCorrection: ' + veto; })]).decisions).toEqual({}); - expect(Object.keys(evaluate([edit(index, q => { q.options[0]!.description += '\nPrior note: "' + veto + '"'; })]).decisions)).toHaveLength(1); - } -}); diff --git a/test/eng-declared-regression-ai.test.ts b/test/eng-declared-regression-ai.test.ts deleted file mode 100644 index 5f1e3eee9..000000000 --- a/test/eng-declared-regression-ai.test.ts +++ /dev/null @@ -1,197 +0,0 @@ -import { expect, test } from 'bun:test'; -import fixture from './fixtures/eng-declared-regression-ai.json'; -import { evaluateEngSeedCoverage } from './helpers/eng-seeded-coverage'; -import type { PlanCountTranscript } from './helpers/plan-count-transcript'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; - -const transcript = fixture.transcript as PlanCountTranscript; -const { start, end } = fixture.provenance.window; -const plan = [ - '## Tests\n\n### CRITICAL regression (mandatory, regression rule)\n\n' + fixture.mandatory, - '## Implementation Tasks\n\n' + fixture.task, - '## Verification\n\n' + fixture.verification, - fixture.reviewReport, -].join('\n\n'); -const evaluate = (text = plan, native = transcript) => evaluateEngSeedCoverage(native, text, start, end); -const retryPlan = [ - '## Architecture\n\n' + fixture.retry.legacyHeading + '\n\n' + fixture.retry.legacy, - '## Tests\n\n' + fixture.retry.heading + '\n\n' + fixture.retry.mandatory, - '## Implementation Tasks\n\n' + fixture.retry.task, - fixture.retry.reviewReport, -].join('\n\n'); -const evaluateRetry = (text = retryPlan) => evaluateEngSeedCoverage(fixture.retry.transcript as PlanCountTranscript, - text, fixture.retry.provenance.window.start, fixture.retry.provenance.window.end); - -test('actual mandatory suite, numbered task and unchanged baseline bind legacy regression', () => { - const result = evaluate(); - expect(Object.keys(result.decisions)).toHaveLength(4); - expect(result.missing).toEqual([]); - expect(result.regression).toBe('plan'); - expect(result.ok).toBe(true); - expect(fixture.provenance.retrospectivePass).toBe(false); -}); - -test('retry same-fixture contract compares new behavior with the unchanged legacy release oracle', () => { - const result = evaluateRetry(); - expect(result.missing).toEqual([]); - expect(result.regression).toBe('plan'); - expect(result.ok).toBe(true); - expect(fixture.retry.provenance.retrospectivePass).toBe(false); -}); - -test('retry requires an unchanged legacy release oracle and actual result parity', () => { - for (const text of [ - retryPlan.replace(fixture.retry.legacy, ''), - retryPlan.replace('stays callable and unchanged this release', 'will be rewritten this release'), - retryPlan.replace('stays callable and unchanged this release', 'might stay callable and unchanged this release'), - retryPlan.replace('- A tenant-keyed flag', 'If approved:\n\n- A tenant-keyed flag'), - retryPlan.replace('- A tenant-keyed flag', 'Unless rejected.\n\n- A tenant-keyed flag'), - retryPlan.replace('- A tenant-keyed flag', 'Proposed baseline:\n\n- A tenant-keyed flag'), - retryPlan.replace('legacyAuthFlow()` stays callable', 'newAuthFlow()` stays callable'), - retryPlan.replace('Run each fixture', 'Run different fixtures'), - retryPlan.replace('and assert identical `Session` shape on success', 'and document different `Session` shape on success'), - retryPlan.replace('identical error code on failure', 'similar error code on failure'), - retryPlan.replace('through `legacyAuthFlow()` and', 'through `newAuthFlow()` and'), - retryPlan.replace('AuthBroker.authenticate()', 'NewBroker.authenticate()'), - retryPlan.replace('and AuthBroker, identical', 'and NewBroker, identical'), - retryPlan.replace('identical Session / error codes', 'identical OtherResponse / error codes'), - retryPlan.replace('Files: src/auth/authFlow.contract.test.ts', 'Files: src/auth/other.contract.test.ts'), - retryPlan.replace('suite green on both paths', 'suite green on the new path'), - retryPlan.replace(fixture.retry.task, ''), - retryPlan.replace('This test\nis also the gate', 'This optional test\nis also the gate'), - ]) { expect(text).not.toBe(retryPlan); expect(evaluateRetry(text).regression, text).toBeUndefined(); } -}); - -test('retry proposals and current withdrawals cannot supply parity coverage', () => { - for (const text of [ - retryPlan.replace(fixture.retry.mandatory, 'If approved, ' + fixture.retry.mandatory), - retryPlan.replace(fixture.retry.mandatory, '"' + fixture.retry.mandatory + '"'), - retryPlan.replace(fixture.retry.mandatory, '```\n' + fixture.retry.mandatory + '\n```'), - retryPlan.replace(fixture.retry.mandatory, fixture.retry.mandatory + '\nThis test is withdrawn.'), - retryPlan.replace(fixture.retry.task, fixture.retry.task + '\nT4 is no longer required.'), - retryPlan.replace(fixture.retry.legacy, fixture.retry.legacy + '\nCorrection: legacyAuthFlow() is changed this release.'), - retryPlan.replace(fixture.retry.legacy, fixture.retry.legacy.split('\n').map(line => '> ' + line).join('\n')), - '# Hypothetical example\n\n' + retryPlan, - '# Proposed work\n\n' + retryPlan, - ]) { expect(text).not.toBe(retryPlan); expect(evaluateRetry(text).regression, text).toBeUndefined(); } -}); - -test('consistent retry identities and unrelated negative outcomes retain parity evidence', () => { - for (const text of [ - retryPlan.replaceAll('AuthBroker', 'SessionBroker').replaceAll('Session', 'Reply'), - retryPlan.replaceAll('authFlow.contract.test.ts', 'loginFlow.contract.test.js').replaceAll('T4', 'T14'), - retryPlan.replace(fixture.retry.task, fixture.retry.task + '\n - Verify revoked tokens are rejected.'), - retryPlan.replace(fixture.retry.legacy, fixture.retry.legacy + '\nRejected alternatives stay documented.'), - ]) expect(evaluateRetry(text).regression).toBe('plan'); -}); - -for (const prefix of [ - '# Source\n\n', - 'An unproven hypothesis.\n\n', - 'The following is a hypothetical example.\n\n', - 'The following is source material only, not the current reviewed plan.\n\n', - '# Current reviewed plan\n\nThe following sections reproduce source material only; they are not requirements of this plan.\n\n', -]) { - test('first and retry evidence retain enclosing source frame: ' + prefix.trim(), () => { - expect(evaluate(prefix + plan).regression).toBeUndefined(); - expect(evaluateRetry(prefix + retryPlan).regression).toBeUndefined(); - }); -} - -test('declaration wording and task identity can vary without changing the required baseline', () => { - for (const text of [ - plan.replaceAll('T4', 'T12'), - plan.replace('capture current', 'pin existing'), - plan.replace('is added as a critical', 'is required as a mandatory'), - plan.replace('auth/legacy tests — ', 'core/auth — '), - plan.replace('wrong-audience, wrong-issuer,', 'wrong-audience, wrong-issuer, malformed,'), - plan.replace(fixture.task, fixture.task + '\n - Verify expired and revoked tokens are rejected.'), - plan.replace(fixture.mandatory, fixture.mandatory + '\nKeep a record of rejected alternatives.'), - '# Historical example\n\nA proposed suite was discussed.\n\n# Current reviewed plan\n\n' + plan, - ]) expect(evaluate(text).regression, text).toBe('plan'); -}); - -test('declaration, task, target and original baseline cannot lend each other missing evidence', () => { - for (const text of [ - plan.replace(fixture.mandatory, ''), - plan.replace(fixture.task, ''), - plan.replace(fixture.verification, ''), - plan.replace('suite (T4)', 'suite (T5)'), - plan.replace('against the untouched', 'against the rewritten'), - plan.replace('first and commit it green. This is the baseline.', 'after rollout and document it.'), - plan.replace('capture current', 'describe future'), - plan.replace('is\nthe oracle the new path is compared to', 'is documentation the new path links to'), - plan.replace('characterization test suite** for `legacyAuthFlow()`', 'characterization test suite** for `newAuthFlow()`'), - plan.replace('suite for `legacyAuthFlow()` prior behavior', 'suite for `newAuthFlow()` prior behavior'), - plan.replace('untouched `legacyAuthFlow()`', 'untouched `newAuthFlow()`'), - ]) { expect(text).not.toBe(plan); expect(evaluate(text).regression, text).toBeUndefined(); } -}); - -test('proposals, future work, conditional and quoted declarations are not required coverage', () => { - for (const text of [ - plan.replace('is added as a critical', 'will be added as a critical'), - plan.replace('is added as a critical', 'might be added as a critical'), - plan.replace('is added as a critical', 'is not added as a critical'), - plan.replace(fixture.mandatory, 'If approved, ' + fixture.mandatory), - plan.replace(fixture.mandatory, 'Example: ' + fixture.mandatory), - plan.replace(fixture.mandatory, 'An unproven hypothesis. ' + fixture.mandatory), - plan.replace(fixture.mandatory, '"' + fixture.mandatory + '"'), - plan.replace(fixture.mandatory, "'" + fixture.mandatory + "'"), - plan.replace(fixture.mandatory, fixture.mandatory.split('\n').map(line => '> ' + line).join('\n')), - plan.replace(fixture.mandatory, '```\n' + fixture.mandatory + '\n```'), - '# Hypothetical example\n\n' + plan, - '# Quoted source\n\n' + plan, - '# Proposed work\n\n' + plan, - plan.replace('## Verification\n\n1. Run', '## Verification\n\n1. If approved, run'), - ]) { expect(text).not.toBe(plan); expect(evaluate(text).regression, text).toBeUndefined(); } -}); - -test('withdrawal of the owned suite, task or baseline prevents credit', () => { - for (const [from, addition] of [ - [fixture.mandatory, 'This suite is withdrawn.'], - [fixture.mandatory, 'The characterization suite is not required.'], - [fixture.mandatory, 'Do not run the suite.'], - [fixture.task, 'Correction: T4 is cancelled.'], - [fixture.task, 'This task is deferred.'], - [fixture.verification, 'Correction: T4 is cancelled.'], - [fixture.verification, 'Skip the characterization suite.'], - ]) expect(evaluate(plan.replace(from!, from + '\n' + addition)).regression, addition).toBeUndefined(); -}); - -test('the required suite cannot replace completed distinct native decisions or final report', () => { - for (let index = 0; index < transcript.calls.length; index++) { - const native = structuredClone(transcript); - native.calls.splice(index, 1); - expect(evaluate(plan, native).ok).toBe(false); - expect(evaluate(plan, native).missing).toHaveLength(1); - } - for (const mutate of [ - (native: PlanCountTranscript) => { native.calls[0]!.failed = true; }, - (native: PlanCountTranscript) => { native.calls[0]!.answeredAt = new Date(start - 1).toISOString(); }, - (native: PlanCountTranscript) => { native.calls[0]!.sessionId = 'foreign-session'; }, - (native: PlanCountTranscript) => { native.calls.push(structuredClone(native.calls[0]!)); }, - ]) { - const native = structuredClone(transcript); mutate(native); - expect(evaluate(plan, native).ok).toBe(false); - } - expect(evaluate(plan.replace(fixture.reviewReport, '')).problems).toContain('final review report absent or empty'); -}); - -test('public declaration still requires the existing owned time and session interval', () => { - const native = structuredClone(transcript); - native.assistantMessages = [{ sessionId: native.calls[0]!.sessionId, timestamp: new Date(start).toISOString(), text: plan }]; - expect(evaluate(fixture.reviewReport, native).regression).toBe('public-narration'); - native.assistantMessages[0]!.timestamp = new Date(start - 1).toISOString(); - expect(evaluate(fixture.reviewReport, native).regression).toBeUndefined(); - native.assistantMessages[0]!.timestamp = new Date(end + 1).toISOString(); - expect(evaluate(fixture.reviewReport, native).regression).toBeUndefined(); - native.assistantMessages[0]!.timestamp = new Date(start).toISOString(); - native.assistantMessages[0]!.sessionId = 'foreign-session'; - expect(evaluate(fixture.reviewReport, native).regression).toBeUndefined(); -}); - -test('new declaration evidence registers only the two existing engineering count owners', () => { - for (const file of ['test/eng-declared-regression-ai.test.ts', 'test/fixtures/eng-declared-regression-ai.json']) { - expect(selectTests([file], E2E_TOUCHFILES, []).selected.sort()).toEqual(['plan-eng-finding-count', 'plan-eng-multi-finding-batching']); - } -}); diff --git a/test/eng-declared-suite-ak.test.ts b/test/eng-declared-suite-ak.test.ts deleted file mode 100644 index 5b565905b..000000000 --- a/test/eng-declared-suite-ak.test.ts +++ /dev/null @@ -1,120 +0,0 @@ -import { expect, test } from 'bun:test'; -import fixture from './fixtures/eng-declared-suite-ak.json'; -import { evaluateEngSeedCoverage } from './helpers/eng-seeded-coverage'; -import type { PlanCountTranscript } from './helpers/plan-count-transcript'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; - -const plan = [fixture.required, '## Implementation Tasks\n\n' + fixture.task, fixture.verification, fixture.reviewReport].join('\n\n'); -const { start, end } = fixture.provenance.window; -const native = () => structuredClone(fixture.transcript) as PlanCountTranscript; -const evaluate = (p = plan, t = native()) => evaluateEngSeedCoverage(t, p, start, end); - -test('the required characterization suite binds the current legacy oracle, task and both router paths', () => { - expect(evaluate().regression).toBe('plan'); - expect(evaluate().ok).toBe(true); -}); - -test('presentation and task/router identity vary without weakening the baseline', () => { - for (const p of [ - plan.replaceAll('T7', 'T19'), - plan.replaceAll('routeAuth', 'dispatchAuth'), - plan.replace('before touching it', 'before refactoring it'), - plan.replace('capture current inputs and outputs', 'record current inputs and outputs'), - plan.replaceAll('tests/regression', 'tests/auth-regression'), - plan.replace('success,\nexpired token, bad signature', 'success,\nexpired token, invalid audience'), - plan + '\n## Assessment of T12\nT12 is cancelled.', - plan + '\n## Payment regression suite\nThe regression suite is no longer required.', - plan.replace(fixture.task, fixture.task + '\nOld note: "T7 is cancelled."'), - plan + '\n## Historical note\n"The legacy regression suite is no longer required."', - plan.replace(fixture.task, '- [ ] T6 — tests/renderer — Test display text\n - Verify: renders the literal text "This is a hypothetical example."\n\n' + fixture.task), - ]) { expect(p).not.toBe(plan); expect(evaluate(p).regression).toBe('plan'); } -}); - -test('the declaration and task require current legacy capture on both paths', () => { - for (const p of [ - plan.replace(fixture.required, ''), plan.replace(fixture.task, ''), plan.replace(fixture.verification, ''), - plan.replace('mandatory rule, no approval needed', 'optional future idea'), - plan.replace('before touching it', 'after rewriting it'), - plan.replace('capture current inputs and outputs', 'describe proposed inputs and outputs'), - plan.replace('the same suite against `routeAuth` on both flag settings', 'the same suite against `routeAuth` on the new setting'), - plan.replace('A behavior difference between\npaths is a test failure', 'A behavior difference between\npaths is acceptable'), - plan.replace('suite for legacyAuthFlow(), run on both router paths', 'suite for newAuthFlow(), run on both router paths'), - plan.replace('suite passes on legacy before any refactor', 'suite passes on legacy after the refactor'), - plan.replace('passes on new path before flag enable', 'passes on new path after flag enable'), - ]) { expect(p).not.toBe(plan); expect(evaluate(p).regression).toBeUndefined(); } -}); - -test('the comparison uses the unchanged legacy result before the refactor', () => { - for (const p of [ - plan.replace('on the unmodified code', 'on the modified code'), - plan.replace('against `legacyAuthFlow()` on the unmodified code', 'against `newAuthFlow()` on the unmodified code'), - plan.replace('must pass before any refactor lands', 'may pass after the refactor lands'), - plan.replace('through `routeAuth` with the flag on `new`', 'through `differentRouter` with the flag on `new`'), - plan.replace('with the flag on `new`', 'with the flag on `legacy`'), - plan.replace('zero differences', 'accepted differences'), - plan.replace(/^1\. Run the characterization.+$/m, ''), - plan.replace(/^3\. Run the characterization.+$/m, ''), - plan.replace(/^1\. Run the characterization/m, '4. Run the characterization'), - plan.replace(/^1\. Run the characterization/m, 'If approved:\n1. Run the characterization'), - plan.replace(/^3\. Run the characterization/m, 'If approved:\n3. Run the characterization'), - ]) { expect(p).not.toBe(plan); expect(evaluate(p).regression).toBeUndefined(); } -}); - -test('quoted, proposed and conditional owners cannot provide the current requirement', () => { - for (const p of [ - '# Source\n\n' + plan, - '# Hypothetical example\n\n' + plan, - 'The following is source text only.\n\n' + plan, - plan.replace(fixture.required, '```md\n' + fixture.required + '\n```'), - plan.replace(fixture.task, fixture.task.split('\n').map(s => '> ' + s).join('\n')), - plan.replace(fixture.verification, '```md\n' + fixture.verification + '\n```'), - plan.replace('### REGRESSION', '### Proposed REGRESSION'), - plan.replace('**Add a characterization', '**If approved, add a characterization'), - plan.replace('**Add a characterization', 'If approved:\n**Add a characterization'), - plan.replace(fixture.task, 'If approved:\n' + fixture.task), - plan.replace('## Implementation Tasks', '## Optional Implementation Tasks'), - plan.replace('## Verification (end to end)', '## Quoted Verification (end to end)'), - plan.replace('**Add a characterization', 'The following is a quoted source excerpt.\n**Add a characterization'), - plan.replace('1. Run the characterization', 'The following is a quoted source excerpt.\n1. Run the characterization'), - plan.replace(' - Verify: suite passes', ' If approved:\n - Verify: suite passes'), - ]) { expect(evaluate(p).regression).toBeUndefined(); } -}); - -test('the owned suite, numbered task and baseline may be explicitly withdrawn', () => { - for (const p of [ - plan.replace(fixture.required, fixture.required + '\nThis suite is withdrawn.'), - plan.replace(fixture.task, fixture.task + '\nT7 is cancelled.'), - plan + '\n## Assessment of T7\nT7 is rejected.', - plan + '\n## Final regression suite assessment\nThe regression suite is no longer required.', - plan + '\n## Payment regression suite\nThe legacy regression suite is no longer required.', - plan.replace(fixture.verification, fixture.verification + '\nThis baseline is no longer required.'), - plan.replace(fixture.verification, fixture.verification + '\nSkip the characterization suite.'), - plan.replace(fixture.task, fixture.task + '\nCorrection: this unchanged-code verification is withdrawn.'), - ]) expect(evaluate(p).regression).toBeUndefined(); -}); - -for (const prefix of ['If approved:', 'The following is a quoted source excerpt.']) { - test(`a previous task cannot hide the next task's owning prefix: ${prefix}`, () => { - const p = plan.replace(fixture.task, '- [ ] T6 — tests/setup — Prepare fixtures\n - Verify: setup is ready.\n\n' + prefix + '\n' + fixture.task); - expect(evaluate(p).regression).toBeUndefined(); - }); -} - -test('all four separate owned decisions and the final review report remain required', () => { - expect(evaluate().missing).toEqual([]); - expect(new Set(Object.values(evaluate().decisions)).size).toBe(4); - expect(evaluate(plan.replace(fixture.reviewReport, '')).ok).toBe(false); - for (const mutate of [ - (t: PlanCountTranscript) => { t.calls[0]!.answered = false; }, - (t: PlanCountTranscript) => { t.calls[0]!.sessionId = 'foreign'; }, - (t: PlanCountTranscript) => { t.calls[0]!.answeredAt = new Date(start - 1).toISOString(); }, - (t: PlanCountTranscript) => { t.calls[0]!.answeredAt = new Date(end + 1).toISOString(); }, - (t: PlanCountTranscript) => { t.calls.push(structuredClone(t.calls[0]!)); }, - ]) { const t = native(); mutate(t); expect(evaluate(plan, t).ok).toBe(false); } -}); - -test('the new exact public regression evidence belongs only to the existing Eng count owner', () => { - for (const file of ['test/eng-declared-suite-ak.test.ts', 'test/fixtures/eng-declared-suite-ak.json']) { - expect(selectTests([file], E2E_TOUCHFILES, []).selected).toEqual(['plan-eng-finding-count']); - } -}); diff --git a/test/eng-error-flow-seed.test.ts b/test/eng-error-flow-seed.test.ts deleted file mode 100644 index 113666531..000000000 --- a/test/eng-error-flow-seed.test.ts +++ /dev/null @@ -1,512 +0,0 @@ -import { expect, test } from 'bun:test'; -import fixture from './fixtures/eng-69193-count-public.json'; -import currentFixture from './fixtures/eng-e366-count-public.json'; -import type { NativePlanQuestionCall } from './helpers/plan-count-transcript'; -import { evaluateEngSeedCoverage, isEngSeedDecisionAUQ } from './helpers/eng-seeded-coverage'; -import { nativePlanCallFingerprint } from './helpers/claude-pty-runner'; - -const original = fixture.calls.find(c=>c.questions[0]!.header === 'D5 error flow') as NativePlanQuestionCall; -const startedAt = Date.parse(fixture.windowStart), finishedAt = Date.parse(fixture.windowEnd); -const check = (call = structuredClone(original)) => evaluateEngSeedCoverage( - { status: 'ready', calls: [call], assistantMessages: [] }, '', startedAt, finishedAt); -const classify = (call = structuredClone(original)) => isEngSeedDecisionAUQ( - nativePlanCallFingerprint(call, 1, false), [], startedAt, finishedAt); -const regressionCalls = () => structuredClone(fixture.calls) as NativePlanQuestionCall[]; -const regression = (plan = fixture.report, calls = regressionCalls()) => evaluateEngSeedCoverage( - { status: 'ready', calls, assistantMessages: [] }, plan, startedAt, finishedAt).regression; - -test('the current native-approved matrix captures legacy first and separately asserts its two approved deltas', () => { - expect(regression()).toBe('plan'); -}); - -function recordEdit(plan: string, id: string, edit: (text: string) => string) { - const sections = plan.split(/(?=^#{1,6} )/m), selected = sections.filter(s => s.startsWith(`### ${id}:`)); - expect(selected).toHaveLength(1); - const before = selected[0]!, after = edit(before); expect(after).not.toBe(before); - return sections.map(s => s === before ? after : s).join(''); -} -const scopeEdit = (id: string, edit: (text: string) => string, plan = fixture.report) => recordEdit(plan, id, - text => text.replace(/^Accepted scope: (.+)$/m, (_line, scope: string) => 'Accepted scope: '+edit(scope))); - -for (const [name, edit] of [ - ['missing baseline', (s: string) => s.replace(/\(1\) [^]*?(?=\(2\))/, '')], - ['baseline after rewrite', (s: string) => s.replace('BEFORE any rewrite', 'AFTER the rewrite')], - ['reversed baseline and replay', (s: string) => s.replace('(1)', '(later)').replace('(2)', '(1)').replace('(later)', '(2)')], - ['new-path baseline', (s: string) => s.replace('against the existing `legacyAuthFlow()`', 'against `AuthBroker.validateAndDispatch()`')], - ['missing replay', (s: string) => s.replace(/\(2\) [^]*?(?=\(3\))/, '')], - ['different replay matrix', (s: string) => s.replace('The identical matrix run', 'A different matrix run')], - ['foreign replay implementation', (s: string) => s.replace('`AuthBroker.validateAndDispatch()`', '`AnotherBroker.validateAndDispatch()`')], - ['missing matrix axis', (s: string) => s.replace('wrong audience; ', '')], - ['IDP failures not per call', (s: string) => s.replace('for each of the 5 calls', 'for one selected call')], - ['one of five IDP calls', (s: string) => s.replace('for each of the 5 calls', 'for each of the 1 calls')], - ['four of five IDP calls', (s: string) => s.replace('for each of the 5 calls', 'for each of the 4 calls')], - ['missing IDP 5xx failures', (s: string) => s.replace('timeout and 5xx', 'timeout')], - ['missing cache assertions', (s: string) => s.replace('cache read/write effect, and ', '')], - ['wrong prior error decision', (s: string) => s.replace('(D5)', '(D19)')], - ['wrong prior cache decision', (s: string) => s.replace('(D4)', '(D19)')], - ['missing prior delta', (s: string) => s.replace('; stale write dropped after invalidation (D4)', '')], - ['extra unapproved delta', (s: string) => s.replace('(D4).', '(D4); permit unknown tenants (D19).')], - ['broader error delta', (s: string) => s.replace('explicit deny + reason code where legacy swallowed', 'deny every formerly valid request')], - ['broader cache delta', (s: string) => s.replace('stale write dropped after invalidation', 'all cache writes dropped')], - ['missing flag requirement', (s: string) => s.replace('Cutover behind a feature flag', 'Cutover immediately')], - ['cutover before capture', (s: string) => s.replace('(1)', '(later)').replace('(5)', '(1)').replace('(later)', '(5)')], - ['cutover before replay', (s: string) => s.replace('(2)', '(later)').replace('(5)', '(2)').replace('(later)', '(5)')], - ['missing selected E2E', (s: string) => s.replace(/\(4\) [^]*?(?=\(5\))/, '')], - ['missing selected E2E flow', (s: string) => s.replace('; IDP revocation → next request denied', '')], - ['E2E before deltas', (s: string) => s.replace('(3)', '(later)').replace('(4)', '(3)').replace('(later)', '(4)')], -] as const) test(`matrix contract rejects ${name}`, () => { - expect(regression(scopeEdit('R6', edit))).toBeUndefined(); -}); - -for (const id of ['R4','R5','R6']) test(`matrix contract binds ${id} to its complete approved native selection`, () => { - for (const edit of [ - (s: string) => s.replace('State: approved', 'State: pending'), - (s: string) => s.replace(/^Actual answer: .+\n/m, ''), - (s: string) => s.replace(/^Actual answer: A/m, 'Actual answer: B'), - (s: string) => s.replace('PLAN.md:', 'OTHER.md:'), - (s: string) => s.replace(/^Header: (.+)$/m, 'Header: $1 changed'), - (s: string) => s.replace(/^Accepted scope: (.+)$/m, '$& This requirement is withdrawn.'), - ]) expect(regression(recordEdit(fixture.report,id,edit))).toBeUndefined(); - const decision = id === 'R4' ? 'D4' : id === 'R5' ? 'D5' : 'D6'; - for (const edit of [ - (c: NativePlanQuestionCall) => { c.answered = false; }, - (c: NativePlanQuestionCall) => { c.failed = true; }, - (c: NativePlanQuestionCall) => { c.answers = {}; }, - (c: NativePlanQuestionCall) => { c.questions[0]!.options[0]!.description += ' New behavior.'; }, - (c: NativePlanQuestionCall) => { c.answeredAt = new Date(finishedAt+1).toISOString(); }, - ]) { - const calls = regressionCalls(), call = calls.find(c=>c.questions[0]!.header.startsWith(decision+' '))!; - edit(call); expect(regression(fixture.report,calls)).toBeUndefined(); - } - const calls = regressionCalls(), call = calls.find(c=>c.questions[0]!.header.startsWith(decision+' '))!; - expect(regression(fixture.report,calls.filter(c=>c!==call))).toBeUndefined(); -}); - -test('both supporting approvals must precede the regression selection', () => { - for (const decision of ['D4','D5']) { - const calls = regressionCalls(); calls.find(c=>c.questions[0]!.header.startsWith(decision+' '))!.answeredAt = - calls.find(c=>c.questions[0]!.header.startsWith('D6 '))!.answeredAt; - expect(regression(fixture.report,calls)).toBeUndefined(); - } -}); - -for (const edit of [ - (s: string) => s.replace('P1 CRITICAL', 'P1 non-CRITICAL'), - (s: string) => s.replace('P1 CRITICAL', 'P1'), -]) test('a noncritical R6 cannot fill mandatory regression coverage', () => { - expect(regression(recordEdit(fixture.report,'R6',edit))).toBeUndefined(); -}); - -for (const [name, edit] of [ - ['missing task', (s: string) => s.replace(/^- \[ \] \*\*T4 [^]*?(?=^- \[ \] \*\*T5)/m, '')], - ['baseline runs after rewrite', (s: string) => s.replace('BEFORE any rewrite', 'AFTER the rewrite')], - ['missing replay', (s: string) => s.replace('then run the matrix against `AuthBroker`', 'stop after recording legacy')], - ['different replay', (s: string) => s.replace('then run the matrix', 'then run another matrix')], - ['missing green baseline', (s: string) => s.replace('suite green against legacy first', 'suite runs on the new path')], - ['wrong outcome equality', (s: string) => s.replace('identical outcomes against `AuthBroker`', 'unverified outcomes against `AuthBroker`')], - ['wrong delta inventory', (s: string) => s.replace('intended-delta assertions for D4/D5', 'intended-delta assertions for D4/D19')], - ['wrong delta count', (s: string) => s.replace('except the two asserted deltas', 'except three asserted deltas')], - ['wrong shared file', (s: string) => s.replace('tests/auth/legacyAuthFlow.characterization.test.ts', 'tests/auth/different.test.ts')], -] as const) test(`ordered task rejects ${name}`, () => { - const parts = fixture.report.split(/(?=^#{1,6} )/m); - const old = parts.find(s=>s.startsWith('## Implementation Tasks\n'))!, changed = edit(old); - expect(changed).not.toBe(old); - expect(regression(parts.map(s=>s===old?changed:s).join(''))).toBeUndefined(); -}); - -for (const status of ['R4 is withdrawn.','D5 is "superseded".','R6 is cancelled.','T4 is optional.', - 'legacyAuthFlow() is modified before T4.']) test(`current cancellation rejects ${status}`, () => { - expect(regression(fixture.report+'\n## Current assessment\n'+status)).toBeUndefined(); - expect(regression(fixture.report+'\n## Current assessment\nPrior note: "'+status.replaceAll('"',"'")+'"')).toBe('plan'); -}); - -test('selector captions and scope numbering are representations of the same owned decisions', () => { - const captioned = regressionCalls().filter(c=>/^D[456] /.test(c.questions[0]!.header)).reduce((plan,c)=>recordEdit(plan,'R'+c.questions[0]!.header.match(/^D(\d+)/)![1], - s=>s.replace(/^Actual answer: A \((D\d+) answer, this session\)$/m, - (_line,id)=>`Actual answer: A — "${c.questions[0]!.options[0]!.label}" (${id} answer)`)), fixture.report); - expect(regression(captioned)).toBe('plan'); - expect(regression(scopeEdit('R6',s=>s.replace(/\(([1-5])\) /g,'Step $1: ')))).toBe('plan'); -}); - -test('current approved deltas reject contradictions but retain historical comparison and dotted identifiers', () => { - for (const [id, change] of [ - ['R4', (s: string) => s + ' Correction: stale writes are accepted after invalidation.'], - ['R4', (s: string) => s.replace('captures the generation before the write', 'captures the generation after the write')], - ['R4', (s: string) => s.replace(') if it advanced.', '). An unrelated guard checks if it advanced.')], - ['R5', (s: string) => s + ' Correction: dispatch also runs when an error is denied.'], - ['R5', (s: string) => s + ' Correction: this remedy is fail-open on unknown errors.'], - ['R5', (s: string) => s.replace('unknown/unexpected error → deny', 'unknown/unexpected error → allow')], - ] as const) expect(regression(scopeEdit(id,change))).toBeUndefined(); - for (const identifier of ['audit.trace.stale_write','metrics/auth.cache.counter']) { - expect(regression(scopeEdit('R4',s=>s.replace('auth_cache.put_dropped_stale',identifier)))).toBe('plan'); - } - for (const id of ['R4','R5','R6']) { - // Text after the record's History field stays historical, not a current - // cancellation. Current cancellation controls modify Accepted scope above. - expect(regression(recordEdit(fixture.report,id,s=>s+'\nThis requirement is withdrawn.\n'))).toBe('plan'); - } -}); - -test('exact public neutral error-flow question establishes only the swallowed-error seed', () => { - expect(classify()).toBe(true); - expect(check().decisions).toEqual({ 'swallowed-errors': `${original.sessionId}:${original.toolUseId}` }); - expect(check().ok).toBe(false); - expect(check().regression).toBeUndefined(); -}); - -function editQuestion(call: NativePlanQuestionCall, edit: (text: string) => string) { - const q = call.questions[0]!, answer = call.answers![q.question]!; - const changed = edit(q.question); - expect(changed).not.toBe(q.question); - q.question = changed; - call.answers = { [changed]: answer }; -} - -for (const [name, edit] of [ - ['different neutral title', (s: string) => s.replace(/^D5 — [^\n]+/, 'D42 — Which error policy should validateAndDispatch() use?')], - ['unquoted structural description', (s: string) => s.replace('three nested "try this, and if it blows up, ignore it" blocks, each ignoring a different kind of failure', '3 nested catch blocks. Every block discards its error')], - ['different quoted metaphor supplies no evidence', (s: string) => s.replace('"try this, and if it blows up, ignore it"', '"nested boxes"')], - ['current evidence survives unrelated quoted history', (s: string) => s + '\nPrior note: "This finding is withdrawn."'], - ['inline identifiers and bold headings', (s: string) => s.replaceAll('validateAndDispatch()', '`validateAndDispatch()`').replace('ELI10:', '**ELI10:**').replace('Project/branch/task:', '**Project/branch/task:**')], -] as const) test(name, () => { - const call = structuredClone(original); editQuestion(call, edit); - expect(classify(call)).toBe(true); -}); - -test('native descriptions do not need duplicate tradeoff bullets, and any offered answer still completes the decision', () => { - for (const choice of original.questions[0]!.options) { - const call = structuredClone(original), q = call.questions[0]!; - q.options.reverse(); call.answers = { [q.question]: choice.label }; - expect(classify(call)).toBe(true); - } -}); - -for (const [name, edit] of [ - ['foreign plan', (s: string) => s.replace('PLAN.md', 'OTHER.md')], - ['foreign same basename', (s: string) => s.replace('PLAN.md', 'archive/PLAN.md')], - ['missing plan ownership', (s: string) => s.replace('(PLAN.md)', '(the current proposal)')], - ['foreign explanation', (s: string) => s.replace('The function that decides', 'Another function that decides')], - ['unrelated title', (s: string) => s.replace(/^D5 — [^\n]+/, 'D5 — Which report format should we use?')], - ['missing explanation', (s: string) => s.replace(/^ELI10: .+\n/m, '')], - ['quoted explanation', (s: string) => s.replace(/^ELI10: (.+)$/m, 'ELI10: `$1`')], - ['blockquoted explanation', (s: string) => s.replace(/^ELI10:/m, '> ELI10:')], - ['historical explanation', (s: string) => s.replace(/^ELI10:/m, 'ELI10: Historical example:')], - ['withdrawn finding', (s: string) => s + '\nThis finding is withdrawn.'], - ['quoted current status', (s: string) => s + '\nThis finding is "not current".'], - ['resolved finding', (s: string) => s + '\nThis finding is fixed.'], - ['already surfaced errors', (s: string) => s + '\nCorrection: validateAndDispatch() now rethrows every error.'], - ['no nested defect', (s: string) => s.replace('three nested "try this, and if it blows up, ignore it" blocks', 'one shallow block')], - ['blocks rethrow instead of discarding', (s: string) => s.replace('each ignoring a different kind of failure', 'each rethrowing every failure')], - ['discard fact exists only in quotation', (s: string) => s.replace('each ignoring a different kind of failure', '"each ignoring a different kind of failure"')], - ['conditional current ownership', (s: string) => s + '\nThis finding applies only if approved.'], -] as const) test(name, () => { - const call = structuredClone(original); editQuestion(call, edit); - expect(classify(call)).toBe(false); - expect(check(call).missing).toContain('swallowed-errors'); -}); - -for (const [name, edit] of [ - ['no-op remedy', (s: string) => 'Keep validateAndDispatch() as written; no error-handling change.'], - ['quoted native remedy', (s: string) => '`'+s+'`'], - ['historical native remedy', (s: string) => 'Historical example: '+s], - ['foreign native function', (s: string) => s.replace('validateAndDispatch()', 'anotherFunction()')], - ['missing typed outcomes', (s: string) => s.replace('a typed `AuthError` subclass', 'an unclassified value')], - ['missing deny mapping', (s: string) => s.replace('explicit deny', 'an unspecified response')], - ['missing reason', (s: string) => s.replace('reason code + ', '')], - ['missing log', (s: string) => s.replace('structured log + ', '')], - ['partial step policy', (s: string) => s.replace('Each step throws', 'Only some steps throw')], - ['partial handler policy', (s: string) => s.replace('maps class', 'maps only some classes')], - ['dispatch reachable on failure', (s: string) => s.replace('Dispatch only reachable on the success path.', 'Dispatch also reachable on the failure path.')], - ['current no-log correction', (s: string) => s + '\nCorrection: Do not log denials.'], - ['current partial-error correction', (s: string) => s + '\nCorrection: Only some errors are surfaced.'], - ['current fail-open correction', (s: string) => s + '\nCorrection: This remedy remains fail-open on unknown errors.'], - ['current swallowed-error correction', (s: string) => s + '\nCorrection: Dispatch errors remain swallowed.'], - ['dispatch contradicts deny boundary', (s: string) => s + '\nCorrection: Dispatch also runs when an error is denied.'], - ['dispatch remains reachable after failure', (s: string) => s + '\nCorrection: Dispatch remains reachable after a validation failure.'], - ['current withdrawn remedy', (s: string) => s + '\nThis option is withdrawn.'], -] as const) test(name, () => { - const call = structuredClone(original), q = call.questions[0]!; - const old = q.options[0]!.description!; - q.options[0]!.description = edit(old); expect(q.options[0]!.description).not.toBe(old); - // The displayed brief remains deliberately intact: it must not replace a - // missing or contradictory contract in the actual native option fields. - expect(classify(call)).toBe(false); -}); - -test('a complete remedy cannot be assembled across options', () => { - const call = structuredClone(original), q = call.questions[0]!; - q.options[0]!.description = q.options[0]!.description!.replace('reason code + structured log + ', ''); - q.options[1]!.description += ' Every deny includes reason code + structured log.'; - expect(classify(call)).toBe(false); -}); - -test('native completion and prior-call ownership still gate the recognized seed', () => { - for (const edit of [ - (c: NativePlanQuestionCall) => { c.answered = false; }, - (c: NativePlanQuestionCall) => { c.failed = true; }, - (c: NativePlanQuestionCall) => { c.answers = {}; }, - (c: NativePlanQuestionCall) => { c.sessionId = ''; }, - (c: NativePlanQuestionCall) => { c.answeredAt = new Date(startedAt-1).toISOString(); }, - (c: NativePlanQuestionCall) => { c.answeredAt = new Date(finishedAt+1).toISOString(); }, - ]) { - const call = structuredClone(original); edit(call); expect(classify(call)).toBe(false); - } - const fingerprint = nativePlanCallFingerprint(original, 1, false); - expect(isEngSeedDecisionAUQ(fingerprint, [original], startedAt, finishedAt)).toBe(false); - expect(isEngSeedDecisionAUQ({ ...fingerprint, signature: 'foreign' }, [], startedAt, finishedAt)).toBe(false); -}); - -const currentStart = Date.parse(currentFixture.windowStart), currentEnd = Date.parse(currentFixture.windowEnd); -const currentCall = (header: string) => structuredClone(currentFixture.calls.find(c=>c.questions[0]!.header === header)!) as NativePlanQuestionCall; -const currentClassify = (call: NativePlanQuestionCall) => { - const index = currentFixture.calls.findIndex(c=>c.toolUseId === call.toolUseId); - return isEngSeedDecisionAUQ(nativePlanCallFingerprint(call, 1, false), currentFixture.calls.slice(0,index) as NativePlanQuestionCall[], currentStart, currentEnd); -}; -for (const [header, seed] of [['Complexity','complexity'],['Error handling','swallowed-errors']] as const) { - test(`exact public ${header} decision retains its owned native seed`, () => { - const call=currentCall(header); - expect(currentClassify(call)).toBe(true); - const result=evaluateEngSeedCoverage({status:'ready',calls:[call],assistantMessages:[]},'',currentStart,currentEnd); - expect(result.decisions).toEqual({[seed]:`${call.sessionId}:${call.toolUseId}`}); - expect(result.ok).toBe(false); - }); - for (const [name,edit] of [ - ['foreign plan',(s:string)=>s.replaceAll('PLAN.md','OTHER.md')], - ['foreign same-basename plan',(s:string)=>s.replaceAll('PLAN.md','archive/PLAN.md')], - ['missing explanation',(s:string)=>s.replace(/^ELI10:.*\n/m,'')], - ['literal explanation',(s:string)=>s.replace(/^ELI10: (.+)$/m,'ELI10: `$1`')], - ['quoted explanation',(s:string)=>s.replace(/^ELI10: (.+)$/m,'ELI10: "$1"')], - ['historical explanation',(s:string)=>s.replace('ELI10:','ELI10: Historical example:')], - ['withdrawn finding',(s:string)=>s+'\nThis finding is withdrawn.'], - ['resolved finding',(s:string)=>s+'\nThis finding is fixed.'], - ] as const) test(`${header} rejects ${name} even with an unhyphenated action`,()=>{ - const call=currentCall(header);editQuestion(call,edit); - call.questions[0]!.options[0]!.description=call.questions[0]!.options[0]!.description!.replace('re-throw','rethrow'); - expect(currentClassify(call)).toBe(false); - }); -} - -for(const [name,edit] of [ - ['premodified catch noun',(s:string)=>s.replace('three try/catch blocks nested inside each other','3 nested catch blocks')], - ['postmodified catch noun',(s:string)=>s.replace('three try/catch blocks nested inside each other','three catch blocks that are nested inside each other')], - ['exhaustive discarded failures',(s:string)=>s.replace('each one quietly eats a different kind of error','every catch silently discards a different failure')], - ['neutral policy title',(s:string)=>s.replace(/^D4 — [^\n]+/,'D4 — Which error boundary should validateAndDispatch() use?')], -] as const) test(`current error subject accepts ${name}`,()=>{ - const call=currentCall('Error handling');editQuestion(call,edit);expect(currentClassify(call)).toBe(true); -}); -for(const [name,edit] of [ - ['ASCII step arrows',(s:string)=>s.replaceAll('→','->')], - ['comma-separated steps',(s:string)=>s.replaceAll(' → ', ', ')], - ['single handler',(s:string)=>s.replace('single catch','one error handler')], - ['unhyphenated rethrow',(s:string)=>s.replace('re-throw','rethrow')], - ['spaced rethrow',(s:string)=>s.replace('re-throw','re throw')], - ['object-form unknown policy',(s:string)=>s.replace('unknown errors deny and re-throw','denies unknown errors and rethrows them')], - ['named outcomes',(s:string)=>s.replace('each known error class to an explicit outcome','every known failure class to an explicit named outcome')], - ['legacy success comparison',(s:string)=>s+' Legacy errors used to return success; this policy denies unknown errors and rethrows them.'], - ['negative success claim',(s:string)=>s+' Known errors never return success.'], - ['negative passive success claim',(s:string)=>s+' ValidationError is not treated as success.'], - ['negative dispatch permission',(s:string)=>s+' For ValidationError, dispatch is never allowed.'], - ['owned function preposition',(s:string)=>s.replace('Rewrite validateAndDispatch() as','For validateAndDispatch(), use')], - ['owned method preposition',(s:string)=>s.replace('Rewrite validateAndDispatch() as','In AuthBroker.validateAndDispatch(), implement')], - ['historical named success',(s:string)=>s+' Legacy ValidationError was treated as success.'], - ['both current error policies deny success',(s:string)=>s+' ValidationError is not allowed and PolicyDenied is never allowed.'], -] as const) test(`current error policy accepts ${name}`,()=>{ - const call=currentCall('Error handling'),o=call.questions[0]!.options[0]!;o.description=edit(o.description!);expect(currentClassify(call)).toBe(true); -}); -for(const [name,edit] of [ - ['no ordered flow',(s:string)=>s.replace('validate → decideAccess → dispatch','the old deeply nested body')], - ['reversed flow',(s:string)=>s.replace('validate → decideAccess → dispatch','dispatch → decideAccess → validate')], - ['no single boundary',(s:string)=>s.replace('single catch','several unrelated catches')], - ['partial known classes',(s:string)=>s.replace('each known error class','some known error classes')], - ['missing explicit outcome',(s:string)=>s.replace('explicit outcome','unspecified side effect')], - ['missing structured log',(s:string)=>s.replace('and structured log','without observability')], - ['unknowns not denied',(s:string)=>s.replace('unknown errors deny and re-throw','unknown errors re-throw')], - ['unknowns not propagated',(s:string)=>s.replace('unknown errors deny and re-throw','unknown errors deny')], - ['unknowns allowed',(s:string)=>s.replace('unknown errors deny and re-throw','unknown errors allow and re-throw')], - ['known errors return allow',(s:string)=>s+' Known errors return allow.'], - ['known errors return success',(s:string)=>s+' Known errors return success.'], - ['known errors map to a success status',(s:string)=>s+' Known errors map to 200.'], - ['known named error becomes success',(s:string)=>s+' ValidationError -> success.'], - ['passive known-error success',(s:string)=>s+' ValidationError is treated as success.'], - ['known-error dispatch permission',(s:string)=>s+' For ValidationError, dispatch is allowed.'], - ['known-error successful outcome',(s:string)=>s+' ValidationError has a successful outcome.'], - ['another named error successful outcome',(s:string)=>s+' IdpUnavailable has a successful outcome.'], - ['a later current assertion overrides earlier negation',(s:string)=>s+' ValidationError is not allowed and PolicyDenied is allowed.'], - ['a later current mapping overrides earlier negation',(s:string)=>s+' Known errors never return success and ValidationError maps to 200.'], - ['a current assertion follows historical success',(s:string)=>s+' Previously, ValidationError was allowed and now PolicyDenied is allowed.'], - ['current dispatch-after-error correction',(s:string)=>s+' Correction: dispatch also runs when an error is denied.'], - ['current logging withdrawal',(s:string)=>s+' Correction: Do not log errors.'], - ['current propagation withdrawal',(s:string)=>s+' Correction: Never re-throw errors.'], - ['current fail-open correction',(s:string)=>s+' Correction: This remedy is fail-open on unknown errors.'], - ['foreign function',(s:string)=>s.replace('validateAndDispatch()','anotherFunction()')], - ['foreign function preposition',(s:string)=>s.replace('Rewrite validateAndDispatch() as','For tokenize(), use')], - ['foreign method preposition',(s:string)=>s.replace('Rewrite validateAndDispatch() as','In TokenCodec.parse(), implement')], - ['literal native policy',(s:string)=>'`'+s+'`'], - ['historical native policy',(s:string)=>'Historical example: '+s], - ['withdrawn native policy',(s:string)=>s+' This option is withdrawn.'], - ['no-op native policy',(_:string)=>'Keep validateAndDispatch() and its current behavior.'], -] as const) test(`current error policy rejects ${name}`,()=>{ - const call=currentCall('Error handling'),o=call.questions[0]!.options[0]!;o.description=edit(o.description!);expect(currentClassify(call)).toBe(false); -}); -test('current error map cannot borrow a known-outcome log from another option',()=>{ - const call=currentCall('Error handling'),q=call.questions[0]!; - q.options[0]!.description=q.options[0]!.description!.replace('and structured log',''); - q.options[1]!.description+=' Every known error gets a structured log.'; - expect(currentClassify(call)).toBe(false); -}); - -for(const [name,edit] of [ - ['decision caption',(s:string)=>s.replace('Complexity gate:','Complexity decision:')], - ['word-form declared count',(s:string)=>s.replace('5 new classes','five new classes')], - ['numeric current count',(s:string)=>s.replace('introduces five new classes','introduces 5 new classes')], -] as const) test(`current class inventory accepts ${name}`,()=>{ - const call=currentCall('Complexity');editQuestion(call,edit);expect(currentClassify(call)).toBe(true); -}); -for(const [name,edit] of [ - ['plain pure function',(s:string)=>s.replace('pure exported function','pure function')], - ['named function before noun',(s:string)=>s.replace('pure exported function decideAccess(claims, ctx)','pure exported decideAccess(claims, ctx) function')], - ['passive accounted fold',(s:string)=>s.replace('TokenStore folds into AuthCache','TokenStore is folded into AuthCache')], - ['current negated policy state',(s:string)=>s+' The RequestPolicy function maintains no mutable tenant state.'], - ['historical policy state',(s:string)=>s+' Previously, the RequestPolicy function maintained mutable tenant state.'], - ['current negated class retention',(s:string)=>s+' Do not retain TokenStore as a separate class.'], -] as const) test(`current class remedy accepts ${name}`,()=>{ - const call=currentCall('Complexity'),o=call.questions[0]!.options[0]!;o.description=edit(o.description!);expect(currentClassify(call)).toBe(true); -}); -for(const [name,edit] of [ - ['different baseline count',(s:string)=>s.replace('5 new classes','4 new classes')], - ['different current count',(s:string)=>s.replace('introduces five new classes','introduces four new classes')], - ['quoted current count',(s:string)=>s.replace('it introduces five new classes across twelve files','"it introduces five new classes across twelve files"')], - ['missing current policy defect',(s:string)=>s.replace('RequestPolicy is described by the plan itself as stateless with no side effects','RequestPolicy owns changing tenant policy state')], - ['missing current store defect',(s:string)=>s.replace('TokenStore is never described','TokenStore has a documented independent responsibility')], - ['current policy is stateful',(s:string)=>s+'\nCorrection: RequestPolicy is now stateful.'], - ['current policy no longer stateless',(s:string)=>s+'\nCorrection: RequestPolicy is no longer stateless.'], - ['current store has its own responsibility',(s:string)=>s+'\nCorrection: TokenStore now has a documented independent responsibility.'], -] as const) test(`current class subject rejects ${name}`,()=>{ - const call=currentCall('Complexity');editQuestion(call,edit);expect(currentClassify(call)).toBe(false); -}); -for(const [name,index,field,edit] of [ - ['missing original inventory',2,'description',(_:string)=>'Keep the original arrangement.'], - ['wrong original member',2,'description',(s:string)=>s.replace('TokenStore','OtherStore')], - ['duplicate original member',2,'description',(s:string)=>s.replace('TokenStore','AuthCache')], - ['wrong original count',2,'label',(s:string)=>s.replace('5 classes','4 classes')], - ['wrong retained count',0,'label',(s:string)=>s.replace('3 units','2 units')], - ['wrong retained member',0,'description',(s:string)=>s.replace('AuthBroker, SessionMint, AuthCache','AuthBroker, SessionMint, OtherCache')], - ['duplicate retained member',0,'description',(s:string)=>s.replace('AuthBroker, SessionMint, AuthCache','AuthBroker, AuthBroker, AuthCache')], - ['missing pure-policy remedy',0,'description',(s:string)=>s.replace('pure exported function','stateful class')], - ['missing store fold',0,'description',(s:string)=>s.replace('TokenStore folds into AuthCache','TokenStore stays independent')], - ['missing single backing adapter',0,'description',(s:string)=>s.replace('one facade over the one backing adapter','a facade over several stores')], - ['current policy state correction',0,'description',(s:string)=>s+' Correction: RequestPolicy remains a separate class with mutable state.'], - ['current store retention correction',0,'description',(s:string)=>s+' Correction: TokenStore remains its own class.'], - ['current function state correction',0,'description',(s:string)=>s+' Correction: The RequestPolicy function now maintains mutable tenant state.'], - ['current imperative class retention',0,'description',(s:string)=>s+' Correction: Retain TokenStore as a separate class.'], - ['current imperative policy restoration',0,'description',(s:string)=>s+' Restore RequestPolicy as a distinct class.'], - ['literal native remedy',0,'description',(s:string)=>'`'+s+'`'], - ['withdrawn native remedy',0,'description',(s:string)=>s+' This option is withdrawn.'], - ['foreign original inventory',2,'description',(s:string)=>'Historical example: '+s], -] as const) test(`current class inventory rejects ${name}`,()=>{ - const call=currentCall('Complexity'),o=call.questions[0]!.options[index]!;o[field]=edit(o[field]!);expect(currentClassify(call)).toBe(false); -}); -test('current class remedy cannot borrow the missing fold from another option',()=>{ - const call=currentCall('Complexity'),q=call.questions[0]!; - q.options[0]!.description=q.options[0]!.description!.replace('TokenStore folds into AuthCache (one facade over the one backing adapter).',''); - q.options[1]!.description+=' TokenStore folds into AuthCache (one facade over the one backing adapter).'; - expect(currentClassify(call)).toBe(false); -}); - - -test('the unchanged complete public report has all four decisions but leaves its critical regression requirement unflagged', () => { - const calls=currentFixture.calls as NativePlanQuestionCall[]; - const result=evaluateEngSeedCoverage({status:'ready',calls,assistantMessages:[]},currentFixture.report,currentStart,currentEnd); - expect(Object.keys(result.decisions).sort()).toEqual(['complexity','sequential-idp','shared-cache','swallowed-errors']); - expect(result.missing).toEqual([]); - expect(result.regression).toBeUndefined(); - expect(result.problems).toContain('mandatory legacy regression coverage absent'); - expect(result.ok).toBe(false); -}); - -// Counterfactual evidence is explicit: the captured report itself never flags -// this risk CRITICAL. Only that missing required flag is added for parser tests. -const criticalCurrentReport = () => recordEdit(currentFixture.report, 'R5', s=>s.replace('Finding: T1, P1,', 'Finding: T1, P1, CRITICAL,')); -const currentRegression = (report=criticalCurrentReport(), calls=currentFixture.calls as NativePlanQuestionCall[]) => - evaluateEngSeedCoverage({status:'ready',calls,assistantMessages:[]},report,currentStart,currentEnd).regression; -const currentScope = (edit:(s:string)=>string) => scopeEdit('R5',edit,criticalCurrentReport()); -const currentTask = (edit:(s:string)=>string) => { - const plan=criticalCurrentReport(), before=plan.match(/^- \[ \] \*\*T1 \([^]*?(?=^- \[ \] \*\*T2)/m)?.[0]; - expect(before).toBeDefined(); const after=edit(before!);expect(after).not.toBe(before); - return plan.replace(before!,after); -}; -test('adding only the mandatory CRITICAL flag exposes the complete native-approved paragraph regression contract',()=>{ - expect(currentRegression()).toBe('plan'); - expect(currentRegression(currentFixture.report)).toBeUndefined(); -}); -for(const [name,edit] of [ - ['missing legacy baseline',(s:string)=>s.replace('write the characterization suite against legacyAuthFlow() BEFORE the rewrite covering','write characterization tests covering')], - ['late legacy baseline',(s:string)=>s.replace('BEFORE the rewrite','AFTER the rewrite')], - ['foreign legacy baseline',(s:string)=>s.replace('legacyAuthFlow()','differentAuthFlow()')], - ['missing selected case',(s:string)=>s.replace('cross-tenant token, ','')], - ['missing selected timeout',(s:string)=>s.replace(' and IDP timeout','')], - ['missing cache assertions',(s:string)=>s.replace(' and cache state','')], - ['different replay suite',(s:string)=>s.replace('the same suite','a different suite')], - ['missing new-flow replay',(s:string)=>s.replace('The new flow must pass the same suite.','')], - ['unapproved difference',(s:string)=>s.replace("D4's explicit deny", "D3's explicit deny")], - ['broader approved difference',(s:string)=>s.replace('explicit deny where legacy swallowed an error','allow on every IDP failure')], - ['unasserted difference',(s:string)=>s.replace('listed and asserted','merely listed')], - ['additional unapproved difference',(s:string)=>s+' Additional product differences are allowed for D7.'], - ['current cache assertion withdrawal',(s:string)=>s+' Cache state is not asserted.'], - ['current outcome assertion withdrawal',(s:string)=>s+' Outcome class is not asserted.'], - ['current new-flow assertion withdrawal',(s:string)=>s+' The new flow is not tested.'], - ['withdrawn requirement',(s:string)=>s+' R5 is withdrawn.'], - ['future requirement',(s:string)=>'If approved: '+s], - ['quoted requirement',(s:string)=>'"'+s+'"'], -] as const) test(`native paragraph regression rejects ${name}`,()=>{ - expect(currentRegression(currentScope(edit))).toBeUndefined(); -}); -for(const [name,edit] of [ - ['wrong native answer',(s:string)=>s.replace('Actual answer: A) Characterization suite','Actual answer: B) Characterization suite')], - ['contradictory selected answer',(s:string)=>s.replace('user chose A','user chose B')], - ['wrong native label',(s:string)=>s.replace('A) Characterization suite\nWrite','A) Different suite\nWrite')], - ['wrong native description',(s:string)=>s.replace('and IDP timeout; assert outcome class','; assert outcome class')], - ['foreign finding source',(s:string)=>s.replaceAll('PLAN.md','OTHER.md')], - ['missing CRITICAL flag',(s:string)=>s.replace('P1, CRITICAL,','P1,')], - ['non-CRITICAL flag',(s:string)=>s.replace('P1, CRITICAL,','P1, non-CRITICAL,')], - ['negated CRITICAL flag',(s:string)=>s.replace('P1, CRITICAL,','P1, no CRITICAL risk,')], - ['historical quoted severity',(s:string)=>s.replace('P1, CRITICAL,','P1, the previous report used the word "CRITICAL",')], - ['pending approval',(s:string)=>s.replace('State: approved','State: proposed')], -] as const) test(`native paragraph regression record rejects ${name}`,()=>{ - expect(currentRegression(recordEdit(criticalCurrentReport(),'R5',edit))).toBeUndefined(); -}); -test('the current paragraph may explicitly forbid any other product differences',()=>{ - expect(currentRegression(currentScope(s=>s+' No other product differences are allowed.'))).toBe('plan'); -}); -for(const [name,edit] of [ - ['missing scheduled baseline',(s:string)=>s.replace('before any rewrite','with the new flow')], - ['late scheduled baseline',(s:string)=>s.replace('before any rewrite','after the rewrite')], - ['missing legacy green',(s:string)=>s.replace('suite green against legacy; later green against new flow','suite green against new flow')], - ['failed legacy baseline',(s:string)=>s.replace('suite green against legacy','suite failing against legacy')], - ['different replay',(s:string)=>s.replace('later green against new flow','later a different suite green against new flow')], - ['unapproved task difference',(s:string)=>s.replace('only listed D4 differences','only listed D3 differences')], - ['missing task owner',(s:string)=>s.replace('(D6)','(D7)')], - ['missing deliverable',(s:string)=>s.replace(/^ - Files:.*\n/m,'')], - ['partial task inventory',(s:string)=>s.replace('10 scenarios','9 scenarios')], -] as const) test(`native paragraph regression task rejects ${name}`,()=>{ - expect(currentRegression(currentTask(edit))).toBeUndefined(); -}); -for(const [name,edit] of [ - ['baseline after implementation',(s:string)=>s.replace('1. Characterization suite','5. Characterization suite')], - ['baseline gate after replay',(s:string)=>s.replace('green on legacy before step 8','green on legacy after step 8')], - ['new implementation starts before baseline',(s:string)=>s.replace('4. `AuthBroker.validateAndDispatch()` rewrite','0. `AuthBroker.validateAndDispatch()` rewrite')], - ['replay before baseline',(s:string)=>s.replace('8. Run the characterization suite','1. Run the characterization suite')], - ['deleted legacy before baseline',(s:string)=>s+'\nCorrection: legacyAuthFlow() is deleted before T1.\n'], -] as const) test(`native paragraph regression ordering rejects ${name}`,()=>{ - const before=criticalCurrentReport(),after=edit(before);expect(after).not.toBe(before); - expect(currentRegression(after)).toBeUndefined(); -}); -for(const [name,edit] of [ - ['missing approved error decision',(calls:NativePlanQuestionCall[])=>calls.filter(c=>c.questions[0]!.header!=='Error handling')], - ['unanswered approved error decision',(calls:NativePlanQuestionCall[])=>{calls.find(c=>c.questions[0]!.header==='Error handling')!.answered=false;return calls;}], - ['changed approved error answer',(calls:NativePlanQuestionCall[])=>{const c=calls.find(c=>c.questions[0]!.header==='Error handling')!,q=c.questions[0]!;c.answers![q.question]=q.options[1]!.label;return calls;}], - ['late approved error decision',(calls:NativePlanQuestionCall[])=>{calls.find(c=>c.questions[0]!.header==='Error handling')!.answeredAt=new Date(Date.parse(calls.find(c=>c.questions[0]!.header==='Regression')!.answeredAt!)+1).toISOString();return calls;}], - ['foreign regression session',(calls:NativePlanQuestionCall[])=>{calls.find(c=>c.questions[0]!.header==='Regression')!.sessionId='another-session';return calls;}], -] as const) test(`native paragraph regression rejects ${name}`,()=>{ - expect(currentRegression(criticalCurrentReport(),edit(structuredClone(currentFixture.calls) as NativePlanQuestionCall[]))).toBeUndefined(); -}); diff --git a/test/eng-finding-fixture.test.ts b/test/eng-finding-fixture.test.ts index a21f01976..c6695dcb3 100644 --- a/test/eng-finding-fixture.test.ts +++ b/test/eng-finding-fixture.test.ts @@ -1,13 +1,6 @@ import { expect, test } from 'bun:test'; -import { execFileSync } from 'node:child_process'; import * as fs from 'node:fs'; -import * as os from 'node:os'; import * as path from 'node:path'; -import { seedEngFindingProject } from './helpers/eng-finding-fixture'; -import { legacyAuthFlow, POLICIES, AuthFailure, type Platform, type Policy } from './fixtures/eng-existing-auth/legacy-auth'; - -const identity = Object.freeze({ tenantId: 'tenant-a', subjectId: 'subject-a' }); -const session = { id: 'opaque-session', expiresAt: 3_600_000 }; function suppliedCountPlan() { // Execute only the actual pure prompt builder, never import its paid test. @@ -50,101 +43,3 @@ test('count fixture retains all five seeded defects and a coherent class invento expect(names).toEqual(['AuthBroker', 'TokenStore', 'SessionMint', 'AuthCache', 'RequestPolicy']); expect(new Set(names).size).toBe(Number(inventory![1])); }); - -test('Eng fixture commits a real legacy flow alongside the unchanged supplied defects', () => { - const cwd = fs.mkdtempSync(path.join(os.tmpdir(), 'eng-finding-fixture-')); - try { - const defects = '# Proposed refactor\nBoth services mutate a global cache.\nNo regression test is planned.\n'; - const input = seedEngFindingProject(cwd, defects); - const git = (...args: string[]) => execFileSync('git', args, { cwd, encoding: 'utf8', timeout: 5000 }); - expect(input.startsWith(defects)).toBe(true); - expect(git('show', 'HEAD:review-input.md')).toBe(input); - expect(git('show', 'HEAD:src/legacy-auth.ts')).toBe(fs.readFileSync(path.resolve(import.meta.dir, 'fixtures/eng-existing-auth/legacy-auth.ts'), 'utf8')); - const pkg = git('show', 'HEAD:package.json'); - expect(pkg).toBe(fs.readFileSync(path.resolve(import.meta.dir, 'fixtures/eng-existing-auth/package.json'), 'utf8')); - expect(JSON.parse(pkg).scripts.test).toBe('bun test'); - expect(input).toContain('POLICIES order, not response-arrival order'); - expect(input).toContain('prior build artifact for rollback'); - expect(input).toContain('reserve concurrency and rate capacity for five policy calls'); - expect(input).not.toContain('reverting that\nflag restores'); - expect(git('diff', 'origin/main...HEAD')).toBe(''); - expect(git('status', '--porcelain')).toBe(''); - expect(fs.readdirSync(path.join(cwd, 'src'))).toEqual(['legacy-auth.ts']); - } finally { fs.rmSync(cwd, { recursive: true, force: true }); } -}); - -test('legacy flow has five sequential independent calls and issues a session only after all allow', async () => { - const called: Policy[] = []; - const pending: Array<(allow: boolean) => void> = []; - let minted = 0; - const result = legacyAuthFlow(identity, { - checkPolicy: (actual, policy) => { - expect(actual).toBe(identity); - called.push(policy); - return new Promise(resolve => pending.push(resolve)); - }, - issueSession: async actual => { expect(actual).toBe(identity); minted++; return session; }, - }); - for (let i = 0; i < POLICIES.length; i++) { - expect(called).toEqual(POLICIES.slice(0, i + 1)); - expect(minted).toBe(0); - pending[i]!(true); - await Promise.resolve(); - } - expect(await result).toBe(session); - expect(minted).toBe(1); -}); - -test.each(['denied', 'provider_unavailable', 'session_unavailable'] as const)('legacy %s remains an explicit failure', async code => { - const cause = new Error('dependency failure'); - let minted = 0; - const platform: Platform = { - checkPolicy: async () => { if (code === 'provider_unavailable') throw cause; return code !== 'denied'; }, - issueSession: async () => { minted++; throw cause; }, - }; - const failure = await legacyAuthFlow(identity, platform).catch(error => error); - expect(failure).toBeInstanceOf(AuthFailure); - expect(failure.code).toBe(code); - expect(failure.cause).toBe(code === 'denied' ? undefined : cause); - expect(minted).toBe(code === 'session_unavailable' ? 1 : 0); -}); - - -test('existing policy-order failure and short-circuit behavior stay unchanged', async () => { - const called: Policy[] = []; - let minted = false; - const result = await legacyAuthFlow(identity, { - checkPolicy: async (_identity, policy) => { - called.push(policy); - if (policy === 'tenant') return false; - if (policy === 'device') throw new Error('later unavailable policy'); - return true; - }, - issueSession: async () => { minted = true; return session; }, - }).catch(error => error); - expect(result).toBeInstanceOf(AuthFailure); - expect(result.code).toBe('denied'); - expect(called).toEqual(['account', 'tenant']); - expect(minted).toBe(false); -}); - - -test.each(['synchronous throw', 'promise rejection'] as const)('legacy preserves the same provider failure contract for %s', async mode => { - const cause = new Error('policy client failure'); - const called: Policy[] = []; - let minted = false; - const platform: Platform = { - checkPolicy: (_identity, policy) => { - called.push(policy); - if (mode === 'synchronous throw') throw cause; - return Promise.reject(cause); - }, - issueSession: async () => { minted = true; return session; }, - }; - const failure = await legacyAuthFlow(identity, platform).catch(error => error); - expect(failure).toBeInstanceOf(AuthFailure); - expect(failure.code).toBe('provider_unavailable'); - expect(failure.cause).toBe(cause); - expect(called).toEqual(['account']); - expect(minted).toBe(false); -}); diff --git a/test/eng-golden-master-al.test.ts b/test/eng-golden-master-al.test.ts deleted file mode 100644 index fc14bd216..000000000 --- a/test/eng-golden-master-al.test.ts +++ /dev/null @@ -1,115 +0,0 @@ -import { expect, test } from 'bun:test'; -import fixture from './fixtures/eng-golden-master-al.json'; -import { evaluateEngSeedCoverage } from './helpers/eng-seeded-coverage'; -import type { PlanCountTranscript } from './helpers/plan-count-transcript'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; - -const plan = [fixture.required, '## Implementation Tasks\n\n' + fixture.task, fixture.verification, fixture.reviewReport].join('\n\n'); -const { start, end } = fixture.provenance.window; -const native = () => structuredClone(fixture.transcript) as PlanCountTranscript; -const evaluate = (p = plan, t = native()) => evaluateEngSeedCoverage(t, p, start, end); - -test('the captured golden-master requirement binds a numbered task to an untouched baseline', () => { - expect(evaluate().regression).toBe('plan'); - expect(evaluate().ok).toBe(true); -}); - -test('task identities and presentation may vary without changing the required oracle', () => { - for (const p of [ - plan.replaceAll('T1', 'T23'), - plan.replace('— legacy —', '— auth/legacy —'), - plan.replaceAll('golden-master', 'golden master'), - plan.replace('Capture current outputs', 'Record current outputs'), - plan.replace('identical behaviour', 'identical behavior'), - plan.replace('success / expired / revoked / wrong-tenant /\nlogout', 'success / invalid audience / expired'), - plan + '\n## Assessment of T8\nT8 is cancelled.', - plan + '\n## Payment regression suite\nThe regression suite is no longer required.', - plan + '\n## Payment golden-master fixtures\nThe golden-master fixtures are no longer required.', - plan + '\n## Historical note\n"The legacy regression suite is no longer required."', - plan.replace(fixture.task, '- [ ] T0 — renderer — Test literal output\n - Verify: renders "This is a hypothetical example."\n\n' + fixture.task), - ]) { expect(p).not.toBe(plan); expect(evaluate(p).regression).toBe('plan'); } -}); - -test('the mandatory declaration, numbered task and linked verification are all necessary', () => { - for (const p of [ - plan.replace(fixture.required, ''), plan.replace(fixture.task, ''), plan.replace(fixture.verification, ''), - plan.replace('regression rule, mandatory', 'optional future idea'), - plan.replace('Capture current outputs', 'Describe proposed outputs'), - plan.replace('BEFORE any change', 'AFTER the rewrite'), - plan.replace('assert identical behaviour', 'accept different behaviour'), - plan.replace('`legacyAuthFlow` golden-master', '`newAuthFlow` golden-master'), - plan.replace('fixtures for legacyAuthFlow', 'fixtures for newAuthFlow'), - plan.replace('fixtures for legacyAuthFlow before any change', 'fixtures for legacyAuthFlow after the rewrite'), - plan.replace('fixtures pass against untouched legacy', 'fixtures pass against modified legacy'), - plan.replace('rerun after every later task', 'rerun optionally after launch'), - plan.replace('1. Run T1 fixtures', '1. Run T9 fixtures'), - plan.replace('2. After each task, rerun the full suite plus T1 fixtures.', '2. After each task, rerun the full suite plus T9 fixtures.'), - plan.replace('before touching anything; they must pass', 'after rewriting legacy; they may pass'), - plan.replace('1. Run T1 fixtures', '3. Run T1 fixtures'), - ]) { expect(p).not.toBe(plan); expect(evaluate(p).regression).toBeUndefined(); } -}); - -test('source, conditional and optional owners cannot provide current mandatory evidence', () => { - for (const p of [ - '# Source\n\n' + plan, - '# Hypothetical example\n\n' + plan, - 'The following is source text only.\n\n' + plan, - plan.replace(fixture.required, '```md\n' + fixture.required + '\n```'), - plan.replace(fixture.task, fixture.task.split('\n').map(s => '> ' + s).join('\n')), - plan.replace(fixture.verification, '```md\n' + fixture.verification + '\n```'), - plan.replace('**CRITICAL', 'If approved:\n**CRITICAL'), - plan.replace('**CRITICAL', 'The following is a quoted source excerpt.\n**CRITICAL'), - plan.replace('**CRITICAL', 'Source excerpt:\n\n**CRITICAL'), - plan.replace(/Capture current outputs[\s\S]*?no existing coverage\./, claim => '`' + claim + '`'), - plan.replace(fixture.task, 'If approved:\n' + fixture.task), - plan.replace(fixture.task, 'The following is a quoted source excerpt.\n' + fixture.task), - plan.replace('## Implementation Tasks', '## Optional Implementation Tasks'), - plan.replace('## Verification', '## Quoted Verification'), - plan.replace('1. Run T1', 'If approved:\n1. Run T1'), - plan.replace('1. Run T1', 'The following is a quoted source excerpt.\n1. Run T1'), - plan.replace('1. Run T1', 'Source excerpt:\n\n1. Run T1'), - plan.replace(' - Verify:', ' If approved:\n - Verify:'), - ]) { expect(p).not.toBe(plan); expect(evaluate(p).regression).toBeUndefined(); } -}); - -test('a previous unrelated task cannot hide a source or conditional prefix', () => { - for (const prefix of ['If approved:', 'The following is a quoted source excerpt.']) { - const p = plan.replace(fixture.task, '- [ ] T0 — setup — Prepare fixtures\n - Verify: setup passes.\n\n' + prefix + '\n' + fixture.task); - expect(evaluate(p).regression).toBeUndefined(); - } -}); - -test('the required suite, numbered task, and unchanged verification remain withdrawable', () => { - for (const p of [ - plan.replace(fixture.required, fixture.required + '\nThis suite is withdrawn.'), - plan.replace(fixture.task, fixture.task + '\nT1 is cancelled.'), - plan + '\n## Assessment of T1\nT1 is rejected.', - plan + '\n## Final regression suite assessment\nThe regression suite is no longer required.', - plan + '\n## Payment regression suite\nThe legacy regression suite is no longer required.', - plan.replace(fixture.verification, fixture.verification + '\nThis baseline is no longer required.'), - plan.replace(fixture.task, fixture.task + '\nCorrection: this unchanged-code verification is withdrawn.'), - plan.replace(fixture.verification, fixture.verification + '\nCorrection: the T1 rerun is withdrawn.'), - plan + '\n## Final regression assessment\nThe golden-master fixtures are no longer required.', - plan + '\n## Payment regression suite\nThe legacy golden-master fixtures are withdrawn.', - plan.replace('T1 (P1,', 'T1 (optional,'), - ]) { expect(p).not.toBe(plan); expect(evaluate(p).regression).toBeUndefined(); } -}); - -test('the four completed owned decisions and final review report remain required', () => { - expect(evaluate().missing).toEqual([]); - expect(new Set(Object.values(evaluate().decisions)).size).toBe(4); - expect(evaluate(plan.replace(fixture.reviewReport, '')).ok).toBe(false); - for (const mutate of [ - (t: PlanCountTranscript) => { t.calls[0]!.answered = false; }, - (t: PlanCountTranscript) => { t.calls[0]!.sessionId = 'foreign'; }, - (t: PlanCountTranscript) => { t.calls[0]!.answeredAt = new Date(start - 1).toISOString(); }, - (t: PlanCountTranscript) => { t.calls[0]!.answeredAt = new Date(end + 1).toISOString(); }, - (t: PlanCountTranscript) => { t.calls.push(structuredClone(t.calls[0]!)); }, - ]) { const t = native(); mutate(t); expect(evaluate(plan, t).ok).toBe(false); } -}); - -test('only the existing Eng finding-count owner selects these public evidence regressions', () => { - for (const file of ['test/eng-golden-master-al.test.ts', 'test/fixtures/eng-golden-master-al.json']) { - expect(selectTests([file], E2E_TOUCHFILES, []).selected).toEqual(['plan-eng-finding-count']); - } -}); diff --git a/test/eng-golden-parity-an.test.ts b/test/eng-golden-parity-an.test.ts deleted file mode 100644 index 66f442d2c..000000000 --- a/test/eng-golden-parity-an.test.ts +++ /dev/null @@ -1,184 +0,0 @@ -import { expect, test } from 'bun:test'; -import { evaluateEngSeedCoverage } from './helpers/eng-seeded-coverage'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; -import fixture from './fixtures/eng-golden-parity-an.json'; -import heldPackets from './fixtures/eng-native-packets-b955.json'; -const heldLegacy=heldPackets.held6bd; -const heldLegacyCheck=(plan=heldLegacy.plan)=>evaluateEngSeedCoverage(heldLegacy.transcript as any,plan,heldLegacy.startedAt,heldLegacy.finishedAt).regression; -test('held6bd legacy: approved required oracle links current legacy body, before-change task and green baseline',()=>expect(heldLegacyCheck()).toBe('plan')); -test('held6bd legacy: current required characterization heading and equivalent baseline fields',()=>expect(heldLegacyCheck(heldLegacy.plan.replace('R5: Regression coverage for legacyAuthFlow() current behavior','R5: Characterization tests for legacyAuthFlow()').replace('pins current\noutcomes for:','records existing\noutcomes for:').replace('suite green on unmodified legacy body','tests pass on untouched legacy implementation'))).toBe('plan')); -for(const [name,edit] of Object.entries({ - 'reopened current row':(s:string)=>s+'\n## Current amendment\nR5 is reopened.\n', - 'pending current verification':(s:string)=>s+'\n## Current amendment\nT1 is pending approval.\n', - 'negated current selected answer':(s:string)=>s.replace('Actual answer: A — characterization + differential harness','Actual answer: A — no characterization + differential harness'), - 'unrelated current answer':(s:string)=>s.replace('Actual answer: A — characterization + differential harness','Actual answer: A — implement a cache'), -}))test('held6bd legacy current approval rejects '+name,()=>expect(heldLegacyCheck(edit(heldLegacy.plan))).toBeUndefined()); -for(const [name,edit] of Object.entries({ - 'duplicate owned ledger':(s:string)=>s.replace('### R6:','### R5: Regression coverage for legacyAuthFlow() current behavior\nState: pending\n\n### R6:'), - 'duplicate required scope':(s:string)=>s.replace('Accepted scope: (1) `legacyAuthFlow.characterization.test`','Accepted scope: withdrawn\nAccepted scope: (1) `legacyAuthFlow.characterization.test`'), - 'quoted foreign current citation':(s:string)=>s.replace('Finding: T1, P1 (CRITICAL)','Finding: T1, P1 (CRITICAL), source "OTHER.md:1"'), - 'optional current baseline':(s:string)=>s+'\n## Current amendment\nT1 is optional.\n', - 'baseline after rewrite':(s:string)=>s.replace('against the current body, before any other change','against the changed body, after the rewrite'), - 'changed current scope order':(s:string)=>s.replace('legacy body BEFORE any delegation is\nadded','legacy body AFTER delegation is\nadded'), - 'mismatched case count':(s:string)=>s.replace('tests for the 8 input classes','tests for the 7 input classes'), - 'duplicate corpus outcome':(s:string)=>s.replace(/(Accepted scope: \(1\)[\s\S]*?outcomes for: valid token, expired), revoked/,'$1, expired'), -}))test('held6bd legacy current class rejects '+name,()=>{const changed=edit(heldLegacy.plan);expect(changed).not.toBe(heldLegacy.plan);expect(heldLegacyCheck(changed)).toBeUndefined();}); -test('held6bd legacy: consistently renumbered current row and task retain ownership',()=>expect(heldLegacyCheck(heldLegacy.plan.replaceAll('R5','R15').replaceAll('D11','D21').replaceAll('T1 (','T11 ('))).toBe('plan')); -test('held6bd legacy: unrelated and historical withdrawals are inert',()=>expect(heldLegacyCheck(heldLegacy.plan+'\n## Notes\nEarlier note: "T1 is withdrawn."\n## Payment regression suite\nThe suite is withdrawn.\n')).toBe('plan')); -for(const [name,edit] of Object.entries({ - 'historical owner':(s:string)=>s.replace('### R5: Regression coverage','### Historical R5: Regression coverage'), - 'foreign source':(s:string)=>s.replaceAll('PLAN.md:','OTHER.md:'), - 'foreign same basename':(s:string)=>s.replaceAll('PLAN.md:','archive/PLAN.md:'), - 'missing required finding':(s:string)=>s.replace('Finding: T1, P1 (CRITICAL)','Finding: T1, P2'), - 'unapproved row':(s:string)=>s.replace(/(### R5:[\s\S]*?)State: approved/,'$1State: pending'), - 'wrong answer owner':(s:string)=>s.replace('(D11 answer)','(D10 answer)'), - 'unknown selected option':(s:string)=>s.replace('Actual answer: A — characterization','Actual answer: C — characterization'), - 'missing selected option':(s:string)=>s.replace('Options: A) Characterization tests plus a','Options: C) Characterization tests plus a'), - 'missing baseline file':(s:string)=>s.replace(' - Files: auth/legacyAuthFlow.characterization.test',' - Files: auth/otherFlow.characterization.test'), - 'foreign task directory':(s:string)=>s.replace(' - Files: auth/legacyAuthFlow.characterization.test',' - Files: other/legacyAuthFlow.characterization.test'), - 'missing same-row task link':(s:string)=>s.replace(' - Surfaced by: Tests — finding 1 (R5/D11)',' - Surfaced by: Tests — finding 1 (R9/D11)'), - 'modified verification':(s:string)=>s.replace('suite green on unmodified legacy body','suite green on modified legacy body'), - 'missing green verification':(s:string)=>s.replace('suite green on unmodified legacy body','suite red on unmodified legacy body'), - 'current task withdrawal':(s:string)=>s+'\n## Current amendment\nT1 is withdrawn.\n', - 'current row withdrawal':(s:string)=>s+'\n## Current amendment\nR5 is not required.\n', - 'quoted current withdrawal':(s:string)=>s+'\n## Current amendment\nThis baseline verification is "cancelled".\n', - 'reversed current order':(s:string)=>s+'\n## Current amendment\nlegacyAuthFlow() is changed before T1.\n', - 'changed baseline expectations':(s:string)=>s+'\n## Current amendment\nChange T1 assertions.\n', - 'same-task duplicate':(s:string)=>s.replace('- [ ] **T2 (','- [ ] **T1 ('), - 'quoted current plan':(s:string)=>'```md\n'+s+'\n```', -}))test('held6bd legacy rejects '+name,()=>{const changed=edit(heldLegacy.plan);expect(changed).not.toBe(heldLegacy.plan);expect(heldLegacyCheck(changed)).toBeUndefined();}); -const times = fixture.calls.map(call => Date.parse(call.answeredAt)); -const check = (plan = fixture.compact) => evaluateEngSeedCoverage( - { status: 'ready', calls: fixture.calls, assistantMessages: [] }, plan, Math.min(...times) - 1, Math.max(...times) + 1); - -const ledgerParity=fixture.ledgerParityCab3; -test('current approved parity ledger binds the required table, same task and legacy-first lane',()=>{ - expect(check(ledgerParity.plan).regression).toBe('plan'); -}); -const swapParityCells=(text:string)=>text.replace(/^\| (?:R5 test shape|Acceptance assertions) \|.*$/gm,line=>{ - const cells=line.split('|');[cells[3],cells[4]]=[cells[4]!,cells[3]!];return cells.join('|'); -}); -test('the selected parity option cannot borrow another comparison column',()=>{ - expect(check(swapParityCells(ledgerParity.plan)).regression).toBeUndefined(); -}); -test('a coherent option and comparison reorder preserves the selected parity oracle',()=>{ - const reordered=swapParityCells(ledgerParity.plan).replace('Question D11: Shared parity suite (recommended) / Characterization suite /','Question D11: Characterization suite / Shared parity suite (recommended) /'); - expect(check(reordered).regression).toBe('plan'); -}); -const ledgerNegative: Array<[string,(text:string)=>string]> = [ - ['missing mandatory test declaration',s=>s.replace(/^\| D11 CRITICAL.*\n/m,'')], - ['optional declaration',s=>s.replace('| D11 CRITICAL |','| D11 optional |')], - ['historical test section',s=>s.replace('## Tests (revised)','## Historical Tests (revised)')], - ['code-only test declaration',s=>s.replace(/^(\| D11 CRITICAL.*)$/m,'```\n$1\n```')], - ['missing outcome from test declaration',s=>s.replace('valid, expired, revoked, tenant suspended, IDP unreachable, missing tenant;','valid, expired, tenant suspended, IDP unreachable, missing tenant;')], - ['missing outcome from accepted scope',s=>s.replace('valid, expired, revoked, tenant suspended, IDP unreachable, missing tenant ID)','valid, expired, tenant suspended, IDP unreachable, missing tenant ID)')], - ['duplicate owned outcome',s=>s.replace('valid, expired, revoked, tenant suspended','valid, expired, expired, tenant suspended')], - ['missing observed output',s=>s.replace('asserts outcome + cache key written;','asserts cache key written;')], - ['missing observed side effect',s=>s.replace('asserts outcome + cache key written;','asserts outcome;')], - ['different declared implementation',s=>s.replace('parameterized over `legacyAuthFlow()` and `AuthBroker`;','parameterized over `legacyAuthFlow()` and `OtherBroker`;')], - ['different declared test file',s=>s.replace('| `auth/authBehavior.contract.test.ts` |','| `auth/other.contract.test.ts` |')], - ['missing rollout gate',s=>s.replace('both green before any tenant is allowlisted','both green eventually')], - ['missing current ledger',s=>s.slice(0,s.indexOf('### R5:'))], - ['foreign ledger source',s=>s.replaceAll('PLAN.md:','foreign/PLAN.md:')], - ['different source document',s=>s.replaceAll('PLAN.md:','OTHER.md:')], - ['unapproved ledger',s=>s.replace('State: approved','State: pending')], - ['duplicate actual answer',s=>s.replace(/^(Actual answer:.*)$/m,'$1\n$1')], - ['wrong decision answer',s=>s.replace('legacy AND AuthBroker (D11)','legacy AND AuthBroker (D10)')], - ['selected characterization instead of parity',s=>s.replace('Actual answer: Shared parity suite run','Actual answer: Characterization suite run')], - ['missing same-implementation parity assertion',s=>s.replace('identical outcome + identical cache key written for each scenario, both impls','outcome and key may differ between implementations')], - ['assertions borrowed from another option',s=>s.replace('identical outcome + identical cache key written for each scenario, both impls | identical outcome + cache key for legacy','outcome only | identical outcome + identical cache key written for each scenario, both impls')], - ['intentional differences allowed',s=>s.replace('Intentional differences: none in this PR.','Intentional differences: permitted in this PR.')], - ['changed legacy baseline',s=>s.replace('it is unchanged code called through a new router','it is rewritten code called through a new router')], - ['missing task',s=>s.replace(/^- \[ \] \*\*T6 .*\n(?: .*(?:\n|$))*/m,'')], - ['wrong task file',s=>s.replace(' - Files: `auth/authBehavior.contract.test.ts`',' - Files: `auth/other.contract.test.ts`')], - ['missing task decision ownership',s=>s.replace('Test review T1 CRITICAL (D11)','Test review T1 CRITICAL (D10)')], - ['wrong verification count',s=>s.replace('Verify: six scenarios','Verify: five scenarios')], - ['only new implementation verified',s=>s.replace('green for both implementations','green for the new implementation')], - ['verification after rollout',s=>s.replace('before any tenant is allowlisted','after a tenant is allowlisted')], - ['no legacy-first lane',s=>s.replace(/^- Lane B:.*\n/m,'')], - ['new implementation supplies baseline',s=>s.replace('T6 parity suite written against `legacyAuthFlow()`','T6 parity suite written against `AuthBroker`')], - ['foreign task in lane',s=>s.replace('Lane B: T6 parity','Lane B: T7 parity')], - ['wrong implementation added to lane',s=>s.replace('then parameterized over `AuthBroker`','then parameterized over `OtherBroker`')], - ['foreign implementation dependency',s=>s.replace("after Lane A's T3 merges","after Lane A's T7 merges")], - ['self-dependent oracle task',s=>s.replace("after Lane A's T3 merges","after Lane A's T6 merges")], - ['duplicate current record',s=>s+'\n'+ledgerParity.parts[3]], - ['ambiguous comparison columns',s=>s.replace('| Choice | Current | A | B | C |','| Choice | Current | A | A | C |')], - ['conditional lane',s=>s.replace('- Lane B:','- If approved, Lane B:')], - ['historical schedule',s=>s.replace('## Worktree parallelization strategy','## Historical worktree parallelization strategy')], - ['task withdrawal',s=>s+'\n## Current assessment\nT6 is withdrawn.\n'], - ['quoted current withdrawal',s=>s+'\n## Current assessment\nT6 is "withdrawn".\n'], - ['decision superseded',s=>s+'\n## Current assessment\nD11 is superseded.\n'], - ['legacy suite cancelled',s=>s+'\n## Current assessment\nThe legacy parity suite is cancelled.\n'], - ['baseline modified first',s=>s+'\n## Current assessment\nlegacyAuthFlow() is modified before T6.\n'], - ['source-only entire declaration',s=>'# Source excerpt\n'+s.replace(/^#/gm,'##')], -]; -test.each(ledgerNegative)('approved parity contract rejects %s',(_,mutate)=>{const altered=mutate(ledgerParity.plan);expect(altered).not.toBe(ledgerParity.plan);expect(check(altered).regression).toBeUndefined();}); -test('same owned task, decision, implementation and file can be renamed coherently',()=>{ - for(const plan of [ledgerParity.plan.replaceAll('T6','T16').replaceAll('D11','D21').replaceAll('R5','R15'), - ledgerParity.plan.replaceAll('AuthBroker','NextAuthenticator').replaceAll('authBehavior.contract.test.ts','compatibility.test.js'), - ledgerParity.plan+'\n## History\nOld note: "T6 is withdrawn."\n', - ledgerParity.plan+'\n## Payment parity suite\nThe parity suite is withdrawn.\n'])expect(check(plan).regression).toBe('plan'); -}); - -test('exact golden requirement binds current outputs, the same task and untouched baseline to flag-off parity', () => { - expect(check().ok).toBe(true); - expect(check().regression).toBe('plan'); -}); - -const negative: Array<[string, (plan: string) => string]> = [ - ['source ancestor', s => '# Source excerpt\n' + s], - ['historical owner', s => s.replace('### Test requirements', '### Historical test requirements')], - ['source declaration prefix', s => s.replace(fixture.declaration, 'Source:\n' + fixture.declaration)], - ['earlier declaration prefix', s => s.replace(fixture.declaration, 'Earlier review assessment:\n' + fixture.declaration)], - ['conditional declaration', s => s.replace(fixture.declaration, 'If approved:\n' + fixture.declaration)], - ['quoted declaration', s => s.replace(fixture.declaration, fixture.declaration.split('\n').map(line => '> ' + line).join('\n'))], - ['literal declaration', s => s.replace(fixture.declaration, '~~~\n' + fixture.declaration + '~~~\n')], - ['optional regression requirement', s => s.replace('REGRESSION RULE, no approval needed', 'optional regression suggestion')], - ['different characterization target', s => s.replaceAll('legacyAuthFlow', 'anotherFlow')], - ['unlinked declared task', s => s.replace('(T3, REGRESSION RULE', '(T8, REGRESSION RULE')], - ['unlinked ordering task', s => s.replace('(T3)** — pin', '(T8)** — pin')], - ['unlinked task file', s => s.replace(' - Files: auth/legacyAuthFlow.regression.test.ts', ' - Files: auth/anotherFlow.regression.test.ts')], - ['missing golden oracle', s => s.replace('These tests are the parity oracle', 'These tests are not the parity oracle')], - ['conditional parity', s => s.replace('These tests are the parity oracle', 'If approved, these tests are the parity oracle')], - ['modified baseline', s => s.replace('against unmodified legacy code', 'against modified legacy code')], - ['reversed baseline ordering', s => s.replace('before any refactor commit', 'after the refactor commit')], - ['future outputs', s => s.replace('pin current outputs', 'pin proposed outputs')], - ['reversed capture ordering', s => s.replace('before any other code moves', 'after the other code moves')], - ['source ordering prefix', s => s.replace(fixture.ordering, 'Source excerpt:\n' + fixture.ordering)], - ['conditional task prefix', s => s.replace(fixture.task, 'If approved:\n' + fixture.task)], - ['source task prefix', s => s.replace(fixture.task, 'Source:\n' + fixture.task)], - ['source baseline verification', s => s.replace(' - Verify: six', ' Source:\n - Verify: six')], - ['conditional baseline verification', s => s.replace(' - Verify: six', ' If approved:\n - Verify: six')], - ['withdrawn same task', s => s + '\n## Final assessment\nT3 is withdrawn.\n'], - ['withdrawn same verification', s => s + '\n## Final assessment\nT3 verification is withdrawn.\n'], - ['directly quoted verification withdrawal', s => s + '\n## Final assessment\nT3 verification is "withdrawn".\n'], - ['current golden suite cancelled', s => s + '\n## Final assessment\nThe legacy golden tests are cancelled.\n'], - ['legacy modified before baseline', s => s + '\n## Final assessment\nlegacyAuthFlow() is modified before T3.\n'], - ['owned test requirement withdrawn', s => s.replace(fixture.declaration, fixture.declaration + 'These tests are withdrawn.\n')], - ['owned baseline withdrawn', s => s.replace(fixture.task, fixture.task + ' Correction: this baseline verification is withdrawn.\n')], - ['quoted owned baseline withdrawal', s => s.replace(fixture.task, fixture.task + ' Correction: this baseline verification is "withdrawn".\n')], - ['quoted legacy golden cancellation', s => s + '\n## Final assessment\nThe legacy golden tests are "cancelled".\n'], - ['owned requirement not current', s => s.replace(fixture.declaration, fixture.declaration + 'This requirement is not current.\n')], -]; -test.each(negative)('%s cannot supply a current unchanged oracle', (_, change) => { - const plan = change(fixture.compact); - expect(plan).not.toBe(fixture.compact); - expect(check(plan).regression).toBeUndefined(); -}); - -test('same-task renumbering, harmless quoted history and unrelated suite preserve the oracle', () => { - expect(check(fixture.compact.replaceAll('T3', 'T8')).ok).toBe(true); - expect(check(fixture.compact + '\n## Notes\nOld note: "T3 verification is withdrawn."\n').ok).toBe(true); - expect(check(fixture.compact + '\n## Payment regression suite\nThe regression suite is withdrawn.\n').ok).toBe(true); - expect(check(fixture.compact.replace(fixture.declaration, 'Old note: "Source:"\n' + fixture.declaration)).ok).toBe(true); -}); - -test('new regression artifacts select only the existing Eng owner and its dependency list stays dense', () => { - for (const path of ['test/eng-golden-parity-an.test.ts', 'test/fixtures/eng-golden-parity-an.json']) - expect(selectTests([path], E2E_TOUCHFILES, []).selected).toEqual(['plan-eng-finding-count']); - const row = E2E_TOUCHFILES['plan-eng-finding-count']; - for (let index = 0; index < row.length; index++) { - expect(Object.hasOwn(row, index)).toBe(true); - expect(typeof row[index]).toBe('string'); - } -}); diff --git a/test/eng-initial-selector-043a.test.ts b/test/eng-initial-selector-043a.test.ts deleted file mode 100644 index 8bb48dcb1..000000000 --- a/test/eng-initial-selector-043a.test.ts +++ /dev/null @@ -1,61 +0,0 @@ -import {test,expect} from 'bun:test'; -import capture from './fixtures/eng-initial-selector-043a.json'; -import {isEngCompletionHandoff} from './helpers/eng-completion-handoff'; -import {nativePlanCallFingerprint} from './helpers/claude-pty-runner'; -import {isEngSeedDecisionAUQ} from './helpers/eng-seeded-coverage'; -import type {NativePlanQuestionCall} from './helpers/plan-count-transcript'; -const actual=()=>({calls:structuredClone(capture.transcript.calls) as NativePlanQuestionCall[],plan:capture.correctedQuestionsPlan}); -type Case=ReturnType; -const accepts=(x:Case)=>isEngCompletionHandoff(nativePlanCallFingerprint(x.calls.at(-1)!,0),x.plan,x.calls.slice(0,-1)); -function check(name:string,want:boolean,change?:(x:Case)=>void){test(name,()=>{const x=actual();change?.(x);expect(accepts(x)).toBe(want);});} -check('counterfactual only substantive Questions corrected; initial selector exception',true); -check('actual five malformed saved Questions still reject',false,x=>{x.plan=capture.originalPlan;}); -for(const state of ['pending','rejected','withdrawn'])check('current state '+state,false,x=>{x.plan=x.plan.replace('State: approved','State: '+state);}); -check('duplicate current state',false,x=>{x.plan=x.plan.replace('State: approved','State: approved\nState: approved');}); -check('changed initial actual answer',false,x=>{x.calls[2]!.answers={[x.calls[2]!.questions[0]!.question]:x.calls[2]!.questions[0]!.options[1]!.label};}); -check('missing initial accepted scope',false,x=>{x.plan=x.plan.replace(/^Accepted scope:.*\n/m,'');}); -check('missing native ACK',false,x=>{x.calls[3]!.answered=false;}); -check('swapped approval pairs',false,x=>{x.plan=x.plan.replace('R1 (D3: A), R2 (D4: A)','R1 (D4: A), R2 (D3: A)');}); -check('foreign owned target',false,x=>{x.plan=x.plan.replace('Reviewed target: `PLAN.md`','Reviewed target: `OTHER.md`');}); -check('missing task',false,x=>{x.plan=x.plan.replace(/^- \[ \] \*\*T6[^\n]*\n/gm,'');}); -check('dependency inversion',false,x=>{x.plan=x.plan.replace('Lane E: W5 → W6','Lane E: W6 → W5');}); -check('unknown lane',false,x=>{x.plan=x.plan.replace('Lane E: W5 → W6','Lane E: W5 → W99');}); -test('actual owned five-to-three structure is a complexity decision',()=>{const c=actual().calls[3]!;expect(isEngSeedDecisionAUQ(nativePlanCallFingerprint(c,0))).toBe(true);}); -check('substantive Header is authoritative',false,x=>{x.plan=x.plan.replace('Header: Wiring','Header: Unrelated');}); -check('substantive option description cannot change cost',false,x=>{x.plan=x.plan.replace('human: ~2h / CC: ~5 min','human: ~20h / CC: ~50 min');}); -check('missing substantive Options',false,x=>{const at=x.plan.indexOf('Question D5:');x.plan=x.plan.slice(0,at)+x.plan.slice(at).replace('Options:','Choices:');}); -check('substantive question cannot claim initial selector exemption',false,x=>{x.plan=x.plan.replace('Question D5:\nD5 —','Question D5: scope only;\nD5 —');x.plan=x.plan.replace('Header: Wiring','Header: Scope');}); -check('foreign earlier citation',false,x=>{const c=x.calls[3]!,q=c.questions[0]!,a=c.answers![q.question]!;q.question=q.question.replaceAll('PLAN.md','other/PLAN.md');c.answers={[q.question]:a};}); -check('duplicate current ownership',false,x=>{x.plan+='\n'+x.plan.split('\n').find(l=>l.startsWith('Reviewed target:'))+'\n';}); -check('current withdrawn approval',false,x=>{x.plan=x.plan.replace('History: none.','R1 approval is withdrawn.\nHistory: none.');}); -check('offered new dependency',false,x=>{x.calls.at(-1)!.questions[0]!.options[0]!.description+=' Add a dependency to the cache.';}); -check('lane list presentation preserves graph',true,x=>{const c=x.calls.at(-1)!,q=c.questions[0]!;q.options[0]!.description=q.options[0]!.description!.replace('lanes A-D can start in parallel worktrees','lanes A/B/C/D in parallel');}); -function seed(name:string,want:boolean,mutate:(q:NativePlanQuestionCall['questions'][number])=>void){test(name,()=>{const c=actual().calls[3]!,q=c.questions[0]!,selected=c.answers![q.question]!;mutate(q);c.answers={[q.question]:selected};expect(isEngSeedDecisionAUQ(nativePlanCallFingerprint(c,0))).toBe(want);});} -seed('wrong declared inventory count',false,q=>{q.question=q.question.replace('all 5 new classes','all 6 new classes');}); -seed('wrong retained class count',false,q=>{q.options[0]!.description=q.options[0]!.description!.replace('AuthBroker, SessionMint, AuthCache as classes','AuthBroker, SessionMint, AuthCache, TokenStore as classes');}); -seed('foreign plan inventory',false,q=>{q.question=q.question.replaceAll('PLAN.md','other/PLAN.md');}); -seed('stateful policy correction',false,q=>{q.options[0]!.description+=' Correction: RequestPolicy remains a class with independent state.';}); -seed('retained independent token store correction',false,q=>{q.options[0]!.description+=' Correction: TokenStore remains a separate class.';}); -seed('conditional Cons do not withdraw offered structure',true,q=>{q.options[0]!.description+=' ❌ If RequestPolicy remains a class, this option’s contract has not been implemented.';}); -seed('missing pure-function contract',false,q=>{q.question=q.question.replace('RequestPolicy as a pure function','RequestPolicy as a class');}); -seed('cannot borrow policy conversion from another option',false,q=>{q.options[1]!.description+=' '+q.options[0]!.description;q.options[0]!.description=q.options[0]!.description!.replace('RequestPolicy becomes decideAccess(claims, ctx) in a policy module','RequestPolicy remains a class');}); -check('unpublished implementation command is not navigation',false,x=>{x.calls.at(-1)!.questions[0]!.options[0]!.description+=' Write Redis configuration.';}); -check('unapproved subprocess is not navigation',false,x=>{x.calls.at(-1)!.questions[0]!.options[0]!.description+=' Run the migration.';}); - -check('stale scope cannot revive the deferred rewrite',false,x=>{x.plan=x.plan.replace('Accepted scope: this PR does not modify `legacyAuthFlow()`; its rewrite/swap moves to a follow-up PR after the new services are exercised. Regression coverage remains a separate pending choice (R3).','Accepted scope: this PR rewrites `legacyAuthFlow()` now; the follow-up is cancelled.');}); -check('stale scope cannot retain removed class',false,x=>{x.plan=x.plan.replace('`TokenStore` not created','`TokenStore` created');}); -check('scope cannot change the selected pure policy contract',false,x=>{x.plan=x.plan.replace('`RequestPolicy` implemented as pure function','`RequestPolicy` implemented as stateful class');}); -for(const at of [3,10])check('explicit foreign repo in native metadata '+at,false,x=>{const c=x.calls[at]!,q=c.questions[0]!,answer=c.answers![q.question]!;q.question=q.question.replace(/^(Project\/branch\/task:.*)$/m,'$1; repo attacker/app');c.answers={[q.question]:answer};}); -check('owned repo prefix cannot authorize another repository',false,x=>{const c=x.calls.at(-1)!,q=c.questions[0]!,answer=c.answers![q.question]!;q.question=q.question.replace(/^(Project\/branch\/task:.*)$/m,'$1; repo gstack-plan-count-eecImF/other');c.answers={[q.question]:answer};}); -check('task self reference cannot supply graph module coverage',false,x=>{const at=x.plan.indexOf('## Worktree parallelization strategy');x.plan=x.plan.slice(0,at)+x.plan.slice(at).replace('| auth/cache |','| foreign/cache |');}); -check('scope selector cannot approve separate remedy through selected option',false,x=>{x.calls[3]!.questions[0]!.options[0]!.description+=' ✅ Approve the cache invalidation remedy now.';}); -seed('current mutable policy contradicts pure function',false,q=>{q.options[0]!.description+=' Correction: RequestPolicy is stateful and stores mutable tenant state.';}); -seed('current owned token persistence contradicts removal',false,q=>{q.options[0]!.description+=' Correction: TokenStore keeps refresh-token persistence in its own class.';}); -seed('explicit non-stateful statement preserves pure function',true,q=>{q.options[0]!.description+=' RequestPolicy is not stateful.';}); -check('declarative selected approval is still a separate remedy',false,x=>{x.calls[3]!.questions[0]!.options[0]!.description+=' ✅ This option approves the cache invalidation remedy now.';}); -check('negative selected approval does not suppress affirmative contrast',false,x=>{x.calls[3]!.questions[0]!.options[0]!.description+=' ✅ This option does not approve the regression remedy but approves the invalidation remedy now.';}); -check('explicit negative approval preserves structure-only choice',true,x=>{x.calls[3]!.questions[0]!.options[0]!.description+=' ✅ This option does not approve the invalidation remedy.';}); -check('conditional approval risk preserves structure-only choice',true,x=>{x.calls[3]!.questions[0]!.options[0]!.description+=' ❌ If this option approves the invalidation remedy, the structure-only contract was violated.';}); -seed('negative class identity cannot suppress affirmative mutable state',false,q=>{q.options[0]!.description+=' Correction: RequestPolicy is not a separate class but stores mutable tenant state.';}); -seed('negative state statements stay negative across contrast',true,q=>{q.options[0]!.description+=' Correction: RequestPolicy is not a class and stores no mutable tenant state.';}); -seed('affirmative state before negative contrast remains contradictory',false,q=>{q.options[0]!.description+=' Correction: RequestPolicy stores mutable tenant state but is not a separate class.';}); diff --git a/test/eng-legacy-contract-am.test.ts b/test/eng-legacy-contract-am.test.ts deleted file mode 100644 index 2bcdf3bec..000000000 --- a/test/eng-legacy-contract-am.test.ts +++ /dev/null @@ -1,64 +0,0 @@ -import {test,expect} from 'bun:test'; -import {evaluateEngSeedCoverage} from './helpers/eng-seeded-coverage'; -import fixture from './fixtures/eng-legacy-contract-am.json'; -const transcript:any={status:'ready',calls:fixture.calls,assistantMessages:[],planReadyRequests:[]}; -const times=fixture.calls.map(c=>Date.parse(c.answeredAt)); -const check=(plan=fixture.compact,calls=transcript.calls)=>evaluateEngSeedCoverage({...transcript,calls},plan,Math.min(...times)-1,Math.max(...times)+1); -test('the actual class inventory decision is a distinct complexity seed',()=>expect(check().missing).toEqual([])); -test('the actual mandatory current-output suite and linked before-rewrite task establish legacy parity',()=>expect(check().problems).toEqual([])); -test('the unchanged legacy function and exact output oracle remain required',()=>{ - expect(check(fixture.compact.replaceAll('legacyAuthFlow','anotherFlow')).regression).toBeUndefined(); - expect(check(fixture.compact.replace('record current outputs','record proposed outputs')).regression).toBeUndefined(); - expect(check(fixture.compact.replace('produces identical decisions and equivalent error surfaces','may produce different decisions and error surfaces')).regression).toBeUndefined(); -}); -const no:Array<[string,(s:string)=>string]>=[ - ['historical source ancestor',s=>'# Source\n'+s], - ['quoted declaration',s=>s.replace(fixture.declaration,fixture.declaration.split('\n').map(l=>'> '+l).join('\n'))], - ['fenced declaration',s=>s.replace(fixture.declaration,'```text\n'+fixture.declaration+'```\n')], - ['optional declaration',s=>s.replace('REGRESSION (mandatory,','REGRESSION (optional,')], - ['source declaration prefix',s=>s.replace('`legacyAuthFlow()` is existing','The following is a quoted source excerpt.\n`legacyAuthFlow()` is existing')], - ['hypothetical declaration prefix',s=>s.replace('`legacyAuthFlow()` is existing','If approved:\n`legacyAuthFlow()` is existing')], - ['after-rewrite capture',s=>s.replace('written BEFORE any rewrite','written AFTER any rewrite')], - ['foreign declared task',s=>s.replace('rewrite (T1)','rewrite (T9)')], - ['foreign task file',s=>s.replace('Files: `auth/legacyAuthFlow.characterization.test.ts`','Files: `auth/other.characterization.test.ts`')], - ['proposed-only task owner',s=>s.replace('## Implementation Tasks','## Proposed Implementation Tasks')], - ['task after rewrite',s=>s.replace('for `legacyAuthFlow()` before any rewrite','for `legacyAuthFlow()` after any rewrite')], - ['missing current baseline',s=>s.replace('suite green on current main','suite green on the new implementation')], - ['missing later rerun',s=>s.replace('; re-run after each later task','; no later runs needed')], - ['conditional verification',s=>s.replace(' - Verify:',' If approved:\n - Verify:')], - ['source verification',s=>s.replace(' - Verify:',' Source excerpt:\n - Verify:')], - ['withdrawn task',s=>s+'\n## Final assessment\nT1 is withdrawn.\n'], - ['withdrawn rerun',s=>s+'\n## Final assessment\nT1 rerun is cancelled.\n'], - ['withdrawn suite',s=>s+'\n## Final assessment\nThe legacy regression suite is withdrawn.\n'], - ['withdrawn baseline verification',s=>s.replace(' - Verify:',' Correction: this baseline verification is withdrawn.\n - Verify:')], -]; -test.each(no)('%s cannot supply the required unchanged legacy oracle',(_,change)=>expect(check(change(fixture.compact)).regression).toBeUndefined()); -test('same file/task identity and harmless unrelated context are preserved',()=>{ - expect(check(fixture.compact.replaceAll('T1','T9').replaceAll('legacyAuthFlow.characterization.test.ts','legacy-behavior.test.ts')).ok).toBe(true); - expect(check(fixture.compact+'\n## Payment regression suite\nThis regression suite is withdrawn.\n').ok).toBe(true); - expect(check(fixture.compact.replace(' - Verify:',' Literal UI label: "This is a hypothetical example."\n - Verify:')).ok).toBe(true); -}); -test('a class-name or historical example cannot replace the class-inventory scope decision',()=>{ - for(const title of ['D1 — Rename the class before building?','Historical example: Reduce the class inventory before building?','D1 — A hypothetical example: reduce the class inventory before building?']){ - const calls=structuredClone(transcript.calls);const q=calls[0].questions[0];const selected=calls[0].answers[q.question];q.question=q.question.replace(/^.*\n/,title+'\n');calls[0].answers={[q.question]:selected};expect(check(fixture.compact,calls).missing).toContain('complexity'); - } -}); - -test('explicit current baseline changes and named verification withdrawal cancel this oracle',()=>{ - for(const suffix of [ - '## Current baseline correction\nlegacyAuthFlow() is modified before T1 records the baseline.', - '## Final verification assessment\nT1 verification is withdrawn.', - '## Final verification assessment\nT1 verification is "withdrawn".', - ])expect(check(fixture.compact+'\n'+suffix).regression).toBeUndefined(); - expect(check(fixture.compact+'\n## History\nOld note: "legacyAuthFlow() is modified before T1 records the baseline."').regression).toBeDefined(); -}); -test('current class inventory ownership excludes literal, source-only and withdrawn actions',()=>{ - const changes=[ - (q:any)=>{q.question=q.question.replace(/^(D1 — )(.*)\n/,'$1`$2`\n')}, - (q:any)=>{q.options=q.options.map((o:any)=>({label:'Quoted source: '+o.label,description:'Source excerpt: '+o.description}))}, - (q:any)=>{q.question+='\nCorrection: this class-inventory decision is withdrawn.'}, - (q:any)=>{q.question+='\nCorrection: this class-inventory decision is "withdrawn".'}, - ]; - for(const change of changes){const calls=structuredClone(transcript.calls),c=calls[0],q=c.questions[0],selected=q.options.findIndex((o:any)=>o.label===c.answers[q.question]);change(q);c.answers={[q.question]:q.options[selected].label};expect(check(fixture.compact,calls).missing).toContain('complexity');} - const calls=structuredClone(transcript.calls),c=calls[0],q=c.questions[0],selected=c.answers[q.question];q.question+='\nOld note: "This class-inventory decision is withdrawn."';c.answers={[q.question]:selected};expect(check(fixture.compact,calls).missing).not.toContain('complexity'); -}); diff --git a/test/eng-mandatory-baseline-as.test.ts b/test/eng-mandatory-baseline-as.test.ts deleted file mode 100644 index e85973db3..000000000 --- a/test/eng-mandatory-baseline-as.test.ts +++ /dev/null @@ -1,93 +0,0 @@ -import { describe, expect, test } from 'bun:test'; -import { readFileSync } from 'node:fs'; -import { createHash } from 'node:crypto'; -import { evaluateEngSeedCoverage } from './helpers/eng-seeded-coverage'; -import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; - -// Exact public Write acknowledged in the first AS attempt. The paid failure -// stays failed; this fixture verifies only the report's mandatory baseline. -const report = readFileSync(new URL('./fixtures/eng-mandatory-baseline-as.md', import.meta.url), 'utf8'); -const declaration = report.match(/^### CRITICAL — regression \(mandatory, REGRESSION RULE\)\n[\s\S]*?(?=\n### )/m)![0]; -const task = report.match(/^- \[ \] \*\*T5 .*\n(?: .*(?:\n|$))*/m)![0]; -const compact = '# Current reviewed plan\n\n## Tests\n\n' + declaration + '\n## Implementation Tasks\n' + task; -const check = (text: string) => evaluateEngSeedCoverage({ status: 'ready', calls: [], assistantMessages: [] }, text, 0, 1); - -const negative: Array<[string, (s: string) => string]> = [ - ['missing declaration', s => s.replace(declaration, '')], - ['optional heading', s => s.replace('mandatory, REGRESSION RULE', 'optional, REGRESSION RULE')], - ['historical owner', s => s.replace('## Tests', '## Historical tests')], - ['source ancestor', s => '# Source excerpt\n' + s.replace('# Current reviewed plan\n', '')], - ['bare source introduction', s => 'Source:\n\n' + s.replace('# Current reviewed plan\n', '')], - ['conditional declaration', s => s.replace('`legacyAuthFlow()` is', 'If approved, `legacyAuthFlow()` is')], - ['source declaration', s => s.replace('`legacyAuthFlow()` is', 'Source:\n`legacyAuthFlow()` is')], - ['quoted declaration', s => s.replace(declaration, declaration.split('\n').map(l => '> ' + l).join('\n'))], - ['fenced declaration', s => s.replace(declaration, '```\n' + declaration + '\n```')], - ['literal declaration', s => s.replace(declaration, declaration.replace(/`/g, '').split('\n').map(l => '`' + l + '`').join('\n'))], - ['quoted declaration sentence', s => s.replace('Before any rewrite:', '"Before any rewrite:').replace('inputs. This suite', 'inputs." This suite')], - ['wrong legacy target', s => s.replaceAll('legacyAuthFlow', 'otherAuthFlow')], - ['capture after rewrite', s => s.replace('Before any rewrite:', 'After the rewrite:')], - ['proposed outputs', s => s.replace('records current', 'records proposed')], - ['new path only', s => s.replace('against the legacy path now', 'against the new path now')], - ['optional assertion', s => s.replace('records current', 'may record current')], - ['missing baseline task', s => s.replace(task, '')], - ['historical task owner', s => s.replace('## Implementation Tasks', '## Historical Implementation Tasks')], - ['conditional task', s => s.replace(task, 'If approved:\n' + task)], - ['source task', s => s.replace(task, 'Source:\n' + task)], - ['quoted task', s => s.replace(task, task.split('\n').map(l => '> ' + l).join('\n'))], - ['wrong file', s => s.replace(' - Files: tests/auth/legacyAuthFlow.characterization.test.ts', ' - Files: tests/auth/other.test.ts')], - ['missing same-file binding', s => s.replace(' - Files: tests/auth/legacyAuthFlow.characterization.test.ts\n', '')], - ['wrong task subject', s => s.replace('suite for `legacyAuthFlow()` current behavior', 'suite for `otherAuthFlow()` current behavior')], - ['missing verification', s => s.replace(' - Verify: suite green against unmodified legacy before any other task merges', '')], - ['changed baseline', s => s.replace('against unmodified legacy', 'against modified legacy')], - ['baseline after merge', s => s.replace('before any other task merges', 'after every other task merges')], - ['missing before-merge gate', s => s.replace(' before any other task merges', '')], - ['neighboring verification', s => s.replace(' - Verify:', '- [ ] T6 — tests/auth — Another suite\n - Verify:')], - ['duplicate task identity', s => s.replace(task, task + task)], - ...['Source:', 'If approved:', 'Assuming approval,', 'Provided approval,', 'Once approved:', 'When approved:', 'Pending approval:'].map(prefix => - [`verification owner ${prefix}`, (s: string) => s.replace(' - Verify:', ` ${prefix}\n - Verify:`)] as [string, (s: string) => string]), - ...['withdrawn', 'superseded', 'optional', 'not current', 'no longer current', 'no longer required'].flatMap(status => [ - [`current T5 ${status}`, (s: string) => s + `\n## Current assessment\nT5 is ${status}.\n`], - [`quoted T5 ${status}`, (s: string) => s + `\n## Current assessment\nT5 is "${status}".\n`], - [`baseline ${status}`, (s: string) => s.replace(task, task + ` This baseline verification is "${status}".\n`)], - ] as Array<[string, (s: string) => string]>), - ['current status row', s => s + '\n## Current assessment\n| T5 | Withdrawn |\n'], - ['quoted status row', s => s + '\n## Current assessment\n| T5 | "Withdrawn" |\n'], - ['withdrawn legacy suite', s => s + '\n## Current assessment\nThe legacy characterization suite is "withdrawn".\n'], - ['declaration withdrawn', s => s.replace(declaration, declaration + '\nThis suite is withdrawn.\n')], - ['baseline changed before task', s => s + '\n## Current assessment\nlegacyAuthFlow() is modified before T5.\n'], -]; - -describe('mandatory legacy baseline before any other task merges', () => { - test('the exact acknowledged report requires the baseline without inventing native decisions', () => { - expect(createHash('sha256').update(report).digest('hex')).toBe('60620ddd798a567423087731775557848c260a82080e9eceb1367a9bb9fc5d23'); - expect(check(report)).toMatchObject({ regression: 'plan', ok: false, - missing: ['complexity', 'shared-cache', 'swallowed-errors', 'sequential-idp'] }); - expect(check(compact).regression).toBe('plan'); - }); - test('task numbering, test paths, markup and line wrapping do not change the obligation', () => { - for (const altered of [compact.replaceAll('T5', 'T31'), compact.replaceAll('tests/auth', 'test/login'), - compact.replaceAll('legacyAuthFlow.characterization.test.ts', 'prior-behavior.test.js'), - compact.replace(/[`*]/g, ''), compact.replace(/\n(?=[a-z])/g, ' ')]) - expect(check(altered).regression).toBe('plan'); - }); - test('this baseline does not require an invented same-suite flag-on rerun', () => { - const baselineOnly = compact.replace(/ and moves to\n`AuthBroker` when the flag is removed \(TODO 1\)/, ''); - expect(baselineOnly).not.toBe(compact); - expect(check(baselineOnly).regression).toBe('plan'); - }); - test('historical quotations, unrelated suite statuses and future completion do not withdraw the baseline', () => { - for (const addition of ['\n## History\n"T5 is withdrawn."', '\n## History\n> T5 is withdrawn.', - '\n## Historical task status\n| T5 | Withdrawn |', '\n## Current assessment\n| T9 | Withdrawn |', - '\n## Payment regression suite\nThe regression suite is withdrawn.', - '\n## Current assessment\nIf T5 is withdrawn, reopen the rollout decision.']) - expect(check(compact + addition).regression).toBe('plan'); - expect(check('Source:\n\n' + compact).regression).toBe('plan'); - }); - test.each(negative)('%s supplies no mandatory baseline', (_, change) => { - const altered = change(compact); expect(altered).not.toBe(compact); expect(check(altered).regression).toBeUndefined(); - }); - test('new regression artifacts select only the existing Eng owner', () => { - for (const file of ['test/eng-mandatory-baseline-as.test.ts', 'test/fixtures/eng-mandatory-baseline-as.md']) - expect(selectTests([file], E2E_TOUCHFILES, []).selected).toEqual(['plan-eng-finding-count']); - }); -}); diff --git a/test/eng-native-seed-contract.test.ts b/test/eng-native-seed-contract.test.ts deleted file mode 100644 index 7d902d2f1..000000000 --- a/test/eng-native-seed-contract.test.ts +++ /dev/null @@ -1,915 +0,0 @@ -import {expect, test} from 'bun:test'; -import captured from './fixtures/eng-native-seed-contract-6f.json'; -import goldenDeclaration from './fixtures/eng-legacy-declaration-90f.json'; -import idpChoice from './fixtures/eng-idp-choice-90f.json'; -import {evaluateEngSeedCoverage, isEngSeedDecisionAUQ} from './helpers/eng-seeded-coverage'; -import {isEngCompletionHandoff} from './helpers/eng-completion-handoff'; -import structureChoice from './fixtures/eng-structure-choice-90f.json'; -import nativePackets from './fixtures/eng-native-packets-b955.json'; - -const held6bd=nativePackets.held6bd; -const heldStructure=()=>structuredClone(held6bd.transcript.calls.find(c=>c.toolUseId==='toolu_01C1daapitaDzziNHqrVQ9qb')!); -const heldStructureResult=(c=heldStructure())=>evaluateEngSeedCoverage({status:'ready',calls:[c],assistantMessages:[]},'',held6bd.startedAt,held6bd.finishedAt).decisions; -const changeHeldStructure=(edit:(q:any)=>void)=>{const c=heldStructure(),q=c.questions[0]!,chosen=q.options.findIndex(o=>o.label===c.answers[q.question]);edit(q);c.answers={[q.question]:q.options[chosen]!.label};return c;}; -test('held6bd structure: actual independently answered four-to-three store consolidation',()=>expect(heldStructureResult()).toEqual({complexity:'8351cb8b-b2d3-424a-8420-137a5ea5be83:toolu_01C1daapitaDzziNHqrVQ9qb'})); -for(const [name,edit] of Object.entries({ - 'same components named as classes':(q:any)=>{q.question=q.question.replace('four things:','four classes:');q.options.forEach((o:any)=>o.label=o.label.replace('components','classes'));}, - 'explicit duplicate responsibility':(q:any)=>{q.question=q.question.replace('TokenStore is never described, and its name says it does what the adapter already does.','TokenStore has no documented purpose. Its name duplicates the existing adapter\'s job.');}, - 'same owned facade responsibility':(q:any)=>{q.options[0].description=q.options[0].description.replace('Exactly one place owns tenant-key construction and invalidation calls on top of the existing adapter','One facade owns tenant-key construction and invalidation over the existing adapter');}, -}))test('held6bd structure class accepts '+name,()=>expect(heldStructureResult(changeHeldStructure(edit)).complexity).toBeDefined()); -for(const [name,edit] of Object.entries({ - 'wrong fold destination':(q:any)=>{q.options[0].label=q.options[0].label.replace('into AuthCache','into OtherCache');}, - 'two different current inventories':(q:any)=>{q.question=q.question.replace('Stakes if','ELI10: The plan adds three components: AuthBroker, SessionMint, AuthCache.\nStakes if');}, - 'current duplicate claim retracted':(q:any)=>{q.question+='\nCorrection: TokenStore does not duplicate the adapter.';}, - 'current responsibility independent':(q:any)=>{q.question+='\nCorrection: TokenStore has a documented independent contract.';}, - 'new independent work':(q:any)=>{q.options[0].description+=' Also add Redis.';}, - 'subordinate new work':(q:any)=>{q.options[0].description+=' Fold TokenStore while disabling tenant validation.';}, - 'same-option negated owner':(q:any)=>{q.options[0].description+=' No single facade owns invalidation.';}, -}))test('held6bd structure class rejects '+name,()=>expect(heldStructureResult(changeHeldStructure(edit))).toEqual({})); - -test('held6bd historical native/seed gates and current report-bottom assertions',()=>{ - const h=structuredClone(held6bd),t=h.transcript as PlanCountTranscript; - const source=fs.readFileSync(path.join(import.meta.dir,'skill-e2e-plan-eng-finding-count.test.ts'),'utf8'); - const start=source.indexOf(" if (!['plan_ready', 'completion_summary'].includes(obs.outcome))"),end=source.indexOf(' // A native completion summary',start); - expect(start).toBeGreaterThan(0);expect(end).toBeGreaterThan(start); - const validate=new Function('fs','planPath','obs','assertReviewReportAtBottom',new Bun.Transpiler({loader:'ts'}).transformSync(source.slice(start,end))); - const dir=fs.mkdtempSync(path.join(os.tmpdir(),'eng-held-captured-')),file=path.join(dir,'report.md'),now=Date.now; - try{ - Date.now=()=>h.finishedAt; - // Only this synthetic file receives the captured timestamp. The owned paid - // report and its cancelled outcome remain immutable. - const write=(body=h.plan)=>{fs.writeFileSync(file,body);fs.utimesSync(file,h.reportMtimeMs/1000,h.reportMtimeMs/1000);};write(); - const admin=new Set();let reviews=0; - t.calls.forEach((call,index)=>{const fp=nativePlanCallFingerprint(call,0,false);if(isEngCompletionHandoff(fp,h.plan,t.calls.slice(0,index)))admin.add(fp.signature);if(isEngSeedDecisionAUQ(fp,t.calls.slice(0,index),h.startedAt,h.finishedAt))reviews++;}); - expect([...admin]).toEqual(['8351cb8b-b2d3-424a-8420-137a5ea5be83:toolu_01Jzx8JVh6SV9sLe9RgMP7GF']);expect(reviews).toBe(4); - expect(hasNativePlanTerminal(t,file,h.startedAt,'plan_ready',admin)).toBe(true); - expect(hasNativePlanTerminal(t,file,h.startedAt,'plan_ready',new Set())).toBe(false); - const obs={outcome:'plan_ready',transcript:t,reviewCount:reviews,step0Count:0,fingerprints:[],elapsedMs:h.finishedAt-h.startedAt,evidence:'Captured public native ExitPlanMode'}; - // The original deterministic export remains a historical compatibility - // check. New semantic acceptance is tested through the actual registered - // PTY path in eng-semantic-terminal; these old reports receive no new credit. - const check=(input=obs)=>{ - validate(fs,file,input,assertReviewReportAtBottom); - if(!evaluateEngSeedCoverage(input.transcript,fs.readFileSync(file,'utf8'),h.startedAt,h.finishedAt).ok) throw Error('SEED COVERAGE FAIL'); - }; - expect(()=>check()).not.toThrow(); - expect(()=>check({...obs,outcome:'cancelled'})).toThrow('finding-count FAILED'); - for(const mutate of [ - (copy:PlanCountTranscript)=>{copy.planReadyRequests=[];}, - (copy:PlanCountTranscript)=>{copy.planReadyRequests![0]!.failed=true;}, - (copy:PlanCountTranscript)=>{copy.calls[6]!.answeredAt=t.calls.at(-1)!.answeredAt;}, - (copy:PlanCountTranscript)=>{copy.calls[6]!.answered=false;copy.calls[6]!.unansweredQuestionIndices=[0];}, - ]){const copy=structuredClone(t);mutate(copy);expect(hasNativePlanTerminal(copy,file,h.startedAt,'plan_ready',admin)).toBe(false);} - const missing=structuredClone(obs);missing.transcript.calls=missing.transcript.calls.filter(c=>c.toolUseId!=='toolu_01C1daapitaDzziNHqrVQ9qb');expect(()=>check(missing)).toThrow('SEED COVERAGE FAIL'); - write(h.plan.replace('suite green on unmodified legacy body','suite red on unmodified legacy body'));expect(()=>check()).toThrow('SEED COVERAGE FAIL'); - write(h.plan+'\n## Unreviewed work\n');expect(()=>check()).toThrow('D19 FAIL'); - expect(h.actualOutcome).toBe('cancelled_no_pass_or_failure_credit'); - }finally{Date.now=now;fs.rmSync(dir,{recursive:true,force:true});} -}); -for(const [name,edit] of Object.entries({ - 'numeric inventory':(q:any)=>{q.question=q.question.replace('four things:','4 components:');}, - 'independent title':(q:any)=>{q.question=q.question.replace(q.question.split('\n')[0],'D23 — Which component arrangement should the token layer use?');}, - 'reordered options':(q:any)=>{q.options.reverse();}, - 'historical contradiction':(q:any)=>{q.question+='\nEarlier note: "TokenStore now has an independent purpose."';}, -}))test('held6bd structure accepts '+name,()=>expect(heldStructureResult(changeHeldStructure(edit)).complexity).toBeDefined()); -for(const [name,edit] of Object.entries({ - 'foreign source':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','OTHER.md');}, - 'same-basename foreign source':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','archive/PLAN.md');}, - 'duplicated source':(q:any)=>{q.question+='\nProject/branch/task: OTHER.md';}, - 'quoted explanation':(q:any)=>{q.question=q.question.replace(/^ELI10: (.+)$/m,'ELI10: "$1"');}, - 'conditional evidence':(q:any)=>{q.question=q.question.replace('ELI10: ','ELI10: If approved, ');}, - 'withdrawn choice':(q:any)=>{q.question+='\nThis decision is withdrawn.';}, - 'reopened choice':(q:any)=>{q.question+='\nThis decision is "reopened".';}, - 'false count':(q:any)=>{q.question=q.question.replace('four things:','five things:');}, - 'duplicate inventory':(q:any)=>{q.question=q.question.replace('AuthBroker, SessionMint, AuthCache, and TokenStore','AuthBroker, SessionMint, AuthCache, and AuthCache');}, - 'foreign retained service':(q:any)=>{q.options[0].label=q.options[0].label.replace('SessionMint','OtherService');}, - 'equal counts':(q:any)=>{q.options[0].label=q.options[0].label.replace('3 components:','4 components:');}, - 'no keep alternative':(q:any)=>{q.options[1]={label:'Discuss storage',description:'No arrangement.'};}, - 'no fold':(q:any)=>{q.options[0].label=q.options[0].label.replace('(fold TokenStore into AuthCache)','');}, - 'remedy borrowed':(q:any)=>{q.options[1].description+=' '+q.options[0].description;q.options[0].description='Undecided storage.';}, - 'quoted remedy':(q:any)=>{q.options[0].description='"'+q.options[0].description+'"';}, - 'independent current store':(q:any)=>{q.question+='\nCorrection: TokenStore now has a documented independent purpose.';}, - 'store retained':(q:any)=>{q.options[0].description+=' But keep TokenStore as a separate class.';}, - 'negated fold':(q:any)=>{q.options[0].description+=' Do not fold TokenStore.';}, - 'declarative negated fold':(q:any)=>{q.options[0].description+=' This option never folds TokenStore into AuthCache.';}, - 'adapter replaced':(q:any)=>{q.options[0].description+=' Replace the existing adapter.';}, - 'foreign remedy':(q:any)=>{q.options[0].description+=' This remedy applies to another project.';}, -}))test('held6bd structure rejects '+name,()=>expect(heldStructureResult(changeHeldStructure(edit))).toEqual({})); -import fs from 'node:fs'; -import path from 'node:path'; -import os from 'node:os'; -import {createHash} from 'node:crypto'; -import {nativePlanCallFingerprint, assertReviewReportAtBottom, classifyPlanCountFrame, hasNativePlanTerminal, isQuestionlessNativePlanExit} from './helpers/claude-pty-runner'; -import type {NativePlanQuestionCall, PlanCountTranscript} from './helpers/plan-count-transcript'; - -const transcript=()=>structuredClone(captured.transcript) as PlanCountTranscript; -const evaluate=(calls=transcript().calls, plan=captured.report)=>evaluateEngSeedCoverage( - {...transcript(),calls,assistantMessages:[]},plan,captured.startedAt,captured.finishedAt); -const seeds=[[4,'complexity'],[5,'shared-cache'],[7,'swallowed-errors'],[9,'sequential-idp']] as const; - -test('current counted alternatives own a complexity reduction without borrowing the preceding fold',()=>{ - const call=structuredClone(structureChoice.calls[1]) as NativePlanQuestionCall; - const result=evaluateEngSeedCoverage({status:'ready',calls:[call],assistantMessages:[]},'',0,Date.parse(structureChoice.captureAt)); - expect(result.decisions).toEqual({complexity:`${call.sessionId}:${call.toolUseId}`}); -}); - -const structureCall=()=>structuredClone(structureChoice.calls[1]) as NativePlanQuestionCall; -const structureResult=(call=structureCall())=>evaluateEngSeedCoverage({status:'ready',calls:[call],assistantMessages:[]},'',0,Date.parse(structureChoice.captureAt)); -const alterStructure=(edit:(q:NativePlanQuestionCall['questions'][number])=>void)=>{ - const call=structureCall();edit(call.questions[0]!); - call.answers={[call.questions[0]!.question]:call.questions[0]!.options[0]!.label};return call; -}; -for(const [name,edit] of Object.entries({ - 'renamed title':(q:any)=>{q.question=q.question.replace('Which class/module arrangement for the remaining new units?','Which structure should the remaining components use?');}, - 'classes instead of units':(q:any)=>{q.options.forEach((o:any)=>{o.label=o.label.replace(' units:',' classes:');});}, - 'reordered inventory':(q:any)=>{q.question=q.question.replace('AuthBroker, SessionMint, AuthCache and RequestPolicy','RequestPolicy, AuthCache, AuthBroker and SessionMint');}, - 'reordered choices':(q:any)=>{q.options.reverse();}, - 'word counts':(q:any)=>{q.options[0].label=q.options[0].label.replace('3 units:','Three components:');q.options[1].label=q.options[1].label.replace('4 units:','Four components:');}, - 'quoted old withdrawal':(q:any)=>{q.question+='\nEarlier note: "D6 is reopened."';}, - 'prior fold omitted':(q:any)=>{q.question=q.question.replace('after D4 (strangler) and D5 (TokenStore folded), ','').replace('drops the new-unit count from 5 to 3','drops the remaining class count from 4 to 3');}, -}))test('current structure comparison accepts '+name,()=>expect(structureResult(alterStructure(edit)).decisions.complexity).toBeDefined()); -for(const [name,edit] of Object.entries({ - 'foreign plan':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','OTHER.md');}, - 'foreign plan directory':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','archive/PLAN.md');}, - 'quoted current source':(q:any)=>{q.question=q.question.replace(/^Project\/branch\/task: (.+)$/m,'Project/branch/task: "$1"');}, - 'historical source':(q:any)=>{q.question=q.question.replace('Project/branch/task: ','Project/branch/task: Historical example: ');}, - 'quoted explanation':(q:any)=>{q.question=q.question.replace(/^ELI10: (.+)$/m,'ELI10: "$1"');}, - 'missing current inventory':(q:any)=>{q.question=q.question.replace('AuthBroker, SessionMint, AuthCache and RequestPolicy','the previous classes');}, - 'foreign current component':(q:any)=>{q.question=q.question.replace('AuthCache and RequestPolicy','OtherCache and RequestPolicy');}, - 'duplicated current component':(q:any)=>{q.question=q.question.replace('AuthBroker, SessionMint, AuthCache and RequestPolicy','AuthBroker, SessionMint, AuthBroker and RequestPolicy');}, - 'no current lifecycle defect':(q:any)=>{q.question=q.question.replace('so a class adds ceremony without adding safety','so either approach is equally necessary');}, - 'independent current lifecycle':(q:any)=>{q.question+='\nCorrection: RequestPolicy now requires an independent lifecycle.';}, - 'missing baseline option':(q:any)=>{q.options[1].label='Discuss the arrangement';}, - 'reversed option counts':(q:any)=>{q.options[0].label=q.options[0].label.replace('3 units:','4 units:');q.options[1].label=q.options[1].label.replace('4 units:','3 units:');}, - 'equal option counts':(q:any)=>{q.options[0].label=q.options[0].label.replace('3 units:','4 units:');}, - 'duplicate reduced inventory':(q:any)=>{q.options[0].label=q.options[0].label.replace('AuthBroker, SessionMint, AuthCache','AuthBroker, SessionMint, AuthBroker');}, - 'foreign reduced component':(q:any)=>{q.options[0].label=q.options[0].label.replace('AuthCache','OtherCache');}, - 'unrelated removed component':(q:any)=>{q.options[0].label=q.options[0].label.replace('AuthBroker, SessionMint, AuthCache','RequestPolicy, SessionMint, AuthCache');}, - 'pure function borrowed from other option':(q:any)=>{q.options[1].description+=' Pure function.';q.options[0].description=q.options[0].description.replace('pure function','method');}, - 'pure function borrowed from question':(q:any)=>{q.question+='\nNet: use a pure function.';q.options[0].description=q.options[0].description.replace('pure function','method');}, - 'quoted reduced remedy':(q:any)=>{q.options[0].label='"'+q.options[0].label+'"';q.options[0].description='"'+q.options[0].description.replaceAll('\n',' ')+'"';}, - 'negated conversion':(q:any)=>{q.options[0].description+='\nDo not convert RequestPolicy.';}, - 'retained lifecycle':(q:any)=>{q.options[0].description+='\nRequestPolicy still retains its lifecycle.';}, - 'retained class':(q:any)=>{q.options[0].description+='\nRequestPolicy is still a class.';}, - 'mutable result':(q:any)=>{q.options[0].description+='\nThe result is not an immutable type.';}, - 'deferred remedy':(q:any)=>{q.options[0].description+='\nThis remedy is deferred.';}, - 'quoted deferred remedy':(q:any)=>{q.options[0].description+='\nThis remedy is "deferred".';}, - 'reopened decision':(q:any)=>{q.question+='\nD6 is reopened.';}, - 'quoted reopened decision':(q:any)=>{q.question+='\nD6 is "reopened".';}, - 'withdrawn decision':(q:any)=>{q.question+='\nThis decision is withdrawn.';}, - 'conditional decision':(q:any)=>{q.question=q.question.replace('ELI10: ','ELI10: If approved, ');}, -}))test('current structure comparison rejects '+name,()=>expect(structureResult(alterStructure(edit)).decisions).toEqual({})); -test('structure decision needs its own current native completion and stable guard identity',()=>{ - const call=structureCall(),finished=Date.parse(structureChoice.captureAt); - const guard=(c=call,prior:NativePlanQuestionCall[]=[])=>isEngSeedDecisionAUQ(nativePlanCallFingerprint(c,0,true),prior,0,finished); - expect(guard()).toBe(true); - expect(guard(call,[structuredClone(structureChoice.calls[0]) as NativePlanQuestionCall])).toBe(true); - expect(guard(call,[call])).toBe(false); - for(const edit of [(c:any)=>{c.answered=false;},(c:any)=>{c.failed=true;},(c:any)=>{c.answers={};},(c:any)=>{c.unansweredQuestionIndices=[0];},(c:any)=>{c.answeredAt=new Date(finished+1).toISOString();}]){ - const invalid=structureCall();edit(invalid);expect(guard(invalid)).toBe(false);expect(structureResult(invalid).decisions).toEqual({}); - } - const alien=structuredClone(structureChoice.calls[0]) as NativePlanQuestionCall;alien.sessionId+='-foreign';expect(guard(call,[alien])).toBe(false); - const both=structureCall();both.questions.push(transcript().calls[5]!.questions[0]!);both.answers={...both.answers,...transcript().calls[5]!.answers}; - expect(guard(both)).toBe(false);expect(structureResult(both).decisions).toEqual({}); -}); -const idpCall=()=>structuredClone(idpChoice.call) as NativePlanQuestionCall; -const idpResult=(call=idpCall())=>evaluateEngSeedCoverage({status:'ready',calls:[call],assistantMessages:[]},'',0,Date.parse(idpChoice.captureAt)); -const alterIdp=(edit:(q:NativePlanQuestionCall['questions'][number])=>void)=>{ - const call=idpCall();edit(call.questions[0]!); - call.answers={[call.questions[0]!.question]:call.questions[0]!.options[0]!.label};return call; -}; -test('IDP choice owns its current sequential defect and concurrent bounded remedy',()=>{ - const call=idpCall();expect(idpResult(call).decisions).toEqual({'sequential-idp':`${call.sessionId}:${call.toolUseId}`}); -}); -for(const [name,edit] of Object.entries({ - 'numeric count':(q:any)=>{q.question=q.question.replaceAll('five','5');}, - 'current ordering vocabulary':(q:any)=>{q.question=q.question.replace('Today the five checks run one after another','Currently the five calls run sequentially');}, - 'plain Promise.all with same timeout':(q:any)=>{q.options[0].label=q.options[0].label.replace('Promise.allSettled','Promise.all');}, - 'reordered choices':(q:any)=>{q.options.reverse();}, - 'timeout in same description':(q:any)=>{q.options[0].description+=' Every call has a per-call timeout of 2000 ms.';q.options[0].label=q.options[0].label.replace(' + per-call timeout (default 2000 ms)','');}, - 'quoted earlier cancellation':(q:any)=>{q.question+='\nEarlier note: "D13 is deferred."';}, - 'planned future concurrency':(q:any)=>{q.question+='\nUnder the proposed option, the five IDP calls run concurrently.';}, - 'parallel scheduling title':(q:any)=>{q.question=q.question.replace('issued concurrently','issued in parallel');}, -}))test('IDP choice accepts '+name,()=>expect(idpResult(alterIdp(edit)).decisions['sequential-idp']).toBeDefined()); -for(const [name,edit] of Object.entries({ - 'foreign source':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','OTHER.md');}, - 'foreign source directory':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','archive/PLAN.md');}, - 'quoted source':(q:any)=>{q.question=q.question.replace(/^Project\/branch\/task: (.+)$/m,'Project/branch/task: "$1"');}, - 'historical source':(q:any)=>{q.question=q.question.replace('Project/branch/task: ','Project/branch/task: Historical example: ');}, - 'quoted current defect':(q:any)=>{q.question=q.question.replace(/^ELI10: (.+)$/m,'ELI10: "$1"');}, - 'unrelated title':(q:any)=>{q.question=q.question.replace('How should the five IDP validation calls be issued concurrently?','Which monitoring dashboard should we use?');}, - 'dependent source calls':(q:any)=>{q.question=q.question.replace('five independent IDP calls','five dependent IDP calls');}, - 'missing current ordering':(q:any)=>{q.question=q.question.replace('Today the five checks run one after another','The five checks have no specified ordering');}, - 'reversed current ordering':(q:any)=>{q.question=q.question.replace('Today the five checks run one after another','Today the five checks run concurrently');}, - 'current parallel correction':(q:any)=>{q.question+='\nCorrection: the five IDP calls already run concurrently.';}, - 'current dependency correction':(q:any)=>{q.question+='\nCorrection: the IDP calls are not independent.';}, - 'no offered timeout':(q:any)=>{q.options[0]={label:'Promise.allSettled',description:'Launch all calls concurrently and report every result.'};}, - 'timeout borrowed from sequential choice':(q:any)=>{q.options[0]={label:'Promise.allSettled',description:'Launch all calls concurrently and report every result.'};q.options[2].description+=' Per-call timeout 2000 ms.';}, - 'timeout borrowed from question':(q:any)=>{q.options[0]={label:'Promise.allSettled',description:'Launch all calls concurrently and report every result.'};q.question+='\nRecommendation: per-call timeout.';}, - 'only sequential timeout remedy':(q:any)=>{q.options[0]={label:'Keep the five calls sequential with per-call timeout',description:'Run each call after the previous call completes.'};}, - 'same-option no timeout':(q:any)=>{q.options[0].description+='\nCorrection: no per-call timeout.';}, - 'same-option sequential correction':(q:any)=>{q.options[0].description+='\nCorrection: keep the five calls sequential.';}, - 'same-option no concurrency':(q:any)=>{q.options[0].description+='\nDo not use Promise.allSettled.';}, - 'same-option negated timeout addition':(q:any)=>{q.options[0].description+='\nDo not add a per-call timeout.';}, - 'same-option calls remain sequential':(q:any)=>{q.options[0].description+='\nCorrection: The IDP calls remain sequential.';}, - 'parallel title foreign source':(q:any)=>{q.question=q.question.replace('issued concurrently','issued in parallel').replaceAll('PLAN.md','OTHER.md');}, - 'parallel title without timeout':(q:any)=>{q.question=q.question.replace('issued concurrently','issued in parallel');q.options[0]={label:'Promise.allSettled',description:'Launch all calls concurrently and report every result.'};}, - 'parallel title sequential correction':(q:any)=>{q.question=q.question.replace('issued concurrently','issued in parallel');q.options[0].description+='\nCorrection: The IDP calls remain sequential.';}, - 'quoted offered remedy':(q:any)=>{q.options[0].label='"'+q.options[0].label+'"';q.options[0].description='"'+q.options[0].description.replaceAll('\n',' ')+'"';}, - 'conditional remedy':(q:any)=>{q.options[0].description='If approved, '+q.options[0].description;}, - 'withdrawn decision':(q:any)=>{q.question+='\nD13 is withdrawn.';}, - 'reopened decision':(q:any)=>{q.question+='\nD13 is reopened.';}, - 'scalar quoted deferred decision':(q:any)=>{q.question+='\nD13 is "deferred".';}, - 'deferred offered remedy':(q:any)=>{q.options[0].description+='\nThis remedy is deferred.';}, - 'scalar quoted pending remedy':(q:any)=>{q.options[0].description+='\nThis remedy is "pending".';}, -}))test('IDP choice rejects '+name,()=>expect(idpResult(alterIdp(edit)).decisions).toEqual({})); -test('IDP choice requires its own completed native answer and distinct seed identity',()=>{ - const call=idpCall(),finished=Date.parse(idpChoice.captureAt); - const guard=(c=call,prior:NativePlanQuestionCall[]=[])=>isEngSeedDecisionAUQ(nativePlanCallFingerprint(c,0,true),prior,0,finished); - expect(guard()).toBe(true);expect(guard(call,[call])).toBe(false); - for(const edit of [(c:any)=>{c.answered=false;},(c:any)=>{c.failed=true;},(c:any)=>{c.answers={};},(c:any)=>{c.unansweredQuestionIndices=[0];},(c:any)=>{c.answeredAt=new Date(finished+1).toISOString();}]){ - const invalid=idpCall();edit(invalid);expect(guard(invalid)).toBe(false);expect(idpResult(invalid).decisions).toEqual({}); - } - const foreign=structuredClone(transcript().calls[5]) as NativePlanQuestionCall;foreign.sessionId+='-foreign';expect(guard(call,[foreign])).toBe(false); - const bundled=idpCall();bundled.questions.push(transcript().calls[5]!.questions[0]!);bundled.answers={...bundled.answers,...transcript().calls[5]!.answers}; - expect(guard(bundled)).toBe(false);expect(idpResult(bundled).decisions).toEqual({}); -}); -for(const [index,seed] of seeds) test('actual native decision owns '+seed,()=>{ - expect(Object.keys(evaluate([transcript().calls[index]!],'').decisions)).toEqual([seed]); -}); -test('actual final callback assertions pass without changing the recorded failed attempt',()=>{ - expect(captured.originalOutcome).toBe('no_review_questions'); - expect(captured.originalCounts).toEqual({review:0,setup:14}); - const result=evaluate(); - expect(result.ok).toBe(true); - expect(new Set(Object.values(result.decisions)).size).toBe(4); - expect(result.regression).toBe('plan'); - expect(assertReviewReportAtBottom(captured.report).ok).toBe(true); -}); - -const change=(index:number,edit:(q:NativePlanQuestionCall['questions'][number])=>void)=>{ - const c=transcript().calls[index]!,q=c.questions[0]!;edit(q);c.answers={[q.question]:q.options[0]!.label};return c; -}; -for(const [name,edit] of Object.entries({ - 'quoted provenance':(q:any)=>{q.question=q.question.replace(/^Project\/branch\/task: (.+)$/m,'Project/branch/task: "$1"');}, - 'source history':(q:any)=>{q.question=q.question.replace('Project/branch/task: ','Project/branch/task: Historical example: ');}, - 'foreign source':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','OTHER.md');}, - 'foreign suffix':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','OTHER-PLAN.md');}, - 'foreign directory':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','archive/PLAN.md');}, - 'quoted explanation':(q:any)=>{q.question=q.question.replace(/^ELI10: (.+)$/m,'ELI10: "$1"');}, - 'conditional explanation':(q:any)=>{q.question=q.question.replace('ELI10: ','ELI10: If approved, ');}, - 'withdrawn question':(q:any)=>{q.question+='\nThis decision is withdrawn.';}, - 'withdrawn remedy':(q:any)=>{q.options[0].description+='\nThis remedy is withdrawn.';}, - 'quoted inactive status':(q:any)=>{q.question+='\nThis decision is "withdrawn".';}, - 'remedy borrowed from Net':(q:any)=>{q.question+='\nNet: '+q.options[0].description;q.options[0].description='Discuss the next steps.';}, - 'quoted remedy':(q:any)=>{q.options[0].label='"'+q.options[0].label+'"';q.options[0].description='"'+q.options[0].description.replaceAll('\n',' ')+'"';}, -})) test('source-bound semantic classes reject '+name,()=>{ - for(const [index] of seeds.slice(0,3))expect(evaluate([change(index,edit)],'').decisions).toEqual({}); -}); -for(const [index,label,edit] of [ - [4,'inventory count',(q:any)=>{q.question=q.question.replace('five new building blocks','six new building blocks');}], - [4,'second store',(q:any)=>{q.options[0].description=q.options[0].description.replace('One owner for cached token state; no second store','Two owners for cached token state; a second store');}], - [4,'retained class independence',(q:any)=>{q.question+='\nTokenStore already has a documented independent purpose.';}], - [5,'other service',(q:any)=>{q.options[0].label=q.options[0].label.replace('pass to both constructors','pass to another constructor');}], - [5,'shared test instance',(q:any)=>{q.options[0].description=q.options[0].description.replace('fresh AuthCache','shared AuthCache');}], - [5,'already repaired cache',(q:any)=>{q.question+='\nThe services are already injected.';}], - [7,'partial mapping',(q:any)=>{q.options[0].label=q.options[0].label.replace('each error class','some error classes');}], - [7,'fail open',(q:any)=>{q.options[0].label=q.options[0].label.replace('fail closed','fail open');}], - [7,'swallowed errors',(q:any)=>{q.options[0].description=q.options[0].description.replace('nothing is silently swallowed','errors are silently swallowed');}], - [7,'already repaired function',(q:any)=>{q.question+='\nvalidateAndDispatch() already no longer swallows failures.';}], -] as const)test('same-option remedy requires '+label,()=>expect(evaluate([change(index,edit)],'').decisions).toEqual({})); - -test('semantic wording and type names do not require the captured sentence',()=>{ - const edits=[ - [4,(q:any)=>{q.question=q.question.replace('five new building blocks','5 new components').replace('TokenStore is never described','TokenStore is undefined');q.options[0].label=q.options[0].label.replace('Consolidate:','Merge:').replace('typed value/config','config');}], - [5,(q:any)=>{q.options[0].label=q.options[0].label.replace('AuthCache once','one AuthCache').replace('pass to both constructors','injected to both services');q.options[0].description=q.options[0].description.replace('fresh AuthCache','isolated instance');}], - [7,(q:any)=>{q.options[0].label=q.options[0].label.replace('AuthFailure','RejectedAuth').replace('one boundary catch','single catch at the boundary');}], - ] as const; - for(const [index,edit] of edits)expect(Object.keys(evaluate([change(index,edit)],'').decisions)).toHaveLength(1); -}); - -test('historical seed guard remains available while the actual callback uses one terminal assessment',()=>{ - const source=fs.readFileSync(path.join(import.meta.dir,'skill-e2e-plan-eng-finding-count.test.ts'),'utf8'); - const guard=(fp:any,prior:NativePlanQuestionCall[])=>isEngSeedDecisionAUQ(fp,prior,captured.startedAt,captured.finishedAt); - const calls=transcript().calls; - expect(calls.filter((c,i)=>guard(nativePlanCallFingerprint(c,0,true),calls.slice(0,i)))).toHaveLength(4); - expect(source).not.toContain('createEngBatchingIssueCounter'); - expect(source).toContain('evaluateEngTerminalReview(followUpPrompt'); - expect(source).not.toContain('isReviewAUQ:'); - expect(source).not.toContain('isCompletionHandoffAUQ:'); - expect(source).toContain('assertReviewReportAtBottom(planContent)'); - expect(source).toContain('reviewCountCeiling: Infinity'); - expect(source).toContain('const deadlineAt = startedAt + 1_500_000'); - expect(source).toContain('timeoutMs: deadlineAt - Date.now()'); - expect(source).toContain('approveEngTestPlanEdits: true'); - expect(source).toContain('preconfiguredReviewActor: true'); - expect(source).toContain('evaluateTerminal: async input =>'); -}); -test('native guard rejects incomplete, unowned, duplicate and foreign calls',()=>{ - const c=transcript().calls[4]!; - const guard=(call=c,prior:NativePlanQuestionCall[]=[],edit=(fp:any)=>{})=>{ - const fp=nativePlanCallFingerprint(call,0,true);edit(fp); - return isEngSeedDecisionAUQ(fp,prior,captured.startedAt,captured.finishedAt); - }; - expect(guard()).toBe(true); - for(const [start,end] of [[NaN,captured.finishedAt],[0,Infinity],[captured.finishedAt,captured.startedAt]])expect(isEngSeedDecisionAUQ(nativePlanCallFingerprint(c,0,true),[],start,end)).toBe(false); - for(const edit of [(c:any)=>{c.answered=false;},(c:any)=>{c.failed=true;},(c:any)=>{c.answeredAt=new Date(captured.startedAt-1).toISOString();}, - (c:any)=>{c.answeredAt=new Date(captured.finishedAt+1).toISOString();},(c:any)=>{c.answers={};}]){ - const bad=structuredClone(c);edit(bad);expect(guard(bad)).toBe(false); - } - for(const edit of [(fp:any)=>{delete fp.nativeCall;},(fp:any)=>{fp.signature+='-foreign';},(fp:any)=>{fp.options[0].label='forged';}, - (fp:any)=>{fp.nativeQuestionIndex=1;}])expect(guard(c,[],edit)).toBe(false); - expect(guard(c,[c])).toBe(false); - const foreign=structuredClone(c);foreign.sessionId+='-foreign';expect(guard(c,[foreign])).toBe(false); - const reask=structuredClone(c);reask.toolUseId+='-reasked';expect(guard(reask,[c])).toBe(false); - const combined=structuredClone(c);combined.questions.push(transcript().calls[5]!.questions[0]!); - combined.answers={...combined.answers,...transcript().calls[5]!.answers};expect(guard(combined)).toBe(false); -}); - -const declaration=/^\*\*CRITICAL regression contract \(D9\):\*\*.+$/m.exec(captured.report)![0]; -const baselineTask=/^- \[ \] \*\*T1[^\n]+\n(?: - [^\n]+\n?)+/m.exec(captured.report)![0]; -const minimalBaseline=()=>`# Current reviewed plan\n\n## Tests\n${declaration}\n\n## Implementation Tasks\n${baselineTask}`; -const regression=(plan:string)=>evaluate([],plan).regression; -const goldenPlan = () => goldenDeclaration.report; -const goldenContract = /^\*\*Regression contract[^\n]*\n.*?(?=\n\n)/ms.exec(goldenPlan())![0]; -const goldenTask = /^- \[ \] \*\*T9[^\n]*\n(?: - [^\n]+\n?)+/m.exec(goldenPlan())![0]; -const goldenLedger = goldenPlan().slice(goldenPlan().indexOf('### R5:')); -test('the final 90f declaration binds named legacy outcomes to its required task and unchanged baseline', () => { - expect(goldenDeclaration.reportSha256).toBe('d4ae545eec013da903b5b3c0b459f9f8d2543ea838001271bd7c82b27ac848c3'); - expect(goldenDeclaration.originalOutcome).toBe('seed_coverage_failed'); - expect(goldenDeclaration.nativeCall.toolUseId).toBe('toolu_01DszoYCjnkNfCj5FxaZJsjQ'); - expect(goldenDeclaration.nativeCall.answered).toBe(true); - expect(regression(goldenPlan())).toBe('plan'); -}); -for (const [name, edit] of Object.entries({ - 'missing declaration': (s:string) => s.replace(goldenContract, ''), - 'missing task': (s:string) => s.replace(goldenTask, ''), - 'missing ledger': (s:string) => s.replace(goldenLedger, ''), - 'missing required status': (s:string) => s.replace('R5, D11, Iron Rule', 'R5, D11'), - 'negated required status': (s:string) => s.replace('R5, D11, Iron Rule', 'R5, D11, not Iron Rule'), - 'optional declaration': (s:string) => s.replace('Regression contract (', 'Optional regression contract ('), - 'conditional capture': (s:string) => s.replace('fixtures pinning current', 'fixtures if approved pinning current'), - 'future outcome oracle': (s:string) => s.replace('pinning current outputs', 'pinning proposed outputs'), - 'foreign legacy target': (s:string) => s.replaceAll('legacyAuthFlow', 'otherAuthFlow'), - 'wrong reviewed source': (s:string) => s.replace('Reviewed target: `PLAN.md`', 'Reviewed target: `OTHER.md`'), - 'wrong reviewed branch': (s:string) => s.replace('on `main`', 'on `feature`'), - 'conflicting reviewed source': (s:string) => s + '\n## Current ownership\nReviewed target: OTHER.md on main\n', - 'foreign finding source': (s:string) => s.replaceAll('PLAN.md:', 'archive/PLAN.md:'), - 'noncritical finding': (s:string) => s.replace('Finding: T1, P1 CRITICAL', 'Finding: T1, P2'), - 'negated critical finding': (s:string) => s.replace('Finding: T1, P1 CRITICAL', 'Finding: T1, P1 not CRITICAL'), - 'pending ownership': (s:string) => s.replace('State: approved', 'State: pending'), - 'duplicate owner': (s:string) => s + '\n' + goldenLedger, - 'wrong record owner': (s:string) => s.replace('### R5:', '### R15:'), - 'duplicate finding': (s:string) => s.replace('State: approved', 'Finding: T1, P1 CRITICAL, PLAN.md:14\nState: approved'), - 'different decision answer': (s:string) => s.replace('A (D11)', 'A (D12)'), - 'selected option omits characterization': (s:string) => s.replace('Actual answer: A', 'Actual answer: B'), - 'selected option has negated characterization': (s:string) => s.replace('Options: A) Characterization', 'Options: A) No characterization'), - 'duplicate offered identity': (s:string) => s.replace('; B) Parity + routing only', '; A) Parity + routing only'), - 'missing named preservation': (s:string) => s.replace(/^Behavior to preserve.+$/m, ''), - 'flagged outcome ownership': (s:string) => s.replace('Behavior to preserve (legacy tenants, flag off)', 'Behavior to preserve (flagged tenants, flag on)'), - 'missing accepted scope': (s:string) => s.replace(/^Accepted scope:.+$/m, ''), - 'conditional accepted scope': (s:string) => s.replace('Accepted scope: (1)', 'Accepted scope: If approved, (1)'), - 'wrong task decision': (s:string) => s.replace('Tests — T1 (PLAN.md:14-16, :27-28), D11', 'Tests — T1 (PLAN.md:14-16, :27-28), D12'), - 'mixed task decisions': (s:string) => s.replace('Tests — T1 (PLAN.md:14-16, :27-28), D11', 'Tests — T1 (PLAN.md:14-16, :27-28), D11, D12'), - 'foreign task source': (s:string) => s.replace('Tests — T1 (PLAN.md:14-16', 'Tests — T1 (archive/PLAN.md:14-16'), - 'negated critical task': (s:string) => s.replace('T9 (P1 CRITICAL', 'T9 (P1 not CRITICAL'), - 'noncritical task': (s:string) => s.replace('T9 (P1 CRITICAL', 'T9 (P2'), - 'negated task': (s:string) => s.replace('Write the `legacyAuthFlow`', 'Do not write the `legacyAuthFlow`'), - 'optional task': (s:string) => s.replace('Write the `legacyAuthFlow`', 'Optionally write the `legacyAuthFlow`'), - 'task count alone': (s:string) => s.replace('Write the `legacyAuthFlow` characterization suite (6 golden fixtures)', 'Create a suite (6 golden fixtures)'), - 'wrong task count': (s:string) => s.replace('suite (6 golden fixtures)', 'suite (5 golden fixtures)'), - 'missing task files': (s:string) => s.replace(/^ - Files:.+$/m, ''), - 'foreign task files': (s:string) => s.replace('legacyAuthFlow.characterization.test', 'otherAuthFlow.characterization.test'), - 'missing task verification': (s:string) => s.replace(/^ - Verify:.+$/m, ''), - 'different baseline': (s:string) => s.replace('unmodified main', 'unmodified feature'), - 'modified baseline': (s:string) => s.replace('unmodified main', 'modified main'), - 'post-refactor baseline': (s:string) => s.replace('before any refactor lands', 'after any refactor lands'), - 'negated baseline': (s:string) => s.replace('suite green on', 'suite not green on'), - 'conditional baseline': (s:string) => s.replace('suite green on', 'if convenient, suite green on'), - 'duplicate task': (s:string) => s.replace(goldenTask, goldenTask + '\n' + goldenTask), - 'neighbor task baseline': (s:string) => s.replace(' - Verify:', '- [ ] **T99** — Other tests\n - Verify:'), - 'quoted declaration': (s:string) => s.replace(goldenContract, goldenContract.split('\n').map(l => '> ' + l).join('\n')), - 'fenced task': (s:string) => s.replace(goldenTask, '```\n' + goldenTask + '\n```'), - 'historical ledger': (s:string) => s.replace('## Review ledger', '## Historical review ledger'), - 'task withdrawal': (s:string) => s + '\n## Current amendments\nT9 is withdrawn.\n', - 'decision withdrawal': (s:string) => s + '\n## Current amendments\nD11 is withdrawn.\n', - 'record withdrawal': (s:string) => s + '\n## Current amendments\nR5 is withdrawn.\n', - 'baseline reversed': (s:string) => s + '\n## Current amendments\nlegacyAuthFlow will be changed before T9.\n', -})) test('owned legacy declaration rejects ' + name, () => expect(regression(edit(goldenPlan()))).toBeUndefined()); -for (const outcome of ['valid', 'expired', 'revoked', 'malformed token', 'suspended tenant', 'IDP unavailable']) { - for (const owner of ['declaration', 'preservation', 'scope']) test('owned legacy declaration retains ' + outcome + ' in ' + owner, () => { - const source = goldenPlan(); - const field = owner === 'declaration' ? goldenContract : owner === 'preservation' - ? /^Behavior to preserve.+$/m.exec(source)![0] : /^Accepted scope:.+$/m.exec(source)![0]; - const mutated = field.replace(new RegExp('\\b' + outcome + '(?:s)?(?:[,;] )?'), ''); - expect(mutated).not.toBe(field); - expect(regression(source.replace(field, mutated))).toBeUndefined(); - }); -} -test('owned legacy declarations support equivalent oracle verbs, selected identities and optional function parentheses', () => { - for (const verb of ['recording existing', 'capturing prior']) expect(regression(goldenPlan().replace('pinning current', verb))).toBe('plan'); - expect(regression(goldenPlan().replace('Options: A)', 'Options: D)').replace('Actual answer: A (D11)', 'Actual answer: D (D11)'))).toBe('plan'); - expect(regression(goldenPlan().replaceAll('`legacyAuthFlow`', '`legacyAuthFlow()`').replace('Write the', 'Implement the') - .replace('suite green on unmodified main before any refactor lands', 'suite passes on untouched main before the rewrite begins'))).toBe('plan'); - expect(regression(goldenPlan() + '\n## Other suite\nBilling characterization suite is withdrawn.\n')).toBe('plan'); -}); -test('the captured legacy contract and its owned task are sufficient without unrelated report text',()=>expect(regression(minimalBaseline())).toBe('plan')); -for(const [name,edit] of Object.entries({ - 'missing declaration':(s:string)=>s.replace(declaration,''), - 'missing task':(s:string)=>s.replace(baselineTask,''), - 'foreign target':(s:string)=>s.replaceAll('legacyAuthFlow','otherAuthFlow'), - 'missing baseline verification':(s:string)=>s.replace(/^ - Verify:.+$/m,''), - 'modified baseline':(s:string)=>s.replace('unmodified `main`','modified `main`'), - 'different baseline':(s:string)=>s.replace('unmodified `main`','unmodified `feature`'), - 'wrong task decision':(s:string)=>s.replace('R4/D9 CRITICAL','R4/D99 CRITICAL'), - 'different outcome count':(s:string)=>s.replace('suite (7 outcomes','suite (6 outcomes'), - 'different verification count':(s:string)=>s.replace('each of the 7 outcomes','each of the 6 outcomes'), - 'baseline after wrap':(s:string)=>s.replace('BEFORE the Phase 1 flag wrap','AFTER the Phase 1 flag wrap'), - 'task after wrap':(s:string)=>s.replace('before any flag wrap','after any flag wrap'), - 'negated write':(s:string)=>s.replace('Write the `legacyAuthFlow()`','Do not write the `legacyAuthFlow()`'), - 'negated land':(s:string)=>s.replace('land it green','do not land it green'), - 'conditional task':(s:string)=>s.replace('Write the `legacyAuthFlow()`','If approved, write the `legacyAuthFlow()`'), - 'quoted declaration':(s:string)=>s.replace(declaration,'"'+declaration+'"'), - 'quoted task':(s:string)=>s.replace(baselineTask,'"'+baselineTask.trim().replaceAll('\n',' ')+'"'), - 'historical section':(s:string)=>s.replace('## Tests','## Historical Tests'), - 'conditional declaration':(s:string)=>s.replace('characterization suite at','if approved, characterization suite at'), - 'quoted document':(s:string)=>'Quoted source material only:\n'+s.replace('# Current reviewed plan','# Report'), - 'task cancellation':(s:string)=>s+'\n## Current amendments\nT1 is withdrawn.\n', - 'decision cancellation':(s:string)=>s+'\n## Current amendments\nD9 is "withdrawn".\n', - 'verification cancellation':(s:string)=>s+'\n## Current amendments\nThis verification is optional.\n', - 'reversed implementation order':(s:string)=>s+'\n## Current amendments\nlegacyAuthFlow() will be changed before T1.\n', -}))test('legacy baseline rejects '+name,()=>expect(regression(edit(minimalBaseline()))).toBeUndefined()); -test('legacy baseline accepts equivalent mandatory verbs, preserves quoted history and other suite ownership',()=>{ - expect(regression(minimalBaseline().replace('CRITICAL regression contract','Required regression contract').replace('Written and green','Implemented and green').replace('land it green','land it passing'))).toBe('plan'); - expect(regression(minimalBaseline()+'\n## Current amendments\nEarlier note: "T1 is withdrawn."\n')).toBe('plan'); - expect(regression(minimalBaseline()+'\n## Billing regression suite\nThis suite is withdrawn.\n')).toBe('plan'); -}); -test('final assertion gate still rejects missing seeds, missing legacy coverage and missing report',()=>{ - for(const [index,seed] of seeds){const input=transcript().calls.filter((_,i)=>i!==index);expect(evaluate(input).missing).toContain(seed);expect(evaluate(input).ok).toBe(false);} - expect(evaluate(transcript().calls,'## GSTACK REVIEW REPORT\nEng complete.\n').ok).toBe(false); - expect(evaluate(transcript().calls,minimalBaseline()).ok).toBe(false); -}); - -for(const [index,verb] of [[4,'consolidate'],[5,'inject'],[7,'map']] as const)test('same-option explicit cancellation rejects '+verb,()=>{ - expect(evaluate([change(index,q=>{q.options[0]!.description+='\nCorrection: Do not '+verb+' this remedy.';})],'').decisions).toEqual({}); -}); - -test('historical native exit/seed gates retain current report-bottom assertions',()=>{ - const source=fs.readFileSync(path.join(import.meta.dir,'skill-e2e-plan-eng-finding-count.test.ts'),'utf8'); - const start=source.indexOf(" if (!['plan_ready', 'completion_summary'].includes(obs.outcome))"),end=source.indexOf(' // A native completion summary',start); - expect(start).toBeGreaterThan(0);expect(end).toBeGreaterThan(start); - const validate=new Function('fs','planPath','obs','assertReviewReportAtBottom', - new Bun.Transpiler({loader:'ts'}).transformSync(source.slice(start,end))); - const dir=fs.mkdtempSync(path.join(os.tmpdir(),'eng-seed-native-')),file=path.join(dir,'report.md'); - try{ - const write=(body=captured.report)=>{fs.writeFileSync(file,body);fs.utimesSync(file,captured.reportMtimeMs/1000,captured.reportMtimeMs/1000);};write(); - expect(createHash('sha256').update(captured.report).digest('hex')).toBe(captured.reportSha256); - const t=transcript(),nonReview=new Set();let review=0; - t.calls.forEach((call,i)=>{const fp=nativePlanCallFingerprint(call,0,true);if(isEngSeedDecisionAUQ(fp,t.calls.slice(0,i),captured.startedAt,captured.finishedAt))review++;else nonReview.add(fp.signature);}); - const frame=classifyPlanCountFrame(captured.screen); - expect(frame).toBe('plan_ready');expect(review).toBe(4); - expect(hasNativePlanTerminal(t,file,captured.startedAt,'plan_ready')).toBe(true); - expect(isQuestionlessNativePlanExit(t,file,captured.startedAt,captured.screen,new Set(t.calls.map(c=>`${c.sessionId}:${c.toolUseId}`)))).toBe(true); - expect(isQuestionlessNativePlanExit(t,file,captured.startedAt,captured.screen,nonReview)).toBe(false); - const obs={outcome:frame,transcript:t,reviewCount:review,step0Count:nonReview.size,fingerprints:[],elapsedMs:0,evidence:captured.screen}; - const check=(input=obs)=>{ - validate(fs,file,input,assertReviewReportAtBottom); - if(!evaluateEngSeedCoverage(input.transcript,fs.readFileSync(file,'utf8'),captured.startedAt,captured.finishedAt).ok) throw Error('SEED COVERAGE FAIL'); - }; - expect(()=>check()).not.toThrow(); - expect(()=>check({...obs,outcome:'no_review_questions' as any})).toThrow('finding-count FAILED'); - const missing={...obs,transcript:{...t,calls:t.calls.filter((_,i)=>i!==4)}};expect(()=>check(missing)).toThrow('SEED COVERAGE FAIL'); - write('## GSTACK REVIEW REPORT\nEng complete.\n');expect(()=>check()).toThrow('SEED COVERAGE FAIL'); - write(captured.report+'\n## Work after report\nExtra\n');expect(()=>check()).toThrow('D19 FAIL'); - }finally{fs.rmSync(dir,{recursive:true,force:true});} -}); - -for(const [index,claim] of [ - [4,'Correction: A second store still remains.'], - [5,'Correction: Do not inject AuthCache.'], - [5,'Correction: Tests do not get a fresh AuthCache.'], - [5,'Correction: Tests share one AuthCache.'], - [7,'Correction: Not every failure class has a named outcome.'], - [7,'Correction: Errors are still swallowed.'], - [7,'Correction: Some errors are silently ignored.'], -] as const)test('a current contradictory remedy cannot retain earlier positive words: '+claim,()=>{ - expect(evaluate([change(index,q=>{q.options[0]!.description+='\n'+claim;})],'').decisions).toEqual({}); - expect(Object.keys(evaluate([change(index,q=>{q.options[0]!.description+='\nEarlier note: "'+claim+'"';})],'').decisions)).toHaveLength(1); -}); - -for(const outcomes of [', denied','denied, denied '])test('legacy outcomes cannot use empty or duplicate labels: '+outcomes,()=>{ - const plan=minimalBaseline().replace(/one test per current outcome: [^.]+\./,'one test per current outcome: '+outcomes+'.') - .replaceAll('7 outcomes','2 outcomes'); - expect(regression(plan)).toBeUndefined(); -}); - -for(const status of ['deferred','not required','not needed','superseded','no longer needed'])test('current baseline ownership respects '+status,()=>{ - for(const id of ['T1','D9']){ - expect(regression(minimalBaseline()+`\n## Current amendments\n${id} is ${status}.\n`)).toBeUndefined(); - expect(regression(minimalBaseline()+`\n## Current amendments\n${id} is "${status}".\n`)).toBeUndefined(); - expect(regression(minimalBaseline()+`\n## Current amendments\nEarlier note: "${id} is ${status}."\n`)).toBe('plan'); - } -}); - - -// Current choice identity is in the title; current defect and exact inventory -// belong to this same native question's source and explanation. -import currentChoiceCab3 from './fixtures/eng-current-choice-cab3.json'; - -const countedCf74 = (index: number) => structuredClone(currentChoiceCab3.currentCountCf74.calls[index]) as NativePlanQuestionCall; -const countedResultCf74 = (call: NativePlanQuestionCall) => evaluateEngSeedCoverage( - {status:'ready', calls:[call], assistantMessages:[]}, '', currentChoiceCab3.currentCountCf74.startedAt, currentChoiceCab3.currentCountCf74.finishedAt).decisions; -const countedChangeCf74 = (index:number, edit:(q:NativePlanQuestionCall['questions'][number])=>void) => { - const c=countedCf74(index), old=c.questions[0]!.question, answer=c.answers![old]!; - edit(c.questions[0]!);c.answers={[c.questions[0]!.question]:c.questions[0]!.options.some(o=>o.label===answer)?answer:c.questions[0]!.options[0]!.label};return c; -}; -for(const [index,name] of [[0,'current undefined-class removal'],[1,'current facade reduction']] as const) - test('cf74 counted complexity: '+name+' uses its complete current question and one offered alternative',()=>{ - const c=countedCf74(index);expect(countedResultCf74(c)).toEqual({complexity:`${c.sessionId}:${c.toolUseId}`}); - expect(isEngSeedDecisionAUQ(nativePlanCallFingerprint(c,0,true),[],currentChoiceCab3.currentCountCf74.startedAt,currentChoiceCab3.currentCountCf74.finishedAt)).toBe(true); - }); -function countedCheckCf74(index:number,name:string,expected:boolean,edit:(q:NativePlanQuestionCall['questions'][number])=>void) { - test(`cf74 counted complexity ${index}: ${name}`,()=>{ - const c=countedChangeCf74(index,edit); - expect(countedResultCf74(c)).toEqual(expected?{complexity:`${c.sessionId}:${c.toolUseId}`}:{ }); - }); -} -for(const index of [0,1]) { - for(const [name,edit] of Object.entries({ - 'foreign source':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','OTHER.md');}, - 'same-basename foreign path':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','archive/PLAN.md');}, - 'duplicate source':(q:any)=>{q.question+='\nProject/branch/task: OTHER.md';}, - 'only quoted source':(q:any)=>{q.question=q.question.replace(/^Project\/branch\/task: (.+)$/m,'Project/branch/task: "$1"');}, - 'quoted explanation':(q:any)=>{q.question=q.question.replace(/^ELI10: (.+)$/m,'ELI10: "$1"');}, - 'single-quoted explanation':(q:any)=>{q.question=q.question.replace(/^ELI10: (.+)$/m,"ELI10: '$1'");}, - 'historical explanation':(q:any)=>{q.question=q.question.replace('ELI10: ','ELI10: Historical example: ');}, - 'quoted title':(q:any)=>{q.question=q.question.replace(/^([^\n]+)/,'"$1"');}, - 'conditional choice':(q:any)=>{q.question+='\nThis decision applies only if approved.';}, - 'withdrawn choice':(q:any)=>{q.question+='\nThis decision is withdrawn.';}, - 'quoted current withdrawn choice':(q:any)=>{q.question+='\nThis decision is "withdrawn".';}, - 'reopened choice':(q:any)=>{q.question+='\nThis decision is reopened.';}, - 'missing current explanation':(q:any)=>{q.question=q.question.replace(/^ELI10: .+$/m,'ELI10: A general design discussion.');}, - })) countedCheckCf74(index,name,false,edit); - const reduction=(q:any)=>q.options.find((o:any)=>/^(?:Defer TokenStore|Drop the facade)/.test(o.label)); - for(const [name,edit] of Object.entries({ - 'quoted complete remedy':(q:any)=>{const o=reduction(q);o.description='"'+o.description+'"';}, - 'withdrawn remedy':(q:any)=>{reduction(q).description+='\nThis remedy is withdrawn.';}, - 'quoted current withdrawn remedy':(q:any)=>{reduction(q).description+='\nThis option is "withdrawn".';}, - 'conditional remedy':(q:any)=>{reduction(q).description+='\nThis remedy applies only if approved.';}, - 'foreign remedy':(q:any)=>{reduction(q).description+='\nThis remedy applies to another project.';}, - 'adapter replacement':(q:any)=>{reduction(q).description+='\nReplace the existing adapter.';}, - 'subordinate extra work':(q:any)=>{reduction(q).description+='\nWhile implementing a new database.';}, - 'same option adds another feature':(q:any)=>{reduction(q).description+='\nAlso add a new persistence engine.';}, - 'descriptive negated removal':(q:any)=>{reduction(q).description+='\nThis option never drops '+(index===0?'TokenStore.':'the facade.');}, - 'imperative negated removal':(q:any)=>{reduction(q).description+='\nDo not drop '+(index===0?'TokenStore.':'the facade.');}, - 'only history contains remedy':(q:any)=>{const o=reduction(q);o.description='Earlier note: "'+o.description+'"';}, - })) countedCheckCf74(index,name,false,edit); - for(const [name,edit] of Object.entries({ - 'options may reorder':(q:any)=>{q.options.reverse();}, - 'decision number may change':(q:any)=>{q.question=q.question.replace(/^D\d+/,'D77');}, - 'current question can cite earlier quoted history':(q:any)=>{q.question+='\nEarlier note: "This decision is withdrawn."';}, - 'formatting does not own the evidence':(q:any)=>{q.question=q.question.replaceAll('`','');}, - })) countedCheckCf74(index,name,true,edit); -} -for(const [name,edit] of Object.entries({ - 'mismatched baseline count':(q:any)=>{q.question=q.question.replace('12 files, 4 new classes','12 files, 5 new classes');}, - 'missing current baseline count':(q:any)=>{q.question=q.question.replace('12 files, 4 new classes','an unspecified scope');}, - 'duplicate baseline count':(q:any)=>{q.question=q.question.replace('12 files, 4 new classes','12 files, 4 new classes; 12 files, 5 new classes');}, - 'no stated contract gap':(q:any)=>{q.question=q.question.replace('without saying what it stores that the adapter does not','with a documented persistence contract');}, - 'foreign component has the missing contract':(q:any)=>{q.question=q.question.replace('class called TokenStore (PLAN.md:35)','class called OtherStore (PLAN.md:35)');}, - 'existing adapter does not store tokens':(q:any)=>{q.question=q.question.replace('adapter stores tokens','adapter does not store tokens');}, - 'existing adapter does not expire tokens':(q:any)=>{q.question=q.question.replace('evicts expired ones','retains expired ones');}, - 'current independent responsibility':(q:any)=>{q.question+='\nCorrection: TokenStore has a documented independent persistence purpose.';}, - 'already removed from current refactor':(q:any)=>{q.question+='\nCorrection: TokenStore is already removed from this refactor.';}, - 'no current keep option':(q:any)=>{q.options[1].label='Discuss storage';}, - 'keep option actually removes class':(q:any)=>{q.options[1].description+='\nAlso remove TokenStore.';}, - 'removal offers no smaller count':(q:any)=>{q.options[0].description=q.options[0].description.replace('Drops one of the 4 new classes','Keeps all 4 new classes');}, - 'same-option negated count':(q:any)=>{q.options[0].description=q.options[0].description.replace('Drops one of the 4 new classes','Never drops one of the 4 new classes');}, - 'same-option retained class':(q:any)=>{q.options[0].description+='\nTokenStore remains in this refactor.';}, - 'adapter ownership moved to keep option':(q:any)=>{const s='One token source of truth: the retained adapter behind the AuthCache facade.';q.options[0].description=q.options[0].description.replace(s,'No current storage choice.');q.options[1].description+=' '+s;}, - 'adapter ownership negated':(q:any)=>{q.options[0].description=q.options[0].description.replace('One token source of truth','Not one token source of truth');}, -})) countedCheckCf74(0,name,false,edit); -for(const [name,edit] of Object.entries({ - 'wrong total count':(q:any)=>{q.question=q.question.replace('four new types:', 'five new types:');}, - 'wrong grouped service count':(q:any)=>{q.question=q.question.replace('two services (AuthBroker, SessionMint)','three services (AuthBroker, SessionMint)');}, - 'duplicate grouped service':(q:any)=>{q.question=q.question.replace('two services (AuthBroker, SessionMint)','two services (AuthBroker, AuthBroker)');}, - 'foreign current service':(q:any)=>{q.question=q.question.replace('two services (AuthBroker, SessionMint)','two services (AuthBroker, OtherService)');}, - 'independent current facade':(q:any)=>{q.question+='\nCorrection: AuthCache now has independent behavior.';}, - 'no current facade behavior gap':(q:any)=>{q.question=q.question.replace('it adds no behavior of its own','it owns independent policy behavior');}, - 'quoted gap only':(q:any)=>{q.question=q.question.replace('so it adds no behavior of its own','so "it adds no behavior of its own"');}, - 'smaller-count arithmetic wrong':(q:any)=>{q.options[1].description=q.options[1].description.replace('Three new types instead of four','Two new types instead of four');}, - 'before-count arithmetic wrong':(q:any)=>{q.options[1].description=q.options[1].description.replace('Three new types instead of four','Three new types instead of five');}, - 'negated smaller count':(q:any)=>{q.options[1].description=q.options[1].description.replace('Three new types instead of four','Not three new types instead of four');}, - 'current keep count contradicts baseline':(q:any)=>{q.options[0].description=q.options[0].description.replace('carrying 3 new ones','carrying 2 new ones');}, - 'no keep option':(q:any)=>{q.options[0].label='Discuss interfaces';}, - 'keep option removes facade':(q:any)=>{q.options[0].description+='\nAlso drop the facade.';}, - 'smaller alternative retains facade':(q:any)=>{q.options[1].description+='\nKeep the AuthCache facade.';}, - 'direct adapter action only in another option':(q:any)=>{q.options[1].label='Drop the facade';q.options[0].description+=' Use the adapter directly.';}, - 'count only in another option':(q:any)=>{const s='Three new types instead of four';q.options[1].description=q.options[1].description.replace(s,'A different arrangement');q.options[0].description+=' '+s;}, - 'existing adapter tests not retained':(q:any)=>{q.options[1].description=q.options[1].description.replace("adapter's existing tests",'new implementation tests');}, -})) countedCheckCf74(1,name,false,edit); -countedCheckCf74(0,'equivalent current question and numeric baseline',true,q=>{ - q.question=q.question.replace('Does TokenStore stay in this refactor, or is it cut/deferred?','Keep TokenStore in this refactor or remove it?').replace('12 files, 4 new classes','12 files, four new classes'); -}); -countedCheckCf74(1,'flat explicit inventory and numeric reduction',true,q=>{ - q.question=q.question.replace('four new types: two services (AuthBroker, SessionMint), RequestPolicy, and AuthCache.','4 new classes: AuthBroker, SessionMint, RequestPolicy, and AuthCache.'); - q.options[1]!.description=q.options[1]!.description!.replace('Three new types instead of four','3 new classes instead of 4'); -}); -countedCheckCf74(0,'duplicate current metadata count is ambiguous',false,q=>{q.question=q.question.replace('12 files, 4 new classes','12 files, 4 new classes; 12 files, 4 new classes');}); -countedCheckCf74(0,'later contradictory removal count cannot borrow earlier reduction',false,q=>{q.options[0]!.description+=' Drops one of the 5 new classes.';}); -countedCheckCf74(1,'duplicate complete current inventory is ambiguous',false,q=>{q.question=q.question.replace('ELI10: ','ELI10: The plan adds four new types: AuthBroker, SessionMint, RequestPolicy, and AuthCache. ');}); -countedCheckCf74(1,'later contradictory option count stays operative',false,q=>{q.options[1]!.description+=' Four new types instead of four.';}); -test('cf74 counted complexity keeps complete native ACK and distinct-seed requirements',()=>{ - const x=currentChoiceCab3.currentCountCf74, c=countedCf74(0), fp=nativePlanCallFingerprint(c,0,true); - for(const option of c.questions[0]!.options){c.answers={[c.questions[0]!.question]:option.label};expect(countedResultCf74(c).complexity).toBeDefined();} - expect(isEngSeedDecisionAUQ(fp,[countedCf74(0)],x.startedAt,x.finishedAt)).toBe(false); - expect(isEngSeedDecisionAUQ(nativePlanCallFingerprint(countedCf74(1),0,true),[countedCf74(0)],x.startedAt,x.finishedAt)).toBe(false); - for(const edit of [(c:any)=>{c.answered=false;},(c:any)=>{c.failed=true;},(c:any)=>{c.answers={};},(c:any)=>{c.unansweredQuestionIndices=[0];},(c:any)=>{c.answeredAt=new Date(x.finishedAt+1).toISOString();}]){const v=countedCf74(0);edit(v);expect(countedResultCf74(v)).toEqual({});} - const packet=countedCf74(0),other=countedCf74(1);packet.questions.push(other.questions[0]!);packet.answers![other.questions[0]!.question]=other.answers![other.questions[0]!.question]!; - expect(countedResultCf74(packet)).toEqual({}); -}); -const cab3Call=(index:number)=>structuredClone(currentChoiceCab3.calls[index]) as NativePlanQuestionCall; -const cab3Result=(c:NativePlanQuestionCall)=>evaluateEngSeedCoverage({status:'ready',calls:[c],assistantMessages:[]},'',0,Date.parse(currentChoiceCab3.captureAt)).decisions; -const cab3Change=(index:number,edit:(q:NativePlanQuestionCall['questions'][number])=>void)=>{const c=cab3Call(index);edit(c.questions[0]!);c.answers={[c.questions[0]!.question]:c.questions[0]!.options[0]!.label};return c;}; -for(const [index,seed] of [[0,'complexity'],[1,'swallowed-errors']] as const){ - test('cab3 current owned choice identifies '+seed,()=>{ - const c=cab3Call(index);expect(cab3Result(c)).toEqual({[seed]:`${c.sessionId}:${c.toolUseId}`}); - expect(isEngSeedDecisionAUQ(nativePlanCallFingerprint(c,0,true),[],0,Date.parse(currentChoiceCab3.captureAt))).toBe(true); - for(const option of c.questions[0]!.options){c.answers={[c.questions[0]!.question]:option.label};expect(cab3Result(c)[seed]).toBeDefined();} - }); - test('cab3 current choice preserves formatting, ordering and historical examples: '+seed,()=>{ - for(const edit of [ - (q:any)=>{q.question=q.question.replaceAll('`','');}, - (q:any)=>{q.options.reverse();}, - (q:any)=>{q.question+='\nEarlier note: "This decision is withdrawn."';}, - (q:any)=>{q.question=q.question.replace(/^D\d+ — /,'D42: ');}, - ])expect(cab3Result(cab3Change(index,edit))[seed]).toBeDefined(); - }); - test('cab3 current choice requires its own source and current evidence: '+seed,()=>{ - for(const edit of [ - (q:any)=>{q.question=q.question.replaceAll('PLAN.md','OTHER.md');}, - (q:any)=>{q.question=q.question.replaceAll('PLAN.md','archive/PLAN.md');}, - (q:any)=>{q.question=q.question.replace(/^Project\/branch\/task: (.+)$/m,'Project/branch/task: "$1"');}, - (q:any)=>{q.question=q.question.replace('Project/branch/task: ','Project/branch/task: Historical example: ');}, - (q:any)=>{q.question=q.question.replace(/^ELI10: (.+)$/m,'ELI10: "$1"');}, - (q:any)=>{q.question=q.question.replace('ELI10: ','ELI10: If approved, ');}, - (q:any)=>{q.question+='\nThis decision is withdrawn.';}, - (q:any)=>{q.question+='\nThis decision is "reopened".';}, - (q:any)=>{q.question+='\nThis finding applies only if approved.';}, - (q:any)=>{q.question=q.question.replace(/^([^\n]+)/,'"$1"');}, - ]){const c=cab3Change(index,edit);expect(cab3Result(c),JSON.stringify(c.questions)).toEqual({});} - }); - test('cab3 current choice cannot borrow an option or bypass native completion: '+seed,()=>{ - for(const edit of [ - (q:any)=>{q.options[0].description='No current remedy.';}, - (q:any)=>{q.options[0].description='"'+q.options[0].description.replaceAll('\n',' ')+'"';}, - (q:any)=>{q.options[0].description+='\nThis remedy is withdrawn.';}, - (q:any)=>{q.options[0].description+='\nThis remedy is "deferred".';}, - (q:any)=>{q.options[0].description+='\nThis remedy applies only if approved.';}, - ])expect(cab3Result(cab3Change(index,edit))).toEqual({}); - for(const edit of [(c:any)=>{c.answered=false;},(c:any)=>{c.failed=true;},(c:any)=>{c.answers={};},(c:any)=>{c.unansweredQuestionIndices=[0];},(c:any)=>{c.questions[0].multiSelect=true;}]){const c=cab3Call(index);edit(c);expect(cab3Result(c)).toEqual({});} - }); -} -test('cab3 store consolidation proves the current inventory and one fewer store',()=>{ - for(const edit of [ - (q:any)=>{q.question=q.question.replace('four components:','4 components:');q.options[0].label=q.options[0].label.replace('3 components:','three components:');q.options[1].label=q.options[1].label.replace('4 components:','four components:');}, - (q:any)=>{q.question=q.question.replace('Component arrangement: keep TokenStore as a separate class, or fold it into AuthCache?','How should the TokenStore and AuthCache components be arranged?');}, - (q:any)=>{q.options[0].label=q.options[0].label.replace('drop TokenStore','remove TokenStore');}, - ])expect(cab3Result(cab3Change(0,edit)).complexity).toBeDefined(); - for(const edit of [ - (q:any)=>{q.question=q.question.replace('four components:','five components:');}, - (q:any)=>{q.question=q.question.replace('AuthCache, and TokenStore.','AuthCache, and OtherStore.');}, - (q:any)=>{q.options[0].label=q.options[0].label.replace('3 components:','4 components:');}, - (q:any)=>{q.options[1].label=q.options[1].label.replace('4 components:','3 components:');}, - (q:any)=>{q.options[0].label=q.options[0].label.replace('AuthCache; drop','AuthBroker; drop');}, - (q:any)=>{q.options[0].label=q.options[0].label.replace('drop TokenStore','keep TokenStore');}, - (q:any)=>{q.options[0].description+='\nDo not remove TokenStore.';}, - (q:any)=>{q.options[0].description+='\nTokenStore remains a separate store.';}, - (q:any)=>{q.question+='\nCorrection: TokenStore has an independent persistence purpose.';}, - (q:any)=>{q.question=q.question.replace("a third layer doing the adapter's job",'an independent component with a separate contract');}, - ])expect(cab3Result(cab3Change(0,edit))).toEqual({}); -}); -test('cab3 typed error choice owns both visible known outcomes and unknown propagation',()=>{ - for(const edit of [ - (q:any)=>{q.question=q.question.replace('quietly eat one kind of error','silently swallow one error class');}, - (q:any)=>{q.options[0].label=q.options[0].label.replace('AuthResult','AuthOutcome');}, - (q:any)=>{q.options[0].description=q.options[0].description.replace('Unknown errors propagate','Unknown failures are rethrown');}, - ])expect(cab3Result(cab3Change(1,edit))['swallowed-errors']).toBeDefined(); - for(const edit of [ - (q:any)=>{q.question=q.question.replace('quietly eat one kind of error','explicitly surface each error');}, - (q:any)=>{q.question+='\nCorrection: validateAndDispatch() no longer swallows failures.';}, - (q:any)=>{q.options[0].description=q.options[0].description.replace('Unknown errors propagate','Unknown errors are swallowed');}, - (q:any)=>{q.options[0].description=q.options[0].description.replace('Every known error class becomes a visible outcome','Some known error classes are ignored');}, - (q:any)=>{q.options[0].description+='\nNot every known error class becomes a visible outcome.';}, - (q:any)=>{q.options[0].description+='\nDo not propagate unknown errors.';}, - (q:any)=>{q.options[0].description+='\nErrors are still swallowed.';}, - (q:any)=>{q.options[0].description+='\nThis remedy applies to another function.';}, - (q:any)=>{q.options[1].description+=' Unknown errors propagate.';q.options[0].description=q.options[0].description.replace('Unknown errors propagate','Unknown errors are unspecified');}, - ])expect(cab3Result(cab3Change(1,edit))).toEqual({}); -}); - - -test('cab3 choice attribution cannot bypass guards through a more explicit title',()=>{ - for(const [index,title] of [[0,'Component classes: keep TokenStore separate, or fold it into AuthCache?'],[1,'Rewrite validateAndDispatch() to fix nested swallowed errors, or add logs?']] as const){ - expect(cab3Result(cab3Change(index,q=>{q.question=q.question.replace(/^D\d+ — [^\n]+/,'D20 — '+title);}))[index===0?'complexity':'swallowed-errors']).toBeDefined(); - for(const suffix of ['\nThis decision is withdrawn.','\nThis decision is "reopened".']) expect(cab3Result(cab3Change(index,q=>{q.question=q.question.replace(/^D\d+ — [^\n]+/,'D20 — '+title)+suffix;}))).toEqual({}); - expect(cab3Result(cab3Change(index,q=>{q.question=q.question.replace(/^D\d+ — [^\n]+/,'D20 — '+title).replaceAll('PLAN.md','OTHER.md');}))).toEqual({}); - } -}); -test('cab3 owned remedies reject explicit contradictory retention and silent errors',()=>{ - for(const [index,tail] of [[0,'Keep TokenStore as a separate store.'],[0,'Retain TokenStore as a separate class.'],[1,'Known errors are still hidden.'],[1,'Unknown errors do not propagate.']] as const){ - expect(cab3Result(cab3Change(index,q=>{q.options[0]!.description+='\n'+tail;}))).toEqual({}); - expect(cab3Result(cab3Change(index,q=>{q.options[0]!.description+='\nEarlier note: "'+tail+'"';}))[index===0?'complexity':'swallowed-errors']).toBeDefined(); - } -}); - -// A whole-candidate scope question can remove one current undefined class; -// it need not restate an arrangement decision or borrow a later cumulative count. -const wholeCandidate=()=>structuredClone(currentChoiceCab3.wholeCandidateRetry.call) as NativePlanQuestionCall; -const wholeFinished=Date.parse(currentChoiceCab3.wholeCandidateRetry.captureAt); -const wholeResult=(call=wholeCandidate())=>evaluateEngSeedCoverage({status:'ready',calls:[call],assistantMessages:[]},'',0,wholeFinished).decisions; -const wholeChange=(edit:(q:NativePlanQuestionCall['questions'][number])=>void)=>{const c=wholeCandidate();edit(c.questions[0]!);c.answers={[c.questions[0]!.question]:c.questions[0]!.options[0]!.label};return c;}; -test('whole-candidate complexity: actual owned class removal needs no prior or later decision',()=>{ - const c=wholeCandidate();expect(wholeResult(c)).toEqual({complexity:`${c.sessionId}:${c.toolUseId}`}); - expect(isEngSeedDecisionAUQ(nativePlanCallFingerprint(c,0,true),[],0,wholeFinished)).toBe(true); - for(const option of c.questions[0]!.options){c.answers={[c.questions[0]!.question]:option.label};expect(wholeResult(c).complexity).toBeDefined();} -}); -test('whole-candidate complexity: presentation and equivalent current alternatives preserve identity',()=>{ - for(const edit of [ - (q:any)=>{q.question=q.question.replaceAll('`','');}, - (q:any)=>{q.question=q.question.replace('TokenStore: keep it in this PR, or defer/cut it?','TokenStore: include it in the current PR or remove it?');}, - (q:any)=>{q.question=q.question.replace('one of 4 new classes','one of four new classes');}, - (q:any)=>{q.options.reverse();}, - (q:any)=>{q.question=q.question.replace(/^D4 — /,'D42: ');}, - (q:any)=>{q.question+='\nEarlier note: "TokenStore has an independent persistence purpose."';}, - ])expect(wholeResult(wholeChange(edit)).complexity).toBeDefined(); -}); -test('whole-candidate complexity: current source, baseline and defect cannot be borrowed',()=>{ - for(const edit of [ - (q:any)=>{q.question=q.question.replaceAll('PLAN.md','OTHER.md');}, - (q:any)=>{q.question=q.question.replaceAll('PLAN.md','archive/PLAN.md');}, - (q:any)=>{q.question=q.question.replace(/^Project\/branch\/task: (.+)$/m,'Project/branch/task: "$1"');}, - (q:any)=>{q.question=q.question.replace('Project/branch/task: ','Project/branch/task: Historical example: ');}, - (q:any)=>{q.question=q.question.replace(/^ELI10: (.+)$/m,'ELI10: "$1"');}, - (q:any)=>{q.question=q.question.replace('ELI10: ','ELI10: If approved, ');}, - (q:any)=>{q.question=q.question.replace(/^([^\n]+)/,'"$1"');}, - (q:any)=>{q.question=q.question.replace('one of 4 new classes','one of 1 new classes');}, - (q:any)=>{q.question=q.question.replace('as one of 4 new classes','as an existing class');}, - (q:any)=>{q.question=q.question.replace('but never says what it does','and defines its independent persistence contract');}, - (q:any)=>{q.question=q.question.replace('already stores tokens keyed by','does not store tokens keyed by');}, - (q:any)=>{q.question+='\nCorrection: TokenStore has a documented independent persistence purpose.';}, - (q:any)=>{q.question+='\nCorrection: TokenStore is already removed from this PR.';}, - (q:any)=>{q.question+='\nThis decision is reopened.';}, - (q:any)=>{q.question+='\nThis decision is "reopened".';}, - (q:any)=>{q.question+='\nThis finding applies only if approved.';}, - ])expect(wholeResult(wholeChange(edit))).toEqual({}); -}); -test('whole-candidate complexity: removal and retained store belong to the same current option',()=>{ - const reductions=(q:any)=>q.options.filter((o:any)=>/^(?:B|C)\)/.test(o.label)); - for(const edit of [ - (q:any)=>{for(const o of reductions(q))o.description='No current remedy.';}, - (q:any)=>{for(const o of reductions(q))o.description='"'+o.description.replaceAll('\n',' ')+'"';}, - (q:any)=>{for(const o of reductions(q))o.description+='\nThis remedy is withdrawn.';}, - (q:any)=>{for(const o of reductions(q))o.description+='\nDo not remove TokenStore.';}, - (q:any)=>{for(const o of reductions(q))o.description+='\nTokenStore remains in this PR.';}, - (q:any)=>{for(const o of reductions(q))o.description+='\nThis remedy applies only if approved.';}, - (q:any)=>{q.options[0].description='Removes this class from the PR.';q.options[2].description='Adapter remains the single source of truth for cached tokens.';}, - (q:any)=>{q.options=q.options.filter((o:any)=>!o.label.startsWith('A) Include'));}, - (q:any)=>{for(const o of q.options)o.label=o.label.replace(/Defer|Cut/g,'Keep');}, - ])expect(wholeResult(wholeChange(edit))).toEqual({}); -}); -test('whole-candidate complexity: native completion, session ownership and one-seed deduplication remain required',()=>{ - const c=wholeCandidate(),guard=(x=c,prior:NativePlanQuestionCall[]=[])=>isEngSeedDecisionAUQ(nativePlanCallFingerprint(x,0,true),prior,0,wholeFinished); - expect(guard()).toBe(true);expect(guard(c,[c])).toBe(false); - for(const edit of [(x:any)=>{x.answered=false;},(x:any)=>{x.failed=true;},(x:any)=>{x.answers={};},(x:any)=>{x.unansweredQuestionIndices=[0];},(x:any)=>{x.answeredAt=new Date(wholeFinished+1).toISOString();}]){const x=wholeCandidate();edit(x);expect(guard(x)).toBe(false);expect(wholeResult(x)).toEqual({});} - const foreign=wholeCandidate();foreign.sessionId+='-foreign';foreign.toolUseId+='-other';expect(guard(c,[foreign])).toBe(false); - const other=wholeCandidate();other.toolUseId+='-other';expect(guard(c,[other])).toBe(false); -}); - -for(const tail of ['This option never removes an undefined class from this PR.','This is not one fewer file/class.']) - test('whole-candidate complexity: declarative negation '+tail,()=>{ - const x=wholeChange(q=>{for(const o of q.options.filter(o=>/^(?:B|C)\)/.test(o.label)))o.description=tail+' Adapter remains the single source of truth for cached tokens.';}); - expect(wholeResult(x)).toEqual({}); - }); - - -// Original packet identities and every answer are retained. Single-question -// projections below isolate semantic controls; they never re-credit the paid run. -const packet = (n:number) => structuredClone(nativePackets.calls[n]) as NativePlanQuestionCall; -const packetResult = (calls:NativePlanQuestionCall[]) => evaluateEngSeedCoverage( - {status:'ready',calls,assistantMessages:[]},'',nativePackets.startedAt,nativePackets.finishedAt); -const packetGuard = (c:NativePlanQuestionCall,prior:NativePlanQuestionCall[]=[]) => isEngSeedDecisionAUQ( - nativePlanCallFingerprint(c,0,true),prior,nativePackets.startedAt,nativePackets.finishedAt); -const singlePacketQuestion = (n:number,index:number) => { - const c=packet(n),q=c.questions[index]!; - c.questions=[q];c.answers={[q.question]:c.answers![q.question]!};return c; -}; -const editedPacketQuestion=(n:number,index:number,edit:(q:NativePlanQuestionCall['questions'][number])=>void)=>{ - const c=singlePacketQuestion(n,index),q=c.questions[0]!,chosen=q.options.findIndex(o=>o.label===c.answers![q.question]); - edit(q);c.answers={[q.question]:q.options[chosen]!.label};return c; -}; - -test('b955 native packets: actual whole-call options authenticate one seed with independently answered unrelated tabs',()=>{ - const c=packet(1),fp=nativePlanCallFingerprint(c,0,true); - expect(fp.options).toHaveLength(c.questions.reduce((n,q)=>n+q.options.length,0)); - expect(packetResult([c]).decisions['shared-cache']).toBe(`${c.sessionId}:${c.toolUseId}`); - expect(packetGuard(c)).toBe(true); - for(const position of [0,fp.options.length-1]){ - const bad=structuredClone(fp);bad.options[position]!.label='forged option'; - expect(isEngSeedDecisionAUQ(bad,[],nativePackets.startedAt,nativePackets.finishedAt)).toBe(false); - } - const firstOnly={...fp,options:fp.options.slice(0,c.questions[0]!.options.length)}; - expect(isEngSeedDecisionAUQ(firstOnly,[],nativePackets.startedAt,nativePackets.finishedAt)).toBe(false); - const reordered=packet(1);reordered.questions.reverse();expect(packetGuard(reordered)).toBe(true); -}); - -test('b955 native packets: current structure alternatives offer a real reduction with unchanged feature choices',()=>{ - const c=packet(0);expect(packetGuard(c)).toBe(true); - expect(packetResult([c]).decisions).toEqual({complexity:`${c.sessionId}:${c.toolUseId}`}); -}); -test('b955 native packets: original error question owns each swallowed class and an offered flatten/typed/rethrow remedy',()=>{ - const c=singlePacketQuestion(2,0);expect(packetGuard(c)).toBe(true); - expect(packetResult([c]).decisions).toEqual({'swallowed-errors':`${c.sessionId}:${c.toolUseId}`}); -}); -test('b955 native packets: one acknowledged packet containing two seeds cannot supply either distinct decision',()=>{ - const c=packet(2);expect(packetGuard(c)).toBe(false);expect(packetResult([c]).decisions).toEqual({}); - const actual=packetResult([packet(0),packet(1),c]); - expect(Object.keys(actual.decisions).sort()).toEqual(['complexity','shared-cache']); - expect(actual.missing).toEqual(['swallowed-errors','sequential-idp']); - expect(nativePackets.originalOutcome).toBe('no_review_questions'); -}); -for(const [name,edit] of Object.entries({ - 'foreign PLAN path':(q:any)=>{q.question=q.question.replaceAll('PLAN.md','archive/PLAN.md');}, - 'foreign primary source':(q:any)=>{q.question=q.question.replace('PLAN.md Multi-tenant Auth Refactor','OTHER.md Other Refactor; compare PLAN.md Multi-tenant Auth Refactor');}, - 'quoted source':(q:any)=>{q.question=q.question.replace(/^Project\/branch\/task: (.+)$/m,'Project/branch/task: "$1"');}, - 'historical source':(q:any)=>{q.question=q.question.replace('Project/branch/task: ','Project/branch/task: Historical example: ');}, - 'quoted explanation':(q:any)=>{q.question=q.question.replace(/^ELI10: (.+)$/m,'ELI10: "$1"');}, - 'duplicated source':(q:any)=>{q.question+='\nProject/branch/task: OTHER.md';}, - 'conditional finding':(q:any)=>{q.question=q.question.replace('ELI10: ','ELI10: If approved, ');}, - 'withdrawn decision':(q:any)=>{q.question+='\nThis decision is withdrawn.';}, - 'current quoted withdrawal':(q:any)=>{q.question+='\nThis decision is "withdrawn".';}, - 'reopened decision':(q:any)=>{q.question+='\nThis decision is reopened.';}, -}))test('b955 native packets reject '+name,()=>{ - for(const [n,index] of [[0,0],[2,0]])expect(packetResult([editedPacketQuestion(n!,index!,edit)]).decisions).toEqual({}); -}); -for(const [name,edit] of Object.entries({ - 'numeric counts in explanation':(q:any)=>{q.question=q.question.replace('A) three classes:','A) 3 classes:').replace('B) two classes:','B) 2 classes:');}, - 'renamed structure title':(q:any)=>{q.question=q.question.replace(q.question.split('\n')[0],'D7 — Which component arrangement preserves the accepted feature choices?');}, - 'reordered native options':(q:any)=>{q.options.reverse();}, - 'historical contradiction inert':(q:any)=>{q.question+='\nEarlier note: "AuthCache now has independent behavior."';}, -}))test('b955 structure comparison accepts '+name,()=>expect(packetResult([editedPacketQuestion(0,0,edit)]).decisions.complexity).toBeDefined()); -for(const [name,edit] of Object.entries({ - 'no fixed feature choices':(q:any)=>{q.question=q.question.replace('deliver the same features (D4-D6 held fixed, legacy flow untouched behind a flag)','deliver different features');}, - 'foreign retained service':(q:any)=>{q.question=q.question.replaceAll('SessionMint','OtherService');}, - 'no current facade':(q:any)=>{q.question=q.question.replace('AuthCache as the one facade over the existing adapter','a new component with an unknown role');}, - 'equal option counts':(q:any)=>{q.options[1].label=q.options[1].label.replace('2 classes','3 classes');}, - 'mismatched body count':(q:any)=>{q.question=q.question.replace('B) two classes:','B) three classes:');}, - 'no offered reduction':(q:any)=>{q.options[1]={label:'B) Discuss the cache',description:'No change yet.'};}, - 'no same-option adapter reuse':(q:any)=>{q.options[1].label=q.options[1].label.replace('services use adapter directly','new services');q.options[1].description='Unspecified behavior.';}, - 'remedy borrowed from unselected option':(q:any)=>{q.options[0].description+=' Services use adapter directly.';q.options[1].label='B) 2 classes';q.options[1].description='Unspecified behavior.';}, - 'foreign comparison letter':(q:any)=>{q.question=q.question.replace('B) two classes:','Z) two classes:');}, - 'native baseline is another component':(q:any)=>{q.options[0].label=q.options[0].label.replace('AuthCache facade','ForeignCache wrapper');}, - 'foreign offered remedy':(q:any)=>{q.options[1].description+=' This remedy applies to another project.';}, - 'same-option retains facade':(q:any)=>{q.options[1].description+=' Correction: Keep the AuthCache facade.';}, - 'matching explanation retains facade':(q:any)=>{q.question=q.question.replace('C) one service:', 'But keep the AuthCache facade. C) one service:');}, - 'matching explanation cancels drop':(q:any)=>{q.question=q.question.replace('C) one service:', 'Do not drop the facade. C) one service:');}, - 'same-option negated removal':(q:any)=>{q.options[1].description+=' Do not drop the facade.';}, - 'same-option replaced adapter':(q:any)=>{q.options[1].description+=' Replace the existing adapter.';}, - 'same-option withdrawn':(q:any)=>{q.options[1].description+=' This option is withdrawn.';}, - 'independent current facade':(q:any)=>{q.question+='\nCorrection: AuthCache now has independent behavior.';}, - 'unapproved additional feature':(q:any)=>{q.question+='\nCorrection: The smaller arrangement changes the accepted feature choices.';}, -}))test('b955 structure comparison rejects '+name,()=>expect(packetResult([editedPacketQuestion(0,0,edit)]).decisions).toEqual({})); -for(const [name,edit] of Object.entries({ - 'current defect equivalent wording':(q:any)=>{q.question=q.question.replace('where each catch quietly eats one kind of error','where every catch silently swallows a different error class');}, - 'typed error name changes':(q:any)=>{q.options[0].label=q.options[0].label.replace('AuthError','ValidationFailure');}, - 'same-option propagation wording':(q:any)=>{q.options[0].label=q.options[0].label.replace('rethrow','propagate');}, - 'reordered options':(q:any)=>{q.options.reverse();}, - 'historical correction inert':(q:any)=>{q.question+='\nEarlier note: "validateAndDispatch() no longer swallows failures."';}, -}))test('b955 current error choice accepts '+name,()=>expect(packetResult([editedPacketQuestion(2,0,edit)]).decisions['swallowed-errors']).toBeDefined()); -for(const [name,edit] of Object.entries({ - 'non-swallowing current behavior':(q:any)=>{q.question=q.question.replace('where each catch quietly eats one kind of error','where every catch already surfaces each error');}, - 'typed name alone':(q:any)=>{q.options[0]={label:'A) Typed AuthError',description:'Add the named type.'};}, - 'no propagation':(q:any)=>{q.options[0].label=q.options[0].label.replace(', rethrow','');}, - 'no flattening':(q:any)=>{q.options[0].label=q.options[0].label.replace('Flatten + ','');q.options[0].description=q.options[0].description.replace('Function shrinks to sequential named steps','Function remains deeply nested');}, - 'partial classes':(q:any)=>{q.options[0].description=q.options[0].description.replace('Each former swallowed class','Some former swallowed classes');}, - 'missing typed result':(q:any)=>{q.options[0].description=q.options[0].description.replace('becomes a typed error','is logged');}, - 'borrowed class coverage':(q:any)=>{q.options[1].description+=' '+q.options[0].description;q.options[0].description='Add the named type.';}, - 'quoted remedy':(q:any)=>{q.options[0].label='"'+q.options[0].label+'"';q.options[0].description='"'+q.options[0].description+'"';}, - 'negated propagation':(q:any)=>{q.options[0].description+=' Do not rethrow errors.';}, - 'declarative negation':(q:any)=>{q.options[0].description+=' This option does not rethrow errors.';}, - 'errors still swallowed':(q:any)=>{q.options[0].description+=' Correction: Errors are still swallowed.';}, - 'incomplete mapping':(q:any)=>{q.options[0].description+=' Not every failure class has a named outcome.';}, - 'partial former swallowed classes':(q:any)=>{q.options[0].description+=' Only some former swallowed classes become a typed error.';}, - 'negated former class coverage':(q:any)=>{q.options[0].description+=' Not every previously swallowed class becomes a typed error.';}, - 'foreign remedy':(q:any)=>{q.options[0].description+=' This remedy applies to another function.';}, - 'withdrawn remedy':(q:any)=>{q.options[0].description+=' This option is withdrawn.';}, - 'already fixed current source':(q:any)=>{q.question+='\nCorrection: validateAndDispatch() now rethrows every error.';}, -}))test('b955 current error choice rejects '+name,()=>expect(packetResult([editedPacketQuestion(2,0,edit)]).decisions).toEqual({})); -test('b955 whole-call adapter keeps native completion and fingerprint integrity checks',()=>{ - for(const edit of [(c:any)=>{c.answered=false;},(c:any)=>{c.failed=true;},(c:any)=>{delete c.answers[c.questions[1].question];c.unansweredQuestionIndices=[1];},(c:any)=>{c.answers[c.questions[2].question]='Not offered';},(c:any)=>{c.answeredAt=new Date(nativePackets.finishedAt+1).toISOString();}]){ - const c=packet(1);edit(c);expect(packetGuard(c)).toBe(false); - } - const c=packet(1);expect(packetGuard(c,[c])).toBe(false); - const foreign=packet(0);foreign.sessionId+='-foreign';expect(packetGuard(c,[foreign])).toBe(false); - const reask=packet(1);reask.toolUseId+='-reask';expect(packetGuard(reask,[c])).toBe(false); -}); diff --git a/test/eng-next-handoff-ah.test.ts b/test/eng-next-handoff-ah.test.ts index 3ccdc9e46..0401c9e50 100644 --- a/test/eng-next-handoff-ah.test.ts +++ b/test/eng-next-handoff-ah.test.ts @@ -3,139 +3,9 @@ import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; import actual from './fixtures/eng-next-handoff-ah.json'; -import { isEngCompletionHandoff } from './helpers/eng-completion-handoff'; -import { hasNativePlanTerminal, nativePlanCallFingerprint, planCountQuestionPhase } from './helpers/claude-pty-runner'; -import type { NativePlanQuestionCall, PlanCountTranscript } from './helpers/plan-count-transcript'; -import { readPlanCountTranscript } from './helpers/plan-count-transcript'; +import { hasNativePlanTerminal } from './helpers/claude-pty-runner'; +import type { PlanCountTranscript } from './helpers/plan-count-transcript'; import { isCurrentPlanApprovalScreen } from './helpers/plan-count-pending-exit'; -import { E2E_TOUCHFILES, matchGlob } from './helpers/touchfiles'; - -const call = () => structuredClone(actual.fingerprint.nativeCall) as NativePlanQuestionCall; -const fp = (c = call()) => nativePlanCallFingerprint(c, 0, false); -const accepts = (c = call(), plan = actual.plan) => isEngCompletionHandoff(fp(c), plan); - -function maintenanceRecap() { - const make = (id: string, header: string, text: string, selected: string, description: string): NativePlanQuestionCall => ({ - sessionId:'maintenance-session', toolUseId:id, questions:[{header,question:text,multiSelect:false, - options:[{label:selected,description},{label:'Skip',description:'Do not approve this action.'}]}], - answered:true, failed:false, answers:{[text]:selected}, unansweredQuestionIndices:[], answeredAt:'2026-09-11T00:00:01Z', - }); - const routing = make('routing','Routing',"Add gstack skill routing rules to CLAUDE.md?",'Add routing rules to CLAUDE.md (recommended)','Append the routing rules after review.'); - const policy = make('policy','TODO 1','D7 — TODO 1: RetryPolicy needs a follow-up.','7A) Add to TODOS.md (recommended)',"Captured in the plan's TODOS section now; write it after exit."); - const cleanup = make('cleanup','TODO 2','D8 — TODO 2: Remove LegacyBridge after rollout.','8A) Add to TODOS.md (recommended)',"Captured in the plan's TODOS section now; write it after exit."); - const next = make('next','Next step','D9 — Next steps. Eng review is CLEARED. There is no UI scope. CEO review is optional. What next?', - 'Ready to implement — run /ship when done (recommended)','Exit plan mode with the reviewed plan. Post-exit: append routing rules to CLAUDE.md and create TODOS.md with the two accepted items.'); - next.answeredAt='2026-09-11T00:00:02Z'; next.questions[0]!.options[1]={label:'Run /plan-ceo-review',description:'Optional strategy review.'}; - return {next, prior:[routing,policy,cleanup], plan:'## TODOS\n### Revisit RetryPolicy\nAn approved follow-up.\n### Remove LegacyBridge\nAfter rollout.\n## Implementation Tasks\n'}; -} -test('completed navigation can recap earlier approved routing and published TODOs', () => { - const a=maintenanceRecap(), check=(x= a)=>isEngCompletionHandoff(fp(x.next),x.plan,x.prior); - expect(check()).toBe(true); - const renamed=structuredClone(a); renamed.plan=renamed.plan.replaceAll('RetryPolicy','TenantPolicy'); - question(renamed.prior[1]!,s=>s.replaceAll('RetryPolicy','TenantPolicy')); expect(check(renamed)).toBe(true); - const reworded=structuredClone(a);question(reworded.next,s=>s.replace('D9 — Next steps. Eng review is CLEARED','D14: Next step: Engineering review is complete')); - reworded.next.questions[0]!.header='Next steps';reworded.next.questions[0]!.options[0]!.description='Exit plan mode with the reviewed plan. After exiting: write TODOS.md with 2 accepted items; add gstack routing rules to CLAUDE.md.'; - expect(check(reworded)).toBe(true); - const batched=structuredClone(a);batched.prior[1]!.questions.push(...batched.prior[2]!.questions); - Object.assign(batched.prior[1]!.answers,batched.prior[2]!.answers);batched.prior.pop();expect(check(batched)).toBe(true); - for(const mutate of [ - (x:typeof a)=>{x.prior.shift();}, - (x:typeof a)=>{x.prior[0]!.sessionId='foreign';}, - (x:typeof a)=>{x.prior[0]!.failed=true;}, - (x:typeof a)=>{x.prior[0]!.answeredAt=x.next.answeredAt;}, - (x:typeof a)=>{x.prior[0]!.unansweredQuestionIndices=[0];}, - (x:typeof a)=>{x.prior.push(structuredClone(x.prior[0]!));}, - (x:typeof a)=>{x.prior[0]!.questions[0]!.options[1]=structuredClone(x.prior[0]!.questions[0]!.options[0]!);}, - (x:typeof a)=>{const revoked=structuredClone(x.prior[0]!);revoked.toolUseId='revoked';revoked.answers![revoked.questions[0]!.question]='Skip';x.prior.push(revoked);}, - (x:typeof a)=>{x.prior[1]!.answers![x.prior[1]!.questions[0]!.question]='Skip';}, - (x:typeof a)=>{x.prior[1]!.questions[0]!.options[0]!.description='A new proposed TODO.';}, - (x:typeof a)=>{question(x.prior[1]!,s=>s+' This approval is withdrawn.');}, - (x:typeof a)=>{question(x.next,s=>'Example: '+s);}, - (x:typeof a)=>{question(x.next,s=>s+' This review is cancelled.');}, - (x:typeof a)=>{question(x.next,s=>s.replace('is CLEARED','will be CLEARED'));}, - (x:typeof a)=>{question(x.next,s=>s+' Only if more tests pass.');}, - (x:typeof a)=>{x.next.questions[0]!.options[0]!.description+=' Add another requirement.';}, - (x:typeof a)=>{x.next.questions[0]!.options[0]!.description=x.next.questions[0]!.options[0]!.description!.replace('two','three');}, - (x:typeof a)=>{x.plan=x.plan.replace('## TODOS','## Historical TODOs');}, - (x:typeof a)=>{x.plan=x.plan.replace('RetryPolicy','OtherPolicy');}, - (x:typeof a)=>{x.plan=x.plan.replace('An approved follow-up.','This TODO is withdrawn.');}, - (x:typeof a)=>{x.plan='```md\n'+x.plan+'\n```';}, - ]){const x=structuredClone(a);mutate(x);expect(check(x)).toBe(false);} - expect(isEngCompletionHandoff(fp(a.next),a.plan)).toBe(false); -}); -function question(c: NativePlanQuestionCall, f: (s: string) => string) { - const q = c.questions[0]!, answer = c.answers![q.question]; - q.question = f(q.question); c.answers = { [q.question]: answer! }; return c; -} - -test('actual completed Next navigation is administrative and never starts review', () => { - expect(accepts()).toBe(true); - for (const started of [false, true]) { - expect(planCountQuestionPhase(fp(), started, () => false, undefined, undefined, - f => isEngCompletionHandoff(f, actual.plan))).toEqual({ preReview: false, reviewStarted: started, administrative: 'completion-handoff' }); - } -}); - -test('published confirmation and characterization references do not introduce work', () => { - expect(actual.source.stat.mtimeMs).toBeLessThan(Date.parse(call().answeredAt!)); - expect(accepts(call(), actual.plan.replaceAll('P0', 'P7'))).toBe(true); - const c = call(); c.questions[0]!.options.reverse(); - expect(accepts(c)).toBe(true); - c.answers![c.questions[0]!.question] = c.questions[0]!.options[0]!.label; - expect(accepts(c)).toBe(true); - expect(accepts(call(), actual.plan.replace(' - Surfaced by: Architecture issue 3 (D7)', ' - Correction: T2 is cancelled.\n - Surfaced by: Architecture issue 3 (D7)'))).toBe(true); - expect(accepts(call(), actual.plan.replace('Write characterization tests for `legacyAuthFlow()` before any rewrite', 'Write characterization tests for `legacyAuthFlow()` before any rewrite\nVerify expired and revoked tokens are rejected.'))).toBe(true); - expect(accepts(call(), actual.plan.replace('Invariants and Latency target above.', 'Invariants and Latency target above.\nKeep a record of rejected alternatives after the author confirms Context.'))).toBe(true); -}); - -test('incomplete, foreign, ambiguous and changed choices cannot be administrative', () => { - for (const mutate of [ - (c: NativePlanQuestionCall) => { c.answered = false; }, - (c: NativePlanQuestionCall) => { c.failed = true; }, - (c: NativePlanQuestionCall) => { c.answeredAt = 'invalid'; }, - (c: NativePlanQuestionCall) => { c.unansweredQuestionIndices = [0]; }, - (c: NativePlanQuestionCall) => { c.answers = {}; }, - (c: NativePlanQuestionCall) => { c.answers![c.questions[0]!.question] = 'unoffered'; }, - (c: NativePlanQuestionCall) => { c.questions[0]!.multiSelect = true; }, - (c: NativePlanQuestionCall) => { c.questions.push(structuredClone(c.questions[0]!)); }, - (c: NativePlanQuestionCall) => { c.questions[0]!.options.push({ label: 'Add another requirement' }); }, - (c: NativePlanQuestionCall) => { c.questions[0]!.options[0]!.description += ' Add a new datastore first.'; }, - (c: NativePlanQuestionCall) => { c.questions[0]!.options[1]!.description = 'Change the implementation architecture first.'; }, - (c: NativePlanQuestionCall) => { c.questions[0]!.options[0]!.description = c.questions[0]!.options[0]!.description!.replace('T1', 'T99'); }, - ]) { const c = call(); mutate(c); expect(accepts(c)).toBe(false); } - expect(isEngCompletionHandoff({ ...fp(), signature: 'foreign:call' }, actual.plan)).toBe(false); - expect(isEngCompletionHandoff({ ...fp(), nativeQuestionIndex: 1 }, actual.plan)).toBe(false); - expect(isEngCompletionHandoff({ ...fp(), options: [] }, actual.plan)).toBe(false); -}); - -test('nonasserted, prospective, conditional and reopened navigation stays substantive', () => { - for (const text of [ - '> ', 'Example: ', 'An unproven hypothesis. ', '```text\n', - ]) expect(accepts(question(call(), s => text + s))).toBe(false); - for (const change of [ - (s: string) => s.replace('all required reviews are complete', 'all required reviews will be complete'), - (s: string) => s.replace('all required reviews are complete', 'all required reviews are not complete'), - (s: string) => s.replace('all required reviews are complete', 'all required reviews are complete if more tests pass'), - (s: string) => s + '\nA new implementation prerequisite is required.', - (s: string) => s.replace('Recommendation: A', 'Recommendation: C'), - ]) expect(accepts(question(call(), change))).toBe(false); -}); - -test('missing, refuted or quoted published prerequisites/tasks cannot be borrowed', () => { - for (const plan of [ - '', '```markdown\n' + actual.plan + '\n```', actual.plan.split('\n').map(s => '> ' + s).join('\n'), - actual.plan.replace('## Context', '## Example context'), - actual.plan.replace('Implementation does not start until the author confirms', 'Implementation starts without the author confirming'), - actual.plan.replace('### Prerequisite P0', '### Example prerequisite P0'), - actual.plan.replace('## Implementation Tasks', '## Historical Tasks'), - actual.plan.replace('Write characterization tests for `legacyAuthFlow()` before any rewrite', 'Write characterization tests after rewriting `legacyAuthFlow()`'), - actual.plan.replace('**T1 (P1', '**T99 (P1'), - actual.plan.replace('## Context', 'Example only:\n## Context'), - actual.plan.replace('## Implementation Tasks', 'Example only:\n## Implementation Tasks'), - actual.plan.replace('Invariants and Latency target above.', 'Invariants and Latency target above.\nCorrection: Prerequisite P0 is cancelled; the author no longer needs to confirm Context.'), - actual.plan.replace('Write characterization tests for `legacyAuthFlow()` before any rewrite', 'Write characterization tests for `legacyAuthFlow()` before any rewrite\nCorrection: T1 is cancelled; no characterization tests are required.'), - ]) expect(accepts(call(), plan)).toBe(false); -}); test('exact final exit/report replay retains all freshness, identity and answer gates', () => { const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-eng-next-ah-')); @@ -147,7 +17,7 @@ test('exact final exit/report replay retains all freshness, identity and answer Date.now = () => Date.parse(actual.captureAt); const t = structuredClone(actual.transcript) as PlanCountTranscript; const id = actual.fingerprint.signature; - const admin = new Set(accepts() ? [id] : []); + const admin = new Set([id]); const check = (v = t, a = admin) => hasNativePlanTerminal(v, file, actual.startedAt, 'plan_ready', a); expect(isCurrentPlanApprovalScreen(actual.screen)).toBe(true); expect(check()).toBe(true); @@ -168,193 +38,14 @@ test('exact final exit/report replay retains all freshness, identity and answer } finally { Date.now = now; fs.rmSync(dir, { recursive: true, force: true }); } }); -test('new handoff evidence belongs to its existing paid caller', () => { - for (const file of ['test/eng-next-handoff-ah.test.ts', 'test/fixtures/eng-next-handoff-ah.json']) { - const owners = Object.entries(E2E_TOUCHFILES).filter(([, globs]) => globs.some(glob => matchGlob(file, glob))).map(([name]) => name); - expect(owners).toEqual(['plan-eng-finding-count']); - } -}); - const b176 = actual.sourceBoundB176; const recorded = () => structuredClone(b176.transcript) as PlanCountTranscript; -const recordedCall = () => recorded().calls.at(-1)!; -const recordedPrior = () => recorded().calls.slice(0,-1); -const recordedCheck = (c=recordedCall(),plan=b176.plan,prior=recordedPrior()) => - isEngCompletionHandoff(nativePlanCallFingerprint(c,Date.parse(b176.capturedAt),true),plan,prior); -const changedApproval = [ - 'D1 approval is revoked.', - 'This decision is reopened.', - 'Routing rules: pending approval.', -]; -test.each(changedApproval)('current navigation cannot withdraw its referenced approval: %s',text=>{ - const c=recordedCall();question(c,s=>s+'\n'+text);expect(recordedCheck(c)).toBe(false); - const option=recordedCall();option.questions[0]!.options[1]!.description+=' '+text;expect(recordedCheck(option)).toBe(false); - const prior=recordedPrior();question(prior[0]!,s=>s+'\n'+text);expect(recordedCheck(recordedCall(),b176.plan,prior)).toBe(false); - expect(recordedCheck(recordedCall(),b176.plan+'\n'+text)).toBe(false); -}); -test.each([ - 'The implementation now requires a production deployment before fixtures.', - 'Implementation needs a production deployment before fixtures.', - 'A production deployment is now required before fixtures.', - 'T2 depends on a production deployment before fixtures.', -])('current navigation cannot add an unbound declarative obligation: %s',text=>{ - const c=recordedCall();question(c,s=>s+'\n'+text);expect(recordedCheck(c)).toBe(false); - const option=recordedCall();option.questions[0]!.options[1]!.description+=' '+text;expect(recordedCheck(option)).toBe(false); -}); -test.each(['Example only:','Sample plan:','Hypothetical:','Source excerpt:'])('report evidence cannot borrow a source-introduced owner: %s',prefix=>{ - for(const heading of ['# Plan:','## GSTACK REVIEW REPORT','## Decision ledger','## Implementation Tasks','## Accepted TODOs','## Implementation order']){ - expect(b176.plan.includes(heading),heading).toBe(true); - expect(recordedCheck(recordedCall(),b176.plan.replace(heading,prefix+'\n'+heading)),heading).toBe(false); - } -}); -test('an inactive source section cannot own or invalidate the next current sibling',()=>{ - const sample='Source excerpt:\n## Old task illustration\nD1 approval is revoked.\nT99 is an illustration.\n\n'; - expect(recordedCheck(recordedCall(),b176.plan.replace('## Implementation Tasks',sample+'## Implementation Tasks'))).toBe(true); -}); -test('both maintenance forms retain the earlier native-answer cardinality and uniqueness checks',()=>{ - for(const mutate of [ - (c:NativePlanQuestionCall)=>{for(let n=0;n<5;n++){const q=structuredClone(c.questions[0]!);q.question+=' extra '+n;c.questions.push(q);c.answers![q.question]=q.options[0]!.label;}}, - (c:NativePlanQuestionCall)=>{for(let n=0;n<4;n++)c.questions[0]!.options.push({label:'Other '+n});}, - (c:NativePlanQuestionCall)=>{c.questions.push(structuredClone(c.questions[0]!));c.answers!['unowned-key']='unused';}, - ]){ - const prior=recordedPrior();mutate(prior[0]!);expect(recordedCheck(recordedCall(),b176.plan,prior)).toBe(false); - const old=maintenanceRecap();mutate(old.prior[0]!);expect(isEngCompletionHandoff(fp(old.next),old.plan,old.prior)).toBe(false); - } -}); -test('review-discovered current changes and example evidence cannot release the pending exit',()=>{ - const dir=fs.mkdtempSync(path.join(os.tmpdir(),'eng-b176-review-')),file=path.join(dir,'report.md'),now=Date.now; - try{ - Date.now=()=>Date.parse(b176.capturedAt); - const examples=[ - {plan:b176.plan,addition:undefined,expected:true}, - {plan:b176.plan,addition:'D1 approval is revoked.',expected:false}, - {plan:b176.plan,addition:'The implementation now requires a production deployment before fixtures.',expected:false}, - {plan:'Example only:\n'+b176.plan,addition:undefined,expected:false}, - {plan:b176.plan.replace('## Implementation Tasks','Example only:\n## Implementation Tasks'),addition:undefined,expected:false}, - ]; - for(const {plan,addition,expected} of examples){ - const t=recorded(),c=t.calls.at(-1)!;if(addition)question(c,s=>s+'\n'+addition); - const f=nativePlanCallFingerprint(c,Date.parse(b176.capturedAt),true); - const administrative=new Set(isEngCompletionHandoff(f,plan,t.calls.slice(0,-1))?[f.signature]:[]); - fs.writeFileSync(file,plan);fs.utimesSync(file,b176.sourceReport.mtimeMs/1000,b176.sourceReport.mtimeMs/1000); - expect(hasNativePlanTerminal(t,file,b176.startedAt,'plan_ready',administrative)).toBe(expected); - } - }finally{Date.now=now;fs.rmSync(dir,{recursive:true,force:true});} -}); -test('actual b176 answered navigation recaps owned prior maintenance and the published first task',()=>{ - expect(recordedCheck()).toBe(true); - expect(b176.originalOutcome).toBe('CANCELLED'); - expect(b176.originalD12PreReview).toBe(true); - expect(b176.originalNativeTerminal).toBe(false); - for(const started of [true,false])expect(planCountQuestionPhase(nativePlanCallFingerprint(recordedCall(),Date.parse(b176.capturedAt),true),started,()=>false,undefined,undefined, - f=>isEngCompletionHandoff(f,b176.plan,recordedPrior()))).toEqual({preReview:false,reviewStarted:started,administrative:'completion-handoff'}); -}); -test('recorded navigation is keyed by current references, not the observed numbering or optional label',()=>{ - const c=recordedCall();c.questions[0]!.options[1]!.label='Run /plan-ceo-review (optional)';expect(recordedCheck(c)).toBe(true); - const renamed=JSON.parse(JSON.stringify({c:recordedCall(),plan:b176.plan,prior:recordedPrior()}).replaceAll('D10','D20').replaceAll('D11','D21')); - expect(recordedCheck(renamed.c,renamed.plan,renamed.prior)).toBe(true); - const wording=recordedCall();question(wording,s=>s.replace('engineering review is done','engineering review is complete').replace('every finding has an approved fix','all decisions are settled')); - wording.questions[0]!.options[0]!.description=wording.questions[0]!.options[0]!.description!.replace('start with','begin with').replace('then write','then create'); - expect(recordedCheck(wording)).toBe(true); -}); -test.each(['missing-answer','failed','pending-tab','unoffered','future','bad-clock','wrong-fingerprint','wrong-option','extra-question','extra-option','ceo-selected'])('recorded handoff rejects incomplete or conflicting native state: %s',kind=>{ - const c=recordedCall(); - if(kind==='missing-answer'){c.answered=false;c.answers={};} - if(kind==='failed')c.failed=true; - if(kind==='pending-tab')c.unansweredQuestionIndices=[0]; - if(kind==='unoffered')c.answers![c.questions[0]!.question]='Unstated route'; - if(kind==='future')c.answeredAt=new Date(Date.now()+60_000).toISOString(); - if(kind==='bad-clock')c.answeredAt='invalid'; - if(kind==='extra-question')c.questions.push(structuredClone(c.questions[0]!)); - if(kind==='extra-option')c.questions[0]!.options.push({label:'Add Redis',description:'New work.'}); - if(kind==='ceo-selected')c.answers![c.questions[0]!.question]=c.questions[0]!.options[1]!.label; - const f=nativePlanCallFingerprint(c,Date.parse(b176.capturedAt),true); - if(kind==='wrong-fingerprint')f.signature='foreign:call'; - if(kind==='wrong-option')f.options[0]!.label='Other route'; - expect(isEngCompletionHandoff(f,b176.plan,recordedPrior())).toBe(false); -}); -test.each(['future-review','negative-review','conditional-review','historical','quoted','revoked-review','unapproved-finding','new-command','quoted-command','new-prerequisite','unknown-task','wrong-first-task','unapproved-routing','unapproved-todo'])('recorded next-step content cannot hide new or incomplete work: %s',kind=>{ - const c=recordedCall(); - if(kind==='future-review')question(c,s=>s.replace('engineering review is done','engineering review will be done')); - if(kind==='negative-review')question(c,s=>s.replace('engineering review is done','engineering review is not done')); - if(kind==='conditional-review')question(c,s=>s.replace('engineering review is done','engineering review is done if new tests pass')); - if(kind==='historical')question(c,s=>'Historical: '+s); - if(kind==='quoted')question(c,s=>'> '+s); - if(kind==='revoked-review')question(c,s=>s+'\nThe engineering review is reopened.'); - if(kind==='unapproved-finding')question(c,s=>s.replace('every finding has an approved fix','not every finding has an approved fix')); - if(kind==='new-command')question(c,s=>s+'\nThen add a new datastore.'); - if(kind==='quoted-command')c.questions[0]!.options[1]!.description+=' Also "deploy production now".'; - if(kind==='new-prerequisite')question(c,s=>s+'\nA new prerequisite is required before implementation.'); - if(kind==='unknown-task')question(c,s=>s.replace('T1–T9','T1–T99')); - if(kind==='wrong-first-task')c.questions[0]!.options[0]!.description=c.questions[0]!.options[0]!.description!.replace('T1 fixtures','T2 fixtures'); - if(kind==='unapproved-routing')c.questions[0]!.options[0]!.description=c.questions[0]!.options[0]!.description!.replace('(D1)','(D2)'); - if(kind==='unapproved-todo')c.questions[0]!.options[0]!.description=c.questions[0]!.options[0]!.description!.replace('D10/D11','D10/D99'); - expect(recordedCheck(c),kind).toBe(false); -}); -test.each(['missing-routing','missing-todo','failed-answer','foreign-session','future-answer','duplicate-answer','declined-todo','declined-routing','foreign-source','changed-remedy','withdrawn'])('recorded maintenance remains bound to complete earlier approvals: %s',kind=>{ - const prior=recordedPrior(); - if(kind==='missing-routing')prior.shift(); - if(kind==='missing-todo')prior.pop(); - if(kind==='failed-answer')prior.at(-1)!.failed=true; - if(kind==='foreign-session')prior.at(-1)!.sessionId='foreign'; - if(kind==='future-answer')prior.at(-1)!.answeredAt=recordedCall().answeredAt; - if(kind==='duplicate-answer')prior.push(structuredClone(prior[0]!)); - if(kind==='declined-todo')prior.at(-1)!.answers![prior.at(-1)!.questions[0]!.question]='Skip — not valuable enough'; - if(kind==='declined-routing')prior[0]!.answers![prior[0]!.questions[0]!.question]="No thanks, I'll invoke skills manually"; - if(kind==='foreign-source')question(prior.at(-1)!,s=>s.replace('PLAN.md','FOREIGN.md')); - if(kind==='changed-remedy'){const c=prior.find(c=>c.questions[0]!.header==='Wiring')!;c.answers![c.questions[0]!.question]=c.questions[0]!.options[1]!.label;} - if(kind==='withdrawn')question(prior.at(-1)!,s=>s+'\nThis decision is withdrawn.'); - expect(recordedCheck(recordedCall(),b176.plan,prior),kind).toBe(false); -}); -test.each(['foreign-title','foreign-project','foreign-branch','foreign-source','archived','quoted','fenced','duplicate-owner','pending-remedy','changed-answer','missing-todo','wrong-todo-count','missing-task','changed-first-task','late-first-task','incomplete-report','negative-report'])('recorded handoff cannot borrow a foreign or incomplete report: %s',kind=>{ - let plan=b176.plan; - if(kind==='foreign-title')plan=plan.replaceAll('Multi-tenant Auth Refactor','Different refactor'); - if(kind==='foreign-project')plan=plan.replace('gstack-plan-count-G18mVB','foreign-project'); - if(kind==='foreign-branch')plan=plan.replace('branch main,','branch foreign,'); - if(kind==='foreign-source')plan=plan.replace('Reviewed target: PLAN.md','Reviewed target: FOREIGN.md'); - if(kind==='archived')plan='# Archived\n'+plan.replace(/^# /gm,'## ').replace(/^## /gm,'### '); - if(kind==='quoted')plan=plan.split('\n').map(line=>'> '+line).join('\n'); - if(kind==='fenced')plan='```md\n'+plan+'\n```'; - if(kind==='duplicate-owner')plan+=plan.split('\n').find(line=>line.startsWith('\n## Implementation plan\n# Plan: User Dashboard Page\n\n## Context\nWe're shipping a new user dashboard at `/dashboard` showing recent activity,\nnotifications panel, and quick-action buttons. Users land here after login.\n\n## UI Scope\n- New React page component `UserDashboard.tsx` at `src/pages/`\n- Three new sub-components: `ActivityFeed`, `NotificationsPanel`, `QuickActions`\n- Tailwind CSS for layout, mobile-first responsive (breakpoints: sm/md/lg)\n- Empty state, loading skeleton, error state for each panel\n- Hover states + focus-visible outlines on every interactive element\n- Modal dialog for \"Mark all as read\" on notifications panel\n- Toast notification system for action feedback\n\n## Backend\n- New REST endpoint `GET /api/dashboard` returns `{ activity, notifications, quickActions }`\n- Backed by existing PostgreSQL tables; no schema changes\n\n## Out of scope\n- Dark mode (separate plan)\n- Personalization / customization (separate plan)\n\n## Existing product and application contracts\n\nThis is the existing single-role member workspace, not a new product or a new\nonboarding flow. Members currently visit three separate pages after login to\nresume work, check alerts, and inspect recent changes. In the team's last task\nwalkthrough, finding the next item took a median 75 seconds. The dashboard's\nsuccess measure is login-to-first-completed-task time, targeting 45 seconds,\nwith completed-task rate and permission-error rate as guardrails. Existing\nanalytics records login, action start, action completion, and permission errors;\nthe new page still needs its own exposure and interaction instrumentation.\n\nActivity is the immutable audit history of workspace changes. Notifications are\nmember-specific alerts with persistent read state; acknowledging an alert does\nnot alter audit history. The existing action registry supplies three actions\n(create an item, resume assigned work, invite a member), with stable IDs, labels,\nroute targets, and server-side eligibility predicates. These are links into\nexisting workflows; action ranking and a new configuration service do not exist.\n\nThe application already uses cookie sessions and workspace membership middleware.\nIts request context supplies the authenticated member and workspace IDs. Existing\nrepository methods apply both IDs where appropriate; callers do not accept a\nworkspace ID from query parameters. Mutations already require CSRF tokens. The\nnew dashboard endpoint must compose these methods and follow the same boundaries;\nits handler, authorization integration, and failure paths have not been written.\n\nExisting list methods return the latest 20 records plus a cursor and have indexed\nworkspace/member and created-at access paths. The existing full activity and\nnotification pages own older-page navigation. The member-scoped bulk-read API is\nidempotent and marks only notifications at or before the supplied snapshot time,\nso later arrivals remain unread. Existing HTTP clients expose typed unauthenticated,\nforbidden, validation, retryable-service, and network errors. Each dashboard panel\nstill needs to map these results to its loading, empty, error, retry, and success\nstates; the aggregate endpoint's response composition and partial-failure behavior\nremain new implementation work. No schema migration or new mutation API is needed.\n\nThe app already has Tailwind spacing/color/type tokens, a responsive page shell,\nbuttons, links, and a dialog primitive with focus trapping, Escape dismissal, and\nfocus return. These primitives do not implement any dashboard panel, confirmation\nflow, or toast system. The new modal and toast feedback must also work with keyboard\nand screen readers; existing accessibility policy requires named controls, a live\nregion for nonblocking feedback, sufficient contrast, and reduced-motion support.\nThe dashboard still needs its own layout, content hierarchy, mobile behavior, and\nstate-specific copy at sm/md/lg breakpoints.\n\nVitest, React Testing Library, and Playwright already run in CI. Existing fixtures\ncover authenticated members, another workspace, empty lists, and service failures;\nthere are no dashboard-specific tests yet. Existing staging feature flags and\nrequest/error metrics support a member-cohort rollout and rollback to the current\nlanding page. The dashboard's rollout criteria, endpoint performance checks,\ninteraction tests, and accessibility verification must be specified and added.\n\nAll dashboard screen, panel, aggregate-endpoint, modal, and toast work listed above\nis new. The existing contracts describe dependencies to reuse, not completed work\nor prior approval of an implementation approach.\n\n\n- `GET /api/dashboard` is implemented by a `DashboardComposer` that runs the activity, notifications, and quick-action eligibility fetches concurrently with a per-fetch timeout budget (default 2s), returns HTTP 200 with a per-panel envelope `{ ok: true, data } | { ok: false, error: 'forbidden' | 'retryable' | 'timeout' | 'validation' }` for each of `activity`, `notifications`, `quickActions`, plus a single `serverTime` captured at handler entry. The handler reads member and workspace IDs only from the request context and ignores any query parameters. Whole-request 401 only when the session is invalid; 403 only when membership fails. Unknown exceptions propagate to the existing error middleware with a request id. Verification: Vitest cases for 0/1/2/3 sub-failures, per-fetch timeout, and predicate throw all return 200 and never 500; integration test asserts `?workspaceId=` returns no other-workspace data.\n- The post-login redirect to `/dashboard` applies to all authenticated members (single-role workspace), is behind the `dashboard_home` feature flag, uses history `replace`, and rolls out 5% \u2192 25% \u2192 100%. The redirect applies only when the login has no return-to destination; a protected deep link keeps its target. Every login emits a `dashboard_variant{variant: treatment|control}` event from the flag evaluation so cohort and control sessions are attributable; control = members logging in during the same window who are not redirected. Permission-error rate = existing permission-error events divided by action-start events per session cohort. Stage rules: 5% \u2192 25% requires at least 3 business days and at least 500 cohort sessions, or 10 business days, whichever comes first, with completed-task rate and permission-error rate within \u00b12% of control (no-regression gate only); 25% \u2192 100% requires the same no-regression gate over at least 5 business days and reports whether the cohort login-to-first-completed-task median improved \u2265 20% vs control; if the safety gates hold for 10 business days at 25% without the improvement, advance to 100% and record the miss. Alert vs kill thresholds: `dashboard_panel_failure` alerts at > 5% over 5 minutes and the flag is reverted if it stays > 5% for 30 minutes; `dashboard_cohort_completion_regression` alerts at \u22125% vs control over 1 hour and the flag is reverted if it persists for 3 hours. Flag removal: after 2 weeks at 100% with no kill-rule trigger, remove the flag and the redirect fallback branch (the previous landing page route itself stays reachable) regardless of whether the 45s target is met; the 45s target is the reported success measure, and if it is not met a follow-up TODO is opened for the hierarchy variant (single primary CTA above the fold). The dashboard route is always reachable by URL and never redirects away on data failure. The login `next` parameter accepts same-origin relative paths only. Verification: integration tests for flag on/off and for `next=//evil.com` rejection; manual check that all-three-panels-failed still renders the page with Retry.\n- Before cohort stage 5% begins, the production login\u2192first-completed-task distribution (median, p75, p90) is queried from existing analytics events (login, action start, action completion) and recorded in this plan's Context section as the real baseline replacing the 75s walkthrough figure. Segmentation into navigation time vs post-arrival time requires a page-arrival event joinable per session; if none exists, the unsegmented median is reported and the dashboard exposure event shipped with v1 provides the arrival marker for the 25% stage analysis.\n- A shared `PanelFrame` component owns loading (skeletons matching final layout), empty, error, and retry chrome for all three panels; panels supply state-specific copy: QuickActions empty = \"No actions available right now\"; NotificationsPanel empty = \"You're all caught up\" with one CTA that is the primary action (omitted when no action is eligible or the quickActions envelope failed); ActivityFeed empty = \"No recent changes yet\". Each panel shows a Retry control on error and the other panels render normally on partial failure. Verification: RTL tests for all four states per panel and for a one-panel-failed response.\n- QuickActions renders only eligible actions from the registry and gives \"resume assigned work\" primary visual weight and initial keyboard focus on load when eligible. NotificationsPanel shows an unread count badge and mirrors the count in the document title. NotificationsPanel and ActivityFeed render relative timestamps with an absolute-time tooltip, use an injected clock, re-render labels on a 60-second interval (paused while the document is hidden), clamp future timestamps to \"just now\", and link \"View all\" to the existing full activity and notifications pages (QuickActions has no list and no such link). Verification: RTL tests with a fake clock including a 60s advance, tooltip presence, future-timestamp clamp, and focus assertion.\n- \"Mark all as read\" opens a confirmation dialog built on the existing dialog primitive (focus trap, Escape, focus return). On confirm the list updates optimistically, the confirm control is disabled while the request is in flight, the request calls the existing member-scoped bulk-read API with `serverTime` from the dashboard response as its snapshot-time argument (the contract states this API marks only notifications at or before the supplied snapshot time; never the client clock; if `serverTime` is absent the action is disabled with a reload toast), a stale-CSRF failure refreshes the token once and retries once, and any failure rolls back the optimistic state and shows an error toast with Retry (403 shows \"You no longer have access\"). Undo is not provided. Verification: RTL rollback test; double-click sends exactly one request; integration test inserting a notification between confirm and response asserts it remains unread.\n- A shared `Toast` primitive lives in the shared UI layer next to the existing dialog primitive, is mounted once at the app root, exposes `useToast`, announces via an ARIA live region, queues at most 3 visible toasts, disables motion under `prefers-reduced-motion`, includes the request id in error toasts, and throws in development when used without its provider. Verification: RTL tests for queue cap, live-region text, and reduced-motion behavior.\n- All notification and activity text renders as text nodes; no server-provided string is rendered as HTML. Verification: RTL test asserting a `