mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-03 01:46:55 +02:00
* refactor(resolvers): split review.ts into MECE resolver modules (pure move) Move every function from scripts/resolvers/review.ts, unchanged, into: - review-dashboard.ts: review dashboard, plan-file review report - plan-gates.ts: approval check, exit-plan-mode gate, plan-file discovery, plan-completion audit/gate (ship + review), plan verification exec - spec-review.ts: both spec review loops, benefits-from, anti-shortcut clause - outside-voice-steps.ts: Codex second opinion, adversarial step, Codex plan review, Codex doc review, disabled-outside record - review-scope.ts: scope drift, cross-review dedup, shared-code reuse review.ts is deleted; index.ts imports the new modules. gen-skill-docs output is byte-identical for every host (--host all). Test imports and source-path references are re-pointed; the two source-text report/gate tests in gen-skill-docs.test.ts become behavioral renders across every consuming skill and host. All 46 touchfile entries that named review.ts now name all five modules, guarded by a recorded selection golden. * test(browse): black-box auth matrix for every server route and both surfaces Drives buildFetchHandler fetchLocal/fetchTunnel with no token, wrong token, root token, scoped token and the SSE cookie for all 33 routes, plus unmatched paths and wrong methods. Denials assert today's exact status, body and content type; allowed credentials assert the handler was reached. Written against the unchanged if-chain server so the W3 route-table refactor must keep it green. * refactor(shard-engine): move scripts/test-strict-output.ts to scripts/lib/shard-engine.ts The shared shard engine grows from the existing strict-output module (runShardChild, killProcessGroup, signal forwarding, strict classifier). scripts/test-strict-output.ts stays as a re-export so existing importers, mock.module paths and the strict-output/run-shard-child tests are unchanged. The engine inherits the global touchfile entry; the free runner's CLI-routing fixture copies the new module. * refactor(resolvers): decompose the three >150-line review resolvers (output-neutral) Split generateAdversarialStep, generateCodexPlanReview and generatePlanCompletionAuditInner into per-section helpers whose template literals are copied verbatim, so every function in the new modules is at or under 150 lines. gen-skill-docs output is byte-identical for every host (--host all, compared against96764e80with a fixed --link-root). * refactor(resolvers): one outside-voice failure policy (deliberate prose unification) outsideVoiceFailurePolicy(ctx, opts) in outside-voice.ts now renders the auth / timeout / empty-response bullets for all four call sites that hand-typed them (Codex second opinion, adversarial step, Codex plan review, design outside voices). Options are explicit per site (timeoutMinutes, onTimeout, stderrOnEmpty, fallback, escape) with no defaults. Deliberate generated-prose changes (every host): - office-hours: 'Fall back to <native> subagent.' becomes 'Fall back to the <native> subagent below.' - plan-devex-review: the plain 'Auth failure (stderr contains ...)' bullets become the canonical bold bullets; auth also triggers on 'API key'; 'auth failed' becomes 'authentication failed'. - review/ship adversarial: 'exceeded 9 minutes and was terminated' becomes 'timed out after 9 minutes and was terminated'; the timeout is still MISSING COVERAGE. - design outside voices: unchanged. Adds ratchet (d) (test/outside-voice-failure-policy.test.ts) with a reasoned allowlist for /codex's own CLI errors, the MISSING COVERAGE retention test, refreshed codex/factory ship goldens, and outside-voice.ts in every touchfile entry of review.ts and design.ts (selection golden extended). * test(pty): fake PTY session driver with an injectable clock through the runner launch seam The three plan-skill runners take an optional PtyDriver (launch, now, monotonic, sleep); omitted, they use the real launcher and clocks exactly as before. test/helpers/pty/fake-session.ts feeds scripted frames through that seam, and claude-pty-runner.runners.unit.test.ts runs observation, counting and floor for success, deadline timeout, permission prompt and plan-ready outcomes with no CLI or real timers. These cases must stay green unchanged through the W4 split and the runPtySession extraction. Touchfiles: every entry that lists claude-pty-runner.ts or pty-screen.ts now also lists test/helpers/pty/**. * refactor(shard-engine): run both lanes on the shared engine; lane policy injected Engine (scripts/lib/shard-engine.ts) gains the W2 primitives: per-shard tmp/Chromium sandbox + async cleanup backstop, log-path allocation and full-stream log capture, one duration-seed reader/writer with a lane predicate, LanePolicy (seed predicate + zero-execution verdict), strictShardStatus, and the shared CLI flag loop. runShardChild takes an optional companion (signal/settle) and waits a bounded 250ms to reap a wall-killed child. Free lane stops spawning shards itself: runFreeShard uses runShardChild with trackShardBrowser as the companion (win32 path unchanged: no process group, no negative-pid kill). Its sync state-dir removal stays lane policy. Paid lane uses the sandbox, log, seed, verdict and flag primitives; the hollow-shard guard applies PAID_LANE_POLICY. Lane outcomes are unchanged (free keeps >= 0 seeds and file-count zero-exec rule; paid keeps > 0 seeds, warning under selection and passed-empty under EVALS_ALL). paid-free-boundary's closure assertion now names the engine module, where the strict classifier lives. * test(shard-engine): engine unit tests, fixture-corpus equivalence, per-lane CLI parity - test/shard-engine.test.ts: failing/unhandled/module-load output fails both lanes, per-lane zero-execution and seed rules, whole-group kill on a wall timeout (both lanes), mocked-win32 path with no negative-pid kill, companion settle order, log capture, sandbox isolation, flag loop. - test/shard-engine-equivalence.test.ts + test/fixtures/shard-equivalence: seven outcome fixtures plus one real shard, run through both lanes and compared with classifications recorded from the base runners (96764e80). - test/shard-cli-parity.test.ts + test/fixtures/shard-cli-parity: flag set, defaults, validation errors and the Unknown argument error per lane match the base runners. * refactor(shard-engine): decompose runFreeShard and runPaidShard to <= 150 lines Output-neutral extraction under the fixture-corpus equivalence and runner tests: captureFreeStream, explainFreeVerdict and logFreeRecovery (free); paidShardCommand, settleShardSpool, settleBootstrapRetention and printLogTail (paid). The bootstrap scope-creation block that bootstrap-retention.test.ts evaluates stays verbatim. * refactor(pty): split claude-pty-runner.ts into test/helpers/pty/* behind a barrel Pure move: every line of the former 5,047-line runner lands verbatim in one module (four private helpers gain `export` for cross-module use): binary, screen (absorbs test/helpers/pty-screen.ts, which now re-exports it), launch, session (PtyDriver), judge, classify, auq, plan-native, boundaries, runners/{observation,counting,floor}. claude-pty-runner.ts re-exports the original public surface by name; pty/ modules import siblings directly. Tests that read the runner's source text: - rewritten as behavioral: the unit test's model-pin tripwire (fake CLI argv: fallback chain, --model before extraArgs, hermetic --strict-mcp-config), pty-skill-seeding-wiring (runners through the fake driver; launcher through a fake CLI reporting CLAUDE_CONFIG_DIR). The "three wrappers forward model" grep is replaced by the runners' fake-driver launch assertions. - pty-screen-session / pty-screen-supervision: stop copying runner source; they mock.module the real pty/screen.ts (and the fixture cleanup) instead. - re-pointed to the owning module (they execute a sliced runner body with injected boundaries; no seam exists for those boundaries yet): eng-seeded-completion-ai, plan-floor-permission, plan-create-prepublication, plan-count-completion; hermetic-wiring's source guard now reads pty/launch.ts and scans every pty/ module for raw process.env spreads. - plan-count-timeout and pty-output-wake mock the viewport at pty/screen.ts. * test(ratchet-c): enforcing module/function size ratchet and moved-code touchfile coverage Ratchet (c) ships enforcing: test/helpers/module-size.ts counts file and top-level function lengths by brace matching over masked source (strings, comments, regex literals and template text masked; ${} expressions kept), covering function declarations, arrow functions assigned to consts and route-table handler properties, with no parser dependency. Its self-test uses template literals and code-fence braces copied from scripts/resolvers/review.ts and design.ts. test/fixtures/module-size-ratchet.json binds scripts/lib/shard-engine.ts (<= 800 lines, <= 150 per function) and records the residual runner sizes (free 2352, paid 1921) as non-growth caps; allowlist entries are keyed on file plus matched text and need a reason. Failure output lists file:line, the rule, Fix: and the allowlist path. touchfiles.test.ts gains the moved-code superset check over test/fixtures/touchfile-move-goldens/ (W2 golden recorded at96764e80: test-strict-output.ts and test-paid-shards.ts global, test-free-shards.ts none). * refactor(browse): declared route table replaces the buildFetchHandler if-chain The ~1,300-line if-chain in buildFetchHandler becomes a route table: each entry declares method, path, auth kind and surfaces, and one auth gate in browse/src/routes/table.ts returns the per-kind denial (root-bearer, scoped, root-or-sse-cookie: 401 Unauthorized; root-token: 403 Root token required; extension-origin: 403 Forbidden). Unmatched requests take the declared fallthrough (root-bearer check, then plain-text 404). Handlers move to browse/src/routes/{core,pairing,pty,tokens,tunnel,activity,commands,files, inspector}.ts and receive a RouteContext with auth checks as functions instead of closing over factory locals. Dispatch order is unchanged: tunnel filter, beforeRoute overlay, gate, handler. TUNNEL_PATHS stays a literal in server.ts. Behavior-preserving: the black-box auth matrix from the previous commit passes unchanged. /memory and /inspector/events are declared root-bearer because the blanket check always ran before their SSE-cookie branch. Source-text route tests are rewritten as behavioral tests through buildFetchHandler or a route's real handler with a stub RouteContext (browse/test/route-test-harness.ts). Checks with no runtime seam are re-pointed to the route modules: Surface type, /inspector/events SSE helper, sanitizeReplacer imports, /pty-inject-scan sidecar-client import, and the ngrok config lookup and startTunnel wiring that stay in server.ts. * test(browse): stubbed-handler auth matrix and route inventory for the route table Every ROUTES entry runs through the real dispatcher and gate with stub handlers on each declared surface and six credentials; denials assert the exact status and body each auth kind returned at96764e8, admitted credentials assert the handler ran (with the gate's TokenInfo for scoped routes). Also pins the reviewed route inventory (method, path, auth kind, surfaces), that every entry declares auth and surfaces, that the table's tunnel paths equal the TUNNEL_PATHS literal with GET /connect admitted, the unmatched fallthrough, and that the root token is rejected on every tunnel route through buildFetchHandler. * test(browse): ratchet (b) keeps route dispatch inside the route table Scans browse/src/server.ts and browse/src/routes/*.ts for pathname comparisons; only the table matcher and the tunnel-surface filter are allowed, listed with reasons in browse/test/fixtures/route-dispatch-allowlist.json (keyed on file plus line text). Also checks every entry declares auth and surfaces and that gstack registers no beforeRoute overlay itself. Self-tests plant a violation and assert the file:line, Fix: and allowlist path in the message, that a shifted line stays allowlisted, and that a reasonless entry is rejected. * test: touchfile superset check for modules moved out of browse/src/server.ts Records the paid evals selected by touching browse/src/server.ts at96764e80(17 E2E, 1 LLM judge) and asserts every browse/src/routes/*.ts module selects a superset. The test reads every golden in test/fixtures/moved-module-selection/ so other moved-code goldens can sit beside it. * test(shard-engine): give non-timeout corpus fixtures CI headroom; keep the POSIX golden off the Windows lane Only the wall-timeout fixture keeps a 3s wall; the rest get 60s so a loaded host cannot turn a pass into a timeout. Base and branch runners still agree on every classification under the new walls. The Windows exclusion entry moves the free runner's ratchet (c) residual cap to 2356 lines. * refactor(pty): one runPtySession loop drives observation, counting and floor test/helpers/pty/session.ts owns launch -> start -> (poll -> tick)* -> timeout and the failure contract the three runners each hand-rolled: the run's own error wins over capture and close errors, close always runs, owned fixture cleanup runs last (also when launch fails). Each runner now supplies a PtySessionPlan: its boot/command step, poll cadence (2s observation/floor sleep; counting's output wake + 250ms coalesce), tick policy (permission handling, native identity, terminal rules stay per runner because they differ) and capture hooks. The runner bodies are decomposed into top-level steps so no function exceeds 150 lines; behavior is unchanged and the fake-driver cases from the first W4 commit pass unmodified. The counting capture step and the native completion-summary predicate are now named functions (countingCapture, isNativeCompletionSummary), so plan-create-prepublication and plan-count-completion call them directly instead of executing sliced source. The two harnesses that still execute a sliced runner body with injected boundaries (eng-seeded-completion-ai, plan-floor-permission) pass the PtyDriver seam instead of overriding Date/Bun.sleep. * test(ratchet-c): register route modules, review resolver modules and server.ts residual cap * refactor(pty): decompose launchClaudePty and engNumberedFindingAUQ under 150 lines launchClaudePty (349 lines) becomes launch preparation (args, hermetic child env, owned state roots), recorder creation, spawn, the trust-dialog watcher, close, and the session handle over one PtyProcess state object. The failure order is unchanged: abort the viewport, dispose any recorders created so far, dispose the viewport, rethrow. The --model / --strict-mcp-config ordering and seedSkills wiring stay pinned by the behavioral fake-CLI tests. engNumberedFindingAUQ (345 lines) keeps its guards and dispatch; each self-contained issue family (declared cache, library retry hooks, cache owner, injected singleton, shared writers, injected export) moves verbatim into its own function. Every pty/ module is now <= 800 lines and every top-level function <= 150 lines. * test(pty): split claude-pty-runner.unit.test.ts along the pty/ module seams The 188 unit tests move verbatim into claude-pty-runner.{screen,classify, auq,launch,plan-native,boundaries}.unit.test.ts (test names unchanged; each file imports only what it uses from the barrel). The five files that no longer read a SKILL.md template join the test-of-test ratchet baseline with a reason. * test(touchfiles): moved PTY modules keep their paid-eval selection test/fixtures/touchfile-selection/w4-pty.json records, at96764e8, the paid evals selected by touching test/helpers/claude-pty-runner.ts (20) and test/helpers/pty-screen.ts (20). touchfiles.test.ts now asserts every .ts file under test/helpers/pty/ (and pty/screen.ts for both sources) selects a superset, reading every golden in that directory so later moves can add one; a planted-violation case pins the report and its Fix line. * fix(browse): unexchanged pair setup keys no longer authenticate bearer requests validateToken accepted a gsk_setup_ key as a bearer on /command, /batch and /file (found while building the W3 auth matrix). A setup key now only authenticates the /connect exchange. * W1: one state-root owner (lib/state-root.ts + bin/gstack-state-root.sh), gstack-paths --explain and fail-stop, parity tests * W1: guarded migration of every executable state-root site; uninstall deletes only ~/.gstack Bins, careful/freeze hooks, setup, upgrade migrations, browse/src, design, ios-qa daemon, lib and scripts resolve the state root through bin/gstack-state-root.sh (bash) or lib/state-root.ts (TS). Bins source the twin and stop with a reinstall message when it is missing; hooks source it and never spawn gstack-paths. browse/src/config.ts and lib/cso/state.ts delegate to resolveStateRoot. Analytics writers and readers move together so the usage log stays one file. gstack-uninstall deletes state only at ~/.gstack, refuses (exit 2) when it resolves to /, $HOME or an ancestor, the checkout or the git root, and leaves any other resolved root in place with the removal command. Fixtures that copy single bins now copy the twin. * W1: privacy keys and trust-policy deny tiers merge across state roots; gstack-config reporting; test hermeticity readConfigKey / gstack_read_config_key return the most restrictive telemetry, memorable_recall, codex_reviews and update_check across the resolved root and ~/.gstack; other keys read the resolved root only. gstack-config set reports an overriding root with the exact override command, list shows the winning root and a root-variable disagreement line. gstack-gbrain-repo-policy get merges deny/read-only tiers. gstack-egress reads through readConfigKey. test-setup.ts strips inherited GSTACK_STATE_ROOT/GSTACK_STATE_DIR and redirects the legacy root. * W1: shared hook logging helper (hosts/claude/hooks/hook-log.ts) One hook-errors.log writer: root from resolveStateRoot, 0600 on every append, opt-in rate limit used only by memorable-user-prompt. The five hooks route through it. * W1: docs/state-root.md and README troubleshooting pointer Precedence table, a real --explain example, the move-your-state recipe, merged privacy keys, the uninstall rule, the resolver-failure fix, and the plugin-mode note (evidence gate: no official plugin distribution). * W1b: template and resolver prose resolve state through guarded gstack-paths; ratchet (a) Every gstack-paths eval in templates and resolvers carries the fail-stop guard; executable ~/.gstack paths in bash blocks (context recovery preamble, eureka log, analytics, project artifacts, upgrade snooze, setup-gbrain lock, retro snapshots, ship consent marker) use $GSTACK_STATE_ROOT, and the writer prose that pairs with them points at the printed PROJECT_DIR / RETRO_FILE. ship drops export GSTACK_STATE_ROOT. SKILL.md regenerated (claude + codex), ship goldens re-pinned, parity and context-budget caps raised to the measured sizes with notes. test/state-root-ratchet.test.ts enforces the rule with a reasoned allowlist; W1 touchfile entries plus a superset golden. * refactor: apply W1 state-root edits in W2/W3/W5-owned files; one moved-code touchfile golden for all workstreams * test: fold the moved-code touchfile golden into touchfiles.test.ts; fix integration fixture closure and caps * v1.91.11.0: CHANGELOG, TODOS, docs and conventions for the refactor wave * test: re-measure plan-ceo/design-consultation caps and ship goldens after the guarded plan-discovery and spec-review blocks; add the state-root twin to the workflow-boundaries fixture * fix(windows): migrations resolve their directory with either path separator; state-root parity compares under the HOME Git Bash actually sees * fix(review,ship): state plan-check timing after smoke expiry and test_stub Skip semantics (review workflow judge clarity) * test(qa-eval): webhook fix eval asks for the fix loop's post-repair probes; eight-scenario coverage stays in the report-only case and the harness recheck * test(qa-eval): re-pin the webhook prompt contract to the fix-loop stage; R29 coverage omissions stay bound by the report-only case * fix(review,ship): plan checks publish a checkpoint before each probe; only the smoke expiry stop is skipped * fix(qa): carry #2999's checkpoint receipt link, report-template line and full-revision placeholder (identical hunks) * test(qa-callers): disable git auto maintenance in the caller fixture Git 2.47+ runs auto maintenance detached after commit; on the CI runner's git 2.55 it rewrote .git/objects fan-out directories while the write observer was running, which surfaced as unauthorized mutations. Same gc.auto=0 / maintenance.auto=false guard the shared-libs fixture already uses. * test(plan-mode-no-op): require prose evidence for the prose-fallback members so a spinner-frame judge verdict cannot end the run as asked * test(ship-docsync): carry #2999's seeded-attempt docsync harness (identical files) The doc-sync fault cases replayed attempt 1 before reaching their gate and ran out of their 285s budget. The fixture now seeds attempt 1 and the parent starts at the gate under test. Taken byte-identical from origin/capy/audit-fix-wave (fb526898,e6ac813d,6ce10ff7,d0c53577,77cce3be). Local: stale-before, recovery and late-result 6/6 PASS (97-164s); the whole file 12/12 PASS.
1529 lines
102 KiB
TypeScript
1529 lines
102 KiB
TypeScript
/** Native completion/report regression; no model/API calls. */
|
||
import { describe, expect, test } from 'bun:test';
|
||
import * as fs from 'node:fs';
|
||
import * as os from 'node:os';
|
||
import * as path from 'node:path';
|
||
import { pathToFileURL } from 'node:url';
|
||
import { hasNativePlanCompletion, hasNativePlanTerminal, isPlanReadyVisible, classifyPlanCountFrame, isNumberedOptionListVisible, isPermissionDialogVisible, isProseAUQVisible } from './helpers/claude-pty-runner';
|
||
import type { PlanCountTranscript } from './helpers/plan-count-transcript';
|
||
import capturedL from './fixtures/devex-review-l-calls.json';
|
||
import designStatusCapture from './fixtures/design-count-native-issue-fields.json';
|
||
import designEnvelope from './fixtures/design-completion-envelope-90f.json';
|
||
import { nativePlanCallFingerprint, isNativeCompletionSummary } from './helpers/claude-pty-runner';
|
||
import type { NativePlanQuestionCall } from './helpers/plan-count-transcript';
|
||
import captured_ceo_completion_handoff_m from './fixtures/ceo-completion-handoff-m-call.json';
|
||
import nextStepCapture_ceo_completion_handoff_m from './fixtures/ceo-handoff-n-calls.json';
|
||
import captured_ceo_completion_handoff_o from './fixtures/ceo-completion-handoff-o-call.json';
|
||
import capturedQ_ceo_completion_handoff_o from './fixtures/ceo-completion-handoff-q-call.json';
|
||
import fs_ceo_handoff_y from 'node:fs';
|
||
import os_ceo_handoff_y from 'node:os';
|
||
import path_ceo_handoff_y from 'node:path';
|
||
import fixture_ceo_handoff_y from './fixtures/ceo-handoff-y-call.json';
|
||
import captured_dx_manual_handoff_ao from './fixtures/dx-manual-handoff-ao.json';
|
||
import fixture_plan_count_dx_handoff_o from './fixtures/devex-handoff-o-call.json';
|
||
import actual_eng_next_handoff_ah from './fixtures/eng-next-handoff-ah.json';
|
||
import { isCurrentPlanApprovalScreen } from './helpers/plan-count-pending-exit';
|
||
import { createHash } from 'node:crypto';
|
||
import capture_eng_task_pause_navigation_f359 from './fixtures/eng-task-pause-navigation-f359.json';
|
||
import fixture_design_count_native_8525 from './fixtures/design-count-native-8525.json';
|
||
|
||
describe('captured Design completion envelope', () => {
|
||
function completedEnvelope() {
|
||
const dir=fs.mkdtempSync(path.join(os.tmpdir(),'design-completion-envelope-'));
|
||
const file=path.join(dir,'design-review.md');
|
||
const transcript=structuredClone(designEnvelope.transcript) as PlanCountTranscript;
|
||
const final=transcript.assistantMessages.at(-1)!;
|
||
final.text=final.text.replaceAll(designEnvelope.originalPlanPath,file);
|
||
const write=(report=designEnvelope.report)=>{fs.writeFileSync(file,report);fs.utimesSync(file,designEnvelope.reportMtimeMs/1000,designEnvelope.reportMtimeMs/1000);};write();
|
||
const check=()=>hasNativePlanTerminal(transcript,file,designEnvelope.startedAt,'completion_summary');
|
||
return {file,transcript,final,write,check,cleanup:()=>fs.rmSync(dir,{recursive:true,force:true})};
|
||
}
|
||
test('saved report annotation and punctuated DONE retain the manual plan-mode handoff',()=>{
|
||
const f=completedEnvelope();try {
|
||
expect(classifyPlanCountFrame(designEnvelope.screen)).toBeNull();
|
||
expect(hasNativePlanCompletion(f.transcript,f.file,designEnvelope.startedAt)).toBe(false);
|
||
expect(f.check()).toBe(true);
|
||
expect(hasNativePlanTerminal(f.transcript,f.file,designEnvelope.startedAt,'plan_ready')).toBe(false);
|
||
}finally{f.cleanup();}
|
||
});
|
||
test('the actual runner branch recognizes the idle native envelope without answering an exit gate',()=>{
|
||
const f=completedEnvelope();try {
|
||
const screen=designEnvelope.screen.replaceAll(designEnvelope.originalPlanPath,f.file);
|
||
const check=()=>isNativeCompletionSummary({expectedPlanPath:f.file,nativeCompletion:hasNativePlanCompletion(f.transcript,f.file,designEnvelope.startedAt),
|
||
renderedFrame:classifyPlanCountFrame(screen),visible:screen,transcript:f.transcript,startedAt:designEnvelope.startedAt,administrative:new Set()});
|
||
expect(check()).toBe(true);
|
||
f.transcript.calls[0]!.answered=false;expect(check()).toBe(false);
|
||
}finally{f.cleanup();}
|
||
});
|
||
for(const [name,edit] of Object.entries({
|
||
'exclamation status':(s:string)=>s.replace('STATUS: DONE.','STATUS: DONE!'),
|
||
'status without extra closure':(s:string)=>s.replace(/\*\*STATUS: DONE\.\*\*[^\n]+/,'STATUS: DONE.'),
|
||
'equivalent read-only closure':(s:string)=>s.replace('No unresolved decisions. Staying in plan mode; nothing outside the plan file was edited.','No open design findings. Remaining in plan mode; no implementation has been started.'),
|
||
'equivalent persisted annotation':(s:string)=>s.replace('report is the last section, read-back verified','saved; review report is the final heading; verified by readback'),
|
||
'report field':(s:string)=>s.replace('- Plan: ','- Report: '),
|
||
}))test('completion envelope accepts '+name,()=>{
|
||
const f=completedEnvelope();try{f.final.text=edit(f.final.text);expect(f.check()).toBe(true);}finally{f.cleanup();}
|
||
});
|
||
for(const [name,edit] of Object.entries({
|
||
'foreign path':(s:string)=>s.replace('design-review.md','other-review.md'),
|
||
'path in quoted prose':(s:string)=>s.replace(/(- Plan: )(.+)/,'$1"$2"'),
|
||
'second path':(s:string)=>s.replace('(report is the last section, read-back verified)','(report is the last section, read-back verified) /other/plan.md'),
|
||
'second artifact field':(s:string)=>s+'\n- Report: /other/plan.md (saved)',
|
||
'historical annotation':(s:string)=>s.replace('read-back verified','historical example'),
|
||
'conditional annotation':(s:string)=>s.replace('read-back verified','read-back verified if approved'),
|
||
'pending read-back':(s:string)=>s.replace('read-back verified','read-back pending'),
|
||
'negated save':(s:string)=>s.replace('report is the last section, read-back verified','not saved'),
|
||
'quoted annotation':(s:string)=>s.replace('report is the last section, read-back verified','"saved"'),
|
||
'pending status':(s:string)=>s.replace('STATUS: DONE.','STATUS: PENDING.'),
|
||
'quoted status':(s:string)=>s.replace('**STATUS: DONE.**','`STATUS: DONE.`'),
|
||
'conditional status':(s:string)=>s.replace('STATUS: DONE.','STATUS: DONE if approved.'),
|
||
'pending native action':(s:string)=>s.replace('No unresolved decisions. Staying in plan mode; nothing outside the plan file was edited.','Please approve the remaining design choice.'),
|
||
'implementation action':(s:string)=>s.replace('Staying in plan mode','Starting implementation now'),
|
||
'conditional plan-mode closure':(s:string)=>s.replace('Staying in plan mode','Staying in plan mode until you approve the remaining decision'),
|
||
'unresolved qualifier':(s:string)=>s.replace('No unresolved decisions.','No unresolved decisions except the contrast issue.'),
|
||
'current reopened report':(s:string)=>s+'\nThe report is withdrawn.',
|
||
'current pending design':(s:string)=>s+'\nDesign review remains unresolved.',
|
||
'current quoted failure':(s:string)=>s+'\nThe report is "failed".',
|
||
'quoted full envelope':(s:string)=>'> '+s.replaceAll('\n','\n> '),
|
||
'historical envelope':(s:string)=>'Historical example:\n\n'+s,
|
||
'repeated status':(s:string)=>s+'\nSTATUS: BLOCKED',
|
||
}))test('completion envelope rejects '+name,()=>{
|
||
const f=completedEnvelope();try{f.final.text=edit(f.final.text);expect(f.check()).toBe(false);}finally{f.cleanup();}
|
||
});
|
||
test('the real envelope still requires completed owned calls and its fresh completed Design report',()=>{
|
||
const f=completedEnvelope();try {
|
||
const original=structuredClone(f.transcript);
|
||
for(const edit of [(t:PlanCountTranscript)=>{t.calls[0]!.answered=false;},(t:PlanCountTranscript)=>{t.calls[0]!.failed=true;},
|
||
(t:PlanCountTranscript)=>{t.calls[0]!.sessionId='foreign';},(t:PlanCountTranscript)=>{t.calls.at(-1)!.answeredAt=f.final.timestamp;},
|
||
(t:PlanCountTranscript)=>{t.assistantMessages.at(-1)!.timestamp='2999-01-01T00:00:00Z';},
|
||
(t:PlanCountTranscript)=>{t.planReadyRequests=[{sessionId:t.calls[0]!.sessionId,toolUseId:'failed-exit',timestamp:f.final.timestamp,failed:true}];}]){
|
||
Object.assign(f.transcript,structuredClone(original));edit(f.transcript);expect(f.check()).toBe(false);
|
||
}
|
||
Object.assign(f.transcript,structuredClone(original));
|
||
for(const body of ['# Draft',designEnvelope.report+'\n## Further work',designEnvelope.report.replace('DESIGN CLEARED','NOT CLEARED'),
|
||
designEnvelope.report.replace('NO UNRESOLVED DECISIONS','**UNRESOLVED DECISIONS:**\n- Missing decision')]){f.write(body);expect(f.check()).toBe(false);}
|
||
f.write();fs.utimesSync(f.file,1,1);expect(f.check()).toBe(false);
|
||
fs.utimesSync(f.file,Date.parse(f.final.timestamp)/1000+1,Date.parse(f.final.timestamp)/1000+1);expect(f.check()).toBe(false);
|
||
fs.rmSync(f.file);expect(f.check()).toBe(false);
|
||
}finally{f.cleanup();}
|
||
});
|
||
});
|
||
|
||
const CAPTURED_CALL = {
|
||
"sessionId": "b5c582af-870e-48ac-ac1e-c85458932136",
|
||
"toolUseId": "toolu_017PNDKgKXk1VyxRT9BVdDbt",
|
||
"answered": true,
|
||
"failed": false,
|
||
"answeredAt": "2026-09-08T17:45:31.244Z",
|
||
"questions": [
|
||
{
|
||
"header": "P5: v1\u2192v2 break",
|
||
"question": "D7 \u2014 Pass 5, Finding 6: The v1\u2192v2 upgrade removes `Client.evaluate()` immediately with zero migration support, guaranteeing broken production installs on pip upgrade. <gstack-qid:plan-devex-review-v1-v2-migration>",
|
||
"options": [
|
||
{
|
||
"label": "A \u2014 Migration guide + deprecation alias (Recommended)",
|
||
"description": "Plan must include: (1) migration guide in changelog ('replace .evaluate() with .run()'), (2) v2 ships a .evaluate() compatibility alias that raises DeprecationWarning('use Client.run() instead'), (3) one-paragraph upgrade section in README. Upgrade is boring, not scary. (human: ~2h / CC: ~20min)"
|
||
},
|
||
{
|
||
"label": "B \u2014 Migration guide only (no alias)",
|
||
"description": "Add a migration guide in the changelog with the one-line fix. No compatibility alias \u2014 v1 users who miss the guide still break. Better than nothing but still relies on developers reading the changelog."
|
||
},
|
||
{
|
||
"label": "C \u2014 Accept immediate removal, document in release notes",
|
||
"description": "Keep the current plan (immediate removal, no alias). Add a prominent breaking-change banner to release notes. Honest about the break, but production projects that auto-upgrade still break silently."
|
||
}
|
||
]
|
||
}
|
||
],
|
||
"answers": {
|
||
"D7 \u2014 Pass 5, Finding 6: The v1\u2192v2 upgrade removes `Client.evaluate()` immediately with zero migration support, guaranteeing broken production installs on pip upgrade. <gstack-qid:plan-devex-review-v1-v2-migration>": "A \u2014 Migration guide + deprecation alias (Recommended)"
|
||
}
|
||
};
|
||
const CAPTURED_FINAL = {
|
||
"sessionId": "b5c582af-870e-48ac-ac1e-c85458932136",
|
||
"text": "DX review complete. Here's what was found and resolved:\n\n**6 findings across 5 review passes \u2014 all resolved interactively:**\n\n| # | Pass | Issue | Fix |\n|---|------|-------|-----|\n| D2 | Getting Started | `examples/first_eval.py` referenced in README but absent from package | Remove reference; make terminal demo the single path |\n| D3 | Getting Started | Demo triggers mandatory 5-min CI gate despite using bundled offline data; TTHW 6 min vs <2 min target | Exempt `evalkit.demo` from CI gate; TTHW drops to ~1-2 min |\n| D4 | API Design | `run_eval(dataset, evaluator)` and `run_batch(evaluator, dataset)` have reversed positional order \u2014 silent wrong-result bugs | Enforce keyword-only args on both functions |\n| D5 | Error Messages | `AuthError(\"request failed\")` \u2014 no code, no cause, no fix | Replace with `AuthError(\"Invalid API key \u2014 check EVALKIT_API_KEY or visit ...\", code=\"auth_invalid_key\")` |\n| D6 | Documentation | No expected output shown after demo command | Add expected output block to README |\n| D7 | Upgrade Path | `Client.evaluate()` removed immediately with no migration path | Ship deprecation shim + migration guide + README upgrade section |\n\n**Score:** 4/10 \u2192 8/10. TTHW: 6 min \u2192 ~1-2 min (Champion tier).\n\n**Critical blocker:** The CI gate finding (D3) is architecturally non-trivial \u2014 the plan currently says \"retains this gate\" but the TTHW target requires exempting demo mode. `/plan-eng-review` should validate the gate bypass implementation before this ships.\n\nPlan written to: `/tmp/gstack-paid-shard-qDnJUI/tmp/gstack-e2e-plan-devex-VWDg0F/gstack-test-plan-devex.md`",
|
||
"timestamp": "2026-09-08T17:49:33.033Z"
|
||
};
|
||
const CAPTURED_PATH = "/tmp/gstack-paid-shard-qDnJUI/tmp/gstack-e2e-plan-devex-VWDg0F/gstack-test-plan-devex.md";
|
||
const REPORT = '# Reviewed plan\n\n## GSTACK REVIEW REPORT\n\n' +
|
||
'| Review | Status | Findings |\n|---|---|---|\n| DX Review | clean | resolved |\n\n' +
|
||
'VERDICT: DX CLEARED — eng review required\n\nNO UNRESOLVED DECISIONS\n';
|
||
|
||
describe('typed native Design completion with a separately pending Eng gate', () => {
|
||
function capturedFixture() {
|
||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'design-status-completion-'));
|
||
const file = path.join(dir, 'plan.md');
|
||
const transcript: PlanCountTranscript = {
|
||
status: 'ready', calls: structuredClone(designStatusCapture.calls),
|
||
assistantMessages: structuredClone(designStatusCapture.assistantMessages),
|
||
};
|
||
const final = transcript.assistantMessages.at(-1)!;
|
||
const originalPath = /Plan written to `([^`]+)`/.exec(final.text)![1]!;
|
||
final.text = final.text.replace(originalPath, file);
|
||
const lastAnswer = Math.max(...transcript.calls.map(c => Date.parse(c.answeredAt!)));
|
||
const modified = (lastAnswer + 1_000) / 1_000;
|
||
// The original report file was not retained. This is an explicitly
|
||
// synthetic valid report; only the native calls/summary above are replayed.
|
||
const report = REPORT.replaceAll('DX', 'Design').replace('Design CLEARED', 'DESIGN CLEARED');
|
||
const write = (content = report) => { fs.writeFileSync(file, content); fs.utimesSync(file, modified, modified); };
|
||
write();
|
||
const startedAt = Date.parse(transcript.assistantMessages[0]!.timestamp) - 1_000;
|
||
const check = () => hasNativePlanTerminal(transcript, file, startedAt, 'completion_summary');
|
||
return { file, transcript, final, report, write, check, cleanup: () => fs.rmSync(dir, { recursive: true, force: true }) };
|
||
}
|
||
|
||
test('captured standalone STATUS: DONE binds to the fresh Design report without completing Eng', () => {
|
||
const f = capturedFixture();
|
||
try {
|
||
expect(f.final.text).toContain('VERDICT: NOT CLEARED — Eng Review missing');
|
||
expect(f.final.text).toContain('**STATUS: DONE**');
|
||
expect(f.check()).toBe(true);
|
||
const original = f.final.text;
|
||
for (const text of [
|
||
original.replace('## Completion', '### Completion summary').replace('**STATUS: DONE**', 'STATUS: DONE'),
|
||
original.replace('Plan written to', 'Plan saved to'),
|
||
'## Completion\n\nSTATUS: DONE\n\n- Plan saved to `' + f.file + '`.\n- Design results are in the report; Eng review remains pending.',
|
||
]) { f.final.text = text; expect(f.check()).toBe(true); }
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test('completion needs its own current status and persisted path, not quoted or provisional prose', () => {
|
||
const f = capturedFixture();
|
||
try {
|
||
const original = f.final.text;
|
||
for (const text of [
|
||
original.replace('**STATUS: DONE**', '**STATUS: BLOCKED**'),
|
||
original.replace('**STATUS: DONE**', '**STATUS: DONE if approved**'),
|
||
original.replace('**STATUS: DONE**', '> STATUS: DONE'),
|
||
original.replace('**STATUS: DONE**', '`STATUS: DONE`'),
|
||
original.replace('**STATUS: DONE**', '```text\nSTATUS: DONE\n```'),
|
||
original.replace('## Completion', '## Example of Completion'),
|
||
original.replace('## Completion', 'Source example:\n\n## Completion'),
|
||
original.replace('Plan written to', 'Plan will be written to'),
|
||
original.replace('- Plan written to', '> Plan written to'),
|
||
original.replace(f.file, f.file + '.other'),
|
||
original + '\n\n## Completion\nSTATUS: DONE',
|
||
original + '\n\nSTATUS: BLOCKED',
|
||
original + '\n\nDesign review remains unresolved.',
|
||
original + '\n\nThe report is withdrawn.',
|
||
original + '\n\nThe Design review requires approval.',
|
||
original + '\n\nPlease confirm?',
|
||
]) { f.final.text = text; expect(f.check(), text).toBe(false); }
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test('typed status preserves native answer ownership, timestamps and required Design report validation', () => {
|
||
const f = capturedFixture();
|
||
try {
|
||
const original = structuredClone(f.transcript);
|
||
for (const change of [
|
||
(t: PlanCountTranscript) => { t.calls[0]!.answered = false; },
|
||
(t: PlanCountTranscript) => { t.calls[0]!.failed = true; },
|
||
(t: PlanCountTranscript) => { t.calls[0]!.sessionId = 'foreign'; },
|
||
(t: PlanCountTranscript) => { t.assistantMessages.at(-1)!.timestamp = '2999-01-01T00:00:00Z'; },
|
||
(t: PlanCountTranscript) => { t.calls.at(-1)!.answeredAt = t.assistantMessages.at(-1)!.timestamp; },
|
||
(t: PlanCountTranscript) => { t.assistantMessages.push({ ...t.assistantMessages.at(-1)!, timestamp: '2026-09-15T08:46:00Z', text: 'Still reviewing.' }); },
|
||
]) {
|
||
Object.assign(f.transcript, structuredClone(original)); change(f.transcript);
|
||
expect(f.check()).toBe(false);
|
||
}
|
||
Object.assign(f.transcript, structuredClone(original));
|
||
for (const content of [
|
||
'# Draft', f.report + '\n## Unreviewed tail\n', REPORT,
|
||
f.report.replace('clean', 'pending'), f.report.replace('DESIGN CLEARED', 'NOT CLEARED'),
|
||
f.report.replace('NO UNRESOLVED DECISIONS', '**UNRESOLVED DECISIONS:**\n- Missing contrast decision'),
|
||
]) { f.write(content); expect(f.check()).toBe(false); }
|
||
f.write(); fs.utimesSync(f.file, 1, 1); expect(f.check()).toBe(false);
|
||
fs.rmSync(f.file); expect(f.check()).toBe(false);
|
||
} finally { f.cleanup(); }
|
||
});
|
||
});
|
||
|
||
function fixture() {
|
||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'native-plan-completion-'));
|
||
const file = path.join(dir, 'review.md');
|
||
const startedAt = Date.parse('2026-09-08T17:39:30.000Z');
|
||
fs.writeFileSync(file, REPORT);
|
||
const modified = Date.parse('2026-09-08T17:48:30.000Z') / 1000;
|
||
fs.utimesSync(file, modified, modified);
|
||
const transcript: PlanCountTranscript = {
|
||
status: 'ready', calls: [structuredClone(CAPTURED_CALL)],
|
||
assistantMessages: [{ ...CAPTURED_FINAL, text: CAPTURED_FINAL.text.replace(CAPTURED_PATH, file) }],
|
||
};
|
||
return { dir, file, startedAt, transcript, cleanup: () => fs.rmSync(dir, { recursive: true, force: true }) };
|
||
}
|
||
|
||
describe('Design manual completion binds its current exit gate to the fresh report', () => {
|
||
const designFixture = () => {
|
||
const f = fixture();
|
||
fs.writeFileSync(f.file, REPORT.replaceAll('DX', 'Design').replace('Design CLEARED', 'DESIGN CLEARED'));
|
||
const time = Date.parse('2026-09-08T17:48:30Z') / 1000; fs.utimesSync(f.file, time, time);
|
||
return f;
|
||
};
|
||
// AZ's final public declaration; detailed score/decision tables are omitted.
|
||
const done = (file: string) => 'Exit gate passed: every issue has its own recorded decision, the plan file\'s last heading is the review report, it has the table and VERDICT, and the final line is the unbolded sentinel.\n\n' +
|
||
' **STATUS: DONE.** Design review of PLAN.md is complete. The reviewed plan is at `' + file + '`.';
|
||
test('recognizes the bound Design declaration independently of terminal layout or ExitPlanMode', () => {
|
||
const f = designFixture();
|
||
try {
|
||
for (const text of [
|
||
done(f.file), done(f.file).replace('**STATUS: DONE.** ', '').replace('Design review of PLAN.md is complete', 'The Design review has been completed')
|
||
.replace('The reviewed plan is at', 'The review report was written to'),
|
||
done(f.file) + '\n\nEng review is pending. Implementation should not start until you leave plan mode.',
|
||
done(f.file) + '\n\n"Design review is pending."',
|
||
done(f.file) + '\n\n"This report is historical."',
|
||
]) {
|
||
f.transcript.assistantMessages[0]!.text = text;
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'completion_summary'), text).toBe(true);
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
}
|
||
} finally {f.cleanup();}
|
||
});
|
||
test('rejects unbound, quoted, historical, provisional and conflicting current completion', () => {
|
||
const f = designFixture();
|
||
try {
|
||
const complete = done(f.file);
|
||
for (const text of [
|
||
complete.replace('Design review of PLAN.md is complete.', 'Everything is done.'),
|
||
complete.replaceAll('Design review', 'Eng review'),
|
||
complete.replace('PLAN.md', 'OTHER.md'), complete.replace(f.file, f.file + '.other'),
|
||
complete.replace('Exit gate passed:', 'Exit gate pending:'), complete.replace('Exit gate passed:', 'Exit gate did not pass:'),
|
||
complete.replace('is complete.', 'is not complete.'), complete.replace('is complete.', 'will be complete.'),
|
||
complete.replace('is complete.', 'is complete if approved.'),
|
||
complete.replace('Exit gate passed:', 'If approved, exit gate passed:'),
|
||
complete.replace('**STATUS:', 'Historical example:\n**STATUS:'),
|
||
complete.replace('The reviewed plan', '\n\nThe reviewed plan'),
|
||
'> ' + complete.replaceAll('\n', '\n> '), '```text\n' + complete + '\n```',
|
||
'An example follows:\n\n"' + complete.replaceAll('\n', ' ') + '"',
|
||
'Source example:\n\n' + complete,
|
||
complete.replace('Design review of PLAN.md is complete.', '"Design review of PLAN.md is complete."'),
|
||
complete + '\n\nThe reviewed plan is at `' + f.file + '.other`.',
|
||
complete + '\n\nDesign review is pending.',
|
||
complete + '\n\nDesign review is "still pending".',
|
||
complete + '\n\nDesign review is complete only if approved.',
|
||
complete + '\n\nThis review is subject to approval.',
|
||
complete + '\n\nThe exit gate failed.',
|
||
complete + '\n\nThe Design review is cancelled.',
|
||
complete + '\n\nThis report is historical.',
|
||
complete.replace('Exit gate passed:', 'This gate belongs to a different plan. Exit gate passed:'),
|
||
complete + '\n\nThis gate belongs to a different plan.',
|
||
complete + '\n\nThe exit gate has been "failed".',
|
||
complete + '\n\nThe exit gate did not pass.',
|
||
complete + '\n\nThe exit gate has not passed.',
|
||
complete + '\n\nThis report is ‘withdrawn’.',
|
||
complete + '\n\nOne design decision remains unresolved.',
|
||
complete + '\n\nThis review requires approval.',
|
||
complete + '\n\nIf approved, this review is complete.',
|
||
complete + '\n\nWaiting for your decision.',
|
||
complete + '\n\nPlease confirm?',
|
||
]) {
|
||
f.transcript.assistantMessages[0]!.text = text;
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'completion_summary'), text).toBe(false);
|
||
}
|
||
} finally {f.cleanup();}
|
||
});
|
||
test('the Design route preserves native ownership, answer order and artifact readiness', () => {
|
||
const f = designFixture();
|
||
try {
|
||
f.transcript.assistantMessages[0]!.text = done(f.file);
|
||
for (const change of [
|
||
(t: any) => {t.status = 'missing';}, (t: any) => {t.calls = [];},
|
||
(t: any) => {t.calls[0].answered = false;}, (t: any) => {t.calls[0].failed = true;},
|
||
(t: any) => {t.calls[0].sessionId = 'foreign';}, (t: any) => {t.calls[0].answeredAt = '2026-09-08T17:50:00Z';},
|
||
(t: any) => {t.calls[0].answeredAt = undefined;},
|
||
(t: any) => {t.assistantMessages[0].timestamp = '2999-01-01T00:00:00Z';},
|
||
(t: any) => {t.planReadyRequests = [{sessionId: CAPTURED_CALL.sessionId, toolUseId: 'exit', timestamp: '2026-09-08T17:49:40Z', failed: true}];},
|
||
(t: any) => {t.assistantMessages.push({...t.assistantMessages[0], timestamp: '2026-09-08T17:50:00Z', text: 'Still reviewing.'});},
|
||
]) {
|
||
const t = structuredClone(f.transcript); change(t);
|
||
expect(hasNativePlanTerminal(t, f.file, f.startedAt, 'completion_summary'), change.toString()).toBe(false);
|
||
}
|
||
for (const timestamp of ['2026-09-08T17:44:00Z', '2026-09-08T17:50:00Z']) {
|
||
const time = Date.parse(timestamp) / 1000; fs.utimesSync(f.file, time, time);
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'completion_summary')).toBe(false);
|
||
}
|
||
for (const report of ['# Draft\n', REPORT + '\n## Still editing\n', REPORT,
|
||
REPORT.replaceAll('DX', 'Design').replace('Design CLEARED', 'DESIGN CLEARED').replace('clean', 'pending'),
|
||
REPORT.replaceAll('DX', 'Design').replace('Design CLEARED', 'DESIGN CLEARED')
|
||
.replace('NO UNRESOLVED DECISIONS', 'VERDICT: NOT CLEARED\n\nNO UNRESOLVED DECISIONS'),
|
||
]) {
|
||
fs.writeFileSync(f.file, report);
|
||
const time = Date.parse('2026-09-08T17:48:30Z') / 1000; fs.utimesSync(f.file, time, time);
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'completion_summary')).toBe(false);
|
||
}
|
||
fs.rmSync(f.file);
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'completion_summary')).toBe(false);
|
||
} finally {f.cleanup();}
|
||
});
|
||
test('a cleared prefix cannot hide a conditional, historical, cancelled or foreign report verdict', () => {
|
||
const f = designFixture();
|
||
try {
|
||
f.transcript.assistantMessages[0]!.text = done(f.file);
|
||
const report = fs.readFileSync(f.file, 'utf8');
|
||
for (const verdict of [
|
||
'DESIGN CLEARED only if approval is granted',
|
||
'DESIGN CLEARED is not the current verdict; review remains incomplete',
|
||
'DESIGN CLEARED is historical; this report has been cancelled',
|
||
'DESIGN CLEARED. This report is "historical".',
|
||
'DESIGN CLEARED. This report belongs to a different plan.',
|
||
'DESIGN CLEARED?', 'DESIGN CLEARED is a placeholder verdict',
|
||
]) {
|
||
fs.writeFileSync(f.file, report.replace('DESIGN CLEARED — eng review required', verdict));
|
||
const time = Date.parse('2026-09-08T17:48:30Z') / 1000; fs.utimesSync(f.file, time, time);
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'completion_summary'), verdict).toBe(false);
|
||
}
|
||
fs.writeFileSync(f.file, report.replace('DESIGN CLEARED — eng review required',
|
||
'DESIGN CLEARED (2026-09-11, commit 1a3f728) — eng review required. "This report is historical."'));
|
||
const time = Date.parse('2026-09-08T17:48:30Z') / 1000; fs.utimesSync(f.file, time, time);
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'completion_summary')).toBe(true);
|
||
} finally {f.cleanup();}
|
||
});
|
||
});
|
||
|
||
describe('native plan completion and final report', () => {
|
||
test('recognizes captured completed native DevEx prose despite a nonmatching terminal heading', () => {
|
||
const f = fixture();
|
||
try {
|
||
const visible = '●' + f.transcript.assistantMessages[0]!.text.replace(/ /g, '') + '\nCrunched for 10m 9s ·done 5:49PM\n❯ ';
|
||
expect(classifyPlanCountFrame(visible)).toBeNull();
|
||
expect(hasNativePlanCompletion(f.transcript, f.file, f.startedAt)).toBe(true);
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test('rejects pending, failed, missing, mixed-session and untimestamped native coverage', () => {
|
||
const f = fixture();
|
||
try {
|
||
for (const change of [
|
||
(t: any) => { t.status = 'missing'; },
|
||
(t: any) => { t.status = 'error'; },
|
||
(t: any) => { t.calls = []; },
|
||
(t: any) => { t.calls[0].answered = false; },
|
||
(t: any) => { t.calls[0].failed = true; },
|
||
(t: any) => { t.calls[0].answeredAt = undefined; },
|
||
(t: any) => { t.calls[0].sessionId = 'foreign'; },
|
||
(t: any) => { t.assistantMessages.push({ ...t.assistantMessages[0], sessionId: 'foreign' }); },
|
||
(t: any) => { t.assistantMessages = []; },
|
||
]) {
|
||
const t = structuredClone(f.transcript); change(t);
|
||
expect(hasNativePlanCompletion(t, f.file, f.startedAt)).toBe(false);
|
||
}
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test('requires the latest native announcement to follow every answer', () => {
|
||
const f = fixture();
|
||
try {
|
||
f.transcript.calls[0]!.answeredAt = '2026-09-08T17:50:00.000Z';
|
||
expect(hasNativePlanCompletion(f.transcript, f.file, f.startedAt)).toBe(false);
|
||
f.transcript.calls[0]!.answeredAt = CAPTURED_CALL.answeredAt;
|
||
f.transcript.assistantMessages.push({ ...CAPTURED_FINAL, text: 'Please answer the remaining question.', timestamp: '2026-09-08T17:50:00.000Z' });
|
||
expect(hasNativePlanCompletion(f.transcript, f.file, f.startedAt)).toBe(false);
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test('quoted completions, refusals, provisional writes, wrong paths and waiting questions are not done', () => {
|
||
const f = fixture();
|
||
try {
|
||
const complete = f.transcript.assistantMessages[0]!.text;
|
||
for (const text of [
|
||
'> ' + complete, '```text\n' + complete + '\n```',
|
||
'Example completion:\n' + complete,
|
||
"I cannot complete this review.\nPlan written to: `" + f.file + '`',
|
||
'Now writing the complete plan file with all resolved findings and the review report.',
|
||
complete.replace(f.file, f.file + '.other'),
|
||
complete.replace('Plan written to:', 'Should I proceed?\nPlan written to:'),
|
||
complete.replace('Plan written to:', 'Waiting for your decision.\nPlan written to:'),
|
||
]) {
|
||
f.transcript.assistantMessages[0]!.text = text;
|
||
expect(hasNativePlanCompletion(f.transcript, f.file, f.startedAt), text).toBe(false);
|
||
}
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test('missing, stale, changed-after-announcement and provisional reports cannot complete', () => {
|
||
const f = fixture();
|
||
try {
|
||
for (const date of ['2026-09-08T17:38:00Z', '2026-09-08T17:44:00Z', '2026-09-08T17:50:00Z']) {
|
||
const time = Date.parse(date) / 1000; fs.utimesSync(f.file, time, time);
|
||
expect(hasNativePlanCompletion(f.transcript, f.file, f.startedAt)).toBe(false);
|
||
}
|
||
for (const report of ['# Draft\n', '## GSTACK REVIEW REPORT\n', REPORT + '\n## Still editing\n']) {
|
||
fs.writeFileSync(f.file, report); const time = Date.parse('2026-09-08T17:48:30Z') / 1000; fs.utimesSync(f.file, time, time);
|
||
expect(hasNativePlanCompletion(f.transcript, f.file, f.startedAt)).toBe(false);
|
||
}
|
||
fs.rmSync(f.file);
|
||
expect(hasNativePlanCompletion(f.transcript, f.file, f.startedAt)).toBe(false);
|
||
expect(hasNativePlanCompletion(f.transcript, 'review.md', f.startedAt)).toBe(false);
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test('a fenced report example or an unclosed fence never supplies completion markers', () => {
|
||
const f = fixture();
|
||
try {
|
||
for (const report of [
|
||
'# Plan\nExample report:\n```markdown\n' + REPORT + '\n```\n',
|
||
'# Plan\nExample report:\n~~~markdown\n' + REPORT + '\n~~~\n',
|
||
'```markdown\n' + REPORT,
|
||
'~~~markdown\n' + REPORT,
|
||
// An apparent shorter/mismatched/annotated close leaves the report
|
||
// inside the original code fence, rather than making it a real report.
|
||
'````markdown\n```\n' + REPORT + '\n````\n',
|
||
'~~~markdown\n```\n' + REPORT + '\n~~~\n',
|
||
'```markdown\n```still-code\n' + REPORT + '\n```\n',
|
||
' ~~~~markdown\n ~~~\n' + REPORT + '\n ~~~~\n',
|
||
REPORT + '\n```unclosed\n',
|
||
]) {
|
||
fs.writeFileSync(f.file, report);
|
||
const time = Date.parse('2026-09-08T17:48:30Z') / 1000;
|
||
fs.utimesSync(f.file, time, time);
|
||
expect(hasNativePlanCompletion(f.transcript, f.file, f.startedAt), report).toBe(false);
|
||
}
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test('real code fences before the final report do not hide that report', () => {
|
||
const f = fixture();
|
||
try {
|
||
for (const prefix of [
|
||
'```typescript\nconst answer = 42;\n```\n',
|
||
'~~~typescript\nconst answer = 42;\n~~~~~\n',
|
||
' ````typescript\n```still-code\n ````` \t\n',
|
||
'```markdown\n' + REPORT + '\n```\n',
|
||
]) {
|
||
fs.writeFileSync(f.file, '# Plan\n' + prefix + '\n' + REPORT);
|
||
const time = Date.parse('2026-09-08T17:48:30Z') / 1000;
|
||
fs.utimesSync(f.file, time, time);
|
||
expect(hasNativePlanCompletion(f.transcript, f.file, f.startedAt), prefix).toBe(true);
|
||
}
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test.skipIf(process.platform === 'win32')('real PTY completes from a native report without a display alias', async () => {
|
||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'native-completion-pty-'));
|
||
const fake = path.join(dir, 'fake-claude');
|
||
const worker = path.join(dir, 'worker.ts');
|
||
const output = path.join(dir, 'plan.md');
|
||
const record = path.join(dir, 'inputs.jsonl');
|
||
const result = path.join(dir, 'result.json');
|
||
const runner = pathToFileURL(path.join(import.meta.dir, 'helpers/claude-pty-runner.ts')).href;
|
||
fs.writeFileSync(fake, `#!${process.execPath}\n` + String.raw`
|
||
import * as fs from 'node:fs';
|
||
import * as path from 'node:path';
|
||
const sessionId = 'native-completion-fixture';
|
||
const project = path.join(process.env.CLAUDE_CONFIG_DIR, 'projects', sessionId);
|
||
fs.mkdirSync(project, { recursive: true });
|
||
const file = path.join(project, sessionId + '.jsonl');
|
||
const native = (role, content, extra = {}) => fs.appendFileSync(file, JSON.stringify({
|
||
cwd: process.cwd(), sessionId, isSidechain: false,
|
||
timestamp: new Date().toISOString(), message: { role, content }, ...extra,
|
||
}) + '\n');
|
||
let sent = false;
|
||
process.stdin.setRawMode?.(true);
|
||
process.stdin.on('data', data => {
|
||
fs.appendFileSync(process.env.PROBE_INPUTS, JSON.stringify(data.toString()) + '\n');
|
||
if (sent) return; sent = true;
|
||
const q = { header: 'Error quality', question: 'D1 — Fix the missing error recovery path?', options: [{ label: 'Add recovery hint' }, { label: 'Keep vague message' }] };
|
||
native('assistant', [{ type: 'tool_use', id: 'finding', name: 'AskUserQuestion', input: { questions: [q] } }]);
|
||
const answeredAt = new Date(Date.now() - 100).toISOString();
|
||
native('user', [{ type: 'tool_result', tool_use_id: 'finding', content: 'Answered.' }], {
|
||
timestamp: answeredAt, toolUseResult: { answers: { [q.question]: q.options[0].label } },
|
||
});
|
||
fs.writeFileSync(process.env.PROBE_PLAN, process.env.PROBE_REPORT);
|
||
const text = 'DX review complete. One finding resolved interactively.\n\nPlan written to: ' + String.fromCharCode(96) + process.env.PROBE_PLAN + String.fromCharCode(96);
|
||
native('assistant', [{ type: 'text', text }], { timestamp: new Date(Date.now() + 5).toISOString() });
|
||
process.stdout.write('407 +NO UNRESOLVED DECISIONS\n●' + text.replace(/ /g, '') + '\nCrunched for 10m 9s ·done 5:49PM\n❯ ');
|
||
});
|
||
process.stdin.resume();
|
||
process.stdout.write('PTY_READY:'+process.env.PROBE_INPUTS+'\x1b[2J\x1b[H');
|
||
`);
|
||
fs.chmodSync(fake, 0o755);
|
||
fs.writeFileSync(worker, `import { runPlanSkillCounting } from ${JSON.stringify(runner)};\n` +
|
||
`const result = await runPlanSkillCounting({skillName:'plan-devex-review',slashCommand:'/plan-devex-review',followUpPrompt:'# Native completion fixture',expectedPlanPath:${JSON.stringify(output)},isLastStep0AUQ:()=>false,isReviewAUQ:()=>true,reviewCountCeiling:8,timeoutMs:33000,startupReadyMarker:${JSON.stringify('PTY_READY:'+record)},env:${JSON.stringify({PROBE_PLAN:output,PROBE_INPUTS:record,PROBE_REPORT:REPORT})}});\n` +
|
||
`await Bun.write(${JSON.stringify(result)},JSON.stringify(result));\n`);
|
||
const child = Bun.spawn([process.execPath, worker], {
|
||
env: { ...process.env, EVALS_HERMETIC: '1', EVALS_RUN_ID: '', BROWSE_TERMINAL_BINARY: fake },
|
||
stdout: 'pipe', stderr: 'pipe',
|
||
});
|
||
const timer = setTimeout(() => child.kill('SIGKILL'), 35000);
|
||
try {
|
||
const [code, stdout, stderr] = await Promise.all([child.exited, new Response(child.stdout).text(), new Response(child.stderr).text()]);
|
||
expect(code, stdout + stderr).toBe(0);
|
||
const observation = JSON.parse(fs.readFileSync(result, 'utf8'));
|
||
expect(observation.outcome).toBe('completion_summary');
|
||
expect(observation.summary).toContain('native review completion and final report verified');
|
||
expect(observation.reviewCount).toBe(1);
|
||
expect(observation.step0Count).toBe(0);
|
||
expect(observation.transcript.calls).toHaveLength(1);
|
||
expect(fs.readFileSync(record, 'utf8').trim().split('\n').map(line => JSON.parse(line))).toEqual(['/plan-devex-review\r']);
|
||
expect(fs.existsSync(output)).toBe(true);
|
||
} finally { clearTimeout(timer); child.kill('SIGKILL'); fs.rmSync(dir, { recursive: true, force: true }); }
|
||
}, 40000);
|
||
});
|
||
|
||
// Last answered call and last finalized native text at the exact G soft-terminal race.
|
||
const CAPTURED_ENG_STREAM = {
|
||
"status": "ready",
|
||
"calls": [
|
||
{
|
||
"sessionId": "4c3d29fb-2a00-4e5d-b3c6-9b584986e340",
|
||
"toolUseId": "toolu_011KrMyU6fjW7RQ7aAVJ4DfA",
|
||
"questions": [
|
||
{
|
||
"question": "D7 — Performance: 5 sequential IDP calls that could be Promise.all'd\nProject/branch/task: Multi-tenant Auth Refactor on main\nELI10: During token validation, the code makes 5 API calls to the identity provider (IDP) one at a time. But the calls are independent — none uses the result of the previous one. Changing from sequential to Promise.all([…]) takes one line of code and cuts auth latency by ~5x. At 100ms per call, sequential = ~500ms; parallel = ~100ms. This matters most under load: auth is on the critical path of every authenticated request.\nStakes if we pick wrong: Every authenticated request pays a 400ms tax that could be eliminated with one line. Under load, slow auth creates timeout cascades — auth is typically among the first things to degrade.\nRecommendation: A — trivially fix it now. The plan itself says “calls are independent” and “trivially” parallelizable. The only reason not to is if there’s a hidden ordering dependency the plan doesn’t mention.\nCompleteness: Note: options differ in kind, not coverage — no completeness score.\nPros / cons:\nA) Parallelize the 5 IDP calls with Promise.all in this PR (recommended)\n ✅ 5x latency reduction on the auth critical path for free — one line change\n ✅ Under load, parallel calls bound latency to the slowest single call instead of sum of all\n ❌ If any hidden ordering dependency exists (not mentioned in the plan), it will surface as a test failure — easily caught\nB) Defer parallelization to a follow-up performance PR\n ✅ Keeps this PR focused on the refactor\n ❌ The plan already flagged this as trivial; deferring creates a PR that self-documents debt without resolving it\nNet: The plan already identified this as trivial. Doing it now costs 5 minutes and makes the auth path meaningfully faster for all users. <gstack-qid:plan-eng-review-perf-idp-sequential-calls>",
|
||
"header": "Performance",
|
||
"options": [
|
||
{
|
||
"label": "A) Parallelize IDP calls now (Recommended)",
|
||
"description": "Promise.all the 5 independent calls. ~5 min, 5x auth latency improvement."
|
||
},
|
||
{
|
||
"label": "B) Defer to a follow-up PR",
|
||
"description": "Keeps this PR focused on refactor; acknowledged debt."
|
||
}
|
||
],
|
||
"multiSelect": false
|
||
}
|
||
],
|
||
"answered": true,
|
||
"failed": false,
|
||
"answers": {
|
||
"D7 — Performance: 5 sequential IDP calls that could be Promise.all'd\nProject/branch/task: Multi-tenant Auth Refactor on main\nELI10: During token validation, the code makes 5 API calls to the identity provider (IDP) one at a time. But the calls are independent — none uses the result of the previous one. Changing from sequential to Promise.all([…]) takes one line of code and cuts auth latency by ~5x. At 100ms per call, sequential = ~500ms; parallel = ~100ms. This matters most under load: auth is on the critical path of every authenticated request.\nStakes if we pick wrong: Every authenticated request pays a 400ms tax that could be eliminated with one line. Under load, slow auth creates timeout cascades — auth is typically among the first things to degrade.\nRecommendation: A — trivially fix it now. The plan itself says “calls are independent” and “trivially” parallelizable. The only reason not to is if there’s a hidden ordering dependency the plan doesn’t mention.\nCompleteness: Note: options differ in kind, not coverage — no completeness score.\nPros / cons:\nA) Parallelize the 5 IDP calls with Promise.all in this PR (recommended)\n ✅ 5x latency reduction on the auth critical path for free — one line change\n ✅ Under load, parallel calls bound latency to the slowest single call instead of sum of all\n ❌ If any hidden ordering dependency exists (not mentioned in the plan), it will surface as a test failure — easily caught\nB) Defer parallelization to a follow-up performance PR\n ✅ Keeps this PR focused on the refactor\n ❌ The plan already flagged this as trivial; deferring creates a PR that self-documents debt without resolving it\nNet: The plan already identified this as trivial. Doing it now costs 5 minutes and makes the auth path meaningfully faster for all users. <gstack-qid:plan-eng-review-perf-idp-sequential-calls>": "A) Parallelize IDP calls now (Recommended)"
|
||
},
|
||
"unansweredQuestionIndices": [],
|
||
"answeredAt": "2026-09-08T18:46:43.934Z"
|
||
}
|
||
],
|
||
"assistantMessages": [
|
||
{
|
||
"sessionId": "4c3d29fb-2a00-4e5d-b3c6-9b584986e340",
|
||
"text": "Tests decision: add characterization tests for legacyAuthFlow() before rewriting. Moving to Section 4: Performance Review.\n\n**Finding 4 [P2] (confidence: 9/10):** The plan states: \"Token validation issues 5 sequential API calls to the IDP; they could be parallelized via Promise.all trivially (calls are independent).\"\n\nThis is a free 5x latency improvement being left on the table. At 100ms per IDP call (typical), sequential = ~500ms per auth validation. Parallel via Promise.all = ~100ms.",
|
||
"timestamp": "2026-09-08T18:46:30.531Z"
|
||
}
|
||
]
|
||
};
|
||
const CAPTURED_DX_HANDOFF = {
|
||
"sessionId": "cc1b321f-f306-4086-a236-34dd30f457be",
|
||
"toolUseId": "toolu_01AGy3uosDrjJJUv1aKGkhRY",
|
||
"questions": [
|
||
{
|
||
"question": "DX review done. 5 findings resolved, 7 implementation tasks written. What's next?\n\nEvalKit SDK / DX POLISH review / main\nThe DX review found API design issues (argument order, auth error), a broken getting-started path, a TTHW gap, and a cliff-edge upgrade. All resolved with user decisions. The full output is at gstack-test-plan-devex.md. Eng review is the required gate before shipping.\n<gstack-qid:devex-next-steps>",
|
||
"header": "Next step",
|
||
"multiSelect": false,
|
||
"options": [
|
||
{
|
||
"label": "Run /plan-eng-review (Recommended)",
|
||
"description": "Required gate. The arg-order fix and --skip-ci flag both have architectural implications that eng review should validate."
|
||
},
|
||
{
|
||
"label": "Ready to implement",
|
||
"description": "Start building the 7 tasks. Run /devex-review after shipping to measure actual TTHW vs the <2-min target."
|
||
},
|
||
{
|
||
"label": "Skip, handle manually",
|
||
"description": "You'll manage next steps outside this session."
|
||
}
|
||
]
|
||
}
|
||
],
|
||
"answered": true,
|
||
"failed": false,
|
||
"answers": {
|
||
"DX review done. 5 findings resolved, 7 implementation tasks written. What's next?\n\nEvalKit SDK / DX POLISH review / main\nThe DX review found API design issues (argument order, auth error), a broken getting-started path, a TTHW gap, and a cliff-edge upgrade. All resolved with user decisions. The full output is at gstack-test-plan-devex.md. Eng review is the required gate before shipping.\n<gstack-qid:devex-next-steps>": "Run /plan-eng-review (Recommended)"
|
||
},
|
||
"unansweredQuestionIndices": [],
|
||
"answeredAt": "2026-09-08T18:46:16.771Z"
|
||
};
|
||
|
||
describe('guarded native terminal and report lifecycle', () => {
|
||
test('G Eng streamed summary cannot finish while native text/file is still pending', () => {
|
||
const f = fixture();
|
||
try {
|
||
const visible = '● Performance decision: parallelize the 5 IDP calls. All 4 review sections complete.\n\nCompletionSummary:\n- Step 0: Scope reduced\n- Architecture Review:1 issue found\n- Code Quality Review:1 issue found\n';
|
||
expect(classifyPlanCountFrame(visible)).toBe('completion_summary');
|
||
const t: PlanCountTranscript = structuredClone(CAPTURED_ENG_STREAM);
|
||
const answerTime = Math.max(...t.calls.map(c => Date.parse(c.answeredAt!)));
|
||
fs.utimesSync(f.file, (answerTime + 1) / 1000, (answerTime + 1) / 1000);
|
||
expect(hasNativePlanTerminal(t, f.file, f.startedAt, 'completion_summary')).toBe(false);
|
||
t.assistantMessages.push({ sessionId: t.calls[0]!.sessionId, timestamp: new Date(answerTime + 2).toISOString(),
|
||
text: 'Completion Summary:\n- Architecture Review: 1 issue resolved\n- Code Quality Review: 1 issue resolved\n- Test Review: 1 gap resolved\n- Performance Review: 1 issue resolved\n' });
|
||
for (const content of ['', '# Draft\n', '## GSTACK REVIEW REPORT\n', REPORT.replace('NO UNRESOLVED DECISIONS', ''), REPORT.replace('| DX Review | clean | resolved |\n', '')]) {
|
||
fs.writeFileSync(f.file, content); fs.utimesSync(f.file, (answerTime + 3) / 1000, (answerTime + 3) / 1000);
|
||
expect(hasNativePlanTerminal(t, f.file, f.startedAt, 'completion_summary'), content).toBe(false);
|
||
}
|
||
fs.writeFileSync(f.file, REPORT); fs.utimesSync(f.file, (answerTime + 3) / 1000, (answerTime + 3) / 1000);
|
||
expect(hasNativePlanTerminal(t, f.file, f.startedAt, 'completion_summary')).toBe(true);
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test('the actual post-report DX handoff does not require rewriting an unchanged report', () => {
|
||
const f = fixture();
|
||
try {
|
||
const t: PlanCountTranscript = structuredClone(f.transcript);
|
||
const sessionId = t.calls[0]!.sessionId;
|
||
t.calls.push({ ...structuredClone(CAPTURED_DX_HANDOFF), sessionId });
|
||
t.planReadyRequests = [{ sessionId, toolUseId: 'exit', timestamp: '2026-09-08T18:46:27.000Z', failed: false }];
|
||
expect(hasNativePlanTerminal(t, f.file, f.startedAt, 'plan_ready')).toBe(true);
|
||
// Unknown, free-form, partly unanswered and substantive menus remain
|
||
// freshness boundaries, even if they reuse the same next-step header.
|
||
for (const mutate of [
|
||
(c: any) => { c.questions[0].question = c.questions[0].question.replace('devex-next-steps', 'new-change'); },
|
||
(c: any) => { c.answers[c.questions[0].question] = 'Also change the SDK auth behavior'; },
|
||
(c: any) => { c.questions[0].options.push({ label: 'Change the error message now' }); },
|
||
(c: any) => { c.unansweredQuestionIndices = [1]; },
|
||
]) {
|
||
const changed = structuredClone(t); mutate(changed.calls[1]);
|
||
expect(hasNativePlanTerminal(changed, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
}
|
||
t.planReadyRequests[0]!.failed = true;
|
||
expect(hasNativePlanTerminal(t, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test('pending/refused/mixed calls, stale or incomplete reports and quoted summaries cannot complete', () => {
|
||
const f = fixture();
|
||
try {
|
||
const t = structuredClone(f.transcript);
|
||
t.planReadyRequests = [{ sessionId: t.calls[0]!.sessionId, toolUseId: 'exit', timestamp: CAPTURED_FINAL.timestamp, failed: false }];
|
||
expect(hasNativePlanTerminal(t, f.file, f.startedAt, 'plan_ready')).toBe(true);
|
||
const retried = structuredClone(t);
|
||
retried.calls.unshift({ ...structuredClone(retried.calls[0]!), toolUseId: 'failed-before-retry', answered: false, failed: true, answers: undefined });
|
||
expect(hasNativePlanTerminal(retried, f.file, f.startedAt, 'plan_ready')).toBe(true);
|
||
retried.calls[0]!.questions[0]!.question = 'A different unresolved question';
|
||
expect(hasNativePlanTerminal(retried, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
for (const mutate of [
|
||
(t: any) => { t.calls[0].answered = false; },
|
||
(t: any) => { t.calls[0].failed = true; },
|
||
(t: any) => { t.calls[0].sessionId = 'other'; },
|
||
(t: any) => { t.planReadyRequests[0].failed = true; },
|
||
(t: any) => { t.planReadyRequests[0].timestamp = t.calls[0].answeredAt; },
|
||
(t: any) => { t.planReadyRequests = []; },
|
||
(t: any) => { t.status = 'missing'; },
|
||
]) { const changed = structuredClone(t); mutate(changed); expect(hasNativePlanTerminal(changed, f.file, f.startedAt, 'plan_ready')).toBe(false); }
|
||
const summary = 'Completion Summary:\n- Review: 4 findings resolved\n';
|
||
for (const text of ['> ' + summary, 'Example:\n' + summary, '```text\n' + summary + '```', 'Here is an example:\n\n' + summary, 'Here is a quoted result:\n```text\n' + summary + '```', 'I cannot complete the review.\n' + summary, 'Completion Summary:\n', summary + 'Waiting for your answer.']) {
|
||
t.assistantMessages[0]!.text = text;
|
||
expect(hasNativePlanTerminal(t, f.file, f.startedAt, 'completion_summary'), text).toBe(false);
|
||
}
|
||
const old = Date.parse('2026-09-08T17:40:00Z') / 1000; fs.utimesSync(f.file, old, old);
|
||
expect(hasNativePlanTerminal(t, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
} finally { f.cleanup(); }
|
||
});
|
||
});
|
||
|
||
test.skipIf(process.platform === 'win32')('real PTY waits through streamed summary, another question and partial report writes', async () => {
|
||
await Promise.all(['completion_summary', 'plan_ready', 'compact_plan_ready', 'ready_with_unsanctioned_write'].map(async (terminal) => {
|
||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'guarded-terminal-pty-'));
|
||
const fake = path.join(dir, 'fake-claude');
|
||
const worker = path.join(dir, 'worker.ts');
|
||
const output = path.join(dir, 'plan.md');
|
||
const record = path.join(dir, 'inputs.jsonl');
|
||
const events = path.join(dir, 'events.jsonl');
|
||
const result = path.join(dir, 'result.json');
|
||
const runner = pathToFileURL(path.join(import.meta.dir, 'helpers/claude-pty-runner.ts')).href;
|
||
fs.writeFileSync(fake, `#!${process.execPath}\n` + String.raw`
|
||
import * as fs from 'node:fs';
|
||
import * as path from 'node:path';
|
||
const sessionId = 'guarded-terminal-fixture';
|
||
const project = path.join(process.env.CLAUDE_CONFIG_DIR, 'projects', sessionId);
|
||
fs.mkdirSync(project, { recursive: true });
|
||
const file = path.join(project, sessionId + '.jsonl');
|
||
const native = (role, content, extra = {}) => fs.appendFileSync(file, JSON.stringify({
|
||
cwd: process.cwd(), sessionId, isSidechain: false, timestamp: new Date().toISOString(), message: { role, content }, ...extra,
|
||
}) + '\n');
|
||
const event = name => fs.appendFileSync(process.env.PROBE_EVENTS, JSON.stringify({ name, at: Date.now() }) + '\n');
|
||
const q = { header: 'Tests', question: 'D2 — Test “Would you like to proceed?” <gstack-qid:plan-eng-test-retry>', options: [{ label: 'Add retry assertion' }, { label: 'Leave untested' }] };
|
||
const ask = id => native('assistant', [{ type: 'tool_use', id, name: 'AskUserQuestion', input: { questions: [q] } }]);
|
||
const answer = id => native('user', [{ type: 'tool_result', tool_use_id: id, content: 'Answered.' }], { toolUseResult: { answers: { [q.question]: q.options[0].label } } });
|
||
const summary = 'Completion Summary:\n- Architecture Review: 1 issue resolved\n- Test Review: 1 missing test resolved\n';
|
||
const compact = process.env.PROBE_TERMINAL === 'compact_plan_ready';
|
||
const ready = () => process.stdout.write(process.env.PROBE_TERMINAL === 'ready_with_unsanctioned_write'
|
||
? '\x1b[2J\x1b[H⏺ Write(src/unreviewed.ts)\nClaude has written up a plan and is ready to execute.\n'
|
||
: compact
|
||
? '\x1b[2J\x1b[HExit plan mode?\n\nClaude wants to exit plan mode\n\n❯ 1. Yes, and switch to default (ask each time) for this session\n 2. No\n'
|
||
: '\nClaude has written up a plan and is ready to execute. Would you like to proceed?\n❯ 1. Yes, and use auto mode\n 2. Yes, manually approve edits\n 3. Tell Claude what to change\n');
|
||
let stage = 0;
|
||
process.stdin.setRawMode?.(true);
|
||
process.stdin.on('data', data => {
|
||
fs.appendFileSync(process.env.PROBE_INPUTS, JSON.stringify(data.toString()) + '\n');
|
||
if (stage === 0) {
|
||
stage = 1; ask('first'); answer('first');
|
||
event('streamed-heading'); process.stdout.write('● Completion Summary:\n- Architecture Review: 1 issue resolved\n');
|
||
if (compact) {
|
||
// Native metadata for an unrelated question and a prior permission
|
||
// cannot authorize this ExitPlanMode menu. No report exists yet.
|
||
ask('second');
|
||
process.stdout.write('Do you want to make this edit to plan.md?\n❯ 1. Yes\n 2. Yes, and allow all edits\n 3. No\nEsc to cancel · Tab to amend\n⎿ Added 1 line\n');
|
||
event('unverified-compact-gate'); ready();
|
||
}
|
||
setTimeout(() => {
|
||
if (!compact) ask('second'); event('pending-question');
|
||
process.stdout.write('\n☐ Tests\n' + q.question + '\n❯ 1. Add retry assertion\n 2. Leave untested\nEnter to select · ↑/↓ to navigate · Esc to cancel\n');
|
||
}, 3500);
|
||
return;
|
||
}
|
||
if (stage === 1 && data.toString().includes('1')) {
|
||
stage = 2; answer('second'); event('answered-second');
|
||
fs.writeFileSync(process.env.PROBE_PLAN, '# Draft\n');
|
||
// A finalized summary plus the finished report is itself a valid terminal.
|
||
// Keep approval-only cases streamed until their native approval arrives.
|
||
if (process.env.PROBE_TERMINAL === 'completion_summary')
|
||
native('assistant', [{ type: 'text', text: summary }], { timestamp: new Date(Date.now() + 1).toISOString() });
|
||
process.stdout.write('\n● ' + summary);
|
||
if (compact) ready();
|
||
setTimeout(() => {
|
||
event('partial-report'); fs.writeFileSync(process.env.PROBE_PLAN, process.env.PROBE_REPORT.split('VERDICT:')[0]);
|
||
}, 3000);
|
||
setTimeout(() => {
|
||
event('complete-report'); fs.writeFileSync(process.env.PROBE_PLAN, process.env.PROBE_REPORT);
|
||
if (process.env.PROBE_TERMINAL !== 'completion_summary') {
|
||
native('assistant', [{ type: 'tool_use', id: 'rejected-ready', name: 'ExitPlanMode', input: {} }]);
|
||
native('user', [{ type: 'tool_result', tool_use_id: 'rejected-ready', is_error: true, content: 'Gate not ready' }]);
|
||
ready();
|
||
setTimeout(() => {
|
||
event('valid-native-ready'); native('assistant', [{ type: 'tool_use', id: 'valid-ready', name: 'ExitPlanMode', input: {} }]); ready();
|
||
}, 2500);
|
||
}
|
||
}, 6000);
|
||
} else if (stage === 2) event('unexpected-plan-approval-input');
|
||
});
|
||
process.stdin.resume();
|
||
process.stdout.write('PTY_READY:'+process.env.PROBE_INPUTS+'\x1b[2J\x1b[H');
|
||
`);
|
||
fs.chmodSync(fake, 0o755);
|
||
fs.writeFileSync(worker, `import { runPlanSkillCounting } from ${JSON.stringify(runner)};\n` +
|
||
`const result = await runPlanSkillCounting({skillName:'plan-eng-review',slashCommand:'/plan-eng-review',followUpPrompt:'# Completion fixture',expectedPlanPath:${JSON.stringify(output)},isLastStep0AUQ:()=>false,isReviewAUQ:()=>true,reviewCountCeiling:8,timeoutMs:43000,startupReadyMarker:${JSON.stringify('PTY_READY:'+record)},env:${JSON.stringify({PROBE_PLAN:output,PROBE_INPUTS:record,PROBE_EVENTS:events,PROBE_REPORT:REPORT,PROBE_TERMINAL:terminal})}});\n` +
|
||
`await Bun.write(${JSON.stringify(result)},JSON.stringify(result));\n`);
|
||
const child = Bun.spawn([process.execPath, worker], { env: { ...process.env,
|
||
EVALS_HERMETIC: '1', EVALS_RUN_ID: '', BROWSE_TERMINAL_BINARY: fake }, stdout: 'pipe', stderr: 'pipe' });
|
||
const timer = setTimeout(() => child.kill('SIGKILL'), 45000);
|
||
try {
|
||
const [code, stdout, stderr] = await Promise.all([child.exited, new Response(child.stdout).text(), new Response(child.stderr).text()]);
|
||
expect(code, stdout + stderr).toBe(0);
|
||
const observation = JSON.parse(fs.readFileSync(result, 'utf8'));
|
||
expect(fs.existsSync(output), `D19: report must exist before stopping; ${observation.summary}`).toBe(true);
|
||
expect(observation.outcome, JSON.stringify(observation)).toBe(
|
||
terminal === 'ready_with_unsanctioned_write' ? 'silent_write' :
|
||
terminal === 'compact_plan_ready' ? 'plan_ready' : terminal,
|
||
);
|
||
expect(observation.reviewCount).toBe(2);
|
||
expect(fs.readFileSync(output, 'utf8')).toBe(REPORT);
|
||
const recorded = fs.readFileSync(events, 'utf8').trim().split('\n').map(line => JSON.parse(line));
|
||
expect(recorded.map(e => e.name)).toEqual(['streamed-heading', ...(terminal === 'compact_plan_ready' ? ['unverified-compact-gate'] : []), 'pending-question', 'answered-second', 'partial-report', 'complete-report', ...(terminal !== 'completion_summary' ? ['valid-native-ready'] : [])]);
|
||
expect(fs.readFileSync(record, 'utf8').trim().split('\n').map(line => JSON.parse(line))).toEqual(['/plan-eng-review\r', '1']);
|
||
if (terminal !== 'completion_summary') expect(observation.transcript.planReadyRequests.map(r => r.failed)).toEqual([true, false]);
|
||
} finally { clearTimeout(timer); child.kill('SIGKILL'); await child.exited; fs.rmSync(dir, { recursive: true, force: true }); }
|
||
}));
|
||
}, 50000);
|
||
|
||
// Actual finalized CEO prose uses a relative output name, not the DevEx fallback's
|
||
// exact “Plan written to:” sentence. Its caller-owned file is checked independently.
|
||
test('captured completed CEO prose supports the ordinary summary path', () => {
|
||
const f = fixture();
|
||
try {
|
||
f.transcript.assistantMessages = [{
|
||
sessionId: f.transcript.calls[0]!.sessionId,
|
||
timestamp: "2026-09-08T17:58:52.319Z",
|
||
text: "CEO review complete. Here's the summary:\n\n**Plan:** Payment Processing — Test Coverage\n**Mode:** HOLD SCOPE | **Verdict:** CLEARED (with 4 additions)\n\n**4 gaps found and resolved through interactive review:**\n\n1. **CRITICAL** — \"assert correct receipt is generated\" is too vague. Spec: assert `charge_id`, `amount`, `currency` (at minimum) match the charge response.\n2. **WARNING** — \"fails clean\" doesn't name what's raised. Spec: `assert raises <PaymentFailedError>` (resolve exact class from error hierarchy at implementation).\n3. **WARNING** — Retry count is implicit. Spec: `assert len(mock_stripe_call_history) == 2` explicitly in Test 2.\n4. **WARNING** — No test file path. Spec: resolve by convention-matching existing test structure.\n\nEverything else (architecture, security, data flow, performance, observability, deployment, trajectory) is clean. Section 11 skipped — no UI scope. Review output written to `gstack-test-plan-ceo-paired.md`."
|
||
}];
|
||
expect(hasNativePlanCompletion(f.transcript, f.file, f.startedAt)).toBe(false);
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'completion_summary')).toBe(true);
|
||
f.transcript.calls = [];
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'completion_summary')).toBe(false);
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
|
||
const CAPTURED_NUMBERED_DX_HANDOFF = {
|
||
"sessionId": "0b7d664e-8d15-4c0e-8be2-d03a833d5d33",
|
||
"toolUseId": "toolu_01RgoUjV9vFRaSFoySjm5QqY",
|
||
"questions": [
|
||
{
|
||
"question": "D8 — DX review complete. 5 friction points resolved, DX score 4/10 → 8/10, TTHW target achievable. What’s next? <gstack-qid:devex-next-steps>",
|
||
"header": "Next steps",
|
||
"options": [
|
||
{
|
||
"label": "Run /plan-eng-review next (recommended)",
|
||
"description": "Three of the five fixes (CI skip flag, auth error upgrade, deprecated alias) have architectural implications. Eng review validates the contracts before implementation."
|
||
},
|
||
{
|
||
"label": "Ready to implement — run /devex-review after shipping",
|
||
"description": "Skip eng review for now. Implement the five recommendations, then run /devex-review on the live beta to verify TTHW <2 min is achieved."
|
||
},
|
||
{
|
||
"label": "Skip — I'll handle next steps manually",
|
||
"description": "No further review automation. You have the five recommendations; implementation is up to you."
|
||
}
|
||
],
|
||
"multiSelect": false
|
||
}
|
||
],
|
||
"answered": true,
|
||
"failed": false,
|
||
"answers": {
|
||
"D8 — DX review complete. 5 friction points resolved, DX score 4/10 → 8/10, TTHW target achievable. What’s next? <gstack-qid:devex-next-steps>": "Run /plan-eng-review next (recommended)"
|
||
},
|
||
"unansweredQuestionIndices": [],
|
||
"answeredAt": "2026-09-08T20:49:56.950Z"
|
||
};
|
||
|
||
|
||
describe('numbered completed DX handoff preserves native report freshness', () => {
|
||
function completedHandoffTranscript(f: ReturnType<typeof fixture>): PlanCountTranscript {
|
||
const transcript = structuredClone(f.transcript);
|
||
const sessionId = transcript.calls[0]!.sessionId;
|
||
transcript.calls.push({ ...structuredClone(CAPTURED_NUMBERED_DX_HANDOFF), sessionId });
|
||
transcript.planReadyRequests = [{
|
||
sessionId,
|
||
toolUseId: 'toolu_01QwXDuSELZNrgexVf88bQxG',
|
||
timestamp: '2026-09-08T20:50:30.273Z',
|
||
failed: false,
|
||
}];
|
||
return transcript;
|
||
}
|
||
|
||
test('the captured three-choice handoff changes orchestration without requiring another report write', () => {
|
||
const f = fixture();
|
||
try {
|
||
const transcript = completedHandoffTranscript(f);
|
||
expect(hasNativePlanTerminal(transcript, f.file, f.startedAt, 'plan_ready')).toBe(true);
|
||
transcript.planReadyRequests![0]!.failed = true;
|
||
expect(hasNativePlanTerminal(transcript, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
transcript.planReadyRequests = [];
|
||
expect(hasNativePlanTerminal(transcript, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
} finally {
|
||
f.cleanup();
|
||
}
|
||
});
|
||
|
||
test('substantive, incomplete, unanswered and mixed menus remain report freshness boundaries', () => {
|
||
const f = fixture();
|
||
try {
|
||
for (const mutate of [
|
||
(call: any) => { call.questions[0].question = call.questions[0].question.replace('review complete.', 'review still has unresolved auth behavior.'); },
|
||
(call: any) => { call.questions[0].question = call.questions[0].question.replace('devex-next-steps', 'devex-auth-finding'); },
|
||
(call: any) => { call.questions[0].header = 'Auth policy'; },
|
||
(call: any) => { call.questions[0].options.push({ label: 'Add retry support to the SDK now' }); },
|
||
(call: any) => { call.questions[0].options[1].label = 'Ready to implement — change authentication first'; },
|
||
(call: any) => { call.questions[0].options[2].label = 'Skip — keep the broken authentication'; },
|
||
(call: any) => { call.answers[call.questions[0].question] = 'First implement the missing retry fix'; },
|
||
(call: any) => { call.unansweredQuestionIndices = [0]; },
|
||
(call: any) => { call.questions.push(structuredClone(CAPTURED_CALL.questions[0])); },
|
||
]) {
|
||
const transcript = completedHandoffTranscript(f);
|
||
mutate(transcript.calls[1]);
|
||
expect(hasNativePlanTerminal(transcript, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
}
|
||
} finally {
|
||
f.cleanup();
|
||
}
|
||
});
|
||
|
||
test('a later actual review answer still requires a fresh complete report', () => {
|
||
const f = fixture();
|
||
try {
|
||
const transcript = completedHandoffTranscript(f);
|
||
transcript.calls.splice(1, 0, {
|
||
...structuredClone(CAPTURED_CALL),
|
||
toolUseId: 'later-review-decision',
|
||
answeredAt: '2026-09-08T20:49:00.000Z',
|
||
});
|
||
expect(hasNativePlanTerminal(transcript, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
fs.utimesSync(f.file, Date.parse('2026-09-08T20:49:20Z') / 1000, Date.parse('2026-09-08T20:49:20Z') / 1000);
|
||
expect(hasNativePlanTerminal(transcript, f.file, f.startedAt, 'plan_ready')).toBe(true);
|
||
fs.writeFileSync(f.file, '## GSTACK REVIEW REPORT\n');
|
||
expect(hasNativePlanTerminal(transcript, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
} finally {
|
||
f.cleanup();
|
||
}
|
||
});
|
||
});
|
||
|
||
|
||
// Source-I Eng displayed this compact native approval after ExitPlanMode.
|
||
// Its classification must not itself claim a completed review/report.
|
||
const CAPTURED_COMPACT_PLAN_GATE = `Exit plan mode?
|
||
|
||
Claude wants to exit plan mode
|
||
|
||
❯ 1. Yes, and switch to default (ask each time) for this session
|
||
2. No`;
|
||
|
||
test('the captured compact ExitPlanMode menu is an approval gate, not an ordinary answer', () => {
|
||
expect(isPlanReadyVisible(CAPTURED_COMPACT_PLAN_GATE)).toBe(true);
|
||
expect(classifyPlanCountFrame(CAPTURED_COMPACT_PLAN_GATE)).toBe('plan_ready');
|
||
const f = fixture();
|
||
try {
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
f.transcript.planReadyRequests = [{ sessionId: f.transcript.calls[0]!.sessionId,
|
||
toolUseId: 'exit', timestamp: CAPTURED_FINAL.timestamp, failed: false }];
|
||
fs.writeFileSync(f.file, '# Draft\n');
|
||
expect(hasNativePlanTerminal(f.transcript, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
} finally { f.cleanup(); }
|
||
});
|
||
|
||
test('compact approval recognition rejects quoted, incomplete and superseded menus', () => {
|
||
for (const visible of [
|
||
'Should the application show an Exit plan mode? confirmation?',
|
||
'> ' + CAPTURED_COMPACT_PLAN_GATE.replaceAll('\n', '\n> '),
|
||
CAPTURED_COMPACT_PLAN_GATE.replace('Claude wants to exit plan mode', 'The documentation mentions exiting plan mode'),
|
||
CAPTURED_COMPACT_PLAN_GATE.replace(' 2. No', ''),
|
||
CAPTURED_COMPACT_PLAN_GATE + '\nTests\nAdd retry coverage?\n❯ 1. Add test\n 2. Skip\nEnter to select · Esc to cancel',
|
||
]) expect(isPlanReadyVisible(visible), visible).toBe(false);
|
||
});
|
||
|
||
describe('untagged completed DX handoff', () => {
|
||
function nativeHandoff(f: ReturnType<typeof fixture>) {
|
||
const transcript = structuredClone(f.transcript);
|
||
const handoff = { ...structuredClone(capturedL.calls[1]!), sessionId: transcript.calls[0]!.sessionId };
|
||
transcript.calls.push(handoff);
|
||
transcript.planReadyRequests = [{sessionId: handoff.sessionId, toolUseId: 'L-native-exit',
|
||
timestamp: new Date(Date.parse(handoff.answeredAt) + 1000).toISOString(), failed: false}];
|
||
return transcript;
|
||
}
|
||
test('actual completed navigation does not make the existing report stale', () => {
|
||
const f = fixture();
|
||
try {
|
||
expect(hasNativePlanTerminal(nativeHandoff(f), f.file, f.startedAt, 'plan_ready')).toBe(true);
|
||
} finally { f.cleanup(); }
|
||
});
|
||
test('missing identity needs positive completion and resolved findings, with native approval and a fresh report', () => {
|
||
const f = fixture();
|
||
try {
|
||
for (const mutate of [
|
||
(t: PlanCountTranscript) => { const q=t.calls[1]!.questions[0]!; q.question += ' <gstack-qid:devex-auth-finding>'; },
|
||
(t: PlanCountTranscript) => { t.calls[1]!.questions[0]!.question += ' <gstack-qid:devex-next-steps> <gstack-qid:broken'; },
|
||
(t: PlanCountTranscript) => { const q=t.calls[1]!.questions[0]!; q.question=q.question.replace('is complete.', 'is not complete.'); },
|
||
(t: PlanCountTranscript) => { const q=t.calls[1]!.questions[0]!; q.question=q.question.replace('Five issues found and resolved', 'Five issues need decisions'); },
|
||
(t: PlanCountTranscript) => { const q=t.calls[1]!.questions[0]!; q.question=q.question.replace('Five issues found and resolved', 'No issues resolved'); },
|
||
(t: PlanCountTranscript) => { const q=t.calls[1]!.questions[0]!; q.question=q.question.replace('Five issues found and resolved', 'Not five issues found and resolved'); },
|
||
(t: PlanCountTranscript) => { const q=t.calls[1]!.questions[0]!; q.question=q.question.replace('What next?', 'One gap remains. What next?'); },
|
||
(t: PlanCountTranscript) => { t.calls[1]!.questions[0]!.options.push({label:'Add a new authentication policy'}); },
|
||
(t: PlanCountTranscript) => { t.calls[1]!.answered=false; },
|
||
(t: PlanCountTranscript) => { t.calls[1]!.failed=true; },
|
||
(t: PlanCountTranscript) => { t.calls[1]!.answers={}; },
|
||
(t: PlanCountTranscript) => { t.planReadyRequests=[]; },
|
||
(t: PlanCountTranscript) => { t.planReadyRequests![0]!.failed=true; },
|
||
]) {
|
||
const t=nativeHandoff(f);
|
||
const originalQuestion=t.calls[1]!.questions[0]!.question;
|
||
const originalAnswer=t.calls[1]!.answers?.[originalQuestion];
|
||
mutate(t);
|
||
const nextQuestion=t.calls[1]!.questions[0]!.question;
|
||
if (nextQuestion !== originalQuestion && originalAnswer && t.calls[1]!.answers?.[originalQuestion]) {
|
||
delete t.calls[1]!.answers![originalQuestion];
|
||
t.calls[1]!.answers![nextQuestion]=originalAnswer;
|
||
}
|
||
expect(hasNativePlanTerminal(t, f.file, f.startedAt, 'plan_ready')).toBe(false);
|
||
}
|
||
const t=nativeHandoff(f);
|
||
t.calls.splice(1,0,{...structuredClone(t.calls[0]!),toolUseId:'new-late-finding',
|
||
answeredAt:new Date(Date.parse(t.calls[1]!.answeredAt!) - 1000).toISOString()});
|
||
expect(hasNativePlanTerminal(t,f.file,f.startedAt,'plan_ready')).toBe(false);
|
||
fs.writeFileSync(f.file,'## GSTACK REVIEW REPORT\n');
|
||
expect(hasNativePlanTerminal(nativeHandoff(f),f.file,f.startedAt,'plan_ready')).toBe(false);
|
||
} finally {f.cleanup();}
|
||
});
|
||
});
|
||
|
||
describe('ceo-completion-handoff-m', () => {
|
||
const captured = captured_ceo_completion_handoff_m;
|
||
const nextStepCapture = nextStepCapture_ceo_completion_handoff_m;
|
||
const calls = () => structuredClone(captured.calls) as NativePlanQuestionCall[];
|
||
const handoff = () => calls().at(-1)!;
|
||
const fingerprint = (call: NativePlanQuestionCall) => nativePlanCallFingerprint(call, 0, false);
|
||
describe('CEO completion described by a native navigation choice', () => {
|
||
});
|
||
|
||
describe('native next-review navigation with a resolved CEO recap', () => {
|
||
const retryCalls = () => structuredClone(captured.distinctRetry.calls) as NativePlanQuestionCall[];
|
||
const retryHandoff = () => retryCalls().at(-1)!;
|
||
});
|
||
|
||
describe('CEO completed next-step identity in native option order', () => {
|
||
const input = () => structuredClone(nextStepCapture.calls) as NativePlanQuestionCall[];
|
||
const actual = () => input().at(-1)!;
|
||
test('the actual report precedes handoff but the captured absent Exit remains incomplete', () => {
|
||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ceo-native-next-step-'));
|
||
const report = path.join(dir, 'plan.md');
|
||
try {
|
||
fs.writeFileSync(report, nextStepCapture.report.content);
|
||
const written = Date.parse(nextStepCapture.report.successfulUpdateAt) / 1000;
|
||
fs.utimesSync(report, written, written);
|
||
const calls = input();
|
||
expect(Date.parse(calls.at(-2)!.answeredAt!)).toBeLessThan(written * 1000);
|
||
expect(Date.parse(calls.at(-1)!.answeredAt!)).toBeGreaterThan(written * 1000);
|
||
const transcript = { status: 'ready' as const, calls, assistantMessages: [],
|
||
planReadyRequests: structuredClone(nextStepCapture.planReadyRequests) };
|
||
const admin = new Set([fingerprint(calls.at(-1)!).signature]);
|
||
expect(hasNativePlanTerminal(transcript, report, Date.parse('2026-09-09T01:06:22Z'), 'plan_ready', admin)).toBe(false);
|
||
} finally {
|
||
fs.rmSync(dir, { recursive: true, force: true });
|
||
}
|
||
});
|
||
});
|
||
});
|
||
|
||
describe('ceo-completion-handoff-o', () => {
|
||
const captured = captured_ceo_completion_handoff_o;
|
||
const capturedQ = capturedQ_ceo_completion_handoff_o;
|
||
const calls = () => structuredClone(captured.calls) as NativePlanQuestionCall[];
|
||
const handoff = () => calls().at(-1)!;
|
||
describe('closed CEO navigation with the native review-prefixed identity', () => {
|
||
});
|
||
|
||
describe('CEO completion recap after native project metadata', () => {
|
||
const qCalls = () => structuredClone(capturedQ.calls) as NativePlanQuestionCall[];
|
||
const qHandoff = () => qCalls().at(-1)!;
|
||
test('actual full report and Exit chronology retain last substantive-answer freshness', () => {
|
||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ceo-metadata-navigation-'));
|
||
const report = path.join(dir, 'plan.md');
|
||
try {
|
||
fs.writeFileSync(report, capturedQ.reportContent);
|
||
const reportAt = Date.parse(capturedQ.reportAt) / 1000;
|
||
fs.utimesSync(report, reportAt, reportAt);
|
||
const transcript = { status: 'ready' as const, calls: qCalls(), assistantMessages: [], planReadyRequests: structuredClone(capturedQ.planReadyRequests) };
|
||
const administrative = new Set([`${qHandoff().sessionId}:${qHandoff().toolUseId}`]);
|
||
const start = Date.parse('2026-09-09T03:25:54Z');
|
||
expect(hasNativePlanTerminal(transcript, report, start, 'plan_ready')).toBe(false);
|
||
expect(hasNativePlanTerminal(transcript, report, start, 'plan_ready', administrative)).toBe(true);
|
||
transcript.planReadyRequests[0]!.failed = true;
|
||
expect(hasNativePlanTerminal(transcript, report, start, 'plan_ready', administrative)).toBe(false);
|
||
transcript.planReadyRequests = structuredClone(capturedQ.planReadyRequests);
|
||
fs.utimesSync(report, start / 1000, start / 1000);
|
||
expect(hasNativePlanTerminal(transcript, report, start, 'plan_ready', administrative)).toBe(false);
|
||
fs.writeFileSync(report, 'Incomplete plan');
|
||
fs.utimesSync(report, reportAt, reportAt);
|
||
expect(hasNativePlanTerminal(transcript, report, start, 'plan_ready', administrative)).toBe(false);
|
||
} finally { fs.rmSync(dir, { recursive: true, force: true }); }
|
||
});
|
||
});
|
||
});
|
||
|
||
describe('ceo-handoff-y', () => {
|
||
const fs = fs_ceo_handoff_y;
|
||
const os = os_ceo_handoff_y;
|
||
const path = path_ceo_handoff_y;
|
||
const fixture = fixture_ceo_handoff_y;
|
||
const fp=(c:NativePlanQuestionCall)=>nativePlanCallFingerprint(c,0,false);
|
||
describe('Y bare next-Eng navigation is administrative, not completion evidence',()=>{
|
||
test('independent fresh report and native Exit still gate completion; menu alone cannot pass',()=>{
|
||
const dir=fs.mkdtempSync(path.join(os.tmpdir(),'gstack-handoff-y-free-'));const report=path.join(dir,'report.md');
|
||
try{fs.writeFileSync(report,fixture.report);const calls=structuredClone(fixture.calls) as NativePlanQuestionCall[];const transcript={status:'ready' as const,calls,assistantMessages:[],planReadyRequests:structuredClone(fixture.planReadyRequests)};const handoff=calls.at(-1)!;const admin=new Set([fp(handoff).signature]);const issueAt=Date.parse(calls.at(-2)!.answeredAt!),handoffAt=Date.parse(handoff.answeredAt!);const started=Date.parse(calls[0]!.answeredAt!)-1000;
|
||
// Controlled metadata only: original Y report mtime was not captured.
|
||
const between=(issueAt+handoffAt)/2;fs.utimesSync(report,between/1000,between/1000);
|
||
expect(hasNativePlanTerminal(transcript,report,started,'plan_ready')).toBe(false);expect(hasNativePlanTerminal(transcript,report,started,'plan_ready',admin)).toBe(true);
|
||
fs.utimesSync(report,(issueAt-1)/1000,(issueAt-1)/1000);expect(hasNativePlanTerminal(transcript,report,started,'plan_ready',admin)).toBe(false);
|
||
fs.utimesSync(report,between/1000,between/1000);expect(hasNativePlanTerminal({...transcript,planReadyRequests:[]},report,started,'plan_ready',admin)).toBe(false);
|
||
expect(hasNativePlanTerminal({...transcript,calls:[handoff]},report,started,'plan_ready',admin)).toBe(false);
|
||
}finally{fs.rmSync(dir,{recursive:true,force:true});}
|
||
});
|
||
});
|
||
});
|
||
|
||
describe('dx-manual-handoff-ao', () => {
|
||
const fs = fs_ceo_handoff_y;
|
||
const os = os_ceo_handoff_y;
|
||
const path = path_ceo_handoff_y;
|
||
const captured = captured_dx_manual_handoff_ao;
|
||
type Edit=(calls:NativePlanQuestionCall[], transcript:PlanCountTranscript, report:string)=>void;
|
||
function replay(edit?:Edit){
|
||
const dir=fs.mkdtempSync(path.join(os.tmpdir(),'dx-manual-handoff-ao-'));
|
||
try {
|
||
const report=path.join(dir,'report.md');fs.writeFileSync(report,captured.reportContent);
|
||
const written=captured.provenance.reportMtimeMs/1000;fs.utimesSync(report,written,written);
|
||
const transcript={status:'ready',calls:structuredClone(captured.calls),assistantMessages:[],planReadyRequests:structuredClone(captured.planReadyRequests)} as PlanCountTranscript;
|
||
edit?.(transcript.calls,transcript,report);
|
||
return hasNativePlanTerminal(transcript,report,captured.provenance.startedAt,'plan_ready');
|
||
} finally {fs.rmSync(dir,{recursive:true,force:true});}
|
||
}
|
||
function change(call:NativePlanQuestionCall,from:string,to:string){
|
||
const q=call.questions[0]!;expect(q.question).toContain(from);
|
||
const selected=call.answers![q.question];q.question=q.question.replace(from,to);call.answers={[q.question]:selected!};
|
||
}
|
||
describe('AO completed manual DX handoff preserves report freshness',()=>{
|
||
test('exact owned report precedes navigation only, with the current Exit gate recognized',()=>{
|
||
expect(captured.calls).toHaveLength(2);expect(captured.events).toHaveLength(4);
|
||
expect(Date.parse(captured.calls[0]!.answeredAt!)).toBeLessThan(captured.provenance.reportMtimeMs);
|
||
expect(Date.parse(captured.calls[1]!.answeredAt!)).toBeGreaterThan(captured.provenance.reportMtimeMs);
|
||
expect(classifyPlanCountFrame(captured.screen)).toBe('plan_ready');
|
||
expect(replay()).toBe(true);
|
||
});
|
||
test('equivalent completed recap and manual roles retain current authority',()=>{
|
||
for(const [from,to] of [
|
||
[' (5/10 -> 8.5/10)',''],
|
||
['5/10 -> 8.5/10','6/10 → 9/10'],
|
||
['What should happen next?',"What's next?"],
|
||
['The DX review found','The DX review identified'],
|
||
['All are written into the plan as tasks T1 to T9.','All DX decisions and tasks are recorded in the plan.'],
|
||
])expect(replay(calls=>change(calls[1]!,from!,to!)),to).toBe(true);
|
||
expect(replay(calls=>calls[1]!.questions[0]!.options.reverse())).toBe(true);
|
||
expect(replay(calls=>{const c=calls[1]!;change(c,'Net: hand off now as you asked, or chain the eng review here.','Net: hand off now as you asked, or chain the eng review here.\n> Historical example: add a new task before leaving.');})).toBe(true);
|
||
expect(replay(calls=>{const o=calls[1]!.questions[0]!.options[0]!;o.description=o.description!.replace('Plan exits now with all DX decisions and tasks recorded; nothing else is started.','Exit the plan now with all DX tasks and decisions recorded. No further review is started.');})).toBe(true);
|
||
});
|
||
test('source, conditional or withdrawn completion facts cannot make a stale report current',()=>{
|
||
const edits:Array<[string,string]>=[
|
||
['D13 — DX review','Source: D13 — DX review'],
|
||
['DX review complete','DX review is not complete'],
|
||
['DX review complete','DX review complete only after another decision'],
|
||
['ELI10: The DX review found','ELI10: Earlier review assessment: The DX review found'],
|
||
['ELI10: The DX review found','ELI10: If approved, the DX review found'],
|
||
['All are written into the plan as tasks T1 to T9.','Example: All are written into the plan as tasks T1 to T9.'],
|
||
['All are written into the plan as tasks T1 to T9.','Previously, all are written into the plan as tasks T1 to T9.'],
|
||
['All are written into the plan as tasks T1 to T9.','"All are written into the plan as tasks T1 to T9."'],
|
||
['All are written into the plan as tasks T1 to T9.','All will be written into the plan as tasks T1 to T9.'],
|
||
['Project/branch/task:','Source:\nProject/branch/task:'],
|
||
['Project/branch/task: ','Project/branch/task: If approved, '],
|
||
['Project/branch/task: ','Project/branch/task: Source excerpt, not a current assessment: '],
|
||
];
|
||
for(const [from,to] of edits)expect(replay(calls=>change(calls[1]!,from,to)),to).toBe(false);
|
||
for(const suffix of [' This review is not complete.',' These tasks are not recorded.',' This review is "withdrawn".',' One DX decision remains unresolved.',' We must fix another issue.',' Add another migration task.',' Should we approve another change?',' <gstack-qid:foreign>']) {
|
||
expect(replay(calls=>{const c=calls[1]!;change(c,c.questions[0]!.question,c.questions[0]!.question+suffix);}),suffix).toBe(false);
|
||
}
|
||
});
|
||
test('selected manual action must close with recorded decisions and no new work',()=>{
|
||
for(const prefix of ['Source: ','Earlier review assessment: ','If approved, ','> '])expect(replay(calls=>{const o=calls[1]!.questions[0]!.options[0]!;o.description=prefix+o.description;}),prefix).toBe(false);
|
||
for(const suffix of [' Also update the plan before exit.',' Run /plan-eng-review now.',' This plan is not complete.',' The tasks are "withdrawn".',' This manual handoff is cancelled.'])expect(replay(calls=>{calls[1]!.questions[0]!.options[0]!.description+=suffix;}),suffix).toBe(false);
|
||
for(const from of ['all DX decisions and tasks recorded','nothing else is started'])expect(replay(calls=>{const o=calls[1]!.questions[0]!.options[0]!;o.description=o.description!.replace(from,'more work remains');}),from).toBe(false);
|
||
for(const index of [1,2])expect(replay(calls=>{const c=calls[1]!,q=c.questions[0]!;c.answers={[q.question]:q.options[index]!.label};})).toBe(false);
|
||
});
|
||
test('completed owned native answer identity remains mandatory',()=>{
|
||
const mutations:Array<(c:NativePlanQuestionCall)=>void>=[
|
||
c=>{c.answered=false;},c=>{c.failed=true;},c=>{c.sessionId='foreign';},c=>{c.toolUseId='';},
|
||
c=>{c.answeredAt='invalid';},c=>{c.answeredAt=new Date(Date.now()+60_000).toISOString();},
|
||
c=>{c.unansweredQuestionIndices=[0];},c=>{delete c.unansweredQuestionIndices;},
|
||
c=>{c.questions[0]!.multiSelect=true;},c=>{c.questions[0]!.header='Issue decision';},
|
||
c=>{c.questions.push(structuredClone(c.questions[0]!));},
|
||
c=>{c.answers={wrong:c.questions[0]!.options[0]!.label};},
|
||
c=>{c.answers![c.questions[0]!.question]='Not an offered answer';},
|
||
c=>{c.answers!.extra='foreign';},
|
||
c=>{c.questions[0]!.options[1]!.label=c.questions[0]!.options[0]!.label;},
|
||
];
|
||
for(const edit of mutations)expect(replay(calls=>edit(calls[1]!)),edit.toString()).toBe(false);
|
||
});
|
||
test('other modifying answers and complete report/current Exit gates remain unchanged',()=>{
|
||
expect(replay(calls=>{calls[0]!.answeredAt=new Date(captured.provenance.reportMtimeMs+1).toISOString();})).toBe(false);
|
||
expect(replay((_calls,_t,report)=>{const time=Date.parse(captured.calls[0]!.answeredAt!)/1000-1;fs.utimesSync(report,time,time);})).toBe(false);
|
||
expect(replay((_calls,_t,report)=>fs.writeFileSync(report,'# Completion summary\nDone.'))).toBe(false);
|
||
expect(replay((_calls,_t,report)=>fs.unlinkSync(report))).toBe(false);
|
||
for(const mutate of [
|
||
(t:PlanCountTranscript)=>{t.planReadyRequests=[];},
|
||
(t:PlanCountTranscript)=>{t.planReadyRequests![0]!.failed=true;},
|
||
(t:PlanCountTranscript)=>{t.planReadyRequests![0]!.sessionId='foreign';},
|
||
(t:PlanCountTranscript)=>{t.planReadyRequests![0]!.timestamp=captured.calls[1]!.answeredAt!;},
|
||
(t:PlanCountTranscript)=>{t.status='missing';},
|
||
])expect(replay((_calls,t)=>mutate(t))).toBe(false);
|
||
});
|
||
});
|
||
});
|
||
|
||
describe('plan-count-dx-handoff-o', () => {
|
||
const fixture = fixture_plan_count_dx_handoff_o;
|
||
function replay(mutate?: (call: NativePlanQuestionCall, transcript: PlanCountTranscript) => void): boolean {
|
||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'dx-native-handoff-'));
|
||
const report = path.join(dir, 'review.md');
|
||
const transcript = structuredClone(fixture.transcript) as PlanCountTranscript;
|
||
const call = transcript.calls.at(-1)!;
|
||
mutate?.(call, transcript);
|
||
fs.writeFileSync(report, fixture.report);
|
||
const writtenAt = Date.parse(fixture.reportWrite.resultAt);
|
||
fs.utimesSync(report, writtenAt / 1000, writtenAt / 1000);
|
||
try {
|
||
return hasNativePlanTerminal(transcript, report,
|
||
Date.parse(fixture.startedAt), 'plan_ready');
|
||
} finally {
|
||
fs.rmSync(dir, { recursive: true, force: true });
|
||
}
|
||
}
|
||
|
||
test('actual completed DX navigation preserves the prior full report and all thirteen native calls', () => {
|
||
expect(fixture.transcript.calls).toHaveLength(13);
|
||
expect(fixture.transcript.calls.every(call => call.answered && !call.failed)).toBe(true);
|
||
expect(fixture.reportWrite.failed).toBe(false);
|
||
expect(replay()).toBe(true);
|
||
});
|
||
|
||
test('closed navigation cannot hide a real new issue, unfinished review, or conditional closure', () => {
|
||
const variants = [
|
||
(q: any) => { q.header = 'New finding'; },
|
||
(q: any) => { q.question = q.question.replace('D11 — What next?', 'D11 — Should we fix the missing authorization check?'); },
|
||
(q: any) => { q.question = q.question.replace('The DX review is done.', 'The DX review is not done.'); },
|
||
(q: any) => { q.question = q.question.replace('The DX review is done.', 'The DX review is done only after tests pass.'); },
|
||
(q: any) => { q.question += '\nOne documentation gap remains unresolved.'; },
|
||
(q: any) => { q.question += '\nPlease fix the missing authorization check first.'; },
|
||
(q: any) => { q.question += '\nShould we add the missing test?'; },
|
||
(q: any) => { q.question += '\nOnce the tests pass, the DX review will be complete.'; },
|
||
(q: any) => { q.question += '\nAll decisions resolved after tests pass.'; },
|
||
(q: any) => { q.question += '\nDX review complete after tests pass.'; },
|
||
(q: any) => { q.question += '\nWe could fix the missing authorization check before Eng.'; },
|
||
(q: any) => { q.question = q.question.replace('ELI10: The DX review is done.', '```text\nThe DX review is done.\n```'); },
|
||
(q: any) => { q.question = q.question.replace('ELI10: The DX review is done.', '```text\nThe DX review is done.'); },
|
||
(q: any) => { q.question = q.question.replace('ELI10: The DX review is done.', '> The DX review is done.'); },
|
||
(q: any) => { q.options[2].description = 'Proceed to fix the missing contrast test before Eng.'; },
|
||
(q: any) => { q.options[2].description = 'One issue remains unresolved; handle it manually.'; },
|
||
(q: any) => { q.options[2].label = 'C) Add the missing test'; },
|
||
(q: any) => { q.options.push({ label: 'D) Add a migration guide', description: 'A new required deliverable.' }); },
|
||
(q: any) => { q.question = q.question.replace('plan-devex-review-next-steps', 'plan-devex-review-new-issue'); },
|
||
(q: any) => { q.question += '\n<gstack-qid:broken'; },
|
||
(q: any) => { q.multiSelect = true; },
|
||
];
|
||
for (const mutate of variants) {
|
||
expect(replay(call => {
|
||
const question = call.questions[0]!;
|
||
mutate(question);
|
||
// Keep a real offered answer after text mutations so the semantic guard,
|
||
// rather than a stale answer key, is what must reject the altered call.
|
||
call.answers = { [question.question]: question.options[0]!.label };
|
||
}), String(mutate)).toBe(false);
|
||
}
|
||
});
|
||
|
||
test('native failure, offered answer, report freshness and real Exit remain required', () => {
|
||
expect(replay(call => { call.answered = false; })).toBe(false);
|
||
expect(replay(call => { call.failed = true; })).toBe(false);
|
||
expect(replay(call => { call.unansweredQuestionIndices = [0]; })).toBe(false);
|
||
expect(replay(call => { call.answers = { [call.questions[0]!.question]: 'Add a new test first' }; })).toBe(false);
|
||
expect(replay((_call, transcript) => { transcript.planReadyRequests = []; })).toBe(false);
|
||
expect(replay((_call, transcript) => { transcript.planReadyRequests![0]!.failed = true; })).toBe(false);
|
||
expect(replay((_call, transcript) => { transcript.planReadyRequests![0]!.sessionId = 'foreign'; })).toBe(false);
|
||
expect(replay((_call, transcript) => {
|
||
transcript.calls[11]!.answeredAt = new Date(Date.parse(fixture.reportWrite.resultAt) + 1000).toISOString();
|
||
})).toBe(false);
|
||
});
|
||
|
||
test('pure navigation preserves actual option order and ordinary next-review sequencing', () => {
|
||
expect(replay(call => { call.questions[0]!.options.reverse(); })).toBe(true);
|
||
expect(replay(call => {
|
||
const q = call.questions[0]!;
|
||
q.options[0]!.description += ' After Eng review is complete, proceed to implementation.';
|
||
})).toBe(true);
|
||
});
|
||
});
|
||
|
||
describe('eng-next-handoff-ah', () => {
|
||
const fs = fs_ceo_handoff_y;
|
||
const os = os_ceo_handoff_y;
|
||
const path = path_ceo_handoff_y;
|
||
const actual = actual_eng_next_handoff_ah;
|
||
test('exact final exit/report replay retains all freshness, identity and answer gates', () => {
|
||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-eng-next-ah-'));
|
||
const file = path.join(dir, 'reviewed.md');
|
||
const now = Date.now;
|
||
try {
|
||
fs.writeFileSync(file, actual.plan);
|
||
fs.utimesSync(file, actual.source.stat.mtimeMs / 1000, actual.source.stat.mtimeMs / 1000);
|
||
Date.now = () => Date.parse(actual.captureAt);
|
||
const t = structuredClone(actual.transcript) as PlanCountTranscript;
|
||
const id = actual.fingerprint.signature;
|
||
const admin = new Set([id]);
|
||
const check = (v = t, a = admin) => hasNativePlanTerminal(v, file, actual.startedAt, 'plan_ready', a);
|
||
expect(isCurrentPlanApprovalScreen(actual.screen)).toBe(true);
|
||
expect(check()).toBe(true);
|
||
expect(check(t, new Set())).toBe(false);
|
||
expect(check(t, new Set(['foreign:call']))).toBe(false);
|
||
for (const mutate of [
|
||
(v: PlanCountTranscript) => { v.planReadyRequests = []; },
|
||
(v: PlanCountTranscript) => { v.planReadyRequests!.at(-1)!.failed = true; },
|
||
(v: PlanCountTranscript) => { v.planReadyRequests!.at(-1)!.sessionId = 'foreign'; },
|
||
(v: PlanCountTranscript) => { v.planReadyRequests!.at(-1)!.timestamp = '2026-09-10T03:29:40.000Z'; },
|
||
(v: PlanCountTranscript) => { v.planReadyRequests!.at(-1)!.timestamp = new Date(Date.now() + 1).toISOString(); },
|
||
(v: PlanCountTranscript) => { v.calls.at(-1)!.answered = false; },
|
||
(v: PlanCountTranscript) => { v.calls.at(-2)!.answeredAt = '2026-09-10T03:29:00.000Z'; },
|
||
]) { const v = structuredClone(t); mutate(v); expect(check(v)).toBe(false); }
|
||
fs.writeFileSync(file, actual.plan.replace('NO UNRESOLVED DECISIONS', 'Report still pending'));
|
||
fs.utimesSync(file, actual.source.stat.mtimeMs / 1000, actual.source.stat.mtimeMs / 1000);
|
||
expect(check()).toBe(false);
|
||
} finally { Date.now = now; fs.rmSync(dir, { recursive: true, force: true }); }
|
||
});
|
||
|
||
const b176 = actual.sourceBoundB176;
|
||
const recorded = () => structuredClone(b176.transcript) as PlanCountTranscript;
|
||
test('actual pending ExitPlanMode needs the classified recap plus the unchanged fresh report and native gates',()=>{
|
||
const dir=fs.mkdtempSync(path.join(os.tmpdir(),'eng-b176-terminal-')),file=path.join(dir,'report.md'),now=Date.now;
|
||
try{
|
||
Date.now=()=>Date.parse(b176.capturedAt);fs.writeFileSync(file,b176.plan);fs.utimesSync(file,b176.sourceReport.mtimeMs/1000,b176.sourceReport.mtimeMs/1000);
|
||
const t=recorded(),signature=b176.fingerprint.signature;
|
||
const admin=new Set([signature]);
|
||
const check=(transcript=t,administrative=admin)=>hasNativePlanTerminal(transcript,file,b176.startedAt,'plan_ready',administrative);
|
||
expect(isCurrentPlanApprovalScreen(b176.screen)).toBe(true);
|
||
expect(check()).toBe(true);expect(check(t,new Set())).toBe(false);expect(check(t,new Set(['foreign:call']))).toBe(false);
|
||
for(const mutate of [
|
||
(v:PlanCountTranscript)=>{v.planReadyRequests=[];},
|
||
(v:PlanCountTranscript)=>{v.planReadyRequests![0]!.failed=true;},
|
||
(v:PlanCountTranscript)=>{v.planReadyRequests![0]!.sessionId='foreign';},
|
||
(v:PlanCountTranscript)=>{v.planReadyRequests![0]!.timestamp=new Date(Date.now()+1).toISOString();},
|
||
(v:PlanCountTranscript)=>{v.planReadyRequests![0]!.timestamp=v.calls.at(-1)!.answeredAt!;},
|
||
(v:PlanCountTranscript)=>{v.calls.at(-1)!.answered=false;},
|
||
(v:PlanCountTranscript)=>{v.calls.at(-2)!.answeredAt=new Date(b176.sourceReport.mtimeMs+1).toISOString();},
|
||
]){const v=recorded();mutate(v);expect(check(v)).toBe(false);}
|
||
fs.utimesSync(file,(b176.startedAt-1)/1000,(b176.startedAt-1)/1000);expect(check()).toBe(false);
|
||
fs.writeFileSync(file,b176.plan.replace('NO UNRESOLVED DECISIONS','PENDING'));fs.utimesSync(file,b176.sourceReport.mtimeMs/1000,b176.sourceReport.mtimeMs/1000);expect(check()).toBe(false);
|
||
}finally{Date.now=now;fs.rmSync(dir,{recursive:true,force:true});}
|
||
});
|
||
});
|
||
|
||
describe('eng-task-pause-navigation-f359', () => {
|
||
const fs = fs_ceo_handoff_y;
|
||
const os = os_ceo_handoff_y;
|
||
const path = path_ceo_handoff_y;
|
||
const capture = capture_eng_task_pause_navigation_f359;
|
||
const actual=()=>({call:structuredClone(capture.transcript.calls.at(-1)!) as NativePlanQuestionCall,prior:structuredClone(capture.transcript.calls.slice(0,-1)) as NativePlanQuestionCall[],plan:capture.plan});
|
||
test('handoff alone never supplies a native terminal or refreshes modifying answers',()=>{
|
||
const x=actual(),fp=nativePlanCallFingerprint(x.call,0,false),admin=new Set([fp.signature]);
|
||
const dir=fs.mkdtempSync(path.join(os.tmpdir(),'gstack-eng-task-pause-')),file=path.join(dir,'reviewed.md'),now=Date.now;
|
||
try {
|
||
fs.writeFileSync(file,x.plan);fs.utimesSync(file,capture.reportSource.mtimeMs/1000,capture.reportSource.mtimeMs/1000);
|
||
Date.now=()=>Date.parse('2026-09-16T07:04:00.000Z');
|
||
const t=structuredClone(capture.transcript) as PlanCountTranscript;
|
||
const check=(v=t,a=admin)=>hasNativePlanTerminal(v,file,Date.parse('2026-09-16T06:40:00.000Z'),'plan_ready',a);
|
||
expect(check()).toBe(false); // Actual capture precedes the native exit.
|
||
// The later retained native exit is real; this is a gate replay, not a
|
||
// replacement verdict for the original paid timeout/failure.
|
||
expect(createHash('sha256').update(JSON.stringify(t.calls)).digest('hex')).toBe(capture.terminalCapture.callsSha256);
|
||
t.planReadyRequests=structuredClone(capture.terminalCapture.planReadyRequests);
|
||
t.assistantMessages=structuredClone(capture.terminalCapture.assistantMessages);
|
||
expect(check()).toBe(true);expect(check(t,new Set())).toBe(false);expect(check(t,new Set(['foreign:call']))).toBe(false);
|
||
for(const mutate of [
|
||
(v:PlanCountTranscript)=>{v.planReadyRequests![0]!.failed=true;},
|
||
(v:PlanCountTranscript)=>{v.planReadyRequests![0]!.sessionId='foreign';},
|
||
(v:PlanCountTranscript)=>{v.planReadyRequests![0]!.timestamp=x.call.answeredAt!;},
|
||
(v:PlanCountTranscript)=>{v.planReadyRequests=[];},
|
||
(v:PlanCountTranscript)=>{v.calls[5]!.answeredAt=x.call.answeredAt;},
|
||
(v:PlanCountTranscript)=>{v.calls[5]!.answered=false;v.calls[5]!.unansweredQuestionIndices=[0];},
|
||
]){const v=structuredClone(t);mutate(v);expect(check(v)).toBe(false);}
|
||
}finally{Date.now=now;fs.rmSync(dir,{recursive:true,force:true});}
|
||
});
|
||
});
|
||
|
||
describe('design-count-native-8525', () => {
|
||
const fixture = fixture_design_count_native_8525;
|
||
function completion() {
|
||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'design-8525-replay-'));
|
||
const file = path.join(dir, path.basename(fixture.provenance.planPath));
|
||
const transcript = structuredClone(fixture.transcript) as PlanCountTranscript;
|
||
const edit = fixture.provenance.operations.filter(o => o.tool === 'Edit').at(-1)!;
|
||
const mtime = Date.parse(edit.acknowledgedAt) / 1000;
|
||
const write = (content = fixture.report) => { fs.writeFileSync(file, content); fs.utimesSync(file, mtime, mtime); };
|
||
write();
|
||
// The replay starts before the first retained native assistant message.
|
||
const startedAt = Math.min(...transcript.assistantMessages.map(m => Date.parse(m.timestamp))) - 1_000;
|
||
const final = transcript.assistantMessages.at(-1)!;
|
||
const check = () => hasNativePlanTerminal(transcript, file, startedAt, 'completion_summary');
|
||
return { dir, file, transcript, final, write, check, cleanup: () => fs.rmSync(dir, {recursive:true, force:true}) };
|
||
}
|
||
|
||
test('exact native final text and reconstructed read-back-verified report supply completion', () => {
|
||
const f = completion(); try { expect(f.check()).toBe(true); } finally { f.cleanup(); }
|
||
});
|
||
test('current typed status accepts presentation, field order and current report prose independently', () => {
|
||
const f=completion();try {
|
||
for (const heading of ['## Completion','### Completion summary','## Review complete','## Design review complete','**Review completion:**']) {
|
||
for (const status of ['STATUS: DONE','**STATUS:** DONE — review saved and verified.','**STATUS: DONE**']) {
|
||
for (const fields of [
|
||
[status,`What changed: \`${path.basename(f.file)}\` now carries the current review report.`],
|
||
[`Report: ${f.file} contains the reviewed plan and verification.`,status],
|
||
[status,`- Plan saved to \`${f.file}\`.`],
|
||
]) {f.final.text=heading+'\n\n'+fields.join('\n\n');expect(f.check(),f.final.text).toBe(true);}
|
||
}
|
||
}
|
||
} finally {f.cleanup();}
|
||
});
|
||
for (const [name, change] of Object.entries({
|
||
'blocked':(s:string)=>s.replace('DONE —','BLOCKED —'),
|
||
'concerns':(s:string)=>s.replace('DONE —','DONE_WITH_CONCERNS —'),
|
||
'pending':(s:string)=>s.replace('DONE —','NEEDS_CONTEXT —'),
|
||
'conditional status':(s:string)=>s.replace('DONE —','DONE if approved —'),
|
||
'conditional reason':(s:string)=>s.replace('completed with evidence','will be completed with evidence'),
|
||
'quoted status':(s:string)=>s.replace('**STATUS:**','> **STATUS:**'),
|
||
'literal status':(s:string)=>s.replace(/\*\*STATUS:\*\* (.+)/,'`STATUS: $1`'),
|
||
'fenced status':(s:string)=>s.replace(/\*\*STATUS:\*\* (.+)/,'```text\nSTATUS: $1\n```'),
|
||
'duplicate status':(s:string)=>s+'\nSTATUS: DONE',
|
||
'conflicting status':(s:string)=>s+'\nSTATUS: BLOCKED',
|
||
'historical context':(s:string)=>'Previous result:\n\n'+s,
|
||
'copied section':(s:string)=>'Source example:\n\n'+s,
|
||
'quoted section':(s:string)=>'> '+s.replaceAll('\n','\n> '),
|
||
'duplicate section':(s:string)=>s+'\n## Review complete\nSTATUS: DONE',
|
||
'unavailable report':(s:string)=>s.replace('now carries','is unavailable; would contain'),
|
||
'proposed write':(s:string)=>s.replace('now carries','will contain'),
|
||
'historical report':(s:string)=>s.replace('now carries','previously contained'),
|
||
'wrong path':(s:string)=>s.replaceAll('gstack-test-plan-design.md','wrong-plan.md'),
|
||
'ambiguous path':(s:string)=>s.replace('now carries','and `another-plan.md` now carry'),
|
||
'different absolute directory':(s:string)=>s.replaceAll('gstack-test-plan-design.md','/elsewhere/gstack-test-plan-design.md'),
|
||
'relative traversal':(s:string)=>s.replaceAll('gstack-test-plan-design.md','../gstack-test-plan-design.md'),
|
||
'quoted artifact line':(s:string)=>s.replace('**What changed:**','> **What changed:**'),
|
||
'literal artifact prose':(s:string)=>s.replace(/\*\*What changed:\*\* (.+)/,'**What changed:** "$1"'),
|
||
'missing artifact field':(s:string)=>s.replace(/^\*\*What changed:\*\*.+\n/m,''),
|
||
'withdrawn report':(s:string)=>s+'\nThe report is withdrawn.',
|
||
'remaining decision':(s:string)=>s+'\nOne design decision is unresolved.',
|
||
})) test(`typed delivery rejects ${name}`, () => {const f=completion();try {f.final.text=change(f.final.text);expect(f.check()).toBe(false);}finally{f.cleanup();}});
|
||
test('typed delivery retains source session, answer chronology, fresh file and complete Design report checks', () => {
|
||
const f=completion();try {
|
||
const original=structuredClone(f.transcript);
|
||
for (const change of [
|
||
(t:PlanCountTranscript)=>{t.calls[1]!.answered=false;},
|
||
(t:PlanCountTranscript)=>{t.calls[1]!.failed=true;},
|
||
(t:PlanCountTranscript)=>{t.calls[1]!.sessionId='foreign';},
|
||
(t:PlanCountTranscript)=>{t.calls[1]!.answeredAt=t.assistantMessages.at(-1)!.timestamp;},
|
||
(t:PlanCountTranscript)=>{t.assistantMessages.at(-1)!.timestamp='2999-01-01T00:00:00Z';},
|
||
]) {Object.assign(f.transcript,structuredClone(original));change(f.transcript);expect(f.check()).toBe(false);}
|
||
Object.assign(f.transcript,structuredClone(original));
|
||
for (const body of ['# Draft',fixture.report+'\n## Implementation changes\n',fixture.report.replace('| 1 | clean |','| 1 | pending |'),fixture.report.replace('DESIGN CLEARED','NOT CLEARED'),fixture.report.replace('NO UNRESOLVED DECISIONS','**UNRESOLVED DECISIONS:**\n- Still open')]) {f.write(body);expect(f.check()).toBe(false);}
|
||
f.write();fs.utimesSync(f.file,1,1);expect(f.check()).toBe(false);
|
||
fs.rmSync(f.file);expect(f.check()).toBe(false);
|
||
const alternate=path.join(f.dir,'alternate.md');fs.writeFileSync(alternate,fixture.report);fs.symlinkSync(alternate,f.file);expect(f.check()).toBe(false);
|
||
}finally{f.cleanup();}
|
||
});
|
||
const cf74 = fixture.cf74Retry;
|
||
|
||
function cf74Completion() {
|
||
const dir=fs.mkdtempSync(path.join(os.tmpdir(),'design-cf74-completion-'));
|
||
const file=path.join(dir,path.basename(cf74.provenance.planPath));
|
||
const transcript=structuredClone(cf74.transcript) as PlanCountTranscript;
|
||
const final=transcript.assistantMessages.at(-1)!;
|
||
final.text=final.text.replaceAll(cf74.provenance.planPath,file);
|
||
const startedAt=Math.min(...transcript.calls.map(c=>Date.parse(c.answeredAt!)))-1000;
|
||
const write=(body=cf74.report)=>{fs.writeFileSync(file,body);fs.utimesSync(file,cf74.provenance.reportMtimeMs/1000,cf74.provenance.reportMtimeMs/1000);};
|
||
write();
|
||
return {dir,file,transcript,final,startedAt,write,check:()=>hasNativePlanTerminal(transcript,file,startedAt,'completion_summary'),cleanup:()=>fs.rmSync(dir,{recursive:true,force:true})};
|
||
}
|
||
|
||
test('cf74 actual completed native report envelope binds the fresh owned Design report',()=>{
|
||
const f=cf74Completion();try{expect(f.check()).toBe(true);}finally{f.cleanup();}
|
||
});
|
||
for(const heading of ['## Completion report','### Completion summary','## Completion'])for(const field of ['Plan written:','Plan saved:','Plan written to'])
|
||
test(`cf74 complete typed delivery: ${heading}/${field}`,()=>{
|
||
const f=cf74Completion();try{f.final.text=f.final.text.replace('## Completion report',heading).replace('Plan written:',field);expect(f.check()).toBe(true);}finally{f.cleanup();}
|
||
});
|
||
for(const [name,change]of Object.entries({
|
||
'pending status':(s:string)=>s.replace('STATUS: DONE','STATUS: PENDING'),
|
||
'conditional status':(s:string)=>s.replace('STATUS: DONE','STATUS: DONE if approved'),
|
||
'quoted status':(s:string)=>s.replace('**STATUS: DONE**','`STATUS: DONE`'),
|
||
'duplicate status':(s:string)=>s+'\nSTATUS: DONE',
|
||
'quoted whole report':(s:string)=>'> '+s.replaceAll('\n','\n> '),
|
||
'historical report':(s:string)=>s.replace('## Completion report','Historical source:\n\n## Completion report'),
|
||
'duplicate report':(s:string)=>s+'\n## Completion report\nSTATUS: DONE',
|
||
'future write':(s:string)=>s.replace('Plan written:','Plan will be written:'),
|
||
'conditional write':(s:string)=>s.replace('Plan written:', 'Plan written if approved:'),
|
||
'quoted written field':(s:string)=>s.replace('- **Plan written:**','> **Plan written:**'),
|
||
'ambiguous path':(s:string)=>s.replace(' — accepted behavior',' and another-report.md — accepted behavior'),
|
||
'foreign path':(s:string)=>s.replaceAll('gstack-test-plan-design.md','foreign-report.md'),
|
||
'withdrawn report':(s:string)=>s+'\nThe review report is withdrawn.',
|
||
'unresolved decision':(s:string)=>s+'\nOne design decision is unresolved.',
|
||
'quoted current unresolved status':(s:string)=>s+'\nOne design decision is "unresolved".',
|
||
}))test(`cf74 typed completion rejects ${name}`,()=>{const f=cf74Completion();try{f.final.text=change(f.final.text);expect(f.check()).toBe(false);}finally{f.cleanup();}});
|
||
test('cf74 typed envelope cannot bypass fresh own Design report and native chronology',()=>{
|
||
const f=cf74Completion();try{
|
||
const base=structuredClone(f.transcript);
|
||
for(const change of [
|
||
(t:PlanCountTranscript)=>{t.calls[0]!.answered=false;},(t:PlanCountTranscript)=>{t.calls[0]!.failed=true;},
|
||
(t:PlanCountTranscript)=>{t.calls[0]!.answers={};},(t:PlanCountTranscript)=>{t.calls[0]!.unansweredQuestionIndices=[0];},
|
||
(t:PlanCountTranscript)=>{t.calls[0]!.sessionId='foreign';},(t:PlanCountTranscript)=>{t.calls[0]!.answeredAt=t.assistantMessages.at(-1)!.timestamp;},
|
||
]){Object.assign(f.transcript,structuredClone(base));change(f.transcript);expect(f.check()).toBe(false);}
|
||
Object.assign(f.transcript,structuredClone(base));
|
||
for(const report of [cf74.report.replace('| 1 | clean |','| 1 | pending |'),cf74.report.replace('DESIGN CLEARED','DESIGN NOT CLEARED'),cf74.report.replace('NO UNRESOLVED DECISIONS','**UNRESOLVED DECISIONS:**\n- One pending'),cf74.report+'\n## Another section\n', '# Draft']){f.write(report);expect(f.check()).toBe(false);}
|
||
f.write();fs.utimesSync(f.file,1,1);expect(f.check()).toBe(false);
|
||
fs.rmSync(f.file);expect(f.check()).toBe(false);
|
||
const target=path.join(f.dir,'other.md');fs.writeFileSync(target,cf74.report);fs.symlinkSync(target,f.file);expect(f.check()).toBe(false);
|
||
}finally{f.cleanup();}
|
||
});
|
||
});
|