diff --git a/CHANGELOG.md b/CHANGELOG.md index f9dccbd09..40d3a733f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,42 @@ ### Gate integrity - `test/bun-entry-isolation.test.ts` is strengthened. On POSIX the required shell spelling still resolves to `/dev/null`. The literal `--config=/dev/null` in a shipped shell file is now an offender, as is an undefined or redefined `GSTACK_BUN_CONFIG`. The shell-caller detector also skips a `--config` value before deciding whether a call starts a program, so `bun --no-env-file --config= -e` no longer passes unchecked; the tree has no such call. Negative controls: the old spelling, a variable without the default, an empty `--config=`, a missing, late or altered definition, and a reverted bin each fail the check. No threshold, budget, timeout, trial count or case kind changed. +## [1.91.223.0] - 2026-10-25 + +**/plan-tune's inferred profile finally moves: a clicked answer is logged as the question's registered option key, not its label.** #2623 showed every inferred dimension stuck at 0.5 on a 1,286-event log: the AskUserQuestion hook recorded labels like "A) Fix it now (recommended)", while the signal map that turns answers into dimension nudges is keyed by registered option keys like `fix-now`, so 0 of 1,286 answers matched. Now five hook-captured label answers to the registered architecture-finding question move `architecture_care` from 0.5 to 0.632, and `--trace` lists all five. + +### Fixed +- **`bin/gstack-question-log`**: on a hook-captured event (`source: hook`) for a registered `question_id`, the clicked label is mapped to that question's registered option key before it is written, and the label is kept in a new `choice_label` field. A label in `recommended` is mapped the same way, so `followed_recommendation` compares keys. Ad-hoc and `hook-` ids and unmatched or ambiguous labels are written as given. Rows from other writers (the agent's own log call, auto-decide, Codex import, typed free text) are unchanged, because readers such as the plan-ceo-review auto-decide check match the full mode names those rows carry. Existing logs are not rewritten. +- **`scripts/psychographic-signals.ts`**: new `resolveOptionKey(question_id, label)`. It strips the `A)` selector and `(recommended)` suffix, then matches the label's leading words against the question's own options only: each key's own words (`fix-now` → "fix now") plus a short cue list ("Add to TODOs" → `defer`, "Ship as-is" → `skip`, "HOLD SCOPE" → `hold`). The longest cue wins and a tie stays unmapped. + +### Tests +- `test/plan-tune.test.ts`: every registered option resolves from its own words, bare and as an `A) … (recommended)` label; 20 common phrasings, including the four plan-ceo-review mode names, map to their keys; negative controls (unregistered ids, a question with no options, a label naming no option, a mid-sentence cue, another question's key, a number that looks like a selector) stay unmapped. Two capture tests run the real bin: hook labels become keys with `choice_label` kept, while agent rows (including a `HOLD SCOPE` auto-decide row), ad-hoc ids, unmatched labels and free text are untouched, and a picked label follows a key recommendation. Both capture tests fail against the old bin. + +### Gate integrity +- No test expectation, threshold, budget, timeout, trial count or case kind changed. Existing tests that pin label rows from the agent, auto-decide and Codex import paths pass unchanged. + +## [1.91.222.0] - 2026-10-25 + +**Five flaky tests now fail only for real defects, and a judge reply that isn't valid JSON says so in its failure line (daily flaky-fix pass, GSTA-91).** `context-restore-fragment-match` failed 0/3 on main cb99df4, although every run restored the right checkpoint: to find the checkpoint whose title matches, the agent often prints all three, and the case failed whenever another checkpoint's body appeared anywhere in the transcript. `test/gstack-ff-update.test.ts` counted every `sleep 300` on the machine, so heartbeats from parallel test files changed its count (remote pregate at #3412's head: expected "1", received "3"). The `GSTACK_CAPY=1` install-admission case in `test/gstack-skill-start.test.ts` makes ten `gstack-skill-start` spawns under bun's 5 s default timeout and took 3.6–5.7 s on a 4-core machine. The QA-only fixture's late scenarios mocked a 1000 ms capture budget but ran each attempt's real setup inside it. The setup-browser-cookies workflow judge in train-4 gate 40173759210 returned JSON with an unescaped quote in its reasoning, and the flake record read only "JSON Parse error: Expected '}'". + +### Fixed +- **`test/helpers/context-restore-fragment.ts`** (new): `fragmentRestoreVerdict` reads only the agent's report (top-level assistant text and the final result), never tool output. The report must carry the payments checkpoint's body and no other checkpoint's body. `test/skill-e2e-context-skills.test.ts` uses it for the fragment-match case. +- **`test/helpers/llm-judge.ts`**: when a judge response is not valid JSON, `callJudge` throws a `SyntaxError` that names the judge output, its stop reason and length, and the retained `llm-judge-response-parse-error` event. It still fails closed and parses no score; the request is unchanged. + +### Tests +- `test/context-restore-fragment-verdict.test.ts` replays the three captured false reds from main (`test/fixtures/context-restore-fragment-match-captures.json`) as passes. Controls that must fail: captures rewritten to resume alpha or omega, synthetic runs that read and resume only alpha or omega, resume summaries that mix two checkpoints, and a payments body that appears only in tool output. Restoring the old any-byte surface fails all four positive controls. Focused paid runs of the case: 0/3 on main, 3/3 after (about $0.11 a trial). +- `test/llm-judge-frontier.test.ts` replays the exact captured cookie-judge reply: it fails closed with no score, one request and the new diagnosis. It fails on the previous helper. +- `test/gstack-ff-update.test.ts`: the reap case runs `gstack-ff-update` with a heartbeat interval unique to the test process and counts only sleepers of that interval, starting from zero. The old count failed with a loop starting unrelated `sleep 300` processes beside it (expected "5", received "14"); the new one passes, and still fails when the bin stops reaping its sleeper. +- `test/qa-only-fixture.test.ts`: in the late scenarios the fixture clock stands still from the start of each attempt until its runner starts, so the real setup work (directory copies and five `git` calls) costs no fixture time. The runner and cleanup stay on the real clock, as the deadline-evidence check requires. Each late capture must now get exactly the 1000 ms budget. A new `late-slow-retry` scenario stalls each attempt's setup for 1100 ms, longer than the whole budget, and its attempt 2 still has to succeed. Before, a loaded machine either used up the work budget in setup (`QA-only work budget exhausted`) or Bun's 1580 ms deadline killed a setup `git` child (exit `null` at `skill-e2e-qa-workflow.test.ts:133`); this is what failed late-retry in the 2026-10-24 and 2026-10-25 remote pregates. Under 24 busy loops on 4 cores: late-retry 1/20 → 20/20, late-failure-retry 2/20 → 20/20, late-timeout 6/20 → 20/20, late-slow-retry 20/20. Reverting the clock fails all four late scenarios unloaded; reverting the deadline fails `late-slow-retry`. +- `test/gstack-skill-start.test.ts`: the `GSTACK_CAPY=1` install-admission case gets an explicit 20 s timeout. Measured on a 4-core Capy machine with a temporary 60 s cap: 3.6–5.7 s through Capy's git shim (9 of 10 over 5 s), 2.6–4.9 s with the real git binary. Four of its ten spawns are full admissions at about 800 ms each. + +### Gate integrity +- One test budget changed, approved by GStack Minder on 2026-10-25: the install-admission case's timeout goes from bun's 5 s default to 20 s, about 3.5 times the 5.7 s worst case measured above. No assertion changed. +- The QA-only late scenarios register each attempt with its real outer budget (1000 + 80 + 5000 = 6080 ms, the value `outerMs` already asserts) instead of a shortened 1580 ms Bun deadline, which left cleanup 500 ms of real time under load. Measured child totals with the fix under 24 busy loops, 20 runs each: late-timeout 2.4–2.7 s, late-retry 4.1–4.3 s, late-failure-retry 4.7–5.0 s, late-slow-retry 6.2–6.6 s, against the parent's unchanged 15 s spawn timeout. The mocked 1000 ms capture budget, the 80 ms drain and every existing assertion are unchanged. Verdict: justified, because the deadline was a fixture artifact shorter than the budget the case declares, and the clock freeze, not the deadline, is what removes the load sensitivity. +- `context-restore-fragment-match`: the failure surface moves from the whole transcript to the agent's report. Alpha or omega bytes in tool output no longer fail the case; the payments body in tool output no longer counts as loaded, so the report must show it. A run that resumes another checkpoint, or mixes two, still fails. `test/test-of-test-ratchet.test.ts` gains one BASELINE entry, the replay test, with its reason, and the new helper joins the touchfiles of the nine context-skills cases, as the derived-closure rule requires. +- `scripts/harness-version.json` records `test/helpers/llm-judge.ts` as non-behavioral at HARNESS_VERSION 6: only a parse failure's message changes, before any score is parsed, so no case's verdict or pass-rate series changes. +- No threshold, trial count, case kind, rubric, judge request or prompt changed. + ## [1.91.221.0] - 2026-10-25 **The session-start updater no longer skips because of its own session's workflow lease, and the autostash tests stop failing their first pass in Free Tests.** Since #3385, `gstack-session-update` skips the fast-forward while another session holds a workflow lease, and it means to ignore the session that started it (`--exclude-ancestry-of`). The check runs in the forked background updater, though, and passed the hook's own `$$`, which has exited by then, so the ancestry walk excluded nothing. A Claude Code process that had already run a skill (and so held a lease) and then started a session again in the same process (`/clear`, resume or compaction) logged `SKIP leased pid=` and never updated. `test/session-update-autostash.test.ts` hit the same gap plus a fixture leak: its first two fixtures spawned the updater with the whole parent environment, so it read the test preload's `GSTACK_LEASE_DIR`, which every test file in the same Bun process shares. A lease another file left there gave "Received: SKIP leased pid=3196/1m" where UPDATING or PULL_FAILED was expected, in 13 Free Tests runs since #3385 merged (each passed on the serial retry). diff --git a/VERSION b/VERSION index 13b9186ef..999c96dfa 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.91.221.0 +1.91.223.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index a582bf0f8..fc6fd4bae 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.91.221.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.91.223.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/bin/gstack-question-log b/bin/gstack-question-log index 01f7b368c..1de42718d 100755 --- a/bin/gstack-question-log +++ b/bin/gstack-question-log @@ -18,8 +18,13 @@ # door_type — one-way | two-way # (optional — looked up from registry if omitted) # options_count — number of options presented (positive integer) -# user_choice — key user selected (free string; registry-options preferred) -# recommended — option key the agent recommended (optional) +# user_choice — key user selected (free string; registry-options preferred). +# On hook-captured events (source=hook) for a registered +# question_id, the clicked UI label is mapped to its +# registered option key (#2623) and kept in choice_label. +# Unmatched labels and other sources stay as given. +# recommended — option key the agent recommended (optional; mapped +# the same way on hook-captured events) # followed_recommendation — bool (optional — computed if both present) # session_id — stable session identifier # ts — ISO 8601 timestamp (auto-injected if missing) @@ -47,8 +52,9 @@ INPUT="$1" TMPERR=$(mktemp "${TMPDIR:-/tmp}/gstack-question-log.XXXXXX") || { echo "gstack-question-log: could not create a temp file in ${TMPDIR:-/tmp}, so the question was not logged. Fix: point TMPDIR at a writable directory." >&2; exit 1; } trap 'rm -f "$TMPERR"' EXIT set +e -VALIDATED=$(printf '%s' "$INPUT" | GSTACK_LIB_DIR="$SCRIPT_DIR/../lib" bun --no-env-file --config="${GSTACK_BUN_CONFIG:-/dev/null}" -e " +VALIDATED=$(printf '%s' "$INPUT" | GSTACK_LIB_DIR="$SCRIPT_DIR/../lib" GSTACK_SCRIPTS_DIR="$SCRIPT_DIR/../scripts" bun --no-env-file --config="${GSTACK_BUN_CONFIG:-/dev/null}" -e " const { hasInjection } = await import(process.env.GSTACK_LIB_DIR + '/jsonl-store.ts'); +const { resolveOptionKey } = await import(process.env.GSTACK_SCRIPTS_DIR + '/psychographic-signals.ts'); const path = require('path'); const raw = await Bun.stdin.text(); let j; @@ -174,6 +180,23 @@ if (j.recommended !== undefined) { if (j.recommended.length > 64) j.recommended = j.recommended.slice(0, 64); } +// Map the clicked UI label to its registered option key (#2623). SIGNAL_MAP +// and question preferences are keyed by the registry's option keys, but the +// PostToolUse hook only sees the label. Hook events and registered ids only; +// an unmatched or ambiguous label stays as given. Agent-written rows keep +// what the agent logged: skills such as plan-ceo-review log full mode names +// that their own readers match on. +if (j.source === 'hook') { + const key = resolveOptionKey(j.question_id, j.user_choice); + if (key !== undefined && key !== j.user_choice) { + j.choice_label = j.user_choice; + j.user_choice = key; + } + if (j.recommended !== undefined) { + j.recommended = resolveOptionKey(j.question_id, j.recommended) ?? j.recommended; + } +} + // followed_recommendation — compute if both sides present. An __unknown__ // choice means extraction failed, not that the user rejected the // recommendation — leave the field absent so metrics can't be poisoned. diff --git a/package.json b/package.json index 0b396bbf1..28dbec666 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.91.221", + "version": "1.91.223", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/scripts/harness-version.json b/scripts/harness-version.json index bcf395dd3..0cb61ce66 100644 --- a/scripts/harness-version.json +++ b/scripts/harness-version.json @@ -29,7 +29,7 @@ "test/helpers/eval-budgets.ts": {"blob":"667d0e09b20fe5d7366f57b77b5fb9b04baa37ef","since":6}, "test/helpers/eval-store.ts": {"blob":"a50ea2c28f508ad9f35ea9353dd2f502ee0dcfc2","since":6,"nonBehavioral":"credential rejections are classed infra instead of assertion: no case's pass or fail changes; only failed trials whose provider rejected the run's key move class, so no rerun spends on a rejected key"}, "test/helpers/hermetic-env.ts": {"blob":"cc072895461dc52ea317d1beff72121342420d30","since":6}, - "test/helpers/llm-judge.ts": {"blob":"d0bc6bbbe9f478603d11a1ece95e66c812ad34e6","since":6}, + "test/helpers/llm-judge.ts": {"blob":"a3177665b662932ce485ccc976c2b36a1531a0bb","since":6,"nonBehavioral":"callJudge rewrites only the message of a JSON parse failure (names the judge output, stop reason, length and the retained llm-judge-response-parse-error event); it still throws a SyntaxError before any score is parsed, the request is unchanged, and no case's pass or fail changes"}, "test/helpers/paid-test-set.ts": {"blob":"ff9bfe43c43ec2bdd11114f7b8ec7c5736116f75","since":6}, "test/helpers/pty-screen.ts": {"blob":"4f77569f9ce11f9cee9f347216e468cd7c956820","since":6}, "test/helpers/pty/auq.ts": {"blob":"16e5ea2bb5b9f28f44808a1a09024dbd3e6f28c4","since":6}, diff --git a/scripts/psychographic-signals.ts b/scripts/psychographic-signals.ts index decc8c6d5..43d4a89ab 100644 --- a/scripts/psychographic-signals.ts +++ b/scripts/psychographic-signals.ts @@ -223,6 +223,89 @@ export const SIGNAL_MAP: Record> = { }, }; +/** + * Label cues: option key → leading words of UI labels that mean that key. + * + * The AskUserQuestion hook only sees the label the user clicked ("A) Fix it + * now (recommended)"), never a stable key, while SIGNAL_MAP is keyed by the + * registry's option keys (#2623). resolveOptionKey() maps a label back to a + * key when bin/gstack-question-log records a hook event. Every key also + * matches its own words ("fix-now" → "fix now"), so only phrasings beyond + * that need an entry here. Cues are + * lowercase words separated by single spaces and match at the start of the + * normalized label; candidates are limited to the question's own registered + * options, and the longest cue wins. + */ +const OPTION_LABEL_CUES: Record = { + accept: ['yes', 'approve', 'agree', 'add', 'include', 'apply', 'adopt', 'enable', 'go ahead', 'do it', 'proceed', 'run', 'create', 'merge', 'roll back', 'migrate', 'open', 'promote', 'reorganize', 'polish'], + reject: ['no', 'disagree', 'decline', 'cancel', 'abort', 'dont', 'do not', 'keep current', 'keep as is', 'keep it as is'], + skip: ['no', 'no thanks', 'not now', 'dont', 'do not', 'pass', 'drop', 'ignore', 'leave it', 'ship as is', 'ship it as is'], + defer: ['later', 'fix later', 'add to todos', 'add to todo', 'add a todo', 'todo', 'postpone', 'backlog', 'park', 'track it'], + 'fix-now': ['fix', 'fix it', 'fix them', 'fix before shipping'], + 'fix-anyway': ['fix', 'fix it'], + investigate: ['dig in', 'look into', 'debug', 'diagnose', 'root cause'], + 'ack-and-ship': ['ack', 'acknowledge', 'ship', 'ship anyway', 'ship it'], + 'accept-risk': ['accept', 'accept the risk', 'live with it'], + 'false-positive': ['not a bug', 'mark false positive', 'mark as false positive', 'mark it false positive'], + 'add-test': ['add', 'add tests', 'add the test', 'write', 'write the test', 'test it'], + expand: ['scope expansion'], + selective: ['cherry pick', 'selective expansion'], + hold: ['keep', 'keep scope', 'hold scope'], + reduce: ['cut', 'trim', 'shrink', 'scope down', 'scope reduction'], + revise: ['edit', 'change', 'rework'], + restart: ['start over', 'redo'], + 'build-now': ['build', 'build it now', 'include in this pr'], + 'keep-local': ['keep'], + 'keep-vendored': ['keep'], + 'yes-upgrade': ['yes', 'upgrade'], + 'always-auto': ['always'], + 'not-now': ['later', 'snooze'], + 'never-ask': ['never', 'disable'], +}; + +function normalizeOptionLabel(label: string): string { + return label + .toLowerCase() + .replace(/\s*\(recommended\)\s*$/, '') + .replace(/['\u2019]/g, '') + .replace(/^\s*(?:option\s+)?[a-z0-9]\s*(?:[).:]|-(?=\s))\s+/, '') + .replace(/[^a-z0-9]+/g, ' ') + .trim(); +} + +/** + * Map a chosen UI label to the registered option key of its question. + * + * Returns the key when exactly one of the question's registered options + * matches with the longest cue; undefined for unregistered ids, questions + * without options, unmatched labels and ties, so an unclear answer stays + * unmapped instead of nudging the wrong dimension. + */ +export function resolveOptionKey(question_id: string, choice: string): string | undefined { + const options = (QUESTIONS as Record)[question_id]?.options as string[] | undefined; + if (!options?.length) return undefined; + if (options.includes(choice)) return choice; + const label = normalizeOptionLabel(choice); + if (!label) return undefined; + let best: string | undefined; + let bestWords = 0; + let tie = false; + for (const key of options) { + for (const cue of [key.replace(/-/g, ' '), ...(OPTION_LABEL_CUES[key] ?? [])]) { + if (label !== cue && !label.startsWith(cue + ' ')) continue; + const words = cue.split(' ').length; + if (words > bestWords) { + best = key; + bestWords = words; + tie = false; + } else if (words === bestWords && best !== key) { + tie = true; + } + } + } + return tie ? undefined : best; +} + /** * Apply a user choice for a question to the running dimension totals. * diff --git a/test/context-restore-fragment-verdict.test.ts b/test/context-restore-fragment-verdict.test.ts new file mode 100644 index 000000000..6fed164e0 --- /dev/null +++ b/test/context-restore-fragment-verdict.test.ts @@ -0,0 +1,121 @@ +import { describe, test, expect } from 'bun:test'; +import * as fs from 'fs'; +import * as path from 'path'; +import { parseNDJSON } from './helpers/session-runner'; +import { fragmentRestoreVerdict } from './helpers/context-restore-fragment'; + +const MARKERS = { restored: 'FRAGMATCH_PAYMENTS_BUILD', others: ['FRAGMATCH_ALPHA_BUILD', 'FRAGMATCH_OMEGA_BUILD'] }; +const BODIES: Record = { + '20260101-120000-alpha-feature.md': 'branch: feat/alpha\n## Working on: alpha feature\n### Summary\nAlpha content FRAGMATCH_ALPHA_BUILD\n', + '20260202-120000-middle-payments.md': 'branch: feat/payments\n## Working on: middle payments\n### Summary\nPayments content FRAGMATCH_PAYMENTS_BUILD\n', + '20260303-120000-omega-release.md': 'branch: feat/omega\n## Working on: omega release\n### Summary\nOmega content FRAGMATCH_OMEGA_BUILD\n', +}; +const ALL_FILES = Object.keys(BODIES); + +const captures: { sessions: Array<{ session_id: string; ndjson: string[] }> } = JSON.parse( + fs.readFileSync(path.join(import.meta.dir, 'fixtures', 'context-restore-fragment-match-captures.json'), 'utf8'), +); + +function replay(lines: string[]) { + const parsed = parseNDJSON(lines); + return { ...parsed, output: parsed.resultLine?.result || '' }; +} + +function toolOutput(run: ReturnType): string { + return run.toolCalls.map((call) => call.output).join('\n'); +} + +function isReportLine(line: string): boolean { + const event = JSON.parse(line); + return event.type === 'result' + || (event.type === 'assistant' && event.message.content.some((block: any) => block.type === 'text')); +} + +function rewriteReport(lines: string[], rewrite: (line: string) => string): string[] { + return lines.map((line) => (isReportLine(line) ? rewrite(line) : line)); +} + +function synthetic(files: string[], report: string): string[] { + const lines: string[] = [ + JSON.stringify({ type: 'assistant', parent_tool_use_id: null, message: { content: [{ type: 'tool_use', id: 'skill', name: 'Skill', input: { skill: 'context-restore', args: 'payments' } }] } }), + JSON.stringify({ type: 'user', parent_tool_use_id: null, message: { content: [{ type: 'tool_result', tool_use_id: 'skill', content: 'Launching skill: context-restore' }] } }), + ]; + files.forEach((file, i) => { + lines.push(JSON.stringify({ type: 'assistant', parent_tool_use_id: null, message: { content: [{ type: 'tool_use', id: `read-${i}`, name: 'Read', input: { file_path: `.gstack-home/projects/p/checkpoints/${file}` } }] } })); + lines.push(JSON.stringify({ type: 'user', parent_tool_use_id: null, message: { content: [{ type: 'tool_result', tool_use_id: `read-${i}`, content: BODIES[file] }] } })); + }); + if (report) { + lines.push(JSON.stringify({ type: 'assistant', parent_tool_use_id: null, message: { content: [{ type: 'text', text: report }] } })); + } + lines.push(JSON.stringify({ type: 'result', subtype: report ? 'success' : 'error_max_turns', result: report })); + return lines; +} + +function resumeReport(file: string, ...summaries: string[]): string { + return `I loaded \`${file}\`.\n\nRESUMING CONTEXT\n### Summary\n${summaries.join('\n')}\n`; +} + +describe('context-restore-fragment-match verdict', () => { + test('captured runs hold three sessions that each printed another checkpoint body', () => { + expect(captures.sessions).toHaveLength(3); + for (const session of captures.sessions) { + const run = replay(session.ndjson); + expect(run.toolCalls.map((call) => call.tool)).toContain('Skill'); + expect(toolOutput(run)).toContain('FRAGMATCH_PAYMENTS_BUILD'); + expect(MARKERS.others.some((marker) => toolOutput(run).includes(marker))).toBe(true); + } + }); + + test.each(captures.sessions.map((s) => [s.session_id, s.ndjson] as const))( + 'captured run %s: inspected every checkpoint, restored payments -> pass', + (_id, ndjson) => { + expect(fragmentRestoreVerdict(replay(ndjson), MARKERS)).toEqual({ loadedTarget: true, didNotLoadOthers: true }); + }, + ); + + test('synthetic: prints every checkpoint, reports payments -> pass', () => { + const run = replay(synthetic(ALL_FILES, resumeReport(ALL_FILES[1], 'Payments content FRAGMATCH_PAYMENTS_BUILD'))); + expect(fragmentRestoreVerdict(run, MARKERS)).toEqual({ loadedTarget: true, didNotLoadOthers: true }); + }); + + test.each([ + ['alpha', 'FRAGMATCH_ALPHA_BUILD'], + ['omega', 'FRAGMATCH_OMEGA_BUILD'], + ])('captured run rewritten to resume %s -> fail', (_name, marker) => { + for (const session of captures.sessions) { + const run = replay(rewriteReport(session.ndjson, (line) => line.replaceAll('FRAGMATCH_PAYMENTS_BUILD', marker))); + expect(toolOutput(run)).toContain('FRAGMATCH_PAYMENTS_BUILD'); + expect(fragmentRestoreVerdict(run, MARKERS)).toEqual({ loadedTarget: false, didNotLoadOthers: false }); + } + }); + + test.each([ + ['alpha', ALL_FILES[0], 'Alpha content FRAGMATCH_ALPHA_BUILD'], + ['omega', ALL_FILES[2], 'Omega content FRAGMATCH_OMEGA_BUILD'], + ])('synthetic: reads only %s and resumes it -> fail', (_name, file, summary) => { + const run = replay(synthetic([file], resumeReport(file, summary))); + expect(fragmentRestoreVerdict(run, MARKERS)).toEqual({ loadedTarget: false, didNotLoadOthers: false }); + }); + + test('captured run with omega content mixed into the resume summary -> fail', () => { + for (const session of captures.sessions) { + const mixed = rewriteReport(session.ndjson, (line) => line.replaceAll( + 'Payments content FRAGMATCH_PAYMENTS_BUILD', 'Payments content FRAGMATCH_PAYMENTS_BUILD\\nOmega content FRAGMATCH_OMEGA_BUILD')); + expect(fragmentRestoreVerdict(replay(mixed), MARKERS)).toEqual({ loadedTarget: true, didNotLoadOthers: false }); + } + }); + + test('synthetic: resume summary mixes alpha and payments -> fail', () => { + const run = replay(synthetic(ALL_FILES, resumeReport(ALL_FILES[1], + 'Payments content FRAGMATCH_PAYMENTS_BUILD', 'Alpha content FRAGMATCH_ALPHA_BUILD'))); + expect(fragmentRestoreVerdict(run, MARKERS)).toEqual({ loadedTarget: true, didNotLoadOthers: false }); + }); + + test('payments body only in tool output, no report of it -> fail', () => { + const noReport = replay(synthetic([ALL_FILES[1]], '')); + expect(toolOutput(noReport)).toContain('FRAGMATCH_PAYMENTS_BUILD'); + expect(fragmentRestoreVerdict(noReport, MARKERS).loadedTarget).toBe(false); + const nameOnly = replay(synthetic(ALL_FILES, `I loaded \`${ALL_FILES[1]}\`.`)); + expect(fragmentRestoreVerdict(nameOnly, MARKERS).loadedTarget).toBe(false); + }); +}); diff --git a/test/fixtures/context-restore-fragment-match-captures.json b/test/fixtures/context-restore-fragment-match-captures.json new file mode 100644 index 000000000..36e729068 --- /dev/null +++ b/test/fixtures/context-restore-fragment-match-captures.json @@ -0,0 +1,51 @@ +{ + "provenance": { + "case": "context-restore-fragment-match", + "run": "local paid baseline: scripts/test-paid-shards.ts --tier periodic --case context-restore-fragment-match --trials 3 (0/3, all on didNotLoadOthers)", + "head": "cb99df4a2fc45bc983e0a64f862f79fab0ce899a", + "claude_code": "2.1.292", + "source_sha256": "2d05a7257f7d559d60ec2e644178a3d65cd0799121260633c931d5b54755b495", + "projection": "Raw NDJSON lines of each session's assistant, user and result events, unchanged; system events, thinking-only assistant events and the synthetic user message carrying the extracted SKILL.md body dropped." + }, + "sessions": [ + { + "session_id": "d1147327-49e3-4371-ab54-784d135573ac", + "ndjson": [ + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2Sp1UcGZ3JFpbUbxQg\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"tool_use\",\"id\":\"toolu_01HHxMjn7G6dQ36WFwvafm5W\",\"name\":\"Skill\",\"input\":{\"skill\":\"context-restore\",\"args\":\"payments\"},\"caller\":{\"type\":\"direct\"}}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":6256,\"cache_read_input_tokens\":12042,\"cache_creation\":{\"ephemeral_5m_input_tokens\":6256,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":16,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"d1147327-49e3-4371-ab54-784d135573ac\",\"uuid\":\"691912ce-6742-4e21-8f4c-a3cfa15b4112\",\"timestamp\":\"2026-10-25T13:21:55.808Z\",\"request_id\":\"req_011CgP2Som6KWQBvN4W6W3B8\",\"wire_tool_inputs\":{\"toolu_01HHxMjn7G6dQ36WFwvafm5W\":{\"skill\":\"context-restore\",\"args\":\"payments\"}}}", + "{\"type\":\"user\",\"message\":{\"role\":\"user\",\"content\":[{\"type\":\"tool_result\",\"tool_use_id\":\"toolu_01HHxMjn7G6dQ36WFwvafm5W\",\"content\":\"Launching skill: context-restore\"}]},\"parent_tool_use_id\":null,\"session_id\":\"d1147327-49e3-4371-ab54-784d135573ac\",\"uuid\":\"9e05cd18-bf0c-4d75-93d8-db0f817d8c87\",\"timestamp\":\"2026-10-25T13:21:55.840Z\",\"tool_use_result\":{\"success\":true,\"commandName\":\"context-restore\",\"allowedTools\":[\"Bash\",\"Read\",\"Glob\",\"Grep\",\"AskUserQuestion\"]}}", + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2SwJ2idXv6jZBjmMRS\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"tool_use\",\"id\":\"toolu_01EyWkH3v7Gu7cBDAgVzCiSu\",\"name\":\"Bash\",\"input\":{\"command\":\"cd /tmp/gstack-paid-shard-O15pOD/tmp/skill-e2e-ctx-fragment-svqfPa; ls bin; export GSTACK_HOME=\\\"$PWD/.gstack-home\\\"; B=./bin\\nGSTACK_STATE_ROOT=$($B/gstack-paths --get GSTACK_STATE_ROOT); echo ROOT=$GSTACK_STATE_ROOT\\nSLUG=$($B/gstack-slug --get SLUG 2>/dev/null); echo SLUG=$SLUG\\nCHECKPOINT_DIR=\\\"$GSTACK_STATE_ROOT/projects/$SLUG/checkpoints\\\"; ls -la \\\"$CHECKPOINT_DIR\\\"; find .gstack-home -type f\"},\"caller\":{\"type\":\"direct\"}}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":6339,\"cache_read_input_tokens\":18298,\"cache_creation\":{\"ephemeral_5m_input_tokens\":6339,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":8,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"d1147327-49e3-4371-ab54-784d135573ac\",\"uuid\":\"c6cad35f-a6b6-4eb2-8741-39112ddc1cae\",\"timestamp\":\"2026-10-25T13:21:58.959Z\",\"request_id\":\"req_011CgP2Sw39kvpLEXQaPZoUk\",\"wire_tool_inputs\":{\"toolu_01EyWkH3v7Gu7cBDAgVzCiSu\":{\"command\":\"cd /tmp/gstack-paid-shard-O15pOD/tmp/skill-e2e-ctx-fragment-svqfPa; ls bin; export GSTACK_HOME=\\\"$PWD/.gstack-home\\\"; B=./bin\\nGSTACK_STATE_ROOT=$($B/gstack-paths --get GSTACK_STATE_ROOT); echo ROOT=$GSTACK_STATE_ROOT\\nSLUG=$($B/gstack-slug --get SLUG 2>/dev/null); echo SLUG=$SLUG\\nCHECKPOINT_DIR=\\\"$GSTACK_STATE_ROOT/projects/$SLUG/checkpoints\\\"; ls -la \\\"$CHECKPOINT_DIR\\\"; find .gstack-home -type f\"}}}", + "{\"type\":\"user\",\"message\":{\"role\":\"user\",\"content\":[{\"tool_use_id\":\"toolu_01EyWkH3v7Gu7cBDAgVzCiSu\",\"type\":\"tool_result\",\"content\":\"gstack-config\\ngstack-learnings-log\\ngstack-learnings-search\\ngstack-paths\\ngstack-remote-identity.sh\\ngstack-repo-mode\\ngstack-slug\\ngstack-state-root.sh\\ngstack-timeline-log\\ngstack-timeline-read\\ngstack-update-check\\nROOT=/tmp/gstack-paid-shard-O15pOD/tmp/skill-e2e-ctx-fragment-svqfPa/.gstack-home\\nSLUG=skill-e2e-ctx-fragment-svqfPa\\ntotal 16\\ndrwxr-xr-x 2 user user 4096 Oct 25 13:21 .\\ndrwxr-xr-x 3 user user 60 Oct 25 13:21 ..\\n-rw-r--r-- 1 user user 157 Oct 25 13:21 20260101-120000-alpha-feature.md\\n-rw-r--r-- 1 user user 168 Oct 25 13:21 20260202-120000-middle-payments.md\\n-rw-r--r-- 1 user user 157 Oct 25 13:21 20260303-120000-omega-release.md\\n.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260101-120000-alpha-feature.md\\n.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260202-120000-middle-payments.md\\n.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260303-120000-omega-release.md\\n.gstack-home/slug-cache/_tmp_gstack-paid-shard-O15pOD_tmp_skill-e2e-ctx-fragment-svqfPa_.git\\n.gstack-home/slug-cache/_tmp_gstack-paid-shard-O15pOD_tmp_skill-e2e-ctx-fragment-svqfPa\",\"is_error\":false}]},\"parent_tool_use_id\":null,\"session_id\":\"d1147327-49e3-4371-ab54-784d135573ac\",\"uuid\":\"1beb6923-1013-4359-a964-2b06e7594c17\",\"timestamp\":\"2026-10-25T13:21:59.621Z\",\"tool_use_result\":{\"stdout\":\"gstack-config\\ngstack-learnings-log\\ngstack-learnings-search\\ngstack-paths\\ngstack-remote-identity.sh\\ngstack-repo-mode\\ngstack-slug\\ngstack-state-root.sh\\ngstack-timeline-log\\ngstack-timeline-read\\ngstack-update-check\\nROOT=/tmp/gstack-paid-shard-O15pOD/tmp/skill-e2e-ctx-fragment-svqfPa/.gstack-home\\nSLUG=skill-e2e-ctx-fragment-svqfPa\\ntotal 16\\ndrwxr-xr-x 2 user user 4096 Oct 25 13:21 .\\ndrwxr-xr-x 3 user user 60 Oct 25 13:21 ..\\n-rw-r--r-- 1 user user 157 Oct 25 13:21 20260101-120000-alpha-feature.md\\n-rw-r--r-- 1 user user 168 Oct 25 13:21 20260202-120000-middle-payments.md\\n-rw-r--r-- 1 user user 157 Oct 25 13:21 20260303-120000-omega-release.md\\n.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260101-120000-alpha-feature.md\\n.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260202-120000-middle-payments.md\\n.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260303-120000-omega-release.md\\n.gstack-home/slug-cache/_tmp_gstack-paid-shard-O15pOD_tmp_skill-e2e-ctx-fragment-svqfPa_.git\\n.gstack-home/slug-cache/_tmp_gstack-paid-shard-O15pOD_tmp_skill-e2e-ctx-fragment-svqfPa\",\"stderr\":\"\",\"interrupted\":false,\"isImage\":false,\"noOutputExpected\":false,\"bashEditDiff\":{\"files\":[{\"filePath\":\"/tmp/gstack-paid-shard-O15pOD/tmp/skill-e2e-ctx-fragment-svqfPa/.gstack-home/slug-cache/_tmp_gstack-paid-shard-O15pOD_tmp_skill-e2e-ctx-fragment-svqfPa\",\"hunks\":[{\"oldStart\":0,\"oldLines\":0,\"newStart\":1,\"newLines\":1,\"lines\":[\"+v2:skill-e2e-ctx-fragment-svqfPa\"]}],\"created\":true},{\"filePath\":\"/tmp/gstack-paid-shard-O15pOD/tmp/skill-e2e-ctx-fragment-svqfPa/.gstack-home/slug-cache/_tmp_gstack-paid-shard-O15pOD_tmp_skill-e2e-ctx-fragment-svqfPa_.git\",\"hunks\":[{\"oldStart\":0,\"oldLines\":0,\"newStart\":1,\"newLines\":1,\"lines\":[\"+v2:skill-e2e-ctx-fragment-svqfPa\"]}],\"created\":true}],\"moreFiles\":0,\"changedFiles\":[\"/tmp/gstack-paid-shard-O15pOD/tmp/skill-e2e-ctx-fragment-svqfPa/.gstack-home/slug-cache/_tmp_gstack-paid-shard-O15pOD_tmp_skill-e2e-ctx-fragment-svqfPa\",\"/tmp/gstack-paid-shard-O15pOD/tmp/skill-e2e-ctx-fragment-svqfPa/.gstack-home/slug-cache/_tmp_gstack-paid-shard-O15pOD_tmp_skill-e2e-ctx-fragment-svqfPa_.git\"]}}}", + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2TDR8mJC7cqYJgSDrA\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"tool_use\",\"id\":\"toolu_01E9t3wnBeL5brVygUoRfuoh\",\"name\":\"Bash\",\"input\":{\"command\":\"cd /tmp/gstack-paid-shard-O15pOD/tmp/skill-e2e-ctx-fragment-svqfPa/.gstack-home/projects/*/checkpoints; for f in *; do echo \\\"== $f\\\"; cat \\\"$f\\\"; done; cd ../../../..; export GSTACK_HOME=\\\"$PWD/.gstack-home\\\"; ls .gstack-home/projects/*/checkpoints/*.md | ./bin/gstack-slug --classify-checkpoints; echo rc=$?\"},\"caller\":{\"type\":\"direct\"}}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":1039,\"cache_read_input_tokens\":24637,\"cache_creation\":{\"ephemeral_5m_input_tokens\":1039,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":16,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"d1147327-49e3-4371-ab54-784d135573ac\",\"uuid\":\"9fa3b490-1be5-4221-b712-c671e2422c46\",\"timestamp\":\"2026-10-25T13:22:01.775Z\",\"request_id\":\"req_011CgP2TD6nbxFnYubbkBVGv\",\"wire_tool_inputs\":{\"toolu_01E9t3wnBeL5brVygUoRfuoh\":{\"command\":\"cd /tmp/gstack-paid-shard-O15pOD/tmp/skill-e2e-ctx-fragment-svqfPa/.gstack-home/projects/*/checkpoints; for f in *; do echo \\\"== $f\\\"; cat \\\"$f\\\"; done; cd ../../../..; export GSTACK_HOME=\\\"$PWD/.gstack-home\\\"; ls .gstack-home/projects/*/checkpoints/*.md | ./bin/gstack-slug --classify-checkpoints; echo rc=$?\"}}}", + "{\"type\":\"user\",\"message\":{\"role\":\"user\",\"content\":[{\"tool_use_id\":\"toolu_01E9t3wnBeL5brVygUoRfuoh\",\"type\":\"tool_result\",\"content\":\"== 20260101-120000-alpha-feature.md\\n---\\nstatus: in-progress\\nbranch: feat/alpha\\ntimestamp: 2026-01-01T12:00:00Z\\n---\\n## Working on: alpha feature\\n\\n### Summary\\nAlpha content FRAGMATCH_ALPHA_BUILD\\n== 20260202-120000-middle-payments.md\\n---\\nstatus: in-progress\\nbranch: feat/payments\\ntimestamp: 2026-02-02T12:00:00Z\\n---\\n## Working on: middle payments\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n== 20260303-120000-omega-release.md\\n---\\nstatus: in-progress\\nbranch: feat/omega\\ntimestamp: 2026-03-03T12:00:00Z\\n---\\n## Working on: omega release\\n\\n### Summary\\nOmega content FRAGMATCH_OMEGA_BUILD\\nunstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260101-120000-alpha-feature.md\\nunstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260202-120000-middle-payments.md\\nunstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260303-120000-omega-release.md\\nrc=0\",\"is_error\":false}]},\"parent_tool_use_id\":null,\"session_id\":\"d1147327-49e3-4371-ab54-784d135573ac\",\"uuid\":\"96853ca6-5db9-445b-ab54-ac9103ca336a\",\"timestamp\":\"2026-10-25T13:22:02.045Z\",\"tool_use_result\":{\"stdout\":\"== 20260101-120000-alpha-feature.md\\n---\\nstatus: in-progress\\nbranch: feat/alpha\\ntimestamp: 2026-01-01T12:00:00Z\\n---\\n## Working on: alpha feature\\n\\n### Summary\\nAlpha content FRAGMATCH_ALPHA_BUILD\\n== 20260202-120000-middle-payments.md\\n---\\nstatus: in-progress\\nbranch: feat/payments\\ntimestamp: 2026-02-02T12:00:00Z\\n---\\n## Working on: middle payments\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n== 20260303-120000-omega-release.md\\n---\\nstatus: in-progress\\nbranch: feat/omega\\ntimestamp: 2026-03-03T12:00:00Z\\n---\\n## Working on: omega release\\n\\n### Summary\\nOmega content FRAGMATCH_OMEGA_BUILD\\nunstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260101-120000-alpha-feature.md\\nunstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260202-120000-middle-payments.md\\nunstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-svqfPa/checkpoints/20260303-120000-omega-release.md\\nrc=0\",\"stderr\":\"\",\"interrupted\":false,\"isImage\":false,\"noOutputExpected\":false}}", + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2TPmyymz1BeaE3qngr\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"I loaded the checkpoint matching \\\"payments\\\": `20260202-120000-middle-payments.md`. It's the only one of the three saved checkpoints whose title contains that word. All three are unstamped and none come from another project, so there's no project mismatch.\\n\\n```\\nRESUMING CONTEXT\\n════════════════════════════════════════\\nTitle: middle payments\\nBranch: feat/payments\\nSaved: Feb 2, 2026, 12:00 UTC\\nDuration: (not recorded)\\nStatus: in-progress\\n════════════════════════════════════════\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n\\n### Remaining Work\\n(none recorded in the checkpoint)\\n\\n### Notes\\n(none recorded)\\n```\\n\\n**Branch:** this context was saved on `feat/payments`, but you're on `main`. You may want to switch to `feat/payments` before continuing.\\n\\nThe other two checkpoints, which I did not load:\\n- `alpha feature` (`feat/alpha`, 2026-01-01)\\n- `omega release` (`feat/omega`, 2026-03-03), the newest\\n\\nYou asked me not to use AskUserQuestion, so I skipped the follow-up prompt. The checkpoint lists no remaining work. You can continue on the payments work, see the full saved file (it's only the short content above), or stop here.\"}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":718,\"cache_read_input_tokens\":25676,\"cache_creation\":{\"ephemeral_5m_input_tokens\":718,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":7,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"d1147327-49e3-4371-ab54-784d135573ac\",\"uuid\":\"833cfc7c-5f23-4673-b45c-d7d70c0e3f35\",\"timestamp\":\"2026-10-25T13:22:06.611Z\",\"request_id\":\"req_011CgP2TPV8JNDtZyf4D3mYB\"}", + "{\"duration_api_ms\":11552,\"stop_reason\":\"end_turn\",\"session_id\":\"d1147327-49e3-4371-ab54-784d135573ac\",\"total_cost_usd\":0.1106026,\"usage\":{\"input_tokens\":8,\"cache_creation_input_tokens\":14352,\"cache_read_input_tokens\":80653,\"output_tokens\":1134,\"output_tokens_details\":{\"thinking_tokens\":86},\"server_tool_use\":{\"web_search_requests\":0,\"web_fetch_requests\":0},\"service_tier\":\"standard\",\"cache_creation\":{\"ephemeral_1h_input_tokens\":0,\"ephemeral_5m_input_tokens\":14352},\"inference_geo\":\"global\",\"iterations\":[{\"input_tokens\":2,\"output_tokens\":484,\"cache_read_input_tokens\":25676,\"cache_creation_input_tokens\":718,\"cache_creation\":{\"ephemeral_5m_input_tokens\":718,\"ephemeral_1h_input_tokens\":0},\"type\":\"message\"}],\"speed\":\"standard\",\"fallback_credit\":null},\"modelUsage\":{\"claude-opus-5-5\":{\"inputTokens\":8,\"outputTokens\":1134,\"cacheReadInputTokens\":80653,\"cacheCreationInputTokens\":14352,\"webSearchRequests\":0,\"costUSD\":0.1106026,\"contextWindow\":1000000,\"maxOutputTokens\":128000,\"thinkingTokens\":86,\"canonicalModel\":\"claude-opus-5-5\",\"provider\":\"firstParty\",\"costBasis\":\"list\"}},\"permission_denials\":[],\"terminal_reason\":\"completed\",\"fast_mode_state\":\"off\",\"fast_mode_disabled_reason\":\"sdk_opt_in_required\",\"subagent_stats\":{\"spawned\":0,\"requested\":{\"background\":0,\"foreground\":0,\"unset\":0},\"started_in_background\":0,\"max_depth\":0,\"spawned_by_subagents\":0,\"completed\":0,\"failed\":0,\"killed\":{\"parent\":0,\"user\":0,\"system\":0},\"refused\":{\"depth_limit\":0,\"concurrency_limit\":0,\"budget\":0},\"by_type\":{}},\"safety_stops\":0,\"is_error\":false,\"num_turns\":5,\"subtype\":\"success\",\"api_error_status\":null,\"result\":\"I loaded the checkpoint matching \\\"payments\\\": `20260202-120000-middle-payments.md`. It's the only one of the three saved checkpoints whose title contains that word. All three are unstamped and none come from another project, so there's no project mismatch.\\n\\n```\\nRESUMING CONTEXT\\n════════════════════════════════════════\\nTitle: middle payments\\nBranch: feat/payments\\nSaved: Feb 2, 2026, 12:00 UTC\\nDuration: (not recorded)\\nStatus: in-progress\\n════════════════════════════════════════\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n\\n### Remaining Work\\n(none recorded in the checkpoint)\\n\\n### Notes\\n(none recorded)\\n```\\n\\n**Branch:** this context was saved on `feat/payments`, but you're on `main`. You may want to switch to `feat/payments` before continuing.\\n\\nThe other two checkpoints, which I did not load:\\n- `alpha feature` (`feat/alpha`, 2026-01-01)\\n- `omega release` (`feat/omega`, 2026-03-03), the newest\\n\\nYou asked me not to use AskUserQuestion, so I skipped the follow-up prompt. The checkpoint lists no remaining work. You can continue on the payments work, see the full saved file (it's only the short content above), or stop here.\",\"ttft_ms\":1839,\"type\":\"result\",\"duration_ms\":12657,\"uuid\":\"170f9374-7a93-44aa-9ce6-c54054faf5cc\",\"ttft_stream_ms\":902,\"time_to_request_ms\":199,\"first_content_frame_ms\":902,\"queued_turn_count\":0,\"result_index\":0}" + ] + }, + { + "session_id": "8889e81b-d4bf-4eba-9ade-cd8805219972", + "ndjson": [ + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2SpsKBS3fLGqXc3Bwm\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"tool_use\",\"id\":\"toolu_01DewVZTi4xMmGQYH14JLR52\",\"name\":\"Skill\",\"input\":{\"skill\":\"context-restore\",\"args\":\"payments\"},\"caller\":{\"type\":\"direct\"}}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":6260,\"cache_read_input_tokens\":12042,\"cache_creation\":{\"ephemeral_5m_input_tokens\":6260,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":16,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"8889e81b-d4bf-4eba-9ade-cd8805219972\",\"uuid\":\"73df9278-656a-4350-9f2b-11f00fc83e23\",\"timestamp\":\"2026-10-25T13:21:55.982Z\",\"request_id\":\"req_011CgP2SpdBaxUoNcNgUqkdU\",\"wire_tool_inputs\":{\"toolu_01DewVZTi4xMmGQYH14JLR52\":{\"skill\":\"context-restore\",\"args\":\"payments\"}}}", + "{\"type\":\"user\",\"message\":{\"role\":\"user\",\"content\":[{\"type\":\"tool_result\",\"tool_use_id\":\"toolu_01DewVZTi4xMmGQYH14JLR52\",\"content\":\"Launching skill: context-restore\"}]},\"parent_tool_use_id\":null,\"session_id\":\"8889e81b-d4bf-4eba-9ade-cd8805219972\",\"uuid\":\"d884aeed-d430-4d45-93b4-29855244b2ab\",\"timestamp\":\"2026-10-25T13:21:56.013Z\",\"tool_use_result\":{\"success\":true,\"commandName\":\"context-restore\",\"allowedTools\":[\"Bash\",\"Read\",\"Glob\",\"Grep\",\"AskUserQuestion\"]}}", + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2Swzw8ptggHdz4yMNm\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"tool_use\",\"id\":\"toolu_01YASRbf3r6DvzQdp2WBoi5v\",\"name\":\"Bash\",\"input\":{\"command\":\"cd /tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T; export GSTACK_HOME=\\\"$PWD/.gstack-home\\\"; ls bin; B=./bin\\nGSTACK_STATE_ROOT=$($B/gstack-paths --get GSTACK_STATE_ROOT); echo ROOT=$GSTACK_STATE_ROOT\\nSLUG=$($B/gstack-slug --get SLUG 2>/dev/null); echo SLUG=$SLUG\\nCHECKPOINT_DIR=\\\"$GSTACK_STATE_ROOT/projects/$SLUG/checkpoints\\\"; ls -la \\\"$CHECKPOINT_DIR\\\"; find \\\"$GSTACK_HOME\\\" -name \\\"*.md\\\"\",\"description\":\"Locate saved checkpoints\"},\"caller\":{\"type\":\"direct\"}}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":6340,\"cache_read_input_tokens\":18302,\"cache_creation\":{\"ephemeral_5m_input_tokens\":6340,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":8,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"8889e81b-d4bf-4eba-9ade-cd8805219972\",\"uuid\":\"88f53b0b-92b0-433b-be2e-30a6e9c4f180\",\"timestamp\":\"2026-10-25T13:21:59.368Z\",\"request_id\":\"req_011CgP2SwkZ9Z8jUt6x2i38U\",\"wire_tool_inputs\":{\"toolu_01YASRbf3r6DvzQdp2WBoi5v\":{\"command\":\"cd /tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T; export GSTACK_HOME=\\\"$PWD/.gstack-home\\\"; ls bin; B=./bin\\nGSTACK_STATE_ROOT=$($B/gstack-paths --get GSTACK_STATE_ROOT); echo ROOT=$GSTACK_STATE_ROOT\\nSLUG=$($B/gstack-slug --get SLUG 2>/dev/null); echo SLUG=$SLUG\\nCHECKPOINT_DIR=\\\"$GSTACK_STATE_ROOT/projects/$SLUG/checkpoints\\\"; ls -la \\\"$CHECKPOINT_DIR\\\"; find \\\"$GSTACK_HOME\\\" -name \\\"*.md\\\"\",\"description\":\"Locate saved checkpoints\"}}}", + "{\"type\":\"user\",\"message\":{\"role\":\"user\",\"content\":[{\"tool_use_id\":\"toolu_01YASRbf3r6DvzQdp2WBoi5v\",\"type\":\"tool_result\",\"content\":\"gstack-config\\ngstack-learnings-log\\ngstack-learnings-search\\ngstack-paths\\ngstack-remote-identity.sh\\ngstack-repo-mode\\ngstack-slug\\ngstack-state-root.sh\\ngstack-timeline-log\\ngstack-timeline-read\\ngstack-update-check\\nROOT=/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home\\nSLUG=skill-e2e-ctx-fragment-mF7D6T\\ntotal 16\\ndrwxr-xr-x 2 user user 4096 Oct 25 13:21 .\\ndrwxr-xr-x 3 user user 60 Oct 25 13:21 ..\\n-rw-r--r-- 1 user user 157 Oct 25 13:21 20260101-120000-alpha-feature.md\\n-rw-r--r-- 1 user user 168 Oct 25 13:21 20260202-120000-middle-payments.md\\n-rw-r--r-- 1 user user 157 Oct 25 13:21 20260303-120000-omega-release.md\\n/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/projects/skill-e2e-ctx-fragment-mF7D6T/checkpoints/20260101-120000-alpha-feature.md\\n/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/projects/skill-e2e-ctx-fragment-mF7D6T/checkpoints/20260202-120000-middle-payments.md\\n/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/projects/skill-e2e-ctx-fragment-mF7D6T/checkpoints/20260303-120000-omega-release.md\",\"is_error\":false}]},\"parent_tool_use_id\":null,\"session_id\":\"8889e81b-d4bf-4eba-9ade-cd8805219972\",\"uuid\":\"d8dab3b1-3cb6-4750-a8b1-30e12536377f\",\"timestamp\":\"2026-10-25T13:21:59.932Z\",\"tool_use_result\":{\"stdout\":\"gstack-config\\ngstack-learnings-log\\ngstack-learnings-search\\ngstack-paths\\ngstack-remote-identity.sh\\ngstack-repo-mode\\ngstack-slug\\ngstack-state-root.sh\\ngstack-timeline-log\\ngstack-timeline-read\\ngstack-update-check\\nROOT=/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home\\nSLUG=skill-e2e-ctx-fragment-mF7D6T\\ntotal 16\\ndrwxr-xr-x 2 user user 4096 Oct 25 13:21 .\\ndrwxr-xr-x 3 user user 60 Oct 25 13:21 ..\\n-rw-r--r-- 1 user user 157 Oct 25 13:21 20260101-120000-alpha-feature.md\\n-rw-r--r-- 1 user user 168 Oct 25 13:21 20260202-120000-middle-payments.md\\n-rw-r--r-- 1 user user 157 Oct 25 13:21 20260303-120000-omega-release.md\\n/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/projects/skill-e2e-ctx-fragment-mF7D6T/checkpoints/20260101-120000-alpha-feature.md\\n/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/projects/skill-e2e-ctx-fragment-mF7D6T/checkpoints/20260202-120000-middle-payments.md\\n/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/projects/skill-e2e-ctx-fragment-mF7D6T/checkpoints/20260303-120000-omega-release.md\",\"stderr\":\"\",\"interrupted\":false,\"isImage\":false,\"noOutputExpected\":false,\"bashEditDiff\":{\"files\":[{\"filePath\":\"/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/slug-cache/_tmp_gstack-paid-shard-XShYTB_tmp_skill-e2e-ctx-fragment-mF7D6T\",\"hunks\":[{\"oldStart\":0,\"oldLines\":0,\"newStart\":1,\"newLines\":1,\"lines\":[\"+v2:skill-e2e-ctx-fragment-mF7D6T\"]}],\"created\":true},{\"filePath\":\"/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/slug-cache/_tmp_gstack-paid-shard-XShYTB_tmp_skill-e2e-ctx-fragment-mF7D6T_.git\",\"hunks\":[{\"oldStart\":0,\"oldLines\":0,\"newStart\":1,\"newLines\":1,\"lines\":[\"+v2:skill-e2e-ctx-fragment-mF7D6T\"]}],\"created\":true}],\"moreFiles\":0,\"changedFiles\":[\"/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/slug-cache/_tmp_gstack-paid-shard-XShYTB_tmp_skill-e2e-ctx-fragment-mF7D6T\",\"/tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/slug-cache/_tmp_gstack-paid-shard-XShYTB_tmp_skill-e2e-ctx-fragment-mF7D6T_.git\"]}}}", + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2TEimtnXFfvDRRUFde\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"tool_use\",\"id\":\"toolu_0148N1vosxsxAic6XBPYrXGQ\",\"name\":\"Bash\",\"input\":{\"command\":\"cd /tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/projects/*/checkpoints; for f in *; do echo \\\"== $f\\\"; cat \\\"$f\\\"; done\",\"description\":\"Read checkpoint files\"},\"caller\":{\"type\":\"direct\"}}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":1098,\"cache_read_input_tokens\":24642,\"cache_creation\":{\"ephemeral_5m_input_tokens\":1098,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":16,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"8889e81b-d4bf-4eba-9ade-cd8805219972\",\"uuid\":\"83ba03a6-8c2c-4fb6-8a99-168b4da4b9ab\",\"timestamp\":\"2026-10-25T13:22:01.555Z\",\"request_id\":\"req_011CgP2TEStrRVjCJmLjGxUQ\",\"wire_tool_inputs\":{\"toolu_0148N1vosxsxAic6XBPYrXGQ\":{\"command\":\"cd /tmp/gstack-paid-shard-XShYTB/tmp/skill-e2e-ctx-fragment-mF7D6T/.gstack-home/projects/*/checkpoints; for f in *; do echo \\\"== $f\\\"; cat \\\"$f\\\"; done\",\"description\":\"Read checkpoint files\"}}}", + "{\"type\":\"user\",\"message\":{\"role\":\"user\",\"content\":[{\"tool_use_id\":\"toolu_0148N1vosxsxAic6XBPYrXGQ\",\"type\":\"tool_result\",\"content\":\"== 20260101-120000-alpha-feature.md\\n---\\nstatus: in-progress\\nbranch: feat/alpha\\ntimestamp: 2026-01-01T12:00:00Z\\n---\\n## Working on: alpha feature\\n\\n### Summary\\nAlpha content FRAGMATCH_ALPHA_BUILD\\n== 20260202-120000-middle-payments.md\\n---\\nstatus: in-progress\\nbranch: feat/payments\\ntimestamp: 2026-02-02T12:00:00Z\\n---\\n## Working on: middle payments\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n== 20260303-120000-omega-release.md\\n---\\nstatus: in-progress\\nbranch: feat/omega\\ntimestamp: 2026-03-03T12:00:00Z\\n---\\n## Working on: omega release\\n\\n### Summary\\nOmega content FRAGMATCH_OMEGA_BUILD\",\"is_error\":false}]},\"parent_tool_use_id\":null,\"session_id\":\"8889e81b-d4bf-4eba-9ade-cd8805219972\",\"uuid\":\"686ae24d-a3eb-4707-ae3b-330574244652\",\"timestamp\":\"2026-10-25T13:22:01.697Z\",\"tool_use_result\":{\"stdout\":\"== 20260101-120000-alpha-feature.md\\n---\\nstatus: in-progress\\nbranch: feat/alpha\\ntimestamp: 2026-01-01T12:00:00Z\\n---\\n## Working on: alpha feature\\n\\n### Summary\\nAlpha content FRAGMATCH_ALPHA_BUILD\\n== 20260202-120000-middle-payments.md\\n---\\nstatus: in-progress\\nbranch: feat/payments\\ntimestamp: 2026-02-02T12:00:00Z\\n---\\n## Working on: middle payments\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n== 20260303-120000-omega-release.md\\n---\\nstatus: in-progress\\nbranch: feat/omega\\ntimestamp: 2026-03-03T12:00:00Z\\n---\\n## Working on: omega release\\n\\n### Summary\\nOmega content FRAGMATCH_OMEGA_BUILD\",\"stderr\":\"\",\"interrupted\":false,\"isImage\":false,\"noOutputExpected\":false}}", + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2TNGCLmG6DD1SMJXr4\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"I loaded the checkpoint matching \\\"payments\\\": `20260202-120000-middle-payments.md`. Three checkpoints were saved in this project (alpha-feature, middle-payments, omega-release), and only this one has \\\"payments\\\" in its title. Omega-release is newer, but you asked for this one by name.\\n\\n```\\nRESUMING CONTEXT\\n════════════════════════════════════════\\nTitle: middle payments\\nBranch: feat/payments\\nSaved: 2026-02-02 12:00:00 UTC\\nDuration: (not recorded)\\nStatus: in-progress\\n════════════════════════════════════════\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n\\n### Remaining Work\\n(none recorded)\\n\\n### Notes\\n(none recorded)\\n```\\n\\n- **Branch:** this context was saved on `feat/payments`, but you're on `main`. You may want to switch branches before continuing.\\n- **Project check:** the checkpoint has no project identity stamp. That's fine here, because no other project has saved checkpoints in this folder.\\n\\nI didn't change any code, and I skipped the usual follow-up question as you asked. Your options are to keep working on this context, see the full file (it's short and shown above in full), or stop here.\"}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":635,\"cache_read_input_tokens\":25740,\"cache_creation\":{\"ephemeral_5m_input_tokens\":635,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":2,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"8889e81b-d4bf-4eba-9ade-cd8805219972\",\"uuid\":\"70979650-0199-44e3-956e-0824d1de507a\",\"timestamp\":\"2026-10-25T13:22:06.359Z\",\"request_id\":\"req_011CgP2TN24xUJz1E2rmAPb1\"}", + "{\"duration_api_ms\":11304,\"stop_reason\":\"end_turn\",\"session_id\":\"8889e81b-d4bf-4eba-9ade-cd8805219972\",\"total_cost_usd\":0.10918220000000001,\"usage\":{\"input_tokens\":8,\"cache_creation_input_tokens\":14333,\"cache_read_input_tokens\":80726,\"output_tokens\":1067,\"output_tokens_details\":{\"thinking_tokens\":71},\"server_tool_use\":{\"web_search_requests\":0,\"web_fetch_requests\":0},\"service_tier\":\"standard\",\"cache_creation\":{\"ephemeral_1h_input_tokens\":0,\"ephemeral_5m_input_tokens\":14333},\"inference_geo\":\"global\",\"iterations\":[{\"input_tokens\":2,\"output_tokens\":439,\"cache_read_input_tokens\":25740,\"cache_creation_input_tokens\":635,\"cache_creation\":{\"ephemeral_5m_input_tokens\":635,\"ephemeral_1h_input_tokens\":0},\"type\":\"message\"}],\"speed\":\"standard\",\"fallback_credit\":null},\"modelUsage\":{\"claude-opus-5-5\":{\"inputTokens\":8,\"outputTokens\":1067,\"cacheReadInputTokens\":80726,\"cacheCreationInputTokens\":14333,\"webSearchRequests\":0,\"costUSD\":0.10918220000000001,\"contextWindow\":1000000,\"maxOutputTokens\":128000,\"thinkingTokens\":71,\"canonicalModel\":\"claude-opus-5-5\",\"provider\":\"firstParty\",\"costBasis\":\"list\"}},\"permission_denials\":[],\"terminal_reason\":\"completed\",\"fast_mode_state\":\"off\",\"fast_mode_disabled_reason\":\"sdk_opt_in_required\",\"subagent_stats\":{\"spawned\":0,\"requested\":{\"background\":0,\"foreground\":0,\"unset\":0},\"started_in_background\":0,\"max_depth\":0,\"spawned_by_subagents\":0,\"completed\":0,\"failed\":0,\"killed\":{\"parent\":0,\"user\":0,\"system\":0},\"refused\":{\"depth_limit\":0,\"concurrency_limit\":0,\"budget\":0},\"by_type\":{}},\"safety_stops\":0,\"is_error\":false,\"num_turns\":5,\"subtype\":\"success\",\"api_error_status\":null,\"result\":\"I loaded the checkpoint matching \\\"payments\\\": `20260202-120000-middle-payments.md`. Three checkpoints were saved in this project (alpha-feature, middle-payments, omega-release), and only this one has \\\"payments\\\" in its title. Omega-release is newer, but you asked for this one by name.\\n\\n```\\nRESUMING CONTEXT\\n════════════════════════════════════════\\nTitle: middle payments\\nBranch: feat/payments\\nSaved: 2026-02-02 12:00:00 UTC\\nDuration: (not recorded)\\nStatus: in-progress\\n════════════════════════════════════════\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n\\n### Remaining Work\\n(none recorded)\\n\\n### Notes\\n(none recorded)\\n```\\n\\n- **Branch:** this context was saved on `feat/payments`, but you're on `main`. You may want to switch branches before continuing.\\n- **Project check:** the checkpoint has no project identity stamp. That's fine here, because no other project has saved checkpoints in this folder.\\n\\nI didn't change any code, and I skipped the usual follow-up question as you asked. Your options are to keep working on this context, see the full file (it's short and shown above in full), or stop here.\",\"ttft_ms\":1797,\"type\":\"result\",\"duration_ms\":12189,\"uuid\":\"3003a3de-20c9-4a22-8387-f4c23597d9c3\",\"ttft_stream_ms\":873,\"time_to_request_ms\":179,\"first_content_frame_ms\":874,\"queued_turn_count\":0,\"result_index\":0}" + ] + }, + { + "session_id": "6d684089-4cb5-4016-bc86-203a9a8cdab3", + "ndjson": [ + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2SoyF1qZULHrLjmM3K\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"tool_use\",\"id\":\"toolu_01X9nJt9XVFgAKfbeHTY7jhD\",\"name\":\"Skill\",\"input\":{\"skill\":\"context-restore\",\"args\":\"payments\"},\"caller\":{\"type\":\"direct\"}}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":6260,\"cache_read_input_tokens\":12042,\"cache_creation\":{\"ephemeral_5m_input_tokens\":6260,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":16,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"6d684089-4cb5-4016-bc86-203a9a8cdab3\",\"uuid\":\"7756244d-e82e-4aeb-9437-732412ccb96f\",\"timestamp\":\"2026-10-25T13:21:56.218Z\",\"request_id\":\"req_011CgP2Soi7ME2b1kXN8NCRG\",\"wire_tool_inputs\":{\"toolu_01X9nJt9XVFgAKfbeHTY7jhD\":{\"skill\":\"context-restore\",\"args\":\"payments\"}}}", + "{\"type\":\"user\",\"message\":{\"role\":\"user\",\"content\":[{\"type\":\"tool_result\",\"tool_use_id\":\"toolu_01X9nJt9XVFgAKfbeHTY7jhD\",\"content\":\"Launching skill: context-restore\"}]},\"parent_tool_use_id\":null,\"session_id\":\"6d684089-4cb5-4016-bc86-203a9a8cdab3\",\"uuid\":\"376841c6-0fe2-417f-8da9-5d79c52f0694\",\"timestamp\":\"2026-10-25T13:21:56.247Z\",\"tool_use_result\":{\"success\":true,\"commandName\":\"context-restore\",\"allowedTools\":[\"Bash\",\"Read\",\"Glob\",\"Grep\",\"AskUserQuestion\"]}}", + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2Sy7Apg39RghKe7nhs\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"tool_use\",\"id\":\"toolu_016oZY36mo3ibs1cL3rNt82T\",\"name\":\"Bash\",\"input\":{\"command\":\"cd /tmp/gstack-paid-shard-V2G2le/tmp/skill-e2e-ctx-fragment-GZoJH5; ls bin; export GSTACK_HOME=\\\"$PWD/.gstack-home\\\"; B=./bin\\nGSTACK_STATE_ROOT=$($B/gstack-paths --get GSTACK_STATE_ROOT 2>&1); echo \\\"ROOT=$GSTACK_STATE_ROOT\\\"\\nSLUG=$($B/gstack-slug --get SLUG 2>&1); echo \\\"SLUG=$SLUG\\\"\\nfind .gstack-home -name \\\"*.md\\\" | sort -r\",\"description\":\"Locate gstack state and checkpoints\"},\"caller\":{\"type\":\"direct\"}}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":6340,\"cache_read_input_tokens\":18302,\"cache_creation\":{\"ephemeral_5m_input_tokens\":6340,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":8,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"6d684089-4cb5-4016-bc86-203a9a8cdab3\",\"uuid\":\"46203839-ca20-4fe1-a98b-d965e965f665\",\"timestamp\":\"2026-10-25T13:22:00.005Z\",\"request_id\":\"req_011CgP2SxnaBEsoXQB2CY6kN\",\"wire_tool_inputs\":{\"toolu_016oZY36mo3ibs1cL3rNt82T\":{\"command\":\"cd /tmp/gstack-paid-shard-V2G2le/tmp/skill-e2e-ctx-fragment-GZoJH5; ls bin; export GSTACK_HOME=\\\"$PWD/.gstack-home\\\"; B=./bin\\nGSTACK_STATE_ROOT=$($B/gstack-paths --get GSTACK_STATE_ROOT 2>&1); echo \\\"ROOT=$GSTACK_STATE_ROOT\\\"\\nSLUG=$($B/gstack-slug --get SLUG 2>&1); echo \\\"SLUG=$SLUG\\\"\\nfind .gstack-home -name \\\"*.md\\\" | sort -r\",\"description\":\"Locate gstack state and checkpoints\"}}}", + "{\"type\":\"user\",\"message\":{\"role\":\"user\",\"content\":[{\"tool_use_id\":\"toolu_016oZY36mo3ibs1cL3rNt82T\",\"type\":\"tool_result\",\"content\":\"gstack-config\\ngstack-learnings-log\\ngstack-learnings-search\\ngstack-paths\\ngstack-remote-identity.sh\\ngstack-repo-mode\\ngstack-slug\\ngstack-state-root.sh\\ngstack-timeline-log\\ngstack-timeline-read\\ngstack-update-check\\nROOT=/tmp/gstack-paid-shard-V2G2le/tmp/skill-e2e-ctx-fragment-GZoJH5/.gstack-home\\nSLUG=skill-e2e-ctx-fragment-GZoJH5\\n.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260303-120000-omega-release.md\\n.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260202-120000-middle-payments.md\\n.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260101-120000-alpha-feature.md\",\"is_error\":false}]},\"parent_tool_use_id\":null,\"session_id\":\"6d684089-4cb5-4016-bc86-203a9a8cdab3\",\"uuid\":\"d7748b04-b48b-498b-89e5-ce4df4e6d58c\",\"timestamp\":\"2026-10-25T13:22:00.921Z\",\"tool_use_result\":{\"stdout\":\"gstack-config\\ngstack-learnings-log\\ngstack-learnings-search\\ngstack-paths\\ngstack-remote-identity.sh\\ngstack-repo-mode\\ngstack-slug\\ngstack-state-root.sh\\ngstack-timeline-log\\ngstack-timeline-read\\ngstack-update-check\\nROOT=/tmp/gstack-paid-shard-V2G2le/tmp/skill-e2e-ctx-fragment-GZoJH5/.gstack-home\\nSLUG=skill-e2e-ctx-fragment-GZoJH5\\n.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260303-120000-omega-release.md\\n.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260202-120000-middle-payments.md\\n.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260101-120000-alpha-feature.md\",\"stderr\":\"\",\"interrupted\":false,\"isImage\":false,\"noOutputExpected\":false,\"bashEditDiff\":{\"files\":[{\"filePath\":\"/tmp/gstack-paid-shard-V2G2le/tmp/skill-e2e-ctx-fragment-GZoJH5/.gstack-home/slug-cache/_tmp_gstack-paid-shard-V2G2le_tmp_skill-e2e-ctx-fragment-GZoJH5\",\"hunks\":[{\"oldStart\":0,\"oldLines\":0,\"newStart\":1,\"newLines\":1,\"lines\":[\"+v2:skill-e2e-ctx-fragment-GZoJH5\"]}],\"created\":true},{\"filePath\":\"/tmp/gstack-paid-shard-V2G2le/tmp/skill-e2e-ctx-fragment-GZoJH5/.gstack-home/slug-cache/_tmp_gstack-paid-shard-V2G2le_tmp_skill-e2e-ctx-fragment-GZoJH5_.git\",\"hunks\":[{\"oldStart\":0,\"oldLines\":0,\"newStart\":1,\"newLines\":1,\"lines\":[\"+v2:skill-e2e-ctx-fragment-GZoJH5\"]}],\"created\":true}],\"moreFiles\":0,\"changedFiles\":[\"/tmp/gstack-paid-shard-V2G2le/tmp/skill-e2e-ctx-fragment-GZoJH5/.gstack-home/slug-cache/_tmp_gstack-paid-shard-V2G2le_tmp_skill-e2e-ctx-fragment-GZoJH5\",\"/tmp/gstack-paid-shard-V2G2le/tmp/skill-e2e-ctx-fragment-GZoJH5/.gstack-home/slug-cache/_tmp_gstack-paid-shard-V2G2le_tmp_skill-e2e-ctx-fragment-GZoJH5_.git\"]}}}", + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2TJxWctw7y2Vz5qBJH\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"tool_use\",\"id\":\"toolu_01FMDVzkCc6y3vknp52LsMBF\",\"name\":\"Bash\",\"input\":{\"command\":\"cd /tmp/gstack-paid-shard-V2G2le/tmp/skill-e2e-ctx-fragment-GZoJH5; export GSTACK_HOME=\\\"$PWD/.gstack-home\\\"; D=.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints\\nls $D/*.md | ./bin/gstack-slug --classify-checkpoints 2>&1; echo ---; cat $D/20260202-120000-middle-payments.md; echo ---; head -12 $D/20260303-120000-omega-release.md $D/20260101-120000-alpha-feature.md\",\"description\":\"Classify checkpoints and read payments file\"},\"caller\":{\"type\":\"direct\"}}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":718,\"cache_read_input_tokens\":24642,\"cache_creation\":{\"ephemeral_5m_input_tokens\":718,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":16,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"6d684089-4cb5-4016-bc86-203a9a8cdab3\",\"uuid\":\"bf17bfed-5ea2-42c9-8b0d-df4adf7e20ff\",\"timestamp\":\"2026-10-25T13:22:03.980Z\",\"request_id\":\"req_011CgP2TJg9vKd8HjaMC2abX\",\"wire_tool_inputs\":{\"toolu_01FMDVzkCc6y3vknp52LsMBF\":{\"command\":\"cd /tmp/gstack-paid-shard-V2G2le/tmp/skill-e2e-ctx-fragment-GZoJH5; export GSTACK_HOME=\\\"$PWD/.gstack-home\\\"; D=.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints\\nls $D/*.md | ./bin/gstack-slug --classify-checkpoints 2>&1; echo ---; cat $D/20260202-120000-middle-payments.md; echo ---; head -12 $D/20260303-120000-omega-release.md $D/20260101-120000-alpha-feature.md\",\"description\":\"Classify checkpoints and read payments file\"}}}", + "{\"type\":\"user\",\"message\":{\"role\":\"user\",\"content\":[{\"tool_use_id\":\"toolu_01FMDVzkCc6y3vknp52LsMBF\",\"type\":\"tool_result\",\"content\":\"unstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260101-120000-alpha-feature.md\\nunstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260202-120000-middle-payments.md\\nunstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260303-120000-omega-release.md\\n---\\n---\\nstatus: in-progress\\nbranch: feat/payments\\ntimestamp: 2026-02-02T12:00:00Z\\n---\\n## Working on: middle payments\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n---\\n==> .gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260303-120000-omega-release.md <==\\n---\\nstatus: in-progress\\nbranch: feat/omega\\ntimestamp: 2026-03-03T12:00:00Z\\n---\\n## Working on: omega release\\n\\n### Summary\\nOmega content FRAGMATCH_OMEGA_BUILD\\n\\n==> .gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260101-120000-alpha-feature.md <==\\n---\\nstatus: in-progress\\nbranch: feat/alpha\\ntimestamp: 2026-01-01T12:00:00Z\\n---\\n## Working on: alpha feature\\n\\n### Summary\\nAlpha content FRAGMATCH_ALPHA_BUILD\",\"is_error\":false}]},\"parent_tool_use_id\":null,\"session_id\":\"6d684089-4cb5-4016-bc86-203a9a8cdab3\",\"uuid\":\"21ec78ab-abbc-4ea3-8ba6-544d6b05fc91\",\"timestamp\":\"2026-10-25T13:22:04.223Z\",\"tool_use_result\":{\"stdout\":\"unstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260101-120000-alpha-feature.md\\nunstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260202-120000-middle-payments.md\\nunstamped\\t.gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260303-120000-omega-release.md\\n---\\n---\\nstatus: in-progress\\nbranch: feat/payments\\ntimestamp: 2026-02-02T12:00:00Z\\n---\\n## Working on: middle payments\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n---\\n==> .gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260303-120000-omega-release.md <==\\n---\\nstatus: in-progress\\nbranch: feat/omega\\ntimestamp: 2026-03-03T12:00:00Z\\n---\\n## Working on: omega release\\n\\n### Summary\\nOmega content FRAGMATCH_OMEGA_BUILD\\n\\n==> .gstack-home/projects/skill-e2e-ctx-fragment-GZoJH5/checkpoints/20260101-120000-alpha-feature.md <==\\n---\\nstatus: in-progress\\nbranch: feat/alpha\\ntimestamp: 2026-01-01T12:00:00Z\\n---\\n## Working on: alpha feature\\n\\n### Summary\\nAlpha content FRAGMATCH_ALPHA_BUILD\",\"stderr\":\"\",\"interrupted\":false,\"isImage\":false,\"noOutputExpected\":false}}", + "{\"type\":\"assistant\",\"message\":{\"model\":\"claude-opus-5-5\",\"id\":\"msg_011CgP2TZ93aTmrLHgHkg2Je\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"I loaded the checkpoint `20260202-120000-middle-payments.md`. It's the only one of the three saved checkpoints whose title contains \\\"payments\\\". It isn't the newest: `omega-release` (2026-03-03) is, so I chose this one because its title matched, not because it was the latest.\\n\\n```\\nRESUMING CONTEXT\\n════════════════════════════════════════\\nTitle: middle payments\\nBranch: feat/payments\\nSaved: Feb 2, 2026, 12:00 UTC\\nDuration: not recorded\\nStatus: in-progress\\n════════════════════════════════════════\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n\\n### Remaining Work\\nNone recorded in the checkpoint.\\n\\n### Notes\\nNone recorded.\\n```\\n\\n- **Branch:** this was saved on `feat/payments`, but you're on `main`. You may want to switch branches before continuing.\\n- **Project check:** none of the checkpoints have a project identity stamp, but none belong to another project, so this one counts as this project's.\\n- **Other checkpoints:** `alpha feature` on `feat/alpha` (2026-01-01) and `omega release` on `feat/omega` (2026-03-03).\\n\\nYou asked me not to use AskUserQuestion, so I skipped the skill's usual \\\"what next?\\\" prompt. I made no code changes; restoring only reads the saved files.\"}],\"container\":null,\"stop_reason\":null,\"stop_sequence\":null,\"stop_details\":null,\"usage\":{\"input_tokens\":2,\"cache_creation_input_tokens\":859,\"cache_read_input_tokens\":25360,\"cache_creation\":{\"ephemeral_5m_input_tokens\":859,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":8,\"service_tier\":\"standard\",\"inference_geo\":\"global\",\"speed\":\"standard\"},\"input_transformations\":[],\"diagnostics\":null,\"context_management\":null},\"parent_tool_use_id\":null,\"session_id\":\"6d684089-4cb5-4016-bc86-203a9a8cdab3\",\"uuid\":\"e8864f23-50e9-4a59-ab4a-b984590d2559\",\"timestamp\":\"2026-10-25T13:22:09.545Z\",\"request_id\":\"req_011CgP2TYqh2XjMw3Nc9cuQu\"}", + "{\"duration_api_ms\":14272,\"stop_reason\":\"end_turn\",\"session_id\":\"6d684089-4cb5-4016-bc86-203a9a8cdab3\",\"total_cost_usd\":0.1112262,\"usage\":{\"input_tokens\":8,\"cache_creation_input_tokens\":14177,\"cache_read_input_tokens\":80346,\"output_tokens\":1212,\"output_tokens_details\":{\"thinking_tokens\":88},\"server_tool_use\":{\"web_search_requests\":0,\"web_fetch_requests\":0},\"service_tier\":\"standard\",\"cache_creation\":{\"ephemeral_1h_input_tokens\":0,\"ephemeral_5m_input_tokens\":14177},\"inference_geo\":\"global\",\"iterations\":[{\"input_tokens\":2,\"output_tokens\":501,\"cache_read_input_tokens\":25360,\"cache_creation_input_tokens\":859,\"cache_creation\":{\"ephemeral_5m_input_tokens\":859,\"ephemeral_1h_input_tokens\":0},\"type\":\"message\"}],\"speed\":\"standard\",\"fallback_credit\":null},\"modelUsage\":{\"claude-opus-5-5\":{\"inputTokens\":8,\"outputTokens\":1212,\"cacheReadInputTokens\":80346,\"cacheCreationInputTokens\":14177,\"webSearchRequests\":0,\"costUSD\":0.1112262,\"contextWindow\":1000000,\"maxOutputTokens\":128000,\"thinkingTokens\":88,\"canonicalModel\":\"claude-opus-5-5\",\"provider\":\"firstParty\",\"costBasis\":\"list\"}},\"permission_denials\":[],\"terminal_reason\":\"completed\",\"fast_mode_state\":\"off\",\"fast_mode_disabled_reason\":\"sdk_opt_in_required\",\"subagent_stats\":{\"spawned\":0,\"requested\":{\"background\":0,\"foreground\":0,\"unset\":0},\"started_in_background\":0,\"max_depth\":0,\"spawned_by_subagents\":0,\"completed\":0,\"failed\":0,\"killed\":{\"parent\":0,\"user\":0,\"system\":0},\"refused\":{\"depth_limit\":0,\"concurrency_limit\":0,\"budget\":0},\"by_type\":{}},\"safety_stops\":0,\"is_error\":false,\"num_turns\":5,\"subtype\":\"success\",\"api_error_status\":null,\"result\":\"I loaded the checkpoint `20260202-120000-middle-payments.md`. It's the only one of the three saved checkpoints whose title contains \\\"payments\\\". It isn't the newest: `omega-release` (2026-03-03) is, so I chose this one because its title matched, not because it was the latest.\\n\\n```\\nRESUMING CONTEXT\\n════════════════════════════════════════\\nTitle: middle payments\\nBranch: feat/payments\\nSaved: Feb 2, 2026, 12:00 UTC\\nDuration: not recorded\\nStatus: in-progress\\n════════════════════════════════════════\\n\\n### Summary\\nPayments content FRAGMATCH_PAYMENTS_BUILD\\n\\n### Remaining Work\\nNone recorded in the checkpoint.\\n\\n### Notes\\nNone recorded.\\n```\\n\\n- **Branch:** this was saved on `feat/payments`, but you're on `main`. You may want to switch branches before continuing.\\n- **Project check:** none of the checkpoints have a project identity stamp, but none belong to another project, so this one counts as this project's.\\n- **Other checkpoints:** `alpha feature` on `feat/alpha` (2026-01-01) and `omega release` on `feat/omega` (2026-03-03).\\n\\nYou asked me not to use AskUserQuestion, so I skipped the skill's usual \\\"what next?\\\" prompt. I made no code changes; restoring only reads the saved files.\",\"ttft_ms\":2268,\"type\":\"result\",\"duration_ms\":15609,\"uuid\":\"39ae4b99-23c4-4444-b94e-4cf48961a993\",\"ttft_stream_ms\":1914,\"time_to_request_ms\":192,\"first_content_frame_ms\":1915,\"queued_turn_count\":0,\"result_index\":0}" + ] + } + ] +} diff --git a/test/gstack-ff-update.test.ts b/test/gstack-ff-update.test.ts index 739e5385b..d6910abf4 100644 --- a/test/gstack-ff-update.test.ts +++ b/test/gstack-ff-update.test.ts @@ -336,10 +336,12 @@ describe.skipIf(process.platform === 'win32')('gstack-ff-update: the install-sco test('the heartbeat and its sleeper are reaped when the bin exits', () => { const fx = makeFixture(); release(fx, { 'notes.txt': 'top UPSTREAM\nmiddle\nbottom\n' }); - const before = execFileSync('bash', ['-c', 'pgrep -f "^sleep 300$" | wc -l'], { encoding: 'utf8', timeout: 30_000 }).trim(); - expect(ff(fx).status).toBe(0); - const after = execFileSync('bash', ['-c', 'sleep 0.3; pgrep -f "^sleep 300$" | wc -l'], { encoding: 'utf8', timeout: 30_000 }).trim(); - expect(after).toBe(before); + const beat = String(3000 + (process.pid % 997)); + const sleepers = (delay = '') => execFileSync('bash', ['-c', `${delay}pgrep -f "^sleep ${beat}$" | wc -l`], { encoding: 'utf8', timeout: 30_000 }).trim(); + const before = sleepers(); + expect(before).toBe('0'); + expect(ff(fx, 'origin/main', { GSTACK_FF_HEARTBEAT_SECONDS: beat }).status).toBe(0); + expect(sleepers('sleep 0.3; ')).toBe(before); }); }); diff --git a/test/gstack-skill-start.test.ts b/test/gstack-skill-start.test.ts index 6669cb711..63f1f9999 100644 --- a/test/gstack-skill-start.test.ts +++ b/test/gstack-skill-start.test.ts @@ -932,7 +932,7 @@ describe('gstack-skill-start install admission', () => { const elsewhere = verdict(gh, {}, path.join(gh, 'named.json')); fs.rmSync(path.join(gh, 'install-verdicts'), { recursive: true, force: true }); expect(start(gh, 'autoplan', { ...capy, GSTACK_INSTALL_VERDICT: elsewhere }).status).toBe(0); - }); + }, 20_000); test('an installed runtime root follows its .source-path to the source checkout for the verdict and HEAD', () => { const gh = scratch(); diff --git a/test/helpers/context-restore-fragment.ts b/test/helpers/context-restore-fragment.ts new file mode 100644 index 000000000..199f4f6e3 --- /dev/null +++ b/test/helpers/context-restore-fragment.ts @@ -0,0 +1,34 @@ +/** + * Verdict for the context-restore-fragment-match E2E: which checkpoint did the + * agent restore? The evidence is the agent's own report (top-level assistant + * text plus the final result), never tool output. Restore has to inspect the + * candidates to find the one whose title matches, so every checkpoint body can + * appear in Bash or Read output without being the restored context. + */ +export interface FragmentRestoreRun { + output?: string; + transcript?: any[]; +} + +export function agentReport(result: FragmentRestoreRun): string { + const parts: string[] = []; + for (const event of result.transcript || []) { + if (event?.type !== 'assistant' || event.parent_tool_use_id) continue; + for (const block of event.message?.content || []) { + if (block?.type === 'text' && typeof block.text === 'string') parts.push(block.text); + } + } + if (result.output) parts.push(result.output); + return parts.join('\n'); +} + +export function fragmentRestoreVerdict( + result: FragmentRestoreRun, + markers: { restored: string; others: string[] }, +): { loadedTarget: boolean; didNotLoadOthers: boolean } { + const report = agentReport(result); + return { + loadedTarget: report.includes(markers.restored), + didNotLoadOthers: markers.others.every((marker) => !report.includes(marker)), + }; +} diff --git a/test/helpers/free-fixtures-data.ts b/test/helpers/free-fixtures-data.ts index d8b7c8440..d7bd733fa 100644 --- a/test/helpers/free-fixtures-data.ts +++ b/test/helpers/free-fixtures-data.ts @@ -60,6 +60,7 @@ export const FREE_FIXTURES: Readonly> = { 'test/fixtures/codex-sandbox/**': ['test/codex-eval-recording.test.ts', 'test/codex-model-probe.test.ts', 'test/outside-review-result.test.ts', 'test/outside-voice-invocation.test.ts'], 'test/fixtures/context-bill/**': ['test/context-bill.test.ts', 'test/context-budget-ratchet.test.ts'], 'test/fixtures/context-budget.json': ['test/helpers/capture-context-budget.ts', 'test/test-value-bar.test.ts'], + 'test/fixtures/context-restore-fragment-match-captures.json': ['test/context-restore-fragment-verdict.test.ts'], 'test/fixtures/coverage-audit-cd-chain.json': ['test/coverage-audit-evidence.test.ts'], 'test/fixtures/coverage-audit-parallel-column.json': ['test/coverage-audit-evidence.test.ts'], 'test/fixtures/coverage-audit-separator-collision.json': ['test/coverage-audit-evidence.test.ts'], diff --git a/test/helpers/llm-judge.ts b/test/helpers/llm-judge.ts index d0bc6bbbe..a3177665b 100644 --- a/test/helpers/llm-judge.ts +++ b/test/helpers/llm-judge.ts @@ -187,6 +187,13 @@ export async function callJudge( .filter(block => block.type === 'text') .map(block => block.text) .join('\n'); + const parseJudgeText = (json: string): T => { + try { + return JSON.parse(json) as T; + } catch (error) { + throw new SyntaxError(`Judge response is not valid JSON (stop_reason=${response.stop_reason}, ${json.length} chars; full text in llm-judge-response-parse-error): ${error instanceof Error ? error.message : String(error)}`); + } + }; try { if (response.stop_reason === 'max_tokens') { throw new Error(`Judge response truncated at max_tokens=${maxTokens} (model=${resolvedModel})`); @@ -194,11 +201,11 @@ export async function callJudge( if (response.stop_reason === 'refusal') throw new JudgeRefusalError(response); if (opts?.jsonSchema !== undefined) { if (response.stop_reason !== 'end_turn') throw new Error(`Structured judge did not complete: stop_reason=${response.stop_reason}`); - return JSON.parse(text) as T; + return parseJudgeText(text); } const jsonMatch = text.match(/\{[\s\S]*\}/); if (!jsonMatch) throw new Error(`Judge returned non-JSON: ${text.slice(0, 200)}`); - return JSON.parse(jsonMatch[0]) as T; + return parseJudgeText(jsonMatch[0]); } catch (error) { // The canonical full stderr spool retains this public response even when // parsing fails before a caller can record a judgment. Never copy content diff --git a/test/helpers/touchfiles-data.ts b/test/helpers/touchfiles-data.ts index 1e9b5f62d..8c8856d66 100644 --- a/test/helpers/touchfiles-data.ts +++ b/test/helpers/touchfiles-data.ts @@ -682,15 +682,15 @@ export const E2E_TOUCHFILES: Record = { // test/skill-e2e-context-skills.test.ts. These are periodic-tier because // each one spawns claude -p and costs ~$0.20-$0.40. Collectively they // verify the thing the /checkpoint → /context-save rename was for. - 'context-save-routing': [ 'context-save/**', 'scripts/resolvers/preamble.ts', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts'], - 'context-save-then-restore-roundtrip': ['bin/gstack-state-root.sh', 'lib/state-root.ts', 'context-save/**', 'context-restore/**', 'bin/gstack-slug', 'bin/gstack-remote-identity.sh', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts'], - 'context-restore-fragment-match': [ 'context-restore/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts'], - 'context-restore-provenance-order': ['context-save/**', 'context-restore/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts'], - 'context-restore-empty-state': [ 'context-restore/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts'], - 'context-restore-list-delegates': [ 'context-restore/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts'], - 'context-restore-legacy-compat': [ 'context-restore/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts'], - 'context-save-list-current-branch': [ 'context-save/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts'], - 'context-save-list-all-branches': [ 'context-save/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts'], + 'context-save-routing': [ 'context-save/**', 'scripts/resolvers/preamble.ts', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts', 'test/helpers/context-restore-fragment.ts'], + 'context-save-then-restore-roundtrip': ['bin/gstack-state-root.sh', 'lib/state-root.ts', 'context-save/**', 'context-restore/**', 'bin/gstack-slug', 'bin/gstack-remote-identity.sh', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts', 'test/helpers/context-restore-fragment.ts'], + 'context-restore-fragment-match': [ 'context-restore/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts', 'test/helpers/context-restore-fragment.ts'], + 'context-restore-provenance-order': ['context-save/**', 'context-restore/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts', 'test/helpers/context-restore-fragment.ts'], + 'context-restore-empty-state': [ 'context-restore/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts', 'test/helpers/context-restore-fragment.ts'], + 'context-restore-list-delegates': [ 'context-restore/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts', 'test/helpers/context-restore-fragment.ts'], + 'context-restore-legacy-compat': [ 'context-restore/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts', 'test/helpers/context-restore-fragment.ts'], + 'context-save-list-current-branch': [ 'context-save/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts', 'test/helpers/context-restore-fragment.ts'], + 'context-save-list-all-branches': [ 'context-save/**', 'test/skill-e2e-context-skills.test.ts', 'bin/gstack-skill-start', 'test/helpers/shipped-skill-routing.ts', 'test/helpers/context-restore-fragment.ts'], // Document-release 'document-release': [ 'test/helpers/outside-voice-evidence.ts', 'bin/gstack-version-bump', 'document-release/**', 'test/skill-e2e-workflow.test.ts', 'test/fixtures/coverage-audit-fixture.ts', 'test/helpers/coverage-audit.ts', 'test/helpers/coverage-audit-evidence.ts', 'test/helpers/office-hours-attempt.ts'], diff --git a/test/llm-judge-frontier.test.ts b/test/llm-judge-frontier.test.ts index 5e57344a6..e326ff0d0 100644 --- a/test/llm-judge-frontier.test.ts +++ b/test/llm-judge-frontier.test.ts @@ -218,6 +218,22 @@ describe('frontier Claude judge compatibility', () => { expect(create).toHaveBeenCalledTimes(1); }); + // Train-4 gate 40173759210 slice 4 attempt 1: Sonnet's free-form cookie workflow judgment left the quotes around "one `--domain` per run" unescaped. + const COOKIE_GATE_UNESCAPED_REPLY = "{\"clarity\": 4, \"completeness\": 4, \"actionability\": 4, \"reasoning\": \"The workflow is well ordered: Aside check, then consent questions before any `$B` command, then daemon prep, import, and an honest three-line report. It states the boundaries precisely: profile directory vs display name, `--all` exclusions, Chromium-only reset with its non-rollback caveat, and the rule that copied cookies, counts, and HTTP 200 never prove sign-in. The error and failure-reason tables cover recovery and platform limits. Gaps and ambiguities remain. (1) Step 3.1 sets env vars and runs `$B stop` before `$B status` and `goto`, which is workable but slightly awkward. (2) Step 2 says to ask everything before any `$B` command, yet it also needs the Aside probe, which comes earlier in step 1 and so is an implicit exception. (3) It is unclear how the picker or direct import maps the receipt's states to the three report lines, such as `Imported: ` for partial or failed results, or the storage-reset line when the picker is used. (4) The `--all` flow is not given a command example or a step. (5) For the multiple-domain case the instruction is \"one `--domain` per run\", but it does not say how to handle the target tab or the receipt across runs. (6) `decryption_failed` says to close the source browser, which sits in tension with the Chrome 136+ note, though it is not contradictory. (7) The `handoff` path requires a display, and the fallback when none is available is not defined beyond 'offer only with a display'. (8) The Windows example's `$B reload` without verification is fine, but the unverified case is not tied to the report. Overall it is strong with a few minor missing decisions.\"}"; + + test('the captured cookie judge reply fails closed and names the judge output, stop reason and retained text', async () => { + create.mockResolvedValue({ id: 'msg_011CgNbBNCNcZ2C6WJDCmjgU', model: 'claude-sonnet-5-5', stop_reason: 'end_turn', + content: [{ type: 'text', text: COOKIE_GATE_UNESCAPED_REPLY }], usage: { input_tokens: 4521, output_tokens: 572 } } as never); + let failure: unknown; + try { await callJudge('score the workflow', 'claude-sonnet-5-5'); } catch (error) { failure = error; } + expect(failure).toBeInstanceOf(SyntaxError); + expect((failure as Error).message).toStartWith(`Judge response is not valid JSON (stop_reason=end_turn, ${COOKIE_GATE_UNESCAPED_REPLY.length} chars; full text in llm-judge-response-parse-error): `); + expect(create).toHaveBeenCalledTimes(1); + expect(diagnostics).toHaveBeenCalledTimes(1); + expect(JSON.parse(diagnostics.mock.calls[0][0])).toMatchObject({ type: 'llm-judge-response-parse-error', stopReason: 'end_turn', + textBlocks: [COOKIE_GATE_UNESCAPED_REPLY], error: { name: 'SyntaxError', message: (failure as Error).message } }); + }); + test('a weak structured score still fails the substance bar', async () => { create.mockResolvedValue({ stop_reason: 'end_turn', content: [{ type: 'text', text: JSON.stringify({ reason_substance: 1, reasoning: 'Boilerplate.' }) }] } as never); diff --git a/test/plan-tune.test.ts b/test/plan-tune.test.ts index a615e95fe..413ed07f9 100644 --- a/test/plan-tune.test.ts +++ b/test/plan-tune.test.ts @@ -33,6 +33,7 @@ import { validateRegistrySignalKeys, newDimensionTotals, normalizeToDimensionValue, + resolveOptionKey, ALL_DIMENSIONS, } from '../scripts/psychographic-signals'; import { @@ -290,6 +291,125 @@ describe('psychographic signal map', () => { }); }); +// ----------------------------------------------------------------------- +// #2623 — chosen UI label → registered option key at capture time +// ----------------------------------------------------------------------- + +describe('option label → registered key (#2623)', () => { + const words = (key: string) => key.replace(/-/g, ' '); + const title = (s: string) => s.charAt(0).toUpperCase() + s.slice(1); + + test('every registered option resolves from its own words, as a bare label and as an AUQ label', () => { + const misses: string[] = []; + for (const q of Object.values(QUESTIONS as Record)) { + for (const key of (q.options ?? []) as string[]) { + for (const label of [key, words(key), `A) ${title(words(key))} (recommended)`, `B. ${title(words(key))}`]) { + if (resolveOptionKey(q.id, label) !== key) misses.push(`${q.id}: ${JSON.stringify(label)} → ${resolveOptionKey(q.id, label)}`); + } + } + } + expect(misses).toEqual([]); + }); + + test('common phrasings map to the question\'s own key', () => { + const cases: Array<[string, string, string]> = [ + ['plan-eng-review-arch-finding', 'A) Fix it now (recommended)', 'fix-now'], + ['plan-eng-review-arch-finding', 'Fix later', 'defer'], + ['plan-eng-review-arch-finding', 'B) Defer to TODOS.md', 'defer'], + ['plan-eng-review-arch-finding', 'C) Accept the risk and ship', 'accept-risk'], + ['plan-ceo-review-expansion-proposal', 'A) Add to plan (recommended)', 'accept'], + ['plan-ceo-review-expansion-proposal', 'B) Add to TODOs', 'defer'], + ['plan-ceo-review-mode', 'A) Expand scope (Recommended)', 'expand'], + ['plan-ceo-review-mode', 'A) SCOPE EXPANSION', 'expand'], + ['plan-ceo-review-mode', 'B) SELECTIVE EXPANSION', 'selective'], + ['plan-ceo-review-mode', 'C) HOLD SCOPE (recommended)', 'hold'], + ['plan-ceo-review-mode', 'D) SCOPE REDUCTION', 'reduce'], + ['plan-ceo-review-mode', 'Cherry-pick expansions', 'selective'], + ['plan-ceo-review-mode', 'Cut to the minimum', 'reduce'], + ['ship-pre-landing-review-fix', 'Ship as-is', 'skip'], + ['ship-test-failure-triage', 'Ship anyway, the failures are known', 'ack-and-ship'], + ['review-finding-fix', 'Not a bug, the input is trusted', 'false-positive'], + ['land-and-deploy-merge-confirm', 'Yes, merge it', 'accept'], + ['land-and-deploy-merge-confirm', 'Don’t merge yet', 'reject'], + ['plan-eng-review-test-gap', 'Add the test now', 'add-test'], + ['office-hours-mode-goal', 'Hackathon / demo', 'hackathon'], + ]; + for (const [id, label, key] of cases) { + expect({ id, label, key: resolveOptionKey(id, label) }).toEqual({ id, label, key }); + } + }); + + test('negative controls stay unmapped', () => { + // Unregistered (ad-hoc or hook-hash) ids: no registered options to map to. + expect(resolveOptionKey('plan-eng-review-ci-typecheck', 'Fix now')).toBeUndefined(); + expect(resolveOptionKey('hook-0123456789', 'Accept')).toBeUndefined(); + // Registered question with no options list. + expect(resolveOptionKey('plan-devex-review-persona', 'Accept')).toBeUndefined(); + // A label that names none of the question's options. + expect(resolveOptionKey('plan-eng-review-arch-finding', 'Restore the parallel typecheck job')).toBeUndefined(); + // Cues anchor at the start of the label, never mid-sentence. + expect(resolveOptionKey('review-sql-safety', 'Ship it, then fix')).toBeUndefined(); + expect(resolveOptionKey('review-sql-safety', "Don't fix")).toBeUndefined(); + // Another question's key is not a candidate. + expect(resolveOptionKey('qa-tier', 'Fix now')).toBeUndefined(); + // A selector-looking number is not stripped into a match. + expect(resolveOptionKey('qa-tier', '1.5x deeper')).toBeUndefined(); + expect(resolveOptionKey('plan-eng-review-arch-finding', '')).toBeUndefined(); + }); + + function logAndRead(entries: Array>): Array> { + const tmpHome = fs.mkdtempSync(path.join(require('os').tmpdir(), 'gstack-2623-')); + try { + const { spawnSync } = require('child_process'); + const env = { ...process.env, GSTACK_HOME: tmpHome, GSTACK_QUESTION_LOG_NO_DERIVE: '1' }; + const logBin = path.join(ROOT, 'bin', 'gstack-question-log'); + for (const e of entries) { + const r = spawnSync(logBin, [JSON.stringify({ skill: 'plan-eng-review', question_summary: 'q?', ...e })], { env, cwd: ROOT, encoding: 'utf-8', timeout: 30_000 }); + expect(r.status).toBe(0); + } + const [slug] = fs.readdirSync(path.join(tmpHome, 'projects')); + return fs + .readFileSync(path.join(tmpHome, 'projects', slug, 'question-log.jsonl'), 'utf-8') + .trim() + .split('\n') + .map((l) => JSON.parse(l)); + } finally { + fs.rmSync(tmpHome, { recursive: true, force: true }); + } + } + + test('gstack-question-log records the key and keeps the label for hook events on registered ids only', () => { + const [hook, agentKey, agentMode, adHoc, unmatched, freeText] = logAndRead([ + { question_id: 'plan-eng-review-arch-finding', user_choice: 'A) Fix it now (recommended)', recommended: 'Fix it now', source: 'hook' }, + { question_id: 'plan-eng-review-arch-finding', user_choice: 'defer', recommended: 'fix-now' }, + { skill: 'plan-ceo-review', question_id: 'plan-ceo-review-mode', user_choice: 'HOLD SCOPE', recommended: 'HOLD SCOPE', auto_decided: true }, + { question_id: 'plan-eng-review-ci-typecheck', user_choice: 'Fix now', source: 'hook' }, + { question_id: 'plan-eng-review-arch-finding', user_choice: 'Restore the parallel typecheck job', source: 'hook' }, + { question_id: 'plan-eng-review-arch-finding', user_choice: 'Fix it but rename the module first', free_text: 'Fix it but rename the module first', source: 'auq-other' }, + ]); + expect(hook).toMatchObject({ user_choice: 'fix-now', choice_label: 'A) Fix it now (recommended)', recommended: 'fix-now', followed_recommendation: true }); + expect(agentKey).toMatchObject({ user_choice: 'defer', recommended: 'fix-now', followed_recommendation: false }); + expect(agentKey.choice_label).toBeUndefined(); + // Agent-written rows keep what the agent logged; the auto-decide harness + // matches plan-ceo-review's full mode names. + expect(agentMode).toMatchObject({ source: 'agent', user_choice: 'HOLD SCOPE', recommended: 'HOLD SCOPE', followed_recommendation: true }); + expect(agentMode.choice_label).toBeUndefined(); + expect(adHoc).toMatchObject({ user_choice: 'Fix now' }); + expect(adHoc.choice_label).toBeUndefined(); + expect(unmatched).toMatchObject({ user_choice: 'Restore the parallel typecheck job' }); + expect(unmatched.choice_label).toBeUndefined(); + expect(freeText).toMatchObject({ user_choice: 'Fix it but rename the module first', source: 'auq-other' }); + expect(freeText.choice_label).toBeUndefined(); + }); + + test('a picked label scores as following a key recommendation', () => { + const [row] = logAndRead([ + { question_id: 'plan-ceo-review-mode', user_choice: 'A) Expand scope (Recommended)', recommended: 'expand', source: 'hook' }, + ]); + expect(row).toMatchObject({ user_choice: 'expand', followed_recommendation: true }); + }); +}); + // ----------------------------------------------------------------------- // Archetypes // ----------------------------------------------------------------------- diff --git a/test/qa-only-fixture.test.ts b/test/qa-only-fixture.test.ts index 004b5c65e..af853d920 100644 --- a/test/qa-only-fixture.test.ts +++ b/test/qa-only-fixture.test.ts @@ -19,7 +19,7 @@ test('QA-only fixture helpers and owned sections select the no-fix consumer', () }); test.each(['success', 'max-turns', 'dependencies', 'metadata', 'edit', 'source-write', 'source-delete', 'source-add', 'setup', 'runner', 'recording', 'runner-recording', 'artifact-error', - 'process-error', 'slow-setup', 'exhausted-setup', 'late-timeout', 'late-retry', 'late-failure-retry', 'unknown-timeout', + 'process-error', 'slow-setup', 'exhausted-setup', 'late-timeout', 'late-retry', 'late-slow-retry', 'late-failure-retry', 'unknown-timeout', 'cleanup', 'cleanup-retry', 'runner-cleanup', 'cleanup-recording', 'remove-error', 'server-stop', 'missing-deadline', 'unguarded-probe', 'forged-receipt', 'deadline-write', 'reset-deadline', 'preparation-missing', 'preparation-failed', 'preparation-unacknowledged', 'preparation-wrong-path', 'preparation-late', @@ -47,11 +47,12 @@ if (scenario === 'remove-error') spyOn(fs, 'rmSync').mockImplementation((target, return originalRm(target, options); }); mock.module(path.join(root, 'test/helpers/eval-store.ts'), () => ({ getProjectEvalDir: () => ${JSON.stringify(path.join(dir, 'evals'))} })); -const late = scenario.startsWith('late-'), workMs = 1000, drainMs = 80; +const late = scenario.startsWith('late-'), lateRetry = ['late-retry', 'late-slow-retry'].includes(scenario), workMs = 1000, drainMs = 80; const realNow = Date.now, controlledClock = ['slow-setup', 'exhausted-setup'].includes(scenario); const clockStart = realNow(); -let clock = clockStart; +let clock = clockStart, frozenAt; if (controlledClock) Date.now = () => clock; +else if (late) Date.now = () => frozenAt ?? realNow(); let finalized = false, outerMs; const collector = { addTest: row => { const cwd = directories.at(-1); @@ -77,8 +78,7 @@ mock.module(path.join(root, 'test/helpers/e2e-helpers.ts'), () => ({ testConcurrentIfSelected: (id, body, budget) => { if (id !== 'qa-only-no-fix') return; outerMs = budget; - const deadline = late ? (budget >= workMs + 5000 ? budget - 5000 + 500 : budget) : 5000; - test(id, body, deadline); + test(id, late ? () => { frozenAt = realNow(); return body(); } : body, late ? budget : 5000); }, copyDirSync: (source, target) => { if (scenario === 'setup') throw new Error('QA-only setup failed'); @@ -87,6 +87,7 @@ mock.module(path.join(root, 'test/helpers/e2e-helpers.ts'), () => ({ setupBrowseShims: cwd => { directories.push(cwd); fs.mkdirSync(path.join(cwd, 'browse/bin'), { recursive: true }); if (controlledClock) clock += scenario === 'slow-setup' ? 295000 : 300000; + if (scenario === 'late-slow-retry') Bun.sleepSync(workMs + 100); }, logCost() {}, createEvalCollector: () => collector, finalizeEvalCollector: async () => { finalized = true; }, recordE2E: (_collector, name, suite, result, extra) => collector.addTest({ name, suite, @@ -96,6 +97,7 @@ mock.module(path.join(root, 'test/helpers/e2e-helpers.ts'), () => ({ mock.module(path.join(root, 'test/helpers/session-runner.ts'), () => ({ SESSION_DRAIN_GRACE_MS: late ? drainMs : 5000, runSkillTest: async options => { + frozenAt = undefined; const cwd = options.workingDirectory; const fact = { cwd, exists: fs.existsSync(cwd), recordedBeforeCapture: records.length, reportBefore: fs.existsSync(path.join(cwd, 'qa-reports/qa-only-report.md')), captureMs: options.timeout, @@ -145,7 +147,7 @@ mock.module(path.join(root, 'test/helpers/session-runner.ts'), () => ({ fact.existsAfterDrain = fs.existsSync(cwd); return { ...result, exitReason: 'timeout' }; } - if (late && (scenario !== 'late-retry' || attempts.length === 1)) { + if (late && (!lateRetry || attempts.length === 1)) { await new Promise(resolve => setTimeout(resolve, options.timeout + drainMs)); fact.existsAfterDrain = fs.existsSync(cwd); return { ...result, exitReason: 'timeout' }; @@ -237,7 +239,7 @@ await import(path.join(root, 'test/skill-e2e-qa-workflow.test.ts')); expect(records[index].cleanupDone).toBe(true); expect(cleanups[index].budget).toBeGreaterThan(0); expect(cleanups[index].budget).toBeLessThanOrEqual(4000); - expect(records[index].passed).toBe(scenario === 'recording' || !shouldFail && !(['late-retry', 'cleanup-retry'].includes(scenario) && index === 0)); + expect(records[index].passed).toBe(scenario === 'recording' || !shouldFail && !(['late-retry', 'late-slow-retry', 'cleanup-retry'].includes(scenario) && index === 0)); if (scenario.includes('cleanup') && !(scenario === 'cleanup-retry' && index === 1)) expect(records[index].error).toContain('QA-only browser cleanup: invalid daemon identity'); } for (const [index, attempt] of attempts.entries()) { @@ -257,6 +259,7 @@ await import(path.join(root, 'test/skill-e2e-qa-workflow.test.ts')); } if (scenario.startsWith('late-')) { expect(outerMs).toBe(1000 + 80 + 5000); + for (const attempt of attempts) expect(attempt.captureMs).toBe(1000); expect(records[0]).toMatchObject({ passed: false, exit_reason: 'timeout', cost_usd: 0.25 }); expect(records[0].error).toContain('timeout'); if (count === 2) expect(directories[0]).not.toBe(directories[1]); diff --git a/test/skill-e2e-context-skills.test.ts b/test/skill-e2e-context-skills.test.ts index 31f214505..c76697a4a 100644 --- a/test/skill-e2e-context-skills.test.ts +++ b/test/skill-e2e-context-skills.test.ts @@ -23,6 +23,7 @@ import { } from './helpers/e2e-helpers'; import { extractSkillBody } from './helpers/skill-fixture'; import { readShippedSkillRouting } from './helpers/shipped-skill-routing'; +import { fragmentRestoreVerdict } from './helpers/context-restore-fragment'; import { spawnSync } from 'child_process'; import * as fs from 'fs'; import * as path from 'path'; @@ -260,11 +261,14 @@ Do NOT use AskUserQuestion.`, logCost('context-restore-fragment-match', result); - // Broader surface — agent may stop at Skill call without echoing the - // body marker. The payments file's body is in tool outputs (Read/Bash). - const out = fullOutputSurface(result); - const loadedPayments = out.includes('FRAGMATCH_PAYMENTS_BUILD'); - const didNotLoadOthers = !out.includes('FRAGMATCH_ALPHA_BUILD') && !out.includes('FRAGMATCH_OMEGA_BUILD'); + // What was restored is what the agent reports, not what its tools printed: + // finding the matching title can cat every checkpoint, so all three bodies + // may appear in Bash/Read output. The report must carry the payments body + // and no other checkpoint's body. + const { loadedTarget: loadedPayments, didNotLoadOthers } = fragmentRestoreVerdict(result, { + restored: 'FRAGMATCH_PAYMENTS_BUILD', + others: ['FRAGMATCH_ALPHA_BUILD', 'FRAGMATCH_OMEGA_BUILD'], + }); const routedToRestore = skillCalls(result).includes('context-restore'); const exitOk = ['success', 'error_max_turns'].includes(result.exitReason); diff --git a/test/test-of-test-ratchet.test.ts b/test/test-of-test-ratchet.test.ts index bffa2c402..983429d25 100644 --- a/test/test-of-test-ratchet.test.ts +++ b/test/test-of-test-ratchet.test.ts @@ -60,6 +60,7 @@ const BASELINE = [ 'test/codex-carve-fixture.test.ts', 'test/codex-eval-recording.test.ts', 'test/codex-resume-flag-semantics.test.ts', + 'test/context-restore-fragment-verdict.test.ts', // GSTA-91: replays three captured false reds (0/3 on main cb99df4) through the paid context-restore-fragment-match case's fragmentRestoreVerdict 'test/cookie-workflow-judge-input.test.ts', // slices test/skill-llm-eval.test.ts source (recorded 2026-10 when the ratchet learned paid-source slicing) 'test/cso-ntfs-fixture.test.ts', 'test/cso-windows-build-contract.test.ts',