Files
gstack/test/helpers/claude-pty-runner.screen.unit.test.ts
ac20ef1e13 v1.91.16.0 fix: safe installs and upgrades, skills start everywhere, truthful completion (parallel wave) (#3014)
* fix(make-pdf,design): real TOC page numbers and anchors; design models move to gpt-5.5 + gpt-image-2 (#2903, #2807)

* ci: pin every action to a SHA, Scorecard, checksummed Bun, scoped Codex CLI in the CI image, macOS named-regressions job (#1948, #1706, #1997)

* fix(cso,browse): Windows/macOS lease identity, retention skips bad runs, truthful 'not assessed' reports, env.kv false positive, Windows browse spawn fixes (#2894, #2912, #2048, #2637, #2805)

* fix(outside-voice): one Codex model precedence for probe and dispatch, bounded timeouts with KILL escalation, probe cache identity, guarded mktemp, skills hidden in nested Codex (#2776, #2914, #2787, #2881, #2847)

* feat(setup): install registry, ./setup --status, staged activation, per-scope migrations, host tiers and conformance kit, OpenCode/Hermes fixes, Copilot CLI (experimental), disabled_skills, slimmer CLAUDE.md (#1925, #2879, #2347, #2338, #2626, #2629, #2825, #2015, #393, #1206, #2096)

Co-authored-by: Andrey Esipov <61357035+andrey-esipov@users.noreply.github.com>
Co-authored-by: Matthew Williams <5863886+ridermw@users.noreply.github.com>
Co-authored-by: Rocky Wu <92764750+lolisaigao1234@users.noreply.github.com>

* fix(preamble): skills start in worktree-isolated Claude Code sessions — one literal skill-start line, context-recovery helper, no eval in skill blocks, $N token tripwire, honest plan-mode precedence (#2763, #2896, #2851)

* fix(setup): per-install renders for renamed, vendored and custom-config installs, space-safe alias roots, per-host /cso launcher path (#1882, #2906)

* test: free tests never write the real HOME (home-write tripwire), no orphaned browse daemons, symlinked-TMPDIR (macOS) fixes, gbrain-on-PATH isolation (#2895, #2816, #2829, #2841, #2982, #2911, #2909, #2908, #2910)

* chore: integrate wave follow-ups — regenerate all hosts, README model docs, TODOS, touchfiles, duration seeds, Bun 1.3 telemetry test fix, replay fixtures pinned to their recorded bytes

* v1.91.14.0: release metadata

* test(browse): group-kill cleanup falls back to the leader when macOS returns EPERM for a live group

* ci: Codex login steps run under bash (the CI container's default sh rejects pipefail)

* fix: full-suite repairs — private state roots for 7 real-HOME writers, truthful home-guard attribution with --attribute-home, gstack-paths shim in ship-hook fixtures, Copilot in team-init probes, test-of-test detector, cookie review fixture pinned to the approved excerpt

* fix(windows): LF for Dockerfiles, normalized home-guard exclusions, path.relative fixture containment, ship push test spawn timeouts

* test(design-detect): widen the whole-scan budget test's timing margins so a loaded macOS runner can't time out the single-batch call

* fix(evals): qa protocol names defined for the judge, async PTY judge so overlapping sessions keep their cleanup deadline, bws setup cases use the standard JUDGE_MS tier

* test(pty): the non-blocking judge case is POSIX-only (its fake judge is a shebang script, like the PTY harness it covers)

* test(evals): ship-coverage-audit names absolute skill paths, uses Read, and skips the Step 7 artifact nothing consumes (removes the mechanism behind 3 of 4 historical timeouts)

* test: deflake terminal-agent concurrent startup (write the placeholder record before spawning, like the product) and qa-evidence resumed pipe (barrier on a full high-water mark instead of a sleep)

* fix(design): daemon log stream handles open errors instead of crashing the daemon; shutdown test awaits the stubbed exit

* test(evals): skillify checks the product's resolved state root (no GSTACK_HOME bypass), negative cases check every root; tpa shims live outside the agent's cwd and answer consistently

* test(evals): coverage-audit detector splits parallel columns at │ rails and [+] groups; coverage prompts name absolute skill paths; TODOs from the gate eval census

* fix(plan-eng-review): scope gate always sends the menu (no headless 'Scope pending' exit); PTY harness ends idle turn-ends and API-error panels promptly with a precise diagnostic

* test(evals): one position-bound coverage-read rule credits read-only && chains (labels, context ranges, git displays in either order) and still rejects forged output

* fix(ship): documentation section has one saved source inserted by cat (never retyped); spawned-marker evidence is the child output; refreshed codex/factory ship goldens

* test(docsync): the byte-exact insert positive is POSIX-only (the bash-command observer refuses backslash paths by design)

* test(evals): plan-design-with-ui focus-title rule accepts the comma-less 'dimensions or focus' wording (replayed on the captured CI run, negative controls kept)

* fix(qa-callers): exploratory preflight says the scope/method Reads come after exploratory.md; design-review scan check credits an executed scan proven by one engine run over the changed files

---------

Co-authored-by: Andrey Esipov <61357035+andrey-esipov@users.noreply.github.com>
Co-authored-by: Matthew Williams <5863886+ridermw@users.noreply.github.com>
Co-authored-by: Rocky Wu <92764750+lolisaigao1234@users.noreply.github.com>
2026-10-03 07:54:09 -07:00

261 lines
11 KiB
TypeScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* Deterministic unit tests for the ANSI/TUI screen detectors (test/helpers/pty/screen.ts).
* Split along the W4 module seams from the former claude-pty-runner.unit.test.ts;
* tests import the public barrel, test/helpers/claude-pty-runner.ts.
*/
import { describe, test, expect } from 'bun:test';
import { readFileSync } from 'node:fs';
import {
isPermissionDialogVisible,
isNumberedOptionListVisible,
isAutoDecidedVisible,
classifyVisible,
classifyPlanCountFrame,
idleTurnEnd,
} from './claude-pty-runner';
describe('saved preference annotation', () => {
test('recognizes the explicit preference attribution from the timed-out CEO capture', () => {
const visible = 'Now I have a clear picture of the branch. Let me proceed with the full review. ' +
'Mode is HOLD SCOPE (auto-decided from plan-tune preference).';
expect(isAutoDecidedVisible(visible)).toBe(true);
expect(classifyVisible(visible)?.outcome).toBe('auto_decided');
expect(classifyVisible(visible.replace(/\s+/g, ''))?.outcome).toBe('auto_decided');
});
test('retains the canonical annotation and its precedence over plan-ready', () => {
const visible = 'Auto-decided review mode → HOLD SCOPE (your preference). Change with /plan-tune.\nReady to execute?';
expect(classifyVisible(visible)?.outcome).toBe('auto_decided');
});
test('does not equate an unrequested choice or plan-tune advice with a saved preference', () => {
for (const visible of [
'Mode is HOLD SCOPE (AUTO_DECIDED).',
'I auto-decided HOLD SCOPE because this is a refactor.',
'I auto-decided HOLD SCOPE. You can set a plan-tune preference later.',
'Mode is HOLD SCOPE (not auto-decided from plan-tune preference).',
'Mode is HOLD SCOPE (will be auto-decided from plan-tune preference).',
]) expect(isAutoDecidedVisible(visible)).toBe(false);
});
});
describe('mode option rendering', () => {
});
describe('isPermissionDialogVisible', () => {
test('matches "Bash command requires permission" prompts', () => {
const sample = `
Some preamble output
Bash command \`gstack-config get telemetry\` requires permission to run.
❯ 1. Yes
2. Yes, and always allow
3. No, abort
`;
expect(isPermissionDialogVisible(sample)).toBe(true);
});
test('matches "allow all edits" file-edit prompts', () => {
// Isolated to the "allow all edits" clause only — no overlapping
// "Do you want to proceed?" co-trigger, so this asserts the clause works.
const sample = `
Edit to ~/.gstack/config.yaml
❯ 1. Yes
2. Yes, allow all edits during this session
3. No
`;
expect(isPermissionDialogVisible(sample)).toBe(true);
});
test('matches the "Do you want to proceed?" file-edit confirmation by itself', () => {
// Separate fixture so weakening this clause is detected by a dedicated test.
const sample = `
Edit to ~/.gstack/config.yaml
Do you want to proceed?
❯ 1. Yes
2. No
`;
expect(isPermissionDialogVisible(sample)).toBe(true);
});
test('matches workspace-trust "always allow access to" prompt', () => {
const sample = `
Do you trust the files in this folder?
❯ 1. Yes, proceed
2. Yes, and always allow access to /Users/me/repo
3. No, exit
`;
expect(isPermissionDialogVisible(sample)).toBe(true);
});
test('recognizes the captured collapsed native overwrite confirmation', () => {
const sample = [
'Doyouwanttooverwritegstack-test-plan-design.md?',
'❯1.Yes',
'2.Yes,andswitchtoacceptedits(auto-approvefileeditsandcommonfilecommands)forthissession',
'3.No',
'Esctocancel·Tabtoamend',
].join('\n');
expect(isPermissionDialogVisible(sample)).toBe(true);
expect(isPermissionDialogVisible(sample.replace('Esctocancel·Tabtoamend', 'Enter to select'))).toBe(false);
});
test('the captured paired-CEO Edit grant is permission, not another review finding', () => {
const sample = [
'Do youwt to makehis dittogstack-test-plan-ceo-paired.md?',
'❯1.Yes',
'2.Yes,andswitchtoacceptedits(auto-approvefileeditsandcommonfilecommands)forthissession;Yes,and',
'alwaysallowaccessto/tmp/gstack-paid-shard-EbUl9j/tmp/gstack-e2e-plan-ceo-paired-gkjAd5forthissession',
'(shift+tab)', '3.No', 'Esctocancel·Tabtoamend',
].join('\r');
expect(isPermissionDialogVisible(sample)).toBe(true);
expect(classifyPlanCountFrame(sample)).toBe('permission');
});
test('recognizes permission labels whose cursor-positioning spaces disappeared', () => {
expect(isPermissionDialogVisible('Yes,andalwaysallowaccessto/tmp/fixtureforthissession')).toBe(true);
expect(isPermissionDialogVisible('Yes,allowalleditsduringthissession')).toBe(true);
expect(isPermissionDialogVisible('Bashcommandrequirespermission')).toBe(true);
});
test('does NOT match a skill AskUserQuestion list', () => {
const sample = `
D1 — Premise challenge: do users actually want this?
❯ 1. Yes, validated
2. No, premise is wrong
3. Need more info
`;
expect(isPermissionDialogVisible(sample)).toBe(false);
});
test('does NOT match a plan-ready confirmation', () => {
const sample = `
Ready to execute the plan?
❯ 1. Yes
2. No, keep planning
`;
expect(isPermissionDialogVisible(sample)).toBe(false);
});
test('does NOT match a skill question that contains the bare phrase "Do you want to proceed?"', () => {
// Co-trigger requirement: "Do you want to proceed?" alone is not enough.
// It must appear with "Edit to <path>" or "Write to <path>" to count as
// a permission dialog. This guards against a skill question like
// "Do you want to proceed with HOLD SCOPE?" being mis-classified.
const sample = `
Choose your scope mode for this review.
Do you want to proceed?
❯ 1. HOLD SCOPE
2. SCOPE EXPANSION
3. SELECTIVE EXPANSION
`;
expect(isPermissionDialogVisible(sample)).toBe(false);
});
test('does NOT mis-match when adversarial prose includes "Edit to <path>" alongside the bare proceed phrase', () => {
// Adversarial fixture: a skill question whose body legitimately mentions
// "Edit to <path>" in prose AND ends with "Do you want to proceed?". The
// current co-trigger regex would mis-classify this as a permission
// dialog. We DO want this test to fail until the regex is tightened
// further (e.g., proximity constraint, or anchoring "Edit to" to a
// line-start). For now this is documented as a known limitation: a
// skill question that talks about "Edit to" in prose IS still treated
// as a permission dialog. The test asserts the current behavior so a
// future fix can flip it intentionally.
const sample = `
Plan: I will Edit to ./plan.md to capture the decision.
Do you want to proceed?
❯ 1. HOLD SCOPE
2. SCOPE EXPANSION
`;
// KNOWN LIMITATION: the co-trigger fires here. Documented as a
// post-merge follow-up. Flip this assertion once the regex tightens.
expect(isPermissionDialogVisible(sample)).toBe(true);
});
test('matches the captured Autoplan settings-overwrite card as a numbered permission dialog', () => {
const captured = JSON.parse(readFileSync(new URL('../fixtures/autoplan-settings-overwrite.json', import.meta.url), 'utf8'));
expect(isNumberedOptionListVisible(captured.frame.text)).toBe(true);
expect(isPermissionDialogVisible(captured.frame.text)).toBe(true);
});
});
describe('isNumberedOptionListVisible', () => {
test('matches a basic ❯ 1. + 2. cursor list', () => {
const sample = `
❯ 1. Option one
2. Option two
3. Option three
`;
expect(isNumberedOptionListVisible(sample)).toBe(true);
});
test('returns false on a single-option prompt', () => {
const sample = `
❯ 1. Only option
`;
expect(isNumberedOptionListVisible(sample)).toBe(false);
});
test('returns false when no cursor renders', () => {
const sample = `
Just some prose with 1. a numbered point and 2. another.
`;
expect(isNumberedOptionListVisible(sample)).toBe(false);
});
test('overlaps permission dialogs (this is why D5 short-circuits)', () => {
// The whole point of D5: this string matches BOTH classifiers, so the
// runner must consult isPermissionDialogVisible to disambiguate.
const sample = `
Bash command \`do-thing\` requires permission to run.
❯ 1. Yes
2. No
`;
expect(isNumberedOptionListVisible(sample)).toBe(true);
expect(isPermissionDialogVisible(sample)).toBe(true);
});
});
describe('idle turn end (captured CI screens)', () => {
const captures = JSON.parse(readFileSync(new URL('../fixtures/pty-idle-turn-end.json', import.meta.url), 'utf8'));
const scope: string = captures.scopePendingIdle.visible;
const api: string = captures.apiErrorIdle.visible;
test('the 988e985 pending line, end-of-turn line and empty prompt are an idle turn end', () => {
expect(idleTurnEnd(scope)).toEqual({ done: '✻Cooked for 24s · done 12:46 AM' });
expect(classifyVisible(scope)).toBeNull();
});
test('the 6a6aa32 provider error panel is reported with its idle turn end', () => {
expect(idleTurnEnd(api)).toEqual({ done: '✻ Sautéedfor1m 20s · done11:34PM',
apiError: '●API Error: Connection lost mid-response. The response above may be incomplete.' });
});
test('negative controls: still thinking, a resumed turn, extra footer text and an open question are not idle', () => {
expect(idleTurnEnd(scope.slice(0, scope.indexOf('●Scope')))).toBeNull();
expect(idleTurnEnd(scope + '\r✶ Thinking… (3s · thinking)')).toBeNull();
const footer = scope.lastIndexOf('← for agents');
expect(idleTurnEnd(`${scope.slice(0, footer)}← for agents · 1 background task${scope.slice(footer + '← for agents'.length)}`)).toBeNull();
const question = '☐ Scope\nWhat should I review?\n❯ 1. The current branch diff\n 2. A plan or design doc\nEnter to select · ↑/↓ to navigate · Esc to cancel';
expect(idleTurnEnd(question)).toBeNull();
expect(idleTurnEnd(scope + '\r' + question)).toBeNull();
});
test('an API Error mention in prose or in an earlier turn is not this turn\'s error panel', () => {
expect(idleTurnEnd('The docs mention API Error: as an example.\r✻ Worked for 3s · done 1:00 PM\r❯ \r← for agents')).toEqual({ done: '✻ Worked for 3s · done 1:00 PM' });
const earlier = api.replace(/✻ Sautéedfor1m 20s · done11:34PM[\s\S]*$/, '✻ Sautéedfor1m 20s · done11:34PM\r❯ /plan-design-review\r●Done.\r✻ Worked for 2s · done 11:35PM\r❯ \r← for agents');
expect(idleTurnEnd(earlier)).toEqual({ done: '✻ Worked for 2s · done 11:35PM' });
});
});