mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-01 22:45:02 +01:00
v1.92.32.0 fix(flaky): Windows hook kill, publication-gates deadline, xvfb marker, devex floor idle API error (2026-10-29 flaky pass, GSTA-162) (#3607)
* fix(flaky): xvfb welcome marker written atomically; ship-publication-gates kills a stuck block's whole tree before cleanup; hook spawns name a kill instead of reading it as exit 1 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(flaky): the PTY floor runner ends on an idle plan-mode turn and reports an API error as api_error, not observer_timeout Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(browse): the real Playwright pipe smoke names the worker reply and Playwright's launch error when the launch never reaches cookie_read Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): Step 11's review plan alone decides which voices run; greptile fixes requeue 9 → 10 instead of an inline loop (ship/SKILL.md workflow judge, 40815828320) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(flaky): the publication-gates case deadline covers a block's output pipes, not only bash's exit; deadline tests clean up in finally Step 11 native adversarial finding: a block whose background child holds stdout hung past the 27s deadline. New POSIX case fails at93d0c41f4(hung 30s) and passes here. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): GSTA-162 flaky pass, second PR (Unreleased) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(pregate): record the PTY floor and screen changes as non-behavioral at HARNESS_VERSION 6; the planDispatch comment names /ship's adversarial step instead of an undefined Step 11 reference Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): Gate integrity line on the HARNESS_VERSION decision reads as two sentences Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(flaky): the floor runner assesses a question already on screen before an idle provider error ends the check; harness-version reason names the ledger failure_cause shift (fix-check finding) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(flaky): the floor runner's idle api_error exit waits for a 10 s settle, re-reading the transcript, so a late transcript line still credits its question (fix-check residual) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): drop the Step 11 dispatch hunk; main pins ship/sections/adversarial.md at its GST-257 no-loss bytes (review-packets-removed.test.ts), and the ship/SKILL.md workflow judge passes on main48599bbwithout it (c:3 c:4 a:4) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
Garry Tanandcapy-ai[bot] authored and GitHub committed
1 parent
7fa7f5f903
commit
84c5019e23
14 files changed
+233
-49
No files matched your search
@@ -1,5 +1,26 @@
|
||||
# Changelog
|
||||
|
||||
## [1.92.32.0] - 2026-11-01
|
||||
|
||||
**Four flakes from the 2026-10-29 flaky pass stop failing correct runs.** Three free-tier tests failed on harness timing (a half-written marker file, a Windows kill that left a process tree behind, a hook kill set at 5 s), and the plan-devex-finding-floor paid case waited out 600 s after a provider error. Each repair has a free test that fails without it.
|
||||
|
||||
### Fixed
|
||||
- **`browse/test/xvfb.test.ts`**: the welcome marker is written with `atomicWriteSync`, so the test never parses a half-written file (Linux Free Tests 40791645916 and 40847105381: "JSON Parse error: Unexpected EOF").
|
||||
- **`test/ship-publication-gates.test.ts`**: each publication block runs under a 27 s case deadline, below bun's 30 s test timeout. A block still running at the deadline, or one whose background child still holds its output, is killed with its whole process tree before cleanup, and the error names the stage and what the earlier stages took. Four Windows runs (40803875149, 40820968738, 40823026209, 40826743317) hit bun's timeout, which killed only the bash child, so cleanup failed with EBUSY and hid the timeout.
|
||||
- **`test/hook-scripts.test.ts`**: a hook that is killed or cannot spawn throws with its signal and elapsed time instead of reading as exit 1. Three Windows runs (40792942747, 40805828986, 40816343665) went red at 5.0 to 5.17 s, which was the old kill.
|
||||
- **plan-devex-finding-floor** (`test/helpers/pty/screen.ts`, `test/helpers/pty/runners/floor.ts`): the floor runner stops at an idle plan-mode turn and reports a provider error ("API Error: Connection lost mid-response", paid run 40851840940) as `api_error`, instead of waiting out 600 s as `observer_timeout`. A question already on screen is assessed first, and the idle screen must hold still for 10 s, so a question is still credited even when its transcript line lands late. The plan-mode footer counts as idle only without its running "esc to interrupt" segment.
|
||||
|
||||
### Changed
|
||||
- `browse/test/cookie-import-native.test.ts`: when the real Playwright pipe smoke never reaches `cookie_read`, the failure names the worker's reply and Playwright's launch error. The same failure repeated on PR #3606's Free Tests run 40888372424 (shard 10).
|
||||
|
||||
### Gate integrity
|
||||
- `test/hook-scripts.test.ts`: the hook spawn kill moves from 5 s to 25 s (`HOOK_SPAWN_TIMEOUT_MS`), under bun's 30 s per-test timeout. Measured from the shard logs of 77 Windows Free Tests runs on 2026-10-29 (40811056776 through 40888378463): 10,608 passing check-careful tests, p95 0.98 s, p99.9 1.85 s, max 4.22 s; the file's first hook test p50 1.54 s, p95 2.14 s, max 4.22 s. The old 5 s kill sat 0.8 s above the slowest pass, and the one red in that set (40816343665) stopped at 5.00 s, the kill itself. 25 s is about six times the slowest pass and stays under bun's 30 s per-test timeout, so a stalled hook is reported by the harness with its signal and elapsed time instead of by bun's generic timeout. The kill lives only in the test (no product code reads it), no test asserts a duration, and the file still runs 266 tests at base and head. A hook slowed into the 5 to 25 s range no longer fails this file.
|
||||
- `scripts/harness-version.json`: `test/helpers/pty/runners/floor.ts` and `test/helpers/pty/screen.ts` are recorded as non-behavioral at HARNESS_VERSION 6, so no pass-rate series resets. The floor runner ends on `api_error` only after the idle screen held still for 10 s with no question to credit, and an idle turn with no question stays idle. Such a trial failed before as `observer_timeout` at the deadline and fails now, earlier, as `api_error` or with the idle-turn summary. A running footer still waits for the deadline. The session ledger can now record `failure_cause` `api_error` or `refusal` for such a session, while `failure_class` and pass rates stay the same.
|
||||
- `test/ship-publication-gates.test.ts`: the per-block 30 s spawn timeout becomes one 27 s deadline for the whole case, setup included, with a process-tree kill.
|
||||
|
||||
### Tests
|
||||
- Unit cases for the idle plan-mode turn end with negative controls (a running footer, extra footer text, a cut footer), a `planModeApiErrorIdle` capture in `test/fixtures/pty-idle-turn-end.json`, a POSIX case where a prose finding question printed before the error panel is still credited, also when its transcript line lands after the screen goes idle (it failed as `api_error` before the reorder), and a POSIX case where a block's background child holds its output past the deadline (it hung 30 s before the fix).
|
||||
|
||||
## [1.92.31.0] - 2026-11-01
|
||||
|
||||
**Nine paid ship rule-case flakes from the 2026-10-29 window stop failing correct runs.** Six qa-callers trials and five ship-docsync trials failed on calls the declared interface could not tell apart from a violation, or on a turn budget the actor spent out of order. Each was replayed from its retained capture, and each repair has a free test that fails without it.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# gstack digest v1.92.31.0 — regenerate/re-copy after upgrading gstack
|
||||
# gstack digest v1.92.32.0 — regenerate/re-copy after upgrading gstack
|
||||
|
||||
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
|
||||
for agent hosts without a full skill install. The full skills add workflows,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { afterAll, describe, expect, test } from 'bun:test';
|
||||
import { copyFileSync, mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { copyFileSync, existsSync, mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { release, tmpdir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
@@ -358,6 +358,7 @@ describe('production Node Playwright worker', () => {
|
||||
const { chromium } = require('playwright');
|
||||
const fixture = mkdtempSync(path.join(root, 'pipe-'));
|
||||
const observation = path.join(fixture, 'launch.json');
|
||||
const launchError = path.join(fixture, 'launch-error.txt');
|
||||
const playwrightEntry = path.join(fixture, 'instrumented-playwright.cjs');
|
||||
writeFileSync(playwrightEntry, `
|
||||
const cp = require('node:child_process');
|
||||
@@ -369,9 +370,14 @@ describe('production Node Playwright worker', () => {
|
||||
};
|
||||
const { chromium } = require(${JSON.stringify(require.resolve('playwright'))});
|
||||
exports.chromium = { async launchPersistentContext(root, options) {
|
||||
const context = await chromium.launchPersistentContext(root, options);
|
||||
await context.addCookies([{ name: 'synthetic', value: 'synthetic', domain: 'example.test', path: '/' }]);
|
||||
return context;
|
||||
try {
|
||||
const context = await chromium.launchPersistentContext(root, options);
|
||||
await context.addCookies([{ name: 'synthetic', value: 'synthetic', domain: 'example.test', path: '/' }]);
|
||||
return context;
|
||||
} catch (error) {
|
||||
require('node:fs').writeFileSync(${JSON.stringify(launchError)}, String(error?.stack ?? error));
|
||||
throw error;
|
||||
}
|
||||
} };
|
||||
`);
|
||||
const result = spawnSync(node!, ['--input-type=commonjs', '-e', NATIVE_COOKIE_NODE_SCRIPT], {
|
||||
@@ -383,7 +389,8 @@ describe('production Node Playwright worker', () => {
|
||||
expect(result.status).toBe(0);
|
||||
const progress = expectNativeProgress(result.stderr);
|
||||
expect(progress).toContainEqual({ stage: 'browser_launch' });
|
||||
expect(progress).toContainEqual({ stage: 'cookie_read' });
|
||||
const launchFailure = existsSync(launchError) ? readFileSync(launchError, 'utf8') : '(no launch error recorded)';
|
||||
expect(progress, `worker reply: ${result.stdout.trim()}; Playwright launch error: ${launchFailure}`).toContainEqual({ stage: 'cookie_read' });
|
||||
expect(JSON.parse(result.stdout).cookies).toHaveLength(1);
|
||||
const launched = JSON.parse(readFileSync(observation, 'utf8'));
|
||||
expect(launched.args).toContain('--remote-debugging-pipe');
|
||||
|
||||
@@ -373,6 +373,7 @@ describe.skipIf(process.platform !== 'linux')('daemon-owned display lifecycle',
|
||||
fs.writeFileSync(preload, `
|
||||
import * as fs from 'node:fs';
|
||||
import { BrowserManager } from ${JSON.stringify(path.resolve(import.meta.dir, '../src/browser-manager.ts'))};
|
||||
import { atomicWriteSync } from ${JSON.stringify(path.resolve(import.meta.dir, '../../lib/fs-atomic.ts'))};
|
||||
const getPage = BrowserManager.prototype.getPage;
|
||||
const wrapped = new WeakSet();
|
||||
let rejectWelcome;
|
||||
@@ -383,7 +384,7 @@ describe.skipIf(process.platform !== 'linux')('daemon-owned display lifecycle',
|
||||
const goto = page.goto.bind(page);
|
||||
page.goto = async (url, options) => {
|
||||
if (new URL(url).pathname === '/welcome') {
|
||||
fs.writeFileSync(${JSON.stringify(entered)}, JSON.stringify(this.getXvfbHandle()));
|
||||
atomicWriteSync(${JSON.stringify(entered)}, JSON.stringify(this.getXvfbHandle()));
|
||||
if (${JSON.stringify(mode)} === 'shutdown') {
|
||||
return new Promise((resolve, reject) => {
|
||||
rejectWelcome = () => reject(new Error('Welcome interrupted during shutdown'));
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gstack",
|
||||
"version": "1.92.31",
|
||||
"version": "1.92.32",
|
||||
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
||||
"license": "MIT",
|
||||
"type": "module",
|
||||
|
||||
@@ -41,9 +41,9 @@
|
||||
"test/helpers/pty/launch.ts": {"blob":"e4413de380b799659170e7da248d51afda3b0e3f","since":6,"nonBehavioral":"credential rejections are classed infra instead of assertion: no case's pass or fail changes; only failed trials whose provider rejected the run's key move class, so no rerun spends on a rejected key"},
|
||||
"test/helpers/pty/plan-native.ts": {"blob":"ddba9cb68c7c0da935cff7c1cea258dee2dff740","since":6},
|
||||
"test/helpers/pty/runners/counting.ts": {"blob":"8c5c05e54abf961259d34d9c3d77db67138d7794","since":6},
|
||||
"test/helpers/pty/runners/floor.ts": {"blob":"54e2145832bb98913644d10ff303792d67f9362e","since":6,"nonBehavioral":"Removes the plan-design 0D focus picker from the shared harness; floor.ts gains an optional pickSetupChoice hook that no other caller passes, so no other case's input or verdict can change. plan-design-finding-floor, the only case whose seed raises that menu, passes the picker from its owned fixture test/fixtures/design-floor-focus-picker.ts, so its owned bytes start its own new series."},
|
||||
"test/helpers/pty/runners/floor.ts": {"blob":"a9938cb98f285474ebc4860cd61012504a36466e","since":6,"nonBehavioral":"The PTY floor and observation runners end a plan-mode turn at an idle prompt (the plan-mode footer without its running 'esc to interrupt' segment, after a 10 s settle) instead of waiting out the deadline. The floor runner exits as api_error only after the idle screen with its error panel held still for 10 s and every tick's question and assessment steps found nothing to credit (the transcript is re-read each tick), so a native or prose question already on screen is credited as before, even when its transcript line lands after the screen draws done (test/helpers/claude-pty-runner.runners.unit.test.ts, the prose-question-before-panel case). With no qualifying question an idle turn stays idle, so a trial that failed at the deadline as observer_timeout now fails earlier as api_error. An observation returns what the deadline would have returned. A running footer, extra footer text and a cut footer still wait for the deadline. No pass becomes a fail or a fail a pass. The session ledger may now record failure_cause api_error or refusal instead of observer_timeout or session_timeout for a plan-mode session whose last idle turn shows a panel; failure_class, the gate red class and pass rates are unchanged, so no pass-rate series resets"},
|
||||
"test/helpers/pty/runners/observation.ts": {"blob":"1c2b75fbb984708725abaa8939125f7cab969f9f","since":6},
|
||||
"test/helpers/pty/screen.ts": {"blob":"f3743152fc9bb7a8e8c70002f75bf45914f260d5","since":6,"nonBehavioral":"credential rejections are classed infra instead of assertion: no case's pass or fail changes; only failed trials whose provider rejected the run's key move class, so no rerun spends on a rejected key"},
|
||||
"test/helpers/pty/screen.ts": {"blob":"1384f765e48e511d5f843df2660b2ec70bbcb6b2","since":6,"nonBehavioral":"The PTY floor and observation runners end a plan-mode turn at an idle prompt (the plan-mode footer without its running 'esc to interrupt' segment, after a 10 s settle) instead of waiting out the deadline. The floor runner exits as api_error only after the idle screen with its error panel held still for 10 s and every tick's question and assessment steps found nothing to credit (the transcript is re-read each tick), so a native or prose question already on screen is credited as before, even when its transcript line lands after the screen draws done (test/helpers/claude-pty-runner.runners.unit.test.ts, the prose-question-before-panel case). With no qualifying question an idle turn stays idle, so a trial that failed at the deadline as observer_timeout now fails earlier as api_error. An observation returns what the deadline would have returned. A running footer, extra footer text and a cut footer still wait for the deadline. No pass becomes a fail or a fail a pass. The session ledger may now record failure_cause api_error or refusal instead of observer_timeout or session_timeout for a plan-mode session whose last idle turn shows a panel; failure_class, the gate red class and pass rates are unchanged, so no pass-rate series resets"},
|
||||
"test/helpers/pty/session.ts": {"blob":"54fe04b61d953e4e55e2fc251532c04a49fab66f","since":6,"nonBehavioral":"credential rejections are classed infra instead of assertion: no case's pass or fail changes; only failed trials whose provider rejected the run's key move class, so no rerun spends on a rejected key"},
|
||||
"test/helpers/session-drain-policy.ts": {"blob":"6211239dd0485e6495a700b824a7e0e61d1bcd97","since":6},
|
||||
"test/helpers/session-ledger.ts": {"blob":"a10bf28bf100f135b93c73818670dd147253cdd3","since":6,"nonBehavioral":"credential rejections are classed infra instead of assertion: no case's pass or fail changes; only failed trials whose provider rejected the run's key move class, so no rerun spends on a rejected key"},
|
||||
|
||||
Vendored
+9
-1
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"provenance": "terminal.visible.log captures (session.visibleSince(command mark), byte-exact, ESC/CR residue kept) from paid-slice-5-a1 artifacts of test/skill-e2e-plan-mode-no-op.test.ts.",
|
||||
"provenance": "terminal.visible.log captures (session.visibleSince(command mark), byte-exact, ESC/CR residue kept); each entry names its run, head and test. The first three are paid-slice-5-a1 artifacts of test/skill-e2e-plan-mode-no-op.test.ts; planModeApiErrorIdle is paid-slice-9-a1 of test/skill-e2e-plan-devex-finding-floor.test.ts.",
|
||||
"scopePendingIdle": {
|
||||
"runId": 37082906717,
|
||||
"headSha": "988e985d",
|
||||
@@ -23,5 +23,13 @@
|
||||
"recordedPassed": true,
|
||||
"note": "Whole capture. The pending line followed by the full prose menu, then the idle prompt; classified 'asked'.",
|
||||
"visible": "\u000f\u001b[?2004h\u001b[?1000h\u001b[?1002h\u001b[?1003h\u001b[?1006h\u001b[?25l\n❯ /plan-eng-review \n*Computing… \n❯ \nesc to interrupt · ← for agents\u001b[?25h\u001b[?25l\n /plan-eng-review\u001b[?25h\u001b[?25l\n✢\u001b[?25h\u001b[?25l\n·\u001b[?25h\u001b[?25l\n…\u001b[?25h\u001b[?25l\n ● high · /effort\u001b[?25h\u001b[?25l\n✢g\u001b[?25h\u001b[?25l\n*n\u001b[?25h\u001b[?25l\n✶\u001b[?25h\u001b[?25l\ni…\u001b[?25h\u001b[?25l\n✻\u001b[?25h\u001b[?25l\n✽tg\u001b[?25h\u001b[?25l\nun\u001b[?25h\u001b[?25l\n✻pi\u001b[?25h\u001b[?25l\nGallop\u001b[?25h\u001b[?25l\n(1s · thinking)\u001b[?25h\u001b[?25l\n✶lpthinking\u001b[?25h\u001b[?25l\n*\u001b[?25h\u001b[?25l\n✢ao2\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n·Glthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\nlthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✢athinking\u001b[?25h\u001b[?25l\n*Gthinking\u001b[?25h\u001b[?25l\n✶\u001b[?25h\u001b[?25l\n✻3\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✽thinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✻thinking\u001b[?25h\u001b[?25l\n✶thinking\u001b[?25h\u001b[?25l\n*\u001b[?25h\u001b[?25l\n✢4\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n·thinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✢thinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n*thinking\u001b[?25h\u001b[?25l\n✶\u001b[?25h\u001b[?25l\n✻5\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✽thinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✻thinking\u001b[?25h\u001b[?25l\n✶thinking\u001b[?25h\u001b[?25l\n*\u001b[?25h\u001b[?25l\n✢6thinking\u001b[?25h\u001b[?25l\n·thinking\u001b[?25h\u001b[?25l\n…thinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✢gthinking\u001b[?25h\u001b[?25l\n*nthinking\u001b[?25h\u001b[?25l\n✶\u001b[?25h\u001b[?25l\n✻i…7thinking\u001b[?25h\u001b[?25l\n✽pgthinking\u001b[?25h\u001b[?25l\nonthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✻lithinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✶lp\u001b[?25h\u001b[?25l\n*\u001b[?25h\u001b[?25l\n✢ao8thinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n·Glthinking\u001b[?25h\u001b[?25l\nlthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✢athinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n*G\u001b[?25h\u001b[?25l\n✶\u001b[?25h\u001b[?25l\n✻9thinking\u001b[?25h\u001b[?25l\n✽thinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✻thinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✶\u001b[?25h\u001b[?25l\n*\u001b[?25h\u001b[?25l\n✢10s · thinking)\u001b[?25h\u001b[?25l\n·thinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✢thinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n*\u001b[?25h\u001b[?25l\n✶\u001b[?25h\u001b[?25l\n✻1thinking\u001b[?25h\u001b[?25l\n✽thinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\nthinking\u001b[?25h\u001b[?25l\n✻thinking\u001b[?25h\u001b[?25l\n✶still thinking)\u001b[?25h\u001b[?25l\n*\u001b[?25h\u001b[?25l\n✢2still thinking\u001b[?25h\u001b[?25l\n·still thinking\u001b[?25h\u001b[?25l\nstill thinking\u001b[?25h\u001b[?25l\nstill thinking\u001b[?25h\u001b[?25l\nstill thinking\u001b[?25h\u001b[?25l\n✢still thinking\u001b[?25h\u001b[?25l\n*still thinking\u001b[?25h\u001b[?25l\n*Galloping… still thinking\u001b[?25h\u001b[?25l\n✶3\u001b[?25h\u001b[?25l\n✻still thinking\u001b[?25h\u001b[?25l\n✽still thinking\u001b[?25h\u001b[?25l\nstill thinking\u001b[?25h\u001b[?25l\nstill thinking\u001b[?25h\u001b[?25l\nstill thinking\u001b[?25h\u001b[?25l\n✻still thinking\u001b[?25h\u001b[?25l\n✶still thinking\u001b[?25h\u001b[?25l\n*\u001b[?25h\u001b[?25l\n4\u001b[?25h\u001b[?25l\n✢Galloping… still thinking\u001b[?25h\u001b[?25l\n·still thinking\u001b[?25h\u001b[?25l\nstill thinking\u001b[?25h\u001b[?25l\nstill thinking\u001b[?25h\u001b[?25l\nstill thinking\u001b[?25h\u001b[?25l\n✢still thinking\u001b[?25h\u001b[?25l\n*still thinking\u001b[?25h\u001b[?25l\n✶\u001b[?25h\u001b[?25l\n5\u001b[?25h\u001b[?25l\n✻still thinking\u001b[?25h\u001b[?25l\n✽still thinking\u001b[?25h\u001b[?25l\n✽Galloping… still thinking\u001b[?25h\u001b[?25l\nstill thinking\u001b[?25h\u001b[?25l\nstill thinking\u001b[?25h\u001b[?25l\n✻still thinking\u001b[?25h\u001b[?25l\n✶still thinking\u001b[?25h\u001b[?25l\n*\u001b[?25h\u001b[?25l\n*Galloping… ↓ 1.1k tokenLine truncated
|
||||
},
|
||||
"planModeApiErrorIdle": {
|
||||
"runId": 40851840940,
|
||||
"headSha": "6c67e023",
|
||||
"testName": "seeded forcing finding causes the agent to fire at least one AskUserQuestion",
|
||||
"recordedPassed": false,
|
||||
"note": "Last 4000 characters. Plan mode. After the D1 setup reply the provider dropped the response mid-turn; the turn ended at the idle plan-mode prompt and the floor runner waited out its 600000ms budget (outcome=timeout, ledger end observer_timeout).",
|
||||
"visible": "escing…\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rCoalescing…\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻Coalescing…\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✽Coalescing…1m 0s· ↓ 3.8k tokens)\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rCoalescing…\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rCoalescing…\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rCoalescing…\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻Coalescing…1\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶Coalescing…\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢Coalescing…\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rCoalescing…\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r · thinking)\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r·thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢2thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r…\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶g\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✽nthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\ri…thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻cg3thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*sn\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rei\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r·lcthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rasthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢oe4thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*Clthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\ra\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✽othinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rCthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻5thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r·thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢6thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✽thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻7thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r·thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r8thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻…\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✽gthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rnthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻i…thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r9\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶cgthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢snthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r·eithinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rlcthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢asthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r10s · ↓ 3.8k tokens · thinking)\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*oethinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻Clthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✽athinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rothinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✻C1thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rthinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✶\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r*\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rstill thinking)\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r·still thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rstill thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rstill thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r✢2still thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\rstill thinking\r\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r●I pulled comparison data via WebSearch since Aside isn'tinstalled,treatingtheresultsasreportedindustryclaims\r rather than hard measurements. I built a table comparing Stripe, typical token-auth RESTAPIs,andthisplan's\ronboardingflow—butsincenonesharethesamestart/endpoints,I'mflaggingthatthiscomparesDXchoices,notactual\r times.\r\r✢ Coalescing… (1m 12s · ↓ 4.7k tokens · thought for 11s)\r ⎿ Tip: Use /btw to ask aquik side questin withot interrupting Claude's cLine truncated
|
||||
}
|
||||
}
|
||||
@@ -209,6 +209,42 @@ describe('runPlanSkillFloorCheck through the fake driver', () => {
|
||||
expect(obs.summary).toBe('no qualifying finding question within 30000ms');
|
||||
});
|
||||
|
||||
test('the 40851840940 provider error at an idle plan-mode prompt ends the floor check with its panel', async () => {
|
||||
const plan: string = JSON.parse(fs.readFileSync(new URL('../fixtures/pty-idle-turn-end.json', import.meta.url), 'utf8')).planModeApiErrorIdle.visible;
|
||||
const { obs } = await floor([{ onInput: '/plan-devex-review PLAN.md\r', screen: plan, effect: delivered }]);
|
||||
expect(obs.outcome).toBe('api_error');
|
||||
expect(obs.auqObserved).toBe(false);
|
||||
expect(obs.elapsedMs).toBeLessThan(30_000);
|
||||
expect(obs.summary).toBe('the CLI turn ended at an idle prompt (✻Brewed for 1m 13s · done 9:51 AM) after its public error panel: ' +
|
||||
'●API Error: Connection lost mid-response. The response above may be incomplete.');
|
||||
const running = plan.replace('⏸planmodeon (shift+tabtocycle)·←foragents', '⏸planmodeon (shift+tabtocycle)·esctointerrupt·←foragents');
|
||||
const live = await floor([{ onInput: '/plan-devex-review PLAN.md\r', screen: running, effect: delivered }]);
|
||||
expect(live.obs).toMatchObject({ outcome: 'timeout', summary: 'no qualifying finding question within 30000ms' });
|
||||
});
|
||||
|
||||
test.skipIf(process.platform === 'win32')('a prose finding question earlier in the idle turn is assessed before the provider error panel ends the check, even when its transcript line lands after the screen goes idle', async () => {
|
||||
const plan: string = JSON.parse(fs.readFileSync(new URL('../fixtures/pty-idle-turn-end.json', import.meta.url), 'utf8')).planModeApiErrorIdle.visible;
|
||||
const prose = 'Which TTHW target should this journey be measured against?\nA) Champion (< 2 min)\nB) Competitive (2-5 min)';
|
||||
const screen = plan.replace('●API Error:', `${prose.replace(/\n/g, '\r\n')}\r\n●API Error:`);
|
||||
const stubDir = fs.mkdtempSync(path.join(os.tmpdir(), 'floor-judge-stub-'));
|
||||
const stub = path.join(stubDir, 'claude');
|
||||
fs.writeFileSync(stub, '#!/bin/sh\ncat >/dev/null\necho \'{"kind":"finding","seedId":"seed-1","questionId":"prose-1","optionId":"prose-1","reason":"the stub judge credits the staged prose question"}\'\n', { mode: 0o755 });
|
||||
const saved = process.env.BROWSE_TERMINAL_BINARY;
|
||||
process.env.BROWSE_TERMINAL_BINARY = stub;
|
||||
try {
|
||||
const asked = (ctx: FakeSessionContext) => journal(ctx, floorSession(ctx), 'assistant', [{ type: 'text', text: prose }]);
|
||||
const sameTick = await floor([{ onInput: '/plan-devex-review PLAN.md\r', screen, effect: ctx => { delivered(ctx); asked(ctx); } }]);
|
||||
const lateLine = await floor([{ onInput: '/plan-devex-review PLAN.md\r', screen, effect: delivered }, { afterMs: 4000, screen, effect: asked }]);
|
||||
for (const { obs } of [sameTick, lateLine]) {
|
||||
expect(obs).toMatchObject({ outcome: 'auq_observed', auqObserved: true });
|
||||
expect(obs.summary).toContain('Current prose question addresses the owned seeded finding');
|
||||
}
|
||||
} finally {
|
||||
if (saved === undefined) delete process.env.BROWSE_TERMINAL_BINARY; else process.env.BROWSE_TERMINAL_BINARY = saved;
|
||||
fs.rmSync(stubDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('permission prompt: an unowned permission is neither granted nor counted', async () => {
|
||||
const { obs, fake } = await floor([
|
||||
{ onInput: '/plan-devex-review PLAN.md\r', screen: PERMISSION, effect: delivered },
|
||||
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
classifyPlanCountFrame,
|
||||
idleTurnEnd,
|
||||
} from './claude-pty-runner';
|
||||
import { screenPanelEnd } from './pty/screen';
|
||||
|
||||
describe('saved preference annotation', () => {
|
||||
test('recognizes the explicit preference attribution from the timed-out CEO capture', () => {
|
||||
@@ -252,6 +253,25 @@ describe('idle turn end (captured CI screens)', () => {
|
||||
expect(idleTurnEnd(scope + '\r' + question)).toBeNull();
|
||||
});
|
||||
|
||||
test('the 40851840940 plan-mode idle footer ends the turn and names its provider error for the ledger', () => {
|
||||
const plan: string = captures.planModeApiErrorIdle.visible;
|
||||
const panel = '●API Error: Connection lost mid-response. The response above may be incomplete.';
|
||||
expect(idleTurnEnd(plan)).toEqual({ done: '✻Brewed for 1m 13s · done 9:51 AM', apiError: panel });
|
||||
expect(screenPanelEnd(plan)).toEqual({ end: 'api_error', evidence: panel });
|
||||
expect(idleTurnEnd(plan.replace('⏸planmodeon (shift+tabtocycle)·←foragents', '⏸ plan mode on (shift+tab to cycle) · ← for agents')))
|
||||
.toMatchObject({ apiError: panel });
|
||||
});
|
||||
|
||||
test('negative controls: the running plan-mode footer and extra plan-mode footer text are not idle', () => {
|
||||
const plan: string = captures.planModeApiErrorIdle.visible;
|
||||
const footer = '⏸planmodeon (shift+tabtocycle)·←foragents';
|
||||
expect(plan.split(footer)).toHaveLength(2);
|
||||
expect(idleTurnEnd(plan.replace(footer, '⏸planmodeon (shift+tabtocycle)·esctointerrupt·←foragents'))).toBeNull();
|
||||
expect(idleTurnEnd(plan.replace(footer, footer + ' · 1 background task'))).toBeNull();
|
||||
expect(idleTurnEnd(plan.replace(footer, '⏸planmodeon (shift+tabtocycle)'))).toBeNull();
|
||||
expect(screenPanelEnd(plan.replace(footer, '⏸planmodeon (shift+tabtocycle)·esctointerrupt·←foragents'))).toBeNull();
|
||||
});
|
||||
|
||||
test('an API Error mention in prose or in an earlier turn is not this turn\'s error panel', () => {
|
||||
expect(idleTurnEnd('The docs mention API Error: as an example.\r✻ Worked for 3s · done 1:00 PM\r❯ \r← for agents')).toEqual({ done: '✻ Worked for 3s · done 1:00 PM' });
|
||||
const earlier = api.replace(/✻ Sautéedfor1m 20s · done11:34PM[\s\S]*$/, '✻ Sautéedfor1m 20s · done11:34PM\r❯ /plan-design-review\r●Done.\r✻ Worked for 2s · done 11:35PM\r❯ \r← for agents');
|
||||
|
||||
@@ -18,7 +18,7 @@ import { capturePlanCountQuestion, createPlanCountPermissionGuard, matchesNative
|
||||
import { resolveClaudeBinary } from '../binary';
|
||||
import { SANCTIONED_WRITE_SUBSTRINGS, isProseAUQVisible, planCountSubmissionInput } from '../classify';
|
||||
import type { ClaudePtySession } from '../launch';
|
||||
import { isNumberedOptionListVisible, isPermissionDialogVisible, isPlanReadyVisible, isRejectedSlashCommand, stripPtyResidue } from '../screen';
|
||||
import { idleTurnEnd, isNumberedOptionListVisible, isPermissionDialogVisible, isPlanReadyVisible, isRejectedSlashCommand, stripPtyResidue } from '../screen';
|
||||
import { realPtyDriver, runPtySession, type PtyDriver, type PtyStep } from '../session';
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────────
|
||||
@@ -41,7 +41,8 @@ export interface PlanSkillFloorObservation {
|
||||
| 'exited'
|
||||
| 'timeout'
|
||||
| 'assessment_error'
|
||||
| 'permission_unanswered';
|
||||
| 'permission_unanswered'
|
||||
| 'api_error';
|
||||
summary: string;
|
||||
/** Public current viewport for an accepted finding; terminal tail otherwise. */
|
||||
evidence: string;
|
||||
@@ -192,7 +193,7 @@ export async function runPlanSkillFloorCheck(opts: PlanSkillFloorOptions): Promi
|
||||
since: 0, commandStartedAt: 0, targetDelivery: undefined, saveSnapshot: undefined, nativeCandidates: [],
|
||||
validatedPendingQuestion: undefined, sampledAt: undefined, lastCheckpointAt: 0, lastCheckpointState: '',
|
||||
artifactError: undefined, finished: false, started: false, start: 0, deadlineAt: 0, screenDeadlineAt: 0,
|
||||
permissionUnansweredSince: undefined,
|
||||
permissionUnansweredSince: undefined, idleVisible: undefined, idleSince: 0,
|
||||
} as unknown as FloorRun;
|
||||
return runPtySession<PlanSkillFloorObservation>({
|
||||
driver, budgetMs: timeoutMs,
|
||||
@@ -271,6 +272,9 @@ interface FloorRun {
|
||||
screenDeadlineAt: number;
|
||||
/** When the current permission render was first left unanswered by this actor. */
|
||||
permissionUnansweredSince: number | undefined;
|
||||
/** The idle turn end with an error panel being settled, and when it was first seen unchanged. */
|
||||
idleVisible: string | undefined;
|
||||
idleSince: number;
|
||||
}
|
||||
|
||||
async function floorStart(run: FloorRun, session: ClaudePtySession): Promise<undefined> {
|
||||
@@ -401,9 +405,24 @@ async function floorTick(run: FloorRun, session: ClaudePtySession): Promise<PtyS
|
||||
}
|
||||
const question = floorQuestion(run, session, currentCalls);
|
||||
if (question) return question;
|
||||
return floorAssess(run, session, visible);
|
||||
const assessed = floorAssess(run, session, visible);
|
||||
if (assessed !== 'continue') return assessed;
|
||||
const idle = idleTurnEnd(visible);
|
||||
if (!idle?.apiError || visible !== run.idleVisible) {
|
||||
run.idleVisible = idle?.apiError ? visible : undefined;
|
||||
run.idleSince = driver.now();
|
||||
return 'continue';
|
||||
}
|
||||
if (driver.now() - run.idleSince < IDLE_SETTLE_MS) return 'continue';
|
||||
return finish({ auqObserved: false, outcome: 'api_error',
|
||||
summary: `the CLI turn ended at an idle prompt (${idle.done}) after its public error panel: ${idle.apiError}`,
|
||||
evidence: visible.slice(-3000), elapsedMs: driver.now() - run.startedAt });
|
||||
}
|
||||
|
||||
/** An idle turn end must hold still this long, re-reading the transcript each tick, before its error panel ends the
|
||||
* check, so a question whose transcript line lands after the screen draws "done" is still credited. */
|
||||
const IDLE_SETTLE_MS = 10_000;
|
||||
|
||||
/** This actor answers only owned file grants; any other permission render fails fast with a named cause. */
|
||||
const PERMISSION_UNANSWERED_MS = 20_000;
|
||||
|
||||
|
||||
@@ -424,7 +424,7 @@ export function idlePanelEnd(visible: string): { end: 'refusal' | 'api_error'; e
|
||||
}
|
||||
|
||||
/** The last idle turn's error panel, else the CLI's rejected-credentials line anywhere on screen: after one the
|
||||
* CLI can sit at a footer the idle check does not know (plan mode, manual mode), and no turn can run. */
|
||||
* CLI can sit at a footer the idle check does not know (manual mode), and no turn can run. */
|
||||
export function screenPanelEnd(visible: string): { end: 'refusal' | 'api_error'; evidence: string } | null {
|
||||
const credential = visible.split(/\r\n?|\n/).reverse().find(line => CREDENTIAL_REJECTION.test(line))?.trim();
|
||||
return idlePanelEnd(visible) ?? (credential ? { end: 'api_error', evidence: credential } : null);
|
||||
@@ -432,8 +432,9 @@ export function screenPanelEnd(visible: string): { end: 'refusal' | 'api_error';
|
||||
|
||||
/** The CLI's end-of-turn line, e.g. "✻ Cooked for 24s · done 12:46 AM" (spaces may collapse in the PTY text). */
|
||||
const TURN_DONE_RE = /✻[ \t]*\p{L}[\p{L}'’-]*[ \t]*for[ \t]*(?=\d)(?:\d+h[ \t]*)?(?:\d+m[ \t]*)?(?:\d+s)?[ \t]*·[ \t]*done[^\n]*/gu;
|
||||
/** An empty input prompt and its known idle footers, nothing else. */
|
||||
const IDLE_PROMPT_RE = /^[\s─]*❯[\s─]*(?:(?:←[ \t]*for[ \t]*agents|\?[ \t]*for[ \t]*shortcuts)[\s─]*)*$/;
|
||||
/** An empty input prompt and its known idle footers, nothing else. The plan-mode footer is idle only
|
||||
* without its running "esc to interrupt" segment (run 40851840940, plan-devex-finding-floor). */
|
||||
const IDLE_PROMPT_RE = /^[\s─]*❯[\s─]*(?:(?:(?:⏸[ \t]*plan[ \t]*mode[ \t]*on[ \t]*\(shift\+tab[ \t]*to[ \t]*cycle\)[ \t]*·[ \t]*)?←[ \t]*for[ \t]*agents|\?[ \t]*for[ \t]*shortcuts)[\s─]*)*$/;
|
||||
|
||||
/**
|
||||
* The CLI ended its turn and sits at an empty prompt: the last end-of-turn
|
||||
|
||||
+20
-15
@@ -11,14 +11,26 @@ usePrivateStateRoot();
|
||||
const CAREFUL_SCRIPT = path.join(ROOT, 'careful', 'bin', 'check-careful.sh');
|
||||
const FREEZE_SCRIPT = path.join(ROOT, 'freeze', 'bin', 'check-freeze.sh');
|
||||
|
||||
function runHook(scriptPath: string, input: object, env?: Record<string, string>, cwd?: string): { exitCode: number; output: any; raw: string } {
|
||||
// A hook spawn's kill sits under the runner's 30s per-test timeout, not at 5s.
|
||||
// Windows Free Tests, 39 runs: every check-careful test p99.9 1.8s, max 3.3s
|
||||
// (n=8736), except the file's first hook spawn (p50 1.5s, max 3.0s passing),
|
||||
// which the old 5s kill cut down three times on 2026-10-29 and reported as
|
||||
// "exited 1". A killed or unspawnable hook now throws with its elapsed time.
|
||||
const HOOK_SPAWN_TIMEOUT_MS = 25_000;
|
||||
|
||||
function spawnHook(scriptPath: string, rawInput: string, env?: Record<string, string>, cwd?: string): { exitCode: number; output: any; raw: string } {
|
||||
const started = performance.now();
|
||||
const result = spawnSync('bash', [scriptPath], {
|
||||
input: JSON.stringify(input),
|
||||
input: rawInput,
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
env: { ...process.env, ...env },
|
||||
cwd,
|
||||
timeout: 5000,
|
||||
timeout: HOOK_SPAWN_TIMEOUT_MS,
|
||||
});
|
||||
if (result.error || result.signal) {
|
||||
const elapsed = Math.round(performance.now() - started);
|
||||
throw new Error(`${path.basename(scriptPath)} did not exit on its own: ${result.signal ?? result.error?.message} after ${elapsed}ms (kill at ${HOOK_SPAWN_TIMEOUT_MS}ms); stderr: ${result.stderr?.toString().trim() || '(empty)'}`);
|
||||
}
|
||||
const raw = result.stdout.toString().trim();
|
||||
let output: any = {};
|
||||
try {
|
||||
@@ -27,6 +39,10 @@ function runHook(scriptPath: string, input: object, env?: Record<string, string>
|
||||
return { exitCode: result.status ?? 1, output, raw };
|
||||
}
|
||||
|
||||
function runHook(scriptPath: string, input: object, env?: Record<string, string>, cwd?: string): { exitCode: number; output: any; raw: string } {
|
||||
return spawnHook(scriptPath, JSON.stringify(input), env, cwd);
|
||||
}
|
||||
|
||||
// Scratch git repo with a resolvable origin default branch — the HIGH-tier
|
||||
// force-push check reads `git symbolic-ref refs/remotes/origin/HEAD` from the
|
||||
// hook's cwd, and Conductor worktrees don't reliably carry that ref.
|
||||
@@ -46,18 +62,7 @@ function withGitRepo(defaultBranch: string, currentBranch: string, fn: (repoDir:
|
||||
}
|
||||
|
||||
function runHookRaw(scriptPath: string, rawInput: string, env?: Record<string, string>): { exitCode: number; output: any; raw: string } {
|
||||
const result = spawnSync('bash', [scriptPath], {
|
||||
input: rawInput,
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
env: { ...process.env, ...env },
|
||||
timeout: 5000,
|
||||
});
|
||||
const raw = result.stdout.toString().trim();
|
||||
let output: any = {};
|
||||
try {
|
||||
output = JSON.parse(raw);
|
||||
} catch {}
|
||||
return { exitCode: result.status ?? 1, output, raw };
|
||||
return spawnHook(scriptPath, rawInput, env);
|
||||
}
|
||||
|
||||
function carefulInput(command: string) {
|
||||
|
||||
@@ -2,6 +2,7 @@ import { expect, test } from 'bun:test';
|
||||
import { existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join, resolve } from 'node:path';
|
||||
import { killGroup } from '../lib/pregate/supervise';
|
||||
import { expectMentions } from './helpers/prompt-structure';
|
||||
|
||||
const template = readFileSync(new URL('../ship/sections/pr-body.md.tmpl', import.meta.url), 'utf8');
|
||||
@@ -34,11 +35,23 @@ const composeBlock = template.match(/```bash\n(: "\$\{DOCS_SECTION_FILE:[\s\S]*?
|
||||
const editBlock = fenceAfter('**Existing open PR/MR**');
|
||||
const createBlock = fenceAfter('**No open PR/MR:**');
|
||||
|
||||
/**
|
||||
* One deadline per case, under the runner's 30s per-test timeout. On Windows Free
|
||||
* Tests the first posting case in the file took 2.4-25.2s (40 runs) and four times
|
||||
* hit bun's timeout, which killed only the bash child: its descendants kept the root
|
||||
* as their working directory, so cleanup failed with EBUSY and hid the timeout. A
|
||||
* block still running at the deadline is killed with its whole tree, and the error
|
||||
* names the stage and what the earlier stages took.
|
||||
*/
|
||||
const CASE_DEADLINE_MS = 27_000;
|
||||
|
||||
/**
|
||||
* A repo whose origin is GitHub, a stub gh that records argv and stdin, and
|
||||
* the real gstack-post and gstack-prose-lint at the installed path the blocks call.
|
||||
*/
|
||||
function publicationRepo(root: string) {
|
||||
function publicationRepo(root: string, deadlineMs = CASE_DEADLINE_MS) {
|
||||
const started = performance.now();
|
||||
const deadline = started + deadlineMs;
|
||||
const log = join(root, 'log');
|
||||
const stubs = join(root, 'stubs');
|
||||
const bin = join(root, '.claude/skills/gstack/bin');
|
||||
@@ -61,10 +74,25 @@ for a in "$@"; do case "$a" in --title=*) printf '%s\\n' "\${a#--title=}" > "${l
|
||||
[ "$1 $2" = "pr create" ] && echo "https://github.com/o/r/pull/9"
|
||||
exit 0
|
||||
`, { mode: 0o755 });
|
||||
const run = (script: string) => Bun.spawnSync(['bash', '-c', script], {
|
||||
cwd: root, env: { ...process.env, ...gitEnv, HOME: root, PATH: `${stubs}:${process.env.PATH}`, DOCS_SECTION_FILE: join(root, 'section.md'), TMPDIR: root },
|
||||
stdout: 'pipe', stderr: 'pipe', timeout: 30_000,
|
||||
});
|
||||
const stages = [`setup ${Math.round(performance.now() - started)}ms`];
|
||||
const run = async (stage: string, script: string) => {
|
||||
const stageStart = performance.now();
|
||||
const child = Bun.spawn(['bash', '-c', script], {
|
||||
cwd: root, env: { ...process.env, ...gitEnv, HOME: root, PATH: `${stubs}:${process.env.PATH}`, DOCS_SECTION_FILE: join(root, 'section.md'), TMPDIR: root },
|
||||
stdout: 'pipe', stderr: 'pipe', detached: process.platform !== 'win32',
|
||||
});
|
||||
const output = Promise.all([new Response(child.stdout).text(), new Response(child.stderr).text()]);
|
||||
const finished = Promise.all([child.exited, output]);
|
||||
const expired = await Promise.race([finished.then(() => false), Bun.sleep(Math.max(0, deadline - performance.now())).then(() => true)]);
|
||||
if (expired) {
|
||||
killGroup(child.pid, 'SIGKILL');
|
||||
await child.exited;
|
||||
throw new Error(`${stage} still running at the ${deadlineMs}ms case deadline after ${Math.round(performance.now() - stageStart)}ms; process tree killed; earlier stages: ${stages.join(', ')}`);
|
||||
}
|
||||
stages.push(`${stage} ${Math.round(performance.now() - stageStart)}ms`);
|
||||
const [stdout, stderr] = await output;
|
||||
return { exitCode: child.exitCode, stdout, stderr };
|
||||
};
|
||||
const calls = () => readdirSync(log).filter(f => f.endsWith('.args')).sort().map(f => ({
|
||||
args: readFileSync(join(log, f), 'utf8').split('\0').slice(0, -1),
|
||||
stdin: readFileSync(join(log, f.replace('.args', '.stdin')), 'utf8'),
|
||||
@@ -73,16 +101,16 @@ exit 0
|
||||
}
|
||||
|
||||
/** Compose the body with the real compose block and write the Step 18 title file. */
|
||||
function composed(root: string, opts: { title?: string; top?: string } = {}) {
|
||||
async function composed(root: string, opts: { title?: string; top?: string } = {}) {
|
||||
const repo = publicationRepo(root);
|
||||
writeFileSync(join(root, 'section.md'), '**Status:** current — no edits.');
|
||||
const compose = repo.run(writeDrafts(root, opts.top)(composeBlock));
|
||||
const compose = await repo.run('compose', writeDrafts(root, opts.top)(composeBlock));
|
||||
expect(compose.exitCode, compose.stderr.toString()).toBe(0);
|
||||
const bodyFile = /^PR_BODY_FILE: (\S+)$/m.exec(compose.stdout.toString())![1]!;
|
||||
writeFileSync(join(root, '.gstack/tmp/pr-title.abc123'), (opts.title ?? 'v1.2.3.4 fix: publication checks') + '\n');
|
||||
const fill = (block: string) => `PR_BODY_FILE=${JSON.stringify(bodyFile)}\n` +
|
||||
block.replace('<title-file-name>', 'pr-title.abc123').replaceAll('<pr-number>', '9').replace('<base>', 'main');
|
||||
return { ...repo, bodyFile, publish: (block: string) => repo.run(fill(block)) };
|
||||
return { ...repo, bodyFile, publish: (block: string) => repo.run('publish', fill(block)) };
|
||||
}
|
||||
|
||||
// The posting cases drive a POSIX stub gh that gstack-post spawns; Windows CreateProcess cannot
|
||||
@@ -90,15 +118,15 @@ function composed(root: string, opts: { title?: string; top?: string } = {}) {
|
||||
// test/gstack-post.test.ts.
|
||||
const postsThroughStubGh = (failure: string) => process.platform === 'win32' && failure === 'none';
|
||||
for (const failure of ['none', 'body', 'title'] as const) {
|
||||
test.skipIf(postsThroughStubGh(failure))(`publication posts only scanned bytes through gstack-post: ${failure}`, () => {
|
||||
test.skipIf(postsThroughStubGh(failure))(`publication posts only scanned bytes through gstack-post: ${failure}`, async () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'ship-post-'));
|
||||
try {
|
||||
const secret = 'AKIA' + '1234567890ABCDEF';
|
||||
const c = composed(root, {
|
||||
const c = await composed(root, {
|
||||
top: failure === 'body' ? `## Summary\nkey ${secret}\n\n## Documentation` : undefined,
|
||||
title: failure === 'title' ? `v1.2.3.4 fix: ${secret}` : undefined,
|
||||
});
|
||||
const result = c.publish(createBlock);
|
||||
const result = await c.publish(createBlock);
|
||||
if (failure === 'none') {
|
||||
expect(result.exitCode, result.stderr.toString()).toBe(0);
|
||||
expect(result.stdout.toString()).toContain('https://github.com/o/r/pull/9');
|
||||
@@ -114,18 +142,56 @@ for (const failure of ['none', 'body', 'title'] as const) {
|
||||
});
|
||||
}
|
||||
|
||||
test.skipIf(process.platform === 'win32')('MEDIUM findings publish nothing until the block reruns with the printed token', () => {
|
||||
/** True when the pid is gone within a second of the kill. */
|
||||
async function exitsSoon(pid: number) {
|
||||
for (let i = 0; i < 50; i++) {
|
||||
try { process.kill(pid, 0); await Bun.sleep(20); } catch { return true; }
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
test('a block still running at the case deadline is killed with its whole tree before cleanup', async () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'ship-post-deadline-'));
|
||||
try {
|
||||
const pidFile = join(root, 'holder.pid');
|
||||
const repo = publicationRepo(root, 1_500);
|
||||
await expect(repo.run('publish', `sh -c 'echo $$ > "${pidFile}"; exec sleep 30'\necho unreachable`))
|
||||
.rejects.toThrow(/^publish still running at the 1500ms case deadline after \d+ms; process tree killed; earlier stages: setup \d+ms$/);
|
||||
if (process.platform !== 'win32') expect(await exitsSoon(Number(readFileSync(pidFile, 'utf8')))).toBe(true);
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
expect(existsSync(root)).toBe(false);
|
||||
} finally {
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// A block that exits while a background child still holds its output is still running: the deadline covers
|
||||
// the pipes too, not only bash's exit. POSIX only: on Windows the exited bash leaves no tree for taskkill /T.
|
||||
test.skipIf(process.platform === 'win32')('a block whose background child holds its output past the deadline is killed too', async () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'ship-post-pipe-'));
|
||||
try {
|
||||
const pidFile = join(root, 'holder.pid');
|
||||
const repo = publicationRepo(root, 1_500);
|
||||
await expect(repo.run('publish', `sleep 30 &\necho $! > "${pidFile}"`))
|
||||
.rejects.toThrow(/^publish still running at the 1500ms case deadline after \d+ms; process tree killed; earlier stages: setup \d+ms$/);
|
||||
expect(await exitsSoon(Number(readFileSync(pidFile, 'utf8')))).toBe(true);
|
||||
} finally {
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test.skipIf(process.platform === 'win32')('MEDIUM findings publish nothing until the block reruns with the printed token', async () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'ship-post-medium-'));
|
||||
try {
|
||||
const c = composed(root, { top: '## Summary\nThanks jane.roe@acme-corp.io\n\n## Documentation' });
|
||||
const first = c.publish(editBlock);
|
||||
const c = await composed(root, { top: '## Summary\nThanks jane.roe@acme-corp.io\n\n## Documentation' });
|
||||
const first = await c.publish(editBlock);
|
||||
expect(first.exitCode).toBe(2);
|
||||
expect(first.stdout.toString()).toContain('RULE: pii.email LINE: 2 PART: body');
|
||||
expect(c.calls()).toEqual([]);
|
||||
const token = /^TOKEN: (\S+)$/m.exec(first.stdout.toString())![1]!;
|
||||
const flagged = 'pr-body <pr-number> --body-file "${PR_BODY_FILE:?restore the composed body path}"';
|
||||
expect(editBlock).toContain(flagged);
|
||||
const then = c.publish(editBlock.replace(flagged, `${flagged} --confirm ${token}`));
|
||||
const then = await c.publish(editBlock.replace(flagged, `${flagged} --confirm ${token}`));
|
||||
expect(then.exitCode, then.stderr.toString()).toBe(0);
|
||||
const calls = c.calls();
|
||||
expect(calls.map(call => call.args)).toEqual([
|
||||
|
||||
Reference in new issue
Block a user