Merge remote-tracking branch 'origin/main' into capy/bun-entry-isolation

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

# Conflicts:
#	CHANGELOG.md
This commit is contained in:
garrytan committed 2026-10-19 22:05:05 +00:00
commit ad797caa85
34 files changed
+886 -117

No files matched your search

+1 -1
View File
@@ -91,7 +91,7 @@ The server writes `.gstack/browse.json` (atomic write via tmp + rename, mode 0o6
{ "pid": 12345, "port": 34567, "token": "uuid-v4", "startedAt": "...", "binaryVersion": "abc123" }
```
The CLI reads this file to find the server. If the file is missing or the daemon process is dead, the CLI spawns a new server. A process that is alive but not answering `/health` is busy, not dead: the CLI probes for a bounded ~8s, then reports busy with a nonzero exit — only an explicit `--force-restart` kills a live daemon. Process liveness uses signal-0 (`isProcessAlive`, EPERM counts as alive) on every platform, with the health check (GET /health) as the responsiveness signal. Daemon stdout/stderr persists to `<project>/.gstack/browse-daemon.log`.
The CLI reads this file to find the server. If the file is missing or the daemon process is dead, the CLI spawns a new server. A process that is alive but not answering `/health` is busy, not dead: the CLI probes for a bounded ~8s, then reports busy with a nonzero exit — only an explicit `--force-restart` kills a live daemon. Process liveness uses signal-0 (`isProcessAlive`, EPERM counts as alive) on every platform, with the health check (GET /health) as the responsiveness signal. Its `status` covers the daemon and its Chromium connection only; the active tab's main thread is reported separately (`activeTab.responsive`, a probe bounded to 1s), so one page stuck in a long script never makes the CLI refuse commands. Daemon stdout/stderr persists to `<project>/.gstack/browse-daemon.log`.
### Port selection
+8
View File
@@ -626,6 +626,14 @@ Storage stays intact by default. With explicit approval on a Chromium target, `-
| `tab-each <command> [args...]` | Fan out a command across every open tab; returns JSON |
| `frame <sel\|@ref\|--name n\|--url pattern\|main>` | Switch to iframe context (or back to main); clears refs |
**A busy tab is not a busy daemon.** `/health` `status` is daemon liveness
(Chromium connected); the active tab's main thread is reported separately as
`activeTab: { id, responsive }`, probed for at most 1s. While one page is stuck
in a long script, `status`, `tabs`, `tab <id>`, `newtab`, `closetab`,
`state save` and `stop` keep working, and `status` and `tabs` mark the tab as
not responding. A command that needs the stuck tab times out after 30s and
names it, so you can switch away or close it without `--force-restart`.
### Extraction
| Command | Description |
+43
View File
@@ -22,6 +22,49 @@
- **…add a bin or a hook:** start it with the isolated shebang, and pass `bun --no-env-file --config=/dev/null` (or `bunIsolationArgs()` in TypeScript) wherever you start Bun. If you don't, the ratchet names the line.
- **…relied on a repo `.env` to configure gstack:** export the variable in your shell or set it with `gstack-config`.
## [1.91.156.0] - 2026-10-19
**The weekly marathon's office-hours case can run its review formatter again, and its validator now checks cap fixes the way the formatter does.** Both defects came from #3164 and were in the test, not the skill. The marathon fixture copied `bin/gstack-office-hours-review` with a hand-kept list of two library files; #3164 added a third import (`lib/spec-review-cap.ts`), so every formatter command in the fixture failed at module load, the agent correctly declared the review UNREVIEWED, and the case went red in marathon run 39032308051 (#3246). With the module restored, a focused rerun showed the second defect: when the review stops at the round cap and the agent applies the remaining blocking fixes with `--fixes`, the formatter records them as verified, but the test never gave the validator that list, so it expected "fixes unconfirmed" and rejected a correct report.
### Fixed
- **Office-hours cap fixes:** `observedSpecFixes()` in `test/helpers/office-hours-completion.ts` returns the last valid `--fixes` list the agent saved with Write, and the marathon test passes it to `validateOfficeHoursReviewArtifacts`. A free test in `test/office-hours-completion.test.ts` runs the real `finalize --fixes` at the cap and checks three cases: the observed list validates, no list fails with the marathon rerun's `Spec Review does not match` error, and a list naming a different finding also fails.
- **Office-hours marathon fixture:** `test/helpers/office-hours-formatter-fixture.ts` copies the formatter together with every local module it imports, so a new import can no longer be left out. A free test in `test/office-hours-review.test.ts` loads the copied formatter and fails on a missing module; the old three-file list reproduces the exact `Cannot find module '../lib/spec-review-cap.ts'` error. The case's touchfiles now include `lib/spec-review-cap.ts` and the new helper.
- **Gate integrity:** no threshold, trial count, case kind or judge changed. The marathon assertions are unchanged; only the fixture's file copy changed.
## [1.91.155.0] - 2026-10-19
**A busy page no longer locks up the browse daemon.** When one tab kept its main thread busy (a heavy dev bundle, a big grid render, a runaway script), every `browse` command, including `status`, `tabs`, `tab <id>`, `closetab` and `stop`, failed with "Daemon busy" after about 9.6 s, and the only way out was `--force-restart`, which threw away cookies, tabs and logins. Fixes #3247. From #3250 by @AndreIntelas.
### Fixed
- **`/health` `status` reports daemon liveness (Chromium connected)**, and the active tab's responsiveness is reported separately as `activeTab: {id, responsive}`. The CLI no longer gates every command on a 2 s `page.evaluate` race against the active tab.
- **`tabs` and `state save` bound their per-tab reads**, so a stuck tab shows as `(not responding)` instead of hanging the command, and one pending probe is shared per stuck tab rather than one per poll. A busy tab keeps its last good storage in saved state.
- **A command that really needs the stuck tab times out with an error naming that tab** and how to get out (switch tabs or close it). The CLI's timeout branch now matches Bun's `TimeoutError`; the old `AbortError` check never fired.
### Changed
- **Gate integrity:** `browse/test/terminal-agent.test.ts` adds `activeTab` to the exact `/health` keys and updates its stub for the now-synchronous `isHealthy` and the new `probeActiveTab`; its no-token assertions are unchanged. `browse/test/busy-active-tab.test.ts` and `browse/test/command-timeout-report.test.ts` are new. No threshold, trial count or case kind changed.
## [1.91.154.0] - 2026-10-19
**/cso can verify Rails repairs in a read-only checkout, its retained Git history no longer brings back credential files the snapshot leaves out, a repair whose external assertions fail is now reported as `ASSERTION_FAILED` instead of a misleading `INCOMPATIBLE_INPUT`, and a security assertion can accept several safe rejection statuses while still requiring the secret to be absent.** The helper keeps `.env`, `master.key`, keys and agent configuration out of the readable tree, but `history.txt` captured `git log -p` in full, so a committed `.env` came back through its diff whenever the value wasn't credential-shaped enough for the redactor. Paid qualification run 39186328569 caught this when the private corpus's planted `.env` value showed up in the judge's helper state.
### Fixed
- **Rails `verify` on read-only sources:** the helper writes its own test `config/database.yml` (plus an initializer and `storage.yml`) into the execution copy. The copy keeps the source's file modes, so in a read-only checkout (the eval producer seals every source file at 0444, and Nix or Bazel trees are the same) the write failed with `EACCES`. That surfaced as the generic `INVALID_SCHEMA` "unexpected or unsafe input", and no Rails repair could reach `runtime_tested`. Each helper file now replaces the copied file instead of opening it for writing, so it also never writes through a copied symlink.
- **`history.txt`:** every diff section that touches a path the snapshot excludes (`exclusion()`, the same rule the readable tree uses) is replaced with `diff withheld: <path> (<reason>)`. This covers either side of a rename and any diff header that can't be parsed. Commit preambles and other files' diffs stay.
- **`verify` with an authenticated witness:** a receipt or observation mismatch is still `INCOMPATIBLE_INPUT`. Receipts that bind correctly but record failed external assertions now raise `ASSERTION_FAILED`, with the phase and the booted, legitimate and security outcome. Nothing new is certified either way. In run 39186328569, three repairs that rejected the attack with a different status than the oracle expected surfaced as `INCOMPATIBLE_INPUT`.
### Changed
- **A security assertion's `expected.status` may list up to eight safe statuses** (for example `[403, 404]`), so a repair that refuses an attack with a different reasonable rejection still passes. A set is accepted only on a security assertion that also carries a `vulnerable` expectation, and only with `excludes` set, so the response must still withhold the vulnerable marker. The fixed and vulnerable oracles must stay provably mutually exclusive: a set that contains the vulnerable status is accepted only when the vulnerable response's `includes` contains the excluded marker. Legitimate controls and `vulnerable` expectations still take exactly one status, so a patch that answers every route with 403 still fails its controls. A no-op patch, a listed rejection that still leaks the marker, and an unlisted status all stay `inconclusive`.
## [1.91.153.0] - 2026-10-19
**An outside review whose findings are untagged prose no longer reads `clean` because a "no issues" phrase appears somewhere in it, and a clean Codex review that says "no actionable regressions" no longer reads `unverified`.** Reported in #3294 by @SchizasNik, with the false-pass half pointed out by @xg-gh-25. `lib/outside-review-result.ts` matched its no-findings vocabulary anywhere in the reply, so "The parser change has no new issues. However, … drops users_email_idx" passed /ship's structured gate. It also lacked `regressions`, so `codex review --uncommitted`'s "I found no actionable regressions in the staged, unstaged, or untracked changes." was missing coverage.
### Fixed
- **A no-findings statement counts only as the review's conclusion.** It must open the reply's last sentence or last ", and" clause (or the `because` clause of a review's `Recommendation:` line) and carry no contrast or exception ("but", "however", "except", "other than", "elsewhere"). When other prose comes first, anything after the statement other than a predicate ("were found", "is evident") or a colon explanation must scope it to the change as a whole, so "No issues in the parser." and "No regressions were found by the existing tests." do not conclude the review. Anything else stays `unverified` / `untagged_review`, the same reason code and troubleshooting anchor as before.
- **`regressions` and `defects` join the vocabulary**, and `NO_FINDINGS` written as inline code still counts.
### Gate integrity
- The structured and review gates' no-findings check (`classifyOutsideReview`) is stricter about position and wider in vocabulary. New free regressions in `test/outside-review-result.test.ts` cover the reporter's exact reply through their CLI invocation, two clean `codex review --uncommitted` replies captured from codex-cli 0.160.0 with gpt-6-sol (both read `unverified` before), fifteen mixed replies that must stay `unverified` under both gates (the two strings from the issue, the phrase before, after and inside untagged findings, exception conclusions, conclusions scoped to one place or to the tests, a quoted `NO_FINDINGS`, a no-issues line inside a code fence), and Recommendation-clause cases including a wrapped clause with "but". A GPT-6.1 Sol outside review probed the classifier; its false passes are among those controls. A known residual, unchanged from before: untagged findings followed by "No issues found in the changed parser." still read `clean`, because "changed" names the change and the parser cannot tell a finding from a summary without tags. All four new tests fail on the previous classifier. Every existing negative control passes unchanged, including the stored Codex clean review that still reads `unverified` ("no actionable correctness issues"). No threshold, trial count or case kind changed.
## [1.91.152.0] - 2026-10-19
**`/cso` now starts and finishes on ordinary repositories that hold a nested Git repository, a long patch history, or a Docker socket on macOS 27.** Three user reports (#3255, #3248) each stopped a run on a normal repo.
+1 -1
View File
@@ -1 +1 @@
1.91.152.0
1.91.156.0
+1 -1
View File
@@ -1,4 +1,4 @@
# gstack digest v1.91.152.0 — regenerate/re-copy after upgrading gstack
# gstack digest v1.91.156.0 — regenerate/re-copy after upgrading gstack
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
for agent hosts without a full skill install. The full skills add workflows,
+122 -30
View File
@@ -290,6 +290,44 @@ export async function handleChromiumDisconnect(browser: Browser | null): Promise
process.exit(1);
}
/**
* How long a tab-independent read waits on one tab's renderer (#3247). A page
* stuck in a long script never answers page.evaluate() or page.title(); before
* this bound, one such tab made /health call the whole daemon unhealthy and
* hung `tabs`. Kept well under the CLI's /health request timeout
* (HEALTH_REQUEST_TIMEOUT_MS in cli.ts) so the CLI reads the answer instead of
* timing out on it.
*/
export const TAB_PROBE_TIMEOUT_MS = 1_000;
/**
* Budget for saveState()'s per-tab storage read. Longer than the probe: a tab
* that misses it falls back to the last storage it answered with, which may
* be stale, so only a tab that is really stuck takes that path.
*/
const TAB_STORAGE_READ_TIMEOUT_MS = 5_000;
type TabStorage = { localStorage: Record<string, string>; sessionStorage: Record<string, string> };
type TabReadKind = 'probe' | 'title' | 'storage';
function urlOrigin(url: string): string {
return URL.canParse(url) ? new URL(url).origin : '';
}
/**
* Resolve with `work`'s value, or with undefined once `ms` passes. The losing
* read settles on its own when the renderer frees up, so callers pass a promise
* that cannot reject.
*/
function withinTabBudget<T>(work: Promise<T>, ms: number): Promise<T | undefined> {
let timer: ReturnType<typeof setTimeout> | undefined;
const deadline = new Promise<undefined>((resolve) => {
timer = setTimeout(() => resolve(undefined), ms);
(timer as { unref?: () => void }).unref?.();
});
return Promise.race([work, deadline]).finally(() => clearTimeout(timer));
}
export type { RefEntry };
// Re-export TabSession for consumers
@@ -300,7 +338,7 @@ export interface BrowserState {
pages: Array<{
url: string;
isActive: boolean;
storage: { localStorage: Record<string, string>; sessionStorage: Record<string, string> } | null;
storage: TabStorage | null;
/**
* HTML content loaded via load-html (setContent), replayed after context recreation.
* In-memory only — never persisted to disk (HTML may contain secrets or customer data).
@@ -360,6 +398,13 @@ export class BrowserManager {
private extensionContext = false;
/** Pending and recorded navigation-guard verdicts per page (D2). */
private navigationGuards = new WeakMap<Page, { pending: Set<Promise<void>>; blocked: string | null }>();
/** One pending read per page and kind (#3247). A read that outlives its
* budget stays queued in the renderer until the page frees up, so a stuck
* tab reuses it instead of collecting a new one on every /health poll,
* `tabs` call or persist tick. */
private pendingTabReads = new WeakMap<Page, Map<TabReadKind, Promise<unknown>>>();
/** The last storage each page answered with, and the origin it came from. */
private lastTabStorage = new WeakMap<Page, { origin: string; storage: TabStorage }>();
// Our own 'disconnected' listeners per browser. Detach only these: removing
// every 'disconnected' listener also strips Playwright's internal one that
// resolves browser.close(), so each close waited the full race timeout (and
@@ -1090,20 +1135,43 @@ export class BrowserManager {
this.closeOwnedDisplay();
}
/** Health check — verifies Chromium is connected AND responsive */
async isHealthy(): Promise<boolean> {
if (!this.browser || !this.browser.isConnected()) return false;
try {
const page = this.pages.get(this.activeTabId);
if (!page) return true; // connected but no pages — still healthy
await Promise.race([
page.evaluate('1'),
new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 2000)),
]);
return true;
} catch {
return false;
/**
* Daemon liveness for /health: Chromium is connected. The active tab is NOT
* part of it (#3247): a page stuck in a long script is one busy tab, not a
* dead daemon, and the CLI gates every command on this answer — including
* the tabs, tab, closetab and stop commands that get out of that state.
* probeActiveTab() reports the tab on its own.
*/
isHealthy(): boolean {
return Boolean(this.browser?.isConnected());
}
/**
* Does the active tab's main thread answer within `timeoutMs`? Null when
* there is no active page. A rejected evaluate (crashed or closing target)
* also counts as not responsive.
*/
async probeActiveTab(timeoutMs = TAB_PROBE_TIMEOUT_MS): Promise<{ id: number; responsive: boolean } | null> {
const id = this.activeTabId;
const page = this.pages.get(id);
if (!page) return null;
const probe = this.sharedTabRead(page, 'probe', () => page.evaluate('1').then(() => true, () => false));
return { id, responsive: (await withinTabBudget(probe, timeoutMs)) === true };
}
/** Join the page's pending read of this kind, or start one. `read` must not
* reject. */
private sharedTabRead<T>(page: Page, kind: TabReadKind, read: () => Promise<T>): Promise<T> {
let reads = this.pendingTabReads.get(page);
if (!reads) {
reads = new Map();
this.pendingTabReads.set(page, reads);
}
const pending = reads.get(kind);
if (pending) return pending as Promise<T>;
const started = read().finally(() => reads.delete(kind));
reads.set(kind, started);
return started;
}
// ─── Tab Management ────────────────────────────────────────
@@ -1301,17 +1369,23 @@ export class BrowserManager {
return released;
}
async getTabListWithTitles(): Promise<Array<{ id: number; url: string; title: string; active: boolean }>> {
const tabs: Array<{ id: number; url: string; title: string; active: boolean }> = [];
for (const [id, page] of this.pages) {
tabs.push({
/**
* Every tab with its title. Titles are read in parallel, each bounded by
* TAB_PROBE_TIMEOUT_MS: a tab whose renderer is stuck comes back with
* `busy: true` and an empty title instead of hanging the list (#3247).
*/
async getTabListWithTitles(): Promise<Array<{ id: number; url: string; title: string; active: boolean; busy: boolean }>> {
return Promise.all([...this.pages].map(async ([id, page]) => {
const title = await withinTabBudget(
this.sharedTabRead(page, 'title', () => page.title().catch(() => '')), TAB_PROBE_TIMEOUT_MS);
return {
id,
url: page.url(),
title: await page.title().catch(() => ''),
title: title ?? '',
active: id === this.activeTabId,
});
}
return tabs;
busy: title === undefined,
};
}));
}
// ─── Session Access ────────────────────────────────────────
@@ -1715,7 +1789,9 @@ export class BrowserManager {
// ─── State Save/Restore (shared by recreateContext + handoff) ─
/**
* Capture browser state: cookies, localStorage, sessionStorage, URLs, active tab.
* Skips pages that fail storage reads (e.g., already closed).
* Skips the storage of pages that fail the read (e.g., already closed).
* A page whose renderer is stuck past TAB_STORAGE_READ_TIMEOUT_MS does not
* hold up `state save` (#3247); see readTabStorage() for what it gets.
*/
async saveState(): Promise<BrowserState> {
if (!this.context) throw new Error('Browser not launched');
@@ -1725,13 +1801,7 @@ export class BrowserManager {
for (const [id, page] of this.pages) {
const url = page.url();
let storage = null;
try {
storage = await page.evaluate(() => ({
localStorage: { ...localStorage },
sessionStorage: { ...sessionStorage },
}));
} catch {}
const storage = await this.readTabStorage(page, url);
// Capture load-html content so a later context recreation (viewport --scale)
// can replay it via setTabContent. Never persisted to disk.
@@ -1753,6 +1823,28 @@ export class BrowserManager {
return { cookies, pages };
}
/**
* One tab's localStorage/sessionStorage for saveState(). A failed read
* (closed page, opaque origin) is null. A read that times out returns the
* last storage this page gave for the same origin: a periodic
* BROWSE_PERSIST_STATE snapshot must not replace good storage on disk with
* null just because the page is busy.
*/
private async readTabStorage(page: Page, url: string): Promise<TabStorage | null> {
const origin = urlOrigin(url);
const read = await withinTabBudget(this.sharedTabRead(page, 'storage', () => page.evaluate(() => ({
localStorage: { ...localStorage },
sessionStorage: { ...sessionStorage },
})).catch(() => null)), TAB_STORAGE_READ_TIMEOUT_MS);
if (read === undefined) {
const last = this.lastTabStorage.get(page);
return last?.origin === origin ? last.storage : null;
}
if (read) this.lastTabStorage.set(page, { origin, storage: read });
else this.lastTabStorage.delete(page);
return read;
}
/**
* Restore browser state into the current context: cookies, pages, storage.
* Navigates to saved URLs, restores storage, wires page events.
+65 -33
View File
@@ -215,39 +215,50 @@ function readState(): ServerState | null {
// isProcessAlive is imported from ./error-handling
/**
* HTTP health check — definitive proof the server is alive and responsive.
* Used in all polling loops instead of isProcessAlive() (which is slow on Windows).
*/
export async function isServerHealthy(port: number, timeoutMs = 2000): Promise<boolean> {
try {
const resp = await fetch(`http://127.0.0.1:${port}/health`, {
signal: AbortSignal.timeout(timeoutMs),
});
if (!resp.ok) return false;
const health = await resp.json() as any;
return health.status === 'healthy';
} catch {
return false;
}
}
/** Client timeout for one GET /health. Must stay above the daemon's active-tab
* probe (TAB_PROBE_TIMEOUT_MS in browser-manager.ts): when the two were equal,
* the CLI aborted just before a busy tab's reply arrived and could not tell a
* busy tab from a hung daemon (#3247). */
export const HEALTH_REQUEST_TIMEOUT_MS = 2000;
/** Best-effort tab count via GET /health (no auth, bounded). Returns null
* when the daemon doesn't answer in time or predates the `tabs` field —
* callers degrade to a countless phrasing, never block on this. */
async function fetchDaemonTabCount(port: number, timeoutMs = 2000): Promise<number | null> {
/** GET /health (no auth, bounded). Null when the daemon doesn't answer in
* time or the reply is not a 200. */
async function fetchDaemonHealth(port: number, timeoutMs = HEALTH_REQUEST_TIMEOUT_MS): Promise<Record<string, unknown> | null> {
try {
const resp = await fetch(`http://127.0.0.1:${port}/health`, {
signal: AbortSignal.timeout(timeoutMs),
});
if (!resp.ok) return null;
const health = await resp.json() as any;
return typeof health.tabs === 'number' ? health.tabs : null;
return await resp.json() as Record<string, unknown>;
} catch {
return null;
}
}
/**
* HTTP health check — definitive proof the server is alive and responsive.
* Used in all polling loops instead of isProcessAlive() (which is slow on Windows).
* `status` is daemon liveness only; a busy active tab does not fail it (#3247).
*/
export async function isServerHealthy(port: number, timeoutMs = HEALTH_REQUEST_TIMEOUT_MS): Promise<boolean> {
return (await fetchDaemonHealth(port, timeoutMs))?.status === 'healthy';
}
/** Best-effort tab count via GET /health. Returns null when the daemon
* doesn't answer in time or predates the `tabs` field — callers degrade to a
* countless phrasing, never block on this. */
async function fetchDaemonTabCount(port: number): Promise<number | null> {
const tabs = (await fetchDaemonHealth(port))?.tabs;
return typeof tabs === 'number' ? tabs : null;
}
/** The active tab's id when /health reports its main thread as not
* responding (#3247), else null — including daemons that predate the field. */
async function fetchBusyActiveTab(port: number): Promise<number | null> {
const activeTab = (await fetchDaemonHealth(port))?.activeTab as { id?: unknown; responsive?: unknown } | null | undefined;
return activeTab?.responsive === false && typeof activeTab.id === 'number' ? activeTab.id : null;
}
// ─── Process Management ─────────────────────────────────────────
async function killServer(pid: number): Promise<void> {
if (!isProcessAlive(pid)) return;
@@ -420,7 +431,7 @@ async function probeHealthWithBackoff(
await Bun.sleep(intervalMs);
const remainingMs = deadline - Date.now();
if (remainingMs <= 0) return false;
if (await isServerHealthy(port, Math.min(2000, remainingMs))) return true;
if (await isServerHealthy(port, Math.min(HEALTH_REQUEST_TIMEOUT_MS, remainingMs))) return true;
}
}
@@ -1034,6 +1045,32 @@ export function extractTabId(args: string[]): { tabId: number | undefined; args:
return { tabId, args: stripped };
}
/** Did this fetch fail because its AbortSignal fired? Bun rejects an
* AbortSignal.timeout() fetch with TimeoutError, not AbortError — matching
* only AbortError left the timeout reports below unreachable. */
export function isFetchTimeout(err: unknown): boolean {
const name = (err as { name?: unknown } | null)?.name;
return name === 'AbortError' || name === 'TimeoutError';
}
/**
* What to print when a command's reply did not arrive within `timeoutMs`.
* #1781: a timeout on a heavy page usually means busy, not dead, so a live
* daemon is never restarted here. #3247: when /health names the active tab
* as not responding, say which tab and the commands that leave it without
* losing the session.
*/
export async function commandTimeoutReport(state: { pid: number; port: number } | null, timeoutMs: number): Promise<string[]> {
const after = `[browse] Command timed out after ${timeoutMs / 1000}s`;
if (!state?.pid || !isProcessAlive(state.pid)) return [after];
const busyTab = await fetchBusyActiveTab(state.port);
if (busyTab === null) return [`${after} (server still alive — busy, not restarting). Retry, or raise load.`];
return [
`${after}: tab ${busyTab} is not responding (its main thread is busy).`,
`[browse] The daemon, its other tabs and cookies still work: switch with 'browse tab <id>', close this one with 'browse closetab ${busyTab}', or retry once the page settles.`,
];
}
// ─── Command Dispatch ──────────────────────────────────────────
export async function sendCommand(state: ServerState, command: string, args: string[], retries = 0): Promise<void> {
// Precedence: CLI --tab-id flag > BROWSE_TAB env var.
@@ -1085,19 +1122,14 @@ export async function sendCommand(state: ServerState, command: string, args: str
process.exit(1);
}
} catch (err: any) {
if (command === 'cookie-import-browser' && (['AbortError', 'TimeoutError'].includes(err.name)
if (command === 'cookie-import-browser' && (isFetchTimeout(err)
|| ['ECONNREFUSED', 'ECONNRESET'].includes(err.code) || err.message?.includes('fetch failed'))) {
throw new Error('Cookie import response was lost or timed out. It may have partially completed; inspect the destination before retrying manually. The command was not replayed.');
}
if (err.name === 'AbortError') {
// #1781: a 30s timeout on a heavy page usually means busy, not dead.
// Don't kill a live server (that's what triggered the crash-loop) — report
// and exit so the user can retry rather than losing their (headed) window.
const ts = readState();
const alive = ts?.pid ? isProcessAlive(ts.pid) : false;
console.error(alive
? '[browse] Command timed out after 30s (server still alive — busy, not restarting). Retry, or raise load.'
: '[browse] Command timed out after 30s');
if (isFetchTimeout(err)) {
// Report and exit so the user can retry rather than losing their
// (headed) window — killing a live server triggered the crash-loop.
for (const line of await commandTimeoutReport(readState(), timeoutMs)) console.error(line);
process.exit(1);
}
// Connection error — server may have crashed, OR may just be busy.
+6 -1
View File
@@ -265,7 +265,7 @@ export async function handleMetaCommand(
case 'tabs': {
const tabs = await bm.getTabListWithTitles();
return tabs.map(t =>
`${t.active ? '→ ' : ' '}[${t.id}] ${t.title || '(untitled)'} — ${t.url}`
`${t.active ? '→ ' : ' '}[${t.id}] ${t.busy ? '(not responding)' : t.title || '(untitled)'} — ${t.url}`
).join('\n');
}
@@ -404,12 +404,17 @@ export async function handleMetaCommand(
case 'status': {
const tabs = bm.getTabCount();
const mode = bm.getConnectionMode();
const activeTab = await bm.probeActiveTab();
return [
`Status: healthy`,
`Mode: ${mode}`,
`URL: ${bm.getCurrentUrl()}`,
`Tabs: ${tabs}`,
`PID: ${process.pid}`,
// #3247: name the busy tab and the ways out that keep the session.
...(activeTab && !activeTab.responsive
? [`Active tab ${activeTab.id}: not responding. Switch with: browse tab <id>, or close it: browse closetab ${activeTab.id}`]
: []),
].join('\n');
}
+7 -1
View File
@@ -77,13 +77,19 @@ export const coreRoutes: RouteEntry[] = [
// Health check — no auth required, does NOT reset idle timer. NEVER carries
// a token in any mode: token bootstrap is POST /extension-token and shell
// auth is POST /pty-session. Liveness/status only.
//
// `status` is daemon liveness (Chromium connected); the CLI gates every
// command on it. `activeTab.responsive` is the active tab's main thread,
// probed for at most TAB_PROBE_TIMEOUT_MS and reported beside it: one busy
// page must not read as a dead daemon (#3247).
{
method: '*', path: '/health', auth: 'none', surfaces: ['local'],
handler: async (_req, _r, ctx) => {
const { browserManager } = ctx;
const healthy = await browserManager.isHealthy();
const healthy = browserManager.isHealthy();
return json({
status: healthy ? 'healthy' : 'unhealthy',
activeTab: healthy ? await browserManager.probeActiveTab() : null,
mode: browserManager.getConnectionMode(),
uptime: Math.floor((Date.now() - ctx.startTime) / 1000),
tabs: browserManager.getTabCount(),
+132
View File
@@ -0,0 +1,132 @@
/**
* #3247: one busy tab must not make the daemon look dead. A renderer stuck in
* a long script never answers page.evaluate() or page.title(). /health folded
* that into its `status`, so the CLI refused every command (status, tabs,
* tab, newtab, stop) with "Daemon busy" until --force-restart threw the
* session away, and `tabs` and `state save` hung on the tab as well.
*
* A real Chromium tab is held in a busy loop while a second tab stays idle;
* each surface must answer within its bound and name the busy tab, without
* piling up reads on the stuck renderer or dropping its persisted storage.
*/
import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import type { Page } from 'playwright';
import { BrowserManager, TAB_PROBE_TIMEOUT_MS } from '../src/browser-manager';
import { HEALTH_REQUEST_TIMEOUT_MS } from '../src/cli';
import { handleMetaCommand } from '../src/meta-commands';
import { persistSessionState } from '../src/session-persist';
import { callRoute, stubRouteContext } from './route-test-harness';
import { startTestServer } from './test-server';
// Outlasts every check below; closing the busy tab at the end stops it sooner.
const BUSY_MS = 30_000;
let testServer: ReturnType<typeof startTestServer>;
let bm: BrowserManager;
let busyId = 0;
let idleId = 0;
let busyPage: Page;
let profileDir = '';
const originalProfile = process.env.CHROMIUM_PROFILE;
beforeAll(async () => {
// Per-file profile, set in a hook (see commands.test.ts for why).
profileDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-busy-tab-profile-'));
process.env.CHROMIUM_PROFILE = profileDir;
testServer = startTestServer(0);
bm = new BrowserManager();
await bm.launch();
busyId = bm.getActiveTabId();
await bm.getPage().goto(`${testServer.url}/basic.html`);
idleId = await bm.newTab(`${testServer.url}/basic.html`);
bm.switchTab(busyId, { bringToFront: false });
busyPage = bm.getPage();
// The good snapshot a BROWSE_PERSIST_STATE tick takes before the page hangs.
await busyPage.evaluate(() => localStorage.setItem('probe', 'kept'));
await bm.saveState();
await busyPage.evaluate((ms) => {
setTimeout(() => { const end = Date.now() + ms; while (Date.now() < end) { /* busy */ } }, 50);
}, BUSY_MS);
await Bun.sleep(300); // let the loop start
}, 30_000);
afterAll(async () => {
try { testServer.server.stop(true); } catch {}
try { await Promise.race([bm?.close(), Bun.sleep(3000)]); } catch {}
if (originalProfile === undefined) delete process.env.CHROMIUM_PROFILE;
else process.env.CHROMIUM_PROFILE = originalProfile;
if (profileDir) fs.rmSync(profileDir, { recursive: true, force: true });
});
describe('busy active tab (#3247)', () => {
test('/health says the daemon is alive and names the busy tab before the CLI gives up', async () => {
// Equal budgets were the bug: the CLI aborted just before the reply landed.
expect(TAB_PROBE_TIMEOUT_MS).toBeLessThan(HEALTH_REQUEST_TIMEOUT_MS);
const ctx = stubRouteContext({
browserManager: bm,
terminal: { readPort: () => null, grantToken: async () => true, restartSession: async () => true },
});
const started = Date.now();
const body = await (await callRoute('GET', '/health', ctx)).json();
expect(Date.now() - started).toBeLessThan(HEALTH_REQUEST_TIMEOUT_MS);
expect(body.status).toBe('healthy');
expect(body.activeTab).toEqual({ id: busyId, responsive: false });
});
test('tabs lists every tab without waiting on the busy one', async () => {
const started = Date.now();
const out = await handleMetaCommand('tabs', [], bm, async () => {});
expect(Date.now() - started).toBeLessThan(HEALTH_REQUEST_TIMEOUT_MS);
expect(out).toContain(`[${busyId}] (not responding)`);
expect(out).toContain(`[${idleId}] Test Page - Basic`);
});
test('status stays healthy and names the busy tab with a way out', async () => {
const out = await handleMetaCommand('status', [], bm, async () => {});
expect(out).toContain('Status: healthy');
expect(out).toContain(`Active tab ${busyId}`);
expect(out).toContain(`closetab ${busyId}`);
});
test('a stuck tab gets one pending probe, not one per poll', async () => {
let evaluates = 0;
const evaluate = busyPage.evaluate.bind(busyPage);
Object.assign(busyPage, { evaluate: (...args: Parameters<Page['evaluate']>) => { evaluates++; return evaluate(...args); } });
try {
for (let i = 0; i < 3; i++) expect(await bm.probeActiveTab()).toEqual({ id: busyId, responsive: false });
} finally {
delete (busyPage as { evaluate?: unknown }).evaluate; // back to the prototype method
}
// The /health check above may already hold the pending probe.
expect(evaluates).toBeLessThanOrEqual(1);
}, 10_000);
test('a persisted snapshot taken while the tab is busy keeps its last good storage', async () => {
const file = path.join(profileDir, 'session-state.json');
await persistSessionState(bm, file);
const saved = JSON.parse(fs.readFileSync(file, 'utf8'));
expect(saved.pages).toHaveLength(2);
expect(saved.pages.find((p: { isActive: boolean }) => p.isActive).storage.localStorage).toEqual({ probe: 'kept' });
expect(saved.pages.find((p: { isActive: boolean }) => !p.isActive).storage).not.toBeNull();
}, 15_000);
test('the idle tab still answers', async () => {
bm.switchTab(idleId, { bringToFront: false });
try {
expect(await bm.probeActiveTab()).toEqual({ id: idleId, responsive: true });
expect(await bm.getPage().title()).toBe('Test Page - Basic');
} finally {
bm.switchTab(busyId, { bringToFront: false });
}
});
test('closetab closes the busy tab and the session carries on', async () => {
const out = await handleMetaCommand('closetab', [String(busyId)], bm, async () => {});
expect(out).toContain(`Closed tab ${busyId}`);
expect(bm.getTabCount()).toBe(1);
expect(await bm.probeActiveTab()).toEqual({ id: idleId, responsive: true });
}, 15_000);
});
@@ -0,0 +1,66 @@
/**
* What the CLI prints when a command's reply never arrives (#1781, #3247).
*
* Bun rejects an AbortSignal.timeout() fetch with TimeoutError, so the old
* AbortError-only check skipped the busy report and printed "The operation
* timed out." instead. When the daemon's /health names its active tab as not
* responding, the report names that tab and the commands that leave it
* without a restart; a dead daemon gets the plain line.
*/
import { afterEach, describe, expect, test } from 'bun:test';
import { commandTimeoutReport, isFetchTimeout } from '../src/cli';
let servers: Array<ReturnType<typeof Bun.serve>> = [];
afterEach(() => { for (const s of servers) s.stop(true); servers = []; });
/** A fake daemon whose /health answers `body`. */
function fakeHealth(body: unknown): number {
const srv = Bun.serve({ port: 0, hostname: '127.0.0.1', fetch: () => Response.json(body) });
servers.push(srv);
if (!srv.port) throw new Error('fake /health: no port');
return srv.port;
}
/** A pid that no longer runs: a child that has already exited. */
function deadPid(): number {
return Bun.spawnSync([process.execPath, '-e', '']).pid;
}
describe('isFetchTimeout', () => {
test('a real fetch whose AbortSignal.timeout fires counts as a timeout', async () => {
const silent = Bun.serve({ port: 0, hostname: '127.0.0.1', fetch: () => new Promise<Response>(() => {}) });
servers.push(silent);
const err = await fetch(`http://127.0.0.1:${silent.port}/`, { signal: AbortSignal.timeout(50) })
.then(() => null, (e: unknown) => e);
expect(isFetchTimeout(err)).toBe(true);
expect(isFetchTimeout(new TypeError('fetch failed'))).toBe(false);
});
});
describe('commandTimeoutReport', () => {
test('names the busy tab and the way out, never a restart', async () => {
const port = fakeHealth({ status: 'healthy', activeTab: { id: 7, responsive: false } });
const out = (await commandTimeoutReport({ pid: process.pid, port }, 30_000)).join('\n');
expect(out).toContain('tab 7');
expect(out).toContain('closetab 7');
expect(out).not.toContain('--force-restart');
});
test('a live daemon with a responsive or unreported tab gets the busy-not-restarting line', async () => {
for (const body of [
{ status: 'healthy', activeTab: { id: 1, responsive: true } },
{ status: 'healthy' }, // daemon from before activeTab existed
]) {
const out = (await commandTimeoutReport({ pid: process.pid, port: fakeHealth(body) }, 30_000)).join('\n');
expect(out).toContain('still alive');
expect(out).not.toContain('closetab');
}
});
test('a dead daemon gets only the timeout line', async () => {
const port = fakeHealth({ status: 'healthy', activeTab: { id: 7, responsive: false } });
const out = await commandTimeoutReport({ pid: deadPid(), port }, 30_000);
expect(out).toHaveLength(1);
expect(out[0]).toContain('timed out after 30s');
});
});
+5 -2
View File
@@ -94,13 +94,16 @@ describe('/health does NOT surface ptyToken', () => {
test('/health response body does not include ptyToken', async () => {
// It's allowed to include `terminalPort` (a port number, not auth).
const ctx = stubRouteContext({
browserManager: { isHealthy: async () => true, getConnectionMode: () => 'launched', getTabCount: () => 1 } as any,
browserManager: {
isHealthy: () => true, probeActiveTab: async () => ({ id: 1, responsive: true }),
getConnectionMode: () => 'launched', getTabCount: () => 1,
} as any,
terminal: { readPort: () => 4242, grantToken: async () => true, restartSession: async () => true },
});
const text = await (await callRoute('GET', '/health', ctx)).text();
const body = JSON.parse(text);
expect(body.terminalPort).toBe(4242);
expect(Object.keys(body).sort()).toEqual(['mode', 'status', 'tabs', 'terminalPort', 'uptime']);
expect(Object.keys(body).sort()).toEqual(['activeTab', 'mode', 'status', 'tabs', 'terminalPort', 'uptime']);
expect(text).not.toContain('ptyToken');
expect(text).not.toContain('gstack_pty');
});
+3 -1
View File
@@ -81,7 +81,9 @@ Response: (plain text result of the command)
```
#### GET /health
Server status. No auth required. Returns status, tabs, mode, uptime.
Server status. No auth required. Returns status, activeTab, tabs, mode, uptime.
`status` is daemon liveness; `activeTab` (`{ id, responsive }`, or null with no
page) says whether the active tab's main thread answered within 1s.
Never carries a token — extension token bootstrap is `POST /extension-token`
(local listener only, validates the pinned `chrome-extension://` Origin and a
loopback Host; 403 otherwise). Not reachable over the tunnel (404).
+11 -3
View File
@@ -172,9 +172,17 @@ the readiness dashboard and the PR body, and never count it as a pass.
review, including the `execution` gate behind `/codex` consult and challenge.
A reply whose last `Verdict:` line says `not ready` but carries no readable
severity prints `FINDINGS: unspecified` and exits 3 like a blocking finding.
- **No findings.** A review with no tag and no label is `clean` only when it
says so explicitly (`NO_FINDINGS`, "no issues", "did not find any bugs").
Otherwise it is `unverified` (see below), never `clean`. A `/codex` consult
- **No findings.** A review with no tag and no label is `clean` only when its
conclusion says so: the last sentence (or its last ", and" clause), or the
`because` clause of its `Recommendation:` line, opens with `NO_FINDINGS`,
"No issues found", "I found no actionable regressions" or "I did not find
any bugs", with no "but", "however", "except", "other than" or "elsewhere".
When other prose comes first, anything after that statement except a
predicate ("were found", "is evident") or a colon explanation must scope it
to the change as a whole ("in this change", "in the staged changes"); "No
issues in the parser." does not conclude the review. A "no issues" phrase
anywhere else in the reply does not count. Otherwise the review is
`unverified` (see below), never `clean`. A `/codex` consult
answer with no severity labels at all is not a review summary and stays
`clean`.
- **Design proposals** (the design-direction voices in /design-consultation
+30 -10
View File
@@ -180,7 +180,7 @@ export interface HttpAssertion {
method: 'GET' | 'POST' | 'PUT' | 'PATCH' | 'DELETE';
headers?: Record<string, string>;
body?: string;
expected: { status: number; includes?: string; excludes?: string };
expected: { status: number | number[]; includes?: string; excludes?: string };
vulnerable?: { status: number; includes?: string; excludes?: string };
}
export interface Command {
@@ -648,16 +648,34 @@ export function validateVerificationObservation(value: unknown): VerificationObs
inputHash: v.inputHash,
};
}
function assertion(value: unknown, name: string): HttpAssertion {
const MAX_SAFE_STATUSES = 8;
function assertion(value: unknown, name: string, safeStatusSet = false): HttpAssertion {
const v = object(value, name),
expected = object(v.expected, `${name}.expected`);
exact(v, ['name', 'path', 'method', 'headers', 'body', 'expected', 'vulnerable'], name);
const oracle = (x: Record<string, any>, n: string) => {
const httpStatus = (s: unknown) => Number.isInteger(s) && (s as number) >= 100 && (s as number) <= 599;
const oracle = (x: Record<string, any>, n: string, statusSet = false) => {
exact(x, ['status', 'includes', 'excludes'], n);
if (!Number.isInteger(x.status) || x.status < 100 || x.status > 599)
throw new CsoError('INVALID_SCHEMA', `${n}.status must be an HTTP status`);
if (!Array.isArray(x.status)) {
if (!httpStatus(x.status)) throw new CsoError('INVALID_SCHEMA', `${n}.status must be an HTTP status`);
} else if (!statusSet) throw new CsoError('INVALID_SCHEMA', `${n}.status must be an HTTP status`);
else if (
!x.status.length ||
x.status.length > MAX_SAFE_STATUSES ||
!x.status.every(httpStatus) ||
new Set(x.status).size !== x.status.length
)
throw new CsoError(
'INVALID_SCHEMA',
`${n}.status must be 1..${MAX_SAFE_STATUSES} distinct HTTP statuses`,
);
else if (typeof x.excludes !== 'string' || !x.excludes)
throw new CsoError(
'INVALID_SCHEMA',
`${n} with a set of safe statuses must exclude the vulnerable marker`,
);
return {
status: x.status,
status: Array.isArray(x.status) ? [...x.status] : x.status,
...(x.includes === undefined ? {} : { includes: string(x.includes, `${n}.includes`) }),
...(x.excludes === undefined ? {} : { excludes: string(x.excludes, `${n}.excludes`) }),
};
@@ -683,7 +701,7 @@ function assertion(value: unknown, name: string): HttpAssertion {
method: oneOf(v.method, ['GET', 'POST', 'PUT', 'PATCH', 'DELETE'], `${name}.method`),
...(Object.keys(headers).length ? { headers } : {}),
...(v.body === undefined ? {} : { body: string(v.body, `${name}.body`, 65536) }),
expected: oracle(expected, `${name}.expected`),
expected: oracle(expected, `${name}.expected`, safeStatusSet && v.vulnerable !== undefined),
...(v.vulnerable === undefined ? {} : { vulnerable: oracle(object(v.vulnerable), `${name}.vulnerable`) }),
};
}
@@ -692,13 +710,13 @@ export function securityAssertions(security: VerificationRequest['security']): H
return Array.isArray(security) ? security : [security];
}
function securityAssertion(value: unknown): VerificationRequest['security'] {
if (!Array.isArray(value)) return assertion(value, 'security');
if (!Array.isArray(value)) return assertion(value, 'security', true);
if (!value.length || value.length > MAX_SECURITY_ASSERTIONS)
throw new CsoError(
'INVALID_SCHEMA',
`security must be one assertion or 1..${MAX_SECURITY_ASSERTIONS} assertions`,
);
const assertions = value.map((x, i) => assertion(x, `security[${i}]`));
const assertions = value.map((x, i) => assertion(x, `security[${i}]`, true));
if (new Set(assertions.map((x) => x.name)).size !== assertions.length)
throw new CsoError('INVALID_SCHEMA', 'security assertion names must be distinct');
return assertions;
@@ -805,7 +823,9 @@ export function validateVerificationRequest(input: unknown): VerificationRequest
'Verification needs a legitimate control, distinct before/fixed security oracles, existing tests, immutable test files, and boundary files',
);
const mutuallyExclusive = ({ expected: secure, vulnerable }: HttpAssertion) =>
secure.status !== vulnerable!.status ||
(Array.isArray(secure.status)
? !secure.status.includes(vulnerable!.status)
: secure.status !== vulnerable!.status) ||
(secure.includes !== undefined &&
vulnerable!.excludes !== undefined &&
secure.includes.includes(vulnerable!.excludes)) ||
+23
View File
@@ -57,6 +57,29 @@ export function gitDiffHeaderPaths(line: string): [string, string] | undefined {
}
}
/**
* Replace every diff section touching a path `withheld` names with one marker line, so a file the snapshot keeps
* out of the readable tree cannot come back through retained history. An unparseable diff header is withheld too.
*/
export function withholdHistory(raw: string, withheld: (path: string) => string | undefined): string {
const output: string[] = [];
let skipping = false;
for (const line of raw.split('\n')) {
if (line.startsWith('commit ')) skipping = false;
else if (line.startsWith('diff --git ')) {
const paths = gitDiffHeaderPaths(line)?.map((value) => value.replace(/^[ab]\//, '')),
reason = paths ? paths.map(withheld).find(Boolean) : 'unparseable diff header';
skipping = Boolean(reason);
if (reason) {
output.push(`diff withheld: ${paths?.[1] ?? 'unknown path'} (${reason})`);
continue;
}
}
if (!skipping) output.push(line);
}
return output.join('\n');
}
/** Return only exact path hunks, keeping one commit preamble per matching commit. */
export function historyForPath(raw: string, path: string): string | undefined {
const expected = new Set([`a/${path}`, `b/${path}`]),
+20 -7
View File
@@ -1106,6 +1106,25 @@ export async function superviseRegistrySocket(options: {
* Requires qualified runtime images to contain the fixed, compiled
* /opt/cso/preparation@1.0.0 helper contract.
*/
/**
* Write the helper's own files (Rails' test database.yml and friends) into an execution copy. The copy keeps the
* source's modes, and a sealed or read-only checkout makes those files 0444, so each copied file is replaced, never
* opened for writing (or written through).
*/
export function writeOfflineTransformations(
source: string,
transformations: OfflinePreparationRequest['transformations'],
): void {
for (const transformation of transformations) {
if (sha256(transformation.content) !== transformation.sha256)
fail('INCOMPATIBLE_INPUT', 'Offline transformation content hash changed');
const file = contained(source, transformation.path);
secureDirectory(resolve(file, '..'));
fs.rmSync(file, { force: true });
fs.writeFileSync(file, transformation.content, { mode: 0o600, flag: 'wx' });
}
}
export class DockerPreparationSandboxRunner implements PreparationSandboxRunner {
readonly qualification;
private readonly runtime;
@@ -1514,13 +1533,7 @@ export class DockerPreparationSandboxRunner implements PreparationSandboxRunner
errorOnExist: false,
preserveTimestamps: false,
});
for (const transformation of request.transformations) {
if (sha256(transformation.content) !== transformation.sha256)
fail('INCOMPATIBLE_INPUT', 'Offline transformation content hash changed');
const file = contained(source, transformation.path);
secureDirectory(resolve(file, '..'));
fs.writeFileSync(file, transformation.content, { mode: 0o600 });
}
writeOfflineTransformations(source, request.transformations);
for (const item of request.metadata) {
if (sha256(item.content) !== item.sha256)
fail('INCOMPATIBLE_INPUT', `Offline metadata hash changed: ${item.path}`);
+4 -1
View File
@@ -14,6 +14,7 @@ import {
snapshotPathId,
MAX_OUTPUT,
} from './contracts';
import { withholdHistory } from './history';
import { childEnvironment, executable, git, redact, runProcess } from './process';
import { boundedList, readJson, secureDirectory, writeHelperJson, writeJson } from './state';
import { scan } from '../redact-engine';
@@ -952,7 +953,9 @@ export async function capture(
if (!(error instanceof CsoError) || error.code !== 'MISSING_INPUT') throw error;
}
guard();
const commits = (patched ?? '').split(/^(?=commit [0-9a-f]{40}(?:[0-9a-f]{24})?$)/m).filter(Boolean),
const commits = withholdHistory(patched ?? '', exclusion)
.split(/^(?=commit [0-9a-f]{40}(?:[0-9a-f]{24})?$)/m)
.filter(Boolean),
sizes = commits.map((commit) => Buffer.byteLength(commit)),
whole = sizes.reduce((sum, size) => sum + size, 0) <= MAX_OUTPUT;
let kept = 0,
+10 -3
View File
@@ -1219,14 +1219,21 @@ export function certify(params: {
beforeReceipt.observationHash !== witnessObservationHash(beforeObservation) ||
afterReceipt.observationHash !== witnessObservationHash(afterObservation) ||
params.before.existingTests !== beforeReceipt.diagnosticTestsPassed ||
params.after.existingTests !== afterReceipt.diagnosticTestsPassed ||
!beforeReceipt.externalAssertionsPassed ||
!afterReceipt.externalAssertionsPassed
params.after.existingTests !== afterReceipt.diagnosticTestsPassed
)
throw new CsoError(
'INCOMPATIBLE_INPUT',
'Authenticated assertion witness receipts do not match the verifier observations',
);
for (const [phase, receipt] of [
['before', beforeReceipt],
['after', afterReceipt],
] as const)
if (!receipt.externalAssertionsPassed)
throw new CsoError(
'ASSERTION_FAILED',
`External assertions failed in the ${phase} phase: booted ${params[phase].booted}, legitimate ${params[phase].legitimate}, security ${params[phase].security}`,
);
const stableExecutions = (receipt: AssertionWitnessReceipt) =>
receipt.executions.map(({ outputHash: _, ...execution }) => execution);
if (canonical(stableExecutions(beforeReceipt)) !== canonical(stableExecutions(afterReceipt)))
+1 -1
View File
@@ -18,7 +18,7 @@ interface Config {
}
function matches(status: number, body: string, oracle: HttpAssertion['expected']): boolean {
return (
status === oracle.status &&
(Array.isArray(oracle.status) ? oracle.status.includes(status) : status === oracle.status) &&
(oracle.includes === undefined || body.includes(oracle.includes)) &&
(oracle.excludes === undefined || !body.includes(oracle.excludes))
);
+52 -4
View File
@@ -135,8 +135,56 @@ const SEVERITY_LABELS = [
function severityWords(text: string): Severity[] {
return SEVERITY_LABELS.flatMap(re => [...text.matchAll(re)].map(m => SEVERITY_WORDS[m[1]!.toLowerCase()]!));
}
/** An explicit conclusion that the review found nothing to report. */
const NO_FINDINGS = /\bNO_FINDINGS\b|\bno (?:actionable |significant |new |concrete )?(?:bugs?|issues?|findings?|problems?)\b|\b(?:did not|didn't|cannot|can't|could not|couldn't) (?:find|identify) any (?:actionable |new |concrete )?(?:bugs|issues|findings|problems)\b/i;
/**
* A no-findings statement opening a sentence: NO_FINDINGS (its underscore is
* gone after plainReview), "No issues found", "I found no actionable
* regressions", "I did not find any bugs". Never a phrase inside another
* clause ("renaming the flag is no problem for callers, but ...").
*/
const FINDING_NOUN = '(?:(?:actionable|significant|new|concrete)\\s+)?(?:bugs?|issues?|findings?|problems?|regressions?|defects?)\\b';
const NO_FINDINGS_LEAD = '(?:(?:overall|in summary|in short|summary|conclusion|result)[,:]?\\s+|(?:after|having|on|upon)\\s+(?:reviewing|reading|checking|inspecting|reviewed|read|checked|inspected)\\b[^,.;]{0,80},\\s+)?';
const NO_FINDINGS = new RegExp(`^${NO_FINDINGS_LEAD}(?:NO_?FINDINGS\\b|(?:(?:i|we)\\s+(?:found|see|saw|identified)\\s+|there\\s+(?:are|were|is)\\s+)?no\\s+${FINDING_NOUN}|(?:i|we)\\s+(?:did not|didn't|could not|couldn't|cannot|can't)\\s+(?:find|identify|see)\\s+any\\s+${FINDING_NOUN})`, 'i');
/** A contrast or exception qualifies the claim, so the sentence does not conclude that nothing was found. */
const NO_FINDINGS_EXCEPTION = /(?<![\w-])(?:but|however|although|though|yet|except|excluding|unless|only|apart from|aside from|other than|besides|beyond|elsewhere|otherwise)(?![\w-])/i;
const NO_FINDINGS_PREDICATE = /^\s*(?:(?:was|were|is|are|has been|have been)\s+)?(?:found|identified|detected|evident|apparent|introduced|remain|exist|to report)?/i;
const WHOLE_CHANGE_SCOPE = /^(?:in|within|across|throughout|with|from|for)\s.*\b(?:chang(?:e|es|ed)|diffs?|patch(?:es)?|staged|unstaged|untracked|branch|commits?|pull request|PR)\b/i;
/**
* The review's conclusion says it found nothing: its last prose sentence or
* ", and" clause (a wrapped line that continues in lowercase joined), or the because-clause of its
* Recommendation line, is an unqualified no-findings statement. When other
* prose precedes it, anything after the statement other than a predicate
* ("were found", "is evident") or a colon explanation must scope it to the
* change as a whole: "No issues in the parser." or "No regressions were found
* by the tests." says nothing about the rest of the reply.
*/
function concludesNoFindings(code: string): boolean {
let prose = '', because: string | undefined, recommendationOpen = false;
for (const line of code.split(/\r?\n/)) {
const text = plainReview(line).trim();
const listed = /^[\t ]*(?:#{1,6}|[-+*])[\t ]/.test(line);
if (recommendationOpen && text && !listed) {
because = `${because} ${text}`;
recommendationOpen = !/[.!?]$/.test(text);
continue;
}
recommendationOpen = false;
if (!text || /^(?:Recommendation|Verdict)[\t ]*:/i.test(text)) {
const clause = text.match(/^Recommendation[\t ]*:.*?\bbecause\b[\t ]*(\S.*)$/i)?.[1];
if (clause) { because = clause; recommendationOpen = !/[.!?]$/.test(clause); }
prose += '\n';
} else if (/^[\t ]*#{1,6}[\t ]/.test(line)) prose += `\n${text}\n`;
else prose += listed || prose.endsWith('\n') || /[.!?;:]$/.test(prose) || !/^[a-z]/.test(text) ? `\n${text}` : ` ${text}`;
}
const sentences = prose.split(/(?<=[.!?;])\s+|\n|,\s+and\s+/i).map(sentence => sentence.trim()).filter(Boolean);
const total = sentences.length + (because ? 1 : 0);
return [sentences.at(-1), because].some(conclusion => {
const statement = conclusion?.match(NO_FINDINGS)?.[0];
if (!conclusion || !statement || NO_FINDINGS_EXCEPTION.test(conclusion)) return false;
const qualifier = conclusion.slice(statement.length).replace(NO_FINDINGS_PREDICATE, '').replace(/[\s.!]+$/, '').trim();
return total === 1 || !qualifier || qualifier.startsWith(':') || WHOLE_CHANGE_SCOPE.test(qualifier);
});
}
/** Formatting the requested marker in bold, inline code, or a list does not invalidate a completed review. */
function plainReview(text: string): string {
@@ -159,7 +207,7 @@ function withoutCode(text: string): string {
}
if (marker) { fence = marker; return ''; }
return line.replace(/`([^`]+)`/g, (span, inner: string, at: number) =>
/^\[?(?:P[0-3]|critical|high|medium|low)\]?$/i.test(inner.trim()) && CODE_LABEL_BEFORE.test(line.slice(0, at)) ? inner : ' ');
inner.trim() === 'NO_FINDINGS' || /^\[?(?:P[0-3]|critical|high|medium|low)\]?$/i.test(inner.trim()) && CODE_LABEL_BEFORE.test(line.slice(0, at)) ? inner : ' ');
}).join('\n');
}
@@ -218,7 +266,7 @@ export function classifyOutsideReview(input: OutsideReviewInput): OutsideReviewC
const notReady = !findings.highest && verdictNotReady(plainCode);
if (blocking || notReady) return { ...result('findings'), ...(unparsed ? { unparsed } : {}) };
if (unparsed) return { ...result('unverified', 'unparsed_severity', unparsed), unparsed };
if ((input.gate === 'structured' || input.gate === 'review') && !findings.highest && !NO_FINDINGS.test(input.text)) {
if ((input.gate === 'structured' || input.gate === 'review') && !findings.highest && !concludesNoFindings(code)) {
return result('unverified', 'untagged_review', 'missing severity or explicit no-findings conclusion');
}
return result('clean');
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "gstack",
"version": "1.91.152",
"version": "1.91.156",
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
"license": "MIT",
"type": "module",
+21 -2
View File
@@ -18,8 +18,8 @@ const finding:any={title:'Cross-tenant read',rootCause:'Tenant query omits calle
const request=(before:string,after:string)=>({findingId:'a'.repeat(32),runtimeProfile:'node',port:3456,start:{executable:'/usr/local/bin/node',args:['app.js']},legitimate:[{name:'health',path:'/health',method:'GET',expected:{status:200,includes:'ok'}}],security:{name:'tenant isolation',path:'/user?id=2',method:'GET',expected:{status:403},vulnerable:{status:200,includes:'tenant-b'}},existingTests:[{executable:'/usr/local/bin/node',args:['test.js']}],testFiles:['test.js'],fixtures:{'tenant.json':'{}'},boundaryFiles:['app.js'],changes:[{path:'app.js',beforeSha256:sha256(before),after,effect:'source'}],review:{reviewer:'independent-reviewer',independent:true,rootCauseRepaired:true,featurePreserved:true,boundaryMocks:false,rationale:'Predicate now binds the caller tenant and health behavior is preserved',reviewedPatchHash:''}});
// Structural signed fixture for bundle-reader mutation tests. Production
// issuance uses AssertionWitnessSession in verifyRepair instead.
function authenticatedBundleFixture(certified:ReturnType<typeof certify>,params:any,outcomes?:Record<'before'|'after',{executedTests:number;passingTests:number}>):any{
const verification=structuredClone(certified.manifest),identityRequest=params.identityRequest??params.request,pair=crypto.generateKeyPairSync('ed25519'),publicKey=pair.publicKey.export({format:'der',type:'spki'}).toString('hex'),keyId=sha256(Buffer.from(publicKey,'hex')),createdAt=verification.createdAt,runner={testToolchain:params.testToolchain,startPlanHash:verification.startPlanHash,testPlanHash:verification.testPlanHash,commandsHash:sha256(canonical(params.request.existingTests)),minimumPassingTestsHash:sha256(canonical(params.request.existingTests.map(()=>1)))},make=(phase:'before'|'after')=>{const observation=verification[phase],counts=outcomes?.[phase]??{executedTests:1,passingTests:1},binding:any={schemaVersion:1,protocol:'gstack-cso-assertion-witness-v1',nonce:sha256(`${phase}:${createdAt}`),phase,issuedAt:createdAt,expiresAt:new Date(Date.parse(createdAt)+300_000).toISOString(),runId:params.runId,findingId:identityRequest.findingId,policyHash:verification.policyHash,auditPolicyHash:verification.auditPolicyHash,runtime:{image:params.runtime.image,verifierImage:params.verifier.image,platform:params.runtime.platform,profile:params.runtime.id},runner,sourceHash:phase==='before'?verification.beforeSourceHash:verification.afterSourceHash,dependencyHash:phase==='before'?verification.beforeDependencies:verification.afterDependencies,configurationHash:phase==='before'?verification.beforeConfiguration:verification.afterConfiguration,requestHash:verification.requestHash,patchHash:verification.patchHash,harnessHash:verification.harnessHash,assertionHash:sha256(canonical({legitimate:identityRequest.legitimate,security:identityRequest.security})),fixturesHash:verification.fixturesHash},executions=params.request.existingTests.map((command:any)=>({commandHash:sha256(canonical(command)),exitCode:0,outputHash:sha256(`structural signed fixture ${phase}`),minimumPassingTests:1,...counts,reportedPassed:counts.passingTests>=1})),unsigned:any={schemaVersion:1,binding,keyId,publicKey,observationHash:witnessObservationHash(observation),externalAssertionsPassed:true,diagnosticTestsPassed:executions.every((execution:any)=>execution.reportedPassed),executions};return{...unsigned,signature:crypto.sign(null,Buffer.from(canonical(unsigned)),pair.privateKey).toString('hex')}};const witness={before:make('before'),after:make('after')};verification.assertionAssurance='authenticated_out_of_process';verification.testCompletionAssurance='self_reported';verification.witnessHash=assertionWitnessPairHash(witness);verification.result='runtime_tested';verification.id='';const id=verificationIdentity(verification);verification.id=id;
function authenticatedBundleFixture(certified:ReturnType<typeof certify>,params:any,outcomes?:Record<'before'|'after',{executedTests:number;passingTests:number}>,externalAssertionsPassed:Record<'before'|'after',boolean>={before:true,after:true}):any{
const verification=structuredClone(certified.manifest),identityRequest=params.identityRequest??params.request,pair=crypto.generateKeyPairSync('ed25519'),publicKey=pair.publicKey.export({format:'der',type:'spki'}).toString('hex'),keyId=sha256(Buffer.from(publicKey,'hex')),createdAt=verification.createdAt,runner={testToolchain:params.testToolchain,startPlanHash:verification.startPlanHash,testPlanHash:verification.testPlanHash,commandsHash:sha256(canonical(params.request.existingTests)),minimumPassingTestsHash:sha256(canonical(params.request.existingTests.map(()=>1)))},make=(phase:'before'|'after')=>{const observation=verification[phase],counts=outcomes?.[phase]??{executedTests:1,passingTests:1},binding:any={schemaVersion:1,protocol:'gstack-cso-assertion-witness-v1',nonce:sha256(`${phase}:${createdAt}`),phase,issuedAt:createdAt,expiresAt:new Date(Date.parse(createdAt)+300_000).toISOString(),runId:params.runId,findingId:identityRequest.findingId,policyHash:verification.policyHash,auditPolicyHash:verification.auditPolicyHash,runtime:{image:params.runtime.image,verifierImage:params.verifier.image,platform:params.runtime.platform,profile:params.runtime.id},runner,sourceHash:phase==='before'?verification.beforeSourceHash:verification.afterSourceHash,dependencyHash:phase==='before'?verification.beforeDependencies:verification.afterDependencies,configurationHash:phase==='before'?verification.beforeConfiguration:verification.afterConfiguration,requestHash:verification.requestHash,patchHash:verification.patchHash,harnessHash:verification.harnessHash,assertionHash:sha256(canonical({legitimate:identityRequest.legitimate,security:identityRequest.security})),fixturesHash:verification.fixturesHash},executions=params.request.existingTests.map((command:any)=>({commandHash:sha256(canonical(command)),exitCode:0,outputHash:sha256(`structural signed fixture ${phase}`),minimumPassingTests:1,...counts,reportedPassed:counts.passingTests>=1})),unsigned:any={schemaVersion:1,binding,keyId,publicKey,observationHash:witnessObservationHash(observation),externalAssertionsPassed:externalAssertionsPassed[phase],diagnosticTestsPassed:executions.every((execution:any)=>execution.reportedPassed),executions};return{...unsigned,signature:crypto.sign(null,Buffer.from(canonical(unsigned)),pair.privateKey).toString('hex')}};const witness={before:make('before'),after:make('after')};verification.assertionAssurance='authenticated_out_of_process';verification.testCompletionAssurance='self_reported';verification.witnessHash=assertionWitnessPairHash(witness);verification.result='runtime_tested';verification.id='';const id=verificationIdentity(verification);verification.id=id;
return{schemaVersion:3,runId:params.runId,id,createdAt,expiresAt:new Date(Date.parse(createdAt)+30*86400_000).toISOString(),requiredInputs:{sourceHash:params.manifest.executionHash,originalHash:params.manifest.originalHash,runtimeImage:params.runtime.image,platform:params.runtime.platform,archives:params.archives,...(params.dependencyClosures?{dependencyClosures:params.dependencyClosures}:{})},request:identityRequest,verification,transformations:params.manifest.entries.filter((entry:any)=>entry.transformation),...(params.preparation?{preparation:params.preparation}:{}),witness};
}
@@ -170,6 +170,14 @@ describe('tested repair certificate gate',()=>{
const params:any={runId:'run',manifest:{originalHash:'d'.repeat(64),executionHash:treeHash(before),entries:[]},request:req,runtime:{id:'node',image:'runtime@sha256:'+'a'.repeat(64),platform:'linux/amd64'},verifier:{image:'runtime@sha256:'+'a'.repeat(64)},before:{booted:true,legitimate:true,security:'intended_failure',existingTests:true,output:'before',inputHash:''},after:{booted:true,legitimate:true,security:'pass',existingTests:true,output:'after',inputHash:''},beforeRoot:before,afterRoot:after,policyHash:'c'.repeat(64),auditPolicyHash:'e'.repeat(64),archives:[],testToolchain:'runtime'},unsigned=certify(params),witness=authenticatedBundleFixture(unsigned,params,{before:{executedTests:2,passingTests:2},after:{executedTests:2,passingTests:1}}).witness;
expect(()=>certify({...params,witness})).toThrow('existing-test execution count or outcome');
});
test('a witness whose external assertions failed is an assertion failure, and a mismatched observation stays incompatible input',()=>{
const root=tmp(),before=path.join(root,'before'),after=path.join(root,'after');fs.mkdirSync(before);fs.writeFileSync(path.join(before,'app.js'),'vulnerable\n');fs.writeFileSync(path.join(before,'test.js'),'one registered test\n');const raw:any=request('vulnerable\n','fixed\n');raw.review.reviewedPatchHash=patchHash(raw);const req=validateVerificationRequest(raw);preparePatchedSource(before,after,req);
const params:any={runId:'run',manifest:{originalHash:'d'.repeat(64),executionHash:treeHash(before),entries:[]},request:req,runtime:{id:'node',image:'runtime@sha256:'+'a'.repeat(64),platform:'linux/amd64'},verifier:{image:'runtime@sha256:'+'a'.repeat(64)},before:{booted:true,legitimate:true,security:'intended_failure',existingTests:true,output:'before',inputHash:''},after:{booted:true,legitimate:true,security:'inconclusive',existingTests:true,output:'after',inputHash:''},beforeRoot:before,afterRoot:after,policyHash:'c'.repeat(64),auditPolicyHash:'e'.repeat(64),archives:[],testToolchain:'runtime'},unsigned=certify(params);
const failed=authenticatedBundleFixture(unsigned,params,undefined,{before:true,after:false}).witness;
expect(()=>certify({...params,witness:failed})).toThrow(expect.objectContaining({code:'ASSERTION_FAILED',message:'External assertions failed in the after phase: booted true, legitimate true, security inconclusive'}));
const mismatched=authenticatedBundleFixture({...unsigned,manifest:{...unsigned.manifest,after:{...unsigned.manifest.after,output:'a different observation'}}},params,undefined,{before:true,after:false}).witness;
expect(()=>certify({...params,witness:mismatched})).toThrow(expect.objectContaining({code:'INCOMPATIBLE_INPUT'}));
});
test('canonical Python starts use direct prepared launchers and reject root framework shadows',()=>{
const pin=(name:string)=>`${name}==1.0.0 --hash=sha256:${'a'.repeat(64)}\n`;
const flask=tmp();fs.writeFileSync(path.join(flask,'requirements.txt'),pin('Flask'));fs.writeFileSync(path.join(flask,'app.py'),'from flask import Flask\napp = Flask(__name__)\n');const flaskCommand=canonicalStartPlan(flask,'python',3456).command;expect(flaskCommand.executable).toBe('/work/.venv/bin/python');expect(flaskCommand.args.slice(0,3)).toEqual(['-I','-S','-c']);expect(flaskCommand.args[3]).toContain('from flask.cli import main');expect(flaskCommand.args.slice(4)).toEqual(['--app','app:app','run','--host','127.0.0.1','--port','3456']);fs.writeFileSync(path.join(flask,'flask.py'),'raise RuntimeError("shadow")\n');expect(()=>canonicalStartPlan(flask,'python',3456)).toThrow('root import shadow');
@@ -186,6 +194,17 @@ describe('tested repair certificate gate',()=>{
const runDir=tmp(),snapshot=path.join(runDir,'snapshot');fs.mkdirSync(snapshot);const files:Record<string,string>={'package.json':JSON.stringify({scripts:{test:'jest'}}),'package-lock.json':'{}\n','app.js':'server.listen(3456)\n','app.test.js':'test("ok",()=>{})\n'};for(const [name,body] of Object.entries(files))fs.writeFileSync(path.join(snapshot,name),body);const plan=canonicalTestPlan(snapshot,'node'),raw:any=request(files['app.js'],'server.listen(3456)\n');raw.start={executable:'/usr/local/bin/node',args:['app.js']};raw.existingTests=plan.commands;raw.testFiles=plan.files;raw.changes=[{path:'package-lock.json',beforeSha256:sha256(files['package-lock.json']),after:'{"changed":true}\n',effect:'dependency'}];raw.review.reviewedPatchHash=patchHash(raw);const entries=Object.entries(files).map(([path,body])=>({path,originalHash:sha256(body),executionHash:sha256(body),bytes:Buffer.byteLength(body),mode:0o600})),manifest:any={version:3,createdAt:'2026-01-01T00:00:00Z',expiresAt:'2026-01-08T00:00:00Z',root:'/repo',headCommit:'a'.repeat(40),originalHash:'b'.repeat(64),executionHash:treeHash(snapshot),entries};let executed=false;await expect(verifyRepair({runId:'run',runDir,manifest,rawRequest:raw,runtime:{id:'node',stack:'node',image:'runtime@sha256:'+'a'.repeat(64),platform:'linux/amd64'} as any,verifier:{id:'node',stack:'node',image:'runtime@sha256:'+'a'.repeat(64),platform:'linux/amd64'} as any,policyHash:'c'.repeat(64),archives:[],executor:{observe:async()=>{executed=true;throw new Error('must not run');}},persist:false})).rejects.toMatchObject({code:'PREREQUISITE'});expect(executed).toBe(false);
});
test('overlapping security oracles and no-op repairs cannot be certified',()=>{const overlapping:any=request('vulnerable\n','fixed\n');overlapping.review.reviewedPatchHash='pending';overlapping.security.vulnerable={...overlapping.security.expected};expect(()=>validateVerificationRequest(overlapping)).toThrow('mutually exclusive');const noop:any=request('vulnerable\n','vulnerable\n');noop.review.reviewedPatchHash='pending';expect(()=>validateVerificationRequest(noop)).toThrow('material patch');});
test('a security expectation may list several safe statuses only while it excludes the vulnerable marker',()=>{
const make=(edit:(raw:any)=>void)=>{const raw:any=request('vulnerable\n','fixed\n');raw.review.reviewedPatchHash='pending';edit(raw);return raw;};
expect(validateVerificationRequest(make(raw=>{raw.security.expected={status:[403,404],excludes:'tenant-b'};})).security).toMatchObject({expected:{status:[403,404],excludes:'tenant-b'}});
expect(validateVerificationRequest(make(raw=>{raw.security.expected={status:[200,403],excludes:'tenant-b'};})).security).toMatchObject({expected:{status:[200,403]}});
expect(()=>validateVerificationRequest(make(raw=>{raw.security.expected={status:[200,403],excludes:'unrelated'};}))).toThrow('mutually exclusive');
expect(()=>validateVerificationRequest(make(raw=>{raw.security.expected={status:[403,404]};}))).toThrow('must exclude the vulnerable marker');
for(const status of [[],[403,403],[99,403],[400,401,403,404,405,409,410,422,429]])expect(()=>validateVerificationRequest(make(raw=>{raw.security.expected={status,excludes:'tenant-b'};}))).toThrow('distinct HTTP statuses');
expect(()=>validateVerificationRequest(make(raw=>{raw.security.vulnerable={status:[200,206],includes:'tenant-b'};}))).toThrow('security.vulnerable.status must be an HTTP status');
expect(()=>validateVerificationRequest(make(raw=>{raw.legitimate[0].expected={status:[200,403],excludes:'denied'};}))).toThrow('legitimate[0].expected.status must be an HTTP status');
expect(()=>validateVerificationRequest(make(raw=>{raw.security=[{...raw.security,expected:{status:[403,404],excludes:'tenant-b'}},{...raw.security,name:'second',expected:{status:[403,404]}}];}))).toThrow('security[1].expected with a set of safe statuses');
});
test('review is a separate content-bound but explicitly self-attested artifact',()=>{const raw:any=request('vulnerable\n','fixed\n');raw.review.reviewedPatchHash=patchHash(raw);const req=validateVerificationRequest(raw),artifact=makeReviewArtifact('run',req,'producer-session');raw.review.artifactId=artifact.id;const bound=validateVerificationRequest(raw);expect(validateReviewArtifact(artifact,'run',bound)).toMatchObject({producer:'producer-session',assurance:'self_attested'});expect(()=>makeReviewArtifact('run',req,req.review.reviewer)).toThrow('distinct');expect(()=>validateReviewArtifact({...artifact,rationale:'changed'},'run',bound)).toThrow('does not bind');});
test('executor observations have a strict helper-owned shape',()=>{expect(()=>validateVerificationObservation({booted:true,legitimate:true,security:'pass',existingTests:true,output:'ok',inputHash:'',tested:true})).toThrow('Unexpected');expect(()=>validateVerificationObservation({booted:'yes',legitimate:true,security:'pass',existingTests:true,output:'ok',inputHash:''})).toThrow('booleans');});
test('secret-bearing fixtures cannot enter a replayable verification bundle',async()=>{const runDir=tmp(),snapshot=path.join(runDir,'snapshot');fs.mkdirSync(snapshot);fs.writeFileSync(path.join(snapshot,'app.js'),'vulnerable\n');const raw:any=request('vulnerable\n','fixed\n');raw.review.reviewedPatchHash=patchHash(raw);raw.fixtures={'secret.txt':CREDENTIAL_CANARY};const manifest:any={entries:[{path:'app.js',transformation:undefined}]};const runtime:any={id:'node',image:'runtime@sha256:'+'a'.repeat(64),platform:'linux/amd64'};expect(verifyRepair({runId:'run',runDir,manifest,rawRequest:raw,runtime,verifier:runtime,policyHash:'c'.repeat(64),archives:[],executor:{observe:async()=>{throw new Error('must not execute');}},persist:false})).rejects.toMatchObject({code:'REDACTION_FAILED'});});
+14
View File
@@ -92,6 +92,20 @@ describe('CSO Git metadata hardening',()=>{
expect(history).not.toContain('replacement-history');
});
test.skipIf(process.platform==='win32')('retained history withholds diffs of credential files the snapshot excludes',async()=>{
const {repo,runDir,git}=fixture();
fs.writeFileSync(path.join(repo,'.env'),'UPSTREAM_TOKEN=planted-history-value\n');fs.writeFileSync(path.join(repo,'tracked.ts'),'export const tracked = false\n');
git('add','.env','tracked.ts');git('commit','-qm','configure upstream');
await capture(repo,runDir);
const history=fs.readFileSync(path.join(runDir,'history.txt'),'utf8');
expect(history).not.toContain('planted-history-value');
expect(history).toContain('diff withheld: .env (credential or execution configuration)');
expect(history).toContain('+export const tracked = false');
expect(history).toContain('Subject: configure upstream');
});
test.skipIf(process.platform==='win32')('rejects repository config includes before Git can consume them',async()=>{
const {root,repo,runDir}=fixture(),included=path.join(root,'included.conf');
fs.writeFileSync(included,'[core]\n\tworktree = /tmp/cso-decoy\n');
+33 -1
View File
@@ -1,5 +1,5 @@
import { describe,expect,test } from 'bun:test';
import { gitDiffHeaderPaths,historyForPath } from '../lib/cso/history';
import { gitDiffHeaderPaths,historyForPath,withholdHistory } from '../lib/cso/history';
describe('CSO retained Git history path filtering',()=>{
test('matches exact paths without leaking a prefix sibling hunk',()=>{
@@ -25,6 +25,38 @@ diff --git a/foo-extra b/foo-extra
expect(historyForPath(raw,'café file.ts')).toContain('+unicode-only');
});
test('withholds every diff section touching an excluded path, keeping commit preambles and other files',()=>{
const raw=`commit ${'c'.repeat(40)}
Subject: add config
diff --git a/.env b/.env
--- /dev/null
+++ b/.env
+PLANTED_VALUE=excluded-file-body
diff --git a/app.ts b/app.ts
--- a/app.ts
+++ b/app.ts
+kept-line
commit ${'d'.repeat(40)}
Subject: rename
diff --git a/config/old.txt b/config/master.key
rename from config/old.txt
rename to config/master.key
+renamed-into-excluded
diff --git "a/foo b/foo
+unparseable-body
`;
const kept=withholdHistory(raw,(path)=>/(^|\/)(\.env|master\.key)$/.test(path)?'credential or execution configuration':undefined);
expect(kept).not.toContain('excluded-file-body');
expect(kept).not.toContain('renamed-into-excluded');
expect(kept).not.toContain('unparseable-body');
expect(kept).toContain('diff withheld: .env (credential or execution configuration)');
expect(kept).toContain('diff withheld: config/master.key (credential or execution configuration)');
expect(kept).toContain('diff withheld: unknown path (unparseable diff header)');
expect(kept).toContain('+kept-line');
expect(kept).toContain('Subject: add config');
expect(kept).toContain('Subject: rename');
});
test('rejects malformed quoted headers instead of broadening the match',()=>{
expect(gitDiffHeaderPaths('diff --git "a/foo b/foo')).toBeUndefined();
expect(historyForPath('diff --git "a/foo b/foo\n+secret\n','foo')).toBeUndefined();
+27 -1
View File
@@ -3,8 +3,9 @@ import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { spawnSync } from 'node:child_process';
import { createHash } from 'node:crypto';
import { createPreparedExport, recordNpmArchiveEntry } from '../lib/cso/preparation-container';
import { materializePreparedExport } from '../lib/cso/preparation-docker';
import { materializePreparedExport, writeOfflineTransformations } from '../lib/cso/preparation-docker';
import { heldApplicationReady, validateSingleContainerProcessOutput } from '../lib/cso/docker';
const roots: string[] = [];
@@ -68,3 +69,28 @@ describe('CSO preparation container archive limits', () => {
expect(heldApplicationReady('PID COMMAND\n1 sh\n')).toBe(false);
});
});
describe('offline transformations in an execution copy', () => {
test.skipIf(process.platform === 'win32')('replace a read-only copied file under a read-only directory instead of writing through it', () => {
const source = temporary(), config = path.join(source, 'config'), database = path.join(config, 'database.yml'), outside = path.join(temporary(), 'outside.yml');
fs.mkdirSync(config);
fs.writeFileSync(database, 'production: sealed\n');
fs.chmodSync(database, 0o444);
fs.chmodSync(config, 0o555);
fs.writeFileSync(outside, 'untouched\n');
const content = 'test:\n adapter: sqlite3\n', sha256 = createHash('sha256').update(content).digest('hex');
writeOfflineTransformations(source, [
{ path: 'config/database.yml', content, sha256, reason: 'helper test configuration' },
{ path: 'config/initializers/zzzz_cso_test.rb', content, sha256, reason: 'helper test configuration' },
]);
expect(fs.readFileSync(database, 'utf8')).toBe(content);
expect(fs.statSync(database).mode & 0o777).toBe(0o600);
expect(fs.readFileSync(path.join(config, 'initializers', 'zzzz_cso_test.rb'), 'utf8')).toBe(content);
expect(() => writeOfflineTransformations(source, [{ path: 'config/database.yml', content, sha256: 'f'.repeat(64), reason: 'tampered' }])).toThrow('Offline transformation content hash changed');
fs.rmSync(database);
fs.symlinkSync(outside, database);
writeOfflineTransformations(source, [{ path: 'config/database.yml', content, sha256, reason: 'helper test configuration' }]);
expect(fs.lstatSync(database).isSymbolicLink()).toBe(false);
expect(fs.readFileSync(outside, 'utf8')).toBe('untouched\n');
});
});
+13
View File
@@ -65,6 +65,19 @@ describe('CSO verifier security judgement',()=>{
try{expect((await judgeSecurity('before',payloads,disproved.port)).security).toBe('pass');}finally{disproved.server.stop(true);}
});
test('a set of safe statuses passes any listed rejection that withholds the marker and nothing else',async()=>{
const dotfile:HttpAssertion={name:'dotfile',path:'/.env',method:'GET',expected:{status:[403,404],excludes:'DOTFILE'},vulnerable:{status:200,includes:'DOTFILE'}};
const answer=async(status:number,body:string)=>{
const server=Bun.serve({port:0,hostname:'127.0.0.1',fetch:()=>new Response(body,{status})});
try{return [(await judgeSecurity('before',dotfile,Number(server.port))).security,(await judgeSecurity('after',dotfile,Number(server.port))).security];}finally{server.stop(true);}
};
expect(await answer(404,'not found')).toEqual(['pass','pass']);
expect(await answer(403,'denied')).toEqual(['pass','pass']);
expect(await answer(200,'DOTFILE=1')).toEqual(['intended_failure','inconclusive']);
expect(await answer(403,'denied DOTFILE=1')).toEqual(['inconclusive','inconclusive']);
expect(await answer(400,'bad request')).toEqual(['inconclusive','inconclusive']);
});
test('a single assertion keeps its original before and after outcomes',async()=>{
const vulnerable=serve(new Set()),fixed=serve(ALL);
try{
+16 -1
View File
@@ -1,7 +1,7 @@
import { isDeepStrictEqual } from 'node:util';
import * as path from 'node:path';
import { renderOfficeHoursReviewerPrompt, validateOfficeHoursReview, renderOfficeHoursReview, extractOfficeHoursReviewBlock, parseOfficeHoursVerdictReceipt, verifyOfficeHoursVerdictReceipt, officeHoursDesignChanges, verifyOfficeHoursCitations, type OfficeHoursReview } from '../../lib/office-hours-review';
import { verifySpecFixesAtCap, type SpecFix } from '../../lib/spec-review-cap';
import { parseSpecFixes, verifySpecFixesAtCap, type SpecFix } from '../../lib/spec-review-cap';
export type OfficeHoursFile = { path: string; content: string | null };
/** The helper's captured diff before a later round; a missing snapshot fails closed. */
@@ -32,6 +32,21 @@ export interface OfficeHoursCompletionEvidence {
fixes?: SpecFix[] | null;
}
/**
* The cap fixes the agent saved for the finalizer: the last observed Write whose
* content is a valid `--fixes` list. Null when it saved none. The validator
* recomputes the cap record from this list, so a list that was not the one
* passed to finalize still produces a mismatch.
*/
export function observedSpecFixes(toolCalls: OfficeHoursCompletionEvidence['toolCalls']): SpecFix[] | null {
for (const call of toolCalls.toReversed()) {
if (call.tool !== 'Write') continue;
try { return parseSpecFixes(JSON.parse(String(call.input?.content ?? ''))); }
catch { /* not a fixes list */ }
}
return null;
}
export interface OfficeHoursReviewEvidence {
verdict: string;
concerns: string;
@@ -0,0 +1,31 @@
/**
* Copy bin/gstack-office-hours-review and every local module it imports into a
* fixture, so the copied formatter loads the same modules as the checkout.
* Marathon 39032308051 copied a hand-kept file list that missed
* lib/spec-review-cap.ts, and every formatter command failed at module load.
*/
import * as fs from 'node:fs';
import * as path from 'node:path';
const ROOT = path.resolve(import.meta.dir, '../..');
export function copyOfficeHoursFormatter(fixtureRoot: string): string {
const pending = ['bin/gstack-office-hours-review'];
const copied = new Set<string>();
while (pending.length) {
const relative = pending.pop()!;
if (copied.has(relative)) continue;
copied.add(relative);
const source = path.join(ROOT, relative);
const target = path.join(fixtureRoot, relative);
fs.mkdirSync(path.dirname(target), { recursive: true });
fs.copyFileSync(source, target);
for (const [, specifier] of fs.readFileSync(source, 'utf8').matchAll(/\b(?:from|import)\s+'(\.\.?\/[^']+)'/g)) {
const resolved = path.relative(ROOT, path.resolve(path.dirname(source), specifier!));
pending.push(path.extname(resolved) ? resolved : `${resolved}.ts`);
}
}
const formatter = path.join(fixtureRoot, 'bin/gstack-office-hours-review');
fs.chmodSync(formatter, 0o755);
return formatter;
}
+1 -1
View File
@@ -1099,7 +1099,7 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
'test/skill-e2e-arm-benchmark.test.ts',
'ship/SKILL.md', 'test/helpers/e2e-gate.ts'],
'office-hours-section-loading': [ 'office-hours/**', 'bin/gstack-office-hours-review', 'lib/office-hours-review.ts', 'lib/fs-atomic.ts', 'scripts/resolvers/review-dashboard.ts', 'scripts/resolvers/plan-gates.ts', 'scripts/resolvers/spec-review.ts', 'scripts/resolvers/outside-voice-steps.ts', 'scripts/resolvers/review-scope.ts', 'scripts/resolvers/outside-voice.ts', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/carve-guards.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/office-hours-completion.ts', 'test/helpers/llm-judge.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-office-hours-section-loading.test.ts', 'test/helpers/agent-sdk-runner.ts', 'test/helpers/auq-native-capture.ts', 'test/helpers/auto-decision-state.ts', 'test/helpers/autoplan-artifact-digest.ts', 'test/helpers/autoplan-artifact-permission.ts', 'test/helpers/autoplan-artifact-recorder.ts', 'test/helpers/capture-parity-baseline.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty/**', 'test/helpers/dx-selected-navigation.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/eng-cache-writer-decision.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/helpers/native-auto-decide.ts', 'test/helpers/owned-claude-transcript.ts', 'test/helpers/parity-harness.ts', 'test/helpers/plan-count-artifacts.ts', 'test/helpers/plan-count-file-permission.ts', 'test/helpers/plan-count-fixture.ts', 'test/helpers/plan-count-pending-exit.ts', 'test/helpers/plan-count-pending-question.ts', 'test/helpers/plan-count-transcript.ts', 'test/helpers/plan-floor-review.ts', 'test/helpers/plan-floor-target.ts', 'test/helpers/plan-scope-selection.ts', 'test/helpers/plan-seed-submission.ts', 'test/helpers/plan-skill-question-events.ts', 'test/helpers/plan-skill-question-hook-scope.ts', 'test/helpers/plan-skill-questions.ts', 'test/helpers/pty-screen.ts', 'test/helpers/pty-trust-dialog.ts', 'test/helpers/skill-census.ts'],
'office-hours-section-loading': [ 'office-hours/**', 'bin/gstack-office-hours-review', 'lib/office-hours-review.ts', 'lib/fs-atomic.ts', 'lib/spec-review-cap.ts', 'test/helpers/office-hours-formatter-fixture.ts', 'scripts/resolvers/review-dashboard.ts', 'scripts/resolvers/plan-gates.ts', 'scripts/resolvers/spec-review.ts', 'scripts/resolvers/outside-voice-steps.ts', 'scripts/resolvers/review-scope.ts', 'scripts/resolvers/outside-voice.ts', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/carve-guards.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/office-hours-completion.ts', 'test/helpers/llm-judge.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-office-hours-section-loading.test.ts', 'test/helpers/agent-sdk-runner.ts', 'test/helpers/auq-native-capture.ts', 'test/helpers/auto-decision-state.ts', 'test/helpers/autoplan-artifact-digest.ts', 'test/helpers/autoplan-artifact-permission.ts', 'test/helpers/autoplan-artifact-recorder.ts', 'test/helpers/capture-parity-baseline.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty/**', 'test/helpers/dx-selected-navigation.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/eng-cache-writer-decision.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/helpers/native-auto-decide.ts', 'test/helpers/owned-claude-transcript.ts', 'test/helpers/parity-harness.ts', 'test/helpers/plan-count-artifacts.ts', 'test/helpers/plan-count-file-permission.ts', 'test/helpers/plan-count-fixture.ts', 'test/helpers/plan-count-pending-exit.ts', 'test/helpers/plan-count-pending-question.ts', 'test/helpers/plan-count-transcript.ts', 'test/helpers/plan-floor-review.ts', 'test/helpers/plan-floor-target.ts', 'test/helpers/plan-scope-selection.ts', 'test/helpers/plan-seed-submission.ts', 'test/helpers/plan-skill-question-events.ts', 'test/helpers/plan-skill-question-hook-scope.ts', 'test/helpers/plan-skill-questions.ts', 'test/helpers/pty-screen.ts', 'test/helpers/pty-trust-dialog.ts', 'test/helpers/skill-census.ts'],
'office-hours-design-draft': [ 'office-hours/**', 'lib/office-hours-review.ts', 'scripts/resolvers/review-dashboard.ts', 'scripts/resolvers/plan-gates.ts', 'scripts/resolvers/spec-review.ts', 'scripts/resolvers/outside-voice-steps.ts', 'scripts/resolvers/review-scope.ts', 'scripts/resolvers/outside-voice.ts', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/carve-guards.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/office-hours-completion.ts', 'test/helpers/llm-judge.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-office-hours-design-draft.test.ts', 'test/helpers/agent-sdk-runner.ts', 'test/helpers/auq-native-capture.ts', 'test/helpers/auto-decision-state.ts', 'test/helpers/autoplan-artifact-digest.ts', 'test/helpers/autoplan-artifact-permission.ts', 'test/helpers/autoplan-artifact-recorder.ts', 'test/helpers/capture-parity-baseline.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty/**', 'test/helpers/dx-selected-navigation.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/eng-cache-writer-decision.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/helpers/native-auto-decide.ts', 'test/helpers/owned-claude-transcript.ts', 'test/helpers/parity-harness.ts', 'test/helpers/plan-count-artifacts.ts', 'test/helpers/plan-count-file-permission.ts', 'test/helpers/plan-count-fixture.ts', 'test/helpers/plan-count-pending-exit.ts', 'test/helpers/plan-count-pending-question.ts', 'test/helpers/plan-count-transcript.ts', 'test/helpers/plan-floor-review.ts', 'test/helpers/plan-floor-target.ts', 'test/helpers/plan-scope-selection.ts', 'test/helpers/plan-seed-submission.ts', 'test/helpers/plan-skill-question-events.ts', 'test/helpers/plan-skill-question-hook-scope.ts', 'test/helpers/plan-skill-questions.ts', 'test/helpers/pty-screen.ts', 'test/helpers/pty-trust-dialog.ts', 'test/helpers/skill-census.ts'],
'dx-error-contract-review': ['test/skill-e2e-dx-error-contract.test.ts', 'test/fixtures/dx-error-contract-case.ts', 'lib/dx-error-contract.ts', 'scripts/resolvers/dx.ts', 'scripts/resolvers/index.ts', 'scripts/gen-skill-docs.ts', 'plan-devex-review/**', 'bin/gstack-autoplan-snapshot.ts', 'lib/autoplan-prompts.ts', 'lib/autoplan-reconcile.ts', 'lib/coordinator-contract.ts', 'test/helpers/session-runner.ts', 'test/helpers/eval-store.ts', 'test/helpers/eval-budgets.ts', 'lib/claude-bin.ts', 'lib/eval-model.ts'],
'plan-devex-peer-comparison-classification': [
+37 -1
View File
@@ -4,7 +4,7 @@ import { describe, expect, test } from 'bun:test';
import * as fs from 'node:fs';
import * as path from 'node:path';
import * as os from 'node:os';
import { validateOfficeHoursCompletion, validateOfficeHoursDesignDraft, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation, type OfficeHoursCompletionEvidence } from './helpers/office-hours-completion';
import { observedSpecFixes, validateOfficeHoursCompletion, validateOfficeHoursDesignDraft, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation, type OfficeHoursCompletionEvidence } from './helpers/office-hours-completion';
import { E2E_TOUCHFILES } from './helpers/touchfiles-data';
import { selectTests } from './helpers/test-selection';
import { expectMentions } from './helpers/prompt-structure';
@@ -954,6 +954,42 @@ Next: /plan-eng-review after observing the workflow. The user declined launching
} finally { fs.rmSync(dir, { recursive: true, force: true }); }
});
test('cap fixes the agent saved and passed to finalize validate; without them the report mismatches (marathon rerun 2026-10-19)', () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'office-hours-cap-fixes-'));
try {
const { evidence, artifacts } = JSON.parse(JSON.stringify(structured())
.replaceAll('/tmp/office-hours-fixture', dir.replaceAll('\\', '/')));
evidence.toolCalls.filter((call: { output?: string }) => call.output?.startsWith('OFFICE_HOURS_VERDICT '))
.forEach((call: { output?: string }, i: number) => { call.output = officeHoursVerdictReceipt(i + 1, artifacts[i].path, artifacts[i].content); });
const applied = 'An absent mapped column stops the import and names the missing column.';
const fixes = [{ id: 'R2-1', applied_text: applied }];
const reviewDir = path.dirname(artifacts[0].path);
const fixesPath = path.join(reviewDir, 'fixes.json');
const reportPath = path.join(dir, 'REPORT.md');
fs.mkdirSync(path.dirname(evidence.designPath), { recursive: true });
fs.writeFileSync(evidence.designPath, evidence.designContent.replace('# Design: RosterCheck\n', `# Design: RosterCheck\n\n${applied}\n`));
fs.writeFileSync(reportPath, evidence.output);
fs.mkdirSync(reviewDir, { recursive: true });
for (const artifact of artifacts) fs.writeFileSync(artifact.path, artifact.content);
const snapshots = snapshotsIn(reviewDir);
for (const snapshot of snapshots) fs.writeFileSync(snapshot.path, snapshot.content);
fs.writeFileSync(fixesPath, JSON.stringify(fixes));
const result = Bun.spawnSync([process.execPath, path.resolve(import.meta.dir, '../bin/gstack-office-hours-review'),
'finalize', '--design', evidence.designPath, '--report', reportPath, '--fixes', fixesPath, ...artifacts.map((artifact: { path: string }) => artifact.path)],
{ cwd: dir, timeout: 5000 });
expect(result.exitCode, result.stderr.toString()).toBe(0);
expect(JSON.parse(result.stdout.toString()).cap.verified).toEqual(['R2-1']);
evidence.output = fs.readFileSync(reportPath, 'utf8');
evidence.designContent = fs.readFileSync(evidence.designPath, 'utf8');
evidence.toolCalls.push({ tool: 'Write', input: { file_path: fixesPath, content: JSON.stringify(fixes) } });
expect(observedSpecFixes(evidence.toolCalls)).toEqual(fixes);
expect(() => checkArtifacts({ ...evidence, fixes: observedSpecFixes(evidence.toolCalls) }, artifacts, snapshots)).not.toThrow();
expect(() => checkArtifacts(evidence, artifacts, snapshots)).toThrow('Spec Review does not match');
expect(() => checkArtifacts({ ...evidence, fixes: [{ id: 'R2-2', applied_text: applied }] }, artifacts, snapshots)).toThrow('Spec Review does not match');
expect(observedSpecFixes(evidence.toolCalls.filter((call: { input?: { file_path?: string } }) => call.input?.file_path !== fixesPath))).toBeNull();
} finally { fs.rmSync(dir, { recursive: true, force: true }); }
});
test('allows a Markdown separator after the exact block, without hiding extra outcome prose', () => {
const { evidence, artifacts } = structured();
evidence.output = evidence.output.replace('## Handoff', '---\n\n## Handoff');
+15
View File
@@ -9,6 +9,7 @@ import {
} from '../lib/office-hours-review';
import { parseSpecFixes, verifySpecFixesAtCap, renderSpecCap } from '../lib/spec-review-cap';
import { expectMentions } from './helpers/prompt-structure';
import { copyOfficeHoursFormatter } from './helpers/office-hours-formatter-fixture';
const cli = path.resolve(import.meta.dir, '../bin/gstack-office-hours-review');
function review(round = 1, count = 1, severity: 'blocking' | 'minor' = 'blocking'): OfficeHoursReview {
@@ -638,3 +639,17 @@ describe('office-hours review CLI', () => {
}
}));
});
describe('office-hours formatter fixture copy', () => {
test('the copied formatter loads every local module it imports (marathon 39032308051)', () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'office-hours-formatter-copy-'));
try {
const formatter = copyOfficeHoursFormatter(dir);
const result = Bun.spawnSync([process.execPath, formatter], { cwd: dir, stdout: 'pipe', stderr: 'pipe', timeout: 5000 });
expect(result.stderr.toString()).not.toContain('Cannot find module');
expect(result.stderr.toString()).toContain('gstack-office-hours-review: usage:');
} finally {
fs.rmSync(dir, { recursive: true, force: true });
}
});
});
+61
View File
@@ -65,6 +65,67 @@ describe('classifyOutsideReview: separate execution, findings and verdict', () =
});
});
// #3294: "clean" comes only from a no-findings statement that concludes the reply,
// never from a "no X" phrase among untagged findings anywhere in it.
describe('a no-findings statement counts only as the reply\'s conclusion (#3294)', () => {
const REPORTER = 'I found no actionable regressions in the staged, unstaged, or untracked changes.';
const FINDING = 'The fallback path now swallows ENOENT and returns an empty config.';
test("the reporter's exact codex review reply reads clean through the reporter's CLI invocation", () => {
expect(classifyOutsideReview({ text: REPORTER, gate: 'structured' }).verdict).toBe('clean');
const r = cli(['--label', 'probe', '--exit', '0', '--stderr', file(''), 'structured', file(REPORTER)]);
expect([r.status, r.stdout]).toEqual([0, 'VERDICT: clean\nFINDINGS: none\n']);
});
test('clean conclusions, with the added regressions/defects vocabulary and the requested marker formatted', () => {
for (const text of ['No regressions found.', 'I did not find any defects.', 'There are no actionable regressions.',
'NO_FINDINGS', '**NO_FINDINGS**', '`NO_FINDINGS`', '- NO_FINDINGS',
`The change adds a retry cap and its tests.\n${REPORTER}`, 'Reviewed the retry helper.\n\nNo issues found in this change.',
'After reviewing the patch, I found no issues.', 'I found no issues in this read-only change.',
'Summary of the retry helper\nNo issues found.', 'I found no actionable regressions in the staged\nchanges.',
// Clean `codex review --uncommitted` replies captured from codex-cli 0.160.0 with gpt-6-sol.
'The change validates the retry limit before execution. No actionable regression was found.',
'The change validates `max` before retrying. No affected callers or tests are present in the repository, and no actionable regression is evident.']) {
expect({ text, verdict: classifyOutsideReview({ text, gate: 'structured' }).verdict }).toEqual({ text, verdict: 'clean' });
}
});
test('mixed replies: untagged findings beside a "no X" phrase are unverified, never clean', () => {
for (const text of [
'The parser change has no new issues. However, db/002_drop_idx.sql drops users_email_idx, which will regress login latency under load.',
'Renaming the flag is no problem for callers, but the fallback path now swallows ENOENT and returns an empty config.',
`No issues in the parser.\n\n${FINDING}`,
`${FINDING}\n\nNo issues in the parser.`,
`${FINDING} No regressions in the existing tests.`,
'No regressions in the existing tests, but the fallback path now swallows ENOENT.',
`${FINDING}\nI found no issues other than the fallback path.`,
`${FINDING}\nNo other issues.`,
'No issues found, except that the fallback path now swallows ENOENT.',
'No issues in the parser, and the fallback path now swallows ENOENT.',
'Missing configuration now silently disables authentication. No regressions were found by the existing tests.',
'Missing configuration now silently disables authentication. No issues are apparent in the parser.',
'The fallback path now swallows ENOENT, and no issues elsewhere.',
'The diff adds a NO_FINDINGS check, but the regex misses lowercase input.',
`${FINDING}\n\`\`\`\nNo issues found.\n\`\`\``,
]) {
for (const gate of ['structured', 'review'] as const) {
const r = classifyOutsideReview({ text: gate === 'review' ? `${text}\nRecommendation: ship because it looks fine.` : text, gate });
expect({ text, gate, verdict: r.verdict, reason: r.reason }).toEqual({ text, gate, verdict: 'unverified', reason: 'untagged_review' });
}
}
});
test("a review's Recommendation because-clause concludes under the same rules", () => {
const verdict = (text: string) => classifyOutsideReview({ text, gate: 'review' }).verdict;
expect(verdict('Recommendation: ship because I found no regressions in this change.')).toBe('clean');
expect(verdict(`The change adds a retry cap.\nRecommendation: ship because no issues remain.`)).toBe('clean');
expect(verdict('No issues in the parser.\nRecommendation: fix the fallback because it swallows ENOENT.')).toBe('unverified');
expect(verdict(`${FINDING}\nRecommendation: ship because no regressions in the existing tests.`)).toBe('unverified');
expect(verdict(`${FINDING}\nRecommendation: ship because no issues remain, but the fallback needs a follow-up.`)).toBe('unverified');
expect(verdict('Recommendation: address the migration because no regressions were found in the parser change,\nbut the migration drops users_email_idx.')).toBe('unverified');
});
});
describe('validateOutsideReview keeps the text-only shape for direct importers', () => {
test('existing shapes are unchanged', () => {
expect(validateOutsideReview('', 'review')).toEqual({ completed: false, reason: 'empty response' });
@@ -14,7 +14,8 @@ import * as path from 'node:path';
import { describeE2ETier } from './helpers/e2e-gate';
import { setupSkillDir, skillFromWorktree, captureSectionReads } from './helpers/auq-sdk-capture';
import { CARVE_GUARDS } from './helpers/carve-guards';
import { validateOfficeHoursCompletion, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation } from './helpers/office-hours-completion';
import { copyOfficeHoursFormatter } from './helpers/office-hours-formatter-fixture';
import { observedSpecFixes, validateOfficeHoursCompletion, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation } from './helpers/office-hours-completion';
const describeMarathon = describeE2ETier('marathon');
const runId = `office-hours-section-loading-${process.env.EVALS_RUN_ID ?? 'local'}`;
@@ -42,15 +43,9 @@ describeMarathon('/office-hours full section-loading workflow (marathon)', () =>
const planDir = setupSkillDir({
skillName: guard.skill, skillMd, sectionsFrom, tmpPrefix: 'gstack-office-hours-secload-',
});
// Copy the real formatter and its small dependency set into the fixture.
// Copy the real formatter and its local imports into the fixture.
// This exercises production code while keeping all outputs in the sandbox.
for (const relative of ['bin/gstack-office-hours-review', 'lib/office-hours-review.ts', 'lib/fs-atomic.ts']) {
const target = path.join(planDir, relative);
fs.mkdirSync(path.dirname(target), { recursive: true });
fs.copyFileSync(path.resolve(__dirname, '..', relative), target);
}
const formatter = path.join(planDir, 'bin/gstack-office-hours-review');
fs.chmodSync(formatter, 0o755);
const formatter = copyOfficeHoursFormatter(planDir);
// Two rounds keep the live run inside its budget: round 1 (full) and round 2 (delta) run here; round 3 and CONVERGENCE are covered by free validator tests.
const reviewRounds = 2;
const capture = await captureSectionReads({
@@ -83,6 +78,7 @@ Completion delivery, after the full workflow and design approval:
const reviewEvidence = validateOfficeHoursReviewArtifacts({
...capture, designPath,
designContent: fs.existsSync(designPath) ? fs.readFileSync(designPath, 'utf-8') : null,
fixes: observedSpecFixes(capture.toolCalls),
}, artifacts, snapshots, reviewRounds);
validateOfficeHoursReviewerHandoffs({
...capture, designPath,