mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-04 15:57:33 +01:00
Merge remote-tracking branch 'origin/main' into crew/port-3129-when-to-invoke
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> # Conflicts: # CHANGELOG.md
This commit is contained in:
commit
ff9305ef58
72 files changed
+1774
-303
No files matched your search
+1
-1
@@ -91,7 +91,7 @@ The server writes `.gstack/browse.json` (atomic write via tmp + rename, mode 0o6
|
||||
{ "pid": 12345, "port": 34567, "token": "uuid-v4", "startedAt": "...", "binaryVersion": "abc123" }
|
||||
```
|
||||
|
||||
The CLI reads this file to find the server. If the file is missing or the daemon process is dead, the CLI spawns a new server. A process that is alive but not answering `/health` is busy, not dead: the CLI probes for a bounded ~8s, then reports busy with a nonzero exit — only an explicit `--force-restart` kills a live daemon. Process liveness uses signal-0 (`isProcessAlive`, EPERM counts as alive) on every platform, with the health check (GET /health) as the responsiveness signal. Daemon stdout/stderr persists to `<project>/.gstack/browse-daemon.log`.
|
||||
The CLI reads this file to find the server. If the file is missing or the daemon process is dead, the CLI spawns a new server. A process that is alive but not answering `/health` is busy, not dead: the CLI probes for a bounded ~8s, then reports busy with a nonzero exit — only an explicit `--force-restart` kills a live daemon. Process liveness uses signal-0 (`isProcessAlive`, EPERM counts as alive) on every platform, with the health check (GET /health) as the responsiveness signal. Its `status` covers the daemon and its Chromium connection only; the active tab's main thread is reported separately (`activeTab.responsive`, a probe bounded to 1s), so one page stuck in a long script never makes the CLI refuse commands. Daemon stdout/stderr persists to `<project>/.gstack/browse-daemon.log`.
|
||||
|
||||
### Port selection
|
||||
|
||||
|
||||
+9
-1
@@ -584,7 +584,7 @@ SITE=app.example.com
|
||||
DOMAIN=example.com
|
||||
$B goto "https://$SITE"
|
||||
$B cookie-import-browser opera-gx --domain "$DOMAIN" --profile Default # or: opera
|
||||
$B reload # confirm the intended account
|
||||
$B reload # view the page; only --verify-auth proves sign-in
|
||||
```
|
||||
|
||||
Run `$B cookie-import-browser` with no flags to see which browsers were detected. `--profile` can be omitted when only one profile has cookies for the domain. If the receipt reports App-Bound Encryption, run `$B handoff`, sign in to the intended account in the window that opens, then `$B resume` (needs a display).
|
||||
@@ -626,6 +626,14 @@ Storage stays intact by default. With explicit approval on a Chromium target, `-
|
||||
| `tab-each <command> [args...]` | Fan out a command across every open tab; returns JSON |
|
||||
| `frame <sel\|@ref\|--name n\|--url pattern\|main>` | Switch to iframe context (or back to main); clears refs |
|
||||
|
||||
**A busy tab is not a busy daemon.** `/health` `status` is daemon liveness
|
||||
(Chromium connected); the active tab's main thread is reported separately as
|
||||
`activeTab: { id, responsive }`, probed for at most 1s. While one page is stuck
|
||||
in a long script, `status`, `tabs`, `tab <id>`, `newtab`, `closetab`,
|
||||
`state save` and `stop` keep working, and `status` and `tabs` mark the tab as
|
||||
not responding. A command that needs the stuck tab times out after 30s and
|
||||
names it, so you can switch away or close it without `--force-restart`.
|
||||
|
||||
### Extraction
|
||||
|
||||
| Command | Description |
|
||||
|
||||
@@ -19,6 +19,83 @@
|
||||
|
||||
One new free test checks that the root router routes every voice trigger, and it skips the `connect-chrome` alias symlink. No threshold, trial count, case kind or judge changed. No context-budget ceiling rose except the root router's.
|
||||
|
||||
## [1.91.156.0] - 2026-10-19
|
||||
|
||||
**The weekly marathon's office-hours case can run its review formatter again, and its validator now checks cap fixes the way the formatter does.** Both defects came from #3164 and were in the test, not the skill. The marathon fixture copied `bin/gstack-office-hours-review` with a hand-kept list of two library files; #3164 added a third import (`lib/spec-review-cap.ts`), so every formatter command in the fixture failed at module load, the agent correctly declared the review UNREVIEWED, and the case went red in marathon run 39032308051 (#3246). With the module restored, a focused rerun showed the second defect: when the review stops at the round cap and the agent applies the remaining blocking fixes with `--fixes`, the formatter records them as verified, but the test never gave the validator that list, so it expected "fixes unconfirmed" and rejected a correct report.
|
||||
|
||||
### Fixed
|
||||
- **Office-hours cap fixes:** `observedSpecFixes()` in `test/helpers/office-hours-completion.ts` returns the last valid `--fixes` list the agent saved with Write, and the marathon test passes it to `validateOfficeHoursReviewArtifacts`. A free test in `test/office-hours-completion.test.ts` runs the real `finalize --fixes` at the cap and checks three cases: the observed list validates, no list fails with the marathon rerun's `Spec Review does not match` error, and a list naming a different finding also fails.
|
||||
- **Office-hours marathon fixture:** `test/helpers/office-hours-formatter-fixture.ts` copies the formatter together with every local module it imports, so a new import can no longer be left out. A free test in `test/office-hours-review.test.ts` loads the copied formatter and fails on a missing module; the old three-file list reproduces the exact `Cannot find module '../lib/spec-review-cap.ts'` error. The case's touchfiles now include `lib/spec-review-cap.ts` and the new helper.
|
||||
- **Gate integrity:** no threshold, trial count, case kind or judge changed. The marathon assertions are unchanged; only the fixture's file copy changed.
|
||||
|
||||
## [1.91.155.0] - 2026-10-19
|
||||
|
||||
**A busy page no longer locks up the browse daemon.** When one tab kept its main thread busy (a heavy dev bundle, a big grid render, a runaway script), every `browse` command, including `status`, `tabs`, `tab <id>`, `closetab` and `stop`, failed with "Daemon busy" after about 9.6 s, and the only way out was `--force-restart`, which threw away cookies, tabs and logins. Fixes #3247. From #3250 by @AndreIntelas.
|
||||
|
||||
### Fixed
|
||||
- **`/health` `status` reports daemon liveness (Chromium connected)**, and the active tab's responsiveness is reported separately as `activeTab: {id, responsive}`. The CLI no longer gates every command on a 2 s `page.evaluate` race against the active tab.
|
||||
- **`tabs` and `state save` bound their per-tab reads**, so a stuck tab shows as `(not responding)` instead of hanging the command, and one pending probe is shared per stuck tab rather than one per poll. A busy tab keeps its last good storage in saved state.
|
||||
- **A command that really needs the stuck tab times out with an error naming that tab** and how to get out (switch tabs or close it). The CLI's timeout branch now matches Bun's `TimeoutError`; the old `AbortError` check never fired.
|
||||
|
||||
### Changed
|
||||
- **Gate integrity:** `browse/test/terminal-agent.test.ts` adds `activeTab` to the exact `/health` keys and updates its stub for the now-synchronous `isHealthy` and the new `probeActiveTab`; its no-token assertions are unchanged. `browse/test/busy-active-tab.test.ts` and `browse/test/command-timeout-report.test.ts` are new. No threshold, trial count or case kind changed.
|
||||
## [1.91.154.0] - 2026-10-19
|
||||
|
||||
**/cso can verify Rails repairs in a read-only checkout, its retained Git history no longer brings back credential files the snapshot leaves out, a repair whose external assertions fail is now reported as `ASSERTION_FAILED` instead of a misleading `INCOMPATIBLE_INPUT`, and a security assertion can accept several safe rejection statuses while still requiring the secret to be absent.** The helper keeps `.env`, `master.key`, keys and agent configuration out of the readable tree, but `history.txt` captured `git log -p` in full, so a committed `.env` came back through its diff whenever the value wasn't credential-shaped enough for the redactor. Paid qualification run 39186328569 caught this when the private corpus's planted `.env` value showed up in the judge's helper state.
|
||||
|
||||
### Fixed
|
||||
- **Rails `verify` on read-only sources:** the helper writes its own test `config/database.yml` (plus an initializer and `storage.yml`) into the execution copy. The copy keeps the source's file modes, so in a read-only checkout (the eval producer seals every source file at 0444, and Nix or Bazel trees are the same) the write failed with `EACCES`. That surfaced as the generic `INVALID_SCHEMA` "unexpected or unsafe input", and no Rails repair could reach `runtime_tested`. Each helper file now replaces the copied file instead of opening it for writing, so it also never writes through a copied symlink.
|
||||
- **`history.txt`:** every diff section that touches a path the snapshot excludes (`exclusion()`, the same rule the readable tree uses) is replaced with `diff withheld: <path> (<reason>)`. This covers either side of a rename and any diff header that can't be parsed. Commit preambles and other files' diffs stay.
|
||||
- **`verify` with an authenticated witness:** a receipt or observation mismatch is still `INCOMPATIBLE_INPUT`. Receipts that bind correctly but record failed external assertions now raise `ASSERTION_FAILED`, with the phase and the booted, legitimate and security outcome. Nothing new is certified either way. In run 39186328569, three repairs that rejected the attack with a different status than the oracle expected surfaced as `INCOMPATIBLE_INPUT`.
|
||||
|
||||
### Changed
|
||||
- **A security assertion's `expected.status` may list up to eight safe statuses** (for example `[403, 404]`), so a repair that refuses an attack with a different reasonable rejection still passes. A set is accepted only on a security assertion that also carries a `vulnerable` expectation, and only with `excludes` set, so the response must still withhold the vulnerable marker. The fixed and vulnerable oracles must stay provably mutually exclusive: a set that contains the vulnerable status is accepted only when the vulnerable response's `includes` contains the excluded marker. Legitimate controls and `vulnerable` expectations still take exactly one status, so a patch that answers every route with 403 still fails its controls. A no-op patch, a listed rejection that still leaks the marker, and an unlisted status all stay `inconclusive`.
|
||||
|
||||
## [1.91.153.0] - 2026-10-19
|
||||
|
||||
**An outside review whose findings are untagged prose no longer reads `clean` because a "no issues" phrase appears somewhere in it, and a clean Codex review that says "no actionable regressions" no longer reads `unverified`.** Reported in #3294 by @SchizasNik, with the false-pass half pointed out by @xg-gh-25. `lib/outside-review-result.ts` matched its no-findings vocabulary anywhere in the reply, so "The parser change has no new issues. However, … drops users_email_idx" passed /ship's structured gate. It also lacked `regressions`, so `codex review --uncommitted`'s "I found no actionable regressions in the staged, unstaged, or untracked changes." was missing coverage.
|
||||
|
||||
### Fixed
|
||||
- **A no-findings statement counts only as the review's conclusion.** It must open the reply's last sentence or last ", and" clause (or the `because` clause of a review's `Recommendation:` line) and carry no contrast or exception ("but", "however", "except", "other than", "elsewhere"). When other prose comes first, anything after the statement other than a predicate ("were found", "is evident") or a colon explanation must scope it to the change as a whole, so "No issues in the parser." and "No regressions were found by the existing tests." do not conclude the review. Anything else stays `unverified` / `untagged_review`, the same reason code and troubleshooting anchor as before.
|
||||
- **`regressions` and `defects` join the vocabulary**, and `NO_FINDINGS` written as inline code still counts.
|
||||
|
||||
### Gate integrity
|
||||
- The structured and review gates' no-findings check (`classifyOutsideReview`) is stricter about position and wider in vocabulary. New free regressions in `test/outside-review-result.test.ts` cover the reporter's exact reply through their CLI invocation, two clean `codex review --uncommitted` replies captured from codex-cli 0.160.0 with gpt-6-sol (both read `unverified` before), fifteen mixed replies that must stay `unverified` under both gates (the two strings from the issue, the phrase before, after and inside untagged findings, exception conclusions, conclusions scoped to one place or to the tests, a quoted `NO_FINDINGS`, a no-issues line inside a code fence), and Recommendation-clause cases including a wrapped clause with "but". A GPT-6.1 Sol outside review probed the classifier; its false passes are among those controls. A known residual, unchanged from before: untagged findings followed by "No issues found in the changed parser." still read `clean`, because "changed" names the change and the parser cannot tell a finding from a summary without tags. All four new tests fail on the previous classifier. Every existing negative control passes unchanged, including the stored Codex clean review that still reads `unverified` ("no actionable correctness issues"). No threshold, trial count or case kind changed.
|
||||
|
||||
## [1.91.152.0] - 2026-10-19
|
||||
|
||||
**`/cso` now starts and finishes on ordinary repositories that hold a nested Git repository, a long patch history, or a Docker socket on macOS 27.** Three user reports (#3255, #3248) each stopped a run on a normal repo.
|
||||
|
||||
### Fixed
|
||||
- **An untracked nested Git repository or a registered submodule no longer stops `gstack-cso start` with `UNSAFE_PATH`.** Git lists such a directory as one entry (`inner/` for an untracked repo, `sub` for a gitlink). The snapshot now records it as one explicit `excluded: nested Git repository` entry that is never read or copied, and the snapshot coverage lists it under exclusions. The capture pins the boundary's device, inode and mode, so a directory swapped in during capture fails as `SNAPSHOT_RACE`. A symlink to a directory is still refused. Ported from #3256 by Chao Chen (@surlymochan), who keeps authorship of the fix. (#3255, bug 1)
|
||||
- **A patch history over 1 MiB no longer leaves `history-inputs` not assessed.** The helper reads up to 100 commits of patches, then keeps the newest whole commits that fit the 1 MiB redaction bound. It never cuts inside a commit, so a secret can't be split across the cut and escape redaction. When even the newest patch is too large, it keeps commit metadata only. A truncated `history.txt` starts with a `[gstack-cso history truncated: …]` line, `history-status.json` records `truncated` and the retained commit count, and the history coverage becomes `partial` with that gap. The redaction bound is unchanged. (#3255, bug 2)
|
||||
|
||||
### For contributors
|
||||
- **`test/cso-ordinary-repos.test.ts`** covers each report end to end through the compiled launcher and `capture()`. Its negative controls show that a symlinked directory is still refused, that a nested repository swapped during capture fails as a race (this check fails if the device and inode comparison is removed), that retained and omitted history never shows a credential canary, and that Docker endpoint admission still refuses non-sockets, dangling and looping links, `..` paths and TCP. The #3248 socket fallback (`realpathSocket`) shipped in 1.91.96.0. The new endpoint-level case fails when admission calls plain `realpathSync`.
|
||||
|
||||
## [1.91.151.0] - 2026-10-19
|
||||
|
||||
**Three contributor fixes: a saved "never ask" now sticks for /plan-eng-review's structure question, /context-save and /context-restore close three checkpoint gaps, and a new how-to covers the first production deploy.** Ported from fork PRs so they can run the paid gate; each commit keeps its author.
|
||||
|
||||
### Fixed
|
||||
- **/plan-eng-review's complexity-gate structure question has a registered id (`plan-eng-review-structure`).** Each run used to invent its own slug, so a `never-ask` saved through /plan-tune never matched the next run and the question came back every time. On `AUTO_DECIDE` the recommended arrangement is taken; feature cuts are still asked one by one. `gstack-question-preference --check` now also falls back to `$GSTACK_HOME/global-question-preferences.json` (the file the question-preference hook already reads) when the project file has no entry; a project entry still wins and the one-way-door and split-chain guards run first. From #3249 by @bezdekt.
|
||||
- **/context-save keeps provenance markers as English literals and backs file claims in Decisions Made; /context-restore treats a translated or unknown marker as `(path assumed)` and asks when no branch can pick between recent checkpoints with different titles.** Fixes #3217. From #3244 by @surlymochan.
|
||||
|
||||
### Added
|
||||
- **`docs/howto-bring-up-a-first-deploy.md`:** a vendor-neutral walkthrough for when /setup-deploy stops at "Nothing is deployed yet" (host, services, env and secrets, DNS, verification, and what to have ready when you rerun /setup-deploy). Linked from the README and docs/skills.md. Fixes #3150. From #3264 by @szsunyuan.
|
||||
## [1.91.150.0] - 2026-10-19
|
||||
|
||||
**The paid evals that still pinned older models now run on the current generation, and two cases that went red on the move are repaired at their cause.** Garry's direction (2026-10-18) is to stop testing older frontier models, so these files are re-baselined as a fresh measured reference rather than held to a threshold change.
|
||||
|
||||
### Changed
|
||||
- **Old-model pins:** the `claude-opus-4-7` pins in `skill-e2e-plan`, `skill-e2e-plan-prosons`, `skill-e2e-design`, `skill-e2e-office-hours-phase4` and `skill-e2e-qa-bugs` now follow `resolveEvalModel('capture')` (Opus 5.5). The overlay benchmark fixtures pin `claude-opus-5-5` and `claude-sonnet-5-5`, the cookie workflow judge pins `claude-sonnet-5-5`, and `codex-sol-scope-termination` runs on `gpt-6.1-sol` while still rendering the `gpt-5.6-sol` overlay.
|
||||
- **/setup-browser-cookies:** the workflow is now five ordered steps. It checks for Aside first and ends there on `READY`, since no import is needed. It then asks every decision (source and profile, domains and target, verification, storage reset, all cookies) before any `$B` command, prepares gstack's browser (verification variables before the daemon starts, the fallback, the `Mode: cdp` stop), imports, and reports on three lines: Imported, Storage reset, Sign-in. The `{{BROWSE_SETUP}}` block, which contradicted the Aside probe, is gone from this skill.
|
||||
- **Gate integrity:** no threshold, trial count, case kind or assertion changed. Free-test edits follow the new cookie template: two phrase and order assertions, the approval fixture's reverse edits, and the positional-literals checksum list, which no longer includes this skill.
|
||||
|
||||
### Fixed
|
||||
- **plan-review-prosons-hardstop-neg:** on Opus 5.5 one trial in three stopped with a provider refusal (`stop_reason: refusal`, category `reasoning_extraction`) on the turn that writes the captured question. Replays of the refused request were stopped 7 of 16 times with the capture prompt's "verbatim" and "Do NOT paraphrase. This is a format-capture test." wording and 0 of 24 without it. The shared prosons capture prompt now asks for "the text of the single AskUserQuestion you would have made", with the same required parts. The decision content and every assertion are unchanged; all four prosons cases pass 3/3.
|
||||
- **Cookie workflow judge:** on `claude-sonnet-5-5` it scored clarity 3, completeness 4, actionability 3 against the old template. The rewritten template scores 4/4/4 with unchanged thresholds.
|
||||
|
||||
## [1.91.149.0] - 2026-10-19
|
||||
|
||||
**Paid eval cases can now run on the Codex host, and the Monday census runs two arms: Claude Opus 5.5 and Codex gpt-6.1-sol.** `EVALS_HOST=codex` runs the 52 adapter-only gate cases through a full Codex runtime per case. On its first pilot gpt-6.1-sol passed 48 of 52 with no retries, at $4.63 for the 52 cases, against $28.93 for the same cases on Fable (0.16×). Claude-only cases are skipped on Codex with a recorded reason and never count as failed. GSTA-49 TM2.
|
||||
|
||||
@@ -656,6 +656,7 @@ Other references: [docs/gbrain-sync.md](docs/gbrain-sync.md) (sync-specific guid
|
||||
|-----|---------------|
|
||||
| [Skill Deep Dives](docs/skills.md) | Philosophy, examples, and workflow for every skill (includes Greptile integration) |
|
||||
| [Diagrams & Document Formats](docs/howto-diagrams-and-formats.md) | Mermaid/excalidraw fences in PDFs, image sizing and safety defaults, `--to html\|docx`, `/diagram` triplets |
|
||||
| [First Production Deploy](docs/howto-bring-up-a-first-deploy.md) | What to do when `/setup-deploy` says nothing is deployed yet: hosting, env and secrets, DNS, checking it works, then back to `/setup-deploy` |
|
||||
| [Builder Ethos](ETHOS.md) | Builder philosophy: Boil the Ocean, Search Before Building, three layers of knowledge |
|
||||
| [Using GBrain with GStack](USING_GBRAIN_WITH_GSTACK.md) | Every path, flag, bin helper, and troubleshooting step for `/setup-gbrain` |
|
||||
| [GBrain Sync](docs/gbrain-sync.md) | Cross-machine memory setup, privacy modes, troubleshooting |
|
||||
|
||||
@@ -1645,19 +1645,24 @@ and `test/dx-selected-navigation-ap.test.ts`. One shared table run once against
|
||||
only after `engFirstReviewAUQ` checks native completion once at entry; today each branch gates it
|
||||
separately, so the change alters a paid verdict and needs its own paid run.
|
||||
|
||||
### P3: Re-pin the five remaining claude-opus-4-7 paid files
|
||||
### P3: Follow-ups from re-baselining the old-model paid pins on the current generation
|
||||
|
||||
**What:** The 2026-09 audit moved seven paid evals to the default capture model (`resolveEvalModel('capture')`).
|
||||
`skill-e2e-design`, `skill-e2e-office-hours-phase4`, `skill-e2e-plan-prosons` and `skill-e2e-plan` keep
|
||||
`claude-opus-4-7` because six cases failed on the default model in one run (plan-design-review-plan-mode timeout,
|
||||
office-hours-phase4-fork format, plan-review-prosons-neutral-neg missing output, plan-ceo-review-selective and
|
||||
plan-eng-review 600 s timeouts, plan-ceo-review-expansion-energy posture score 3). plan-review-report, the one PR-gate
|
||||
case in `skill-e2e-plan`, left the pin on 2026-10-18 and follows the capture default. `skill-e2e-qa-bugs` returned
|
||||
to `claude-opus-4-7` after `qa-b6-static` timed out on the default model in two of three runs (census 36597762183
|
||||
and a targeted local rerun): each time the stream stopped mid-message, with no pending tool, right after the model
|
||||
found the disabled submit button, and emitted nothing until the 300 s case deadline. They measure an old model.
|
||||
**What:** GSTA-49 TM3 (2026-10-18) moved every remaining old-model paid pin to the current generation: the
|
||||
`claude-opus-4-7` pins in `skill-e2e-{plan,plan-prosons,design,office-hours-phase4,qa-bugs}` follow
|
||||
`resolveEvalModel('capture')`, the overlay fixtures pin `claude-opus-5-5` and `claude-sonnet-5-5`, the cookie workflow
|
||||
judge pins `claude-sonnet-5-5`, and `codex-sol-scope-termination` runs on `gpt-6.1-sol` (still rendering the
|
||||
`gpt-5.6-sol` overlay). Three things remain:
|
||||
|
||||
**Re-entry:** fix the prompt, budget or rubric so each case passes on the default model in one run, then drop the pin.
|
||||
- The base `claude-dedicated-tools-vs-bash` fixture now duplicates `claude-dedicated-tools-vs-bash-opus-5-5`.
|
||||
- The `opus-4-7-*` fixtures measure `model-overlays/opus-4-7.md` on Opus 5.5, which gets the generic `claude`
|
||||
overlay in product.
|
||||
- Opus 5.5 refuses (stop category `reasoning_extraction`) capture prompts that ask for the "verbatim text" of a
|
||||
question "you would have made" with "Do NOT paraphrase"; that wording stopped 7 of 16 replays of a prosons
|
||||
request and 0 of 24 without it. `skill-e2e-plan-prosons` dropped it; `skill-e2e-plan-format` and
|
||||
`skill-e2e-office-hours-phase4` still use it.
|
||||
|
||||
**Re-entry:** retire one dedicated-tools fixture, decide the opus-4-7 overlay's future, and drop the extraction
|
||||
wording from the remaining capture prompts with a paid run each.
|
||||
|
||||
### P3: Retire the unused CEO payment seeder
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# gstack digest v1.91.149.0 — regenerate/re-copy after upgrading gstack
|
||||
# gstack digest v1.91.156.0 — regenerate/re-copy after upgrading gstack
|
||||
|
||||
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
|
||||
for agent hosts without a full skill install. The full skills add workflows,
|
||||
|
||||
@@ -2,6 +2,9 @@
|
||||
# gstack-question-preference — read/write/check explicit per-question preferences.
|
||||
#
|
||||
# Preference file: ~/.gstack/projects/{SLUG}/question-preferences.json
|
||||
# Global fallback: ~/.gstack/global-question-preferences.json, the file the
|
||||
# question-preference hook reads (--check only; a project entry for the same
|
||||
# id wins). Hand-edited; --write stays per-project.
|
||||
# Schema: { "<question_id>": "always-ask" | "never-ask" | "ask-only-for-one-way" }
|
||||
#
|
||||
# Subcommands:
|
||||
@@ -42,6 +45,7 @@ GSTACK_HOME="$_gstack_sr_root"
|
||||
eval "$("$SCRIPT_DIR/gstack-slug" 2>/dev/null || true)"
|
||||
SLUG="${SLUG:-unknown}"
|
||||
PREF_FILE="$GSTACK_HOME/projects/$SLUG/question-preferences.json"
|
||||
GLOBAL_PREF_FILE="$GSTACK_HOME/global-question-preferences.json"
|
||||
EVENT_FILE="$GSTACK_HOME/projects/$SLUG/question-events.jsonl"
|
||||
mkdir -p "$GSTACK_HOME/projects/$SLUG"
|
||||
|
||||
@@ -101,13 +105,23 @@ do_check() {
|
||||
fi
|
||||
ensure_file
|
||||
cd "$ROOT_DIR"
|
||||
PREF_FILE_PATH="$PREF_FILE" QID="$QID" QSUMMARY="$QSUMMARY" bun -e "
|
||||
PREF_FILE_PATH="$PREF_FILE" GLOBAL_PREF_FILE_PATH="$GLOBAL_PREF_FILE" QID="$QID" QSUMMARY="$QSUMMARY" bun -e "
|
||||
import('./scripts/one-way-doors.ts').then((oneway) => {
|
||||
const fs = require('fs');
|
||||
const qid = process.env.QID;
|
||||
const summary = process.env.QSUMMARY || undefined;
|
||||
const prefs = JSON.parse(fs.readFileSync(process.env.PREF_FILE_PATH, 'utf-8'));
|
||||
const pref = prefs[qid];
|
||||
let globalPrefs = {};
|
||||
try {
|
||||
if (fs.existsSync(process.env.GLOBAL_PREF_FILE_PATH)) {
|
||||
const parsed = JSON.parse(fs.readFileSync(process.env.GLOBAL_PREF_FILE_PATH, 'utf-8'));
|
||||
if (parsed === null || typeof parsed !== 'object' || Array.isArray(parsed)) throw new Error('not an object');
|
||||
globalPrefs = parsed;
|
||||
}
|
||||
} catch (e) {
|
||||
console.error('NOTE: global-question-preferences.json is unreadable; ignoring it.');
|
||||
}
|
||||
const pref = qid in prefs ? prefs[qid] : globalPrefs[qid];
|
||||
|
||||
// Always check one-way status first — safety overrides preferences.
|
||||
// summary (when piped) lets the keyword net catch ad-hoc destructive
|
||||
|
||||
+122
-30
@@ -290,6 +290,44 @@ export async function handleChromiumDisconnect(browser: Browser | null): Promise
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
/**
|
||||
* How long a tab-independent read waits on one tab's renderer (#3247). A page
|
||||
* stuck in a long script never answers page.evaluate() or page.title(); before
|
||||
* this bound, one such tab made /health call the whole daemon unhealthy and
|
||||
* hung `tabs`. Kept well under the CLI's /health request timeout
|
||||
* (HEALTH_REQUEST_TIMEOUT_MS in cli.ts) so the CLI reads the answer instead of
|
||||
* timing out on it.
|
||||
*/
|
||||
export const TAB_PROBE_TIMEOUT_MS = 1_000;
|
||||
|
||||
/**
|
||||
* Budget for saveState()'s per-tab storage read. Longer than the probe: a tab
|
||||
* that misses it falls back to the last storage it answered with, which may
|
||||
* be stale, so only a tab that is really stuck takes that path.
|
||||
*/
|
||||
const TAB_STORAGE_READ_TIMEOUT_MS = 5_000;
|
||||
|
||||
type TabStorage = { localStorage: Record<string, string>; sessionStorage: Record<string, string> };
|
||||
type TabReadKind = 'probe' | 'title' | 'storage';
|
||||
|
||||
function urlOrigin(url: string): string {
|
||||
return URL.canParse(url) ? new URL(url).origin : '';
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve with `work`'s value, or with undefined once `ms` passes. The losing
|
||||
* read settles on its own when the renderer frees up, so callers pass a promise
|
||||
* that cannot reject.
|
||||
*/
|
||||
function withinTabBudget<T>(work: Promise<T>, ms: number): Promise<T | undefined> {
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
const deadline = new Promise<undefined>((resolve) => {
|
||||
timer = setTimeout(() => resolve(undefined), ms);
|
||||
(timer as { unref?: () => void }).unref?.();
|
||||
});
|
||||
return Promise.race([work, deadline]).finally(() => clearTimeout(timer));
|
||||
}
|
||||
|
||||
export type { RefEntry };
|
||||
|
||||
// Re-export TabSession for consumers
|
||||
@@ -300,7 +338,7 @@ export interface BrowserState {
|
||||
pages: Array<{
|
||||
url: string;
|
||||
isActive: boolean;
|
||||
storage: { localStorage: Record<string, string>; sessionStorage: Record<string, string> } | null;
|
||||
storage: TabStorage | null;
|
||||
/**
|
||||
* HTML content loaded via load-html (setContent), replayed after context recreation.
|
||||
* In-memory only — never persisted to disk (HTML may contain secrets or customer data).
|
||||
@@ -360,6 +398,13 @@ export class BrowserManager {
|
||||
private extensionContext = false;
|
||||
/** Pending and recorded navigation-guard verdicts per page (D2). */
|
||||
private navigationGuards = new WeakMap<Page, { pending: Set<Promise<void>>; blocked: string | null }>();
|
||||
/** One pending read per page and kind (#3247). A read that outlives its
|
||||
* budget stays queued in the renderer until the page frees up, so a stuck
|
||||
* tab reuses it instead of collecting a new one on every /health poll,
|
||||
* `tabs` call or persist tick. */
|
||||
private pendingTabReads = new WeakMap<Page, Map<TabReadKind, Promise<unknown>>>();
|
||||
/** The last storage each page answered with, and the origin it came from. */
|
||||
private lastTabStorage = new WeakMap<Page, { origin: string; storage: TabStorage }>();
|
||||
// Our own 'disconnected' listeners per browser. Detach only these: removing
|
||||
// every 'disconnected' listener also strips Playwright's internal one that
|
||||
// resolves browser.close(), so each close waited the full race timeout (and
|
||||
@@ -1090,20 +1135,43 @@ export class BrowserManager {
|
||||
this.closeOwnedDisplay();
|
||||
}
|
||||
|
||||
/** Health check — verifies Chromium is connected AND responsive */
|
||||
async isHealthy(): Promise<boolean> {
|
||||
if (!this.browser || !this.browser.isConnected()) return false;
|
||||
try {
|
||||
const page = this.pages.get(this.activeTabId);
|
||||
if (!page) return true; // connected but no pages — still healthy
|
||||
await Promise.race([
|
||||
page.evaluate('1'),
|
||||
new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 2000)),
|
||||
]);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
/**
|
||||
* Daemon liveness for /health: Chromium is connected. The active tab is NOT
|
||||
* part of it (#3247): a page stuck in a long script is one busy tab, not a
|
||||
* dead daemon, and the CLI gates every command on this answer — including
|
||||
* the tabs, tab, closetab and stop commands that get out of that state.
|
||||
* probeActiveTab() reports the tab on its own.
|
||||
*/
|
||||
isHealthy(): boolean {
|
||||
return Boolean(this.browser?.isConnected());
|
||||
}
|
||||
|
||||
/**
|
||||
* Does the active tab's main thread answer within `timeoutMs`? Null when
|
||||
* there is no active page. A rejected evaluate (crashed or closing target)
|
||||
* also counts as not responsive.
|
||||
*/
|
||||
async probeActiveTab(timeoutMs = TAB_PROBE_TIMEOUT_MS): Promise<{ id: number; responsive: boolean } | null> {
|
||||
const id = this.activeTabId;
|
||||
const page = this.pages.get(id);
|
||||
if (!page) return null;
|
||||
const probe = this.sharedTabRead(page, 'probe', () => page.evaluate('1').then(() => true, () => false));
|
||||
return { id, responsive: (await withinTabBudget(probe, timeoutMs)) === true };
|
||||
}
|
||||
|
||||
/** Join the page's pending read of this kind, or start one. `read` must not
|
||||
* reject. */
|
||||
private sharedTabRead<T>(page: Page, kind: TabReadKind, read: () => Promise<T>): Promise<T> {
|
||||
let reads = this.pendingTabReads.get(page);
|
||||
if (!reads) {
|
||||
reads = new Map();
|
||||
this.pendingTabReads.set(page, reads);
|
||||
}
|
||||
const pending = reads.get(kind);
|
||||
if (pending) return pending as Promise<T>;
|
||||
const started = read().finally(() => reads.delete(kind));
|
||||
reads.set(kind, started);
|
||||
return started;
|
||||
}
|
||||
|
||||
// ─── Tab Management ────────────────────────────────────────
|
||||
@@ -1301,17 +1369,23 @@ export class BrowserManager {
|
||||
return released;
|
||||
}
|
||||
|
||||
async getTabListWithTitles(): Promise<Array<{ id: number; url: string; title: string; active: boolean }>> {
|
||||
const tabs: Array<{ id: number; url: string; title: string; active: boolean }> = [];
|
||||
for (const [id, page] of this.pages) {
|
||||
tabs.push({
|
||||
/**
|
||||
* Every tab with its title. Titles are read in parallel, each bounded by
|
||||
* TAB_PROBE_TIMEOUT_MS: a tab whose renderer is stuck comes back with
|
||||
* `busy: true` and an empty title instead of hanging the list (#3247).
|
||||
*/
|
||||
async getTabListWithTitles(): Promise<Array<{ id: number; url: string; title: string; active: boolean; busy: boolean }>> {
|
||||
return Promise.all([...this.pages].map(async ([id, page]) => {
|
||||
const title = await withinTabBudget(
|
||||
this.sharedTabRead(page, 'title', () => page.title().catch(() => '')), TAB_PROBE_TIMEOUT_MS);
|
||||
return {
|
||||
id,
|
||||
url: page.url(),
|
||||
title: await page.title().catch(() => ''),
|
||||
title: title ?? '',
|
||||
active: id === this.activeTabId,
|
||||
});
|
||||
}
|
||||
return tabs;
|
||||
busy: title === undefined,
|
||||
};
|
||||
}));
|
||||
}
|
||||
|
||||
// ─── Session Access ────────────────────────────────────────
|
||||
@@ -1715,7 +1789,9 @@ export class BrowserManager {
|
||||
// ─── State Save/Restore (shared by recreateContext + handoff) ─
|
||||
/**
|
||||
* Capture browser state: cookies, localStorage, sessionStorage, URLs, active tab.
|
||||
* Skips pages that fail storage reads (e.g., already closed).
|
||||
* Skips the storage of pages that fail the read (e.g., already closed).
|
||||
* A page whose renderer is stuck past TAB_STORAGE_READ_TIMEOUT_MS does not
|
||||
* hold up `state save` (#3247); see readTabStorage() for what it gets.
|
||||
*/
|
||||
async saveState(): Promise<BrowserState> {
|
||||
if (!this.context) throw new Error('Browser not launched');
|
||||
@@ -1725,13 +1801,7 @@ export class BrowserManager {
|
||||
|
||||
for (const [id, page] of this.pages) {
|
||||
const url = page.url();
|
||||
let storage = null;
|
||||
try {
|
||||
storage = await page.evaluate(() => ({
|
||||
localStorage: { ...localStorage },
|
||||
sessionStorage: { ...sessionStorage },
|
||||
}));
|
||||
} catch {}
|
||||
const storage = await this.readTabStorage(page, url);
|
||||
|
||||
// Capture load-html content so a later context recreation (viewport --scale)
|
||||
// can replay it via setTabContent. Never persisted to disk.
|
||||
@@ -1753,6 +1823,28 @@ export class BrowserManager {
|
||||
return { cookies, pages };
|
||||
}
|
||||
|
||||
/**
|
||||
* One tab's localStorage/sessionStorage for saveState(). A failed read
|
||||
* (closed page, opaque origin) is null. A read that times out returns the
|
||||
* last storage this page gave for the same origin: a periodic
|
||||
* BROWSE_PERSIST_STATE snapshot must not replace good storage on disk with
|
||||
* null just because the page is busy.
|
||||
*/
|
||||
private async readTabStorage(page: Page, url: string): Promise<TabStorage | null> {
|
||||
const origin = urlOrigin(url);
|
||||
const read = await withinTabBudget(this.sharedTabRead(page, 'storage', () => page.evaluate(() => ({
|
||||
localStorage: { ...localStorage },
|
||||
sessionStorage: { ...sessionStorage },
|
||||
})).catch(() => null)), TAB_STORAGE_READ_TIMEOUT_MS);
|
||||
if (read === undefined) {
|
||||
const last = this.lastTabStorage.get(page);
|
||||
return last?.origin === origin ? last.storage : null;
|
||||
}
|
||||
if (read) this.lastTabStorage.set(page, { origin, storage: read });
|
||||
else this.lastTabStorage.delete(page);
|
||||
return read;
|
||||
}
|
||||
|
||||
/**
|
||||
* Restore browser state into the current context: cookies, pages, storage.
|
||||
* Navigates to saved URLs, restores storage, wires page events.
|
||||
|
||||
+65
-33
@@ -215,39 +215,50 @@ function readState(): ServerState | null {
|
||||
|
||||
// isProcessAlive is imported from ./error-handling
|
||||
|
||||
/**
|
||||
* HTTP health check — definitive proof the server is alive and responsive.
|
||||
* Used in all polling loops instead of isProcessAlive() (which is slow on Windows).
|
||||
*/
|
||||
export async function isServerHealthy(port: number, timeoutMs = 2000): Promise<boolean> {
|
||||
try {
|
||||
const resp = await fetch(`http://127.0.0.1:${port}/health`, {
|
||||
signal: AbortSignal.timeout(timeoutMs),
|
||||
});
|
||||
if (!resp.ok) return false;
|
||||
const health = await resp.json() as any;
|
||||
return health.status === 'healthy';
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
/** Client timeout for one GET /health. Must stay above the daemon's active-tab
|
||||
* probe (TAB_PROBE_TIMEOUT_MS in browser-manager.ts): when the two were equal,
|
||||
* the CLI aborted just before a busy tab's reply arrived and could not tell a
|
||||
* busy tab from a hung daemon (#3247). */
|
||||
export const HEALTH_REQUEST_TIMEOUT_MS = 2000;
|
||||
|
||||
/** Best-effort tab count via GET /health (no auth, bounded). Returns null
|
||||
* when the daemon doesn't answer in time or predates the `tabs` field —
|
||||
* callers degrade to a countless phrasing, never block on this. */
|
||||
async function fetchDaemonTabCount(port: number, timeoutMs = 2000): Promise<number | null> {
|
||||
/** GET /health (no auth, bounded). Null when the daemon doesn't answer in
|
||||
* time or the reply is not a 200. */
|
||||
async function fetchDaemonHealth(port: number, timeoutMs = HEALTH_REQUEST_TIMEOUT_MS): Promise<Record<string, unknown> | null> {
|
||||
try {
|
||||
const resp = await fetch(`http://127.0.0.1:${port}/health`, {
|
||||
signal: AbortSignal.timeout(timeoutMs),
|
||||
});
|
||||
if (!resp.ok) return null;
|
||||
const health = await resp.json() as any;
|
||||
return typeof health.tabs === 'number' ? health.tabs : null;
|
||||
return await resp.json() as Record<string, unknown>;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* HTTP health check — definitive proof the server is alive and responsive.
|
||||
* Used in all polling loops instead of isProcessAlive() (which is slow on Windows).
|
||||
* `status` is daemon liveness only; a busy active tab does not fail it (#3247).
|
||||
*/
|
||||
export async function isServerHealthy(port: number, timeoutMs = HEALTH_REQUEST_TIMEOUT_MS): Promise<boolean> {
|
||||
return (await fetchDaemonHealth(port, timeoutMs))?.status === 'healthy';
|
||||
}
|
||||
|
||||
/** Best-effort tab count via GET /health. Returns null when the daemon
|
||||
* doesn't answer in time or predates the `tabs` field — callers degrade to a
|
||||
* countless phrasing, never block on this. */
|
||||
async function fetchDaemonTabCount(port: number): Promise<number | null> {
|
||||
const tabs = (await fetchDaemonHealth(port))?.tabs;
|
||||
return typeof tabs === 'number' ? tabs : null;
|
||||
}
|
||||
|
||||
/** The active tab's id when /health reports its main thread as not
|
||||
* responding (#3247), else null — including daemons that predate the field. */
|
||||
async function fetchBusyActiveTab(port: number): Promise<number | null> {
|
||||
const activeTab = (await fetchDaemonHealth(port))?.activeTab as { id?: unknown; responsive?: unknown } | null | undefined;
|
||||
return activeTab?.responsive === false && typeof activeTab.id === 'number' ? activeTab.id : null;
|
||||
}
|
||||
|
||||
// ─── Process Management ─────────────────────────────────────────
|
||||
async function killServer(pid: number): Promise<void> {
|
||||
if (!isProcessAlive(pid)) return;
|
||||
@@ -420,7 +431,7 @@ async function probeHealthWithBackoff(
|
||||
await Bun.sleep(intervalMs);
|
||||
const remainingMs = deadline - Date.now();
|
||||
if (remainingMs <= 0) return false;
|
||||
if (await isServerHealthy(port, Math.min(2000, remainingMs))) return true;
|
||||
if (await isServerHealthy(port, Math.min(HEALTH_REQUEST_TIMEOUT_MS, remainingMs))) return true;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1034,6 +1045,32 @@ export function extractTabId(args: string[]): { tabId: number | undefined; args:
|
||||
return { tabId, args: stripped };
|
||||
}
|
||||
|
||||
/** Did this fetch fail because its AbortSignal fired? Bun rejects an
|
||||
* AbortSignal.timeout() fetch with TimeoutError, not AbortError — matching
|
||||
* only AbortError left the timeout reports below unreachable. */
|
||||
export function isFetchTimeout(err: unknown): boolean {
|
||||
const name = (err as { name?: unknown } | null)?.name;
|
||||
return name === 'AbortError' || name === 'TimeoutError';
|
||||
}
|
||||
|
||||
/**
|
||||
* What to print when a command's reply did not arrive within `timeoutMs`.
|
||||
* #1781: a timeout on a heavy page usually means busy, not dead, so a live
|
||||
* daemon is never restarted here. #3247: when /health names the active tab
|
||||
* as not responding, say which tab and the commands that leave it without
|
||||
* losing the session.
|
||||
*/
|
||||
export async function commandTimeoutReport(state: { pid: number; port: number } | null, timeoutMs: number): Promise<string[]> {
|
||||
const after = `[browse] Command timed out after ${timeoutMs / 1000}s`;
|
||||
if (!state?.pid || !isProcessAlive(state.pid)) return [after];
|
||||
const busyTab = await fetchBusyActiveTab(state.port);
|
||||
if (busyTab === null) return [`${after} (server still alive — busy, not restarting). Retry, or raise load.`];
|
||||
return [
|
||||
`${after}: tab ${busyTab} is not responding (its main thread is busy).`,
|
||||
`[browse] The daemon, its other tabs and cookies still work: switch with 'browse tab <id>', close this one with 'browse closetab ${busyTab}', or retry once the page settles.`,
|
||||
];
|
||||
}
|
||||
|
||||
// ─── Command Dispatch ──────────────────────────────────────────
|
||||
export async function sendCommand(state: ServerState, command: string, args: string[], retries = 0): Promise<void> {
|
||||
// Precedence: CLI --tab-id flag > BROWSE_TAB env var.
|
||||
@@ -1085,19 +1122,14 @@ export async function sendCommand(state: ServerState, command: string, args: str
|
||||
process.exit(1);
|
||||
}
|
||||
} catch (err: any) {
|
||||
if (command === 'cookie-import-browser' && (['AbortError', 'TimeoutError'].includes(err.name)
|
||||
if (command === 'cookie-import-browser' && (isFetchTimeout(err)
|
||||
|| ['ECONNREFUSED', 'ECONNRESET'].includes(err.code) || err.message?.includes('fetch failed'))) {
|
||||
throw new Error('Cookie import response was lost or timed out. It may have partially completed; inspect the destination before retrying manually. The command was not replayed.');
|
||||
}
|
||||
if (err.name === 'AbortError') {
|
||||
// #1781: a 30s timeout on a heavy page usually means busy, not dead.
|
||||
// Don't kill a live server (that's what triggered the crash-loop) — report
|
||||
// and exit so the user can retry rather than losing their (headed) window.
|
||||
const ts = readState();
|
||||
const alive = ts?.pid ? isProcessAlive(ts.pid) : false;
|
||||
console.error(alive
|
||||
? '[browse] Command timed out after 30s (server still alive — busy, not restarting). Retry, or raise load.'
|
||||
: '[browse] Command timed out after 30s');
|
||||
if (isFetchTimeout(err)) {
|
||||
// Report and exit so the user can retry rather than losing their
|
||||
// (headed) window — killing a live server triggered the crash-loop.
|
||||
for (const line of await commandTimeoutReport(readState(), timeoutMs)) console.error(line);
|
||||
process.exit(1);
|
||||
}
|
||||
// Connection error — server may have crashed, OR may just be busy.
|
||||
|
||||
@@ -265,7 +265,7 @@ export async function handleMetaCommand(
|
||||
case 'tabs': {
|
||||
const tabs = await bm.getTabListWithTitles();
|
||||
return tabs.map(t =>
|
||||
`${t.active ? '→ ' : ' '}[${t.id}] ${t.title || '(untitled)'} — ${t.url}`
|
||||
`${t.active ? '→ ' : ' '}[${t.id}] ${t.busy ? '(not responding)' : t.title || '(untitled)'} — ${t.url}`
|
||||
).join('\n');
|
||||
}
|
||||
|
||||
@@ -404,12 +404,17 @@ export async function handleMetaCommand(
|
||||
case 'status': {
|
||||
const tabs = bm.getTabCount();
|
||||
const mode = bm.getConnectionMode();
|
||||
const activeTab = await bm.probeActiveTab();
|
||||
return [
|
||||
`Status: healthy`,
|
||||
`Mode: ${mode}`,
|
||||
`URL: ${bm.getCurrentUrl()}`,
|
||||
`Tabs: ${tabs}`,
|
||||
`PID: ${process.pid}`,
|
||||
// #3247: name the busy tab and the ways out that keep the session.
|
||||
...(activeTab && !activeTab.responsive
|
||||
? [`Active tab ${activeTab.id}: not responding. Switch with: browse tab <id>, or close it: browse closetab ${activeTab.id}`]
|
||||
: []),
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
|
||||
@@ -77,13 +77,19 @@ export const coreRoutes: RouteEntry[] = [
|
||||
// Health check — no auth required, does NOT reset idle timer. NEVER carries
|
||||
// a token in any mode: token bootstrap is POST /extension-token and shell
|
||||
// auth is POST /pty-session. Liveness/status only.
|
||||
//
|
||||
// `status` is daemon liveness (Chromium connected); the CLI gates every
|
||||
// command on it. `activeTab.responsive` is the active tab's main thread,
|
||||
// probed for at most TAB_PROBE_TIMEOUT_MS and reported beside it: one busy
|
||||
// page must not read as a dead daemon (#3247).
|
||||
{
|
||||
method: '*', path: '/health', auth: 'none', surfaces: ['local'],
|
||||
handler: async (_req, _r, ctx) => {
|
||||
const { browserManager } = ctx;
|
||||
const healthy = await browserManager.isHealthy();
|
||||
const healthy = browserManager.isHealthy();
|
||||
return json({
|
||||
status: healthy ? 'healthy' : 'unhealthy',
|
||||
activeTab: healthy ? await browserManager.probeActiveTab() : null,
|
||||
mode: browserManager.getConnectionMode(),
|
||||
uptime: Math.floor((Date.now() - ctx.startTime) / 1000),
|
||||
tabs: browserManager.getTabCount(),
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
/**
|
||||
* #3247: one busy tab must not make the daemon look dead. A renderer stuck in
|
||||
* a long script never answers page.evaluate() or page.title(). /health folded
|
||||
* that into its `status`, so the CLI refused every command (status, tabs,
|
||||
* tab, newtab, stop) with "Daemon busy" until --force-restart threw the
|
||||
* session away, and `tabs` and `state save` hung on the tab as well.
|
||||
*
|
||||
* A real Chromium tab is held in a busy loop while a second tab stays idle;
|
||||
* each surface must answer within its bound and name the busy tab, without
|
||||
* piling up reads on the stuck renderer or dropping its persisted storage.
|
||||
*/
|
||||
import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import type { Page } from 'playwright';
|
||||
import { BrowserManager, TAB_PROBE_TIMEOUT_MS } from '../src/browser-manager';
|
||||
import { HEALTH_REQUEST_TIMEOUT_MS } from '../src/cli';
|
||||
import { handleMetaCommand } from '../src/meta-commands';
|
||||
import { persistSessionState } from '../src/session-persist';
|
||||
import { callRoute, stubRouteContext } from './route-test-harness';
|
||||
import { startTestServer } from './test-server';
|
||||
|
||||
// Outlasts every check below; closing the busy tab at the end stops it sooner.
|
||||
const BUSY_MS = 30_000;
|
||||
|
||||
let testServer: ReturnType<typeof startTestServer>;
|
||||
let bm: BrowserManager;
|
||||
let busyId = 0;
|
||||
let idleId = 0;
|
||||
let busyPage: Page;
|
||||
let profileDir = '';
|
||||
const originalProfile = process.env.CHROMIUM_PROFILE;
|
||||
|
||||
beforeAll(async () => {
|
||||
// Per-file profile, set in a hook (see commands.test.ts for why).
|
||||
profileDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-busy-tab-profile-'));
|
||||
process.env.CHROMIUM_PROFILE = profileDir;
|
||||
testServer = startTestServer(0);
|
||||
bm = new BrowserManager();
|
||||
await bm.launch();
|
||||
busyId = bm.getActiveTabId();
|
||||
await bm.getPage().goto(`${testServer.url}/basic.html`);
|
||||
idleId = await bm.newTab(`${testServer.url}/basic.html`);
|
||||
bm.switchTab(busyId, { bringToFront: false });
|
||||
busyPage = bm.getPage();
|
||||
// The good snapshot a BROWSE_PERSIST_STATE tick takes before the page hangs.
|
||||
await busyPage.evaluate(() => localStorage.setItem('probe', 'kept'));
|
||||
await bm.saveState();
|
||||
await busyPage.evaluate((ms) => {
|
||||
setTimeout(() => { const end = Date.now() + ms; while (Date.now() < end) { /* busy */ } }, 50);
|
||||
}, BUSY_MS);
|
||||
await Bun.sleep(300); // let the loop start
|
||||
}, 30_000);
|
||||
|
||||
afterAll(async () => {
|
||||
try { testServer.server.stop(true); } catch {}
|
||||
try { await Promise.race([bm?.close(), Bun.sleep(3000)]); } catch {}
|
||||
if (originalProfile === undefined) delete process.env.CHROMIUM_PROFILE;
|
||||
else process.env.CHROMIUM_PROFILE = originalProfile;
|
||||
if (profileDir) fs.rmSync(profileDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('busy active tab (#3247)', () => {
|
||||
test('/health says the daemon is alive and names the busy tab before the CLI gives up', async () => {
|
||||
// Equal budgets were the bug: the CLI aborted just before the reply landed.
|
||||
expect(TAB_PROBE_TIMEOUT_MS).toBeLessThan(HEALTH_REQUEST_TIMEOUT_MS);
|
||||
const ctx = stubRouteContext({
|
||||
browserManager: bm,
|
||||
terminal: { readPort: () => null, grantToken: async () => true, restartSession: async () => true },
|
||||
});
|
||||
const started = Date.now();
|
||||
const body = await (await callRoute('GET', '/health', ctx)).json();
|
||||
expect(Date.now() - started).toBeLessThan(HEALTH_REQUEST_TIMEOUT_MS);
|
||||
expect(body.status).toBe('healthy');
|
||||
expect(body.activeTab).toEqual({ id: busyId, responsive: false });
|
||||
});
|
||||
|
||||
test('tabs lists every tab without waiting on the busy one', async () => {
|
||||
const started = Date.now();
|
||||
const out = await handleMetaCommand('tabs', [], bm, async () => {});
|
||||
expect(Date.now() - started).toBeLessThan(HEALTH_REQUEST_TIMEOUT_MS);
|
||||
expect(out).toContain(`[${busyId}] (not responding)`);
|
||||
expect(out).toContain(`[${idleId}] Test Page - Basic`);
|
||||
});
|
||||
|
||||
test('status stays healthy and names the busy tab with a way out', async () => {
|
||||
const out = await handleMetaCommand('status', [], bm, async () => {});
|
||||
expect(out).toContain('Status: healthy');
|
||||
expect(out).toContain(`Active tab ${busyId}`);
|
||||
expect(out).toContain(`closetab ${busyId}`);
|
||||
});
|
||||
|
||||
test('a stuck tab gets one pending probe, not one per poll', async () => {
|
||||
let evaluates = 0;
|
||||
const evaluate = busyPage.evaluate.bind(busyPage);
|
||||
Object.assign(busyPage, { evaluate: (...args: Parameters<Page['evaluate']>) => { evaluates++; return evaluate(...args); } });
|
||||
try {
|
||||
for (let i = 0; i < 3; i++) expect(await bm.probeActiveTab()).toEqual({ id: busyId, responsive: false });
|
||||
} finally {
|
||||
delete (busyPage as { evaluate?: unknown }).evaluate; // back to the prototype method
|
||||
}
|
||||
// The /health check above may already hold the pending probe.
|
||||
expect(evaluates).toBeLessThanOrEqual(1);
|
||||
}, 10_000);
|
||||
|
||||
test('a persisted snapshot taken while the tab is busy keeps its last good storage', async () => {
|
||||
const file = path.join(profileDir, 'session-state.json');
|
||||
await persistSessionState(bm, file);
|
||||
const saved = JSON.parse(fs.readFileSync(file, 'utf8'));
|
||||
expect(saved.pages).toHaveLength(2);
|
||||
expect(saved.pages.find((p: { isActive: boolean }) => p.isActive).storage.localStorage).toEqual({ probe: 'kept' });
|
||||
expect(saved.pages.find((p: { isActive: boolean }) => !p.isActive).storage).not.toBeNull();
|
||||
}, 15_000);
|
||||
|
||||
test('the idle tab still answers', async () => {
|
||||
bm.switchTab(idleId, { bringToFront: false });
|
||||
try {
|
||||
expect(await bm.probeActiveTab()).toEqual({ id: idleId, responsive: true });
|
||||
expect(await bm.getPage().title()).toBe('Test Page - Basic');
|
||||
} finally {
|
||||
bm.switchTab(busyId, { bringToFront: false });
|
||||
}
|
||||
});
|
||||
|
||||
test('closetab closes the busy tab and the session carries on', async () => {
|
||||
const out = await handleMetaCommand('closetab', [String(busyId)], bm, async () => {});
|
||||
expect(out).toContain(`Closed tab ${busyId}`);
|
||||
expect(bm.getTabCount()).toBe(1);
|
||||
expect(await bm.probeActiveTab()).toEqual({ id: idleId, responsive: true });
|
||||
}, 15_000);
|
||||
});
|
||||
@@ -0,0 +1,66 @@
|
||||
/**
|
||||
* What the CLI prints when a command's reply never arrives (#1781, #3247).
|
||||
*
|
||||
* Bun rejects an AbortSignal.timeout() fetch with TimeoutError, so the old
|
||||
* AbortError-only check skipped the busy report and printed "The operation
|
||||
* timed out." instead. When the daemon's /health names its active tab as not
|
||||
* responding, the report names that tab and the commands that leave it
|
||||
* without a restart; a dead daemon gets the plain line.
|
||||
*/
|
||||
import { afterEach, describe, expect, test } from 'bun:test';
|
||||
import { commandTimeoutReport, isFetchTimeout } from '../src/cli';
|
||||
|
||||
let servers: Array<ReturnType<typeof Bun.serve>> = [];
|
||||
afterEach(() => { for (const s of servers) s.stop(true); servers = []; });
|
||||
|
||||
/** A fake daemon whose /health answers `body`. */
|
||||
function fakeHealth(body: unknown): number {
|
||||
const srv = Bun.serve({ port: 0, hostname: '127.0.0.1', fetch: () => Response.json(body) });
|
||||
servers.push(srv);
|
||||
if (!srv.port) throw new Error('fake /health: no port');
|
||||
return srv.port;
|
||||
}
|
||||
|
||||
/** A pid that no longer runs: a child that has already exited. */
|
||||
function deadPid(): number {
|
||||
return Bun.spawnSync([process.execPath, '-e', '']).pid;
|
||||
}
|
||||
|
||||
describe('isFetchTimeout', () => {
|
||||
test('a real fetch whose AbortSignal.timeout fires counts as a timeout', async () => {
|
||||
const silent = Bun.serve({ port: 0, hostname: '127.0.0.1', fetch: () => new Promise<Response>(() => {}) });
|
||||
servers.push(silent);
|
||||
const err = await fetch(`http://127.0.0.1:${silent.port}/`, { signal: AbortSignal.timeout(50) })
|
||||
.then(() => null, (e: unknown) => e);
|
||||
expect(isFetchTimeout(err)).toBe(true);
|
||||
expect(isFetchTimeout(new TypeError('fetch failed'))).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('commandTimeoutReport', () => {
|
||||
test('names the busy tab and the way out, never a restart', async () => {
|
||||
const port = fakeHealth({ status: 'healthy', activeTab: { id: 7, responsive: false } });
|
||||
const out = (await commandTimeoutReport({ pid: process.pid, port }, 30_000)).join('\n');
|
||||
expect(out).toContain('tab 7');
|
||||
expect(out).toContain('closetab 7');
|
||||
expect(out).not.toContain('--force-restart');
|
||||
});
|
||||
|
||||
test('a live daemon with a responsive or unreported tab gets the busy-not-restarting line', async () => {
|
||||
for (const body of [
|
||||
{ status: 'healthy', activeTab: { id: 1, responsive: true } },
|
||||
{ status: 'healthy' }, // daemon from before activeTab existed
|
||||
]) {
|
||||
const out = (await commandTimeoutReport({ pid: process.pid, port: fakeHealth(body) }, 30_000)).join('\n');
|
||||
expect(out).toContain('still alive');
|
||||
expect(out).not.toContain('closetab');
|
||||
}
|
||||
});
|
||||
|
||||
test('a dead daemon gets only the timeout line', async () => {
|
||||
const port = fakeHealth({ status: 'healthy', activeTab: { id: 7, responsive: false } });
|
||||
const out = await commandTimeoutReport({ pid: deadPid(), port }, 30_000);
|
||||
expect(out).toHaveLength(1);
|
||||
expect(out[0]).toContain('timed out after 30s');
|
||||
});
|
||||
});
|
||||
@@ -94,13 +94,16 @@ describe('/health does NOT surface ptyToken', () => {
|
||||
test('/health response body does not include ptyToken', async () => {
|
||||
// It's allowed to include `terminalPort` (a port number, not auth).
|
||||
const ctx = stubRouteContext({
|
||||
browserManager: { isHealthy: async () => true, getConnectionMode: () => 'launched', getTabCount: () => 1 } as any,
|
||||
browserManager: {
|
||||
isHealthy: () => true, probeActiveTab: async () => ({ id: 1, responsive: true }),
|
||||
getConnectionMode: () => 'launched', getTabCount: () => 1,
|
||||
} as any,
|
||||
terminal: { readPort: () => 4242, grantToken: async () => true, restartSession: async () => true },
|
||||
});
|
||||
const text = await (await callRoute('GET', '/health', ctx)).text();
|
||||
const body = JSON.parse(text);
|
||||
expect(body.terminalPort).toBe(4242);
|
||||
expect(Object.keys(body).sort()).toEqual(['mode', 'status', 'tabs', 'terminalPort', 'uptime']);
|
||||
expect(Object.keys(body).sort()).toEqual(['activeTab', 'mode', 'status', 'tabs', 'terminalPort', 'uptime']);
|
||||
expect(text).not.toContain('ptyToken');
|
||||
expect(text).not.toContain('gstack_pty');
|
||||
});
|
||||
|
||||
@@ -5,16 +5,16 @@
|
||||
"": {
|
||||
"name": "gstack",
|
||||
"dependencies": {
|
||||
"@huggingface/transformers": "^4.3.0",
|
||||
"@huggingface/transformers": "^4.3.1",
|
||||
"@ngrok/ngrok": "^1.7.0",
|
||||
"cross-spawn": "^7.0.6",
|
||||
"diff": "^9.0.0",
|
||||
"html-to-docx": "1.8.0",
|
||||
"html-to-vdom": "0.7.0",
|
||||
"marked": "^18.0.11",
|
||||
"marked": "^18.1.1",
|
||||
"playwright": "^1.62.1",
|
||||
"sharp": "^0.35.4",
|
||||
"socks": "^2.8.9",
|
||||
"socks": "^2.8.10",
|
||||
},
|
||||
"devDependencies": {
|
||||
"@anthropic-ai/claude-agent-sdk": "0.2.141",
|
||||
@@ -71,7 +71,7 @@
|
||||
|
||||
"@huggingface/tokenizers": ["@huggingface/tokenizers@0.2.0", "", {}, "sha512-LidMHe1FpcSYH4vcSjXooda34pC0M8m1gmDOv7SQi6Iv+ib1zX7J3sHBCC9/3FPCkn2k1b55FneVqMRxrr99pg=="],
|
||||
|
||||
"@huggingface/transformers": ["@huggingface/transformers@4.3.0", "", { "dependencies": { "@huggingface/jinja": "^0.5.10", "@huggingface/tokenizers": "^0.2.0", "onnxruntime-node": "1.30.0", "onnxruntime-web": "1.31.0-dev.20260914-8d85527a0", "sharp": "^0.35.4" } }, "sha512-fL1A/WUZwouPrOlYxU5dzIwD2T5J781JiB2jDR8bFe5DwCj0Gfudq+NEXCMno49kQgajHA7xQkrRLJlqG1veEA=="],
|
||||
"@huggingface/transformers": ["@huggingface/transformers@4.3.1", "", { "dependencies": { "@huggingface/jinja": "^0.5.10", "@huggingface/tokenizers": "^0.2.0", "onnxruntime-node": "1.30.0", "onnxruntime-web": "1.31.0-dev.20260914-8d85527a0", "sharp": "^0.35.4" } }, "sha512-5+voSeUCFK6PCKlY15dPhNOKq3/zOTxDN7oC+Wy9dFb1QZ0TSQomZRRy7URRK8YRuUoE9fGMQTTPC5d/pWBJhA=="],
|
||||
|
||||
"@img/colour": ["@img/colour@1.1.0", "", {}, "sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ=="],
|
||||
|
||||
@@ -413,7 +413,7 @@
|
||||
|
||||
"long": ["long@5.3.2", "", {}, "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA=="],
|
||||
|
||||
"marked": ["marked@18.0.11", "", { "bin": { "marked": "bin/marked.js" } }, "sha512-HnslJfsZkRPBDJRHvVtAaWlZHEpSu7u8LgQuJCELjRKuWR+hpq4A7sLq3p8HaI9ypVoXDXxV34CsQJEe1+J5Aw=="],
|
||||
"marked": ["marked@18.1.1", "", { "bin": { "marked": "bin/marked.js" } }, "sha512-zEGgrtUG8crk+TrMacJSVABU4DYDDkS8uqgU3ohrn/2fW9C2AEfUrO3m6IhpIpQpl7YcVpiXXC9cSMWDANT3dg=="],
|
||||
|
||||
"matcher": ["matcher@4.0.0", "", { "dependencies": { "escape-string-regexp": "^4.0.0" } }, "sha512-S6x5wmcDmsDRRU/c2dkccDwQPXoFczc5+HpQ2lON8pnvHlnvHAHj5WlLVvw6n6vNyHuVugYrFohYxbS+pvFpKQ=="],
|
||||
|
||||
@@ -529,7 +529,7 @@
|
||||
|
||||
"smart-buffer": ["smart-buffer@4.2.0", "", {}, "sha512-94hK0Hh8rPqQl2xXc3HsaBoOXKV20MToPkcXvwbISWLEs+64sBq5kFgn2kJDHb1Pry9yrP0dxrCI9RRci7RXKg=="],
|
||||
|
||||
"socks": ["socks@2.8.9", "", { "dependencies": { "ip-address": "^10.1.1", "smart-buffer": "^4.2.0" } }, "sha512-LJhUYUvItdQ0LkJTmPeaEObWXAqFyfmP85x0tch/ez9cahmhlBBLbIqDFnvBnUJGagb0JbIQrkBs1wJ+yRYpEw=="],
|
||||
"socks": ["socks@2.8.10", "", { "dependencies": { "ip-address": "^10.1.1", "smart-buffer": "^4.2.0" } }, "sha512-e0VyvkVTwVYViNovRkZ9aodhxVlyoMn7eJhVUPxZ+eK9P/7CBkxvvsBOHqFPEH416726W8tLXXXjKwqgTErrCQ=="],
|
||||
|
||||
"statuses": ["statuses@2.0.2", "", {}, "sha512-DvEy55V3DB7uknRo+4iOGT5fP1slR8wQohVdknigZPMpMstaKJQWhwiYBACJE3Ul2pTnATihhBYnRhZQHGBiRw=="],
|
||||
|
||||
|
||||
@@ -500,6 +500,12 @@ has written to). Prefixed lines are never candidates for "latest":
|
||||
- Otherwise: load the **first plain path line returned by Step 1 above**, which
|
||||
is the newest `YYYYMMDD-HHMMSS` checkpoint of this project for the current
|
||||
branch, or, if the current branch has none, the newest across all branches.
|
||||
- **No useful branch discriminator.** When `CURRENT_BRANCH` is empty, `HEAD`, or
|
||||
otherwise cannot discriminate (cwd is not a git repo, or branch is unknown), do
|
||||
not silently load newest if the recent candidates (e.g. last 24 hours, or the
|
||||
three newest) carry **different titles**. Show those titles with timestamps via
|
||||
AskUserQuestion, mark newest as the recommended option, and wait. Only auto-load
|
||||
newest when every recent candidate shares one title (same topic).
|
||||
- **Do not silently take an older checkpoint.** If Step 1 printed `NEWER_TASK`,
|
||||
or a `BELOW_CWD`/`POINTER` file is newer than the first plain path line (or
|
||||
there is none), show both (title, branch, saved time, and `project_root` for a
|
||||
@@ -525,6 +531,10 @@ checkpoint(s) from another project; only this project's are listed." If it print
|
||||
order, and drop nothing. Put an item under **Verify first** when it:
|
||||
- ends in `(path assumed)` or `(code read)`;
|
||||
- ends in `(path run)` but its text reports a failure;
|
||||
- ends in a parenthesised trailing marker that is **not** one of the five English
|
||||
literals `(path run)`, `(path read)`, `(path assumed)`, `(target state checked)`,
|
||||
`(code read)` — treat a translated or unknown marker as `(path assumed)` (→ Verify
|
||||
first), never as unmarked Next steps;
|
||||
- has no marker and is a writing step (migration, sync, insert, import, a dialog that
|
||||
writes) or names a concrete path (a runnable command, CLI flag or switch, config key
|
||||
or value, or file or directory path).
|
||||
|
||||
@@ -176,6 +176,12 @@ has written to). Prefixed lines are never candidates for "latest":
|
||||
- Otherwise: load the **first plain path line returned by Step 1 above**, which
|
||||
is the newest `YYYYMMDD-HHMMSS` checkpoint of this project for the current
|
||||
branch, or, if the current branch has none, the newest across all branches.
|
||||
- **No useful branch discriminator.** When `CURRENT_BRANCH` is empty, `HEAD`, or
|
||||
otherwise cannot discriminate (cwd is not a git repo, or branch is unknown), do
|
||||
not silently load newest if the recent candidates (e.g. last 24 hours, or the
|
||||
three newest) carry **different titles**. Show those titles with timestamps via
|
||||
AskUserQuestion, mark newest as the recommended option, and wait. Only auto-load
|
||||
newest when every recent candidate shares one title (same topic).
|
||||
- **Do not silently take an older checkpoint.** If Step 1 printed `NEWER_TASK`,
|
||||
or a `BELOW_CWD`/`POINTER` file is newer than the first plain path line (or
|
||||
there is none), show both (title, branch, saved time, and `project_root` for a
|
||||
@@ -201,6 +207,10 @@ checkpoint(s) from another project; only this project's are listed." If it print
|
||||
order, and drop nothing. Put an item under **Verify first** when it:
|
||||
- ends in `(path assumed)` or `(code read)`;
|
||||
- ends in `(path run)` but its text reports a failure;
|
||||
- ends in a parenthesised trailing marker that is **not** one of the five English
|
||||
literals `(path run)`, `(path read)`, `(path assumed)`, `(target state checked)`,
|
||||
`(code read)` — treat a translated or unknown marker as `(path assumed)` (→ Verify
|
||||
first), never as unmarked Next steps;
|
||||
- has no marker and is a writing step (migration, sync, insert, import, a dialog that
|
||||
writes) or names a concrete path (a runnable command, CLI flag or switch, config key
|
||||
or value, or file or directory path).
|
||||
|
||||
@@ -427,6 +427,21 @@ depends on the target's state, not on the code that writes. Examples:
|
||||
`Open. Run the suite with CONFIG=staging. (path run) exit 0` and
|
||||
`Open. Switch B reads config Y. (path assumed)`.
|
||||
|
||||
**Markers stay English literals.** `/context-restore` sorts on the exact strings
|
||||
`(path run)`, `(path read)`, `(path assumed)`, `(target state checked)`, and
|
||||
`(code read)`. Never translate them, even when the rest of the checkpoint is in
|
||||
another language. After drafting Remaining Work, scan each item that should have
|
||||
a marker: if it ends in a parenthesised token that is not one of those five, or
|
||||
has no marker when the table requires one, rewrite that item before writing the
|
||||
file (preferred), or add a Remaining Work item that names the bad line so restore
|
||||
puts it under Verify first.
|
||||
|
||||
**Decisions Made and file claims.** Provenance above covers Remaining Work only.
|
||||
Any statement in Decisions Made about the *contents* of a concrete file (item
|
||||
counts, line counts, section names) must be backed by a `grep`/`wc`/`Read` in this
|
||||
turn, or rewritten with an honest `(path assumed)` note in Remaining Work that
|
||||
restore will verify first. Do not record an unchecked file claim as fact.
|
||||
|
||||
If the user provided a title, use it. Otherwise, infer a concise title (3-6 words)
|
||||
from the work being done.
|
||||
|
||||
|
||||
@@ -103,6 +103,21 @@ depends on the target's state, not on the code that writes. Examples:
|
||||
`Open. Run the suite with CONFIG=staging. (path run) exit 0` and
|
||||
`Open. Switch B reads config Y. (path assumed)`.
|
||||
|
||||
**Markers stay English literals.** `/context-restore` sorts on the exact strings
|
||||
`(path run)`, `(path read)`, `(path assumed)`, `(target state checked)`, and
|
||||
`(code read)`. Never translate them, even when the rest of the checkpoint is in
|
||||
another language. After drafting Remaining Work, scan each item that should have
|
||||
a marker: if it ends in a parenthesised token that is not one of those five, or
|
||||
has no marker when the table requires one, rewrite that item before writing the
|
||||
file (preferred), or add a Remaining Work item that names the bad line so restore
|
||||
puts it under Verify first.
|
||||
|
||||
**Decisions Made and file claims.** Provenance above covers Remaining Work only.
|
||||
Any statement in Decisions Made about the *contents* of a concrete file (item
|
||||
counts, line counts, section names) must be backed by a `grep`/`wc`/`Read` in this
|
||||
turn, or rewritten with an honest `(path assumed)` note in Remaining Work that
|
||||
restore will verify first. Do not record an unchecked file claim as fact.
|
||||
|
||||
If the user provided a title, use it. Otherwise, infer a concise title (3-6 words)
|
||||
from the work being done.
|
||||
|
||||
|
||||
@@ -14,8 +14,10 @@ Selection IDs for the six retained paid cases stay unchanged.
|
||||
## Blocking checks
|
||||
|
||||
Both arms use the real Claude Code preset; ON appends the resolved overlay and
|
||||
OFF does not. Each retained fixture runs ten samples per arm, separately for
|
||||
`claude-opus-4-7` and `claude-sonnet-4-6`. Each model keeps its own results.
|
||||
OFF does not. Each retained fixture runs ten samples per arm on its pinned
|
||||
model: `claude-opus-5-5`, plus `claude-sonnet-5-5` for the dedicated-tools `-sonnet` case.
|
||||
Each model keeps its own results. The `opus-4-7-*` cases keep their overlay
|
||||
(`model-overlays/opus-4-7.md`) but run on Opus 5.5 since GSTA-49.
|
||||
|
||||
Every planned sample in both arms must execute successfully, produce exactly
|
||||
one successful native terminal result without an error flag, preserve workspace
|
||||
@@ -38,7 +40,7 @@ are not reinterpreted.
|
||||
| `opus-4-7-effort-match-trivial` | Read-only workspace; native final JSON contains the exact version string | Correct answer in every ON sample | Native `usage.output_tokens_details.thinking_tokens` |
|
||||
| `opus-4-7-literal-interpretation` | Only `src/auth.ts`, `src/billing.ts`, and `src/notifications.ts` may change; public tests stay frozen; independent behavior checks run outside the writable fixture | All three target behaviors pass in every ON sample | Correct target behaviors, 0..3 |
|
||||
|
||||
Each family also has a `-sonnet` case. Output validation checks the native final
|
||||
The dedicated-tools family also has `-sonnet` and `-opus-5-5` cases. Output validation checks the native final
|
||||
answer, so a correct word mentioned earlier cannot conceal a wrong final answer.
|
||||
The read-only prompts explicitly request the JSON shape. These exact-answer
|
||||
prompts are new in v2: compare ON/OFF within the same contract and fixture, not
|
||||
|
||||
@@ -81,7 +81,9 @@ Response: (plain text result of the command)
|
||||
```
|
||||
|
||||
#### GET /health
|
||||
Server status. No auth required. Returns status, tabs, mode, uptime.
|
||||
Server status. No auth required. Returns status, activeTab, tabs, mode, uptime.
|
||||
`status` is daemon liveness; `activeTab` (`{ id, responsive }`, or null with no
|
||||
page) says whether the active tab's main thread answered within 1s.
|
||||
Never carries a token — extension token bootstrap is `POST /extension-token`
|
||||
(local listener only, validates the pinned `chrome-extension://` Origin and a
|
||||
loopback Host; 403 otherwise). Not reachable over the tunnel (404).
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
# How to bring up your first production deploy
|
||||
|
||||
Use this when `/setup-deploy` stops with "Nothing is deployed yet". It walks through a first production deploy, so you finish with a live URL that `/setup-deploy` can record and `/land-and-deploy` can verify. It names no hosting provider or deploy tool; use whichever fits your project.
|
||||
|
||||
`/setup-deploy` writes nothing at that stop. It configures how `/land-and-deploy` checks a deploy, so it needs a production deploy that already exists.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- The app builds and starts locally with the commands production will run.
|
||||
- The code is in its GitHub repository. `/setup-deploy` and `/land-and-deploy` read it through the GitHub CLI, so `gh auth status` should pass.
|
||||
- An account with the hosting provider you choose, with billing set up if it needs it.
|
||||
- Optional: a domain you control, if users won't use the host's default address.
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Choose where the app runs
|
||||
|
||||
Pick a host for the kind of app you have: static files, a long-running server or container, or functions. List what the app needs at runtime: the language runtime and its version, a database or other services, background jobs, and any disk that has to survive a restart.
|
||||
|
||||
Then choose how production deploys start. `/land-and-deploy` only watches a trigger that already exists; it never creates one. When `/setup-deploy` asks how deploys are triggered, the choices are:
|
||||
|
||||
- automatically on push to the default branch
|
||||
- a CI workflow in the repository
|
||||
- a deploy script or CLI command
|
||||
- manually
|
||||
|
||||
A trigger that runs on merge to the default branch keeps `/land-and-deploy` hands-off. With a command trigger, nothing deploys until the command runs: `/land-and-deploy` asks you to run it after the merge, or runs it only with your explicit approval.
|
||||
|
||||
### 2. Create the services the app depends on
|
||||
|
||||
Create production instances of every service the app uses, such as its database, cache, file storage, email or payments, separate from your development ones. Run the database schema setup or migrations against production before the app serves traffic.
|
||||
|
||||
### 3. Set environment variables and secrets
|
||||
|
||||
1. List every variable the app reads, at build time and at runtime.
|
||||
2. Set each one in the host's environment or secrets settings.
|
||||
3. Keep secret values out of the repository. Commit only an example file with placeholder values, such as `.env.example`.
|
||||
4. Enter secret values yourself. Don't paste them into a chat with your agent.
|
||||
|
||||
Some hosts inject variables only at runtime. A value the build needs, such as a public client-side key, has to be available when the build runs.
|
||||
|
||||
### 4. Run the first deploy
|
||||
|
||||
Start a deploy with the trigger you chose: push to the default branch, run the workflow, or run the command. Watch the build and release logs until they finish, and note the address the host assigns. If the build fails, fix it and deploy again before you go on.
|
||||
|
||||
### 5. Point your domain at it (optional)
|
||||
|
||||
1. At your DNS provider, add the records the host asks for: usually a CNAME record for a subdomain, or A and AAAA records for the root domain.
|
||||
2. Wait until the name resolves:
|
||||
|
||||
```bash
|
||||
dig +short app.example.com
|
||||
```
|
||||
|
||||
3. Check that HTTPS works for the domain with a valid certificate:
|
||||
|
||||
```bash
|
||||
curl -sI https://app.example.com | head -1
|
||||
```
|
||||
|
||||
Decide which address is the production URL, your domain or the host's default address. That is the URL you give `/setup-deploy`.
|
||||
|
||||
### 6. Check that it works
|
||||
|
||||
1. Open the production URL in a browser and walk through one real flow, such as signing in or loading data.
|
||||
2. If the app has a health endpoint, check that it returns a 2xx status:
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w "%{http_code}\n" https://app.example.com/health
|
||||
```
|
||||
|
||||
A useful health endpoint exercises the app, for example by reaching its database, instead of returning a fixed page.
|
||||
3. Make sure you can tell which revision is live: a status command that prints the deployed commit, or a page or endpoint that shows it. `/land-and-deploy` treats a page that loads as site health, not as proof that the merged commit is live.
|
||||
4. Check the host's logs for errors after your first requests.
|
||||
|
||||
### 7. Hand the deploy back to gstack
|
||||
|
||||
Run `/setup-deploy` again. If the project now has a config file for a platform it recognizes, it starts from that platform's settings; otherwise it asks. Either way, have these answers ready:
|
||||
|
||||
- how deploys are triggered (step 1)
|
||||
- the production URL (step 5)
|
||||
- how to check that a deploy succeeded: a health check URL, a status command, the CI workflow's status, or just checking that the URL loads
|
||||
- the merge method your repository allows (squash, merge or rebase), and any command to run before merging
|
||||
|
||||
It writes the answers to the `## Deploy Configuration` section of the project's CLAUDE.md, then tries the health check and the status command once. From then on, `/land-and-deploy` uses that section to merge your PR, watch the deploy and check production. Its first run walks you through a dry run of the pipeline before anything irreversible happens.
|
||||
|
||||
## Verification
|
||||
|
||||
- `/setup-deploy` ends with `DEPLOY CONFIGURATION — COMPLETE`.
|
||||
- The project's CLAUDE.md has the section, with your production URL:
|
||||
|
||||
```bash
|
||||
grep -A 20 "## Deploy Configuration" CLAUDE.md
|
||||
```
|
||||
|
||||
- Its health check did not print `UNREACHABLE`, and its status command did not print `COMMAND_FAILED`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**`/setup-deploy` stops at "Nothing is deployed yet" again.**
|
||||
That answer means there is no production deploy to record. Once the app is live, pick the answer that says how it deploys: on push, through a CI workflow, with a script or command, or by hand.
|
||||
|
||||
**The health check fails or prints `UNREACHABLE`.**
|
||||
Open the URL yourself. A 404 means the health path doesn't exist, so give `/setup-deploy` a path that returns 2xx, or the plain production URL. `/setup-deploy` keeps the configuration even when this check fails, so you can fix the URL later by running it again or by editing the section in CLAUDE.md.
|
||||
|
||||
**The app works on the host's address but not on your domain.**
|
||||
Check the DNS records (`dig +short <domain>`) and the certificate (`curl -sI https://<domain>`). New records can take a while to resolve everywhere.
|
||||
|
||||
**The build works locally but fails on the host.**
|
||||
Compare the host's build environment with yours. A missing build-time variable or a different runtime version is the usual cause.
|
||||
|
||||
**You deploy by running a command by hand.**
|
||||
Record it as the deploy trigger. After each merge, `/land-and-deploy` tells you production hasn't moved yet and asks you to run it.
|
||||
|
||||
## Related
|
||||
|
||||
- [`/setup-deploy`](./skills.md#setup-deploy) and [`/land-and-deploy`](./skills.md#land-and-deploy): what each skill does
|
||||
- [`/canary`](./skills.md#canary): post-deploy monitoring once production is live
|
||||
- Reference: [`setup-deploy/SKILL.md`](../setup-deploy/SKILL.md), [`land-and-deploy/SKILL.md`](../land-and-deploy/SKILL.md)
|
||||
@@ -1162,6 +1162,8 @@ Claude: Detected: Fly.io (fly.toml found)
|
||||
Written to CLAUDE.md. Run /land-and-deploy when ready.
|
||||
```
|
||||
|
||||
If you answer "Nothing is deployed yet", `/setup-deploy` stops and writes nothing, because there is no production deploy to record. [How to bring up your first production deploy](./howto-bring-up-a-first-deploy.md) covers that first step; run `/setup-deploy` again once the app is live.
|
||||
|
||||
---
|
||||
|
||||
## `/codex`
|
||||
|
||||
+11
-3
@@ -172,9 +172,17 @@ the readiness dashboard and the PR body, and never count it as a pass.
|
||||
review, including the `execution` gate behind `/codex` consult and challenge.
|
||||
A reply whose last `Verdict:` line says `not ready` but carries no readable
|
||||
severity prints `FINDINGS: unspecified` and exits 3 like a blocking finding.
|
||||
- **No findings.** A review with no tag and no label is `clean` only when it
|
||||
says so explicitly (`NO_FINDINGS`, "no issues", "did not find any bugs").
|
||||
Otherwise it is `unverified` (see below), never `clean`. A `/codex` consult
|
||||
- **No findings.** A review with no tag and no label is `clean` only when its
|
||||
conclusion says so: the last sentence (or its last ", and" clause), or the
|
||||
`because` clause of its `Recommendation:` line, opens with `NO_FINDINGS`,
|
||||
"No issues found", "I found no actionable regressions" or "I did not find
|
||||
any bugs", with no "but", "however", "except", "other than" or "elsewhere".
|
||||
When other prose comes first, anything after that statement except a
|
||||
predicate ("were found", "is evident") or a colon explanation must scope it
|
||||
to the change as a whole ("in this change", "in the staged changes"); "No
|
||||
issues in the parser." does not conclude the review. A "no issues" phrase
|
||||
anywhere else in the reply does not count. Otherwise the review is
|
||||
`unverified` (see below), never `clean`. A `/codex` consult
|
||||
answer with no severity labels at all is not a review summary and stays
|
||||
`clean`.
|
||||
- **Design proposals** (the design-direction voices in /design-consultation
|
||||
|
||||
+9
-6
@@ -545,13 +545,16 @@ function historyCoverage(status: any, scope: string, auditPaths?: string[]): Cov
|
||||
? {
|
||||
domain: 'history-inputs',
|
||||
scope,
|
||||
status: auditPaths ? 'partial' : 'assessed',
|
||||
status: auditPaths || status.truncated ? 'partial' : 'assessed',
|
||||
method: 'bounded helper-retained Git history',
|
||||
gaps: auditPaths
|
||||
? [
|
||||
`History outside the --paths audit scope (${auditPaths.join(', ')}) was not retained or assessed`,
|
||||
]
|
||||
: [],
|
||||
gaps: [
|
||||
...(auditPaths
|
||||
? [
|
||||
`History outside the --paths audit scope (${auditPaths.join(', ')}) was not retained or assessed`,
|
||||
]
|
||||
: []),
|
||||
...(typeof status.truncated === 'string' ? [`History truncated: ${status.truncated}`] : []),
|
||||
],
|
||||
exclusions: [],
|
||||
evidence: [
|
||||
`Captured ${String(status.commits ?? 'bounded commits')} from ${String(status.range ?? 'the pinned source range')}`,
|
||||
|
||||
+30
-10
@@ -180,7 +180,7 @@ export interface HttpAssertion {
|
||||
method: 'GET' | 'POST' | 'PUT' | 'PATCH' | 'DELETE';
|
||||
headers?: Record<string, string>;
|
||||
body?: string;
|
||||
expected: { status: number; includes?: string; excludes?: string };
|
||||
expected: { status: number | number[]; includes?: string; excludes?: string };
|
||||
vulnerable?: { status: number; includes?: string; excludes?: string };
|
||||
}
|
||||
export interface Command {
|
||||
@@ -648,16 +648,34 @@ export function validateVerificationObservation(value: unknown): VerificationObs
|
||||
inputHash: v.inputHash,
|
||||
};
|
||||
}
|
||||
function assertion(value: unknown, name: string): HttpAssertion {
|
||||
const MAX_SAFE_STATUSES = 8;
|
||||
function assertion(value: unknown, name: string, safeStatusSet = false): HttpAssertion {
|
||||
const v = object(value, name),
|
||||
expected = object(v.expected, `${name}.expected`);
|
||||
exact(v, ['name', 'path', 'method', 'headers', 'body', 'expected', 'vulnerable'], name);
|
||||
const oracle = (x: Record<string, any>, n: string) => {
|
||||
const httpStatus = (s: unknown) => Number.isInteger(s) && (s as number) >= 100 && (s as number) <= 599;
|
||||
const oracle = (x: Record<string, any>, n: string, statusSet = false) => {
|
||||
exact(x, ['status', 'includes', 'excludes'], n);
|
||||
if (!Number.isInteger(x.status) || x.status < 100 || x.status > 599)
|
||||
throw new CsoError('INVALID_SCHEMA', `${n}.status must be an HTTP status`);
|
||||
if (!Array.isArray(x.status)) {
|
||||
if (!httpStatus(x.status)) throw new CsoError('INVALID_SCHEMA', `${n}.status must be an HTTP status`);
|
||||
} else if (!statusSet) throw new CsoError('INVALID_SCHEMA', `${n}.status must be an HTTP status`);
|
||||
else if (
|
||||
!x.status.length ||
|
||||
x.status.length > MAX_SAFE_STATUSES ||
|
||||
!x.status.every(httpStatus) ||
|
||||
new Set(x.status).size !== x.status.length
|
||||
)
|
||||
throw new CsoError(
|
||||
'INVALID_SCHEMA',
|
||||
`${n}.status must be 1..${MAX_SAFE_STATUSES} distinct HTTP statuses`,
|
||||
);
|
||||
else if (typeof x.excludes !== 'string' || !x.excludes)
|
||||
throw new CsoError(
|
||||
'INVALID_SCHEMA',
|
||||
`${n} with a set of safe statuses must exclude the vulnerable marker`,
|
||||
);
|
||||
return {
|
||||
status: x.status,
|
||||
status: Array.isArray(x.status) ? [...x.status] : x.status,
|
||||
...(x.includes === undefined ? {} : { includes: string(x.includes, `${n}.includes`) }),
|
||||
...(x.excludes === undefined ? {} : { excludes: string(x.excludes, `${n}.excludes`) }),
|
||||
};
|
||||
@@ -683,7 +701,7 @@ function assertion(value: unknown, name: string): HttpAssertion {
|
||||
method: oneOf(v.method, ['GET', 'POST', 'PUT', 'PATCH', 'DELETE'], `${name}.method`),
|
||||
...(Object.keys(headers).length ? { headers } : {}),
|
||||
...(v.body === undefined ? {} : { body: string(v.body, `${name}.body`, 65536) }),
|
||||
expected: oracle(expected, `${name}.expected`),
|
||||
expected: oracle(expected, `${name}.expected`, safeStatusSet && v.vulnerable !== undefined),
|
||||
...(v.vulnerable === undefined ? {} : { vulnerable: oracle(object(v.vulnerable), `${name}.vulnerable`) }),
|
||||
};
|
||||
}
|
||||
@@ -692,13 +710,13 @@ export function securityAssertions(security: VerificationRequest['security']): H
|
||||
return Array.isArray(security) ? security : [security];
|
||||
}
|
||||
function securityAssertion(value: unknown): VerificationRequest['security'] {
|
||||
if (!Array.isArray(value)) return assertion(value, 'security');
|
||||
if (!Array.isArray(value)) return assertion(value, 'security', true);
|
||||
if (!value.length || value.length > MAX_SECURITY_ASSERTIONS)
|
||||
throw new CsoError(
|
||||
'INVALID_SCHEMA',
|
||||
`security must be one assertion or 1..${MAX_SECURITY_ASSERTIONS} assertions`,
|
||||
);
|
||||
const assertions = value.map((x, i) => assertion(x, `security[${i}]`));
|
||||
const assertions = value.map((x, i) => assertion(x, `security[${i}]`, true));
|
||||
if (new Set(assertions.map((x) => x.name)).size !== assertions.length)
|
||||
throw new CsoError('INVALID_SCHEMA', 'security assertion names must be distinct');
|
||||
return assertions;
|
||||
@@ -805,7 +823,9 @@ export function validateVerificationRequest(input: unknown): VerificationRequest
|
||||
'Verification needs a legitimate control, distinct before/fixed security oracles, existing tests, immutable test files, and boundary files',
|
||||
);
|
||||
const mutuallyExclusive = ({ expected: secure, vulnerable }: HttpAssertion) =>
|
||||
secure.status !== vulnerable!.status ||
|
||||
(Array.isArray(secure.status)
|
||||
? !secure.status.includes(vulnerable!.status)
|
||||
: secure.status !== vulnerable!.status) ||
|
||||
(secure.includes !== undefined &&
|
||||
vulnerable!.excludes !== undefined &&
|
||||
secure.includes.includes(vulnerable!.excludes)) ||
|
||||
|
||||
@@ -57,6 +57,29 @@ export function gitDiffHeaderPaths(line: string): [string, string] | undefined {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace every diff section touching a path `withheld` names with one marker line, so a file the snapshot keeps
|
||||
* out of the readable tree cannot come back through retained history. An unparseable diff header is withheld too.
|
||||
*/
|
||||
export function withholdHistory(raw: string, withheld: (path: string) => string | undefined): string {
|
||||
const output: string[] = [];
|
||||
let skipping = false;
|
||||
for (const line of raw.split('\n')) {
|
||||
if (line.startsWith('commit ')) skipping = false;
|
||||
else if (line.startsWith('diff --git ')) {
|
||||
const paths = gitDiffHeaderPaths(line)?.map((value) => value.replace(/^[ab]\//, '')),
|
||||
reason = paths ? paths.map(withheld).find(Boolean) : 'unparseable diff header';
|
||||
skipping = Boolean(reason);
|
||||
if (reason) {
|
||||
output.push(`diff withheld: ${paths?.[1] ?? 'unknown path'} (${reason})`);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if (!skipping) output.push(line);
|
||||
}
|
||||
return output.join('\n');
|
||||
}
|
||||
|
||||
/** Return only exact path hunks, keeping one commit preamble per matching commit. */
|
||||
export function historyForPath(raw: string, path: string): string | undefined {
|
||||
const expected = new Set([`a/${path}`, `b/${path}`]),
|
||||
|
||||
@@ -1106,6 +1106,25 @@ export async function superviseRegistrySocket(options: {
|
||||
* Requires qualified runtime images to contain the fixed, compiled
|
||||
* /opt/cso/preparation@1.0.0 helper contract.
|
||||
*/
|
||||
/**
|
||||
* Write the helper's own files (Rails' test database.yml and friends) into an execution copy. The copy keeps the
|
||||
* source's modes, and a sealed or read-only checkout makes those files 0444, so each copied file is replaced, never
|
||||
* opened for writing (or written through).
|
||||
*/
|
||||
export function writeOfflineTransformations(
|
||||
source: string,
|
||||
transformations: OfflinePreparationRequest['transformations'],
|
||||
): void {
|
||||
for (const transformation of transformations) {
|
||||
if (sha256(transformation.content) !== transformation.sha256)
|
||||
fail('INCOMPATIBLE_INPUT', 'Offline transformation content hash changed');
|
||||
const file = contained(source, transformation.path);
|
||||
secureDirectory(resolve(file, '..'));
|
||||
fs.rmSync(file, { force: true });
|
||||
fs.writeFileSync(file, transformation.content, { mode: 0o600, flag: 'wx' });
|
||||
}
|
||||
}
|
||||
|
||||
export class DockerPreparationSandboxRunner implements PreparationSandboxRunner {
|
||||
readonly qualification;
|
||||
private readonly runtime;
|
||||
@@ -1514,13 +1533,7 @@ export class DockerPreparationSandboxRunner implements PreparationSandboxRunner
|
||||
errorOnExist: false,
|
||||
preserveTimestamps: false,
|
||||
});
|
||||
for (const transformation of request.transformations) {
|
||||
if (sha256(transformation.content) !== transformation.sha256)
|
||||
fail('INCOMPATIBLE_INPUT', 'Offline transformation content hash changed');
|
||||
const file = contained(source, transformation.path);
|
||||
secureDirectory(resolve(file, '..'));
|
||||
fs.writeFileSync(file, transformation.content, { mode: 0o600 });
|
||||
}
|
||||
writeOfflineTransformations(source, request.transformations);
|
||||
for (const item of request.metadata) {
|
||||
if (sha256(item.content) !== item.sha256)
|
||||
fail('INCOMPATIBLE_INPUT', `Offline metadata hash changed: ${item.path}`);
|
||||
|
||||
+112
-23
@@ -14,6 +14,7 @@ import {
|
||||
snapshotPathId,
|
||||
MAX_OUTPUT,
|
||||
} from './contracts';
|
||||
import { withholdHistory } from './history';
|
||||
import { childEnvironment, executable, git, redact, runProcess } from './process';
|
||||
import { boundedList, readJson, secureDirectory, writeHelperJson, writeJson } from './state';
|
||||
import { scan } from '../redact-engine';
|
||||
@@ -68,6 +69,14 @@ export function sourceCapMessage(bytes: number, files: number): string {
|
||||
}
|
||||
/** Transformation recorded for a source path outside the start --paths prefixes; its payload is never copied or assessed. */
|
||||
export const OUTSIDE_AUDIT_PATHS = 'withheld: outside the --paths audit scope';
|
||||
const HISTORY_MARKER_RESERVE = 1024;
|
||||
/** First line of a history.txt that holds fewer commits or less detail than Git returned. */
|
||||
export function historyMarker(note: string): string {
|
||||
return `[gstack-cso history truncated: ${note}]\n`;
|
||||
}
|
||||
/** Transformation recorded for a repository boundary Git listed as one entry: an untracked nested repository or a registered submodule gitlink. */
|
||||
export const NESTED_REPOSITORY =
|
||||
'excluded: nested Git repository; Git listed it as a single boundary entry and never descends into it, so its own source and history are not audited with this repository';
|
||||
export function inAuditPaths(path: string, prefixes: readonly string[]): boolean {
|
||||
return prefixes.some((prefix) => path === prefix || path.startsWith(`${prefix}/`));
|
||||
}
|
||||
@@ -517,7 +526,12 @@ async function paths(
|
||||
for (const value of data.split('\0')) {
|
||||
admission.time();
|
||||
if (!value) continue;
|
||||
seen.add(relativePath(value));
|
||||
// `ls-files --others` reports a repository boundary as one entry with a
|
||||
// trailing slash: an untracked nested repository that Git refuses to
|
||||
// descend into. The path itself is the boundary, exactly like a
|
||||
// registered submodule's gitlink, so record it under its own name
|
||||
// instead of rejecting the whole run.
|
||||
seen.add(relativePath(value.replace(/\/$/, '')));
|
||||
admission.count(seen.size);
|
||||
}
|
||||
const result = [...seen].sort();
|
||||
@@ -643,6 +657,7 @@ export async function capture(
|
||||
gitHashes = new Map<string, string>(),
|
||||
gitModes = new Map<string, string>(),
|
||||
sensitiveEvidence: any[] = [],
|
||||
boundaries = new Map<string, fs.Stats>(),
|
||||
absentPaths = new Set<string>();
|
||||
let total = 0;
|
||||
guard();
|
||||
@@ -690,15 +705,35 @@ export async function capture(
|
||||
};
|
||||
for (const path of list) {
|
||||
guard();
|
||||
let size: number;
|
||||
let source: fs.Stats;
|
||||
try {
|
||||
size = fs.lstatSync(join(repo, path)).size;
|
||||
source = fs.lstatSync(join(repo, path));
|
||||
} catch (error: any) {
|
||||
if (error?.code === 'ENOENT') {
|
||||
absentPaths.add(path);
|
||||
continue;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
const size = source.size;
|
||||
// A directory in Git's inventory is a repository boundary, not source:
|
||||
// an untracked nested repository (listed with a trailing slash) or a
|
||||
// registered submodule's gitlink. Git reports it as one entry and never
|
||||
// descends, so its contents are not this repository's source. Record an
|
||||
// explicit exclusion bound to the boundary's identity, exactly like a
|
||||
// host dependency tree, instead of reading it or failing the run.
|
||||
if (source.isDirectory()) {
|
||||
boundaries.set(path, source);
|
||||
entries.push({
|
||||
path,
|
||||
pathId: snapshotPathId(repo, path),
|
||||
originalHash: 'not-read',
|
||||
bytes: source.size,
|
||||
mode: source.mode & 0o777,
|
||||
transformation: NESTED_REPOSITORY,
|
||||
});
|
||||
guard();
|
||||
continue;
|
||||
} // tracked deletions are represented by absence and the diff manifest
|
||||
// Host dependency trees and paths outside --paths aren't copied, assessed,
|
||||
// or counted toward the source cap. Their omission is still explicit.
|
||||
@@ -807,6 +842,22 @@ export async function capture(
|
||||
for (const e of entries) {
|
||||
guard();
|
||||
if (e.originalHash === 'not-read') {
|
||||
const boundary = boundaries.get(e.path);
|
||||
if (boundary) {
|
||||
// A nested repository is recorded by identity, never read: a
|
||||
// swap, symlink or replacement directory fails the capture.
|
||||
const current = fs.lstatSync(containedFile(repo, e.path));
|
||||
if (
|
||||
current.isSymbolicLink() ||
|
||||
!current.isDirectory() ||
|
||||
current.dev !== boundary.dev ||
|
||||
current.ino !== boundary.ino ||
|
||||
current.mode !== boundary.mode
|
||||
)
|
||||
throw new CsoError('SNAPSHOT_RACE', `${message}: ${e.path}`);
|
||||
guard();
|
||||
continue;
|
||||
}
|
||||
const current = fs.lstatSync(containedFile(repo, e.path)),
|
||||
compared = gitHashes.get(e.path);
|
||||
if (
|
||||
@@ -875,33 +926,71 @@ export async function capture(
|
||||
});
|
||||
} else {
|
||||
const range = manifest.baseCommit ? `${manifest.baseCommit}..${headCommit}` : headCommit;
|
||||
const log = (patches: boolean, maxBytes?: number) =>
|
||||
git(
|
||||
repo,
|
||||
[
|
||||
'-c',
|
||||
'core.quotePath=false',
|
||||
'log',
|
||||
'--no-ext-diff',
|
||||
'--no-textconv',
|
||||
'--max-count=100',
|
||||
'--format=commit %H%nAuthor: %an%nDate: %aI%nSubject: %s',
|
||||
...(patches ? ['--unified=3', '-p'] : []),
|
||||
range,
|
||||
'--',
|
||||
...(auditPaths ?? []).map((prefix) => `:(literal)${prefix}`),
|
||||
],
|
||||
home,
|
||||
maxBytes,
|
||||
);
|
||||
admission.time();
|
||||
const raw = await git(
|
||||
repo,
|
||||
[
|
||||
'-c',
|
||||
'core.quotePath=false',
|
||||
'log',
|
||||
'--no-ext-diff',
|
||||
'--no-textconv',
|
||||
'--max-count=100',
|
||||
'--format=commit %H%nAuthor: %an%nDate: %aI%nSubject: %s',
|
||||
'--unified=3',
|
||||
'-p',
|
||||
range,
|
||||
'--',
|
||||
...(auditPaths ?? []).map((prefix) => `:(literal)${prefix}`),
|
||||
],
|
||||
home,
|
||||
);
|
||||
let patched: string | undefined;
|
||||
try {
|
||||
patched = await log(true, SNAPSHOT_MANIFEST_LIMIT);
|
||||
} catch (error) {
|
||||
if (!(error instanceof CsoError) || error.code !== 'MISSING_INPUT') throw error;
|
||||
}
|
||||
guard();
|
||||
const safe = redact(raw);
|
||||
const commits = withholdHistory(patched ?? '', exclusion)
|
||||
.split(/^(?=commit [0-9a-f]{40}(?:[0-9a-f]{24})?$)/m)
|
||||
.filter(Boolean),
|
||||
sizes = commits.map((commit) => Buffer.byteLength(commit)),
|
||||
whole = sizes.reduce((sum, size) => sum + size, 0) <= MAX_OUTPUT;
|
||||
let kept = 0,
|
||||
bytes = 0,
|
||||
safe = patched === '' ? '' : undefined,
|
||||
truncated: string | undefined,
|
||||
retainedCommits: number | string = 'at most 100';
|
||||
while (kept < commits.length && (whole || bytes + sizes[kept] <= MAX_OUTPUT - HISTORY_MARKER_RESERVE))
|
||||
bytes += sizes[kept++];
|
||||
for (; kept > 0 && safe === undefined; kept--) {
|
||||
admission.time();
|
||||
const omitted = commits.length - kept,
|
||||
note = omitted
|
||||
? `the newest ${kept} of ${commits.length} commits were retained with patches; ${omitted} older commits were omitted to stay within the ${MAX_OUTPUT}-byte history limit`
|
||||
: undefined,
|
||||
text = redact((note ? historyMarker(note) : '') + commits.slice(0, kept).join(''));
|
||||
if (Buffer.byteLength(text) > MAX_OUTPUT) continue;
|
||||
safe = text;
|
||||
truncated = note;
|
||||
if (omitted) retainedCommits = kept;
|
||||
}
|
||||
if (safe === undefined) {
|
||||
const metadata = await log(false);
|
||||
guard();
|
||||
truncated = `patches were withheld because the newest commit's patch or the patch log exceeds the ${MAX_OUTPUT}-byte history limit; only commit metadata was retained`;
|
||||
safe = redact(historyMarker(truncated) + metadata);
|
||||
retainedCommits = 'metadata for at most 100';
|
||||
}
|
||||
atomicWriteSync(join(runDir, 'history.txt'), safe, { mode: 0o600 });
|
||||
writeJson(join(runDir, 'history-status.json'), {
|
||||
status: 'captured',
|
||||
range: auditPaths ? `${range}, limited to --paths ${auditPaths.join(', ')}` : range,
|
||||
commits: 'at most 100',
|
||||
commits: retainedCommits,
|
||||
bytes: Buffer.byteLength(safe),
|
||||
...(truncated ? { truncated } : {}),
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
|
||||
+10
-3
@@ -1219,14 +1219,21 @@ export function certify(params: {
|
||||
beforeReceipt.observationHash !== witnessObservationHash(beforeObservation) ||
|
||||
afterReceipt.observationHash !== witnessObservationHash(afterObservation) ||
|
||||
params.before.existingTests !== beforeReceipt.diagnosticTestsPassed ||
|
||||
params.after.existingTests !== afterReceipt.diagnosticTestsPassed ||
|
||||
!beforeReceipt.externalAssertionsPassed ||
|
||||
!afterReceipt.externalAssertionsPassed
|
||||
params.after.existingTests !== afterReceipt.diagnosticTestsPassed
|
||||
)
|
||||
throw new CsoError(
|
||||
'INCOMPATIBLE_INPUT',
|
||||
'Authenticated assertion witness receipts do not match the verifier observations',
|
||||
);
|
||||
for (const [phase, receipt] of [
|
||||
['before', beforeReceipt],
|
||||
['after', afterReceipt],
|
||||
] as const)
|
||||
if (!receipt.externalAssertionsPassed)
|
||||
throw new CsoError(
|
||||
'ASSERTION_FAILED',
|
||||
`External assertions failed in the ${phase} phase: booted ${params[phase].booted}, legitimate ${params[phase].legitimate}, security ${params[phase].security}`,
|
||||
);
|
||||
const stableExecutions = (receipt: AssertionWitnessReceipt) =>
|
||||
receipt.executions.map(({ outputHash: _, ...execution }) => execution);
|
||||
if (canonical(stableExecutions(beforeReceipt)) !== canonical(stableExecutions(afterReceipt)))
|
||||
|
||||
+1
-1
@@ -18,7 +18,7 @@ interface Config {
|
||||
}
|
||||
function matches(status: number, body: string, oracle: HttpAssertion['expected']): boolean {
|
||||
return (
|
||||
status === oracle.status &&
|
||||
(Array.isArray(oracle.status) ? oracle.status.includes(status) : status === oracle.status) &&
|
||||
(oracle.includes === undefined || body.includes(oracle.includes)) &&
|
||||
(oracle.excludes === undefined || !body.includes(oracle.excludes))
|
||||
);
|
||||
|
||||
@@ -135,8 +135,56 @@ const SEVERITY_LABELS = [
|
||||
function severityWords(text: string): Severity[] {
|
||||
return SEVERITY_LABELS.flatMap(re => [...text.matchAll(re)].map(m => SEVERITY_WORDS[m[1]!.toLowerCase()]!));
|
||||
}
|
||||
/** An explicit conclusion that the review found nothing to report. */
|
||||
const NO_FINDINGS = /\bNO_FINDINGS\b|\bno (?:actionable |significant |new |concrete )?(?:bugs?|issues?|findings?|problems?)\b|\b(?:did not|didn't|cannot|can't|could not|couldn't) (?:find|identify) any (?:actionable |new |concrete )?(?:bugs|issues|findings|problems)\b/i;
|
||||
/**
|
||||
* A no-findings statement opening a sentence: NO_FINDINGS (its underscore is
|
||||
* gone after plainReview), "No issues found", "I found no actionable
|
||||
* regressions", "I did not find any bugs". Never a phrase inside another
|
||||
* clause ("renaming the flag is no problem for callers, but ...").
|
||||
*/
|
||||
const FINDING_NOUN = '(?:(?:actionable|significant|new|concrete)\\s+)?(?:bugs?|issues?|findings?|problems?|regressions?|defects?)\\b';
|
||||
const NO_FINDINGS_LEAD = '(?:(?:overall|in summary|in short|summary|conclusion|result)[,:]?\\s+|(?:after|having|on|upon)\\s+(?:reviewing|reading|checking|inspecting|reviewed|read|checked|inspected)\\b[^,.;]{0,80},\\s+)?';
|
||||
const NO_FINDINGS = new RegExp(`^${NO_FINDINGS_LEAD}(?:NO_?FINDINGS\\b|(?:(?:i|we)\\s+(?:found|see|saw|identified)\\s+|there\\s+(?:are|were|is)\\s+)?no\\s+${FINDING_NOUN}|(?:i|we)\\s+(?:did not|didn't|could not|couldn't|cannot|can't)\\s+(?:find|identify|see)\\s+any\\s+${FINDING_NOUN})`, 'i');
|
||||
/** A contrast or exception qualifies the claim, so the sentence does not conclude that nothing was found. */
|
||||
const NO_FINDINGS_EXCEPTION = /(?<![\w-])(?:but|however|although|though|yet|except|excluding|unless|only|apart from|aside from|other than|besides|beyond|elsewhere|otherwise)(?![\w-])/i;
|
||||
const NO_FINDINGS_PREDICATE = /^\s*(?:(?:was|were|is|are|has been|have been)\s+)?(?:found|identified|detected|evident|apparent|introduced|remain|exist|to report)?/i;
|
||||
const WHOLE_CHANGE_SCOPE = /^(?:in|within|across|throughout|with|from|for)\s.*\b(?:chang(?:e|es|ed)|diffs?|patch(?:es)?|staged|unstaged|untracked|branch|commits?|pull request|PR)\b/i;
|
||||
|
||||
/**
|
||||
* The review's conclusion says it found nothing: its last prose sentence or
|
||||
* ", and" clause (a wrapped line that continues in lowercase joined), or the because-clause of its
|
||||
* Recommendation line, is an unqualified no-findings statement. When other
|
||||
* prose precedes it, anything after the statement other than a predicate
|
||||
* ("were found", "is evident") or a colon explanation must scope it to the
|
||||
* change as a whole: "No issues in the parser." or "No regressions were found
|
||||
* by the tests." says nothing about the rest of the reply.
|
||||
*/
|
||||
function concludesNoFindings(code: string): boolean {
|
||||
let prose = '', because: string | undefined, recommendationOpen = false;
|
||||
for (const line of code.split(/\r?\n/)) {
|
||||
const text = plainReview(line).trim();
|
||||
const listed = /^[\t ]*(?:#{1,6}|[-+*])[\t ]/.test(line);
|
||||
if (recommendationOpen && text && !listed) {
|
||||
because = `${because} ${text}`;
|
||||
recommendationOpen = !/[.!?]$/.test(text);
|
||||
continue;
|
||||
}
|
||||
recommendationOpen = false;
|
||||
if (!text || /^(?:Recommendation|Verdict)[\t ]*:/i.test(text)) {
|
||||
const clause = text.match(/^Recommendation[\t ]*:.*?\bbecause\b[\t ]*(\S.*)$/i)?.[1];
|
||||
if (clause) { because = clause; recommendationOpen = !/[.!?]$/.test(clause); }
|
||||
prose += '\n';
|
||||
} else if (/^[\t ]*#{1,6}[\t ]/.test(line)) prose += `\n${text}\n`;
|
||||
else prose += listed || prose.endsWith('\n') || /[.!?;:]$/.test(prose) || !/^[a-z]/.test(text) ? `\n${text}` : ` ${text}`;
|
||||
}
|
||||
const sentences = prose.split(/(?<=[.!?;])\s+|\n|,\s+and\s+/i).map(sentence => sentence.trim()).filter(Boolean);
|
||||
const total = sentences.length + (because ? 1 : 0);
|
||||
return [sentences.at(-1), because].some(conclusion => {
|
||||
const statement = conclusion?.match(NO_FINDINGS)?.[0];
|
||||
if (!conclusion || !statement || NO_FINDINGS_EXCEPTION.test(conclusion)) return false;
|
||||
const qualifier = conclusion.slice(statement.length).replace(NO_FINDINGS_PREDICATE, '').replace(/[\s.!]+$/, '').trim();
|
||||
return total === 1 || !qualifier || qualifier.startsWith(':') || WHOLE_CHANGE_SCOPE.test(qualifier);
|
||||
});
|
||||
}
|
||||
|
||||
/** Formatting the requested marker in bold, inline code, or a list does not invalidate a completed review. */
|
||||
function plainReview(text: string): string {
|
||||
@@ -159,7 +207,7 @@ function withoutCode(text: string): string {
|
||||
}
|
||||
if (marker) { fence = marker; return ''; }
|
||||
return line.replace(/`([^`]+)`/g, (span, inner: string, at: number) =>
|
||||
/^\[?(?:P[0-3]|critical|high|medium|low)\]?$/i.test(inner.trim()) && CODE_LABEL_BEFORE.test(line.slice(0, at)) ? inner : ' ');
|
||||
inner.trim() === 'NO_FINDINGS' || /^\[?(?:P[0-3]|critical|high|medium|low)\]?$/i.test(inner.trim()) && CODE_LABEL_BEFORE.test(line.slice(0, at)) ? inner : ' ');
|
||||
}).join('\n');
|
||||
}
|
||||
|
||||
@@ -218,7 +266,7 @@ export function classifyOutsideReview(input: OutsideReviewInput): OutsideReviewC
|
||||
const notReady = !findings.highest && verdictNotReady(plainCode);
|
||||
if (blocking || notReady) return { ...result('findings'), ...(unparsed ? { unparsed } : {}) };
|
||||
if (unparsed) return { ...result('unverified', 'unparsed_severity', unparsed), unparsed };
|
||||
if ((input.gate === 'structured' || input.gate === 'review') && !findings.highest && !NO_FINDINGS.test(input.text)) {
|
||||
if ((input.gate === 'structured' || input.gate === 'review') && !findings.highest && !concludesNoFindings(code)) {
|
||||
return result('unverified', 'untagged_review', 'missing severity or explicit no-findings conclusion');
|
||||
}
|
||||
return result('clean');
|
||||
|
||||
+4
-4
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gstack",
|
||||
"version": "1.91.149",
|
||||
"version": "1.91.156",
|
||||
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
||||
"license": "MIT",
|
||||
"type": "module",
|
||||
@@ -69,16 +69,16 @@
|
||||
"eval:bg:release": "bun run scripts/eval-bg.ts release"
|
||||
},
|
||||
"dependencies": {
|
||||
"@huggingface/transformers": "^4.3.0",
|
||||
"@huggingface/transformers": "^4.3.1",
|
||||
"@ngrok/ngrok": "^1.7.0",
|
||||
"cross-spawn": "^7.0.6",
|
||||
"diff": "^9.0.0",
|
||||
"html-to-docx": "1.8.0",
|
||||
"html-to-vdom": "0.7.0",
|
||||
"marked": "^18.0.11",
|
||||
"marked": "^18.1.1",
|
||||
"playwright": "^1.62.1",
|
||||
"sharp": "^0.35.4",
|
||||
"socks": "^2.8.9"
|
||||
"socks": "^2.8.10"
|
||||
},
|
||||
"engines": {
|
||||
"bun": ">=1.4.2"
|
||||
|
||||
@@ -441,18 +441,19 @@ Initial scope selectors need no grid or **pre-answer** ledger write. Ask and
|
||||
wait before changes.
|
||||
|
||||
1. Explain the complexity. Ask each proposed feature cut/deferral separately;
|
||||
wait before changing scope. With no proposed cuts, keep the feature list and
|
||||
go directly to the structure question.
|
||||
2. Always ask the structure question when this gate trips, even with no cuts.
|
||||
Compare only the file/class arrangement. Use labels `Original arrangement`
|
||||
and `Smaller arrangement`; put files/classes in each description. Both retain
|
||||
wait before changing scope.
|
||||
2. Always resolve the structure question when this gate trips, even with no cuts,
|
||||
as question_id `plan-eng-review-structure` (no per-plan slug); `AUTO_DECIDE`
|
||||
picks the recommendation. Compare only the file/class arrangement, labeled
|
||||
`Original arrangement` and `Smaller arrangement`, with files/classes per
|
||||
option. Both retain
|
||||
the same approved feature list, contracts and approved
|
||||
security/error/test/performance fixes. Include `Pending remedies not decided here: <ids>` in the
|
||||
question; unapproved fixes stay pending. If no smaller arrangement preserves
|
||||
these commitments, explain that and offer confirmation of the original
|
||||
arrangement or a pause to investigate a smaller one. Wait for the answer.
|
||||
A pause leaves the arrangement undecided: investigate only the agreed question,
|
||||
then return to this structure selector. Do not continue to C until it is settled.
|
||||
then return to the selector. Do not continue to C until it is settled.
|
||||
3. Save the actual feature and structure answers as one scope record: `feature
|
||||
answers: <refs>; structure: <A/B + ref>; accepted scope: <exact scope>;
|
||||
pending remedies: <ids or none>`.
|
||||
|
||||
@@ -339,18 +339,19 @@ Initial scope selectors need no grid or **pre-answer** ledger write. Ask and
|
||||
wait before changes.
|
||||
|
||||
1. Explain the complexity. Ask each proposed feature cut/deferral separately;
|
||||
wait before changing scope. With no proposed cuts, keep the feature list and
|
||||
go directly to the structure question.
|
||||
2. Always ask the structure question when this gate trips, even with no cuts.
|
||||
Compare only the file/class arrangement. Use labels `Original arrangement`
|
||||
and `Smaller arrangement`; put files/classes in each description. Both retain
|
||||
wait before changing scope.
|
||||
2. Always resolve the structure question when this gate trips, even with no cuts,
|
||||
as question_id `plan-eng-review-structure` (no per-plan slug); `AUTO_DECIDE`
|
||||
picks the recommendation. Compare only the file/class arrangement, labeled
|
||||
`Original arrangement` and `Smaller arrangement`, with files/classes per
|
||||
option. Both retain
|
||||
the same approved feature list, contracts and approved
|
||||
security/error/test/performance fixes. Include `Pending remedies not decided here: <ids>` in the
|
||||
question; unapproved fixes stay pending. If no smaller arrangement preserves
|
||||
these commitments, explain that and offer confirmation of the original
|
||||
arrangement or a pause to investigate a smaller one. Wait for the answer.
|
||||
A pause leaves the arrangement undecided: investigate only the agreed question,
|
||||
then return to this structure selector. Do not continue to C until it is settled.
|
||||
then return to the selector. Do not continue to C until it is settled.
|
||||
3. Save the actual feature and structure answers as one scope record: `feature
|
||||
answers: <refs>; structure: <A/B + ref>; accepted scope: <exact scope>;
|
||||
pending remedies: <ids or none>`.
|
||||
|
||||
@@ -336,6 +336,14 @@ export const QUESTIONS = {
|
||||
signal_key: 'scope-appetite',
|
||||
description: "Plan touches 8+ files — reduce scope or hold?",
|
||||
},
|
||||
'plan-eng-review-structure': {
|
||||
id: 'plan-eng-review-structure',
|
||||
skill: 'plan-eng-review',
|
||||
category: 'routing',
|
||||
door_type: 'two-way',
|
||||
options: ['original-arrangement', 'smaller-arrangement'],
|
||||
description: "Complexity gate — Original arrangement or Smaller arrangement (file/class layout only)?",
|
||||
},
|
||||
'plan-eng-review-test-gap': {
|
||||
id: 'plan-eng-review-test-gap',
|
||||
skill: 'plan-eng-review',
|
||||
|
||||
@@ -145,83 +145,59 @@ Skills that run plan reviews (`/plan-*-review`, `/codex review`) include the EXI
|
||||
|
||||
# Setup Browser Cookies
|
||||
|
||||
## 1. Choose the browser
|
||||
## 1. Check for Aside
|
||||
|
||||
Use this checkout as the gstack root if it contains `BROWSER.md` and `browse/SKILL.md`; otherwise use the installed root containing `bin/gstack-skill-start`, never a generated host stub. Read that root's `browse/SKILL.md` **BROWSER SETUP** section and run its probe first. On `READY`, stop importing: use Aside's sessions or ask the user to sign in there. Otherwise follow the probe's fallback handling, then continue below.
|
||||
Use this checkout as the gstack root if it contains `BROWSER.md` and `browse/SKILL.md`; otherwise use the installed root containing `bin/gstack-skill-start`, never a generated host stub. Read that root's `browse/SKILL.md` **BROWSER SETUP** section and run its Aside probe.
|
||||
|
||||
## SETUP (run this check BEFORE any browse command)
|
||||
- **Aside probe prints `READY`:** no import is needed, because Aside already has the user's signed-in sessions. Tell the user to use Aside, or to sign in there, and end this skill.
|
||||
- **Any other result:** continue with gstack's own headless browser (`$B`).
|
||||
|
||||
## 2. Get every decision from the user first
|
||||
|
||||
Ask these together, before running any `$B` command. Never guess an answer or treat the default Comet browser as consent.
|
||||
|
||||
1. **Source browser and profile.** `--profile` takes the profile's directory (`Default`, `Profile 2`), not its display name. A profile that cannot be read is unknown, not empty.
|
||||
2. **Domains to import,** and the **target**: one page on a chosen domain that the session should open, by URL.
|
||||
3. **Sign-in verification (`--verify-auth`)?** Default no. It reloads the target and reports **verified** only after a successful same-origin response with exactly one visible element, found by a CSS selector, whose whitespace-normalized text equals the expected identity. If yes, the user supplies the selector and the expected identity privately.
|
||||
4. **Clear stale storage (`--clear-storage`)?** Default no; needs explicit approval. Chromium only. It clears localStorage for the target's origin (shared by that origin's tabs) and sessionStorage for the target tab, and nothing else; other browser engines reject the reset but still import. A failed reset may already have cleared some storage and is not rolled back.
|
||||
5. **Every cookie in the profile (`--all`)?** Default no; needs explicit consent. It cannot be combined with a domain or with clearing storage.
|
||||
|
||||
Use the picker when any of answers 1-2 is still open; it asks for them. Use direct import when the user has named the browser, the profile directory and the domains.
|
||||
|
||||
## 3. Prepare gstack's browser
|
||||
|
||||
1. If the user wants verification, have them set `GSTACK_COOKIE_AUTH_SELECTOR` and `GSTACK_COOKIE_AUTH_EXPECTED_IDENTITY` privately in the shell that runs `$B`. The browser daemon reads them only when it starts, so if a gstack daemon is already running, run `$B stop` first. Never invent the values.
|
||||
2. Follow `browse/SKILL.md`'s **Browser fallback** section to find `$B`, building it if the check prints `NEEDS_SETUP`.
|
||||
3. Run `$B status`. If it reports `Mode: cdp`, `$B` is attached to the user's real browser and already has its sessions: tell the user no import is needed and end this skill.
|
||||
4. Open the target URL from answer 2 and leave that tab alone until the import finishes:
|
||||
|
||||
```bash
|
||||
_ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
|
||||
B=""
|
||||
[ -n "$_ROOT" ] && [ -x "$_ROOT/.claude/skills/gstack/browse/dist/browse" ] && B="$_ROOT/.claude/skills/gstack/browse/dist/browse"
|
||||
[ -z "$B" ] && B="$HOME/.claude/skills/gstack/browse/dist/browse"
|
||||
if [ -x "$B" ] && "${B%/browse/*}/bin/gstack-browser-ensure"; then
|
||||
echo "READY: $B"
|
||||
else
|
||||
echo "NEEDS_SETUP"
|
||||
fi
|
||||
$B goto "<url>"
|
||||
```
|
||||
|
||||
`gstack-browser-ensure` installs Chromium on first use when the install skipped it (lazy browser); it prints `BROWSER_OK` or `BROWSER_UNAVAILABLE <reason>` first.
|
||||
## 4. Import
|
||||
|
||||
If `NEEDS_SETUP`:
|
||||
1. Tell the user: "gstack browse needs a one-time build (~10 seconds). OK to proceed?" Then STOP and wait.
|
||||
2. Run: `cd <SKILL_DIR> && ./setup`
|
||||
3. If `bun` is not installed:
|
||||
```bash
|
||||
if ! command -v bun >/dev/null 2>&1; then
|
||||
BUN_VERSION="1.4.2"
|
||||
BUN_INSTALL_SHA="bab8acfb046aac8c72407bdcce903957665d655d7acaa3e11c7c4616beae68dd"
|
||||
tmpfile=$(mktemp "${TMPDIR:-/tmp}/bun-install.XXXXXX")
|
||||
curl -fsSL "https://bun.sh/install" -o "$tmpfile"
|
||||
# shasum is macOS/perl; coreutils-only Linux ships sha256sum instead —
|
||||
# resolve whichever exists so the verify never fails on a missing tool.
|
||||
if command -v sha256sum >/dev/null 2>&1; then
|
||||
actual_sha=$(sha256sum < "$tmpfile" | awk '{print $(1)}')
|
||||
else
|
||||
actual_sha=$(shasum -a 256 < "$tmpfile" | awk '{print $(1)}')
|
||||
fi
|
||||
if [ "$actual_sha" != "$BUN_INSTALL_SHA" ]; then
|
||||
echo "ERROR: bun install script checksum mismatch" >&2
|
||||
echo " expected: $BUN_INSTALL_SHA" >&2
|
||||
echo " got: $actual_sha" >&2
|
||||
rm "$tmpfile"; exit 1
|
||||
fi
|
||||
bash "$tmpfile" "bun-v$BUN_VERSION"
|
||||
rm "$tmpfile"
|
||||
fi
|
||||
```
|
||||
|
||||
```bash
|
||||
$B status
|
||||
```
|
||||
If status says `Mode: cdp`, stop: the real browser already has sessions.
|
||||
|
||||
## 2. Confirm options before import
|
||||
|
||||
Open the known target and keep its tab unchanged. Both options default off and require explicit request:
|
||||
|
||||
- **`--verify-auth` / picker checkbox:** reloads the target. Have the user privately configure daemon `GSTACK_COOKIE_AUTH_SELECTOR` and `GSTACK_COOKIE_AUTH_EXPECTED_IDENTITY` **before startup**. Never invent values or assume CLI env reconfigures an existing daemon. Missing config rejects before mutation. Require a successful same-origin response and exactly one visible element whose whitespace-normalized text exactly matches the expected identity.
|
||||
- **`--clear-storage`:** for suspected stale storage, obtain explicit approval. Chromium only: clears captured-origin localStorage (shared across same-origin context tabs) and target-tab sessionStorage. Other origins, other tabs' sessionStorage, IndexedDB, and service workers stay intact. It uses an isolated world/native deadline; other engines reject reset, not imports/auth checks. Never auto-approve or claim rollback after partial failure.
|
||||
|
||||
## 3. Select source and scope
|
||||
Picker (the user confirms the browser, profile and domains there, ticks verification if they asked for it, then tells you when done; the link works once, for five minutes, so rerun the command if it expires):
|
||||
|
||||
```bash
|
||||
$B cookie-import-browser
|
||||
```
|
||||
|
||||
Ask the user to choose browser, account/profile, and domains, then say when done. Never guess accounts or treat default Comet as consent. Unreadable profiles are unknown, not empty. Rerun for an expired five-minute one-use link.
|
||||
Direct import: run `$B cookie-import-browser` with the chosen browser name, one `--domain` per run and `--profile` set to the profile directory (not its display name); add `--verify-auth` or `--clear-storage` only if the user said yes in step 2.
|
||||
|
||||
Direct import: pass the chosen browser and `--domain` after navigating to a matching target. `--profile` takes a directory, not a display name; omit only for an unambiguous relevant profile, otherwise use the picker. `--all` requires consent for all non-expired profile cookies; it cannot accompany `--domain` or `--clear-storage`.
|
||||
If the import rejects because verification settings are missing, nothing changed: tell the user which variable is missing and return to step 3. Read that same root's `BROWSER.md`, **Choosing a source and checking sign-in**, for other errors and receipt failure reasons; it has the error tables, examples, profile labels, supported sources and platform setup.
|
||||
|
||||
Read that same root's `BROWSER.md`, **Choosing a source and checking sign-in**, for examples, profile labels, supported sources and platform setup.
|
||||
## 5. Report honestly
|
||||
|
||||
## 4. Report honestly
|
||||
Report from the import receipt, not from raw `$B cookies`, as three lines:
|
||||
|
||||
Report receipt/picker counts, partial/zero/error and reset outcomes, not raw `$B cookies`. Imports affect the context, not one tab. **Not checked** means no requested check; **not verified** means it failed; **verified** requires positive target evidence. Zero imports, counts, or HTTP 200 never prove login.
|
||||
- `Imported: <count> (<imported|partial|empty|failed>, as the receipt says) from <browser> for <domains>`
|
||||
- `Storage reset: <done|failed|not requested>`
|
||||
- `Sign-in: <verified|not verified|not checked>`
|
||||
|
||||
Never request/publish cookie values, passwords, identity/profile text, session details, or raw errors in public logs.
|
||||
**Not checked** means verification was not requested; **not verified** means it ran and failed; **verified** needs the identity match. Cookie counts, zero imports and HTTP 200 never prove sign-in. Imported cookies apply to the whole browser context, not one tab.
|
||||
|
||||
Never request or publish cookie values, passwords, identity or profile text, session details, or raw errors in public logs.
|
||||
|
||||
## Platform boundaries
|
||||
|
||||
|
||||
@@ -21,41 +21,59 @@ allowed-tools:
|
||||
|
||||
# Setup Browser Cookies
|
||||
|
||||
## 1. Choose the browser
|
||||
## 1. Check for Aside
|
||||
|
||||
Use this checkout as the gstack root if it contains `BROWSER.md` and `browse/SKILL.md`; otherwise use the installed root containing `bin/gstack-skill-start`, never a generated host stub. Read that root's `browse/SKILL.md` **BROWSER SETUP** section and run its probe first. On `READY`, stop importing: use Aside's sessions or ask the user to sign in there. Otherwise follow the probe's fallback handling, then continue below.
|
||||
Use this checkout as the gstack root if it contains `BROWSER.md` and `browse/SKILL.md`; otherwise use the installed root containing `bin/gstack-skill-start`, never a generated host stub. Read that root's `browse/SKILL.md` **BROWSER SETUP** section and run its Aside probe.
|
||||
|
||||
{{BROWSE_SETUP}}
|
||||
- **Aside probe prints `READY`:** no import is needed, because Aside already has the user's signed-in sessions. Tell the user to use Aside, or to sign in there, and end this skill.
|
||||
- **Any other result:** continue with gstack's own headless browser (`$B`).
|
||||
|
||||
## 2. Get every decision from the user first
|
||||
|
||||
Ask these together, before running any `$B` command. Never guess an answer or treat the default Comet browser as consent.
|
||||
|
||||
1. **Source browser and profile.** `--profile` takes the profile's directory (`Default`, `Profile 2`), not its display name. A profile that cannot be read is unknown, not empty.
|
||||
2. **Domains to import,** and the **target**: one page on a chosen domain that the session should open, by URL.
|
||||
3. **Sign-in verification (`--verify-auth`)?** Default no. It reloads the target and reports **verified** only after a successful same-origin response with exactly one visible element, found by a CSS selector, whose whitespace-normalized text equals the expected identity. If yes, the user supplies the selector and the expected identity privately.
|
||||
4. **Clear stale storage (`--clear-storage`)?** Default no; needs explicit approval. Chromium only. It clears localStorage for the target's origin (shared by that origin's tabs) and sessionStorage for the target tab, and nothing else; other browser engines reject the reset but still import. A failed reset may already have cleared some storage and is not rolled back.
|
||||
5. **Every cookie in the profile (`--all`)?** Default no; needs explicit consent. It cannot be combined with a domain or with clearing storage.
|
||||
|
||||
Use the picker when any of answers 1-2 is still open; it asks for them. Use direct import when the user has named the browser, the profile directory and the domains.
|
||||
|
||||
## 3. Prepare gstack's browser
|
||||
|
||||
1. If the user wants verification, have them set `GSTACK_COOKIE_AUTH_SELECTOR` and `GSTACK_COOKIE_AUTH_EXPECTED_IDENTITY` privately in the shell that runs `$B`. The browser daemon reads them only when it starts, so if a gstack daemon is already running, run `$B stop` first. Never invent the values.
|
||||
2. Follow `browse/SKILL.md`'s **Browser fallback** section to find `$B`, building it if the check prints `NEEDS_SETUP`.
|
||||
3. Run `$B status`. If it reports `Mode: cdp`, `$B` is attached to the user's real browser and already has its sessions: tell the user no import is needed and end this skill.
|
||||
4. Open the target URL from answer 2 and leave that tab alone until the import finishes:
|
||||
|
||||
```bash
|
||||
$B status
|
||||
$B goto "<url>"
|
||||
```
|
||||
If status says `Mode: cdp`, stop: the real browser already has sessions.
|
||||
|
||||
## 2. Confirm options before import
|
||||
## 4. Import
|
||||
|
||||
Open the known target and keep its tab unchanged. Both options default off and require explicit request:
|
||||
|
||||
- **`--verify-auth` / picker checkbox:** reloads the target. Have the user privately configure daemon `GSTACK_COOKIE_AUTH_SELECTOR` and `GSTACK_COOKIE_AUTH_EXPECTED_IDENTITY` **before startup**. Never invent values or assume CLI env reconfigures an existing daemon. Missing config rejects before mutation. Require a successful same-origin response and exactly one visible element whose whitespace-normalized text exactly matches the expected identity.
|
||||
- **`--clear-storage`:** for suspected stale storage, obtain explicit approval. Chromium only: clears captured-origin localStorage (shared across same-origin context tabs) and target-tab sessionStorage. Other origins, other tabs' sessionStorage, IndexedDB, and service workers stay intact. It uses an isolated world/native deadline; other engines reject reset, not imports/auth checks. Never auto-approve or claim rollback after partial failure.
|
||||
|
||||
## 3. Select source and scope
|
||||
Picker (the user confirms the browser, profile and domains there, ticks verification if they asked for it, then tells you when done; the link works once, for five minutes, so rerun the command if it expires):
|
||||
|
||||
```bash
|
||||
$B cookie-import-browser
|
||||
```
|
||||
|
||||
Ask the user to choose browser, account/profile, and domains, then say when done. Never guess accounts or treat default Comet as consent. Unreadable profiles are unknown, not empty. Rerun for an expired five-minute one-use link.
|
||||
Direct import: run `$B cookie-import-browser` with the chosen browser name, one `--domain` per run and `--profile` set to the profile directory (not its display name); add `--verify-auth` or `--clear-storage` only if the user said yes in step 2.
|
||||
|
||||
Direct import: pass the chosen browser and `--domain` after navigating to a matching target. `--profile` takes a directory, not a display name; omit only for an unambiguous relevant profile, otherwise use the picker. `--all` requires consent for all non-expired profile cookies; it cannot accompany `--domain` or `--clear-storage`.
|
||||
If the import rejects because verification settings are missing, nothing changed: tell the user which variable is missing and return to step 3. Read that same root's `BROWSER.md`, **Choosing a source and checking sign-in**, for other errors and receipt failure reasons; it has the error tables, examples, profile labels, supported sources and platform setup.
|
||||
|
||||
Read that same root's `BROWSER.md`, **Choosing a source and checking sign-in**, for examples, profile labels, supported sources and platform setup.
|
||||
## 5. Report honestly
|
||||
|
||||
## 4. Report honestly
|
||||
Report from the import receipt, not from raw `$B cookies`, as three lines:
|
||||
|
||||
Report receipt/picker counts, partial/zero/error and reset outcomes, not raw `$B cookies`. Imports affect the context, not one tab. **Not checked** means no requested check; **not verified** means it failed; **verified** requires positive target evidence. Zero imports, counts, or HTTP 200 never prove login.
|
||||
- `Imported: <count> (<imported|partial|empty|failed>, as the receipt says) from <browser> for <domains>`
|
||||
- `Storage reset: <done|failed|not requested>`
|
||||
- `Sign-in: <verified|not verified|not checked>`
|
||||
|
||||
Never request/publish cookie values, passwords, identity/profile text, session details, or raw errors in public logs.
|
||||
**Not checked** means verification was not requested; **not verified** means it ran and failed; **verified** needs the identity match. Cookie counts, zero imports and HTTP 200 never prove sign-in. Imported cookies apply to the whole browser context, not one tab.
|
||||
|
||||
Never request or publish cookie values, passwords, identity or profile text, session details, or raw errors in public logs.
|
||||
|
||||
## Platform boundaries
|
||||
|
||||
|
||||
@@ -180,14 +180,14 @@ TODO: consider migrating this example to a larger configuration framework.
|
||||
name: 'codex-sol-scope-termination',
|
||||
suite: 'codex-e2e-sol-scope',
|
||||
budgetMs: CAPTURE_MS,
|
||||
model: 'gpt-5.6-sol',
|
||||
model: 'gpt-6.1-sol',
|
||||
outputLimit: Infinity,
|
||||
run: (signal) => runCodexSkill({
|
||||
skillDir,
|
||||
skillName: 'gstack-investigate',
|
||||
cwd: scratch,
|
||||
sandbox: 'workspace-write',
|
||||
model: 'gpt-5.6-sol',
|
||||
model: 'gpt-6.1-sol',
|
||||
configOverrides: ['model_reasoning_effort="high"'],
|
||||
ignoreUserConfig: true,
|
||||
timeoutMs: CODEX_TIMEOUT_MS,
|
||||
|
||||
@@ -37,6 +37,8 @@ describe('Remaining Work provenance contract', () => {
|
||||
expect(table).toContain('`Open.`');
|
||||
expect(table).toContain('`[done]`');
|
||||
expect(table.indexOf('a writing step')).toBeLessThan(table.indexOf('names a concrete path'));
|
||||
expect(table).toContain('**Markers stay English literals.**');
|
||||
expect(table).toContain('**Decisions Made and file claims.**');
|
||||
const format = between(save, '### Remaining Work', '### Notes');
|
||||
expect(format).toContain('`Open.`');
|
||||
});
|
||||
@@ -49,6 +51,8 @@ describe('Remaining Work provenance contract', () => {
|
||||
for (const marker of ['(path assumed)', '(code read)']) expect(verifyFirst).toContain(`\`${marker}\``);
|
||||
expect(verifyFirst).toContain('reports a failure');
|
||||
expect(verifyFirst).toContain('has no marker');
|
||||
expect(verifyFirst).toContain('translated or unknown marker');
|
||||
expect(restore).toContain('**No useful branch discriminator.**');
|
||||
const nextSteps = between(rules, 'Every other item goes under **Next steps**', 'Verifying means');
|
||||
for (const marker of ['(path run)', '(path read)', '(target state checked)']) expect(nextSteps).toContain(`\`${marker}\``);
|
||||
expect(rules).toContain(
|
||||
|
||||
@@ -244,8 +244,8 @@ describe('cookie workflow judge input', () => {
|
||||
expect(body).toContain('Use this checkout as the gstack root if it contains `BROWSER.md` and `browse/SKILL.md`');
|
||||
expect(body).toContain('the installed root containing `bin/gstack-skill-start`, never a generated host stub');
|
||||
expect(body).toContain("Read that root's `browse/SKILL.md`");
|
||||
expect(body).toContain('On `READY`, stop importing');
|
||||
expect(body.indexOf('**BROWSER SETUP**')).toBeLessThan(body.indexOf('## SETUP'));
|
||||
expect(body).toContain('**Aside probe prints `READY`:** no import is needed');
|
||||
expect(body.indexOf('**BROWSER SETUP**')).toBeLessThan(body.indexOf('**Browser fallback**'));
|
||||
expect(body).toContain("Read that same root's `BROWSER.md`, **Choosing a source and checking sign-in**");
|
||||
expect(body).not.toContain('../BROWSER.md');
|
||||
}
|
||||
|
||||
@@ -18,8 +18,8 @@ const finding:any={title:'Cross-tenant read',rootCause:'Tenant query omits calle
|
||||
const request=(before:string,after:string)=>({findingId:'a'.repeat(32),runtimeProfile:'node',port:3456,start:{executable:'/usr/local/bin/node',args:['app.js']},legitimate:[{name:'health',path:'/health',method:'GET',expected:{status:200,includes:'ok'}}],security:{name:'tenant isolation',path:'/user?id=2',method:'GET',expected:{status:403},vulnerable:{status:200,includes:'tenant-b'}},existingTests:[{executable:'/usr/local/bin/node',args:['test.js']}],testFiles:['test.js'],fixtures:{'tenant.json':'{}'},boundaryFiles:['app.js'],changes:[{path:'app.js',beforeSha256:sha256(before),after,effect:'source'}],review:{reviewer:'independent-reviewer',independent:true,rootCauseRepaired:true,featurePreserved:true,boundaryMocks:false,rationale:'Predicate now binds the caller tenant and health behavior is preserved',reviewedPatchHash:''}});
|
||||
// Structural signed fixture for bundle-reader mutation tests. Production
|
||||
// issuance uses AssertionWitnessSession in verifyRepair instead.
|
||||
function authenticatedBundleFixture(certified:ReturnType<typeof certify>,params:any,outcomes?:Record<'before'|'after',{executedTests:number;passingTests:number}>):any{
|
||||
const verification=structuredClone(certified.manifest),identityRequest=params.identityRequest??params.request,pair=crypto.generateKeyPairSync('ed25519'),publicKey=pair.publicKey.export({format:'der',type:'spki'}).toString('hex'),keyId=sha256(Buffer.from(publicKey,'hex')),createdAt=verification.createdAt,runner={testToolchain:params.testToolchain,startPlanHash:verification.startPlanHash,testPlanHash:verification.testPlanHash,commandsHash:sha256(canonical(params.request.existingTests)),minimumPassingTestsHash:sha256(canonical(params.request.existingTests.map(()=>1)))},make=(phase:'before'|'after')=>{const observation=verification[phase],counts=outcomes?.[phase]??{executedTests:1,passingTests:1},binding:any={schemaVersion:1,protocol:'gstack-cso-assertion-witness-v1',nonce:sha256(`${phase}:${createdAt}`),phase,issuedAt:createdAt,expiresAt:new Date(Date.parse(createdAt)+300_000).toISOString(),runId:params.runId,findingId:identityRequest.findingId,policyHash:verification.policyHash,auditPolicyHash:verification.auditPolicyHash,runtime:{image:params.runtime.image,verifierImage:params.verifier.image,platform:params.runtime.platform,profile:params.runtime.id},runner,sourceHash:phase==='before'?verification.beforeSourceHash:verification.afterSourceHash,dependencyHash:phase==='before'?verification.beforeDependencies:verification.afterDependencies,configurationHash:phase==='before'?verification.beforeConfiguration:verification.afterConfiguration,requestHash:verification.requestHash,patchHash:verification.patchHash,harnessHash:verification.harnessHash,assertionHash:sha256(canonical({legitimate:identityRequest.legitimate,security:identityRequest.security})),fixturesHash:verification.fixturesHash},executions=params.request.existingTests.map((command:any)=>({commandHash:sha256(canonical(command)),exitCode:0,outputHash:sha256(`structural signed fixture ${phase}`),minimumPassingTests:1,...counts,reportedPassed:counts.passingTests>=1})),unsigned:any={schemaVersion:1,binding,keyId,publicKey,observationHash:witnessObservationHash(observation),externalAssertionsPassed:true,diagnosticTestsPassed:executions.every((execution:any)=>execution.reportedPassed),executions};return{...unsigned,signature:crypto.sign(null,Buffer.from(canonical(unsigned)),pair.privateKey).toString('hex')}};const witness={before:make('before'),after:make('after')};verification.assertionAssurance='authenticated_out_of_process';verification.testCompletionAssurance='self_reported';verification.witnessHash=assertionWitnessPairHash(witness);verification.result='runtime_tested';verification.id='';const id=verificationIdentity(verification);verification.id=id;
|
||||
function authenticatedBundleFixture(certified:ReturnType<typeof certify>,params:any,outcomes?:Record<'before'|'after',{executedTests:number;passingTests:number}>,externalAssertionsPassed:Record<'before'|'after',boolean>={before:true,after:true}):any{
|
||||
const verification=structuredClone(certified.manifest),identityRequest=params.identityRequest??params.request,pair=crypto.generateKeyPairSync('ed25519'),publicKey=pair.publicKey.export({format:'der',type:'spki'}).toString('hex'),keyId=sha256(Buffer.from(publicKey,'hex')),createdAt=verification.createdAt,runner={testToolchain:params.testToolchain,startPlanHash:verification.startPlanHash,testPlanHash:verification.testPlanHash,commandsHash:sha256(canonical(params.request.existingTests)),minimumPassingTestsHash:sha256(canonical(params.request.existingTests.map(()=>1)))},make=(phase:'before'|'after')=>{const observation=verification[phase],counts=outcomes?.[phase]??{executedTests:1,passingTests:1},binding:any={schemaVersion:1,protocol:'gstack-cso-assertion-witness-v1',nonce:sha256(`${phase}:${createdAt}`),phase,issuedAt:createdAt,expiresAt:new Date(Date.parse(createdAt)+300_000).toISOString(),runId:params.runId,findingId:identityRequest.findingId,policyHash:verification.policyHash,auditPolicyHash:verification.auditPolicyHash,runtime:{image:params.runtime.image,verifierImage:params.verifier.image,platform:params.runtime.platform,profile:params.runtime.id},runner,sourceHash:phase==='before'?verification.beforeSourceHash:verification.afterSourceHash,dependencyHash:phase==='before'?verification.beforeDependencies:verification.afterDependencies,configurationHash:phase==='before'?verification.beforeConfiguration:verification.afterConfiguration,requestHash:verification.requestHash,patchHash:verification.patchHash,harnessHash:verification.harnessHash,assertionHash:sha256(canonical({legitimate:identityRequest.legitimate,security:identityRequest.security})),fixturesHash:verification.fixturesHash},executions=params.request.existingTests.map((command:any)=>({commandHash:sha256(canonical(command)),exitCode:0,outputHash:sha256(`structural signed fixture ${phase}`),minimumPassingTests:1,...counts,reportedPassed:counts.passingTests>=1})),unsigned:any={schemaVersion:1,binding,keyId,publicKey,observationHash:witnessObservationHash(observation),externalAssertionsPassed:externalAssertionsPassed[phase],diagnosticTestsPassed:executions.every((execution:any)=>execution.reportedPassed),executions};return{...unsigned,signature:crypto.sign(null,Buffer.from(canonical(unsigned)),pair.privateKey).toString('hex')}};const witness={before:make('before'),after:make('after')};verification.assertionAssurance='authenticated_out_of_process';verification.testCompletionAssurance='self_reported';verification.witnessHash=assertionWitnessPairHash(witness);verification.result='runtime_tested';verification.id='';const id=verificationIdentity(verification);verification.id=id;
|
||||
return{schemaVersion:3,runId:params.runId,id,createdAt,expiresAt:new Date(Date.parse(createdAt)+30*86400_000).toISOString(),requiredInputs:{sourceHash:params.manifest.executionHash,originalHash:params.manifest.originalHash,runtimeImage:params.runtime.image,platform:params.runtime.platform,archives:params.archives,...(params.dependencyClosures?{dependencyClosures:params.dependencyClosures}:{})},request:identityRequest,verification,transformations:params.manifest.entries.filter((entry:any)=>entry.transformation),...(params.preparation?{preparation:params.preparation}:{}),witness};
|
||||
}
|
||||
|
||||
@@ -170,6 +170,14 @@ describe('tested repair certificate gate',()=>{
|
||||
const params:any={runId:'run',manifest:{originalHash:'d'.repeat(64),executionHash:treeHash(before),entries:[]},request:req,runtime:{id:'node',image:'runtime@sha256:'+'a'.repeat(64),platform:'linux/amd64'},verifier:{image:'runtime@sha256:'+'a'.repeat(64)},before:{booted:true,legitimate:true,security:'intended_failure',existingTests:true,output:'before',inputHash:''},after:{booted:true,legitimate:true,security:'pass',existingTests:true,output:'after',inputHash:''},beforeRoot:before,afterRoot:after,policyHash:'c'.repeat(64),auditPolicyHash:'e'.repeat(64),archives:[],testToolchain:'runtime'},unsigned=certify(params),witness=authenticatedBundleFixture(unsigned,params,{before:{executedTests:2,passingTests:2},after:{executedTests:2,passingTests:1}}).witness;
|
||||
expect(()=>certify({...params,witness})).toThrow('existing-test execution count or outcome');
|
||||
});
|
||||
test('a witness whose external assertions failed is an assertion failure, and a mismatched observation stays incompatible input',()=>{
|
||||
const root=tmp(),before=path.join(root,'before'),after=path.join(root,'after');fs.mkdirSync(before);fs.writeFileSync(path.join(before,'app.js'),'vulnerable\n');fs.writeFileSync(path.join(before,'test.js'),'one registered test\n');const raw:any=request('vulnerable\n','fixed\n');raw.review.reviewedPatchHash=patchHash(raw);const req=validateVerificationRequest(raw);preparePatchedSource(before,after,req);
|
||||
const params:any={runId:'run',manifest:{originalHash:'d'.repeat(64),executionHash:treeHash(before),entries:[]},request:req,runtime:{id:'node',image:'runtime@sha256:'+'a'.repeat(64),platform:'linux/amd64'},verifier:{image:'runtime@sha256:'+'a'.repeat(64)},before:{booted:true,legitimate:true,security:'intended_failure',existingTests:true,output:'before',inputHash:''},after:{booted:true,legitimate:true,security:'inconclusive',existingTests:true,output:'after',inputHash:''},beforeRoot:before,afterRoot:after,policyHash:'c'.repeat(64),auditPolicyHash:'e'.repeat(64),archives:[],testToolchain:'runtime'},unsigned=certify(params);
|
||||
const failed=authenticatedBundleFixture(unsigned,params,undefined,{before:true,after:false}).witness;
|
||||
expect(()=>certify({...params,witness:failed})).toThrow(expect.objectContaining({code:'ASSERTION_FAILED',message:'External assertions failed in the after phase: booted true, legitimate true, security inconclusive'}));
|
||||
const mismatched=authenticatedBundleFixture({...unsigned,manifest:{...unsigned.manifest,after:{...unsigned.manifest.after,output:'a different observation'}}},params,undefined,{before:true,after:false}).witness;
|
||||
expect(()=>certify({...params,witness:mismatched})).toThrow(expect.objectContaining({code:'INCOMPATIBLE_INPUT'}));
|
||||
});
|
||||
test('canonical Python starts use direct prepared launchers and reject root framework shadows',()=>{
|
||||
const pin=(name:string)=>`${name}==1.0.0 --hash=sha256:${'a'.repeat(64)}\n`;
|
||||
const flask=tmp();fs.writeFileSync(path.join(flask,'requirements.txt'),pin('Flask'));fs.writeFileSync(path.join(flask,'app.py'),'from flask import Flask\napp = Flask(__name__)\n');const flaskCommand=canonicalStartPlan(flask,'python',3456).command;expect(flaskCommand.executable).toBe('/work/.venv/bin/python');expect(flaskCommand.args.slice(0,3)).toEqual(['-I','-S','-c']);expect(flaskCommand.args[3]).toContain('from flask.cli import main');expect(flaskCommand.args.slice(4)).toEqual(['--app','app:app','run','--host','127.0.0.1','--port','3456']);fs.writeFileSync(path.join(flask,'flask.py'),'raise RuntimeError("shadow")\n');expect(()=>canonicalStartPlan(flask,'python',3456)).toThrow('root import shadow');
|
||||
@@ -186,6 +194,17 @@ describe('tested repair certificate gate',()=>{
|
||||
const runDir=tmp(),snapshot=path.join(runDir,'snapshot');fs.mkdirSync(snapshot);const files:Record<string,string>={'package.json':JSON.stringify({scripts:{test:'jest'}}),'package-lock.json':'{}\n','app.js':'server.listen(3456)\n','app.test.js':'test("ok",()=>{})\n'};for(const [name,body] of Object.entries(files))fs.writeFileSync(path.join(snapshot,name),body);const plan=canonicalTestPlan(snapshot,'node'),raw:any=request(files['app.js'],'server.listen(3456)\n');raw.start={executable:'/usr/local/bin/node',args:['app.js']};raw.existingTests=plan.commands;raw.testFiles=plan.files;raw.changes=[{path:'package-lock.json',beforeSha256:sha256(files['package-lock.json']),after:'{"changed":true}\n',effect:'dependency'}];raw.review.reviewedPatchHash=patchHash(raw);const entries=Object.entries(files).map(([path,body])=>({path,originalHash:sha256(body),executionHash:sha256(body),bytes:Buffer.byteLength(body),mode:0o600})),manifest:any={version:3,createdAt:'2026-01-01T00:00:00Z',expiresAt:'2026-01-08T00:00:00Z',root:'/repo',headCommit:'a'.repeat(40),originalHash:'b'.repeat(64),executionHash:treeHash(snapshot),entries};let executed=false;await expect(verifyRepair({runId:'run',runDir,manifest,rawRequest:raw,runtime:{id:'node',stack:'node',image:'runtime@sha256:'+'a'.repeat(64),platform:'linux/amd64'} as any,verifier:{id:'node',stack:'node',image:'runtime@sha256:'+'a'.repeat(64),platform:'linux/amd64'} as any,policyHash:'c'.repeat(64),archives:[],executor:{observe:async()=>{executed=true;throw new Error('must not run');}},persist:false})).rejects.toMatchObject({code:'PREREQUISITE'});expect(executed).toBe(false);
|
||||
});
|
||||
test('overlapping security oracles and no-op repairs cannot be certified',()=>{const overlapping:any=request('vulnerable\n','fixed\n');overlapping.review.reviewedPatchHash='pending';overlapping.security.vulnerable={...overlapping.security.expected};expect(()=>validateVerificationRequest(overlapping)).toThrow('mutually exclusive');const noop:any=request('vulnerable\n','vulnerable\n');noop.review.reviewedPatchHash='pending';expect(()=>validateVerificationRequest(noop)).toThrow('material patch');});
|
||||
test('a security expectation may list several safe statuses only while it excludes the vulnerable marker',()=>{
|
||||
const make=(edit:(raw:any)=>void)=>{const raw:any=request('vulnerable\n','fixed\n');raw.review.reviewedPatchHash='pending';edit(raw);return raw;};
|
||||
expect(validateVerificationRequest(make(raw=>{raw.security.expected={status:[403,404],excludes:'tenant-b'};})).security).toMatchObject({expected:{status:[403,404],excludes:'tenant-b'}});
|
||||
expect(validateVerificationRequest(make(raw=>{raw.security.expected={status:[200,403],excludes:'tenant-b'};})).security).toMatchObject({expected:{status:[200,403]}});
|
||||
expect(()=>validateVerificationRequest(make(raw=>{raw.security.expected={status:[200,403],excludes:'unrelated'};}))).toThrow('mutually exclusive');
|
||||
expect(()=>validateVerificationRequest(make(raw=>{raw.security.expected={status:[403,404]};}))).toThrow('must exclude the vulnerable marker');
|
||||
for(const status of [[],[403,403],[99,403],[400,401,403,404,405,409,410,422,429]])expect(()=>validateVerificationRequest(make(raw=>{raw.security.expected={status,excludes:'tenant-b'};}))).toThrow('distinct HTTP statuses');
|
||||
expect(()=>validateVerificationRequest(make(raw=>{raw.security.vulnerable={status:[200,206],includes:'tenant-b'};}))).toThrow('security.vulnerable.status must be an HTTP status');
|
||||
expect(()=>validateVerificationRequest(make(raw=>{raw.legitimate[0].expected={status:[200,403],excludes:'denied'};}))).toThrow('legitimate[0].expected.status must be an HTTP status');
|
||||
expect(()=>validateVerificationRequest(make(raw=>{raw.security=[{...raw.security,expected:{status:[403,404],excludes:'tenant-b'}},{...raw.security,name:'second',expected:{status:[403,404]}}];}))).toThrow('security[1].expected with a set of safe statuses');
|
||||
});
|
||||
test('review is a separate content-bound but explicitly self-attested artifact',()=>{const raw:any=request('vulnerable\n','fixed\n');raw.review.reviewedPatchHash=patchHash(raw);const req=validateVerificationRequest(raw),artifact=makeReviewArtifact('run',req,'producer-session');raw.review.artifactId=artifact.id;const bound=validateVerificationRequest(raw);expect(validateReviewArtifact(artifact,'run',bound)).toMatchObject({producer:'producer-session',assurance:'self_attested'});expect(()=>makeReviewArtifact('run',req,req.review.reviewer)).toThrow('distinct');expect(()=>validateReviewArtifact({...artifact,rationale:'changed'},'run',bound)).toThrow('does not bind');});
|
||||
test('executor observations have a strict helper-owned shape',()=>{expect(()=>validateVerificationObservation({booted:true,legitimate:true,security:'pass',existingTests:true,output:'ok',inputHash:'',tested:true})).toThrow('Unexpected');expect(()=>validateVerificationObservation({booted:'yes',legitimate:true,security:'pass',existingTests:true,output:'ok',inputHash:''})).toThrow('booleans');});
|
||||
test('secret-bearing fixtures cannot enter a replayable verification bundle',async()=>{const runDir=tmp(),snapshot=path.join(runDir,'snapshot');fs.mkdirSync(snapshot);fs.writeFileSync(path.join(snapshot,'app.js'),'vulnerable\n');const raw:any=request('vulnerable\n','fixed\n');raw.review.reviewedPatchHash=patchHash(raw);raw.fixtures={'secret.txt':CREDENTIAL_CANARY};const manifest:any={entries:[{path:'app.js',transformation:undefined}]};const runtime:any={id:'node',image:'runtime@sha256:'+'a'.repeat(64),platform:'linux/amd64'};expect(verifyRepair({runId:'run',runDir,manifest,rawRequest:raw,runtime,verifier:runtime,policyHash:'c'.repeat(64),archives:[],executor:{observe:async()=>{throw new Error('must not execute');}},persist:false})).rejects.toMatchObject({code:'REDACTION_FAILED'});});
|
||||
|
||||
@@ -92,6 +92,20 @@ describe('CSO Git metadata hardening',()=>{
|
||||
expect(history).not.toContain('replacement-history');
|
||||
});
|
||||
|
||||
test.skipIf(process.platform==='win32')('retained history withholds diffs of credential files the snapshot excludes',async()=>{
|
||||
const {repo,runDir,git}=fixture();
|
||||
fs.writeFileSync(path.join(repo,'.env'),'UPSTREAM_TOKEN=planted-history-value\n');fs.writeFileSync(path.join(repo,'tracked.ts'),'export const tracked = false\n');
|
||||
git('add','.env','tracked.ts');git('commit','-qm','configure upstream');
|
||||
|
||||
await capture(repo,runDir);
|
||||
const history=fs.readFileSync(path.join(runDir,'history.txt'),'utf8');
|
||||
|
||||
expect(history).not.toContain('planted-history-value');
|
||||
expect(history).toContain('diff withheld: .env (credential or execution configuration)');
|
||||
expect(history).toContain('+export const tracked = false');
|
||||
expect(history).toContain('Subject: configure upstream');
|
||||
});
|
||||
|
||||
test.skipIf(process.platform==='win32')('rejects repository config includes before Git can consume them',async()=>{
|
||||
const {root,repo,runDir}=fixture(),included=path.join(root,'included.conf');
|
||||
fs.writeFileSync(included,'[core]\n\tworktree = /tmp/cso-decoy\n');
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe,expect,test } from 'bun:test';
|
||||
import { gitDiffHeaderPaths,historyForPath } from '../lib/cso/history';
|
||||
import { gitDiffHeaderPaths,historyForPath,withholdHistory } from '../lib/cso/history';
|
||||
|
||||
describe('CSO retained Git history path filtering',()=>{
|
||||
test('matches exact paths without leaking a prefix sibling hunk',()=>{
|
||||
@@ -25,6 +25,38 @@ diff --git a/foo-extra b/foo-extra
|
||||
expect(historyForPath(raw,'café file.ts')).toContain('+unicode-only');
|
||||
});
|
||||
|
||||
test('withholds every diff section touching an excluded path, keeping commit preambles and other files',()=>{
|
||||
const raw=`commit ${'c'.repeat(40)}
|
||||
Subject: add config
|
||||
diff --git a/.env b/.env
|
||||
--- /dev/null
|
||||
+++ b/.env
|
||||
+PLANTED_VALUE=excluded-file-body
|
||||
diff --git a/app.ts b/app.ts
|
||||
--- a/app.ts
|
||||
+++ b/app.ts
|
||||
+kept-line
|
||||
commit ${'d'.repeat(40)}
|
||||
Subject: rename
|
||||
diff --git a/config/old.txt b/config/master.key
|
||||
rename from config/old.txt
|
||||
rename to config/master.key
|
||||
+renamed-into-excluded
|
||||
diff --git "a/foo b/foo
|
||||
+unparseable-body
|
||||
`;
|
||||
const kept=withholdHistory(raw,(path)=>/(^|\/)(\.env|master\.key)$/.test(path)?'credential or execution configuration':undefined);
|
||||
expect(kept).not.toContain('excluded-file-body');
|
||||
expect(kept).not.toContain('renamed-into-excluded');
|
||||
expect(kept).not.toContain('unparseable-body');
|
||||
expect(kept).toContain('diff withheld: .env (credential or execution configuration)');
|
||||
expect(kept).toContain('diff withheld: config/master.key (credential or execution configuration)');
|
||||
expect(kept).toContain('diff withheld: unknown path (unparseable diff header)');
|
||||
expect(kept).toContain('+kept-line');
|
||||
expect(kept).toContain('Subject: add config');
|
||||
expect(kept).toContain('Subject: rename');
|
||||
});
|
||||
|
||||
test('rejects malformed quoted headers instead of broadening the match',()=>{
|
||||
expect(gitDiffHeaderPaths('diff --git "a/foo b/foo')).toBeUndefined();
|
||||
expect(historyForPath('diff --git "a/foo b/foo\n+secret\n','foo')).toBeUndefined();
|
||||
|
||||
@@ -0,0 +1,299 @@
|
||||
// Regressions for /cso on ordinary user repositories: an untracked nested Git
|
||||
// repository (#3255 bug 1), a patch history over the 1 MiB bound (#3255 bug 2),
|
||||
// and a Docker socket that realpath refuses to open (#3248).
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, spyOn, test } from 'bun:test';
|
||||
import * as fs from 'node:fs';
|
||||
import * as net from 'node:net';
|
||||
import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { NESTED_REPOSITORY, capture, historyMarker } from '../lib/cso/snapshot';
|
||||
import { newRun, readJson } from '../lib/cso/state';
|
||||
import { MAX_OUTPUT } from '../lib/cso/contracts';
|
||||
import { dockerEndpoint } from '../lib/cso/docker';
|
||||
import { TRUSTED_DIRECTORIES } from '../lib/cso/process';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
const launcher = path.join(ROOT, 'bin', process.platform === 'win32' ? 'gstack-cso-launcher.exe' : 'gstack-cso-launcher');
|
||||
const MIB = 1024 * 1024;
|
||||
const CREDENTIAL_CANARY = ['ghp_', 'abcdefghijklmnopqrstuvwxyz1234567890'].join('');
|
||||
const OLD_CANARY = ['ghp_', 'zyxwvutsrqponmlkjihgfedcba0987654321'].join('');
|
||||
const MARKER = '[gstack-cso history truncated:';
|
||||
const roots: string[] = [];
|
||||
const originalHome = process.env.GSTACK_HOME;
|
||||
let state = '';
|
||||
beforeAll(() => {
|
||||
state = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'cso-ordinary-state-')));
|
||||
process.env.GSTACK_HOME = state;
|
||||
});
|
||||
afterEach(() => {
|
||||
for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
afterAll(() => {
|
||||
if (originalHome === undefined) delete process.env.GSTACK_HOME;
|
||||
else process.env.GSTACK_HOME = originalHome;
|
||||
fs.rmSync(state, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function tmp(): string {
|
||||
const root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'cso-ordinary-')));
|
||||
roots.push(root);
|
||||
return root;
|
||||
}
|
||||
function git(repo: string, ...args: string[]): string {
|
||||
const result = spawnSync('/usr/bin/git', ['-C', repo, ...args], { encoding: 'utf8', env: { HOME: path.dirname(repo), PATH: '/usr/bin:/bin' }, timeout: 60_000 });
|
||||
if (result.status) throw new Error(result.stderr);
|
||||
return result.stdout;
|
||||
}
|
||||
function init(repo: string): string {
|
||||
fs.mkdirSync(repo, { recursive: true });
|
||||
git(repo, 'init', '-q');
|
||||
git(repo, 'config', 'user.email', 'fixture@example.test');
|
||||
git(repo, 'config', 'user.name', 'Fixture');
|
||||
return repo;
|
||||
}
|
||||
function commit(repo: string, files: Record<string, string>, subject: string): void {
|
||||
for (const [file, body] of Object.entries(files)) fs.writeFileSync(path.join(repo, file), body);
|
||||
git(repo, 'add', '-A');
|
||||
git(repo, 'commit', '-qm', subject);
|
||||
}
|
||||
function filler(tag: string, bytes: number): string {
|
||||
const lines: string[] = [];
|
||||
for (let i = 0, size = 0; size < bytes; i++) {
|
||||
const line = `export const ${tag}${i} = ${i};`;
|
||||
lines.push(line);
|
||||
size += line.length + 1;
|
||||
}
|
||||
return lines.join('\n') + '\n';
|
||||
}
|
||||
function nestedRepository(outer: string, name: string, files: Record<string, string>): string {
|
||||
const inner = init(path.join(outer, name));
|
||||
commit(inner, files, 'inner');
|
||||
return inner;
|
||||
}
|
||||
function cso(repo: string, args: string[]) {
|
||||
return spawnSync(launcher, args, { cwd: repo, encoding: 'utf8', env: { HOME: path.dirname(repo), GSTACK_HOME: state, PATH: '/usr/bin:/bin' }, timeout: 120_000, maxBuffer: 64 * MIB });
|
||||
}
|
||||
function runFiles(dir: string): string[] {
|
||||
return fs.readdirSync(dir, { recursive: true, withFileTypes: true }).filter((item) => item.isFile()).map((item) => path.join(item.parentPath, item.name));
|
||||
}
|
||||
|
||||
describe('CSO start on a repository with an untracked nested repository (#3255 bug 1)', () => {
|
||||
test('start succeeds, records the boundary as an explicit exclusion, and never reads the nested source', () => {
|
||||
const repo = init(path.join(tmp(), 'repo'));
|
||||
commit(repo, { 'README.md': 'outer\n' }, 'outer');
|
||||
nestedRepository(repo, 'inner', { 'secret.txt': `TOKEN=${CREDENTIAL_CANARY}\n` });
|
||||
const started = cso(repo, ['start', '--repo', repo, '--scope', 'auth', '--offline']);
|
||||
expect(started.stderr).toBe('');
|
||||
expect(started.status).toBe(0);
|
||||
const run = JSON.parse(started.stdout),
|
||||
dir = path.join(state, 'security', 'cso', run.repoId, run.runId),
|
||||
report = readJson(path.join(dir, 'report.json')),
|
||||
snapshot = report.coverage.find((item: any) => item.domain === 'snapshot-inputs');
|
||||
expect(snapshot.exclusions).toEqual([`inner: ${NESTED_REPOSITORY}`]);
|
||||
expect(snapshot.gaps).toEqual([]);
|
||||
for (const file of runFiles(dir)) expect(fs.readFileSync(file, 'utf8')).not.toContain(CREDENTIAL_CANARY);
|
||||
expect(fs.existsSync(path.join(dir, 'snapshot', 'inner'))).toBe(false);
|
||||
expect(fs.existsSync(path.join(dir, 'readable', 'inner'))).toBe(false);
|
||||
});
|
||||
|
||||
test('negative control: an untracked symlink to a directory is still refused, not treated as a boundary', () => {
|
||||
const root = tmp(),
|
||||
repo = init(path.join(root, 'repo')),
|
||||
outside = path.join(root, 'outside');
|
||||
commit(repo, { 'README.md': 'outer\n' }, 'outer');
|
||||
nestedRepository(repo, 'inner', { 'f.txt': 'x\n' });
|
||||
fs.mkdirSync(outside);
|
||||
fs.writeFileSync(path.join(outside, 'host.txt'), 'host\n');
|
||||
fs.symlinkSync(outside, path.join(repo, 'escape'));
|
||||
const started = cso(repo, ['start', '--repo', repo, '--scope', 'auth', '--offline']);
|
||||
expect(started.status).not.toBe(0);
|
||||
expect(JSON.parse(started.stderr).error).toMatchObject({ code: 'UNSAFE_PATH', message: expect.stringContaining('escape') });
|
||||
});
|
||||
|
||||
test('negative control: a nested repository swapped for another directory during capture fails as a snapshot race', async () => {
|
||||
const root = tmp(),
|
||||
repo = init(path.join(root, 'repo')),
|
||||
target = path.join(repo, 'vendor', 'inner'),
|
||||
replacement = path.join(root, 'replacement');
|
||||
commit(repo, { 'README.md': 'outer\n' }, 'outer');
|
||||
fs.mkdirSync(path.join(repo, 'vendor'));
|
||||
nestedRepository(repo, 'vendor/inner', { 'f.txt': 'x\n' });
|
||||
nestedRepository(root, 'replacement', { 'f.txt': 'y\n' });
|
||||
const original = fs.lstatSync;
|
||||
let hits = 0;
|
||||
const counting = spyOn(fs, 'lstatSync').mockImplementation(((candidate: any, options?: any) => {
|
||||
if (path.resolve(String(candidate)) === target) hits++;
|
||||
return options === undefined ? original(candidate) : original(candidate, options);
|
||||
}) as typeof fs.lstatSync);
|
||||
try {
|
||||
await capture(repo, newRun(repo).dir, 'HEAD');
|
||||
} finally {
|
||||
counting.mockRestore();
|
||||
}
|
||||
const last = hits;
|
||||
expect(last).toBeGreaterThan(1);
|
||||
hits = 0;
|
||||
let swapped = false;
|
||||
const swapping = spyOn(fs, 'lstatSync').mockImplementation(((candidate: any, options?: any) => {
|
||||
if (path.resolve(String(candidate)) === target && ++hits === last) {
|
||||
fs.renameSync(target, `${target}.parked`);
|
||||
fs.renameSync(replacement, target);
|
||||
swapped = true;
|
||||
}
|
||||
return options === undefined ? original(candidate) : original(candidate, options);
|
||||
}) as typeof fs.lstatSync);
|
||||
try {
|
||||
await expect(capture(repo, newRun(repo).dir, 'HEAD')).rejects.toMatchObject({ code: 'SNAPSHOT_RACE', message: 'Source changed before snapshot persistence: vendor/inner' });
|
||||
} finally {
|
||||
swapping.mockRestore();
|
||||
}
|
||||
expect(swapped).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('CSO history capture over the 1 MiB bound (#3255 bug 2)', () => {
|
||||
test('keeps the newest whole commits that fit, marks the truncation, and still redacts', async () => {
|
||||
const repo = init(path.join(tmp(), 'repo'));
|
||||
commit(repo, { 'old.ts': filler('old', 450 * 1024) + `// ${OLD_CANARY}\n` }, 'oldest');
|
||||
commit(repo, { 'mid.ts': filler('mid', 450 * 1024) }, 'middle');
|
||||
commit(repo, { 'new.ts': filler('new', 300 * 1024) + `// ${CREDENTIAL_CANARY}\n` }, 'newest');
|
||||
const run = newRun(repo);
|
||||
await capture(repo, run.dir);
|
||||
const status = readJson(path.join(run.dir, 'history-status.json')),
|
||||
history = fs.readFileSync(path.join(run.dir, 'history.txt'), 'utf8');
|
||||
expect(status).toMatchObject({ status: 'captured', commits: 2 });
|
||||
expect(status.truncated).toBe(`the newest 2 of 3 commits were retained with patches; 1 older commits were omitted to stay within the ${MAX_OUTPUT}-byte history limit`);
|
||||
expect(history.startsWith(historyMarker(status.truncated))).toBe(true);
|
||||
expect(Buffer.byteLength(history)).toBeLessThanOrEqual(MAX_OUTPUT);
|
||||
expect(status.bytes).toBe(Buffer.byteLength(history));
|
||||
expect(history).toContain('Subject: newest');
|
||||
expect(history).toContain('Subject: middle');
|
||||
expect(history).not.toContain('Subject: oldest');
|
||||
expect(history).toContain('diff --git a/new.ts b/new.ts');
|
||||
expect(history).not.toContain(CREDENTIAL_CANARY);
|
||||
expect(history).not.toContain(OLD_CANARY);
|
||||
});
|
||||
|
||||
test('falls back to commit metadata when the newest patch alone exceeds the bound, and still redacts', async () => {
|
||||
const repo = init(path.join(tmp(), 'repo'));
|
||||
commit(repo, { 'a.ts': 'export const a = 1;\n' }, 'first');
|
||||
commit(repo, { 'big.ts': filler('big', 1.2 * MIB) }, `rotate ${CREDENTIAL_CANARY}`);
|
||||
const run = newRun(repo);
|
||||
await capture(repo, run.dir);
|
||||
const status = readJson(path.join(run.dir, 'history-status.json')),
|
||||
history = fs.readFileSync(path.join(run.dir, 'history.txt'), 'utf8');
|
||||
expect(status).toMatchObject({ status: 'captured', commits: 'metadata for at most 100' });
|
||||
expect(status.truncated).toMatch(/^patches were withheld because/);
|
||||
expect(history.startsWith(historyMarker(status.truncated))).toBe(true);
|
||||
expect(history).toContain('Subject: first');
|
||||
expect(history).toContain('Subject: rotate');
|
||||
expect(history).not.toContain('diff --git');
|
||||
expect(history).not.toContain(CREDENTIAL_CANARY);
|
||||
});
|
||||
|
||||
test('a history within the bound is retained whole with no marker', async () => {
|
||||
const repo = init(path.join(tmp(), 'repo'));
|
||||
commit(repo, { 'a.ts': 'export const a = 1;\n' }, 'first');
|
||||
commit(repo, { 'b.ts': `// ${CREDENTIAL_CANARY}\n` }, 'second');
|
||||
const run = newRun(repo);
|
||||
await capture(repo, run.dir);
|
||||
const status = readJson(path.join(run.dir, 'history-status.json')),
|
||||
history = fs.readFileSync(path.join(run.dir, 'history.txt'), 'utf8');
|
||||
expect(status).toEqual({ status: 'captured', range: status.range, commits: 'at most 100', bytes: Buffer.byteLength(history) });
|
||||
expect(history.startsWith('commit ')).toBe(true);
|
||||
expect(history).toContain('Subject: first');
|
||||
expect(history).not.toContain(MARKER);
|
||||
expect(history).not.toContain(CREDENTIAL_CANARY);
|
||||
});
|
||||
|
||||
test('start reports the truncated history as partial coverage and history stays readable', () => {
|
||||
const repo = init(path.join(tmp(), 'repo'));
|
||||
commit(repo, { 'old.ts': filler('old', 700 * 1024) }, 'oldest');
|
||||
commit(repo, { 'new.ts': filler('new', 700 * 1024) }, 'newest');
|
||||
const started = cso(repo, ['start', '--repo', repo, '--scope', 'auth', '--offline']);
|
||||
expect(started.stderr).toBe('');
|
||||
expect(started.status).toBe(0);
|
||||
const run = JSON.parse(started.stdout),
|
||||
report = readJson(path.join(state, 'security', 'cso', run.repoId, run.runId, 'report.json')),
|
||||
coverage = report.coverage.find((item: any) => item.domain === 'history-inputs');
|
||||
expect(coverage.status).toBe('partial');
|
||||
expect(coverage.gaps).toEqual([`History truncated: the newest 1 of 2 commits were retained with patches; 1 older commits were omitted to stay within the ${MAX_OUTPUT}-byte history limit`]);
|
||||
expect(coverage.evidence[0]).toStartWith('Captured 1 from ');
|
||||
const whole = cso(repo, ['history', run.runId]);
|
||||
expect(whole.status).toBe(0);
|
||||
expect(whole.stdout.startsWith(MARKER)).toBe(true);
|
||||
const scoped = cso(repo, ['history', run.runId, 'new.ts']);
|
||||
expect(scoped.status).toBe(0);
|
||||
expect(scoped.stdout.startsWith('commit ')).toBe(true);
|
||||
expect(scoped.stdout).toContain('diff --git a/new.ts b/new.ts');
|
||||
});
|
||||
});
|
||||
|
||||
const dockerInstalled = TRUSTED_DIRECTORIES.some((directory) => fs.existsSync(path.join(directory, 'docker')));
|
||||
describe.skipIf(process.platform === 'win32' || !dockerInstalled)('CSO Docker endpoint admission when realpath refuses sockets (#3248)', () => {
|
||||
let root = '';
|
||||
let socketPath = '';
|
||||
let server: net.Server;
|
||||
const realpath = fs.realpathSync;
|
||||
const refusing = ((candidate: fs.PathLike, ...rest: any[]) => {
|
||||
let socket = false;
|
||||
try {
|
||||
socket = fs.statSync(candidate).isSocket();
|
||||
} catch {}
|
||||
if (socket) {
|
||||
const error = new Error('EOPNOTSUPP: operation not supported on socket') as NodeJS.ErrnoException;
|
||||
error.code = 'EOPNOTSUPP';
|
||||
throw error;
|
||||
}
|
||||
return realpath(candidate, ...rest);
|
||||
}) as typeof fs.realpathSync;
|
||||
beforeAll(async () => {
|
||||
root = realpath(fs.mkdtempSync(path.join(os.tmpdir(), 'cso-endpoint-')));
|
||||
socketPath = path.join(root, 'docker.sock');
|
||||
server = net.createServer();
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
server.once('error', reject);
|
||||
server.listen(socketPath, () => resolve());
|
||||
});
|
||||
});
|
||||
afterAll(async () => {
|
||||
await new Promise<void>((resolve) => server.close(() => resolve()));
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
afterEach(() => {
|
||||
(fs.realpathSync as any).mockRestore?.();
|
||||
});
|
||||
const admit = (host: string) => dockerEndpoint(root, { HOME: root, DOCKER_HOST: host });
|
||||
|
||||
test('admits a socket reached through a symlinked directory and a symlinked leaf, pinned to its identity', async () => {
|
||||
const linkedDirectory = path.join(root, 'run'),
|
||||
leaf = path.join(root, 'colima.sock');
|
||||
fs.symlinkSync(root, linkedDirectory);
|
||||
fs.symlinkSync('docker.sock', leaf);
|
||||
const stub = spyOn(fs, 'realpathSync').mockImplementation(refusing);
|
||||
const identity = fs.statSync(socketPath);
|
||||
for (const host of [`unix://${socketPath}`, `unix://${path.join(linkedDirectory, 'colima.sock')}`]) {
|
||||
const endpoint = await admit(host);
|
||||
expect(endpoint).toMatchObject({ socket: socketPath, uri: `unix://${socketPath}`, device: identity.dev, inode: identity.ino });
|
||||
}
|
||||
expect(stub.mock.calls.some(([candidate]) => String(candidate) === socketPath)).toBe(true);
|
||||
});
|
||||
|
||||
test('negative controls: non-sockets, dangling or looping links, and non-local URIs are still refused', async () => {
|
||||
spyOn(fs, 'realpathSync').mockImplementation(refusing);
|
||||
const file = path.join(root, 'plain.sock'),
|
||||
dangling = path.join(root, 'dangling.sock'),
|
||||
loopA = path.join(root, 'loop-a.sock'),
|
||||
loopB = path.join(root, 'loop-b.sock');
|
||||
fs.writeFileSync(file, '');
|
||||
fs.symlinkSync(path.join(root, 'missing.sock'), dangling);
|
||||
fs.symlinkSync(loopB, loopA);
|
||||
fs.symlinkSync(loopA, loopB);
|
||||
await expect(admit(`unix://${file}`)).rejects.toMatchObject({ code: 'ISOLATION_FAILED', message: 'Docker endpoint is not a local Unix socket' });
|
||||
for (const candidate of [dangling, loopA])
|
||||
await expect(admit(`unix://${candidate}`)).rejects.toMatchObject({ code: 'ISOLATION_FAILED', message: 'Pinned local Docker socket is unavailable' });
|
||||
await expect(admit(`unix://${root}/../${path.basename(root)}/docker.sock`)).rejects.toMatchObject({ code: 'ISOLATION_FAILED' });
|
||||
await expect(admit('tcp://127.0.0.1:2375')).rejects.toMatchObject({ code: 'ISOLATION_FAILED' });
|
||||
});
|
||||
});
|
||||
@@ -3,8 +3,9 @@ import * as fs from 'node:fs';
|
||||
import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { createPreparedExport, recordNpmArchiveEntry } from '../lib/cso/preparation-container';
|
||||
import { materializePreparedExport } from '../lib/cso/preparation-docker';
|
||||
import { materializePreparedExport, writeOfflineTransformations } from '../lib/cso/preparation-docker';
|
||||
import { heldApplicationReady, validateSingleContainerProcessOutput } from '../lib/cso/docker';
|
||||
|
||||
const roots: string[] = [];
|
||||
@@ -68,3 +69,28 @@ describe('CSO preparation container archive limits', () => {
|
||||
expect(heldApplicationReady('PID COMMAND\n1 sh\n')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('offline transformations in an execution copy', () => {
|
||||
test.skipIf(process.platform === 'win32')('replace a read-only copied file under a read-only directory instead of writing through it', () => {
|
||||
const source = temporary(), config = path.join(source, 'config'), database = path.join(config, 'database.yml'), outside = path.join(temporary(), 'outside.yml');
|
||||
fs.mkdirSync(config);
|
||||
fs.writeFileSync(database, 'production: sealed\n');
|
||||
fs.chmodSync(database, 0o444);
|
||||
fs.chmodSync(config, 0o555);
|
||||
fs.writeFileSync(outside, 'untouched\n');
|
||||
const content = 'test:\n adapter: sqlite3\n', sha256 = createHash('sha256').update(content).digest('hex');
|
||||
writeOfflineTransformations(source, [
|
||||
{ path: 'config/database.yml', content, sha256, reason: 'helper test configuration' },
|
||||
{ path: 'config/initializers/zzzz_cso_test.rb', content, sha256, reason: 'helper test configuration' },
|
||||
]);
|
||||
expect(fs.readFileSync(database, 'utf8')).toBe(content);
|
||||
expect(fs.statSync(database).mode & 0o777).toBe(0o600);
|
||||
expect(fs.readFileSync(path.join(config, 'initializers', 'zzzz_cso_test.rb'), 'utf8')).toBe(content);
|
||||
expect(() => writeOfflineTransformations(source, [{ path: 'config/database.yml', content, sha256: 'f'.repeat(64), reason: 'tampered' }])).toThrow('Offline transformation content hash changed');
|
||||
fs.rmSync(database);
|
||||
fs.symlinkSync(outside, database);
|
||||
writeOfflineTransformations(source, [{ path: 'config/database.yml', content, sha256, reason: 'helper test configuration' }]);
|
||||
expect(fs.lstatSync(database).isSymbolicLink()).toBe(false);
|
||||
expect(fs.readFileSync(outside, 'utf8')).toBe('untouched\n');
|
||||
});
|
||||
});
|
||||
@@ -1,6 +1,6 @@
|
||||
import { afterAll, afterEach, describe, expect, spyOn, test } from 'bun:test';
|
||||
import * as fs from 'node:fs';import * as os from 'node:os';import * as path from 'node:path';import { spawn, spawnSync } from 'node:child_process';
|
||||
import { assertSnapshot, capture } from '../lib/cso/snapshot';import { CsoError, snapshotPathHandle } from '../lib/cso/contracts';import { runProcess, sanitizeForJson, sanitizeHelperForJson } from '../lib/cso/process';import { discardAtomicNoReplaceTemp, finalizeReplayTemporary,reclaimDeadAttemptScratch, loadReport, newRun, privateRoot, readJson, recoverAtomicNoReplaceJson, retention, saveReport, secureDirectory, stateRoot, withLock, writeHelperJson, writeJson } from '../lib/cso/state';
|
||||
import { assertSnapshot, capture, NESTED_REPOSITORY } from '../lib/cso/snapshot';import { CsoError, snapshotPathHandle } from '../lib/cso/contracts';import { runProcess, sanitizeForJson, sanitizeHelperForJson } from '../lib/cso/process';import { discardAtomicNoReplaceTemp, finalizeReplayTemporary,reclaimDeadAttemptScratch, loadReport, newRun, privateRoot, readJson, recoverAtomicNoReplaceJson, retention, saveReport, secureDirectory, stateRoot, withLock, writeHelperJson, writeJson } from '../lib/cso/state';
|
||||
const roots:string[]=[];const tmp=()=>{const p=fs.mkdtempSync(path.join(os.tmpdir(),'cso-snapshot-'));roots.push(p);return p;};afterEach(()=>{for(const p of roots.splice(0))fs.rmSync(p,{recursive:true,force:true});});
|
||||
const originalState=process.env.GSTACK_HOME,state=fs.mkdtempSync(path.join(os.tmpdir(),'cso-state-'));process.env.GSTACK_HOME=state;afterAll(()=>{if(originalState===undefined)delete process.env.GSTACK_HOME;else process.env.GSTACK_HOME=originalState;fs.rmSync(state,{recursive:true,force:true});});
|
||||
function git(repo:string,...args:string[]){const r=spawnSync('/usr/bin/git',['-C',repo,...args],{encoding:'utf8',env:{PATH:'/usr/bin:/bin',HOME:repo},timeout:30_000});if(r.status)throw new Error(r.stderr);return r.stdout;}
|
||||
@@ -19,6 +19,8 @@ const unlink=fs.unlinkSync.bind(fs);spyOn(fsModule,'unlinkSync').mockImplementat
|
||||
const {withLock}=await import(${JSON.stringify(path.resolve(import.meta.dir,'../lib/cso/state.ts'))});let outcome='success';try{withLock(dir,()=>{const active=path.join(barrier,'work-active');try{write(active,id,{flag:'wx'});}catch{fs.appendFileSync(path.join(barrier,'overlap'),'overlap\\n');}mark('entered-'+id);if(process.env.RACE_HOLD==='1')waitFor('release-'+id);try{if(fs.readFileSync(active,'utf8')===id)fs.unlinkSync(active);}catch{}});}catch(error){outcome=String(error?.code??error);}write(path.join(barrier,'result-'+id),outcome);`;}
|
||||
describe('CSO dirty snapshot boundary',()=>{
|
||||
test('captures tracked modifications and nonignored untracked source without changing Git state',async()=>{const p=repo();fs.writeFileSync(path.join(p,'tracked.txt'),'dirty\n');fs.writeFileSync(path.join(p,'new.txt'),'untracked\n');fs.writeFileSync(path.join(p,'.gitignore'),'ignored.txt\n');fs.writeFileSync(path.join(p,'ignored.txt'),'ignored\n');const before=git(p,'status','--porcelain=v1','-z'),run=newRun(p),m=await capture(p,run.dir,'HEAD');expect(fs.readFileSync(path.join(run.dir,'snapshot','tracked.txt'),'utf8')).toBe('dirty\n');expect(fs.readFileSync(path.join(run.dir,'snapshot','new.txt'),'utf8')).toBe('untracked\n');expect(fs.existsSync(path.join(run.dir,'snapshot','ignored.txt'))).toBe(false);expect(git(p,'status','--porcelain=v1','-z')).toBe(before);expect(m.changedPaths).toContain('new.txt');});
|
||||
test('records an untracked nested repository as one boundary entry instead of failing the run',async()=>{const p=repo(),inner=path.join(p,'inner');fs.mkdirSync(inner);git(inner,'init','-q');git(inner,'config','user.email','fixture@example.test');git(inner,'config','user.name','Fixture');fs.writeFileSync(path.join(inner,'f.txt'),'x\n');git(inner,'add','.');git(inner,'commit','-qm','inner');const run=newRun(p),manifest=await capture(p,run.dir,'HEAD');assertSnapshot(run.dir,manifest);const entry=manifest.entries.find(e=>e.path==='inner');expect(entry?.transformation).toBe(NESTED_REPOSITORY);expect(entry?.originalHash).toBe('not-read');expect(manifest.entries.map(e=>e.path).filter(p=>p==='tracked.txt'||p.startsWith('inner'))).toEqual(['inner','tracked.txt']);expect(manifest.changedPaths).toContain('inner');expect(fs.existsSync(path.join(run.dir,'snapshot','inner'))).toBe(false);expect(fs.existsSync(path.join(run.dir,'readable','inner'))).toBe(false);});
|
||||
test('records a registered submodule gitlink the same way as an untracked nested repository',async()=>{const p=repo(),src=tmp();git(src,'init','-q');git(src,'config','user.email','fixture@example.test');git(src,'config','user.name','Fixture');fs.writeFileSync(path.join(src,'g.txt'),'g\n');git(src,'add','.');git(src,'commit','-qm','sub');git(p,'-c','protocol.file.allow=always','submodule','add','-q',src,'sub');const run=newRun(p),manifest=await capture(p,run.dir,'HEAD');assertSnapshot(run.dir,manifest);expect(manifest.entries.find(e=>e.path==='sub')?.transformation).toBe(NESTED_REPOSITORY);expect(fs.existsSync(path.join(run.dir,'snapshot','sub','g.txt'))).toBe(false);});
|
||||
test('admits source membership and time before an unbounded startup walk',async()=>{const p=repo();fs.writeFileSync(path.join(p,'second.txt'),'second\n');const tooMany=newRun(p);await expect(capture(p,tooMany.dir,undefined,undefined,{maxEntries:1})).rejects.toMatchObject({code:'MISSING_INPUT'});expect(fs.existsSync(path.join(tooMany.dir,'snapshot'))).toBe(false);const expired=newRun(p);await expect(capture(p,expired.dir,undefined,undefined,{deadlineMs:Date.now()-1})).rejects.toMatchObject({code:'DEADLINE'});expect(fs.existsSync(path.join(expired.dir,'snapshot'))).toBe(false);});
|
||||
test('includes chmod-only changes in diff scope',async()=>{const p=repo();fs.chmodSync(path.join(p,'tracked.txt'),0o755);const manifest=await capture(p,newRun(p).dir,'HEAD');expect(manifest.changedPaths).toContain('tracked.txt');});
|
||||
test('does not invoke configured clean filters, hooks, fsmonitor, or PATH shims',async()=>{const p=repo(),marker=path.join(p,'marker');fs.writeFileSync(path.join(p,'.gitattributes'),'*.txt filter=hostile\n');git(p,'config','filter.hostile.clean',`/bin/sh -c 'touch ${marker}; cat'`);git(p,'config','core.fsmonitor',`/bin/sh -c 'touch ${marker}'`);const fake=path.join(p,'bin');fs.mkdirSync(fake);fs.writeFileSync(path.join(fake,'git'),`#!/bin/sh\ntouch '${marker}'\nexit 99\n`,{mode:0o755});const old=process.env.PATH;process.env.PATH=`${fake}:${old}`;try{await capture(p,newRun(p).dir,'HEAD');}finally{process.env.PATH=old;}expect(fs.existsSync(marker)).toBe(false);});
|
||||
|
||||
@@ -65,6 +65,19 @@ describe('CSO verifier security judgement',()=>{
|
||||
try{expect((await judgeSecurity('before',payloads,disproved.port)).security).toBe('pass');}finally{disproved.server.stop(true);}
|
||||
});
|
||||
|
||||
test('a set of safe statuses passes any listed rejection that withholds the marker and nothing else',async()=>{
|
||||
const dotfile:HttpAssertion={name:'dotfile',path:'/.env',method:'GET',expected:{status:[403,404],excludes:'DOTFILE'},vulnerable:{status:200,includes:'DOTFILE'}};
|
||||
const answer=async(status:number,body:string)=>{
|
||||
const server=Bun.serve({port:0,hostname:'127.0.0.1',fetch:()=>new Response(body,{status})});
|
||||
try{return [(await judgeSecurity('before',dotfile,Number(server.port))).security,(await judgeSecurity('after',dotfile,Number(server.port))).security];}finally{server.stop(true);}
|
||||
};
|
||||
expect(await answer(404,'not found')).toEqual(['pass','pass']);
|
||||
expect(await answer(403,'denied')).toEqual(['pass','pass']);
|
||||
expect(await answer(200,'DOTFILE=1')).toEqual(['intended_failure','inconclusive']);
|
||||
expect(await answer(403,'denied DOTFILE=1')).toEqual(['inconclusive','inconclusive']);
|
||||
expect(await answer(400,'bad request')).toEqual(['inconclusive','inconclusive']);
|
||||
});
|
||||
|
||||
test('a single assertion keeps its original before and after outcomes',async()=>{
|
||||
const vulnerable=serve(new Set()),fixed=serve(ALL);
|
||||
try{
|
||||
|
||||
@@ -76,10 +76,16 @@ describe('engineering review routing contracts', () => {
|
||||
test('high complexity stops before Section 1 and asks cuts and structure separately', () => {
|
||||
expectMentions(compact(complexity), [['stop', 'classes/services', 'section']], 'compact(complexity)');
|
||||
ordered(compact(complexity), ['1. Explain the complexity', 'Ask each proposed feature cut/deferral separately',
|
||||
'2. Always ask the structure question', '3. Save the actual feature and structure answers']);
|
||||
'2. Always resolve the structure question', '3. Save the actual feature and structure answers']);
|
||||
expect(compact(complexity)).toContain('Pending remedies not decided here: <ids>');
|
||||
});
|
||||
|
||||
test('the structure question uses its registered id and honors a stored preference', () => {
|
||||
ordered(compact(complexity), ['2. Always resolve the structure question',
|
||||
'as question_id `plan-eng-review-structure` (no per-plan slug)', '`AUTO_DECIDE` picks the recommendation',
|
||||
'Compare only the file/class arrangement']);
|
||||
});
|
||||
|
||||
test('no safe smaller arrangement does not authorize scope cuts or bypass the pause', () => {
|
||||
ordered(compact(complexity), ['If no smaller arrangement preserves these commitments',
|
||||
'A pause leaves the arrangement undecided', 'Do not continue to C until it is settled']);
|
||||
|
||||
Vendored
+6
-6
@@ -202,7 +202,7 @@ export const OVERLAY_FIXTURES: OverlayFixture[] = [
|
||||
{
|
||||
id: 'claude-dedicated-tools-vs-bash',
|
||||
overlayPath: 'model-overlays/claude.md',
|
||||
model: 'claude-opus-4-7',
|
||||
model: 'claude-opus-5-5',
|
||||
trials: 10,
|
||||
concurrency: 3,
|
||||
direction: 'lower_is_better',
|
||||
@@ -241,7 +241,7 @@ export const OVERLAY_FIXTURES: OverlayFixture[] = [
|
||||
{
|
||||
id: 'opus-4-7-effort-match-trivial',
|
||||
overlayPath: 'model-overlays/opus-4-7.md',
|
||||
model: 'claude-opus-4-7',
|
||||
model: 'claude-opus-5-5',
|
||||
trials: 10,
|
||||
concurrency: 3,
|
||||
direction: 'lower_is_better',
|
||||
@@ -269,7 +269,7 @@ export const OVERLAY_FIXTURES: OverlayFixture[] = [
|
||||
{
|
||||
id: 'opus-4-7-literal-interpretation',
|
||||
overlayPath: 'model-overlays/opus-4-7.md',
|
||||
model: 'claude-opus-4-7',
|
||||
model: 'claude-opus-5-5',
|
||||
trials: 10,
|
||||
concurrency: 3,
|
||||
direction: 'higher_is_better',
|
||||
@@ -290,16 +290,16 @@ export const OVERLAY_FIXTURES: OverlayFixture[] = [
|
||||
},
|
||||
|
||||
// =========================================================================
|
||||
// Sonnet 4.6 variants of the Opus-4.7 fixtures.
|
||||
// Sonnet 5.5 variant of the Opus 5.5 dedicated-tools fixture.
|
||||
//
|
||||
// Same overlays, prompts, metrics and trial counts; different pinned model.
|
||||
// Same overlay, prompt, metric and trial count; different pinned model.
|
||||
// Model effects remain separate instead of pooling their observations.
|
||||
// =========================================================================
|
||||
|
||||
{
|
||||
id: 'claude-dedicated-tools-vs-bash-sonnet',
|
||||
overlayPath: 'model-overlays/claude.md',
|
||||
model: 'claude-sonnet-4-6',
|
||||
model: 'claude-sonnet-5-5',
|
||||
trials: 10,
|
||||
concurrency: 3,
|
||||
direction: 'lower_is_better',
|
||||
|
||||
@@ -132,6 +132,57 @@ describe('--check with preferences set', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// The global file ($GSTACK_HOME/global-question-preferences.json, the same file
|
||||
// the question-preference hook reads) is a hand-edited fallback for --check
|
||||
// only: one setting covers every project, a project entry for the same id
|
||||
// wins, and the one-way / split-chain guards still run.
|
||||
describe('--check global preference fallback', () => {
|
||||
function plantGlobal(contents: string) {
|
||||
fs.writeFileSync(path.join(tmpHome, 'global-question-preferences.json'), contents);
|
||||
}
|
||||
|
||||
test('global never-ask with no project entry → AUTO_DECIDE', () => {
|
||||
plantGlobal(JSON.stringify({ 'plan-eng-review-structure': 'never-ask' }));
|
||||
const r = run('--check', 'plan-eng-review-structure');
|
||||
expect(r.status).toBe(0);
|
||||
expect(r.stdout.trim()).toBe('AUTO_DECIDE');
|
||||
});
|
||||
|
||||
test('project entry wins over the global entry', () => {
|
||||
plantGlobal(JSON.stringify({ 'plan-eng-review-structure': 'never-ask' }));
|
||||
run('--write', JSON.stringify({ question_id: 'plan-eng-review-structure', preference: 'always-ask', source: 'plan-tune' }));
|
||||
const r = run('--check', 'plan-eng-review-structure');
|
||||
expect(r.stdout.trim()).toBe('ASK_NORMALLY');
|
||||
});
|
||||
|
||||
test('global never-ask on a one-way id → ASK_NORMALLY with safety note', () => {
|
||||
plantGlobal(JSON.stringify({ 'ship-test-failure-triage': 'never-ask' }));
|
||||
const r = run('--check', 'ship-test-failure-triage');
|
||||
expect(r.stdout).toContain('ASK_NORMALLY');
|
||||
expect(r.stdout).toContain('one-way door overrides');
|
||||
});
|
||||
|
||||
test('global never-ask on a split-chain id → ASK_NORMALLY', () => {
|
||||
plantGlobal(JSON.stringify({ 'plan-eng-review-split-option-a': 'never-ask' }));
|
||||
const r = run('--check', 'plan-eng-review-split-option-a');
|
||||
expect(r.stdout.split('\n')[0]).toBe('ASK_NORMALLY');
|
||||
});
|
||||
|
||||
test.each(['{not json', 'null', '["never-ask"]'])('unusable global file %p is ignored with a note', (contents) => {
|
||||
plantGlobal(contents);
|
||||
const r = run('--check', 'plan-eng-review-structure');
|
||||
expect(r.status).toBe(0);
|
||||
expect(r.stdout.trim()).toBe('ASK_NORMALLY');
|
||||
expect(r.stderr).toContain('global-question-preferences.json is unreadable');
|
||||
});
|
||||
|
||||
test('--write stays per-project and never touches the global file', () => {
|
||||
plantGlobal('{}');
|
||||
run('--write', JSON.stringify({ question_id: 'plan-eng-review-structure', preference: 'never-ask', source: 'plan-tune' }));
|
||||
expect(fs.readFileSync(path.join(tmpHome, 'global-question-preferences.json'), 'utf-8')).toBe('{}');
|
||||
});
|
||||
});
|
||||
|
||||
// #2024: the keyword net only fires when the question TEXT reaches the
|
||||
// classifier. --summary-stdin pipes it (stdin, not argv — summaries carry
|
||||
// quotes/newlines/shell metacharacters). Without the summary, an unregistered
|
||||
|
||||
@@ -4,7 +4,7 @@ import { join } from 'node:path';
|
||||
import { buildWorkflowJudgePrompt, type WorkflowJudgeFile, type WorkflowJudgeInput } from './workflow-judge-input';
|
||||
|
||||
export const COOKIE_WORKFLOW_JUDGE = {
|
||||
model: 'claude-sonnet-4-6',
|
||||
model: 'claude-sonnet-5-5',
|
||||
judgeContext: 'a fallback-browser cookie import workflow',
|
||||
judgeGoal: 'how to select an authorized source browser, profile, and domain without guessing an account; configure optional authentication verification before mutation; obtain explicit consent for precisely scoped storage reset; distinguish copied cookies from positive sign-in evidence; and recover within the documented platform and privacy boundaries',
|
||||
thresholds: { clarity: 4, completeness: 3, actionability: 4 },
|
||||
|
||||
@@ -48,14 +48,31 @@ const LATER_ENTRYPOINT_EDITS: Array<[string, string]> = [
|
||||
['bash "$tmpfile" "bun-v$BUN_VERSION"', 'BUN_VERSION="$BUN_VERSION" bash "$tmpfile"'],
|
||||
];
|
||||
|
||||
// The GSTA-49 cookie-workflow repair rewrote the entrypoint's steps 1-5 (split
|
||||
// the two probe READY outcomes, dropped the duplicate SETUP block for the browse
|
||||
// fallback, moved source selection before the target, asked about verification
|
||||
// before the daemon starts, added a report shape) and one BROWSER.md comment.
|
||||
// Reversing it first restores the pre-repair text, which the edits below then
|
||||
// take back further.
|
||||
const LATER_COOKIE_ORDER_EDITS: Array<[string, string]> = [
|
||||
["## 1. Check for Aside\n\nUse this checkout as the gstack root if it contains `BROWSER.md` and `browse/SKILL.md`; otherwise use the installed root containing `bin/gstack-skill-start`, never a generated host stub. Read that root's `browse/SKILL.md` **BROWSER SETUP** section and run its Aside probe.\n\n- **Aside probe prints `READY`:** no import is needed, because Aside already has the user's signed-in sessions. Tell the user to use Aside, or to sign in there, and end this skill.\n- **Any other result:** continue with gstack's own headless browser (`$B`).\n\n## 2. Get every decision from the user first\n\nAsk these together, before running any `$B` command. Never guess an answer or treat the default Comet browser as consent.\n\n1. **Source browser and profile.** `--profile` takes the profile's directory (`Default`, `Profile 2`), not its display name. A profile that cannot be read is unknown, not empty.\n2. **Domains to import,** and the **target**: one page on a chosen domain that the session should open, by URL.\n3. **Sign-in verification (`--verify-auth`)?** Default no. It reloads the target and reports **verified** only after a successful same-origin response with exactly one visible element, found by a CSS selector, whose whitespace-normalized text equals the expected identity. If yes, the user supplies the selector and the expected identity privately.\n4. **Clear stale storage (`--clear-storage`)?** Default no; needs explicit approval. Chromium only. It clears localStorage for the target's origin (shared by that origin's tabs) and sessionStorage for the target tab, and nothing else; other browser engines reject the reset but still import. A failed reset may already have cleared some storage and is not rolled back.\n5. **Every cookie in the profile (`--all`)?** Default no; needs explicit consent. It cannot be combined with a domain or with clearing storage.\n\nUse the picker when any of answers 1-2 is still open; it asks for them. Use direct import when the user has named the browser, the profile directory and the domains.\n\n## 3. Prepare gstack's browser\n\n1. If the user wants verification, have them set `GSTACK_COOKIE_AUTH_SELECTOR` and `GSTACK_COOKIE_AUTH_EXPECTED_IDENTITY` privately in the shell that runs `$B`. The browser daemon reads them only when it starts, so if a gstack daemon is already running, run `$B stop` first. Never invent the values.\n2. Follow `browse/SKILL.md`'s **Browser fallback** section to find `$B`, building it if the check prints `NEEDS_SETUP`.\n3. Run `$B status`. If it reports `Mode: cdp`, `$B` is attached to the user's real browser and already has its sessions: tell the user no import is needed and end this skill.\n4. Open the target URL from answer 2 and leave that tab alone until the import finishes:\n\n```bash\n$B goto \"<url>\"\n```\n\n## 4. Import\n\nPicker (the user confirms the browser, profile and domains there, ticks verification if they asked for it, then tells you when done; the link works once, for five minutes, so rerun the command if it expires):\n\n```bash\n$B cookie-import-browser\n```\n\nDirect import: run `$B cookie-import-browser` with the chosen browser name, one `--domain` per run and `--profile` set to the profile directory (not its display name); add `--verify-auth` or `--clear-storage` only if the user said yes in step 2.\n\nIf the import rejects because verification settings are missing, nothing changed: tell the user which variable is missing and return to step 3. Read that same root's `BROWSER.md`, **Choosing a source and checking sign-in**, for other errors and receipt failure reasons; it has the error tables, examples, profile labels, supported sources and platform setup.\n\n## 5. Report honestly\n\nReport from the import receipt, not from raw `$B cookies`, as three lines:\n\n- `Imported: <count> (<imported|partial|empty|failed>, as the receipt says) from <browser> for <domains>`\n- `Storage reset: <done|failed|not requested>`\n- `Sign-in: <verified|not verified|not checked>`\n\n**Not checked** means verification was not requested; **not verified** means it ran and failed; **verified** needs the identity match. Cookie counts, zero imports and HTTP 200 never prove sign-in. Imported cookies apply to the whole browser context, not one tab.\n\nNever request or publish cookie values, passwords, identity or profile text, session details, or raw errors in public logs.\n\n## Platform boundaries", "## 1. Choose the browser\n\nUse this checkout as the gstack root if it contains `BROWSER.md` and `browse/SKILL.md`; otherwise use the installed root containing `bin/gstack-skill-start`, never a generated host stub. Read that root's `browse/SKILL.md` **BROWSER SETUP** section and run its probe first. On `READY`, stop importing: use Aside's sessions or ask the user to sign in there. Otherwise follow the probe's fallback handling, then continue below.\n\n## SETUP (run this check BEFORE any browse command)\n\n```bash\n_ROOT=$(git rev-parse --show-toplevel 2>/de
Line truncated
|
||||
["# view the page; only --verify-auth proves sign-in", "# confirm the intended account"],
|
||||
];
|
||||
|
||||
export function approvedCookieWorkflowSource(source: string): string {
|
||||
let removedLines = 0;
|
||||
let removedEntrypointLines = 0;
|
||||
let historical = source
|
||||
let reordered = source;
|
||||
for (const [later, earlier] of LATER_COOKIE_ORDER_EDITS) {
|
||||
if (!reordered.includes(later)) continue;
|
||||
removedEntrypointLines += later.split('\n').length - earlier.split('\n').length;
|
||||
reordered = reordered.replace(later, earlier);
|
||||
}
|
||||
let historical = reordered
|
||||
.replace('sha256sum < "$tmpfile" | awk \'{print $(1)}\'', 'sha256sum "$tmpfile" | awk \'{print $1}\'')
|
||||
.replace('shasum -a 256 < "$tmpfile" | awk \'{print $(1)}\'', 'shasum -a 256 "$tmpfile" | awk \'{print $1}\'')
|
||||
.replace(LATER_BROWSER_BLOCK, block => { removedLines = block.split('\n').length - 1; return ''; })
|
||||
.replace(LATER_ENTRYPOINT_BLOCK, block => { removedEntrypointLines = block.split('\n').length - 1; return ''; });
|
||||
.replace(LATER_ENTRYPOINT_BLOCK, block => { removedEntrypointLines += block.split('\n').length - 1; return ''; });
|
||||
for (const [later, earlier] of [...LATER_BROWSER_EDITS, ...LATER_PREAMBLE_EDITS, ...LATER_WAVE_EDITS, ...LATER_ENTRYPOINT_EDITS]) historical = historical.replace(later, earlier);
|
||||
return historical
|
||||
.replace(/(--- BEGIN FILE "BROWSER\.md" \(lines \d+-)(\d+)(; section\) ---)/, (_, head, end, tail) => `${head}${Number(end) - removedLines}${tail}`)
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { isDeepStrictEqual } from 'node:util';
|
||||
import * as path from 'node:path';
|
||||
import { renderOfficeHoursReviewerPrompt, validateOfficeHoursReview, renderOfficeHoursReview, extractOfficeHoursReviewBlock, parseOfficeHoursVerdictReceipt, verifyOfficeHoursVerdictReceipt, officeHoursDesignChanges, verifyOfficeHoursCitations, type OfficeHoursReview } from '../../lib/office-hours-review';
|
||||
import { verifySpecFixesAtCap, type SpecFix } from '../../lib/spec-review-cap';
|
||||
import { parseSpecFixes, verifySpecFixesAtCap, type SpecFix } from '../../lib/spec-review-cap';
|
||||
|
||||
export type OfficeHoursFile = { path: string; content: string | null };
|
||||
/** The helper's captured diff before a later round; a missing snapshot fails closed. */
|
||||
@@ -32,6 +32,21 @@ export interface OfficeHoursCompletionEvidence {
|
||||
fixes?: SpecFix[] | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The cap fixes the agent saved for the finalizer: the last observed Write whose
|
||||
* content is a valid `--fixes` list. Null when it saved none. The validator
|
||||
* recomputes the cap record from this list, so a list that was not the one
|
||||
* passed to finalize still produces a mismatch.
|
||||
*/
|
||||
export function observedSpecFixes(toolCalls: OfficeHoursCompletionEvidence['toolCalls']): SpecFix[] | null {
|
||||
for (const call of toolCalls.toReversed()) {
|
||||
if (call.tool !== 'Write') continue;
|
||||
try { return parseSpecFixes(JSON.parse(String(call.input?.content ?? ''))); }
|
||||
catch { /* not a fixes list */ }
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export interface OfficeHoursReviewEvidence {
|
||||
verdict: string;
|
||||
concerns: string;
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
/**
|
||||
* Copy bin/gstack-office-hours-review and every local module it imports into a
|
||||
* fixture, so the copied formatter loads the same modules as the checkout.
|
||||
* Marathon 39032308051 copied a hand-kept file list that missed
|
||||
* lib/spec-review-cap.ts, and every formatter command failed at module load.
|
||||
*/
|
||||
import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '../..');
|
||||
|
||||
export function copyOfficeHoursFormatter(fixtureRoot: string): string {
|
||||
const pending = ['bin/gstack-office-hours-review'];
|
||||
const copied = new Set<string>();
|
||||
while (pending.length) {
|
||||
const relative = pending.pop()!;
|
||||
if (copied.has(relative)) continue;
|
||||
copied.add(relative);
|
||||
const source = path.join(ROOT, relative);
|
||||
const target = path.join(fixtureRoot, relative);
|
||||
fs.mkdirSync(path.dirname(target), { recursive: true });
|
||||
fs.copyFileSync(source, target);
|
||||
for (const [, specifier] of fs.readFileSync(source, 'utf8').matchAll(/\b(?:from|import)\s+'(\.\.?\/[^']+)'/g)) {
|
||||
const resolved = path.relative(ROOT, path.resolve(path.dirname(source), specifier!));
|
||||
pending.push(path.extname(resolved) ? resolved : `${resolved}.ts`);
|
||||
}
|
||||
}
|
||||
const formatter = path.join(fixtureRoot, 'bin/gstack-office-hours-review');
|
||||
fs.chmodSync(formatter, 0o755);
|
||||
return formatter;
|
||||
}
|
||||
@@ -1099,7 +1099,7 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
|
||||
'test/skill-e2e-arm-benchmark.test.ts',
|
||||
'ship/SKILL.md', 'test/helpers/e2e-gate.ts'],
|
||||
|
||||
'office-hours-section-loading': [ 'office-hours/**', 'bin/gstack-office-hours-review', 'lib/office-hours-review.ts', 'lib/fs-atomic.ts', 'scripts/resolvers/review-dashboard.ts', 'scripts/resolvers/plan-gates.ts', 'scripts/resolvers/spec-review.ts', 'scripts/resolvers/outside-voice-steps.ts', 'scripts/resolvers/review-scope.ts', 'scripts/resolvers/outside-voice.ts', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/carve-guards.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/office-hours-completion.ts', 'test/helpers/llm-judge.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-office-hours-section-loading.test.ts', 'test/helpers/agent-sdk-runner.ts', 'test/helpers/auq-native-capture.ts', 'test/helpers/auto-decision-state.ts', 'test/helpers/autoplan-artifact-digest.ts', 'test/helpers/autoplan-artifact-permission.ts', 'test/helpers/autoplan-artifact-recorder.ts', 'test/helpers/capture-parity-baseline.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty/**', 'test/helpers/dx-selected-navigation.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/eng-cache-writer-decision.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/helpers/native-auto-decide.ts', 'test/helpers/owned-claude-transcript.ts', 'test/helpers/parity-harness.ts', 'test/helpers/plan-count-artifacts.ts', 'test/helpers/plan-count-file-permission.ts', 'test/helpers/plan-count-fixture.ts', 'test/helpers/plan-count-pending-exit.ts', 'test/helpers/plan-count-pending-question.ts', 'test/helpers/plan-count-transcript.ts', 'test/helpers/plan-floor-review.ts', 'test/helpers/plan-floor-target.ts', 'test/helpers/plan-scope-selection.ts', 'test/helpers/plan-seed-submission.ts', 'test/helpers/plan-skill-question-events.ts', 'test/helpers/plan-skill-question-hook-scope.ts', 'test/helpers/plan-skill-questions.ts', 'test/helpers/pty-screen.ts', 'test/helpers/pty-trust-dialog.ts', 'test/helpers/skill-census.ts'],
|
||||
'office-hours-section-loading': [ 'office-hours/**', 'bin/gstack-office-hours-review', 'lib/office-hours-review.ts', 'lib/fs-atomic.ts', 'lib/spec-review-cap.ts', 'test/helpers/office-hours-formatter-fixture.ts', 'scripts/resolvers/review-dashboard.ts', 'scripts/resolvers/plan-gates.ts', 'scripts/resolvers/spec-review.ts', 'scripts/resolvers/outside-voice-steps.ts', 'scripts/resolvers/review-scope.ts', 'scripts/resolvers/outside-voice.ts', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/carve-guards.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/office-hours-completion.ts', 'test/helpers/llm-judge.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-office-hours-section-loading.test.ts', 'test/helpers/agent-sdk-runner.ts', 'test/helpers/auq-native-capture.ts', 'test/helpers/auto-decision-state.ts', 'test/helpers/autoplan-artifact-digest.ts', 'test/helpers/autoplan-artifact-permission.ts', 'test/helpers/autoplan-artifact-recorder.ts', 'test/helpers/capture-parity-baseline.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty/**', 'test/helpers/dx-selected-navigation.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/eng-cache-writer-decision.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/helpers/native-auto-decide.ts', 'test/helpers/owned-claude-transcript.ts', 'test/helpers/parity-harness.ts', 'test/helpers/plan-count-artifacts.ts', 'test/helpers/plan-count-file-permission.ts', 'test/helpers/plan-count-fixture.ts', 'test/helpers/plan-count-pending-exit.ts', 'test/helpers/plan-count-pending-question.ts', 'test/helpers/plan-count-transcript.ts', 'test/helpers/plan-floor-review.ts', 'test/helpers/plan-floor-target.ts', 'test/helpers/plan-scope-selection.ts', 'test/helpers/plan-seed-submission.ts', 'test/helpers/plan-skill-question-events.ts', 'test/helpers/plan-skill-question-hook-scope.ts', 'test/helpers/plan-skill-questions.ts', 'test/helpers/pty-screen.ts', 'test/helpers/pty-trust-dialog.ts', 'test/helpers/skill-census.ts'],
|
||||
'office-hours-design-draft': [ 'office-hours/**', 'lib/office-hours-review.ts', 'scripts/resolvers/review-dashboard.ts', 'scripts/resolvers/plan-gates.ts', 'scripts/resolvers/spec-review.ts', 'scripts/resolvers/outside-voice-steps.ts', 'scripts/resolvers/review-scope.ts', 'scripts/resolvers/outside-voice.ts', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/carve-guards.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/office-hours-completion.ts', 'test/helpers/llm-judge.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-office-hours-design-draft.test.ts', 'test/helpers/agent-sdk-runner.ts', 'test/helpers/auq-native-capture.ts', 'test/helpers/auto-decision-state.ts', 'test/helpers/autoplan-artifact-digest.ts', 'test/helpers/autoplan-artifact-permission.ts', 'test/helpers/autoplan-artifact-recorder.ts', 'test/helpers/capture-parity-baseline.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty/**', 'test/helpers/dx-selected-navigation.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/eng-cache-writer-decision.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/helpers/native-auto-decide.ts', 'test/helpers/owned-claude-transcript.ts', 'test/helpers/parity-harness.ts', 'test/helpers/plan-count-artifacts.ts', 'test/helpers/plan-count-file-permission.ts', 'test/helpers/plan-count-fixture.ts', 'test/helpers/plan-count-pending-exit.ts', 'test/helpers/plan-count-pending-question.ts', 'test/helpers/plan-count-transcript.ts', 'test/helpers/plan-floor-review.ts', 'test/helpers/plan-floor-target.ts', 'test/helpers/plan-scope-selection.ts', 'test/helpers/plan-seed-submission.ts', 'test/helpers/plan-skill-question-events.ts', 'test/helpers/plan-skill-question-hook-scope.ts', 'test/helpers/plan-skill-questions.ts', 'test/helpers/pty-screen.ts', 'test/helpers/pty-trust-dialog.ts', 'test/helpers/skill-census.ts'],
|
||||
'dx-error-contract-review': ['test/skill-e2e-dx-error-contract.test.ts', 'test/fixtures/dx-error-contract-case.ts', 'lib/dx-error-contract.ts', 'scripts/resolvers/dx.ts', 'scripts/resolvers/index.ts', 'scripts/gen-skill-docs.ts', 'plan-devex-review/**', 'bin/gstack-autoplan-snapshot.ts', 'lib/autoplan-prompts.ts', 'lib/autoplan-reconcile.ts', 'lib/coordinator-contract.ts', 'test/helpers/session-runner.ts', 'test/helpers/eval-store.ts', 'test/helpers/eval-budgets.ts', 'lib/claude-bin.ts', 'lib/eval-model.ts'],
|
||||
'plan-devex-peer-comparison-classification': [
|
||||
|
||||
@@ -654,7 +654,7 @@ describe('Plan format actual capture and judge lifecycle', () => {
|
||||
const starts = events.filter(event => event.kind === 'start');
|
||||
expect(starts.map(({ timeout, maxTurns, model }) => ({ timeout, maxTurns, model })))
|
||||
.toEqual([1, 2].map(() => ({ timeout: 300, maxTurns: 10,
|
||||
model: file.includes('plan-prosons') ? 'claude-opus-4-7' : resolveEvalModel('capture') })));
|
||||
model: resolveEvalModel('capture') })));
|
||||
expect(events.filter(event => event.kind === 'ready').map(event => event.fixtureExists)).toEqual([true, true]);
|
||||
expect(entries).toHaveLength(2);
|
||||
expect(entries.map(entry => entry.attempt)).toEqual([1, 2]);
|
||||
|
||||
@@ -4,7 +4,7 @@ import { describe, expect, test } from 'bun:test';
|
||||
import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import * as os from 'node:os';
|
||||
import { validateOfficeHoursCompletion, validateOfficeHoursDesignDraft, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation, type OfficeHoursCompletionEvidence } from './helpers/office-hours-completion';
|
||||
import { observedSpecFixes, validateOfficeHoursCompletion, validateOfficeHoursDesignDraft, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation, type OfficeHoursCompletionEvidence } from './helpers/office-hours-completion';
|
||||
import { E2E_TOUCHFILES } from './helpers/touchfiles-data';
|
||||
import { selectTests } from './helpers/test-selection';
|
||||
import { expectMentions } from './helpers/prompt-structure';
|
||||
@@ -954,6 +954,42 @@ Next: /plan-eng-review after observing the workflow. The user declined launching
|
||||
} finally { fs.rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
|
||||
test('cap fixes the agent saved and passed to finalize validate; without them the report mismatches (marathon rerun 2026-10-19)', () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'office-hours-cap-fixes-'));
|
||||
try {
|
||||
const { evidence, artifacts } = JSON.parse(JSON.stringify(structured())
|
||||
.replaceAll('/tmp/office-hours-fixture', dir.replaceAll('\\', '/')));
|
||||
evidence.toolCalls.filter((call: { output?: string }) => call.output?.startsWith('OFFICE_HOURS_VERDICT '))
|
||||
.forEach((call: { output?: string }, i: number) => { call.output = officeHoursVerdictReceipt(i + 1, artifacts[i].path, artifacts[i].content); });
|
||||
const applied = 'An absent mapped column stops the import and names the missing column.';
|
||||
const fixes = [{ id: 'R2-1', applied_text: applied }];
|
||||
const reviewDir = path.dirname(artifacts[0].path);
|
||||
const fixesPath = path.join(reviewDir, 'fixes.json');
|
||||
const reportPath = path.join(dir, 'REPORT.md');
|
||||
fs.mkdirSync(path.dirname(evidence.designPath), { recursive: true });
|
||||
fs.writeFileSync(evidence.designPath, evidence.designContent.replace('# Design: RosterCheck\n', `# Design: RosterCheck\n\n${applied}\n`));
|
||||
fs.writeFileSync(reportPath, evidence.output);
|
||||
fs.mkdirSync(reviewDir, { recursive: true });
|
||||
for (const artifact of artifacts) fs.writeFileSync(artifact.path, artifact.content);
|
||||
const snapshots = snapshotsIn(reviewDir);
|
||||
for (const snapshot of snapshots) fs.writeFileSync(snapshot.path, snapshot.content);
|
||||
fs.writeFileSync(fixesPath, JSON.stringify(fixes));
|
||||
const result = Bun.spawnSync([process.execPath, path.resolve(import.meta.dir, '../bin/gstack-office-hours-review'),
|
||||
'finalize', '--design', evidence.designPath, '--report', reportPath, '--fixes', fixesPath, ...artifacts.map((artifact: { path: string }) => artifact.path)],
|
||||
{ cwd: dir, timeout: 5000 });
|
||||
expect(result.exitCode, result.stderr.toString()).toBe(0);
|
||||
expect(JSON.parse(result.stdout.toString()).cap.verified).toEqual(['R2-1']);
|
||||
evidence.output = fs.readFileSync(reportPath, 'utf8');
|
||||
evidence.designContent = fs.readFileSync(evidence.designPath, 'utf8');
|
||||
evidence.toolCalls.push({ tool: 'Write', input: { file_path: fixesPath, content: JSON.stringify(fixes) } });
|
||||
expect(observedSpecFixes(evidence.toolCalls)).toEqual(fixes);
|
||||
expect(() => checkArtifacts({ ...evidence, fixes: observedSpecFixes(evidence.toolCalls) }, artifacts, snapshots)).not.toThrow();
|
||||
expect(() => checkArtifacts(evidence, artifacts, snapshots)).toThrow('Spec Review does not match');
|
||||
expect(() => checkArtifacts({ ...evidence, fixes: [{ id: 'R2-2', applied_text: applied }] }, artifacts, snapshots)).toThrow('Spec Review does not match');
|
||||
expect(observedSpecFixes(evidence.toolCalls.filter((call: { input?: { file_path?: string } }) => call.input?.file_path !== fixesPath))).toBeNull();
|
||||
} finally { fs.rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
|
||||
test('allows a Markdown separator after the exact block, without hiding extra outcome prose', () => {
|
||||
const { evidence, artifacts } = structured();
|
||||
evidence.output = evidence.output.replace('## Handoff', '---\n\n## Handoff');
|
||||
|
||||
@@ -2,6 +2,7 @@ import { expect, test } from 'bun:test';
|
||||
import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import { CAPTURE_MS, CAPTURE_LONG_MS } from './helpers/eval-budgets';
|
||||
import { resolveEvalModel } from '../lib/eval-model';
|
||||
const source = fs.readFileSync(path.join(import.meta.dir, 'skill-e2e-office-hours-phase4.test.ts'), 'utf8');
|
||||
const question = (word = 'architectural', options = 'A) Put retrieval on the server\nB) Put retrieval on the client') =>
|
||||
`Where should retrieval live?\nThis ${word} choice decides which component owns the retrieval algorithm and the cross-host API contract.\n${options}\nRecommendation: A because all hosts need a consistent algorithm with one owner.\n`;
|
||||
@@ -19,7 +20,7 @@ async function runCaller(captured: string | undefined, scenario: 'success' | 'ti
|
||||
const result = { exitReason: scenario === 'timeout' ? 'timeout' : 'success', browseErrors: [], durationMs: 123, costUsd: 0.1, output: 'public completion' };
|
||||
const args: Record<string,any> = {
|
||||
expect, beforeAll: (fn: () => void) => setups.push(fn), afterAll: (fn: () => Promise<void>) => finalizers.push(fn),
|
||||
CAPTURE_MS, CAPTURE_LONG_MS, ROOT: sourceRoot, runId: 'synthetic-run',
|
||||
CAPTURE_MS, CAPTURE_LONG_MS, resolveEvalModel, ROOT: sourceRoot, runId: 'synthetic-run',
|
||||
describeIfSelected: (_title: string, _names: string[], fn: () => void) => fn(),
|
||||
testConcurrentIfSelected: (name: string, fn: () => Promise<void>, timeout: number) => { expect(name).toBe('office-hours-phase4-fork'); expect(timeout).toBe(CAPTURE_LONG_MS); callbacks.push(fn); },
|
||||
createEvalCollector: () => collector, finalizeEvalCollector: async () => {}, logCost: () => {},
|
||||
@@ -35,7 +36,7 @@ async function runCaller(captured: string | undefined, scenario: 'success' | 'ti
|
||||
fs: { mkdtempSync:(prefix:string)=>prefix+'owned', mkdirSync:()=>{}, rmSync:()=>{}, existsSync:(name:string)=>files.has(name),
|
||||
writeFileSync:(name:string,body:string)=>files.set(name,body), readFileSync:(name:string)=>name===fixturePath.join(sourceRoot,'office-hours','SKILL.md') ? '## AskUserQuestion Format\nformat\n## Phase 4: Alternatives Generation\nworkflow\n## Phase 4.5\nnext' : files.get(name) },
|
||||
runSkillTest: async (opts: any) => {
|
||||
calls.push(opts); expect(opts.timeout).toBe(CAPTURE_MS); expect(opts.maxTurns).toBe(12); expect(opts.model).toBe('claude-opus-4-7');
|
||||
calls.push(opts); expect(opts.timeout).toBe(CAPTURE_MS); expect(opts.maxTurns).toBe(12); expect(opts.model).toBe(resolveEvalModel('capture'));
|
||||
expect(opts.prompt).toContain('Do NOT call any tool to ask the user.');
|
||||
if (captured !== undefined) files.set(fixturePath.join(opts.workingDirectory, 'phase4-capture.md'), captured);
|
||||
return result;
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
} from '../lib/office-hours-review';
|
||||
import { parseSpecFixes, verifySpecFixesAtCap, renderSpecCap } from '../lib/spec-review-cap';
|
||||
import { expectMentions } from './helpers/prompt-structure';
|
||||
import { copyOfficeHoursFormatter } from './helpers/office-hours-formatter-fixture';
|
||||
|
||||
const cli = path.resolve(import.meta.dir, '../bin/gstack-office-hours-review');
|
||||
function review(round = 1, count = 1, severity: 'blocking' | 'minor' = 'blocking'): OfficeHoursReview {
|
||||
@@ -638,3 +639,17 @@ describe('office-hours review CLI', () => {
|
||||
}
|
||||
}));
|
||||
});
|
||||
|
||||
describe('office-hours formatter fixture copy', () => {
|
||||
test('the copied formatter loads every local module it imports (marathon 39032308051)', () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'office-hours-formatter-copy-'));
|
||||
try {
|
||||
const formatter = copyOfficeHoursFormatter(dir);
|
||||
const result = Bun.spawnSync([process.execPath, formatter], { cwd: dir, stdout: 'pipe', stderr: 'pipe', timeout: 5000 });
|
||||
expect(result.stderr.toString()).not.toContain('Cannot find module');
|
||||
expect(result.stderr.toString()).toContain('gstack-office-hours-review: usage:');
|
||||
} finally {
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -65,6 +65,67 @@ describe('classifyOutsideReview: separate execution, findings and verdict', () =
|
||||
});
|
||||
});
|
||||
|
||||
// #3294: "clean" comes only from a no-findings statement that concludes the reply,
|
||||
// never from a "no X" phrase among untagged findings anywhere in it.
|
||||
describe('a no-findings statement counts only as the reply\'s conclusion (#3294)', () => {
|
||||
const REPORTER = 'I found no actionable regressions in the staged, unstaged, or untracked changes.';
|
||||
const FINDING = 'The fallback path now swallows ENOENT and returns an empty config.';
|
||||
|
||||
test("the reporter's exact codex review reply reads clean through the reporter's CLI invocation", () => {
|
||||
expect(classifyOutsideReview({ text: REPORTER, gate: 'structured' }).verdict).toBe('clean');
|
||||
const r = cli(['--label', 'probe', '--exit', '0', '--stderr', file(''), 'structured', file(REPORTER)]);
|
||||
expect([r.status, r.stdout]).toEqual([0, 'VERDICT: clean\nFINDINGS: none\n']);
|
||||
});
|
||||
|
||||
test('clean conclusions, with the added regressions/defects vocabulary and the requested marker formatted', () => {
|
||||
for (const text of ['No regressions found.', 'I did not find any defects.', 'There are no actionable regressions.',
|
||||
'NO_FINDINGS', '**NO_FINDINGS**', '`NO_FINDINGS`', '- NO_FINDINGS',
|
||||
`The change adds a retry cap and its tests.\n${REPORTER}`, 'Reviewed the retry helper.\n\nNo issues found in this change.',
|
||||
'After reviewing the patch, I found no issues.', 'I found no issues in this read-only change.',
|
||||
'Summary of the retry helper\nNo issues found.', 'I found no actionable regressions in the staged\nchanges.',
|
||||
// Clean `codex review --uncommitted` replies captured from codex-cli 0.160.0 with gpt-6-sol.
|
||||
'The change validates the retry limit before execution. No actionable regression was found.',
|
||||
'The change validates `max` before retrying. No affected callers or tests are present in the repository, and no actionable regression is evident.']) {
|
||||
expect({ text, verdict: classifyOutsideReview({ text, gate: 'structured' }).verdict }).toEqual({ text, verdict: 'clean' });
|
||||
}
|
||||
});
|
||||
|
||||
test('mixed replies: untagged findings beside a "no X" phrase are unverified, never clean', () => {
|
||||
for (const text of [
|
||||
'The parser change has no new issues. However, db/002_drop_idx.sql drops users_email_idx, which will regress login latency under load.',
|
||||
'Renaming the flag is no problem for callers, but the fallback path now swallows ENOENT and returns an empty config.',
|
||||
`No issues in the parser.\n\n${FINDING}`,
|
||||
`${FINDING}\n\nNo issues in the parser.`,
|
||||
`${FINDING} No regressions in the existing tests.`,
|
||||
'No regressions in the existing tests, but the fallback path now swallows ENOENT.',
|
||||
`${FINDING}\nI found no issues other than the fallback path.`,
|
||||
`${FINDING}\nNo other issues.`,
|
||||
'No issues found, except that the fallback path now swallows ENOENT.',
|
||||
'No issues in the parser, and the fallback path now swallows ENOENT.',
|
||||
'Missing configuration now silently disables authentication. No regressions were found by the existing tests.',
|
||||
'Missing configuration now silently disables authentication. No issues are apparent in the parser.',
|
||||
'The fallback path now swallows ENOENT, and no issues elsewhere.',
|
||||
'The diff adds a NO_FINDINGS check, but the regex misses lowercase input.',
|
||||
`${FINDING}\n\`\`\`\nNo issues found.\n\`\`\``,
|
||||
]) {
|
||||
for (const gate of ['structured', 'review'] as const) {
|
||||
const r = classifyOutsideReview({ text: gate === 'review' ? `${text}\nRecommendation: ship because it looks fine.` : text, gate });
|
||||
expect({ text, gate, verdict: r.verdict, reason: r.reason }).toEqual({ text, gate, verdict: 'unverified', reason: 'untagged_review' });
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("a review's Recommendation because-clause concludes under the same rules", () => {
|
||||
const verdict = (text: string) => classifyOutsideReview({ text, gate: 'review' }).verdict;
|
||||
expect(verdict('Recommendation: ship because I found no regressions in this change.')).toBe('clean');
|
||||
expect(verdict(`The change adds a retry cap.\nRecommendation: ship because no issues remain.`)).toBe('clean');
|
||||
expect(verdict('No issues in the parser.\nRecommendation: fix the fallback because it swallows ENOENT.')).toBe('unverified');
|
||||
expect(verdict(`${FINDING}\nRecommendation: ship because no regressions in the existing tests.`)).toBe('unverified');
|
||||
expect(verdict(`${FINDING}\nRecommendation: ship because no issues remain, but the fallback needs a follow-up.`)).toBe('unverified');
|
||||
expect(verdict('Recommendation: address the migration because no regressions were found in the parser change,\nbut the migration drops users_email_idx.')).toBe('unverified');
|
||||
});
|
||||
});
|
||||
|
||||
describe('validateOutsideReview keeps the text-only shape for direct importers', () => {
|
||||
test('existing shapes are unchanged', () => {
|
||||
expect(validateOutsideReview('', 'review')).toEqual({ completed: false, reason: 'empty response' });
|
||||
|
||||
@@ -28,7 +28,7 @@ test('every fixture owns one paid wrapper; public models/trials/concurrency/turn
|
||||
expect(isPaidTestFile('test/overlay-lifecycle.test.ts')).toBe(false);
|
||||
expect(OVERLAY_FIXTURES.every(f => f.trials === 10 && f.concurrency === 3)).toBe(true);
|
||||
expect(OVERLAY_FIXTURES.map(f => f.maxTurns ?? 5)).toEqual([15,8,15,15,15]);
|
||||
expect(OVERLAY_FIXTURES.map(f => f.model)).toEqual([...Array(3).fill('claude-opus-4-7'), 'claude-sonnet-4-6', 'claude-opus-5-5']);
|
||||
expect(OVERLAY_FIXTURES.map(f => f.model)).toEqual([...Array(3).fill('claude-opus-5-5'), 'claude-sonnet-5-5', 'claude-opus-5-5']);
|
||||
expect([OVERLAY_CASE_WORK_MS, OVERLAY_RECORD_GRACE_MS, OVERLAY_CASE_OUTER_MS, OVERLAY_MIN_FILE_WALL_MS]).toEqual([1_800_000, 5_000, 1_810_000, 1_830_000]);
|
||||
});
|
||||
|
||||
|
||||
@@ -65,7 +65,7 @@ describe('SDK overlay measurements', () => {
|
||||
});
|
||||
|
||||
describe('correctness versus efficacy', () => {
|
||||
describe.each(['claude-opus-4-7', 'claude-sonnet-4-6'])('dedicated-tool gate for %s', (model) => {
|
||||
describe.each(['claude-opus-5-5', 'claude-sonnet-5-5'])('dedicated-tool gate for %s', (model) => {
|
||||
const f = { ...OVERLAY_FIXTURES.find(f => f.metricName === 'bash_tool_calls' && f.model === model)!, trials: 3 };
|
||||
// Contract v4: a trial whose final JSON fails the fixture's verify is a valid
|
||||
// measurement with a wrong answer, not an invalid measurement.
|
||||
@@ -276,9 +276,8 @@ describe('literal fixture task correctness', () => {
|
||||
expect(OVERLAY_FIXTURES).toHaveLength(5);
|
||||
expect(OVERLAY_FIXTURES.some((f) => f.id.includes('fanout') || f.comparison?.unsupportedHypothesis)).toBe(false);
|
||||
expect(OVERLAY_FIXTURES.every((f) => f.trials === 10 && f.comparison)).toBe(true);
|
||||
expect(OVERLAY_FIXTURES.filter((f) => f.model === 'claude-opus-4-7')).toHaveLength(3);
|
||||
expect(OVERLAY_FIXTURES.filter((f) => f.model === 'claude-sonnet-4-6')).toHaveLength(1);
|
||||
expect(OVERLAY_FIXTURES.filter((f) => f.model === 'claude-opus-5-5')).toHaveLength(1);
|
||||
expect(OVERLAY_FIXTURES.filter((f) => f.model === 'claude-opus-5-5')).toHaveLength(4);
|
||||
expect(OVERLAY_FIXTURES.filter((f) => f.model === 'claude-sonnet-5-5')).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -319,7 +319,7 @@ test('Eng loads its one remedy procedure before Scope Challenge findings and ret
|
||||
const scope = compactProse(sections.slice(sections.indexOf('## Scope Challenge'), sections.indexOf('## Review Sections')));
|
||||
expectAll(scope, [/decision-brief format for this complexity gate/i,
|
||||
/ask each proposed feature cut\/deferral separately; wait before changing scope/i,
|
||||
/unapproved fixes stay pending/i, /always ask the structure question when this gate trips/i,
|
||||
/unapproved fixes stay pending/i, /always resolve the structure question when this gate trips/i,
|
||||
/offer confirmation of the original arrangement or a pause to investigate a smaller one\. wait for the answer/i,
|
||||
/apply only accepted scope changes/i, /"No issues found" for an empty list/i, /findings and scope answers approve no remedies/i]);
|
||||
const complexityRule = at(scope, /initial scope selectors need no grid or \*\*pre-answer\*\* ledger write/i);
|
||||
|
||||
@@ -124,7 +124,7 @@ Write DESIGN.md and CLAUDE.md (or update it) in the working directory.`,
|
||||
timeout: CAPTURE_LONG_MS,
|
||||
testName: 'design-consultation-core',
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
});
|
||||
|
||||
logCost('/design-consultation core', result);
|
||||
@@ -298,7 +298,7 @@ Run Phase 0's DESIGN.md format check exactly as written (the gstack bin director
|
||||
timeout: CAPTURE_LONG_MS,
|
||||
testName: 'design-consultation-existing',
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
});
|
||||
|
||||
logCost('/design-consultation existing', result);
|
||||
|
||||
@@ -22,6 +22,7 @@
|
||||
import { expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { CAPTURE_MS, CAPTURE_LONG_MS } from './helpers/eval-budgets';
|
||||
import { runSkillTest } from './helpers/session-runner';
|
||||
import { resolveEvalModel } from '../lib/eval-model';
|
||||
import type { EvalTestEntry } from './helpers/eval-store';
|
||||
import {
|
||||
ROOT, runId,
|
||||
@@ -166,7 +167,7 @@ After writing the file with that ONE Phase 4 question, stop. Do not continue to
|
||||
timeout: CAPTURE_MS,
|
||||
testName: 'office-hours-phase4-fork',
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
});
|
||||
|
||||
logCost('/office-hours Phase 4 fork', result);
|
||||
|
||||
@@ -14,7 +14,8 @@ import * as path from 'node:path';
|
||||
import { describeE2ETier } from './helpers/e2e-gate';
|
||||
import { setupSkillDir, skillFromWorktree, captureSectionReads } from './helpers/auq-sdk-capture';
|
||||
import { CARVE_GUARDS } from './helpers/carve-guards';
|
||||
import { validateOfficeHoursCompletion, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation } from './helpers/office-hours-completion';
|
||||
import { copyOfficeHoursFormatter } from './helpers/office-hours-formatter-fixture';
|
||||
import { observedSpecFixes, validateOfficeHoursCompletion, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation } from './helpers/office-hours-completion';
|
||||
|
||||
const describeMarathon = describeE2ETier('marathon');
|
||||
const runId = `office-hours-section-loading-${process.env.EVALS_RUN_ID ?? 'local'}`;
|
||||
@@ -42,15 +43,9 @@ describeMarathon('/office-hours full section-loading workflow (marathon)', () =>
|
||||
const planDir = setupSkillDir({
|
||||
skillName: guard.skill, skillMd, sectionsFrom, tmpPrefix: 'gstack-office-hours-secload-',
|
||||
});
|
||||
// Copy the real formatter and its small dependency set into the fixture.
|
||||
// Copy the real formatter and its local imports into the fixture.
|
||||
// This exercises production code while keeping all outputs in the sandbox.
|
||||
for (const relative of ['bin/gstack-office-hours-review', 'lib/office-hours-review.ts', 'lib/fs-atomic.ts']) {
|
||||
const target = path.join(planDir, relative);
|
||||
fs.mkdirSync(path.dirname(target), { recursive: true });
|
||||
fs.copyFileSync(path.resolve(__dirname, '..', relative), target);
|
||||
}
|
||||
const formatter = path.join(planDir, 'bin/gstack-office-hours-review');
|
||||
fs.chmodSync(formatter, 0o755);
|
||||
const formatter = copyOfficeHoursFormatter(planDir);
|
||||
// Two rounds keep the live run inside its budget: round 1 (full) and round 2 (delta) run here; round 3 and CONVERGENCE are covered by free validator tests.
|
||||
const reviewRounds = 2;
|
||||
const capture = await captureSectionReads({
|
||||
@@ -83,6 +78,7 @@ Completion delivery, after the full workflow and design approval:
|
||||
const reviewEvidence = validateOfficeHoursReviewArtifacts({
|
||||
...capture, designPath,
|
||||
designContent: fs.existsSync(designPath) ? fs.readFileSync(designPath, 'utf-8') : null,
|
||||
fixes: observedSpecFixes(capture.toolCalls),
|
||||
}, artifacts, snapshots, reviewRounds);
|
||||
validateOfficeHoursReviewerHandoffs({
|
||||
...capture, designPath,
|
||||
|
||||
@@ -30,6 +30,7 @@ import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { CAPTURE_MS } from './helpers/eval-budgets';
|
||||
import { NEUTRAL_POSTURE_RE } from './helpers/prosons-posture';
|
||||
import { runSkillTest } from './helpers/session-runner';
|
||||
import { resolveEvalModel } from '../lib/eval-model';
|
||||
import { runRecordedOfficeHoursAttempt, OFFICE_HOURS_BUN_GRACE_MS } from './helpers/office-hours-attempt';
|
||||
import {
|
||||
ROOT, runId,
|
||||
@@ -134,7 +135,7 @@ function setupPlanDir(tmpPrefix: string, planContent: string, skillName: string)
|
||||
}
|
||||
|
||||
function captureInstruction(outFile: string): string {
|
||||
return `Write the verbatim text of the single AskUserQuestion you would have made to ${outFile} (full text including D<N> header, ELI10, Stakes, Recommendation, Pros/cons, and Net line — the complete rich markdown body). Do NOT call any tool to ask the user. Do NOT paraphrase. This is a format-capture test.`;
|
||||
return `Write the text of the single AskUserQuestion you would have made to ${outFile} (full text including D<N> header, ELI10, Stakes, Recommendation, Pros/cons, and Net line — the complete rich markdown body). Do NOT call any tool to ask the user.`;
|
||||
}
|
||||
|
||||
// --- Case 1: Format positive — all v1.7.0.0 tokens present ---
|
||||
@@ -155,7 +156,7 @@ describeIfSelected('Plan Prosons — Format Positive', ['plan-review-prosons-for
|
||||
testConcurrentIfSelected('plan-review-prosons-format', async () => {
|
||||
await runRecordedOfficeHoursAttempt({
|
||||
collector: evalCollector, name: '/plan-review-prosons-format', suite: 'Plan Prosons — Format Positive',
|
||||
model: 'claude-opus-4-7', budgetMs: CAPTURE_MS,
|
||||
model: resolveEvalModel('capture'), budgetMs: CAPTURE_MS,
|
||||
run: signal => runSkillTest({
|
||||
signal,
|
||||
prompt: `Read plan-ceo-review/SKILL.md for the review workflow.
|
||||
@@ -170,7 +171,7 @@ After writing the file, stop.`,
|
||||
timeout: CAPTURE_MS,
|
||||
testName: 'plan-review-prosons-format',
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
}),
|
||||
validate: async (result, signal) => {
|
||||
logCost('/plan-review prosons format positive', result);
|
||||
@@ -217,7 +218,7 @@ describeIfSelected('Plan Prosons — Hard-stop Negative', ['plan-review-prosons-
|
||||
testConcurrentIfSelected('plan-review-prosons-hardstop-neg', async () => {
|
||||
await runRecordedOfficeHoursAttempt({
|
||||
collector: evalCollector, name: '/plan-review-prosons-hardstop-neg', suite: 'Plan Prosons — Hard-stop Negative',
|
||||
model: 'claude-opus-4-7', budgetMs: CAPTURE_MS,
|
||||
model: resolveEvalModel('capture'), budgetMs: CAPTURE_MS,
|
||||
run: signal => runSkillTest({
|
||||
signal,
|
||||
prompt: `Read plan-ceo-review/SKILL.md.
|
||||
@@ -232,7 +233,7 @@ After writing the file, stop.`,
|
||||
timeout: CAPTURE_MS,
|
||||
testName: 'plan-review-prosons-hardstop-neg',
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
}),
|
||||
validate: async (result, signal) => {
|
||||
logCost('/plan-review prosons hard-stop negative', result);
|
||||
@@ -270,7 +271,7 @@ describeIfSelected('Plan Prosons — Neutral-posture Negative', ['plan-review-pr
|
||||
testConcurrentIfSelected('plan-review-prosons-neutral-neg', async () => {
|
||||
await runRecordedOfficeHoursAttempt({
|
||||
collector: evalCollector, name: '/plan-review-prosons-neutral-neg', suite: 'Plan Prosons — Neutral Negative',
|
||||
model: 'claude-opus-4-7', budgetMs: CAPTURE_MS,
|
||||
model: resolveEvalModel('capture'), budgetMs: CAPTURE_MS,
|
||||
run: signal => runSkillTest({
|
||||
signal,
|
||||
prompt: `Read plan-ceo-review/SKILL.md.
|
||||
@@ -285,7 +286,7 @@ After writing the file, stop.`,
|
||||
timeout: CAPTURE_MS,
|
||||
testName: 'plan-review-prosons-neutral-neg',
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
}),
|
||||
validate: async (result, signal) => {
|
||||
logCost('/plan-review prosons neutral negative', result);
|
||||
@@ -324,7 +325,7 @@ describeIfSelected('Plan Prosons — Hard-stop Positive', ['plan-ceo-review-pros
|
||||
testConcurrentIfSelected('plan-ceo-review-prosons-cadence', async () => {
|
||||
await runRecordedOfficeHoursAttempt({
|
||||
collector: evalCollector, name: '/plan-ceo-review-prosons-cadence', suite: 'Plan Prosons — Hard-stop Positive',
|
||||
model: 'claude-opus-4-7', budgetMs: CAPTURE_MS,
|
||||
model: resolveEvalModel('capture'), budgetMs: CAPTURE_MS,
|
||||
run: signal => runSkillTest({
|
||||
signal,
|
||||
prompt: `Read plan-ceo-review/SKILL.md.
|
||||
@@ -339,7 +340,7 @@ After writing the file, stop.`,
|
||||
timeout: CAPTURE_MS,
|
||||
testName: 'plan-ceo-review-prosons-cadence',
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
}),
|
||||
validate: async (result, signal) => {
|
||||
logCost('/plan-review prosons hard-stop positive', result);
|
||||
|
||||
@@ -91,7 +91,7 @@ Focus on reviewing the plan content: architecture, error handling, security, and
|
||||
timeout: 540_000,
|
||||
testName: 'plan-ceo-review',
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
});
|
||||
|
||||
logCost('/plan-ceo-review', result);
|
||||
@@ -178,7 +178,7 @@ Focus on reviewing the plan content: architecture, error handling, security, and
|
||||
timeout: CAPTURE_LONG_MS,
|
||||
testName: 'plan-ceo-review-selective',
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
});
|
||||
|
||||
logCost('/plan-ceo-review (SELECTIVE)', result);
|
||||
@@ -246,7 +246,7 @@ Write your expansion proposals to ${planDir}/proposals.md with ONLY the proposal
|
||||
timeout: CAPTURE_LONG_MS,
|
||||
testName: 'plan-ceo-review-expansion-energy',
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
});
|
||||
|
||||
logCost('/plan-ceo-review (EXPANSION ENERGY)', result);
|
||||
@@ -355,7 +355,7 @@ Focus on architecture, code quality, tests, and performance sections.`,
|
||||
timeout: CAPTURE_LONG_MS,
|
||||
testName: 'plan-eng-review',
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
});
|
||||
|
||||
logCost('/plan-eng-review', result);
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { CAPTURE_MS, CAPTURE_LONG_MS } from './helpers/eval-budgets';
|
||||
import { runSkillTest } from './helpers/session-runner';
|
||||
import { resolveEvalModel } from '../lib/eval-model';
|
||||
import { outcomeJudge } from './helpers/llm-judge';
|
||||
import { judgePassed } from './helpers/eval-store';
|
||||
import {
|
||||
@@ -108,7 +109,7 @@ CRITICAL RULES:
|
||||
timeout: CAPTURE_MS,
|
||||
testName: `qa-${label}`,
|
||||
runId,
|
||||
model: 'claude-opus-4-7',
|
||||
model: resolveEvalModel('capture'),
|
||||
});
|
||||
|
||||
logCost(`/qa ${label}`, result);
|
||||
|
||||
@@ -75,7 +75,7 @@ for (const host of ['claude', 'codex'] as const) for (const args of [[], tenArgu
|
||||
const file = backslashPath ? join(root, 'checksum\\path', 'install-script') : join(root, 'install-script');
|
||||
mkdirSync(dirname(file), { recursive: true });
|
||||
writeFileSync(file, 'synthetic installer bytes\n');
|
||||
for (const name of ['open-gstack-browser', 'pair-agent', 'setup-browser-cookies']) {
|
||||
for (const name of ['open-gstack-browser', 'pair-agent']) {
|
||||
const lines = apply(skill(host, name)).split('\n').filter(line => line.includes('actual_sha=$('));
|
||||
expect(lines.map(line => line.trim())).toEqual([
|
||||
literals.checksum,
|
||||
|
||||
Reference in new issue
Block a user