CalvinBackup/gstack
1
0 Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-11-05 16:34:00 +01:00
Code Issues Packages Projects Releases Wiki Activity

v1.92.61.0 stamp: merge main 68195c372e into #3829 (land)

Browse Source
This commit is contained in:
garrytan committed 2026-11-02 05:42:48 +00:00
2 parents 1ec2f430d4 68195c372e
commit f0893cb592
170 files changed
+5907 -744
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/workflows/free-durations-record.yml
+2 -2
View File
@@ -164,7 +164,7 @@ jobs:
- Files that failed while recording (durations kept; this PR's own free checks decide): $failed_line
- Seed growth ratchet (60 s ceiling, shrink-only allowlist): $ratchet_line
Data only. This run dispatched Free Tests (the required free-tests check), Windows Free Tests and Quality gate on this branch; those are this PR's checks. The pull_request runs wait as "action required" because a token-opened PR never starts them without approval. Stamp and merge per docs/ship-policy.md; the stamp push runs the full PR checks.
Data only. This run dispatched Free Tests (the required free-tests check), Windows Smoke and Quality gate on this branch; those are this PR's checks. The pull_request runs wait as "action required" because a token-opened PR never starts them without approval. Stamp and merge per docs/ship-policy.md; the stamp push runs the full PR checks.
EOF
if [ -n "$number" ]; then
gh pr edit "$number" --body-file pr-body.md
@@ -172,5 +172,5 @@ jobs:
gh pr create --base main --head "$branch" --title "test(seed): refresh the free-suite duration seeds" --body-file pr-body.md
fi
gh workflow run free-tests.yml --repo "$GITHUB_REPOSITORY" --ref "$branch"
gh workflow run windows-free-tests.yml --repo "$GITHUB_REPOSITORY" --ref "$branch"
gh workflow run windows-smoke.yml --repo "$GITHUB_REPOSITORY" --ref "$branch"
gh workflow run quality-gate.yml --repo "$GITHUB_REPOSITORY" --ref "$branch"
.github/workflows/windows-free-tests.yml
+18 -20
View File
@@ -1,9 +1,15 @@
name: Windows Free Tests
# The Windows-safe subset of the free suite on GitHub's free `windows-latest`
# runners. scripts/test-free-shards.ts --windows-only is the single source of
# truth for which files run: POSIX-bound tests (bin/ shebang spawns, raw /tmp
# paths, /bin/sh, mode bitmasks, the Bun-driven browse server) are excluded by
# Nightly, not a PR gate. Windows has minimal support (docs/support-tiers.md):
# the required Windows check is windows-smoke.yml (./setup plus the core
# skills). This workflow runs the curated Windows-safe subset of the free suite
# on main every night and on workflow_dispatch, and never blocks a pull
# request. A red nightly run gets a fix only when it shows an install, startup
# or core-workflow break; other Windows-only failures close as won't fix.
#
# scripts/test-free-shards.ts --windows-only is the single source of truth for
# which files run: POSIX-bound tests (bin/ shebang spawns, raw /tmp paths,
# /bin/sh, mode bitmasks, the Bun-driven browse server) are excluded by
# pattern there, with explicit force-include and known-incompatible lists.
# Every run prints the curated count and top exclusion reasons in the job
# summary so a shrinking subset is visible.
@@ -27,13 +33,9 @@ name: Windows Free Tests
# dispatch and duration-recording runs never consume.
on:
pull_request:
branches: [main]
# Also on main pushes, as free-tests.yml does: trains dispatch only the
# Linux free workflow, so without this no Windows run ever sees a train's
# combined tree or main after a merge.
push:
branches: [main]
# 09:17 UTC, about 2 AM Pacific, on the default branch (main).
schedule:
- cron: '17 9 * * *'
workflow_dispatch:
inputs:
record_durations:
@@ -113,8 +115,10 @@ jobs:
if-no-files-found: error
windows-free-shard:
# gstack-pregate reads a job keyed on a dispatch input as manual, never a
# PR obligation; this lane runs only nightly or by hand.
needs: [windows-plan, reuse]
if: ${{ !cancelled() && needs.windows-plan.result == 'success' && (needs.reuse.result != 'success' || needs.reuse.outputs.reused != 'true') }}
if: ${{ !cancelled() && !inputs.record_durations && needs.windows-plan.result == 'success' && (needs.reuse.result != 'success' || needs.reuse.outputs.reused != 'true') }}
runs-on: windows-latest
timeout-minutes: 15
strategy:
@@ -139,14 +143,8 @@ jobs:
name: windows-plan
path: ${{ runner.temp }}
# bun's install cache lives under ~/.bun/install/cache on every platform.
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
with:
path: ~/.bun/install/cache
key: windows-bun-${{ hashFiles('bun.lock') }}
# A lockfile bump starts from the previous cache instead of cold.
restore-keys: |
windows-bun-
# No bun install cache: restoring the ~450 MB entry took 33-42 s per
# shard, a cold install 16-17 s (GSTA-203 PR 1 A/B, 3 runs per arm).
- name: Configure git identity (required by tests that init temp repos)
run: |
.github/workflows/windows-setup-e2e.yml
+2 -10
View File
@@ -92,16 +92,8 @@ jobs:
with:
bun-version: 1.4.2
# Same lockfile-keyed install cache as windows-free-tests.yml (install
# was 45s of a 64s job, all network).
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
with:
path: ~/.bun/install/cache
key: windows-bun-${{ hashFiles('bun.lock') }}
# A lockfile bump starts from the previous cache instead of cold
# (restore alone costs ~43s; without this a bump pays it for nothing).
restore-keys: |
windows-bun-
# No bun install cache, as in windows-free-tests.yml: a cold install is
# faster on windows-latest than restoring the ~450 MB cache entry.
- name: Configure git identity
run: |
.github/workflows/windows-smoke.yml
+170
View File
@@ -0,0 +1,170 @@
name: Windows Smoke
# The required Windows check under minimal support (docs/support-tiers.md):
# with a default Git for Windows install, ./setup installs and the core skills
# start. One windows-latest job, under 5 minutes:
# 1. ./setup --host claude into an isolated HOME under Git Bash, with
# gstack's own browser (Playwright Chromium).
# 2. The core skills start: gstack-skill-start for /ship, /review,
# /qa-only, /investigate and /browse answers SKILL_START_PROTO: 1.
# 3. /browse on gstack's own browser: the installed browse runs a piped
# chain flow against a local page from an unrelated directory.
# 4. The fastest Windows-safe free tests for those paths (list below).
# The full curated Windows suite is windows-free-tests.yml: nightly on main
# and on dispatch, never a PR gate.
#
# No bun install cache: a cold install took 16-17 s on windows-latest, a
# restore of the ~450 MB cache 33-42 s (GSTA-203 PR 1 A/B, 3 runs per arm).
on:
pull_request:
branches: [main]
push:
branches: [main]
workflow_dispatch:
concurrency:
group: windows-smoke-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
permissions:
contents: read
jobs:
windows-smoke:
runs-on: windows-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.4.2
- name: Configure git identity (required by tests that init temp repos)
run: |
git config --global user.email "windows-smoke@gstack.test"
git config --global user.name "Windows Smoke"
git config --global init.defaultBranch main
shell: bash
- name: Install dependencies
run: bun install --frozen-lockfile
shell: bash
- name: ./setup installs into an isolated HOME
run: |
set -euo pipefail
SMOKE_HOME="$RUNNER_TEMP/smoke-home"
mkdir -p "$SMOKE_HOME"
HOME="$SMOKE_HOME" USERPROFILE="$(cygpath -w "$SMOKE_HOME")" ./setup --host claude
test -x "$SMOKE_HOME/.claude/skills/gstack/bin/gstack-skill-start" || (echo "MISSING: installed gstack-skill-start" && exit 1)
shell: bash
- name: The core skills start
run: |
set -euo pipefail
SMOKE_HOME="$RUNNER_TEMP/smoke-home"
USERPROFILE="$(cygpath -w "$SMOKE_HOME")"
export HOME="$SMOKE_HOME" USERPROFILE
for skill in ship review qa-only investigate browse; do
set +e
OUT=$(timeout 120 "$SMOKE_HOME/.claude/skills/gstack/bin/gstack-skill-start" --skill "$skill" --model claude 2>&1 < /dev/null)
CODE=$?
set -e
echo "::group::$skill"; echo "$OUT"; echo "::endgroup::"
test "$CODE" -eq 0 || (echo "/$skill: gstack-skill-start exited $CODE" && exit 1)
grep -q '^SKILL_START_PROTO: 1' <<< "$OUT" || (echo "/$skill: no SKILL_START_PROTO: 1 line" && exit 1)
done
shell: bash
- name: /browse on gstack's own browser runs a piped chain flow from an unrelated directory
run: |
set -euo pipefail
SMOKE_HOME="$RUNNER_TEMP/smoke-home"
USERPROFILE="$(cygpath -w "$SMOKE_HOME")"
export HOME="$SMOKE_HOME" USERPROFILE
BROWSE="$SMOKE_HOME/.claude/skills/gstack/browse/dist/browse.exe"
[ -f "$BROWSE" ] || BROWSE="$SMOKE_HOME/.claude/skills/gstack/browse/dist/browse"
test -f "$BROWSE" || (echo "MISSING: installed browse" && exit 1)
bun -e 'Bun.serve({ hostname: "127.0.0.1", port: 47124, fetch: () => new Response("<title>gstack smoke</title><h1>gstack-windows-smoke-ok</h1>", { headers: { "content-type": "text/html" } }) }); setInterval(() => {}, 1 << 30);' &
PAGE_PID=$!
trap 'kill "$PAGE_PID" 2>/dev/null || true; "$BROWSE" stop >/dev/null 2>&1 || true' EXIT
mkdir -p "$RUNNER_TEMP/unrelated-dir" && cd "$RUNNER_TEMP/unrelated-dir"
set +e
OUT=$(printf '%s\n' '[["goto","http://127.0.0.1:47124/"],["text"]]' | "$BROWSE" chain 2>&1)
CODE=$?
set -e
echo "$OUT"
if [ "$CODE" -ne 0 ]; then
find "$RUNNER_TEMP/unrelated-dir" "$SMOKE_HOME" -maxdepth 4 \( -name 'browse-daemon.log' -o -name 'browse-startup-error.log' \) -print -exec tail -n 60 {} \; 2>/dev/null || true
echo "browse chain exited $CODE" && exit 1
fi
grep -q 'gstack-windows-smoke-ok' <<< "$OUT" || (echo "chain output lacks the page text: the flow did not run" && exit 1)
shell: bash
- name: Fastest Windows-safe free tests for install and the core skills
# Windows-measured seconds from scripts/free-test-durations-windows.json.
# setup-windows-fallback (0.1): setup's _link_or_copy copies, never links, on Windows
# setup-staged-copy (0.6): a copy install is staged and swapped in only when complete
# setup-windows-rerun-refresh (1.7): re-running ./setup refreshes copied skills (#2444)
# setup-skill-copies (0.1): copied SKILL.md hashes and the backup of an edited copy
# setup-bun-cmd-and-pipe-bugs (0.4): setup's bun.cmd and pipe handling
# gstack-launch-probe (0.1): setup's Smart App Control launch probe (#2595)
# state-root-parity (2.1): every skill start resolves one state root in TS and bash
# bun-isolation-launch (1.1): Bun entries ignore .env and bunfig under Git Bash (#3415)
# gstack-lease-git-bash (3.7): the skill-start lease under Git Bash (#3330)
# gstack-wtree-windows-paths (1.3): /review and /ship record a start on C:/ paths (#3101)
# ship-hook-windows-paths (2.3): /ship's pre-push guard in a Windows worktree (#2805)
# qa-only-browser-probe (2.4): /qa-only's browser probe fixture
# find-browse (0.1): /browse resolves its binary (.exe)
# cli-chain-stdin (0.3): /browse chain reads its flow from stdin on Windows (#3039)
# windows-spawn-hide (0.1): /browse's spawns hide their console window (#1835)
env:
GSTACK_FREE_RETRY_FLAKY: '1'
GSTACK_FLAKE_LEDGER: ${{ runner.temp }}/flake-ledger.jsonl
TEMP: ${{ runner.temp }}
TMP: ${{ runner.temp }}
run: |
set -euo pipefail
cat > "$RUNNER_TEMP/windows-smoke-files.txt" <<'LIST'
test/setup-windows-fallback.test.ts
test/setup-staged-copy.test.ts
test/setup-windows-rerun-refresh.test.ts
test/setup-skill-copies.test.ts
test/setup-bun-cmd-and-pipe-bugs.test.ts
test/gstack-launch-probe.test.ts
test/state-root-parity.test.ts
test/bun-isolation-launch.test.ts
test/gstack-lease-git-bash.test.ts
test/gstack-wtree-windows-paths.test.ts
test/ship-hook-windows-paths.test.ts
test/qa-only-browser-probe.test.ts
browse/test/find-browse.test.ts
browse/test/cli-chain-stdin.test.ts
browse/test/windows-spawn-hide.test.ts
LIST
bun run scripts/test-free-shards.ts --files-from "$RUNNER_TEMP/windows-smoke-files.txt"
shell: bash
- name: Upload full test logs
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7
with:
name: windows-smoke-logs
path: |
.context/free-test-logs/gstack-free-test-*.log
${{ runner.temp }}/gstack-free-test-*.log
include-hidden-files: true
if-no-files-found: ignore
- name: Upload flake ledger
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7
with:
name: flake-ledger-windows-smoke
path: ${{ runner.temp }}/flake-ledger.jsonl
if-no-files-found: ignore
retention-days: 90
.gstack/pregate.json
+1 -2
View File
@@ -16,8 +16,7 @@
"supabase/**": []
},
"selection": {
"free-tests.yml/free-suite": "bun run scripts/test-free-shards.ts --list",
"windows-free-tests.yml/windows-free-shard": "bun run scripts/test-free-shards.ts --windows-only --list"
"free-tests.yml/free-suite": "bun run scripts/test-free-shards.ts --list"
},
"run_profiles": {
"ci-reuse/free-tests": { "workflow": "free-tests.yml", "events": ["pull_request", "workflow_dispatch"], "jobs": ["free-plan", "typecheck", "quick-bun-default", "cso-macos-launcher", "macos-named-regressions", "cso-windows-launcher", "cso-docker-integration", "free-tests"], "artifact": "tested-tree", "os": "Linux" },
.gstack/ship-policy.json
+1 -1
View File
@@ -14,6 +14,6 @@
"pr_count": { "default_pr_count": 1, "split_requires_owner": true },
"maintainers": ["garrytan"],
"train": { "gate_workflow": "evals.yml", "free_workflow": "free-tests.yml", "full_gate_check": "gstack/full-gate", "record_artifact": "gstack-train-record", "max_size": 5, "fill_hours": 1, "screen_selection": "gstack-paid", "trial_reservation_usd": 1.5 },
"changelog_fragments": { "dir": "changelog.d", "required_checks": ["free-tests", "windows-free-tests", "Check VERSION is not stale vs queue", "quality", "check-freshness", "actionlint"] },
"changelog_fragments": { "dir": "changelog.d", "required_checks": ["free-tests", "windows-smoke", "Check VERSION is not stale vs queue", "quality", "check-freshness", "actionlint"] },
"min_gstack_version": "1.91.340.0"
}
AGENTS.md
+5 -3
View File
@@ -285,9 +285,11 @@ Support tiers (full: macOS, Linux; minimal: Windows, Gemini models) are in
[docs/support-tiers.md](docs/support-tiers.md).
- **macOS** + **Linux**: full test suite supported.
- **Windows**: the curated Windows-safe subset runs in the `windows-free-tests`
CI workflow across six `windows-latest` jobs, packed by Windows-measured
durations. Setup script (`./setup`) requires Git Bash or
- **Windows**: minimal support; [docs/support-tiers.md](docs/support-tiers.md)
says what is promised. Every PR must pass the `windows-smoke` check (`./setup`,
the core skills start, the fastest Windows-safe tests); the curated
Windows-safe subset runs nightly on main in `windows-free-tests` and never
blocks a PR. Setup script (`./setup`) requires Git Bash or
MSYS today; native PowerShell support is a future expansion. The `bin/gstack-paths`
helper resolves state roots through `CLAUDE_PLUGIN_DATA` / `GSTACK_HOME` so plugin
installs work on every platform.
CHANGELOG.md
+174
View File
@@ -1,5 +1,179 @@
# Changelog
## [1.92.61.0] - 2026-11-02
- **The security class reaches outbound URLs, archives and every `child_process` spelling** (GST-850). `lib/review-risk.ts` now flags any call to an outbound HTTP client (`fetch`, `http(s).get` or `.request`, axios, got, undici, `requests`, `httpx`, urllib), `new URL` from input, redirect toggles, proxy settings and host allowlists; tar, zip and unzip extraction, `path.join` or `resolve` on request or entry input, and `'..'` checks; and any changed line that names the `child_process` module. The new rules parse no import clause or call arguments, and a test holds each under 100 ms on 200,000-character hostile lines.
- **The money class reads metering and charge code.** Paths and lines naming meter, metering, metered, charge, billable, billed or credits, plus token usage, a billing proxy and cost or charge clients, now route to the money reader. gbrain-evals #289, which changed a metering proxy and billing clients and read `required=none`, is now a regression test that reads money. "costume", a CLI's usage text and a plain proxy do not count.
- **When a diff publishes numbers, the batched specialist reads the write-up's text files** instead of leaving docs out of its diff. The collector records the files a diff creates.
- **UI text and SVGs stay code (light).** An allowlist docs path for chart SVGs and locale values was built and dropped for review cost: its first review round found a new bypass class in each version. Every bypass found so far is a test row that classes as code.
- Replayed on the 26 GSTA-167 no-loss units, no unit loses a reviewer role. The replay checked which reviewers get dispatched, not R51's re-scored findings. Over the last 100 main commits, 3 move from light to full and no commit with a gate or risk class drops depth.
## [1.92.60.0] - 2026-11-02
Free test shards can now run their files as several sub-shards side by side inside one CI job. `bun run scripts/test-free-shards.ts --ci-run <plan> --shard <i> --result <file> --jobs <k>` splits the shard into k parts that each run in their own sandboxed `bun test` process. The default stays at 1, so CI runs exactly as before until a workflow opts in. Bad `--jobs` values stop with `FREE_JOBS_INVALID`, a sub-shard that runs out of time prints `FREE_SUBSHARD_TIMEOUT` with a serial reproduce command, and every CI shard result now records per-file test and `expect()` counts. See docs/TESTING_INTERNALS.md#sub-shards-inside-one-ci-job.
## [1.92.59.0] - 2026-11-02
**`test:health` and the factory scorecard leave reused main pushes out of their free-suite flake counts (#3725).** Since #3709, a main push whose merged pull request already passed on the same tree reuses that run. It finishes green, runs no shards and uploads nothing, so both readers counted it as a run with zero flakes. As reuse grows, that pulls real flake rates under the 5% bar, and `bun run test:health --enforce` stops flagging files without any error.
### Fixed
- **One rule for both readers** (`reuseChecker` in `scripts/lib/ci-history.ts`). A run with no artifacts and a green `reuse` job is a reused main push. `readCiFlakeLedgers` leaves it out of the window and its denominator and fills the window with older runs, so it still holds 20 runs that ran their shards. `readFreeRetries` in `lib/factory-scorecard.ts` leaves it out of the free-retry count. Both reports name how many runs they left out.
- **Bounded and visible.** A run with any artifact costs no extra call. A run with none costs one job listing, at most 40 per read. Past that cap, or when a listing fails, the run counts as it did before and the report says so.
### Tests
- `test/test-health-report.test.ts`: reused runs among real flaky runs (2 flaky of 20 real runs, with 30 reused) failed on the old code with 2/45 = 4.4%, which passed the 5% bar; now 2/20 = 10% fails it. The lookup cap and a failed lookup both count the run as before, and the report says so.
- `test/factory-scorecard.test.ts`: two reused runs in the window failed on the old code ("in 1 of 5 runs"); now the free-retry line matches the real runs only ("in 1 of 3 runs") and names the 2 runs it left out.
## [1.92.58.0] - 2026-11-02
**A parent agent can now run the CEO-only review through the driver: `gstack-autoplan next --ceo-only` (GST-537).** `/autoplan --ceo-only` only worked inside a Claude Code session, so a crew or eval harness driving `gstack-autoplan` from another host had to run the whole review and stop at the approval gate. Now the same review-only run works from the driver on any host.
### Added
- **`gstack-autoplan next --ceo-only`** (first `next` only): opens the CEO phase alone. When it closes, the run prints `REVIEWED:`, writes `<dir>/ceo-review.md` with the same sections as the skill path (hashed in `run.json`), and ends `GSTACK_RESULT: skill=autoplan status=reviewed run=<dir>`. It writes no gate, never edits the plan and never approves anything.
- **Decision rows with status `auto_kept`**: each item a gate would have asked is recorded as kind `auto`, the plan's direction kept, with no answer time. An item only the owner can accept (a Critical finding left open when CEO closed deferred) is left undecided, with no chosen option, and the report lists it as left open for the owner. Tasks in a CEO-only run wait on no decision. `run.json` now accepts `reviewed`, and `gstack-artifact validate` passes a finished CEO-only run.
- **Coded refusals**: `--ceo-only` beside `--ui`, `--developer-tool` or `--agent-primary` (on the command line or in the plan's scope marker), a scope marker that names `--ceo-only`, or a plan that is the report file `<dir>/ceo-review.md` (directly, through a symlink or a hard link) refuses with `CEO_ONLY_CONFLICT`, so the report never overwrites the plan; on a directory that holds a full run, `CEO_ONLY_AFTER_INIT`. `answer`, `recommend`, `revise` and `gstack-approvals import` refuse a CEO-only run with `CEO_ONLY_NO_GATE`. A CEO-only run stopped by `--spend-cap` recovers the usual way: `budget` then `resume`, or `close --deferred`, which ends it `reviewed`.
### Fixed
- `gstack-approvals import --autoplan` no longer imports from a run whose `run.json` is present but not valid JSON; it refuses with `ARTIFACT_INVALID_JSON` instead of guessing the run's identity.
## [1.92.57.0] - 2026-11-02
**Hermes can start gstack skills again (GST-882).** On stock Hermes, the Tirith scanner flagged every gstack skill's first command as HIGH ("Nested executable body could not be resolved"), so `gstack-skill-start` was denied in 8 of 8 trials. Tirith-only findings can't go on Hermes's permanent allowlist, so there was no workaround. The start line Tirith couldn't follow was a grouped fallback that picked the script through a variable.
### Fixed
- **The skill-start line on Codex, Hermes and the other env-var hosts is plain shell.** A skill installed for one folder (Hermes, and the Codex install `./setup` writes) now checks for that folder's `bin/gstack-skill-start` and then runs it by its literal path. Tirith 0.4.2 allows that line where it blocked the old one. A checked-in render keeps the same lookup order (an exported `GSTACK_ROOT`, then the repo's own install, then the global one) as plain lines, without the `{ …; exit 1;}` group. When no install exists, both print the same "no install found" line with the setup command, and the start command then fails.
- **An install-root render's first block no longer honors an exported `GSTACK_ROOT`.** Tirith blocks any start command picked through a variable. Later blocks still honor it. `docs/troubleshooting.md` names this exception.
### Tests
- `test/skill-start-plain.test.ts`: on every host and both render kinds, the start fence has no grouped fallback, and a missing install fails loudly under `bash -u`. The install-root start command is a literal path that runs, and a checked-in render finds the global install. On each env-var host, the start lookup picks the same root as later blocks for global, repo-local and exported installs.
## [1.92.56.0] - 2026-11-02
**`gstack-ship-receipt write` records the session kind on Windows.** Without `--session-kind` or `GSTACK_SESSION_KIND`, it asks the `gstack-session-kind` bash helper. It used to spawn that helper by path, which never starts on Windows, so every /ship receipt there carried no `session_kind`. It now starts the helper through `lib/script-invocation.ts`, which uses Git Bash on Windows. This is the /ship half of #3541.
### Fixed
- **`bin/gstack-ship-receipt`** starts `gstack-session-kind` with `runScriptSync`.
### Tests
- `test/ship-receipt-session-kind.test.ts` (windows-free-tests too): a receipt written without a session kind records what `gstack-session-kind` prints.
## [1.92.55.0] - 2026-11-02
- **`/ship` finish no longer stops on a branch whose restamp moved its own stale stamp after a lost slot** (`gstack-ship-release finish`, GST-895). When a branch loses its merge slot, merges main and runs `gstack-restamp --next`, the restamp moves the branch's own `since:` rows from the lost version to the new one. Finish only accepted a moved version at or above the reviewed tree's VERSION, which now came from main, so it read those rows as a behavior change and stopped with "insert 5–11.5 → 12–14 → 16" (gbrain #8134 on 1.92.46.0). Now a line the branch added itself, by the same rule restamp uses (added since the merge base with main, and not added by main), may move any version newer than the branch's fork point, and finish routes it as release metadata. A row main carries, a row main edited after the merge, and a branch row at or before the fork point still stop. Only `stamp_files` paths count, and nothing else on the line may change.
### Tests
- `test/ship-rereview-triggers.test.ts`: after a lost slot, a merge of main and a restamp, the branch's moved row routes as metadata; moving main's own row, a merged main row that main later edited, a branch row at the fork version, or one older than the fork point each stop. The first case fails on main with `behavior=1`.
## [1.92.54.0] - 2026-11-02
**`train land` and a single-PR `land` can no longer publish onto main at the same time, and a slot that forgot its version bump can no longer merge (#3599, #3531; GSTA-157).** Before, `train land` took no lock, so it and `land` could both read the same main and publish a combined tree neither tested. And train 33's `verify --train` said "may merge" for #3527's head after a bare commit dropped the stamp that restamp had written but not staged, so #3527 shipped under #3526's version.
### Fixed
- **`train land` takes the shared land lock** (`refs/notes/gstack-land`, now in lib/ship-queue-land-lock.ts) as `train <nonce> slot <i> (#<pr>)`. Whichever of `land` and `train land` starts second refuses with `QUEUE_LAND_LOCKED`, naming the holder. Once a slot has merged, the train keeps the lock in phase `between` until its last slot lands, through a refusal on the next slot too, so no single land slips in between slots and sets off a freeze. A stop that freezes the queue leaves the lock in phase `freeze`, which only the admitted repair train takes over while the freeze lasts; once the freeze is cleared, the next `land` or `train land` takes over a freeze lock whose release push failed. A lock ref deleted on origin now reads as no lock, not as this clone's cached copy.
- **`--break-lock` prints the holder** (`LOCK_HOLDER:`), keeps land's staleness, mid-push and drift rules, never breaks a `freeze` lock, waits 10 minutes on a `between` lock, and refuses a live train run's lock unless `--break-train-lock` is also given. `train land` accepts both flags.
- **`verify --train` checks the slot stamp itself**: the head's VERSION must be above main's and its CHANGELOG headed with that version directly above main's top release, or it refuses with the new `QUEUE_TRAIN_UNSTAMPED`, naming the rule and both versions. It prints `STAMP_VERSION:` from the head.
- **`verify --train` calls a PR merged only when GitHub reports it MERGED.** An open release-only PR (#3530's shape) now gets the pre-merge check, and any other open or unreadable PR refuses with `QUEUE_PROOF_MISSING` without freezing the queue. So does a verify that could not read the PR's current head (`--no-fetch` or a failed fetch), instead of judging the unstamped pin.
- **`gstack-restamp` stages the files it writes** and prints `STAGED:`. It refuses before writing when a stamp file holds unstaged edits of your own (`RESTAMP_UNSTAGED_EDITS`); when staging fails, it says `RESTAMP_NOT_STAGED` and the next run stages the files.
- Two comments in lib/ship-queue-train-gate.ts now say "one spend cap per command gate across every prefix of the train", which is what the code does.
### Tests
- `test/ship-queue-land-lock.test.ts`: two writers racing one lock sha, the four lock read states, owner kinds, the `--break-lock` rules, and both lock directions, the between phase, refusal and resume, and the freeze lock's repair takeover through `bin/gstack-ship-queue`.
- `test/ship-queue-train.test.ts`: the #3527 and #3530 shapes, the CHANGELOG rule, a claim-at-open pin, and no false freeze. `test/gstack-restamp-queue.test.ts`: staging after a no-commit merge, the unstaged-edits refusal, and a staging failure and its rerun.
## [1.92.53.0] - 2026-11-02
**`reap` and `up` now destroy only your own Ubicloud VMs (GST-573).** Before, every `up` destroyed any VM in the project whose TTL had ended, whoever created it, which broke the rule that cloud deletes stay with the owner who made the resource. Now reap destroys only the caller's own expired VMs and its own old TTL-less ones, and only counts other owners' expired VMs in its log (`reap: left N expired VMs of other owners; only their owners reap them`).
### Changed
- **Owner-scoped reap.** Another owner's VM, even an expired one, and an untagged one are never reaped. A lane's leftover VM is destroyed by that owner's own next `up` or `reap`, so a lane that never runs again leaves its VM until someone with that owner runs `reap` or `down`.
## [1.92.52.0] - 2026-11-02
**/ship on Capy runs Exploratory QA again.** Since v1.91.345.0 (#3648) the driver sent Step 9.2.1 out as a QA reader subagent, but the section says only the parent runs report-only discovery, so the reader refused on every pass ("parent-only") and the parent ran the probes afterward anyway, an extra reader call each pass (the C4-36 and C5-36 replays).
### Fixed
- **The parent runs Step 9.2.1's probes itself, as the section says.** Phase 3 does the section's method Reads and no probe; Phase 4's one program runs the smoke and plan checks after Step 8.1 collects the plan checks and before Scope Check, dedup and Fix-First, and the record's Checks heading carries the `Exploratory QA:` line. The call count is unchanged, and the section text is untouched.
- **`gstack-review-dispatch pass` refuses a QA extra.** An extra named `qa`, or one whose prompt holds Step 9.2.1's "Only the parent runs report-only discovery.", stops the pass before any reader is created. A prompt file that cannot be read stops as an unreadable prompt, not a raw read error. The Capy skill lists Steps 7, 8 and 14.5 as extras, not Step 9.2.1.
## [1.92.51.0] - 2026-11-01
**PRs no longer wait on the six-shard Windows suite. A short `windows-smoke` check is the required Windows check, and the full suite runs nightly on main.** Windows has minimal support (docs/support-tiers.md). Since 2026-11-01 06:00 UTC, Windows Free Tests finished last on 71 of 73 PR heads, with a wall time of 892 s at p50 and 1,886 s at p90 over 71 green PR runs, and each run used about 59 Windows runner-minutes. The new smoke check finished in 198 s on this PR (run 41325664750). With smoke in its place, the slowest required check on a PR drops from 892 s to 562 s at p50 and from 1,877 s to 1,193 s at p90.
### Changed
- **New `windows-smoke.yml`** (check `windows-smoke`, on pull requests, pushes to main and by hand). It installs gstack with `./setup --host claude` into a separate home, starts /ship, /review, /qa-only, /investigate and /browse with `gstack-skill-start`, runs the installed browse chain against gstack's own browser, and runs 15 Windows-safe test files. Each file's reason is written next to it in the workflow. No cache: a cold `bun install` took 16 to 17 s, while restoring the cache took 33 to 42 s in a three-round A/B.
- **`windows-free-tests.yml` runs nightly** (09:17 UTC) on main, and by hand. It no longer runs on pull requests or pushes and never blocks a merge. A red nightly gets a fix only when it shows a broken install, startup or core workflow. Other Windows-only failures close as won't fix with a link to docs/support-tiers.md.
- **Bun cache removed** from `windows-free-tests.yml` and `windows-setup-e2e.yml`. The 18 cache entries on pull-request refs (8.05 GB) were deleted. Main's entries were left alone.
- **`gstack-ship-queue ready --dispatch-ci`** skips a named workflow that main has but the PR branch predates, and prints a note instead of failing. GitHub reads the workflow file from the branch, so dispatching `windows-smoke.yml` on an older branch could only fail.
### Gate integrity
- `.gstack/ship-policy.json` `required_checks`: `windows-free-tests` is replaced by `windows-smoke`. The ci-hint, `--dispatch-ci` docs, train CLI help and `free-durations-record.yml` now name `windows-smoke.yml`.
- `.gstack/pregate.json`: the `windows-free-tests.yml/windows-free-shard` remote lane is removed, since that suite no longer runs on PRs. Linux and macOS lanes are unchanged.
- `scripts/test-health-report.ts` measures Windows from `windows-smoke.yml` PR runs.
- No test file was deleted, skipped or re-tiered. `scripts/harness-version.json` records `scripts/test-pr-profile.ts` as a non-behavioral change, because it only lists `windows-smoke.yml` among files no paid case reads.
## [1.92.50.0] - 2026-11-01
### Fixed
- **`/autoplan` no longer locks a Claude Code session after a snapshot init run as `cd <dir> && …` (#3651).** When that `cd` leaves the session's folder, Claude Code adds a `Shell cwd was reset to …` line to the command's output. The guard now removes exactly that one line before reading the init or checkpoint JSON; any other extra text still fails. An init whose output still can't be read now counts as a failed init with a "re-run" hint, so a later clean init in the same session works again. Before, every later phase read was denied with code `evidence` until a new session.
### Fixed
- **`/eval-plan` pilots with a subscription arm can now be approved, and the cost projection is per arm (#3653).** `gstack-eval-plan pilot start --seat-metered <arm>` declares an arm that runs on a subscription with no extra charge (for example `claude -p` on a seat). Its attempts hold none of the spend cap, settle as `unknown` (an arm that settles a real charge is projected like any priced arm), and get their own `PILOT_SEAT_METERED:` line, so the verdict covers the arms that cost money. `pilot report` now projects each arm from its own attempts (one `PILOT_ARM:` line each) and adds them up. Before, it averaged all arms together: a priced arm's cost was multiplied across arms with no price, and two arms at different prices got a band so wide that a pilot that fit the cap was told it exceeded it. The skill also says to stop a seat-metered run at its first quota or rate-limit error, with no retry (idea from #3273 by @merit-blake).
**`/autoplan --ceo-only` runs the CEO review by itself and asks nothing from start to finish (GST-537).** Garry asked for a CEO review that "just automatically does the right thing", and the planning eval needed a gstack CEO arm that a harness can run without a simulated user. Before, /autoplan always ran the Eng review too and stopped at a final approval question, and standalone /plan-ceo-review asks for the mode, offers to chain to the next skill and adds a Codex voice.
### Added
- **`/autoplan --ceo-only`** (`autoplan/sections/ceo-only.md`): runs Phase 0, the outside-reviewer preflight and the CEO phase only. Every question takes its recommendation and gets an audit-trail row; consent and destructive options take the conservative choice. The review mode comes from plan-ceo-review's 0E rule. User challenges are never applied; each one is listed at the top of `<run dir>/ceo-review.md`, with scope added, conservative choices, findings and the audit trail. The run ends with `GSTACK_RESULT: skill=autoplan status=reviewed run=<dir>` and approves nothing.
- **`--no-outside`** with `--ceo-only` skips the outside voice, so one model does the whole review.
- **`gstack-ceo-mode-handoff --ceo-only`** prints `Auto-selected review mode → <mode> (CEO-only run: Step 0E's rule)` instead of crediting a saved preference.
- **`status=reviewed`** joins the `GSTACK_RESULT` statuses (`docs/unattended.md`, "CEO-only runs").
### For contributors
- The paid case `autoplan-ceo-only` fails if the mode asks anything, skips a later phase's file, or leaves new files in HOME outside its folders. It takes 6 to 16 minutes, so it runs in the weekly marathon lane, not on autoplan PRs. The mode needs Claude Code for now.
**/autoplan fixes from a field report (GBRA-223): findings without a line number pair across voices, open Critical and High findings reach the gate, a spend cap binds on any host, `run` works with the API runner, and an earlier item-by-item answer carries forward.** OpenClaw's /autoplan run on a real plan hit six problems at once. We reproduced each on main with a failing test first (GST-672). A follow-up from GEVA-2 adds a record of whose words answered a gate.
### Fixed
- **Plan reviews that name a file but no line now pair across voices.** A same-file pair, with a line missing on either side, is associated as `unverified` instead of both findings counting as new. Title matching ignores a leading DX cap code (`CAP4:<path>:`) and never pairs findings on two different files. The same-file fallback runs only after every title and line match is placed, and a same-file pair never merges two findings into one gate item.
- **DX cap findings carry real titles.** `CAP4` and `CAP6` findings are titled `CAP<n>:<path>:<item> <what fails>`, so the gate shows the problem and not only the item name.
- **Every open Critical or High finding becomes a gate item.** `a` applies the reviewer's fix to the plan and is recommended. `b` carries it into the build as a blocking task. `rec` and `p1b` apply the fix. An unanswered item keeps the plan as written unless the reply is `p1b`, and a partial reply never locks that choice early. A finding carried with `b` stays open, so a rerun that raises it again asks again.
- **A spend cap binds on any host.** With `--spend-cap` set and no `--estimate-usd`, each host-subagent attempt reserves the default $5 per voice, on Capy or off it, in any session kind, and so does an API call whose model has no price. `next --estimate-usd <n>` sets the reservation for the rest of the run. `run` reserves each API call's priced worst case (prompt size plus the output limit) and settles it at its real token cost, so `--spend-cap 0` refuses before any call.
- **`gstack-autoplan run` with `--native-runner api --runner api` reaches the gate.** Results are checked as autoplan results, so they bind. A response cut off at the output limit is named `truncated` and interrupts the run instead of binding a partial review. The default output limit is 16384 tokens for both providers (OpenAI counts reasoning tokens toward it), `--max-tokens <n>` sets it, and an outside result refused at submit leaves the run native-only instead of aborting it.
- **An earlier item-by-item answer carries forward when nothing changed.** When every item on the new gate is carried by origin, the owner's `p1` choice is replayed and their tokens are recorded verbatim as the basis. Any new or changed item still goes to the owner.
- **A spend cap stays closed when a call fails.** For the native voice, a call cut off at its output limit is settled at what it cost, and a retry with a larger `--max-tokens` is admitted against the cap afresh. A call that timed out, dropped, was cancelled or had its result refused after it was sent is settled at its usage, or its full reservation when the usage is unknown, because the provider may have billed it. A call still in flight when the process died is charged its reservation by the next `run`, which then sends a fresh attempt. A call refused before anything was sent (no API key, for example) charges nothing, and the rerun reuses its reservation. A failed outside call is still counted at its full reservation while the phase continues native-only.
- **A finding you chose to fix stays open until the revised plan is reviewed.** If the reviewer raises it again after a revise that missed it, it comes back to the gate instead of being treated as settled. A withdrawn fix leaves its finding open.
- **Two findings that only sit near each other are no longer merged.** A user challenge one line away from an unrelated Critical no longer hides the Critical. Findings are merged only when their titles match or the parent says they are the same problem.
- **A plan gate answered in an agent's own words is not owner approval.** `gstack-ship-receipt` rejects a plan-sha approval citation whose gate was answered with `--words-by agent:<id>`.
- **A later gate answer replaces the earlier answer's authorship.** When someone else answers again, the earlier answerer's words, `words_by` and carry fields are cleared, so the approval record names who actually answered.
### Added
- **`gstack-autoplan answer --words-by agent:<id>`** records that the words came from the answering agent itself, as `answer_words` and `words_by`, never as `owner_reply`. The default stays `owner`. A later gate never carries those words forward, and `gstack-approvals import` lists that row under the agent, while `--by` names only the rows that hold the owner's words.
### Tests
- New failing-first tests in test/autoplan-pairing.test.ts, test/autoplan-gate-judgment.test.ts, test/autoplan-budget.test.ts, test/autoplan-run-api.test.ts (a local fake provider, no key or network), test/autoplan-carry.test.ts and test/approval-record.test.ts.
## [1.92.49.0] - 2026-11-01
**The timeline Stop hook returns within its 2-second budget even when the host keeps stdin open.** The hook read its payload with a blocking read that waits for end of input, so a host that writes the Stop payload and leaves the pipe open held the hook for as long as the pipe stayed open. A local run with the pipe held for 8 seconds took 8 seconds, past the hook's 2-second budget and on the way to Claude Code's 15-second hook window (#3628, reported on Windows). Rewritten from PR #3628 by shahab (mashtii85).
### Fixed
- **`hosts/claude/hooks/timeline-stop-hook.ts`** reads stdin for at most 500 ms and stops at the first complete JSON payload, then repairs the timeline as before. The hook still exits 0 on every path.
### Tests
- `test/timeline-stop-hook-stdin.test.ts` (runs on windows-latest): a complete payload on a pipe that never closes returns within budget and still writes the repair; an open pipe with no data returns within budget. Both failed before the fix (10 s, killed).
## [1.92.48.0] - 2026-11-01
**On Windows, `gstack-review-scope summary` works again, and the JSONL merge driver no longer fails on non-English text or rewrites files with CRLF.** Before, the summary started the bash script `gstack-review-read` directly, which Windows cannot run, so it always refused with an empty "gstack-review-read failed:" message and /ship could not write its review-loop file (#3541). The `jsonl-append` merge driver wrote its result with Python's `print()`, so a line the console code page could not encode (Cyrillic text under cp1250, for example) made it exit 1 and git reported a conflict with no markers, and every merge on Windows turned LF files into CRLF (#3659).
### Fixed
- **`gstack-review-scope summary`** runs `gstack-review-read` through `lib/script-invocation.ts` (Git Bash on Windows). When the helper cannot start, the refusal names the launch error instead of an empty string.
- **`gstack-jsonl-merge`** writes UTF-8 bytes with LF line endings on every platform and code page.
### Tests
- `test/review-scope-windows-launch.test.ts` runs the summary against a seeded review log through the same launcher; it runs on windows-latest.
- `test/jsonl-merge-encoding.test.ts` merges a Cyrillic line with `PYTHONIOENCODING=cp1252` and checks the bytes are UTF-8 with no CR. It failed before the fix on Linux too.
- `test/script-invocation-tripwire.test.ts` now also catches a bin spawned through `path.join(path.dirname(...import.meta.url...), ...)` or `import.meta.dir`, the shape that hid #3541. `bin/gstack-ship-receipt`'s `gstack-session-kind` spawn is listed as a known site until its own follow-up.
## [1.92.47.0] - 2026-11-01
- **`/ship` Step 5 runs its test lanes one at a time on machines with 4 or fewer cores, and `--lanes serial|parallel` picks the mode by hand** (`gstack-ship-preflight`, GST-851). Preflight used to start every `--lane` at once. On garrytan/gstack-evals#115 six lanes (a 1,451-test bun suite, tsc, a Docker tier and an arm tarball trial) started within 60 ms of each other on a 4-core, 16 GB machine, and running lanes together has taken machines down before (gstack-evals #101, #106, #107). Now, with no flag, preflight runs one lane at a time when `os.availableParallelism()` is 4 or less and all of them at once above that. `--lanes serial` or `--lanes parallel` overrides the default, and any other value, or `--lanes` with no value, is a usage error (exit 2). A new `LANES: serial|parallel (--lanes|<n> cores)` line before the `LANE` lines says which mode ran and why. Each lane still prints its own `LANE` line with its own `tests_ran` count, in the order the lanes were given. Serial runs take the sum of the lane times instead of the longest one. There is no ship-policy key for this: a new key needs key versioning and a `min_gstack_version` bump, and the core-count default already covers the 4-core case.
CONTRIBUTING.md
+3 -3
View File
@@ -73,8 +73,8 @@ project where you actually felt the pain.
**What CI does on a fork PR.** GitHub never gives a fork PR this repository's
secrets, so CI runs everything that needs none: the required `free-tests` check
(the Linux free suite, typecheck, the macOS and Windows gates), Windows Free
Tests, Skill Docs Freshness, Workflow Lint, Version Gate, Quality gate, the
(the Linux free suite, typecheck, the macOS and Windows gates), Windows
Smoke, Skill Docs Freshness, Workflow Lint, Version Gate, Quality gate, the
make-pdf gate, and any path-triggered gates your change touches. E2E Evals
still builds the CI image from your `Dockerfile.ci` without publishing it, and
skips its paid eval jobs, which need provider API keys. That skip is expected
@@ -192,7 +192,7 @@ the new defaults.
| Run full free acceptance before publishing | `bun run test` | Free, a few minutes | Bun 1.4.2 |
| Run the full free suite from a small machine | `bun run test:ubicloud` | Free suite on a billed 16-vCPU VM, about 5 minutes | `UBICLOUD_API_KEY`, `UBI_OWNER` |
| Clear pregate's test tier on a machine under 8 cores ([remote gate](docs/pregate.md#remote-gate)) | `gstack-pregate --stage tests --remote --spend-cap 2 --timeout 2700 --allow-repo-commands` (same command keyed or keyless) | Keyed: one billed 16-vCPU VM, about 7 minutes and $0.06 for the whole free tier, worst case $0.38; keyless: free, local parallel shards, practical for small selections only (98 files took 103 s on 4 cores) | Keyed: `UBICLOUD_API_KEY`, `UBI_OWNER`; keyless: nothing |
| Run the curated Windows-safe subset | `bun run test:windows` (CI: six `windows-latest` jobs) | Free | Windows, Git Bash |
| Run the curated Windows-safe subset | `bun run test:windows` (CI: six `windows-latest` jobs, nightly on main; PRs run the shorter `windows-smoke` check) | Free | Windows, Git Bash |
| Refresh the Windows duration seed | `gh workflow run windows-free-tests.yml --ref <branch> -f record_durations=true`, then commit the `free-test-durations-windows` artifact | Free CI runners | Pushed branch, `gh` with workflow rights |
| Run a native Windows qualification campaign | `gh workflow run native-qualification.yml --ref <branch> -f mode=<mode>` ([modes](docs/TESTING_INTERNALS.md#windows-free-lane)) | Free CI runners | Pushed branch, `gh` with workflow rights |
| Preview which paid cases my diff selects | `bun run eval:select` (PR profile; `--profile full` for the plain touchfile selection) | Free | — |
TODOS.md
+6
View File
@@ -17,6 +17,12 @@
- **Derive `carry.covers` from depth.** A carried receipt's `covers` is the fixed `CARRY_COVERS` list (lib/ship-receipt.ts); derive it from the recorded `review_depth` so a light review's carry does not claim the checks a full review ran (plan item 5, deferred at /ship). **Effort:** S. **Priority:** P1.
- **Bound the queue's recompute time.** `gstack-ship-queue` recomputes each receipt's needed depth at READY and at train build; add a test that a five-PR train recomputes within a fixed budget, so a large diff cannot stall the train. **Effort:** S. **Priority:** P1.
### P2/P3: Windows smoke follow-ups (filed 2026-11-01, from the GSTA-203 PR 1 /ship review)
- **Branches cut before windows-smoke.yml still owe the old Windows lane (P2)** — pregate reads lanes from the branch's own workflows, so such a branch keeps an uncleared `windows-free-tests.yml/windows-free-shard` obligation, and `ready --dispatch-ci` skips `windows-smoke.yml` there. The lane clears it by passing `windows-free-tests.yml` to `--dispatch-ci` (the old cost) or with a main-merge grant. Have the skip note name that route, or dispatch the old workflow in smoke's place. The skip also treats a branch that deletes or renames `windows-smoke.yml` as one that predates it; check the merge base too. **Effort:** S. **Priority:** P2.
- **Someone owns the nightly (P2)** — nothing reads the scheduled `windows-free-tests.yml` result or its flake ledgers. Add a test-health-report metric for scheduled runs on main, or have the daily flaky pass read the latest nightly. **Effort:** S. **Priority:** P2.
- **Smoke /browse step waits for its page server (P3)** — `windows-smoke.yml` starts `Bun.serve` in the background and calls `browse chain goto` right away; poll the port first. **Effort:** XS. **Priority:** P3.
- **Smoke seed ceiling vs the weekly reseed (P3)** — `test/windows-native-workflows.test.ts` holds each smoke file to 5 s in the Windows seed, which `free-durations-record.yml` rewrites weekly; one slow recording turns the seed PR red. Report the overrun there or re-pick files at reseed. **Effort:** S. **Priority:** P3.
### P2/P3: pregate head-policy follow-ups (filed 2026-10-31, from the GST-419/GST-420 /ship review)
- **`finish` fallback drops head-policy lanes (P2)** — `lib/ship-release.ts` reruns pregate on the stamped tree under the base policy, so after a version bump a lane the branch added under `--policy-from head` is gone and `finish` reports `PREGATE: fail (rerun on the stamped tree)`. GSTA-194 owns that file and has the note. **Effort:** S. **Priority:** P2.
- **The 3600 s preflight and `finish` wrapper deadline (P3)** — it caps the whole pregate run, so several runners that each fit `timeout_s` can still be cut off together. Derive the wrapper deadline from `timeout_s` and the lane count, or document the cap. **Effort:** S. **Priority:** P3.
VERSION
+1 -1
View File
@@ -1 +1 @@
1.92.47.0
1.92.61.0
agents-digest/gstack-AGENTS.md
+1 -1
View File
@@ -1,4 +1,4 @@
# gstack digest v1.92.47.0 — regenerate/re-copy after upgrading gstack
# gstack digest v1.92.61.0 — regenerate/re-copy after upgrading gstack
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
for agent hosts without a full skill install. The full skills add workflows,
autoplan/DRIVER.md
+11 -11
View File
@@ -57,28 +57,28 @@ line yourself; gstack never writes the plan.
### Commands
- `gstack-autoplan next --out <dir> --plan <plan.md> [scope flags] [--light] [--spend-cap <usd>] [--max-rounds <n>] [--scaffold]`
first call: init the run, print PHASE:, SCOPE:, DRIVER: and one ATTEMPT: per voice; --max-rounds caps review rounds per phase (default 2 unattended), and an unattended Capy run reserves 5.00 per host attempt unless --estimate-usd <n> says otherwise, so --spend-cap binds.
- `gstack-autoplan next --out <dir> --plan <plan.md> [scope flags] [--light] [--spend-cap <usd>] [--max-rounds <n>] [--estimate-usd <n>] [--scaffold] [--ceo-only]`
first call: init the run, print PHASE:, SCOPE:, DRIVER: and one ATTEMPT: per voice; --max-rounds caps review rounds per phase (default 2 unattended); a run with --spend-cap, in any session kind (or any unattended Capy run), reserves 5.00 per voice for each attempt, so the cap binds; --estimate-usd <n> sets that reservation and the run keeps it for later phases; --ceo-only (command line only, never a scope marker) reviews CEO alone and ends `status=reviewed` with `<dir>/ceo-review.md` (`REVIEWED:`): no gate, and the gate verbs refuse it (`CEO_ONLY_NO_GATE`).
- `gstack-autoplan run --out <dir> --plan <plan.md> --native-model <id> --outside-model <id> --native-runner api --runner api [--max-tokens <n>] [--spend-cap <usd>]`
hosts without subagents: one command calls both models directly (ANTHROPIC_API_KEY, OPENAI_API_KEY) and loops next/submit to the gate; a response cut off at the output limit is reported truncated, so raise --max-tokens (default 16384 for both providers; OpenAI counts reasoning tokens toward it); each api call reserves its priced worst case (a model with no price reserves the per-voice default under a cap), a native call that went out but bound no result settles at its usage, or its full reservation when the usage is unknown, and a failed outside call stays counted at its reservation.
- `gstack-autoplan next --out <dir> [--redispatch <attempt>]`
later calls: open the next phase or reprint open attempts; never redispatches on its own.
- `gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --record <file.md> --model <id> [--attempt <id>]`
bind a free-form review record (canonical fence, one findings table or list, or `No findings.`).
- `gstack-autoplan submit ... --record <file.md> --attest "<reason>" --attempt <id>`
bind a record with no INPUT: line as receipt: attested (a wrong INPUT: always refuses).
- `gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --result <file> --model <id> --attempt <id>`
bind a canonical result (the RESULT FORMAT below).
- `gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --result <file> --model <id> --attempt <id> [--usage-usd <n>] [--reconcile <jsonl>]`
bind a canonical result (the RESULT FORMAT below); `--usage-usd` settles the attempt at what the reviewer actually cost (else it stays priced unknown at its reservation); on the submit that closes the phase, `--reconcile` takes your own pairing of findings both voices raised, one `{"outside_id","native_id","disposition","resolution"}` row each, over the structural one.
- `gstack-autoplan submit --out <dir> --phase <p> --voice outside --unavailable|--failed "<reason>"`
close the phase native-only when no outside voice can run.
- `gstack-autoplan answer --out <dir> --gate-rev <n> --reply "<reply>" --by human|agent:<id> [--owner-reply <text>|-] [--rec-hash <h>]`
answer the gate with the reply grammar below; an agent answer stores the owner's words verbatim.
- `gstack-autoplan answer --out <dir> --gate-rev <n> --reply "<reply>" --by human|agent:<id> [--owner-reply <text>|-] [--words-by agent:<id>] [--rec-hash <h>]`
answer the gate with the reply grammar below; an agent answer stores the owner's words verbatim, and passes --words-by agent:<id> whenever the words are its own (the default is owner).
- `gstack-autoplan recommend --out <dir> --gate-rev <n> --item <id> --option <k> --basis "<why>"`
change one recommendation before relay; then render and relay again (a `rec` needs the new `--rec-hash`).
- `gstack-autoplan revise --out <dir> --plan <plan v2> --incorporates <decision ids>`
bind the amended plan; reopens every phase whose reviewed plan changed, Eng last.
- `gstack-autoplan revise --out <dir> --settle <decision id|d4>`
discharge a packaging or ordering change with no plan edit; reopens nothing.
- `gstack-autoplan revise --out <dir> --withdraw <decision id>`
drop a pending amendment instead of applying it.
- `gstack-autoplan revise --out <dir> --settle <decision id|d4> | --withdraw <decision id>`
--settle discharges a packaging or ordering change with no plan edit and reopens nothing; --withdraw drops a pending amendment instead of applying it.
- `gstack-autoplan close --out <dir> --phase <p> --deferred "<reason>" --by human|agent:<id>`
close a phase accepted-with-findings: its open findings go to tasks.jsonl, and Critical, urgent or user-challenge findings become gate items; also the recovery from `STOP: reason=budget_exhausted`.
- `gstack-autoplan budget --out <dir> [--spend-cap <usd>] [--max-rounds <n>] --by human|agent:<id>`
@@ -111,7 +111,7 @@ A reply binds to the `gate_rev` you read (`--gate-rev`). Bare `yes`/`no` is unpa
- `rec`: every item takes its recommendation, plan changes included (each prints APPLY:); `all` and `do it` are aliases.
- `as-is`: keep every direction; no plan change.
- `d3b uc1a`: one <id><option> token per item you set; unanswered auto items take their default.
- `d3b uc1a`: one <id><option> token per item you set; unanswered auto items take their default (a finding item keeps the plan as written; `p1b` applies it).
- `rec except d3b`: recommendations, with the listed overrides.
## Run directory
autoplan/SKILL.md
+1
View File
@@ -269,6 +269,7 @@ sections. Read a section in full before doing its step; do not work from memory.
| closing a review phase, after its reviews finish and before announcing completion or loading the next phase (read afresh at each exit) | `sections/phase-close.md` |
| presenting the Final Approval Gate (Phase 4) — the aggregator computes $AGGREGATED_TASKS that the gate message substitutes | `sections/tasks-aggregator.md` |
| the preamble echoed `SESSION_KIND: unattended` (read before Phase 1 and at the Phase 4 gate) | `sections/unattended.md` |
| the invocation names `--ceo-only` (read before any preamble instruction block) | `sections/ceo-only.md` |
---
autoplan/bin/guard-journal.ts
+8 -2
View File
@@ -17,6 +17,10 @@ type Event = ClaudeParentPublicEvent;
type Use = NativePublicToolEvent & { order: number };
/** Claude Code's reply to a Read of an unchanged file it already returned in full. */
export const DEDUP_REPLY = 'Wasted call — file unchanged since your last Read. Refer to that earlier tool_result instead.';
/** Claude Code appends exactly this one line to a Bash result when the command's `cd` left the session cwd (#3651). */
const SHELL_CWD_RESET = /\nShell cwd was reset to [^\n]+$/;
/** A Bash result's JSON, without that one host footer; anything else must still be strict JSON. */
export function bashJson(text: string): any { return JSON.parse(text.replace(SHELL_CWD_RESET, '')); }
/** The longest Read input or Bash command the index keeps; a literal init is far shorter. */
const INDEX_INPUT_MAX = 4096;
@@ -69,11 +73,13 @@ function select(tests: GuardJournalTests, currentToolUseId: string) {
const replies = (resultsOf.get(init.use.toolUseId) ?? []).flatMap(r => r.events)
.filter(e => e.kind === 'result' && e.toolUseId === init.use.toolUseId && e.order > init.use.order);
let reply: unknown;
try { reply = replies.length === 1 ? JSON.parse(textResult(replies[0]!) ?? 'null') : undefined; } catch { reply = undefined; }
try { reply = replies.length === 1 ? bashJson(textResult(replies[0]!) ?? 'null') : undefined; } catch { reply = undefined; }
// A failed or unreadable init binds nothing and is skipped, as invocation() does.
if (!object(reply)) continue;
// A reused:true init keeps the earlier binding, as invocation() does.
if (object(reply) && reply.reused === true && bound && bound.activePlan === reply.activePlan &&
bound.restorePath === reply.restorePath) continue;
chainStart = init.record; bound = object(reply) ? reply : undefined;
chainStart = init.record; bound = reply;
}
let start = 0;
for (let i = chainStart ?? records.length - 1; i >= 0; i--)
autoplan/bin/phase-publication-hook.ts
+12 -9
View File
@@ -10,7 +10,7 @@ import { extractImplementationPlan, checkPhaseImplementation, acceptedBlocks, ca
import { autoplanPhaseCompletions } from '../../lib/autoplan-phase-publication';
import { nativePathSpelling, ownedNativePath, sameNativePath, type ClaudeParentPublicEvent } from '../../lib/claude-journal-records';
import { ownedRecordLimit, ownedRetainedLimit, type JournalPrefix, type OwnedRead, type OwnedReadMeasure } from '../../lib/claude-owned-journal';
import { readGuardJournal, textResult, DEDUP_REPLY } from './guard-journal';
import { bashJson, readGuardJournal, textResult, DEDUP_REPLY } from './guard-journal';
import { REASONS, reasonCode, reasonText, type Detail, type ReasonCode } from './guard-reasons';
import { logGuardDecision } from './guard-log';
import { approveOwnedRead, renderSectionBase, userRenderRoot } from './owned-read';
@@ -227,7 +227,7 @@ function checkpointResult(result: Event, tools: Tools, init: Invocation): { phas
if (use?.name !== 'Bash' || use.order >= result.order) return;
const text = textResult(result);
if (text === undefined) return;
const output = JSON.parse(text);
const output = bashJson(text);
if (!object(output) || !['ceo', 'design', 'dx', 'eng'].includes(output.phase) ||
!ownPath(output.snapshotPath) || typeof output.nativePrompt !== 'string' || !object(output.baselineEdits)) return;
const { nativePrompt, baselineEdits, ...identity } = output;
@@ -260,6 +260,7 @@ function initArguments(command: unknown, root: string): string[] | undefined {
const INIT_MISMATCH_CAUSE = 'Snapshot init returned paths that do not match the canonical form of its own command arguments';
const INIT_FAILED_CAUSE = 'The most recent snapshot init call failed; re-run the same init command with the same three paths';
const INIT_UNPARSEABLE_CAUSE = 'The most recent snapshot init printed something other than its JSON result; re-run the same init command with the same three paths, without a leading cd';
/** Diagnostic only, never binds: a Bash call that ran snapshot init in a shape the guard cannot bind (#3045). */
const UNBINDABLE_INIT = /(?:gstack-autoplan-snapshot|SNAPSHOT_TOOL|\$\{?[A-Z_]+\}?"?)\S*\s+init\s/;
@@ -268,8 +269,8 @@ function invocation(events: Event[], root: string): Invocation {
let bound: Invocation | undefined;
let chosen: Record<string, any> | undefined;
let unbindable = false;
// True while the most recent bindable init use errored (a later success clears it).
let lastErrored = false;
// The cause while the most recent bindable init use failed (a later success clears it).
let lastFailure: string | undefined;
const tools = byTool(events);
for (const use of events) {
if (use.kind !== 'use' || use.name !== 'Bash') continue;
@@ -278,10 +279,12 @@ function invocation(events: Event[], root: string): Invocation {
const results = tools.results(use.toolUseId).filter(x => x.order > use.order);
if (results.length !== 1) fail('init_mismatch');
// An errored init published nothing; a later correct init may still bind.
if (results[0]!.isError === true) { lastErrored = true; continue; }
if (results[0]!.isError === true) { lastFailure = INIT_FAILED_CAUSE; continue; }
const text = textResult(results[0]!);
if (text === undefined) fail('init_failed');
const result = JSON.parse(text);
if (text === undefined) { lastFailure = INIT_FAILED_CAUSE; continue; }
let result: any;
// Output that is not the init's JSON bound nothing; a later correct init may still bind.
try { result = bashJson(text); } catch { lastFailure = INIT_UNPARSEABLE_CAUSE; continue; }
// init reports canonical destinations (macOS /tmp is /private/tmp); canonicalize the argv the same way.
let active: string, restore: string;
try { active = canonicalDestination(args[1]!); restore = canonicalDestination(args[2]!); }
@@ -289,13 +292,13 @@ function invocation(events: Event[], root: string): Invocation {
if (!object(result) || !samePath(result.sourcePlan, fs.realpathSync(args[0]!)) || !samePath(result.activePlan, active) ||
!samePath(result.restorePath, restore) || typeof result.reused !== 'boolean' || !positive(result.originalBytes) ||
!/^[a-f0-9]{64}$/.test(result.originalSha256)) fail('init_mismatch', { cause: INIT_MISMATCH_CAUSE });
lastErrored = false;
lastFailure = undefined;
if (result.reused && bound && bound.activePlan === result.activePlan && bound.restorePath === result.restorePath) continue;
chosen = result;
bound = { activePlan: result.activePlan, restorePath: result.restorePath,
originalSha256: result.originalSha256, start: results[0]!.order };
}
if (lastErrored) fail('init_failed', { cause: INIT_FAILED_CAUSE });
if (lastFailure) fail('init_failed', { cause: lastFailure });
if ((!chosen || !bound) && unbindable) fail('init_unbindable');
if (!chosen || !bound) fail('init_required');
const restore = read(bound.restorePath, true), active = read(bound.activePlan);
autoplan/sections/ceo-only.md
+41
View File
@@ -0,0 +1,41 @@
<!-- AUTO-GENERATED from ceo-only.md.tmpl — do not edit directly -->
<!-- Regenerate: bun run gen:skill-docs -->
## CEO-only run (`--ceo-only`)
This section governs the whole run and wins over the skeleton. The run reviews
the plan; it never approves it.
**Zero questions.** Never call AskUserQuestion (native or `mcp__*`) or
ExitPlanMode, never ask the user in prose, never wait for a reply, whatever
`SESSION_KIND` says. A preamble instruction block that would ask takes its
non-consenting default (off, not now, skip). Each decision takes its
`(recommended)` option, else the 6 principles; consent, destructive or
irreversible options take the conservative choice. Log each as a Decision Audit
Trail row.
**Scope.** Run Phases 0, 0.5 and 1 only; send "Phases 2, 2.5, 3 and 4 skipped
(CEO-only run)" and record it in ACTIVE_PLAN. With `--no-outside`, skip every
outside dispatch: `outside_status: disabled (--no-outside)`.
**Mode.** Instead of SELECTIVE EXPANSION, use plan-ceo-review 0E step 2's rule
(an explicit mode in the request wins), skip step 3, run
`~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff "<mode>" --ceo-only --decisions "<rows or none>"`
and send its line verbatim.
**Never applied:** a user challenge (keep the direction; list it below).
**Skipped and logged `skipped (ceo-only)`:** office-hours and BENEFITS_FROM
offers, 0E's question, plan archive cleanup, Next Steps chaining and CEO plan
promotion; the row satisfies its checklist entry.
**State.** Write only in `$_AP`, ACTIVE_PLAN and `GSTACK_STATE_ROOT`
(the override; docs/state-root.md).
**Finish.** After Phase 1 closes, write `$_AP/ceo-review.md`: `# CEO-only
review: <plan title>`; a line with the mode, outside-voice status and "Not
approved"; then `## User challenges (direction kept)` (plan says, proposal,
why, cost if wrong), `## Scope added`, `## Conservative choices`,
`## Findings` (severity, section, fix, disposition) and `## Decision Audit
Trail` (copied), each "None." when empty. Send the CEO Completion Summary and
both paths, skip Completion's logs, approval and `/ship`, and end with:
`GSTACK_RESULT: skill=autoplan status=reviewed run=<absolute $_AP>`
autoplan/sections/ceo-only.md.tmpl
+39
View File
@@ -0,0 +1,39 @@
## CEO-only run (`--ceo-only`)
This section governs the whole run and wins over the skeleton. The run reviews
the plan; it never approves it.
**Zero questions.** Never call AskUserQuestion (native or `mcp__*`) or
ExitPlanMode, never ask the user in prose, never wait for a reply, whatever
`SESSION_KIND` says. A preamble instruction block that would ask takes its
non-consenting default (off, not now, skip). Each decision takes its
`(recommended)` option, else the 6 principles; consent, destructive or
irreversible options take the conservative choice. Log each as a Decision Audit
Trail row.
**Scope.** Run Phases 0, 0.5 and 1 only; send "Phases 2, 2.5, 3 and 4 skipped
(CEO-only run)" and record it in ACTIVE_PLAN. With `--no-outside`, skip every
outside dispatch: `outside_status: disabled (--no-outside)`.
**Mode.** Instead of SELECTIVE EXPANSION, use plan-ceo-review 0E step 2's rule
(an explicit mode in the request wins), skip step 3, run
`~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff "<mode>" --ceo-only --decisions "<rows or none>"`
and send its line verbatim.
**Never applied:** a user challenge (keep the direction; list it below).
**Skipped and logged `skipped (ceo-only)`:** office-hours and BENEFITS_FROM
offers, 0E's question, plan archive cleanup, Next Steps chaining and CEO plan
promotion; the row satisfies its checklist entry.
**State.** Write only in `$_AP`, ACTIVE_PLAN and `GSTACK_STATE_ROOT`
(the override; docs/state-root.md).
**Finish.** After Phase 1 closes, write `$_AP/ceo-review.md`: `# CEO-only
review: <plan title>`; a line with the mode, outside-voice status and "Not
approved"; then `## User challenges (direction kept)` (plan says, proposal,
why, cost if wrong), `## Scope added`, `## Conservative choices`,
`## Findings` (severity, section, fix, disposition) and `## Decision Audit
Trail` (copied), each "None." when empty. Send the CEO Completion Summary and
both paths, skip Completion's logs, approval and `/ship`, and end with:
`GSTACK_RESULT: skill=autoplan status=reviewed run=<absolute $_AP>`
autoplan/sections/manifest.json
+6
View File
@@ -45,6 +45,12 @@
"file": "unattended.md",
"title": "Unattended profile: parent-driven next/submit loop, gate answers, export, terminal line",
"trigger": "the preamble echoed `SESSION_KIND: unattended` (read before Phase 1 and at the Phase 4 gate)"
},
{
"id": "ceo-only",
"file": "ceo-only.md",
"title": "CEO-only run: zero questions, CEO phase only, review report, no gate",
"trigger": "the invocation names `--ceo-only` (read before any preamble instruction block)"
}
]
}
bin/gstack-autoplan
+64 -23
View File
@@ -2,15 +2,15 @@
/**
* gstack-autoplan — the unattended /autoplan runner as an explicit, durable state machine (plan B2).
*
* gstack-autoplan next --out <dir> [--plan <file>] [--plan-heading <text>] [--ui|--no-ui] [--developer-tool] [--agent-primary] [--docs-only]
* gstack-autoplan next --out <dir> [--plan <file>] [--plan-heading <text>] [--ui|--no-ui] [--developer-tool] [--agent-primary] [--docs-only] [--ceo-only]
* [--no-outside] [--light] [--spend-cap <usd>] [--estimate-usd <n>] [--deadline <min>] [--redispatch <attempt>] [--json]
* gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --result <file> | --record <file> [--attest <reason>]
* --model <id> [--attempt <id>] [--runner <r>] [--usage-usd <n>] [--reconcile <jsonl>] [--json]
* gstack-autoplan submit --out <dir> --phase <p> --voice outside --unavailable <reason> | --failed <reason> [--json]
* gstack-autoplan revise --out <dir> --plan <file> [--incorporates <id,...>] [--withdraw <id>] [--settle <id>] [--json] review a revised plan in this run
* gstack-autoplan run --out <dir> --plan <file> --native-model <id> --outside-model <id> [--runner codex-cli|api] [--native-runner api]
* [--deadline <min>] [--spend-cap <usd>] [--estimate-usd <n>] [--json] next in a loop; needs runners for both voices
* gstack-autoplan answer --out <dir> --gate-rev <rev> --reply "<grammar>" --by <human|agent:<id>> [--owner-reply <text>|-] [--rec-hash <h>] [--json]
* [--deadline <min>] [--spend-cap <usd>] [--estimate-usd <n>] [--max-tokens <n>] [--json] next in a loop; needs runners for both voices
* gstack-autoplan answer --out <dir> --gate-rev <rev> --reply "<grammar>" --by <human|agent:<id>> [--owner-reply <text>|-] [--words-by owner|agent:<id>] [--rec-hash <h>] [--json]
* gstack-autoplan recommend --out <dir> --gate-rev <n> --item <id> --option <k> --basis "<why>" [--by <who>]
* gstack-autoplan resume --out <dir> [--from <checkpoint>] [--json] take ownership; intent without a terminal record becomes EXECUTION_UNKNOWN
* gstack-autoplan status --out <dir> [--json] query: the manifest, no GSTACK_RESULT line
@@ -31,10 +31,11 @@
*/
import * as fs from 'node:fs';
import * as path from 'node:path';
import { amendmentSteps, answerGate, closeDeferred, finishGate, recommendGate } from '../lib/autoplan-gate';
import { amendmentSteps, answerGate, closeDeferred, recommendGate } from '../lib/autoplan-gate';
import { checkCeoOnlyFlags, finishRun, refuseCeoOnly } from '../lib/autoplan-ceo-only';
import { gateCost, gateCostPayload, renderGateCost, type GateCost } from '../lib/autoplan-gate-cost';
import { attemptEstimate, changeBudget, escalateLine, liftStop } from '../lib/autoplan-budget';
import { DEFAULT_CARRY_CHECK_CAP_USD, carryLines, submitCarryCheck } from '../lib/autoplan-carry';
import { DEFAULT_CARRY_CHECK_CAP_USD, submitCarryCheck } from '../lib/autoplan-carry';
import { ANSWERED_BY_RE } from '../lib/gate-list';
import { checkpointLines, checkpointRun, restoreCheckpoint, type CheckpointMoment } from '../lib/autoplan-checkpoint';
import { admit, touchInstallLease } from '../lib/autoplan-admission';
@@ -43,11 +44,13 @@ import { capyRelayLine } from '../lib/captain-brief';
import { exportPublic } from '../lib/autoplan-public';
import { migrateRun } from '../lib/autoplan-migrate';
import { reviseRun } from '../lib/autoplan-revise';
import { pendingAmendments, type RunState } from '../lib/autoplan-state';
import { FILES, RunError, currentPhase, initRun, markOutside, nextStep, readState, resumeRun, statusLines, submitResult, withRunLock, writeState } from '../lib/autoplan-run';
import { pendingAmendments, type RunState, type Voice } from '../lib/autoplan-state';
import { FILES, RunError, currentPhase, fitReservation, initRun, markOutside, markSent, nextStep, readState, resumeRun, retireAttempt, statusLines, submitResult, withRunLock, writeState, type NextOptions } from '../lib/autoplan-run';
import { driverFile, type Phase } from '../lib/autoplan-prompts';
import { EXIT, exitTableHelp } from '../lib/headless-artifacts';
import { createRunner, familyConflict, type RunnerId } from '../lib/outside-voice-runner';
import { DEFAULT_MAX_TOKENS, apiTimeoutMs } from '../lib/outside-voice-api';
import { estimateCostUsd, pricingRow } from '../lib/pricing';
import { resultLine } from '../lib/result-codes';
import { release } from '../lib/spend-ledger';
import { COORDINATOR_CONTRACT } from '../lib/coordinator-contract';
@@ -55,11 +58,12 @@ import { COORDINATOR_CONTRACT } from '../lib/coordinator-contract';
const HELP = `gstack-autoplan — unattended /autoplan: next/submit loop, answer, resume (docs/unattended.md)
DRIVER: ${driverFile()}
next --out <dir> [--plan <file>] [--plan-heading <text>] [--scaffold] [--ui|--no-ui] [--developer-tool] [--agent-primary] [--no-dx] [--docs-only]
next --out <dir> [--plan <file>] [--plan-heading <text>] [--scaffold] [--ui|--no-ui] [--developer-tool] [--agent-primary] [--no-dx] [--docs-only] [--ceo-only]
[--no-outside] [--light] [--spend-cap <usd>] [--max-rounds <n>] [--estimate-usd <n>] [--deadline <min>] [--redispatch <attempt>] [--json]
--max-rounds (first next): review rounds per phase (default 2 unattended); at the cap a phase closes accepted-with-findings
--docs-only (first next; also detected when every Implementation-plan path is docs-class): CEO (native voice only) + Eng
--estimate-usd: reserved per new attempt; unattended Capy runs reserve a default for host attempts so --spend-cap binds
--ceo-only (first next, command line only): the CEO phase alone, no gate; ends status=reviewed with <out>/ceo-review.md, never approved
--estimate-usd: reserved per new attempt, kept for the rest of the run; with --spend-cap any run (or an unattended run on Capy) reserves a default so the cap binds
--scaffold (first next): a plan missing one of its two sections is reviewed from <out>/plan-source.md with the heading added
--light loads each phase's checklist plus the sections its scope needs (the unattended default); interactive kinds load everything
--no-carry-forward (first next): every later gate revision goes to the owner, even one that asks nothing new
@@ -80,9 +84,12 @@ DRIVER: ${driverFile()}
comes into scope, the edit applies without re-review (APPLIED:); --withdraw (alone or with --plan) drops one;
--settle records a packaging/ordering change that needs no plan edit and reopens nothing (ids: decision id or d4)
run --out <dir> --plan <file> --native-model <id> --outside-model <id> [--runner codex-cli|api] [--native-runner api] ...
next in a loop, only when a runner executes both voices here; --deadline cancels the children it started
next in a loop, only when a runner executes both voices here (hosts without subagents: --native-runner api --runner api);
--deadline cancels the children it started; --max-tokens caps each api call's output (default 16384; OpenAI counts reasoning tokens in it), and a
response cut off at that limit is reported truncated; submit's result checks judge every completed response
answer --out <dir> --gate-rev <rev> --reply "<rec | rec except d3a | d3b uc1a | as-is>" --by <human|agent:<id>> [--json]
--by agent:<id> needs --owner-reply "<owner's words>" (- reads stdin; --owner-reply-file <path>), stored verbatim;
--words-by agent:<id> (the same agent) when those words are its own: recorded as its words, never the owner's, never carried;
--rec-hash <h> binds a rec to the render the owner saw (required once recommend ran); accepted changes leave amend_pending
recommend --out <dir> --gate-rev <n> --item <id> --option <k> --basis "<why>" [--by <who>]
set one item's recommendation before relay (provenance recorded, p1 follows); re-render and relay afterwards
@@ -178,24 +185,26 @@ function emit(json: boolean, lines: string[], payload: Record<string, unknown>):
else process.stdout.write(lines.join('\n') + '\n');
}
function doNext(out: string, rest: string[], json: boolean): number {
function doNext(out: string, rest: string[], json: boolean, price?: NextOptions['price']): number {
const cwd = process.cwd();
const initLines: string[] = [];
const fresh = !fs.existsSync(path.join(out, FILES.run));
if (!fresh) checkCeoOnlyFlags(rest, undefined, readState(out));
if (fresh) {
const plan = flag(rest, '--plan');
if (!plan) throw new RunError('RUN_NOT_INITIALIZED', `${out} has no run.json; pass --plan <file> on the first next`, 3);
const ceoOnly = checkCeoOnlyFlags(rest, path.resolve(plan), undefined, out);
const ui = rest.includes('--ui') ? true : rest.includes('--no-ui') ? false : undefined;
const maxRounds = num(rest, '--max-rounds');
if (maxRounds !== undefined && (!Number.isInteger(maxRounds) || maxRounds < 1)) throw new RunError('ARTIFACT_SCHEMA', '--max-rounds needs an integer >= 1', 1);
const carryCheckCapUsd = num(rest, '--carry-check-cap');
if (carryCheckCapUsd !== undefined && !(carryCheckCapUsd >= 0)) throw new RunError('ARTIFACT_SCHEMA', '--carry-check-cap needs a number >= 0', 1);
initRun({ plan, out, cwd, planHeading: flag(rest, '--plan-heading'), outsideEnabled: !rest.includes('--no-outside'), spendCap: num(rest, '--spend-cap') ?? null, maxRounds,
carryForward: !rest.includes('--no-carry-forward'), carryCheckCapUsd,
carryForward: !rest.includes('--no-carry-forward'), carryCheckCapUsd, ceoOnly,
deadlineMin: num(rest, '--deadline'), light: rest.includes('--light'), scaffold: rest.includes('--scaffold'), scope: { ui, developerTool: rest.includes('--developer-tool'), agentPrimary: rest.includes('--agent-primary'), dx: rest.includes('--no-dx') ? false : undefined, ...(rest.includes('--docs-only') ? { docsOnly: true } : {}) } }, initLines);
}
const redispatch = rest.flatMap((a, i) => (a === '--redispatch' && rest[i + 1] ? [rest[i + 1]!] : []));
const r = nextStep({ out, cwd, estimateUsd: num(rest, '--estimate-usd'), redispatch });
const r = nextStep({ out, cwd, estimateUsd: num(rest, '--estimate-usd'), redispatch, price });
moment = fresh ? 'init' : r.attempts.some(a => a.new) || r.lines.some(l => l.startsWith('PHASE_DEFERRED: ')) ? 'transition' : undefined;
const status = r.state.status;
const lines = [...initLines, ...r.lines];
@@ -248,6 +257,7 @@ function doRevise(out: string, rest: string[], json: boolean): number {
const plan = flag(rest, '--plan'); const incorporates = listFlag(rest, '--incorporates'); const withdraw = listFlag(rest, '--withdraw'); const settle = listFlag(rest, '--settle');
if (!plan && !withdraw.length && !settle.length) return usage('revise needs --plan <file> (and optionally --incorporates <id,...>), --withdraw <decision id> or --settle <id>');
if (!plan && incorporates.length) return usage('--incorporates binds accepted changes to a revised plan: pass --plan <file>');
refuseCeoOnly(readState(out), 'revise');
const r = reviseRun({ out, plan: plan ? path.resolve(plan) : undefined, incorporates, withdraw, settle, cwd: process.cwd() });
const lines = [...r.lines];
if (r.state.status === 'running' || r.state.status === 'amend_pending') lines.push(nextLine(r.state, out));
@@ -268,7 +278,7 @@ function doClose(out: string, rest: string[], json: boolean): number {
const r = closeDeferred(state, phase as Phase, { reason: reason.trim(), by, cause: 'owner' });
moment = 'transition';
const lines = [...r.lines, ...(stopped ? liftStop(state, `${phase} closed deferred by ${by}`) : [])];
if (!currentPhase(state)) { finishGate(state); lines.push(...carryLines(state), ...(state.status === 'gate_pending' ? gateLines(state, out) : [])); }
if (!currentPhase(state)) lines.push(...finishRun(state), ...(state.status === 'gate_pending' ? gateLines(state, out) : []));
else { state.status = 'running'; lines.push(`NEXT: gstack-autoplan next --out ${out}`); }
writeState(state);
lines.push(resultLineFor(state.status, out));
@@ -305,10 +315,25 @@ async function doRun(out: string, rest: string[], json: boolean): Promise<number
if (!['codex-cli', 'api'].includes(outsideRunner)) return usage('--runner must be codex-cli or api for run (host-subagent cannot execute here; use next/submit)');
const conflict = familyConflict(outsideModel, nativeModel);
if (conflict) throw new RunError('MODEL_FAMILY_CONFLICT', conflict, 3);
const maxTokens = num(rest, '--max-tokens');
if (maxTokens !== undefined && (!Number.isInteger(maxTokens) || maxTokens <= 0)) return usage('--max-tokens needs a positive integer');
const deadline = num(rest, '--deadline');
const deadlineAt = deadline ? Date.now() + deadline * 60_000 : undefined;
const modelOf = (voice: Voice) => (voice === 'native' ? nativeModel : outsideModel);
const isApi = (voice: Voice) => (voice === 'native' ? nativeRunner : outsideRunner) === 'api';
// An api call reserves its worst case (prompt bytes / 3 input tokens plus the output limit it is sent) and settles at its usage.
const price = (voice: Voice, promptFile: string) => {
const row = isApi(voice) ? pricingRow(modelOf(voice)) : undefined;
return row ? +((fs.statSync(promptFile).size / 3 * row.input_per_mtok + (maxTokens ?? DEFAULT_MAX_TOKENS) * row.output_per_mtok) / 1e6).toFixed(6) : undefined;
};
const usdOf = (voice: Voice, used: { input_tokens?: number; output_tokens?: number } | undefined) =>
isApi(voice) && pricingRow(modelOf(voice)) && typeof used?.input_tokens === 'number' && typeof used?.output_tokens === 'number' ? estimateCostUsd({ input: used.input_tokens, output: used.output_tokens }, modelOf(voice)) : undefined;
// Every exit after a paid call went out settles its attempt: a failure, a cancel, a refused result, or a process that died
// mid-call (sent_at, found on the next run) settles at its usage, or its reservation when the usage is unknown. Only a call
// refused before any request left this machine bills nothing, so its attempt stays open for the rerun.
const NOT_SENT = new Set(['runner_unavailable', 'cli_missing', 'model_unselected']);
for (let guard = 0; guard < 40; guard++) {
const code = withRunLock(out, () => doNext(out, rest, true));
const code = withRunLock(out, () => doNext(out, rest, true, price));
if (code !== EXIT.ok) return code;
const state = readState(out);
if (state.status !== 'awaiting_result') {
@@ -316,27 +341,39 @@ async function doRun(out: string, rest: string[], json: boolean): Promise<number
process.stdout.write(resultLineFor(state.status, out) + '\n'); return EXIT.ok;
}
const phase = state.phase_order.find(p => state.phases[p]?.snapshot && !state.phases[p]?.closed_at)!;
const lost = (['native', 'outside'] as const).flatMap(voice => (state.phases[phase]![voice]!.attempts ?? []).filter(a => a.state === 'dispatched' && a.sent_at).map(a => [voice, a.attempt] as const));
if (lost.length) { withRunLock(out, () => { for (const [voice, attempt] of lost) retireAttempt(out, phase as Phase, voice, attempt, 'unknown'); }); continue; }
for (const voice of ['native', 'outside'] as const) {
const v = state.phases[phase]![voice]!;
const open = v.attempts?.find(a => a.state === 'dispatched');
if (!open) continue;
const runnerId = voice === 'native' ? nativeRunner : outsideRunner;
const cost = price(voice, open.prompt);
if (cost !== undefined) withRunLock(out, () => fitReservation(out, phase as Phase, voice, open.attempt, cost));
const runner = createRunner(runnerId);
const remaining = deadlineAt ? deadlineAt - Date.now() : 300_000;
const remaining = deadlineAt ? deadlineAt - Date.now() : runnerId === 'api' ? apiTimeoutMs(maxTokens) : 300_000;
if (remaining <= 0) { withRunLock(out, () => { const s = readState(out); s.status = 'interrupted'; writeState(s); }); process.stdout.write(resultLineFor('interrupted', out) + '\n'); return EXIT.fail; }
const timer = deadlineAt ? setTimeout(() => runner.cancel(), remaining) : undefined;
const res = await runner.run({ promptFile: open.prompt, outFile: `${open.result}.raw`, model: voice === 'native' ? nativeModel : outsideModel, timeoutMs: Math.min(remaining, 600_000), cwd: process.cwd(), gate: 'review' });
withRunLock(out, () => markSent(out, phase as Phase, voice, open.attempt, true));
const res = await runner.run({ promptFile: open.prompt, outFile: `${open.result}.raw`, model: voice === 'native' ? nativeModel : outsideModel, timeoutMs: Math.min(remaining, runnerId === 'api' ? apiTimeoutMs(maxTokens) : 600_000), cwd: process.cwd(), gate: 'execution', maxTokens: runnerId === 'api' ? maxTokens ?? DEFAULT_MAX_TOKENS : maxTokens });
if (timer) clearTimeout(timer);
if (!json) process.stdout.write(`VOICE: ${phase} ${voice} ${res.status} runner=${res.runner} model=${res.model ?? 'unknown'}\n`);
if (res.status === 'cancelled') { withRunLock(out, () => { const s = readState(out); s.status = 'interrupted'; writeState(s); }); process.stdout.write(resultLineFor('interrupted', out) + '\n'); return EXIT.fail; }
const retire = () => { if (NOT_SENT.has(res.reason ?? '')) markSent(out, phase as Phase, voice, open.attempt, false); else retireAttempt(out, phase as Phase, voice, open.attempt, usdOf(voice, res.usage) ?? 'unknown'); };
if (res.status === 'cancelled') { withRunLock(out, () => { retire(); const s = readState(out); s.status = 'interrupted'; writeState(s); }); process.stdout.write(resultLineFor('interrupted', out) + '\n'); return EXIT.fail; }
if (res.status !== 'completed' && res.status !== 'unverified') {
if (voice === 'outside') { withRunLock(out, () => markOutside(out, phase as Phase, 'unavailable', `${res.runner} ${res.status}${res.detail ? `: ${res.detail}` : ''}`)); continue; }
process.stderr.write(`gstack-autoplan run: native voice ${res.status}${res.detail ? `: ${res.detail}` : ''}\n`);
withRunLock(out, () => { const s = readState(out); s.status = 'interrupted'; writeState(s); });
process.stderr.write(`gstack-autoplan run: native voice ${res.status}${res.reason ? `: ${res.reason}` : ''}${res.detail ? `: ${res.detail}` : ''}\n`);
withRunLock(out, () => { retire(); const s = readState(out); s.status = 'interrupted'; writeState(s); });
process.stdout.write(resultLineFor('interrupted', out) + '\n');
return EXIT.fail;
}
withRunLock(out, () => doSubmit(out, ['--phase', phase, '--voice', voice, '--result', `${open.result}.raw`, '--model', res.model ?? (voice === 'native' ? nativeModel : outsideModel), '--attempt', open.attempt, '--runner', res.runner], true));
const usd = usdOf(voice, res.usage);
const usage = usd === undefined ? [] : ['--usage-usd', String(usd)];
try { withRunLock(out, () => doSubmit(out, ['--phase', phase, '--voice', voice, '--result', `${open.result}.raw`, '--model', res.model ?? modelOf(voice), '--attempt', open.attempt, '--runner', res.runner, ...usage], true)); }
catch (e) {
if (voice !== 'outside' || !(e instanceof RunError)) { withRunLock(out, () => { retire(); const s = readState(out); s.status = 'interrupted'; writeState(s); }); throw e; }
withRunLock(out, () => markOutside(out, phase as Phase, 'unavailable', `${res.runner} result refused: ${e.code}`));
}
}
}
return EXIT.fail;
@@ -383,13 +420,16 @@ async function main(argv: string[]): Promise<number> {
if (cmd === 'answer') {
const rev = flag(rest, '--gate-rev'); const reply = flag(rest, '--reply'); const by = flag(rest, '--by');
if (reply === undefined) return usage('answer needs --reply "<text>"');
if (rev === undefined) { const s = readState(out); process.stderr.write(resultLine('gstack-autoplan answer', 'GATE_REV_STALE', `no --gate-rev given; current gate_rev=${s.gate_rev ?? 0}`) + '\n'); return EXIT.refused; }
if (rev === undefined) { const s = readState(out); refuseCeoOnly(s, 'answer'); process.stderr.write(resultLine('gstack-autoplan answer', 'GATE_REV_STALE', `no --gate-rev given; current gate_rev=${s.gate_rev ?? 0}`) + '\n'); return EXIT.refused; }
const replyFile = flag(rest, '--owner-reply-file'); const ownerFlag = flag(rest, '--owner-reply');
if (replyFile !== undefined && ownerFlag !== undefined) return usage('--owner-reply and --owner-reply-file are exclusive');
const wordsBy = flag(rest, '--words-by') ?? 'owner';
if (wordsBy !== 'owner' && (wordsBy !== by || !by.startsWith('agent:'))) return usage(`--words-by must be owner or the answering agent (--by ${by ?? '<unset>'}), got ${wordsBy}`);
const ownerReply = replyFile !== undefined ? fs.readFileSync(replyFile, 'utf8') : ownerFlag === '-' ? fs.readFileSync(0, 'utf8') : ownerFlag;
return mutating(out, json, () => {
const state = readState(out);
const r = answerGate(state, Number(rev), reply, by, { recHash: flag(rest, '--rec-hash'), ownerReply });
refuseCeoOnly(state, 'answer');
const r = answerGate(state, Number(rev), reply, by, { recHash: flag(rest, '--rec-hash'), ownerReply, ...(wordsBy === 'owner' ? {} : { wordsBy }) });
writeState(state);
const lines = [...r.lines];
if (r.outcome === 'reopened' || r.outcome === 'amend_pending') lines.push(nextLine(state, out));
@@ -403,6 +443,7 @@ async function main(argv: string[]): Promise<number> {
if (rev === undefined || !item || !option || !basis?.trim()) return usage('recommend needs --gate-rev <n> --item <id> --option <k> --basis "<why>"');
return mutating(out, json, () => {
const state = readState(out);
refuseCeoOnly(state, 'recommend');
const lines = recommendGate(state, Number(rev), item.toLowerCase(), option.toLowerCase(), basis.trim(), flag(rest, '--by') ?? 'parent');
writeState(state);
emit(json, [...lines, resultLineFor(state.status, out)], { run: state.run, status: state.status, gate_rev: state.gate_rev, out });
bin/gstack-autoplan-snapshot.ts
+1 -1
View File
@@ -55,7 +55,7 @@ For each finding: what's wrong, severity (critical/high/medium), and the fix.`,
reviewing this plan. You have NOT seen any prior review. Evaluate:
1. Getting started: how many steps from zero to hello world? What's the TTHW?
2. API/CLI ergonomics: naming consistency, sensible defaults, progressive disclosure?
3. Error handling: Score every error path against the agent-ready error contract (https://github.com/garrytan/gstack/blob/main/docs/agent-ready-errors.md): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or the precise thing a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). Usage and validation errors rejected before any work need only what, fix, code, redaction and channel; items apply per surface, N/A needs a reason, an item the plan never specifies for a path it introduces is a gap, and unknown is reserved for existing behavior you could not inspect. Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure that names no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item>" (caps the Error Messages score at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item>" (caps it at 6/10).
3. Error handling: Score every error path against the agent-ready error contract (https://github.com/garrytan/gstack/blob/main/docs/agent-ready-errors.md): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or what a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). Usage and validation errors rejected before any work need only what, fix, code, redaction and channel; items apply per surface, N/A needs a reason, an item the plan never specifies for a path it introduces is a gap, and unknown is reserved for existing behavior you could not inspect. Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure naming no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item> <what fails>" (caps Error Messages at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item> <what fails>" (caps it at 6/10).
4. Documentation: copy-paste examples? Information architecture? Interactive elements?
5. Escape hatches: can developers override every opinionated default?
For each finding: what's wrong, severity (critical/high/medium), and the fix.`,
bin/gstack-ceo-mode-handoff
+8 -2
View File
@@ -1,10 +1,11 @@
#!/usr/bin/env bash
# gstack-ceo-mode-handoff — print /plan-ceo-review's Step 0E mode handoff line.
#
# Usage: gstack-ceo-mode-handoff <MODE> [--auto] [--decisions "<rows>"]
# Usage: gstack-ceo-mode-handoff <MODE> [--auto | --ceo-only] [--decisions "<rows>"]
# MODE: HOLD SCOPE | SCOPE EXPANSION | SELECTIVE EXPANSION | SCOPE REDUCTION
# (underscores accepted for spaces)
# --auto the mode came from a saved AUTO_DECIDE preference
# --ceo-only the mode came from 0E's rule in an /autoplan --ceo-only run
# --decisions governing approved rows; defaults to "none"
#
# The handoff used to be chat-only, and the model often composed it in its
@@ -15,10 +16,12 @@ set -euo pipefail
mode="${1:-}"
[ "$#" -gt 0 ] && shift
auto=0
ceo_only=0
decisions="none"
while [ "$#" -gt 0 ]; do
case "$1" in
--auto) auto=1; shift ;;
--ceo-only) ceo_only=1; shift ;;
--decisions)
[ "$#" -ge 2 ] || { echo "gstack-ceo-mode-handoff: --decisions needs a value" >&2; exit 2; }
decisions="$2"; shift 2 ;;
@@ -31,11 +34,14 @@ case "$mode" in
"HOLD SCOPE"|"SCOPE EXPANSION"|"SELECTIVE EXPANSION"|"SCOPE REDUCTION") ;;
*) echo "gstack-ceo-mode-handoff: unknown mode: ${mode:-<empty>}" >&2; exit 2 ;;
esac
[ "$auto$ceo_only" != 11 ] || { echo "gstack-ceo-mode-handoff: --auto and --ceo-only conflict" >&2; exit 2; }
decisions="${decisions//$'\n'/ }"
decisions="${decisions%.}"
[ -n "${decisions// /}" ] || decisions="none"
if [ "$auto" = 1 ]; then
if [ "$ceo_only" = 1 ]; then
echo "Auto-selected review mode → $mode (CEO-only run: Step 0E's rule). Approved decisions: $decisions."
elif [ "$auto" = 1 ]; then
echo "Auto-decided review mode → $mode (your preference). Change with /plan-tune. Approved decisions: $decisions."
else
echo "Mode: $mode; approved decisions: $decisions."
bin/gstack-doctor-components.sh
+6
View File
@@ -338,6 +338,9 @@ gstack_result_anchor() {
RECEIPT_GIT_TOO_OLD) printf '%s' 'receipt-git-too-old' ;;
CHANGELOG_FRAGMENT_INVALID) printf '%s' 'changelog-fragment-invalid' ;;
QUEUE_LAND_LOCKED) printf '%s' 'queue-land-locked' ;;
QUEUE_TRAIN_UNSTAMPED) printf '%s' 'queue-train-unstamped' ;;
RESTAMP_NOT_STAGED) printf '%s' 'restamp-not-staged' ;;
RESTAMP_UNSTAGED_EDITS) printf '%s' 'restamp-unstaged-edits' ;;
QUEUE_LAND_CONFLICT) printf '%s' 'queue-land-conflict' ;;
QUEUE_LAND_DRIFT) printf '%s' 'queue-land-drift' ;;
QUEUE_LAND_BASE_BUSY) printf '%s' 'queue-land-base-busy' ;;
@@ -375,6 +378,9 @@ gstack_result_anchor() {
OFFICE_HOURS_PROBE_REFUSED) printf '%s' 'office-hours-probe-refused' ;;
TASKS_SUPERSEDE_REFUSED) printf '%s' 'tasks-supersede-refused' ;;
REALITY_RECEIPT_UNRESOLVED) printf '%s' 'reality-receipt-unresolved' ;;
CEO_ONLY_CONFLICT) printf '%s' 'ceo-only-conflict' ;;
CEO_ONLY_AFTER_INIT) printf '%s' 'ceo-only-after-init' ;;
CEO_ONLY_NO_GATE) printf '%s' 'ceo-only-no-gate' ;;
*) return 1 ;;
esac
}
bin/gstack-eval-plan
+6 -4
View File
@@ -6,7 +6,7 @@
* gstack-eval-plan arms [--arms <model,model>] [--policy-doc <file>] [--input-tokens N] [--output-tokens N] [--json]
* gstack-eval-plan dry-run --input <rows.jsonl> --arms <model,model> [--fields a,b] [--output-budget N]
* [--retry-prompt <file>] [--max-rows N] [--out <dir>] [--json]
* gstack-eval-plan pilot start --out <dir> --cap <usd> --arms <model[#loadout],...> --dry-run <dry-run.json> [--n 3]
* gstack-eval-plan pilot start --out <dir> --cap <usd> --arms <model[#loadout],...> --dry-run <dry-run.json> [--n 3] [--seat-metered <arm,...>]
* gstack-eval-plan pilot settle --out <dir> --attempt <id> --usd <n|unknown>
* gstack-eval-plan pilot release --out <dir> --attempt <id>
* gstack-eval-plan pilot report --out <dir> --total <items> [--workers W] [--json]
@@ -53,9 +53,11 @@ const HELP = `gstack-eval-plan — preregistration, priced arms, $0 dry run, pri
dry-run --input rows.jsonl --arms m,m [--fields a,b] [--output-budget N] [--retry-prompt f] [--max-rows N] [--cap usd] [--out dir] [--json]
stub model over the real rows; checklist: price table, output budget, retry prompt text,
and with --cap reservation-fits-cap (one call: largest row + output budget, priced)
pilot start --out dir --cap usd --arms m,m#no-gstack --dry-run dry-run.json [--n 3]
pilot start --out dir --cap usd --arms m,m#no-gstack --dry-run dry-run.json [--n 3] [--seat-metered arm,...]
--seat-metered: arms on a subscription with no incremental charge; they reserve $0,
settle unknown, and sit outside the cap and the projection until one settles a charge
pilot settle --out dir --attempt id --usd <n|unknown> pilot release --out dir --attempt id
pilot report --out dir --total items [--workers W] [--json] spend table + extrapolation with a 95% band, then the approval ask
pilot report --out dir --total items [--workers W] [--json] spend table + per-arm extrapolation with a 95% band, then the approval ask
results --check <prereg> <results.md> [--json]
one table in the results file has a filled row for every Arms identity, each no-gstack arm
beside its variants; reads text only (docs/TESTING_INTERNALS.md#no-gstack-arm)
@@ -185,7 +187,7 @@ function doPilot(rest: string[]): number {
if (cap === undefined || !arms.length || !dry || !fs.existsSync(dry)) return usage('pilot start needs --cap <usd> --arms <m,m> --dry-run <existing dry-run.json>');
const rules = lintModels(args, arms.map(model => ({ model, scope: scopeOf(args), where: 'pilot start --arms' })), 'pilot start');
if (!rules.ok) return EXIT.fail;
const r = startPilot({ out, cap, n: num(args, '--n', 3)!, arms, dryRun: dry });
const r = startPilot({ out, cap, n: num(args, '--n', 3)!, arms, seatMetered: csv(flag(args, '--seat-metered')), dryRun: dry });
if (r.error && !r.record) { process.stderr.write(resultLine('gstack-eval-plan pilot start', r.error.code, r.error.message) + '\n'); return EXIT.refused; }
for (const a of r.record.attempts) process.stdout.write(`PILOT_ATTEMPT: ${a.attempt} model=${a.model} estimate=$${a.estimate_usd.toFixed(4)}\n`);
process.stdout.write(`PILOT: status=${r.record.status} reserved=${r.record.attempts.length} cap=$${cap.toFixed(2)} ledger=${path.join(out, 'spend.json')}\n`);
bin/gstack-jsonl-merge
+5 -1
View File
@@ -91,8 +91,12 @@ for path in paths:
sys.exit(1)
# Timestamp-ordered entries first (group 0), then hash-ordered (group 1).
# UTF-8 bytes and LF on every platform: print() encodes with the stdout code
# page and writes CRLF on Windows (#3659).
out = sys.stdout.buffer
for line, _ in sorted(seen.items(), key=lambda item: item[1]):
print(line)
out.write(line.encode('utf-8') + b'\n')
out.flush()
PYEOF
_PYEXIT=$?
bin/gstack-outside-voice
+12 -6
View File
@@ -4,7 +4,7 @@
*
* gstack-outside-voice run --runner codex-cli|api|host-subagent --prompt <file> --out <file>
* [--model <id>] [--result <file>] [--native-model <id>] [--fallback <runner>]
* [--gate review|structured|spec|proposal] [--timeout-ms N] [--cwd <dir>] [--json]
* [--gate review|structured|spec|proposal] [--timeout-ms N] [--max-tokens N] [--cwd <dir>] [--json]
* gstack-outside-voice runners [--model <id>] [--json] the runner table: capabilities and availability here
* gstack-outside-voice family <model-id> the model's family (same-family rule)
*
@@ -20,18 +20,21 @@ import { EXIT, exitTableHelp } from '../lib/headless-artifacts';
import type { OutsideGate } from '../lib/outside-review-result';
import { RUNNER_IDS, createRunner, familyConflict, modelFamily, outsideStatusLine, runnerTable, selectRunner, type RunnerId } from '../lib/outside-voice-runner';
import { outsideConsent } from '../lib/outside-consent';
import { apiTimeoutMs } from '../lib/outside-voice-api';
import { resultLine } from '../lib/result-codes';
const HELP = `gstack-outside-voice — one outside voice, three runners (docs/unattended.md)
run --runner <r> --prompt <file> --out <file> [--model <id>] [--result <file>] [--native-model <id>]
[--fallback <r>] [--gate review|structured|spec|proposal] [--timeout-ms N] [--cwd <dir>] [--json]
[--fallback <r>] [--gate review|structured|spec|proposal] [--timeout-ms N] [--max-tokens N] [--cwd <dir>] [--json]
runners [--model <id>] [--json] the runner table: id, capabilities, available here
family <model-id> print the model family (anthropic, openai, google, ...)
Runners: codex-cli (repository review via codex exec), api (supplied-input review via
OPENAI_API_KEY/ANTHROPIC_API_KEY; needs --model), host-subagent (binds --result from a
cross-family subagent the host ran; needs --model). --native-model enforces the same-family
cross-family subagent the host ran; needs --model). --max-tokens caps the api runner's output
(Anthropic default 16384; its default time limit grows with it); a response cut off at the limit
is unavailable (truncated). --native-model enforces the same-family
rule. GSTACK_OUTSIDE_RUNNER names the fallback when --fallback is absent.
Prints OUTSIDE_STATUS: <status> provider=<runner> model=<id> family=<f> and writes <out>.meta.json.
codex_reviews=disabled (or an unreadable config) refuses every runner before it starts:
@@ -75,9 +78,12 @@ async function main(argv: string[]): Promise<number> {
if (!fs.existsSync(promptFile)) return usage(`prompt file not found: ${promptFile}`);
const gate = (flag(rest, '--gate') ?? 'review') as OutsideGate;
if (!['review', 'structured', 'spec', 'proposal'].includes(gate)) return usage('--gate must be review, structured, spec or proposal');
const maxTokensText = flag(rest, '--max-tokens');
const maxTokens = maxTokensText === undefined ? undefined : Number(maxTokensText);
if (maxTokens !== undefined && (!Number.isInteger(maxTokens) || maxTokens <= 0)) return usage('--max-tokens needs a positive integer');
const timeoutText = flag(rest, '--timeout-ms');
const timeoutMs = timeoutText === undefined ? 300_000 : Number(timeoutText);
if (!Number.isInteger(timeoutMs) || timeoutMs <= 0) return usage('--timeout-ms needs a positive integer');
const timeoutMs = timeoutText === undefined ? undefined : Number(timeoutText);
if (timeoutMs !== undefined && (!Number.isInteger(timeoutMs) || timeoutMs <= 0)) return usage('--timeout-ms needs a positive integer');
const fallbackText = flag(rest, '--fallback') ?? process.env.GSTACK_OUTSIDE_RUNNER;
if (fallbackText !== undefined && !isRunner(fallbackText)) return usage(`--fallback must be one of ${RUNNER_IDS.join(', ')}`);
const model = flag(rest, '--model');
@@ -110,7 +116,7 @@ async function main(argv: string[]): Promise<number> {
const stop = () => runner.cancel();
process.on('SIGTERM', stop);
process.on('SIGINT', stop);
const result = await runner.run({ promptFile: path.resolve(promptFile), outFile: path.resolve(outFile), model, timeoutMs, cwd, gate, resultFile: resultFile ? path.resolve(resultFile) : undefined });
const result = await runner.run({ promptFile: path.resolve(promptFile), outFile: path.resolve(outFile), model, timeoutMs: timeoutMs ?? (choice.runner === 'api' ? apiTimeoutMs(maxTokens) : 300_000), cwd, gate, maxTokens, resultFile: resultFile ? path.resolve(resultFile) : undefined });
fs.mkdirSync(path.dirname(path.resolve(outFile)), { recursive: true });
fs.writeFileSync(`${path.resolve(outFile)}.meta.json`, JSON.stringify(result, null, 2) + '\n');
if (json) process.stdout.write(JSON.stringify(result) + '\n');
bin/gstack-restamp
+71 -10
View File
@@ -10,11 +10,18 @@
* Runs gstack-version-bump write, the policy's release_tool, the CHANGELOG
* re-heading and the stamp_paths rewrites in an isolated staging worktree,
* shows the diff limited to stamp files, and writes the branch only after
* every stage validated (journal: .gstack/tmp/restamp-journal.json).
* Idempotent; --dry-run writes nothing. Workflow command: ends with the
* every stage validated (journal: .gstack/tmp/restamp-journal.json), then
* stages what it wrote (`git add -A -- <FILES>`, STAGED:), so a bare commit after
* a --no-commit base merge cannot drop the stamp (#3531). A stamp file holding
* unstaged edits of the caller's own refuses before anything is written
* (RESTAMP_UNSTAGED_EDITS); a staging failure leaves the files written and
* .gstack/tmp/restamp-staging.json, which the noop rerun stages, or drops when a
* file changed since (RESTAMP_NOT_STAGED). Idempotent; --dry-run writes nothing. Workflow command: ends with the
* GSTACK_RESULT line. Logic: lib/restamp.ts; policy: lib/ship-policy.ts.
*/
import { spawnSync } from 'node:child_process';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { EXIT, exitTableHelp, formatResult } from '../lib/headless-artifacts';
import { GATE_AHEAD_REL, JOURNAL_REL, restamp, type RestampResult } from '../lib/restamp';
import { RESULT_CODES, type ResultCodeName } from '../lib/result-codes';
@@ -39,7 +46,7 @@ const HELP = `gstack-restamp — stamp the version, release tool, CHANGELOG and
--dry-run print the full diff, write nothing
--json JSON only on stdout
Prints STAMP_VERSION: <X>, the stage table, the stamp diff and
Prints STAMP_VERSION: <X>, the stage table, the stamp diff, STAGED: <files it staged> and
GSTACK_RESULT: skill=restamp status=complete|refused|incomplete run=${JOURNAL_REL}.
${exitTableHelp()}`;
@@ -48,6 +55,49 @@ function flag(args: string[], name: string): string | undefined {
return i >= 0 ? args[i + 1] : undefined;
}
const STAGING_REL = '.gstack/tmp/restamp-staging.json';
interface Pending { files: string[]; blobs: Record<string, string | null> }
interface Staging { staged?: string[]; dropped?: string[]; failure?: string }
const gitIn = (root: string, args: string[]) => spawnSync('git', args, { cwd: root, encoding: 'utf8', timeout: 60_000 });
/** Each file's blob sha in the worktree (one git call), null for a file that does not exist. */
function blobsOf(root: string, files: string[]): Record<string, string | null> {
const present = files.filter(f => fs.existsSync(path.join(root, f)));
const shas = present.length ? gitIn(root, ['hash-object', '--', ...present]).stdout.trim().split('\n') : [];
return Object.fromEntries(files.map(f => [f, present.includes(f) ? shas[present.indexOf(f)] ?? '' : null]));
}
function readPending(root: string): Pending | null {
try { return JSON.parse(fs.readFileSync(path.join(root, STAGING_REL), 'utf8')) as Pending; } catch { return null; }
}
/** Files with unstaged edits that are the caller's own (not bytes a pending restamp wrote and could not stage). */
function ownUnstaged(root: string): string[] {
const pending = readPending(root);
const unstaged = gitIn(root, ['diff', '--name-only', '-z']).stdout.split('\0').filter(Boolean);
const now = pending ? blobsOf(root, unstaged.filter(f => f in pending.blobs)) : {};
return unstaged.filter(f => !(pending && f in now && pending.blobs[f] === now[f]));
}
/**
* Stage what restamp wrote (`written`), or what a pending marker says is still unstaged; a marker file whose bytes
* changed since (a hand commit, another branch, a base merge) is stale for that file and dropped, never a failure.
* Null when there is nothing to stage.
*/
function stage(root: string, written: string[] | null): Staging | null {
const marker = path.join(root, STAGING_REL);
const pending = written ? { files: written, blobs: blobsOf(root, written) } : readPending(root);
if (!pending?.files.length) return null;
if (written) { fs.mkdirSync(path.dirname(marker), { recursive: true }); fs.writeFileSync(marker, JSON.stringify(pending, null, 2) + '\n'); }
const now = written ? pending.blobs : blobsOf(root, pending.files);
const dropped = pending.files.filter(f => now[f] !== pending.blobs[f]);
const files = pending.files.filter(f => !dropped.includes(f));
if (!files.length) { fs.rmSync(marker, { force: true }); return { staged: [], dropped }; }
const add = gitIn(root, ['add', '-A', '--', ...files]);
if (add.status !== 0) return { failure: `git add of ${files.join(' ')} failed (${(add.stderr || add.stdout).trim().split('\n')[0]}); the stamp stays written and ${STAGING_REL} names it` };
fs.rmSync(marker, { force: true });
return { staged: gitIn(root, ['diff', '--cached', '--name-only', '-z', '--', ...files]).stdout.split('\0').filter(Boolean), dropped };
}
function main(argv: string[]): number {
if (argv.includes('--help') || argv.includes('-h')) { process.stdout.write(HELP + '\n'); return EXIT.ok; }
const json = argv.includes('--json');
@@ -65,23 +115,32 @@ function main(argv: string[]): number {
const after = flag(argv, '--after');
const base = resolveBaseBranch(repoRoot, flag(argv, '--base'));
const result = restamp({
const options = {
repoRoot, base, version: flag(argv, '--version'), next: argv.includes('--next'), bump, queueMode,
after: after ? { pr: after, ref: flag(argv, '--predecessor-ref') } : undefined, pr: flag(argv, '--pr'),
expectBase: flag(argv, '--expect-base'), was: flag(argv, '--was'), expectOrder: flag(argv, '--expect-order')?.split(',').map(s => s.trim()).filter(Boolean),
policyFrom, allowRepoCommands: argv.includes('--allow-repo-commands'), dryRun: argv.includes('--dry-run'),
failAfter: process.env.GSTACK_RESTAMP_FAIL_AFTER,
});
return report(result, json);
};
const writes = !after && !options.dryRun;
const own = writes ? ownUnstaged(repoRoot) : [];
const dry = own.length ? restamp({ ...options, dryRun: true }) : null;
if (dry && (dry.status === 'refused' || dry.status === 'failed')) return report(dry, json);
const clash = (dry?.files ?? []).filter(f => own.includes(f));
if (clash.length) return report({ status: 'refused', code: 'RESTAMP_UNSTAGED_EDITS', message: `${clash.join(', ')} ${clash.length === 1 ? 'holds' : 'hold'} unstaged edits of your own, which staging the stamp would sweep into the stamp commit; nothing written`, stages: [] }, json);
const result = restamp(options);
const staging = writes && (result.status === 'stamped' || result.status === 'noop') ? stage(repoRoot, result.status === 'stamped' ? result.files ?? [] : null) : null;
return report(result, json, staging);
}
function report(r: RestampResult, json: boolean): number {
const status = r.status === 'refused' ? 'refused' : r.status === 'failed' ? 'incomplete' : 'complete';
const code = r.status === 'refused' ? EXIT.refused : r.status === 'failed' ? EXIT.fail : EXIT.ok;
function report(r: RestampResult, json: boolean, staging: Staging | null = null): number {
const status = r.status === 'refused' ? 'refused' : r.status === 'failed' || staging?.failure ? 'incomplete' : 'complete';
const code = r.status === 'refused' ? EXIT.refused : r.status === 'failed' || staging?.failure ? EXIT.fail : EXIT.ok;
if (json) {
process.stdout.write(JSON.stringify({ ...r, result: status }) + '\n');
process.stdout.write(JSON.stringify({ ...r, ...(staging ?? {}), ...(staging?.failure && !r.code ? { code: 'RESTAMP_NOT_STAGED' } : {}), result: status }) + '\n');
return code;
}
if (staging?.failure) process.stderr.write(`gstack-restamp: ${staging.failure}; fix: ${RESULT_CODES.RESTAMP_NOT_STAGED.fix} (RESTAMP_NOT_STAGED)\n`);
if (r.code) {
const row = RESULT_CODES[r.code as ResultCodeName];
process.stderr.write(`gstack-restamp: ${r.message}; fix: ${r.fix ?? row.fix} (${r.code})\n`);
@@ -95,6 +154,8 @@ function report(r: RestampResult, json: boolean): number {
if (r.gateAhead) process.stdout.write(`GATE_AHEAD: head=${r.gateAhead.gated_head} tree=${r.gateAhead.gated_tree} predecessor=#${r.gateAhead.predecessor_pr}@${r.gateAhead.predecessor_head.slice(0, 12)} worktree=${r.gateAhead.worktree} evidence_branch=${r.gateAhead.evidence_branch_key}\n`);
if (r.diff) process.stdout.write(r.diff + (r.diff.endsWith('\n') ? '' : '\n'));
process.stdout.write(`FILES: ${(r.files ?? []).join(' ') || '(none)'}\n`);
if (staging?.staged) process.stdout.write(`STAGED: ${staging.staged.join(' ') || '(none)'}\n`);
if (staging?.dropped?.length) process.stdout.write(`NOTE: ${staging.dropped.join(' ')} changed since an earlier restamp wrote ${staging.dropped.length === 1 ? 'it' : 'them'}; not staged, and that run's staging marker is dropped\n`);
if (r.mergeCondition) process.stdout.write(`MERGE_CONDITION: sha=${r.mergeCondition}\n`);
if (r.status === 'gate-ahead' && r.gateAhead) process.stdout.write(`next: run the verification gate in ${r.gateAhead.worktree}, then \`gstack-evidence bundle --out <file>\` there; the branch is untouched. At this PR's turn rerun \`gstack-restamp --next\` here and read \`gstack-tree-receipt --from ${GATE_AHEAD_REL}\` before deciding what to rerun.\n`);
else if ((r.status === 'stamped' || r.status === 'noop') && r.queueMode === 'stamp-at-merge') process.stdout.write(`next: NEW_VERSION=${r.newVersion}; commit and push only the stamp diff above (FILES), with no edit of your own: the train's verify accepts the CHANGELOG only as the base's plus this PR's entry re-headed; merge with the expected head ${r.mergeCondition ?? 'sha'} and recompute on a stale queue.\n`);
bin/gstack-review-scope
+3 -2
View File
@@ -23,6 +23,7 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import { EXIT, exitTableHelp } from '../lib/headless-artifacts';
import { resultLineWithDocs } from '../lib/result-codes';
import { runScriptSync } from '../lib/script-invocation';
import { isUnavailable, loadPolicyFromBase, policyUnavailableLine, resolveBaseBranch } from '../lib/ship-policy';
import { EXIT as POST_EXIT, runPost } from '../lib/gstack-post';
import {
@@ -237,8 +238,8 @@ function cmdRender(rest: string[]): number {
function cmdSummary(rest: string[], json: boolean): number {
const since = flag(rest, '--since');
if (!since || !Number.isFinite(Date.parse(since))) return refuse('summary', 'REVIEW_SCOPE_INPUT', '--since <ISO-8601 timestamp> (the invocation\'s first Step 9 entry) is required');
const read = spawnSync(path.join(path.dirname(Bun.fileURLToPath(import.meta.url)), 'gstack-review-read'), [], { encoding: 'utf8', timeout: 60_000, maxBuffer: 64 * 1024 * 1024 });
if (read.status !== 0) return refuse('summary', 'REVIEW_SCOPE_INPUT', `gstack-review-read failed: ${(read.stderr ?? '').trim().split('\n')[0] ?? ''}`);
const read = runScriptSync(path.join(path.dirname(Bun.fileURLToPath(import.meta.url)), 'gstack-review-read'), [], { encoding: 'utf8', timeout: 60_000, maxBuffer: 64 * 1024 * 1024 });
if (read.status !== 0) return refuse('summary', 'REVIEW_SCOPE_INPUT', `gstack-review-read failed: ${read.error?.message ?? (read.stderr ?? '').trim().split('\n')[0] ?? ''}`);
const rows: Record<string, unknown>[] = [];
for (const line of (read.stdout ?? '').split('---CONFIG---')[0]!.split('\n')) {
if (!line.trim() || line === 'NO_REVIEWS') continue;
bin/gstack-ship-queue
+3 -3
View File
@@ -14,7 +14,7 @@
* gstack-ship-queue gate-ahead --list --pr <n>
* gstack-ship-queue verify --pr <n> --screen (train policy: the head's screen verdict from its run)
* gstack-ship-queue verify --post-merge <sha> [--timeout <s>] (after a merge: main's push workflows at that commit until they settle)
* gstack-ship-queue land --pr <n> [--dry-run] [--no-wait] [--break-lock] (the merge step; lib/ship-queue-land.ts)
* gstack-ship-queue land --pr <n> [--dry-run] [--no-wait] [--break-lock [--break-train-lock]] (the merge step; lib/ship-queue-land.ts)
* gstack-ship-queue land --pr <n> --repair (after QUEUE_LAND_DRIFT: open the outside merge's release PR)
* gstack-ship-queue land --census
* gstack-ship-queue train build|status|verdict|bisect|rebuild|watch|prune ... (merge trains; train --help)
@@ -57,7 +57,7 @@ const HELP = `gstack-ship-queue — the shared merge queue's protocol steps (doc
verify --pr <n> --screen (train policy) the PR head's screen verdict read from its run (SCREEN_VERDICT)
verify --post-merge <sha> [--timeout <s>] wait for every check run at a merged commit to settle: POST_MERGE: green|red|pending
and one CHECK: line per workflow; resumable, default timeout 2700 s
land --pr <n> [--dry-run] [--no-wait] [--timeout <min>] [--break-lock] [--allow-repo-commands]
land --pr <n> [--dry-run] [--no-wait] [--timeout <min>] [--break-lock [--break-train-lock]] [--allow-repo-commands]
the merge step: lock, merge the base, stamp (fragments folded), push the stamp commit,
wait for its checks, squash with --match-head-commit, confirm the base tree; resumable
land --pr <n> --repair after QUEUE_LAND_DRIFT: open the PR that gives the merge outside land its own
@@ -172,7 +172,7 @@ async function main(argv: string[]): Promise<number> {
if (args.includes('--census')) return finish('land', l.census(ctx));
if (!pr || !/^\d+$/.test(pr)) return usage('land needs --pr <number> (or --census)');
const timeout = flag(args, '--timeout');
return finish('land', l.land(ctx, { pr, repair: args.includes('--repair'), dryRun: args.includes('--dry-run'), noWait: args.includes('--no-wait'), breakLock: args.includes('--break-lock'), allowRepoCommands: args.includes('--allow-repo-commands'), waitTimeoutMin: timeout && /^\d+$/.test(timeout) ? Number(timeout) : undefined }));
return finish('land', l.land(ctx, { pr, repair: args.includes('--repair'), dryRun: args.includes('--dry-run'), noWait: args.includes('--no-wait'), breakLock: args.includes('--break-lock'), breakTrainLock: args.includes('--break-train-lock'), allowRepoCommands: args.includes('--allow-repo-commands'), waitTimeoutMin: timeout && /^\d+$/.test(timeout) ? Number(timeout) : undefined }));
}
if (!pr || !/^\d+$/.test(pr)) return usage(`${cmd} needs --pr <number>`);
const gateExit = flag(args, '--gate-exit');
bin/gstack-ship-receipt
+2 -1
View File
@@ -37,6 +37,7 @@ import * as path from 'node:path';
import { EXIT, exitTableHelp, type ShipReceipt } from '../lib/headless-artifacts';
import { resultLine, resultLineWithDocs } from '../lib/result-codes';
import { versionAtRef } from '../lib/restamp';
import { runScriptSync } from '../lib/script-invocation';
import { readPregateFile, verifyPregate } from '../lib/pregate/index';
import { isUnavailable, loadPolicyFromBase, matchesAny, policyUnavailableLine, resolveBaseBranch, runningGstackVersion, stampFilesInTree, validatePolicy, type ShipPolicy } from '../lib/ship-policy';
import { approvalForReceipt } from '../lib/approval-citations';
@@ -179,7 +180,7 @@ function cmdWrite(rest: string[], json: boolean): number {
const tree = receiptFile ? JSON.parse(fs.readFileSync(receiptFile, 'utf8')) : null;
let sessionKind = flag(rest, '--session-kind') ?? process.env.GSTACK_SESSION_KIND;
if (!sessionKind) {
const r = spawnSync(path.join(path.dirname(Bun.fileURLToPath(import.meta.url)), 'gstack-session-kind'), [], { encoding: 'utf8', timeout: 15_000 });
const r = runScriptSync(path.join(path.dirname(Bun.fileURLToPath(import.meta.url)), 'gstack-session-kind'), [], { encoding: 'utf8', timeout: 15_000 });
sessionKind = r.status === 0 ? r.stdout.trim() || undefined : undefined;
}
let pregate: PregateField | undefined;
captain/templates/slot-call.md
+6 -4
View File
@@ -12,10 +12,12 @@ Run: gstack-restamp --next --base <BASE> --pr <n> --expect-base <sha> --expect-o
Slot 2 and later (and slot 1 when BASE moved): merge BASE into your pin first (expect a
CHANGELOG-only conflict; resolve it with mergeChangelogs from lib/changelog-check.ts, the
one-liner in docs/ship-policy.md's merge-train section, never by hand; grep for conflict
markers before you stamp). Commit only the BASE merge and the stamp files this call names:
<STAMP_FILES> (plus the file carrying a `*_KEY_VERSION` your PR introduced, on your own
line only, when this call names it), never a fifth: a file the restamp lists that this call
does not name is not a stamp, leave it out and say so.
markers before you stamp). The restamp stages what it writes (STAGED:); commit what it
staged, which must be only the BASE merge and the stamp files this call names: <STAMP_FILES>
(plus the file carrying a `*_KEY_VERSION` your PR introduced, on your own line only, when this
call names it), never a fifth: a file the call does not name is not a stamp, `git restore
--staged --worktree` it and say so. verify reads the head's own STAMP_VERSION and refuses an unstamped
head (QUEUE_TRAIN_UNSTAMPED).
A pin stacked on a PR that merged earlier conflicts beyond the CHANGELOG where the parent's
squash touched your lines: take your pin's bytes there (verify checks the tree against
T<i>); mergeChangelogs carries your edits to the parent's released entry as Gate integrity
capy/skill/gstack/SKILL.md
+1 -1
View File
@@ -46,7 +46,7 @@ Capy checklist for `/autoplan`: drive `gstack-autoplan`, never a hand adaptation
Token caps on /ship: pass a brief's cap as `--token-cap <n>`. /ship reads `cost.tokens` and `cost.credits` from `tools.thread.get({ thread_id: "<your own jam id>" })` (its own jam; a subagent reading root `"this"` starts `--subagent --reading-from this --meter-scope tree`) inside the same execute program as the `gstack-ship-budget` command that takes them as `--reading <n> --credits <c>`, never alone. `ship/DRIVER.md` starts `--mode observe`: each program ends with `note`, and only a note past the cap stops (`docs/ship-budget.md`).
/ship on Capy is about a dozen calls, each one execute program (`ship/DRIVER.md`). Each review pass and Step 11 is one call to the checked-in pass program (fill only `input`): `const t0 = await now(); const p = (await tools.capy.bash({ command: ". '<launch file>' && gstack-review-dispatch program", description: "Pass" })).output.split("\n"); return p[0] + "\n" + (await new (Object.getPrototypeOf(async () => {}).constructor)("tools", "sleep", "now", "input", JSON.parse(p[1]))(tools, sleep, now, { startedAt: t0, launch: "<launch file>", cwd: "<repo>", base: "<base>", out: "<run dir>/p<n>", pass: "<run>-p<n>", laneModel: "<your model id>", planFlags: [], logStart: ["review"], jam: "<jam id>", record: "<rec>", step: "9.2" }));` It plans, logs `logStart` (`reservation` mode from `RESERVATION_ID:`), sends each brief on `machine: "shared"`, then polls: one `tools.subagent.list` per ~20 s tick, returning 200 s after the program's first line, where a reader is settled only at status `done` or `failed` or with a `summary` (a just-created `idle` reader has not run); it starts readers under `rejected` with `tools.subagent.start` as slots free and stops on one stuck 20 minutes. It ends on `NEXT:` (`poll`: rerun) or `STOP:`. `models` maps roles to models (`outside-voice`: another family); Step 11 adds `stage: "voices"`; a full repeat pass, `diffBase: "<RANGE_BASE>"`. Steps 7, 8, 9.2.1 and 14.5 (`step: "14.5"`) use `extras`, JSON `{name, prompt_file, first, model}` (prompts verbatim from their sections, Step 7 `first`); they cannot block, so `model: { model_id: "anthropic/claude-sonnet-5-5", reasoning_mode: "low" }` (`anthropic/claude-haiku-5-5` also fits). Waits: never let 5 minutes pass between calls. While readers run, never end the turn on `wait`; rerun the loader. Otherwise re-issue `wait` with `duration_seconds: 200` and operation ids. At Step 22 report the `CALLS:` line of `gstack-ship-budget calls --events <file>` on your jam's transcript since /ship began. A small waiting child saves nothing.
/ship on Capy is about a dozen calls, each one execute program (`ship/DRIVER.md`). Each review pass and Step 11 is one call to the checked-in pass program (fill only `input`): `const t0 = await now(); const p = (await tools.capy.bash({ command: ". '<launch file>' && gstack-review-dispatch program", description: "Pass" })).output.split("\n"); return p[0] + "\n" + (await new (Object.getPrototypeOf(async () => {}).constructor)("tools", "sleep", "now", "input", JSON.parse(p[1]))(tools, sleep, now, { startedAt: t0, launch: "<launch file>", cwd: "<repo>", base: "<base>", out: "<run dir>/p<n>", pass: "<run>-p<n>", laneModel: "<your model id>", planFlags: [], logStart: ["review"], jam: "<jam id>", record: "<rec>", step: "9.2" }));` It plans, logs `logStart` (`reservation` mode from `RESERVATION_ID:`), sends each brief on `machine: "shared"`, then polls: one `tools.subagent.list` per ~20 s tick, returning 200 s after the program's first line, where a reader is settled only at status `done` or `failed` or with a `summary` (a just-created `idle` reader has not run); it starts readers under `rejected` with `tools.subagent.start` as slots free and stops on one stuck 20 minutes. It ends on `NEXT:` (`poll`: rerun) or `STOP:`. `models` maps roles to models (`outside-voice`: another family); Step 11 adds `stage: "voices"`; a full repeat pass, `diffBase: "<RANGE_BASE>"`. Steps 7, 8 and 14.5 (`step: "14.5"`) use `extras`, JSON `{name, prompt_file, first, model}` (prompts verbatim from their sections, Step 7 `first`); they cannot block, so `model: { model_id: "anthropic/claude-sonnet-5-5", reasoning_mode: "low" }` (`anthropic/claude-haiku-5-5` also fits). Waits: never let 5 minutes pass between calls. While readers run, never end the turn on `wait`; rerun the loader. Otherwise re-issue `wait` with `duration_seconds: 200` and operation ids. At Step 22 report the `CALLS:` line of `gstack-ship-budget calls --events <file>` on your jam's transcript since /ship began. A small waiting child saves nothing.
## Plan, approve, ship
changelog.d/gst-850-allowlist-docs-paths.md
-5
View File
@@ -1,5 +0,0 @@
- **The security class reaches outbound URLs, archives and every `child_process` spelling** (GST-850). `lib/review-risk.ts` now flags any call to an outbound HTTP client (`fetch`, `http(s).get` or `.request`, axios, got, undici, `requests`, `httpx`, urllib), `new URL` from input, redirect toggles, proxy settings and host allowlists; tar, zip and unzip extraction, `path.join` or `resolve` on request or entry input, and `'..'` checks; and any changed line that names the `child_process` module. The new rules parse no import clause or call arguments, and a test holds each under 100 ms on 200,000-character hostile lines.
- **The money class reads metering and charge code.** Paths and lines naming meter, metering, metered, charge, billable, billed or credits, plus token usage, a billing proxy and cost or charge clients, now route to the money reader. gbrain-evals #289, which changed a metering proxy and billing clients and read `required=none`, is now a regression test that reads money. "costume", a CLI's usage text and a plain proxy do not count.
- **When a diff publishes numbers, the batched specialist reads the write-up's text files** instead of leaving docs out of its diff. The collector records the files a diff creates.
- **UI text and SVGs stay code (light).** An allowlist docs path for chart SVGs and locale values was built and dropped for review cost: its first review round found a new bypass class in each version. Every bypass found so far is a test row that classes as code.
- Replayed on the 26 GSTA-167 no-loss units, no unit loses a reviewer role. The replay checked which reviewers get dispatched, not R51's re-scored findings. Over the last 100 main commits, 3 move from light to full and no commit with a gate or risk class drops depth.
docs/TESTING_INTERNALS.md
+77 -14
View File
@@ -1002,7 +1002,7 @@ with `UBICLOUD: left=<0|1|unknown> owner=<o> vm=<name> others=<n>`, checked
against the project's VM list after teardown; anything but `left=0` turns a
passing command into exit 1. pregate-remote.sh reads that line from the file
named by `UBI_LEFT_FILE`, never from the command's output. No owner-age sweep runs by default (the TTL reaper
below covers every owner); set
below covers this owner's expired VMs); set
`UBI_GC_HOURS` to a positive number (or run `gc HOURS`; `gc 0` is a no-op) to
sweep only this owner's VMs older than that. For other commands, use the runner
directly: `scripts/ubicloud/ubi-runner.sh run --setup <script> -- '<command>'`,
@@ -1011,12 +1011,12 @@ or its `up` / `ssh` / `sync` / `pull` / `down` steps (`--help` lists them).
The runner is also the `/ubicloud` skill's engine (`bin/gstack-ubi` execs it).
A generated name ends in `-t<hours>` (`UBI_TTL_HOURS`, 1-999, default 3; pregate-remote.sh
uses its stop deadline plus 15 minutes, rounded up: `-t1` today), the creator's own statement of how long the VM may live: `reap`
destroys every VM in the project whose TTL ended more than ten minutes ago,
whichever owner made it, and the caller's own `ubirun-<UBI_OWNER>-*` names
without a TTL (older runner copies) once they are `UBI_LEGACY_HOURS` old
(default 4). Another owner's TTL-less VM and an untagged one are never reaped,
because only a TTL lets reap judge another lane's VM. Every `up` runs it
first (`UBI_REAP=0` skips it). Names outside the `ubirun-` prefix are never reaped. The
destroys only the caller's own VMs: its `ubirun-<UBI_OWNER>-*` names whose TTL
ended more than ten minutes ago, and those without a TTL (older runner copies)
once they are `UBI_LEGACY_HOURS` old (default 4). Another owner's VM and an
untagged one are never reaped, even after their TTL ends; reap only counts them
in its log, because cloud deletes stay with the owner that created the VM
(GST-573). Every `up` runs it first (`UBI_REAP=0` skips it). Names outside the `ubirun-` prefix are never reaped. The
quota preflight adds the size's vCPUs to the project's running VMs and refuses
before the create request when that exceeds the known ceiling
(`UBI_QUOTA_VCPUS`, or the maximum the last refusal printed, remembered in the
@@ -1070,7 +1070,7 @@ machine swap, in this order:
one side local and the other remote.
4. Profiling very large imports (a whole-repo ingest, the full free suite under
a profiler) is warned about before it starts: it holds the VM for hours, and
any other lane's `up` reaps it once its TTL (`UBI_TTL_HOURS`, default 3)
the same owner's next `up` or `reap` destroys it once its TTL (`UBI_TTL_HOURS`, default 3)
plus ten minutes has passed, so set `UBI_TTL_HOURS` above the expected
duration and name that duration in the thread before launching.
@@ -1428,10 +1428,13 @@ No free test may write the developer's real home. `scripts/lib/free-home-guard.t
snapshots `~/.gstack`, `~/.claude`, `~/.codex`, `~/.agents` and `~/.config/gstack`
(live session logs excluded) and fails the run when an entry changes.
- A shard that runs alone (`--shard`, CI, the exclusive host-state shard, the
flaky retry) owns its window, so the failure names its files.
- Full-suite shards run concurrently on one HOME and cannot tell whose write a
change was. The runner guards that whole phase once and names no shard.
- A shard that runs alone (`--shard`, a CI shard at `--jobs 1` or its trailing
serial sub-shard, the exclusive host-state shard, the flaky retry) owns its
window, so the failure names its files.
- Full-suite shards, and the parallel sub-shards of a CI shard at `--jobs k > 1`
([sub-shards](#sub-shards-inside-one-ci-job)), run concurrently on one HOME and
cannot tell whose write a change was. The runner guards that whole phase once
and names no shard.
- `bun run scripts/test-free-shards.ts --attribute-home` names the writer: it
runs every file alone with a private HOME (browser cache and git identity
still come from the real home) and reports each file that wrote a watched
@@ -1450,7 +1453,13 @@ Monday "Weekly test health" workflow (`bun run test:health --enforce`) reads
main's ledgers through `scripts/lib/ci-history.ts` and fails when a file flakes
in more than 5% of main free-tests runs over a window of at least 20 runs; a
window thinner than 20 runs in the last 7 days widens to the newest 20 runs, and
fewer than 20 runs is report-only. `bun run test:health flake-summary <dir>`
fewer than 20 runs is report-only. A main push that reused its pull request's
run ([CI reuse](pregate.md#ci-reuse)) ran no shards, so it stays out of the
window and its denominator, and older runs take its place. Such a run has no
artifacts and a green `reuse` job; checking that costs one job listing per
run with no artifacts, at most 40 per read. Past that cap, or when a listing
fails, the run counts as before and the report says so.
`bun run test:health flake-summary <dir>`
prints a run's ledgers as a table. A flaky file is fixed at source; the retry
never masks it. `bun run eval:pass-rates` lists only the local ledger
(`flakeLedgerPath()`, overridden by `GSTACK_FLAKE_LEDGER`).
@@ -1547,7 +1556,7 @@ refresh to a maintainer.
(`ubicloud-standard-4`); both are Ubicloud standard runners. When either recording adds
or drops a test file, it force-updates the `bot/free-duration-seeds` branch,
opens or refreshes one seed PR whose body states the ratchet result, and
dispatches Free Tests, Windows Free Tests and Quality gate on it. Those
dispatches Free Tests, Windows Smoke and Quality gate on it. Those
dispatched runs are the seed PR's checks, and Free Tests supplies the required
`free-tests` check. GitHub holds the pull_request runs of a token-opened PR as
"action required" until someone approves them. A file that fails while recording turns its
@@ -1559,8 +1568,62 @@ refresh to a maintainer.
besides the bot's one refresh commit, later runs leave it alone. Only runs on
`main` push the branch; a dispatch from another ref reports only.
### Sub-shards inside one CI job
`--ci-run <plan> --shard i --result <file> --jobs k` runs one planned CI shard as
k sub-shards side by side on the same machine (`scripts/lib/shard-jobs.ts`). Each
sub-shard is an ordinary free shard child: one `bun test` process, serial
inside, with its own sandbox, process-group kill and log. Bun's worker
`--parallel` mode is still not used (decision V3/D6 in
`scripts/test-free-shards.ts`). The default is `k = 1`, which runs today's one
child, so no workflow changes behavior until it passes `--jobs`.
- `--jobs` applies only to `--ci-run`. It takes a positive whole number up to the
shard's file count (an empty shard takes 1); anything else stops with
`FREE_JOBS_INVALID`. `GSTACK_FREE_JOBS` sets the local full-suite process count
and never changes `--ci-run`. The exclusive host-state shard has one file, so
a workflow passes it `--jobs 1` or no `--jobs`.
- Files are split into disjoint sub-shards packed by the duration seed. Each
sub-shard has an id (`<plan id prefix>:<shard>.<n>`) bound to its file list.
Files in `TREE_MUTATING` or `SLOW_UNDER_LOAD` run after the parallel
sub-shards settle, serially, in one trailing sub-shard.
- The parallel phase uses the concurrent-phase real-home guard, the same one
full-suite mode uses; the serial sub-shard guards itself and names its files.
- The CI shard keeps its k = 1 wall deadline: the parallel sub-shards get it,
and the serial sub-shard gets whatever is left of it, so k never extends a
deadline. The parent waits for every child to settle.
- The parent merges every sub-shard result into the shard's one `outcome`
before the single flaky-retry pass, so the five-file retry cap stays per CI
shard. A missing, duplicate or incomplete sub-shard result fails the shard
with `FREE_SUBSHARD_REJECTED` and vetoes the retry. `--ci-verify` re-checks
the `jobs` and `subShards` records a `k > 1` result carries.
- Sub-shard runs never write `scripts/free-test-durations.json`; only
`--record-durations` does, and only from one-file children.
`FREE_SUBSHARD_TIMEOUT` means a sub-shard hit its wall at `k > 1`. The message
names the file in flight, k, the runner and the CI run id, and prints a serial
reproduce command. If the file passes serially, add it to `SLOW_UNDER_LOAD` in
`scripts/lib/shard-jobs.ts` with that run id and a reason. Never raise the
timeout. `SLOW_UNDER_LOAD` is empty today.
Every `--ci-run` result, at any k, records per-file test and `expect()` counts
in `outcome.counts` (`tests`, `pass`, `fail`, `skip`, `todo`, `expects` per
file, plus shard totals). They come from bun's JUnit reporter, which the runner
writes into the shard sandbox; on the pinned Bun 1.4.2 it attributes both counts
per file inside a multi-file child and leaves the console stream unchanged.
`complete: false` (with a `problem`) means the report was missing or did not add
up to bun's own test count. Counts are evidence for comparing k values; they
never decide a verdict.
### Windows free lane
Windows has minimal support ([support tiers](support-tiers.md)). The required
PR check is `windows-smoke.yml`: one `windows-latest` job runs `./setup`, starts
the core skills, drives `/browse` on gstack's own browser, and runs the short
test list in its last step, where each file's reason sits beside it.
`test/windows-native-workflows.test.ts` keeps that list Windows-safe and fast.
The curated suite below runs nightly on main and on dispatch, never on a PR.
`windows-free-tests.yml` plans on Linux (`--windows-only --ci-plan --shards 6`),
runs one strict serial shard per `windows-latest` job
(`bun run test:windows --ci-run`), and verifies every shard receipt on Linux
docs/agent-ready-errors.md
+2 -2
View File
@@ -37,8 +37,8 @@ unknown is reserved for existing behavior you could not inspect. In a live audit
you never triggered are unknown, not extrapolated.
**Caps.** Name the path and item behind every cap and show the corrected error line.
- **4/10 (Critical finding, title `CAP4:<path>:<item>`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as `--yes`).
- **6/10 (High finding, title `CAP6:<path>:<item>`):** a path lacks an applicable fix, recovery, context or trace item.
- **4/10 (Critical finding, title `CAP4:<path>:<item> <what fails>`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as `--yes`).
- **6/10 (High finding, title `CAP6:<path>:<item> <what fails>`):** a path lacks an applicable fix, recovery, context or trace item.
Every mode, DX TRIAGE included, reports every CAP4 finding.
docs/autoplan-guard-troubleshooting.md
+7
View File
@@ -257,6 +257,13 @@ the same three paths. An errored init is skipped, so a later successful init
still binds. This denial applies when no init ever succeeded, or when an
errored init follows a good one.
An init whose output is not its JSON result (extra text, or no single text
block) also counts as failed, and the message adds "without a leading cd" to
the re-run hint. The one exception is the `Shell cwd was reset to <dir>` line
Claude Code adds as the last line of a Bash result when the command's `cd`
left the session's folder: the guard removes exactly that line before reading
an init or checkpoint result (#3651).
<a id="init-unbindable"></a>
### `init_unbindable`
Init ran through a shell variable, substitution, chaining, a pipe or a
docs/captain.md
+4 -3
View File
@@ -172,7 +172,7 @@ not a new ask ([carry-forward](unattended.md#carry-forward)).
**The record.** `gstack-approvals import --autoplan <run dir> --by "<owner>"` (with
`--stack <b1,b2,...>` for a stacked PR set) writes the approval row (date, the ask, the owner's exact words, the lane that answered as relayer,
what it covers); `gstack-approvals render` puts it in the report and the PR body. Before
what it covers; a gate answered with `--words-by agent:<id>` is listed under that agent as its own decision, not the owner's words); `gstack-approvals render` puts it in the report and the PR body. Before
training the PR the controller compares that row's words and `gate_rev` with the ask it
relayed and the reply it carried back; a row that does not match is not READY (shape A).
The brief's done line names the row. A plan approval is not a READY field; the
@@ -317,8 +317,9 @@ stamp on the bare pin leaves the PR conflicting on the CHANGELOG. The lane merge
into its pin first (expect a CHANGELOG-only conflict; resolve it with `mergeChangelogs`
from `lib/changelog-check.ts`, the `bun -e` one-liner in
[ship-policy.md](ship-policy.md#run-a-merge-train), never by hand, and grep for conflict
markers before stamping), then restamps, then commits only the merge and the stamp
files. In practice slot 1 needs the same merge whenever the base moved after the pin was
markers before stamping), then restamps (the restamp stages the stamp files it wrote,
so a plain `git commit` after a `--no-commit` merge keeps them), then commits only the
merge and the stamp files. In practice slot 1 needs the same merge whenever the base moved after the pin was
taken. A stamp commit with one extra byte freezes the queue for every
lane behind it.
docs/pregate.md
+3 -1
View File
@@ -848,7 +848,9 @@ path records and uploads `tested-tree`, after its verification passes, so a
red, partial or reused run never leaves a record and a reuse is never reused.
The pull request lookup needs no `pull-requests` permission on a public
repository; a private fork of this setup would need `pull-requests: read`,
and without it every push falls back to the full run.
and without it every push falls back to the full run. Windows Free Tests now
runs nightly and by hand only (docs/support-tiers.md), never on a push, so its
`reuse` job is always skipped and every nightly run takes the strict path.
<a id="guards"></a>
## Guards and the lane-done rule
docs/recommended-harness.md
+1 -1
View File
@@ -284,7 +284,7 @@ measured later does not count anything that happened after that week.
| Gate runs green / red / cancelled | completed runs of the gate workflow (`workflow_dispatch` by default; `--gate-event any`); cost from the run's `report-verdict-a*` artifact (`collector-outcomes.json` `.verdict.costUsd`), else the `**$N** rule/judge cost` or `cost=$N` line of the PR's verdict comment | green on 95%+ of runs, cancelled near 0, cost known for every run |
| Known gate cost per merged PR | known cost / merged PRs; unknown-cost runs make it a floor | known, stable or falling week over week |
| Red-train rate | runs whose head branch starts with `--train-prefix`, cancelled excluded; red = conclusion other than success; red backstops and screens shown beside it | under 5% (each red train is a bisect plus a rebuild, about two gates) |
| Flaky events | `flake-records-a*` records of the newest gate runs (count, known rerun spend) and `flake-ledger-*` entries of free runs on the default branch; a run whose artifacts cannot be read is counted and named, never scored 0 | 0 repeat cases; under 1 flaky case per 10 gates; free retries under 1% of runs |
| Flaky events | `flake-records-a*` records of the newest gate runs (count, known rerun spend) and `flake-ledger-*` entries of free runs on the default branch; a run whose artifacts cannot be read is counted and named, never scored 0, and a main push that reused its pull request's run (no artifacts, `reuse` job green) is left out and named | 0 repeat cases; under 1 flaky case per 10 gates; free retries under 1% of runs |
| Issues opened / closed | GitHub search, created and closed in the window | closed ≥ opened; nothing severe older than a week |
| Releases cut | releases published in the window, drafts excluded | at least one tagged release a week, on schedule |
docs/review-policy.md
+3 -2
View File
@@ -565,8 +565,9 @@ ones, and the arm lost blockers. `bin/gstack-review-dispatch` closes both ends:
prompts prints `VERIFY: no brief records` (exit 3), never clean.
6. `pass --base <b> --out <dir> --pass <id> -- <plan flags>` runs one pass from a
staged manifest (`<dir>/pass-manifest.json`): audit readers marked first, then
the specialists with QA and the other audits, then the Red Team, then Step 11's
voices. Each reviewer keeps its plan entry and is rendered by the same code as
the specialists with the other audit readers, then the Red Team, then Step 11's
voices. A QA extra is refused before any reader is created: Step 9.2.1's probes
are the parent's. Each reviewer keeps its plan entry and is rendered by the same code as
`render`, once its inputs exist; the Red Team's findings file is the specialists'
`merged.jsonl` lines byte for byte. A brief leaves the bin in indexed chunks of at
most 65,536 characters (`chunk`), so no tool output line carries a whole brief and a
docs/ship-policy.md
+59 -17
View File
@@ -92,7 +92,7 @@ hard-code:
"changelog": "CHANGELOG.md",
"history": "squash-ok",
"stamp_sensitive_gates": ["tests"],
"changelog_fragments": { "dir": "changelog.d", "required_checks": ["free-tests", "windows-free-tests", "Check VERSION is not stale vs queue", "quality", "check-freshness", "actionlint"] },
"changelog_fragments": { "dir": "changelog.d", "required_checks": ["free-tests", "windows-smoke", "Check VERSION is not stale vs queue", "quality", "check-freshness", "actionlint"] },
"min_gstack_version": "1.91.340.0"
}
```
@@ -210,16 +210,18 @@ READY PR as conflicting each time main stamps; the train merges the CHANGELOG
by section, so the lane never merges main for it. The conflict does mean GitHub
has no merge ref for the PR, so no `pull_request` CI runs on the head and
`ready` prints `ci: none at <head12>` with a `ci-hint:` line. The lane then
runs `ready … --dispatch-ci free-tests.yml,windows-free-tests.yml [--timeout
runs `ready … --dispatch-ci free-tests.yml,windows-smoke.yml [--timeout
<s>]` (under a train policy): it dispatches each named workflow on the branch
(`gh workflow run <wf> --ref <branch>`, skipped when a run at this head already
concluded), waits for the runs (`CI_DISPATCH: <wf> --ref <branch> run <id> <url>
<conclusion>`), clears every requires-remote pregate lane whose workflow is one
of them with the run URL (`PREGATE_CLEARED: <lane> <url>`; a lane of another
workflow stays uncleared and is named in the refusal), and then reads the head
as usual, which passes because the dispatched check runs attach to the commit;
the `ci-hint:` line then says those non-pull_request runs count. A red run
clears no lane and refuses (`QUEUE_NOT_READY`); a run still going at `--timeout`
concluded, and skipped with a note when the base has the workflow file but the
branch predates it, since GitHub reads the file from the branch), waits for the
runs (`CI_DISPATCH: <wf> --ref <branch> run <id> <url> <conclusion>`), clears
every requires-remote pregate lane whose workflow is one of them with the run
URL (`PREGATE_CLEARED: <lane> <url>`; a lane of another workflow stays
uncleared and is named in the refusal), and then reads the head as usual,
which passes because the dispatched check runs attach to the commit; the
`ci-hint:` line then says those non-pull_request runs count. A red run clears
no lane and refuses (`QUEUE_NOT_READY`); a run still going at `--timeout`
(default 2700 s) is `QUEUE_TRAIN_PENDING`, and rerunning the command keeps
waiting; a fork head is never dispatched here (`QUEUE_TRAIN_UNTRUSTED`). By hand the same four steps are `gh workflow
run <wf> --ref <branch>`, wait, `gstack-pregate clear --remote <lane> --run-url
@@ -399,6 +401,28 @@ release paths (version files, CHANGELOG, fragments, `release_outputs`,
train head and branch are S_k, and the one full gate runs on S_k, the exact
tree the last merge publishes.
**One land lock.** `train land` takes the same repository land lock as
[`land`](#land), `refs/notes/gstack-land`, as `train <nonce> slot <i> (#<pr>)`,
before its first read of main, so a standalone land and a train slot never
both publish onto one main: whichever starts second refuses with
`QUEUE_LAND_LOCKED`, naming the holder. Once a slot has merged, the train keeps
the lock in phase `between`, which only the same train's next slot takes over,
until its last landable slot lands; a refusal before a later slot's push (a
failed `gh` or `git fetch` read, say) keeps it too, so no standalone land slips
in while the train waits on a rerun. Before any slot merged, a refusal before
the push releases it. After a push, a failure keeps the lock in its phase and a
rerun of the same slot takes it back. A stop that freezes the queue
leaves the lock in phase `freeze`, which `--break-lock` never breaks: only the
repair train the freeze admits takes it over, and it is released when that
train clears the freeze. `--break-lock` prints the holder (`LOCK_HOLDER:`),
keeps land's staleness, mid-push and drift rules, refuses a `between` lock
younger than 10 minutes, and refuses a live train run's lock unless
`--break-train-lock` is also given. `train watch --land` goes through `train
land`, so it stops with `NEXT:` on `QUEUE_LAND_LOCKED` like any refusal. A train
`train land` refuses (one built without stamped prefixes or a gate) still merges
through `verify --train` and `gh pr merge`, which neither take nor check this
lock.
Slots merge with `train land --train <nonce> --pr <n>`, never a slot call:
1. The record must be the gated head's own chain. Walking first parents from
@@ -849,8 +873,8 @@ git fetch origin main
git merge --no-ff --no-commit origin/main # expect a CHANGELOG.md conflict only
bun --no-env-file --config=/dev/null -e "import { mergeChangelogsWithNotes } from './lib/changelog-check'; import { spawnSync } from 'node:child_process'; import { writeFileSync } from 'node:fs'; const s = (n) => spawnSync('git', ['show', ':' + n + ':CHANGELOG.md'], { encoding: 'utf8', maxBuffer: 1 << 26, timeout: 30000 }).stdout; const m = mergeChangelogsWithNotes(s(2), s(3), s(1)); for (const n of m.notes) console.error('CHANGELOG: carried as a Gate integrity correction: ' + n); if (!m.text) { console.error('CHANGELOG needs a hand merge'); process.exit(1); } writeFileSync('CHANGELOG.md', m.text);"
git add CHANGELOG.md && git commit --no-edit
gstack-restamp --next --base main --pr 42 --expect-base <main sha> --expect-order <order> --allow-repo-commands
git add CHANGELOG.md VERSION package.json <release outputs> && git commit -m "v<X> stamp" && git push
gstack-restamp --next --base main --pr 42 --expect-base <main sha> --expect-order <order> --allow-repo-commands # stages what it writes (STAGED:)
git commit -m "v<X> stamp" && git push
```
A conflict in any file other than CHANGELOG.md means the merge isn't the train's prefix. Stop and report it. When the helper says the CHANGELOG needs a hand merge, take main's CHANGELOG unchanged and put your own entry above it. `verify` accepts the resulting head because it contains main and its tree equals T<i> modulo stamps, with its CHANGELOG judged as main's plus lines your pin added, however the conflict was resolved; the `head:` line says "T<i> with main merged in".
@@ -863,8 +887,12 @@ introduced). A line the base added since the pin's fork point is the base's,
whether the base merge is committed or still staged (`stage stamp_paths` says
"N line(s) the base added since the fork point are the base's"; train 24,
2026-10-27, moved main's `DOCS_CAPS_KEY_VERSION` in both slots before this
rule). A file the restamp lists that the slot call did not name is not a stamp:
leave it out and report it. The tree receipt refuses such a move after the fact
rule). Restamp stages what it writes and prints `STAGED:`; commit what restamp
staged. A file the slot call does not name is not a stamp: `git restore --staged
--worktree` it and report it. A stamp file holding unstaged edits of your own refuses before
anything is written (`RESTAMP_UNSTAGED_EDITS`), and a staging failure leaves the
files written for the noop rerun to stage (`RESTAMP_NOT_STAGED`), so a bare
commit after a `--no-commit` base merge can no longer drop the stamp (#3527). The tree receipt refuses such a move after the fact
only while the moved version is older than the base's at the build, so the
restamp, not the receipt, is what keeps the base's key in place.
@@ -935,7 +963,18 @@ saw, itself the prefix before modulo stamps) plus the PR's own entry under the
same rule. One CHANGELOG rule applies before and after, so a head the pre-merge
verify accepted confirms after its squash merge with the same CHANGELOG, and a
stacked child whose one `[Unreleased]` became two released sections is not a
mismatch (train 24, 2026-10-27, froze on exactly that). Anything else is
mismatch (train 24, 2026-10-27, froze on exactly that). Before
`MERGE_CONDITION`, the head must also carry its own stamp: its `VERSION` above
main's, and its CHANGELOG headed with that version directly above main's top
release, with no `[Unreleased]` section main does not have; otherwise
`QUEUE_TRAIN_UNSTAMPED` names the rule and both versions (train 33 merged #3527
under #3526's version without it). verify prints `STAMP_VERSION:` from the head
itself. A post-merge confirmation needs GitHub to report the PR MERGED: an open
PR whose stamp-only diff already matches main (a release-only PR, #3530) takes
the pre-merge path, and any other open or unreadable PR refuses
`QUEUE_PROOF_MISSING` without freezing, as does a verify that could not read
the PR's current head (`--no-fetch` or a failed fetch) instead of judging the
unstamped pin. Anything else is
`QUEUE_TRAIN_TREE_MISMATCH` (or `QUEUE_TRAIN_MOVED`). When nothing of the train
has merged, that is a plain refusal, and a new train is built on current main. When
earlier slots already merged, main holds a prefix no gate covered on its own,
@@ -1627,9 +1666,12 @@ command:
2. **Lock.** Take the repository land lock, `refs/notes/gstack-land` (its own
ref, never the trains notes ref), by exact lease. It records the PR, holder,
phase and journal. A held lock is `QUEUE_LAND_LOCKED` naming the holder;
`--break-lock` takes over a crashed holder's lock and is refused while that
holder entered a push or a merge less than 10 minutes ago, and for a drift
lock. A refusal before the push releases the lock.
`--break-lock` prints the holder (`LOCK_HOLDER:`), takes over a crashed
holder's lock and is refused while that holder entered a push or a merge
less than 10 minutes ago, and for a drift lock. `train land` shares this
lock ([one land lock](#train-land)); a train slot's lock needs
`--break-train-lock` as well while its train still lands, and a `freeze`
lock is never broken. A refusal before the push releases the lock.
3. **Merge and stamp.** In a scratch worktree, merge the base into the pin.
Conflicts in release fields only (the version file and `release_outputs`
take the base's side, the CHANGELOG merges by section, a manifest or npm
docs/skills.md
+9 -5
View File
@@ -962,8 +962,12 @@ blank (`PREREG_INCOMPLETE`). `arms` resolves the `eval-arms` model role against
`lib/pricing.ts` and stops on an unpriced model (`EVAL_PRICE_MISSING`); `dry-run`
runs every row through the stub model for $0 and flags malformed rows
(`EVAL_DRY_RUN_FAILED`); `pilot start|settle|release|report` runs three priced
items, records actual spend, extrapolates to the full set against the cap
(`EVAL_BUDGET_EXHAUSTED`) and prints the spend table the approval question shows.
items, records actual spend, extrapolates each priced arm to the full set against
the cap (`EVAL_BUDGET_EXHAUSTED`) and prints the spend table the approval question
shows. An arm on a subscription with no extra charge is declared with
`pilot start --seat-metered <arm,...>`: it reserves $0, settles as `unknown`, and
stays out of the projection until one of its attempts settles a real charge.
A run stops at the first quota or rate-limit error on such an arm.
`results --check <prereg> <results.md>` refuses a finished results file unless one
table holds a filled row for every arm, each `<model>#no-gstack` row beside its
variants (`EVAL_REPORT_NO_GSTACK_ROW`); no number leaves the run before it passes.
@@ -1002,8 +1006,8 @@ VM. `/ubicloud` runs it there through one entry point, `bin/gstack-ubi`
(`scripts/ubicloud/ubi-runner.sh`), with the discipline a shared quota needs:
every VM is named `ubirun-<owner>-<epoch>-<suffix>-t<hours>`, so its owner
(`UBI_OWNER`, required to create one) and TTL (`UBI_TTL_HOURS`, default 3) are
visible to everyone; every `up` first reaps VMs whose TTL ended, whichever owner
made them, and its own owner's VMs with no TTL once they are 4 hours old; every run
visible to everyone; every `up` first reaps its own owner's VMs whose TTL ended,
and those with no TTL once they are 4 hours old, never another owner's; every run
ends with `UBICLOUD: left=0` or fails; the
quota preflight refuses before the create request when the project's running
VMs plus the new size exceed the known ceiling (`UBI_QUOTA_VCPUS`, or the
@@ -1026,7 +1030,7 @@ Claude: UBICLOUD_API_KEY: set; UBI_OWNER=lane7
gstack-ubi verify (.gstack/ubicloud.json: bootstrap.sh, "bash scripts/verify.sh", standard-16)
ubi-runner: reap: no expired VMs
ubi-runner: creating eu-central-h1/ubirun-lane7-1761500000-9f3a2c10-t3 (standard-16)
ubi-runner: ubirun-lane7-1761500000-9f3a2c10-t3 carries a 3h TTL (any later run of this script reaps it); if this process dies first: gstack-ubi down ubirun-lane7-1761500000-9f3a2c10-t3
ubi-runner: ubirun-lane7-1761500000-9f3a2c10-t3 carries a 3h TTL (this owner's next up or reap destroys it once it expires); if this process dies first: gstack-ubi down ubirun-lane7-1761500000-9f3a2c10-t3
ubi-runner: heartbeat: command 300s elapsed, last output 2s ago
ubi-runner: command exited 0 after 1140s
ubi-runner: destroyed eu-central-h1/ubirun-lane7-1761500000-9f3a2c10-t3
docs/troubleshooting.md
+111 -8
View File
@@ -314,6 +314,20 @@ network.
**Expected result.** `OUTSIDE_STATUS: completed`.
<a id="outside-review-truncated"></a>
### `... outside review unavailable: the reviewer hit its output token limit and the response is cut off (...)`
**Meaning.** The api runner's provider stopped at the output token limit
(Anthropic `stop_reason: max_tokens`, OpenAI `finish_reason: length`), so the
review ends mid-sentence and its findings may be missing.
**Fix.** Re-run with a larger `--max-tokens` (`gstack-outside-voice run` or
`gstack-autoplan run`), or shorten the prompt. `gstack-autoplan run` sends a
default of 16384 to both providers, and OpenAI counts reasoning tokens toward
it; `gstack-outside-voice run` sends that default to Anthropic only.
**Expected result.** `OUTSIDE_STATUS: completed`.
<a id="outside-review-missing-markers"></a>
### `... outside review unavailable: the response lacks the markers this gate requires (...)`
@@ -866,7 +880,11 @@ pick a new `--output`), then copy the recovery file to where you want it.
**Meaning.** On Codex, Factory, OpenCode, Cursor, Copilot and Kiro every bash
block finds gstack on its own: an exported `GSTACK_ROOT` (with `bin/` and
`lib/`), then a repo-local install, then the host's global install
(`$CODEX_HOME/skills/gstack` on Codex). None existed.
(`$CODEX_HOME/skills/gstack` on Codex). None existed. A skill installed for one
folder (Hermes, or the Codex install `./setup` writes) is the exception for its
first block: that block runs the folder's own `bin/gstack-skill-start` and
ignores an exported `GSTACK_ROOT`. When the folder is missing, the block prints
this line and its start command then fails.
**Fix.**
@@ -3263,6 +3281,42 @@ result.
**Fix.** Run `gstack-autoplan status --out <dir>` and submit for the attempt
the last `next` printed (`ATTEMPT: ... phase=<p> voice=<v>`).
<a id="ceo-only-conflict"></a>
### `CEO_ONLY_CONFLICT`
**Meaning.** `gstack-autoplan next --ceo-only` was combined with `--ui`,
`--developer-tool` or `--agent-primary` (on the command line or in the plan's
`gstack-autoplan-scope` marker), each of which adds a phase a CEO-only run
never opens; or the marker names `--ceo-only` (the mode is a command-line
choice for one run, so a plan can't pin it); or the plan is `<out>/ceo-review.md`
(directly, through a symlink or a hard link), the report the run would write
over it. The finish checks that last case again before it writes the report.
**Fix.** Drop the conflicting flag (or the marker token), move the plan out of
the report path, or run the full review without `--ceo-only`.
<a id="ceo-only-after-init"></a>
### `CEO_ONLY_AFTER_INIT`
**Meaning.** `next --ceo-only` named a run directory that already holds a full
review run. A run keeps the mode it started with.
**Fix.** Start the CEO-only run in a new `--out` directory, passing
`--ceo-only` on its first `next`.
<a id="ceo-only-no-gate"></a>
### `CEO_ONLY_NO_GATE`
**Meaning.** `answer`, `recommend` or `revise` was run on a CEO-only run, or
`gstack-approvals import` was pointed at one. A CEO-only
run writes no gate, ends `status=reviewed` and is never approved, so there is
nothing to answer, change or import. (`budget` and `close --deferred` still
work: they are how a budget-stopped CEO-only run recovers.)
**Fix.** Read `<run dir>/ceo-review.md`. For a gate and an approval, start a
full review in a new directory: `gstack-autoplan next --out <new dir> --plan
<file>` without `--ceo-only`.
<a id="attempt-mismatch"></a>
### `ATTEMPT_MISMATCH`
@@ -4590,7 +4644,10 @@ train still building, running or merging, `train watch` on an unknown record,
unreadable run), `verify --pr <n> --train <nonce>` when main's tree mismatches
and whether a slot merged on GitHub cannot be read (so whether the queue must
freeze is unknown; it never answers "nothing merged" from an unreadable
state), `train land` when the base or the train branch cannot be fetched or
state), `verify --pr <n> --train <nonce>` when GitHub does not report the PR
MERGED and it is not a release-only PR whose stamp-only diff already matches
main, or when it could not read the PR's current head (`--no-fetch` or a failed
fetch), without freezing the queue, `train land` when the base or the train branch cannot be fetched or
the record's T0 and S_1..S_k are not the gated head's first-parent chain
(each S_i's second parent slot i's pin), `train land` when main's tree is the
slot's S_i but GitHub does not report the PR as MERGED (a record whose slot
@@ -4696,14 +4753,60 @@ the name already landed. Then rerun.
<a id="queue-land-locked"></a>
### `QUEUE_LAND_LOCKED`
**Meaning.** Another `gstack-ship-queue land` holds the repository land lock
(`refs/notes/gstack-land`), or took it from under this run, so this one would
race it for the next version. The line names the holder, its PR and phase.
**Meaning.** Another `gstack-ship-queue land`, or a `train land` slot, holds the
repository land lock (`refs/notes/gstack-land`), or took it from under this run,
so this one would race it onto main. The line names the holder: `#<pr>` for a
land, `train <nonce> slot <i> (#<pr>)` for a train, with its holder and phase. A
train keeps the lock in phase `between` from one merged slot to the next, so no
standalone land slips in between slots.
**Fix.** Wait for the holder to finish. If it crashed, rerun with
`--break-lock`; that is refused while the holder entered a push or a merge
less than 10 minutes ago. A lock in phase `drift` is released by landing its
repair PR ([QUEUE_LAND_DRIFT](#queue-land-drift)).
`--break-lock`, which prints the holder (`LOCK_HOLDER:`). That is refused while
the holder entered a push or a merge less than 10 minutes ago, while a train is
between slots for less than 10 minutes, and for a live train run unless you also
pass `--break-train-lock`. A lock in phase `drift` is released by landing its
repair PR ([QUEUE_LAND_DRIFT](#queue-land-drift)); a lock in phase `freeze` is
released by the verify that clears the freeze
([QUEUE_TRAIN_FROZEN](#queue-train-frozen)). If that release push failed, the
next `land` or `train land` takes the lock over once the freeze record it fetches
shows the freeze cleared after the lock was written; `--break-lock` never does.
<a id="queue-train-unstamped"></a>
### `QUEUE_TRAIN_UNSTAMPED`
**Meaning.** `verify --pr <n> --train <nonce>` found the slot head it would let
merge without its own stamp: the head's `VERSION` is not above main's, or its
CHANGELOG's top section is not headed with that version directly above main's
top release (or keeps an `[Unreleased]` section main does not have). The line
names the rule, the head's version and main's. Train 33 merged #3527 this way
(a no-commit base merge, a restamp whose files were never staged, a bare
commit) and shipped it under #3526's version. Nothing was merged.
**Fix.** On the lane, run `gstack-restamp --next` on the head, commit what
restamp staged, push, and rerun the verify.
<a id="restamp-not-staged"></a>
### `RESTAMP_NOT_STAGED`
**Meaning.** `gstack-restamp` wrote the stamp but `git add` failed, so a bare
`git commit` would leave the stamp out. The files stay written, and
`.gstack/tmp/restamp-staging.json` records what is still to stage.
**Fix.** Clear what blocked `git add` (a stale `.git/index.lock`, for
example) and rerun `gstack-restamp`; the rerun is a noop that stages the files
it wrote. When any of them changed since restamp wrote it (you committed by
hand, switched branches or merged the base), the marker is stale: the noop
rerun drops it and stages nothing, and a stamp still owed is restamped as usual.
<a id="restamp-unstaged-edits"></a>
### `RESTAMP_UNSTAGED_EDITS`
**Meaning.** A file `gstack-restamp` would stamp holds unstaged edits of yours.
Restamp stages what it writes, so those edits would ride along in the stamp
commit. Nothing was written.
**Fix.** Stage (`git add`) or stash your own edits to the named files, then
rerun `gstack-restamp`.
<a id="queue-land-conflict"></a>
### `QUEUE_LAND_CONFLICT`
docs/unattended.md
+102 -11
View File
@@ -65,10 +65,12 @@ A run that writes a record ends with one line the parent greps:
GSTACK_RESULT: skill=<name> status=<status> run=<dir>
```
`status` is one of `complete`, `gate_pending`, `incomplete`, `interrupted` or
`refused`; `run` is where the run's record is. `/autoplan`
`status` is one of `complete`, `reviewed`, `gate_pending`, `incomplete`,
`interrupted` or `refused`; `run` is where the run's record is. `/autoplan`
always ends `gate_pending` when it reaches the final gate, because the gate is
written and never approved. Pure queries (`gstack-artifact schema`,
written and never approved. `/autoplan --ceo-only` ends `reviewed`: it runs the
CEO phase alone, asks nothing, writes no gate and approves nothing (see
[CEO-only runs](#ceo-only-runs)). Pure queries (`gstack-artifact schema`,
`gstack-gate render`) print no result line.
Not every workflow writes this line or a run directory today:
@@ -257,7 +259,7 @@ tree no review saw, so /ship runs again;
| File | Schema | Contents |
|------|--------|----------|
| `run.json` | `run` v1 | The manifest: `run` id, `status`, `required_phases`, each phase's `native` and `outside` outcome (`status`, `model`, `provider`), input `snapshot` hashes, `counts`, every artifact's `path` (relative to the run directory) and `sha256`, `session_kind`, `gate_rev`, `deviations`, `consumed_by`. |
| `decisions.jsonl` | `decisions` v1 | One row per decision: `id` (`<run>-<label>` at gate revision 1, `<run>-r<rev>-<label>` after a reopen, since labels restart at each gate), `title`, `options`, `recommended`, `kind` (`auto`, `approval`, `user_challenge`), `status` (`pending`, `approved`, `overridden`, `rejected`), `gate_rev`, and once answered `chosen`, `answered_at`, `reply`. |
| `decisions.jsonl` | `decisions` v1 | One row per decision: `id` (`<run>-<label>` at gate revision 1, `<run>-r<rev>-<label>` after a reopen, since labels restart at each gate), `title`, `options`, `recommended`, `kind` (`auto`, `approval`, `user_challenge`), `status` (`pending`, `approved`, `overridden`, `rejected`, or `auto_kept` in a [CEO-only run](#ceo-only-runs)), `gate_rev`, and once answered `chosen`, `answered_at`, `reply`. |
| `findings.jsonl` | `findings` v1 | One row per reviewer finding: `id`, `phase`, `voice` (`native`/`outside`), `model`, `severity`, `title`, `disposition`, `plan_items`, `native_counterpart` (string or null), `source`. Rows share the review-log row shape, so `gstack-review-log --findings` derives review status from this file. |
| `tasks.jsonl` | `tasks` v2 | Implementation tasks with `id`, `title`, `status`, `depends_on`, `findings` (ids in `findings.jsonl`), `blocked_by`, `acceptance`, optional `pr`/`item`/`tier`. |
| `timing.json` | `timing` v1 | Per-phase `started_at`, `ended_at`, `wall_s`, `outside_s`, `native_s`; `started_at` and `total_wall_s` for the run; the Phase 0 `estimate`. |
@@ -374,7 +376,14 @@ the plan.
refuses to open without one (`CONSENSUS_MISSING`).
3. **Loop `next` until `status=gate_pending`.** Eng always runs last. The gate
(`gate.json`, `gate_rev`) lists user challenges as approval items,
disagreements as auto items and `p1` plan approval; nothing is approved.
disagreements as auto items, each open Critical or High finding as an
auto item (a twin pair is one item; `a` applies the reviewer's fix to the
plan and is recommended, `b` carries it into the build as a blocking task)
and `p1` plan approval; nothing is approved. An unanswered auto item whose
recommendation changes the plan keeps the plan's direction, so only `rec`,
`p1b` (approve with the recommended changes) or an explicit `<id>a` applies
a fix; a finding carried with `b` stays
open, so a rerun that raises it again asks again.
4. **Answer the gate** with the reply grammar and the revision you read:
```bash
$AP answer --out <dir> --gate-rev 1 --reply "rec" --by human # every recommendation, plan changes included (APPLY: ...; amend_pending)
@@ -385,12 +394,18 @@ the plan.
means an agent relayed them, and then `--owner-reply` (or `-` for stdin,
`--owner-reply-file <path>`) stores the owner's words verbatim
(`OWNER_REPLY_REQUIRED`); owner words that parse to different choices are
refused (`GATE_OWNER_REPLY_MISMATCH`). An unattended run refuses an answer
refused (`GATE_OWNER_REPLY_MISMATCH`). The words default to the owner's: an
agent that answers in its own words (a Captain deciding on its own
authority) must pass `--words-by agent:<id>` with its own `--by` id. They
are then recorded as `answer_words` with `words_by`, never as
`owner_reply`; a later gate never carries them (`CARRY: no`), and
`gstack-approvals import` lists that row under the agent, not the owner.
An unattended run refuses an answer
without `--by` (`ANSWER_BY_REQUIRED`, exit 3) and records the value as
`answered_by` on every decision row and in the review record's gate line,
so a gate an agent closed is distinguishable from one the owner closed.
`gstack-autoplan recommend` changes one recommendation before relay; it
bumps the gate's `rec_rev`, and a later `rec` needs the `--rec-hash` from
bumps the gate's `rec_rev`, and a later `rec` or `p1b` needs the `--rec-hash` from
the render the owner saw (`GATE_REC_STALE`). Every `GATE:` line is followed
by `GATE_SPEND:` and `GATE_REC_COST:` (also `gate_cost` in `--json`; see
`autoplan/DRIVER.md`).
@@ -440,7 +455,19 @@ Concurrency and recovery: every state write is a locked read-modify-write
(`test/autoplan-run-recovery.test.ts`, `test/autoplan-run-concurrency.test.ts`,
`test/autoplan-answer-invalidation.test.ts`). `--spend-cap` admits each attempt
through `spend.json` (`SPEND_CAP_EXCEEDED` refuses, exit 3); host-subagent
spend is `unknown` and counted in `worst_case`. `--deadline <min>` makes the
spend is `unknown` and counted in `worst_case`. Under a cap, in any session
kind (or unattended on Capy), each host-subagent attempt, and each api attempt
whose model has no price, reserves the default per voice unless
`next --estimate-usd <n>` sets the reservation, which the run keeps for later
phases (`run` reserves an api attempt's priced worst case, prompt bytes / 3
input tokens plus the output limit, and settles it at its token usage; every
native call that went out and bound no result, whether it failed, was cancelled,
had its result refused, or was in flight when the process died (the next `run`
finds it), settles at its usage, or at its full reservation when the usage is
unknown, while one refused before anything was sent, such as with no API key,
keeps its attempt open for the rerun, and a failed outside call leaves the phase
native-only and stays counted at its reservation); `submit
--usage-usd <n>` settles an attempt at its real cost. `--deadline <min>` makes the
`run` subcommand cancel the children it started and marks the run
`interrupted`.
@@ -520,9 +547,8 @@ A gate revision after the first (`gate_rev` 2 and later, written after a
it when nothing new needs their decision. `gstack-autoplan` prints
`CARRIED: gate_rev=<n> <- gate_rev=<m> words="<their words>" by=<relayer> reason=<why>`,
then the usual `APPROVED:` lines, and journals `gate_carried`. The carry needs
all of: the earlier gate approved (`p1a` or `p1b`) with `rec` or "do it"
(not item tokens), no item there left pending, and the owner's words
recorded; that gate was not itself carried; every accepted change incorporated
all of: the earlier gate approved (`p1a` or `p1b`), no item there left
pending, and the owner's words recorded; that gate was not itself carried; every accepted change incorporated
or settled; the plan file's bytes equal those recorded at the gate or at the
`revise` that took the changes (any later edit refuses, an ELI5-only one
included); no urgent or
@@ -531,6 +557,13 @@ Anything else prints `CARRY: no (<why>)` (journal `carry_refused`) and the gate
goes to the owner as before. `--no-carry-forward` on the first `next` turns it
off for the run.
An earlier answer given item by item (`<id><option>` tokens, not `rec` or "do
it") carries only what the owner already answered: every item on the new gate
must be carried by origin (the same proposal, so the same choice), and only an
unreviewed post-cap delta (below) may be new. The carry replays the owner's own
`p1` choice and records their tokens verbatim as its basis; any new or changed
item, "do it" cases included, goes to the owner.
- **"do it".** When the owner's words were "do it" (their words, not a
`captain:` relay of them), a new `auto` item that changes no scope, cost or
direction, whose recommended option changes no plan text, and is no round
@@ -560,6 +593,64 @@ and `gstack-owner-brief render --captain` opens with "Nothing pending.
Carried forward from gate_rev <m>", followed by any items taken under "do it"
without being shown to the owner.
## CEO-only runs
`/autoplan --ceo-only` runs the CEO review by itself and asks nothing. It runs
in the session that invokes the skill and needs Claude Code: other hosts inline
autoplan's sections and do not carry this one yet. A parent agent on any host
gets the same run from the driver with `gstack-autoplan next --ceo-only` (see
below). It is for a harness or a crew that wants the strategy pass without the
full gauntlet, such as an eval comparing gstack's CEO review with a plain
prompt.
- **What runs:** Phase 0, the outside-reviewer preflight and the CEO phase.
Design, DX, Eng and the final gate do not run.
- **How it decides:** every question takes its recommendation and is logged in
the Decision Audit Trail. The review mode comes from plan-ceo-review's 0E
rule (file count, then greenfield, added capability or fix), not autoplan's
fixed SELECTIVE EXPANSION, and the mode line says `CEO-only run`. Consent,
destructive and irreversible options take the conservative choice.
- **What it never does:** apply a user challenge (the plan's direction stays;
each challenge is listed), chain to another skill, promote the CEO plan, or
approve anything.
- **One model:** add `--no-outside` to skip the outside voice, so every
reviewer runs on the host model.
- **Where it writes:** the run directory (`.gstack/tmp/autoplan/` in the
repository), the reviewed plan, and CEO plans and task files under
`GSTACK_STATE_ROOT`. Point that variable inside a trial folder to keep every
write there.
- **What it leaves:** the reviewed plan (its original is kept at the restore
point), `<run dir>/ceo-review.md` (user challenges, scope added,
conservative choices, findings and the audit trail), and the last line
`GSTACK_RESULT: skill=autoplan status=reviewed run=<dir>`.
`test/skill-e2e-autoplan-ceo-only.test.ts` runs it on a plan that makes the
interactive CEO review ask a question and fails if this mode asks anything.
**From the driver.** `gstack-autoplan next --out <dir> --plan <file>
--ceo-only` opens only the CEO phase; the parent dispatches and submits its
voices as usual. When CEO closes, the run prints `REVIEWED:` and ends
`GSTACK_RESULT: skill=autoplan status=reviewed run=<dir>`, with
`<dir>/ceo-review.md` in the same sections, hashed in `run.json`. Each item a
gate would have asked becomes a `decisions.jsonl` row of kind `auto` and status
`auto_kept` (nobody answered: the plan's direction is kept, and a round-cap item
only the owner can accept, such as a Critical left open, stays undecided with
no chosen option), and tasks are blocked on no decision. An open Critical or
High finding, which a full gate asks about, gets no row: nobody decides it, and
it stays in the report's Findings and in `tasks.jsonl` for the owner. The driver never edits the plan, writes no `gate.json` and never
approves, so `gstack-artifact validate` accepts the run as it stands.
- The flag goes on the first `next` only. It refuses beside `--ui`,
`--developer-tool` or `--agent-primary` on the command line or in the plan's
scope marker, when the marker names it, and on a plan at the report path
(`CEO_ONLY_CONFLICT`); a directory that holds a full run refuses it
(`CEO_ONLY_AFTER_INIT`).
- `answer`, `recommend` and `revise` refuse a CEO-only run, and so does
`gstack-approvals import` (`CEO_ONLY_NO_GATE`): there is no gate to answer and
nothing was approved. A run stopped by `--spend-cap` recovers as a full run
does: `budget` then `resume`, or `close --phase ceo --deferred`, which ends
it `reviewed`.
## Owner brief
`gstack-owner-brief render --out <dir>` is what the owner sees at the final
eval-plan/SKILL.md
+12 -4
View File
@@ -138,7 +138,7 @@ Write (or open) the preregistration file. Each section is required and must be f
| Arms | one line per arm: the arm identity first (`<model>` or `<model>#<loadout>`), settings, per-item cost from Step 1b |
| Comparison | `stack: loaded` when any arm loads gstack or another skill stack; otherwise `stack: none` and a `why:` line |
| Decoys | inputs the scorer must reject or score low; how many, how they were made, what a decoy pass proves |
| Stop rules | the spend cap, a bar missed on the pilot, a scorer below its bar, `budget_exhausted` |
| Stop rules | the spend cap, a bar missed on the pilot, a scorer below its bar, `budget_exhausted`, the first quota or rate-limit error on a seat-metered arm |
| Held-out exposure | every held-out set the plan touches: prior exposure (which runs, which models), the reservation on it |
| No-gstack arm | required with `stack: loaded`: `arms:`, `inputs:`, `prompt:`, `panel:`, `report:` (below) |
@@ -254,7 +254,14 @@ anything else (`EVAL_DRY_RUN_FAILED`).
`--out`): admission is `spent + reserved + unknown + estimate <= cap`, under a file lock
shared with any concurrent worker. Run each `PILOT_ATTEMPT:` through the project's own
eval command, then settle it with the actual cost from the provider's usage field, or
`unknown` when the call was seat-metered or CLI-metered:
`unknown` when the call was seat-metered or CLI-metered.
An arm on a subscription with no extra charge (say `claude -p` on a seat) is declared at
start with `--seat-metered <arm,...>`, each also in `--arms`. Its attempts hold none of the
cap, settle as `unknown` and get a `PILOT_SEAT_METERED:` line instead of a projection; an
arm that settles a real charge is projected as priced. The first quota or rate-limit error
on such an arm ends the run: no retry, no new attempt; release the unstarted ones. Settle
each attempt, then report:
```bash
~/.claude/skills/gstack/bin/gstack-eval-plan pilot settle --out <outdir> --attempt <attempt-id> --usd <usd|unknown>
@@ -263,8 +270,9 @@ eval command, then settle it with the actual cost from the provider's usage fiel
`pilot report` prints the spend table (`spent`, `reserved`, `unknown charges`, the worst case
with one in-flight call per worker, the worst-case overrun against the cap) and
`PILOT_PROJECTION:` — mean cost per item, its sample deviation, the projection to
`--total` items across the arms, and a 95% band. Regenerate the table at every later gate
one `PILOT_ARM:` line per money arm (its own mean, deviation, projection and band), then
`PILOT_PROJECTION:` — the arms summed: cost per item, deviation, the projection to
`--total` items, and a 95% band. Regenerate the table at every later gate
of the run. Stop and report, without asking for approval, when the pilot missed a bar from
the preregistration, when `status=budget_exhausted`, or when `verdict=exceeds`.
eval-plan/SKILL.md.tmpl
+12 -4
View File
@@ -60,7 +60,7 @@ Write (or open) the preregistration file. Each section is required and must be f
| Arms | one line per arm: the arm identity first (`<model>` or `<model>#<loadout>`), settings, per-item cost from Step 1b |
| Comparison | `stack: loaded` when any arm loads gstack or another skill stack; otherwise `stack: none` and a `why:` line |
| Decoys | inputs the scorer must reject or score low; how many, how they were made, what a decoy pass proves |
| Stop rules | the spend cap, a bar missed on the pilot, a scorer below its bar, `budget_exhausted` |
| Stop rules | the spend cap, a bar missed on the pilot, a scorer below its bar, `budget_exhausted`, the first quota or rate-limit error on a seat-metered arm |
| Held-out exposure | every held-out set the plan touches: prior exposure (which runs, which models), the reservation on it |
| No-gstack arm | required with `stack: loaded`: `arms:`, `inputs:`, `prompt:`, `panel:`, `report:` (below) |
@@ -176,7 +176,14 @@ anything else (`EVAL_DRY_RUN_FAILED`).
`--out`): admission is `spent + reserved + unknown + estimate <= cap`, under a file lock
shared with any concurrent worker. Run each `PILOT_ATTEMPT:` through the project's own
eval command, then settle it with the actual cost from the provider's usage field, or
`unknown` when the call was seat-metered or CLI-metered:
`unknown` when the call was seat-metered or CLI-metered.
An arm on a subscription with no extra charge (say `claude -p` on a seat) is declared at
start with `--seat-metered <arm,...>`, each also in `--arms`. Its attempts hold none of the
cap, settle as `unknown` and get a `PILOT_SEAT_METERED:` line instead of a projection; an
arm that settles a real charge is projected as priced. The first quota or rate-limit error
on such an arm ends the run: no retry, no new attempt; release the unstarted ones. Settle
each attempt, then report:
```bash
~/.claude/skills/gstack/bin/gstack-eval-plan pilot settle --out <outdir> --attempt <attempt-id> --usd <usd|unknown>
@@ -185,8 +192,9 @@ eval command, then settle it with the actual cost from the provider's usage fiel
`pilot report` prints the spend table (`spent`, `reserved`, `unknown charges`, the worst case
with one in-flight call per worker, the worst-case overrun against the cap) and
`PILOT_PROJECTION:` — mean cost per item, its sample deviation, the projection to
`--total` items across the arms, and a 95% band. Regenerate the table at every later gate
one `PILOT_ARM:` line per money arm (its own mean, deviation, projection and band), then
`PILOT_PROJECTION:` — the arms summed: cost per item, deviation, the projection to
`--total` items, and a 95% band. Regenerate the table at every later gate
of the run. Stop and report, without asking for approval, when the pilot missed a bar from
the preregistration, when `status=budget_exhausted`, or when `verdict=exceeds`.
hosts/claude/hooks/timeline-stop-hook.ts
+35 -10
View File
@@ -59,6 +59,8 @@ import { readVersionedSlugCache } from '../../../lib/bin-context';
import { logHookError as sharedLogHookError } from './hook-log';
const DEADLINE_MS = 2000;
/** A host may keep stdin open after its payload (#3628); never wait for EOF past this. */
const STDIN_DEADLINE_MS = 500;
const REPAIR_SLUG_TIMEOUT_MS = 30_000;
const LOCK_TTL_MS = 60_000;
const MISS_TTL_MS = 5 * 60_000;
@@ -161,9 +163,35 @@ function slugOverride(): string | undefined {
return slug && slug !== '.' && slug !== '..' ? slug : undefined;
}
function readStopCwd(): string {
/** The Stop payload: returns at EOF, at the first complete JSON object, or at the deadline. */
function readStdinBounded(ms: number): Promise<string> {
return new Promise((resolve) => {
let buf = '';
const done = () => {
clearTimeout(timer);
process.stdin.removeAllListeners('data');
process.stdin.destroy();
resolve(buf);
};
const timer = setTimeout(done, ms);
process.stdin.setEncoding('utf8');
process.stdin.on('data', (chunk: string) => {
buf += chunk;
try {
JSON.parse(buf);
done();
} catch {
// incomplete payload: keep reading until EOF or the deadline
}
});
process.stdin.on('end', done);
process.stdin.on('error', done);
});
}
async function readStopCwd(): Promise<string> {
try {
const stdin = fs.readFileSync(0, 'utf8');
const stdin = await readStdinBounded(STDIN_DEADLINE_MS);
if (stdin.trim()) {
const payload = JSON.parse(stdin) as { cwd?: string };
if (payload.cwd && fs.existsSync(payload.cwd)) return payload.cwd;
@@ -209,12 +237,12 @@ function backgroundRepair(cwd: string): void {
});
}
function main(): void {
async function main(): Promise<void> {
if (process.argv[2] === '--repair' && process.argv[3]) {
backgroundRepair(process.argv[3]);
return;
}
const cwd = readStopCwd();
const cwd = await readStopCwd();
const slug = slugOverride() ?? readVersionedSlugCache(resolveStateRoot(), cwd);
if (!slug) {
spawnRepair(cwd);
@@ -318,9 +346,6 @@ function repairTimeline(slug: string, deadline: number): void {
}
}
try {
main();
} catch (err) {
logHookError(`unexpected: ${err instanceof Error ? (err.stack ?? err.message) : String(err)}`);
}
process.exit(0);
main()
.catch((err) => logHookError(`unexpected: ${err instanceof Error ? (err.stack ?? err.message) : String(err)}`))
.finally(() => process.exit(0));
lib/approval-citations.ts
+1
View File
@@ -78,6 +78,7 @@ function planGate(dir: string, run: Record<string, any>): Resolution {
const p1 = readJsonl(path.join(dir, 'decisions.jsonl')).find(d => d.label === 'P1' && d.gate_rev === rev);
const approved = run.status === 'complete' && typeof gate.plan === 'string' && !['c', 'e'].includes(gate.plan) && p1 && p1.chosen === gate.plan && p1.status !== 'pending';
if (!approved) return { state: 'rejected', why: `run ${run.run} holds this plan but its gate is not APPROVED (status ${run.status}, p1 ${p1?.chosen ?? 'unanswered'})` };
if (p1.words_by) return { state: 'rejected', why: `run ${run.run}: its gate was answered in ${p1.words_by}'s own words, not the owner's` };
return { state: 'resolved', evidence: `plan ${run.run} gate_rev=${rev} p1=${gate.plan} answered_by=${gate.answered_by ?? p1.answered_by ?? 'unrecorded'}`, text: fs.readFileSync(planFile, 'utf8') };
}
lib/approval-record.ts
+10 -4
View File
@@ -235,7 +235,7 @@ export function renderRecord(read: ReadResult, loc: StoreLocation, o: { heading?
return [...head, ...lines, '', `Source: gstack-approvals record for ${cell(loc.branch)} ${APPROVALS_FOOTER_END}`, ''].join('\n');
}
interface DecisionLike { id?: string; title?: string; status?: string; chosen?: string | null; recommended?: string | null; gate_rev?: number; answered_at?: string; answered_by?: string; reply?: string; owner_reply?: string; carried_from?: string; label?: string; chosen_text?: string;
interface DecisionLike { id?: string; title?: string; status?: string; chosen?: string | null; recommended?: string | null; gate_rev?: number; answered_at?: string; answered_by?: string; reply?: string; owner_reply?: string; answer_words?: string; words_by?: string; carried_from?: string; label?: string; chosen_text?: string;
carried_gate_from?: string; carried_gate_rev?: number; carried_answered_by?: string; carried_under?: string; carry_check?: string }
const stripRef = (t: string) => t.replace(/\s*\[[^\]]*\]\s*$/, '').trim();
@@ -254,7 +254,12 @@ export function importAutoplan(runDir: string, branch: string, by?: string): App
if (!line.trim()) continue;
try { rows.push(JSON.parse(line)); } catch { return { code: 'ARTIFACT_MALFORMED_JSONL', message: `decisions.jsonl has a line that is not JSON` }; }
}
const run = (() => { try { return JSON.parse(fs.readFileSync(path.join(runDir, 'run.json'), 'utf8')); } catch { return {}; } })();
const runFile = path.join(runDir, 'run.json');
let run: Record<string, any> = {};
if (fs.existsSync(runFile)) {
try { run = JSON.parse(fs.readFileSync(runFile, 'utf8')); } catch (e: any) { return { code: 'ARTIFACT_INVALID_JSON', message: `run.json in ${runDir} is unreadable (${e?.message ?? e}); an approval names the run it came from` }; }
}
if (run.ceo_only || run.status === 'reviewed') return { code: 'CEO_ONLY_NO_GATE', message: `${runDir} is a CEO-only run (status ${run.status}): it has no gate and was never approved` };
const plan = reviewedPlan(runDir, run);
const planTitle = firstTitle(plan.text) ?? 'the plan';
const planSha = plan.sha;
@@ -278,7 +283,8 @@ export function importAutoplan(runDir: string, branch: string, by?: string): App
const answerer = first.answered_by ?? 'unknown';
const carried = p1?.carried_gate_from ? p1 : undefined;
const relayed = carried?.carried_answered_by && carried.carried_answered_by !== 'human' ? `, relayed by ${carried.carried_answered_by}` : '';
const who = carried ? `${by ?? 'the owner'} (words from gate_rev ${carried.carried_gate_rev}${relayed}), carried forward by gstack-autoplan`
const who = first.words_by ? `${first.words_by}, its own decision (not the owner's words)`
: carried ? `${by ?? 'the owner'} (words from gate_rev ${carried.carried_gate_rev}${relayed}), carried forward by gstack-autoplan`
: answerer === 'human' ? (by ? `${by}, typed at the gate` : 'the owner, typed at the gate') : `${by ?? 'the owner'}, relayed by ${answerer}`;
const unseen = items.filter(r => r.carried_under).map(r => stripRef(r.title ?? ''));
const checked = [...new Set(items.map(r => r.carry_check).filter((c): c is string => !!c))];
@@ -286,7 +292,7 @@ export function importAutoplan(runDir: string, branch: string, by?: string): App
unseen.length ? `taken under "do it", not seen by the owner: ${unseen.join('; ')}` : '', checked.length ? `post-cap delta checked by ${checked.join(', ')}` : ''].filter(Boolean).join('; ');
const built = buildRow({
ask: `Approve the plan "${planTitle}" (autoplan run ${runId}, ${carried ? `gate_rev ${rev} carried forward from gate_rev ${carried.carried_gate_rev}, ` : ''}${items.length + (p1 ? 1 : 0)} decisions)`.slice(0, CAPS.ask),
words: first.owner_reply ?? first.reply ?? '(no words recorded)',
words: first.owner_reply ?? first.answer_words ?? first.reply ?? '(no words recorded)',
by: who.slice(0, CAPS.by),
scope: scope.slice(0, CAPS.scope),
pending: (stillPending.length ? `${stillPending.length} decision(s) unanswered: ${stillPending.map(r => stripRef(r.title ?? '')).join('; ')}` : outcome === 'pending' ? 'the plan approval itself' : 'none').slice(0, CAPS.pending),
lib/autoplan-budget.ts
+18 -10
View File
@@ -16,12 +16,12 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import { ANSWERED_BY_RE, type GateItem } from './gate-list';
import { settledIds } from './autoplan-confirm';
import type { ConsensusRecord } from './autoplan-reconcile';
import { sameProblem, type ConsensusRecord } from './autoplan-reconcile';
import { FILES, RunError, journal, now, readJournal, voiceRows, type RunState, type RunStop } from './autoplan-state';
import { readLedger, release, setCap, summarize } from './spend-ledger';
export const DEFAULT_MAX_ROUNDS = 2;
/** Reserved per voice for a host-subagent attempt with no `--estimate-usd` in an unattended Capy run. */
/** Reserved per voice for a host-subagent attempt with no `--estimate-usd` in an unattended run with a spend cap or on Capy. */
export const DEFAULT_HOST_ESTIMATE_USD = 5;
/** Round r of a phase: 1 plus the times it was reopened. */
@@ -35,17 +35,20 @@ export const atCap = (state: RunState, phase: string): boolean => { const cap =
/** The phase is about to open past its cap (a rerun-all reply reopened it). */
export const overCap = (state: RunState, phase: string): boolean => { const cap = roundCap(state, phase); return cap !== null && roundOf(state, phase) > cap; };
/** Open High or Critical findings of a closed phase: the deferred counts, else bound rows not settled or answered, an outside row paired with a native one counted once. */
/** Open High or Critical findings of a closed phase: the deferred counts, else bound rows not settled or answered, an outside row naming the same problem (`sameProblem`) as a High or Critical native one counted once. */
function openHigh(state: RunState, phase: string): number {
const ps = state.phases[phase];
if (ps?.deferred) return (ps.deferred.counts.High ?? 0) + (ps.deferred.counts.Critical ?? 0);
const file = path.join(state.out, `${phase}-consensus.json`);
let rows: ConsensusRecord['rows'] = [];
try { rows = fs.existsSync(file) ? (JSON.parse(fs.readFileSync(file, 'utf8')) as ConsensusRecord).rows : []; } catch {}
const paired = new Set(rows.filter(r => r.native_id).map(r => r.outside_id));
const high = (s: string | undefined) => s === 'High' || s === 'Critical';
const native = voiceRows(state, phase, 'native');
const nativeHigh = new Set(native.filter(r => high(r.severity)).map(r => r.id));
const paired = new Set(rows.filter(r => r.native_id && sameProblem(r) && nativeHigh.has(r.native_id)).map(r => r.outside_id));
const answered = settledIds(state);
return [...voiceRows(state, phase, 'native'), ...voiceRows(state, phase, 'outside')]
.filter(r => !paired.has(r.id) && !(r as { settled_by?: string }).settled_by && !answered.has(r.id) && (r.severity === 'High' || r.severity === 'Critical')).length;
return [...native, ...voiceRows(state, phase, 'outside')]
.filter(r => !paired.has(r.id) && !(r as { settled_by?: string }).settled_by && !answered.has(r.id) && high(r.severity)).length;
}
/** At the round cap with High or Critical findings still open, more rounds buy nothing: the owner decides. */
@@ -59,11 +62,16 @@ export function roundsLine(state: RunState): string {
return `ROUNDS: ${state.phase_order.map(cell).join(' ')} max_rounds=${state.max_rounds ?? 'none'} host_spend=unknown`;
}
/** The estimate one new attempt reserves: the flag, else the Capy unattended default, else 0 (host spend unknown, not admitted). */
/**
* The estimate one new attempt reserves: the flag, else the run's persisted `--estimate-usd`, else the default whenever
* the run has a spend cap (any session kind) or an unattended run is on Capy, else 0 (host spend unknown, not admitted).
* `run` prices its api attempts from the price table instead (NextOptions.price); an unpriced model falls back to this.
*/
export function attemptEstimate(state: RunState, flag: number | undefined, env: NodeJS.ProcessEnv): { usd: number; line?: string } {
if (flag !== undefined) return { usd: flag };
if (state.session_kind !== 'unattended' || env.GSTACK_CAPY !== '1') return { usd: 0 };
return { usd: DEFAULT_HOST_ESTIMATE_USD, line: `ESTIMATE: ${DEFAULT_HOST_ESTIMATE_USD.toFixed(2)} per voice reserved for each host-subagent attempt (Capy unattended default; --estimate-usd <n> sets it); actual host spend stays unknown` };
const set = flag ?? state.estimate_usd;
if (set !== undefined) return { usd: set };
if (state.spend_cap_usd === null && (state.session_kind !== 'unattended' || env.GSTACK_CAPY !== '1')) return { usd: 0 };
return { usd: DEFAULT_HOST_ESTIMATE_USD, line: `ESTIMATE: ${DEFAULT_HOST_ESTIMATE_USD.toFixed(2)} per voice reserved for each host-subagent attempt (default under a spend cap, or unattended on Capy; --estimate-usd <n> sets it for the rest of the run); actual host spend stays unknown` };
}
/** Stop the run: terminal `incomplete` with the structured budget_exhausted stop, journaled. Returns the STOP and RECOVER lines. */
lib/autoplan-carry.ts
+12 -8
View File
@@ -7,11 +7,13 @@
* carried answer is the owner's own earlier words, and every refusal says which condition failed.
*
* A carry needs all of: carry not turned off (`--no-carry-forward`); gate_rev N approved (p1 a or b)
* with the owner's words recorded, and not itself a carry; every change accepted at N
* with the owner's words recorded, and not itself a carry; nothing left pending there; every change accepted at N
* incorporated or settled; the plan file unchanged since the gate or the revise that took those
* changes, and the reviewed plan equal to it; no open urgent finding; and every new item already
* carried. Two kinds of new item do not block: items a "do it" covers (the owner's words are
* "do it": an auto item that changes no scope, cost or direction and is no round grant), and a
* carried. An answer given item by item (tokens, not `rec`) carries only that: every item on
* N+1 already carried by origin, its p1 choice replayed and its tokens recorded verbatim as the
* basis; any other new or changed item goes to the owner (an unreviewed capped delta, below, is the one exception). After `rec` or "do it", two kinds of new
* item do not block: items a "do it" covers (the owner's words are "do it": an auto item that changes no scope, cost or direction and is no round grant), and a
* capped-delta item recommending "accept", which first gets one reviewer on just the delta
* (`CARRY_CHECK:`, answered with `gstack-autoplan check`); a High or Critical finding there
* means no carry, and the gate goes to the owner as before.
@@ -92,7 +94,8 @@ function deltaSince(state: RunState, rev: number): { diff: string; sha: string }
return { diff, sha: sha256(diff) };
}
interface Source { id: string; gate_rev: number; words: string; by?: string }
/** `p1`: an item-token answer's own p1 choice, replayed as given. */
interface Source { id: string; gate_rev: number; words: string; by?: string; p1?: string }
type Plan = { source: Source; doIt: GateItem[]; deltas: GateItem[] } | { reason: string; from?: number };
/** Every condition but the delta check; returns why not, or what the carry takes. */
@@ -105,7 +108,8 @@ function assess(state: RunState, prev: GateList, list: GateList): Plan {
if (unanswered) return { reason: `gate_rev ${from} left ${unanswered.label ?? unanswered.id} unanswered`, from };
if (p1.chosen !== 'a' && p1.chosen !== 'b') return { reason: `gate_rev ${from} answer was p1${p1.chosen}, not an approval`, from };
if (p1.carried_gate_from) return { reason: `gate_rev ${from} was itself carried; the owner answers this one`, from };
if (parseReply(p1.reply ?? '', prev).mode !== 'rec') return { reason: `gate_rev ${from} was answered with item tokens, not rec or "do it"`, from };
const tokens = parseReply(p1.reply ?? '', prev).mode !== 'rec';
if (p1.words_by) return { reason: `gate_rev ${from} was answered in ${p1.words_by}'s own words, not the owner's`, from };
const words = p1.owner_reply ?? (p1.answered_by === 'human' ? p1.reply : undefined);
if (!words?.trim()) return { reason: `no owner words recorded at gate_rev ${from}`, from };
const amendments = (state.amendments ?? []).filter(a => a.gate_rev === from);
@@ -125,10 +129,10 @@ function assess(state: RunState, prev: GateList, list: GateList): Plan {
if (item.id === 'p1' || item.carried) continue;
const option = item.options.find(o => o.key === item.recommended);
if (item.cap?.kind === 'delta' && !option?.changes_plan && item.recommended === 'a') { deltas.push(item); continue; }
if (isDoIt(words) && item.kind === 'auto' && !item.cap && !option?.changes_plan && !['scope', 'cost', 'direction'].includes(item.changes ?? '')) { doIt.push(item); continue; }
if (!tokens && isDoIt(words) && item.kind === 'auto' && !item.cap && !option?.changes_plan && !['scope', 'cost', 'direction'].includes(item.changes ?? '')) { doIt.push(item); continue; }
return { reason: `new item ${item.id}: ${item.title}`, from };
}
return { source: { id: p1.id, gate_rev: from, words, by: p1.answered_by }, doIt, deltas };
return { source: { id: p1.id, gate_rev: from, words, by: p1.answered_by, ...(tokens ? { p1: p1.chosen } : {}) }, doIt, deltas };
}
/** Called by finishGate once gate_rev N+1 is written (status gate_pending): carry, set up the delta check, or say why not. */
@@ -178,7 +182,7 @@ function dispatchCheck(state: RunState, prev: GateList, list: GateList, plan: Ex
function carry(state: RunState, list: GateList, source: Source, doIt: GateItem[], deltas: GateItem[], check?: CarryCheck): CarryRecord {
const rev = list.gate_rev;
const p1 = list.items.find(i => i.id === 'p1')!;
const reply = list.items.map(i => `${i.id}${i.id === 'p1' ? p1.recommended : i.carried?.chosen ?? i.recommended}`).join(' ');
const reply = list.items.map(i => `${i.id}${i.id === 'p1' ? source.p1 ?? p1.recommended : i.carried?.chosen ?? i.recommended}`).join(' ');
const reason = [doIt.length || deltas.length ? '' : 'no new choice', doIt.length ? `"do it" covers ${doIt.map(i => i.id).join(',')}` : '', deltas.length ? `delta check passed on ${deltas.map(i => i.id).join(',')}` : '',
'plan unchanged since the accepted changes'].filter(Boolean).join('; ');
const answered = answerGate(state, rev, reply, 'agent:gstack-autoplan', { carry: { from: source.id, gate_rev: source.gate_rev, words: source.words, by: source.by, reason } });
lib/autoplan-ceo-only.ts
+105
View File
@@ -0,0 +1,105 @@
/**
* autoplan-ceo-only — the driver's CEO-only run (`gstack-autoplan next
* --ceo-only`, GST-537), the twin of `/autoplan --ceo-only` (the skill's
* CEO-only section, docs/unattended.md#ceo-only-runs). The flag is a command-line choice for one run:
* it conflicts with the flags that add a phase, a plan's scope marker may not
* name it, and a run directory keeps the mode it started with. The run opens
* only the CEO phase. `finishRun` is the one finish path next, phase close,
* resume and close share: a full run writes the gate (`finishGate`); a
* CEO-only run writes `ceo-review.md`, records each would-be gate item as a
* decision row of kind `auto` and status `auto_kept` (nobody answered: the
* plan's direction is kept, and an item only the owner can accept stays
* undecided), and ends `reviewed`. It never writes gate.json,
* never calls approve() and never edits the plan, so the commands that need a
* gate or an approvable run refuse it (CEO_ONLY_NO_GATE); `budget` and
* `close --deferred` still recover a run stopped by its spend cap.
*/
import * as crypto from 'node:crypto';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { buildGate, finishGate } from './autoplan-gate';
import { FINDING_ORIGIN } from './autoplan-confirm';
import { carryLines } from './autoplan-carry';
import { aggregateFindings, exportRun } from './autoplan-export';
import { decisionRows, keepDirection, type GateItem } from './gate-list';
import { MARKER_RE } from './autoplan-scope';
import { RunError, journal, voiceRows, writeDecisions, type RunState } from './autoplan-state';
import type { DecisionRow, FindingRow } from './headless-artifacts';
import { sha256File } from './headless-artifacts';
export const CEO_ONLY_FLAG = '--ceo-only';
export const CEO_ONLY_CONFLICTS = ['--ui', '--developer-tool', '--agent-primary'] as const;
export const CEO_ONLY_REPORT = 'ceo-review.md';
/**
* The first next's checks: true when this next asks for a CEO-only run. A marker naming the flag refuses whether or not
* the flag was passed; the flag refuses beside a phase-adding flag (argument or marker), on a plan at the report path,
* or on a directory that holds a full run.
*/
export function checkCeoOnlyFlags(args: readonly string[], plan: string | undefined, existing: RunState | undefined, out?: string): boolean {
const marker = plan && fs.existsSync(plan) ? MARKER_RE.exec(fs.readFileSync(plan, 'utf8').split('\n', 1)[0]!.replace(/^\uFEFF/, '').trimEnd()) : null;
const tokens = marker?.[1]!.trim().split(/\s+/) ?? [];
if (tokens.includes(CEO_ONLY_FLAG)) throw new RunError('CEO_ONLY_CONFLICT', `the scope marker in ${plan} names ${CEO_ONLY_FLAG}`, 3);
if (!args.includes(CEO_ONLY_FLAG)) return false;
const conflicts = CEO_ONLY_CONFLICTS.filter(f => args.includes(f) || tokens.includes(f));
if (conflicts.length) throw new RunError('CEO_ONLY_CONFLICT', `${CEO_ONLY_FLAG} with ${conflicts.join(', ')}`, 3);
const real = (p: string) => { try { return fs.realpathSync(p); } catch { return path.resolve(p); } };
if (plan && out && (path.resolve(plan) === path.resolve(out, CEO_ONLY_REPORT) || real(plan) === path.join(real(out), CEO_ONLY_REPORT))) throw new RunError('CEO_ONLY_CONFLICT', `the plan ${plan} is the report path`, 3);
if (existing && !existing.ceo_only) throw new RunError('CEO_ONLY_AFTER_INIT', `${existing.out} holds full run ${existing.run} (status ${existing.status})`, 3);
return true;
}
/** answer, recommend and revise need a gate or an approvable run: a CEO-only run has neither (budget and close stay: they recover a budget stop). */
export function refuseCeoOnly(state: RunState, verb: string): void {
if (state.ceo_only) throw new RunError('CEO_ONLY_NO_GATE', `${verb} on CEO-only run ${state.run} (status ${state.status}); report: ${path.join(state.out, CEO_ONLY_REPORT)}`, 3);
}
/** Every phase closed: a full run writes its gate (carry lines follow); a CEO-only run writes its report and ends reviewed. */
export function finishRun(state: RunState): string[] {
if (!state.ceo_only) { finishGate(state); return carryLines(state); }
const report = path.join(state.out, CEO_ONLY_REPORT);
const id = (p: string) => fs.statSync(p, { bigint: true });
const at = fs.existsSync(report) && !fs.lstatSync(report).isSymbolicLink() ? id(report) : undefined;
const plan = at?.nlink === 1n ? [state.plan_source, state.plan_input, state.active_plan].find(p => p && fs.existsSync(p) && id(p).ino === at.ino && id(p).dev === at.dev) : undefined;
if (plan) throw new RunError('CEO_ONLY_CONFLICT', `the plan ${plan} is the report path ${report}`, 3);
const list = buildGate(state, 1);
const items = list.items.filter(i => i.id !== 'p1' && !i.origin?.endsWith(`|${FINDING_ORIGIN}`));
const kept = (item: GateItem) => (item.cap && item.kind === 'approval' ? null : keepDirection(item));
const rows: DecisionRow[] = decisionRows({ ...list, items }).map((row, i) => ({ ...row, kind: 'auto', status: 'auto_kept', chosen: kept(items[i]!), default_if_unanswered: kept(items[i]!), source: 'ceo-only' }));
writeDecisions(state.out, rows);
state.status = 'reviewed';
exportRun(state);
const findings = aggregateFindings(state);
const tmp = `${report}.${crypto.randomBytes(4).toString('hex')}.tmp`;
fs.writeFileSync(tmp, renderReport(state, items, rows, findings), { flag: 'wx' });
fs.renameSync(tmp, report);
state.artifacts[CEO_ONLY_REPORT] = { path: CEO_ONLY_REPORT, sha256: sha256File(report) };
journal(state.out, { event: 'ceo_only_reviewed', decisions: rows.length, findings: findings.length, report: CEO_ONLY_REPORT });
return [`REVIEWED: ceo-only run=${state.run} decisions=${rows.length} (auto_kept) findings=${findings.length} report=${report} (not approved; no gate)`];
}
type Challenge = FindingRow & { user_challenge?: boolean };
const none = (lines: string[]) => (lines.length ? lines : ['None.']);
/** The report both paths write: user challenges (direction kept), scope added, conservative choices, findings and the audit trail. */
function renderReport(state: RunState, items: GateItem[], rows: DecisionRow[], findings: FindingRow[]): string {
const plan = fs.existsSync(state.active_plan) ? fs.readFileSync(state.active_plan, 'utf8') : '';
const title = /^#\s+(.+)$/m.exec(plan)?.[1]?.trim() ?? path.basename(state.plan_input ?? state.plan_source);
const outside = state.phases.ceo?.outside;
const outsideLine = outside?.status === 'completed' ? `completed (${outside.model})` : `${outside?.status ?? 'disabled'}${outside?.reason ? ` (${outside.reason})` : outside?.detail ? ` (${outside.detail})` : ''}`;
const challenges = [...voiceRows(state, 'ceo', 'native'), ...voiceRows(state, 'ceo', 'outside')].filter(f => (f as Challenge).user_challenge);
const where = (f: FindingRow) => (f.file ? `${f.file}${f.line ? `:${f.line}` : ''}` : 'plan');
return [
`# CEO-only review: ${title}`, '',
`Mode: gstack-autoplan driver, CEO phase only (run ${state.run}). Outside voice: ${outsideLine}. Not approved.`, '',
'## User challenges (direction kept)', '',
...none(challenges.map(f => `- [${f.severity}] ${f.title} (${f.id}, ${f.voice}): proposal: ${f.resolution ?? 'unstated'}; the plan's direction is kept`)), '',
'## Scope added', '', 'None. The driver never edits the plan; scope proposals are listed under Findings.', '',
'## Conservative choices', '',
...none(rows.map((r, i) => `- ${r.label}: ${r.title}: ${r.chosen === null ? 'left open for the owner (not accepted)' : `kept ${r.chosen} (${items[i]!.options.find(o => o.key === r.chosen)?.text ?? r.chosen})`}; the reviewers recommended ${r.recommended}`)), '',
'## Findings', '',
...none(findings.map(f => `- [${f.severity}] ${f.title} (${f.id}; ${where(f)}): fix: ${f.resolution ?? 'unstated'}; disposition: ${f.disposition ?? 'open'}${(f as { reconciliation?: string }).reconciliation ? `; voices: ${(f as { reconciliation?: string }).reconciliation}` : ''}`)), '',
'## Decision Audit Trail', '',
...none(rows.map(r => `- ${r.id}: ${r.kind} ${r.status}, chosen ${r.chosen ?? 'none'}, recommended ${r.recommended}${r.recommended_basis ? ` (${r.recommended_basis})` : ''}`)), '',
].join('\n');
}
lib/autoplan-checkpoint.ts
+2 -2
View File
@@ -1,7 +1,7 @@
/**
* autoplan-checkpoint — an opt-in durable copy of a run directory (next-wave
* B12). With GSTACK_AUTOPLAN_CHECKPOINT_DIR set, every mutating command that
* leaves the run at rest (gate_pending, amend_pending, complete, incomplete)
* leaves the run at rest (gate_pending, amend_pending, complete, reviewed, incomplete)
* publishes one immutable generation under `<dir>/<run>/generations/<name>/`:
* `checkpoint.json` (every file's sha256), the run files under `run/`, the
* owner's plan file under `plan-input/` when it lives outside the run, and
@@ -34,7 +34,7 @@ import { readConfigKey } from './state-root';
const MANIFEST = 'checkpoint.json';
const CURRENT = 'current.json';
const GENERATIONS = 'generations';
const AT_REST = ['gate_pending', 'amend_pending', 'complete', 'incomplete'];
const AT_REST = ['gate_pending', 'amend_pending', 'complete', 'reviewed', 'incomplete'];
const FOREIGN_LOCK_STALE_MS = 10 * 60_000;
interface CheckpointManifest {
schema_version: 1; run: string; status: string; gate_rev: number; revision: number; at: string; host: string; gstack_version?: string;
lib/autoplan-confirm.ts
+12 -5
View File
@@ -13,7 +13,7 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import type { FindingRow } from './headless-artifacts';
import { readJsonl } from './headless-artifacts';
import { jaccard, tokens } from './autoplan-reconcile';
import { titleScore } from './autoplan-reconcile';
import { readDecisions, type PhaseState, type RunState } from './autoplan-state';
export interface DiffLine { op: ' ' | '+' | '-'; text: string; a?: number; b?: number }
@@ -108,11 +108,18 @@ export function reviewRows(state: RunState, prev: PhaseState | undefined): Findi
const earlierRows = (state: RunState, phase: string): FindingRow[] =>
state.history.filter(h => h.event === 'reopened' && h.phase === phase).flatMap(h => reviewRows(state, h.previous as PhaseState | undefined));
/** Earlier finding id -> the answered decision whose gate item named it. */
export function settledIds(state: RunState): Map<string, string> {
/** The origin tag of an open Critical/High finding's gate item: only `a` (its fix applied to the plan) settles it. */
export const FINDING_ORIGIN = 'finding';
/**
* Earlier finding id -> the answered decision whose gate item named it. A finding carried into the build stays open,
* and `findings: false` also leaves an applied one open, so a confirm round re-raising it is never suppressed.
*/
export function settledIds(state: RunState, o: { findings?: boolean } = {}): Map<string, string> {
const out = new Map<string, string>();
for (const d of readDecisions(state.out)) {
if (d.chosen == null || d.status === 'pending' || !d.origin) continue;
if (d.origin.endsWith(`|${FINDING_ORIGIN}`) && (d.chosen !== 'a' || o.findings === false)) continue;
for (const id of (d.origin.split('|')[1] ?? '').split('+')) if (id && id !== 'null') out.set(id, d.id);
}
return out;
@@ -125,13 +132,13 @@ export function settledIds(state: RunState): Map<string, string> {
export function annotateConfirm(state: RunState, phase: string, rows: FindingRow[], ctx: { planName: string; planSource: string; snapshotPath: string }): { correlated: number; settled: number } {
const earlier = earlierRows(state, phase);
if (!earlier.length) return { correlated: 0, settled: 0 };
const settled = settledIds(state);
const settled = settledIds(state, { findings: false });
const added = new Set((phaseDelta(state, phase, ctx.snapshotPath)?.ops ?? []).filter(o => o.op === '+' && o.text.trim()).map(o => o.text.trim()));
const planLines = (read(ctx.planSource) ?? '').split('\n');
const isPlan = (f: FindingRow) => f.file === ctx.planName;
let correlated = 0, settledCount = 0;
for (const row of rows as Array<FindingRow & { correlates?: string[]; settled_by?: string }>) {
const same = earlier.filter(e => jaccard(tokens(e.title), tokens(row.title)) >= 0.6 || (!!e.file && !isPlan(e) && e.file === row.file && e.line === row.line));
const same = earlier.filter(e => titleScore(e.title, row.title) >= 0.6 || (!!e.file && !isPlan(e) && e.file === row.file && e.line === row.line));
if (!same.length) continue;
row.correlates = same.map(e => e.id);
correlated++;
lib/autoplan-export.ts
+5 -4
View File
@@ -24,21 +24,22 @@ export const PLAN_WARN_BYTES = 30_000;
export function buildTasks(state: RunState, findings: FindingRow[]): TaskRow[] {
const tasks: TaskRow[] = [];
const covered = new Set<string>();
const blockedBy = state.ceo_only ? [] : [decisionId(state.run, state.gate_rev ?? 1, 'p1')];
let n = 0;
for (const f of findings as Array<FindingRow & { settled_by?: string }>) {
if (f.settled_by || covered.has(f.id)) continue;
if (f.disposition === 'deferred') {
covered.add(f.id);
tasks.push({ schema_version: 2, run: state.run, id: `${state.run}-T${++n}`, title: `deferred from ${f.phase}: ${f.title}`, status: 'pending', depends_on: [], findings: [f.id],
blocked_by: [decisionId(state.run, state.gate_rev ?? 1, 'p1')], acceptance: f.resolution ? f.resolution : `the build resolves: ${f.title}`, priority: f.severity, ...(f.file ? { files: [f.file] } : {}), item: f.phase });
blocked_by: blockedBy, acceptance: f.resolution ? f.resolution : `the build resolves: ${f.title}`, priority: f.severity, ...(f.file ? { files: [f.file] } : {}), item: f.phase });
continue;
}
if (!['Critical', 'High'].includes(f.severity) || f.disposition === 'rejected') continue;
const twin = typeof f.native_counterpart === 'string' ? f.native_counterpart : findings.find(x => x.native_counterpart === f.id)?.id;
const ids = twin ? [f.id, twin] : [f.id];
ids.forEach(id => covered.add(id));
tasks.push({ schema_version: 2, run: state.run, id: `${state.run}-T${++n}`, title: f.title, status: 'blocked_on_decision', depends_on: [], findings: ids,
blocked_by: [decisionId(state.run, state.gate_rev ?? 1, 'p1')], acceptance: f.resolution ? f.resolution : `reviewer re-verifies: ${f.title}`, priority: f.severity, ...(f.file ? { files: [f.file] } : {}), item: f.phase });
tasks.push({ schema_version: 2, run: state.run, id: `${state.run}-T${++n}`, title: f.title, status: state.ceo_only ? 'pending' : 'blocked_on_decision', depends_on: [], findings: ids,
blocked_by: blockedBy, acceptance: f.resolution ? f.resolution : `reviewer re-verifies: ${f.title}`, priority: f.severity, ...(f.file ? { files: [f.file] } : {}), item: f.phase });
}
return tasks;
}
@@ -100,7 +101,7 @@ function runLog(state: RunState, findings: number, tasks: number, decisions: { t
...[...(state.stops ?? []), ...(state.stop ? [state.stop] : [])].map(s => `Stopped ${s.reason} at ${s.at}: cap ${s.cap_usd ?? 'none'}, spent ${s.spent}, reserved ${s.reserved}, unknown ${s.unknown}${s === state.stop ? '' : ' (lifted)'}`),
];
if (budget.length) lines.push(`Budget: max_rounds ${state.max_rounds ?? 'none'}${state.round_grants ? `, grants ${JSON.stringify(state.round_grants)}` : ''}.`, ...budget, '');
lines.push(`Gate: ${state.gate_rev ? `gate_rev ${state.gate_rev}, ${decisions.total} decisions (${decisions.pending} pending)${state.gate && typeof (state.gate as any).reply === 'string' ? `; answered "${(state.gate as any).reply}"${typeof (state.gate as any).answered_by === 'string' ? ` answered by ${(state.gate as any).answered_by}` : ''}` : ''}` : 'not reached'}. Findings: ${findings}. Tasks: ${tasks}.`);
lines.push(`Gate: ${state.ceo_only ? 'none (CEO-only run: ceo-review.md, never approved)' : state.gate_rev ? `gate_rev ${state.gate_rev}, ${decisions.total} decisions (${decisions.pending} pending)${state.gate && typeof (state.gate as any).reply === 'string' ? `; answered "${(state.gate as any).reply}"${typeof (state.gate as any).answered_by === 'string' ? ` answered by ${(state.gate as any).answered_by}` : ''}` : ''}` : 'not reached'}. Findings: ${findings}. Tasks: ${tasks}.`);
return lines.join('\n') + '\n';
}
lib/autoplan-gate.ts
+32 -12
View File
@@ -3,7 +3,7 @@
* (plan B2, B9, B11). `closePhase` reconciles the outside pass against the
* native one from canonical ids (lib/autoplan-reconcile.ts), writes the
* consensus record and the timing row, and marks the phase closed.
* `finishGate` turns disagreements and user challenges into one gate list
* `finishGate` turns disagreements, user challenges and open Critical/High findings into one gate list
* (lib/gate-list.ts: `d<n>` auto items, `uc<n>` approval items, `p1` the plan
* approval), writes decisions.jsonl, tasks.jsonl and the exported files, and
* leaves the run `gate_pending`: unattended never approves. `answerGate`
@@ -20,15 +20,15 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import { exportRun } from './autoplan-export';
import { driftLine, reviewHash } from './autoplan-plan-text';
import { consensusRecord, reconcile, remedyKey, renderConsensus, SETTLEABLE_CHANGES, type ConsensusRecord, type ReconciliationOverride, type ReconciliationRow } from './autoplan-reconcile';
import { consensusRecord, reconcile, remedyKey, renderConsensus, sameProblem, SETTLEABLE_CHANGES, type ConsensusRecord, type ReconciliationOverride, type ReconciliationRow } from './autoplan-reconcile';
import type { Phase } from './autoplan-prompts';
import { FILES, RunError, journal as journalRow, now, readDecisions, voiceRows, voiceTerminal, writeDecisions, type Amendment, type RunState, type VoiceState } from './autoplan-state';
import { applyCapChoices, pendingDeltas, releaseUndispatched, roundCap, roundOf } from './autoplan-budget';
import { evaluateCarry, gateEvidence, voidPendingCheck } from './autoplan-carry';
import { buildEli5, capyRelayLine } from './captain-brief';
import { previousReview, reviewRows, settledIds } from './autoplan-confirm';
import { FINDING_ORIGIN, previousReview, reviewRows, settledIds } from './autoplan-confirm';
import { phaseClose } from './autoplan-timing';
import { ANSWERED_BY_RE, applyReply, decisionId, decisionRows, keepDirection, parseReply, pendingHint, recHash, validateGateList, type GateChoice, type GateItem, type GateList, type GateOption, type ParsedReply } from './gate-list';
import { ANSWERED_BY_RE, applyReply, autoDefault, decisionId, decisionRows, keepDirection, parseReply, pendingHint, recHash, validateGateList, type GateChoice, type GateItem, type GateList, type GateOption, type ParsedReply } from './gate-list';
import type { FindingRow } from './headless-artifacts';
import { readJsonl, sha256File } from './headless-artifacts';
import { closeDraftDirectionsFile, loadBriefInputs } from './owner-brief';
@@ -205,14 +205,15 @@ export function buildGate(state: RunState, gateRev: number): GateList {
const byId = new Map([...native, ...outside].map(f => [f.id, f]));
const reran = gateRev > 1 && state.history.some(h => h.event === 'reopened' && h.phase === phase);
const origin = (ids: string, options: GateOption[]) => [phase, ids, ps.closed_at, ps.snapshot, optionHash(options)].join('|');
const push = (item: GateItem, ids: string) => items.push({ ...item, origin: origin(ids, item.options) });
const push = (item: GateItem, ids: string, tag?: string) => items.push({ ...item, origin: [origin(ids, item.options), ...(tag ? [tag] : [])].join('|') });
const challenged = new Set<string>();
const twins = new Map(consensus.rows.filter(r => sameProblem(r) && (r.disposition === 'confirmed' || r.disposition === 'unverified')).map(r => [r.native_id!, r.outside_id]));
for (const row of consensus.rows) {
const o = byId.get(row.outside_id)! as FindingRow & { user_challenge?: boolean; changes?: string; settled_by?: string };
if (o.settled_by) continue;
const n = (row.native_id ? byId.get(row.native_id) : undefined) as (FindingRow & { user_challenge?: boolean; changes?: string }) | undefined;
const ids = `${row.native_id}+${row.outside_id}`;
if (o.user_challenge && n?.user_challenge && row.disposition !== 'new') {
if (o.user_challenge && n?.user_challenge && row.disposition !== 'new' && row.basis !== 'file') {
challenged.add(o.id); challenged.add(n.id);
const title = `${n.title} [${row.native_id} + ${row.outside_id}]`;
if (reran && !n.changes && !o.changes) { push(agentDecision(`d${++d}`, title, phase, 'both voices recommend changing your stated direction'), ids); continue; }
@@ -223,6 +224,7 @@ export function buildGate(state: RunState, gateRev: number): GateList {
continue;
}
if (row.disposition === 'disagree') {
challenged.add(o.id); challenged.add(n!.id);
push({ id: `d${++d}`, title: `${n!.title} [${row.native_id} vs ${row.outside_id}]`, kind: 'auto', recommended: 'a', phase,
why: `voices disagree: ${row.resolution}`, cost: `b reopens ${phase}${phase === 'eng' ? '' : ' and eng'}`,
options: [{ key: 'a', text: `native position (${row.severity.native})`, changes_plan: false }, { key: 'b', text: `outside position (${row.severity.outside})`, changes_plan: true }] }, ids);
@@ -235,6 +237,20 @@ export function buildGate(state: RunState, gateRev: number): GateList {
push({ id: `d${++d}`, title: `${f.title} [${f.id}]`, kind: 'auto', recommended: 'a', phase, why, ...(f.changes ? { changes: f.changes } : {}),
cost: `b reopens ${phase}${phase === 'eng' ? '' : ' and eng'}`, options: [KEEP('keep your direction'), { key: 'b', text: 'accept the change', changes_plan: true }] }, f.id);
}
// An open Critical or High finding the items above do not cover: rec applies its fix to the plan.
const pairedOutside = new Set(twins.values());
for (const f of [...native, ...outside] as Array<FindingRow & { user_challenge?: boolean; settled_by?: string }>) {
if (f.voice === 'outside' && pairedOutside.has(f.id)) continue;
const group = [f, ...(twins.has(f.id) ? [byId.get(twins.get(f.id)!)!] : [])] as Array<FindingRow & { user_challenge?: boolean; settled_by?: string }>;
if (group.some(g => challenged.has(g.id) || g.user_challenge || g.settled_by)) continue;
const severity = group.some(g => g.severity === 'Critical') ? 'Critical' : group.some(g => g.severity === 'High') ? 'High' : null;
if (!severity) continue;
const ids = group.map(g => g.id);
const fixes = [...new Set(group.map(g => g.resolution).filter((r): r is string => !!r))];
push({ id: `d${++d}`, title: `${severity}: ${f.title} [${ids.join(' + ')}]`, kind: 'auto', recommended: 'a', phase, finding: ids,
why: `an open ${severity} finding (${group.map(g => g.voice).join(' + ')}); rec applies its fix to the plan`, cost: `a reopens ${phase}${phase === 'eng' ? '' : ' and eng'} once the plan is revised`,
options: [{ key: 'a', text: `apply the fix to the plan: ${fixes.length ? fixes.join('; ') : `resolve "${f.title}"`}`, changes_plan: true }, { key: 'b', text: 'carry it into the build as a blocking task (no plan change)', changes_plan: false }] }, ids.join('+'), FINDING_ORIGIN);
}
}
items.push(...capItems(state, { d: () => `d${++d}`, uc: () => `uc${++uc}` }));
items.push(planApproval(state, items));
@@ -277,7 +293,7 @@ export function recommendGate(state: RunState, gateRev: number, itemId: string,
writeDecisions(state.out, readDecisions(state.out).map(r => {
const i = updated.get(r.id);
if (!i) return r;
return { ...r, recommended: i.recommended, ...(i.kind === 'auto' ? { default_if_unanswered: i.recommended } : {}), ...(i.recommended_basis ? { recommended_basis: i.recommended_basis } : {}), ...(i.recommended_by ? { recommended_by: i.recommended_by } : {}) };
return { ...r, recommended: i.recommended, ...(i.kind === 'auto' ? { default_if_unanswered: autoDefault(i) } : {}), ...(i.recommended_basis ? { recommended_basis: i.recommended_basis } : {}), ...(i.recommended_by ? { recommended_by: i.recommended_by } : {}) };
}));
const hash = recHash(list);
journalRow(state.out, { event: 'recommended', gate_rev: gateRev, item: itemId, option, basis, by, rec_rev: list.rec_rev, rec_hash: hash });
@@ -344,6 +360,8 @@ export interface AnswerOptions {
recHash?: string;
/** The owner's verbatim words; required for every `--by agent:<id>` answer. */
ownerReply?: string;
/** `agent:<id>` when those words are the answering agent's own: recorded as `answer_words`/`words_by`, never as `owner_reply`, and never carried. */
wordsBy?: string;
/** A whole-gate carry (lib/autoplan-carry.ts): the earlier p1 decision and the owner's words it reuses. Replaces ownerReply and its re-parse. */
carry?: { from: string; gate_rev: number; words: string; by?: string; reason: string };
}
@@ -396,22 +414,24 @@ export function answerGate(state: RunState, gateRev: number, reply: string, answ
const currentHash = recHash(list);
if (o.recHash !== undefined && o.recHash !== currentHash) throw new RunError('GATE_REC_STALE', `--rec-hash ${o.recHash} was rendered before the recommendations changed; current rec_hash=${currentHash}`, 3);
if (parsed.mode === 'rec' && list.rec_rev === undefined && o.recHash === undefined) throw new RunError('GATE_REC_STALE', `this gate was written before \`rec\`/\`all\` took plan changes (gate.json has no rec_rev), so the owner saw a different meaning; render it again (gstack-gate render ${gateFile} --all), relay it, and answer with --rec-hash ${currentHash}, or reply with explicit <id><option> tokens`, 3);
if (parsed.mode === 'rec' && (list.rec_rev ?? 0) > 0 && o.recHash === undefined) throw new RunError('GATE_REC_STALE', `recommend changed this gate (rec_rev=${list.rec_rev}); pass --rec-hash from the render the owner saw (current rec_hash=${currentHash})`, 3);
const takesRecs = parsed.mode === 'rec' || parsed.choices.some(c => c.id === 'p1' && c.chosen === 'b' && !c.prior);
if (takesRecs && (list.rec_rev ?? 0) > 0 && o.recHash === undefined) throw new RunError('GATE_REC_STALE', `recommend changed this gate (rec_rev=${list.rec_rev}); \`rec\` and \`p1b\` take its recommendations, so pass --rec-hash from the render the owner saw (current rec_hash=${currentHash})`, 3);
const ownerParsed = o.ownerReply !== undefined && !o.carry ? checkOwnerReply(list, gateRev, prior, parsed, o.ownerReply) : undefined;
const provenance = o.carry ? { owner_reply: o.carry.words, carried_gate_from: o.carry.from, carried_gate_rev: o.carry.gate_rev, carry_reason: o.carry.reason, ...(o.carry.by ? { carried_answered_by: o.carry.by } : {}) }
: o.ownerReply !== undefined && o.wordsBy ? { answer_words: o.ownerReply, answer_words_parsed: ownerParsed, words_by: o.wordsBy }
: o.ownerReply !== undefined ? { owner_reply: o.ownerReply, owner_reply_parsed: ownerParsed } : {};
const itemOf = (id: string) => list.items.find(i => i.id === id)!;
const optionOf = (c: GateChoice) => itemOf(c.id).options.find(o => o.key === c.chosen)!;
const plan = parsed.choices.find(c => c.id === 'p1');
const lines: string[] = o.carry ? [] : voidPendingCheck(state, `the owner answered gate_rev ${gateRev}`);
const answeredAt = now();
const record = () => writeDecisions(state.out, applyReply(rows, parsed, answeredAt, answeredBy, provenance).map(r => {
const record = (answered = parsed) => writeDecisions(state.out, applyReply(rows, answered, answeredAt, answeredBy, provenance).map(r => {
const c = r.gate_rev === list.gate_rev && r.chosen && !r.chosen_text ? itemOf((r.label ?? '').toLowerCase())?.options.find(o => o.key === r.chosen) : undefined;
return c ? { ...r, chosen_text: c.text } : r;
}));
if (!plan || parsed.pending.length) {
// Approvals unanswered: record what was answered, keep the gate pending; nothing reviewed changes.
record();
// Approvals unanswered: record what was answered, but not an auto default that p1b would change (the final answer decides it).
record({ ...parsed, choices: parsed.choices.filter(c => c.how !== 'auto_default' || c.chosen === itemOf(c.id).recommended) });
const pending = parsed.pending.length ? parsed.pending : ['p1'];
state.gate = { ...(state.gate ?? {}), last_reply: reply, pending };
lines.push(`GATE_PENDING: ${pending.join(', ')} unanswered (${pendingHint(list, pending)})`);
@@ -421,7 +441,7 @@ export function answerGate(state: RunState, gateRev: number, reply: string, answ
const carried = (c: GateChoice) => itemOf(c.id).carried?.chosen === c.chosen;
const changing = proceeds ? parsed.choices.filter(c => c.id !== 'p1' && !carried(c) && itemOf(c.id).reopens !== false && (optionOf(c).changes_plan ?? c.how === 'override')) : [];
const tokens = (cs: GateChoice[]) => cs.map(c => `${c.id}${c.chosen}`);
const overrides = parsed.choices.filter(c => c.how === 'override' && c.id !== 'p1');
const overrides = parsed.choices.filter(c => c.id !== 'p1' && (c.how === 'override' || (c.how === 'auto_default' && c.chosen !== itemOf(c.id).recommended)));
const reopenSet = new Set<Phase>(plan.chosen === 'd' ? state.phase_order : []);
record();
state.gate = { ...(state.gate ?? {}), answered_at: answeredAt, reply, gate_rev: gateRev, plan: plan.chosen, overrides: tokens(overrides), ...(answeredBy ? { answered_by: answeredBy } : {}), ...provenance };
lib/autoplan-reconcile.ts
+29 -7
View File
@@ -196,20 +196,30 @@ export interface ReconciliationRow {
outside_id: string; native_id: string | null; disposition: ReconciliationDisposition;
/** Why this disposition: the matched location or title, or the parent's override reason. */
resolution: string;
basis: 'file_line' | 'title' | 'override' | 'none';
basis: 'file_line' | 'file' | 'title' | 'override' | 'none';
severity: { outside: Severity; native: Severity | null };
/** A parent override's one shared proposal for both voices' user challenge; the gate renders it as option b. */
proposal?: string;
}
export interface ReconciliationOverride { outside_id: string; native_id?: string | null; disposition: ReconciliationDisposition; resolution: string; proposal?: string }
export const tokens = (title: string) => new Set(title.toLowerCase().replace(/[^a-z0-9 ]+/g, ' ').split(/\s+/).filter(t => t.length > 2));
/** A DX cap code (`CAP4:<path>:<item>`) names a location, not the problem, so title comparison skips it. */
export const tokens = (title: string) => new Set(title.replace(/^\s*CAP[46]\s*:[^:]*:\s*/i, '').toLowerCase().replace(/[^a-z0-9 ]+/g, ' ').split(/\s+/).filter(t => t.length > 2));
export function jaccard(a: Set<string>, b: Set<string>): number {
if (!a.size || !b.size) return 0;
let inter = 0;
for (const t of a) if (b.has(t)) inter++;
return inter / (a.size + b.size - inter);
}
const capPath = (title: string) => /^\s*CAP[46]\s*:([^:]*):/i.exec(title)?.[1]!.trim();
/** Title similarity. Two CAP-coded titles on different paths name different problems, so they never match. */
export function titleScore(a: string, b: string): number {
const pa = capPath(a), pb = capPath(b);
if (pa !== undefined && pb !== undefined && pa !== pb) return 0;
return jaccard(tokens(a), tokens(b));
}
/** The voices name one problem: a title association or the parent's override. A shared location (file, or file:line with unrelated titles) associates two findings but never makes them one. */
export const sameProblem = (r: ReconciliationRow) => !!r.native_id && (r.basis === 'title' || r.basis === 'override' || (r.basis === 'file_line' && !r.resolution.startsWith('same location')));
const level = (s: Severity) => SEVERITIES.indexOf(s);
/** A remedy compared as words: case, punctuation and spacing ignored. */
export const remedyKey = (text: string | undefined) => (text ?? '').toLowerCase().replace(/[^a-z0-9]+/g, ' ').trim();
@@ -225,7 +235,9 @@ function contradictory(a: string, b: string): boolean {
/**
* Structural reconciliation. Findings associate by a title overlap of at least
* half the words, or by file:line (±3 lines). Association alone never
* half the words (a leading CAP code ignored; never across two different files),
* or by file:line (±3 lines), or, when either finding has no line, by the same
* file. Association alone never
* confirms: a title association is `confirmed` only when both voices state the
* same remedy, `disagree` when severities differ by two or more levels or the
* remedies contradict (one keeps, drops or negates what the other changes),
@@ -250,6 +262,8 @@ export function reconcile(native: FindingRow[], outside: FindingRow[], overrides
}
const used = new Set<string>();
const rows: ReconciliationRow[] = [];
const later: Array<[number, FindingRow]> = [];
const unmatched = (o: FindingRow): ReconciliationRow => ({ outside_id: o.id, native_id: null, disposition: 'new', resolution: 'not raised by the native voice', basis: 'none', severity: { outside: o.severity, native: null } });
for (const o of outside) {
const override = byOutside.get(o.id);
if (override) {
@@ -260,17 +274,17 @@ export function reconcile(native: FindingRow[], outside: FindingRow[], overrides
continue;
}
const near = (n: FindingRow) => !!o.file && o.line !== undefined && n.file === o.file && n.line !== undefined && Math.abs(n.line - o.line) <= 3;
const ot = tokens(o.title);
let match: FindingRow | undefined;
let best = 0;
for (const n of native) {
if (used.has(n.id)) continue;
const score = jaccard(ot, tokens(n.title)) + (near(n) ? 0.01 : 0);
if (used.has(n.id) || (o.file && n.file && o.file !== n.file)) continue;
const score = titleScore(o.title, n.title) + (near(n) ? 0.01 : 0);
if (score >= 0.5 && score > best) { best = score; match = n; }
}
const titled = !!match;
match ??= native.find(n => !used.has(n.id) && near(n));
if (!match) { rows.push({ outside_id: o.id, native_id: null, disposition: 'new', resolution: 'not raised by the native voice', basis: 'none', severity: { outside: o.severity, native: null } }); continue; }
if (!match && o.file) { later.push([rows.length, o]); rows.push(unmatched(o)); continue; }
if (!match) { rows.push(unmatched(o)); continue; }
used.add(match.id);
const basis: ReconciliationRow['basis'] = near(match) ? 'file_line' : 'title';
const where = near(match) ? `${o.file}:${o.line}` : 'same title';
@@ -286,6 +300,14 @@ export function reconcile(native: FindingRow[], outside: FindingRow[], overrides
if (remedyKey(match.resolution) !== remedyKey(o.resolution)) { pair('unverified', `same title, remedies differ (${where}): ${match.title}`); continue; }
pair('confirmed', `both voices, same remedy (${where}): ${match.title}`);
}
// Plan reviews name a file without a line: same file, a line missing on either side, once every title and line match is placed.
for (const [i, o] of later) {
const match = native.filter(n => !used.has(n.id) && n.file === o.file && (o.line === undefined || n.line === undefined))
.reduce<FindingRow | undefined>((b, n) => (!b || titleScore(o.title, n.title) > titleScore(o.title, b.title) ? n : b), undefined);
if (!match) continue;
used.add(match.id);
rows[i] = { outside_id: o.id, native_id: match.id, disposition: 'unverified', resolution: `same file, agreement unverified (${o.file}): ${match.title}`, basis: 'file', severity: { outside: o.severity, native: match.severity } };
}
return { rows, native_only: native.filter(n => !used.has(n.id)).map(n => n.id), errors };
}
lib/autoplan-run.ts
+75 -26
View File
@@ -19,8 +19,8 @@ import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { canonicalRows, parseReceipt, parseRecordFindings, parseReviewResult, type CanonicalFinding } from './autoplan-reconcile';
import { amendmentSteps, applyGuidance, closeDeferred, closePhase, finishGate } from './autoplan-gate';
import { carryLines } from './autoplan-carry';
import { amendmentSteps, applyGuidance, closeDeferred, closePhase } from './autoplan-gate';
import { finishRun } from './autoplan-ceo-only';
import { GSTACK_VERSION } from './autoplan-migrate';
import { driftLine, normalizeFindingFile, reviewHash, scaffold } from './autoplan-plan-text';
import { PHASES, SNAPSHOT_TOOL, attemptFile, bindPromptResult, detectScope, driverFile, extractHeading, phaseOrder, reviewSkillFile, scopeLines, scopeSourceLabel, sectionLoadFor, snapshotTool, writePrompts, type Phase, type Scope, type ScopeFlags } from './autoplan-prompts';
@@ -31,7 +31,7 @@ import type { RunStatus } from './headless-artifacts';
import { sha256File } from './headless-artifacts';
import { outsideConsent } from './outside-consent';
import { familyConflict, modelFamily } from './outside-voice-runner';
import { renderSpend, reserve, settle, summarize, readLedger } from './spend-ledger';
import { SpendCapExceeded, release, renderSpend, reserve, settle, summarize, readLedger } from './spend-ledger';
import { DEFAULT_MAX_ROUNDS, attemptEstimate, overCap, resumeStop, roundCap, roundOf, roundsLine, stopForBudget, stopLines } from './autoplan-budget';
import { annotateConfirm, confirmBlock } from './autoplan-confirm';
@@ -72,7 +72,7 @@ export function withRunLock<T>(out: string, fn: () => T, opts: { steal?: boolean
// ---------------------------------------------------------------------------
// init
// ---------------------------------------------------------------------------
export interface InitOptions { plan: string; out: string; cwd: string; scope?: ScopeFlags; planHeading?: string; outsideEnabled?: boolean; spendCap?: number | null; sessionKind?: string; deadlineMin?: number; light?: boolean; runId?: string; scaffold?: boolean; maxRounds?: number; carryForward?: boolean; carryCheckCapUsd?: number }
export interface InitOptions { plan: string; out: string; cwd: string; scope?: ScopeFlags; ceoOnly?: boolean; planHeading?: string; outsideEnabled?: boolean; spendCap?: number | null; sessionKind?: string; deadlineMin?: number; light?: boolean; runId?: string; scaffold?: boolean; maxRounds?: number; carryForward?: boolean; carryCheckCapUsd?: number }
/** `lines` receives what init itself reports (SCAFFOLDED:). A plan with one section heading missing refuses, or with `scaffold` is reviewed from a scaffolded copy. */
export function initRun(o: InitOptions, lines: string[] = []): RunState {
@@ -103,14 +103,14 @@ export function initRun(o: InitOptions, lines: string[] = []): RunState {
const restore = path.join(out, FILES.restore);
const init = snapshotTool(['init', source, active, restore], o.cwd);
const scope = detectScope(active, o.cwd, o.scope ?? {}, path.resolve(o.plan));
const order = phaseOrder(scope);
const order: Phase[] = o.ceoOnly ? ['ceo'] : phaseOrder(scope);
const run = o.runId ?? `${path.basename(out)}-${new Date().toISOString().replace(/[-:T.Z]/g, '').slice(0, 14)}`;
// codex_reviews gates the outside voice here, before any dispatch: a refusal is a terminal not_run outcome per phase.
const consent = (o.outsideEnabled ?? true) ? outsideConsent({ userInvoked: false }) : undefined;
const consentMissing = consent && !consent.allowed ? consent.lines[0] : undefined;
const sessionKind = o.sessionKind ?? process.env.GSTACK_SESSION_KIND ?? 'unattended';
const phases: Record<string, PhaseState> = {};
for (const p of PHASES) if (!order.includes(p)) phases[p] = { skipped: scope.docs_only ? 'docs-only plan' : p === 'design' ? 'no UI scope detected' : 'no developer-facing scope detected', native: { status: 'skipped' }, outside: { status: 'skipped' } };
for (const p of PHASES) if (!order.includes(p)) phases[p] = { skipped: o.ceoOnly ? 'ceo-only run' : scope.docs_only ? 'docs-only plan' : p === 'design' ? 'no UI scope detected' : 'no developer-facing scope detected', native: { status: 'skipped' }, outside: { status: 'skipped' } };
const state: RunState = {
schema_version: 1, run, status: 'running', required_phases: order, phases, artifacts: {}, revision: 0, out,
plan_source: init.sourcePlan, active_plan: init.activePlan, restore: init.restorePath, scope, phase_order: order,
@@ -120,12 +120,12 @@ export function initRun(o: InitOptions, lines: string[] = []): RunState {
deviations: ['autoplan guard: not enforced by this host; publication order is unverified (GUARD_NOT_INSTALLED)'],
...(o.deadlineMin ? { deadline_at: new Date(Date.now() + o.deadlineMin * 60_000).toISOString() } : {}),
...(o.light ? { light: true } : {}),
gstack_version: GSTACK_VERSION, plan_input: input, scope_flags: { ...(o.scope ?? {}) },
...(o.ceoOnly ? { ceo_only: true } : {}), gstack_version: GSTACK_VERSION, plan_input: input, scope_flags: { ...(o.scope ?? {}) },
...(o.planHeading ? { plan_heading: o.planHeading } : {}), ...(scaffolded ? { scaffolded: true } : {}),
...(o.carryForward === false ? { carry_forward: false } : {}), ...(o.carryCheckCapUsd !== undefined ? { carry_check_cap_usd: o.carryCheckCapUsd } : {}),
};
fs.writeFileSync(path.join(out, 'timing.json'), JSON.stringify({ schema_version: 1, run, phases: [], started_at: now(), session_kind: state.session_kind }, null, 2) + '\n');
journal(out, { event: 'init', run, plan_source: source, scope, phase_order: order, ...(consentMissing ? { outside: 'not_run', reason: 'consent_missing' } : {}) });
journal(out, { event: 'init', run, plan_source: source, scope, phase_order: order, ...(o.ceoOnly ? { ceo_only: true } : {}), ...(consentMissing ? { outside: 'not_run', reason: 'consent_missing' } : {}) });
writeState(state);
return state;
}
@@ -142,7 +142,11 @@ export function voiceState(state: RunState, phase: Phase, voice: Voice): VoiceSt
}
const openAttempt = (v: VoiceState) => v.attempts?.find(a => a.state === 'dispatched' || a.state === 'execution_unknown');
export interface NextOptions { out: string; cwd: string; estimateUsd?: number; redispatch?: string[] }
export interface NextOptions {
out: string; cwd: string; estimateUsd?: number; redispatch?: string[];
/** `run`: a voice's priced reservation from its prompt (undefined when unpriced); `--estimate-usd` wins over it. */
price?: (voice: Voice, promptFile: string) => number | undefined;
}
export interface NextResult { state: RunState; phase?: Phase; attempts: Array<{ attempt: string; voice: Voice; prompt: string; result: string; model_family: string; new: boolean }>; lines: string[] }
function openPhase(state: RunState, phase: Phase, cwd: string): void {
@@ -194,7 +198,7 @@ export function nextStep(o: NextOptions): NextResult {
const lines: string[] = [];
const result: NextResult = { state, attempts: [], lines };
if (state.status === 'incomplete' && state.stop) { lines.push(...stopLines(state)); return result; }
if (state.status === 'gate_pending' || state.status === 'complete' || state.status === 'incomplete' || state.status === 'refused') return result;
if (['gate_pending', 'complete', 'reviewed', 'incomplete', 'refused'].includes(state.status)) return result;
if (state.deadline_at && Date.parse(state.deadline_at) < Date.now()) { state.status = 'interrupted'; writeState(state); lines.push(`DEADLINE: passed at ${state.deadline_at}`); return result; }
const pending = pendingAmendments(state).map(a => a.decision);
if (pending.length) {
@@ -206,7 +210,7 @@ export function nextStep(o: NextOptions): NextResult {
lines.push(...closeDeferred(state, phase, { reason: `round cap: round ${roundOf(state, phase)} would pass max ${roundCap(state, phase)}`, by: 'runner', cause: 'cap' }).lines);
phase = currentPhase(state);
}
if (!phase) { finishGate(state); lines.push(...carryLines(state)); writeState(state); return result; }
if (!phase) { lines.push(...finishRun(state)); writeState(state); return result; }
const first = !state.phase_order.some(p => state.phases[p]?.snapshot);
openPhase(state, phase, o.cwd);
const ps = state.phases[phase]!;
@@ -219,15 +223,20 @@ export function nextStep(o: NextOptions): NextResult {
if (ps.outside?.reason === 'docs_only') lines.push(`OUTSIDE: ${phase} not_run reason=docs_only (native voice only)`);
if (ps.sections) lines.push(`Skipped sections: ${ps.sections.skipped.length ? ps.sections.skipped.join(',') : 'none'} (scope: ui=${state.scope.ui ? 'yes' : 'no'},dx=${state.scope.dx ? 'yes' : 'no'},source=${scopeSourceLabel(state.scope)}; checklist=${ps.sections.checklist ? 'loaded' : 'none'})`);
const prompts = ps.prompts!;
if (o.estimateUsd !== undefined) state.estimate_usd = o.estimateUsd;
const estimate = attemptEstimate(state, o.estimateUsd, process.env);
if (estimate.line) lines.push(estimate.line);
const usdFor = (voice: Voice) => (state.estimate_usd === undefined ? o.price?.(voice, prompts[voice]) : undefined) ?? estimate.usd;
if (estimate.line && (['native', 'outside'] as const).some(voice => usdFor(voice) === estimate.usd)) lines.push(estimate.line);
if (first) lines.push(reviewEstimateLine(state, estimate.usd));
const fresh = (['native', 'outside'] as const).filter(voice => { const v = voiceState(state, phase!, voice); const open = openAttempt(v); return !voiceTerminal(v) && (!open || (open.state === 'execution_unknown' && o.redispatch?.includes(open.attempt))); });
const resultOf = (voice: Voice, attempt: string) => path.join(state.out, attemptFile(phase!, voice, attempt.slice(state.run.length + 1), 'md'));
fresh.forEach((voice, i) => bindPromptResult(prompts[voice], resultOf(voice, `${state.run}-a${state.attempt_counter + i + 1}`)));
const before = summarize(readLedger(path.join(state.out, FILES.spend), state.spend_cap_usd));
if (state.spend_cap_usd !== null && fresh.length && before.worst_case_usd + fresh.length * estimate.usd > state.spend_cap_usd + 1e-9) {
const stop = stopForBudget(state, phase, fresh.length * estimate.usd);
const asked = fresh.reduce((sum, voice) => sum + usdFor(voice), 0);
if (state.spend_cap_usd !== null && fresh.length && before.worst_case_usd + asked > state.spend_cap_usd + 1e-9) {
const stop = stopForBudget(state, phase, asked);
writeState(state);
throw new RunError('SPEND_CAP_EXCEEDED', `spent ${before.spent_usd} + reserved ${before.reserved_usd} + unknown ${before.unknown_usd} + ${fresh.length} x estimate ${estimate.usd} > cap ${state.spend_cap_usd}; the run stopped incomplete (budget_exhausted)`, 3, { status: 'incomplete', lines: [...lines, ...stop] });
throw new RunError('SPEND_CAP_EXCEEDED', `spent ${before.spent_usd} + reserved ${before.reserved_usd} + unknown ${before.unknown_usd} + ${fresh.length} attempt(s) reserving ${+asked.toFixed(6)} > cap ${state.spend_cap_usd}; the run stopped incomplete (budget_exhausted)`, 3, { status: 'incomplete', lines: [...lines, ...stop] });
}
for (const voice of ['native', 'outside'] as const) {
const v = voiceState(state, phase, voice);
@@ -243,11 +252,12 @@ export function nextStep(o: NextOptions): NextResult {
continue;
}
const attempt = `${state.run}-a${++state.attempt_counter}`;
try { reserve(path.join(state.out, FILES.spend), attempt, estimate.usd, { cap: state.spend_cap_usd, label: `${phase}/${voice}` }); }
const resultFile = resultOf(voice, attempt);
const usd = usdFor(voice);
try { reserve(path.join(state.out, FILES.spend), attempt, usd, { cap: state.spend_cap_usd, label: `${phase}/${voice}` }); }
catch (e: any) { if (e.code === 'SPEND_CAP_EXCEEDED') { state.attempt_counter--; writeState(state); throw new RunError('SPEND_CAP_EXCEEDED', e.message, 3); } throw e; }
const row: AttemptState = { attempt, state: 'dispatched', prompt: prompts[voice], result: path.join(state.out, attemptFile(phase, voice, attempt.slice(state.run.length + 1), 'md')), estimate_usd: estimate.usd, dispatched_at: now() };
bindPromptResult(row.prompt, row.result);
journal(state.out, { event: 'dispatched', attempt, phase, voice, snapshot: ps.snapshot, prompt: path.relative(state.out, row.prompt), prompt_sha256: sha256File(row.prompt), prompt_bytes: fs.statSync(row.prompt).size, result: path.relative(state.out, row.result), estimate_usd: estimate.usd });
const row: AttemptState = { attempt, state: 'dispatched', prompt: prompts[voice], result: resultFile, estimate_usd: usd, dispatched_at: now() };
journal(state.out, { event: 'dispatched', attempt, phase, voice, snapshot: ps.snapshot, prompt: path.relative(state.out, row.prompt), prompt_sha256: sha256File(row.prompt), prompt_bytes: fs.statSync(row.prompt).size, result: path.relative(state.out, row.result), estimate_usd: usd });
(v.attempts ??= []).push(row);
v.attempt = attempt; v.status = 'running';
result.attempts.push({ attempt, voice, prompt: row.prompt, result: row.result, model_family: familyHint(state, voice), new: true });
@@ -384,8 +394,9 @@ export function submitResult(o: SubmitOptions): SubmitResult {
function closeAndAdvance(state: RunState, phase: Phase, reconcile?: string): string[] {
const close = closePhase(state, phase, { overridesFile: reconcile });
const gated = !currentPhase(state);
if (gated) finishGate(state); else state.status = 'running';
return [`PHASE_CLOSED: ${phase} confirmed=${close.consensus.confirmed}${close.consensus.unverified ? ` unverified=${close.consensus.unverified}` : ''} disagree=${close.consensus.disagree} new=${close.consensus.new} native_only=${close.consensus.native_only} coverage=${close.consensus.coverage}`, ...(gated ? carryLines(state) : [])];
const finished = gated ? finishRun(state) : [];
if (!gated) state.status = 'running';
return [`PHASE_CLOSED: ${phase} confirmed=${close.consensus.confirmed}${close.consensus.unverified ? ` unverified=${close.consensus.unverified}` : ''} disagree=${close.consensus.disagree} new=${close.consensus.new} native_only=${close.consensus.native_only} coverage=${close.consensus.coverage}`, ...finished];
}
/**
@@ -405,6 +416,44 @@ export function closeOutside(state: RunState, phase: Phase, status: 'unavailable
return lines;
}
/** Raise an open attempt's reservation to what the call about to run can cost; the cap still binds (SPEND_CAP_EXCEEDED). */
export function fitReservation(out: string, phase: Phase, voice: Voice, attempt: string, usd: number): void {
const state = readState(out);
const a = voiceState(state, phase, voice).attempts?.find(x => x.attempt === attempt);
if (!a || usd <= a.estimate_usd + 1e-9) return;
const file = path.join(out, FILES.spend);
release(file, attempt);
const label = `${phase}/${voice}`;
try { reserve(file, attempt, usd, { cap: state.spend_cap_usd, label }); }
catch (e) {
reserve(file, attempt, a.estimate_usd, { label });
if (e instanceof SpendCapExceeded) throw new RunError('SPEND_CAP_EXCEEDED', `${attempt} (${phase} ${voice}) was admitted at ${a.estimate_usd}; the call it is about to make can cost ${usd}, past the cap ${state.spend_cap_usd}`, 3);
throw e;
}
a.estimate_usd = usd;
writeState(state);
}
/** Record that `run`'s paid call for an open attempt is going out (true) or never left this machine (false). */
export function markSent(out: string, phase: Phase, voice: Voice, attempt: string, sent: boolean): void {
const state = readState(out);
const a = voiceState(state, phase, voice).attempts?.find(x => x.attempt === attempt);
if (!a) return;
if (sent) a.sent_at = now(); else delete a.sent_at;
writeState(state);
}
/** A paid call that did not bind a result (failed, refused, lost with its process): settle what it cost and supersede its open attempt, so a retry is admitted afresh. */
export function retireAttempt(out: string, phase: Phase, voice: Voice, attempt: string, usd: number | 'unknown'): void {
const state = readState(out);
const a = voiceState(state, phase, voice).attempts?.find(x => x.attempt === attempt);
if (!a || a.state !== 'dispatched') return;
a.state = 'superseded'; a.terminal_at = now();
settle(path.join(out, FILES.spend), attempt, usd);
journal(out, { event: 'superseded', attempt, phase, voice, usage_usd: usd });
writeState(state);
}
/** Mark the outside voice of the current phase unavailable/failed (the runner could not run); closes the phase if native is bound. */
export function markOutside(out: string, phase: Phase, status: 'unavailable' | 'failed', detail: string): RunState {
const state = readState(out);
@@ -466,15 +515,15 @@ export function resumeRun(out: string): ResumeResult {
if (ps.native?.status === 'completed' && voiceTerminal(ps.outside)) closePhase(state, phase, {});
}
const stopped = resumeStop(state);
const gated = !currentPhase(state) && !['gate_pending', 'amend_pending', 'complete', 'incomplete', 'refused'].includes(state.status);
if (gated) finishGate(state);
else if (state.status === 'interrupted' || state.status === 'running' || state.status === 'awaiting_result') {
const gated = !currentPhase(state) && !['gate_pending', 'amend_pending', 'complete', 'reviewed', 'incomplete', 'refused'].includes(state.status);
const finished = gated ? finishRun(state) : [];
if (!gated && (state.status === 'interrupted' || state.status === 'running' || state.status === 'awaiting_result')) {
const anyOpen = state.phase_order.some(p => (['native', 'outside'] as const).some(v => openAttempt(voiceState(state, p, v))));
state.status = anyOpen ? 'awaiting_result' : 'running';
}
journal(out, { event: 'resumed', unknown, rebound, status: state.status });
writeState(state);
const lines = [...recovered, ...stopped, ...(gated ? carryLines(state) : []), `RESUMED: run=${state.run} status=${state.status} execution_unknown=${unknown.length} rebound=${rebound.length}`];
const lines = [...recovered, ...stopped, ...finished, `RESUMED: run=${state.run} status=${state.status} execution_unknown=${unknown.length} rebound=${rebound.length}`];
for (const a of unknown) lines.push(`EXECUTION_UNKNOWN: ${a}`);
return { state, unknown, rebound, lines };
}
@@ -498,6 +547,6 @@ export function statusLines(state: RunState): string[] {
for (const d of state.capped_deltas ?? []) lines.push(`CAPPED_DELTA: v${d.k} ${d.phase} +${d.added} -${d.removed} ${d.status}`);
return lines;
}
export const TERMINAL: readonly RunStatus[] = ['complete', 'gate_pending', 'amend_pending', 'incomplete', 'refused', 'interrupted'];
export const TERMINAL: readonly RunStatus[] = ['complete', 'reviewed', 'gate_pending', 'amend_pending', 'incomplete', 'refused', 'interrupted'];
export { SNAPSHOT_TOOL, RunError, FILES, readState, writeState, readJournal, journal };
export type { RunState, Voice, VoiceState, AttemptState };
lib/autoplan-scope.ts
+1 -1
View File
@@ -65,7 +65,7 @@ export function detectDocsOnly(implementationPlan: string, root: string): DocsOn
export interface ScopeFlags { ui?: boolean; developerTool?: boolean; agentPrimary?: boolean; dx?: boolean; docsOnly?: boolean }
export const SCOPE_FLAG_TOKENS = ['--ui', '--no-ui', '--no-dx', '--developer-tool', '--agent-primary', '--docs-only'] as const;
const MARKER_RE = /^<!--\s*gstack-autoplan-scope:(.*?)-->\s*$/;
export const MARKER_RE = /^<!--\s*gstack-autoplan-scope:(.*?)-->\s*$/;
export function scopeTokens(f: ScopeFlags): string[] {
return [...(f.ui === true ? ['--ui'] : f.ui === false ? ['--no-ui'] : []), ...(f.dx === false ? ['--no-dx'] : []),
lib/autoplan-state.ts
+4 -2
View File
@@ -13,7 +13,7 @@ import { readJsonl } from './headless-artifacts';
import type { ResultCodeName } from './result-codes';
export type Voice = 'native' | 'outside';
export interface AttemptState { attempt: string; state: 'dispatched' | 'submitted' | 'execution_unknown' | 'cancelled' | 'superseded' | 'deferred'; prompt: string; result: string; estimate_usd: number; dispatched_at: string; terminal_at?: string }
export interface AttemptState { attempt: string; state: 'dispatched' | 'submitted' | 'execution_unknown' | 'cancelled' | 'superseded' | 'deferred'; prompt: string; result: string; estimate_usd: number; dispatched_at: string; terminal_at?: string; /** `run` set it before the paid call went out; an open attempt still carrying it was in flight when its process died. */ sent_at?: string }
export interface VoiceState extends VoiceOutcome {
attempt?: string; attempts?: AttemptState[]; result_sha256?: string; findings?: number; label?: string; usage?: Record<string, unknown>; detail?: string;
/** verified: the INPUT receipt matched the snapshot; attested: a --record bound without one, on the parent's word (attest_reason). */
@@ -58,7 +58,7 @@ export interface DispatchCounts { total: number; by_phase: Record<string, Partia
export interface RunState extends RunManifest {
revision: number; out: string; plan_source: string; active_plan: string; restore: string; scope: Scope; phase_order: Phase[];
phases: Record<string, PhaseState>; native_model?: string; outside_model?: string; outside_enabled: boolean; attempt_counter: number;
spend_cap_usd: number | null; deadline_at?: string; light?: boolean; history: Array<Record<string, unknown>>; created_at: string; updated_at: string;
spend_cap_usd: number | null; /** `next --estimate-usd`, kept for every later attempt. */ estimate_usd?: number; deadline_at?: string; light?: boolean; history: Array<Record<string, unknown>>; created_at: string; updated_at: string;
/** The gstack that last wrote this run (lib/autoplan-migrate.ts); absent on runs written before 1.91.96. */
gstack_version?: string;
/** The --plan file the owner edits (drift hints, revise), the --plan-heading it was cut from, and the scope flags revise re-detects with. */
@@ -72,6 +72,8 @@ export interface RunState extends RunManifest {
dispatches?: DispatchCounts; est_input_tokens?: number;
/** Carry-forward (lib/autoplan-carry.ts): false with --no-carry-forward; the per-gate cap on the delta check; the latest carry decision. */
carry_forward?: boolean; carry_check_cap_usd?: number; carry?: import('./autoplan-carry').CarryRecord;
/** `next --ceo-only` (lib/autoplan-ceo-only.ts): the CEO phase alone, no gate, ends `reviewed`; never set by a scope marker. */
ceo_only?: boolean;
}
export const TERMINAL_VOICE_STATUSES = ['completed', 'disabled', 'unavailable', 'failed', 'skipped'] as const;
lib/dx-error-contract.ts
+4 -4
View File
@@ -14,9 +14,9 @@ export const DX_ERROR_ITEMS = ['what', 'fix', 'recovery', 'code', 'context', 'tr
export const DX_ERROR_DOC = 'https://github.com/garrytan/gstack/blob/main/docs/agent-ready-errors.md';
export const DX_ERROR_COMPACT =
`Score every error path against the agent-ready error contract (${DX_ERROR_DOC}): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or the precise thing a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). ` +
`Score every error path against the agent-ready error contract (${DX_ERROR_DOC}): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or what a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). ` +
'Usage and validation errors rejected before any work need only what, fix, code, redaction and channel; items apply per surface, N/A needs a reason, an item the plan never specifies for a path it introduces is a gap, and unknown is reserved for existing behavior you could not inspect. ' +
'Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure that names no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item>" (caps the Error Messages score at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item>" (caps it at 6/10).';
'Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure naming no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item> <what fails>" (caps Error Messages at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item> <what fails>" (caps it at 6/10).';
export const DX_ERROR_CONTRACT = `#### The agent-ready error contract (${DX_ERROR_RUBRIC})
@@ -53,7 +53,7 @@ unknown is reserved for existing behavior you could not inspect. In a live audit
you never triggered are unknown, not extrapolated.
**Caps.** Name the path and item behind every cap and show the corrected error line.
- **4/10 (Critical finding, title \`CAP4:<path>:<item>\`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as \`--yes\`).
- **6/10 (High finding, title \`CAP6:<path>:<item>\`):** a path lacks an applicable fix, recovery, context or trace item.
- **4/10 (Critical finding, title \`CAP4:<path>:<item> <what fails>\`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as \`--yes\`).
- **6/10 (High finding, title \`CAP6:<path>:<item> <what fails>\`):** a path lacks an applicable fix, recovery, context or trace item.
Every mode, DX TRIAGE included, reports every CAP4 finding.`;
lib/eval-plan.ts
+50 -21
View File
@@ -366,15 +366,24 @@ function renderReservationCheck(c: NonNullable<DryRunResult['checks']['reservati
// ---------------------------------------------------------------------------
export const PILOT_FILES = { spend: 'spend.json', pilot: 'pilot.json' } as const;
export interface PilotAttempt { attempt: string; model: string; row: number; estimate_usd: number }
export interface PilotRecord { schema_version: 1; n: number; arms: string[]; cap_usd: number; dry_run: string; attempts: PilotAttempt[]; started_at: string; status: 'reserved' | 'budget_exhausted' }
export interface PilotStartOptions { out: string; cap: number; n: number; arms: string[]; dryRun: string; tokens?: TokenEstimate; now?: Date }
export interface PilotRecord { schema_version: 1; n: number; arms: string[]; seat_metered?: string[]; cap_usd: number; dry_run: string; attempts: PilotAttempt[]; started_at: string; status: 'reserved' | 'budget_exhausted' }
export interface PilotStartOptions { out: string; cap: number; n: number; arms: string[]; seatMetered?: string[]; dryRun: string; tokens?: TokenEstimate; now?: Date }
/** Reserve n attempts per arm under the cap; a refused reservation ends the pilot `budget_exhausted` with nothing else reserved. */
/**
* Reserve n attempts per arm under the cap; a refused reservation ends the pilot `budget_exhausted` with nothing else reserved.
* A seat-metered arm (operator-declared: a subscription call with no incremental charge) reserves $0 and stays outside the projection.
*/
export function startPilot(o: PilotStartOptions): { record: PilotRecord; error?: { code: ResultCodeName; message: string } } {
const dry = JSON.parse(fs.readFileSync(o.dryRun, 'utf8')) as DryRunResult;
if (!dry.pass) return { record: null as unknown as PilotRecord, error: { code: 'EVAL_DRY_RUN_FAILED', message: `${o.dryRun} did not pass; the pilot never starts on a failed dry run` } };
const tokens = o.tokens ?? { input: dry.rows ? Math.ceil(dry.est_input_tokens / dry.rows) : DEFAULT_TOKENS.input, output: dry.checks.output_budget.budget };
const record: PilotRecord = { schema_version: 1, n: o.n, arms: o.arms, cap_usd: o.cap, dry_run: o.dryRun, attempts: [], started_at: (o.now ?? new Date()).toISOString(), status: 'reserved' };
const seat = o.seatMetered ?? [];
const stray = seat.filter(a => !o.arms.includes(a));
if (stray.length) throw new Error(`--seat-metered ${stray.join(',')} needs to name arms listed in --arms`);
const prior = path.join(o.out, PILOT_FILES.pilot);
const priorSeat = fs.existsSync(prior) ? (JSON.parse(fs.readFileSync(prior, 'utf8')) as PilotRecord).seat_metered ?? [] : seat;
if ([...priorSeat].sort().join(',') !== [...seat].sort().join(',')) throw new Error(`${o.out} holds a pilot started with --seat-metered ${priorSeat.join(',') || '(none)'}; seat-metered arms are fixed when a pilot starts, so this one needs a new --out`);
const record: PilotRecord = { schema_version: 1, n: o.n, arms: o.arms, ...(seat.length ? { seat_metered: seat } : {}), cap_usd: o.cap, dry_run: o.dryRun, attempts: [], started_at: (o.now ?? new Date()).toISOString(), status: 'reserved' };
const keyOf = (a: string) => `${armModel(a).replace(/[^A-Za-z0-9.-]/g, '_')}${armLoadout(a) !== null ? `.${armLoadout(a)!.replace(/[^A-Za-z0-9.-]/g, '_')}` : ''}`;
const keys = o.arms.map(keyOf);
const duplicates = [...new Set(o.arms.filter((_, i) => keys.indexOf(keys[i]!) !== i).map(a => o.arms.filter(b => keyOf(b) === keyOf(a)).join(' = ')))];
@@ -383,11 +392,12 @@ export function startPilot(o: PilotStartOptions): { record: PilotRecord; error?:
const ledgerFile = path.join(o.out, PILOT_FILES.spend);
let error: { code: ResultCodeName; message: string } | undefined;
outer: for (const model of o.arms) {
const seated = seat.includes(model);
if (!hasPricing(armModel(model))) { error = { code: 'EVAL_PRICE_MISSING', message: `${armModel(model)} has no row in lib/pricing.ts` }; break; }
for (let i = 1; i <= o.n; i++) {
const estimate = estimateCostUsd(tokens, armModel(model));
const estimate = seated ? 0 : estimateCostUsd(tokens, armModel(model));
const attempt = `pilot-${keyOf(model)}-${i}`;
try { reserve(ledgerFile, attempt, estimate, { cap: o.cap, label: `pilot ${model} #${i}`, now: o.now }); }
try { reserve(ledgerFile, attempt, estimate, { cap: o.cap, label: `pilot ${model} #${i}${seated ? ' (seat-metered)' : ''}`, now: o.now }); }
catch (e) {
if (e instanceof SpendCapExceeded) { record.status = 'budget_exhausted'; error = { code: 'EVAL_BUDGET_EXHAUSTED', message: e.message }; break outer; }
throw e;
@@ -408,31 +418,44 @@ export function releasePilot(out: string, attempt: string): SpendLedger {
/** Two-sided 95% Student t for n-1 degrees of freedom (n settled items); 2.0 past the table. */
const T95: Record<number, number> = { 2: 12.706, 3: 4.303, 4: 3.182, 5: 2.776, 6: 2.571, 7: 2.447, 8: 2.365, 9: 2.306, 10: 2.262 };
export interface Projection { n: number; per_item_usd: number | null; sd_usd: number | null; total: number; projected_usd: number | null; band_usd: [number, number] | null; verdict: 'within' | 'exceeds' | 'unknown' }
export interface ArmProjection { arm: string; n: number; per_item_usd: number | null; sd_usd: number | null; projected_usd: number | null; band_usd: [number, number] | null }
export interface Projection { n: number; per_item_usd: number | null; sd_usd: number | null; total: number; projected_usd: number | null; band_usd: [number, number] | null; verdict: 'within' | 'exceeds' | 'unknown'; arms: ArmProjection[] }
export interface SpendTable { summary: SpendSummary; workers: number; max_estimate_usd: number; worst_case_usd: number; worst_case_overrun_usd: number }
export interface PilotReport { record: PilotRecord; spend: SpendTable; projection: Projection; settled_known: number; settled_unknown: number }
export interface PilotReport { record: PilotRecord; spend: SpendTable; projection: Projection; settled_known: number; settled_unknown: number; seat_metered: Array<{ arm: string; calls: number; settled: number }> }
export function pilotReport(out: string, o: { total: number; workers?: number }): PilotReport {
const record = JSON.parse(fs.readFileSync(path.join(out, PILOT_FILES.pilot), 'utf8')) as PilotRecord;
const ledger = readLedger(path.join(out, PILOT_FILES.spend), record.cap_usd);
const summary = summarize(ledger);
const known = ledger.settled.filter(s => typeof s.usd === 'number').map(s => s.usd as number);
const unknown = ledger.settled.length - known.length;
const armOf = new Map(record.attempts.map(a => [a.attempt, a.model]));
const charged = new Set(ledger.settled.filter(s => typeof s.usd === 'number' && s.usd > 0).map(s => armOf.get(s.attempt)));
const seat = new Set((record.seat_metered ?? []).filter(arm => !charged.has(arm)));
const unknown = ledger.settled.filter(s => s.usd === 'unknown' && !seat.has(armOf.get(s.attempt) ?? '')).length;
const workers = Math.max(1, o.workers ?? 1);
const maxEstimate = Math.max(0, ...record.attempts.map(a => a.estimate_usd));
const worst = summary.worst_case_usd + workers * maxEstimate;
const n = known.length;
const mean = n ? known.reduce((s, x) => s + x, 0) / n : null;
const sd = n > 1 && mean !== null ? Math.sqrt(known.reduce((s, x) => s + (x - mean) ** 2, 0) / (n - 1)) : null;
const perArmTotal = o.total * Math.max(1, record.arms.length);
const projected = mean === null ? null : mean * perArmTotal;
const half = sd !== null && mean !== null ? (T95[n] ?? 2.0) * (sd / Math.sqrt(n)) * perArmTotal : null;
const band: [number, number] | null = projected !== null && half !== null ? [Math.max(0, projected - half), projected + half] : null;
const verdict: Projection['verdict'] = projected === null || band === null || unknown > 0 ? 'unknown' : band[1] + summary.worst_case_usd <= record.cap_usd ? 'within' : 'exceeds';
const arms: ArmProjection[] = record.arms.filter(arm => !seat.has(arm)).map(arm => {
const xs = ledger.settled.filter(s => armOf.get(s.attempt) === arm && typeof s.usd === 'number').map(s => s.usd as number);
const n = xs.length;
const mean = n ? xs.reduce((s, x) => s + x, 0) / n : null;
const sd = n > 1 && mean !== null ? Math.sqrt(xs.reduce((s, x) => s + (x - mean) ** 2, 0) / (n - 1)) : null;
const projected = mean === null ? null : mean * o.total;
const half = sd !== null ? (T95[n] ?? 2.0) * (sd / Math.sqrt(n)) * o.total : null;
return { arm, n, per_item_usd: mean, sd_usd: sd, projected_usd: projected, band_usd: projected !== null && half !== null ? [Math.max(0, projected - half), projected + half] : null };
});
const priced = arms.filter(a => a.per_item_usd !== null);
const n = priced.reduce((s, a) => s + a.n, 0);
const mean = priced.length || !arms.length ? priced.reduce((s, a) => s + a.per_item_usd!, 0) : null;
const sd = mean !== null && priced.every(a => a.sd_usd !== null) ? priced.reduce((s, a) => s + a.sd_usd!, 0) : null;
const projected = mean === null ? null : mean * o.total;
const band: [number, number] | null = projected !== null && priced.every(a => a.band_usd)
? [priced.reduce((s, a) => s + a.band_usd![0], 0), priced.reduce((s, a) => s + a.band_usd![1], 0)] : null;
const verdict: Projection['verdict'] = band === null || unknown > 0 || priced.length < arms.length ? 'unknown' : band[1] + summary.worst_case_usd <= record.cap_usd ? 'within' : 'exceeds';
return {
record, settled_known: n, settled_unknown: unknown,
spend: { summary, workers, max_estimate_usd: maxEstimate, worst_case_usd: worst, worst_case_overrun_usd: Math.max(0, worst - record.cap_usd) },
projection: { n, per_item_usd: mean, sd_usd: sd, total: o.total, projected_usd: projected, band_usd: band, verdict },
projection: { n, per_item_usd: mean, sd_usd: sd, total: o.total, projected_usd: projected, band_usd: band, verdict, arms },
seat_metered: [...seat].map(arm => ({ arm, calls: record.attempts.filter(a => a.model === arm).length, settled: ledger.settled.filter(s => armOf.get(s.attempt) === arm).length })),
};
}
@@ -449,11 +472,17 @@ export function renderSpendTable(t: SpendTable, cap: number): string {
export function renderPilotReport(r: PilotReport): string {
const p = r.projection;
const band = (b: [number, number] | null) => (b ? `[${usd(b[0])}, ${usd(b[1])}]` : 'unknown');
const notReady = r.settled_unknown > 0
? 'unknown charges make the band meaningless; settle them with actual cost, or start a new pilot in a new `--out` with `pilot start --seat-metered <arm>` for an arm on a subscription with no incremental charge'
: 'settle at least 2 attempts per priced arm with actual cost';
return [
`PILOT: n=${r.record.n} arms=${r.record.arms.join(',')} settled=${r.settled_known} unknown=${r.settled_unknown} status=${r.record.status}`,
renderSpendTable(r.spend, r.record.cap_usd),
`PILOT_PROJECTION: items=${p.total} per_item=${usd(p.per_item_usd)} sd=${usd(p.sd_usd)} projected=${usd(p.projected_usd)} band=${p.band_usd ? `[${usd(p.band_usd[0])}, ${usd(p.band_usd[1])}]` : 'unknown'} cap=$${r.record.cap_usd.toFixed(2)} verdict=${p.verdict}`,
p.verdict === 'unknown' ? `APPROVAL: not ready — ${r.settled_known < 2 ? 'settle at least 2 pilot attempts with actual cost' : 'unknown charges make the band meaningless'}; settle, then rerun report` : `APPROVAL: run ${p.total} items at projected ${usd(p.projected_usd)} (95% band ${usd(p.band_usd![0])}–${usd(p.band_usd![1])}) under cap $${r.record.cap_usd.toFixed(2)}: ${p.verdict === 'within' ? 'fits; ask the owner before any further priced call' : 'EXCEEDS the cap; stop, raise the cap on purpose or cut arms or items'}`,
...p.arms.map(a => `PILOT_ARM: arm=${a.arm} n=${a.n} per_item=${usd(a.per_item_usd)} sd=${usd(a.sd_usd)} projected=${usd(a.projected_usd)} band=${band(a.band_usd)}`),
...r.seat_metered.map(s => `PILOT_SEAT_METERED: arm=${s.arm} calls=${s.calls} settled=${s.settled} (declared seat-metered: no incremental charge, outside the cap and the projection; usage-limited)`),
`PILOT_PROJECTION: items=${p.total} known_arms=${p.arms.filter(a => a.per_item_usd !== null).length}/${p.arms.length} per_item=${usd(p.per_item_usd)} sd=${usd(p.sd_usd)} projected=${usd(p.projected_usd)} band=${band(p.band_usd)} cap=$${r.record.cap_usd.toFixed(2)} verdict=${p.verdict}`,
p.verdict === 'unknown' ? `APPROVAL: not ready — ${notReady}; then rerun report` : `APPROVAL: run ${p.total} items at projected ${usd(p.projected_usd)} (95% band ${usd(p.band_usd![0])}–${usd(p.band_usd![1])}) under cap $${r.record.cap_usd.toFixed(2)}: ${p.verdict === 'within' ? 'fits; ask the owner before any further priced call' : 'EXCEEDS the cap; stop, raise the cap on purpose or cut arms or items'}`,
].join('\n') + '\n';
}
lib/factory-scorecard.ts
+31 -13
View File
@@ -25,7 +25,8 @@
* flaky events flake-records-a<n> artifacts of gate runs (paid, one record per case
* that failed once and passed its one rerun) and flake-ledger-* artifacts
* of <free workflow> runs on <base> (free-suite serial retries); the same
* two sources the daily flaky pass reads
* two sources the daily flaky pass reads; a main push that reused its PR's
* run (no artifacts, `reuse` job green; reuseChecker) is left out
* issues search/issues created / closed inside the window
* releases releases published inside the window (drafts excluded)
*
@@ -36,13 +37,13 @@ import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import {
GH_CLIENT, createdFilter, downloadRunArtifacts, filesUnder, historyFetcher, listWorkflowRuns, readFlakeLedgerDir, unavailableReason,
GH_CLIENT, createdFilter, downloadRunArtifacts, filesUnder, historyFetcher, listWorkflowRuns, readFlakeLedgerDir, reuseChecker, unavailableReason,
type GhClient,
} from '../scripts/lib/ci-history';
export interface ScorecardSource {
getJson(apiPath: string): unknown;
/** Downloaded, unzipped artifact directories of one run whose names match, keyed by artifact name; one listing per call. */
/** Downloaded, unzipped artifact directories of one run whose names match, keyed by artifact name; one listing per call, and `match` sees every name in it. */
artifactDirs(runId: number, match: (name: string) => boolean): Map<string, string>;
}
@@ -93,6 +94,8 @@ export interface FreeRetryRun { id: number; conclusion: string | null; retries:
export interface Scorecard {
repo: string; base: string; windowStart: string; windowEnd: string;
prs: MergedPr[]; gates: GateRun[]; freeRuns: FreeRetryRun[]; artifactRunsRead: number; passCap: number;
/** Reused main pushes left out of freeRuns (reuseChecker), and the note naming runs it could not check. */
freeRunsSkipped: number; freeRunsNote: string | null;
/** Gate runs whose artifacts could not be read; they count without cost or flake data. */
artifactErrors: number;
issuesOpened: number | null; issuesClosed: number | null; releases: Array<{ tag: string; publishedAt: string }>;
@@ -308,14 +311,29 @@ export function readGateRuns(source: ScorecardSource, o: ScorecardOptions, w: Sc
return { gates, artifactRunsRead, artifactErrors };
}
/** Free-suite runs on base in the window and the flaky-pass entries their flake-ledger-* artifacts hold. */
export function readFreeRetries(source: ScorecardSource, o: ScorecardOptions, w: ScoreWindow): FreeRetryRun[] {
/**
* Free-suite runs on base in the window and the flaky-pass entries their
* flake-ledger-* artifacts hold. A read run that reused its pull request's run
* (reuseChecker) is left out and counted in freeRunsSkipped; an unreadable run
* and a run past maxArtifactRuns are kept, as before.
*/
export function readFreeRetries(source: ScorecardSource, o: ScorecardOptions, w: ScoreWindow): Pick<Scorecard, 'freeRuns' | 'freeRunsSkipped' | 'freeRunsNote'> {
const client: GhClient = { getJson: p => source.getJson(p), downloadZip: () => { throw new Error('not used'); } };
const runs = listWorkflowRuns(client, o.repo, o.freeWorkflow, { since: w.start, until: w.end, branch: o.base });
const retries = (runId: number): number | null => {
try { return [...source.artifactDirs(runId, name => name.startsWith('flake-ledger-')).values()].flatMap(readFlakeLedgerDir).length; } catch { return null; }
};
return runs.map((run, index) => ({ id: run.id, conclusion: run.conclusion, retries: index < o.maxArtifactRuns ? retries(run.id) : 0 }));
const reuse = reuseChecker(client, o.repo);
const freeRuns: FreeRetryRun[] = [];
let freeRunsSkipped = 0;
runs.forEach((run, index) => {
if (index >= o.maxArtifactRuns) { freeRuns.push({ id: run.id, conclusion: run.conclusion, retries: 0 }); return; }
const names: string[] = [];
let retries: number | null;
try {
retries = [...source.artifactDirs(run.id, name => { names.push(name); return name.startsWith('flake-ledger-'); }).values()].flatMap(readFlakeLedgerDir).length;
} catch { retries = null; }
if (retries !== null && reuse.reused(run, names)) { freeRunsSkipped++; return; }
freeRuns.push({ id: run.id, conclusion: run.conclusion, retries });
});
return { freeRuns, freeRunsSkipped, freeRunsNote: reuse.note() ?? null };
}
function searchCount(source: ScorecardSource, q: string): number | null {
@@ -340,14 +358,14 @@ export function readReleases(source: ScorecardSource, repo: string, w: ScoreWind
export function collectScorecard(source: ScorecardSource, o: ScorecardOptions): Scorecard {
const w = scoreWindow(o);
const card: Scorecard = { repo: o.repo, base: o.base, windowStart: w.start, windowEnd: w.end,
prs: [], gates: [], freeRuns: [], artifactRunsRead: 0, passCap: o.passCap, artifactErrors: 0, issuesOpened: null, issuesClosed: null, releases: [], unavailable: [] };
prs: [], gates: [], freeRuns: [], freeRunsSkipped: 0, freeRunsNote: null, artifactRunsRead: 0, passCap: o.passCap, artifactErrors: 0, issuesOpened: null, issuesClosed: null, releases: [], unavailable: [] };
const guard = (what: string, fn: () => void) => { try { fn(); } catch (e) { card.unavailable.push(`${what}: ${unavailableReason(e)}`); } };
guard('merged PRs', () => { card.prs = readMergedPrs(source, o, w); });
guard('gate runs', () => {
const prCost = new Map(card.prs.filter(p => p.commentCostUsd !== null).map(p => [p.number, p.commentCostUsd!] as const));
Object.assign(card, readGateRuns(source, o, w, prCost));
});
guard('free retries', () => { card.freeRuns = readFreeRetries(source, o, w); });
guard('free retries', () => { Object.assign(card, readFreeRetries(source, o, w)); });
guard('issues', () => {
card.issuesOpened = searchCount(source, `repo:${o.repo} is:issue created:${searchRange(w)}`);
card.issuesClosed = searchCount(source, `repo:${o.repo} is:issue closed:${searchRange(w)}`);
@@ -412,7 +430,7 @@ export function scorecardRows(card: Scorecard): ScorecardRow[] {
return { id, metric, value: failed ? unavailableText(failed) : value(), benchmark: BENCHMARKS[id]!, rule };
};
const paidFlakes = () => `${flakes} paid case(s) passed on their one rerun (rerun spend ${usd(flakeUsd)})${card.artifactErrors ? `; artifacts unreadable on ${card.artifactErrors} run(s)` : ''}`;
const freeRetries = () => `${retries} free-suite retr${retries === 1 ? 'y' : 'ies'} in ${retryRuns} of ${card.freeRuns.length} runs${unreadableFree ? ` (${unreadableFree} unreadable)` : ''}`;
const freeRetries = () => `${retries} free-suite retr${retries === 1 ? 'y' : 'ies'} in ${retryRuns} of ${card.freeRuns.length} runs${unreadableFree ? ` (${unreadableFree} unreadable)` : ''}${card.freeRunsSkipped ? `; ${card.freeRunsSkipped} reused main push(es) left out` : ''}${card.freeRunsNote ? `; ${card.freeRunsNote}` : ''}`;
return [
row('merged', 'Merged PRs', 'merged PRs', () => `${card.prs.length} (crew ${crewPrs.length}, outside ${card.prs.length - crewPrs.length})`,
`merged_at inside the window, base ${card.base}; crew by login list or author_association OWNER/MEMBER/COLLABORATOR`),
@@ -429,7 +447,7 @@ export function scorecardRows(card: Scorecard): ScorecardRow[] {
row('red_train', 'Red-train rate', 'gate runs', () => `${pct(trains.filter(t => t.outcome === 'red').length, trains.length)} (${trains.filter(t => t.outcome === 'red').length} red of ${trains.length} trains; ${by('red', 'backstop')} red backstop, ${by('red', 'screen')} red screens)`,
`runs whose head branch starts with the train prefix, cancelled runs excluded; red means the run's conclusion was not success`),
row('flaky', 'Flaky events', [], () => `${unavailable('gate runs') ? unavailableText('gate runs') : paidFlakes()}; ${unavailable('free retries') ? unavailableText('free retries') : freeRetries()}`,
`flake-records artifacts of the newest ${card.artifactRunsRead} gate runs and flake-ledger artifacts of free runs on the base branch`),
`flake-records artifacts of the newest ${card.artifactRunsRead} gate runs and flake-ledger artifacts of free runs on the base branch, reused main pushes left out`),
row('issues', 'Issues opened / closed', 'issues', () => `${card.issuesOpened ?? 'n/a'} / ${card.issuesClosed ?? 'n/a'}`, 'GitHub search: created, closed inside the window'),
row('releases', 'Releases cut', 'releases', () => `${card.releases.length}${card.releases.length ? ` (${card.releases.map(r => r.tag).join(', ')})` : ''}`, 'releases published inside the window, drafts excluded'),
];
lib/gate-list.ts
+19 -4
View File
@@ -2,7 +2,9 @@
* gate-list — one gate list instead of four-option sequences (plan B9).
* A gate list is numbered items, each with a stable id, a recommended default
* and lettered options; `auto` items take their recommendation when
* unanswered, `approval` items stay pending until answered. One reply
* unanswered (one whose recommendation changes the plan keeps the plan's
* direction instead, unless the reply approves with the recommended changes,
* `p1b`), `approval` items stay pending until answered. One reply
* grammar: `rec` (every item takes its recommendation, plan changes
* included), `<id><option>` tokens (`d3b uc1a`), or `rec except <tokens>`
* (the same operation as listing the overrides). `all`, `approve all`,
@@ -39,6 +41,8 @@ export interface GateItem {
carried?: { from: string; chosen: string; gate_rev: number };
/** A round-cap item (lib/autoplan-budget.ts): `b` grants the phase one more round and reopens it. */
cap?: { phase: string; kind: 'deferred' | 'blocker' | 'delta'; ref?: string };
/** An open Critical or High finding (its ids, a paired twin included): `a` applies the reviewer's fix to the plan. */
finding?: string[];
}
export interface GateList { schema_version: 1; run: string; gate_rev: number; items: GateItem[]; title?: string;
/** Bumped by every `recommend`, so a `rec` relayed before it no longer matches `recHash`. */
@@ -196,6 +200,7 @@ export function parseReply(reply: string, list: GateList, gateRev?: number, prio
const choices: GateChoice[] = [];
const pending: string[] = [];
const how = (item: GateItem, chosen: string) => (chosen === item.recommended ? 'recommended' : 'override') as GateChoice['how'];
const withChanges = overrides.get('p1') === 'b';
for (const item of list.items) {
const chosen = overrides.get(item.id);
const earlier = prior[item.id] ?? item.carried?.chosen;
@@ -203,7 +208,7 @@ export function parseReply(reply: string, list: GateList, gateRev?: number, prio
else if (earlier !== undefined && item.options.some(o => o.key === earlier)) choices.push({ id: item.id, chosen: earlier, kind: item.kind, how: how(item, earlier), prior: true });
else if (asIs) { const keep = keepDirection(item); choices.push({ id: item.id, chosen: keep, kind: item.kind, how: how(item, keep) }); }
else if (rec) choices.push({ id: item.id, chosen: item.recommended, kind: item.kind, how: 'recommended' });
else if (item.kind === 'auto') choices.push({ id: item.id, chosen: item.recommended, kind: item.kind, how: 'auto_default' });
else if (item.kind === 'auto') choices.push({ id: item.id, chosen: withChanges ? item.recommended : autoDefault(item), kind: item.kind, how: 'auto_default' });
else pending.push(item.id);
}
return { ...base, valid: true, choices, pending, mode: rec ? 'rec' : asIs ? 'as-is' : 'tokens' };
@@ -214,6 +219,11 @@ export function keepDirection(item: GateItem): string {
return item.options.find(o => o.changes_plan === false)?.key ?? (item.id === 'p1' && item.options.some(o => o.key === 'a') ? 'a' : item.recommended);
}
/** What an unanswered `auto` item resolves to: its recommendation, unless that changes the plan (then the plan's direction). */
export function autoDefault(item: GateItem): string {
return item.options.find(o => o.key === item.recommended)?.changes_plan ? keepDirection(item) : item.recommended;
}
/** The decisions.jsonl id for a gate item. Item ids restart at every gate revision, so revisions after the first carry `r<rev>-`. */
export function decisionId(run: string, gateRev: number, itemId: string): string {
return gateRev > 1 ? `${run}-r${gateRev}-${itemId}` : `${run}-${itemId}`;
@@ -224,7 +234,7 @@ export function decisionRows(list: GateList, extra: Partial<DecisionRow> = {}):
return list.items.map(item => ({
schema_version: 1, run: list.run, id: decisionId(list.run, list.gate_rev, item.id), label: item.id.toUpperCase(), title: item.title,
options: item.options.map(o => o.key), recommended: item.recommended, kind: item.kind, status: 'pending',
default_if_unanswered: item.kind === 'auto' ? item.recommended : null, gate_rev: list.gate_rev,
default_if_unanswered: item.kind === 'auto' ? autoDefault(item) : null, gate_rev: list.gate_rev,
...(item.phase ? { phase: item.phase } : {}), ...(item.cost ? { cost: item.cost } : {}),
...(item.recommended_basis ? { recommended_basis: item.recommended_basis } : {}), ...(item.changes ? { changes: item.changes } : {}),
...(item.origin ? { origin: item.origin } : {}), ...(item.reopens === false ? { source: 'agent' } : {}), ...extra,
@@ -234,14 +244,19 @@ export function decisionRows(list: GateList, extra: Partial<DecisionRow> = {}):
/** `--by` values: the owner typing tokens, or an agent relaying or deciding them. */
export const ANSWERED_BY_RE = /^(human|agent:[A-Za-z0-9_.:-]+)$/;
/** Who answered and in whose words: a new answer replaces all of it, so an earlier answerer's fields never survive under a later one. */
const PROVENANCE = ['answered_by', 'owner_reply', 'owner_reply_parsed', 'answer_words', 'answer_words_parsed', 'words_by', 'carried_gate_from', 'carried_gate_rev', 'carry_reason', 'carried_answered_by'] as const;
export function applyReply(rows: DecisionRow[], parsed: ParsedReply, answeredAt: string, answeredBy?: string, extra: Partial<DecisionRow> = {}): DecisionRow[] {
if (!parsed.valid) return rows;
const byId = new Map(parsed.choices.map(c => [c.id, c]));
return rows.map(row => {
const choice = byId.get(row.label?.toLowerCase() ?? '') ?? byId.get(row.id.slice(row.run.length + 1).replace(/^r\d+-/, ''));
if (!choice || choice.prior || row.gate_rev !== parsed.gate_rev) return row;
const fresh = { ...row };
for (const k of PROVENANCE) delete (fresh as Record<string, unknown>)[k];
return {
...row, chosen: choice.chosen, answered_at: answeredAt, reply: parsed.reply, ...(answeredBy ? { answered_by: answeredBy } : {}), ...extra,
...fresh, chosen: choice.chosen, answered_at: answeredAt, reply: parsed.reply, ...(answeredBy ? { answered_by: answeredBy } : {}), ...extra,
status: choice.how === 'override' ? 'overridden' : 'approved',
};
});
lib/gate-outcomes.ts
+5
View File
@@ -48,6 +48,11 @@ export const GATE_OUTCOMES = {
summary: 'the reviewer declined to review',
fix: 're-run the review; if it declines again, review the change natively',
},
truncated: {
state: 'unavailable', anchor: 'outside-review-truncated',
summary: 'the reviewer hit its output token limit and the response is cut off',
fix: 're-run with a larger --max-tokens (or a smaller prompt); a cut-off response never counts as a review',
},
missing_markers: {
state: 'unavailable', anchor: 'outside-review-missing-markers',
summary: 'the response lacks the markers this gate requires',
lib/headless-artifacts.ts
+8 -6
View File
@@ -20,10 +20,10 @@ export type Severity = 'Critical' | 'High' | 'Medium' | 'Low' | 'Informational';
export type Disposition = 'open' | 'accepted' | 'partially_accepted' | 'rejected' | 'deferred' | 'fixed';
export type Voice = 'native' | 'outside';
export type DecisionKind = 'auto' | 'approval' | 'user_challenge';
export type DecisionStatus = 'pending' | 'approved' | 'overridden' | 'rejected';
export type DecisionStatus = 'pending' | 'approved' | 'overridden' | 'rejected' | 'auto_kept';
export type TaskStatus = 'pending' | 'ready' | 'blocked_on_dependency' | 'blocked_on_decision' | 'in_progress' | 'done';
export type VoiceStatus = 'completed' | 'unavailable' | 'disabled' | 'skipped' | 'failed' | 'pending' | 'running';
export type ResultStatus = 'complete' | 'gate_pending' | 'amend_pending' | 'incomplete' | 'interrupted' | 'refused';
export type ResultStatus = 'complete' | 'reviewed' | 'gate_pending' | 'amend_pending' | 'incomplete' | 'interrupted' | 'refused';
export type RunStatus = ResultStatus | 'awaiting_result' | 'running';
export interface FindingRow {
@@ -41,6 +41,8 @@ export interface DecisionRow {
recommended_basis?: string; changes?: string; origin?: string; carried_from?: string; chosen_text?: string;
/** An agent-relayed answer: the owner's verbatim words, and whether they parsed under the gate grammar. */
owner_reply?: string; owner_reply_parsed?: boolean; recommended_by?: string;
/** An agent's own words (`--words-by agent:<id>`): never the owner's, so never carried or imported as the owner's. */
answer_words?: string; answer_words_parsed?: boolean; words_by?: string;
/** A whole-gate carry (lib/autoplan-carry.ts): the earlier p1 decision whose owner words this answer reuses, its gate_rev, why, who relayed them, items taken under "do it", and the delta check attempt. */
carried_gate_from?: string; carried_gate_rev?: number; carry_reason?: string; carried_answered_by?: string; carried_under?: string; carry_check?: string;
}
@@ -119,8 +121,8 @@ export interface ShipReceipt {
base_red?: { cases: string[]; files: string[]; base: string; lane: string; expires_at: string };
}
export const RESULT_STATUSES: readonly ResultStatus[] = ['complete', 'gate_pending', 'amend_pending', 'incomplete', 'interrupted', 'refused'];
const TERMINAL_RUN: readonly string[] = ['complete', 'gate_pending', 'amend_pending', 'refused', 'incomplete'];
export const RESULT_STATUSES: readonly ResultStatus[] = ['complete', 'reviewed', 'gate_pending', 'amend_pending', 'incomplete', 'interrupted', 'refused'];
const TERMINAL_RUN: readonly string[] = ['complete', 'reviewed', 'gate_pending', 'amend_pending', 'refused', 'incomplete'];
const TERMINAL_VOICE: readonly string[] = ['completed', 'unavailable', 'disabled', 'skipped', 'failed'];
const RESOLVED: readonly string[] = ['accepted', 'partially_accepted', 'rejected', 'deferred', 'fixed'];
@@ -174,7 +176,7 @@ export const ARTIFACT_SCHEMAS = {
required: ['schema_version', 'run', 'id', 'title', 'options', 'recommended', 'kind', 'status', 'gate_rev'],
properties: {
schema_version: { const: 1 }, run: str, id: str, label: str, title: str, options: strList, recommended: nullableStr,
kind: { enum: ['auto', 'approval', 'user_challenge'] }, status: { enum: ['pending', 'approved', 'overridden', 'rejected'] },
kind: { enum: ['auto', 'approval', 'user_challenge'] }, status: { enum: ['pending', 'approved', 'overridden', 'rejected', 'auto_kept'] },
default_if_unanswered: nullableStr, gate_rev: { type: 'integer', minimum: 1 }, phase: str, chosen: nullableStr,
answered_at: str, reply: str, source: str, cost: str,
recommended_basis: str, changes: str, origin: str, carried_from: str, chosen_text: str,
@@ -205,7 +207,7 @@ export const ARTIFACT_SCHEMAS = {
required: ['schema_version', 'run', 'status', 'required_phases', 'phases', 'artifacts'],
properties: {
schema_version: { const: 1 }, run: str,
status: { enum: ['complete', 'gate_pending', 'amend_pending', 'incomplete', 'interrupted', 'refused', 'awaiting_result', 'running'] },
status: { enum: ['complete', 'reviewed', 'gate_pending', 'amend_pending', 'incomplete', 'interrupted', 'refused', 'awaiting_result', 'running'] },
base: str, host: str, session_kind: str, required_phases: strList,
phases: { type: 'object', description: 'phase id → { snapshot, native, outside }', properties: {} },
counts: { type: 'object', description: 'decisions, decisions_pending, findings, tasks: each must equal the rows in its file' },
lib/outside-voice-api.ts
+11 -5
View File
@@ -13,7 +13,13 @@ import { modelFamily, type ModelFamily } from './outside-voice-runner';
export interface ApiCallRequest { model: string; prompt: string; timeoutMs: number; env?: Record<string, string | undefined>; signal?: AbortSignal; maxTokens?: number }
export interface ApiUsage { input_tokens?: number; output_tokens?: number }
export interface ApiCallResult { text: string; model: string; usage: ApiUsage; provider: 'openai' | 'anthropic'; status: number }
/** `truncated`: the provider stopped at the output token limit (Anthropic `max_tokens`, OpenAI `length`), so the text is cut off. */
export interface ApiCallResult { text: string; model: string; usage: ApiUsage; provider: 'openai' | 'anthropic'; status: number; truncated: boolean }
/** Anthropic requires max_tokens; a full review needs more than 8192. `run` sends this to OpenAI too, where reasoning tokens count toward it. */
export const DEFAULT_MAX_TOKENS = 16_384;
/** The api runner's default time limit: five minutes, or about 40 ms per output token the call may write. */
export const apiTimeoutMs = (maxTokens?: number): number => Math.max(300_000, (maxTokens ?? DEFAULT_MAX_TOKENS) * 40);
export const API_FAMILIES: readonly ModelFamily[] = ['openai', 'anthropic'];
@@ -57,17 +63,17 @@ export async function callModel(req: ApiCallRequest): Promise<ApiCallResult> {
const model = req.model.includes('/') ? req.model.slice(req.model.indexOf('/') + 1) : req.model;
if (auth.provider === 'anthropic') {
const base = (env.ANTHROPIC_BASE_URL ?? 'https://api.anthropic.com').replace(/\/$/, '');
const body = JSON.stringify({ model, max_tokens: req.maxTokens ?? 8192, messages: [{ role: 'user', content: req.prompt }] });
const body = JSON.stringify({ model, max_tokens: req.maxTokens ?? DEFAULT_MAX_TOKENS, messages: [{ role: 'user', content: req.prompt }] });
const { status, json } = await post(`${base}/v1/messages`, { 'x-api-key': auth.key, 'anthropic-version': '2023-06-01' }, body, req, 'anthropic');
if (status < 200 || status >= 300) throw new Error(`anthropic ${status}: ${json?.error?.message ?? json?.raw ?? JSON.stringify(json)}`);
const text = Array.isArray(json?.content) ? json.content.filter((c: any) => c?.type === 'text').map((c: any) => c.text).join('\n') : '';
return { text, model: typeof json?.model === 'string' ? json.model : model, usage: { input_tokens: json?.usage?.input_tokens, output_tokens: json?.usage?.output_tokens }, provider: 'anthropic', status };
return { text, model: typeof json?.model === 'string' ? json.model : model, usage: { input_tokens: json?.usage?.input_tokens, output_tokens: json?.usage?.output_tokens }, provider: 'anthropic', status, truncated: json?.stop_reason === 'max_tokens' };
}
const base = (env.OPENAI_BASE_URL ?? 'https://api.openai.com/v1').replace(/\/$/, '');
const body = JSON.stringify({ model, messages: [{ role: 'user', content: req.prompt }] });
const body = JSON.stringify({ model, messages: [{ role: 'user', content: req.prompt }], ...(req.maxTokens ? { max_completion_tokens: req.maxTokens } : {}) });
const { status, json } = await post(`${base}/chat/completions`, { authorization: `Bearer ${auth.key}` }, body, req, 'openai');
if (status < 200 || status >= 300) throw new Error(`openai ${status}: ${json?.error?.message ?? json?.raw ?? JSON.stringify(json)}`);
const content = json?.choices?.[0]?.message?.content;
const text = typeof content === 'string' ? content : Array.isArray(content) ? content.map((c: any) => c?.text ?? '').join('\n') : '';
return { text, model: typeof json?.model === 'string' ? json.model : model, usage: { input_tokens: json?.usage?.prompt_tokens, output_tokens: json?.usage?.completion_tokens }, provider: 'openai', status };
return { text, model: typeof json?.model === 'string' ? json.model : model, usage: { input_tokens: json?.usage?.prompt_tokens, output_tokens: json?.usage?.completion_tokens }, provider: 'openai', status, truncated: json?.choices?.[0]?.finish_reason === 'length' };
}
lib/outside-voice-runner.ts
+4 -1
View File
@@ -36,6 +36,8 @@ export interface RunnerRequest {
promptFile: string; outFile: string; model?: string; timeoutMs: number; cwd: string; gate: OutsideGate;
/** host-subagent: the file the host's subagent wrote. */
resultFile?: string;
/** api: the output token limit (lib/outside-voice-api.ts DEFAULT_MAX_TOKENS for Anthropic when unset). */
maxTokens?: number;
env?: Record<string, string | undefined>;
/** Directory of the gstack bins (gstack-codex-probe); defaults to this module's sibling bin/. */
binDir?: string;
@@ -199,10 +201,11 @@ class ApiRunner implements OutsideRunner {
if (!avail.ok) return finish({ ...base, status: 'unavailable', exit: 1, reason: 'runner_unavailable', detail: avail.reason }, started);
const { callModel } = await import('./outside-voice-api');
try {
const res = await callModel({ model: req.model!, prompt, timeoutMs: req.timeoutMs, env: req.env, signal: this.controller.signal });
const res = await callModel({ model: req.model!, prompt, timeoutMs: req.timeoutMs, env: req.env, signal: this.controller.signal, maxTokens: req.maxTokens });
fs.mkdirSync(path.dirname(req.outFile), { recursive: true });
fs.writeFileSync(req.outFile, res.text);
const usage: RunnerUsage = { ...res.usage, usd: 'unknown' };
if (res.truncated) return finish({ ...base, model: res.model, family: modelFamily(res.model), status: 'unavailable', exit: 0, usage, reason: 'truncated', detail: `stopped at the output limit after ${res.usage.output_tokens ?? 'an unknown number of'} output tokens; raise --max-tokens` }, started, req.outFile);
const { status, verdict } = classify(res.text, req.gate, { exit: 0 });
return finish({ ...base, model: res.model, family: modelFamily(res.model), status, exit: 0, usage, verdict, ...(verdict.reason ? { reason: verdict.reason } : {}), ...(verdict.detail ? { detail: verdict.detail } : {}) }, started, req.outFile);
} catch (e: any) {
lib/result-codes-autoplan.ts
+26
View File
@@ -0,0 +1,26 @@
/**
* Result codes for the CEO-only driver run (`gstack-autoplan next --ceo-only`,
* lib/autoplan-ceo-only.ts). Kept beside lib/result-codes.ts, which spreads
* this table into RESULT_CODES; the same rules apply: one stable
* docs/troubleshooting.md anchor, a summary and a fix clause each. The run is
* documented in docs/unattended.md#ceo-only-runs.
*/
import type { ResultCode } from './result-codes';
export const AUTOPLAN_RESULT_CODES = {
CEO_ONLY_CONFLICT: {
anchor: 'ceo-only-conflict',
summary: '--ceo-only was combined with a flag that adds a phase (--ui, --developer-tool, --agent-primary), or a plan\'s scope marker names it',
fix: 'drop the conflicting flag, or run without --ceo-only for the full review; --ceo-only goes on the command line of the first next only, never in a gstack-autoplan-scope marker',
},
CEO_ONLY_AFTER_INIT: {
anchor: 'ceo-only-after-init',
summary: 'next --ceo-only named a run directory that already holds a full review run',
fix: 'a run keeps the mode it started with: pass --ceo-only on the first next with a new --out directory',
},
CEO_ONLY_NO_GATE: {
anchor: 'ceo-only-no-gate',
summary: 'answer, recommend, revise and approvals import need a gate or an approvable run, and a CEO-only run has neither: it ends status=reviewed and is never approved',
fix: 'read <run dir>/ceo-review.md; to get a gate and an approval, start a full review with gstack-autoplan next --out <new dir> --plan <file> (no --ceo-only)',
},
} as const satisfies Record<string, ResultCode>;
lib/result-codes-release.ts
+18 -2
View File
@@ -418,8 +418,24 @@ export const RELEASE_RESULT_CODES = {
},
QUEUE_LAND_LOCKED: {
anchor: 'queue-land-locked',
summary: 'another gstack-ship-queue land holds the repository land lock (refs/notes/gstack-land), so this one would race it for the next version',
fix: 'wait for the holder named in the line to finish; if it crashed, gstack-ship-queue land --pr <n> --break-lock (refused while its journal shows a push in flight)',
summary: 'another gstack-ship-queue land or train land slot holds the repository land lock (refs/notes/gstack-land), so this one would race it onto main',
fix: 'wait for the holder named in the line to finish; if it crashed, rerun with --break-lock (refused while it is mid-push or mid-merge, between train slots less than 10 minutes, and for a live train run without --break-train-lock)',
},
// #3531: verify --train's slot stamp proof and gstack-restamp's staging of what it writes.
QUEUE_TRAIN_UNSTAMPED: {
anchor: 'queue-train-unstamped',
summary: 'the slot head verify --train would let merge is not stamped above main: its VERSION is not above main\'s, or its CHANGELOG is not headed with that version directly above main\'s top release',
fix: 'on the lane, run gstack-restamp --next on the head, commit what restamp staged, push, then rerun gstack-ship-queue verify --pr <n> --train <nonce>',
},
RESTAMP_NOT_STAGED: {
anchor: 'restamp-not-staged',
summary: 'gstack-restamp wrote the stamp but could not stage it, so a bare git commit would leave the stamp out',
fix: 'clear what blocked git add (an index.lock), then rerun gstack-restamp: the noop rerun stages the files it wrote, or drops the marker when they changed since',
},
RESTAMP_UNSTAGED_EDITS: {
anchor: 'restamp-unstaged-edits',
summary: 'a file gstack-restamp would stamp holds unstaged edits of the caller, which staging the stamp would sweep into the stamp commit; nothing was written',
fix: 'stage (git add) or stash your own edits to the files named, then rerun gstack-restamp',
},
QUEUE_LAND_CONFLICT: {
anchor: 'queue-land-conflict',
lib/result-codes.ts
+2
View File
@@ -15,6 +15,7 @@ import { CAPY_RESULT_CODES } from './result-codes-capy';
import { BUDGET_RESULT_CODES } from './result-codes-budget';
import { EVAL_RESULT_CODES } from './result-codes-eval';
import { PLAN_RESULT_CODES } from './result-codes-plan';
import { AUTOPLAN_RESULT_CODES } from './result-codes-autoplan';
export interface ResultCode {
/** Stable `<a id>` in docs/troubleshooting.md. */
@@ -770,6 +771,7 @@ export const RESULT_CODES = {
...BUDGET_RESULT_CODES,
...EVAL_RESULT_CODES,
...PLAN_RESULT_CODES,
...AUTOPLAN_RESULT_CODES,
} as const satisfies Record<string, ResultCode>;
export type ResultCodeName = keyof typeof RESULT_CODES;
lib/review-pass.ts
+8 -2
View File
@@ -3,7 +3,7 @@
*
* gstack-review-dispatch pass turns gstack-review-plan's dispatches into a manifest: stage `first`
* (audit readers that may write tests, so they settle before any reviewer reads the tree), stage
* `specialists` (the specialists, plus the other audit and QA readers), stage `red-team` and stage
* `specialists` (the specialists, plus the other audit readers), stage `red-team` and stage
* `voices` (Step 11). Each reviewer keeps its plan entry (role, effort, exclusions) and is rendered
* once, when its inputs exist, by lib/review-brief.ts; the Red Team's findings file is the
* specialists' merged.jsonl lines, byte for byte. Briefs travel to the dispatch program in bounded
@@ -28,7 +28,7 @@ export interface PassReader {
role?: BriefRole; name?: string; effort?: string; exclude?: string; checklists?: string[];
/** The targeted reader: the plan's TARGETED classes and the merge base it reads on every pass. */
classes?: string[]; diff_base?: string;
/** An audit or QA reader: its prompt is a file the parent wrote from its section; its output is returned, never collected. */
/** An audit reader: its prompt is a file the parent wrote from its section; its output is returned, never collected. */
prompt_file?: string; model?: PassModel;
file?: string; sha256?: string; bytes?: number; chunks?: number; findings_sha256?: string;
/** The fingerprint the reader's reply must report: the checkout's HEAD^{tree} and toplevel when its brief was rendered (GST-482). */
@@ -47,6 +47,9 @@ export interface Manifest {
}
export interface FixCheckFiles { fix_list: string; hunks: string; tests: string; callers?: string }
export interface Extra { name: string; prompt_file: string; first?: boolean; model?: PassModel }
/** Step 9.2.1's own first line: its probes are the parent's, so a QA subagent only refuses (C4-36, C5-36: "parent-only"). */
export const PARENT_ONLY_QA = 'Only the parent runs report-only discovery.';
const qaRefusal = (name: string) => new BriefError(`Step 9.2.1 is the parent's report-only discovery (ship/sections/review-army.md: "${PARENT_ONLY_QA}"), never an audit reader: ${name}`);
/** Brief characters per transport chunk: even with every character escaped, one chunk line stays far under 1 MiB. */
export const CHUNK_CHARS = 65_536;
@@ -96,6 +99,9 @@ export function buildManifest(o: {
}
for (const e of o.extras ?? []) {
if (!/^[a-z0-9-]+$/.test(e.name)) throw new BriefError(`an audit reader's name is lowercase letters, digits and dashes: ${e.name}`);
let brief = '';
try { brief = fs.readFileSync(e.prompt_file, 'utf8'); } catch { brief = ''; }
if (/(^|-)qa(-|$)/.test(e.name) || brief.includes(PARENT_ONLY_QA)) throw qaRefusal(e.name);
readers.push({ reader: `audit/${e.name}`, stage: e.first ? 'first' : 'specialists', title: `ship ${o.pass} audit/${e.name}`, state: 'planned', prompt_file: path.resolve(e.prompt_file), model: e.model });
}
if (new Set(readers.map(r => r.reader)).size !== readers.length) throw new BriefError('two readers share a key in this pass');
lib/ship-bin-common.ts
+28 -5
View File
@@ -10,7 +10,7 @@ import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { fragmentSlug } from './changelog-fragments';
import { parseUnifiedDiff, stampOnlyLine, versionAtRef } from './restamp';
import { branchAddedLines, forkVersion, parseUnifiedDiff, stampMentions, stampOnlyLine, versionAtRef } from './restamp';
import { matchesAny } from './ship-policy';
import { extractVersion } from './version-source';
import { BIN_DIR, runBashSync, runScriptSync, scriptInvocation } from './script-invocation';
@@ -96,19 +96,23 @@ export function versionOnlyEdit(cwd: string, ref: string, p: string): boolean {
* never a behavior change. A manifest counts only while its edit is version-only (a dependency, script or
* lockfile change is a build input); bun.lock never counts.
*/
export function releasePaths(policy: Record<string, any> | null, cwd: string, ref: string): string[] {
export function releasePaths(policy: Record<string, any> | null, cwd: string, ref: string, base: string): string[] {
const list = (v: unknown) => (Array.isArray(v) ? v.filter((s): s is string => typeof s === 'string') : []);
const fragment = fragmentPath(policy, cwd);
return [...new Set(['VERSION', 'CHANGELOG.md', 'TODOS.md', 'agents-digest/gstack-AGENTS.md', ...VERSIONED_MANIFESTS.filter(p => versionOnlyEdit(cwd, ref, p)),
...(typeof policy?.changelog === 'string' ? [policy.changelog] : []), ...(fragment ? [fragment] : []), ...list(policy?.release_outputs), ...list(policy?.stamp_paths),
...stampedFiles(policy, cwd, ref)])];
...stampedFiles(policy, cwd, ref, base)])];
}
/**
* `stamp_files` paths whose working-tree edit against `ref` is a version stamp alone: every changed line swaps the
* old version for the new one under a built-in form or a `stamp_patterns` template (lib/tree-receipt.ts classifyFile).
* A moved version is floored at `ref`'s VERSION, except on a line the branch added itself (gstack-restamp's own rule:
* added since the merge base with origin/<base>, and not added by the base since): after a lost slot and a merge of the
* base its stale stamp is older than `ref`'s VERSION, so it must be newer than the fork point's version instead, the
* bound gstack-restamp's stamp_paths stage uses (GST-895).
*/
export function stampedFiles(policy: Record<string, any> | null, cwd: string, ref: string): string[] {
export function stampedFiles(policy: Record<string, any> | null, cwd: string, ref: string, base: string): string[] {
const globs = Array.isArray(policy?.stamp_files) ? policy.stamp_files.filter((s: unknown): s is string => typeof s === 'string') : [];
if (!globs.length) return [];
const before = versionAtRef(cwd, ref);
@@ -119,8 +123,27 @@ export function stampedFiles(policy: Record<string, any> | null, cwd: string, re
const templates = Array.isArray(policy?.stamp_patterns) ? policy.stamp_patterns.filter((s: unknown): s is string => typeof s === 'string') : [];
const diff = git(cwd, ['diff', '--unified=0', '--no-color', '--no-ext-diff', '--no-textconv', '--no-renames', ref]);
if (diff.status !== 0) return [];
let fork: string | undefined;
let since: string | undefined;
const own = new Map<string, Set<string>>();
const ownLines = (rel: string): Set<string> => {
if (own.has(rel)) return own.get(rel)!;
since ??= git(cwd, ['merge-base', `origin/${base}`, 'HEAD']).stdout.trim();
let lines = new Set<string>();
if (since) {
try { lines = branchAddedLines(cwd, since, [rel], ref, branchAddedLines(cwd, since, [rel], `origin/${base}`)).get(rel)?.text ?? lines; } catch {}
}
own.set(rel, lines);
return lines;
};
const stamp = (rel: string, removed: string, added: string): boolean => {
if (stampOnlyLine(removed, added, before.version, after, templates)) return true;
if (!ownLines(rel).has(removed)) return false;
fork ??= forkVersion(cwd, `origin/${base}`, before.version);
return stampOnlyLine(removed, added, fork, after, templates) && (!stampMentions(removed, templates).has(fork) || stampMentions(added, templates).has(fork));
};
return parseUnifiedDiff(diff.stdout).filter(f => f.status === 'M' && !f.binary && !f.modeChanged && matchesAny(globs, f.path) && f.hunks.length > 0 &&
f.hunks.every(h => h.removed.length === h.added.length && h.removed.every((line, i) => stampOnlyLine(line, h.added[i]!, before.version, after, templates)))).map(f => f.path);
f.hunks.every(h => h.removed.length === h.added.length && h.removed.every((line, i) => stamp(f.path, line, h.added[i]!)))).map(f => f.path);
}
/** Under `changelog_fragments`, the branch's own Step 13 entry `<dir>/<branch-slug>.md` (docs/ship-policy.md#changelog-fragments); null without the key. */
lib/ship-queue-dispatch-ci.ts
+10 -1
View File
@@ -9,7 +9,10 @@
* run at this head already concluded), a wait for each run, and one
* `PREGATE_CLEARED:` per requires-remote lane whose workflow is one of them
* and whose run concluded success. A red run clears no lane and refuses. A
* fork head is never dispatched here (QUEUE_TRAIN_UNTRUSTED).
* fork head is never dispatched here (QUEUE_TRAIN_UNTRUSTED). A workflow the
* base has but the head predates (windows-smoke.yml on a branch cut before it
* landed) is skipped with a note: GitHub reads the workflow file from the
* branch, so the dispatch could only fail.
*/
import * as fs from 'node:fs';
import * as path from 'node:path';
@@ -17,6 +20,7 @@ import type { PregateFile } from './headless-artifacts';
import { PREGATE_OUT_REL, clearObligation, readPregateFile, writePregateFile } from './pregate/index';
import type { QueueContext } from './ship-queue';
import type { QueueFailure } from './ship-queue-gate';
import { git } from './ship-queue-gate';
import { prInfo, repoSlug, runView } from './ship-queue-train-github';
const sleep = (ms: number) => new Promise(resolve => setTimeout(resolve, ms));
@@ -47,6 +51,11 @@ export async function dispatchCi(ctx: QueueContext, o: DispatchCiOptions): Promi
const prior = atHead.find(r => r.status === 'completed');
if (prior) { lines.push(`CI_DISPATCH: ${wf} not dispatched (run ${prior.databaseId} already concluded ${prior.conclusion ?? 'unknown'} at this head)`); concluded.set(wf, prior); continue; }
const live = atHead[0];
const wfPath = `.github/workflows/${wf}`;
if (!live && git(ctx.repoRoot, ['cat-file', '-e', `${head}^{commit}`]).ok && git(ctx.repoRoot, ['cat-file', '-e', `${ctx.remote ?? 'origin'}/${ctx.base}:${wfPath}`]).ok && !git(ctx.repoRoot, ['cat-file', '-e', `${head}:${wfPath}`]).ok) {
lines.push(`CI_DISPATCH: ${wf} not dispatched (${wfPath} is on ${ctx.base} but not in ${head.slice(0, 12)}; the branch predates it, so its pull_request run comes with the stamp commit)`);
continue;
}
if (!live) {
const d = ctx.gh(['workflow', 'run', wf, '--ref', branch]);
if (!d.ok) return { lines, failure: { code: 'QUEUE_NOT_READY', message: `gh workflow run ${wf} --ref ${branch} failed (${(d.err ?? d.out).trim().replace(/\s+/g, ' ').slice(0, 120) || 'no output'})` } };
lib/ship-queue-land-lock.ts
+137
View File
@@ -0,0 +1,137 @@
/**
* The repository land lock (docs/ship-policy.md#land, #3599): one ref, refs/notes/gstack-land, whose commit
* holds lock.json, shared by `gstack-ship-queue land` (owner `{ pr }`) and `train land` (owner
* `{ train, slot, pr }`), so a standalone land and a train slot never both publish onto one main. Every write
* is an exact lease on the sha read, so two writers racing one sha leave one winner. A lock is resumed only by
* the same owner kind, identity and journal; anything else is a takeover the caller allows (a drift repair, a
* train's next slot, the admitted repair train) or a --break-lock under the staleness, mid-push, drift, freeze
* and live-train rules.
*/
import { git, type QueueFailure } from './ship-queue-gate';
export const LAND_LOCK_REF = 'refs/notes/gstack-land';
export const PUSH_STALE_MS = 10 * 60_000;
export type LandLockPhase = 'stamping' | 'pushing' | 'waiting' | 'merging' | 'drift' | 'between' | 'freeze';
export interface LandLock { pr: string; holder: string; started_at: string; phase: LandLockPhase; journal: string; phase_at?: string; repair_pr?: string; drift_of?: string; train?: string; slot?: number }
export type LockOwner = { pr: string; train?: undefined; slot?: undefined } | { pr: string; train: string; slot: number };
export type LockRead = { state: 'absent'; sha: '' } | { state: 'released'; sha: string } | { state: 'held'; sha: string; lock: LandLock } | { state: 'unreadable'; sha: string; detail: string };
type Released = { released: true; by: string; at: string };
const isSha = (s: string | undefined) => /^[0-9a-f]{40,64}$/.test(s ?? '');
/** How a lock names its owner in every message: `#<pr>`, or `train <nonce> slot <i> (#<pr>)`. */
export const lockLabel = (lock: LandLock): string => (lock.train ? `train ${lock.train} slot ${lock.slot} (#${lock.pr})` : `#${lock.pr}`);
/** The same owner kind, identity and journal: a resume of this run's journal, never just the same PR. */
export const ownsLock = (lock: LandLock, owner: LockOwner, journal: string): boolean =>
lock.pr === owner.pr && (lock.train ?? null) === (owner.train ?? null) && (lock.slot ?? null) === (owner.slot ?? null) && lock.journal === journal;
export function readLock(root: string, remote = 'origin'): LockRead {
const fetched = git(root, ['fetch', '-q', remote, `+${LAND_LOCK_REF}:${LAND_LOCK_REF}`]);
if (!fetched.ok) {
const listed = git(root, ['ls-remote', remote, LAND_LOCK_REF]);
if (listed.ok && !listed.out.trim()) git(root, ['update-ref', '-d', LAND_LOCK_REF]);
}
const sha = git(root, ['rev-parse', '--verify', '--quiet', LAND_LOCK_REF]).out;
if (!sha) return { state: 'absent', sha: '' };
const body = git(root, ['show', `${sha}:lock.json`]);
if (!body.ok) return { state: 'unreadable', sha, detail: 'no lock.json' };
let value: Partial<LandLock> & { released?: boolean };
try { value = JSON.parse(body.out); } catch { return { state: 'unreadable', sha, detail: 'lock.json is not JSON' }; }
if (value?.released === true) return { state: 'released', sha };
if (!/^\d+$/.test(String(value?.pr)) || typeof value.phase !== 'string' || typeof value.holder !== 'string') return { state: 'unreadable', sha, detail: 'lock.json names no PR, holder and phase' };
return { state: 'held', sha, lock: value as LandLock };
}
export function writeLock(root: string, expected: string, value: LandLock | Released, remote = 'origin'): { ok: boolean; sha: string } {
const blob = git(root, ['hash-object', '-w', '--stdin'], undefined, JSON.stringify(value, null, 2) + '\n').out;
const tree = git(root, ['mktree'], undefined, `100644 blob ${blob}\tlock.json\n`).out;
const message = 'released' in value ? `land lock released by ${value.by}` : `land lock ${value.train ? `train ${value.train} slot ${value.slot} ` : ''}#${value.pr} ${value.phase}`;
const commit = git(root, ['commit-tree', tree, ...(expected ? ['-p', expected] : []), '-m', message]).out;
if (![blob, tree, commit].every(isSha)) return { ok: false, sha: '' };
const push = git(root, ['push', '-q', `--force-with-lease=${LAND_LOCK_REF}:${expected}`, remote, `${commit}:${LAND_LOCK_REF}`]);
if (push.ok) git(root, ['update-ref', LAND_LOCK_REF, commit]);
return { ok: push.ok, sha: commit };
}
export interface HeldLandLock {
/** The lock as this run last wrote it. */
readonly lock: LandLock;
/** This owner's lock as it was before this run took it back (a resume), else null. */
readonly resumed: LandLock | null;
/** Move to a phase under the exact lease; a failure is another lander's lock (QUEUE_LAND_LOCKED) or a refused push (QUEUE_NOT_READY). */
set(phase: LandLockPhase, extra?: Partial<LandLock>): QueueFailure | null;
release(): boolean;
}
export interface AcquireOptions {
root: string; remote?: string; holder: string; owner: LockOwner; journal: string; now: () => Date;
/** The first phase, given this owner's own lock when this run resumes it. */
phase: (own: LandLock | null) => LandLockPhase;
breakLock?: boolean; breakTrainLock?: boolean;
/** A takeover the caller allows (a drift repair, a train's next slot, the admitted repair train): its LOCK: line, else null. */
takeover?: (lock: LandLock) => string | null;
/** Whether a train still lands (its record is not terminal); unknown counts as live. */
liveTrain?: (nonce: string) => boolean;
/** Fields every write of this run carries (land's drift_of), given this owner's own lock when resumed. */
extra?: (own: LandLock | null) => Partial<LandLock>;
}
/** Why --break-lock may not break a foreign lock, or null when it may. */
function unbreakable(lock: LandLock, o: AcquireOptions, age: number): QueueFailure | null {
const label = lockLabel(lock);
if (lock.phase === 'drift' || lock.drift_of) return { code: 'QUEUE_LAND_DRIFT', message: `#${lock.drift_of ?? lock.pr} holds a drift lock; --break-lock never breaks it, only its repair PR's land takes it over`, fix: `gstack-ship-queue land --pr ${lock.drift_of ?? lock.pr} --repair` };
if (lock.phase === 'freeze') return { code: 'QUEUE_TRAIN_FROZEN', message: `${label} holds the land lock for a frozen queue; --break-lock never breaks it, only the repair train the freeze admits takes it over` };
if (lock.train && (o.liveTrain?.(lock.train) ?? true) && !o.breakTrainLock) return { code: 'QUEUE_LAND_LOCKED', message: `${label} is a live train run's lock (phase ${lock.phase}); --break-lock refused without --break-train-lock` };
if ((lock.phase === 'pushing' || lock.phase === 'merging') && age < PUSH_STALE_MS) return { code: 'QUEUE_LAND_LOCKED', message: `${label}'s holder is mid-${lock.phase === 'pushing' ? 'push' : 'merge'} (${Math.round(age / 1000)} s ago); --break-lock refused` };
if (lock.phase === 'between' && age < PUSH_STALE_MS) return { code: 'QUEUE_LAND_LOCKED', message: `${label} is between slots (${Math.round(age / 1000)} s ago); --break-lock refused until it is ${PUSH_STALE_MS / 60_000} minutes old` };
return null;
}
/**
* Take the land lock for `owner`: resume this owner's own lock, take over one the caller allows, or (with
* --break-lock, which always names the holder) break a foreign one the rules permit; then write the first
* phase under the exact lease on the sha read.
*/
export function acquireLandLock(o: AcquireOptions): { held: HeldLandLock; lines: string[] } | { failure: QueueFailure; lines: string[] } {
const remote = o.remote ?? 'origin';
const lines: string[] = [];
const read = readLock(o.root, remote);
let sha = read.sha;
let current: LandLock | null = read.state === 'held' ? read.lock : null;
const own = current && ownsLock(current, o.owner, o.journal) ? current : null;
if (read.state === 'unreadable') {
if (!o.breakLock) return { failure: { code: 'QUEUE_LAND_LOCKED', message: `the land lock at ${read.sha.slice(0, 12)} cannot be read (${read.detail}); --break-lock replaces it` }, lines };
lines.push(`LOCK: broke an unreadable lock at ${read.sha.slice(0, 12)} (${read.detail})`);
} else if (current && !own) {
const allowed = o.takeover?.(current) ?? null;
if (allowed) lines.push(allowed);
else {
const age = o.now().getTime() - Date.parse(current.phase_at ?? current.started_at);
if (!o.breakLock) return { failure: { code: 'QUEUE_LAND_LOCKED', message: `${lockLabel(current)} holds the land lock (${current.holder}, phase ${current.phase}, since ${current.started_at})` }, lines };
lines.push(`LOCK_HOLDER: ${lockLabel(current)} holder=${current.holder} phase=${current.phase} since=${current.started_at}`);
const refusal = unbreakable(current, o, age);
if (refusal) return { failure: refusal, lines };
lines.push(`LOCK: broke ${lockLabel(current)}'s lock (${current.holder}, phase ${current.phase})`);
}
}
const startedAt = own?.started_at ?? o.now().toISOString();
const always = o.extra?.(own) ?? {};
const set = (phase: LandLockPhase, extra: Partial<LandLock> = {}): QueueFailure | null => {
const next: LandLock = { pr: o.owner.pr, ...(o.owner.train ? { train: o.owner.train, slot: o.owner.slot } : {}), holder: o.holder, started_at: startedAt, phase, journal: o.journal, phase_at: o.now().toISOString(), ...always, ...extra };
const w = writeLock(o.root, sha, next, remote);
if (!w.ok) {
return readLock(o.root, remote).sha === sha
? { code: 'QUEUE_NOT_READY', message: `the land lock push to ${remote} failed and the lock there is unchanged; nothing was pushed` }
: { code: 'QUEUE_LAND_LOCKED', message: 'the land lock moved under this run (another lander took it)' };
}
sha = w.sha; current = next;
return null;
};
const failed = set(o.phase(own));
if (failed) return { failure: failed, lines };
const held: HeldLandLock = {
get lock() { return current!; },
resumed: own,
set,
release: () => writeLock(o.root, sha, { released: true, by: o.holder, at: o.now().toISOString() }, remote).ok,
};
return { held, lines };
}
lib/ship-queue-land.ts
+33 -52
View File
@@ -17,15 +17,15 @@ import * as path from 'node:path';
import { mergeChangelogs, parseChangelog } from './changelog-check';
import { fragmentSlug, readFragments } from './changelog-fragments';
import { restamp, versionAtRef } from './restamp';
import { acquireLandLock, lockLabel, readLock, writeLock, type LandLockPhase } from './ship-queue-land-lock';
import { matchesAny, type ShipPolicy } from './ship-policy';
import { ciAtHead, loadBasePolicy, type QueueContext, type StepOutput } from './ship-queue';
import { QUEUE_DIR_REL, git, writeJsonAtomic, type QueueFailure } from './ship-queue-gate';
import { bindShippedReceipt } from './ship-receipt-depth';
import { loadReady, rebuildFromNotes, repoIdentity } from './ship-queue-train-record';
import { TERMINAL_STATES, freezeLockCleared, loadReady, loadTrain, rebuildFromNotes, repoIdentity } from './ship-queue-train-record';
export const LAND_LOCK_REF = 'refs/notes/gstack-land';
export { LAND_LOCK_REF } from './ship-queue-land-lock';
export const MAX_RESTAMPS = 2;
const PUSH_STALE_MS = 10 * 60_000;
const MANIFESTS = ['package.json', 'package-lock.json', 'npm-shrinkwrap.json'];
const isSha = (s: string | undefined) => /^[0-9a-f]{40,64}$/.test(s ?? '');
/** The first path segment of `rel` inside `wt` that is a symlink, or null: land writes only inside the checkout. */
@@ -49,35 +49,13 @@ export interface LandJournal {
drift_main?: string; repair_pr?: string;
phase: 'locked' | 'stamped' | 'pushed' | 'merged' | 'landed' | 'drift';
}
export interface LandLock { pr: string; holder: string; started_at: string; phase: 'stamping' | 'pushing' | 'waiting' | 'merging' | 'drift'; journal: string; phase_at?: string; repair_pr?: string; drift_of?: string }
export interface LandOptions { pr: string; repair?: boolean; dryRun?: boolean; noWait?: boolean; breakLock?: boolean; allowRepoCommands?: boolean; waitTimeoutMin?: number; now?: () => Date; holder?: string }
export interface LandOptions { pr: string; repair?: boolean; dryRun?: boolean; noWait?: boolean; breakLock?: boolean; breakTrainLock?: boolean; allowRepoCommands?: boolean; waitTimeoutMin?: number; now?: () => Date; holder?: string }
const fail = (lines: string[], failure: QueueFailure, run: string): StepOutput => ({ ok: false, lines, failure, run });
export const landJournalRel = (pr: string) => `${QUEUE_DIR_REL}/land/${pr}.json`;
function readJson<T>(file: string): T | null { try { return JSON.parse(fs.readFileSync(file, 'utf8')) as T; } catch { return null; } }
/** The remote land lock: a ref whose commit holds lock.json, or a released marker. Updated only by exact lease. */
function readLock(root: string): { sha: string; lock: LandLock | null } {
git(root, ['fetch', '-q', 'origin', `+${LAND_LOCK_REF}:${LAND_LOCK_REF}`]);
const sha = git(root, ['rev-parse', '--verify', '--quiet', LAND_LOCK_REF]).out;
if (!sha) return { sha: '', lock: null };
const body = git(root, ['show', `${sha}:lock.json`]);
let lock: LandLock | { released: true } | null = null;
try { lock = body.ok ? JSON.parse(body.out) : null; } catch { lock = null; }
return { sha, lock: lock && 'pr' in lock && /^\d+$/.test(String(lock.pr)) ? lock : null };
}
function writeLock(root: string, expected: string, value: LandLock | { released: true; by: string; at: string }): { ok: boolean; sha: string } {
const blob = git(root, ['hash-object', '-w', '--stdin'], undefined, JSON.stringify(value, null, 2) + '\n').out;
const tree = git(root, ['mktree'], undefined, `100644 blob ${blob}\tlock.json\n`).out;
const commit = git(root, ['commit-tree', tree, ...(expected ? ['-p', expected] : []), '-m', 'released' in value ? `land lock released by ${value.by}` : `land lock #${value.pr} ${value.phase}`]).out;
if (![blob, tree, commit].every(isSha)) return { ok: false, sha: '' };
const push = git(root, ['push', '-q', `--force-with-lease=${LAND_LOCK_REF}:${expected}`, 'origin', `${commit}:${LAND_LOCK_REF}`]);
if (push.ok) git(root, ['update-ref', LAND_LOCK_REF, commit]);
return { ok: push.ok, sha: commit };
}
/**
* Release-version conflicts only: the version file and release outputs take the base's side, the
* CHANGELOG merges by section (mergeChangelogs), and a manifest or npm lockfile is re-merged with
@@ -328,7 +306,7 @@ function repair(ctx: QueueContext, policy: ShipPolicy, j: LandJournal | null, fi
if (!n) return fail(lines, { code: 'QUEUE_NOT_READY', message: `gh pr create failed: ${(pr.err ?? pr.out).trim().slice(0, 160)}` }, run);
j.repair_pr = n; writeJsonAtomic(file, j);
const held = readLock(root);
if (held.lock?.pr === j.pr && held.lock.phase === 'drift') writeLock(root, held.sha, { ...held.lock, repair_pr: n });
if (held.state === 'held' && !held.lock.train && held.lock.pr === j.pr && held.lock.phase === 'drift') writeLock(root, held.sha, { ...held.lock, repair_pr: n });
return { ok: true, run, lines: [...lines, `REPAIR_PR: #${n} ${branch} ${pushed}`, `NEXT: run /ship on #${n}, then gstack-ship-queue land --pr ${n} (it takes over #${j.pr}'s drift lock and releases it)`] };
} finally {
git(root, ['worktree', 'remove', '--force', wt]);
@@ -379,28 +357,31 @@ export function land(ctx: QueueContext, o: LandOptions): StepOutput {
return { ok: true, run, lines: [...lines, ...s.lines, `STAMP_VERSION: ${s.version}`, `STAMP_TREE: ${s.tree}`, 'DRY-RUN: nothing locked, pushed or merged'] };
}
let { sha: lockSha, lock } = readLock(root);
const drifted = lock && lock.pr !== o.pr && lock.phase === 'drift' ? lock.pr : null;
const repairOf = drifted ? readJson<LandJournal>(path.join(root, landJournalRel(drifted))) : null;
const driftOf = drifted && (lock!.repair_pr === o.pr || repairOf?.repair_pr === o.pr) ? drifted : lock?.pr === o.pr ? lock.drift_of : undefined;
if (drifted && driftOf) lines.push(`LOCK: taking over #${drifted}'s drift lock for its repair`);
else if (lock && lock.pr !== o.pr) {
const age = now().getTime() - Date.parse(lock.phase_at ?? lock.started_at);
if (!o.breakLock) return fail(lines, { code: 'QUEUE_LAND_LOCKED', message: `#${lock.pr} holds the land lock (${lock.holder}, phase ${lock.phase}, since ${lock.started_at})` }, run);
if (lock.phase === 'drift' || lock.drift_of) return fail(lines, { code: 'QUEUE_LAND_DRIFT', message: `#${lock.drift_of ?? lock.pr} holds a drift lock; --break-lock never breaks it, only its repair PR's land takes it over`, fix: `gstack-ship-queue land --pr ${lock.drift_of ?? lock.pr} --repair` }, run);
if ((lock.phase === 'pushing' || lock.phase === 'merging') && age < PUSH_STALE_MS) return fail(lines, { code: 'QUEUE_LAND_LOCKED', message: `#${lock.pr}'s holder is mid-${lock.phase === 'pushing' ? 'push' : 'merge'} (${Math.round(age / 1000)} s ago); --break-lock refused` }, run);
lines.push(`LOCK: broke #${lock.pr}'s lock (${lock.holder}, phase ${lock.phase})`);
}
const setLock = (phase: LandLock['phase']): QueueFailure | null => {
const next: LandLock = { pr: o.pr, holder, started_at: lock?.pr === o.pr ? lock.started_at : now().toISOString(), phase, journal: run, phase_at: now().toISOString(), ...(driftOf ? { drift_of: driftOf } : {}) };
const w = writeLock(root, lockSha, next);
if (!w.ok) return { code: 'QUEUE_LAND_LOCKED', message: 'the land lock moved under this run (another lander took it)' };
lockSha = w.sha; lock = next;
return null;
};
let driftOf: string | undefined;
const got = acquireLandLock({
root, holder, owner: { pr: o.pr }, journal: run, now, breakLock: o.breakLock, breakTrainLock: o.breakTrainLock,
takeover: held => {
if (held.phase === 'freeze') {
const cleared = freezeLockCleared(root, ctx.remote ?? 'origin', keys, held);
return cleared ? `LOCK: taking over ${lockLabel(held)}'s freeze lock; that freeze was cleared at ${cleared}` : null;
}
if (held.train || held.phase !== 'drift' || (held.repair_pr !== o.pr && readJson<LandJournal>(path.join(root, landJournalRel(held.pr)))?.repair_pr !== o.pr)) return null;
driftOf = held.pr;
return `LOCK: taking over #${held.pr}'s drift lock for its repair`;
},
liveTrain: nonce => {
if (!loadTrain(nonce)) rebuildFromNotes(root, ctx.remote ?? 'origin');
const t = loadTrain(nonce);
return !t || !TERMINAL_STATES.includes(t.state);
},
extra: own => { driftOf ??= own?.drift_of; return driftOf ? { drift_of: driftOf } : {}; },
phase: () => (j?.phase === 'pushed' ? 'waiting' : j?.phase === 'merged' ? 'merging' : 'stamping'),
});
lines.push(...got.lines);
if ('failure' in got) return fail(lines, got.failure, run);
const lock = got.held;
const setLock = (phase: LandLockPhase) => lock.set(phase);
const save = () => writeJsonAtomic(file, j);
const failed = setLock(j?.phase === 'pushed' ? 'waiting' : j?.phase === 'merged' ? 'merging' : 'stamping');
if (failed) return fail(lines, failed, run);
if (!j) { j = { schema_version: 1, pr: o.pr, branch: info.headRefName, reviewed_pin: pin, last_pushed_head: recovered ? info.headRefOid : null, stamp_commit: null, stamp_tree: null, base_sha: null, version: null, date: null, restamps: 0, phase: 'locked' }; save(); }
for (;;) {
@@ -409,7 +390,7 @@ export function land(ctx: QueueContext, o: LandOptions): StepOutput {
if (j.phase === 'pushed' && j.base_sha !== baseNow) {
if (j.restamps >= MAX_RESTAMPS) {
j.restamps = 0; save();
if (!driftOf) writeLock(root, lockSha, { released: true, by: holder, at: now().toISOString() });
if (!driftOf) lock.release();
return fail(lines, { code: 'QUEUE_LAND_BASE_BUSY', message: `origin/${ctx.base} moved during CI ${MAX_RESTAMPS + 1} times; the lock is released and a later run restamps from the pin` }, run);
}
lines.push(`BASE_MOVED: ${j.base_sha?.slice(0, 12)} -> ${baseNow.slice(0, 12)}; restamping`);
@@ -419,7 +400,7 @@ export function land(ctx: QueueContext, o: LandOptions): StepOutput {
const s = buildStamp(ctx, policy, { pin, baseSha: baseNow, pr: o.pr, title: info.title, allowRepoCommands: !!o.allowRepoCommands, now: j.date ? new Date(`${j.date}T12:00:00Z`) : now() });
if ('code' in s) {
if (driftOf) setLock('stamping');
else writeLock(root, lockSha, { released: true, by: holder, at: now().toISOString() });
else lock.release();
return fail([...lines], s, run);
}
lines.push(...s.lines);
@@ -471,8 +452,8 @@ export function land(ctx: QueueContext, o: LandOptions): StepOutput {
j.phase = 'landed'; save();
const driftJournal = driftOf ? readJson<LandJournal>(path.join(root, landJournalRel(driftOf))) : null;
if (driftOf && driftJournal) writeJsonAtomic(path.join(root, landJournalRel(driftOf)), { ...driftJournal, phase: 'landed' });
const rel = writeLock(root, lockSha, { released: true, by: holder, at: now().toISOString() });
return { ok: true, run, lines: [...lines, `MERGED: #${o.pr} as ${main} (${j.version})`, `LANDED: origin/${ctx.base} tree ${tree} equals the stamped tree`, `LOCK: ${rel.ok ? 'released' : 'release push failed (the next land takes it over)'}`] };
const released = lock.release();
return { ok: true, run, lines: [...lines, `MERGED: #${o.pr} as ${main} (${j.version})`, `LANDED: origin/${ctx.base} tree ${tree} equals the stamped tree`, `LOCK: ${released ? 'released' : 'release push failed (the next land takes it over)'}`] };
}
if (j.phase === 'drift') return fail(lines, { code: 'QUEUE_LAND_DRIFT', message: 'a previous land of this PR found drift', fix: `gstack-ship-queue land --pr ${o.pr} --repair` }, run);
}
lib/ship-queue-train-cli.ts
+3 -3
View File
@@ -36,7 +36,7 @@ export const TRAIN_HELP = ` train build --prs <n>@<sha>[,...] [--order <n>,...]
train status [--train <nonce>] never spends: rebuild records from refs/notes/gstack-trains, reconcile
unconfirmed dispatches by nonce
train verdict --train <nonce> read the gate run and its record: GREEN, RED, INCOMPLETE or PENDING
train land --train <nonce> --pr <n> [--no-wait] [--timeout <min>]
train land --train <nonce> --pr <n> [--no-wait] [--timeout <min>] [--break-lock [--break-train-lock]]
publish slot n's prefix (stamped S_i, else gated T_i): update mode commits its tree on
[pin, main], pushes under the pin's lease, waits for checks and merges; direct mode
(trains with gates, no strict protection, a green pin) runs gh pr merge
@@ -156,7 +156,7 @@ export async function trainMain(ctx: QueueContext, args: string[]): Promise<numb
if (!pr || !/^\d+$/.test(pr)) return usage('train land needs --pr <number>');
const timeout = positive(flag(rest, '--timeout'), 60);
if (Number.isNaN(timeout)) return usage('--timeout needs minutes');
return guarded('train land', async () => (await import('./ship-queue-train-land')).trainLand(ctx, { train: nonce, pr, noWait: rest.includes('--no-wait'), timeoutMin: timeout }));
return guarded('train land', async () => (await import('./ship-queue-train-land')).trainLand(ctx, { train: nonce, pr, noWait: rest.includes('--no-wait'), timeoutMin: timeout, breakLock: rest.includes('--break-lock'), breakTrainLock: rest.includes('--break-train-lock') }));
}
if (sub === 'rebuild') return guarded('train rebuild', () => trainRebuild(ctx, { train: nonce, infra: rest.includes('--infra') }));
if (sub === 'bisect') {
@@ -174,7 +174,7 @@ export async function readyMain(ctx: QueueContext, args: string[], base: { pr: s
if (args.includes('--broad') && !screen) return usage('--broad selects the screen policy; it needs --screen');
const dispatchCi = flag(args, '--dispatch-ci');
const workflows = dispatchCi === undefined ? [] : dispatchCi.split(',').map(s => s.trim()).filter(Boolean);
if (dispatchCi !== undefined && (!workflows.length || workflows.some(w => !/^[\w.-]+\.ya?ml$/.test(w)))) return usage('--dispatch-ci names workflow files under .github/workflows, comma-separated (free-tests.yml,windows-free-tests.yml)');
if (dispatchCi !== undefined && (!workflows.length || workflows.some(w => !/^[\w.-]+\.ya?ml$/.test(w)))) return usage('--dispatch-ci names workflow files under .github/workflows, comma-separated (free-tests.yml,windows-smoke.yml)');
const timeoutSec = positive(flag(args, '--timeout'), 2700);
if (!Number.isFinite(timeoutSec)) return usage('--timeout needs a positive number of seconds');
const fixer = flag(args, '--fixer');
lib/ship-queue-train-gate.ts
+2 -2
View File
@@ -11,7 +11,7 @@
* a second red is real; both attempts' jobs are kept.
* - `command`: a clean detached checkout of the prefix head runs on a Ubicloud VM through
* scripts/ubicloud/ubi-runner.sh, forwarding only `pass_env`; the VM's `git write-tree` must equal the prefix
* tree and exit 0 is GREEN. One spend cap covers every run of the train.
* tree and exit 0 is GREEN. One spend cap per command gate covers that gate's runs across every prefix of the train.
*
* The verdict is per prefix; the train is GREEN when every prefix is, and otherwise the first red prefix after a
* green one names the culprit (classifyPrefixes). Slot 1's READY may declare FIXES_BASE_RED: T0 is then allowed
@@ -366,7 +366,7 @@ async function settleCommand(env: TrainEnv, t: TrainRecord, a: AttemptRecord, g:
lines.push(`COMMAND: gate ${a.gate} on T${p.index} ${r.outcome} (${r.detail})`);
}
/** Run every command unit that has no finished attempt, all prefixes in parallel, under the train's one spend cap. */
/** Run every command unit that has no finished attempt, all prefixes in parallel, under one spend cap per command gate across every prefix of the train. */
async function runCommands(env: TrainEnv, t: TrainRecord, units: GateUnit[], lines: string[]): Promise<QueueFailure | null> {
const commands = units.filter(u => u.workflow === 'command');
if (!commands.length) return null;
lib/ship-queue-train-land.ts
+76 -5
View File
@@ -17,16 +17,24 @@
* Either way main's tree must be the prefix after the merge; any other main is the freeze path
* (QUEUE_TRAIN_TREE_MISMATCH), as for verify. Fork heads are refused. Journaled beside the train record; a rerun
* resumes: open at the pin, open at the update commit, or merged (confirmed through main's tree).
*
* Train land takes the shared land lock (lib/ship-queue-land-lock.ts, #3599) as `train <nonce> slot <i> (#pr)`
* before its first main read, so no standalone land publishes beside a slot. After a slot merges with landable
* slots left, the lock stays in a train-owned `between` phase that the same train's next slot takes over; a stop
* that freezes the queue leaves it in a `freeze` phase only the admitted repair train takes over. The release is
* decided once, at the trainLand boundary (settleLock).
*/
import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { ciAtHead, type QueueContext } from './ship-queue';
import { git, writeJsonAtomic } from './ship-queue-gate';
import { git, writeJsonAtomic, type QueueFailure } from './ship-queue-gate';
import { acquireLandLock, lockLabel, type HeldLandLock, type LandLock, type LandLockPhase } from './ship-queue-land-lock';
import { waitChecks } from './ship-queue-land';
import { verifyPostMerge } from './ship-queue-post-merge';
import { bindTrainReceipt } from './ship-queue-train-gate';
import { checkRun, prInfo, type PrInfo } from './ship-queue-train-github';
import { activeFreeze, attemptsOf, nowIso, trainFile, type LandingRecord, type TrainRecord } from './ship-queue-train-record';
import { TERMINAL_STATES, activeFreeze, attemptsOf, freezeLockCleared, nowIso, trainFile, type LandingRecord, type TrainRecord } from './ship-queue-train-record';
import { fetchBase, fetchTrainBranch, findTrain, frozenLines, nextFor, outFailed, outPending, outRefused, save, trainEnv, type TrainEnv, type TrainOut } from './ship-queue-train';
import { confirmMerged, stopped } from './ship-queue-train-verify';
@@ -36,7 +44,10 @@ const STAMP_IDENTITY = { GIT_AUTHOR_NAME: 'gstack-ship-queue', GIT_AUTHOR_EMAIL:
const isSha = (s: string) => /^[0-9a-f]{40,64}$/.test(s);
interface SlotJournal { schema_version: 1; pr: string; pin: string; stamp_commit: string; base: string }
interface Link { index: number; commit: string; tree: string; pin: string }
interface Slot { env: TrainEnv; t: TrainRecord; i: number; pin: string; target: Link & { version?: string }; pred: Link; run: string; rerun: string; lines: string[] }
interface Slot { env: TrainEnv; t: TrainRecord; i: number; pin: string; target: Link & { version?: string }; pred: Link; run: string; rerun: string; lines: string[]; lock: { held: HeldLandLock | null; effect: boolean } }
/** Move this slot's land lock to a phase; a failure means another lander took it, so nothing remote may follow. */
const lockPhase = (s: Slot, phase: LandLockPhase): QueueFailure | null => s.lock.held?.set(phase) ?? null;
/** The checks train land requires on the landing commit: changelog_fragments.required_checks, else train.land.required_checks. */
export const landRequiredChecks = (env: TrainEnv): string[] => env.policy.policy.changelog_fragments?.required_checks ?? env.train.land?.required_checks ?? [];
@@ -131,6 +142,9 @@ async function mergeAt(ctx: QueueContext, s: Slot, head: string, base: string):
const target = prInfo(ctx, pr);
if ('error' in target) return outPending({ code: 'QUEUE_PROOF_MISSING', message: target.error }, s.lines, s.rerun, s.run);
if (target.baseRefName && target.baseRefName !== ctx.base) return retargeted(s, target.baseRefName, ctx.base);
const moved = lockPhase(s, 'merging');
if (moved) return outFailed(moved, s.lines, s.rerun, s.run);
s.lock.effect = true;
const m = ctx.gh(['pr', 'merge', pr, `--${method}`, '--match-head-commit', head]);
if (!m.ok) return outFailed({ code: 'QUEUE_NOT_READY', message: `gh pr merge refused: ${(m.err ?? m.out).trim().slice(0, 200)}` }, s.lines, s.rerun, s.run);
main = mainNow(ctx, s.env);
@@ -189,6 +203,8 @@ async function landUpdate(ctx: QueueContext, s: Slot, info: PrInfo, base: string
writeJsonAtomic(file, j);
recordLanding(s, 'update', commit, base);
const pushed = info.headRefOid !== commit;
const pushing = lockPhase(s, 'pushing');
if (pushing) return outFailed(pushing, s.lines, s.rerun, s.run);
if (pushed) {
const push = git(root, ['push', '-q', `--force-with-lease=refs/heads/${info.headRefName}:${info.headRefOid}`, env.remote, `${commit}:refs/heads/${info.headRefName}`]);
if (!push.ok) {
@@ -198,6 +214,9 @@ async function landUpdate(ctx: QueueContext, s: Slot, info: PrInfo, base: string
if (!remote.ok || now !== commit) return outFailed({ code: 'QUEUE_NOT_READY', message: `the ${what} push failed and ${info.headRefName} still reads ${remote.ok ? info.headRefOid.slice(0, 12) : 'unknown'}: ${push.err.split('\n')[0]}` }, s.lines, s.rerun, s.run);
}
}
s.lock.effect = true;
const waiting = lockPhase(s, 'waiting');
if (waiting) return outFailed(waiting, s.lines, s.rerun, s.run);
s.lines.push(`${pushed ? 'PUSHED' : 'RESUMED'}: ${commit} (tree ${label(s)}, parents [pin, main ${base.slice(0, 12)}]) ${pushed ? 'to' : 'is already the head of'} ${info.headRefName}`);
if (o.noWait && pushed) return outPending({ code: 'QUEUE_NOT_READY', message: `checks run on the ${what} commit ${commit.slice(0, 12)}` }, s.lines, `gstack-pr-wait ${pr} --until green && ${s.rerun}`, s.run);
if (!o.noWait) waitChecks(ctx, commit, o.timeoutMin ?? 60);
@@ -220,7 +239,43 @@ async function landUpdate(ctx: QueueContext, s: Slot, info: PrInfo, base: string
const retargeted = (s: Slot, b: string, base: string) => outRefused({ code: 'QUEUE_NOT_READY', message: `#${s.t.slots[s.i]!.pr} targets ${b}, not ${base}; train land merges only PRs on the base the train built` }, s.lines, `retarget #${s.t.slots[s.i]!.pr} to ${base}, then ${s.rerun}`, s.run);
export async function trainLand(ctx: QueueContext, o: { train: string; pr: string; noWait?: boolean; timeoutMin?: number }): Promise<TrainOut> {
interface TrainLandOptions { train: string; pr: string; noWait?: boolean; timeoutMin?: number; breakLock?: boolean; breakTrainLock?: boolean }
/**
* The takeovers train land allows: its own train's between-slots lock, the freeze lock when the freeze admits this
* train as its repair (or, with the notes ref fetched, a freeze record cleared after the lock was written: absence
* is never proof), and a lock whose train record is terminal.
*/
function trainTakeover(env: TrainEnv, t: TrainRecord, i: number, lock: LandLock): string | null {
if (lock.train === t.nonce && lock.phase === 'between') return `LOCK: train ${t.nonce} slot ${i + 1} takes over its train's between-slots lock (from slot ${lock.slot})`;
if (lock.phase === 'freeze') {
const cleared = freezeLockCleared(env.ctx.repoRoot, env.remote, env.repo.keys, lock);
if (activeFreeze(env.repo.keys)?.repair?.train === t.nonce) return `LOCK: repair train ${t.nonce} takes over the freeze lock of ${lockLabel(lock)}`;
return cleared ? `LOCK: train ${t.nonce} takes over ${lockLabel(lock)}'s freeze lock; that freeze was cleared at ${cleared}` : null;
}
if (!lock.train || lock.train === t.nonce) return null;
const other = findTrain(env, lock.train);
return other && TERMINAL_STATES.includes(other.state) ? `LOCK: train ${t.nonce} takes over ${lockLabel(lock)}'s lock; that train is ${other.state}` : null;
}
/**
* The one release point: a freeze keeps the lock in `freeze`; once a slot merged, a train still merging keeps it
* `between` (a merged slot, a pending wait or a refusal before this slot's push); a failure after a remote effect
* keeps its phase for the resume; anything else releases it.
*/
function settleLock(s: Slot, out: TrainOut): TrainOut {
const held = s.lock.held!;
const midway = s.t.state === 'merging' && s.t.merged.length > 0;
const keep = (phase: LandLockPhase, line: string) => { const failed = held.set(phase); return failed ? `LOCK: ${phase} could not be written (${failed.message})` : line; };
let line: string;
if (activeFreeze(s.env.repo.keys)) line = keep('freeze', 'LOCK: freeze (kept for the repair train the freeze admits; --break-lock never breaks it)');
else if (midway && (out.exit === 0 || !s.lock.effect)) line = keep('between', `LOCK: between (train ${s.t.nonce} keeps the land lock for its next slot)`);
else if (out.exit !== 0 && s.lock.effect) line = `LOCK: kept in phase ${held.lock.phase}; ${s.rerun} resumes it`;
else line = held.release() ? 'LOCK: released' : 'LOCK: release push failed (the next lander takes it over)';
return { ...out, lines: [...out.lines, line] };
}
export async function trainLand(ctx: QueueContext, o: TrainLandOptions): Promise<TrainOut> {
const fetchErr = fetchBase(ctx);
const env = trainEnv(ctx);
if ('code' in env) return outRefused(env, [], 'none', 'none');
@@ -241,8 +296,24 @@ export async function trainLand(ctx: QueueContext, o: { train: string; pr: strin
if (!chain) return outRefused({ code: 'QUEUE_PROOF_MISSING', message: `the record's T0 and ${t.stamped ? 'S' : 'T'}1..${t.stamped ? 'S' : 'T'}${k} (with each slot's pin as second parent) are not the first-parent chain of the gated head ${t.head.slice(0, 12)}` }, [], `${Q} train status --train ${t.nonce}`, run);
const pin = t.slots[i]!.head;
const rerun = `${Q} train land --train ${t.nonce} --pr ${o.pr}`;
const s: Slot = { env, t, i, pin, target: { ...chain[i + 1]!, version: t.stamped?.[i]?.version }, pred: chain[i]!, run, rerun, lines: [] };
const s: Slot = { env, t, i, pin, target: { ...chain[i + 1]!, version: t.stamped?.[i]?.version }, pred: chain[i]!, run, rerun, lines: [], lock: { held: null, effect: false } };
s.lines.push(`TRAIN_LAND: ${t.nonce} slot ${i + 1}/${k} #${o.pr} pin ${pin} -> ${label(s)}${s.target.version ? ` v${s.target.version}` : ''} tree ${s.target.tree}`);
const got = acquireLandLock({
root: ctx.repoRoot, remote: env.remote, holder: `${os.hostname()}:${process.pid}`, owner: { train: t.nonce, slot: i + 1, pr: o.pr }, journal: `train/${t.nonce}/slot-${i + 1}`,
now: () => new Date(), breakLock: o.breakLock, breakTrainLock: o.breakTrainLock,
phase: own => (own && ['pushing', 'waiting', 'merging'].includes(own.phase) ? own.phase : 'stamping'),
takeover: lock => trainTakeover(env, t, i, lock),
liveTrain: nonce => { const other = findTrain(env, nonce); return !other || !TERMINAL_STATES.includes(other.state); },
});
s.lines.push(...got.lines);
if ('failure' in got) return outRefused(got.failure, s.lines, `wait for the holder named above, then ${rerun}`, run);
s.lock = { held: got.held, effect: !!got.held.resumed && ['pushing', 'waiting', 'merging'].includes(got.held.resumed.phase) };
return settleLock(s, await landSlot(ctx, s, chain, o));
}
/** Train land under the lock: main read, confirmation, post-merge checks of the previous slot, then direct or update. */
async function landSlot(ctx: QueueContext, s: Slot, chain: Link[], o: TrainLandOptions): Promise<TrainOut> {
const { env, t, i, pin, run, rerun } = s;
const main = mainNow(ctx, env);
if ('error' in main) return outRefused({ code: 'QUEUE_PROOF_MISSING', message: main.error }, s.lines, rerun, run);
if (main.tree === s.target.tree) return confirmed(s, main.sha, `main: ${main.sha} is ${label(s)} exactly`);
lib/ship-queue-train-record.ts
+12 -3
View File
@@ -187,6 +187,15 @@ const firstOf = <T>(k: RepoKeys, read: (key: string) => T | null): T | null => {
export const loadFreeze = (repoKeys: RepoKeys) => firstOf(repoKeys, key => readJson<FreezeRecord>(freezePath(key), 'freeze'));
export const loadReady = (repoKeys: RepoKeys, pr: string, head: string) => firstOf(repoKeys, key => readJson<ReadyReceipt>(readyPath(key, pr, head), 'ready'));
export const activeFreeze = (repoKeys: RepoKeys): FreezeRecord | null => firstOf(repoKeys, key => { const f = readJson<FreezeRecord>(freezePath(key), 'freeze'); return f && !f.cleared_at ? f : null; });
/**
* When a freeze-phase land lock's freeze is over, the cleared_at that proves it: the notes ref fetched, no freeze
* active, and the last freeze cleared at or after the lock was written. Absence of a record is never proof.
*/
export function freezeLockCleared(repoRoot: string, remote: string, repoKeys: RepoKeys, lock: { phase_at?: string; started_at: string }): string | null {
if (!rebuildFromNotes(repoRoot, remote).fetched || activeFreeze(repoKeys)) return null;
const last = loadFreeze(repoKeys);
return last?.cleared_at && Date.parse(last.cleared_at) >= Date.parse(lock.phase_at ?? lock.started_at) ? last.cleared_at : null;
}
export const loadWave = (nonce: string) => (/^[A-Za-z0-9._-]{1,128}$/.test(nonce) ? readJson<WaveRecord>(wavePath(nonce), 'wave') : null);
/** Every wave of this repository, newest first. */
@@ -325,13 +334,13 @@ export function forgetNotes(repoRoot: string, names: string[], remote = 'origin'
}
/** Rebuild local records from the notes ref: a note whose record is missing locally, or newer than the local copy, is written. */
export function rebuildFromNotes(repoRoot: string, remote = 'origin'): { restored: number; read: number; detail: string } {
export function rebuildFromNotes(repoRoot: string, remote = 'origin'): { restored: number; read: number; detail: string; fetched: boolean } {
const f = fetchNotes(repoRoot, remote);
const list = git(repoRoot, ['notes', `--ref=${NOTES_REF}`, 'list']);
if (!list.ok || !list.out.trim()) return { restored: 0, read: 0, detail: f.detail };
if (!list.ok || !list.out.trim()) return { restored: 0, read: 0, detail: f.detail, fetched: f.ok };
const records = list.out.split('\n').map(l => l.split(' ')[0]!).filter(Boolean)
.map(blob => parseNote(git(repoRoot, ['cat-file', 'blob', blob]).out)).filter((r): r is AnyRecord => !!r);
const fresh = records.filter(newerOrMissing);
for (const r of fresh) writeJsonAtomic(recordPath(r), r);
return { restored: fresh.length, read: records.length, detail: f.detail };
return { restored: fresh.length, read: records.length, detail: f.detail, fetched: f.ok };
}
lib/ship-queue-train-verify.ts
+47 -4
View File
@@ -6,8 +6,9 @@
* head, or at a head whose tree is the slot's prefix after it modulo stamps
* (from slot 2 on, the lane merges main into its pin so GitHub can merge it),
* with main's tree equal to the prefix before the PR (pre-merge:
* MERGE_CONDITION) or after it (post-merge confirmation), modulo stamps
* (lib/tree-receipt.ts). When a train stops partway (a mismatch or a moved
* MERGE_CONDITION, once the head carries its own stamp above main's version,
* #3531) or after it (post-merge confirmation, once GitHub reports the PR
* MERGED), modulo stamps (lib/tree-receipt.ts). When a train stops partway (a mismatch or a moved
* head after earlier slots merged) the queue freezes: the freeze is written
* locally and to the notes ref first, then the backstop gate is dispatched on
* main's published prefix at once. One repair train is admitted under the
@@ -25,6 +26,9 @@ import {
} from './ship-queue-train';
import { mergeChangelogs, ownEntryAboveMain, parseChangelog } from './changelog-check';
import { buildReceipt } from './tree-receipt';
import { versionAtRef } from './restamp';
import { cmpVersion, parseVersion } from './version-source';
import { readLock, writeLock } from './ship-queue-land-lock';
import { genericBackstop } from './ship-queue-train-gate';
const Q = 'gstack-ship-queue';
@@ -110,6 +114,31 @@ function published(env: TrainEnv, t: TrainRecord, i: number, main: string): { sa
return { same: true, detail: `same modulo stamps (${r.files.filter(f => f.verdict === 'stamp').length} stamp file(s)) except ${cl}, main ${parent.slice(0, 12)}'s ${cl} plus #${t.slots[i]!.pr}'s own entry` };
}
/**
* The slot stamp a hand-stamped head must carry before it may merge (#3531, train 33 shipped #3527 under #3526's
* version): its VERSION above main's and, with a CHANGELOG, its top section headed with that version directly above
* main's top release, with no [Unreleased] section when main has none. The failing rule is named.
*/
function stampProof(env: TrainEnv, head: string, main: string): { version: string } | { rule: string; detail: string } {
const root = env.ctx.repoRoot;
const hv = versionAtRef(root, head);
const mv = versionAtRef(root, main);
if ('error' in hv) return { rule: 'version', detail: `the head's version cannot be read (${hv.error})` };
if ('error' in mv) return { rule: 'version', detail: `main's version cannot be read (${mv.error})` };
const [h, m] = [parseVersion(hv.version), parseVersion(mv.version)];
if (!h || !m || cmpVersion(h, m) <= 0) return { rule: 'version', detail: `the head's ${hv.rel} ${hv.version} is not above main's ${mv.version}` };
const cl = env.policy.policy.changelog;
if (!cl) return { version: hv.version };
const sections = (rev: string) => parseChangelog(git(root, ['show', `${rev}:${cl}`]).out).sections;
const [ours, theirs] = [sections(head), sections(main)];
const mainTop = theirs.find(s => !s.unreleased)?.version ?? null;
if (ours[0]?.version !== hv.version) return { rule: 'changelog', detail: `${cl}'s top section at the head is [${ours[0] ? ours[0].version ?? 'Unreleased' : 'none'}], not [${hv.version}]` };
const below = ours.slice(1).find(s => !s.unreleased)?.version ?? null;
if (below !== mainTop) return { rule: 'changelog', detail: `${cl}'s release below [${hv.version}] at the head is [${below ?? 'none'}], not main's top release [${mainTop ?? 'none'}]` };
if (!theirs.some(s => s.unreleased) && ours.some(s => s.unreleased)) return { rule: 'changelog', detail: `${cl} at the head keeps an [Unreleased] section that main does not have` };
return { version: hv.version };
}
/**
* Freeze the queue: the record goes to local state and the notes ref before
* anything remote, then the backstop gate is dispatched on main's sha.
@@ -225,8 +254,13 @@ export async function verifyTrain(ctx: QueueContext, o: { pr: string; train: str
const before = t.prefixes[i]!;
const post = published(env, t, i, main);
const lines = [`TRAIN: ${t.nonce} slot ${i + 1}/${t.slots.length} #${o.pr} pinned ${t.slots[i]!.head}`];
if (post.same) return confirmMerged(env, t, o.pr, main, after.index, [...lines, `main: ${main} = T${after.index} (${post.detail})`]);
const pre = sameModuloStamps(env, before.commit, main);
if (post.same) {
const state = prState(ctx, o.pr);
if (state === 'MERGED') return confirmMerged(env, t, o.pr, main, after.index, [...lines, `main: ${main} = T${after.index} (${post.detail})`]);
if (state !== 'OPEN' || !pre.same) return outRefused({ code: 'QUEUE_PROOF_MISSING', message: `main ${main.slice(0, 12)} is T${after.index} modulo stamps (${post.detail}), but GitHub reports #${o.pr} as ${state}, not MERGED; nothing is confirmed and the queue is not frozen` }, lines, `gh pr view ${o.pr} --json state,mergeCommit; rerun once GitHub reports #${o.pr} MERGED: ${Q} verify --pr ${o.pr} --train ${t.nonce}`, run);
lines.push(`main: also T${after.index} modulo stamps, but #${o.pr} is OPEN, so this is the pre-merge check (a stamp-only PR)`);
}
if (!pre.same) {
return stopped(env, t, 'QUEUE_TRAIN_TREE_MISMATCH', `main ${main.slice(0, 12)} is neither T${before.index} nor T${after.index} modulo stamps (vs T${after.index}: ${post.detail})`, main, lines, !o.noFetch);
}
@@ -239,7 +273,10 @@ export async function verifyTrain(ctx: QueueContext, o: { pr: string; train: str
if ('error' in info) return outRefused({ code: 'QUEUE_PROOF_MISSING', message: info.error }, lines, `${Q} verify --pr ${o.pr} --train ${t.nonce}`, run);
const shipped = bindReceipt(info.body, o.pr, [t.slots[i]!.head, head]);
if (!shipped.ok) return outRefused({ code: shipped.code, message: shipped.message }, [...lines, shipped.line], `#${o.pr}'s lane runs /ship and READYs again; then ${Q} train build --prs <pins>`, run);
return outOk([...lines, `main: ${main} = T${before.index} (${pre.detail})`, `head: ${head} (${stampOnly.detail})`, `${shipped.line} (binds ${shipped.head === t.slots[i]!.head ? 'the pinned head' : 'the current head'})`,
if (!headRef) return outRefused({ code: 'QUEUE_PROOF_MISSING', message: `#${o.pr}'s current head was not read (${o.noFetch ? '--no-fetch' : 'the head fetch failed'}), and the pin it falls back to is unstamped by design, so the stamp cannot be proved` }, lines, `${Q} verify --pr ${o.pr} --train ${t.nonce}`, run);
const stamp = stampProof(env, head, main);
if ('rule' in stamp) return outRefused({ code: 'QUEUE_TRAIN_UNSTAMPED', message: `#${o.pr}'s head ${head.slice(0, 12)} is not stamped above main: ${stamp.rule}: ${stamp.detail}` }, lines, `#${o.pr}'s lane runs gstack-restamp --next on its head, commits what restamp staged, pushes, then ${Q} verify --pr ${o.pr} --train ${t.nonce}`, run);
return outOk([...lines, `main: ${main} = T${before.index} (${pre.detail})`, `head: ${head} (${stampOnly.detail})`, `${shipped.line} (binds ${shipped.head === t.slots[i]!.head ? 'the pinned head' : 'the current head'})`, `STAMP_VERSION: ${stamp.version}`,
`VERIFY_TRAIN: #${o.pr} may merge as slot ${i + 1} of train ${t.nonce} (green run ${gate.run?.id ?? '-'}, ${env.train.full_gate_check} success)`, `MERGE_CONDITION: sha=${head}`],
`merge #${o.pr} with expected head ${head.slice(0, 12)} (gstack-restamp stamps at merge), then ${Q} verify --pr ${o.pr} --train ${t.nonce}`, run);
}
@@ -262,6 +299,12 @@ export function confirmMerged(env: TrainEnv, t: TrainRecord, pr: string, main: s
freeze.updated_at = freeze.cleared_at;
persist(env.ctx.repoRoot, [freeze], { remote: env.remote });
lines.push(`FREEZE: cleared (${freeze.cleared_by})`);
const held = readLock(env.ctx.repoRoot, env.remote);
if (held.state === 'held' && held.lock.phase === 'freeze') {
lines.push(writeLock(env.ctx.repoRoot, held.sha, { released: true, by: `verify #${pr} via train ${t.nonce}`, at: nowIso() }, env.remote).ok
? 'LOCK: released the freeze lock'
: 'LOCK: the freeze lock could not be released; the next land or train land takes it over once it reads this cleared freeze');
}
const rebuild = freeze.unmerged.length ? `${Q} train build --prs ${pinList(freeze.unmerged)}` : '';
const unread = freeze.unreadable ?? [];
const check = unread.length ? `${unread.map(u => `gh pr view ${u.pr} --json state,headRefOid`).join(' && ')} (unreadable at the freeze: ${unread.map(u => `#${u.pr}@${u.head.slice(0, 12)}`).join(', ')}); seat each that is OPEN in a train` : '';
lib/ship-queue.ts
+1 -1
View File
@@ -175,7 +175,7 @@ function conflictHint(mergeable: unknown, ci: { green: boolean; none: boolean },
if (mergeable !== 'CONFLICTING') return null;
const ref = branch && /^[\w.][\w./-]*$/.test(branch) ? `--ref ${branch}` : '--ref <branch>';
if (ci.none && fork) return `ci-hint: PR conflicts with the base (no merge ref, so pull_request CI did not run on ${head.slice(0, 12)}) and its head is on a fork, so the free workflows cannot be dispatched here; the coordinator decides (advisory)`;
if (ci.none) return `ci-hint: PR conflicts with the base (no merge ref, so pull_request CI did not run on ${head.slice(0, 12)}); dispatch the free workflows on the branch (gh workflow run free-tests.yml ${ref} && gh workflow run windows-free-tests.yml ${ref}), clear the pregate lanes with their run URLs (gstack-pregate clear --remote <lane> --run-url <run>), then rerun ready (advisory)`;
if (ci.none) return `ci-hint: PR conflicts with the base (no merge ref, so pull_request CI did not run on ${head.slice(0, 12)}); dispatch the free workflows on the branch (gh workflow run free-tests.yml ${ref} && gh workflow run windows-smoke.yml ${ref}), clear the pregate lanes with their run URLs (gstack-pregate clear --remote <lane> --run-url <run>), then rerun ready (advisory)`;
if (ci.green) return `ci-hint: PR conflicts with the base (no merge ref); the runs at ${head.slice(0, 12)} were not pull_request runs (dispatched on the branch, or push-triggered), and they count (advisory)`;
return null;
}
lib/ship-release.ts
+3 -3
View File
@@ -285,17 +285,17 @@ export async function finish(argv: readonly string[], cwd: string): Promise<Repo
const r = new Report();
const list = lanes(argv) as { label: string; command: string }[];
const policy = shipPolicy(cwd, base);
if (!commitRelease(cwd, releasePaths(policy, cwd, 'HEAD'), flag('--trailer'), r)) return r;
if (!commitRelease(cwd, releasePaths(policy, cwd, 'HEAD', base), flag('--trailer'), r)) return r;
for (const cmd of all('--build')) {
const b = sh(cwd, cmd);
r.add(`BUILD: ${b.status === 0 ? 'ok' : `FAIL exit=${b.status}`} ${cmd.slice(0, 120)}`);
if (b.status !== 0) return r.stop(`Build failed or prerequisite missing: ${firstLine(b.stderr || b.stdout)}; repair it, then repeat stage 1`);
}
if (!all('--build').length) r.add('BUILD: none passed (record the inspected sources, or pass --build per declared command)');
const release = releasePaths(policy, cwd, flag('--reviewed-tree')!);
const release = releasePaths(policy, cwd, flag('--reviewed-tree')!, base);
const routed = route(cwd, base, flag('--reviewed-tree')!, flag('--reviewed-base'), release, { docsRechecked: has('--docs-rechecked'), contextReviewed: flag('--context-reviewed') }, r);
if (!routed) return r;
if (!docsFresh(cwd, base, flag('--docs-post')!, [...routed.base, ...routed.context, ...stampedFiles(policy, cwd, flag('--reviewed-tree')!)], all('--docs-requested'), flag('--docs-result'), r)) return r;
if (!docsFresh(cwd, base, flag('--docs-post')!, [...routed.base, ...routed.context, ...stampedFiles(policy, cwd, flag('--reviewed-tree')!, base)], all('--docs-requested'), flag('--docs-result'), r)) return r;
if (!list.length && !has('--no-lanes-approved')) return r.stop('no test lanes: require Step 5\'s explicit untested-scope approval (rerun with --no-lanes-approved) or run Steps 5-15');
const allow = evidenceExempt(release).join(',');
const check = (l: { label: string; command: string }) => bin(cwd, 'gstack-evidence', ['check', '--label', l.label, '--expect-cmd', l.command, '--max-age', '24', '--allow-paths', allow], 60_000).stdout;
package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "gstack",
"version": "1.92.47",
"version": "1.92.61",
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
"license": "MIT",
"type": "module",
plan-devex-review/dx-hall-of-fame.md
+2 -2
View File
@@ -76,8 +76,8 @@ unknown is reserved for existing behavior you could not inspect. In a live audit
you never triggered are unknown, not extrapolated.
**Caps.** Name the path and item behind every cap and show the corrected error line.
- **4/10 (Critical finding, title `CAP4:<path>:<item>`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as `--yes`).
- **6/10 (High finding, title `CAP6:<path>:<item>`):** a path lacks an applicable fix, recovery, context or trace item.
- **4/10 (Critical finding, title `CAP4:<path>:<item> <what fails>`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as `--yes`).
- **6/10 (High finding, title `CAP6:<path>:<item> <what fails>`):** a path lacks an applicable fix, recovery, context or trace item.
Every mode, DX TRIAGE included, reports every CAP4 finding.
plan-devex-review/sections/review-sections.md
+1 -1
View File
@@ -162,7 +162,7 @@ persona from 0A.
Load reference: Read the "## Pass 3" section from `~/.claude/skills/gstack/plan-devex-review/dx-hall-of-fame.md`
before scoring: the full contract, calibration examples and the standards mapping.
Score every error path against the agent-ready error contract (https://github.com/garrytan/gstack/blob/main/docs/agent-ready-errors.md): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or the precise thing a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). Usage and validation errors rejected before any work need only what, fix, code, redaction and channel; items apply per surface, N/A needs a reason, an item the plan never specifies for a path it introduces is a gap, and unknown is reserved for existing behavior you could not inspect. Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure that names no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item>" (caps the Error Messages score at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item>" (caps it at 6/10).
Score every error path against the agent-ready error contract (https://github.com/garrytan/gstack/blob/main/docs/agent-ready-errors.md): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or what a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). Usage and validation errors rejected before any work need only what, fix, code, redaction and channel; items apply per surface, N/A needs a reason, an item the plan never specifies for a path it introduces is a gap, and unknown is reserved for existing behavior you could not inspect. Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure naming no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item> <what fails>" (caps Error Messages at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item> <what fails>" (caps it at 6/10).
The "## Pass 3" Hall of Fame section loaded above holds the full contract: the item table,
the per-surface channel rules and the evidence states.
scripts/harness-version.json
+1 -1
View File
@@ -17,7 +17,7 @@
"scripts/lib/published-text.ts": {"blob":"3f2e4d616b5dd0204035710c66b64d49055c5714","since":6},
"scripts/lib/shard-engine.ts": {"blob":"dc714d7d897aed185c0ec3d2c92ad22d954039e5","since":6,"nonBehavioral":"createShardSandbox drops the Capy launch markers (GSTACK_SESSION_KIND, GSTACK_CAPY, GSTACK_EPHEMERAL, GSTACK_INSTALL_VERDICT) from shard environments; CI never sets them, so no case's input or verdict changes there, and on a Capy machine fixtures now see the session kind they set themselves instead of the installer's"},
"scripts/test-paid-shards.ts": {"blob":"5439d0a8f6e1569fa7c924af20d67c30c81d5d30","since":6,"nonBehavioral":"credential rejections are classed infra instead of assertion: no case's pass or fail changes; only failed trials whose provider rejected the run's key move class, so no rerun spends on a rejected key"},
"scripts/test-pr-profile.ts": {"blob":"911e393d0c36c050c3a68db5a81ef21701bfd5ae","since":6,"nonBehavioral":"selection only: fragment exemption narrowed to .md files directly in the fragments dir; changed-file lists include a rename's old path; no case prompt, fixture, judge or verdict logic changes (GST-400)"},
"scripts/test-pr-profile.ts": {"blob":"44ae88bebf8f716f70a3bd836c8bedb03e6653e8","since":6,"nonBehavioral":"FREE_ONLY_PR_FILES gains scripts/lib/shard-jobs.ts (free-runner module) and .github/workflows/windows-smoke.yml (a workflow no paid case reads); both change only which PR paths select paid cases, never how a case runs or is judged"},
"scripts/test-strict-output.ts": {"blob":"01185d20dc9761b04b9b2f751ec98664fe0eaf93","since":6},
"test/helpers/agent-sdk-runner.ts": {"blob":"210bba92d241a90df578b2c7fc943ac5755d0025","since":6},
"test/helpers/auq-substance-panel.ts": {"blob":"4881260ef793668e4963a062db6d06a5e1f7a40f","since":6},
scripts/lib/ci-history.ts
+48 -7
View File
@@ -259,7 +259,38 @@ export function readFlakeLedgerDir(dir: string): FlakeLedgerEntry[] {
export const FLAKE_POLICY = { maxRate: 0.05, minRuns: 20 } as const;
export interface FlakyFile { file: string; flakyRuns: number; entries: number; rate: number; enforced: boolean; failing: boolean }
export interface FlakeWindow { runs: number; flakyRuns: number; since: string; files: FlakyFile[]; enforceable: boolean }
/** `skipped`: reused main pushes left out of the window (reuseChecker); `note`: runs that could not be checked. */
export interface FlakeWindow { runs: number; flakyRuns: number; since: string; files: FlakyFile[]; enforceable: boolean; skipped?: number; note?: string }
/** The most job listings one reuseChecker spends. */
export const REUSE_LOOKUPS_MAX = 40;
/**
* Finds runs that reused their merged pull request's verified run (#3709): such
* a run uploads no artifact and its `reuse` job succeeds, because every suite
* job was skipped. It holds no flake evidence, so flake counts leave it out of
* their runs and their denominator (#3725). A run with any artifact costs no
* call. A run with none costs one job listing, at most REUSE_LOOKUPS_MAX per
* checker; past the cap, or when the listing fails, the run counts as it did
* before #3725 and note() says so.
*/
export function reuseChecker(client: GhClient, repo: string) {
let lookups = 0;
let unchecked = 0;
let reason = '';
return {
reused(run: Pick<CiRun, 'id' | 'attempt'>, artifactNames: string[]): boolean {
if (artifactNames.length) return false;
if (lookups >= REUSE_LOOKUPS_MAX) { unchecked++; reason ||= `cap of ${REUSE_LOOKUPS_MAX} job listings reached`; return false; }
lookups++;
try {
return listRunJobs(client, repo, run.id, run.attempt).some(job => job.name === 'reuse' && job.conclusion === 'success');
} catch (error) { unchecked++; reason ||= unavailableReason(error); return false; }
},
note: (): string | undefined => (unchecked
? `${unchecked} run(s) with no artifacts not checked for main-push reuse (${reason}); counted as before #3725` : undefined),
};
}
/**
* Rank files by the share of runs in which they needed the retry pass. A file
@@ -286,9 +317,10 @@ export function rankFlakyFiles(runEntries: Map<number, FlakeLedgerEntry[]>, runC
/**
* Read the flake ledgers of main's completed free-tests runs: the runs created
* in the last `sinceDays`, widened to the newest `minRuns` runs (within
* `maxLookbackDays`) so the 5% rule has a window of at least 20 runs. A run
* with no `flake-ledger-*` artifact had no flaky passes (the upload is skipped
* when the ledger file does not exist).
* `maxLookbackDays`) so the 5% rule has a window of at least 20 runs. Reused
* main pushes (reuseChecker) are left out and older runs take their place. A
* run with no `flake-ledger-*` artifact had no flaky passes (the upload is
* skipped when the ledger file does not exist).
*/
export function readCiFlakeLedgers(opts: { client: GhClient; repo: string; sinceDays: number; now: number; cacheDir: string;
minRuns?: number; maxLookbackDays?: number; workflow?: string }): FlakeWindow {
@@ -297,16 +329,25 @@ export function readCiFlakeLedgers(opts: { client: GhClient; repo: string; since
const all = listWorkflowRuns(opts.client, opts.repo, opts.workflow ?? 'free-tests.yml',
{ since: iso(Math.max(opts.sinceDays, opts.maxLookbackDays ?? 60)), branch: 'main' })
.filter(run => run.conclusion === 'success' || run.conclusion === 'failure');
const recent = all.filter(run => run.createdAt >= iso(opts.sinceDays));
const window = recent.length >= minRuns ? recent : all.slice(0, minRuns);
const since = iso(opts.sinceDays);
const recent = all.filter(run => run.createdAt >= since).length;
const fetcher = historyFetcher(opts.client);
const reuse = reuseChecker(opts.client, opts.repo);
const window: CiRun[] = [];
let skipped = 0;
for (const [index, run] of all.entries()) {
if (index >= recent && window.length >= minRuns) break;
if (reuse.reused(run, fetcher.listArtifacts(opts.repo, run.id).map(artifact => artifact.name))) skipped++;
else window.push(run);
}
const runEntries = new Map<number, FlakeLedgerEntry[]>();
for (const run of window) {
const dirs = downloadRunArtifacts({ repo: opts.repo, run, cacheDir: opts.cacheDir, fetcher, match: name => name.startsWith('flake-ledger-') });
const entries = dirs.flatMap(readFlakeLedgerDir);
if (entries.length) runEntries.set(run.id, entries);
}
return rankFlakyFiles(runEntries, window.length, window[window.length - 1]?.createdAt ?? iso(opts.sinceDays));
const note = reuse.note();
return { ...rankFlakyFiles(runEntries, window.length, window[window.length - 1]?.createdAt ?? since), skipped, ...(note ? { note } : {}) };
}
/** Markdown table of one run's merged flake ledgers, for a job summary. */
scripts/lib/shard-jobs.ts
+422
View File
@@ -0,0 +1,422 @@
/**
* Sub-shards inside one CI job: `--ci-run <plan> --shard i --result r --jobs k`
* (GSTA-203 PR 2, principle "buy cores, not shards").
*
* A CI shard's files split into k disjoint, identity-bound sub-shards that run
* side by side through the free runner's own process-shard path (one bun child
* each, serial inside, own sandbox, group kill, log), never Bun's worker
* `--parallel` (decision V3/D6 in scripts/test-free-shards.ts). The rules:
*
* - k = 1 (the default) is today's single child, unchanged.
* - Files in TREE_MUTATING or SLOW_UNDER_LOAD run after the parallel
* sub-shards settle, serially, in one trailing sub-shard.
* - The parallel phase uses the concurrent-phase home guard; every child
* uses the shared guard inside it, exactly as full-suite mode does.
* - The whole CI shard keeps its k = 1 wall: the parallel sub-shards get it,
* and the serial sub-shard gets what is left of it, so k never extends a
* deadline; every launched child settles before the parent goes on.
* - The parent merges every sub-shard result before the single retry pass,
* so the five-file retry cap stays per CI shard; a missing, duplicate or
* incomplete sub-shard result fails the shard.
* - Sub-shard runs never write the duration seed (only `--record-durations`
* does, from one-file children).
*
* Per-file test and expect() counts come from bun's JUnit reporter, which
* attributes both per file inside a multi-file child on the pinned Bun 1.4.2
* (the console stream is byte-identical with or without it). Counts are
* evidence for k > 1 comparisons; they never decide a verdict.
*/
import * as fs from 'node:fs';
import * as os from 'node:os';
import type { FreeCiPlan, FreeCiResult, FreeShardOutcome } from '../test-free-shards';
import { guardFreeHome, sharedFreeHome, type FreeHomeGuard, type FreeHomeGuardFactory } from './free-home-guard';
import { normalizeRelativePath } from './shard-engine';
/**
* Files that pass alone but time out as a sub-shard under load. Each entry
* names the CI run id where it timed out (FREE_SUBSHARD_TIMEOUT prints it).
* They run in the trailing serial sub-shard under `--jobs`; at k = 1 nothing
* changes. Keys are pinned against the live file census by
* test/shard-jobs.test.ts. Never raise a timeout instead of adding an entry.
*/
export const SLOW_UNDER_LOAD: Record<string, { reason: string; runId: string }> = {};
export const FREE_JOBS_INVALID = 'FREE_JOBS_INVALID';
export const FREE_SUBSHARD_TIMEOUT = 'FREE_SUBSHARD_TIMEOUT';
export const FREE_SUBSHARD_REJECTED = 'FREE_SUBSHARD_REJECTED';
const DOCS = 'docs/TESTING_INTERNALS.md#sub-shards-inside-one-ci-job';
const JOBS_FIX = `--jobs applies only to --ci-run and defaults to 1; GSTACK_FREE_JOBS sets the local full-suite process count and never changes --ci-run. See ${DOCS}`;
/** `--jobs <k>` at parse time: digits only, at least 1. */
export function parseJobsFlag(raw: string | undefined): number {
if (raw === undefined || !/^\d+$/.test(raw) || Number(raw) < 1) {
throw new Error(`${FREE_JOBS_INVALID}: --jobs needs a positive whole number; got ${JSON.stringify(raw ?? '')}. ${JOBS_FIX}`);
}
return Number(raw);
}
export function rejectJobsWithoutCiRun(): never {
throw new Error(`${FREE_JOBS_INVALID}: --jobs needs --ci-run. ${JOBS_FIX}`);
}
/** `--jobs <k>` against the planned shard: k may not exceed its file count (an empty shard allows 1). */
export function validateJobsForShard(jobs: number, fileCount: number, shard: number): void {
const max = Math.max(1, fileCount);
if (jobs > max) {
throw new Error(`${FREE_JOBS_INVALID}: --jobs ${jobs} is more than CI shard ${shard}'s ${fileCount} file(s); use 1 to ${max}. ${JOBS_FIX}`);
}
}
export interface FileTestCounts { tests: number; pass: number; fail: number; skip: number; todo: number; expects: number }
export interface ShardTestCounts {
source: 'bun-junit';
/** Every reported file was planned and the per-file tests add up to bun's own `Ran N tests` count. */
complete: boolean;
problem?: string;
files: Record<string, FileTestCounts>;
totals: FileTestCounts;
}
const zeroCounts = (): FileTestCounts => ({ tests: 0, pass: 0, fail: 0, skip: 0, todo: 0, expects: 0 });
const addCounts = (into: FileTestCounts, from: FileTestCounts) => {
for (const key of Object.keys(into) as Array<keyof FileTestCounts>) into[key] += from[key];
};
const TAG = /<(\/?)(testsuites|testsuite|testcase|failure|error|skipped)\b((?:[^>"']|"[^"]*"|'[^']*')*?)(\/?)>/g;
const ATTR = /([\w:-]+)="([^"]*)"/g;
const ENTITIES: Record<string, string> = { amp: '&', lt: '<', gt: '>', quot: '"', apos: "'" };
const decode = (text: string) => text.replace(/&(#x[0-9a-f]+|#\d+|\w+);/gi, (whole, entity: string) => entity[0] === '#'
? String.fromCodePoint(entity[1].toLowerCase() === 'x' ? parseInt(entity.slice(2), 16) : Number(entity.slice(1)))
: ENTITIES[entity] ?? whole);
const attributes = (text: string): Record<string, string> =>
Object.fromEntries([...text.matchAll(ATTR)].map(([, key, value]) => [key, decode(value)]));
/**
* Per-file counts from bun's JUnit XML. A testcase is a fail when it holds a
* failure or error, a todo when skipped with message TODO, a skip when
* skipped otherwise, else a pass; expects sum each testcase's assertions.
* Printed paths are matched back to planned files. Bun prints them relative
* to the child's real cwd, so a plain relative path must match exactly; an
* absolute or ../-prefixed path matches only when what precedes the planned
* file is the child's root. Files that never ran a test are absent.
*/
export function parseJunitCounts(xml: string | null, planned: readonly string[], testsRan: number | null, root = process.cwd()): ShardTestCounts {
const files: Record<string, FileTestCounts> = {};
const totals = zeroCounts();
const result = (complete: boolean, problem?: string): ShardTestCounts =>
({ source: 'bun-junit', complete, ...(problem ? { problem } : {}), files, totals });
if (xml === null) return result(false, 'bun wrote no JUnit report');
const plannedSet = new Set(planned);
const unknown = new Set<string>();
const roots = new Set([root, (() => { try { return fs.realpathSync(root); } catch { return root; } })()].map(normalizeRelativePath));
const isRoot = (prefix: string) => [...roots].some(dir => dir === prefix || dir.endsWith(`/${prefix}`));
const canonical = (printed: string) => {
const normalized = normalizeRelativePath(printed);
const stripped = normalized.replace(/^(?:\.{1,2}\/)+/, '');
if (plannedSet.has(stripped)) return stripped;
if (stripped === normalized && !/^(?:\/|[A-Za-z]:\/)/.test(stripped)) return stripped;
const matches = planned.filter(file => stripped.endsWith(`/${file}`) && isRoot(stripped.slice(0, -file.length - 1)));
return matches.length === 1 ? matches[0] : stripped;
};
let current: { file: string; state: 'pass' | 'fail' | 'skip' | 'todo'; expects: number } | null = null;
const finish = () => {
if (!current) return;
const entry = files[current.file] ??= zeroCounts();
entry.tests += 1;
entry[current.state] += 1;
entry.expects += current.expects;
if (!plannedSet.has(current.file)) unknown.add(current.file);
current = null;
};
for (const [, closing, name, body, selfClosing] of xml.matchAll(TAG)) {
if (name === 'testcase') {
if (closing) { finish(); continue; }
const attrs = attributes(body);
current = { file: canonical(attrs.file ?? ''), state: 'pass', expects: Number(attrs.assertions ?? 0) || 0 };
if (selfClosing) finish();
} else if (current && !closing && (name === 'failure' || name === 'error')) {
current.state = 'fail';
} else if (current && !closing && name === 'skipped' && current.state === 'pass') {
current.state = attributes(body).message === 'TODO' ? 'todo' : 'skip';
}
}
finish();
for (const entry of Object.values(files)) addCounts(totals, entry);
if (unknown.size) return result(false, `JUnit named unplanned file(s): ${[...unknown].sort().join(', ')}`);
if (testsRan === null) return result(false, "bun's summary has no test count");
if (totals.tests !== testsRan) return result(false, `JUnit counted ${totals.tests} tests, bun's summary ${testsRan}`);
return result(true);
}
const JUNIT_MAX_BYTES = 64 * 1024 * 1024;
/** The child-written report, only when it is a regular file of bounded size (never a link, FIFO or device). */
export function readJunitReport(file: string): string | null {
let fd: number;
try { fd = fs.openSync(file, fs.constants.O_RDONLY | (fs.constants.O_NOFOLLOW ?? 0) | (fs.constants.O_NONBLOCK ?? 0)); } catch { return null; }
try {
const stat = fs.fstatSync(fd);
return stat.isFile() && stat.size <= JUNIT_MAX_BYTES ? fs.readFileSync(fd, 'utf8') : null;
} catch { return null; } finally { fs.closeSync(fd); }
}
/** One shard's counts from its sub-shards' counts; complete only when every part is and no file repeats. */
export function mergeTestCounts(parts: Array<ShardTestCounts | undefined>): ShardTestCounts | undefined {
if (parts.every(part => !part)) return undefined;
const files: Record<string, FileTestCounts> = {};
const totals = zeroCounts();
const problems: string[] = [];
parts.forEach((part, index) => {
if (!part) { problems.push(`sub-shard ${index + 1} has no counts`); return; }
if (!part.complete) problems.push(`sub-shard ${index + 1}: ${part.problem ?? 'incomplete'}`);
for (const [file, counts] of Object.entries(part.files)) {
if (files[file]) problems.push(`${file} counted twice`);
files[file] = { ...counts };
}
addCounts(totals, part.totals);
});
return { source: 'bun-junit', complete: problems.length === 0, ...(problems.length ? { problem: problems.join('; ') } : {}), files, totals };
}
export interface SubShardAssignment {
/** `<plan id prefix>:<shard>.<index>`: binds a result to one plan, shard and file set. */
id: string;
index: number;
phase: 'parallel' | 'serial';
files: string[];
}
/**
* Split one CI shard into disjoint sub-shards: up to k parallel ones packed by
* `pack`, then one serial sub-shard for the files `isSerial` names. Throws
* unless the assignments cover the shard's files exactly once.
*/
export function assignSubShards(
planId: string, shard: { shard: number; files: string[] }, jobs: number,
isSerial: (file: string) => boolean, pack: (files: string[], count: number) => string[][],
): SubShardAssignment[] {
const serial = shard.files.filter(isSerial);
const parallel = shard.files.filter(file => !isSerial(file));
const groups = parallel.length ? pack(parallel, Math.min(jobs, parallel.length)).filter(files => files.length) : [];
const assignments: SubShardAssignment[] = [
...groups.map(files => ({ phase: 'parallel' as const, files })),
...(serial.length ? [{ phase: 'serial' as const, files: serial }] : []),
].map((entry, index) => ({ id: `${planId.slice(0, 12)}:${shard.shard}.${index + 1}`, index: index + 1, ...entry }));
const assigned = assignments.flatMap(entry => entry.files);
if (new Set(assigned).size !== assigned.length || assigned.length !== shard.files.length
|| !shard.files.every(file => assigned.includes(file))) {
throw new Error(`${FREE_SUBSHARD_REJECTED}: sub-shard assignment for CI shard ${shard.shard} does not cover its files exactly once`);
}
return assignments;
}
export type SubShardSettlement = { assignment: SubShardAssignment } & ({ outcome: FreeShardOutcome } | { error: string });
/** Why one sub-shard's result cannot stand for its assignment; empty when it can. */
export function subShardProblems(settled: SubShardSettlement): string[] {
const { assignment } = settled;
if (!('outcome' in settled)) return [`${assignment.id} has no result (${settled.error})`];
const { outcome } = settled;
const problems: string[] = [];
if (JSON.stringify(outcome.files) !== JSON.stringify(assignment.files)) problems.push(`${assignment.id} reported another file set`);
const summary = outcome.summary;
if (!summary || (summary.sawTerminalSummary && summary.filesRan !== assignment.files.length)) {
problems.push(`${assignment.id} reported ${summary?.filesRan ?? 'no'} executed file(s) for ${assignment.files.length} assigned`);
} else if (outcome.status === 'passed' && (summary.filesRan !== assignment.files.length || summary.sawTerminalSummary !== true
|| outcome.exitCode !== 0 || outcome.failingFiles.length || outcome.unattributedFailures)) {
problems.push(`${assignment.id} passed without a complete execution summary`);
}
return problems;
}
export interface SubShardRecord {
id: string;
phase: SubShardAssignment['phase'];
files: string[];
status: FreeShardOutcome['status'] | 'missing';
exitCode: number | null;
elapsedMs: number;
summary?: FreeShardOutcome['summary'];
failingFiles: string[];
unattributedFailures: number;
}
/**
* One CI shard outcome from every sub-shard: the planned file order, summed
* summaries, the union of failing files, and the worst status. Every rejected
* sub-shard adds unattributed evidence, which also vetoes the retry pass.
*/
export function mergeSubShardOutcomes(
shard: { shard: number; files: string[] }, settled: SubShardSettlement[], elapsedMs: number, extraProblems: string[] = [],
): { outcome: FreeShardOutcome; records: SubShardRecord[]; problems: string[] } {
const problems = [...extraProblems];
const assigned = settled.flatMap(entry => entry.assignment.files);
if (new Set(assigned).size !== assigned.length) problems.push('a file is assigned to more than one sub-shard');
if (assigned.length !== shard.files.length || !shard.files.every(file => assigned.includes(file))) problems.push('sub-shards do not cover the CI shard');
for (const entry of settled) problems.push(...subShardProblems(entry));
const outcomes = settled.flatMap(entry => ('outcome' in entry ? [entry.outcome] : []));
const missing = settled.length - outcomes.length;
const status: FreeShardOutcome['status'] = outcomes.some(outcome => outcome.status === 'timed-out') ? 'timed-out'
: problems.length || outcomes.some(outcome => outcome.status !== 'passed') ? 'failed' : 'passed';
const testsRan = outcomes.map(outcome => outcome.summary?.testsRan ?? null);
const failures = outcomes.flatMap(outcome => outcome.failures ?? []);
const inFlight = outcomes.flatMap(outcome => outcome.inFlight ?? []);
const counts = mergeTestCounts(settled.map(entry => ('outcome' in entry ? entry.outcome.counts : undefined)));
const outcome: FreeShardOutcome = {
shard: shard.shard,
files: shard.files,
status,
exitCode: status === 'passed' ? 0 : outcomes.find(entry => entry.exitCode)?.exitCode ?? 1,
elapsedMs,
groupPid: null,
failingFiles: status === 'passed' ? [] : [...new Set(outcomes.flatMap(entry => entry.failingFiles))],
unattributedFailures: status === 'passed' ? 0 : outcomes.reduce((sum, entry) => sum + entry.unattributedFailures, 0) + problems.length,
...(failures.length ? { failures } : {}),
summary: {
testsRan: missing || testsRan.some(value => value === null) ? null : testsRan.reduce<number>((sum, value) => sum + value!, 0),
filesRan: outcomes.reduce((sum, entry) => sum + (entry.summary?.filesRan ?? 0), 0),
sawTerminalSummary: !missing && outcomes.every(entry => entry.files.length === 0 || entry.summary?.sawTerminalSummary === true),
},
...(counts ? { counts } : {}),
...(inFlight.length ? { inFlight } : {}),
};
const records = settled.map((entry): SubShardRecord => {
const { id, phase, files } = entry.assignment;
if (!('outcome' in entry)) return { id, phase, files, status: 'missing', exitCode: null, elapsedMs: 0, failingFiles: [], unattributedFailures: 1 };
const { status: subStatus, exitCode, elapsedMs: subElapsed, summary, failingFiles, unattributedFailures } = entry.outcome;
return { id, phase, files, status: subStatus, exitCode, elapsedMs: subElapsed, summary, failingFiles, unattributedFailures };
});
return { outcome, records, problems };
}
/** `--ci-verify` re-checks a k > 1 result's sub-shard records; empty means they stand. */
export function subShardRecordProblems(result: FreeCiResult): string[] {
if (result.jobs === undefined && result.subShards === undefined) return [];
const records = result.subShards ?? [];
if (!Number.isInteger(result.jobs) || result.jobs! < 2 || !records.length) return ['sub-shard result without a valid --jobs and records'];
const files = records.flatMap(record => record.files);
const problems: string[] = [];
if (new Set(files).size !== files.length || files.length !== result.outcome.files.length
|| !result.outcome.files.every(file => files.includes(file))) problems.push('sub-shard records do not cover the shard exactly once');
const prefix = `${result.planId.slice(0, 12)}:${result.outcome.shard}.`;
if (records.some((record, index) => record.id !== `${prefix}${index + 1}`)
|| records.filter(record => record.phase === 'parallel').length > result.jobs!
|| records.some((record, index) => record.phase === 'serial' ? index !== records.length - 1 : record.phase !== 'parallel')) {
problems.push('sub-shard ids or phases do not match the plan and --jobs');
}
if (records.some(record => record.status === 'missing' || record.summary?.filesRan !== record.files.length
|| (record.files.length > 0 && record.summary?.sawTerminalSummary !== true))) problems.push('a sub-shard did not run every assigned file');
if (result.outcome.status === 'passed' && records.some(record => record.status !== 'passed')) problems.push('a passing shard holds a failed sub-shard');
return problems;
}
export interface CiRunContext { planPath: string; runId: string; runner: string }
/** Where this run is: the CI run id (with attempt) and runner, else `local` and the host. */
export function ciRunContext(planPath: string, env: NodeJS.ProcessEnv = process.env): CiRunContext {
const runId = env.GITHUB_RUN_ID ? `${env.GITHUB_RUN_ID}${env.GITHUB_RUN_ATTEMPT ? `/attempt ${env.GITHUB_RUN_ATTEMPT}` : ''}` : 'local';
const cpus = os.availableParallelism?.() ?? os.cpus().length;
return { planPath, runId, runner: `${env.RUNNER_NAME || os.hostname()} (${process.platform}, ${cpus} CPUs)` };
}
const quote = (value: string) => `'${value.replaceAll("'", "'\\''")}'`;
/** The FREE_SUBSHARD_TIMEOUT text: the file, k, runner, a serial reproduce command and the fix. */
export function subShardTimeoutText(
shard: number, jobs: number, record: SubShardRecord, inFlight: string[], wallTimeoutMs: number, context: CiRunContext, shardWallMs = wallTimeoutMs,
): string[] {
const suspects = inFlight.filter(file => record.files.includes(file));
const files = suspects.length ? suspects : record.files;
const seconds = (ms: number) => Math.round(ms / 1000);
const fix = record.phase !== 'serial'
? `if it passes serially, add it to SLOW_UNDER_LOAD in scripts/lib/shard-jobs.ts with run id ${context.runId}; never raise the timeout`
: wallTimeoutMs < shardWallMs
? `the parallel sub-shards left this serial sub-shard ${seconds(wallTimeoutMs)}s of the ${seconds(shardWallMs)}s shard wall; if the k=1 command passes, the parallel phase used up the budget and this shard needs a lower --jobs; never raise the timeout`
: 'this file already runs alone after the parallel sub-shards; fix or split it, never raise the timeout';
return [
`[test:free] ${FREE_SUBSHARD_TIMEOUT}: CI shard ${shard} sub-shard ${record.id} (${record.files.length} files) hit its ${seconds(wallTimeoutMs)}s wall at --jobs ${jobs} on ${context.runner}, run ${context.runId}.`,
`[test:free] in flight: ${suspects.length ? suspects.join(', ') : 'unknown (no open file chunk; see the sub-shard log)'}`,
`[test:free] reproduce serially: printf '%s\\n' ${files.map(quote).join(' ')} > files.txt && GSTACK_FREE_JOBS=1 bun run scripts/test-free-shards.ts --files-from files.txt`,
`[test:free] or the whole CI shard at k=1: bun run scripts/test-free-shards.ts --ci-run ${quote(context.planPath)} --shard ${shard} --result shard-${shard}.json --jobs 1`,
`[test:free] ${fix}. See ${DOCS}`,
];
}
export interface RunShardOptions { wallTimeoutMs: number; verbose: boolean; testCounts: true; homeGuard?: FreeHomeGuardFactory }
export interface CiShardInput {
plan: FreeCiPlan;
shard: FreeCiPlan['shards'][number];
revision: string;
/** Sub-shard count; 1 is today's single child. */
jobs: number;
wallTimeoutMs: number;
verbose: boolean;
context: CiRunContext;
}
export interface CiShardDeps {
runShard(files: string[], shardNumber: number, totalShards: number, options: RunShardOptions): Promise<FreeShardOutcome>;
/** The free runner's single flaky-retry pass over every collected outcome. */
retry(outcomes: FreeShardOutcome[]): Promise<{ exitCode: number; retry: FreeShardOutcome | null }>;
pack(files: string[], count: number): string[][];
isSerial(file: string): boolean;
terminationRequested(): boolean;
phaseGuard?(files: string[], shards: number): FreeHomeGuard;
log?(line: string): void;
}
/** Run one planned CI shard at k sub-shards and build its result JSON. */
export async function runCiShard(input: CiShardInput, deps: CiShardDeps): Promise<{ result: FreeCiResult; exitCode: number }> {
const { plan, shard, revision, jobs, wallTimeoutMs, verbose } = input;
const log = deps.log ?? ((line: string) => console.log(line));
if (jobs === 1) {
const outcome = await deps.runShard(shard.files, shard.shard, plan.shards.length, { wallTimeoutMs, verbose, testCounts: true });
const retried = await deps.retry([outcome]);
return { result: { planId: plan.id, revision, outcome, retry: retried.retry }, exitCode: retried.exitCode };
}
const startedAt = Date.now();
const assignments = assignSubShards(plan.id, shard, jobs, deps.isSerial, deps.pack);
const total = assignments.length;
log(`[test:free] CI shard ${shard.shard}/${plan.shards.length} at --jobs ${jobs}: ${assignments
.map(entry => `sub-shard ${entry.id} = runner shard ${entry.index}/${total} (${entry.phase}, ${entry.files.length} files)`).join('; ')}`);
const walls = new Map<string, number>();
const run = (entry: SubShardAssignment, homeGuard?: FreeHomeGuardFactory, wall = wallTimeoutMs): Promise<SubShardSettlement> => {
walls.set(entry.id, wall);
return deps.runShard(entry.files, entry.index, total, { wallTimeoutMs: wall, verbose, testCounts: true, ...(homeGuard ? { homeGuard } : {}) })
.then(outcome => ({ assignment: entry, outcome }), (error: unknown) => ({ assignment: entry, error: error instanceof Error ? error.message : String(error) }));
};
const parallel = assignments.filter(entry => entry.phase === 'parallel');
const serial = assignments.filter(entry => entry.phase === 'serial');
const phaseProblems: string[] = [];
const settled: SubShardSettlement[] = [];
if (parallel.length) {
const guard = deps.phaseGuard ?? ((files: string[], shards: number) => guardFreeHome(files, process.env, { kind: 'concurrent', shards }));
const phaseHome = guard(parallel.flatMap(entry => entry.files), parallel.length);
settled.push(...await Promise.all(parallel.map(entry => run(entry, sharedFreeHome))));
const homeChange = phaseHome.verify();
if (homeChange) phaseProblems.push(homeChange);
}
for (const entry of serial) {
const left = startedAt + wallTimeoutMs - Date.now();
settled.push(deps.terminationRequested() ? { assignment: entry, error: 'cancelled before the serial phase' }
: left <= 0 ? { assignment: entry, error: 'no wall left after the parallel phase' } : await run(entry, undefined, left));
}
const merged = mergeSubShardOutcomes(shard, settled, Date.now() - startedAt, phaseProblems);
for (const problem of merged.problems) console.error(`[test:free] ${FREE_SUBSHARD_REJECTED}: CI shard ${shard.shard}: ${problem}`);
for (const record of merged.records.filter(entry => entry.status === 'timed-out')) {
for (const line of subShardTimeoutText(shard.shard, jobs, record, merged.outcome.inFlight ?? [], walls.get(record.id) ?? wallTimeoutMs, input.context, wallTimeoutMs)) console.error(line);
}
log(`[test:free] CI shard ${shard.shard}: ${merged.outcome.status} across ${total} sub-shards in ${Math.round(merged.outcome.elapsedMs / 1000)}s`);
const retried = await deps.retry([merged.outcome]);
return {
result: { planId: plan.id, revision, outcome: merged.outcome, retry: retried.retry, jobs, subShards: merged.records },
exitCode: retried.exitCode,
};
}
/** The duration seed takes only one-file children's walls, never a sub-shard's or a shard's. */
export function assertSoloSeedSample(file: string, outcome: Pick<FreeShardOutcome, 'files' | 'summary'>): void {
if (outcome.files.length !== 1 || outcome.files[0] !== file || (outcome.summary?.filesRan ?? 0) > 1) {
throw new Error(`Duration seed sample for ${file} came from a ${outcome.files.length}-file child; seeds are recorded only from one-file serial children.`);
}
}
scripts/lib/windows-curation.ts
+8
View File
@@ -315,6 +315,10 @@ export const KNOWN_WINDOWS_INCOMPATIBLE: Array<{ file: string; reason: string }>
file: 'test/ship-queue-train-land.test.ts',
reason: 'drives train land against test/helpers/ship-queue-train-fixture.ts plus an extensionless node gh wrapper on a colon-joined PATH; on windows-latest spawnSync(\'gh\') resolves the runner\'s gh.exe instead, as for test/ship-queue-train.test.ts and test/ship-queue-land.test.ts (windows-free-shard 6, run 40824965988)',
},
{
file: 'test/ship-queue-land-lock.test.ts',
reason: 'drives land and train land against test/helpers/ship-queue-train-fixture.ts plus an extensionless node gh wrapper on a colon-joined PATH; on windows-latest spawnSync(\'gh\') resolves the runner\'s gh.exe instead, as for test/ship-queue-train-land.test.ts and test/ship-queue-land.test.ts (windows-free-shard 6, run 40824965988)',
},
{
file: 'test/ship-queue-train.test.ts',
reason: 'drives bin/gstack-ship-queue against the stateful #!/bin/sh gh shim in test/helpers/ship-queue-train-fixture.ts; on windows-latest spawnSync(\'gh\') resolves the runner\'s gh.exe instead of the shim (windows-free-shard run 39152702056)',
@@ -402,10 +406,14 @@ export const KNOWN_WINDOWS_SAFE: Array<{ file: string; reason: string }> = [
{ file: 'test/cso-windows-docker.test.ts', reason: '/cso docker.exe discovery under known-folder roots (real path, no reparse points) and the native-transport-unsupported outcome; pure win32 path logic' },
{ file: 'test/copilot-windows-bash.test.ts', reason: 'the Copilot glossary line that runs bash blocks in Git for Windows Bash; reads host config only' },
{ file: 'test/ship-hook-windows-paths.test.ts', reason: 'runs bin/ helpers through explicit bash and Bun argv with forward-slash paths; never executes a shebang; the path-spelling simulation is skipIf win32' },
{ file: 'test/review-scope-windows-launch.test.ts', reason: '#3541: runs gstack-review-scope through lib/script-invocation.ts, which must reach the bash gstack-review-read through Git Bash (red at main: windows-free-shard 3, run 41228091417); os.tmpdir state only' },
{ file: 'test/jsonl-merge-encoding.test.ts', reason: '#3659: runs gstack-jsonl-merge through lib/script-invocation.ts (Git Bash on win32); the CRLF half of the bug only reproduces with a Windows Python' },
{ file: 'test/gstack-wtree-windows-paths.test.ts', reason: '#3101: runs bin/gstack-wtree through explicit bash in an os.tmpdir linked worktree with forward-slash paths; never executes a shebang; the drive-letter simulation is skipIf win32' },
{ file: 'test/setup-hook-syntax-gate.test.ts', reason: '#3088/#3100: runs bin/gstack-hook-check and the setup hook sections through explicit bash with forward-slash paths; never executes a shebang. Its Bun drive-path spelling is the Windows bug it guards' },
{ file: 'test/bun-config-shell-bins.test.ts', reason: '#3415: runs the record bins through lib/script-invocation.ts (Git Bash on Windows) with MSYS path conversion off, the case it guards; its bun and cygpath stand-ins and simulated-OSTYPE cases are skipIf win32' },
{ file: 'test/ship-receipt-session-kind.test.ts', reason: '#3541: runs gstack-ship-receipt write through lib/script-invocation.ts, which must reach the bash gstack-session-kind through Git Bash; os.tmpdir git fixture only' },
{ file: 'test/path-spelling.test.ts', reason: 'sources bin/gstack-path-spelling.sh through explicit bash and checks lib/path-spelling.ts; on Windows it exercises the real Git Bash spellings' },
{ file: 'test/timeline-stop-hook-stdin.test.ts', reason: '#3628: the hook runs through Bun argv with a stdin pipe held open; the return-within-budget timing must meet real Windows process startup' },
{ file: 'test/state-root-parity.test.ts', reason: 'runs the bash twin and lib/state-root.ts over an env table with PATH empty; no shebang execution, raw-string comparison is platform-neutral' },
{ file: 'test/generator-eexist.test.ts',
reason: 'E4: runs the generators through Bun argv with the Windows EEXIST emulation preload; no shebang execution' },
scripts/pr-dependencies.ts
+1 -1
View File
@@ -89,7 +89,7 @@ export const SELECTION_DATA_MODULES: readonly string[] = [
// The build entry list: every compiled binary, which skills reach through their `--outfile` path instead.
'scripts/build.sh',
// The reason-code table: its `fix:` prose names the command a person reruns, not a file the module consumes.
'lib/result-codes.ts', 'lib/result-codes-release.ts', 'lib/result-codes-capy.ts', 'lib/result-codes-budget.ts', 'lib/result-codes-eval.ts', 'lib/result-codes-plan.ts',
'lib/result-codes.ts', 'lib/result-codes-release.ts', 'lib/result-codes-capy.ts', 'lib/result-codes-budget.ts', 'lib/result-codes-eval.ts', 'lib/result-codes-plan.ts', 'lib/result-codes-autoplan.ts',
// The exercised-path map: observed commands per case, read only by exercisedConsumers() (shadow data; a change selects no paid case).
'test/helpers/exercised-commands.json',
];
scripts/resolvers/autoplan-driver.ts
+6 -6
View File
@@ -17,17 +17,17 @@ import type { ResolverFn } from './types';
/** One entry per command line the loop runs, as the field-wave plan specifies them. */
export const AUTOPLAN_DRIVER_COMMANDS: Array<{ cmd: string; does: string }> = [
{ cmd: 'gstack-autoplan next --out <dir> --plan <plan.md> [scope flags] [--light] [--spend-cap <usd>] [--max-rounds <n>] [--scaffold]', does: `first call: init the run, print PHASE:, SCOPE:, DRIVER: and one ATTEMPT: per voice; --max-rounds caps review rounds per phase (default ${DEFAULT_MAX_ROUNDS} unattended), and an unattended Capy run reserves ${DEFAULT_HOST_ESTIMATE_USD.toFixed(2)} per host attempt unless --estimate-usd <n> says otherwise, so --spend-cap binds` },
{ cmd: 'gstack-autoplan next --out <dir> --plan <plan.md> [scope flags] [--light] [--spend-cap <usd>] [--max-rounds <n>] [--estimate-usd <n>] [--scaffold] [--ceo-only]', does: `first call: init the run, print PHASE:, SCOPE:, DRIVER: and one ATTEMPT: per voice; --max-rounds caps review rounds per phase (default ${DEFAULT_MAX_ROUNDS} unattended); a run with --spend-cap, in any session kind (or any unattended Capy run), reserves ${DEFAULT_HOST_ESTIMATE_USD.toFixed(2)} per voice for each attempt, so the cap binds; --estimate-usd <n> sets that reservation and the run keeps it for later phases; --ceo-only (command line only, never a scope marker) reviews CEO alone and ends \`status=reviewed\` with \`<dir>/ceo-review.md\` (\`REVIEWED:\`): no gate, and the gate verbs refuse it (\`CEO_ONLY_NO_GATE\`)` },
{ cmd: 'gstack-autoplan run --out <dir> --plan <plan.md> --native-model <id> --outside-model <id> --native-runner api --runner api [--max-tokens <n>] [--spend-cap <usd>]', does: 'hosts without subagents: one command calls both models directly (ANTHROPIC_API_KEY, OPENAI_API_KEY) and loops next/submit to the gate; a response cut off at the output limit is reported truncated, so raise --max-tokens (default 16384 for both providers; OpenAI counts reasoning tokens toward it); each api call reserves its priced worst case (a model with no price reserves the per-voice default under a cap), a native call that went out but bound no result settles at its usage, or its full reservation when the usage is unknown, and a failed outside call stays counted at its reservation' },
{ cmd: 'gstack-autoplan next --out <dir> [--redispatch <attempt>]', does: 'later calls: open the next phase or reprint open attempts; never redispatches on its own' },
{ cmd: 'gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --record <file.md> --model <id> [--attempt <id>]', does: 'bind a free-form review record (canonical fence, one findings table or list, or `No findings.`)' },
{ cmd: 'gstack-autoplan submit ... --record <file.md> --attest "<reason>" --attempt <id>', does: 'bind a record with no INPUT: line as receipt: attested (a wrong INPUT: always refuses)' },
{ cmd: 'gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --result <file> --model <id> --attempt <id>', does: 'bind a canonical result (the RESULT FORMAT below)' },
{ cmd: 'gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --result <file> --model <id> --attempt <id> [--usage-usd <n>] [--reconcile <jsonl>]', does: 'bind a canonical result (the RESULT FORMAT below); `--usage-usd` settles the attempt at what the reviewer actually cost (else it stays priced unknown at its reservation); on the submit that closes the phase, `--reconcile` takes your own pairing of findings both voices raised, one `{"outside_id","native_id","disposition","resolution"}` row each, over the structural one' },
{ cmd: 'gstack-autoplan submit --out <dir> --phase <p> --voice outside --unavailable|--failed "<reason>"', does: 'close the phase native-only when no outside voice can run' },
{ cmd: 'gstack-autoplan answer --out <dir> --gate-rev <n> --reply "<reply>" --by human|agent:<id> [--owner-reply <text>|-] [--rec-hash <h>]', does: 'answer the gate with the reply grammar below; an agent answer stores the owner\'s words verbatim' },
{ cmd: 'gstack-autoplan answer --out <dir> --gate-rev <n> --reply "<reply>" --by human|agent:<id> [--owner-reply <text>|-] [--words-by agent:<id>] [--rec-hash <h>]', does: 'answer the gate with the reply grammar below; an agent answer stores the owner\'s words verbatim, and passes --words-by agent:<id> whenever the words are its own (the default is owner)' },
{ cmd: 'gstack-autoplan recommend --out <dir> --gate-rev <n> --item <id> --option <k> --basis "<why>"', does: 'change one recommendation before relay; then render and relay again (a `rec` needs the new `--rec-hash`)' },
{ cmd: 'gstack-autoplan revise --out <dir> --plan <plan v2> --incorporates <decision ids>', does: 'bind the amended plan; reopens every phase whose reviewed plan changed, Eng last' },
{ cmd: 'gstack-autoplan revise --out <dir> --settle <decision id|d4>', does: 'discharge a packaging or ordering change with no plan edit; reopens nothing' },
{ cmd: 'gstack-autoplan revise --out <dir> --withdraw <decision id>', does: 'drop a pending amendment instead of applying it' },
{ cmd: 'gstack-autoplan revise --out <dir> --settle <decision id|d4> | --withdraw <decision id>', does: '--settle discharges a packaging or ordering change with no plan edit and reopens nothing; --withdraw drops a pending amendment instead of applying it' },
{ cmd: 'gstack-autoplan close --out <dir> --phase <p> --deferred "<reason>" --by human|agent:<id>', does: 'close a phase accepted-with-findings: its open findings go to tasks.jsonl, and Critical, urgent or user-challenge findings become gate items; also the recovery from `STOP: reason=budget_exhausted`' },
{ cmd: 'gstack-autoplan budget --out <dir> [--spend-cap <usd>] [--max-rounds <n>] --by human|agent:<id>', does: 'the owner\'s journaled cap raise; after a budget stop, `resume` continues the run' },
{ cmd: 'gstack-autoplan export --out <dir>', does: 'write the artifact set; then `gstack-artifact validate <dir>/run.json`' },
@@ -40,7 +40,7 @@ export const AUTOPLAN_DRIVER_COMMANDS: Array<{ cmd: string; does: string }> = [
export const GATE_REPLY_FORMS: Array<{ reply: string; means: string }> = [
{ reply: 'rec', means: 'every item takes its recommendation, plan changes included (each prints APPLY:); `all` and `do it` are aliases' },
{ reply: 'as-is', means: 'keep every direction; no plan change' },
{ reply: 'd3b uc1a', means: 'one <id><option> token per item you set; unanswered auto items take their default' },
{ reply: 'd3b uc1a', means: 'one <id><option> token per item you set; unanswered auto items take their default (a finding item keeps the plan as written; `p1b` applies it)' },
{ reply: 'rec except d3b', means: 'recommendations, with the listed overrides' },
];
scripts/resolvers/preamble/generate-preamble-bash.ts
+4 -5
View File
@@ -1,6 +1,6 @@
import type { TemplateContext } from '../types';
import { getHostConfig } from '../../../hosts/index';
import { runtimeRootPrelude } from '../runtime-root';
import { skillStartLines } from '../runtime-root';
/**
* Preamble bootstrap (token-reduction Phase 1).
@@ -25,11 +25,10 @@ import { runtimeRootPrelude } from '../runtime-root';
*/
export function generatePreambleBash(ctx: TemplateContext): string {
const hostConfig = getHostConfig(ctx.host);
const runtimeRoot = hostConfig.usesEnvVars ? `${runtimeRootPrelude(ctx)}\n` : '';
const start = hostConfig.usesEnvVars ? skillStartLines(ctx) : null;
const runtimeRoot = start ? `${start.lookup}\n` : '';
const brainHealthFlag = ctx.host === 'gbrain' || ctx.host === 'hermes' ? ' --brain-health' : '';
const startCommand = hostConfig.usesEnvVars
? `"${ctx.paths.binDir}/gstack-skill-start"`
: `${ctx.paths.binDir}/gstack-skill-start`;
const startCommand = start ? start.command : `${ctx.paths.binDir}/gstack-skill-start`;
const entry = ['plan-design-review', 'plan-eng-review'].includes(ctx.skillName)
? `## Preamble (after scope gate)
scripts/resolvers/runtime-root.ts
+41 -5
View File
@@ -3,8 +3,9 @@
* bash finds gstack on Codex and every other `usesEnvVars` host.
*
* Those hosts run each fenced block in a fresh shell, so `GSTACK_ROOT`, `B`,
* `D` and `P` set by an earlier block are gone. The preamble's root resolution and
* the per-fence prelude come from `runtimeRootPrelude()`; `insertRuntimePreludes()`
* `D` and `P` set by an earlier block are gone. The preamble's skill-start fence
* comes from `skillStartLines()` and the per-fence prelude from
* `runtimeRootPrelude()`; `insertRuntimePreludes()`
* is the single post-render pass (gen-skill-docs, before rewriteInstallRoot)
* that puts it into every fence that uses these variables without assigning
* them. Claude output is returned unchanged.
@@ -13,6 +14,9 @@
* on a per-install render, the literal install root (no git call); otherwise the
* repo-local install, then the host's global root (`CODEX_HOME` on Codex). No
* root prints one line naming what was tried and the setup command, and exits.
* The skill-start fence differs (GST-882): an install-root render calls the
* literal install root's start script and ignores an exported `GSTACK_ROOT`, and
* with no root either render prints that line and lets the call fail.
*/
import { getHostConfig } from '../../hosts/index';
import { toShellPath, type TemplateContext } from './types';
@@ -37,13 +41,45 @@ const uses = (name: string) => new RegExp(`\\$\\{?${name}\\b`);
*/
function resolveRoot(ctx: TemplateContext): string {
const host = getHostConfig(ctx.host);
const fix = `Fix: ./setup --host ${host.name} from your gstack checkout; ./setup --status shows it.`;
const fix = installFix(ctx);
const exported = '[ -d "${GSTACK_ROOT:-/-}/bin" ]&&[ -d "$GSTACK_ROOT/lib" ]||';
if (ctx.installRoot) {
return `${exported}{ GSTACK_ROOT="${ctx.installRoot.replace(/\/+$/, '')}";[ -d "$GSTACK_ROOT/bin" ]||{ echo "gstack: no install found (tried $GSTACK_ROOT). ${fix}">&2;exit 1;};}`;
}
const global = host.name === 'codex' ? '${CODEX_HOME:-~/.codex}/skills/gstack' : `~/${host.globalRoot}`;
return `${exported}{ _r=$(git rev-parse --show-toplevel 2>/dev/null)/${host.localSkillRoot};[ -d "$_r/bin" ]||_r=${global};[ -d "$_r/bin" ]||{ echo "gstack: no install found (tried $_r). ${fix}">&2;exit 1;};GSTACK_ROOT=$_r;}`;
return `${exported}{ _r=$(git rev-parse --show-toplevel 2>/dev/null)/${host.localSkillRoot};[ -d "$_r/bin" ]||_r=${globalRoot(ctx)};[ -d "$_r/bin" ]||{ echo "gstack: no install found (tried $_r). ${fix}">&2;exit 1;};GSTACK_ROOT=$_r;}`;
}
const installFix = (ctx: TemplateContext) => `Fix: ./setup --host ${getHostConfig(ctx.host).name} from your gstack checkout; ./setup --status shows it.`;
const globalRoot = (ctx: TemplateContext) => {
const host = getHostConfig(ctx.host);
return host.name === 'codex' ? '${CODEX_HOME:-~/.codex}/skills/gstack' : `~/${host.globalRoot}`;
};
/**
* The skill-start fence on env-var hosts (GST-882). Hermes's Tirith blocks a
* start command it cannot resolve statically, a grouped fallback or a
* variable-selected path alike, and no allowlist entry clears that finding. An
* install-root render therefore names the literal start path; a checked-in
* render keeps resolveRoot's lookup order as plain lines. Either way a missing
* install prints the fix, and the call then fails.
*/
export function skillStartLines(ctx: TemplateContext): { lookup: string; command: string } {
const fix = installFix(ctx);
if (ctx.installRoot) {
const root = ctx.installRoot.replace(/\/+$/, '');
return {
lookup: `[ -x "${root}/bin/gstack-skill-start" ]||echo "gstack: no install found (tried ${root}). ${fix}">&2`,
command: `"${root}/bin/gstack-skill-start"`,
};
}
return {
lookup: [
`[ -d "\${GSTACK_ROOT:-/-}/lib" ]&&[ -d "$GSTACK_ROOT/bin" ]||GSTACK_ROOT=$(git rev-parse --show-toplevel 2>/dev/null)/${getHostConfig(ctx.host).localSkillRoot}`,
`[ -d "$GSTACK_ROOT/bin" ]||GSTACK_ROOT=${globalRoot(ctx)}`,
`[ -d "$GSTACK_ROOT/bin" ]||echo "gstack: no install found (tried $GSTACK_ROOT). ${fix}">&2`,
].join('\n'),
command: '"$GSTACK_ROOT/bin/gstack-skill-start"',
};
}
const DERIVED: Record<string, string> = { BIN: 'bin', BROWSE: 'browse/dist', DESIGN: 'design/dist', MAKE_PDF: 'make-pdf/dist' };
scripts/test-free-shards.ts
+51 -12
View File
@@ -77,6 +77,11 @@
* bun run scripts/test-free-shards.ts --attribute-home # name real-home writers: each file alone, private HOME
* bun run scripts/test-free-shards.ts --ci-plan plan.json --shards 20
* bun run scripts/test-free-shards.ts --ci-run plan.json --shard 1 --result result.json
* bun run scripts/test-free-shards.ts --ci-run plan.json --shard 1 --result result.json --jobs 4
* # k sub-shards side by side inside one CI job (default 1 = one child, today's behavior;
* # scripts/lib/shard-jobs.ts). --jobs applies only to --ci-run: a positive whole number up
* # to the shard's file count, else FREE_JOBS_INVALID. GSTACK_FREE_JOBS sets the local
* # full-suite process count and never changes --ci-run; --jobs is its only knob.
* bun run scripts/test-free-shards.ts --ci-verify plan.json --results results/
*/
@@ -91,6 +96,10 @@ import { resolveStateRoot } from '../lib/state-root';
import { attributeFreeHomeWriters, guardFreeHome, sharedFreeHome, type FreeHomeGuardFactory } from './lib/free-home-guard';
import { appendStepSummary, ciHealthSummary, SEED_REFRESH_COMMAND, unseededWarning, windowsCurationLine } from './lib/free-ci-health';
import { curateWindowsSafe, type CurationResult } from './lib/windows-curation';
import {
assertSoloSeedSample, ciRunContext, parseJobsFlag, parseJunitCounts, readJunitReport, rejectJobsWithoutCiRun, runCiShard, SLOW_UNDER_LOAD, subShardRecordProblems,
validateJobsForShard, type ShardTestCounts, type SubShardRecord,
} from './lib/shard-jobs';
export { curateWindowsSafe, detectWindowsFragility, KNOWN_WINDOWS_INCOMPATIBLE, type CurationResult } from './lib/windows-curation';
import {
BunTestOutputClassifier,
@@ -447,6 +456,9 @@ export interface FreeCiResult {
revision: string;
outcome: FreeShardOutcome;
retry: FreeShardOutcome | null;
/** Present only at --jobs k > 1: k and one record per sub-shard. */
jobs?: number;
subShards?: SubShardRecord[];
}
/** Preserve the full-suite retry cap across independently running CI jobs. */
@@ -472,6 +484,8 @@ export function verifyFreeCiResults(plan: FreeCiPlan, results: FreeCiResult[]):
|| JSON.stringify(outcome.files) !== JSON.stringify(shard.files)) throw new Error('CI result identity, shard or file coverage mismatch');
seen.add(outcome.shard);
if (!hasCompleteCiSummary(outcome)) throw new Error('Missing or incomplete CI execution summary');
const subShardProblems = subShardRecordProblems(result);
if (subShardProblems.length) throw new Error(`CI shard ${outcome.shard} sub-shards rejected: ${subShardProblems.join('; ')}`);
if (outcome.status === 'passed') {
if (outcome.exitCode !== 0 || outcome.failingFiles.length || outcome.unattributedFailures || result.retry) throw new Error('Inconsistent passing CI result');
} else {
@@ -542,6 +556,8 @@ type CliOptions = {
result: string | null;
results: string | null;
filesFrom: string | null;
/** Set only by --jobs (with --ci-run); absent means k = 1. */
jobs?: number;
};
export function parseCliOptions(argv: string[]): CliOptions {
@@ -556,6 +572,7 @@ export function parseCliOptions(argv: string[]): CliOptions {
let wallTimeoutExplicit = false;
let quick = false;
let attributeHome = false;
let jobs: number | undefined;
const paths: Record<'ciPlan' | 'ciRun' | 'ciVerify' | 'result' | 'results' | 'filesFrom', string | null> = {
ciPlan: null, ciRun: null, ciVerify: null, result: null, results: null, filesFrom: null,
};
@@ -579,6 +596,7 @@ export function parseCliOptions(argv: string[]): CliOptions {
'--result': pathFlag('--result', 'result'),
'--results': pathFlag('--results', 'results'),
'--files-from': pathFlag('--files-from', 'filesFrom'),
'--jobs': (next) => { jobs = parseJobsFlag(next()); },
'--shards': (next) => {
const value = next();
if (!value) throw new Error('Missing value for --shards');
@@ -602,8 +620,9 @@ export function parseCliOptions(argv: string[]): CliOptions {
if (ciModes > 1 || (ciModes && (quick || listOnly || dryRun || recordDurations || attributeHome))) throw new Error('CI modes cannot be combined with other selection modes');
if (paths.ciRun && (shardIndex === null || !paths.result)) throw new Error('--ci-run requires --shard and --result');
if (paths.ciVerify && !paths.results) throw new Error('--ci-verify requires --results');
if (jobs !== undefined && !paths.ciRun) rejectJobsWithoutCiRun();
if (quick && (recordDurations || windowsOnly || shardIndex !== null)) throw new Error('--quick cannot change recording, Windows or shard selection');
return { dryRun, listOnly, recordDurations, windowsOnly, verbose, shardCount, shardIndex, wallTimeoutMs, wallTimeoutExplicit, quick, attributeHome, ...paths };
return { dryRun, listOnly, recordDurations, windowsOnly, verbose, shardCount, shardIndex, wallTimeoutMs, wallTimeoutExplicit, quick, attributeHome, ...paths, ...(jobs === undefined ? {} : { jobs }) };
}
const describeLines = (lines: string[]) => `${lines.slice(0, 5).join(', ')}${lines.length > 5 ? ` (+${lines.length - 5} more)` : ''}`;
@@ -1075,6 +1094,10 @@ export interface FreeShardOutcome {
unattributedFailures: number;
/** Named failures with their excerpts, so a flaky pass records what failed the first time. */
failures?: FreeRunFailure[];
/** Per-file test and expect() counts from bun's JUnit report (--ci-run only). */
counts?: ShardTestCounts;
/** Wedge suspects at a wall-timeout kill (FreeRunReport.inFlight). */
inFlight?: string[];
}
export interface ShardCommand {
@@ -1087,8 +1110,10 @@ export interface RunFreeShardOptions {
wallTimeoutMs?: number;
rootDir?: string;
env?: NodeJS.ProcessEnv;
/** Override the spawned command. Tests inject fake pass/fail/slow commands. */
commandFor?: (files: string[]) => ShardCommand;
/** Override the spawned command. Tests inject fake pass/fail/slow commands; `junitPath` is set with testCounts. */
commandFor?: (files: string[], junitPath?: string) => ShardCommand;
/** Ask bun for a JUnit report in the shard sandbox and record per-file counts on the outcome (--ci-run). */
testCounts?: boolean;
/** Suppress ALL child output from the console (tests). The classifier and the log file still see every byte. */
quiet?: boolean;
/** Forward the full child stream to the console (legacy firehose). Default: the quiet filtered console. */
@@ -1633,12 +1658,14 @@ export async function runFreeShard(
const shardLog = openShardLog(logPath, label, 0o600);
log(`[test:free] full log: ${logPath}`);
const { command, args } = options.commandFor
? options.commandFor(files)
: { command: process.execPath, args: buildShardArgs(files, { rootDir }) };
// realpath: the browser tracker refuses a state dir whose path resolves elsewhere.
const { stateDir, env } = createShardSandbox('gstack-free-shard-', options.env ?? process.env, { realpath: true });
// The JUnit reporter leaves the console stream byte-identical (Bun 1.4.2); it only adds per-file counts.
const junitPath = options.testCounts ? path.join(stateDir, 'junit.xml') : undefined;
const { command, args } = options.commandFor
? options.commandFor(files, junitPath)
: { command: process.execPath, args: [...buildShardArgs(files, { rootDir }), ...(junitPath ? ['--reporter=junit', `--reporter-outfile=${junitPath}`] : [])] };
let junitXml = null as string | null;
// CLI renders otherwise share the repo's .gstack/browse.json, where concurrent
// shards and prior daemons replace each other's state; override inherited state.
env.BROWSE_STATE_FILE = path.join(stateDir, '.gstack', 'browse.json');
@@ -1703,6 +1730,7 @@ export async function runFreeShard(
shardLog.write(diagnostic + '\n');
}
await new Promise<void>((resolve) => shardLog.stream.end(() => resolve()));
if (junitPath) junitXml = readJunitReport(junitPath);
try {
if (!cleanupError) fs.rmSync(stateDir, { recursive: true, force: true });
} catch {
@@ -1745,6 +1773,8 @@ export async function runFreeShard(
shard: shardNumber, files, status, exitCode, elapsedMs: Date.now() - startedAt, groupPid, failingFiles, unattributedFailures,
...(status === 'passed' ? {} : { failures: report.failures }),
summary: { testsRan: report.testsRan, filesRan: report.filesRan, sawTerminalSummary: report.sawTerminalSummary },
...(junitPath ? { counts: parseJunitCounts(junitXml, files, report.testsRan, rootDir) } : {}),
...(status === 'timed-out' && report.inFlight.length ? { inFlight: report.inFlight } : {}),
};
log(shardEpilogue(outcome, totalShards));
for (const line of buildRunEpilogue(status, report, outcome.elapsedMs, logPath)) log(line);
@@ -1799,6 +1829,7 @@ async function recordFreeTestDurations(files: string[], jobs: number, wallTimeou
const outcome = await runFreeShard([file], index + 1, files.length, {
wallTimeoutMs, quiet: true, homeGuard: sharedFreeHome,
});
assertSoloSeedSample(file, outcome);
durations[normalizeRelativePath(file)] = outcome.elapsedMs;
if (outcome.status !== 'passed') failed.push(file);
});
@@ -1944,14 +1975,22 @@ async function main(): Promise<number> {
}
const shard = plan.shards[options.shardIndex! - 1];
if (!shard || shard.shard !== options.shardIndex) throw new Error('CI shard index is outside the plan');
const outcome = await runFreeShard(shard.files, shard.shard, plan.shards.length, {
const jobs = options.jobs ?? 1;
validateJobsForShard(jobs, shard.files.length, shard.shard);
const durations = loadFreeTestDurations() ?? {};
const { result, exitCode } = await runCiShard({
plan, shard, revision, jobs, verbose: options.verbose, context: ciRunContext(options.ciRun!),
wallTimeoutMs: options.wallTimeoutExplicit ? options.wallTimeoutMs
: wallTimeoutForPackedShard(shard.predictedMs, options.wallTimeoutMs, shard.files.length),
verbose: options.verbose,
}, {
runShard: (files, shardNumber, totalShards, runOptions) => runFreeShard(files, shardNumber, totalShards, runOptions),
retry: outcomes => retryFailedFreeFiles(outcomes, plan.shards.length, options),
pack: (files, count) => packShardsByDuration(files, count, durations).shards,
isSerial: file => file in TREE_MUTATING || file in SLOW_UNDER_LOAD,
terminationRequested: isTerminationRequested,
});
const retried = await retryFailedFreeFiles([outcome], plan.shards.length, options);
writeJson(options.result!, { planId: plan.id, revision, outcome, retry: retried.retry } satisfies FreeCiResult);
return retried.exitCode;
writeJson(options.result!, result);
return exitCode;
}
let files = allFiles;
scripts/test-health-report.ts
+14 -8
View File
@@ -128,12 +128,18 @@ export function flakeCheck(window: FlakeWindow | Error): Check {
fix: 'Fix the race at source (docs/TESTING_INTERNALS.md#flake-ledger); never raise the timeout or add a retry. Reproduce with `bun test <file>` under load.' };
if (window instanceof Error) return { ...base, status: 'unavailable', detail: `unavailable: ${unavailableReason(window)}` };
const failing = window.files.filter(file => file.failing);
const left = flakeWindowNotes(window);
if (!window.enforceable) {
return { ...base, status: 'pass', detail: `report-only: ${window.runs} main run(s) since ${window.since.slice(0, 10)} (< 20 needed to enforce)` };
return { ...base, status: 'pass', detail: `report-only: ${window.runs} main run(s) since ${window.since.slice(0, 10)} (< 20 needed to enforce)${left}` };
}
if (failing.length === 0) return { ...base, status: 'pass', detail: `no file above 5% over ${window.runs} main runs` };
if (failing.length === 0) return { ...base, status: 'pass', detail: `no file above 5% over ${window.runs} main runs${left}` };
return { ...base, status: 'fail',
detail: failing.map(file => `${file.file}: flaky in ${file.flakyRuns}/${window.runs} main runs (${pct(file.rate)} > 5%)`).join('; ') };
detail: failing.map(file => `${file.file}: flaky in ${file.flakyRuns}/${window.runs} main runs (${pct(file.rate)} > 5%)`).join('; ') + left };
}
/** The reused main pushes a flake window left out and the runs it could not check, or ''. */
function flakeWindowNotes(window: FlakeWindow): string {
return `${window.skipped ? `; ${window.skipped} reused main push(es) left out` : ''}${window.note ? `; ${window.note}` : ''}`;
}
export function unseededCheck(root: string): Check {
@@ -286,19 +292,19 @@ export function collectHealth(inputs: HealthInputs): HealthReport {
metrics.push(...censusMetrics(inputs, artifactDirs));
const flakes = attempt(() => readCiFlakeLedgers({ client, repo, sinceDays, now, cacheDir }));
metrics.push(metric('free-flaky-pass-rate', 'Free-suite flaky-pass rate', 'share of main free-tests runs (success or failure) with at least one flaky-pass ledger entry; per-file rate = runs with that file / runs', () => {
metrics.push(metric('free-flaky-pass-rate', 'Free-suite flaky-pass rate', 'share of main free-tests runs (success or failure) with at least one flaky-pass ledger entry; per-file rate = runs with that file / runs; a main push that reused its pull request\'s run (no artifacts, `reuse` job green) is left out', () => {
const window = need(flakes);
return { value: `${pct(window.runs ? window.flakyRuns / window.runs : 0)} (${window.flakyRuns}/${window.runs} main runs since ${window.since.slice(0, 10)} needed the retry pass; ${window.files.length} file(s))`,
return { value: `${pct(window.runs ? window.flakyRuns / window.runs : 0)} (${window.flakyRuns}/${window.runs} main runs since ${window.since.slice(0, 10)} needed the retry pass; ${window.files.length} file(s)${flakeWindowNotes(window)})`,
detail: window.files.slice(0, 15).map(file => `${file.file}: ${file.flakyRuns}/${window.runs} runs (${pct(file.rate)})${file.failing ? ' FAIL > 5%' : ''}`) };
}));
metrics.push(metric('free-wall', 'Free-suite and Windows wall', 'p50 of run_started_at to updated_at for successful main runs of free-tests.yml and successful pull_request runs of windows-free-tests.yml (its workflow_dispatch duration recordings excluded)', () => {
metrics.push(metric('free-wall', 'Free-suite and Windows wall', 'p50 of run_started_at to updated_at for successful main runs of free-tests.yml and successful pull_request runs of windows-smoke.yml, the required Windows check (its workflow_dispatch runs excluded)', () => {
const walls = (workflow: string, filter: { branch?: string; event?: string }) =>
listWorkflowRuns(client, repo, workflow, { since, ...filter }).filter(run => run.conclusion === 'success').map(run => span(run.startedAt, run.updatedAt)!);
const free = walls('free-tests.yml', { branch: 'main' });
const windows = walls('windows-free-tests.yml', { event: 'pull_request' });
const windows = walls('windows-smoke.yml', { event: 'pull_request' });
const windowsValue = windows.length ? `${minutes(percentile(windows, 0.5))} (${windows.length} runs)` : 'no successful PR runs in window';
return { value: `free-tests ${minutes(percentile(free, 0.5))} (${free.length} runs); windows-free-tests (successful PR runs) ${windowsValue}` };
return { value: `free-tests ${minutes(percentile(free, 0.5))} (${free.length} runs); windows-smoke (successful PR runs) ${windowsValue}` };
}));
metrics.push(metric('free-serial', 'Free-suite serial time', 'sum of scripts/free-test-durations.json (the committed seed)', () => {
scripts/test-pr-profile.ts
+2
View File
@@ -237,6 +237,7 @@ export const PAID_WORKFLOW_FILES = [
export const FREE_ONLY_PR_FILES = [
'scripts/test-free-shards.ts',
'scripts/lib/free-home-guard.ts', // Imported only by the free shard runner.
'scripts/lib/shard-jobs.ts', // Free runner's --ci-run --jobs sub-shards; ship-measure loads it through test-free-shards.ts, so its touchfiles list it.
'test/helpers/auq-parallel-worker.ts',
'test/helpers/stored-zip.ts', // Read only by the free test-health report tests.
'scripts/free-test-durations.json',
@@ -294,6 +295,7 @@ export const FREE_ONLY_PR_FILES = [
'.github/workflows/windows-free-tests.yml',
'.github/workflows/windows-pregate-e2e.yml',
'.github/workflows/windows-setup-e2e.yml',
'.github/workflows/windows-smoke.yml',
] as const;
/**
scripts/ubicloud/ubi-runner.sh
+21 -16
View File
@@ -44,12 +44,13 @@
#
# TTL and reaper: a name this script generates ends in -t<hours> (UBI_TTL_HOURS,
# 1-999, default 3), the creator's own declaration of how long the VM may live.
# `reap` destroys every VM in the project whose TTL expired more than ten
# minutes ago, whichever owner made it, and this owner's own ubirun-<owner>-*
# names with no TTL once they are UBI_LEGACY_HOURS old (default 4; another
# owner's or an untagged TTL-less VM is never reaped); every `up` runs it first (UBI_REAP=0 skips it), so a runner
# killed by a sleeping machine leaves nothing behind for longer than its TTL.
# Names outside the ubirun- prefix are never reaped.
# `reap` destroys only this owner's VMs: its ubirun-<owner>-* names whose TTL
# expired more than ten minutes ago, and those with no TTL once they are
# UBI_LEGACY_HOURS old (default 4). Another owner's VM, an untagged one and
# names outside the ubirun- prefix are never reaped, expired or not; reap only
# counts other owners' expired VMs in its log (cloud deletes stay owner-scoped,
# GST-573). Every `up` runs it first (UBI_REAP=0 skips it), so a lane killed by
# a sleeping machine has its leftovers destroyed by its own next `up`.
#
# Leftover line: `run` and `verify` end with
# UBICLOUD: left=<0|1|unknown> owner=<owner> vm=<name> others=<n>
@@ -754,23 +755,26 @@ vm_rows() {
done
}
# reap: destroy every VM whose name carries a TTL that ended more than
# REAP_GRACE seconds ago, whichever owner created it (the TTL is that owner's
# own declaration), and the caller's own TTL-less ubirun-<UBI_OWNER>-* names at
# least LEGACY_HOURS old. Another owner's TTL-less VM, an untagged one, and names
# outside the prefix are never touched: only a TTL lets reap judge another lane's VM.
# reap: destroy the caller's own VMs whose TTL ended more than REAP_GRACE
# seconds ago, and its TTL-less ubirun-<UBI_OWNER>-* names at least LEGACY_HOURS
# old. Every other VM is never touched, whatever its TTL says: a delete stays
# with the owner that created the VM. Other owners' expired VMs are only counted.
cmd_reap() {
local now rows loc name owner expires created n=0 failed=0
local now rows loc name owner expires created n=0 failed=0 others=0
[[ "$LEGACY_HOURS" =~ ^[1-9][0-9]{0,3}$ ]] || die "UBI_LEGACY_HOURS must be a whole number of hours from 1 to 9999 with no leading zero (how old a TTL-less ubirun-* VM must be before reap destroys it)"
now=$(date +%s)
rows=$(vm_rows) || { log "reap: could not list the project's VMs; nothing destroyed"; return 1; }
while read -r loc name owner expires created; do
[ -n "$name" ] || continue
if [ "$owner" != "$OWNER" ]; then
[ "$expires" = - ] || [ "$now" -lt $(( expires + REAP_GRACE )) ] || others=$(( others + 1 ))
continue
fi
if [ "$expires" != - ]; then
[ "$now" -ge $(( expires + REAP_GRACE )) ] || continue
log "reaping expired $loc/$name (owner $owner; its TTL ended $(( (now - expires) / 60 )) min ago)"
else
[ -n "${UBI_OWNER:-}" ] && [ "$owner" = "$OWNER" ] && [ "$created" != - ] || continue
[ -n "${UBI_OWNER:-}" ] && [ "$created" != - ] || continue
[ $(( now - 10#$created )) -ge $(( LEGACY_HOURS * 3600 )) ] || continue
log "reaping TTL-less $loc/$name (owner $owner; $(( (now - 10#$created) / 3600 ))h old, legacy sweep at ${LEGACY_HOURS}h)"
fi
@@ -778,6 +782,7 @@ cmd_reap() {
cmd_down "$name" -l "$loc" || { failed=$(( failed + 1 )); log "failed to reap $loc/$name"; }
done <<<"$rows"
[ "$n" -gt 0 ] || log "reap: no expired VMs"
[ "$others" -eq 0 ] || log "reap: left $others expired VMs of other owners; only their owners reap them"
[ "$failed" -eq 0 ] || { log "reap: $failed of $n expired VMs could not be destroyed"; return 1; }
}
@@ -1363,7 +1368,7 @@ cmd_run() {
trap 'exit 130' INT TERM QUIT
trap on_hup HUP
if [[ "$name" =~ -t([0-9]+)$ ]]; then
log "$name carries a ${BASH_REMATCH[1]}h TTL (any later run of this script reaps it); if this process dies first: $0 down $name"
log "$name carries a ${BASH_REMATCH[1]}h TTL (this owner's next up or reap destroys it once it expires); if this process dies first: $0 down $name"
elif [[ "$name" =~ ^$PREFIX-$OWNER-[0-9]{10}- ]]; then
log "$name has no TTL (reap destroys it once it is ${LEGACY_HOURS}h old); if this process dies before its teardown line, destroy it with: $0 down $name"
else
@@ -1579,8 +1584,8 @@ usage: ubi-runner.sh <command> [args]
owner print this caller's owner tag ($OWNER)
records VMs this machine created and has not confirmed gone ($VM_RECORD)
gc HOURS destroy this owner's VMs older than HOURS (never other owners')
reap destroy every VM whose -t<hours> TTL has ended (any owner) and this
owner's TTL-less ubirun-* VMs older than UBI_LEGACY_HOURS
reap destroy this owner's VMs whose -t<hours> TTL has ended and its
TTL-less ubirun-* VMs older than UBI_LEGACY_HOURS (never other owners')
cli ARGS... raw Ubicloud CLI passthrough (e.g. cli vm list)
signals: INT and TERM tear the VM down. HUP, once a remote phase is live, detaches
ship/DRIVER.md
+5 -5
View File
@@ -15,7 +15,7 @@ the next act. Read a section only when a phase or the list at the end sends you,
up front. A STOP or decision this file does not cover: read `ship/SKILL.md` at that step, finish it
there, come back. Unattended, take a question's recommended option (log it); gate
dispositions and approvals stay pending and end the run. Keep a private invocation record (`--record`): write its six headings at Phase 1, then fill each as its phase ends: **Checks**
(Phases 2-4: lanes, pregate, evals, `Coverage:`, the plan checks, `Scope Check:`), **Decisions** (any
(Phases 2-4: lanes, pregate, evals, `Coverage:`, the plan checks, `Exploratory QA:`, `Scope Check:`), **Decisions** (any
phase: each answered or auto-taken question, verbatim), **Reviews** (Phases 3-5: the plan line,
start tokens, `Pre-Landing Review:`, `Gate edits:`, the reviewed tree), **Release** (Phases 5-7:
version, CHANGELOG, commits), **Documentation** (Step 14.5) and **Next steps** (every phase: the next act).
@@ -24,7 +24,7 @@ version, CHANGELOG, commits), **Documentation** (Step 14.5) and **Next steps** (
outside-voice and documentation brief, its severity and confidence rules and gates come from its
section verbatim (placeholders filled); this file
never shortens a reviewer's brief. A blocking reader's prompt is its `gstack-review-dispatch render`
file, unchanged; its findings reach `merged.jsonl` only via `collect`; `judge` refuses a drop or downgrade. Audit readers (Steps 7, 8, 9.2.1, 14.5) cannot block, so each runs on a
file, unchanged; its findings reach `merged.jsonl` only via `collect`; `judge` refuses a drop or downgrade. Audit readers (Steps 7, 8, 14.5) cannot block, so each runs on a
small model at low effort; their findings go to the PR body.
**Calls.** One pass with no fixes is 9 calls: Phases 1-8 and the report; at most 12 with a STOP's
@@ -77,13 +77,13 @@ Never read TODOS.md or the full diff into the parent. A `PERSIST:` hint never re
## 3. Audits and review: Steps 7, 8, 9, 9.1, 9.2, 9.2.1
One program: the audit readers `READER:` names (Step 7 from `sections/test-coverage.md`, Step 8 from
`sections/plan-completion.md`, each run as written; Step 7 may write tests, so it settles first), Step 9.2.1's QA reader (that step of
`sections/review-army.md` as its brief) and the pass (above): Steps 9-9.2 as written. `Coverage:` and Step 8's line come from `--audit` when no
`sections/plan-completion.md`, each run as written; Step 7 may write tests, so it settles first), 9.2.1's Reads only and the pass
(above): Steps 9-9.2 from `sections/review-army.md` as written. `Coverage:` and Step 8's line come from `--audit` when no
reader ran.
## 4. Judgment: Steps 8.1, 8.2, 9.3, 9.4 and Step 11
Steps 8.1 and 8.2 (`Scope Check:`) on the todos and commits files, then the core checklist, Steps
Step 8.1, 9.2.1's probes, 8.2 (`Scope Check:`) on the todos and commits files, then the core checklist, Steps
9.3, 9.4 and the repeat decision as written (`gstack-review-plan` included;
`JUDGMENT: ok` before any fix), fixes and persistence in
one program. Prints `Pre-Landing Review:`, `Gate edits:`. `NEXT: stop` stops; `full` or
ship/DRIVER.md.tmpl
+5 -5
View File
@@ -13,7 +13,7 @@ the next act. Read a section only when a phase or the list at the end sends you,
up front. A STOP or decision this file does not cover: read `ship/SKILL.md` at that step, finish it
there, come back. Unattended, take a question's recommended option (log it); gate
dispositions and approvals stay pending and end the run. Keep a private invocation record (`--record`): write its six headings at Phase 1, then fill each as its phase ends: **Checks**
(Phases 2-4: lanes, pregate, evals, `Coverage:`, the plan checks, `Scope Check:`), **Decisions** (any
(Phases 2-4: lanes, pregate, evals, `Coverage:`, the plan checks, `Exploratory QA:`, `Scope Check:`), **Decisions** (any
phase: each answered or auto-taken question, verbatim), **Reviews** (Phases 3-5: the plan line,
start tokens, `Pre-Landing Review:`, `Gate edits:`, the reviewed tree), **Release** (Phases 5-7:
version, CHANGELOG, commits), **Documentation** (Step 14.5) and **Next steps** (every phase: the next act).
@@ -22,7 +22,7 @@ version, CHANGELOG, commits), **Documentation** (Step 14.5) and **Next steps** (
outside-voice and documentation brief, its severity and confidence rules and gates come from its
section verbatim (placeholders filled); this file
never shortens a reviewer's brief. A blocking reader's prompt is its `gstack-review-dispatch render`
file, unchanged; its findings reach `merged.jsonl` only via `collect`; `judge` refuses a drop or downgrade. Audit readers (Steps 7, 8, 9.2.1, 14.5) cannot block, so each runs on a
file, unchanged; its findings reach `merged.jsonl` only via `collect`; `judge` refuses a drop or downgrade. Audit readers (Steps 7, 8, 14.5) cannot block, so each runs on a
small model at low effort; their findings go to the PR body.
**Calls.** One pass with no fixes is 9 calls: Phases 1-8 and the report; at most 12 with a STOP's
@@ -75,13 +75,13 @@ Never read TODOS.md or the full diff into the parent. A `PERSIST:` hint never re
## 3. Audits and review: Steps 7, 8, 9, 9.1, 9.2, 9.2.1
One program: the audit readers `READER:` names (Step 7 from `sections/test-coverage.md`, Step 8 from
`sections/plan-completion.md`, each run as written; Step 7 may write tests, so it settles first), Step 9.2.1's QA reader (that step of
`sections/review-army.md` as its brief) and the pass (above): Steps 9-9.2 as written. `Coverage:` and Step 8's line come from `--audit` when no
`sections/plan-completion.md`, each run as written; Step 7 may write tests, so it settles first), 9.2.1's Reads only and the pass
(above): Steps 9-9.2 from `sections/review-army.md` as written. `Coverage:` and Step 8's line come from `--audit` when no
reader ran.
## 4. Judgment: Steps 8.1, 8.2, 9.3, 9.4 and Step 11
Steps 8.1 and 8.2 (`Scope Check:`) on the todos and commits files, then the core checklist, Steps
Step 8.1, 9.2.1's probes, 8.2 (`Scope Check:`) on the todos and commits files, then the core checklist, Steps
9.3, 9.4 and the repeat decision as written (`gstack-review-plan` included;
`JUDGMENT: ok` before any fix), fixes and persistence in
one program. Prints `Pre-Landing Review:`, `Gate edits:`. `NEXT: stop` stops; `full` or
Loaded 100 of 170 files, more files were not shown because too many files have changed in this diff. Show more