From 917a74e033b7ecf05b80ba6398f6385dabd1ee10 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Fri, 28 Aug 2026 17:26:57 +0000 Subject: [PATCH] docs: close cross-model doc-review gaps for v1.72.0.0 setup's manual bun-verify instruction gets the same sha256sum-first fallback the automated snippet got (coreutils-only Linux); BROWSER.md's BROWSE_STATE_FILE row now lists the under-root world-writable refusal; test-cost ceilings in CLAUDE.md/CONTRIBUTING.md updated for the five new gate E2E cases (~$4.20 E2E / ~$4.35 evals). Co-Authored-By: Claude Fable 5 --- BROWSER.md | 2 +- CLAUDE.md | 6 +++--- CONTRIBUTING.md | 6 +++--- setup | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/BROWSER.md b/BROWSER.md index d2e83fda0..b4f62bd41 100644 --- a/BROWSER.md +++ b/BROWSER.md @@ -1269,7 +1269,7 @@ the global `~/.gstack/browser-skills/foo/` only inside project-a. |----------|---------|-------------| | `BROWSE_PORT` | 0 (random 10000–49151) | Fixed port for the HTTP server (debug override) | | `BROWSE_IDLE_TIMEOUT` | 1800000 (30 min) | Idle shutdown timeout in ms | -| `BROWSE_STATE_FILE` | `.gstack/browse.json` | Path to state file. Its parent dir gets owner-only (0700) hardening only when gstack owns it — shared sticky dirs (`/tmp`, `/var/tmp`), foreign-owned dirs, and symlinked dirs are left untouched with a one-time warning (v1.72.0.0+) | +| `BROWSE_STATE_FILE` | `.gstack/browse.json` | Path to state file. Its parent dir gets owner-only (0700) hardening only when gstack owns it — shared sticky dirs (`/tmp`, `/var/tmp`), foreign-owned dirs, symlinked dirs, and (under root) any world-writable dir are left untouched with a one-time warning (v1.72.0.0+) | | `BROWSE_SERVER_SCRIPT` | auto-detected | Path to `server.ts` | | `BROWSE_CDP_URL` | (none) | Set to `channel:chrome` for real-browser mode | | `BROWSE_CDP_PORT` | 0 | CDP port (used internally) | diff --git a/CLAUDE.md b/CLAUDE.md index 94f54c107..3a19fc98f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,13 +5,13 @@ ```bash bun install # install dependencies bun run test # run free tests via the strict parallel runner (~90-100s full suite) -bun run test:evals # run paid evals: LLM judge + E2E (diff-based, ~$4/run max) +bun run test:evals # run paid evals: LLM judge + E2E (diff-based, ~$4.35/run max) bun run test:evals:all # run ALL paid evals regardless of diff bun run test:gate # run gate-tier tests only (CI default, blocks merge) bun run test:periodic # run periodic-tier tests only (weekly cron / manual) bun run test:gate:sharded # gate tier via the sharded paid runner (one Bun process per test file) bun run test:periodic:sharded # periodic tier via the sharded paid runner (implies EVALS_ALL=1) -bun run test:e2e # run E2E tests only (diff-based, ~$3.85/run max) +bun run test:e2e # run E2E tests only (diff-based, ~$4.20/run max) bun run test:e2e:all # run ALL E2E tests regardless of diff bun run eval:select # show which tests would run based on current diff bun run dev # run CLI in dev mode, e.g. bun run dev goto https://example.com @@ -67,7 +67,7 @@ in sync. ```bash bun run test # run before every commit — free, ~90-100s for the full ~7,000-test suite -bun run test:evals # run before shipping — paid, diff-based (~$4/run max) +bun run test:evals # run before shipping — paid, diff-based (~$4.35/run max) ``` `bun run test` routes through `scripts/test-free-shards.ts` (N concurrent diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1dad1c54e..a301a1ff8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -149,14 +149,14 @@ Bun auto-loads `.env` — no extra config. Conductor workspaces inherit `.env` f | Tier | Command | Cost | What it tests | |------|---------|------|---------------| | 1 — Static | `bun run test` | Free | Command validation, snapshot flags, SKILL.md correctness, TODOS-format.md refs, observability unit tests | -| 2 — E2E | `bun run test:e2e` | ~$3.85 | Full skill execution via `claude -p` subprocess | +| 2 — E2E | `bun run test:e2e` | ~$4.20 | Full skill execution via `claude -p` subprocess | | 3 — LLM eval | `EVALS=1 bun test test/skill-llm-eval.test.ts` | ~$0.15 standalone | LLM-as-judge scoring of generated SKILL.md docs | | 2+3 | `bun run test:evals` | ~$4 combined | E2E + LLM-as-judge (runs both) | ```bash bun run test # Tier 1 only (run before every commit, ~90-100s for the full ~7,000-test suite) bun run test:e2e # Tier 2: E2E only (needs EVALS=1, can't run inside Claude Code) -bun run test:evals # Tier 2 + 3 combined (~$4/run) +bun run test:evals # Tier 2 + 3 combined (~$4.35/run) ``` ### Tier 1: Static validation (free) @@ -176,7 +176,7 @@ eval files, and misses the strict classifier. No API keys needed. - **Catalog budget** (`test/catalog-budget.test.ts`) — Caps the aggregate discovery surface: the sum of every skill's frontmatter `name` + `description` (what every host loads at discovery, every session) must stay under 1,150 token-equivalents, with a 260-byte per-skill cap. Counting goes through the shared census in `test/helpers/skill-census.ts` (physical files vs authored skills vs registry entries — three deliberately different counts). Adding a skill? The failure message carries the re-measure + ratchet protocol. - **Context-budget ratchet** (`test/context-budget-ratchet.test.ts`) — CI ceilings on the two token ledgers the catalog budget doesn't cover: the always-on full-frontmatter aggregate and each skill's per-invocation eager tokens (SKILL.md + forced-read references), graded against `test/fixtures/context-budget.json` via `lib/context-bill.ts`. New skills fail until they have a ceiling; ceilings for removed skills must be pruned. Legitimate growth or a landed reduction: re-run `bun test/helpers/capture-context-budget.ts` and commit the refreshed fixture in the same commit, so the change is a visible decision in the diff. -### Tier 2: E2E via `claude -p` (~$3.85/run) +### Tier 2: E2E via `claude -p` (~$4.20/run) Spawns `claude -p` as a subprocess with `--output-format stream-json --verbose`, streams NDJSON for real-time progress, and scans for browse errors. This is the closest thing to "does this skill actually work end-to-end?" diff --git a/setup b/setup index 21ec6c0b0..c7b95b7bc 100755 --- a/setup +++ b/setup @@ -55,7 +55,7 @@ if ! command -v bun >/dev/null 2>&1; then echo ' BUN_VERSION="1.3.10"' >&2 echo ' tmpfile=$(mktemp)' >&2 echo ' curl -fsSL "https://bun.sh/install" -o "$tmpfile"' >&2 - echo ' echo "Verify checksum before running: shasum -a 256 $tmpfile"' >&2 + echo ' echo "Verify checksum before running: sha256sum $tmpfile # or: shasum -a 256 $tmpfile"' >&2 echo ' BUN_VERSION="$BUN_VERSION" bash "$tmpfile" && rm "$tmpfile"' >&2 exit 1 fi