mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-09 06:28:59 +02:00
docs: close cross-model doc-review gaps for v1.72.0.0
setup's manual bun-verify instruction gets the same sha256sum-first fallback the automated snippet got (coreutils-only Linux); BROWSER.md's BROWSE_STATE_FILE row now lists the under-root world-writable refusal; test-cost ceilings in CLAUDE.md/CONTRIBUTING.md updated for the five new gate E2E cases (~$4.20 E2E / ~$4.35 evals). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
eb8742635c
commit
917a74e033
+1
-1
@@ -1269,7 +1269,7 @@ the global `~/.gstack/browser-skills/foo/` only inside project-a.
|
||||
|----------|---------|-------------|
|
||||
| `BROWSE_PORT` | 0 (random 10000–49151) | Fixed port for the HTTP server (debug override) |
|
||||
| `BROWSE_IDLE_TIMEOUT` | 1800000 (30 min) | Idle shutdown timeout in ms |
|
||||
| `BROWSE_STATE_FILE` | `.gstack/browse.json` | Path to state file. Its parent dir gets owner-only (0700) hardening only when gstack owns it — shared sticky dirs (`/tmp`, `/var/tmp`), foreign-owned dirs, and symlinked dirs are left untouched with a one-time warning (v1.72.0.0+) |
|
||||
| `BROWSE_STATE_FILE` | `.gstack/browse.json` | Path to state file. Its parent dir gets owner-only (0700) hardening only when gstack owns it — shared sticky dirs (`/tmp`, `/var/tmp`), foreign-owned dirs, symlinked dirs, and (under root) any world-writable dir are left untouched with a one-time warning (v1.72.0.0+) |
|
||||
| `BROWSE_SERVER_SCRIPT` | auto-detected | Path to `server.ts` |
|
||||
| `BROWSE_CDP_URL` | (none) | Set to `channel:chrome` for real-browser mode |
|
||||
| `BROWSE_CDP_PORT` | 0 | CDP port (used internally) |
|
||||
|
||||
@@ -5,13 +5,13 @@
|
||||
```bash
|
||||
bun install # install dependencies
|
||||
bun run test # run free tests via the strict parallel runner (~90-100s full suite)
|
||||
bun run test:evals # run paid evals: LLM judge + E2E (diff-based, ~$4/run max)
|
||||
bun run test:evals # run paid evals: LLM judge + E2E (diff-based, ~$4.35/run max)
|
||||
bun run test:evals:all # run ALL paid evals regardless of diff
|
||||
bun run test:gate # run gate-tier tests only (CI default, blocks merge)
|
||||
bun run test:periodic # run periodic-tier tests only (weekly cron / manual)
|
||||
bun run test:gate:sharded # gate tier via the sharded paid runner (one Bun process per test file)
|
||||
bun run test:periodic:sharded # periodic tier via the sharded paid runner (implies EVALS_ALL=1)
|
||||
bun run test:e2e # run E2E tests only (diff-based, ~$3.85/run max)
|
||||
bun run test:e2e # run E2E tests only (diff-based, ~$4.20/run max)
|
||||
bun run test:e2e:all # run ALL E2E tests regardless of diff
|
||||
bun run eval:select # show which tests would run based on current diff
|
||||
bun run dev <cmd> # run CLI in dev mode, e.g. bun run dev goto https://example.com
|
||||
@@ -67,7 +67,7 @@ in sync.
|
||||
|
||||
```bash
|
||||
bun run test # run before every commit — free, ~90-100s for the full ~7,000-test suite
|
||||
bun run test:evals # run before shipping — paid, diff-based (~$4/run max)
|
||||
bun run test:evals # run before shipping — paid, diff-based (~$4.35/run max)
|
||||
```
|
||||
|
||||
`bun run test` routes through `scripts/test-free-shards.ts` (N concurrent
|
||||
|
||||
+3
-3
@@ -149,14 +149,14 @@ Bun auto-loads `.env` — no extra config. Conductor workspaces inherit `.env` f
|
||||
| Tier | Command | Cost | What it tests |
|
||||
|------|---------|------|---------------|
|
||||
| 1 — Static | `bun run test` | Free | Command validation, snapshot flags, SKILL.md correctness, TODOS-format.md refs, observability unit tests |
|
||||
| 2 — E2E | `bun run test:e2e` | ~$3.85 | Full skill execution via `claude -p` subprocess |
|
||||
| 2 — E2E | `bun run test:e2e` | ~$4.20 | Full skill execution via `claude -p` subprocess |
|
||||
| 3 — LLM eval | `EVALS=1 bun test test/skill-llm-eval.test.ts` | ~$0.15 standalone | LLM-as-judge scoring of generated SKILL.md docs |
|
||||
| 2+3 | `bun run test:evals` | ~$4 combined | E2E + LLM-as-judge (runs both) |
|
||||
|
||||
```bash
|
||||
bun run test # Tier 1 only (run before every commit, ~90-100s for the full ~7,000-test suite)
|
||||
bun run test:e2e # Tier 2: E2E only (needs EVALS=1, can't run inside Claude Code)
|
||||
bun run test:evals # Tier 2 + 3 combined (~$4/run)
|
||||
bun run test:evals # Tier 2 + 3 combined (~$4.35/run)
|
||||
```
|
||||
|
||||
### Tier 1: Static validation (free)
|
||||
@@ -176,7 +176,7 @@ eval files, and misses the strict classifier. No API keys needed.
|
||||
- **Catalog budget** (`test/catalog-budget.test.ts`) — Caps the aggregate discovery surface: the sum of every skill's frontmatter `name` + `description` (what every host loads at discovery, every session) must stay under 1,150 token-equivalents, with a 260-byte per-skill cap. Counting goes through the shared census in `test/helpers/skill-census.ts` (physical files vs authored skills vs registry entries — three deliberately different counts). Adding a skill? The failure message carries the re-measure + ratchet protocol.
|
||||
- **Context-budget ratchet** (`test/context-budget-ratchet.test.ts`) — CI ceilings on the two token ledgers the catalog budget doesn't cover: the always-on full-frontmatter aggregate and each skill's per-invocation eager tokens (SKILL.md + forced-read references), graded against `test/fixtures/context-budget.json` via `lib/context-bill.ts`. New skills fail until they have a ceiling; ceilings for removed skills must be pruned. Legitimate growth or a landed reduction: re-run `bun test/helpers/capture-context-budget.ts` and commit the refreshed fixture in the same commit, so the change is a visible decision in the diff.
|
||||
|
||||
### Tier 2: E2E via `claude -p` (~$3.85/run)
|
||||
### Tier 2: E2E via `claude -p` (~$4.20/run)
|
||||
|
||||
Spawns `claude -p` as a subprocess with `--output-format stream-json --verbose`, streams NDJSON for real-time progress, and scans for browse errors. This is the closest thing to "does this skill actually work end-to-end?"
|
||||
|
||||
|
||||
@@ -55,7 +55,7 @@ if ! command -v bun >/dev/null 2>&1; then
|
||||
echo ' BUN_VERSION="1.3.10"' >&2
|
||||
echo ' tmpfile=$(mktemp)' >&2
|
||||
echo ' curl -fsSL "https://bun.sh/install" -o "$tmpfile"' >&2
|
||||
echo ' echo "Verify checksum before running: shasum -a 256 $tmpfile"' >&2
|
||||
echo ' echo "Verify checksum before running: sha256sum $tmpfile # or: shasum -a 256 $tmpfile"' >&2
|
||||
echo ' BUN_VERSION="$BUN_VERSION" bash "$tmpfile" && rm "$tmpfile"' >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
Reference in New Issue
Block a user