mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-18 10:52:24 +02:00
docs: sync every doc surface with the v1.63 release
/document-release audit (4-lane, all claims verified against branch code): - README: gstack-egress + gstack-context-bill rows in the standalone-binaries table; Privacy & Telemetry gains the receipted-egress bullet (attempted- egress framing per the shipped threat model). - ARCHITECTURE: /health is liveness-only, POST /extension-token endpoint row + bootstrap mechanics paragraph; new Egress receipt ledger subsection under Security model; eval persistence covers the sharded runner, GSTACK_EVAL_DIR, and the finalized-run baseline rule. - CLAUDE.md: sharded test scripts in Commands; sharded semantics in the detached-evals section; PTY skill seeding in the hermetic section; egress invariant block beside the other server-egress invariants; catalog-budget ceiling beside the 160KB token ceiling; project-tree entries for lib/egress-receipt.ts, lib/context-bill.ts, scripts/test-paid-shards.ts. - CONTRIBUTING: seedSkills + live-tree seeding in the hermetic paragraph; sharded runner in detached runs; catalog-budget in the Tier 1 list. - BROWSER: extension token bootstrap section, tunnel egress receipts section, identity-pin migration note in manual install. - REMOTE_BROWSER_ACCESS: tunnel-start receipt bullet in the security model. - gbrain docs: /sync-gbrain + brain-sync egress-receipt behavior documented; dead consumer-token instructions removed (consumer machinery deleted this release); new fail-closed refusal added to the error catalog. - CHANGELOG: measured-vs-ceiling catalog numbers, contributor notes for the external-service tier move and the PTY single-line AskUserQuestion parser, release date. - TODOS: /health token-distribution TODO resolved by this release, removed; port-wave follow-up sections re-labeled to the shipped version. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
562273d793
commit
e7110aac93
@@ -9,6 +9,8 @@ bun run test:evals # run paid evals: LLM judge + E2E (diff-based, ~$4/run max)
|
||||
bun run test:evals:all # run ALL paid evals regardless of diff
|
||||
bun run test:gate # run gate-tier tests only (CI default, blocks merge)
|
||||
bun run test:periodic # run periodic-tier tests only (weekly cron / manual)
|
||||
bun run test:gate:sharded # gate tier via the sharded paid runner (one Bun process per test file)
|
||||
bun run test:periodic:sharded # periodic tier via the sharded paid runner (implies EVALS_ALL=1)
|
||||
bun run test:e2e # run E2E tests only (diff-based, ~$3.85/run max)
|
||||
bun run test:e2e:all # run ALL E2E tests regardless of diff
|
||||
bun run eval:select # show which tests would run based on current diff
|
||||
@@ -48,13 +50,21 @@ MCP servers / skills), a temp `GSTACK_HOME`, and `--strict-mcp-config`.
|
||||
Local eval signal matches CI. Debug against real operator state with
|
||||
`EVALS_HERMETIC=0` (restores the legacy env AND drops the strict-MCP flag).
|
||||
Per-test `env:` overrides merge last, so deliberate contamination
|
||||
(`CONDUCTOR_WORKSPACE_PATH`, per-test `GSTACK_HOME`) keeps working. Wiring
|
||||
is pinned by `test/hermetic-wiring.test.ts` (static tripwire) and two
|
||||
gate-tier canaries in `test/skill-e2e-hermetic-canary.test.ts`.
|
||||
(`CONDUCTOR_WORKSPACE_PATH`, per-test `GSTACK_HOME`) keeps working. The
|
||||
hermetic config dir seeds NO skills by default; a PTY test that types a
|
||||
`/skill` slash command must pass `seedSkills: true` to the PTY runner, which
|
||||
points the child's `CLAUDE_CONFIG_DIR` at `hermeticSkillsConfigDir()` — a
|
||||
seeded registry that symlinks the LIVE working tree's SKILL.md files (by
|
||||
design: the skills ARE the subject under test; a snapshot would measure stale
|
||||
copies). Wiring is pinned by `test/hermetic-wiring.test.ts` (static tripwire),
|
||||
two gate-tier canaries in `test/skill-e2e-hermetic-canary.test.ts`, and the
|
||||
seeding tripwires in `test/hermetic-skills-seeding.test.ts` /
|
||||
`test/pty-skill-seeding-wiring.test.ts`.
|
||||
|
||||
E2E tests stream progress in real-time (tool-by-tool via `--output-format stream-json
|
||||
--verbose`). Results are persisted to `~/.gstack-dev/evals/` with auto-comparison
|
||||
against the previous run.
|
||||
against the previous finalized run (in-flight `_partial` files are never used as
|
||||
a baseline, so a run can't compare against itself).
|
||||
|
||||
**Diff-based test selection:** `test:evals` and `test:e2e` auto-select tests based
|
||||
on `git diff` against the base branch. Each test declares its file dependencies in
|
||||
@@ -110,6 +120,7 @@ gstack/
|
||||
│ ├── host-adapters/ # Host-specific adapters (OpenClaw tool mapping)
|
||||
│ ├── resolvers/ # Template resolver modules (preamble, design, review, gbrain, etc.)
|
||||
│ ├── skill-check.ts # Health dashboard
|
||||
│ ├── test-paid-shards.ts # Sharded paid-tier runner (one Bun process per shard)
|
||||
│ └── dev-skill.ts # Watch mode
|
||||
├── test/ # Skill validation + eval tests
|
||||
│ ├── helpers/ # skill-parser.ts, session-runner.ts, llm-judge.ts, eval-store.ts
|
||||
@@ -147,7 +158,7 @@ gstack/
|
||||
│ ├── test/ # Integration tests
|
||||
│ └── dist/ # Compiled binary
|
||||
├── extension/ # Chrome extension (side panel + activity feed + CSS inspector)
|
||||
├── lib/ # Shared libraries (worktree.ts)
|
||||
├── lib/ # Shared libraries (worktree.ts, egress-receipt.ts, context-bill.ts, redact-engine.ts)
|
||||
├── docs/designs/ # Design documents
|
||||
├── setup-deploy/ # /setup-deploy skill (one-time deploy config)
|
||||
├── .github/ # CI workflows + Docker image
|
||||
@@ -184,6 +195,16 @@ behavior). If you blow past 40K, the right fix is usually: (1) look at WHAT grew
|
||||
or as a reference doc, (3) only compress carefully-tuned prose as a last resort —
|
||||
cuts to the coverage audit, review army, or voice directive have real quality cost.
|
||||
|
||||
A second, harder ceiling guards the DISCOVERY surface: `test/catalog-budget.test.ts`
|
||||
caps the aggregate frontmatter `name` + `description` across all skills at 1,150
|
||||
token-equivalents (260-byte per-skill sub-cap), counted through the shared census
|
||||
in `test/helpers/skill-census.ts`. This one is enforced, not a warning — every
|
||||
host loads the full catalog every session, so growth here taxes every
|
||||
conversation. The failure message carries the re-measure + ratchet protocol.
|
||||
`bin/gstack-context-bill` shows the full token bill-of-materials for a skills
|
||||
tree (always-on vs per-invocation, `--diff`, `--budget`; `--exact` opts into the
|
||||
real tokenizer and POSTs file text to api.anthropic.com with an egress receipt).
|
||||
|
||||
**Merge conflicts on SKILL.md files:** NEVER resolve conflicts on generated SKILL.md
|
||||
files by accepting either side. Instead: (1) resolve conflicts on the `.tmpl` templates
|
||||
and `scripts/gen-skill-docs.ts` (the sources of truth), (2) run `bun run gen:skill-docs`
|
||||
@@ -319,6 +340,21 @@ response in `server.ts`, read
|
||||
`browse/test/server-sanitize-surrogates.test.ts` pins the wiring with invariant
|
||||
tests, so bypasses fail CI.
|
||||
|
||||
**Egress receipts at every off-machine sink** (v1.63.0.0+). Every gstack-initiated
|
||||
send off the machine MUST write a hash-chained receipt to
|
||||
`~/.gstack/security/egress.jsonl` BEFORE the send: TypeScript callers use
|
||||
`writeReceipt` from `lib/egress-receipt.ts`; shell scripts source
|
||||
`bin/gstack-egress-lib.sh` and use `_receipted_curl` / `_receipted_git`. Failure
|
||||
polarity is per-class: fail-closed for sensitive sinks (brain-sync, memory-ingest,
|
||||
gbrain-sync, telemetry, ngrok tunnels, mcp-verify, supabase-provision), fail-open
|
||||
+ stderr warning for user-facing ones (design OpenAI calls, update-check,
|
||||
dashboards, git-class ops). The zero-exception scanner in
|
||||
`test/egress-receipt-wiring.test.ts` fails CI on any unreceipted `curl` /
|
||||
`git push` / `fetch` to a non-loopback host — if you add a new off-machine sink,
|
||||
wire it through the helpers and add it to the enumerated sink list. Inspect with
|
||||
`bin/gstack-egress` (`list` | `verify`, exit 3 on tamper | `grants`). Threat
|
||||
model: forensic observability of ATTEMPTED egress, not an exfiltration control.
|
||||
|
||||
**SSE endpoint helper** (v1.51.0.0+). New SSE endpoints in `server.ts` MUST route
|
||||
through `createSseEndpoint(req, config)` from `browse/src/sse-helpers.ts`. The
|
||||
helper owns the cleanup contract (abort + enqueue-throw + heartbeat-throw, all
|
||||
@@ -870,7 +906,17 @@ the run can also die to idle-sleep. `gstack-detach` fixes both: a fresh session
|
||||
machine-wide `gstack-evals` lock (concurrent worktrees serialize instead of
|
||||
saturating the shared model API), a per-tier watchdog, and a **run-scoped** log
|
||||
under `~/.gstack-dev/eval-runs/` (no shared-`/tmp` collision). Each prints its
|
||||
log path. Or call `gstack-detach [--lock NAME] [--timeout SECS] [--label LBL] --
|
||||
log path. `eval:bg:gate` / `eval:bg:periodic` run their tier through the
|
||||
sharded paid runner (`scripts/test-paid-shards.ts`, also exposed as
|
||||
`test:gate:sharded` / `test:periodic:sharded`): one Bun process per test
|
||||
file, an external wall-clock timeout that kills the shard's process GROUP
|
||||
(stray `claude`/`codex` grandchildren included), a per-shard
|
||||
`GSTACK_EVAL_DIR=<evalDir>/shards/<slug>/` honored by the `EvalCollector`
|
||||
constructor, and an aggregate that separates failed vs timed-out vs
|
||||
never-started shards — the detach timeouts (25200s gate / 28800s periodic)
|
||||
are sized against worst-case shard wall clock. `eval:list` / `eval:compare` /
|
||||
`eval:summary` read the shard dirs too. Or call
|
||||
`gstack-detach [--lock NAME] [--timeout SECS] [--label LBL] --
|
||||
<cmd>` directly for any long agent job. Export `ANTHROPIC_API_KEY` first (never
|
||||
pass keys in argv).
|
||||
- Then **poll the printed logfile** with a death-aware watcher: break on the
|
||||
|
||||
Reference in New Issue
Block a user