mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-15 01:15:29 +02:00
docs: sync every doc surface with the v1.63 release
/document-release audit (4-lane, all claims verified against branch code): - README: gstack-egress + gstack-context-bill rows in the standalone-binaries table; Privacy & Telemetry gains the receipted-egress bullet (attempted- egress framing per the shipped threat model). - ARCHITECTURE: /health is liveness-only, POST /extension-token endpoint row + bootstrap mechanics paragraph; new Egress receipt ledger subsection under Security model; eval persistence covers the sharded runner, GSTACK_EVAL_DIR, and the finalized-run baseline rule. - CLAUDE.md: sharded test scripts in Commands; sharded semantics in the detached-evals section; PTY skill seeding in the hermetic section; egress invariant block beside the other server-egress invariants; catalog-budget ceiling beside the 160KB token ceiling; project-tree entries for lib/egress-receipt.ts, lib/context-bill.ts, scripts/test-paid-shards.ts. - CONTRIBUTING: seedSkills + live-tree seeding in the hermetic paragraph; sharded runner in detached runs; catalog-budget in the Tier 1 list. - BROWSER: extension token bootstrap section, tunnel egress receipts section, identity-pin migration note in manual install. - REMOTE_BROWSER_ACCESS: tunnel-start receipt bullet in the security model. - gbrain docs: /sync-gbrain + brain-sync egress-receipt behavior documented; dead consumer-token instructions removed (consumer machinery deleted this release); new fail-closed refusal added to the error catalog. - CHANGELOG: measured-vs-ceiling catalog numbers, contributor notes for the external-service tier move and the PTY single-line AskUserQuestion parser, release date. - TODOS: /health token-distribution TODO resolved by this release, removed; port-wave follow-up sections re-labeled to the shipped version. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
562273d793
commit
e7110aac93
+20
-5
@@ -160,6 +160,7 @@ Runs automatically with `bun test`. No API keys needed.
|
||||
- **Skill validation tests** (`test/skill-validation.test.ts`) — Validates that SKILL.md files reference only real commands and flags, and that command descriptions meet quality thresholds.
|
||||
- **Generator tests** (`test/gen-skill-docs.test.ts`) — Tests the template system: verifies placeholders resolve correctly, output includes value hints for flags (e.g. `-d <N>` not just `-d`), enriched descriptions for key commands (e.g. `is` lists valid states, `press` lists key examples).
|
||||
- **Tier-alignment invariant** (`test/e2e-tier-alignment.test.ts`) — For every self-gated `test/skill-e2e-*.test.ts` named in a touchfiles dep list, the file's `EVALS_TIER` self-gate must match its declared tier in `E2E_TIERS`. Kills the "inert demotion" class where a test is re-tiered in `touchfiles.ts` but the file still gates on the old tier and keeps running in the wrong lane. Unmapped or mixed-tier files are reported, never silently skipped.
|
||||
- **Catalog budget** (`test/catalog-budget.test.ts`) — Caps the aggregate discovery surface: the sum of every skill's frontmatter `name` + `description` (what every host loads at discovery, every session) must stay under 1,150 token-equivalents, with a 260-byte per-skill cap. Counting goes through the shared census in `test/helpers/skill-census.ts` (physical files vs authored skills vs registry entries — three deliberately different counts). Adding a skill? The failure message carries the re-measure + ratchet protocol.
|
||||
|
||||
### Tier 2: E2E via `claude -p` (~$3.85/run)
|
||||
|
||||
@@ -184,10 +185,16 @@ seeded `CLAUDE_CONFIG_DIR`, a temp `GSTACK_HOME`, and `--strict-mcp-config`. You
|
||||
operator `~/.claude` config, MCP servers (gbrain, Conductor), skills, `~/.gstack`
|
||||
decision logs, and `CONDUCTOR_*` env never leak into the child, so local eval
|
||||
signal matches CI instead of disagreeing for reasons unrelated to the code under
|
||||
test. Set `EVALS_HERMETIC=0` to debug against your real operator state (this also
|
||||
test. The hermetic `CLAUDE_CONFIG_DIR` seeds no skills by default; a PTY test
|
||||
that types a `/skill` slash command passes `seedSkills: true` to the PTY runner,
|
||||
which swaps in `hermeticSkillsConfigDir()` — a seeded skill registry that
|
||||
symlinks the LIVE working tree's SKILL.md files (by design: the skills are the
|
||||
subject under test, so a snapshot would measure stale copies). Set
|
||||
`EVALS_HERMETIC=0` to debug against your real operator state (this also
|
||||
drops `--strict-mcp-config`). The wiring is pinned by `test/hermetic-wiring.test.ts`
|
||||
(a free static tripwire) and two gate-tier isolation canaries in
|
||||
`test/skill-e2e-hermetic-canary.test.ts`.
|
||||
(a free static tripwire), two gate-tier isolation canaries in
|
||||
`test/skill-e2e-hermetic-canary.test.ts`, and the skill-seeding tripwires in
|
||||
`test/hermetic-skills-seeding.test.ts` / `test/pty-skill-seeding-wiring.test.ts`.
|
||||
|
||||
### E2E observability
|
||||
|
||||
@@ -227,8 +234,16 @@ bun run eval:bg:gate # detached gate-tier suite
|
||||
bun run eval:bg:periodic # detached periodic-tier suite
|
||||
```
|
||||
|
||||
Each prints its log path. Humans running `bun run test:evals` foreground in their
|
||||
own terminal don't need this — Ctrl-C is intended there.
|
||||
Each prints its log path. The gate and periodic variants run their tier through
|
||||
the sharded paid runner (`scripts/test-paid-shards.ts`, also available directly
|
||||
as `bun run test:gate:sharded` / `bun run test:periodic:sharded`): one Bun
|
||||
process per test file, an external wall-clock timeout that kills the shard's
|
||||
whole process group (stray `claude`/`codex` grandchildren included), a per-shard
|
||||
eval dir (`GSTACK_EVAL_DIR=<evalDir>/shards/<slug>/`), and an aggregate that
|
||||
distinguishes failed vs timed-out vs never-started shards. `eval:list`,
|
||||
`eval:compare`, and `eval:summary` are shard-aware. Humans running
|
||||
`bun run test:evals` foreground in their own terminal don't need this — Ctrl-C
|
||||
is intended there.
|
||||
|
||||
**Eval comparison commentary:** `eval:compare` generates natural-language Takeaway sections interpreting what changed between runs — flagging regressions, noting improvements, calling out efficiency gains (fewer turns, faster, cheaper), and producing an overall summary. This is driven by `generateCommentary()` in `eval-store.ts`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user