diff --git a/CLAUDE.md b/CLAUDE.md index ced80711a..fefd383cd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -226,6 +226,16 @@ conversation. The failure message carries the re-measure + ratchet protocol. tree (always-on vs per-invocation, `--diff`, `--budget`; `--exact` opts into the real tokenizer and POSTs file text to api.anthropic.com with an egress receipt). +A third enforced ceiling is the context-budget ratchet: +`test/context-budget-ratchet.test.ts` (free, runs in `bun run test`) grades the +two ledgers the other guards don't cover — the always-on FULL-frontmatter +aggregate and each skill's per-invocation eager tokens (SKILL.md + forced-read +references) — against `test/fixtures/context-budget.json`. A skill that grows +past its ceiling fails; a new skill fails until it's consciously budgeted. For +legitimate growth or a landed reduction, re-run +`bun test/helpers/capture-context-budget.ts` and commit the refreshed fixture +in the same commit, so ceilings ratchet down and every win is locked. + **Merge conflicts on SKILL.md files:** NEVER resolve conflicts on generated SKILL.md files by accepting either side. Instead: (1) resolve conflicts on the `.tmpl` templates and `scripts/gen-skill-docs.ts` (the sources of truth), (2) run `bun run gen:skill-docs` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 77e188ae8..1dad1c54e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -174,6 +174,7 @@ eval files, and misses the strict classifier. No API keys needed. - **Generator tests** (`test/gen-skill-docs.test.ts`) — Tests the template system: verifies placeholders resolve correctly, output includes value hints for flags (e.g. `-d ` not just `-d`), enriched descriptions for key commands (e.g. `is` lists valid states, `press` lists key examples). - **Tier-alignment invariant** (`test/e2e-tier-alignment.test.ts`) — For every self-gated `test/skill-e2e-*.test.ts` named in a touchfiles dep list, the file's `EVALS_TIER` self-gate must match its declared tier in `E2E_TIERS`. Kills the "inert demotion" class where a test is re-tiered in `touchfiles.ts` but the file still gates on the old tier and keeps running in the wrong lane. Unmapped or mixed-tier files are reported, never silently skipped. - **Catalog budget** (`test/catalog-budget.test.ts`) — Caps the aggregate discovery surface: the sum of every skill's frontmatter `name` + `description` (what every host loads at discovery, every session) must stay under 1,150 token-equivalents, with a 260-byte per-skill cap. Counting goes through the shared census in `test/helpers/skill-census.ts` (physical files vs authored skills vs registry entries — three deliberately different counts). Adding a skill? The failure message carries the re-measure + ratchet protocol. +- **Context-budget ratchet** (`test/context-budget-ratchet.test.ts`) — CI ceilings on the two token ledgers the catalog budget doesn't cover: the always-on full-frontmatter aggregate and each skill's per-invocation eager tokens (SKILL.md + forced-read references), graded against `test/fixtures/context-budget.json` via `lib/context-bill.ts`. New skills fail until they have a ceiling; ceilings for removed skills must be pruned. Legitimate growth or a landed reduction: re-run `bun test/helpers/capture-context-budget.ts` and commit the refreshed fixture in the same commit, so the change is a visible decision in the diff. ### Tier 2: E2E via `claude -p` (~$3.85/run) @@ -418,7 +419,8 @@ When you add a new skill template, all hosts get it automatically: 1. Create `{skill}/SKILL.md.tmpl` 2. Run `bun run gen:skill-docs --host all` 3. The dynamic template discovery picks it up, no static list to update -4. Commit `{skill}/SKILL.md`, external host output is generated at setup time and gitignored +4. Budget it: run `bun test/helpers/capture-context-budget.ts` and commit the refreshed `test/fixtures/context-budget.json` — the context-budget ratchet fails any skill without a ceiling +5. Commit `{skill}/SKILL.md`, external host output is generated at setup time and gitignored ## Conductor workspaces