CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-09-18 02:42:25 +02:00
Code Issues Packages Projects Releases Wiki Activity

chore(test): re-baseline size floors + ratchet ceilings down (EOV1/OV9 protocol)

Browse Source 
parity-baseline-v1.69.1.0.json captured with carved-skill unions (53 skills);
skill-size-budget repointed with the derivation comment citing the Phase 1
context-bill receipt (the ~13KB/skill cut trips the old 80% floor on tier-1
skills first — setup-browser-cookies headroom 10.8KB < the cut). The v1.47
fixture stays on disk for history; the parity-suite growth baseline
(v1.64.1.0) is untouched. Context-budget ceilings re-captured: review
29,309->26,192; learn ->10,969; ios-clean ->10,764 — Phase 1's win is locked.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-25 15:37:00 +00:00
co-authored by Claude Fable 5
parent 17bebe33ab
commit 135bf5d420
3 changed files with 703 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/skill-size-budget.test.ts
+11 -6
View File
@@ -14,8 +14,13 @@
* compares LIVE eval runs (tool calls, turns, cost); this one compares
* static SKILL.md sizes. Both gate-tier.
*
* The baseline lives at test/fixtures/parity-baseline-v1.47.0.0.json,
* captured by scripts/capture-baseline.ts before any Phase A work landed.
* The baseline lives at test/fixtures/parity-baseline-v1.69.1.0.json,
* re-captured 2026-08-25 during token-reduction Phase 1 (bash consolidation
* moved ~11-13KB of inline preamble bash per skill into bin/gstack-skill-start
* and bin/gstack-skill-end — a deliberate corpus-wide shrink; receipt:
* gstack-context-bill --diff in PR #2691). The prior v1.47.0.0 fixture stays
* on disk for history. Live pins at capture time: this test (shrink floor)
* and test/parity-suite.test.ts vs parity-baseline-v1.64.1.0.json (growth).
*
* Override:
* - GSTACK_SIZE_BUDGET_RATIO=<n> changes the per-skill regression ratio.
@@ -37,7 +42,7 @@ import { logBudgetOverride } from './helpers/budget-override';
import { CARVED_SKILLS } from './helpers/carve-guards';
const REPO_ROOT = path.resolve(import.meta.dir, '..');
const BASELINE_PATH = path.join(REPO_ROOT, 'test', 'fixtures', 'parity-baseline-v1.47.0.0.json');
const BASELINE_PATH = path.join(REPO_ROOT, 'test', 'fixtures', 'parity-baseline-v1.69.1.0.json');
// Default per-skill ratio is 1.50 (50% growth tolerance). Adjusted v1.52.0.0
// (cathedral cap audit) from 1.05 → 1.50: a 5% ratio tripped on legitimate
@@ -57,11 +62,11 @@ interface Regression {
}
describe('SKILL.md size budget regression (gate, free)', () => {
test('parity-baseline-v1.47.0.0.json exists', () => {
test('parity-baseline-v1.69.1.0.json exists', () => {
expect(fs.existsSync(BASELINE_PATH)).toBe(true);
});
test('no skill exceeds v1.47.0.0 baseline size × ratio', () => {
test('no skill exceeds v1.69.1.0 baseline size × ratio', () => {
const baseline: ParityBaseline = JSON.parse(fs.readFileSync(BASELINE_PATH, 'utf-8'));
const current = captureBaseline({ repoRoot: REPO_ROOT });
@@ -157,7 +162,7 @@ describe('SKILL.md size budget regression (gate, free)', () => {
* sectioned invariant in parity-harness.ts (minBytes on skeleton+sections).
* Add the remaining three here as they carve.
*/
test('no skill shrinks past 80% of v1.47.0.0 baseline (catches accidental body strip)', () => {
test('no skill shrinks past 80% of v1.69.1.0 baseline (catches accidental body strip)', () => {
const baseline: ParityBaseline = JSON.parse(fs.readFileSync(BASELINE_PATH, 'utf-8'));
const current = captureBaseline({ repoRoot: REPO_ROOT });
const MIN_RATIO = 0.80; // a skill at <80% of its v1.44 size signals mass-deletion