fix(test): adversarial findings — stable root key, symlink-alias dedupe, fixture-shape guard

Adversarial review (Claude subagent) verified the fixture's root-skill key was
the capture machine's checkout dirname: any non-gstack-named clone (every
Conductor worktree) failed the free suite, and the documented re-run-the-capture
recovery baked the local dirname into the committed fixture — silent corruption
through the tool's own protocol. The root skill is now pinned to ROOT_SKILL_KEY
('gstack', its frontmatter name). Symlink aliases are realpath-deduped (census
precedent): connect-chrome no longer gets its own ceiling, so Windows checkouts
that materialize the symlink as a plain file can't fail the stale-ceiling
set-equality test. New guards: fixture-shape validation (a string alwaysOnTotal
can no longer silently disable the ceiling), a mutation pin that the filter
shrinks the always-on ledger vs the raw bill, an alwaysOnTotal violation test
(the branch was load-bearing with only under-budget coverage), and an atomic
temp+rename fixture write. Fixture regenerated: 59 ceilings, alwaysOnTotal 6344.
Deferred with a TODO: anchoring transformFrontmatter's denylist strip to the
frontmatter block (latent, zero live collisions, pre-existing path).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-25 02:53:25 +00:00
co-authored by Claude Fable 5
parent d80d3c3ed7
commit c11e44b93a
5 changed files with 107 additions and 12 deletions
+43 -7
View File
@@ -56,17 +56,50 @@ export interface ContextBudget {
eagerPerInvocation: Record<string, number>;
}
/**
* The root SKILL.md's bill name falls back to the checkout directory's
* basename (path.relative gives '' at the root), which is machine-specific:
* a Conductor worktree named anything but "gstack" would mismatch the fixture
* key, and the documented "re-run the capture" recovery would then bake the
* local dirname INTO the committed fixture. Pin it to the skill's frontmatter
* name instead — stable across every clone.
*/
export const ROOT_SKILL_KEY = 'gstack';
/**
* The bill the ratchet grades: repo tree minus test-fixture skill dirs, with
* POSIX-normalized names and ALL totals rebuilt from the filtered list (a
* partially-updated totals object would hand fixture-polluted numbers to any
* future consumer of the perInvocation/totalMd fields).
* POSIX-normalized names, the root skill pinned to ROOT_SKILL_KEY, symlink
* aliases deduped by realpath (connect-chrome -> open-gstack-browser; on
* Windows checkouts the symlink materializes as a plain file and the alias
* dir vanishes, so budgeting it would make the stale-ceiling test
* platform-dependent — same dedupe the skill census uses), and ALL totals
* rebuilt from the filtered list (a partially-updated totals object would
* hand fixture-polluted numbers to any future consumer of the
* perInvocation/totalMd fields).
*/
export function buildRatchetBill(root: string = REPO_ROOT): Bill {
const bill = buildBill(root);
const skills = bill.skills
.map((s) => ({ ...s, name: toPosixName(s.name) }))
const candidates = bill.skills
.map((s) => ({
...s,
name: s.dir === bill.root ? ROOT_SKILL_KEY : toPosixName(s.name),
}))
.filter((s) => !isFixtureSkill(s.name));
// One ceiling per PHYSICAL skill: group by realpath, prefer the entry whose
// dir IS the realpath (the real dir) over symlink aliases.
const byReal = new Map<string, (typeof candidates)[number]>();
for (const s of candidates) {
let real: string;
try {
real = fs.realpathSync(s.dir);
} catch {
real = s.dir;
}
const cur = byReal.get(real);
if (!cur || (s.dir === real && cur.dir !== real)) byReal.set(real, s);
}
const kept = new Set(byReal.values());
const skills = candidates.filter((s) => kept.has(s));
return {
...bill,
skills,
@@ -102,10 +135,13 @@ export function captureContextBudget(root: string = REPO_ROOT): ContextBudget {
};
}
// CLI: write the fixture.
// CLI: write the fixture atomically (temp + rename) — an interrupted capture
// must never leave truncated JSON that breaks the suite at module load.
if (import.meta.main) {
const budget = captureContextBudget();
fs.writeFileSync(BUDGET_FIXTURE_PATH, JSON.stringify(budget, null, 2) + '\n');
const tmp = `${BUDGET_FIXTURE_PATH}.tmp-${process.pid}`;
fs.writeFileSync(tmp, JSON.stringify(budget, null, 2) + '\n');
fs.renameSync(tmp, BUDGET_FIXTURE_PATH);
const n = Object.keys(budget.eagerPerInvocation).length;
console.log(
`Wrote ${path.relative(REPO_ROOT, BUDGET_FIXTURE_PATH)}: alwaysOnTotal=${budget.alwaysOnTotal} tok, ${n} eager ceilings`,