test: pin the added-subsection note and readSkillEffective's changed-plus-added order (ship Step 7, GSTA-96)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-10-26 04:59:51 +00:00
1 parent 5db328f2b9
commit 01818d5dda
2 files changed
+65

No files matched your search

+1
View File
@@ -11,6 +11,7 @@
### Tests
- `test/helpers/skill-startup.ts` applies added subsections when it builds the effective text, so the hoisted-equals-inline check covers office-hours' new layout.
- `test/preamble-hoist.test.ts` checks, on every host and tier, that a cut-down variant carries the "add to its end" note exactly when it adds a subsection, and that `readSkillEffective` puts a changed subsection in place and appends added ones in order.
- `test/helpers/carve-guards.ts`: office-hours' parity cap goes down from 1.2279 to 1.1796, its measured union. No cap or pin goes up.
**Tier 2+ skills load the shared preamble once per session instead of once per invocation.** Every tier 2+ SKILL.md carried the same ~23 KB of preamble (AskUserQuestion format, voice, writing style, completeness, completion protocol and more), so each skill invocation re-read text the conversation already held. That text now renders once per host into `sections/preamble.md` beside each tier 2+ SKILL.md. The SKILL.md keeps a short "Shared preamble (load first)" pointer naming the file and its content id, and asks for the Read unless the skill's own earlier Read of that file, headed by that id, is still in context, so other text that merely carries the id cannot stand in for the shared rules. Measured with Anthropic token counting: review's SKILL.md drops from 21,884 to 16,103 tokens, ship from 22,688 to 16,907, office-hours from 27,100 to 23,085 and design-review from 39,040 to 33,261. The shared file is 5,953 tokens, so a session's first invocation costs about 170 tokens more than before (about 1,900 for office-hours, about 3,800 for plan-eng-review) and every later one saves about 5,800 (4,000 for office-hours, 2,200 for plan-eng-review). A /autoplan run over all four review phases reads 17,941 fewer tokens (191,871 → 173,930), because its review-skill methodology loads no longer repeat the preamble.
+64
View File
@@ -112,4 +112,68 @@ describe('shared-preamble hoist', () => {
expectAbsent(pointer, ['whose first heading names'], label(ctx));
}
});
// Generated by /ship Step 7 coverage audit (GSTA-96, PR #3453).
// Value: protects=a cut-down preamble variant that appends a subsection says so in its note, and a replace-only variant keeps the replace-only note;
// fails_when=the note collapses to "Only these subsections replace the shared ones" while office-hours appends ### Open-ended questions;
// why_new=the hoisted-equals-inline case passes for either note, since readSkillEffective accepts both; goldens cover ship only;
// seam=none; no_claim=that a model treats an added subsection differently (resolver output only)
test('a cut-down variant names an added subsection in its note, and only then', () => {
const REPLACE = 'Only these subsections replace the shared ones; the rest of this section is shared.';
const ADD = 'Only these subsections replace the shared ones or, under a new heading, add to its end; the rest of this section is shared.';
const subheadings = (section: string) => {
const heads: string[] = [];
let fence: string | null = null;
for (const line of section.split('\n')) {
const delimiter = /^ {0,3}(`{3,}|~{3,})/.exec(line)?.[1];
if (delimiter) {
if (!fence) fence = delimiter;
else if (delimiter[0] === fence[0] && delimiter.length >= fence.length) fence = null;
} else if (!fence && line.startsWith('### ')) heads.push(line);
}
return heads;
};
let adding = 0;
for (const ctx of contexts()) {
if (!hoistsPreamble(ctx)) continue;
const shared = new Map(sections(generateSharedPreambleFile(ctx)).map(section => [section.split('\n', 1)[0]!, section]));
for (const section of sections(generateSkillPreamble(ctx))) {
const note = section.includes(ADD) ? ADD : section.includes(REPLACE) ? REPLACE : null;
if (!note) continue;
const base = shared.get(section.split('\n', 1)[0]!);
expect(base, label(ctx)).toBeDefined();
const sharedHeads = new Set(subheadings(base!));
const adds = subheadings(section).some(heading => !sharedHeads.has(heading));
expect(note, label(ctx)).toBe(adds ? ADD : REPLACE);
if (adds) adding++;
}
}
expect(adding).toBeGreaterThan(0);
});
// Generated by /ship Step 7 coverage audit (GSTA-96, PR #3453).
// Value: protects=readSkillEffective rebuilds a variant section as shared A, changed B, then added C and D, in that order, and keeps the skill's own sections after it;
// fails_when=added subsections are dropped, put before the shared ones, reordered, or a changed subsection leaves the original in place;
// why_new=the generated-skill case has only office-hours' single added subsection and no variant that both changes and adds;
// seam=none; no_claim=that the generator emits this shape (the resolver cases above cover real renders)
test('readSkillEffective applies a changed subsection and appends added ones in order', () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-preamble-effective-'));
try {
const note = 'Only these subsections replace the shared ones or, under a new heading, add to its end; the rest of this section is shared.';
fs.mkdirSync(path.join(root, 'sections'));
fs.writeFileSync(path.join(root, 'sections', SHARED_PREAMBLE_FILE),
'# gstack shared preamble 0123456789ab\n\n## Rules\n\nShared intro.\n\n### A\n\nShared A.\n\n### B\n\nShared B.\n');
fs.writeFileSync(path.join(root, 'SKILL.md'), [
'# fixture', '', SHARED_POINTER_HEADING, '', 'Pointer text.', '',
'## Rules', '', note, '', '### B', '', 'Changed B.', '', '### C', '', 'Added C.', '', '### D', '', 'Added D.', '',
'## Own', '', 'Own body.', '',
].join('\n'));
expect(readSkillEffective(path.join(root, 'SKILL.md'))).toBe([
'# fixture', '', '## Rules', '', 'Shared intro.', '', '### A', '', 'Shared A.', '', '### B', '', 'Changed B.', '',
'### C', '', 'Added C.', '', '### D', '', 'Added D.', '', '## Own', '', 'Own body.', '',
].join('\n'));
} finally {
fs.rmSync(root, { recursive: true, force: true });
}
});
});