Files
gstack/test/skill-gates-advisory-bytes.test.ts
T
Garry Tanandcapy-ai[bot] ef99eb5a55 feat: retire /investigate after a no-gain eval (GST-1717) (#4110)
* feat: retire /investigate after a no-gain eval (GST-1717)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(skill-start): core routing offer drops every non-core rule line (GST-1717)

The retired /investigate rule became 'Bugs/errors → fix them directly', which no
longer matches '→ invoke /', so proactive=core leaked it into the offered section.
Match any '- … → ' rule line.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* revert(review): keep review/checklist.md at its no-loss-pinned bytes (GST-1717)

The pin names replay GSTA-224 H1; a reviewer checklist edit merges only after a
passing no-loss replay (crew rule 15). The /investigate flake line stays until a
replayed repin.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: repair free-suite fallout from retiring /investigate (GST-1717)

CI run 42340956269 at 83a8a9a: drop fixtures, ledger rows, allowlist entries and
counts for the retired skill and its deleted paid files; record the harness
change as non-behavioral; the test-of-test detector accepts path.join(ROOT, 'bin');
uncovered-prompt fixtures use a tracked prompt, since an untracked path now
reads as a deleted one.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(relink): back up a retired skill's real SKILL.md before cleaning it (GST-1717)

With the skill's source gone, the customized-file check never ran, so a
banner-carrying local copy of investigate/SKILL.md was deleted instead of moved
to the backup root (red-team review, pass 1).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(qm): restore the review checklist's flake-line rewrite (GST-1717)

review/checklist.md keeps its pinned /investigate line, so the QM edition needs
its replaceText rule back; qm/ stays fresh against the generator.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(changelog): name the setup flag break, the relink backup and the pinned checklist line (GST-1717)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(setup,relink): back up a retired skill's real SKILL.md instead of deleting it (GST-1717)

Scoped fix round under grant a-360dd62bd46a (outside-voice blocking finding and
adversarial #1, #4). setup's prune compares a copied host entry with the render
before removing that render, and backs up a copy that differs or whose render is
gone; relink no longer takes the marker's rm -rf for a skill whose source is
gone, so a real SKILL.md is backed up and the user's files stay. setup reports a
failed remove-source of the retired gate hooks on stderr.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pr-profile): a changed prompt counts as deleted only when it is gone from disk (GST-1717)

An untracked prompt file on disk still needs prompt validation (targeted review
finding); harness change recorded as non-behavioral.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(changelog): exact backup wording, the hook-removal warning and GST-1744 (GST-1717)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(setup): one backup folder per host for a retired skill's copies (GST-1717)

Under grant a-c95576b8b1f9 (fix-check#1): every copy host moved its
gstack-investigate/SKILL.md to the same backup path, so a later host's copy
overwrote an earlier host's edits. A sandboxed ./setup --host factory run showed
gen-skill-docs removes the retired render before the prune, so every real copy
is backed up; each host now gets its own folder.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(changelog): say exactly what setup backs up for a retired skill (GST-1717)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(changelog): end the two colon sentences prose-lint flagged (GST-1717)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(retired-skills): name plan-eng-review once behind a pregate-literal waiver (GST-1717)

pregate's literals check read the GSTACK_RETIRED_SKILLS change as an old
value still in the tests. plan-eng-review is still retired; investigate was
appended. The fixture names it once, with the waiver.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-11-07 06:52:04 +00:00

48 lines
3.0 KiB
TypeScript

/**
* The enforcement hooks are opt-in and leave the advisory skills alone
* (GST-1461): turning a gate on changes no byte of /plan-ceo-review's
* SKILL.md. The generator renders the committed bytes with the switch on, and
* an armed gate run writes nothing under the installed skills dir.
*/
import { afterEach, beforeEach, describe, expect, test } from 'bun:test';
import { spawnSync } from 'node:child_process';
import { createHash } from 'node:crypto';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { bunIsolationArgs } from '../lib/bun-isolation';
import { ROOT, hookEnv, result, runHook, say, tmpDir, tool, user } from './helpers/skill-contract-fixture';
function filesUnder(dir: string): string[] {
return fs.readdirSync(dir, { recursive: true }).map(String).filter(f => fs.statSync(path.join(dir, f)).isFile()).map(f => path.join(dir, f));
}
const digest = (dir: string) => Object.fromEntries(filesUnder(dir).map(f => [path.relative(dir, f), createHash('sha256').update(fs.readFileSync(f)).digest('hex')]));
let state = '', home = '';
beforeEach(() => { state = tmpDir('gstack-advisory-state-'); home = tmpDir('gstack-advisory-home-'); });
afterEach(() => { for (const d of [state, home]) fs.rmSync(d, { recursive: true, force: true }); });
describe('the /plan-ceo-review gate leaves its skill text alone', () => {
test('gen:skill-docs --dry-run with ceo_review_gate on (env and config) finds plan-ceo-review fresh', () => {
fs.writeFileSync(path.join(state, 'config.yaml'), 'ceo_review_gate: on\n');
const r = spawnSync(process.execPath, [...bunIsolationArgs(), path.join(ROOT, 'scripts', 'gen-skill-docs.ts'), '--dry-run'], {
cwd: ROOT, env: { ...process.env, GSTACK_STATE_ROOT: state, GSTACK_CEO_REVIEW_GATE: 'on' }, encoding: 'utf8', timeout: 120_000,
});
expect(r.stdout).toContain('FRESH: plan-ceo-review/SKILL.md');
expect(r.stdout).not.toMatch(/STALE: plan-ceo-review\//);
});
test('an armed CEO run that blocks a stop writes nothing under the installed skills dir', () => {
const skills = path.join(home, '.claude', 'skills');
fs.mkdirSync(path.join(skills, 'plan-ceo-review'), { recursive: true });
fs.copyFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), path.join(skills, 'plan-ceo-review', 'SKILL.md'));
const before = digest(skills);
const transcript = path.join(home, 't.jsonl');
fs.writeFileSync(transcript, `${[user('review my plan'), tool('s1', 'Skill', { skill: 'plan-ceo-review' }), result('s1', 'Launching skill: plan-ceo-review'), say('Review complete.')].join('\n')}\n`);
const r = runHook(path.join(ROOT, 'hosts', 'claude', 'hooks', 'ceo-review-gate.ts'),
JSON.stringify({ session_id: 'sess-adv', transcript_path: transcript, cwd: home, hook_event_name: 'Stop', stop_hook_active: false }),
hookEnv(state, skills, { HOME: home, GSTACK_CEO_REVIEW_GATE: 'on', GSTACK_CEO_REVIEW_OUTPUT: '' }));
expect(JSON.parse(r.stdout).decision).toBe('block');
expect(digest(skills)).toEqual(before);
});
});