diff --git a/.gstack/pregate.json b/.gstack/pregate.json index 04502a58f..2fc8cc546 100644 --- a/.gstack/pregate.json +++ b/.gstack/pregate.json @@ -15,7 +15,8 @@ "licenses/**": [], "contrib/**": [], "supabase/**": [], - "qm/sandbox/skills/gstack-review/scripts/diff-base.sh": ["test/qm-edition.test.ts"] + "qm/sandbox/skills/gstack-review/scripts/diff-base.sh": ["test/qm-edition.test.ts"], + "lib/shell-writes.ts": ["test/skill-contract.test.ts"] }, "selection": { "free-tests.yml/free-suite": "bun run scripts/test-free-shards.ts --list" diff --git a/changelog.d/remove-test-after-edit-stop-hook-check.md b/changelog.d/remove-test-after-edit-stop-hook-check.md new file mode 100644 index 000000000..8fa40aaa7 --- /dev/null +++ b/changelog.d/remove-test-after-edit-stop-hook-check.md @@ -0,0 +1 @@ +- **The optional Stop hook no longer checks for a test run after the last edit, and /verify asks for it in plain words instead.** With /verify loaded, Opus 5.5 and GPT-6.1 Sol already ran a test after their last edit in 96% and 89% of runs, and the hook's check added between -0.5 and +2 points, inside the noise on both models. The hook's other checks (report labels, specialist dispatches, the adversarial pass) are unchanged. If you wrote your own skill whose contract lists `check_after_last_edit`, that contract is now invalid. The hook then checks none of that skill's items and prints one line naming the file until you delete the item. /verify now says to run the full check after your last edit. It also no longer says it never edits CHANGELOG, because a changelog entry the repo expects with a change is part of the change; /ship still folds branch entries into the release entry. diff --git a/docs/skill-routing-and-contracts.md b/docs/skill-routing-and-contracts.md index 5308919f6..c85997cc9 100644 --- a/docs/skill-routing-and-contracts.md +++ b/docs/skill-routing-and-contracts.md @@ -73,10 +73,11 @@ Every item has an `id` (lowercase letters, digits, `.`, `_`, `-`) and a `kind`, | kind | Fields | Met when | |---|---|---| | `report_labels` | `heading`, `labels[]` | The last assistant message containing the heading has a `Label: value` line for every label, and no value is empty or a placeholder (`[...]`, `<...>`, TBD, TODO, `...`, or a template like `DONE \| BLOCKED`) | -| `check_after_last_edit` | optional `match` (regular expression) | No edit happened, or a Bash test or repro command ran after the last one. An edit is an Edit, Write, MultiEdit or NotebookEdit call, or a Bash command that writes a file inside the session directory (the transcript's first `cwd`): a `>`, `>>` or `&>` redirect, `tee`, `sed -i` or `perl -i`, `patch` or `git apply`, or an inline or heredoc python, node, ruby, bun or deno script that opens a literal path for writing (directly or through a variable assigned one, as in `python3 - <<'EOF'`), including a script the same command wrote first. A check later in the same Bash command counts (`sed -i ... && bun test`), and so does the command whose output the write captures (`bun test > out.log`, `bun test \| tee test.log`). Reads, output to stdout, fd duplications (`2>&1`), `/dev` paths, targets outside the session directory (`/tmp`, `~`, `../`) and targets named by a variable (`"$f"`) are not edits; `lib/shell-writes.ts` holds the parser (GST-1633). By default a command counts when, with quoted text removed, one of its parts (split on `&&`, `\|\|`, `;`, `\|`) runs a test runner (pytest, jest, vitest, rspec, tsc, ...), a test or check verb (`npm test`, `go test`, `cargo run`, `make check`, `bun run lint`), code through an interpreter (`node repro.js`, `python -c ...`, `bun scripts/repro.ts`), `curl` against a URL, or a script whose name says test, spec, repro, check, verify or lint, directly or behind a wrapper (`bundle exec rspec`, `uv run pytest`, `docker compose run app pytest`, `xvfb-run npm test`, `bash scripts/test.sh`). Installs, builds, cleans and version probes (`bun install`, `npm install`, `go build`, `make clean`, `node --version`), `git commit --no-verify`, `cat test/a.test.ts` and the shell's `test -f` do not count | | `selected_specialists_dispatched` | optional `marker`, `empty_marker` | The selection line was printed in an assistant message or a Bash result (never a fetched page or file), its names are plain (letters, digits, spaces, `_`, `.`, `-`, at most 40 characters), and each specialist it names has an Agent dispatch that mentions it. The marker is literal text in which a whole-word N stands for a number; the default is `Dispatching N specialists:`. N = 0 (`Dispatching 0 specialists: (small diff)`) or a "specialists skipped" line means zero specialists were selected, which passes. Text in parentheses is not a name (`Dispatching 1 specialists: State Machines (small diff)` selects State Machines) | | `agent_dispatch` | `match` (regular expression, case-insensitive) | An Agent dispatch's description or prompt matches `match`. A `marker` is rejected until the checker can bind it to the dispatch it names (GST-1242) | +There is no item for "run a test after your last edit". /verify asks for that in its text, and Opus 5.5 and GPT-6.1 Sol already did it in 96% and 89% of runs; a Stop-hook item for it added no measurable gain (gstack-evals#356), so it was removed. + A run the skill ended early counts as met: a final `STATUS: BLOCKED` (or `Status: BLOCKED` in the skill's report) or a final message with a line that starts "Nothing to review" (/review's early exit). The checker's default marker strings live in one table, `DEFAULT_MATCHERS` in `lib/skill-contract.ts`; a skill's own `marker`, `empty_marker` or `match` overrides them. @@ -92,14 +93,12 @@ Items are checked against one run of the skill: from its last load to the end of kind: report_labels heading: DEBUG REPORT labels: [Symptom, Root cause, Fix, Evidence] - - id: repro-after-fix - kind: check_after_last_edit -A skill you wrote, `bugfix`, declares that contract. A session loads it, edits `src/auth.ts` and answers "Fixed: the null check was missing." with no report and no test run. The Stop hook blocks once with: +A skill you wrote, `bugfix`, declares that contract. A session loads it, edits `src/auth.ts` and answers "Fixed: the null check was missing." with no report. The Stop hook blocks once with: - SKILL_CONTRACT_UNMET bugfix (contract v1): missing debug-report, repro-after-fix. Fix: debug-report: print the DEBUG REPORT with real values for Symptom, Root cause, Fix, Evidence; repro-after-fix: run the test or repro again after your last edit and show its result. If an item truly does not apply, run '/bin/gstack-skill-contract' waive --reason "" (this session only; counted as a miss for follow). This blocks once. + SKILL_CONTRACT_UNMET bugfix (contract v1): missing debug-report. Fix: debug-report: print the DEBUG REPORT with real values for Symptom, Root cause, Fix, Evidence. If an item truly does not apply, run '/bin/gstack-skill-contract' waive --reason "" (this session only; counted as a miss for follow). This blocks once. -The model runs `bun test`, prints the DEBUG REPORT, and stops. The record reads `met_after_block`. +The model prints the DEBUG REPORT and stops. The record reads `met_after_block`. ### When the Stop hook checks @@ -109,7 +108,7 @@ The hook checks the most recently loaded skill that declares a contract, and onl - the session is headless or unattended (`gstack-session-kind`), or - the turn does not end on a question to the user (a last paragraph ending in "?", an AskUserQuestion call, or `NEEDS_CONTEXT`). -So an interactive session that pauses to ask a question is never blocked. When Claude Code says a Stop hook already continued this turn (`stop_hook_active`), the hook never blocks: after its own block it records the result (`met_after_block` or `miss`). On a final turn it also allows the stop while Agent dispatches are still running (no result yet, or launched in the background with no completion notice naming that agent), because their outputs may still arrive, and records the run as `deferred`. Once the same agents have been running for 10 minutes since the stop that first saw them, they count as stalled and the contract is checked at the next stop. Because Claude Code can fire Stop before the turn's last lines reach the transcript file (about 200 ms late in a smoke run), the hook re-reads the file until it holds the payload's `last_assistant_message`, for at most 2 seconds (`readSettledTranscript` in `lib/skill-gates.ts`, the loop the skill gates share). If the message never shows up, it counts that message as the turn's final text. A single read could miss the test rerun after the last edit and block a finished run, or miss the skill load and let an unfinished one through. +So an interactive session that pauses to ask a question is never blocked. When Claude Code says a Stop hook already continued this turn (`stop_hook_active`), the hook never blocks: after its own block it records the result (`met_after_block` or `miss`). On a final turn it also allows the stop while Agent dispatches are still running (no result yet, or launched in the background with no completion notice naming that agent), because their outputs may still arrive, and records the run as `deferred`. Once the same agents have been running for 10 minutes since the stop that first saw them, they count as stalled and the contract is checked at the next stop. Because Claude Code can fire Stop before the turn's last lines reach the transcript file (about 200 ms late in a smoke run), the hook re-reads the file until it holds the payload's `last_assistant_message`, for at most 2 seconds (`readSettledTranscript` in `lib/skill-gates.ts`, the loop the skill gates share). If the message never shows up, it counts that message as the turn's final text. A single read could miss the final report and block a finished run, or miss the skill load and let an unfinished one through. It blocks at most once per skill run. If the model stops again with the contract still unmet, the stop is allowed and the run is recorded as a miss. diff --git a/lib/shell-writes.ts b/lib/shell-writes.ts deleted file mode 100644 index 4502f77db..000000000 --- a/lib/shell-writes.ts +++ /dev/null @@ -1,341 +0,0 @@ -/** - * shell-writes — the repo files a Bash command writes, for the skill contract's - * check_after_last_edit item (GST-1633). A fix written through the shell (a - * heredoc into python or cat, sed -i, tee, a > or >> redirect, patch, git apply) - * is an edit just like an Edit call, so a test run before it does not count. - * - * Conservative: a write counts only when its target resolves inside the session - * directory. Reads, output to stdout, fd duplications (2>&1), /dev paths, targets - * under another directory (/tmp, ~, ../) and targets named by a variable never - * count. patch, git apply and xargs sed -i with no visible target count when they - * run inside the session directory, since they only write the tree they run in. - */ -import * as path from 'path'; - -const posix = path.posix; -/** Stands in for the session directory when the transcript does not name one. */ -const UNKNOWN_DIR = '/\0session'; - -type Tok = { kind: 'word'; raw: string; value: string } | { kind: 'op'; value: string } | { kind: 'redir'; op: string } | { kind: 'heredoc'; body: string }; -export interface ShellCommand { words: string[]; text: string; redirects: { op: string; target: string }[]; heredocs: string[]; cwd: string | null; pipeline: number } - -/** The index just past the bracket that closes the one at src[from] (quotes are not tracked). */ -function closing(src: string, from: number, open: string, close: string): number { - let depth = 0; - for (let j = from; j < src.length; j++) { - if (src[j] === open) depth++; - else if (src[j] === close && --depth === 0) return j + 1; - } - return src.length; -} - -function tokenize(src: string): Tok[] { - const toks: Tok[] = []; - const pending: { delim: string; strip: boolean; at: number }[] = []; - let raw = ''; - let value = ''; - let inWord = false; - let i = 0; - const flush = () => { if (inWord) toks.push({ kind: 'word', raw, value }); raw = ''; value = ''; inWord = false; }; - const take = (end: number, v = src.slice(i, end)) => { raw += src.slice(i, end); value += v; inWord = true; i = end; }; - const bodies = () => { - const due = pending.splice(0); - due.forEach((h, n) => { - const lines: string[] = []; - while (i < src.length) { - const nl = src.indexOf('\n', i); - const end = nl < 0 ? src.length : nl; - const line = src.slice(i, end); - i = end + 1; - if ((h.strip ? line.replace(/^\t+/, '') : line) === h.delim) break; - lines.push(line); - } - toks.splice(h.at, 0, { kind: 'heredoc', body: lines.join('\n') }); - for (const later of due.slice(n + 1)) later.at++; - }); - }; - while (i < src.length) { - const c = src[i]; - const rest = src.slice(i); - if (c === '\\') { - if (src[i + 1] === '\n') i += 2; - else take(Math.min(i + 2, src.length), src[i + 1] ?? ''); - } else if (c === "'") { - const end = src.indexOf("'", i + 1); - take(end < 0 ? src.length : end + 1, src.slice(i + 1, end < 0 ? src.length : end)); - } else if (c === '"') { - let j = i + 1; - let v = ''; - for (; j < src.length && src[j] !== '"'; j++) { - if (src[j] === '\\' && '"\\$`'.includes(src[j + 1] ?? '')) j++; - v += src[j]; - } - take(Math.min(j + 1, src.length), v); - } else if (rest.startsWith('$(') || rest.startsWith('${')) { - take(closing(src, i + 1, src[i + 1], src[i + 1] === '(' ? ')' : '}')); - } else if (c === '`') { - const end = src.indexOf('`', i + 1); - take(end < 0 ? src.length : end + 1); - } else if (rest.startsWith('((') && !inWord) { - take(closing(src, i, '(', ')')); - flush(); - } else if (c === '#' && !inWord) { - while (i < src.length && src[i] !== '\n') i++; - } else if (c === '\n') { - flush(); - toks.push({ kind: 'op', value: '\n' }); - i++; - bodies(); - } else if (c === ' ' || c === '\t') { - flush(); - i++; - } else { - const redir = /^(?:&>>|&>|\d*>>|\d*>&|\d*>\||\d*>|<<<|<<-|<<|\d*<&|\d*<>|\d*<)/.exec(rest); - const op = /^(?:\|\||&&|\|&|;;|[;|&()])/.exec(rest); - if (redir && !(inWord && /^\d/.test(redir[0]))) { - flush(); - i += redir[0].length; - if (redir[0] === '<<' || redir[0] === '<<-') { - while (src[i] === ' ' || src[i] === '\t') i++; - const m = /^(['"])(.*?)\1|^[^\s;&|<>()]+/.exec(src.slice(i)); - pending.push({ delim: m ? (m[2] ?? m[0]).replace(/\\/g, '') : '', strip: redir[0] === '<<-', at: toks.length }); - i += m ? m[0].length : 0; - continue; - } - const dup = /[<>]&$/.test(redir[0]) ? /^\s*(?:\d+|-)/.exec(src.slice(i)) : null; - if (dup) i += dup[0].length; - else toks.push({ kind: 'redir', op: redir[0].replace(/^(\d*)>&$/, '&>') }); - } else if (op) { - flush(); - toks.push({ kind: 'op', value: op[0] }); - i += op[0].length; - } else { - take(i + 1); - } - } - } - flush(); - bodies(); - return toks; -} - -const ASSIGN = /^[A-Za-z_][A-Za-z0-9_]*=/; -const RESERVED = new Set(['!', 'if', 'then', 'else', 'elif', 'fi', 'do', 'done', 'while', 'until', 'time', '{', '}']); -/** Words that run the command after them, with the options of each that take a value. */ -const PREFIXES: Record> = { - env: new Set(['-u', '--unset', '-C', '--chdir', '-S']), sudo: new Set(['-u', '-g', '-C', '-D', '-h', '-p', '-U']), command: new Set(), exec: new Set(['-a']), - nice: new Set(['-n']), nohup: new Set(), time: new Set(), timeout: new Set(['-s', '-k', '--signal', '--kill-after']), xargs: new Set(['-I', '-n', '-P', '-L', '-d', '-E', '-s', '-a']), -}; -const SHELLS = /^(?:bash|sh|zsh|dash)$/; -const base = (word: string) => word.split('/').pop() ?? ''; - -/** The words that run: assignments, reserved words and prefix commands (with their options) removed. */ -function program(words: string[]): string[] { - let w = words; - for (let guard = 0; guard < 8 && w.length; guard++) { - while (w.length && (ASSIGN.test(w[0]) || RESERVED.has(w[0]))) w = w.slice(1); - const valued = PREFIXES[base(w[0] ?? '')]; - if (!valued) break; - const name = base(w[0]); - let k = 1; - while (k < w.length && (w[k].startsWith('-') || ASSIGN.test(w[k]))) k += valued.has(w[k]) ? 2 : 1; - if (name === 'timeout' && k < w.length) k++; - w = w.slice(k); - } - return w; -} - -/** A cd target resolved against the current directory; null when it cannot be known (cd, cd ~, cd -, cd $DIR). */ -function changeDir(here: string | null, target: string | undefined): string | null { - if (!target || /^[~$-]|\$|`/.test(target)) return null; - if (posix.isAbsolute(target)) return posix.normalize(target); - return here === null ? null : posix.resolve(here, target); -} - -/** The simple commands of a Bash line in order, with heredoc bodies attached, cd tracked and bash -c unwrapped. */ -export function shellCommands(line: string, cwd: string | null, depth = 0): ShellCommand[] { - const out: ShellCommand[] = []; - const scopes: (string | null)[] = []; - let here = cwd; - let pipeline = 0; - let cur = { words: [] as string[], raws: [] as string[], redirects: [] as ShellCommand['redirects'], heredocs: [] as string[] }; - const end = () => { - const words = program(cur.words); - const redirects = words[0] === '[[' ? [] : cur.redirects; - const cmd: ShellCommand = { words, text: cur.raws.join(' '), redirects, heredocs: cur.heredocs, cwd: here, pipeline }; - cur = { words: [], raws: [], redirects: [], heredocs: [] }; - if (!cmd.words.length && !redirects.length && !cmd.heredocs.length) return; - const shell = SHELLS.test(base(words[0] ?? '')) && words[1] === '-c' && typeof words[2] === 'string' && depth < 3; - if (shell) { - out.push(...shellCommands(words[2], here, depth + 1).map(c => ({ ...c, pipeline }))); - if (redirects.length) out.push({ ...cmd, words: [], text: '' }); - return; - } - out.push(cmd); - if (words[0] === 'cd' || words[0] === 'pushd') here = changeDir(here, words.slice(1).find(w => !w.startsWith('-')) ?? '~'); - else if (words[0] === 'popd') here = null; - }; - const toks = tokenize(line); - for (let k = 0; k < toks.length; k++) { - const t = toks[k]; - if (t.kind === 'op') { - end(); - if (t.value !== '|' && t.value !== '|&') pipeline++; - if (t.value === '(') scopes.push(here); - else if (t.value === ')' && scopes.length) here = scopes.pop() ?? null; - } else if (t.kind === 'heredoc') cur.heredocs.push(t.body); - else if (t.kind === 'redir') { - const next = toks[k + 1]; - if (next?.kind === 'word') { cur.redirects.push({ op: t.op, target: next.value }); cur.raws.push(`${t.op}${next.raw}`); k++; } - } else { - cur.words.push(t.value); - cur.raws.push(t.raw); - } - } - end(); - return out; -} - -/** A path relative to the session directory when it resolves inside it, else null. '' is the directory itself. */ -function inside(p: string, cwd: string | null, root: string): string | null { - if (!p || /^[~$&]|\$|`|^\/dev\/|^[A-Za-z]:[\\/]|^\\/.test(p)) return null; - const abs = posix.isAbsolute(p) ? posix.normalize(p) : cwd === null ? null : posix.resolve(cwd, p); - if (abs === null) return null; - if (abs === root) return ''; - return abs.startsWith(`${root.replace(/\/+$/, '')}/`) ? abs.slice(root.replace(/\/+$/, '').length + 1) : null; -} - -const WRITE_REDIRECT = /^(?:\d*>>?|\d*>\||&>>?)$/; -const INTERPRETER = /^(?:python[\d.]*|node|nodejs|ruby|bun|deno)$/; -const CODE_FLAG = /^(?:-c|-e|--eval|eval)$/; -const VALUE_FLAG = /^(?:-m|-W|-X|-r|--require|-I)$/; -const PATH_EXPR = String.raw`((?:pathlib\.)?Path\(\s*[rbuRBU]{0,2}(?:'[^'\n]*'|"[^"\n]*")\s*\)|[rbuRBU]{0,2}(?:'[^'\n]*'|"[^"\n]*")|[A-Za-z_$][\w$]*)`; -const MODE = String.raw`\s*(?:mode\s*=\s*)?[rbuRBU]?['"]([^'"]*)['"]`; -/** Calls that write a file named by their first path argument; the second group, when present, is an open mode. */ -const WRITE_CALLS = [ - new RegExp(String.raw`\bopen\(\s*${PATH_EXPR}\s*,${MODE}`, 'g'), - new RegExp(String.raw`${PATH_EXPR}\s*\.\s*open\(${MODE}`, 'g'), - new RegExp(String.raw`${PATH_EXPR}\s*\.\s*write_(?:text|bytes)\s*\(`, 'g'), - new RegExp(String.raw`\b(?:writeFileSync|writeFile|appendFileSync|appendFile|Bun\.write|File\.write|File\.binwrite|Deno\.writeTextFile(?:Sync)?|Deno\.writeFile(?:Sync)?)\s*\(\s*${PATH_EXPR}\s*,`, 'g'), -]; -const ASSIGNMENT = new RegExp(String.raw`(?:^|[\s;(])(?:const\s+|let\s+|var\s+)?([A-Za-z_$][\w$]*)\s*=\s*${PATH_EXPR}\s*(?:;|$)`, 'gm'); - -/** The string literals a path expression can name: a literal, Path(literal), or a variable assigned one of those. */ -function literals(expr: string, assigned: Map): string[] { - const m = /^(?:(?:pathlib\.)?Path\(\s*)?[rbuRBU]{0,2}(['"])(.*)\1\s*\)?$/.exec(expr.trim()); - if (m) return m[2].includes('{') ? [] : [m[2]]; - return assigned.get(expr.trim()) ?? []; -} - -/** The files a script's code writes, by literal path or a variable holding one. */ -function codeWrites(code: string): string[] { - const assigned = new Map(); - for (const m of code.matchAll(ASSIGNMENT)) { - const lits = literals(m[2], new Map()); - if (lits.length) assigned.set(m[1], [...(assigned.get(m[1]) ?? []), ...lits]); - } - return WRITE_CALLS.flatMap(re => [...code.matchAll(re)].filter(m => m[2] === undefined || /[wax+]/.test(m[2])).flatMap(m => literals(m[1], assigned))); -} - -/** The code an interpreter command runs, when the line shows it: an inline -c/-e script, its stdin, or a script this line wrote. */ -function interpreterCode(c: ShellCommand, scripts: Map): string[] { - const [, ...args] = c.words; - let script: string | undefined; - for (let k = 0; k < args.length; k++) { - if (CODE_FLAG.test(args[k])) return [args[k + 1] ?? '']; - if (VALUE_FLAG.test(args[k])) { if (args[k] === '-m') return []; k++; continue; } - if (!args[k].startsWith('-') || args[k] === '-') { script = args[k]; break; } - } - const file = (p: string) => scripts.get(c.cwd === null && !posix.isAbsolute(p) ? '' : posix.resolve(c.cwd ?? '/', p)) ?? ''; - if (script !== undefined && script !== '-') return [file(script)]; - return [...c.heredocs, ...c.redirects.filter(r => r.op === '<<<').map(r => r.target), ...c.redirects.filter(r => r.op === '<').map(r => file(r.target))]; -} - -/** The +++ paths of a unified diff, with git's a/ b/ prefixes removed. */ -const diffTargets = (diff: string) => [...diff.matchAll(/^\+\+\+ (\S+)/gm)].map(m => m[1].replace(/^[ab]\//, '')).filter(p => p !== '/dev/null'); - -/** - * The repo files one simple command writes, or null when it writes none. self is false when every write is the - * command's own output (a redirect or tee), so the command itself can still be the check that follows the write. - */ -export function writeTargets(c: ShellCommand, root: string, scripts: Map = new Map()): { files: string[]; self: boolean } | null { - const files: string[] = []; - const add = (p: string, cwd = c.cwd) => { const f = inside(p, cwd, root); if (f !== null) files.push(f || '.'); }; - for (const r of c.redirects) if (WRITE_REDIRECT.test(r.op)) add(r.target); - const [first = '', ...args] = c.words; - const name = base(first); - const positional = args.filter(a => !a.startsWith('-')); - if (name === 'tee') positional.forEach(f => add(f)); - const output = files.length; - if (name === 'sed' || name === 'gsed' || name === 'perl') { - let inPlace = false; - let script = false; - const rest: string[] = []; - for (let k = 0; k < args.length; k++) { - const a = args[k]; - if (/^(?:-e|--expression|-f|--file)$/.test(a)) { script = true; k++; } - else if (a === '-i' && args[k + 1] === '') { inPlace = true; k++; } - else if (a.startsWith('--in-place')) inPlace = true; - else if (/^-[A-Za-z0-9]/.test(a)) { - const letters = a.slice(1); - const flags = letters.slice(0, letters.search(name === 'perl' ? /[CdDFImMx]|$/ : /[ef]|$/)); - const at = flags.indexOf('i'); - if (at >= 0 && (name !== 'perl' || !flags.slice(0, at).includes('e'))) inPlace = true; - if (name === 'perl' && letters.endsWith('e')) { script = true; k++; } - } else rest.push(a); - } - const targets = (script ? rest : rest.slice(1)).filter(f => f !== '{}'); - if (inPlace) targets.length ? targets.forEach(f => add(f)) : add('.'); - } else if (name === 'git') { - let k = 0; - let cwd = c.cwd; - for (; k < args.length && args[k].startsWith('-'); k++) { - if (args[k] === '-C') cwd = changeDir(cwd, args[++k]); - else if (args[k] === '-c') k++; - } - const sub = args.slice(k + 1); - const readOnly = sub.includes('--cached') || (sub.some(a => /^--(?:check|stat|numstat|summary)$/.test(a)) && !sub.includes('--apply')); - if (args[k] === 'apply' && !readOnly) { - const targets = c.heredocs.flatMap(diffTargets); - targets.length ? targets.forEach(f => add(f, cwd)) : add('.', cwd); - } - } else if (name === 'patch' && !args.some(a => a === '--dry-run' || a === '-C' || a === '--check')) { - let cwd = c.cwd; - const named: string[] = []; - for (let k = 0; k < args.length; k++) { - if (args[k] === '-d' || args[k] === '--directory') cwd = changeDir(cwd, args[++k]); - else if (args[k] === '-o' || args[k] === '--output') named.push(args[++k] ?? ''); - else if (/^(?:-i|--input|-B|--prefix|-r|--reject-file|-z|--suffix|-D|--ifdef|-F|--fuzz|-V)$/.test(args[k])) k++; - else if (!args[k].startsWith('-')) { named.push(args[k]); break; } - } - const targets = [...named, ...c.heredocs.flatMap(diffTargets)]; - targets.length ? targets.forEach(f => add(f, cwd)) : add('.', cwd); - } else if (INTERPRETER.test(name)) { - interpreterCode(c, scripts).flatMap(codeWrites).forEach(f => add(f)); - } - return files.length ? { files: [...new Set(files)], self: files.length > output } : null; -} - -/** - * The last repo write in a Bash line: the files it writes and the text of the commands that ran after it, which - * includes the writer's own pipeline when the write is only its output (`bun test | tee test.log`). Null when the - * line writes no file inside the session directory. cwd is where the line ran, root the session directory; both - * null when the transcript does not say, and then only relative targets count. - */ -export function lastBashWrite(line: string, cwd: string | null, root: string | null): { files: string[]; after: string } | null { - const top = root !== null && posix.isAbsolute(root) ? posix.normalize(root) : UNKNOWN_DIR; - const here = cwd !== null && posix.isAbsolute(cwd) ? posix.normalize(cwd) : top; - const cmds = shellCommands(line, here); - const scripts = new Map(); - let last: { at: number; files: string[]; self: boolean } | null = null; - for (const [at, c] of cmds.entries()) { - const w = writeTargets(c, top, scripts); - if (w) last = { at, ...w }; - const body = c.heredocs.join('\n'); - for (const r of c.redirects) if (WRITE_REDIRECT.test(r.op) && c.cwd !== null) scripts.set(posix.resolve(c.cwd, r.target), body); - } - if (!last) return null; - const { at, files, self } = last; - const start = self ? at + 1 : cmds.findIndex(c => c.pipeline === cmds[at].pipeline); - return { files, after: cmds.slice(start).map(c => c.text).join('\n') }; -} diff --git a/lib/skill-contract.ts b/lib/skill-contract.ts index d10da8dae..cdcab0fe2 100644 --- a/lib/skill-contract.ts +++ b/lib/skill-contract.ts @@ -18,12 +18,11 @@ import * as fs from 'fs'; import * as path from 'path'; import { atomicWriteSync } from './fs-atomic'; -import { lastBashWrite } from './shell-writes'; import { readFrontmatter, scanInstalledSkills } from './skill-route'; import { resolveStateRoot } from './state-root'; export const TRANSCRIPT_MAX_BYTES = 32 * 1024 * 1024; -export const ITEM_KINDS = ['report_labels', 'check_after_last_edit', 'selected_specialists_dispatched', 'agent_dispatch'] as const; +export const ITEM_KINDS = ['report_labels', 'selected_specialists_dispatched', 'agent_dispatch'] as const; export type ItemKind = typeof ITEM_KINDS[number]; const ITEM_ID = /^[a-z0-9][a-z0-9_.-]{0,63}$/; const MAX_ITEMS = 20; @@ -150,7 +149,6 @@ export function itemSummary(item: ContractItem): string { if (item.summary) return item.summary; switch (item.kind) { case 'report_labels': return `a ${item.heading} with ${(item.labels ?? []).join(', ')}`; - case 'check_after_last_edit': return 'a test or repro run after your last edit'; case 'selected_specialists_dispatched': return 'one Agent dispatch per selected specialist'; case 'agent_dispatch': return `an Agent dispatch matching /${item.match}/`; } @@ -158,10 +156,9 @@ export function itemSummary(item: ContractItem): string { export type TranscriptEvent = | { kind: 'prompt' | 'meta' | 'text'; line: number; text: string } - | { kind: 'tool_use'; line: number; id: string; name: string; input: Record; cwd?: string } + | { kind: 'tool_use'; line: number; id: string; name: string; input: Record } | { kind: 'tool_result'; line: number; id: string; text: string; isError: boolean }; -/** cwd is the session directory: the first record's working directory, when the transcript names one. */ -export interface Transcript { events: TranscriptEvent[]; sessionId: string | null; cwd?: string | null } +export interface Transcript { events: TranscriptEvent[]; sessionId: string | null } const META_PREFIX = /^\s*<(?:local-command-stdout|local-command-stderr|system-reminder|task-notification|user-prompt-submit-hook|bash-input|bash-stdout|bash-stderr)\b/; @@ -179,7 +176,6 @@ function blockText(content: unknown): string { export function parseTranscript(text: string, opts: { includeSidechain?: boolean } = {}): Transcript { const events: TranscriptEvent[] = []; let sessionId: string | null = null; - let cwd: string | null = null; let lines = 0; let objects = 0; let messages = 0; @@ -198,14 +194,13 @@ export function parseTranscript(text: string, opts: { includeSidechain?: boolean messages++; if (rec.isSidechain === true && !opts.includeSidechain) continue; if (typeof rec.sessionId === 'string') sessionId = rec.sessionId; - if (cwd === null && typeof rec.cwd === 'string') cwd = rec.cwd; const blocks = typeof content === 'string' ? [{ type: 'text', text: content }] : content; for (const block of blocks) { if (!isObject(block)) continue; if (rec.type === 'assistant') { if (block.type === 'text' && typeof block.text === 'string') events.push({ kind: 'text', line, text: block.text }); if (block.type === 'tool_use' && typeof block.name === 'string') { - events.push({ kind: 'tool_use', line, id: String(block.id ?? ''), name: block.name, input: isObject(block.input) ? block.input : {}, ...(typeof rec.cwd === 'string' ? { cwd: rec.cwd } : {}) }); + events.push({ kind: 'tool_use', line, id: String(block.id ?? ''), name: block.name, input: isObject(block.input) ? block.input : {} }); } } else if (block.type === 'tool_result') { events.push({ kind: 'tool_result', line, id: String(block.tool_use_id ?? ''), text: blockText(block.content), isError: block.is_error === true }); @@ -216,7 +211,7 @@ export function parseTranscript(text: string, opts: { includeSidechain?: boolean } if (lines > 0 && objects === 0) throw new UnknownShape('no line is a JSON object'); if (objects > 0 && messages === 0) throw new UnknownShape('no user or assistant message records'); - return { events, sessionId, cwd }; + return { events, sessionId }; } /** @@ -271,69 +266,8 @@ export function skillLoads(t: Transcript): SkillLoad[] { } export const STATUS_RE = /^[ \t>*_-]*(?:\*\*|__)?(?:STATUS|Status)(?:\*\*|__)?[ \t]*(?:\*\*|__)?:[ \t]*(?:\*\*|__)?[ \t]*(DONE_WITH_CONCERNS|DONE|BLOCKED|NEEDS_CONTEXT)\b/m; -const EDIT_TOOLS = /^(?:Edit|Write|MultiEdit|NotebookEdit)$/; const AGENT_TOOLS = /^(?:Agent|Task)$/; -const TEST_RUNNER = /^(?:pytest|py\.test|jest|vitest|mocha|rspec|phpunit|ctest|tox|nox|tsc|rake|playwright|cypress)$/i; -const CHECK_SCRIPT = /(?:^|[-_.:])(?:tests?|spec|repro|check|typecheck|verify|lint)(?:[-_.:]|$)/i; -const CHECK_VERB = /^(?:tests?|t|check|verify|spec|vet|clippy|nextest|lint|e2e)$/i; -const SUBCOMMAND_TOOL = /^(?:make|cargo|go|dotnet|swift|mix|bazel|just|dart|flutter|rails|mvn|gradlew?|xcodebuild|bun|npm|pnpm|yarn|deno)$/i; -const RUNS_PROGRAM = /^(?:cargo|go|dotnet|swift)$/i; -const PACKAGE_EXEC = /^(?:npx|bunx|pnpx)$/i; -const INTERPRETER = /^(?:node|python3?|ruby|php|perl|tsx|ts-node)$/i; -const FILE_INTERPRETER = /^(?:bun|deno)$/i; -const COMMAND_WRAPPER = /^(?:bundle|uv|poetry|pipenv|docker|podman|xvfb-run|bash|sh|zsh)$/; -const COMMAND_PREFIX = /^(?:env|sudo|time|nice|timeout|command|exec|\w+=\S*|-\S*|\d+[smh]?)$/; -const SCRIPT_ARG = /[\\/]|\.(?:[cm]?[jt]sx?|py|rb|php|pl|sh)$/; -/** Options whose next word is their value (`make -C web test`, `pnpm --filter app test`, `xcodebuild -scheme App test`). */ -const VALUE_FLAG = /^(?:-[CfpwF]|--(?:prefix|filter|cwd|dir|directory|workspace|package|manifest-path|project|file)|-(?:scheme|project|workspace|destination|configuration|target|sdk))$/; -const WRAPPER_DEPTH = 3; -const baseName = (word: string) => word.split(/[\\/]/).pop() ?? ''; -const isCheckScript = (word: string) => { const name = baseName(word); return name !== 'test' && CHECK_SCRIPT.test(name); }; - -/** The words after the program that are not options or option values. */ -function positionals(words: string[]): string[] { - const out: string[] = []; - for (let i = 1; i < words.length; i++) { - if (words[i] === '--') break; - if (!words[i].startsWith('-')) out.push(words[i]); - else if (VALUE_FLAG.test(words[i])) i++; - } - return out; -} - -/** - * One command's words (prefixes removed): does it run tests, a check, or a repro? Installs, builds and version probes - * do not. Wrappers nest at most WRAPPER_DEPTH deep, so a long chain of wrapper words costs linear time, not exponential. - */ -function runsCheck(words: string[], depth = 0): boolean { - const program = baseName(words[0] ?? ''); - const args = positionals(words); - if (TEST_RUNNER.test(program) || isCheckScript(words[0] ?? '')) return true; - if (depth < WRAPPER_DEPTH && PACKAGE_EXEC.test(program)) return runsCheck(words.slice(1).filter(w => !w.startsWith('-')), depth + 1); - if (COMMAND_WRAPPER.test(program)) return depth < WRAPPER_DEPTH && [1, 2, 3, 4, 5].some(i => i < words.length && runsCheck(words.slice(i), depth + 1)); - if (INTERPRETER.test(program)) return args.length > 0 || words.slice(1).some(w => /^-[ecm]$|^--test$/.test(w)); - if (program === 'curl') return args.some(a => /^https?:\/\/|^localhost|^127\./.test(a)); - if (FILE_INTERPRETER.test(program) && args[0] && SCRIPT_ARG.test(args[0])) return true; - if (!SUBCOMMAND_TOOL.test(program) || !args[0]) return false; - const [verb, next] = args; - if (CHECK_VERB.test(verb) || isCheckScript(verb) || args.some(a => CHECK_VERB.test(a))) return true; - if (verb === 'run' || verb === 'exec' || verb === 'x') return RUNS_PROGRAM.test(program) || (!!next && (CHECK_VERB.test(next) || isCheckScript(next) || TEST_RUNNER.test(baseName(next)) || SCRIPT_ARG.test(next))); - return /^(?:yarn|pnpm)$/i.test(program) && TEST_RUNNER.test(verb); -} const SAFE_SPECIALIST = /^[A-Za-z0-9][A-Za-z0-9 _.-]{0,39}$/; - -/** - * A test or repro run: some segment of the command (split on && || ; | and newlines) runs a test runner, a test verb - * (`npm test`, `go test`, `make check`), a test, check or repro script, or code through an interpreter, directly or - * behind a wrapper (bundle exec, uv run, docker compose run, xvfb-run, bash). Installs, builds and version probes do not count. - */ -export function isCheckCommand(cmd: string): boolean { - return cmd.split(/&&|\|\||[;|\n]/).some(segment => { - const words = segment.trim().replace(/^[({]\s*/, '').split(/\s+/); - while (words.length > 1 && COMMAND_PREFIX.test(words[0])) words.shift(); - return runsCheck(words); - }); -} const WAIVE_RE = /gstack-skill-contract["']?\s+waive\s+["']?([A-Za-z0-9][A-Za-z0-9_.-]*)["']?/; type ToolUse = Extract; @@ -341,7 +275,6 @@ type ToolResult = Extract; const isToolUse = (e: TranscriptEvent): e is ToolUse => e.kind === 'tool_use'; const bashCommand = (e: TranscriptEvent) => (isToolUse(e) && e.name === 'Bash' ? String(e.input.command ?? '') : null); const agentText = (e: ToolUse) => [e.input.description, e.input.prompt, e.input.subagent_type, e.input.name].filter(v => typeof v === 'string').join('\n'); -const cut = (s: string, n = 80) => (s.length > n ? `${s.slice(0, n - 3)}...` : s).replace(/\s+/g, ' '); export function labelValue(text: string, label: string): string | null { const re = new RegExp(`^[ \\t>*_|-]*(?:\\*\\*|__)?${escapeRe(label)}[ \\t]*(?:\\*\\*|__)?[ \\t]*:[ \\t]*(?:\\*\\*|__)?(.*)$`, 'im'); @@ -363,7 +296,7 @@ function parseNames(raw: string): string[] { interface ItemCheck { met: boolean; detail: string; fix: string } -function checkItem(item: ContractItem, run: TranscriptEvent[], root: string | null): ItemCheck { +function checkItem(item: ContractItem, run: TranscriptEvent[]): ItemCheck { const texts = run.filter(e => e.kind === 'text').map(e => (e as { text: string }).text); const agents = run.filter((e): e is ToolUse => isToolUse(e) && AGENT_TOOLS.test(e.name)); switch (item.kind) { @@ -379,28 +312,6 @@ function checkItem(item: ContractItem, run: TranscriptEvent[], root: string | nu ? { met: false, detail: `${heading} lacks a real value for ${bad.join(', ')}`, fix: item.fix ?? `fill ${bad.join(', ')} in the ${heading} with real values` } : { met: true, detail: `${heading} has all ${labels.length} labels`, fix: '' }; } - case 'check_after_last_edit': { - let lastEdit = -1; - let edit = ''; - let sameCall = ''; - run.forEach((e, i) => { - if (!isToolUse(e)) return; - const cmd = bashCommand(e); - const write = cmd === null ? null : lastBashWrite(cmd, e.cwd ?? root, root); - if (write) [lastEdit, edit, sameCall] = [i, `Bash write (${cut(write.files.join(', '), 60)})`, write.after]; - else if (EDIT_TOOLS.test(e.name)) [lastEdit, edit, sameCall] = [i, e.name, '']; - }); - if (lastEdit < 0) return { met: true, detail: 'no edits since the skill loaded', fix: '' }; - const re = item.match ? new RegExp(item.match, 'i') : null; - const check = [sameCall || null, ...run.slice(lastEdit + 1).map(bashCommand)].find(cmd => { - if (!cmd || /gstack-skill-contract/.test(cmd)) return false; - const bare = cmd.replace(/"(?:[^"\\]|\\.)*"|'[^']*'/g, '""'); - return re ? re.test(bare) : isCheckCommand(bare); - }); - return check - ? { met: true, detail: `ran "${cut(check)}" after the last edit`, fix: '' } - : { met: false, detail: `no test or repro command after the last ${edit}`, fix: item.fix ?? 'run the test or repro again after your last edit and show its result' }; - } case 'selected_specialists_dispatched': { const marker = markerPattern(item.marker ?? DEFAULT_MATCHERS.specialistSelection); const empty = markerPattern(item.empty_marker ?? DEFAULT_MATCHERS.specialistsSkipped); @@ -485,7 +396,7 @@ export function checkContract(t: Transcript, skill: string, contract: Contract, const earlyExit = !load ? null : statusLine === 'BLOCKED' ? 'STATUS: BLOCKED' : endedEarly ? DEFAULT_MATCHERS.earlyExit : null; const items: ItemResult[] = load ? contract.items.map(item => { if (earlyExit) return { id: item.id, kind: item.kind, status: 'met', detail: `the skill ended the run early (${earlyExit})`, fix: '' }; - const r = checkItem(item, run, t.cwd ?? null); + const r = checkItem(item, run); if (r.met) return { id: item.id, kind: item.kind, status: 'met', detail: r.detail, fix: '' }; const reason = waived.get(item.id); return reason ? { id: item.id, kind: item.kind, status: 'waived', detail: r.detail, fix: r.fix, reason } : { id: item.id, kind: item.kind, status: 'missing', detail: r.detail, fix: r.fix }; diff --git a/scripts/resolvers/contract-done-means.ts b/scripts/resolvers/contract-done-means.ts index d4d07a6c7..4e406367c 100644 --- a/scripts/resolvers/contract-done-means.ts +++ b/scripts/resolvers/contract-done-means.ts @@ -16,7 +16,6 @@ const code = (s: string) => '\u0060' + s + '\u0060'; export type ContractItem = | { id: string; kind: 'report_labels'; heading: string; labels: string[] } - | { id: string; kind: 'check_after_last_edit' } | { id: string; kind: 'selected_specialists_dispatched'; marker: string } | { id: string; kind: 'agent_dispatch'; match: string }; @@ -27,7 +26,6 @@ export interface SkillContract { const KIND_FIELDS: Record = { report_labels: ['heading', 'labels'], - check_after_last_edit: [], selected_specialists_dispatched: ['marker'], agent_dispatch: ['match'], }; @@ -107,7 +105,6 @@ const list = (xs: string[]) => xs.length < 2 ? xs.join('') : xs.slice(0, -1).joi function plainWords(item: ContractItem): string { switch (item.kind) { case 'report_labels': return 'a ' + code(item.heading) + ' with ' + list(item.labels) + ' filled in (' + code('none: ') + ' where the skill\'s own stop rule leaves one empty)'; - case 'check_after_last_edit': return 'a test or repro command run after your last edit'; case 'selected_specialists_dispatched': return 'the specialist selection printed (' + code(item.marker) + ' ...), then one Agent dispatch per specialist it names (none when a small diff skips them)'; case 'agent_dispatch': return 'an Agent dispatch for the ' + item.id.replace(/-/g, ' ') + ', its prompt including "' + item.match + '"'; } diff --git a/test/fixtures/skill-contract/block-message.txt b/test/fixtures/skill-contract/block-message.txt index 8b9187077..d651c9dbb 100644 --- a/test/fixtures/skill-contract/block-message.txt +++ b/test/fixtures/skill-contract/block-message.txt @@ -1 +1 @@ -SKILL_CONTRACT_UNMET verify (contract v1): missing debug-report, repro-after-fix. Fix: debug-report: print the DEBUG REPORT with real values for Symptom, Root cause, Fix, Evidence; repro-after-fix: run the test or repro again after your last edit and show its result. If an item truly does not apply, run '{bin}' waive --reason "" (this session only; counted as a miss for follow). This blocks once. +SKILL_CONTRACT_UNMET verify (contract v1): missing debug-report. Fix: debug-report: print the DEBUG REPORT with real values for Symptom, Root cause, Fix, Evidence. If an item truly does not apply, run '{bin}' waive --reason "" (this session only; counted as a miss for follow). This blocks once. diff --git a/test/helpers/skill-contract-fixture.ts b/test/helpers/skill-contract-fixture.ts index a0e714f5d..982eff18b 100644 --- a/test/helpers/skill-contract-fixture.ts +++ b/test/helpers/skill-contract-fixture.ts @@ -29,8 +29,6 @@ contract: kind: report_labels heading: DEBUG REPORT labels: [Symptom, Root cause, Fix, Evidence] - - id: repro-after-fix - kind: check_after_last_edit --- # /verify (fixture) diff --git a/test/skill-contract-stop-hook.test.ts b/test/skill-contract-stop-hook.test.ts index 320af1bcd..adffa02a7 100644 --- a/test/skill-contract-stop-hook.test.ts +++ b/test/skill-contract-stop-hook.test.ts @@ -55,7 +55,7 @@ describe('block once, with the pinned text', () => { expect(out.reason).toBe(PINNED); expect(out.reason).not.toMatch(/skill_contracts|gstack-config|\boff\b/); const run = Object.values(record().runs)[0] as any; - expect(run).toMatchObject({ skill: 'verify', outcome: 'blocked', missing: ['debug-report', 'repro-after-fix'] }); + expect(run).toMatchObject({ skill: 'verify', outcome: 'blocked', missing: ['debug-report'] }); expect(fs.statSync(path.join(state, 'skill-contracts', 'sess-1.json')).mode & 0o777).toBe(0o600); }); @@ -65,7 +65,7 @@ describe('block once, with the pinned text', () => { const named = /run '([^']+)' waive /.exec(reason)?.[1]; expect(named).toBe(BIN); const { spawnSync } = require('node:child_process') as typeof import('node:child_process'); - const waive = spawnSync(named!, ['waive', 'repro-after-fix', '--reason', 'docs-only change'], { env: hookEnv(state, skills), encoding: 'utf8', timeout: 20_000 }); + const waive = spawnSync(named!, ['waive', 'debug-report', '--reason', 'docs-only change'], { env: hookEnv(state, skills), encoding: 'utf8', timeout: 20_000 }); expect(waive.status).toBe(0); }); @@ -151,11 +151,11 @@ describe('block once, with the pinned text', () => { test('a waiver read from the transcript satisfies the block but counts against follow', () => { const lines = [...J1_TRANSCRIPT.slice(0, -1), - tool('w1', 'Bash', { command: 'gstack-skill-contract waive repro-after-fix --reason "no test runner in this repo"' }), - result('w1', 'SKILL_CONTRACT_WAIVED repro-after-fix: no test runner in this repo'), - say(DEBUG_REPORT)]; + tool('w1', 'Bash', { command: 'gstack-skill-contract waive debug-report --reason "a docs-only change, no bug to report"' }), + result('w1', 'SKILL_CONTRACT_WAIVED debug-report: a docs-only change, no bug to report'), + say('Updated the README.')]; expect(stop(transcript(lines)).stdout).toBe(''); - expect((Object.values(record().runs)[0] as any)).toMatchObject({ outcome: 'met_waived', waived: ['repro-after-fix'] }); + expect((Object.values(record().runs)[0] as any)).toMatchObject({ outcome: 'met_waived', waived: ['debug-report'] }); }); }); diff --git a/test/skill-contract.test.ts b/test/skill-contract.test.ts index ab67d10d9..7090e5902 100644 --- a/test/skill-contract.test.ts +++ b/test/skill-contract.test.ts @@ -8,11 +8,10 @@ import { spawnSync } from 'node:child_process'; import * as fs from 'node:fs'; import * as path from 'node:path'; import { - DEFAULT_MATCHERS, EXIT, STATUS_RE, blockMessage, checkContract, isCheckCommand, isMiss, isPlaceholder, parseContract, parseTranscript, reportRows, skillLoads, UnknownShape, + DEFAULT_MATCHERS, EXIT, STATUS_RE, blockMessage, checkContract, isMiss, isPlaceholder, parseContract, parseTranscript, reportRows, skillLoads, UnknownShape, type CheckResult, type Contract, } from '../lib/skill-contract'; -import { lastBashWrite } from '../lib/shell-writes'; import { readFrontmatter } from '../lib/skill-route'; import { CONTRACT_HOOK, DEBUG_REPORT, INTERACTIVE_TURNS, J1_TRANSCRIPT, ROOT, SKILLS, hookEnv, meta, result, runHook, say, skillsDir, slash, tmpDir, tool, user, @@ -29,7 +28,7 @@ const check = (lines: string[], skill = 'verify') => checkContract(parseTranscri describe('schema v1', () => { test('block and flow style parse to the same items', () => { - expect(contractOf('verify').items.map(i => [i.id, i.kind])).toEqual([['debug-report', 'report_labels'], ['repro-after-fix', 'check_after_last_edit']]); + expect(contractOf('verify').items.map(i => [i.id, i.kind])).toEqual([['debug-report', 'report_labels']]); expect(contractOf('review').items.map(i => i.kind)).toEqual(['selected_specialists_dispatched', 'agent_dispatch', 'report_labels']); }); @@ -43,7 +42,8 @@ describe('schema v1', () => { ['contract:\n version: 1\n items: [{id: a, kind: agent_dispatch, match: "("}]', 'not a valid regular expression'], ['contract:\n version: 1\n items: [{id: a, kind: agent_dispatch}]', 'agent_dispatch needs match'], ['contract:\n version: 1\n items: [{id: a, kind: report_labels, heading: X, labels: []}]', 'needs labels'], - ['contract:\n version: 1\n items: [{id: Bad_ID, kind: check_after_last_edit}]', 'id must match'], + ['contract:\n version: 1\n items: [{id: Bad_ID, kind: agent_dispatch, match: x}]', 'id must match'], + ['contract:\n version: 1\n items: [{id: a, kind: check_after_last_edit}]', 'unknown kind'], ['contract: [unclosed', 'not valid YAML'], ]; for (const [fm, msg] of bad) { @@ -73,10 +73,10 @@ describe('transcript parsing', () => { }); describe('checkContract is a pure function of the transcript', () => { - test('J1 transcript: both items missing, and the block text is the pinned one', () => { + test('J1 transcript: the report is missing, and the block text is the pinned one', () => { const r = check(J1_TRANSCRIPT); expect(r.loaded).toBe(true); - expect(r.missing).toEqual(['debug-report', 'repro-after-fix']); + expect(r.missing).toEqual(['debug-report']); expect(blockMessage(r, '/opt/gstack/bin/gstack-skill-contract')).toBe(fs.readFileSync(path.join(ROOT, 'test', 'fixtures', 'skill-contract', 'block-message.txt'), 'utf8').trimEnd().replace('{bin}', '/opt/gstack/bin/gstack-skill-contract')); expect(check(J1_TRANSCRIPT)).toEqual(r); }); @@ -89,17 +89,6 @@ describe('checkContract is a pure function of the transcript', () => { for (const v of ['N/A, no test runner', 'none found', 'src/a.ts:3']) expect(isPlaceholder(v)).toBe(false); }); - test('check_after_last_edit: only a test or repro command after the LAST edit counts; quoted words do not', () => { - const edited = J1_TRANSCRIPT.slice(0, -1); - const runAfter = (command: string) => check([...edited, tool('b', 'Bash', { command }), result('b', 'ok')]).items[1].status; - expect(runAfter('bun test test/auth.test.ts')).toBe('met'); - expect(runAfter('git commit -m "add test"')).toBe('missing'); - expect(runAfter('cat src/auth.ts')).toBe('missing'); - const before = check([...J1_TRANSCRIPT.slice(0, 4), tool('b', 'Bash', { command: 'bun test' }), ...J1_TRANSCRIPT.slice(4)]); - expect(before.items[1].status).toBe('missing'); - expect(check([slash('verify'), say('No code change was needed.')]).items[1]).toMatchObject({ status: 'met', detail: 'no edits since the skill loaded' }); - }); - test('selected_specialists_dispatched (lane B marker), agent_dispatch and the /review report', () => { const sel = say('Dispatching 2 specialists: Testing, Data Migration. Skipped: Security (scope not detected).'); const agent = (id: string, description: string, prompt = 'p') => [tool(id, 'Agent', { description, prompt, subagent_type: 'general-purpose' }), result(id, '[]')]; @@ -142,20 +131,19 @@ describe('checkContract is a pure function of the transcript', () => { const printed = spawnSync('bash', ['-c', `printf %s ${cmd}`], { cwd: tmpDir('gstack-contract-quote-'), encoding: 'utf8', timeout: 10_000 }); expect(printed.stdout).toBe(bin); expect(fs.existsSync(path.join(printed.stdout, '..', 'pwned'))).toBe(false); - const base = [...J1_TRANSCRIPT.slice(0, -1), say(DEBUG_REPORT)]; - const waived = check([...base, tool('w', 'Bash', { command: `${cmd} waive repro-after-fix --reason "docs-only change"` }), result('w', 'ack')]); - expect(waived).toMatchObject({ met: true, waived: ['repro-after-fix'] }); + const waived = check([...J1_TRANSCRIPT, tool('w', 'Bash', { command: `${cmd} waive debug-report --reason "no bug, a docs-only change"` }), result('w', 'ack')]); + expect(waived).toMatchObject({ met: true, waived: ['debug-report'] }); }); test('waivers come from waive Bash calls in the transcript; an errored waive or another skill\'s waive does not count', () => { const waive = (cmd: string, isError = false) => [tool('w', 'Bash', { command: cmd }), result('w', 'ack', isError)]; - const base = [...J1_TRANSCRIPT.slice(0, -1), say(DEBUG_REPORT)]; - const ok = check([...base, ...waive('~/.claude/skills/gstack/bin/gstack-skill-contract waive repro-after-fix --reason "docs-only change"')]); - expect(ok).toMatchObject({ met: true, follow: false, waived: ['repro-after-fix'] }); - expect(ok.items[1].reason).toBe('docs-only change'); - expect(check([...base, ...waive('gstack-skill-contract waive repro-after-fix --reason x', true)]).met).toBe(false); - expect(check([...base, ...waive('gstack-skill-contract waive repro-after-fix --reason x --skill review')]).met).toBe(false); - expect(check([...base, ...waive('gstack-skill-contract waive repro-after-fix')]).met).toBe(false); + const base = J1_TRANSCRIPT; + const ok = check([...base, ...waive('~/.claude/skills/gstack/bin/gstack-skill-contract waive debug-report --reason "docs-only change"')]); + expect(ok).toMatchObject({ met: true, follow: false, waived: ['debug-report'] }); + expect(ok.items[0].reason).toBe('docs-only change'); + expect(check([...base, ...waive('gstack-skill-contract waive debug-report --reason x', true)]).met).toBe(false); + expect(check([...base, ...waive('gstack-skill-contract waive debug-report --reason x --skill review')]).met).toBe(false); + expect(check([...base, ...waive('gstack-skill-contract waive debug-report')]).met).toBe(false); }); test('status line, pending question and running agents', () => { @@ -172,7 +160,7 @@ describe('checkContract is a pure function of the transcript', () => { test('items are checked from the skill\'s LAST load: an earlier run\'s report does not satisfy a new one', () => { const r = check([...INTERACTIVE_TURNS.flat(), user('another bug'), tool('s', 'Skill', { skill: 'verify' }), tool('e', 'Edit', { file_path: 'x' })]); - expect(r.missing).toEqual(['debug-report', 'repro-after-fix']); + expect(r.missing).toEqual(['debug-report']); }); test('an early exit counts as met: /review "Nothing to review" or STATUS: BLOCKED, /verify Status: BLOCKED in its DEBUG REPORT', () => { @@ -183,7 +171,7 @@ describe('checkContract is a pure function of the transcript', () => { expect(early([slash('review'), say('Base branch could not be found.\nSTATUS: BLOCKED')], 'review').met).toBe(true); const blockedReport = DEBUG_REPORT.replace('Status: DONE', 'Status: BLOCKED'); expect(early([...J1_TRANSCRIPT.slice(0, -1), say(blockedReport)], 'verify')).toMatchObject({ met: true, statusLine: 'BLOCKED' }); - expect(early([...J1_TRANSCRIPT.slice(0, -1), say(DEBUG_REPORT)], 'verify').missing).toEqual(['repro-after-fix']); + expect(early([...J1_TRANSCRIPT.slice(0, -1), say(DEBUG_REPORT)], 'verify').missing).toEqual([]); expect(early([slash('review'), say('I would print STATUS: BLOCKED if the base were missing, but it is here. Done.')], 'review').met).toBe(false); expect(early([slash('review'), say('Nothing to review under docs/; checking lib/ next.'), say('Done.')], 'review').met).toBe(false); expect(early([...J1_TRANSCRIPT.slice(0, -1), say('There is nothing to review in the docs folder. Fixed it.')], 'verify').met).toBe(false); @@ -196,95 +184,6 @@ describe('checkContract is a pure function of the transcript', () => { expect(loads('/home/u/src/gstack/review/SKILL.md')).toEqual([]); }); - test('check_after_last_edit: a command counts by the program it runs, not by a word inside it', () => { - for (const cmd of ['bun test', 'cd repo && npm test', 'FOO=1 timeout 60 pytest -x', './scripts/run-tests.sh', 'node repro.js | tail -5', 'bin/gstack-verify', - 'bundle exec rspec', 'uv run pytest', 'poetry run pytest', 'docker compose run app pytest', 'xvfb-run -a npm test', 'bash scripts/test.sh', 'mix test', 'bazel test //...']) - expect([cmd, isCheckCommand(cmd)]).toEqual([cmd, true]); - for (const cmd of ['npm test', 'npm run test:unit', 'yarn jest', 'go test ./...', 'cargo test', 'make check', 'python -m pytest', 'bun repro.ts', 'npx vitest run', 'curl -s http://localhost:3000/api']) - expect([cmd, isCheckCommand(cmd)]).toEqual([cmd, true]); - for (const cmd of ['git commit -m wip --no-verify', 'git add test/a.test.ts', 'cat test/a.test.ts', 'git diff --check', 'ls go', 'test -f README.md', './scripts/fetch-latest.sh', 'bash scripts/deploy.sh', - 'bun install', 'npm install lodash', 'bun add zod', 'go build ./...', 'go mod tidy', 'node --version', 'make clean', 'make', 'npm run build', 'curl --version']) - expect([cmd, isCheckCommand(cmd)]).toEqual([cmd, false]); - }); - - test('check_after_last_edit: a Bash write to a repo file is an edit, so a fix written through the shell needs a check after it (GST-1633)', () => { - const loaded = J1_TRANSCRIPT.slice(0, 4); - const bash = (id: string, command: string) => [tool(id, 'Bash', { command }), result(id, 'ok')]; - const after = (...commands: string[]) => check([...loaded, ...bash('t0', 'bun test'), ...commands.flatMap((c, i) => bash(`b${i}`, c))]).items[1]; - const writes = [ - "python3 - <<'EOF'\np = 'src/auth.ts'\ns = open(p).read()\nopen(p, 'w').write(s.replace('a', 'b'))\nEOF", - "sed -i 's/session.user/session?.user/' src/auth.ts", - "echo 'export const x = 1' | tee src/auth.ts", - "printf 'fixed\\n' >> src/auth.ts", - "cat > src/auth.ts <<'EOF'\nexport function login() {}\nEOF", - "git apply <<'EOF'\n--- a/src/auth.ts\n+++ b/src/auth.ts\n@@ -1 +1 @@\n-a\n+b\nEOF", - 'patch -p1 < fix.diff', - "cd src && perl -pi -e 's/a/b/' auth.ts", - 'python3 -c "from pathlib import Path; Path(\'src/auth.ts\').write_text(\'x\')"', - 'node -e "require(\'fs\').writeFileSync(\'/repo/src/auth.ts\', \'x\')"', - "cat > /tmp/fix.py <<'EOF'\nwith open('src/auth.ts', 'w') as f:\n f.write('x')\nEOF\npython3 /tmp/fix.py", - 'grep -rl session.user src | xargs sed -i s/session.user/session?.user/', - ]; - for (const w of writes) { - expect([w, after(w).status]).toEqual([w, 'missing']); - expect([w, after(w, 'bun test test/auth.test.ts').status]).toEqual([w, 'met']); - expect([w, after(`${w}\nbun test`).status]).toEqual([w, 'met']); - expect([w, after(`bun test\n${w}`).status]).toEqual([w, 'missing']); - } - expect(after(writes[1]).detail).toBe('no test or repro command after the last Bash write (src/auth.ts)'); - expect(after(writes[0], 'git diff src/auth.ts').status).toBe('missing'); - }); - - test('check_after_last_edit: reads, stdout, writes outside the repo and fd redirects are not edits (GST-1633)', () => { - const edited = [...J1_TRANSCRIPT.slice(0, -1), tool('t0', 'Bash', { command: 'bun test' }), result('t0', 'ok')]; - const after = (command: string) => check([...edited, tool('b', 'Bash', { command }), result('b', 'ok')]).items[1].status; - const controls = [ - 'cat src/auth.ts', 'grep -n user src/auth.ts', 'sed -n 1,20p src/auth.ts', 'echo done', 'echo "a > src/auth.ts"', "grep '>' src/auth.ts", - "python3 - <<'EOF'\nprint(open('src/auth.ts').read())\nEOF", 'python3 -c "open(\'/tmp/x\', \'w\').write(\'y\')"', - 'echo x > /tmp/out.txt', "cat > /tmp/fix.py <<'EOF'\nx = 1\nEOF", 'cd /tmp && echo x > notes.txt', 'echo x | tee /tmp/log', 'sed -i s/a/b/ /tmp/a.txt', - 'echo x > ../elsewhere/a.txt', 'echo x > /home/u/notes.txt', 'echo x > ~/notes.txt', 'echo x > "$OUT"', - 'bun test 2>&1 | tail -5', 'ls >/dev/null 2>&1', 'git status 2>&1', '[[ 2 > 1 ]] && echo ok', '(( 2 > 1 )) && echo ok', - 'git apply --check fix.diff', 'patch --dry-run -p1 < fix.diff', "cat <<'EOF'\necho x > src/auth.ts\nEOF", - ]; - for (const c of controls) expect([c, after(c)]).toEqual([c, 'met']); - expect(after('bun test > test-output.log')).toBe('met'); - expect(after('bun test 2>&1 | tee test-output.log')).toBe('met'); - const noCwd = (lines: string[]) => lines.map(l => { const o = JSON.parse(l); delete o.cwd; return JSON.stringify(o); }); - const bare = (command: string) => check(noCwd([...edited, tool('b', 'Bash', { command }), result('b', 'ok')])).items[1].status; - expect(bare('sed -i s/a/b/ src/auth.ts')).toBe('missing'); - expect(bare('echo x > /repo/src/auth.ts')).toBe('met'); - expect(bare('cd /tmp && sed -i s/a/b/ a.txt')).toBe('met'); - }); - - test('lastBashWrite: the files a Bash line writes and the commands that ran after the last one (GST-1633)', () => { - const w = (line: string, cwd = '/repo') => lastBashWrite(line, cwd, '/repo'); - expect(w('bash -c "sed -i s/a/b/ src/a.py && bun test"')).toEqual({ files: ['src/a.py'], after: 'bun test' }); - expect(w('(cd /tmp && echo x > a) && echo y > b.txt')).toEqual({ files: ['b.txt'], after: 'echo y >b.txt' }); - expect(w("sed -i '' 's/a/b/' src/a.py")).toEqual({ files: ['src/a.py'], after: '' }); - expect(w('echo x1>src/a.py')?.files).toEqual(['src/a.py']); - expect(w("cat < x.txt; cat < /tmp/y\na\nA\nb\nB\npytest")).toEqual({ files: ['x.txt'], after: 'cat >x.txt\ncat >/tmp/y\npytest' }); - expect(w("python3 - <<'PY'\nimport pathlib\np = pathlib.Path('src/a.py')\np.write_text('x')\nPY")?.files).toEqual(['src/a.py']); - expect(w('git apply fix.diff')?.files).toEqual(['.']); - expect(w('git -C /tmp/other apply fix.diff')).toBeNull(); - expect(w('sed -i s/a/b/ a.py', '/repo/src')?.files).toEqual(['src/a.py']); - expect(w("for f in src/*.py; do sed -i 's/a/b/' \"$f\"; done")).toBeNull(); - expect(w("node -e \"fs.writeFileSync(path.join('src', 'a'), 'x')\"")).toBeNull(); - for (const read of ['perl -Ilib t/basic.t', "perl -MList::Util=sum -le 'print sum(1, 2)'", 'sed -fscript.sed src/a.py', "sed -es/i/j/ src/a.py"]) expect([read, w(read)]).toEqual([read, null]); - expect(w("perl -Ilib -pi -e 's/a/b/' src/a.pm")?.files).toEqual(['src/a.pm']); - expect(w("perl -i.bak -pe 's/a/b/' src/a.pm")?.files).toEqual(['src/a.pm']); - }); - - test('check_after_last_edit: an option value is never the verb, and a long chain of wrapper words stays fast', () => { - for (const cmd of ['make -C web test', 'npm --prefix web test', 'pnpm --filter app test', 'bun --cwd app test', 'yarn workspace app test', - 'xcodebuild -scheme App test', 'cargo +nightly test', 'mvn clean test', 'bun run typecheck', 'node --test']) - expect([cmd, isCheckCommand(cmd)]).toEqual([cmd, true]); - for (const cmd of ['make -C web build', 'npm --prefix web install', 'pnpm --filter app build', 'xcodebuild -scheme App build']) - expect([cmd, isCheckCommand(cmd)]).toEqual([cmd, false]); - const started = performance.now(); - expect(isCheckCommand(`${Array(40).fill('sh').join(' ')} echo`)).toBe(false); - expect(performance.now() - started).toBeLessThan(500); - }); - test('report re-checks a deferred run: unmet is a miss, met is met', () => { const run = { skill: 'review', load_line: 1, outcome: 'deferred' as const, missing: [], waived: [], deferred_agents: ['a1'], updated_at: '' }; const rec = { version: 1 as const, session_id: 's', transcript_path: 't', runs: { k: run } }; @@ -341,7 +240,7 @@ describe('gstack-skill-contract CLI', () => { expect(met.stdout).toStartWith('SKILL_CONTRACT_MET verify (contract v1)'); const unmet = cli('check', '--transcript', write(J1_TRANSCRIPT), '--skill', 'verify', '--json'); expect(unmet.code).toBe(EXIT.UNMET); - expect(JSON.parse(unmet.stdout)).toMatchObject({ exit: 1, skill: 'verify', missing: ['debug-report', 'repro-after-fix'], follow: false }); + expect(JSON.parse(unmet.stdout)).toMatchObject({ exit: 1, skill: 'verify', missing: ['debug-report'], follow: false }); }); test('check: stable error exits 2, 3, 4, 5, 6', () => { @@ -357,10 +256,10 @@ describe('gstack-skill-contract CLI', () => { }); test('waive is stateless: validates and acknowledges, writes nothing', () => { - const ok = cli('waive', 'repro-after-fix', '--reason', 'docs-only change', '--skill', 'verify'); - expect(ok).toEqual({ code: 0, stdout: 'SKILL_CONTRACT_WAIVED repro-after-fix (verify): docs-only change (this session only; counted as a miss for follow)\n', stderr: '' }); + const ok = cli('waive', 'debug-report', '--reason', 'docs-only change', '--skill', 'verify'); + expect(ok).toEqual({ code: 0, stdout: 'SKILL_CONTRACT_WAIVED debug-report (verify): docs-only change (this session only; counted as a miss for follow)\n', stderr: '' }); expect(fs.readdirSync(state)).toEqual([]); - expect(cli('waive', 'repro-after-fix').code).toBe(EXIT.USAGE); + expect(cli('waive', 'debug-report').code).toBe(EXIT.USAGE); expect(cli('waive', 'Not An Id', '--reason', 'x').code).toBe(EXIT.USAGE); expect(cli('waive', 'nope', '--reason', 'x', '--skill', 'verify').code).toBe(EXIT.USAGE); }); @@ -377,7 +276,7 @@ describe('gstack-skill-contract CLI', () => { const rep = JSON.parse(r.stdout); expect(rep.runs).toBe(2); expect(rep.misses.map((m: any) => [m.session, m.outcome])).toEqual([['s-missed', 'miss']]); - expect(rep.items).toEqual([{ skill: 'verify', item: 'debug-report', misses: 1 }, { skill: 'verify', item: 'repro-after-fix', misses: 1 }]); + expect(rep.items).toEqual([{ skill: 'verify', item: 'debug-report', misses: 1 }]); expect(cli('report').stdout).toStartWith('SKILL_CONTRACT_REPORT runs=2 misses=1'); }); diff --git a/test/skill-descriptions.test.ts b/test/skill-descriptions.test.ts index ddac7e8fc..4ca62e9c5 100644 --- a/test/skill-descriptions.test.ts +++ b/test/skill-descriptions.test.ts @@ -149,7 +149,6 @@ describe('contract-done-means renderer', () => { version: 1, items: [ { id: 'debug-report', kind: 'report_labels', heading: 'DEBUG REPORT', labels: ['Symptom', 'Root cause', 'Status'] }, - { id: 'check-after-last-edit', kind: 'check_after_last_edit' }, { id: 'specialists-dispatched', kind: 'selected_specialists_dispatched', marker: 'Dispatching N specialists:' }, { id: 'adversarial-pass', kind: 'agent_dispatch', match: 'chaos engineer' }, ], @@ -159,7 +158,6 @@ describe('contract-done-means renderer', () => { test('names each item kind in plain words', () => { expect(out).toContain('## Done means'); expect(out).toContain('- ' + BT + 'debug-report' + BT + ': a ' + BT + 'DEBUG REPORT' + BT + ' with Symptom, Root cause and Status filled in (' + BT + 'none: ' + BT + " where the skill's own stop rule leaves one empty)."); - expect(out).toContain('- ' + BT + 'check-after-last-edit' + BT + ': a test or repro command run after your last edit.'); expect(out).toContain('- ' + BT + 'specialists-dispatched' + BT + ': the specialist selection printed (' + BT + 'Dispatching N specialists:' + BT + ' ...), then one Agent dispatch per specialist it names (none when a small diff skips them).'); expect(out).toContain('- ' + BT + 'adversarial-pass' + BT + ': an Agent dispatch for the adversarial pass, its prompt including "chaos engineer".'); }); @@ -174,18 +172,19 @@ describe('contract-done-means renderer', () => { const one = (item: Record) => ({ version: 1, items: [item] }); const MALFORMED: Array<[string, unknown, RegExp]> = [ ['not a mapping', 'yes', /must be a mapping/], - ['wrong version', { version: 2, items: [{ id: 'a', kind: 'check_after_last_edit' }] }, /version must be 1/], + ['wrong version', { version: 2, items: [{ id: 'a', kind: 'agent_dispatch', match: 'x' }] }, /version must be 1/], ['no items', { version: 1, items: [] }, /items must be a non-empty list/], - ['unknown top-level key', { version: 1, items: [{ id: 'a', kind: 'check_after_last_edit' }], extra: 1 }, /unknown key "extra"/], + ['unknown top-level key', { version: 1, items: [{ id: 'a', kind: 'agent_dispatch', match: 'x' }], extra: 1 }, /unknown key "extra"/], ['unknown kind', one({ id: 'a', kind: 'vibes' }), /kind "vibes" is not one of/], - ['bad id', one({ id: 'Debug Report', kind: 'check_after_last_edit' }), /id must be lowercase kebab-case/], - ['duplicate id', { version: 1, items: [{ id: 'a', kind: 'check_after_last_edit' }, { id: 'a', kind: 'agent_dispatch', match: 'x' }] }, /duplicate id "a"/], + ['removed kind', one({ id: 'a', kind: 'check_after_last_edit' }), /kind "check_after_last_edit" is not one of/], + ['bad id', one({ id: 'Debug Report', kind: 'agent_dispatch', match: 'x' }), /id must be lowercase kebab-case/], + ['duplicate id', { version: 1, items: [{ id: 'a', kind: 'agent_dispatch', match: 'y' }, { id: 'a', kind: 'agent_dispatch', match: 'x' }] }, /duplicate id "a"/], ['labels missing', one({ id: 'a', kind: 'report_labels', heading: 'R' }), /labels must be a non-empty list/], ['placeholder label', one({ id: 'a', kind: 'report_labels', heading: 'R', labels: ['[what the user observed]'] }), /placeholder/], ['heading missing', one({ id: 'a', kind: 'report_labels', labels: ['Symptom'] }), /heading must be a non-empty string/], ['marker missing', one({ id: 'a', kind: 'selected_specialists_dispatched' }), /marker must be a non-empty string/], ['match missing', one({ id: 'a', kind: 'agent_dispatch' }), /match must be a non-empty string/], - ['field from another kind', one({ id: 'a', kind: 'check_after_last_edit', match: 'x' }), /unknown key "match"/], + ['field from another kind', one({ id: 'a', kind: 'agent_dispatch', match: 'x', heading: 'R' }), /unknown key "heading"/], ]; for (const [name, value, want] of MALFORMED) { test('a malformed block fails loudly: ' + name, () => { diff --git a/test/skill-route.test.ts b/test/skill-route.test.ts index 9127d8dba..e73cb8223 100644 --- a/test/skill-route.test.ts +++ b/test/skill-route.test.ts @@ -105,7 +105,7 @@ describe('skill-route hook', () => { expect(expectHookSchema(route(prompt('fix this bug'), { GSTACK_SKILL_ROUTING: 'on', GSTACK_SKILL_ROUTE_HINT: 'step' }).stdout)) .toContain('Its first step: Reproduce the failure before you change code'); expect(expectHookSchema(route(prompt('fix this bug'), { GSTACK_SKILL_ROUTING: 'on', GSTACK_SKILL_ROUTE_HINT: 'contract' }).stdout)) - .toContain('Done means: a DEBUG REPORT with Symptom, Root cause, Fix, Evidence; a test or repro run after your last edit.'); + .toContain('Done means: a DEBUG REPORT with Symptom, Root cause, Fix, Evidence.'); }); test('with nothing set SWITCH_DEFAULT decides; off in config, and off in env over config on', () => { diff --git a/verify/SKILL.md b/verify/SKILL.md index ee6e6e266..6180ded2d 100644 --- a/verify/SKILL.md +++ b/verify/SKILL.md @@ -157,7 +157,8 @@ Skills that run plan reviews (`/plan-*-review`, `/codex review`) include the EXI # /verify — build, check, fix, repeat Use this while building, before /ship. It runs checks and fixes code. It never -reviews, bumps a version, edits CHANGELOG, pushes, or opens a PR: that is /ship. +reviews, bumps a version, pushes, or opens a PR: that is /ship. A changelog entry +the repo expects with a change is part of the change. ## 1. See what will run @@ -197,7 +198,7 @@ and a log path. Exit 0 is green; only `PASS` is green. A `FAIL` whose own error says a service, secret or env var is missing is the same: report it, do not stub or skip it. - Rerun one check with `"$C" run --only ` while fixing it, then a full - `run` before you stop. + `run` after your last edit, before you stop. Never weaken, skip, delete or edit a test, a checker config or a package.json script to get green. diff --git a/verify/SKILL.md.tmpl b/verify/SKILL.md.tmpl index 059a412c1..2bd61cbda 100644 --- a/verify/SKILL.md.tmpl +++ b/verify/SKILL.md.tmpl @@ -28,7 +28,8 @@ allowed-tools: # /verify — build, check, fix, repeat Use this while building, before /ship. It runs checks and fixes code. It never -reviews, bumps a version, edits CHANGELOG, pushes, or opens a PR: that is /ship. +reviews, bumps a version, pushes, or opens a PR: that is /ship. A changelog entry +the repo expects with a change is part of the change. ## 1. See what will run @@ -68,7 +69,7 @@ and a log path. Exit 0 is green; only `PASS` is green. A `FAIL` whose own error says a service, secret or env var is missing is the same: report it, do not stub or skip it. - Rerun one check with `"$C" run --only ` while fixing it, then a full - `run` before you stop. + `run` after your last edit, before you stop. Never weaken, skip, delete or edit a test, a checker config or a package.json script to get green.