diff --git a/CHANGELOG.md b/CHANGELOG.md index 17e5ed851..a9593397a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,19 @@ # Changelog +## [1.94.18.0] - 2026-11-03 + +**/investigate, /review, /qa and /verify now say when to use them, and /investigate and /review declare what "done" means (GST-1088 lane B).** Untold, Opus 5.5 never opened /investigate in 252 trials (Run D, gstack-evals#140). A when-to-use description raised GPT-6.1 Sol's open rate from 9.5% to 27.0% in the same run. Claude Code's catalog shows only a description's first sentence, so each of the four now opens with a "Use when ..." sentence. The routing hook and the contract check that read the new frontmatter ship separately (lane A), both off by default. + +### Changed +- **Descriptions**: the first sentence of each is now a when-clause, for example /investigate's "Use when a test fails, code errors or output is wrong: find the root cause, then fix it." Hosts that show the full description also get the longer list of situations. The catalog stays within its existing budget. +- **Triggers**: /investigate, /review and /verify list the prompts people actually type, such as "test fails", "look over my diff" and "make it green". /qa keeps its three triggers because its size cap had 9 bytes of room. + +### Added +- **`contract:` frontmatter** (frozen v1 schema) on /investigate (the DEBUG REPORT labels, and a test or repro run after the last edit) and /review (one Agent dispatch per selected specialist, the adversarial pass, the Pre-Landing Review report). `{{CONTRACT_DONE_MEANS}}` (`scripts/resolvers/contract-done-means.ts`) renders each block as a short "Done means" list at the top of the skill, so the list and the block can't drift. + +### Tests +- `test/skill-descriptions.test.ts`: each of the four opens with a when-clause within the catalog, lead and host caps; triggers are short and don't overlap; ten sample prompts match exactly the intended skill; every template's contract block passes the v1 schema and names markers the skill text actually prints; the renderer covers each item kind; and malformed blocks fail with the file and the fix. + ## [1.94.17.0] - 2026-11-03 - A pregate guard whose exact command already passed at this tree no longer runs again (GST-1186). When the base branch's `ship_lanes_from` maps the command, it is carried from the verified CI run through the same `gstack-evidence record-ci` path /ship lanes use (`GUARD: carried runs=…`); otherwise a counted local ledger pass of that command at the same working tree stands (CI rows count only through a live `record-ci` check) (`GUARD: reused …`, and the new read-only `gstack-evidence passed -- `). Repos whose suites are guards, like founderstar, stop rerunning about six minutes of suites per /ship. See docs/pregate.md#guard-carry. diff --git a/VERSION b/VERSION index 54c4aa075..ce198dd4a 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.94.17.0 +1.94.18.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index be1ec0e3d..c591dfe8a 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.94.17.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.94.18.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/gstack/llms.txt b/gstack/llms.txt index 3dbe42205..ec8a7721d 100644 --- a/gstack/llms.txt +++ b/gstack/llms.txt @@ -37,7 +37,7 @@ Conventions: - [/gstack-upgrade](gstack-upgrade/SKILL.md): Upgrade gstack to the latest version. - [/guard](guard/SKILL.md): Full safety mode: destructive command warnings + directory-scoped edits. - [/health](health/SKILL.md): Code quality dashboard. -- [/investigate](investigate/SKILL.md): Systematic debugging with root cause investigation. +- [/investigate](investigate/SKILL.md): Use when a test fails, code errors or output is wrong: find the root cause, then fix it. - [/ios-clean](ios-clean/SKILL.md): Remove the DebugBridge SPM package and all #if DEBUG wiring from an iOS app. - [/ios-design-review](ios-design-review/SKILL.md): Visual design audit for iOS apps on real hardware. - [/ios-fix](ios-fix/SKILL.md): Autonomous iOS bug fixer. @@ -56,10 +56,10 @@ Conventions: - [/plan-devex-review](plan-devex-review/SKILL.md): Interactive developer experience plan review. - [/plan-eng-review](plan-eng-review/SKILL.md): Eng manager-mode plan review. - [/plan-tune](plan-tune/SKILL.md): Self-tuning question sensitivity + developer psychographic for gstack (v1: observational). -- [/qa](qa/SKILL.md): Test the site, API or CLI and fix the bugs it finds. +- [/qa](qa/SKILL.md): Use when you test the site, API or CLI and fix bugs. - [/qa-only](qa-only/SKILL.md): Report browser/API/CLI/job/worker/webhook bugs. - [/retro](retro/SKILL.md): Weekly engineering retrospective. -- [/review](review/SKILL.md): Review this PR's diff. +- [/review](review/SKILL.md): Use when you review this PR's diff. - [/scrape](scrape/SKILL.md): Pull data from a web page through the Aside browser — your real, already signed-in sessions. - [/setup-browser-cookies](setup-browser-cookies/SKILL.md): Import cookies from your real Chromium browser into the headless browse session. - [/setup-deploy](setup-deploy/SKILL.md): Configure deployment settings for /land-and-deploy. @@ -71,7 +71,7 @@ Conventions: - [/test-audit](test-audit/SKILL.md): Find low-value or duplicate tests and the test-only code they keep alive. - [/ubicloud](ubicloud/SKILL.md): Run a heavy suite, build or eval on an ephemeral 16-vCPU Ubicloud VM: TTL-named VMs, a reaper, quota preflight and a hang watchdog. - [/unfreeze](unfreeze/SKILL.md): Clear the freeze boundary set by /freeze, allowing edits to all directories again. -- [/verify](verify/SKILL.md): Build loop while coding: run the repo's checks and the tests /ship would pick, fix, repeat until green. +- [/verify](verify/SKILL.md): Use when coding: run the checks and tests /ship picks until green. - [/write-skill](write-skill/SKILL.md): Write a new agent skill or improve one without bloat: the edit ladder, levels, drift and size checks, full diets. ## Browse Commands diff --git a/investigate/SKILL.md b/investigate/SKILL.md index cb6a50df2..20f6455a7 100644 --- a/investigate/SKILL.md +++ b/investigate/SKILL.md @@ -2,7 +2,7 @@ name: investigate preamble-tier: 2 version: 1.0.0 -description: Systematic debugging with root cause investigation. (gstack) +description: "Use when a test fails, code errors or output is wrong: find the root cause, then fix it. (gstack)" allowed-tools: - Bash - Read @@ -15,9 +15,29 @@ allowed-tools: triggers: - debug this - fix this bug + - test fails + - failing test - why is this broken - - root cause analysis + - this doesn't work + - build is broken + - stack trace - investigate this error + - root cause analysis +contract: + version: 1 + items: + - id: debug-report + kind: report_labels + heading: DEBUG REPORT + labels: + - Symptom + - Root cause + - Fix + - Evidence + - Regression test + - Status + - id: check-after-last-edit + kind: check_after_last_edit hooks: PreToolUse: - matcher: "Edit" @@ -61,6 +81,13 @@ gbrain: +## Done means + +Finish only when each of these exists (contract v1): + +- `debug-report`: a `DEBUG REPORT` with Symptom, Root cause, Fix, Evidence, Regression test and Status filled in (`none: ` where the skill's own stop rule leaves one empty). +- `check-after-last-edit`: a test or repro command run after your last edit. + ## Shared preamble (load first) This skill follows gstack shared preamble `4e82fa4c839f` (AskUserQuestion format, voice, writing, completeness and completion rules). If your Read of that file, headed by this id, is still in context, do not read it again; otherwise Read `~/.claude/skills/gstack/investigate/sections/preamble.md` in full before the command below. A section or subsection here with a shared heading replaces the shared one. diff --git a/investigate/SKILL.md.tmpl b/investigate/SKILL.md.tmpl index c7469537d..a9f7b2ecb 100644 --- a/investigate/SKILL.md.tmpl +++ b/investigate/SKILL.md.tmpl @@ -3,10 +3,10 @@ name: investigate preamble-tier: 2 version: 1.0.0 description: | - Systematic debugging with root cause investigation. Iron Law: no fixes - without root cause. - Use when asked to "debug this", "fix this bug", "why is this broken", - "investigate this error", or "root cause analysis". (gstack) + Use when a test fails, code errors or output is wrong: find the root cause, then fix it. + Systematic debugging; Iron Law: no fixes without root cause. Fits a failing + build, test, script, service or config, an error message or stack trace, + wrong or missing output, or broken data, before any edit that fixes it. (gstack) allowed-tools: - Bash - Read @@ -19,9 +19,29 @@ allowed-tools: triggers: - debug this - fix this bug + - test fails + - failing test - why is this broken - - root cause analysis + - this doesn't work + - build is broken + - stack trace - investigate this error + - root cause analysis +contract: + version: 1 + items: + - id: debug-report + kind: report_labels + heading: DEBUG REPORT + labels: + - Symptom + - Root cause + - Fix + - Evidence + - Regression test + - Status + - id: check-after-last-edit + kind: check_after_last_edit hooks: PreToolUse: - matcher: "Edit" @@ -63,6 +83,8 @@ gbrain: render_as: "## Recent eureka moments (cross-project)" --- +{{CONTRACT_DONE_MEANS}} + {{PREAMBLE}} # Systematic Debugging diff --git a/package.json b/package.json index 64d469b2b..1ca02c0a8 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.94.17", + "version": "1.94.18", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/qa/SKILL.md b/qa/SKILL.md index 5a348f9c7..5c8b6ab91 100644 --- a/qa/SKILL.md +++ b/qa/SKILL.md @@ -2,7 +2,7 @@ name: qa preamble-tier: 4 version: 2.0.0 -description: Test the site, API or CLI and fix the bugs it finds. (gstack) +description: Use when you test the site, API or CLI and fix bugs. (gstack) allowed-tools: - Bash - Read diff --git a/qa/SKILL.md.tmpl b/qa/SKILL.md.tmpl index 7c411d8c0..7bc6706b1 100644 --- a/qa/SKILL.md.tmpl +++ b/qa/SKILL.md.tmpl @@ -3,9 +3,9 @@ name: qa preamble-tier: 4 version: 2.0.0 description: | - Test the site, API or CLI and fix the bugs it finds. + Use when you test the site, API or CLI and fix bugs. Covers browser/API/CLI/job/worker/webhook bugs and commits verified fixes atomically. - Use when asked to "qa", "QA", "qa test this", "test this site", "find bugs", "find bugs on site", + Fits "qa", "QA", "qa test this", "test this site", "find bugs", "find bugs on site", "test and fix", or "fix what's broken". Proactively suggest when the user says a feature is ready for testing or asks "does this work?". Three tiers: Quick (critical/high only), diff --git a/review/SKILL.md b/review/SKILL.md index 369f3730a..dbd2e97e0 100644 --- a/review/SKILL.md +++ b/review/SKILL.md @@ -2,7 +2,7 @@ name: review preamble-tier: 4 version: 1.0.0 -description: Review this PR's diff. (gstack) +description: Use when you review this PR's diff. (gstack) allowed-tools: - Bash - Read @@ -15,13 +15,37 @@ allowed-tools: - WebSearch triggers: - review this pr + - look over my diff - code review - check my diff + - review my changes - pre-landing review +contract: + version: 1 + items: + - id: specialists-dispatched + kind: selected_specialists_dispatched + marker: "Dispatching N specialists:" + - id: adversarial-pass + kind: agent_dispatch + match: Think like an attacker and a chaos engineer + - id: review-report + kind: report_labels + heading: Pre-Landing Review + labels: + - Gate edits --- +## Done means + +Finish only when each of these exists (contract v1): + +- `specialists-dispatched`: the specialist selection printed (`Dispatching N specialists:` ...), then one Agent dispatch per specialist it names (none when a small diff skips them). +- `adversarial-pass`: an Agent dispatch for the adversarial pass, its prompt including "Think like an attacker and a chaos engineer". +- `review-report`: a `Pre-Landing Review` with Gate edits filled in (`none: ` where the skill's own stop rule leaves one empty). + ## Shared preamble (load first) This skill follows gstack shared preamble `4e82fa4c839f` (AskUserQuestion format, voice, writing, completeness and completion rules). If your Read of that file, headed by this id, is still in context, do not read it again; otherwise Read `~/.claude/skills/gstack/review/sections/preamble.md` in full before the command below. A section or subsection here with a shared heading replaces the shared one. diff --git a/review/SKILL.md.tmpl b/review/SKILL.md.tmpl index 60ca668bd..7ae3e4eef 100644 --- a/review/SKILL.md.tmpl +++ b/review/SKILL.md.tmpl @@ -3,10 +3,11 @@ name: review preamble-tier: 4 version: 1.0.0 description: | - Review this PR's diff. Checks it against the base branch for SQL safety, LLM trust - boundary violations, conditional side effects, and other structural issues. Use when - asked to "review this PR", "code review", "pre-landing review", or "check my diff". - Proactively suggest when the user is about to merge or land code changes. (gstack) + Use when you review this PR's diff. + Checks it for SQL safety, LLM trust boundaries, conditional side effects and + bugs that pass CI and break production. Fits "look over my diff", "review my + changes", "code review", "check my diff" or "pre-landing review". + Proactively suggest before a merge. (gstack) codex_default_prompt: "Use $gstack-review to review this diff against its base before it lands: find the bugs that pass CI and break in production, fix the clear ones, and list what remains." allowed-tools: - Bash @@ -20,11 +21,29 @@ allowed-tools: - WebSearch triggers: - review this pr + - look over my diff - code review - check my diff + - review my changes - pre-landing review +contract: + version: 1 + items: + - id: specialists-dispatched + kind: selected_specialists_dispatched + marker: "Dispatching N specialists:" + - id: adversarial-pass + kind: agent_dispatch + match: Think like an attacker and a chaos engineer + - id: review-report + kind: report_labels + heading: Pre-Landing Review + labels: + - Gate edits --- +{{CONTRACT_DONE_MEANS}} + {{PREAMBLE}} {{BASE_BRANCH_DETECT}} diff --git a/scripts/resolvers/contract-done-means.ts b/scripts/resolvers/contract-done-means.ts new file mode 100644 index 000000000..5e0a0e1a5 --- /dev/null +++ b/scripts/resolvers/contract-done-means.ts @@ -0,0 +1,142 @@ +/** + * {{CONTRACT_DONE_MEANS}}: a skill's `contract:` frontmatter block (GST-1088, frozen v1 + * schema) rendered as a short "Done means" list at the top of the body, so the + * outputs the model reads as required and the block the Stop check reads cannot + * drift. parseSkillContract is the schema; test/skill-descriptions.test.ts runs it + * over every template, so a malformed block fails CI instead of failing open. + */ + +import * as fs from 'fs'; +import * as path from 'path'; +import type { TemplateContext } from './types'; +import { getHostConfig } from '../../hosts/index'; + +const ROOT = path.resolve(import.meta.dir, '..', '..'); +const code = (s: string) => '\u0060' + s + '\u0060'; + +export type ContractItem = + | { id: string; kind: 'report_labels'; heading: string; labels: string[] } + | { id: string; kind: 'check_after_last_edit' } + | { id: string; kind: 'selected_specialists_dispatched'; marker: string } + | { id: string; kind: 'agent_dispatch'; match: string }; + +export interface SkillContract { + version: 1; + items: ContractItem[]; +} + +const KIND_FIELDS: Record = { + report_labels: ['heading', 'labels'], + check_after_last_edit: [], + selected_specialists_dispatched: ['marker'], + agent_dispatch: ['match'], +}; + +// The resolver and section that render the step each kind checks today. A host that suppresses the resolver, or a loadout that excludes the section, never shows that step, so its items are left out of the list there. +const BACKED_BY: Partial> = { + selected_specialists_dispatched: { resolver: 'REVIEW_ARMY', section: 'review-army' }, + agent_dispatch: { resolver: 'ADVERSARIAL_STEP', section: 'adversarial' }, +}; + +const PLACEHOLDER = /^\[.*\]$|^<.*>$|^\.\.\.$|^(todo|tbd|n\/a)$/i; + +function fail(where: string, problem: string): never { + throw new Error('CONTRACT_INVALID ' + where + ': ' + problem + '. Fix: edit ' + where + ', then bun run gen:skill-docs'); +} + +const isMap = (v: unknown): v is Record => typeof v === 'object' && v !== null && !Array.isArray(v); +const isText = (v: unknown): v is string => typeof v === 'string' && v.trim() !== ''; + +/** Validates a parsed `contract:` value against the frozen v1 schema; where names the template in errors. */ +export function parseSkillContract(value: unknown, where: string): SkillContract { + if (!isMap(value)) fail(where, 'contract must be a mapping { version: 1, items: [...] }'); + for (const key of Object.keys(value)) { + if (key !== 'version' && key !== 'items') fail(where, 'contract has unknown key "' + key + '"'); + } + if (value.version !== 1) fail(where, 'contract version must be 1, got ' + JSON.stringify(value.version)); + if (!Array.isArray(value.items) || value.items.length === 0) fail(where, 'contract items must be a non-empty list'); + const seen = new Set(); + const items = value.items.map((raw: unknown, i: number) => { + const at = 'items[' + i + ']'; + if (!isMap(raw)) fail(where, at + ' must be a mapping'); + const kind = raw.kind as ContractItem['kind']; + if (typeof raw.kind !== 'string' || !Object.hasOwn(KIND_FIELDS, kind)) { + fail(where, at + ' kind ' + JSON.stringify(raw.kind) + ' is not one of ' + Object.keys(KIND_FIELDS).join(', ')); + } + if (typeof raw.id !== 'string' || !/^[a-z][a-z0-9]*(-[a-z0-9]+)*$/.test(raw.id)) { + fail(where, at + ' id must be lowercase kebab-case, got ' + JSON.stringify(raw.id)); + } + if (seen.has(raw.id)) fail(where, at + ' duplicate id "' + raw.id + '"'); + seen.add(raw.id); + for (const key of Object.keys(raw)) { + if (key !== 'id' && key !== 'kind' && !KIND_FIELDS[kind].includes(key)) fail(where, at + ' (' + kind + ') has unknown key "' + key + '"'); + } + for (const field of KIND_FIELDS[kind]) { + if (field !== 'labels' && !isText(raw[field])) fail(where, at + ' ' + field + ' must be a non-empty string'); + } + if (kind === 'report_labels') { + if (!Array.isArray(raw.labels) || raw.labels.length === 0) fail(where, at + ' labels must be a non-empty list'); + for (const label of raw.labels as unknown[]) { + if (!isText(label)) fail(where, at + ' labels must be non-empty strings'); + if (PLACEHOLDER.test(label.trim())) fail(where, at + ' label ' + JSON.stringify(label) + ' is a placeholder'); + } + } + return raw as ContractItem; + }); + return { version: 1, items }; +} + +/** The template's contract, or null without one; required makes a missing block an error. */ +export function contractFromTemplate(tmplContent: string, where: string, opts: { required?: boolean } = {}): SkillContract | null { + const block = tmplContent.replace(/\r\n/g, '\n').match(/^---\n([\s\S]*?)\n---(?:\n|$)/); + if (!block || !/^contract:/m.test(block[1])) { + if (opts.required) fail(where, '{{CONTRACT_DONE_MEANS}} needs a contract block in the frontmatter'); + return null; + } + let frontmatter: unknown; + try { + frontmatter = Bun.YAML.parse(block[1]); + } catch (error) { + fail(where, 'frontmatter is not valid YAML (' + (error instanceof Error ? error.message : String(error)) + ')'); + } + return parseSkillContract(isMap(frontmatter) ? frontmatter.contract : undefined, where); +} + +const list = (xs: string[]) => xs.length < 2 ? xs.join('') : xs.slice(0, -1).join(', ') + ' and ' + xs[xs.length - 1]; + +function plainWords(item: ContractItem): string { + switch (item.kind) { + case 'report_labels': return 'a ' + code(item.heading) + ' with ' + list(item.labels) + ' filled in (' + code('none: ') + ' where the skill\'s own stop rule leaves one empty)'; + case 'check_after_last_edit': return 'a test or repro command run after your last edit'; + case 'selected_specialists_dispatched': return 'the specialist selection printed (' + code(item.marker) + ' ...), then one Agent dispatch per specialist it names (none when a small diff skips them)'; + case 'agent_dispatch': return 'an Agent dispatch for the ' + item.id.replace(/-/g, ' ') + ', its prompt including "' + item.match + '"'; + } +} + +/** Whether bin/gstack-skill-contract ships. The PR that adds it flips this, regenerates and re-pins; test/skill-descriptions.test.ts fails while the two disagree. */ +export const CONTRACT_CHECKER_SHIPPED = false; + +/** The list; checker adds the check and waive commands, which only a tree that ships the checker may mention. */ +export function renderDoneMeans(contract: SkillContract, checker = CONTRACT_CHECKER_SHIPPED): string { + if (contract.items.length === 0) return ''; + return [ + '## Done means', + '', + 'Finish only when each of these exists (contract v1):', + '', + ...contract.items.map(item => '- ' + code(item.id) + ': ' + plainWords(item) + '.'), + ...(checker ? ['', code('gstack-skill-contract check') + ' reads them from the transcript (docs/skill-routing-and-contracts.md). ' + + 'For an item that truly does not apply, run ' + code('gstack-skill-contract waive --reason ""') + ' (this session only; it counts as a miss).'] : []), + ].join('\n'); +} + +export function generateContractDoneMeans(ctx: TemplateContext): string { + const where = path.relative(ROOT, ctx.tmplPath).split(path.sep).join('/'); + const contract = contractFromTemplate(fs.readFileSync(ctx.tmplPath, 'utf-8'), where, { required: true })!; + const suppressed = new Set(getHostConfig(ctx.host).suppressedResolvers ?? []); + const shown = (item: ContractItem) => { + const backing = BACKED_BY[item.kind]; + return !backing || (!suppressed.has(backing.resolver) && !ctx.modules?.excluded.has('section:' + ctx.skillName + '/' + backing.section)); + }; + return renderDoneMeans({ ...contract, items: contract.items.filter(shown) }); +} diff --git a/scripts/resolvers/index.ts b/scripts/resolvers/index.ts index e205fcae9..2fd070d74 100644 --- a/scripts/resolvers/index.ts +++ b/scripts/resolvers/index.ts @@ -58,6 +58,7 @@ import { generateShipDriverStops } from './ship-driver'; import { generateQAScope, generateQAExploratory, generateQAFunctional, generateQAResource, generateQAReview, generateQAReviewPreflight, generateQAMethodReads } from './qa'; import { generateVoiceTriggerRouting } from './voice-triggers'; import { generateTranscriptDiscovery, generateTranscriptSources } from './transcripts'; +import { generateContractDoneMeans } from './contract-done-means'; export const RESOLVERS: Record = { AUTOPLAN_PUBLICATION_HOOK: generateAutoplanPublicationHook, @@ -197,4 +198,5 @@ export const RESOLVERS: Record = { SECTION_INDEX, TRANSCRIPT_SOURCES: generateTranscriptSources, TRANSCRIPT_DISCOVERY: generateTranscriptDiscovery, + CONTRACT_DONE_MEANS: generateContractDoneMeans, }; diff --git a/test/fixtures/skill-md-size-allowlist.json b/test/fixtures/skill-md-size-allowlist.json index a0627d4b9..46678fc24 100644 --- a/test/fixtures/skill-md-size-allowlist.json +++ b/test/fixtures/skill-md-size-allowlist.json @@ -6,52 +6,52 @@ "agy/gstack-plan-design-review": 109107, "agy/gstack-plan-devex-review": 114948, "agy/gstack-plan-eng-review": 143581, - "agy/gstack-review": 104182, + "agy/gstack-review": 104488, "codex/gstack-autoplan": 108400, "codex/gstack-plan-design-review": 110464, "codex/gstack-plan-devex-review": 116456, "codex/gstack-plan-eng-review": 144746, - "codex/gstack-review": 103705, + "codex/gstack-review": 104011, "copilot/gstack-autoplan": 109100, "copilot/gstack-plan-design-review": 109373, "copilot/gstack-plan-devex-review": 114410, "copilot/gstack-plan-eng-review": 143746, - "copilot/gstack-review": 103763, + "copilot/gstack-review": 104069, "cursor/gstack-autoplan": 108332, "cursor/gstack-plan-design-review": 107915, "cursor/gstack-plan-devex-review": 113680, "cursor/gstack-plan-eng-review": 142336, - "cursor/gstack-review": 122053, + "cursor/gstack-review": 122541, "factory/gstack-autoplan": 110610, "factory/gstack-plan-design-review": 111013, "factory/gstack-plan-devex-review": 117043, "factory/gstack-plan-eng-review": 145444, - "factory/gstack-review": 125826, + "factory/gstack-review": 126314, "gbrain/gstack-autoplan": 108739, "gbrain/gstack-plan-eng-review": 124963, "gemini/gstack-autoplan": 108890, "gemini/gstack-plan-design-review": 109201, "gemini/gstack-plan-devex-review": 114238, "gemini/gstack-plan-eng-review": 143579, - "gemini/gstack-review": 103585, + "gemini/gstack-review": 103891, "hermes/gstack-autoplan": 108742, "hermes/gstack-plan-eng-review": 124999, "kiro/gstack-autoplan": 108106, "kiro/gstack-plan-design-review": 107761, "kiro/gstack-plan-devex-review": 113532, "kiro/gstack-plan-eng-review": 142198, - "kiro/gstack-review": 121877, + "kiro/gstack-review": 122365, "openclaw/gstack-autoplan": 108562, "openclaw/gstack-plan-eng-review": 123478, "opencode/gstack-autoplan": 109081, "opencode/gstack-plan-design-review": 109229, "opencode/gstack-plan-devex-review": 114288, "opencode/gstack-plan-eng-review": 143605, - "opencode/gstack-review": 122675, + "opencode/gstack-review": 123163, "slate/gstack-autoplan": 108219, "slate/gstack-plan-design-review": 107838, "slate/gstack-plan-devex-review": 113606, "slate/gstack-plan-eng-review": 142267, - "slate/gstack-review": 121965 + "slate/gstack-review": 122453 } } diff --git a/test/harness-register.test.ts b/test/harness-register.test.ts index 6544b11d1..b483d491c 100644 --- a/test/harness-register.test.ts +++ b/test/harness-register.test.ts @@ -248,7 +248,7 @@ describe('GSTA-135: registered skills get used', () => { } const oc = register(opts(d, 'openclaw', { mode: 'all', skillsDir: path.join(d, 'oc') })); const review = fs.readFileSync(path.join(oc.dest, 'gstack-review', 'SKILL.md'), 'utf8'); - expect(descriptionOf(review)[0]).not.toMatch(/^Use /); + expect(descriptionOf(review)[0]).not.toMatch(/^Use for: /); }, 120_000); test('OpenClaw hybrid is the default: the task skills listed directly, the router for the rest; mode switches refresh and uninstall cleanly', () => { diff --git a/test/helpers/carve-guards.ts b/test/helpers/carve-guards.ts index 901100cc1..aa6b49cd5 100644 --- a/test/helpers/carve-guards.ts +++ b/test/helpers/carve-guards.ts @@ -490,10 +490,10 @@ do not launch the downstream skill or open a browser.`, gateAfterStop: undefined, // operational multi-STOP skill, like ship }, behavioral: 'plan', - maxSkeletonBytes: 59_073, // GSTA-96 hoist (2026-10-26): lowered from 80_900 by the shared preamble that moved to sections/preamble.md, keeping the pre-hoist headroom of 319 bytes; measured 58,754. Before: Shared-code identity/skip/action rules + critical-severity validation; measured 74,493 (2026-09-17). + v1.91.12.0 merge of #2999 (review clarity repairs: await reads, research alongside dispatch, /review deadline and setup authority, findings sources) with #3002 (guarded state-root lines, plan-check checkpoints); each fit alone; measured 74,881 (2026-10-01). + Oct 8 wave: shared voice directive bans "load-bearing" and adds the reply-language rule, offset by tightening the bounded closer (net +17 bytes per tier-2+ skill); measured 74,884 (2026-10-08). + gate integrity (HONEST_WORK_GATE_INTEGRITY W2): Step 3.5 Diff scans with the shared GATE_SCAN_BLOCK, the Step 4 carry, the four-option `review-gate-disposition` question in 5c, gate record fields in 5.8 and the Gate edits report line; measured 80,054 (2026-10-10). + Oct 10 multi-agent wave PR D2 (sideways review): Step 4's enum-only paragraph becomes the sideways sweep with the forced-CRITICAL list, plus the flake-fix number sentence in the Gate Integrity paragraph; measured skeleton 80,812 (2026-10-10). + maxSkeletonBytes: 59_774, // GST-1088 lane B (raised from 59,073 to measured, no headroom): the contract: frontmatter block and the F1 "Done means" list rendered from it at the top of the body, plus widened triggers; F1 is a gate-rev-2 recommendation on the plan Garry approved with "do it" on gate rev 1 (2026-11-03), contingent on his gate-rev-2 answer, and lane C's diet is expected to bring /review back under 59,073; measured 59,774. Before: GSTA-96 hoist (2026-10-26): lowered from 80_900 by the shared preamble that moved to sections/preamble.md, keeping the pre-hoist headroom of 319 bytes; measured 58,754. Before: Shared-code identity/skip/action rules + critical-severity validation; measured 74,493 (2026-09-17). + v1.91.12.0 merge of #2999 (review clarity repairs: await reads, research alongside dispatch, /review deadline and setup authority, findings sources) with #3002 (guarded state-root lines, plan-check checkpoints); each fit alone; measured 74,881 (2026-10-01). + Oct 8 wave: shared voice directive bans "load-bearing" and adds the reply-language rule, offset by tightening the bounded closer (net +17 bytes per tier-2+ skill); measured 74,884 (2026-10-08). + gate integrity (HONEST_WORK_GATE_INTEGRITY W2): Step 3.5 Diff scans with the shared GATE_SCAN_BLOCK, the Step 4 carry, the four-option `review-gate-disposition` question in 5c, gate record fields in 5.8 and the Gate edits report line; measured 80,054 (2026-10-10). + Oct 10 multi-agent wave PR D2 (sideways review): Step 4's enum-only paragraph becomes the sideways sweep with the forced-CRITICAL list, plus the flake-fix number sentence in the Gate Integrity paragraph; measured skeleton 80,812 (2026-10-10). minUnionBytes: 89_000, // Phase 4 wave 1; measured union 93,357 mustContain: ['confidence', 'P1', 'P2', 'Review Army', 'adversarial'], - maxSizeRatio: 1.3180, // + GSTA-167 variant G part 1 (plan approved by Garry 2026-10-30, autoplan-20261030161754 "Do it", which names about +0.3k prompt tokens per voice): the native adversarial and outside-voice prompts carry the blocking floor and the Severity and Blocks lines, +537 bytes; measured 1.3180 (2026-10-30). Before: 1.3136. GSTA-96 hoist merged with GSTA-92 phase 2 (2026-10-26): both owner-approved raises together, measured 142,550 / 108,523 = 1.3135. Before: Shared-code feature + critical-severity validation: 128,042 union bytes / 108,523 baseline = 1.1799; preserves content floors. + v1.91.12.0 merge of #2999 (above, plus plan-completion fallback intent and specialist checklist-by-path) with #3002; measured 1.1843 (2026-10-01). + v1.91.19.0 wave: verdict-form Codex callers with sandbox preflight (INV-1/B1), P0/P1 structured gate (B1b), bound-plan discovery and data-not-instructions guard (F1/D3), visible LEARNINGS failures (B5); measured 129,941 / 108,523 = 1.1974 (2026-10-03). + v1.91.18.0 (#3023) merged into the v1.91.19.0 wave; measured 130,025 / 108,523 = 1.1981 (2026-10-04). + Oct 6 fix wave: quota_exhausted preflight branch (#3051); measured 130,359 / 108,523 = 1.2012 (2026-10-06). + Oct 6 fix wave A3/CEO-12: free text reaches commands only through agent-written mktemp files (mktemp block, write rule, path rebuild and refusal at each posting, prompt and brief site); measured 131,314 / 108,523 = 1.2100 (2026-10-06). Combined Oct 6 wave head (A and B): measured 131,966 / 108,523 = 1.2160 (2026-10-06). + Oct 7 wave B3: the Codex probe runs as a command (select-model capture lines, check-sandbox/probe-model/run-with-timeout subcommands) instead of being sourced; measured 132,199 / 108,523 = 1.2182 (2026-10-07). + Oct 7 wave CEO-28/DX-11: review prompts ask for a severity per finding and design proposals use the proposal gate; measured 132,258 / 108,523 = 1.2187 (2026-10-07). + Oct 7 wave guard (CEO-20): conditional `run_in_background` request plus the background-run recovery at each foreground dispatch site; measured 132,233 / 108,523 = 1.2185 (2026-10-07). Merged with the guard area: measured 132,522 / 108,523 = 1.2211 (2026-10-07). + Oct 8 wave: the defensive-security framing and fixture summary mode reach the security specialist and the Red Team (stated once in the dispatch, referenced by the Red Team); measured 133,902 / 108,523 = 1.2339 (2026-10-08). + gate integrity W2 (above); measured 1.282 (2026-10-10). + Oct 10 multi-agent wave PR D2 (sideways review): sideways sweep and flake-fix number in Step 4; `Diff read` and `Undeclared behavior changes` in the review-mode Scope Check (plan-completion section); measured 140,497 / 108,523 = 1.2946 (2026-10-10). + harness lane E (2026-10-26): the lead-judgment stage in the finding merge and the `cause` anchor in the specialist prompts, shared with /ship through scripts/resolvers/review-army.ts; measured 141,949 / 108,523 = 1.3080. + maxSizeRatio: 1.3261, // + GST-1088 lane B (raised from 1.3180 to measured, no headroom): the contract: frontmatter block and the F1 "Done means" list, contingent on Garry's gate-rev-2 answer; lane C's diet is expected to bring /review back under 1.3180; measured 143,907 / 108,523 = 1.3261. Before: + GSTA-167 variant G part 1 (plan approved by Garry 2026-10-30, autoplan-20261030161754 "Do it", which names about +0.3k prompt tokens per voice): the native adversarial and outside-voice prompts carry the blocking floor and the Severity and Blocks lines, +537 bytes; measured 1.3180 (2026-10-30). Before: 1.3136. GSTA-96 hoist merged with GSTA-92 phase 2 (2026-10-26): both owner-approved raises together, measured 142,550 / 108,523 = 1.3135. Before: Shared-code feature + critical-severity validation: 128,042 union bytes / 108,523 baseline = 1.1799; preserves content floors. + v1.91.12.0 merge of #2999 (above, plus plan-completion fallback intent and specialist checklist-by-path) with #3002; measured 1.1843 (2026-10-01). + v1.91.19.0 wave: verdict-form Codex callers with sandbox preflight (INV-1/B1), P0/P1 structured gate (B1b), bound-plan discovery and data-not-instructions guard (F1/D3), visible LEARNINGS failures (B5); measured 129,941 / 108,523 = 1.1974 (2026-10-03). + v1.91.18.0 (#3023) merged into the v1.91.19.0 wave; measured 130,025 / 108,523 = 1.1981 (2026-10-04). + Oct 6 fix wave: quota_exhausted preflight branch (#3051); measured 130,359 / 108,523 = 1.2012 (2026-10-06). + Oct 6 fix wave A3/CEO-12: free text reaches commands only through agent-written mktemp files (mktemp block, write rule, path rebuild and refusal at each posting, prompt and brief site); measured 131,314 / 108,523 = 1.2100 (2026-10-06). Combined Oct 6 wave head (A and B): measured 131,966 / 108,523 = 1.2160 (2026-10-06). + Oct 7 wave B3: the Codex probe runs as a command (select-model capture lines, check-sandbox/probe-model/run-with-timeout subcommands) instead of being sourced; measured 132,199 / 108,523 = 1.2182 (2026-10-07). + Oct 7 wave CEO-28/DX-11: review prompts ask for a severity per finding and design proposals use the proposal gate; measured 132,258 / 108,523 = 1.2187 (2026-10-07). + Oct 7 wave guard (CEO-20): conditional `run_in_background` request plus the background-run recovery at each foreground dispatch site; measured 132,233 / 108,523 = 1.2185 (2026-10-07). Merged with the guard area: measured 132,522 / 108,523 = 1.2211 (2026-10-07). + Oct 8 wave: the defensive-security framing and fixture summary mode reach the security specialist and the Red Team (stated once in the dispatch, referenced by the Red Team); measured 133,902 / 108,523 = 1.2339 (2026-10-08). + gate integrity W2 (above); measured 1.282 (2026-10-10). + Oct 10 multi-agent wave PR D2 (sideways review): sideways sweep and flake-fix number in Step 4; `Diff read` and `Undeclared behavior changes` in the review-mode Scope Check (plan-completion section); measured 140,497 / 108,523 = 1.2946 (2026-10-10). + harness lane E (2026-10-26): the lead-judgment stage in the finding merge and the `cause` anchor in the specialist prompts, shared with /ship through scripts/resolvers/review-army.ts; measured 141,949 / 108,523 = 1.3080. }, codex: { skill: 'codex', diff --git a/test/skill-descriptions.test.ts b/test/skill-descriptions.test.ts new file mode 100644 index 000000000..d6eff7b43 --- /dev/null +++ b/test/skill-descriptions.test.ts @@ -0,0 +1,228 @@ +import { describe, expect, test } from 'bun:test'; +import { existsSync, readdirSync, readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { sentences } from '../lib/skill-triggers'; +import { CONTRACT_CHECKER_SHIPPED, contractFromTemplate, generateContractDoneMeans, parseSkillContract, renderDoneMeans } from '../scripts/resolvers/contract-done-means'; +import { HOST_PATHS } from '../scripts/resolvers/types'; + +/** + * GST-1088 discovery targets: each says when to use it in the part of its + * description every host shows, and every contract block matches the frozen + * v1 schema, so a malformed block fails here instead of failing open in the + * Stop check. + */ + +const ROOT = join(import.meta.dir, '..'); +const SKILLS = ['investigate', 'review', 'qa', 'verify']; +// splitCatalogDescription cuts a longer Claude lead with "...", which drops the end of the when-clause. +const LEAD_CAP = 200; +// test/catalog-budget.test.ts PER_SKILL_BYTE_CAP: name + Claude description bytes. +const CATALOG_ENTRY_CAP = 260; +// Codex, Gemini, Copilot and agy reject a longer description (hosts/*.ts descriptionLimit). +const DESCRIPTION_CAP = 1024; +const TRIGGER_CAP = 32; +const BT = '\u0060'; + +const fix = (skill: string) => 'Fix: edit ' + skill + '/SKILL.md.tmpl, then bun run gen:skill-docs'; +const read = (rel: string) => readFileSync(join(ROOT, rel), 'utf8').replace(/\r\n/g, '\n'); + +function frontmatter(rel: string): Record { + const m = read(rel).match(/^---\n([\s\S]*?)\n---\n/); + if (!m) throw new Error(rel + ': no frontmatter'); + return Bun.YAML.parse(m[1]) as Record; +} + +const body = (rel: string) => read(rel).replace(/^---\n[\s\S]*?\n---\n/, ''); +const triggersOf = (skill: string) => (frontmatter(skill + '/SKILL.md.tmpl').triggers ?? []) as string[]; + +describe('discovery-target descriptions (GST-1088)', () => { + for (const skill of SKILLS) { + const tmpl = skill + '/SKILL.md.tmpl'; + + test(skill + ': the template description opens with a "Use when ..." sentence', () => { + const first = sentences(String(frontmatter(tmpl).description ?? ''))[0] ?? ''; + expect(/^Use when\b/.test(first), tmpl + ': the first sentence is "' + first.slice(0, 80) + '", not a "Use when ..." clause. ' + fix(skill)).toBe(true); + }); + + test(skill + ': the Claude catalog entry keeps the when-clause within its caps', () => { + const rel = skill + '/SKILL.md'; + const fm = frontmatter(rel); + const description = String(fm.description ?? ''); + const lead = description.replace(/\s*\(gstack\)\s*$/, ''); + expect(/^Use when\b/.test(lead), rel + ': the catalog description "' + lead.slice(0, 80) + '" does not start with "Use when". ' + fix(skill)).toBe(true); + expect(lead.length, rel + ': the catalog lead is ' + lead.length + ' chars against the ' + LEAD_CAP + '-char cap. ' + fix(skill)).toBeLessThanOrEqual(LEAD_CAP); + const entry = Buffer.byteLength(String(fm.name)) + Buffer.byteLength(description); + expect(entry, rel + ': name + description is ' + entry + ' bytes against the ' + CATALOG_ENTRY_CAP + '-byte cap. ' + fix(skill)).toBeLessThanOrEqual(CATALOG_ENTRY_CAP); + }); + + test(skill + ': the full description fits every host limit', () => { + const full = String(frontmatter(tmpl).description ?? '').trim(); + expect(full.length, tmpl + ': the description is ' + full.length + ' chars against the ' + DESCRIPTION_CAP + '-char cap. ' + fix(skill)).toBeLessThanOrEqual(DESCRIPTION_CAP); + }); + + test(skill + ': triggers are short and distinct', () => { + const own = triggersOf(skill); + expect(own.length, tmpl + ': no triggers. ' + fix(skill)).toBeGreaterThan(0); + expect(new Set(own).size, tmpl + ': duplicate triggers. ' + fix(skill)).toBe(own.length); + for (const t of own) { + expect(t.length, tmpl + ': trigger "' + t + '" is ' + t.length + ' chars against the ' + TRIGGER_CAP + '-char cap. ' + fix(skill)).toBeLessThanOrEqual(TRIGGER_CAP); + expect(t, tmpl + ': trigger "' + t + '" must be lowercase. ' + fix(skill)).toBe(t.toLowerCase()); + for (const other of SKILLS.filter(s => s !== skill)) { + const clash = triggersOf(other).find(o => o.includes(t) || t.includes(o)); + expect(clash, tmpl + ': trigger "' + t + '" overlaps ' + other + "'s \"" + clash + '". ' + fix(skill)).toBeUndefined(); + } + } + }); + } + + const PROMPTS: Array<[string, string]> = [ + ['this test fails, fix it', 'investigate'], + ['why is this broken?', 'investigate'], + ['the build is broken, here is the stack trace', 'investigate'], + ['review this PR', 'review'], + ['can you look over my diff', 'review'], + ['review my changes before I merge', 'review'], + ['test the site and fix what you find', 'qa'], + ['can you qa test this page', 'qa'], + ['run the checks and make it green', 'verify'], + ['verify my change', 'verify'], + ]; + for (const [prompt, want] of PROMPTS) { + test('untold prompt "' + prompt + '" carries a ' + want + ' trigger and no other target trigger', () => { + const p = prompt.toLowerCase(); + const hits = SKILLS.filter(s => triggersOf(s).some(t => p.includes(t))); + expect(hits, '"' + prompt + '" matches [' + hits.join(', ') + '], want [' + want + ']. ' + fix(want)).toEqual([want]); + }); + } +}); + +describe('contract blocks (frozen v1 schema)', () => { + const templates = readdirSync(ROOT, { withFileTypes: true }) + .filter(d => d.isDirectory() && existsSync(join(ROOT, d.name, 'SKILL.md.tmpl'))) + .map(d => d.name) + .sort(); + + test('investigate and review declare contracts', () => { + for (const skill of ['investigate', 'review']) { + const tmpl = skill + '/SKILL.md.tmpl'; + expect(contractFromTemplate(read(tmpl), tmpl), tmpl + ': no contract block. ' + fix(skill)).not.toBeNull(); + } + }); + + for (const skill of templates) { + const tmpl = skill + '/SKILL.md.tmpl'; + const contract = contractFromTemplate(read(tmpl), tmpl); + if (!contract) { + test(skill + ': no contract, no Done-means placeholder', () => { + expect(read(tmpl).includes('{{CONTRACT_DONE_MEANS}}'), tmpl + ': {{CONTRACT_DONE_MEANS}} without a contract block. ' + fix(skill)).toBe(false); + }); + continue; + } + + test(skill + ': the body carries the Done-means placeholder and the render carries the list', () => { + expect(body(tmpl).includes('{{CONTRACT_DONE_MEANS}}'), tmpl + ': a contract block needs {{CONTRACT_DONE_MEANS}} at the top of the body. ' + fix(skill)).toBe(true); + expect(read(skill + '/SKILL.md').includes(renderDoneMeans(contract)), skill + '/SKILL.md: the Done-means list is stale. ' + fix(skill)).toBe(true); + }); + + test(skill + ': every contract marker appears in the skill text', () => { + const sectionDir = join(ROOT, skill, 'sections'); + const sections = existsSync(sectionDir) + ? readdirSync(sectionDir).filter(f => f.endsWith('.md')).map(f => read(skill + '/sections/' + f)) + : []; + const text = [body(tmpl), ...sections].join('\n'); + for (const item of contract.items) { + const markers = item.kind === 'report_labels' ? [item.heading, ...item.labels.map(l => l + ':')] + : item.kind === 'selected_specialists_dispatched' ? [item.marker] + : item.kind === 'agent_dispatch' ? [item.match] + : []; + for (const marker of markers) { + expect(text.includes(marker), tmpl + ': contract item ' + item.id + ' needs "' + marker + '", which the skill text never prints. ' + fix(skill)).toBe(true); + } + } + }); + } +}); + +describe('contract-done-means renderer', () => { + const all = parseSkillContract({ + version: 1, + items: [ + { id: 'debug-report', kind: 'report_labels', heading: 'DEBUG REPORT', labels: ['Symptom', 'Root cause', 'Status'] }, + { id: 'check-after-last-edit', kind: 'check_after_last_edit' }, + { id: 'specialists-dispatched', kind: 'selected_specialists_dispatched', marker: 'Dispatching N specialists:' }, + { id: 'adversarial-pass', kind: 'agent_dispatch', match: 'chaos engineer' }, + ], + }, 'fixture/SKILL.md.tmpl'); + const out = renderDoneMeans(all, true); + + test('names each item kind in plain words', () => { + expect(out).toContain('## Done means'); + expect(out).toContain('- ' + BT + 'debug-report' + BT + ': a ' + BT + 'DEBUG REPORT' + BT + ' with Symptom, Root cause and Status filled in (' + BT + 'none: ' + BT + " where the skill's own stop rule leaves one empty)."); + expect(out).toContain('- ' + BT + 'check-after-last-edit' + BT + ': a test or repro command run after your last edit.'); + expect(out).toContain('- ' + BT + 'specialists-dispatched' + BT + ': the specialist selection printed (' + BT + 'Dispatching N specialists:' + BT + ' ...), then one Agent dispatch per specialist it names (none when a small diff skips them).'); + expect(out).toContain('- ' + BT + 'adversarial-pass' + BT + ': an Agent dispatch for the adversarial pass, its prompt including "chaos engineer".'); + }); + + test('points to check and waive in the routing-and-contracts doc, never the off switch', () => { + expect(out).toContain(BT + 'gstack-skill-contract check' + BT); + expect(out).toContain(BT + 'gstack-skill-contract waive --reason ""' + BT); + expect(out).toContain('docs/skill-routing-and-contracts.md'); + expect(out).not.toContain('skill_contracts'); + }); + + const one = (item: Record) => ({ version: 1, items: [item] }); + const MALFORMED: Array<[string, unknown, RegExp]> = [ + ['not a mapping', 'yes', /must be a mapping/], + ['wrong version', { version: 2, items: [{ id: 'a', kind: 'check_after_last_edit' }] }, /version must be 1/], + ['no items', { version: 1, items: [] }, /items must be a non-empty list/], + ['unknown top-level key', { version: 1, items: [{ id: 'a', kind: 'check_after_last_edit' }], extra: 1 }, /unknown key "extra"/], + ['unknown kind', one({ id: 'a', kind: 'vibes' }), /kind "vibes" is not one of/], + ['bad id', one({ id: 'Debug Report', kind: 'check_after_last_edit' }), /id must be lowercase kebab-case/], + ['duplicate id', { version: 1, items: [{ id: 'a', kind: 'check_after_last_edit' }, { id: 'a', kind: 'agent_dispatch', match: 'x' }] }, /duplicate id "a"/], + ['labels missing', one({ id: 'a', kind: 'report_labels', heading: 'R' }), /labels must be a non-empty list/], + ['placeholder label', one({ id: 'a', kind: 'report_labels', heading: 'R', labels: ['[what the user observed]'] }), /placeholder/], + ['heading missing', one({ id: 'a', kind: 'report_labels', labels: ['Symptom'] }), /heading must be a non-empty string/], + ['marker missing', one({ id: 'a', kind: 'selected_specialists_dispatched' }), /marker must be a non-empty string/], + ['match missing', one({ id: 'a', kind: 'agent_dispatch' }), /match must be a non-empty string/], + ['field from another kind', one({ id: 'a', kind: 'check_after_last_edit', match: 'x' }), /unknown key "match"/], + ]; + for (const [name, value, want] of MALFORMED) { + test('a malformed block fails loudly: ' + name, () => { + let message = ''; + try { parseSkillContract(value, 'demo/SKILL.md.tmpl'); } catch (e) { message = (e as Error).message; } + expect(message).toMatch(want); + expect(message).toContain('CONTRACT_INVALID demo/SKILL.md.tmpl'); + expect(message).toContain('Fix: edit demo/SKILL.md.tmpl, then bun run gen:skill-docs'); + }); + } + + test('check and waive appear only once bin/gstack-skill-contract ships', () => { + const shipped = existsSync(join(ROOT, 'bin', 'gstack-skill-contract')); + expect(CONTRACT_CHECKER_SHIPPED, 'CONTRACT_CHECKER_SHIPPED in scripts/resolvers/contract-done-means.ts must be ' + shipped + ' while bin/gstack-skill-contract ' + (shipped ? 'exists' : 'is missing') + '. Fix: set it, then bun run gen:skill-docs').toBe(shipped); + for (const skill of ['investigate', 'review']) { + expect(read(skill + '/SKILL.md').includes('gstack-skill-contract'), skill + '/SKILL.md names gstack-skill-contract while bin/gstack-skill-contract ' + (shipped ? 'exists' : 'is missing') + '. Fix: bun run gen:skill-docs').toBe(shipped); + } + if (shipped) expect(existsSync(join(ROOT, 'docs', 'skill-routing-and-contracts.md')), 'docs/skill-routing-and-contracts.md is missing').toBe(true); + expect(renderDoneMeans(all, false)).not.toContain('gstack-skill-contract'); + }); + + test('a host that suppresses, or a loadout that excludes, the step an item checks leaves that item out', () => { + const render = (host: string, excluded: string[] = []) => generateContractDoneMeans({ skillName: 'review', tmplPath: join(ROOT, 'review', 'SKILL.md.tmpl'), host, paths: HOST_PATHS[host], modules: excluded.length ? { name: 'test', excluded: new Set(excluded) } : undefined } as unknown as Parameters[0]); + const ids = (host: string, excluded: string[] = []) => [...render(host, excluded).matchAll(/^- \u0060([a-z-]+)\u0060:/gm)].map(m => m[1]); + expect(ids('claude', ['section:review/review-army', 'section:review/adversarial'])).toEqual(['review-report']); + expect(ids('claude')).toEqual(['specialists-dispatched', 'adversarial-pass', 'review-report']); + expect(ids('codex')).toEqual(['adversarial-pass', 'review-report']); + expect(ids('hermes')).toEqual(['review-report']); + expect(renderDoneMeans({ version: 1, items: [] })).toBe(''); + }); + + test('a placeholder with no contract block fails the render', () => { + expect(() => contractFromTemplate('---\nname: demo\n---\n{{CONTRACT_DONE_MEANS}}\n', 'demo/SKILL.md.tmpl', { required: true })) + .toThrow(/CONTRACT_INVALID demo\/SKILL.md.tmpl: \{\{CONTRACT_DONE_MEANS\}\} needs a contract block/); + }); + + test('invalid frontmatter YAML fails the render', () => { + expect(() => contractFromTemplate('---\nname: demo\ncontract: [unclosed\n---\n', 'demo/SKILL.md.tmpl')) + .toThrow(/CONTRACT_INVALID demo\/SKILL.md.tmpl: frontmatter is not valid YAML/); + }); +}); diff --git a/verify/SKILL.md b/verify/SKILL.md index ea9e4c937..b321341dc 100644 --- a/verify/SKILL.md +++ b/verify/SKILL.md @@ -2,11 +2,13 @@ name: verify preamble-tier: 1 version: 0.1.0 -description: "Build loop while coding: run the repo's checks and the tests /ship would pick, fix, repeat until green. (gstack)" +description: "Use when coding: run the checks and tests /ship picks until green. (gstack)" triggers: - verify my change - run the checks - make the tests pass + - make it green + - is it green - check my work before ship allowed-tools: - Bash diff --git a/verify/SKILL.md.tmpl b/verify/SKILL.md.tmpl index 8f21e5625..78bc39853 100644 --- a/verify/SKILL.md.tmpl +++ b/verify/SKILL.md.tmpl @@ -3,14 +3,16 @@ name: verify preamble-tier: 1 version: 0.1.0 description: | - Build loop while coding: run the repo's checks and the tests /ship would - pick, fix, repeat until green. Never green when no test covers a change. - Use when asked to "verify", "check my work", "run the checks", "make it - green", or after each change before /ship. (gstack) + Use when coding: run the checks and tests /ship picks until green. + Build loop: run the repo's checks and the tests /ship would pick, fix, repeat. + Never green when no test covers a change. Fits "verify", "check my work", "run + the checks", "make it green", or after each change before /ship. (gstack) triggers: - verify my change - run the checks - make the tests pass + - make it green + - is it green - check my work before ship allowed-tools: - Bash