mirror of
https://github.com/garrytan/gstack.git
synced 2026-06-01 15:51:41 +02:00
v1.54.0.0 feat: carve /ship into skeleton + on-demand sections (-59% always-loaded) (#1806)
* feat(test): transcript-section-logger + ship-action fingerprint (T10) Pure-analysis module over a SkillTestResult/NDJSON transcript: - extractSectionReads(): which sections/*.md a run opened (post-carve check) - extractShipActions(): observable action fingerprint (merge/test/bump/ changelog/commit/push/pr) that works on the MONOLITH too, so a baseline captured before the carve can detect a sectioned-ship regression - baseline read/write + compareShipActions() for baseline-first dogf(T10) Baseline-first answers the Codex outside-voice critique that a logger in the same PR as the carve is post-failure telemetry without a pre-carve reference. 11 unit tests, all green. Paid monolith baseline capture runs separately. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): section discovery + generation machinery (T9) - discover-skills.ts: discoverSectionTemplates() scans <skill>/sections/*.md.tmpl - gen-skill-docs.ts: extract resolvePlaceholders + applyHostRewrites + buildContext as shared helpers (processTemplate and the new processSectionTemplate both call them, so a sanitization/rewrite fix can't miss sections) [C1] - processSectionTemplate: body-fragment generation (no frontmatter/catalog/voice), parent-skill TemplateContext (skillName pinned to parent, not 'sections', so appliesTo gating + tier behave identically), per-host output routing - --host all now fails the build on ANY host failure, not just claude, so a stale external-host output can't slip the freshness gate [Codex outside-voice #9] Inert until a skill is carved (no sections/ dirs exist yet). Refactor is output-neutral: gen:skill-docs --dry-run --host all reports 0 STALE. 5 discovery unit tests + 389 gen-skill-docs tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(setup): install sections/ for cherry-pick targets (claude + kiro) (T9) Two install targets cherry-pick SKILL.md and would leave a carved skill's sections/ behind, 404ing a runtime 'Read sections/<name>.md': - link_claude_skill_dirs: link the sections/ subdir via _link_or_copy (windows gets a fresh copy on every ./setup) - kiro per-skill loop: sed-rewrite + copy each sections/* so paths resolve under ~/.kiro, not ~/.codex/~/.claude codex/factory/opencode link the whole generated dir, so sections ride free. Addresses Codex outside-voice #4/#6 (runtime pathing landmine). Inert until a skill is carved. Static-tripwire test + windows-fallback invariant green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ship): gstack-version-bump CLI — tested idempotency classify + write (T9) Hybrid CLI extraction (CM1): the deterministic core of ship Step 12 becomes a tested CLI instead of bash prose the agent re-derives each run. - classify: FRESH/ALREADY_BUMPED/DRIFT_STALE_PKG/DRIFT_UNEXPECTED from VERSION vs origin/<base>:VERSION vs package.json.version (pure reader) - write: validated dual-write to VERSION + package.json (FRESH bump) - repair: DRIFT_STALE_PKG sync, no re-bump Bump-LEVEL choice + queue collision stay agent judgment; slot pick stays bin/gstack-next-version. This removes the re-bump-a-shipped-branch footgun from skippable prose into code that can't be skipped or misread. 15 tests (exhaustive state matrix + write/repair fs + real-git classify). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(parity): sectioned-skill parity capability — guards the carve (T9) Carved skills (skeleton + sections/*.md) need parity checks that see relocated content, or moving a phrase into a section reads as 'lost': - readSkillForParity(): union skeleton + all sections/*.md - checkSkillParity sectioned mode: content checks against the union; minBytes/ maxSizeRatio against union bytes (total behavior preserved); maxSkeletonBytes asserts the always-loaded skeleton actually shrank. Lowering minBytes to fit a small skeleton would otherwise make the size floor toothless [Codex #12]. Built + tested BEFORE the carve so ship's invariant can flip to sectioned in the same commit it lands. Monolith path byte-identical (verified: pre-existing investigate 1.053 ratio drift fails the same with this change stashed). 7 sectioned-parity tests + existing parity tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ship): carve into skeleton + on-demand sections (Claude) (T9) ship/SKILL.md drops 167KB → 68.7KB (~59% of the always-loaded skill) by moving 8 prose-heavy steps into ship/sections/*.md, read on demand: tests, test-coverage, plan-completion, review-army, greptile, adversarial, changelog, pr-body. Step 12's version logic now calls the tested gstack-version-bump CLI instead of inline bash. Claude-first (S2): {{SECTION:id}} emits a STOP-Read pointer on Claude (skeleton + generated section files) and INLINES the content on every other host, so external hosts keep the full monolith — verified factory at 162KB with no sections dir. {{SECTION_INDEX:ship}} renders the situation→section table from the PASSIVE manifest (CM2 / v2_PLAN.md:663); required-reads live only in test fixtures. Multi-pass resolve expands inlined sections' own resolvers. Parity: ship invariant flipped to sectioned (union content checks + maxSkeletonBytes asserts the shrink). Carve-fallout fixed across gen-skill-docs/skill-validation/ golden/plan-completion/#1539/size-budget tests via skeleton+sections union reads. Free suite green except the pre-existing investigate parity drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ship): manifest-consistency + context-parity + requiredReads helper (T9) Free deterministic guards for the carve: - required-reads.ts + unit test: assertRequiredReads(run, requiredFiles) — the mechanical layer-5 check that the agent Read the sections its situation needs (required set comes from the fixture, not the passive manifest) - section-manifest-consistency: 3-tier orphan classification (generated orphan + hand-edited generated file → FAIL; manifest orphan → WARN per v2_PLAN.md) and pins the PASSIVE-manifest contract (no applies_when/required_for) - template-context-parity: generated sections have zero unresolved placeholders and gated resolvers (ADVERSARIAL_STEP/CONFIDENCE_CALIBRATION/CHANGELOG_WORKFLOW) rendered — proving sections resolve with the parent skillName, not 'sections' 16 tests, all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ship): section-loading E2E + idempotency CLI detection (T9) - skill-e2e-ship-section-loading.test.ts (new, periodic): runs real /ship in plan mode against a fresh version-changing fixture and asserts the agent Read the required sections (review-army + changelog). Runs against the INSTALLED skill (~/.claude/skills/gstack/ship), not repo paths, so install-layout 404s surface [Codex outside-voice #5]. Layer-5 mechanical guard against silent section-skip. - skill-e2e-ship-idempotency.test.ts: detection updated for the carve — Step 12 now runs gstack-version-bump classify (JSON "state":"ALREADY_BUMPED") instead of the inline bash echo (STATE: ALREADY_BUMPED). Accept both; add a gstack-version-bump-write re-bump regression signal. - touchfiles: register ship-section-loading (periodic) + extend idempotency deps with bin/gstack-version-bump + scripts/resolvers/sections.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ship): union-read redaction wiring test for the carve (T9) main's PR-body redaction-at-sink lives in sections/pr-body.md.tmpl after the carve, not the skeleton template. Read skeleton + section templates union so the redaction-wiring assertions follow the relocated content. 9/9 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v1.54.0.0 feat: carve /ship into skeleton + on-demand sections (-59% always-loaded) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
@@ -1,5 +1,44 @@
|
||||
# Changelog
|
||||
|
||||
## [1.54.0.0] - 2026-05-30
|
||||
|
||||
## **The heaviest skill stopped taxing every session. /ship's always-loaded cost dropped 59%, and its prose now loads only when a step needs it.**
|
||||
|
||||
`/ship` was a 167KB wall that every session paid for in full, whether you were bumping a version or writing a changelog or none of it. It is now a 69KB decision-tree skeleton plus eight `sections/*.md` files the agent opens on demand. The eight steps that are long prose (the test run, coverage audit, plan-completion, the review army, Greptile triage, the adversarial pass, the changelog, the PR body) moved into sections behind STOP-Read pointers, so a run only reads the chapters its situation calls for. The version-bump logic that used to be ~90 lines of inline bash, the single worst re-bump footgun in the workflow, is now the tested `gstack-version-bump` CLI (classify / write / repair). Other hosts (codex, factory, kiro, opencode) keep the full inline skill unchanged, so nothing regresses off Claude. This release dogfooded itself: the version you are reading was bumped by `gstack-version-bump`.
|
||||
|
||||
### The numbers that matter
|
||||
|
||||
Measured directly from the generated skill (`wc -c ship/SKILL.md`) and the new section files, regenerated for all hosts:
|
||||
|
||||
| Metric | Before (v1.53) | After (v1.54) | Δ |
|
||||
|--------|----------------|---------------|---|
|
||||
| ship always-loaded | 167 KB (~41.8K tokens) | 69 KB (~17.2K tokens) | -59% |
|
||||
| ship prose loaded per run | all of it | only applicable sections | on-demand |
|
||||
| ship version logic | ~90 lines inline bash | tested CLI, 15 unit tests | extracted |
|
||||
| External-host ship | 167 KB inline | 162 KB inline (unchanged behavior) | no regression |
|
||||
|
||||
The skeleton is what loads the instant `/ship` is invoked, so the ~24.6K-token drop is paid back on every single ship, not just once.
|
||||
|
||||
### What this means for you
|
||||
|
||||
A `/ship` run starts ~3x lighter and pulls in each heavy step's instructions only when it reaches that step, so the agent spends less of its window holding prose it is not using yet. You will not notice any behavior change. The workflow is identical step for step; the difference is what is in context when. If you ever want to read a step in isolation, the chapters live at `~/.claude/skills/gstack/ship/sections/`.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
#### Added
|
||||
- `bin/gstack-version-bump` — tested version-state CLI (classify / write / repair) with 15 unit tests covering the full FRESH / ALREADY_BUMPED / DRIFT_STALE_PKG / DRIFT_UNEXPECTED matrix.
|
||||
- `ship/sections/*.md` — eight on-demand sections (tests, test-coverage, plan-completion, review-army, greptile, adversarial, changelog, pr-body) with a passive `manifest.json` registry.
|
||||
- Section pipeline in `gen-skill-docs`: `{{SECTION:id}}` (STOP-Read pointer on Claude, inline on other hosts) and `{{SECTION_INDEX}}` (situation to section table rendered from the manifest).
|
||||
- `test/helpers/transcript-section-logger.ts` + `required-reads.ts` and section-loading / manifest-consistency / context-parity tests guarding the carve.
|
||||
|
||||
#### Changed
|
||||
- `/ship` is a skeleton + sections on Claude; external hosts still receive the full inline skill (no behavior change off Claude).
|
||||
- Step 12 calls `gstack-version-bump` instead of inline bash.
|
||||
- Parity harness understands carved skills (checks skeleton + sections union; asserts the skeleton actually shrank).
|
||||
|
||||
#### For contributors
|
||||
- `setup` links `sections/` into the prefixed Claude + Kiro skill dirs; `--host all` now fails the build on any host failure, not just claude.
|
||||
- New section templates live at `<skill>/sections/*.md.tmpl`; regenerate with `bun run gen:skill-docs`.
|
||||
## [1.53.1.0] - 2026-05-30
|
||||
|
||||
## **Workspace and scripted setup never hang on a hidden prompt again. Installing the plan-tune hooks is now flag-driven with safe defaults.**
|
||||
|
||||
Executable
+212
@@ -0,0 +1,212 @@
|
||||
#!/usr/bin/env bun
|
||||
// gstack-version-bump — deterministic version-state classifier + writer for /ship.
|
||||
//
|
||||
// Extracted from ship Step 12 prose (v2 plan T9, hybrid CLI extraction). The
|
||||
// idempotency classification and the dual-write to VERSION + package.json are
|
||||
// pure deterministic logic; running them as tested code removes the single
|
||||
// worst /ship footgun — re-bumping an already-shipped branch — from prose the
|
||||
// agent could skip or misread when the step lives in a lazy-loaded section.
|
||||
//
|
||||
// What STAYS agent judgment (NOT here): the bump-LEVEL decision (micro/patch vs
|
||||
// minor/major, which may AskUserQuestion on feature signals) and the queue
|
||||
// collision prompt. The slot pick itself is bin/gstack-next-version. This CLI
|
||||
// only answers "what state am I in?" and "write this exact version".
|
||||
//
|
||||
// Subcommands:
|
||||
// classify --base <branch> [--version-path <p>]
|
||||
// Compares VERSION vs origin/<base>:VERSION vs package.json.version.
|
||||
// Emits JSON: { state, baseVersion, currentVersion, pkgVersion, pkgExists }
|
||||
// state ∈ FRESH | ALREADY_BUMPED | DRIFT_STALE_PKG | DRIFT_UNEXPECTED
|
||||
// Exit 0 on a decidable state (incl. DRIFT_UNEXPECTED — it's a real state
|
||||
// the caller must handle), exit 2 on bad args / unresolvable base.
|
||||
//
|
||||
// write --version <X.Y.Z.W> [--version-path <p>]
|
||||
// Validates the 4-digit pattern, writes VERSION + package.json.version.
|
||||
// Use for the FRESH bump (or an approved queue rebump). Exit 3 on a
|
||||
// half-write (VERSION written, package.json failed) so the caller knows
|
||||
// drift exists; the next classify() will report DRIFT_STALE_PKG.
|
||||
//
|
||||
// repair [--version-path <p>]
|
||||
// DRIFT_STALE_PKG path: sync package.json.version to the current VERSION
|
||||
// file. No bump. Validates the VERSION pattern first.
|
||||
//
|
||||
// Contract: classify NEVER writes. write/repair mutate VERSION + package.json
|
||||
// only. No git mutation, no network. Mirrors gstack-next-version's reader/writer
|
||||
// split so /ship composes them.
|
||||
|
||||
import { existsSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { join } from "node:path";
|
||||
|
||||
const VERSION_RE = /^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$/;
|
||||
const DEFAULT = "0.0.0.0";
|
||||
|
||||
type State = "FRESH" | "ALREADY_BUMPED" | "DRIFT_STALE_PKG" | "DRIFT_UNEXPECTED";
|
||||
|
||||
function fail(msg: string, code = 2): never {
|
||||
process.stderr.write(`gstack-version-bump: ${msg}\n`);
|
||||
process.exit(code);
|
||||
}
|
||||
|
||||
function argVal(args: string[], flag: string): string | undefined {
|
||||
const i = args.indexOf(flag);
|
||||
return i >= 0 && i + 1 < args.length ? args[i + 1] : undefined;
|
||||
}
|
||||
|
||||
/** Resolve the VERSION file path: --version-path, else .gstack/version-path, else "VERSION". */
|
||||
function resolveVersionPath(cwd: string, explicit?: string): string {
|
||||
if (explicit) return join(cwd, explicit);
|
||||
const pin = join(cwd, ".gstack", "version-path");
|
||||
if (existsSync(pin)) {
|
||||
const p = readFileSync(pin, "utf-8").trim();
|
||||
if (p) return join(cwd, p);
|
||||
}
|
||||
return join(cwd, "VERSION");
|
||||
}
|
||||
|
||||
function readVersionFile(p: string): string {
|
||||
try {
|
||||
const v = readFileSync(p, "utf-8").replace(/[\r\n\s]/g, "");
|
||||
return v || DEFAULT;
|
||||
} catch {
|
||||
return DEFAULT;
|
||||
}
|
||||
}
|
||||
|
||||
/** package.json version + existence, parsed without spawning node. */
|
||||
function readPkgVersion(cwd: string): { exists: boolean; version: string } {
|
||||
const pkgPath = join(cwd, "package.json");
|
||||
if (!existsSync(pkgPath)) return { exists: false, version: "" };
|
||||
let raw: string;
|
||||
try {
|
||||
raw = readFileSync(pkgPath, "utf-8");
|
||||
} catch {
|
||||
return { exists: true, version: "" };
|
||||
}
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(raw);
|
||||
} catch {
|
||||
fail("package.json is not valid JSON. Fix the file before re-running /ship.", 2);
|
||||
}
|
||||
const version = (parsed as { version?: unknown })?.version;
|
||||
return { exists: true, version: typeof version === "string" ? version : "" };
|
||||
}
|
||||
|
||||
function writePkgVersion(cwd: string, version: string): void {
|
||||
const pkgPath = join(cwd, "package.json");
|
||||
const raw = readFileSync(pkgPath, "utf-8");
|
||||
const parsed = JSON.parse(raw) as Record<string, unknown>;
|
||||
parsed.version = version;
|
||||
writeFileSync(pkgPath, JSON.stringify(parsed, null, 2) + "\n");
|
||||
}
|
||||
|
||||
function baseVersion(cwd: string, base: string, versionRel: string): string {
|
||||
// Verify the base ref resolves, mirroring the Step 12 guard.
|
||||
try {
|
||||
execFileSync("git", ["rev-parse", "--verify", `origin/${base}`], { cwd, stdio: "ignore" });
|
||||
} catch {
|
||||
fail(`Unable to resolve origin/${base}. Run 'git fetch origin' or verify the base branch exists.`, 2);
|
||||
}
|
||||
try {
|
||||
const out = execFileSync("git", ["show", `origin/${base}:${versionRel}`], { cwd }).toString();
|
||||
const v = out.replace(/[\r\n\s]/g, "");
|
||||
return v || DEFAULT;
|
||||
} catch {
|
||||
// VERSION absent on base (new repo / new file) → treat as 0.0.0.0.
|
||||
return DEFAULT;
|
||||
}
|
||||
}
|
||||
|
||||
function classifyState(current: string, base: string, pkgExists: boolean, pkgVersion: string): State {
|
||||
if (current === base) {
|
||||
// VERSION unchanged vs base. A diverging package.json means someone hand-edited
|
||||
// package.json bypassing /ship — unsafe to guess which is authoritative.
|
||||
if (pkgExists && pkgVersion && pkgVersion !== current) return "DRIFT_UNEXPECTED";
|
||||
return "FRESH";
|
||||
}
|
||||
// VERSION already moved past base.
|
||||
if (pkgExists && pkgVersion && pkgVersion !== current) return "DRIFT_STALE_PKG";
|
||||
return "ALREADY_BUMPED";
|
||||
}
|
||||
|
||||
function cmdClassify(args: string[], cwd: string): void {
|
||||
const base = argVal(args, "--base");
|
||||
if (!base) fail("classify requires --base <branch>", 2);
|
||||
const versionPath = resolveVersionPath(cwd, argVal(args, "--version-path"));
|
||||
const versionRel = argVal(args, "--version-path") ?? "VERSION";
|
||||
const current = readVersionFile(versionPath);
|
||||
const baseV = baseVersion(cwd, base!, versionRel);
|
||||
const pkg = readPkgVersion(cwd);
|
||||
const state = classifyState(current, baseV, pkg.exists, pkg.version);
|
||||
process.stdout.write(
|
||||
JSON.stringify({
|
||||
state,
|
||||
baseVersion: baseV,
|
||||
currentVersion: current,
|
||||
pkgVersion: pkg.version || null,
|
||||
pkgExists: pkg.exists,
|
||||
}) + "\n",
|
||||
);
|
||||
// DRIFT_UNEXPECTED is a real, decidable state — the caller stops on it, but the
|
||||
// classification itself succeeded, so exit 0. (Bad args / unresolvable base are
|
||||
// the only exit-2 cases.)
|
||||
}
|
||||
|
||||
function cmdWrite(args: string[], cwd: string): void {
|
||||
const version = argVal(args, "--version");
|
||||
if (!version) fail("write requires --version <X.Y.Z.W>", 2);
|
||||
if (!VERSION_RE.test(version!)) {
|
||||
fail(`NEW_VERSION (${version}) does not match MAJOR.MINOR.PATCH.MICRO. Aborting.`, 2);
|
||||
}
|
||||
const versionPath = resolveVersionPath(cwd, argVal(args, "--version-path"));
|
||||
writeFileSync(versionPath, version + "\n");
|
||||
if (existsSync(join(cwd, "package.json"))) {
|
||||
try {
|
||||
writePkgVersion(cwd, version!);
|
||||
} catch {
|
||||
fail(
|
||||
"failed to update package.json. VERSION was written but package.json is now stale. " +
|
||||
"Re-run — classify will report DRIFT_STALE_PKG and repair will sync it.",
|
||||
3,
|
||||
);
|
||||
}
|
||||
}
|
||||
process.stdout.write(JSON.stringify({ wrote: version, packageJson: existsSync(join(cwd, "package.json")) }) + "\n");
|
||||
}
|
||||
|
||||
function cmdRepair(args: string[], cwd: string): void {
|
||||
const versionPath = resolveVersionPath(cwd, argVal(args, "--version-path"));
|
||||
const current = readVersionFile(versionPath);
|
||||
if (!VERSION_RE.test(current)) {
|
||||
fail(
|
||||
`VERSION file contents (${current}) do not match MAJOR.MINOR.PATCH.MICRO. ` +
|
||||
"Refusing to propagate invalid semver into package.json. Fix VERSION, then re-run /ship.",
|
||||
2,
|
||||
);
|
||||
}
|
||||
if (!existsSync(join(cwd, "package.json"))) {
|
||||
fail("repair: no package.json to sync.", 2);
|
||||
}
|
||||
try {
|
||||
writePkgVersion(cwd, current);
|
||||
} catch {
|
||||
fail("drift repair failed — could not update package.json.", 3);
|
||||
}
|
||||
process.stdout.write(JSON.stringify({ repaired: current }) + "\n");
|
||||
}
|
||||
|
||||
// Exported for unit tests (pure logic, no I/O).
|
||||
export { classifyState, VERSION_RE, type State };
|
||||
|
||||
if (import.meta.main) {
|
||||
const [sub, ...rest] = process.argv.slice(2);
|
||||
const cwd = process.cwd();
|
||||
switch (sub) {
|
||||
case "classify": cmdClassify(rest, cwd); break;
|
||||
case "write": cmdWrite(rest, cwd); break;
|
||||
case "repair": cmdRepair(rest, cwd); break;
|
||||
default:
|
||||
fail("usage: gstack-version-bump <classify|write|repair> [flags]", 2);
|
||||
}
|
||||
}
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gstack",
|
||||
"version": "1.53.1.0",
|
||||
"version": "1.54.0.0",
|
||||
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
||||
"license": "MIT",
|
||||
"type": "module",
|
||||
|
||||
@@ -26,6 +26,34 @@ export function discoverTemplates(root: string): Array<{ tmpl: string; output: s
|
||||
return results;
|
||||
}
|
||||
|
||||
/**
|
||||
* Discover on-demand section templates: `<skill>/sections/*.md.tmpl`.
|
||||
*
|
||||
* Returns the relative tmpl path, its generated output path (`.tmpl` stripped),
|
||||
* and the owning skill directory so the generator can build a TemplateContext
|
||||
* with the PARENT skill's name (not "sections") — see processSectionTemplate.
|
||||
*
|
||||
* Scans one level of subdirs (same depth as discoverTemplates), looking only
|
||||
* inside a `sections/` child. Skills without a sections/ dir contribute nothing,
|
||||
* so this is a no-op for every skill that hasn't been carved.
|
||||
*/
|
||||
export function discoverSectionTemplates(
|
||||
root: string,
|
||||
): Array<{ tmpl: string; output: string; skillDir: string }> {
|
||||
const results: Array<{ tmpl: string; output: string; skillDir: string }> = [];
|
||||
for (const dir of subdirs(root)) {
|
||||
const sectionsDir = path.join(root, dir, 'sections');
|
||||
if (!fs.existsSync(sectionsDir) || !fs.statSync(sectionsDir).isDirectory()) continue;
|
||||
for (const entry of fs.readdirSync(sectionsDir, { withFileTypes: true })) {
|
||||
if (!entry.isFile() || !entry.name.endsWith('.md.tmpl')) continue;
|
||||
const rel = `${dir}/sections/${entry.name}`;
|
||||
results.push({ tmpl: rel, output: rel.replace(/\.tmpl$/, ''), skillDir: dir });
|
||||
}
|
||||
}
|
||||
// Deterministic order so CI freshness checks don't flap on FS iteration order.
|
||||
return results.sort((a, b) => a.tmpl.localeCompare(b.tmpl));
|
||||
}
|
||||
|
||||
export function discoverSkillFiles(root: string): string[] {
|
||||
const dirs = ['', ...subdirs(root)];
|
||||
const results: string[] = [];
|
||||
|
||||
+202
-57
@@ -11,7 +11,7 @@
|
||||
|
||||
import { COMMAND_DESCRIPTIONS } from '../browse/src/commands';
|
||||
import { SNAPSHOT_FLAGS } from '../browse/src/snapshot';
|
||||
import { discoverTemplates } from './discover-skills';
|
||||
import { discoverTemplates, discoverSectionTemplates } from './discover-skills';
|
||||
import { writeLlmsTxt } from './gen-llms-txt';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
@@ -574,6 +574,102 @@ function extractHookSafetyProse(tmplContent: string): string | null {
|
||||
|
||||
const GENERATED_HEADER = `<!-- AUTO-GENERATED from {{SOURCE}} — do not edit directly -->\n<!-- Regenerate: bun run gen:skill-docs -->\n`;
|
||||
|
||||
/**
|
||||
* Apply a host's configured path + tool rewrites. Extracted so both SKILL.md
|
||||
* (via processExternalHost) and section files (via processSectionTemplate) get
|
||||
* identical per-host treatment — a section's cross-references must rewrite the
|
||||
* same way the parent skill's do, or external hosts get wrong paths.
|
||||
*/
|
||||
function applyHostRewrites(content: string, hostConfig: HostConfig): string {
|
||||
let result = content;
|
||||
for (const rewrite of hostConfig.pathRewrites) {
|
||||
result = result.replaceAll(rewrite.from, rewrite.to);
|
||||
}
|
||||
if (hostConfig.toolRewrites) {
|
||||
for (const [from, to] of Object.entries(hostConfig.toolRewrites)) {
|
||||
result = result.replaceAll(from, to);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve {{PLACEHOLDER}} / {{NAME:arg}} tokens against the RESOLVERS registry,
|
||||
* honoring host suppression and appliesTo gating, then assert nothing is left
|
||||
* unresolved. Extracted so SKILL.md and section templates resolve through the
|
||||
* exact same path — a security/sanitization fix to one can't miss the other.
|
||||
*/
|
||||
function resolvePlaceholders(
|
||||
tmplContent: string,
|
||||
ctx: TemplateContext,
|
||||
hostConfig: HostConfig,
|
||||
relTmplPath: string,
|
||||
): string {
|
||||
// effectiveSuppressedResolvers() honors --respect-detection: when gbrain is
|
||||
// detected locally, GBRAIN_* resolvers un-suppress. Shared by SKILL.md and
|
||||
// section generation so both paths get the same gbrain-aware behavior.
|
||||
const suppressed = effectiveSuppressedResolvers(hostConfig);
|
||||
const onePass = (input: string): string =>
|
||||
input.replace(/\{\{(\w+(?::[^}]+)?)\}\}/g, (_match, fullKey) => {
|
||||
const parts = fullKey.split(':');
|
||||
const resolverName = parts[0];
|
||||
const args = parts.slice(1);
|
||||
if (suppressed.has(resolverName)) return '';
|
||||
const entry = RESOLVERS[resolverName];
|
||||
if (!entry) throw new Error(`Unknown placeholder {{${resolverName}}} in ${relTmplPath}`);
|
||||
const { resolve, appliesTo } = unwrapResolver(entry);
|
||||
if (appliesTo && !appliesTo(ctx)) return '';
|
||||
return args.length > 0 ? resolve(ctx, args) : resolve(ctx);
|
||||
});
|
||||
|
||||
// Multi-pass: a resolver may emit content that itself contains {{TOKENS}} — the
|
||||
// {{SECTION:id}} resolver inlines a section template (with its own resolvers)
|
||||
// for non-Claude hosts. .replace() doesn't re-scan inserted text, so loop until
|
||||
// the output stabilizes. Bounded to avoid an infinite loop if a resolver ever
|
||||
// emits its own placeholder; 6 passes is far more nesting than any skill needs.
|
||||
let content = tmplContent;
|
||||
for (let pass = 0; pass < 6; pass++) {
|
||||
const next = onePass(content);
|
||||
if (next === content) break;
|
||||
content = next;
|
||||
}
|
||||
|
||||
const remaining = content.match(/\{\{(\w+(?::[^}]+)?)\}\}/g);
|
||||
if (remaining) {
|
||||
throw new Error(`Unresolved placeholders in ${relTmplPath}: ${remaining.join(', ')}`);
|
||||
}
|
||||
return content;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the TemplateContext from a template's frontmatter. Shared by SKILL.md
|
||||
* and section generation so sections inherit the SAME context the parent skill
|
||||
* resolves with (skillName, tier, benefitsFrom, interactive) — enforced by
|
||||
* test/template-context-parity.test.ts. skillNameOverride lets section
|
||||
* generation pin the parent skill's name instead of deriving "sections".
|
||||
*/
|
||||
function buildContext(
|
||||
tmplContent: string,
|
||||
tmplPath: string,
|
||||
host: Host,
|
||||
skillNameOverride?: string,
|
||||
): TemplateContext {
|
||||
const { name: extractedName } = extractNameAndDescription(tmplContent);
|
||||
const skillName = skillNameOverride || extractedName || path.basename(path.dirname(tmplPath));
|
||||
const benefitsMatch = tmplContent.match(/^benefits-from:\s*\[([^\]]*)\]/m);
|
||||
const benefitsFrom = benefitsMatch
|
||||
? benefitsMatch[1].split(',').map(s => s.trim()).filter(Boolean)
|
||||
: undefined;
|
||||
const tierMatch = tmplContent.match(/^preamble-tier:\s*(\d+)$/m);
|
||||
const preambleTier = tierMatch ? parseInt(tierMatch[1], 10) : undefined;
|
||||
const interactiveMatch = tmplContent.match(/^interactive:\s*(true|false)\s*$/m);
|
||||
const interactive = interactiveMatch ? interactiveMatch[1] === 'true' : undefined;
|
||||
return {
|
||||
skillName, tmplPath, benefitsFrom, host, paths: HOST_PATHS[host],
|
||||
preambleTier, model: MODEL_ARG_VAL, interactive, explainLevel: EXPLAIN_LEVEL,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Process external host output: routing, frontmatter, path rewrites, metadata.
|
||||
* Shared between Codex and Factory (and future external hosts).
|
||||
@@ -619,17 +715,9 @@ function processExternalHost(
|
||||
result = result.slice(0, bodyStart) + '\n' + safetyProse + '\n' + result.slice(bodyStart);
|
||||
}
|
||||
|
||||
// Config-driven path rewrites (order matters, replaceAll)
|
||||
for (const rewrite of hostConfig.pathRewrites) {
|
||||
result = result.replaceAll(rewrite.from, rewrite.to);
|
||||
}
|
||||
|
||||
// Config-driven tool rewrites
|
||||
if (hostConfig.toolRewrites) {
|
||||
for (const [from, to] of Object.entries(hostConfig.toolRewrites)) {
|
||||
result = result.replaceAll(from, to);
|
||||
}
|
||||
}
|
||||
// Config-driven path + tool rewrites (shared with processSectionTemplate so
|
||||
// section cross-references get the same per-host treatment as SKILL.md).
|
||||
result = applyHostRewrites(result, hostConfig);
|
||||
|
||||
// Config-driven: generate metadata (e.g., openai.yaml for Codex)
|
||||
if (hostConfig.generation.generateMetadata && !symlinkLoop) {
|
||||
@@ -650,53 +738,18 @@ function processTemplate(tmplPath: string, host: Host = 'claude'): { outputPath:
|
||||
// Determine skill directory relative to ROOT
|
||||
const skillDir = path.relative(ROOT, path.dirname(tmplPath));
|
||||
|
||||
// Extract skill name from frontmatter early — needed for both TemplateContext and external host output paths.
|
||||
// When frontmatter name: differs from directory name (e.g., run-tests/ with name: test),
|
||||
// the frontmatter name is used for external skill naming and setup script symlinks.
|
||||
// Extract name/description: name drives external skill naming + setup symlinks
|
||||
// (and TemplateContext.skillName via buildContext); description feeds external
|
||||
// host metadata. When frontmatter name: differs from directory name (e.g.
|
||||
// run-tests/ with name: test), the frontmatter name wins.
|
||||
const { name: extractedName, description: extractedDescription } = extractNameAndDescription(tmplContent);
|
||||
const skillName = extractedName || path.basename(path.dirname(tmplPath));
|
||||
|
||||
|
||||
// Extract benefits-from list from frontmatter (inline YAML: benefits-from: [a, b])
|
||||
const benefitsMatch = tmplContent.match(/^benefits-from:\s*\[([^\]]*)\]/m);
|
||||
const benefitsFrom = benefitsMatch
|
||||
? benefitsMatch[1].split(',').map(s => s.trim()).filter(Boolean)
|
||||
: undefined;
|
||||
|
||||
// Extract preamble-tier from frontmatter (1-4, controls which preamble sections are included)
|
||||
const tierMatch = tmplContent.match(/^preamble-tier:\s*(\d+)$/m);
|
||||
const preambleTier = tierMatch ? parseInt(tierMatch[1], 10) : undefined;
|
||||
|
||||
// Extract interactive flag from frontmatter (generator-only; controls plan-mode handshake inclusion)
|
||||
const interactiveMatch = tmplContent.match(/^interactive:\s*(true|false)\s*$/m);
|
||||
const interactive = interactiveMatch ? interactiveMatch[1] === 'true' : undefined;
|
||||
|
||||
const ctx: TemplateContext = { skillName, tmplPath, benefitsFrom, host, paths: HOST_PATHS[host], preambleTier, model: MODEL_ARG_VAL, interactive, explainLevel: EXPLAIN_LEVEL };
|
||||
|
||||
// Replace placeholders (supports parameterized: {{NAME:arg1:arg2}})
|
||||
// Config-driven: suppressedResolvers return empty string for this host.
|
||||
// effectiveSuppressedResolvers() honors --respect-detection: when gbrain
|
||||
// is detected locally, GBRAIN_* resolvers un-suppress so brain-aware
|
||||
// blocks render for users who have gbrain installed.
|
||||
const currentHostConfig = getHostConfig(host);
|
||||
const suppressed = effectiveSuppressedResolvers(currentHostConfig);
|
||||
let content = tmplContent.replace(/\{\{(\w+(?::[^}]+)?)\}\}/g, (match, fullKey) => {
|
||||
const parts = fullKey.split(':');
|
||||
const resolverName = parts[0];
|
||||
const args = parts.slice(1);
|
||||
if (suppressed.has(resolverName)) return '';
|
||||
const entry = RESOLVERS[resolverName];
|
||||
if (!entry) throw new Error(`Unknown placeholder {{${resolverName}}} in ${relTmplPath}`);
|
||||
const { resolve, appliesTo } = unwrapResolver(entry);
|
||||
if (appliesTo && !appliesTo(ctx)) return '';
|
||||
return args.length > 0 ? resolve(ctx, args) : resolve(ctx);
|
||||
});
|
||||
const ctx = buildContext(tmplContent, tmplPath, host);
|
||||
const skillName = ctx.skillName;
|
||||
|
||||
// Check for any remaining unresolved placeholders
|
||||
const remaining = content.match(/\{\{(\w+(?::[^}]+)?)\}\}/g);
|
||||
if (remaining) {
|
||||
throw new Error(`Unresolved placeholders in ${relTmplPath}: ${remaining.join(', ')}`);
|
||||
}
|
||||
// Replace placeholders + assert none remain (shared path with section generation).
|
||||
let content = resolvePlaceholders(tmplContent, ctx, currentHostConfig, relTmplPath);
|
||||
|
||||
// Preprocess voice triggers: fold into description, strip field from frontmatter.
|
||||
// Must run BEFORE transformFrontmatter so all hosts see the updated description,
|
||||
@@ -742,6 +795,58 @@ function processTemplate(tmplPath: string, host: Host = 'claude'): { outputPath:
|
||||
return { outputPath, content, symlinkLoop, catalogParts };
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate one on-demand section file (`<skill>/sections/<name>.md.tmpl` →
|
||||
* `<name>.md`). Sections are BODY FRAGMENTS — no frontmatter, no catalog trim,
|
||||
* no voice triggers. They resolve placeholders through the SAME path as
|
||||
* SKILL.md (resolvePlaceholders) using the PARENT skill's TemplateContext
|
||||
* (so appliesTo gating + tier behave identically — a section's {{PREAMBLE}}-
|
||||
* style resolver renders the same content it would in the parent, not empty).
|
||||
*
|
||||
* Output routing mirrors SKILL.md: Claude writes in-tree at
|
||||
* `<skill>/sections/<name>.md`; external hosts write to
|
||||
* `<hostSubdir>/skills/<externalName>/sections/<name>.md`. External hosts get
|
||||
* applyHostRewrites so cross-references resolve per host.
|
||||
*/
|
||||
function processSectionTemplate(
|
||||
sectionTmplPath: string,
|
||||
skillDir: string,
|
||||
host: Host = 'claude',
|
||||
): { outputPath: string; content: string } {
|
||||
const tmplContent = fs.readFileSync(sectionTmplPath, 'utf-8');
|
||||
const relTmplPath = path.relative(ROOT, sectionTmplPath);
|
||||
const hostConfig = getHostConfig(host);
|
||||
|
||||
// Read the owning SKILL.md.tmpl so the section inherits the parent's name +
|
||||
// tier + benefits-from (TemplateContext parity). Fall back to the dir name.
|
||||
const parentTmplPath = path.join(ROOT, skillDir, 'SKILL.md.tmpl');
|
||||
const parentContent = fs.existsSync(parentTmplPath) ? fs.readFileSync(parentTmplPath, 'utf-8') : '';
|
||||
const parentName = (parentContent && extractNameAndDescription(parentContent).name) || skillDir;
|
||||
const ctx = buildContext(parentContent || tmplContent, parentTmplPath, host, parentName);
|
||||
|
||||
// Resolve placeholders against the section body (shared guard catches stragglers).
|
||||
let content = resolvePlaceholders(tmplContent, ctx, hostConfig, relTmplPath);
|
||||
|
||||
// External hosts: rewrite cross-reference paths/tools (no frontmatter to transform).
|
||||
if (host !== 'claude') {
|
||||
content = applyHostRewrites(content, hostConfig);
|
||||
}
|
||||
|
||||
// Plain generated header (no frontmatter to insert after).
|
||||
content = GENERATED_HEADER.replace('{{SOURCE}}', path.basename(sectionTmplPath)) + content;
|
||||
|
||||
const fileName = path.basename(sectionTmplPath).replace(/\.tmpl$/, '');
|
||||
let outputPath: string;
|
||||
if (host === 'claude') {
|
||||
outputPath = path.join(ROOT, skillDir, 'sections', fileName);
|
||||
} else {
|
||||
const externalName = externalSkillName(skillDir, parentName);
|
||||
outputPath = path.join(ROOT, hostConfig.hostSubdir, 'skills', externalName, 'sections', fileName);
|
||||
}
|
||||
if (!DRY_RUN) fs.mkdirSync(path.dirname(outputPath), { recursive: true });
|
||||
return { outputPath, content };
|
||||
}
|
||||
|
||||
// ─── Main ───────────────────────────────────────────────────
|
||||
|
||||
function findTemplates(): string[] {
|
||||
@@ -833,6 +938,42 @@ for (const currentHost of hostsToRun) {
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Section generation (v2 plan T9, Claude-first carve) ───
|
||||
// On-demand sections/*.md for carved skills. Generated for CLAUDE ONLY:
|
||||
// every other host inlines section content via the {{SECTION:id}} resolver
|
||||
// (keeping the full monolith skill), so they need no section files and we
|
||||
// sidestep host-portable section paths until that plumbing lands. No-op for
|
||||
// any skill without a sections/ dir. Mirrors the SKILL.md DRY_RUN handling so
|
||||
// sections participate in the freshness gate.
|
||||
for (const sec of currentHost === 'claude' ? discoverSectionTemplates(ROOT) : []) {
|
||||
if (currentHostConfig.generation.includeSkills?.length &&
|
||||
!currentHostConfig.generation.includeSkills.includes(sec.skillDir)) continue;
|
||||
if (currentHostConfig.generation.skipSkills?.length &&
|
||||
currentHostConfig.generation.skipSkills.includes(sec.skillDir)) continue;
|
||||
|
||||
const { outputPath, content } = processSectionTemplate(path.join(ROOT, sec.tmpl), sec.skillDir, currentHost);
|
||||
const relOutput = path.relative(ROOT, outputPath);
|
||||
|
||||
if (DRY_RUN) {
|
||||
const existing = fs.existsSync(outputPath) ? fs.readFileSync(outputPath, 'utf-8') : '';
|
||||
if (existing !== content) {
|
||||
console.log(`STALE: ${relOutput}`);
|
||||
hasChanges = true;
|
||||
} else {
|
||||
console.log(`FRESH: ${relOutput}`);
|
||||
}
|
||||
} else {
|
||||
fs.writeFileSync(outputPath, content);
|
||||
console.log(`GENERATED: ${relOutput}`);
|
||||
}
|
||||
|
||||
tokenBudget.push({
|
||||
skill: relOutput,
|
||||
lines: content.split('\n').length,
|
||||
tokens: Math.round(content.length / 4),
|
||||
});
|
||||
}
|
||||
|
||||
// Generate gstack-lite and gstack-full for OpenClaw host
|
||||
if (currentHost === 'openclaw' && !DRY_RUN) {
|
||||
const openclawDir = path.join(ROOT, 'openclaw');
|
||||
@@ -959,10 +1100,14 @@ The orchestrator will persist the plan link to its own memory/knowledge store.
|
||||
}
|
||||
}
|
||||
|
||||
// --host all: report failures. Only exit(1) if claude failed.
|
||||
// --host all: any host failure fails the build. Previously only claude failures
|
||||
// exited nonzero, which let a stale or broken external-host output (e.g. a
|
||||
// section that failed to generate for Factory) slip through the freshness gate
|
||||
// silently. With sections fanned out across every host, "all hosts regenerated
|
||||
// in the same commit" is only a real gate if every host failure is fatal here.
|
||||
if (failures.length > 0 && HOST_ARG_VAL === 'all') {
|
||||
console.error(`\n${failures.length} host(s) failed: ${failures.map(f => f.host).join(', ')}`);
|
||||
if (failures.some(f => f.host === 'claude')) process.exit(1);
|
||||
process.exit(1);
|
||||
}
|
||||
// Single host dry-run failure already handled above
|
||||
|
||||
|
||||
@@ -34,6 +34,7 @@ import { generateGBrainContextLoad, generateGBrainSaveResults, generateBrainPref
|
||||
import { generateQuestionPreferenceCheck, generateQuestionLog, generateInlineTuneFeedback } from './question-tuning';
|
||||
import { generateMakePdfSetup } from './make-pdf';
|
||||
import { generateTasksSectionEmit, generateTasksSectionAggregate } from './tasks-section';
|
||||
import { SECTION, SECTION_INDEX } from './sections';
|
||||
import { generateRedactTaxonomyTable, generateRedactInvocationBlock } from './redact-doc';
|
||||
|
||||
export const RESOLVERS: Record<string, ResolverValue> = {
|
||||
@@ -98,4 +99,6 @@ export const RESOLVERS: Record<string, ResolverValue> = {
|
||||
MAKE_PDF_SETUP: generateMakePdfSetup,
|
||||
TASKS_SECTION_EMIT: generateTasksSectionEmit,
|
||||
TASKS_SECTION_AGGREGATE: generateTasksSectionAggregate,
|
||||
SECTION,
|
||||
SECTION_INDEX,
|
||||
};
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
/**
|
||||
* Section resolvers (v2 plan T9, Claude-first carve).
|
||||
*
|
||||
* A carved skill keeps its prose-heavy steps in `<skill>/sections/<id>.md`, read
|
||||
* on demand. The SAME template ships to every host, so these resolvers make the
|
||||
* carve host-aware:
|
||||
*
|
||||
* - On CLAUDE: {{SECTION:id}} emits a STOP-Read pointer to the generated section
|
||||
* file (the skeleton), and the section .md is generated + installed separately.
|
||||
* - On every OTHER host: {{SECTION:id}} INLINES the section template's content,
|
||||
* so external hosts keep the full monolith ship skill (no section files, no
|
||||
* host-portable-path problem). Inlined content keeps its own {{RESOLVER}}
|
||||
* tokens, which the generator's multi-pass resolve expands.
|
||||
*
|
||||
* {{SECTION_INDEX:skill}} renders the situation→section table from the PASSIVE
|
||||
* manifest on Claude (empty on other hosts — they have no sections). The manifest
|
||||
* is the single source of id/file/title/trigger text (CM2; v2_PLAN.md:663).
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import type { ResolverFn, TemplateContext } from './types';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..', '..');
|
||||
|
||||
interface SectionEntry {
|
||||
id: string;
|
||||
file: string;
|
||||
title: string;
|
||||
trigger: string;
|
||||
}
|
||||
interface SectionManifest {
|
||||
skill: string;
|
||||
sections: SectionEntry[];
|
||||
}
|
||||
|
||||
function loadManifest(skill: string): SectionManifest {
|
||||
const p = path.join(ROOT, skill, 'sections', 'manifest.json');
|
||||
const raw = fs.readFileSync(p, 'utf-8');
|
||||
return JSON.parse(raw) as SectionManifest;
|
||||
}
|
||||
|
||||
function findSection(skill: string, id: string): SectionEntry {
|
||||
const entry = loadManifest(skill).sections.find(s => s.id === id);
|
||||
if (!entry) {
|
||||
throw new Error(`{{SECTION:${id}}} — no section "${id}" in ${skill}/sections/manifest.json`);
|
||||
}
|
||||
return entry;
|
||||
}
|
||||
|
||||
/**
|
||||
* {{SECTION:id}} — pointer on Claude, inline on other hosts.
|
||||
* Claude path uses the stable gstack-root install (`{skillRoot}/{skill}/sections/`),
|
||||
* which always exists, instead of a naked relative path (Codex outside-voice #7).
|
||||
*/
|
||||
export const SECTION: ResolverFn = (ctx: TemplateContext, args?: string[]): string => {
|
||||
const id = args?.[0];
|
||||
if (!id) throw new Error('{{SECTION:id}} requires a section id');
|
||||
const entry = findSection(ctx.skillName, id);
|
||||
|
||||
if (ctx.host === 'claude') {
|
||||
const sectionPath = `${ctx.paths.skillRoot}/${ctx.skillName}/sections/${entry.file}`;
|
||||
return [
|
||||
`> **STOP.** Before ${entry.trigger}, Read \`${sectionPath}\` and execute it`,
|
||||
`> in full. Do not work from memory — that section is the source of truth for this step.`,
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
// Non-Claude hosts inline the section template content (monolith preserved).
|
||||
// Inner {{RESOLVER}} tokens are expanded by the generator's multi-pass resolve.
|
||||
const tmplPath = path.join(ROOT, ctx.skillName, 'sections', `${entry.file}.tmpl`);
|
||||
return fs.readFileSync(tmplPath, 'utf-8').trimEnd();
|
||||
};
|
||||
|
||||
/**
|
||||
* {{SECTION_INDEX:skill}} — situation→section table from the passive manifest.
|
||||
* Claude only; other hosts inline everything so an index would be noise.
|
||||
*/
|
||||
export const SECTION_INDEX: ResolverFn = (ctx: TemplateContext, args?: string[]): string => {
|
||||
if (ctx.host !== 'claude') return '';
|
||||
const skill = args?.[0] ?? ctx.skillName;
|
||||
const manifest = loadManifest(skill);
|
||||
const lines: string[] = [
|
||||
'## Section index — Read each section when its situation applies',
|
||||
'',
|
||||
'This skill is a decision-tree skeleton. The steps below point to on-demand',
|
||||
'sections. Read a section in full before doing its step; do not work from memory.',
|
||||
'',
|
||||
'| When | Read this section |',
|
||||
'|------|-------------------|',
|
||||
];
|
||||
for (const s of manifest.sections) {
|
||||
lines.push(`| ${s.trigger} | \`sections/${s.file}\` |`);
|
||||
}
|
||||
return lines.join('\n');
|
||||
};
|
||||
@@ -569,6 +569,14 @@ link_claude_skill_dirs() {
|
||||
# Validate target isn't a symlink before creating the link
|
||||
if [ -L "$target/SKILL.md" ]; then rm "$target/SKILL.md"; fi
|
||||
_link_or_copy "$gstack_dir/$dir_name/SKILL.md" "$target/SKILL.md"
|
||||
# Link the sections/ subdir for carved skills (v2 plan T9). The prefixed
|
||||
# Claude skill dir otherwise holds only SKILL.md, so a runtime
|
||||
# "Read sections/<name>.md" 404s. Route through _link_or_copy so Windows
|
||||
# gets a fresh copy (and re-copies on every ./setup, refreshing staleness).
|
||||
if [ -d "$gstack_dir/$dir_name/sections" ]; then
|
||||
if [ -e "$target/sections" ] || [ -L "$target/sections" ]; then rm -rf "$target/sections"; fi
|
||||
_link_or_copy "$gstack_dir/$dir_name/sections" "$target/sections"
|
||||
fi
|
||||
linked+=("$link_name")
|
||||
fi
|
||||
done
|
||||
@@ -1144,6 +1152,20 @@ if [ "$INSTALL_KIRO" -eq 1 ]; then
|
||||
-e "s|~/.codex/skills/gstack|~/.kiro/skills/gstack|g" \
|
||||
-e "s|~/.claude/skills/gstack|~/.kiro/skills/gstack|g" \
|
||||
"$skill_dir/SKILL.md" > "$target_dir/SKILL.md"
|
||||
# Carved skills (v2 plan T9): rewrite + copy each sections/*.md the same way,
|
||||
# so a runtime "Read sections/<name>.md" resolves under ~/.kiro and doesn't
|
||||
# leak a ~/.codex or ~/.claude path. Kiro builds from the codex output, so
|
||||
# these section files only exist for skills that have been carved.
|
||||
if [ -d "$skill_dir/sections" ]; then
|
||||
mkdir -p "$target_dir/sections"
|
||||
for section_file in "$skill_dir/sections"/*; do
|
||||
[ -f "$section_file" ] || continue
|
||||
sed -e 's|\$HOME/.codex/skills/gstack|$HOME/.kiro/skills/gstack|g' \
|
||||
-e "s|~/.codex/skills/gstack|~/.kiro/skills/gstack|g" \
|
||||
-e "s|~/.claude/skills/gstack|~/.kiro/skills/gstack|g" \
|
||||
"$section_file" > "$target_dir/sections/$(basename "$section_file")"
|
||||
done
|
||||
fi
|
||||
done
|
||||
echo "gstack ready (kiro)."
|
||||
echo " browse: $BROWSE_BIN"
|
||||
|
||||
+68
-1927
File diff suppressed because it is too large
Load Diff
+47
-616
@@ -71,6 +71,10 @@ Never skip a verification step because a prior `/ship` run already performed it.
|
||||
|
||||
---
|
||||
|
||||
{{SECTION_INDEX:ship}}
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Pre-flight
|
||||
|
||||
1. Check the current branch. If on the base branch or the repo's default branch, **abort**: "You're on the base branch. Ship from a feature branch."
|
||||
@@ -139,432 +143,53 @@ git fetch origin <base> && git merge origin/<base> --no-edit
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Test Framework Bootstrap
|
||||
{{SECTION:tests}}
|
||||
|
||||
{{TEST_BOOTSTRAP}}
|
||||
{{SECTION:test-coverage}}
|
||||
|
||||
---
|
||||
{{SECTION:plan-completion}}
|
||||
|
||||
## Step 5: Run tests (on merged code)
|
||||
{{SECTION:review-army}}
|
||||
|
||||
**Do NOT run `RAILS_ENV=test bin/rails db:migrate`** — `bin/test-lane` already calls
|
||||
`db:test:prepare` internally, which loads the schema into the correct lane database.
|
||||
Running bare test migrations without INSTANCE hits an orphan DB and corrupts structure.sql.
|
||||
{{SECTION:greptile}}
|
||||
|
||||
Run both test suites in parallel:
|
||||
|
||||
```bash
|
||||
bin/test-lane 2>&1 | tee /tmp/ship_tests.txt &
|
||||
npm run test 2>&1 | tee /tmp/ship_vitest.txt &
|
||||
wait
|
||||
```
|
||||
|
||||
After both complete, read the output files and check pass/fail.
|
||||
|
||||
**If any test fails:** Do NOT immediately stop. Apply the Test Failure Ownership Triage:
|
||||
|
||||
{{TEST_FAILURE_TRIAGE}}
|
||||
|
||||
**After triage:** If any in-branch failures remain unfixed, **STOP**. Do not proceed. If all failures were pre-existing and handled (fixed, TODOed, assigned, or skipped), continue to Step 6.
|
||||
|
||||
**If all pass:** Continue silently — just note the counts briefly.
|
||||
|
||||
---
|
||||
|
||||
## Step 6: Eval Suites (conditional)
|
||||
|
||||
Evals are mandatory when prompt-related files change. Skip this step entirely if no prompt files are in the diff.
|
||||
|
||||
**1. Check if the diff touches prompt-related files:**
|
||||
|
||||
```bash
|
||||
git diff origin/<base> --name-only
|
||||
```
|
||||
|
||||
Match against these patterns (from CLAUDE.md):
|
||||
- `app/services/*_prompt_builder.rb`
|
||||
- `app/services/*_generation_service.rb`, `*_writer_service.rb`, `*_designer_service.rb`
|
||||
- `app/services/*_evaluator.rb`, `*_scorer.rb`, `*_classifier_service.rb`, `*_analyzer.rb`
|
||||
- `app/services/concerns/*voice*.rb`, `*writing*.rb`, `*prompt*.rb`, `*token*.rb`
|
||||
- `app/services/chat_tools/*.rb`, `app/services/x_thread_tools/*.rb`
|
||||
- `config/system_prompts/*.txt`
|
||||
- `test/evals/**/*` (eval infrastructure changes affect all suites)
|
||||
|
||||
**If no matches:** Print "No prompt-related files changed — skipping evals." and continue to Step 9.
|
||||
|
||||
**2. Identify affected eval suites:**
|
||||
|
||||
Each eval runner (`test/evals/*_eval_runner.rb`) declares `PROMPT_SOURCE_FILES` listing which source files affect it. Grep these to find which suites match the changed files:
|
||||
|
||||
```bash
|
||||
grep -l "changed_file_basename" test/evals/*_eval_runner.rb
|
||||
```
|
||||
|
||||
Map runner → test file: `post_generation_eval_runner.rb` → `post_generation_eval_test.rb`.
|
||||
|
||||
**Special cases:**
|
||||
- Changes to `test/evals/judges/*.rb`, `test/evals/support/*.rb`, or `test/evals/fixtures/` affect ALL suites that use those judges/support files. Check imports in the eval test files to determine which.
|
||||
- Changes to `config/system_prompts/*.txt` — grep eval runners for the prompt filename to find affected suites.
|
||||
- If unsure which suites are affected, run ALL suites that could plausibly be impacted. Over-testing is better than missing a regression.
|
||||
|
||||
**3. Run affected suites at `EVAL_JUDGE_TIER=full`:**
|
||||
|
||||
`/ship` is a pre-merge gate, so always use full tier (Sonnet structural + Opus persona judges).
|
||||
|
||||
```bash
|
||||
EVAL_JUDGE_TIER=full EVAL_VERBOSE=1 bin/test-lane --eval test/evals/<suite>_eval_test.rb 2>&1 | tee /tmp/ship_evals.txt
|
||||
```
|
||||
|
||||
If multiple suites need to run, run them sequentially (each needs a test lane). If the first suite fails, stop immediately — don't burn API cost on remaining suites.
|
||||
|
||||
**4. Check results:**
|
||||
|
||||
- **If any eval fails:** Show the failures, the cost dashboard, and **STOP**. Do not proceed.
|
||||
- **If all pass:** Note pass counts and cost. Continue to Step 9.
|
||||
|
||||
**5. Save eval output** — include eval results and cost dashboard in the PR body (Step 19).
|
||||
|
||||
**Tier reference (for context — /ship always uses `full`):**
|
||||
| Tier | When | Speed (cached) | Cost |
|
||||
|------|------|----------------|------|
|
||||
| `fast` (Haiku) | Dev iteration, smoke tests | ~5s (14x faster) | ~$0.07/run |
|
||||
| `standard` (Sonnet) | Default dev, `bin/test-lane --eval` | ~17s (4x faster) | ~$0.37/run |
|
||||
| `full` (Opus persona) | **`/ship` and pre-merge** | ~72s (baseline) | ~$1.27/run |
|
||||
|
||||
---
|
||||
|
||||
## Step 7: Test Coverage Audit
|
||||
|
||||
**Dispatch this step as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent runs the coverage audit in a fresh context window — the parent only sees the conclusion, not intermediate file reads. This is context-rot defense.
|
||||
|
||||
**Subagent prompt:** Pass the following instructions to the subagent, with `<base>` substituted with the base branch:
|
||||
|
||||
> You are running a ship-workflow test coverage audit. Run `git diff <base>...HEAD` as needed. Do not commit or push — report only.
|
||||
>
|
||||
> {{TEST_COVERAGE_AUDIT_SHIP}}
|
||||
>
|
||||
> After your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):
|
||||
> `{"coverage_pct":N,"gaps":N,"diagram":"<full markdown coverage diagram for PR body>","tests_added":["path",...]}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
1. Read the subagent's final output. Parse the LAST line as JSON.
|
||||
2. Store `coverage_pct` (for Step 20 metrics), `gaps` (user summary), `tests_added` (for the commit).
|
||||
3. Embed `diagram` verbatim in the PR body's `## Test Coverage` section (Step 19).
|
||||
4. Print a one-line summary: `Coverage: {coverage_pct}%, {gaps} gaps. {tests_added.length} tests added.`
|
||||
|
||||
**If the subagent fails, times out, or returns invalid JSON:** Fall back to running the audit inline in the parent. Do not block /ship on subagent failure — partial results are better than none.
|
||||
|
||||
---
|
||||
|
||||
## Step 8: Plan Completion Audit
|
||||
|
||||
**Dispatch this step as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent reads the plan file and every referenced code file in its own fresh context. Parent gets only the conclusion.
|
||||
|
||||
**Subagent prompt:** Pass these instructions to the subagent:
|
||||
|
||||
> You are running a ship-workflow plan completion audit. The base branch is `<base>`. Use `git diff <base>...HEAD` to see what shipped. Do not commit or push — report only.
|
||||
>
|
||||
> {{PLAN_COMPLETION_AUDIT_SHIP}}
|
||||
>
|
||||
> After your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):
|
||||
> `{"total_items":N,"done":N,"changed":N,"deferred":N,"unverifiable":N,"summary":"<markdown checklist for PR body>"}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
1. Parse the LAST line of the subagent's output as JSON.
|
||||
2. Store `done`, `deferred`, `unverifiable` for Step 20 metrics; use `summary` in PR body.
|
||||
3. If `deferred > 0` or `unverifiable > 0` and no user override, present the items via the appropriate AskUserQuestion (see Gate Logic priority order above) before continuing.
|
||||
4. Embed `summary` in PR body's `## Plan Completion` section (Step 19). If `unverifiable > 0` and the user picked option A in the UNVERIFIABLE gate, also embed `## Plan Completion — Manual Verifications` listing each user-confirmed item.
|
||||
|
||||
**If the subagent fails or returns invalid JSON:** Fall back to running the audit inline (parent processes the same plan-extraction + classification logic). If the inline fallback also fails (e.g., plan file unreadable, parser error), do NOT silently pass — surface the failure as an explicit AskUserQuestion: "Plan Completion audit could not run ({reason}). Options: (A) Skip audit and ship anyway — record that the audit was skipped in PR body and Step 20 metrics; (B) Stop and fix the audit." Default and recommended option is (B). Silent fail-open is the failure shape that VAS-449 surfaced.
|
||||
|
||||
---
|
||||
|
||||
{{PLAN_VERIFICATION_EXEC}}
|
||||
|
||||
{{LEARNINGS_SEARCH:query=release ship version changelog merge pr}}
|
||||
|
||||
{{SCOPE_DRIFT}}
|
||||
|
||||
---
|
||||
|
||||
## Step 9: Pre-Landing Review
|
||||
|
||||
Review the diff for structural issues that tests don't catch.
|
||||
|
||||
1. Read `.claude/skills/review/checklist.md`. If the file cannot be read, **STOP** and report the error.
|
||||
|
||||
2. Run `git diff origin/<base>` to get the full diff (scoped to feature changes against the freshly-fetched base branch).
|
||||
|
||||
3. Apply the review checklist in two passes:
|
||||
- **Pass 1 (CRITICAL):** SQL & Data Safety, LLM Output Trust Boundary
|
||||
- **Pass 2 (INFORMATIONAL):** All remaining categories
|
||||
|
||||
{{CONFIDENCE_CALIBRATION}}
|
||||
|
||||
{{DESIGN_REVIEW_LITE}}
|
||||
|
||||
Include any design findings alongside the code review findings. They follow the same Fix-First flow below.
|
||||
|
||||
{{REVIEW_ARMY}}
|
||||
|
||||
{{CROSS_REVIEW_DEDUP}}
|
||||
|
||||
4. **Classify each finding from both the checklist pass and specialist review (Step 9.1-Step 9.2) as AUTO-FIX or ASK** per the Fix-First Heuristic in
|
||||
checklist.md. Critical findings lean toward ASK; informational lean toward AUTO-FIX.
|
||||
|
||||
5. **Auto-fix all AUTO-FIX items.** Apply each fix. Output one line per fix:
|
||||
`[AUTO-FIXED] [file:line] Problem → what you did`
|
||||
|
||||
6. **If ASK items remain,** present them in ONE AskUserQuestion:
|
||||
- List each with number, severity, problem, recommended fix
|
||||
- Per-item options: A) Fix B) Skip
|
||||
- Overall RECOMMENDATION
|
||||
- If 3 or fewer ASK items, you may use individual AskUserQuestion calls instead
|
||||
|
||||
7. **After all fixes (auto + user-approved):**
|
||||
- If ANY fixes were applied: commit fixed files by name (`git add <fixed-files> && git commit -m "fix: pre-landing review fixes"`), then **STOP** and tell the user to run `/ship` again to re-test.
|
||||
- If no fixes applied (all ASK items skipped, or no issues found): continue to Step 12.
|
||||
|
||||
8. Output summary: `Pre-Landing Review: N issues — M auto-fixed, K asked (J fixed, L skipped)`
|
||||
|
||||
If no issues found: `Pre-Landing Review: No issues found.`
|
||||
|
||||
9. Persist the review result to the review log:
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-review-log '{"skill":"review","timestamp":"TIMESTAMP","status":"STATUS","issues_found":N,"critical":N,"informational":N,"quality_score":SCORE,"specialists":SPECIALISTS_JSON,"findings":FINDINGS_JSON,"commit":"'"$(git rev-parse --short HEAD)"'","via":"ship"}'
|
||||
```
|
||||
Substitute TIMESTAMP (ISO 8601), STATUS ("clean" if no issues, "issues_found" otherwise),
|
||||
and N values from the summary counts above. The `via:"ship"` distinguishes from standalone `/review` runs.
|
||||
- `quality_score` = the PR Quality Score computed in Step 9.2 (e.g., 7.5). If specialists were skipped (small diff), use `10.0`
|
||||
- `specialists` = the per-specialist stats object compiled in Step 9.2. Each specialist that was considered gets an entry: `{"dispatched":true/false,"findings":N,"critical":N,"informational":N}` if dispatched, or `{"dispatched":false,"reason":"scope|gated"}` if skipped. Example: `{"testing":{"dispatched":true,"findings":2,"critical":0,"informational":2},"security":{"dispatched":false,"reason":"scope"}}`
|
||||
- `findings` = array of per-finding records. For each finding (from checklist pass and specialists), include: `{"fingerprint":"path:line:category","severity":"CRITICAL|INFORMATIONAL","action":"ACTION"}`. ACTION is `"auto-fixed"`, `"fixed"` (user approved), or `"skipped"` (user chose Skip).
|
||||
|
||||
Save the review output — it goes into the PR body in Step 19.
|
||||
|
||||
---
|
||||
|
||||
## Step 10: Address Greptile review comments (if PR exists)
|
||||
|
||||
**Dispatch the fetch + classification as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent pulls every Greptile comment, runs the escalation detection algorithm, and classifies each comment. Parent receives a structured list and handles user interaction + file edits.
|
||||
|
||||
**Subagent prompt:**
|
||||
|
||||
> You are classifying Greptile review comments for a /ship workflow. Read `.claude/skills/review/greptile-triage.md` and follow the fetch, filter, classify, and **escalation detection** steps. Do NOT fix code, do NOT reply to comments, do NOT commit — report only.
|
||||
>
|
||||
> For each comment, assign: `classification` (`valid_actionable`, `already_fixed`, `false_positive`, `suppressed`), `escalation_tier` (1 or 2), the file:line or [top-level] tag, body summary, and permalink URL.
|
||||
>
|
||||
> If no PR exists, `gh` fails, the API errors, or there are zero comments, output: `{"total":0,"comments":[]}` and stop.
|
||||
>
|
||||
> Otherwise, output a single JSON object on the LAST LINE of your response:
|
||||
> `{"total":N,"comments":[{"classification":"...","escalation_tier":N,"ref":"file:line","summary":"...","permalink":"url"},...]}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
Parse the LAST line as JSON.
|
||||
|
||||
If `total` is 0, skip this step silently. Continue to Step 12.
|
||||
|
||||
Otherwise, print: `+ {total} Greptile comments ({valid_actionable} valid, {already_fixed} already fixed, {false_positive} FP)`.
|
||||
|
||||
For each comment in `comments`:
|
||||
|
||||
**VALID & ACTIONABLE:** Use AskUserQuestion with:
|
||||
- The comment (file:line or [top-level] + body summary + permalink URL)
|
||||
- `RECOMMENDATION: Choose A because [one-line reason]`
|
||||
- Options: A) Fix now, B) Acknowledge and ship anyway, C) It's a false positive
|
||||
- If user chooses A: apply the fix, commit the fixed files (`git add <fixed-files> && git commit -m "fix: address Greptile review — <brief description>"`), reply using the **Fix reply template** from greptile-triage.md (include inline diff + explanation), and save to both per-project and global greptile-history (type: fix).
|
||||
- If user chooses C: reply using the **False Positive reply template** from greptile-triage.md (include evidence + suggested re-rank), save to both per-project and global greptile-history (type: fp).
|
||||
|
||||
**VALID BUT ALREADY FIXED:** Reply using the **Already Fixed reply template** from greptile-triage.md — no AskUserQuestion needed:
|
||||
- Include what was done and the fixing commit SHA
|
||||
- Save to both per-project and global greptile-history (type: already-fixed)
|
||||
|
||||
**FALSE POSITIVE:** Use AskUserQuestion:
|
||||
- Show the comment and why you think it's wrong (file:line or [top-level] + body summary + permalink URL)
|
||||
- Options:
|
||||
- A) Reply to Greptile explaining the false positive (recommended if clearly wrong)
|
||||
- B) Fix it anyway (if trivial)
|
||||
- C) Ignore silently
|
||||
- If user chooses A: reply using the **False Positive reply template** from greptile-triage.md (include evidence + suggested re-rank), save to both per-project and global greptile-history (type: fp)
|
||||
|
||||
**SUPPRESSED:** Skip silently — these are known false positives from previous triage.
|
||||
|
||||
**After all comments are resolved:** If any fixes were applied, the tests from Step 5 are now stale. **Re-run tests** (Step 5) before continuing to Step 12. If no fixes were applied, continue to Step 12.
|
||||
|
||||
---
|
||||
|
||||
{{ADVERSARIAL_STEP}}
|
||||
|
||||
{{LEARNINGS_LOG}}
|
||||
|
||||
{{GBRAIN_SAVE_RESULTS}}
|
||||
|
||||
### Refresh learnings for the headline feature on this branch
|
||||
|
||||
The top-of-skill learnings pull was keyed to "release ship" broadly. Before the VERSION/CHANGELOG step, re-pull learnings keyed to THIS branch's headline feature so any prior version-bump or CHANGELOG pitfalls for similar features surface.
|
||||
|
||||
Pick ONE keyword that names the headline feature you're shipping. The keyword should be a noun: the primary skill or module name, the central feature noun, or the binary you changed. The keyword MUST be alphanumeric or hyphen only — no quotes, slashes, dots, colons, or whitespace. If your candidate has any of those, simplify to just the alphanumeric stem.
|
||||
|
||||
Worked examples (ship-specific): good keywords are `learnings-search`, `pacing`, `worktree-ship`. Bad: `the branch headline`, `v1.31.1.0`, `feat: token-or search`.
|
||||
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-learnings-search --query "<your-keyword>" --limit 5 2>/dev/null || true
|
||||
```
|
||||
|
||||
If any learnings come back, name which one applies to the version bump or CHANGELOG framing in one sentence. If none come back, continue without reference — the absence is itself useful information.
|
||||
{{SECTION:adversarial}}
|
||||
|
||||
## Step 12: Version bump (auto-decide)
|
||||
|
||||
**Idempotency check:** Before bumping, classify the state by comparing `VERSION` against the base branch AND against `package.json`'s `version` field. Four states: FRESH (do bump), ALREADY_BUMPED (skip bump), DRIFT_STALE_PKG (sync pkg only, no re-bump), DRIFT_UNEXPECTED (stop and ask).
|
||||
|
||||
```bash
|
||||
if ! git rev-parse --verify origin/<base> >/dev/null 2>&1; then
|
||||
echo "ERROR: Unable to resolve origin/<base>. Run 'git fetch origin' or verify the base branch exists."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
BASE_VERSION=$(git show origin/<base>:VERSION 2>/dev/null | tr -d '\r\n[:space:]' || echo "0.0.0.0")
|
||||
CURRENT_VERSION=$(cat VERSION 2>/dev/null | tr -d '\r\n[:space:]' || echo "0.0.0.0")
|
||||
[ -z "$BASE_VERSION" ] && BASE_VERSION="0.0.0.0"
|
||||
[ -z "$CURRENT_VERSION" ] && CURRENT_VERSION="0.0.0.0"
|
||||
PKG_VERSION=""
|
||||
PKG_EXISTS=0
|
||||
if [ -f package.json ]; then
|
||||
PKG_EXISTS=1
|
||||
if command -v node >/dev/null 2>&1; then
|
||||
PKG_VERSION=$(node -e 'const p=require("./package.json");process.stdout.write(p.version||"")' 2>/dev/null)
|
||||
PARSE_EXIT=$?
|
||||
elif command -v bun >/dev/null 2>&1; then
|
||||
PKG_VERSION=$(bun -e 'const p=require("./package.json");process.stdout.write(p.version||"")' 2>/dev/null)
|
||||
PARSE_EXIT=$?
|
||||
else
|
||||
echo "ERROR: package.json exists but neither node nor bun is available. Install one and re-run."
|
||||
exit 1
|
||||
fi
|
||||
if [ "$PARSE_EXIT" != "0" ]; then
|
||||
echo "ERROR: package.json is not valid JSON. Fix the file before re-running /ship."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
echo "BASE: $BASE_VERSION VERSION: $CURRENT_VERSION package.json: ${PKG_VERSION:-<none>}"
|
||||
|
||||
if [ "$CURRENT_VERSION" = "$BASE_VERSION" ]; then
|
||||
if [ "$PKG_EXISTS" = "1" ] && [ -n "$PKG_VERSION" ] && [ "$PKG_VERSION" != "$CURRENT_VERSION" ]; then
|
||||
echo "STATE: DRIFT_UNEXPECTED"
|
||||
echo "package.json version ($PKG_VERSION) disagrees with VERSION ($CURRENT_VERSION) while VERSION matches base."
|
||||
echo "This looks like a manual edit to package.json bypassing /ship. Reconcile manually, then re-run."
|
||||
exit 1
|
||||
fi
|
||||
echo "STATE: FRESH"
|
||||
else
|
||||
if [ "$PKG_EXISTS" = "1" ] && [ -n "$PKG_VERSION" ] && [ "$PKG_VERSION" != "$CURRENT_VERSION" ]; then
|
||||
echo "STATE: DRIFT_STALE_PKG"
|
||||
else
|
||||
echo "STATE: ALREADY_BUMPED"
|
||||
fi
|
||||
fi
|
||||
```
|
||||
|
||||
Read the `STATE:` line and dispatch:
|
||||
|
||||
- **FRESH** → proceed with the bump action below (steps 1–4).
|
||||
- **ALREADY_BUMPED** → skip the bump by default, BUT check for queue drift first: call `bin/gstack-next-version` with the implied bump level (derived from `CURRENT_VERSION` vs `BASE_VERSION`), compare its `.version` against `CURRENT_VERSION`. If they differ (queue moved since last ship), use **AskUserQuestion**: "VERSION drift detected: you claim v<CURRENT> but next available is v<NEW> (queue moved). A) Rebump to v<NEW> and rewrite CHANGELOG header + PR title (recommended), B) Keep v<CURRENT> — will be rejected by CI version-gate until resolved." If A, treat this as FRESH with `NEW_VERSION=<new>` and run steps 1-4 (which will also trigger Step 13 CHANGELOG header rewrite and Step 19 PR title rewrite). If B, reuse `CURRENT_VERSION` and warn that CI will likely reject. If util is offline, warn and reuse `CURRENT_VERSION`.
|
||||
- **DRIFT_STALE_PKG** → a prior `/ship` bumped `VERSION` but failed to update `package.json`. Run the sync-only repair block below (after step 4). Do NOT re-bump. Reuse `CURRENT_VERSION` for CHANGELOG and PR body. (Queue check still runs in ALREADY_BUMPED terms after repair.)
|
||||
- **DRIFT_UNEXPECTED** → `/ship` has halted (exit 1). Resolve manually; /ship cannot tell which file is authoritative.
|
||||
|
||||
1. Read the current `VERSION` file (4-digit format: `MAJOR.MINOR.PATCH.MICRO`)
|
||||
|
||||
2. **Auto-decide the bump level based on the diff:**
|
||||
- Count lines changed (`git diff origin/<base>...HEAD --stat | tail -1`)
|
||||
- Check for feature signals: new route/page files (e.g. `app/*/page.tsx`, `pages/*.ts`), new DB migration/schema files, new test files alongside new source files, or branch name starting with `feat/`
|
||||
- **MICRO** (4th digit): < 50 lines changed, trivial tweaks, typos, config
|
||||
- **PATCH** (3rd digit): 50+ lines changed, no feature signals detected
|
||||
- **MINOR** (2nd digit): **ASK the user** if ANY feature signal is detected, OR 500+ lines changed, OR new modules/packages added
|
||||
- **MAJOR** (1st digit): **ASK the user** — only for milestones or breaking changes
|
||||
|
||||
Save the chosen level as `BUMP_LEVEL` (one of `major`, `minor`, `patch`, `micro`). This is the user-intended level. The next step decides *placement* — the level stays the same even if queue-aware allocation has to advance past a claimed slot.
|
||||
|
||||
3. **Queue-aware version pick (workspace-aware ship, v1.6.4.0+).** Call `bin/gstack-next-version` to see what's already claimed by open PRs + active sibling Conductor worktrees, then render the queue state to the user:
|
||||
The deterministic version-state logic is the tested **`gstack-version-bump`** CLI
|
||||
(classify / write / repair). The bump-LEVEL decision and queue-collision handling
|
||||
stay agent judgment; the slot pick stays `gstack-next-version`.
|
||||
|
||||
1. **Classify state** — pure reader, never writes:
|
||||
```bash
|
||||
QUEUE_JSON=$(bun run bin/gstack-next-version \
|
||||
--base <base> \
|
||||
--bump "$BUMP_LEVEL" \
|
||||
--current-version "$BASE_VERSION" 2>/dev/null || echo '{"offline":true}')
|
||||
NEW_VERSION=$(echo "$QUEUE_JSON" | jq -r '.version // empty')
|
||||
CLAIMED_COUNT=$(echo "$QUEUE_JSON" | jq -r '.claimed | length')
|
||||
ACTIVE_SIBLING_COUNT=$(echo "$QUEUE_JSON" | jq -r '.active_siblings | length')
|
||||
OFFLINE=$(echo "$QUEUE_JSON" | jq -r '.offline // false')
|
||||
REASON=$(echo "$QUEUE_JSON" | jq -r '.reason // ""')
|
||||
bun run ~/.claude/skills/gstack/bin/gstack-version-bump classify --base <base>
|
||||
```
|
||||
Read the JSON `state` and dispatch:
|
||||
- **FRESH** → do the bump (steps 2-4).
|
||||
- **ALREADY_BUMPED** → skip the bump, but run the queue-drift check (step 3) with the reported `currentVersion`. If the queue moved (next free version differs), **AskUserQuestion**: rebump to the new version (rewrites CHANGELOG header + PR title) or keep current (CI version-gate will reject until resolved).
|
||||
- **DRIFT_STALE_PKG** → run `gstack-version-bump repair` (syncs package.json to VERSION). No re-bump; reuse `currentVersion` for CHANGELOG + PR.
|
||||
- **DRIFT_UNEXPECTED** → **STOP**. package.json disagrees with VERSION while VERSION matches base — a manual edit bypassed /ship. Reconcile manually, then re-run.
|
||||
|
||||
- If `OFFLINE=true` or the util fails (auth expired, no `gh`/`glab`, network): fall back to local `BUMP_LEVEL` arithmetic (bump `BASE_VERSION` at the chosen level). Print `⚠ workspace-aware ship offline — using local bump only`. Continue.
|
||||
- If `CLAIMED_COUNT > 0`: render the queue table to the user so they can see landing order at a glance:
|
||||
```
|
||||
Queue on <base> (vBASE_VERSION):
|
||||
#<pr> <branch> → v<version> [⚠ collision with #<other>]
|
||||
Active sibling workspaces (WIP, not yet PR'd):
|
||||
<path> → v<version> (committed Nh ago)
|
||||
Your branch will claim: vNEW_VERSION (<reason>)
|
||||
```
|
||||
- If `ACTIVE_SIBLING_COUNT > 0` and any active sibling's VERSION is `>= NEW_VERSION`, use **AskUserQuestion**: "Sibling workspace <path> has v<X> committed <N>h ago but hasn't PR'd yet. Wait for them to ship first, or advance past? A) Advance past (recommended for unrelated work), B) Abort /ship and sync up with sibling first."
|
||||
- Validate `NEW_VERSION` matches `MAJOR.MINOR.PATCH.MICRO`. If util returns an empty or malformed version, fall back to local bump.
|
||||
2. **Decide the bump level** from the diff (agent judgment):
|
||||
- **MICRO**: <50 lines, trivial tweaks/config. **PATCH**: 50+ lines, no feature signals.
|
||||
- **MINOR**: **ASK** if any feature signal (new route/page, migration, new module), OR 500+ lines. **MAJOR**: **ASK** — milestones or breaking changes only.
|
||||
Save as `BUMP_LEVEL`. The level is the user-intended bump; queue-aware placement may advance the slot without changing the level.
|
||||
|
||||
4. **Validate** `NEW_VERSION` and write it to **both** `VERSION` and `package.json`. This block runs only when `STATE: FRESH`.
|
||||
3. **Queue-aware pick** (workspace-aware ship):
|
||||
```bash
|
||||
QUEUE_JSON=$(bun run ~/.claude/skills/gstack/bin/gstack-next-version --base <base> --bump "$BUMP_LEVEL" --current-version "$BASE_VERSION" 2>/dev/null || echo '{"offline":true}')
|
||||
NEW_VERSION=$(echo "$QUEUE_JSON" | jq -r '.version // empty')
|
||||
```
|
||||
If `offline`/util fails: fall back to local `BUMP_LEVEL` arithmetic and print `⚠ workspace-aware ship offline — using local bump only`. If `claimed` is non-empty, render the queue table so the user sees landing order. If an active sibling workspace holds a version `>= NEW_VERSION`, **AskUserQuestion**: advance past (unrelated work) or abort and sync with the sibling.
|
||||
|
||||
```bash
|
||||
if ! printf '%s' "$NEW_VERSION" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'; then
|
||||
echo "ERROR: NEW_VERSION ($NEW_VERSION) does not match MAJOR.MINOR.PATCH.MICRO pattern. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
echo "$NEW_VERSION" > VERSION
|
||||
if [ -f package.json ]; then
|
||||
if command -v node >/dev/null 2>&1; then
|
||||
node -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$NEW_VERSION" || {
|
||||
echo "ERROR: failed to update package.json. VERSION was written but package.json is now stale. Fix and re-run — the new idempotency check will detect the drift."
|
||||
exit 1
|
||||
}
|
||||
elif command -v bun >/dev/null 2>&1; then
|
||||
bun -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$NEW_VERSION" || {
|
||||
echo "ERROR: failed to update package.json. VERSION was written but package.json is now stale."
|
||||
exit 1
|
||||
}
|
||||
else
|
||||
echo "ERROR: package.json exists but neither node nor bun is available."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
```
|
||||
4. **Write the bump** (FRESH, or an approved rebump):
|
||||
```bash
|
||||
bun run ~/.claude/skills/gstack/bin/gstack-version-bump write --version "$NEW_VERSION"
|
||||
```
|
||||
The CLI validates the 4-digit `MAJOR.MINOR.PATCH.MICRO` pattern and writes **both** VERSION and package.json. On a half-write (VERSION written, package.json failed) it exits 3 — re-run, and classify will report DRIFT_STALE_PKG for `repair` to fix.
|
||||
|
||||
**DRIFT_STALE_PKG repair path** — runs when idempotency reports `STATE: DRIFT_STALE_PKG`. No re-bump; sync `package.json.version` to the current `VERSION` and continue. Reuse `CURRENT_VERSION` for CHANGELOG and PR body.
|
||||
|
||||
```bash
|
||||
REPAIR_VERSION=$(cat VERSION | tr -d '\r\n[:space:]')
|
||||
if ! printf '%s' "$REPAIR_VERSION" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'; then
|
||||
echo "ERROR: VERSION file contents ($REPAIR_VERSION) do not match MAJOR.MINOR.PATCH.MICRO pattern. Refusing to propagate invalid semver into package.json. Fix VERSION manually, then re-run /ship."
|
||||
exit 1
|
||||
fi
|
||||
if command -v node >/dev/null 2>&1; then
|
||||
node -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$REPAIR_VERSION" || {
|
||||
echo "ERROR: drift repair failed — could not update package.json."
|
||||
exit 1
|
||||
}
|
||||
else
|
||||
bun -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$REPAIR_VERSION" || {
|
||||
echo "ERROR: drift repair failed."
|
||||
exit 1
|
||||
}
|
||||
fi
|
||||
echo "Drift repaired: package.json synced to $REPAIR_VERSION. No version bump performed."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
{{CHANGELOG_WORKFLOW}}
|
||||
|
||||
---
|
||||
{{SECTION:changelog}}
|
||||
|
||||
## Step 14: TODOS.md (auto-update)
|
||||
|
||||
@@ -770,211 +395,7 @@ git push -u origin <branch-name>
|
||||
|
||||
---
|
||||
|
||||
## Step 18: Documentation sync (via subagent, before PR creation)
|
||||
|
||||
**Dispatch /document-release as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent gets a fresh context window — zero rot from the preceding 17 steps. It also runs the **full** `/document-release` workflow (with CHANGELOG clobber protection, doc exclusions, risky-change gates, named staging, race-safe PR body editing) rather than a weaker reimplementation.
|
||||
|
||||
**Sequencing:** This step runs AFTER Step 17 (Push) and BEFORE Step 19 (Create PR). The PR is created once from final HEAD with the `## Documentation` section baked into the initial body. No create-then-re-edit dance.
|
||||
|
||||
**Subagent prompt:**
|
||||
|
||||
> You are executing the /document-release workflow after a code push. Read the full skill file `${HOME}/.claude/skills/gstack/document-release/SKILL.md` and execute its complete workflow end-to-end, including CHANGELOG clobber protection, doc exclusions, risky-change gates, and named staging. Do NOT attempt to edit the PR body — no PR exists yet. Branch: `<branch>`, base: `<base>`.
|
||||
>
|
||||
> After completing the workflow, output a single JSON object on the LAST LINE of your response (no other text after it):
|
||||
> `{"files_updated":["README.md","CLAUDE.md",...],"commit_sha":"abc1234","pushed":true,"documentation_section":"<markdown block for PR body's ## Documentation section>"}`
|
||||
>
|
||||
> If no documentation files needed updating, output:
|
||||
> `{"files_updated":[],"commit_sha":null,"pushed":false,"documentation_section":null}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
1. Parse the LAST line of the subagent's output as JSON.
|
||||
2. Store `documentation_section` — Step 19 embeds it in the PR body (or omits the section if null).
|
||||
3. If `files_updated` is non-empty, print: `Documentation synced: {files_updated.length} files updated, committed as {commit_sha}`.
|
||||
4. If `files_updated` is empty, print: `Documentation is current — no updates needed.`
|
||||
|
||||
**If the subagent fails or returns invalid JSON:** Print a warning and proceed to Step 19 without a `## Documentation` section. Do not block /ship on subagent failure. The user can run `/document-release` manually after the PR lands.
|
||||
|
||||
---
|
||||
|
||||
## Step 19: Create PR/MR
|
||||
|
||||
**Idempotency check:** Check if a PR/MR already exists for this branch.
|
||||
|
||||
**If GitHub:**
|
||||
```bash
|
||||
gh pr view --json url,number,state -q 'if .state == "OPEN" then "PR #\(.number): \(.url)" else "NO_PR" end' 2>/dev/null || echo "NO_PR"
|
||||
```
|
||||
|
||||
**If GitLab:**
|
||||
```bash
|
||||
glab mr view -F json 2>/dev/null | jq -r 'if .state == "opened" then "MR_EXISTS" else "NO_MR" end' 2>/dev/null || echo "NO_MR"
|
||||
```
|
||||
|
||||
If an **open** PR/MR already exists: **update** the PR body using `gh pr edit --body-file "$PR_BODY_FILE"` (GitHub) or `glab mr update -d ...` (GitLab). Always regenerate the PR body from scratch using this run's fresh results (test output, coverage audit, review findings, adversarial review, TODOS summary, documentation_section from Step 18). Never reuse stale PR body content from a prior run. **Run the same redaction scan-at-sink (PR body + title) as the create path (Step 19) before editing — scan the temp file, then `gh pr edit --body-file` from it.**
|
||||
|
||||
**Always update the PR title to start with `v$NEW_VERSION`.** PR titles use the workspace-aware format `v<NEW_VERSION> <type>: <summary>` — version ALWAYS first, no exceptions, no "custom title kept intentionally" escape hatch. The shared helper `bin/gstack-pr-title-rewrite.sh` is the single source of truth for the rule.
|
||||
|
||||
1. Read the current title: `CURRENT=$(gh pr view --json title -q .title)` (or `glab mr view -F json | jq -r .title`).
|
||||
2. Compute the corrected title: `NEW_TITLE=$(~/.claude/skills/gstack/bin/gstack-pr-title-rewrite.sh "$NEW_VERSION" "$CURRENT")`. The helper handles three cases: title already correct (no-op), title has a different `v<X.Y.Z.W>` prefix (replace it), or title has no version prefix (prepend one).
|
||||
3. If `NEW_TITLE` differs from `CURRENT`, run `gh pr edit --title "$NEW_TITLE"` (or `glab mr update -t "$NEW_TITLE"`).
|
||||
4. **Self-check:** re-fetch the title and assert it starts with `v$NEW_VERSION `. If it does not, retry the edit once. If still wrong, surface the failure to the user.
|
||||
|
||||
This keeps the title truthful when Step 12's queue-drift detection rebumps a stale version, and forces the format on PRs that were created without it.
|
||||
|
||||
Print the existing URL and continue to Step 20.
|
||||
|
||||
If no PR/MR exists: create a pull request (GitHub) or merge request (GitLab) using the platform detected in Step 0.
|
||||
|
||||
The PR/MR body should contain these sections:
|
||||
|
||||
```
|
||||
## Summary
|
||||
<Summarize ALL changes being shipped. Run `git log <base>..HEAD --oneline` to enumerate
|
||||
every commit. Exclude the VERSION/CHANGELOG metadata commit (that's this PR's bookkeeping,
|
||||
not a substantive change). Group the remaining commits into logical sections (e.g.,
|
||||
"**Performance**", "**Dead Code Removal**", "**Infrastructure**"). Every substantive commit
|
||||
must appear in at least one section. If a commit's work isn't reflected in the summary,
|
||||
you missed it.>
|
||||
|
||||
## Test Coverage
|
||||
<coverage diagram from Step 7, or "All new code paths have test coverage.">
|
||||
<If Step 7 ran: "Tests: {before} → {after} (+{delta} new)">
|
||||
|
||||
## Pre-Landing Review
|
||||
<findings from Step 9 code review, or "No issues found.">
|
||||
|
||||
## Design Review
|
||||
<If design review ran: "Design Review (lite): N findings — M auto-fixed, K skipped. AI Slop: clean/N issues.">
|
||||
<If no frontend files changed: "No frontend files changed — design review skipped.">
|
||||
|
||||
## Eval Results
|
||||
<If evals ran: suite names, pass/fail counts, cost dashboard summary. If skipped: "No prompt-related files changed — evals skipped.">
|
||||
|
||||
## Greptile Review
|
||||
<If Greptile comments were found: bullet list with [FIXED] / [FALSE POSITIVE] / [ALREADY FIXED] tag + one-line summary per comment>
|
||||
<If no Greptile comments found: "No Greptile comments.">
|
||||
<If no PR existed during Step 10: omit this section entirely>
|
||||
|
||||
## Scope Drift
|
||||
<If scope drift ran: "Scope Check: CLEAN" or list of drift/creep findings>
|
||||
<If no scope drift: omit this section>
|
||||
|
||||
## Plan Completion
|
||||
<If plan file found: completion checklist summary from Step 8>
|
||||
<If no plan file: "No plan file detected.">
|
||||
<If plan items deferred: list deferred items>
|
||||
|
||||
## Linked Spec
|
||||
<Auto-detect: look for /spec archives matching this branch via:
|
||||
eval "$(${ctx.paths.binDir}/gstack-paths)"
|
||||
eval "$(${ctx.paths.binDir}/gstack-slug)"
|
||||
CURRENT_BRANCH=$(git branch --show-current)
|
||||
SPEC_ARCHIVES="$GSTACK_STATE_ROOT/projects/$SLUG/specs"
|
||||
# Find newest archive whose spec_branch frontmatter matches current branch (or one of its
|
||||
# parents — if spec spawned worktree spec/<slug>-$$, the spawned worktree IS where /ship runs).
|
||||
SPEC_FILE=$(grep -l "^spec_branch: $CURRENT_BRANCH$" "$SPEC_ARCHIVES"/*.md 2>/dev/null | head -1)
|
||||
[ -z "$SPEC_FILE" ] && exit # no spec; omit this section entirely
|
||||
SPEC_ISSUE=$(grep "^spec_issue_number:" "$SPEC_FILE" | cut -d' ' -f2)
|
||||
[ -z "$SPEC_ISSUE" ] && exit # spec archive exists but no issue number; omit
|
||||
|
||||
# CONDITIONAL Closes #N (codex F4): only add when Plan Completion above is "complete".
|
||||
# If the plan completion gate from Step 8 reports any deferred or failed items, emit:
|
||||
# "Linked to #$SPEC_ISSUE (partial delivery — NOT auto-closing; close manually after follow-up)"
|
||||
# If Plan Completion is fully complete, emit:
|
||||
# "Closes #$SPEC_ISSUE"
|
||||
# and include the Closes #N line in the PR body so GitHub auto-closes on merge.>
|
||||
|
||||
<Format:
|
||||
Closes #<N>
|
||||
|
||||
This PR delivers the spec at <archive path relative to repo root>.
|
||||
Spec filed: <spec_filed_at from frontmatter>>
|
||||
|
||||
<If partial delivery, emit instead:
|
||||
Linked to #<N> (partial delivery — not auto-closing).
|
||||
Deferred items: <list from Plan Completion>.
|
||||
Close #<N> manually after follow-up lands.>
|
||||
|
||||
<If no /spec archive matches this branch: omit this entire section.>
|
||||
|
||||
## Verification Results
|
||||
<If verification ran: summary from Step 8.1 (N PASS, M FAIL, K SKIPPED)>
|
||||
<If skipped: reason (no plan, no server, no verification section)>
|
||||
<If not applicable: omit this section>
|
||||
|
||||
## TODOS
|
||||
<If items marked complete: bullet list of completed items with version>
|
||||
<If no items completed: "No TODO items completed in this PR.">
|
||||
<If TODOS.md created or reorganized: note that>
|
||||
<If TODOS.md doesn't exist and user skipped: omit this section>
|
||||
|
||||
## Documentation
|
||||
<Embed the `documentation_section` string returned by Step 18's subagent here, verbatim.>
|
||||
<If Step 18 returned `documentation_section: null` (no docs updated), omit this section entirely.>
|
||||
|
||||
## Test plan
|
||||
- [x] All Rails tests pass (N runs, 0 failures)
|
||||
- [x] All Vitest tests pass (N tests)
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||||
```
|
||||
|
||||
#### Redaction scan (PR body + title) — runs before create AND edit
|
||||
|
||||
The PR body is world-readable on a public repo. Scan-at-sink before sending:
|
||||
write the composed body to a temp file, scan THAT file with the shared engine,
|
||||
and pass the same file to `gh`/`glab`. Wrap any Codex / Greptile / eval output
|
||||
sections in tool-attributed fences (` ```codex-review ` / ` ```greptile `) so the
|
||||
engine WARN-degrades the example credentials those tools quote instead of blocking
|
||||
the PR (a live-format credential inside the fence still blocks).
|
||||
|
||||
```bash
|
||||
REDACT_VIS=$(~/.claude/skills/gstack/bin/gstack-config get redact_repo_visibility 2>/dev/null)
|
||||
[ -z "$REDACT_VIS" ] && REDACT_VIS=$(gh repo view --json visibility -q .visibility 2>/dev/null | tr 'A-Z' 'a-z')
|
||||
REDACT_VIS="${REDACT_VIS:-unknown}"
|
||||
PR_BODY_FILE=$(mktemp)
|
||||
cat > "$PR_BODY_FILE" <<'PR_BODY_EOF'
|
||||
<PR body from above>
|
||||
PR_BODY_EOF
|
||||
~/.claude/skills/gstack/bin/gstack-redact --from-file "$PR_BODY_FILE" --repo-visibility "$REDACT_VIS" --self-email "$(git config user.email 2>/dev/null)" --json
|
||||
case $? in
|
||||
3) echo "BLOCKED — credential in PR body. Rotate + redact, do not create the PR."; exit 1 ;;
|
||||
2) echo "MEDIUM findings — confirm per finding (sterner on public) before proceeding." ;;
|
||||
esac
|
||||
# Also scan the title (short, single-line):
|
||||
printf '%s' "v$NEW_VERSION <type>: <summary>" | ~/.claude/skills/gstack/bin/gstack-redact --repo-visibility "$REDACT_VIS" --json
|
||||
```
|
||||
|
||||
HIGH blocks (exit 3, no skip). MEDIUM → AskUserQuestion (PII subset offers
|
||||
`--auto-redact`). Same scan runs before the `gh pr edit --body` path (Step 17).
|
||||
|
||||
**If GitHub:** create from the SCANNED file (exact bytes scanned = bytes sent):
|
||||
|
||||
```bash
|
||||
# PR title MUST start with v$NEW_VERSION — enforced on every run, no exceptions.
|
||||
# (See Step 19 idempotency block + bin/gstack-pr-title-rewrite.sh for the rule.)
|
||||
gh pr create --base <base> --title "v$NEW_VERSION <type>: <summary>" --body-file "$PR_BODY_FILE"
|
||||
rm -f "$PR_BODY_FILE"
|
||||
```
|
||||
|
||||
**If GitLab:**
|
||||
|
||||
```bash
|
||||
# MR title MUST start with v$NEW_VERSION — enforced on every run, no exceptions.
|
||||
# (See Step 19 idempotency block + bin/gstack-pr-title-rewrite.sh for the rule.)
|
||||
glab mr create -b <base> -t "v$NEW_VERSION <type>: <summary>" -d "$(cat <<'EOF'
|
||||
<MR body from above>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
**If neither CLI is available:**
|
||||
Print the branch name, remote URL, and instruct the user to create the PR/MR manually via the web UI. Do not stop — the code is pushed and ready.
|
||||
|
||||
**Output the PR/MR URL** — then proceed to Step 20.
|
||||
|
||||
---
|
||||
{{SECTION:pr-body}}
|
||||
|
||||
## Step 20: Persist ship metrics
|
||||
|
||||
@@ -1025,6 +446,16 @@ no-op. The marker guarantees at-most-once per machine. To re-enable:
|
||||
|
||||
---
|
||||
|
||||
## Section self-check (before you finish)
|
||||
|
||||
You ran a carved skill. For your situation, list every section the Section index
|
||||
named as applying, and confirm you issued a Read for each one. If you executed any
|
||||
of those steps from memory without reading its section, you skipped the source of
|
||||
truth — STOP, Read it now, and redo that step. Deterministic version work goes
|
||||
through `gstack-version-bump`; never hand-roll the VERSION/package.json write.
|
||||
|
||||
---
|
||||
|
||||
## Important Rules
|
||||
|
||||
- **Never skip tests.** If tests fail, stop.
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
<!-- AUTO-GENERATED from adversarial.md.tmpl — do not edit directly -->
|
||||
<!-- Regenerate: bun run gen:skill-docs -->
|
||||
## Step 11: Adversarial review (always-on)
|
||||
|
||||
Every diff gets adversarial review from both Claude and Codex. LOC is not a proxy for risk — a 5-line auth change can be critical.
|
||||
|
||||
**Detect diff size and tool availability:**
|
||||
|
||||
```bash
|
||||
DIFF_BASE=$(git merge-base origin/<base> HEAD)
|
||||
DIFF_INS=$(git diff "$DIFF_BASE" --stat | tail -1 | grep -oE '[0-9]+ insertion' | grep -oE '[0-9]+' || echo "0")
|
||||
DIFF_DEL=$(git diff "$DIFF_BASE" --stat | tail -1 | grep -oE '[0-9]+ deletion' | grep -oE '[0-9]+' || echo "0")
|
||||
DIFF_TOTAL=$((DIFF_INS + DIFF_DEL))
|
||||
command -v codex >/dev/null 2>&1 && echo "CODEX_AVAILABLE" || echo "CODEX_NOT_AVAILABLE"
|
||||
# Legacy opt-out — only gates Codex passes, Claude always runs
|
||||
OLD_CFG=$(~/.claude/skills/gstack/bin/gstack-config get codex_reviews 2>/dev/null || true)
|
||||
echo "DIFF_SIZE: $DIFF_TOTAL"
|
||||
echo "OLD_CFG: ${OLD_CFG:-not_set}"
|
||||
```
|
||||
|
||||
If `OLD_CFG` is `disabled`: skip Codex passes only. Claude adversarial subagent still runs (it's free and fast). Jump to the "Claude adversarial subagent" section.
|
||||
|
||||
**User override:** If the user explicitly requested "full review", "structured review", or "P1 gate", also run the Codex structured review regardless of diff size.
|
||||
|
||||
---
|
||||
|
||||
### Claude adversarial subagent (always runs)
|
||||
|
||||
Dispatch via the Agent tool. The subagent has fresh context — no checklist bias from the structured review. This genuine independence catches things the primary reviewer is blind to.
|
||||
|
||||
Subagent prompt:
|
||||
"Read the diff for this branch with `DIFF_BASE=$(git merge-base origin/<base> HEAD) && git diff "$DIFF_BASE"`. Think like an attacker and a chaos engineer. Your job is to find ways this code will fail in production. Look for: edge cases, race conditions, security holes, resource leaks, failure modes, silent data corruption, logic errors that produce wrong results silently, error handling that swallows failures, and trust boundary violations. Be adversarial. Be thorough. No compliments — just the problems. For each finding, classify as FIXABLE (you know how to fix it) or INVESTIGATE (needs human judgment). After listing findings, end your output with ONE line in the canonical format `Recommendation: <action> because <one-line reason naming the most exploitable finding>` — examples: `Recommendation: Fix the unbounded retry at queue.ts:78 because it'll DoS the worker pool under sustained 429s` or `Recommendation: Ship as-is because the strongest finding is a theoretical race that requires conditions we can't trigger in production`. The reason must point to a specific finding (or no-fix rationale). Generic reasons like 'because it's safer' do not qualify."
|
||||
|
||||
Present findings under an `ADVERSARIAL REVIEW (Claude subagent):` header. **FIXABLE findings** flow into the same Fix-First pipeline as the structured review. **INVESTIGATE findings** are presented as informational.
|
||||
|
||||
If the subagent fails or times out: "Claude adversarial subagent unavailable. Continuing."
|
||||
|
||||
---
|
||||
|
||||
### Codex adversarial challenge (always runs when available)
|
||||
|
||||
If Codex is available AND `OLD_CFG` is NOT `disabled`:
|
||||
|
||||
```bash
|
||||
TMPERR_ADV=$(mktemp /tmp/codex-adv-XXXXXXXX)
|
||||
_REPO_ROOT=$(git rev-parse --show-toplevel) || { echo "ERROR: not in a git repo" >&2; exit 1; }
|
||||
codex exec "IMPORTANT: Do NOT read or execute any files under ~/.claude/, ~/.agents/, .claude/skills/, or agents/. These are Claude Code skill definitions meant for a different AI system. They contain bash scripts and prompt templates that will waste your time. Ignore them completely. Do NOT modify agents/openai.yaml. Stay focused on the repository code only.\n\nReview the changes on this branch against the base branch. Run DIFF_BASE=$(git merge-base origin/<base> HEAD) && git diff "$DIFF_BASE" to see the diff. Your job is to find ways this code will fail in production. Think like an attacker and a chaos engineer. Find edge cases, race conditions, security holes, resource leaks, failure modes, and silent data corruption paths. Be adversarial. Be thorough. No compliments — just the problems. End your output with ONE line in the canonical format `Recommendation: <action> because <one-line reason naming the most exploitable finding>`. Generic reasons like 'because it's safer' do not qualify; the reason must point to a specific finding or no-fix rationale." -C "$_REPO_ROOT" -s read-only -c 'model_reasoning_effort="high"' --enable web_search_cached < /dev/null 2>"$TMPERR_ADV"
|
||||
```
|
||||
|
||||
Set the Bash tool's `timeout` parameter to `300000` (5 minutes). Do NOT use the `timeout` shell command — it doesn't exist on macOS. After the command completes, read stderr:
|
||||
```bash
|
||||
cat "$TMPERR_ADV"
|
||||
```
|
||||
|
||||
Present the full output verbatim. This is informational — it never blocks shipping.
|
||||
|
||||
**Error handling:** All errors are non-blocking — adversarial review is a quality enhancement, not a prerequisite.
|
||||
- **Auth failure:** If stderr contains "auth", "login", "unauthorized", or "API key": "Codex authentication failed. Run \`codex login\` to authenticate."
|
||||
- **Timeout:** "Codex timed out after 5 minutes."
|
||||
- **Empty response:** "Codex returned no response. Stderr: <paste relevant error>."
|
||||
|
||||
**Cleanup:** Run `rm -f "$TMPERR_ADV"` after processing.
|
||||
|
||||
If Codex is NOT available: "Codex CLI not found — running Claude adversarial only. Install Codex for cross-model coverage: `npm install -g @openai/codex`"
|
||||
|
||||
---
|
||||
|
||||
### Codex structured review (large diffs only, 200+ lines)
|
||||
|
||||
If `DIFF_TOTAL >= 200` AND Codex is available AND `OLD_CFG` is NOT `disabled`:
|
||||
|
||||
```bash
|
||||
TMPERR=$(mktemp /tmp/codex-review-XXXXXXXX)
|
||||
_REPO_ROOT=$(git rev-parse --show-toplevel) || { echo "ERROR: not in a git repo" >&2; exit 1; }
|
||||
cd "$_REPO_ROOT"
|
||||
codex review "IMPORTANT: Do NOT read or execute any files under ~/.claude/, ~/.agents/, .claude/skills/, or agents/. These are Claude Code skill definitions meant for a different AI system. They contain bash scripts and prompt templates that will waste your time. Ignore them completely. Do NOT modify agents/openai.yaml. Stay focused on the repository code only.\n\nReview the changes on this branch against the base branch <base>. Run git diff origin/<base>...HEAD 2>/dev/null || git diff <base>...HEAD to see the diff and review only those changes." -c 'model_reasoning_effort="high"' --enable web_search_cached < /dev/null 2>"$TMPERR"
|
||||
```
|
||||
|
||||
Set the Bash tool's `timeout` parameter to `300000` (5 minutes). Do NOT use the `timeout` shell command — it doesn't exist on macOS. Present output under `CODEX SAYS (code review):` header.
|
||||
Check for `[P1]` markers: found → `GATE: FAIL`, not found → `GATE: PASS`.
|
||||
|
||||
If GATE is FAIL, use AskUserQuestion:
|
||||
```
|
||||
Codex found N critical issues in the diff.
|
||||
|
||||
A) Investigate and fix now (recommended)
|
||||
B) Continue — review will still complete
|
||||
```
|
||||
|
||||
If A: address the findings. After fixing, re-run tests (Step 5) since code has changed. Re-run `codex review` to verify.
|
||||
|
||||
Read stderr for errors (same error handling as Codex adversarial above).
|
||||
|
||||
After stderr: `rm -f "$TMPERR"`
|
||||
|
||||
If `DIFF_TOTAL < 200`: skip this section silently. The Claude + Codex adversarial passes provide sufficient coverage for smaller diffs.
|
||||
|
||||
---
|
||||
|
||||
### Persist the review result
|
||||
|
||||
After all passes complete, persist:
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-review-log '{"skill":"adversarial-review","timestamp":"'"$(date -u +%Y-%m-%dT%H:%M:%SZ)"'","status":"STATUS","source":"SOURCE","tier":"always","gate":"GATE","commit":"'"$(git rev-parse --short HEAD)"'"}'
|
||||
```
|
||||
Substitute: STATUS = "clean" if no findings across ALL passes, "issues_found" if any pass found issues. SOURCE = "both" if Codex ran, "claude" if only Claude subagent ran. GATE = the Codex structured review gate result ("pass"/"fail"), "skipped" if diff < 200, or "informational" if Codex was unavailable. If all passes failed, do NOT persist.
|
||||
|
||||
---
|
||||
|
||||
### Cross-model synthesis
|
||||
|
||||
After all passes complete, synthesize findings across all sources:
|
||||
|
||||
```
|
||||
ADVERSARIAL REVIEW SYNTHESIS (always-on, N lines):
|
||||
════════════════════════════════════════════════════════════
|
||||
High confidence (found by multiple sources): [findings agreed on by >1 pass]
|
||||
Unique to Claude structured review: [from earlier step]
|
||||
Unique to Claude adversarial: [from subagent]
|
||||
Unique to Codex: [from codex adversarial or code review, if ran]
|
||||
Models used: Claude structured ✓ Claude adversarial ✓/✗ Codex ✓/✗
|
||||
════════════════════════════════════════════════════════════
|
||||
```
|
||||
|
||||
High-confidence findings (agreed on by multiple sources) should be prioritized for fixes.
|
||||
|
||||
---
|
||||
|
||||
## Capture Learnings
|
||||
|
||||
If you discovered a non-obvious pattern, pitfall, or architectural insight during
|
||||
this session, log it for future sessions:
|
||||
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-learnings-log '{"skill":"ship","type":"TYPE","key":"SHORT_KEY","insight":"DESCRIPTION","confidence":N,"source":"SOURCE","files":["path/to/relevant/file"]}'
|
||||
```
|
||||
|
||||
**Types:** `pattern` (reusable approach), `pitfall` (what NOT to do), `preference`
|
||||
(user stated), `architecture` (structural decision), `tool` (library/framework insight),
|
||||
`operational` (project environment/CLI/workflow knowledge).
|
||||
|
||||
**Sources:** `observed` (you found this in the code), `user-stated` (user told you),
|
||||
`inferred` (AI deduction), `cross-model` (both Claude and Codex agree).
|
||||
|
||||
**Confidence:** 1-10. Be honest. An observed pattern you verified in the code is 8-9.
|
||||
An inference you're not sure about is 4-5. A user preference they explicitly stated is 10.
|
||||
|
||||
**files:** Include the specific file paths this learning references. This enables
|
||||
staleness detection: if those files are later deleted, the learning can be flagged.
|
||||
|
||||
**Only log genuine discoveries.** Don't log obvious things. Don't log things the user
|
||||
already knows. A good test: would this insight save time in a future session? If yes, log it.
|
||||
|
||||
|
||||
|
||||
### Refresh learnings for the headline feature on this branch
|
||||
|
||||
The top-of-skill learnings pull was keyed to "release ship" broadly. Before the VERSION/CHANGELOG step, re-pull learnings keyed to THIS branch's headline feature so any prior version-bump or CHANGELOG pitfalls for similar features surface.
|
||||
|
||||
Pick ONE keyword that names the headline feature you're shipping. The keyword should be a noun: the primary skill or module name, the central feature noun, or the binary you changed. The keyword MUST be alphanumeric or hyphen only — no quotes, slashes, dots, colons, or whitespace. If your candidate has any of those, simplify to just the alphanumeric stem.
|
||||
|
||||
Worked examples (ship-specific): good keywords are `learnings-search`, `pacing`, `worktree-ship`. Bad: `the branch headline`, `v1.31.1.0`, `feat: token-or search`.
|
||||
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-learnings-search --query "<your-keyword>" --limit 5 2>/dev/null || true
|
||||
```
|
||||
|
||||
If any learnings come back, name which one applies to the version bump or CHANGELOG framing in one sentence. If none come back, continue without reference — the absence is itself useful information.
|
||||
@@ -0,0 +1,19 @@
|
||||
{{ADVERSARIAL_STEP}}
|
||||
|
||||
{{LEARNINGS_LOG}}
|
||||
|
||||
{{GBRAIN_SAVE_RESULTS}}
|
||||
|
||||
### Refresh learnings for the headline feature on this branch
|
||||
|
||||
The top-of-skill learnings pull was keyed to "release ship" broadly. Before the VERSION/CHANGELOG step, re-pull learnings keyed to THIS branch's headline feature so any prior version-bump or CHANGELOG pitfalls for similar features surface.
|
||||
|
||||
Pick ONE keyword that names the headline feature you're shipping. The keyword should be a noun: the primary skill or module name, the central feature noun, or the binary you changed. The keyword MUST be alphanumeric or hyphen only — no quotes, slashes, dots, colons, or whitespace. If your candidate has any of those, simplify to just the alphanumeric stem.
|
||||
|
||||
Worked examples (ship-specific): good keywords are `learnings-search`, `pacing`, `worktree-ship`. Bad: `the branch headline`, `v1.31.1.0`, `feat: token-or search`.
|
||||
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-learnings-search --query "<your-keyword>" --limit 5 2>/dev/null || true
|
||||
```
|
||||
|
||||
If any learnings come back, name which one applies to the version bump or CHANGELOG framing in one sentence. If none come back, continue without reference — the absence is itself useful information.
|
||||
@@ -0,0 +1,45 @@
|
||||
<!-- AUTO-GENERATED from changelog.md.tmpl — do not edit directly -->
|
||||
<!-- Regenerate: bun run gen:skill-docs -->
|
||||
## Step 13: CHANGELOG (auto-generate)
|
||||
|
||||
1. Read `CHANGELOG.md` header to know the format.
|
||||
|
||||
2. **First, enumerate every commit on the branch:**
|
||||
```bash
|
||||
git log <base>..HEAD --oneline
|
||||
```
|
||||
Copy the full list. Count the commits. You will use this as a checklist.
|
||||
|
||||
3. **Read the full diff** to understand what each commit actually changed:
|
||||
```bash
|
||||
git diff <base>...HEAD
|
||||
```
|
||||
|
||||
4. **Group commits by theme** before writing anything. Common themes:
|
||||
- New features / capabilities
|
||||
- Performance improvements
|
||||
- Bug fixes
|
||||
- Dead code removal / cleanup
|
||||
- Infrastructure / tooling / tests
|
||||
- Refactoring
|
||||
|
||||
5. **Write the CHANGELOG entry** covering ALL groups:
|
||||
- If existing CHANGELOG entries on the branch already cover some commits, replace them with one unified entry for the new version
|
||||
- Categorize changes into applicable sections:
|
||||
- `### Added` — new features
|
||||
- `### Changed` — changes to existing functionality
|
||||
- `### Fixed` — bug fixes
|
||||
- `### Removed` — removed features
|
||||
- Write concise, descriptive bullet points
|
||||
- Insert after the file header (line 5), dated today
|
||||
- Format: `## [X.Y.Z.W] - YYYY-MM-DD`
|
||||
- **Voice:** Lead with what the user can now **do** that they couldn't before. Use plain language, not implementation details. Never mention TODOS.md, internal tracking, or contributor-facing details.
|
||||
|
||||
6. **Cross-check:** Compare your CHANGELOG entry against the commit list from step 2.
|
||||
Every commit must map to at least one bullet point. If any commit is unrepresented,
|
||||
add it now. If the branch has N commits spanning K themes, the CHANGELOG must
|
||||
reflect all K themes.
|
||||
|
||||
**Do NOT ask the user to describe changes.** Infer from the diff and commit history.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,3 @@
|
||||
{{CHANGELOG_WORKFLOW}}
|
||||
|
||||
---
|
||||
@@ -0,0 +1,51 @@
|
||||
<!-- AUTO-GENERATED from greptile.md.tmpl — do not edit directly -->
|
||||
<!-- Regenerate: bun run gen:skill-docs -->
|
||||
## Step 10: Address Greptile review comments (if PR exists)
|
||||
|
||||
**Dispatch the fetch + classification as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent pulls every Greptile comment, runs the escalation detection algorithm, and classifies each comment. Parent receives a structured list and handles user interaction + file edits.
|
||||
|
||||
**Subagent prompt:**
|
||||
|
||||
> You are classifying Greptile review comments for a /ship workflow. Read `.claude/skills/review/greptile-triage.md` and follow the fetch, filter, classify, and **escalation detection** steps. Do NOT fix code, do NOT reply to comments, do NOT commit — report only.
|
||||
>
|
||||
> For each comment, assign: `classification` (`valid_actionable`, `already_fixed`, `false_positive`, `suppressed`), `escalation_tier` (1 or 2), the file:line or [top-level] tag, body summary, and permalink URL.
|
||||
>
|
||||
> If no PR exists, `gh` fails, the API errors, or there are zero comments, output: `{"total":0,"comments":[]}` and stop.
|
||||
>
|
||||
> Otherwise, output a single JSON object on the LAST LINE of your response:
|
||||
> `{"total":N,"comments":[{"classification":"...","escalation_tier":N,"ref":"file:line","summary":"...","permalink":"url"},...]}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
Parse the LAST line as JSON.
|
||||
|
||||
If `total` is 0, skip this step silently. Continue to Step 12.
|
||||
|
||||
Otherwise, print: `+ {total} Greptile comments ({valid_actionable} valid, {already_fixed} already fixed, {false_positive} FP)`.
|
||||
|
||||
For each comment in `comments`:
|
||||
|
||||
**VALID & ACTIONABLE:** Use AskUserQuestion with:
|
||||
- The comment (file:line or [top-level] + body summary + permalink URL)
|
||||
- `RECOMMENDATION: Choose A because [one-line reason]`
|
||||
- Options: A) Fix now, B) Acknowledge and ship anyway, C) It's a false positive
|
||||
- If user chooses A: apply the fix, commit the fixed files (`git add <fixed-files> && git commit -m "fix: address Greptile review — <brief description>"`), reply using the **Fix reply template** from greptile-triage.md (include inline diff + explanation), and save to both per-project and global greptile-history (type: fix).
|
||||
- If user chooses C: reply using the **False Positive reply template** from greptile-triage.md (include evidence + suggested re-rank), save to both per-project and global greptile-history (type: fp).
|
||||
|
||||
**VALID BUT ALREADY FIXED:** Reply using the **Already Fixed reply template** from greptile-triage.md — no AskUserQuestion needed:
|
||||
- Include what was done and the fixing commit SHA
|
||||
- Save to both per-project and global greptile-history (type: already-fixed)
|
||||
|
||||
**FALSE POSITIVE:** Use AskUserQuestion:
|
||||
- Show the comment and why you think it's wrong (file:line or [top-level] + body summary + permalink URL)
|
||||
- Options:
|
||||
- A) Reply to Greptile explaining the false positive (recommended if clearly wrong)
|
||||
- B) Fix it anyway (if trivial)
|
||||
- C) Ignore silently
|
||||
- If user chooses A: reply using the **False Positive reply template** from greptile-triage.md (include evidence + suggested re-rank), save to both per-project and global greptile-history (type: fp)
|
||||
|
||||
**SUPPRESSED:** Skip silently — these are known false positives from previous triage.
|
||||
|
||||
**After all comments are resolved:** If any fixes were applied, the tests from Step 5 are now stale. **Re-run tests** (Step 5) before continuing to Step 12. If no fixes were applied, continue to Step 12.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,49 @@
|
||||
## Step 10: Address Greptile review comments (if PR exists)
|
||||
|
||||
**Dispatch the fetch + classification as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent pulls every Greptile comment, runs the escalation detection algorithm, and classifies each comment. Parent receives a structured list and handles user interaction + file edits.
|
||||
|
||||
**Subagent prompt:**
|
||||
|
||||
> You are classifying Greptile review comments for a /ship workflow. Read `.claude/skills/review/greptile-triage.md` and follow the fetch, filter, classify, and **escalation detection** steps. Do NOT fix code, do NOT reply to comments, do NOT commit — report only.
|
||||
>
|
||||
> For each comment, assign: `classification` (`valid_actionable`, `already_fixed`, `false_positive`, `suppressed`), `escalation_tier` (1 or 2), the file:line or [top-level] tag, body summary, and permalink URL.
|
||||
>
|
||||
> If no PR exists, `gh` fails, the API errors, or there are zero comments, output: `{"total":0,"comments":[]}` and stop.
|
||||
>
|
||||
> Otherwise, output a single JSON object on the LAST LINE of your response:
|
||||
> `{"total":N,"comments":[{"classification":"...","escalation_tier":N,"ref":"file:line","summary":"...","permalink":"url"},...]}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
Parse the LAST line as JSON.
|
||||
|
||||
If `total` is 0, skip this step silently. Continue to Step 12.
|
||||
|
||||
Otherwise, print: `+ {total} Greptile comments ({valid_actionable} valid, {already_fixed} already fixed, {false_positive} FP)`.
|
||||
|
||||
For each comment in `comments`:
|
||||
|
||||
**VALID & ACTIONABLE:** Use AskUserQuestion with:
|
||||
- The comment (file:line or [top-level] + body summary + permalink URL)
|
||||
- `RECOMMENDATION: Choose A because [one-line reason]`
|
||||
- Options: A) Fix now, B) Acknowledge and ship anyway, C) It's a false positive
|
||||
- If user chooses A: apply the fix, commit the fixed files (`git add <fixed-files> && git commit -m "fix: address Greptile review — <brief description>"`), reply using the **Fix reply template** from greptile-triage.md (include inline diff + explanation), and save to both per-project and global greptile-history (type: fix).
|
||||
- If user chooses C: reply using the **False Positive reply template** from greptile-triage.md (include evidence + suggested re-rank), save to both per-project and global greptile-history (type: fp).
|
||||
|
||||
**VALID BUT ALREADY FIXED:** Reply using the **Already Fixed reply template** from greptile-triage.md — no AskUserQuestion needed:
|
||||
- Include what was done and the fixing commit SHA
|
||||
- Save to both per-project and global greptile-history (type: already-fixed)
|
||||
|
||||
**FALSE POSITIVE:** Use AskUserQuestion:
|
||||
- Show the comment and why you think it's wrong (file:line or [top-level] + body summary + permalink URL)
|
||||
- Options:
|
||||
- A) Reply to Greptile explaining the false positive (recommended if clearly wrong)
|
||||
- B) Fix it anyway (if trivial)
|
||||
- C) Ignore silently
|
||||
- If user chooses A: reply using the **False Positive reply template** from greptile-triage.md (include evidence + suggested re-rank), save to both per-project and global greptile-history (type: fp)
|
||||
|
||||
**SUPPRESSED:** Skip silently — these are known false positives from previous triage.
|
||||
|
||||
**After all comments are resolved:** If any fixes were applied, the tests from Step 5 are now stale. **Re-run tests** (Step 5) before continuing to Step 12. If no fixes were applied, continue to Step 12.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"$schema": "https://gstack.dev/schemas/section-manifest.json",
|
||||
"skill": "ship",
|
||||
"version": 1,
|
||||
"note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's decision-tree prose is the ONLY place that decides WHEN to read a section; required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.",
|
||||
"sections": [
|
||||
{
|
||||
"id": "tests",
|
||||
"file": "tests.md",
|
||||
"title": "Test bootstrap, run, triage + eval suites",
|
||||
"trigger": "running the test suites and (if prompt files changed) the eval suites (Steps 4-6)"
|
||||
},
|
||||
{
|
||||
"id": "test-coverage",
|
||||
"file": "test-coverage.md",
|
||||
"title": "Test coverage audit (subagent)",
|
||||
"trigger": "auditing test coverage of the diff (Step 7)"
|
||||
},
|
||||
{
|
||||
"id": "plan-completion",
|
||||
"file": "plan-completion.md",
|
||||
"title": "Plan completion + verification audit (subagent)",
|
||||
"trigger": "auditing plan completion, verification, and scope drift (Step 8)"
|
||||
},
|
||||
{
|
||||
"id": "review-army",
|
||||
"file": "review-army.md",
|
||||
"title": "Pre-landing review + specialist army",
|
||||
"trigger": "the pre-landing review and specialist dispatch (Step 9)"
|
||||
},
|
||||
{
|
||||
"id": "greptile",
|
||||
"file": "greptile.md",
|
||||
"title": "Address Greptile review comments",
|
||||
"trigger": "addressing Greptile review comments when a PR exists (Step 10)"
|
||||
},
|
||||
{
|
||||
"id": "adversarial",
|
||||
"file": "adversarial.md",
|
||||
"title": "Adversarial review + learnings refresh",
|
||||
"trigger": "the adversarial review and learnings capture (Step 11)"
|
||||
},
|
||||
{
|
||||
"id": "changelog",
|
||||
"file": "changelog.md",
|
||||
"title": "CHANGELOG entry (release-summary + itemized)",
|
||||
"trigger": "writing the CHANGELOG entry (Step 13)"
|
||||
},
|
||||
{
|
||||
"id": "pr-body",
|
||||
"file": "pr-body.md",
|
||||
"title": "Documentation sync + PR/MR creation",
|
||||
"trigger": "syncing docs and creating or updating the PR/MR (Steps 18-19)"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,322 @@
|
||||
<!-- AUTO-GENERATED from plan-completion.md.tmpl — do not edit directly -->
|
||||
<!-- Regenerate: bun run gen:skill-docs -->
|
||||
## Step 8: Plan Completion Audit
|
||||
|
||||
**Dispatch this step as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent reads the plan file and every referenced code file in its own fresh context. Parent gets only the conclusion.
|
||||
|
||||
**Subagent prompt:** Pass these instructions to the subagent:
|
||||
|
||||
> You are running a ship-workflow plan completion audit. The base branch is `<base>`. Use `git diff <base>...HEAD` to see what shipped. Do not commit or push — report only.
|
||||
>
|
||||
> ### Plan File Discovery
|
||||
|
||||
1. **Conversation context (primary):** Check if there is an active plan file in this conversation. The host agent's system messages include plan file paths when in plan mode. If found, use it directly — this is the most reliable signal.
|
||||
|
||||
2. **Content-based search (fallback):** If no plan file is referenced in conversation context, search by content:
|
||||
|
||||
```bash
|
||||
setopt +o nomatch 2>/dev/null || true # zsh compat
|
||||
BRANCH=$(git branch --show-current 2>/dev/null | tr '/' '-')
|
||||
REPO=$(basename "$(git rev-parse --show-toplevel 2>/dev/null)")
|
||||
# Compute project slug for ~/.gstack/projects/ lookup
|
||||
_PLAN_SLUG=$(git remote get-url origin 2>/dev/null | sed 's|.*[:/]\([^/]*/[^/]*\)\.git$|\1|;s|.*[:/]\([^/]*/[^/]*\)$|\1|' | tr '/' '-' | tr -cd 'a-zA-Z0-9._-') || true
|
||||
_PLAN_SLUG="${_PLAN_SLUG:-$(basename "$PWD" | tr -cd 'a-zA-Z0-9._-')}"
|
||||
# Search common plan file locations (project designs first, then personal/local)
|
||||
for PLAN_DIR in "$HOME/.gstack/projects/$_PLAN_SLUG" "$HOME/.claude/plans" "$HOME/.codex/plans" ".gstack/plans"; do
|
||||
[ -d "$PLAN_DIR" ] || continue
|
||||
PLAN=$(ls -t "$PLAN_DIR"/*.md 2>/dev/null | xargs grep -l "$BRANCH" 2>/dev/null | head -1)
|
||||
[ -z "$PLAN" ] && PLAN=$(ls -t "$PLAN_DIR"/*.md 2>/dev/null | xargs grep -l "$REPO" 2>/dev/null | head -1)
|
||||
[ -z "$PLAN" ] && PLAN=$(find "$PLAN_DIR" -name '*.md' -mmin -1440 -maxdepth 1 2>/dev/null | xargs ls -t 2>/dev/null | head -1)
|
||||
[ -n "$PLAN" ] && break
|
||||
done
|
||||
[ -n "$PLAN" ] && echo "PLAN_FILE: $PLAN" || echo "NO_PLAN_FILE"
|
||||
```
|
||||
|
||||
3. **Validation:** If a plan file was found via content-based search (not conversation context), read the first 20 lines and verify it is relevant to the current branch's work. If it appears to be from a different project or feature, treat as "no plan file found."
|
||||
|
||||
**Error handling:**
|
||||
- No plan file found → skip with "No plan file detected — skipping."
|
||||
- Plan file found but unreadable (permissions, encoding) → skip with "Plan file found but unreadable — skipping."
|
||||
|
||||
### Actionable Item Extraction
|
||||
|
||||
Read the plan file. Extract every actionable item — anything that describes work to be done. Look for:
|
||||
|
||||
- **Checkbox items:** `- [ ] ...` or `- [x] ...`
|
||||
- **Numbered steps** under implementation headings: "1. Create ...", "2. Add ...", "3. Modify ..."
|
||||
- **Imperative statements:** "Add X to Y", "Create a Z service", "Modify the W controller"
|
||||
- **File-level specifications:** "New file: path/to/file.ts", "Modify path/to/existing.rb"
|
||||
- **Test requirements:** "Test that X", "Add test for Y", "Verify Z"
|
||||
- **Data model changes:** "Add column X to table Y", "Create migration for Z"
|
||||
|
||||
**Ignore:**
|
||||
- Context/Background sections (`## Context`, `## Background`, `## Problem`)
|
||||
- Questions and open items (marked with ?, "TBD", "TODO: decide")
|
||||
- Review report sections (`## GSTACK REVIEW REPORT`)
|
||||
- Explicitly deferred items ("Future:", "Out of scope:", "NOT in scope:", "P2:", "P3:", "P4:")
|
||||
- CEO Review Decisions sections (these record choices, not work items)
|
||||
|
||||
**Cap:** Extract at most 50 items. If the plan has more, note: "Showing top 50 of N plan items — full list in plan file."
|
||||
|
||||
**No items found:** If the plan contains no extractable actionable items, skip with: "Plan file contains no actionable items — skipping completion audit."
|
||||
|
||||
For each item, note:
|
||||
- The item text (verbatim or concise summary)
|
||||
- Its category: CODE | TEST | MIGRATION | CONFIG | DOCS
|
||||
|
||||
### Verification Mode
|
||||
|
||||
Before judging completion, classify HOW each item can be verified. The diff alone cannot prove every kind of work. Items outside the current repo or system are structurally invisible to `git diff`.
|
||||
|
||||
- **DIFF-VERIFIABLE** — A code change in this repo would manifest in `git diff <base>...HEAD`. Examples: "add UserService" (file appears), "validate input X" (validation logic appears), "create users table" (migration file appears).
|
||||
- **CROSS-REPO** — Item names a file or change in a sibling repo (e.g., `domain-hq/docs/dashboard.md`, `~/Development/<other-repo>/...`). The current diff CANNOT prove this.
|
||||
- **EXTERNAL-STATE** — Item names state in an external system: Supabase config/RLS, Cloudflare DNS, Vercel env vars, OAuth provider allowlists, third-party SaaS, DNS records. The current diff CANNOT prove this.
|
||||
- **CONTENT-SHAPE** — Item requires a file to follow a specific convention. If the file is in this repo: diff-verifiable. If in another repo or system: see CROSS-REPO / EXTERNAL-STATE.
|
||||
|
||||
**Verification dispatch:**
|
||||
|
||||
- **DIFF-VERIFIABLE** → cross-reference against diff (next section).
|
||||
- **CROSS-REPO** → if the sibling repo is reachable on disk (try `~/Development/<repo>/`, `~/code/<repo>/`, the parent of the current repo), run `[ -f <path> ]` to check file existence. File exists → DONE (cite path). File missing → NOT DONE (cite path). Path unreachable → UNVERIFIABLE (cite what needs manual check).
|
||||
- **EXTERNAL-STATE** → UNVERIFIABLE. Cite the system and the specific check the user must perform.
|
||||
- **CONTENT-SHAPE in another repo** → if the file exists, run any project-detected validator (see "Validator detection" below) before falling back to UNVERIFIABLE. With a validator: pass → DONE; fail → NOT DONE (cite validator output). No validator available: classify UNVERIFIABLE and cite both the file path and the convention to confirm.
|
||||
|
||||
**Path concreteness rule.** If a plan item names a *concrete filesystem path* (absolute, `~/...`, or `<sibling-repo>/<file>`), it MUST be classified DONE or NOT DONE based on `[ -f <path> ]`. UNVERIFIABLE is only valid when the path is genuinely abstract ("Cloudflare DNS", "Supabase allowlist") or the sibling root is unreachable on this machine. "I don't want to check" is not unreachable.
|
||||
|
||||
**Validator detection.** Before falling back to UNVERIFIABLE on a CONTENT-SHAPE item, scan the target repo's `package.json` for any script matching `validate-*`, `lint-wiki`, `check-docs`, or similar. If found, invoke it with the relevant path argument (e.g., `npm run validate-wiki -- <path>`). For multi-target validators (e.g., `validate-wiki --all`), run once and reconcile per-item from the output. A passing validator promotes the item from UNVERIFIABLE to DONE; a failing one demotes to NOT DONE.
|
||||
|
||||
**Honesty rule.** Do NOT classify an item as DONE just because related code shipped. Code that *handles* a deliverable is not the deliverable. Shipping a markdown-extraction library is not the same as shipping the markdown file. When in doubt between DONE and UNVERIFIABLE, prefer UNVERIFIABLE — better to surface a confirmation prompt than silently miss a deliverable.
|
||||
|
||||
### Cross-Reference Against Diff
|
||||
|
||||
Run `git diff origin/<base>...HEAD` and `git log origin/<base>..HEAD --oneline` to understand what was implemented.
|
||||
|
||||
For each extracted plan item, run the verification dispatch from the previous section, then classify:
|
||||
|
||||
- **DONE** — Clear evidence the item shipped. Cite the specific file(s) changed in the diff for DIFF-VERIFIABLE items, or the verified path that exists for CROSS-REPO items with a reachable sibling repo.
|
||||
- **PARTIAL** — Some work toward this item exists but is incomplete (e.g., model created but controller missing, function exists but edge cases not handled).
|
||||
- **NOT DONE** — Verification ran and produced negative evidence (file missing, code absent in diff, sibling-repo file confirmed absent).
|
||||
- **CHANGED** — The item was implemented using a different approach than the plan described, but the same goal is achieved. Note the difference.
|
||||
- **UNVERIFIABLE** — The diff and any reachable sibling-repo checks cannot prove or disprove this. Always applies to EXTERNAL-STATE items and to CROSS-REPO items where the sibling repo isn't reachable. Cite the specific manual verification the user must perform (e.g., "check Cloudflare DNS shows DNS-only mode for dashboard.example.com", "confirm /docs/dashboard.md exists in domain-hq repo").
|
||||
|
||||
**Be conservative with DONE** — require clear evidence. A file being touched is not enough; the specific functionality described must be present.
|
||||
**Be generous with CHANGED** — if the goal is met by different means, that counts as addressed.
|
||||
**Be honest with UNVERIFIABLE** — better to surface 5 items the user must manually confirm than silently classify them DONE.
|
||||
|
||||
### Output Format
|
||||
|
||||
```
|
||||
PLAN COMPLETION AUDIT
|
||||
═══════════════════════════════
|
||||
Plan: {plan file path}
|
||||
|
||||
## Implementation Items
|
||||
[DONE] Create UserService — src/services/user_service.rb (+142 lines)
|
||||
[PARTIAL] Add validation — model validates but missing controller checks
|
||||
[NOT DONE] Add caching layer — no cache-related changes in diff
|
||||
[CHANGED] "Redis queue" → implemented with Sidekiq instead
|
||||
|
||||
## Test Items
|
||||
[DONE] Unit tests for UserService — test/services/user_service_test.rb
|
||||
[NOT DONE] E2E test for signup flow
|
||||
|
||||
## Migration Items
|
||||
[DONE] Create users table — db/migrate/20240315_create_users.rb
|
||||
|
||||
## Cross-Repo / External Items
|
||||
[DONE] sibling-repo has /docs/dashboard.md — verified at ~/Development/sibling-repo/docs/dashboard.md
|
||||
[UNVERIFIABLE] Cloudflare DNS-only on api.example.com — external system, manual check required
|
||||
[UNVERIFIABLE] Supabase auth allowlist contains user email — external system, confirm in Supabase dashboard
|
||||
|
||||
─────────────────────────────────
|
||||
COMPLETION: 5/9 DONE, 1 PARTIAL, 1 NOT DONE, 1 CHANGED, 2 UNVERIFIABLE
|
||||
─────────────────────────────────
|
||||
```
|
||||
|
||||
### Gate Logic
|
||||
|
||||
After producing the completion checklist, evaluate in priority order:
|
||||
|
||||
1. **Any NOT DONE items** (highest priority — known missing work). Use AskUserQuestion:
|
||||
- Show the completion checklist above
|
||||
- "{N} items from the plan are NOT DONE. These were part of the original plan but are missing from the implementation."
|
||||
- RECOMMENDATION: depends on item count and severity. If 1-2 minor items (docs, config), recommend B. If core functionality is missing, recommend A.
|
||||
- Options:
|
||||
A) Stop — implement the missing items before shipping
|
||||
B) Ship anyway — defer these to a follow-up (will create P1 TODOs in Step 5.5)
|
||||
C) These items were intentionally dropped — remove from scope
|
||||
- If A: STOP. List the missing items for the user to implement.
|
||||
- If B: Continue. For each NOT DONE item, create a P1 TODO in Step 5.5 with "Deferred from plan: {plan file path}".
|
||||
- If C: Continue. Note in PR body: "Plan items intentionally dropped: {list}."
|
||||
|
||||
2. **Any UNVERIFIABLE items** (silent gaps — the diff cannot prove them either way). Only fires after NOT DONE is resolved or absent.
|
||||
|
||||
**Per-item confirmation is mandatory.** Do NOT use a single AskUserQuestion to blanket-confirm all UNVERIFIABLE items. Blanket confirmation is the failure mode that surfaced in VAS-449 (user clicks A without opening any file). Instead:
|
||||
|
||||
- Loop through UNVERIFIABLE items one at a time.
|
||||
- For each item, use AskUserQuestion with the item's *specific* manual check (e.g., "Confirm: does `~/Development/domain-hq/docs/dashboard.md` exist?", not "Have you checked all items?").
|
||||
- Options per item:
|
||||
Y) Confirmed done — cite what you verified (free-text, embedded in PR body)
|
||||
N) Not done — block ship; treat as NOT DONE and re-enter the priority-1 gate
|
||||
D) Intentionally dropped — note in PR body: "Plan item intentionally dropped: {item}"
|
||||
- RECOMMENDATION per item: Y if the item is concrete and easily verified; N if it's critical-path (auth, DNS, deliverables to other repos) and the user shows hesitation.
|
||||
|
||||
**Exit conditions:**
|
||||
- Any N: STOP. Surface the missing items, suggest re-running /ship after they're addressed.
|
||||
- All Y or D: Continue. Embed `## Plan Completion — Manual Verifications` section in PR body listing each Y'd item with the user's free-text evidence and each D'd item with "intentionally dropped".
|
||||
|
||||
**Cap.** If there are more than 5 UNVERIFIABLE items, present them as a numbered list first and ask whether the user wants to (1) confirm each individually, (2) stop and reduce scope, or (3) explicitly accept blanket-confirmation with the warning that this is the VAS-449 failure shape. Default and recommended option is (1).
|
||||
|
||||
3. **Only PARTIAL items (no NOT DONE, no UNVERIFIABLE):** Continue with a note in the PR body. Not blocking.
|
||||
|
||||
4. **All DONE or CHANGED:** Pass. "Plan completion: PASS — all items addressed." Continue.
|
||||
|
||||
**No plan file found:** Skip entirely. "No plan file detected — skipping plan completion audit."
|
||||
|
||||
**Include in PR body (Step 8):** Add a `## Plan Completion` section with the checklist summary.
|
||||
>
|
||||
> After your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):
|
||||
> `{"total_items":N,"done":N,"changed":N,"deferred":N,"unverifiable":N,"summary":"<markdown checklist for PR body>"}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
1. Parse the LAST line of the subagent's output as JSON.
|
||||
2. Store `done`, `deferred`, `unverifiable` for Step 20 metrics; use `summary` in PR body.
|
||||
3. If `deferred > 0` or `unverifiable > 0` and no user override, present the items via the appropriate AskUserQuestion (see Gate Logic priority order above) before continuing.
|
||||
4. Embed `summary` in PR body's `## Plan Completion` section (Step 19). If `unverifiable > 0` and the user picked option A in the UNVERIFIABLE gate, also embed `## Plan Completion — Manual Verifications` listing each user-confirmed item.
|
||||
|
||||
**If the subagent fails or returns invalid JSON:** Fall back to running the audit inline (parent processes the same plan-extraction + classification logic). If the inline fallback also fails (e.g., plan file unreadable, parser error), do NOT silently pass — surface the failure as an explicit AskUserQuestion: "Plan Completion audit could not run ({reason}). Options: (A) Skip audit and ship anyway — record that the audit was skipped in PR body and Step 20 metrics; (B) Stop and fix the audit." Default and recommended option is (B). Silent fail-open is the failure shape that VAS-449 surfaced.
|
||||
|
||||
---
|
||||
|
||||
## Step 8.1: Plan Verification
|
||||
|
||||
Automatically verify the plan's testing/verification steps using the `/qa-only` skill.
|
||||
|
||||
### 1. Check for verification section
|
||||
|
||||
Using the plan file already discovered in Step 8, look for a verification section. Match any of these headings: `## Verification`, `## Test plan`, `## Testing`, `## How to test`, `## Manual testing`, or any section with verification-flavored items (URLs to visit, things to check visually, interactions to test).
|
||||
|
||||
**If no verification section found:** Skip with "No verification steps found in plan — skipping auto-verification."
|
||||
**If no plan file was found in Step 8:** Skip (already handled).
|
||||
|
||||
### 2. Check for running dev server
|
||||
|
||||
Before invoking browse-based verification, check if a dev server is reachable:
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}' http://localhost:3000 2>/dev/null || \
|
||||
curl -s -o /dev/null -w '%{http_code}' http://localhost:8080 2>/dev/null || \
|
||||
curl -s -o /dev/null -w '%{http_code}' http://localhost:5173 2>/dev/null || \
|
||||
curl -s -o /dev/null -w '%{http_code}' http://localhost:4000 2>/dev/null || echo "NO_SERVER"
|
||||
```
|
||||
|
||||
**If NO_SERVER:** Skip with "No dev server detected — skipping plan verification. Run /qa separately after deploying."
|
||||
|
||||
### 3. Invoke /qa-only inline
|
||||
|
||||
Read the `/qa-only` skill from disk:
|
||||
|
||||
```bash
|
||||
cat ${CLAUDE_SKILL_DIR}/../qa-only/SKILL.md
|
||||
```
|
||||
|
||||
**If unreadable:** Skip with "Could not load /qa-only — skipping plan verification."
|
||||
|
||||
Follow the /qa-only workflow with these modifications:
|
||||
- **Skip the preamble** (already handled by /ship)
|
||||
- **Use the plan's verification section as the primary test input** — treat each verification item as a test case
|
||||
- **Use the detected dev server URL** as the base URL
|
||||
- **Skip the fix loop** — this is report-only verification during /ship
|
||||
- **Cap at the verification items from the plan** — do not expand into general site QA
|
||||
|
||||
### 4. Gate logic
|
||||
|
||||
- **All verification items PASS:** Continue silently. "Plan verification: PASS."
|
||||
- **Any FAIL:** Use AskUserQuestion:
|
||||
- Show the failures with screenshot evidence
|
||||
- RECOMMENDATION: Choose A if failures indicate broken functionality. Choose B if cosmetic only.
|
||||
- Options:
|
||||
A) Fix the failures before shipping (recommended for functional issues)
|
||||
B) Ship anyway — known issues (acceptable for cosmetic issues)
|
||||
- **No verification section / no server / unreadable skill:** Skip (non-blocking).
|
||||
|
||||
### 5. Include in PR body
|
||||
|
||||
Add a `## Verification Results` section to the PR body (Step 19):
|
||||
- If verification ran: summary of results (N PASS, M FAIL, K SKIPPED)
|
||||
- If skipped: reason for skipping (no plan, no server, no verification section)
|
||||
|
||||
## Prior Learnings
|
||||
|
||||
Search for relevant learnings from previous sessions:
|
||||
|
||||
```bash
|
||||
_CROSS_PROJ=$(~/.claude/skills/gstack/bin/gstack-config get cross_project_learnings 2>/dev/null || echo "unset")
|
||||
echo "CROSS_PROJECT: $_CROSS_PROJ"
|
||||
if [ "$_CROSS_PROJ" = "true" ]; then
|
||||
~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 --query "release ship version changelog merge pr" --cross-project 2>/dev/null || true
|
||||
else
|
||||
~/.claude/skills/gstack/bin/gstack-learnings-search --limit 10 --query "release ship version changelog merge pr" 2>/dev/null || true
|
||||
fi
|
||||
```
|
||||
|
||||
If `CROSS_PROJECT` is `unset` (first time): Use AskUserQuestion:
|
||||
|
||||
> gstack can search learnings from your other projects on this machine to find
|
||||
> patterns that might apply here. This stays local (no data leaves your machine).
|
||||
> Recommended for solo developers. Skip if you work on multiple client codebases
|
||||
> where cross-contamination would be a concern.
|
||||
|
||||
Options:
|
||||
- A) Enable cross-project learnings (recommended)
|
||||
- B) Keep learnings project-scoped only
|
||||
|
||||
If A: run `~/.claude/skills/gstack/bin/gstack-config set cross_project_learnings true`
|
||||
If B: run `~/.claude/skills/gstack/bin/gstack-config set cross_project_learnings false`
|
||||
|
||||
Then re-run the search with the appropriate flag.
|
||||
|
||||
If learnings are found, incorporate them into your analysis. When a review finding
|
||||
matches a past learning, display:
|
||||
|
||||
**"Prior learning applied: [key] (confidence N/10, from [date])"**
|
||||
|
||||
This makes the compounding visible. The user should see that gstack is getting
|
||||
smarter on their codebase over time.
|
||||
|
||||
## Step 8.2: Scope Drift Detection
|
||||
|
||||
Before reviewing code quality, check: **did they build what was requested — nothing more, nothing less?**
|
||||
|
||||
1. Read `TODOS.md` (if it exists). Read PR description (`gh pr view --json body --jq .body 2>/dev/null || true`).
|
||||
Read commit messages (`git log origin/<base>..HEAD --oneline`).
|
||||
**If no PR exists:** rely on commit messages and TODOS.md for stated intent — this is the common case since /review runs before /ship creates the PR.
|
||||
2. Identify the **stated intent** — what was this branch supposed to accomplish?
|
||||
3. Run `DIFF_BASE=$(git merge-base origin/<base> HEAD) && git diff "$DIFF_BASE" --stat` and compare the files changed against the stated intent.
|
||||
|
||||
4. Evaluate with skepticism (incorporating plan completion results if available from an earlier step or adjacent section):
|
||||
|
||||
**SCOPE CREEP detection:**
|
||||
- Files changed that are unrelated to the stated intent
|
||||
- New features or refactors not mentioned in the plan
|
||||
- "While I was in there..." changes that expand blast radius
|
||||
|
||||
**MISSING REQUIREMENTS detection:**
|
||||
- Requirements from TODOS.md/PR description not addressed in the diff
|
||||
- Test coverage gaps for stated requirements
|
||||
- Partial implementations (started but not finished)
|
||||
|
||||
5. Output (before the main review begins):
|
||||
\`\`\`
|
||||
Scope Check: [CLEAN / DRIFT DETECTED / REQUIREMENTS MISSING]
|
||||
Intent: <1-line summary of what was requested>
|
||||
Delivered: <1-line summary of what the diff actually does>
|
||||
[If drift: list each out-of-scope change]
|
||||
[If missing: list each unaddressed requirement]
|
||||
\`\`\`
|
||||
|
||||
6. This is **INFORMATIONAL** — does not block the review. Proceed to the next step.
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
@@ -0,0 +1,31 @@
|
||||
## Step 8: Plan Completion Audit
|
||||
|
||||
**Dispatch this step as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent reads the plan file and every referenced code file in its own fresh context. Parent gets only the conclusion.
|
||||
|
||||
**Subagent prompt:** Pass these instructions to the subagent:
|
||||
|
||||
> You are running a ship-workflow plan completion audit. The base branch is `<base>`. Use `git diff <base>...HEAD` to see what shipped. Do not commit or push — report only.
|
||||
>
|
||||
> {{PLAN_COMPLETION_AUDIT_SHIP}}
|
||||
>
|
||||
> After your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):
|
||||
> `{"total_items":N,"done":N,"changed":N,"deferred":N,"unverifiable":N,"summary":"<markdown checklist for PR body>"}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
1. Parse the LAST line of the subagent's output as JSON.
|
||||
2. Store `done`, `deferred`, `unverifiable` for Step 20 metrics; use `summary` in PR body.
|
||||
3. If `deferred > 0` or `unverifiable > 0` and no user override, present the items via the appropriate AskUserQuestion (see Gate Logic priority order above) before continuing.
|
||||
4. Embed `summary` in PR body's `## Plan Completion` section (Step 19). If `unverifiable > 0` and the user picked option A in the UNVERIFIABLE gate, also embed `## Plan Completion — Manual Verifications` listing each user-confirmed item.
|
||||
|
||||
**If the subagent fails or returns invalid JSON:** Fall back to running the audit inline (parent processes the same plan-extraction + classification logic). If the inline fallback also fails (e.g., plan file unreadable, parser error), do NOT silently pass — surface the failure as an explicit AskUserQuestion: "Plan Completion audit could not run ({reason}). Options: (A) Skip audit and ship anyway — record that the audit was skipped in PR body and Step 20 metrics; (B) Stop and fix the audit." Default and recommended option is (B). Silent fail-open is the failure shape that VAS-449 surfaced.
|
||||
|
||||
---
|
||||
|
||||
{{PLAN_VERIFICATION_EXEC}}
|
||||
|
||||
{{LEARNINGS_SEARCH:query=release ship version changelog merge pr}}
|
||||
|
||||
{{SCOPE_DRIFT}}
|
||||
|
||||
---
|
||||
@@ -0,0 +1,207 @@
|
||||
<!-- AUTO-GENERATED from pr-body.md.tmpl — do not edit directly -->
|
||||
<!-- Regenerate: bun run gen:skill-docs -->
|
||||
## Step 18: Documentation sync (via subagent, before PR creation)
|
||||
|
||||
**Dispatch /document-release as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent gets a fresh context window — zero rot from the preceding 17 steps. It also runs the **full** `/document-release` workflow (with CHANGELOG clobber protection, doc exclusions, risky-change gates, named staging, race-safe PR body editing) rather than a weaker reimplementation.
|
||||
|
||||
**Sequencing:** This step runs AFTER Step 17 (Push) and BEFORE Step 19 (Create PR). The PR is created once from final HEAD with the `## Documentation` section baked into the initial body. No create-then-re-edit dance.
|
||||
|
||||
**Subagent prompt:**
|
||||
|
||||
> You are executing the /document-release workflow after a code push. Read the full skill file `${HOME}/.claude/skills/gstack/document-release/SKILL.md` and execute its complete workflow end-to-end, including CHANGELOG clobber protection, doc exclusions, risky-change gates, and named staging. Do NOT attempt to edit the PR body — no PR exists yet. Branch: `<branch>`, base: `<base>`.
|
||||
>
|
||||
> After completing the workflow, output a single JSON object on the LAST LINE of your response (no other text after it):
|
||||
> `{"files_updated":["README.md","CLAUDE.md",...],"commit_sha":"abc1234","pushed":true,"documentation_section":"<markdown block for PR body's ## Documentation section>"}`
|
||||
>
|
||||
> If no documentation files needed updating, output:
|
||||
> `{"files_updated":[],"commit_sha":null,"pushed":false,"documentation_section":null}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
1. Parse the LAST line of the subagent's output as JSON.
|
||||
2. Store `documentation_section` — Step 19 embeds it in the PR body (or omits the section if null).
|
||||
3. If `files_updated` is non-empty, print: `Documentation synced: {files_updated.length} files updated, committed as {commit_sha}`.
|
||||
4. If `files_updated` is empty, print: `Documentation is current — no updates needed.`
|
||||
|
||||
**If the subagent fails or returns invalid JSON:** Print a warning and proceed to Step 19 without a `## Documentation` section. Do not block /ship on subagent failure. The user can run `/document-release` manually after the PR lands.
|
||||
|
||||
---
|
||||
|
||||
## Step 19: Create PR/MR
|
||||
|
||||
**Idempotency check:** Check if a PR/MR already exists for this branch.
|
||||
|
||||
**If GitHub:**
|
||||
```bash
|
||||
gh pr view --json url,number,state -q 'if .state == "OPEN" then "PR #\(.number): \(.url)" else "NO_PR" end' 2>/dev/null || echo "NO_PR"
|
||||
```
|
||||
|
||||
**If GitLab:**
|
||||
```bash
|
||||
glab mr view -F json 2>/dev/null | jq -r 'if .state == "opened" then "MR_EXISTS" else "NO_MR" end' 2>/dev/null || echo "NO_MR"
|
||||
```
|
||||
|
||||
If an **open** PR/MR already exists: **update** the PR body using `gh pr edit --body-file "$PR_BODY_FILE"` (GitHub) or `glab mr update -d ...` (GitLab). Always regenerate the PR body from scratch using this run's fresh results (test output, coverage audit, review findings, adversarial review, TODOS summary, documentation_section from Step 18). Never reuse stale PR body content from a prior run. **Run the same redaction scan-at-sink (PR body + title) as the create path (Step 19) before editing — scan the temp file, then `gh pr edit --body-file` from it.**
|
||||
|
||||
**Always update the PR title to start with `v$NEW_VERSION`.** PR titles use the workspace-aware format `v<NEW_VERSION> <type>: <summary>` — version ALWAYS first, no exceptions, no "custom title kept intentionally" escape hatch. The shared helper `bin/gstack-pr-title-rewrite.sh` is the single source of truth for the rule.
|
||||
|
||||
1. Read the current title: `CURRENT=$(gh pr view --json title -q .title)` (or `glab mr view -F json | jq -r .title`).
|
||||
2. Compute the corrected title: `NEW_TITLE=$(~/.claude/skills/gstack/bin/gstack-pr-title-rewrite.sh "$NEW_VERSION" "$CURRENT")`. The helper handles three cases: title already correct (no-op), title has a different `v<X.Y.Z.W>` prefix (replace it), or title has no version prefix (prepend one).
|
||||
3. If `NEW_TITLE` differs from `CURRENT`, run `gh pr edit --title "$NEW_TITLE"` (or `glab mr update -t "$NEW_TITLE"`).
|
||||
4. **Self-check:** re-fetch the title and assert it starts with `v$NEW_VERSION `. If it does not, retry the edit once. If still wrong, surface the failure to the user.
|
||||
|
||||
This keeps the title truthful when Step 12's queue-drift detection rebumps a stale version, and forces the format on PRs that were created without it.
|
||||
|
||||
Print the existing URL and continue to Step 20.
|
||||
|
||||
If no PR/MR exists: create a pull request (GitHub) or merge request (GitLab) using the platform detected in Step 0.
|
||||
|
||||
The PR/MR body should contain these sections:
|
||||
|
||||
```
|
||||
## Summary
|
||||
<Summarize ALL changes being shipped. Run `git log <base>..HEAD --oneline` to enumerate
|
||||
every commit. Exclude the VERSION/CHANGELOG metadata commit (that's this PR's bookkeeping,
|
||||
not a substantive change). Group the remaining commits into logical sections (e.g.,
|
||||
"**Performance**", "**Dead Code Removal**", "**Infrastructure**"). Every substantive commit
|
||||
must appear in at least one section. If a commit's work isn't reflected in the summary,
|
||||
you missed it.>
|
||||
|
||||
## Test Coverage
|
||||
<coverage diagram from Step 7, or "All new code paths have test coverage.">
|
||||
<If Step 7 ran: "Tests: {before} → {after} (+{delta} new)">
|
||||
|
||||
## Pre-Landing Review
|
||||
<findings from Step 9 code review, or "No issues found.">
|
||||
|
||||
## Design Review
|
||||
<If design review ran: "Design Review (lite): N findings — M auto-fixed, K skipped. AI Slop: clean/N issues.">
|
||||
<If no frontend files changed: "No frontend files changed — design review skipped.">
|
||||
|
||||
## Eval Results
|
||||
<If evals ran: suite names, pass/fail counts, cost dashboard summary. If skipped: "No prompt-related files changed — evals skipped.">
|
||||
|
||||
## Greptile Review
|
||||
<If Greptile comments were found: bullet list with [FIXED] / [FALSE POSITIVE] / [ALREADY FIXED] tag + one-line summary per comment>
|
||||
<If no Greptile comments found: "No Greptile comments.">
|
||||
<If no PR existed during Step 10: omit this section entirely>
|
||||
|
||||
## Scope Drift
|
||||
<If scope drift ran: "Scope Check: CLEAN" or list of drift/creep findings>
|
||||
<If no scope drift: omit this section>
|
||||
|
||||
## Plan Completion
|
||||
<If plan file found: completion checklist summary from Step 8>
|
||||
<If no plan file: "No plan file detected.">
|
||||
<If plan items deferred: list deferred items>
|
||||
|
||||
## Linked Spec
|
||||
<Auto-detect: look for /spec archives matching this branch via:
|
||||
eval "$(${ctx.paths.binDir}/gstack-paths)"
|
||||
eval "$(${ctx.paths.binDir}/gstack-slug)"
|
||||
CURRENT_BRANCH=$(git branch --show-current)
|
||||
SPEC_ARCHIVES="$GSTACK_STATE_ROOT/projects/$SLUG/specs"
|
||||
# Find newest archive whose spec_branch frontmatter matches current branch (or one of its
|
||||
# parents — if spec spawned worktree spec/<slug>-$$, the spawned worktree IS where /ship runs).
|
||||
SPEC_FILE=$(grep -l "^spec_branch: $CURRENT_BRANCH$" "$SPEC_ARCHIVES"/*.md 2>/dev/null | head -1)
|
||||
[ -z "$SPEC_FILE" ] && exit # no spec; omit this section entirely
|
||||
SPEC_ISSUE=$(grep "^spec_issue_number:" "$SPEC_FILE" | cut -d' ' -f2)
|
||||
[ -z "$SPEC_ISSUE" ] && exit # spec archive exists but no issue number; omit
|
||||
|
||||
# CONDITIONAL Closes #N (codex F4): only add when Plan Completion above is "complete".
|
||||
# If the plan completion gate from Step 8 reports any deferred or failed items, emit:
|
||||
# "Linked to #$SPEC_ISSUE (partial delivery — NOT auto-closing; close manually after follow-up)"
|
||||
# If Plan Completion is fully complete, emit:
|
||||
# "Closes #$SPEC_ISSUE"
|
||||
# and include the Closes #N line in the PR body so GitHub auto-closes on merge.>
|
||||
|
||||
<Format:
|
||||
Closes #<N>
|
||||
|
||||
This PR delivers the spec at <archive path relative to repo root>.
|
||||
Spec filed: <spec_filed_at from frontmatter>>
|
||||
|
||||
<If partial delivery, emit instead:
|
||||
Linked to #<N> (partial delivery — not auto-closing).
|
||||
Deferred items: <list from Plan Completion>.
|
||||
Close #<N> manually after follow-up lands.>
|
||||
|
||||
<If no /spec archive matches this branch: omit this entire section.>
|
||||
|
||||
## Verification Results
|
||||
<If verification ran: summary from Step 8.1 (N PASS, M FAIL, K SKIPPED)>
|
||||
<If skipped: reason (no plan, no server, no verification section)>
|
||||
<If not applicable: omit this section>
|
||||
|
||||
## TODOS
|
||||
<If items marked complete: bullet list of completed items with version>
|
||||
<If no items completed: "No TODO items completed in this PR.">
|
||||
<If TODOS.md created or reorganized: note that>
|
||||
<If TODOS.md doesn't exist and user skipped: omit this section>
|
||||
|
||||
## Documentation
|
||||
<Embed the `documentation_section` string returned by Step 18's subagent here, verbatim.>
|
||||
<If Step 18 returned `documentation_section: null` (no docs updated), omit this section entirely.>
|
||||
|
||||
## Test plan
|
||||
- [x] All Rails tests pass (N runs, 0 failures)
|
||||
- [x] All Vitest tests pass (N tests)
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||||
```
|
||||
|
||||
#### Redaction scan (PR body + title) — runs before create AND edit
|
||||
|
||||
The PR body is world-readable on a public repo. Scan-at-sink before sending:
|
||||
write the composed body to a temp file, scan THAT file with the shared engine,
|
||||
and pass the same file to `gh`/`glab`. Wrap any Codex / Greptile / eval output
|
||||
sections in tool-attributed fences (` ```codex-review ` / ` ```greptile `) so the
|
||||
engine WARN-degrades the example credentials those tools quote instead of blocking
|
||||
the PR (a live-format credential inside the fence still blocks).
|
||||
|
||||
```bash
|
||||
REDACT_VIS=$(~/.claude/skills/gstack/bin/gstack-config get redact_repo_visibility 2>/dev/null)
|
||||
[ -z "$REDACT_VIS" ] && REDACT_VIS=$(gh repo view --json visibility -q .visibility 2>/dev/null | tr 'A-Z' 'a-z')
|
||||
REDACT_VIS="${REDACT_VIS:-unknown}"
|
||||
PR_BODY_FILE=$(mktemp)
|
||||
cat > "$PR_BODY_FILE" <<'PR_BODY_EOF'
|
||||
<PR body from above>
|
||||
PR_BODY_EOF
|
||||
~/.claude/skills/gstack/bin/gstack-redact --from-file "$PR_BODY_FILE" --repo-visibility "$REDACT_VIS" --self-email "$(git config user.email 2>/dev/null)" --json
|
||||
case $? in
|
||||
3) echo "BLOCKED — credential in PR body. Rotate + redact, do not create the PR."; exit 1 ;;
|
||||
2) echo "MEDIUM findings — confirm per finding (sterner on public) before proceeding." ;;
|
||||
esac
|
||||
# Also scan the title (short, single-line):
|
||||
printf '%s' "v$NEW_VERSION <type>: <summary>" | ~/.claude/skills/gstack/bin/gstack-redact --repo-visibility "$REDACT_VIS" --json
|
||||
```
|
||||
|
||||
HIGH blocks (exit 3, no skip). MEDIUM → AskUserQuestion (PII subset offers
|
||||
`--auto-redact`). Same scan runs before the `gh pr edit --body` path (Step 17).
|
||||
|
||||
**If GitHub:** create from the SCANNED file (exact bytes scanned = bytes sent):
|
||||
|
||||
```bash
|
||||
# PR title MUST start with v$NEW_VERSION — enforced on every run, no exceptions.
|
||||
# (See Step 19 idempotency block + bin/gstack-pr-title-rewrite.sh for the rule.)
|
||||
gh pr create --base <base> --title "v$NEW_VERSION <type>: <summary>" --body-file "$PR_BODY_FILE"
|
||||
rm -f "$PR_BODY_FILE"
|
||||
```
|
||||
|
||||
**If GitLab:**
|
||||
|
||||
```bash
|
||||
# MR title MUST start with v$NEW_VERSION — enforced on every run, no exceptions.
|
||||
# (See Step 19 idempotency block + bin/gstack-pr-title-rewrite.sh for the rule.)
|
||||
glab mr create -b <base> -t "v$NEW_VERSION <type>: <summary>" -d "$(cat <<'EOF'
|
||||
<MR body from above>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
**If neither CLI is available:**
|
||||
Print the branch name, remote URL, and instruct the user to create the PR/MR manually via the web UI. Do not stop — the code is pushed and ready.
|
||||
|
||||
**Output the PR/MR URL** — then proceed to Step 20.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,205 @@
|
||||
## Step 18: Documentation sync (via subagent, before PR creation)
|
||||
|
||||
**Dispatch /document-release as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent gets a fresh context window — zero rot from the preceding 17 steps. It also runs the **full** `/document-release` workflow (with CHANGELOG clobber protection, doc exclusions, risky-change gates, named staging, race-safe PR body editing) rather than a weaker reimplementation.
|
||||
|
||||
**Sequencing:** This step runs AFTER Step 17 (Push) and BEFORE Step 19 (Create PR). The PR is created once from final HEAD with the `## Documentation` section baked into the initial body. No create-then-re-edit dance.
|
||||
|
||||
**Subagent prompt:**
|
||||
|
||||
> You are executing the /document-release workflow after a code push. Read the full skill file `${HOME}/.claude/skills/gstack/document-release/SKILL.md` and execute its complete workflow end-to-end, including CHANGELOG clobber protection, doc exclusions, risky-change gates, and named staging. Do NOT attempt to edit the PR body — no PR exists yet. Branch: `<branch>`, base: `<base>`.
|
||||
>
|
||||
> After completing the workflow, output a single JSON object on the LAST LINE of your response (no other text after it):
|
||||
> `{"files_updated":["README.md","CLAUDE.md",...],"commit_sha":"abc1234","pushed":true,"documentation_section":"<markdown block for PR body's ## Documentation section>"}`
|
||||
>
|
||||
> If no documentation files needed updating, output:
|
||||
> `{"files_updated":[],"commit_sha":null,"pushed":false,"documentation_section":null}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
1. Parse the LAST line of the subagent's output as JSON.
|
||||
2. Store `documentation_section` — Step 19 embeds it in the PR body (or omits the section if null).
|
||||
3. If `files_updated` is non-empty, print: `Documentation synced: {files_updated.length} files updated, committed as {commit_sha}`.
|
||||
4. If `files_updated` is empty, print: `Documentation is current — no updates needed.`
|
||||
|
||||
**If the subagent fails or returns invalid JSON:** Print a warning and proceed to Step 19 without a `## Documentation` section. Do not block /ship on subagent failure. The user can run `/document-release` manually after the PR lands.
|
||||
|
||||
---
|
||||
|
||||
## Step 19: Create PR/MR
|
||||
|
||||
**Idempotency check:** Check if a PR/MR already exists for this branch.
|
||||
|
||||
**If GitHub:**
|
||||
```bash
|
||||
gh pr view --json url,number,state -q 'if .state == "OPEN" then "PR #\(.number): \(.url)" else "NO_PR" end' 2>/dev/null || echo "NO_PR"
|
||||
```
|
||||
|
||||
**If GitLab:**
|
||||
```bash
|
||||
glab mr view -F json 2>/dev/null | jq -r 'if .state == "opened" then "MR_EXISTS" else "NO_MR" end' 2>/dev/null || echo "NO_MR"
|
||||
```
|
||||
|
||||
If an **open** PR/MR already exists: **update** the PR body using `gh pr edit --body-file "$PR_BODY_FILE"` (GitHub) or `glab mr update -d ...` (GitLab). Always regenerate the PR body from scratch using this run's fresh results (test output, coverage audit, review findings, adversarial review, TODOS summary, documentation_section from Step 18). Never reuse stale PR body content from a prior run. **Run the same redaction scan-at-sink (PR body + title) as the create path (Step 19) before editing — scan the temp file, then `gh pr edit --body-file` from it.**
|
||||
|
||||
**Always update the PR title to start with `v$NEW_VERSION`.** PR titles use the workspace-aware format `v<NEW_VERSION> <type>: <summary>` — version ALWAYS first, no exceptions, no "custom title kept intentionally" escape hatch. The shared helper `bin/gstack-pr-title-rewrite.sh` is the single source of truth for the rule.
|
||||
|
||||
1. Read the current title: `CURRENT=$(gh pr view --json title -q .title)` (or `glab mr view -F json | jq -r .title`).
|
||||
2. Compute the corrected title: `NEW_TITLE=$(~/.claude/skills/gstack/bin/gstack-pr-title-rewrite.sh "$NEW_VERSION" "$CURRENT")`. The helper handles three cases: title already correct (no-op), title has a different `v<X.Y.Z.W>` prefix (replace it), or title has no version prefix (prepend one).
|
||||
3. If `NEW_TITLE` differs from `CURRENT`, run `gh pr edit --title "$NEW_TITLE"` (or `glab mr update -t "$NEW_TITLE"`).
|
||||
4. **Self-check:** re-fetch the title and assert it starts with `v$NEW_VERSION `. If it does not, retry the edit once. If still wrong, surface the failure to the user.
|
||||
|
||||
This keeps the title truthful when Step 12's queue-drift detection rebumps a stale version, and forces the format on PRs that were created without it.
|
||||
|
||||
Print the existing URL and continue to Step 20.
|
||||
|
||||
If no PR/MR exists: create a pull request (GitHub) or merge request (GitLab) using the platform detected in Step 0.
|
||||
|
||||
The PR/MR body should contain these sections:
|
||||
|
||||
```
|
||||
## Summary
|
||||
<Summarize ALL changes being shipped. Run `git log <base>..HEAD --oneline` to enumerate
|
||||
every commit. Exclude the VERSION/CHANGELOG metadata commit (that's this PR's bookkeeping,
|
||||
not a substantive change). Group the remaining commits into logical sections (e.g.,
|
||||
"**Performance**", "**Dead Code Removal**", "**Infrastructure**"). Every substantive commit
|
||||
must appear in at least one section. If a commit's work isn't reflected in the summary,
|
||||
you missed it.>
|
||||
|
||||
## Test Coverage
|
||||
<coverage diagram from Step 7, or "All new code paths have test coverage.">
|
||||
<If Step 7 ran: "Tests: {before} → {after} (+{delta} new)">
|
||||
|
||||
## Pre-Landing Review
|
||||
<findings from Step 9 code review, or "No issues found.">
|
||||
|
||||
## Design Review
|
||||
<If design review ran: "Design Review (lite): N findings — M auto-fixed, K skipped. AI Slop: clean/N issues.">
|
||||
<If no frontend files changed: "No frontend files changed — design review skipped.">
|
||||
|
||||
## Eval Results
|
||||
<If evals ran: suite names, pass/fail counts, cost dashboard summary. If skipped: "No prompt-related files changed — evals skipped.">
|
||||
|
||||
## Greptile Review
|
||||
<If Greptile comments were found: bullet list with [FIXED] / [FALSE POSITIVE] / [ALREADY FIXED] tag + one-line summary per comment>
|
||||
<If no Greptile comments found: "No Greptile comments.">
|
||||
<If no PR existed during Step 10: omit this section entirely>
|
||||
|
||||
## Scope Drift
|
||||
<If scope drift ran: "Scope Check: CLEAN" or list of drift/creep findings>
|
||||
<If no scope drift: omit this section>
|
||||
|
||||
## Plan Completion
|
||||
<If plan file found: completion checklist summary from Step 8>
|
||||
<If no plan file: "No plan file detected.">
|
||||
<If plan items deferred: list deferred items>
|
||||
|
||||
## Linked Spec
|
||||
<Auto-detect: look for /spec archives matching this branch via:
|
||||
eval "$(${ctx.paths.binDir}/gstack-paths)"
|
||||
eval "$(${ctx.paths.binDir}/gstack-slug)"
|
||||
CURRENT_BRANCH=$(git branch --show-current)
|
||||
SPEC_ARCHIVES="$GSTACK_STATE_ROOT/projects/$SLUG/specs"
|
||||
# Find newest archive whose spec_branch frontmatter matches current branch (or one of its
|
||||
# parents — if spec spawned worktree spec/<slug>-$$, the spawned worktree IS where /ship runs).
|
||||
SPEC_FILE=$(grep -l "^spec_branch: $CURRENT_BRANCH$" "$SPEC_ARCHIVES"/*.md 2>/dev/null | head -1)
|
||||
[ -z "$SPEC_FILE" ] && exit # no spec; omit this section entirely
|
||||
SPEC_ISSUE=$(grep "^spec_issue_number:" "$SPEC_FILE" | cut -d' ' -f2)
|
||||
[ -z "$SPEC_ISSUE" ] && exit # spec archive exists but no issue number; omit
|
||||
|
||||
# CONDITIONAL Closes #N (codex F4): only add when Plan Completion above is "complete".
|
||||
# If the plan completion gate from Step 8 reports any deferred or failed items, emit:
|
||||
# "Linked to #$SPEC_ISSUE (partial delivery — NOT auto-closing; close manually after follow-up)"
|
||||
# If Plan Completion is fully complete, emit:
|
||||
# "Closes #$SPEC_ISSUE"
|
||||
# and include the Closes #N line in the PR body so GitHub auto-closes on merge.>
|
||||
|
||||
<Format:
|
||||
Closes #<N>
|
||||
|
||||
This PR delivers the spec at <archive path relative to repo root>.
|
||||
Spec filed: <spec_filed_at from frontmatter>>
|
||||
|
||||
<If partial delivery, emit instead:
|
||||
Linked to #<N> (partial delivery — not auto-closing).
|
||||
Deferred items: <list from Plan Completion>.
|
||||
Close #<N> manually after follow-up lands.>
|
||||
|
||||
<If no /spec archive matches this branch: omit this entire section.>
|
||||
|
||||
## Verification Results
|
||||
<If verification ran: summary from Step 8.1 (N PASS, M FAIL, K SKIPPED)>
|
||||
<If skipped: reason (no plan, no server, no verification section)>
|
||||
<If not applicable: omit this section>
|
||||
|
||||
## TODOS
|
||||
<If items marked complete: bullet list of completed items with version>
|
||||
<If no items completed: "No TODO items completed in this PR.">
|
||||
<If TODOS.md created or reorganized: note that>
|
||||
<If TODOS.md doesn't exist and user skipped: omit this section>
|
||||
|
||||
## Documentation
|
||||
<Embed the `documentation_section` string returned by Step 18's subagent here, verbatim.>
|
||||
<If Step 18 returned `documentation_section: null` (no docs updated), omit this section entirely.>
|
||||
|
||||
## Test plan
|
||||
- [x] All Rails tests pass (N runs, 0 failures)
|
||||
- [x] All Vitest tests pass (N tests)
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||||
```
|
||||
|
||||
#### Redaction scan (PR body + title) — runs before create AND edit
|
||||
|
||||
The PR body is world-readable on a public repo. Scan-at-sink before sending:
|
||||
write the composed body to a temp file, scan THAT file with the shared engine,
|
||||
and pass the same file to `gh`/`glab`. Wrap any Codex / Greptile / eval output
|
||||
sections in tool-attributed fences (` ```codex-review ` / ` ```greptile `) so the
|
||||
engine WARN-degrades the example credentials those tools quote instead of blocking
|
||||
the PR (a live-format credential inside the fence still blocks).
|
||||
|
||||
```bash
|
||||
REDACT_VIS=$(~/.claude/skills/gstack/bin/gstack-config get redact_repo_visibility 2>/dev/null)
|
||||
[ -z "$REDACT_VIS" ] && REDACT_VIS=$(gh repo view --json visibility -q .visibility 2>/dev/null | tr 'A-Z' 'a-z')
|
||||
REDACT_VIS="${REDACT_VIS:-unknown}"
|
||||
PR_BODY_FILE=$(mktemp)
|
||||
cat > "$PR_BODY_FILE" <<'PR_BODY_EOF'
|
||||
<PR body from above>
|
||||
PR_BODY_EOF
|
||||
~/.claude/skills/gstack/bin/gstack-redact --from-file "$PR_BODY_FILE" --repo-visibility "$REDACT_VIS" --self-email "$(git config user.email 2>/dev/null)" --json
|
||||
case $? in
|
||||
3) echo "BLOCKED — credential in PR body. Rotate + redact, do not create the PR."; exit 1 ;;
|
||||
2) echo "MEDIUM findings — confirm per finding (sterner on public) before proceeding." ;;
|
||||
esac
|
||||
# Also scan the title (short, single-line):
|
||||
printf '%s' "v$NEW_VERSION <type>: <summary>" | ~/.claude/skills/gstack/bin/gstack-redact --repo-visibility "$REDACT_VIS" --json
|
||||
```
|
||||
|
||||
HIGH blocks (exit 3, no skip). MEDIUM → AskUserQuestion (PII subset offers
|
||||
`--auto-redact`). Same scan runs before the `gh pr edit --body` path (Step 17).
|
||||
|
||||
**If GitHub:** create from the SCANNED file (exact bytes scanned = bytes sent):
|
||||
|
||||
```bash
|
||||
# PR title MUST start with v$NEW_VERSION — enforced on every run, no exceptions.
|
||||
# (See Step 19 idempotency block + bin/gstack-pr-title-rewrite.sh for the rule.)
|
||||
gh pr create --base <base> --title "v$NEW_VERSION <type>: <summary>" --body-file "$PR_BODY_FILE"
|
||||
rm -f "$PR_BODY_FILE"
|
||||
```
|
||||
|
||||
**If GitLab:**
|
||||
|
||||
```bash
|
||||
# MR title MUST start with v$NEW_VERSION — enforced on every run, no exceptions.
|
||||
# (See Step 19 idempotency block + bin/gstack-pr-title-rewrite.sh for the rule.)
|
||||
glab mr create -b <base> -t "v$NEW_VERSION <type>: <summary>" -d "$(cat <<'EOF'
|
||||
<MR body from above>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
**If neither CLI is available:**
|
||||
Print the branch name, remote URL, and instruct the user to create the PR/MR manually via the web UI. Do not stop — the code is pushed and ready.
|
||||
|
||||
**Output the PR/MR URL** — then proceed to Step 20.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,405 @@
|
||||
<!-- AUTO-GENERATED from review-army.md.tmpl — do not edit directly -->
|
||||
<!-- Regenerate: bun run gen:skill-docs -->
|
||||
## Step 9: Pre-Landing Review
|
||||
|
||||
Review the diff for structural issues that tests don't catch.
|
||||
|
||||
1. Read `.claude/skills/review/checklist.md`. If the file cannot be read, **STOP** and report the error.
|
||||
|
||||
2. Run `git diff origin/<base>` to get the full diff (scoped to feature changes against the freshly-fetched base branch).
|
||||
|
||||
3. Apply the review checklist in two passes:
|
||||
- **Pass 1 (CRITICAL):** SQL & Data Safety, LLM Output Trust Boundary
|
||||
- **Pass 2 (INFORMATIONAL):** All remaining categories
|
||||
|
||||
## Confidence Calibration
|
||||
|
||||
Every finding MUST include a confidence score (1-10):
|
||||
|
||||
| Score | Meaning | Display rule |
|
||||
|-------|---------|-------------|
|
||||
| 9-10 | Verified by reading specific code. Concrete bug or exploit demonstrated. | Show normally |
|
||||
| 7-8 | High confidence pattern match. Very likely correct. | Show normally |
|
||||
| 5-6 | Moderate. Could be a false positive. | Show with caveat: "Medium confidence, verify this is actually an issue" |
|
||||
| 3-4 | Low confidence. Pattern is suspicious but may be fine. | Suppress from main report. Include in appendix only. |
|
||||
| 1-2 | Speculation. | Only report if severity would be P0. |
|
||||
|
||||
**Finding format:**
|
||||
|
||||
\`[SEVERITY] (confidence: N/10) file:line — description\`
|
||||
|
||||
Example:
|
||||
\`[P1] (confidence: 9/10) app/models/user.rb:42 — SQL injection via string interpolation in where clause\`
|
||||
\`[P2] (confidence: 5/10) app/controllers/api/v1/users_controller.rb:18 — Possible N+1 query, verify with production logs\`
|
||||
|
||||
### Pre-emit verification gate (#1539 — kills the "field doesn't exist" FP class)
|
||||
|
||||
Before any finding is promoted to the report, the gate requires:
|
||||
|
||||
1. **Quote the specific code line that motivates the finding** — file:line plus
|
||||
the verbatim text of the line(s) that triggered it. If the finding is "field
|
||||
X doesn't exist on model Y", quote the lines of class Y where the field
|
||||
would live. If "dict.get() might return None", quote the dict initialization.
|
||||
If "race condition between A and B", quote both A and B.
|
||||
|
||||
2. **If you cannot quote the motivating line(s), the finding is unverified.**
|
||||
Force its confidence to 4-5 (suppressed from the main report). It still goes
|
||||
into the appendix so reviewers can audit calibration, but the user does NOT
|
||||
see it in the critical-pass output. Do not work around this by inventing
|
||||
speculative confidence 7+ — that defeats the gate.
|
||||
|
||||
**Framework-meta nudge:** When the symbol is generated by a framework
|
||||
metaclass, descriptor, ORM Meta inner-class, or migration history (Django
|
||||
`Meta`, Rails `has_many`/`scope`, SQLAlchemy `relationship`/`Column`,
|
||||
TypeORM decorators, Sequelize `init`/`belongsTo`, Prisma generated client),
|
||||
quote the meta-construct (the `Meta` block, the migration, the decorator,
|
||||
the schema file) instead of expecting the literal name in the class body.
|
||||
The verification is "I read the source that creates this symbol", not "I
|
||||
grep'd for the name and didn't find it." Deeper framework-aware verification
|
||||
(model introspection, migration-history-aware checks, ORM dialect detection)
|
||||
is deliberately out of scope for the lighter gate — see the deferred
|
||||
`~/.gstack-dev/plans/1539-framework-aware-review.md` design doc.
|
||||
|
||||
The FP classes the gate kills (measured against Django Sprint 2.5 #1539):
|
||||
|
||||
| FP class | Why the gate catches it |
|
||||
|---|---|
|
||||
| "field doesn't exist on model" | Requires quoting the model class body or Meta; the field's absence becomes obvious |
|
||||
| "dict.get() might be None" | Requires quoting the dict initialization (e.g. Django form's `cleaned_data` is `{}`-initialized) |
|
||||
| "save() might lose fields" | Requires quoting the ORM signature or model definition |
|
||||
| "update_fields might miss X" | Requires quoting the field set; if X doesn't exist, the FP is self-evident |
|
||||
|
||||
**Calibration learning:** If you report a finding with confidence < 7 and the user
|
||||
confirms it IS a real issue, that is a calibration event. Your initial confidence was
|
||||
too low. Log the corrected pattern as a learning so future reviews catch it with
|
||||
higher confidence.
|
||||
|
||||
## Design Review (conditional, diff-scoped)
|
||||
|
||||
Check if the diff touches frontend files using `gstack-diff-scope`:
|
||||
|
||||
```bash
|
||||
source <(~/.claude/skills/gstack/bin/gstack-diff-scope <base> 2>/dev/null)
|
||||
```
|
||||
|
||||
**If `SCOPE_FRONTEND=false`:** Skip design review silently. No output.
|
||||
|
||||
**If `SCOPE_FRONTEND=true`:**
|
||||
|
||||
1. **Check for DESIGN.md.** If `DESIGN.md` or `design-system.md` exists in the repo root, read it. All design findings are calibrated against it — patterns blessed in DESIGN.md are not flagged. If not found, use universal design principles.
|
||||
|
||||
2. **Read `.claude/skills/review/design-checklist.md`.** If the file cannot be read, skip design review with a note: "Design checklist not found — skipping design review."
|
||||
|
||||
3. **Read each changed frontend file** (full file, not just diff hunks). Frontend files are identified by the patterns listed in the checklist.
|
||||
|
||||
4. **Apply the design checklist** against the changed files. For each item:
|
||||
- **[HIGH] mechanical CSS fix** (`outline: none`, `!important`, `font-size < 16px`): classify as AUTO-FIX
|
||||
- **[HIGH/MEDIUM] design judgment needed**: classify as ASK
|
||||
- **[LOW] intent-based detection**: present as "Possible — verify visually or run /design-review"
|
||||
|
||||
5. **Include findings** in the review output under a "Design Review" header, following the output format in the checklist. Design findings merge with code review findings into the same Fix-First flow.
|
||||
|
||||
6. **Log the result** for the Review Readiness Dashboard:
|
||||
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-review-log '{"skill":"design-review-lite","timestamp":"TIMESTAMP","status":"STATUS","findings":N,"auto_fixed":M,"commit":"COMMIT"}'
|
||||
```
|
||||
|
||||
Substitute: TIMESTAMP = ISO 8601 datetime, STATUS = "clean" if 0 findings or "issues_found", N = total findings, M = auto-fixed count, COMMIT = output of `git rev-parse --short HEAD`.
|
||||
|
||||
7. **Codex design voice** (optional, automatic if available):
|
||||
|
||||
```bash
|
||||
command -v codex >/dev/null 2>&1 && echo "CODEX_AVAILABLE" || echo "CODEX_NOT_AVAILABLE"
|
||||
```
|
||||
|
||||
If Codex is available, run a lightweight design check on the diff:
|
||||
|
||||
```bash
|
||||
TMPERR_DRL=$(mktemp /tmp/codex-drl-XXXXXXXX)
|
||||
_REPO_ROOT=$(git rev-parse --show-toplevel) || { echo "ERROR: not in a git repo" >&2; exit 1; }
|
||||
codex exec "Review the git diff on this branch. Run 7 litmus checks (YES/NO each): 1. Brand/product unmistakable in first screen? 2. One strong visual anchor present? 3. Page understandable by scanning headlines only? 4. Each section has one job? 5. Are cards actually necessary? 6. Does motion improve hierarchy or atmosphere? 7. Would design feel premium with all decorative shadows removed? Flag any hard rejections: 1. Generic SaaS card grid as first impression 2. Beautiful image with weak brand 3. Strong headline with no clear action 4. Busy imagery behind text 5. Sections repeating same mood statement 6. Carousel with no narrative purpose 7. App UI made of stacked cards instead of layout 5 most important design findings only. Reference file:line." -C "$_REPO_ROOT" -s read-only -c 'model_reasoning_effort="high"' --enable web_search_cached < /dev/null 2>"$TMPERR_DRL"
|
||||
```
|
||||
|
||||
Use a 5-minute timeout (`timeout: 300000`). After the command completes, read stderr:
|
||||
```bash
|
||||
cat "$TMPERR_DRL" && rm -f "$TMPERR_DRL"
|
||||
```
|
||||
|
||||
**Error handling:** All errors are non-blocking. On auth failure, timeout, or empty response — skip with a brief note and continue.
|
||||
|
||||
Present Codex output under a `CODEX (design):` header, merged with the checklist findings above.
|
||||
|
||||
Include any design findings alongside the code review findings. They follow the same Fix-First flow below.
|
||||
|
||||
## Step 9.1: Review Army — Specialist Dispatch
|
||||
|
||||
### Detect stack and scope
|
||||
|
||||
```bash
|
||||
source <(~/.claude/skills/gstack/bin/gstack-diff-scope <base> 2>/dev/null) || true
|
||||
# Detect stack for specialist context
|
||||
STACK=""
|
||||
[ -f Gemfile ] && STACK="${STACK}ruby "
|
||||
[ -f package.json ] && STACK="${STACK}node "
|
||||
[ -f requirements.txt ] || [ -f pyproject.toml ] && STACK="${STACK}python "
|
||||
[ -f go.mod ] && STACK="${STACK}go "
|
||||
[ -f Cargo.toml ] && STACK="${STACK}rust "
|
||||
echo "STACK: ${STACK:-unknown}"
|
||||
DIFF_BASE=$(git merge-base origin/<base> HEAD)
|
||||
DIFF_INS=$(git diff "$DIFF_BASE" --stat | tail -1 | grep -oE '[0-9]+ insertion' | grep -oE '[0-9]+' || echo "0")
|
||||
DIFF_DEL=$(git diff "$DIFF_BASE" --stat | tail -1 | grep -oE '[0-9]+ deletion' | grep -oE '[0-9]+' || echo "0")
|
||||
DIFF_LINES=$((DIFF_INS + DIFF_DEL))
|
||||
echo "DIFF_LINES: $DIFF_LINES"
|
||||
# Detect test framework for specialist test stub generation
|
||||
TEST_FW=""
|
||||
{ [ -f jest.config.ts ] || [ -f jest.config.js ]; } && TEST_FW="jest"
|
||||
[ -f vitest.config.ts ] && TEST_FW="vitest"
|
||||
{ [ -f spec/spec_helper.rb ] || [ -f .rspec ]; } && TEST_FW="rspec"
|
||||
{ [ -f pytest.ini ] || [ -f conftest.py ]; } && TEST_FW="pytest"
|
||||
[ -f go.mod ] && TEST_FW="go-test"
|
||||
echo "TEST_FW: ${TEST_FW:-unknown}"
|
||||
```
|
||||
|
||||
### Read specialist hit rates (adaptive gating)
|
||||
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-specialist-stats 2>/dev/null || true
|
||||
```
|
||||
|
||||
### Select specialists
|
||||
|
||||
Based on the scope signals above, select which specialists to dispatch.
|
||||
|
||||
**Always-on (dispatch on every review with 50+ changed lines):**
|
||||
1. **Testing** — read `~/.claude/skills/gstack/review/specialists/testing.md`
|
||||
2. **Maintainability** — read `~/.claude/skills/gstack/review/specialists/maintainability.md`
|
||||
|
||||
**If DIFF_LINES < 50:** Skip all specialists. Print: "Small diff ($DIFF_LINES lines) — specialists skipped." Continue to the Fix-First flow (item 4).
|
||||
|
||||
**Conditional (dispatch if the matching scope signal is true):**
|
||||
3. **Security** — if SCOPE_AUTH=true, OR if SCOPE_BACKEND=true AND DIFF_LINES > 100. Read `~/.claude/skills/gstack/review/specialists/security.md`
|
||||
4. **Performance** — if SCOPE_BACKEND=true OR SCOPE_FRONTEND=true. Read `~/.claude/skills/gstack/review/specialists/performance.md`
|
||||
5. **Data Migration** — if SCOPE_MIGRATIONS=true. Read `~/.claude/skills/gstack/review/specialists/data-migration.md`
|
||||
6. **API Contract** — if SCOPE_API=true. Read `~/.claude/skills/gstack/review/specialists/api-contract.md`
|
||||
7. **Design** — if SCOPE_FRONTEND=true. Use the existing design review checklist at `~/.claude/skills/gstack/review/design-checklist.md`
|
||||
|
||||
### Adaptive gating
|
||||
|
||||
After scope-based selection, apply adaptive gating based on specialist hit rates:
|
||||
|
||||
For each conditional specialist that passed scope gating, check the `gstack-specialist-stats` output above:
|
||||
- If tagged `[GATE_CANDIDATE]` (0 findings in 10+ dispatches): skip it. Print: "[specialist] auto-gated (0 findings in N reviews)."
|
||||
- If tagged `[NEVER_GATE]`: always dispatch regardless of hit rate. Security and data-migration are insurance policy specialists — they should run even when silent.
|
||||
|
||||
**Force flags:** If the user's prompt includes `--security`, `--performance`, `--testing`, `--maintainability`, `--data-migration`, `--api-contract`, `--design`, or `--all-specialists`, force-include that specialist regardless of gating.
|
||||
|
||||
Note which specialists were selected, gated, and skipped. Print the selection:
|
||||
"Dispatching N specialists: [names]. Skipped: [names] (scope not detected). Gated: [names] (0 findings in N+ reviews)."
|
||||
|
||||
---
|
||||
|
||||
### Dispatch specialists in parallel
|
||||
|
||||
For each selected specialist, launch an independent subagent via the Agent tool.
|
||||
**Launch ALL selected specialists in a single message** (multiple Agent tool calls)
|
||||
so they run in parallel. Each subagent has fresh context — no prior review bias.
|
||||
|
||||
**Each specialist subagent prompt:**
|
||||
|
||||
Construct the prompt for each specialist. The prompt includes:
|
||||
|
||||
1. The specialist's checklist content (you already read the file above)
|
||||
2. Stack context: "This is a {STACK} project."
|
||||
3. Past learnings for this domain (if any exist):
|
||||
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-learnings-search --type pitfall --query "{specialist domain}" --limit 5 2>/dev/null || true
|
||||
```
|
||||
|
||||
If learnings are found, include them: "Past learnings for this domain: {learnings}"
|
||||
|
||||
4. Instructions:
|
||||
|
||||
"You are a specialist code reviewer. Read the checklist below, then run
|
||||
`DIFF_BASE=$(git merge-base origin/<base> HEAD) && git diff "$DIFF_BASE"` to get the full diff. Apply the checklist against the diff.
|
||||
|
||||
For each finding, output a JSON object on its own line:
|
||||
{\"severity\":\"CRITICAL|INFORMATIONAL\",\"confidence\":N,\"path\":\"file\",\"line\":N,\"category\":\"category\",\"summary\":\"description\",\"fix\":\"recommended fix\",\"fingerprint\":\"path:line:category\",\"specialist\":\"name\"}
|
||||
|
||||
Required fields: severity, confidence, path, category, summary, specialist.
|
||||
Optional: line, fix, fingerprint, evidence, test_stub.
|
||||
|
||||
If you can write a test that would catch this issue, include it in the `test_stub` field.
|
||||
Use the detected test framework ({TEST_FW}). Write a minimal skeleton — describe/it/test
|
||||
blocks with clear intent. Skip test_stub for architectural or design-only findings.
|
||||
|
||||
If no findings: output `NO FINDINGS` and nothing else.
|
||||
Do not output anything else — no preamble, no summary, no commentary.
|
||||
|
||||
Stack context: {STACK}
|
||||
Past learnings: {learnings or 'none'}
|
||||
|
||||
CHECKLIST:
|
||||
{checklist content}"
|
||||
|
||||
**Subagent configuration:**
|
||||
- Use `subagent_type: "general-purpose"`
|
||||
- Do NOT use `run_in_background` — all specialists must complete before merge
|
||||
- If any specialist subagent fails or times out, log the failure and continue with results from successful specialists. Specialists are additive — partial results are better than no results.
|
||||
|
||||
---
|
||||
|
||||
### Step 9.2: Collect and merge findings
|
||||
|
||||
After all specialist subagents complete, collect their outputs.
|
||||
|
||||
**Parse findings:**
|
||||
For each specialist's output:
|
||||
1. If output is "NO FINDINGS" — skip, this specialist found nothing
|
||||
2. Otherwise, parse each line as a JSON object. Skip lines that are not valid JSON.
|
||||
3. Collect all parsed findings into a single list, tagged with their specialist name.
|
||||
|
||||
**Fingerprint and deduplicate:**
|
||||
For each finding, compute its fingerprint:
|
||||
- If `fingerprint` field is present, use it
|
||||
- Otherwise: `{path}:{line}:{category}` (if line is present) or `{path}:{category}`
|
||||
|
||||
Group findings by fingerprint. For findings sharing the same fingerprint:
|
||||
- Keep the finding with the highest confidence score
|
||||
- Tag it: "MULTI-SPECIALIST CONFIRMED ({specialist1} + {specialist2})"
|
||||
- Boost confidence by +1 (cap at 10)
|
||||
- Note the confirming specialists in the output
|
||||
|
||||
**Apply confidence gates:**
|
||||
- Confidence 7+: show normally in the findings output
|
||||
- Confidence 5-6: show with caveat "Medium confidence — verify this is actually an issue"
|
||||
- Confidence 3-4: move to appendix (suppress from main findings)
|
||||
- Confidence 1-2: suppress entirely
|
||||
|
||||
**Compute PR Quality Score:**
|
||||
After merging, compute the quality score:
|
||||
`quality_score = max(0, 10 - (critical_count * 2 + informational_count * 0.5))`
|
||||
Cap at 10. Log this in the review result at the end.
|
||||
|
||||
**Output merged findings:**
|
||||
Present the merged findings in the same format as the current review:
|
||||
|
||||
```
|
||||
SPECIALIST REVIEW: N findings (X critical, Y informational) from Z specialists
|
||||
|
||||
[For each finding, in order: CRITICAL first, then INFORMATIONAL, sorted by confidence descending]
|
||||
[SEVERITY] (confidence: N/10, specialist: name) path:line — summary
|
||||
Fix: recommended fix
|
||||
[If MULTI-SPECIALIST CONFIRMED: show confirmation note]
|
||||
|
||||
PR Quality Score: X/10
|
||||
```
|
||||
|
||||
These findings flow into the Fix-First flow (item 4) alongside the checklist pass (Step 9).
|
||||
The Fix-First heuristic applies identically — specialist findings follow the same AUTO-FIX vs ASK classification.
|
||||
|
||||
**Compile per-specialist stats:**
|
||||
After merging findings, compile a `specialists` object for the review-log persist.
|
||||
For each specialist (testing, maintainability, security, performance, data-migration, api-contract, design, red-team):
|
||||
- If dispatched: `{"dispatched": true, "findings": N, "critical": N, "informational": N}`
|
||||
- If skipped by scope: `{"dispatched": false, "reason": "scope"}`
|
||||
- If skipped by gating: `{"dispatched": false, "reason": "gated"}`
|
||||
- If not applicable (e.g., red-team not activated): omit from the object
|
||||
|
||||
Include the Design specialist even though it uses `design-checklist.md` instead of the specialist schema files.
|
||||
Remember these stats — you will need them for the review-log entry in Step 5.8.
|
||||
|
||||
---
|
||||
|
||||
### Red Team dispatch (conditional)
|
||||
|
||||
**Activation:** Only if DIFF_LINES > 200 OR any specialist produced a CRITICAL finding.
|
||||
|
||||
If activated, dispatch one more subagent via the Agent tool (foreground, not background).
|
||||
|
||||
The Red Team subagent receives:
|
||||
1. The red-team checklist from `~/.claude/skills/gstack/review/specialists/red-team.md`
|
||||
2. The merged specialist findings from Step 9.2 (so it knows what was already caught)
|
||||
3. The git diff command
|
||||
|
||||
Prompt: "You are a red team reviewer. The code has already been reviewed by N specialists
|
||||
who found the following issues: {merged findings summary}. Your job is to find what they
|
||||
MISSED. Read the checklist, run `DIFF_BASE=$(git merge-base origin/<base> HEAD) && git diff "$DIFF_BASE"`, and look for gaps.
|
||||
Output findings as JSON objects (same schema as the specialists). Focus on cross-cutting
|
||||
concerns, integration boundary issues, and failure modes that specialist checklists
|
||||
don't cover."
|
||||
|
||||
If the Red Team finds additional issues, merge them into the findings list before
|
||||
the Fix-First flow (item 4). Red Team findings are tagged with `"specialist":"red-team"`.
|
||||
|
||||
If the Red Team returns NO FINDINGS, note: "Red Team review: no additional issues found."
|
||||
If the Red Team subagent fails or times out, skip silently and continue.
|
||||
|
||||
### Step 9.3: Cross-review finding dedup
|
||||
|
||||
Before classifying findings, check if any were previously skipped by the user in a prior review on this branch.
|
||||
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-review-read
|
||||
```
|
||||
|
||||
Parse the output: only lines BEFORE `---CONFIG---` are JSONL entries (the output also contains `---CONFIG---` and `---HEAD---` footer sections that are not JSONL — ignore those).
|
||||
|
||||
For each JSONL entry that has a `findings` array:
|
||||
1. Collect all fingerprints where `action: "skipped"`
|
||||
2. Note the `commit` field from that entry
|
||||
|
||||
If skipped fingerprints exist, get the list of files changed since that review:
|
||||
|
||||
```bash
|
||||
git diff --name-only <prior-review-commit> HEAD
|
||||
```
|
||||
|
||||
For each current finding (from both the checklist pass (Step 9) and specialist review (Step 9.1-9.2)), check:
|
||||
- Does its fingerprint match a previously skipped finding?
|
||||
- Is the finding's file path NOT in the changed-files set?
|
||||
|
||||
If both conditions are true: suppress the finding. It was intentionally skipped and the relevant code hasn't changed.
|
||||
|
||||
Print: "Suppressed N findings from prior reviews (previously skipped by user)"
|
||||
|
||||
**Only suppress `skipped` findings — never `fixed` or `auto-fixed`** (those might regress and should be re-checked).
|
||||
|
||||
If no prior reviews exist or none have a `findings` array, skip this step silently.
|
||||
|
||||
Output a summary header: `Pre-Landing Review: N issues (X critical, Y informational)`
|
||||
|
||||
4. **Classify each finding from both the checklist pass and specialist review (Step 9.1-Step 9.2) as AUTO-FIX or ASK** per the Fix-First Heuristic in
|
||||
checklist.md. Critical findings lean toward ASK; informational lean toward AUTO-FIX.
|
||||
|
||||
5. **Auto-fix all AUTO-FIX items.** Apply each fix. Output one line per fix:
|
||||
`[AUTO-FIXED] [file:line] Problem → what you did`
|
||||
|
||||
6. **If ASK items remain,** present them in ONE AskUserQuestion:
|
||||
- List each with number, severity, problem, recommended fix
|
||||
- Per-item options: A) Fix B) Skip
|
||||
- Overall RECOMMENDATION
|
||||
- If 3 or fewer ASK items, you may use individual AskUserQuestion calls instead
|
||||
|
||||
7. **After all fixes (auto + user-approved):**
|
||||
- If ANY fixes were applied: commit fixed files by name (`git add <fixed-files> && git commit -m "fix: pre-landing review fixes"`), then **STOP** and tell the user to run `/ship` again to re-test.
|
||||
- If no fixes applied (all ASK items skipped, or no issues found): continue to Step 12.
|
||||
|
||||
8. Output summary: `Pre-Landing Review: N issues — M auto-fixed, K asked (J fixed, L skipped)`
|
||||
|
||||
If no issues found: `Pre-Landing Review: No issues found.`
|
||||
|
||||
9. Persist the review result to the review log:
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-review-log '{"skill":"review","timestamp":"TIMESTAMP","status":"STATUS","issues_found":N,"critical":N,"informational":N,"quality_score":SCORE,"specialists":SPECIALISTS_JSON,"findings":FINDINGS_JSON,"commit":"'"$(git rev-parse --short HEAD)"'","via":"ship"}'
|
||||
```
|
||||
Substitute TIMESTAMP (ISO 8601), STATUS ("clean" if no issues, "issues_found" otherwise),
|
||||
and N values from the summary counts above. The `via:"ship"` distinguishes from standalone `/review` runs.
|
||||
- `quality_score` = the PR Quality Score computed in Step 9.2 (e.g., 7.5). If specialists were skipped (small diff), use `10.0`
|
||||
- `specialists` = the per-specialist stats object compiled in Step 9.2. Each specialist that was considered gets an entry: `{"dispatched":true/false,"findings":N,"critical":N,"informational":N}` if dispatched, or `{"dispatched":false,"reason":"scope|gated"}` if skipped. Example: `{"testing":{"dispatched":true,"findings":2,"critical":0,"informational":2},"security":{"dispatched":false,"reason":"scope"}}`
|
||||
- `findings` = array of per-finding records. For each finding (from checklist pass and specialists), include: `{"fingerprint":"path:line:category","severity":"CRITICAL|INFORMATIONAL","action":"ACTION"}`. ACTION is `"auto-fixed"`, `"fixed"` (user approved), or `"skipped"` (user chose Skip).
|
||||
|
||||
Save the review output — it goes into the PR body in Step 19.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,55 @@
|
||||
## Step 9: Pre-Landing Review
|
||||
|
||||
Review the diff for structural issues that tests don't catch.
|
||||
|
||||
1. Read `.claude/skills/review/checklist.md`. If the file cannot be read, **STOP** and report the error.
|
||||
|
||||
2. Run `git diff origin/<base>` to get the full diff (scoped to feature changes against the freshly-fetched base branch).
|
||||
|
||||
3. Apply the review checklist in two passes:
|
||||
- **Pass 1 (CRITICAL):** SQL & Data Safety, LLM Output Trust Boundary
|
||||
- **Pass 2 (INFORMATIONAL):** All remaining categories
|
||||
|
||||
{{CONFIDENCE_CALIBRATION}}
|
||||
|
||||
{{DESIGN_REVIEW_LITE}}
|
||||
|
||||
Include any design findings alongside the code review findings. They follow the same Fix-First flow below.
|
||||
|
||||
{{REVIEW_ARMY}}
|
||||
|
||||
{{CROSS_REVIEW_DEDUP}}
|
||||
|
||||
4. **Classify each finding from both the checklist pass and specialist review (Step 9.1-Step 9.2) as AUTO-FIX or ASK** per the Fix-First Heuristic in
|
||||
checklist.md. Critical findings lean toward ASK; informational lean toward AUTO-FIX.
|
||||
|
||||
5. **Auto-fix all AUTO-FIX items.** Apply each fix. Output one line per fix:
|
||||
`[AUTO-FIXED] [file:line] Problem → what you did`
|
||||
|
||||
6. **If ASK items remain,** present them in ONE AskUserQuestion:
|
||||
- List each with number, severity, problem, recommended fix
|
||||
- Per-item options: A) Fix B) Skip
|
||||
- Overall RECOMMENDATION
|
||||
- If 3 or fewer ASK items, you may use individual AskUserQuestion calls instead
|
||||
|
||||
7. **After all fixes (auto + user-approved):**
|
||||
- If ANY fixes were applied: commit fixed files by name (`git add <fixed-files> && git commit -m "fix: pre-landing review fixes"`), then **STOP** and tell the user to run `/ship` again to re-test.
|
||||
- If no fixes applied (all ASK items skipped, or no issues found): continue to Step 12.
|
||||
|
||||
8. Output summary: `Pre-Landing Review: N issues — M auto-fixed, K asked (J fixed, L skipped)`
|
||||
|
||||
If no issues found: `Pre-Landing Review: No issues found.`
|
||||
|
||||
9. Persist the review result to the review log:
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-review-log '{"skill":"review","timestamp":"TIMESTAMP","status":"STATUS","issues_found":N,"critical":N,"informational":N,"quality_score":SCORE,"specialists":SPECIALISTS_JSON,"findings":FINDINGS_JSON,"commit":"'"$(git rev-parse --short HEAD)"'","via":"ship"}'
|
||||
```
|
||||
Substitute TIMESTAMP (ISO 8601), STATUS ("clean" if no issues, "issues_found" otherwise),
|
||||
and N values from the summary counts above. The `via:"ship"` distinguishes from standalone `/review` runs.
|
||||
- `quality_score` = the PR Quality Score computed in Step 9.2 (e.g., 7.5). If specialists were skipped (small diff), use `10.0`
|
||||
- `specialists` = the per-specialist stats object compiled in Step 9.2. Each specialist that was considered gets an entry: `{"dispatched":true/false,"findings":N,"critical":N,"informational":N}` if dispatched, or `{"dispatched":false,"reason":"scope|gated"}` if skipped. Example: `{"testing":{"dispatched":true,"findings":2,"critical":0,"informational":2},"security":{"dispatched":false,"reason":"scope"}}`
|
||||
- `findings` = array of per-finding records. For each finding (from checklist pass and specialists), include: `{"fingerprint":"path:line:category","severity":"CRITICAL|INFORMATIONAL","action":"ACTION"}`. ACTION is `"auto-fixed"`, `"fixed"` (user approved), or `"skipped"` (user chose Skip).
|
||||
|
||||
Save the review output — it goes into the PR body in Step 19.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,259 @@
|
||||
<!-- AUTO-GENERATED from test-coverage.md.tmpl — do not edit directly -->
|
||||
<!-- Regenerate: bun run gen:skill-docs -->
|
||||
## Step 7: Test Coverage Audit
|
||||
|
||||
**Dispatch this step as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent runs the coverage audit in a fresh context window — the parent only sees the conclusion, not intermediate file reads. This is context-rot defense.
|
||||
|
||||
**Subagent prompt:** Pass the following instructions to the subagent, with `<base>` substituted with the base branch:
|
||||
|
||||
> You are running a ship-workflow test coverage audit. Run `git diff <base>...HEAD` as needed. Do not commit or push — report only.
|
||||
>
|
||||
> 100% coverage is the goal — every untested path is a path where bugs hide and vibe coding becomes yolo coding. Evaluate what was ACTUALLY coded (from the diff), not what was planned.
|
||||
|
||||
### Test Framework Detection
|
||||
|
||||
Before analyzing coverage, detect the project's test framework:
|
||||
|
||||
1. **Read CLAUDE.md** — look for a `## Testing` section with test command and framework name. If found, use that as the authoritative source.
|
||||
2. **If CLAUDE.md has no testing section, auto-detect:**
|
||||
|
||||
```bash
|
||||
setopt +o nomatch 2>/dev/null || true # zsh compat
|
||||
# Detect project runtime
|
||||
[ -f Gemfile ] && echo "RUNTIME:ruby"
|
||||
[ -f package.json ] && echo "RUNTIME:node"
|
||||
[ -f requirements.txt ] || [ -f pyproject.toml ] && echo "RUNTIME:python"
|
||||
[ -f go.mod ] && echo "RUNTIME:go"
|
||||
[ -f Cargo.toml ] && echo "RUNTIME:rust"
|
||||
# Check for existing test infrastructure
|
||||
ls jest.config.* vitest.config.* playwright.config.* cypress.config.* .rspec pytest.ini phpunit.xml 2>/dev/null
|
||||
ls -d test/ tests/ spec/ __tests__/ cypress/ e2e/ 2>/dev/null
|
||||
```
|
||||
|
||||
3. **If no framework detected:** falls through to the Test Framework Bootstrap step (Step 4) which handles full setup.
|
||||
|
||||
**0. Before/after test count:**
|
||||
|
||||
```bash
|
||||
# Count test files before any generation
|
||||
find . -name '*.test.*' -o -name '*.spec.*' -o -name '*_test.*' -o -name '*_spec.*' | grep -v node_modules | wc -l
|
||||
```
|
||||
|
||||
Store this number for the PR body.
|
||||
|
||||
**1. Trace every codepath changed** using `git diff origin/<base>...HEAD`:
|
||||
|
||||
Read every changed file. For each one, trace how data flows through the code — don't just list functions, actually follow the execution:
|
||||
|
||||
1. **Read the diff.** For each changed file, read the full file (not just the diff hunk) to understand context.
|
||||
2. **Trace data flow.** Starting from each entry point (route handler, exported function, event listener, component render), follow the data through every branch:
|
||||
- Where does input come from? (request params, props, database, API call)
|
||||
- What transforms it? (validation, mapping, computation)
|
||||
- Where does it go? (database write, API response, rendered output, side effect)
|
||||
- What can go wrong at each step? (null/undefined, invalid input, network failure, empty collection)
|
||||
3. **Diagram the execution.** For each changed file, draw an ASCII diagram showing:
|
||||
- Every function/method that was added or modified
|
||||
- Every conditional branch (if/else, switch, ternary, guard clause, early return)
|
||||
- Every error path (try/catch, rescue, error boundary, fallback)
|
||||
- Every call to another function (trace into it — does IT have untested branches?)
|
||||
- Every edge: what happens with null input? Empty array? Invalid type?
|
||||
|
||||
This is the critical step — you're building a map of every line of code that can execute differently based on input. Every branch in this diagram needs a test.
|
||||
|
||||
**2. Map user flows, interactions, and error states:**
|
||||
|
||||
Code coverage isn't enough — you need to cover how real users interact with the changed code. For each changed feature, think through:
|
||||
|
||||
- **User flows:** What sequence of actions does a user take that touches this code? Map the full journey (e.g., "user clicks 'Pay' → form validates → API call → success/failure screen"). Each step in the journey needs a test.
|
||||
- **Interaction edge cases:** What happens when the user does something unexpected?
|
||||
- Double-click/rapid resubmit
|
||||
- Navigate away mid-operation (back button, close tab, click another link)
|
||||
- Submit with stale data (page sat open for 30 minutes, session expired)
|
||||
- Slow connection (API takes 10 seconds — what does the user see?)
|
||||
- Concurrent actions (two tabs, same form)
|
||||
- **Error states the user can see:** For every error the code handles, what does the user actually experience?
|
||||
- Is there a clear error message or a silent failure?
|
||||
- Can the user recover (retry, go back, fix input) or are they stuck?
|
||||
- What happens with no network? With a 500 from the API? With invalid data from the server?
|
||||
- **Empty/zero/boundary states:** What does the UI show with zero results? With 10,000 results? With a single character input? With maximum-length input?
|
||||
|
||||
Add these to your diagram alongside the code branches. A user flow with no test is just as much a gap as an untested if/else.
|
||||
|
||||
**3. Check each branch against existing tests:**
|
||||
|
||||
Go through your diagram branch by branch — both code paths AND user flows. For each one, search for a test that exercises it:
|
||||
- Function `processPayment()` → look for `billing.test.ts`, `billing.spec.ts`, `test/billing_test.rb`
|
||||
- An if/else → look for tests covering BOTH the true AND false path
|
||||
- An error handler → look for a test that triggers that specific error condition
|
||||
- A call to `helperFn()` that has its own branches → those branches need tests too
|
||||
- A user flow → look for an integration or E2E test that walks through the journey
|
||||
- An interaction edge case → look for a test that simulates the unexpected action
|
||||
|
||||
Quality scoring rubric:
|
||||
- ★★★ Tests behavior with edge cases AND error paths
|
||||
- ★★ Tests correct behavior, happy path only
|
||||
- ★ Smoke test / existence check / trivial assertion (e.g., "it renders", "it doesn't throw")
|
||||
|
||||
### E2E Test Decision Matrix
|
||||
|
||||
When checking each branch, also determine whether a unit test or E2E/integration test is the right tool:
|
||||
|
||||
**RECOMMEND E2E (mark as [→E2E] in the diagram):**
|
||||
- Common user flow spanning 3+ components/services (e.g., signup → verify email → first login)
|
||||
- Integration point where mocking hides real failures (e.g., API → queue → worker → DB)
|
||||
- Auth/payment/data-destruction flows — too important to trust unit tests alone
|
||||
|
||||
**RECOMMEND EVAL (mark as [→EVAL] in the diagram):**
|
||||
- Critical LLM call that needs a quality eval (e.g., prompt change → test output still meets quality bar)
|
||||
- Changes to prompt templates, system instructions, or tool definitions
|
||||
|
||||
**STICK WITH UNIT TESTS:**
|
||||
- Pure function with clear inputs/outputs
|
||||
- Internal helper with no side effects
|
||||
- Edge case of a single function (null input, empty array)
|
||||
- Obscure/rare flow that isn't customer-facing
|
||||
|
||||
### REGRESSION RULE (mandatory)
|
||||
|
||||
**IRON RULE:** When the coverage audit identifies a REGRESSION — code that previously worked but the diff broke — a regression test is written immediately. No AskUserQuestion. No skipping. Regressions are the highest-priority test because they prove something broke.
|
||||
|
||||
A regression is when:
|
||||
- The diff modifies existing behavior (not new code)
|
||||
- The existing test suite (if any) doesn't cover the changed path
|
||||
- The change introduces a new failure mode for existing callers
|
||||
|
||||
When uncertain whether a change is a regression, err on the side of writing the test.
|
||||
|
||||
Format: commit as `test: regression test for {what broke}`
|
||||
|
||||
**4. Output ASCII coverage diagram:**
|
||||
|
||||
Include BOTH code paths and user flows in the same diagram. Mark E2E-worthy and eval-worthy paths:
|
||||
|
||||
```
|
||||
CODE PATHS USER FLOWS
|
||||
[+] src/services/billing.ts [+] Payment checkout
|
||||
├── processPayment() ├── [★★★ TESTED] Complete purchase — checkout.e2e.ts:15
|
||||
│ ├── [★★★ TESTED] happy + declined + timeout ├── [GAP] [→E2E] Double-click submit
|
||||
│ ├── [GAP] Network timeout └── [GAP] Navigate away mid-payment
|
||||
│ └── [GAP] Invalid currency
|
||||
└── refundPayment() [+] Error states
|
||||
├── [★★ TESTED] Full refund — :89 ├── [★★ TESTED] Card declined message
|
||||
└── [★ TESTED] Partial (non-throw only) — :101 └── [GAP] Network timeout UX
|
||||
|
||||
LLM integration: [GAP] [→EVAL] Prompt template change — needs eval test
|
||||
|
||||
COVERAGE: 5/13 paths tested (38%) | Code paths: 3/5 (60%) | User flows: 2/8 (25%)
|
||||
QUALITY: ★★★:2 ★★:2 ★:1 | GAPS: 8 (2 E2E, 1 eval)
|
||||
```
|
||||
|
||||
Legend: ★★★ behavior + edge + error | ★★ happy path | ★ smoke check
|
||||
[→E2E] = needs integration test | [→EVAL] = needs LLM eval
|
||||
|
||||
**Fast path:** All paths covered → "Step 7: All new code paths have test coverage ✓" Continue.
|
||||
|
||||
**5. Generate tests for uncovered paths:**
|
||||
|
||||
If test framework detected (or bootstrapped in Step 4):
|
||||
- Prioritize error handlers and edge cases first (happy paths are more likely already tested)
|
||||
- Read 2-3 existing test files to match conventions exactly
|
||||
- Generate unit tests. Mock all external dependencies (DB, API, Redis).
|
||||
- For paths marked [→E2E]: generate integration/E2E tests using the project's E2E framework (Playwright, Cypress, Capybara, etc.)
|
||||
- For paths marked [→EVAL]: generate eval tests using the project's eval framework, or flag for manual eval if none exists
|
||||
- Write tests that exercise the specific uncovered path with real assertions
|
||||
- Run each test. Passes → commit as `test: coverage for {feature}`
|
||||
- Fails → fix once. Still fails → revert, note gap in diagram.
|
||||
|
||||
Caps: 30 code paths max, 20 tests generated max (code + user flow combined), 2-min per-test exploration cap.
|
||||
|
||||
If no test framework AND user declined bootstrap → diagram only, no generation. Note: "Test generation skipped — no test framework configured."
|
||||
|
||||
**Diff is test-only changes:** Skip Step 7 entirely: "No new application code paths to audit."
|
||||
|
||||
**6. After-count and coverage summary:**
|
||||
|
||||
```bash
|
||||
# Count test files after generation
|
||||
find . -name '*.test.*' -o -name '*.spec.*' -o -name '*_test.*' -o -name '*_spec.*' | grep -v node_modules | wc -l
|
||||
```
|
||||
|
||||
For PR body: `Tests: {before} → {after} (+{delta} new)`
|
||||
Coverage line: `Test Coverage Audit: N new code paths. M covered (X%). K tests generated, J committed.`
|
||||
|
||||
**7. Coverage gate:**
|
||||
|
||||
Before proceeding, check CLAUDE.md for a `## Test Coverage` section with `Minimum:` and `Target:` fields. If found, use those percentages. Otherwise use defaults: Minimum = 60%, Target = 80%.
|
||||
|
||||
Using the coverage percentage from the diagram in substep 4 (the `COVERAGE: X/Y (Z%)` line):
|
||||
|
||||
- **>= target:** Pass. "Coverage gate: PASS ({X}%)." Continue.
|
||||
- **>= minimum, < target:** Use AskUserQuestion:
|
||||
- "AI-assessed coverage is {X}%. {N} code paths are untested. Target is {target}%."
|
||||
- RECOMMENDATION: Choose A because untested code paths are where production bugs hide.
|
||||
- Options:
|
||||
A) Generate more tests for remaining gaps (recommended)
|
||||
B) Ship anyway — I accept the coverage risk
|
||||
C) These paths don't need tests — mark as intentionally uncovered
|
||||
- If A: Loop back to substep 5 (generate tests) targeting the remaining gaps. After second pass, if still below target, present AskUserQuestion again with updated numbers. Maximum 2 generation passes total.
|
||||
- If B: Continue. Include in PR body: "Coverage gate: {X}% — user accepted risk."
|
||||
- If C: Continue. Include in PR body: "Coverage gate: {X}% — {N} paths intentionally uncovered."
|
||||
|
||||
- **< minimum:** Use AskUserQuestion:
|
||||
- "AI-assessed coverage is critically low ({X}%). {N} of {M} code paths have no tests. Minimum threshold is {minimum}%."
|
||||
- RECOMMENDATION: Choose A because less than {minimum}% means more code is untested than tested.
|
||||
- Options:
|
||||
A) Generate tests for remaining gaps (recommended)
|
||||
B) Override — ship with low coverage (I understand the risk)
|
||||
- If A: Loop back to substep 5. Maximum 2 passes. If still below minimum after 2 passes, present the override choice again.
|
||||
- If B: Continue. Include in PR body: "Coverage gate: OVERRIDDEN at {X}%."
|
||||
|
||||
**Coverage percentage undetermined:** If the coverage diagram doesn't produce a clear numeric percentage (ambiguous output, parse error), **skip the gate** with: "Coverage gate: could not determine percentage — skipping." Do not default to 0% or block.
|
||||
|
||||
**Test-only diffs:** Skip the gate (same as the existing fast-path).
|
||||
|
||||
**100% coverage:** "Coverage gate: PASS (100%)." Continue.
|
||||
|
||||
### Test Plan Artifact
|
||||
|
||||
After producing the coverage diagram, write a test plan artifact so `/qa` and `/qa-only` can consume it:
|
||||
|
||||
```bash
|
||||
eval "$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)" && mkdir -p ~/.gstack/projects/$SLUG
|
||||
USER=$(whoami)
|
||||
DATETIME=$(date +%Y%m%d-%H%M%S)
|
||||
```
|
||||
|
||||
Write to `~/.gstack/projects/{slug}/{user}-{branch}-ship-test-plan-{datetime}.md`:
|
||||
|
||||
```markdown
|
||||
# Test Plan
|
||||
Generated by /ship on {date}
|
||||
Branch: {branch}
|
||||
Repo: {owner/repo}
|
||||
|
||||
## Affected Pages/Routes
|
||||
- {URL path} — {what to test and why}
|
||||
|
||||
## Key Interactions to Verify
|
||||
- {interaction description} on {page}
|
||||
|
||||
## Edge Cases
|
||||
- {edge case} on {page}
|
||||
|
||||
## Critical Paths
|
||||
- {end-to-end flow that must work}
|
||||
```
|
||||
>
|
||||
> After your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):
|
||||
> `{"coverage_pct":N,"gaps":N,"diagram":"<full markdown coverage diagram for PR body>","tests_added":["path",...]}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
1. Read the subagent's final output. Parse the LAST line as JSON.
|
||||
2. Store `coverage_pct` (for Step 20 metrics), `gaps` (user summary), `tests_added` (for the commit).
|
||||
3. Embed `diagram` verbatim in the PR body's `## Test Coverage` section (Step 19).
|
||||
4. Print a one-line summary: `Coverage: {coverage_pct}%, {gaps} gaps. {tests_added.length} tests added.`
|
||||
|
||||
**If the subagent fails, times out, or returns invalid JSON:** Fall back to running the audit inline in the parent. Do not block /ship on subagent failure — partial results are better than none.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,23 @@
|
||||
## Step 7: Test Coverage Audit
|
||||
|
||||
**Dispatch this step as a subagent** using the Agent tool with `subagent_type: "general-purpose"`. The subagent runs the coverage audit in a fresh context window — the parent only sees the conclusion, not intermediate file reads. This is context-rot defense.
|
||||
|
||||
**Subagent prompt:** Pass the following instructions to the subagent, with `<base>` substituted with the base branch:
|
||||
|
||||
> You are running a ship-workflow test coverage audit. Run `git diff <base>...HEAD` as needed. Do not commit or push — report only.
|
||||
>
|
||||
> {{TEST_COVERAGE_AUDIT_SHIP}}
|
||||
>
|
||||
> After your analysis, output a single JSON object on the LAST LINE of your response (no other text after it):
|
||||
> `{"coverage_pct":N,"gaps":N,"diagram":"<full markdown coverage diagram for PR body>","tests_added":["path",...]}`
|
||||
|
||||
**Parent processing:**
|
||||
|
||||
1. Read the subagent's final output. Parse the LAST line as JSON.
|
||||
2. Store `coverage_pct` (for Step 20 metrics), `gaps` (user summary), `tests_added` (for the commit).
|
||||
3. Embed `diagram` verbatim in the PR body's `## Test Coverage` section (Step 19).
|
||||
4. Print a one-line summary: `Coverage: {coverage_pct}%, {gaps} gaps. {tests_added.length} tests added.`
|
||||
|
||||
**If the subagent fails, times out, or returns invalid JSON:** Fall back to running the audit inline in the parent. Do not block /ship on subagent failure — partial results are better than none.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,349 @@
|
||||
<!-- AUTO-GENERATED from tests.md.tmpl — do not edit directly -->
|
||||
<!-- Regenerate: bun run gen:skill-docs -->
|
||||
## Step 4: Test Framework Bootstrap
|
||||
|
||||
## Test Framework Bootstrap
|
||||
|
||||
**Detect existing test framework and project runtime:**
|
||||
|
||||
```bash
|
||||
setopt +o nomatch 2>/dev/null || true # zsh compat
|
||||
# Detect project runtime
|
||||
[ -f Gemfile ] && echo "RUNTIME:ruby"
|
||||
[ -f package.json ] && echo "RUNTIME:node"
|
||||
[ -f requirements.txt ] || [ -f pyproject.toml ] && echo "RUNTIME:python"
|
||||
[ -f go.mod ] && echo "RUNTIME:go"
|
||||
[ -f Cargo.toml ] && echo "RUNTIME:rust"
|
||||
[ -f composer.json ] && echo "RUNTIME:php"
|
||||
[ -f mix.exs ] && echo "RUNTIME:elixir"
|
||||
# Detect sub-frameworks
|
||||
[ -f Gemfile ] && grep -q "rails" Gemfile 2>/dev/null && echo "FRAMEWORK:rails"
|
||||
[ -f package.json ] && grep -q '"next"' package.json 2>/dev/null && echo "FRAMEWORK:nextjs"
|
||||
# Check for existing test infrastructure
|
||||
ls jest.config.* vitest.config.* playwright.config.* .rspec pytest.ini pyproject.toml phpunit.xml 2>/dev/null
|
||||
ls -d test/ tests/ spec/ __tests__/ cypress/ e2e/ 2>/dev/null
|
||||
# Check opt-out marker
|
||||
[ -f .gstack/no-test-bootstrap ] && echo "BOOTSTRAP_DECLINED"
|
||||
```
|
||||
|
||||
**If test framework detected** (config files or test directories found):
|
||||
Print "Test framework detected: {name} ({N} existing tests). Skipping bootstrap."
|
||||
Read 2-3 existing test files to learn conventions (naming, imports, assertion style, setup patterns).
|
||||
Store conventions as prose context for use in Phase 8e.5 or Step 7. **Skip the rest of bootstrap.**
|
||||
|
||||
**If BOOTSTRAP_DECLINED** appears: Print "Test bootstrap previously declined — skipping." **Skip the rest of bootstrap.**
|
||||
|
||||
**If NO runtime detected** (no config files found): Use AskUserQuestion:
|
||||
"I couldn't detect your project's language. What runtime are you using?"
|
||||
Options: A) Node.js/TypeScript B) Ruby/Rails C) Python D) Go E) Rust F) PHP G) Elixir H) This project doesn't need tests.
|
||||
If user picks H → write `.gstack/no-test-bootstrap` and continue without tests.
|
||||
|
||||
**If runtime detected but no test framework — bootstrap:**
|
||||
|
||||
### B2. Research best practices
|
||||
|
||||
Use WebSearch to find current best practices for the detected runtime:
|
||||
- `"[runtime] best test framework 2025 2026"`
|
||||
- `"[framework A] vs [framework B] comparison"`
|
||||
|
||||
If WebSearch is unavailable, use this built-in knowledge table:
|
||||
|
||||
| Runtime | Primary recommendation | Alternative |
|
||||
|---------|----------------------|-------------|
|
||||
| Ruby/Rails | minitest + fixtures + capybara | rspec + factory_bot + shoulda-matchers |
|
||||
| Node.js | vitest + @testing-library | jest + @testing-library |
|
||||
| Next.js | vitest + @testing-library/react + playwright | jest + cypress |
|
||||
| Python | pytest + pytest-cov | unittest |
|
||||
| Go | stdlib testing + testify | stdlib only |
|
||||
| Rust | cargo test (built-in) + mockall | — |
|
||||
| PHP | phpunit + mockery | pest |
|
||||
| Elixir | ExUnit (built-in) + ex_machina | — |
|
||||
|
||||
### B3. Framework selection
|
||||
|
||||
Use AskUserQuestion:
|
||||
"I detected this is a [Runtime/Framework] project with no test framework. I researched current best practices. Here are the options:
|
||||
A) [Primary] — [rationale]. Includes: [packages]. Supports: unit, integration, smoke, e2e
|
||||
B) [Alternative] — [rationale]. Includes: [packages]
|
||||
C) Skip — don't set up testing right now
|
||||
RECOMMENDATION: Choose A because [reason based on project context]"
|
||||
|
||||
If user picks C → write `.gstack/no-test-bootstrap`. Tell user: "If you change your mind later, delete `.gstack/no-test-bootstrap` and re-run." Continue without tests.
|
||||
|
||||
If multiple runtimes detected (monorepo) → ask which runtime to set up first, with option to do both sequentially.
|
||||
|
||||
### B4. Install and configure
|
||||
|
||||
1. Install the chosen packages (npm/bun/gem/pip/etc.)
|
||||
2. Create minimal config file
|
||||
3. Create directory structure (test/, spec/, etc.)
|
||||
4. Create one example test matching the project's code to verify setup works
|
||||
|
||||
If package installation fails → debug once. If still failing → revert with `git checkout -- package.json package-lock.json` (or equivalent for the runtime). Warn user and continue without tests.
|
||||
|
||||
### B4.5. First real tests
|
||||
|
||||
Generate 3-5 real tests for existing code:
|
||||
|
||||
1. **Find recently changed files:** `git log --since=30.days --name-only --format="" | sort | uniq -c | sort -rn | head -10`
|
||||
2. **Prioritize by risk:** Error handlers > business logic with conditionals > API endpoints > pure functions
|
||||
3. **For each file:** Write one test that tests real behavior with meaningful assertions. Never `expect(x).toBeDefined()` — test what the code DOES.
|
||||
4. Run each test. Passes → keep. Fails → fix once. Still fails → delete silently.
|
||||
5. Generate at least 1 test, cap at 5.
|
||||
|
||||
Never import secrets, API keys, or credentials in test files. Use environment variables or test fixtures.
|
||||
|
||||
### B5. Verify
|
||||
|
||||
```bash
|
||||
# Run the full test suite to confirm everything works
|
||||
{detected test command}
|
||||
```
|
||||
|
||||
If tests fail → debug once. If still failing → revert all bootstrap changes and warn user.
|
||||
|
||||
### B5.5. CI/CD pipeline
|
||||
|
||||
```bash
|
||||
# Check CI provider
|
||||
ls -d .github/ 2>/dev/null && echo "CI:github"
|
||||
ls .gitlab-ci.yml .circleci/ bitrise.yml 2>/dev/null
|
||||
```
|
||||
|
||||
If `.github/` exists (or no CI detected — default to GitHub Actions):
|
||||
Create `.github/workflows/test.yml` with:
|
||||
- `runs-on: ubuntu-latest`
|
||||
- Appropriate setup action for the runtime (setup-node, setup-ruby, setup-python, etc.)
|
||||
- The same test command verified in B5
|
||||
- Trigger: push + pull_request
|
||||
|
||||
If non-GitHub CI detected → skip CI generation with note: "Detected {provider} — CI pipeline generation supports GitHub Actions only. Add test step to your existing pipeline manually."
|
||||
|
||||
### B6. Create TESTING.md
|
||||
|
||||
First check: If TESTING.md already exists → read it and update/append rather than overwriting. Never destroy existing content.
|
||||
|
||||
Write TESTING.md with:
|
||||
- Philosophy: "100% test coverage is the key to great vibe coding. Tests let you move fast, trust your instincts, and ship with confidence — without them, vibe coding is just yolo coding. With tests, it's a superpower."
|
||||
- Framework name and version
|
||||
- How to run tests (the verified command from B5)
|
||||
- Test layers: Unit tests (what, where, when), Integration tests, Smoke tests, E2E tests
|
||||
- Conventions: file naming, assertion style, setup/teardown patterns
|
||||
|
||||
### B7. Update CLAUDE.md
|
||||
|
||||
First check: If CLAUDE.md already has a `## Testing` section → skip. Don't duplicate.
|
||||
|
||||
Append a `## Testing` section:
|
||||
- Run command and test directory
|
||||
- Reference to TESTING.md
|
||||
- Test expectations:
|
||||
- 100% test coverage is the goal — tests make vibe coding safe
|
||||
- When writing new functions, write a corresponding test
|
||||
- When fixing a bug, write a regression test
|
||||
- When adding error handling, write a test that triggers the error
|
||||
- When adding a conditional (if/else, switch), write tests for BOTH paths
|
||||
- Never commit code that makes existing tests fail
|
||||
|
||||
### B8. Commit
|
||||
|
||||
```bash
|
||||
git status --porcelain
|
||||
```
|
||||
|
||||
Only commit if there are changes. Stage all bootstrap files (config, test directory, TESTING.md, CLAUDE.md, .github/workflows/test.yml if created):
|
||||
`git commit -m "chore: bootstrap test framework ({framework name})"`
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Run tests (on merged code)
|
||||
|
||||
**Do NOT run `RAILS_ENV=test bin/rails db:migrate`** — `bin/test-lane` already calls
|
||||
`db:test:prepare` internally, which loads the schema into the correct lane database.
|
||||
Running bare test migrations without INSTANCE hits an orphan DB and corrupts structure.sql.
|
||||
|
||||
Run both test suites in parallel:
|
||||
|
||||
```bash
|
||||
bin/test-lane 2>&1 | tee /tmp/ship_tests.txt &
|
||||
npm run test 2>&1 | tee /tmp/ship_vitest.txt &
|
||||
wait
|
||||
```
|
||||
|
||||
After both complete, read the output files and check pass/fail.
|
||||
|
||||
**If any test fails:** Do NOT immediately stop. Apply the Test Failure Ownership Triage:
|
||||
|
||||
## Test Failure Ownership Triage
|
||||
|
||||
When tests fail, do NOT immediately stop. First, determine ownership:
|
||||
|
||||
### Step T1: Classify each failure
|
||||
|
||||
For each failing test:
|
||||
|
||||
1. **Get the files changed on this branch:**
|
||||
```bash
|
||||
git diff origin/<base>...HEAD --name-only
|
||||
```
|
||||
|
||||
2. **Classify the failure:**
|
||||
- **In-branch** if: the failing test file itself was modified on this branch, OR the test output references code that was changed on this branch, OR you can trace the failure to a change in the branch diff.
|
||||
- **Likely pre-existing** if: neither the test file nor the code it tests was modified on this branch, AND the failure is unrelated to any branch change you can identify.
|
||||
- **When ambiguous, default to in-branch.** It is safer to stop the developer than to let a broken test ship. Only classify as pre-existing when you are confident.
|
||||
|
||||
This classification is heuristic — use your judgment reading the diff and the test output. You do not have a programmatic dependency graph.
|
||||
|
||||
### Step T2: Handle in-branch failures
|
||||
|
||||
**STOP.** These are your failures. Show them and do not proceed. The developer must fix their own broken tests before shipping.
|
||||
|
||||
### Step T3: Handle pre-existing failures
|
||||
|
||||
Check `REPO_MODE` from the preamble output.
|
||||
|
||||
**If REPO_MODE is `solo`:**
|
||||
|
||||
Use AskUserQuestion:
|
||||
|
||||
> These test failures appear pre-existing (not caused by your branch changes):
|
||||
>
|
||||
> [list each failure with file:line and brief error description]
|
||||
>
|
||||
> Since this is a solo repo, you're the only one who will fix these.
|
||||
>
|
||||
> RECOMMENDATION: Choose A — fix now while the context is fresh. Completeness: 9/10.
|
||||
> A) Investigate and fix now (human: ~2-4h / CC: ~15min) — Completeness: 10/10
|
||||
> B) Add as P0 TODO — fix after this branch lands — Completeness: 7/10
|
||||
> C) Skip — I know about this, ship anyway — Completeness: 3/10
|
||||
|
||||
**If REPO_MODE is `collaborative` or `unknown`:**
|
||||
|
||||
Use AskUserQuestion:
|
||||
|
||||
> These test failures appear pre-existing (not caused by your branch changes):
|
||||
>
|
||||
> [list each failure with file:line and brief error description]
|
||||
>
|
||||
> This is a collaborative repo — these may be someone else's responsibility.
|
||||
>
|
||||
> RECOMMENDATION: Choose B — assign it to whoever broke it so the right person fixes it. Completeness: 9/10.
|
||||
> A) Investigate and fix now anyway — Completeness: 10/10
|
||||
> B) Blame + assign GitHub issue to the author — Completeness: 9/10
|
||||
> C) Add as P0 TODO — Completeness: 7/10
|
||||
> D) Skip — ship anyway — Completeness: 3/10
|
||||
|
||||
### Step T4: Execute the chosen action
|
||||
|
||||
**If "Investigate and fix now":**
|
||||
- Switch to /investigate mindset: root cause first, then minimal fix.
|
||||
- Fix the pre-existing failure.
|
||||
- Commit the fix separately from the branch's changes: `git commit -m "fix: pre-existing test failure in <test-file>"`
|
||||
- Continue with the workflow.
|
||||
|
||||
**If "Add as P0 TODO":**
|
||||
- If `TODOS.md` exists, add the entry following the format in `review/TODOS-format.md` (or `.claude/skills/review/TODOS-format.md`).
|
||||
- If `TODOS.md` does not exist, create it with the standard header and add the entry.
|
||||
- Entry should include: title, the error output, which branch it was noticed on, and priority P0.
|
||||
- Continue with the workflow — treat the pre-existing failure as non-blocking.
|
||||
|
||||
**If "Blame + assign GitHub issue" (collaborative only):**
|
||||
- Find who likely broke it. Check BOTH the test file AND the production code it tests:
|
||||
```bash
|
||||
# Who last touched the failing test?
|
||||
git log --format="%an (%ae)" -1 -- <failing-test-file>
|
||||
# Who last touched the production code the test covers? (often the actual breaker)
|
||||
git log --format="%an (%ae)" -1 -- <source-file-under-test>
|
||||
```
|
||||
If these are different people, prefer the production code author — they likely introduced the regression.
|
||||
- Create an issue assigned to that person (use the platform detected in Step 0):
|
||||
- **If GitHub:**
|
||||
```bash
|
||||
gh issue create \
|
||||
--title "Pre-existing test failure: <test-name>" \
|
||||
--body "Found failing on branch <current-branch>. Failure is pre-existing.\n\n**Error:**\n```\n<first 10 lines>\n```\n\n**Last modified by:** <author>\n**Noticed by:** gstack /ship on <date>" \
|
||||
--assignee "<github-username>"
|
||||
```
|
||||
- **If GitLab:**
|
||||
```bash
|
||||
glab issue create \
|
||||
-t "Pre-existing test failure: <test-name>" \
|
||||
-d "Found failing on branch <current-branch>. Failure is pre-existing.\n\n**Error:**\n```\n<first 10 lines>\n```\n\n**Last modified by:** <author>\n**Noticed by:** gstack /ship on <date>" \
|
||||
-a "<gitlab-username>"
|
||||
```
|
||||
- If neither CLI is available or `--assignee`/`-a` fails (user not in org, etc.), create the issue without assignee and note who should look at it in the body.
|
||||
- Continue with the workflow.
|
||||
|
||||
**If "Skip":**
|
||||
- Continue with the workflow.
|
||||
- Note in output: "Pre-existing test failure skipped: <test-name>"
|
||||
|
||||
**After triage:** If any in-branch failures remain unfixed, **STOP**. Do not proceed. If all failures were pre-existing and handled (fixed, TODOed, assigned, or skipped), continue to Step 6.
|
||||
|
||||
**If all pass:** Continue silently — just note the counts briefly.
|
||||
|
||||
---
|
||||
|
||||
## Step 6: Eval Suites (conditional)
|
||||
|
||||
Evals are mandatory when prompt-related files change. Skip this step entirely if no prompt files are in the diff.
|
||||
|
||||
**1. Check if the diff touches prompt-related files:**
|
||||
|
||||
```bash
|
||||
git diff origin/<base> --name-only
|
||||
```
|
||||
|
||||
Match against these patterns (from CLAUDE.md):
|
||||
- `app/services/*_prompt_builder.rb`
|
||||
- `app/services/*_generation_service.rb`, `*_writer_service.rb`, `*_designer_service.rb`
|
||||
- `app/services/*_evaluator.rb`, `*_scorer.rb`, `*_classifier_service.rb`, `*_analyzer.rb`
|
||||
- `app/services/concerns/*voice*.rb`, `*writing*.rb`, `*prompt*.rb`, `*token*.rb`
|
||||
- `app/services/chat_tools/*.rb`, `app/services/x_thread_tools/*.rb`
|
||||
- `config/system_prompts/*.txt`
|
||||
- `test/evals/**/*` (eval infrastructure changes affect all suites)
|
||||
|
||||
**If no matches:** Print "No prompt-related files changed — skipping evals." and continue to Step 9.
|
||||
|
||||
**2. Identify affected eval suites:**
|
||||
|
||||
Each eval runner (`test/evals/*_eval_runner.rb`) declares `PROMPT_SOURCE_FILES` listing which source files affect it. Grep these to find which suites match the changed files:
|
||||
|
||||
```bash
|
||||
grep -l "changed_file_basename" test/evals/*_eval_runner.rb
|
||||
```
|
||||
|
||||
Map runner → test file: `post_generation_eval_runner.rb` → `post_generation_eval_test.rb`.
|
||||
|
||||
**Special cases:**
|
||||
- Changes to `test/evals/judges/*.rb`, `test/evals/support/*.rb`, or `test/evals/fixtures/` affect ALL suites that use those judges/support files. Check imports in the eval test files to determine which.
|
||||
- Changes to `config/system_prompts/*.txt` — grep eval runners for the prompt filename to find affected suites.
|
||||
- If unsure which suites are affected, run ALL suites that could plausibly be impacted. Over-testing is better than missing a regression.
|
||||
|
||||
**3. Run affected suites at `EVAL_JUDGE_TIER=full`:**
|
||||
|
||||
`/ship` is a pre-merge gate, so always use full tier (Sonnet structural + Opus persona judges).
|
||||
|
||||
```bash
|
||||
EVAL_JUDGE_TIER=full EVAL_VERBOSE=1 bin/test-lane --eval test/evals/<suite>_eval_test.rb 2>&1 | tee /tmp/ship_evals.txt
|
||||
```
|
||||
|
||||
If multiple suites need to run, run them sequentially (each needs a test lane). If the first suite fails, stop immediately — don't burn API cost on remaining suites.
|
||||
|
||||
**4. Check results:**
|
||||
|
||||
- **If any eval fails:** Show the failures, the cost dashboard, and **STOP**. Do not proceed.
|
||||
- **If all pass:** Note pass counts and cost. Continue to Step 9.
|
||||
|
||||
**5. Save eval output** — include eval results and cost dashboard in the PR body (Step 19).
|
||||
|
||||
**Tier reference (for context — /ship always uses `full`):**
|
||||
| Tier | When | Speed (cached) | Cost |
|
||||
|------|------|----------------|------|
|
||||
| `fast` (Haiku) | Dev iteration, smoke tests | ~5s (14x faster) | ~$0.07/run |
|
||||
| `standard` (Sonnet) | Default dev, `bin/test-lane --eval` | ~17s (4x faster) | ~$0.37/run |
|
||||
| `full` (Opus persona) | **`/ship` and pre-merge** | ~72s (baseline) | ~$1.27/run |
|
||||
|
||||
---
|
||||
@@ -0,0 +1,93 @@
|
||||
## Step 4: Test Framework Bootstrap
|
||||
|
||||
{{TEST_BOOTSTRAP}}
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Run tests (on merged code)
|
||||
|
||||
**Do NOT run `RAILS_ENV=test bin/rails db:migrate`** — `bin/test-lane` already calls
|
||||
`db:test:prepare` internally, which loads the schema into the correct lane database.
|
||||
Running bare test migrations without INSTANCE hits an orphan DB and corrupts structure.sql.
|
||||
|
||||
Run both test suites in parallel:
|
||||
|
||||
```bash
|
||||
bin/test-lane 2>&1 | tee /tmp/ship_tests.txt &
|
||||
npm run test 2>&1 | tee /tmp/ship_vitest.txt &
|
||||
wait
|
||||
```
|
||||
|
||||
After both complete, read the output files and check pass/fail.
|
||||
|
||||
**If any test fails:** Do NOT immediately stop. Apply the Test Failure Ownership Triage:
|
||||
|
||||
{{TEST_FAILURE_TRIAGE}}
|
||||
|
||||
**After triage:** If any in-branch failures remain unfixed, **STOP**. Do not proceed. If all failures were pre-existing and handled (fixed, TODOed, assigned, or skipped), continue to Step 6.
|
||||
|
||||
**If all pass:** Continue silently — just note the counts briefly.
|
||||
|
||||
---
|
||||
|
||||
## Step 6: Eval Suites (conditional)
|
||||
|
||||
Evals are mandatory when prompt-related files change. Skip this step entirely if no prompt files are in the diff.
|
||||
|
||||
**1. Check if the diff touches prompt-related files:**
|
||||
|
||||
```bash
|
||||
git diff origin/<base> --name-only
|
||||
```
|
||||
|
||||
Match against these patterns (from CLAUDE.md):
|
||||
- `app/services/*_prompt_builder.rb`
|
||||
- `app/services/*_generation_service.rb`, `*_writer_service.rb`, `*_designer_service.rb`
|
||||
- `app/services/*_evaluator.rb`, `*_scorer.rb`, `*_classifier_service.rb`, `*_analyzer.rb`
|
||||
- `app/services/concerns/*voice*.rb`, `*writing*.rb`, `*prompt*.rb`, `*token*.rb`
|
||||
- `app/services/chat_tools/*.rb`, `app/services/x_thread_tools/*.rb`
|
||||
- `config/system_prompts/*.txt`
|
||||
- `test/evals/**/*` (eval infrastructure changes affect all suites)
|
||||
|
||||
**If no matches:** Print "No prompt-related files changed — skipping evals." and continue to Step 9.
|
||||
|
||||
**2. Identify affected eval suites:**
|
||||
|
||||
Each eval runner (`test/evals/*_eval_runner.rb`) declares `PROMPT_SOURCE_FILES` listing which source files affect it. Grep these to find which suites match the changed files:
|
||||
|
||||
```bash
|
||||
grep -l "changed_file_basename" test/evals/*_eval_runner.rb
|
||||
```
|
||||
|
||||
Map runner → test file: `post_generation_eval_runner.rb` → `post_generation_eval_test.rb`.
|
||||
|
||||
**Special cases:**
|
||||
- Changes to `test/evals/judges/*.rb`, `test/evals/support/*.rb`, or `test/evals/fixtures/` affect ALL suites that use those judges/support files. Check imports in the eval test files to determine which.
|
||||
- Changes to `config/system_prompts/*.txt` — grep eval runners for the prompt filename to find affected suites.
|
||||
- If unsure which suites are affected, run ALL suites that could plausibly be impacted. Over-testing is better than missing a regression.
|
||||
|
||||
**3. Run affected suites at `EVAL_JUDGE_TIER=full`:**
|
||||
|
||||
`/ship` is a pre-merge gate, so always use full tier (Sonnet structural + Opus persona judges).
|
||||
|
||||
```bash
|
||||
EVAL_JUDGE_TIER=full EVAL_VERBOSE=1 bin/test-lane --eval test/evals/<suite>_eval_test.rb 2>&1 | tee /tmp/ship_evals.txt
|
||||
```
|
||||
|
||||
If multiple suites need to run, run them sequentially (each needs a test lane). If the first suite fails, stop immediately — don't burn API cost on remaining suites.
|
||||
|
||||
**4. Check results:**
|
||||
|
||||
- **If any eval fails:** Show the failures, the cost dashboard, and **STOP**. Do not proceed.
|
||||
- **If all pass:** Note pass counts and cost. Continue to Step 9.
|
||||
|
||||
**5. Save eval output** — include eval results and cost dashboard in the PR body (Step 19).
|
||||
|
||||
**Tier reference (for context — /ship always uses `full`):**
|
||||
| Tier | When | Speed (cached) | Cost |
|
||||
|------|------|----------------|------|
|
||||
| `fast` (Haiku) | Dev iteration, smoke tests | ~5s (14x faster) | ~$0.07/run |
|
||||
| `standard` (Sonnet) | Default dev, `bin/test-lane --eval` | ~17s (4x faster) | ~$0.37/run |
|
||||
| `full` (Opus persona) | **`/ship` and pre-merge** | ~72s (baseline) | ~$1.27/run |
|
||||
|
||||
---
|
||||
@@ -0,0 +1,57 @@
|
||||
/**
|
||||
* Unit coverage for discoverSectionTemplates — the section-discovery half of the
|
||||
* v2 plan T9 pipeline. Drives it against a temp fixture tree so it doesn't
|
||||
* depend on which skills have been carved in the real repo.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, afterAll } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import { discoverSectionTemplates } from '../scripts/discover-skills';
|
||||
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'sections-disc-'));
|
||||
afterAll(() => { try { fs.rmSync(root, { recursive: true, force: true }); } catch { /* noop */ } });
|
||||
|
||||
// ship/ has two section templates + a non-template file; review/ has none;
|
||||
// hidden + node_modules dirs must be skipped by the shared subdirs() filter.
|
||||
fs.mkdirSync(path.join(root, 'ship', 'sections'), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, 'ship', 'SKILL.md.tmpl'), '---\nname: ship\n---\nbody');
|
||||
fs.writeFileSync(path.join(root, 'ship', 'sections', 'version-bump.md.tmpl'), 'bump');
|
||||
fs.writeFileSync(path.join(root, 'ship', 'sections', 'changelog.md.tmpl'), 'changelog');
|
||||
fs.writeFileSync(path.join(root, 'ship', 'sections', 'manifest.json'), '{}'); // not a .md.tmpl
|
||||
fs.mkdirSync(path.join(root, 'review'), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, 'review', 'SKILL.md.tmpl'), '---\nname: review\n---\nbody');
|
||||
fs.mkdirSync(path.join(root, 'node_modules', 'sections'), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, 'node_modules', 'sections', 'x.md.tmpl'), 'nope');
|
||||
|
||||
describe('discoverSectionTemplates', () => {
|
||||
const found = discoverSectionTemplates(root);
|
||||
|
||||
test('finds only *.md.tmpl files inside <skill>/sections/', () => {
|
||||
expect(found.map(f => f.tmpl)).toEqual([
|
||||
'ship/sections/changelog.md.tmpl',
|
||||
'ship/sections/version-bump.md.tmpl',
|
||||
]);
|
||||
});
|
||||
|
||||
test('strips .tmpl for the output path and records the owning skill dir', () => {
|
||||
const bump = found.find(f => f.tmpl.endsWith('version-bump.md.tmpl'))!;
|
||||
expect(bump.output).toBe('ship/sections/version-bump.md');
|
||||
expect(bump.skillDir).toBe('ship');
|
||||
});
|
||||
|
||||
test('ignores non-template files (manifest.json) and skipped dirs (node_modules)', () => {
|
||||
expect(found.some(f => f.tmpl.includes('manifest.json'))).toBe(false);
|
||||
expect(found.some(f => f.tmpl.includes('node_modules'))).toBe(false);
|
||||
});
|
||||
|
||||
test('returns deterministic (sorted) order', () => {
|
||||
const tmpls = found.map(f => f.tmpl);
|
||||
expect([...tmpls].sort()).toEqual(tmpls);
|
||||
});
|
||||
|
||||
test('skills without a sections/ dir contribute nothing', () => {
|
||||
expect(found.some(f => f.skillDir === 'review')).toBe(false);
|
||||
});
|
||||
});
|
||||
+68
-1927
File diff suppressed because it is too large
Load Diff
+39
-138
@@ -805,6 +805,10 @@ Only *actions* are idempotent:
|
||||
- Step 19: If PR exists, update the body instead of creating a new PR
|
||||
Never skip a verification step because a prior `/ship` run already performed it.
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Pre-flight
|
||||
@@ -2098,150 +2102,37 @@ If any learnings come back, name which one applies to the version bump or CHANGE
|
||||
|
||||
## Step 12: Version bump (auto-decide)
|
||||
|
||||
**Idempotency check:** Before bumping, classify the state by comparing `VERSION` against the base branch AND against `package.json`'s `version` field. Four states: FRESH (do bump), ALREADY_BUMPED (skip bump), DRIFT_STALE_PKG (sync pkg only, no re-bump), DRIFT_UNEXPECTED (stop and ask).
|
||||
|
||||
```bash
|
||||
if ! git rev-parse --verify origin/<base> >/dev/null 2>&1; then
|
||||
echo "ERROR: Unable to resolve origin/<base>. Run 'git fetch origin' or verify the base branch exists."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
BASE_VERSION=$(git show origin/<base>:VERSION 2>/dev/null | tr -d '\r\n[:space:]' || echo "0.0.0.0")
|
||||
CURRENT_VERSION=$(cat VERSION 2>/dev/null | tr -d '\r\n[:space:]' || echo "0.0.0.0")
|
||||
[ -z "$BASE_VERSION" ] && BASE_VERSION="0.0.0.0"
|
||||
[ -z "$CURRENT_VERSION" ] && CURRENT_VERSION="0.0.0.0"
|
||||
PKG_VERSION=""
|
||||
PKG_EXISTS=0
|
||||
if [ -f package.json ]; then
|
||||
PKG_EXISTS=1
|
||||
if command -v node >/dev/null 2>&1; then
|
||||
PKG_VERSION=$(node -e 'const p=require("./package.json");process.stdout.write(p.version||"")' 2>/dev/null)
|
||||
PARSE_EXIT=$?
|
||||
elif command -v bun >/dev/null 2>&1; then
|
||||
PKG_VERSION=$(bun -e 'const p=require("./package.json");process.stdout.write(p.version||"")' 2>/dev/null)
|
||||
PARSE_EXIT=$?
|
||||
else
|
||||
echo "ERROR: package.json exists but neither node nor bun is available. Install one and re-run."
|
||||
exit 1
|
||||
fi
|
||||
if [ "$PARSE_EXIT" != "0" ]; then
|
||||
echo "ERROR: package.json is not valid JSON. Fix the file before re-running /ship."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
echo "BASE: $BASE_VERSION VERSION: $CURRENT_VERSION package.json: ${PKG_VERSION:-<none>}"
|
||||
|
||||
if [ "$CURRENT_VERSION" = "$BASE_VERSION" ]; then
|
||||
if [ "$PKG_EXISTS" = "1" ] && [ -n "$PKG_VERSION" ] && [ "$PKG_VERSION" != "$CURRENT_VERSION" ]; then
|
||||
echo "STATE: DRIFT_UNEXPECTED"
|
||||
echo "package.json version ($PKG_VERSION) disagrees with VERSION ($CURRENT_VERSION) while VERSION matches base."
|
||||
echo "This looks like a manual edit to package.json bypassing /ship. Reconcile manually, then re-run."
|
||||
exit 1
|
||||
fi
|
||||
echo "STATE: FRESH"
|
||||
else
|
||||
if [ "$PKG_EXISTS" = "1" ] && [ -n "$PKG_VERSION" ] && [ "$PKG_VERSION" != "$CURRENT_VERSION" ]; then
|
||||
echo "STATE: DRIFT_STALE_PKG"
|
||||
else
|
||||
echo "STATE: ALREADY_BUMPED"
|
||||
fi
|
||||
fi
|
||||
```
|
||||
|
||||
Read the `STATE:` line and dispatch:
|
||||
|
||||
- **FRESH** → proceed with the bump action below (steps 1–4).
|
||||
- **ALREADY_BUMPED** → skip the bump by default, BUT check for queue drift first: call `bin/gstack-next-version` with the implied bump level (derived from `CURRENT_VERSION` vs `BASE_VERSION`), compare its `.version` against `CURRENT_VERSION`. If they differ (queue moved since last ship), use **AskUserQuestion**: "VERSION drift detected: you claim v<CURRENT> but next available is v<NEW> (queue moved). A) Rebump to v<NEW> and rewrite CHANGELOG header + PR title (recommended), B) Keep v<CURRENT> — will be rejected by CI version-gate until resolved." If A, treat this as FRESH with `NEW_VERSION=<new>` and run steps 1-4 (which will also trigger Step 13 CHANGELOG header rewrite and Step 19 PR title rewrite). If B, reuse `CURRENT_VERSION` and warn that CI will likely reject. If util is offline, warn and reuse `CURRENT_VERSION`.
|
||||
- **DRIFT_STALE_PKG** → a prior `/ship` bumped `VERSION` but failed to update `package.json`. Run the sync-only repair block below (after step 4). Do NOT re-bump. Reuse `CURRENT_VERSION` for CHANGELOG and PR body. (Queue check still runs in ALREADY_BUMPED terms after repair.)
|
||||
- **DRIFT_UNEXPECTED** → `/ship` has halted (exit 1). Resolve manually; /ship cannot tell which file is authoritative.
|
||||
|
||||
1. Read the current `VERSION` file (4-digit format: `MAJOR.MINOR.PATCH.MICRO`)
|
||||
|
||||
2. **Auto-decide the bump level based on the diff:**
|
||||
- Count lines changed (`git diff origin/<base>...HEAD --stat | tail -1`)
|
||||
- Check for feature signals: new route/page files (e.g. `app/*/page.tsx`, `pages/*.ts`), new DB migration/schema files, new test files alongside new source files, or branch name starting with `feat/`
|
||||
- **MICRO** (4th digit): < 50 lines changed, trivial tweaks, typos, config
|
||||
- **PATCH** (3rd digit): 50+ lines changed, no feature signals detected
|
||||
- **MINOR** (2nd digit): **ASK the user** if ANY feature signal is detected, OR 500+ lines changed, OR new modules/packages added
|
||||
- **MAJOR** (1st digit): **ASK the user** — only for milestones or breaking changes
|
||||
|
||||
Save the chosen level as `BUMP_LEVEL` (one of `major`, `minor`, `patch`, `micro`). This is the user-intended level. The next step decides *placement* — the level stays the same even if queue-aware allocation has to advance past a claimed slot.
|
||||
|
||||
3. **Queue-aware version pick (workspace-aware ship, v1.6.4.0+).** Call `bin/gstack-next-version` to see what's already claimed by open PRs + active sibling Conductor worktrees, then render the queue state to the user:
|
||||
The deterministic version-state logic is the tested **`gstack-version-bump`** CLI
|
||||
(classify / write / repair). The bump-LEVEL decision and queue-collision handling
|
||||
stay agent judgment; the slot pick stays `gstack-next-version`.
|
||||
|
||||
1. **Classify state** — pure reader, never writes:
|
||||
```bash
|
||||
QUEUE_JSON=$(bun run bin/gstack-next-version \
|
||||
--base <base> \
|
||||
--bump "$BUMP_LEVEL" \
|
||||
--current-version "$BASE_VERSION" 2>/dev/null || echo '{"offline":true}')
|
||||
NEW_VERSION=$(echo "$QUEUE_JSON" | jq -r '.version // empty')
|
||||
CLAIMED_COUNT=$(echo "$QUEUE_JSON" | jq -r '.claimed | length')
|
||||
ACTIVE_SIBLING_COUNT=$(echo "$QUEUE_JSON" | jq -r '.active_siblings | length')
|
||||
OFFLINE=$(echo "$QUEUE_JSON" | jq -r '.offline // false')
|
||||
REASON=$(echo "$QUEUE_JSON" | jq -r '.reason // ""')
|
||||
bun run $GSTACK_ROOT/bin/gstack-version-bump classify --base <base>
|
||||
```
|
||||
Read the JSON `state` and dispatch:
|
||||
- **FRESH** → do the bump (steps 2-4).
|
||||
- **ALREADY_BUMPED** → skip the bump, but run the queue-drift check (step 3) with the reported `currentVersion`. If the queue moved (next free version differs), **AskUserQuestion**: rebump to the new version (rewrites CHANGELOG header + PR title) or keep current (CI version-gate will reject until resolved).
|
||||
- **DRIFT_STALE_PKG** → run `gstack-version-bump repair` (syncs package.json to VERSION). No re-bump; reuse `currentVersion` for CHANGELOG + PR.
|
||||
- **DRIFT_UNEXPECTED** → **STOP**. package.json disagrees with VERSION while VERSION matches base — a manual edit bypassed /ship. Reconcile manually, then re-run.
|
||||
|
||||
- If `OFFLINE=true` or the util fails (auth expired, no `gh`/`glab`, network): fall back to local `BUMP_LEVEL` arithmetic (bump `BASE_VERSION` at the chosen level). Print `⚠ workspace-aware ship offline — using local bump only`. Continue.
|
||||
- If `CLAIMED_COUNT > 0`: render the queue table to the user so they can see landing order at a glance:
|
||||
```
|
||||
Queue on <base> (vBASE_VERSION):
|
||||
#<pr> <branch> → v<version> [⚠ collision with #<other>]
|
||||
Active sibling workspaces (WIP, not yet PR'd):
|
||||
<path> → v<version> (committed Nh ago)
|
||||
Your branch will claim: vNEW_VERSION (<reason>)
|
||||
```
|
||||
- If `ACTIVE_SIBLING_COUNT > 0` and any active sibling's VERSION is `>= NEW_VERSION`, use **AskUserQuestion**: "Sibling workspace <path> has v<X> committed <N>h ago but hasn't PR'd yet. Wait for them to ship first, or advance past? A) Advance past (recommended for unrelated work), B) Abort /ship and sync up with sibling first."
|
||||
- Validate `NEW_VERSION` matches `MAJOR.MINOR.PATCH.MICRO`. If util returns an empty or malformed version, fall back to local bump.
|
||||
2. **Decide the bump level** from the diff (agent judgment):
|
||||
- **MICRO**: <50 lines, trivial tweaks/config. **PATCH**: 50+ lines, no feature signals.
|
||||
- **MINOR**: **ASK** if any feature signal (new route/page, migration, new module), OR 500+ lines. **MAJOR**: **ASK** — milestones or breaking changes only.
|
||||
Save as `BUMP_LEVEL`. The level is the user-intended bump; queue-aware placement may advance the slot without changing the level.
|
||||
|
||||
4. **Validate** `NEW_VERSION` and write it to **both** `VERSION` and `package.json`. This block runs only when `STATE: FRESH`.
|
||||
3. **Queue-aware pick** (workspace-aware ship):
|
||||
```bash
|
||||
QUEUE_JSON=$(bun run $GSTACK_ROOT/bin/gstack-next-version --base <base> --bump "$BUMP_LEVEL" --current-version "$BASE_VERSION" 2>/dev/null || echo '{"offline":true}')
|
||||
NEW_VERSION=$(echo "$QUEUE_JSON" | jq -r '.version // empty')
|
||||
```
|
||||
If `offline`/util fails: fall back to local `BUMP_LEVEL` arithmetic and print `⚠ workspace-aware ship offline — using local bump only`. If `claimed` is non-empty, render the queue table so the user sees landing order. If an active sibling workspace holds a version `>= NEW_VERSION`, **AskUserQuestion**: advance past (unrelated work) or abort and sync with the sibling.
|
||||
|
||||
```bash
|
||||
if ! printf '%s' "$NEW_VERSION" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'; then
|
||||
echo "ERROR: NEW_VERSION ($NEW_VERSION) does not match MAJOR.MINOR.PATCH.MICRO pattern. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
echo "$NEW_VERSION" > VERSION
|
||||
if [ -f package.json ]; then
|
||||
if command -v node >/dev/null 2>&1; then
|
||||
node -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$NEW_VERSION" || {
|
||||
echo "ERROR: failed to update package.json. VERSION was written but package.json is now stale. Fix and re-run — the new idempotency check will detect the drift."
|
||||
exit 1
|
||||
}
|
||||
elif command -v bun >/dev/null 2>&1; then
|
||||
bun -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$NEW_VERSION" || {
|
||||
echo "ERROR: failed to update package.json. VERSION was written but package.json is now stale."
|
||||
exit 1
|
||||
}
|
||||
else
|
||||
echo "ERROR: package.json exists but neither node nor bun is available."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
```
|
||||
|
||||
**DRIFT_STALE_PKG repair path** — runs when idempotency reports `STATE: DRIFT_STALE_PKG`. No re-bump; sync `package.json.version` to the current `VERSION` and continue. Reuse `CURRENT_VERSION` for CHANGELOG and PR body.
|
||||
|
||||
```bash
|
||||
REPAIR_VERSION=$(cat VERSION | tr -d '\r\n[:space:]')
|
||||
if ! printf '%s' "$REPAIR_VERSION" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'; then
|
||||
echo "ERROR: VERSION file contents ($REPAIR_VERSION) do not match MAJOR.MINOR.PATCH.MICRO pattern. Refusing to propagate invalid semver into package.json. Fix VERSION manually, then re-run /ship."
|
||||
exit 1
|
||||
fi
|
||||
if command -v node >/dev/null 2>&1; then
|
||||
node -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$REPAIR_VERSION" || {
|
||||
echo "ERROR: drift repair failed — could not update package.json."
|
||||
exit 1
|
||||
}
|
||||
else
|
||||
bun -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$REPAIR_VERSION" || {
|
||||
echo "ERROR: drift repair failed."
|
||||
exit 1
|
||||
}
|
||||
fi
|
||||
echo "Drift repaired: package.json synced to $REPAIR_VERSION. No version bump performed."
|
||||
```
|
||||
|
||||
---
|
||||
4. **Write the bump** (FRESH, or an approved rebump):
|
||||
```bash
|
||||
bun run $GSTACK_ROOT/bin/gstack-version-bump write --version "$NEW_VERSION"
|
||||
```
|
||||
The CLI validates the 4-digit `MAJOR.MINOR.PATCH.MICRO` pattern and writes **both** VERSION and package.json. On a half-write (VERSION written, package.json failed) it exits 3 — re-run, and classify will report DRIFT_STALE_PKG for `repair` to fix.
|
||||
|
||||
## Step 13: CHANGELOG (auto-generate)
|
||||
|
||||
@@ -2746,6 +2637,16 @@ no-op. The marker guarantees at-most-once per machine. To re-enable:
|
||||
|
||||
---
|
||||
|
||||
## Section self-check (before you finish)
|
||||
|
||||
You ran a carved skill. For your situation, list every section the Section index
|
||||
named as applying, and confirm you issued a Read for each one. If you executed any
|
||||
of those steps from memory without reading its section, you skipped the source of
|
||||
truth — STOP, Read it now, and redo that step. Deterministic version work goes
|
||||
through `gstack-version-bump`; never hand-roll the VERSION/package.json write.
|
||||
|
||||
---
|
||||
|
||||
## Important Rules
|
||||
|
||||
- **Never skip tests.** If tests fail, stop.
|
||||
|
||||
+39
-138
@@ -807,6 +807,10 @@ Only *actions* are idempotent:
|
||||
- Step 19: If PR exists, update the body instead of creating a new PR
|
||||
Never skip a verification step because a prior `/ship` run already performed it.
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Pre-flight
|
||||
@@ -2476,150 +2480,37 @@ If any learnings come back, name which one applies to the version bump or CHANGE
|
||||
|
||||
## Step 12: Version bump (auto-decide)
|
||||
|
||||
**Idempotency check:** Before bumping, classify the state by comparing `VERSION` against the base branch AND against `package.json`'s `version` field. Four states: FRESH (do bump), ALREADY_BUMPED (skip bump), DRIFT_STALE_PKG (sync pkg only, no re-bump), DRIFT_UNEXPECTED (stop and ask).
|
||||
|
||||
```bash
|
||||
if ! git rev-parse --verify origin/<base> >/dev/null 2>&1; then
|
||||
echo "ERROR: Unable to resolve origin/<base>. Run 'git fetch origin' or verify the base branch exists."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
BASE_VERSION=$(git show origin/<base>:VERSION 2>/dev/null | tr -d '\r\n[:space:]' || echo "0.0.0.0")
|
||||
CURRENT_VERSION=$(cat VERSION 2>/dev/null | tr -d '\r\n[:space:]' || echo "0.0.0.0")
|
||||
[ -z "$BASE_VERSION" ] && BASE_VERSION="0.0.0.0"
|
||||
[ -z "$CURRENT_VERSION" ] && CURRENT_VERSION="0.0.0.0"
|
||||
PKG_VERSION=""
|
||||
PKG_EXISTS=0
|
||||
if [ -f package.json ]; then
|
||||
PKG_EXISTS=1
|
||||
if command -v node >/dev/null 2>&1; then
|
||||
PKG_VERSION=$(node -e 'const p=require("./package.json");process.stdout.write(p.version||"")' 2>/dev/null)
|
||||
PARSE_EXIT=$?
|
||||
elif command -v bun >/dev/null 2>&1; then
|
||||
PKG_VERSION=$(bun -e 'const p=require("./package.json");process.stdout.write(p.version||"")' 2>/dev/null)
|
||||
PARSE_EXIT=$?
|
||||
else
|
||||
echo "ERROR: package.json exists but neither node nor bun is available. Install one and re-run."
|
||||
exit 1
|
||||
fi
|
||||
if [ "$PARSE_EXIT" != "0" ]; then
|
||||
echo "ERROR: package.json is not valid JSON. Fix the file before re-running /ship."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
echo "BASE: $BASE_VERSION VERSION: $CURRENT_VERSION package.json: ${PKG_VERSION:-<none>}"
|
||||
|
||||
if [ "$CURRENT_VERSION" = "$BASE_VERSION" ]; then
|
||||
if [ "$PKG_EXISTS" = "1" ] && [ -n "$PKG_VERSION" ] && [ "$PKG_VERSION" != "$CURRENT_VERSION" ]; then
|
||||
echo "STATE: DRIFT_UNEXPECTED"
|
||||
echo "package.json version ($PKG_VERSION) disagrees with VERSION ($CURRENT_VERSION) while VERSION matches base."
|
||||
echo "This looks like a manual edit to package.json bypassing /ship. Reconcile manually, then re-run."
|
||||
exit 1
|
||||
fi
|
||||
echo "STATE: FRESH"
|
||||
else
|
||||
if [ "$PKG_EXISTS" = "1" ] && [ -n "$PKG_VERSION" ] && [ "$PKG_VERSION" != "$CURRENT_VERSION" ]; then
|
||||
echo "STATE: DRIFT_STALE_PKG"
|
||||
else
|
||||
echo "STATE: ALREADY_BUMPED"
|
||||
fi
|
||||
fi
|
||||
```
|
||||
|
||||
Read the `STATE:` line and dispatch:
|
||||
|
||||
- **FRESH** → proceed with the bump action below (steps 1–4).
|
||||
- **ALREADY_BUMPED** → skip the bump by default, BUT check for queue drift first: call `bin/gstack-next-version` with the implied bump level (derived from `CURRENT_VERSION` vs `BASE_VERSION`), compare its `.version` against `CURRENT_VERSION`. If they differ (queue moved since last ship), use **AskUserQuestion**: "VERSION drift detected: you claim v<CURRENT> but next available is v<NEW> (queue moved). A) Rebump to v<NEW> and rewrite CHANGELOG header + PR title (recommended), B) Keep v<CURRENT> — will be rejected by CI version-gate until resolved." If A, treat this as FRESH with `NEW_VERSION=<new>` and run steps 1-4 (which will also trigger Step 13 CHANGELOG header rewrite and Step 19 PR title rewrite). If B, reuse `CURRENT_VERSION` and warn that CI will likely reject. If util is offline, warn and reuse `CURRENT_VERSION`.
|
||||
- **DRIFT_STALE_PKG** → a prior `/ship` bumped `VERSION` but failed to update `package.json`. Run the sync-only repair block below (after step 4). Do NOT re-bump. Reuse `CURRENT_VERSION` for CHANGELOG and PR body. (Queue check still runs in ALREADY_BUMPED terms after repair.)
|
||||
- **DRIFT_UNEXPECTED** → `/ship` has halted (exit 1). Resolve manually; /ship cannot tell which file is authoritative.
|
||||
|
||||
1. Read the current `VERSION` file (4-digit format: `MAJOR.MINOR.PATCH.MICRO`)
|
||||
|
||||
2. **Auto-decide the bump level based on the diff:**
|
||||
- Count lines changed (`git diff origin/<base>...HEAD --stat | tail -1`)
|
||||
- Check for feature signals: new route/page files (e.g. `app/*/page.tsx`, `pages/*.ts`), new DB migration/schema files, new test files alongside new source files, or branch name starting with `feat/`
|
||||
- **MICRO** (4th digit): < 50 lines changed, trivial tweaks, typos, config
|
||||
- **PATCH** (3rd digit): 50+ lines changed, no feature signals detected
|
||||
- **MINOR** (2nd digit): **ASK the user** if ANY feature signal is detected, OR 500+ lines changed, OR new modules/packages added
|
||||
- **MAJOR** (1st digit): **ASK the user** — only for milestones or breaking changes
|
||||
|
||||
Save the chosen level as `BUMP_LEVEL` (one of `major`, `minor`, `patch`, `micro`). This is the user-intended level. The next step decides *placement* — the level stays the same even if queue-aware allocation has to advance past a claimed slot.
|
||||
|
||||
3. **Queue-aware version pick (workspace-aware ship, v1.6.4.0+).** Call `bin/gstack-next-version` to see what's already claimed by open PRs + active sibling Conductor worktrees, then render the queue state to the user:
|
||||
The deterministic version-state logic is the tested **`gstack-version-bump`** CLI
|
||||
(classify / write / repair). The bump-LEVEL decision and queue-collision handling
|
||||
stay agent judgment; the slot pick stays `gstack-next-version`.
|
||||
|
||||
1. **Classify state** — pure reader, never writes:
|
||||
```bash
|
||||
QUEUE_JSON=$(bun run bin/gstack-next-version \
|
||||
--base <base> \
|
||||
--bump "$BUMP_LEVEL" \
|
||||
--current-version "$BASE_VERSION" 2>/dev/null || echo '{"offline":true}')
|
||||
NEW_VERSION=$(echo "$QUEUE_JSON" | jq -r '.version // empty')
|
||||
CLAIMED_COUNT=$(echo "$QUEUE_JSON" | jq -r '.claimed | length')
|
||||
ACTIVE_SIBLING_COUNT=$(echo "$QUEUE_JSON" | jq -r '.active_siblings | length')
|
||||
OFFLINE=$(echo "$QUEUE_JSON" | jq -r '.offline // false')
|
||||
REASON=$(echo "$QUEUE_JSON" | jq -r '.reason // ""')
|
||||
bun run $GSTACK_ROOT/bin/gstack-version-bump classify --base <base>
|
||||
```
|
||||
Read the JSON `state` and dispatch:
|
||||
- **FRESH** → do the bump (steps 2-4).
|
||||
- **ALREADY_BUMPED** → skip the bump, but run the queue-drift check (step 3) with the reported `currentVersion`. If the queue moved (next free version differs), **AskUserQuestion**: rebump to the new version (rewrites CHANGELOG header + PR title) or keep current (CI version-gate will reject until resolved).
|
||||
- **DRIFT_STALE_PKG** → run `gstack-version-bump repair` (syncs package.json to VERSION). No re-bump; reuse `currentVersion` for CHANGELOG + PR.
|
||||
- **DRIFT_UNEXPECTED** → **STOP**. package.json disagrees with VERSION while VERSION matches base — a manual edit bypassed /ship. Reconcile manually, then re-run.
|
||||
|
||||
- If `OFFLINE=true` or the util fails (auth expired, no `gh`/`glab`, network): fall back to local `BUMP_LEVEL` arithmetic (bump `BASE_VERSION` at the chosen level). Print `⚠ workspace-aware ship offline — using local bump only`. Continue.
|
||||
- If `CLAIMED_COUNT > 0`: render the queue table to the user so they can see landing order at a glance:
|
||||
```
|
||||
Queue on <base> (vBASE_VERSION):
|
||||
#<pr> <branch> → v<version> [⚠ collision with #<other>]
|
||||
Active sibling workspaces (WIP, not yet PR'd):
|
||||
<path> → v<version> (committed Nh ago)
|
||||
Your branch will claim: vNEW_VERSION (<reason>)
|
||||
```
|
||||
- If `ACTIVE_SIBLING_COUNT > 0` and any active sibling's VERSION is `>= NEW_VERSION`, use **AskUserQuestion**: "Sibling workspace <path> has v<X> committed <N>h ago but hasn't PR'd yet. Wait for them to ship first, or advance past? A) Advance past (recommended for unrelated work), B) Abort /ship and sync up with sibling first."
|
||||
- Validate `NEW_VERSION` matches `MAJOR.MINOR.PATCH.MICRO`. If util returns an empty or malformed version, fall back to local bump.
|
||||
2. **Decide the bump level** from the diff (agent judgment):
|
||||
- **MICRO**: <50 lines, trivial tweaks/config. **PATCH**: 50+ lines, no feature signals.
|
||||
- **MINOR**: **ASK** if any feature signal (new route/page, migration, new module), OR 500+ lines. **MAJOR**: **ASK** — milestones or breaking changes only.
|
||||
Save as `BUMP_LEVEL`. The level is the user-intended bump; queue-aware placement may advance the slot without changing the level.
|
||||
|
||||
4. **Validate** `NEW_VERSION` and write it to **both** `VERSION` and `package.json`. This block runs only when `STATE: FRESH`.
|
||||
3. **Queue-aware pick** (workspace-aware ship):
|
||||
```bash
|
||||
QUEUE_JSON=$(bun run $GSTACK_ROOT/bin/gstack-next-version --base <base> --bump "$BUMP_LEVEL" --current-version "$BASE_VERSION" 2>/dev/null || echo '{"offline":true}')
|
||||
NEW_VERSION=$(echo "$QUEUE_JSON" | jq -r '.version // empty')
|
||||
```
|
||||
If `offline`/util fails: fall back to local `BUMP_LEVEL` arithmetic and print `⚠ workspace-aware ship offline — using local bump only`. If `claimed` is non-empty, render the queue table so the user sees landing order. If an active sibling workspace holds a version `>= NEW_VERSION`, **AskUserQuestion**: advance past (unrelated work) or abort and sync with the sibling.
|
||||
|
||||
```bash
|
||||
if ! printf '%s' "$NEW_VERSION" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'; then
|
||||
echo "ERROR: NEW_VERSION ($NEW_VERSION) does not match MAJOR.MINOR.PATCH.MICRO pattern. Aborting."
|
||||
exit 1
|
||||
fi
|
||||
echo "$NEW_VERSION" > VERSION
|
||||
if [ -f package.json ]; then
|
||||
if command -v node >/dev/null 2>&1; then
|
||||
node -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$NEW_VERSION" || {
|
||||
echo "ERROR: failed to update package.json. VERSION was written but package.json is now stale. Fix and re-run — the new idempotency check will detect the drift."
|
||||
exit 1
|
||||
}
|
||||
elif command -v bun >/dev/null 2>&1; then
|
||||
bun -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$NEW_VERSION" || {
|
||||
echo "ERROR: failed to update package.json. VERSION was written but package.json is now stale."
|
||||
exit 1
|
||||
}
|
||||
else
|
||||
echo "ERROR: package.json exists but neither node nor bun is available."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
```
|
||||
|
||||
**DRIFT_STALE_PKG repair path** — runs when idempotency reports `STATE: DRIFT_STALE_PKG`. No re-bump; sync `package.json.version` to the current `VERSION` and continue. Reuse `CURRENT_VERSION` for CHANGELOG and PR body.
|
||||
|
||||
```bash
|
||||
REPAIR_VERSION=$(cat VERSION | tr -d '\r\n[:space:]')
|
||||
if ! printf '%s' "$REPAIR_VERSION" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'; then
|
||||
echo "ERROR: VERSION file contents ($REPAIR_VERSION) do not match MAJOR.MINOR.PATCH.MICRO pattern. Refusing to propagate invalid semver into package.json. Fix VERSION manually, then re-run /ship."
|
||||
exit 1
|
||||
fi
|
||||
if command -v node >/dev/null 2>&1; then
|
||||
node -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$REPAIR_VERSION" || {
|
||||
echo "ERROR: drift repair failed — could not update package.json."
|
||||
exit 1
|
||||
}
|
||||
else
|
||||
bun -e 'const fs=require("fs"),p=require("./package.json");p.version=process.argv[1];fs.writeFileSync("package.json",JSON.stringify(p,null,2)+"\n")' "$REPAIR_VERSION" || {
|
||||
echo "ERROR: drift repair failed."
|
||||
exit 1
|
||||
}
|
||||
fi
|
||||
echo "Drift repaired: package.json synced to $REPAIR_VERSION. No version bump performed."
|
||||
```
|
||||
|
||||
---
|
||||
4. **Write the bump** (FRESH, or an approved rebump):
|
||||
```bash
|
||||
bun run $GSTACK_ROOT/bin/gstack-version-bump write --version "$NEW_VERSION"
|
||||
```
|
||||
The CLI validates the 4-digit `MAJOR.MINOR.PATCH.MICRO` pattern and writes **both** VERSION and package.json. On a half-write (VERSION written, package.json failed) it exits 3 — re-run, and classify will report DRIFT_STALE_PKG for `repair` to fix.
|
||||
|
||||
## Step 13: CHANGELOG (auto-generate)
|
||||
|
||||
@@ -3124,6 +3015,16 @@ no-op. The marker guarantees at-most-once per machine. To re-enable:
|
||||
|
||||
---
|
||||
|
||||
## Section self-check (before you finish)
|
||||
|
||||
You ran a carved skill. For your situation, list every section the Section index
|
||||
named as applying, and confirm you issued a Read for each one. If you executed any
|
||||
of those steps from memory without reading its section, you skipped the source of
|
||||
truth — STOP, Read it now, and redo that step. Deterministic version work goes
|
||||
through `gstack-version-bump`; never hand-roll the VERSION/package.json write.
|
||||
|
||||
---
|
||||
|
||||
## Important Rules
|
||||
|
||||
- **Never skip tests.** If tests fail, stop.
|
||||
|
||||
+48
-25
@@ -8,6 +8,24 @@ import * as os from 'os';
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
const MAX_SKILL_DESCRIPTION_LENGTH = 1024;
|
||||
|
||||
// Carved-skill aware (v2 plan T9): ship is now a skeleton SKILL.md + sections/*.md.
|
||||
// Read the union so assertions about content that MOVED into a section still pass.
|
||||
// The skeleton is a subset of the union, so skeleton-only assertions also hold,
|
||||
// and negative assertions stay safe (the absent phrases live in neither file).
|
||||
function readSkillUnion(skill: string): string {
|
||||
let t = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
||||
const secDir = path.join(ROOT, skill, 'sections');
|
||||
if (fs.existsSync(secDir)) {
|
||||
for (const f of fs.readdirSync(secDir).sort()) {
|
||||
if (f.endsWith('.md')) t += '\n' + fs.readFileSync(path.join(secDir, f), 'utf-8');
|
||||
}
|
||||
}
|
||||
return t;
|
||||
}
|
||||
function readShipUnion(): string {
|
||||
return readSkillUnion('ship');
|
||||
}
|
||||
|
||||
function extractDescription(content: string): string {
|
||||
const fmEnd = content.indexOf('\n---', 4);
|
||||
expect(fmEnd).toBeGreaterThan(0);
|
||||
@@ -485,7 +503,7 @@ describe('gen-skill-docs', () => {
|
||||
|
||||
describe('BASE_BRANCH_DETECT resolver', () => {
|
||||
// Find a generated SKILL.md that uses the placeholder (ship is guaranteed to)
|
||||
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipContent = readShipUnion();
|
||||
|
||||
test('resolver output contains PR base detection command', () => {
|
||||
expect(shipContent).toContain('gh pr view --json baseRefName');
|
||||
@@ -518,7 +536,7 @@ describe('BASE_BRANCH_DETECT resolver', () => {
|
||||
|
||||
describe('GitLab support in generated skills', () => {
|
||||
const retroContent = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
|
||||
const shipSkillContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipSkillContent = readShipUnion();
|
||||
|
||||
test('retro contains GitLab MR number extraction', () => {
|
||||
expect(retroContent).toContain('[#!]');
|
||||
@@ -634,13 +652,13 @@ describe('REVIEW_DASHBOARD resolver', () => {
|
||||
}
|
||||
|
||||
test('review dashboard appears in ship generated file', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('reviews.jsonl');
|
||||
expect(content).toContain('REVIEW READINESS DASHBOARD');
|
||||
});
|
||||
|
||||
test('dashboard treats review as a valid Eng Review source', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('plan-eng-review, review, plan-design-review');
|
||||
expect(content).toContain('`review` (diff-scoped pre-landing review)');
|
||||
expect(content).toContain('`plan-eng-review` (plan-stage architecture review)');
|
||||
@@ -708,7 +726,7 @@ describe('REVIEW_DASHBOARD resolver', () => {
|
||||
});
|
||||
|
||||
test('ship does NOT contain review chaining', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).not.toContain('Review Chaining');
|
||||
});
|
||||
});
|
||||
@@ -717,7 +735,7 @@ describe('REVIEW_DASHBOARD resolver', () => {
|
||||
|
||||
describe('TEST_COVERAGE_AUDIT placeholders', () => {
|
||||
const planSkill = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
|
||||
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipSkill = readShipUnion();
|
||||
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
||||
|
||||
test('plan and ship modes share codepath tracing methodology', () => {
|
||||
@@ -874,7 +892,7 @@ describe('TEST_COVERAGE_AUDIT placeholders', () => {
|
||||
// --- {{TEST_FAILURE_TRIAGE}} resolver tests ---
|
||||
|
||||
describe('TEST_FAILURE_TRIAGE resolver', () => {
|
||||
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipSkill = readShipUnion();
|
||||
|
||||
test('contains all 4 triage steps', () => {
|
||||
expect(shipSkill).toContain('Step T1: Classify each failure');
|
||||
@@ -938,7 +956,7 @@ describe('PLAN_FILE_REVIEW_REPORT resolver', () => {
|
||||
// --- {{PLAN_COMPLETION_AUDIT}} resolver tests ---
|
||||
|
||||
describe('PLAN_COMPLETION_AUDIT placeholders', () => {
|
||||
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipSkill = readShipUnion();
|
||||
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
||||
|
||||
test('ship SKILL.md contains plan completion audit step', () => {
|
||||
@@ -989,7 +1007,7 @@ describe('PLAN_COMPLETION_AUDIT placeholders', () => {
|
||||
// --- {{PLAN_VERIFICATION_EXEC}} resolver tests ---
|
||||
|
||||
describe('PLAN_VERIFICATION_EXEC placeholder', () => {
|
||||
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipSkill = readShipUnion();
|
||||
|
||||
test('ship SKILL.md contains plan verification step', () => {
|
||||
expect(shipSkill).toContain('Step 8.1');
|
||||
@@ -1018,7 +1036,7 @@ describe('PLAN_VERIFICATION_EXEC placeholder', () => {
|
||||
// --- Coverage gate tests ---
|
||||
|
||||
describe('Coverage gate in ship', () => {
|
||||
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipSkill = readShipUnion();
|
||||
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
||||
|
||||
test('ship SKILL.md contains coverage gate with thresholds', () => {
|
||||
@@ -1047,7 +1065,7 @@ describe('Coverage gate in ship', () => {
|
||||
// --- Ship metrics logging ---
|
||||
|
||||
describe('Ship metrics logging', () => {
|
||||
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipSkill = readShipUnion();
|
||||
|
||||
test('ship SKILL.md contains metrics persistence step', () => {
|
||||
expect(shipSkill).toContain('Step 20');
|
||||
@@ -1063,7 +1081,7 @@ describe('Ship metrics logging', () => {
|
||||
describe('Plan file discovery shared helper', () => {
|
||||
// The shared helper should appear in ship (via PLAN_COMPLETION_AUDIT_SHIP)
|
||||
// and in review (via PLAN_COMPLETION_AUDIT_REVIEW)
|
||||
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipSkill = readShipUnion();
|
||||
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
||||
|
||||
test('plan file discovery appears in both ship and review', () => {
|
||||
@@ -1276,7 +1294,8 @@ describe('Codex filesystem boundary', () => {
|
||||
|
||||
test('boundary instruction appears in all skills that call codex', () => {
|
||||
for (const skill of CODEX_CALLING_SKILLS) {
|
||||
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
||||
// Union: ship's codex call lives in sections/adversarial.md after the carve.
|
||||
const content = readSkillUnion(skill);
|
||||
expect(content).toContain(BOUNDARY_MARKER);
|
||||
}
|
||||
});
|
||||
@@ -1393,7 +1412,7 @@ describe('INVOKE_SKILL resolver', () => {
|
||||
// --- {{CHANGELOG_WORKFLOW}} resolver tests ---
|
||||
|
||||
describe('CHANGELOG_WORKFLOW resolver', () => {
|
||||
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipContent = readShipUnion();
|
||||
|
||||
test('ship SKILL.md contains changelog workflow', () => {
|
||||
expect(shipContent).toContain('CHANGELOG (auto-generate)');
|
||||
@@ -1410,10 +1429,13 @@ describe('CHANGELOG_WORKFLOW resolver', () => {
|
||||
});
|
||||
|
||||
test('template uses {{CHANGELOG_WORKFLOW}} placeholder', () => {
|
||||
const tmpl = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md.tmpl'), 'utf-8');
|
||||
expect(tmpl).toContain('{{CHANGELOG_WORKFLOW}}');
|
||||
// Should NOT contain the old inline changelog content
|
||||
expect(tmpl).not.toContain('Group commits by theme');
|
||||
// Post-carve (T9): the skeleton points to the changelog section, which carries
|
||||
// the resolver. Neither should inline the old changelog content.
|
||||
const skel = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md.tmpl'), 'utf-8');
|
||||
const changelogSection = fs.readFileSync(path.join(ROOT, 'ship', 'sections', 'changelog.md.tmpl'), 'utf-8');
|
||||
expect(skel).toContain('{{SECTION:changelog}}');
|
||||
expect(changelogSection).toContain('{{CHANGELOG_WORKFLOW}}');
|
||||
expect(skel + changelogSection).not.toContain('Group commits by theme');
|
||||
});
|
||||
|
||||
test('changelog workflow includes keep-changelog format', () => {
|
||||
@@ -1450,7 +1472,7 @@ describe('parameterized resolver support', () => {
|
||||
// --- Preamble routing injection tests ---
|
||||
|
||||
describe('preamble routing injection', () => {
|
||||
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipContent = readShipUnion();
|
||||
|
||||
test('preamble bash checks for routing section in CLAUDE.md', () => {
|
||||
expect(shipContent).toContain('grep -q "## Skill routing" CLAUDE.md');
|
||||
@@ -1594,7 +1616,7 @@ describe('DESIGN_SKETCH extended with outside voices', () => {
|
||||
// --- Extended DESIGN_REVIEW_LITE resolver tests ---
|
||||
|
||||
describe('DESIGN_REVIEW_LITE extended with Codex', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
|
||||
test('contains Codex design voice block', () => {
|
||||
expect(content).toContain('Codex design voice');
|
||||
@@ -1897,7 +1919,7 @@ describe('Codex generation (--host codex)', () => {
|
||||
});
|
||||
|
||||
test('Claude output unchanged: ship skill still uses .claude/skills/ paths', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('~/.claude/skills/gstack');
|
||||
expect(content).not.toContain('.agents/skills');
|
||||
expect(content).not.toContain('~/.codex/');
|
||||
@@ -2586,7 +2608,7 @@ describe('community fixes wave', () => {
|
||||
|
||||
// #573 — Feature signals: ship/SKILL.md contains feature signal detection
|
||||
test('ship/SKILL.md contains feature signal detection in Step 4', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content.toLowerCase()).toContain('feature signal');
|
||||
});
|
||||
|
||||
@@ -2736,7 +2758,8 @@ describe('codex commands must not use inline $(git rev-parse --show-toplevel) fo
|
||||
];
|
||||
|
||||
for (const rel of checkedFiles) {
|
||||
const content = fs.readFileSync(path.join(ROOT, rel), 'utf-8');
|
||||
// ship's codex/adversarial command moved into sections/adversarial.md (T9 carve).
|
||||
const content = rel === 'ship/SKILL.md' ? readShipUnion() : fs.readFileSync(path.join(ROOT, rel), 'utf-8');
|
||||
expect(content).not.toContain('--base <base> -c \'model_reasoning_effort="high"\'');
|
||||
expect(content).toContain('Run git diff origin/<base>...HEAD 2>/dev/null || git diff <base>...HEAD');
|
||||
}
|
||||
@@ -2750,7 +2773,7 @@ describe('LEARNINGS_SEARCH resolver', () => {
|
||||
|
||||
for (const skill of SEARCH_SKILLS) {
|
||||
test(`${skill} generated SKILL.md contains learnings search`, () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
||||
const content = readSkillUnion(skill); // ship: moved to sections/plan-completion.md
|
||||
expect(content).toContain('Prior Learnings');
|
||||
expect(content).toContain('gstack-learnings-search');
|
||||
});
|
||||
@@ -2811,7 +2834,7 @@ describe('CONFIDENCE_CALIBRATION resolver', () => {
|
||||
|
||||
for (const skill of CONFIDENCE_SKILLS) {
|
||||
test(`${skill} generated SKILL.md contains confidence calibration`, () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
||||
const content = readSkillUnion(skill); // ship: moved to sections/review-army.md
|
||||
expect(content).toContain('Confidence Calibration');
|
||||
expect(content).toContain('confidence score');
|
||||
});
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
/**
|
||||
* Tests for the gstack-version-bump CLI (v2 plan T9 hybrid extraction). Covers
|
||||
* the idempotency classifier (pure) + the write/repair mutations (temp fs).
|
||||
* The classifier is the one that prevents re-bumping an already-shipped branch —
|
||||
* the worst /ship footgun — so it gets exhaustive state coverage.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, afterAll } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import { execFileSync } from 'child_process';
|
||||
import { classifyState, VERSION_RE } from '../bin/gstack-version-bump';
|
||||
|
||||
const BIN = path.join(import.meta.dir, '..', 'bin', 'gstack-version-bump');
|
||||
|
||||
describe('classifyState (idempotency)', () => {
|
||||
test('FRESH when VERSION matches base and pkg agrees', () => {
|
||||
expect(classifyState('1.1.0.0', '1.1.0.0', true, '1.1.0.0')).toBe('FRESH');
|
||||
});
|
||||
test('FRESH when VERSION matches base and no package.json', () => {
|
||||
expect(classifyState('1.1.0.0', '1.1.0.0', false, '')).toBe('FRESH');
|
||||
});
|
||||
test('ALREADY_BUMPED when VERSION moved past base and pkg agrees (re-run)', () => {
|
||||
expect(classifyState('1.2.0.0', '1.1.0.0', true, '1.2.0.0')).toBe('ALREADY_BUMPED');
|
||||
});
|
||||
test('ALREADY_BUMPED when VERSION moved past base, no package.json', () => {
|
||||
expect(classifyState('1.2.0.0', '1.1.0.0', false, '')).toBe('ALREADY_BUMPED');
|
||||
});
|
||||
test('DRIFT_STALE_PKG when VERSION bumped but pkg lagging', () => {
|
||||
expect(classifyState('1.2.0.0', '1.1.0.0', true, '1.1.0.0')).toBe('DRIFT_STALE_PKG');
|
||||
});
|
||||
test('DRIFT_UNEXPECTED when VERSION matches base but pkg diverges (manual edit)', () => {
|
||||
expect(classifyState('1.1.0.0', '1.1.0.0', true, '1.2.0.0')).toBe('DRIFT_UNEXPECTED');
|
||||
});
|
||||
});
|
||||
|
||||
describe('VERSION_RE', () => {
|
||||
test('accepts 4-digit semver', () => {
|
||||
expect(VERSION_RE.test('1.2.3.4')).toBe(true);
|
||||
});
|
||||
test('rejects 3-digit and garbage', () => {
|
||||
expect(VERSION_RE.test('1.2.3')).toBe(false);
|
||||
expect(VERSION_RE.test('v1.2.3.4')).toBe(false);
|
||||
expect(VERSION_RE.test('1.2.3.4-rc')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('write (FRESH bump)', () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'vbump-write-'));
|
||||
afterAll(() => { try { fs.rmSync(dir, { recursive: true, force: true }); } catch { /* noop */ } });
|
||||
|
||||
test('writes VERSION + package.json.version, preserving other pkg fields', () => {
|
||||
fs.writeFileSync(path.join(dir, 'VERSION'), '1.0.0.0\n');
|
||||
fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ name: 'x', version: '1.0.0.0', scripts: { t: 'y' } }, null, 2) + '\n');
|
||||
const out = execFileSync('bun', [BIN, 'write', '--version', '1.1.0.0'], { cwd: dir }).toString();
|
||||
expect(JSON.parse(out)).toEqual({ wrote: '1.1.0.0', packageJson: true });
|
||||
expect(fs.readFileSync(path.join(dir, 'VERSION'), 'utf-8').trim()).toBe('1.1.0.0');
|
||||
const pkg = JSON.parse(fs.readFileSync(path.join(dir, 'package.json'), 'utf-8'));
|
||||
expect(pkg.version).toBe('1.1.0.0');
|
||||
expect(pkg.scripts).toEqual({ t: 'y' }); // untouched
|
||||
});
|
||||
|
||||
test('rejects a malformed version with exit 2', () => {
|
||||
let code = 0;
|
||||
try { execFileSync('bun', [BIN, 'write', '--version', '1.2.3'], { cwd: dir, stdio: 'pipe' }); }
|
||||
catch (e: any) { code = e.status; }
|
||||
expect(code).toBe(2);
|
||||
});
|
||||
|
||||
test('VERSION-only repo (no package.json) writes just VERSION', () => {
|
||||
const d2 = fs.mkdtempSync(path.join(os.tmpdir(), 'vbump-noPkg-'));
|
||||
fs.writeFileSync(path.join(d2, 'VERSION'), '0.1.0.0\n');
|
||||
const out = execFileSync('bun', [BIN, 'write', '--version', '0.2.0.0'], { cwd: d2 }).toString();
|
||||
expect(JSON.parse(out)).toEqual({ wrote: '0.2.0.0', packageJson: false });
|
||||
expect(fs.readFileSync(path.join(d2, 'VERSION'), 'utf-8').trim()).toBe('0.2.0.0');
|
||||
fs.rmSync(d2, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
|
||||
describe('repair (DRIFT_STALE_PKG)', () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'vbump-repair-'));
|
||||
afterAll(() => { try { fs.rmSync(dir, { recursive: true, force: true }); } catch { /* noop */ } });
|
||||
|
||||
test('syncs package.json.version up to VERSION, no re-bump', () => {
|
||||
fs.writeFileSync(path.join(dir, 'VERSION'), '2.0.0.0\n');
|
||||
fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ name: 'x', version: '1.9.0.0' }, null, 2) + '\n');
|
||||
const out = execFileSync('bun', [BIN, 'repair'], { cwd: dir }).toString();
|
||||
expect(JSON.parse(out)).toEqual({ repaired: '2.0.0.0' });
|
||||
expect(JSON.parse(fs.readFileSync(path.join(dir, 'package.json'), 'utf-8')).version).toBe('2.0.0.0');
|
||||
expect(fs.readFileSync(path.join(dir, 'VERSION'), 'utf-8').trim()).toBe('2.0.0.0'); // unchanged
|
||||
});
|
||||
|
||||
test('refuses to propagate an invalid VERSION (exit 2)', () => {
|
||||
fs.writeFileSync(path.join(dir, 'VERSION'), 'not-a-version\n');
|
||||
let code = 0;
|
||||
try { execFileSync('bun', [BIN, 'repair'], { cwd: dir, stdio: 'pipe' }); }
|
||||
catch (e: any) { code = e.status; }
|
||||
expect(code).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('classify (idempotency over a real git base)', () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'vbump-classify-'));
|
||||
afterAll(() => { try { fs.rmSync(dir, { recursive: true, force: true }); } catch { /* noop */ } });
|
||||
|
||||
// Build a tiny repo with an "origin/main" carrying VERSION=1.0.0.0.
|
||||
const git = (...a: string[]) => execFileSync('git', a, { cwd: dir, stdio: 'pipe' });
|
||||
fs.writeFileSync(path.join(dir, 'VERSION'), '1.0.0.0\n');
|
||||
fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ name: 'x', version: '1.0.0.0' }, null, 2) + '\n');
|
||||
git('init', '-q', '-b', 'main');
|
||||
git('config', 'user.email', 't@t'); git('config', 'user.name', 't');
|
||||
git('add', '-A'); git('commit', '-q', '-m', 'base');
|
||||
// Fake an "origin/main" remote-tracking ref pointing at this commit.
|
||||
const head = execFileSync('git', ['rev-parse', 'HEAD'], { cwd: dir }).toString().trim();
|
||||
fs.mkdirSync(path.join(dir, '.git', 'refs', 'remotes', 'origin'), { recursive: true });
|
||||
fs.writeFileSync(path.join(dir, '.git', 'refs', 'remotes', 'origin', 'main'), head + '\n');
|
||||
|
||||
test('reports FRESH before any bump', () => {
|
||||
const out = execFileSync('bun', [BIN, 'classify', '--base', 'main'], { cwd: dir }).toString();
|
||||
expect(JSON.parse(out).state).toBe('FRESH');
|
||||
});
|
||||
|
||||
test('reports ALREADY_BUMPED after VERSION+pkg move together', () => {
|
||||
fs.writeFileSync(path.join(dir, 'VERSION'), '1.1.0.0\n');
|
||||
fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ name: 'x', version: '1.1.0.0' }, null, 2) + '\n');
|
||||
const out = execFileSync('bun', [BIN, 'classify', '--base', 'main'], { cwd: dir }).toString();
|
||||
const parsed = JSON.parse(out);
|
||||
expect(parsed.state).toBe('ALREADY_BUMPED');
|
||||
expect(parsed.baseVersion).toBe('1.0.0.0');
|
||||
expect(parsed.currentVersion).toBe('1.1.0.0');
|
||||
});
|
||||
});
|
||||
@@ -33,6 +33,22 @@ export interface ParityInvariant {
|
||||
maxSizeRatio?: number;
|
||||
/** Minimum byte size (catches over-stripping cliffs). */
|
||||
minBytes?: number;
|
||||
/**
|
||||
* Carved skill (v2 plan T9): the skill is a skeleton SKILL.md plus on-demand
|
||||
* sections/*.md. When true:
|
||||
* - mustContain / mustHaveHeadings run against skeleton + ALL sections unioned,
|
||||
* so a phrase that moved into a section still counts (content preserved, just
|
||||
* relocated — that's the whole point of the carve).
|
||||
* - minBytes / maxSizeRatio run against the UNION bytes, not the skeleton alone
|
||||
* (total behavior must not shrink; the win is what's no longer always-loaded,
|
||||
* which the union size deliberately does NOT measure — maxSkeletonBytes does).
|
||||
* - maxSkeletonBytes asserts the always-loaded skeleton actually shrank.
|
||||
* Without this, lowering minBytes to fit a 65KB skeleton would make the size
|
||||
* floor toothless (Codex outside-voice #12).
|
||||
*/
|
||||
sectioned?: boolean;
|
||||
/** Max bytes for the always-loaded skeleton SKILL.md (carved skills only). */
|
||||
maxSkeletonBytes?: number;
|
||||
}
|
||||
|
||||
export interface ParityCheckResult {
|
||||
@@ -41,6 +57,35 @@ export interface ParityCheckResult {
|
||||
failures: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a skill's check text + sizes. For a carved skill, union the skeleton with
|
||||
* every sections/*.md so relocated content still counts and the union size
|
||||
* measures total preserved behavior; skeletonBytes is reported separately so the
|
||||
* always-loaded shrink can be asserted. For a monolith, text == skeleton.
|
||||
*/
|
||||
export function readSkillForParity(
|
||||
repoRoot: string,
|
||||
skill: string,
|
||||
sectioned: boolean,
|
||||
): { text: string; unionBytes: number; skeletonBytes: number } {
|
||||
const skeleton = fs.readFileSync(path.join(repoRoot, skill, 'SKILL.md'), 'utf-8');
|
||||
const skeletonBytes = Buffer.byteLength(skeleton, 'utf-8');
|
||||
if (!sectioned) return { text: skeleton, unionBytes: skeletonBytes, skeletonBytes };
|
||||
|
||||
let text = skeleton;
|
||||
let unionBytes = skeletonBytes;
|
||||
const sectionsDir = path.join(repoRoot, skill, 'sections');
|
||||
if (fs.existsSync(sectionsDir)) {
|
||||
for (const f of fs.readdirSync(sectionsDir).sort()) {
|
||||
if (!f.endsWith('.md')) continue;
|
||||
const sec = fs.readFileSync(path.join(sectionsDir, f), 'utf-8');
|
||||
text += '\n' + sec;
|
||||
unionBytes += Buffer.byteLength(sec, 'utf-8');
|
||||
}
|
||||
}
|
||||
return { text, unionBytes, skeletonBytes };
|
||||
}
|
||||
|
||||
export function checkSkillParity(
|
||||
invariant: ParityInvariant,
|
||||
current: SkillBaselineEntry,
|
||||
@@ -48,38 +93,54 @@ export function checkSkillParity(
|
||||
repoRoot: string,
|
||||
): ParityCheckResult {
|
||||
const failures: string[] = [];
|
||||
const needText = !!(invariant.mustContain?.length || invariant.mustHaveHeadings?.length);
|
||||
|
||||
// SIZE checks
|
||||
// Resolve the text + size to check against. Carved skills union skeleton +
|
||||
// sections; monoliths use the skeleton alone. Read on demand so size-only
|
||||
// invariants don't pay for a file read they don't need (monolith path).
|
||||
let checkText: string | null = null;
|
||||
let checkBytes = current.skillMdBytes;
|
||||
if (invariant.sectioned) {
|
||||
try {
|
||||
const r = readSkillForParity(repoRoot, invariant.skill, true);
|
||||
checkText = r.text;
|
||||
checkBytes = r.unionBytes;
|
||||
if (invariant.maxSkeletonBytes !== undefined && r.skeletonBytes > invariant.maxSkeletonBytes) {
|
||||
failures.push(`skeleton ${r.skeletonBytes} > maxSkeletonBytes ${invariant.maxSkeletonBytes}`);
|
||||
}
|
||||
} catch (err) {
|
||||
failures.push(`cannot read carved skill ${invariant.skill}: ${(err as Error).message}`);
|
||||
}
|
||||
} else if (needText) {
|
||||
try {
|
||||
checkText = fs.readFileSync(path.join(repoRoot, invariant.skill, 'SKILL.md'), 'utf-8');
|
||||
} catch (err) {
|
||||
failures.push(`cannot read ${path.join(repoRoot, invariant.skill, 'SKILL.md')}: ${(err as Error).message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// SIZE checks (union bytes for carved skills, skeleton bytes for monoliths)
|
||||
if (invariant.maxSizeRatio !== undefined && baseline) {
|
||||
const ratio = current.skillMdBytes / baseline.skillMdBytes;
|
||||
const ratio = checkBytes / baseline.skillMdBytes;
|
||||
if (ratio > invariant.maxSizeRatio) {
|
||||
failures.push(`size ratio ${ratio.toFixed(3)} > maxSizeRatio ${invariant.maxSizeRatio}`);
|
||||
}
|
||||
}
|
||||
if (invariant.minBytes !== undefined && current.skillMdBytes < invariant.minBytes) {
|
||||
failures.push(`size ${current.skillMdBytes} < minBytes ${invariant.minBytes}`);
|
||||
if (invariant.minBytes !== undefined && checkBytes < invariant.minBytes) {
|
||||
failures.push(`size ${checkBytes} < minBytes ${invariant.minBytes}`);
|
||||
}
|
||||
|
||||
// CONTENT checks (read live file for fresh content)
|
||||
if (invariant.mustContain?.length || invariant.mustHaveHeadings?.length) {
|
||||
const skillMdPath = path.join(repoRoot, invariant.skill, 'SKILL.md');
|
||||
let content: string | null = null;
|
||||
try {
|
||||
content = fs.readFileSync(skillMdPath, 'utf-8');
|
||||
} catch (err) {
|
||||
failures.push(`cannot read ${skillMdPath}: ${(err as Error).message}`);
|
||||
}
|
||||
if (content) {
|
||||
const lower = content.toLowerCase();
|
||||
for (const phrase of invariant.mustContain ?? []) {
|
||||
if (!lower.includes(phrase.toLowerCase())) {
|
||||
failures.push(`missing required phrase: "${phrase}"`);
|
||||
}
|
||||
// CONTENT checks
|
||||
if (needText && checkText !== null) {
|
||||
const lower = checkText.toLowerCase();
|
||||
for (const phrase of invariant.mustContain ?? []) {
|
||||
if (!lower.includes(phrase.toLowerCase())) {
|
||||
failures.push(`missing required phrase: "${phrase}"`);
|
||||
}
|
||||
for (const heading of invariant.mustHaveHeadings ?? []) {
|
||||
if (!content.includes(heading)) {
|
||||
failures.push(`missing required heading: "${heading}"`);
|
||||
}
|
||||
}
|
||||
for (const heading of invariant.mustHaveHeadings ?? []) {
|
||||
if (!checkText.includes(heading)) {
|
||||
failures.push(`missing required heading: "${heading}"`);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -146,7 +207,13 @@ export const PARITY_INVARIANTS: ParityInvariant[] = [
|
||||
minBytes: 30_000,
|
||||
},
|
||||
{
|
||||
// Carved (v2 plan T9): skeleton SKILL.md + sections/*.md. Content checks run
|
||||
// against the union (relocated phrases still count); size floors run against
|
||||
// the union (total behavior preserved); maxSkeletonBytes asserts the
|
||||
// always-loaded skeleton actually shrank from the ~167KB monolith.
|
||||
skill: 'ship',
|
||||
sectioned: true,
|
||||
maxSkeletonBytes: 90_000,
|
||||
mustContain: [
|
||||
'VERSION',
|
||||
'CHANGELOG',
|
||||
@@ -156,7 +223,7 @@ export const PARITY_INVARIANTS: ParityInvariant[] = [
|
||||
],
|
||||
mustHaveHeadings: ['## Preamble', '## When to invoke'],
|
||||
maxSizeRatio: 1.05,
|
||||
minBytes: 80_000,
|
||||
minBytes: 120_000,
|
||||
},
|
||||
{
|
||||
skill: 'plan-ceo-review',
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
/**
|
||||
* requiredReads enforcement (v2 plan T9, mitigation layer 5 — the only CI-failing
|
||||
* layer against silent section-skip).
|
||||
*
|
||||
* Given a /ship run's tool calls and the set of section files the run's SITUATION
|
||||
* required, assert the agent actually Read each one. The required set comes from
|
||||
* the TEST FIXTURE (which situation it set up), NOT from the manifest — the
|
||||
* manifest is passive (CM2). This keeps "when is a section required" in exactly
|
||||
* one machine-checkable place: the eval fixtures.
|
||||
*
|
||||
* Builds on extractSectionReads from transcript-section-logger so section-path
|
||||
* matching (the `/sections/<file>.md` segment, host-layout agnostic) lives in one
|
||||
* place.
|
||||
*/
|
||||
|
||||
import { extractSectionReads, type TranscriptResultLike } from './transcript-section-logger';
|
||||
|
||||
export interface RequiredReadsResult {
|
||||
required: string[];
|
||||
read: string[];
|
||||
missing: string[];
|
||||
ok: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param result the skill run (anything with toolCalls)
|
||||
* @param requiredFiles section basenames the situation required, e.g.
|
||||
* ['version-bump.md','changelog.md'] (or with a sections/
|
||||
* prefix — normalized to basename here)
|
||||
*/
|
||||
export function assertRequiredReads(
|
||||
result: TranscriptResultLike,
|
||||
requiredFiles: string[],
|
||||
): RequiredReadsResult {
|
||||
const read = extractSectionReads(result);
|
||||
const readSet = new Set(read);
|
||||
const required = requiredFiles.map(f => f.replace(/^.*\//, '')); // tolerate sections/<f>
|
||||
const missing = required.filter(f => !readSet.has(f));
|
||||
return { required, read, missing, ok: missing.length === 0 };
|
||||
}
|
||||
@@ -120,7 +120,8 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
|
||||
'plan-ceo-mode-routing': ['plan-ceo-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'test/helpers/claude-pty-runner.ts'],
|
||||
'plan-design-with-ui-scope': ['plan-design-review/**', 'test/fixtures/plans/ui-heavy-feature.md', 'test/helpers/claude-pty-runner.ts'],
|
||||
'budget-regression-pty': ['test/helpers/eval-store.ts', 'test/skill-budget-regression.test.ts'],
|
||||
'ship-idempotency-pty': ['ship/**', 'bin/gstack-next-version', 'lib/worktree.ts', 'test/helpers/claude-pty-runner.ts'],
|
||||
'ship-idempotency-pty': ['ship/**', 'bin/gstack-next-version', 'bin/gstack-version-bump', 'scripts/resolvers/sections.ts', 'lib/worktree.ts', 'test/helpers/claude-pty-runner.ts'],
|
||||
'ship-section-loading': ['ship/**', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/required-reads.ts', 'test/helpers/transcript-section-logger.ts', 'test/helpers/claude-pty-runner.ts'],
|
||||
'autoplan-chain-pty': ['autoplan/**', 'plan-ceo-review/**', 'plan-design-review/**', 'plan-eng-review/**', 'plan-devex-review/**', 'test/fixtures/plans/ui-heavy-feature.md', 'test/helpers/claude-pty-runner.ts'],
|
||||
'e2e-harness-audit': ['plan-ceo-review/**', 'plan-eng-review/**', 'plan-design-review/**', 'plan-devex-review/**', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/agent-sdk-runner.ts', 'test/helpers/claude-pty-runner.ts'],
|
||||
|
||||
@@ -508,6 +509,7 @@ export const E2E_TIERS: Record<string, 'gate' | 'periodic'> = {
|
||||
'plan-design-with-ui-scope': 'gate', // ~$0.80/run
|
||||
'budget-regression-pty': 'gate', // free, library-only assertion
|
||||
'ship-idempotency-pty': 'periodic', // ~$3/run, real /ship in plan mode
|
||||
'ship-section-loading': 'periodic', // ~$3/run, real /ship; asserts section reads
|
||||
'autoplan-chain-pty': 'periodic', // ~$8/run, all 3 phases sequential
|
||||
|
||||
// Per-finding count + review-report-at-bottom — periodic because each
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
/**
|
||||
* Transcript section logger (v2 plan T10).
|
||||
*
|
||||
* Two jobs, both pure analysis over a SkillTestResult / NDJSON transcript:
|
||||
*
|
||||
* 1. extractSectionReads() — which `sections/*.md` files a run actually Read.
|
||||
* Used by the sectioned world (post-carve) to verify the agent opened the
|
||||
* chapters its situation required.
|
||||
*
|
||||
* 2. extractShipActions() — an observable ACTION fingerprint of a /ship run
|
||||
* (ran tests, bumped VERSION, wrote CHANGELOG, created PR, ...). This works
|
||||
* on BOTH the monolith and the sectioned skill, which is the whole point:
|
||||
* capture a baseline on the current monolith ship FIRST, then assert the
|
||||
* sectioned ship still performs the same actions. A section-read check alone
|
||||
* can't catch "agent read the chapter but skipped the step"; the action
|
||||
* fingerprint can.
|
||||
*
|
||||
* Why baseline-first (Codex outside-voice critique on the T9 plan): a logger
|
||||
* shipped in the same PR as the carve is post-failure telemetry unless it has a
|
||||
* pre-carve reference. captureShipBaseline() records the monolith's action
|
||||
* fingerprint so compareShipActions() can flag a regression introduced by the
|
||||
* carve.
|
||||
*
|
||||
* Pure functions, no I/O except the explicit read/write baseline helpers. The
|
||||
* unit tests drive these with synthetic transcripts — no paid run needed to
|
||||
* validate the logic.
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
|
||||
/** Minimal shape we need from SkillTestResult — kept structural so callers can
|
||||
* pass a full SkillTestResult or a hand-built fixture in unit tests. */
|
||||
export interface ToolCallLike {
|
||||
tool: string;
|
||||
input: unknown;
|
||||
output?: string;
|
||||
}
|
||||
export interface TranscriptResultLike {
|
||||
toolCalls: ToolCallLike[];
|
||||
output?: string;
|
||||
}
|
||||
|
||||
/** Pull the file_path off a tool-call input, tolerating unknown shapes. */
|
||||
function readFilePath(input: unknown): string | null {
|
||||
if (input && typeof input === 'object') {
|
||||
const fp = (input as Record<string, unknown>).file_path;
|
||||
if (typeof fp === 'string') return fp;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Pull the command string off a Bash tool-call input. */
|
||||
function bashCommand(input: unknown): string | null {
|
||||
if (input && typeof input === 'object') {
|
||||
const cmd = (input as Record<string, unknown>).command;
|
||||
if (typeof cmd === 'string') return cmd;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every `sections/<name>.md` file the run Read, normalized to the section
|
||||
* basename (e.g. "version-bump.md"). Deduped, in first-Read order. Matching is
|
||||
* on the path segment `/sections/<file>.md` so it works regardless of whether
|
||||
* the host resolved a relative, absolute, or prefixed install path.
|
||||
*/
|
||||
export function extractSectionReads(result: TranscriptResultLike): string[] {
|
||||
const seen = new Set<string>();
|
||||
const ordered: string[] = [];
|
||||
for (const call of result.toolCalls) {
|
||||
if (call.tool !== 'Read') continue;
|
||||
const fp = readFilePath(call.input);
|
||||
if (!fp) continue;
|
||||
const m = fp.match(/(?:^|\/)sections\/([A-Za-z0-9._-]+\.md)$/);
|
||||
if (!m) continue;
|
||||
const name = m[1];
|
||||
if (!seen.has(name)) {
|
||||
seen.add(name);
|
||||
ordered.push(name);
|
||||
}
|
||||
}
|
||||
return ordered;
|
||||
}
|
||||
|
||||
/**
|
||||
* The canonical /ship action vocabulary. Each action is detected from the Bash
|
||||
* commands the agent ran (plus a couple of Write/Edit signals). Order is the
|
||||
* rough ship sequence; detection is order-independent.
|
||||
*
|
||||
* Keep this list aligned with the ship skeleton's numbered steps. The
|
||||
* section-loading eval asserts the sectioned ship still triggers the same
|
||||
* actions a monolith run did for the same fixture situation.
|
||||
*/
|
||||
export const SHIP_ACTIONS = [
|
||||
'merged_base', // git merge <base>
|
||||
'ran_tests', // bun test / npm test / the project test cmd
|
||||
'bumped_version', // wrote VERSION / package.json version / ran gstack-version-bump
|
||||
'wrote_changelog', // edited CHANGELOG.md
|
||||
'committed', // git commit
|
||||
'pushed', // git push
|
||||
'opened_pr', // gh pr create / glab mr create
|
||||
] as const;
|
||||
export type ShipAction = (typeof SHIP_ACTIONS)[number];
|
||||
|
||||
const BASH_ACTION_PATTERNS: Array<{ action: ShipAction; re: RegExp }> = [
|
||||
{ action: 'merged_base', re: /\bgit\s+merge\b/ },
|
||||
{ action: 'ran_tests', re: /\b(bun\s+test|npm\s+(run\s+)?test|yarn\s+test|pytest|go\s+test|cargo\s+test|rspec)\b/ },
|
||||
{ action: 'bumped_version', re: /gstack-version-bump\b|gstack-next-version\b|>\s*VERSION\b|npm\s+version\b/ },
|
||||
{ action: 'wrote_changelog', re: /CHANGELOG\.md/ },
|
||||
{ action: 'committed', re: /\bgit\s+commit\b/ },
|
||||
{ action: 'pushed', re: /\bgit\s+push\b/ },
|
||||
{ action: 'opened_pr', re: /\bgh\s+pr\s+create\b|\bglab\s+mr\s+create\b/ },
|
||||
];
|
||||
|
||||
/**
|
||||
* The observable action fingerprint of a ship run. Works on monolith AND
|
||||
* sectioned skills because it reads what the agent DID (Bash + file writes),
|
||||
* not which prose it loaded.
|
||||
*/
|
||||
export function extractShipActions(result: TranscriptResultLike): ShipAction[] {
|
||||
const found = new Set<ShipAction>();
|
||||
for (const call of result.toolCalls) {
|
||||
if (call.tool === 'Bash') {
|
||||
const cmd = bashCommand(call.input);
|
||||
if (!cmd) continue;
|
||||
for (const { action, re } of BASH_ACTION_PATTERNS) {
|
||||
if (re.test(cmd)) found.add(action);
|
||||
}
|
||||
} else if (call.tool === 'Write' || call.tool === 'Edit') {
|
||||
const fp = readFilePath(call.input);
|
||||
if (fp && /CHANGELOG\.md$/.test(fp)) found.add('wrote_changelog');
|
||||
if (fp && /(?:^|\/)VERSION$/.test(fp)) found.add('bumped_version');
|
||||
}
|
||||
}
|
||||
// Preserve canonical order.
|
||||
return SHIP_ACTIONS.filter(a => found.has(a));
|
||||
}
|
||||
|
||||
export interface ShipBaseline {
|
||||
tag: string;
|
||||
/** Fixture/situation id this baseline was captured for. */
|
||||
situation: string;
|
||||
/** Action fingerprint observed on the monolith ship. */
|
||||
actions: ShipAction[];
|
||||
/** Section reads observed (empty on the monolith — present after carve). */
|
||||
sectionReads: string[];
|
||||
capturedAt: string;
|
||||
}
|
||||
|
||||
const DEFAULT_BASELINE_DIR = path.join(os.homedir(), '.gstack-dev', 'ship-baselines');
|
||||
|
||||
/** Where a baseline for a given situation lives. */
|
||||
export function baselinePath(situation: string, dir = DEFAULT_BASELINE_DIR): string {
|
||||
return path.join(dir, `${situation}.json`);
|
||||
}
|
||||
|
||||
/** Persist a ship baseline (used once on the monolith, before the carve). */
|
||||
export function writeShipBaseline(baseline: ShipBaseline, dir = DEFAULT_BASELINE_DIR): string {
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
const p = baselinePath(baseline.situation, dir);
|
||||
fs.writeFileSync(p, JSON.stringify(baseline, null, 2) + '\n');
|
||||
return p;
|
||||
}
|
||||
|
||||
/** Read a previously-captured baseline, or null if none exists yet. */
|
||||
export function readShipBaseline(situation: string, dir = DEFAULT_BASELINE_DIR): ShipBaseline | null {
|
||||
try {
|
||||
return JSON.parse(fs.readFileSync(baselinePath(situation, dir), 'utf-8')) as ShipBaseline;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export interface ShipActionDiff {
|
||||
/** Actions the baseline performed that the current run did NOT (the regression set). */
|
||||
missing: ShipAction[];
|
||||
/** Actions the current run performed that the baseline did not (usually fine). */
|
||||
added: ShipAction[];
|
||||
/** True when no baseline action was dropped. */
|
||||
ok: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compare a current sectioned-ship run against the monolith baseline. A dropped
|
||||
* action (in baseline, not in current) is the carve regression we care about:
|
||||
* the sectioned ship stopped doing something the monolith did.
|
||||
*/
|
||||
export function compareShipActions(baseline: ShipBaseline, current: ShipAction[]): ShipActionDiff {
|
||||
const cur = new Set(current);
|
||||
const base = new Set(baseline.actions);
|
||||
const missing = baseline.actions.filter(a => !cur.has(a));
|
||||
const added = current.filter(a => !base.has(a));
|
||||
return { missing, added, ok: missing.length === 0 };
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
/**
|
||||
* Unit coverage for the sectioned-parity capability (v2 plan T9, guards the
|
||||
* carve). Proves that a carved skill's relocated content still counts (union of
|
||||
* skeleton + sections), the always-loaded skeleton shrink is asserted
|
||||
* separately (maxSkeletonBytes), and size floors run against the union so they
|
||||
* stay meaningful (Codex outside-voice #12). Synthetic fixture — no ship carve
|
||||
* needed to validate the logic.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, afterAll } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import { checkSkillParity, readSkillForParity, type ParityInvariant } from './helpers/parity-harness';
|
||||
import type { SkillBaselineEntry } from './helpers/capture-parity-baseline';
|
||||
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'parity-sectioned-'));
|
||||
afterAll(() => { try { fs.rmSync(root, { recursive: true, force: true }); } catch { /* noop */ } });
|
||||
|
||||
// Carved "ship": a small skeleton + two sections holding the relocated prose.
|
||||
fs.mkdirSync(path.join(root, 'ship', 'sections'), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, 'ship', 'SKILL.md'),
|
||||
'## Preamble\nskeleton body, decision tree, VERSION bump step calls the CLI.\n## When to invoke\n');
|
||||
fs.writeFileSync(path.join(root, 'ship', 'sections', 'changelog.md'), '# Changelog\nWrite the CHANGELOG entry here.\n');
|
||||
fs.writeFileSync(path.join(root, 'ship', 'sections', 'review-army.md'), '# Review\nDispatch the pre-landing review army.\n');
|
||||
|
||||
// A monolith control skill.
|
||||
fs.mkdirSync(path.join(root, 'mono'), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, 'mono', 'SKILL.md'), '## Preamble\nVERSION CHANGELOG review all inline here.\n');
|
||||
|
||||
const skeletonBytes = Buffer.byteLength(fs.readFileSync(path.join(root, 'ship', 'SKILL.md'), 'utf-8'), 'utf-8');
|
||||
const unionBytes = readSkillForParity(root, 'ship', true).unionBytes;
|
||||
const baseline: SkillBaselineEntry = { skillMdBytes: unionBytes } as SkillBaselineEntry;
|
||||
|
||||
describe('readSkillForParity', () => {
|
||||
test('unions skeleton + sections for carved skills', () => {
|
||||
const r = readSkillForParity(root, 'ship', true);
|
||||
expect(r.text).toContain('CHANGELOG'); // from changelog.md
|
||||
expect(r.text).toContain('review army'); // from review-army.md
|
||||
expect(r.skeletonBytes).toBe(skeletonBytes);
|
||||
expect(r.unionBytes).toBeGreaterThan(r.skeletonBytes);
|
||||
});
|
||||
test('monolith text == skeleton, union == skeleton', () => {
|
||||
const r = readSkillForParity(root, 'mono', false);
|
||||
expect(r.unionBytes).toBe(r.skeletonBytes);
|
||||
});
|
||||
});
|
||||
|
||||
describe('checkSkillParity (sectioned)', () => {
|
||||
test('finds phrases that moved into sections (union content check)', () => {
|
||||
const inv: ParityInvariant = {
|
||||
skill: 'ship', sectioned: true,
|
||||
mustContain: ['VERSION', 'CHANGELOG', 'review army'],
|
||||
mustHaveHeadings: ['## Preamble', '## When to invoke'],
|
||||
};
|
||||
const res = checkSkillParity(inv, { skillMdBytes: skeletonBytes } as SkillBaselineEntry, baseline, root);
|
||||
expect(res.passed).toBe(true);
|
||||
});
|
||||
|
||||
test('maxSkeletonBytes catches a skeleton that did not shrink', () => {
|
||||
const inv: ParityInvariant = { skill: 'ship', sectioned: true, maxSkeletonBytes: 10 };
|
||||
const res = checkSkillParity(inv, { skillMdBytes: skeletonBytes } as SkillBaselineEntry, baseline, root);
|
||||
expect(res.passed).toBe(false);
|
||||
expect(res.failures.join()).toContain('maxSkeletonBytes');
|
||||
});
|
||||
|
||||
test('minBytes runs against the union, not the skeleton (content preserved)', () => {
|
||||
// A floor between skeletonBytes and unionBytes must PASS for sectioned skills,
|
||||
// because the union (total behavior) is what must not shrink.
|
||||
const floor = Math.floor((skeletonBytes + unionBytes) / 2);
|
||||
const inv: ParityInvariant = { skill: 'ship', sectioned: true, minBytes: floor };
|
||||
const res = checkSkillParity(inv, { skillMdBytes: skeletonBytes } as SkillBaselineEntry, baseline, root);
|
||||
expect(res.passed).toBe(true);
|
||||
});
|
||||
|
||||
test('flags a phrase that truly went missing', () => {
|
||||
const inv: ParityInvariant = { skill: 'ship', sectioned: true, mustContain: ['this-phrase-is-not-anywhere'] };
|
||||
const res = checkSkillParity(inv, { skillMdBytes: skeletonBytes } as SkillBaselineEntry, baseline, root);
|
||||
expect(res.passed).toBe(false);
|
||||
expect(res.failures.join()).toContain('missing required phrase');
|
||||
});
|
||||
|
||||
test('maxSizeRatio uses union bytes vs baseline (carve preserves ~total size)', () => {
|
||||
const inv: ParityInvariant = { skill: 'ship', sectioned: true, maxSizeRatio: 1.05 };
|
||||
const res = checkSkillParity(inv, { skillMdBytes: skeletonBytes } as SkillBaselineEntry, baseline, root);
|
||||
expect(res.passed).toBe(true); // union == baseline here → ratio 1.0
|
||||
});
|
||||
});
|
||||
@@ -83,9 +83,22 @@ describe("#1539 generated SKILL.md files — gate propagated to all consumers",
|
||||
"ship/SKILL.md",
|
||||
];
|
||||
|
||||
// ship's confidence-calibration gate moved into sections/review-army.md (T9 carve);
|
||||
// read the skeleton+sections union so the gate is still found.
|
||||
const readUnion = (rel: string): string => {
|
||||
let body = fs.readFileSync(path.join(ROOT, rel), "utf-8");
|
||||
const secDir = path.join(ROOT, path.dirname(rel), "sections");
|
||||
if (fs.existsSync(secDir)) {
|
||||
for (const f of fs.readdirSync(secDir).sort()) {
|
||||
if (f.endsWith(".md")) body += "\n" + fs.readFileSync(path.join(secDir, f), "utf-8");
|
||||
}
|
||||
}
|
||||
return body;
|
||||
};
|
||||
|
||||
for (const rel of consumers) {
|
||||
test(`${rel} carries the Pre-emit verification gate`, () => {
|
||||
const body = fs.readFileSync(path.join(ROOT, rel), "utf-8");
|
||||
const body = readUnion(rel);
|
||||
expect(body).toMatch(/Pre-emit verification gate/);
|
||||
expect(body).toMatch(/Quote the specific code line/);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
/**
|
||||
* Unit tests for assertRequiredReads (v2 plan T9 mitigation layer 5). Pure logic
|
||||
* over synthetic tool-call transcripts — the section-loading E2E (paid) drives
|
||||
* this against real /ship runs.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { assertRequiredReads } from './helpers/required-reads';
|
||||
import type { ToolCallLike } from './helpers/transcript-section-logger';
|
||||
|
||||
const read = (fp: string): ToolCallLike => ({ tool: 'Read', input: { file_path: fp }, output: '' });
|
||||
|
||||
describe('assertRequiredReads', () => {
|
||||
test('passes when every required section was Read', () => {
|
||||
const result = {
|
||||
toolCalls: [
|
||||
read('/Users/x/.claude/skills/gstack/ship/sections/version-bump.md'),
|
||||
read('ship/sections/changelog.md'),
|
||||
],
|
||||
};
|
||||
const r = assertRequiredReads(result, ['version-bump.md', 'changelog.md']);
|
||||
expect(r.ok).toBe(true);
|
||||
expect(r.missing).toEqual([]);
|
||||
});
|
||||
|
||||
test('flags a required section the agent never opened', () => {
|
||||
const result = { toolCalls: [read('ship/sections/changelog.md')] };
|
||||
const r = assertRequiredReads(result, ['version-bump.md', 'changelog.md']);
|
||||
expect(r.ok).toBe(false);
|
||||
expect(r.missing).toEqual(['version-bump.md']);
|
||||
});
|
||||
|
||||
test('tolerates a sections/ prefix in the required list', () => {
|
||||
const result = { toolCalls: [read('/abs/gstack/ship/sections/review-army.md')] };
|
||||
expect(assertRequiredReads(result, ['sections/review-army.md']).ok).toBe(true);
|
||||
});
|
||||
|
||||
test('empty required set always passes', () => {
|
||||
expect(assertRequiredReads({ toolCalls: [] }, []).ok).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,77 @@
|
||||
/**
|
||||
* Section manifest ↔ filesystem consistency (v2 plan T9 / Phase C orphan check).
|
||||
*
|
||||
* Implements the 3-tier orphan classification from v2_PLAN.md:
|
||||
* - generated orphan (sections/X.md with no sections/X.md.tmpl) → FAIL
|
||||
* - hand-edited generated file (X.md missing the AUTO-GENERATED header) → FAIL
|
||||
* - manifest orphan (sections/X.md.tmpl not listed in manifest) → WARN (v2.0)
|
||||
*
|
||||
* Also pins the PASSIVE-manifest contract (CM2 / v2_PLAN.md:663): manifest entries
|
||||
* carry only id/file/title/trigger — no machine predicate (applies_when/required_for).
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
const SHIP_SECTIONS = path.join(ROOT, 'ship', 'sections');
|
||||
const manifest = JSON.parse(fs.readFileSync(path.join(SHIP_SECTIONS, 'manifest.json'), 'utf-8'));
|
||||
|
||||
const sectionTmpls = fs.readdirSync(SHIP_SECTIONS).filter(f => f.endsWith('.md.tmpl'));
|
||||
const sectionMds = fs.readdirSync(SHIP_SECTIONS).filter(f => f.endsWith('.md') && !f.endsWith('.md.tmpl'));
|
||||
|
||||
describe('section manifest ↔ filesystem consistency', () => {
|
||||
test('manifest parses with skill + sections array', () => {
|
||||
expect(manifest.skill).toBe('ship');
|
||||
expect(Array.isArray(manifest.sections)).toBe(true);
|
||||
expect(manifest.sections.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
test('every manifest entry has a .md.tmpl source AND a generated .md', () => {
|
||||
for (const s of manifest.sections) {
|
||||
expect(fs.existsSync(path.join(SHIP_SECTIONS, `${s.file}.tmpl`))).toBe(true);
|
||||
expect(fs.existsSync(path.join(SHIP_SECTIONS, s.file))).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
test('manifest is PASSIVE — no applies_when / required_for predicate (CM2)', () => {
|
||||
for (const s of manifest.sections) {
|
||||
expect(s).not.toHaveProperty('applies_when');
|
||||
expect(s).not.toHaveProperty('required_for');
|
||||
// The allowed passive shape:
|
||||
expect(typeof s.id).toBe('string');
|
||||
expect(typeof s.file).toBe('string');
|
||||
expect(typeof s.title).toBe('string');
|
||||
expect(typeof s.trigger).toBe('string');
|
||||
}
|
||||
});
|
||||
|
||||
test('no generated orphan: every sections/X.md has a sections/X.md.tmpl → FAIL', () => {
|
||||
const orphans = sectionMds.filter(md => !sectionTmpls.includes(`${md}.tmpl`));
|
||||
expect(orphans).toEqual([]);
|
||||
});
|
||||
|
||||
test('no hand-edited generated file: every sections/X.md has the AUTO-GENERATED header → FAIL', () => {
|
||||
for (const md of sectionMds) {
|
||||
const head = fs.readFileSync(path.join(SHIP_SECTIONS, md), 'utf-8').slice(0, 120);
|
||||
expect(head).toContain('AUTO-GENERATED');
|
||||
}
|
||||
});
|
||||
|
||||
test('manifest orphan check (WARN in v2.0): every .md.tmpl is listed', () => {
|
||||
const listed = new Set(manifest.sections.map((s: { file: string }) => `${s.file}.tmpl`));
|
||||
const unlisted = sectionTmpls.filter(t => !listed.has(t));
|
||||
if (unlisted.length > 0) {
|
||||
// v2_PLAN.md: WARN now, FAIL in v2.1. Surface, don't fail the build yet.
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn(`[section-manifest] manifest orphan(s) (not in manifest.json): ${unlisted.join(', ')}`);
|
||||
}
|
||||
expect(unlisted.length).toBeLessThanOrEqual(unlisted.length); // always passes; WARN only
|
||||
});
|
||||
|
||||
test('section ids are unique', () => {
|
||||
const ids = manifest.sections.map((s: { id: string }) => s.id);
|
||||
expect(new Set(ids).size).toBe(ids.length);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,48 @@
|
||||
/**
|
||||
* Static invariant: the two install targets that cherry-pick SKILL.md (Claude
|
||||
* prefixed dirs + Kiro) must ALSO install the sections/ subdir, or a carved
|
||||
* skill's runtime "Read sections/<name>.md" 404s. codex/factory/opencode link
|
||||
* the whole generated dir, so sections ride along for free there.
|
||||
*
|
||||
* Matches the repo's static-tripwire style (setup-windows-fallback,
|
||||
* cdp-session-cleanup). End-to-end "sections resolve in a temp install" runs in
|
||||
* the group-5/6 functional pass once real ship/sections/ exist.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
|
||||
const SETUP = fs.readFileSync(path.join(import.meta.dir, '..', 'setup'), 'utf-8');
|
||||
|
||||
/** Body of a shell function `name() { ... }` up to the closing line `}`. */
|
||||
function fnBody(src: string, name: string): string {
|
||||
const start = src.indexOf(`${name}() {`);
|
||||
if (start === -1) return '';
|
||||
const end = src.indexOf('\n}', start);
|
||||
return src.slice(start, end === -1 ? undefined : end);
|
||||
}
|
||||
|
||||
describe('setup links sections/ for cherry-pick install targets', () => {
|
||||
test('link_claude_skill_dirs links sections/ via _link_or_copy', () => {
|
||||
const body = fnBody(SETUP, 'link_claude_skill_dirs');
|
||||
expect(body).toContain('sections');
|
||||
// sections install must route through the windows-safe helper, not raw ln.
|
||||
expect(body).toMatch(/_link_or_copy\s+"\$gstack_dir\/\$dir_name\/sections"\s+"\$target\/sections"/);
|
||||
expect(body).toMatch(/if \[ -d "\$gstack_dir\/\$dir_name\/sections" \]/);
|
||||
});
|
||||
|
||||
test('kiro per-skill loop rewrites + copies sections/*', () => {
|
||||
// Kiro builds from the codex output and sed-rewrites paths; sections must get
|
||||
// the same rewrite so they resolve under ~/.kiro, not ~/.codex or ~/.claude.
|
||||
expect(SETUP).toMatch(/if \[ -d "\$skill_dir\/sections" \]/);
|
||||
expect(SETUP).toMatch(/mkdir -p "\$target_dir\/sections"/);
|
||||
expect(SETUP).toContain('$target_dir/sections/$(basename "$section_file")');
|
||||
});
|
||||
|
||||
test('no raw ln introduced (windows-fallback invariant still holds)', () => {
|
||||
// Every new line touching sections uses _link_or_copy or sed redirect, never ln.
|
||||
const sectionLines = SETUP.split('\n').filter(l => l.includes('sections') && /\bln\s+-/.test(l));
|
||||
expect(sectionLines).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -2,10 +2,23 @@ import { describe, test, expect } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
|
||||
const SHIP_SKILL = path.join(__dirname, '..', 'ship', 'SKILL.md');
|
||||
const SHIP_DIR = path.join(__dirname, '..', 'ship');
|
||||
|
||||
// Carved (v2 plan T9): the Plan Completion gate moved into sections/plan-completion.md.
|
||||
// Read the skeleton + sections union so these invariants follow the content.
|
||||
function readShipUnion(): string {
|
||||
let t = fs.readFileSync(path.join(SHIP_DIR, 'SKILL.md'), 'utf8');
|
||||
const secDir = path.join(SHIP_DIR, 'sections');
|
||||
if (fs.existsSync(secDir)) {
|
||||
for (const f of fs.readdirSync(secDir).sort()) {
|
||||
if (f.endsWith('.md')) t += '\n' + fs.readFileSync(path.join(secDir, f), 'utf8');
|
||||
}
|
||||
}
|
||||
return t;
|
||||
}
|
||||
|
||||
describe('ship/SKILL.md — Plan Completion gate invariants (VAS-449 remediation)', () => {
|
||||
const skill = fs.readFileSync(SHIP_SKILL, 'utf8');
|
||||
const skill = readShipUnion();
|
||||
|
||||
test('Path concreteness rule: filesystem-pathed items must be test -f checked', () => {
|
||||
expect(skill).toContain('**Path concreteness rule.**');
|
||||
|
||||
@@ -9,7 +9,20 @@ import * as path from "path";
|
||||
import { scan } from "../lib/redact-engine";
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, "..");
|
||||
const TMPL = fs.readFileSync(path.join(ROOT, "ship", "SKILL.md.tmpl"), "utf-8");
|
||||
// Carved (v2 plan T9): ship is a skeleton template + sections/*.md.tmpl. The
|
||||
// PR-body redaction wiring moved into sections/pr-body.md.tmpl, so assert against
|
||||
// the union of the skeleton template and its section templates.
|
||||
function readShipTemplateUnion(): string {
|
||||
let t = fs.readFileSync(path.join(ROOT, "ship", "SKILL.md.tmpl"), "utf-8");
|
||||
const secDir = path.join(ROOT, "ship", "sections");
|
||||
if (fs.existsSync(secDir)) {
|
||||
for (const f of fs.readdirSync(secDir).sort()) {
|
||||
if (f.endsWith(".md.tmpl")) t += "\n" + fs.readFileSync(path.join(secDir, f), "utf-8");
|
||||
}
|
||||
}
|
||||
return t;
|
||||
}
|
||||
const TMPL = readShipTemplateUnion();
|
||||
|
||||
describe("/ship redaction wiring", () => {
|
||||
test("scans the PR body via the shared bin before create", () => {
|
||||
|
||||
@@ -197,20 +197,26 @@ describeE2E('/ship idempotency E2E (periodic, real-PTY)', () => {
|
||||
}
|
||||
}
|
||||
|
||||
// Positive: the idempotency-check echoed ALREADY_BUMPED.
|
||||
if (/STATE:\s*ALREADY_BUMPED/.test(visible)) {
|
||||
// Positive: idempotency classify reported ALREADY_BUMPED. Post-carve
|
||||
// (T9), Step 12 runs `gstack-version-bump classify` which emits JSON
|
||||
// (`"state":"ALREADY_BUMPED"`); the legacy inline bash echoed
|
||||
// `STATE: ALREADY_BUMPED`. Accept either so the test survives the carve.
|
||||
if (/STATE:\s*ALREADY_BUMPED|"state":\s*"ALREADY_BUMPED"/.test(visible)) {
|
||||
outcome = 'detected';
|
||||
evidence = visible.slice(-3000);
|
||||
break;
|
||||
}
|
||||
|
||||
// Negative regressions:
|
||||
// - bump-action bash block ran (would echo on FRESH path)
|
||||
// - classify reported FRESH (CLI JSON or legacy echo) → would re-bump
|
||||
// - agent attempted git commit -m "chore: bump version"
|
||||
// - agent attempted git push
|
||||
// - agent rendered an Edit/Write to CHANGELOG.md or VERSION (acceptable in plan mode but flagged here)
|
||||
// - agent ran the CLI write path (gstack-version-bump write) — a
|
||||
// re-bump on an already-shipped branch
|
||||
if (
|
||||
/"state":\s*"FRESH"/.test(visible) ||
|
||||
/STATE:\s*FRESH(?![\w-])/i.test(visible) ||
|
||||
/gstack-version-bump\s+write/i.test(visible) ||
|
||||
/git\s+commit\s+.*chore:\s*bump\s+version/i.test(visible) ||
|
||||
/git\s+push.*origin/i.test(visible)
|
||||
) {
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
/**
|
||||
* /ship section-loading E2E (periodic, paid, real-PTY) — v2 plan T9 mitigation
|
||||
* layer 5, the ONLY CI-failing guard against silent section-skip.
|
||||
*
|
||||
* After the carve, ship is a skeleton whose STOP-Read directives point at
|
||||
* sections/*.md. This test runs the REAL /ship skill in plan mode against a
|
||||
* fresh version-changing fixture and asserts the agent actually Read the
|
||||
* sections its situation requires (review-army + changelog at minimum — every
|
||||
* version-changing ship needs the pre-landing review and a CHANGELOG entry).
|
||||
*
|
||||
* Runs against the INSTALLED skill at ~/.claude/skills/gstack/ship (Codex
|
||||
* outside-voice #5: an E2E that reads repo paths would miss install-layout
|
||||
* 404s). Section reads are detected from the PTY scrollback — when the agent
|
||||
* Reads a section the tool render shows the `sections/<file>.md` path.
|
||||
*
|
||||
* Plan-mode framing keeps the agent from committing/pushing; producing a plan
|
||||
* is the terminal signal. Cost: ~$2-4/run. Periodic tier.
|
||||
*
|
||||
* Situation matrix (T1 = B): this file covers the fresh version-changing ship;
|
||||
* the already-bumped re-run is covered by skill-e2e-ship-idempotency.test.ts,
|
||||
* and a no-plan-file variant can be added to FIXTURES below.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { spawnSync } from 'child_process';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import {
|
||||
launchClaudePty,
|
||||
isPermissionDialogVisible,
|
||||
isNumberedOptionListVisible,
|
||||
} from './helpers/claude-pty-runner';
|
||||
|
||||
const shouldRun = !!process.env.EVALS && process.env.EVALS_TIER === 'periodic';
|
||||
const describeE2E = shouldRun ? describe : describe.skip;
|
||||
|
||||
/** Fresh fixture: feature branch with a real change but VERSION still == base,
|
||||
* so /ship must bump (FRESH) and walk the full pre-landing + changelog flow. */
|
||||
function buildFreshFixture(): { workTree: string; root: string } {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-ship-secload-'));
|
||||
const workTree = path.join(root, 'workspace');
|
||||
const bareRemote = path.join(root, 'origin.git');
|
||||
fs.mkdirSync(workTree, { recursive: true });
|
||||
const sh = (cmd: string, cwd: string): void => {
|
||||
const r = spawnSync('bash', ['-c', cmd], { cwd, stdio: 'pipe', timeout: 15_000 });
|
||||
if (r.status !== 0) throw new Error(`fixture setup failed at "${cmd}":\n${r.stderr?.toString()}`);
|
||||
};
|
||||
sh(`git init --bare "${bareRemote}"`, root);
|
||||
sh('git init -b main', workTree);
|
||||
sh('git config user.email "t@t.com" && git config user.name "T" && git config commit.gpgsign false', workTree);
|
||||
fs.writeFileSync(path.join(workTree, 'VERSION'), '0.0.1\n');
|
||||
fs.writeFileSync(path.join(workTree, 'package.json'), JSON.stringify({ name: 'fx', version: '0.0.1', private: true }, null, 2) + '\n');
|
||||
fs.writeFileSync(path.join(workTree, 'CHANGELOG.md'), '# Changelog\n\n## [0.0.1] - 2026-01-01\n\n- Initial release\n');
|
||||
fs.writeFileSync(path.join(workTree, 'app.js'), '// base\n');
|
||||
sh('git add -A && git commit -m "chore: initial v0.0.1"', workTree);
|
||||
sh(`git remote add origin "${bareRemote}" && git push -u origin main`, workTree);
|
||||
// Feature branch: a real code change, VERSION untouched → FRESH (needs a bump).
|
||||
sh('git checkout -b feat/new-thing', workTree);
|
||||
fs.writeFileSync(path.join(workTree, 'app.js'), '// base\nexport function newThing() { return 42; }\n');
|
||||
fs.writeFileSync(path.join(workTree, 'app.test.js'), 'test("newThing", () => {});\n');
|
||||
sh('git add -A && git commit -m "feat: add newThing"', workTree);
|
||||
sh('git push -u origin feat/new-thing', workTree);
|
||||
return { workTree, root };
|
||||
}
|
||||
|
||||
// Sections every version-changing ship must consult.
|
||||
const REQUIRED_SECTIONS = ['review-army.md', 'changelog.md'];
|
||||
|
||||
describeE2E('/ship section-loading E2E (periodic, real-PTY, installed skill)', () => {
|
||||
test(
|
||||
'fresh version-changing ship Reads the required sections',
|
||||
async () => {
|
||||
const { workTree, root } = buildFreshFixture();
|
||||
const session = await launchClaudePty({
|
||||
permissionMode: 'plan',
|
||||
cwd: workTree,
|
||||
timeoutMs: 720_000,
|
||||
env: { GH_TOKEN: 'mock-not-real', NO_COLOR: '1' },
|
||||
});
|
||||
|
||||
const readSections = new Set<string>();
|
||||
let planReady = false;
|
||||
try {
|
||||
await Bun.sleep(8000);
|
||||
const since = session.mark();
|
||||
session.send('/ship\r');
|
||||
const start = Date.now();
|
||||
let lastPermSig = '';
|
||||
while (Date.now() - start < 600_000) {
|
||||
await Bun.sleep(3000);
|
||||
if (session.exited()) break;
|
||||
const visible = session.visibleSince(since);
|
||||
const tail = visible.slice(-1500);
|
||||
if (isNumberedOptionListVisible(tail) && isPermissionDialogVisible(tail)) {
|
||||
const sig = visible.slice(-500);
|
||||
if (sig !== lastPermSig) { lastPermSig = sig; session.send('1\r'); await Bun.sleep(1500); continue; }
|
||||
}
|
||||
// Detect section reads from the scrollback (tool render shows the path).
|
||||
for (const m of visible.matchAll(/sections\/([A-Za-z0-9._-]+\.md)/g)) readSections.add(m[1]);
|
||||
if (/ready to execute|Would you like to proceed|GSTACK REVIEW REPORT/i.test(visible)) {
|
||||
planReady = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
await session.close();
|
||||
try { fs.rmSync(root, { recursive: true, force: true }); } catch { /* ignore */ }
|
||||
}
|
||||
|
||||
const missing = REQUIRED_SECTIONS.filter(s => !readSections.has(s));
|
||||
expect({ planReady, read: [...readSections], missing }).toEqual({
|
||||
planReady: true,
|
||||
read: expect.any(Array),
|
||||
missing: [],
|
||||
});
|
||||
},
|
||||
900_000,
|
||||
);
|
||||
});
|
||||
@@ -156,7 +156,11 @@ describe('SKILL.md size budget regression (gate, free)', () => {
|
||||
const baseline: ParityBaseline = JSON.parse(fs.readFileSync(BASELINE_PATH, 'utf-8'));
|
||||
const current = captureBaseline({ repoRoot: REPO_ROOT });
|
||||
const MIN_RATIO = 0.80; // a skill at <80% of its v1.44 size signals mass-deletion
|
||||
const SECTIONS_EXTRACTED = new Set<string>(); // populate in v2.0.0.0 when sections/ lands
|
||||
// Carved skills (v2 plan T9): the skeleton SKILL.md intentionally shrinks
|
||||
// because prose moved into sections/*.md. The union size is guarded instead
|
||||
// by the sectioned ship invariant in parity-harness.ts (minBytes on the
|
||||
// skeleton+sections union), so exempt the skeleton from the body-strip floor.
|
||||
const SECTIONS_EXTRACTED = new Set<string>(['ship']);
|
||||
|
||||
const undershoots: Array<{
|
||||
skill: string; beforeBytes: number; afterBytes: number; ratio: number;
|
||||
|
||||
@@ -7,6 +7,22 @@ import * as path from 'path';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
|
||||
// Carved-skill aware (v2 plan T9): ship is a skeleton SKILL.md + sections/*.md.
|
||||
// Read the union so validations of content that moved into a section still hold.
|
||||
// `_SHIP_MD` is a distinct path expression so a mechanical read-replace can't
|
||||
// recurse into this helper.
|
||||
const _SHIP_MD = path.join(ROOT, 'ship', 'SKILL.md');
|
||||
function readShipUnion(): string {
|
||||
let t = fs.readFileSync(_SHIP_MD, 'utf-8');
|
||||
const secDir = path.join(ROOT, 'ship', 'sections');
|
||||
if (fs.existsSync(secDir)) {
|
||||
for (const f of fs.readdirSync(secDir).sort()) {
|
||||
if (f.endsWith('.md')) t += '\n' + fs.readFileSync(path.join(secDir, f), 'utf-8');
|
||||
}
|
||||
}
|
||||
return t;
|
||||
}
|
||||
|
||||
describe('SKILL.md command validation', () => {
|
||||
test('all $B commands in SKILL.md are valid browse commands', () => {
|
||||
const result = validateSkill(path.join(ROOT, 'SKILL.md'));
|
||||
@@ -315,7 +331,8 @@ describe('Cross-skill path consistency', () => {
|
||||
for (const file of filesToCheck) {
|
||||
const filePath = path.join(ROOT, file);
|
||||
if (!fs.existsSync(filePath)) continue;
|
||||
const content = fs.readFileSync(filePath, 'utf-8');
|
||||
// ship's greptile handling moved into sections/greptile.md (T9 carve).
|
||||
const content = file === 'ship/SKILL.md' ? readShipUnion() : fs.readFileSync(filePath, 'utf-8');
|
||||
|
||||
const hasBoth = (content.includes('per-project') && content.includes('global')) ||
|
||||
(content.includes('$REMOTE_SLUG/greptile-history') && content.includes('~/.gstack/greptile-history'));
|
||||
@@ -437,7 +454,7 @@ describe('Greptile history format consistency', () => {
|
||||
|
||||
test('review/SKILL.md and ship/SKILL.md both reference greptile-triage.md for write details', () => {
|
||||
const reviewContent = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
||||
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipContent = readShipUnion();
|
||||
|
||||
expect(reviewContent.toLowerCase()).toContain('greptile-triage.md');
|
||||
expect(shipContent.toLowerCase()).toContain('greptile-triage.md');
|
||||
@@ -530,7 +547,7 @@ describe('TODOS-format.md reference consistency', () => {
|
||||
});
|
||||
|
||||
test('skills that write TODOs reference TODOS-format.md', () => {
|
||||
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipContent = readShipUnion();
|
||||
const ceoPlanContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
||||
const engPlanContent = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
|
||||
|
||||
@@ -788,7 +805,7 @@ describe('Enum & Value Completeness in review checklist', () => {
|
||||
expect(checklist).toContain('ASK');
|
||||
|
||||
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review/SKILL.md'), 'utf-8');
|
||||
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship/SKILL.md'), 'utf-8');
|
||||
const shipSkill = readShipUnion();
|
||||
expect(reviewSkill).toContain('AUTO-FIX');
|
||||
expect(reviewSkill).toContain('[AUTO-FIXED]');
|
||||
expect(shipSkill).toContain('AUTO-FIX');
|
||||
@@ -1014,7 +1031,7 @@ describe('Test Bootstrap ({{TEST_BOOTSTRAP}}) integration', () => {
|
||||
});
|
||||
|
||||
test('TEST_BOOTSTRAP appears in ship/SKILL.md', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('Test Framework Bootstrap');
|
||||
expect(content).toContain('Step 4');
|
||||
});
|
||||
@@ -1063,7 +1080,7 @@ describe('Test Bootstrap ({{TEST_BOOTSTRAP}}) integration', () => {
|
||||
|
||||
test('WebSearch is in allowed-tools for qa, ship, design-review', () => {
|
||||
const qa = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
||||
const ship = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const ship = readShipUnion();
|
||||
const qaDesign = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
|
||||
expect(qa).toContain('WebSearch');
|
||||
expect(ship).toContain('WebSearch');
|
||||
@@ -1112,7 +1129,7 @@ describe('Phase 8e.5 regression test generation', () => {
|
||||
|
||||
describe('Step 3.4 test coverage audit', () => {
|
||||
test('ship/SKILL.md contains Step 7', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('Step 7: Test Coverage Audit');
|
||||
// The coverage diagram collapses code-path and user-flow counts onto one
|
||||
// summary line. Verify that summary is present (labels are stable).
|
||||
@@ -1120,7 +1137,7 @@ describe('Step 3.4 test coverage audit', () => {
|
||||
});
|
||||
|
||||
test('Step 3.4 includes quality scoring rubric', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('★★★');
|
||||
expect(content).toContain('★★');
|
||||
expect(content).toContain('edge cases AND error paths');
|
||||
@@ -1128,36 +1145,36 @@ describe('Step 3.4 test coverage audit', () => {
|
||||
});
|
||||
|
||||
test('Step 3.4 includes before/after test count', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('Count test files before');
|
||||
expect(content).toContain('Count test files after');
|
||||
});
|
||||
|
||||
test('ship PR body includes Test Coverage section', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('## Test Coverage');
|
||||
});
|
||||
|
||||
test('ship rules include test generation rule', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('Step 7 generates coverage tests');
|
||||
expect(content).toContain('Never commit failing tests');
|
||||
});
|
||||
|
||||
test('Step 3.4 includes vibe coding philosophy', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('vibe coding becomes yolo coding');
|
||||
});
|
||||
|
||||
test('Step 3.4 traces actual codepaths, not just syntax', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('Trace every codepath');
|
||||
expect(content).toContain('Trace data flow');
|
||||
expect(content).toContain('Diagram the execution');
|
||||
});
|
||||
|
||||
test('Step 3.4 maps user flows and interaction edge cases', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('Map user flows');
|
||||
expect(content).toContain('Interaction edge cases');
|
||||
expect(content).toContain('Double-click');
|
||||
@@ -1167,7 +1184,7 @@ describe('Step 3.4 test coverage audit', () => {
|
||||
});
|
||||
|
||||
test('Step 3.4 diagram includes user-flow coverage summary', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
// The diagram was compressed from separate CODE PATH COVERAGE / USER FLOW
|
||||
// COVERAGE section headers into a single summary line. Assert on the
|
||||
// labels that still appear on that summary line.
|
||||
@@ -1203,7 +1220,7 @@ describe('ship step numbering', () => {
|
||||
});
|
||||
|
||||
test('ship/SKILL.md main headings use clean integer step numbers', () => {
|
||||
const skill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const skill = readShipUnion();
|
||||
// Headings like "## Step 7: Test Coverage Audit" — NOT sub-steps like "## Step 8.1:"
|
||||
const headings = Array.from(skill.matchAll(/^## Step (\d+(?:\.\d+)?):/gm)).map(
|
||||
(m) => m[1]
|
||||
@@ -1381,7 +1398,7 @@ describe('Codex skill', () => {
|
||||
});
|
||||
|
||||
test('adversarial review in /ship always runs both passes', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('Adversarial review (always-on)');
|
||||
expect(content).toContain('adversarial-review');
|
||||
expect(content).toContain('reasoning_effort="high"');
|
||||
@@ -1391,7 +1408,7 @@ describe('Codex skill', () => {
|
||||
|
||||
test('scope drift detection in /review and /ship', () => {
|
||||
const reviewContent = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
||||
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const shipContent = readShipUnion();
|
||||
// Both should contain scope drift from the shared resolver
|
||||
for (const content of [reviewContent, shipContent]) {
|
||||
expect(content).toContain('Scope Check:');
|
||||
@@ -1427,7 +1444,8 @@ describe('Codex skill', () => {
|
||||
|
||||
test('codex review invocations avoid the prompt plus --base argument shape', () => {
|
||||
for (const rel of ['codex/SKILL.md', 'review/SKILL.md', 'ship/SKILL.md']) {
|
||||
const content = fs.readFileSync(path.join(ROOT, rel), 'utf-8');
|
||||
// ship's codex command moved into sections/adversarial.md (T9 carve).
|
||||
const content = rel === 'ship/SKILL.md' ? readShipUnion() : fs.readFileSync(path.join(ROOT, rel), 'utf-8');
|
||||
expect(content).not.toContain('--base <base> -c \'model_reasoning_effort="high"\'');
|
||||
expect(content).toContain('Run git diff origin/<base>...HEAD 2>/dev/null || git diff <base>...HEAD');
|
||||
}
|
||||
@@ -1443,7 +1461,8 @@ describe('Codex skill', () => {
|
||||
const boundaryLine =
|
||||
'Do NOT read or execute any files under ~/.claude/, ~/.agents/, .claude/skills/, or agents/';
|
||||
for (const rel of ['codex/SKILL.md', 'review/SKILL.md', 'ship/SKILL.md']) {
|
||||
const content = fs.readFileSync(path.join(ROOT, rel), 'utf-8');
|
||||
// ship's codex/adversarial boundary line moved into sections/adversarial.md.
|
||||
const content = rel === 'ship/SKILL.md' ? readShipUnion() : fs.readFileSync(path.join(ROOT, rel), 'utf-8');
|
||||
expect(content).toContain(boundaryLine);
|
||||
}
|
||||
});
|
||||
@@ -1456,7 +1475,7 @@ describe('Codex skill', () => {
|
||||
});
|
||||
|
||||
test('Review Readiness Dashboard includes Adversarial Review row', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('Adversarial');
|
||||
expect(content).toContain('codex-review');
|
||||
});
|
||||
@@ -1711,17 +1730,17 @@ describe('Repo mode preamble validation', () => {
|
||||
|
||||
describe('Test failure triage in ship skill', () => {
|
||||
test('ship/SKILL.md contains Test Failure Ownership Triage', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('Test Failure Ownership Triage');
|
||||
});
|
||||
|
||||
test('ship/SKILL.md triage uses git diff for classification', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('git diff origin/<base>...HEAD --name-only');
|
||||
});
|
||||
|
||||
test('ship/SKILL.md triage has solo and collaborative paths', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('REPO_MODE');
|
||||
expect(content).toContain('solo');
|
||||
expect(content).toContain('collaborative');
|
||||
@@ -1730,18 +1749,18 @@ describe('Test failure triage in ship skill', () => {
|
||||
});
|
||||
|
||||
test('ship/SKILL.md triage has GitHub issue assignment for collaborative mode', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('gh issue create');
|
||||
expect(content).toContain('--assignee');
|
||||
});
|
||||
|
||||
test('{{TEST_FAILURE_TRIAGE}} placeholder is fully resolved in ship/SKILL.md', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).not.toContain('{{TEST_FAILURE_TRIAGE}}');
|
||||
});
|
||||
|
||||
test('ship/SKILL.md uses in-branch language for stop condition', () => {
|
||||
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
||||
const content = readShipUnion();
|
||||
expect(content).toContain('In-branch test failures');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
/**
|
||||
* Section TemplateContext parity (v2 plan T9 / Codex consult absorbed-refinement #1).
|
||||
*
|
||||
* Section generation must use the SAME TemplateContext as the parent skill —
|
||||
* crucially the same skillName, so resolver `appliesTo` gating + tier behave
|
||||
* identically. If a section resolved with skillName "sections" (the bug
|
||||
* processSectionTemplate guards against), gated resolvers like ADVERSARIAL_STEP /
|
||||
* CONFIDENCE_CALIBRATION would render empty.
|
||||
*
|
||||
* We assert on the GENERATED section output: gated resolver content is present and
|
||||
* no placeholder is left unresolved. That can only be true if the parent ctx
|
||||
* (skillName=ship) drove the resolve.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
const SHIP_SECTIONS = path.join(ROOT, 'ship', 'sections');
|
||||
|
||||
function readSection(file: string): string {
|
||||
return fs.readFileSync(path.join(SHIP_SECTIONS, file), 'utf-8');
|
||||
}
|
||||
|
||||
describe('section TemplateContext parity (skillName pinned to parent)', () => {
|
||||
test('no generated section has unresolved {{PLACEHOLDER}} tokens', () => {
|
||||
for (const md of fs.readdirSync(SHIP_SECTIONS).filter(f => f.endsWith('.md') && !f.endsWith('.md.tmpl'))) {
|
||||
const content = readSection(md);
|
||||
const unresolved = content.match(/\{\{[A-Z_]+(?::[^}]+)?\}\}/g);
|
||||
expect({ md, unresolved }).toEqual({ md, unresolved: null });
|
||||
}
|
||||
});
|
||||
|
||||
test('adversarial section rendered the ADVERSARIAL_STEP resolver (proves ship ctx)', () => {
|
||||
const content = readSection('adversarial.md');
|
||||
// The codex filesystem-boundary line only appears when ADVERSARIAL_STEP resolves.
|
||||
expect(content).toContain('Do NOT read or execute any files under');
|
||||
expect(content.length).toBeGreaterThan(500);
|
||||
});
|
||||
|
||||
test('review-army section rendered CONFIDENCE_CALIBRATION + REVIEW_ARMY (gated resolvers)', () => {
|
||||
const content = readSection('review-army.md');
|
||||
expect(content).toContain('Confidence Calibration');
|
||||
expect(content).toContain('confidence score');
|
||||
});
|
||||
|
||||
test('tests section rendered TEST_BOOTSTRAP + TEST_FAILURE_TRIAGE', () => {
|
||||
const content = readSection('tests.md');
|
||||
expect(content).toContain('Test Failure Ownership Triage');
|
||||
});
|
||||
|
||||
test('changelog section rendered CHANGELOG_WORKFLOW', () => {
|
||||
const content = readSection('changelog.md');
|
||||
expect(content).toContain('CHANGELOG');
|
||||
expect(content.length).toBeGreaterThan(300);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,136 @@
|
||||
/**
|
||||
* Unit tests for the transcript section logger (T10). Pure-function coverage —
|
||||
* no paid run needed. Drives the analyzers with synthetic tool-call transcripts.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, afterAll } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import {
|
||||
extractSectionReads,
|
||||
extractShipActions,
|
||||
compareShipActions,
|
||||
writeShipBaseline,
|
||||
readShipBaseline,
|
||||
baselinePath,
|
||||
SHIP_ACTIONS,
|
||||
type ToolCallLike,
|
||||
type ShipBaseline,
|
||||
} from './helpers/transcript-section-logger';
|
||||
|
||||
const read = (fp: string): ToolCallLike => ({ tool: 'Read', input: { file_path: fp }, output: '' });
|
||||
const bash = (command: string): ToolCallLike => ({ tool: 'Bash', input: { command }, output: '' });
|
||||
|
||||
describe('extractSectionReads', () => {
|
||||
test('picks up section reads via the /sections/<file>.md segment', () => {
|
||||
const result = {
|
||||
toolCalls: [
|
||||
read('/Users/x/.claude/skills/gstack-ship/sections/version-bump.md'),
|
||||
read('ship/sections/changelog.md'),
|
||||
read('/abs/.factory/skills/gstack-ship/sections/review-army.md'),
|
||||
],
|
||||
};
|
||||
expect(extractSectionReads(result)).toEqual(['version-bump.md', 'changelog.md', 'review-army.md']);
|
||||
});
|
||||
|
||||
test('ignores non-section reads and non-Read tools', () => {
|
||||
const result = {
|
||||
toolCalls: [
|
||||
read('ship/SKILL.md'),
|
||||
read('/some/sections-like/notsections/x.md'),
|
||||
bash('cat ship/sections/version-bump.md'), // bash, not a Read
|
||||
],
|
||||
};
|
||||
expect(extractSectionReads(result)).toEqual([]);
|
||||
});
|
||||
|
||||
test('dedupes and preserves first-read order', () => {
|
||||
const result = {
|
||||
toolCalls: [
|
||||
read('ship/sections/tests.md'),
|
||||
read('ship/sections/version-bump.md'),
|
||||
read('ship/sections/tests.md'),
|
||||
],
|
||||
};
|
||||
expect(extractSectionReads(result)).toEqual(['tests.md', 'version-bump.md']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractShipActions', () => {
|
||||
test('detects the full action fingerprint from bash + writes', () => {
|
||||
const result = {
|
||||
toolCalls: [
|
||||
bash('git merge origin/main'),
|
||||
bash('bun test'),
|
||||
bash('gstack-version-bump --bump minor'),
|
||||
{ tool: 'Edit', input: { file_path: 'CHANGELOG.md' }, output: '' },
|
||||
bash('git commit -m "v1.2.0.0 feat"'),
|
||||
bash('git push origin HEAD'),
|
||||
bash('gh pr create --base main'),
|
||||
],
|
||||
};
|
||||
expect(extractShipActions(result)).toEqual([...SHIP_ACTIONS]);
|
||||
});
|
||||
|
||||
test('returns canonical order regardless of execution order', () => {
|
||||
const result = {
|
||||
toolCalls: [
|
||||
bash('gh pr create --base main'),
|
||||
bash('git merge origin/main'),
|
||||
],
|
||||
};
|
||||
expect(extractShipActions(result)).toEqual(['merged_base', 'opened_pr']);
|
||||
});
|
||||
|
||||
test('VERSION write counts as a version bump even without the CLI', () => {
|
||||
const result = { toolCalls: [{ tool: 'Write', input: { file_path: 'VERSION' }, output: '' }] };
|
||||
expect(extractShipActions(result)).toEqual(['bumped_version']);
|
||||
});
|
||||
|
||||
test('empty run produces empty fingerprint', () => {
|
||||
expect(extractShipActions({ toolCalls: [] })).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('compareShipActions', () => {
|
||||
const baseline: ShipBaseline = {
|
||||
tag: 'monolith',
|
||||
situation: 'fresh-version-changing',
|
||||
actions: ['merged_base', 'ran_tests', 'bumped_version', 'wrote_changelog', 'committed', 'pushed', 'opened_pr'],
|
||||
sectionReads: [],
|
||||
capturedAt: '2026-05-30T00:00:00Z',
|
||||
};
|
||||
|
||||
test('flags a dropped action as the carve regression', () => {
|
||||
const current = baseline.actions.filter(a => a !== 'bumped_version');
|
||||
const diff = compareShipActions(baseline, current);
|
||||
expect(diff.ok).toBe(false);
|
||||
expect(diff.missing).toEqual(['bumped_version']);
|
||||
});
|
||||
|
||||
test('passes when the sectioned run performs every baseline action', () => {
|
||||
const diff = compareShipActions(baseline, [...baseline.actions, 'merged_base']);
|
||||
expect(diff.ok).toBe(true);
|
||||
expect(diff.missing).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('baseline persistence', () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ship-baseline-'));
|
||||
afterAll(() => { try { fs.rmSync(dir, { recursive: true, force: true }); } catch { /* noop */ } });
|
||||
|
||||
test('round-trips a baseline to disk', () => {
|
||||
const baseline: ShipBaseline = {
|
||||
tag: 'monolith', situation: 'no-plan-file',
|
||||
actions: ['ran_tests', 'committed'], sectionReads: [], capturedAt: '2026-05-30T00:00:00Z',
|
||||
};
|
||||
const p = writeShipBaseline(baseline, dir);
|
||||
expect(p).toBe(baselinePath('no-plan-file', dir));
|
||||
expect(readShipBaseline('no-plan-file', dir)).toEqual(baseline);
|
||||
});
|
||||
|
||||
test('returns null when no baseline captured yet', () => {
|
||||
expect(readShipBaseline('never-captured', dir)).toBeNull();
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user