Files
gstack/test/plan-reality-receipts.test.ts
Garry Tanandcapy-ai[bot] 8695938e74 v1.92.15.0 feat: plan reviews check facts and keep one task list; /office-hours hands its wireframe to design review (#340, #513, #973, #1048, #1791) (#3775)
* G1: gstack-plan-reality check opens every file:line receipt (#973)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G1: office-hours spec reviewer checks paths and flags through a bin-enforced probe (#973)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2: plan-review reruns reconcile the last task list (gstack-tasks prior/supersede) (#1791)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2: plan reviews write task lists through gstack-tasks start/add/close

The emit block shrinks to three calls: prior (now defaulting --dir and
--branch, and printing how to reconcile), start (creates the empty
branch-named list) and add (one validated row per task). close records a
done or dropped prior task. The plan-review result codes move to
lib/result-codes-plan.ts so lib/result-codes.ts stays under the module-size
ratchet; generated plan-review sections shrink, so their size pins go down.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* F2: office-hours saves the sketch beside the design doc; /plan-design-review reads it

Step 5 copies sketch.png/html to <PROJECT_DIR>/{user}-{branch}-sketch-{datetime}.*
and writes a Wireframe: line (state-root relative) or Wireframe: none (<reason>).
The design review section finds the design doc before Pass 1 and opens the
wireframe, text-only when the file is missing. G4's signals line rides along
in the same generated office-hours files; tests in planning-wave-prose.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G3: plan reviews tie storage to queries, not file count (#1048)

One bullet in each of plan-eng-review and plan-ceo-review: fields queried,
constrained or updated on their own get explicit columns or tables; JSON suits
a document read and written whole. Written fresh; the reporter's patch is not
ported.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* evals: GSTA-228 planning-wave cases with branch, main and no-gstack arms

planted facts (G1, behavior), data model (G3, behavior), task-list rerun (G2,
rule) and closing signals (G4, behavior). GSTACK_WAVE_ARM picks the arm; only
the branch arm asserts the preregistered bar, every arm writes a score row.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* changelog: GSTA-228 planning wave fragment

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G1: the probe takes subcommand words for fixed-subcommand CLIs

The planted-facts eval's one branch miss: the reviewer asked to probe
"git log", the probe refused it, and git's top-level help cannot show a log
flag, so the fabricated --since-commit stayed "unverified".
probe --help <command> [<subcommand> [<subcommand>]] now runs
<cli> <sub> --help for git, gh, docker, cargo, go, npm and uv (never node,
bun, python3, make, yarn, pnpm or a gstack bin, where the word could name a
script), in an empty temporary directory with pagers set to cat, falling back
to -h when --help has no usage line (a minimized system's missing man page).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2: task text goes through a rows file, never a command line

The free suite's generated-bash lint (INV-3) refused the add command's
<title>/<finding> placeholders: free text reaches a command only as a file the
agent writes with its file-write tool. start now also creates an empty rows
file in the repository's git-excluded .gstack/tmp and records the run; the
agent writes task and disposition rows there; add validates every row,
stamps phase, run, branch and commit, appends tasks to the list and
dispositions to task-dispositions-<phase>-<branch>.jsonl, then supersedes the
branch's earlier lists. close is gone (dispositions are rows).

The new planning-wave cases get measured periodic walls in
scripts/paid-test-durations.json so the periodic census still packs into 40
slices, the schema case declares the preamble sources its eng-template owners
need, and the paid file count is 114.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2: fold start into add so each plan review emits one bash fence

The rows file lives at .gstack/tmp/tasks-<phase>-rows.jsonl; add writes this
run's list (empty when there are no rows), appends dispositions, deletes the
rows file and supersedes the branch's earlier lists. Lowers 27 size pins.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2 tests: give gstack-tasks a private HOME and state root

CI shard 13 home containment caught gstack-slug writing ~/.gstack/slug-cache
entries from the tasks-prior fixtures.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* changelog: gstack-tasks has no start command

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G1/G2: tighten the probe and the rows file after ship review

The probe now accepts only each CLI's listed built-in subcommands (no uv run,
docker run, go test, npm exec or gh extensions), refuses gstack bins that do not
handle --help and never retries them with -h, and runs with a throwaway HOME and
an environment without keys. gstack-tasks resolves the slug through
runScriptSync (Windows) and prior deletes a rows file an unfinished run left.
Adds tests for the dropped disposition and the stale rows file.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2: add refuses while an earlier task is neither carried nor closed

Ship review (state matrix): a rerun that skipped RECONCILE silently retired
open tasks. add now names each undecided <run_id>/<id> and writes nothing.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G1/G2: fixes from the ship fix-check and Step 11 voices

Probe: find the repository root without running git, drop relative and
in-repository PATH entries for lookup and for the child, keep HOME and
toolchain locations (rustup, mise, asdf) but no keys, drop cargo fmt/clippy
(external cargo-* programs), restore planted env in the test.
gstack-tasks: resolve the exclude path against the cwd (subdirectories),
resolve --dir, stamp an empty branch on a detached HEAD like the aggregator,
name the file from --branch, and refuse a same-second second list instead of
appending. Receipts: line 0, reversed ranges, dangling links and paths
through files are unresolved instead of crashing.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* changelog: probe limits and the reconcile refusal

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: probe subcommand counts per CLI (ship docs audit)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-11-01 06:11:17 +00:00

110 lines
5.9 KiB
TypeScript

/**
* G1 (#973): `gstack-plan-reality check` opens every file:line receipt. A
* receipt naming a missing file, or a line past the end of a real file, is
* unresolved and keeps the phase open; a receipt this checkout cannot judge
* (outside every root, a sha a shallow clone lacks) is reported, never
* blocking. The pure `checkRealityRows` is unchanged; only the bin resolves.
*/
import { describe, expect, test } from 'bun:test';
import { spawnSync } from 'node:child_process';
import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { parseRealityLines } from '../lib/plan-reality';
import { receiptTokens, resolveReceipts } from '../lib/plan-reality-receipts';
const ROOT = path.resolve(import.meta.dir, '..');
const BIN = path.join(ROOT, 'bin', 'gstack-plan-reality');
function fixtureRepo(): string {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'reality-receipts-'));
fs.mkdirSync(path.join(dir, 'lib'));
fs.writeFileSync(path.join(dir, 'lib', 'a.ts'), 'one\ntwo\nthree\n');
spawnSync('git', ['init', '-q'], { cwd: dir, timeout: 10_000 });
return dir;
}
const rows = (receipts: Record<string, string>) => ['premise-table', 'already-done', 'surface-check', 'binding-decisions', 'numbers', 'deferred-asks', 'pr-count']
.map(id => `REALITY: ${id} pass checked ${receipts[id] ?? 'lib/a.ts:1'}`).join('\n') + '\n';
function check(cwd: string, review: string, extra: string[] = []) {
const r = spawnSync('bun', [BIN, 'check', '--phase', 'eng', '--session-kind', 'unattended', ...extra, '-'], { cwd, input: review, encoding: 'utf8', timeout: 30_000 });
return { code: r.status, out: r.stdout, err: r.stderr };
}
describe('gstack-plan-reality check resolves receipts', () => {
test('a missing file and a line past EOF are unresolved and keep the phase open', () => {
const dir = fixtureRepo();
const r = check(dir, rows({ 'premise-table': 'lib/does-not-exist.ts:12', 'already-done': 'lib/a.ts:99' }));
expect(r.out).toContain('unresolved=premise-table,already-done');
expect(r.out).toContain('verdict=incomplete');
expect(r.out).toContain('RECEIPT_UNRESOLVED: premise-table lib/does-not-exist.ts:12 no such file');
expect(r.out).toContain('RECEIPT_UNRESOLVED: already-done lib/a.ts:99 line 99 is past the end (3 lines)');
expect(r.err).toContain('(REALITY_RECEIPT_UNRESOLVED)');
expect(r.code).toBe(1);
});
test('real receipts and ranges pass', () => {
const dir = fixtureRepo();
const r = check(dir, rows({ 'premise-table': 'lib/a.ts:1-3', 'already-done': 'lib/a.ts:2' }));
expect(r.out).toContain('verdict=complete');
expect(r.out).not.toContain('unresolved=');
expect(r.code).toBe(0);
});
test('receipts this checkout cannot judge are reported, never blocking', () => {
const dir = fixtureRepo();
const r = check(dir, rows({ 'premise-table': '~/notes/plan.md:1', 'already-done': 'gate.r1.json:1', 'surface-check': 'deadbeefcafe' }));
expect(r.out).toContain('verdict=complete');
expect(r.err.match(/^RECEIPT_UNVERIFIABLE: /gm)?.length).toBe(3);
expect(r.code).toBe(0);
});
test('--root adds a root and the default is the git top level', () => {
const dir = fixtureRepo();
const other = fixtureRepo();
fs.writeFileSync(path.join(other, 'lib', 'b.ts'), 'x\n');
const review = rows({ 'premise-table': 'lib/b.ts:1' });
expect(check(dir, review).out).toContain('unresolved=premise-table');
expect(check(dir, review, ['--root', dir, '--root', other]).out).toContain('verdict=complete');
expect(check(path.join(dir, 'lib'), rows({})).out).toContain('verdict=complete');
});
});
describe('resolveReceipts', () => {
test('every receipt token on a row is checked; host:port is not a receipt', () => {
expect(receiptTokens('see lib/a.ts:1 and lib/b.ts:2, served on localhost:3000')).toEqual(['lib/a.ts:1', 'lib/b.ts:2']);
const dir = fixtureRepo();
const out = resolveReceipts(parseRealityLines('REALITY: numbers pass ok lib/a.ts:1 lib/gone.ts:4'), { roots: [dir] });
expect(out.map(r => [r.receipt, r.outcome])).toEqual([['lib/a.ts:1', 'resolved'], ['lib/gone.ts:4', 'unresolved']]);
});
test('a symlink out of the root, a directory and an oversized file are never read', () => {
const dir = fixtureRepo();
const outside = fs.mkdtempSync(path.join(os.tmpdir(), 'reality-outside-'));
fs.writeFileSync(path.join(outside, 'secret.txt'), 'TOKEN=abc\n');
fs.symlinkSync(path.join(outside, 'secret.txt'), path.join(dir, 'lib', 'link.txt'));
fs.writeFileSync(path.join(dir, 'lib', 'big.txt'), 'x'.repeat(64));
const out = resolveReceipts(parseRealityLines('REALITY: numbers pass ok lib/link.txt:1 lib/:1 lib/big.txt:1'), { roots: [dir], maxBytes: 32 });
expect(out.map(r => r.outcome)).toEqual(['unverifiable', 'unresolved', 'unverifiable']);
expect(JSON.stringify(out)).not.toContain('TOKEN');
});
test('line 0, a reversed range, a dangling link and a path through a file are unresolved, never a crash', () => {
const dir = fixtureRepo();
fs.symlinkSync(path.join(dir, 'lib', 'gone.ts'), path.join(dir, 'lib', 'dangling.ts'));
const out = resolveReceipts(parseRealityLines('REALITY: numbers pass ok lib/a.ts:0 lib/a.ts:3-1 lib/dangling.ts:1 lib/a.ts/x.ts:1 lib/a.ts:1-3'), { roots: [dir] });
expect(out.map(r => r.outcome)).toEqual(['unresolved', 'unresolved', 'unresolved', 'unresolved', 'resolved']);
});
test('a commit sha in the repository resolves; an absent one is unverifiable', () => {
const dir = fixtureRepo();
const git = (...a: string[]) => spawnSync('git', a, { cwd: dir, encoding: 'utf8', timeout: 10_000 });
git('add', '.');
git('-c', 'user.name=t', '-c', 'user.email=t@t', 'commit', '-qm', 'seed');
const sha = git('rev-parse', 'HEAD').stdout.trim();
const out = resolveReceipts(parseRealityLines(`REALITY: numbers pass ok ${sha.slice(0, 10)} abcdef1234`), { roots: [dir] });
expect(out.map(r => r.outcome)).toEqual(['resolved', 'unverifiable']);
});
});