Files
gstack/test/design-approval-flow.test.ts
Garry TanandMatt Van Horn fe6d1ae62a v1.91.17.0 fix: mvanhorn wave (checkpoint provenance, never-overwrite design images) (#3022)
* fix: mark Remaining Work provenance so restore verifies guessed steps first (#3004)

/context-save now starts every Remaining Work item with `Open.` and ends it
with how this session knows it: `(path run)` with the observed outcome,
`(path read)` or `(path assumed)` for concrete paths, and
`(target state checked)`, `(code read)` or `(path assumed)` for writing
steps. /context-restore keeps saved order and original text but splits the
list into Next steps and Verify first (assumed, code-read, failed runs and
unmarked concrete or writing items), prints a banner for pre-marker
checkpoints, and option A walks saved order so an unverified first item is
verified before any later step runs.

Adds a prompt-byte contract test for the marker wire format and a periodic
behavior E2E (context-restore-provenance-order) that seeds an assumed item
ahead of a run item plus a code-read write.

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>

* fix(design): never overwrite a paid design image (#1529)

Every image the API returns now goes through one persistence path
(design/src/persist.ts): mkdir, exclusive `wx` claim that bumps
x.png -> x-2.png ... x-999.png, write, fsync, close. A failure after the
bytes arrived removes the claimed file and keeps a 0600 recovery copy in
the temp dir; empty image data is a failure, never a 0-byte PNG. The API
call is the only thing inside each retry loop, so a local save failure
never buys a second image (variants, iterate fallback, generate --retry).

generate, iterate, evolve and variants (both branches) print the same run
fields (requested, saved, selected, failures, recovered) on every exit,
with exit 0 / 2 (nothing saved) / 3 (stopped after saving). Each
generate --retry attempt is its own saved image listed in `attempts`.
compare accepts --images-file and JSON-array --images. The gallery lists
bumped names, orders history by mtime then board-images.json, and marks
approvals by approved_path. New bin/gstack-design-claim publishes staged
images without clobbering; bin/gstack-design-approved resolves an
approval record (legacy letter-only records still resolve; a missing
approved_path is an error, never a substitute).

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>

* fix(design skills): read the paths $D printed, never fixed variant names (#1529)

Every design flow now captures each generating $D command's JSON and exit
code without set -e, does round accounting before the first board, check
or inline view (stopping when nothing was saved), and carries the printed
paths forward. Boards are built from board-images.json via
compare --images-file after archiving any earlier feedback.json, and
approvals map the confirmed letter through that file and record
approved_path. design-shotgun subagents generate into a per-run mktemp
staging directory and publish with bin/gstack-design-claim instead of a
shared /tmp/variant-X.png plus cp. design-html and taste memory resolve
approvals with gstack-design-approved; design-review verifies against the
printed target path.

test/design-printed-paths.test.ts bans fixed variant names, variant globs,
variant-{letter} reads, cp/mv of $D output and literal check paths in the
generated skills and resolvers (one content-anchored allowlist entry: the
design-html discovery probe), and pins the accounting/board marker order.
test/design-approval-flow.test.ts runs the generated board and approval
blocks through a stubbed two-round flow.

* chore: release v1.91.17.0 (mvanhorn fix wave: checkpoint provenance, never-overwrite design images)

* fix(design): keep iterate sessions in the OS temp dir and normalize tripwire paths on Windows

* fix(setup-deploy): report failing status commands, specify merge-method, health and workflow checks

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
2026-10-03 16:13:45 -07:00

98 lines
5.4 KiB
TypeScript

/**
* Stubbed two-round design flow: round two lands at bumped names, the board
* block archives round one's Submit, approval maps the letter through this
* board's board-images.json, and a fresh session (relocated directory)
* resolves the approved image. Runs the generated SKILL.md bash blocks with
* the image API stubbed (no spend).
*/
import { afterEach, beforeEach, expect, test } from 'bun:test';
import { spawnSync } from 'node:child_process';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
const ROOT = path.resolve(import.meta.dir, '..');
const SKILL = fs.readFileSync(path.join(ROOT, 'plan-design-review/SKILL.md'), 'utf8');
function bashBlockAfter(marker: string): string {
const at = SKILL.indexOf(marker);
expect(at).toBeGreaterThan(-1);
return SKILL.slice(at).match(/```bash\n([\s\S]*?)```/)![1];
}
const BOARD = bashBlockAfter('<!-- design:board -->');
const APPROVAL = SKILL.slice(SKILL.indexOf('**Save the approved choice.**')).match(/```bash\n([\s\S]*?)```/)![1];
let dir: string;
let designDir: string;
let env: Record<string, string>;
beforeEach(() => {
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'design-approval-flow-'));
designDir = path.join(dir, 'state', 'designs', 'home-20261003');
const preload = path.join(dir, 'stub.ts');
fs.writeFileSync(preload, `let n = 0;
globalThis.fetch = (async () => Response.json({ id: "r", output: [{ type: "image_generation_call", result: Buffer.from("round image " + (n++)).toString("base64") }] })) as typeof fetch;
`);
const design = path.join(dir, 'design');
fs.writeFileSync(design, `#!/bin/sh
if [ "$1" = compare ]; then printf '%s\\n' "$*" > "$COMPARE_ARGS"; exit 0; fi
exec ${JSON.stringify(process.execPath)} --no-env-file --preload ${JSON.stringify(preload)} ${JSON.stringify(path.join(ROOT, 'design/src/cli.ts'))} "$@"
`, { mode: 0o755 });
env = {
PATH: process.env.PATH!, HOME: dir, GSTACK_HOME: path.join(dir, 'gstack'), TMPDIR: dir,
OPENAI_API_KEY: 'fixture-not-a-real-key', D: design, _DESIGN_DIR: designDir, COMPARE_ARGS: path.join(dir, 'compare-args'),
};
});
afterEach(() => { fs.rmSync(dir, { recursive: true, force: true }); });
const sh = (script: string) => spawnSync('bash', ['-c', script], { cwd: dir, env, encoding: 'utf8', timeout: 60_000 });
const variants = () => JSON.parse(sh('"$D" variants --brief "home page" --count 2 --output-dir "$_DESIGN_DIR/"').stdout);
const approve = (letter: string) => sh(APPROVAL.replaceAll('<VARIANT>', letter).replaceAll('<FEEDBACK>', 'calmer').replaceAll('<SCREEN>', 'home'));
// Value: protects=round two's approval comes from round two's board, never round one's stale feedback.json or the newest-file listing;
// fails_when=the board block stops archiving feedback.json or approval maps letters through the directory; why_new=eng feedback-binding; seam=none
test('round two board archives round one Submit and approval maps to the bumped image', () => {
const round1 = variants();
fs.writeFileSync(path.join(designDir, 'board-images.json'), JSON.stringify(round1.paths));
fs.writeFileSync(path.join(designDir, 'feedback.json'), JSON.stringify({ preferred: 'B', regenerated: false }));
const round2 = variants();
expect(round2.paths.map((p: string) => path.basename(p))).toEqual(['variant-A-2.png', 'variant-B-2.png']);
fs.writeFileSync(path.join(designDir, 'board-images.json'), JSON.stringify(round2.paths));
const board = sh(BOARD);
expect(board.status, board.stderr).toBe(0);
expect(fs.existsSync(path.join(designDir, 'feedback.json'))).toBe(false);
const archived = fs.readdirSync(designDir).filter(f => /^feedback-\d{8}T\d{6}Z\.json$/.test(f));
expect(archived).toHaveLength(1);
expect(fs.readFileSync(env.COMPARE_ARGS, 'utf8')).toContain(`--images-file ${path.join(designDir, 'board-images.json')}`);
const approved = approve('A');
expect(approved.status, approved.stderr).toBe(0);
expect(approved.stdout).toContain(`APPROVED_IMAGE: ${path.join(designDir, 'variant-A-2.png')}`);
const record = JSON.parse(fs.readFileSync(path.join(designDir, 'approved.json'), 'utf8'));
expect(record).toMatchObject({ approved_variant: 'A', approved_path: 'variant-A-2.png', screen: 'home', feedback: 'calmer' });
const missing = approve('C');
expect(missing.stdout).toContain('NO_BOARD_IMAGE');
expect(JSON.parse(fs.readFileSync(path.join(designDir, 'approved.json'), 'utf8')).approved_path).toBe('variant-A-2.png');
});
// Value: protects=a fresh session in a relocated state root opens round two's approved image, not variant-A.png;
// fails_when=approved_path is stored absolute or readers resolve by letter; why_new=DX relocation + CEO fresh-session recovery; seam=none
test('fresh session resolves the approved bumped image after relocation', () => {
variants();
const round2 = variants();
fs.writeFileSync(path.join(designDir, 'board-images.json'), JSON.stringify(round2.paths));
expect(approve('A').status).toBe(0);
const moved = path.join(dir, 'moved', 'home-20261003');
fs.cpSync(designDir, moved, { recursive: true });
fs.rmSync(path.join(dir, 'state'), { recursive: true });
const r = spawnSync(process.execPath, [path.join(ROOT, 'bin/gstack-design-approved'), path.join(moved, 'approved.json')], { encoding: 'utf8', timeout: 30_000 });
expect(r.status, r.stderr).toBe(0);
expect(r.stdout.trim()).toBe(path.join(moved, 'variant-A-2.png'));
expect(fs.readFileSync(r.stdout.trim(), 'utf8')).toBe('round image 0');
});