mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-04 07:47:27 +01:00
v1.92.82.0 fix(ship): verify re-derives each merged finding from the reader's output (GST-541) (#3717)
* fix(ship): verify re-derives each merged row by index (GST-541) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): verify fails closed on a missing output, a forged merged copy and uncollected rows (GST-541 review) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): a missing recorded output file is dirty even with no findings (GST-541 fix-check) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): a reply collect refused as no review keeps its NOT_REVIEWED row (Rule 25 with #3701) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): only a reply refused for its tree skips verify's count check (GST-541 fix-check) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): verify re-derives a tree refusal from the reply before it skips the count check or voids an output (GST-541 fix-check 2) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): an output with no readable recorded file is dirty even with no ids (GST-541 fix-check 3) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): verify accepts an output as a review only when its reply names the render's tree (GST-541 red team) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review-ledger): judge binding compares real paths, so a run reached through a symlink verifies clean Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
1022f4fc51
commit
26539fb0d9
7 files changed
+213
-17
No files matched your search
@@ -1,5 +1,19 @@
|
||||
# Changelog
|
||||
|
||||
## [1.92.82.0] - 2026-11-02
|
||||
|
||||
**`gstack-review-dispatch verify` now catches a merged findings file edited after collect.** Verify used to compare only output hashes and line counts. So a `merged.jsonl` row whose label was lowered, or whose text was replaced with another line's summary, passed with `VERIFY: clean` once the judge agreed with the edited row. Verify now re-parses each reader's unchanged output and checks every merged row against the finding at its index, label and raw text byte for byte. A mismatch prints `merged_line_differs` on the OUTPUT and JUDGE rows, and verify is dirty. A missing output file or an output event with no file recorded (even for a reply with no findings), a judgment run on a copy of the merged file, merged rows that collect never wrote, and an output whose finding count no longer matches its recorded ids are dirty too. A reply refused for naming the wrong checkout is still allowed to hold findings collect never merged, but only when verify re-checks its tree line and finds it wrong.
|
||||
|
||||
### Known
|
||||
- Deleting a whole output event together with its merged rows still verifies clean, because the ledger keeps no anchor for it. It is listed for the structured-findings plan.
|
||||
|
||||
### Tests
|
||||
- `test/review-dispatch.test.ts`: a forged label, a summary copied onto a blocker's id, a deleted output file, a judgment on a forged copy and an extra merged row all verify dirty. Each verified clean at main c830c9b.
|
||||
- `test/review-dispatch.test.ts`: an output event edited to look like a wrong-tree refusal (no ids, `review: false`, `tree_ok: false`) verifies dirty, even after a re-sent brief is answered, and so does one edited to drop its file and ids. A real wrong-tree reply stays `NOT_REVIEWED` until a re-sent brief gets a review, then verifies clean.
|
||||
|
||||
### Docs
|
||||
- `docs/review-policy.md` (verify) describes the per-row check.
|
||||
|
||||
## [1.92.81.0] - 2026-11-02
|
||||
|
||||
**`gstack-ship-release finish` now accepts the owner's edit after a fix-check instead of asking for another review round (GST-965).** On gbrain #8425, the fix-check found a blocker, the owner's pre-agreed fallback was applied and "then ship" was granted. Finish still classed the edit as behavior and stopped with `insert 5–11.5 → 12–14 → 16`, a third round the R51 cap forbids, so /ship could not get through Step 16 without running its stages by hand.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# gstack digest v1.92.81.0 — regenerate/re-copy after upgrading gstack
|
||||
# gstack digest v1.92.82.0 — regenerate/re-copy after upgrading gstack
|
||||
|
||||
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
|
||||
for agent hosts without a full skill install. The full skills add workflows,
|
||||
|
||||
+12
-1
@@ -579,7 +579,18 @@ ones, and the arm lost blockers. `bin/gstack-review-dispatch` closes both ends:
|
||||
missing coverage leaves the review incomplete. A mismatched send re-sent
|
||||
byte-exact before anything was collected is void and counted (`voided=N` on the
|
||||
verdict line); one that produced an output stays `MISMATCH`. Then output hashes
|
||||
against the merged lines they produced, and the judgment rerun against the
|
||||
against the merged lines they produced, and each merged line against the finding collect
|
||||
parses at that id's index from the unchanged output file, label and raw text byte for byte
|
||||
(`merged_line_differs` on the OUTPUT and JUDGE rows otherwise, GST-541), so an edited
|
||||
merged.jsonl cannot lower a finding or swap its text. A recorded output file that is missing
|
||||
(`output_file_missing`), a judgment run on a merged file collect did not write
|
||||
(`judge_merged_not_collected`) and merged rows collect never recorded
|
||||
(`merged_rows_not_collected`) are dirty too, and so is an output whose re-parsed findings
|
||||
outnumber or undercount the ids collect recorded (`merged_count_differs`). The one exception
|
||||
is a reply collect refused for its tree: verify re-checks the reply's `REVIEWED_TREE` line
|
||||
against the render's tree, and such an output stays `NOT_REVIEWED` until a re-sent brief is
|
||||
answered by a reply that passes the same check. Deleting a whole output event with its rows is
|
||||
not caught yet, because the ledger has no anchor for it. The judgment is then rerun against the
|
||||
recorded judged set, covering every collected id. A ledger from a run that kept no
|
||||
prompts prints `VERIFY: no brief records` (exit 3), never clean.
|
||||
6. `pass --base <b> --out <dir> --pass <id> -- <plan flags>` runs one pass from a
|
||||
|
||||
+57
-13
@@ -16,7 +16,7 @@
|
||||
*/
|
||||
import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import { isReviewedTreeLine, promptHash, type TreeCheck } from './review-brief';
|
||||
import { checkReviewedTree, isReviewedTreeLine, promptHash, type TreeCheck } from './review-brief';
|
||||
|
||||
export interface LedgerEvent { event: 'rendered' | 'sent' | 'output' | 'judgment' | 'judge' | 'program'; [k: string]: unknown }
|
||||
export const readerKey = (role: string, name?: string) => (name ? `${role}/${name}` : role);
|
||||
@@ -236,6 +236,21 @@ export function recordJudgment(ledger: string, merged: MergedLine[], judged: Jud
|
||||
return res;
|
||||
}
|
||||
|
||||
/** A recorded output's file text while it still hashes to what collect recorded. */
|
||||
function unchangedOutput(o: LedgerEvent): string | undefined {
|
||||
try {
|
||||
const text = typeof o.file === 'string' ? fs.readFileSync(o.file, 'utf8') : undefined;
|
||||
return text !== undefined && promptHash(text) === o.sha256 ? text : undefined;
|
||||
} catch { return undefined; }
|
||||
}
|
||||
|
||||
/** Whether a reply names the tree its render recorded (GST-482), checked on the reply itself, never on an output event's editable fields. */
|
||||
const treeHolds = (render: LedgerEvent | undefined, reply: string | undefined) => !render || !('tree' in render)
|
||||
|| (reply !== undefined && checkReviewedTree(reply, typeof render.tree === 'string' ? render.tree : null).ok);
|
||||
|
||||
/** One file however its path is spelled: collect and judge may reach a run directory through a symlink. */
|
||||
const realPath = (f: string) => { try { return fs.realpathSync(f); } catch { return path.resolve(f); } };
|
||||
|
||||
/**
|
||||
* The blocking findings a run's ledger holds (GST-951): every collected line whose label ranks CRITICAL, raised to its judged
|
||||
* severity when the newest judge accepted a judged set that is unchanged since; lead judgment's additions count and a folded
|
||||
@@ -284,15 +299,16 @@ export function ledgerBlockers(ledger: string): LedgerBlocker[] {
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a recorded output was a review: it has findings, or its file (unchanged since collect) carries the reader's clean marker, so
|
||||
* a ledger written before collect recorded the field is judged by the same rule; with no readable file, collect's own review field.
|
||||
* Whether a recorded output was a review: its unchanged reply names its render's tree, and it has findings or carries the reader's
|
||||
* clean marker, so a ledger written before collect recorded the field is judged by the same rule; with no readable file, collect's own
|
||||
* review field.
|
||||
*/
|
||||
function reviewed(o: LedgerEvent): boolean {
|
||||
function reviewed(o: LedgerEvent, render?: LedgerEvent): boolean {
|
||||
if (o.tree_ok === false || unavailableOutput(o)) return false;
|
||||
const text = unchangedOutput(o);
|
||||
if (!treeHolds(render, text)) return false;
|
||||
if (Array.isArray(o.ids) && o.ids.length) return true;
|
||||
let text: string | undefined;
|
||||
try { text = typeof o.file === 'string' ? fs.readFileSync(o.file, 'utf8') : undefined; } catch { /* unreadable */ }
|
||||
return text !== undefined && promptHash(text) === o.sha256 ? hasCleanMarker(String(o.reader), text) : o.review === true;
|
||||
return text !== undefined ? hasCleanMarker(String(o.reader), text) : o.review === true;
|
||||
}
|
||||
|
||||
/** Every id collect recorded, in order. */
|
||||
@@ -314,6 +330,7 @@ export function verify(ledger: string): { rows: VerifyRow[]; clean: boolean; bri
|
||||
return { at, next, reader, label: nth > 1 ? `${reader}@${nth}` : reader };
|
||||
});
|
||||
let voided = 0;
|
||||
const differs: string[] = [];
|
||||
for (const p of episodes) {
|
||||
const r = events[p.at]!;
|
||||
const inside = events.map((e, i) => ({ e, i })).filter(({ e, i }) => i > p.at && i < p.next && e.reader === p.reader);
|
||||
@@ -331,7 +348,7 @@ export function verify(ledger: string): { rows: VerifyRow[]; clean: boolean; bri
|
||||
voided += epVoided;
|
||||
const lastMatch = [...sent].reverse().find(({ e }) => e.sha256 === r.sha256);
|
||||
const outputs = lastMatch ? inside.filter(({ e, i }) => e.event === 'output' && i > lastMatch.i) : [];
|
||||
const collected = outputs.some(({ e }) => reviewed(e));
|
||||
const collected = outputs.some(({ e }) => reviewed(e, r));
|
||||
const unavailable = !collected && outputs.some(({ e }) => unavailableOutput(e));
|
||||
const status = !sent.length ? 'NOT_SENT' : mismatched ? 'MISMATCH' : collected ? 'MATCH' : unavailable ? 'UNAVAILABLE' : 'NOT_COLLECTED';
|
||||
const tampered = fileNow && fileNow !== r.sha256 ? ' file_changed_since_render' : '';
|
||||
@@ -346,20 +363,38 @@ export function verify(ledger: string): { rows: VerifyRow[]; clean: boolean; bri
|
||||
const lastSent = p ? events.slice(p.at + 1, i).filter(e => e.event === 'sent' && e.reader === reader).at(-1) : undefined;
|
||||
if (!p) { status = 'MISMATCH'; notes.push('brief_not_rendered'); }
|
||||
else if (!lastSent || lastSent.sha256 !== events[p.at]!.sha256) { status = 'MISMATCH'; notes.push('brief_not_sent_verbatim'); }
|
||||
if (typeof o.file === 'string' && fs.existsSync(o.file) && promptHash(fs.readFileSync(o.file, 'utf8')) !== o.sha256) { status = 'MISMATCH'; notes.push('output_file_changed'); }
|
||||
let text: string | undefined;
|
||||
try { text = typeof o.file === 'string' ? fs.readFileSync(o.file, 'utf8') : undefined; } catch { /* missing or unreadable */ }
|
||||
const render = p ? events[p.at]! : undefined;
|
||||
if (text !== undefined && promptHash(text) !== o.sha256) { status = 'MISMATCH'; notes.push('output_file_changed'); }
|
||||
const ids = Array.isArray(o.ids) ? (o.ids as string[]) : [];
|
||||
let lines: MergedLine[] = [];
|
||||
try { lines = typeof o.merged === 'string' ? readJsonl<MergedLine>(o.merged).filter(l => ids.includes(l.id)) : []; } catch { notes.push('merged_unreadable'); status = 'MISMATCH'; }
|
||||
if (lines.length !== ids.length) { status = 'MISMATCH'; notes.push(`merged_lines=${lines.length}/${ids.length}`); }
|
||||
if (lines.some(l => l.output_sha256 !== o.sha256)) { status = 'MISMATCH'; notes.push('merged_line_from_other_output'); }
|
||||
if (status === 'MATCH' && !reviewed(o)) {
|
||||
// Each merged row must be the finding collect parses at that id's index from the unchanged output, label and raw text byte for byte.
|
||||
if (text === undefined) {
|
||||
status = 'MISMATCH';
|
||||
notes.push(typeof o.file === 'string' ? 'output_file_missing' : 'output_file_unrecorded');
|
||||
differs.push(...ids);
|
||||
} else if (promptHash(text) === o.sha256) {
|
||||
const parsed = parseFindings(reader, text);
|
||||
const off = ids.filter((id, k) => { const m = lines.find(l => l.id === id); return !m || parsed[k]?.label !== m.label || parsed[k]?.raw !== m.raw; });
|
||||
if (unavailableOutput(o) && parsed.length) { status = 'MISMATCH'; notes.push('unavailable_output_has_findings'); }
|
||||
else if (!unavailableOutput(o)) {
|
||||
if (off.length) { status = 'MISMATCH'; notes.push(`merged_line_differs=${JSON.stringify(off)}`); differs.push(...off); }
|
||||
// A reply collect refused for its tree (GST-482) records no ids whatever it holds; the refusal is re-derived from the reply.
|
||||
if (parsed.length !== ids.length && !(o.tree_ok === false && !ids.length && !treeHolds(render, text))) { status = 'MISMATCH'; notes.push(`merged_count_differs=${parsed.length}/${ids.length}`); }
|
||||
}
|
||||
}
|
||||
if (status === 'MATCH' && !reviewed(o, render)) {
|
||||
// A non-review the same dispatch later replaced with a review (the brief re-sent verbatim, then answered) is void, counted like a voided send.
|
||||
const later = p ? events.slice(i + 1, p.next) : [];
|
||||
const review = later.findIndex(e => e.event === 'output' && e.reader === reader && reviewed(e));
|
||||
const review = later.findIndex(e => e.event === 'output' && e.reader === reader && reviewed(e, render));
|
||||
const replaced = review > 0 && later.slice(0, review).some(e => e.event === 'sent' && e.reader === reader && e.sha256 === events[p!.at]!.sha256);
|
||||
if (unavailableOutput(o)) { status = 'UNAVAILABLE'; notes.push(`reason=${JSON.stringify(o.unavailable)}`); }
|
||||
else if (replaced) { status = 'VOIDED'; voided++; notes.push('replaced_by_a_review'); }
|
||||
else if (o.tree_ok === false) { status = 'NOT_REVIEWED'; notes.push(`wrong_tree reported=${JSON.stringify(o.reported_tree)} expected=${JSON.stringify(o.tree)}`); }
|
||||
else if (o.tree_ok === false || !treeHolds(render, text)) { status = 'NOT_REVIEWED'; notes.push(`wrong_tree reported=${JSON.stringify(o.reported_tree)} expected=${JSON.stringify(o.tree)}`); }
|
||||
else { status = 'NOT_REVIEWED'; notes.push('no_finding_no_clean_marker'); }
|
||||
}
|
||||
rows.push({ kind: 'OUTPUT', reader: p?.label ?? reader, status, detail: `sha=${short(o.sha256)} findings=${ids.length}${notes.length ? ' ' + notes.join(' ') : ''}` });
|
||||
@@ -373,13 +408,22 @@ export function verify(ledger: string): { rows: VerifyRow[]; clean: boolean; bri
|
||||
const judgedIds = new Set(Array.isArray(v.merged_ids) ? (v.merged_ids as string[]) : []);
|
||||
const missing = collectedIds(events.slice(0, lastJudge)).filter(id => !judgedIds.has(id));
|
||||
if (missing.length) notes.push(`merged_ids_missing=${missing.join(',')}`);
|
||||
// The judged set must come from the merged file collect wrote, holding only rows collect recorded.
|
||||
const outputs = events.slice(0, lastJudge).filter(e => e.event === 'output' && Array.isArray(e.ids) && e.ids.length);
|
||||
if (outputs.some(e => typeof e.merged !== 'string' || realPath(e.merged) !== realPath(String(v.merged)))) notes.push('judge_merged_not_collected');
|
||||
let rerun: Judgment | null = null;
|
||||
try {
|
||||
const judgedText = fs.readFileSync(String(v.judged), 'utf8');
|
||||
if (promptHash(judgedText) !== v.judged_sha256) notes.push('judged_changed_since_judge');
|
||||
rerun = judge(readJsonl<MergedLine>(String(v.merged)), readJsonl<JudgedLine>(String(v.judged)));
|
||||
const merged = readJsonl<MergedLine>(String(v.merged));
|
||||
const collected = new Set(collectedIds(events.slice(0, lastJudge)));
|
||||
const extra = merged.filter(m => !collected.has(m.id)).map(m => m.id);
|
||||
if (extra.length) notes.push(`merged_rows_not_collected=${JSON.stringify(extra)}`);
|
||||
rerun = judge(merged, readJsonl<JudgedLine>(String(v.judged)));
|
||||
} catch { notes.push('judged_or_merged_unreadable'); }
|
||||
if (rerun && !rerun.ok) notes.push(`rerun_rejects dropped=${JSON.stringify(rerun.dropped)} downgraded=${JSON.stringify(rerun.downgraded)}`);
|
||||
const forged = differs.filter(id => judgedIds.has(id));
|
||||
if (forged.length) notes.push(`merged_line_differs=${JSON.stringify(forged)}`);
|
||||
const status = !v.ok ? 'REJECTED' : notes.length ? 'MISMATCH' : 'OK';
|
||||
rows.push({ kind: 'JUDGE', reader: '-', status, detail: `dropped=${JSON.stringify(v.dropped)} downgraded=${JSON.stringify(v.downgraded)} added=${v.added} promoted=${v.promoted} folded=${v.folded}${notes.length ? ' ' + notes.join(' ') : ''}` });
|
||||
} else if (events.some(e => e.event === 'output')) rows.push({ kind: 'JUDGE', reader: '-', status: 'MISSING', detail: 'outputs were collected but no judged set was checked' });
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gstack",
|
||||
"version": "1.92.81",
|
||||
"version": "1.92.82",
|
||||
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
||||
"license": "MIT",
|
||||
"type": "module",
|
||||
|
||||
@@ -683,6 +683,133 @@ describe('gstack-review-dispatch sent, collect, judge and verify: nothing change
|
||||
const v = run(['verify', '--ledger', ledger]);
|
||||
expect([v.status, v.stdout.trim()]).toEqual([3, "VERIFY: no brief records (this run's ledger kept no prompts; dispatched-vs-rendered cannot be checked)"]);
|
||||
});
|
||||
|
||||
// Value: protects=verify re-derives every merged row from the reader's unchanged output by its index, so a merged.jsonl edited
|
||||
// after collect cannot lower a finding or swap a blocker's text for a summary and still verify clean; fails_when=a forged label
|
||||
// or a copied raw line on a real blocker's id verifies clean (GST-541: both did at main c830c9b), or a clean run collected through a
|
||||
// symlinked run directory and judged there with relative paths verifies dirty (GST-541 ship2-p11); why_new=verify compared only
|
||||
// output hashes and line counts; seam=verify; no_claim=that an output collected without a recorded file can be re-derived
|
||||
test('verify re-derives each merged row by index: a forged label or a summary copied onto a blocker id is dirty', () => {
|
||||
const blocker = '{"severity":"CRITICAL","path":"log.ts","line":3,"category":"security","summary":"bearer token written to a public log","specialist":"testing"}';
|
||||
const summary = '{"severity":"INFORMATIONAL","path":"a.ts","line":9,"category":"style","summary":"I found nothing CRITICAL.","specialist":"testing"}';
|
||||
const setup = () => {
|
||||
const p = pass();
|
||||
run(['sent', '--ledger', p.ledger, '--role', 'specialist', '--name', 'testing', '--sent', p.file]);
|
||||
const out = p.w('spec.out', blocker + '\n' + summary + '\n');
|
||||
expect(run(['collect', '--ledger', p.ledger, '--role', 'specialist', '--name', 'testing', '--output', out, '--merged', p.merged]).status).toBe(0);
|
||||
return p;
|
||||
};
|
||||
const judgeAll = (p: ReturnType<typeof pass>, severity: string) => run(['judge', '--ledger', p.ledger, '--merged', p.merged, '--judged', p.w('judged.jsonl',
|
||||
readJsonl<MergedLine>(p.merged).map(m => JSON.stringify({ id: m.id, severity: m.id.endsWith('#1') ? severity : 'INFORMATIONAL' })).join('\n') + '\n')]);
|
||||
const clean = setup();
|
||||
expect(judgeAll(clean, 'CRITICAL').status).toBe(0);
|
||||
expect(run(['verify', '--ledger', clean.ledger]).stdout).toContain('VERIFY: clean');
|
||||
const forge = (p: ReturnType<typeof pass>, edit: (rows: MergedLine[]) => void) => {
|
||||
const rows = readJsonl<MergedLine>(p.merged);
|
||||
edit(rows);
|
||||
fs.writeFileSync(p.merged, rows.map(r => JSON.stringify(r)).join('\n') + '\n');
|
||||
expect(judgeAll(p, 'INFORMATIONAL').stdout.trim()).toBe('JUDGMENT: ok added=0 promoted=0 folded=0');
|
||||
return run(['verify', '--ledger', p.ledger]);
|
||||
};
|
||||
const label = forge(setup(), rows => { rows[0]!.label = 'INFORMATIONAL'; });
|
||||
expect([label.status, label.stdout]).toEqual([1, expect.stringContaining('merged_line_differs=["specialist/testing#1"]')]);
|
||||
const copied = forge(setup(), rows => { rows[0]!.label = rows[1]!.label; rows[0]!.raw = rows[1]!.raw; });
|
||||
expect([copied.status, copied.stdout]).toEqual([1, expect.stringContaining('merged_line_differs=["specialist/testing#1"]')]);
|
||||
expect(copied.stdout).toMatch(/^JUDGE - MISMATCH .*merged_line_differs/m);
|
||||
const gone = setup();
|
||||
fs.rmSync(path.join(gone.out, 'spec.out'));
|
||||
const missing = forge(gone, rows => { rows[0]!.label = 'INFORMATIONAL'; });
|
||||
expect([missing.status, missing.stdout.includes('output_file_missing'), missing.stdout.includes('merged_line_differs=["specialist/testing#1","specialist/testing#2"]')]).toEqual([1, true, true]);
|
||||
const copy = setup();
|
||||
const forgedCopy = copy.w('merged2.jsonl', readJsonl<MergedLine>(copy.merged).map((r, k) => JSON.stringify(k ? r : { ...r, label: 'INFORMATIONAL' })).join('\n') + '\n');
|
||||
expect(run(['judge', '--ledger', copy.ledger, '--merged', forgedCopy, '--judged', copy.w('judged.jsonl', readJsonl<MergedLine>(copy.merged).map(m => JSON.stringify({ id: m.id, severity: 'INFORMATIONAL' })).join('\n') + '\n')]).status).toBe(0);
|
||||
const viaCopy = run(['verify', '--ledger', copy.ledger]);
|
||||
expect([viaCopy.status, viaCopy.stdout]).toEqual([1, expect.stringMatching(/^JUDGE - MISMATCH .*judge_merged_not_collected/m)]);
|
||||
const relinked = pass();
|
||||
const linkDir = path.join(tmp(), 'run');
|
||||
fs.symlinkSync(relinked.out, linkDir);
|
||||
const ledgerViaLink = path.join(linkDir, 'review-ledger.jsonl');
|
||||
run(['sent', '--ledger', ledgerViaLink, '--role', 'specialist', '--name', 'testing', '--sent', relinked.file]);
|
||||
const linkOut = relinked.w('spec.out', blocker + '\n' + summary + '\n');
|
||||
expect(run(['collect', '--ledger', ledgerViaLink, '--role', 'specialist', '--name', 'testing', '--output', path.join(linkDir, path.basename(linkOut)), '--merged', path.join(linkDir, 'merged.jsonl')]).status).toBe(0);
|
||||
relinked.w('judged.jsonl', readJsonl<MergedLine>(relinked.merged).map(m => JSON.stringify({ id: m.id, severity: m.label })).join('\n') + '\n');
|
||||
expect(run(['judge', '--ledger', 'review-ledger.jsonl', '--merged', 'merged.jsonl', '--judged', 'judged.jsonl'], linkDir).status).toBe(0);
|
||||
const linked = run(['verify', '--ledger', ledgerViaLink]);
|
||||
expect([linked.status, linked.stdout.includes('judge_merged_not_collected'), linked.stdout.includes('VERIFY: clean')]).toEqual([0, false, true]);
|
||||
const quiet = pass();
|
||||
run(['sent', '--ledger', quiet.ledger, '--role', 'specialist', '--name', 'testing', '--sent', quiet.file]);
|
||||
const none = quiet.w('none.out', 'NO FINDINGS\n');
|
||||
expect(run(['collect', '--ledger', quiet.ledger, '--role', 'specialist', '--name', 'testing', '--output', none, '--merged', quiet.merged]).status).toBe(0);
|
||||
fs.writeFileSync(quiet.merged, '');
|
||||
expect(run(['judge', '--ledger', quiet.ledger, '--merged', quiet.merged, '--judged', quiet.w('judged.jsonl', '')]).status).toBe(0);
|
||||
fs.rmSync(none);
|
||||
const noneGone = run(['verify', '--ledger', quiet.ledger]);
|
||||
expect([noneGone.status, noneGone.stdout.includes('output_file_missing')]).toEqual([1, true]);
|
||||
const extra = forge(setup(), rows => { rows[0]!.label = 'INFORMATIONAL'; rows.push({ ...rows[1]!, id: 'specialist/testing#9' }); });
|
||||
expect([extra.status, extra.stdout.includes('merged_rows_not_collected=["specialist/testing#9"]')]).toEqual([1, true]);
|
||||
});
|
||||
// Value: protects=an output event edited to read like a refused reply (no ids, review false, tree_ok false) or to drop its file
|
||||
// cannot drop its findings and still verify clean, while a reply collect really refused for its tree stays NOT_REVIEWED until a re-sent brief is answered;
|
||||
// fails_when=verify skips the count check on the editable review or tree_ok field, so a clean marker keeps the output MATCH or a
|
||||
// later review voids it, or an output with no recorded file passes on its review field (fix-checks at 721539c, 56ed6e3 and 50dc73c: VERIFY: clean); why_new=the GST-482 count exemption trusted collect's
|
||||
// fields; seam=verify; no_claim=that a deleted output event is caught (#3719)
|
||||
test('an output event edited to look refused, or a refused one edited to look accepted, is dirty; a real wrong-tree reply is NOT_REVIEWED, then voided by a review', () => {
|
||||
const head = spawnSync('git', ['rev-parse', 'HEAD'], { cwd: ROOT, encoding: 'utf8', timeout: 30_000 }).stdout.trim();
|
||||
const findings = '1. **CRITICAL**: auth.ts:3 leaks a token\nBlocks: yes\n\nRecommendation: request changes\n';
|
||||
const voice = (reply: string) => {
|
||||
const out = tmp();
|
||||
const ledger = path.join(out, 'review-ledger.jsonl');
|
||||
const merged = path.join(out, 'merged.jsonl');
|
||||
const brief = field(run(['render', '--role', 'outside-voice', '--out', out, '--diff-base', head]).stdout, 'file');
|
||||
const send = () => run(['sent', '--ledger', ledger, '--role', 'outside-voice', '--sent', brief]);
|
||||
let n = 0;
|
||||
const take = (text: string) => { const f = path.join(out, `ov${++n}.out`); fs.writeFileSync(f, text); return run(['collect', '--ledger', ledger, '--role', 'outside-voice', '--output', f, '--merged', merged]); };
|
||||
send();
|
||||
const c = take(reply);
|
||||
const forge = (edit: Record<string, unknown>) => {
|
||||
fs.writeFileSync(ledger, readJsonl<Record<string, unknown>>(ledger).map(e => JSON.stringify(e.event === 'output' ? { ...e, ...edit } : e)).join('\n') + '\n');
|
||||
fs.writeFileSync(merged, '');
|
||||
};
|
||||
const verify = () => {
|
||||
fs.writeFileSync(path.join(out, 'judged.jsonl'), '');
|
||||
expect(run(['judge', '--ledger', ledger, '--merged', merged, '--judged', path.join(out, 'judged.jsonl')]).status).toBe(0);
|
||||
return run(['verify', '--ledger', ledger]);
|
||||
};
|
||||
return { c, send, take, forge, verify };
|
||||
};
|
||||
const noIds = voice(findings);
|
||||
expect(noIds.c.status).toBe(0);
|
||||
noIds.forge({ ids: [], review: false });
|
||||
const v = noIds.verify();
|
||||
expect([v.status, v.stdout.includes('merged_count_differs=1/0'), v.stdout.includes('VERIFY: dirty')]).toEqual([1, true, true]);
|
||||
const unfiled = voice(findings);
|
||||
unfiled.forge({ file: null, ids: [] });
|
||||
const u = unfiled.verify();
|
||||
expect([u.status, u.stdout.includes('output_file_unrecorded'), u.stdout.includes('VERIFY: dirty')]).toEqual([1, true, true]);
|
||||
const flagged = voice(findings);
|
||||
flagged.forge({ ids: [], review: false, tree_ok: false });
|
||||
flagged.send();
|
||||
expect(flagged.take('Recommendation: approve\n').status).toBe(0);
|
||||
const f = flagged.verify();
|
||||
expect([f.status, f.stdout.includes('merged_count_differs=1/0'), f.stdout.includes('VERIFY: dirty')]).toEqual([1, true, true]);
|
||||
const wrong = voice(`REVIEWED_TREE: ${'b'.repeat(40)} /elsewhere\n` + findings);
|
||||
expect([wrong.c.status, wrong.c.stdout.startsWith('COLLECT_WRONG_TREE: reader=outside-voice')]).toEqual([1, true]);
|
||||
const w = wrong.verify();
|
||||
expect([w.status, /^OUTPUT outside-voice NOT_REVIEWED .*wrong_tree/m.test(w.stdout), w.stdout.includes('merged_count_differs')]).toEqual([1, true, false]);
|
||||
wrong.send();
|
||||
expect(wrong.take('Recommendation: approve\n').status).toBe(0);
|
||||
const r = wrong.verify();
|
||||
expect([r.status, r.stdout.includes('replaced_by_a_review'), r.stdout.includes('VERIFY: clean briefs=1 outputs=2 voided=1')]).toEqual([0, true, true]);
|
||||
const elsewhere = pass();
|
||||
run(['sent', '--ledger', elsewhere.ledger, '--role', 'specialist', '--name', 'testing', '--sent', elsewhere.file]);
|
||||
const refused = run(['collect', '--ledger', elsewhere.ledger, '--role', 'specialist', '--name', 'testing', '--output', elsewhere.w('else.out', `REVIEWED_TREE: ${'b'.repeat(40)} /elsewhere\nNO FINDINGS\n`), '--merged', elsewhere.merged]);
|
||||
expect(refused.stdout.startsWith('COLLECT_WRONG_TREE: reader=specialist/testing')).toBe(true);
|
||||
fs.writeFileSync(elsewhere.ledger, readJsonl<Record<string, unknown>>(elsewhere.ledger).map(e => JSON.stringify(e.event === 'output' ? { ...e, tree_ok: true, review: true, reported_tree: e.tree } : e)).join('\n') + '\n');
|
||||
fs.writeFileSync(elsewhere.merged, '');
|
||||
expect(run(['judge', '--ledger', elsewhere.ledger, '--merged', elsewhere.merged, '--judged', elsewhere.w('judged.jsonl', '')]).status).toBe(0);
|
||||
const e = run(['verify', '--ledger', elsewhere.ledger]);
|
||||
expect([e.status, /^OUTPUT specialist\/testing NOT_REVIEWED .*wrong_tree/m.test(e.stdout), e.stdout.includes('VERIFY: dirty')]).toEqual([1, true, true]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the workspace paragraph (GST-482)', () => {
|
||||
|
||||
Reference in new issue
Block a user