mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-01 22:45:02 +01:00
fix(ship): a label word used as a word is no finding, and a paragraph finding keeps its detail bullets (GST-435)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
caaa2a96b8
commit
4333632e2c
6 files changed
+141
-10
No files matched your search
@@ -1,5 +1,8 @@
|
||||
**A reviewer that says it could not review no longer counts as a clean review in /ship.** In the GSTA-196 replay a specialist replied in prose that the diff base did not exist and it was not returning NO FINDINGS. `collect` parsed that reply to zero findings, and `verify` marked the brief collected, so the review would have read clean from a reader that never saw the diff. What reviewers see is unchanged: every brief renders byte for byte as before.
|
||||
**/ship's review collection reads reviewer replies the way they were meant: a reviewer that says it could not review no longer counts as clean, and a closing "nothing here is CRITICAL" no longer counts as a CRITICAL finding.** In the GSTA-196 replay a specialist replied in prose that the diff base did not exist and it was not returning NO FINDINGS. `collect` parsed that reply to zero findings, and `verify` marked the brief collected, so the review would have read clean from a reader that never saw the diff. What reviewers see is unchanged: every brief renders byte for byte as before.
|
||||
|
||||
### Fixed
|
||||
- **`gstack-review-dispatch collect` needs the clean marker a reader's own brief asks for before it counts a reply with no findings.** That is `NO FINDINGS` alone on a line for the specialists, the batched specialist, the targeted reader, the Red Team and fix-check; the `Recommendation:` line for the outside voice; and either one for the native adversarial voice. A reply with no finding and no marker (a refusal, an empty reply, a reader that never got the diff) is recorded with `review: false`, prints `COLLECT_NOT_REVIEWED` and exits 1.
|
||||
- **`gstack-review-dispatch verify` reports that reader's brief `NOT_COLLECTED` and its output `NOT_REVIEWED`, so VERIFY reads dirty.** When the output file is still there and unchanged, verify checks the marker in it again, so a ledger written before this change is judged by the same rule. The GSTA-189 replay's combined ledger still verifies clean, with five `NO FINDINGS` specialists and one outside voice ending in `Recommendation:`.
|
||||
- **A severity word used as a word is no longer a finding.** A fix-check reader on gstack-evals#94 closed with "Neither finding is CRITICAL.", `collect` stored that line as a CRITICAL finding, and `judge` would not let the lead drop it. Now a label that is negated ("not CRITICAL", "no High findings"), used as a yardstick ("below CRITICAL"), quoted or listed as vocabulary ("Critical|High") is skipped, and so is a closing "Overall:" or "Summary:" line after the findings. A label the line opens with, a `Severity:` field and a table cell still count.
|
||||
- **A paragraph finding keeps its detail bullets.** The GSTA-196 native adversarial voice wrote four findings, each with unlabeled bullets and then a `Severity:` line. Every bullet closed the open finding, so each `Severity:` line opened a new one: 4 findings collected as 8. They now collect as 4, each at the stronger of its two labels.
|
||||
- **No blocker is lost.** Main's parser and this one were run over every recorded reader output: 25 raw outputs from the GSTA-189 and GSTA-196 replays and 2,172 finding records from arms C6, L6, A3668 (both suites), G1 and L. The 8 GSTA-196 lines merge into 4 findings, none at a weaker label. One INVESTIGATE finding, recorded in both arm G1 and arm L, loses a phantom CRITICAL ("no CRITICAL" inside its sentence), and another loses a phantom High ("resolves below High"). None of the 260 records matched to a judged blocker changes.
|
||||
@@ -473,7 +473,12 @@ ones, and the arm lost blockers. `bin/gstack-review-dispatch` closes both ends:
|
||||
second line for the same field starts another: `Severity` raises its label,
|
||||
`Blocks: yes` makes it CRITICAL and `Blocks: no` never lowers it; an inline
|
||||
`Blocks: yes` is CRITICAL too, and with no finding above it is one. The native
|
||||
adversarial voice's FIXABLE collects as CRITICAL (normalizer v2, GSTA-167). A phantom finding costs a fold; a lost one costs the gate.
|
||||
adversarial voice's FIXABLE collects as CRITICAL (normalizer v2, GSTA-167). A label word used as a word is not a label: negated
|
||||
within three words in its clause ("not CRITICAL", "Neither finding is CRITICAL", "no High findings"), a yardstick ("below
|
||||
CRITICAL", "worse than High"), quoted, or listed as vocabulary ("Critical|High"); a label the line opens with always counts, and a
|
||||
closing "Overall:", "Summary:" or "Verdict:" line after the findings starts none (GST-435). An unlabeled bullet under a paragraph
|
||||
finding whose Severity or Blocks line has not come yet is that finding's detail, so the field line after it lands on the finding.
|
||||
A phantom finding costs a fold; a lost one costs the gate.
|
||||
A reply with no finding counts as a review only with the clean marker its own brief asks for: `NO FINDINGS` alone on a
|
||||
line for the specialists, the batched specialist, the targeted reader, the Red Team and fix-check, the `Recommendation:`
|
||||
line for the outside voice, either for the native adversarial voice. Without one (a refusal, an empty reply, a reader
|
||||
|
||||
+32
-7
@@ -59,6 +59,27 @@ const stronger = (a: string, b: string) => ((rank(a) ?? 0) >= (rank(b) ?? 0) ? a
|
||||
/** A prose voice's `Blocks: yes` (GSTA-167 normalizer v2): the finding is CRITICAL whatever its label; `Blocks: no` never lowers one. */
|
||||
const BLOCKS_YES = /(?<![A-Za-z0-9])Blocks:?\**:?\s*\**\s*yes(?![A-Za-z0-9])/i;
|
||||
const ALL_LABELS = new RegExp(ANY_LABEL.source, 'g');
|
||||
/** The label the line opens with, after a list marker, heading mark, table bar, bracket, bold or quote: always a label. */
|
||||
const OPENS = /^(?:[-*+>]\s+|\d+[.)]\s+|#{1,6}\s*)?[\s[(*_`"'“‘|]*$/;
|
||||
/**
|
||||
* A label word used as a word, not as a label (GST-435): negated within three words in its clause ("not CRITICAL", "Neither finding is
|
||||
* CRITICAL", "no High findings"), used as a yardstick ("below CRITICAL", "worse than High"), quoted ("'High'", `CRITICAL`) or listed as
|
||||
* vocabulary ("Critical|High"). The clause runs back to the last sentence, colon, comma, dash or bracket.
|
||||
*/
|
||||
const NEGATED = /\b(?:not|no|neither|nor|none|never|nothing|without|zero|isn't|aren't|wasn't|weren't|doesn't|don't|didn't|cannot|can't)\b(?:\s+[\w'’-]+){0,3}?\s+[*_`"'“‘]*$/i;
|
||||
const YARDSTICK = /\b(?:below|under|than|short of|instead of|up to|at most)\s+[*_`"'“‘]*$/i;
|
||||
const QUOTES: Record<string, string> = { '"': '"', "'": "'", '`': '`', '“': '”', '‘': '’' };
|
||||
function aWord(t: string, at: number, len: number): boolean {
|
||||
const before = t.slice(0, at);
|
||||
if (OPENS.test(before)) return false;
|
||||
const after = t.slice(at + len);
|
||||
if (QUOTES[before.slice(-1)] !== undefined && after.startsWith(QUOTES[before.slice(-1)]!)) return true;
|
||||
if (/^\|[A-Za-z]/.test(after) || /[A-Za-z0-9]\|$/.test(before)) return true;
|
||||
const clause = before.slice(Math.max(...['.', ';', ':', '!', '?', ',', '—', '–', '(', ')'].map(c => before.lastIndexOf(c))) + 1);
|
||||
return NEGATED.test(' ' + clause) || YARDSTICK.test(' ' + clause);
|
||||
}
|
||||
/** A closing summary line ("Overall: …", "In summary, …", "Verdict: …") after the findings restates them; it starts no finding. */
|
||||
const SUMMARY = /^(?:\*\*)?(?:in\s+)?(?:summary|overall|verdict|conclusion|bottom line)\b/i;
|
||||
/** A file:line locus ("auth.ts:9"): prose naming one after a finding's fields starts the next finding. */
|
||||
const LOCUS = /[\w./-]+\.\w+:\d+/;
|
||||
/** A line that is only a finding's `Severity` or `Blocks` field ("Severity: High", "- Blocks: yes", "Severity High"). */
|
||||
@@ -72,11 +93,13 @@ export interface MergedLine { id: string; reader: string; label: string; raw: st
|
||||
* of the two labels). A `Severity` or `Blocks` field line continues the open finding, once per field: `Severity` raises its label,
|
||||
* `Blocks: yes` makes it CRITICAL, `Blocks: no` never lowers it; with no finding open it is one, carrying the paragraph's subject.
|
||||
* A list item, a heading, a bold or post-blank paragraph at the margin, or a line naming a file:line after the open finding's fields
|
||||
* starts the next finding. An inline `Blocks: yes` is CRITICAL too. The native voice's FIXABLE collects as CRITICAL; other labels
|
||||
* stay as written for lead judgment. When in doubt the parser merges rather than drops: a lost blocker costs the gate.
|
||||
* starts the next finding, except an unlabeled bullet under a paragraph finding whose fields have not come yet, which is its detail.
|
||||
* An inline `Blocks: yes` is CRITICAL too. A label word used as a word (negated, a yardstick, quoted or listed as vocabulary) is
|
||||
* not a label, and a closing summary line after the findings starts none (GST-435). The native voice's FIXABLE collects as CRITICAL;
|
||||
* other labels stay as written for lead judgment. When in doubt the parser merges rather than drops: a lost blocker costs the gate.
|
||||
*/
|
||||
export function parseFindings(reader: string, output: string): Array<{ label: string; raw: string }> {
|
||||
type Finding = { label: string; raw: string; fields: Set<string> };
|
||||
type Finding = { label: string; raw: string; fields: Set<string>; list: boolean };
|
||||
const out: Finding[] = [];
|
||||
let section: string | null = null;
|
||||
let current: Finding | null = null;
|
||||
@@ -84,14 +107,14 @@ export function parseFindings(reader: string, output: string): Array<{ label: st
|
||||
let gap = false;
|
||||
const role = reader.split('/')[0];
|
||||
const named = (l: string) => (role === 'adversarial-native' && l.toUpperCase() === 'FIXABLE' ? 'CRITICAL' : l);
|
||||
const strongest = (t: string) => [...t.matchAll(ALL_LABELS)].map(m => named(m[1]!)).reduce<string | undefined>((a, b) => (a === undefined ? b : stronger(a, b)), undefined);
|
||||
const push = (label: string, raw: string, fields: string[] = []): Finding => { const f = { label, raw, fields: new Set(fields) }; out.push(f); return f; };
|
||||
const strongest = (t: string) => [...t.matchAll(ALL_LABELS)].filter(m => !aWord(t, m.index!, m[1]!.length)).map(m => named(m[1]!)).reduce<string | undefined>((a, b) => (a === undefined ? b : stronger(a, b)), undefined);
|
||||
const push = (label: string, raw: string, fields: string[] = []): Finding => { const f = { label, raw, fields: new Set(fields), list: LIST_ITEM.test(raw) }; out.push(f); return f; };
|
||||
for (const line of output.replace(/\r\n/g, '\n').split('\n')) {
|
||||
const t = line.trim();
|
||||
if (!t) { gap = true; continue; }
|
||||
const afterGap = gap;
|
||||
gap = false;
|
||||
if (/^NO FINDINGS\.?$/.test(t) || /^Recommendation:/i.test(t)) { current = null; continue; }
|
||||
if (/^NO FINDINGS\.?$/.test(t) || /^Recommendation:/i.test(t) || (out.length && SUMMARY.test(t))) { current = null; continue; }
|
||||
const indented = line !== line.trimStart();
|
||||
const label = strongest(t);
|
||||
const blocks = BLOCKS_YES.test(t);
|
||||
@@ -118,7 +141,9 @@ export function parseFindings(reader: string, output: string): Array<{ label: st
|
||||
if (typeof o.severity === 'string') { current = push(o.severity, t); continue; }
|
||||
} catch { /* prose that starts with a brace */ }
|
||||
}
|
||||
const fresh = listItem || (!indented && (t.startsWith('**') || afterGap)) || (!!current?.fields.size && locus);
|
||||
// An unlabeled bullet under a paragraph finding whose fields have not come yet is its detail, not the next finding (GSTA-196 native voice).
|
||||
const detail = listItem && !label && !section && !!current && !current.list && !current.fields.size;
|
||||
const fresh = (listItem && !detail) || (!indented && !detail && (t.startsWith('**') || afterGap)) || (!!current?.fields.size && locus);
|
||||
if (fresh) { current = null; head = t; }
|
||||
const inherited = section && listItem ? section : undefined;
|
||||
const found = label && inherited ? stronger(inherited, label) : label ?? inherited;
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
{"severity":"INFORMATIONAL","path":"CHANGELOG.md","line":5,"summary":"The reworded sentence describes /ship behaviour that gstack 1.91.358.0 does not have. Under bump never, Step 12 prints 'VERSION: <v> kept', and Step 13's Kept branch (ship/sections/changelog.md:49-51) puts an entry 'where the repo's own rule puts unit-PR notes (its CHANGELOG preamble or a fragment directory it names), else under ## [Unreleased]'. Only NO_VERSION or changelog: null skip the entry. The preamble still names no location for unit-PR notes (for example the PR body), so a lane that reads Step 13 literally still reaches the [Unreleased] fallback on its next /ship. That half of the original finding is still open. Fix: say where the notes go ('unit-PR notes go in the PR body'), or set changelog: null in ship-policy.json so Step 13 really writes nothing."}
|
||||
{"severity":"INFORMATIONAL","path":"CHANGELOG.md","line":5,"summary":"The fix leaves the original finding's fragment question open. changelog.d/gsta-30-laneb-quick-3-results.md is still in the tree, and CREW.md still says 'changelog entries go in changelog.d/ fragments'. The new sentence covers CHANGELOG.md only. docs/ship-policy.md:42 says bump never cannot be combined with changelog_fragments, so nothing folds that fragment at release, and a lane following CREW.md can keep adding fragments the preamble never mentions. Fix: name changelog.d/ in the preamble, either as banned or as folded by the Captain at release, and decide what happens to the GSTA-30 fragment."}
|
||||
|
||||
Neither finding is CRITICAL. The change is one prose line, and pregate, tests (1264), typecheck, errors and wiki all pass after the fix. I checked the claim against the installed gstack 1.91.358.0: ship/sections/changelog.md and docs/ship-policy.md.
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
Adversarial review (native) of `1b68e637c07c..0a8efed`. HEAD is still `0a8efeda8f79`, the working tree is clean and the same 10 files changed, so the findings below are the ones I reported earlier, rechecked against this range.
|
||||
|
||||
**Coverage:** I read the full diffs of the non-fixture source: `.github/workflows/{ci,sealed,train-gate}.yml`, `scripts/gate.ts`, `gate/manifest.json`, `package.json` and `README.md`. **I reviewed the test files in summary mode only**, so their coverage of the review is reduced:
|
||||
- `tests/ci/gate.test.ts` is new (+299). It covers the train chain walk, `routeDispatch` refusals, `publishable` binding and the gate-surface rule, and `runCases`/junit judging.
|
||||
- `tests/ci/sealed-workflow.test.ts` (+19/−6) adds the push trigger.
|
||||
- `tests/ci/workflow-hygiene.test.ts` is new (+85). It covers SHA pins, read-only tokens, no secrets, the checksummed Bun install, the gstack pin and parity between the manifest and ci.yml.
|
||||
|
||||
**1. A PR in the train controls the workflow that judges the train (INVESTIGATE).** `workflow_dispatch` runs the copy of `train-gate.yml` at the dispatched ref, `train/<nonce>`, and that copy contains every train PR's edits. So a PR that edits `train-gate.yml` changes the `route`, `test` and `publish` jobs themselves.
|
||||
- It can drop the call to the base's `gate.ts publish`. It can also post the `full_gate_check` check run unconditionally with the publish job's `checks: write`, or give itself more `permissions:` and use repository secrets.
|
||||
- The header says "a pull request in the train cannot change how the train is judged". In practice only the case list and `gate.ts` come from the base.
|
||||
- The "a gate surface PR rides alone" rule (`publishable`, `slots > 1`) is enforced by the publish job, which is code the PR can rewrite. So the rule holds only against PRs that don't break it.
|
||||
- `ci.yml`'s new `workflow_dispatch` has the same property.
|
||||
- Real mitigations would run the judging workflow from the default branch, for example `repository_dispatch` or a dispatch on `main` that takes the train head as an input, or have gstack verify which workflow file produced the check. Either one changes the protocol with gstack.
|
||||
- Today an attacker needs a malicious workflow edit to pass review and READY, and no workflow references a secret.
|
||||
|
||||
Severity: High. Blocks: no.
|
||||
|
||||
**2. A case timeout leaks processes and can lose the record (FIXABLE).** `runCases` calls `spawnSync(..., { timeout })` (`scripts/gate.ts:252`), which kills only the `bash` wrapper. Grandchildren such as `bun test` survive with inherited stdio and keep running into later cases.
|
||||
- A 20-minute case timeout (`CASE_TIMEOUT_MS`) plus the other cases can go past the job's 30-minute `timeout-minutes`.
|
||||
- GitHub then kills `gate.ts` before it writes `train-record.json`. That run has no record and fails as infrastructure, not as a red verdict.
|
||||
- Fix: spawn each case detached and kill its whole process group on timeout, and keep the per-case cap well under the job limit (or size the job to cases × cap).
|
||||
|
||||
Severity: Low. Blocks: no.
|
||||
|
||||
**3. Local `run` hides a broken train chain (FIXABLE).** In `gate.ts run` (`:386`), when `trainBase` returns `{ error }`, the code falls back to `merge-base HEAD origin/<default>` without saying so. A malformed train run locally is judged against the wrong base.
|
||||
- CI always passes `--base`, so this affects only local `bun run gate`.
|
||||
- Fix: fail on an error result, the way `route` and `publish` already do.
|
||||
|
||||
Severity: Low. Blocks: no.
|
||||
|
||||
**4. No train policy is on the base, so trains cannot run yet (INVESTIGATE).** `.gstack/ship-policy.json` does not exist at HEAD. Until a commit with a `train` block lands on main, `route` refuses every train and bisect dispatch with "needs a train block". The workflow cannot gate a train at merge, and this PR does not add the file.
|
||||
- Related: `bun.lock` is not in `gate_surface`. A multi-PR train can change resolved dependency versions within `package.json` ranges, for example `@types/bun ^1.4.4`, which changes what `typecheck` and `unit` run against.
|
||||
|
||||
Severity: Low. Blocks: no.
|
||||
|
||||
**Ruled out:**
|
||||
- `grep -q` after `tee` in the sealed case is not a SIGPIPE risk. `SEALED:` is the last and only line on gstack-pregate's stdout.
|
||||
- The empty wiki `--base` on dispatch is safe: `resolveBase` falls back to `origin/HEAD`.
|
||||
- `Bun.Glob("**/tsconfig*.json")` matches `tsconfig.json` at the repository root.
|
||||
- The bisect artifact name matches gstack's `bisectArtifact`.
|
||||
- `route` refuses when the bash T1 lookup and `trainBase` disagree.
|
||||
|
||||
Recommendation: Ship as-is and send finding 1 to the train-protocol owner because the strongest issue, a train PR rewriting `train-gate.yml` to post `full_gate_check` itself, needs a malicious workflow edit to pass review, and fixing it means changing the dispatch protocol with gstack, not this diff.
|
||||
@@ -4,7 +4,7 @@ import * as fs from 'node:fs';
|
||||
import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import { BriefError, promptHash, renderBrief, sectionTexts } from '../lib/review-brief';
|
||||
import { collect, judge, parseFindings, type MergedLine } from '../lib/review-ledger';
|
||||
import { collect, judge, parseFindings, readJsonl, type MergedLine } from '../lib/review-ledger';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
const BIN = path.join(ROOT, 'bin', 'gstack-review-dispatch');
|
||||
@@ -401,6 +401,57 @@ describe('gstack-review-dispatch sent, collect, judge and verify: nothing change
|
||||
}
|
||||
});
|
||||
|
||||
// Value: protects=a severity word used as a word (negated, a yardstick, quoted, listed as vocabulary, or in a closing summary)
|
||||
// creates no finding, so judge can bind a reader's own "nothing here is CRITICAL" (GST-435: GEVA-77's fix-check closing line
|
||||
// "Neither finding is CRITICAL." collected as CRITICAL fix-check#3 and judge refused to drop it); and every labeled prose finding
|
||||
// shape still collects at its label; fails_when=the closing line is a finding, or a guard case (a label the line opens with, a
|
||||
// Severity field, a table row, a mid-sentence label) is lost; why_new=any label word anywhere on a line was a finding;
|
||||
// seam=parseFindings, collect and judge; no_claim=every English negation (the replay over recorded outputs bounds the rest)
|
||||
test('a severity word that is negated, compared, quoted or in a closing summary is no finding; labeled prose findings stay', () => {
|
||||
const fc = fs.readFileSync(path.join(ROOT, 'test/fixtures/review-ledger/geva-77-fix-check.txt'), 'utf8');
|
||||
expect(parseFindings('fix-check', fc).map(f => f.label)).toEqual(['INFORMATIONAL', 'INFORMATIONAL']);
|
||||
const d = tmp();
|
||||
const merged = path.join(d, 'merged.jsonl');
|
||||
const r = collect(path.join(d, 'l.jsonl'), 'fix-check', fc, merged);
|
||||
expect(r.lines.map(l => [l.id, l.label])).toEqual([['fix-check#1', 'INFORMATIONAL'], ['fix-check#2', 'INFORMATIONAL']]);
|
||||
const judged = path.join(d, 'judged.jsonl');
|
||||
fs.writeFileSync(judged, r.lines.map(l => JSON.stringify({ id: l.id, severity: 'INFORMATIONAL' })).join('\n') + '\n');
|
||||
expect(judge(readJsonl<MergedLine>(merged), readJsonl(judged)).ok).toBe(true);
|
||||
for (const word of ['Neither finding is CRITICAL.', 'This is not CRITICAL.', 'There are no CRITICAL findings.', 'None of these is High.', 'The race sits below CRITICAL on the floor.',
|
||||
'It ranks lower than High.', "Pass 1 rated it 'High'; the fix closes it.", 'I used the `CRITICAL` label for none of them.', 'Severity vocabulary: Critical|High|Medium|Low'])
|
||||
expect([word, parseFindings('outside-voice', word)]).toEqual([word, []]);
|
||||
expect(parseFindings('outside-voice', '1. **High**: retry loop never backs off (q.ts:78)\n\nOverall: one High finding, nothing CRITICAL.').map(f => f.label)).toEqual(['High']);
|
||||
expect(parseFindings('outside-voice', 'Severity: Low (not High)\n').map(f => f.label)).toEqual(['Low']);
|
||||
const kept: Array<[string, string]> = [['CRITICAL: x.ts:4 drops rows', 'CRITICAL'], ['1. The token is never checked (auth.ts:9) — High', 'High'], ['- **High**: leak in log.ts:3', 'High'],
|
||||
['| High | race in queue.ts:4 |', 'High'], ['| `High` | race in queue.ts:4 |', 'High'], ['The retry drops writes, a CRITICAL defect (q.ts:78).', 'CRITICAL'], ['[P1] token not expired', 'P1'],
|
||||
['Not checking expiry at auth.ts:9 lets stale tokens through. Severity: High', 'High'], ['Summary: CRITICAL auth bypass at a.ts:3', 'CRITICAL'], ["The lock isn't released on error (pool.ts:12): High", 'High']];
|
||||
for (const [line, label] of kept) expect([line, parseFindings('outside-voice', line).map(f => f.label)]).toEqual([line, [label]]);
|
||||
});
|
||||
|
||||
// Value: protects=the mirror of GST-435: a refusal whose prose names severity words ("no CRITICAL or High findings to report") is
|
||||
// neither findings nor clean; collect says COLLECT_NOT_REVIEWED and exits 1; fails_when=its words collect as phantom findings or it
|
||||
// reads as reviewed; why_new=GST-434 and GST-435 meet in the same parser; seam=collect; no_claim=refusals that carry a clean marker
|
||||
test('a refusal that names severity words collects no finding and is a non-review, never clean', () => {
|
||||
const p = pass();
|
||||
run(['sent', '--ledger', p.ledger, '--role', 'specialist', '--name', 'testing', '--sent', p.file]);
|
||||
const reply = p.w('reply.out', 'I could not access the diff, so there are no CRITICAL or High findings to report.\n');
|
||||
const col = run(['collect', '--ledger', p.ledger, '--role', 'specialist', '--name', 'testing', '--output', reply, '--merged', p.merged]);
|
||||
expect([col.status, col.stdout.startsWith('COLLECT_NOT_REVIEWED: reader=specialist/testing findings=0 ')]).toEqual([1, true]);
|
||||
expect(run(['verify', '--ledger', p.ledger]).stdout).toContain('BRIEF specialist/testing NOT_COLLECTED');
|
||||
});
|
||||
|
||||
// Value: protects=a paragraph finding's unlabeled detail bullets and the Severity/Blocks line after them stay one finding (GSTA-196
|
||||
// native voice: 4 findings collected as 8, the closing Severity lines opening phantoms judge had to fold); a list item's sibling and
|
||||
// a labeled bullet still start their own findings; fails_when=the sample collects more or fewer than 4, or a labeled bullet merges
|
||||
// away; why_new=every margin list item closed the open finding; seam=parseFindings; no_claim=indented detail lists (already attached)
|
||||
test('unlabeled detail bullets and the Severity line after them stay on the open paragraph finding', () => {
|
||||
const native = fs.readFileSync(path.join(ROOT, 'test/fixtures/review-ledger/gsta-196-native.txt'), 'utf8');
|
||||
expect(parseFindings('adversarial-native', native).map(f => [f.label, f.raw.slice(0, 12), f.raw.endsWith('| Severity: ' + (f.label === 'High' ? 'High' : 'Low') + '. Blocks: no.')]))
|
||||
.toEqual([['High', '**1. A PR in', true], ['CRITICAL', '**2. A case ', true], ['CRITICAL', '**3. Local `', true], ['Low', '**4. No trai', true]]);
|
||||
expect(parseFindings('outside-voice', '1. FIXABLE: race in a.ts:3\n- Unlabeled sibling at b.ts:9\nSeverity: Medium').map(f => f.label)).toEqual(['FIXABLE', 'Medium']);
|
||||
expect(parseFindings('outside-voice', '**Retry storm in r.ts:2 (INVESTIGATE).**\n- detail\n- High: a second defect in s.ts:4\nSeverity: Medium').map(f => f.label)).toEqual(['INVESTIGATE', 'High']);
|
||||
});
|
||||
|
||||
test('verify on a ledger that kept no prompts says so and exits 3', () => {
|
||||
const d = tmp();
|
||||
const ledger = path.join(d, 'ledger.jsonl');
|
||||
|
||||
Reference in new issue
Block a user