mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-21 07:33:14 +01:00
* feat(review): variant G part 4: data files no code names are docs in classifyDiff, untested state-matrix cells map below the floor per cell (GSTA-167 PR-a)
Part 4a: lib/review-depth.ts owns DATA_PATH (JSON, JSONL, NDJSON, CSV, TSV, raster
images; YAML, TOML and SVG stay code) and applies the data rule only in classifyDiff,
to candidates lib/review-collect.ts finds no non-docs consumer for (git grep of
basenames at head, merge base and, live, the working tree; data-to-data chains four
deep; over 2,000 candidates keeps all code; a grep failure fails closed). classifyPath
and every path-only caller are unchanged; the receipt gate scan decides through
gateScanNeeded (path-only), so it cannot be skipped. gstack-ship-preflight computes
RUNTIME_LINES with classifyDiff over gstack-review-plan --json (now with consumed).
Part 4b: lib/review-scope.ts mapStateMatrixOutput maps an "untested cell" finding to
INFORMATIONAL when its cell has test null in the same output's matrix and it is the
only finding on that cell; gstack-state-matrix findings prints it for Step 9.2 before
the merge. state-machines.md is byte-identical; check still reads the raw output.
Reviewer-brief pins repinned under no-loss replay PR-a (Arm 4: plan identity at the 14
cohort heads; free exact re-score of saved C, D and A' findings).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* feat(review): variant G parts 1 and 3: blocking floor in both prose voices, specialists at medium (GSTA-167)
Part 1: the native adversarial and outside-voice prompts end with the blocking
floor ("would fail a test, break production, or lose data or secrets") and ask
each finding for Severity and Blocks: yes|no. Additive rule: a finding is
CRITICAL when its label rule says so or it says Blocks: yes; Blocks: no never
lowers one (proseVoiceSeverity, BLOCKING_FLOOR in lib/review-scope.ts).
Part 3: ROLE_EFFORT puts specialists at medium; Red Team, native adversarial
and fix-check stay full; light's batched specialist (carries the Red Team
prompt) stays full; --no-tiers sends specialists to host.
Repinned reviewer briefs with no-loss replay G1 (live Arm G1, not yet run):
ship/sections/review-army.md, ship/sections/adversarial.md,
review/sections/adversarial.md, lib/review-depth.ts. Merge only on G1's pass.
Review-skill parity ratio 1.3136 -> 1.3180 and the gstack-review size pins
+537 for the approved +0.3k tokens per voice.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* feat(review): variant G part 2: lean ensemble, one batched specialist at every depth (GSTA-167)
planReview dispatches one 'specialist batched' at standard and full depth (and
light, which adds the Red Team checklist): Testing and Security always on code
and gate class, plus State-machines, Data-migration, API-contract and Design
when scope asks; a forced checklist always joins. Maintainability, Performance
and Simplification are read only when forced (SKIP names them). LIGHT_FIXED is
[testing, security]; BATCH_SCOPED joins DEPTH_RULES so old receipts re-plan.
The Red Team rule and prompt are unchanged; it reads the batched reader's
merged findings. specialistsFor stays as the legacy selection the dispatch
compare script models.
Repinned reviewer briefs with no-loss replay L (live Arm L, scored L∘G1 vs G1;
not yet run): ship/sections/review-army.md, lib/review-depth.ts. Merge only on
L's pass.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* feat(review): variant G on #3648's dispatch: Blocks: yes collects as CRITICAL, batched checklists by name (GSTA-167)
Integrates parts 1 and 2 with gstack-review-dispatch (#3648):
- lib/review-ledger.ts parseFindings: a prose voice's "Blocks: yes" makes its
own line, or the finding on the line just before it, CRITICAL in
merged.jsonl (normalizer v2, certified by round 5's G1), so judge refuses a
lead that keeps it below the floor; with no finding before it the Blocks
line is its own CRITICAL finding. "Blocks: no" never lowers a label.
- lib/review-brief.ts: the outside-voice brief no longer appends "Give each
finding a severity"; the section's prompt now asks for Severity and Blocks,
so the brief is the certified section prompt plus the branch diff.
- bin/gstack-review-dispatch render --checklists takes the plan's
checklists= names (review/specialists/<name>.md) as well as paths and
refuses an unknown one, since the batched reader is now every depth's
specialist dispatch.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* chore(changelog): GSTA-167 variant G reviewer stack fragment
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(review): consumer scan survives grep.lineNumber/grep.column; collect reads separate-line Severity/Blocks
/ship pass 1 (PR #3660): the Red Team CRITICAL at lib/review-collect.ts:120 and the
informational findings in the same batch. proseVoiceSeverity had no caller and is
removed; the normalizer v2 table now runs through parseFindings, the path collect uses.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(review): collect keeps every prose blocker and one id per finding
/ship pass 2 (PR #3660): an unlabeled line carrying an inline Blocks: yes collects as
CRITICAL again; Severity/Blocks continuation needs a colon, stops at a list item, heading,
bold paragraph opener or a repeated field, and never swallows a top-level bullet or its
section label; the native voice's FIXABLE collects as CRITICAL under its role with or
without a dispatch name; the consumer scan adds --no-color.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(review): a later finding's fields never land on the one before it; explicit Severity wins its line
/ship pass 2 Red Team (PR #3660): prose after a finding's Severity/Blocks field, a
post-blank paragraph or a line in an indented block starts the next finding, and a
line's explicit Severity: label decides it over other label words on the line.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(review): FIXABLE outranks a Severity label on its line; a blocker keeps its own id and subject
/ship fix-check (PR #3660): labels are named before the strongest is taken, so a one-line
native FIXABLE ... Severity: Medium collects as CRITICAL; a later finding starts at a
file:line after the open finding's fields, field bullets stay on their finding, a field
line may omit the colon, and continuation prose never replaces a list item's subject.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* docs(changelog): GSTA-167 fragment adds round 6 and the collect and consumer-scan fixes
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
---------
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
72 lines
4.7 KiB
TypeScript
72 lines
4.7 KiB
TypeScript
/**
|
|
* GSTA-167 variant G part 1: both prose voices carry the blocking floor verbatim and end each
|
|
* finding with Severity and Blocks, and the Step 11 rule is additive (normalizer v2): a finding is
|
|
* CRITICAL when its label rule says so or it says Blocks: yes; Blocks: no never lowers one.
|
|
*/
|
|
import { describe, expect, test } from 'bun:test';
|
|
import * as fs from 'node:fs';
|
|
import * as path from 'node:path';
|
|
import { parseFindings, rank } from '../lib/review-ledger';
|
|
import { BLOCKING_FLOOR } from '../lib/review-scope';
|
|
import { generateAdversarialStep } from '../scripts/resolvers/outside-voice-steps';
|
|
import { HOST_PATHS } from '../scripts/resolvers/types';
|
|
|
|
const ROOT = path.join(import.meta.dir, '..');
|
|
const render = (skillName: 'ship' | 'review', host: 'claude' | 'codex') => generateAdversarialStep({ skillName, tmplPath: '', host, paths: HOST_PATHS[host] } as any);
|
|
const prompt = (text: string, opener: string) => {
|
|
const start = text.indexOf(opener);
|
|
expect(start).toBeGreaterThan(-1);
|
|
return text.slice(start, text.indexOf('"\n', start));
|
|
};
|
|
const BLOCKS = `A finding blocks only if it ${BLOCKING_FLOOR}; end each finding with \`Severity: Critical|High|Medium|Low\` and \`Blocks: yes|no\`.`;
|
|
|
|
describe('prose voices carry the blocking floor (GSTA-167 part 1)', () => {
|
|
// Value: protects=the outside voice and the native adversarial pass rate against the floor the policy states, so a real
|
|
// blocker labeled Medium or INVESTIGATE can still say Blocks: yes; fails_when=either prompt loses the floor sentence or the
|
|
// Blocks line on any host or skill; seam=generateAdversarialStep
|
|
for (const skill of ['ship', 'review'] as const) for (const host of ['claude', 'codex'] as const) {
|
|
test(`${skill} on ${host}: both prompts end with the floor, Severity and Blocks; the structured review prompt is unchanged`, () => {
|
|
const text = render(skill, host);
|
|
const native = prompt(text, 'Think like an attacker and a chaos engineer. Your job');
|
|
const outside = prompt(text, 'Review the changes on this branch against the base branch.');
|
|
for (const p of [native, outside]) expect(p).toContain(BLOCKS);
|
|
expect(native).toContain('classify as FIXABLE (you know how to fix it) or INVESTIGATE (needs human judgment).');
|
|
expect(text).toContain('Prepare a structured review prompt requesting severity-tagged findings ([P0]-[P3]) or an explicit NO_FINDINGS conclusion.');
|
|
expect(text).toContain('**FIXABLE or `Blocks: yes` findings** are CRITICAL');
|
|
expect(text).toContain('A Critical, High or `Blocks: yes` finding is CRITICAL; `Blocks: no` never lowers one');
|
|
});
|
|
}
|
|
|
|
// Value: protects=F10, every prose copy of the floor stays the same words (wrapped copies compared with whitespace
|
|
// collapsed); fails_when=a copy drifts from BLOCKING_FLOOR; seam=none
|
|
test('every prose copy of the floor matches BLOCKING_FLOOR once whitespace is collapsed', () => {
|
|
const files = ['scripts/resolvers/fix-check.ts', 'review/SKILL.md.tmpl', 'ship/SKILL.md.tmpl', 'capy/skill/gstack/factory.md', 'capy/README.md', 'docs/review-policy.md'];
|
|
const drift = files.filter(f => !fs.readFileSync(path.join(ROOT, f), 'utf8').replace(/\s+/g, ' ').includes(BLOCKING_FLOOR));
|
|
expect(drift).toEqual([]);
|
|
});
|
|
});
|
|
|
|
describe('normalizer v2 as collected: Blocks: yes is CRITICAL, Blocks: no never lowers a label', () => {
|
|
// Value: protects=the rule round 5's G1 was scored under, on the path production runs (gstack-review-dispatch collect);
|
|
// fails_when=Blocks: yes collects below CRITICAL or Blocks: no lowers a label; seam=parseFindings
|
|
test.each([
|
|
['FIXABLE: race in a.ts:3\nSeverity: Medium\nBlocks: no', 'CRITICAL'],
|
|
['INVESTIGATE: odd lock in b.ts:9\nSeverity: Low\nBlocks: yes', 'CRITICAL'],
|
|
['INVESTIGATE: odd lock in b.ts:9\nBlocks: no', 'INVESTIGATE'],
|
|
['FIXABLE: race in a.ts:3. Severity: Medium. Blocks: no', 'CRITICAL'],
|
|
['1. FIXABLE: race in a.ts:3 — Severity: Low, Blocks: no', 'CRITICAL'],
|
|
['- FIXABLE: race in a.ts:3\n- Severity: Medium\n- Blocks: no', 'CRITICAL'],
|
|
] as const)('native %j -> %s', (output, want) => expect(parseFindings('adversarial-native', output).map(g => g.label)).toEqual([want]));
|
|
test.each([
|
|
['1. Drops the last row (q.ts:7). Severity: Critical. Blocks: no', 'Critical'],
|
|
['1. Cache evicts sessions (c.ts:4)\n Severity: High', 'High'],
|
|
['1. Retry storm (r.ts:2)\n Severity: Medium\n Blocks: yes', 'CRITICAL'],
|
|
['1. Index drift (i.ts:5). Severity: Low. Blocks: Yes, it loses the row', 'CRITICAL'],
|
|
['1. Naming (n.ts:1)\n Severity: Low\n Blocks: no', 'Low'],
|
|
] as const)('%j -> %s', (output, want) => {
|
|
const got = parseFindings('outside-voice', output);
|
|
expect(got.map(g => g.label)).toEqual([want]);
|
|
expect(rank(got[0]!.label)).not.toBeNull();
|
|
});
|
|
});
|