From 0eb9c0d88b56f9862183d9f0e562b3ba009953a9 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Mon, 2 Nov 2026 03:36:03 -0800 Subject: [PATCH 01/19] v1.92.72.0 fix(sealed): refuse published records that put a label next to an identifier (GST-620) (#3747) * feat(sealed): refuse published records that put a label next to an identifier (GST-620) sealed.label_keys turns on sealed.label_join: a published JSON, JSONL, CSV, TSV or Markdown-table record that holds a label field and an identifier (an identifier_keys glob or a hex value) is refused when its fingerprint is new against every parent. Unreadable supported files are sealed.label_join_unchecked. Without label_keys the guard reads no blobs and its output matches main byte for byte (golden in test/fixtures/sealed-guard-off-golden.json). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes sealed.label_join: a hex object key is an identifier of the object it holds; label_keys and identifier_keys refuse dotted names; register the off golden as a free fixture. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: adversarial review fixes (GST-620) A field may count as both a label and an identifier (a pushed label_keys can no longer hide a join); TSV cells may be quoted; a file that never names a label key is skipped before parsing; blobs are parsed as they are read; label_join refusals no longer suggest allow_paths. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(sealed): strip quotes before the label-key prefilter so a quoted header fragment still joins Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(sealed): the label-key prefilter ignores double quotes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(sealed): hex lists count as identifiers; a \u escape always parses (Step 14.5 audit) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 16 ++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- bin/gstack-pregate | 3 +- docs/pregate.md | 62 +++++- docs/troubleshooting.md | 11 +- lib/pregate/config.ts | 2 +- lib/sealed-guard.ts | 32 +-- lib/sealed-join.ts | 227 ++++++++++++++++++++ package.json | 2 +- test/fixtures/module-size-ratchet.json | 1 + test/fixtures/sealed-guard-off-golden.json | 65 ++++++ test/helpers/free-fixtures-data.ts | 1 + test/sealed-guard.test.ts | 61 ++++++ test/sealed-join.test.ts | 237 +++++++++++++++++++++ 15 files changed, 697 insertions(+), 27 deletions(-) create mode 100644 lib/sealed-join.ts create mode 100644 test/fixtures/sealed-guard-off-golden.json create mode 100644 test/sealed-join.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 2946d21bc..237f5c19e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,21 @@ # Changelog +## [1.92.72.0] - 2026-11-02 + +**The sealed guard can now refuse a published record that puts a label next to an identifier.** gstack-evals hides which test units are decoys. Its /ship on garrytan/gstack-evals#99 published a panel where each unit's `stratum`, `tags` and `decoy_set` sat next to its `head`, `tree_sha256`, `plan_bytes` and `files`, values a model under test can compute from its own copy of the work. A model could look itself up and learn it was the decoy. No answer leaked, so `gstack-pregate sealed` said `clear` (GSTA-2, GST-620). The new rule is opt-in. A repo that sets `sealed.label_keys` gets it, and every other repo sees the same output as before, byte for byte. + +### Added +- **`sealed.label_keys` and `sealed.identifier_keys` in `.gstack/pregate.json`.** With `label_keys` set, `sealed.label_join` checks every .json, .jsonl, .ndjson, .csv, .tsv and Markdown pipe table a published commit adds, modifies, renames or copies. A record that holds a label field and an identifier (a field matching an `identifier_keys` glob, by default `head`, `*_sha256`, `*_bytes`, `files` and similar names, or any 7 to 64 character hex string, as a value, in a list, or as the key of an object) is a join. The guard fingerprints each join in memory and compares it with the parent's version of the file, so unchanged rows and reformatting stay clear while a replaced, added or widened row refuses even when the row count stays the same. Findings name the file, the commit and the field names (`keys=stratum,tags,head,plan_bytes`), never the values, on stderr, in `--json` and in the pre-push hook. +- **`sealed.label_join_unchecked`.** A file whose text never names a label key, ignoring double quotes, cannot hold a join and is skipped without parsing, so a commented `tsconfig.json` passes. A file with a `\u` escape is always parsed. A supported file over 4 MiB, one that names a label key and does not parse, one nested deeper than 64 levels, or a blob git cannot read refuses instead of passing unread. The `SEALED:` line counts these as `unchecked=`. Like every sealed finding, it can be waived through the base branch's `.gstack/sealed-allow.tsv`; `allow_paths` never exempt either rule. +- **The policy only tightens.** A field can count as both a label and an identifier, so adding identifier names to `label_keys` never hides a join. Both lists are the union of the base branch's and the pushed policy's, each resolved to its default first, so a pushed config can turn the rule on or widen it but cannot turn it off or narrow it. + +### Changed +- **docs/pregate.md#sealed** documents the rule, its normalization (`Tree SHA-256` matches as `tree_sha_256`) and its known limits: joins across files, joins by reference, columnar JSON, labels in file names and prose. Its version-skew note now names the new keys. A gstack older than this release refuses them as unknown, so raise `min_gstack_version` before adopting them. The unknown-key error now lists `label_keys` and `identifier_keys` among the valid keys. + +### Tests +- `test/sealed-guard.test.ts`: the GSTA-2 panel refuses through `gstack-pregate sealed`, `--json` and the pre-push hook, and warns on a private destination, with no value in any output. It failed on main. A golden captured from main 2d1b55d76 covers seven pushes (refused, clear, exempt, waived, warned, error, no policy) without `label_keys`; stdout, stderr, `--json`, the hook's stderr and every exit code match it byte for byte. +- `test/sealed-join.test.ts`: each format, human table headers, hex values under unlisted keys, maps keyed by sha, quoted TSV, a CSV header split by quotes (`"Strat"um`), a commented tsconfig.json that stays clear, dotted key names refused in the config, identifier names added to a pushed `label_keys`, decimal strings that stay clear, replacement at the same count, delete plus add, an added identifier field, labels then identifiers a commit later, renames, merges, five unchecked cases, waivers, `allow_paths`, pushed attempts to narrow or disable the rule, and `label_keys: []` matching an absent key. + ## [1.92.71.0] - 2026-11-02 **Four free-suite waits now end when the thing they wait for happens: the six files involved spend 18–29 s less in summed per-file runtime, with every assertion intact.** `./setup` returns from a finished child up to ~0.9 s sooner too. diff --git a/VERSION b/VERSION index 2d527513b..8d5fb4735 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.92.71.0 +1.92.72.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index 181ad1acd..5c5850c4e 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.92.71.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.92.72.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/bin/gstack-pregate b/bin/gstack-pregate index 772bec1b1..cfee7e707 100755 --- a/bin/gstack-pregate +++ b/bin/gstack-pregate @@ -274,8 +274,9 @@ function cmdSealed(root: string, rest: string[], json: boolean): number { const blocks = report.status === 'refused' || report.status === 'error'; if (json) { process.stdout.write(JSON.stringify({ ...report, ok: !blocks }) + '\n'); return blocks ? EXIT.fail : EXIT.ok; } process.stderr.write(renderSealed(report, 'gstack-pregate sealed')); + const unchecked = report.findings.filter(f => f.rule === 'sealed.label_join_unchecked').length; const policy = report.policyLabels.length ? report.policyLabels.join(' + ') : `no sealed key on ${baseRef ?? `${remote}/${base}`} or ${end}`; - process.stdout.write(`SEALED: ${report.status} findings=${report.findings.length} commits=${report.commits} visibility=${report.visibility}${report.visibility === 'unknown' ? ' (counted public)' : ''} waived=${report.waived.length} exempt=${report.exempt.length} policy=${policy}\n`); + process.stdout.write(`SEALED: ${report.status} findings=${report.findings.length} commits=${report.commits} visibility=${report.visibility}${report.visibility === 'unknown' ? ' (counted public)' : ''} waived=${report.waived.length} exempt=${report.exempt.length}${unchecked ? ` unchecked=${unchecked}` : ''} policy=${policy}\n`); return blocks ? EXIT.fail : EXIT.ok; } diff --git a/docs/pregate.md b/docs/pregate.md index c7238d475..e82fb15a0 100644 --- a/docs/pregate.md +++ b/docs/pregate.md @@ -1093,7 +1093,8 @@ with a timing flake: never raise the bound. "answer_markers": ["SEALED ANSWER:"], "seed_patterns": ["shuffle_seed=\\d+"], "local_paths": true, - "case_insensitive": true } } + "case_insensitive": true, + "label_keys": ["stratum", "decoy_set"] } } ``` A repo with a `sealed` key gets a configured local check against accidental @@ -1139,11 +1140,60 @@ gstack-pregate sealed [--range ] [--remote origin] [--base ] [--js sealed data. Each exempt path is listed on every push (`exempt commit (sealed.allow_paths )`) and counted as `exempt=` on the `SEALED:` line. -- **Version skew.** A gstack older than the release whose CHANGELOG adds - `sealed.allow_paths` refuses `allow_paths` and `case_insensitive` as - unknown keys, so every push and `gstack-pregate run` in the repository - fail on that machine. Raise `min_gstack_version` in - `.gstack/ship-policy.json` to that release before landing either key. +- **Label joins.** Off unless `label_keys` names at least one field. Then + `sealed.label_join` refuses a published record that puts a label field next + to an identifier, the way a panel row with `stratum` beside `head` and + `plan_bytes` tells a model under test which unit it is. A record is a JSON + object at any depth (its own fields plus one level of nested objects, named + like `meta.head`), a JSONL or NDJSON line, a CSV or TSV row, or a row of a + Markdown pipe table (CSV and TSV cells may be quoted). Field names and table headers are compared after + trimming, lowercasing and turning spaces and hyphens into underscores + (`Tree SHA-256` is `tree_sha_256`); this is the rule's own normalization, + separate from `case_insensitive`. An identifier is a field whose name + matches an `identifier_keys` glob (`*` is the only wildcard; the default + is `head`, `head_sha`, `sha`, `commit`, `tree`, `tree_sha*`, `*_sha`, + `*_sha256`, `sha256`, `digest`, `*_digest`, `hash`, `*_hash`, + `*_bytes`, `bytes`, `size`, `files`, `lines`), or any string value (or a + list holding one) of 7 to 64 hex characters with at least one digit and + one letter a to f. An + object key of that shape counts as an identifier of the object it holds + (shown as `(key)`), so a map keyed by sha joins too. Numbers count only + through the field name. A field can be both a label and an identifier, and + a record joins when it has a label field and a different identifier field, + so adding identifier names to `label_keys` never hides a join. Both lists take bare field names: a nested field + matches by its own name (`stratum`, never `meta.stratum`). For each supported file a commit adds, + modifies, renames or copies, the guard fingerprints every joined record + (sha256 over its label and identifier fields and their values, kept in + memory) and compares the set with the parent's version of the file (the + rename or copy source; every parent for a merge). A fingerprint no parent + has is a finding, so unchanged records and formatting-only edits stay clear, + and a replaced, added or widened record refuses even when the count stays + the same. A file whose text never names a label key, ignoring double + quotes (so `"Strat"um` still names `stratum`), cannot hold a join, so + it is skipped without parsing (a commented `tsconfig.json` passes). A file + with a `\u` escape, which could spell a label key, is always parsed, so a + commented one is `sealed.label_join_unchecked`. A + supported file over 4 MiB, one that names a label key and does not parse (one bad + JSONL line is enough), one nested deeper than 64 levels, or a blob git + cannot read is `sealed.label_join_unchecked`. It refuses too, and the + `SEALED:` line counts it as `unchecked=`. Findings print field names + (`keys=`, label fields first), never values; `--json` carries them as + `keys`. `allow_paths` never exempt either rule; a base-branch waiver does. + Both lists are the union of the base branch's and the pushed policy's, each + resolved to its default first, so a push can turn the rule on or widen it + but never turn it off or narrow it. Without `label_keys` the guard reads no + file contents and its output is unchanged. Known limits: joins across files, + joins by reference inside a file (a `decoy_set` list naming units described + elsewhere), a map from an identifier straight to a label value with no + label field name, columnar JSON (`columns` plus `rows`), labels encoded in + file names, and prose. +- **Version skew.** A gstack older than the release that adds a `sealed` key + refuses it as unknown, so every push and `gstack-pregate run` in the + repository fail on that machine. `allow_paths` and `case_insensitive` came + with the release whose CHANGELOG adds `sealed.allow_paths`; `label_keys` + and `identifier_keys` came with the release whose CHANGELOG adds + `sealed.label_join`. Raise `min_gstack_version` in + `.gstack/ship-policy.json` to that release before landing the key. - **Policy.** The union of the destination base branch's `sealed` and the pushed commit's: a push can tighten it, never relax it (`allow_paths` come from the base branch alone, and a pushed deny glob cannot override one). The diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index d73967bb1..5cd386160 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -3566,7 +3566,7 @@ base branch's copy or the user-owned file. For the user-owned file, edit it directly at the path the message names. A user file inside the repository, or reached through a symlink into it, is ignored rather than read (`--explain` names it). An unknown `sealed` key that a newer gstack defines (such as -`allow_paths` or `case_insensitive`) means this machine's gstack is older than +`allow_paths`, `case_insensitive`, `label_keys` or `identifier_keys`) means this machine's gstack is older than the policy: upgrade gstack here, and raise `min_gstack_version` in the repository's `.gstack/ship-policy.json` before landing such a key. @@ -4175,7 +4175,10 @@ migration is a code change, so the gate reruns. sealed path or transcript (binary, empty and later-deleted files included), or adds a line with a sealed-answer marker, a seed or a local home path, and the destination is public or its visibility is unknown (a new or unreadable -repository counts as public). An unreadable or malformed sealed policy refuses +repository counts as public). With `sealed.label_keys` set, a new record that +puts a label field next to an identifier (`sealed.label_join`), or a supported +file the guard could not read (`sealed.label_join_unchecked`), refuses the +same way. An unreadable or malformed sealed policy refuses too. The managed pre-push hook and `gstack-pregate sealed` print one line per finding: rule, `file[:line]`, commit. They never print the matched text. @@ -4184,7 +4187,9 @@ reaches the public repository (a later commit that deletes it still publishes it). A publishable file kept next to sealed data (a `MANIFEST.json` of hashes and counts) can be exempted from the path rules with a `sealed.allow_paths` glob in the base branch's `.gstack/pregate.json`; the text rules still read it -([docs/pregate.md#sealed](pregate.md#sealed)). A reviewed false positive gets +([docs/pregate.md#sealed](pregate.md#sealed)). For `sealed.label_join`, drop +the identifier fields (or the labels) from the published file, or keep the file +out of the public repository; `allow_paths` never exempt it. A reviewed false positive gets one line in the base branch's `.gstack/sealed-allow.tsv`: `rulepathshareason`, where `sha` is the full commit sha (a shortened one is ignored and named on stderr). A private diff --git a/lib/pregate/config.ts b/lib/pregate/config.ts index ac2a511d1..72fd16c11 100644 --- a/lib/pregate/config.ts +++ b/lib/pregate/config.ts @@ -348,7 +348,7 @@ export function explainConfig(): string { 'local: / lanes a local run satisfies (platform-agnostic toolchain, or CI off); their tests run here, not requires-remote', 'guards: [{ id, when: [globs], command, fix?, timeout_s? }]: each guard whose globs match a touched file runs in its own process (--allow-repo-commands); a failure fails pregate with the guard id and fix; never downgradable', `serial_patterns: test-file globs the lanes check runs one process per file, apart from the unit files (default ${DEFAULT_SERIAL_PATTERNS.join(', ')})`, - 'sealed: { paths, transcripts, answer_markers, seed_patterns, local_paths, allow_paths, case_insensitive, visibility }: sealed or eval output the pre-push hook and `gstack-pregate sealed` refuse to publish to a public destination; allow_paths (base branch only) exempt publishable files from the path globs (docs/pregate.md#sealed)', + 'sealed: { paths, transcripts, answer_markers, seed_patterns, local_paths, allow_paths, case_insensitive, label_keys, identifier_keys, visibility }: sealed or eval output the pre-push hook and `gstack-pregate sealed` refuse to publish to a public destination; allow_paths (base branch only) exempt publishable files from the path globs; label_keys turns on sealed.label_join, which refuses a published record that puts one of those fields next to an identifier (identifier_keys globs, or a hex value) (docs/pregate.md#sealed)', 'price_file: repo-relative path of the GSTACK_EVAL_CALLS v1 call-plan file `gstack-eval-plan price-gate` prices a paid gate from; never executed (docs/eval-calls.md)', 'stall_timeout_s: seconds without test output before the lanes check kills the runner\'s process group and records it incomplete (default 300; 0 disables; --stall-timeout overrides)', 'timeout_s: seconds each runner process may take (the unit-file group, each serial file, each guard without its own timeout_s, and the remote gate\'s deadline) when --timeout is not passed; 1 to 3600, default 600', diff --git a/lib/sealed-guard.ts b/lib/sealed-guard.ts index 7ba8b261d..05875c249 100644 --- a/lib/sealed-guard.ts +++ b/lib/sealed-guard.ts @@ -11,6 +11,8 @@ * local_paths true: a local home path (/home//, /Users//) in added text * allow_paths globs exempt from the path rules (read from the base branch only; never folds case) * case_insensitive true: `paths` and `transcripts` match regardless of case + * label_keys field names whose records may not also carry an identifier (lib/sealed-join.ts; off when empty) + * identifier_keys field-name globs that count as identifiers (default DEFAULT_IDENTIFIER_KEYS) * visibility "public" | "private": the destination override (reviewed on the base branch) * * The publication boundary is every commit the push publishes: the local sha @@ -32,19 +34,20 @@ import { spawnSync } from 'node:child_process'; import * as fs from 'node:fs'; import * as path from 'node:path'; import { containedRelative, globToRegExp } from './ship-policy'; +import { DEFAULT_IDENTIFIER_KEYS, identifierKeyError, joinFindings, labelKeyError } from './sealed-join'; import { resolveStateRoot } from './state-root'; export const SEALED_CONFIG_PATH = '.gstack/pregate.json'; export const SEALED_ALLOW_PATH = '.gstack/sealed-allow.tsv'; export const VISIBILITY_OVERRIDE_KEY = 'sealed.visibility'; -export type SealedRule = 'sealed.path' | 'sealed.transcript' | 'sealed.answer' | 'sealed.seed' | 'sealed.local_path'; -export interface SealedPolicy { paths: string[]; transcripts: string[]; answer_markers: string[]; seed_patterns: string[]; local_paths: boolean; allow_paths: string[]; case_insensitive: boolean; visibility?: 'public' | 'private' } -export interface SealedFinding { rule: SealedRule; file: string; line?: number; commit: string } +export type SealedRule = 'sealed.path' | 'sealed.transcript' | 'sealed.answer' | 'sealed.seed' | 'sealed.local_path' | 'sealed.label_join' | 'sealed.label_join_unchecked'; +export interface SealedPolicy { paths: string[]; transcripts: string[]; answer_markers: string[]; seed_patterns: string[]; local_paths: boolean; allow_paths: string[]; case_insensitive: boolean; label_keys: string[]; identifier_keys: string[]; visibility?: 'public' | 'private' } +export interface SealedFinding { rule: SealedRule; file: string; line?: number; commit: string; keys?: string[] } export type ExemptFinding = SealedFinding & { glob: string }; export interface AllowEntry { rule: string; path: string; sha: string; reason: string } export type Visibility = 'public' | 'private' | 'unknown'; -const KEYS = ['paths', 'transcripts', 'answer_markers', 'seed_patterns', 'local_paths', 'allow_paths', 'case_insensitive', 'visibility']; +const KEYS = ['paths', 'transcripts', 'answer_markers', 'seed_patterns', 'local_paths', 'allow_paths', 'case_insensitive', 'label_keys', 'identifier_keys', 'visibility']; const MAX_ENTRIES = 64; const MAX_REGEX = 256; const LOCAL_PATH_RE = /(?:^|[^A-Za-z0-9_.~-]|\$\{!?(?:[A-Za-z_][A-Za-z0-9_]*(?:\[[^\]\s]{0,64}\])?|[0-9]+|[@*#?$!-]):?-)(?:\/home\/[^/\s'"`]+\/|\/Users\/[^/\s'"`]+\/|[A-Za-z]:\\+Users\\+[^\\\s'"`]+\\)/; @@ -68,6 +71,7 @@ export function sealedErrors(raw: unknown): Array<{ path: string; message: strin const regex = (r: string) => { if (!r || r.length > MAX_REGEX) return `must be 1-${MAX_REGEX} characters`; try { new RegExp(r); return null; } catch (e: any) { return `is not a valid regex (${e.message})`; } }; const exception = (g: string) => { const name = g.split('/').pop()!; return glob(g) ?? (name.includes('**') || /^[*?]+(\.|$)/.test(name) ? 'has a wildcard file name, which would exempt whole directories or file types; name the file (e.g. **/sealed/*/MANIFEST.json)' : null); }; list('paths', glob); list('transcripts', glob); list('answer_markers', regex); list('seed_patterns', regex); list('allow_paths', exception); + list('label_keys', labelKeyError); list('identifier_keys', identifierKeyError); for (const k of ['local_paths', 'case_insensitive']) if (k in o && typeof o[k] !== 'boolean') errors.push({ path: `sealed.${k}`, message: 'must be true or false' }); if ('visibility' in o && o.visibility !== 'public' && o.visibility !== 'private') errors.push({ path: 'sealed.visibility', message: 'must be "public" or "private"' }); return errors; @@ -93,15 +97,15 @@ export function sealedPolicyAt(repoRoot: string, treeish: string, label: string) const errors = sealedErrors(raw.sealed); if (errors.length) return { error: errors.map(e => `${e.path}: ${e.message}`).join('; '), label }; const s = raw.sealed; - return { policy: { paths: s.paths ?? [], transcripts: s.transcripts ?? [], answer_markers: s.answer_markers ?? [], seed_patterns: s.seed_patterns ?? [], local_paths: s.local_paths === true, allow_paths: s.allow_paths ?? [], case_insensitive: s.case_insensitive === true, visibility: s.visibility }, label }; + return { policy: { paths: s.paths ?? [], transcripts: s.transcripts ?? [], answer_markers: s.answer_markers ?? [], seed_patterns: s.seed_patterns ?? [], local_paths: s.local_paths === true, allow_paths: s.allow_paths ?? [], case_insensitive: s.case_insensitive === true, label_keys: s.label_keys ?? [], identifier_keys: s.identifier_keys ?? DEFAULT_IDENTIFIER_KEYS, visibility: s.visibility }, label }; } -/** Base ∪ pushed: deny lists concatenate, `local_paths` and `case_insensitive` OR (both only tighten); `allow_paths` relax, so they come from the base alone; visibility stays private only when the base says so and the push does not say public. */ +/** Base ∪ pushed: deny lists, `label_keys` and `identifier_keys` (each resolved to its default per policy first) concatenate, `local_paths` and `case_insensitive` OR (both only tighten); `allow_paths` relax, so they come from the base alone; visibility stays private only when the base says so and the push does not say public. */ export function unionPolicies(base: SealedPolicy | null, pushed: SealedPolicy | null): SealedPolicy | null { if (!base && !pushed) return null; - const u = (k: 'paths' | 'transcripts' | 'answer_markers' | 'seed_patterns') => [...new Set([...(base?.[k] ?? []), ...(pushed?.[k] ?? [])])]; + const u = (k: 'paths' | 'transcripts' | 'answer_markers' | 'seed_patterns' | 'label_keys' | 'identifier_keys') => [...new Set([...(base?.[k] ?? []), ...(pushed?.[k] ?? [])])]; const visibility = pushed?.visibility === 'public' || base?.visibility === 'public' ? 'public' : base?.visibility === 'private' ? 'private' : undefined; - return { paths: u('paths'), transcripts: u('transcripts'), answer_markers: u('answer_markers'), seed_patterns: u('seed_patterns'), local_paths: !!(base?.local_paths || pushed?.local_paths), allow_paths: base?.allow_paths ?? [], case_insensitive: !!(base?.case_insensitive || pushed?.case_insensitive), visibility }; + return { paths: u('paths'), transcripts: u('transcripts'), answer_markers: u('answer_markers'), seed_patterns: u('seed_patterns'), local_paths: !!(base?.local_paths || pushed?.local_paths), allow_paths: base?.allow_paths ?? [], case_insensitive: !!(base?.case_insensitive || pushed?.case_insensitive), label_keys: u('label_keys'), identifier_keys: u('identifier_keys'), visibility }; } /** Reviewed waivers: `rulepathshareason` lines in the base branch's allow file; a waiver counts only when its sha equals the finding's full commit sha. */ @@ -190,7 +194,7 @@ function textFindings(repoRoot: string, commits: string[], p: SealedPolicy): Sea export function scanCommits(repoRoot: string, commits: string[], policy: SealedPolicy): SealedFinding[] { if (!commits.length) return []; const seen = new Set(); - return [...pathFindings(repoRoot, commits, policy), ...textFindings(repoRoot, commits, policy)].filter(f => { + return [...pathFindings(repoRoot, commits, policy), ...textFindings(repoRoot, commits, policy), ...joinFindings(repoRoot, commits, policy)].filter(f => { const key = `${f.rule}\0${f.file}\0${f.line ?? ''}\0${f.commit}`; if (seen.has(key)) return false; seen.add(key); @@ -299,15 +303,17 @@ export function renderSealed(r: SealedReport, context: string): string { lines.push(`${context}: ignored waiver ${i.rule} ${i.path} sha ${i.sha} (${why}; this finding is in commit ${i.commit}, review it before waiving)`); } const vis = `visibility: ${r.visibility}${r.visibility === 'unknown' ? ', counted public' : ''}; ${r.visibilitySource}`; - const body = r.findings.map(f => ` ${f.rule} ${where(f)} commit ${f.commit.slice(0, 12)}`); + const body = r.findings.map(f => ` ${f.rule} ${where(f)} commit ${f.commit.slice(0, 12)}${f.keys ? ` keys=${f.keys.join(',')}` : ''}`); + const join = r.findings.some(f => f.rule.startsWith('sealed.label_join')) ? [`sealed.label_join: a record puts a sealed.label_keys field next to an identifier (keys= lists the label fields, then the identifier fields); drop one side from the published file or keep the file out of the public repository. sealed.label_join_unchecked: a supported file the guard could not read (over 4 MiB, unparseable, or nested deeper than 64); allow_paths never exempt either rule`] : []; if (r.status === 'error') { lines.push(`\n⛔ ${context} BLOCKED the publication of ${r.target} — ${r.error}.`); lines.push(`Fix the sealed policy (gstack-pregate validate), then retry; an unreadable policy never passes (SEALED_OUTPUT_REFUSED)`); } else if (r.status === 'refused') { - lines.push(`\n⛔ ${context} BLOCKED the publication of ${r.target} (${vis}) — sealed or eval output in ${r.commits} published commit(s):`, ...body); - lines.push(`Remove it from every published commit before this branch reaches a public repository, exempt a publishable path with a sealed.allow_paths glob in the base branch's ${SEALED_CONFIG_PATH}, or waive one reviewed finding with a \`rulepathfull-commit-shareason\` line in the base branch's ${SEALED_ALLOW_PATH}. A private destination: set ${VISIBILITY_OVERRIDE_KEY} "private" in the base branch's ${SEALED_CONFIG_PATH} (SEALED_OUTPUT_REFUSED)`); + lines.push(`\n⛔ ${context} BLOCKED the publication of ${r.target} (${vis}) — sealed or eval output in ${r.commits} published commit(s):`, ...body, ...join); + const exemptable = r.findings.some(f => !f.rule.startsWith('sealed.label_join')) ? ` exempt a publishable path with a sealed.allow_paths glob in the base branch's ${SEALED_CONFIG_PATH},` : ''; + lines.push(`Remove it from every published commit before this branch reaches a public repository,${exemptable} or waive one reviewed finding with a \`rulepathfull-commit-shareason\` line in the base branch's ${SEALED_ALLOW_PATH}. A private destination: set ${VISIBILITY_OVERRIDE_KEY} "private" in the base branch's ${SEALED_CONFIG_PATH} (SEALED_OUTPUT_REFUSED)`); } else if (r.status === 'warned') { - lines.push(`${context}: warning — sealed or eval output in a push to a private destination (${vis}); it would be refused for a public one:`, ...body); + lines.push(`${context}: warning — sealed or eval output in a push to a private destination (${vis}); it would be refused for a public one:`, ...body, ...join); } return lines.length ? lines.join('\n') + '\n' : ''; } diff --git a/lib/sealed-join.ts b/lib/sealed-join.ts new file mode 100644 index 000000000..3c048ddf4 --- /dev/null +++ b/lib/sealed-join.ts @@ -0,0 +1,227 @@ +/** + * sealed-join — the `sealed.label_join` rule of the sealed-output guard + * (lib/sealed-guard.ts), on only when the policy names `sealed.label_keys`. + * + * A record that puts a label field (a decoy stratum, a flaw tag) next to an + * identifier the model under test can compute from its own copy of the work + * (a head sha, a tree hash, a byte count) tells that model which label it + * carries. For every .json, .jsonl, .ndjson, .csv, .tsv or .md (pipe table) + * file a published commit adds, modifies, renames or copies, each joined + * record is fingerprinted (sha256 over its label and identifier key/value + * pairs, in memory only) at the commit and at every parent's version (the + * rename or copy source); a fingerprint no parent has is a finding. A file + * over 4 MiB, one that does not parse, nests deeper than 64 or cannot be read + * is `sealed.label_join_unchecked`. Findings carry field names, never values. + */ +import { spawnSync } from 'node:child_process'; +import { createHash } from 'node:crypto'; +import type { SealedFinding } from './sealed-guard'; + +export const DEFAULT_IDENTIFIER_KEYS = ['head', 'head_sha', 'sha', 'commit', 'tree', 'tree_sha*', '*_sha', '*_sha256', 'sha256', 'digest', '*_digest', 'hash', '*_hash', '*_bytes', 'bytes', 'size', 'files', 'lines']; +export const JOIN_MAX_BYTES = 4 * 1024 * 1024; +const MAX_DEPTH = 64; +const MAX_KEY = 128; +const SUPPORTED = /\.(json|jsonl|ndjson|csv|tsv|md)$/i; +const HEX_ID = /^(?=[0-9a-f]*[0-9])(?=[0-9a-f]*[a-f])[0-9a-f]{7,64}$/i; +const ZERO = /^0+$/; +const TABLE_DELIMITER = /^\s*\|?\s*:?-+:?\s*(\|\s*:?-+:?\s*)*\|?\s*$/; + +export interface JoinPolicy { label_keys: string[]; identifier_keys: string[] } +type Field = { name: string; key: string; value: unknown }; +type Joins = Map; + +/** The rule's own key normalization: trimmed, lowercased, spaces and hyphens as underscores. */ +export const normalizeKey = (k: string) => k.trim().toLowerCase().replace(/[ -]/g, '_'); + +export function labelKeyError(s: string): string | null { + return s.trim() && s.length <= MAX_KEY && !/[*?[\]{}.]/.test(s) ? null : `must be a bare field name of 1-${MAX_KEY} characters with no wildcard or dot (meta.stratum matches as stratum)`; +} +export function identifierKeyError(s: string): string | null { + return s.trim() && s.length <= MAX_KEY && !/[?[\]{}.]/.test(s) ? null : `must be a bare field name of 1-${MAX_KEY} characters with no dot; * is the only wildcard`; +} + +const present = (v: unknown) => typeof v === 'number' || (typeof v === 'string' && v.trim() !== '') || (Array.isArray(v) && v.length > 0); +const hexValue = (v: unknown): boolean => typeof v === 'string' ? HEX_ID.test(v.trim()) : Array.isArray(v) && v.some(e => typeof e === 'string' && HEX_ID.test(e.trim())); +const shown = (name: string) => name.split('.').map(s => HEX_ID.test(s) ? '(hex)' : s).join('.').slice(0, 64); + +function classifier(p: JoinPolicy): (fields: Field[]) => { fp: string; labels: string[]; ids: string[] } | null { + const labels = new Set(p.label_keys.map(normalizeKey)); + const ids = p.identifier_keys.map(g => new RegExp(`^${normalizeKey(g).replace(/[.+^${}()|[\]\\?]/g, '\\$&').replace(/\*/g, '.*')}$`)); + return fields => { + const l = fields.filter(f => labels.has(f.key)); + const i = fields.filter(f => (present(f.value) && ids.some(re => re.test(f.key))) || hexValue(f.value)); + if (!l.some(a => i.some(b => b !== a))) return null; + const fp = createHash('sha256').update([...new Set([...l, ...i])].map(f => `${f.name}=${JSON.stringify(f.value)}`).sort().join('\0')).digest('hex'); + return { fp, labels: l.map(f => shown(f.name)), ids: i.map(f => shown(f.name)) }; + }; +} + +/** Every JSON object at any depth as one record: its own fields plus one level of nested object fields (`meta.head`), and the hex key that holds it as `(key)`; false past the depth cap. */ +function jsonRecords(root: unknown, out: Field[][]): boolean { + const field = (name: string, value: unknown, key: string): Field => ({ name, key: normalizeKey(key), value }); + const stack: Array<[unknown, number, string?]> = [[root, 0]]; + while (stack.length) { + const [v, depth, heldBy] = stack.pop()!; + if (!v || typeof v !== 'object') continue; + if (depth > MAX_DEPTH) return false; + if (Array.isArray(v)) { for (const e of v) stack.push([e, depth + 1]); continue; } + const fields: Field[] = heldBy && HEX_ID.test(heldBy) ? [field('(key)', heldBy, '(key)')] : []; + for (const [k, val] of Object.entries(v)) { + const name = normalizeKey(k); + fields.push(field(name, val, k)); + if (val && typeof val === 'object' && !Array.isArray(val)) for (const [k2, v2] of Object.entries(val)) fields.push(field(`${name}.${normalizeKey(k2)}`, v2, k2)); + stack.push([val, depth + 1, k]); + } + out.push(fields); + } + return true; +} + +function delimitedRows(text: string, sep: string): string[][] | null { + const rows: string[][] = []; let row: string[] = []; let cell = ''; let quoted = false; + for (let i = 0; i < text.length; i++) { + const c = text[i]!; + if (quoted) { + if (c !== '"') cell += c; + else if (text[i + 1] === '"') { cell += '"'; i++; } + else quoted = false; + } else if (c === '"' && cell.trim() === '') { quoted = true; cell = ''; } + else if (c === sep) { row.push(cell); cell = ''; } + else if (c === '\n' || c === '\r') { if (c === '\r' && text[i + 1] === '\n') i++; row.push(cell); rows.push(row); row = []; cell = ''; } + else cell += c; + } + if (quoted) return null; + if (cell !== '' || row.length) { row.push(cell); rows.push(row); } + return rows; +} + +function markdownTables(text: string): string[][][] { + const cells = (line: string) => line.trim().replace(/^\|/, '').replace(/(? c.trim().replace(/^[`*]+|[`*]+$/g, '')); + const lines = text.split(/\r?\n/); + const tables: string[][][] = []; + for (let i = 0; i + 1 < lines.length; i++) { + if (!lines[i]!.includes('|') || !lines[i + 1]!.includes('|') || !TABLE_DELIMITER.test(lines[i + 1]!)) continue; + const table = [cells(lines[i]!)]; + let j = i + 2; + for (; j < lines.length && lines[j]!.includes('|'); j++) table.push(cells(lines[j]!)); + tables.push(table); + i = j - 1; + } + return tables; +} + +function tableRecords(rows: string[][], out: Field[][]): void { + const header = (rows[0] ?? []).map(normalizeKey); + for (const row of rows.slice(1)) if (row.some(c => c.trim())) out.push(header.map((h, i) => ({ name: h, key: h, value: (row[i] ?? '').trim() }))); +} + +/** The records of one supported file's text, or null when it cannot be checked (parse failure or depth cap). */ +export function fileRecords(file: string, text: string): Field[][] | null { + const body = text.replace(/^\uFEFF/, ''); + const ext = (/\.([a-z]+)$/i.exec(file)?.[1] ?? '').toLowerCase(); + const out: Field[][] = []; + try { + if (ext === 'json') return !body.trim() || jsonRecords(JSON.parse(body), out) ? out : null; + if (ext === 'jsonl' || ext === 'ndjson') { + for (const line of body.split(/\r?\n/)) if (line.trim() && !jsonRecords(JSON.parse(line), out)) return null; + return out; + } + } catch { return null; } + if (ext === 'md') { for (const t of markdownTables(body)) tableRecords(t, out); return out; } + const rows = delimitedRows(body, ext === 'tsv' ? '\t' : ','); + if (!rows) return null; + tableRecords(rows.filter(r => r.some(c => c.trim())), out); + return out; +} + +/** Joined-record fingerprints of one file's text; null when it cannot be checked. */ +export function fileJoins(file: string, text: string, p: JoinPolicy): Joins | null { + const folded = normalizeKey(text.replace(/"/g, '')); + if (!text.includes('\\u') && !p.label_keys.some(k => folded.includes(normalizeKey(k)))) return new Map(); + const records = fileRecords(file, text); + if (!records) return null; + const classify = classifier(p); + const joins: Joins = new Map(); + for (const r of records) { const j = classify(r); if (j) joins.set(j.fp, { labels: j.labels, ids: j.ids }); } + return joins; +} + +function gitBuffer(repoRoot: string, args: string[], input: string, label: string): Buffer { + const r = spawnSync('git', args, { cwd: repoRoot, input, timeout: 120_000, maxBuffer: 512 * 1024 * 1024 }); + if (r.status !== 0) throw new Error(`git ${label} failed: ${(r.stderr ?? '').toString().trim().slice(0, 200)}`); + return r.stdout; +} + +interface Change { commit: string; file: string; blob: string; parents: string[] } + +/** Supported regular files a commit adds, modifies, renames or copies against every parent (`-m`: a merge's file must differ from each parent). */ +function changedFiles(repoRoot: string, commits: string[]): Change[] { + const parentCount = new Map(gitBuffer(repoRoot, ['rev-list', '--no-walk', '--parents', '--stdin'], commits.join('\n') + '\n', 'rev-list --parents').toString('utf8').split('\n').filter(Boolean).map(l => { const s = l.split(' '); return [s[0]!, Math.max(1, s.length - 1)] as const; })); + const tokens = gitBuffer(repoRoot, ['-c', 'core.quotePath=false', 'diff-tree', '--stdin', '-r', '--root', '-m', '-M', '-C', '--raw', '-z', '--no-abbrev'], commits.join('\n') + '\n', 'diff-tree -m').toString('utf8').split('\0'); + const sections = new Map>>(); + let current: Map | null = null; + for (let i = 0; i < tokens.length; i++) { + const t = tokens[i]!; + if (!t) continue; + if (!t.startsWith(':')) { current = new Map(); sections.set(t, [...(sections.get(t) ?? []), current]); continue; } + const [, dstMode, src, dst, status] = t.slice(1).split(' ') as [string, string, string, string, string]; + const file = /^[RC]/.test(status) ? tokens[i += 2]! : tokens[++i]!; + if (status.startsWith('D') || !/^100(644|755)$/.test(dstMode) || !SUPPORTED.test(file)) continue; + current?.set(file, { src, dst }); + } + const out: Change[] = []; + for (const [commit, parts] of sections) { + if (parts.length < (parentCount.get(commit) ?? 1)) continue; + for (const [file, { dst }] of parts[0]!) { + if (parts.every(p => p.has(file))) out.push({ commit, file, blob: dst, parents: parts.map(p => p.get(file)!.src).filter(s => !ZERO.test(s)) }); + } + } + return out; +} + +/** Each blob's text by sha, handed to `use` as it is read (one `cat-file --batch-check`, then chunked `--batch` reads); null over the cap or when unreadable. */ +function readBlobs(repoRoot: string, shas: string[], use: (sha: string, text: string | null) => void): void { + if (!shas.length) return; + const small: Array<[string, number]> = []; + for (const line of gitBuffer(repoRoot, ['cat-file', '--batch-check'], shas.join('\n') + '\n', 'cat-file --batch-check').toString('utf8').split('\n').filter(Boolean)) { + const [sha, type, size] = line.split(' '); + if (type !== 'blob' || Number(size) > JOIN_MAX_BYTES) use(sha!, null); + else small.push([sha!, Number(size)]); + } + const decoder = new TextDecoder(); + for (let start = 0; start < small.length;) { + let end = start; let bytes = 0; + while (end < small.length && (end === start || bytes + small[end]![1] <= 64 * 1024 * 1024)) bytes += small[end++]![1]; + const buf = gitBuffer(repoRoot, ['cat-file', '--batch'], small.slice(start, end).map(s => s[0]).join('\n') + '\n', 'cat-file --batch'); + let pos = 0; + for (let k = start; k < end; k++) { + const nl = buf.indexOf(0x0a, pos); + const size = Number(buf.subarray(pos, nl).toString('utf8').split(' ')[2]); + use(small[k]![0], decoder.decode(buf.subarray(nl + 1, nl + 1 + size))); + pos = nl + 1 + size + 1; + } + start = end; + } +} + +/** `sealed.label_join` and `sealed.label_join_unchecked` findings for the published commits; [] without label_keys (no git reads). */ +export function joinFindings(repoRoot: string, commits: string[], p: JoinPolicy): SealedFinding[] { + if (!p.label_keys.length || !commits.length) return []; + const changes = changedFiles(repoRoot, commits); + const ext = (file: string) => (/\.[a-z]+$/i.exec(file)?.[0] ?? '').toLowerCase(); + const wanted = new Map>(); + for (const c of changes) for (const sha of [c.blob, ...c.parents]) wanted.set(sha, (wanted.get(sha) ?? new Set()).add(ext(c.file))); + const cache = new Map(); + readBlobs(repoRoot, [...wanted.keys()], (sha, text) => { for (const e of wanted.get(sha)!) cache.set(`${sha}\0${e}`, text === null ? null : fileJoins(`f${e}`, text, p)); }); + const joinsOf = (sha: string, file: string) => cache.get(`${sha}\0${ext(file)}`) ?? null; + const out: SealedFinding[] = []; + for (const c of changes) { + const now = joinsOf(c.blob, c.file); + if (!now) { out.push({ rule: 'sealed.label_join_unchecked', file: c.file, commit: c.commit }); continue; } + const before = new Set(c.parents.flatMap(s => [...(joinsOf(s, c.file)?.keys() ?? [])])); + const labels = new Set(); const ids = new Set(); + for (const [fp, j] of now) if (!before.has(fp)) { j.labels.forEach(k => labels.add(k)); j.ids.forEach(k => ids.add(k)); } + if (labels.size) out.push({ rule: 'sealed.label_join', file: c.file, commit: c.commit, keys: [...[...labels].sort(), ...[...ids].sort()] }); + } + return out; +} diff --git a/package.json b/package.json index 4f5009fa6..2a2d10ae8 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.92.71", + "version": "1.92.72", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/test/fixtures/module-size-ratchet.json b/test/fixtures/module-size-ratchet.json index 1bbc21be7..da9804538 100644 --- a/test/fixtures/module-size-ratchet.json +++ b/test/fixtures/module-size-ratchet.json @@ -161,6 +161,7 @@ { "file": "lib/pregate/checks/timing.ts", "movedFrom": "(new: pre-gate timing-assertion check, field list 2 W4)" }, { "file": "lib/pregate/runtime.ts", "movedFrom": "(new: the verified launch Bun for repo commands, field wave CD1)" }, { "file": "lib/sealed-guard.ts", "movedFrom": "(new: the sealed-output guard shared by the pre-push hook and gstack-pregate sealed, field wave CD4)" }, + { "file": "lib/sealed-join.ts", "movedFrom": "(new: the sealed.label_join rule, label fields published beside identifiers, GST-620)" }, { "file": "lib/pregate/regate.ts", "movedFrom": "(new: the regate verdict from a fresh tree receipt plus migration collisions, field wave CD3)" }, { "file": "lib/autoplan-scope.ts", "movedFrom": "(new: UI evidence from rendered surfaces and the plan scope marker, shared by the runner and the snapshot tool, field wave B2c)" }, { "file": "lib/source-root.ts", "movedFrom": "(new: the source checkout behind an install root, from its recorded .source-path, field wave B2e)" }, diff --git a/test/fixtures/sealed-guard-off-golden.json b/test/fixtures/sealed-guard-off-golden.json new file mode 100644 index 000000000..ebe386370 --- /dev/null +++ b/test/fixtures/sealed-guard-off-golden.json @@ -0,0 +1,65 @@ +{ + "refused": { + "code": 1, + "stdout": "SEALED: refused findings=2 commits=1 visibility=unknown (counted public) waived=0 exempt=0 policy=origin/main + commit \n", + "stderr": "\n⛔ gstack-pregate sealed BLOCKED the publication of /origin.git (visibility: unknown, counted public; not a GitHub remote) — sealed or eval output in 1 published commit(s):\n sealed.transcript runs/c1/transcript.jsonl commit \n sealed.answer runs/c1/summary.md:1 commit \nRemove it from every published commit before this branch reaches a public repository, exempt a publishable path with a sealed.allow_paths glob in the base branch's .gstack/pregate.json, or waive one reviewed finding with a `rulepathfull-commit-shareason` line in the base branch's .gstack/sealed-allow.tsv. A private destination: set sealed.visibility \"private\" in the base branch's .gstack/pregate.json (SEALED_OUTPUT_REFUSED)\n", + "jsonCode": 1, + "json": "{\"status\":\"refused\",\"policyLabels\":[\"origin/main\",\"commit \"],\"commits\":1,\"findings\":[{\"rule\":\"sealed.transcript\",\"file\":\"runs/c1/transcript.jsonl\",\"commit\":\"\"},{\"rule\":\"sealed.answer\",\"file\":\"runs/c1/summary.md\",\"line\":1,\"commit\":\"\"}],\"exempt\":[],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"not a GitHub remote\",\"target\":\"/origin.git\",\"ok\":false}\n", + "hookCode": 1, + "hookStderr": "\n⛔ gstack sealed-output guard BLOCKED the publication of /origin.git (visibility: unknown, counted public; not a GitHub remote) — sealed or eval output in 1 published commit(s):\n sealed.transcript runs/c1/transcript.jsonl commit \n sealed.answer runs/c1/summary.md:1 commit \nRemove it from every published commit before this branch reaches a public repository, exempt a publishable path with a sealed.allow_paths glob in the base branch's .gstack/pregate.json, or waive one reviewed finding with a `rulepathfull-commit-shareason` line in the base branch's .gstack/sealed-allow.tsv. A private destination: set sealed.visibility \"private\" in the base branch's .gstack/pregate.json (SEALED_OUTPUT_REFUSED)\n" + }, + "clear": { + "code": 0, + "stdout": "SEALED: clear findings=0 commits=1 visibility=unknown (counted public) waived=0 exempt=0 policy=origin/main + commit \n", + "stderr": "", + "jsonCode": 0, + "json": "{\"status\":\"clear\",\"policyLabels\":[\"origin/main\",\"commit \"],\"commits\":1,\"findings\":[],\"exempt\":[],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"not a GitHub remote\",\"target\":\"/origin.git\",\"ok\":true}\n", + "hookCode": 0, + "hookStderr": "" + }, + "exempt": { + "code": 0, + "stdout": "SEALED: clear findings=0 commits=1 visibility=unknown (counted public) waived=0 exempt=1 policy=origin/main + commit \n", + "stderr": "gstack-pregate sealed: exempt sealed.path sealed/s1/MANIFEST.json commit (sealed.allow_paths **/sealed/*/MANIFEST.json)\n", + "jsonCode": 0, + "json": "{\"status\":\"clear\",\"policyLabels\":[\"origin/main\",\"commit \"],\"commits\":1,\"findings\":[],\"exempt\":[{\"rule\":\"sealed.path\",\"file\":\"sealed/s1/MANIFEST.json\",\"commit\":\"\",\"glob\":\"**/sealed/*/MANIFEST.json\"}],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"not a GitHub remote\",\"target\":\"/origin.git\",\"ok\":true}\n", + "hookCode": 0, + "hookStderr": "gstack sealed-output guard: exempt sealed.path sealed/s1/MANIFEST.json commit (sealed.allow_paths **/sealed/*/MANIFEST.json)\n" + }, + "waived": { + "code": 0, + "stdout": "SEALED: clear findings=0 commits=1 visibility=unknown (counted public) waived=1 exempt=0 policy=origin/main + commit \n", + "stderr": "gstack-pregate sealed: waived sealed.transcript runs/c1/transcript.jsonl commit (.gstack/sealed-allow.tsv: synthetic fixture)\n", + "jsonCode": 0, + "json": "{\"status\":\"clear\",\"policyLabels\":[\"origin/main\",\"commit \"],\"commits\":1,\"findings\":[],\"exempt\":[],\"waived\":[{\"rule\":\"sealed.transcript\",\"file\":\"runs/c1/transcript.jsonl\",\"commit\":\"\",\"reason\":\"synthetic fixture\"}],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"not a GitHub remote\",\"target\":\"/origin.git\",\"ok\":true}\n", + "hookCode": 0, + "hookStderr": "gstack sealed-output guard: waived sealed.transcript runs/c1/transcript.jsonl commit (.gstack/sealed-allow.tsv: synthetic fixture)\n" + }, + "warned": { + "code": 0, + "stdout": "SEALED: warned findings=1 commits=1 visibility=private waived=0 exempt=0 policy=origin/main + commit \n", + "stderr": "gstack-pregate sealed: warning — sealed or eval output in a push to a private destination (visibility: private; sealed.visibility override); it would be refused for a public one:\n sealed.transcript runs/c1/transcript.jsonl commit \n", + "jsonCode": 0, + "json": "{\"status\":\"warned\",\"policyLabels\":[\"origin/main\",\"commit \"],\"commits\":1,\"findings\":[{\"rule\":\"sealed.transcript\",\"file\":\"runs/c1/transcript.jsonl\",\"commit\":\"\"}],\"exempt\":[],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"private\",\"visibilitySource\":\"sealed.visibility override\",\"target\":\"/origin.git\",\"ok\":true}\n", + "hookCode": 0, + "hookStderr": "gstack sealed-output guard: warning — sealed or eval output in a push to a private destination (visibility: private; sealed.visibility override); it would be refused for a public one:\n sealed.transcript runs/c1/transcript.jsonl commit \n" + }, + "error": { + "code": 1, + "stdout": "SEALED: error findings=0 commits=0 visibility=unknown (counted public) waived=0 exempt=0 policy=no sealed key on refs/remotes/origin/main or HEAD\n", + "stderr": "\n⛔ gstack-pregate sealed BLOCKED the publication of /origin.git — the sealed policy in commit is unreadable: sealed.answer_markers[0]: \"(\" is not a valid regex (Invalid regular expression: missing )).\nFix the sealed policy (gstack-pregate validate), then retry; an unreadable policy never passes (SEALED_OUTPUT_REFUSED)\n", + "jsonCode": 1, + "json": "{\"status\":\"error\",\"error\":\"the sealed policy in commit is unreadable: sealed.answer_markers[0]: \\\"(\\\" is not a valid regex (Invalid regular expression: missing ))\",\"policyLabels\":[],\"commits\":0,\"findings\":[],\"exempt\":[],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"\",\"target\":\"/origin.git\",\"ok\":false}\n", + "hookCode": 1, + "hookStderr": "\n⛔ gstack sealed-output guard BLOCKED the publication of /origin.git — the sealed policy in commit is unreadable: sealed.answer_markers[0]: \"(\" is not a valid regex (Invalid regular expression: missing )).\nFix the sealed policy (gstack-pregate validate), then retry; an unreadable policy never passes (SEALED_OUTPUT_REFUSED)\n" + }, + "no-policy": { + "code": 0, + "stdout": "SEALED: no-policy findings=0 commits=0 visibility=unknown (counted public) waived=0 exempt=0 policy=no sealed key on refs/remotes/origin/main or HEAD\n", + "stderr": "", + "jsonCode": 0, + "json": "{\"status\":\"no-policy\",\"policyLabels\":[],\"commits\":0,\"findings\":[],\"exempt\":[],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"\",\"target\":\"/origin.git\",\"ok\":true}\n", + "hookCode": 0, + "hookStderr": "" + } +} diff --git a/test/helpers/free-fixtures-data.ts b/test/helpers/free-fixtures-data.ts index 9d9131d47..fe6bb9e66 100644 --- a/test/helpers/free-fixtures-data.ts +++ b/test/helpers/free-fixtures-data.ts @@ -210,6 +210,7 @@ export const FREE_FIXTURES: Readonly> = { 'test/fixtures/review-design-lite-reports-ci-36633323521.json': ['test/review-finalization-budget.test.ts'], 'test/fixtures/review-gate-integrity-opus-reports.json': ['test/gate-integrity-templates.test.ts'], 'test/fixtures/review-wrong-tree/**': ['test/review-dispatch.test.ts'], + 'test/fixtures/sealed-guard-off-golden.json': ['test/sealed-guard.test.ts'], 'test/fixtures/session-update-heartbeat.sh': ['test/session-update-heartbeat.test.ts'], 'test/fixtures/shard-cli-parity/**': ['test/shard-cli-parity.test.ts'], 'test/fixtures/shard-equivalence/**': ['test/shard-engine-equivalence.test.ts'], diff --git a/test/sealed-guard.test.ts b/test/sealed-guard.test.ts index 878f36dc0..f93e1eaac 100644 --- a/test/sealed-guard.test.ts +++ b/test/sealed-guard.test.ts @@ -506,3 +506,64 @@ describe('waivers name the full commit sha (GST-268)', () => { expect(JSON.parse(preflight(s, ['--range', 'nosuch..HEAD', '--json']).out).ignoredWaivers).toEqual([]); }); }); + +const PANEL = JSON.stringify({ units: [ + { stratum: 'decoy', tags: ['W:T1'], plan_bytes: 3760, files: 5, head: '5b7db3aa' + '1'.repeat(32), tree_sha256: '15762a' + '0'.repeat(56) + 'aa' }, + { stratum: 'seeded', tags: ['B:T2'], plan_bytes: 4112, files: 7, head: '9c1e0bbb' + '2'.repeat(32), tree_sha256: '88aa11' + '0'.repeat(56) + 'bb' }, +] }, null, 2) + '\n'; +const PANEL_VALUES = [/5b7db3aa/, /15762a0/, /9c1e0bbb/, /88aa110/, /\b3760\b/, /\b4112\b/, /W:T1/, /B:T2/, /\bdecoy\b/, /\bseeded\b/]; + +describe('sealed.label_join: labels published beside self-computable identifiers (GST-620)', () => { + test('the GSTA-2 panel (stratum and tags beside head, tree_sha256, plan_bytes, files) refuses on a public destination and warns on a private one, naming fields, never values', () => { + const labeled = { paths: ['**/heldout*/**'], label_keys: ['stratum', 'tags', 'decoy_set'] }; + const s = scratch({ '.gstack/pregate.json': policy(labeled) }); + const sha = commit(s, { 'wiki/board/panel/panel.json': PANEL }); + const keys = ['stratum', 'tags', 'files', 'head', 'plan_bytes', 'tree_sha256']; + const text = preflight(s); + expect(text.code).toBe(1); + expect(text.err).toContain(`sealed.label_join wiki/board/panel/panel.json commit ${sha.slice(0, 12)} keys=${keys.join(',')}`); + expect(text.out).toMatch(/^SEALED: refused findings=1 commits=1 /m); + const json = preflight(s, ['--json']); + expect(JSON.parse(json.out).findings).toEqual([{ rule: 'sealed.label_join', file: 'wiki/board/panel/panel.json', commit: sha, keys }]); + const h = hook(s); + expect(h.code).toBe(1); + expect(h.err).toContain(`sealed.label_join wiki/board/panel/panel.json commit ${sha.slice(0, 12)} keys=${keys.join(',')}`); + for (const out of [text.out, text.err, json.out, h.err]) for (const value of PANEL_VALUES) expect(out).not.toMatch(value); + const priv = scratch({ '.gstack/pregate.json': policy({ ...labeled, visibility: 'private' }) }); + commit(priv, { 'wiki/board/panel/panel.json': PANEL }); + const p = preflight(priv); + expect(p.code).toBe(0); + expect(p.err).toMatch(/warning.*sealed\.label_join\s+wiki\/board\/panel\/panel\.json/s); + }); +}); + +const OFF_GOLDEN = path.join(ROOT, 'test', 'fixtures', 'sealed-guard-off-golden.json'); +/** Representative pushes under policies without label_keys, each read three ways: the preflight's text and --json, and the pre-push hook. */ +function offRuns(): Record { + const table = '| unit | stratum | head |\n|---|---|---|\n| u1 | decoy | 5b7db3a |\n'; + const fixtures: Record Scratch> = { + refused: () => { const s = scratch({ '.gstack/pregate.json': policy(SEALED) }); commit(s, { 'wiki/board/panel/panel.json': PANEL, 'runs/c1/transcript.jsonl': '{}\n', 'runs/c1/summary.md': 'SEALED ANSWER: x\n' + table }); return s; }, + clear: () => { const s = scratch({ '.gstack/pregate.json': policy({ paths: ['heldout/**'] }) }); commit(s, { 'wiki/board/panel/panel.json': PANEL, 'wiki/board.md': table, 'wiki/units.csv': 'stratum,head\ndecoy,5b7db3a\n' }); return s; }, + exempt: () => { const s = scratch({ '.gstack/pregate.json': policy(MANIFEST) }); commit(s, { 'sealed/s1/MANIFEST.json': PANEL, 'wiki/board/panel/panel.json': PANEL }); return s; }, + waived: () => { const s = scratch({ '.gstack/pregate.json': policy(SEALED) }); const sha = commit(s, { 'runs/c1/transcript.jsonl': '{}\n', 'wiki/panel.json': PANEL }); landOnMain(s, { '.gstack/sealed-allow.tsv': `sealed.transcript\truns/c1/transcript.jsonl\t${sha}\tsynthetic fixture\n` }); return s; }, + warned: () => { const s = scratch({ '.gstack/pregate.json': policy({ ...SEALED, visibility: 'private' }) }); commit(s, { 'runs/c1/transcript.jsonl': '{}\n', 'wiki/panel.json': PANEL }); return s; }, + error: () => { const s = scratch({ '.gstack/pregate.json': policy(SEALED) }); commit(s, { '.gstack/pregate.json': policy({ ...SEALED, answer_markers: ['('] }), 'wiki/panel.json': PANEL }); return s; }, + 'no-policy': () => { const s = scratch({}); commit(s, { 'wiki/panel.json': PANEL }); return s; }, + }; + const runs: Record = {}; + for (const [name, make] of Object.entries(fixtures)) { + const s = make(); + const norm = (t: string) => t.split(s.root).join('').replace(/[0-9a-f]{12,64}/g, ''); + const text = preflight(s); const json = preflight(s, ['--json']); const h = hook(s); + runs[name] = { code: text.code, stdout: norm(text.out), stderr: norm(text.err), jsonCode: json.code, json: norm(json.out), hookCode: h.code, hookStderr: norm(h.err) }; + } + return runs; +} + +describe('off unless configured (GST-620 item 13)', () => { + test('without sealed.label_keys, stdout, stderr, --json, the hook and every exit code match the capture from main 2d1b55d76 byte for byte', () => { + const runs = offRuns(); + if (process.env.SEALED_GUARD_GOLDEN_CAPTURE) { fs.writeFileSync(process.env.SEALED_GUARD_GOLDEN_CAPTURE, JSON.stringify(runs, null, 2) + '\n'); return; } + expect(runs).toEqual(JSON.parse(fs.readFileSync(OFF_GOLDEN, 'utf8'))); + }, 120_000); +}); diff --git a/test/sealed-join.test.ts b/test/sealed-join.test.ts new file mode 100644 index 000000000..3b92ce1a8 --- /dev/null +++ b/test/sealed-join.test.ts @@ -0,0 +1,237 @@ +/** + * GST-620: `sealed.label_join`, the sealed guard's rule for a published + * record that puts a label field (`sealed.label_keys`) next to an identifier + * the model under test can compute from its own copy of the work. The repro + * and the byte-for-byte "off" golden live in test/sealed-guard.test.ts; this + * file covers formats, the per-record content diff against every parent, + * fail-closed unchecked files, waivers, allow_paths and the monotonic policy. + */ +import { afterEach, describe, expect, test } from 'bun:test'; +import { spawnSync } from 'node:child_process'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { sealedErrors } from '../lib/sealed-guard'; +import { DEFAULT_IDENTIFIER_KEYS, joinFindings } from '../lib/sealed-join'; + +const ROOT = path.resolve(import.meta.dir, '..'); +const PREGATE = path.join(ROOT, 'bin', 'gstack-pregate'); +const HEAD_A = '5b7db3aa' + '1'.repeat(32); +const HEAD_B = '9c1e0bbb' + '2'.repeat(32); +const LABELED = { label_keys: ['stratum', 'tags', 'decoy_set'] }; +const P = { label_keys: ['stratum', 'tags'], identifier_keys: DEFAULT_IDENTIFIER_KEYS }; + +interface Scratch { root: string; work: string; env: Record } +const scratches: Scratch[] = []; +afterEach(() => { while (scratches.length) fs.rmSync(scratches.pop()!.root, { recursive: true, force: true }); }); + +function git(s: Scratch, args: string[], ok = true): string { + const r = spawnSync('git', args, { cwd: s.work, encoding: 'utf8', timeout: 30_000, env: { ...process.env, ...s.env } }); + if (ok && r.status !== 0) throw new Error(`git ${args.join(' ')}: ${r.stderr}`); + return (r.stdout ?? '').trim(); +} +function write(s: Scratch, files: Record): void { + for (const [rel, body] of Object.entries(files)) { + const abs = path.join(s.work, rel); + if (body === null) { fs.rmSync(abs, { force: true }); continue; } + fs.mkdirSync(path.dirname(abs), { recursive: true }); + fs.writeFileSync(abs, body); + } +} +function commit(s: Scratch, files: Record, msg = 'results'): string { + write(s, files); + git(s, ['add', '-A']); git(s, ['commit', '-q', '-m', msg]); + return git(s, ['rev-parse', 'HEAD']); +} +/** A bare origin whose main carries the base files, and a clone on branch `results`. */ +function scratch(baseFiles: Record): Scratch { + const root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'sealed-join-'))); + const work = path.join(root, 'work'); const origin = path.join(root, 'origin.git'); const state = path.join(root, 'state'); + fs.mkdirSync(work); fs.mkdirSync(state); + const env = { HOME: root, GSTACK_HOME: state, GSTACK_STATE_ROOT: state, GIT_CONFIG_GLOBAL: path.join(root, 'gitconfig'), GIT_CONFIG_NOSYSTEM: '1' }; + fs.writeFileSync(env.GIT_CONFIG_GLOBAL, '[user]\n\tname = t\n\temail = t@example.com\n[commit]\n\tgpgsign = false\n[init]\n\tdefaultBranch = main\n'); + const s = { root, work, env }; + scratches.push(s); + spawnSync('git', ['init', '-q', '--bare', origin], { env: { ...process.env, ...env }, timeout: 30_000 }); + git(s, ['init', '-q', '-b', 'main']); + commit(s, { 'README.md': '# evals\n', ...baseFiles }, 'base'); + git(s, ['remote', 'add', 'origin', origin]); git(s, ['push', '-q', 'origin', 'main']); + git(s, ['checkout', '-q', '-b', 'results']); + return s; +} +function landOnMain(s: Scratch, files: Record): void { + git(s, ['checkout', '-q', 'main']); commit(s, files, 'base change'); git(s, ['push', '-q', 'origin', 'main']); git(s, ['checkout', '-q', 'results']); +} +const policy = (sealed: unknown) => JSON.stringify({ sealed }, null, 2) + '\n'; +const panel = (units: object[]) => JSON.stringify({ units }, null, 2) + '\n'; +const unit = (name: string, head: string, extra: object = {}) => ({ name, stratum: name === 'u1' ? 'decoy' : 'seeded', tags: ['W:T1'], head, plan_bytes: 3760, ...extra }); +const found = (s: Scratch) => joinFindings(s.work, git(s, ['rev-list', 'origin/main..HEAD']).split('\n').filter(Boolean), P); +function preflight(s: Scratch, args: string[] = []) { + const r = spawnSync(PREGATE, ['sealed', '--base', 'main', ...args], { cwd: s.work, encoding: 'utf8', timeout: 60_000, env: { ...process.env, ...s.env } }); + return { code: r.status, out: r.stdout ?? '', err: r.stderr ?? '' }; +} + +describe('formats and matching', () => { + test('a JSONL line, a nested meta.head, CSV, TSV, a markdown table with human headers and a hex value under an unlisted key each join; decimal-only strings, labels alone and identifiers alone do not', () => { + const s = scratch({}); + const sha = commit(s, { + 'a.jsonl': '{"x":1}\n{"stratum":"decoy","meta":{"head":"5b7db3a"}}\n', + 'b.csv': 'unit,Stratum,Plan Bytes\nu1,decoy,"3,760"\n', + 'c.tsv': 'stratum\ttree_sha256\ndecoy\tabc\n', + 'd.md': '# board\n\n| Unit | Stratum | `head` | Tree SHA-256 |\n|---|:---:|---|---|\n| u1 | decoy | x | y |\n', + 'e.json': '[{"tags":["B:T2"],"id":"5b7db3a"}]\n', + 'f.json': '[{"stratum":"decoy","ts":"202610311200","name":"unit-1","count":12}]\n', + 'g.json': '[{"stratum":"decoy"},{"head":"5b7db3aa"}]\n', + 'h.md': '| head | size |\n|---|---|\n| 5b7db3a | 3 |\n', + 'i.txt': '{"stratum":"decoy","head":"5b7db3a"}\n', + 'j.json': `{"units":{"${HEAD_A}":{"stratum":"decoy","tags":["t"]}}}\n`, + 'k.jsonl': `{"${HEAD_B}":{"stratum":"seeded"}}\n`, + 'l.json': '{"units":{"unit-one":{"stratum":"decoy"}}}\n', + 'm.tsv': '"Stratum"\t"head"\n"decoy"\t"5b7db3a"\n', + 'n.csv': '"Strat"um,head\ndecoy,5b7db3a\n', + 'tsconfig.json': '{\n // a comment, so not strict JSON\n "compilerOptions": { "strict": true }\n}\n', + }); + expect(found(s).sort((a, b) => a.file.localeCompare(b.file))).toEqual([ + { rule: 'sealed.label_join', file: 'a.jsonl', commit: sha, keys: ['stratum', 'meta.head'] }, + { rule: 'sealed.label_join', file: 'b.csv', commit: sha, keys: ['stratum', 'plan_bytes'] }, + { rule: 'sealed.label_join', file: 'c.tsv', commit: sha, keys: ['stratum', 'tree_sha256'] }, + { rule: 'sealed.label_join', file: 'd.md', commit: sha, keys: ['stratum', 'head', 'tree_sha_256'] }, + { rule: 'sealed.label_join', file: 'e.json', commit: sha, keys: ['tags', 'id'] }, + { rule: 'sealed.label_join', file: 'j.json', commit: sha, keys: ['stratum', 'tags', '(key)'] }, + { rule: 'sealed.label_join', file: 'k.jsonl', commit: sha, keys: ['stratum', '(key)'] }, + { rule: 'sealed.label_join', file: 'm.tsv', commit: sha, keys: ['stratum', 'head'] }, + { rule: 'sealed.label_join', file: 'n.csv', commit: sha, keys: ['stratum', 'head'] }, + ]); + }); + + test('without label_keys the rule reads nothing: no git call, even outside a repository', () => { + expect(joinFindings(os.tmpdir(), ['0'.repeat(40)], { label_keys: [], identifier_keys: DEFAULT_IDENTIFIER_KEYS })).toEqual([]); + }); + + test('label_keys must be bare field names and identifier_keys allow only *; neither takes a dotted path; both must be string arrays', () => { + expect(sealedErrors({ label_keys: ['stratum'], identifier_keys: ['*_sha', 'head'] })).toEqual([]); + expect(sealedErrors({ label_keys: ['strat*'] }).map(e => e.path)).toEqual(['sealed.label_keys[0]']); + expect(sealedErrors({ identifier_keys: ['he?d', ''] }).map(e => e.path)).toEqual(['sealed.identifier_keys[0]', 'sealed.identifier_keys[1]']); + expect(sealedErrors({ label_keys: 'stratum' })).toEqual([{ path: 'sealed.label_keys', message: 'must be an array of strings' }]); + expect(sealedErrors({ label_keys: ['meta.stratum'], identifier_keys: ['meta.head'] }).map(e => e.path)).toEqual(['sealed.label_keys[0]', 'sealed.identifier_keys[0]']); + }); +}); + +describe('content diff against every parent', () => { + test('an unchanged record and a formatting-only edit stay clear; a replacement at the same count, a delete plus add and an added identifier field refuse', () => { + const s = scratch({ 'wiki/panel.json': panel([unit('u1', HEAD_A), unit('u2', HEAD_B)]) }); + const reformat = commit(s, { 'wiki/panel.json': JSON.stringify({ units: [unit('u1', HEAD_A), unit('u2', HEAD_B), { note: 'no labels here' }] }) + '\n' }); + const replaced = commit(s, { 'wiki/panel.json': panel([unit('u1', HEAD_A), unit('u2', 'c'.repeat(39) + '3')]) }); + const swapped = commit(s, { 'wiki/panel.json': panel([unit('u2', 'c'.repeat(39) + '3'), unit('u3', 'd'.repeat(39) + '4')]) }); + const widened = commit(s, { 'wiki/panel.json': panel([unit('u2', 'c'.repeat(39) + '3'), unit('u3', 'd'.repeat(39) + '4', { tree_sha256: 'e'.repeat(63) + '5' })]) }); + const commits = found(s).map(f => f.commit); + expect(commits).not.toContain(reformat); + expect(commits.sort()).toEqual([replaced, swapped, widened].sort()); + }); + + test('labels in one commit and identifiers added to the same records in the next: the second commit refuses', () => { + const s = scratch({}); + const labels = commit(s, { 'units.jsonl': '{"name":"u1","stratum":"decoy"}\n' }); + const ids = commit(s, { 'units.jsonl': '{"name":"u1","stratum":"decoy","head":"5b7db3a"}\n' }); + expect(found(s).map(f => f.commit)).toEqual([ids]); + expect(found(s).map(f => f.commit)).not.toContain(labels); + }); + + test('a pure rename is clear and a rename that adds a unit refuses; a clean merge of two published sides is clear and a resolution that adds a joined row refuses', () => { + const units = ['u1', 'u2', 'u3', 'u4', 'u5', 'u6'].map((n, i) => unit(n, String(i + 1).repeat(4) + 'ab' + 'f'.repeat(34))); + const s = scratch({ 'wiki/panel.json': panel(units) }); + git(s, ['mv', 'wiki/panel.json', 'wiki/board.json']); git(s, ['commit', '-q', '-m', 'rename']); + write(s, { 'wiki/board.json': panel([...units, unit('u7', HEAD_A)]) }); + git(s, ['mv', 'wiki/board.json', 'wiki/board2.json']); git(s, ['add', '-A']); git(s, ['commit', '-q', '-m', 'rename and add']); + const renamedAndAdded = git(s, ['rev-parse', 'HEAD']); + expect(found(s).map(f => f.commit)).toEqual([renamedAndAdded]); + + const m = scratch({ 'wiki/panel.json': panel([unit('u1', HEAD_A)]) }); + git(m, ['checkout', '-q', '-b', 'side']); + const side = commit(m, { 'wiki/panel.json': panel([unit('u1', HEAD_A), unit('u2', HEAD_B)]) }); + git(m, ['checkout', '-q', 'results']); + const ours = commit(m, { 'wiki/panel.json': panel([unit('u3', 'c'.repeat(39) + '3'), unit('u1', HEAD_A)]) }); + const merge = (resolution: object[]) => { + git(m, ['merge', '-q', '--no-ff', '--no-commit', 'side'], false); + write(m, { 'wiki/panel.json': panel(resolution) }); + git(m, ['add', '-A']); git(m, ['commit', '-q', '--no-edit', '-m', 'merge']); + return git(m, ['rev-parse', 'HEAD']); + }; + const clean = merge([unit('u3', 'c'.repeat(39) + '3'), unit('u1', HEAD_A), unit('u2', HEAD_B)]); + expect(found(m).map(f => f.commit).sort()).toEqual([side, ours].sort()); + expect(found(m).map(f => f.commit)).not.toContain(clean); + git(m, ['reset', '-q', '--hard', ours]); + const added = merge([unit('u3', 'c'.repeat(39) + '3'), unit('u1', HEAD_A), unit('u2', HEAD_B), unit('u4', 'd'.repeat(39) + '4')]); + const atMerge = found(m).filter(f => f.commit === added); + expect(atMerge).toEqual([{ rule: 'sealed.label_join', file: 'wiki/panel.json', commit: added, keys: ['stratum', 'tags', 'head', 'plan_bytes'] }]); + }); +}); + +describe('fail closed, waivers, allow_paths and the policy union (gstack-pregate sealed)', () => { + test('an oversized valid JSON, malformed JSON, a malformed JSONL line, nesting deeper than 64 and an unreadable blob are each sealed.label_join_unchecked; an empty file is clear', () => { + const s = scratch({ '.gstack/pregate.json': policy(LABELED) }); + let deep: unknown = { stratum: 'decoy' }; + for (let i = 0; i < 70; i++) deep = { next: deep }; + const sha = commit(s, { + 'big.json': JSON.stringify({ pad: 'x'.repeat(4 * 1024 * 1024 + 10), units: [] }), + 'bad.json': '{"stratum": "decoy",', + 'bad.jsonl': '{"stratum":"decoy"}\nnot json\n', + 'deep.json': JSON.stringify(deep), + 'gone.json': '{"note":"this blob is removed after the commit"}\n', + 'empty.json': '', + }); + const blob = git(s, ['rev-parse', 'HEAD:gone.json']); + fs.rmSync(path.join(s.work, '.git', 'objects', blob.slice(0, 2), blob.slice(2))); + const r = preflight(s); + expect(r.code).toBe(1); + for (const f of ['big.json', 'bad.json', 'bad.jsonl', 'deep.json', 'gone.json']) expect(r.err).toContain(`sealed.label_join_unchecked ${f} commit ${sha.slice(0, 12)}`); + expect(r.err).not.toContain('empty.json'); + expect(r.out).toMatch(/^SEALED: refused findings=5 commits=1 .* exempt=0 unchecked=5 policy=/m); + const json = JSON.parse(preflight(s, ['--json']).out); + expect(json.findings.map((f: { rule: string; file: string }) => `${f.rule} ${f.file}`).sort()).toEqual(['bad.json', 'bad.jsonl', 'big.json', 'deep.json', 'gone.json'].map(f => `sealed.label_join_unchecked ${f}`)); + }); + + test('allow_paths never exempt the rule; a short-sha waiver is ignored and the full sha waives', () => { + const s = scratch({ '.gstack/pregate.json': policy({ ...LABELED, paths: ['heldout/**'], allow_paths: ['wiki/board/panel.json'] }) }); + const sha = commit(s, { 'wiki/board/panel.json': panel([unit('u1', HEAD_A)]) }); + const r = preflight(s); + expect(r.code).toBe(1); + expect(r.out).toMatch(/findings=1 .* exempt=0 policy=/); + expect(r.err).not.toContain('sealed.allow_paths glob'); + landOnMain(s, { '.gstack/sealed-allow.tsv': `sealed.label_join\twiki/board/panel.json\t${sha.slice(0, 12)}\tsynthetic fixture\n` }); + const short = preflight(s); + expect(short.code).toBe(1); + expect(short.err).toContain('ignored waiver sealed.label_join wiki/board/panel.json'); + landOnMain(s, { '.gstack/sealed-allow.tsv': `sealed.label_join\twiki/board/panel.json\t${sha}\tsynthetic fixture\n` }); + const full = preflight(s); + expect(full.code).toBe(0); + expect(full.err).toContain('waived sealed.label_join wiki/board/panel.json'); + }); + + test('a push can enable or widen the rule but never disable or narrow it', () => { + const leak = { 'wiki/panel.json': panel([unit('u1', HEAD_A)]) }; + for (const pushed of [{ paths: ['heldout/**'] }, { ...LABELED, identifier_keys: ['zzz'] }, { label_keys: [] as string[], identifier_keys: [] as string[] }, { label_keys: [...LABELED.label_keys, 'head', 'plan_bytes'] }]) { + const s = scratch({ '.gstack/pregate.json': policy(LABELED) }); + commit(s, { '.gstack/pregate.json': policy(pushed), ...leak }); + const r = preflight(s); + expect(r.code).toBe(1); + expect(r.err).toContain('sealed.label_join wiki/panel.json'); + expect(r.err).toContain('head,plan_bytes'); + } + const enable = scratch({ '.gstack/pregate.json': policy({ paths: ['heldout/**'] }) }); + commit(enable, { '.gstack/pregate.json': policy(LABELED), ...leak }); + expect(preflight(enable).err).toContain('sealed.label_join wiki/panel.json'); + }); + + test('label_keys: [] reports exactly what an absent label_keys reports', () => { + const run = (sealed: object) => { + const s = scratch({ '.gstack/pregate.json': policy(sealed) }); + commit(s, { 'wiki/panel.json': panel([unit('u1', HEAD_A)]), 'runs/c1/transcript.jsonl': '{}\n' }); + const norm = (t: string) => t.split(s.root).join('').replace(/[0-9a-f]{12,64}/g, ''); + const text = preflight(s); const json = preflight(s, ['--json']); + return [text.code, norm(text.out), norm(text.err), json.code, norm(json.out)]; + }; + const base = { transcripts: ['runs/**/transcript*.jsonl'] }; + expect(run({ ...base, label_keys: [] })).toEqual(run(base)); + }); +}); From a00149e86eddf0f5f5387185c9214ef285a468b9 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Mon, 2 Nov 2026 03:45:58 -0800 Subject: [PATCH 02/19] v1.92.73.0 fix(review): read printed: fingerprint lines, stop on a foreign start checkout, give Step 11 its start token (GST-776) (#3798) * fix(review): readers start their reply with the bare REVIEWED_TREE line; a printed: label is read (GST-776) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): the Capy pass program refuses a foreign start checkout and takes Step 11's adversarial-review start (GST-776) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): a start directory outside any checkout still dispatches; the Capy skill's Step 11 note fits its context budget (GST-776) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): the start probe stops on an unreadable start directory, runs once per stage, and logStart is validated; brief wording stays main's (GST-776) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): GST-776 review dispatch fixes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review): a wholly bold printed: REVIEWED_TREE line is read (GST-776 fix-check) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review): the GBRA-218 one-line shape (echoed command, then printed: REVIEWED_TREE) is read; a wrong tree is still refused (GST-776) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): the logStart STOP text no longer names gstack-review-log --start ahead of the pass call (reader-drift order test, GST-776) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(budget): capy/skill/gstack ceiling 5316 after the main merge Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Revert "test(budget): capy/skill/gstack ceiling 5316 after the main merge" This reverts commit 61a9051b4ac1165d36e1a0aa2266ecc10674725a. * docs(capy): trim the /ship paragraph to stay under the 5315 context ceiling after the main merge Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 8 ++++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- capy/skill/gstack/SKILL.md | 2 +- docs/review-policy.md | 9 +++- lib/capy-review-pass.js | 14 +++++- lib/review-brief.ts | 7 ++- package.json | 2 +- test/fixtures/context-budget.json | 2 +- test/review-dispatch.test.ts | 39 ++++++++++++++++ test/review-pass.test.ts | 76 ++++++++++++++++++++++++++----- 11 files changed, 144 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 237f5c19e..0d90faced 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,13 @@ # Changelog +## [1.92.73.0] - 2026-11-02 + +### Fixed + +- **A reviewer that wrote `printed: REVIEWED_TREE: …` is no longer failed.** Two readers in GBRA-218 echoed the command and then wrote the fingerprint after a `printed:` label, so /ship read no line, failed them and restarted the pass twice. The label is now read, plain, bold or inside the line's backticks, and the tree and checkout it names are still compared exactly. A wrong tree or checkout is still refused. +- **The Capy pass program stops before it sends a reader into the wrong checkout.** Capy starts every shared reader in the thread's working directory and cannot start one anywhere else. In GBRA-218 an outside voice ignored its `cd` and reviewed that checkout, so Step 11 lost its outside coverage. Now, before a stage takes a start token or sends its first reader, the program compares that directory with the checkout /ship ships. It stops when they differ or the directory can't be read. **Run /ship in the thread's own checkout**, not in a worktree beside it. A start directory outside any checkout, such as a multi-repo workspace root, still dispatches. +- **Step 11 takes the right start token.** Reusing the pass input's `logStart: ["review"]` for `stage: "voices"` used to run the voices with no `adversarial-review` start, which left Step 11.5 nothing to bind. Step 11 now takes `adversarial-review` by default and stops before anything runs when given any other list. The gstack Capy skill says Step 11 swaps `logStart` for `stage: "voices"`. + ## [1.92.72.0] - 2026-11-02 **The sealed guard can now refuse a published record that puts a label next to an identifier.** gstack-evals hides which test units are decoys. Its /ship on garrytan/gstack-evals#99 published a panel where each unit's `stratum`, `tags` and `decoy_set` sat next to its `head`, `tree_sha256`, `plan_bytes` and `files`, values a model under test can compute from its own copy of the work. A model could look itself up and learn it was the decoy. No answer leaked, so `gstack-pregate sealed` said `clear` (GSTA-2, GST-620). The new rule is opt-in. A repo that sets `sealed.label_keys` gets it, and every other repo sees the same output as before, byte for byte. diff --git a/VERSION b/VERSION index 8d5fb4735..4633cc70e 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.92.72.0 +1.92.73.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index 5c5850c4e..a86825dff 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.92.72.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.92.73.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/capy/skill/gstack/SKILL.md b/capy/skill/gstack/SKILL.md index bb617b0c0..1e3263109 100644 --- a/capy/skill/gstack/SKILL.md +++ b/capy/skill/gstack/SKILL.md @@ -46,7 +46,7 @@ Capy checklist for `/autoplan`: drive `gstack-autoplan`, never a hand adaptation Token caps on /ship: pass a brief's cap as `--token-cap `. /ship reads `cost.tokens` and `cost.credits` from `tools.thread.get({ thread_id: "" })` (its own jam; a subagent reading root `"this"` starts `--subagent --reading-from this --meter-scope tree`) inside the same execute program as the `gstack-ship-budget` command that takes them as `--reading --credits `, never alone. `ship/DRIVER.md` starts `--mode observe`: each program ends with `note`, and only a note past the cap stops (`docs/ship-budget.md`). -/ship on Capy is about a dozen calls, each an execute program (`ship/DRIVER.md`). Each review pass and Step 11 is one call to the checked-in pass program (fill only `input`): `const t0 = await now(); const p = (await tools.capy.bash({ command: ". '' && gstack-review-dispatch program", description: "Pass" })).output.split("\n"); return p[0] + "\n" + (await new (Object.getPrototypeOf(async () => {}).constructor)("tools", "sleep", "now", "input", JSON.parse(p[1]))(tools, sleep, now, { startedAt: t0, launch: "", cwd: "", base: "", out: "/p", pass: "-p", laneModel: "", planFlags: [], logStart: ["review"], jam: "", record: "", step: "9.2" }));` It plans, logs `logStart` (`reservation` mode from `RESERVATION_ID:`), sends each brief on `machine: "shared"`, then polls: one `tools.subagent.list` per ~20 s tick, returning 200 s after the program's first line, where a reader is settled only at status `done` or `failed` or with a `summary` (a just-created `idle` reader has not run); it starts readers under `rejected` with `tools.subagent.start` as slots free and stops on one stuck 20 minutes. It ends on `NEXT:` (`poll`: rerun) or `STOP:`. `models` maps roles to models (`outside-voice`: another family, pinned in `voice-models.json`); Step 11 adds `stage: "voices"`; a full repeat pass, `diffBase: ""`. Steps 7, 8 and 14.5 (`step: "14.5"`) use `extras`, JSON `{name, prompt_file, first, model}` (prompts verbatim from their sections, Step 7 `first`); they cannot block, so `model: { model_id: "anthropic/claude-sonnet-5-5", reasoning_mode: "low" }` (14.5 once). Waits: never let 5 minutes pass between calls. While readers run, never end the turn on `wait`; rerun the loader. Else re-issue `wait` with `duration_seconds: 200` and operation ids. At Step 22 report the `CALLS:` line of `gstack-ship-budget calls --events ` on your transcript since /ship began. A small waiting child saves nothing. +/ship on Capy is about a dozen calls, each an execute program (`ship/DRIVER.md`). Each review pass and Step 11 is one call to the checked-in pass program (fill only `input`): `const t0 = await now(); const p = (await tools.capy.bash({ command: ". '' && gstack-review-dispatch program", description: "Pass" })).output.split("\n"); return p[0] + "\n" + (await new (Object.getPrototypeOf(async () => {}).constructor)("tools", "sleep", "now", "input", JSON.parse(p[1]))(tools, sleep, now, { startedAt: t0, launch: "", cwd: "", base: "", out: "/p", pass: "-p", laneModel: "", planFlags: [], logStart: ["review"], jam: "", record: "", step: "9.2" }));` It plans, logs `logStart` (`reservation` mode from `RESERVATION_ID:`), sends each brief on `machine: "shared"`, then polls: one `tools.subagent.list` per ~20 s tick, returning 200 s after the program's first line, where a reader is settled only at status `done` or `failed` or with a `summary` (a just-created `idle` reader has not run); it starts readers under `rejected` with `tools.subagent.start` as slots free and stops on one stuck 20 minutes. It ends on `NEXT:` (`poll`: rerun) or `STOP:`. `models` maps roles to models (`outside-voice`: another family, pinned in `voice-models.json`); Step 11 swaps `logStart` for `stage: "voices"`; a full repeat pass, `diffBase: ""`. Steps 7, 8 and 14.5 (`step: "14.5"`) use `extras`, JSON `{name, prompt_file, first, model}` (prompts verbatim from their sections, Step 7 `first`); they cannot block, so `model: { model_id: "anthropic/claude-sonnet-5-5", reasoning_mode: "low" }` (14.5 once). Waits: never let 5 minutes pass between calls. While readers run, never end the turn on `wait`; rerun the loader. Else re-issue `wait` with `duration_seconds: 200` and operation ids. At Step 22 report the `CALLS:` line of `gstack-ship-budget calls --events ` on your transcript since /ship began. A small waiting child saves nothing. ## Plan, approve, ship diff --git a/docs/review-policy.md b/docs/review-policy.md index 2cf6a453b..c8238d7d6 100644 --- a/docs/review-policy.md +++ b/docs/review-policy.md @@ -593,7 +593,14 @@ ones, and the arm lost blockers. `bin/gstack-review-dispatch` closes both ends: `COLLECT_WRONG_TREE`, a later commit in the same checkout `COLLECT_TREE_MOVED`; nothing is merged, the reader becomes `failed`, and `verify` reports `NOT_COLLECTED`. An outside voice's tree failure is recorded `unavailable` only when its reply holds no finding. The line catches a reader that - worked elsewhere by accident; it is not proof of reading, since diff text can predict it. + worked elsewhere by accident; it is not proof of reading, since diff text can predict it. A `printed:` label + before the line (plain, bold or inside its backticks) is still read, and its tree and toplevel are still compared + exactly (GST-776). Capy cannot start a subagent in a chosen directory, so before a stage takes a start token or sends + its first reader, the Capy pass program compares the checkout with the one shared readers start in (the thread's + working directory) and stops when that is another checkout or cannot be read: run /ship in the thread's own checkout. + A start directory outside any checkout, such as a multi-repo workspace root, holds no tree to review by mistake and + dispatches. Step 11's voices take the `adversarial-review` start + token by default, and a `logStart` list without it stops before anything runs. ## Where this is enforced diff --git a/lib/capy-review-pass.js b/lib/capy-review-pass.js index 9511695e9..886976f34 100644 --- a/lib/capy-review-pass.js +++ b/lib/capy-review-pass.js @@ -30,6 +30,9 @@ const finish = (tail) => { return all.slice(0, -1).join('\n').slice(0, Math.max(0, RETURN_BYTES - last.length)) + last; }; const stop = (why) => finish(['STOP: ' + why]); +const logStart = input.stage === 'voices' ? (input.logStart ?? ['adversarial-review']) : (input.logStart || []); +if (!Array.isArray(logStart) || !logStart.every((s) => typeof s === 'string' && s.trim())) return stop('input.logStart must be a list of review-log start skills (strings), got ' + JSON.stringify(input.logStart) + '; nothing ran'); +if (input.stage === 'voices' && !(logStart.length && logStart.every((s) => s.split(' ')[0] === 'adversarial-review'))) return stop('Step 11 (stage "voices") takes the adversarial-review start token, which Step 11.5 binds, so its logStart is ["adversarial-review"] (with any --reservation in input.reservation) or omitted, never ' + JSON.stringify(input.logStart) + '; nothing ran'); const runDir = input.out.replace(/\/+$/, '').replace(/\/[^/]+$/, ''); const nextPhase = input.stage === 'voices' ? 5 : input.step === '14.5' ? 7 : 4; const withParts = async (tail) => { @@ -133,8 +136,17 @@ try { if (voices.laneModel) laneFamily = family(voices.laneModel); } if (input.stage === 'voices' && !m.readers.some((r) => r.stage === 'voices')) return finish(['VOICES: none (the pass plan lists no Step 11 voice)', 'NEXT: judge']); + if (input.cwd && !m.readers.some((r) => active.includes(r.stage) && !['planned', 'rendered'].includes(r.state))) { + const top = 'if d=$(git rev-parse --show-toplevel 2>&1); then cd "$d" && pwd -P; else case "$d" in *"not a git repository"*) echo NOT_A_CHECKOUT ;; *) echo "$d"; exit 1 ;; esac; fi'; + const ship = await sh(top); + const start = await tools.capy.bash({ command: top, description: 'Ship review pass ' + input.pass + ' reader start checkout', timeout: 300 }); + const want = ship.out.trim(), got = (start.output || '').trim(); + if (ship.code !== 0 || want === 'NOT_A_CHECKOUT') return stop('cannot read the checkout input.cwd names (' + input.cwd + '): ' + want.slice(-300)); + if (start.exit_code !== 0) return stop('cannot read the directory readers on the shared machine start in: ' + got.slice(-300) + '; no start token was taken and no reader was dispatched'); + if (got !== 'NOT_A_CHECKOUT' && got !== want) return stop('readers on the shared machine start in ' + got + ', not in ' + want + ', the checkout this pass ships, and Capy cannot start a subagent in another directory; no start token was taken and no reader was dispatched. Run /ship in ' + got + " (the thread's working directory) on this branch, with a fresh --out"); + } const seen = {}; - for (const s of input.logStart || []) { + for (const s of logStart) { const nth = (seen[s] = (seen[s] || 0) + 1); const key = (input.stage === 'voices' ? 'voices:' : 'pass:') + s + (nth > 1 ? '#' + nth : ''); if ((m.log_starts || {})[key]) continue; diff --git a/lib/review-brief.ts b/lib/review-brief.ts index b233aa40b..30a9565f3 100644 --- a/lib/review-brief.ts +++ b/lib/review-brief.ts @@ -172,12 +172,17 @@ export function checkoutRoot(): string { } /** - * One REVIEWED_TREE line as " ", or null. A list marker or heading mark may lead it, and a whole-line `**…**` or + * One REVIEWED_TREE line as " ", or null. A list marker, heading mark or `printed:` label (GST-776, also after the echoed `cd … && echo …` command on the same line) may lead it, and a whole-line `**…**` or * backtick wrapper, a bold label and backticks around the sha or the whole path are removed; the path's own characters are kept. */ function parseTreeLine(line: string): string | null { + const label = /^(?:\*\*[Pp]rinted:\*\*|[Pp]rinted:)\s+/; let t = line.trim().replace(/^(?:[-*>]\s+|\d+[.)]\s+|#+\s+)/, ''); + const echoed = /\s(?:\*\*[Pp]rinted:\*\*|[Pp]rinted:)\s+(?=\S)/.exec(t); + if (echoed && t.startsWith('cd ')) t = t.slice(echoed.index + 1); + t = t.replace(label, ''); for (const w of ['**', '`']) if (t.length > 2 * w.length && t.startsWith(w) && t.endsWith(w)) t = t.slice(w.length, -w.length).trim(); + t = t.replace(label, ''); const m = /^(?:\*\*REVIEWED_TREE:\*\*|REVIEWED_TREE:)\s+`?([0-9a-f]{40,64})`?\s+(.+)$/.exec(t); if (!m) return null; const p = m[2]!.length > 2 && m[2]!.startsWith('`') && m[2]!.endsWith('`') ? m[2]!.slice(1, -1) : m[2]!; diff --git a/package.json b/package.json index 2a2d10ae8..ad4e3a294 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.92.72", + "version": "1.92.73", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/test/fixtures/context-budget.json b/test/fixtures/context-budget.json index 5f0d85790..619885589 100644 --- a/test/fixtures/context-budget.json +++ b/test/fixtures/context-budget.json @@ -9,7 +9,7 @@ "browser-skills/hackernews-frontpage": 371, "canary": 7738, "captain": 6551, - "capy/skill/gstack": 5312, + "capy/skill/gstack": 5315, "careful": 1239, "codex": 9890, "context-restore": 6287, diff --git a/test/review-dispatch.test.ts b/test/review-dispatch.test.ts index f6874e886..d0c01e345 100644 --- a/test/review-dispatch.test.ts +++ b/test/review-dispatch.test.ts @@ -794,3 +794,42 @@ describe('reading the REVIEWED_TREE line (GST-482 review fixes)', () => { expect(r.stderr).toContain('BRIEF_RENDER_FAILED: cannot read HEAD^{tree} and the toplevel of'); }); }); + +describe('the REVIEWED_TREE line as GBRA-218 readers wrote it (GST-776)', () => { + const top = fs.realpathSync(spawnSync('git', ['rev-parse', '--show-toplevel'], { cwd: ROOT, encoding: 'utf8', timeout: 30_000 }).stdout.trim()); + const tree = spawnSync('git', ['rev-parse', 'HEAD^{tree}'], { cwd: ROOT, encoding: 'utf8', timeout: 30_000 }).stdout.trim(); + const command = `cd '${top}' && echo "REVIEWED_TREE: $(git rev-parse 'HEAD^{tree}') $(git rev-parse --show-toplevel)"`; + // Value: protects=a reader that reviewed the right checkout is not failed for how it wrote its fingerprint; fails_when=a reply that echoes the command and then writes `printed: REVIEWED_TREE: …` (two Sonnet 5.5 low readers) is failed as reported="none", or a `printed:` label lets a wrong tree, toplevel or other label through; + // why_new=GBRA-218 pass restarts on gstack 1.92.25.0; seam=lib/review-brief.ts checkReviewedTree via gstack-review-dispatch collect; no_claim=the brief's wording, which stays main's until a no-loss replay + test('the echoed command then a printed: line is read; the same shape with a wrong tree or checkout is still refused', () => { + const reply = (fingerprint: string) => `${command}\nprinted: REVIEWED_TREE: ${fingerprint}\nNO FINDINGS\n`; + const oneLine = (fingerprint: string) => `${command} printed: REVIEWED_TREE: ${fingerprint}\n\nNO FINDINGS\n`; + expect(checkReviewedTree(oneLine(`${tree} ${top}`), `${tree} ${top}`)).toMatchObject({ ok: true }); + expect(checkReviewedTree(oneLine(`${'b'.repeat(40)} ${top}`), `${tree} ${top}`)).toMatchObject({ ok: false, moved: true }); + expect(checkReviewedTree(oneLine(`${tree} /elsewhere`), `${tree} ${top}`)).toMatchObject({ ok: false, moved: false }); + expect(checkReviewedTree(`${command} Note: REVIEWED_TREE: ${tree} ${top}`, `${tree} ${top}`).reported).toBe('none'); + expect(checkReviewedTree(reply(`${tree} ${top}`), `${tree} ${top}`)).toMatchObject({ ok: true }); + expect(checkReviewedTree(reply(`${'b'.repeat(40)} ${top}`), `${tree} ${top}`)).toMatchObject({ ok: false, moved: true, reported: `${'b'.repeat(40)} ${top}` }); + expect(checkReviewedTree(reply(`${tree} /elsewhere`), `${tree} ${top}`)).toMatchObject({ ok: false, moved: false, reported: `${tree} /elsewhere` }); + expect(checkReviewedTree(`Note: REVIEWED_TREE: ${tree} ${top}\nNO FINDINGS`, `${tree} ${top}`).reported).toBe('none'); + for (const line of ['`printed: REVIEWED_TREE: ' + tree + ' ' + top + '`', `**Printed:** REVIEWED_TREE: ${tree} ${top}`, `**printed: REVIEWED_TREE: ${tree} ${top}**`]) { + expect(checkReviewedTree(line, `${tree} ${top}`).ok, line).toBe(true); + expect(checkReviewedTree(line.replace(tree, 'c'.repeat(40)), `${tree} ${top}`).ok, line).toBe(false); + } + + const out = tmp(); + const ledger = path.join(out, 'review-ledger.jsonl'); + const merged = path.join(out, 'merged.jsonl'); + const file = field(run(['render', '--role', 'specialist', '--name', 'testing', '--out', out, '--diff-base', SHA]).stdout, 'file'); + run(['sent', '--ledger', ledger, '--role', 'specialist', '--name', 'testing', '--sent', file]); + const collectReply = (text: string) => { + const f = path.join(out, 'reply.out'); + fs.writeFileSync(f, text); + return spawnSync(BIN, ['collect', '--ledger', ledger, '--role', 'specialist', '--name', 'testing', '--output', f, '--merged', merged], { cwd: ROOT, encoding: 'utf8', timeout: 60_000 }); + }; + const wrong = collectReply(reply(`${'b'.repeat(40)} ${top}`)); + expect([wrong.status, wrong.stdout.startsWith('COLLECT_TREE_MOVED: reader=specialist/testing')]).toEqual([1, true]); + const right = collectReply(reply(`${tree} ${top}`)); + expect(right.stdout).toStartWith('COLLECT: reader=specialist/testing findings=0 '); + }); +}); diff --git a/test/review-pass.test.ts b/test/review-pass.test.ts index dc367ddc0..10ffa4f8d 100644 --- a/test/review-pass.test.ts +++ b/test/review-pass.test.ts @@ -332,6 +332,27 @@ describe('Capy pass program (fake host, real bins)', () => { expect(h2.starts).toEqual([[1]]); }, 240_000); + // Value: protects=Step 11.5 always has an adversarial-review start token to bind; fails_when=Step 11 reuses the pass input's logStart: ["review"] and runs the voices with no adversarial-review start, or a missing token is found only after the voices ran; + // why_new=GBRA-218 reran a whole pass after the loader example's logStart was reused for stage voices; seam=lib/capy-review-pass.js log starts; no_claim=gstack-review-read's binding itself + test('Step 11 takes the adversarial-review start token by default and refuses a list without it before anything runs', async () => { + const { dir, env } = repo(); + const out = path.join(dir, '.state/run/p1'); + const h = host(dir, env); + const models = { 'outside-voice': { model_id: 'openai/gpt-6.1-sol' } }; + const base = { base: 'main', out, pass: 'x12p1', planFlags: ['--depth-floor', 'full'], logStart: ['review'] }; + expect(await runPass(h, base)).toContain('NEXT: judge (Phase 4)'); + const created = h.creates.length; + const reused = await runPass(h, { ...base, stage: 'voices', models }); + expect(reused).toStartWith('STOP: Step 11 (stage "voices") takes the adversarial-review start token'); + expect(h.creates.length).toBe(created); + const voices = await runPass(h, { ...base, logStart: undefined, stage: 'voices', models }); + expect(voices).toMatch(/REVIEW_LOG_START: adversarial-review token=\S+/); + expect(voices.indexOf('REVIEW_LOG_START: adversarial-review')).toBeLessThan(voices.indexOf('DISPATCHED:')); + expect(voices).toContain('NEXT: judge the voices'); + const m = JSON.parse(fs.readFileSync(path.join(out, 'pass-manifest.json'), 'utf8')); + expect(Object.keys(m.log_starts ?? {}).sort()).toEqual(['pass:review', 'voices:adversarial-review']); + }, 240_000); + test('Step 11 voices run from the same manifest; the outside voice needs a model of another family', async () => { const { dir, env } = repo(); const out = path.join(dir, '.state/run/p1'); @@ -630,11 +651,10 @@ describe('Capy pass program (fake host, real bins)', () => { // why_new=replay C2-34 (GSTA-193) on PR 2; seam=none; no_claim=the host's status timing beyond one late summary test('extras as JSON, the checkout as cwd, and a late summary all run through without a STOP', async () => { const { dir, env } = repo(); - const elsewhere = tmp('rp-cwd-'); const out = path.join(dir, '.state/run/p1'); fs.mkdirSync(out, { recursive: true }); fs.writeFileSync(path.join(out, 'plan.prompt.md'), 'Plan completion brief'); - const h = host(elsewhere, env, { lateSummary: true }); + const h = host(dir, env, { lateSummary: true }); const res = await runPass(h, { base: 'main', out, pass: 'x13p1', cwd: dir, planFlags: ['--depth-floor', 'full'], extras: [{ name: 'plan', prompt_file: path.join(out, 'plan.prompt.md'), model: { model_id: 'anthropic/claude-haiku-5-5', reasoning_mode: 'low' } }] }); expect(res).not.toContain('STOP:'); @@ -914,12 +934,46 @@ describe('readers on a shared machine review the checkout the pass ships (GST-48 sh('for i in $(seq 1 20); do echo "export const w$i = $i;" >> src/a.ts; done && git commit -qam allow', wt); return { dir, env, wt: fs.realpathSync(wt) }; } + // Value: protects=no reader is started where Capy will put it (the shared machine's start checkout) when /ship ships another checkout; fails_when=the program dispatches readers that start in the primary checkout while the pass ships a worktree (GBRA-218's outside voice ignored `cd &&` and reviewed the primary checkout, losing Step 11 outside coverage); + // why_new=GBRA-218 on gstack 1.92.25.0; Capy subagents take no working directory; seam=lib/capy-review-pass.js before any log start or create; no_claim=that Capy can start a reader in a chosen directory + test('a shared machine whose start checkout is not the worktree refuses before any start token or reader; a start directory outside any checkout dispatches', async () => { + const { dir, env, wt } = worktree(); + const out = path.join(wt, '.state/run/p1'); + const h = host(dir, env); + const res = await runPass(h, { base: 'main', out, pass: 'w0p1', cwd: wt, planFlags: ['--depth-floor', 'full'], logStart: ['review'] }); + expect(res).toContain(`STOP: readers on the shared machine start in ${fs.realpathSync(dir)}, not in ${wt}`); + expect(res).not.toContain('REVIEW_LOG_START:'); + expect(h.creates.length).toBe(0); + const voices = await runPass(h, { base: 'main', out, pass: 'w0p1', cwd: wt, planFlags: ['--depth-floor', 'full'], stage: 'voices', models: { 'outside-voice': { model_id: 'openai/gpt-6.1-sol' } } }); + expect(voices).toContain('STOP: readers on the shared machine start in '); + expect(h.creates.length).toBe(0); + const started = host(wt, env); + expect(await runPass(started, { base: 'main', out: path.join(wt, '.state/run/p2'), pass: 'w0p2', cwd: wt, planFlags: ['--depth-floor', 'full'] })).toContain('NEXT: judge (Phase 4)'); + expect(started.creates.length).toBeGreaterThan(0); + const outside = host(path.dirname(wt), env); + expect(await runPass(outside, { base: 'main', out: path.join(wt, '.state/run/p3'), pass: 'w0p3', cwd: wt, planFlags: ['--depth-floor', 'full'] })).toContain('NEXT: judge (Phase 4)'); + const broken = tmp('rp-broken-'); + fs.writeFileSync(path.join(broken, '.git'), 'garbage\n'); + const unreadable = host(broken, env); + expect(await runPass(unreadable, { base: 'main', out: path.join(wt, '.state/run/p4'), pass: 'w0p4', cwd: wt, planFlags: ['--depth-floor', 'full'] })).toContain('STOP: cannot read the directory readers on the shared machine start in: fatal: invalid gitfile format'); + expect(unreadable.creates.length).toBe(0); + const slow = host(wt, env, { pollsToFinish: 30 }); + let probes = 0; + const bash = slow.tools.capy.bash; + slow.tools.capy.bash = async (a) => { if (a.command.startsWith('if d=$(git rev-parse')) probes++; return bash(a); }; + const input = { base: 'main', out: path.join(wt, '.state/run/p5'), pass: 'w0p5', cwd: wt, planFlags: ['--depth-floor', 'full'] }; + expect(await runPass(slow, input)).toContain('NEXT: poll'); + expect(await runPass(slow, input)).toContain('NEXT: poll'); + expect(probes).toBe(1); + expect(await runPass(host(wt, env), { ...input, logStart: [7] })).toStartWith('STOP: input.logStart must be a list'); + }, 240_000); + // Value: protects=a reader that never left the machine's default checkout cannot hand /ship a review of another tree; fails_when=collect merges a reply whose REVIEWED_TREE names the primary checkout while the pass ships a worktree, or a reply with no such line; // why_new=GBRA-218: pass 1's specialist reviewed /workspace/gbrain while /ship shipped ~/.capy/work/wt-allow; seam=gstack-review-dispatch collect; no_claim=a reader that fingerprints the right checkout but reads another test('a reader that reviews the primary checkout instead of the worktree is refused and nothing it said is merged', async () => { const { dir, env, wt } = worktree(); const out = path.join(wt, '.state/run/p1'); - const h = host(dir, env, { reportFrom: () => dir, output: t => (t.includes('specialist') ? '[CRITICAL] (confidence: 9/10) src/other.ts:3 a file outside this PR' : cleanReply(t)) }); + const h = host(wt, env, { reportFrom: () => dir, output: t => (t.includes('specialist') ? '[CRITICAL] (confidence: 9/10) src/other.ts:3 a file outside this PR' : cleanReply(t)) }); const res = await runPass(h, { base: 'main', out, pass: 'w1p1', cwd: wt, planFlags: ['--depth-floor', 'full'] }); expect(res).toContain('COLLECT_WRONG_TREE: reader='); expect(res).toContain(`reported="${treeLine(dir).slice(15).trim()}"`); @@ -934,20 +988,20 @@ describe('readers on a shared machine review the checkout the pass ships (GST-48 test('a reply that names no checkout is refused the same way', async () => { const { dir, env, wt } = worktree(); const out = path.join(wt, '.state/run/p1'); - const res = await runPass(host(dir, env, { reportFrom: () => '' }), { base: 'main', out, pass: 'w2p1', cwd: wt, planFlags: ['--depth-floor', 'full'] }); + const res = await runPass(host(wt, env, { reportFrom: () => '' }), { base: 'main', out, pass: 'w2p1', cwd: wt, planFlags: ['--depth-floor', 'full'] }); expect(res).toContain('COLLECT_WRONG_TREE: reader='); expect(res).toContain('reported="none"'); expect(res).not.toContain('NEXT: judge'); }, 240_000); - // Value: protects=/ship from a worktree on a shared machine works when readers follow their brief, though they start in another checkout; fails_when=a brief does not name the shipped checkout, or a correct reply is refused; + // Value: protects=/ship from a worktree that is the shared machine's start checkout works, its readers' briefs naming it (GST-776 refuses any other start checkout up front); fails_when=a brief does not name the shipped checkout, or a correct reply is refused; // why_new=GBRA-218 (no brief named a directory); seam=lib/review-pass.ts renderStage and the Capy pass program; no_claim=model compliance with the paragraph test('a reader that works in the checkout its brief names passes, and every brief names the worktree', async () => { const { dir, env, wt } = worktree(); const out = path.join(wt, '.state/run/p1'); fs.mkdirSync(out, { recursive: true }); fs.writeFileSync(path.join(out, 'coverage.prompt.md'), 'Run Step 7 as written.\n'); - const h = host(dir, env); + const h = host(wt, env); const res = await runPass(h, { base: 'main', out, pass: 'w3p1', cwd: wt, planFlags: ['--depth-floor', 'full'], extras: [{ name: 'coverage', prompt_file: path.join(out, 'coverage.prompt.md'), first: true }] }); expect(res).not.toContain('STOP:'); @@ -962,7 +1016,7 @@ describe('readers on a shared machine review the checkout the pass ships (GST-48 const out = path.join(wt, '.state/run/p1'); fs.mkdirSync(out, { recursive: true }); fs.writeFileSync(path.join(out, 'docs.prompt.md'), 'Run the Step 14.5 audit as written.\n'); - const h = host(dir, env, { reportFrom: t => (t.endsWith('audit/docs') ? dir : briefDirOf(h, t)) }); + const h = host(wt, env, { reportFrom: t => (t.endsWith('audit/docs') ? dir : briefDirOf(h, t)) }); const res = await runPass(h, { base: 'main', out, pass: 'w4p1', cwd: wt, planFlags: ['--depth-floor', 'full'], extras: [{ name: 'docs', prompt_file: path.join(out, 'docs.prompt.md'), first: true }] }); expect(res).toContain('COLLECT_WRONG_TREE: reader=audit/docs'); @@ -973,7 +1027,7 @@ describe('readers on a shared machine review the checkout the pass ships (GST-48 test('a rerun after a wrong-tree STOP stops on the failed reader and never collects its reply twice', async () => { const { dir, env, wt } = worktree(); const out = path.join(wt, '.state/run/p1'); - const h = host(dir, env, { reportFrom: () => dir }); + const h = host(wt, env, { reportFrom: () => dir }); const input = { base: 'main', out, pass: 'w5p1', cwd: wt, planFlags: ['--depth-floor', 'full'] }; expect(await runPass(h, input)).toContain('COLLECT_WRONG_TREE: reader='); const again = await runPass(h, input); @@ -987,7 +1041,7 @@ describe('readers on a shared machine review the checkout the pass ships (GST-48 const wt = path.join(tmp('rp-wt-'), 'same'); expect(spawnSync('git', ['worktree', 'add', '-q', '--detach', wt, 'feat'], { cwd: dir, env, encoding: 'utf8', timeout: 60_000 }).status).toBe(0); const out = path.join(wt, '.state/run/p1'); - const res = await runPass(host(dir, env, { reportFrom: () => dir }), { base: 'main', out, pass: 'w6p1', cwd: fs.realpathSync(wt), planFlags: ['--depth-floor', 'full'] }); + const res = await runPass(host(wt, env, { reportFrom: () => dir }), { base: 'main', out, pass: 'w6p1', cwd: fs.realpathSync(wt), planFlags: ['--depth-floor', 'full'] }); expect(res).toContain('COLLECT_WRONG_TREE: reader='); expect(res).not.toContain('COLLECT_TREE_MOVED'); }, 240_000); @@ -996,7 +1050,7 @@ describe('readers on a shared machine review the checkout the pass ships (GST-48 const { dir, env, wt } = worktree(); const out = path.join(wt, '.state/run/p1'); let moved = false; - const h = host(dir, env, { reportFrom: t => { + const h = host(wt, env, { reportFrom: t => { if (!moved) { moved = true; spawnSync('bash', ['-c', 'echo "export const late = 1;" >> src/a.ts && git commit -qam late'], { cwd: wt, env, encoding: 'utf8', timeout: 60_000 }); } return briefDirOf(h, t); } }); @@ -1008,7 +1062,7 @@ describe('readers on a shared machine review the checkout the pass ships (GST-48 test('an outside voice that reviewed another tree is recorded as unavailable coverage, never clean, and the voices still finish', async () => { const { dir, env, wt } = worktree(); const out = path.join(wt, '.state/run/p1'); - const h = host(dir, env, { reportFrom: t => (t.endsWith('outside-voice') ? dir : briefDirOf(h, t)) }); + const h = host(wt, env, { reportFrom: t => (t.endsWith('outside-voice') ? dir : briefDirOf(h, t)) }); const base = { base: 'main', out, pass: 'w8p1', cwd: wt, planFlags: ['--depth-floor', 'full'] }; expect(await runPass(h, base)).toContain('NEXT: judge (Phase 4)'); const models = { 'outside-voice': { model_id: 'openai/gpt-6.1-sol' }, 'adversarial-native': { model_id: 'anthropic/claude-opus-5-5', reasoning_mode: 'high' } }; From f865cd9fc7c89f6ac64e6dcbe219a07a0ba026ad Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Mon, 2 Nov 2026 03:55:57 -0800 Subject: [PATCH 03/19] v1.92.74.0 pregate: clear --run-url refuses a failed or unfinished run of the lane's own workflow (GSTA-203 PR 8) (#3841) * test(pregate): clear --run-url accepts a run of another workflow on a lane with no profile (red, GSTA-203 PR 6) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(pregate): clear --run-url refuses a run of another workflow on a lane with no profile (GSTA-203 PR 6) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(pregate): the wrong-workflow refusal names its fix and survives a null run read (GSTA-203 PR 6) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (each refusal names its own fix) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(pregate): a job link with ?pr= or #step clears a lane with no profile (red, GSTA-203 PR 6 fix-check) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(pregate): clear --run-url accepts job links copied with ?pr= or #step (GSTA-203 PR 6 fix-check) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(pregate): the real workflows owe windows-smoke.yml/windows-smoke and only its run clears it; the ready conflict hint names each lane's own workflow (GSTA-203 PR 6) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(pregate): clear --run-url accepts a failed or running run of the lane's own workflow (red, GSTA-203 PR 8) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(pregate): clear --run-url refuses a failed or unfinished run of the lane's own workflow (GSTA-203 PR 8) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(pregate): a --run-url clear records the run attempt it judged (GSTA-203 PR 8 review) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(pregate): a --run-url clear records the run attempt it judged, not the link passed (GSTA-203 PR 8 review) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(pregate): --run-url clears wait for a green run; troubleshooting lists the no-profile refusals (GSTA-203 PR 8 docs audit) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 4 ++++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- docs/pregate.md | 30 +++++++++++++++++++++--------- docs/troubleshooting.md | 11 ++++++++++- lib/pregate/index.ts | 12 +++++++++--- package.json | 2 +- test/pregate-clear-runs.test.ts | 31 ++++++++++++++++++++++++------- 8 files changed, 71 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d90faced..93ceeccfa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,9 @@ # Changelog +## [1.92.74.0] - 2026-11-02 + +- **`gstack-pregate clear --run-url` needs a green run.** On a lane with no run profile, a run of the lane's own workflow that failed, was cancelled or timed out now refuses `PREGATE_RECEIPT_INVALID` (`run_not_success`), and one still queued or running refuses `run_in_progress`. The fix line says to rerun the workflow and clear with the green run. The check reads the run's newest attempt, so a job or attempt link is judged by the whole run, and the lane records the attempt it judged (`…/actions/runs//attempts/`) instead of the link you passed. A green run still clears. docs/pregate.md now names `windows-smoke.yml/windows-smoke` as the free tier's Windows smoke lane, beside the narrower `free-tests.yml/cso-windows-launcher` and `windows-setup-e2e.yml/windows-setup` lanes. `test/pregate-clear-runs.test.ts` failed before the fix. + ## [1.92.73.0] - 2026-11-02 ### Fixed diff --git a/VERSION b/VERSION index 4633cc70e..61e08b1b6 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.92.73.0 +1.92.74.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index a86825dff..21a2aa351 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.92.73.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.92.74.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/docs/pregate.md b/docs/pregate.md index e82fb15a0..e0e18d938 100644 --- a/docs/pregate.md +++ b/docs/pregate.md @@ -383,7 +383,7 @@ The record is `.gstack/tmp/pregate.json` (`gstack-artifact validate --as pregate (`PREGATE_STALE`: rerun after the final stamp), green (`PREGATE_LANE_FAILED`, `PREGATE_INCOMPLETE`, or `PREGATE_ENVIRONMENT` when every incomplete check is the machine's) and every `requires-remote` obligation carries a receipt -(`PREGATE_REMOTE_UNCLEARED`: dispatch the job, then +(`PREGATE_REMOTE_UNCLEARED`: dispatch the job, wait for it to finish green, then `gstack-pregate clear --remote --run-url `). The ship receipt then carries `pregate` and `requires_remote`. `gstack-pregate verify` runs the same check standalone. @@ -754,17 +754,24 @@ reads `remote-gate tree= tests= runs=/,… profiles=

,… heads= verified=

| --ledger ... | --no-ledger) [--out ] [--json] * * `classify` reads a JSON array of findings (the Step 9.3 result) and stamps * each with `locus` (diff | fix-range | outside), `kind` (defect | test-gap | @@ -30,6 +30,7 @@ import { parseHunks, renderDeferredIssue, renderDeferredSection, reviewLoopLine, scopeFindings, scopeLine, summarizeReviewLoop, ALREADY_FILED_REASON, DEFERRED_MARKER, findingIdentity, findingWhere, REVIEW_SCOPE_POLICIES, type DeferredRecord, type ReviewScopePolicy, type ScopeFinding, type ScopedFinding, } from '../lib/review-scope'; +import { ledgerBlockers, type LedgerBlocker } from '../lib/review-ledger'; const HELP = `gstack-review-scope — finding scope for /ship's review loop (docs/ship-policy.md#review-scope) @@ -42,8 +43,12 @@ const HELP = `gstack-review-scope — finding scope for /ship's review loop (doc render --deferred [--findings ] the PR-body sections: ## Deferred findings (references, never closing keywords) and ## Review notes (scope=note findings) - summary --since [--out ] the invocation's passes from this branch's review log: REVIEW_LOOP: passes=… - blocking_found=… deferred=… noted=… (the ship receipt's --review-loop input) + summary --since (--run | --ledger ... | --no-ledger) [--out ] + the invocation's passes from this branch's review log: REVIEW_LOOP: passes=… + blocking_found=… deferred=… noted=… (the ship receipt's --review-loop input); + blocking_found also counts every CRITICAL the dispatch ledgers collected or judged + (--run finds each review-ledger.jsonl under the run directory); unrecorded=N names + blockers no row gives a disposition. An unreadable ledger: REVIEW_SCOPE_LEDGER Defaults: --policy from the base branch's .gstack/ship-policy.json review_scope (frozen); an unresolvable base refuses (POLICY_SOURCE_UNAVAILABLE) unless --policy is passed; --findings - reads stdin; --branch the current branch. @@ -117,7 +122,7 @@ function readFindings(spec: string | undefined, what: string): ScopeFinding[] { if (!list || !list.every((f: unknown) => f && typeof f === 'object' && !Array.isArray(f))) throw new Error(`${what}: expected a JSON array of finding objects`); return list as ScopeFinding[]; } -function refuse(cmd: string, code: 'REVIEW_SCOPE_INPUT' | 'REVIEW_SCOPE_RANGE', detail: string): number { +function refuse(cmd: string, code: 'REVIEW_SCOPE_INPUT' | 'REVIEW_SCOPE_RANGE' | 'REVIEW_SCOPE_LEDGER', detail: string): number { process.stderr.write(resultLineWithDocs(`gstack-review-scope ${cmd}`, code, detail) + '\n'); return EXIT.fail; } @@ -245,7 +250,26 @@ function cmdSummary(rest: string[], json: boolean): number { if (!line.trim() || line === 'NO_REVIEWS') continue; try { rows.push(JSON.parse(line)); } catch { /* malformed row: gstack-review-read already warned */ } } - const summary = summarizeReviewLoop(rows, since); + const files: string[] = []; + for (let i = 0; i < rest.length; i++) { + if (rest[i] !== '--ledger') continue; + if (!rest[i + 1] || rest[i + 1]!.startsWith('--')) return refuse('summary', 'REVIEW_SCOPE_INPUT', '--ledger needs a file'); + files.push(rest[++i]!); + } + const run = flag(rest, '--run'); + if (run) { + let found: string[]; + try { found = (fs.readdirSync(run, { recursive: true }) as string[]).filter(p => path.basename(p) === 'review-ledger.jsonl').sort().map(p => path.join(run, p)); } catch (e: any) { return refuse('summary', 'REVIEW_SCOPE_LEDGER', `--run ${run}: ${e.code ?? e.message}`); } + if (!found.length) return refuse('summary', 'REVIEW_SCOPE_LEDGER', `--run ${run} holds no review-ledger.jsonl: name the run directory the passes wrote, or each --ledger`); + files.push(...found); + } + const noLedger = rest.includes('--no-ledger'); + if (noLedger && files.length) return refuse('summary', 'REVIEW_SCOPE_INPUT', '--no-ledger contradicts --run and --ledger'); + // The ledgers hold every reader's findings, the voices' too, before any row classifies them; a summary without them can undercount. + if (!noLedger && !files.length) return refuse('summary', 'REVIEW_SCOPE_LEDGER', 'name the review\'s dispatch ledgers (--run or --ledger ), or --no-ledger when no reader ran through gstack-review-dispatch'); + let ledgers: LedgerBlocker[][]; + try { ledgers = files.map(ledgerBlockers); } catch (e: any) { return refuse('summary', 'REVIEW_SCOPE_LEDGER', e.message); } + const summary = summarizeReviewLoop(rows, since, ledgers); const line = reviewLoopLine(summary); // A /ship invocation always runs Step 9 once; zero matching rows means the wrong --since or a review log written elsewhere, never a loop to record. if (summary.passes === 0) return refuse('summary', 'REVIEW_SCOPE_INPUT', `no review-log row of this branch is a via:ship review at or after ${since}; check REVIEW_SINCE (${line})`); diff --git a/docs/review-policy.md b/docs/review-policy.md index c8238d7d6..a5fd51cb7 100644 --- a/docs/review-policy.md +++ b/docs/review-policy.md @@ -387,14 +387,17 @@ narrowed to the gate scan's tagged hunks. ## What the receipt records -`gstack-review-scope summary --since ` reads this invocation's -review-log rows and writes the counters `gstack-ship-receipt write --review-loop` +`gstack-review-scope summary --since --run ` reads this invocation's +review-log rows and the dispatch ledgers under the run directory (`--ledger ` names one; +`--no-ledger` when no reader ran through `gstack-review-dispatch`) and writes the counters `gstack-ship-receipt write --review-loop` puts in the receipt: | Field | Meaning | |---|---| | `review_loop.passes` | full review passes this ship ran (1 or 2; up to 3 under `review_scope: all`) | -| `review_loop.blocking_found` | distinct blocking findings across the passes | +| `review_loop.blocking_found` | every blocker found (GST-951): the rows' blocking findings (classified, or an unclassified CRITICAL), plus each ledger CRITICAL (as collected or judged) that no row finding from the same reviewer at the same path:line accounts for (each row finding pairs with one per ledger), so an adversarial, outside-voice or not-yet-classified blocker counts and one whose newest classification is below the floor does not. An unreadable ledger, merged file or judged set, or a judged set changed since `judge`, is `REVIEW_SCOPE_LEDGER`, never a count | +| `review_loop.unrecorded` | blockers found that no row gives a disposition (fixed, deferred, noted, skipped, a gate answer); absent when 0 | +| `review_loop.ledgers_read` | dispatch ledgers the count read; absent under `--no-ledger` | | `review_loop.deferred` | blocking findings in unchanged code filed as issues (a Step 11 late defer, logged as a `review-late-defer` row so it never displaces the newest `review` pass, counts here and is not a pass) | | `review_loop.noted` | below-floor findings listed in the PR body | | `review_loop.fixed` | findings fixed in this PR | @@ -611,7 +614,7 @@ ones, and the arm lost blockers. `bin/gstack-review-dispatch` closes both ends: - `ship/sections/review-army.md` Step 9: the plan's `REVIEW_DEPTH`, `GATE_SCAN` and `--depth-floor` on every run, the depth fields of the review-log row, passes, `RANGE_BASE`, classify, defer, render, the cap check before any edit, the Step 11 late-defer row, scratch files under `.gstack/tmp/`, the cap decisions; `ship/sections/adversarial.md` Step 11: the outside voice reads the same range and scopes queued fixes the same way. - `lib/review-depth.ts`, `bin/gstack-review-plan`: the diff class, the depth (`depthFor`, `DEPTH_RULES`, `depthOfCommitInput`), each dispatch's `exclude` pathspecs and `DIFF_CMD`, the transition table, the numeric-claim scan and each role's effort tier (`test/review-depth.test.ts`, `test/review-plan-cli.test.ts`). - `lib/review-collect.ts`: `collectCommit` (git objects only, from any checkout) and `collectLive` (the working tree), per-path lines and the base commit's registry (`test/review-collect.test.ts`). -- `lib/review-scope.ts`, `bin/gstack-review-scope`: `classify`, `defer`, `render`, `summary` (with `fix_checks`, `dispatches`, `est_input_tokens`); refusals are `REVIEW_SCOPE_INPUT` and `REVIEW_SCOPE_RANGE` ([troubleshooting.md](troubleshooting.md#review-scope-input)). +- `lib/review-scope.ts`, `bin/gstack-review-scope`: `classify`, `defer`, `render`, `summary` (with `fix_checks`, `dispatches`, `est_input_tokens`, `unrecorded`, `ledgers_read`); refusals are `REVIEW_SCOPE_INPUT`, `REVIEW_SCOPE_RANGE` and `REVIEW_SCOPE_LEDGER` ([troubleshooting.md](troubleshooting.md#review-scope-input)). - `lib/ship-policy.ts`: the `review_scope` key and its `ship: review_scope …` consequence line. - `review/checklist.md`: `prereg-with-results` under Gate Integrity and `test-keyboard-count-loop` under Pass 2; `bin/gstack-coauthor-trailers` for the contributor-credit trailers of a port. - `lib/ship-receipt-prose.ts`, `gstack-ship-receipt read --check-prose`: the body's quoted counters against the receipt (`RECEIPT_PROSE_MISMATCH`, [ship-policy.md](ship-policy.md#ship-receipt)). diff --git a/docs/ship-policy.md b/docs/ship-policy.md index 26f3ef71e..1e08417be 100644 --- a/docs/ship-policy.md +++ b/docs/ship-policy.md @@ -1268,11 +1268,11 @@ construction: the run and names the finding (path, severity, problem) for the owner instead of a third round; the owner may grant one scoped fix-check. -`gstack-review-scope summary --since ` reads the branch's review-log -rows for this invocation and writes the counters the ship receipt carries +`gstack-review-scope summary --since --run ` reads the branch's review-log +rows for this invocation and its dispatch ledgers, and writes the counters the ship receipt carries (`gstack-ship-receipt write … --review-loop @`): `review_loop: { passes, -blocking_found, deferred, noted, fixed, scope }`, plus `fix_checks`, `dispatches` -and `est_input_tokens` when the run recorded them, printed on the receipt summary +blocking_found, deferred, noted, fixed, scope }`, plus `fix_checks`, `dispatches`, +`est_input_tokens`, `unrecorded` and `ledgers_read` when the run recorded them, printed on the receipt summary line as `passes= blocking_found= deferred=`, so a scorecard can track passes per PR. @@ -1286,8 +1286,9 @@ instead of `defer`. `review_scope: all` is the opt-back to the earlier loop: every finding is `scope: fix` on every pass, reviewers read the whole diff each time, and only the three-cycle fix cap stops the loop. `show-effective` prints the choice as a `ship: review_scope …` line, and the codes a refusal ends with are -[`REVIEW_SCOPE_INPUT`](troubleshooting.md#review-scope-input) and -[`REVIEW_SCOPE_RANGE`](troubleshooting.md#review-scope-range). +[`REVIEW_SCOPE_INPUT`](troubleshooting.md#review-scope-input), +[`REVIEW_SCOPE_RANGE`](troubleshooting.md#review-scope-range) and +[`REVIEW_SCOPE_LEDGER`](troubleshooting.md#review-scope-ledger). ## `base_red`: pausing /ship on a red base diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 12fae0efc..6634be10f 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -5660,6 +5660,24 @@ pass). the invocation record, and start a full pass (`--pass 1`) when the branch was rebased; /ship's own rule is to merge the base in, never rebase a reviewed head. + +### `REVIEW_SCOPE_LEDGER` + +**Meaning.** `gstack-review-scope summary` could not count the review's +dispatch ledgers, so it wrote no `REVIEW_LOOP:` line rather than a count that +might be low (GST-951). Either no ledger was named (`--run`, `--ledger` or +`--no-ledger`), `--run` holds no `review-ledger.jsonl`, or a ledger, the +`merged.jsonl` it names or its judged set is unreadable, or the judged set +changed after `judge` checked it. The ledgers hold every reader's findings +(specialists, Red Team, both adversarial voices, targeted, fix-check) before any +review-log row classifies them, which is why the count reads them. + +**Fix.** Pass `--run `, the directory the review passes wrote (on +Capy, `ship/DRIVER.md`'s run directory), or each `--ledger `. Pass +`--no-ledger` only when no reader ran through `gstack-review-dispatch`. When a +file moved, restore it; when a judged set changed, rerun `gstack-review-dispatch +judge` on it. + ### `RECEIPT_CARRY_REFUSED` diff --git a/docs/unattended.md b/docs/unattended.md index cb6ea320a..8506acf25 100644 --- a/docs/unattended.md +++ b/docs/unattended.md @@ -172,7 +172,8 @@ unpushed head is `QUEUE_RECEIPT_STALE` to the queue. The receipt's `review_loop` object, written from `gstack-review-scope summary --since `, is the loop's machine-readable outcome: `passes` (Step 9 full passes this ship ran, 1 or 2; up to 3 under `review_scope: all`), -`blocking_found` (distinct blocking findings across them), `deferred` +`blocking_found` (every blocker the readers found, rows and dispatch ledgers +both, [review-policy.md](review-policy.md#what-the-receipt-records)), `deferred` (blocking findings in unchanged code filed as issues, Step 11's late defers included), `noted`, `fixed` and `scope`. The summary line a parent reads prints it as `passes= blocking_found= diff --git a/lib/headless-artifacts.ts b/lib/headless-artifacts.ts index 433bec17b..06d9f5d93 100644 --- a/lib/headless-artifacts.ts +++ b/lib/headless-artifacts.ts @@ -106,7 +106,7 @@ export interface ShipReceipt { // Related open PRs by other authors (Lane D): `supersedes` / `credit` only on the owner's confirmation. issues?: { fixes: number[]; refs: number[]; flakes: Array<{ issue: number; run: string }>; related?: Array<{ pr: number; repo?: string; author: string; closes: number[]; supersedes?: boolean; credit?: boolean }> }; /** Harness lane E: the review loop's counters (gstack-review-scope summary) and the carry record when the receipt followed a head whose product diff was unchanged (gstack-ship-receipt carry). */ - review_loop?: { passes: number; blocking_found: number; deferred: number; noted?: number; fixed?: number; scope?: string; fix_checks?: number; rounds?: number; dispatches?: Record; est_input_tokens?: number }; + review_loop?: { passes: number; blocking_found: number; deferred: number; noted?: number; fixed?: number; scope?: string; fix_checks?: number; rounds?: number; dispatches?: Record; est_input_tokens?: number; unrecorded?: number; ledgers_read?: number }; carry?: { from: string; reason: string; product_patch_id: string; changed_paths: string[]; merge_commits: number; range_diff: string; covers: string[]; reruns: string[]; trees?: string[]; ancestry?: 'linear' | 'rewritten' }; /** The reviewed head's tree and its product patch-id against the base (gstack-ship-receipt write; carry rebinds both): what a carry across rewritten history compares. */ head_tree?: string; product_patch_id?: string; @@ -238,7 +238,7 @@ export const ARTIFACT_SCHEMAS = { thread: str, tree: { enum: ['same-modulo-stamps', 'changed', 'not-compared'] }, gate_reuse: str, policy: str, queue_mode: str, preregistration: str, history: str, pregate: { ...str, description: 'pass|warn|requires-remote cleared … @ from pregate.json, current for this head' }, requires_remote: strList, approval: { type: 'object', description: 'approval classes: { source, status: checked|unavailable, classes: [{ class, scope, effects, citation: verified|unverified, approved_by, date, via, binding, evidence }], report_only }' }, - review_loop: { type: 'object', required: ['passes', 'blocking_found', 'deferred'], properties: { passes: { type: 'integer', minimum: 0 }, blocking_found: { type: 'integer', minimum: 0 }, deferred: { type: 'integer', minimum: 0 }, noted: { type: 'integer', minimum: 0 }, fixed: { type: 'integer', minimum: 0 }, scope: str, fix_checks: { type: 'integer', minimum: 0 }, rounds: { type: 'integer', minimum: 0 }, dispatches: { type: 'object', description: 'reviewer dispatches by role' }, est_input_tokens: { type: 'integer', minimum: 0 } } }, + review_loop: { type: 'object', required: ['passes', 'blocking_found', 'deferred'], properties: { passes: { type: 'integer', minimum: 0 }, blocking_found: { type: 'integer', minimum: 0 }, deferred: { type: 'integer', minimum: 0 }, noted: { type: 'integer', minimum: 0 }, fixed: { type: 'integer', minimum: 0 }, scope: str, fix_checks: { type: 'integer', minimum: 0 }, rounds: { type: 'integer', minimum: 0 }, dispatches: { type: 'object', description: 'reviewer dispatches by role' }, est_input_tokens: { type: 'integer', minimum: 0 }, unrecorded: { type: 'integer', minimum: 0 }, ledgers_read: { type: 'integer', minimum: 0 } } }, carry: { type: 'object', required: ['from', 'reason', 'product_patch_id'], properties: { from: str, reason: str, product_patch_id: str, changed_paths: strList, merge_commits: { type: 'integer', minimum: 0 }, range_diff: str, covers: strList, reruns: strList, trees: strList, ancestry: { enum: ['linear', 'rewritten'] } } }, head_tree: str, product_patch_id: str, gstack_version: str, base_red: { type: 'object', required: ['cases', 'files', 'base', 'lane', 'expires_at'], properties: { cases: strList, files: strList, base: str, lane: str, expires_at: str } }, diff --git a/lib/result-codes-release.ts b/lib/result-codes-release.ts index 3797aeae2..f85c80bc0 100644 --- a/lib/result-codes-release.ts +++ b/lib/result-codes-release.ts @@ -360,6 +360,11 @@ export const RELEASE_RESULT_CODES = { summary: 'gstack-review-scope could not read the diff it scopes findings against: the base or the range base does not resolve, or the range base is not an ancestor of HEAD', fix: 'fetch origin/, check --range-base against the saved pass heads, and start a full pass (--pass 1) when history was rewritten', }, + REVIEW_SCOPE_LEDGER: { + anchor: 'review-scope-ledger', + summary: 'gstack-review-scope summary could not count the dispatch ledgers: none was named (no --run, --ledger or --no-ledger), --run holds no review-ledger.jsonl, or a ledger, its merged file or its judged set is unreadable or changed since judge', + fix: 'pass --run (the directory the review passes wrote) or each --ledger ; --no-ledger only when no reader ran through gstack-review-dispatch; restore or re-collect a ledger file that moved', + }, RECEIPT_CARRY_REFUSED: { anchor: 'receipt-carry-refused', summary: 'gstack-ship-receipt carry could not carry the /ship receipt to HEAD: the PR body has no receipt, the working tree has uncommitted changes, the receipt head is not an ancestor of HEAD and the receipt records no tree to compare, or the product diff could not be compared', diff --git a/lib/review-ledger.ts b/lib/review-ledger.ts index cfbc91656..5f2973f66 100644 --- a/lib/review-ledger.ts +++ b/lib/review-ledger.ts @@ -236,6 +236,53 @@ export function recordJudgment(ledger: string, merged: MergedLine[], judged: Jud return res; } +/** + * The blocking findings a run's ledger holds (GST-951): every collected line whose label ranks CRITICAL, raised to its judged + * severity when the newest judge accepted a judged set that is unchanged since; lead judgment's additions count and a folded + * duplicate does not. Every reader counts the same way, whatever it is and whether or not Step 9.4 classified its findings. + * Each carries its locus (a JSON finding's path:line, else the first path:line its text names, an extensionless bin/ path + * included) so the summary can find its row. + * A ledger, merged file or judged set it cannot read, or a judged set changed since judge, throws: the count is never a guess. + */ +export interface LedgerBlocker { id: string; locus: string | null } +function lineLocus(raw: string): string | null { + try { + const o = JSON.parse(raw); + if (o && typeof o.path === 'string' && Number.isInteger(o.line)) return `${o.path.replace(/^\.\//, '')}:${o.line}`; + } catch { /* a prose finding */ } + return /(?:[\w.-]+\/)+[\w.-]+:\d+|[\w.-]+\.\w+:\d+/.exec(raw)?.[0]?.replace(/^\.\//, '') ?? null; +} +export function ledgerBlockers(ledger: string): LedgerBlocker[] { + const fail = (why: string): never => { throw new Error(`${ledger}: ${why}`); }; + const read = (file: unknown, what: string): string => { + try { return fs.readFileSync(String(file), 'utf8'); } catch { return fail(`${what} ${String(file)} is unreadable`); } + }; + const jsonLines = (text: string, what: string): any[] => + text.split('\n').filter(l => l.trim()).map(l => { try { return JSON.parse(l); } catch { return fail(`${what} has a line that is not JSON`); } }); + const events = jsonLines(read(ledger, 'ledger'), 'the ledger') as LedgerEvent[]; + const ranks = new Map(); + const loci = new Map(); + for (const o of events.filter(e => e.event === 'output' && Array.isArray(e.ids) && e.ids.length)) { + const lines = jsonLines(read(o.merged, `merged file of ${String(o.reader)}`), String(o.merged)) as MergedLine[]; + for (const id of o.ids as string[]) { + const line = lines.find(l => l.id === id) ?? fail(`collected id ${id} is missing from ${String(o.merged)}`); + ranks.set(id, rank(line.label) ?? 0); + loci.set(id, lineLocus(line.raw)); + } + } + const verdict = [...events].reverse().find(e => e.event === 'judge'); + if (verdict?.ok === true) { + const text = read(verdict.judged, 'judged set'); + if (promptHash(text) !== verdict.judged_sha256) fail(`judged set ${String(verdict.judged)} changed since judge`); + for (const j of jsonLines(text, String(verdict.judged)) as JudgedLine[]) { + if (j.duplicate_of) { ranks.delete(j.id); continue; } + ranks.set(j.id, Math.max(ranks.get(j.id) ?? 0, rank(j.severity) ?? 0)); + if (!loci.has(j.id)) loci.set(j.id, lineLocus(j.reason ?? '')); + } + } + return [...ranks].filter(([, r]) => r >= 4).map(([id]) => ({ id, locus: loci.get(id) ?? null })); +} + /** * Whether a recorded output was a review: it has findings, or its file (unchanged since collect) carries the reader's clean marker, so * a ledger written before collect recorded the field is judged by the same rule; with no readable file, collect's own review field. diff --git a/lib/review-scope.ts b/lib/review-scope.ts index 833562591..6ab96923f 100644 --- a/lib/review-scope.ts +++ b/lib/review-scope.ts @@ -14,6 +14,7 @@ * Pure functions here; bin/gstack-review-scope runs git and gh. */ import { SPECIALISTS } from './review-depth'; +import { rank, type LedgerBlocker } from './review-ledger'; import { parseSpecialistOutput } from './state-matrix'; export type Locus = 'diff' | 'fix-range' | 'outside'; @@ -306,6 +307,10 @@ export interface ReviewLoopSummary { dispatches?: Record; /** Prompt plus diff bytes the dispatches read, / 4: a relative estimate, never a bill. */ est_input_tokens?: number; + /** Blocking findings no row gives a disposition (fixed, deferred, noted, skipped or a gate answer): found, outcome unrecorded. */ + unrecorded?: number; + /** Dispatch ledgers whose collected and judged findings were counted (gstack-review-dispatch). */ + ledgers_read?: number; } /** Review-log rows of one /ship invocation (skill review, via ship, at or after `since`) → the loop summary. */ @@ -348,12 +353,41 @@ function rowDispatches(row: Record): Array<{ role: string; bytes: n .map(([name]) => ({ role: name === 'red-team' ? 'red-team' : 'specialist', bytes: null })); } -export function summarizeReviewLoop(rows: readonly Record[], since: string): ReviewLoopSummary { +/** Row actions that record what happened to a finding. */ +const DISPOSITIONS = ['fixed', 'auto-fixed', 'deferred', 'noted', 'skipped', 'kept', 'restored', 'open']; +/** A row finding that blocks: classified blocking, or an unclassified CRITICAL defect (logged before Step 9.4's classify). */ +const rowBlocks = (f: Record) => f.blocking === true || (f.blocking === undefined && f.advisory !== true && rank(f.severity) === 4); +/** The reviewer behind a row's raised_by or a ledger id: every specialist is one; the lead's additions and unknowns name none. */ +function raiserOf(raiser: unknown): string | null { + const role = typeof raiser === 'string' ? raiser.split(/[/#]/)[0]!.trim() : ''; + if (!role || role === 'unknown' || role === 'lead' || role === 'core') return null; + return ['qa', 'red-team', 'adversarial-native', 'outside-voice', 'fix-check', 'targeted'].includes(role) ? role : 'specialist'; +} +/** A row finding's path:line, from its path and line or its fingerprint (`path:line:category`). */ +function rowLocus(f: Record): string | null { + if (typeof f.path === 'string' && Number.isInteger(f.line)) return `${f.path.replace(/^\.\//, '')}:${f.line}`; + const m = /^(.+?):(\d+):/.exec(typeof f.fingerprint === 'string' ? f.fingerprint : ''); + return m ? `${m[1]!.replace(/^\.\//, '')}:${m[2]}` : null; +} + +/** + * The loop summary. blocking_found counts every blocker found (GST-951). Each CRITICAL the dispatch ledgers hold + * (`ledgerBlockers`, one list per ledger) is paired with at most one row finding per ledger at the same path:line, raised + * by the same reviewer (or naming none; a fix-check re-reads every reviewer's fixes), that may be it (no severity yet, CRITICAL as logged or before a mapper lowered + * it, or blocking), so one finding re-raised by a later pass's ledger pairs again. A paired finding blocks unless its newest + * classification put it below the floor (blocking: false), and its row's action is its disposition. An unpaired ledger + * CRITICAL blocks and is unrecorded. A row finding blocks by its newest classification, else as an unclassified CRITICAL. + * A ledger CRITICAL with no locus, or a second one at a paired line, counts on its own. A different defect the same reviewer + * raises at the exact line of a finding a row classified below the floor pairs with that finding: path:line is the only + * identity rows and ledgers share until rows carry the ledger id. + */ +export function summarizeReviewLoop(rows: readonly Record[], since: string, ledgers: readonly (readonly LedgerBlocker[])[] = []): ReviewLoopSummary { const sinceMs = Date.parse(since); const inRun = (r: Record) => r && r.via === 'ship' && Number.isFinite(rowTime(r)) && (!Number.isFinite(sinceMs) || rowTime(r) >= sinceMs); const loopRows = rows.filter(r => inRun(r) && (r.skill === 'review' || r.skill === LATE_DEFER_SKILL || r.skill === FIX_CHECK_SKILL)); const passes = loopRows.filter(r => r.skill === 'review'); - const blocking = new Set(); + type Seen = { locus: string | null; raiser: string | null; classified: boolean; blocks: boolean; candidate: boolean; disposed: boolean }; + const seen = new Map(); const deferred = new Set(); const noted = new Set(); const fixed = new Set(); @@ -363,13 +397,36 @@ export function summarizeReviewLoop(rows: readonly Record[], since: for (const f of Array.isArray(row.findings) ? row.findings : []) { if (!f || typeof f !== 'object') continue; const id = findingIdentity(f); - if (f.blocking === true) blocking.add(id); + const prior = seen.get(id); + const classified = typeof f.blocking === 'boolean'; + seen.set(id, { + locus: rowLocus(f) ?? prior?.locus ?? null, + raiser: prior?.raiser ?? raiserOf(f.raised_by), + classified: classified || !!prior?.classified, + blocks: classified ? f.blocking : prior?.classified ? prior.blocks : !!prior?.blocks || rowBlocks(f), + candidate: !!prior?.candidate || f.severity === undefined || rank(f.severity) === 4 || rank(f.raw_severity) === 4 || f.blocking === true, + disposed: !!prior?.disposed || DISPOSITIONS.includes(f.action), + }); if (f.action === 'deferred') deferred.add(id); if (f.action === 'noted') noted.add(id); if (f.action === 'fixed' || f.action === 'auto-fixed') fixed.add(id); } } - const out: ReviewLoopSummary = { passes: passes.length, blocking_found: blocking.size, deferred: deferred.size, noted: noted.size, fixed: fixed.size, scope }; + const paired = new Set(); + let ledgerOnly = 0; + const pairedIds = new Set(); + for (const [n, list] of ledgers.entries()) { + for (const b of list) { + const by = b.id.startsWith('fix-check') ? null : raiserOf(b.id); + const id = b.locus ? [...seen].find(([k, s]) => s.candidate && s.locus === b.locus && (!s.raiser || !by || s.raiser === by) && !paired.has(`${n}\0${k}`))?.[0] : undefined; + if (id) { paired.add(`${n}\0${id}`); pairedIds.add(id); } else ledgerOnly++; + } + } + const blockers = [...seen].filter(([k, s]) => s.blocks || (pairedIds.has(k) && !s.classified)); + const unrecorded = ledgerOnly + blockers.filter(([, s]) => !s.disposed).length; + const out: ReviewLoopSummary = { passes: passes.length, blocking_found: blockers.length + ledgerOnly, deferred: deferred.size, noted: noted.size, fixed: fixed.size, scope }; + if (unrecorded) out.unrecorded = unrecorded; + if (ledgers.length) out.ledgers_read = ledgers.length; const fixChecks = loopRows.filter(r => r.skill === FIX_CHECK_SKILL).length; if (fixChecks) out.fix_checks = fixChecks; out.rounds = passes.length + fixChecks; @@ -390,5 +447,6 @@ export function dispatchTotal(d: Record | undefined): number | u export function reviewLoopLine(s: ReviewLoopSummary): string { const total = dispatchTotal(s.dispatches); return `REVIEW_LOOP: passes=${s.passes} blocking_found=${s.blocking_found} deferred=${s.deferred} noted=${s.noted} fixed=${s.fixed} scope=${s.scope}` + + (s.unrecorded ? ` unrecorded=${s.unrecorded}` : '') + (s.ledgers_read ? ` ledgers_read=${s.ledgers_read}` : '') + (s.fix_checks ? ` fix_checks=${s.fix_checks}` : '') + (total !== undefined ? ` dispatches=${total}` : '') + (s.est_input_tokens !== undefined ? ` est_input_tokens=${s.est_input_tokens}` : ''); } diff --git a/lib/ship-receipt.ts b/lib/ship-receipt.ts index aba864ed5..b628a0aba 100644 --- a/lib/ship-receipt.ts +++ b/lib/ship-receipt.ts @@ -328,6 +328,8 @@ export function parseReviewLoop(value: unknown): ReviewLoopReceipt | undefined { if (n('fix_checks') !== undefined) out.fix_checks = n('fix_checks'); if (n('rounds') !== undefined) out.rounds = n('rounds'); if (n('est_input_tokens') !== undefined) out.est_input_tokens = n('est_input_tokens'); + if (n('unrecorded')) out.unrecorded = n('unrecorded'); + if (n('ledgers_read')) out.ledgers_read = n('ledgers_read'); if (v.dispatches && typeof v.dispatches === 'object' && !Array.isArray(v.dispatches)) { const d = Object.fromEntries(Object.entries(v.dispatches as Record) .filter(([k, c]) => /^[a-z][a-z0-9-]{0,39}$/.test(k) && typeof c === 'number' && Number.isInteger(c) && c >= 0).slice(0, 12)) as Record; diff --git a/package.json b/package.json index 5795fa2b6..7f8863c51 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.92.74", + "version": "1.92.75", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/ship/DRIVER.md b/ship/DRIVER.md index 8e3b4f4ca..a7da6798b 100644 --- a/ship/DRIVER.md +++ b/ship/DRIVER.md @@ -14,7 +14,7 @@ gates and the report. `STOP:` (exit 3) means nothing after it ran; `NEXT:` names the next act. Read a section only when a phase or the list at the end sends you, never up front. A STOP or decision this file does not cover: read `ship/SKILL.md` at that step, finish it there, come back. Unattended, take a question's recommended option (log it); gate -dispositions and approvals stay pending and end the run. Keep a private invocation record (`--record`): write its six headings at Phase 1, then fill each as its phase ends: **Checks** +dispositions and approvals stay pending and end the run. Keep a private invocation record (`--record`): write its six headings at Phase 1, fill each as its phase ends: **Checks** (Phases 2-4: lanes, pregate, evals, `Coverage:`, the plan checks, `Exploratory QA:`, `Scope Check:`), **Decisions** (any phase: each answered or auto-taken question, verbatim), **Reviews** (Phases 3-5: the plan line, start tokens, `Pre-Landing Review:`, `Gate edits:`, the reviewed tree), **Release** (Phases 5-7: @@ -119,7 +119,7 @@ STALE or MISSING reruns once), `PREGATE:`; print them, the last `Gate edits:` an ## 8. Publish: Steps 18, 19, 19-C, 20, 21, 22 - Step 18: the open-PR lookup; a title file starting `v `. `UNATTENDED_SHIP:` with no open PR at HEAD takes Step 18-H. -- Step 19 body: the receipt block, then the sections `sections/pr-body.md` lists (Pre-Landing Review: `REVIEW_LOOP:`, `Gate edits:`, deferred, notes), led under `CAPY_PR: on` (Step 19-C) by What changed, Why, Blast radius and Approvals (`gstack-approvals render`). `gstack-prose-lint --sections 'Why,Summary,Blast radius' --gate`; `gstack-issue-links`; `gstack-review-scope summary --since "$REVIEW_SINCE" && gstack-ship-receipt write --pr --base --gate @ --pregate --issues @ --review-loop @ --docs @ --body `; `gstack-post pr-body` and `pr-title` (a new PR via the host's PR tool; a draft while a gate finding is `open`). `SHIP_RECEIPT:` opens your report. +- Step 19 body: the receipt block, then the sections `sections/pr-body.md` lists (Pre-Landing Review: `REVIEW_LOOP:`, `Gate edits:`, deferred, notes), led under `CAPY_PR: on` (Step 19-C) by What changed, Why, Blast radius and Approvals (`gstack-approvals render`). `gstack-prose-lint --sections 'Why,Summary,Blast radius' --gate`; `gstack-issue-links`; `gstack-review-scope summary --since "$REVIEW_SINCE" --run && gstack-ship-receipt write --pr --base --gate @ --pregate --issues @ --review-loop @ --docs @ --body `; `gstack-post pr-body` and `pr-title` (a new PR via the host's PR tool; draft while a gate finding is `open`). `SHIP_RECEIPT:` opens your report. - Step 20: the metrics row via `gstack-review-log`. - Step 22: `gstack-ship-budget close --reading R --credits C` (report `BUDGET_FINAL:`, `BUDGET_TIMELINE:` (tokens, credits), any `METER:` line), then `gstack-ship-receipt result --pr `: `GSTACK_RESULT: skill=ship status=complete run=`. Under a merge queue, `gstack-ship-queue ready --pr [--screen|--no-screen]` at the pushed head; paste its output. diff --git a/ship/DRIVER.md.tmpl b/ship/DRIVER.md.tmpl index cefeda67f..f7b98d4c8 100644 --- a/ship/DRIVER.md.tmpl +++ b/ship/DRIVER.md.tmpl @@ -12,7 +12,7 @@ gates and the report. `STOP:` (exit 3) means nothing after it ran; `NEXT:` names the next act. Read a section only when a phase or the list at the end sends you, never up front. A STOP or decision this file does not cover: read `ship/SKILL.md` at that step, finish it there, come back. Unattended, take a question's recommended option (log it); gate -dispositions and approvals stay pending and end the run. Keep a private invocation record (`--record`): write its six headings at Phase 1, then fill each as its phase ends: **Checks** +dispositions and approvals stay pending and end the run. Keep a private invocation record (`--record`): write its six headings at Phase 1, fill each as its phase ends: **Checks** (Phases 2-4: lanes, pregate, evals, `Coverage:`, the plan checks, `Exploratory QA:`, `Scope Check:`), **Decisions** (any phase: each answered or auto-taken question, verbatim), **Reviews** (Phases 3-5: the plan line, start tokens, `Pre-Landing Review:`, `Gate edits:`, the reviewed tree), **Release** (Phases 5-7: @@ -117,7 +117,7 @@ STALE or MISSING reruns once), `PREGATE:`; print them, the last `Gate edits:` an ## 8. Publish: Steps 18, 19, 19-C, 20, 21, 22 - Step 18: the open-PR lookup; a title file starting `v `. `UNATTENDED_SHIP:` with no open PR at HEAD takes Step 18-H. -- Step 19 body: the receipt block, then the sections `sections/pr-body.md` lists (Pre-Landing Review: `REVIEW_LOOP:`, `Gate edits:`, deferred, notes), led under `CAPY_PR: on` (Step 19-C) by What changed, Why, Blast radius and Approvals (`gstack-approvals render`). `gstack-prose-lint --sections 'Why,Summary,Blast radius' --gate`; `gstack-issue-links`; `gstack-review-scope summary --since "$REVIEW_SINCE" && gstack-ship-receipt write --pr --base --gate @ --pregate --issues @ --review-loop @ --docs @ --body `; `gstack-post pr-body` and `pr-title` (a new PR via the host's PR tool; a draft while a gate finding is `open`). `SHIP_RECEIPT:` opens your report. +- Step 19 body: the receipt block, then the sections `sections/pr-body.md` lists (Pre-Landing Review: `REVIEW_LOOP:`, `Gate edits:`, deferred, notes), led under `CAPY_PR: on` (Step 19-C) by What changed, Why, Blast radius and Approvals (`gstack-approvals render`). `gstack-prose-lint --sections 'Why,Summary,Blast radius' --gate`; `gstack-issue-links`; `gstack-review-scope summary --since "$REVIEW_SINCE" --run && gstack-ship-receipt write --pr --base --gate @ --pregate --issues @ --review-loop @ --docs @ --body `; `gstack-post pr-body` and `pr-title` (a new PR via the host's PR tool; draft while a gate finding is `open`). `SHIP_RECEIPT:` opens your report. - Step 20: the metrics row via `gstack-review-log`. - Step 22: `gstack-ship-budget close --reading R --credits C` (report `BUDGET_FINAL:`, `BUDGET_TIMELINE:` (tokens, credits), any `METER:` line), then `gstack-ship-receipt result --pr `: `GSTACK_RESULT: skill=ship status=complete run=`. Under a merge queue, `gstack-ship-queue ready --pr [--screen|--no-screen]` at the pushed head; paste its output. diff --git a/ship/sections/pr-body.md b/ship/sections/pr-body.md index 64e2f50cc..79af0c5d9 100644 --- a/ship/sections/pr-body.md +++ b/ship/sections/pr-body.md @@ -220,7 +220,7 @@ per-lane counts (`{"":{"pass":N,"fail":N,"skip":N}}`) to ```bash rm -f .gstack/tmp/review-loop.json -~/.claude/skills/gstack/bin/gstack-review-scope summary --since "$REVIEW_SINCE" --out .gstack/tmp/review-loop.json && +~/.claude/skills/gstack/bin/gstack-review-scope summary --since "$REVIEW_SINCE" --no-ledger --out .gstack/tmp/review-loop.json && ~/.claude/skills/gstack/bin/gstack-ship-receipt write --pr --base --gate @.gstack/tmp/ship-gate.json --pregate .gstack/tmp/pregate.json --issues @.gstack/tmp/issues.json --review-loop @.gstack/tmp/review-loop.json --docs @.gstack/tmp/docs-audit.json --body "$PR_BODY_FILE" ``` @@ -233,7 +233,9 @@ When the owner accepted a stop after a fix-check edit, add and put its printed `Review-loop exception:` line in the body; never a prose line under `REVIEW_LOOP:`. The summary reads this branch's review log for the invocation's passes (its -`REVIEW_LOOP:` line is the one the Pre-Landing Review section carries). The `&&` +`REVIEW_LOOP:` line is the one the Pre-Landing Review section carries). Here every +finding reaches a row through Step 9.4; a run whose readers were collected by +`gstack-review-dispatch` (ship/DRIVER.md) passes `--run ` instead. The `&&` matters: a summary that refuses (no pass matched `REVIEW_SINCE`) must not let `write` record a stale or empty loop file. diff --git a/ship/sections/pr-body.md.tmpl b/ship/sections/pr-body.md.tmpl index 8786e9daa..611c1e553 100644 --- a/ship/sections/pr-body.md.tmpl +++ b/ship/sections/pr-body.md.tmpl @@ -203,7 +203,7 @@ per-lane counts (`{"":{"pass":N,"fail":N,"skip":N}}`) to ```bash rm -f .gstack/tmp/review-loop.json -~/.claude/skills/gstack/bin/gstack-review-scope summary --since "$REVIEW_SINCE" --out .gstack/tmp/review-loop.json && +~/.claude/skills/gstack/bin/gstack-review-scope summary --since "$REVIEW_SINCE" --no-ledger --out .gstack/tmp/review-loop.json && ~/.claude/skills/gstack/bin/gstack-ship-receipt write --pr --base --gate @.gstack/tmp/ship-gate.json --pregate .gstack/tmp/pregate.json --issues @.gstack/tmp/issues.json --review-loop @.gstack/tmp/review-loop.json --docs @.gstack/tmp/docs-audit.json --body "$PR_BODY_FILE" ``` @@ -216,7 +216,9 @@ When the owner accepted a stop after a fix-check edit, add and put its printed `Review-loop exception:` line in the body; never a prose line under `REVIEW_LOOP:`. The summary reads this branch's review log for the invocation's passes (its -`REVIEW_LOOP:` line is the one the Pre-Landing Review section carries). The `&&` +`REVIEW_LOOP:` line is the one the Pre-Landing Review section carries). Here every +finding reaches a row through Step 9.4; a run whose readers were collected by +`gstack-review-dispatch` (ship/DRIVER.md) passes `--run ` instead. The `&&` matters: a summary that refuses (no pass matched `REVIEW_SINCE`) must not let `write` record a stale or empty loop file. diff --git a/test/helpers/ship-frozen-scope-actor.ts b/test/helpers/ship-frozen-scope-actor.ts index 85c6dca67..19a2a8d28 100644 --- a/test/helpers/ship-frozen-scope-actor.ts +++ b/test/helpers/ship-frozen-scope-actor.ts @@ -190,7 +190,7 @@ function reviewRows(repo: string, env: NodeJS.ProcessEnv) { } function loopSummary(repo: string, env: NodeJS.ProcessEnv, since: string) { - return JSON.parse(command(repo, env, path.join(ROOT, 'bin/gstack-review-scope'), ['summary', '--since', since, '--json']).stdout.trim()); + return JSON.parse(command(repo, env, path.join(ROOT, 'bin/gstack-review-scope'), ['summary', '--no-ledger', '--since', since, '--json']).stdout.trim()); } export async function shipFrozenScopeFailures(fixture: ReturnType, result: Pick) { diff --git a/test/review-scope-ledger-count.test.ts b/test/review-scope-ledger-count.test.ts new file mode 100644 index 000000000..955d57a61 --- /dev/null +++ b/test/review-scope-ledger-count.test.ts @@ -0,0 +1,249 @@ +/** + * GST-951: the /ship receipt's review_loop counters (gstack-review-scope summary) missed real blockers three times on + * 2026-11-02, because summary read only the review-log rows while /ship's readers report through the dispatch ledgers + * (gstack-review-dispatch collect and judge). Each case below is a minimal reconstruction of that PR's rows and ledgers, + * built from its public PR body and receipt (the rows and ledgers themselves stayed on the lanes' machines): the rows + * reproduce the receipt's published counters on the old summary, and the ledgers hold what the PR body says the readers found. + * Ledgers are written with the real collect and judge, so the parser and judge rules apply as in a run. + */ +import { afterEach, describe, expect, setDefaultTimeout, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import { EXIT } from '../lib/headless-artifacts'; +import { collect, readJsonl, recordJudgment, type JudgedLine, type MergedLine } from '../lib/review-ledger'; +import { parseReviewLoop } from '../lib/ship-receipt'; +import { cleanup, makeFixtureRepo, runBin, type FixtureRepo } from './helpers/restamp-fixture'; + +setDefaultTimeout(60_000); +const repos: FixtureRepo[] = []; +afterEach(() => { while (repos.length) cleanup(repos.pop()!); }); + +const SINCE = '2026-11-02T01:00:00Z'; +const at = (min: number) => new Date(Date.parse(SINCE) + min * 60_000).toISOString(); +const review = (min: number, pass: number, findings: object[]) => ({ skill: 'review', via: 'ship', timestamp: at(min), status: findings.length ? 'issues_found' : 'clean', review_scope: 'frozen', pass, findings }); +const fixCheck = (min: number, findings: object[]) => ({ skill: 'review-fix-check', via: 'ship', timestamp: at(min), review_scope: 'frozen', findings }); +const note = (fp: string, raised_by: string) => ({ fingerprint: fp, severity: 'INFORMATIONAL', action: 'noted', blocking: false, raised_by }); + +/** One pass directory: each reader's output collected into its ledger, then lead judgment (default: every label kept). */ +function pass(dir: string, readers: Array<[string, string]>, judgeWith = (merged: MergedLine[]): JudgedLine[] => merged.map(m => ({ id: m.id, severity: m.label }))): string { + fs.mkdirSync(dir, { recursive: true }); + const ledger = path.join(dir, 'review-ledger.jsonl'); + const mergedFile = path.join(dir, 'merged.jsonl'); + fs.writeFileSync(mergedFile, ''); + for (const [reader, output] of readers) expect(collect(ledger, reader, output, mergedFile).review).toBe(true); + const merged = readJsonl(mergedFile); + const judged = judgeWith(merged); + const judgedFile = path.join(dir, 'judged.jsonl'); + fs.writeFileSync(judgedFile, judged.map(j => JSON.stringify(j) + '\n').join('')); + expect(recordJudgment(ledger, merged, judged, judgedFile, mergedFile).ok).toBe(true); + return ledger; +} + +function branchWith(rows: object[]): FixtureRepo { + const repo = makeFixtureRepo({ prefix: 'gst951-' }); + repos.push(repo); + repo.git('checkout', '-qb', 'feat'); + const slug = runBin(repo, 'gstack-slug', ['--get', 'SLUG']).out.trim(); + const logDir = path.join(repo.home, 'projects', slug); + fs.mkdirSync(logDir, { recursive: true }); + fs.writeFileSync(path.join(logDir, 'feat-reviews.jsonl'), rows.map(r => JSON.stringify(r) + '\n').join('')); + return repo; +} +const summary = (repo: FixtureRepo, ...args: string[]) => runBin(repo, 'gstack-review-scope', ['summary', '--since', SINCE, '--json', ...args]); +const loop = (r: { status: number | null; out: string; err: string }) => { expect(r.err).not.toContain('(REVIEW_SCOPE_'); expect(r.status).toBe(EXIT.ok); return JSON.parse(r.out); }; + +const CLEAN = 'NO FINDINGS'; +const OUTSIDE_CLEAN = 'No issues found in the diff.\nRecommendation: ship because no finding reaches the floor'; + +describe('GST-951: the receipt counts every blocker the readers found', () => { + // #3841 (reconstruction; GSTA-203 PR 8 trail rows 9 and 14): Step 11's native adversarial FIXABLE (CRITICAL) finding was + // fixed in round 2, but no row recorded it (pass 2's row held no findings), so summary printed blocking_found=0 fixed=0 and + // the lane hand-set blocking_found=1 fixed=1. Its two other findings are the noted pair. + const ADVERSARIAL_3841 = [ + '1. lib/pregate-clear.ts:140 The receipt stores the raw run URL, not the attempt it judged. FIXABLE', + '2. lib/pregate.ts:88 The refusal text does not name the attempt. INVESTIGATE', + '3. docs/pregate.md:12 The docs do not say a rerun replaces the attempt. INVESTIGATE', + 'Recommendation: fix 1 because a later attempt would be judged by the first', + ].join('\n'); + test('#3841 as /ship logs it: the queued adversarial blocker in pass 2\'s row counts as 1 found and 1 fixed', () => { + const repo = branchWith([ + review(0, 1, []), + review(40, 2, [{ fingerprint: 'lib/pregate-clear.ts:140:security', action: 'fixed', raised_by: 'adversarial-native' }, + note('lib/pregate.ts:88:docs', 'adversarial-native'), note('docs/pregate.md:12:docs', 'adversarial-native')]), + ]); + const run = path.join(repo.root, 'run'); + pass(path.join(run, 'p1'), [['specialist/testing', CLEAN], ['red-team', CLEAN]]); + pass(path.join(run, 'p1-voices'), [['outside-voice', OUTSIDE_CLEAN], ['adversarial-native', ADVERSARIAL_3841]]); + pass(path.join(run, 'p2'), [['specialist/testing', CLEAN], ['red-team', CLEAN]]); + expect(loop(summary(repo, '--no-ledger'))).toMatchObject({ passes: 2, blocking_found: 0, noted: 2, fixed: 1 }); + const s = loop(summary(repo, '--run', run)); + expect(s).toMatchObject({ passes: 2, blocking_found: 1, noted: 2, fixed: 1, ledgers_read: 3, rounds: 2 }); + expect(s.unrecorded).toBeUndefined(); + }); + + test('#3841 as the lane logged it: an adversarial-only blocker counts, and its missing fix record shows as unrecorded', () => { + const repo = branchWith([ + review(0, 1, []), + review(40, 2, [note('lib/pregate.ts:88:docs', 'adversarial-native'), note('docs/pregate.md:12:docs', 'adversarial-native')]), + ]); + const run = path.join(repo.root, 'run'); + pass(path.join(run, 'p1'), [['specialist/testing', CLEAN], ['red-team', CLEAN]]); + pass(path.join(run, 'p1-voices'), [['outside-voice', OUTSIDE_CLEAN], ['adversarial-native', ADVERSARIAL_3841]]); + expect(loop(summary(repo, '--no-ledger'))).toMatchObject({ passes: 2, blocking_found: 0, noted: 2, fixed: 0 }); + const s = loop(summary(repo, '--run', run)); + expect(s).toMatchObject({ passes: 2, blocking_found: 1, noted: 2, fixed: 0, unrecorded: 1, ledgers_read: 2, rounds: 2 }); + }); + + // #3838 (reconstruction): the rows reproduce its receipt (passes=2 blocking_found=0 noted=8 fixed=4 fix_checks=3). Pass 2's + // red-team CRITICAL was logged fixed without classify's blocking stamp; Step 11's native adversarial CRITICAL (CSS + // image-set loading an outside image) was fixed in 0f61362c1 and no row holds it. + test('#3838: CRITICALs fixed without a classified row count', () => { + const repo = branchWith([ + review(0, 1, ['a', 'b', 'c'].map(x => ({ fingerprint: `lib/review-depth.ts:1${x}:style`, severity: 'INFORMATIONAL', action: 'fixed', blocking: false, raised_by: 'security' }))), + review(50, 2, [{ fingerprint: 'lib/review-depth.ts:212:security', severity: 'CRITICAL', action: 'fixed', raised_by: 'red-team' }, + note('lib/review-depth.ts:220:style', 'red-team'), note('lib/review-depth.ts:230:style', 'red-team')]), + fixCheck(60, [note('lib/review-depth.ts:240:style', 'fix-check'), note('lib/review-depth.ts:241:style', 'fix-check')]), + fixCheck(90, [note('lib/review-depth.ts:250:style', 'fix-check')]), + fixCheck(120, [note('lib/review-depth.ts:260:perf', 'fix-check'), note('lib/review-depth.ts:261:perf', 'fix-check'), note('lib/review-depth.ts:262:perf', 'fix-check')]), + ]); + const run = path.join(repo.root, 'run'); + pass(path.join(run, 'p1'), [['specialist/security', '- [INFORMATIONAL] lib/review-depth.ts:1a a CSS escape\n- [INFORMATIONAL] lib/review-depth.ts:1b a zero-padded reference\n- [INFORMATIONAL] lib/review-depth.ts:1c a split handler line'], ['red-team', CLEAN]]); + pass(path.join(run, 'p2'), [['red-team', '[CRITICAL] lib/review-depth.ts:212 a stray quote hides an external href from the drawing check']]); + pass(path.join(run, 'p2-voices'), [['outside-voice', OUTSIDE_CLEAN], ['adversarial-native', 'lib/review-depth.ts:300 CSS image-set() loads an outside image. FIXABLE\nRecommendation: fix because the drawing loads a remote URL']]); + expect(loop(summary(repo, '--no-ledger'))).toMatchObject({ passes: 2, blocking_found: 1, noted: 8, fixed: 4, fix_checks: 3, rounds: 5 }); + const s = loop(summary(repo, '--run', run)); + expect(s).toMatchObject({ passes: 2, blocking_found: 2, deferred: 0, noted: 8, fixed: 4, fix_checks: 3, rounds: 5, unrecorded: 1, ledgers_read: 3 }); + }); + + // #3837 (reconstruction): pass 1's row was logged before the outside voice's seven findings were classified, so they sit in the + // row with no severity, blocking stamp or action; the fix batch under the Captain's exception fixed them, and the receipt said + // blocking_found=0 fixed=0 noted=11 fix_checks=3. + test('#3837: blockers logged before classification count', () => { + const seven = Array.from({ length: 7 }, (_, i) => ({ fingerprint: `lib/inventory.ts:${10 + i}:logic`, raised_by: 'outside-voice' })); + const repo = branchWith([ + review(0, 1, [...seven, ...Array.from({ length: 9 }, (_, i) => note(`docs/inventory.md:${i + 1}:docs`, 'specialist'))]), + review(60, 2, [note('lib/inventory.ts:90:docs', 'red-team'), note('lib/inventory.ts:91:docs', 'red-team')]), + fixCheck(80, []), fixCheck(100, []), fixCheck(120, []), + ]); + const run = path.join(repo.root, 'run'); + pass(path.join(run, 'p1'), [['specialist-batched/1', CLEAN], ['red-team', CLEAN]]); + pass(path.join(run, 'p1-voices'), [['adversarial-native', CLEAN], ['outside-voice', [ + ...seven.map((f, i) => `${i + 1}. ${f.fingerprint.split(':').slice(0, 2).join(':')} finding ${i + 1} Severity: High Blocks: yes`), + 'Recommendation: fix 1-6 because a self-grandfathered row skips every plan rule', + ].join('\n')]]); + expect(loop(summary(repo, '--no-ledger'))).toMatchObject({ passes: 2, blocking_found: 0, noted: 11, fixed: 0, fix_checks: 3, rounds: 5 }); + const s = loop(summary(repo, '--run', run)); + expect(s).toMatchObject({ passes: 2, blocking_found: 7, noted: 11, fixed: 0, unrecorded: 7, ledgers_read: 2, rounds: 5 }); + const line = runBin(repo, 'gstack-review-scope', ['summary', '--since', SINCE, '--run', run]); + expect(line.out.trim()).toBe('REVIEW_LOOP: passes=2 blocking_found=7 deferred=0 noted=11 fixed=0 scope=frozen unrecorded=7 ledgers_read=2 fix_checks=3'); + expect(parseReviewLoop(s)).toMatchObject({ blocking_found: 7, unrecorded: 7, ledgers_read: 2, rounds: 5 }); + }); + + test('a blocker both sources hold counts once, and a disposed one is not unrecorded', () => { + const repo = branchWith([review(0, 1, [{ fingerprint: 'src/a.ts:3:logic', severity: 'CRITICAL', blocking: true, action: 'fixed', raised_by: 'red-team' }])]); + const run = path.join(repo.root, 'run'); + pass(path.join(run, 'p1'), [['specialist/testing', CLEAN], ['red-team', '[CRITICAL] src/a.ts:3 the guard is inverted']]); + const s = loop(summary(repo, '--run', run)); + expect(s).toMatchObject({ blocking_found: 1, fixed: 1, ledgers_read: 1 }); + expect(s.unrecorded).toBeUndefined(); + }); + + test('a CRITICAL a row classified below the floor is not a blocker; distinct blockers from one reviewer each count', () => { + const repo = branchWith([review(0, 1, [ + { fingerprint: 'src/a.ts:3:testing', path: 'src/a.ts', line: 3, severity: 'CRITICAL', kind: 'test-gap', blocking: false, action: 'noted', raised_by: 'testing' }, + { fingerprint: 'src/b.ts:9:security', path: 'src/b.ts', line: 9, severity: 'CRITICAL', blocking: true, action: 'fixed', raised_by: 'security' }, + { fingerprint: 'src/c.ts:1:logic', severity: 'CRITICAL', advisory: true, action: 'noted', raised_by: 'security' }, + ])]); + const run = path.join(repo.root, 'run'); + pass(path.join(run, 'p1'), [['specialist/testing', '[CRITICAL] src/a.ts:3 the denied path has no test\n[CRITICAL] src/d.ts:7 the retry loop never ends']]); + const s = loop(summary(repo, '--run', run)); + expect(s).toMatchObject({ blocking_found: 2, fixed: 1, noted: 2, unrecorded: 1, ledgers_read: 1 }); + }); + + test('Step 11 review cases: a row finding pairs with one ledger CRITICAL; the newest classification and a mapped severity decide; a bare fixed row is a disposition', () => { + const row = (fp: string, f: object) => ({ fingerprint: fp, ...f }); + const repo = branchWith([ + review(0, 1, [ + row('src/auth.ts:42:testing', { path: 'src/auth.ts', line: 42, severity: 'CRITICAL', kind: 'test-gap', blocking: false, action: 'noted', raised_by: 'testing' }), + row('src/m.ts:5:state-matrix', { path: 'src/m.ts', line: 5, severity: 'INFORMATIONAL', raw_severity: 'CRITICAL', kind: 'test-gap', blocking: false, action: 'noted', raised_by: 'state-machines' }), + row('src/f.ts:3:logic', { action: 'fixed', raised_by: 'outside-voice' }), + row('src/n.ts:8:logic', { severity: 'CRITICAL', raised_by: 'red-team' }), + row('src/g.ts:7:logic', { action: 'fixed', raised_by: 'core' }), + ]), + review(30, 2, [row('src/n.ts:8:logic', { severity: 'CRITICAL', blocking: false, action: 'noted', raised_by: 'red-team' })]), + ]); + const run = path.join(repo.root, 'run'); + pass(path.join(run, 'p1'), [ + ['specialist/testing', '[CRITICAL] src/auth.ts:42 the denied path has no test\n[CRITICAL] src/m.ts:5 untested cell'], + ['outside-voice', '1. src/auth.ts:42 an unauthenticated caller reaches the handler. Severity: High Blocks: yes\n2. src/f.ts:3 the guard is inverted. Blocks: yes\nRecommendation: fix 1 because it is an open handler'], + ['red-team', '[CRITICAL] src/n.ts:8 looked wrong, classified below the floor later'], + ], m => [...m.map(l => ({ id: l.id, severity: l.label })), { id: 'lead#1', severity: 'CRITICAL', reason: 'src/g.ts:7 the lead found an unchecked write' }]); + const s = loop(summary(repo, '--run', run)); + expect(s).toMatchObject({ passes: 2, blocking_found: 3, fixed: 2, unrecorded: 1, ledgers_read: 1 }); + }); + + test('round 2 cases: another reviewer at a below-floor line, an extensionless path, one finding in two ledgers, no locus', () => { + const repo = branchWith([ + review(0, 1, [ + { fingerprint: 'src/auth.ts:42:testing', path: 'src/auth.ts', line: 42, severity: 'CRITICAL', blocking: false, action: 'noted', raised_by: 'testing' }, + { fingerprint: 'bin/gstack-review-scope:262:logic', path: 'bin/gstack-review-scope', line: 262, severity: 'CRITICAL', blocking: true, action: 'fixed', raised_by: 'red-team' }, + { fingerprint: 'x.ts:5:logic', path: 'x.ts', line: 5, severity: 'CRITICAL', blocking: true, action: 'fixed', raised_by: 'red-team' }, + ]), + fixCheck(30, []), + ]); + const run = path.join(repo.root, 'run'); + pass(path.join(run, 'p1'), [ + ['specialist/testing', '[CRITICAL] src/auth.ts:42 the denied path has no test'], + ['outside-voice', '1. src/auth.ts:42 an unauthenticated caller reaches the handler. Blocks: yes\nRecommendation: fix 1 because it is open'], + ['red-team', '[CRITICAL] bin/gstack-review-scope:262 the refusal is skipped\n[CRITICAL] x.ts:5 the guard is inverted'], + ]); + pass(path.join(run, 'fix-check'), [['fix-check', '[CRITICAL] x.ts:5 the guard is still inverted\n[CRITICAL] the retry loop never ends']]); + const s = loop(summary(repo, '--run', run)); + expect(s).toMatchObject({ blocking_found: 4, fixed: 2, noted: 1, unrecorded: 2, ledgers_read: 2 }); + }); + + test('judgment decides: a folded duplicate counts once, a lead addition counts, a rejected judgment keeps the collected labels', () => { + const repo = branchWith([review(0, 1, [])]); + const run = path.join(repo.root, 'run'); + pass(path.join(run, 'fold'), [['red-team', '[CRITICAL] src/a.ts:3 the guard is inverted\n[CRITICAL] src/a.ts:4 the guard is inverted again']], + m => [{ id: m[0]!.id, severity: 'CRITICAL' }, { id: m[1]!.id, severity: 'CRITICAL', duplicate_of: m[0]!.id }, { id: 'lead#1', severity: 'CRITICAL', reason: 'src/e.ts:2 the lead found an unchecked write' }]); + expect(loop(summary(repo, '--run', run))).toMatchObject({ blocking_found: 2, unrecorded: 2, ledgers_read: 1 }); + const rejected = path.join(repo.root, 'rejected'); + fs.mkdirSync(rejected); + const ledger = path.join(rejected, 'review-ledger.jsonl'); + const mergedFile = path.join(rejected, 'merged.jsonl'); + fs.writeFileSync(mergedFile, ''); + collect(ledger, 'red-team', '[CRITICAL] src/a.ts:3 the guard is inverted', mergedFile); + const merged = readJsonl(mergedFile); + const judgedFile = path.join(rejected, 'judged.jsonl'); + fs.writeFileSync(judgedFile, JSON.stringify({ id: merged[0]!.id, severity: 'INFORMATIONAL' }) + '\n'); + expect(recordJudgment(ledger, merged, [{ id: merged[0]!.id, severity: 'INFORMATIONAL' }], judgedFile, mergedFile).ok).toBe(false); + expect(loop(summary(repo, '--ledger', ledger))).toMatchObject({ blocking_found: 1, unrecorded: 1, ledgers_read: 1 }); + }); + + test('summary refuses REVIEW_SCOPE_LEDGER instead of counting without a ledger it needs, and writes nothing', () => { + const repo = branchWith([review(0, 1, [])]); + const run = path.join(repo.root, 'run'); + const ledger = pass(path.join(run, 'p1'), [['red-team', '[CRITICAL] src/a.ts:3 the guard is inverted']]); + const out = path.join(repo.root, 'loop.json'); + const refused = (r: { status: number | null; err: string }, why: string) => { expect(r.status).toBe(EXIT.fail); expect(r.err).toContain('(REVIEW_SCOPE_LEDGER;'); expect(r.err).toContain(why); expect(fs.existsSync(out)).toBe(false); }; + refused(summary(repo, '--out', out), '--no-ledger when no reader ran through gstack-review-dispatch'); + fs.mkdirSync(path.join(repo.root, 'empty')); + refused(summary(repo, '--run', path.join(repo.root, 'empty'), '--out', out), 'holds no review-ledger.jsonl'); + refused(summary(repo, '--ledger', path.join(run, 'nope.jsonl'), '--out', out), 'is unreadable'); + const judged = path.join(run, 'p1', 'judged.jsonl'); + const kept = fs.readFileSync(judged, 'utf8'); + fs.writeFileSync(judged, kept.replace('CRITICAL', 'INFORMATIONAL')); + refused(summary(repo, '--ledger', ledger, '--out', out), 'changed since judge'); + fs.writeFileSync(judged, kept); + fs.rmSync(path.join(run, 'p1', 'merged.jsonl')); + refused(summary(repo, '--run', run, '--out', out), 'is unreadable'); + refused(summary(repo, '--run', path.join(repo.root, 'nonesuch'), '--out', out), 'ENOENT'); + const bare = summary(repo, '--out', out, '--ledger'); + expect(bare.err).toContain('(REVIEW_SCOPE_INPUT;'); + expect(bare.err).toContain('--ledger needs a file'); + const both = summary(repo, '--run', run, '--no-ledger'); + expect(both.status).toBe(EXIT.fail); + expect(both.err).toContain('(REVIEW_SCOPE_INPUT;'); + }); +}); diff --git a/test/review-scope-windows-launch.test.ts b/test/review-scope-windows-launch.test.ts index 68c077bd5..5a7405455 100644 --- a/test/review-scope-windows-launch.test.ts +++ b/test/review-scope-windows-launch.test.ts @@ -50,7 +50,7 @@ test('summary reads the review log through gstack-review-read (#3541)', () => { ]; fs.writeFileSync(path.join(state, 'projects', SLUG, 'feat-reviews.jsonl'), rows.map(r => JSON.stringify(r)).join('\n') + '\n'); const out = path.join(tmp, 'review-loop.json'); - const r = runScriptSync(path.join(ROOT, 'bin', 'gstack-review-scope'), ['summary', '--since', '2026-10-26T09:00:00Z', '--out', out, '--json'], { + const r = runScriptSync(path.join(ROOT, 'bin', 'gstack-review-scope'), ['summary', '--no-ledger', '--since', '2026-10-26T09:00:00Z', '--out', out, '--json'], { cwd: repo, env, encoding: 'utf8', timeout: 120_000, }); expect(r.error).toBeUndefined(); diff --git a/test/review-scope.test.ts b/test/review-scope.test.ts index e6bdc2ace..8b40ad3af 100644 --- a/test/review-scope.test.ts +++ b/test/review-scope.test.ts @@ -423,30 +423,30 @@ describe('gstack-review-scope (bin)', () => { fs.writeFileSync(path.join(logDir, `${branch}-reviews.jsonl`), rows.map(r => JSON.stringify(r)).join('\n') + '\n{not json\n'); const expected = { passes: 2, blocking_found: 2, deferred: 1, noted: 1, fixed: 1, scope: 'frozen', rounds: 2 }; const out = path.join(repo.root, 'review-loop.json'); - const r = runBin(repo, 'gstack-review-scope', ['summary', '--since', '2026-10-26T09:30:00Z', '--out', out, '--json']); + const r = runBin(repo, 'gstack-review-scope', ['summary', '--no-ledger', '--since', '2026-10-26T09:30:00Z', '--out', out, '--json']); expect(r.status).toBe(EXIT.ok); expect(r.err).toContain('REVIEW_LOOP: passes=2 blocking_found=2 deferred=1 noted=1 fixed=1 scope=frozen'); expect(JSON.parse(r.out)).toEqual(expected); expect(JSON.parse(fs.readFileSync(out, 'utf8'))).toEqual(expected); - const plain = runBin(repo, 'gstack-review-scope', ['summary', '--since', '2026-10-26T08:00:00Z']); + const plain = runBin(repo, 'gstack-review-scope', ['summary', '--no-ledger', '--since', '2026-10-26T08:00:00Z']); expect(plain.status).toBe(EXIT.ok); expect(plain.out.trim()).toBe('REVIEW_LOOP: passes=3 blocking_found=3 deferred=1 noted=1 fixed=2 scope=frozen'); const noSince = runBin(repo, 'gstack-review-scope', ['summary', '--json']); expect(noSince.status).toBe(EXIT.fail); expect(noSince.err).toContain('(REVIEW_SCOPE_INPUT;'); expect(noSince.out).toBe(''); - const badSince = runBin(repo, 'gstack-review-scope', ['summary', '--since', 'yesterday']); + const badSince = runBin(repo, 'gstack-review-scope', ['summary', '--no-ledger', '--since', 'yesterday']); expect(badSince.status).toBe(EXIT.fail); expect(badSince.err).toContain('(REVIEW_SCOPE_INPUT;'); // zero matching passes is a wrong --since or a log written elsewhere, never a loop to record: refused, nothing written const bare = fixture(); const noneOut = path.join(bare.root, 'review-loop.json'); - const none = runBin(bare, 'gstack-review-scope', ['summary', '--since', '2026-10-26T09:30:00Z', '--out', noneOut, '--json']); + const none = runBin(bare, 'gstack-review-scope', ['summary', '--no-ledger', '--since', '2026-10-26T09:30:00Z', '--out', noneOut, '--json']); expect(none.status).toBe(EXIT.fail); expect(none.err).toContain('(REVIEW_SCOPE_INPUT;'); expect(none.err).toContain('REVIEW_LOOP: passes=0'); expect(fs.existsSync(noneOut)).toBe(false); - const late = runBin(repo, 'gstack-review-scope', ['summary', '--since', '2026-10-27T00:00:00Z']); + const late = runBin(repo, 'gstack-review-scope', ['summary', '--no-ledger', '--since', '2026-10-27T00:00:00Z']); expect(late.status).toBe(EXIT.fail); }); }); diff --git a/test/review-targeted.test.ts b/test/review-targeted.test.ts index 375cf1d01..9ef2217d3 100644 --- a/test/review-targeted.test.ts +++ b/test/review-targeted.test.ts @@ -324,7 +324,7 @@ describe('the targeted role end to end on a fixture repository', () => { fs.mkdirSync(logDir, { recursive: true }); const row = { skill: 'review', via: 'ship', timestamp: '2026-10-30T10:00:00Z', status: 'clean', review_scope: 'frozen', findings: [], dispatches: [{ role: 'specialist' }, { role: 'adversarial-native' }, { role: 'targeted' }] }; fs.writeFileSync(path.join(logDir, 'feat-reviews.jsonl'), JSON.stringify(row) + '\n'); - const summary = runBin(repo, 'gstack-review-scope', ['summary', '--since', '2026-10-30T09:00:00Z', '--json']); + const summary = runBin(repo, 'gstack-review-scope', ['summary', '--no-ledger', '--since', '2026-10-30T09:00:00Z', '--json']); expect(summary.status, summary.err).toBe(0); expect(JSON.parse(summary.out).dispatches).toEqual({ specialist: 1, 'adversarial-native': 1, targeted: 1 }); diff --git a/test/size-ledger/ship/gst-951-receipt-counts-blockers.json b/test/size-ledger/ship/gst-951-receipt-counts-blockers.json new file mode 100644 index 000000000..7509a6666 --- /dev/null +++ b/test/size-ledger/ship/gst-951-receipt-counts-blockers.json @@ -0,0 +1,4 @@ +{ + "bytes": 177, + "why": "GST-951: summary names its ledgers (--run in DRIVER, --no-ledger in pr-body)" +} From 3d99c224972942c447c1f94c9061da900e667aab Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Mon, 2 Nov 2026 04:16:51 -0800 Subject: [PATCH 05/19] v1.92.76.0 wave(review): #3809 #3813 #3826 #3831 #3838 (#3848) * fix(retro): keep retro history in the state root, shared across worktrees (#3749) /retro saved snapshots under .context/retros inside the working tree, so a run in a linked worktree saw none of the main checkout's history, reported "First retro recorded", and wrote a snapshot that git worktree remove deletes. gstack-retro-metrics --history [--next ] now resolves the history directory as /projects//retros (the slug is keyed on the git common dir, so every worktree shares it), moves any legacy .context/retros from this worktree and the main checkout into it, lists prior snapshots and prints the next snapshot path. Step 12/13 and the gbrain prior-retros query use it. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(gate-diff): read a bare base as origin/ in gate-diff and diff-scope (#3650) gstack-gate-diff and gstack-diff-scope resolved a bare base such as main to the local branch whenever it shared history with HEAD. A local main that is behind origin/main (a linked worktree, or a Capy clone, where local main never moves) then set the merge base, so /ship's gate scan and specialist scope also read commits already upstream. Both now prefer origin/ when it resolves, the base /ship fetches and merges against. Fixes #3650 Co-authored-by: exGeni <135786749+exGeni@users.noreply.github.com> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(retro): make history migration safe against symlinks and concurrent runs (#3749 review) Review pass 1 findings: a .context/retros that resolves outside its worktree (or to the shared history itself) is left alone, so migration can no longer delete the history it points at; moves are no-clobber with retry; a copy identical to any same-date snapshot collapses; RETRO_FILE is reserved as {} so overlapping runs in two worktrees never share a path; --next must be a YYYY-MM-DD date; a missing option value no longer loops forever. Step 12 names RETRO_HISTORY_ERROR instead of reading it as a stale helper. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): fragment for the #3749 retro history fix Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (diff-scope reads slash-named bases through origin too) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(retro): trim Step 12/13 prose back under the parity skeleton cap Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: Step 11 review fixes (a fork branch, a colliding tag or another remote never replaces the base) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(retro): keep today="" in Step 12 so the judge excerpt keeps both date inputs Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): describe the fork, tag and remote guards (#3650) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(retro): compare #3749 history paths with forward slashes so the Windows shard passes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(review): review-plan and gate-diff plan the same with a stale local main or a shadowing tag (GST-883, failing first) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review): gate-diff and review-plan resolve a branch base to origin/ by full ref name; review-plan scans the merge base its own diff used (GST-883) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): GST-883 fragment Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(review-risk): hostile-input time bounds for every risk pattern; the R51 rm and rmSync rules fail (GST-898) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review-risk): the R51 rm and rmSync rules run in linear time (GST-898) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(review-packets-removed): repin lib/review-risk.ts on the GST-898 verdict-identity replay Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review-risk): padding cannot push recursive: true out of the rmSync window; repin on the GST-898 no-loss replay Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): GST-898 fragment wording Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(review): planner misreads: SVG chart as runtime, session data as auth, bare base in diff-scope and receipt depth (GST-911, GST-884, failing first) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review): a drawing-only SVG is data and a scripted one is security; session data files are not auth; diff-scope and the receipt depth read origin/ by full ref name (GST-911, GST-884) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review): the SVG drawing check refuses CSS escapes and reads zero-padded character references; the security class reads a file's changed SVG lines joined (GST-911 review pass 1) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(review): a stray quote cannot hide an external href in a drawing SVG (GST-911 review pass 2, failing first) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review): the SVG drawing check reads every name= without consuming its value, so a stray quote cannot hide an external href (GST-911 review pass 2) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(review): CSS in a drawing SVG holds only declarations and url(#id): image-set(), image(), element() keep it code (GST-911 Step 11, failing first) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review): a drawing SVG's CSS holds declarations only, no @ rule and no function but url(#id), which closes image-set(), image(), element() and every other CSS load (GST-911 Step 11) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(review): a style element's CSS is read past comments and CDATA, unclosed style tags are refused in linear time, and every drawing verdict holds at 1 MB and 8 MB (GST-911, failing first) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review): walk a style element's CSS with indexOf, dropping comments and reading CDATA, so no hidden ends it early and unclosed style tags refuse in linear time (GST-911) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(review-policy): the linear-time claim covers R51's rm and rmSync rules (GST-898) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(review-policy): state the rmSync padding rule exactly (GST-898) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review-depth): linear url() refusal in svgDrawing (wave review-2, red-team CRITICAL) SVG_REFUSED's url() entry backtracked quadratically on url( followed by a long whitespace run (80k spaces: 820 ms; 1 MB: over 230 s), and svgDrawing runs in collectLive and in collectCommit, which ship-queue ready, train build and land call with no timeout. The rewritten entry refuses exactly what #3838's pattern refused. Repin lib/review-depth.ts under no-loss replay GST-911-redos (verdict identity: 60,000 seeded strings, the edges and every SVG blob in history, zero differences). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (wave review-2, pass 1) - bin/gstack-diff-scope: _resolve_base takes a related local before another remote's twin, the order resolveBase uses, so the two tools pick the same base for a bare name when origin/ is unrelated. - lib/review-depth.ts: a drawing's DOCTYPE names no document but W3C's SVG DTD; SYSTEM and other PUBLIC identifiers keep the SVG code. Repin lib/review-depth.ts under no-loss replay GST-911-redos (the DOCTYPE change only keeps more SVGs code, so it drops no reviewer). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: freeze the exported SVG_REFUSED list (wave review-2, Step 11) Exported only so test/review-risk-redos.test.ts can reach the url() entry; a frozen readonly copy keeps any importer from changing verdicts while DEPTH_RULES still reports the original rules. Repin lib/review-depth.ts under no-loss replay GST-911-redos (no pattern changed). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Revert "fix: freeze the exported SVG_REFUSED list (wave review-2, Step 11)" The fix-check (wave review-2) showed Object.freeze is shallow: each RegExp can still be changed in place, so the freeze did not close adversarial-native#1, and it moved the tree past the one the Step 11 voices reviewed. The finding ships as an open note (its reviewer rated it Low, not blocking; no production caller mutates the list). Restores the lib/review-depth.ts pin under no-loss replay GST-911-redos. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): wave review-2 entry for the integrator fixes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(review-policy): a drawing's DOCTYPE names no outside document (wave review-2 docs audit) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> Co-authored-by: exGeni <135786749+exGeni@users.noreply.github.com> --- CHANGELOG.md | 42 +++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- bin/gstack-diff-scope | 69 ++++- bin/gstack-gate-diff | 6 +- bin/gstack-retro-metrics | 77 ++++- bin/gstack-review-plan | 7 +- docs/designs/HONEST_WORK_GATE_INTEGRITY.md | 9 +- docs/review-policy.md | 31 +- docs/skills.md | 2 +- lib/gate-diff/git.ts | 58 +++- lib/review-collect.ts | 33 +- lib/review-depth.ts | 96 +++++- lib/review-risk.ts | 18 +- lib/ship-preflight.ts | 2 +- lib/ship-receipt-depth.ts | 7 +- package.json | 2 +- retro/SKILL.md | 33 +- retro/SKILL.md.tmpl | 33 +- test/diff-scope.test.ts | 65 ++++ test/gate-diff-cli.test.ts | 53 ++++ ...ession-3749-retro-history-worktree.test.ts | 155 ++++++++++ test/review-packets-removed.test.ts | 7 +- test/review-plan-origin-base.test.ts | 95 ++++++ test/review-planner-misreads.test.ts | 291 ++++++++++++++++++ test/review-risk-redos.test.ts | 102 ++++++ 26 files changed, 1188 insertions(+), 109 deletions(-) create mode 100644 test/regression-3749-retro-history-worktree.test.ts create mode 100644 test/review-plan-origin-base.test.ts create mode 100644 test/review-planner-misreads.test.ts create mode 100644 test/review-risk-redos.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 341227af6..0c95b691a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,47 @@ # Changelog +## [1.92.76.0] - 2026-11-02 + +**/retro now remembers its history across git worktrees, so a run in a linked worktree compares against last week instead of announcing "First retro recorded".** Retro snapshots used to live in `.context/retros/` inside the working tree. A worktree (Conductor, Claude Code) saw none of the main checkout's history, and its own snapshot was deleted along with the worktree (#3749). + +### Fixed +- **`bin/gstack-retro-metrics --history [--next ]`**: keeps retro history in `~/.gstack/projects//retros/` (the state root), which every worktree of a repository shares. It lists earlier snapshots as `RETRO_PRIOR` lines, newest first, and reserves the next `-.json` as `RETRO_FILE`, so two retros running at once in different worktrees never write the same file. +- **Moving old history**: snapshots in `.context/retros/` of the current worktree and the main checkout move to the shared folder on the first run. A copy identical to a stored snapshot from the same date is dropped, and a different snapshot with the same name takes the next free number, so nothing is overwritten. A `.context/retros` that points outside its worktree (a symlink to the shared folder or anywhere else) is left alone, and moves never replace an existing file. +- **`retro/SKILL.md`**: Steps 12 and 13 use the helper, the gbrain prior-retros query reads the shared folder, and a history error is reported to the user instead of being mistaken for an outdated helper. `docs/skills.md` says where snapshots are saved. +- `gstack-retro-metrics` no longer loops forever when an option such as `--next` is given without a value. + +### Tests +- `test/regression-3749-retro-history-worktree.test.ts` runs the real Step 12 block in a main checkout with a linked worktree: the worktree sees the main checkout's snapshot, its new snapshot path is outside the worktree and survives `git worktree remove`, migration keeps every snapshot, a symlinked `.context/retros` cannot delete history, and two runs get different snapshot paths. It failed 4 of 4 on the code before the fix. + +**A stale local `main` no longer turns a /ship review into a full-depth gate review (GST-883).** `gstack-review-plan` handed the raw base name to `gstack-gate-diff`, which diffed against the local branch. With a local `main` four commits behind `origin/main`, the gate scan read a test edit that was already on `origin/main`, so a plain code diff planned as gate class at full depth. On gstack-evals#123 that cost about 15-20M tokens of readers the change did not need (GEVA-117). + +### Fixed +- **`gstack-gate-diff `** resolves a branch name to its remote-tracking branch (`origin/main` for `main`, origin first), the way `gstack-review-plan`'s own diff and review dispatch already did (#3780). It uses the local branch only when no remote has the branch, and says so on stderr: `no remote-tracking branch for `, or `/ shares no history with HEAD`. A repository with no remote at all uses the local branch without a note. Branches are read by their full ref names, so a tag named `main` or `origin/main` can no longer stand in for the base and hide committed edits from the scan. +- **`gstack-review-plan`** passes the merge base its own diff used to `gstack-diff-scope` and `gstack-gate-diff`, so the scope flags, the gate scan and the line counts all read the same commits. Its diff reads `refs/remotes/origin/` and `refs/heads/` by full name too. + +### Tests +- `test/review-plan-origin-base.test.ts`: a fixture whose `origin/main` is one commit ahead of the local `main`, with a test edit on `origin/main` and a branch forked from it. `gstack-review-plan --base main` and `gstack-gate-diff main` plan the same class, depth and gate-tagged count as with a fresh `main` (before the fix: gate, full, `RH-1:1` against code, light, none). A tag named `origin/main` or `main` at HEAD leaves the plan and the scanned range unchanged. Also covers the stderr note for a branch no remote has, an unrelated `origin/main`, and the quiet path with no remote. + +- **One hostile line no longer stalls review classification** (GST-898). R51's `rm -rf` rule in `lib/review-risk.ts` took quadratic time on a long flag run. One added line of `rm -` and 100,000 `r` characters held `matchRisk` for about 3.7 s, and each doubling took four times as long. The `rmSync(… recursive: true)` rule did the same on a line of repeated `rmSync(` calls. Both now run in linear time (the same line takes about 1 ms). The `rm` rule is rewritten with lookaheads and matches exactly the lines it matched before. The `rmSync` rule looks for `recursive: true` in the first 500 characters of the call, and a call with more than 500 characters before its closing parenthesis now counts as security on its own, so padding can't push the flag out of view. A slow match never lowered review depth: `gstack-review-dispatch pass` stops when the plan times out, and the queue recomputes depth itself. +- A new test holds every risk pattern under 100 ms on 100,000-character hostile lines and checks both rules against R51's on 60,000 seeded lines each. Over the last 100 main commits, no commit changes depth, class or risk class, and none of the 69,023 changed lines gets a different verdict from either rule. + +### Fixed + +- **/ship no longer reviews a results write-up as if it were gate code.** Three planner misreads on gstack-evals#132 (GST-911) pushed a docs PR to the five-reader full panel. A chart SVG made of drawing elements now counts as data, like the same chart as a PNG; an SVG with a script, `foreignObject`, event handler or `javascript:` URL stays code and now gets the security class. A data file named for sessions (`recovered-sessions.jsonl`) no longer sets the auth scope, so it adds no auth reader and no gate class; session code and session config still do. +- **`gstack-diff-scope` and the receipt depth read `origin/` by its full ref name (GST-884).** A local `main` behind `origin/main` no longer adds upstream files to the scope, and a tag named `main` or `origin/main` can no longer stand in for the branch. + +### Tests + +- `test/review-planner-misreads.test.ts`: a 600-point matplotlib-style chart plans docs, light, runtime 0 (before: code, 636 runtime lines); six ways an SVG can run script keep it code with the security class, and other non-drawing constructs (external images, any CSS beyond declarations and `url(#id)`, animations, entity declarations) keep it code; session data files leave `SCOPE_AUTH` false while session code and config set it; a stale local `main` and tags named `main` or `origin/main` leave the scope and the receipt base unchanged. + +- `gstack-gate-diff` and `gstack-diff-scope` now read a bare base name such as `main` as origin's tracking branch (`refs/remotes/origin/main`) when it exists and shares history with HEAD. That's the base /ship fetches and merges against. Before, a local `main` that was behind `origin/main` set the merge base. That happens in a linked worktree, and on a Capy machine, where local `main` never moves. So the gate scan and the specialist scope also read commits already on `origin/main`. One /ship run in #3650 listed a `.github/workflows/ci.yml` hunk from upstream and dispatched an extra performance reviewer. `gstack-gate-diff` now says `local main is not origin/main; base is origin/main` on stderr when that happens. A base you name with its remote (`upstream/main`), a commit or `HEAD` is used as given. A tag named `origin/main` or a fork's `origin/upstream/main` branch can't stand in for the base, and another remote's branch only replaces a local base that is missing or unrelated (#3650). + +### Fixed + +- **The SVG drawing check runs in linear time.** One changed SVG with a long run of spaces inside `url(` could stall `gstack-review-plan` and the queue's depth check for hours (80,000 spaces took 0.8 s, a 1 MB run over 230 s). The check now reads it in about a millisecond and refuses exactly what it refused before. +- **An SVG whose DOCTYPE points at an outside document stays code.** Only a bare `` or W3C's own SVG DTD still counts as a drawing, so an SVG with a `SYSTEM` DTD keeps its full review. +- **`gstack-diff-scope` picks the same base as `gstack-gate-diff`.** When `origin/` is missing or unrelated, a related local `` now comes before another remote's branch in both tools. + ## [1.92.75.0] - 2026-11-02 - **The /ship receipt counts every blocker its readers found** (GST-951). `gstack-review-scope summary` read only the review-log rows, but /ship's readers report through the dispatch ledgers (`gstack-review-dispatch collect` and `judge`). So a blocker that only the native adversarial pass raised, a CRITICAL logged fixed without classify's blocking stamp, and findings logged before they were classified all left `blocking_found` at 0. That happened on #3841, #3838 and #3837. The summary now also reads each ledger's collected and judged CRITICAL findings, pairs each one with at most one row finding from the same reviewer at the same path:line so a blocker held in both places counts once and one classified below the floor does not, and counts an unclassified CRITICAL row as blocking. `unrecorded=N` names blockers no row gives a disposition, and `ledgers_read=N` says how many it read. diff --git a/VERSION b/VERSION index c285791ec..730cd3e00 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.92.75.0 +1.92.76.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index a7cd1c88f..f297fb296 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.92.75.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.92.76.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/bin/gstack-diff-scope b/bin/gstack-diff-scope index 3ec81cf09..7609aefbf 100755 --- a/bin/gstack-diff-scope +++ b/bin/gstack-diff-scope @@ -36,6 +36,46 @@ _default_base() { BASE="${1:-$(_default_base)}" HEAD_REF="" if [ "${2:-}" = "--head" ]; then HEAD_REF="${3:-}"; fi +# A bare branch name reads origin's tracking branch, by its full ref, when that +# shares history with HEAD: /ship merges against origin/, and a linked +# worktree or a Capy clone never moves the local branch, so a stale one widens +# the scope (#3650). A remote-qualified name is taken as given. +_NOT_BRANCH='^[0-9a-f]{7,64}$|[~^]|@\{|^[A-Z_]*HEAD$|^refs/' +_QUALIFIED=false +for _remote in $(git remote 2>/dev/null); do + case "$BASE" in "$_remote"/*) _QUALIFIED=true ;; esac +done +if [[ ! "$BASE" =~ $_NOT_BRANCH ]] && [ "$_QUALIFIED" = false ] \ + && git rev-parse --verify -q "refs/remotes/origin/${BASE}^{commit}" >/dev/null 2>&1 \ + && git merge-base "refs/remotes/origin/${BASE}" "${HEAD_REF:-HEAD}" >/dev/null 2>&1; then + BASE="refs/remotes/origin/${BASE}" +fi + +# The base as a commit (GST-884, mirroring lib/gate-diff/git.ts resolveBase from +# GST-883): a branch name reads refs/remotes/ (origin/main as given), then +# refs/remotes/origin/ that shares history with the head, then refs/heads/ +# that does, then another remote's twin that does, then refs/heads/ as it is, +# all by full ref name and in resolveBase's order, so a stale local main never widens +# the scope and a tag named main or origin/main cannot stand in for the branch. +# A sha, a relative ref (HEAD~1, main^2, @{u}) or a full refs/ name resolves as +# given. Prints nothing when nothing resolves. +_is_branch_name() { + case "$1" in refs/*|*~*|*^*|*@\{*|HEAD|*_HEAD) return 1 ;; esac + [[ ! "$1" =~ ^[0-9a-f]{7,64}$ ]] +} +_commit() { git rev-parse --verify -q "$1^{commit}" 2>/dev/null; } +_resolve_base() { + local ref="$1" head="$2" sha loc remote + if ! _is_branch_name "$ref"; then _commit "$ref"; return; fi + sha=$(_commit "refs/remotes/$ref") && { echo "$sha"; return; } + loc=$(_commit "refs/heads/$ref") + for remote in $(git remote 2>/dev/null | grep -x origin) ${loc:+:local} $(git remote 2>/dev/null | grep -vx origin); do + if [ "$remote" = :local ]; then sha=$loc; else sha=$(_commit "refs/remotes/$remote/$ref") || continue; fi + git merge-base "$sha" "$head" >/dev/null 2>&1 && { echo "$sha"; return; } + done + [ -n "$loc" ] && { echo "$loc"; return; } + _commit "$ref" +} FRONTEND=false BACKEND=false @@ -53,6 +93,19 @@ AUTH=false # the root-anchored globs alone (#3335). TEST_PATH_RE='(^|/)(tests?|spec|__tests__)/|(^|/)tests?\.py$|(^|/)test_[^/]+\.py$|_test\.(go|py|rb|ts|js|exs|dart)$|\.(test|spec)\.[jt]sx?$|_spec\.rb$|Test\.(java|kt)$' +# Data files (lib/review-depth.ts DATA_PATH) outside the config locations its +# DATA_CONFIG and agent-prose rules keep code: .github/, config/, dotfiles, +# *.config.*, .gstack/. Case-insensitive, as DATA_PATH is. +DATA_FILE_RE='\.(json|jsonl|ndjson|csv|tsv|png|jpe?g|gif|webp|bmp|ico|avif)$' +DATA_CONFIG_RE='(^|/)\.github/|(^|/)config/|(^|/)\.[^/]+$|(^|/)[^/]+\.config\.[^/]+$|^\.gstack/' +_session_data() { + local r=1 + shopt -s nocasematch + if [[ "$1" =~ $DATA_FILE_RE ]] && [[ ! "$1" =~ $DATA_CONFIG_RE ]]; then r=0; fi + shopt -u nocasematch + return "$r" +} + _print_flags() { echo "SCOPE_FRONTEND=$FRONTEND" echo "SCOPE_BACKEND=$BACKEND" @@ -68,7 +121,8 @@ _print_flags() { # Base reachability (#2526): a shallow CI checkout or an unfetched ref makes # `git diff` return an empty list — all-false with exit 0, a green that means # "we could not look". Distinguish it before diffing. -if ! git rev-parse --verify -q "${BASE}^{commit}" >/dev/null 2>&1; then +BASE_SHA=$(_resolve_base "$BASE" "${HEAD_REF:-HEAD}" || true) +if [ -z "$BASE_SHA" ]; then _print_flags echo "SCOPE_ERROR=no_base" echo "# base ref '${BASE}' is not resolvable — shallow checkout or missing fetch. Run: git fetch origin ${BASE}" @@ -91,16 +145,16 @@ if [ -n "$HEAD_REF" ] || [ "${2:-}" = "--head" ]; then echo "# --head '${HEAD_REF}' is not a commit in this repository" exit 2 fi - if ! git merge-base "${BASE}" "${HEAD_REF}" >/dev/null 2>&1; then + if ! git merge-base "${BASE_SHA}" "${HEAD_REF}" >/dev/null 2>&1; then _print_flags echo "SCOPE_ERROR=no_merge_base" echo "# '${BASE}' and '${HEAD_REF}' share no merge base" exit 2 fi - _collect < <(git diff -z "${BASE}...${HEAD_REF}" --name-only 2>/dev/null) + _collect < <(git diff -z "${BASE_SHA}...${HEAD_REF}" --name-only 2>/dev/null) else # Committed diff vs base (merge-base form; two-dot fallback when no merge base). -_collect < <(git diff -z "${BASE}...HEAD" --name-only 2>/dev/null || git diff -z "${BASE}" --name-only 2>/dev/null || true) +_collect < <(git diff -z "${BASE_SHA}...HEAD" --name-only 2>/dev/null || git diff -z "${BASE_SHA}" --name-only 2>/dev/null || true) # Working-tree changes (staged + unstaged). `git diff HEAD` fails on a repo # with no commits; tolerated. _collect < <(git diff -z HEAD --name-only 2>/dev/null || true) @@ -190,9 +244,12 @@ for f in ${FILES_LIST[@]+"${FILES_LIST[@]}"}; do *.graphql|*.gql|openapi.*|swagger.*) m_api=true ;; esac - # Auth: authentication, authorization, sessions, permissions + # Auth: authentication, authorization, sessions, permissions. A data file + # named for sessions (recovered-sessions.jsonl, GST-911) holds rows, not + # session handling, so it is not auth; session config still is. case "$f" in - *auth*|*session*|*jwt*|*oauth*|*permission*|*role*) m_auth=true ;; + *auth*|*jwt*|*oauth*|*permission*|*role*) m_auth=true ;; + *session*) _session_data "$f" || m_auth=true ;; esac # Backend: code that isn't a frontend component/view file. Includes ESM/CJS diff --git a/bin/gstack-gate-diff b/bin/gstack-gate-diff index 0a1af715c..400a9a56d 100755 --- a/bin/gstack-gate-diff +++ b/bin/gstack-gate-diff @@ -39,7 +39,8 @@ const BIN_DIR = path.dirname(Bun.fileURLToPath(import.meta.url)); const USAGE = `usage: gstack-gate-diff [base-ref] [--format table|jsonl] [--commit ] [--help] Lists every hunk in a test, CI, runner/lint-config, snapshot or golden path of the -candidate (merge-base of and HEAD -> working tree + untracked files), +candidate (merge-base of and HEAD -> working tree + untracked files; +a branch name resolves to origin/, the local ref only when no remote has it), tags the gate edits it recognises (RH-1 RH-3 RH-4 RH-12 RH-13 RH-14 RH-15 RH-16), and prints the read-level listing after one GATE_SUMMARY: line. --commit scans that commit's first-parent delta instead of the working tree. @@ -148,7 +149,8 @@ function main(argv: string[]): number { process.stdout.write(`GATE_ERROR=no_base ref=${baseRef} fix=${noBaseFix(baseRef, root)}\n`); return 2; } - if (base.local) console.error(`gstack-gate-diff: ${base.local === 'unrelated' ? `local ${baseRef} shares no history with HEAD` : `no local branch ${baseRef}`}; base is ${base.ref}`); + if (base.local === 'fallback') console.error(`gstack-gate-diff: ${base.unrelatedRemote ? `${base.unrelatedRemote} shares no history with HEAD` : `no remote-tracking branch for ${baseRef}`}; base is the local ${baseRef}, which may lag its remote`); + else if (base.local) console.error(`gstack-gate-diff: ${base.local === 'unrelated' ? `local ${baseRef} shares no history with HEAD` : base.local === 'differs' ? `local ${baseRef} is not ${base.ref}` : `no local branch ${baseRef}`}; base is ${base.ref}`); const mb = mergeBase(base.sha, head, root); if (!mb) { process.stdout.write(`GATE_ERROR=no_base ref=${baseRef} fix=${noBaseFix(baseRef, root)}\n`); diff --git a/bin/gstack-retro-metrics b/bin/gstack-retro-metrics index 40bb4f43c..d49abd3cb 100755 --- a/bin/gstack-retro-metrics +++ b/bin/gstack-retro-metrics @@ -28,6 +28,19 @@ # --since midnight-aligned ISO date ("2026-03-11T00:00:00") or a relative # expression ("24 hours ago"). Default: "7 days ago". # --until optional window end (compare mode's prior window). +# +# gstack-retro-metrics --history [--next ] +# +# Retro history for this project (#3749): prints RETRO_HISTORY_DIR (the +# state root's projects//retros, shared by every worktree of the +# repository) and one RETRO_PRIOR line per snapshot, newest first. Moves any +# legacy .context/retros/*.json from this worktree and the main checkout +# into it first: a copy identical to a stored snapshot of the same date +# collapses, a same-name snapshot that differs takes the next free number +# for its date, and a .context/retros that resolves outside its worktree +# is left alone. --next also reserves the next unused +# -.json (written as {} so a concurrent run cannot take it) and +# prints it as RETRO_FILE. GSTACK_BUN_CONFIG=/dev/null; case "${OSTYPE:-}" in msys*|cygwin*) if command -v cygpath >/dev/null 2>&1; then GSTACK_BUN_CONFIG=NUL; fi ;; esac @@ -36,9 +49,11 @@ SINCE="7 days ago" UNTIL="" while [ $# -gt 0 ]; do case "$1" in - --base) BASE="$2"; shift 2 ;; - --since) SINCE="$2"; shift 2 ;; - --until) UNTIL="$2"; shift 2 ;; + --base) BASE="$2"; shift 2 || break ;; + --since) SINCE="$2"; shift 2 || break ;; + --until) UNTIL="$2"; shift 2 || break ;; + --history) HISTORY=1; shift ;; + --next) NEXT_DATE="$2"; shift 2 || break ;; *) shift ;; esac done @@ -51,6 +66,62 @@ _BIN="$_SCRIPT_DIR" gstack_state_root_select _GH="$_gstack_sr_root" +if [ -n "${HISTORY:-}" ]; then + _SLUG=$("$_BIN/gstack-slug" --get SLUG 2>/dev/null || true) + if [ -z "$_SLUG" ]; then + echo "RETRO_HISTORY_ERROR: could not resolve the project slug (gstack-slug failed). fix: reinstall with ./setup or /gstack-upgrade" + exit 0 + fi + _HD="$_GH/projects/$_SLUG/retros" + mkdir -p "$_HD" || { echo "RETRO_HISTORY_ERROR: cannot create $_HD"; exit 0; } + if [ -n "${NEXT_DATE:-}" ]; then + case "$NEXT_DATE" in + [0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]) ;; + *) echo "RETRO_HISTORY_ERROR: --next takes a YYYY-MM-DD date, got '$NEXT_DATE'"; exit 0 ;; + esac + fi + _HDR=$(cd -P "$_HD" && pwd) + _TOP=$(git rev-parse --show-toplevel 2>/dev/null || true) + _MAIN=$(git worktree list --porcelain 2>/dev/null | sed -n '1s/^worktree //p') + _MOVED=0 + for _TREE in "$_TOP" "$_MAIN"; do + [ -n "$_TREE" ] && [ -d "$_TREE/.context/retros" ] || continue + _SRC=$(cd -P "$_TREE/.context/retros" 2>/dev/null && pwd) || continue + _TR=$(cd -P "$_TREE" 2>/dev/null && pwd) || continue + case "$_SRC" in "$_TR"/*) ;; *) continue ;; esac + [ "$_SRC" != "$_HDR" ] || continue + for _F in "$_SRC"/*.json; do + [ -f "$_F" ] && [ ! -L "$_F" ] || continue + _B=$(basename "$_F") + _DUP="" + for _E in "$_HD/$_B" "$_HD/${_B%-*}"-*.json; do + [ -f "$_E" ] && cmp -s "$_F" "$_E" && { _DUP=1; break; } + done + if [ -n "$_DUP" ]; then rm -f "$_F"; continue; fi + _DEST="$_HD/$_B" + _N=1 + while [ -e "$_F" ] && [ "$_N" -le 1000 ]; do + [ -e "$_DEST" ] || mv -n "$_F" "$_DEST" 2>/dev/null + [ -e "$_F" ] || { _MOVED=$((_MOVED + 1)); break; } + _N=$((_N + 1)); _DEST="$_HD/${_B%-*}-$_N.json" + done + done + rmdir "$_TREE/.context/retros" 2>/dev/null || true + done + echo "RETRO_HISTORY_DIR: $_HD" + [ "$_MOVED" -gt 0 ] && echo "RETRO_HISTORY_MIGRATED: $_MOVED from .context/retros" + ls -t "$_HD"/*.json 2>/dev/null | sed 's/^/RETRO_PRIOR: /' + if [ -n "${NEXT_DATE:-}" ]; then + _N=1 + until ( set -C; printf '{}\n' > "$_HD/$NEXT_DATE-$_N.json" ) 2>/dev/null; do + _N=$((_N + 1)) + [ "$_N" -le 1000 ] || { echo "RETRO_HISTORY_ERROR: cannot reserve a snapshot file in $_HD"; exit 0; } + done + echo "RETRO_FILE: $_HD/$NEXT_DATE-$_N.json" + fi + exit 0 +fi + echo "RETRO_METRICS_PROTO: 1" if ! git rev-parse --git-dir >/dev/null 2>&1; then diff --git a/bin/gstack-review-plan b/bin/gstack-review-plan index 93f6aee5e..40610335b 100755 --- a/bin/gstack-review-plan +++ b/bin/gstack-review-plan @@ -116,8 +116,9 @@ function main(): number { const m = /GATE_SUMMARY: .*?tagged=\{([^}]*)\}/.exec(g.stdout ?? ''); return g.status === 0 && m ? m[1]!.split(',').reduce((n, kv) => n + (Number(kv.split(':')[1]) || 0), 0) : null; }; - const scope = scopeOf([base]); - let gateTagged = flag('--gate-tagged') !== undefined ? Number(flag('--gate-tagged')) : gateTagsOf([base]); + const diffBase = collected.mergeBase ?? base; + const scope = scopeOf([diffBase]); + let gateTagged = flag('--gate-tagged') !== undefined ? Number(flag('--gate-tagged')) : gateTagsOf([diffBase]); if (gateTagged !== null && !Number.isFinite(gateTagged)) gateTagged = null; const force = flag('--force') === 'all' ? 'all' as const : list('--force'); const tiersSetting = runScriptSync(path.join(binDir, 'gstack-config'), ['get', 'review_effort_tiers'], { encoding: 'utf8', timeout: 15_000 }); @@ -149,7 +150,7 @@ function main(): number { const gateScan = { tagged: gateTagged, head }; if (has('--json')) { const { runtimeLines, stepDepth, depthReasons, ...rest } = plan; - process.stdout.write(JSON.stringify({ ...rest, targeted: { ...targeted, brief, targeted_patch_id: patch }, step_depth: stepDepth, runtime_lines: runtimeLines, depth_reasons: depthReasons, depth_floor: floor ?? null, gate_scan: gateScan, fix_range: fixRange ?? null, registry: { sha: registry.sha, outputs: registry.outputs }, per_path: perPath, diff_lines: diffLines, pass, audit_effort: auditEffort, scope, gate_tagged: gateTagged, paths, deleted, stamp_only: stampOnly, consumed: collected.consumed }) + '\n'); + process.stdout.write(JSON.stringify({ ...rest, targeted: { ...targeted, brief, targeted_patch_id: patch }, step_depth: stepDepth, runtime_lines: runtimeLines, depth_reasons: depthReasons, depth_floor: floor ?? null, gate_scan: gateScan, fix_range: fixRange ?? null, registry: { sha: registry.sha, outputs: registry.outputs }, per_path: perPath, diff_lines: diffLines, pass, audit_effort: auditEffort, scope, gate_tagged: gateTagged, paths, deleted, stamp_only: stampOnly, consumed: collected.consumed, drawings: collected.drawings }) + '\n'); return EXIT.ok; } const out = [`REVIEW_CLASS: ${plan.class} (${plan.reasons.slice(0, 3).join('; ') || 'no signals'})`, reviewDepthLine(plan), `GATE_SCAN: ${gateTagged === null ? 'unavailable' : `tagged=${gateTagged}`} head=${head}`, diff --git a/docs/designs/HONEST_WORK_GATE_INTEGRITY.md b/docs/designs/HONEST_WORK_GATE_INTEGRITY.md index adfad8d8c..28a8563d9 100644 --- a/docs/designs/HONEST_WORK_GATE_INTEGRITY.md +++ b/docs/designs/HONEST_WORK_GATE_INTEGRITY.md @@ -180,9 +180,12 @@ only history need is calibration, which `--commit ` covers by scanning one commit's first-parent delta without the working tree.) Positional base defaults the way `gstack-diff-scope` does (the remote's `HEAD` target, origin first, then the other remotes → `main` → `master`); an -already-qualified ref is used as given, and a branch name with no local -branch resolves through its remote-tracking twin (`origin/main` for `main`: -the merge-base is the same commit), noted once on stderr. stdout: one +already-qualified ref is used as given, and a branch name resolves through +its remote-tracking twin (`origin/main` for `main`), so a stale local `main` +never adds commits the branch only inherited (GST-883). Only when no remote +has the branch does the local ref stand, noted once on stderr (a repository +with no remote at all uses it quietly). Branches are read by full ref name, +so a tag named `main` or `origin/main` cannot stand in for them. stdout: one `GATE_SUMMARY: listed=N read=N tagged={RH-1:2,RH-15:1} unmatched=N coverage= languages_unlisted=<…> candidate=` line, then the read-level listing (bounded; the full listing goes to `--out`). diff --git a/docs/review-policy.md b/docs/review-policy.md index a5fd51cb7..db10c3753 100644 --- a/docs/review-policy.md +++ b/docs/review-policy.md @@ -181,7 +181,14 @@ follows that file, four files deep. Precedence: agent prose (`.gstack/**`), a te fixture path, a gate path, a config file (`.github/**`, anything under a `config/` directory, dotfiles, `*.config.*`, `package.json`, lockfiles, `tsconfig*.json`, `manifest.json` and the other tool manifests `lib/review-depth.ts` lists) and a named -consumer each keep the path code, and YAML, TOML and SVG are always code. More than +consumer each keep the path code, and YAML and TOML are always code. An SVG counts +as data only when its whole text at the head is a drawing (GST-911): SVG drawing +elements and RDF metadata, CSS made of declarations only (no `@` rule and no +function but a same-file `url(#id)`, which closes `@import`, `image-set()`, +`image()`, `element()` and every other CSS load), no backslash escape, every +`href` a same-file `#` reference or an inline raster image, no event handler and no +entity declaration, and no DOCTYPE but a bare `` or W3C's own SVG DTD, read after character references are decoded; anything else, a +script or `foreignObject` included, keeps it code. More than 2,000 data paths keep every one code; a `git grep` failure is `consumer_scan_failed` and fails the depth closed. Only `classifyDiff` applies the rule (`lib/review-collect.ts` finds the consumers, `gstack-ship-preflight`'s `RUNTIME_LINES` reads the same @@ -215,23 +222,25 @@ input, redirect toggles, proxy settings and host allowlists, archive extraction and path traversal (`tar.x`, adm-zip, yauzl, extract-zip, `extractall`, `tar -x`, `unzip`, `path.join` or `resolve` on request or entry input, a `'..'` check added or removed), and any changed line that names the `child_process` or -`node:child_process` module, in any import or require spelling. The GST-850 +`node:child_process` module, in any import or require spelling. A changed SVG +line that can run code (a script or `foreignObject` element, an event handler, a +`javascript:` URL) is security too (GST-911). The GST-850 rules are coarse on purpose and parse no import clause or call arguments; `test/review-risk-linear.test.ts` holds each of them under 100 ms on 200,000-character hostile lines, its anchor followed by a long run of the -characters it repeats included. That claim covers only the rules GST-850 adds, -not R51's older ones. **data** is user data: a path naming pii, gdpr, ccpa, privacy, +characters it repeats included. `test/review-risk-redos.test.ts` (GST-898) holds every risk pattern, R51's `rm` and `rmSync` rules included, under 100 ms on 100,000-character hostile lines. An `rmSync(` call with more than 500 characters before the first `)` after it counts as security on its own. **data** is user data: a path naming pii, gdpr, ccpa, privacy, personal, account, customer, cookie, persistence, storage or database, or a line that drops a table or column, deletes rows or bulk-deletes records. Auth scope, migrations and deleted code paths were already gate class. `RISK_RULES` hashes the patterns and `DEPTH_RULES` includes it. -**UI text and SVGs** stay code (light unless a risk class applies). A docs path -for UI copy, locale values and chart SVGs was built as an allowlist in GST-850 and -dropped for review cost: its first review round found a new bypass class in each -version, the same pattern as three rounds of R51. When a diff publishes numbers, -the batched specialist's diff keeps the write-up's text files instead of leaving -docs out. +**UI text** stays code (light unless a risk class applies). A docs path for UI +copy and locale values was built as an allowlist in GST-850 and dropped for +review cost: its first review round found a new bypass class in each version, +the same pattern as three rounds of R51. Chart SVGs use the drawing allowlist +above, which reads the whole file rather than changed lines. When a diff +publishes numbers, the batched specialist's diff keeps the write-up's text files +instead of leaving docs out. **Batched checklists (the lean ensemble, GSTA-167 part 2).** One specialist subagent reads Testing and Security on every code or gate diff, plus @@ -350,7 +359,7 @@ and matching is token-bounded: `costume` never reads as cost. |---|---| | money | a path segment naming cost, spend, budget, ledger, billing, price, fee or invoice; or a changed line with an identifier ending in `_usd` or `Usd`, any `spend_cap` spelling, or a call to a paid provider `lib/pricing.ts` prices | | gate | a hunk the gate scan tagged: a relaxed threshold, a skipped test, a loosened check | -| auth | `SCOPE_AUTH` from `gstack-diff-scope`, on a code path with an auth, session, jwt, permission or role token in its name | +| auth | `SCOPE_AUTH` from `gstack-diff-scope`, on a code path with an auth, session, jwt, permission or role token in its name; a data file named for sessions (`recovered-sessions.jsonl`) outside a config location does not set `SCOPE_AUTH` | | secret | a secrets, credentials, keychain or token path, or a changed line that reads a key-named environment variable (`OPENAI_API_KEY`, `GH_TOKEN`) or a credential store | An input the classifier cannot read (the diff, `gstack-diff-scope`, the gate diff --git a/docs/skills.md b/docs/skills.md index 335da1795..3622a6d8f 100644 --- a/docs/skills.md +++ b/docs/skills.md @@ -1119,7 +1119,7 @@ Claude: Week of Mar 1: 47 commits (3 contributors), 3.2k LOC, 38% tests, 12 PRs, [Top 3 team wins, 3 things to improve, 3 habits for next week] ``` -It saves a JSON snapshot to `.context/retros/` so the next run can show trends. +It saves a JSON snapshot under `~/.gstack/projects//retros/`, shared by every worktree of the repository, so the next run can show trends from any checkout. An older `.context/retros/` is moved there on the first run. --- diff --git a/lib/gate-diff/git.ts b/lib/gate-diff/git.ts index a9d4f07c0..ecc03e578 100644 --- a/lib/gate-diff/git.ts +++ b/lib/gate-diff/git.ts @@ -85,24 +85,52 @@ export function isShallow(cwd: string): boolean { /** * The base as a ref that resolves here and, when `head` is given, shares - * history with it: the ref itself, else its remote-tracking branch - * (`origin/main` for `main`). A clone that checked out only the feature - * branch has no local `main`, and the merge-base against `origin/main` is the - * same commit; a stale local `main` that shares no history with HEAD (a - * fetched-over clone) yields to the twin too, so the fix line's fetch into the - * tracking ref is a fix and not a loop. + * history with it. A bare branch name resolves to origin's tracking branch + * (`refs/remotes/origin/main` for `main`), the base /ship merges against and + * the one review-collect's collectLive and review-dispatch read: a stale local + * `main` behind it would scan upstream commits the branch only inherited + * (#3650, GST-883), so a local branch that differs is reported as `differs`. + * Without a related origin twin the related local branch wins over any other + * remote, so a fork's branch cannot replace it; it is `fallback` when origin + * lacks the branch or shares no history with HEAD (named as `unrelatedRemote`), + * and plain with no origin. Another remote's tracking branch is used only when + * no related local branch exists. A clone that checked out only the feature + * branch has no local `main` (`missing`); a local `main` that shares no + * history with HEAD is `unrelated`, so the fix line's fetch into the tracking + * ref is a fix and not a loop. Branches are read by their full names + * (`refs/heads/`, `refs/remotes/`), so a tag named `main` or `origin/main` + * cannot stand in for them, and a remote-qualified name (`upstream/main`) is + * taken as given. */ -export function resolveBase(ref: string, cwd: string, head?: string): { ref: string; sha: string; local?: 'missing' | 'unrelated' } | null { +export function resolveBase(ref: string, cwd: string, head?: string): { ref: string; sha: string; local?: 'missing' | 'unrelated' | 'differs' | 'fallback'; unrelatedRemote?: string } | null { const related = (sha: string) => !head || mergeBase(sha, head, cwd) !== null; - const direct = resolvesToCommit(ref, cwd); - if (direct && related(direct)) return { ref, sha: direct }; - if (!isBranchName(ref)) return direct ? { ref, sha: direct } : null; - for (const remote of remoteNames(cwd)) { - const tracking = `${remote}/${ref}`; - const sha = resolvesToCommit(tracking, cwd); - if (sha && related(sha)) return { ref: tracking, sha, local: direct ? 'unrelated' : 'missing' }; + const direct = () => resolvesToCommit(ref, cwd); + if (!isBranchName(ref)) { const sha = direct(); return sha ? { ref, sha } : null; } + const qualified = resolvesToCommit(`refs/remotes/` + ref, cwd); + if (qualified) return { ref, sha: qualified }; + const local = resolvesToCommit(`refs/heads/` + ref, cwd); + const remotes = remoteNames(cwd); + const localRelated = !!local && related(local); + const twin = (remote: string) => { + const sha = resolvesToCommit(`refs/remotes/` + remote + '/' + ref, cwd); + if (!sha || !related(sha)) return null; + const tracking = remote + '/' + ref; + return sha === local ? { ref: tracking, sha } : { ref: tracking, sha, local: !local ? 'missing' as const : localRelated ? 'differs' as const : 'unrelated' as const }; + }; + const hasOrigin = remotes.includes('origin'); + const origin = hasOrigin ? twin('origin') : null; + if (origin) return origin; + const originRef = resolvesToCommit(`refs/remotes/origin/` + ref, cwd); + const fallback = hasOrigin ? { local: 'fallback' as const, ...(originRef ? { unrelatedRemote: 'origin/' + ref } : {}) } : {}; + if (localRelated) return { ref, sha: local!, ...fallback }; + for (const remote of remotes) { + if (remote === 'origin') continue; + const other = twin(remote); + if (other) return other; } - return direct ? { ref, sha: direct } : null; + if (local) return { ref, sha: local, ...(remotes.length ? { local: 'fallback' as const, ...(originRef ? { unrelatedRemote: 'origin/' + ref } : {}) } : {}) }; + const sha = direct(); + return sha ? { ref, sha } : null; } /** A word for a printed shell line: as is when it is plain, single-quoted otherwise (ref names may hold `;`, `$`, `(`, `{1,2}`). */ diff --git a/lib/review-collect.ts b/lib/review-collect.ts index 8df418713..7e456bd5e 100644 --- a/lib/review-collect.ts +++ b/lib/review-collect.ts @@ -12,7 +12,7 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import { GENERATED_PATH, parseRegistry } from './regen'; import { RESULT_CODES } from './result-codes'; -import { classifyPath, dataCandidate, type ClassInput, type DepthError, type PathLines, type RegistryInfo } from './review-depth'; +import { classifyPath, dataCandidate, svgCandidate, svgDrawing, type ClassInput, type DepthError, type PathLines, type RegistryInfo } from './review-depth'; export interface Collected extends ClassInput { perPath: PathLines[]; @@ -30,6 +30,7 @@ export interface Collected extends ClassInput { mergeBase: string | null; untracked: string[]; consumed: string[] | 'all'; + drawings: string[]; } function git(root: string, args: string[]): { ok: boolean; status: number | null; out: string; err: string } { @@ -109,7 +110,7 @@ export const CONSUMER_SCAN_DEPTH = 4; * a collector error, which fails the depth closed. */ export function scanConsumers(root: string, paths: readonly string[], revs: readonly string[], live: boolean, classRoot: string | readonly string[], errors: DepthError[]): string[] | 'all' { - const candidates = [...new Set(paths)].filter(p => dataCandidate(p, classRoot)).sort(); + const candidates = [...new Set(paths)].filter(p => dataCandidate(p, classRoot) || svgCandidate(p, classRoot)).sort(); if (candidates.length > CONSUMER_SCAN_CAP) return 'all'; const base = (p: string) => p.slice(p.lastIndexOf('/') + 1); const named = new Map>(); @@ -149,9 +150,20 @@ export function scanConsumers(root: string, paths: readonly string[], revs: read return candidates.filter(p => consumed.has(p)); } -function result(diff: ReturnType, extra: Pick): Collected { +/** The largest SVG whose text is read for the drawing check; a larger one stays code. */ +export const DRAWING_READ_CAP = 8 * 1024 * 1024; + +/** The changed SVG candidates whose whole text, read by `read` (null when it cannot be read), is a drawing (lib/review-depth.ts svgDrawing). */ +function drawingsAmong(perPath: readonly PathLines[], classRoot: string | readonly string[], read: (p: string) => Buffer | null): string[] { + return perPath.filter(e => !e.binary && svgCandidate(e.path, classRoot)).map(e => e.path).filter(p => { + const buf = read(p); + return !!buf && buf.length <= DRAWING_READ_CAP && !buf.includes(0) && svgDrawing(buf.toString('utf8')); + }).sort(); +} + +function result(diff: ReturnType, extra: Pick & { drawings?: string[] }): Collected { const paths = diff.perPath.map(e => e.path).sort(); - return { ...diff, ...extra, paths, gateTagged: null, diffLines: diff.perPath.reduce((n, e) => n + e.lines, 0) }; + return { ...diff, drawings: [], ...extra, paths, gateTagged: null, diffLines: diff.perPath.reduce((n, e) => n + e.lines, 0) }; } /** The range `baseSha`...`headSha` from git objects alone; the registry is the base commit's, never the head's. */ @@ -168,13 +180,17 @@ export function collectCommit(root: string, baseSha: string, headSha: string): C const tree = git(root, ['ls-tree', '-r', '--name-only', '-z', headSha]); const skillDirs = [...new Set(tree.out.split('\0').flatMap(p => /^([^/]+)\/SKILL\.md(\.tmpl)?$/.exec(p)?.[1] ?? []))].sort(); const consumed = scanConsumers(root, [...diff.perPath.map(e => e.path), ...diff.deleted], [headSha, mergeBase], false, skillDirs, errors); - return result(diff, { root: skillDirs, registry, errors, mergeBase, untracked: [], consumed }); + const drawings = drawingsAmong(diff.perPath, skillDirs, p => { + const blob = spawnSync('git', ['cat-file', 'blob', `${headSha}:${p}`], { cwd: root, timeout: 60_000, maxBuffer: DRAWING_READ_CAP + 1 }); + return blob.status === 0 && !blob.error ? blob.stdout : null; + }); + return result(diff, { root: skillDirs, registry, errors, mergeBase, untracked: [], consumed, drawings }); } /** The working tree against the merge base with origin/ (or ), untracked files counted per path; the registry is the base ref's commit. */ export function collectLive(root: string, base: string): Collected { const errors: DepthError[] = []; - const ref = git(root, ['rev-parse', '--verify', '-q', `origin/${base}^{commit}`]).ok ? `origin/${base}` : base; + const ref = [`refs/remotes/origin/${base}`, `refs/heads/${base}`].find(r => git(root, ['rev-parse', '--verify', '-q', `${r}^{commit}`]).ok) ?? base; const registry = loadRegistryAt(root, ref); const mb = git(root, ['merge-base', ref, 'HEAD']); if (!mb.ok) { @@ -196,5 +212,8 @@ export function collectLive(root: string, base: string): Collected { if (!binary) { diff.added[p] = text.split('\n'); diff.created.push(p); } } const consumed = scanConsumers(root, [...diff.perPath.map(e => e.path), ...diff.deleted], [mergeBase], true, root, errors); - return result(diff, { root, registry, errors, mergeBase, untracked, consumed }); + const drawings = drawingsAmong(diff.perPath, root, p => { + try { return fs.statSync(path.join(root, p)).size <= DRAWING_READ_CAP ? fs.readFileSync(path.join(root, p)) : null; } catch { return null; } + }); + return result(diff, { root, registry, errors, mergeBase, untracked, consumed, drawings }); } diff --git a/lib/review-depth.ts b/lib/review-depth.ts index 268d40de7..59340ac8e 100644 --- a/lib/review-depth.ts +++ b/lib/review-depth.ts @@ -13,7 +13,7 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import type { ResultCodeName } from './result-codes'; import { matchesAny } from './ship-policy'; -import { matchRisk, RISK_RULES, type RiskClass } from './review-risk'; +import { decodeCharRefs, matchRisk, RISK_RULES, SVG_PATH, svgActive, type RiskClass } from './review-risk'; export type DiffClass = 'docs' | 'tests' | 'code' | 'gate'; export type PathClass = 'docs' | 'tests' | 'code'; @@ -58,7 +58,7 @@ const GATE_PATH = [ /^lib\/review-(depth|collect|scope|targeted|risk)\.ts$/, /^review\/specialists\/targeted\.md$/, /^bin\/gstack-review-plan$/, /^lib\/ship-queue[^/]*\.ts$/, /^lib\/regen\.ts$/, /^\.gstack\/generated\.json$/, /^ship\/sections\/(review-army|adversarial)\.md\.tmpl$/, /^scripts\/resolvers\/review-army\.ts$/, ]; -/** Data files: JSON, CSV, TSV and raster images. YAML, TOML and SVG are never data here: deploy configs, API specs and executable SVG stay code. */ +/** Data files: JSON, CSV, TSV and raster images. YAML and TOML are never data here: deploy configs and API specs stay code. An SVG is data only as a drawing (svgDrawing). */ export const DATA_PATH = /\.(json|jsonl|ndjson|csv|tsv|png|jpe?g|gif|webp|bmp|ico|avif)$/i; /** Data-shaped files tools read by convention, with no code naming them: always code. */ const DATA_CONFIG = [ @@ -66,10 +66,89 @@ const DATA_CONFIG = [ /(^|\/)(package|package-lock|npm-shrinkwrap|composer|deno|jsr|tsconfig[^/]*|jsconfig[^/]*|manifest|app|vercel|netlify|firebase|renovate|turbo|nx|lerna|biome|angular|nest-cli|lint-staged|components|pyrightconfig|cdk|serverless|appsettings[^/]*)\.json$/i, ]; +/** + * The SVG elements a chart is drawn with, lowercased, under any namespace prefix: shapes, text, paint servers, clip paths, + * filters (feImage loads a document, so it is left out), CSS, and the RDF metadata plotting libraries write. No script, + * foreignObject, link, animation or embedded document. + */ +const SVG_ELEMENTS = new Set([ + 'svg', 'g', 'defs', 'symbol', 'use', 'path', 'rect', 'circle', 'ellipse', 'line', 'polyline', 'polygon', 'text', 'tspan', 'textpath', 'title', 'desc', + 'metadata', 'clippath', 'mask', 'pattern', 'marker', 'lineargradient', 'radialgradient', 'stop', 'style', 'image', 'filter', + 'feblend', 'fecolormatrix', 'fecomponenttransfer', 'fecomposite', 'feconvolvematrix', 'fediffuselighting', 'fedisplacementmap', 'fedistantlight', + 'fedropshadow', 'feflood', 'fefunca', 'fefuncb', 'fefuncg', 'fefuncr', 'fegaussianblur', 'femerge', 'femergenode', 'femorphology', 'feoffset', + 'fepointlight', 'fespecularlighting', 'fespotlight', 'fetile', 'feturbulence', + 'rdf', 'work', 'agent', 'type', 'date', 'format', 'creator', 'description', 'license', +]); +/** Markup a drawing may hold besides elements: comments, CDATA, the XML declaration, and a DOCTYPE that names no document but W3C's SVG DTD. */ +const SVG_MARKUP = /^<(?:!--|!\[CDATA\[|\?xml\s|!DOCTYPE\s+svg(?:\s+PUBLIC\s+(["'])-\/\/W3C\/\/DTD SVG [\w .]+\/\/EN\1\s+(["'])https?:\/\/www\.w3\.org\/[\w.\/-]+\.dtd\2)?\s*>)/i; +/** + * What a drawing never holds: a named entity beyond the five XML ones, a backslash (a CSS escape such as @\69mport), a CSS + * import, a url() to another document, a CSS expression or binding. + */ +export const SVG_REFUSED = [/&(?!(?:amp|lt|gt|quot|apos);)/, /\\/, /@import/i, /url\((?!\s*#|['"]\s*#)/i, /expression\s*\(/i, /-moz-binding|\bbehavior\s*:/i]; +/** A drawing's CSS, a style element's text or a style attribute's value, holds declarations only: no @ rule and no function but url(#id). */ +const SVG_STYLE_OPEN = /<(?:[A-Za-z_][\w.-]*:)?style\b/gi; +const SVG_STYLE_CLOSE = /^<\/(?:[A-Za-z_][\w.-]*:)?style\s*>/i; +const SVG_CSS_REF = /url\(\s*(['"]?)#[\w.:-]*\1\s*\)/gi; +const SVG_CSS_REFUSED = /[@(]/; +/** An href or src value a drawing may hold: a same-document reference, or a raster image inline. */ +const SVG_HREF = /^\s*(?:#|data:image\/(?:png|jpe?g|gif|webp);base64,[A-Za-z0-9+/=\s]*$)/i; + +/** + * Whether an SVG's whole text is a drawing (GST-911): every element in SVG_ELEMENTS, no event handler, every href or src a + * same-document reference or an inline raster image, CSS of declarations only, and nothing SVG_REFUSED names, read after character references are + * decoded. Every name= in the text is read, so a stray quote cannot hide a later attribute inside its value; an unquoted value + * stops at the next =, which keeps the scan linear. A style element's CSS is walked with indexOf to its closing tag, dropping + * comments and reading CDATA as CSS, so neither can hide a closing tag; any other markup inside it, or a style element that never + * closes, refuses. Anything else, an SVG that can run code included, stays code. + */ +export function svgDrawing(text: string): boolean { + const t = decodeCharRefs(text); + if (SVG_REFUSED.some(r => r.test(t)) || svgActive(t)) return false; + const cssDrawing = (css: string) => !SVG_CSS_REFUSED.test(css.replace(SVG_CSS_REF, '')); + for (let at = 0; ;) { + SVG_STYLE_OPEN.lastIndex = at; + const open = SVG_STYLE_OPEN.exec(t); + if (!open) break; + let i = t.indexOf('>', open.index) + 1; + if (!i) return false; + let css = ''; + for (;;) { + const lt = t.indexOf('<', i); + if (lt === -1) return false; + css += t.slice(i, lt); + if (t.startsWith('', lt + 4) + 3; + if (i === 2) return false; + } else if (t.startsWith('', lt + 9); + if (end === -1) return false; + css += t.slice(lt + 9, end); + i = end + 3; + } else if (SVG_STYLE_CLOSE.test(t.slice(lt, lt + 256))) { + at = lt + 1; + break; + } else return false; + } + if (!cssDrawing(css)) return false; + } + for (let at = t.indexOf('<'); at !== -1; at = t.indexOf('<', at + 1)) { + if (SVG_MARKUP.test(t.slice(at, at + 512))) continue; + const tag = /^<\/?(?:[A-Za-z_][\w.-]*:)?([A-Za-z_][\w.-]*)/.exec(t.slice(at, at + 256)); + if (!tag || !SVG_ELEMENTS.has(tag[1]!.toLowerCase())) return false; + } + for (const m of t.matchAll(/(?=]+)))/g)) { + const local = m[1]!.slice(m[1]!.lastIndexOf(':') + 1).toLowerCase(); + const value = m[2] ?? m[3] ?? m[4] ?? ''; + if (local.startsWith('on') || ((local === 'href' || local === 'src') && !SVG_HREF.test(value)) || (local === 'style' && !cssDrawing(value))) return false; + } + return true; +} + /** A hash of the constants and patterns that decide a depth, not of this file's bytes: a comment edit keeps it, a threshold edit changes it. */ export const DEPTH_RULES = createHash('sha256').update(JSON.stringify({ - RISK_RULES, SMALL_DIFF_LINES, RED_TEAM_LINES, LARGE_DIFF_LINES, FULL_PASS_CAP, LIGHT_FIXED, BATCH_SCOPED, SCOPED_RANGE_LINES, ROLE_EFFORT, - patterns: Object.entries({ GATE_PATH, AGENT_PROSE, TEST_PATH, RELEASE_METADATA, DOC_PATH: [DOC_PATH], DATA_PATH: [DATA_PATH], DATA_CONFIG }).map(([k, v]) => [k, v.map(r => `/${r.source}/${r.flags}`)]), + RISK_RULES, SMALL_DIFF_LINES, RED_TEAM_LINES, LARGE_DIFF_LINES, FULL_PASS_CAP, LIGHT_FIXED, BATCH_SCOPED, SCOPED_RANGE_LINES, ROLE_EFFORT, SVG_ELEMENTS: [...SVG_ELEMENTS], + patterns: Object.entries({ GATE_PATH, AGENT_PROSE, TEST_PATH, RELEASE_METADATA, DOC_PATH: [DOC_PATH], DATA_PATH: [DATA_PATH], DATA_CONFIG, SVG_MARKUP: [SVG_MARKUP], SVG_REFUSED, SVG_HREF: [SVG_HREF], SVG_STYLE_OPEN: [SVG_STYLE_OPEN], SVG_STYLE_CLOSE: [SVG_STYLE_CLOSE], SVG_CSS_REF: [SVG_CSS_REF], SVG_CSS_REFUSED: [SVG_CSS_REFUSED] }).map(([k, v]) => [k, v.map(r => `/${r.source}/${r.flags}`)]), })).digest('hex').slice(0, 16); /** @@ -97,6 +176,11 @@ export function dataCandidate(p: string, root?: string | readonly string[]): boo return DATA_PATH.test(p) && classifyPath(p, root) === 'code' && ![...AGENT_PROSE, ...GATE_PATH, ...DATA_CONFIG].some(r => r.test(p)); } +/** An SVG the data rule may move to docs once the collector reads its whole text as a drawing (ClassInput.drawings); never a data file another file is named through. */ +export function svgCandidate(p: string, root?: string | readonly string[]): boolean { + return SVG_PATH.test(p) && classifyPath(p, root) === 'code' && ![...AGENT_PROSE, ...GATE_PATH, ...DATA_CONFIG].some(r => r.test(p)); +} + /** Whether a range needs the gate scan: any path that is not docs by the path-only rule. The data rule never skips it. */ export function gateScanNeeded(paths: readonly string[], root?: string | readonly string[]): boolean { return paths.some(p => classifyPath(p, root) !== 'docs'); @@ -133,6 +217,8 @@ export interface ClassInput { consumed?: readonly string[] | 'all'; /** Paths the diff creates (untracked files included): their added lines are the whole file. */ created?: readonly string[]; + /** The SVG candidates (svgCandidate) whose whole text the collector read as a drawing (svgDrawing); any other SVG stays code. */ + drawings?: readonly string[]; /** Added and deleted lines per path (lib/review-collect.ts): the risk classes read them; without them no line signal applies. */ added?: Record; removed?: Record; @@ -141,7 +227,7 @@ export interface ClassResult { class: DiffClass; reasons: string[]; errors: Dept const sanitize = (s: string) => s.replace(/[\0-\x1f]+/g, ' ').slice(0, 200); const isOutput = (i: ClassInput, p: string) => !!i.registry?.outputs.length && matchesAny(i.registry.outputs, p); -const unconsumedData = (i: ClassInput, p: string) => Array.isArray(i.consumed) && !i.consumed.includes(p) && dataCandidate(p, i.root); +const unconsumedData = (i: ClassInput, p: string) => Array.isArray(i.consumed) && !i.consumed.includes(p) && (dataCandidate(p, i.root) || (!!i.drawings?.includes(p) && svgCandidate(p, i.root))); const pathClass = (i: ClassInput, p: string): PathClass => ((i.stampOnly ?? []).includes(p) || unconsumedData(i, p) ? 'docs' : classifyPath(p, i.root)); /** The diff's class, raised to gate by any gate signal; `errors` lists every input that fails the depth closed; `runtimeLines` counts code-class lines outside the base registry's outputs. */ diff --git a/lib/review-risk.ts b/lib/review-risk.ts index e4f22b195..b6c5b92f9 100644 --- a/lib/review-risk.ts +++ b/lib/review-risk.ts @@ -7,7 +7,8 @@ * (a redaction, sanitizer, consent or trust module; a shell, eval or raw HTML sink; outbound URL handling; archive * extraction and path traversal; any line naming the child_process module); data is user data (a path naming personal or * stored data, or a line that drops or bulk-deletes it). Path and identifier signals are token-bounded: costume never - * names cost, and a CLI's usage text never names money. + * names cost, and a CLI's usage text never names money. A changed SVG line that can run code (a script or + * foreignObject element, an event handler, a javascript: URL; character references decoded) is security (GST-911). */ import { createHash } from 'node:crypto'; @@ -54,9 +55,19 @@ export const SECURITY_LINE = [ /\b(?:includes|startsWith|endsWith|indexOf)\s*\(\s*['"`]\.\.[/\\]{0,2}['"`]|[!=]==?\s*['"`]\.\.['"`]|['"`]\.\.['"`]\s*[!=]==?|\\\.\\\./, /(? s.replace(/&#(?:[xX]0*([0-9a-fA-F]+)|0*(\d+));?/g, (_, hex: string | undefined, dec: string | undefined) => { + const digits = hex ?? dec!; + const c = digits.length > 8 ? Infinity : hex !== undefined ? parseInt(hex, 16) : Number(dec); + return c <= 0x10ffff ? String.fromCodePoint(c) : ''; +}); +export const svgActive = (text: string) => { const d = decodeCharRefs(text); const squeezed = d.replace(/[\s\0-\x1f]+/g, ''); return SVG_ACTIVE.some(r => r.test(d) || r.test(squeezed)); }; const DATA_PATH_TOKENS = ['pii', 'gdpr', 'ccpa', 'privacy', 'personal', 'account', 'accounts', 'customer', 'customers', 'cookie', 'cookies', 'persistence', 'storage', 'database']; export const DATA_LINE = [ /\bDELETE\s+FROM\b/, /\b(?:drop\s+(?:table|column|database|schema)|truncate\s+table)\b/i, /\bdelete\s+from\s+[\w."`]+\s*(?:where\b|;|$|["'`)])/i, @@ -70,7 +81,7 @@ const namesMoney = (t: readonly string[], words: readonly string[]) => t.some(x export const RISK_RULES = createHash('sha256').update(JSON.stringify({ RISK_CLASSES, MONEY_PATH_TOKENS, AUTH_PATH_TOKENS, AUTH_PATH_PREFIXES, KEY_NAME, SECURITY_PATH_TOKENS, DATA_PATH_TOKENS, MONEY_LINE_TOKENS, MONEY_PAIRS, - patterns: Object.entries({ MONEY_LINE, SECRET_LINE, SECRET_PATH: [SECRET_PATH], SECURITY_LINE, DATA_LINE }).map(([k, v]) => [k, v.map(r => '/' + r.source + '/' + r.flags)]), + patterns: Object.entries({ MONEY_LINE, SECRET_LINE, SECRET_PATH: [SECRET_PATH], SECURITY_LINE, DATA_LINE, SVG_PATH: [SVG_PATH], SVG_ACTIVE }).map(([k, v]) => [k, v.map(r => '/' + r.source + '/' + r.flags)]), })).digest('hex').slice(0, 16); export const tokens = (p: string) => p.replace(/([a-z0-9])([A-Z])/g, '$1 $2').toLowerCase().split(/[^a-z0-9]+/).filter(Boolean); @@ -95,6 +106,7 @@ export function matchRisk(i: RiskInput): Map { if (namesMoney(t, MONEY_PATH_TOKENS)) hit('money', 'money path: ' + p); if (t.some(x => SECURITY_PATH_TOKENS.includes(x))) hit('security', 'security path: ' + p); if (t.some(x => DATA_PATH_TOKENS.includes(x))) hit('data', 'data path: ' + p); + if (SVG_PATH.test(p) && [i.added?.[p], i.removed?.[p]].some(l => l && svgActive(l.join('\n')))) hit('security', 'active SVG lines: ' + p); for (const line of [...(i.added?.[p] ?? []), ...(i.removed?.[p] ?? [])]) { if (!why.has('money') && (MONEY_LINE.some(r => r.test(line)) || namesMoney(tokens(line), MONEY_LINE_TOKENS))) hit('money', 'money line: ' + p); if (!why.has('secret') && SECRET_LINE.some(r => r.test(line))) hit('secret', 'secret line: ' + p); diff --git a/lib/ship-preflight.ts b/lib/ship-preflight.ts index 519ee5475..405a4eda8 100644 --- a/lib/ship-preflight.ts +++ b/lib/ship-preflight.ts @@ -117,7 +117,7 @@ function merge(cwd: string, base: string, r: Report): boolean { let runtime = 'unknown'; try { const j = JSON.parse(plan.stdout); - const c = classifyDiff({ paths: j.paths, perPath: j.per_path, stampOnly: j.stamp_only, registry: j.registry, consumed: j.consumed, root: cwd, gateTagged: 0 }); + const c = classifyDiff({ paths: j.paths, perPath: j.per_path, stampOnly: j.stamp_only, registry: j.registry, consumed: j.consumed, drawings: j.drawings, root: cwd, gateTagged: 0 }); runtime = String(c.runtimeLines ?? 'unknown'); } catch { /* unknown */ } r.add(`RUNTIME_LINES: ${runtime} (gstack-ship-budget check --runtime-lines at the Step 9 and 12 boundaries)`); diff --git a/lib/ship-receipt-depth.ts b/lib/ship-receipt-depth.ts index ad8933cfb..4452321e9 100644 --- a/lib/ship-receipt-depth.ts +++ b/lib/ship-receipt-depth.ts @@ -40,6 +40,9 @@ function git(root: string, args: string[]): { ok: boolean; out: string } { return { ok: r.status === 0, out: (r.stdout ?? '').trim() }; } +/** A base as a commit; origin/ reads refs/remotes/origin/ by its full name, so a tag named origin/main cannot stand in for it (GST-884). */ +const baseCommit = (root: string, ref: string) => git(root, ['rev-parse', '--verify', '--quiet', `${/^origin\//.test(ref) ? `refs/remotes/${ref}` : ref}^{commit}`]); + /** gstack-diff-scope on the committed range `base`...`head` (`--head`): the same signals at write, carry and the queue. */ function scopeOfRange(root: string, binDir: string, base: string, head: string): Record { const sr = runScriptSync(path.join(binDir, 'gstack-diff-scope'), [base, '--head', head], { cwd: root, encoding: 'utf8', timeout: 60_000, maxBuffer: 16 * 1024 * 1024 }); @@ -58,7 +61,7 @@ function scopeOfRange(root: string, binDir: string, base: string, head: string): */ export function receiptDepthAtHead(repoRoot: string, base: string, binDir: string, head: string): ReceiptDepth { const branch = base.replace(/^origin\//, ''); - const baseSha = git(repoRoot, ['rev-parse', '--verify', '--quiet', `origin/${branch}^{commit}`]); + const baseSha = baseCommit(repoRoot, `origin/${branch}`); const baseRev = baseSha.ok ? baseSha.out : branch; const c = collectCommit(repoRoot, baseRev, head); const scope = scopeOfRange(repoRoot, binDir, baseRev, head); @@ -156,7 +159,7 @@ export interface DepthVerdict { ok: boolean; line: string; message?: string } /** The head's committed range against the base, read once for the depth verdict and the targeted line. */ interface HeadInputs { baseSha: { ok: boolean; out: string }; c: ReturnType; scope: Record } function headInputs(root: string, baseRef: string, head: string): HeadInputs { - const baseSha = git(root, ['rev-parse', '--verify', '--quiet', `${baseRef}^{commit}`]); + const baseSha = baseCommit(root, baseRef); const baseRev = baseSha.ok ? baseSha.out : baseRef; return { baseSha, c: collectCommit(root, baseRev, head), scope: scopeOfRange(root, BIN_DIR, baseRev, head) }; } diff --git a/package.json b/package.json index 7f8863c51..c5271f2da 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.92.75", + "version": "1.92.76", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/retro/SKILL.md b/retro/SKILL.md index 0b71c2dfd..85b3ba8df 100644 --- a/retro/SKILL.md +++ b/retro/SKILL.md @@ -18,8 +18,8 @@ gbrain: context_queries: - id: prior-retros kind: filesystem - # /retro writes .context/retros/*.json (repo-local; see Step 13). - glob: ".context/retros/*.json" + # /retro writes its history here, shared by every worktree (Step 13). + glob: "{gstack_state_root}/projects/{repo_slug}/retros/*.json" sort: mtime_desc limit: 5 render_as: "## Prior retros for this project" @@ -504,13 +504,17 @@ End the section with: `N markers, M with no trigger.` If none: `No shortcut debt ### Step 12: Load History & Compare -Before saving the new snapshot, check for prior retro history: +Load history, shared across worktrees (``: session-reminder date): ```bash -setopt +o nomatch 2>/dev/null || true # zsh compat -ls -t .context/retros/*.json 2>/dev/null +_RM="$HOME/.claude/skills/gstack/bin/gstack-retro-metrics" +[ -x "$_RM" ] || _RM=".claude/skills/gstack/bin/gstack-retro-metrics" +today="" +"$_RM" --history --next "$today" ``` +`RETRO_PRIOR:` lists snapshots, newest first (skip `{}` stubs). On `RETRO_HISTORY_ERROR:` show its fix, skip history and Step 13. No `RETRO_FILE:` otherwise: stale helper; suggest `/gstack-upgrade`. + **If prior retros exist:** Load the most recent one with the same `window` using the Read tool; if none matches, disclose that and skip historical deltas. Calculate deltas for available key metrics and include a **Trends vs Last Retro** section (in `compare` mode use the freshly computed prior period instead): ``` Last Now Delta @@ -530,20 +534,7 @@ Deep sessions: 3 → 5 ↑2 After computing all metrics (including streak) and loading any prior history for comparison, draft the tweetable summary and the "3 Things to Improve" items using the format in Step 14, then save a JSON snapshot. The Step 14 narrative must reuse this exact summary and these exact items. `streak_days` is the live **team** streak from Step 11 (0 when broken); put the personal streak in `user_streak_days`. -```bash -mkdir -p .context/retros -``` - -Determine the next unused sequence number for today (substitute the session-reminder date for ``): -```bash -setopt +o nomatch 2>/dev/null || true # zsh compat -today="" -next=1 -while [ -e ".context/retros/${today}-${next}.json" ]; do next=$((next + 1)); done -# Save as .context/retros/${today}-${next}.json -``` - -Use the Write tool to save the JSON file with this schema: +Read Step 12's reserved `RETRO_FILE`, then Write it with this schema: ```json { "date": "2026-03-08", @@ -945,7 +936,7 @@ When the user runs `/retro compare` (or `/retro compare 14d`): - Never compare teammates against each other negatively. Each person's section stands on its own. - Size the report to the window and the team: every section carries specific evidence, and a section with nothing to say is cut, not padded - Use markdown tables and code blocks for data, prose for narrative -- Output the narrative directly to the conversation; the only report file is the JSON snapshot (`.context/retros/` for repo retros, `$GSTACK_STATE_ROOT/retros/` for global) +- Output the narrative directly to the conversation; the only report file is the JSON snapshot (`$GSTACK_STATE_ROOT/projects//retros/` for repo retros, `$GSTACK_STATE_ROOT/retros/` for global) ## Important Rules @@ -957,4 +948,4 @@ When the user runs `/retro compare` (or `/retro compare 14d`): - Treat merge commits as PR boundaries - Do not read CLAUDE.md or unrelated docs — this skill is self-contained; the CHANGELOG and optional inputs explicitly named above are exceptions - On first run (no prior retros), skip saved-history comparisons gracefully; explicit `compare` mode still computes its prior window -- **Global mode:** Does NOT require being inside a git repo. Saves snapshots to `~/.gstack/retros/` (not `.context/retros/`). Gracefully skip AI tools that aren't installed. Only compare against prior global retros with the same window value. If streak hits 365d cap, display as "365+ days". +- **Global mode:** Does NOT require being inside a git repo. Saves snapshots to `~/.gstack/retros/` (not the per-project history). Gracefully skip AI tools that aren't installed. Only compare against prior global retros with the same window value. If streak hits 365d cap, display as "365+ days". diff --git a/retro/SKILL.md.tmpl b/retro/SKILL.md.tmpl index 32dd21b7a..00a7d8961 100644 --- a/retro/SKILL.md.tmpl +++ b/retro/SKILL.md.tmpl @@ -22,8 +22,8 @@ gbrain: context_queries: - id: prior-retros kind: filesystem - # /retro writes .context/retros/*.json (repo-local; see Step 13). - glob: ".context/retros/*.json" + # /retro writes its history here, shared by every worktree (Step 13). + glob: "{gstack_state_root}/projects/{repo_slug}/retros/*.json" sort: mtime_desc limit: 5 render_as: "## Prior retros for this project" @@ -349,13 +349,17 @@ End the section with: `N markers, M with no trigger.` If none: `No shortcut debt ### Step 12: Load History & Compare -Before saving the new snapshot, check for prior retro history: +Load history, shared across worktrees (``: session-reminder date): ```bash -setopt +o nomatch 2>/dev/null || true # zsh compat -ls -t .context/retros/*.json 2>/dev/null +_RM="{{RUNTIME_ROOT}}/bin/gstack-retro-metrics" +[ -x "$_RM" ] || _RM=".claude/skills/gstack/bin/gstack-retro-metrics" +today="" +"$_RM" --history --next "$today" ``` +`RETRO_PRIOR:` lists snapshots, newest first (skip `{}` stubs). On `RETRO_HISTORY_ERROR:` show its fix, skip history and Step 13. No `RETRO_FILE:` otherwise: stale helper; suggest `/gstack-upgrade`. + **If prior retros exist:** Load the most recent one with the same `window` using the Read tool; if none matches, disclose that and skip historical deltas. Calculate deltas for available key metrics and include a **Trends vs Last Retro** section (in `compare` mode use the freshly computed prior period instead): ``` Last Now Delta @@ -375,20 +379,7 @@ Deep sessions: 3 → 5 ↑2 After computing all metrics (including streak) and loading any prior history for comparison, draft the tweetable summary and the "3 Things to Improve" items using the format in Step 14, then save a JSON snapshot. The Step 14 narrative must reuse this exact summary and these exact items. `streak_days` is the live **team** streak from Step 11 (0 when broken); put the personal streak in `user_streak_days`. -```bash -mkdir -p .context/retros -``` - -Determine the next unused sequence number for today (substitute the session-reminder date for ``): -```bash -setopt +o nomatch 2>/dev/null || true # zsh compat -today="" -next=1 -while [ -e ".context/retros/${today}-${next}.json" ]; do next=$((next + 1)); done -# Save as .context/retros/${today}-${next}.json -``` - -Use the Write tool to save the JSON file with this schema: +Read Step 12's reserved `RETRO_FILE`, then Write it with this schema: ```json { "date": "2026-03-08", @@ -766,7 +757,7 @@ When the user runs `/retro compare` (or `/retro compare 14d`): - Never compare teammates against each other negatively. Each person's section stands on its own. - Size the report to the window and the team: every section carries specific evidence, and a section with nothing to say is cut, not padded - Use markdown tables and code blocks for data, prose for narrative -- Output the narrative directly to the conversation; the only report file is the JSON snapshot (`.context/retros/` for repo retros, `$GSTACK_STATE_ROOT/retros/` for global) +- Output the narrative directly to the conversation; the only report file is the JSON snapshot (`$GSTACK_STATE_ROOT/projects//retros/` for repo retros, `$GSTACK_STATE_ROOT/retros/` for global) ## Important Rules @@ -778,4 +769,4 @@ When the user runs `/retro compare` (or `/retro compare 14d`): - Treat merge commits as PR boundaries - Do not read CLAUDE.md or unrelated docs — this skill is self-contained; the CHANGELOG and optional inputs explicitly named above are exceptions - On first run (no prior retros), skip saved-history comparisons gracefully; explicit `compare` mode still computes its prior window -- **Global mode:** Does NOT require being inside a git repo. Saves snapshots to `~/.gstack/retros/` (not `.context/retros/`). Gracefully skip AI tools that aren't installed. Only compare against prior global retros with the same window value. If streak hits 365d cap, display as "365+ days". +- **Global mode:** Does NOT require being inside a git repo. Saves snapshots to `~/.gstack/retros/` (not the per-project history). Gracefully skip AI tools that aren't installed. Only compare against prior global retros with the same window value. If streak hits 365d cap, display as "365+ days". diff --git a/test/diff-scope.test.ts b/test/diff-scope.test.ts index e33469c1a..dc2f75835 100644 --- a/test/diff-scope.test.ts +++ b/test/diff-scope.test.ts @@ -9,6 +9,7 @@ import { mkdtempSync, writeFileSync, mkdirSync, rmSync, readFileSync } from 'fs' import { join } from 'path'; import { tmpdir } from 'os'; import { spawnSync } from 'child_process'; +import { resolveBase } from '../lib/gate-diff/git'; const SCRIPT = join(import.meta.dir, '..', 'bin', 'gstack-diff-scope'); @@ -330,6 +331,70 @@ describe('exit-code contract (#2526)', () => { }); }); +describe('a bare base name reads origin/ when it exists (#3650)', () => { + function behindBase(base: string) { + const dir = createRepo(['README2.md']); + const git = (...args: string[]) => spawnSync('git', args, { cwd: dir, stdio: 'pipe', timeout: 5000 }).stdout.toString().trim(); + const seed = git('rev-parse', 'main'); + git('checkout', '-q', '-B', base, 'main'); + writeFileSync(join(dir, 'service.py'), 'print(1)\n'); + git('add', '.'); + git('commit', '-qm', 'upstream backend change'); + git('update-ref', `refs/remotes/origin/${base}`, 'HEAD'); + git('checkout', '-q', 'feature/test'); + git('rebase', '-q', `origin/${base}`); + git('branch', '-f', base, seed); + expect(git('diff', '--name-only', `${base}...HEAD`).split('\n').sort()).toEqual(['README2.md', 'service.py']); + return { dir, git }; + } + const scopeAt = (dir: string, base: string) => Object.fromEntries(spawnSync('bash', [SCRIPT, base], { cwd: dir, stdio: 'pipe', timeout: 5000 }).stdout.toString().trim().split('\n').map(l => l.split('='))); + + test('a local main behind origin/main does not widen the scope to upstream commits', () => { + const { dir } = behindBase('main'); + const scope = runScope(dir); + expect([scope.SCOPE_BACKEND, scope.SCOPE_DOCS]).toEqual(['false', 'true']); + }); + + test('a slash-named base reads its remote-tracking twin too, and HEAD or a sha stays as given', () => { + const { dir, git } = behindBase('release/x'); + expect(scopeAt(dir, 'release/x').SCOPE_BACKEND).toBe('false'); + git('update-ref', 'refs/remotes/origin/HEAD', git('rev-parse', 'release/x')); + expect(scopeAt(dir, 'HEAD').SCOPE_DOCS).toBe('false'); + git('update-ref', `refs/remotes/origin/${git('rev-parse', 'release/x')}`, 'HEAD'); + expect(scopeAt(dir, git('rev-parse', 'release/x')).SCOPE_BACKEND).toBe('true'); + }); + + test('a colliding tag, a fork branch over an explicit remote base, or an unrelated twin never replaces the base (#3650 review)', () => { + const { dir, git } = behindBase('main'); + const upstreamCommit = git('rev-parse', 'origin/main'); + git('tag', 'origin/main', 'HEAD'); + expect([scopeAt(dir, 'main').SCOPE_BACKEND, scopeAt(dir, 'main').SCOPE_DOCS]).toEqual(['false', 'true']); + git('remote', 'add', 'upstream', 'file:///nonexistent-upstream'); + git('update-ref', 'refs/remotes/upstream/main', git('rev-parse', 'main')); + git('update-ref', 'refs/remotes/origin/upstream/main', 'HEAD'); + expect(scopeAt(dir, 'upstream/main').SCOPE_BACKEND).toBe('true'); + git('tag', '-d', 'origin/main'); + git('checkout', '-q', '--orphan', 'island'); + git('commit', '-q', '--allow-empty', '-m', 'unrelated'); + git('update-ref', 'refs/remotes/origin/main', 'HEAD'); + git('checkout', '-q', 'feature/test'); + expect(scopeAt(dir, 'main').SCOPE_BACKEND).toBe('true'); + expect(upstreamCommit).not.toBe(''); + }); + + test('with origin/main unrelated, a related local main comes before a related other remote, the base resolveBase picks', () => { + const { dir, git } = behindBase('main'); + git('remote', 'add', 'upstream', 'file:///nonexistent-upstream'); + git('update-ref', 'refs/remotes/upstream/main', git('rev-parse', 'origin/main')); + git('checkout', '-q', '--orphan', 'island'); + git('commit', '-q', '--allow-empty', '-m', 'unrelated'); + git('update-ref', 'refs/remotes/origin/main', 'HEAD'); + git('checkout', '-q', 'feature/test'); + expect(resolveBase('main', dir, 'HEAD')?.sha).toBe(git('rev-parse', 'main')); + expect(scopeAt(dir, 'main').SCOPE_BACKEND).toBe('true'); + }); +}); + describe('uncommitted work is visible (#2299)', () => { test('uncommitted change on a branch with no commits sets the scope', () => { const dir = mkdtempSync(join(tmpdir(), 'diff-scope-dirty-')); diff --git a/test/gate-diff-cli.test.ts b/test/gate-diff-cli.test.ts index 3a8545e09..315bfccda 100644 --- a/test/gate-diff-cli.test.ts +++ b/test/gate-diff-cli.test.ts @@ -153,6 +153,59 @@ describe('gstack-gate-diff arguments and exit contract', () => { // fetches from it; fails_when=remoteNames or resolveBase hard-code origin, or noBaseFix falls back to the literal 'origin' // while a real remote exists; why_new=every existing fixture has either an origin remote or none; seam=none; // no_claim=nothing about the order when origin and another remote both carry the branch. + // Value: protects=a bare base name scans from the same merge base /ship merges against (origin/) when the local + // branch is related but behind (#3650); fails_when=resolveBase prefers any related local branch, so a stale local main + // widens the scan to commits already upstream; why_new=the stale-main test above covers only unrelated history; + // seam=none; no_claim=nothing about a local base that is ahead of its remote-tracking twin. + test('a related local main that is behind origin/main yields to origin/main (#3650)', () => { + const src = repo({ fixture: 'rh1-skip', commit: true }); + const seed = gitIn(src, ['rev-parse', 'main']).trim(); + gitIn(src, ['checkout', '-q', 'main']); + fs.mkdirSync(path.join(src, '.github', 'workflows'), { recursive: true }); + fs.writeFileSync(path.join(src, '.github', 'workflows', 'ci.yml'), 'on: push\njobs: {}\n'); + gitIn(src, ['add', '-A']); + gitIn(src, ['commit', '-q', '-m', 'upstream ci change']); + gitIn(src, ['checkout', '-q', 'feature']); + gitIn(src, ['rebase', '-q', 'main']); + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-gate-diff-behind-')); + repos.push(dir); + gitIn(dir, ['clone', '-q', '--branch', 'feature', `file://${src}`, '.']); + gitIn(dir, ['branch', '-f', 'main', seed]); + const bare = run(dir, ['main']); + const remote = run(dir, ['origin/main']); + expect([bare.status, remote.status]).toEqual([0, 0]); + expect(summary(bare.stdout).candidate).toBe(summary(remote.stdout).candidate); + expect(bare.stdout).toBe(remote.stdout); + expect(bare.stderr).toContain('gstack-gate-diff: local main is not origin/main; base is origin/main'); + }); + + // Value: protects=a ref a contributor controls (a fork's origin/upstream/main branch, a tag named origin/main, another + // remote's main) never replaces the base the caller named or a related local base, so it cannot hide a gate edit; + // fails_when=resolveBase rewrites remote-qualified names, resolves the twin by its short name, or lets any remote beat a + // related local base; why_new=the cases above have one remote and no colliding refs; seam=none; no_claim=nothing about + // a remote's HEAD symref. + test('a fork branch, a colliding tag or another remote never replaces the base (#3650 review)', () => { + const src = repo({ fixture: 'rh1-skip', commit: true }); + const seed = gitIn(src, ['rev-parse', 'main']).trim(); + const clone = (remote: string) => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-gate-diff-refs-')); + repos.push(dir); + gitIn(dir, ['clone', '-q', '-o', remote, '--branch', 'feature', `file://${src}`, '.']); + return dir; + }; + const named = clone('upstream'); + gitIn(named, ['remote', 'add', 'origin', 'file:///nonexistent-fork']); + gitIn(named, ['update-ref', 'refs/remotes/origin/upstream/main', 'HEAD']); + expect(summary(run(named, ['upstream/main']).stdout).tagged).toBe('RH-1:1'); + const tagged = clone('origin'); + gitIn(tagged, ['tag', 'origin/main', 'HEAD']); + expect(summary(run(tagged, ['main']).stdout).tagged).toBe('RH-1:1'); + const fork = clone('zfork'); + gitIn(fork, ['branch', '-f', 'main', seed]); + gitIn(fork, ['update-ref', 'refs/remotes/zfork/main', 'HEAD']); + expect(summary(run(fork, ['main']).stdout).tagged).toBe('RH-1:1'); + }); + test('a clone whose only remote is upstream resolves the base to upstream/main, and the fix line fetches from upstream', () => { const src = repo({ fixture: 'rh1-skip', commit: true }); const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-gate-diff-upstream-')); diff --git a/test/regression-3749-retro-history-worktree.test.ts b/test/regression-3749-retro-history-worktree.test.ts new file mode 100644 index 000000000..af3daa419 --- /dev/null +++ b/test/regression-3749-retro-history-worktree.test.ts @@ -0,0 +1,155 @@ +/** + * Regression for #3749: /retro kept its history in .context/retros/ inside the + * working tree, so a run in a linked worktree (Conductor, Claude Code) found + * none of the main checkout's snapshots, reported "First retro recorded", and + * wrote its own snapshot where `git worktree remove` deletes it. History now + * lives in the state root, keyed by project slug (shared by every worktree of + * a repository), and an existing .context/retros/ is migrated on first use. + * + * Runs the real Step 12 bash block (history and the Step 13 snapshot path) from the generated + * retro/SKILL.md with the gstack install pointed at this checkout. + */ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { spawnSync } from 'child_process'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; + +const ROOT = path.resolve(import.meta.dir, '..'); +const SKILL = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8'); +const TMPL = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md.tmpl'), 'utf-8'); + +function bashBlockAfter(heading: string, contains: string): string { + const start = SKILL.indexOf(heading); + if (start < 0) throw new Error(`retro/SKILL.md: heading not found: ${heading}`); + const end = SKILL.indexOf('\n### ', start + heading.length); + const re = /```bash\n([\s\S]*?)```/g; + re.lastIndex = start; + for (let m = re.exec(SKILL); m && m.index < end; m = re.exec(SKILL)) { + if (m[1].includes(contains)) { + return m[1].replaceAll('$HOME/.claude/skills/gstack', ROOT).replaceAll('~/.claude/skills/gstack', ROOT); + } + } + throw new Error(`retro/SKILL.md: no bash block containing ${contains} under ${heading}`); +} + +const LOAD = bashBlockAfter('### Step 12: Load History & Compare', 'retro').replace('', '2026-11-01'); + +let tmp: string; +let home: string; + +/** Git Bash on Windows prints the state root as given plus `/projects/...`; compare with forward slashes. */ +const fwd = (p: string) => p.replaceAll('\\', '/'); + +function sh(script: string, cwd: string) { + const env: Record = {}; + for (const [k, v] of Object.entries(process.env)) if (v !== undefined) env[k] = v; + for (const k of ['GSTACK_STATE_ROOT', 'GSTACK_STATE_DIR', 'GSTACK_PROJECT_SLUG', 'CLAUDE_PLUGIN_DATA', 'CLAUDE_PROJECT_DIR']) delete env[k]; + const r = spawnSync('bash', ['-c', script], { cwd, env: { ...env, HOME: tmp, GSTACK_HOME: home }, encoding: 'utf-8', timeout: 30_000 }); + return { stdout: fwd(r.stdout ?? ''), stderr: r.stderr ?? '', status: r.status ?? -1 }; +} + +function git(cwd: string, ...args: string[]) { + const r = spawnSync('git', args, { cwd, encoding: 'utf-8', timeout: 30_000 }); + if (r.status !== 0) throw new Error(`git ${args.join(' ')}: ${r.stderr}`); +} + +function snapshot(dir: string, name: string, body: object): string { + fs.mkdirSync(dir, { recursive: true }); + const file = path.join(dir, name); + fs.writeFileSync(file, JSON.stringify(body)); + return file; +} + +/** A main checkout with one commit (fixture-local identity) and a linked worktree under it. */ +function layout() { + const main = path.join(tmp, 'widget'); + fs.mkdirSync(main); + git(main, 'init', '-q', '-b', 'main'); + git(main, 'remote', 'add', 'origin', 'https://github.com/acme/widget.git'); + git(main, 'config', 'user.name', 'Fixture'); + git(main, 'config', 'user.email', 'fixture@example.com'); + git(main, 'commit', '-q', '--allow-empty', '-m', 'init'); + git(main, 'worktree', 'add', '-q', '-b', 'demo', path.join(main, '.claude', 'worktrees', 'demo')); + const wt = fs.realpathSync(path.join(main, '.claude', 'worktrees', 'demo')); + const slug = sh(`'${ROOT}/bin/gstack-slug' --get SLUG`, main).stdout.trim(); + expect(slug).not.toBe(''); + return { main, wt, history: path.join(home, 'projects', slug, 'retros') }; +} + +beforeEach(() => { + tmp = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'retro-3749-'))); + home = path.join(tmp, '.gstack'); +}); + +afterEach(() => { + fs.rmSync(tmp, { recursive: true, force: true }); +}); + +describe('#3749 retro history is shared across linked worktrees', () => { + test('Step 12 in a linked worktree finds the main checkout\'s earlier snapshot', () => { + const { main, wt, history } = layout(); + snapshot(path.join(main, '.context', 'retros'), '2026-10-25-1.json', { date: '2026-10-25', window: '7d' }); + + const r = sh(LOAD, wt); + expect(r.status).toBe(0); + expect(r.stdout).toContain(fwd(path.join(history, '2026-10-25-1.json'))); + expect(fs.existsSync(path.join(history, '2026-10-25-1.json'))).toBe(true); + expect(fs.existsSync(path.join(main, '.context', 'retros'))).toBe(false); + }); + + test('the snapshot path from a linked worktree is outside the worktree, and the main checkout sees it', () => { + const { main, wt, history } = layout(); + const save = sh(LOAD, wt); + expect(save.status).toBe(0); + const file = save.stdout.match(/^RETRO_FILE: (.+)$/m)?.[1]; + expect(file).toBe(fwd(path.join(history, '2026-11-01-1.json'))); + expect(file!.startsWith(fwd(wt))).toBe(false); + fs.writeFileSync(file!, JSON.stringify({ date: '2026-11-01', window: '7d' })); + + git(main, 'worktree', 'remove', '--force', wt); + const fromMain = sh(LOAD, main).stdout; + expect(fromMain).toContain(file!); + expect(fromMain).toContain(`RETRO_FILE: ${fwd(path.join(history, '2026-11-01-2.json'))}`); + }); + + test('migration keeps every snapshot: duplicates collapse, same-name differing snapshots get the next number', () => { + const { main, wt, history } = layout(); + for (const tree of [main, wt]) snapshot(path.join(tree, '.context', 'retros'), '2026-10-18-1.json', { date: '2026-10-18', window: '7d' }); + snapshot(path.join(main, '.context', 'retros'), '2026-10-25-1.json', { date: '2026-10-25', from: 'main' }); + snapshot(path.join(wt, '.context', 'retros'), '2026-10-25-1.json', { date: '2026-10-25', from: 'worktree' }); + + const r = sh(LOAD, wt); + expect(r.status).toBe(0); + expect(fs.readdirSync(history).sort()).toEqual(['2026-10-18-1.json', '2026-10-25-1.json', '2026-10-25-2.json', '2026-11-01-1.json']); + const froms = ['2026-10-25-1.json', '2026-10-25-2.json'].map((f) => JSON.parse(fs.readFileSync(path.join(history, f), 'utf-8')).from).sort(); + expect(froms).toEqual(['main', 'worktree']); + for (const tree of [main, wt]) expect(fs.existsSync(path.join(tree, '.context', 'retros'))).toBe(false); + }); + + test('a .context/retros symlinked to the shared history is left alone, so migration cannot delete it', () => { + const { wt, history } = layout(); + const kept = snapshot(history, '2026-10-25-1.json', { date: '2026-10-25', window: '7d' }); + fs.mkdirSync(path.join(wt, '.context'), { recursive: true }); + fs.symlinkSync(history, path.join(wt, '.context', 'retros')); + + const r = sh(LOAD, wt); + expect(r.status).toBe(0); + expect(fs.readFileSync(kept, 'utf-8')).toContain('2026-10-25'); + expect(r.stdout).toContain(`RETRO_PRIOR: ${fwd(kept)}`); + }); + + test('RETRO_FILE is reserved, so two runs in different worktrees never get the same path', () => { + const { main, wt, history } = layout(); + const first = sh(LOAD, wt).stdout.match(/^RETRO_FILE: (.+)$/m)?.[1]; + const second = sh(LOAD, main).stdout.match(/^RETRO_FILE: (.+)$/m)?.[1]; + expect(first).toBe(fwd(path.join(history, '2026-11-01-1.json'))); + expect(second).toBe(fwd(path.join(history, '2026-11-01-2.json'))); + expect(fs.readFileSync(first!, 'utf-8').trim()).toBe('{}'); + }); + + test('the gbrain prior-retros query reads the same state-root history', () => { + expect(TMPL).toContain('glob: "{gstack_state_root}/projects/{repo_slug}/retros/*.json"'); + expect(TMPL).not.toContain('glob: ".context/retros/*.json"'); + }); +}); diff --git a/test/review-packets-removed.test.ts b/test/review-packets-removed.test.ts index c8ab8f706..1ae42964e 100644 --- a/test/review-packets-removed.test.ts +++ b/test/review-packets-removed.test.ts @@ -57,7 +57,10 @@ const NO_LOSS_REPLAYS = { 'L': "GSTA-167 variant G Arm L: live no-loss replay of one batched reader (testing and security plus scoped checklists) and the Red Team on its findings, 2 passes, scored L∘G1 against G1 on the paired rule (part 2)", 'GST-257': "GST-257 PR 1 dispatch-superset no-loss check (test/review-targeted.test.ts): on every fixture diff the branch dispatches main's reviewers at 8be7f21 unchanged plus only the targeted reader, and over the last 100 non-release commits on main it lost none; adds the targeted role and brief to the G1 and L bytes, whose existing reviewer text is unchanged", 'GST-850': "GST-850 dispatch-superset no-loss check: on the 26 GSTA-167 no-loss units the branch plan (wider security and money classes, write-up text in the batched diff) drops no reviewer role main dispatches, so no saved blocker can be lost, and over the last 100 main commits no commit with a gate or risk class drops depth (test/review-risk-routing.test.ts, test/review-risk-linear.test.ts); reviewer text is unchanged", + 'GST-911-redos': "GST-911 ReDoS fix verdict-identity no-loss check (wave review-e7b551ebd865): SVG_REFUSED's url() entry, rewritten to run in linear time, refuses exactly the strings #3838's pattern refused (test/review-risk-redos.test.ts: 60,000 seeded strings and the hand-picked edges, and every SVG blob in the repository's history, with zero differences); SVG_MARKUP now refuses a DOCTYPE naming any document but W3C's SVG DTD (test/review-planner-misreads.test.ts), which only keeps more SVGs code and so drops no reviewer; GST-911's no-loss replay holds unchanged; reviewer text is unchanged", + 'GST-911': "GST-911 no-loss replay (GSTA-249/13): the planner over the last 100 first-parent main commits to f5cb6ec1e, origin/main's code against this branch, gate scan fixed at 0, changes no commit's depth, class, risk classes, targeted classes or SCOPE_AUTH; the three misreads (a drawing-only SVG, a session-named data file, a bare base) are proven on fixtures (test/review-planner-misreads.test.ts), where every SVG that can run script stays code with the security class; reviewer text is unchanged", 'R51': "R51 risk-tier routing no-loss re-score (GSTA-167): the saved panel findings (349 records over 24 of the 26 no-loss units) re-scored against the branch's plan lose no blocker whose only finders are roles it drops, and every gate, security, user-data and money fixture keeps main's reviewers plus the full panel (test/review-targeted.test.ts, test/review-risk-routing.test.ts); reviewer text is unchanged", + 'GST-898': "GST-898 verdict-identity no-loss check: the linear rm rule, and the rmSync rule (which also raises security on any call with more than 500 characters before its closing parenthesis), give the same verdict as R51's on every changed line of the last 100 main commits (69,023 lines, a563f47 back 100 first-parent commits) and on 60,000 seeded lines per rule (test/review-risk-redos.test.ts), and no commit changes depth, class or risk class, so every diff dispatches main's reviewers unchanged; reviewer text is unchanged", } as const; type Replay = keyof typeof NO_LOSS_REPLAYS; const REVIEWER_BRIEFS: Record = { @@ -79,8 +82,8 @@ const REVIEWER_BRIEFS: Record = { 'review/specialists/testing.md': { sha: 'd6cc2bef7be562387d0c0a22fefe646dff9c295f1d34d8453a6cb496584150e4', replay: '#3638' }, 'review/specialists/targeted.md': { sha: '7f92a0d0a8319c3c0e076fb3b1a4689c45120b2be2c14dfc7d4cb03aafa9d6a5', replay: 'GST-257' }, 'scripts/resolvers/fix-check.ts': { sha: '712bf5b0c439e96bac32f19cbe25394dabb4c94aa66600100bfde9b100e15a08', replay: '#3638' }, - 'lib/review-depth.ts': { sha: '839205a409014fb2cca6988a01aa04c98c495d8d24bd94895a86bf7794627338', replay: 'GST-850' }, - 'lib/review-risk.ts': { sha: '18f0330ddfda0032eb93516b0eb941fa4244e0a68b028e19c436c74609093b03', replay: 'GST-850' }, + 'lib/review-depth.ts': { sha: '166dd7b1148a272b345dd9d2ceccd14348cc59c52ef9c88109436f6f61cf6b69', replay: 'GST-911-redos' }, + 'lib/review-risk.ts': { sha: '126eff0f1de6982e03dc24d8017c67e6807c91dfa518b18d2db3aa910aac911f', replay: 'GST-911' }, }; describe('reviewer briefs stay at their no-loss-tested bytes (GSTA-167)', () => { diff --git a/test/review-plan-origin-base.test.ts b/test/review-plan-origin-base.test.ts new file mode 100644 index 000000000..cb5f27bce --- /dev/null +++ b/test/review-plan-origin-base.test.ts @@ -0,0 +1,95 @@ +/** + * gstack-review-plan and gstack-gate-diff diff against origin/ (GST-883): with a local that lags + * origin/, review-plan handed the raw name to gstack-gate-diff, which merged against the stale local ref and + * scanned upstream commits the branch only inherited, so a code diff planned as gate class at full depth. + */ +import { afterEach, describe, expect, test } from 'bun:test'; +import { spawnSync } from 'node:child_process'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; + +const PLAN = path.join(import.meta.dir, '..', 'bin', 'gstack-review-plan'); +const GATE_DIFF = path.join(import.meta.dir, '..', 'bin', 'gstack-gate-diff'); +const FIXTURES = path.join(import.meta.dir, 'fixtures', 'gate-diff'); +const dirs: string[] = []; +afterEach(() => { while (dirs.length) fs.rmSync(dirs.pop()!, { recursive: true, force: true }); }); + +function fixture() { + const root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'rp-origin-'))); + dirs.push(root); + const work = path.join(root, 'work'); + const env = { ...process.env, GSTACK_STATE_ROOT: path.join(root, 'state'), GIT_AUTHOR_NAME: 't', GIT_AUTHOR_EMAIL: 't@t', GIT_COMMITTER_NAME: 't', GIT_COMMITTER_EMAIL: 't@t', GIT_CONFIG_GLOBAL: path.join(root, 'gitconfig') }; + fs.writeFileSync(path.join(root, 'gitconfig'), '[commit]\n\tgpgsign = false\n'); + const sh = (cwd: string, c: string) => { const r = spawnSync('bash', ['-c', c], { cwd, env, encoding: 'utf8', timeout: 60_000 }); if (r.status !== 0) throw new Error(`${c}: ${r.stderr}`); return r.stdout.trim(); }; + sh(root, 'git init -q --bare -b main origin.git && git clone -q origin.git work 2>/dev/null'); + sh(work, `git apply '${FIXTURES}/rh1-skip.base.patch' && git add -A && git commit -qm base && git push -q origin main`); + sh(root, `git clone -q origin.git other && cd other && git apply '${FIXTURES}/rh1-skip.patch' && git commit -qam upstream && git push -q origin main`); + sh(work, "git fetch -q origin && git checkout -qb feat origin/main && echo 'export const v = 1;' > src/v.ts && git add -A && git commit -qm feat"); + return { work, env, sh }; +} + +const planOf = (work: string, env: NodeJS.ProcessEnv) => { + const r = spawnSync(PLAN, ['--base', 'main', '--json'], { cwd: work, env, encoding: 'utf8', timeout: 180_000 }); + expect(r.status, r.stdout + r.stderr).toBe(0); + const p = JSON.parse(r.stdout) as { class: string; depth: string; gate_scan: unknown }; + return { class: p.class, depth: p.depth, gate_scan: p.gate_scan }; +}; +const gateDiff = (work: string, env: NodeJS.ProcessEnv, base: string) => { + const r = spawnSync(GATE_DIFF, [base], { cwd: work, env, encoding: 'utf8', timeout: 120_000 }); + expect(r.status, r.stdout + r.stderr).toBe(0); + return { tagged: /tagged=\{([^}]*)\}/.exec(r.stdout)?.[1], candidate: /candidate=(\S+)/.exec(r.stdout)?.[1], stderr: r.stderr }; +}; + +describe('review-plan and gate-diff read the origin merge base (GST-883)', () => { + // Value: protects=a stale local never turns inherited upstream commits into gate hunks that force full depth; + // fails_when=gstack-gate-diff merges against the local while origin/ is ahead, or review-plan hands it a base the plan's own diff did not use; + // why_new=GEVA-117 paid ~15-20M tokens of full-depth readers for a gate hunk already on origin/main; seam=bin/gstack-review-plan, bin/gstack-gate-diff; + // no_claim=which gate rules fire + test('local main behind origin/main plans the same class, depth and gate-tagged count as a fresh main', () => { + const { work, env, sh } = fixture(); + expect(sh(work, 'git merge-base main HEAD')).not.toBe(sh(work, 'git merge-base origin/main HEAD')); + const stale = { plan: planOf(work, env), tags: gateDiff(work, env, 'main').tagged }; + sh(work, 'git branch -f main origin/main'); + const fresh = { plan: planOf(work, env), tags: gateDiff(work, env, 'main').tagged }; + expect(fresh).toEqual({ plan: { class: 'code', depth: 'light', gate_scan: expect.anything() }, tags: '' }); + expect(stale).toEqual(fresh); + }, 240_000); + + // Value: protects=a tag cannot stand in for the base branch, so it cannot move the merge base to HEAD and hide committed edits; + // fails_when=resolveBase or collectLive read `origin/main` or `main` by short name while a tag of that name exists; + // why_new=outside-voice finding on the whole diff (shadow tag reproduced); seam=bin/gstack-review-plan, bin/gstack-gate-diff + for (const [tag, branch] of [['origin/main', 'refs/remotes/origin/main'], ['main', 'refs/heads/main']] as const) { + test(`a tag named ${tag} at HEAD does not shadow ${branch}`, () => { + const { work, env, sh } = fixture(); + if (branch === 'refs/heads/main') sh(work, 'git remote remove origin'); + const before = { plan: planOf(work, env), candidate: gateDiff(work, env, 'main').candidate }; + sh(work, `git tag '${tag}' HEAD`); + const after = { plan: planOf(work, env), candidate: gateDiff(work, env, 'main').candidate }; + expect(after).toEqual(before); + expect(after.candidate?.startsWith(sh(work, `git merge-base ${branch} HEAD`).slice(0, 12))).toBe(true); + }, 240_000); + } + + // Value: protects=without a remote-tracking twin the local base still works and the fallback is named on stderr; + // fails_when=the origin preference turns a remote-less branch into no_base, or falls back silently; why_new=the fallback is new; seam=bin/gstack-gate-diff + test('no origin/: gate-diff falls back to the local ref and says so', () => { + const { work, env, sh } = fixture(); + sh(work, 'git branch trunk origin/main'); + const r = gateDiff(work, env, 'trunk'); + expect(r.tagged).toBe(''); + expect(r.stderr).toContain('gstack-gate-diff: no remote-tracking branch for trunk; base is the local trunk, which may lag its remote'); + }, 240_000); + + // Value: protects=the fallback note names why the local ref stands, and a remote-less repository stays quiet; + // fails_when=an unrelated origin/ is reported as missing, or a repository with no remote prints a lag note; why_new=review finding on the fallback; seam=bin/gstack-gate-diff + test('an unrelated origin/ is named in the note; no remote at all uses the local base quietly', () => { + const { work, env, sh } = fixture(); + sh(work, 'git branch -f main origin/main && git update-ref refs/remotes/origin/main "$(git commit-tree "HEAD^{tree}" -m orphan)"'); + expect(gateDiff(work, env, 'main').stderr).toContain('gstack-gate-diff: origin/main shares no history with HEAD; base is the local main, which may lag its remote'); + sh(work, 'git remote remove origin'); + const none = gateDiff(work, env, 'main'); + expect(none.tagged).toBe(''); + expect(none.stderr).not.toContain('base is'); + }, 240_000); +}); diff --git a/test/review-planner-misreads.test.ts b/test/review-planner-misreads.test.ts new file mode 100644 index 000000000..0c34036a0 --- /dev/null +++ b/test/review-planner-misreads.test.ts @@ -0,0 +1,291 @@ +/** + * Three /ship planner misreads from GST-911 (gstack-evals#132, a results write-up) and GST-884: + * a plotting-script SVG counted as runtime code, a data file named for sessions read as auth code, and + * gstack-diff-scope (plus the receipt depth) resolving a bare base name, so a stale local main or a tag + * named main or origin/main moved the diff. Every real signal stays: an SVG that can run code is + * security, session code is auth, and a credential reference is a secret wherever it is. + */ +import { afterEach, describe, expect, test } from 'bun:test'; +import { spawnSync } from 'node:child_process'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { collectCommit, collectLive } from '../lib/review-collect'; +import { depthOfCommitInput, svgDrawing } from '../lib/review-depth'; +import { targetedClasses } from '../lib/review-targeted'; +import { receiptDepthAtHead, receiptDepthVerdict } from '../lib/ship-receipt-depth'; + +const BIN = path.join(import.meta.dir, '..', 'bin'); +const SCOPE = path.join(BIN, 'gstack-diff-scope'); +const dirs: string[] = []; +afterEach(() => { while (dirs.length) fs.rmSync(dirs.pop()!, { recursive: true, force: true }); }); + +function fixture() { + const root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'planner-misreads-'))); + dirs.push(root); + const work = path.join(root, 'work'); + const env = { ...process.env, GSTACK_STATE_ROOT: path.join(root, 'state'), GIT_AUTHOR_NAME: 't', GIT_AUTHOR_EMAIL: 't@t', GIT_COMMITTER_NAME: 't', GIT_COMMITTER_EMAIL: 't@t', GIT_CONFIG_GLOBAL: path.join(root, 'gitconfig'), GIT_CONFIG_NOSYSTEM: '1' }; + fs.writeFileSync(path.join(root, 'gitconfig'), '[commit]\n\tgpgsign = false\n[init]\n\tdefaultBranch = main\n'); + const sh = (cwd: string, c: string) => { const r = spawnSync('bash', ['-c', c], { cwd, env, encoding: 'utf8', timeout: 60_000 }); if (r.status !== 0) throw new Error(`${c}: ${r.stderr}`); return r.stdout.trim(); }; + sh(root, 'git init -q --bare origin.git && git clone -q origin.git work 2>/dev/null'); + const write = (files: Record) => { for (const [p, text] of Object.entries(files)) { const f = path.join(work, p); fs.mkdirSync(path.dirname(f), { recursive: true }); fs.writeFileSync(f, text); } }; + const commit = (files: Record) => { write(files); sh(work, 'git add -A && git commit -qm change'); return sh(work, 'git rev-parse HEAD'); }; + write({ 'README.md': '# evals\n', 'src/run.ts': 'export const run = () => 1;\n' }); + sh(work, 'git add -A && git commit -qm base && git push -q origin main && git checkout -qb feat'); + const base = sh(work, 'git rev-parse HEAD'); + const scope = (args: string[]) => { + const r = spawnSync('bash', [SCOPE, ...args], { cwd: work, env, encoding: 'utf8', timeout: 30_000 }); + return { status: r.status, out: r.stdout, vars: Object.fromEntries(r.stdout.split('\n').filter(l => /^SCOPE_\w+=/.test(l)).map(l => l.split('=') as [string, string])) }; + }; + return { root, work, env, sh, write, commit, base, scope }; +} + +/** A matplotlib-style chart: XML declaration, DOCTYPE, RDF metadata, CSS, markers by internal reference, a clip path. */ +const chart = (points: number, extra = '') => [ + '', + '', + '', + ' ', + ' ', + ' ', + ' ', + ' 2026-10-31T12:00:00', + ' image/svg+xml', + ' ', + ' ', + ' Matplotlib v3.9.2, https://matplotlib.org/', + ' ', + ' ', + ' ', + ' ', + ' ', + ' ', + ' ', + ' ', + ' ', + ' ', + ' ', + ' ', + ' ', + ...Array.from({ length: points }, (_, i) => ` `), + ' ', + ' cost per run & pass rate', + ' ', + ...(extra ? [extra] : []), + ' ', + ' ', + ' ', + ' ', + ' ', + ' ', + '', + '', +].join('\n'); + +const plan = (fx: ReturnType, head: string) => { + const c = collectCommit(fx.work, fx.base, head); + const d = depthOfCommitInput({ ...c, gateTagged: 0 }); + return { class: d.class, depth: d.depth, runtime: d.runtimeLines, risk: d.risk }; +}; + +describe('GST-911 misread 1: a chart SVG is data, an SVG that can run code is security', () => { + // Value: protects=a results write-up whose chart is an SVG plans like the same chart as a PNG (docs, light, no runtime lines); + // fails_when=an SVG with only drawing elements, internal references and CSS counts as runtime code; seam=collectCommit + depthOfCommitInput, collectLive + test('a plotting-script SVG no code names plans as docs, light, runtime 0, like the PNG', () => { + const fx = fixture(); + const svg = plan(fx, fx.commit({ 'results/2026-10-31/chart.svg': chart(600), 'results/2026-10-31/README.md': '# Results\n\n![chart](chart.svg)\n' })); + expect(svg).toEqual({ class: 'docs', depth: 'light', runtime: 0, risk: [] }); + fx.sh(fx.work, 'git rm -q results/2026-10-31/chart.svg && git commit -qm drop'); + const png = plan(fx, fx.commit({ 'results/2026-10-31/chart.png': '\x89PNG\r\n\x1a\n\0\0\0\rIHDR' })); + expect(png).toEqual({ class: 'docs', depth: 'light', runtime: 0, risk: [] }); + fx.write({ 'results/live/chart.svg': chart(40) }); + const live = collectLive(fx.work, 'main'); + expect(depthOfCommitInput({ ...live, gateTagged: 0 }).runtimeLines).toBe(0); + }, 60_000); + + // Value: protects=an SVG that can execute script stays code and gets the security class (full depth), however it spells the script; + // fails_when=a script element, a namespaced script, an event handler, a javascript: URL (plain, entity-encoded or zero-padded) or a + // foreignObject plans as docs or loses the security class, or a handler or URL split across lines loses it; seam=collectCommit + depthOfCommitInput + test.each([ + ['a script element', ' '], + ['a namespaced script', ' window.x = 42'], + ['an event handler', ' '], + ['a javascript: URL', ' x'], + ['an entity-encoded javascript: URL', ' x'], + ['a foreignObject', '
x
'], + ['a zero-padded javascript: URL', ' x'], + ['an event handler split across lines', ' '], + ['a javascript: URL split across lines', ' x'], + ])('%s stays code with the security class', (_name, line) => { + const fx = fixture(); + expect(plan(fx, fx.commit({ 'results/chart.svg': chart(5, line) }))).toMatchObject({ class: 'code', depth: 'full', risk: ['security'] }); + }, 60_000); + + // Value: protects=only an SVG made of drawing elements is data: anything that loads, links or declares outside that set stays code; + // fails_when=an external image or stylesheet, an entity declaration, an external DTD, an animation or a CSS url() to another document reads as docs, + // including through a CSS escape or a zero-padded character reference (review pass 1 findings) or behind a stray quote that + // pairs with a later one (review pass 2 finding), or any CSS beyond declarations and url(#id) (Step 11 finding: image-set()); + // seam=collectCommit + depthOfCommitInput + test.each([ + ['an external image', ' '], + ['a CSS import', ' '], + ['a CSS url() to another document', ' '], + ['an animation', ' '], + ['a stylesheet processing instruction', ''], + ['an entity declaration', ' ]>'], + ['an external DTD', ''], + ['an external DTD behind the SVG public identifier', ''], + ['a CSS-escaped import', ' '], + ['a CSS-escaped url()', ' '], + ['a zero-padded character reference import', ' '], + ['a zero-padded hex character reference url()', ' '], + ['an external image after a stray quote in text', " z=''"], + ['an external use after a stray quote in a comment', " "], + ['an external xlink:href after a stray quote in a desc', " z=''"], + ['an external image after a stray href in text', " href='#'"], + ['a CSS image-set() in a style element', ' '], + ['a CSS image-set() in a style attribute', " "], + ['a CSS image() with a string', ' '], + ['a CSS element() reference', ' '], + ['a CSS import of a bare string', ' '], + ['a style element import', ' '], + ['a style attribute cross-fade()', ' '], + ['a style element whose hides in a comment', ' -->image-set("https://x.test/p.png" 1x)}'], + ['a style element whose hides in CDATA', ' */ svg{fill:image-set("https://x.test/p.png" 1x)} ]]>'], + ])('%s keeps the SVG code', (_name, line) => { + const fx = fixture(); + expect(plan(fx, fx.commit({ 'results/chart.svg': chart(5, line) })).class).toBe('code'); + }, 60_000); + + // Value: protects=a passive edit to an SVG that already carries a script is still code, because the whole file decides, not the + // changed lines; and an SVG code names stays code like any consumed data file; fails_when=either plans as docs; seam=collectCommit + test('an edit inside a scripted SVG, and an SVG code names, stay code', () => { + const fx = fixture(); + fx.commit({ 'assets/live.svg': chart(5, ' ') }); + fx.sh(fx.work, 'git checkout -q main && git merge -q --ff-only feat && git checkout -q feat'); + const scripted = { ...fx, base: fx.sh(fx.work, 'git rev-parse HEAD') }; + expect(plan(scripted, fx.commit({ 'assets/live.svg': chart(6, ' ') })).class).toBe('code'); + const named = plan(fx, fx.commit({ 'assets/icon.svg': chart(3), 'src/icon.ts': "export const icon = new URL('./icon.svg', import.meta.url);\n" })); + expect(named.class).toBe('code'); + expect(named.runtime).toBeGreaterThan(3); + }, 60_000); + + // Value: protects=the drawing check reads a file up to the collector's 8 MB cap in linear time; fails_when=a run of unclosed + // style tags (480 KB) takes over 100 ms or reads as a drawing (review pass notes on #3838); seam=svgDrawing + test('a 480 KB run of unclosed ' + fill('') + '')).toBe(true); + expect(svgDrawing(H + '')).toBe(true); + for (const svg of [ + H + fill('') + '', + H + '', + H + '', + H + ' */ ' + fill('rect{fill:#000} ') + 'svg{fill:' + bad + '} ]]>', + H + ' -->svg{fill:' + bad + '}', + H + '', + H + "z='" + fill('abc ') + "'", + H + fill(' { + // Value: protects=a data file whose name says session never sets SCOPE_AUTH, while session code and session config still do; + // fails_when=recovered-sessions.jsonl sets SCOPE_AUTH, or session.ts, a session store, a sessions controller or config/sessions.json stop setting it; + // seam=bin/gstack-diff-scope + test('gstack-diff-scope: session data files are not auth; session code and config are', () => { + for (const [file, auth] of [ + ['results/recovered-sessions.jsonl', 'false'], ['data/sessions.csv', 'false'], ['logs/session-costs.json', 'false'], ['shots/session.PNG', 'false'], + ['src/auth/session.ts', 'true'], ['lib/session-store.ts', 'true'], ['app/controllers/sessions_controller.rb', 'true'], ['config/sessions.json', 'true'], + ['.github/session.json', 'true'], ['app/.session.json', 'true'], ['session.config.json', 'true'], ['data/oauth-tokens.json', 'true'], + ] as const) { + const fx = fixture(); + fx.commit({ [file]: '{"a": 1}\n', 'src/run.ts': 'export const run = () => 2;\n' }); + expect([file, fx.scope(['main']).vars.SCOPE_AUTH]).toEqual([file, auth]); + } + }, 120_000); + + // Value: protects=the planner adds no auth reader and no gate class for a session-named data file, and a credential in it is still a secret; + // fails_when=targetedClasses returns auth for recovered-sessions.jsonl next to a code change, classifyDiff raises it to gate through SCOPE_AUTH, + // or the secret class misses an env key reference in that file; seam=gstack-diff-scope + collectCommit + targetedClasses + depthOfCommitInput + test('the planner: no auth class and no gate for session rows; real session code is auth; a key reference is a secret', () => { + const fx = fixture(); + const rows = Array.from({ length: 20 }, (_, i) => `{"run": ${i}, "tokens": ${1000 + i}}`).join('\n') + '\n'; + const run = (files: Record) => { + const head = fx.commit(files); + const c = collectCommit(fx.work, fx.base, head); + const scope = fx.scope(['main', '--head', head]).vars; + return { targeted: targetedClasses({ ...c, scope, gateTagged: 0 }).classes, class: depthOfCommitInput({ ...c, scope, gateTagged: 0 }).class }; + }; + expect(run({ 'results/recovered-sessions.jsonl': rows, 'src/run.ts': 'export const run = () => 3;\n' })).toEqual({ targeted: [], class: 'code' }); + expect(run({ 'lib/session-store.ts': 'export const cookie = (sid: string) => `sid=${sid}; HttpOnly`;\n' })).toEqual({ targeted: ['auth'], class: 'gate' }); + expect(run({ 'results/recovered-sessions.jsonl': rows + '{"key": "${OPENAI_API_KEY}"}\n' }).targeted).toEqual(['auth', 'secret']); + }, 120_000); +}); + +describe('GST-884: gstack-diff-scope and the receipt depth read origin/ by its full ref name', () => { + function staleFixture() { + const fx = fixture(); + fx.sh(fx.root, 'git clone -q origin.git other && cd other && mkdir -p db/migrate && echo "create table t;" > db/migrate/001.sql && git add -A && git commit -qm upstream && git push -q origin main'); + fx.sh(fx.work, 'git fetch -q origin && git checkout -q -B feat origin/main && echo "a{}" > styles.css && git add -A && git commit -qm feat'); + return fx; + } + + // Value: protects=a local main behind origin/main never adds upstream files to the scope; fails_when=gstack-diff-scope diffs against the + // stale local main and reports SCOPE_MIGRATIONS for a migration the branch only inherited; seam=bin/gstack-diff-scope + test('a stale local main scopes the same as a fresh one', () => { + const fx = staleFixture(); + expect(fx.sh(fx.work, 'git merge-base main HEAD')).not.toBe(fx.sh(fx.work, 'git merge-base origin/main HEAD')); + const stale = fx.scope(['main']); + fx.sh(fx.work, 'git branch -f main origin/main'); + const fresh = fx.scope(['main']); + expect(fresh.vars).toMatchObject({ SCOPE_FRONTEND: 'true', SCOPE_MIGRATIONS: 'false' }); + expect(stale.out).toBe(fresh.out); + }, 60_000); + + // Value: protects=a tag named main or origin/main cannot stand in for the branch and hide the change; fails_when=gstack-diff-scope resolves + // the base by its short name while a tag of that name exists at HEAD (an empty scope); seam=bin/gstack-diff-scope + for (const [tag, arg] of [['main', 'main'], ['origin/main', 'main'], ['origin/main', 'origin/main']] as const) { + test(`a tag named ${tag} at HEAD does not shadow the base ${arg}`, () => { + const fx = staleFixture(); + fx.sh(fx.work, 'git branch -f main origin/main'); + const before = fx.scope([arg]); + fx.sh(fx.work, `git tag '${tag}' HEAD`); + const after = fx.scope([arg]); + expect(before.vars.SCOPE_FRONTEND).toBe('true'); + expect(after.out).toBe(before.out); + const head = fx.sh(fx.work, 'git rev-parse HEAD'); + expect(fx.scope([arg, '--head', head]).out).toBe(before.out); + }, 60_000); + } + + // Value: protects=the receipt's depth and the queue's recomputation diff against refs/remotes/origin/, never a tag named origin/main; + // fails_when=receiptDepthAtHead or receiptDepthVerdict rev-parse the short name origin/; seam=lib/ship-receipt-depth.ts + test('a tag named origin/main does not move the receipt depth base', () => { + const fx = staleFixture(); + const origin = fx.sh(fx.work, 'git rev-parse refs/remotes/origin/main'); + const head = fx.sh(fx.work, 'git rev-parse HEAD'); + fx.sh(fx.work, "git tag 'origin/main' HEAD && git tag main HEAD"); + const stub = path.join(fx.root, 'stub-bin'); + fs.mkdirSync(stub); + fs.writeFileSync(path.join(stub, 'gstack-diff-scope'), '#!/bin/sh\nexit 0\n', { mode: 0o755 }); + fs.writeFileSync(path.join(stub, 'gstack-gate-diff'), '#!/bin/sh\nexit 1\n', { mode: 0o755 }); + expect(receiptDepthAtHead(fx.work, 'main', stub, head).base).toBe(origin); + const verdict = receiptDepthVerdict(fx.work, 'origin/main', { gstack_version: '1.0.0', review_depth: { depth: 'full', gate_scan: { tagged: 0, head } } } as any, head); + expect(verdict.line).toContain(`inputs=base@${origin.slice(0, 12)}`); + expect(verdict.line).toContain('class=code'); + }, 60_000); +}); diff --git a/test/review-risk-redos.test.ts b/test/review-risk-redos.test.ts new file mode 100644 index 000000000..7d40d9df3 --- /dev/null +++ b/test/review-risk-redos.test.ts @@ -0,0 +1,102 @@ +/** + * GST-898: R51's rm and rmSync rules (lib/review-risk.ts) ran in quadratic time on one hostile added line ('rm -' and + * 100,000 r's took about 3.2 s); every risk pattern finishes in linear time on its anchor followed by a long run of the + * class it repeats, and the rewritten rm rule matches exactly the lines R51's pattern matched. + */ +import { describe, expect, test } from 'bun:test'; +import { SVG_REFUSED, svgDrawing } from '../lib/review-depth'; +import { DATA_LINE, MONEY_LINE, SECRET_LINE, SECRET_PATH, SECURITY_LINE, matchRisk } from '../lib/review-risk'; + +const L = 100_000; +const ms = (f: () => unknown) => { const t = performance.now(); f(); return performance.now() - t; }; +const run = (anchor: string, unit: string, tail = '') => [JSON.stringify(anchor) + ' + ' + JSON.stringify(unit) + '*n' + (tail ? ' + ' + JSON.stringify(tail) : ''), anchor + unit.repeat(Math.ceil(L / unit.length)) + tail] as const; +const security = (line: string) => matchRisk({ codePaths: ['x.sh'], added: { 'x.sh': [line] }, removed: {}, scopeAuth: false }).has('security'); + +describe('linear time on hostile input', () => { + // Value: protects=gstack-review-plan, the targeted pass and the queue depth check never stall on one added line; + // fails_when=any risk pattern backtracks quadratically on its anchor followed by a long run of the class it repeats, + // or on a line of repeated anchors; seam=every pattern in SECURITY_LINE, DATA_LINE, SECRET_LINE, MONEY_LINE and SECRET_PATH + const hostile = [ + run('rm -', 'r'), run('rm -', 'f'), run('rm -r', 'r'), run('rm -f', 'a'), run('rm\t-', 'r'), run('rm ', ' '), run('rm -', 'r', '1'), run('rm -', 'rf', '_'), run('', 'rm -r '), + run('', 'rmSync('), run('rmSync(', ' '), run('rmSync(', 'recursive '), run('rmSync(', 'recursive: '), + run('shell', ' '), run('shell:', ' '), run('shell=', ' '), run('', '; '), run(';', ' '), run('; eval', ' '), run('eval', ' '), run('', ';eval '), + run('execSync', ' '), run('child_process.exec', ' '), run('os.system', ' '), run('new', ' '), run('new Function', ' '), + run('spawn(', ' '), run('Bun.spawn(', ' '), run('Bun.spawn([', ' '), run('innerHTML', ' '), + run('tar -', 'A'), run('tar x', 'A'), run('tar', ' '), run('redirect', ' '), run('redirect:', ' '), run('maxRedirects', ' '), + run('DELETE', ' '), run('drop', ' '), run('truncate', ' '), run('delete from', ' '), run('delete from ', 'a.'), run('', 'delete from a '), run('.deleteMany', ' '), + run('', 'a_'), run('', 'aU'), run('a', 'A'), run('spend', '_'), run('new Anthropic', ' '), run('.messages.create', ' '), + run('$', 'A_'), run('${', 'A_'), run('', '$A_'), run('process.env.', 'A_'), run('process.env[', ' '), run('os.environ[', ' '), run('os.getenv(', ' '), run('ENV[', ' '), + ]; + test.each(Object.entries({ SECURITY_LINE, DATA_LINE, SECRET_LINE, MONEY_LINE, SECRET_PATH: [SECRET_PATH] }))('every %s pattern finishes under 100 ms on each 100k-character hostile line', (_n, patterns) => { + const slow = patterns.flatMap(r => hostile.map(([label, s]) => [r.source.slice(0, 50), label, Math.round(ms(() => r.test(s)))] as const)).filter(([, , t]) => t >= 100); + expect(slow).toEqual([]); + }, 120_000); + test("matchRisk on 'rm -' and 100,000 r's (the GST-898 repro) returns in under 250 ms and raises no class", () => { + const line = 'rm -' + 'r'.repeat(100_000); + let hit = true; + expect(ms(() => { hit = security(line); })).toBeLessThan(250); + expect(hit).toBe(false); + }, 60_000); +}); + +describe('the rm rule matches what R51 matched', () => { + // Value: protects=the rewrite changes no verdict; fails_when=the linear rm pattern matches a line R51's pattern did not, or misses + // one it did; seam=SECURITY_LINE's rm pattern against R51's original source + const r51 = /\brm\s+-(?:[a-z]*r[a-z]*f|[a-z]*f[a-z]*r)\b/; + const rm = SECURITY_LINE.find(r => r.source.startsWith(String.raw`\brm\s+-`))!; + let x = 898; + const next = (n: number) => { x ^= x << 13; x ^= x >>> 17; x ^= x << 5; return (x >>> 0) % n; }; + test('the same verdict on 60,000 seeded short lines and the hand-picked edges', () => { + x = 898; + const prefixes = ['rm -', 'rm -', 'xrm -', ' rm\t-', 'rm-', 'rm -- ', '"rm -', 'brm -', 'rm -r ', '']; + const alphabet = 'rfrfrfFRa1_- .x/m'; + const lines = Array.from({ length: 60_000 }, () => prefixes[next(prefixes.length)] + Array.from({ length: next(11) }, () => alphabet[next(alphabet.length)]).join('')); + const edges = ['rm -rf /', 'rm -fr x', 'rm -Rf x', 'rm -rF x', 'rm -rvf x', 'rm -frv', 'rm -r -f x', 'rm -rf1', 'rm -rf_x', 'rm -rfx', 'rm -rf-x', 'rm -f', 'rm -r', 'rm -rrf', 'rm -ffr', 'rm -rfr', 'farm -rf x', 'rm-rf x', 'rm\t\t-rf', 'sudo rm -rf "$DIR"']; + const differ = [...lines, ...edges].filter(s => rm.test(s) !== r51.test(s)); + expect(differ).toEqual([]); + expect(lines.filter(s => r51.test(s)).length).toBeGreaterThan(1000); + expect(edges.filter(s => rm.test(s))).toEqual(['rm -rf /', 'rm -fr x', 'rm -rvf x', 'rm -rf-x', 'rm -rrf', 'rm -ffr', 'rm -rfr', 'rm\t\t-rf', 'sudo rm -rf "$DIR"']); + }); + test('the rmSync rule matches what R51 matched on 60,000 seeded short lines, and padding past its window still raises security', () => { + x = 898; + const r51Sync = /\brmSync\s*\([^)]*recursive\s*:\s*true/; + const rmSync = SECURITY_LINE.find(r => r.source.startsWith(String.raw`\brmSync`))!; + const parts = ['rmSync(', 'rmSync (', 'xrmSync(', 'recursive', ':', ' ', 'true', ')', 'a', 'recursive: true', 'rmSync', '(', '\t']; + const lines = Array.from({ length: 60_000 }, () => Array.from({ length: next(9) + 1 }, () => parts[next(parts.length)]).join('')); + expect(lines.filter(s => rmSync.test(s) !== r51Sync.test(s))).toEqual([]); + expect(lines.filter(s => r51Sync.test(s)).length).toBeGreaterThan(1000); + expect(security('fs.rmSync(dir, { recursive: true, force: true });')).toBe(true); + expect(security('fs.rmSync(' + 'a'.repeat(480) + ', { recursive: true })')).toBe(true); + expect(security('fs.rmSync(dir, /*' + 'x'.repeat(501) + '*/ { recursive: true, force: true });')).toBe(true); + expect(security('fs.rmSync(dir, /*' + 'x'.repeat(5_000_000) + '*/ { recursive: true });')).toBe(true); + expect(security('fs.rmSync(dir, { force: true });')).toBe(false); + }); +}); + +describe('svgDrawing in linear time', () => { + // Value: protects=gstack-review-plan and the commit collector behind ship-queue ready, train build and land never stall on one + // changed SVG; fails_when=a whole-text SVG pattern backtracks quadratically on a long whitespace run, or the linear url() + // refusal reads any url( differently from #3838's pattern; seam=SVG_REFUSED's url() entry and svgDrawing + const old = /url\(\s*(?!['"]?\s*#)/i; + test('a 1 MB whitespace run in url(), an attribute, a style element and an href returns in under 250 ms each', () => { + const pad = ' '.repeat(1_000_000); + const cases = [ + ['', true], ['', false], + ['', true], ['', true], + ['', true], ['' + pad + '', true], + ] as const; + const wrong = cases.map(([s, want], i) => { let got = !want; const t = ms(() => { got = svgDrawing(s); }); return [i, got === want, Math.round(t)] as const; }).filter(([, ok, t]) => !ok || t >= 250); + expect(wrong).toEqual([]); + }, 60_000); + test("the url() refusal matches #3838's pattern on 60,000 seeded short strings and the hand-picked edges", () => { + const url = SVG_REFUSED.find(r => r.source.startsWith('url'))!; + let x = 911; + const next = (n: number) => { x ^= x << 13; x ^= x >>> 17; x ^= x << 5; return (x >>> 0) % n; }; + const parts = ['url(', 'URL(', ' ', '\t', '\n', '#', "'", '"', 'a', '#a', ')', 'http:', '(', 'u']; + const lines = Array.from({ length: 60_000 }, () => Array.from({ length: next(8) + 1 }, () => parts[next(parts.length)]).join('')); + const edges = ['url(#a)', 'url( #a)', "url('#a')", "url( '#a')", 'url( "#a")', 'url(" #a")', 'url(\t#a)', 'url(x)', 'url( x)', "url('http://x')", 'url(', 'url( ', "url('')"]; + expect([...lines, ...edges].filter(s => url.test(s) !== old.test(s))).toEqual([]); + expect(lines.filter(s => old.test(s)).length).toBeGreaterThan(1000); + expect(lines.filter(s => /url\(/i.test(s) && !old.test(s)).length).toBeGreaterThan(100); + }); +}); From 4bbad24221ff519d1f02942a4f538f2b3e1d1645 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Mon, 2 Nov 2026 04:58:56 -0800 Subject: [PATCH 06/19] v1.92.77.0 wave(github): #3724 #3771 #3833 (#3846) * test-free-shards: --ci-run --jobs k sub-shards inside one CI job (k=1 default), per-file JUnit counts (GSTA-203 PR 2) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ci): ship ffmpeg in the eval image so the marathon design-motion case can run (#3687) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * shard-jobs: serial sub-shard gets what is left of the shard wall; register shard-jobs.ts in ship-measure touchfiles Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * shard-jobs: validate every sub-shard summary, bounded JUnit read, stricter --ci-verify records, unique count paths, no serial launch without wall Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * shard-jobs tests: non-hex plan id fixture; harness-version: shard-jobs FREE_ONLY entry is non-behavioral Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(test:free): exact root-bound JUnit path matching; serial timeout text names the wall the parallel phase left Adversarial pass 2 (GSTA-203 PR 2): a plain relative path now matches a planned file exactly, and an absolute or ../-prefixed path matches only when its prefix is the child's root, so an unplanned browse/test/x.test.ts is never credited to test/x.test.ts. The serial sub-shard's FREE_SUBSHARD_TIMEOUT text says how much of the shard wall the parallel phase left it. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(testing): real-home tripwire covers --jobs k>1 parallel sub-shards (docs audit docs-203-1) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ci(free): per-shard sub-shard k in the CI plan (--shard-jobs), free_jobs and windows_jobs dispatch inputs (k=1 defaults) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ci(free-tests): 10 x ubicloud-standard-8 at 4 sub-shards per shard (GSTA-203 PR 3 A/B winner); Windows stays at k=1 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (changelog names main's 20 x standard-4 baseline and the unchanged 80 vCPUs) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(testing): sub-shard default is the plan's per-shard k (docs audit docs-pr3-2) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ci(free): run the #3680/#3679 BSD-userland regression on macos-named-regressions Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * changelog: macos-named-regressions runs the BSD-userland ubi-runner regression (#3680) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: Windows k=1 applies to the nightly suite since #3806 (docs re-audit after the main merge) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- .github/docker/Dockerfile.ci | 15 +++++ .github/workflows/free-tests.yml | 35 ++++++++---- .github/workflows/windows-free-tests.yml | 21 ++++++- CHANGELOG.md | 12 ++++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- docs/TESTING_INTERNALS.md | 55 +++++++++++++------ package.json | 2 +- scripts/lib/shard-jobs.ts | 31 +++++++++-- scripts/test-free-shards.ts | 50 ++++++++++++----- test/ci-image-cli-pin.test.ts | 19 +++++++ test/free-tests-workflow-wiring.test.ts | 14 +++-- ...on-issue3680-ubicloud-bsd-userland.test.ts | 10 ++++ test/shard-jobs.test.ts | 55 +++++++++++++++++-- test/windows-native-workflows.test.ts | 36 +++++++++++- 15 files changed, 297 insertions(+), 62 deletions(-) diff --git a/.github/docker/Dockerfile.ci b/.github/docker/Dockerfile.ci index 841291cca..1ed8c693d 100644 --- a/.github/docker/Dockerfile.ci +++ b/.github/docker/Dockerfile.ci @@ -193,6 +193,21 @@ RUN for i in 1 2 3; do \ && fc-cache -f \ && rm -rf /var/lib/apt/lists/* +# ffmpeg (with ffprobe): /design-motion's checks and exports shell out to it +# (lib/motion-encode.ts), and the marathon design-motion-showcase case runs +# that flow end to end. The image had none, so every trial stopped at +# FFMPEG_MISSING (#3687). The build fails if the packaged ffmpeg ever lacks a +# filter or encoder the checks and exports use. +RUN for i in 1 2 3; do \ + apt-get update && apt-get install -y --no-install-recommends ffmpeg && break || \ + (echo "ffmpeg install retry $i/3"; sleep 10); \ + done \ + && rm -rf /var/lib/apt/lists/* \ + && command -v ffprobe \ + && filters="$(ffmpeg -hide_banner -filters)" && encoders="$(ffmpeg -hide_banner -encoders)" \ + && for f in psnr gblur palettegen paletteuse signalstats tile; do echo "$filters" | grep -qw "$f" || { echo "ERROR: ffmpeg lacks the $f filter" >&2; exit 1; }; done \ + && for e in libx264 gif; do echo "$encoders" | grep -qw "$e" || { echo "ERROR: ffmpeg lacks the $e encoder" >&2; exit 1; }; done + # Pre-install dependencies (cached layer — only rebuilds when package.json or # bun.lock changes). Copy BOTH so install is deterministic and matches local # resolution. Without bun.lock here, bun install resolved transitive deps diff --git a/.github/workflows/free-tests.yml b/.github/workflows/free-tests.yml index 966419ad3..0d3cff19e 100644 --- a/.github/workflows/free-tests.yml +++ b/.github/workflows/free-tests.yml @@ -41,13 +41,21 @@ on: description: 'Runner-sizing experiments only: shard count, 1-40 (empty keeps FREE_SUITE_SHARDS)' type: string default: '' + free_jobs: + description: 'Runner-sizing experiments only: sub-shards per CI shard (--shard-jobs), 1-8 (empty keeps FREE_SUITE_JOBS)' + type: string + default: '' -# The one source of the free-suite runner and shard count. pull_request and -# push runs never see workflow_dispatch inputs, so free-plan resolves these -# (or a dispatch override) into outputs that free-suite reads. +# The one source of the free-suite runner, shard count and sub-shards per +# shard. pull_request and push runs never see workflow_dispatch inputs, so +# free-plan resolves these (or a dispatch override) into outputs and the plan +# that free-suite reads. FREE_SUITE_JOBS is k in `--ci-plan --shard-jobs k`: +# each shard runs min(k, its parallel files) sandboxed sub-shards side by side +# (scripts/lib/shard-jobs.ts); the exclusive host-state shard stays one child. env: - FREE_SUITE_RUNNER: ubicloud-standard-4 - FREE_SUITE_SHARDS: '20' + FREE_SUITE_RUNNER: ubicloud-standard-8 + FREE_SUITE_SHARDS: '10' + FREE_SUITE_JOBS: '4' # Keyed on the PR number, not head_ref: a bare branch name carries no fork # prefix, so same-name branches from two forks would share one group and a @@ -120,10 +128,12 @@ jobs: env: RUNNER_OVERRIDE: ${{ inputs.free_runner }} SHARDS_OVERRIDE: ${{ inputs.free_shards }} + JOBS_OVERRIDE: ${{ inputs.free_jobs }} run: | set -euo pipefail runner="${RUNNER_OVERRIDE:-$FREE_SUITE_RUNNER}" shards="${SHARDS_OVERRIDE:-$FREE_SUITE_SHARDS}" + jobs="${JOBS_OVERRIDE:-$FREE_SUITE_JOBS}" case "$runner" in ubicloud-standard-2|ubicloud-standard-4|ubicloud-standard-8) ;; *) echo "::error::free_runner '$runner' is not ubicloud-standard-2, -4 or -8"; exit 1 ;; @@ -132,10 +142,14 @@ jobs: ''|*[!0-9]*) echo "::error::free_shards '$shards' is not a whole number"; exit 1 ;; esac if [ "$shards" -lt 1 ] || [ "$shards" -gt 40 ]; then echo "::error::free_shards $shards is outside 1-40"; exit 1; fi - if [ "$runner" != "$FREE_SUITE_RUNNER" ] || [ "$shards" != "$FREE_SUITE_SHARDS" ]; then - echo "- Runner-sizing experiment: $shards shards on $runner (defaults $FREE_SUITE_SHARDS on $FREE_SUITE_RUNNER); not a PR or train Free Tests result" >> "$GITHUB_STEP_SUMMARY" + case "$jobs" in + ''|*[!0-9]*) echo "::error::free_jobs '$jobs' is not a whole number"; exit 1 ;; + esac + if [ "$jobs" -lt 1 ] || [ "$jobs" -gt 8 ]; then echo "::error::free_jobs $jobs is outside 1-8"; exit 1; fi + if [ "$runner" != "$FREE_SUITE_RUNNER" ] || [ "$shards" != "$FREE_SUITE_SHARDS" ] || [ "$jobs" != "$FREE_SUITE_JOBS" ]; then + echo "- Runner-sizing experiment: $shards shards on $runner at $jobs sub-shards each (defaults $FREE_SUITE_SHARDS on $FREE_SUITE_RUNNER at $FREE_SUITE_JOBS); not a PR or train Free Tests result" >> "$GITHUB_STEP_SUMMARY" fi - matrix=$(bun run scripts/test-free-shards.ts --ci-plan "$RUNNER_TEMP/free-plan.json" --shards "$shards") + matrix=$(bun run scripts/test-free-shards.ts --ci-plan "$RUNNER_TEMP/free-plan.json" --shards "$shards" --shard-jobs "$jobs") echo "matrix=$matrix" >> "$GITHUB_OUTPUT" echo "runner=$runner" >> "$GITHUB_OUTPUT" - name: Seed growth ratchet (60 s ceiling, shrink-only allowlist) @@ -186,7 +200,8 @@ jobs: # code that compares a realpath with an unresolved path passes on Linux's # /tmp and fails here (#2982). TMPDIR is deliberately left at the runner's # default, and the job fails if that layout is not symlinked. It also sources - # the Codex probe from zsh, macOS's default shell (#3024; no paid call). + # the Codex probe from zsh, macOS's default shell (#3024; no paid call), and + # runs ubi-runner under macOS's bsdtar and one-true-awk (#3680, #3679). macos-named-regressions: needs: reuse if: ${{ !cancelled() && (needs.reuse.result != 'success' || needs.reuse.outputs.reused != 'true') }} @@ -229,7 +244,7 @@ jobs: resolved="$(realpath "$tmp")" echo "temp root: $tmp -> $resolved" test "$resolved" != "${tmp%/}" - files=(test/gstack-design-detect.test.ts browse/test/chromium-profile-isolation.test.ts test/cso-git-hardening.test.ts test/plan-count-file-permission.test.ts test/spec-quality-gate-secret-sink.test.ts test/gstack-memory-ingest.test.ts test/regression-issue3024-zsh-sourced-helpers.test.ts test/codex-probe-subcommands.test.ts browse/test/proxy-env-daemon.test.ts browse/test/proxy-direct-launch-args.test.ts) + files=(test/gstack-design-detect.test.ts browse/test/chromium-profile-isolation.test.ts test/cso-git-hardening.test.ts test/plan-count-file-permission.test.ts test/spec-quality-gate-secret-sink.test.ts test/gstack-memory-ingest.test.ts test/regression-issue3024-zsh-sourced-helpers.test.ts test/codex-probe-subcommands.test.ts browse/test/proxy-env-daemon.test.ts browse/test/proxy-direct-launch-args.test.ts test/regression-issue3680-ubicloud-bsd-userland.test.ts) for file in "${files[@]}"; do test -f "$file"; done bun test "${files[@]}" diff --git a/.github/workflows/windows-free-tests.yml b/.github/workflows/windows-free-tests.yml index 6146e927d..d94d1d6f2 100644 --- a/.github/workflows/windows-free-tests.yml +++ b/.github/workflows/windows-free-tests.yml @@ -42,6 +42,10 @@ on: description: Record per-file Windows durations (artifact) instead of running the lane type: boolean default: false + windows_jobs: + description: 'Sub-shard experiments only: sub-shards per Windows shard (--shard-jobs), 1-4 (empty keeps WINDOWS_FREE_JOBS). Never cite such a run as a PR result.' + type: string + default: '' # free-durations-record.yml calls the recording path weekly. workflow_call: inputs: @@ -49,6 +53,11 @@ on: type: boolean default: false +# k in `--ci-plan --shard-jobs k`: each Windows shard runs min(k, its parallel +# files) sandboxed sub-shards side by side (scripts/lib/shard-jobs.ts). +env: + WINDOWS_FREE_JOBS: '1' + concurrency: group: windows-free-${{ github.event.pull_request.number || github.run_id }} cancel-in-progress: true @@ -105,8 +114,18 @@ jobs: # 4-vCPU Windows runner, so the lane lands near 4 min with setup. env: GSTACK_FREE_TEST_DURATIONS: scripts/free-test-durations-windows.json + JOBS_OVERRIDE: ${{ inputs.windows_jobs }} run: | - matrix=$(bun run scripts/test-free-shards.ts --windows-only --ci-plan "$RUNNER_TEMP/windows-plan.json" --shards 6) + set -euo pipefail + jobs="${JOBS_OVERRIDE:-$WINDOWS_FREE_JOBS}" + case "$jobs" in + ''|*[!0-9]*) echo "::error::windows_jobs '$jobs' is not a whole number"; exit 1 ;; + esac + if [ "$jobs" -lt 1 ] || [ "$jobs" -gt 4 ]; then echo "::error::windows_jobs $jobs is outside 1-4"; exit 1; fi + if [ "$jobs" != "$WINDOWS_FREE_JOBS" ]; then + echo "- Sub-shard experiment: $jobs sub-shards per Windows shard (default $WINDOWS_FREE_JOBS); not a PR Windows Free Tests result" >> "$GITHUB_STEP_SUMMARY" + fi + matrix=$(bun run scripts/test-free-shards.ts --windows-only --ci-plan "$RUNNER_TEMP/windows-plan.json" --shards 6 --shard-jobs "$jobs") echo "matrix=$matrix" >> "$GITHUB_OUTPUT" - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7 with: diff --git a/CHANGELOG.md b/CHANGELOG.md index 0c95b691a..bf0437f2c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,17 @@ # Changelog +## [1.92.77.0] - 2026-11-02 + +**The weekly marathon evals can run /design-motion again: the CI eval image now ships ffmpeg (#3687).** The `design-motion-showcase` case joined the marathon tier on 2026-10-26, and its first scheduled run (Actions run 41200316994, main @ 088bfc5) failed 0/3. Every trial built and captured the page, then stopped at `FFMPEG_MISSING`, because the image had no ffmpeg and the skill rightly refuses to export before its checks run. The model did nothing wrong; the case could not pass on that image. + +### Fixed +- **`.github/docker/Dockerfile.ci`** installs Ubuntu's `ffmpeg` (6.1.1, with `ffprobe`). The build fails if that ffmpeg lacks a filter or encoder the motion checks and exports use (`psnr`, `gblur`, `palettegen`, `paletteuse`, `signalstats`, `tile`, `libx264`, `gif`). +- **`test/ci-image-cli-pin.test.ts`** fails the free suite if the image stops installing ffmpeg or stops checking it. + +The macOS `macos-named-regressions` CI job now runs `test/regression-issue3680-ubicloud-bsd-userland.test.ts`, so the ubi-runner cases for #3680 (bsdtar) and #3679 (macOS awk) run on a real Mac against `/usr/bin/tar` and `/usr/bin/awk`. Before, the file ran only on Linux CI, where bsdtar and one-true-awk aren't installed, so every case skipped and a regression in either could merge green. A new wiring test fails if the macOS job stops naming the file. + +The Linux free suite now runs on 10 `ubicloud-standard-8` machines, each running its tests as 4 sandboxed sub-shards side by side, instead of main's 20 `ubicloud-standard-4` machines (set by #3710) running one file at a time. Both use 80 vCPUs per run. In a 3-run A/B at one commit against that main setup, the slowest shard dropped from 389–450 s to 224–258 s and the Ubicloud cost per run dropped from about $0.39 to $0.32 at premium prices, with the same per-file test counts and no new failures. The CI plan now stores each shard's sub-shard count (`--ci-plan --shard-jobs k`), keeps the exclusive host-state shard at one child, and `--ci-verify` rejects a shard that ran at a different count. The nightly Windows suite stays at one child per shard, because two sub-shards made Windows file-lock cleanup failures show up in two of three runs. Dispatch-only `free_jobs` and `windows_jobs` inputs exist for future experiments. See docs/TESTING_INTERNALS.md#sub-shards-inside-one-ci-job. + ## [1.92.76.0] - 2026-11-02 **/retro now remembers its history across git worktrees, so a run in a linked worktree compares against last week instead of announcing "First retro recorded".** Retro snapshots used to live in `.context/retros/` inside the working tree. A worktree (Conductor, Claude Code) saw none of the main checkout's history, and its own snapshot was deleted along with the worktree (#3749). diff --git a/VERSION b/VERSION index 730cd3e00..463685b58 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.92.76.0 +1.92.77.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index f297fb296..ac6c1e65e 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.92.76.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.92.77.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/docs/TESTING_INTERNALS.md b/docs/TESTING_INTERNALS.md index 91c400897..441525762 100644 --- a/docs/TESTING_INTERNALS.md +++ b/docs/TESTING_INTERNALS.md @@ -195,15 +195,18 @@ fallback; unknown files get 99th-percentile pessimism, and both full-suite and the long pole. Packed shards get duration-aware walls (`max(base, predicted × 3, files × 5s)`). The legacy `--shards N --shard i` path keeps stable hash indices. Required CI uses -one duration-packed `--ci-plan`, 20 ordinary `--ci-run` shards plus a separate +one duration-packed `--ci-plan`, 10 ordinary `--ci-run` shards plus a separate exclusive-fixture shard, and a `--ci-verify` aggregate. CI shards still run on -independent machines without ordering unrelated jobs. Two workflow-level +independent machines without ordering unrelated jobs. Three workflow-level constants in `free-tests.yml` set that size for every event: `FREE_SUITE_RUNNER` -(`ubicloud-standard-4`) and `FREE_SUITE_SHARDS` (`'20'`), which `free-plan` -resolves into the shards' runner and shard count. A `workflow_dispatch` run may -override them with `free_runner` (`ubicloud-standard-2`, `-4` or `-8`) and -`free_shards` (1–40) for runner-sizing experiments only; its job summary says -so, and it is never cited as a PR or train Free Tests result. Each shard prints +(`ubicloud-standard-8`), `FREE_SUITE_SHARDS` (`'10'`) and `FREE_SUITE_JOBS` +(`'4'`, the [sub-shards](#sub-shards-inside-one-ci-job) each shard runs side by +side), which `free-plan` resolves into the shards' runner, shard count and plan. +GSTA-203 chose them by a 3-run A/B against 20 × `ubicloud-standard-4` at k = 1 +(the PR that set them has the table). A `workflow_dispatch` run may +override them with `free_runner` (`ubicloud-standard-2`, `-4` or `-8`), +`free_shards` (1–40) and `free_jobs` (1–8) for runner-sizing experiments only; +its job summary says so, and it is never cited as a PR or train Free Tests result. Each shard prints `RESOURCES: peak_used_mib=… min_free_disk_mib=… samples=…` from a memory and disk sample every 2 seconds, even when the suite fails. The public `TREE_MUTATING` map now classifies exclusive host-state fixtures; its sole @@ -1553,7 +1556,7 @@ refresh to a maintainer. `scripts/ubicloud/setup-free-suite.sh` and the Windows seed through `windows-free-tests.yml`'s `record_durations` path. The recorder stays on `ubicloud-standard-8` while the free-suite shards run on `FREE_SUITE_RUNNER` - (`ubicloud-standard-4`); both are Ubicloud standard runners. When either recording adds + (`ubicloud-standard-8`); both are Ubicloud standard runners. When either recording adds or drops a test file, it force-updates the `bot/free-duration-seeds` branch, opens or refreshes one seed PR whose body states the ratchet result, and dispatches Free Tests, Windows Smoke and Quality gate on it. Those @@ -1575,14 +1578,24 @@ k sub-shards side by side on the same machine (`scripts/lib/shard-jobs.ts`). Eac sub-shard is an ordinary free shard child: one `bun test` process, serial inside, with its own sandbox, process-group kill and log. Bun's worker `--parallel` mode is still not used (decision V3/D6 in -`scripts/test-free-shards.ts`). The default is `k = 1`, which runs today's one -child, so no workflow changes behavior until it passes `--jobs`. +`scripts/test-free-shards.ts`). Without `--jobs`, a shard runs at the plan's +per-shard k (below); a shard with no planned k runs one child (`k = 1`). -- `--jobs` applies only to `--ci-run`. It takes a positive whole number up to the - shard's file count (an empty shard takes 1); anything else stops with - `FREE_JOBS_INVALID`. `GSTACK_FREE_JOBS` sets the local full-suite process count - and never changes `--ci-run`. The exclusive host-state shard has one file, so - a workflow passes it `--jobs 1` or no `--jobs`. +- Workflows set k in the plan, per shard: `--ci-plan --shards N + --shard-jobs k` stores `jobs = min(k, the shard's files that may run in + parallel)` on each shard, and only when that is above 1. The exclusive + host-state shard holds only `TREE_MUTATING` files, so it stays one child, and + a plan made at k = 1 is byte-identical to one made without `--shard-jobs`. + `--ci-run` runs the shard at the plan's k, and `--ci-verify` rejects any + shard result that ran at a different k. `free-tests.yml` reads k from + `FREE_SUITE_JOBS` and `windows-free-tests.yml` from `WINDOWS_FREE_JOBS`; the + `free_jobs` and `windows_jobs` dispatch inputs override them for experiments + only. +- `--jobs` applies only to `--ci-run` and overrides the plan's k, for local + reproduction. It takes a positive whole number up to the shard's file count + (an empty shard takes 1); anything else stops with `FREE_JOBS_INVALID`, as + does a bad `--shard-jobs` or one without `--ci-plan`. `GSTACK_FREE_JOBS` sets + the local full-suite process count and never changes `--ci-run`. - Files are split into disjoint sub-shards packed by the duration seed. Each sub-shard has an id (`:.`) bound to its file list. Files in `TREE_MUTATING` or `SLOW_UNDER_LOAD` run after the parallel @@ -1606,6 +1619,13 @@ reproduce command. If the file passes serially, add it to `SLOW_UNDER_LOAD` in `scripts/lib/shard-jobs.ts` with that run id and a reason. Never raise the timeout. `SLOW_UNDER_LOAD` is empty today. +The nightly Windows lane stays at `WINDOWS_FREE_JOBS: '1'`. In GSTA-203's A/B, k = 2 +made the slowest Windows shard faster, but two of three runs needed the flaky +retry for file-lock cleanup errors (`EBUSY` in +`browse/test/stealth-webdriver.test.ts` and a native fixture cleanup in +`browse/test/cookie-import-native-job.test.ts`), which k = 1 never showed. Fix +those cleanups before raising it. + Every `--ci-run` result, at any k, records per-file test and `expect()` counts in `outcome.counts` (`tests`, `pass`, `fail`, `skip`, `todo`, `expects` per file, plus shard totals). They come from bun's JUnit reporter, which the runner @@ -1624,8 +1644,9 @@ test list in its last step, where each file's reason sits beside it. `test/windows-native-workflows.test.ts` keeps that list Windows-safe and fast. The curated suite below runs nightly on main and on dispatch, never on a PR. -`windows-free-tests.yml` plans on Linux (`--windows-only --ci-plan --shards 6`), -runs one strict serial shard per `windows-latest` job +`windows-free-tests.yml` plans on Linux (`--windows-only --ci-plan --shards 6 +--shard-jobs $WINDOWS_FREE_JOBS`), runs one strict serial shard per +`windows-latest` job (`bun run test:windows --ci-run`), and verifies every shard receipt on Linux (`--windows-only --ci-verify`). Packing uses Windows-measured durations from `scripts/free-test-durations-windows.json`; the weekly diff --git a/package.json b/package.json index c5271f2da..8d40c09f7 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.92.76", + "version": "1.92.77", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/scripts/lib/shard-jobs.ts b/scripts/lib/shard-jobs.ts index c96100e01..d0510fb77 100644 --- a/scripts/lib/shard-jobs.ts +++ b/scripts/lib/shard-jobs.ts @@ -45,16 +45,39 @@ export const FREE_JOBS_INVALID = 'FREE_JOBS_INVALID'; export const FREE_SUBSHARD_TIMEOUT = 'FREE_SUBSHARD_TIMEOUT'; export const FREE_SUBSHARD_REJECTED = 'FREE_SUBSHARD_REJECTED'; const DOCS = 'docs/TESTING_INTERNALS.md#sub-shards-inside-one-ci-job'; -const JOBS_FIX = `--jobs applies only to --ci-run and defaults to 1; GSTACK_FREE_JOBS sets the local full-suite process count and never changes --ci-run. See ${DOCS}`; +const JOBS_FIX = `--jobs applies only to --ci-run and defaults to the plan's per-shard k (1 when the plan has none); --shard-jobs applies only to --ci-plan; GSTACK_FREE_JOBS sets the local full-suite process count and never changes --ci-run. See ${DOCS}`; -/** `--jobs ` at parse time: digits only, at least 1. */ -export function parseJobsFlag(raw: string | undefined): number { +/** `--jobs ` or `--shard-jobs ` at parse time: digits only, at least 1. */ +export function parseJobsFlag(raw: string | undefined, flag = '--jobs'): number { if (raw === undefined || !/^\d+$/.test(raw) || Number(raw) < 1) { - throw new Error(`${FREE_JOBS_INVALID}: --jobs needs a positive whole number; got ${JSON.stringify(raw ?? '')}. ${JOBS_FIX}`); + throw new Error(`${FREE_JOBS_INVALID}: ${flag} needs a positive whole number; got ${JSON.stringify(raw ?? '')}. ${JOBS_FIX}`); } return Number(raw); } +export function rejectShardJobsWithoutCiPlan(): never { + throw new Error(`${FREE_JOBS_INVALID}: --shard-jobs needs --ci-plan. ${JOBS_FIX}`); +} + +/** + * One planned shard's k under `--ci-plan --shard-jobs k`: at most one parallel + * sub-shard per file that may run in parallel, so a shard of only serial files + * (the exclusive TREE_MUTATING shard) keeps k = 1. Undefined means k = 1, and + * the plan then carries no `jobs` key, so its id matches a plan without k. + */ +export function plannedShardJobs(files: readonly string[], shardJobs: number, isSerial: (file: string) => boolean): number | undefined { + const jobs = Math.min(shardJobs, files.filter(file => !isSerial(file)).length); + return jobs > 1 ? jobs : undefined; +} + +/** Why a plan's per-shard k cannot stand; null when absent or valid. */ +export function plannedShardJobsProblem(shard: { shard: number; files: readonly string[]; jobs?: unknown }, isSerial: (file: string) => boolean): string | null { + if (shard.jobs === undefined) return null; + const parallel = shard.files.filter(file => !isSerial(file)).length; + return Number.isInteger(shard.jobs) && (shard.jobs as number) >= 2 && (shard.jobs as number) <= parallel ? null + : `CI shard ${shard.shard} plans --jobs ${JSON.stringify(shard.jobs)} for ${parallel} parallel file(s)`; +} + export function rejectJobsWithoutCiRun(): never { throw new Error(`${FREE_JOBS_INVALID}: --jobs needs --ci-run. ${JOBS_FIX}`); } diff --git a/scripts/test-free-shards.ts b/scripts/test-free-shards.ts index deda761cf..d61ca6112 100755 --- a/scripts/test-free-shards.ts +++ b/scripts/test-free-shards.ts @@ -77,11 +77,15 @@ * bun run scripts/test-free-shards.ts --attribute-home # name real-home writers: each file alone, private HOME * bun run scripts/test-free-shards.ts --ci-plan plan.json --shards 20 * bun run scripts/test-free-shards.ts --ci-run plan.json --shard 1 --result result.json + * bun run scripts/test-free-shards.ts --ci-plan plan.json --shards 20 --shard-jobs 2 + * # the plan stores a per-shard k: min(k, the shard's files that may run in parallel), + * # kept only when above 1, so the exclusive TREE_MUTATING shard stays one child. * bun run scripts/test-free-shards.ts --ci-run plan.json --shard 1 --result result.json --jobs 4 - * # k sub-shards side by side inside one CI job (default 1 = one child, today's behavior; - * # scripts/lib/shard-jobs.ts). --jobs applies only to --ci-run: a positive whole number up - * # to the shard's file count, else FREE_JOBS_INVALID. GSTACK_FREE_JOBS sets the local - * # full-suite process count and never changes --ci-run; --jobs is its only knob. + * # k sub-shards side by side inside one CI job (default: the plan's per-shard k, else 1 = + * # one child, today's behavior; scripts/lib/shard-jobs.ts). --jobs applies only to + * # --ci-run and overrides the plan: a positive whole number up to the shard's file count, + * # else FREE_JOBS_INVALID. --ci-verify requires every result to have run at the plan's k. + * # GSTACK_FREE_JOBS sets the local full-suite process count and never changes --ci-run. * bun run scripts/test-free-shards.ts --ci-verify plan.json --results results/ */ @@ -97,8 +101,8 @@ import { attributeFreeHomeWriters, guardFreeHome, sharedFreeHome, type FreeHomeG import { appendStepSummary, ciHealthSummary, SEED_REFRESH_COMMAND, unseededWarning, windowsCurationLine } from './lib/free-ci-health'; import { curateWindowsSafe, type CurationResult } from './lib/windows-curation'; import { - assertSoloSeedSample, ciRunContext, parseJobsFlag, parseJunitCounts, readJunitReport, rejectJobsWithoutCiRun, runCiShard, SLOW_UNDER_LOAD, subShardRecordProblems, - validateJobsForShard, type ShardTestCounts, type SubShardRecord, + assertSoloSeedSample, ciRunContext, parseJobsFlag, parseJunitCounts, plannedShardJobs, plannedShardJobsProblem, readJunitReport, rejectJobsWithoutCiRun, + rejectShardJobsWithoutCiPlan, runCiShard, SLOW_UNDER_LOAD, subShardRecordProblems, validateJobsForShard, type ShardTestCounts, type SubShardRecord, } from './lib/shard-jobs'; export { curateWindowsSafe, detectWindowsFragility, KNOWN_WINDOWS_INCOMPATIBLE, type CurationResult } from './lib/windows-curation'; import { @@ -424,19 +428,27 @@ export interface FreeCiPlan { version: 1; revision: string; id: string; - shards: Array<{ shard: number; files: string[]; predictedMs: number }>; + /** `jobs` is the shard's k from --shard-jobs, present only when above 1. */ + shards: Array<{ shard: number; files: string[]; predictedMs: number; jobs?: number }>; } +/** Files that run in the trailing serial sub-shard under k > 1, never beside another child. */ +const runsSerialUnderJobs = (file: string) => file in TREE_MUTATING || file in SLOW_UNDER_LOAD; + const planDigest = (plan: Omit): string => createHash('sha256').update(JSON.stringify(plan)).digest('hex'); /** One immutable plan is shared by isolated CI machines; never repack per job. */ -export function createFreeCiPlan(files: string[], count: number, durations: Record, revision: string): FreeCiPlan { +export function createFreeCiPlan(files: string[], count: number, durations: Record, revision: string, shardJobs = 1): FreeCiPlan { const readers = files.filter(file => !(file in TREE_MUTATING)); const exclusive = files.filter(file => file in TREE_MUTATING).sort(); const packed = packShardsByDuration(readers, count, durations); - const shards = packed.shards.map((files, index) => ({ shard: index + 1, files, predictedMs: packed.predictedMs[index] })); - if (exclusive.length) shards.push({ shard: shards.length + 1, files: exclusive, predictedMs: exclusive.reduce((ms, file) => ms + (durations[file] ?? 0), 0) }); + const planned = packed.shards.map((files, index) => ({ files, predictedMs: packed.predictedMs[index] })); + if (exclusive.length) planned.push({ files: exclusive, predictedMs: exclusive.reduce((ms, file) => ms + (durations[file] ?? 0), 0) }); + const shards = planned.map(({ files, predictedMs }, index) => { + const jobs = plannedShardJobs(files, shardJobs, runsSerialUnderJobs); + return { shard: index + 1, files, predictedMs, ...(jobs ? { jobs } : {}) }; + }); const body = { version: 1 as const, revision, shards }; return { ...body, id: planDigest(body) }; } @@ -447,6 +459,8 @@ export function validateFreeCiPlan(plan: FreeCiPlan, files: string[], revision: || id !== planDigest({ version, revision: plan.revision, shards })) throw new Error('CI plan identity or revision mismatch'); if (shards.some((shard, index) => shard.shard !== index + 1 || !Array.isArray(shard.files) || !Number.isFinite(shard.predictedMs) || shard.predictedMs < 0)) throw new Error('Invalid CI shard plan'); + const jobsProblems = shards.map(shard => plannedShardJobsProblem(shard, runsSerialUnderJobs)).filter(Boolean); + if (jobsProblems.length) throw new Error(`Invalid CI shard plan: ${jobsProblems.join('; ')}`); const planned = shards.flatMap(shard => shard.files).sort(); if (new Set(planned).size !== planned.length || JSON.stringify(planned) !== JSON.stringify([...files].sort())) throw new Error('CI plan must cover every free file exactly once'); } @@ -486,6 +500,7 @@ export function verifyFreeCiResults(plan: FreeCiPlan, results: FreeCiResult[]): if (!hasCompleteCiSummary(outcome)) throw new Error('Missing or incomplete CI execution summary'); const subShardProblems = subShardRecordProblems(result); if (subShardProblems.length) throw new Error(`CI shard ${outcome.shard} sub-shards rejected: ${subShardProblems.join('; ')}`); + if ((result.jobs ?? 1) !== (shard.jobs ?? 1)) throw new Error(`CI shard ${outcome.shard} ran at --jobs ${result.jobs ?? 1}; the plan says ${shard.jobs ?? 1}`); if (outcome.status === 'passed') { if (outcome.exitCode !== 0 || outcome.failingFiles.length || outcome.unattributedFailures || result.retry) throw new Error('Inconsistent passing CI result'); } else { @@ -556,8 +571,10 @@ type CliOptions = { result: string | null; results: string | null; filesFrom: string | null; - /** Set only by --jobs (with --ci-run); absent means k = 1. */ + /** Set only by --jobs (with --ci-run); absent means the plan's per-shard k. */ jobs?: number; + /** Set only by --shard-jobs (with --ci-plan); absent means k = 1 in every shard. */ + shardJobs?: number; }; export function parseCliOptions(argv: string[]): CliOptions { @@ -573,6 +590,7 @@ export function parseCliOptions(argv: string[]): CliOptions { let quick = false; let attributeHome = false; let jobs: number | undefined; + let shardJobs: number | undefined; const paths: Record<'ciPlan' | 'ciRun' | 'ciVerify' | 'result' | 'results' | 'filesFrom', string | null> = { ciPlan: null, ciRun: null, ciVerify: null, result: null, results: null, filesFrom: null, }; @@ -597,6 +615,7 @@ export function parseCliOptions(argv: string[]): CliOptions { '--results': pathFlag('--results', 'results'), '--files-from': pathFlag('--files-from', 'filesFrom'), '--jobs': (next) => { jobs = parseJobsFlag(next()); }, + '--shard-jobs': (next) => { shardJobs = parseJobsFlag(next(), '--shard-jobs'); }, '--shards': (next) => { const value = next(); if (!value) throw new Error('Missing value for --shards'); @@ -621,8 +640,9 @@ export function parseCliOptions(argv: string[]): CliOptions { if (paths.ciRun && (shardIndex === null || !paths.result)) throw new Error('--ci-run requires --shard and --result'); if (paths.ciVerify && !paths.results) throw new Error('--ci-verify requires --results'); if (jobs !== undefined && !paths.ciRun) rejectJobsWithoutCiRun(); + if (shardJobs !== undefined && !paths.ciPlan) rejectShardJobsWithoutCiPlan(); if (quick && (recordDurations || windowsOnly || shardIndex !== null)) throw new Error('--quick cannot change recording, Windows or shard selection'); - return { dryRun, listOnly, recordDurations, windowsOnly, verbose, shardCount, shardIndex, wallTimeoutMs, wallTimeoutExplicit, quick, attributeHome, ...paths, ...(jobs === undefined ? {} : { jobs }) }; + return { dryRun, listOnly, recordDurations, windowsOnly, verbose, shardCount, shardIndex, wallTimeoutMs, wallTimeoutExplicit, quick, attributeHome, ...paths, ...(jobs === undefined ? {} : { jobs }), ...(shardJobs === undefined ? {} : { shardJobs }) }; } const describeLines = (lines: string[]) => `${lines.slice(0, 5).join(', ')}${lines.length > 5 ? ` (+${lines.length - 5} more)` : ''}`; @@ -1954,7 +1974,7 @@ async function main(): Promise { appendStepSummary(line); } warnUnseededFreeFiles(ciFiles, durations); - const plan = createFreeCiPlan(ciFiles, options.shardCount, durations, revision); + const plan = createFreeCiPlan(ciFiles, options.shardCount, durations, revision, options.shardJobs); validateFreeCiPlan(plan, ciFiles, revision); writeJson(options.ciPlan, plan); console.log(JSON.stringify({ shard: plan.shards.map(shard => shard.shard) })); @@ -1975,7 +1995,7 @@ async function main(): Promise { } const shard = plan.shards[options.shardIndex! - 1]; if (!shard || shard.shard !== options.shardIndex) throw new Error('CI shard index is outside the plan'); - const jobs = options.jobs ?? 1; + const jobs = options.jobs ?? shard.jobs ?? 1; validateJobsForShard(jobs, shard.files.length, shard.shard); const durations = loadFreeTestDurations() ?? {}; const { result, exitCode } = await runCiShard({ @@ -1986,7 +2006,7 @@ async function main(): Promise { runShard: (files, shardNumber, totalShards, runOptions) => runFreeShard(files, shardNumber, totalShards, runOptions), retry: outcomes => retryFailedFreeFiles(outcomes, plan.shards.length, options), pack: (files, count) => packShardsByDuration(files, count, durations).shards, - isSerial: file => file in TREE_MUTATING || file in SLOW_UNDER_LOAD, + isSerial: runsSerialUnderJobs, terminationRequested: isTerminationRequested, }); writeJson(options.result!, result); diff --git a/test/ci-image-cli-pin.test.ts b/test/ci-image-cli-pin.test.ts index c4a84f21a..1a94c107c 100644 --- a/test/ci-image-cli-pin.test.ts +++ b/test/ci-image-cli-pin.test.ts @@ -256,3 +256,22 @@ describe('CI Node.js installs match the image and are verified (#3354)', () => { expect(unpack).toBeGreaterThan(verify); }); }); + +// Value: protects=the eval image ships an ffmpeg that runs /design-motion's checks and exports; +// fails_when=Dockerfile.ci stops installing ffmpeg or stops failing the build on a missing filter or encoder; +// why_new=the marathon design-motion-showcase case ran 0/3 on an image without ffmpeg (#3687); seam=none; +// no_claim=reads the Dockerfile only; the image build runs the check itself +describe('ci image ffmpeg for the design-motion evals (#3687)', () => { + const lines = logicalLines(fs.readFileSync(DOCKERFILE, 'utf-8')).filter((line) => !/^\s*#/.test(line)); + + test('apt installs ffmpeg, which the marathon design-motion-showcase case needs', () => { + expect(lines.some((line) => /apt-get install\b[^&|;]*\sffmpeg\b/.test(line))).toBe(true); + }); + + test('the build fails when that ffmpeg cannot run the motion checks and exports', () => { + const verify = lines.find((line) => /\bffmpeg\b.*-filters/.test(line)) ?? ''; + for (const need of ['ffprobe', 'psnr', 'gblur', 'palettegen', 'paletteuse', 'signalstats', 'tile', 'libx264', 'gif']) { + expect(verify, `Dockerfile.ci must check ffmpeg for ${need}`).toContain(need); + } + }); +}); diff --git a/test/free-tests-workflow-wiring.test.ts b/test/free-tests-workflow-wiring.test.ts index b4426e205..0b36507d7 100644 --- a/test/free-tests-workflow-wiring.test.ts +++ b/test/free-tests-workflow-wiring.test.ts @@ -174,7 +174,8 @@ describe('free-tests workflow wiring', () => { expect(workflow.env.FREE_SUITE_SHARDS).toMatch(/^[1-9][0-9]?$/); expect(planner.outputs.runner).toBe('${{ steps.plan.outputs.runner }}'); expect(workflow.jobs['free-suite']['runs-on']).toBe('${{ needs.free-plan.outputs.runner }}'); - expect(plan.env).toEqual({ RUNNER_OVERRIDE: '${{ inputs.free_runner }}', SHARDS_OVERRIDE: '${{ inputs.free_shards }}' }); + expect(workflow.env.FREE_SUITE_JOBS).toMatch(/^[1-8]$/); + expect(plan.env).toEqual({ RUNNER_OVERRIDE: '${{ inputs.free_runner }}', SHARDS_OVERRIDE: '${{ inputs.free_shards }}', JOBS_OVERRIDE: '${{ inputs.free_jobs }}' }); expect(plan.run).not.toContain('${{'); const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'free plan ')); const bin = path.join(directory, 'bin'); @@ -185,7 +186,7 @@ describe('free-tests workflow wiring', () => { const output = path.join(directory, 'output'); fs.writeFileSync(output, ''); const result = spawnSync('bash', ['-c', plan.run], { - env: { ...process.env, ...workflow.env, RUNNER_OVERRIDE: '', SHARDS_OVERRIDE: '', ...overrides, + env: { ...process.env, ...workflow.env, RUNNER_OVERRIDE: '', SHARDS_OVERRIDE: '', JOBS_OVERRIDE: '', ...overrides, PATH: `${toShell(bin)}:${process.env.PATH}`, RUNNER_TEMP: toShell(directory), GITHUB_OUTPUT: toShell(output), GITHUB_STEP_SUMMARY: toShell(path.join(directory, 'summary')) }, encoding: 'utf8', timeout: 5000, @@ -199,12 +200,13 @@ describe('free-tests workflow wiring', () => { const unset = resolve({}); expect(unset.status).toBe(0); expect(unset.outputs.runner).toBe(workflow.env.FREE_SUITE_RUNNER); - expect(unset.args).toContain(`--shards ${workflow.env.FREE_SUITE_SHARDS}`); - const override = resolve({ RUNNER_OVERRIDE: 'ubicloud-standard-4', SHARDS_OVERRIDE: '16' }); + expect(unset.args).toContain(`--shards ${workflow.env.FREE_SUITE_SHARDS} --shard-jobs ${workflow.env.FREE_SUITE_JOBS}`); + const override = resolve({ RUNNER_OVERRIDE: 'ubicloud-standard-4', SHARDS_OVERRIDE: '16', JOBS_OVERRIDE: '3' }); expect(override.status).toBe(0); expect(override.outputs.runner).toBe('ubicloud-standard-4'); - expect(override.args).toContain('--shards 16'); - const refusals: Record[] = [{ RUNNER_OVERRIDE: 'ubuntu-24.04' }, { RUNNER_OVERRIDE: 'ubicloud-standard-8 x' }, { SHARDS_OVERRIDE: '0' }, { SHARDS_OVERRIDE: '41' }, { SHARDS_OVERRIDE: '16; true' }]; + expect(override.args).toContain('--shards 16 --shard-jobs 3'); + const refusals: Record[] = [{ RUNNER_OVERRIDE: 'ubuntu-24.04' }, { RUNNER_OVERRIDE: 'ubicloud-standard-8 x' }, { SHARDS_OVERRIDE: '0' }, { SHARDS_OVERRIDE: '41' }, { SHARDS_OVERRIDE: '16; true' }, + { JOBS_OVERRIDE: '0' }, { JOBS_OVERRIDE: '9' }, { JOBS_OVERRIDE: '2; true' }, { JOBS_OVERRIDE: '-1' }]; for (const bad of refusals) { const refused = resolve(bad); expect(refused.status, JSON.stringify(bad)).not.toBe(0); diff --git a/test/regression-issue3680-ubicloud-bsd-userland.test.ts b/test/regression-issue3680-ubicloud-bsd-userland.test.ts index a6fd98a86..85561a029 100644 --- a/test/regression-issue3680-ubicloud-bsd-userland.test.ts +++ b/test/regression-issue3680-ubicloud-bsd-userland.test.ts @@ -120,3 +120,13 @@ printf 200 } finally { rmSync(f.root, { recursive: true, force: true }); } }, 60_000); }); + +describe('CI runs these cases on the BSD userland itself', () => { + // Value: protects=the cases above run where /usr/bin/tar and /usr/bin/awk are the BSD tools; fails_when=macos-named-regressions stops naming this file, so Linux CI (no bsdtar or one-true-awk installed) skips every case and a #3680 or #3679 regression ships green; why_new=the file was only in the Linux free suite; seam=.github/workflows/free-tests.yml; no_claim=the job's runner image or its tool versions + test('free-tests.yml macos-named-regressions names this file', () => { + const workflow = Bun.YAML.parse(readFileSync(join(ROOT, '.github/workflows/free-tests.yml'), 'utf8')) as any; + const mac = workflow.jobs['macos-named-regressions']; + expect(mac['runs-on']).toMatch(/^macos-/); + expect((mac.steps as any[]).map(step => String(step.run ?? '')).join('\n')).toContain('test/regression-issue3680-ubicloud-bsd-userland.test.ts'); + }); +}); diff --git a/test/shard-jobs.test.ts b/test/shard-jobs.test.ts index 83714013d..af18dfa54 100644 --- a/test/shard-jobs.test.ts +++ b/test/shard-jobs.test.ts @@ -10,11 +10,11 @@ import * as fs from 'node:fs'; import * as os from 'node:os'; import * as path from 'node:path'; import { - collectFreeTestFiles, createFreeCiPlan, eligibleFreeRetryFiles, parseCliOptions, runFreeShard, TREE_MUTATING, verifyFreeCiResults, + collectFreeTestFiles, createFreeCiPlan, eligibleFreeRetryFiles, parseCliOptions, runFreeShard, TREE_MUTATING, validateFreeCiPlan, verifyFreeCiResults, type FreeCiResult, type FreeShardOutcome, } from '../scripts/test-free-shards'; import { - assertSoloSeedSample, assignSubShards, ciRunContext, FREE_JOBS_INVALID, mergeSubShardOutcomes, parseJobsFlag, parseJunitCounts, + assertSoloSeedSample, assignSubShards, ciRunContext, FREE_JOBS_INVALID, mergeSubShardOutcomes, parseJobsFlag, parseJunitCounts, plannedShardJobs, plannedShardJobsProblem, readJunitReport, runCiShard, SLOW_UNDER_LOAD, subShardRecordProblems, subShardTimeoutText, validateJobsForShard, type CiShardDeps, type RunShardOptions, } from '../scripts/lib/shard-jobs'; @@ -89,6 +89,52 @@ describe('--jobs validation (FREE_JOBS_INVALID)', () => { }); }); +describe('per-shard k in the plan (--ci-plan --shard-jobs)', () => { + test('--shard-jobs belongs to --ci-plan only and takes the same positive whole numbers', () => { + const base = ['--ci-plan', 'plan.json', '--shards', '4']; + expect(parseCliOptions([...base, '--shard-jobs', '2']).shardJobs).toBe(2); + expect('shardJobs' in parseCliOptions(base)).toBe(false); + for (const raw of ['0', '2.5', 'x']) expect(() => parseCliOptions([...base, '--shard-jobs', raw])).toThrow(`${FREE_JOBS_INVALID}: --shard-jobs needs a positive whole number`); + expect(() => parseCliOptions(['--shard-jobs', '2'])).toThrow(`${FREE_JOBS_INVALID}: --shard-jobs needs --ci-plan`); + expect(() => parseCliOptions(['--ci-run', 'p.json', '--shard', '1', '--result', 'r.json', '--shard-jobs', '2'])).toThrow(FREE_JOBS_INVALID); + }); + + test('each shard plans min(k, its parallel files); the exclusive shard and one-file shards stay one child, and k = 1 keeps the plan id', () => { + const withExclusive = [...files, EXCLUSIVE]; + const plain = createFreeCiPlan(withExclusive, 3, durations, 'revision-a'); + expect(createFreeCiPlan(withExclusive, 3, durations, 'revision-a', 1)).toEqual(plain); + expect(plain.shards.every(shard => !('jobs' in shard))).toBe(true); + const planned = createFreeCiPlan(withExclusive, 3, durations, 'revision-a', 4); + expect(planned.id).not.toBe(plain.id); + expect(planned.shards.map(shard => shard.files)).toEqual(plain.shards.map(shard => shard.files)); + const exclusiveShard = planned.shards.at(-1)!; + expect(exclusiveShard.files).toEqual([EXCLUSIVE]); + expect('jobs' in exclusiveShard).toBe(false); + for (const shard of planned.shards.slice(0, -1)) expect(shard.jobs).toBe(Math.min(4, shard.files.length) > 1 ? Math.min(4, shard.files.length) : undefined); + expect(() => validateFreeCiPlan(planned, withExclusive, 'revision-a')).not.toThrow(); + expect(plannedShardJobs(['a', 'b', 'c'], 2, file => file === 'a')).toBe(2); + expect(plannedShardJobs(['a', 'b'], 4, file => file === 'a')).toBeUndefined(); + expect(plannedShardJobs([], 4, () => false)).toBeUndefined(); + }); + + test('a plan whose per-shard k is out of range is invalid, and every shard result must have run at its planned k', () => { + const isSerial = (file: string) => file === EXCLUSIVE; + expect(plannedShardJobsProblem({ shard: 1, files }, isSerial)).toBeNull(); + expect(plannedShardJobsProblem({ shard: 1, files, jobs: 7 }, isSerial)).toBeNull(); + for (const jobs of [1, 0, 8, 2.5, '2']) expect(plannedShardJobsProblem({ shard: 1, files, jobs }, isSerial)).toContain('CI shard 1 plans --jobs'); + expect(plannedShardJobsProblem({ shard: 2, files: [EXCLUSIVE, files[0]], jobs: 2 }, isSerial)).toContain('for 1 parallel file(s)'); + const plan = createFreeCiPlan(files, 1, durations, 'revision-a', 2); + expect(plan.shards[0].jobs).toBe(2); + const tampered = structuredClone(plan); + tampered.shards[0].jobs = 99; + expect(() => validateFreeCiPlan(tampered, files, 'revision-a')).toThrow(); + const k1: FreeCiResult = { planId: plan.id, revision: 'revision-a', outcome: passed(plan.shards[0].files), retry: null }; + expect(() => verifyFreeCiResults(plan, [k1])).toThrow('CI shard 1 ran at --jobs 1; the plan says 2'); + const plainPlan = createFreeCiPlan(files, 1, durations, 'revision-a'); + expect(() => verifyFreeCiResults(plainPlan, [{ ...k1, planId: plainPlan.id }])).not.toThrow(); + }); +}); + describe('k = 1 golden: PR 2 changes nothing in CI', () => { test('one child over the whole planned shard with the same wall, one retry pass, and the same result keys', async () => { const h = harness(files); @@ -160,7 +206,8 @@ describe('k > 1: merge, rejection and one retry pass per CI shard', () => { summary: { testsRan: 16, filesRan: 8, sawTerminalSummary: true } }); expect(result.subShards!.map(record => record.status)).toEqual(['passed', 'passed', 'passed', 'passed']); expect(h.retries).toHaveLength(1); - expect(() => verifyFreeCiResults(h.plan, [result])).not.toThrow(); + expect(() => verifyFreeCiResults({ ...h.plan, shards: [{ ...h.shard, jobs: 3 }] }, [result])).not.toThrow(); + expect(() => verifyFreeCiResults(h.plan, [result])).toThrow('CI shard 1 ran at --jobs 3; the plan says 1'); }); test('failures from every sub-shard are collected first, so k never multiplies the five-file retry cap', async () => { @@ -263,7 +310,7 @@ describe('k > 1: merge, rejection and one retry pass per CI shard', () => { }); test('--ci-verify rejects tampered sub-shard records', () => { - const plan = createFreeCiPlan(files, 1, durations, 'revision-a'); + const plan = createFreeCiPlan(files, 1, durations, 'revision-a', 2); const shard = plan.shards[0]; const assignments = assignSubShards(plan.id, shard, 2, () => false, packFiles => [packFiles.slice(0, 3), packFiles.slice(3)]); const merged = mergeSubShardOutcomes(shard, assignments.map(assignment => ({ assignment, outcome: passed(assignment.files) })), 10); diff --git a/test/windows-native-workflows.test.ts b/test/windows-native-workflows.test.ts index 48ce50d9d..a111dc605 100644 --- a/test/windows-native-workflows.test.ts +++ b/test/windows-native-workflows.test.ts @@ -4,7 +4,9 @@ * that moved out of it. */ import { expect, test } from 'bun:test'; -import { existsSync, readFileSync } from 'node:fs'; +import { spawnSync } from 'node:child_process'; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import os from 'node:os'; import path from 'node:path'; import { collectFreeTestFiles, createFreeCiPlan, curateWindowsSafe, parseCliOptions, validateFreeCiPlan } from '../scripts/test-free-shards'; import { windowsCurationLine } from '../scripts/lib/free-ci-health'; @@ -27,7 +29,7 @@ const WINDOWS_CURATED_FLOOR = 795; test('the Windows lane runs nightly and by hand, plans, runs one strict shard per job and verifies every result', () => { expect(Object.keys(windows.on).sort()).toEqual(['schedule', 'workflow_call', 'workflow_dispatch']); expect(windows.on.schedule).toEqual([{ cron: '17 9 * * *' }]); - expect(Object.keys(windows.on.workflow_dispatch.inputs)).toEqual(['record_durations']); + expect(Object.keys(windows.on.workflow_dispatch.inputs)).toEqual(['record_durations', 'windows_jobs']); expect(Object.keys(windows.on.workflow_call.inputs)).toEqual(['record_durations']); const plan = windows.jobs['windows-plan']; expect(plan['runs-on']).toBe('ubuntu-24.04'); @@ -50,6 +52,36 @@ test('the Windows lane runs nightly and by hand, plans, runs one strict shard pe expect(aggregate.steps.find((step: any) => step.name === 'Require exact coverage and complete strict results').run).toContain('--windows-only --ci-verify "$RUNNER_TEMP/windows-plan.json" --results '); }); +test.skipIf(!Bun.which('bash'))('windows-plan passes the one sub-shard constant, or a validated dispatch override, to --shard-jobs', () => { + const planStep = windows.jobs['windows-plan'].steps.find((step: any) => step.id === 'plan'); + expect(windows.env.WINDOWS_FREE_JOBS).toMatch(/^[1-4]$/); + expect(planStep.env.JOBS_OVERRIDE).toBe('${{ inputs.windows_jobs }}'); + expect(planStep.run).not.toContain('${{'); + const directory = mkdtempSync(path.join(os.tmpdir(), 'windows plan ')); + const bin = path.join(directory, 'bin'); + mkdirSync(bin); + writeFileSync(path.join(bin, 'bun'), '#!/bin/sh\nprintf "%s\\n" "$*" > "$RUNNER_TEMP/bun-args"\necho \'{"shard":[1]}\'\n', { mode: 0o755 }); + const toShell = (p: string) => p.split(path.sep).join('/'); + const resolve = (override: string) => { + rmSync(path.join(directory, 'bun-args'), { force: true }); + const result = spawnSync('bash', ['-c', planStep.run], { + env: { ...process.env, ...windows.env, JOBS_OVERRIDE: override, PATH: `${toShell(bin)}:${process.env.PATH}`, RUNNER_TEMP: toShell(directory), + GITHUB_OUTPUT: toShell(path.join(directory, 'output')), GITHUB_STEP_SUMMARY: toShell(path.join(directory, 'summary')) }, + encoding: 'utf8', timeout: 5000, + }); + return { status: result.status, args: existsSync(path.join(directory, 'bun-args')) ? readFileSync(path.join(directory, 'bun-args'), 'utf8') : '' }; + }; + try { + expect(resolve('')).toEqual({ status: 0, args: expect.stringContaining(`--shards 6 --shard-jobs ${windows.env.WINDOWS_FREE_JOBS}`) }); + expect(resolve('2')).toEqual({ status: 0, args: expect.stringContaining('--shards 6 --shard-jobs 2') }); + for (const bad of ['0', '5', '2; true', 'x']) { + const refused = resolve(bad); + expect(refused.status, bad).not.toBe(0); + expect(refused.args, bad).toBe(''); + } + } finally { rmSync(directory, { recursive: true, force: true }); } +}); + test('a Windows CI plan binds the curated file set, so it never verifies as the full Linux suite', () => { for (const argv of [['--windows-only', '--ci-plan', 'p.json'], ['--windows-only', '--ci-run', 'p.json', '--shard', '1', '--result', 'r.json'], ['--windows-only', '--ci-verify', 'p.json', '--results', 'd']]) { expect(parseCliOptions(argv).windowsOnly).toBe(true); From eff19e267854c8fba5a254d8c6d419d9335e9e59 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Mon, 2 Nov 2026 05:07:54 -0800 Subject: [PATCH 07/19] v1.92.78.0 test: seven flaky-test repairs and six failure diagnostics from the 2026-11-01 flaky pass (GSTA-240) (#3804) * test: eight flaky-test repairs and five failure diagnostics from the 2026-11-01 flaky pass (GSTA-240) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(free-shards): keep failing a shard on an unseen browser child, and name it and whether it exited (review finding) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * changelog: pregate-regate copies through gitFixtureTemplate after the main merge Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 17 +++++++++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- bin/gstack-codex-probe | 4 +- bin/gstack-evidence | 2 +- package.json | 2 +- scripts/lib/free-home-guard.ts | 9 +++-- scripts/test-free-shards.ts | 2 + test/auq-parallel.test.ts | 3 +- test/autoplan-publication-guard.test.ts | 4 +- test/bootstrap-retention-shard.test.ts | 2 +- test/codex-probe-cache.test.ts | 25 ++++++++++++ test/evidence.test.ts | 8 ++++ test/free-home-guard.test.ts | 28 ++++++++++++++ test/gstack-design-detect.test.ts | 7 +++- test/office-hours-attempt.test.ts | 3 +- test/pregate-regate.test.ts | 51 +++++++++++++++++++++---- test/session-update-autostash.test.ts | 3 +- test/shard-engine.test.ts | 5 ++- test/ship-budget.test.ts | 2 +- test/ship-queue-train.test.ts | 7 +++- 21 files changed, 162 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bf0437f2c..517e1d270 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,22 @@ # Changelog +## [1.92.78.0] - 2026-11-02 + +**Seven flaky tests from the 2026-11-01 flaky pass stop failing correct runs, and six more now name their cause when they fail.** One product lock broke a fresh lock it had misjudged as abandoned, a /ship lane tripped the free runner's home check with its own log, and the rest were fixtures that raced their own setup, leaned on wall-clock sleeps or read the caller's state root. + +### Fixed +- **`bin/gstack-codex-probe`**: a waiter that cannot read an owner-less lock's age (the holder removed it in that instant, or `stat -f` printed filesystem text on Linux) now treats the lock as fresh instead of abandoned, so it no longer deletes a live owner's lock on a misread age and the probe no longer ends `MODEL_PROBE_INCONCLUSIVE` (Free Tests 41216564482, 41235671867, 41263831361). A stat stub for both race windows fails without the fix; 64 stress runs went from 2 failures to 0. +- **`scripts/lib/free-home-guard.ts`, `bin/gstack-evidence`**: `gstack-evidence run` names its own lane log in `GSTACK_EVIDENCE_LOG`, and the real-home tripwire ignores exactly that file, so a /ship lane no longer trips over its own output. A foreign write under the home's `~/.gstack` still fails. +- **`test/autoplan-publication-guard.test.ts`**: the render-root cases clear the state-root variables they don't name, so a `GSTACK_STATE_ROOT` exported by the gstack launch file no longer picks the root. Three cases failed every time under that variable, and the lookalike controls passed for the wrong reason. +- **`test/gstack-design-detect.test.ts`**: the directory scan proves one batch by counting engine calls instead of sleeping near the 1,500 ms detect timeout (macOS job 124813933194). The product timeout is unchanged. +- **`test/pregate-regate.test.ts`**: the two fixtures are built once and copied per test through the `gitFixtureTemplate` helper from #3586, so 8 builds become 2. +- **`test/ship-queue-train.test.ts`**: the planted seam has its own name, so the fixture's own background `git maintenance` lock no longer collides with it (EEXIST, Free Tests 41288253305, 41288417894), and the test now holds a real `maintenance.lock` beside it. +- **`test/ship-budget.test.ts`**: the fake child writes its args to a temporary file and renames it, so the poll never reads an empty file (Free Tests 41242243205). + +### Changed +- `scripts/test-free-shards.ts` names the recorded browser child it never saw in the daemon's process tree, and whether it was still alive, when it fails a shard for unconfirmed ownership (Free Tests 41264845860 shard 7). The shard still fails. +- `test/bootstrap-retention-shard.test.ts`, `test/office-hours-attempt.test.ts`, `test/session-update-autostash.test.ts`, `test/shard-engine.test.ts` and `test/auq-parallel.test.ts` print the process state, exit status or cleanup events behind their assertion, so the next red names its cause. + ## [1.92.77.0] - 2026-11-02 **The weekly marathon evals can run /design-motion again: the CI eval image now ships ffmpeg (#3687).** The `design-motion-showcase` case joined the marathon tier on 2026-10-26, and its first scheduled run (Actions run 41200316994, main @ 088bfc5) failed 0/3. Every trial built and captured the page, then stopped at `FFMPEG_MISSING`, because the image had no ffmpeg and the skill rightly refuses to export before its checks run. The model did nothing wrong; the case could not pass on that image. diff --git a/VERSION b/VERSION index 463685b58..66231b754 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.92.77.0 +1.92.78.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index ac6c1e65e..9f27e6a29 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.92.77.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.92.78.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/bin/gstack-codex-probe b/bin/gstack-codex-probe index 0292e5674..981ae1499 100755 --- a/bin/gstack-codex-probe +++ b/bin/gstack-codex-probe @@ -662,8 +662,8 @@ _gstack_codex_lock_acquire() { if [ "$_ohost" = "$_host" ] && ! kill -0 "$_pid" 2>/dev/null; then _dead=1; fi [ $((_now - _ts)) -ge "$_stale" ] && _dead=1 else - _ts=$(stat -c %Y "$_dir" 2>/dev/null || stat -f %m "$_dir" 2>/dev/null || echo 0) - case "$_ts" in ''|*[!0-9]*) _ts=0 ;; esac + _ts=$(stat -c %Y "$_dir" 2>/dev/null || stat -f %m "$_dir" 2>/dev/null || echo "$_now") + case "$_ts" in ''|*[!0-9]*) _ts="$_now" ;; esac [ $((_now - _ts)) -ge 5 ] && _dead=1 fi [ "$_dead" -eq 1 ] && _gstack_codex_lock_break "$_dir" "$_owner" diff --git a/bin/gstack-evidence b/bin/gstack-evidence index 59f8bbe30..35ca01d7b 100755 --- a/bin/gstack-evidence +++ b/bin/gstack-evidence @@ -400,7 +400,7 @@ async function cmdRun(argv: string[]): Promise { try { if (shell && !shell.ok) throw new Error(shell.error); const spawnArgv = shell?.ok ? [shell.cmd, ...shell.argv] : cmdArgv; - proc = Bun.spawn(spawnArgv, { stdin: "inherit", stdout: "pipe", stderr: "pipe" }); + proc = Bun.spawn(spawnArgv, { stdin: "inherit", stdout: "pipe", stderr: "pipe", env: log ? { ...process.env, GSTACK_EVIDENCE_LOG: log.path } : process.env }); } catch (e: any) { // Spawn failure (ENOENT on argv-direct form): record exit 127, propagate 127. exitCode = 127; diff --git a/package.json b/package.json index 8d40c09f7..bce0fb11b 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.92.77", + "version": "1.92.78", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/scripts/lib/free-home-guard.ts b/scripts/lib/free-home-guard.ts index 69b4bfd17..fb07555d3 100644 --- a/scripts/lib/free-home-guard.ts +++ b/scripts/lib/free-home-guard.ts @@ -5,8 +5,10 @@ * the real home before and after each free shard, and names the shard's files * when one of those entries changed. It watches only these surfaces; it is not * a universal write-containment check. Live agent-session logs that a - * concurrently running host keeps writing are excluded, and directory mtimes - * are ignored, so only file, link and directory-set changes count. + * concurrently running host keeps writing are excluded, as is the one lane log + * `gstack-evidence run` tees this run's own output into (its exact path, from + * GSTACK_EVIDENCE_LOG). Directory mtimes are ignored, so only file, link and + * directory-set changes count. * * A shard that runs alone owns its window, so its guard names its files. * Concurrent shards share one HOME and cannot tell whose write a change was, so @@ -104,6 +106,7 @@ export type FreeHomeGuardFactory = (files: string[], env: NodeJS.ProcessEnv, sta /** Take the baseline now; verify() returns null or the failure to report. */ export function guardFreeHome(files: string[], env: NodeJS.ProcessEnv = process.env, scope: FreeHomeScope = { kind: 'shard' }): FreeHomeGuard { const home = freeHomeDir(env); + const ownLog = env.GSTACK_EVIDENCE_LOG && path.isAbsolute(env.GSTACK_EVIDENCE_LOG) ? path.relative(home, env.GSTACK_EVIDENCE_LOG) : null; const unreadable = (error: unknown) => `real home surfaces could not be snapshotted (${(error as NodeJS.ErrnoException).code ?? 'error'}); shard files: ${files.join(', ')}`; let before: FreeHomeSnapshot | undefined; let baselineError: string | undefined; @@ -112,7 +115,7 @@ export function guardFreeHome(files: string[], env: NodeJS.ProcessEnv = process. verify() { if (!before) return baselineError!; try { - const changed = diffFreeHome(before, snapshotFreeHome(home)); + const changed = diffFreeHome(before, snapshotFreeHome(home)).filter(entry => entry !== ownLog); return changed.length ? formatFreeHomeChange(changed, files, scope) : null; } catch (error) { return unreadable(error); } }, diff --git a/scripts/test-free-shards.ts b/scripts/test-free-shards.ts index d61ca6112..b4012a60a 100755 --- a/scripts/test-free-shards.ts +++ b/scripts/test-free-shards.ts @@ -1370,6 +1370,8 @@ function trackShardBrowser(stateDir: string, env: NodeJS.ProcessEnv) { check(operation); if (!Number.isSafeInteger(pid) || (pid as number) <= 1) throw new BrowserCleanupError('invalid browser child identity'); const identity = identities.get(pid as number); + if (!identity && identities.get(daemon as number)?.root) throw new BrowserCleanupError( + `browser child ownership is unconfirmed: recorded pid ${pid} (${await inspect(operation, pid as number) ? 'alive' : 'exited'}) was never seen in daemon ${daemon}'s process tree`); if (!identity || identity.daemon !== daemon || identity.root) throw new BrowserCleanupError('browser child ownership is unconfirmed'); if (await live(operation, identity) && (typeof start !== 'string' || !start || await nativeStart(operation, identity) !== start.replace(/\s+/g, ' '))) throw new BrowserCleanupError('browser child identity was replaced'); diff --git a/test/auq-parallel.test.ts b/test/auq-parallel.test.ts index d7296a5b1..5a6facc94 100644 --- a/test/auq-parallel.test.ts +++ b/test/auq-parallel.test.ts @@ -111,7 +111,8 @@ test.each([ expect(facts.receipts.filter((receipt: any) => receipt.outcome === 'timeout')).toHaveLength(3 - scenario.passed); expect(facts.receipts.every((receipt: any) => receipt.timeoutMs === 240_000 && !receipt.answered)).toBe(true); expect(facts.active).toBe(0); expect(facts.answers).toBe(0); expect(facts.pendingTimers).toBe(0); - expect(facts.leftovers).toEqual([]); expect(facts.ownedStateLeftovers).toEqual([]); + expect(facts.leftovers).toEqual([]); + expect(facts.ownedStateLeftovers, JSON.stringify({ fixtures: facts.fixtures, inputs: facts.inputs.map((input: any) => [input.id, input.config, input.state]), events: facts.events.filter((event: any) => ['query-close', 'capture-settle', 'cleanup'].includes(event.kind)) })).toEqual([]); expect(facts.events.filter((event: any) => event.kind === 'query-close')).toHaveLength(scenario.queries); expect(facts.events.filter((event: any) => event.kind === 'cleanup')).toHaveLength(3); }, 20_000); diff --git a/test/autoplan-publication-guard.test.ts b/test/autoplan-publication-guard.test.ts index 44a375db4..d39a6baa2 100644 --- a/test/autoplan-publication-guard.test.ts +++ b/test/autoplan-publication-guard.test.ts @@ -1326,7 +1326,9 @@ describe('Autoplan phase entry through the gbrain :user render (#2569)', () => { edit(rewriteSectionBase(fs.readFileSync(path.join(ROOT, 'autoplan/sections', name), 'utf8'), linkRoot))); return root; } - function withEnv(env: Record, work: () => T): T { + /** The state-root chain the case does not name is cleared, so an ambient GSTACK_STATE_ROOT (every gstack launch file exports one) cannot pick the root. */ + function withEnv(given: Record, work: () => T): T { + const env = { GSTACK_STATE_ROOT: undefined, GSTACK_STATE_DIR: undefined, CLAUDE_PLUGIN_DATA: undefined, ...given }; const previous = Object.fromEntries(Object.keys(env).map(k => [k, process.env[k]])); for (const [k, v] of Object.entries(env)) if (v === undefined) delete process.env[k]; else process.env[k] = v; try { return work(); } diff --git a/test/bootstrap-retention-shard.test.ts b/test/bootstrap-retention-shard.test.ts index cbeac1275..2b120bd4e 100644 --- a/test/bootstrap-retention-shard.test.ts +++ b/test/bootstrap-retention-shard.test.ts @@ -62,7 +62,7 @@ describe.skipIf(process.platform !== 'linux')('bootstrap paid-shard cleanup inte fs.writeFileSync(path.join(root, 'controller.stdout.log'), controller.stdout ?? '', { mode: 0o600 }); fs.writeFileSync(path.join(root, 'controller.stderr.log'), controller.stderr ?? '', { mode: 0o600 }); expect(controller.error).toBeUndefined(); - expect(controller.status).toBe(0); + expect(controller.status, `controller exited ${controller.status} (signal ${controller.signal}); stderr tail: ${(controller.stderr ?? '').slice(-600).replace(/\s*\n\s*/g, ' | ')}`).toBe(0); const outcome = JSON.parse(fs.readFileSync(outcomePath, 'utf8')); if (scenario === 'ack-failure') { expect(controller.stdout).toContain('(fail) qa-bootstrap'); diff --git a/test/codex-probe-cache.test.ts b/test/codex-probe-cache.test.ts index 002fdb921..981fde6dd 100644 --- a/test/codex-probe-cache.test.ts +++ b/test/codex-probe-cache.test.ts @@ -247,6 +247,31 @@ describe('probe lock recovery and failure', () => { expect(calls(f)).toBe(1); })); + test('a lock released while a waiter reads its age is not abandoned; the next owner\'s fresh lock survives', withFixture((f) => { + const sig = sigOf(f); + const lock = path.join(f.locks, sig); + const once = path.join(f.home, 'vanish-once'); + const real = JSON.stringify(Bun.which('stat')); + fs.mkdirSync(lock, { recursive: true }); + fs.writeFileSync(path.join(f.home, 'bin/stat'), `#!/usr/bin/env bash +if [ "$3" = "$RACE_LOCK" ] && [ -f "$RACE_ONCE" ]; then + case "$1" in + -c) rmdir "$RACE_LOCK"; [ "$RACE_PHASE" = early ] && mkdir "$RACE_LOCK"; exit 1 ;; + -f) ${real} "$@"; rc=$?; rm -f "$RACE_ONCE"; [ -d "$RACE_LOCK" ] || mkdir "$RACE_LOCK"; exit "$rc" ;; + esac +fi +exec ${real} "$@" +`, { mode: 0o755 }); + for (const phase of ['early', 'late']) { + fs.writeFileSync(once, 'armed'); + const r = probe(f, { _GSTACK_CODEX_LOCK_WAIT: '1', RACE_LOCK: lock, RACE_ONCE: once, RACE_PHASE: phase }); + expect(fs.existsSync(once)).toBe(false); + expect(r.stdout).toContain('MODEL_PROBE_INCONCLUSIVE (another probe of this model still holds its lock)'); + expect(fs.existsSync(lock)).toBe(true); + expect(calls(f)).toBe(0); + } + })); + test('an unreadable cache or uncreatable lock is unverified and launches no paid probe', withFixture((f) => { fs.mkdirSync(f.cache); const dir = probe(f); diff --git a/test/evidence.test.ts b/test/evidence.test.ts index e40d6769c..0617ef8ed 100644 --- a/test/evidence.test.ts +++ b/test/evidence.test.ts @@ -74,6 +74,14 @@ describe('gstack-evidence run', () => { expect(fs.readFileSync(rec.log_path, 'utf-8')).toContain('ok'); }); + test('the lane command sees its own log path as GSTACK_EVIDENCE_LOG (the free runner\'s home tripwire excludes exactly that file)', () => { + const r = run(['run', '--label', 'focused', '--', 'printf "%s" "$GSTACK_EVIDENCE_LOG"']); + expect(r.status).toBe(0); + const rec = records().pop(); + expect(path.isAbsolute(rec.log_path)).toBe(true); + expect(r.stdout).toBe(rec.log_path); + }); + test('propagates a failing exit code and records it', () => { const r = run(['run', '--label', 'tests', '--', 'exit 3']); expect(r.status).toBe(3); diff --git a/test/free-home-guard.test.ts b/test/free-home-guard.test.ts index d01c8c979..4f387b5bd 100644 --- a/test/free-home-guard.test.ts +++ b/test/free-home-guard.test.ts @@ -78,6 +78,34 @@ describe('free-suite home-write tripwire (#2895)', () => { expect(failure).toContain('only'); }); + test.skipIf(process.platform === 'win32')('the wrapping gstack-evidence lane log is excluded by its exact path; a foreign write beside it still trips', async () => { + const home = fakeHome(); + const logs = path.join(home, '.gstack/projects/p/logs'); + fs.mkdirSync(logs, { recursive: true }); + const laneLog = path.join(logs, 'focused.log'); + fs.writeFileSync(laneLog, 'lane start\n'); + const unexcluded = guardFreeHome(['test/a.test.ts'], { HOME: home }); + const options = (script: string, name: string) => ({ env: { ...process.env, HOME: home, GSTACK_EVIDENCE_LOG: laneLog }, quiet: true, log: () => {}, + logFilePath: path.join(home, `${name}.log`), + commandFor: () => ({ command: process.execPath, args: ['-e', `${script}console.log('Ran 1 tests across 1 files. [1.00ms]')`] }) }); + const grow = `require('fs').appendFileSync(${JSON.stringify(laneLog)}, 'lane output\\n');`; + const errors: string[] = []; + const spy = spyOn(console, 'error').mockImplementation((...args: unknown[]) => { errors.push(args.join(' ')); }); + try { + expect((await runFreeShard(['test/a.test.ts'], 1, 1, options(grow, 'lane'))).status).toBe('passed'); + expect(unexcluded.verify()).toContain('~/.gstack/projects/p/logs/focused.log'); + const foreign = `${grow}require('fs').writeFileSync(${JSON.stringify(path.join(logs, 'other.log'))}, 'x');`; + expect((await runFreeShard(['test/a.test.ts'], 1, 1, options(foreign, 'foreign'))).status).toBe('failed'); + expect(errors.join('\n')).toContain('~/.gstack/projects/p/logs/other.log'); + expect(errors.join('\n')).not.toContain('focused.log'); + } finally { + spy.mockRestore(); + for (const retained of errors.flatMap(line => /retained (\S+)$/.exec(line)?.[1] ?? [])) { + if (/^gstack-free-shard-/.test(path.basename(retained))) fs.rmSync(retained, { recursive: true, force: true }); + } + } + }); + test('a concurrent phase guard names no shard and points at attribution', () => { const home = fakeHome(); const guard = guardFreeHome(['test/a.test.ts', 'test/b.test.ts'], { HOME: home }, { kind: 'concurrent', shards: 3 }); diff --git a/test/gstack-design-detect.test.ts b/test/gstack-design-detect.test.ts index 6349ec7d9..39388ac41 100644 --- a/test/gstack-design-detect.test.ts +++ b/test/gstack-design-detect.test.ts @@ -1460,8 +1460,13 @@ describe('scan: option-like bases and page-controlled inline ignores', () => { fs.mkdirSync(many, { recursive: true }); for (let i = 0; i < 1100; i++) fs.writeFileSync(path.join(many, `f${i}.css`), 'a{}'); try { - const r = run(['scan', '--format', 'gstack', 'src/many'], { env: { IMPECCABLE_BIN: FAKE, IMPECCABLE_FAKE_SLEEP_MS: '1000', GSTACK_DESIGN_DETECT_TIMEOUT_MS: '1500' } }); + const log = path.join(SANDBOX, 'argv-many.log'); + fs.rmSync(log, { force: true }); + const r = run(['scan', '--format', 'gstack', 'src/many'], { env: { IMPECCABLE_BIN: FAKE, IMPECCABLE_FAKE_LOG: log } }); expect(r.code).toBe(2); // a directory target is one batch (the fake reports findings): the budget test needs files + const calls = fs.readFileSync(log, 'utf-8').trim().split('\n').map(l => JSON.parse(l).argv as string[]); + expect(calls).toHaveLength(1); + expect(calls[0].some(x => x.endsWith(path.join('src', 'many')))).toBe(true); const files = fs.readdirSync(many).map(f => path.join('src', 'many', f)); const r2 = run(['scan', '--format', 'gstack', ...files], { env: { IMPECCABLE_BIN: FAKE, IMPECCABLE_FAKE_SLEEP_MS: '1000', GSTACK_DESIGN_DETECT_TIMEOUT_MS: '1500' } }); expect(r2.err).toMatch(/DETECT_TIMEOUT: whole-scan budget 7500ms exceeded, \d+ of 11 batches not run/); diff --git a/test/office-hours-attempt.test.ts b/test/office-hours-attempt.test.ts index c36da4486..c96bc8889 100644 --- a/test/office-hours-attempt.test.ts +++ b/test/office-hours-attempt.test.ts @@ -312,7 +312,8 @@ describe('Office Hours real session runner with fake processes', () => { expect(captured.output).toBe('captured output'); const diagnostic = JSON.parse(fs.readFileSync(path.join(dir, '.gstack/test-transcripts/drain-failure.json'), 'utf8')); expect(diagnostic.stderr).toContain('fixture auth failure'); - expect(running(Number(fs.readFileSync(path.join(dir, 'child.pid'), 'utf8')))).toBe(false); + const child = fs.readFileSync(path.join(dir, 'child.pid'), 'utf8').trim(); + expect(running(Number(child)), `held-pipe child after the runner returned (group ${fs.readFileSync(path.join(dir, 'parent.pid'), 'utf8').trim()}): ${spawnSync('ps', ['-o', 'pid=,ppid=,pgid=,stat=,etime=', '-p', child], { encoding: 'utf8', timeout: 5_000 }).stdout.trim() || 'gone now'}`).toBe(false); }); }, 10_000); } diff --git a/test/pregate-regate.test.ts b/test/pregate-regate.test.ts index 2e31a4c5f..09f01a004 100644 --- a/test/pregate-regate.test.ts +++ b/test/pregate-regate.test.ts @@ -6,11 +6,12 @@ * the base branch and every other open PR, and a lookup that cannot finish is * `unknown`, which withholds `REGATE: not-needed`. */ -import { afterEach, describe, expect, test } from 'bun:test'; +import { afterAll, afterEach, beforeAll, describe, expect, test } from 'bun:test'; import * as fs from 'node:fs'; import * as path from 'node:path'; import { EXIT } from '../lib/headless-artifacts'; -import { BASE_POLICY, cleanup, makeFixtureRepo, makePr, recordAndBundle, restamp, runBin, type FixtureRepo } from './helpers/restamp-fixture'; +import { gitFixtureTemplate, rebindFile, rebindGitDir, type GitFixtureTemplate } from './helpers/git-fixture-template'; +import { BASE_POLICY, cleanup, fixtureRepoAt, makePr, recordAndBundle, restamp, runBin, seedFixtureRepo, type FixtureRepo } from './helpers/restamp-fixture'; const repos: FixtureRepo[] = []; afterEach(() => { while (repos.length) cleanup(repos.pop()!); }); @@ -39,10 +40,14 @@ console.error("gh shim: unsupported "+a.join(" ")); process.exit(1)' "${file}" " } const ghCalls = (repo: FixtureRepo) => fs.readFileSync(path.join(repo.root, 'gh-regate', 'calls.log'), 'utf8'); +type Lanes = Array<{ label: string; command: string }>; +const TESTS_LANE: Lanes = [{ label: 'tests', command: 'true' }]; +const DIGEST_LANES: Lanes = [...TESTS_LANE, { label: 'digest', command: 'cat generated/digest.md' }]; + /** pr-a adds migration v240, is stamped 1.2.4.0 and gated (lanes recorded and bundled); then restamped to 1.2.5.0. */ -function gatedThenRestamped(lanes: Array<{ label: string; command: string }> = [{ label: 'tests', command: 'true' }]) { - const repo = makeFixtureRepo({ prefix: 'pregate-regate-', policy: POLICY }); - repos.push(repo); +function buildGated(root: string, lanes: Lanes) { + seedFixtureRepo(root, { policy: POLICY }); + const repo = fixtureRepoAt(root); makePr(repo, 'pr-a', { expects: '1.2.4.0' }); fs.mkdirSync(path.join(repo.work, 'migrations'), { recursive: true }); fs.writeFileSync(path.join(repo.work, 'migrations', 'v240-add-index.sql'), 'create index a;\n'); @@ -54,7 +59,39 @@ function gatedThenRestamped(lanes: Array<{ label: string; command: string }> = [ const head = repo.git('rev-parse', 'HEAD'); expect(restamp(repo, ['--version', '1.2.5.0']).status).toBe(EXIT.ok); repo.git('commit', '-qam', 'stamp 1.2.5.0'); - return { repo, head, bundle, current: repo.git('rev-parse', 'HEAD') }; + return { head, current: repo.git('rev-parse', 'HEAD') }; +} + +// Each fixture costs four gstack bin runs (two restamps, an evidence run and a bundle); build each lane set once +// and give every test its own copy instead of rebuilding it per test. +const templates = new Map>>(); +beforeAll(() => { + for (const lanes of [TESTS_LANE, DIGEST_LANES]) { + const template = gitFixtureTemplate({ + prefix: 'pregate-regate-', + build: root => buildGated(root, lanes), + rebind: (copy, tpl) => { + rebindGitDir(path.join(copy, 'work', '.git'), tpl, copy); + rebindFile(path.join(copy, 'bundle.json'), tpl, copy); + const projects = path.join(copy, 'home', 'projects'); + for (const slug of fs.existsSync(projects) ? fs.readdirSync(projects) : []) { + const dir = path.join(projects, slug.split(path.basename(tpl)).join(path.basename(copy))); + if (dir !== path.join(projects, slug)) fs.renameSync(path.join(projects, slug), dir); + for (const file of fs.readdirSync(dir, { recursive: true, withFileTypes: true })) if (file.isFile()) rebindFile(path.join(file.parentPath, file.name), tpl, copy); + } + fs.rmSync(path.join(copy, 'home', '.capy'), { recursive: true, force: true }); + }, + }); + template.copy().root && templates.set(lanes, template); + } +}, 60_000); +afterAll(() => { for (const t of templates.values()) t.dispose(); }); + +function gatedThenRestamped(lanes: Lanes = TESTS_LANE) { + const { root, meta: { head, current } } = templates.get(lanes)!.copy(); + const repo = fixtureRepoAt(root); + repos.push(repo); + return { repo, head, bundle: path.join(repo.root, 'bundle.json'), current }; } const regate = (repo: FixtureRepo, args: string[], env: Record) => runBin(repo, 'gstack-pregate', ['regate', '--base', 'main', ...args], env); @@ -97,7 +134,7 @@ describe('gstack-pregate regate: evidence, stamps and lane verdicts', () => { }); test('a stamp-sensitive lane reruns, so a regate is needed; a code edit is not stamp-only and names the file and line', () => { - const { repo, head, bundle } = gatedThenRestamped([{ label: 'tests', command: 'true' }, { label: 'digest', command: 'cat generated/digest.md' }]); + const { repo, head, bundle } = gatedThenRestamped(DIGEST_LANES); const env = ghShim(repo, { list: [] }); const r = regate(repo, ['--against', head, '--bundle', bundle], env); expect(r.status).toBe(EXIT.fail); diff --git a/test/session-update-autostash.test.ts b/test/session-update-autostash.test.ts index f2ab223d9..d5a0cff24 100644 --- a/test/session-update-autostash.test.ts +++ b/test/session-update-autostash.test.ts @@ -284,7 +284,8 @@ describe('gstack-session-update lock identity + TTL (#2613)', () => { const r = runScript(base, install, state); expect(r.status).toBe(0); const log = await waitForLog(state, /UP_TO_DATE|UPDATING/); - expect(log).toMatch(/UP_TO_DATE|UPDATING/); + const lockPid = fs.existsSync(path.join(lockDir, 'pid')) ? fs.readFileSync(path.join(lockDir, 'pid'), 'utf8').trim() : 'no lock'; + expect(log, `planted dead pid ${dead.pid} (alive now: ${isAlive(dead.pid)}); lock pid after 15 s: ${lockPid} (alive: ${isAlive(Number(lockPid))}); script exit ${r.status}, stderr: ${r.stderr}`).toMatch(/UP_TO_DATE|UPDATING/); } finally { fs.rmSync(base, { recursive: true, force: true }); } diff --git a/test/shard-engine.test.ts b/test/shard-engine.test.ts index 272d5d740..60c99de61 100644 --- a/test/shard-engine.test.ts +++ b/test/shard-engine.test.ts @@ -171,7 +171,10 @@ describe('shard engine: process lifecycle', () => { }); expect(paid.status).toBe('timed-out'); expect(summaryExitCode(summarize([paid]))).toBe(1); - expect(await waitDead(Number(fs.readFileSync(pidFile, 'utf8')))).toBe(true); + const paidChild = Number(fs.readFileSync(pidFile, 'utf8')); + const paidChildDead = await waitDead(paidChild); + const procStat = (pid: number) => { try { return fs.readFileSync(`/proc/${pid}/stat`, 'utf8').trim(); } catch { return 'no /proc entry'; } }; + expect(paidChildDead, `paid grandchild ${paidChild} of group ${paid.groupPid} 2 s after the timeout kill: ${procStat(paidChild)}`).toBe(true); }, 30_000); test('win32 semantics: no process group and never a negative-pid kill', async () => { diff --git a/test/ship-budget.test.ts b/test/ship-budget.test.ts index 2ce43a144..5043da5a1 100644 --- a/test/ship-budget.test.ts +++ b/test/ship-budget.test.ts @@ -308,7 +308,7 @@ describe('Claude Code child launch', () => { const file = ho.out.match(/^HANDOFF: (\S+)/m)![1]!; const argsFile = path.join(h.root, 'claude-args.txt'); const fake = path.join(h.root, 'fake-claude'); - fs.writeFileSync(fake, `#!/bin/sh\nprintf '%s\\n' "$@" > "${argsFile}"\n`, { mode: 0o755 }); + fs.writeFileSync(fake, `#!/bin/sh\nprintf '%s\\n' "$@" > "${argsFile}.tmp" && mv "${argsFile}.tmp" "${argsFile}"\n`, { mode: 0o755 }); const r = h.sb(['launch', '--in', file], { GSTACK_CLAUDE_BIN: fake }); expect(r.code).toBe(0); const session = h.field(r.out, 'CHILD', 'session'); diff --git a/test/ship-queue-train.test.ts b/test/ship-queue-train.test.ts index ad5c056a3..9a46e65be 100644 --- a/test/ship-queue-train.test.ts +++ b/test/ship-queue-train.test.ts @@ -35,13 +35,16 @@ const result = (out: string) => line(out, 'GSTACK_RESULT:'); const next = (out: string) => line(out, 'NEXT:'); const remoteRefs = (fx: TrainFixture) => fx.repo.git('ls-remote', 'origin'); -// fails_when=coordinatorClone copies origin.git/objects file by file again, so an entry that appears or vanishes there mid-copy (the detached maintenance.lock every push into the fixture origin leaves) fails the clone, as in two Free Tests reds on 2026-10-28 (GSTA-139 ledger); why_new=no other test clones while origin's objects directory changes; seam=a dangling symlink planted under objects/, which a local copy refuses on every run; no_claim=which process removed the loose object those runs named +// fails_when=coordinatorClone copies origin.git/objects file by file again, so an entry that appears or vanishes there mid-copy (the detached maintenance.lock every push into the fixture origin leaves) fails the clone, as in two Free Tests reds on 2026-10-28 (GSTA-139 ledger); why_new=no other test clones while origin's objects directory changes; seam=a dangling symlink planted under objects/ beside a held maintenance.lock, which a local copy refuses on every run; the seam has its own name because the fixture push's own detached maintenance can hold objects/maintenance.lock at that moment (EEXIST in Free Tests 41288253305 and 41288417894, 2026-11-01); no_claim=which process removed the loose object those runs named test('the coordinator clone reads origin through git, never by copying its objects directory (GSTA-139)', () => { const fx = make(); - fs.symlinkSync(path.join(fx.repo.root, 'gone'), path.join(fx.repo.origin, 'objects', 'maintenance.lock')); + const objects = path.join(fx.repo.origin, 'objects'); + fs.writeFileSync(path.join(objects, 'maintenance.lock'), ''); + fs.symlinkSync(path.join(fx.repo.root, 'gone'), path.join(objects, 'gstack-dangling-seam')); const coord = coordinatorClone(fx.repo); expect(spawnSync('git', ['-C', coord, 'rev-parse', 'origin/main'], { encoding: 'utf8', timeout: 30_000, env: { ...process.env, ...fx.repo.env } }).stdout.trim()).toBe(fx.repo.git('rev-parse', 'origin/main')); expect(fs.existsSync(path.join(coord, '.git', 'objects', 'maintenance.lock'))).toBe(false); + expect(fs.readdirSync(path.join(coord, '.git', 'objects'))).not.toContain('gstack-dangling-seam'); }); describe('the train command surface', () => { From 322906ab2b0975c6b009ae4a2804259be9e8ed70 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Mon, 2 Nov 2026 05:17:17 -0800 Subject: [PATCH 08/19] v1.92.79.0 fix: /plan-tune stops reporting mismatches with no observed answers behind them (#3436) (#3856) * fix(plan-tune): compare declared and inferred only with 3+ observed answers (#3436) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: derive keeps the gap at 3 events; changelog fragment for #3436 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(plan-tune): profile view and design doc honor the 3-answer floor (#3436) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(plan-tune): keep the 10-answer floor in --check-mismatch beside the per-dimension floor Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 11 ++++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- bin/gstack-developer-profile | 23 +++++-- docs/designs/PLAN_TUNING_V0.md | 2 +- package.json | 2 +- plan-tune/SKILL.md | 5 ++ plan-tune/SKILL.md.tmpl | 5 ++ test/gstack-developer-profile.test.ts | 92 +++++++++++++++++++++++++++ 9 files changed, 135 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 517e1d270..1fc172340 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,16 @@ # Changelog +## [1.92.79.0] - 2026-11-02 + +**/plan-tune no longer reports a mismatch between your stated and observed preferences when it has no observed answers behind it (#3436).** `--check-mismatch` compared a declared value against an inferred value that had never moved from its 0.5 starting point, so a strong declaration with no matching answers printed `scope_appetite: declared 0.9 vs inferred 0.5 (gap 0.4)` even though no logged answer touched that dimension. + +### Fixed +- **`bin/gstack-developer-profile`**: `--check-mismatch`, `--gap` and the gap that `--derive` stores compare a dimension only when at least 3 logged answers moved it. `--check-mismatch` still needs 10 logged answers overall before it reports anything. Below the 3-answer floor, `--check-mismatch` prints `no observed evidence` or `too little evidence (n of 3 events)` instead of a gap, and `--gap` lists the dimension under a new `low_evidence` field. A profile derived before `contributing_events` existed keeps the old behavior until its next derive. +- **/plan-tune "Show gap"**: says "not enough observed answers yet" for those dimensions instead of calling them drift. + +### Tests +- `test/gstack-developer-profile.test.ts`: a declared dimension with no evidence is not flagged by `--check-mismatch` or `--gap` (both failed on main), 1 or 2 events report too little evidence, 3 events are compared, three same-day answers that agree with the declaration still report not enough data, and `--derive` stores no gap without evidence. + ## [1.92.78.0] - 2026-11-02 **Seven flaky tests from the 2026-11-01 flaky pass stop failing correct runs, and six more now name their cause when they fail.** One product lock broke a fresh lock it had misjudged as abandoned, a /ship lane tripped the free runner's home check with its own log, and the rest were fixtures that raced their own setup, leaned on wall-clock sleeps or read the caller's state root. diff --git a/VERSION b/VERSION index 66231b754..dfb847afe 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.92.78.0 +1.92.79.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index 9f27e6a29..192027e8e 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.92.78.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.92.79.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/bin/gstack-developer-profile b/bin/gstack-developer-profile index bc489c5c1..409c4f56e 100755 --- a/bin/gstack-developer-profile +++ b/bin/gstack-developer-profile @@ -54,6 +54,7 @@ case "$(uname -s)" in esac PROFILE_FILE="$GSTACK_HOME/developer-profile.json" LEGACY_FILE="$GSTACK_HOME/builder-profile.jsonl" +MIN_EVIDENCE=3 eval "$("$SCRIPT_DIR/gstack-slug" 2>/dev/null || true)" SLUG="${SLUG:-unknown}" @@ -512,14 +513,19 @@ do_gap() { const p = JSON.parse(await Bun.stdin.text()); const declared = p.declared || {}; const inferred = (p.inferred && p.inferred.values) || {}; + const contributing = p.inferred && p.inferred.contributing_events; const dims = ['scope_appetite','risk_tolerance','detail_preference','autonomy','architecture_care']; const gap = {}; + const low_evidence = {}; for (const d of dims) { - if (declared[d] !== undefined && inferred[d] !== undefined) { - gap[d] = +(Math.abs(declared[d] - inferred[d])).toFixed(3); + if (declared[d] === undefined || inferred[d] === undefined) continue; + if (contributing && (contributing[d] || 0) < $MIN_EVIDENCE) { + low_evidence[d] = contributing[d] || 0; + continue; } + gap[d] = +(Math.abs(declared[d] - inferred[d])).toFixed(3); } - console.log(JSON.stringify({ declared, inferred, gap }, null, 2)); + console.log(JSON.stringify({ declared, inferred, gap, low_evidence }, null, 2)); " } @@ -657,6 +663,7 @@ do_derive() { // Recompute gap. const gap = {}; for (const d of Object.keys(values)) { + if ((contributing[d] || 0) < $MIN_EVIDENCE) continue; if (profile.declared && profile.declared[d] !== undefined) { gap[d] = +(Math.abs(profile.declared[d] - values[d])).toFixed(3); } @@ -727,9 +734,8 @@ do_check_mismatch() { const declared = p.declared || {}; const inferred = (p.inferred && p.inferred.values) || {}; const sampleSize = (p.inferred && p.inferred.sample_size) || 0; - const diversity = (p.inferred && p.inferred.diversity) || {}; + const contributing = p.inferred && p.inferred.contributing_events; - // Require enough data before reporting mismatch. if (sampleSize < 10) { console.log('MISMATCH: not enough data (' + sampleSize + ' events; need 10+)'); process.exit(0); @@ -737,8 +743,14 @@ do_check_mismatch() { const THRESHOLD = 0.3; const flagged = []; + const thin = []; for (const d of Object.keys(declared)) { if (inferred[d] === undefined) continue; + const n = contributing ? (contributing[d] || 0) : undefined; + if (n !== undefined && n < $MIN_EVIDENCE) { + thin.push(' ' + d + ': ' + (n === 0 ? 'no observed evidence' : 'too little evidence (' + n + ' of $MIN_EVIDENCE events)')); + continue; + } const gap = Math.abs(declared[d] - inferred[d]); if (gap > THRESHOLD) { flagged.push({ dim: d, declared: declared[d], inferred: inferred[d], gap: +gap.toFixed(3) }); @@ -753,6 +765,7 @@ do_check_mismatch() { console.log(' ' + f.dim + ': declared ' + f.declared + ' vs inferred ' + f.inferred + ' (gap ' + f.gap + ')'); } } + for (const line of thin) console.log(line); " } diff --git a/docs/designs/PLAN_TUNING_V0.md b/docs/designs/PLAN_TUNING_V0.md index e9bf2b440..77bf95e8d 100644 --- a/docs/designs/PLAN_TUNING_V0.md +++ b/docs/designs/PLAN_TUNING_V0.md @@ -105,7 +105,7 @@ After weighing Codex's argument, we chose to roll back CEO EXPANSION and ship an } ``` -`signal_events` counts the logged answers that moved any dimension, and `contributing_events` counts, per dimension, the logged answers that moved it (an answer that writes two deltas to one dimension counts once), so a dimension with no evidence (0) is visible beside one whose evidence balances out; both print 0.5 (#2623). Each value is the sigmoid of its accumulated deltas, and each UTC day's sum per dimension (by the row's `ts`) is clamped to the largest single delta in the signal map (currently ±0.06), so one burst of answers can't pin a dimension; `--trace ` lists every contributing answer, each clamped day and the total. The bound keys on the day, not on `session_id`: the PostToolUse hook and the agent log different ids for the same review (the host's conversation id and the preamble's run id), and an agent can leave the `SESSION_ID` placeholder unsubstituted, so no logged id reliably marks one review. The cost is that several reviews on one day count as one. `contributing_events` still counts an answer logged by both writers twice. Only an ISO 8601 date or date-time `ts` counts as dated (a time without a zone is read as UTC), every other row shares one undated bucket, and `diversity.days_span` counts the same UTC days. Nothing displays `contributing_events` yet (#3436). +`signal_events` counts the logged answers that moved any dimension, and `contributing_events` counts, per dimension, the logged answers that moved it (an answer that writes two deltas to one dimension counts once), so a dimension with no evidence (0) is visible beside one whose evidence balances out; both print 0.5 (#2623). Each value is the sigmoid of its accumulated deltas, and each UTC day's sum per dimension (by the row's `ts`) is clamped to the largest single delta in the signal map (currently ±0.06), so one burst of answers can't pin a dimension; `--trace ` lists every contributing answer, each clamped day and the total. The bound keys on the day, not on `session_id`: the PostToolUse hook and the agent log different ids for the same review (the host's conversation id and the preamble's run id), and an agent can leave the `SESSION_ID` placeholder unsubstituted, so no logged id reliably marks one review. The cost is that several reviews on one day count as one. `contributing_events` still counts an answer logged by both writers twice. Only an ISO 8601 date or date-time `ts` counts as dated (a time without a zone is read as UTC), every other row shares one undated bucket, and `diversity.days_span` counts the same UTC days. `--check-mismatch`, `--gap`, the stored `gap` and /plan-tune's profile view compare a dimension only when its `contributing_events` is at least 3; `--gap` lists the others under `low_evidence` and the stored `gap` omits them (#3436). **Diversity check** (Codex #13): `inferred` is considered "enough data" only when `sample_size >= 20 AND skills_covered >= 3 AND question_ids_covered >= 8 AND days_span >= 7`. Below this, `/plan-tune profile` shows "not enough observed data yet" instead of a potentially-misleading inferred value. diff --git a/package.json b/package.json index bce0fb11b..b8a73016c 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.92.78", + "version": "1.92.79", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/plan-tune/SKILL.md b/plan-tune/SKILL.md index 86b2cbdb7..481d7764e 100644 --- a/plan-tune/SKILL.md +++ b/plan-tune/SKILL.md @@ -341,6 +341,8 @@ Parse the JSON. Present in **plain English**, not raw floats: the inferred column next to declared: "**scope_appetite:** declared 0.8 (boil the ocean) ↔ observed 0.72 (close)" Use words for the gap: 0.0-0.1 "close", 0.1-0.3 "drift", 0.3+ "mismatch". + When `inferred.contributing_events[dim]` is below 3, show "not enough + observed answers yet" for that dimension instead of an observed value or gap word. This display gate is intentionally lower than the E1 **promotion gate** (90+ days stable across 3+ skills, per `docs/designs/PLAN_TUNING_V0.md`). @@ -491,6 +493,9 @@ Parse the JSON. For each dimension where both declared and inferred exist: Consider updating your declared value, or reflect on whether your behavior is actually what you want." +For each dimension in `low_evidence` (fewer than 3 observed answers moved it), +say "not enough observed answers yet" instead of a gap. + Never auto-update declared based on the gap. In v1 the gap is reporting only — the user decides whether declared is wrong or behavior is wrong. diff --git a/plan-tune/SKILL.md.tmpl b/plan-tune/SKILL.md.tmpl index 748dfad62..88b7a8eb3 100644 --- a/plan-tune/SKILL.md.tmpl +++ b/plan-tune/SKILL.md.tmpl @@ -272,6 +272,8 @@ Parse the JSON. Present in **plain English**, not raw floats: the inferred column next to declared: "**scope_appetite:** declared 0.8 (boil the ocean) ↔ observed 0.72 (close)" Use words for the gap: 0.0-0.1 "close", 0.1-0.3 "drift", 0.3+ "mismatch". + When `inferred.contributing_events[dim]` is below 3, show "not enough + observed answers yet" for that dimension instead of an observed value or gap word. This display gate is intentionally lower than the E1 **promotion gate** (90+ days stable across 3+ skills, per `docs/designs/PLAN_TUNING_V0.md`). @@ -422,6 +424,9 @@ Parse the JSON. For each dimension where both declared and inferred exist: Consider updating your declared value, or reflect on whether your behavior is actually what you want." +For each dimension in `low_evidence` (fewer than 3 observed answers moved it), +say "not enough observed answers yet" instead of a gap. + Never auto-update declared based on the gap. In v1 the gap is reporting only — the user decides whether declared is wrong or behavior is wrong. diff --git a/test/gstack-developer-profile.test.ts b/test/gstack-developer-profile.test.ts index df23a9c3b..cffaadebc 100644 --- a/test/gstack-developer-profile.test.ts +++ b/test/gstack-developer-profile.test.ts @@ -622,6 +622,98 @@ describe('gstack-developer-profile --check-mismatch', () => { expect(r.stdout).toContain('scope_appetite'); expect(r.stdout).toContain('autonomy'); }); + + const deriveWithOneSignal = () => { + logQuestion({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', question_summary: 'arch?', user_choice: 'fix-now', session_id: 's0' }); + for (let i = 1; i < 13; i++) { + logQuestion({ skill: 'plan-eng-review', question_id: `adhoc-unregistered-${i}`, question_summary: 'x', user_choice: 'anything', session_id: `s${i}` }); + } + expect(runDev('--derive').status).toBe(0); + const file = path.join(tmpHome, 'developer-profile.json'); + const p = readProfile() as any; + expect(p.inferred.contributing_events.scope_appetite).toBe(0); + p.declared = { scope_appetite: 0.9 }; + fs.writeFileSync(file, JSON.stringify(p)); + }; + + test('a declared dimension with no observed evidence is not flagged (#3436)', () => { + deriveWithOneSignal(); + const r = runDev('--check-mismatch'); + expect(r.status).toBe(0); + expect(r.stdout).not.toContain('scope_appetite: declared'); + expect(r.stdout).toContain('scope_appetite: no observed evidence'); + }); + + test('--gap leaves out a dimension with no observed evidence (#3436)', () => { + deriveWithOneSignal(); + const out = JSON.parse(runDev('--gap').stdout); + expect(out.gap.scope_appetite).toBeUndefined(); + expect(out.low_evidence).toEqual({ scope_appetite: 0 }); + }); + + test('a dimension with 1 or 2 contributing events reports too little evidence', () => { + runDev('--read'); + const file = path.join(tmpHome, 'developer-profile.json'); + const p = readProfile() as any; + p.declared = { scope_appetite: 0.9 }; + p.inferred.values.scope_appetite = 0.45; + p.inferred.sample_size = 10; + p.inferred.contributing_events = { scope_appetite: 2 }; + fs.writeFileSync(file, JSON.stringify(p)); + const r = runDev('--check-mismatch'); + expect(r.stdout).toContain('MISMATCH: none'); + expect(r.stdout).toContain('scope_appetite: too little evidence (2 of 3 events)'); + expect(JSON.parse(runDev('--gap').stdout).low_evidence).toEqual({ scope_appetite: 2 }); + }); + + test('a dimension with 3 contributing events is compared', () => { + runDev('--read'); + const file = path.join(tmpHome, 'developer-profile.json'); + const p = readProfile() as any; + p.declared = { scope_appetite: 0.9 }; + p.inferred.values.scope_appetite = 0.45; + p.inferred.sample_size = 10; + p.inferred.contributing_events = { scope_appetite: 3 }; + fs.writeFileSync(file, JSON.stringify(p)); + expect(runDev('--check-mismatch').stdout).toContain('scope_appetite: declared 0.9 vs inferred 0.45 (gap 0.45)'); + expect(JSON.parse(runDev('--gap').stdout).gap.scope_appetite).toBeCloseTo(0.45, 2); + }); + + test('three same-day answers that agree with the declaration are not a mismatch', () => { + for (let i = 0; i < 3; i++) { + logQuestion({ skill: 'plan-ceo-review', question_id: 'plan-ceo-review-mode', question_summary: 'mode?', user_choice: 'expand', session_id: `s${i}`, ts: '2026-10-20T12:00:00Z' }); + } + expect(runDev('--derive').status).toBe(0); + const file = path.join(tmpHome, 'developer-profile.json'); + const p = readProfile() as any; + expect(p.inferred.contributing_events.scope_appetite).toBe(3); + p.declared = { scope_appetite: 0.9 }; + fs.writeFileSync(file, JSON.stringify(p)); + const r = runDev('--check-mismatch'); + expect(r.stdout).toContain('not enough data'); + expect(r.stdout).not.toContain('scope_appetite: declared'); + }); + + test('derive stores the gap for a dimension with 3 contributing events', () => { + for (let i = 0; i < 3; i++) { + logQuestion({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', question_summary: 'arch?', user_choice: 'fix-now', session_id: `s${i}`, ts: `2026-10-2${i}T12:00:00Z` }); + } + expect(runDev('--derive').status).toBe(0); + const file = path.join(tmpHome, 'developer-profile.json'); + const p = readProfile() as any; + expect(p.inferred.contributing_events.architecture_care).toBe(3); + p.declared = { architecture_care: 0.1 }; + fs.writeFileSync(file, JSON.stringify(p)); + expect(runDev('--derive').status).toBe(0); + expect((readProfile() as any).gap.architecture_care).toBeGreaterThan(0.3); + }); + + test('derive stores no gap for a dimension without evidence', () => { + deriveWithOneSignal(); + expect(runDev('--derive').status).toBe(0); + const p = readProfile() as any; + expect(p.gap.scope_appetite).toBeUndefined(); + }); }); // ----------------------------------------------------------------------- From 0542230e632b5e6b2d776aadbe21fb4b8109d0d5 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Mon, 2 Nov 2026 05:28:00 -0800 Subject: [PATCH 09/19] test(fixtures): leak scan skips Capy's git shim log in fixture homes (#3860) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 8 ++++++++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- package.json | 2 +- test/git-fixture-template.test.ts | 17 +++++++++++++++++ test/helpers/git-fixture-template.ts | 8 ++++++-- 6 files changed, 34 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1fc172340..f32c13f1e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,13 @@ # Changelog +## [1.92.80.0] - 2026-11-02 + +**On a Capy machine, the git fixture tests no longer fail locally because of Capy's git shim log.** Capy's `/usr/bin/git` shim appends to `$HOME/.capy/git-shim.log`, so each fixture home that ran git during its template build ended up with a log naming the template. The leak scan in `test/helpers/git-fixture-template.ts` flagged it, and 34 tests in `test/train-backstop.test.ts`, `test/restamp-mentions.test.ts` and `test/pregate-user-policy.test.ts` failed on every Capy lane, though they passed on the remote gate. + +### Tests +- `templateReferences` skips `.capy/git-shim.log`. Nothing reads that file back, and every other file is still scanned. +- `test/git-fixture-template.test.ts` plants the shim log in a template's `home/.capy/` (the copy succeeds), and plants `home/.capy/state.json` as a control (the copy still throws). + ## [1.92.79.0] - 2026-11-02 **/plan-tune no longer reports a mismatch between your stated and observed preferences when it has no observed answers behind it (#3436).** `--check-mismatch` compared a declared value against an inferred value that had never moved from its 0.5 starting point, so a strong declaration with no matching answers printed `scope_appetite: declared 0.9 vs inferred 0.5 (gap 0.4)` even though no logged answer touched that dimension. diff --git a/VERSION b/VERSION index dfb847afe..f17ba7e50 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.92.79.0 +1.92.80.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index 192027e8e..835c2c05f 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.92.79.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.92.80.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/package.json b/package.json index b8a73016c..2da268dcb 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.92.79", + "version": "1.92.80", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/test/git-fixture-template.test.ts b/test/git-fixture-template.test.ts index 76375466c..f63a6c002 100644 --- a/test/git-fixture-template.test.ts +++ b/test/git-fixture-template.test.ts @@ -71,6 +71,23 @@ describe('git fixture template', () => { expect(() => leaky.copy()).toThrow(/still references its template [^\n]*\n.*config/); }); + test("Capy's git shim log in a fixture home (home/.capy/git-shim.log) is not a leak; any other file naming the template still is", () => { + const plant = (rel: string) => gitFixtureTemplate({ prefix: 'gstack-fixture-template-shim-', build: (root) => { + const meta = build(root); + fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true }); + fs.writeFileSync(path.join(root, rel), '# capy git shim v2: ts cwd ppid parent verb\n2026-11-02T02:15:30Z\t' + path.join(root, 'work') + '\t1\tbun\tcat-file\n'); + return meta; + }, rebind: (copy, tpl) => rebindGitDir(path.join(copy, 'work', '.git'), tpl, copy) }); + const shim = plant(path.join('home', '.capy', 'git-shim.log')); + const other = plant(path.join('home', '.capy', 'state.json')); + try { + const c = shim.copy(); + roots.push(c.root); + expect(git(path.join(c.root, 'work'), 'remote', 'get-url', 'origin')).toBe(path.join(c.root, 'origin.git')); + expect(() => other.copy()).toThrow(/still references its template [^\n]*\n.*state\.json/); + } finally { shim.dispose(); other.dispose(); } + }); + test('native, forward-slash, backslash-escaped and Git Bash spellings are each rewritten, and each one left behind is found', () => { const to = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-fixture-spellings-')); roots.push(to); diff --git a/test/helpers/git-fixture-template.ts b/test/helpers/git-fixture-template.ts index a646747ab..5fb9f72eb 100644 --- a/test/helpers/git-fixture-template.ts +++ b/test/helpers/git-fixture-template.ts @@ -52,7 +52,11 @@ export function rebindGitDir(gitDir: string, from: string, to: string): void { } } -/** Files and symlinks under dir (and the template's meta value) that still name templateRoot; zlib-compressed object storage is skipped. */ +/** + * Files and symlinks under dir (and the template's meta value) that still name templateRoot. Skipped: zlib-compressed object + * storage, and the log Capy's /usr/bin/git shim appends to in whatever HOME it runs under (.capy/git-shim.log), which names + * the template whenever a fixture home ran git during build; nothing reads it back. + */ export function templateReferences(dir: string, templateRoot: string, meta?: unknown): string[] { const needles = pathSpellings(templateRoot).map(s => Buffer.from(s)); const names = (buf: Buffer) => needles.some(n => buf.includes(n)); @@ -62,7 +66,7 @@ export function templateReferences(dir: string, templateRoot: string, meta?: unk const p = path.join(d, entry.name); if (entry.isSymbolicLink()) { if (names(Buffer.from(fs.readlinkSync(p)))) hits.push(p); } else if (entry.isDirectory()) { if (!(path.basename(d) === 'objects' && /^(pack|[0-9a-f]{2})$/.test(entry.name))) walk(p); } - else if (entry.isFile() && names(fs.readFileSync(p))) hits.push(p); + else if (entry.isFile() && !(entry.name === 'git-shim.log' && path.basename(d) === '.capy') && names(fs.readFileSync(p))) hits.push(p); } }; walk(dir); From 1022f4fc51a4771e3838ce6bbf7c19b7e94e67fe Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Mon, 2 Nov 2026 05:38:02 -0800 Subject: [PATCH 10/19] v1.92.81.0 feat(ship-release): finish accepts the owner's edit after a fix-check (GST-965) (#3861) * feat(ship-release): finish accepts the owner's edit after a fix-check (GST-965) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship-release): fix_check.files narrows the accepted scope; gate-edited and over-cap paths stay behavior (GST-965) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(review-policy): the gate-edited and over-cap exclusions hold with --reviewed-base (GST-965) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship-release): a gate-edited test is never owner-accepted; an empty fix_check.files stops (GST-965) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(review-policy): the over-cap exclusion holds with --reviewed-base (GST-965 fix-check note) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 10 ++++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- docs/review-policy.md | 9 +++- lib/ship-release.ts | 57 ++++++++++++++++------ package.json | 2 +- test/ship-rereview-triggers.test.ts | 76 +++++++++++++++++++++++++++++ 7 files changed, 140 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f32c13f1e..209155077 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,15 @@ # Changelog +## [1.92.81.0] - 2026-11-02 + +**`gstack-ship-release finish` now accepts the owner's edit after a fix-check instead of asking for another review round (GST-965).** On gbrain #8425, the fix-check found a blocker, the owner's pre-agreed fallback was applied and "then ship" was granted. Finish still classed the edit as behavior and stopped with `insert 5–11.5 → 12–14 → 16`, a third round the R51 cap forbids, so /ship could not get through Step 16 without running its stages by hand. + +### Added +- **`finish --review-loop-exception `** (`lib/ship-release.ts`) takes the same record `gstack-ship-receipt write --review-loop-exception` takes. Paths changed since `fix_check.head` that the branch already touched at that commit (narrowed to the record's `fix_check.files` when it lists them) route as `accepted=` on the `ROUTE:` line, a `REVIEW_LOOP_EXCEPTION:` line names them with the owner, date and channel, and the docs stage carries them (`accepted=` in the DOCS line). A path outside that scope and a gate-edited test (with or without `--reviewed-base`) still route as behavior, and with `--reviewed-base` so does a file too large for the scoped review. A malformed record, or a `fix_check.head` that is not HEAD or an ancestor of it, stops before the release commit. Without the flag nothing changes. + +### Tests +- `test/ship-rereview-triggers.test.ts`: an owner fallback edit to a source file and a doc after the fix-check stops without the record and passes Step 16 with it; a new file outside the scope (even when `fix_check.files` lists it), a fix-check head off the branch, a malformed record, an unreadable record file and an empty `fix_check.files` stop; a `fix_check.files` list narrows what is accepted; a test weakened after the fix-check stops with and without `--reviewed-base`. + ## [1.92.80.0] - 2026-11-02 **On a Capy machine, the git fixture tests no longer fail locally because of Capy's git shim log.** Capy's `/usr/bin/git` shim appends to `$HOME/.capy/git-shim.log`, so each fixture home that ran git during its template build ended up with a log naming the template. The leak scan in `test/helpers/git-fixture-template.ts` flagged it, and 34 tests in `test/train-backstop.test.ts`, `test/restamp-mentions.test.ts` and `test/pregate-user-policy.test.ts` failed on every Capy lane, though they passed on the remote gate. diff --git a/VERSION b/VERSION index f17ba7e50..9a1e28b7f 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.92.80.0 +1.92.81.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index 835c2c05f..45e52d15a 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.92.80.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.92.81.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/docs/review-policy.md b/docs/review-policy.md index db10c3753..16e93e891 100644 --- a/docs/review-policy.md +++ b/docs/review-policy.md @@ -429,7 +429,14 @@ rules hash, base and registry commits, the gate scan bound to the head and the r the queue refuses a receipt below the depth its head needs ([ship-policy.md#ship-receipt](ship-policy.md#ship-receipt)). A stop the owner accepted after a fix-check edit goes in `review_loop_exception`, never in prose -under `REVIEW_LOOP:`. +under `REVIEW_LOOP:`. Step 16 takes the same record: +`gstack-ship-release finish --review-loop-exception ` routes the +paths the branch touched at `fix_check.head` (narrowed to the record's +`fix_check.files` when it lists them) and changed since as `accepted=` in +`ROUTE:`, prints them on a `REVIEW_LOOP_EXCEPTION:` line, and carries them +through the docs stage. A path outside that scope and a gate-edited test still +route as behavior, with `--reviewed-base` (as /ship passes it) so does a file +too large for the scoped review, and without the record nothing changes. The receipt's summary line prints `passes= blocking_found= deferred=`, which is what a scorecard tracks (target: a median of 1 pass per PR). A diff --git a/lib/ship-release.ts b/lib/ship-release.ts index bdce14414..02f5218a7 100644 --- a/lib/ship-release.ts +++ b/lib/ship-release.ts @@ -15,6 +15,7 @@ import { mergeChangelogs, normalizeChangelog } from './changelog-check'; import { versionAtRef } from './restamp'; import { classifyPath } from './review-depth'; import { comparePath, contextPacket, gateEditedPaths, isTestCode, PACKET_SECTION_CAP, readScopedReview, touchedPaths, type PatchSides } from './ship-patch'; +import { parseReviewLoopException, type ReviewLoopException } from './ship-receipt-depth'; import { parseReceiptDocs, type ReceiptDocs } from './ship-receipt-docs'; import { resolveStateRoot } from './state-root'; import { bumpVersion, cmpVersion, fmtVersion, parseVersion, versionWidth, type Bump } from './version-source'; @@ -27,10 +28,11 @@ export const RELEASE_HELP = `gstack-ship-release — /ship Steps 12 and 15-17 (s --keep-current (or --version ) keeps VERSION; under bump ask a session no person answers needs --answer before a bump is written finish --base --reviewed-tree [--reviewed-base ] --docs-post [--lane