From 225a39b312356d5cacc24a7b473d3888c99ef4f9 Mon Sep 17 00:00:00 2001 From: garrytan Date: Tue, 3 Nov 2026 01:53:33 +0000 Subject: [PATCH] fix(review): review_exclude_paths keeps native binaries, NUL-bearing code and attribute-binary secret lines; exact --review-exclude docs (GST-972) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- bin/gstack-ship-receipt | 4 +- .../gst-972-review-exclude-follow-ups.md | 7 ++ docs/ship-policy.md | 14 ++-- docs/troubleshooting.md | 6 +- lib/review-exclude.ts | 53 ++++++++++--- lib/ship-receipt-depth.ts | 5 +- test/review-exclude.test.ts | 79 ++++++++++++++++++- 7 files changed, 145 insertions(+), 23 deletions(-) create mode 100644 changelog.d/gst-972-review-exclude-follow-ups.md diff --git a/bin/gstack-ship-receipt b/bin/gstack-ship-receipt index 9e417faa9..866e77ec8 100755 --- a/bin/gstack-ship-receipt +++ b/bin/gstack-ship-receipt @@ -98,7 +98,9 @@ write flags: --pr --head --base --version --gated-tree --predecessor --gate (the digest= on gstack-review-dispatch verify's REVIEW_EXCLUDE_RECORD: line, which the reviewers' briefs name; none without that line): must equal the digest HEAD has under the base policy's review_exclude_paths, else - RECEIPT_INVALID field=review_exclude and nothing is written (rerun the review; write alone cannot clear it); recorded as review_exclude. Carry and the queue compare that digest. + RECEIPT_INVALID field=review_exclude and nothing is written; recorded as review_exclude. Carry and the queue compare that digest. + The check binds the digest to the base policy, not to a review: write does not read the review ledger, so the expected= + digest a refusal prints passes too. After a refusal, rerun the review and pass the digest its verify prints. --review-targeted ({classes, rules, brief_sha, status, findings, raised_by, head}: gstack-review-dispatch verify --targeted-out); recorded as review_targeted bound to the receipt's product_patch_id, and refused (RECEIPT_INVALID) when the commit the reader read (head) is not an ancestor or a change since it carries a targeted class (gate scan included). write and carry diff --git a/changelog.d/gst-972-review-exclude-follow-ups.md b/changelog.d/gst-972-review-exclude-follow-ups.md new file mode 100644 index 000000000..279d57153 --- /dev/null +++ b/changelog.d/gst-972-review-exclude-follow-ups.md @@ -0,0 +1,7 @@ +**`review_exclude_paths` keeps code it used to exclude, and the docs say exactly what the receipt check stops (GST-972).** Three ways to get code or a credential out of review under a base pattern are closed, each with a failing-first test in `test/review-exclude.test.ts`, and the receipt docs no longer over-claim. + +### Fixed +- **Native binaries are code.** A `.node`, `.so`, `.dylib` or `.dll` file under a base pattern was excluded as binary content, and so was a compiled executable with a data extension. A binary file now starting with ELF, Mach-O, PE, WebAssembly or `#!` magic is kept as `code` whatever its extension. +- **A NUL byte no longer makes code excludable.** A binary file was content unless its extension was on a short source list, so `.gradle`, `.sql`, `.tf`, `.groovy`, a `.p12` key store or an extensionless private key with a NUL byte was excluded. A binary file is now content only under a known content extension (images, fonts, media, documents, compressed archives, `.bin`, `.dat` and data blobs); every other binary file is kept as `code`. +- **A branch's `.gitattributes` cannot hide a credential line.** Marking `receipts/**` binary left the collector's added lines empty, so the secret-line keep rule saw nothing and the file was excluded. For a file git calls binary only because of its attributes, the rule now reads `git diff --text --no-textconv --no-ext-diff`. +- **`gstack-ship-receipt write --review-exclude` docs are exact.** The help, `docs/troubleshooting.md`, `docs/ship-policy.md` and the queue's refusal line said a receipt write alone cannot clear `field=review_exclude`. Write checks the digest against the base policy and does not read the review ledger, so a write given the refusal's `expected=` digest passes without a review. The text now says that, and that the review rerun is the fix. diff --git a/docs/ship-policy.md b/docs/ship-policy.md index d76edb603..551153b24 100644 --- a/docs/ship-policy.md +++ b/docs/ship-policy.md @@ -1287,8 +1287,8 @@ A pattern that matches no changed file is listed as `unmatched=` on the plan lin **What is excluded, and what is kept.** A matched file is excluded only when it is content: a data file (JSON, JSONL, CSV, TSV, images), a docs-class file, a -`.log` file, or a file git reports as binary (`-\t-` in `--numstat`; readers would only see `Binary files differ`) whose bytes hold a NUL in their first 8,000, so a branch's `.gitattributes` cannot make a text file binary content. A binary file is content even under -`config/` or in a skill directory; only `policy-dir`, `gate-path`, `executable`, `link`, `gate-tagged`, or a source or script extension (`code`) keep it. +`.log` file, or a file git reports as binary (`-\t-` in `--numstat`; readers would only see `Binary files differ`) whose bytes hold a NUL in their first 8,000, so a branch's `.gitattributes` cannot make a text file binary content. A binary file is content only under a known content extension (images, fonts, media, documents, compressed archives, `.bin`, `.dat`, and data blobs such as `.parquet`, `.npy` or `.sqlite`) and only when its bytes do not start with an executable image's magic (ELF, Mach-O, PE, WebAssembly, `#!`). Under those rules it is content even under +`config/` or in a skill directory, and only `policy-dir`, `gate-path`, `executable`, `link` or `gate-tagged` keep it. Every other binary file is `code`: source with a NUL, a native library (`.node`, `.so`, `.dylib`, `.dll`), a compiled executable, a key store, a pickle, or a file with no extension. Everything else stays in every reader's diff with one reason code: | Reason | Kept because | @@ -1300,8 +1300,8 @@ Everything else stays in every reader's diff with one reason code: | `agent-prose` | it is a skill, section, resolver, template or rule file | | `config` | it is a data-shaped config file (`package.json`, `tsconfig*.json`, `config/`, dotfiles) | | `gate-tagged` | the gate scan tagged one of its hunks | -| `code` | it is code or a test, not content; a binary file with a source or script extension (`.js`, `.py`, `.sh`, `.wasm`) counts as code | -| `secret-line` | an added line reads a credential (the targeted pass's secret rule) | +| `code` | it is code or a test, not content; a binary file outside the content extensions above, or one with executable magic, counts as code | +| `secret-line` | an added line reads a credential (the targeted pass's secret rule); for a file git calls binary only because of its attributes, the added lines come from `git diff --text`, which `.gitattributes` cannot change | | `named-by-code` | a code or test file that stays in review (not docs, a data file, a `.log` or binary, and not itself excluded) names its full repository path, at the head or the merge base | | `consumer-scan-unavailable` | the `git grep` for those names failed, so every candidate is kept; also a live binary file whose bytes could not be read | | `gate-scan-unavailable` | the gate scan failed or did not run at the head, so no hunk's tag is known and every candidate is kept | @@ -1355,7 +1355,11 @@ and binds. A different digest means the reviewers read another selection: write refuses with `RECEIPT_INVALID field=review_exclude`, carry with `RECEIPT_CARRY_REFUSED review_exclude changed`, and the queue with `RECEIPT_INVALID field=review_exclude expected= recorded=`. The -fix is to rerun the review, never a receipt write alone. +fix is to rerun the review and write with the digest its `verify` prints. Write +checks the digest against the base policy, so a stale selection cannot be +recorded by accident, but it does not read the review ledger: a write given the +`expected=` digest passes without a review, and only the review run shows that +the reviewers read that selection. **Adopting it.** (1) Open a PR on the base that adds the patterns and `min_gstack_version` at least `REVIEW_EXCLUDE_KEY_VERSION`, and ship it like any diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 42c2a96ce..8f1d5aa9f 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -3340,8 +3340,10 @@ use this code when the receipt's review_exclude digest (or the base policy's [`review_exclude_paths`](ship-policy.md#review-exclude-paths). The line names `expected=` (now), `recorded=` (what the review read, `none` when it read without exclusion) and `source=` (the base commit). The reviewers read another -selection than the one the head now has, so a receipt write alone cannot clear -it: rerun `/ship`'s review on the head (`gstack-review-dispatch pass --base `), +selection than the one the head now has. Write checks the digest against the base +policy only: it does not read the review ledger, so a write that passes the +`expected=` digest clears the field without a review. Do not do that. Rerun +`/ship`'s review on the head (`gstack-review-dispatch pass --base `), run `gstack-review-dispatch verify --ledger `, which prints `REVIEW_EXCLUDE_RECORD: digest=` from the briefs the readers got, then `gstack-ship-receipt write --pr --review-exclude --body `. diff --git a/lib/review-exclude.ts b/lib/review-exclude.ts index 80533bc7d..7f4c48d21 100644 --- a/lib/review-exclude.ts +++ b/lib/review-exclude.ts @@ -122,20 +122,25 @@ function select(i: ExcludeInput, patterns: readonly string[]): { byPattern: Map< return { byPattern, files }; } +/** Executable image magic: ELF, Mach-O (thin and universal; 0xcafebabe is also a Java class), PE, WebAssembly and a script's #!. */ +const NATIVE_MAGIC = [[0x7f, 0x45, 0x4c, 0x46], [0xfe, 0xed, 0xfa, 0xce], [0xfe, 0xed, 0xfa, 0xcf], [0xce, 0xfa, 0xed, 0xfe], [0xcf, 0xfa, 0xed, 0xfe], [0xca, 0xfe, 0xba, 0xbe], [0x4d, 0x5a], [0x00, 0x61, 0x73, 0x6d], [0x23, 0x21]]; +const nativeImage = (b: Buffer) => NATIVE_MAGIC.some(m => m.every((x, k) => b[k] === x)); + /** * The matched paths git calls binary whose bytes hold a NUL in their first 8,000 (git's own test), read from the head blob, * the file on a live tree, or the old blob for a deletion. A branch's .gitattributes can make git call a text file binary; - * only real bytes make it binary content here. + * only real bytes make it binary content here. A binary file that starts with executable image magic also goes in `native`. */ -function binaryContent(i: ExcludeInput, files: ReadonlyMap, claimed: ReadonlySet, unread: Set): Set { +function binaryContent(i: ExcludeInput, files: ReadonlyMap, claimed: ReadonlySet, unread: Set, native: Set): Set { const out = new Set(); + const sniff = (p: string, b: Buffer) => { if (!b.includes(0)) return; out.add(p); if (nativeImage(b)) native.add(p); }; const real = (sha: string) => /^[0-9a-f]{40,64}$/.test(sha) && !/^0+$/.test(sha); const bySha = new Map(); for (const p of claimed) { const m = files.get(p); if (!m) continue; if (!i.head && m.newMode !== '000000') { - try { const fd = fs.openSync(path.join(i.root, p), 'r'); const buf = Buffer.alloc(8000); const n = fs.readSync(fd, buf, 0, 8000, 0); fs.closeSync(fd); if (buf.subarray(0, n).includes(0)) out.add(p); } catch { unread.add(p); } + try { const fd = fs.openSync(path.join(i.root, p), 'r'); const buf = Buffer.alloc(8000); const n = fs.readSync(fd, buf, 0, 8000, 0); fs.closeSync(fd); sniff(p, buf.subarray(0, n)); } catch { unread.add(p); } continue; } const sha = real(m.newSha) ? m.newSha : real(m.oldSha) ? m.oldSha : ''; @@ -144,8 +149,25 @@ function binaryContent(i: ExcludeInput, files: ReadonlyMap, cla for (const [sha, paths] of bySha) { const r = spawnSync('git', ['cat-file', 'blob', sha], { cwd: i.root, timeout: 30_000, maxBuffer: 8000 }); const read = r.status === 0 || (r.error as NodeJS.ErrnoException | undefined)?.code === 'ENOBUFS'; - const binary = read && (r.stdout ?? Buffer.alloc(0)).subarray(0, 8000).includes(0); - for (const p of paths) if (!read) unread.add(p); else if (binary) out.add(p); + const head = (r.stdout ?? Buffer.alloc(0)).subarray(0, 8000); + for (const p of paths) if (!read) unread.add(p); else sniff(p, head); + } + return out; +} + +/** + * The added lines of matched files git calls binary only because of their attributes (the bytes hold no NUL): the + * collector's added lines are empty for them, so the secret-line rule reads `git diff --text` instead, which no + * .gitattributes, textconv or external diff driver changes. A git failure puts the path in `unread`. + */ +function attributeBinaryLines(i: ExcludeInput, paths: readonly string[], unread: Set): Record { + const out: Record = {}; + const range = i.head ? [i.mergeBase, i.head] : [i.mergeBase]; + for (const p of paths) { + const r = git(i.root, ['diff', '--text', '--no-textconv', '--no-ext-diff', '--no-renames', '-U0', ...range, '--', literal(p)]); + if (!r.ok) { unread.add(p); continue; } + let hunk = false; + out[p] = r.out.split('\n').filter(l => (hunk ||= l.startsWith('@@')) && l.startsWith('+')).map(l => l.slice(1)); } return out; } @@ -159,17 +181,21 @@ function contentKind(i: ExcludeInput, p: string, binary: ReadonlySet): C return classifyPath(p, i.classRoot) === 'docs' ? 'docs' : null; } -/** Source and script extensions: a NUL in one does not make it excludable binary content. */ -const CODE_EXT = /\.(?:[cm]?[jt]sx?|py|rb|go|rs|java|kts?|scala|swift|exs?|php|cs|c|h|cc|cpp|hpp|mm?|dart|pl|pm|lua|sh|bash|zsh|fish|ps1|bat|cmd|vue|svelte|html?|wasm)$/i; +/** + * The only extensions a binary file may be excludable content under: images, fonts, media, documents, compressed archives + * and data blobs. Any other binary (source with a NUL, a native library or executable, a key store, a pickle, no + * extension) is code; so is one that starts with executable image magic whatever its extension. + */ +const BINARY_CONTENT_EXT = /\.(?:png|jpe?g|gif|webp|bmp|ico|avif|tiff?|heic|heif|woff2?|ttf|otf|eot|mp3|mp4|m4a|wav|ogg|oga|ogv|webm|mov|avi|mkv|flac|aac|opus|pdf|docx|xlsx|pptx|odt|ods|odp|gz|tgz|zip|tar|bz2|xz|zst|7z|bin|dat|parquet|arrow|feather|avro|orc|npy|npz|h5|hdf5|sqlite3?|db|safetensors)$/i; -/** Why a matched file is kept by its path, mode and lines alone; null for content the policy may exclude. A binary file readers would only see as "Binary files differ" is content unless its directory, mode or gate tags keep it. */ -function pathReason(i: ExcludeInput, p: string, m: Matched, binary: ReadonlySet): KeptReason | null { +/** Why a matched file is kept by its path, mode and lines alone; null for content the policy may exclude. A binary file readers would only see as "Binary files differ" is content unless its directory, mode, gate tags, extension or magic keep it. */ +function pathReason(i: ExcludeInput, p: string, m: Matched, binary: ReadonlySet, native: ReadonlySet): KeptReason | null { if (/^\.(gstack|github)\//.test(p)) return 'policy-dir'; if (isGatePath(p)) return 'gate-path'; if (m.oldMode === '100755' || m.newMode === '100755') return 'executable'; if ([m.oldMode, m.newMode].some(x => x === '120000' || x === '160000')) return 'link'; if (i.gateTagged?.has(p)) return 'gate-tagged'; - if (binary.has(p)) return CODE_EXT.test(p) ? 'code' : null; + if (binary.has(p)) return BINARY_CONTENT_EXT.test(p) && !native.has(p) ? null : 'code'; if (isAgentProse(p)) return 'agent-prose'; if (isDataConfig(p)) return 'config'; if (contentKind(i, p, binary) === null) return 'code'; @@ -227,9 +253,12 @@ export function reviewExclude(i: ExcludeInput): Exclusion { const sel = select(i, patterns); if ('error' in sel) return { state: 'refused', code: 'REVIEW_DIFF_UNREADABLE', lines: [`${i.context}: git could not list the files review_exclude_paths matches (${sel.error}); fix: git fetch origin ${base}, then rerun once; nothing was planned (REVIEW_DIFF_UNREADABLE)`] }; const unread = new Set(); - const binary = binaryContent(i, sel.files, new Set(i.perPath.filter(e => e.binary && sel.files.has(e.path)).map(e => e.path)), unread); + const native = new Set(); + const claimed = i.perPath.filter(e => e.binary && sel.files.has(e.path)).map(e => e.path); + const binary = binaryContent(i, sel.files, new Set(claimed), unread, native); + const text = { ...i, added: { ...i.added, ...attributeBinaryLines(i, claimed.filter(p => !binary.has(p) && !unread.has(p)), unread) } }; const reasons = new Map(); - for (const [p, m] of sel.files) reasons.set(p, unread.has(p) ? 'consumer-scan-unavailable' : pathReason(i, p, m, binary)); + for (const [p, m] of sel.files) reasons.set(p, unread.has(p) ? 'consumer-scan-unavailable' : pathReason(text, p, m, binary, native)); const candidates = new Set([...reasons].filter(([, r]) => r === null).map(([p]) => p)); const named = grepFixed(i, [...candidates].filter(p => !p.includes('\n')), candidates, [i.mergeBase]); for (const p of candidates) { diff --git a/lib/ship-receipt-depth.ts b/lib/ship-receipt-depth.ts index 7cd524523..b48ebdd90 100644 --- a/lib/ship-receipt-depth.ts +++ b/lib/ship-receipt-depth.ts @@ -224,7 +224,8 @@ function headInputs(root: string, baseRef: string, head: string, r?: ShipReceipt /** * The queue's exclusion check: the digest the receipt recorded (none when absent) against the one `head` has under the current * base policy. Only the digest counts (the policy's patterns and the excluded list), so a base merge or a moved base that leaves - * both unchanged binds; a changed selection needs the review rerun, which receipt write alone cannot stand in for. + * both unchanged binds; a changed selection needs the review rerun (write checks the same digest and does not read the + * review ledger). */ export function excludeMismatch(x: Exclusion, r: ShipReceipt, head: string, base: string): string | null { const expected = x.state === 'active' ? x.record.digest : 'none'; @@ -232,7 +233,7 @@ export function excludeMismatch(x: Exclusion, r: ShipReceipt, head: string, base if (expected === recorded) return null; const source = x.state === 'active' ? x.record.source : r.review_exclude?.source ?? base; const h = head.slice(0, 12); - return `field=review_exclude expected=${expected} recorded=${recorded} source=${source}: the reviewers read another selection than ${base}'s review_exclude_paths gives ${h}; the receipt is unchanged; fix: rerun /ship's review on ${h} (gstack-review-dispatch pass --base ${base.replace(/^(refs\/remotes\/)?origin\//, '')}), then gstack-ship-receipt write --pr --review-exclude --body with the digest gstack-review-dispatch verify --ledger prints on REVIEW_EXCLUDE_RECORD:; a receipt write alone does not clear this`; + return `field=review_exclude expected=${expected} recorded=${recorded} source=${source}: the reviewers read another selection than ${base}'s review_exclude_paths gives ${h}; the receipt is unchanged; fix: rerun /ship's review on ${h} (gstack-review-dispatch pass --base ${base.replace(/^(refs\/remotes\/)?origin\//, '')}), then gstack-ship-receipt write --pr --review-exclude --body with the digest gstack-review-dispatch verify --ledger prints on REVIEW_EXCLUDE_RECORD: (write checks the digest against the base policy and does not read the review ledger, so only the rerun shows the reviewers read this selection)`; } /** The receipt's recorded gate scan when it names this head; otherwise null (unavailable). */ const boundScan = (r: ShipReceipt, head: string): number | null => { diff --git a/test/review-exclude.test.ts b/test/review-exclude.test.ts index aefa18c08..de38365b1 100644 --- a/test/review-exclude.test.ts +++ b/test/review-exclude.test.ts @@ -11,7 +11,7 @@ import { EXIT } from '../lib/headless-artifacts'; import { KEEP_ARG_MAX_BYTES, readSelection, reviewExclude, tooLargeLine, unreadableLine } from '../lib/review-exclude'; import { collectCommit, collectLive } from '../lib/review-collect'; import { extractReceipt } from '../lib/ship-receipt'; -import { bindShippedReceipt, receiptReviewAtHead } from '../lib/ship-receipt-depth'; +import { bindShippedReceipt, excludeMismatch, receiptReviewAtHead } from '../lib/ship-receipt-depth'; import { REVIEW_EXCLUDE_KEY_VERSION, isUnavailable } from '../lib/ship-policy'; import { loadPolicyAt } from '../lib/ship-policy-at'; import { cleanup, commit, makeFixtureRepo, runBin, write, type FixtureRepo } from './helpers/pregate-fixture'; @@ -313,6 +313,31 @@ describe('the receipt binds the selection the reviewers read', () => { expect(removed.status).toBe(EXIT.fail); expect(removed.err).toContain(`field=review_exclude expected=none recorded=${digest}`); }, 240_000); + + // Value: protects=every place that describes write --review-exclude says what it checks: the digest HEAD has under the base + // policy, not that a review read that selection; fails_when=the help, troubleshooting, ship-policy or the queue's refusal + // line claims a receipt write alone cannot clear field=review_exclude while a write with the refusal's own expected= digest + // clears it with no review run; why_new=GST-972 item 2 (the #3852 cross-family trail review); seam=gstack-ship-receipt write + // and --help, excludeMismatch, the two docs; no_claim=binding the digest to the review ledger (write never reads it) + test('write binds --review-exclude to the base policy, not to a review, and no description claims more', () => { + const repo = census(); + const body = path.join(repo.root, 'body.md'); + const refused = writeReceipt(repo, body, []); + const expected = refused.err.match(/field=review_exclude expected=([0-9a-f]{64}) /)?.[1]; + expect(expected).toBeDefined(); + expect(writeReceipt(repo, body, ['--review-exclude', expected!, ...targetedAtHead(repo)]).status).toBe(EXIT.ok); + const doc = (rel: string) => fs.readFileSync(path.join(import.meta.dir, '..', rel), 'utf8').replace(/\s+/g, ' '); + const claims = { + help: runBin(repo, 'gstack-ship-receipt', ['--help']).out.replace(/\s+/g, ' '), + troubleshooting: doc('docs/troubleshooting.md'), + shipPolicy: doc('docs/ship-policy.md'), + queue: excludeMismatch({ state: 'none' }, { review_exclude: { digest: expected } } as any, repo.git('rev-parse', 'HEAD'), 'origin/main')!, + }; + for (const [where, text] of Object.entries(claims)) { + expect({ where, overClaim: /write alone (cannot|does not) clear|never a receipt write alone/i.test(text) }).toEqual({ where, overClaim: false }); + expect({ where, says: text.includes('does not read the review ledger') }).toEqual({ where, says: true }); + } + }, 240_000); }); describe('a branch cannot hide code under a base pattern', () => { @@ -348,6 +373,58 @@ describe('a branch cannot hide code under a base pattern', () => { expect(p).toMatchObject({ depth: 'full', errors: expect.arrayContaining([expect.objectContaining({ code: 'binary_code_path', input: 'assets/loader.js' })]) }); }, 120_000); + // Value: protects=a compiled binary under a base pattern stays in review, because it is code; fails_when=a .node, .so, .so.N, + // .dylib or .dll file, or an ELF, Mach-O or PE image under a content extension (.bin, .dat), is excluded as binary content; + // why_new=GST-972 item 3; seam=bin/gstack-review-plan; no_claim=what a reader can see inside a binary + test('native binaries are code: .node, .so, .dylib, .dll and executable images under a content extension stay kept', () => { + const repo = makeFixtureRepo({ prefix: 'gstack-exclude-', baseFiles: { '.gstack/ship-policy.json': policyText(['assets/**']) } }); repos.push(repo); + fs.mkdirSync(path.join(repo.work, 'assets'), { recursive: true }); + const elf = Buffer.concat([Buffer.from([0x7f, 0x45, 0x4c, 0x46, 2, 1, 1, 0]), Buffer.alloc(24)]); + const macho = Buffer.concat([Buffer.from([0xcf, 0xfa, 0xed, 0xfe]), Buffer.alloc(28)]); + const pe = Buffer.concat([Buffer.from('MZ'), Buffer.alloc(30)]); + const files = { 'assets/addon.node': elf, 'assets/libx.dylib': macho, 'assets/libx.so': elf, 'assets/libx.so.1': elf, 'assets/setup.dat': pe, 'assets/tool.bin': elf, 'assets/x.dll': pe, 'assets/real.gz': GZ }; + for (const [rel, bytes] of Object.entries(files)) fs.writeFileSync(path.join(repo.work, rel), bytes); + commit(repo, {}); + const p = JSON.parse(plan(repo, ['--json']).out); + expect(p.review_exclude.kept_files).toEqual(Object.keys(files).filter(f => f !== 'assets/real.gz').map(f => ({ path: f, reason: 'code' }))); + expect(p.review_exclude).toMatchObject({ excluded: 1, content: { binary: 1 } }); + }, 120_000); + + // Value: protects=a NUL byte cannot turn code or a key file into excludable content; fails_when=a NUL-bearing file with a + // source extension outside the old CODE_EXT list (.gradle, .sql, .tf, .groovy), a key store (.p12) or an extensionless key + // is excluded as binary; why_new=GST-972 item 6 (#3852 targeted gate reader at 61803d9); seam=bin/gstack-review-plan; + // no_claim=the list of binary content extensions is complete (an unknown one is kept, never excluded) + test('a NUL-bearing file is binary content only under a known content extension; code and key files stay kept', () => { + const repo = makeFixtureRepo({ prefix: 'gstack-exclude-', baseFiles: { '.gstack/ship-policy.json': policyText(['assets/**']) } }); repos.push(repo); + fs.mkdirSync(path.join(repo.work, 'assets'), { recursive: true }); + const nul = (code: string) => Buffer.from(`/* \0 */ ${code}\n`); + const kept = { 'assets/Build.groovy': nul('println "x"'), 'assets/build.gradle': nul('apply plugin: "java"'), 'assets/id_ed25519': nul('-----BEGIN OPENSSH PRIVATE KEY-----'), 'assets/main.tf': nul('resource "x" "y" {}'), 'assets/query.sql': nul('DROP TABLE users;'), 'assets/server.p12': Buffer.from([0x30, 0x82, 0, 1, 2, 3]) }; + const content = { 'assets/report.pdf': GZ, 'assets/shot.png': GZ }; + for (const [rel, bytes] of Object.entries({ ...kept, ...content })) fs.writeFileSync(path.join(repo.work, rel), bytes); + commit(repo, {}); + const p = JSON.parse(plan(repo, ['--json']).out); + expect(p.review_exclude.kept_files).toEqual(Object.keys(kept).map(f => ({ path: f, reason: 'code' }))); + expect(p.review_exclude).toMatchObject({ excluded: 2, content: { binary: 2 } }); + }, 120_000); + + // Value: protects=a branch's .gitattributes cannot hide a credential line inside a matched file by making git call it + // binary, at the plan (live tree) and at the receipt (head blobs); fails_when=the secret-line rule reads only the + // collector's added lines, which git leaves empty for an attribute-binary file, so the file is excluded; why_new=GST-972 + // item 5 (#3852 round 3, outside-voice#2); seam=bin/gstack-review-plan and reviewExclude in head mode; no_claim=pregate's gitleaks scan + test('a branch .gitattributes marking receipts binary cannot hide a credential line: the keep rule reads the text', () => { + const repo = makeFixtureRepo({ prefix: 'gstack-exclude-', baseFiles: { '.gstack/ship-policy.json': policyText(['receipts/**']) } }); repos.push(repo); + commit(repo, { '.gitattributes': 'receipts/** binary\n', 'receipts/s.json': '{"cmd": "curl -H $OPENAI_API_KEY"}\n', 'receipts/a.json': '{"ok": true}\n' }); + const p = JSON.parse(plan(repo, ['--json']).out); + expect(p.review_exclude.kept_files).toEqual([{ path: 'receipts/s.json', reason: 'secret-line' }]); + expect(p.review_exclude).toMatchObject({ excluded: 1, content: { data: 1 } }); + const head = repo.git('rev-parse', 'HEAD'); + const baseSha = repo.git('rev-parse', 'origin/main'); + const c = collectCommit(repo.work, baseSha, head); + const x = reviewExclude({ root: repo.work, baseSha, ref: 'origin/main', mergeBase: c.mergeBase!, head, perPath: c.perPath, added: c.added, classRoot: c.root ?? [], gateTagged: new Set(), context: 'test' }); + expect(x.state === 'active' && x.kept).toEqual([{ path: 'receipts/s.json', reason: 'secret-line' }]); + expect(x.state === 'active' && x.record.digest).toBe(p.review_exclude.digest); + }, 120_000); + test('an unavailable gate scan keeps every candidate instead of excluding it', () => { const repo = census(); const head = repo.git('rev-parse', 'HEAD');