feat(sealed): refuse published records that put a label next to an identifier (GST-620)

sealed.label_keys turns on sealed.label_join: a published JSON, JSONL, CSV, TSV or Markdown-table record that holds a label field and an identifier (an identifier_keys glob or a hex value) is refused when its fingerprint is new against every parent. Unreadable supported files are sealed.label_join_unchecked. Without label_keys the guard reads no blobs and its output matches main byte for byte (golden in test/fixtures/sealed-guard-off-golden.json).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-10-31 20:45:44 +00:00
1 parent 2d1b55d762
commit 3fef3423ab
11 files changed
+669 -23

No files matched your search

+2 -1
View File
@@ -292,8 +292,9 @@ function cmdSealed(root: string, rest: string[], json: boolean): number {
const blocks = report.status === 'refused' || report.status === 'error';
if (json) { process.stdout.write(JSON.stringify({ ...report, ok: !blocks }) + '\n'); return blocks ? EXIT.fail : EXIT.ok; }
process.stderr.write(renderSealed(report, 'gstack-pregate sealed'));
const unchecked = report.findings.filter(f => f.rule === 'sealed.label_join_unchecked').length;
const policy = report.policyLabels.length ? report.policyLabels.join(' + ') : `no sealed key on ${baseRef ?? `${remote}/${base}`} or ${end}`;
process.stdout.write(`SEALED: ${report.status} findings=${report.findings.length} commits=${report.commits} visibility=${report.visibility}${report.visibility === 'unknown' ? ' (counted public)' : ''} waived=${report.waived.length} exempt=${report.exempt.length} policy=${policy}\n`);
process.stdout.write(`SEALED: ${report.status} findings=${report.findings.length} commits=${report.commits} visibility=${report.visibility}${report.visibility === 'unknown' ? ' (counted public)' : ''} waived=${report.waived.length} exempt=${report.exempt.length}${unchecked ? ` unchecked=${unchecked}` : ''} policy=${policy}\n`);
return blocks ? EXIT.fail : EXIT.ok;
}
@@ -0,0 +1,13 @@
**The sealed guard can now refuse a published record that puts a label next to an identifier.** gstack-evals hides which test units are decoys. Its /ship on garrytan/gstack-evals#99 published a panel where each unit's `stratum`, `tags` and `decoy_set` sat next to its `head`, `tree_sha256`, `plan_bytes` and `files`, values a model under test can compute from its own copy of the work. A model could look itself up and learn it was the decoy. No answer leaked, so `gstack-pregate sealed` said `clear` (GSTA-2, GST-620). The new rule is opt-in. A repo that sets `sealed.label_keys` gets it, and every other repo sees the same output as before, byte for byte.
### Added
- **`sealed.label_keys` and `sealed.identifier_keys` in `.gstack/pregate.json`.** With `label_keys` set, `sealed.label_join` checks every .json, .jsonl, .ndjson, .csv, .tsv and Markdown pipe table a published commit adds, modifies, renames or copies. A record that holds a label field and an identifier (a field matching an `identifier_keys` glob, by default `head`, `*_sha256`, `*_bytes`, `files` and similar names, or any 7 to 64 character hex string) is a join. The guard fingerprints each join in memory and compares it with the parent's version of the file, so unchanged rows and reformatting stay clear while a replaced, added or widened row refuses even when the row count stays the same. Findings name the file, the commit and the field names (`keys=stratum,tags,head,plan_bytes`), never the values, on stderr, in `--json` and in the pre-push hook.
- **`sealed.label_join_unchecked`.** A supported file over 4 MiB, one that does not parse, one nested deeper than 64 levels, or a blob git cannot read refuses instead of passing unread. The `SEALED:` line counts these as `unchecked=<n>`. Like every sealed finding, it can be waived through the base branch's `.gstack/sealed-allow.tsv`; `allow_paths` never exempt either rule.
- **The policy only tightens.** Both lists are the union of the base branch's and the pushed policy's, each resolved to its default first, so a pushed config can turn the rule on or widen it but cannot turn it off or narrow it.
### Changed
- **docs/pregate.md#sealed** documents the rule, its normalization (`Tree SHA-256` matches as `tree_sha_256`) and its known limits: joins across files, joins by reference, columnar JSON, labels in file names and prose. Its version-skew note now names the new keys. A gstack older than this release refuses them as unknown, so raise `min_gstack_version` before adopting them. The unknown-key error now lists `label_keys` and `identifier_keys` among the valid keys.
### Tests
- `test/sealed-guard.test.ts`: the GSTA-2 panel refuses through `gstack-pregate sealed`, `--json` and the pre-push hook, and warns on a private destination, with no value in any output. It failed on main. A golden captured from main 2d1b55d76 covers seven pushes (refused, clear, exempt, waived, warned, error, no policy) without `label_keys`; stdout, stderr, `--json`, the hook's stderr and every exit code match it byte for byte.
- `test/sealed-join.test.ts`: each format, human table headers, hex values under unlisted keys, decimal strings that stay clear, replacement at the same count, delete plus add, an added identifier field, labels then identifiers a commit later, renames, merges, five unchecked cases, waivers, `allow_paths`, pushed attempts to narrow or disable the rule, and `label_keys: []` matching an absent key.
+44 -6
View File
@@ -946,7 +946,8 @@ with a timing flake: never raise the bound.
"answer_markers": ["SEALED ANSWER:"],
"seed_patterns": ["shuffle_seed=\\d+"],
"local_paths": true,
"case_insensitive": true } }
"case_insensitive": true,
"label_keys": ["stratum", "decoy_set"] } }
```
A repo with a `sealed` key gets a configured local check against accidental
@@ -992,11 +993,48 @@ gstack-pregate sealed [--range <a..b>] [--remote origin] [--base <branch>] [--js
sealed data. Each exempt path is listed on every push (`exempt <rule>
<file> commit <sha> (sealed.allow_paths <glob>)`) and counted as
`exempt=<n>` on the `SEALED:` line.
- **Version skew.** A gstack older than the release whose CHANGELOG adds
`sealed.allow_paths` refuses `allow_paths` and `case_insensitive` as
unknown keys, so every push and `gstack-pregate run` in the repository
fail on that machine. Raise `min_gstack_version` in
`.gstack/ship-policy.json` to that release before landing either key.
- **Label joins.** Off unless `label_keys` names at least one field. Then
`sealed.label_join` refuses a published record that puts a label field next
to an identifier, the way a panel row with `stratum` beside `head` and
`plan_bytes` tells a model under test which unit it is. A record is a JSON
object at any depth (its own fields plus one level of nested objects, named
like `meta.head`), a JSONL or NDJSON line, a CSV or TSV row, or a row of a
Markdown pipe table. Field names and table headers are compared after
trimming, lowercasing and turning spaces and hyphens into underscores
(`Tree SHA-256` is `tree_sha_256`); this is the rule's own normalization,
separate from `case_insensitive`. An identifier is a field whose name
matches an `identifier_keys` glob (`*` is the only wildcard; the default
is `head`, `head_sha`, `sha`, `commit`, `tree`, `tree_sha*`, `*_sha`,
`*_sha256`, `sha256`, `digest`, `*_digest`, `hash`, `*_hash`,
`*_bytes`, `bytes`, `size`, `files`, `lines`), or any string value of 7
to 64 hex characters with at least one digit and one letter a to f. Numbers
count only through the field name. For each supported file a commit adds,
modifies, renames or copies, the guard fingerprints every joined record
(sha256 over its label and identifier fields and their values, kept in
memory) and compares the set with the parent's version of the file (the
rename or copy source; every parent for a merge). A fingerprint no parent
has is a finding, so unchanged records and formatting-only edits stay clear,
and a replaced, added or widened record refuses even when the count stays
the same. A supported file over 4 MiB, one that does not parse (one bad
JSONL line is enough), one nested deeper than 64 levels, or a blob git
cannot read is `sealed.label_join_unchecked`. It refuses too, and the
`SEALED:` line counts it as `unchecked=<n>`. Findings print field names
(`keys=`, label fields first), never values; `--json` carries them as
`keys`. `allow_paths` never exempt either rule; a base-branch waiver does.
Both lists are the union of the base branch's and the pushed policy's, each
resolved to its default first, so a push can turn the rule on or widen it
but never turn it off or narrow it. Without `label_keys` the guard reads no
file contents and its output is unchanged. Known limits: joins across files,
joins by reference inside a file (a `decoy_set` list naming units described
elsewhere), columnar JSON (`columns` plus `rows`), labels encoded in file
names, and prose.
- **Version skew.** A gstack older than the release that adds a `sealed` key
refuses it as unknown, so every push and `gstack-pregate run` in the
repository fail on that machine. `allow_paths` and `case_insensitive` came
with the release whose CHANGELOG adds `sealed.allow_paths`; `label_keys`
and `identifier_keys` came with the release whose CHANGELOG adds
`sealed.label_join`. Raise `min_gstack_version` in
`.gstack/ship-policy.json` to that release before landing the key.
- **Policy.** The union of the destination base branch's `sealed` and the
pushed commit's: a push can tighten it, never relax it (`allow_paths` come
from the base branch alone, and a pushed deny glob cannot override one). The
+8 -3
View File
@@ -3303,7 +3303,7 @@ base branch's copy or the user-owned file. For the user-owned file, edit it
directly at the path the message names. A user file inside the repository, or
reached through a symlink into it, is ignored rather than read (`--explain`
names it). An unknown `sealed` key that a newer gstack defines (such as
`allow_paths` or `case_insensitive`) means this machine's gstack is older than
`allow_paths`, `case_insensitive`, `label_keys` or `identifier_keys`) means this machine's gstack is older than
the policy: upgrade gstack here, and raise `min_gstack_version` in the
repository's `.gstack/ship-policy.json` before landing such a key.
@@ -3892,7 +3892,10 @@ migration is a code change, so the gate reruns.
sealed path or transcript (binary, empty and later-deleted files included), or
adds a line with a sealed-answer marker, a seed or a local home path, and the
destination is public or its visibility is unknown (a new or unreadable
repository counts as public). An unreadable or malformed sealed policy refuses
repository counts as public). With `sealed.label_keys` set, a new record that
puts a label field next to an identifier (`sealed.label_join`), or a supported
file the guard could not read (`sealed.label_join_unchecked`), refuses the
same way. An unreadable or malformed sealed policy refuses
too. The managed pre-push hook and `gstack-pregate sealed` print one line per
finding: rule, `file[:line]`, commit. They never print the matched text.
@@ -3901,7 +3904,9 @@ reaches the public repository (a later commit that deletes it still publishes
it). A publishable file kept next to sealed data (a `MANIFEST.json` of hashes
and counts) can be exempted from the path rules with a `sealed.allow_paths`
glob in the base branch's `.gstack/pregate.json`; the text rules still read it
([docs/pregate.md#sealed](pregate.md#sealed)). A reviewed false positive gets
([docs/pregate.md#sealed](pregate.md#sealed)). For `sealed.label_join`, drop
the identifier fields (or the labels) from the published file, or move it under
a sealed path; `allow_paths` never exempt it. A reviewed false positive gets
one line in the base branch's
`.gstack/sealed-allow.tsv`: `rule<TAB>path<TAB>sha<TAB>reason`, where `sha` is
the full commit sha (a shortened one is ignored and named on stderr). A private
+1 -1
View File
@@ -338,7 +338,7 @@ export function explainConfig(): string {
'local: <workflow.yml>/<job> lanes a local run satisfies (platform-agnostic toolchain, or CI off); their tests run here, not requires-remote',
'guards: [{ id, when: [globs], command, fix?, timeout_s? }]: each guard whose globs match a touched file runs in its own process (--allow-repo-commands); a failure fails pregate with the guard id and fix; never downgradable',
`serial_patterns: test-file globs the lanes check runs one process per file, apart from the unit files (default ${DEFAULT_SERIAL_PATTERNS.join(', ')})`,
'sealed: { paths, transcripts, answer_markers, seed_patterns, local_paths, allow_paths, case_insensitive, visibility }: sealed or eval output the pre-push hook and `gstack-pregate sealed` refuse to publish to a public destination; allow_paths (base branch only) exempt publishable files from the path globs (docs/pregate.md#sealed)',
'sealed: { paths, transcripts, answer_markers, seed_patterns, local_paths, allow_paths, case_insensitive, label_keys, identifier_keys, visibility }: sealed or eval output the pre-push hook and `gstack-pregate sealed` refuse to publish to a public destination; allow_paths (base branch only) exempt publishable files from the path globs; label_keys turns on sealed.label_join, which refuses a published record that puts one of those fields next to an identifier (identifier_keys globs, or a hex value) (docs/pregate.md#sealed)',
'price_file: repo-relative path of the GSTACK_EVAL_CALLS v1 call-plan file `gstack-eval-plan price-gate` prices a paid gate from; never executed (docs/eval-calls.md)',
'stall_timeout_s: seconds without test output before the lanes check kills the runner\'s process group and records it incomplete (default 300; 0 disables; --stall-timeout overrides)',
'timeout_s: seconds each runner process may take (the unit-file group, each serial file, each guard without its own timeout_s, and the remote gate\'s deadline) when --timeout is not passed; 1 to 3600, default 600',
+17 -12
View File
@@ -11,6 +11,8 @@
* local_paths true: a local home path (/home/<u>/, /Users/<u>/) in added text
* allow_paths globs exempt from the path rules (read from the base branch only; never folds case)
* case_insensitive true: `paths` and `transcripts` match regardless of case
* label_keys field names whose records may not also carry an identifier (lib/sealed-join.ts; off when empty)
* identifier_keys field-name globs that count as identifiers (default DEFAULT_IDENTIFIER_KEYS)
* visibility "public" | "private": the destination override (reviewed on the base branch)
*
* The publication boundary is every commit the push publishes: the local sha
@@ -32,19 +34,20 @@ import { spawnSync } from 'node:child_process';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { containedRelative, globToRegExp } from './ship-policy';
import { DEFAULT_IDENTIFIER_KEYS, identifierKeyError, joinFindings, labelKeyError } from './sealed-join';
import { resolveStateRoot } from './state-root';
export const SEALED_CONFIG_PATH = '.gstack/pregate.json';
export const SEALED_ALLOW_PATH = '.gstack/sealed-allow.tsv';
export const VISIBILITY_OVERRIDE_KEY = 'sealed.visibility';
export type SealedRule = 'sealed.path' | 'sealed.transcript' | 'sealed.answer' | 'sealed.seed' | 'sealed.local_path';
export interface SealedPolicy { paths: string[]; transcripts: string[]; answer_markers: string[]; seed_patterns: string[]; local_paths: boolean; allow_paths: string[]; case_insensitive: boolean; visibility?: 'public' | 'private' }
export interface SealedFinding { rule: SealedRule; file: string; line?: number; commit: string }
export type SealedRule = 'sealed.path' | 'sealed.transcript' | 'sealed.answer' | 'sealed.seed' | 'sealed.local_path' | 'sealed.label_join' | 'sealed.label_join_unchecked';
export interface SealedPolicy { paths: string[]; transcripts: string[]; answer_markers: string[]; seed_patterns: string[]; local_paths: boolean; allow_paths: string[]; case_insensitive: boolean; label_keys: string[]; identifier_keys: string[]; visibility?: 'public' | 'private' }
export interface SealedFinding { rule: SealedRule; file: string; line?: number; commit: string; keys?: string[] }
export type ExemptFinding = SealedFinding & { glob: string };
export interface AllowEntry { rule: string; path: string; sha: string; reason: string }
export type Visibility = 'public' | 'private' | 'unknown';
const KEYS = ['paths', 'transcripts', 'answer_markers', 'seed_patterns', 'local_paths', 'allow_paths', 'case_insensitive', 'visibility'];
const KEYS = ['paths', 'transcripts', 'answer_markers', 'seed_patterns', 'local_paths', 'allow_paths', 'case_insensitive', 'label_keys', 'identifier_keys', 'visibility'];
const MAX_ENTRIES = 64;
const MAX_REGEX = 256;
const LOCAL_PATH_RE = /(?:^|[^A-Za-z0-9_.~-]|\$\{!?(?:[A-Za-z_][A-Za-z0-9_]*(?:\[[^\]\s]{0,64}\])?|[0-9]+|[@*#?$!-]):?-)(?:\/home\/[^/\s'"`]+\/|\/Users\/[^/\s'"`]+\/|[A-Za-z]:\\+Users\\+[^\\\s'"`]+\\)/;
@@ -68,6 +71,7 @@ export function sealedErrors(raw: unknown): Array<{ path: string; message: strin
const regex = (r: string) => { if (!r || r.length > MAX_REGEX) return `must be 1-${MAX_REGEX} characters`; try { new RegExp(r); return null; } catch (e: any) { return `is not a valid regex (${e.message})`; } };
const exception = (g: string) => { const name = g.split('/').pop()!; return glob(g) ?? (name.includes('**') || /^[*?]+(\.|$)/.test(name) ? 'has a wildcard file name, which would exempt whole directories or file types; name the file (e.g. **/sealed/*/MANIFEST.json)' : null); };
list('paths', glob); list('transcripts', glob); list('answer_markers', regex); list('seed_patterns', regex); list('allow_paths', exception);
list('label_keys', labelKeyError); list('identifier_keys', identifierKeyError);
for (const k of ['local_paths', 'case_insensitive']) if (k in o && typeof o[k] !== 'boolean') errors.push({ path: `sealed.${k}`, message: 'must be true or false' });
if ('visibility' in o && o.visibility !== 'public' && o.visibility !== 'private') errors.push({ path: 'sealed.visibility', message: 'must be "public" or "private"' });
return errors;
@@ -93,15 +97,15 @@ export function sealedPolicyAt(repoRoot: string, treeish: string, label: string)
const errors = sealedErrors(raw.sealed);
if (errors.length) return { error: errors.map(e => `${e.path}: ${e.message}`).join('; '), label };
const s = raw.sealed;
return { policy: { paths: s.paths ?? [], transcripts: s.transcripts ?? [], answer_markers: s.answer_markers ?? [], seed_patterns: s.seed_patterns ?? [], local_paths: s.local_paths === true, allow_paths: s.allow_paths ?? [], case_insensitive: s.case_insensitive === true, visibility: s.visibility }, label };
return { policy: { paths: s.paths ?? [], transcripts: s.transcripts ?? [], answer_markers: s.answer_markers ?? [], seed_patterns: s.seed_patterns ?? [], local_paths: s.local_paths === true, allow_paths: s.allow_paths ?? [], case_insensitive: s.case_insensitive === true, label_keys: s.label_keys ?? [], identifier_keys: s.identifier_keys ?? DEFAULT_IDENTIFIER_KEYS, visibility: s.visibility }, label };
}
/** Base ∪ pushed: deny lists concatenate, `local_paths` and `case_insensitive` OR (both only tighten); `allow_paths` relax, so they come from the base alone; visibility stays private only when the base says so and the push does not say public. */
/** Base ∪ pushed: deny lists, `label_keys` and `identifier_keys` (each resolved to its default per policy first) concatenate, `local_paths` and `case_insensitive` OR (both only tighten); `allow_paths` relax, so they come from the base alone; visibility stays private only when the base says so and the push does not say public. */
export function unionPolicies(base: SealedPolicy | null, pushed: SealedPolicy | null): SealedPolicy | null {
if (!base && !pushed) return null;
const u = (k: 'paths' | 'transcripts' | 'answer_markers' | 'seed_patterns') => [...new Set([...(base?.[k] ?? []), ...(pushed?.[k] ?? [])])];
const u = (k: 'paths' | 'transcripts' | 'answer_markers' | 'seed_patterns' | 'label_keys' | 'identifier_keys') => [...new Set([...(base?.[k] ?? []), ...(pushed?.[k] ?? [])])];
const visibility = pushed?.visibility === 'public' || base?.visibility === 'public' ? 'public' : base?.visibility === 'private' ? 'private' : undefined;
return { paths: u('paths'), transcripts: u('transcripts'), answer_markers: u('answer_markers'), seed_patterns: u('seed_patterns'), local_paths: !!(base?.local_paths || pushed?.local_paths), allow_paths: base?.allow_paths ?? [], case_insensitive: !!(base?.case_insensitive || pushed?.case_insensitive), visibility };
return { paths: u('paths'), transcripts: u('transcripts'), answer_markers: u('answer_markers'), seed_patterns: u('seed_patterns'), local_paths: !!(base?.local_paths || pushed?.local_paths), allow_paths: base?.allow_paths ?? [], case_insensitive: !!(base?.case_insensitive || pushed?.case_insensitive), label_keys: u('label_keys'), identifier_keys: u('identifier_keys'), visibility };
}
/** Reviewed waivers: `rule<TAB>path<TAB>sha<TAB>reason` lines in the base branch's allow file; a waiver counts only when its sha equals the finding's full commit sha. */
@@ -190,7 +194,7 @@ function textFindings(repoRoot: string, commits: string[], p: SealedPolicy): Sea
export function scanCommits(repoRoot: string, commits: string[], policy: SealedPolicy): SealedFinding[] {
if (!commits.length) return [];
const seen = new Set<string>();
return [...pathFindings(repoRoot, commits, policy), ...textFindings(repoRoot, commits, policy)].filter(f => {
return [...pathFindings(repoRoot, commits, policy), ...textFindings(repoRoot, commits, policy), ...joinFindings(repoRoot, commits, policy)].filter(f => {
const key = `${f.rule}\0${f.file}\0${f.line ?? ''}\0${f.commit}`;
if (seen.has(key)) return false;
seen.add(key);
@@ -299,15 +303,16 @@ export function renderSealed(r: SealedReport, context: string): string {
lines.push(`${context}: ignored waiver ${i.rule} ${i.path} sha ${i.sha} (${why}; this finding is in commit ${i.commit}, review it before waiving)`);
}
const vis = `visibility: ${r.visibility}${r.visibility === 'unknown' ? ', counted public' : ''}; ${r.visibilitySource}`;
const body = r.findings.map(f => ` ${f.rule} ${where(f)} commit ${f.commit.slice(0, 12)}`);
const body = r.findings.map(f => ` ${f.rule} ${where(f)} commit ${f.commit.slice(0, 12)}${f.keys ? ` keys=${f.keys.join(',')}` : ''}`);
const join = r.findings.some(f => f.rule.startsWith('sealed.label_join')) ? [`sealed.label_join: a record puts a sealed.label_keys field next to an identifier (keys= lists the label fields, then the identifier fields); drop one side from the published file or move it under a sealed path. sealed.label_join_unchecked: a supported file the guard could not read (over 4 MiB, unparseable, or nested deeper than 64); allow_paths never exempt either rule`] : [];
if (r.status === 'error') {
lines.push(`\n⛔ ${context} BLOCKED the publication of ${r.target} — ${r.error}.`);
lines.push(`Fix the sealed policy (gstack-pregate validate), then retry; an unreadable policy never passes (SEALED_OUTPUT_REFUSED)`);
} else if (r.status === 'refused') {
lines.push(`\n⛔ ${context} BLOCKED the publication of ${r.target} (${vis}) — sealed or eval output in ${r.commits} published commit(s):`, ...body);
lines.push(`\n⛔ ${context} BLOCKED the publication of ${r.target} (${vis}) — sealed or eval output in ${r.commits} published commit(s):`, ...body, ...join);
lines.push(`Remove it from every published commit before this branch reaches a public repository, exempt a publishable path with a sealed.allow_paths glob in the base branch's ${SEALED_CONFIG_PATH}, or waive one reviewed finding with a \`rule<TAB>path<TAB>full-commit-sha<TAB>reason\` line in the base branch's ${SEALED_ALLOW_PATH}. A private destination: set ${VISIBILITY_OVERRIDE_KEY} "private" in the base branch's ${SEALED_CONFIG_PATH} (SEALED_OUTPUT_REFUSED)`);
} else if (r.status === 'warned') {
lines.push(`${context}: warning — sealed or eval output in a push to a private destination (${vis}); it would be refused for a public one:`, ...body);
lines.push(`${context}: warning — sealed or eval output in a push to a private destination (${vis}); it would be refused for a public one:`, ...body, ...join);
}
return lines.length ? lines.join('\n') + '\n' : '';
}
+232
View File
@@ -0,0 +1,232 @@
/**
* sealed-join — the `sealed.label_join` rule of the sealed-output guard
* (lib/sealed-guard.ts), on only when the policy names `sealed.label_keys`.
*
* A record that puts a label field (a decoy stratum, a flaw tag) next to an
* identifier the model under test can compute from its own copy of the work
* (a head sha, a tree hash, a byte count) tells that model which label it
* carries. For every .json, .jsonl, .ndjson, .csv, .tsv or .md (pipe table)
* file a published commit adds, modifies, renames or copies, each joined
* record is fingerprinted (sha256 over its label and identifier key/value
* pairs, in memory only) at the commit and at every parent's version (the
* rename or copy source); a fingerprint no parent has is a finding. A file
* over 4 MiB, one that does not parse, nests deeper than 64 or cannot be read
* is `sealed.label_join_unchecked`. Findings carry field names, never values.
*/
import { spawnSync } from 'node:child_process';
import { createHash } from 'node:crypto';
import type { SealedFinding } from './sealed-guard';
export const DEFAULT_IDENTIFIER_KEYS = ['head', 'head_sha', 'sha', 'commit', 'tree', 'tree_sha*', '*_sha', '*_sha256', 'sha256', 'digest', '*_digest', 'hash', '*_hash', '*_bytes', 'bytes', 'size', 'files', 'lines'];
export const JOIN_MAX_BYTES = 4 * 1024 * 1024;
const MAX_DEPTH = 64;
const MAX_KEY = 128;
const SUPPORTED = /\.(json|jsonl|ndjson|csv|tsv|md)$/i;
const HEX_ID = /^(?=[0-9a-f]*[0-9])(?=[0-9a-f]*[a-f])[0-9a-f]{7,64}$/i;
const ZERO = /^0+$/;
const TABLE_DELIMITER = /^\s*\|?\s*:?-+:?\s*(\|\s*:?-+:?\s*)*\|?\s*$/;
export interface JoinPolicy { label_keys: string[]; identifier_keys: string[] }
type Field = { name: string; key: string; value: unknown };
type Joins = Map<string, { labels: string[]; ids: string[] }>;
/** The rule's own key normalization: trimmed, lowercased, spaces and hyphens as underscores. */
export const normalizeKey = (k: string) => k.trim().toLowerCase().replace(/[ -]/g, '_');
export function labelKeyError(s: string): string | null {
return s.trim() && s.length <= MAX_KEY && !/[*?[\]{}]/.test(s) ? null : `must be a field name of 1-${MAX_KEY} characters with no wildcard`;
}
export function identifierKeyError(s: string): string | null {
return s.trim() && s.length <= MAX_KEY && !/[?[\]{}]/.test(s) ? null : `must be a field name of 1-${MAX_KEY} characters; * is the only wildcard`;
}
const present = (v: unknown) => typeof v === 'number' || (typeof v === 'string' && v.trim() !== '') || (Array.isArray(v) && v.length > 0);
const hexValue = (v: unknown): boolean => typeof v === 'string' ? HEX_ID.test(v.trim()) : Array.isArray(v) && v.some(e => typeof e === 'string' && HEX_ID.test(e.trim()));
const shown = (name: string) => name.split('.').map(s => HEX_ID.test(s) ? '(hex)' : s).join('.').slice(0, 64);
function classifier(p: JoinPolicy): (fields: Field[]) => { fp: string; labels: string[]; ids: string[] } | null {
const labels = new Set(p.label_keys.map(normalizeKey));
const ids = p.identifier_keys.map(g => new RegExp(`^${normalizeKey(g).replace(/[.+^${}()|[\]\\?]/g, '\\$&').replace(/\*/g, '.*')}$`));
return fields => {
const l: Field[] = []; const i: Field[] = [];
for (const f of fields) {
if (labels.has(f.key)) l.push(f);
else if ((present(f.value) && ids.some(re => re.test(f.key))) || hexValue(f.value)) i.push(f);
}
if (!l.length || !i.length) return null;
const fp = createHash('sha256').update([...l, ...i].map(f => `${f.name}=${JSON.stringify(f.value)}`).sort().join('\0')).digest('hex');
return { fp, labels: l.map(f => shown(f.name)), ids: i.map(f => shown(f.name)) };
};
}
/** Every JSON object at any depth as one record: its own fields plus one level of nested object fields (`meta.head`); false past the depth cap. */
function jsonRecords(root: unknown, out: Field[][]): boolean {
const field = (name: string, value: unknown, key: string): Field => ({ name, key: normalizeKey(key), value });
const stack: Array<[unknown, number]> = [[root, 0]];
while (stack.length) {
const [v, depth] = stack.pop()!;
if (!v || typeof v !== 'object') continue;
if (depth > MAX_DEPTH) return false;
if (Array.isArray(v)) { for (const e of v) stack.push([e, depth + 1]); continue; }
const fields: Field[] = [];
for (const [k, val] of Object.entries(v)) {
const name = normalizeKey(k);
fields.push(field(name, val, k));
if (val && typeof val === 'object' && !Array.isArray(val)) for (const [k2, v2] of Object.entries(val)) fields.push(field(`${name}.${normalizeKey(k2)}`, v2, k2));
stack.push([val, depth + 1]);
}
out.push(fields);
}
return true;
}
function csvRows(text: string): string[][] | null {
const rows: string[][] = []; let row: string[] = []; let cell = ''; let quoted = false;
for (let i = 0; i < text.length; i++) {
const c = text[i]!;
if (quoted) {
if (c !== '"') cell += c;
else if (text[i + 1] === '"') { cell += '"'; i++; }
else quoted = false;
} else if (c === '"' && cell.trim() === '') { quoted = true; cell = ''; }
else if (c === ',') { row.push(cell); cell = ''; }
else if (c === '\n' || c === '\r') { if (c === '\r' && text[i + 1] === '\n') i++; row.push(cell); rows.push(row); row = []; cell = ''; }
else cell += c;
}
if (quoted) return null;
if (cell !== '' || row.length) { row.push(cell); rows.push(row); }
return rows;
}
function markdownTables(text: string): string[][][] {
const cells = (line: string) => line.trim().replace(/^\|/, '').replace(/(?<!\\)\|$/, '').split(/(?<!\\)\|/).map(c => c.trim().replace(/^[`*]+|[`*]+$/g, ''));
const lines = text.split(/\r?\n/);
const tables: string[][][] = [];
for (let i = 0; i + 1 < lines.length; i++) {
if (!lines[i]!.includes('|') || !lines[i + 1]!.includes('|') || !TABLE_DELIMITER.test(lines[i + 1]!)) continue;
const table = [cells(lines[i]!)];
let j = i + 2;
for (; j < lines.length && lines[j]!.includes('|'); j++) table.push(cells(lines[j]!));
tables.push(table);
i = j - 1;
}
return tables;
}
function tableRecords(rows: string[][], out: Field[][]): void {
const header = (rows[0] ?? []).map(normalizeKey);
for (const row of rows.slice(1)) if (row.some(c => c.trim())) out.push(header.map((h, i) => ({ name: h, key: h, value: (row[i] ?? '').trim() })));
}
/** The records of one supported file's text, or null when it cannot be checked (parse failure or depth cap). */
export function fileRecords(file: string, text: string): Field[][] | null {
const body = text.replace(/^\uFEFF/, '');
const ext = (/\.([a-z]+)$/i.exec(file)?.[1] ?? '').toLowerCase();
const out: Field[][] = [];
try {
if (ext === 'json') return !body.trim() || jsonRecords(JSON.parse(body), out) ? out : null;
if (ext === 'jsonl' || ext === 'ndjson') {
for (const line of body.split(/\r?\n/)) if (line.trim() && !jsonRecords(JSON.parse(line), out)) return null;
return out;
}
} catch { return null; }
if (ext === 'md') { for (const t of markdownTables(body)) tableRecords(t, out); return out; }
const rows = ext === 'tsv' ? body.split(/\r?\n/).map(l => l.split('\t')) : csvRows(body);
if (!rows) return null;
tableRecords(rows.filter(r => r.some(c => c.trim())), out);
return out;
}
/** Joined-record fingerprints of one file's text; null when it cannot be checked. */
export function fileJoins(file: string, text: string, p: JoinPolicy): Joins | null {
const records = fileRecords(file, text);
if (!records) return null;
const classify = classifier(p);
const joins: Joins = new Map();
for (const r of records) { const j = classify(r); if (j) joins.set(j.fp, { labels: j.labels, ids: j.ids }); }
return joins;
}
function gitBuffer(repoRoot: string, args: string[], input: string, label: string): Buffer {
const r = spawnSync('git', args, { cwd: repoRoot, input, timeout: 120_000, maxBuffer: 512 * 1024 * 1024 });
if (r.status !== 0) throw new Error(`git ${label} failed: ${(r.stderr ?? '').toString().trim().slice(0, 200)}`);
return r.stdout;
}
interface Change { commit: string; file: string; blob: string; parents: string[] }
/** Supported regular files a commit adds, modifies, renames or copies against every parent (`-m`: a merge's file must differ from each parent). */
function changedFiles(repoRoot: string, commits: string[]): Change[] {
const parentCount = new Map(gitBuffer(repoRoot, ['rev-list', '--no-walk', '--parents', '--stdin'], commits.join('\n') + '\n', 'rev-list --parents').toString('utf8').split('\n').filter(Boolean).map(l => { const s = l.split(' '); return [s[0]!, Math.max(1, s.length - 1)] as const; }));
const tokens = gitBuffer(repoRoot, ['-c', 'core.quotePath=false', 'diff-tree', '--stdin', '-r', '--root', '-m', '-M', '-C', '--raw', '-z', '--no-abbrev'], commits.join('\n') + '\n', 'diff-tree -m').toString('utf8').split('\0');
const sections = new Map<string, Array<Map<string, { src: string; dst: string }>>>();
let current: Map<string, { src: string; dst: string }> | null = null;
for (let i = 0; i < tokens.length; i++) {
const t = tokens[i]!;
if (!t) continue;
if (!t.startsWith(':')) { current = new Map(); sections.set(t, [...(sections.get(t) ?? []), current]); continue; }
const [, dstMode, src, dst, status] = t.slice(1).split(' ') as [string, string, string, string, string];
const file = /^[RC]/.test(status) ? tokens[i += 2]! : tokens[++i]!;
if (status.startsWith('D') || !/^100(644|755)$/.test(dstMode) || !SUPPORTED.test(file)) continue;
current?.set(file, { src, dst });
}
const out: Change[] = [];
for (const [commit, parts] of sections) {
if (parts.length < (parentCount.get(commit) ?? 1)) continue;
for (const [file, { dst }] of parts[0]!) {
if (parts.every(p => p.has(file))) out.push({ commit, file, blob: dst, parents: parts.map(p => p.get(file)!.src).filter(s => !ZERO.test(s)) });
}
}
return out;
}
/** Blob text by sha through one `cat-file --batch-check` and chunked `--batch` reads; `big` over the cap, `missing` when unreadable. */
function readBlobs(repoRoot: string, shas: string[]): Map<string, string | { unread: 'big' | 'missing' }> {
const out = new Map<string, string | { unread: 'big' | 'missing' }>();
if (!shas.length) return out;
const small: Array<[string, number]> = [];
for (const line of gitBuffer(repoRoot, ['cat-file', '--batch-check'], shas.join('\n') + '\n', 'cat-file --batch-check').toString('utf8').split('\n').filter(Boolean)) {
const [sha, type, size] = line.split(' ');
if (type !== 'blob') out.set(sha!, { unread: 'missing' });
else if (Number(size) > JOIN_MAX_BYTES) out.set(sha!, { unread: 'big' });
else small.push([sha!, Number(size)]);
}
const decoder = new TextDecoder();
for (let start = 0; start < small.length;) {
let end = start; let bytes = 0;
while (end < small.length && (end === start || bytes + small[end]![1] <= 64 * 1024 * 1024)) bytes += small[end++]![1];
const buf = gitBuffer(repoRoot, ['cat-file', '--batch'], small.slice(start, end).map(s => s[0]).join('\n') + '\n', 'cat-file --batch');
let pos = 0;
for (let k = start; k < end; k++) {
const nl = buf.indexOf(0x0a, pos);
const size = Number(buf.subarray(pos, nl).toString('utf8').split(' ')[2]);
out.set(small[k]![0], decoder.decode(buf.subarray(nl + 1, nl + 1 + size)));
pos = nl + 1 + size + 1;
}
start = end;
}
return out;
}
/** `sealed.label_join` and `sealed.label_join_unchecked` findings for the published commits; [] without label_keys (no git reads). */
export function joinFindings(repoRoot: string, commits: string[], p: JoinPolicy): SealedFinding[] {
if (!p.label_keys.length || !commits.length) return [];
const changes = changedFiles(repoRoot, commits);
const blobs = readBlobs(repoRoot, [...new Set(changes.flatMap(c => [c.blob, ...c.parents]))]);
const cache = new Map<string, Joins | null>();
const joinsOf = (sha: string, file: string) => {
const key = `${sha}\0${(/\.[a-z]+$/i.exec(file)?.[0] ?? '').toLowerCase()}`;
if (!cache.has(key)) { const text = blobs.get(sha); cache.set(key, typeof text === 'string' ? fileJoins(file, text, p) : null); }
return cache.get(key)!;
};
const out: SealedFinding[] = [];
for (const c of changes) {
const now = joinsOf(c.blob, c.file);
if (!now) { out.push({ rule: 'sealed.label_join_unchecked', file: c.file, commit: c.commit }); continue; }
const before = new Set(c.parents.flatMap(s => [...(joinsOf(s, c.file)?.keys() ?? [])]));
const labels = new Set<string>(); const ids = new Set<string>();
for (const [fp, j] of now) if (!before.has(fp)) { j.labels.forEach(k => labels.add(k)); j.ids.forEach(k => ids.add(k)); }
if (labels.size) out.push({ rule: 'sealed.label_join', file: c.file, commit: c.commit, keys: [...[...labels].sort(), ...[...ids].sort()] });
}
return out;
}
+1
View File
@@ -155,6 +155,7 @@
{ "file": "lib/pregate/checks/timing.ts", "movedFrom": "(new: pre-gate timing-assertion check, field list 2 W4)" },
{ "file": "lib/pregate/runtime.ts", "movedFrom": "(new: the verified launch Bun for repo commands, field wave CD1)" },
{ "file": "lib/sealed-guard.ts", "movedFrom": "(new: the sealed-output guard shared by the pre-push hook and gstack-pregate sealed, field wave CD4)" },
{ "file": "lib/sealed-join.ts", "movedFrom": "(new: the sealed.label_join rule, label fields published beside identifiers, GST-620)" },
{ "file": "lib/pregate/regate.ts", "movedFrom": "(new: the regate verdict from a fresh tree receipt plus migration collisions, field wave CD3)" },
{ "file": "lib/autoplan-scope.ts", "movedFrom": "(new: UI evidence from rendered surfaces and the plan scope marker, shared by the runner and the snapshot tool, field wave B2c)" },
{ "file": "lib/source-root.ts", "movedFrom": "(new: the source checkout behind an install root, from its recorded .source-path, field wave B2e)" },
+65
View File
@@ -0,0 +1,65 @@
{
"refused": {
"code": 1,
"stdout": "SEALED: refused findings=2 commits=1 visibility=unknown (counted public) waived=0 exempt=0 policy=origin/main + commit <sha>\n",
"stderr": "\n⛔ gstack-pregate sealed BLOCKED the publication of <root>/origin.git (visibility: unknown, counted public; not a GitHub remote) — sealed or eval output in 1 published commit(s):\n sealed.transcript runs/c1/transcript.jsonl commit <sha>\n sealed.answer runs/c1/summary.md:1 commit <sha>\nRemove it from every published commit before this branch reaches a public repository, exempt a publishable path with a sealed.allow_paths glob in the base branch's .gstack/pregate.json, or waive one reviewed finding with a `rule<TAB>path<TAB>full-commit-sha<TAB>reason` line in the base branch's .gstack/sealed-allow.tsv. A private destination: set sealed.visibility \"private\" in the base branch's .gstack/pregate.json (SEALED_OUTPUT_REFUSED)\n",
"jsonCode": 1,
"json": "{\"status\":\"refused\",\"policyLabels\":[\"origin/main\",\"commit <sha>\"],\"commits\":1,\"findings\":[{\"rule\":\"sealed.transcript\",\"file\":\"runs/c1/transcript.jsonl\",\"commit\":\"<sha>\"},{\"rule\":\"sealed.answer\",\"file\":\"runs/c1/summary.md\",\"line\":1,\"commit\":\"<sha>\"}],\"exempt\":[],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"not a GitHub remote\",\"target\":\"<root>/origin.git\",\"ok\":false}\n",
"hookCode": 1,
"hookStderr": "\n⛔ gstack sealed-output guard BLOCKED the publication of <root>/origin.git (visibility: unknown, counted public; not a GitHub remote) — sealed or eval output in 1 published commit(s):\n sealed.transcript runs/c1/transcript.jsonl commit <sha>\n sealed.answer runs/c1/summary.md:1 commit <sha>\nRemove it from every published commit before this branch reaches a public repository, exempt a publishable path with a sealed.allow_paths glob in the base branch's .gstack/pregate.json, or waive one reviewed finding with a `rule<TAB>path<TAB>full-commit-sha<TAB>reason` line in the base branch's .gstack/sealed-allow.tsv. A private destination: set sealed.visibility \"private\" in the base branch's .gstack/pregate.json (SEALED_OUTPUT_REFUSED)\n"
},
"clear": {
"code": 0,
"stdout": "SEALED: clear findings=0 commits=1 visibility=unknown (counted public) waived=0 exempt=0 policy=origin/main + commit <sha>\n",
"stderr": "",
"jsonCode": 0,
"json": "{\"status\":\"clear\",\"policyLabels\":[\"origin/main\",\"commit <sha>\"],\"commits\":1,\"findings\":[],\"exempt\":[],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"not a GitHub remote\",\"target\":\"<root>/origin.git\",\"ok\":true}\n",
"hookCode": 0,
"hookStderr": ""
},
"exempt": {
"code": 0,
"stdout": "SEALED: clear findings=0 commits=1 visibility=unknown (counted public) waived=0 exempt=1 policy=origin/main + commit <sha>\n",
"stderr": "gstack-pregate sealed: exempt sealed.path sealed/s1/MANIFEST.json commit <sha> (sealed.allow_paths **/sealed/*/MANIFEST.json)\n",
"jsonCode": 0,
"json": "{\"status\":\"clear\",\"policyLabels\":[\"origin/main\",\"commit <sha>\"],\"commits\":1,\"findings\":[],\"exempt\":[{\"rule\":\"sealed.path\",\"file\":\"sealed/s1/MANIFEST.json\",\"commit\":\"<sha>\",\"glob\":\"**/sealed/*/MANIFEST.json\"}],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"not a GitHub remote\",\"target\":\"<root>/origin.git\",\"ok\":true}\n",
"hookCode": 0,
"hookStderr": "gstack sealed-output guard: exempt sealed.path sealed/s1/MANIFEST.json commit <sha> (sealed.allow_paths **/sealed/*/MANIFEST.json)\n"
},
"waived": {
"code": 0,
"stdout": "SEALED: clear findings=0 commits=1 visibility=unknown (counted public) waived=1 exempt=0 policy=origin/main + commit <sha>\n",
"stderr": "gstack-pregate sealed: waived sealed.transcript runs/c1/transcript.jsonl commit <sha> (.gstack/sealed-allow.tsv: synthetic fixture)\n",
"jsonCode": 0,
"json": "{\"status\":\"clear\",\"policyLabels\":[\"origin/main\",\"commit <sha>\"],\"commits\":1,\"findings\":[],\"exempt\":[],\"waived\":[{\"rule\":\"sealed.transcript\",\"file\":\"runs/c1/transcript.jsonl\",\"commit\":\"<sha>\",\"reason\":\"synthetic fixture\"}],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"not a GitHub remote\",\"target\":\"<root>/origin.git\",\"ok\":true}\n",
"hookCode": 0,
"hookStderr": "gstack sealed-output guard: waived sealed.transcript runs/c1/transcript.jsonl commit <sha> (.gstack/sealed-allow.tsv: synthetic fixture)\n"
},
"warned": {
"code": 0,
"stdout": "SEALED: warned findings=1 commits=1 visibility=private waived=0 exempt=0 policy=origin/main + commit <sha>\n",
"stderr": "gstack-pregate sealed: warning — sealed or eval output in a push to a private destination (visibility: private; sealed.visibility override); it would be refused for a public one:\n sealed.transcript runs/c1/transcript.jsonl commit <sha>\n",
"jsonCode": 0,
"json": "{\"status\":\"warned\",\"policyLabels\":[\"origin/main\",\"commit <sha>\"],\"commits\":1,\"findings\":[{\"rule\":\"sealed.transcript\",\"file\":\"runs/c1/transcript.jsonl\",\"commit\":\"<sha>\"}],\"exempt\":[],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"private\",\"visibilitySource\":\"sealed.visibility override\",\"target\":\"<root>/origin.git\",\"ok\":true}\n",
"hookCode": 0,
"hookStderr": "gstack sealed-output guard: warning — sealed or eval output in a push to a private destination (visibility: private; sealed.visibility override); it would be refused for a public one:\n sealed.transcript runs/c1/transcript.jsonl commit <sha>\n"
},
"error": {
"code": 1,
"stdout": "SEALED: error findings=0 commits=0 visibility=unknown (counted public) waived=0 exempt=0 policy=no sealed key on refs/remotes/origin/main or HEAD\n",
"stderr": "\n⛔ gstack-pregate sealed BLOCKED the publication of <root>/origin.git — the sealed policy in commit <sha> is unreadable: sealed.answer_markers[0]: \"(\" is not a valid regex (Invalid regular expression: missing )).\nFix the sealed policy (gstack-pregate validate), then retry; an unreadable policy never passes (SEALED_OUTPUT_REFUSED)\n",
"jsonCode": 1,
"json": "{\"status\":\"error\",\"error\":\"the sealed policy in commit <sha> is unreadable: sealed.answer_markers[0]: \\\"(\\\" is not a valid regex (Invalid regular expression: missing ))\",\"policyLabels\":[],\"commits\":0,\"findings\":[],\"exempt\":[],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"\",\"target\":\"<root>/origin.git\",\"ok\":false}\n",
"hookCode": 1,
"hookStderr": "\n⛔ gstack sealed-output guard BLOCKED the publication of <root>/origin.git — the sealed policy in commit <sha> is unreadable: sealed.answer_markers[0]: \"(\" is not a valid regex (Invalid regular expression: missing )).\nFix the sealed policy (gstack-pregate validate), then retry; an unreadable policy never passes (SEALED_OUTPUT_REFUSED)\n"
},
"no-policy": {
"code": 0,
"stdout": "SEALED: no-policy findings=0 commits=0 visibility=unknown (counted public) waived=0 exempt=0 policy=no sealed key on refs/remotes/origin/main or HEAD\n",
"stderr": "",
"jsonCode": 0,
"json": "{\"status\":\"no-policy\",\"policyLabels\":[],\"commits\":0,\"findings\":[],\"exempt\":[],\"waived\":[],\"ignoredWaivers\":[],\"visibility\":\"unknown\",\"visibilitySource\":\"\",\"target\":\"<root>/origin.git\",\"ok\":true}\n",
"hookCode": 0,
"hookStderr": ""
}
}
+61
View File
@@ -506,3 +506,64 @@ describe('waivers name the full commit sha (GST-268)', () => {
expect(JSON.parse(preflight(s, ['--range', 'nosuch..HEAD', '--json']).out).ignoredWaivers).toEqual([]);
});
});
const PANEL = JSON.stringify({ units: [
{ stratum: 'decoy', tags: ['W:T1'], plan_bytes: 3760, files: 5, head: '5b7db3aa' + '1'.repeat(32), tree_sha256: '15762a' + '0'.repeat(56) + 'aa' },
{ stratum: 'seeded', tags: ['B:T2'], plan_bytes: 4112, files: 7, head: '9c1e0bbb' + '2'.repeat(32), tree_sha256: '88aa11' + '0'.repeat(56) + 'bb' },
] }, null, 2) + '\n';
const PANEL_VALUES = [/5b7db3aa/, /15762a0/, /9c1e0bbb/, /88aa110/, /\b3760\b/, /\b4112\b/, /W:T1/, /B:T2/, /\bdecoy\b/, /\bseeded\b/];
describe('sealed.label_join: labels published beside self-computable identifiers (GST-620)', () => {
test('the GSTA-2 panel (stratum and tags beside head, tree_sha256, plan_bytes, files) refuses on a public destination and warns on a private one, naming fields, never values', () => {
const labeled = { paths: ['**/heldout*/**'], label_keys: ['stratum', 'tags', 'decoy_set'] };
const s = scratch({ '.gstack/pregate.json': policy(labeled) });
const sha = commit(s, { 'wiki/board/panel/panel.json': PANEL });
const keys = ['stratum', 'tags', 'files', 'head', 'plan_bytes', 'tree_sha256'];
const text = preflight(s);
expect(text.code).toBe(1);
expect(text.err).toContain(`sealed.label_join wiki/board/panel/panel.json commit ${sha.slice(0, 12)} keys=${keys.join(',')}`);
expect(text.out).toMatch(/^SEALED: refused findings=1 commits=1 /m);
const json = preflight(s, ['--json']);
expect(JSON.parse(json.out).findings).toEqual([{ rule: 'sealed.label_join', file: 'wiki/board/panel/panel.json', commit: sha, keys }]);
const h = hook(s);
expect(h.code).toBe(1);
expect(h.err).toContain(`sealed.label_join wiki/board/panel/panel.json commit ${sha.slice(0, 12)} keys=${keys.join(',')}`);
for (const out of [text.out, text.err, json.out, h.err]) for (const value of PANEL_VALUES) expect(out).not.toMatch(value);
const priv = scratch({ '.gstack/pregate.json': policy({ ...labeled, visibility: 'private' }) });
commit(priv, { 'wiki/board/panel/panel.json': PANEL });
const p = preflight(priv);
expect(p.code).toBe(0);
expect(p.err).toMatch(/warning.*sealed\.label_join\s+wiki\/board\/panel\/panel\.json/s);
});
});
const OFF_GOLDEN = path.join(ROOT, 'test', 'fixtures', 'sealed-guard-off-golden.json');
/** Representative pushes under policies without label_keys, each read three ways: the preflight's text and --json, and the pre-push hook. */
function offRuns(): Record<string, unknown> {
const table = '| unit | stratum | head |\n|---|---|---|\n| u1 | decoy | 5b7db3a |\n';
const fixtures: Record<string, () => Scratch> = {
refused: () => { const s = scratch({ '.gstack/pregate.json': policy(SEALED) }); commit(s, { 'wiki/board/panel/panel.json': PANEL, 'runs/c1/transcript.jsonl': '{}\n', 'runs/c1/summary.md': 'SEALED ANSWER: x\n' + table }); return s; },
clear: () => { const s = scratch({ '.gstack/pregate.json': policy({ paths: ['heldout/**'] }) }); commit(s, { 'wiki/board/panel/panel.json': PANEL, 'wiki/board.md': table, 'wiki/units.csv': 'stratum,head\ndecoy,5b7db3a\n' }); return s; },
exempt: () => { const s = scratch({ '.gstack/pregate.json': policy(MANIFEST) }); commit(s, { 'sealed/s1/MANIFEST.json': PANEL, 'wiki/board/panel/panel.json': PANEL }); return s; },
waived: () => { const s = scratch({ '.gstack/pregate.json': policy(SEALED) }); const sha = commit(s, { 'runs/c1/transcript.jsonl': '{}\n', 'wiki/panel.json': PANEL }); landOnMain(s, { '.gstack/sealed-allow.tsv': `sealed.transcript\truns/c1/transcript.jsonl\t${sha}\tsynthetic fixture\n` }); return s; },
warned: () => { const s = scratch({ '.gstack/pregate.json': policy({ ...SEALED, visibility: 'private' }) }); commit(s, { 'runs/c1/transcript.jsonl': '{}\n', 'wiki/panel.json': PANEL }); return s; },
error: () => { const s = scratch({ '.gstack/pregate.json': policy(SEALED) }); commit(s, { '.gstack/pregate.json': policy({ ...SEALED, answer_markers: ['('] }), 'wiki/panel.json': PANEL }); return s; },
'no-policy': () => { const s = scratch({}); commit(s, { 'wiki/panel.json': PANEL }); return s; },
};
const runs: Record<string, unknown> = {};
for (const [name, make] of Object.entries(fixtures)) {
const s = make();
const norm = (t: string) => t.split(s.root).join('<root>').replace(/[0-9a-f]{12,64}/g, '<sha>');
const text = preflight(s); const json = preflight(s, ['--json']); const h = hook(s);
runs[name] = { code: text.code, stdout: norm(text.out), stderr: norm(text.err), jsonCode: json.code, json: norm(json.out), hookCode: h.code, hookStderr: norm(h.err) };
}
return runs;
}
describe('off unless configured (GST-620 item 13)', () => {
test('without sealed.label_keys, stdout, stderr, --json, the hook and every exit code match the capture from main 2d1b55d76 byte for byte', () => {
const runs = offRuns();
if (process.env.SEALED_GUARD_GOLDEN_CAPTURE) { fs.writeFileSync(process.env.SEALED_GUARD_GOLDEN_CAPTURE, JSON.stringify(runs, null, 2) + '\n'); return; }
expect(runs).toEqual(JSON.parse(fs.readFileSync(OFF_GOLDEN, 'utf8')));
}, 120_000);
});
+225
View File
@@ -0,0 +1,225 @@
/**
* GST-620: `sealed.label_join`, the sealed guard's rule for a published
* record that puts a label field (`sealed.label_keys`) next to an identifier
* the model under test can compute from its own copy of the work. The repro
* and the byte-for-byte "off" golden live in test/sealed-guard.test.ts; this
* file covers formats, the per-record content diff against every parent,
* fail-closed unchecked files, waivers, allow_paths and the monotonic policy.
*/
import { afterEach, describe, expect, test } from 'bun:test';
import { spawnSync } from 'node:child_process';
import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { sealedErrors } from '../lib/sealed-guard';
import { DEFAULT_IDENTIFIER_KEYS, joinFindings } from '../lib/sealed-join';
const ROOT = path.resolve(import.meta.dir, '..');
const PREGATE = path.join(ROOT, 'bin', 'gstack-pregate');
const HEAD_A = '5b7db3aa' + '1'.repeat(32);
const HEAD_B = '9c1e0bbb' + '2'.repeat(32);
const LABELED = { label_keys: ['stratum', 'tags', 'decoy_set'] };
const P = { label_keys: ['stratum', 'tags'], identifier_keys: DEFAULT_IDENTIFIER_KEYS };
interface Scratch { root: string; work: string; env: Record<string, string> }
const scratches: Scratch[] = [];
afterEach(() => { while (scratches.length) fs.rmSync(scratches.pop()!.root, { recursive: true, force: true }); });
function git(s: Scratch, args: string[], ok = true): string {
const r = spawnSync('git', args, { cwd: s.work, encoding: 'utf8', timeout: 30_000, env: { ...process.env, ...s.env } });
if (ok && r.status !== 0) throw new Error(`git ${args.join(' ')}: ${r.stderr}`);
return (r.stdout ?? '').trim();
}
function write(s: Scratch, files: Record<string, string | null>): void {
for (const [rel, body] of Object.entries(files)) {
const abs = path.join(s.work, rel);
if (body === null) { fs.rmSync(abs, { force: true }); continue; }
fs.mkdirSync(path.dirname(abs), { recursive: true });
fs.writeFileSync(abs, body);
}
}
function commit(s: Scratch, files: Record<string, string | null>, msg = 'results'): string {
write(s, files);
git(s, ['add', '-A']); git(s, ['commit', '-q', '-m', msg]);
return git(s, ['rev-parse', 'HEAD']);
}
/** A bare origin whose main carries the base files, and a clone on branch `results`. */
function scratch(baseFiles: Record<string, string>): Scratch {
const root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'sealed-join-')));
const work = path.join(root, 'work'); const origin = path.join(root, 'origin.git'); const state = path.join(root, 'state');
fs.mkdirSync(work); fs.mkdirSync(state);
const env = { HOME: root, GSTACK_HOME: state, GSTACK_STATE_ROOT: state, GIT_CONFIG_GLOBAL: path.join(root, 'gitconfig'), GIT_CONFIG_NOSYSTEM: '1' };
fs.writeFileSync(env.GIT_CONFIG_GLOBAL, '[user]\n\tname = t\n\temail = t@example.com\n[commit]\n\tgpgsign = false\n[init]\n\tdefaultBranch = main\n');
const s = { root, work, env };
scratches.push(s);
spawnSync('git', ['init', '-q', '--bare', origin], { env: { ...process.env, ...env }, timeout: 30_000 });
git(s, ['init', '-q', '-b', 'main']);
commit(s, { 'README.md': '# evals\n', ...baseFiles }, 'base');
git(s, ['remote', 'add', 'origin', origin]); git(s, ['push', '-q', 'origin', 'main']);
git(s, ['checkout', '-q', '-b', 'results']);
return s;
}
function landOnMain(s: Scratch, files: Record<string, string>): void {
git(s, ['checkout', '-q', 'main']); commit(s, files, 'base change'); git(s, ['push', '-q', 'origin', 'main']); git(s, ['checkout', '-q', 'results']);
}
const policy = (sealed: unknown) => JSON.stringify({ sealed }, null, 2) + '\n';
const panel = (units: object[]) => JSON.stringify({ units }, null, 2) + '\n';
const unit = (name: string, head: string, extra: object = {}) => ({ name, stratum: name === 'u1' ? 'decoy' : 'seeded', tags: ['W:T1'], head, plan_bytes: 3760, ...extra });
const found = (s: Scratch) => joinFindings(s.work, git(s, ['rev-list', 'origin/main..HEAD']).split('\n').filter(Boolean), P);
function preflight(s: Scratch, args: string[] = []) {
const r = spawnSync(PREGATE, ['sealed', '--base', 'main', ...args], { cwd: s.work, encoding: 'utf8', timeout: 60_000, env: { ...process.env, ...s.env } });
return { code: r.status, out: r.stdout ?? '', err: r.stderr ?? '' };
}
describe('formats and matching', () => {
test('a JSONL line, a nested meta.head, CSV, TSV, a markdown table with human headers and a hex value under an unlisted key each join; decimal-only strings, labels alone and identifiers alone do not', () => {
const s = scratch({});
const sha = commit(s, {
'a.jsonl': '{"x":1}\n{"stratum":"decoy","meta":{"head":"5b7db3a"}}\n',
'b.csv': 'unit,Stratum,Plan Bytes\nu1,decoy,"3,760"\n',
'c.tsv': 'stratum\ttree_sha256\ndecoy\tabc\n',
'd.md': '# board\n\n| Unit | Stratum | `head` | Tree SHA-256 |\n|---|:---:|---|---|\n| u1 | decoy | x | y |\n',
'e.json': '[{"tags":["B:T2"],"id":"5b7db3a"}]\n',
'f.json': '[{"stratum":"decoy","ts":"202610311200","name":"unit-1","count":12}]\n',
'g.json': '[{"stratum":"decoy"},{"head":"5b7db3aa"}]\n',
'h.md': '| head | size |\n|---|---|\n| 5b7db3a | 3 |\n',
'i.txt': '{"stratum":"decoy","head":"5b7db3a"}\n',
});
expect(found(s).sort((a, b) => a.file.localeCompare(b.file))).toEqual([
{ rule: 'sealed.label_join', file: 'a.jsonl', commit: sha, keys: ['stratum', 'meta.head'] },
{ rule: 'sealed.label_join', file: 'b.csv', commit: sha, keys: ['stratum', 'plan_bytes'] },
{ rule: 'sealed.label_join', file: 'c.tsv', commit: sha, keys: ['stratum', 'tree_sha256'] },
{ rule: 'sealed.label_join', file: 'd.md', commit: sha, keys: ['stratum', 'head', 'tree_sha_256'] },
{ rule: 'sealed.label_join', file: 'e.json', commit: sha, keys: ['tags', 'id'] },
]);
});
test('without label_keys the rule reads nothing: no git call, even outside a repository', () => {
expect(joinFindings(os.tmpdir(), ['0'.repeat(40)], { label_keys: [], identifier_keys: DEFAULT_IDENTIFIER_KEYS })).toEqual([]);
});
test('label_keys must be plain field names and identifier_keys allow only *; both must be string arrays', () => {
expect(sealedErrors({ label_keys: ['stratum'], identifier_keys: ['*_sha', 'head'] })).toEqual([]);
expect(sealedErrors({ label_keys: ['strat*'] }).map(e => e.path)).toEqual(['sealed.label_keys[0]']);
expect(sealedErrors({ identifier_keys: ['he?d', ''] }).map(e => e.path)).toEqual(['sealed.identifier_keys[0]', 'sealed.identifier_keys[1]']);
expect(sealedErrors({ label_keys: 'stratum' })).toEqual([{ path: 'sealed.label_keys', message: 'must be an array of strings' }]);
});
});
describe('content diff against every parent', () => {
test('an unchanged record and a formatting-only edit stay clear; a replacement at the same count, a delete plus add and an added identifier field refuse', () => {
const s = scratch({ 'wiki/panel.json': panel([unit('u1', HEAD_A), unit('u2', HEAD_B)]) });
const reformat = commit(s, { 'wiki/panel.json': JSON.stringify({ units: [unit('u1', HEAD_A), unit('u2', HEAD_B), { note: 'no labels here' }] }) + '\n' });
const replaced = commit(s, { 'wiki/panel.json': panel([unit('u1', HEAD_A), unit('u2', 'c'.repeat(39) + '3')]) });
const swapped = commit(s, { 'wiki/panel.json': panel([unit('u2', 'c'.repeat(39) + '3'), unit('u3', 'd'.repeat(39) + '4')]) });
const widened = commit(s, { 'wiki/panel.json': panel([unit('u2', 'c'.repeat(39) + '3'), unit('u3', 'd'.repeat(39) + '4', { tree_sha256: 'e'.repeat(63) + '5' })]) });
const commits = found(s).map(f => f.commit);
expect(commits).not.toContain(reformat);
expect(commits.sort()).toEqual([replaced, swapped, widened].sort());
});
test('labels in one commit and identifiers added to the same records in the next: the second commit refuses', () => {
const s = scratch({});
const labels = commit(s, { 'units.jsonl': '{"name":"u1","stratum":"decoy"}\n' });
const ids = commit(s, { 'units.jsonl': '{"name":"u1","stratum":"decoy","head":"5b7db3a"}\n' });
expect(found(s).map(f => f.commit)).toEqual([ids]);
expect(found(s).map(f => f.commit)).not.toContain(labels);
});
test('a pure rename is clear and a rename that adds a unit refuses; a clean merge of two published sides is clear and a resolution that adds a joined row refuses', () => {
const units = ['u1', 'u2', 'u3', 'u4', 'u5', 'u6'].map((n, i) => unit(n, String(i + 1).repeat(4) + 'ab' + 'f'.repeat(34)));
const s = scratch({ 'wiki/panel.json': panel(units) });
git(s, ['mv', 'wiki/panel.json', 'wiki/board.json']); git(s, ['commit', '-q', '-m', 'rename']);
write(s, { 'wiki/board.json': panel([...units, unit('u7', HEAD_A)]) });
git(s, ['mv', 'wiki/board.json', 'wiki/board2.json']); git(s, ['add', '-A']); git(s, ['commit', '-q', '-m', 'rename and add']);
const renamedAndAdded = git(s, ['rev-parse', 'HEAD']);
expect(found(s).map(f => f.commit)).toEqual([renamedAndAdded]);
const m = scratch({ 'wiki/panel.json': panel([unit('u1', HEAD_A)]) });
git(m, ['checkout', '-q', '-b', 'side']);
const side = commit(m, { 'wiki/panel.json': panel([unit('u1', HEAD_A), unit('u2', HEAD_B)]) });
git(m, ['checkout', '-q', 'results']);
const ours = commit(m, { 'wiki/panel.json': panel([unit('u3', 'c'.repeat(39) + '3'), unit('u1', HEAD_A)]) });
const merge = (resolution: object[]) => {
git(m, ['merge', '-q', '--no-ff', '--no-commit', 'side'], false);
write(m, { 'wiki/panel.json': panel(resolution) });
git(m, ['add', '-A']); git(m, ['commit', '-q', '--no-edit', '-m', 'merge']);
return git(m, ['rev-parse', 'HEAD']);
};
const clean = merge([unit('u3', 'c'.repeat(39) + '3'), unit('u1', HEAD_A), unit('u2', HEAD_B)]);
expect(found(m).map(f => f.commit).sort()).toEqual([side, ours].sort());
expect(found(m).map(f => f.commit)).not.toContain(clean);
git(m, ['reset', '-q', '--hard', ours]);
const added = merge([unit('u3', 'c'.repeat(39) + '3'), unit('u1', HEAD_A), unit('u2', HEAD_B), unit('u4', 'd'.repeat(39) + '4')]);
const atMerge = found(m).filter(f => f.commit === added);
expect(atMerge).toEqual([{ rule: 'sealed.label_join', file: 'wiki/panel.json', commit: added, keys: ['stratum', 'tags', 'head', 'plan_bytes'] }]);
});
});
describe('fail closed, waivers, allow_paths and the policy union (gstack-pregate sealed)', () => {
test('an oversized valid JSON, malformed JSON, a malformed JSONL line, nesting deeper than 64 and an unreadable blob are each sealed.label_join_unchecked; an empty file is clear', () => {
const s = scratch({ '.gstack/pregate.json': policy(LABELED) });
let deep: unknown = { stratum: 'decoy' };
for (let i = 0; i < 70; i++) deep = { next: deep };
const sha = commit(s, {
'big.json': JSON.stringify({ pad: 'x'.repeat(4 * 1024 * 1024 + 10), units: [] }),
'bad.json': '{"stratum": "decoy",',
'bad.jsonl': '{"stratum":"decoy"}\nnot json\n',
'deep.json': JSON.stringify(deep),
'gone.json': '{"note":"this blob is removed after the commit"}\n',
'empty.json': '',
});
const blob = git(s, ['rev-parse', 'HEAD:gone.json']);
fs.rmSync(path.join(s.work, '.git', 'objects', blob.slice(0, 2), blob.slice(2)));
const r = preflight(s);
expect(r.code).toBe(1);
for (const f of ['big.json', 'bad.json', 'bad.jsonl', 'deep.json', 'gone.json']) expect(r.err).toContain(`sealed.label_join_unchecked ${f} commit ${sha.slice(0, 12)}`);
expect(r.err).not.toContain('empty.json');
expect(r.out).toMatch(/^SEALED: refused findings=5 commits=1 .* exempt=0 unchecked=5 policy=/m);
const json = JSON.parse(preflight(s, ['--json']).out);
expect(json.findings.map((f: { rule: string; file: string }) => `${f.rule} ${f.file}`).sort()).toEqual(['bad.json', 'bad.jsonl', 'big.json', 'deep.json', 'gone.json'].map(f => `sealed.label_join_unchecked ${f}`));
});
test('allow_paths never exempt the rule; a short-sha waiver is ignored and the full sha waives', () => {
const s = scratch({ '.gstack/pregate.json': policy({ ...LABELED, paths: ['heldout/**'], allow_paths: ['wiki/board/panel.json'] }) });
const sha = commit(s, { 'wiki/board/panel.json': panel([unit('u1', HEAD_A)]) });
const r = preflight(s);
expect(r.code).toBe(1);
expect(r.out).toMatch(/findings=1 .* exempt=0 policy=/);
landOnMain(s, { '.gstack/sealed-allow.tsv': `sealed.label_join\twiki/board/panel.json\t${sha.slice(0, 12)}\tsynthetic fixture\n` });
const short = preflight(s);
expect(short.code).toBe(1);
expect(short.err).toContain('ignored waiver sealed.label_join wiki/board/panel.json');
landOnMain(s, { '.gstack/sealed-allow.tsv': `sealed.label_join\twiki/board/panel.json\t${sha}\tsynthetic fixture\n` });
const full = preflight(s);
expect(full.code).toBe(0);
expect(full.err).toContain('waived sealed.label_join wiki/board/panel.json');
});
test('a push can enable or widen the rule but never disable or narrow it', () => {
const leak = { 'wiki/panel.json': panel([unit('u1', HEAD_A)]) };
for (const pushed of [{ paths: ['heldout/**'] }, { ...LABELED, identifier_keys: ['zzz'] }, { label_keys: [] as string[], identifier_keys: [] as string[] }]) {
const s = scratch({ '.gstack/pregate.json': policy(LABELED) });
commit(s, { '.gstack/pregate.json': policy(pushed), ...leak });
const r = preflight(s);
expect(r.code).toBe(1);
expect(r.err).toContain('sealed.label_join wiki/panel.json');
expect(r.err).toContain('head,plan_bytes');
}
const enable = scratch({ '.gstack/pregate.json': policy({ paths: ['heldout/**'] }) });
commit(enable, { '.gstack/pregate.json': policy(LABELED), ...leak });
expect(preflight(enable).err).toContain('sealed.label_join wiki/panel.json');
});
test('label_keys: [] reports exactly what an absent label_keys reports', () => {
const run = (sealed: object) => {
const s = scratch({ '.gstack/pregate.json': policy(sealed) });
commit(s, { 'wiki/panel.json': panel([unit('u1', HEAD_A)]), 'runs/c1/transcript.jsonl': '{}\n' });
const norm = (t: string) => t.split(s.root).join('<root>').replace(/[0-9a-f]{12,64}/g, '<sha>');
const text = preflight(s); const json = preflight(s, ['--json']);
return [text.code, norm(text.out), norm(text.err), json.code, norm(json.out)];
};
const base = { transcripts: ['runs/**/transcript*.jsonl'] };
expect(run({ ...base, label_keys: [] })).toEqual(run(base));
});
});