mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-31 22:14:25 +01:00
* fix(land-and-deploy): never merge over red, pending or missing CI (#2995) /land-and-deploy gated only on required checks, so a repo with none (every GitHub Free private repo, any repo without branch protection) skipped the CI wait and could merge over red CI. The verdict now comes from bin/gstack-ci-gate, a tested helper bound to the head SHA: all checks by default, cancel/unknown buckets fail, NO_CHECKS needs explicit per-head approval, and both merge commands (including --auto) run only after a PASS for the same SHA they pin with --match-head-commit. Co-authored-by: harshitgavita-07 <harshit.gavita@gmail.com> * fix(slug): give nested-group projects their own state bucket and verify checkpoint identity (#3003, #2901) gstack-slug kept only the last two remote path segments, so customer-a/product/repo and customer-b/product/repo shared one bucket and /context-restore could present another project's checkpoint. - One canonical-remote rule (bash + TS, golden vectors) used by all six resolvers: lowercased host, no userinfo/port/.git, Azure DevOps and Bitbucket Server normalized. 3+ segment remotes get <last-two>-<16 hex>; two-segment remotes (all of GitHub) are unchanged. - Versioned sticky cache (v2:) keeps cache hits git-free. - context-save stamps the credential-free remote and root; context-restore only treats this project's checkpoints as latest, reports foreign/unverified ones first, and treats a different root with the same remote (Conductor handoff) as information. - gstack-slug --adopt-legacy copies only files the user confirms. - question-preference-hook reads the same versioned cache, so never-ask preferences fire; unknown identity asks instead of auto-deciding. * fix(gbrain): brain context loads, managed sync works, memory sync stops duplicating (#2883, #2985, #2922) - brain-context-load called 'gbrain list_pages' (an MCP tool name), so every skill's brain context was 0 bytes. It now uses 'gbrain list' with gbrain's real filter flags, and prints one line when queries fail. - The code-stage sync always passes --no-pull (managed gbrain >= 0.51 refuses without it); a gbrain that rejects the flag fails with an upgrade message and is never retried without it. stderr is streamed with a bounded tail and quoted in failure summaries. - /sync-gbrain gains --sources (and GSTACK_MEMORY_INGEST_SOURCES); by default it skips types already synced through a federated curated source, so curated pages stop being re-duplicated. Co-authored-by: Vitor Cepeda Lopes <vitor@vitorcepedalopes.com> Co-authored-by: harshitgavita-07 <harshit.gavita@gmail.com> * fix(codex,cso): clean Codex reviews stop failing; semgrep and trivy SARIF results import (#2769, #3011) - Codex review gate is PASS / FAIL / UNVERIFIED. Untagged output (what a clean 'codex review --base' prints) is UNVERIFIED, not FAIL, and never a false PASS; P2/P3-only output passes. - import-sarif resolves semgrep's undeclared %SRCROOT% and trivy's ROOTPATH (including git-URL targets) through explicit, captured mappings; declared bases must sit inside the source root or the run's checkout, URIs are percent-decoded, and rejected results are counted with rule, base and path instead of silently dropped. * fix: version strings are not IPs, failed update checks are not 'up to date', private security reporting (#2856, #2786, #2984) - The pre-push redact hook exempts a root VERSION file that is only a version string and the version token in CHANGELOG release headings; the engine allowlist is untouched. - gstack-update-check caches an unreadable remote as CHECK_FAILED with a short TTL instead of UP_TO_DATE; /gstack-upgrade never reports 'latest' after a failed check. - SECURITY.md points reporters at GitHub private vulnerability reporting. - pr-title-sync (pull_request_target, write token) pins actions/checkout to a reviewed SHA. * fix(autoplan): publication guard accepts real Claude Code journals (#3007, #3009, #2968, #2977, #2986, #3006) The guard required a session journal to start with a user message, but Claude Code starts it with SessionStart hook attachments (and a forked compacted session starts with a compact_boundary), so every phase was denied for anyone with a SessionStart hook. Windows path spellings (C:/, /c/, drive case) were rejected too. - Reader accepts the captured startup, compact, clear and fork shapes (claude 2.1.284 fixture), computes root uniqueness over every candidate, and returns typed reason codes instead of one string. - Hard denials (competing root, foreign cwd, sidechain, agent, cycle, malformed, identity) say what happened and how to run the reviews by hand; unflushed journals still retry. - A well-formed, stable journal whose root shape is unrecognized now warns ('phase publication was NOT verified') and continues, without overriding Claude Code's own permission check; no bypass switch. - Windows paths fold before validation, so '..' is still rejected. - Periodic drift canary runs the reader on fresh journals from the pinned claude. Co-authored-by: Charles M Berg <77517285+CharlesBerg85@users.noreply.github.com> Co-authored-by: laurenbdaniels-coder <264343335+laurenbdaniels-coder@users.noreply.github.com> Co-authored-by: Jordan Mayer <176445995+maydaycyber@users.noreply.github.com> Co-authored-by: elmesery-dev <elmesery.dev@gmail.com> * v1.91.13.0: severe bug fix wave release notes, stale gbrain verbs in docs * test: seed the versioned slug cache in the memory-injection deny case (#2901) The preference hook now resolves the project bucket from gstack-slug's v2 slug-cache entry instead of basename(cwd); this fixture still wrote preferences under the basename bucket without the entry the skill preamble always writes first. * fix: full-suite follow-ups for the fix wave - gstack-slug's --help heredoc is over 512B, so it gets the BASH_COMPAT=50 pipe-deadlock guard like the other bins. - Refresh the codex/factory ship goldens for the plan-gates slug change. - The periodic lane gains the autoplan journal drift canary (107 paid files); raise the eval-slices cap to 25 so every slice still starts at once until its recorded duration lets the planner pack it. * fix(plan-eng-review): clearer Step 0 sequencing, write routes and section outputs The workflow judge scored plan-eng-review's instructions at actionability 3.67 (bar 4) on text unchanged from main. Resolve the ambiguities it named together, without changing any gate or output: Step 0 names the Read's order and keeps one STOP; the pending-call, read-only, forbidden-write and failed-save rules become one Write routes block; header/label limits and grid checks are concrete; Performance gets a checklist; each section has a stated output shape; Lake Score is defined. Same total length (parity cap). --------- Co-authored-by: harshitgavita-07 <harshit.gavita@gmail.com> Co-authored-by: Vitor Cepeda Lopes <vitor@vitorcepedalopes.com> Co-authored-by: Charles M Berg <77517285+CharlesBerg85@users.noreply.github.com> Co-authored-by: laurenbdaniels-coder <264343335+laurenbdaniels-coder@users.noreply.github.com> Co-authored-by: Jordan Mayer <176445995+maydaycyber@users.noreply.github.com> Co-authored-by: elmesery-dev <elmesery.dev@gmail.com>
577 lines
26 KiB
TypeScript
Executable File
577 lines
26 KiB
TypeScript
Executable File
#!/usr/bin/env bun
|
|
/**
|
|
* gstack-redact-prepush — git pre-push hook that scans the diff being pushed for
|
|
* HIGH-severity credentials and blocks the push on a hit.
|
|
*
|
|
* THIS IS A GUARDRAIL, NOT ENFORCEMENT. `git push --no-verify` bypasses it, as
|
|
* does `GSTACK_REDACT_PREPUSH=skip`. It catches accidental credential pushes,
|
|
* the most common real-world leak. It does NOT scan history, binary/LFS/submodule
|
|
* files, or non-added lines. History scanning is /cso's job.
|
|
*
|
|
* Git pre-push interface: refs are read from STDIN, one per line:
|
|
* <local ref> <local sha> <remote ref> <remote sha>
|
|
* We scan the ADDED lines of <remote sha>..<local sha> per ref (what's being
|
|
* pushed). Special cases:
|
|
* - remote sha all-zeroes → new branch: diff against merge-base with the
|
|
* remote's default branch (fallback: scan all commits unique to local ref).
|
|
* - local sha all-zeroes → branch delete: nothing to scan, skip.
|
|
* - force-push → remote..local still gives the net new content.
|
|
*
|
|
* Behavior:
|
|
* - HIGH finding in added lines → print + exit 1 (block), for public AND private.
|
|
* - MEDIUM → warn (non-blocking). LOW/WARN → silent.
|
|
* - GSTACK_REDACT_PREPUSH=skip → log + exit 0 (escape valve).
|
|
*
|
|
* Installed/uninstalled via `gstack-redact install-prepush-hook` (see the
|
|
* gstack-redact CLI), which chains any pre-existing hook.
|
|
*/
|
|
import { spawnSync } from "child_process";
|
|
import * as fs from "fs";
|
|
import * as path from "path";
|
|
import { normalizeWithMap, scan, type Finding } from "../lib/redact-engine";
|
|
import { mkdirpSync } from "../lib/fs-utils";
|
|
import { resolveStateRoot } from "../lib/state-root";
|
|
|
|
const ZERO = /^0+$/;
|
|
let emptyTree: string | undefined;
|
|
function emptyTreeOid(): string {
|
|
if (emptyTree) return emptyTree;
|
|
const oid = gitStrict(["hash-object", "-w", "-t", "tree", "--stdin"]).trim();
|
|
if (!/^(?:[0-9a-f]{40}|[0-9a-f]{64})$/.test(oid)) throw new Error("git could not resolve the empty tree");
|
|
emptyTree = oid;
|
|
return oid;
|
|
}
|
|
|
|
/**
|
|
* Permissive git for legitimately-fallible PROBES (symbolic-ref, rev-parse,
|
|
* merge-base) where a non-zero exit is normal control flow. The DIFF call
|
|
* must NOT use this — see gitStrict (#1946 fail-closed).
|
|
*/
|
|
function git(args: string[]): string {
|
|
const r = spawnSync("git", args, { encoding: "utf8", maxBuffer: 64 * 1024 * 1024 });
|
|
return r.status === 0 ? (r.stdout ?? "") : "";
|
|
}
|
|
|
|
/**
|
|
* Fail-closed git for the diff that decides whether the push is scanned
|
|
* (#1946). status !== 0 covers repo errors; status === null covers a killed
|
|
* process AND maxBuffer overflow — the oversized-diff case is exactly where
|
|
* a large secret-bearing blob is most likely, so "couldn't read the diff"
|
|
* must block, not silently allow.
|
|
*/
|
|
function gitStrict(args: string[]): string {
|
|
const r = spawnSync("git", args, { encoding: "utf8", maxBuffer: 64 * 1024 * 1024 });
|
|
// status !== 0 covers BOTH a non-zero exit AND null (process killed by a
|
|
// signal or maxBuffer overflow — null !== 0 is true).
|
|
if (r.status !== 0) {
|
|
throw new Error(
|
|
`git ${args[0]} failed (status=${r.status ?? "killed/overflow"}): ${(r.stderr ?? "").slice(0, 300)}`,
|
|
);
|
|
}
|
|
return r.stdout ?? "";
|
|
}
|
|
|
|
/** True when the object exists in the local odb (cat-file -e signals via exit code). */
|
|
function objectExists(sha: string): boolean {
|
|
const r = spawnSync("git", ["cat-file", "-e", sha], { encoding: "utf8" });
|
|
return r.status === 0;
|
|
}
|
|
|
|
/**
|
|
* The remote-tracking exclusion used when narrowing to "commits new to the
|
|
* remote" (#2592 catch-up merges, #2573 rebased force-pushes).
|
|
*
|
|
* Narrowed to the PUSH TARGET's namespace (S1): a bare `--remotes` excludes
|
|
* commits reachable from ANY remote-tracking ref, so a secret that had only
|
|
* ever been fetched from (or pushed to) a private/local-path remote was never
|
|
* scanned when later pushed to a PUBLIC remote — "already left this machine"
|
|
* is not "already reached THIS remote". Git hands pre-push the push remote's
|
|
* name as $1 (and its URL as $2); the installed hook wrapper forwards "$@".
|
|
* A URL push has no tracking namespace, even if origin happens to contain
|
|
* the same commits. Direct invocations without Git's argv keep the historical
|
|
* all-remotes behavior for compatibility.
|
|
*/
|
|
type PushTarget = { kind: "remote"; name: string } | { kind: "url" } | { kind: "unknown" };
|
|
let cachedPushTarget: PushTarget | undefined;
|
|
function pushTarget(): PushTarget {
|
|
if (cachedPushTarget) return cachedPushTarget;
|
|
const name = process.argv[2];
|
|
const url = process.argv[3];
|
|
if (!name && !url) return cachedPushTarget = { kind: "unknown" };
|
|
if (!name || !url) return cachedPushTarget = { kind: "url" };
|
|
const remotes = git(["remote"]).split("\n");
|
|
if (!remotes.includes(name)) return cachedPushTarget = { kind: "url" };
|
|
const pushUrls = git(["remote", "get-url", "--push", "--all", name]).trim().split("\n");
|
|
const fetchUrls = git(["remote", "get-url", "--all", name]).trim().split("\n");
|
|
return cachedPushTarget = pushUrls.includes(url) && fetchUrls[0] === url
|
|
? { kind: "remote", name }
|
|
: { kind: "url" };
|
|
}
|
|
|
|
function remotesExclusionArgs(): string[] {
|
|
const target = pushTarget();
|
|
if (target.kind === "remote") return [`--remotes=${target.name}/*`];
|
|
return target.kind === "unknown" ? ["--remotes"] : [];
|
|
}
|
|
|
|
function defaultRemoteBranch(): string | null {
|
|
const target = pushTarget();
|
|
if (target.kind === "url") return null;
|
|
const remote = target.kind === "remote" ? target.name : "origin";
|
|
const sym = git(["symbolic-ref", `refs/remotes/${remote}/HEAD`]).trim();
|
|
if (sym.startsWith(`refs/remotes/${remote}/`) && git(["rev-parse", "--verify", `${sym}^{commit}`]).trim()) return sym;
|
|
for (const b of [`${remote}/main`, `${remote}/master`]) {
|
|
const qualified = `refs/remotes/${b}`;
|
|
if (git(["rev-parse", "--verify", `${qualified}^{commit}`]).trim()) return qualified;
|
|
}
|
|
return null;
|
|
}
|
|
|
|
/**
|
|
* Base commit for a new remote ref, ordered from most precise to most
|
|
* conservative. Returns null when nothing can anchor the range.
|
|
*/
|
|
function unknownRemoteTipBase(localSha: string): string | null {
|
|
// 1. The common case: a merge-base with the remote's default branch.
|
|
const defaultBranch = defaultRemoteBranch();
|
|
const base = defaultBranch ? git(["merge-base", localSha, defaultBranch]).trim() : "";
|
|
if (base) return base;
|
|
|
|
// 2. No merge-base. The target's default branch may be unavailable
|
|
// (named trunk/develop, remote/HEAD unset), or history may be disjoint.
|
|
// Anything reachable from localSha but from NO remote-tracking
|
|
// branch is what this push actually adds; the parent of its oldest commit
|
|
// is the real base.
|
|
//
|
|
// Without this we drop straight to EMPTY_TREE and re-scan content that is
|
|
// already on the remote. That is not merely wasteful, it is wrong in two
|
|
// ways: a secret pushed long ago gets re-reported as if THIS push
|
|
// introduced it (telling the operator to rotate a key over someone else's
|
|
// old commit), and on any real repository the input overshoots the
|
|
// engine's byte cap, so `engine.input_too_large` blocks having scanned
|
|
// NOTHING — "scans more, never less" inverted into "scans nothing".
|
|
//
|
|
// The exclusion is scoped to the PUSH TARGET's tracking refs (see
|
|
// remotesExclusionArgs): content on some OTHER remote has left this machine,
|
|
// but it has not reached the remote being pushed to — a secret that only
|
|
// ever hit a private remote must still be scanned on its way to a public
|
|
// one (S1).
|
|
const exclusions = remotesExclusionArgs();
|
|
const newCommits = git(["rev-list", "--reverse", localSha, ...(exclusions.length ? ["--not", ...exclusions] : [])]).trim();
|
|
if (newCommits) {
|
|
const oldest = newCommits.split("\n")[0];
|
|
const parent = git(["rev-parse", "--verify", `${oldest}^`]).trim();
|
|
if (parent) return parent;
|
|
// Oldest new commit is a root commit: there is no parent to anchor on.
|
|
}
|
|
|
|
// 3. Nothing to anchor on — a genuinely fresh repository with no remote refs.
|
|
// Every commit IS new content, so scanning it all is the correct answer.
|
|
return null;
|
|
}
|
|
|
|
/**
|
|
* The commits this push actually adds — reachable from localSha and from NO
|
|
* remote-tracking ref.
|
|
*
|
|
* ⚠ WHY THIS EXISTS RATHER THAN A TWO-DOT RANGE.
|
|
*
|
|
* `remoteSha..localSha` is "everything new on this branch", which is NOT the
|
|
* same as "everything new to the remote". Merge origin/main into a feature
|
|
* branch and every commit main gained since the branch's last push becomes an
|
|
* added line — content that is already published, already scanned, and not
|
|
* this push's doing.
|
|
*
|
|
* Two things follow, and both were observed:
|
|
*
|
|
* · FALSE HIGH FINDINGS. A placeholder connection string in a test fixture,
|
|
* already merged to main by someone else, blocked an unrelated push as
|
|
* `db.url_with_password` — telling the operator to rotate a credential
|
|
* over a fixture they had never touched. A
|
|
* guard that cries wolf on catch-up merges is a guard people learn to
|
|
* bypass reflexively — which is exactly how a real secret gets through.
|
|
* · OVERSIZED SCANS. The comment on SCAN_CHUNK_BYTES below records a
|
|
* 1,146,782-byte diff from "a feature branch catching up to a busy main"
|
|
* blowing the engine's 1 MiB cap. Same root cause, treated there as a size
|
|
* problem and solved by slicing. Narrowing the range fixes the size too.
|
|
*
|
|
* A two-dot range cannot express this: after merging main, neither the remote
|
|
* tip nor the merge-base with main is an ancestor of the other, so no single
|
|
* base excludes both. `rev-list --not --remotes=<push-remote>/*` is the
|
|
* operation that does, and this file already reasons that way in
|
|
* `unknownRemoteTipBase` step 2. The exclusion is scoped to the push target's
|
|
* tracking namespace (see remotesExclusionArgs): the upstream commits a catch-up
|
|
* merge brings in came from the SAME remote being pushed to, so scoping keeps
|
|
* the #2592 fix intact while a secret known only to some OTHER (private)
|
|
* remote is still scanned on its way to this one (S1).
|
|
*
|
|
* Each commit is diffed alone. `--cc` on a merge shows only the conflict
|
|
* RESOLUTION — content that exists in no parent — so a secret introduced while
|
|
* resolving a merge is still caught, while an ordinary merge contributes
|
|
* nothing. Returns null when the notion does not apply, so callers fall back.
|
|
*/
|
|
function addedLinesFromNewCommits(localSha: string, remoteSha: string): string | null {
|
|
// remoteSha is what git TELLS us the remote has, and it is authoritative in a
|
|
// way `--remotes` is not: remote-tracking refs can be absent (a fresh clone
|
|
// that never fetched, a push to a remote with no tracking ref) or stale. Drop
|
|
// it and a repo with no tracking refs excludes NOTHING — every commit ever
|
|
// made reads as "new", which re-introduces the false positives from the other
|
|
// direction. So it stays the base; `--remotes` only ADDS exclusions on top.
|
|
if (ZERO.test(remoteSha) || !objectExists(remoteSha)) return null;
|
|
|
|
const narrowed = git(["rev-list", localSha, "--not", remoteSha, ...remotesExclusionArgs()]).trim();
|
|
if (!narrowed) return null;
|
|
|
|
// If excluding remote-tracking refs changes nothing, this push has no
|
|
// catch-up commits and the plain range already describes it exactly. Defer to
|
|
// it. That is not just an optimization: it keeps every push that ISN'T a
|
|
// catch-up merge on the original gitStrict diff path, so the fail-closed
|
|
// guarantee (#1946) and its regression test keep exercising the code they
|
|
// were written for. A narrowing that silently retired that test would be a
|
|
// worse trade than the false positives it set out to fix.
|
|
const plain = git(["rev-list", `${remoteSha}..${localSha}`]).trim();
|
|
const asSet = (s: string) => s.split("\n").filter(Boolean).sort().join("\n");
|
|
if (asSet(narrowed) === asSet(plain)) return null;
|
|
|
|
const shas = narrowed.split("\n").filter(Boolean);
|
|
// A rewrite of long history should fall back rather than shell out per commit.
|
|
if (shas.length > 500) return null;
|
|
const out: string[] = [];
|
|
for (const sha of shas) {
|
|
// gitStrict: a failed diff must never read as "nothing added" (#1946).
|
|
out.push(gitStrict([
|
|
"show", "--unified=0", "--no-color", "--no-ext-diff", "--no-textconv",
|
|
"--src-prefix=a/", "--dst-prefix=b/", "--cc", "--format=", sha,
|
|
]));
|
|
}
|
|
return out.join("\n");
|
|
}
|
|
|
|
/** Return the added-line text for a ref update being pushed. */
|
|
function addedLinesFor(localSha: string, remoteSha: string): string {
|
|
// Preferred ONLY when this push carries catch-up commits: scanning them again
|
|
// is the bug. Every other shape falls through to the range logic below.
|
|
const fromNew = addedLinesFromNewCommits(localSha, remoteSha);
|
|
if (fromNew !== null) return collectAddedLines(fromNew);
|
|
|
|
let range: string;
|
|
if (ZERO.test(remoteSha) || !objectExists(remoteSha)) {
|
|
const base = ZERO.test(remoteSha) ? unknownRemoteTipBase(localSha) : null;
|
|
range = base ? `${base}..${localSha}` : `${emptyTreeOid()}..${localSha}`;
|
|
} else {
|
|
// Existing branch (incl. force-push): net new content remote..local.
|
|
range = `${remoteSha}..${localSha}`;
|
|
}
|
|
// -U0: only changed lines; we keep lines starting with '+' (added), drop the
|
|
// +++ file header. Unified diff added lines start with a single '+'.
|
|
// Strict (#1946): a failed diff used to return "" and the push sailed
|
|
// through unscanned — fail open on the exact path the guard exists for.
|
|
//
|
|
// --no-ext-diff: a user's `diff.external` driver replaces the entire diff
|
|
// with its own output — with one set, `git diff` emits zero '+' lines, so an
|
|
// unhardened scanner reads an empty diff and exits 0 on a push full of
|
|
// secrets. Reachable from ordinary user config, not hypothetical. (#2498)
|
|
// --no-textconv: a .gitattributes textconv driver can likewise rewrite
|
|
// content before we ever see it. (#2498)
|
|
const diff = gitStrict([
|
|
"diff", "--unified=0", "--no-color", "--no-ext-diff", "--no-textconv",
|
|
"--src-prefix=a/", "--dst-prefix=b/", range,
|
|
]);
|
|
return collectAddedLines(diff);
|
|
}
|
|
|
|
/**
|
|
* Added-line text from a unified diff. Shared by both range strategies so the
|
|
* hunk-aware header handling below cannot drift between them.
|
|
*/
|
|
function collectAddedLines(diff: string): string {
|
|
const added: string[] = [];
|
|
// Hunk-aware header skip (#2498): `+++ ` is only a FILE HEADER outside a
|
|
// hunk. Inside a hunk, an added content line whose text begins with "++"
|
|
// renders as "+++<content>" — the old blanket startsWith("+++") skip
|
|
// silently dropped exactly those lines from the scan.
|
|
let inHunk = false;
|
|
let file = "";
|
|
let fileLines: string[] = [];
|
|
const flush = () => {
|
|
added.push(...releaseVersionExempt(file, fileLines));
|
|
fileLines = [];
|
|
};
|
|
for (const line of diff.split("\n")) {
|
|
// `diff --` rather than `diff --git`: a merge scanned with --cc emits
|
|
// `diff --cc <path>`, so a --git-only reset left inHunk true across file
|
|
// boundaries and read the next file's `+++ b/...` header as content. Only
|
|
// noise (it over-scans, never under-scans), but the boundary is real.
|
|
if (line.startsWith("diff --")) { flush(); file = ""; inHunk = false; continue; }
|
|
if (line.startsWith("@@")) { inHunk = true; continue; }
|
|
if (!inHunk && line.startsWith("+++ ")) { file = line.startsWith("+++ b/") ? line.slice(6) : ""; continue; }
|
|
if (!inHunk && (line.startsWith("+++") || line.startsWith("---"))) continue;
|
|
if (line.startsWith("+")) fileLines.push(line.slice(1));
|
|
}
|
|
flush();
|
|
return added.join("\n");
|
|
}
|
|
|
|
const VERSION_FILE_CONTENT = /^\d+(?:\.\d+)+$/;
|
|
const CHANGELOG_RELEASE_VERSION = /^(#{1,6}[\t ]*\[?v?)\d+(?:\.\d+){3}(?=[\]\t ]|$)/;
|
|
|
|
/**
|
|
* #2856: a 4-part release version (MAJOR.MINOR.PATCH.MICRO) is shaped exactly like a
|
|
* public IPv4 address, so /ship's own VERSION bump and CHANGELOG heading
|
|
* raised pii.ip_public on every release. Exempt only those two release
|
|
* artifacts, decided from the pushed diff alone (works in shallow clones):
|
|
* a root VERSION whose entire added content is one version string, and the
|
|
* version token of a root CHANGELOG.md release heading (the rest of the
|
|
* heading is still scanned). Every other file — and any other position in
|
|
* these two — is scanned unchanged; the engine's global allowlist is untouched.
|
|
*/
|
|
function releaseVersionExempt(file: string, lines: string[]): string[] {
|
|
if (file === "VERSION" && lines.length === 1 && VERSION_FILE_CONTENT.test(lines[0].trim())) return [];
|
|
if (file === "CHANGELOG.md") return lines.map((l) => l.replace(CHANGELOG_RELEASE_VERSION, "$1VERSION"));
|
|
return lines;
|
|
}
|
|
|
|
/**
|
|
* Byte budget per scan() call. Kept comfortably under redact-engine's
|
|
* DEFAULT_MAX_BYTES (1 MiB) so a slice never trips its oversize guard.
|
|
*/
|
|
const SCAN_CHUNK_BYTES = 768 * 1024;
|
|
const SCAN_OVERLAP_CHARS = 16 * 1024;
|
|
const MAX_SCAN_BYTES = 1024 * 1024;
|
|
|
|
/**
|
|
* Scan complete-line core slices with bounded left and right context. Admit
|
|
* only findings whose original captured start lies inside that core.
|
|
*
|
|
* Why: the engine refuses input over its byte cap and fails closed, which is
|
|
* right for one scan() call but wrong as a push policy — a feature branch
|
|
* catching up to a busy main legitimately produces more added lines than the
|
|
* cap (1,146,782 bytes against the 1 MiB default in the push that prompted
|
|
* this, and only ~7% of that was the lockfile). The push then blocked on
|
|
* `engine.input_too_large` — a size error naming no credential — which trains
|
|
* people to reach for --no-verify, defeating the guardrail far more thoroughly
|
|
* than a large diff does.
|
|
*
|
|
* A match may need context on either adjacent line (`nearWindow` reaches 300
|
|
* normalized characters). The full core lines keep opaque spans intact; the
|
|
* engine's normalization map selects context in normalized units without
|
|
* changing the raw input. A partial context may create an artificial boundary,
|
|
* but its findings are never admitted as core findings.
|
|
*
|
|
* Fail-closed is preserved: a SINGLE line over the budget is still passed to
|
|
* the engine intact, so a genuinely unscannable blob (minified bundle,
|
|
* embedded base64) trips input_too_large and blocks exactly as before.
|
|
*
|
|
* Findings' line/col are slice-relative, which is fine here — this hook only
|
|
* reads severity, id and preview. Do not lift this into the engine, where
|
|
* callers rely on absolute line numbers.
|
|
*/
|
|
function scanAddedLines(added: string, opts: Parameters<typeof scan>[1]): Finding[] {
|
|
const findings: Finding[] = [];
|
|
const cores: Array<{ start: number; end: number }> = [];
|
|
const lines = added.split("\n");
|
|
let start = 0;
|
|
let end = 0;
|
|
let bytes = 0;
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
const newline = i < lines.length - 1 ? 1 : 0;
|
|
const lineBytes = Buffer.byteLength(line, "utf8") + newline;
|
|
if (bytes && bytes + lineBytes > SCAN_CHUNK_BYTES) {
|
|
cores.push({ start, end });
|
|
start = end;
|
|
bytes = 0;
|
|
}
|
|
end += line.length + newline;
|
|
bytes += lineBytes;
|
|
if (lineBytes > SCAN_CHUNK_BYTES) {
|
|
cores.push({ start, end });
|
|
start = end;
|
|
bytes = 0;
|
|
}
|
|
}
|
|
if (bytes) cores.push({ start, end });
|
|
|
|
const contextStart = (boundary: number): number => {
|
|
if (!boundary) return 0;
|
|
let length = Math.min(boundary, SCAN_OVERLAP_CHARS * 2);
|
|
while (true) {
|
|
const from = boundary - length;
|
|
const raw = added.slice(from, boundary);
|
|
if (Buffer.byteLength(raw, "utf8") > MAX_SCAN_BYTES) return from;
|
|
const { normalized, map } = normalizeWithMap(raw);
|
|
if (!from || normalized.length > SCAN_OVERLAP_CHARS) {
|
|
return from + (map[Math.max(0, normalized.length - SCAN_OVERLAP_CHARS - 1)] ?? 0);
|
|
}
|
|
length = Math.min(boundary, length * 2);
|
|
}
|
|
};
|
|
|
|
const contextEnd = (boundary: number): number => {
|
|
if (boundary === added.length) return boundary;
|
|
let length = Math.min(added.length - boundary, SCAN_OVERLAP_CHARS * 2);
|
|
while (true) {
|
|
const to = boundary + length;
|
|
const raw = added.slice(boundary, to);
|
|
if (Buffer.byteLength(raw, "utf8") > MAX_SCAN_BYTES) return to;
|
|
const { normalized, map } = normalizeWithMap(raw);
|
|
if (to === added.length || normalized.length > SCAN_OVERLAP_CHARS) {
|
|
return boundary + (map[Math.min(normalized.length, SCAN_OVERLAP_CHARS + 1)] ?? length);
|
|
}
|
|
length = Math.min(added.length - boundary, length * 2);
|
|
}
|
|
};
|
|
|
|
for (const core of cores) {
|
|
const left = contextStart(core.start);
|
|
const text = added.slice(left, contextEnd(core.end));
|
|
const result = scan(text, opts);
|
|
if (result.oversize) {
|
|
findings.push(...result.findings);
|
|
return findings;
|
|
}
|
|
const starts = [0];
|
|
for (let i = 0; i < text.length; i++) if (text[i] === "\n") starts.push(i + 1);
|
|
for (const finding of result.findings) {
|
|
const offset = starts[finding.line - 1] + finding.col - 1 + left;
|
|
if (offset >= core.start && offset < core.end) findings.push(finding);
|
|
}
|
|
}
|
|
|
|
return findings;
|
|
}
|
|
|
|
function logSkip(reason: string): void {
|
|
try {
|
|
const home = resolveStateRoot();
|
|
const dir = path.join(home, "security");
|
|
// mkdirpSync, not bare mkdirSync: bun-on-Windows EEXIST (#2635). This site
|
|
// is try-wrapped by the caller, so the old failure was a silent skip-log
|
|
// loss rather than a crash — the fix makes the log survive, not un-crash.
|
|
mkdirpSync(dir);
|
|
fs.appendFileSync(
|
|
path.join(dir, "prepush-skip.jsonl"),
|
|
JSON.stringify({ ts: new Date().toISOString(), reason }) + "\n",
|
|
);
|
|
} catch {
|
|
// best-effort; never block a push because logging failed
|
|
}
|
|
}
|
|
|
|
function main() {
|
|
if ((process.env.GSTACK_REDACT_PREPUSH || "").toLowerCase() === "skip") {
|
|
logSkip(process.env.GSTACK_REDACT_PREPUSH_REASON || "env-skip");
|
|
process.stderr.write("gstack-redact-prepush: skipped via GSTACK_REDACT_PREPUSH=skip\n");
|
|
process.exit(0);
|
|
}
|
|
|
|
const stdin = fs.readFileSync(0, "utf8");
|
|
const refs = stdin
|
|
.split("\n")
|
|
.map((l) => l.trim())
|
|
.filter(Boolean)
|
|
.map((l) => l.split(/\s+/));
|
|
|
|
const allHigh: Finding[] = [];
|
|
let mediumCount = 0;
|
|
|
|
for (const fields of refs) {
|
|
// Fail CLOSED on a ref line we cannot parse (#2498): git hands pre-push
|
|
// exactly "<local ref> <local sha> <remote ref> <remote sha>" — anything
|
|
// else means we cannot tell WHAT is being pushed, and silently skipping
|
|
// it would leave that ref unscanned.
|
|
const [, localSha, , remoteSha] = fields;
|
|
const shaShaped = (s: string | undefined) => !!s && /^[0-9a-f]{40,64}$/i.test(s);
|
|
if (fields.length !== 4 || !shaShaped(localSha) || !shaShaped(remoteSha)) {
|
|
process.stderr.write(
|
|
"\n⛔ gstack-redact-prepush BLOCKED the push — could not parse a pre-push ref line, " +
|
|
"so its content cannot be scanned.\n" +
|
|
` line: ${JSON.stringify(fields.join(" "))}\n` +
|
|
"Bypass if you're sure: GSTACK_REDACT_PREPUSH=skip git push (or git push --no-verify)\n",
|
|
);
|
|
process.exit(1);
|
|
}
|
|
if (ZERO.test(localSha!)) continue; // branch delete → nothing pushed
|
|
let added: string;
|
|
try {
|
|
added = addedLinesFor(localSha, remoteSha || "0");
|
|
} catch (err) {
|
|
// Fail CLOSED (#1946): if we can't compute the pushed diff we can't
|
|
// scan it, and unscanned-but-allowed is the failure mode this hook
|
|
// exists to prevent.
|
|
process.stderr.write(
|
|
"\n⛔ gstack-redact-prepush BLOCKED the push — could not compute the pushed diff, " +
|
|
"so it cannot be scanned for credentials.\n" +
|
|
` (${err instanceof Error ? err.message.split("\n")[0] : String(err)})\n` +
|
|
"Bypass if you're sure: GSTACK_REDACT_PREPUSH=skip git push (or git push --no-verify)\n",
|
|
);
|
|
process.exit(1);
|
|
}
|
|
if (!added.trim()) continue;
|
|
// Visibility doesn't change HIGH behavior; pass private so nothing is treated
|
|
// as public-strict (HIGH blocks regardless either way).
|
|
// Sliced (see scanAddedLines) so a large-but-legitimate diff is actually
|
|
// scanned rather than blocked unscanned on the engine's size cap.
|
|
for (const f of scanAddedLines(added, { repoVisibility: "private" })) {
|
|
if (f.severity === "HIGH") allHigh.push(f);
|
|
else if (f.severity === "MEDIUM") mediumCount++;
|
|
}
|
|
}
|
|
|
|
if (mediumCount > 0) {
|
|
process.stderr.write(
|
|
`gstack-redact-prepush: ${mediumCount} MEDIUM finding(s) in pushed diff (PII/internal). ` +
|
|
"Not blocking. Review before this becomes public.\n",
|
|
);
|
|
}
|
|
|
|
if (allHigh.length > 0) {
|
|
// A scan that could not RUN is not a scan that FOUND something. Reporting
|
|
// "credential(s) in the pushed diff — rotate the credential" for an
|
|
// `engine.*` finding tells the operator to rotate a secret that was never
|
|
// detected, on a diff that was never read. Blocking is still right (fail
|
|
// closed), but the reason must be the true one: a guardrail that cries wolf
|
|
// is a guardrail that gets bypassed by reflex, which is worse than none.
|
|
// The computed range can be much larger than the visible tip diff, and a
|
|
// single over-cap line is deliberately withheld rather than partly scanned.
|
|
const unscanned = allHigh.filter((f) => f.id.startsWith("engine."));
|
|
const secrets = allHigh.filter((f) => !f.id.startsWith("engine."));
|
|
|
|
if (secrets.length > 0) {
|
|
process.stderr.write(
|
|
"\n⛔ gstack-redact-prepush BLOCKED the push — credential(s) in the pushed diff:\n\n",
|
|
);
|
|
for (const f of secrets) {
|
|
process.stderr.write(` HIGH ${f.id} ${f.preview}\n`);
|
|
}
|
|
process.stderr.write(
|
|
"\nRotate the credential (a pushed secret is compromised) and remove it from the diff.\n",
|
|
);
|
|
}
|
|
|
|
if (unscanned.length > 0) {
|
|
process.stderr.write(
|
|
"\n⛔ gstack-redact-prepush BLOCKED the push — the diff could NOT be scanned.\n" +
|
|
" No credential was found; none was looked for. Blocking fail-closed.\n\n",
|
|
);
|
|
for (const f of unscanned) {
|
|
process.stderr.write(` ${f.id}: ${f.description}\n`);
|
|
}
|
|
process.stderr.write(
|
|
"\nA long line, large first push, or unavailable remote tip may exceed the\n" +
|
|
"per-slice safety cap. Check the destination and range; scan the diff yourself\n" +
|
|
"before bypassing: `git diff <base>..HEAD | grep -inE \'password|secret|token|api.?key\'`.\n",
|
|
);
|
|
}
|
|
|
|
process.stderr.write(
|
|
"This is a guardrail: `git push --no-verify` or `GSTACK_REDACT_PREPUSH=skip git push` bypass it.\n",
|
|
);
|
|
process.exit(1);
|
|
}
|
|
|
|
process.exit(0);
|
|
}
|
|
|
|
main();
|