mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-02 06:54:50 +01:00
v1.92.61.0 fix(ship): risk classes cover outbound clients, archives, child_process and metering (GST-850) (#3829)
* WIP GST-850: allowlist docs paths, security and money classes (pins not yet repinned; review pending) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * GST-850: repin review-depth, review-risk and review-allowlist under the GST-850 no-loss replay Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * GST-850: round-1 review fixes (locale admission, SVG comments and interpolation, escapes, YAML types, linear import scan) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * GST-850: choice A, drop the allowlist docs paths (UI text and SVGs stay code); repin under GST-850 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * GST-850 B+: coarse linear security rules (child_process module line, any outbound client call), drop stale docs paragraph; repin Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * GST-850 B: narrow the linear-time claim to the new rules; hostile anchor-run inputs for every new pattern Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
Garry Tanandcapy-ai[bot] authored and GitHub committed
1 parent
68195c372e
commit
dd9834d44d
14 files changed
+471
-30
No files matched your search
@@ -1,5 +1,13 @@
|
||||
# Changelog
|
||||
|
||||
## [1.92.61.0] - 2026-11-02
|
||||
|
||||
- **The security class reaches outbound URLs, archives and every `child_process` spelling** (GST-850). `lib/review-risk.ts` now flags any call to an outbound HTTP client (`fetch`, `http(s).get` or `.request`, axios, got, undici, `requests`, `httpx`, urllib), `new URL` from input, redirect toggles, proxy settings and host allowlists; tar, zip and unzip extraction, `path.join` or `resolve` on request or entry input, and `'..'` checks; and any changed line that names the `child_process` module. The new rules parse no import clause or call arguments, and a test holds each under 100 ms on 200,000-character hostile lines.
|
||||
- **The money class reads metering and charge code.** Paths and lines naming meter, metering, metered, charge, billable, billed or credits, plus token usage, a billing proxy and cost or charge clients, now route to the money reader. gbrain-evals #289, which changed a metering proxy and billing clients and read `required=none`, is now a regression test that reads money. "costume", a CLI's usage text and a plain proxy do not count.
|
||||
- **When a diff publishes numbers, the batched specialist reads the write-up's text files** instead of leaving docs out of its diff. The collector records the files a diff creates.
|
||||
- **UI text and SVGs stay code (light).** An allowlist docs path for chart SVGs and locale values was built and dropped for review cost: its first review round found a new bypass class in each version. Every bypass found so far is a test row that classes as code.
|
||||
- Replayed on the 26 GSTA-167 no-loss units, no unit loses a reviewer role. The replay checked which reviewers get dispatched, not R51's re-scored findings. Over the last 100 main commits, 3 move from light to full and no commit with a gate or risk class drops depth.
|
||||
|
||||
## [1.92.60.0] - 2026-11-02
|
||||
|
||||
Free test shards can now run their files as several sub-shards side by side inside one CI job. `bun run scripts/test-free-shards.ts --ci-run <plan> --shard <i> --result <file> --jobs <k>` splits the shard into k parts that each run in their own sandboxed `bun test` process. The default stays at 1, so CI runs exactly as before until a workflow opts in. Bad `--jobs` values stop with `FREE_JOBS_INVALID`, a sub-shard that runs out of time prints `FREE_SUBSHARD_TIMEOUT` with a serial reproduce command, and every CI shard result now records per-file test and `expect()` counts. See docs/TESTING_INTERNALS.md#sub-shards-inside-one-ci-job.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# gstack digest v1.92.60.0 — regenerate/re-copy after upgrading gstack
|
||||
# gstack digest v1.92.61.0 — regenerate/re-copy after upgrading gstack
|
||||
|
||||
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
|
||||
for agent hosts without a full skill install. The full skills add workflows,
|
||||
|
||||
+25
-5
@@ -201,17 +201,37 @@ in a data file still schedules the outside voice.
|
||||
added and deleted lines; any class makes the depth full (`risk:<class>` in
|
||||
`reasons=`, `review_depth.risk` in the receipt). money, auth and secret are the
|
||||
[targeted pass](#targeted-pass-on-money-gate-auth-and-secret-code)'s matchers.
|
||||
**security** is a trust-boundary path (trust, redact, sanitize, neutralize,
|
||||
consent, escape) or line (`shell: true`, `execSync(`, `eval`, `new Function(`,
|
||||
raw HTML sinks, a shell run through argv, recursive deletes). **data** is user data: a path naming pii, gdpr, ccpa, privacy,
|
||||
**money** also reads metering and charge words in a path or on a line (meter,
|
||||
metering, metered, charge, charged, billable, billed, credits) and three
|
||||
pairs: usage beside a token or cost word, proxy beside billing, and a client
|
||||
beside a cost or charge word, so a CLI's usage text and a plain proxy stay out
|
||||
(GST-850, from gbrain-evals #289, whose metering proxy and billing clients read
|
||||
`required=none`). **security** is a trust-boundary path (trust, redact, sanitize, neutralize,
|
||||
consent, escape) or line (`shell: true`, `exec(` and `execSync(`, `eval`, `new Function(`,
|
||||
raw HTML sinks, a shell run through argv, recursive deletes), any call to an
|
||||
outbound HTTP client (`fetch`, `http(s).get` or `.request`, axios, got, undici,
|
||||
`requests`, `httpx`, urllib, with no exemption for a literal URL), `new URL` from
|
||||
input, redirect toggles, proxy settings and host allowlists, archive extraction
|
||||
and path traversal (`tar.x`, adm-zip, yauzl, extract-zip, `extractall`, `tar -x`,
|
||||
`unzip`, `path.join` or `resolve` on request or entry input, a `'..'` check added
|
||||
or removed), and any changed line that names the `child_process` or
|
||||
`node:child_process` module, in any import or require spelling. The GST-850
|
||||
rules are coarse on purpose and parse no import clause or call arguments;
|
||||
`test/review-risk-linear.test.ts` holds each of them under 100 ms on
|
||||
200,000-character hostile lines, its anchor followed by a long run of the
|
||||
characters it repeats included. That claim covers only the rules GST-850 adds,
|
||||
not R51's older ones. **data** is user data: a path naming pii, gdpr, ccpa, privacy,
|
||||
personal, account, customer, cookie, persistence, storage or database, or a line
|
||||
that drops a table or column, deletes rows or bulk-deletes records. Auth scope,
|
||||
migrations and deleted code paths were already gate class. `RISK_RULES` hashes
|
||||
the patterns and `DEPTH_RULES` includes it.
|
||||
|
||||
**UI text and SVGs** stay code (light unless a risk class applies). A docs path
|
||||
for UI copy, locale values and chart SVGs waits for an allowlist classifier: three
|
||||
rounds of R51 review found bypasses in line-pattern versions of both.
|
||||
for UI copy, locale values and chart SVGs was built as an allowlist in GST-850 and
|
||||
dropped for review cost: its first review round found a new bypass class in each
|
||||
version, the same pattern as three rounds of R51. When a diff publishes numbers,
|
||||
the batched specialist's diff keeps the write-up's text files instead of leaving
|
||||
docs out.
|
||||
|
||||
**Batched checklists (the lean ensemble, GSTA-167 part 2).** One specialist
|
||||
subagent reads Testing and Security on every code or gate diff, plus
|
||||
|
||||
+10
-7
@@ -20,6 +20,8 @@ export interface Collected extends ClassInput {
|
||||
errors: DepthError[];
|
||||
deleted: string[];
|
||||
stampOnly: string[];
|
||||
/** Paths the diff creates (untracked files included): their added lines are the whole file. */
|
||||
created: string[];
|
||||
/** Added lines per path, for scanNumbers. */
|
||||
added: Record<string, string[]>;
|
||||
/** Deleted lines per path, for the targeted classes (lib/review-targeted.ts). */
|
||||
@@ -54,11 +56,11 @@ export function loadRegistryAt(root: string, sha: string): RegistryInfo {
|
||||
}
|
||||
|
||||
/** Numstat, deletions and the -U0 patch for `range` (`[mergeBase, head]` or `[mergeBase]` for the working tree). */
|
||||
function readDiff(root: string, range: string[], errors: DepthError[]): Pick<Collected, 'perPath' | 'deleted' | 'stampOnly' | 'added' | 'removed'> {
|
||||
function readDiff(root: string, range: string[], errors: DepthError[]): Pick<Collected, 'perPath' | 'deleted' | 'stampOnly' | 'created' | 'added' | 'removed'> {
|
||||
const numstat = git(root, ['diff', '--numstat', '-z', '--no-renames', ...range]);
|
||||
const status = git(root, ['diff', '--name-status', '-z', '--no-renames', '--diff-filter=D', ...range]);
|
||||
const patch = git(root, ['diff', '-U0', '--no-color', '--no-ext-diff', '--no-renames', ...range]);
|
||||
const empty = { perPath: [], deleted: [], stampOnly: [], added: {}, removed: {} };
|
||||
const empty = { perPath: [], deleted: [], stampOnly: [], created: [], added: {}, removed: {} };
|
||||
for (const [r, what] of [[numstat, '--numstat'], [status, '--name-status'], [patch, '-U0']] as const) {
|
||||
if (!r.ok) { errors.push({ code: 'numstat_failed', detail: oneLine(r.err || 'git diff failed'), input: `git diff ${what} ${range.join(' ')}`, repair: 'check that both commits exist locally (git fetch origin <base>) and rerun' }); return empty; }
|
||||
}
|
||||
@@ -73,6 +75,7 @@ function readDiff(root: string, range: string[], errors: DepthError[]): Pick<Col
|
||||
const added: Record<string, string[]> = {};
|
||||
const removed: Record<string, string[]> = {};
|
||||
const changed: Record<string, string[]> = {};
|
||||
const created: string[] = [];
|
||||
let cur = '';
|
||||
let minus = '';
|
||||
let inHunk = false;
|
||||
@@ -80,7 +83,7 @@ function readDiff(root: string, range: string[], errors: DepthError[]): Pick<Col
|
||||
if (line.startsWith('diff --git ')) { inHunk = false; cur = ''; minus = ''; continue; }
|
||||
if (!inHunk) {
|
||||
if (line.startsWith('--- ')) minus = line === '--- /dev/null' ? '' : line.slice(6).replace(/\t$/, '');
|
||||
else if (line.startsWith('+++ ')) cur = line === '+++ /dev/null' ? minus : line.slice(6).replace(/\t$/, '');
|
||||
else if (line.startsWith('+++ ')) { cur = line === '+++ /dev/null' ? minus : line.slice(6).replace(/\t$/, ''); if (!minus && cur && !scratch(cur)) created.push(cur); }
|
||||
else if (line.startsWith('@@')) inHunk = true;
|
||||
continue;
|
||||
}
|
||||
@@ -90,7 +93,7 @@ function readDiff(root: string, range: string[], errors: DepthError[]): Pick<Col
|
||||
if (line[0] === '+' || line[0] === '-') (changed[cur] ??= []).push(line);
|
||||
}
|
||||
const stampOnly = perPath.map(e => e.path).filter(p => p.endsWith('package.json') && !!changed[p]?.length && changed[p]!.every(l => /^[+-]\s*"version"\s*:\s*"[^"]*"\s*,?\s*$/.test(l)));
|
||||
return { perPath, deleted, stampOnly, added, removed };
|
||||
return { perPath, deleted, stampOnly, created, added, removed };
|
||||
}
|
||||
|
||||
/** More data candidates than this keep every data file code: the consumer scan is not run. */
|
||||
@@ -158,7 +161,7 @@ export function collectCommit(root: string, baseSha: string, headSha: string): C
|
||||
const mb = git(root, ['merge-base', baseSha, headSha]);
|
||||
if (!mb.ok) {
|
||||
errors.push({ code: 'numstat_failed', detail: oneLine(mb.err || 'no merge base'), input: `git merge-base ${baseSha} ${headSha}`, repair: 'check that both commits exist locally (git fetch origin <base>) and rerun' });
|
||||
return result({ perPath: [], deleted: [], stampOnly: [], added: {}, removed: {} }, { root: [], registry, errors, mergeBase: null, untracked: [], consumed: 'all' });
|
||||
return result({ perPath: [], deleted: [], stampOnly: [], created: [], added: {}, removed: {} }, { root: [], registry, errors, mergeBase: null, untracked: [], consumed: 'all' });
|
||||
}
|
||||
const mergeBase = mb.out.trim();
|
||||
const diff = readDiff(root, [mergeBase, headSha], errors);
|
||||
@@ -176,7 +179,7 @@ export function collectLive(root: string, base: string): Collected {
|
||||
const mb = git(root, ['merge-base', ref, 'HEAD']);
|
||||
if (!mb.ok) {
|
||||
errors.push({ code: 'numstat_failed', detail: oneLine(mb.err || `no merge base with ${ref}`), input: `git merge-base ${ref} HEAD`, repair: `fetch the base first (git fetch origin ${base})` });
|
||||
return result({ perPath: [], deleted: [], stampOnly: [], added: {}, removed: {} }, { root, registry, errors, mergeBase: null, untracked: [], consumed: 'all' });
|
||||
return result({ perPath: [], deleted: [], stampOnly: [], created: [], added: {}, removed: {} }, { root, registry, errors, mergeBase: null, untracked: [], consumed: 'all' });
|
||||
}
|
||||
const mergeBase = mb.out.trim();
|
||||
const diff = readDiff(root, [mergeBase], errors);
|
||||
@@ -190,7 +193,7 @@ export function collectLive(root: string, base: string): Collected {
|
||||
const text = binary ? '' : buf.toString('utf8');
|
||||
const lines = text ? text.split('\n').length - (text.endsWith('\n') ? 1 : 0) : 0;
|
||||
diff.perPath.push({ path: p, lines, binary });
|
||||
if (!binary) diff.added[p] = text.split('\n');
|
||||
if (!binary) { diff.added[p] = text.split('\n'); diff.created.push(p); }
|
||||
}
|
||||
const consumed = scanConsumers(root, [...diff.perPath.map(e => e.path), ...diff.deleted], [mergeBase], true, root, errors);
|
||||
return result(diff, { root, registry, errors, mergeBase, untracked, consumed });
|
||||
|
||||
+3
-1
@@ -131,6 +131,8 @@ export interface ClassInput {
|
||||
* (lib/review-collect.ts). A candidate not listed counts as docs; 'all' or no scan keeps every data file code.
|
||||
*/
|
||||
consumed?: readonly string[] | 'all';
|
||||
/** Paths the diff creates (untracked files included): their added lines are the whole file. */
|
||||
created?: readonly string[];
|
||||
/** Added and deleted lines per path (lib/review-collect.ts): the risk classes read them; without them no line signal applies. */
|
||||
added?: Record<string, readonly string[]>;
|
||||
removed?: Record<string, readonly string[]>;
|
||||
@@ -395,7 +397,7 @@ export function planReview(i: PlanInput): ReviewPlan {
|
||||
const skipped: string[] = [];
|
||||
const dispatches: Dispatch[] = [];
|
||||
const add = (role: Role, name: string, effort: Effort, exclude: string[] = [], extra: Partial<Dispatch> = {}) => dispatches.push({ role, name, effort, exclude, ...extra });
|
||||
const docsAndOutputs = i.paths.filter(p => pathClass(i, p) === 'docs' || isOutput(i, p)).map(excludeSpec);
|
||||
const docsAndOutputs = i.paths.filter(p => (pathClass(i, p) === 'docs' && !(i.numbers.publishes && DOC_PATH.test(p))) || isOutput(i, p)).map(excludeSpec);
|
||||
const numbersChanged = i.numbers.publishes && i.reviewedNumbers !== undefined && i.reviewedNumbers !== i.numbers.fingerprint;
|
||||
const firstVoices = h.fullPasses === 0 || !i.adversarialRan;
|
||||
const specialist: Effort = i.tiers === false ? 'host' : ROLE_EFFORT.specialist;
|
||||
|
||||
+31
-10
@@ -4,16 +4,23 @@
|
||||
* and deleted lines, so it imports neither the depth nor the targeted module.
|
||||
*
|
||||
* money, auth and secret are the targeted pass's matchers (GST-257). security is a trust-boundary path or line
|
||||
* (a redaction, sanitizer, consent or trust module; a shell, eval or raw HTML sink); data is user data (a path
|
||||
* naming personal or stored data, or a line that drops or bulk-deletes it). Path and identifier signals are
|
||||
* token-bounded: costume never names cost.
|
||||
* (a redaction, sanitizer, consent or trust module; a shell, eval or raw HTML sink; outbound URL handling; archive
|
||||
* extraction and path traversal; any line naming the child_process module); data is user data (a path naming personal or
|
||||
* stored data, or a line that drops or bulk-deletes it). Path and identifier signals are token-bounded: costume never
|
||||
* names cost, and a CLI's usage text never names money.
|
||||
*/
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
export const RISK_CLASSES = ['money', 'auth', 'secret', 'security', 'data'] as const;
|
||||
export type RiskClass = (typeof RISK_CLASSES)[number];
|
||||
|
||||
export const MONEY_PATH_TOKENS = ['cost', 'costs', 'spend', 'spending', 'budget', 'budgets', 'ledger', 'billing', 'price', 'prices', 'pricing', 'fee', 'fees', 'invoice', 'invoices'];
|
||||
/** Metering and charge words (GST-850: gbrain-evals #289 changed a metering proxy and billing clients with none of the older tokens); they also count on a line. */
|
||||
export const MONEY_LINE_TOKENS = ['meter', 'meters', 'metering', 'metered', 'charge', 'charges', 'charged', 'billable', 'billed', 'credits'];
|
||||
export const MONEY_PATH_TOKENS = ['cost', 'costs', 'spend', 'spending', 'budget', 'budgets', 'ledger', 'billing', 'price', 'prices', 'pricing', 'fee', 'fees', 'invoice', 'invoices', ...MONEY_LINE_TOKENS];
|
||||
/** Words that name money only beside another in the same path or line: token usage, a billing proxy, a cost or charge client. */
|
||||
export const MONEY_PAIRS: readonly (readonly [string, readonly string[]])[] = [
|
||||
['usage', ['token', 'tokens', 'cost', 'costs', 'spend', 'billing', 'price', 'pricing', 'budget']], ['proxy', ['billing', 'bill']], ['client', ['cost', 'costs', 'charge', 'charges', 'billing', 'price', 'pricing', 'spend']],
|
||||
];
|
||||
/** Identifiers ending in _usd, Usd or cap_usd, any spend_cap spelling, and paid-provider call sites (the providers lib/pricing.ts prices). */
|
||||
export const MONEY_LINE = [
|
||||
/\b[A-Za-z0-9_]*_usd\b/i, /\b[a-z][A-Za-z0-9]*Usd\b/, /\bspend[_-]?cap\b/i,
|
||||
@@ -33,22 +40,36 @@ export const SECRET_LINE = [
|
||||
];
|
||||
|
||||
const SECURITY_PATH_TOKENS = ['trust', 'redact', 'redaction', 'redactor', 'sanitize', 'sanitizer', 'neutralize', 'neutralizer', 'consent', 'escape'];
|
||||
const SECURITY_LINE = [
|
||||
/\bshell\s*:\s*true\b/, /\bshell\s*=\s*True\b/, /(?<![.\w])execSync\s*\(/, /\bchild_process\.exec(?:Sync)?\s*\(/, /\bos\.system\s*\(/,
|
||||
/** Coarse, linear-time rules (GST-850 B+): any changed line naming the child_process module, any outbound HTTP client call; quantifiers bounded. */
|
||||
export const SECURITY_LINE = [
|
||||
/\bshell\s*:\s*true\b/, /\bshell\s*=\s*True\b/, /(?<![.\w$])exec(?:Sync)?\s*\(/, /\b(?:child_process|childProcess|cp)\.exec(?:Sync)?\s*\(/, /\bos\.system\s*\(/,
|
||||
/['"`](?:node:)?child_process['"`]/,
|
||||
/(?:(?<![.\w$])|\b(?:globalThis|window|self)\.)fetch\s{0,20}\(/, /\bhttps?\.(?:request|get)\s{0,20}\(/, /\baxios\b/, /(?<![.\w$])got(?:\.[a-z]{1,10})?\s{0,20}\(/, /\bundici\b/,
|
||||
/\b(?:requests|httpx)\.[a-z]{1,10}\s{0,20}\(/, /\burllib(?:3|\.request)?\b|\burlopen\s{0,20}\(/, /\bnew\s{1,20}URL\s{0,20}\(\s{0,20}(?!['"][^'"\\]{0,2000}['"]\s{0,20}\)|import\.meta\.url\s{0,20}\))/, /\b(?:urljoin|url\.resolve)\s{0,20}\(/,
|
||||
/\bredirect\s*:\s*['"](?:follow|manual)['"]|\b(?:maxRedirects|followRedirects?|follow_redirects|allow_redirects)\s*[:=]/,
|
||||
/\b(?:HTTPS?_PROXY|NO_PROXY|ALL_PROXY|https?_proxy|no_proxy)\b|\b(?:Https?)?ProxyAgent\b|\bproxy\s*(?::|=(?![=>]))/, /\b(?:allowed|trusted|blocked|denied)[_-]?(?:hosts?|domains?|origins?|urls?)|\b(?:hosts?|domains?|origins?|urls?)[_-]?(?:allow|deny|block)[_-]?list/i,
|
||||
/\btar\.(?:x|extract)\s*\(|\.(?:extractall|extractAllTo|extractEntryTo)\s*\(|\b(?:tar|tf|zf|zip|archive)\.extract\s*\(|\bshutil\.unpack_archive\s*\(|\b(?:adm-zip|AdmZip|yauzl|extract-zip|unzipper|node-stream-zip)\b/,
|
||||
/\btar\s+(?:-[A-Za-z]*x|x[A-Za-z]*\s|--extract)|\bunzip\s+[-\w"'$]/,
|
||||
/\bpath\.(?:join|resolve)\s*\([^)]{0,300}?(?:\b(?:req|request|params|query|argv|headers?|filename|fileName|entryName|member|upload|untrusted|userInput)\b|\bentry\.(?:path|name|fileName)\b)/,
|
||||
/\b(?:includes|startsWith|endsWith|indexOf)\s*\(\s*['"`]\.\.[/\\]{0,2}['"`]|[!=]==?\s*['"`]\.\.['"`]|['"`]\.\.['"`]\s*[!=]==?|\\\.\\\./,
|
||||
/(?<![.\w])eval\s*\(/, /(?:^|[;&|])\s*eval\s+["'$]/, /\bnew\s+Function\s*\(/,
|
||||
/\b(?:spawn|spawnSync|execFile|execFileSync)\s*\(\s*['"](?:\/bin\/)?(?:ba|z)?sh['"]/, /\bBun\.spawn(?:Sync)?\s*\(\s*\[\s*['"](?:\/bin\/)?(?:ba|z)?sh['"]/,
|
||||
/\brmSync\s*\([^)]*recursive\s*:\s*true/, /\brm\s+-(?:[a-z]*r[a-z]*f|[a-z]*f[a-z]*r)\b/,
|
||||
/\brmSync\s*\([^)]{0,500}recursive\s*:\s*true/, /\brm\s+-(?:[a-z]*r[a-z]*f|[a-z]*f[a-z]*r)\b/,
|
||||
/\b(?:innerHTML|outerHTML)\s*\+?=/, /\bdangerouslySetInnerHTML\b/, /\bv-html\b/, /\{@html\b/, /\binsertAdjacentHTML\s*\(/, /\bdocument\.write\s*\(/,
|
||||
];
|
||||
const DATA_PATH_TOKENS = ['pii', 'gdpr', 'ccpa', 'privacy', 'personal', 'account', 'accounts', 'customer', 'customers', 'cookie', 'cookies', 'persistence', 'storage', 'database'];
|
||||
const DATA_LINE = [
|
||||
export const DATA_LINE = [
|
||||
/\bDELETE\s+FROM\b/, /\b(?:drop\s+(?:table|column|database|schema)|truncate\s+table)\b/i, /\bdelete\s+from\s+[\w."`]+\s*(?:where\b|;|$|["'`)])/i,
|
||||
/\.(?:deleteMany|destroyAll|bulkDelete)\s*\(/, /\b(?:destroy_all|delete_all)\b/,
|
||||
];
|
||||
|
||||
/** Whether a path or line's tokens name money: a metering or charge word (a cost word too, for a path), or a pair. */
|
||||
const namesMoney = (t: readonly string[], words: readonly string[]) => t.some(x => words.includes(x)) || MONEY_PAIRS.some(([w, beside]) => t.includes(w) && t.some(x => beside.includes(x)));
|
||||
|
||||
/** A hash of the patterns that decide the risk classes, not of this file's bytes; lib/review-depth.ts folds it into DEPTH_RULES. */
|
||||
export const RISK_RULES = createHash('sha256').update(JSON.stringify({
|
||||
RISK_CLASSES, MONEY_PATH_TOKENS, AUTH_PATH_TOKENS, AUTH_PATH_PREFIXES, KEY_NAME, SECURITY_PATH_TOKENS, DATA_PATH_TOKENS,
|
||||
MONEY_LINE_TOKENS, MONEY_PAIRS,
|
||||
patterns: Object.entries({ MONEY_LINE, SECRET_LINE, SECRET_PATH: [SECRET_PATH], SECURITY_LINE, DATA_LINE }).map(([k, v]) => [k, v.map(r => '/' + r.source + '/' + r.flags)]),
|
||||
})).digest('hex').slice(0, 16);
|
||||
|
||||
@@ -71,11 +92,11 @@ export function matchRisk(i: RiskInput): Map<RiskClass, string> {
|
||||
const t = tokens(p);
|
||||
if (i.scopeAuth && t.some(x => AUTH_PATH_TOKENS.includes(x) || AUTH_PATH_PREFIXES.some(a => x.startsWith(a)))) hit('auth', 'auth path: ' + p);
|
||||
if (SECRET_PATH.test(p)) hit('secret', 'secret path: ' + p);
|
||||
if (t.some(x => MONEY_PATH_TOKENS.includes(x))) hit('money', 'money path: ' + p);
|
||||
if (namesMoney(t, MONEY_PATH_TOKENS)) hit('money', 'money path: ' + p);
|
||||
if (t.some(x => SECURITY_PATH_TOKENS.includes(x))) hit('security', 'security path: ' + p);
|
||||
if (t.some(x => DATA_PATH_TOKENS.includes(x))) hit('data', 'data path: ' + p);
|
||||
for (const line of [...(i.added?.[p] ?? []), ...(i.removed?.[p] ?? [])]) {
|
||||
if (!why.has('money') && MONEY_LINE.some(r => r.test(line))) hit('money', 'money line: ' + p);
|
||||
if (!why.has('money') && (MONEY_LINE.some(r => r.test(line)) || namesMoney(tokens(line), MONEY_LINE_TOKENS))) hit('money', 'money line: ' + p);
|
||||
if (!why.has('secret') && SECRET_LINE.some(r => r.test(line))) hit('secret', 'secret line: ' + p);
|
||||
if (!why.has('security') && SECURITY_LINE.some(r => r.test(line))) hit('security', 'security line: ' + p);
|
||||
if (!why.has('data') && DATA_LINE.some(r => r.test(line))) hit('data', 'data line: ' + p);
|
||||
|
||||
@@ -21,14 +21,14 @@ import { createHash } from 'node:crypto';
|
||||
import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import { classifyPath } from './review-depth';
|
||||
import { AUTH_PATH_PREFIXES, AUTH_PATH_TOKENS, KEY_NAME, MONEY_LINE, MONEY_PATH_TOKENS, SECRET_LINE, SECRET_PATH, matchRisk } from './review-risk';
|
||||
import { AUTH_PATH_PREFIXES, AUTH_PATH_TOKENS, KEY_NAME, MONEY_LINE, MONEY_LINE_TOKENS, MONEY_PAIRS, MONEY_PATH_TOKENS, SECRET_LINE, SECRET_PATH, matchRisk } from './review-risk';
|
||||
|
||||
export const TARGETED_CLASSES = ['money', 'gate', 'auth', 'secret'] as const;
|
||||
export type TargetedClass = (typeof TARGETED_CLASSES)[number];
|
||||
|
||||
/** A hash of the patterns that decide the classes, not of this file's bytes: a comment edit keeps it, a pattern edit changes it. */
|
||||
export const TARGETED_RULES = createHash('sha256').update(JSON.stringify({
|
||||
TARGETED_CLASSES, MONEY_PATH_TOKENS, AUTH_PATH_TOKENS, AUTH_PATH_PREFIXES, KEY_NAME,
|
||||
TARGETED_CLASSES, MONEY_PATH_TOKENS, MONEY_LINE_TOKENS, MONEY_PAIRS, AUTH_PATH_TOKENS, AUTH_PATH_PREFIXES, KEY_NAME,
|
||||
patterns: Object.entries({ MONEY_LINE, SECRET_LINE, SECRET_PATH: [SECRET_PATH] }).map(([k, v]) => [k, v.map(r => `/${r.source}/${r.flags}`)]),
|
||||
})).digest('hex').slice(0, 16);
|
||||
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gstack",
|
||||
"version": "1.92.60",
|
||||
"version": "1.92.61",
|
||||
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
||||
"license": "MIT",
|
||||
"type": "module",
|
||||
|
||||
+139
@@ -0,0 +1,139 @@
|
||||
{
|
||||
"source": "garrytan/gbrain-evals#289 (merge 513de26d): changed lines of its code-class paths, read as text via gh pr diff",
|
||||
"paths": [
|
||||
"CHANGELOG.md",
|
||||
"docs/benchmarks/cat40-hard/accepted-drift.json",
|
||||
"docs/benchmarks/cat40-hard/calibration.md",
|
||||
"docs/usage-receipt.md",
|
||||
"eval/runner/cat40/loop.ts",
|
||||
"eval/runner/harness-metering-proxy.ts",
|
||||
"eval/runner/providers/openai-compatible.ts",
|
||||
"eval/runner/q1/cell.ts",
|
||||
"eval/runner/systems/baselines.ts",
|
||||
"eval/workload-suites/bench-infra.ts",
|
||||
"scripts/particle-canary.ts",
|
||||
"test/eval/cat40-particle-loop.test.ts",
|
||||
"test/eval/file-agent.test.ts",
|
||||
"test/eval/harness-metering-proxy.test.ts",
|
||||
"test/eval/memory-qa-particle.test.ts",
|
||||
"test/eval/openai-compatible.test.ts",
|
||||
"test/eval/particle-canary.test.ts",
|
||||
"test/eval/q1-cell.test.ts",
|
||||
"test/eval/workload-particle.test.ts"
|
||||
],
|
||||
"added": {
|
||||
"eval/runner/cat40/loop.ts": [
|
||||
"/** A provider's non-2xx answer, after backoff for retryable statuses, or a particle 2xx without a message (its 2xx status). */",
|
||||
" * Particle runs only: the run ended on a turn that timed out, lost its connection after it went out, or was answered",
|
||||
" * 2xx without a message (`compatMayHaveBilled`). Particle may have billed that turn, so a caller never runs the",
|
||||
" * question again.",
|
||||
" * recorded, and a 2xx that is not JSON or has no `choices[0].message` is recorded and thrown as a ProviderError with",
|
||||
" * its 2xx status, never sent again, because the provider answered it and may have billed it (issue #280). A `compat`",
|
||||
" * request is sent with `compatDeadlineMs` instead of Bun's 300-second fetch timeout, and a 504 (a gateway's or the lease",
|
||||
" const answered = new ProviderError(res.status, `${compat} answered ${res.status} ${parsed ? 'without choices[0].message' : 'with a body that is not JSON'}: ${text.slice(0, 300)}`);",
|
||||
" onAttempt?.({ status: 'error', response: json, error: answered.message, ms: Date.now() - t, retry_after_ms: null });",
|
||||
" throw answered;"
|
||||
],
|
||||
"eval/runner/harness-metering-proxy.ts": [
|
||||
" * A particle request that failed after it went out (the upstream deadline",
|
||||
" * passed, the connection reset or closed, or the response body failed to",
|
||||
" * read) is answered 504 `mpw_proxy_upstream_lost`, which no particle client",
|
||||
" * sends again, because Particle may have billed it. Any other upstream",
|
||||
" * failure, such as a refused connection, is a 502",
|
||||
" * `mpw_proxy_upstream_unreachable` that a client may send again (issue #282).",
|
||||
" *",
|
||||
"import { carriesKeyInCleartext, COMPAT_PROVIDERS, compatConnectionLost } from './providers/openai-compatible.ts';",
|
||||
" const lost = provider === 'particle' && ((error as Error).name === 'TimeoutError' || compatConnectionLost(error));",
|
||||
" const message = `metering proxy ${lost ? 'lost the request it sent to' : 'could not reach'} ${provider}: ${(error as Error).message}`;",
|
||||
" return finish(lost ? errorResponse(504, 'mpw_proxy_upstream_lost', message) : errorResponse(502, 'mpw_proxy_upstream_unreachable', message), { ...settle(null, 0, message), error: message });",
|
||||
" return finish(provider === 'particle' ? errorResponse(504, 'mpw_proxy_upstream_lost', message) : errorResponse(502, 'mpw_proxy_upstream_unreachable', message), { ...settle(null, upstream.status, message), error: message });"
|
||||
],
|
||||
"eval/runner/providers/openai-compatible.ts": [
|
||||
"/** A non-2xx answer that is not a budget stop, or a 2xx answer without a usable `choices[0].message` (`compatSend`). */",
|
||||
" * A request the provider may have billed: it timed out (`compatTimedOut`), its connection was lost after it went out",
|
||||
" * (`compatConnectionLost`), or it was answered 2xx without a usable `choices[0].message` (an error carrying that 2xx",
|
||||
" * status, issue #280). A caller never sends it again, with or without a metering proxy in front.",
|
||||
" const status = (e as { status?: unknown } | null)?.status;",
|
||||
" return compatTimedOut(e) || compatConnectionLost(e) || (typeof status === 'number' && status >= 200 && status < 300);",
|
||||
" * decides any retry. A 2xx answer without `choices[0].message`, or whose body is not JSON, is an error carrying its 2xx",
|
||||
" * status and the usage it reported, never an empty answer, and it is not retryable: the provider answered, so it may",
|
||||
" * have billed it (`compatMayHaveBilled`, issue #280). `timeoutMs` (300 s unless given) is the only deadline: Bun's own",
|
||||
" * 300-second fetch timeout is off, so a longer deadline holds.",
|
||||
" catch { throw new CompatHttpError({ provider: req.provider, status: res.status, kind: 'server', retryable: false, retryAfterMs: null, body: `response body is not JSON: ${text.slice(0, 200)}` }); }",
|
||||
" if (!Array.isArray(choices) || !choices[0]?.message) throw new CompatHttpError({ provider: req.provider, status: res.status, kind: 'server', retryable: false, retryAfterMs: null, body: `response has no choices[0].message: ${text.slice(0, 200)}` }, parsed.usage);"
|
||||
],
|
||||
"eval/runner/q1/cell.ts": [
|
||||
" /** `final`: a try that must not be sent again (an OpenAI-compatible request that timed out, lost its connection or was answered 2xx without a message, which the provider may have billed). */"
|
||||
],
|
||||
"eval/runner/systems/baselines.ts": [
|
||||
" /** The last request timed out, lost its connection after it went out or was answered 2xx without a message; Particle may have billed it, so the cell never sends it again. */",
|
||||
" * (`compatDeadlineMs`), and one that times out, loses its connection after it went out or is answered 2xx without a",
|
||||
" * message (`compatMayHaveBilled`) is a `reader_error` that is never sent again, because Particle may have billed it. Each attempt is recorded, the failed ones with null response"
|
||||
],
|
||||
"eval/workload-suites/bench-infra.ts": [
|
||||
"import { COMPAT_PROVIDERS, CompatHttpError, ParticleCreditExhausted, compatMayHaveBilled, compatRequest, compatSend, type CompatRequest, type CompatResponse } from '../runner/providers/openai-compatible.ts';",
|
||||
" * not Bun's 300-second fetch timeout. A request Particle may have billed (`compatMayHaveBilled`: it timed out, lost its",
|
||||
" * connection after it went out, was answered 504, which the harness proxy sends when its upstream request failed after",
|
||||
" * it went out, or was answered 2xx without a message) is never sent again (issue #282).",
|
||||
" const retry = !compatMayHaveBilled(error) && (http ? http.retryable && attempt < 5 : !budget && attempt < 4);"
|
||||
],
|
||||
"scripts/particle-canary.ts": [
|
||||
"import { CompatHttpError, assertMetered, compatBaseUrl, compatMayHaveBilled, compatRequest, compatSend, isLoopbackUrl, type CanarySampling, type CompatLogprob, type CompatRequest } from '../eval/runner/providers/openai-compatible.ts';",
|
||||
"/**",
|
||||
" * Send the canary once per prompt (up to 3 attempts on a 429 or 5xx, honoring a capped Retry-After). A request Particle",
|
||||
" * may have billed (`compatMayHaveBilled`: a 504, or a 2xx without a message) is never sent again (issue #282). A budget",
|
||||
" * stop ends it after its receipt.",
|
||||
" */",
|
||||
" if (!(e instanceof CompatHttpError && e.retryable) || compatMayHaveBilled(e) || attempt === ATTEMPTS - 1) break;"
|
||||
]
|
||||
},
|
||||
"removed": {
|
||||
"eval/runner/cat40/loop.ts": [
|
||||
"/** A provider's non-2xx answer, after backoff for retryable statuses. */",
|
||||
" * Particle runs only: the run ended on a turn that timed out, or lost its connection after it went out",
|
||||
" * (`compatMayHaveBilled`). Particle may have billed that turn, so a caller never runs the question again.",
|
||||
" * recorded, and a 2xx that is not JSON or has no `choices[0].message` is retried as a server error. A `compat` request",
|
||||
" * is sent with `compatDeadlineMs` instead of Bun's 300-second fetch timeout, and a 504 (a gateway's or the lease",
|
||||
" last = new ProviderError(502, `${compat} answered ${res.status} ${parsed ? 'without choices[0].message' : 'with a body that is not JSON'}: ${text.slice(0, 300)}`);",
|
||||
" onAttempt?.({ status: 'error', response: json, error: last.message, ms: Date.now() - t, retry_after_ms: null });",
|
||||
" if (attempt < 5) await new Promise(r => setTimeout(r, Math.min(60_000, 2000 * 2 ** attempt) + Math.random() * 1000));",
|
||||
" continue;"
|
||||
],
|
||||
"eval/runner/harness-metering-proxy.ts": [
|
||||
"import { carriesKeyInCleartext, COMPAT_PROVIDERS } from './providers/openai-compatible.ts';",
|
||||
" const message = `metering proxy could not reach ${provider}: ${(error as Error).message}`;",
|
||||
" return finish(errorResponse(502, 'mpw_proxy_upstream_unreachable', message), { ...settle(null, 0, message), error: message });",
|
||||
" return finish(errorResponse(502, 'mpw_proxy_upstream_unreachable', message), { ...settle(null, upstream.status, message), error: message });"
|
||||
],
|
||||
"eval/runner/providers/openai-compatible.ts": [
|
||||
"/** A non-2xx answer that is not a budget stop. */",
|
||||
" * A request the provider may have billed: it timed out (`compatTimedOut`) or its connection was lost after it went out",
|
||||
" * (`compatConnectionLost`). A caller never sends it again, with or without a metering proxy in front.",
|
||||
" return compatTimedOut(e) || compatConnectionLost(e);",
|
||||
" * decides any retry. A 2xx answer without `choices[0].message` is a server error carrying the usage it reported,",
|
||||
" * never an empty answer. `timeoutMs` (300 s unless given) is the only deadline: Bun's own 300-second fetch timeout is",
|
||||
" * off, so a longer deadline holds.",
|
||||
" catch { throw new CompatHttpError({ provider: req.provider, status: res.status, kind: 'server', retryable: true, retryAfterMs: null, body: `response body is not JSON: ${text.slice(0, 200)}` }); }",
|
||||
" if (!Array.isArray(choices) || !choices[0]?.message) throw new CompatHttpError({ provider: req.provider, status: res.status, kind: 'server', retryable: true, retryAfterMs: null, body: `response has no choices[0].message: ${text.slice(0, 200)}` }, parsed.usage);"
|
||||
],
|
||||
"eval/runner/q1/cell.ts": [
|
||||
" /** `final`: a try that must not be sent again (an OpenAI-compatible request that timed out or lost its connection, which the provider may have billed). */"
|
||||
],
|
||||
"eval/runner/systems/baselines.ts": [
|
||||
" /** The last request timed out or lost its connection after it went out; Particle may have billed it, so the cell never sends it again. */",
|
||||
" * (`compatDeadlineMs`), and one that times out or loses its connection after it",
|
||||
" * went out (`compatMayHaveBilled`) is a `reader_error` that is never sent again, because Particle may have billed it. Each attempt is recorded, the failed ones with null response"
|
||||
],
|
||||
"eval/workload-suites/bench-infra.ts": [
|
||||
"import { COMPAT_PROVIDERS, CompatHttpError, ParticleCreditExhausted, compatRequest, compatSend, type CompatRequest, type CompatResponse } from '../runner/providers/openai-compatible.ts';",
|
||||
" * not Bun's 300-second fetch timeout, and a request that times out is never sent again.",
|
||||
" const timedOut = error.name === 'TimeoutError' || error.name === 'AbortError';",
|
||||
" const retry = http ? http.retryable && attempt < 5 : !budget && !timedOut && attempt < 4;"
|
||||
],
|
||||
"scripts/particle-canary.ts": [
|
||||
"import { CompatHttpError, assertMetered, compatBaseUrl, compatRequest, compatSend, isLoopbackUrl, type CanarySampling, type CompatLogprob, type CompatRequest } from '../eval/runner/providers/openai-compatible.ts';",
|
||||
"/** Send the canary once per prompt (up to 3 attempts on a 429 or 5xx, honoring a capped Retry-After). A budget stop ends it after its receipt. */",
|
||||
" if (!(e instanceof CompatHttpError && e.retryable) || attempt === ATTEMPTS - 1) break;"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -205,6 +205,7 @@ export const FREE_FIXTURES: Readonly<Record<string, readonly string[]>> = {
|
||||
'test/fixtures/remote-identity-vectors.json': ['test/bin-context-windows-slug.test.ts', 'test/gstack-slug-parity.test.ts', 'test/remote-identity-vectors.test.ts'],
|
||||
'test/fixtures/review-depth/**': ['test/review-dispatch-compare.test.ts'],
|
||||
'test/fixtures/review-ledger/**': ['test/review-dispatch.test.ts'],
|
||||
'test/fixtures/review-risk/gbrain-evals-289.json': ['test/review-risk-routing.test.ts'],
|
||||
'test/fixtures/review-design-lite-reports-ci-36633323521.json': ['test/review-finalization-budget.test.ts'],
|
||||
'test/fixtures/review-gate-integrity-opus-reports.json': ['test/gate-integrity-templates.test.ts'],
|
||||
'test/fixtures/session-update-heartbeat.sh': ['test/session-update-heartbeat.test.ts'],
|
||||
|
||||
@@ -56,6 +56,7 @@ const NO_LOSS_REPLAYS = {
|
||||
'G1': "GSTA-167 variant G Arm G1: live no-loss replay of both prose voices with part 1's prompts, 2 passes, normalizer v2, paired rule against pooled C' (parts 1 and 3; specialists at medium as in C and C2)",
|
||||
'L': "GSTA-167 variant G Arm L: live no-loss replay of one batched reader (testing and security plus scoped checklists) and the Red Team on its findings, 2 passes, scored L∘G1 against G1 on the paired rule (part 2)",
|
||||
'GST-257': "GST-257 PR 1 dispatch-superset no-loss check (test/review-targeted.test.ts): on every fixture diff the branch dispatches main's reviewers at 8be7f21 unchanged plus only the targeted reader, and over the last 100 non-release commits on main it lost none; adds the targeted role and brief to the G1 and L bytes, whose existing reviewer text is unchanged",
|
||||
'GST-850': "GST-850 dispatch-superset no-loss check: on the 26 GSTA-167 no-loss units the branch plan (wider security and money classes, write-up text in the batched diff) drops no reviewer role main dispatches, so no saved blocker can be lost, and over the last 100 main commits no commit with a gate or risk class drops depth (test/review-risk-routing.test.ts, test/review-risk-linear.test.ts); reviewer text is unchanged",
|
||||
'R51': "R51 risk-tier routing no-loss re-score (GSTA-167): the saved panel findings (349 records over 24 of the 26 no-loss units) re-scored against the branch's plan lose no blocker whose only finders are roles it drops, and every gate, security, user-data and money fixture keeps main's reviewers plus the full panel (test/review-targeted.test.ts, test/review-risk-routing.test.ts); reviewer text is unchanged",
|
||||
} as const;
|
||||
type Replay = keyof typeof NO_LOSS_REPLAYS;
|
||||
@@ -78,8 +79,8 @@ const REVIEWER_BRIEFS: Record<string, { sha: string; replay: Replay }> = {
|
||||
'review/specialists/testing.md': { sha: 'd6cc2bef7be562387d0c0a22fefe646dff9c295f1d34d8453a6cb496584150e4', replay: '#3638' },
|
||||
'review/specialists/targeted.md': { sha: '7f92a0d0a8319c3c0e076fb3b1a4689c45120b2be2c14dfc7d4cb03aafa9d6a5', replay: 'GST-257' },
|
||||
'scripts/resolvers/fix-check.ts': { sha: '712bf5b0c439e96bac32f19cbe25394dabb4c94aa66600100bfde9b100e15a08', replay: '#3638' },
|
||||
'lib/review-depth.ts': { sha: 'd649569a163c03476c6e0c5ea295f5d02088f96aab85fbc7d558f2ac2b27006a', replay: 'R51' },
|
||||
'lib/review-risk.ts': { sha: 'dbef7475d467e17340b21a2bacb52a16a7ccb1ee66291900ed2d36dfb5cfb7f2', replay: 'R51' },
|
||||
'lib/review-depth.ts': { sha: '839205a409014fb2cca6988a01aa04c98c495d8d24bd94895a86bf7794627338', replay: 'GST-850' },
|
||||
'lib/review-risk.ts': { sha: '18f0330ddfda0032eb93516b0eb941fa4244e0a68b028e19c436c74609093b03', replay: 'GST-850' },
|
||||
};
|
||||
|
||||
describe('reviewer briefs stay at their no-loss-tested bytes (GSTA-167)', () => {
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
/**
|
||||
* GST-850: every risk pattern (lib/review-risk.ts) finishes in linear time on hostile input, and every new one is in
|
||||
* RISK_RULES, which DEPTH_RULES folds in (docs/review-policy.md#risk-classes).
|
||||
*/
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import * as fs from 'node:fs';
|
||||
import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import { DATA_LINE, MONEY_LINE, SECRET_LINE, SECURITY_LINE, matchRisk } from '../lib/review-risk';
|
||||
|
||||
const ROOT = path.join(import.meta.dir, '..');
|
||||
const N = 10_000;
|
||||
const fast = (f: () => unknown) => { const t = performance.now(); f(); return performance.now() - t; };
|
||||
const scan = (lines: string[]) => matchRisk({ codePaths: ['src/a.ts'], added: { 'src/a.ts': lines }, removed: { 'src/a.ts': lines }, scopeAuth: false });
|
||||
|
||||
describe('linear time on hostile input', () => {
|
||||
// Value: protects=gstack-review-plan and the queue verdict never hang on a diff (the R51 resource regex did at about 50
|
||||
// backslashes; GST-850 rounds 1 and 2 found quadratic import and exec-binding scans); fails_when=any risk pattern
|
||||
// backtracks on a long or minified line, or on its anchor followed by a long run of the class it repeats; seam=every
|
||||
// pattern in SECURITY_LINE, DATA_LINE, SECRET_LINE and MONEY_LINE (R51's rm rule is a known pre-existing issue, filed apart), and matchRisk
|
||||
const L = 200_000;
|
||||
const fill = (s: string) => s.repeat(Math.ceil(L / s.length));
|
||||
const hostile = [' ', '(', "'", '"', 'a', '{', '\\', '..', 'import a, {', 'fetch(', 'path.join(', 'rmSync(', ' exec', 'new URL("', 'child_process', 'http.get(', 'requests.',
|
||||
'tar -', 'unzip ', 'DELETE FROM ', 'delete from a ', 'process.env.', 'x_usd', 'aUsd'].map(fill).concat([' exec' + ' '.repeat(L) + '// c', 'import a, {' + 'b,'.repeat(L / 2)],
|
||||
['fetch', 'globalThis.fetch', 'http.get', 'https.request', 'got', 'urlopen', 'urljoin', 'url.resolve', 'exec', 'execSync', 'cp.exec', 'proxy', 'redirect', 'maxRedirects', 'tar.extract', 'unzip', 'tar'].map(a => a + ' '.repeat(L)),
|
||||
['got.', 'requests.', 'httpx.', 'tar -', 'tar x', 'rm -', 'allowed_', 'host_allow', 'node_modules/adm-'].map(a => a + 'x'.repeat(L)),
|
||||
['new' + ' '.repeat(L) + 'URL(', 'new URL(' + ' '.repeat(L), 'new URL("' + 'a'.repeat(L), "'" + 'node:'.repeat(L / 5), 'path.join(' + 'a'.repeat(L), 'path.join(' + ')'.repeat(L),
|
||||
"includes(" + ' '.repeat(L), '==' + ' '.repeat(L) + "'..'", "'..'" + ' '.repeat(L) + '==', '\\.' + '\\'.repeat(L)]);
|
||||
test.each(Object.entries({ SECURITY_LINE, DATA_LINE, SECRET_LINE, MONEY_LINE }))('every %s pattern finishes under 100 ms on each 200k-character hostile line', (_n, patterns) => {
|
||||
const slow = patterns.flatMap(r => hostile.map((s, k) => [r.source.slice(0, 40), k, fast(() => r.test(s))] as const)).filter(([, , ms]) => ms >= 100);
|
||||
expect(slow).toEqual([]);
|
||||
});
|
||||
test('a 512 KB comment of unclosed imports and a minified import line run through matchRisk in under 1 s', () => {
|
||||
expect(fast(() => scan(Array.from({ length: 65_536 }, () => 'import {')))).toBeLessThan(1000);
|
||||
expect(fast(() => scan(['import a, {'.repeat(32_000)]))).toBeLessThan(1000);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the rules hashes cover every new pattern', () => {
|
||||
// Value: protects=a receipt or queue verdict notices a risk-pattern edit; fails_when=an edit to a security pattern or a money
|
||||
// word leaves DEPTH_RULES unchanged; seam=DEPTH_RULES over RISK_RULES
|
||||
test.each([
|
||||
["'billable', 'billed'", "'billable', 'billed', 'tariff'"],
|
||||
['extract-zip|unzipper', 'extract-zip|unzipper|jszip'],
|
||||
["['proxy', ['billing', 'bill']]", "['proxy', ['billing', 'bill', 'meter']]"],
|
||||
])('editing review-risk.ts (%s) changes DEPTH_RULES', (needle, to) => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'risk-rules-'));
|
||||
try {
|
||||
for (const f of ['review-depth.ts', 'review-risk.ts']) fs.copyFileSync(path.join(ROOT, 'lib', f), path.join(dir, f));
|
||||
fs.symlinkSync(path.join(ROOT, 'lib/ship-policy.ts'), path.join(dir, 'ship-policy.ts'));
|
||||
fs.writeFileSync(path.join(dir, 'print.ts'), "import { DEPTH_RULES } from './review-depth';\nconsole.log(DEPTH_RULES);\n");
|
||||
const hash = () => {
|
||||
const r = spawnSync(process.execPath, ['--no-env-file', '--config=/dev/null', path.join(dir, 'print.ts')], { encoding: 'utf8', timeout: 30_000 });
|
||||
expect(r.status, r.stderr).toBe(0);
|
||||
return r.stdout.trim();
|
||||
};
|
||||
const before = hash();
|
||||
const src = fs.readFileSync(path.join(dir, 'review-risk.ts'), 'utf8');
|
||||
expect(src).toContain(needle);
|
||||
fs.writeFileSync(path.join(dir, 'review-risk.ts'), src.replace(needle, to));
|
||||
expect(before).toMatch(/^[0-9a-f]{16}$/);
|
||||
expect(hash()).not.toBe(before);
|
||||
} finally { fs.rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
});
|
||||
@@ -121,3 +121,183 @@ describe('the receipt records the tier and the rounds', () => {
|
||||
expect(receiptSummaryLine(legacy)).not.toContain('rounds=');
|
||||
});
|
||||
});
|
||||
|
||||
const SVG_PATH = 'docs/evals/chart.svg';
|
||||
const CHART = ['<?xml version="1.0" encoding="UTF-8"?>', '<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 200 100" role="img" aria-label="recall by arm">', ' <title>Recall by arm</title>',
|
||||
' <defs><linearGradient id="g"><stop offset="0" stop-color="#4a90d9"/><stop offset="1" stop-color="#1c5fa8"/></linearGradient></defs>', ' <g font-family="sans-serif" font-size="10">',
|
||||
' <rect x="10" y="20" width="40" height="70" fill="url(#g)" style="opacity: 0.9"/>', ' <path d="M10 90 H190" stroke="#333" stroke-width="1"/>', ' <text x="30" y="15" text-anchor="middle">33/41 & up</text>', ' <use href="#g"/>', ' </g>', '</svg>', ''];
|
||||
const classOf = (p: string, added: string[], removed: string[] = [], created: string[] = []) =>
|
||||
classifyDiff({ paths: [p], created, gateTagged: 0, added: { [p]: added }, removed: { [p]: removed }, perPath: [{ path: p, lines: added.length + removed.length, binary: false }] }).class;
|
||||
const svgClass = (lines: string[]) => classOf(SVG_PATH, lines, [], [SVG_PATH]);
|
||||
const swap = (from: string, to: string) => CHART.map(l => l.replace(from, to));
|
||||
|
||||
describe('GST-850 choice A: a clean chart SVG and a locale value edit stay code (light)', () => {
|
||||
// Value: protects=UI text and SVGs keep one batched reader plus the native adversarial pass (the allowlist docs paths were dropped
|
||||
// for review cost); fails_when=a clean chart or a locale value edit plans as docs, or the batched specialist drops a write-up
|
||||
// that publishes numbers; seam=classifyDiff and planReview
|
||||
test('a new clean chart SVG is code, and a write-up with it plans light with the batched reader', () => {
|
||||
expect(svgClass(CHART)).toBe('code');
|
||||
const md = 'docs/evals/run.md';
|
||||
const added = { [md]: ['Recall rose to 33/41.'], [SVG_PATH]: CHART };
|
||||
const p = plan({ paths: [md, SVG_PATH], created: [SVG_PATH], added, removed: {}, perPath: [{ path: md, lines: 1, binary: false }, { path: SVG_PATH, lines: CHART.length, binary: false }], numbers: scanNumbers(added) });
|
||||
expect([p.depth, roles(p)]).toEqual(['light', ['specialist', 'adversarial-native', 'outside-voice']]);
|
||||
});
|
||||
test('the batched specialist reads a write-up that publishes numbers, and leaves docs out when nothing is published', () => {
|
||||
const md = 'docs/evals/run.md', code = 'scripts/score.ts';
|
||||
const base = { paths: [code, md], added: { [code]: ['export const k = 1;'], [md]: ['Recall rose to 33/41.'] }, removed: {}, perPath: [{ path: code, lines: 1, binary: false }, { path: md, lines: 1, binary: false }] };
|
||||
const batched = (numbers: ReturnType<typeof scanNumbers>) => plan({ ...base, numbers }).dispatches.find(d => d.name === 'batched')!.exclude.join(' ');
|
||||
expect(batched(scanNumbers(base.added))).not.toContain(md);
|
||||
expect(batched(none)).toContain(md);
|
||||
});
|
||||
test.each([
|
||||
['json', 'locales/en.json', ' "save": "Save",', ' "save": "Save now",'],
|
||||
['arb', 'lib/l10n/app_en.arb', ' "greeting": "Hello {name}",', ' "greeting": "Hi there {name}",'],
|
||||
['yaml quoted', 'app/i18n/en.yml', ' save: "Save"', ' save: "Save now"'],
|
||||
['yaml plain', 'locales/en.yaml', ' cancel: Cancel', ' cancel: Cancel it'],
|
||||
['po', 'locale/fr/LC_MESSAGES/app.po', 'msgstr "Enregistrer %s"', 'msgstr "Sauvegarder %s"'],
|
||||
['strings', 'Resources/lang/en.lproj/Localizable.strings', '"save" = "Save";', '"save" = "Save now";'],
|
||||
['xliff', 'translations/messages.fr.xlf', ' <target>Enregistrer</target>', ' <target>Sauvegarder</target>'],
|
||||
['ftl', 'locales/en-US/main.ftl', 'welcome = Welcome, { $user }!', 'welcome = Hello, { $user }!'],
|
||||
['properties', 'src/main/resources/i18n/messages_en.properties', 'button.save=Save', 'button.save=Save now'],
|
||||
])('a %s value edit under a locale directory is code', (_n, p, before, after) => {
|
||||
expect(classOf(p, [after], [before])).toBe('code');
|
||||
});
|
||||
});
|
||||
|
||||
describe('GST-850: every bypass of the line-pattern docs paths stays code', () => {
|
||||
// Value: protects=active or bound content never plans as docs; fails_when=any bypass from GST-850 or a sibling spelling classes docs;
|
||||
// seam=classifyDiff
|
||||
test.each([
|
||||
['namespaced <s:script>', ['<svg xmlns:s="http://www.w3.org/2000/svg"><s:script>window.x=42</s:script></svg>']],
|
||||
['a <script> element', swap('<title>', '<script>alert(1)</script><title>')],
|
||||
['an uppercase SCRIPT', swap('<title>', '<SCRIPT>alert(1)</SCRIPT><title>')],
|
||||
['a multi-line onload attribute', ['<svg xmlns="http://www.w3.org/2000/svg"', ' onload', ' ="alert(1)">', '</svg>']],
|
||||
['an onclick handler', swap('<rect ', '<rect onclick="alert(1)" ')],
|
||||
['a Vue :fill binding', swap('<rect ', '<rect :fill="leak(document.cookie)" ')],
|
||||
['a v-bind: binding', swap('<rect ', '<rect v-bind:fill="x" ')],
|
||||
['an Alpine x-bind: binding', swap('<rect ', '<rect x-bind:fill="x" ')],
|
||||
['an @click binding', swap('<rect ', '<rect @click="x()" ')],
|
||||
['mustache text', swap('33/41 & up', '{{ constructor.constructor("alert(1)")() }}')],
|
||||
['a foreignObject', swap('<title>', '<foreignObject><div>x</div></foreignObject><title>')],
|
||||
['an animate on href', swap('<use href="#g"/>', '<use href="#g"><animate attributeName="href" to="javascript:alert(1)"/></use>')],
|
||||
['a set element', swap('<use href="#g"/>', '<set attributeName="fill" to="red"/>')],
|
||||
['a doctype with an entity', ['<!DOCTYPE svg [<!ENTITY x "y">]>', ...CHART.slice(1)]],
|
||||
['an xlink:href', swap('<use href="#g"/>', '<use xlink:href="#g"/>')],
|
||||
['an outside use href', swap('<use href="#g"/>', '<use href="https://evil.example/x.svg#a"/>')],
|
||||
['an href off use', swap('<text x="30"', '<text href="#g" x="30"')],
|
||||
['a style url(', swap('style="opacity: 0.9"', 'style="fill: url(#g)"')],
|
||||
['a style @import', swap('style="opacity: 0.9"', 'style="@import url(x.css)"')],
|
||||
['an outside fill url', swap('fill="url(#g)"', 'fill="url(https://evil.example/x)"')],
|
||||
['a CSS-escaped url', swap('fill="url(#g)"', 'fill="\\75rl(https://evil.example/x)"')],
|
||||
['an xml-stylesheet instruction', ['<?xml-stylesheet href="https://evil.example/x.css"?>', ...CHART.slice(1)]],
|
||||
['a CDATA section', swap('33/41 & up', '<![CDATA[<script>x</script>]]>')],
|
||||
['an image element', swap('<title>', '<image href="https://evil.example/x.png"/><title>')],
|
||||
['a style element', swap('<title>', '<style>@import url(x.css);</style><title>')],
|
||||
['another namespace', swap('xmlns="http://www.w3.org/2000/svg"', 'xmlns="http://www.w3.org/1999/xhtml"')],
|
||||
['a numeric entity in an attribute', swap('fill="url(#g)"', 'fill="url(https://evil.example/x)"')],
|
||||
['an unclosed root', CHART.slice(0, -2)],
|
||||
['two roots', [...CHART, '<svg xmlns="http://www.w3.org/2000/svg"></svg>']],
|
||||
['text after the root', [...CHART, 'trailing']],
|
||||
['an HTML-closed empty comment <!-->', swap('<title>', '<!--><image width="1" height="1"/>--><title>')],
|
||||
['an HTML-closed empty comment <!--->', swap('<title>', '<!---><foreignObject/>--><title>')],
|
||||
['numeric references spelling an interpolation', swap('33/41 & up', '{{run()}}')],
|
||||
['an attribute interpolation', swap('<text x="30"', '<text class="{{run()}}" x="30"')],
|
||||
])('a new SVG with %s is code', (_n, lines) => {
|
||||
expect(svgClass(lines as string[])).toBe('code');
|
||||
});
|
||||
test('an edited SVG is code even when both sides are clean', () => {
|
||||
expect(classOf(SVG_PATH, CHART, swap('33/41', '30/41'))).toBe('code');
|
||||
expect(classOf(SVG_PATH, CHART)).toBe('code');
|
||||
});
|
||||
test.each([
|
||||
['i18n .ts', 'src/i18n/en.ts', " save: 'Save',", " save: 'Save now',"],
|
||||
['string-concat HTML', 'src/App.tsx', "const html = '<h1>' + title + '</h1>';", "const html = '<h1>' + leak() + '</h1>';"],
|
||||
['an edit inside an existing <script>', 'src/App.vue', ' const label = "Save";', ' const label = "Save now";'],
|
||||
['a Vue :title binding', 'src/App.vue', '<div :title="user.name">Profile</div>', '<div :title="leak(document.cookie)">Profile</div>'],
|
||||
['a markup value', 'locales/en.json', ' "save": "Save",', ' "save": "<img src=x onerror=alert(1)>",'],
|
||||
['a changed placeholder', 'locales/en.json', ' "hi": "Hi {name}",', ' "hi": "Hi {user.password}",'],
|
||||
['a new mustache expression', 'locales/en.json', ' "hi": "Hi",', ' "hi": "Hi {{constructor.constructor(1)()}}",'],
|
||||
['a vue-i18n linked message', 'locales/en.json', ' "hi": "Hi",', ' "hi": "Hi @:secret.key",'],
|
||||
['an i18next nesting', 'locales/en.json', ' "hi": "Hi",', ' "hi": "Hi $t(secret)",'],
|
||||
['a renamed key', 'locales/en.json', ' "save": "Save",', ' "store": "Save",'],
|
||||
['an escaped character', 'locales/en.json', ' "save": "Save",', ' "save": "\\u003cscript\\u003e",'],
|
||||
['a script URL', 'locales/en.yml', ' link: "https://example.com"', ' link: "javascript:alert(1)"'],
|
||||
['a locale file outside a locale directory', 'src/en.json', ' "save": "Save",', ' "save": "Save now",'],
|
||||
['a locale file under config/', 'config/locales/en.yml', ' save: "Save"', ' save: "Save now"'],
|
||||
['an escaped script scheme', 'locales/en.json', ' "go": "Continue",', ' "go": "java\\nscript:alert(1)",'],
|
||||
['a protocol-relative link', 'locales/en.json', ' "help": "[docs](/help)",', ' "help": "[docs](//evil.example/help)",'],
|
||||
['a YAML null', 'locales/en.yml', ' label: Save', ' label: null'],
|
||||
['a YAML boolean', 'locales/en.yml', ' label: Save', ' label: false'],
|
||||
['a renamed gettext placeholder', 'locale/fr/LC_MESSAGES/app.po', 'msgstr "Bonjour %(name)s"', 'msgstr "Bonjour %(password)s"'],
|
||||
['an Ansible task under a locale role', 'roles/locale/tasks/main.yml', ' command: a-tool one', ' command: b-tool two'],
|
||||
['a non-message .properties under lang/', 'src/main/resources/lang/app.properties', 'db.url=host-a', 'db.url=host-b'],
|
||||
['a non-locale JSON under lang/', 'tools/lang/rules.json', ' "pattern": "^[a-z]+$",', ' "pattern": ".*",'],
|
||||
['a locale value carrying a risk line', 'src/i18n/en.yml', ' command: echo ok', ' command: rm -rf /tmp/x'],
|
||||
])('%s is code', (_n, p, before, after) => {
|
||||
expect(classOf(p, [after], [before])).toBe('code');
|
||||
});
|
||||
test('an added key and the backtracking input are code, and the input finishes fast', () => {
|
||||
expect(classOf('locales/en.json', [' "extra": "Extra",'])).toBe('code');
|
||||
expect(classOf('src/i18n/en.yml', [' title: Hello', ' sub: World'], [' title: Hello', ' sub: World'])).toBe('code');
|
||||
const t = performance.now();
|
||||
expect(classOf('locales/en.yml', [' save: "' + '\\'.repeat(10_000) + 'x'], [' save: "Save"'])).toBe('code');
|
||||
expect(classOf('locales/en.yml', [' save: "' + '\\'.repeat(9_999) + '"'], [' save: "Save"'])).toBe('code');
|
||||
expect(performance.now() - t).toBeLessThan(100);
|
||||
});
|
||||
});
|
||||
|
||||
describe('GST-850: security reaches outbound URLs, archives, path traversal and every child_process spelling', () => {
|
||||
// Value: protects=an SSRF, archive-extraction, traversal or shell-exec diff reaches the full panel; fails_when=a spelling of any of
|
||||
// them reads light, or a literal URL, a spawnSync import or a tarball create raises a class; seam=matchRisk
|
||||
test.each([
|
||||
"import { exec } from 'node:child_process';", 'import { execSync as run } from "child_process";', "import { spawn, spawnSync } from 'node:child_process';",
|
||||
"import cp from 'node:child_process';", "import * as cp from 'child_process';", "import cp, { spawnSync } from 'child_process';",
|
||||
"const { exec } = require('child_process');", "const cp = require('node:child_process');", "const cp = await import('node:child_process');",
|
||||
'exec(cmd, cb);', 'cp.exec(cmd);', "import { spawnSync } from 'node:child_process';", "import type { ExecOptions } from 'node:child_process';",
|
||||
"const r = await fetch('https://api.github.com/repos');", 'r = requests.request("GET", user_url)', "http.get('http://x.example/', { hostname: req.query.t }, cb);", 'const res = await fetch(url);', "await fetch('https://' + host + '/v1');", 'https.get(target, cb);', 'await axios.get(req.query.u);', 'const body = await got(url).text();',
|
||||
'r = requests.get(url, timeout=5)', 'urllib.request.urlopen(target)', 'const next = new URL(req.query.next, base);', "import { request } from 'undici';",
|
||||
"const res = await fetch('https://x.example', { redirect: 'follow' });", 'maxRedirects: 10,', 'r = httpx.get("https://x.example", follow_redirects=True)', 'const p = process.env.HTTPS_PROXY;', 'const agent = new HttpsProxyAgent(p);',
|
||||
"const ALLOWED_HOSTS = ['api.example.com'];", 'hostAllowlist.push(h);',
|
||||
'await tar.x({ file, cwd: dest });', 'new AdmZip(buf).extractAllTo(dir, true);', "import yauzl from 'yauzl';", "import extract from 'extract-zip';", 'zipfile.ZipFile(p).extractall(d)',
|
||||
'tarfile.open(p).extractall(d)', 'tar -xzf "$TGZ" -C "$DIR"', 'unzip -o "$ZIP" -d "$DIR"', 'fs.writeFileSync(path.join(root, entry.path), data);', 'const dest = path.resolve(out, req.params.name);',
|
||||
"if (rel.startsWith('..')) throw new Error('escape');", "if (parts.includes('..')) return;",
|
||||
])('%s is security', line => {
|
||||
expect(risk(['src/net.ts'], { 'src/net.ts': [line] })).toEqual(['security']);
|
||||
});
|
||||
test('a named import split over lines is security, from either side of the diff', () => {
|
||||
const split = ['import {', ' spawnSync,', ' exec,', "} from 'node:child_process';"];
|
||||
expect(risk(['src/run.ts'], { 'src/run.ts': split })).toEqual(['security']);
|
||||
expect([...matchRisk({ codePaths: ['src/run.ts'], added: {}, removed: { 'src/run.ts': split }, scopeAuth: false }).keys()]).toEqual(['security']);
|
||||
expect(risk(['src/run.ts'], { 'src/run.ts': ['import {', ' type ExecOptions,', ' exec as run,', "} from 'node:child_process';"] })).toEqual(['security']);
|
||||
});
|
||||
test('a literal new URL, tarball creation, parent-dir joins and look-alike names are not security', () => {
|
||||
for (const line of ["const u = new URL('https://example.com');", 'const here = new URL(import.meta.url);', 'tar -czf out.tgz dir', "const root = path.join(import.meta.dir, '..');",
|
||||
'const m = re.exec(line);', "ENV.fetch('HOME')", 'git fetch origin main'])
|
||||
expect(risk(['src/run.ts'], { 'src/run.ts': [line] })).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('GST-850: money reaches metering, charge and credit code', () => {
|
||||
// Value: protects=a billing-client or metering diff gets the money reader; fails_when=gbrain-evals #289 reads required=none again,
|
||||
// or costume, a CLI usage line or a proxy with no billing names money; seam=matchRisk and targetedClasses
|
||||
test('gbrain-evals #289 (metering proxy and particle billing clients) reads money, each code path on its own', () => {
|
||||
const x = JSON.parse(fs.readFileSync(path.join(ROOT, 'test/fixtures/review-risk/gbrain-evals-289.json'), 'utf8'));
|
||||
expect(targetedClasses({ paths: x.paths, added: x.added, removed: x.removed, scope: { SCOPE_AUTH: 'false' }, gateTagged: 0 }).classes).toEqual(['money']);
|
||||
expect(classifyDiff({ paths: x.paths, added: x.added, removed: x.removed, gateTagged: 0 }).risk).toContain('money');
|
||||
const code = Object.keys({ ...x.added, ...x.removed });
|
||||
expect(code.length).toBe(7);
|
||||
for (const p of code) expect(matchRisk({ codePaths: [p], added: { [p]: x.added[p] ?? [] }, removed: { [p]: x.removed[p] ?? [] }, scopeAuth: false }).has('money')).toBe(true);
|
||||
});
|
||||
test('metering, charge and credit words and pairs name money; costume, CLI usage and a plain proxy do not', () => {
|
||||
expect(risk(['eval/runner/harness-metering-proxy.ts'])).toEqual(['money']);
|
||||
expect(risk(['lib/usage-tokens.ts'])).toEqual(['money']);
|
||||
for (const line of ['if (compatMayHaveBilled(e)) break;', 'const left = credits - spent;', 'await meter.record(n);', 'const usd = res.usage.output_tokens * rate;',
|
||||
'const billingProxy = startProxy();', 'export class CostClient {', 'const billable = true;'])
|
||||
expect(risk(['src/run.ts'], { 'src/run.ts': [line] })).toEqual(['money']);
|
||||
expect(risk(['src/costume.ts'])).toEqual([]);
|
||||
expect(risk(['src/cli/usage.ts'], { 'src/cli/usage.ts': ['export function printUsage() { console.log(USAGE); }'] })).toEqual([]);
|
||||
expect(risk(['src/proxy.ts'], { 'src/proxy.ts': ['const target = proxyTarget(host);'] })).toEqual([]);
|
||||
expect(risk(['src/geometry.ts'], { 'src/geometry.ts': ['const diameter = 2 * r; // a parameter'] })).toEqual([]);
|
||||
expect(classifyDiff({ paths: ['docs/usage.md'], gateTagged: 0, added: { 'docs/usage.md': ['Usage: gstack run'] }, removed: {} }).risk).toEqual([]);
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user