mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-12 11:40:51 +01:00
fix(review-risk): keep reservations, estimates, spend guards and pricing tables money (GST-961, GST-962 review)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
5f43fa66a3
commit
c3b5e7040d
7 files changed
+52
-25
No files matched your search
@@ -1,8 +1,8 @@
|
||||
### Fixed
|
||||
|
||||
- **A docs or data PR no longer gets a money reader because a file name says "cost", and reading a `cost_usd` field no longer counts as money code (GST-961, GST-962).** On gstack-evals#138, a results write-up planned as docs at light depth still dispatched a second, targeted money reader, because `wiki/.../cost-gstack-full.tsv` has "cost" in its name. On gstack-evals#139, an analysis module that only sums and prints per-trial `cost_usd` raised the money class. A data file that no code reads now never counts as money, by its name or its rows, and a `_usd` or `Usd` name counts only when it says cap, limit, max, budget, spend, price, fee, bill or payment, or when a `<` or `>` compares it. Spend caps, budgets, billing paths and paid-provider calls still count. Every other class still reads unread data files. Over the last 100 main commits, no commit's depth, class, risk classes or targeted classes changed.
|
||||
- **A docs or data PR no longer gets a money reader because a file name says "cost", and summing or printing a `cost_usd` field no longer counts as money code (GST-961, GST-962).** On gstack-evals#138, a results write-up planned as docs at light depth still dispatched a second, targeted money reader, because `wiki/.../cost-gstack-full.tsv` has "cost" in its name. On gstack-evals#139, an analysis module that only sums and prints per-trial `cost_usd` raised the money class. Now a data file that no code reads, under `docs/`, `wiki/`, `results/` or `reports/`, never counts as money for its name or for a reported cost. A spend cap or paid-provider call in that file still counts. A `_usd` or `Usd` name is not money only when it is a reported cost (made of words like cost, total, per, trial or sum) on a line with no cap, limit, budget, price, charge, reservation or estimate word, no `<` or `>` comparison and no number set on it. Reservations, estimates, amounts, refunds, spend guards and pricing tables all still count. Over the last 100 main commits, no commit's depth, class, risk classes or targeted classes changed.
|
||||
|
||||
### Tests
|
||||
|
||||
- `test/review-targeted.test.ts`: #138's file list plans no targeted reader, and the same file read by code still plans money. Neither a `cost_usd` read nor a `cost_usd` report names money. Positive controls: caps, budgets, limits, spend checks, comparisons, `spend_cap`, a billing path, a provider client, and a secret in an unread data file.
|
||||
- `test/review-risk-redos.test.ts`: `usdMoney` stays under 250 ms on 100,000-character lines of `_usd` names, comparisons and member chains.
|
||||
- `test/review-targeted.test.ts`: #138's file list plans no targeted reader, and the same file read by code still plans money. A `cost_usd` that is only summed or printed names no money. Positive controls: caps, budgets, limits, optional-chain and wrapped comparisons, `Math.min`, charges and refunds, reservations and estimates from `lib/eval-plan.ts` and `lib/autoplan-run.ts`, the `trial_reservation_usd` validator, a pricing table outside a results tree, a spend cap in an unread results file, and a secret in an unread data file.
|
||||
- `test/review-risk-redos.test.ts`: `usdMoney` and its comparison patterns stay under 250 ms on 100,000-character lines.
|
||||
@@ -385,7 +385,7 @@ and matching is token-bounded: `costume` never reads as cost.
|
||||
|
||||
| Class | Signal |
|
||||
|---|---|
|
||||
| money | a path segment naming cost, spend, budget, ledger, billing, price, fee or invoice; or a changed line with an identifier ending in `_usd` or `Usd` whose name says cap, limit, max, budget, ceiling, allowance, spend, price, fee, bill or payment, or that a `<` or `>` compares (reading or reporting `cost_usd` is not money, GST-962); any `spend_cap` spelling; or a call to a paid provider `lib/pricing.ts` prices. A data file no code reads (the same consumer scan the depth uses) never counts as money by its name or its rows (GST-961); every other class still reads it |
|
||||
| money | a path segment naming cost, spend, budget, ledger, billing, price, fee or invoice; or a changed line with an identifier ending in `_usd` or `Usd`, unless every such name on the line is a reported cost (made only of words like cost, total, per, trial, mean or sum) and the line has no cap, limit, budget, price, charge, reservation or estimate word, no `<` or `>` comparison and no number set on it: summing or printing `cost_usd` is not money (GST-962); any `spend_cap` spelling; or a call to a paid provider `lib/pricing.ts` prices. A data file no code reads (the same consumer scan the depth uses) under a `docs/`, `wiki/`, `results/` or `reports/` directory counts as money only for a `spend_cap` or paid-provider line, never for its name or a reported cost (GST-961); every other class still reads it |
|
||||
| gate | a hunk the gate scan tagged: a relaxed threshold, a skipped test, a loosened check |
|
||||
| auth | `SCOPE_AUTH` from `gstack-diff-scope`, on a code path with an auth, session, jwt, permission or role token in its name; a data file named for sessions (`recovered-sessions.jsonl`) outside a config location does not set `SCOPE_AUTH`, and `SCOPE_AUTH` fails safe: any name token containing auth sets it (`auth.ts`, `/auth/`, `oauth`, `useAuth.ts`, `nextauthmiddleware.ts`) except the authorship words `author`, `authors`, `authored`, `authoring`, `authorship`, `authority`, `authorities`, `coauthor` and `coauthors` |
|
||||
| secret | a secrets, credentials, keychain or token path, or a changed line that reads a key-named environment variable (`OPENAI_API_KEY`, `GH_TOKEN`) or a credential store |
|
||||
|
||||
+20
-13
@@ -22,11 +22,17 @@ export const MONEY_PATH_TOKENS = ['cost', 'costs', 'spend', 'spending', 'budget'
|
||||
export const MONEY_PAIRS: readonly (readonly [string, readonly string[]])[] = [
|
||||
['usage', ['token', 'tokens', 'cost', 'costs', 'spend', 'billing', 'price', 'pricing', 'budget']], ['proxy', ['billing', 'bill']], ['client', ['cost', 'costs', 'charge', 'charges', 'billing', 'price', 'pricing', 'spend']],
|
||||
];
|
||||
/** Identifiers ending in _usd or Usd: money only beside a word in MONEY_USD_WORDS or in a comparison (usdMoney). */
|
||||
/** Identifiers ending in _usd or Usd: money unless the line only reads or reports a cost (usdMoney). */
|
||||
export const MONEY_USD = [/\b[A-Za-z0-9_]*_usd\b/i, /\b[a-z][A-Za-z0-9]*Usd\b/];
|
||||
const MONEY_USD_ALL = MONEY_USD.map(r => new RegExp(r.source, r.flags + 'g'));
|
||||
/** What makes a _usd identifier money (GST-962: reading or reporting gstack-evals#139's per-trial cost_usd is not money code). */
|
||||
export const MONEY_USD_WORDS = ['cap', 'caps', 'limit', 'limits', 'max', 'maximum', 'budget', 'budgets', 'ceiling', 'allowance', 'spend', 'spent', 'spending', 'price', 'prices', 'pricing', 'fee', 'fees', 'bill', 'billing', 'paid', 'payment'];
|
||||
/** The words a reported cost's name is made of (GST-962: gstack-evals#139's per-trial cost_usd); any other word, a reservation, estimate or amount, keeps the name money. */
|
||||
export const REPORTED_COST_WORDS = ['cost', 'costs', 'total', 'totals', 'usd', 'per', 'trial', 'trials', 'item', 'items', 'mean', 'avg', 'average', 'median', 'sum', 'streamed', 'transcript', 'observed', 'actual', 'reported', 'recorded'];
|
||||
/** A reported cost is still money on a line naming a cap, limit, budget, price, charge, reservation or estimate. */
|
||||
export const MONEY_USD_WORDS = ['cap', 'caps', 'limit', 'limits', 'max', 'maximum', 'min', 'budget', 'budgets', 'ceiling', 'allowance', 'threshold', 'spend', 'spent', 'spending', 'price', 'prices', 'pricing', 'fee', 'fees', 'bill', 'billing', 'paid', 'payment', 'charge', 'refund', 'amount', 'invoice', 'reserve', 'reservation', 'estimate'];
|
||||
/** A reported cost compared with `<` or `>` (never an arrow or a shift), or set to a number. */
|
||||
export const USD_COMPARE = [/(?<![=\-<>])[<>](?![<>])/, /(?:_usd|Usd)\b["']?\s{0,20}[:=]\s{0,20}-?\d/i];
|
||||
/** Docs and results trees: an unread data file there names no money by its path or a reported cost (GST-961). */
|
||||
export const RESULTS_TREE = /(^|\/)(?:docs?|wiki|results|reports)\//i;
|
||||
/** Any spend_cap spelling and paid-provider call sites (the providers lib/pricing.ts prices). */
|
||||
export const PAID_MONEY_LINE = [
|
||||
/\bspend[_-]?cap\b/i,
|
||||
@@ -34,11 +40,12 @@ export const PAID_MONEY_LINE = [
|
||||
/\bnew\s+(?:Anthropic|OpenAI|AnthropicBedrock|AnthropicVertex)\s*\(/, /\.(?:messages|chat\.completions|responses|embeddings)\.create\s*\(/,
|
||||
];
|
||||
export const MONEY_LINE = [...MONEY_USD, ...PAID_MONEY_LINE];
|
||||
/** A _usd identifier whose name says cap, budget, limit, spend or price, or that a `<` or `>` compares; never one an arrow reads. */
|
||||
export const usdMoney = (line: string) => MONEY_USD_ALL.some(r => [...line.matchAll(r)].some(m =>
|
||||
tokens(m[0]).some(x => MONEY_USD_WORDS.includes(x))
|
||||
|| /^\s{0,20}[<>]/.test(line.slice(m.index + m[0].length, m.index + m[0].length + 21))
|
||||
|| /(?<![=-])[<>]=?\s{0,20}(?:[\w$]{1,100}\.){0,10}$/.test(line.slice(Math.max(0, m.index - 300), m.index))));
|
||||
/** A _usd or Usd name is money unless every one on the line is a reported cost the line only reads: no cap word, comparison or number set. */
|
||||
export const usdMoney = (line: string) => {
|
||||
const names = MONEY_USD_ALL.flatMap(r => [...line.matchAll(r)].map(m => m[0]));
|
||||
if (!names.length) return false;
|
||||
return names.some(n => tokens(n).some(x => !REPORTED_COST_WORDS.includes(x))) || tokens(line).some(x => MONEY_USD_WORDS.includes(x)) || USD_COMPARE.some(r => r.test(line));
|
||||
};
|
||||
export const AUTH_PATH_TOKENS = ['auth', 'authn', 'authz', 'oauth', 'oauth2', 'jwt', 'session', 'sessions', 'permission', 'permissions', 'role', 'roles'];
|
||||
export const AUTH_PATH_PREFIXES = ['authenticat', 'authoriz'];
|
||||
export const SECRET_PATH = /(^|\/)(secrets?|credentials?|keychain|token[s]?)([._/-]|$)/i;
|
||||
@@ -93,8 +100,8 @@ const namesMoney = (t: readonly string[], words: readonly string[]) => t.some(x
|
||||
/** A hash of the patterns that decide the risk classes, not of this file's bytes; lib/review-depth.ts folds it into DEPTH_RULES. */
|
||||
export const RISK_RULES = createHash('sha256').update(JSON.stringify({
|
||||
RISK_CLASSES, MONEY_PATH_TOKENS, AUTH_PATH_TOKENS, AUTH_PATH_PREFIXES, KEY_NAME, SECURITY_PATH_TOKENS, DATA_PATH_TOKENS,
|
||||
MONEY_LINE_TOKENS, MONEY_PAIRS, MONEY_USD_WORDS,
|
||||
patterns: Object.entries({ MONEY_LINE, SECRET_LINE, SECRET_PATH: [SECRET_PATH], SECURITY_LINE, SECURITY_JOINED, DATA_LINE, SVG_PATH: [SVG_PATH], SVG_ACTIVE }).map(([k, v]) => [k, v.map(r => '/' + r.source + '/' + r.flags)]),
|
||||
MONEY_LINE_TOKENS, MONEY_PAIRS, REPORTED_COST_WORDS, MONEY_USD_WORDS,
|
||||
patterns: Object.entries({ MONEY_LINE, USD_COMPARE, RESULTS_TREE: [RESULTS_TREE], SECRET_LINE, SECRET_PATH: [SECRET_PATH], SECURITY_LINE, SECURITY_JOINED, DATA_LINE, SVG_PATH: [SVG_PATH], SVG_ACTIVE }).map(([k, v]) => [k, v.map(r => '/' + r.source + '/' + r.flags)]),
|
||||
})).digest('hex').slice(0, 16);
|
||||
|
||||
export const tokens = (p: string) => p.replace(/([a-z0-9])([A-Z])/g, '$1 $2').toLowerCase().split(/[^a-z0-9]+/).filter(Boolean);
|
||||
@@ -106,7 +113,7 @@ export interface RiskInput {
|
||||
removed?: Record<string, readonly string[]>;
|
||||
/** gstack-diff-scope's SCOPE_AUTH: the auth class needs it and an auth token in the path. */
|
||||
scopeAuth: boolean;
|
||||
/** Code paths no code reads (an unconsumed data file or drawing): nothing runs them, so neither their name nor their rows name money (GST-961). */
|
||||
/** Code paths no code reads (an unconsumed data file or drawing): under RESULTS_TREE their name and reported costs name no money; a spend cap or paid call in them still does (GST-961). */
|
||||
dataPaths?: readonly string[];
|
||||
}
|
||||
|
||||
@@ -118,13 +125,13 @@ export function matchRisk(i: RiskInput): Map<RiskClass, string> {
|
||||
const t = tokens(p);
|
||||
if (i.scopeAuth && t.some(x => AUTH_PATH_TOKENS.includes(x) || AUTH_PATH_PREFIXES.some(a => x.startsWith(a)))) hit('auth', 'auth path: ' + p);
|
||||
if (SECRET_PATH.test(p)) hit('secret', 'secret path: ' + p);
|
||||
const money = !i.dataPaths?.includes(p);
|
||||
const money = !(i.dataPaths?.includes(p) && RESULTS_TREE.test(p));
|
||||
if (money && namesMoney(t, MONEY_PATH_TOKENS)) hit('money', 'money path: ' + p);
|
||||
if (t.some(x => SECURITY_PATH_TOKENS.includes(x))) hit('security', 'security path: ' + p);
|
||||
if (t.some(x => DATA_PATH_TOKENS.includes(x))) hit('data', 'data path: ' + p);
|
||||
if (SVG_PATH.test(p) && [i.added?.[p], i.removed?.[p]].some(l => l && svgActive(l.join('\n')))) hit('security', 'active SVG lines: ' + p);
|
||||
for (const line of [...(i.added?.[p] ?? []), ...(i.removed?.[p] ?? [])]) {
|
||||
if (money && !why.has('money') && (PAID_MONEY_LINE.some(r => r.test(line)) || usdMoney(line) || namesMoney(tokens(line), MONEY_LINE_TOKENS))) hit('money', 'money line: ' + p);
|
||||
if (!why.has('money') && (PAID_MONEY_LINE.some(r => r.test(line)) || (money && (usdMoney(line) || namesMoney(tokens(line), MONEY_LINE_TOKENS))))) hit('money', 'money line: ' + p);
|
||||
if (!why.has('secret') && SECRET_LINE.some(r => r.test(line))) hit('secret', 'secret line: ' + p);
|
||||
if (!why.has('security') && SECURITY_LINE.some(r => r.test(line))) hit('security', 'security line: ' + p);
|
||||
if (!why.has('data') && DATA_LINE.some(r => r.test(line))) hit('data', 'data line: ' + p);
|
||||
|
||||
@@ -21,7 +21,7 @@ import { createHash } from 'node:crypto';
|
||||
import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import { classifyPath, unconsumedData, type ClassInput } from './review-depth';
|
||||
import { AUTH_PATH_PREFIXES, AUTH_PATH_TOKENS, KEY_NAME, MONEY_LINE, MONEY_LINE_TOKENS, MONEY_PAIRS, MONEY_PATH_TOKENS, MONEY_USD_WORDS, SECRET_LINE, SECRET_PATH, matchRisk } from './review-risk';
|
||||
import { AUTH_PATH_PREFIXES, AUTH_PATH_TOKENS, KEY_NAME, MONEY_LINE, MONEY_LINE_TOKENS, MONEY_PAIRS, MONEY_PATH_TOKENS, MONEY_USD_WORDS, REPORTED_COST_WORDS, RESULTS_TREE, SECRET_LINE, SECRET_PATH, USD_COMPARE, matchRisk } from './review-risk';
|
||||
|
||||
export const TARGETED_CLASSES = ['money', 'gate', 'auth', 'secret'] as const;
|
||||
export type TargetedClass = (typeof TARGETED_CLASSES)[number];
|
||||
@@ -31,8 +31,8 @@ export const secretLine = (line: string) => SECRET_LINE.some(r => r.test(line));
|
||||
|
||||
/** A hash of the patterns that decide the classes, not of this file's bytes: a comment edit keeps it, a pattern edit changes it. */
|
||||
export const TARGETED_RULES = createHash('sha256').update(JSON.stringify({
|
||||
TARGETED_CLASSES, MONEY_PATH_TOKENS, MONEY_LINE_TOKENS, MONEY_PAIRS, MONEY_USD_WORDS, AUTH_PATH_TOKENS, AUTH_PATH_PREFIXES, KEY_NAME,
|
||||
patterns: Object.entries({ MONEY_LINE, SECRET_LINE, SECRET_PATH: [SECRET_PATH] }).map(([k, v]) => [k, v.map(r => `/${r.source}/${r.flags}`)]),
|
||||
TARGETED_CLASSES, MONEY_PATH_TOKENS, MONEY_LINE_TOKENS, MONEY_PAIRS, REPORTED_COST_WORDS, MONEY_USD_WORDS, AUTH_PATH_TOKENS, AUTH_PATH_PREFIXES, KEY_NAME,
|
||||
patterns: Object.entries({ MONEY_LINE, USD_COMPARE, RESULTS_TREE: [RESULTS_TREE], SECRET_LINE, SECRET_PATH: [SECRET_PATH] }).map(([k, v]) => [k, v.map(r => `/${r.source}/${r.flags}`)]),
|
||||
})).digest('hex').slice(0, 16);
|
||||
|
||||
export interface TargetedInput {
|
||||
|
||||
@@ -74,7 +74,7 @@ const NO_LOSS_REPLAYS = {
|
||||
'GST-967': "GST-967 dispatch-superset no-loss check (test/review-depth.test.ts, 'GST-967 dispatch-superset no-loss check'): over 1,440 planReview cases the branch plans exactly what main planned at 1022f4f except the 71 gate-class stops before Step 11, which add only Step 11's adversarial, outside and targeted voices, so no reviewer main dispatches is dropped or re-efforted; the brief bodies lib/review-brief.ts cuts from the ship sections hash to main's; only the lead's repeat-decision and Step 11 instructions changed; reviewer text is unchanged",
|
||||
'GST-508+GST-464+GST-967': "wave review-53cfb5bfe011 merge of main 68410828f into #3852: ship/sections/review-army.md and adversarial.md are main's GST-967 bytes plus exactly the GST-508+GST-464 lines (the merged file minus main equals the wave minus fc292e22f, line for line), so GST-967's dispatch-superset no-loss check and GST-508's no-loss replay both hold; merged-head checks in GStack Minder drive handoffs/GSTA-202/merge-6841-checks.md",
|
||||
'GST-508+R51+GST-911+GST-967': "wave review-53cfb5bfe011 merge of main 68410828f into #3852: lib/review-depth.ts is main's GST-967 bytes plus exactly the GST-508+R51+GST-911 lines (the merged file minus main equals the wave minus fc292e22f, line for line); GST-967's dispatch-superset no-loss check (test/review-depth.test.ts) passes on the merged file, and the GST-508+R51+GST-911 checks are rerun at the merged head (GStack Minder drive handoffs/GSTA-202/merge-6841-checks.md)",
|
||||
'GST-961+962': "GST-961+962 verdict no-loss replay: a data file no code reads (the depth's consumer scan) names no money by its path or rows, and a _usd or Usd name is money only when it says cap, limit, max, budget, spend, price, fee, bill or payment or a < or > compares it; over the last 100 first-parent main commits to bf2bcf14b, main's code against this branch, gate scan fixed at 0, no commit changes depth, class, risk classes or targeted classes (~/.capy/work/gst898/replay961.ts on GSTA-249's machine); the 76 changed lines that no longer read as money lines are cost reads and reports (paid-report, eval-plan projections, autoplan estimates), tests, docs and the old pattern source; lib/review-depth.ts only exports unconsumedData; reviewer text is unchanged",
|
||||
'GST-961+962': "GST-961+962 verdict no-loss replay: a data file no code reads (the depth's consumer scan) names no money by its path or rows, under docs/, wiki/, results/ or reports/ (a spend cap or paid call there still does), and a _usd or Usd name is not money only when every such name is a reported cost on a line with no cap, budget, price, charge, reservation or estimate word, no < or > comparison and no number set; over the last 100 first-parent main commits to bf2bcf14b, main's code against this branch, gate scan fixed at 0, no commit changes depth, class, risk classes or targeted classes (~/.capy/work/gst898/replay961.ts on GSTA-249's machine); the 21 changed lines that no longer read as money lines are cost sums and reports (scripts/lib/paid-report.ts, two eval-plan per-item means), tests and docs; lib/review-depth.ts only exports unconsumedData; reviewer text is unchanged",
|
||||
'GST-940': "GST-940 verdict no-loss check: svgDrawing reads a style element's opening tag as quoted pairs with ASCII whitespace, walks its body over the raw text, refuses any & in it, reads comments as CSS and refuses a non-ASCII element or attribute name; on 62,805 unique SVG files (21,658 with a style element) it gives the same verdict as main at 4bbad2422 on every file, and the planner over the last 100 first-parent main commits to 4bbad2422 changes no commit's depth, class or risk classes; the only verdicts it changes are the GST-940 cases in test/review-planner-misreads.test.ts; rerun on the merge with main at 8f08cb76a (GST-967 merged), with the same results; rerun on the merge with main at 5f705f367 (#3852 and #3864 merged), with the same results",
|
||||
} as const;
|
||||
type Replay = keyof typeof NO_LOSS_REPLAYS;
|
||||
@@ -98,7 +98,7 @@ const REVIEWER_BRIEFS: Record<string, { sha: string; replay: Replay }> = {
|
||||
'review/specialists/targeted.md': { sha: '7f92a0d0a8319c3c0e076fb3b1a4689c45120b2be2c14dfc7d4cb03aafa9d6a5', replay: 'GST-257' },
|
||||
'scripts/resolvers/fix-check.ts': { sha: '712bf5b0c439e96bac32f19cbe25394dabb4c94aa66600100bfde9b100e15a08', replay: '#3638' },
|
||||
'lib/review-depth.ts': { sha: '894ef79782232ae8e2b13a25d57d87264125ff6ae5aeb70813c4b11c8cd59a4d', replay: 'GST-961+962' },
|
||||
'lib/review-risk.ts': { sha: 'ac023de9073705a31f864bc9dc9a58262c325185375dff9049c917fbe8d2be76', replay: 'GST-961+962' },
|
||||
'lib/review-risk.ts': { sha: 'aafbf5dfa3c4cdcb3d047178d5cc485216228aff50a3ba98528d4bea67c6bb8b', replay: 'GST-961+962' },
|
||||
};
|
||||
|
||||
/**
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
*/
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
import { SVG_REFUSED, svgDrawing } from '../lib/review-depth';
|
||||
import { DATA_LINE, MONEY_LINE, SECRET_LINE, SECRET_PATH, SECURITY_JOINED, SECURITY_LINE, matchRisk, usdMoney } from '../lib/review-risk';
|
||||
import { DATA_LINE, MONEY_LINE, SECRET_LINE, SECRET_PATH, SECURITY_JOINED, SECURITY_LINE, USD_COMPARE, matchRisk, usdMoney } from '../lib/review-risk';
|
||||
|
||||
const L = 100_000;
|
||||
const ms = (f: () => unknown) => { const t = performance.now(); f(); return performance.now() - t; };
|
||||
@@ -27,7 +27,7 @@ describe('linear time on hostile input', () => {
|
||||
run('', 'a_'), run('', 'aU'), run('a', 'A'), run('spend', '_'), run('new Anthropic', ' '), run('.messages.create', ' '),
|
||||
run('$', 'A_'), run('${', 'A_'), run('', '$A_'), run('process.env.', 'A_'), run('process.env[', ' '), run('os.environ[', ' '), run('os.getenv(', ' '), run('ENV[', ' '),
|
||||
];
|
||||
test.each(Object.entries({ SECURITY_LINE, SECURITY_JOINED, DATA_LINE, SECRET_LINE, MONEY_LINE, SECRET_PATH: [SECRET_PATH] }))('every %s pattern finishes under 100 ms on each 100k-character hostile line', (_n, patterns) => {
|
||||
test.each(Object.entries({ SECURITY_LINE, SECURITY_JOINED, DATA_LINE, SECRET_LINE, MONEY_LINE, USD_COMPARE, SECRET_PATH: [SECRET_PATH] }))('every %s pattern finishes under 100 ms on each 100k-character hostile line', (_n, patterns) => {
|
||||
const slow = patterns.flatMap(r => hostile.map(([label, s]) => [r.source.slice(0, 50), label, Math.round(ms(() => r.test(s)))] as const)).filter(([, , t]) => t >= 100);
|
||||
expect(slow).toEqual([]);
|
||||
}, 120_000);
|
||||
@@ -38,7 +38,7 @@ describe('linear time on hostile input', () => {
|
||||
expect(hit).toBe(false);
|
||||
}, 60_000);
|
||||
test('usdMoney (GST-962) returns in under 250 ms on each 100k-character line of _usd names, comparisons and member chains', () => {
|
||||
const lines = [run('', 'x_usd '), run('', 'aUsd '), run('', '> a.x_usd '), run('', '=> t.cost_usd '), run('>', ' '), run('> ', 'a.', 'x_usd'), run('', 'a_', 'usd'), run('', '<'), run('', 'cost_usd>')];
|
||||
const lines = [run('', 'x_usd '), run('', 'aUsd '), run('', '> a.x_usd '), run('', '=> t.cost_usd '), run('>', ' '), run('> ', 'a.', 'x_usd'), run('', 'a_', 'usd'), run('', '<'), run('', 'cost_usd>'), run('', 'cost_usd: '), run('x_usd', ' '), run('x_usd:', ' ')];
|
||||
const slow = lines.map(([label, s]) => [label, Math.round(ms(() => usdMoney(s)))] as const).filter(([, t]) => t >= 250);
|
||||
expect(slow).toEqual([]);
|
||||
}, 60_000);
|
||||
|
||||
@@ -437,6 +437,9 @@ describe('incidental money signals (GST-961, GST-962)', () => {
|
||||
expect(classesOf(input(['wiki/credentials.json'], { consumed: [] }))).toEqual(['secret']);
|
||||
expect(classesOf(input(['wiki/keys.json'], { consumed: [], added: { 'wiki/keys.json': ['{"k": "$OPENAI_API_KEY"}'] } }))).toEqual(['secret']);
|
||||
expect(classesOf(input(['src/billing/invoice.ts'], { consumed: [] }))).toEqual(['money']);
|
||||
expect(classesOf(input(['prices/us-east.json'], { consumed: [] }))).toEqual(['money']);
|
||||
expect(classesOf(input(['data/rates.json'], { consumed: [], added: { 'data/rates.json': ['{"input_price_usd": 3}'] } }))).toEqual(['money']);
|
||||
expect(classesOf(input([tsv], { consumed: [], added: { [tsv]: ['spend_cap\t500'] } }))).toEqual(['money']);
|
||||
});
|
||||
|
||||
test('a cost_usd field read or reported names no money; a cap, budget, limit, spend or comparison still does', () => {
|
||||
@@ -460,6 +463,23 @@ describe('incidental money signals (GST-961, GST-962)', () => {
|
||||
' if (spent > capUsd) throw new Error("over cap");',
|
||||
'const client = new Anthropic();',
|
||||
'await meter.charge(user, amount);',
|
||||
'if (limit < run?.cost_usd) stop();',
|
||||
'if ((run.cost_usd ?? 0) > limit) stop();',
|
||||
'if (limit <= (row.cost_usd ?? 0)) stop();',
|
||||
'const capped = Math.min(cost_usd, cap);',
|
||||
'if (Number(row.cost_usd) > budget) stop();',
|
||||
'const intent = await stripe.paymentIntents.create({ amount: amount_usd * 100 });',
|
||||
'await refund(user, refund_usd);',
|
||||
'const cost_usd = input_tokens * PRICE_PER_MTOK;',
|
||||
'const check = { attempt, estimate_usd: 0 };',
|
||||
'rows.push({ arm, cost_usd: 0 });',
|
||||
' return { ...r, reservation_usd: estimateCostUsd({ input: Math.ceil(r.in_tokens / r.calls), output: r.max_out_tokens }, r.model) };',
|
||||
' const maxEstimate = Math.max(0, ...record.attempts.map(a => a.estimate_usd));',
|
||||
' const worst = summary.worst_case_usd + workers * maxEstimate;',
|
||||
' reserve(file, attempt, a.estimate_usd, { label });',
|
||||
' a.estimate_usd = usd;',
|
||||
' ship_measure_ask_per_trial_usd) echo "200" ;;',
|
||||
" trial_reservation_usd: v => (typeof v === 'number' && v > 0 && v <= 100000 ? null : 'x'),",
|
||||
]) expect(at(line), line).toEqual(['money']);
|
||||
});
|
||||
|
||||
|
||||
Reference in new issue
Block a user