feat(redact): semantic-pass eval + CLAUDE.md docs + size/parity baselines

- test/redact-semantic-pass.eval.ts: periodic-tier paid eval (EVALS=1) with 10
  should-flag / should-clean fixtures + an injection-resistance case, the only
  way to detect semantic-pass model drift.
- CLAUDE.md: "Redaction guard" section — engine/CLI/hook locations, the
  guardrail-not-enforcement framing, scan-at-sink, no-tier-promotion, the
  tool-attributed-fence convention, the config keys, and the audit log.
- /cso uses the compact (HIGH-tier) taxonomy table so it fits under BOTH the
  v1.47 and the older v1.44.1 parity ceilings; full MEDIUM/LOW lives in
  lib/redact-patterns.ts. Alignment test asserts the HIGH-tier contract.
- Refresh the ship golden baselines (claude/codex/factory) for the PR-body
  redaction wiring.

Full free suite green (incl. skill-size-budget + parity 10/10).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-05-29 07:40:12 -07:00
parent dd4dd9e1f5
commit 02f848dde4
8 changed files with 231 additions and 52 deletions
+3 -29
View File
@@ -884,8 +884,8 @@ INFRASTRUCTURE SURFACE
Scan git history for leaked credentials, check tracked `.env` files, find CI configs with inline secrets.
**Canonical pattern catalog** (shared with `/spec`'s in-flight redaction, generated
from `lib/redact-patterns.ts` — the archaeology greps below target the HIGH-tier
prefixes from this table):
from `lib/redact-patterns.ts` — the archaeology greps below target these HIGH-tier
prefixes; full MEDIUM/LOW taxonomy is in `lib/redact-patterns.ts`):
**HIGH — genuinely-secret credentials. Blocks dispatch/file/edit/commit.**
@@ -909,33 +909,7 @@ prefixes from this table):
| `db.url_with_password` | Database URL with embedded password | postgres://user:pw@host |
| `creds.basic_auth_url` | HTTP(S) URL with embedded basic-auth credentials | https://user:pw@host |
**MEDIUM PII, legal/damaging, internal-leak, and high-FP credential-shaped patterns. AskUserQuestion to confirm (sterner on public repos); never auto-blocked.**
| ID | Catches | Example |
|----|---------|---------|
| `stripe.publishable` | Stripe live publishable key (often intentionally public) | pk_live_… |
| `google.api_key` | Google API key (AIza…; sometimes a public client key) | AIza… |
| `jwt` | JSON Web Token (3-segment base64url) | eyJ….eyJ….sig |
| `env.kv` | Env-style SECRET assignment with high-entropy value | FOO_SECRET=<high-entropy> |
| `pii.email` | Email address | name@host.tld |
| `pii.phone.e164` | Phone number (E.164 / common national formats; US/EU-biased) | +1 415 555 0123 |
| `pii.ssn` | US Social Security Number | 123-45-6789 |
| `pii.cc` | Credit-card number (Luhn-valid) | Luhn-valid 13-19 digits |
| `pii.ip_public` | Public IPv4 address | public IPv4 |
| `pii.wallet` | Crypto wallet address (ETH/BTC) | 0x… / bc1… / 1… |
| `internal.hostname` | Internal hostname (*.internal/.corp/.local/.prod/.staging) | host.corp / host.internal |
| `internal.url_private` | localhost URL with a non-trivial path | http://localhost:PORT/path |
| `legal.nda_marker` | Confidentiality / NDA marker | CONFIDENTIAL / UNDER NDA |
| `legal.named_criticism` | Negative judgment near a capitalized full name (semantic pass is primary) | negative judgment + a full name |
**LOW — surfaced as an FYI, never blocks.**
| ID | Catches | Example |
|----|---------|---------|
| `internal.user_path` | Absolute path under a user home dir | /Users/<name>/… , /home/<name>/… |
| `hygiene.todo` | TODO(owner) marker carried into the artifact | TODO(owner) |
Calibration: a gate that cries wolf gets ignored, so context-variable / high-FP credential shapes (Stripe publishable `pk_live_`, Google `AIza`, JWTs, env-style `*_KEY=`) sit at MEDIUM, not HIGH. The full taxonomy lives in `lib/redact-patterns.ts` and this table is generated from it.
MEDIUM (PII / legal / internal + high-FP credential shapes like `pk_live_`/`AIza`/JWT/`*_KEY=`) confirms via AskUserQuestion; LOW surfaces as an FYI. Full taxonomy: `lib/redact-patterns.ts` (or `/cso`).
**Git history — known secret prefixes:**
```bash
+3 -3
View File
@@ -160,10 +160,10 @@ INFRASTRUCTURE SURFACE
Scan git history for leaked credentials, check tracked `.env` files, find CI configs with inline secrets.
**Canonical pattern catalog** (shared with `/spec`'s in-flight redaction, generated
from `lib/redact-patterns.ts` — the archaeology greps below target the HIGH-tier
prefixes from this table):
from `lib/redact-patterns.ts` — the archaeology greps below target these HIGH-tier
prefixes; full MEDIUM/LOW taxonomy is in `lib/redact-patterns.ts`):
{{REDACT_TAXONOMY_TABLE}}
{{REDACT_TAXONOMY_TABLE:compact}}
**Git history — known secret prefixes:**
```bash