fix: address Codex adversarial findings in /cso v2

Six fixes from Codex adversarial review:

1. Phase 2: Use `git log -G` (regex) instead of `-S` (literal) for
   patterns with alternation (ghp_|gho_|github_pat_, etc.)

2. Phase 12 exclusion #5: Add exception so CI/CD pipeline findings
   from Phase 4 are never auto-discarded when --infra is active

3. Phase 12 exclusion #6: Add exception that unpinned actions and
   missing CODEOWNERS are concrete risks, not "missing hardening"

4. Phase 12 exclusion #15: Add exception that SKILL.md files are
   executable prompt code, not documentation — Phase 8 findings
   in SKILL.md must not be excluded

5. Phase 12 exclusion #1: Add exception that LLM cost/spend
   amplification from Phase 7 is financial risk, not DoS

6. E2E tests: Add exitReason === 'success' assertion to all 3 tests;
   move finalizeEvalCollector to file-level afterAll

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-03-23 06:52:00 -07:00
parent 0d25f275c9
commit bcded824f0
3 changed files with 21 additions and 15 deletions
+7 -1
View File
@@ -12,6 +12,10 @@ import * as os from 'os';
const evalCollector = createEvalCollector('e2e-cso');
afterAll(() => {
finalizeEvalCollector(evalCollector);
});
// --- CSO v2 E2E Tests ---
describeIfSelected('CSO v2 — full audit', ['cso-full-audit'], () => {
@@ -55,7 +59,6 @@ app.listen(3000);
afterAll(() => {
try { fs.rmSync(csoDir, { recursive: true, force: true }); } catch {}
finalizeEvalCollector(evalCollector);
});
test('/cso finds planted vulnerabilities', async () => {
@@ -76,6 +79,7 @@ IMPORTANT:
});
logCost('cso', result);
expect(result.exitReason).toBe('success');
// Should detect hardcoded API key
const output = result.output.toLowerCase();
@@ -163,6 +167,7 @@ IMPORTANT:
});
logCost('cso', result);
expect(result.exitReason).toBe('success');
const output = result.output.toLowerCase();
// Should mention webhook and missing signature verification
@@ -239,6 +244,7 @@ IMPORTANT:
});
logCost('cso', result);
expect(result.exitReason).toBe('success');
const output = result.output.toLowerCase();
// Should mention unpinned action or Dockerfile issues