Commit Graph

5 Commits

Author SHA1 Message Date
Garry Tan bcded824f0 fix: address Codex adversarial findings in /cso v2
Six fixes from Codex adversarial review:

1. Phase 2: Use `git log -G` (regex) instead of `-S` (literal) for
   patterns with alternation (ghp_|gho_|github_pat_, etc.)

2. Phase 12 exclusion #5: Add exception so CI/CD pipeline findings
   from Phase 4 are never auto-discarded when --infra is active

3. Phase 12 exclusion #6: Add exception that unpinned actions and
   missing CODEOWNERS are concrete risks, not "missing hardening"

4. Phase 12 exclusion #15: Add exception that SKILL.md files are
   executable prompt code, not documentation — Phase 8 findings
   in SKILL.md must not be excluded

5. Phase 12 exclusion #1: Add exception that LLM cost/spend
   amplification from Phase 7 is financial risk, not DoS

6. E2E tests: Add exitReason === 'success' assertion to all 3 tests;
   move finalizeEvalCollector to file-level afterAll

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 06:52:00 -07:00
Garry Tan 9fe95bef0c fix: /cso infra E2E test — sharper prompt to prevent exploration waste
The agent was burning 30+ turns exploring a 3-file repo (18 Glob calls,
Explore subagent, 4 SKILL.md reads) before starting the audit. Two Agent
verification subagents then ate ~100s, causing the 240s timeout.

Fix: tell the agent the repo is tiny, list the exact files, skip the
preamble, remove Agent from allowed tools, reduce maxTurns 60→30.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 06:19:06 -07:00
Garry Tan 227504161f fix: /cso infra E2E test — increase timeout to 360s
The infra scope test runs Agent sub-tasks for parallel finding
verification which can take longer than 240s. Increased maxTurns
from 25 to 60 and timeout from 240s to 360s.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 23:35:21 -07:00
Garry Tan d046b13fea fix: /cso E2E tests — correct logCost and recordE2E signatures
logCost requires (label, result), recordE2E requires (collector, name,
suite, result). Fixed all 3 test cases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 23:25:40 -07:00
Garry Tan b856752bfd test: /cso v2 E2E tests — full audit, diff mode, infra scope
Three E2E test cases with planted vulnerabilities:
- cso-full-audit: hardcoded API key + .env tracked by git
- cso-diff-mode: webhook without signature verification on feature branch
- cso-infra-scope: unpinned GitHub Action + Dockerfile without USER

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 23:15:16 -07:00