feat: hermetic Codex runner hardening + Sol scope-termination E2E

The Codex E2E runner copies auth.json only (operator plugins, MCP servers,
rules, and skills no longer leak into hermetic evals), pins CODEX_HOME to
the temp dir, and supports per-run model, TOML overrides, and
--ignore-user-config. New periodic E2E installs the FULL generated
investigate skill on gpt-5.6-sol against a planted one-line bug with decoy
TODOs: the fix must land inside the boundary (untracked files counted via
git status --porcelain), decoys stay byte-identical, the regression oracle
survives unweakened, nothing gets committed, all within 30 tool calls.
The shared .agents tree is snapshotted and restored exactly in beforeAll;
fixture commits disable gpg signing. Wired into the periodic CI matrix,
paid-shard globs, eval scripts, touchfiles/E2E_TIERS
(codex-sol-scope-termination), and diff-based selection. Real-file
periodic-tier classification pins both codex E2Es out of the gate tier.
Free-tier test proves an explicit --model overrides the host default
through the real generation CLI.
This commit is contained in:
Garry Tan
2026-08-18 13:11:15 -07:00
parent 39a60588ed
commit 8229fdc489
10 changed files with 346 additions and 22 deletions
+2
View File
@@ -100,6 +100,8 @@ jobs:
file: test/skill-routing-e2e.test.ts
- name: e2e-codex
file: test/codex-e2e.test.ts
- name: e2e-codex-sol-scope
file: test/codex-e2e-sol-scope.test.ts
- name: e2e-gemini
file: test/gemini-e2e.test.ts
steps: