mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-20 11:52:20 +02:00
feat: hermetic Codex runner hardening + Sol scope-termination E2E
The Codex E2E runner copies auth.json only (operator plugins, MCP servers, rules, and skills no longer leak into hermetic evals), pins CODEX_HOME to the temp dir, and supports per-run model, TOML overrides, and --ignore-user-config. New periodic E2E installs the FULL generated investigate skill on gpt-5.6-sol against a planted one-line bug with decoy TODOs: the fix must land inside the boundary (untracked files counted via git status --porcelain), decoys stay byte-identical, the regression oracle survives unweakened, nothing gets committed, all within 30 tool calls. The shared .agents tree is snapshotted and restored exactly in beforeAll; fixture commits disable gpg signing. Wired into the periodic CI matrix, paid-shard globs, eval scripts, touchfiles/E2E_TIERS (codex-sol-scope-termination), and diff-based selection. Real-file periodic-tier classification pins both codex E2Es out of the gate tier. Free-tier test proves an explicit --model overrides the host default through the real generation CLI.
This commit is contained in:
@@ -15,6 +15,7 @@ export const PAID_TEST_GLOBS = [
|
||||
'test/skill-e2e-*.test.ts',
|
||||
'test/skill-routing-e2e.test.ts',
|
||||
'test/codex-e2e.test.ts',
|
||||
'test/codex-e2e-sol-scope.test.ts',
|
||||
'test/gemini-e2e.test.ts',
|
||||
] as const;
|
||||
|
||||
|
||||
Reference in New Issue
Block a user