mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-10 06:58:59 +02:00
feat: hermetic Codex runner hardening + Sol scope-termination E2E
The Codex E2E runner copies auth.json only (operator plugins, MCP servers, rules, and skills no longer leak into hermetic evals), pins CODEX_HOME to the temp dir, and supports per-run model, TOML overrides, and --ignore-user-config. New periodic E2E installs the FULL generated investigate skill on gpt-5.6-sol against a planted one-line bug with decoy TODOs: the fix must land inside the boundary (untracked files counted via git status --porcelain), decoys stay byte-identical, the regression oracle survives unweakened, nothing gets committed, all within 30 tool calls. The shared .agents tree is snapshotted and restored exactly in beforeAll; fixture commits disable gpg signing. Wired into the periodic CI matrix, paid-shard globs, eval scripts, touchfiles/E2E_TIERS (codex-sol-scope-termination), and diff-based selection. Real-file periodic-tier classification pins both codex E2Es out of the gate tier. Free-tier test proves an explicit --model overrides the host default through the real generation CLI.
This commit is contained in:
@@ -100,6 +100,8 @@ jobs:
|
||||
file: test/skill-routing-e2e.test.ts
|
||||
- name: e2e-codex
|
||||
file: test/codex-e2e.test.ts
|
||||
- name: e2e-codex-sol-scope
|
||||
file: test/codex-e2e-sol-scope.test.ts
|
||||
- name: e2e-gemini
|
||||
file: test/gemini-e2e.test.ts
|
||||
steps:
|
||||
|
||||
Reference in New Issue
Block a user