v1.67.2.0 feat: gpt-5.6-sol bounded-scope profile for Codex installs (#2633)

* feat: model taxonomy gains gpt-5.6-sol + per-host generation defaults

Adds 'gpt-5.6-sol' to the model taxonomy with exact-match-only resolution
(Terra/Luna/suffixed IDs deliberately fall back to generic gpt) and replaces
the hardcoded 'claude' generation default with a validated
HostConfig.defaultModel: codex renders the gpt profile when --model is
absent, every other host keeps claude. Codex ship golden regenerated
accordingly; ADDING_A_HOST documents the new field.

* feat: gpt-5.6-sol bounded-scope overlay + scope-aware resolvers

The Sol profile pins the explicit task as the lake: adjacent work is
report-only, investigation is bounded, runs terminate on one clean
verification pass, and the AskUserQuestion decision-brief format is never
trimmed. The overlay wrapper grants scope-interpretation precedence while
concrete workflow steps, gates, and skill-mandated re-verification loops
still win. Sol-specific Completeness Principle and first-run intro copy.
New SETUP_COMMAND resolver renders './setup --host <host>' for every
non-claude host so generated upgrade skills reinstall their own host.

* feat: setup reads the Codex model from config.toml

New resolve-codex-generation-model.ts reads the top-level model from
${CODEX_HOME:-~/.codex}/config.toml, validates against the model allowlist,
strips control characters from every config-derived string it surfaces,
guards against non-absolute config locations, and warns on Sol near-misses.
setup runs it on EVERY invocation (read-only TOML lookup) so a plain
./setup can never clobber a Sol user's rendered profile with the hardcoded
fallback; --model <id> overrides for one run and prints the persistence
hint. Kiro installs render the claude profile before copying (Kiro fronts
Claude-family models), rewrite the baked setup command to --host kiro, and
restore the resolved Codex profile after; the codex skills path honors
CODEX_HOME. Static pins cover the resolver wiring, fail-closed exit,
quoted argv, and the Kiro sandwich.

* feat: hermetic Codex runner hardening + Sol scope-termination E2E

The Codex E2E runner copies auth.json only (operator plugins, MCP servers,
rules, and skills no longer leak into hermetic evals), pins CODEX_HOME to
the temp dir, and supports per-run model, TOML overrides, and
--ignore-user-config. New periodic E2E installs the FULL generated
investigate skill on gpt-5.6-sol against a planted one-line bug with decoy
TODOs: the fix must land inside the boundary (untracked files counted via
git status --porcelain), decoys stay byte-identical, the regression oracle
survives unweakened, nothing gets committed, all within 30 tool calls.
The shared .agents tree is snapshotted and restored exactly in beforeAll;
fixture commits disable gpg signing. Wired into the periodic CI matrix,
paid-shard globs, eval scripts, touchfiles/E2E_TIERS
(codex-sol-scope-termination), and diff-based selection. Real-file
periodic-tier classification pins both codex E2Es out of the gate tier.
Free-tier test proves an explicit --model overrides the host default
through the real generation CLI.

* chore: bump version and changelog (v1.67.2.0)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: post-ship documentation sync for v1.67.2.0

- README: Codex skills path is CODEX_HOME-aware; state that
  --model overrides detection for one run only (persist via
  the Codex config.toml model key)
- CONTRIBUTING: add the model-overlay axis to the per-host
  config table (per-host defaultModel, override precedence)
- CLAUDE.md: eval results dir is ~/.gstack/projects/<slug>/evals/
  (legacy fallback ~/.gstack-dev/evals/), matching eval-store.ts
  and the eval:* CLI headers

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: post-ship documentation sync (v1.67.2.0)

Sol exact-match and near-miss warning documented in README; CODEX_HOME-aware
uninstall and troubleshooting paths; hermetic auth.json-only detail and the
build-clobber gotcha in CLAUDE.md; eval-store location corrected in
ARCHITECTURE.md; defaultModel row in the ADDING_A_HOST field reference;
resolver test count corrected in the CHANGELOG entry.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-18 17:00:04 -07:00
committed by GitHub
co-authored by Claude Fable 5
parent c86e6472eb
commit 60e51342b5
38 changed files with 1194 additions and 72 deletions
+17 -1
View File
@@ -10,6 +10,10 @@
*/
import { describe, test, expect } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
const ROOT = path.resolve(import.meta.dir, '..');
import {
PAID_TEST_GLOBS,
classifyPaidTestFile,
@@ -32,6 +36,7 @@ describe('paid test enumeration', () => {
expect(isPaidTestFile('test/skill-e2e-qa-workflow.test.ts')).toBe(true);
expect(isPaidTestFile('test/skill-llm-eval.test.ts')).toBe(true);
expect(isPaidTestFile('test/codex-e2e.test.ts')).toBe(true);
expect(isPaidTestFile('test/codex-e2e-sol-scope.test.ts')).toBe(true);
expect(isPaidTestFile('test/skill-e2e-triage-audit.test.ts')).toBe(true);
// Outside the globs: no dash, extra suffix, or a free test.
// 'test/skill-e2e.test.ts' is the DELETED pre-split monolith's name,
@@ -46,7 +51,7 @@ describe('paid test enumeration', () => {
const files = collectPaidTestFiles();
expect(files.length).toBeGreaterThan(0);
expect(files.every(isPaidTestFile)).toBe(true);
expect(PAID_TEST_GLOBS.length).toBe(5);
expect(PAID_TEST_GLOBS.length).toBe(6);
const shards = planPaidShards(files);
expect(shards.flat().sort()).toEqual([...files].sort());
@@ -87,6 +92,17 @@ describe('tier classification', () => {
expect(classifyPaidTestFile(noGuard, 'periodic').included).toBe(true);
expect(classifyPaidTestFile('', 'gate').included).toBe(true);
});
test('the REAL external-CLI test files classify as periodic-only', () => {
// Synthetic guard shapes above can drift from the actual files — the
// inert-demotion defect class. Pin the real sources: a guard-shape edit
// in either file that silently runs it in gate fails here.
for (const file of ['test/codex-e2e.test.ts', 'test/codex-e2e-sol-scope.test.ts']) {
const source = fs.readFileSync(path.join(ROOT, file), 'utf8');
expect(classifyPaidTestFile(source, 'gate').included, `${file} leaked into gate tier`).toBe(false);
expect(classifyPaidTestFile(source, 'periodic').included, `${file} dropped from periodic tier`).toBe(true);
}
});
});
describe('shard execution', () => {