Garry TanandClaude Fable 5 bd11416d80 fix(evals): absorb codex/gemini CLI drift; external-service tests go periodic-tier ...

- codex exec gains --skip-git-repo-check: newer CLIs refuse exec in an
  untrusted non-git dir (our temp skill dirs) — empirically verified.
- gemini: --skip-trust was removed in gemini-cli 0.34 (argv parse error);
  dropped from the session runner and the benchmark adapter. A present-
  but-unusable CLI (deprecated individual code-assist auth path) now
  classifies as SKIP, not a false adapter failure; the benchmark live
  smoke skips on auth/rate_limit error codes (environmental) while still
  failing on timeout/unknown (the drift classes it exists to catch).
- codex-e2e, gemini-e2e, and benchmark-providers gain the canonical
  whole-file EVALS_TIER === 'periodic' guard per CLAUDE.md tiering rule 3
  (external service -> periodic) — the sharded gate runner now excludes
  all three (gate: 45 -> 42 shards).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-08-13 10:34:25 -07:00

..

providers

fix(evals): absorb codex/gemini CLI drift; external-service tests go periodic-tier

2026-08-13 10:34:25 -07:00

agent-sdk-runner.ts

v1.58.1.0 feat: hermetic local E2E + Conductor prose AskUserQuestion (#2004)

2026-06-14 11:40:57 -07:00

auq-sdk-capture.ts

v1.56.0.0 Token-reduction Phase B + AUQ paranoid safety net (#1849)

2026-06-04 11:14:43 -07:00

benchmark-judge.ts

feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040)

2026-04-19 17:50:31 +08:00

benchmark-runner.ts

feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040)

2026-04-19 17:50:31 +08:00

budget-override.test.ts

v1.46.0.0 feat: gstack v2 foundation — catalog tokens drop 56%, eval-first floor covers all 51 skills (#1712)

2026-05-26 16:50:03 -07:00

budget-override.ts

v1.46.0.0 feat: gstack v2 foundation — catalog tokens drop 56%, eval-first floor covers all 51 skills (#1712)

2026-05-26 16:50:03 -07:00

capture-parity-baseline.test.ts

v1.46.0.0 feat: gstack v2 foundation — catalog tokens drop 56%, eval-first floor covers all 51 skills (#1712)

2026-05-26 16:50:03 -07:00

capture-parity-baseline.ts

v1.46.0.0 feat: gstack v2 foundation — catalog tokens drop 56%, eval-first floor covers all 51 skills (#1712)

2026-05-26 16:50:03 -07:00

carve-guard-checks.ts

v1.57.0.0 feat: carve-guard system + carve cso/document-release/design-consultation (#1907)

2026-06-07 19:13:24 -07:00

carve-guards.ts

v1.62.0.0 feat: plan-mode auto-select at the review scope gate (#2533)

2026-08-12 11:12:28 -07:00

claude-pty-runner.scope-gate-floor.unit.test.ts

v1.62.0.0 feat: plan-mode auto-select at the review scope gate (#2533)

2026-08-12 11:12:28 -07:00

claude-pty-runner.ts

feat(evals): seedSkills opt-in for PTY slash-command tests + tripwire

2026-08-12 15:31:47 -07:00

claude-pty-runner.unit.test.ts

v1.62.0.0 feat: plan-mode auto-select at the review scope gate (#2533)

2026-08-12 11:12:28 -07:00

codex-session-runner.ts

fix(evals): absorb codex/gemini CLI drift; external-service tests go periodic-tier

2026-08-13 10:34:25 -07:00

e2e-helpers.ts

v1.39.2.0 feat: GSTACK_* env-shim for Conductor + gbrain/gstack setup docs (#1534)

2026-05-16 12:32:33 -07:00

eval-store.test.ts

feat(evals): env-driven lazy eval dir + shard-aware store and tooling

2026-08-12 15:31:47 -07:00

eval-store.ts

feat(evals): env-driven lazy eval dir + shard-aware store and tooling

2026-08-12 15:31:47 -07:00

gemini-session-runner.test.ts

feat: Gemini CLI E2E tests (v0.9.2.0) (#252)

2026-03-20 08:30:09 -07:00

gemini-session-runner.ts

fix(evals): absorb codex/gemini CLI drift; external-service tests go periodic-tier

2026-08-13 10:34:25 -07:00

hermetic-env.test.ts

v1.58.1.0 feat: hermetic local E2E + Conductor prose AskUserQuestion (#2004)

2026-06-14 11:40:57 -07:00

hermetic-env.ts

feat(evals): register shipped skills for hermetic PTY children

2026-08-12 15:31:47 -07:00

llm-judge.ts

v1.25.1.0 fix: office-hours Phase 4 STOP gate + AskUserQuestion recommendation judge (#1296)

2026-05-01 19:51:51 -07:00

observability.test.ts

v1.60.2.0 fix: free-suite drift on dev machines (eval-list cwd, gemini regex, observability floor) (#2470)

2026-08-07 15:08:12 -07:00

paid-test-set.ts

refactor(evals): single shared paid-test-set module

2026-08-12 15:31:47 -07:00

parity-harness.ts

v1.62.0.0 feat: plan-mode auto-select at the review scope gate (#2533)

2026-08-12 11:12:28 -07:00

pricing.ts

feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040)

2026-04-19 17:50:31 +08:00

required-reads.ts

v1.54.0.0 feat: carve /ship into skeleton + on-demand sections (-59% always-loaded) (#1806)

2026-05-30 12:09:10 -07:00

secret-sink-harness.ts

v1.12.0.0 feat: /setup-gbrain — coding-agent onboarding for gbrain (#1183)

2026-04-24 01:38:21 -07:00

session-runner.test.ts

feat: stream-json NDJSON parser for real-time E2E progress

2026-03-14 03:49:36 -05:00

session-runner.ts

fix(test-harness): spawn claude without shell interpolation

2026-07-14 12:56:07 -07:00

skill-census.ts

test(helpers): shared skill-census helper with three explicit counts

2026-08-12 15:31:46 -07:00

skill-parser.ts

feat: content security — 4-layer prompt injection defense for pair-agent (#815)

2026-04-06 14:41:06 -07:00

tool-map.ts

feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040)

2026-04-19 17:50:31 +08:00

touchfiles.ts

v1.62.0.0 feat: plan-mode auto-select at the review scope gate (#2533)

2026-08-12 11:12:28 -07:00

transcript-section-logger.ts

v1.54.0.0 feat: carve /ship into skeleton + on-demand sections (-59% always-loaded) (#1806)

2026-05-30 12:09:10 -07:00