garrytan e758fbe96d test(evals): record a pre-turn API or CLI failure as infra ...

recordE2E sets failure_class 'infra' on a failed session whose runner
reports error_api, timeout_startup, error_output_stream or a non-zero CLI
exit with zero turns and no assistant event. A model refusal, a timeout
after model work, max turns, or an explicit caller pass/class keeps its
ordinary classification.

2026-09-29 19:51:35 +00:00

..

providers

test: clean up the paid eval lane (B1-B4, B6, B7)

2026-09-29 06:08:49 +00:00

agent-sdk-runner.ts

v1.90.2.0 perf: halve local free-suite time and preserve coverage (#2972)

2026-09-25 13:27:07 -04:00

anthropic-preflight.ts

v1.66.0.0 feat: test/evals/CI speedup — 90s truthful free suite, diff-billed evals, required Linux lane (#2593)

2026-08-15 22:20:30 -07:00

arm-benchmark-harness.ts

v1.75.0.0 feat: ponytail import wave — simplification review lens, arm benchmark, reuse ladder, instruction-tier digest (#2722)

2026-08-29 10:10:35 -07:00

aside-available.ts

v1.81.0.0 feat: Aside is the browser gstack drives first; every browsing skill, the PDF/diagram renderer, and web research; the bundled browser stays the automatic fallback (#2810)

2026-09-06 08:54:25 -07:00

auq-native-capture.ts

v1.89.0.0 feat: add shared-code extraction audit (#2925)

2026-09-24 01:53:58 -04:00

auq-parallel-worker.ts

v1.90.2.0 perf: halve local free-suite time and preserve coverage (#2972)

2026-09-25 13:27:07 -04:00

auq-sdk-capture.ts

test: delete tests of dead eval code (A)

2026-09-29 05:09:36 +00:00

auto-decision-state.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

autoplan-artifact-digest.ts

test: delete tests of dead eval code (A)

2026-09-29 05:09:36 +00:00

autoplan-artifact-permission.ts

test: retire the finding-count cluster and trim its helpers (C)

2026-09-29 06:08:49 +00:00

autoplan-artifact-recorder.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

autoplan-dual-voice-evidence.ts

test: one finite PR world for the shared-libs fixture; name dual-voice probe evidence

2026-09-29 14:28:11 +00:00

autoplan-method-read-audit.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

autoplan-phase-observer.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

autoplan-preconfigured-fixture.ts

v1.86.0.0 feat: route outside reviews by harness (#2850)

2026-09-14 14:32:45 -07:00

bash-script.ts

v1.80.0.0 fix: setup survives a failed Chromium install, hooks share one state root, gstack never clobbers a skill it did not create (#2802)

2026-09-05 14:45:28 -07:00

benchmark-judge.ts

v1.84.1.0 fix: default Codex and Claude to frontier models (#2835)

2026-09-09 08:57:21 -07:00

benchmark-runner.ts

feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040)

2026-04-19 17:50:31 +08:00

bootstrap-retention.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

budget-override.test.ts

v1.74.0.0 test/CI overhaul: green means green, suites restructured for speed (#2721)

2026-08-29 09:06:54 -07:00

budget-override.ts

v1.46.0.0 feat: gstack v2 foundation — catalog tokens drop 56%, eval-first floor covers all 51 skills (#1712)

2026-05-26 16:50:03 -07:00

capture-context-budget.ts

v1.71.0.0 feat: token-load reduction — preamble runtime scripts, gated onboarding, 20 skill carves, CLAUDE.md trim (#2691)

2026-08-27 09:50:31 -07:00

capture-parity-baseline.test.ts

v1.46.0.0 feat: gstack v2 foundation — catalog tokens drop 56%, eval-first floor covers all 51 skills (#1712)

2026-05-26 16:50:03 -07:00

capture-parity-baseline.ts

v1.77.0.0 feat: test-infrastructure overhaul wave 1 — matrix deletion, flake telemetry, sync-spawn wedge class extinct (#2746)

2026-08-31 08:55:30 -07:00

carve-guard-checks.ts

v1.57.0.0 feat: carve-guard system + carve cso/document-release/design-consultation (#1907)

2026-06-07 19:13:24 -07:00

carve-guards.ts

Merge origin/main (v1.91.7.0) into test-audit-reduction

2026-09-29 13:48:02 +00:00

carve-plan-fixture.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

carve-section-case.ts

test: retire the finding-count cluster and trim its helpers (C)

2026-09-29 06:08:49 +00:00

ceo-finding-fixture.ts

test: delete tests of dead eval code (A)

2026-09-29 05:09:36 +00:00

ceo-hold-posture-review.ts

fix(plan-ceo-review): restore experience-first expansion framing, require the mode handoff, skip pacing menus

2026-09-29 16:57:54 +00:00

ceo-mode-option.ts

test(ceo-mode-routing): submit a mode review that scrolled past the viewport

2026-09-29 19:16:01 +00:00

ceo-section-loading-fixture.ts

test(ceo-section-loading): recognize an arrow-ordered stale-fill execution by event roles

2026-09-29 17:05:15 +00:00

ceo-split-question-policy.ts

fix(test): recognize ledger row-ID split candidates so collection stops at the last ACK

2026-09-29 15:18:46 +00:00

claude-pty-runner.scope-gate-floor.unit.test.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

claude-pty-runner.ts

test(eng-batching): grade the floor once the review report is complete

2026-09-29 19:15:06 +00:00

claude-pty-runner.unit.test.ts

test(design-floor): answer the seed-declared all-seven 0D focus menu while it is pending

2026-09-29 17:07:38 +00:00

codex-eval.ts

v1.91.2.0 fix: consolidate gstack reliability wave (#2959)

2026-09-26 18:57:53 -04:00

codex-offering-fixture.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

codex-session-runner.ts

v1.91.2.0 fix: consolidate gstack reliability wave (#2959)

2026-09-26 18:57:53 -04:00

cookie-workflow-judge-input.ts

v1.91.2.0 fix: consolidate gstack reliability wave (#2959)

2026-09-26 18:57:53 -04:00

cookie-workflow-manual-review.ts

v1.90.0.0 feat: make browser cookie imports explicit and safe (#2964)

2026-09-25 12:06:45 -04:00

coverage-audit-evidence.ts

v1.89.0.0 feat: add shared-code extraction audit (#2925)

2026-09-24 01:53:58 -04:00

coverage-audit.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

cso-eval-oracles.ts

test: delete tests of dead eval code (A)

2026-09-29 05:09:36 +00:00

cso-ntfs-fixture.ts

v1.90.0.0 feat: make browser cookie imports explicit and safe (#2964)

2026-09-25 12:06:45 -04:00

cso-qualified-cli.ts

v1.87.0.0 feat: add verified CSO audits and replayable repair bundles (#2852)

2026-09-14 15:14:58 -07:00

cso-runtime-catalog.ts

v1.87.0.0 feat: add verified CSO audits and replayable repair bundles (#2852)

2026-09-14 15:14:58 -07:00

disabled-plan-review-fixture.ts

test(outside-plan-disabled): compare named record timestamps as instants; negated authorship is not a current claim

2026-09-29 17:02:31 +00:00

docsync-contract.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

docsync-fault-actor.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

docsync-fault-eval.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

docsync-fixture.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

docsync-observer.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

dx-selected-navigation.ts

v1.86.0.0 feat: route outside reviews by harness (#2850)

2026-09-14 14:32:45 -07:00

e2e-gate.ts

test: add the non-blocking 'marathon' E2E tier

2026-09-29 15:21:17 +00:00

e2e-gate.unit.test.ts

test: add the non-blocking 'marathon' E2E tier

2026-09-29 15:21:17 +00:00

e2e-helpers.ts

test(evals): record a pre-turn API or CLI failure as infra

2026-09-29 19:51:35 +00:00

emulate-bun-windows-eexist.ts

v1.69.0.0 fix: the silent-failure wave — 6 fixes, 5 community PRs absorbed, tracker closed with receipts (#2666)

2026-08-24 08:55:58 -07:00

eng-cache-writer-decision.ts

v1.86.0.0 feat: route outside reviews by harness (#2850)

2026-09-14 14:32:45 -07:00

eng-seeded-coverage.ts

test(eng-batching): bind unsourced native briefs through the report's target

2026-09-29 19:15:06 +00:00

eval-budgets.ts

test(evals): retire every paid automatic retry

2026-09-29 19:09:51 +00:00

eval-store.test.ts

test(evals): add trial records, panelVerdict, expectContract and trial-outcomes JSONL

2026-09-29 18:56:27 +00:00

eval-store.ts

feat(evals): stamp trial series identities and fit panels to the live registry

2026-09-29 19:30:43 +00:00

fake-bun-cli.ts

v1.86.0.0 feat: route outside reviews by harness (#2850)

2026-09-14 14:32:45 -07:00

fake-impeccable.ts

v1.84.0.0 feat: impeccable interop: detector pre-pass in four design skills, DOM-mode scans, open DESIGN.md format, one typed slop catalog (#2832)

2026-09-08 22:40:22 -07:00

fake-plan-seed.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

fs-caps.ts

v1.72.0.0 feat: Aside recommended driver for third-party web actions (#2710)

2026-08-28 11:43:13 -07:00

health-eval-fixture.ts

v1.87.4.0 fix: preserve health failures and disclose coverage (#2882)

2026-09-15 21:35:11 -07:00

hermetic-env.test.ts

test: pin DISABLE_AUTOUPDATER in hermetic env and capture corrupt-seed warning

2026-09-29 14:22:05 +00:00

hermetic-env.ts

test: pin DISABLE_AUTOUPDATER in hermetic env and capture corrupt-seed warning

2026-09-29 14:22:05 +00:00

hermetic-skill-runtime.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

llm-judge.ts

test(judges): sample the recommendation rubric as a panel; never re-ask armJudge

2026-09-29 19:51:35 +00:00

manual-judge-review-fixture.ts

v1.91.4.0 fix: Windows Opera cookie import wave (#2980, refs #2957) (#2987)

2026-09-28 13:09:52 -07:00

native-auto-decide.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

observability.test.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

office-hours-attempt.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

office-hours-completion.ts

test: move the full office-hours workflow to marathon; add a periodic design-draft checkpoint

2026-09-29 15:29:46 +00:00

outside-voice-evidence.ts

v1.86.0.0 feat: route outside reviews by harness (#2850)

2026-09-14 14:32:45 -07:00

outside-voice-fixture.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

outside-voice-receipt.ts

v1.86.0.0 feat: route outside reviews by harness (#2850)

2026-09-14 14:32:45 -07:00

overlay-attempt.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

overlay-case-policy.ts

test: clean up the paid eval lane (B1-B4, B6, B7)

2026-09-29 06:08:49 +00:00

overlay-case.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

overlay-lifecycle.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

overlay-measurement.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

overlay-workspace.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

owned-claude-transcript.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

paid-test-set.ts

test: clean up the paid eval lane (B1-B4, B6, B7)

2026-09-29 06:08:49 +00:00

parity-harness.ts

v1.87.0.0 feat: add verified CSO audits and replayable repair bundles (#2852)

2026-09-14 15:14:58 -07:00

periodic-exclude-data.ts

feat(evals): per-case pass rates with Wilson intervals, identity series and quarantine policy

2026-09-29 19:11:27 +00:00

plan-count-artifacts.ts

test: judge plan-count completion on structured evidence, not wording

2026-09-29 15:04:28 +00:00

plan-count-file-permission.ts

test(pty): grant an owned Create pane whose title row is cropped

2026-09-29 19:47:15 +00:00

plan-count-fixture.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

plan-count-pending-exit.ts

v1.86.0.0 feat: route outside reviews by harness (#2850)

2026-09-14 14:32:45 -07:00

plan-count-pending-question.ts

v1.89.0.0 feat: add shared-code extraction audit (#2925)

2026-09-24 01:53:58 -04:00

plan-count-transcript.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

plan-floor-review.ts

v1.91.2.0 fix: consolidate gstack reliability wave (#2959)

2026-09-26 18:57:53 -04:00

plan-floor-target.ts

v1.86.0.0 feat: route outside reviews by harness (#2850)

2026-09-14 14:32:45 -07:00

plan-mode-evidence.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

plan-review-board-feedback.ts

test: delete tests of dead eval code (A)

2026-09-29 05:09:36 +00:00

plan-review-cases.ts

test: delete tests of dead eval code (A)

2026-09-29 05:09:36 +00:00

plan-review-decisions.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

plan-scope-selection.ts

v1.86.0.0 feat: route outside reviews by harness (#2850)

2026-09-14 14:32:45 -07:00

plan-seed-submission.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

plan-skill-question-events.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

plan-skill-question-hook-scope.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

plan-skill-questions.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

pricing.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

pty-screen.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

pty-trust-dialog.ts

v1.86.0.0 feat: route outside reviews by harness (#2850)

2026-09-14 14:32:45 -07:00

qa-browser-deadline-evidence.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

qa-callers-fixture.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

qa-checkpoint-evidence.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

qa-evidence-producer.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

qa-functional-eval.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

qa-functional-evidence.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

qa-functional-fixture.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

qa-functional-observer.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

qa-only-cleanup.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

resolve-repo-path.test.ts

test: guard the reduced suite against new test-of-test files

2026-09-29 09:16:19 +00:00

resolve-repo-path.ts

test: guard the reduced suite against new test-of-test files

2026-09-29 09:16:19 +00:00

run-bin.test.ts

v1.74.0.0 test/CI overhaul: green means green, suites restructured for speed (#2721)

2026-08-29 09:06:54 -07:00

run-bin.ts

v1.74.0.0 test/CI overhaul: green means green, suites restructured for speed (#2721)

2026-08-29 09:06:54 -07:00

scratch-repo.ts

Merge origin/main (v1.91.7.0) into test-audit-reduction

2026-09-29 13:48:02 +00:00

secret-sink-harness.ts

v1.12.0.0 feat: /setup-gbrain — coding-agent onboarding for gbrain (#1183)

2026-04-24 01:38:21 -07:00

session-drain-policy.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

session-runner.test.ts

v1.88.1.0 fix: harden credential boundaries and owned state (#2942)

2026-09-23 08:54:53 -04:00

session-runner.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

setup-codex-scope-fixture.ts

v1.91.5.0 feat: balanced free-suite shards, test:ubicloud, and faster PR eval lane (#2989)

2026-09-28 16:00:12 -07:00

setup-gbrain-fixture-command.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

setup-gbrain-fixture.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

setup-gbrain-sandbox.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

shared-libs-eval-fixture.ts

test: one finite PR world for the shared-libs fixture; name dual-voice probe evidence

2026-09-29 14:28:11 +00:00

shared-libs-path-fixture.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

shared-libs-plan-actor.ts

v1.89.0.0 feat: add shared-code extraction audit (#2925)

2026-09-24 01:53:58 -04:00

shared-libs-plan-excerpt.ts

v1.89.0.0 feat: add shared-code extraction audit (#2925)

2026-09-24 01:53:58 -04:00

shared-libs-review-start-evidence.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

ship-hook-actor.ts

v1.91.2.0 fix: consolidate gstack reliability wave (#2959)

2026-09-26 18:57:53 -04:00

ship-skip-actor.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

skill-census.ts

v1.63.0.0 feat: GStack 2 fork port wave — egress receipts, context-bill, sharded gate, /health fix (#2541)

2026-08-14 09:28:56 -07:00

skill-fixture.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00

skill-parser.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

sol-skill-fixture.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

sync-command-capture.test.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

sync-command-capture.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

sync-gbrain-readiness-fixture.ts

fix(sync-gbrain): define Step 4 helper args and one atomic write path

2026-09-29 16:19:39 +00:00

sync-gbrain-readiness-verdict.ts

v1.91.2.0 fix: consolidate gstack reliability wave (#2959)

2026-09-26 18:57:53 -04:00

test-selection.ts

test(evals): classify every live case and re-select a case when its kind changes

2026-09-29 19:11:04 +00:00

third-party-actions.ts

v1.90.2.0 perf: halve local free-suite time and preserve coverage (#2972)

2026-09-25 13:27:07 -04:00

tool-map.ts

v1.81.0.0 feat: Aside is the browser gstack drives first; every browsing skill, the PDF/diagram renderer, and web research; the bundled browser stays the automatic fallback (#2810)

2026-09-06 08:54:25 -07:00

touchfile-closure.ts

test: guard the reduced suite against new test-of-test files

2026-09-29 09:16:19 +00:00

touchfiles-data.ts

test(evals): classify every live case and re-select a case when its kind changes

2026-09-29 19:11:04 +00:00

touchfiles.ts

v1.66.0.0 feat: test/evals/CI speedup — 90s truthful free suite, diff-billed evals, required Linux lane (#2593)

2026-08-15 22:20:30 -07:00

workflow-boundaries-fixture.ts

v1.91.2.0 fix: consolidate gstack reliability wave (#2959)

2026-09-26 18:57:53 -04:00

workflow-excerpt.ts

v1.87.6.0 fix: make checks reliable and everyday validation faster (#2898)

2026-09-22 14:57:52 -04:00

workflow-judge-cache.ts

test(llm-judge): sample every judge as a pre-registered 3-sample panel

2026-09-29 19:11:04 +00:00

workflow-judge-input.ts

v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

2026-09-29 06:07:35 -07:00