Commit Graph
7 Commits
Author SHA1 Message Date
garrytan adced7e046 test(evals): add trial records, panelVerdict, expectContract and trial-outcomes JSONL
EvalTestEntry gains case_id, kind, trial, panel, failure_class and
policy_version, stamped from the runner's TRIAL_ENV on isolated trial
shards. panelVerdict() is the single verdict function (INCOMPLETE on
missing or duplicate trials, contract veto at any count, quarantine
hard-break rule, INFRA/INCOMPLETE machine classification). expectContract()
records failure_class 'contract' on the collector entry and a sidecar
before throwing. trial-outcomes JSONL has a fail-closed writer and a
data-only reader.
2026-09-29 18:56:27 +00:00
garrytan 618c2906b4 chore(typecheck): lock in two fixed test diagnostics 2026-09-29 16:19:52 +00:00
garrytan 4973f96d98 test: move the full office-hours workflow to marathon; add a periodic design-draft checkpoint
The full startup workflow runs 1–3 real spec-review rounds (~280s each) and
hit its 1200s capture in run 36385945043 at finalize. Review depth is the
product's loop, so the case cannot fit a blocking lane without cutting
rounds. It is now marathon tier with every assertion unchanged.

skill-e2e-office-hours-design-draft.test.ts (periodic) runs the same fixed
interview only through the Write that creates the design (269s in that run)
and applies the full validator's design-draft checks, the required section
reads and the launch/foreign-skill-read guards. validateOfficeHoursDesignDraft
is extracted from validateOfficeHoursCompletion, which still applies it.

Selection: office-hours-design-draft is registered periodic; the marathon-only
file is already excluded from the gate and periodic plans by the B5 planner
rule. Tier-alignment regexes and the valid-tier check accept 'marathon'.
A type-only cast in plan-scope-selection.test.ts removes a diagnostic whose
union print order made the ratchet identity unstable; baseline tightened.
2026-09-29 15:29:46 +00:00
garrytan 2ba451a49d fix(typecheck): strip the checkout root from ratchet diagnostic identities 2026-09-29 15:17:03 +00:00
garrytan 028b9040e9 chore: regenerate the test type-debt baseline on top of #2994 2026-09-29 15:09:35 +00:00
garrytan 99058ee3cd test: follow the headed env helper and the typecheck gate in source-shape checks 2026-09-29 14:42:11 +00:00
garrytan 42e53413da feat: require a zero-error product typecheck and a test type-debt ratchet
Adds tsconfig.json (strict) over product code, fixes its remaining 90
diagnostics (type-only, interface corrections, and explicit narrowing),
and adds a typecheck job to the required free-tests aggregate running
bun run typecheck, the test-code ratchet (identity -> count baseline, fails
on new, repeated, or unlocked fixed diagnostics), and the lib/cso format
check. Reuses fixes from #2447 where they still applied.
2026-09-29 14:34:02 +00:00