Files
gstack/test
garrytan 647d7aa48a test: preregistered W2 judge calibration corpora (W2 T10, T8)
24 items per request configuration (arm, qa-workflow, qa-health-rubric,
qa-anti-refusal, cross-skill, voice, workflow-default): 8 pass, 8 fail,
8 threshold-adjacent, with a stratified held-out third. Built from real
gstack skill text and hand-degraded variants (arm: real git diffs).
Expected verdicts were labeled by the author, then blind by a separate
agent run before any schema was written; 12 of 168 disagreed (all
adjacent or reviewer-low-confidence), two unclear pass items were replaced
and re-reviewed, and the rest were adjudicated with notes in corpus.json.
2026-10-03 20:56:58 +00:00
..