Commit Graph
2 Commits
Author SHA1 Message Date
Garry TanandClaude Fable 5 4d5af0efe4 fix(evals): AUQ A/B vendored pre-cut arm + judge-error inconclusive taxonomy
- The PRE arm read a branch-local SHA (3263fffe) that becomes unreachable on
  fresh clones after the squash-merge; the pre-cut render is now a vendored
  fixture.
- A judge failure on one side no longer coerces substance to 0 (which
  fabricated DEGRADATION on POST-side failures and masked regressions on
  PRE-side failures): null substance = inconclusive, format still gates.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-29 05:18:52 +00:00
Garry TanandClaude Fable 5 781f46d025 feat(preamble): AskUserQuestion repetition cut — gated, passed NOT-WORSE A/B
Removes the duplicate statements v1.71's compaction left in the
AskUserQuestion Format section: the completeness rule restated in the prose
triad, the auto-decide marker syntax stated twice, the Conductor-flakiness
explanation stated twice, and the self-check's full triad restatement. Every
verbosity floor and all 14 format pins stay (Layer 0 green).

The gate this decision rested on ran before landing (new periodic
skill-e2e-auq-repetition-cut-ab.test.ts, pre-cut ref 3263fffe vs this
render, same harness as auq-verbose-vs-carved-ab): POST 7/7 format elements,
substance 5 — identical to PRE. No degradation; the load-bearing-repetition
hypothesis did not hold for these duplicates.

Net: -236 bytes per tier-2+ skill (~9.7KB corpus). Golden ship baselines
re-captured for the deliberate change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-28 02:00:50 +00:00