test: browse reference judge gates on its stated floors; the baseline comparison is recorded, not a ratchet (W7d browse judge baseline ratchet; t08 F7)

expect(regressions).toEqual([]) compared a noisy three-sample mean against
eval-baselines.json (clarity 4), silently raising the stated clarity floor of
3. The comparison stays in judge_reasoning; pass and assertions use
BROWSE_JUDGE_FLOORS (clarity 3, completeness 4, actionability 4). Judge
prompt and schema are unchanged. Stored panel means: three known-good (two the
ratchet rejected) pass; three below-floor panels fail.
This commit is contained in:
garrytan committed 2026-10-03 00:44:12 +00:00
1 parent 99720e744e
commit 0725d99d99
4 files changed
+45 -8

No files matched your search

+12
View File
@@ -135,3 +135,15 @@ export function prepareWorkflowJudgeCache(opts: WorkflowCacheOptions): {
},
};
}
/**
* Pass floors for the browse reference judge panel mean. The stored baseline
* is recorded for comparison only: a three-sample mean is too noisy for a
* no-dip ratchet, and gating on it silently raised the clarity floor to 4.
*/
export const BROWSE_JUDGE_FLOORS = { clarity: 3, completeness: 4, actionability: 4 } as const;
export function browseJudgeFloorsMet(scores: Record<keyof typeof BROWSE_JUDGE_FLOORS, number>): boolean {
return (Object.keys(BROWSE_JUDGE_FLOORS) as Array<keyof typeof BROWSE_JUDGE_FLOORS>)
.every(dim => scores[dim] >= BROWSE_JUDGE_FLOORS[dim]);
}