test(evals): record the read-only and detector-row invariants as contracts

shared-libs-opportunity-judgment and review-design-lite are behavior
cases: their recommendation and checklist judgments may vary, but the
read-only invariant (commands, provider requests, fixture bytes, hooks,
state) and the deterministic fake-engine detector rows are contracts.
Both now go through expectContract, so any failure vetoes the panel.
This commit is contained in:
garrytan committed 2026-09-29 19:51:35 +00:00
1 parent b2ca207cf0
commit 2fb55e4c7a
2 files changed
+19 -10

No files matched your search

+3 -1
View File
@@ -8,6 +8,7 @@ import {
createEvalCollector, finalizeEvalCollector,
} from './helpers/e2e-helpers';
import { extractSkillSections, REVIEW_E2E_SECTIONS } from './helpers/skill-fixture';
import { expectContract } from './helpers/eval-store';
import { spawnSync } from 'child_process';
import * as fs from 'fs';
import * as path from 'path';
@@ -291,7 +292,8 @@ Important: The design checklist should catch issues like blacklisted fonts, smal
console.log(`Design review detected ${detected}/7 planted checklist signals; detector rows surfaced: ${detectorSeen}`);
expect(detected).toBeGreaterThanOrEqual(4); // the LLM-checklist bar, unchanged by the detector
expect(detectorSeen).toBe(true); // the fake engine's rows are deterministic; the review must carry them
// The fake engine's rows are deterministic; carrying them is the contract.
expectContract(detectorSeen, 'review-design-lite: the review omitted the mechanical detector rows', { collector: evalCollector, name: '/review design lite' });
}
}, CAPTURE_MS + REVIEW_FINALIZE_MS);
});