test(W7h): replace workflow-judge-cache source-grep counts with an invariant

W7h retirement (t09 F16, t13 F6) of the source-grep counts in 'workflow
registration preserves model work and reserves only terminal-recording
grace'. The stage-order check over runWorkflowJudge (the cheapest guard) and
every behavioral callback test stay.

Retirement card:
- test: workflow-judge-cache.test.ts pins of skill-llm-eval.test.ts source:
  "execution: reused ? 'reused' : 'executed'", 'const workDeadline = started
  + JUDGE_MS;', 'const WORKFLOW_JUDGE_RECORD_MS = 5_000;', 'const
  WORKFLOW_JUDGE_TEST_MS = JUDGE_MS + 10_000;', "testName: 'review/SKILL.md
  workflow'", and the magic counts /}, WORKFLOW_JUDGE_TEST_MS\);/ x17 and
  /}, JUDGE_MS\);/ x7.
- detects: any workflow eval added or removed, any constant reformatted.
- non_test_callers: none.
- stronger_proof / replacement: (a) a new invariant in the same test: every
  `await runWorkflowJudge(` registration closes with WORKFLOW_JUDGE_TEST_MS
  (the property the 17-count stood in for, now independent of how many evals
  exist); (b) the real-callback tests in this file exercise execute-once,
  reuse with provenance (execution reused/executed), timeout, supersede,
  expiry and the recording deadline.
- history: 636175d (#2898); last 7fca42a.
- validation: bun test test/workflow-judge-cache.test.ts.

Controls (scratch worktree, test/skill-llm-eval.test.ts): moving cache.publish
before the score assertions or registering the ship workflow judge with
JUDGE_MS goes red; adding a correctly registered eval and reformatting
5_000 as 5000 (both red under the old pins) stay green (4/4). Log:
.context/mutation-workflow-judge-cache.md.
This commit is contained in:
garrytan committed 2026-10-03 01:34:23 +00:00
1 parent 93d2d43c2c
commit 8ac565d615
1 file changed
+6 -7
+6 -7
View File
@@ -177,13 +177,12 @@ test('workflow registration preserves model work and reserves only terminal-reco
.map(stage => body.indexOf(stage));
expect(stages.every(position => position >= 0)).toBe(true);
expect(stages).toEqual([...stages].sort((a, b) => a - b));
expect(body).toContain("execution: reused ? 'reused' : 'executed'");
expect(body).toContain('const workDeadline = started + JUDGE_MS;');
expect(source).toContain('const WORKFLOW_JUDGE_RECORD_MS = 5_000;');
expect(source).toContain('const WORKFLOW_JUDGE_TEST_MS = JUDGE_MS + 10_000;');
expect(source.match(/\}, WORKFLOW_JUDGE_TEST_MS\);/g)).toHaveLength(17);
expect(source).toContain("testName: 'review/SKILL.md workflow'");
expect(source.match(/\}, JUDGE_MS\);/g)).toHaveLength(7);
const calls = [...source.matchAll(/await runWorkflowJudge\(/g)].map(match => match.index!);
expect(calls.length).toBeGreaterThan(0);
for (const call of calls) {
const timeout = /\n\s*\}, (\w+)\);/.exec(source.slice(call))?.[1];
expect(timeout, `runWorkflowJudge registration at offset ${call} must use WORKFLOW_JUDGE_TEST_MS`).toBe('WORKFLOW_JUDGE_TEST_MS');
}
});
function actualCallback(f: ReturnType<typeof fixture>, overrides: {