v1.91.156.0 fix(evals): office-hours marathon fixture loads the formatter's imports; validator checks the observed cap fixes (#3246) (#3309)

* fix(evals): office-hours marathon fixture copies the formatter import closure; validator gets the observed cap fixes (#3246)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* v1.91.156.0 chore: stamp #3309 (office-hours marathon fixture and cap-fix validation)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

CHANGELOG.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## [1.91.156.0] - 2026-10-19
+
+**The weekly marathon's office-hours case can run its review formatter again, and its validator now checks cap fixes the way the formatter does.** Both defects came from #3164 and were in the test, not the skill. The marathon fixture copied `bin/gstack-office-hours-review` with a hand-kept list of two library files; #3164 added a third import (`lib/spec-review-cap.ts`), so every formatter command in the fixture failed at module load, the agent correctly declared the review UNREVIEWED, and the case went red in marathon run 39032308051 (#3246). With the module restored, a focused rerun showed the second defect: when the review stops at the round cap and the agent applies the remaining blocking fixes with `--fixes`, the formatter records them as verified, but the test never gave the validator that list, so it expected "fixes unconfirmed" and rejected a correct report.
+
+### Fixed
+- **Office-hours cap fixes:** `observedSpecFixes()` in `test/helpers/office-hours-completion.ts` returns the last valid `--fixes` list the agent saved with Write, and the marathon test passes it to `validateOfficeHoursReviewArtifacts`. A free test in `test/office-hours-completion.test.ts` runs the real `finalize --fixes` at the cap and checks three cases: the observed list validates, no list fails with the marathon rerun's `Spec Review does not match` error, and a list naming a different finding also fails.
+- **Office-hours marathon fixture:** `test/helpers/office-hours-formatter-fixture.ts` copies the formatter together with every local module it imports, so a new import can no longer be left out. A free test in `test/office-hours-review.test.ts` loads the copied formatter and fails on a missing module; the old three-file list reproduces the exact `Cannot find module '../lib/spec-review-cap.ts'` error. The case's touchfiles now include `lib/spec-review-cap.ts` and the new helper.
+- **Gate integrity:** no threshold, trial count, case kind or judge changed. The marathon assertions are unchanged; only the fixture's file copy changed.
+
 ## [1.91.155.0] - 2026-10-19
 
 **A busy page no longer locks up the browse daemon.** When one tab kept its main thread busy (a heavy dev bundle, a big grid render, a runaway script), every `browse` command, including `status`, `tabs`, `tab <id>`, `closetab` and `stop`, failed with "Daemon busy" after about 9.6 s, and the only way out was `--force-restart`, which threw away cookies, tabs and logins. Fixes #3247. From #3250 by @AndreIntelas.

VERSION

@@ -1 +1 @@
-1.91.155.0
+1.91.156.0

agents-digest/gstack-AGENTS.md

@@ -1,4 +1,4 @@
-# gstack digest v1.91.155.0 — regenerate/re-copy after upgrading gstack
+# gstack digest v1.91.156.0 — regenerate/re-copy after upgrading gstack
 
 Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
 for agent hosts without a full skill install. The full skills add workflows,

package.json

@@ -1,6 +1,6 @@
 {
   "name": "gstack",
-  "version": "1.91.155",
+  "version": "1.91.156",
   "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
   "license": "MIT",
   "type": "module",

test/helpers/office-hours-completion.ts

@@ -1,7 +1,7 @@
 import { isDeepStrictEqual } from 'node:util';
 import * as path from 'node:path';
 import { renderOfficeHoursReviewerPrompt, validateOfficeHoursReview, renderOfficeHoursReview, extractOfficeHoursReviewBlock, parseOfficeHoursVerdictReceipt, verifyOfficeHoursVerdictReceipt, officeHoursDesignChanges, verifyOfficeHoursCitations, type OfficeHoursReview } from '../../lib/office-hours-review';
-import { verifySpecFixesAtCap, type SpecFix } from '../../lib/spec-review-cap';
+import { parseSpecFixes, verifySpecFixesAtCap, type SpecFix } from '../../lib/spec-review-cap';
 
 export type OfficeHoursFile = { path: string; content: string | null };
 /** The helper's captured diff before a later round; a missing snapshot fails closed. */
@@ -32,6 +32,21 @@ export interface OfficeHoursCompletionEvidence {
   fixes?: SpecFix[] | null;
 }
 
+/**
+ * The cap fixes the agent saved for the finalizer: the last observed Write whose
+ * content is a valid `--fixes` list. Null when it saved none. The validator
+ * recomputes the cap record from this list, so a list that was not the one
+ * passed to finalize still produces a mismatch.
+ */
+export function observedSpecFixes(toolCalls: OfficeHoursCompletionEvidence['toolCalls']): SpecFix[] | null {
+  for (const call of toolCalls.toReversed()) {
+    if (call.tool !== 'Write') continue;
+    try { return parseSpecFixes(JSON.parse(String(call.input?.content ?? ''))); }
+    catch { /* not a fixes list */ }
+  }
+  return null;
+}
+
 export interface OfficeHoursReviewEvidence {
   verdict: string;
   concerns: string;

test/helpers/office-hours-formatter-fixture.ts

@@ -0,0 +1,31 @@
+/**
+ * Copy bin/gstack-office-hours-review and every local module it imports into a
+ * fixture, so the copied formatter loads the same modules as the checkout.
+ * Marathon 39032308051 copied a hand-kept file list that missed
+ * lib/spec-review-cap.ts, and every formatter command failed at module load.
+ */
+import * as fs from 'node:fs';
+import * as path from 'node:path';
+
+const ROOT = path.resolve(import.meta.dir, '../..');
+
+export function copyOfficeHoursFormatter(fixtureRoot: string): string {
+  const pending = ['bin/gstack-office-hours-review'];
+  const copied = new Set<string>();
+  while (pending.length) {
+    const relative = pending.pop()!;
+    if (copied.has(relative)) continue;
+    copied.add(relative);
+    const source = path.join(ROOT, relative);
+    const target = path.join(fixtureRoot, relative);
+    fs.mkdirSync(path.dirname(target), { recursive: true });
+    fs.copyFileSync(source, target);
+    for (const [, specifier] of fs.readFileSync(source, 'utf8').matchAll(/\b(?:from|import)\s+'(\.\.?\/[^']+)'/g)) {
+      const resolved = path.relative(ROOT, path.resolve(path.dirname(source), specifier!));
+      pending.push(path.extname(resolved) ? resolved : `${resolved}.ts`);
+    }
+  }
+  const formatter = path.join(fixtureRoot, 'bin/gstack-office-hours-review');
+  fs.chmodSync(formatter, 0o755);
+  return formatter;
+}

test/helpers/touchfiles-data.ts

@@ -1099,7 +1099,7 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
     'test/skill-e2e-arm-benchmark.test.ts',
     'ship/SKILL.md', 'test/helpers/e2e-gate.ts'],
 
-  'office-hours-section-loading': [ 'office-hours/**', 'bin/gstack-office-hours-review', 'lib/office-hours-review.ts', 'lib/fs-atomic.ts', 'scripts/resolvers/review-dashboard.ts', 'scripts/resolvers/plan-gates.ts', 'scripts/resolvers/spec-review.ts', 'scripts/resolvers/outside-voice-steps.ts', 'scripts/resolvers/review-scope.ts', 'scripts/resolvers/outside-voice.ts', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/carve-guards.ts', 'test/helpers/auq-sdk-capture.ts',  'test/helpers/office-hours-completion.ts', 'test/helpers/llm-judge.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-office-hours-section-loading.test.ts', 'test/helpers/agent-sdk-runner.ts', 'test/helpers/auq-native-capture.ts', 'test/helpers/auto-decision-state.ts', 'test/helpers/autoplan-artifact-digest.ts', 'test/helpers/autoplan-artifact-permission.ts', 'test/helpers/autoplan-artifact-recorder.ts', 'test/helpers/capture-parity-baseline.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty/**', 'test/helpers/dx-selected-navigation.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/eng-cache-writer-decision.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/helpers/native-auto-decide.ts', 'test/helpers/owned-claude-transcript.ts', 'test/helpers/parity-harness.ts', 'test/helpers/plan-count-artifacts.ts', 'test/helpers/plan-count-file-permission.ts', 'test/helpers/plan-count-fixture.ts', 'test/helpers/plan-count-pending-exit.ts', 'test/helpers/plan-count-pending-question.ts', 'test/helpers/plan-count-transcript.ts', 'test/helpers/plan-floor-review.ts', 'test/helpers/plan-floor-target.ts', 'test/helpers/plan-scope-selection.ts', 'test/helpers/plan-seed-submission.ts', 'test/helpers/plan-skill-question-events.ts', 'test/helpers/plan-skill-question-hook-scope.ts', 'test/helpers/plan-skill-questions.ts', 'test/helpers/pty-screen.ts', 'test/helpers/pty-trust-dialog.ts', 'test/helpers/skill-census.ts'],
+  'office-hours-section-loading': [ 'office-hours/**', 'bin/gstack-office-hours-review', 'lib/office-hours-review.ts', 'lib/fs-atomic.ts', 'lib/spec-review-cap.ts', 'test/helpers/office-hours-formatter-fixture.ts', 'scripts/resolvers/review-dashboard.ts', 'scripts/resolvers/plan-gates.ts', 'scripts/resolvers/spec-review.ts', 'scripts/resolvers/outside-voice-steps.ts', 'scripts/resolvers/review-scope.ts', 'scripts/resolvers/outside-voice.ts', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/carve-guards.ts', 'test/helpers/auq-sdk-capture.ts',  'test/helpers/office-hours-completion.ts', 'test/helpers/llm-judge.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-office-hours-section-loading.test.ts', 'test/helpers/agent-sdk-runner.ts', 'test/helpers/auq-native-capture.ts', 'test/helpers/auto-decision-state.ts', 'test/helpers/autoplan-artifact-digest.ts', 'test/helpers/autoplan-artifact-permission.ts', 'test/helpers/autoplan-artifact-recorder.ts', 'test/helpers/capture-parity-baseline.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty/**', 'test/helpers/dx-selected-navigation.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/eng-cache-writer-decision.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/helpers/native-auto-decide.ts', 'test/helpers/owned-claude-transcript.ts', 'test/helpers/parity-harness.ts', 'test/helpers/plan-count-artifacts.ts', 'test/helpers/plan-count-file-permission.ts', 'test/helpers/plan-count-fixture.ts', 'test/helpers/plan-count-pending-exit.ts', 'test/helpers/plan-count-pending-question.ts', 'test/helpers/plan-count-transcript.ts', 'test/helpers/plan-floor-review.ts', 'test/helpers/plan-floor-target.ts', 'test/helpers/plan-scope-selection.ts', 'test/helpers/plan-seed-submission.ts', 'test/helpers/plan-skill-question-events.ts', 'test/helpers/plan-skill-question-hook-scope.ts', 'test/helpers/plan-skill-questions.ts', 'test/helpers/pty-screen.ts', 'test/helpers/pty-trust-dialog.ts', 'test/helpers/skill-census.ts'],
   'office-hours-design-draft': [ 'office-hours/**', 'lib/office-hours-review.ts', 'scripts/resolvers/review-dashboard.ts', 'scripts/resolvers/plan-gates.ts', 'scripts/resolvers/spec-review.ts', 'scripts/resolvers/outside-voice-steps.ts', 'scripts/resolvers/review-scope.ts', 'scripts/resolvers/outside-voice.ts', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/carve-guards.ts', 'test/helpers/auq-sdk-capture.ts',  'test/helpers/office-hours-completion.ts', 'test/helpers/llm-judge.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-office-hours-design-draft.test.ts', 'test/helpers/agent-sdk-runner.ts', 'test/helpers/auq-native-capture.ts', 'test/helpers/auto-decision-state.ts', 'test/helpers/autoplan-artifact-digest.ts', 'test/helpers/autoplan-artifact-permission.ts', 'test/helpers/autoplan-artifact-recorder.ts', 'test/helpers/capture-parity-baseline.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty/**', 'test/helpers/dx-selected-navigation.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/eng-cache-writer-decision.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/helpers/native-auto-decide.ts', 'test/helpers/owned-claude-transcript.ts', 'test/helpers/parity-harness.ts', 'test/helpers/plan-count-artifacts.ts', 'test/helpers/plan-count-file-permission.ts', 'test/helpers/plan-count-fixture.ts', 'test/helpers/plan-count-pending-exit.ts', 'test/helpers/plan-count-pending-question.ts', 'test/helpers/plan-count-transcript.ts', 'test/helpers/plan-floor-review.ts', 'test/helpers/plan-floor-target.ts', 'test/helpers/plan-scope-selection.ts', 'test/helpers/plan-seed-submission.ts', 'test/helpers/plan-skill-question-events.ts', 'test/helpers/plan-skill-question-hook-scope.ts', 'test/helpers/plan-skill-questions.ts', 'test/helpers/pty-screen.ts', 'test/helpers/pty-trust-dialog.ts', 'test/helpers/skill-census.ts'],
   'dx-error-contract-review': ['test/skill-e2e-dx-error-contract.test.ts', 'test/fixtures/dx-error-contract-case.ts', 'lib/dx-error-contract.ts', 'scripts/resolvers/dx.ts', 'scripts/resolvers/index.ts', 'scripts/gen-skill-docs.ts', 'plan-devex-review/**', 'bin/gstack-autoplan-snapshot.ts', 'lib/autoplan-prompts.ts', 'lib/autoplan-reconcile.ts', 'lib/coordinator-contract.ts', 'test/helpers/session-runner.ts', 'test/helpers/eval-store.ts', 'test/helpers/eval-budgets.ts', 'lib/claude-bin.ts', 'lib/eval-model.ts'],
   'plan-devex-peer-comparison-classification': [

test/office-hours-completion.test.ts

@@ -4,7 +4,7 @@ import { describe, expect, test } from 'bun:test';
 import * as fs from 'node:fs';
 import * as path from 'node:path';
 import * as os from 'node:os';
-import { validateOfficeHoursCompletion, validateOfficeHoursDesignDraft, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation, type OfficeHoursCompletionEvidence } from './helpers/office-hours-completion';
+import { observedSpecFixes, validateOfficeHoursCompletion, validateOfficeHoursDesignDraft, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation, type OfficeHoursCompletionEvidence } from './helpers/office-hours-completion';
 import { E2E_TOUCHFILES } from './helpers/touchfiles-data';
 import { selectTests } from './helpers/test-selection';
 import { expectMentions } from './helpers/prompt-structure';
@@ -954,6 +954,42 @@ Next: /plan-eng-review after observing the workflow. The user declined launching
     } finally { fs.rmSync(dir, { recursive: true, force: true }); }
   });
 
+  test('cap fixes the agent saved and passed to finalize validate; without them the report mismatches (marathon rerun 2026-10-19)', () => {
+    const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'office-hours-cap-fixes-'));
+    try {
+      const { evidence, artifacts } = JSON.parse(JSON.stringify(structured())
+        .replaceAll('/tmp/office-hours-fixture', dir.replaceAll('\\', '/')));
+      evidence.toolCalls.filter((call: { output?: string }) => call.output?.startsWith('OFFICE_HOURS_VERDICT '))
+        .forEach((call: { output?: string }, i: number) => { call.output = officeHoursVerdictReceipt(i + 1, artifacts[i].path, artifacts[i].content); });
+      const applied = 'An absent mapped column stops the import and names the missing column.';
+      const fixes = [{ id: 'R2-1', applied_text: applied }];
+      const reviewDir = path.dirname(artifacts[0].path);
+      const fixesPath = path.join(reviewDir, 'fixes.json');
+      const reportPath = path.join(dir, 'REPORT.md');
+      fs.mkdirSync(path.dirname(evidence.designPath), { recursive: true });
+      fs.writeFileSync(evidence.designPath, evidence.designContent.replace('# Design: RosterCheck\n', `# Design: RosterCheck\n\n${applied}\n`));
+      fs.writeFileSync(reportPath, evidence.output);
+      fs.mkdirSync(reviewDir, { recursive: true });
+      for (const artifact of artifacts) fs.writeFileSync(artifact.path, artifact.content);
+      const snapshots = snapshotsIn(reviewDir);
+      for (const snapshot of snapshots) fs.writeFileSync(snapshot.path, snapshot.content);
+      fs.writeFileSync(fixesPath, JSON.stringify(fixes));
+      const result = Bun.spawnSync([process.execPath, path.resolve(import.meta.dir, '../bin/gstack-office-hours-review'),
+        'finalize', '--design', evidence.designPath, '--report', reportPath, '--fixes', fixesPath, ...artifacts.map((artifact: { path: string }) => artifact.path)],
+      { cwd: dir, timeout: 5000 });
+      expect(result.exitCode, result.stderr.toString()).toBe(0);
+      expect(JSON.parse(result.stdout.toString()).cap.verified).toEqual(['R2-1']);
+      evidence.output = fs.readFileSync(reportPath, 'utf8');
+      evidence.designContent = fs.readFileSync(evidence.designPath, 'utf8');
+      evidence.toolCalls.push({ tool: 'Write', input: { file_path: fixesPath, content: JSON.stringify(fixes) } });
+      expect(observedSpecFixes(evidence.toolCalls)).toEqual(fixes);
+      expect(() => checkArtifacts({ ...evidence, fixes: observedSpecFixes(evidence.toolCalls) }, artifacts, snapshots)).not.toThrow();
+      expect(() => checkArtifacts(evidence, artifacts, snapshots)).toThrow('Spec Review does not match');
+      expect(() => checkArtifacts({ ...evidence, fixes: [{ id: 'R2-2', applied_text: applied }] }, artifacts, snapshots)).toThrow('Spec Review does not match');
+      expect(observedSpecFixes(evidence.toolCalls.filter((call: { input?: { file_path?: string } }) => call.input?.file_path !== fixesPath))).toBeNull();
+    } finally { fs.rmSync(dir, { recursive: true, force: true }); }
+  });
+
   test('allows a Markdown separator after the exact block, without hiding extra outcome prose', () => {
     const { evidence, artifacts } = structured();
     evidence.output = evidence.output.replace('## Handoff', '---\n\n## Handoff');

test/office-hours-review.test.ts

@@ -9,6 +9,7 @@ import {
 } from '../lib/office-hours-review';
 import { parseSpecFixes, verifySpecFixesAtCap, renderSpecCap } from '../lib/spec-review-cap';
 import { expectMentions } from './helpers/prompt-structure';
+import { copyOfficeHoursFormatter } from './helpers/office-hours-formatter-fixture';
 
 const cli = path.resolve(import.meta.dir, '../bin/gstack-office-hours-review');
 function review(round = 1, count = 1, severity: 'blocking' | 'minor' = 'blocking'): OfficeHoursReview {
@@ -638,3 +639,17 @@ describe('office-hours review CLI', () => {
     }
   }));
 });
+
+describe('office-hours formatter fixture copy', () => {
+  test('the copied formatter loads every local module it imports (marathon 39032308051)', () => {
+    const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'office-hours-formatter-copy-'));
+    try {
+      const formatter = copyOfficeHoursFormatter(dir);
+      const result = Bun.spawnSync([process.execPath, formatter], { cwd: dir, stdout: 'pipe', stderr: 'pipe', timeout: 5000 });
+      expect(result.stderr.toString()).not.toContain('Cannot find module');
+      expect(result.stderr.toString()).toContain('gstack-office-hours-review: usage:');
+    } finally {
+      fs.rmSync(dir, { recursive: true, force: true });
+    }
+  });
+});

test/skill-e2e-office-hours-section-loading.test.ts

@@ -14,7 +14,8 @@ import * as path from 'node:path';
 import { describeE2ETier } from './helpers/e2e-gate';
 import { setupSkillDir, skillFromWorktree, captureSectionReads } from './helpers/auq-sdk-capture';
 import { CARVE_GUARDS } from './helpers/carve-guards';
-import { validateOfficeHoursCompletion, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation } from './helpers/office-hours-completion';
+import { copyOfficeHoursFormatter } from './helpers/office-hours-formatter-fixture';
+import { observedSpecFixes, validateOfficeHoursCompletion, validateOfficeHoursReviewerHandoffs, validateOfficeHoursReviewArtifacts, validateOfficeHoursReviewPreservation } from './helpers/office-hours-completion';
 
 const describeMarathon = describeE2ETier('marathon');
 const runId = `office-hours-section-loading-${process.env.EVALS_RUN_ID ?? 'local'}`;
@@ -42,15 +43,9 @@ describeMarathon('/office-hours full section-loading workflow (marathon)', () =>
     const planDir = setupSkillDir({
       skillName: guard.skill, skillMd, sectionsFrom, tmpPrefix: 'gstack-office-hours-secload-',
     });
-    // Copy the real formatter and its small dependency set into the fixture.
+    // Copy the real formatter and its local imports into the fixture.
     // This exercises production code while keeping all outputs in the sandbox.
-    for (const relative of ['bin/gstack-office-hours-review', 'lib/office-hours-review.ts', 'lib/fs-atomic.ts']) {
-      const target = path.join(planDir, relative);
-      fs.mkdirSync(path.dirname(target), { recursive: true });
-      fs.copyFileSync(path.resolve(__dirname, '..', relative), target);
-    }
-    const formatter = path.join(planDir, 'bin/gstack-office-hours-review');
-    fs.chmodSync(formatter, 0o755);
+    const formatter = copyOfficeHoursFormatter(planDir);
     // Two rounds keep the live run inside its budget: round 1 (full) and round 2 (delta) run here; round 3 and CONVERGENCE are covered by free validator tests.
     const reviewRounds = 2;
     const capture = await captureSectionReads({
@@ -83,6 +78,7 @@ Completion delivery, after the full workflow and design approval:
     const reviewEvidence = validateOfficeHoursReviewArtifacts({
       ...capture, designPath,
       designContent: fs.existsSync(designPath) ? fs.readFileSync(designPath, 'utf-8') : null,
+      fixes: observedSpecFixes(capture.toolCalls),
     }, artifacts, snapshots, reviewRounds);
     validateOfficeHoursReviewerHandoffs({
       ...capture, designPath,