v1.89.0.0 feat: add shared-code extraction audit (#2925)

* feat: bind shared-code review advice to source and branch

* feat: add shared-code extraction audit and scoped review checks

* test: recognize complete source reads and explicit coverage legends

* chore: bump version and changelog (v1.88.0.0)

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* test: capture native review questions and retain public evidence

Capture the actual first public native question with strict ownership and display matching. Preserve terminal failures and raw evidence, and retain SDK completion checks.

* test: recognize verified review evidence and complete fixtures

Recognize complete source and diagram evidence, concrete design and developer-experience decisions, and the complete planted scenario contracts. Preserve negative controls and grading thresholds.

* fix: preserve decision brief structure in native questions

Keep the required pros-and-cons heading and final Net field in native question text. Regenerate host outputs and document the release and evaluation repairs.

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* docs: update project documentation for v1.88.0.0

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix: correct eval retry accounting and ship workflow gates

* fix: capture native eval evidence and stabilize CI fixtures

* fix: keep shared-code eval skips read-only

Choose explicit no-change answers instead of mixed fix/preservation options.
Reuse the bounded revalidation prompt for path fixtures so required review
metadata is available without repeated discovery. Preserve source checks,
retry limits, and failed native terminal outcomes.

Add captured-question and callback regressions, plus evaluation selection
coverage for the affected fixtures.

---------

Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
Garry Tan
2026-09-24 01:53:58 -04:00
committed by GitHub
co-authored by OpenAI Codex
parent b9706f3635
commit 06ed920a97
177 changed files with 13244 additions and 2477 deletions
@@ -1,26 +1,10 @@
/**
* AskUserQuestion format-compliance gate (gate, paid, SDK capture).
* AskUserQuestion format-compliance gate (gate, paid, native SDK tool capture).
*
* Asserts: /plan-ceo-review's first AskUserQuestion (Step 0F mode selection) is a
* compliant decision brief — all 7 mandated format elements present, with a
* substantive recommendation.
*
* Why SDK capture, not real-PTY (changed v1.59+): the prior version launched an
* interactive `claude` PTY and grepped the rendered TUI after stripAnsi. But
* plan-mode AUQs render as an interactive cursor picker whose cursor-positioning
* escapes stripAnsi CANNOT faithfully flatten — verified directly: the picker
* renders fine for a human (cursorSeen=45) but the flattened text drops `ELI10:`
* and `(recommended)` and `parseNumberedOptions` returns 0. So the old test was
* grading a lossy projection of the TUI, not the question's actual format, and
* failed by construction in this environment.
*
* This version drives the skill via the SDK $OUT_FILE capture path (the agent
* writes the verbatim AskUserQuestion it would have shown to a file — clean text,
* zero rendering loss) and grades that. Same property tested (does the question
* carry every format element), reliably, environment-independent. The rendering
* layer is identical across skills/content, so it is not where format regressions
* hide; the model's composed question is. Shares the engine with the periodic
* A/B and matrix evals (test/helpers/auq-sdk-capture.ts).
* Drives the real four-mode question through AskUserQuestion, then captures its
* exact public permission-callback fields without answering. The 12-turn/240s
* capture keeps its original bounds. Grades remain all 7 decision-brief format
* elements and recommendation substance >=4; capture is not workflow completion.
*/
import { test, expect } from 'bun:test';
import { CAPTURE_MS } from './helpers/eval-budgets';