mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-27 07:01:54 +02:00
v1.89.0.0 feat: add shared-code extraction audit (#2925)
* feat: bind shared-code review advice to source and branch * feat: add shared-code extraction audit and scoped review checks * test: recognize complete source reads and explicit coverage legends * chore: bump version and changelog (v1.88.0.0) Co-Authored-By: OpenAI Codex <noreply@openai.com> * test: capture native review questions and retain public evidence Capture the actual first public native question with strict ownership and display matching. Preserve terminal failures and raw evidence, and retain SDK completion checks. * test: recognize verified review evidence and complete fixtures Recognize complete source and diagram evidence, concrete design and developer-experience decisions, and the complete planted scenario contracts. Preserve negative controls and grading thresholds. * fix: preserve decision brief structure in native questions Keep the required pros-and-cons heading and final Net field in native question text. Regenerate host outputs and document the release and evaluation repairs. Co-Authored-By: OpenAI Codex <noreply@openai.com> * docs: update project documentation for v1.88.0.0 Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: correct eval retry accounting and ship workflow gates * fix: capture native eval evidence and stabilize CI fixtures * fix: keep shared-code eval skips read-only Choose explicit no-change answers instead of mixed fix/preservation options. Reuse the bounded revalidation prompt for path fixtures so required review metadata is available without repeated discovery. Preserve source checks, retry limits, and failed native terminal outcomes. Add captured-question and callback regressions, plus evaluation selection coverage for the affected fixtures. --------- Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
co-authored by
OpenAI Codex
parent
b9706f3635
commit
06ed920a97
@@ -1,26 +1,10 @@
|
||||
/**
|
||||
* AskUserQuestion format-compliance gate (gate, paid, SDK capture).
|
||||
* AskUserQuestion format-compliance gate (gate, paid, native SDK tool capture).
|
||||
*
|
||||
* Asserts: /plan-ceo-review's first AskUserQuestion (Step 0F mode selection) is a
|
||||
* compliant decision brief — all 7 mandated format elements present, with a
|
||||
* substantive recommendation.
|
||||
*
|
||||
* Why SDK capture, not real-PTY (changed v1.59+): the prior version launched an
|
||||
* interactive `claude` PTY and grepped the rendered TUI after stripAnsi. But
|
||||
* plan-mode AUQs render as an interactive cursor picker whose cursor-positioning
|
||||
* escapes stripAnsi CANNOT faithfully flatten — verified directly: the picker
|
||||
* renders fine for a human (cursorSeen=45) but the flattened text drops `ELI10:`
|
||||
* and `(recommended)` and `parseNumberedOptions` returns 0. So the old test was
|
||||
* grading a lossy projection of the TUI, not the question's actual format, and
|
||||
* failed by construction in this environment.
|
||||
*
|
||||
* This version drives the skill via the SDK $OUT_FILE capture path (the agent
|
||||
* writes the verbatim AskUserQuestion it would have shown to a file — clean text,
|
||||
* zero rendering loss) and grades that. Same property tested (does the question
|
||||
* carry every format element), reliably, environment-independent. The rendering
|
||||
* layer is identical across skills/content, so it is not where format regressions
|
||||
* hide; the model's composed question is. Shares the engine with the periodic
|
||||
* A/B and matrix evals (test/helpers/auq-sdk-capture.ts).
|
||||
* Drives the real four-mode question through AskUserQuestion, then captures its
|
||||
* exact public permission-callback fields without answering. The 12-turn/240s
|
||||
* capture keeps its original bounds. Grades remain all 7 decision-brief format
|
||||
* elements and recommendation substance >=4; capture is not workflow completion.
|
||||
*/
|
||||
import { test, expect } from 'bun:test';
|
||||
import { CAPTURE_MS } from './helpers/eval-budgets';
|
||||
|
||||
Reference in New Issue
Block a user