mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-27 15:11:47 +02:00
v1.89.0.0 feat: add shared-code extraction audit (#2925)
* feat: bind shared-code review advice to source and branch * feat: add shared-code extraction audit and scoped review checks * test: recognize complete source reads and explicit coverage legends * chore: bump version and changelog (v1.88.0.0) Co-Authored-By: OpenAI Codex <noreply@openai.com> * test: capture native review questions and retain public evidence Capture the actual first public native question with strict ownership and display matching. Preserve terminal failures and raw evidence, and retain SDK completion checks. * test: recognize verified review evidence and complete fixtures Recognize complete source and diagram evidence, concrete design and developer-experience decisions, and the complete planted scenario contracts. Preserve negative controls and grading thresholds. * fix: preserve decision brief structure in native questions Keep the required pros-and-cons heading and final Net field in native question text. Regenerate host outputs and document the release and evaluation repairs. Co-Authored-By: OpenAI Codex <noreply@openai.com> * docs: update project documentation for v1.88.0.0 Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: correct eval retry accounting and ship workflow gates * fix: capture native eval evidence and stabilize CI fixtures * fix: keep shared-code eval skips read-only Choose explicit no-change answers instead of mixed fix/preservation options. Reuse the bounded revalidation prompt for path fixtures so required review metadata is available without repeated discovery. Preserve source checks, retry limits, and failed native terminal outcomes. Add captured-question and callback regressions, plus evaluation selection coverage for the affected fixtures. --------- Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
co-authored by
OpenAI Codex
parent
b9706f3635
commit
06ed920a97
@@ -46,10 +46,41 @@ Autoplan resolves each review skill from its own installed host registry.
|
||||
existing xterm dependency interprets cursor moves and erases; old menus in the
|
||||
raw stream cannot establish a current prompt. Snapshots preserve
|
||||
`terminal.raw.log`, `terminal.visible.log`, and `terminal.screen.log` separately.
|
||||
Setting `EVALS_RUN_ID` or `GSTACK_EVAL_DIR` retains these snapshots; an output
|
||||
directory without a run ID gets a stable, unique local ID for that writer.
|
||||
Completed native transcript calls establish question counts and phase coverage.
|
||||
Report-aware count tests also require a fresh, complete report and native
|
||||
completion evidence before accepting a completion heading.
|
||||
|
||||
The periodic first-question matrix uses `test/helpers/auq-native-capture.ts`
|
||||
to match the first public `PreToolUse` AskUserQuestion payload to its current
|
||||
native display. It grades that question's exact public fields without answering
|
||||
it or reading model transcripts. `question_captured` records
|
||||
`workflowCompleted: false`. With `GSTACK_EVAL_DIR`, `EVALS_RUN_ID`, or an explicit
|
||||
run ID, `native-auq/<run-id>/<test>-<suffix>/capture.json` under the eval directory
|
||||
retains the public payload, bounded current viewport, and capture outcome.
|
||||
CEO mode selection uses the actual SDK `AskUserQuestion` permission callback
|
||||
in `auq-sdk-capture.ts`, with the existing 12-turn and 240-second limits. It
|
||||
captures the public question and stops without submitting an answer; its
|
||||
`question_captured` outcome also records `workflowCompleted: false`. The retained
|
||||
capture survives fixture cleanup. Provider refusals and malformed questions
|
||||
remain failures. Section-loading captures retain their noninteractive contract.
|
||||
|
||||
Shared-code revalidation fixtures pair public tool calls with their successful
|
||||
results to verify that the current trusted start record was inspected before
|
||||
completion. A discovered path in tool output counts; a path mentioned only in
|
||||
instructions or narration does not. Saved public captures cover absolute and
|
||||
relative paths and discovery followed by a read. The revalidation prompt supplies
|
||||
the path to the trusted start-record directory and declares the existing turn
|
||||
limit. It asks the agent to batch independent reads and retrieve the complete
|
||||
final record; every source, approval, persistence, and completion check still applies. The
|
||||
path-boundary fixtures use this same execution contract for symlinks, submodules,
|
||||
ignored files, index flags, and legacy or filtered evidence. Their skip actor
|
||||
accepts an explicit no-change choice; a preservation word inside an option that
|
||||
also approves changes cannot authorize edits. Captured native questions exercise
|
||||
the actual answer callback, and native turn-limit failures still fail even after
|
||||
a question was answered.
|
||||
|
||||
The engineering and DX finding fixtures check coverage of their seeded issues
|
||||
rather than cap the total number of review questions. Each decision needs a
|
||||
distinct, completed native question with an offered answer; accepting, rejecting,
|
||||
@@ -159,8 +190,8 @@ board actor submits feedback before acknowledging it. The final proof uses
|
||||
the full native question, not the truncated diagnostic snippet, and proposal
|
||||
text mentioning "no UI scope" is not treated as an exit verdict. Unknown-command
|
||||
failures must name the invoked slash command; a child tool rejecting `--help`
|
||||
is not a skill registration failure. Periodic seeded-finding classifiers are
|
||||
unchanged.
|
||||
is not a skill registration failure. Periodic seeded-finding classifiers
|
||||
separately verify fixture-owned findings.
|
||||
|
||||
**Paid suite (sharded runner, local AND CI).** `scripts/test-paid-shards.ts`
|
||||
is the single selection engine: 1 file per shard, `EVALS_JOBS` shard
|
||||
|
||||
Reference in New Issue
Block a user