mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-26 14:50:55 +02:00
v1.89.0.0 feat: add shared-code extraction audit (#2925)
* feat: bind shared-code review advice to source and branch * feat: add shared-code extraction audit and scoped review checks * test: recognize complete source reads and explicit coverage legends * chore: bump version and changelog (v1.88.0.0) Co-Authored-By: OpenAI Codex <noreply@openai.com> * test: capture native review questions and retain public evidence Capture the actual first public native question with strict ownership and display matching. Preserve terminal failures and raw evidence, and retain SDK completion checks. * test: recognize verified review evidence and complete fixtures Recognize complete source and diagram evidence, concrete design and developer-experience decisions, and the complete planted scenario contracts. Preserve negative controls and grading thresholds. * fix: preserve decision brief structure in native questions Keep the required pros-and-cons heading and final Net field in native question text. Regenerate host outputs and document the release and evaluation repairs. Co-Authored-By: OpenAI Codex <noreply@openai.com> * docs: update project documentation for v1.88.0.0 Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: correct eval retry accounting and ship workflow gates * fix: capture native eval evidence and stabilize CI fixtures * fix: keep shared-code eval skips read-only Choose explicit no-change answers instead of mixed fix/preservation options. Reuse the bounded revalidation prompt for path fixtures so required review metadata is available without repeated discovery. Preserve source checks, retry limits, and failed native terminal outcomes. Add captured-question and callback regressions, plus evaluation selection coverage for the affected fixtures. --------- Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
co-authored by
OpenAI Codex
parent
b9706f3635
commit
06ed920a97
@@ -16,7 +16,7 @@ function generateSpecialistSelection(ctx: TemplateContext): string {
|
||||
const isShip = ctx.skillName === 'ship';
|
||||
const stepSel = isShip ? '9.1' : '4.5';
|
||||
const stepMerge = isShip ? '9.2' : '4.6';
|
||||
const nextStep = isShip ? 'the Fix-First flow (item 4)' : 'Step 5';
|
||||
const nextStep = isShip ? 'Step 9.3 (cross-review dedup)' : 'Step 5';
|
||||
return `## Step ${stepSel}: Review Army — Specialist Dispatch
|
||||
|
||||
### Detect stack and scope
|
||||
@@ -60,7 +60,7 @@ Based on the scope signals above, select which specialists to dispatch.
|
||||
1. **Testing** — read \`${ctx.paths.skillRoot}/review/specialists/testing.md\`
|
||||
2. **Maintainability** — read \`${ctx.paths.skillRoot}/review/specialists/maintainability.md\`
|
||||
|
||||
**If DIFF_LINES < 50:** Skip all specialists. Print: "Small diff ($DIFF_LINES lines) — specialists skipped." Continue to ${nextStep}.
|
||||
**If DIFF_LINES < 50:** Skip all specialists. Print: "Small diff ($DIFF_LINES lines) — specialists skipped." Continue to ${nextStep}. This threshold only gates specialist dispatch; any core shared-code check still runs.
|
||||
|
||||
**Conditional (dispatch if the matching scope signal is true):**
|
||||
3. **Security** — if SCOPE_AUTH=true, OR if SCOPE_BACKEND=true AND DIFF_LINES > 100. Read \`${ctx.paths.skillRoot}/review/specialists/security.md\`
|
||||
@@ -114,7 +114,9 @@ For each finding, output a JSON object on its own line:
|
||||
{\\"severity\\":\\"CRITICAL|INFORMATIONAL\\",\\"confidence\\":N,\\"path\\":\\"file\\",\\"line\\":N,\\"category\\":\\"category\\",\\"summary\\":\\"description\\",\\"fix\\":\\"recommended fix\\",\\"fingerprint\\":\\"path:line:category\\",\\"specialist\\":\\"name\\"}
|
||||
|
||||
Required fields: severity, confidence, path, category, summary, specialist.
|
||||
Optional: line, fix, fingerprint, evidence, test_stub.
|
||||
Optional: line, fix, fingerprint, evidence, test_stub, advisory, evidence_paths, helper_target.
|
||||
|
||||
Optional extraction advice belongs to the core shared-code check; do not duplicate its proposals. Report real defects in duplicated code independently. Preserve advisory metadata when returning structural advice, and never label a demonstrated defect advisory merely because sharing a helper could fix it.
|
||||
|
||||
If you can write a test that would catch this issue, include it in the \`test_stub\` field.
|
||||
Use the detected test framework ({TEST_FW}). Write a minimal skeleton — describe/it/test
|
||||
@@ -139,7 +141,7 @@ function generateFindingsMerge(ctx: TemplateContext): string {
|
||||
const isShip = ctx.skillName === 'ship';
|
||||
const stepMerge = isShip ? '9.2' : '4.6';
|
||||
const stepSel = isShip ? '9.1' : '4.5';
|
||||
const fixFirstRef = isShip ? 'the Fix-First flow (item 4)' : 'Step 5 Fix-First';
|
||||
const fixFirstRef = isShip ? 'Step 9.3 dedup, then Step 9.4 Fix-First' : 'Step 5 Fix-First';
|
||||
const critPassRef = isShip ? 'the checklist pass (Step 9)' : 'the CRITICAL pass findings from Step 4';
|
||||
const persistRef = isShip ? 'the review-log persist' : 'the review-log entry in Step 5.8';
|
||||
return `### Step ${stepMerge}: Collect and merge findings
|
||||
@@ -152,12 +154,17 @@ For each specialist's output:
|
||||
2. Otherwise, parse each line as a JSON object. Skip lines that are not valid JSON.
|
||||
3. Collect all parsed findings into a single list, tagged with their specialist name.
|
||||
|
||||
**Validate advisory severity first.** If a current finding has \`"severity":"CRITICAL"\` and \`"advisory":true\`, remove \`advisory\` and retain its \`CRITICAL\` severity. Handle it as a normal defect before fingerprinting, partitioning, deduplication, counting, scoring, and Fix-First. Never downgrade severity to make advisory metadata consistent. Valid INFORMATIONAL advisories remain advisory in every category, including simplification. Apply this validation to core and specialist findings alike before combining them.
|
||||
|
||||
**Fingerprint and deduplicate:**
|
||||
For each finding, compute its fingerprint:
|
||||
- For a shared-code advisory (category \`shared-libs\` or a \`shared-libs:\` fingerprint), call the installed \`sharedLibsFingerprint\` helper from \`${ctx.paths.skillRoot}/lib/review-evidence.ts\` with literal JSON on stdin, as in the core pass. Recompute from \`evidence_paths\` and \`helper_target\`; never trust a supplied hash or generate hash text yourself. Missing/malformed metadata cannot deduplicate or reuse a saved decision.
|
||||
- If \`fingerprint\` field is present, use it
|
||||
- Otherwise: \`{path}:{line}:{category}\` (if line is present) or \`{path}:{category}\`
|
||||
|
||||
Group findings by fingerprint. For findings sharing the same fingerprint:
|
||||
The last two rules apply only to other findings. Preserve \`advisory\`, \`evidence_paths\`, and \`helper_target\` through merging. Core review owns shared-code proposals: consolidate equivalent specialist advice with the core proposal and count overlapping savings once. Keep the actual specialist activity in its stats; core-only advice must not create a specialist dispatch or finding.
|
||||
|
||||
Partition defects and advisories BEFORE grouping by fingerprint. A defect and an advisory must never merge with each other, even if a supplied fingerprint collides. A higher-confidence advisory or prior skipped extraction cannot replace, downgrade, or suppress a demonstrated defect. For findings sharing the same fingerprint within the same partition:
|
||||
- Keep the finding with the highest confidence score
|
||||
- Tag it: "MULTI-SPECIALIST CONFIRMED ({specialist1} + {specialist2})"
|
||||
- Boost confidence by +1 (cap at 10)
|
||||
@@ -169,11 +176,13 @@ Group findings by fingerprint. For findings sharing the same fingerprint:
|
||||
- Confidence 3-4: move to appendix (suppress from main findings)
|
||||
- Confidence 1-2: suppress entirely
|
||||
|
||||
**Advisory carve-out (simplification specialist):**
|
||||
Findings with \`"advisory": true\` are excluded from BOTH the quality_score
|
||||
**Advisory carve-out (all sources, including core shared-code and simplification):**
|
||||
After severity validation, remaining findings with \`"advisory": true\` are excluded from BOTH the quality_score
|
||||
summation and the findings-count header below — they are structure suggestions,
|
||||
not defects, and must not make "5 findings … 10/10" look contradictory. In
|
||||
Fix-First they are ASK-only: NEVER auto-applied, even when mechanical.
|
||||
Fix-First they are ASK-only: NEVER auto-applied, even when mechanical. Also exclude
|
||||
them from unresolved-defect totals and clean-status blockers. Preserve normal
|
||||
Fix-First handling for any real defect affecting the same code.
|
||||
|
||||
**Compute PR Quality Score:**
|
||||
After merging, compute the quality score over NON-advisory findings only:
|
||||
@@ -203,6 +212,8 @@ PR Quality Score: X/10
|
||||
\`Simplification: lean already — nothing to cut.\`
|
||||
- If it was not dispatched, print neither line.
|
||||
|
||||
Do not add core shared-code savings to this specialist footer. Explain any overlap once in the core proposal instead of presenting duplicate savings.
|
||||
|
||||
These findings flow into ${fixFirstRef} alongside ${critPassRef}.
|
||||
The Fix-First heuristic applies identically — specialist findings follow the same AUTO-FIX vs ASK classification (except advisory findings, which are ASK-only per the carve-out above).
|
||||
|
||||
@@ -215,7 +226,8 @@ For each specialist (testing, maintainability, security, performance, data-migra
|
||||
- If not applicable (e.g., red-team not activated): omit from the object
|
||||
|
||||
Advisory findings COUNT in the stats \`findings\` field — the advisory
|
||||
carve-out governs the quality score and the findings-count header only.
|
||||
carve-out governs defect counts, score penalties, and clean-status blockers,
|
||||
not specialist activity. Count only findings that specialist actually returned.
|
||||
Logging simplification's advisories as \`findings: 0\` would auto-gate the
|
||||
lens into permanent silence after 10 dispatches.
|
||||
|
||||
@@ -226,7 +238,7 @@ Remember these stats — you will need them for ${persistRef}.`;
|
||||
function generateRedTeam(ctx: TemplateContext): string {
|
||||
const isShip = ctx.skillName === 'ship';
|
||||
const stepMerge = isShip ? '9.2' : '4.6';
|
||||
const fixFirstRef = isShip ? 'the Fix-First flow (item 4)' : 'Step 5 Fix-First';
|
||||
const fixFirstRef = isShip ? 'Step 9.3 dedup, then Step 9.4 Fix-First' : 'Step 5 Fix-First';
|
||||
return `### Red Team dispatch (conditional)
|
||||
|
||||
**Activation:** Only if DIFF_LINES > 200 OR any specialist produced a CRITICAL finding.
|
||||
@@ -249,7 +261,7 @@ If the Red Team finds additional issues, merge them into the findings list befor
|
||||
${fixFirstRef}. Red Team findings are tagged with \`"specialist":"red-team"\`.
|
||||
|
||||
If the Red Team returns NO FINDINGS, note: "Red Team review: no additional issues found."
|
||||
If the Red Team subagent fails or times out, skip silently and continue.`;
|
||||
${isShip ? 'If the Red Team subagent fails or times out, continue through dedup and persistence with dispatched coverage incomplete. Step 9.4 must not certify that pass as completed or clean.' : 'If the Red Team subagent fails or times out, skip silently and continue.'}`;
|
||||
}
|
||||
|
||||
export function generateReviewArmy(ctx: TemplateContext): string {
|
||||
|
||||
Reference in New Issue
Block a user