mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-26 14:50:55 +02:00
v1.89.0.0 feat: add shared-code extraction audit (#2925)
* feat: bind shared-code review advice to source and branch * feat: add shared-code extraction audit and scoped review checks * test: recognize complete source reads and explicit coverage legends * chore: bump version and changelog (v1.88.0.0) Co-Authored-By: OpenAI Codex <noreply@openai.com> * test: capture native review questions and retain public evidence Capture the actual first public native question with strict ownership and display matching. Preserve terminal failures and raw evidence, and retain SDK completion checks. * test: recognize verified review evidence and complete fixtures Recognize complete source and diagram evidence, concrete design and developer-experience decisions, and the complete planted scenario contracts. Preserve negative controls and grading thresholds. * fix: preserve decision brief structure in native questions Keep the required pros-and-cons heading and final Net field in native question text. Regenerate host outputs and document the release and evaluation repairs. Co-Authored-By: OpenAI Codex <noreply@openai.com> * docs: update project documentation for v1.88.0.0 Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: correct eval retry accounting and ship workflow gates * fix: capture native eval evidence and stabilize CI fixtures * fix: keep shared-code eval skips read-only Choose explicit no-change answers instead of mixed fix/preservation options. Reuse the bounded revalidation prompt for path fixtures so required review metadata is available without repeated discovery. Preserve source checks, retry limits, and failed native terminal outcomes. Add captured-question and callback regressions, plus evaluation selection coverage for the affected fixtures. --------- Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
co-authored by
OpenAI Codex
parent
b9706f3635
commit
06ed920a97
+142
-13
@@ -134,13 +134,13 @@ Completeness: use `Completeness: N/10` only when options differ in coverage. 10
|
||||
|
||||
Accepted shortcuts leave a trail: when the user selects an option that is BOTH Completeness ≤ 7 AND a durable-scope call (architecture or scope-cut — never a turn-level choice), log it via `gstack-decision-log` with the ceiling and the upgrade trigger in the rationale, and — as part of implementing that option, same edit, no follow-up question — mark each cut corner in code with `gstack-shortcut(dec-<id>): <ceiling>, upgrade when <trigger>` in the language's comment syntax. Never agent-initiated: the marker exists only downstream of the user's explicit choice. /retro harvests these into a debt ledger, joined on the decision id.
|
||||
|
||||
Pros / cons: use ✅ and ❌. Minimum 2 pros and 1 con per option when the choice is real; Minimum 40 characters per bullet. Hard-stop escape for one-way/destructive confirmations: `✅ No cons — this is a hard-stop choice`.
|
||||
`Pros / cons:` in question text; descriptions use literal ✅/❌ bullets, not Pro:/Con:. Each real option: ≥2 pros and ≥1 con, ≥40 chars each. One-way/destructive escape: `✅ No cons — this is a hard-stop choice`.
|
||||
|
||||
Neutral posture: `Recommendation: <default> — this is a taste call, no strong preference either way`; `(recommended)` STAYS on the default option for AUTO_DECIDE.
|
||||
|
||||
Effort both-scales: when an option involves effort, label both human-team and CC+gstack time, e.g. `(human: ~2 days / CC: ~15 min)`. Makes AI compression visible at decision time.
|
||||
|
||||
Net line closes the tradeoff. Per-skill instructions may add stricter rules.
|
||||
`Net:` line closes question text. Per-skill instructions may add stricter rules.
|
||||
|
||||
### Handling 5+ options — split, never drop
|
||||
|
||||
@@ -172,10 +172,10 @@ Before calling AskUserQuestion, verify:
|
||||
- [ ] ELI10 paragraph present (stakes line too)
|
||||
- [ ] Recommendation line present with concrete reason
|
||||
- [ ] Completeness scored (coverage) OR kind-note present (kind)
|
||||
- [ ] Every option has ≥2 ✅ and ≥1 ❌, each ≥40 chars (or hard-stop escape)
|
||||
- [ ] `Pros / cons:` in question; options: ≥2 ✅, ≥1 ❌, ≥40 chars/bullet (or escape)
|
||||
- [ ] (recommended) label on one option (even for neutral-posture)
|
||||
- [ ] Dual-scale effort labels on effort-bearing options (human / CC)
|
||||
- [ ] Net line closes the decision
|
||||
- [ ] `Net:` closes question text
|
||||
- [ ] You are calling the tool, not writing prose — unless `CONDUCTOR_SESSION: true` (then prose is the DEFAULT, not the tool) OR the documented failure fallback applies (then: the prose fallback's mandatory triad + a "reply with a letter" instruction, then STOP); in `SESSION_KIND: spawned` (the echoed STATUS line only) you should never reach this checklist — auto-choose the recommended option, no tool call, no prose
|
||||
- [ ] Non-ASCII characters (CJK / accents) written directly, NOT \u-escaped
|
||||
- [ ] If you had 5+ options, you split (or batched into ≤4-groups) — did NOT drop any
|
||||
@@ -681,7 +681,7 @@ SQL & Data Safety, Race Conditions & Concurrency, LLM Output Trust Boundary, She
|
||||
|
||||
Also apply the remaining INFORMATIONAL categories that are still in the checklist (Async/Sync Mixing, Column/Field Name Safety, LLM Prompt Issues, Type Coercion, View/Frontend, Time Window Safety, Completeness Gaps, Distribution & CI/CD).
|
||||
|
||||
**Enum & Value Completeness requires reading code OUTSIDE the diff.** When the diff introduces a new enum value, status, tier, or type constant, use Grep to find all files that reference sibling values, then Read those files to check if the new value is handled. This is the one category where within-diff review is insufficient.
|
||||
**Enum & Value Completeness requires reading code OUTSIDE the diff.** When the diff introduces a new enum value, status, tier, or type constant, use Grep to find all files that reference sibling values, then Read those files to check if the new value is handled. Shared-code analysis also requires reading related callers outside the diff; keep findings anchored to changed code.
|
||||
|
||||
**Search-before-recommending:** When recommending a fix pattern (especially for concurrency, caching, auth, or framework-specific behavior), research through Aside (Web research runs in Aside, above):
|
||||
- Verify the pattern is current best practice for the framework version in use
|
||||
@@ -697,6 +697,52 @@ Takes seconds, prevents recommending outdated patterns. If the Aside check did n
|
||||
|
||||
Follow the output format specified in the checklist. Respect the suppressions — do NOT flag items listed in the "DO NOT flag" section.
|
||||
|
||||
### Shared-code opportunities (core pass)
|
||||
|
||||
Run this check on every diff, including fewer than 50 changed lines and hosts without Review Army. Review the changed code and related unchanged callers using the shared rubric below. Do not run the standalone history/PR sweep or impose candidate quotas. At least one verified authored location must be changed in this diff, and at least two actual authored source locations must need the shared behavior; added or uncommitted source qualifies, invented future callers do not. Trace generated copies to their authored templates/resolvers and exclude generated and third-party copies from evidence and savings.
|
||||
|
||||
### Shared-code evaluation rubric
|
||||
|
||||
- **Prove the callers.** Require at least two verified, first-party authored source
|
||||
locations, with functions and lines. Actual added or uncommitted source qualifies.
|
||||
Only an engineering-plan review may use proposed callers; label those assumptions
|
||||
and distinguish them from existing source. Similar names or formatting alone do
|
||||
not establish equivalent behavior. Generated and third-party copies cannot qualify
|
||||
as callers or contribute savings. Follow generated copies back to authored
|
||||
templates/resolvers. Existing dependencies remain valid reuse targets.
|
||||
- **Reuse before extracting.** Inspect existing libraries and helpers first. Compare
|
||||
behavior, inputs, outputs, error handling, side effects, security requirements,
|
||||
dependencies, and deployment/runtime boundaries. Preserve differences callers need;
|
||||
do not bridge languages or isolated deployments without a practical shared contract.
|
||||
- **Keep the helper small.** Name its destination and contract, the callers to migrate,
|
||||
and the smallest adoption sequence. Avoid option-heavy helpers and coupling unrelated
|
||||
components. Point to existing tests or established use, specify shared-contract and
|
||||
caller-integration coverage, and describe the blast radius of a shared failure.
|
||||
- **Account for the whole change.** Name removed blocks and their replacements. Show
|
||||
estimated implementation lines removed, added, and saved separately from total lines
|
||||
removed, added, and saved including tests and integration. Savings = removed - added.
|
||||
Count moved code on both sides, exclude generated/vendor lines, use ranges when
|
||||
uncertain, and do not count overlapping removals twice across opportunities. State
|
||||
when tests or integration may make the total change grow.
|
||||
- **Rank useful changes.** Favor reliability gains and total net savings, then low
|
||||
adoption and testing risk. Prefer proven code used by several callers. Use recent
|
||||
activity to break ties between comparable benefits, not as evidence by itself.
|
||||
Explain choices centered on older code. Reject similarities with incompatible
|
||||
contracts and opportunities whose benefits do not justify the abstraction.
|
||||
|
||||
The core pass owns optional extraction advice. Present only worthwhile, supported proposals; zero is valid. For each proposal, show the changed anchor and other verified callers, smallest helper/destination, preserved differences, compatibility tests, shared-failure risk, and estimated implementation and total removed/added/saved lines from named blocks. Use `"category":"shared-libs","severity":"INFORMATIONAL","advisory":true`, retain `evidence_paths` (all authored supporting paths) and `helper_target:{"path":"...","symbol":"..."}`. When reusing an existing helper, include its authored path in `evidence_paths` so its contract and raw bytes participate in revalidation; a not-yet-created helper belongs only in `helper_target`. Deduplicate equivalent proposals and overlapping savings. Existing-helper reuse is preferable when compatible.
|
||||
|
||||
**Identity before merge or suppression:** Compute the structural fingerprint through the installed `sharedLibsFingerprint` helper, never write model-generated hash text. Feed the finding as literal JSON on stdin (replace the example values; keep the quoted delimiter), not interpolated shell code:
|
||||
|
||||
```bash
|
||||
GSTACK_SHARED_LIB=~/.claude/skills/gstack/lib/review-evidence.ts
|
||||
bun -e 'const { sharedLibsFingerprint } = await import(process.argv[1]); const value = sharedLibsFingerprint(JSON.parse(await Bun.stdin.text())); if (!value) process.exit(1); console.log(value);' "$GSTACK_SHARED_LIB" <<'GSTACK_SHARED_LIBS_JSON'
|
||||
{"evidence_paths":["src/caller-a.ts","src/caller-b.ts"],"helper_target":{"path":"src/shared.ts","symbol":"sharedHelper"}}
|
||||
GSTACK_SHARED_LIBS_JSON
|
||||
```
|
||||
|
||||
Use the returned fingerprint; malformed/missing metadata has no reusable identity and must be revalidated. A real defect in the same code remains a normal defect with its own evidence and Fix-First handling. An optional extraction must never suppress, downgrade, or replace that defect, even if they share a supplied fingerprint or an extraction was previously skipped.
|
||||
|
||||
## Confidence Calibration
|
||||
|
||||
Every finding MUST include a confidence score (1-10):
|
||||
@@ -771,8 +817,14 @@ higher confidence.
|
||||
|
||||
**Every finding gets action — not just critical ones.**
|
||||
|
||||
**Keep decisions through fix cycles.** Maintain an in-memory action list for this invocation, initialized once and retained when Steps 3–5.7 repeat. Keep defects and advisories separate; for shared-code advice retain the helper-computed fingerprint, `advisory`, `evidence_paths`, and `helper_target` from the actual decision. Record completed AUTO-FIX/fix actions and explicit Skip choices as they happen. A later zero-edit pass may no longer find an approved extraction because it succeeded; that must not erase its `fixed` action or original identity metadata.
|
||||
|
||||
On each repeat pass, re-read all supporting callers and the helper destination before carrying an advisory decision forward. An unrelated auto-fix does not require asking the same question again when the structural identity, proposed contract, and tradeoffs remain unchanged. Compare actual raw source with the evidence read for the decision, including secondary callers and any transformed or indirect paths; changed evidence requires fresh evaluation. If the proposal, behavior, migration, or risk has materially changed, ask a new question instead of inheriting the choice. This invocation-local decision tracking is not cross-review suppression and must never hide a new or recurring defect.
|
||||
|
||||
### Step 5.0: Cross-review finding dedup
|
||||
|
||||
**Validate advisory severity first.** If a current finding has `"severity":"CRITICAL"` and `"advisory":true`, remove `advisory` and retain its `CRITICAL` severity. Handle it as a normal defect before suppression, classification, counting, scoring, and persistence. Never downgrade severity to make advisory metadata consistent. Valid INFORMATIONAL advisories remain advisory in every category, including simplification. A prior saved finding with contradictory CRITICAL/advisory metadata cannot establish a skipped defect or advisory decision: exclude it from reuse and revalidate the current finding.
|
||||
|
||||
Before classifying findings, check if any were previously skipped by the user in a prior review on this branch.
|
||||
|
||||
```bash
|
||||
@@ -781,7 +833,12 @@ Before classifying findings, check if any were previously skipped by the user in
|
||||
|
||||
Parse the output: only lines BEFORE `---CONFIG---` are JSONL entries (the output also contains `---CONFIG---` and `---HEAD---` footer sections that are not JSONL — ignore those).
|
||||
|
||||
For each JSONL entry that has a `findings` array:
|
||||
**Shared-code advisory decisions use the stricter rule below.** Do not send a
|
||||
finding through the ordinary primary-file rule if its category is `shared-libs`,
|
||||
its fingerprint starts `shared-libs:`, or it has `evidence_paths` / `helper_target`.
|
||||
Missing legacy metadata requires revalidation, not fallback to a line fingerprint.
|
||||
|
||||
For each JSONL entry that has a `findings` array, for ordinary findings only:
|
||||
1. Collect all fingerprints where `action: "skipped"`
|
||||
2. Note the `commit` field from that entry
|
||||
|
||||
@@ -794,8 +851,69 @@ git diff --name-only <prior-review-commit> HEAD
|
||||
For each current finding (from both Step 4 critical pass and Step 4.5-4.6 specialists), check:
|
||||
- Does its fingerprint match a previously skipped finding?
|
||||
- Is the finding's file path NOT in the changed-files set?
|
||||
- Is it the same advisory/defect kind? Never use a skipped advisory to suppress a real defect, including a defect with a colliding supplied fingerprint.
|
||||
|
||||
If both conditions are true: suppress the finding. It was intentionally skipped and the relevant code hasn't changed.
|
||||
If all conditions are true: suppress the finding. It was intentionally skipped and the relevant code hasn't changed.
|
||||
|
||||
**Reuse a skipped shared-code advisory only with complete structural evidence:**
|
||||
|
||||
1. Recompute both structural identities with `sharedLibsFingerprint` from
|
||||
`~/.claude/skills/gstack/lib/review-evidence.ts` before deduplication. Both must
|
||||
be valid, both findings must explicitly be advisory, the prior saved hash must
|
||||
match its recomputation, and the prior action must explicitly be `skipped`.
|
||||
Retain `evidence_paths` and `helper_target`; line numbers and a primary path
|
||||
alone cannot identify an extraction.
|
||||
2. Require a prior completed, converged `review` with verified binding and
|
||||
start/end/record fingerprints equal to current `---WTREE---`. Read REVIEW_START
|
||||
without consuming it; its repo, raw branch and fingerprint must match the current
|
||||
repo, branch and snapshot. Missing, changed or unknown fields/token require
|
||||
revalidation. Do not mint a new token to enable suppression.
|
||||
3. Match prior trusted `review_binding.branch_id` to SHA-256 of the exact
|
||||
current raw branch, matching the capture. Compute the digest in code, never
|
||||
as model-generated text. Sanitized log filenames are not branch identity:
|
||||
`topic/a` and `topic-a` can collide.
|
||||
4. Verify EVERY evidence path against the snapshot. Enumerate tracked/non-ignored
|
||||
untracked paths, then raw-read/lstat each file and path component; `ls-files`
|
||||
alone is insufficient. Revalidate symlink targets/ancestors, submodules,
|
||||
ignored/outside files and missing/unreadable paths: the parent fingerprint
|
||||
does not cover them. Inspect effective Git attributes/config without conversion:
|
||||
filter, working-tree-encoding, ident, text/eol and core.autocrlf can hide raw
|
||||
changes. Active/unknown transformations require fresh raw-source review even
|
||||
with an unchanged filtered tree. Disable fsmonitor and optional locks.
|
||||
Exclude assume-unchanged, skip-worktree and sparse index entries. Compare each
|
||||
raw file byte-for-byte with its blob in that exact working-tree snapshot,
|
||||
using Git object reads without external diff/textconv or normalization.
|
||||
Missing blobs, mismatches or unknown coverage require revalidation.
|
||||
Only verified regular, untransformed,
|
||||
in-repository paths enter `covered_paths`.
|
||||
The prior finding's `snapshot_covered_paths` must also cover every evidence
|
||||
path; current eligibility cannot prove what prior filters/index flags hid.
|
||||
Missing prior coverage is legacy metadata; revalidate it.
|
||||
5. Call pure `canReuseSharedLibsAdvisory` with actually read records and verified
|
||||
snapshot fields as literal JSON on stdin. The command below computes the live branch digest;
|
||||
replace the empty example objects and keep the quoted delimiter:
|
||||
|
||||
```bash
|
||||
bun -e '
|
||||
const { createHash } = await import("node:crypto");
|
||||
const { canReuseSharedLibsAdvisory } = await import(process.argv[1]);
|
||||
const input = JSON.parse(await Bun.stdin.text());
|
||||
let branch = Bun.spawnSync(["git", "symbolic-ref", "--quiet", "--short", "HEAD"]);
|
||||
if (branch.exitCode !== 0) branch = Bun.spawnSync(["git", "rev-parse", "HEAD"]);
|
||||
if (branch.exitCode !== 0) { console.log(false); process.exit(0); }
|
||||
const rawBranch = branch.stdout.toString().replace(/\r?\n$/, "");
|
||||
const snapshot = { ...input.currentSnapshot, branch_id: createHash("sha256").update(rawBranch, "utf8").digest("hex") };
|
||||
console.log(canReuseSharedLibsAdvisory(input.priorFinding, input.currentFinding, input.priorReview, snapshot));
|
||||
' "$HOME/.claude/skills/gstack/lib/review-evidence.ts" <<'GSTACK_SHARED_LIBS_REUSE_JSON'
|
||||
{"priorFinding":{},"currentFinding":{},"priorReview":{},"currentSnapshot":{"wtree":"","covered_paths":[]}}
|
||||
GSTACK_SHARED_LIBS_REUSE_JSON
|
||||
```
|
||||
|
||||
Suppress only when ALL eligibility checks passed and the helper returns true.
|
||||
Otherwise re-read all supporting callers and present any still-supported advice
|
||||
for a fresh decision. A changed secondary caller or changed raw bytes matter even
|
||||
when the primary anchor, commit, or normalized Git tree appears unchanged. A real
|
||||
defect always retains normal Fix-First handling independently of this advice.
|
||||
|
||||
Print: "Suppressed N findings from prior reviews (previously skipped by user)"
|
||||
|
||||
@@ -803,7 +921,12 @@ Print: "Suppressed N findings from prior reviews (previously skipped by user)"
|
||||
|
||||
If no prior reviews exist or none have a `findings` array, skip this step silently.
|
||||
|
||||
Output a summary header: `Pre-Landing Review: N issues (X critical, Y informational)`
|
||||
Output a summary header: `Pre-Landing Review: N issues (X critical, Y informational)`.
|
||||
Count only non-advisory defects in that header; list optional advice separately
|
||||
with `[ADVISORY]`. Preserve advisory records and explicit decisions for
|
||||
persistence, but exclude advisories from score penalties, unresolved-defect
|
||||
totals, and clean-status blockers. This does not relax completion, convergence,
|
||||
or missing-reviewer rules.
|
||||
|
||||
### Step 5a: Classify each finding
|
||||
|
||||
@@ -811,6 +934,8 @@ For each finding, classify as AUTO-FIX or ASK per the Fix-First Heuristic in
|
||||
checklist.md. Critical findings lean toward ASK; informational findings lean
|
||||
toward AUTO-FIX.
|
||||
|
||||
**Advisory override:** After the severity validation above, every remaining finding with `advisory:true`, including core shared-code advice, is ASK-only even when mechanical. Never auto-apply an optional extraction. Label it `[ADVISORY]`, show the helper, caller migration, tests, and estimated total savings, and let the user approve or skip it. Advisories are excluded from defect counts, score penalties, unresolved-defect totals, and clean-status blockers. A real defect still follows ordinary Fix-First independently of advice touching the same code.
|
||||
|
||||
**Test stub override:** Any finding that has a `test_stub` field (generated by a specialist)
|
||||
is reclassified as ASK regardless of its original classification. When presenting the ASK
|
||||
item, show the proposed test file path and the test code. The user approves or skips the
|
||||
@@ -823,12 +948,13 @@ already exists, append the new test. Output: `[FIXED + TEST] [file:line] Problem
|
||||
|
||||
Apply each fix directly. For each one, output a one-line summary:
|
||||
`[AUTO-FIXED] [file:line] Problem → what you did`
|
||||
Retain the completed action in the invocation action list before starting any re-review.
|
||||
|
||||
### Step 5c: Batch-ask about ASK items
|
||||
|
||||
If there are ASK items remaining, present them in ONE AskUserQuestion:
|
||||
|
||||
- List each item with a number, the severity label, the problem, and a recommended fix
|
||||
- List each item with a number, the severity label (or `[ADVISORY]` for optional advice), the problem, and a recommended fix
|
||||
- For each item, provide options: A) Fix as recommended, B) Skip
|
||||
- Include an overall RECOMMENDATION
|
||||
|
||||
@@ -848,10 +974,12 @@ RECOMMENDATION: Fix both — #1 is a real race condition, #2 prevents silent dat
|
||||
```
|
||||
|
||||
If 3 or fewer ASK items, you may use individual AskUserQuestion calls instead of batching.
|
||||
Retain each explicit Skip choice and its finding metadata in the invocation action list. Do not record an unanswered question as skipped or ask again about a decision already revalidated in this invocation.
|
||||
|
||||
### Step 5d: Apply user-approved fixes
|
||||
|
||||
Apply fixes for items where the user chose "Fix." Output what was fixed.
|
||||
After applying the approved fix, retain its `fixed` action and the original finding metadata in the invocation action list, even if the changed blocks or helper callers are subsequently removed. Approval alone is not a completed fix.
|
||||
|
||||
If no ASK items exist (everything was AUTO-FIX), skip the question entirely.
|
||||
|
||||
@@ -937,10 +1065,10 @@ Run:
|
||||
|
||||
Substitute:
|
||||
- `TIMESTAMP` = ISO 8601 datetime
|
||||
- `STATUS` = `"clean"` if there are no remaining unresolved findings after Fix-First handling and adversarial review, otherwise `"issues_found"`
|
||||
- `issues_found` = total remaining unresolved findings
|
||||
- `critical` = remaining unresolved critical findings
|
||||
- `informational` = remaining unresolved informational findings
|
||||
- `STATUS` = `"clean"` if there are no remaining unresolved non-advisory defects after Fix-First handling and adversarial review, otherwise `"issues_found"`. Unapproved or skipped advisories never block clean status; incomplete or nonconverged coverage remains governed by the completion rules.
|
||||
- `issues_found` = total remaining unresolved non-advisory defects
|
||||
- `critical` = remaining unresolved non-advisory critical defects
|
||||
- `informational` = remaining unresolved non-advisory informational defects
|
||||
- `quality_score` = the PR Quality Score computed in Step 4.6 (e.g., 7.5). If specialists were skipped (small diff), use `10.0`
|
||||
- `COMMIT` = output of `git rev-parse --short HEAD`
|
||||
|
||||
@@ -977,4 +1105,5 @@ If the review exits early before a real review completes (for example, no diff a
|
||||
- **Fix-first, not read-only.** AUTO-FIX items are applied directly. ASK items are only applied after user approval. Never commit, push, or create PRs — that's /ship's job.
|
||||
- **Be terse.** One line problem, one line fix. No preamble.
|
||||
- **Only flag real problems.** Skip anything that's fine.
|
||||
- **Optional extractions stay advisory.** Shared-code opportunities need verified callers and useful reliability or total savings; similarity alone is not a defect. Keep actual defects independently actionable.
|
||||
- **Use Greptile reply templates from greptile-triage.md.** Every reply includes evidence. Never post vague replies.
|
||||
|
||||
Reference in New Issue
Block a user