mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-05 19:07:12 +02:00
v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)
* feat: add surface-aware exploratory QA and ship documentation gates * test: preserve delegated QA setup authority after main integration * fix(qa): clarify exploration order and preserve report artifacts * test(qa): follow the shared setup reference directly * refactor(ship): make verification and recovery routes explicit * test(ship): align evidence and review guards with explicit routes * fix(workflows): clarify ship recovery and functional QA evidence * fix(workflows): clarify approval recovery and full QA coverage * refactor(workflows): order review transactions and clarify ship state * fix(ship): clarify final verification and fail closed at publication * fix(evals): attribute native atomic documentation writes * fix(ship): clarify recovery and documentation lifecycle guidance * fix(test): preserve observed native placeholder styling in CI * fix(codex): report watchdog timeouts without a process-exit race * Checkpoint functional QA implementation and workflow validation repairs * Fix documentation and shared-review fixture contracts * docs: clarify judge reuse and evaluation supervision * test: align review evidence and selected case contracts * test: verify append-only documentation checkpoints and recovery * fix: qualify QA workflows and CI validation repairs * fix: launch shared-libs fixture scripts on Windows * fix: qualify QA deadlines, fixture isolation, and shard cleanup * fix: preserve qualified QA and cancellation repairs * fix: enforce functional fixture authority and share strict event decoding * fix: retain free-test evidence and explain recovery * fix: reject malformed native evidence after decoder consolidation * test: use reliable capture for telemetry privacy filters * test: refresh measured quick coverage and document validation costs * Fix native fixture receipts and preserve VM validation evidence * Align negative judge controls with upstream clarity policy * Fix report-only QA preparation and public evidence handling * Clarify QA-only preparation and current-report preservation * Stream Ship quality judgments with an explicit 64k response contract * Validate compact judge reasoning locally with supported wire schema * Align functional QA fixture instructions with evidence acceptance * Bind native browser diagnostics to execution evidence and align review verdicts * Preserve native diagnostic line boundaries * Serialize functional QA evidence from native captures * Keep large QA evidence fixture payload out of Windows argv
This commit is contained in:
1 parent
65bfb0ce49
commit
dcaea52800
333 files changed
+41755
-7357
No files matched your search
@@ -1,21 +1,19 @@
|
||||
<!-- AUTO-GENERATED from plan-completion.md.tmpl — do not edit directly -->
|
||||
<!-- Regenerate: bun run gen:skill-docs -->
|
||||
This is the deep pass behind Step 1.5's scope-drift check: discover the plan file, extract its actionable items, classify how each can be verified, and cross-reference them against the diff. Like Step 1.5 itself, the audit is INFORMATIONAL — it never blocks the review.
|
||||
This is Step 1.5's plan-completion audit: discover the plan, extract actionable items, classify their verification and compare with the diff. It is INFORMATIONAL except for the HIGH-impact discrepancy question below; resolve that gate before the final Scope Check.
|
||||
|
||||
### Plan File Discovery
|
||||
|
||||
1. **Conversation context (primary):** Check if there is an active plan file in this conversation. The host agent's system messages include plan file paths when in plan mode. If found, use it directly — this is the most reliable signal.
|
||||
1. **Conversation context (primary):** Use the active plan file from this conversation or its plan-mode system context.
|
||||
|
||||
2. **Content-based search (fallback):** If no plan file is referenced in conversation context, search by content:
|
||||
2. **Content-based search (fallback):** Without a conversation-supplied path, search by content:
|
||||
|
||||
```bash
|
||||
setopt +o nomatch 2>/dev/null || true # zsh compat
|
||||
BRANCH=$(git branch --show-current 2>/dev/null | tr '/' '-' | tr -cd 'a-zA-Z0-9._-')
|
||||
REPO=$(basename "$(git rev-parse --show-toplevel 2>/dev/null)")
|
||||
# Compute project slug for ~/.gstack/projects/ lookup
|
||||
_PLAN_SLUG=$(git remote get-url origin 2>/dev/null | sed 's|.*[:/]\([^/]*/[^/]*\)\.git$|\1|;s|.*[:/]\([^/]*/[^/]*\)$|\1|' | tr '/' '-' | tr -cd 'a-zA-Z0-9._-') || true
|
||||
_PLAN_SLUG="${_PLAN_SLUG:-$(basename "$PWD" | tr -cd 'a-zA-Z0-9._-')}"
|
||||
# Search common plan file locations (project designs first, then personal/local)
|
||||
for PLAN_DIR in "$HOME/.gstack/projects/$_PLAN_SLUG" "$HOME/.claude/plans" "$HOME/.codex/plans" ".gstack/plans"; do
|
||||
[ -d "$PLAN_DIR" ] || continue
|
||||
PLAN=$(ls -t "$PLAN_DIR"/*.md 2>/dev/null | xargs grep -l "$BRANCH" 2>/dev/null | head -1)
|
||||
@@ -26,7 +24,7 @@ done
|
||||
[ -n "$PLAN" ] && echo "PLAN_FILE: $PLAN" || echo "NO_PLAN_FILE"
|
||||
```
|
||||
|
||||
3. **Validation:** If a plan file was found via content-based search (not conversation context), read the first 20 lines and verify it is relevant to the current branch's work. If it appears to be from a different project or feature, treat as "no plan file found."
|
||||
3. **Validation:** For search results, read the first 20 lines and verify the project, feature and current branch. A mismatch means "no plan file found." Conversation-supplied paths bypass this search-result check.
|
||||
|
||||
**Error handling:**
|
||||
- No plan file found → skip with "No plan file detected — skipping."
|
||||
@@ -34,7 +32,14 @@ done
|
||||
|
||||
### Actionable Item Extraction
|
||||
|
||||
Read the plan file. Extract every actionable item — anything that describes work to be done. Look for:
|
||||
**Separate static audit evidence from behavioral checks.** Read the plan and keep two lists:
|
||||
- Deliverables and test-creation work: audit these below.
|
||||
- Commands/assertions that exercise behavior: retain the exact command, expected outcome
|
||||
and source for Step 4.7's required plan checks. They remain pending execution, never DONE
|
||||
from a diff. A mixed item contributes to both lists. Zero audited deliverables do not waive these checks.
|
||||
Keep external-state and human-only checks under the existing audit rules.
|
||||
|
||||
Extract every actionable item into the appropriate list. Look for:
|
||||
|
||||
- **Checkbox items:** `- [ ] ...` or `- [x] ...`
|
||||
- **Numbered steps** under implementation headings: "1. Create ...", "2. Add ...", "3. Modify ..."
|
||||
@@ -52,7 +57,7 @@ Read the plan file. Extract every actionable item — anything that describes wo
|
||||
|
||||
**Cap:** Extract at most 50 items. If the plan has more, note: "Showing top 50 of N plan items — full list in plan file."
|
||||
|
||||
**No items found:** If the plan contains no extractable actionable items, skip with: "Plan file contains no actionable items — skipping completion audit."
|
||||
**No items found:** If both lists are empty, skip the completion audit. If only behavioral checks remain, report zero audited deliverables and retain their pending Step 4.7 list.
|
||||
|
||||
For each item, note:
|
||||
- The item text (verbatim or concise summary)
|
||||
@@ -60,7 +65,7 @@ For each item, note:
|
||||
|
||||
### Verification Mode
|
||||
|
||||
Before judging completion, classify HOW each item can be verified. The diff alone cannot prove every kind of work. Items outside the current repo or system are structurally invisible to `git diff`.
|
||||
Classify how each item can be verified. The diff cannot prove work in another repo or external system.
|
||||
|
||||
- **DIFF-VERIFIABLE** — A code change in this repo would manifest in `git diff <base>...HEAD`. Examples: "add UserService" (file appears), "validate input X" (validation logic appears), "create users table" (migration file appears).
|
||||
- **CROSS-REPO** — Item names a file or change in a sibling repo (e.g., `domain-hq/docs/dashboard.md`, `~/Development/<other-repo>/...`). The current diff CANNOT prove this.
|
||||
@@ -76,7 +81,10 @@ Before judging completion, classify HOW each item can be verified. The diff alon
|
||||
|
||||
**Path concreteness rule.** If a plan item names a *concrete filesystem path* (absolute, `~/...`, or `<sibling-repo>/<file>`), it MUST be classified DONE or NOT DONE based on `[ -f <path> ]`. UNVERIFIABLE is only valid when the path is genuinely abstract ("Cloudflare DNS", "Supabase allowlist") or the sibling root is unreachable on this machine. "I don't want to check" is not unreachable.
|
||||
|
||||
**Validator detection.** Before falling back to UNVERIFIABLE on a CONTENT-SHAPE item, scan the target repo's `package.json` for any script matching `validate-*`, `lint-wiki`, `check-docs`, or similar. If found, invoke it with the relevant path argument (e.g., `npm run validate-wiki -- <path>`). For multi-target validators (e.g., `validate-wiki --all`), run once and reconcile per-item from the output. A passing validator promotes the item from UNVERIFIABLE to DONE; a failing one demotes to NOT DONE.
|
||||
**Validator detection.** Before falling back to UNVERIFIABLE on a CONTENT-SHAPE item, scan the target repo's `package.json` for any script matching `validate-*`, `lint-wiki`, `check-docs`, or similar. File-existence checks and verified read-only content validators are static audit checks, not behavioral probes.
|
||||
Inspect the validator and its hooks before running it; verify read-only effects and access to the target.
|
||||
If that cannot be established, leave the item UNVERIFIABLE and defer the command to Step 4.7's isolation/permission preflight.
|
||||
Do not start applications, exercise APIs or mutate state during this audit. If found and verified safe above, invoke it with the relevant path argument (e.g., `npm run validate-wiki -- <path>`). For multi-target validators (e.g., `validate-wiki --all`), run once and reconcile per-item from the output. A passing validator promotes the item from UNVERIFIABLE to DONE; a failing one demotes to NOT DONE.
|
||||
|
||||
**Honesty rule.** Do NOT classify an item as DONE just because related code shipped. Code that *handles* a deliverable is not the deliverable. Shipping a markdown-extraction library is not the same as shipping the markdown file. When in doubt between DONE and UNVERIFIABLE, prefer UNVERIFIABLE — better to surface a confirmation prompt than silently miss a deliverable.
|
||||
|
||||
@@ -84,7 +92,7 @@ Before judging completion, classify HOW each item can be verified. The diff alon
|
||||
|
||||
Run `git diff origin/<base>...HEAD` and `git log origin/<base>..HEAD --oneline` to understand what was implemented.
|
||||
|
||||
For each extracted plan item, run the verification dispatch from the previous section, then classify:
|
||||
For each audited deliverable, run the verification dispatch from the previous section, then classify:
|
||||
|
||||
- **DONE** — Clear evidence the item shipped. Cite the specific file(s) changed in the diff for DIFF-VERIFIABLE items, or the verified path that exists for CROSS-REPO items with a reachable sibling repo.
|
||||
- **PARTIAL** — Some work toward this item exists but is incomplete (e.g., model created but controller missing, function exists but edge cases not handled).
|
||||
@@ -100,7 +108,7 @@ For each extracted plan item, run the verification dispatch from the previous se
|
||||
|
||||
```
|
||||
PLAN COMPLETION AUDIT
|
||||
═══════════════════════════════
|
||||
════════════════════
|
||||
Plan: {plan file path}
|
||||
|
||||
## Implementation Items
|
||||
@@ -121,9 +129,9 @@ Plan: {plan file path}
|
||||
[UNVERIFIABLE] Cloudflare DNS-only on api.example.com — external system, manual check required
|
||||
[UNVERIFIABLE] Supabase auth allowlist contains user email — external system, confirm in Supabase dashboard
|
||||
|
||||
─────────────────────────────────
|
||||
────────────────────
|
||||
COMPLETION: 4/10 DONE, 1 PARTIAL, 2 NOT DONE, 1 CHANGED, 2 UNVERIFIABLE
|
||||
─────────────────────────────────
|
||||
────────────────────
|
||||
```
|
||||
|
||||
### Fallback Intent Sources (when no plan file found)
|
||||
@@ -186,11 +194,14 @@ The plan completion results augment the existing Scope Drift Detection. If a pla
|
||||
- **Items in the diff that don't match any plan item** become evidence for **SCOPE CREEP** detection.
|
||||
- **HIGH-impact discrepancies** trigger AskUserQuestion:
|
||||
- Show the investigation findings
|
||||
- Options: A) Stop and implement missing items, B) Ship anyway + create P1 TODOs, C) Intentionally dropped
|
||||
- Options: A) Stop this review for implementation, B) Continue this review with P1 TODOs, C) Record the items as intentionally dropped
|
||||
- A ends this invocation before code review or implementation. List the missing work; after implementation, start a fresh /review.
|
||||
- B queues the approved TODO changes for Step 5, not this read-only audit. B/C continue to the final Scope Check and Step 2. None of these choices authorizes shipping or waives required verification.
|
||||
|
||||
This is **INFORMATIONAL** unless HIGH-impact discrepancies are found (then it gates via AskUserQuestion).
|
||||
|
||||
Update the scope drift output to include plan file context:
|
||||
When continuing after the audit (no HIGH-impact gate, or option B/C), emit the
|
||||
single final Scope Check using Step 1.5's provisional notes and this plan context:
|
||||
|
||||
```
|
||||
Scope Check: [CLEAN / DRIFT DETECTED / REQUIREMENTS MISSING]
|
||||
@@ -202,4 +213,6 @@ Plan items: N DONE, M PARTIAL, K NOT DONE
|
||||
[If scope creep: list each out-of-scope change not in the plan]
|
||||
```
|
||||
|
||||
**No plan file found:** Use commit messages and TODOS.md as fallback sources (see above). If no intent sources at all, skip with: "No intent sources detected — skipping completion audit."
|
||||
**No plan file found:** Use commit messages and TODOS.md as fallback sources (see above).
|
||||
Emit Step 1.5's Scope Check once without plan fields. If no intent sources exist, state
|
||||
"No intent sources detected — skipping completion audit." rather than claiming requirements were verified.
|
||||
Reference in new issue
Block a user