v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)

* feat: add surface-aware exploratory QA and ship documentation gates

* test: preserve delegated QA setup authority after main integration

* fix(qa): clarify exploration order and preserve report artifacts

* test(qa): follow the shared setup reference directly

* refactor(ship): make verification and recovery routes explicit

* test(ship): align evidence and review guards with explicit routes

* fix(workflows): clarify ship recovery and functional QA evidence

* fix(workflows): clarify approval recovery and full QA coverage

* refactor(workflows): order review transactions and clarify ship state

* fix(ship): clarify final verification and fail closed at publication

* fix(evals): attribute native atomic documentation writes

* fix(ship): clarify recovery and documentation lifecycle guidance

* fix(test): preserve observed native placeholder styling in CI

* fix(codex): report watchdog timeouts without a process-exit race

* Checkpoint functional QA implementation and workflow validation repairs

* Fix documentation and shared-review fixture contracts

* docs: clarify judge reuse and evaluation supervision

* test: align review evidence and selected case contracts

* test: verify append-only documentation checkpoints and recovery

* fix: qualify QA workflows and CI validation repairs

* fix: launch shared-libs fixture scripts on Windows

* fix: qualify QA deadlines, fixture isolation, and shard cleanup

* fix: preserve qualified QA and cancellation repairs

* fix: enforce functional fixture authority and share strict event decoding

* fix: retain free-test evidence and explain recovery

* fix: reject malformed native evidence after decoder consolidation

* test: use reliable capture for telemetry privacy filters

* test: refresh measured quick coverage and document validation costs

* Fix native fixture receipts and preserve VM validation evidence

* Align negative judge controls with upstream clarity policy

* Fix report-only QA preparation and public evidence handling

* Clarify QA-only preparation and current-report preservation

* Stream Ship quality judgments with an explicit 64k response contract

* Validate compact judge reasoning locally with supported wire schema

* Align functional QA fixture instructions with evidence acceptance

* Bind native browser diagnostics to execution evidence and align review verdicts

* Preserve native diagnostic line boundaries

* Serialize functional QA evidence from native captures

* Keep large QA evidence fixture payload out of Windows argv
This commit is contained in:
Garry Tan authored and GitHub committed 2026-09-29 06:07:35 -07:00
1 parent 65bfb0ce49
commit dcaea52800
333 files changed
+41755 -7357

No files matched your search

+29 -9
View File
@@ -104,12 +104,14 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
'test-coverage.md',
'plan-completion.md',
'review-army.md',
'shared-code-reuse.md',
'greptile.md',
'adversarial.md',
'changelog.md',
'documentation.md',
'pr-body.md',
],
requiredReads: ['review-army.md', 'changelog.md'],
requiredReads: ['review-army.md', 'changelog.md', 'documentation.md'],
scenario:
'This is a FRESH version-changing ship: the branch has a real code change, VERSION still equals the base version (needs a bump), and CHANGELOG.md needs a new entry. Follow the skill flow for a version-changing ship: run the pre-landing review and prepare the CHANGELOG entry. Produce the ship plan / review report. Do NOT actually commit, push, or open a PR.',
staticInvariants: {
@@ -130,8 +132,8 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
mustStayInSkeleton: [
'v$NEW_VERSION',
'gstack-pr-title-rewrite',
'dispatching the /document-release subagent to sync docs',
'Continue to mandatory Step 18 (dispatch /document-release)',
'## Step 14.5: Documentation audit (every ship)',
'No documentation writer runs after push',
'dispatches the /document-release subagent',
],
// ...while the full create/update procedure stays carved into pr-body.md
@@ -352,8 +354,8 @@ do not launch the downstream skill or open a browser.`,
},
'document-release': {
skill: 'document-release',
expectedSections: ['release-body.md'],
requiredReads: ['release-body.md'],
expectedSections: ['audit-scope.md', 'release-body.md'],
requiredReads: ['audit-scope.md', 'release-body.md'],
scenario:
'A PR has shipped a new CLI flag and touched README.md and CHANGELOG.md. Skip the git pre-flight shell commands (assume the diff adds --new-flag and updates those two docs). Run the documentation workflow: build the coverage map, then audit the docs, apply updates, and polish the CHANGELOG voice. Produce the documentation health summary.',
staticInvariants: {
@@ -450,7 +452,7 @@ do not launch the downstream skill or open a browser.`,
// ── Token-reduction Phase 4 wave 1 (v1.69.x branch) ──────────────────────
review: {
skill: 'review',
expectedSections: ['plan-completion.md', 'review-army.md', 'adversarial.md'],
expectedSections: ['plan-completion.md', 'review-army.md', 'shared-code-reuse.md', 'adversarial.md'],
requiredReads: ['plan-completion.md', 'review-army.md'],
scenario:
"The working tree has a real diff against the base branch (assume Step 1's git checks passed; the diff implements the PLAN.md cache layer). Run the /review flow: the scope-drift and plan-completion deep pass against PLAN.md, then the critical pass, then the Review Army specialist dispatch — apply the specialist checklists yourself instead of launching subagents. Produce the review report. Do NOT commit, push, or create a PR.",
@@ -632,14 +634,13 @@ do not launch the downstream skill or open a browser.`,
// ── Token-reduction Phase 4 wave 3 (v1.69.x branch) ──────────────────────
qa: {
skill: 'qa',
expectedSections: ['test-bootstrap.md', 'qa-patterns.md'],
requiredReads: ['qa-patterns.md'],
expectedSections: ['scope.md', 'browser-setup.md', 'exploratory.md', 'system-functional.md', 'browser-verify.md', 'test-bootstrap.md', 'qa-patterns.md'],
requiredReads: ['scope.md', 'browser-setup.md', 'exploratory.md', 'qa-patterns.md'],
scenario:
'Walk /qa in SIMULATION — do not launch a browser, run any aside command, or execute bash; treat the working tree as clean, the tier as Quick, and the target app as http://localhost:3000 with a small feature-branch diff touching one page. Skip the test-framework bootstrap (assume CLAUDE.md documents the test command). Read each pointed section before doing its step, then produce the QA plan as the report: the mode you selected and why, the Phase 1-6 steps you would run, and a worked health-score computation from the rubric. Do NOT use AskUserQuestion.',
staticInvariants: {
mustStayInSkeleton: [
'## Setup',
'## BROWSER SETUP (Aside',
'## Phases 1-6: QA Baseline',
'## Phase 7: Triage',
'## Phase 8: Fix Loop',
@@ -652,6 +653,8 @@ do not launch the downstream skill or open a browser.`,
mustMoveToSection: [
'## Test Framework Bootstrap',
'BOOTSTRAP_DECLINED',
'### Select the surface before setup',
'## BROWSER SETUP (Aside',
'## Health Score Rubric',
'### Diff-aware (automatic when on a feature branch with no URL)',
'Never refuse to use the browser',
@@ -665,6 +668,23 @@ do not launch the downstream skill or open a browser.`,
// 'aside repl' pins the Aside contract; '$B goto' pins the fallback block in the always-loaded skeleton.
mustContain: ['bug', 'aside repl', '$B goto', 'fix', 'Health Score Rubric', 'regression'],
},
'qa-only': {
skill: 'qa-only',
expectedSections: ['exploratory.md'],
requiredReads: ['exploratory.md'],
scenario:
'Walk /qa-only for an isolated CLI fixture using its declared native commands. Read installed scope, exploratory and functional resources; never read browser setup or DX instructions. Report contract outcomes and proposed tests without changing product, tests or Git. Do not use AskUserQuestion.',
staticInvariants: {
mustStayInSkeleton: ['## Request Parameters', 'Never fix bugs or write product tests', '## Output'],
mustPrecedeStop: ['## Request Parameters'],
mustMoveToSection: ['# Shared exploratory QA'],
},
behavioral: 'external',
externalTest: 'test/skill-e2e-qa-functional.test.ts',
maxSkeletonBytes: 45_000,
minUnionBytes: 40_000,
mustContain: ['contract', 'Never fix bugs', 'edit-then-restore', 'test_stub'],
},
browse: {
skill: 'browse',
expectedSections: ['command-list.md'],