mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-03 18:06:54 +02:00
Garry Tan
dcaea52800
* feat: add surface-aware exploratory QA and ship documentation gates * test: preserve delegated QA setup authority after main integration * fix(qa): clarify exploration order and preserve report artifacts * test(qa): follow the shared setup reference directly * refactor(ship): make verification and recovery routes explicit * test(ship): align evidence and review guards with explicit routes * fix(workflows): clarify ship recovery and functional QA evidence * fix(workflows): clarify approval recovery and full QA coverage * refactor(workflows): order review transactions and clarify ship state * fix(ship): clarify final verification and fail closed at publication * fix(evals): attribute native atomic documentation writes * fix(ship): clarify recovery and documentation lifecycle guidance * fix(test): preserve observed native placeholder styling in CI * fix(codex): report watchdog timeouts without a process-exit race * Checkpoint functional QA implementation and workflow validation repairs * Fix documentation and shared-review fixture contracts * docs: clarify judge reuse and evaluation supervision * test: align review evidence and selected case contracts * test: verify append-only documentation checkpoints and recovery * fix: qualify QA workflows and CI validation repairs * fix: launch shared-libs fixture scripts on Windows * fix: qualify QA deadlines, fixture isolation, and shard cleanup * fix: preserve qualified QA and cancellation repairs * fix: enforce functional fixture authority and share strict event decoding * fix: retain free-test evidence and explain recovery * fix: reject malformed native evidence after decoder consolidation * test: use reliable capture for telemetry privacy filters * test: refresh measured quick coverage and document validation costs * Fix native fixture receipts and preserve VM validation evidence * Align negative judge controls with upstream clarity policy * Fix report-only QA preparation and public evidence handling * Clarify QA-only preparation and current-report preservation * Stream Ship quality judgments with an explicit 64k response contract * Validate compact judge reasoning locally with supported wire schema * Align functional QA fixture instructions with evidence acceptance * Bind native browser diagnostics to execution evidence and align review verdicts * Preserve native diagnostic line boundaries * Serialize functional QA evidence from native captures * Keep large QA evidence fixture payload out of Windows argv
55 lines
3.0 KiB
Markdown
55 lines
3.0 KiB
Markdown
# Functional QA Report: {TARGET}
|
|
|
|
| Field | Value |
|
|
|---|---|
|
|
| Date / branch / revision | {DATE / BRANCH / COMMIT AND WORKING-TREE INPUTS} |
|
|
| Caller / authority / depth | {qa-only, qa, review or ship; permitted writes; bound} |
|
|
| Surfaces / scope | {API, CLI, job, worker, webhook; changed and adjacent contracts} |
|
|
| Runtime / native tools | {VERSIONS AND REPOSITORY-SUPPORTED COMMANDS} |
|
|
| Fixture ownership / destinations | {ISOLATED ROOT, STORES, DOWNSTREAM TARGETS} |
|
|
| Duration / stop reason | {MEASURED DURATION, COMPLETE OR BOUND/BLOCKER} |
|
|
|
|
## Contract outcomes
|
|
|
|
| Contract and source | Exact probe / evidence | Expected → observed | Outcome |
|
|
|---|---|---|---|
|
|
| {CONTRACT, DOC/TEST/USER SOURCE} | {COMMAND OR REQUEST, EVIDENCE PATH} | {OUTPUT AND DURABLE EFFECT} | pass / fail / blocked / not run / inconclusive / not applicable (reason) |
|
|
|
|
No visual score applies to this functional section. In a mixed report, keep the
|
|
browser section's score and evidence separate, and link both surfaces' replay
|
|
evidence and regression baselines. Do not combine their scores or outcomes.
|
|
|
|
## Findings
|
|
|
|
### ISSUE-NNN: {Reproduced defect or setup blocker}
|
|
|
|
- Classification / severity: {PRODUCT DEFECT / SETUP / INCONCLUSIVE; IMPACT}.
|
|
- Intended contract and source: {EXPECTED BEHAVIOR, NOT MERELY CURRENT IMPLEMENTATION}.
|
|
- Reproduction: {WORKING DIRECTORY; SAFE SETUP/RESET; ENVIRONMENT NAMES ONLY; EXACT COMMAND OR METHOD/PATH/HEADERS/BODY USING SYNTHETIC VALUES}.
|
|
- Observed: {EXIT/STATUS; STDOUT; STDERR; INITIAL/FINAL DURABLE STATE; REPLAY/RETRY ORDER}.
|
|
- Evidence: {EXACT SAFE OUTPUT AND STATE PATHS; REVISION/RUNTIME; REDACTION AND REPRODUCIBILITY LIMITS}.
|
|
- Diagnosis / next action: {CAUSAL EVIDENCE OR SPECIFIC PREREQUISITE; NO SPECULATIVE FIX}.
|
|
|
|
## Discoveries and permanent tests
|
|
|
|
Link each `exploration-NNN.json` checkpoint, saved before its next probe, in this report.
|
|
Use one Markdown entry per checkpoint, for example:
|
|
|
|
- [checkpoint 001](exploration-001.json) — how this observation shaped the next probe.
|
|
|
|
Use the actual filename and a path relative to this report (or its owned absolute
|
|
path); plain or backticked filenames are not links.
|
|
Include superseded checkpoints as history, not current passing evidence.
|
|
Keep these original notes with the report.
|
|
|
|
| Hypothesis / discovery | Native test or proposed case | Red evidence before repair | Green + original + adjacent evidence | Parent disposition |
|
|
|---|---|---|---|---|
|
|
| {OBSERVATION THAT CHANGED THE NEXT PROBE} | {UNIT / INTEGRATION / E2E; PATH OR REPORT-ONLY PROPOSAL} | {EXACT DEFECT FAILURE OR HEALTHY CONTRACT} | {ACTUAL RESULTS OR NOT RUN} | {AUTHORIZED CHANGE / SUGGESTION / DEFERRED} |
|
|
|
|
## Coverage limits and cleanup
|
|
|
|
List unexecuted charters, unavailable prerequisites, denied effects, ambiguous contracts,
|
|
incomplete observations and remaining risk. Do not count them as passes. Name owned
|
|
processes/state cleaned and anything left behind. State whether later changes invalidated
|
|
evidence. Report-only must identify proposals separately from tests actually created.
|