fix: repair frontier eval budgets and workflow instructions

Preserve frontier models and quality thresholds while fixing truncated judge output, ordered section expansion, consent checks, QA scoring, and ship audit gates. Add regression coverage and refresh generated docs.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
This commit is contained in:
Garry Tan
2026-09-09 04:11:19 +00:00
co-authored by OpenAI Codex
parent 0cbccc1c6c
commit a9f9ec5f08
43 changed files with 916 additions and 518 deletions
+17 -4
View File
@@ -239,23 +239,36 @@ console.log("ASIDE_DIR=" + pwd); await closeTab(pg); console.log("GSTACK_STEP_OK
Compute each category score (0-100), then take the weighted average.
### Counting
- Deduplicate the same root cause across pages. Use one primary category, first applicable: Links (navigation), Accessibility (access barriers), Functional (behavior), Performance (speed), Visual (layout), Content (copy), UX (friction), Console (remaining errors). No double deductions.
- Exclude **untested** categories; label partial scores **provisional** with coverage. None tested: "not scored". Compare only identical coverage.
### Console (weight: 15%)
Deduplicate reproducible errors/exceptions by message+source across pages. Exclude warnings, info, and defects scored elsewhere.
- 0 errors → 100
- 1-3 errors → 70
- 4-10 errors → 40
- 10+ errors → 10
- 11+ errors → 10
### Links (weight: 10%)
Count unique broken destinations, including client-side routes: repeatable 4xx/5xx, missing routes/anchors, or timeouts. Exclude expected auth redirects and resource/API requests.
- 0 broken → 100
- Each broken link → -15 (minimum 0)
### Per-Category Scoring (Visual, Functional, UX, Content, Performance, Accessibility)
Each category starts at 100. Deduct per finding:
Start at 100; deduct per finding:
- Critical issue → -25
- High issue → -15
- Medium issue → -8
- Low issue → -3
Minimum 0 per category.
Floor: 0.
Use the highest applicable severity; record impact/workaround:
- **Critical:** data loss, security/privacy exposure, or core app unusable for all users.
- **High:** core/major task blocked without a workaround.
- **Medium:** task impaired but a workaround exists.
- **Low:** cosmetic/copy/friction issue without lost task completion.
Console/Links use counts instead.
### Weights
| Category | Weight |
@@ -270,7 +283,7 @@ Minimum 0 per category.
| Accessibility | 15% |
### Final Score
`score = Σ (category_score × weight)`
Use decimal weights (15% = 0.15): `score = Σ (category_score × weight) / Σ tested weights`. Round only the final score to the nearest integer (0.5 rounds up).
---