mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-11 15:39:04 +02:00
fix: repair frontier eval budgets and workflow instructions
Preserve frontier models and quality thresholds while fixing truncated judge output, ordered section expansion, consent checks, QA scoring, and ship audit gates. Add regression coverage and refresh generated docs. Co-Authored-By: OpenAI Codex <noreply@openai.com>
This commit is contained in:
co-authored by
OpenAI Codex
parent
0cbccc1c6c
commit
a9f9ec5f08
@@ -239,23 +239,36 @@ console.log("ASIDE_DIR=" + pwd); await closeTab(pg); console.log("GSTACK_STEP_OK
|
||||
|
||||
Compute each category score (0-100), then take the weighted average.
|
||||
|
||||
### Counting
|
||||
- Deduplicate the same root cause across pages. Use one primary category, first applicable: Links (navigation), Accessibility (access barriers), Functional (behavior), Performance (speed), Visual (layout), Content (copy), UX (friction), Console (remaining errors). No double deductions.
|
||||
- Exclude **untested** categories; label partial scores **provisional** with coverage. None tested: "not scored". Compare only identical coverage.
|
||||
|
||||
### Console (weight: 15%)
|
||||
Deduplicate reproducible errors/exceptions by message+source across pages. Exclude warnings, info, and defects scored elsewhere.
|
||||
- 0 errors → 100
|
||||
- 1-3 errors → 70
|
||||
- 4-10 errors → 40
|
||||
- 10+ errors → 10
|
||||
- 11+ errors → 10
|
||||
|
||||
### Links (weight: 10%)
|
||||
Count unique broken destinations, including client-side routes: repeatable 4xx/5xx, missing routes/anchors, or timeouts. Exclude expected auth redirects and resource/API requests.
|
||||
- 0 broken → 100
|
||||
- Each broken link → -15 (minimum 0)
|
||||
|
||||
### Per-Category Scoring (Visual, Functional, UX, Content, Performance, Accessibility)
|
||||
Each category starts at 100. Deduct per finding:
|
||||
Start at 100; deduct per finding:
|
||||
- Critical issue → -25
|
||||
- High issue → -15
|
||||
- Medium issue → -8
|
||||
- Low issue → -3
|
||||
Minimum 0 per category.
|
||||
Floor: 0.
|
||||
|
||||
Use the highest applicable severity; record impact/workaround:
|
||||
- **Critical:** data loss, security/privacy exposure, or core app unusable for all users.
|
||||
- **High:** core/major task blocked without a workaround.
|
||||
- **Medium:** task impaired but a workaround exists.
|
||||
- **Low:** cosmetic/copy/friction issue without lost task completion.
|
||||
Console/Links use counts instead.
|
||||
|
||||
### Weights
|
||||
| Category | Weight |
|
||||
@@ -270,7 +283,7 @@ Minimum 0 per category.
|
||||
| Accessibility | 15% |
|
||||
|
||||
### Final Score
|
||||
`score = Σ (category_score × weight)`
|
||||
Use decimal weights (15% = 0.15): `score = Σ (category_score × weight) / Σ tested weights`. Round only the final score to the nearest integer (0.5 rounds up).
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user