mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-09 14:38:59 +02:00
fix(ci): run the ship-docsync gate E2E in the evals matrix + silent-skip tripwire
The evals.yml matrix is hand-enumerated and the Run step never exported EVALS_TIER, so the new whole-file-gated ship-docsync E2E would have self-skipped even with a row — a hollow green one layer deeper than the documented rehomed-monolith incident. Add the e2e-ship-docsync row with a row-level `tier: gate` property, exported as EVALS_TIER by the Run step (empty = unset for every existing row: all readers are `=== '<tier>'` or truthiness). New free tripwire test/evals-workflow-matrix.test.ts ratchets the class: matrix files must exist; gate-hosting files must have a row; whole-file-gated matrix files must carry a matching row tier; and the burn-down lists enforce their own cleanup. It enumerates the PRE-EXISTING holes found while wiring this (8 gate-hosting files with no row; codex/gemini rows running zero tests; the pty-plan-smoke row hollow since its files adopted describeE2ETier) — tracked in TODOS as the CI gate-lane hollow-coverage burn-down. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
65b9cff299
commit
8f18682880
@@ -135,6 +135,17 @@ jobs:
|
||||
file: test/skill-e2e-coverage-audit.test.ts
|
||||
- name: e2e-triage
|
||||
file: test/skill-e2e-triage.test.ts
|
||||
# ship-docsync is whole-file tier-gated (describeE2ETier('gate') keeps
|
||||
# it out of the periodic shard census), so its row MUST set tier: gate
|
||||
# — without it the self-gate skips every test and the job reports a
|
||||
# hollow green (the same silent-skip class as the rehomed monolith
|
||||
# above, one layer deeper). The Run step exports EVALS_TIER from this
|
||||
# property; rows without it keep EVALS_TIER empty (= unset: every
|
||||
# reader is `=== '<tier>'` or truthiness). Enforced by
|
||||
# test/evals-workflow-matrix.test.ts.
|
||||
- name: e2e-ship-docsync
|
||||
file: test/skill-e2e-ship-docsync.test.ts
|
||||
tier: gate
|
||||
- name: e2e-routing
|
||||
file: test/skill-routing-e2e.test.ts
|
||||
- name: e2e-codex
|
||||
@@ -337,6 +348,12 @@ jobs:
|
||||
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
|
||||
EVALS_CONCURRENCY: "40"
|
||||
PLAYWRIGHT_BROWSERS_PATH: /opt/playwright-browsers
|
||||
# Per-row tier activation for whole-file-gated suites. Empty when the
|
||||
# row declares no tier — every EVALS_TIER reader treats empty as unset
|
||||
# (`=== '<tier>'` comparisons and the truthiness check in
|
||||
# test/helpers/e2e-helpers.ts:70), so untiered rows are byte-for-byte
|
||||
# unaffected.
|
||||
EVALS_TIER: ${{ matrix.suite.tier || '' }}
|
||||
run: EVALS=1 bun test --retry ${{ matrix.suite.retries || 1 }} --concurrent --max-concurrency 40 ${{ matrix.suite.file }}
|
||||
|
||||
- name: Upload eval results
|
||||
|
||||
Reference in New Issue
Block a user