v1.91.371.0 ci(evals): a pr-summary run skips the readiness wait it never used (#3609)

* ci(evals): a pr-summary run skips the readiness wait it never used

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore: CHANGELOG [Unreleased] entry for the pr-summary readiness skip

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore: move the CHANGELOG entry to changelog.d/capy-gsta149-pr-summary-wait.md (fragments policy since #3654)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
Garry Tanandcapy-ai[bot] authored and GitHub committed 2026-10-31 15:20:39 +00:00
1 parent cae783e6f4
commit 48599bb8ae
9 files changed
+31 -10

No files matched your search

+6 -2
View File
@@ -278,10 +278,14 @@ jobs:
# Free Tests red on this head fails the run without paid work (re-run this
# workflow after a green Free Tests rerun). Dispatches never wait: they are
# explicit (eval:bg, census). Version Gate is not an input, so a stamp push
# waits for its own Free Tests only and always gets its gate.
# waits for its own Free Tests only and always gets its gate. A pr-summary
# run (a repository with a train block) plans no paid slices, so it skips
# the wait: the train gate and the dispatched screen carry the paid work,
# and Free Tests stays the required free check.
readiness:
runs-on: ubuntu-24.04
if: github.event_name == 'pull_request' && github.actor != 'dependabot[bot]' && github.event.pull_request.head.repo.full_name == github.repository
needs: [route]
if: github.event_name == 'pull_request' && github.actor != 'dependabot[bot]' && github.event.pull_request.head.repo.full_name == github.repository && needs.route.outputs.mode != 'pr-summary'
timeout-minutes: 90
permissions:
contents: read
+12
View File
@@ -1,5 +1,17 @@
# Changelog
## [1.91.371.0] - 2026-10-31
**A PR push in a repository with a merge train no longer waits ten minutes for a check that does no work.** Under a train policy every PR push runs E2E Evals in `pr-summary` mode, which plans no paid slices, yet its `readiness` job still waited out the 600-second quiet window and green Free Tests before the run could finish. Now `readiness` waits for `route` and skips in `pr-summary` mode, so E2E Evals on a PR push finishes in about a minute (58 s on #3609) instead of after the 10-minute wait.
### Changed
- **`.github/workflows/evals.yml`**: `readiness` needs `route` and runs only when the mode is not `pr-summary`. In `pr` mode (a repository without a train) it still gates recovery and planning as before. In `pr-summary` mode a red Free Tests now shows on Free Tests alone, which stays the required free check; the train gate and the dispatched screen still carry the paid work.
- **`bun run test:health`**: the "PR lane push-to-verdict" rule text says `pr-summary` runs skip the wait since GSTA-149, so its p50 drops by about 10 minutes from this change on.
- **docs/TESTING_INTERNALS.md** describes the skip.
### Tests
- `test/evals-pr-lane-workflow.test.ts` pins `readiness.needs` and the full `if`. `test/evals-train-workflow.test.ts` checks that a `pr-summary` push runs only `build-image`, `route` and `screen-summary`, and that `pr` mode still runs `readiness`.
## [1.91.370.0] - 2026-10-31
**The free suite now runs on `ubicloud-standard-2` instead of `ubicloud-standard-8`, at the same 20 shards, for about a quarter of the runner cost.** Each shard runs its files one at a time, so it waits on disk and processes rather than CPU. In three paired rounds at one commit, the median per-shard time rose by 3 to 8 s, inside the round-to-round spread of −90 to +65 s, while runner-minutes stayed about equal (464 vs 475). Peak memory stayed under 2.5 GiB and free disk above 15 GiB. A 16-shard arm was slower beyond that spread, so the shard count stays at 20.
+1 -1
View File
@@ -1 +1 @@
1.91.370.0
1.91.371.0
+1 -1
View File
@@ -1,4 +1,4 @@
# gstack digest v1.91.370.0 — regenerate/re-copy after upgrading gstack
# gstack digest v1.91.371.0 — regenerate/re-copy after upgrading gstack
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
for agent hosts without a full skill install. The full skills add workflows,
+4 -1
View File
@@ -1282,7 +1282,10 @@ re-run keeps the creation time, so it does not wait out the window again). A
head that is no longer the PR's head reports "superseded by <sha>" and plans
nothing. Selection, trials and verdict rules do not change. Version Gate is not
an input, so a stamp push waits only for its own Free Tests and always gets a
gate. Dispatches (`eval:bg`, the census) never wait.
gate. Dispatches (`eval:bg`, the census) never wait. Neither does a
`pr-summary` run (a repository whose policy declares a train): it plans no paid
slices, so `readiness` is skipped, the run ends with the screening summary, and
a red Free Tests shows on Free Tests alone.
**Verdict reuse (R2a restamp-only, R2 main moved; a gate edit).** A PR head
reuses the verdict of the newest executed gate of the same PR instead of
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "gstack",
"version": "1.91.370",
"version": "1.91.371",
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
"license": "MIT",
"type": "module",
+1 -1
View File
@@ -277,7 +277,7 @@ export function collectHealth(inputs: HealthInputs): HealthReport {
return { value: `${(total / 60_000 * 7 / sinceDays).toFixed(0)} min/week (${cancelled.length} cancelled run(s))` };
}));
metrics.push(metric('push-to-verdict', 'PR lane push-to-verdict', 'created_at to updated_at of evals.yml pull_request runs that reached success or failure (includes queue and the readiness wait)', () => {
metrics.push(metric('push-to-verdict', 'PR lane push-to-verdict', 'created_at to updated_at of evals.yml pull_request runs that reached success or failure (includes queue and the readiness wait, which pr-summary runs skip since GSTA-149, so their verdicts land about 10 min sooner)', () => {
const walls = need(prRuns).filter(run => run.conclusion === 'success' || run.conclusion === 'failure').map(run => span(run.createdAt, run.updatedAt)!);
if (walls.length === 0) throw new Error('no PR run reached a verdict in the window');
return { value: `p50 ${minutes(percentile(walls, 0.5))}, p95 ${minutes(percentile(walls, 0.95))} (${walls.length} runs)` };
+2 -1
View File
@@ -83,7 +83,8 @@ describe('evals.yml readiness-gated dispatch (R1, GSTA-48)', () => {
test('a tiny PR-only job waits for a quiet PR and green Free Tests before recovery and planning', () => {
expect(readiness['runs-on']).toBe('ubuntu-24.04');
expect(readiness.if).toBe("github.event_name == 'pull_request' && github.actor != 'dependabot[bot]' && github.event.pull_request.head.repo.full_name == github.repository");
expect(readiness.if).toBe("github.event_name == 'pull_request' && github.actor != 'dependabot[bot]' && github.event.pull_request.head.repo.full_name == github.repository && needs.route.outputs.mode != 'pr-summary'");
expect(readiness.needs).toEqual(['route']);
expect(readiness.permissions).toEqual({ contents: 'read', actions: 'read', 'pull-requests': 'read' });
expect(Number(readiness['timeout-minutes'])).toBeGreaterThan(80);
const wait = step('readiness', 'Wait for a quiet PR and green Free Tests on this head')!;
+3 -2
View File
@@ -193,9 +193,10 @@ describe('without a train block nothing changes (the inert proof)', () => {
});
describe('under a train block (the gate edits, each only on its mode)', () => {
test('a PR push starts no paid slice: route, readiness and the free screening summary only', () => {
test('a PR push starts no paid slice and waits for nothing: route, the image and the free screening summary only', () => {
const sim = simulate(evals, { github: pr(), inputs: {}, outputs: { route: { ...ROUTE, ...train, mode: 'pr-summary' } } });
expect(sim.jobs).toEqual(['build-image', 'readiness', 'route', 'screen-summary']);
expect(sim.jobs).toEqual(['build-image', 'route', 'screen-summary']);
expect(simulate(evals, { github: pr(), inputs: {}, outputs: { route: { ...ROUTE, mode: 'pr' } } }).jobs).toContain('readiness');
const summary = evals.jobs['screen-summary']!;
expect(summary.permissions).toEqual({ contents: 'read' });
expect(JSON.stringify(summary)).not.toMatch(/secrets\.|restore-deps|bun install/);