CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-09-14 08:59:01 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix(evals): align plan-eng/design plan-mode + finding-floor smokes to their declared periodic tier

Browse Source 
The #2077 demotion of these four stochastic tests to 'periodic' was inert:
E2E_TIERS declared periodic but the files self-gated on EVALS_TIER === 'gate',
so they kept running in the blocking gate lane and never in the weekly lane.

Flip the four self-gates to 'periodic' (headers/describe labels updated), add
a free static tier-alignment invariant test (dep-list filename mapping;
unmapped self-gated files are reported, never silently skipped), and name the
two plan-mode test files in their own touchfiles dep lists so the invariant
binds for them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-11 17:40:17 -07:00
co-authored by Claude Fable 5
parent 94993f7401
commit f349d7c894
6 changed files with 88 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/skill-e2e-plan-eng-finding-floor.test.ts
+4 -4
View File
@@ -1,5 +1,5 @@
/**
* /plan-eng-review AskUserQuestion floor regression (gate, paid, real-PTY).
* /plan-eng-review AskUserQuestion floor regression (periodic, paid, real-PTY).
*
* Catches the May 2026 transcript bug where /plan-eng-review wrote a
* multi-section review plan to ~/.claude/plans/ and called ExitPlanMode
@@ -11,7 +11,7 @@
* render. See claude-pty-runner.ts for why this is separate from the
* runPlanSkillCounting harness used by periodic finding-count tests.
*
* Tier: gate. Budget: 10 min (early exit on success ~30-90s typical).
* Tier: periodic. Budget: 10 min (early exit on success ~30-90s typical).
* Cost: ~$0.50-$1.50 per run depending on early-exit timing.
*/
@@ -19,10 +19,10 @@ import { describe, test } from 'bun:test';
import { runPlanSkillFloorCheck } from './helpers/claude-pty-runner';
import { FORCING_FLOOR_ENG } from './fixtures/forcing-finding-seeds';
const shouldRun = !!process.env.EVALS && process.env.EVALS_TIER === 'gate';
const shouldRun = !!process.env.EVALS && process.env.EVALS_TIER === 'periodic';
const describeE2E = shouldRun ? describe : describe.skip;
describeE2E('/plan-eng-review AskUserQuestion floor (gate)', () => {
describeE2E('/plan-eng-review AskUserQuestion floor (periodic)', () => {
test(
'seeded forcing finding causes the agent to fire at least one AskUserQuestion',
async () => {