Garry Tan cddf8ee3bd fix: simplify planted-bug eval prompts for reliable 25-turn completion ...

The QA agent was spending all 50 turns reading qa/SKILL.md and browsing
without ever writing a report. Replace verbose QA workflow prompt with
concise, direct bug-finding instructions. The /qa quick test already
validates the full QA workflow E2E — planted-bug evals test "can the
agent find bugs with browse", not the QA workflow documentation.

- 25 maxTurns (was 50) — more focused, less cost (~$0.50 vs ~$1.00)
- Direct step-by-step instructions instead of "read qa/SKILL.md"
- 180s timeout (was 300s)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-14 05:51:48 -05:00

..

fixtures

fix: lower planted-bug detection baselines and LLM judge thresholds for reliability

2026-03-14 05:16:17 -05:00

helpers

fix: remove false-positive Exit code 1 pattern, fix NEEDS_SETUP test, update QA tests

2026-03-14 04:48:35 -05:00

gen-skill-docs.test.ts

fix: enrich SKILL.md docs to pass LLM evals, upgrade judge to Sonnet 4.6 (#43)

2026-03-13 22:14:14 -07:00

skill-e2e.test.ts

fix: simplify planted-bug eval prompts for reliable 25-turn completion

2026-03-14 05:51:48 -05:00

skill-llm-eval.test.ts

fix: lower planted-bug detection baselines and LLM judge thresholds for reliability

2026-03-14 05:16:17 -05:00

skill-parser.test.ts

feat: SKILL.md template system, 3-tier testing, DX tools (v0.3.3) (#41)

2026-03-13 21:08:12 -07:00

skill-validation.test.ts

fix: remove false-positive Exit code 1 pattern, fix NEEDS_SETUP test, update QA tests

2026-03-14 04:48:35 -05:00