Garry Tan 9dffb1ed16 feat: LLM-as-judge evals for SKILL.md documentation quality ...

4 eval tests using Anthropic API (claude-haiku, ~$0.01-0.03/run):
- Command reference table: clarity/completeness/actionability >= 4/5
- Snapshot flags section: same thresholds
- browse/SKILL.md overall quality
- Regression: generated version must score >= hand-maintained baseline

Requires ANTHROPIC_API_KEY. Auto-skips without it.
Run: bun run test:eval (or ANTHROPIC_API_KEY=sk-... bun test test/skill-llm-eval.test.ts)

2026-03-13 20:23:31 -05:00

..

helpers

feat: DX tools (skill:check, dev:skill) + Tier 2 E2E test scaffolding

2026-03-13 20:23:18 -05:00

gen-skill-docs.test.ts

test: quality evals for generated SKILL.md descriptions

2026-03-13 20:23:30 -05:00

skill-e2e.test.ts

feat: DX tools (skill:check, dev:skill) + Tier 2 E2E test scaffolding

2026-03-13 20:23:18 -05:00

skill-llm-eval.test.ts

feat: LLM-as-judge evals for SKILL.md documentation quality

2026-03-13 20:23:31 -05:00

skill-parser.test.ts

test: Tier 1 static validation — 34 tests for SKILL.md command correctness

2026-03-13 20:23:18 -05:00

skill-validation.test.ts

test: Tier 1 static validation — 34 tests for SKILL.md command correctness

2026-03-13 20:23:18 -05:00