2 Commits

Author	SHA1	Message	Date
Garry Tan	620f5dbaea	test: /review hardening — NOT-READY env isolation, workdir cleanup, perf ... Applied from the adversarial subagent pass during /review on this branch: - test/benchmark-cli.test.ts — new "NOT READY path fires when auth env vars are stripped" test. The default dry-run test always showed OK on dev machines with auth, hiding regressions in the remediation-hint branch. Stripped env (no auth vars, HOME→empty tmpdir) now force- exercises gpt + gemini NOT READY paths and asserts every NOT READY line includes a concrete remediation hint (install/login/export). (claude adapter's os.homedir() call is Bun-cached; the 2-of-3 adapter coverage is sufficient to exercise the branch.) - test/taste-engine.test.ts — session-cap test rewritten to seed the profile with 50 entries + one real CLI call, instead of 55 sequential subprocess spawns. Same coverage (FIFO eviction at the boundary), ~5s faster CI time. Also pins first-casing-wins on the Geist/GEIST merge assertion — bumpPref() keeps the first-arrival casing, so the test documents that policy. - test/skill-e2e-benchmark-providers.test.ts — workdir creation moved from module-load into beforeAll, cleanup added in afterAll. Previous shape leaked a /tmp/bench-e2e-* dir every CI run. - test/publish-dry-run.test.ts — removed unused empty test/helpers mkdirSync from the sandbox setup. The bin doesn't import from there, so the empty dir was a footgun for future maintainers. - test/helpers/providers/gpt.ts — expanded the inline comment on `--skip-git-repo-check` to explicitly note that `-s read-only` is now load-bearing safety (the trust prompt was the secondary boundary; removing read-only while keeping skip-git-repo-check would be unsafe). Net: 45 passing tests (was 44), session-cap test 5s faster, one real regression surface covered that didn't exist before. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-18 06:54:09 +08:00
Garry Tan	c875e0c3fc	test: lite E2E coverage for benchmark, taste engine, publish ... Fills real coverage gaps in v0.19.0.0 primitives. 44 new deterministic tests (gate tier, ~3s) + 8 live-API tests (periodic tier). New gate-tier test files (free, <3s total): - test/taste-engine.test.ts — 24 tests against gstack-taste-update: schema shape, Laplace-smoothed confidence, 5%/week decay clamped at 0, multi-dimension extraction, case-insensitive matching, session cap, legacy profile migration with session truncation, taste-drift conflict warning, malformed-JSON recovery, missing-variant exit code. - test/publish-dry-run.test.ts — 13 tests against gstack-publish --dry-run: manifest parsing, missing/malformed JSON, per-skill validation errors (missing source file / slug / version / marketplaces), slug filter, unknown-skill exit, per-marketplace auth isolation (fake marketplaces with always-pass / always-fail / missing-binary CLIs), and a sanity check against the real repo manifest. - test/benchmark-cli.test.ts — 11 tests against gstack-model-benchmark --dry-run: provider default, unknown-provider WARN, empty list fallback, flag passthrough (timeout/workdir/judge/output), long-prompt truncation, prompt resolution (inline vs file vs positional), missing prompt exit. New periodic-tier test file (paid, gated EVALS=1): - test/skill-e2e-benchmark-providers.test.ts — 8 tests hitting real claude, codex, gemini CLIs with a trivial prompt (~$0.001/provider). Verifies output parsing, token accounting, cost estimation, timeout error.code semantics, Promise.allSettled parallel isolation. Per-provider availability gate — unauthed providers skip cleanly. This suite already caught one real bug (codex adapter missing --skip-git-repo-check, fixed in 5260987d). Registered `benchmark-providers-live` in touchfiles.ts (periodic tier, triggered by changes to bin/gstack-model-benchmark, providers/**, benchmark-runner.ts, pricing.ts). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-18 06:45:06 +08:00