mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-10 06:58:59 +02:00
Garry TanandClaude Fable 5
967c71d32c
Moved: browser/sidebar/server internals, CHANGELOG release-summary format spec, project tree, hermetic-E2E detail, slop-scan reference, OpenClaw publishing. Kept inline: every hard behavioral rule (dist/ ban, redaction scan-at-sink, egress receipts, bisect commits, eval detach, CHANGELOG entry rules), the machine-managed GBrain block (byte-identical), and the '## Deploying to the active skill' header with gbrain-refresh in range (pinned by test/gbrain-refresh-install-render.test.ts). No voice rewrites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
44 lines
2.7 KiB
Markdown
44 lines
2.7 KiB
Markdown
# Testing internals: env keys, hermetic E2E
|
|
|
|
Moved verbatim from CLAUDE.md (token-load reduction). Read this before
|
|
writing or debugging E2E tests, passing `env:` to a runner, or touching
|
|
`test/helpers/hermetic-env.ts`.
|
|
|
|
**Env keys in Conductor workspaces.** The `GSTACK_*` env-shim (v1.39.2.0+,
|
|
`lib/conductor-env-shim.ts`) promotes `GSTACK_ANTHROPIC_API_KEY` /
|
|
`GSTACK_OPENAI_API_KEY` to their canonical names inside gstack's TS binaries.
|
|
Tests run through gstack entrypoints inherit this promotion automatically.
|
|
Don't echo the key value to stdout, logs, or shell history. The historical
|
|
"never pass `env:` to `runAgentSdkTest`" rule is retired: the failure was
|
|
partial-env replacement (the SDK's `Options.env` REPLACES the child's entire
|
|
environment, so an object without the key broke auth). The runner now always
|
|
passes a COMPLETE hermetic env with per-test `env:` merged last, so per-test
|
|
overrides are safe; ambient `process.env.ANTHROPIC_API_KEY` mutation also
|
|
still works (the env builder reads process.env at call time).
|
|
|
|
**Hermetic local E2E (default).** Every E2E runner (claude -p, PTY, Agent
|
|
SDK, codex, gemini) spawns children through `test/helpers/hermetic-env.ts`:
|
|
allowlist-scrubbed env (operator `CONDUCTOR_*`, `CLAUDE_*`, `GSTACK_*`,
|
|
`MCP_*`, `GBRAIN_*`, and credentials like `GH_TOKEN` never reach children),
|
|
a fresh seeded `CLAUDE_CONFIG_DIR` (no operator `~/.claude` CLAUDE.md /
|
|
MCP servers / skills), a temp `GSTACK_HOME`, and `--strict-mcp-config`.
|
|
Local eval signal matches CI. Debug against real operator state with
|
|
`EVALS_HERMETIC=0` (restores the legacy env AND drops the strict-MCP flag).
|
|
Per-test `env:` overrides merge last, so deliberate contamination
|
|
(`CONDUCTOR_WORKSPACE_PATH`, per-test `GSTACK_HOME`) keeps working. The
|
|
hermetic config dir seeds NO skills by default; a PTY test that types a
|
|
`/skill` slash command must pass `seedSkills: true` to the PTY runner, which
|
|
points the child's `CLAUDE_CONFIG_DIR` at `hermeticSkillsConfigDir()` — a
|
|
seeded registry that symlinks the LIVE working tree's SKILL.md files (by
|
|
design: the skills ARE the subject under test; a snapshot would measure stale
|
|
copies). Wiring is pinned by `test/hermetic-wiring.test.ts` (static tripwire),
|
|
two gate-tier canaries in `test/skill-e2e-hermetic-canary.test.ts`, and the
|
|
seeding tripwires in `test/hermetic-skills-seeding.test.ts` /
|
|
`test/pty-skill-seeding-wiring.test.ts`.
|
|
|
|
E2E tests stream progress in real-time (tool-by-tool via `--output-format stream-json
|
|
--verbose`). Results are persisted to `~/.gstack/projects/<slug>/evals/` (legacy
|
|
fallback `~/.gstack-dev/evals/`) with auto-comparison
|
|
against the previous finalized run (in-flight `_partial` files are never used as
|
|
a baseline, so a run can't compare against itself).
|