From 3139cf40fbf7fd1d87846edd35e4b361e831bfc8 Mon Sep 17 00:00:00 2001
From: Garry Tan <garrytan@gmail.com>
Date: Fri, 24 Apr 2026 01:26:24 -0700
Subject: [PATCH] docs(CLAUDE.md): source ANTHROPIC/OPENAI keys from ~/.zshrc
 for paid evals
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Conductor workspaces don't inherit the interactive shell env, so
both API keys are absent from the default process env even though
they're set in ~/.zshrc. Documents the source-from-zshrc pattern
(grep + eval, never echo the value) plus the Agent SDK gotcha: do
NOT pass env as an object to runAgentSdkTest — mutate process.env
ambiently and restore in finally.

Discovered this during the brain-privacy-gate E2E. First run failed
at SDK auth with 401; second failed because explicit env handoff
bypassed the SDK's own auth routing. Fix pattern now codified so
the next paid-eval session in a Conductor workspace doesn't hit the
same two dead ends.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---
 CLAUDE.md | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/CLAUDE.md b/CLAUDE.md
index ca1c5b99..dfe9df23 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -26,6 +26,26 @@ bun run slop:diff     # slop findings in files changed on this branch only
 
 `test:evals` requires `ANTHROPIC_API_KEY`. Codex E2E tests (`test/codex-e2e.test.ts`)
 use Codex's own auth from `~/.codex/` config — no `OPENAI_API_KEY` env var needed.
+
+**Where the keys live on this machine.** Conductor workspaces don't inherit the
+user's interactive shell env, so `ANTHROPIC_API_KEY` and `OPENAI_API_KEY` aren't
+in the default process env. Before running any paid eval / E2E, source them from
+`~/.zshrc` (that's where Garry keeps them):
+
+```bash
+bash -c '
+  eval "$(grep -E "^export (ANTHROPIC_API_KEY|OPENAI_API_KEY)=" ~/.zshrc)"
+  export ANTHROPIC_API_KEY OPENAI_API_KEY
+  EVALS=1 EVALS_TIER=periodic bun test test/skill-e2e-<whatever>.test.ts
+'
+```
+
+Do not echo the key value anywhere (stdout, logs, shell history). The grep+eval
+pattern keeps it in process env only. When passing to a test's Agent SDK, do NOT
+pass `env: {...}` to `runAgentSdkTest` — the SDK's auth pipeline doesn't pick up
+the key the same way when env is supplied as an object (confirmed failure mode).
+Instead, mutate `process.env.ANTHROPIC_API_KEY` ambiently before the call and
+restore in `finally`.
 E2E tests stream progress in real-time (tool-by-tool via `--output-format stream-json
 --verbose`). Results are persisted to `~/.gstack-dev/evals/` with auto-comparison
 against the previous run.