CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-08-10 00:00:39 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: wire detached-eval guidance into /ship + correct CLAUDE.md flags

Browse Source 
- /ship eval step (sections/tests.md): long eval suites launch via gstack-detach
  (own session, machine lock, EXIT sentinel) so a turn boundary can't kill a
  30+ min run mid-ship — the exact failure observed during this branch's ship.
- CLAUDE.md: correct the now-stale /tmp reference; document the --lock (serialize
  worktrees, no API saturation), --timeout watchdog, run-scoped log, and the
  guaranteed EXIT sentinel the poller breaks on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-06-12 06:24:39 -07:00
co-authored by Claude Fable 5
parent 6eea549121
commit a38089aac6
3 changed files with 45 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+13 -6
View File
@@ -839,13 +839,20 @@ the run can also die to idle-sleep. `gstack-detach` fixes both: a fresh session
(escapes the group SIGTERM) wrapped in `caffeinate -i` (blocks idle-sleep).
- Use the `eval:bg*` scripts (`eval:bg`, `eval:bg:all`, `eval:bg:gate`,
`eval:bg:periodic`) — they wrap the eval command in `gstack-detach` and stream to
`/tmp/gstack-evals.log`. Or call `gstack-detach <log> -- <cmd>` directly for any
long agent job. Export `ANTHROPIC_API_KEY` first (never pass keys in argv).
- Then **poll the logfile** with a death-aware watcher: break on a `### DONE ###` /
`EXIT=` marker OR on "runner process gone before DONE" (silence is not success —
cover the crash/kill path, not just the happy path). The detached run survives
`eval:bg:periodic`) — they wrap the eval command in `gstack-detach` with the
machine-wide `gstack-evals` lock (concurrent worktrees serialize instead of
saturating the shared model API), a per-tier watchdog, and a **run-scoped** log
under `~/.gstack-dev/eval-runs/` (no shared-`/tmp` collision). Each prints its
log path. Or call `gstack-detach [--lock NAME] [--timeout SECS] [--label LBL] --
<cmd>` directly for any long agent job. Export `ANTHROPIC_API_KEY` first (never
pass keys in argv).
- Then **poll the printed logfile** with a death-aware watcher: break on the
guaranteed `### gstack-detach EXIT=<code> ###` sentinel (success AND failure are
both marked, so silence is never mistaken for success). The detached run survives
even if your watcher gets reaped, so re-checking the log always works.
- Why the lock: a shared dev box with several Conductor worktrees will rate-limit
the model API if two eval suites run at once (15-way concurrency each), which
mass-times-out E2E tests. The lock makes the second run WAIT, not collide.
- Humans running `bun run test:evals` foreground in their own terminal don't need
this — Ctrl-C is intended there. Detachment is for agent-launched runs only.