docs: post-ship documentation sync (v1.67.2.0)

Sol exact-match and near-miss warning documented in README; CODEX_HOME-aware
uninstall and troubleshooting paths; hermetic auth.json-only detail and the
build-clobber gotcha in CLAUDE.md; eval-store location corrected in
ARCHITECTURE.md; defaultModel row in the ADDING_A_HOST field reference;
resolver test count corrected in the CHANGELOG entry.
This commit is contained in:
Garry Tan
2026-08-18 16:41:22 -07:00
parent a7632d20be
commit 8011b51ae6
6 changed files with 20 additions and 9 deletions
+2 -2
View File
@@ -415,7 +415,7 @@ The `parseNDJSON()` function is pure — no I/O, no side effects — making it i
**Machine-readable diagnostics:** Each test result includes `exit_reason` (success, timeout, error_max_turns, error_api, exit_code_N), `timeout_at_turn`, and `last_tool_call`. This enables `jq` queries like:
```bash
jq '.tests[] | select(.exit_reason == "timeout") | .last_tool_call' ~/.gstack-dev/evals/_partial-e2e.json
jq '.tests[] | select(.exit_reason == "timeout") | .last_tool_call' ~/.gstack/projects/<slug>/evals/_partial-e2e.json
```
### Eval persistence (`test/helpers/eval-store.ts`)
@@ -425,7 +425,7 @@ The `EvalCollector` accumulates test results and writes them in two ways:
1. **Incremental:** `savePartial()` writes `_partial-e2e.json` after each test (atomic: write `.tmp`, `fs.renameSync`). Survives kills.
2. **Final:** `finalize()` writes a timestamped eval file (e.g. `e2e-20260314-143022.json`). The partial file is never cleaned up — it persists alongside the final file for observability.
`eval:compare` diffs two eval runs. `eval:summary` aggregates stats across all runs in `~/.gstack-dev/evals/`. Both are shard-aware (v1.63.0.0): the sharded paid runner (`scripts/test-paid-shards.ts`, run via `test:gate:sharded` / `test:periodic:sharded` — the `eval:bg:gate` / `eval:bg:periodic` scripts now point at these) gives each shard's collector its own directory at `<evalDir>/shards/<slug>/` through the `GSTACK_EVAL_DIR` env var (honored by the `EvalCollector` constructor), and `eval:list` / `eval:compare` / `eval:summary` scan one level of `shards/<slug>/` subdirectories. Baseline lookups exclude `_partial` accumulators (`isPartialEval` / `findLatestFinalizedRun` in `eval-store.ts`), so auto-comparison never uses the current run's own partial file as its baseline.
`eval:compare` diffs two eval runs. `eval:summary` aggregates stats across all runs in `~/.gstack/projects/<slug>/evals/` (legacy fallback `~/.gstack-dev/evals/`). Both are shard-aware (v1.63.0.0): the sharded paid runner (`scripts/test-paid-shards.ts`, run via `test:gate:sharded` / `test:periodic:sharded` — the `eval:bg:gate` / `eval:bg:periodic` scripts now point at these) gives each shard's collector its own directory at `<evalDir>/shards/<slug>/` through the `GSTACK_EVAL_DIR` env var (honored by the `EvalCollector` constructor), and `eval:list` / `eval:compare` / `eval:summary` scan one level of `shards/<slug>/` subdirectories. Baseline lookups exclude `_partial` accumulators (`isPartialEval` / `findLatestFinalizedRun` in `eval-store.ts`), so auto-comparison never uses the current run's own partial file as its baseline.
### Test tiers
+1 -1
View File
@@ -44,7 +44,7 @@ If you run Codex on `gpt-5.6-sol`, rerun `./setup --host codex` once. Your skill
- `setup` resolves the Codex generation model on every run (a read-only TOML lookup), so any install path preserves a Sol user's rendered profile; the codex install summary prints the active profile and its source.
#### For contributors
- New free-tier coverage: every resolver branch including hostile-config shapes (12 tests), overlay content pins, the explicit `--model` override through the real generation CLI, real-file periodic-tier classification for both codex E2E files, and invalid `defaultModel` validation.
- New free-tier coverage: every resolver branch including hostile-config shapes (10 tests), overlay content pins, the explicit `--model` override through the real generation CLI, real-file periodic-tier classification for both codex E2E files, and invalid `defaultModel` validation.
- Static pins in `test/setup-codex-model.test.ts` hold the load-bearing setup properties: unconditional resolver, quoted `--explicit` argv, fail-closed empty-resolver exit, the Kiro claude-render sandwich, and the `--host kiro` rewrite.
- The Sol E2E snapshots the exact prior `.agents` render and restores it in `beforeAll`, so the shared tree never stays Sol-flavored for goldens, parallel shards, or symlinked installs. Fixture commits disable gpg signing so the eval runs under any global git config.
+9 -3
View File
@@ -26,8 +26,10 @@ bun run slop # full slop-scan report (all files)
bun run slop:diff # slop findings in files changed on this branch only
```
`test:evals` requires `ANTHROPIC_API_KEY`. Codex E2E tests (`test/codex-e2e.test.ts`)
use Codex's own auth from `~/.codex/` config — no `OPENAI_API_KEY` env var needed.
`test:evals` requires `ANTHROPIC_API_KEY`. Codex E2E tests (`test/codex-e2e.test.ts`,
`test/codex-e2e-sol-scope.test.ts`) use Codex's own auth — the hermetic runner copies
only `auth.json` from `${CODEX_HOME:-~/.codex}` and pins `CODEX_HOME` in the child
env — no `OPENAI_API_KEY` env var needed.
**Env keys in Conductor workspaces.** The `GSTACK_*` env-shim (v1.39.2.0+,
`lib/conductor-env-shim.ts`) promotes `GSTACK_ANTHROPIC_API_KEY` /
@@ -194,7 +196,11 @@ SKILL.md files are **generated** from `.tmpl` templates. To update docs:
Generation uses each host's `defaultModel` (`claude` for existing hosts, `gpt`
for Codex) unless `--model` is explicit. Codex installs additionally read the
top-level model from `${CODEX_HOME:-~/.codex}/config.toml`; rerun
`./setup --host codex` after changing that model.
`./setup --host codex` after changing that model. Note: `bun run build` and a
bare `gen:skill-docs --host codex` render the host default (gpt) — if your
Codex config.toml pins a different model, rerun `./setup --host codex`
afterwards to restore your profile (single-owner persistence is filed in
TODOS.md).
To add a new browse command: add it to `browse/src/commands.ts` and rebuild.
To add a snapshot flag: add it to `SNAPSHOT_FLAGS` in `browse/src/snapshot.ts` and rebuild.
+5 -3
View File
@@ -126,7 +126,9 @@ For Codex, setup reads the top-level `model` from
`${CODEX_HOME:-~/.codex}/config.toml` and generates the matching behavioral
profile. `gpt-5.6-sol` automatically receives bounded-scope instructions that
finish the requested lake without expanding into adjacent cleanup or speculative
hardening. Override detection with `./setup --host codex --model <id>` — the
hardening. The Sol profile is exact-match only: dated snapshots and other 5.6
variants get the generic GPT profile, and setup warns on near-misses like
`gpt-5.6-sol-2026-08-01`. Override detection with `./setup --host codex --model <id>` — the
override applies to that run only; set `model` in your Codex `config.toml` to
make it stick across upgrades. After changing your Codex model, rerun
`./setup --host codex` to regenerate the skills.
@@ -382,7 +384,7 @@ rm -rf ~/.claude/skills/gstack
rm -rf ~/.gstack
# 5. Remove integrations (skip any you never installed)
rm -rf ~/.codex/skills/gstack* 2>/dev/null
rm -rf "${CODEX_HOME:-$HOME/.codex}/skills/gstack"* 2>/dev/null
rm -rf ~/.factory/skills/gstack* 2>/dev/null
rm -rf ~/.kiro/skills/gstack* 2>/dev/null
rm -rf ~/.openclaw/skills/gstack* 2>/dev/null
@@ -499,7 +501,7 @@ Data is stored in [Supabase](https://supabase.com) (open source Firebase alterna
**Want namespaced commands?** `cd ~/.claude/skills/gstack && ./setup --prefix` — switches from `/qa` to `/gstack-qa`. Useful if you run other skill packs alongside gstack.
**Codex says "Skipped loading skill(s) due to invalid SKILL.md"?** Your Codex skill descriptions are stale. Fix: `cd ~/.codex/skills/gstack && git pull && ./setup --host codex` — or for repo-local installs: `cd "$(readlink -f .agents/skills/gstack)" && git pull && ./setup --host codex`
**Codex says "Skipped loading skill(s) due to invalid SKILL.md"?** Your Codex skill descriptions are stale. Fix: `cd "${CODEX_HOME:-$HOME/.codex}/skills/gstack" && git pull && ./setup --host codex` — or for repo-local installs: `cd "$(readlink -f .agents/skills/gstack)" && git pull && ./setup --host codex`
**Windows users:** gstack works on Windows 11 via Git Bash or WSL. Node.js is required in addition to Bun — Bun has a known bug with Playwright's pipe transport on Windows ([bun#4253](https://github.com/oven-sh/bun/issues/4253)). The browse server automatically falls back to Node.js. Make sure both `bun` and `node` are on your PATH.
+2
View File
@@ -815,6 +815,8 @@ or reject the value with guidance like openclaw/hermes/gbrain get.
**Why:** Silent success with zero effect is the worst failure shape.
**Priority:** P4. **Effort:** S.
---
## browse server: terminal-agent teardown follow-ups (filed v1.41 via /plan-eng-review)
### ✅ DONE (v1.44.0.0): Identity-based terminal-agent kill (replace pkill regex with PID)
+1
View File
@@ -157,6 +157,7 @@ Key fields:
| Field | Purpose |
|-------|---------|
| `defaultModel` | Model overlay rendered when generation gets no explicit `--model` (validated against `ALL_MODEL_NAMES` in `scripts/models.ts`) |
| `frontmatter.mode` | `allowlist` (keep only listed) or `denylist` (strip listed) |
| `frontmatter.descriptionLimit` | Max chars, `null` for no limit |
| `frontmatter.descriptionLimitBehavior` | `error` (fail build), `truncate`, `warn` |