mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-09 14:38:59 +02:00
docs: post-ship documentation sync (v1.67.2.0)
Sol exact-match and near-miss warning documented in README; CODEX_HOME-aware uninstall and troubleshooting paths; hermetic auth.json-only detail and the build-clobber gotcha in CLAUDE.md; eval-store location corrected in ARCHITECTURE.md; defaultModel row in the ADDING_A_HOST field reference; resolver test count corrected in the CHANGELOG entry.
This commit is contained in:
+2
-2
@@ -415,7 +415,7 @@ The `parseNDJSON()` function is pure — no I/O, no side effects — making it i
|
||||
|
||||
**Machine-readable diagnostics:** Each test result includes `exit_reason` (success, timeout, error_max_turns, error_api, exit_code_N), `timeout_at_turn`, and `last_tool_call`. This enables `jq` queries like:
|
||||
```bash
|
||||
jq '.tests[] | select(.exit_reason == "timeout") | .last_tool_call' ~/.gstack-dev/evals/_partial-e2e.json
|
||||
jq '.tests[] | select(.exit_reason == "timeout") | .last_tool_call' ~/.gstack/projects/<slug>/evals/_partial-e2e.json
|
||||
```
|
||||
|
||||
### Eval persistence (`test/helpers/eval-store.ts`)
|
||||
@@ -425,7 +425,7 @@ The `EvalCollector` accumulates test results and writes them in two ways:
|
||||
1. **Incremental:** `savePartial()` writes `_partial-e2e.json` after each test (atomic: write `.tmp`, `fs.renameSync`). Survives kills.
|
||||
2. **Final:** `finalize()` writes a timestamped eval file (e.g. `e2e-20260314-143022.json`). The partial file is never cleaned up — it persists alongside the final file for observability.
|
||||
|
||||
`eval:compare` diffs two eval runs. `eval:summary` aggregates stats across all runs in `~/.gstack-dev/evals/`. Both are shard-aware (v1.63.0.0): the sharded paid runner (`scripts/test-paid-shards.ts`, run via `test:gate:sharded` / `test:periodic:sharded` — the `eval:bg:gate` / `eval:bg:periodic` scripts now point at these) gives each shard's collector its own directory at `<evalDir>/shards/<slug>/` through the `GSTACK_EVAL_DIR` env var (honored by the `EvalCollector` constructor), and `eval:list` / `eval:compare` / `eval:summary` scan one level of `shards/<slug>/` subdirectories. Baseline lookups exclude `_partial` accumulators (`isPartialEval` / `findLatestFinalizedRun` in `eval-store.ts`), so auto-comparison never uses the current run's own partial file as its baseline.
|
||||
`eval:compare` diffs two eval runs. `eval:summary` aggregates stats across all runs in `~/.gstack/projects/<slug>/evals/` (legacy fallback `~/.gstack-dev/evals/`). Both are shard-aware (v1.63.0.0): the sharded paid runner (`scripts/test-paid-shards.ts`, run via `test:gate:sharded` / `test:periodic:sharded` — the `eval:bg:gate` / `eval:bg:periodic` scripts now point at these) gives each shard's collector its own directory at `<evalDir>/shards/<slug>/` through the `GSTACK_EVAL_DIR` env var (honored by the `EvalCollector` constructor), and `eval:list` / `eval:compare` / `eval:summary` scan one level of `shards/<slug>/` subdirectories. Baseline lookups exclude `_partial` accumulators (`isPartialEval` / `findLatestFinalizedRun` in `eval-store.ts`), so auto-comparison never uses the current run's own partial file as its baseline.
|
||||
|
||||
### Test tiers
|
||||
|
||||
|
||||
+1
-1
@@ -44,7 +44,7 @@ If you run Codex on `gpt-5.6-sol`, rerun `./setup --host codex` once. Your skill
|
||||
- `setup` resolves the Codex generation model on every run (a read-only TOML lookup), so any install path preserves a Sol user's rendered profile; the codex install summary prints the active profile and its source.
|
||||
|
||||
#### For contributors
|
||||
- New free-tier coverage: every resolver branch including hostile-config shapes (12 tests), overlay content pins, the explicit `--model` override through the real generation CLI, real-file periodic-tier classification for both codex E2E files, and invalid `defaultModel` validation.
|
||||
- New free-tier coverage: every resolver branch including hostile-config shapes (10 tests), overlay content pins, the explicit `--model` override through the real generation CLI, real-file periodic-tier classification for both codex E2E files, and invalid `defaultModel` validation.
|
||||
- Static pins in `test/setup-codex-model.test.ts` hold the load-bearing setup properties: unconditional resolver, quoted `--explicit` argv, fail-closed empty-resolver exit, the Kiro claude-render sandwich, and the `--host kiro` rewrite.
|
||||
- The Sol E2E snapshots the exact prior `.agents` render and restores it in `beforeAll`, so the shared tree never stays Sol-flavored for goldens, parallel shards, or symlinked installs. Fixture commits disable gpg signing so the eval runs under any global git config.
|
||||
|
||||
|
||||
@@ -26,8 +26,10 @@ bun run slop # full slop-scan report (all files)
|
||||
bun run slop:diff # slop findings in files changed on this branch only
|
||||
```
|
||||
|
||||
`test:evals` requires `ANTHROPIC_API_KEY`. Codex E2E tests (`test/codex-e2e.test.ts`)
|
||||
use Codex's own auth from `~/.codex/` config — no `OPENAI_API_KEY` env var needed.
|
||||
`test:evals` requires `ANTHROPIC_API_KEY`. Codex E2E tests (`test/codex-e2e.test.ts`,
|
||||
`test/codex-e2e-sol-scope.test.ts`) use Codex's own auth — the hermetic runner copies
|
||||
only `auth.json` from `${CODEX_HOME:-~/.codex}` and pins `CODEX_HOME` in the child
|
||||
env — no `OPENAI_API_KEY` env var needed.
|
||||
|
||||
**Env keys in Conductor workspaces.** The `GSTACK_*` env-shim (v1.39.2.0+,
|
||||
`lib/conductor-env-shim.ts`) promotes `GSTACK_ANTHROPIC_API_KEY` /
|
||||
@@ -194,7 +196,11 @@ SKILL.md files are **generated** from `.tmpl` templates. To update docs:
|
||||
Generation uses each host's `defaultModel` (`claude` for existing hosts, `gpt`
|
||||
for Codex) unless `--model` is explicit. Codex installs additionally read the
|
||||
top-level model from `${CODEX_HOME:-~/.codex}/config.toml`; rerun
|
||||
`./setup --host codex` after changing that model.
|
||||
`./setup --host codex` after changing that model. Note: `bun run build` and a
|
||||
bare `gen:skill-docs --host codex` render the host default (gpt) — if your
|
||||
Codex config.toml pins a different model, rerun `./setup --host codex`
|
||||
afterwards to restore your profile (single-owner persistence is filed in
|
||||
TODOS.md).
|
||||
|
||||
To add a new browse command: add it to `browse/src/commands.ts` and rebuild.
|
||||
To add a snapshot flag: add it to `SNAPSHOT_FLAGS` in `browse/src/snapshot.ts` and rebuild.
|
||||
|
||||
@@ -126,7 +126,9 @@ For Codex, setup reads the top-level `model` from
|
||||
`${CODEX_HOME:-~/.codex}/config.toml` and generates the matching behavioral
|
||||
profile. `gpt-5.6-sol` automatically receives bounded-scope instructions that
|
||||
finish the requested lake without expanding into adjacent cleanup or speculative
|
||||
hardening. Override detection with `./setup --host codex --model <id>` — the
|
||||
hardening. The Sol profile is exact-match only: dated snapshots and other 5.6
|
||||
variants get the generic GPT profile, and setup warns on near-misses like
|
||||
`gpt-5.6-sol-2026-08-01`. Override detection with `./setup --host codex --model <id>` — the
|
||||
override applies to that run only; set `model` in your Codex `config.toml` to
|
||||
make it stick across upgrades. After changing your Codex model, rerun
|
||||
`./setup --host codex` to regenerate the skills.
|
||||
@@ -382,7 +384,7 @@ rm -rf ~/.claude/skills/gstack
|
||||
rm -rf ~/.gstack
|
||||
|
||||
# 5. Remove integrations (skip any you never installed)
|
||||
rm -rf ~/.codex/skills/gstack* 2>/dev/null
|
||||
rm -rf "${CODEX_HOME:-$HOME/.codex}/skills/gstack"* 2>/dev/null
|
||||
rm -rf ~/.factory/skills/gstack* 2>/dev/null
|
||||
rm -rf ~/.kiro/skills/gstack* 2>/dev/null
|
||||
rm -rf ~/.openclaw/skills/gstack* 2>/dev/null
|
||||
@@ -499,7 +501,7 @@ Data is stored in [Supabase](https://supabase.com) (open source Firebase alterna
|
||||
|
||||
**Want namespaced commands?** `cd ~/.claude/skills/gstack && ./setup --prefix` — switches from `/qa` to `/gstack-qa`. Useful if you run other skill packs alongside gstack.
|
||||
|
||||
**Codex says "Skipped loading skill(s) due to invalid SKILL.md"?** Your Codex skill descriptions are stale. Fix: `cd ~/.codex/skills/gstack && git pull && ./setup --host codex` — or for repo-local installs: `cd "$(readlink -f .agents/skills/gstack)" && git pull && ./setup --host codex`
|
||||
**Codex says "Skipped loading skill(s) due to invalid SKILL.md"?** Your Codex skill descriptions are stale. Fix: `cd "${CODEX_HOME:-$HOME/.codex}/skills/gstack" && git pull && ./setup --host codex` — or for repo-local installs: `cd "$(readlink -f .agents/skills/gstack)" && git pull && ./setup --host codex`
|
||||
|
||||
**Windows users:** gstack works on Windows 11 via Git Bash or WSL. Node.js is required in addition to Bun — Bun has a known bug with Playwright's pipe transport on Windows ([bun#4253](https://github.com/oven-sh/bun/issues/4253)). The browse server automatically falls back to Node.js. Make sure both `bun` and `node` are on your PATH.
|
||||
|
||||
|
||||
@@ -815,6 +815,8 @@ or reject the value with guidance like openclaw/hermes/gbrain get.
|
||||
**Why:** Silent success with zero effect is the worst failure shape.
|
||||
**Priority:** P4. **Effort:** S.
|
||||
|
||||
---
|
||||
|
||||
## browse server: terminal-agent teardown follow-ups (filed v1.41 via /plan-eng-review)
|
||||
|
||||
### ✅ DONE (v1.44.0.0): Identity-based terminal-agent kill (replace pkill regex with PID)
|
||||
|
||||
@@ -157,6 +157,7 @@ Key fields:
|
||||
|
||||
| Field | Purpose |
|
||||
|-------|---------|
|
||||
| `defaultModel` | Model overlay rendered when generation gets no explicit `--model` (validated against `ALL_MODEL_NAMES` in `scripts/models.ts`) |
|
||||
| `frontmatter.mode` | `allowlist` (keep only listed) or `denylist` (strip listed) |
|
||||
| `frontmatter.descriptionLimit` | Max chars, `null` for no limit |
|
||||
| `frontmatter.descriptionLimitBehavior` | `error` (fail build), `truncate`, `warn` |
|
||||
|
||||
Reference in New Issue
Block a user