mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-16 09:55:29 +02:00
* fix(freeze): hook reads the same state root /freeze writes — fails closed under GSTACK_HOME (#1459, #1509) check-freeze.sh resolved its state dir as ${CLAUDE_PLUGIN_DATA:-$HOME/.gstack} while every writer (/freeze, /guard, /unfreeze, /investigate) resolves through bin/gstack-paths, GSTACK_HOME first. With GSTACK_HOME set, /freeze wrote freeze-dir.txt under GSTACK_HOME, the hook read $HOME/.gstack, found no file, and allowed everything — a deny-tier boundary failing open. One resolver now: gstack_hook_state_root() in careful/bin/hook-extract.sh (already sourced by both check-freeze.sh and check-careful.sh) implements the exact gstack-paths chain, including the CLAUDE_PLUGIN_ROOT guard that keeps a CLAUDE_PLUGIN_DATA leaked from another plugin from redirecting our state. check-freeze.sh and gstack_hook_log_fire both call it; nothing spawns gstack-paths from a hook. Tests: the GSTACK_HOME deny regression, GSTACK_HOME-over-CLAUDE_PLUGIN_DATA precedence, plugin-root guard both ways, and a byte-parity check against bin/gstack-paths across six env combinations. Existing freeze tests now pass CLAUDE_PLUGIN_ROOT like a real plugin install would. Idea from PR #1509 (@NikhileshNanduri); implemented natively against the shared resolver rather than a second fallback chain. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(relink): never delete or link over a skill gstack does not own (#2119) gstack-relink runs on every ./setup. Its cleanup did `rm -rf` on any same-name entry whose SKILL.md was a symlink, with no readlink check, and its link step did `mkdir -p` then `ln -snf` onto any existing SKILL.md — on Linux that replaces a user's real file with a symlink into gstack (macOS refused by accident). setup's Windows mode-flip cleanup deleted any real dir whose name matched a gstack skill. A personal `qa` skill, or a fork installed under another path, was destroyed by the installer of a tool it never asked for. Ownership is now proven, never assumed. An entry is ours when it is a symlink resolving into INSTALL_DIR or RENDER_DIR, a real dir whose SKILL.md is such a symlink, or a real dir carrying the .gstack-owned marker setup now writes for Windows copy installs (legacy copies count when byte-identical to the source or carrying gen-skill-docs' AUTO-GENERATED header). Anything else — including an entry whose readlink fails — is foreign: left untouched, reported on stderr, and listed in relink's summary line. The same rule replaces setup's Windows name-match deletion; setup:1040 and gstack-uninstall:204 already gated on readlink, so this closes the last unguarded deleter of the class. Tests: foreign real dir in flat mode, foreign flat entry on a prefix flip, foreign directory symlink, RENDER_DIR-targeted entry (ours), marker-carrying copy (ours), marker-less copy (foreign); the Windows cleanup test now proves provenance three ways and keeps the user's own same-name skill. Idea and two regression cases from PR #2119 (@smblight); implemented on the destination entry, not only the symlink target. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(setup): Chromium bootstrap is best-effort and bounded — skills always register (#1900, #1901, #1902, #913, #2233) setup runs under `set -e`, and the Chromium bootstrap in section 2 sat ahead of skill registration in section 4 with a bare `bunx playwright install chromium`, an unbounded download, and an explicit `exit 1` after the post-install launch probe. On an offline, proxied, or AppArmor-restricted box the user ended with ZERO skills registered and a re-run that died at the same line; a wedged download hung setup indefinitely. Every browser failure now records a reason code in _PW_FAIL_REASON and setup continues: skipped (GSTACK_SKIP_PLAYWRIGHT=1, #913), chromium-install, chromium-install-timeout (the download is bounded by the existing _wait_with_deadline helper, default 600s, env GSTACK_PLAYWRIGHT_INSTALL_TIMEOUT, process tree killed via _kill_tree), chromium-install-locked (another setup holds the lock: this one registers skills and re-probes next time instead of exiting), windows-no-node, windows-node-modules, post-install-launch (with the GSTACK_CHROMIUM_NO_SANDBOX=1 hint for Ubuntu 24.04's userns policy, #2157). The daemon font refresh is skipped when Chromium is unavailable. The final summary names the skills that need the browser (/qa, /qa-only, /design-review, /browse, make-pdf, /pair-agent) and the fix for the recorded reason, and logs the reason code (never a path) through gstack-telemetry-log when telemetry is on. Tests: static invariants over the anchor-sliced block (no exit, every reason code, deadline helper, trap chaining, guarded refresh, summary contents) plus an integration harness that executes the real block with a stubbed probe and installer: install failure, hang killed at the deadline with the tree kill recorded, non-numeric knob fallback, live lock (continues, installer not run, lock preserved), stale lock reclaimed, post-install probe failure, and the skip flag. Credit @DavidMiserak (PR #1900) for the best-effort shape; re-implemented on the current block. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(designs): preserve the time-attack fork-port residual evaluation The read-only evaluation of what remains portable from time-attack/gstack (583 raw candidates, 415 canonical, 287 with a residual, 48 adversarially refuted, 14 standing) lived only on a throwaway VM. This records the report, the lite residual index, the absorbed/superseded ledger, the refuter verdicts, and SHAS.md with the fork tip, upstream HEAD, merge-base, and a sha256 per file, so every scheduled fix in this wave series traces to its evidence. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: file the fork-port residual deferrals and document the Chromium bootstrap knobs TODOS.md gains the seven items the CEO and eng reviews of the fork-port residual plan deliberately deferred (shared ownership helper, config-key reader tripwire, "pre-existing" vocabulary, opt-in reply_language, .auth.json writer removal, the fork-derived-change rule for CONTRIBUTING, hook slug parity audit), each with rationale, and updates the two residual bullets for PR #2232 and PR #2233 with their dispositions. README's Troubleshooting section explains the best-effort Chromium bootstrap and its three knobs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(relink): canonicalize link targets before the ownership check Pre-landing review finding: the ownership gate compared readlink output textually against INSTALL_DIR and RENDER_DIR, so two shapes of gstack's OWN entries read as foreign and were left behind on a mode flip — a legacy relative link (`gstack/qa/SKILL.md`, resolved against $PWD instead of the link's directory) and an entry linked against the real path of a symlinked install dir (~/.claude/skills/gstack -> checkout). Both now resolve: relative targets anchor at the link's directory, the directory part is canonicalized with pwd -P (the basename stays verbatim so a dangling managed target is not misread), and both spellings of each root are accepted. Two regression tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(telemetry): one-shot setup events never sweep other sessions' pending markers gstack-telemetry-log finalizes every .pending-<session> marker that is not the caller's own as outcome:unknown and deletes it. setup's onboarding events (_setup_welcome, _setup_playwright) have no session of their own, so a Chromium bootstrap failure during a live skill session recorded a false unknown for that session and removed its marker. New --no-sweep flag skips the stale-marker pass; both setup call sites use it (the synthetic --session-id did not prevent the sweep). Surfaced by the Codex adversarial pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(hooks): partial upgrades fail closed for freeze and fall back for careful A hook script and its sourced helper can be copied at different times. With an older careful/bin/hook-extract.sh that lacks gstack_hook_state_root: - check-freeze.sh now emits a deny ("fail closed, re-run ./setup or /unfreeze") instead of dying under set -e with no decision JSON. - check-careful.sh falls back to ${GSTACK_HOME:-$HOME/.gstack} so project rules under the plain chain still load and a decision is always emitted (a warn hook must never break on a stale helper). gstack_hook_state_root prints its root without a trailing newline and both callers capture it with a printf-x sentinel, so a GSTACK_HOME ending in a newline round-trips byte-for-byte with the writer's %q form. gstack_hook_log_fire stays on ${GSTACK_HOME:-$HOME/.gstack}/analytics, the same two-step chain every other analytics writer and reader uses, so the usage log remains one file under a plugin install. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(setup): never link over, copy over, or reap a skill gstack does not own (#2119) The relink gate alone left three destructive sites open: - link_claude_skill_dirs runs BEFORE relink on every ./setup and used `ln -snf` (Linux replaces a user's real SKILL.md with a symlink into gstack) or, on Windows, rm -rf + cp followed by a marker that made the user's directory "ours" on the next flip. It and _install_alias_skill_md now consult _claude_entry_is_ours first and skip loudly. - cleanup_prefixed_claude_symlinks kept a bare name-match deletion and a `*gstack*` substring match. Symlink arms use anchored `gstack/` segment patterns; the Windows real-file arm proves provenance (marker, byte-identity with our source, or the full two-line gen-skill-docs banner within the first 40 lines, never a one-line substring another generator could emit). cleanup_old_claude_symlinks uses the same banner rule. - gstack-relink's fast path judged absolute targets before canonicalizing, so `/x/gstack/../foreign/SKILL.md` counted as ours; dot-segment targets now canonicalize first. Its banner rule matches setup's. The `.gstack-owned` marker records the owning payload's realpath. Entries skipped by setup or relink are listed in the final setup summary. Chromium bootstrap refinements from the pre-landing review: an INT/TERM trap kills the installer's process tree; the Windows npm chain no longer masks an install failure; GSTACK_SKIP_PLAYWRIGHT=1 is reported as a choice rather than a failure and sends no telemetry; the timeout knob is normalized (0, 000, non-numeric, or more than nine digits fall back to the 600s default instead of killing on the first poll or never killing). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: README Chromium note outside the CLAUDE.md fence; report banner stripped; deferrals name the four gate sites - README: the Chromium troubleshooting paragraph sat inside the CLAUDE.md snippet code fence, so copy-paste put it into users' CLAUDE.md. Moved to the troubleshooting list. - docs/designs/fork-port-residual-2026-09/REPORT.md: the scratch-run preamble banner is gone; SHAS.md re-hashed. - TODOS: the ownership-gate deferral names the four sites and the marker-path idea for the fork-with-banner residual. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(todos): the bootstrap block coverage gap is pinned except the quarantine helper Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(setup,relink): ownership proof has two strengths; weak proof never deletes a directory or discards a differing file The first #2119 gate treated a byte-identical or banner-bearing real-file SKILL.md as full ownership, so a prefix flip could rm -rf a user's directory (their own qa skill started from a gstack SKILL.md, plus my-templates/) and the link pass could replace their customized file with a symlink. Two strengths now: - STRONG: the .gstack-owned marker (we created the directory), or a directory holding nothing but symlinks and the marker (deleting it loses no data). Only strong proof removes a directory whole. - WEAK: byte-identity with our source or the two-line gen-skill-docs banner on a real file. Weak proof covers that SKILL.md and our runtime-asset links only; a differing file is moved to ${GSTACK_HOME:-~/.gstack}/backups/skills/<ts>/<skill>/ before we link over it, and setup/relink print one summary line naming what moved. The marker is written on every platform now (path-independent proof for Windows copies and for checkouts whose path carries no gstack segment), but only for a directory gstack creates: a directory we merely link into (unclaimed, or a legacy install) never becomes deletable whole. A directory with no SKILL.md at all is unclaimed: the link pass may add our file, the cleanup pass has nothing to remove. Also from the review passes: the banner check reads 8192 bytes, not 40 lines (investigate, office-hours, plan-ceo-review and design-consultation carry the banner past line 40 and were left "foreign" on pre-marker Windows installs); a link into a checkout named without a gstack segment (git worktree add ../gstack-<branch>) is ours when that tree carries setup + VERSION + bin/; relink's fast path is gone so both files canonicalize before judging; relink's root alias (_gstack-command) is gated and stamped like every other entry; relink reports the bare entry name with setup's wording and setup dedupes when forwarding (_run_relink_quiet); the summary names the browser skills as examples. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(setup): Chromium-install lock reclaim is atomic and pid-validated; abandoned locks expire; the tree kill walks /proc without pgrep - A pid file holding "", "-1" or "0" counted as a live holder (kill -0 -1 signals every process and succeeds), locking Chromium out for good. A pid must be a positive integer; anything else is stale. - Two setups judging the same lock stale raced on rm -rf + mkdir and the loser deleted the winner's fresh lock. The stale dir is renamed first (atomic), so exactly one reclaims. - A lock dir with no pid file (killed between mkdir and echo) was never reclaimed; it now expires once older than the install bound. - _kill_tree needed pgrep; debian-slim and git-bash ship none, so the bound killed only the wrapper subshell and the installer kept running. Without pgrep the children are found by walking /proc/*/stat. - The timeout knob is normalized in one place with one comment; the trap's exit 130 is the only exit the block may contain. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(freeze): an unexpected non-zero death denies via an EXIT backstop instead of exiting with no decision set -e plus a failing pipeline (a tool on PATH exiting non-zero, a deleted cwd) ended the deny-tier hook with no JSON, which Claude Code treats as non-blocking: the edit outside the boundary proceeded. The EXIT trap now prints a deny for any non-zero exit that happens before a decision was written; every deliberate output sets _FREEZE_DECIDED first so a late failure never prints a second object. Tests also pin careful's state-root precedence (GSTACK_HOME over CLAUDE_PLUGIN_DATA, plugin data when CLAUDE_PLUGIN_ROOT names gstack) and the specific "out of date" deny for a helper without gstack_hook_state_root. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(telemetry): guard the stale-marker sweep with an if, not a break inside the loop Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(todos): the ownership gate lives in six sites, and the cleanup arms inline their own chain Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: the two remaining linker harnesses extract the ownership helpers; the marker is the one allowed dotfile setup-claude-skill-assets and user-render-out-dir-install slice link_claude_skill_dirs out of setup without the helpers it now calls, so the extracted function died with "command not found" (or, inside an if, degraded into "foreign, skipped"). Both harnesses now carry the full helper set and the globals. The hidden-files census allows .gstack-owned, which the linker writes for directories it creates rather than copying from the skill source. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(setup,relink): weak proof never costs the user a file — assets, flips, failed backups, foreign dir links, alias markers Third review cycle on the ownership model, every item reproduced against a fixture before the fix: - Runtime assets (sections/, templates/, checklist.md, ...) were refreshed with rm -rf regardless of who owned the directory, so an unclaimed or weakly-owned directory lost the user's same-named real files. Real assets are now replaced only in a directory gstack created or strongly owns (marker, or SKILL.md symlink into gstack), plus the legacy Windows real-copy shape; elsewhere they are kept and reported. Symlinks are never content and are always refreshed. - The prefix-flip cleanup deleted a customized banner-bearing SKILL.md that the link pass would have backed up. Both cleanups now compare the file against the source (raw, or with its name: line rewritten to the entry name, which is how alias and prefixed copies legitimately differ) and move a differing file to the backup root. - A failed backup (unwritable root) returned success and the caller linked over the file anyway. It now fails, and the entry is left untouched and reported. - A foreign DIRECTORY symlink whose target had no SKILL.md fell through to the "unclaimed directory" rule and was replaced by a real directory. A symlink that does not resolve into gstack is foreign, full stop. - The alias installers stamped .gstack-owned into pre-existing directories; they now follow the same created-or-already-marked rule. - A directory counts as "only links" only when every link resolves into gstack: a user's own symlink makes it mixed, so their link survives. - The gstack-tree heuristic requires bin/gstack-relink, not just a VERSION file, a setup script and a bin/ directory. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(setup): lock reclaim hands a fresh lock back; a live holder past the bound is stale; /proc walk strips through the last paren - Reclaim renamed the lock by path after judging it stale, so a second setup that had already reclaimed and re-created it lost its fresh lock and two installers ran. After the rename the moved directory's pid is re-read: a new live holder, or a fresh lock whose pid is not written yet, is moved straight back. - A pid file whose process is alive but whose lock is older than the install bound is stale too (the holder is past its own deadline, or the pid was recycled to an unrelated long-lived process); it was locked forever. - The /proc fallback stripped the comm field to the FIRST ") ", so a comm containing ") " hid a child from the kill. proc(5) says the last paren. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(freeze): mark the decision written after the helper prints, not before If gstack_hook_decision ever failed between the flag and its output the backstop would have stayed silent; setting the flag after the print keeps the deny backstop armed until a decision is actually on stdout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore: bump version and changelog (v1.80.0.0) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: update project documentation for v1.80.0.0 README troubleshooting + manual uninstall cover the skill ownership gate (.gstack-owned marker, ~/.gstack/backups/skills/<ts>/, foreign same-name skills left untouched). CLAUDE.md and CONTRIBUTING carry the ownership and best-effort Chromium bootstrap invariants for people editing setup and gstack-relink. PROJECT_STRUCTURE gains careful/, freeze/, guard/, unfreeze/, gstack-upgrade/, gstack-relink, and the setup/relink/hook test files. TESTING_INTERNALS documents the anchor-sliced setup harness convention. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(setup): the final summary reports customized SKILL.md files moved to the backup root The linker moved a weakly-proven, customized SKILL.md aside before linking over it but never said so; only relink printed a "Moved N" line, and by the time relink runs the file is already a symlink. The summary now names each moved file and where it went, next to the foreign-entry report. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: run assembled setup harness scripts from a temp file, not `bash -c` argv (Windows MSYS2 8 KB truncation) windows-free-tests (run 33907177851) failed in test/setup-alias-name-uniqueness.test.ts with bash: -c: line 178: unexpected EOF while looking for matching `' The harness slices functions out of `setup` and passed the joined script as one `bash -c` argv element. The ownership gate grew that script from 6.7 KB to 15.7 KB, and on Windows bash is an MSYS2 program: when its parent is a non-MSYS process (bun), msys-2.0.dll's build_argv() runs any argument containing `?*["'(){}` through globify()/glob(), which copies the pattern into a fixed `Char patbuf[8192]` and silently stops after 8192 - MB_CUR_MAX (8186 chars under C.UTF-8); GLOB_NOCHECK then returns the truncated text as the argument. Character 8186 lands inside the single-quoted sed token on line 178. Rebuilding the exact script with CI path shapes and cutting it at 8186-8190 characters reproduces the identical message locally; cmd.exe's 8191-UTF-16 cap and CreateProcess's 32767 do not fit the evidence. Fix: test/helpers/bash-script.ts writes the script to a temp file and runs `bash <path>` — a short glob-free argument that never enters globify. Every setup harness that assembled a script for `bash -c` (11 files, 22 sites) uses it; timeouts and env are preserved verbatim, spawn/timeout errors are appended to stderr, temp cleanup is best-effort. `spawnSync('bash', [<Windows absolute path>])` already passes on windows-latest in setup-help, uninstall-windows-copies and the migration tests. The Windows-curated list is byte-identical before and after. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(test-free-shards): the rerun-refresh harness spawns bash <tempfile> via test/helpers/bash-script.ts, not bash -c Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
815 lines
46 KiB
Markdown
815 lines
46 KiB
Markdown
# gstack development
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
bun install # install dependencies
|
|
bun run test # run free tests via the strict parallel runner (~90-100s full suite)
|
|
bun run test:evals # run paid evals: LLM judge + E2E (diff-based, ~$4.35/run max)
|
|
bun run test:evals:all # run ALL paid evals regardless of diff
|
|
bun run test:gate # run gate-tier tests only (CI default, blocks merge)
|
|
bun run test:periodic # run periodic-tier tests only (weekly cron / manual)
|
|
bun run test:gate:sharded # gate tier via the sharded paid runner (one Bun process per test file)
|
|
bun run test:periodic:sharded # periodic tier via the sharded paid runner (implies EVALS_ALL=1)
|
|
bun run test:e2e # run E2E tests only (diff-based, ~$4.20/run max)
|
|
bun run test:e2e:all # run ALL E2E tests regardless of diff
|
|
bun run eval:select # show which tests would run based on current diff
|
|
bun run dev <cmd> # run CLI in dev mode, e.g. bun run dev goto https://example.com
|
|
bun run build # gen docs + compile binaries
|
|
bun run gen:skill-docs # regenerate SKILL.md files from templates
|
|
bun run skill:check # health dashboard for all skills
|
|
bun run dev:skill # watch mode: auto-regen + validate on change
|
|
bun run eval:list # list all eval runs from ~/.gstack/projects/<slug>/evals/
|
|
bun run eval:compare # compare two eval runs (auto-picks most recent)
|
|
bun run eval:summary # aggregate stats across all eval runs
|
|
bun run eval:flake-rank # rank tests by flake signal (retried passes first; --json, --dir, --since-days)
|
|
bun run slop # full slop-scan report (all files)
|
|
bun run slop:diff # slop findings in files changed on this branch only
|
|
```
|
|
|
|
`test:evals` requires `ANTHROPIC_API_KEY`. Codex E2E tests (`test/codex-e2e.test.ts`,
|
|
`test/codex-e2e-sol-scope.test.ts`) use Codex's own auth — the hermetic runner copies
|
|
only `auth.json` from `${CODEX_HOME:-~/.codex}` and pins `CODEX_HOME` in the child
|
|
env — no `OPENAI_API_KEY` env var needed.
|
|
|
|
**Hermetic E2E + env keys:** every E2E runner spawns children through
|
|
`test/helpers/hermetic-env.ts` (allowlist-scrubbed env, fresh seeded
|
|
`CLAUDE_CONFIG_DIR`, temp `GSTACK_HOME`, `--strict-mcp-config`); per-test
|
|
`env:` overrides merge last onto a COMPLETE hermetic env, so they're safe.
|
|
A PTY test that types a `/skill` command must pass `seedSkills: true`.
|
|
Debug against real operator state with `EVALS_HERMETIC=0`. Full detail
|
|
(env-shim, seeding tripwires, wiring tests):
|
|
[docs/TESTING_INTERNALS.md](docs/TESTING_INTERNALS.md).
|
|
|
|
**Diff-based test selection:** `test:evals` and `test:e2e` auto-select tests based
|
|
on `git diff` against the base branch. Each test declares its file dependencies in
|
|
`test/helpers/touchfiles.ts`. Changes to global touchfiles (session-runner, eval-store,
|
|
touchfiles.ts itself) trigger all tests. Use `EVALS_ALL=1` or the `:all` script
|
|
variants to force all tests. Run `eval:select` to preview which tests would run.
|
|
|
|
**Two-tier system:** Tests are classified as `gate` or `periodic` in `E2E_TIERS`
|
|
(in `test/helpers/touchfiles.ts` — a facade over `touchfiles-data.ts` +
|
|
`test-selection.ts`). CI runs gate tests per PR via evals.yml's sliced lane
|
|
(planner manifest → executors → fail-closed report; engine =
|
|
scripts/test-paid-shards.ts, the same runner as local eval:bg:gate); the free
|
|
suite runs on every PR via `.github/workflows/free-tests.yml` (a REQUIRED
|
|
check, secretless — fork PRs get real signal); ALL periodic tests run weekly
|
|
via evals-periodic.yml (EVALS_ALL, minus the reasoned exclusions in
|
|
`test/helpers/periodic-exclude-data.ts` — reason + tracking required per
|
|
entry), plus a weekly EVALS_ALL gate census. Use `EVALS_TIER=gate` or
|
|
`EVALS_TIER=periodic` to filter locally. When adding new E2E tests, classify them:
|
|
1. Safety guardrail or deterministic functional test? -> `gate`
|
|
2. Quality benchmark, Opus model test, or non-deterministic? -> `periodic`
|
|
3. Requires external service (Codex, Gemini)? -> `periodic`
|
|
|
|
Tier declarations are enforced by `test/e2e-tier-alignment.test.ts` (free, runs
|
|
in `bun test`): a `skill-e2e-*` file named in a touchfiles dep list whose
|
|
`EVALS_TIER` self-gate disagrees with its declared tier in `E2E_TIERS` fails the
|
|
suite. Files not named in any dep list are reported, not enforced — keep both
|
|
in sync.
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
bun run test # run before every commit — free, ~90-100s for the full ~8,700-test suite
|
|
bun run test:evals # run before shipping — paid, diff-based (~$4.35/run max)
|
|
```
|
|
|
|
`bun run test` routes through `scripts/test-free-shards.ts` (N concurrent
|
|
shard processes, serial within each, packed by recorded per-file durations
|
|
when `scripts/free-test-durations.json` exists — refresh occasionally with
|
|
`bun run test:free --record-durations`; strict-output classification per
|
|
shard: a shard without bun's terminal summary line FAILS — silent truncation
|
|
cannot report green). The former trailing serial tree-mutating shard is
|
|
gone: `TREE_MUTATING` is empty (gen-skill-docs has a main() guard and
|
|
`--out-dir` renders every host, so tests render into mkdtemps — see
|
|
docs/TESTING_INTERNALS.md). Never type bare `bun test` for the suite: it
|
|
walks the whole repo, loading paid eval files and missing the strict
|
|
classifier.
|
|
It covers skill validation, gen-skill-docs quality checks, and browse
|
|
integration tests. `bun run test:evals` runs LLM-judge quality evals and E2E
|
|
tests via `claude -p`. Both must pass before creating a PR.
|
|
|
|
## Project structure
|
|
|
|
Full annotated tree: [docs/PROJECT_STRUCTURE.md](docs/PROJECT_STRUCTURE.md).
|
|
Quick map: `browse/` headless-browser CLI, `design/` design binary,
|
|
`hosts/` typed host configs, `scripts/` build+DX tooling (gen-skill-docs,
|
|
resolvers), `test/` validation+evals, `lib/` shared libraries, `bin/` CLI
|
|
utilities, `extension/` Chrome extension, one directory per skill
|
|
(`ship/`, `review/`, `qa/`, ...), `.github/` CI, `contrib/` contributor
|
|
tools, `docs/designs/` design documents.
|
|
|
|
## SKILL.md workflow
|
|
|
|
SKILL.md files are **generated** from `.tmpl` templates. To update docs:
|
|
|
|
1. Edit the `.tmpl` file (e.g. `SKILL.md.tmpl` or `browse/SKILL.md.tmpl`)
|
|
2. Run `bun run gen:skill-docs` (or `bun run build` which does it automatically)
|
|
3. Commit both the `.tmpl` and generated `.md` files
|
|
|
|
Generation uses each host's `defaultModel` (`claude` for existing hosts, `gpt`
|
|
for Codex) unless `--model` is explicit. Codex installs additionally read the
|
|
top-level model from `${CODEX_HOME:-~/.codex}/config.toml`; rerun
|
|
`./setup --host codex` after changing that model. Note: `bun run build` and a
|
|
bare `gen:skill-docs --host codex` render the host default (gpt) — if your
|
|
Codex config.toml pins a different model, rerun `./setup --host codex`
|
|
afterwards to restore your profile (single-owner persistence is filed in
|
|
TODOS.md).
|
|
|
|
To add a new browse command: add it to `browse/src/commands.ts` and rebuild.
|
|
To add a snapshot flag: add it to `SNAPSHOT_FLAGS` in `browse/src/snapshot.ts` and rebuild.
|
|
|
|
**Token ceiling:** Generated SKILL.md files trip a warning above 160KB (~40K tokens).
|
|
This is a "watch for feature bloat" guardrail, not a hard gate. Modern flagship
|
|
models have 200K-1M context windows, so 40K is 4-20% of window, and prompt caching
|
|
makes the marginal cost of larger skills small. The ceiling exists to catch runaway
|
|
preamble/resolver growth, not to force compression on carefully-tuned big skills
|
|
(`ship`, `plan-ceo-review`, `office-hours` legitimately pack 25-35K tokens of
|
|
behavior). If you blow past 40K, the right fix is usually: (1) look at WHAT grew,
|
|
(2) if one resolver added 10K+ in a single PR, question whether it belongs inline
|
|
or as a reference doc, (3) only compress carefully-tuned prose as a last resort —
|
|
cuts to the coverage audit, review army, or voice directive have real quality cost.
|
|
|
|
A second, harder ceiling guards the DISCOVERY surface: `test/catalog-budget.test.ts`
|
|
caps the aggregate frontmatter `name` + `description` across all skills at 1,150
|
|
token-equivalents (260-byte per-skill sub-cap), counted through the shared census
|
|
in `test/helpers/skill-census.ts`. This one is enforced, not a warning — every
|
|
host loads the full catalog every session, so growth here taxes every
|
|
conversation. The failure message carries the re-measure + ratchet protocol.
|
|
`bin/gstack-context-bill` shows the full token bill-of-materials for a skills
|
|
tree (always-on vs per-invocation, `--diff`, `--budget`; `--exact` opts into the
|
|
real tokenizer and POSTs file text to api.anthropic.com with an egress receipt).
|
|
|
|
The context-budget ratchet (`test/context-budget-ratchet.test.ts`, free, runs
|
|
in `bun run test`) pins ABSOLUTE ceilings on two more ledgers: the always-on
|
|
FULL-frontmatter aggregate (catalog-budget counts only name+description) and
|
|
each skill's per-invocation eager tokens (SKILL.md + forced-read references —
|
|
size floors and parity ratios guard these relatively, not absolutely), graded
|
|
against `test/fixtures/context-budget.json`. A skill that grows past its
|
|
ceiling fails; a new skill fails until it's consciously budgeted. For
|
|
legitimate growth or a landed reduction, re-run
|
|
`bun test/helpers/capture-context-budget.ts` and commit the refreshed fixture
|
|
in the same commit, so ceilings ratchet down and every win is locked.
|
|
|
|
**Merge conflicts on SKILL.md files:** NEVER resolve conflicts on generated SKILL.md
|
|
files by accepting either side. Instead: (1) resolve conflicts on the `.tmpl` templates
|
|
and `scripts/gen-skill-docs.ts` (the sources of truth), (2) run `bun run gen:skill-docs`
|
|
to regenerate all SKILL.md files, (3) stage the regenerated files. Accepting one side's
|
|
generated output silently drops the other side's template changes.
|
|
|
|
## Platform-agnostic design
|
|
|
|
Skills must NEVER hardcode framework-specific commands, file patterns, or directory
|
|
structures. Instead:
|
|
|
|
1. **Read CLAUDE.md** for project-specific config (test commands, eval commands, etc.)
|
|
2. **If missing, AskUserQuestion** — let the user tell you or let gstack search the repo
|
|
3. **Persist the answer to CLAUDE.md** so we never have to ask again
|
|
|
|
This applies to test commands, eval commands, deploy commands, and any other
|
|
project-specific behavior. The project owns its config; gstack reads it.
|
|
|
|
## Writing SKILL templates
|
|
|
|
SKILL.md.tmpl files are **prompt templates read by Claude**, not bash scripts.
|
|
Each bash code block runs in a separate shell — variables do not persist between blocks.
|
|
|
|
Rules:
|
|
- **Use natural language for logic and state.** Don't use shell variables to pass
|
|
state between code blocks. Instead, tell Claude what to remember and reference
|
|
it in prose (e.g., "the base branch detected in Step 0").
|
|
- **Don't hardcode branch names.** Detect `main`/`master`/etc dynamically via
|
|
`gh pr view` or `gh repo view`. Use `{{BASE_BRANCH_DETECT}}` for PR-targeting
|
|
skills. Use "the base branch" in prose, `<base>` in code block placeholders.
|
|
- **Keep bash blocks self-contained.** Each code block should work independently.
|
|
If a block needs context from a previous step, restate it in the prose above.
|
|
- **Express conditionals as English.** Instead of nested `if/elif/else` in bash,
|
|
write numbered decision steps: "1. If X, do Y. 2. Otherwise, do Z."
|
|
|
|
## Writing style (V1)
|
|
|
|
Default output from every tier-≥2 skill follows the Writing Style section in
|
|
`scripts/resolvers/preamble.ts`: jargon glossed on first use (curated list in
|
|
`scripts/jargon-list.json`, baked at gen-skill-docs time), questions framed in
|
|
outcome terms ("what breaks for your users if...") not implementation terms,
|
|
short sentences, decisions close with user impact. Power users who want the
|
|
tighter V0 prose set `gstack-config set explain_level terse` (binary switch,
|
|
no middle mode). See `docs/designs/PLAN_TUNING_V1.md` for the full design
|
|
rationale. The review pacing overhaul that originally tried to ride alongside
|
|
writing-style was extracted to V1.1 — see `docs/designs/PACING_UPDATES_V0.md`.
|
|
|
|
## Browser interaction
|
|
|
|
When you need to interact with a browser (QA, dogfooding, cookie setup), use the
|
|
`/browse` skill or run the browse binary directly via `$B <command>`. NEVER use
|
|
`mcp__claude-in-chrome__*` tools — they are slow, unreliable, and not what this
|
|
project uses.
|
|
|
|
**Server / sidebar / extension internals:** before editing `browse/src/server.ts`,
|
|
`extension/`, the sidebar PTY, any SSE endpoint, or CDP session code, read
|
|
[docs/BROWSER_INTERNALS.md](docs/BROWSER_INTERNALS.md) — sidebar message flow,
|
|
WebSocket auth, tunnel dual-listener rules, Unicode sanitization at egress,
|
|
SSE/CDP helpers, setup symlink hardening, and the sidebar security stack all
|
|
live there, each pinned by a CI tripwire.
|
|
|
|
**Egress receipts at every off-machine sink** (v1.63.0.0+). Every gstack-initiated
|
|
send off the machine MUST write a hash-chained receipt to
|
|
`~/.gstack/security/egress.jsonl` BEFORE the send: TypeScript callers use
|
|
`writeReceipt` from `lib/egress-receipt.ts`; shell scripts source
|
|
`bin/gstack-egress-lib.sh` and use `_receipted_curl` / `_receipted_git`. Failure
|
|
polarity is per-class: fail-closed for sensitive sinks (brain-sync, memory-ingest,
|
|
gbrain-sync, telemetry, ngrok tunnels, mcp-verify, supabase-provision), fail-open
|
|
+ stderr warning for user-facing ones (design OpenAI calls, update-check,
|
|
dashboards, git-class ops). The new-sink scanner in
|
|
`test/egress-receipt-wiring.test.ts` fails CI on an unreceipted `curl` /
|
|
`git push` / `fetch` to a non-loopback host unless the file carries a reasoned
|
|
entry in its `SCANNER_EXEMPT` list (user-directed page fetches, reachability
|
|
probes, instruction strings, skill prose) — if you add a new off-machine sink,
|
|
wire it through the helpers and add it to the enumerated sink list. Inspect with
|
|
`bin/gstack-egress` (`list` | `verify`, exit 3 on tamper | `grants`). Threat
|
|
model: forensic observability of ATTEMPTED egress, not an exfiltration control.
|
|
|
|
## Dev symlink awareness
|
|
|
|
When developing gstack, `.claude/skills/gstack` may be a symlink back to this
|
|
working directory (gitignored). This means skill changes are **live immediately**,
|
|
great for rapid iteration, risky during big refactors where half-written skills
|
|
could break other Claude Code sessions using gstack concurrently.
|
|
|
|
**Check once per session:** Run `ls -la .claude/skills/gstack` to see if it's a
|
|
symlink or a real copy. If it's a symlink to your working directory, be aware that:
|
|
- Template changes + `bun run gen:skill-docs` immediately affect all gstack invocations
|
|
- Breaking changes to SKILL.md.tmpl files can break concurrent gstack sessions
|
|
- During large refactors, remove the symlink (`rm .claude/skills/gstack`) so the
|
|
global install at `~/.claude/skills/gstack/` is used instead
|
|
|
|
**Prefix setting:** Setup creates real directories (not symlinks) at the top level
|
|
with a SKILL.md symlink inside (e.g., `qa/SKILL.md -> gstack/qa/SKILL.md`), plus
|
|
links to each skill's runtime assets (sections/, templates, checklists — everything
|
|
except SKILL.md, tests, build output, and `.tmpl` sources). Alias skills
|
|
(`_gstack-command`, `connect-chrome`) install as rewritten copies, never symlinks.
|
|
This ensures Claude discovers them as top-level skills, not nested under `gstack/`.
|
|
Names are either short (`qa`) or namespaced (`gstack-qa`), controlled by
|
|
`skill_prefix` in `~/.gstack/config.yaml`. Pass `--no-prefix` or `--prefix` to
|
|
skip the interactive prompt.
|
|
|
|
**Ownership gate (#2119):** `setup` writes a `.gstack-owned` marker into every
|
|
skill directory it creates, and `setup` (the linker, the alias installer, and
|
|
both prefix-flip cleanups) and `bin/gstack-relink` only delete or link over an
|
|
entry they can prove is gstack's. Strong proof (a symlink resolving into gstack,
|
|
or the marker) allows deleting or refreshing the whole directory. Weak proof (a
|
|
real SKILL.md byte-identical to the source, or carrying gen-skill-docs' two-line
|
|
banner) covers only that one file, and a weakly-proven file that differs is
|
|
moved to `~/.gstack/backups/skills/<ts>/<skill>/SKILL.md` before gstack links
|
|
over it. Anything else is a foreign skill: skipped, and named in setup's final
|
|
summary. The rule lives in two copies (`setup` and `bin/gstack-relink`); keep
|
|
them in sync until the shared helper filed in TODOS.md lands. Pinned by
|
|
`test/setup-link-ownership.test.ts`, `test/setup-cleanup-orphans.test.ts`, and
|
|
`test/relink.test.ts`.
|
|
|
|
**Note:** Vendoring gstack into a project's repo is deprecated. Use global install
|
|
+ `./setup --team` instead. See README.md for team mode instructions.
|
|
|
|
**For plan reviews:** When reviewing plans that modify skill templates or the
|
|
gen-skill-docs pipeline, consider whether the changes should be tested in isolation
|
|
before going live (especially if the user is actively using gstack in other windows).
|
|
|
|
**Upgrade migrations:** When a change modifies on-disk state (directory structure,
|
|
config format, stale files) in ways that could break existing user installs, add a
|
|
migration script to `gstack-upgrade/migrations/`. Read CONTRIBUTING.md's "Upgrade
|
|
migrations" section for the format and testing requirements. The upgrade skill runs
|
|
these automatically after `./setup` during `/gstack-upgrade`.
|
|
|
|
## Compiled binaries — never commit browse/dist/, design/dist/, or make-pdf/dist/
|
|
|
|
The `browse/dist/`, `design/dist/`, and `make-pdf/dist/` directories contain
|
|
compiled Bun binaries (`browse`, `find-browse`, `design`, ~62MB each). These are
|
|
Mach-O arm64 only — they do NOT work on Linux, Windows, or Intel Macs. The
|
|
`./setup` script builds from source for every platform.
|
|
|
|
These directories are **untracked and gitignored** (`.gitignore:3-6`; the
|
|
`browse/dist/` binaries were untracked in `64d5a3e4`, v0.11.16.0; the others were
|
|
never tracked). They will NOT appear in `git status`. If a dist binary ever does
|
|
show up in `git status`, something force-added it (`git add -f`) — do not commit
|
|
it; unstage it and find out how it got there.
|
|
|
|
When staging files, always use specific filenames (`git add file1 file2`) — never
|
|
`git add .` or `git add -A`, which can sweep in build outputs and junk.
|
|
|
|
## Redaction guard (PII / secrets / legal content)
|
|
|
|
Shared redaction engine catches credentials, PII, and legal/damaging content
|
|
before it reaches an external sink (codex dispatch, GitHub issue/PR body, pushed
|
|
commit). It is a **guardrail, not airtight enforcement** — `git push --no-verify`,
|
|
direct `gh issue create`, and `GSTACK_REDACT_PREPUSH=skip` all bypass it. It
|
|
catches accidents and carelessness, the 99% case. Do not claim it stops a
|
|
determined leaker (a CHANGELOG line that does would fail a hostile screenshotter).
|
|
|
|
- **Engine + taxonomy:** `lib/redact-patterns.ts` (the single source of truth —
|
|
3 tiers; HIGH = genuinely-secret credentials that block, MEDIUM = PII/legal/
|
|
internal + high-FP credential shapes that confirm via AskUserQuestion, LOW =
|
|
FYI) and `lib/redact-engine.ts` (pure `scan()` + `applyRedactions()`).
|
|
Calibration matters: a gate that cries wolf gets ignored, so context-variable
|
|
shapes (Stripe `pk_live_`, Google `AIza`, JWT, env `*_KEY=`) sit at MEDIUM.
|
|
- **CLI:** `bin/gstack-redact` (exit 0 clean / 2 MEDIUM / 3 HIGH; `--json`,
|
|
`--auto-redact`, `--repo-visibility`, `--from-file`). `bin/gstack-redact-prepush`
|
|
is the opt-in git hook.
|
|
- **Skill docs are generated** from `scripts/resolvers/redact-doc.ts`
|
|
(`{{REDACT_INVOCATION_BLOCK:<sink>}}`) so /spec,
|
|
/cso, /ship, /document-release, /document-generate never drift from the engine.
|
|
- **Scan-at-sink:** always scan the EXACT bytes that will be sent — write to a
|
|
temp file, scan that file, pass the SAME file to `gh`/`git`. Never scan a string
|
|
then re-render (that reopens a scan-vs-send gap).
|
|
- **Visibility (no tier promotion):** resolve once per run, order = local config
|
|
(`gstack-config get redact_repo_visibility`, ~/.gstack so never committed) → gh
|
|
→ glab → unknown(=public-strict). Public repos get STERNER per-finding
|
|
confirmation (no batch-acknowledge, no silent-proceed); MEDIUM is never
|
|
auto-promoted to HIGH.
|
|
- **Tool-attributed fences:** wrap Codex/Greptile/eval output in ` ```codex-review `
|
|
/ ` ```greptile ` fences so example credentials those tools quote WARN-degrade
|
|
instead of blocking. A live-format credential inside the fence still blocks.
|
|
- **Config keys:** `redact_repo_visibility` (public|private|unknown, local-only
|
|
override for repos gh/glab can't read), `redact_prepush_hook` (true|false).
|
|
There is intentionally NO key to disable HIGH blocking.
|
|
- **Audit:** the /spec semantic pass appends a content-free record (categories +
|
|
body sha256, no spec text) to `~/.gstack/security/semantic-reviews.jsonl` (0600).
|
|
|
|
## Commit style
|
|
|
|
**Always bisect commits.** Every commit should be a single logical change. When
|
|
you've made multiple changes (e.g., a rename + a rewrite + new tests), split them
|
|
into separate commits before pushing. Each commit should be independently
|
|
understandable and revertable.
|
|
|
|
Examples of good bisection:
|
|
- Rename/move separate from behavior changes
|
|
- Test infrastructure (touchfiles, helpers) separate from test implementations
|
|
- Template changes separate from generated file regeneration
|
|
- Mechanical refactors separate from new features
|
|
|
|
When the user says "bisect commit" or "bisect and push," split staged/unstaged
|
|
changes into logical commits and push.
|
|
|
|
## Slop-scan: AI code quality, not AI code hiding
|
|
|
|
We use [slop-scan](https://github.com/benvinegar/slop-scan) to catch patterns where
|
|
AI-generated code is genuinely worse than what a human would write. We are NOT trying
|
|
to pass as human code. We are AI-coded and proud of it. The goal is code quality.
|
|
|
|
```bash
|
|
npx slop-scan scan . # human-readable report
|
|
npx slop-scan scan . --json # machine-readable for diffing
|
|
```
|
|
|
|
Config: `slop-scan.config.json` at repo root (currently excludes `**/vendor/**`).
|
|
|
|
Before fixing any finding, read [docs/SLOP_SCAN.md](docs/SLOP_SCAN.md):
|
|
it separates genuine quality fixes (empty catches around file ops →
|
|
`safeUnlink()`, process kills → `safeKill()`) from linter gaming we
|
|
reject (string-matching error messages, tightening best-effort cleanup).
|
|
Utilities live in `browse/src/error-handling.ts`. Don't chase the score.
|
|
|
|
## Community PR guardrails
|
|
|
|
When reviewing or merging community PRs, **always AskUserQuestion** before accepting
|
|
any commit that:
|
|
|
|
1. **Touches ETHOS.md** — this file is Garry's personal builder philosophy. No edits
|
|
from external contributors or AI agents, period.
|
|
2. **Removes or softens promotional material** — YC references, founder perspective,
|
|
and product voice are intentional. PRs that frame these as "unnecessary" or
|
|
"too promotional" must be rejected.
|
|
3. **Changes Garry's voice** — the tone, humor, directness, and perspective in skill
|
|
templates, CHANGELOG, and docs are not generic. PRs that rewrite voice to be
|
|
more "neutral" or "professional" must be rejected.
|
|
|
|
Even if the agent strongly believes a change improves the project, these three
|
|
categories require explicit user approval via AskUserQuestion. No exceptions.
|
|
No auto-merging. No "I'll just clean this up."
|
|
|
|
## Checking out PRs from garrytan-agents
|
|
|
|
When the user says "check out <PR link>" and the PR is from `garrytan-agents/gstack`
|
|
(or any other fork that is NOT a collaborator on `garrytan/gstack`), do NOT just
|
|
`gh pr checkout`. Fork PRs don't receive base-repo secrets (`ANTHROPIC_API_KEY`,
|
|
`OPENAI_API_KEY`, etc.), so the eval/E2E CI jobs fail with empty-env auth errors
|
|
regardless of what's set on the base repo.
|
|
|
|
**Workflow:** push the branch to `garrytan/gstack` (the base repo) and re-target
|
|
the PR from there.
|
|
|
|
Concretely, after `gh pr checkout <N>`:
|
|
|
|
1. Note the original PR number and head branch name.
|
|
2. Push the same branch to the base repo: `git push origin HEAD:<branch-name>`
|
|
(origin = `garrytan/gstack`, since the worktree is set up with that remote).
|
|
3. Close the fork PR (`gh pr close <N> --comment "moving to base-repo branch for secret access"`).
|
|
4. Open a new PR from the base-repo branch: `gh pr create --base main --head <branch-name>`.
|
|
5. New PR's workflows will get secrets automatically.
|
|
|
|
Why not fix it on the fork side? `garrytan-agents` isn't a collaborator on
|
|
`garrytan/gstack`. Adding it as a collaborator (option A) or flipping the
|
|
repo-wide "send secrets to fork PRs" toggle (option B) would let secrets reach
|
|
fork PRs from anyone — broader blast radius than just moving this one branch.
|
|
Option C (this section) keeps secret-distribution scope tight.
|
|
|
|
If the user asks you to skip the move (e.g., "just leave it as a fork PR"),
|
|
respect that — eval CI will fail with empty-env auth, but check-freshness,
|
|
workflow-lint, and windows-tests will still pass on the fork PR.
|
|
|
|
## CHANGELOG + VERSION style
|
|
|
|
**Versioning invariant (workspace-aware ship).** VERSION is a monotonic ordered
|
|
release identifier, not a strict semver commitment. The bump level
|
|
(major/minor/patch/micro) expresses intent at ship time. Queue-advancing past a
|
|
claimed version within the same bump level is explicitly permitted — if branch A
|
|
claims v1.7.0.0 as a MINOR and branch B is also a MINOR, B lands at v1.8.0.0
|
|
(still a MINOR relative to main). Downstream consumers must NOT rely on
|
|
"MINOR = feature-only, PATCH = fix-only" as a strict contract. This is why
|
|
`bin/gstack-next-version` advances within the chosen bump level rather than
|
|
repicking the level when collisions happen.
|
|
|
|
**package.json carries the npm-valid translation, not VERSION verbatim.**
|
|
VERSION stays the 4-digit source of truth (e.g. `1.67.0.0`); package.json and
|
|
any subdirectory manifests with a `version` field get the 3-digit npm-valid
|
|
translation (`1.67.0`), and lockfile `version` fields sync only when the
|
|
lockfile already exists. `bin/gstack-version-bump` (via `lib/version-source.ts`)
|
|
owns the translation and judges drift on translated forms — do NOT "fix" the
|
|
apparent mismatch by hand, and do not write a 4-digit version into
|
|
package.json (npm rejects it). Rationale and translation rules live in the
|
|
`lib/version-source.ts` header; `test/gstack-version-bump.test.ts` pins the
|
|
contract.
|
|
|
|
**Scale-aware bumps — use common sense.** When the diff is big, bump MINOR (or
|
|
MAJOR), not PATCH. PATCH is for bug fixes and small additions; MINOR is for
|
|
substantial new capability or substantial reduction; MAJOR is for breaking
|
|
changes. Rough guideposts (don't treat as rules, treat as smell-checks):
|
|
|
|
- **PATCH (X.Y.Z+1.0)**: bug fix, doc tweak, small additive change, single
|
|
test/file added. Net diff under ~500 lines, no new user-facing capability.
|
|
- **MINOR (X.Y+1.0.0)**: new capability shipped (skill, harness, command, big
|
|
refactor), substantial code reduction (compression, migration), or coordinated
|
|
multi-file change. Net diff over ~2000 lines added/removed, OR a user-visible
|
|
feature you'd put in a tweet.
|
|
- **MAJOR (X+1.0.0.0)**: breaking change to public surface (CLI flag rename,
|
|
skill removed, config format changed), OR a release big enough to be the
|
|
headline of a blog post.
|
|
|
|
If you find yourself debating "is 10K added + 24K removed really a PATCH?" — it
|
|
isn't. Bump MINOR. Same for "this adds a whole new test harness with 6 new E2E
|
|
tests + helper utilities" — MINOR. The bump level is communication to the user
|
|
about what kind of release this is; don't undersell it.
|
|
|
|
When merging origin/main brings a higher VERSION, re-evaluate the bump level
|
|
against the SCALE of your branch's work, not just whether main moved forward.
|
|
If main bumped MINOR and your branch is also a substantial change, you bump
|
|
MINOR again on top (e.g., main at v1.14.0.0, your branch lands v1.15.0.0).
|
|
|
|
**VERSION and CHANGELOG are branch-scoped.** Every feature branch that ships gets its
|
|
own version bump and CHANGELOG entry. The entry describes what THIS branch adds —
|
|
not what was already on main.
|
|
|
|
**The CHANGELOG entry is the diff between main and the shipping branch — what users
|
|
get when they upgrade. NOT how the branch got there.** A reader landing on the entry
|
|
should learn what they can do now that they couldn't before; they should not learn
|
|
about the branch's internal version bumps, the bugs we caught and fixed mid-branch,
|
|
the plan reviews we ran, or the commits we squashed. That is branch development
|
|
narrative. It belongs in PR descriptions and commit messages, not CHANGELOG.
|
|
|
|
**Never reference branch-internal versions in a CHANGELOG entry.** If your branch
|
|
bumped VERSION from v1.5.0.0 → v1.5.1.0 → v1.6.0.0 during development and only the
|
|
final v1.6.0.0 ships to main, the entry must read as if v1.5.1.0 never existed.
|
|
Concretely, NEVER write:
|
|
- "v1.5.1.0 had a bug that v1.6.0.0 fixes" — readers don't know about v1.5.1.0; it's
|
|
a branch-internal artifact.
|
|
- "The shipping headline of v1.5.1.0 was broken because..." — same reason. From main's
|
|
perspective, v1.5.1.0 was never released.
|
|
- "Pre-fix tests encoded the broken behavior" — that's a contributor's victory lap,
|
|
not a user benefit.
|
|
- "Two surgical edits, both in the dispatch path" — micro-narrative of the patch.
|
|
|
|
Instead, describe the released system: "Browser-skills run end-to-end with the
|
|
expected tab-access semantics." If a property of the shipped system is worth calling
|
|
out (e.g., "skill spawns get permissive tab access; pair-agent tunnel tokens require
|
|
ownership"), document it as a property, not as a fix. The shipped system is what
|
|
the user gets; the path to that system is invisible to them.
|
|
|
|
**When to write the CHANGELOG entry:**
|
|
- At `/ship` time (Step 13), not during development or mid-branch.
|
|
- The entry covers ALL commits on this branch vs the base branch.
|
|
- Never fold new work into an existing CHANGELOG entry from a prior version that
|
|
already landed on main. If main has v0.10.0.0 and your branch adds features,
|
|
bump to v0.10.1.0 with a new entry — don't edit the v0.10.0.0 entry.
|
|
|
|
**Key questions before writing:**
|
|
1. What branch am I on? What did THIS branch change?
|
|
2. Is the base branch version already released? (If yes, bump and create new entry.)
|
|
3. Does an existing entry on this branch already cover earlier work? (If yes, replace
|
|
it with one unified entry for the final version.)
|
|
|
|
**Merging main does NOT mean adopting main's version.** When you merge origin/main into
|
|
a feature branch, main may bring new CHANGELOG entries and a higher VERSION. Your branch
|
|
still needs its OWN version bump on top. If main is at v0.13.8.0 and your branch adds
|
|
features, bump to v0.13.9.0 with a new entry. Never jam your changes into an entry that
|
|
already landed on main. Your entry goes on top because your branch lands next.
|
|
|
|
**After merging main, always check:**
|
|
- Does CHANGELOG have your branch's own entry separate from main's entries?
|
|
- Is VERSION higher than main's VERSION?
|
|
- Is your entry the topmost entry in CHANGELOG (above main's latest)?
|
|
If any answer is no, fix it before continuing.
|
|
|
|
**After any CHANGELOG edit that moves, adds, or removes entries,** immediately run
|
|
`grep "^## \[" CHANGELOG.md` to verify no duplicates and a sensible reverse-chronological
|
|
order. Gaps between version numbers are fine. A branch that ships at v1.6.4.0 without
|
|
a prior v1.5.2.0 or v1.5.3.0 entry on main is correct — those were branch-internal
|
|
version numbers that never landed. Do not back-fill gaps with placeholder entries.
|
|
|
|
**Never orphan branch-internal versions.** If your branch bumped VERSION several times
|
|
during development (v1.5.1.0 → v1.5.2.0 → v1.6.4.0, say) and those earlier entries were
|
|
never released to main, the final ship consolidates ALL of them into a single entry at
|
|
the final version (v1.6.4.0). Collapse them — delete the old entries and move their
|
|
content into the final entry, re-version table columns accordingly. Readers see one
|
|
release, not a branch diary. Gaps are fine (v1.6.3.0 → v1.6.4.0 with no v1.5.x
|
|
in between on main is correct).
|
|
|
|
CHANGELOG.md is **for users**, not contributors. Write it like product release notes:
|
|
|
|
- Lead with what the user can now **do** that they couldn't before. Sell the feature.
|
|
- Use plain language, not implementation details. "You can now..." not "Refactored the..."
|
|
- **Never mention TODOS.md, internal tracking, eval infrastructure, or contributor-facing
|
|
details.** These are invisible to users and meaningless to them.
|
|
- Put contributor/internal changes in a separate "For contributors" section at the bottom.
|
|
- Every entry should make someone think "oh nice, I want to try that."
|
|
- No jargon: say "every question now tells you which project and branch you're in" not
|
|
"AskUserQuestion format standardized across skill templates via preamble resolver."
|
|
|
|
**Only document what shipped between main and this change.** Readers do not care how
|
|
we got here. Keep out of the CHANGELOG, always:
|
|
|
|
- Branch resyncs, merge commits with main, rebase activity.
|
|
- Plan approvals, review outcomes (CEO / eng / design / outside-voice / codex findings),
|
|
AskUserQuestion decisions, scope negotiations.
|
|
- "Work queued," "plan approved," "in-progress," "will ship later" — the CHANGELOG
|
|
documents what DID ship, not what MIGHT ship.
|
|
- Version-bump housekeeping when no user-facing work actually landed.
|
|
|
|
If the diff between the base branch version and this version has no user-facing change
|
|
(only merges, only CHANGELOG edits, only placeholder work), the honest entry is one
|
|
sentence: "Version bump for branch-ahead discipline. No user-facing changes yet." Stop
|
|
there. Do not pad. Do not explain the plan that will ship eventually. Do not narrate
|
|
the branch's history. When real work lands, the entry will replace this at /ship time.
|
|
|
|
### Entry format
|
|
|
|
Every `## [X.Y.Z]` entry starts with a release summary (two-line bold
|
|
headline, lead paragraph, numbers table, closing paragraph) followed by an
|
|
`### Itemized changes` section. Read
|
|
[docs/CHANGELOG_STYLE.md](docs/CHANGELOG_STYLE.md) for the full format spec
|
|
and voice rules BEFORE writing an entry. Always credit community
|
|
contributions with `Contributed by @username`.
|
|
|
|
## AI effort compression
|
|
|
|
When estimating or discussing effort, always show both human-team and CC+gstack time:
|
|
|
|
| Task type | Human team | CC+gstack | Compression |
|
|
|-----------|-----------|-----------|-------------|
|
|
| Boilerplate / scaffolding | 2 days | 15 min | ~100x |
|
|
| Test writing | 1 day | 15 min | ~50x |
|
|
| Feature implementation | 1 week | 30 min | ~30x |
|
|
| Bug fix + regression test | 4 hours | 15 min | ~20x |
|
|
| Architecture / design | 2 days | 4 hours | ~5x |
|
|
| Research / exploration | 1 day | 3 hours | ~3x |
|
|
|
|
Completeness is cheap. Don't recommend shortcuts when the complete implementation
|
|
is achievable. Boil the ocean — the complete thing is the goal; only genuinely
|
|
unrelated multi-quarter migrations are separate scope, never an excuse for a
|
|
shortcut. See the Completeness Principle in the skill preamble for the full
|
|
philosophy.
|
|
|
|
## Search before building
|
|
|
|
Before designing any solution that involves concurrency, unfamiliar patterns,
|
|
infrastructure, or anything where the runtime/framework might have a built-in:
|
|
|
|
1. Search for "{runtime} {thing} built-in"
|
|
2. Search for "{thing} best practice {current year}"
|
|
3. Check official runtime/framework docs
|
|
|
|
Three layers of knowledge: tried-and-true (Layer 1), new-and-popular (Layer 2),
|
|
first-principles (Layer 3). Prize Layer 3 above all. See ETHOS.md for the full
|
|
builder philosophy.
|
|
|
|
## Local plans
|
|
|
|
Contributors can store long-range vision docs and design documents in `~/.gstack-dev/plans/`.
|
|
These are local-only (not checked in). When reviewing TODOS.md, check `plans/` for candidates
|
|
that may be ready to promote to TODOs or implement.
|
|
|
|
## E2E eval failure blame protocol
|
|
|
|
When an E2E eval fails during `/ship` or any other workflow, **never claim "not
|
|
related to our changes" without proving it.** These systems have invisible couplings —
|
|
a preamble text change affects agent behavior, a new helper changes timing, a
|
|
regenerated SKILL.md shifts prompt context.
|
|
|
|
**Required before attributing a failure to "pre-existing":**
|
|
1. Run the same eval on main (or base branch) and show it fails there too
|
|
2. If it passes on main but fails on the branch — it IS your change. Trace the blame.
|
|
3. If you can't run on main, say "unverified — may or may not be related" and flag it
|
|
as a risk in the PR body
|
|
|
|
"Pre-existing" without receipts is a lazy claim. Prove it or don't say it.
|
|
|
|
## Long-running tasks: don't give up
|
|
|
|
When running evals, E2E tests, or any long-running background task, **poll until
|
|
completion**. Use `sleep 180 && echo "ready"` + `TaskOutput` in a loop every 3
|
|
minutes. Never switch to blocking mode and give up when the poll times out. Never
|
|
say "I'll be notified when it completes" and stop checking — keep the loop going
|
|
until the task finishes or the user tells you to stop.
|
|
|
|
The full E2E suite can take 30-45 minutes. That's 10-15 polling cycles. Do all of
|
|
them. Report progress at each check (which tests passed, which are running, any
|
|
failures so far). The user wants to see the run complete, not a promise that
|
|
you'll check later.
|
|
|
|
## Running evals as an agent: always detach (SIGTERM-proof)
|
|
|
|
When **you (an agent/harness)** launch a long eval/benchmark run, run it through
|
|
`bin/gstack-detach` — NEVER as a plain backgrounded Bash task. A plain background
|
|
task lives in the harness's process group, so a SIGTERM ("polite quit") on a turn
|
|
boundary, a stopped Monitor, or an interruption kills the run mid-flight (observed:
|
|
`script "test:gate" was terminated by signal SIGTERM` ~40 min into a run). On macOS
|
|
the run can also die to idle-sleep. `gstack-detach` fixes both: a fresh session
|
|
(escapes the group SIGTERM) wrapped in `caffeinate -i` (blocks idle-sleep).
|
|
|
|
- Use the `eval:bg*` scripts (`eval:bg`, `eval:bg:all`, `eval:bg:gate`,
|
|
`eval:bg:periodic`) — they wrap the eval command in `gstack-detach` with the
|
|
machine-wide `gstack-evals` lock (concurrent worktrees serialize instead of
|
|
saturating the shared model API), a per-tier watchdog, and a **run-scoped** log
|
|
under `~/.gstack-dev/eval-runs/` (no shared-`/tmp` collision). Each prints its
|
|
log path. `eval:bg:gate` / `eval:bg:periodic` run their tier through the
|
|
sharded paid runner (`scripts/test-paid-shards.ts`, also exposed as
|
|
`test:gate:sharded` / `test:periodic:sharded`): one Bun process per test
|
|
file, an external wall-clock timeout that kills the shard's process GROUP
|
|
(stray `claude`/`codex` grandchildren included), a per-shard
|
|
`GSTACK_EVAL_DIR=<evalDir>/shards/<slug>/` honored by the `EvalCollector`
|
|
constructor, and an aggregate that separates failed vs timed-out vs
|
|
never-started shards — the detach timeouts (25200s gate / 37800s periodic;
|
|
floor enforced against the live shard census by
|
|
test/eval-detach-timeout-floor.test.ts)
|
|
are sized against worst-case shard wall clock. `EVALS_JOBS` sets the shard
|
|
process count (default 8); `EVALS_CONCURRENCY` is bun's --max-concurrency
|
|
WITHIN a shard (default 2) — they are deliberately separate knobs. `eval:list` / `eval:compare` /
|
|
`eval:summary` / `eval:flake-rank` read the shard dirs too. Or call
|
|
`gstack-detach [--lock NAME] [--timeout SECS] [--label LBL] --
|
|
<cmd>` directly for any long agent job. Export `ANTHROPIC_API_KEY` first (never
|
|
pass keys in argv).
|
|
- Then **poll the printed logfile** with a death-aware watcher: break on the
|
|
guaranteed `### gstack-detach EXIT=<code> ###` sentinel (success AND failure are
|
|
both marked, so silence is never mistaken for success). The detached run survives
|
|
even if your watcher gets reaped, so re-checking the log always works.
|
|
- Why the lock: a shared dev box with several Conductor worktrees will rate-limit
|
|
the model API if two eval suites run at once (15-way concurrency each), which
|
|
mass-times-out E2E tests. The lock makes the second run WAIT, not collide.
|
|
- Humans running `bun run test:evals` foreground in their own terminal don't need
|
|
this — Ctrl-C is intended there. Detachment is for agent-launched runs only.
|
|
|
|
## E2E test fixtures: extract, don't copy
|
|
|
|
**NEVER copy a full SKILL.md file into an E2E test fixture.** SKILL.md files are
|
|
1500-2000 lines. When `claude -p` reads a file that large, context bloat causes
|
|
timeouts, flaky turn limits, and tests that take 5-10x longer than necessary.
|
|
|
|
Instead, extract only the section the test actually needs:
|
|
|
|
```typescript
|
|
// BAD — agent reads 1900 lines, burns tokens on irrelevant sections
|
|
fs.copyFileSync(path.join(ROOT, 'ship', 'SKILL.md'), path.join(dir, 'ship-SKILL.md'));
|
|
|
|
// GOOD — agent reads ~60 lines, finishes in 38s instead of timing out
|
|
const full = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
const start = full.indexOf('## Review Readiness Dashboard');
|
|
const end = full.indexOf('\n---\n', start);
|
|
fs.writeFileSync(path.join(dir, 'ship-SKILL.md'), full.slice(start, end > start ? end : undefined));
|
|
```
|
|
|
|
Also when running targeted E2E tests to debug failures:
|
|
- Run in **foreground** (`bun test ...`), not background with `&` and `tee`
|
|
- Never `pkill` running eval processes and restart — you lose results and waste money
|
|
- One clean run beats three killed-and-restarted runs
|
|
|
|
## Publishing native OpenClaw skills to ClawHub
|
|
|
|
Native OpenClaw skills live in `openclaw/skills/gstack-openclaw-*/SKILL.md`.
|
|
The command is `clawhub publish` (NOT `clawhub skill publish`) — full
|
|
workflow, auth, and verification:
|
|
[docs/OPENCLAW_PUBLISHING.md](docs/OPENCLAW_PUBLISHING.md).
|
|
|
|
## Deploying to the active skill
|
|
|
|
The active skill lives at `~/.claude/skills/gstack/`. After making changes:
|
|
|
|
1. Push your branch
|
|
2. Fetch and reset in the skill directory: `cd ~/.claude/skills/gstack && git fetch origin && git reset --hard origin/main`
|
|
3. Rebuild: `cd ~/.claude/skills/gstack && bun run build`
|
|
|
|
**If you use gbrain:** the `git reset --hard` in step 2 reverts the brain-aware
|
|
(`GBRAIN_CONTEXT_LOAD` / `GBRAIN_SAVE_RESULTS`) blocks that `gstack-config
|
|
gbrain-refresh` renders into the install (those generated blocks differ from
|
|
`main` by design). After deploying, re-run `gstack-config gbrain-refresh` to
|
|
restore them across all your projects' Claude sessions. It's idempotent.
|
|
|
|
Or copy the binaries directly:
|
|
- `cp browse/dist/browse ~/.claude/skills/gstack/browse/dist/browse`
|
|
- `cp design/dist/design ~/.claude/skills/gstack/design/dist/design`
|
|
|
|
## Skill routing
|
|
|
|
When the user's request matches an available skill, invoke it via the Skill tool. When in doubt, invoke the skill.
|
|
|
|
Key routing rules:
|
|
- Product ideas/brainstorming → invoke /office-hours
|
|
- Strategy/scope → invoke /plan-ceo-review
|
|
- Architecture → invoke /plan-eng-review
|
|
- Design system/plan review → invoke /design-consultation or /plan-design-review
|
|
- Full review pipeline → invoke /autoplan
|
|
- Bugs/errors → invoke /investigate
|
|
- QA/testing site behavior → invoke /qa or /qa-only
|
|
- Code review/diff check → invoke /review
|
|
- Visual polish → invoke /design-review
|
|
- Ship/deploy/PR → invoke /ship or /land-and-deploy
|
|
- Save progress → invoke /context-save
|
|
- Resume context → invoke /context-restore
|
|
|
|
## Cross-session decision memory
|
|
|
|
Durable decisions and their rationale are captured in an append-only, event-sourced
|
|
store at `~/.gstack/projects/<slug>/decisions.jsonl` so neither you nor the user
|
|
re-litigates a settled call or loses the "why" across sessions. This is the reliable,
|
|
file-only path: it works with gbrain OFF. (gbrain semantic recall is an optional
|
|
enhancement layered on top, never a dependency.)
|
|
|
|
- **Resurface** active decisions before re-deciding: `bin/gstack-decision-search`
|
|
(`--recent N`, `--scope repo|branch|issue`, `--query KW`, `--all`, `--json`).
|
|
Add `--semantic` (with `--query`) to append related hits from gbrain memory when
|
|
it's up; it degrades silently to the reliable file results when gbrain is off.
|
|
Session start already surfaces scope-relevant active decisions via Context Recovery.
|
|
If a decision is listed, treat it as settled with its rationale; if you're about to
|
|
reverse it, say so explicitly.
|
|
- **Capture** a DURABLE decision when you or the user make one:
|
|
`bin/gstack-decision-log '{"decision":"...","rationale":"...","scope":"repo|branch|issue","source":"user|skill|agent","confidence":1-10}'`.
|
|
Reverse a prior call with `--supersede <id>`; expunge an accidental secret with
|
|
`--redact <id>`; rewrite the log to the active set with `--compact`. Non-interactive
|
|
(never prompts), injection-sanitized, and HIGH-secret-blocking on write.
|
|
- **Durable means:** architecture choice, scope cut, tool/vendor choice, or a reversal
|
|
of a prior call. NOT a turn-level edit, a phrasing tweak, or anything trivially
|
|
re-derivable. Capture is curated at the source — log durable decisions only, or the
|
|
store becomes noise.
|
|
|
|
## GBrain Search Guidance (configured by /sync-gbrain)
|
|
<!-- gstack-gbrain-search-guidance:start -->
|
|
|
|
GBrain is set up and synced on this machine. The agent should prefer gbrain
|
|
over Grep when the question is semantic or when you don't know the exact
|
|
identifier yet.
|
|
|
|
**This worktree is pinned to a worktree-scoped code source** via the
|
|
`.gbrain-source` file in the repo root (kubectl-style context). Any
|
|
`gbrain code-def`, `code-refs`, `code-callers`, `code-callees`, or `query`
|
|
call from anywhere under this worktree routes to that source by default —
|
|
no `--source` flag needed. Conductor sibling worktrees of the same repo
|
|
each have their own pin and their own indexed pages, so semantic results
|
|
match the actual code on disk in this worktree.
|
|
|
|
Two indexed corpora available via the `gbrain` CLI:
|
|
- This worktree's code (auto-pinned via `.gbrain-source`).
|
|
- `~/.gstack/` curated memory (registered as `gstack-brain-<user>` source via
|
|
the existing federation pipeline).
|
|
|
|
Prefer gbrain when:
|
|
- "Where is X handled?" / semantic intent, no exact string yet:
|
|
`gbrain search "<terms>"` or `gbrain query "<question>"`
|
|
- "Where is symbol Y defined?" / symbol-based code questions:
|
|
`gbrain code-def <symbol>` or `gbrain code-refs <symbol>`
|
|
- "What calls Y?" / "What does Y depend on?":
|
|
`gbrain code-callers <symbol>` / `gbrain code-callees <symbol>`
|
|
- "What did we decide last time?" / past plans, retros, learnings:
|
|
`gbrain search "<terms>" --source gstack-brain-<user>`
|
|
|
|
Grep is still right for known exact strings, regex, multiline patterns, and
|
|
file globs. Run `/sync-gbrain` after meaningful code changes; for ongoing
|
|
auto-sync across all worktrees, run `gbrain autopilot --install` once per
|
|
machine — gbrain's daemon handles incremental refresh on a schedule.
|
|
|
|
Safety: don't run `/sync-gbrain` while `gbrain autopilot` is active — the
|
|
orchestrator refuses destructive source ops when it detects a running autopilot
|
|
to avoid racing it (#1734). Prefer registering user repos with `gbrain sources
|
|
add --path <dir>` (no `--url`): URL-managed sources can auto-reclone, and the
|
|
sync code walk for them requires an explicit `--allow-reclone` opt-in.
|
|
|
|
<!-- gstack-gbrain-search-guidance:end -->
|