mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-17 13:44:32 +01:00
Merge main 1ccab0d into gsta258-l1-release-gate (rule 19 grant, GSTA-36)
Resolve scripts/harness-version.json against #3903: take main's manifest and carry this branch's test-pr-profile.ts pin into scripts/harness-pins/scripts/test-pr-profile.ts.json. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
commit
c0d33a2195
133 files changed
+1414
-240
No files matched your search
@@ -1001,8 +1001,10 @@ paid test file, its fixtures and its skills' prompt files, including the
|
||||
generated SKILL.md) plus `HARNESS_VERSION`, keyed with model, Claude CLI version
|
||||
and policy version (`scripts/eval-trial-series.ts`). Each trial also records the
|
||||
full consumed-input fingerprint as provenance. `scripts/harness-version.json`
|
||||
pins every shared-harness file (`GLOBAL_TOUCHFILES` plus the session runners, the
|
||||
PTY harness, the e2e gate and the LLM judge) by blob hash. When
|
||||
holds the version and the harness patterns, and `scripts/harness-pins/<path>.json`
|
||||
pins each shared-harness file (`GLOBAL_TOUCHFILES` plus the session runners, the
|
||||
PTY harness, the e2e gate and the LLM judge) by blob hash, one file per pin, so
|
||||
two PRs that record decisions for different harness files never conflict. When
|
||||
`test/harness-version.test.ts` fails, run one of:
|
||||
|
||||
```bash
|
||||
@@ -1010,8 +1012,8 @@ bun run scripts/bump-harness-version.ts --non-behavioral "<why no verdict can ch
|
||||
bun run scripts/bump-harness-version.ts --bump "<what behavior changed>" # new series for every case
|
||||
```
|
||||
|
||||
On a merge conflict in the manifest, take the higher version, then rerun the
|
||||
script.
|
||||
On a merge conflict in the manifest or a pin file, take the higher version, then
|
||||
rerun the script.
|
||||
|
||||
## Ubicloud VMs (`bun run test:ubicloud`)
|
||||
|
||||
|
||||
+6
-4
@@ -80,14 +80,16 @@ It is off unless you turn it on, because turning it on changes the harness's own
|
||||
|---|---|---|
|
||||
| Plugin folder | `${OPENCLAW_STATE_DIR:-~/.openclaw}/extensions/gstack-start` (index.js, openclaw.plugin.json, package.json, note.txt) | `${HERMES_HOME:-~/.hermes}/plugins/gstack-start` (plugin.yaml, __init__.py, note.txt) |
|
||||
| Hook | `before_prompt_build`, returning the note as `prependContext` | `pre_llm_call`, returning the note as `context` |
|
||||
| Settings it changes, through the harness's own CLI | `openclaw config set plugins.entries.gstack-start.hooks.allowConversationAccess true` and `...enabled true` in `openclaw.json`. OpenClaw runs `before_prompt_build` from a plugin it did not bundle only with that grant, which also lets the plugin's hooks see conversation content (the plugin reads only its own note.txt) | `hermes plugins enable gstack-start` adds it to `plugins.enabled` in `config.yaml` (and creates `config.yaml` when there is none) |
|
||||
| Settings it changes, through the harness's own CLI | `openclaw config set plugins.entries.gstack-start.hooks.allowConversationAccess true` and `...enabled true` in `openclaw.json`. OpenClaw runs `before_prompt_build` from a plugin it did not bundle only with that grant, which also lets the plugin's hooks see conversation content (the plugin reads only its own note.txt) | `hermes plugins enable gstack-start` adds it to `plugins.enabled` in `config.yaml` |
|
||||
| Check | `openclaw plugins inspect gstack-start --runtime --json` lists `before_prompt_build` under `typedHooks` | `hermes plugins list` shows `gstack-start` enabled |
|
||||
|
||||
**Turn it on.** `gstack-harness register openclaw --start-hook` (or `hermes`), or `gstack-config set harness_start_hook on` followed by `./setup`, or `GSTACK_HARNESS_START_HOOK=1 ./setup --host openclaw`. The first of these that is set decides: the `--start-hook`/`--no-start-hook` flag, then `GSTACK_HARNESS_START_HOOK` (`1`/`on` or `0`/`off`), then `harness_start_hook`, then the choice saved in the registration's ledger. Whatever decides, the choice is saved, so refresh and upgrades keep it. The hook goes only into the harness profile's own plugin folder; a registration with a custom `--skills-dir`, or OpenClaw run with `OPENCLAW_CONFIG_PATH` pointing at another config, gets no hook (`HARNESS_HOOK_UNSUPPORTED` when you ask for one). A hook you already turned on stays on when a later run happens to have `OPENCLAW_CONFIG_PATH` set.
|
||||
**Turn it on.** `gstack-harness register openclaw --start-hook` (or `hermes`), or `gstack-config set harness_start_hook on` followed by `./setup`, or `GSTACK_HARNESS_START_HOOK=1 ./setup --host openclaw`. The first of these that is set decides: the `--start-hook`/`--no-start-hook` flag, then `GSTACK_HARNESS_START_HOOK` (`1`/`on` or `0`/`off`), then `harness_start_hook`, then the choice saved in the registration's ledger. Whatever decides, the choice is saved, so refresh and upgrades keep it. The hook goes only into the harness profile's own plugin folder; a registration with a custom `--skills-dir`, or OpenClaw run with `OPENCLAW_CONFIG_PATH` pointing at another config, gets no hook (`HARNESS_HOOK_UNSUPPORTED` with `--start-hook`; with `GSTACK_HARNESS_START_HOOK` or `harness_start_hook`, `register` and `./setup --host` say why and exit 4). `OPENCLAW_CONFIG_PATH` naming the profile's own `openclaw.json`, by any path that resolves to it, is fine. A hook you already turned on stays on when a later run happens to have `OPENCLAW_CONFIG_PATH` set.
|
||||
|
||||
**Check it.** The receipt and `gstack-harness status` show `start hook: active` (configured for new sessions; a running OpenClaw gateway or Hermes gateway picks it up when it reloads or restarts), `written` (files in place but not active, with the reason and the command that fixes it), `disabled` (you turned it off in the harness), `unchecked` (a run that left a hook already on alone, or `status --no-check`; `gstack-harness status` checks it) or `off`. When the run that turns the hook on (or `--start-hook`) leaves it inactive, `gstack-harness register` and `./setup --host` exit 4 after registering the skills (`HARNESS_HOOK_INACTIVE`). Later refreshes and upgrades try once more each time, without failing, until the hook is active or you disable the plugin in the harness.
|
||||
**Run the harness's own setup first.** gstack turns the hook on only in a harness that already has its config (`openclaw.json` from `openclaw onboard`, `config.yaml` from `hermes setup`). Without one, the harness CLI would create a config holding nothing but gstack's plugin setting, with no model, provider or terminal settings, so gstack runs no harness command: the plugin's files are written, the hook reports `written` with the setup to run, and `register` and `./setup --host` exit 4. Once the harness is set up, the next `./setup` or `gstack-harness refresh` turns the hook on, changing only the `gstack-start` plugin entry.
|
||||
|
||||
**Turn it off.** `gstack-harness register openclaw --no-start-hook` (or `hermes`), or `gstack-config set harness_start_hook off` followed by `./setup`. Either turns the plugin off through the harness's CLI and removes its folder. You can also switch it off in the harness (`openclaw config set plugins.entries.gstack-start.enabled false`, `hermes plugins disable gstack-start`); gstack then leaves it off and never turns it back on unless you pass `--start-hook` again. `gstack-harness unregister` and `gstack-uninstall` remove it with the rest (Hermes only disables it when its folder holds an edited file or a kept `.edited-*` copy, so your edit survives), and print the harness command to run when the harness CLI is not on `PATH`.
|
||||
**Check it.** The receipt and `gstack-harness status` show `start hook: active` (configured for new sessions; a running OpenClaw gateway or Hermes gateway picks it up when it reloads or restarts), `written` (files in place but not active, with the reason and the command that fixes it), `disabled` (you turned it off in the harness), `unknown` (the harness CLI is not on `PATH`, as inside OpenClaw's or Hermes's own terminal tool, and the config file cannot settle it; gstack reads `plugins.enabled` and `plugins.disabled` from Hermes's `config.yaml` and the plugin entry from `openclaw.json` instead), `unchecked` (a run that left a hook already on alone, or `status --no-check`; `gstack-harness status` checks it) or `off`. When the run that turns the hook on (or `--start-hook`) leaves it inactive, `gstack-harness register` and `./setup --host` exit 4 after registering the skills (`HARNESS_HOOK_INACTIVE`). Later refreshes and upgrades try once more each time, without failing, until the hook is active or you disable the plugin in the harness.
|
||||
|
||||
**Turn it off.** `gstack-harness register openclaw --no-start-hook` (or `hermes`), or `gstack-config set harness_start_hook off` followed by `./setup`. Either turns the plugin off through the harness's CLI and removes its folder. You can also switch it off in the harness (`openclaw config set plugins.entries.gstack-start.enabled false`, `hermes plugins disable gstack-start`); gstack then leaves it off and never turns it back on unless you pass `--start-hook` again. `gstack-harness unregister` and `gstack-uninstall` remove it with the rest (Hermes only disables it when its folder holds an edited file or a kept `.edited-*` copy, so your edit survives), and print the harness command to run when the harness CLI is not on `PATH`. In a profile with no harness config yet (no `openclaw.json` or `config.yaml`), the hook was never turned on, so turning it off or unregistering runs no harness command and leaves the config alone.
|
||||
|
||||
**What it costs.** The note is about 50 tokens and lands on every user turn, and each copy stays in the session's history, so a long session pays more (mostly cache reads). HARN-18's panel measures the added cost per trial. OpenClaw also logs that it cannot verify where the plugin came from, because gstack writes it instead of installing it from npm or ClawHub.
|
||||
|
||||
|
||||
+5
-2
@@ -201,9 +201,12 @@ Two stages with separate clocks, both printed:
|
||||
file is read whole from disk. A file over the size limit, or whose full diff
|
||||
does not parse, is scanned on its added lines alone, without that context.
|
||||
- `strays`: a new file at the repository root, a new top-level directory, or a
|
||||
scratch name (`zz-*`, `tmp*`, `scratch*`) anywhere.
|
||||
scratch name (`zz-*`, `tmp*`, `scratch*`) or editor leftover (`*.orig`,
|
||||
`*.rej`, `*.swp`, `*.swo`, `*.bak`, `*.tmp`, `*~`, `.DS_Store`, `Thumbs.db`)
|
||||
anywhere.
|
||||
- `literals`: for every changed exported constant or named timeout/retry
|
||||
literal, the whole test tree (nightly- and Heavy-only files included) is
|
||||
literal (a named literal the branch only adds, such as `limit: 300` in a new
|
||||
object, is not a change), the whole test tree (nightly- and Heavy-only files included) is
|
||||
searched for the old value in files that name the constant or, when the old
|
||||
value is longer than one character, its module (the module's file name not
|
||||
followed by another extension, or a path ending
|
||||
|
||||
+4
-1
@@ -1618,7 +1618,10 @@ the reviewed range (the reviewed head excluded, at most 100 commits) whose
|
||||
product change the current base already holds: `git merge-tree` of that commit
|
||||
into the base changes no product path. A conflicted product path never counts,
|
||||
so a parent that never landed is never treated as one, even when the child's
|
||||
merge dropped its change. The comparison then runs from that commit: an equal
|
||||
merge dropped its change. The comparison then runs from that commit, leaving out
|
||||
every path whose content at the reviewed head is the base's (a base merge the
|
||||
child made before its receipt brought it in) unless HEAD changed it against the
|
||||
base (GST-1214): an equal
|
||||
patch-id carries with reason `stack parent <sha12> landed`, base context next
|
||||
to the child's unchanged hunks is `RECEIPT_CARRY_SCOPED_REVIEW`, and any other
|
||||
difference refuses with `RECEIPT_CARRY_PRODUCT_CHANGED`.
|
||||
|
||||
@@ -50,14 +50,14 @@ per-failure-class breakdown from `test:health`.
|
||||
full consumed-input fingerprint (all touchfiles, globals included) as
|
||||
`series_fingerprint`, plus `harness_version`, as provenance.
|
||||
(`caseSeriesIdentitiesV2` in `scripts/eval-trial-series.ts`.)
|
||||
- **Harness guard.** `scripts/harness-version.json` pins the blob of every
|
||||
shared execution-harness file: `GLOBAL_TOUCHFILES` plus the session runners,
|
||||
the PTY harness, the e2e gate and the LLM judge (36 files). A free test
|
||||
- **Harness guard.** `scripts/harness-pins/<path>.json` pins the blob of every
|
||||
shared execution-harness file (`scripts/harness-version.json` keeps the version and patterns): `GLOBAL_TOUCHFILES` plus the session runners,
|
||||
the PTY harness, the e2e gate and the LLM judge (42 files). A free test
|
||||
(`test/harness-version.test.ts`) fails when one changes without a decision.
|
||||
`bun run scripts/bump-harness-version.ts --bump "<reason>"` raises
|
||||
`HARNESS_VERSION` and starts a new series for every case;
|
||||
`--non-behavioral "<reason>"` records the new blobs as compatible with no
|
||||
reset. Entries are one line per file so concurrent PRs that touch different
|
||||
reset. Each pin is its own file so concurrent PRs that touch different
|
||||
files merge cleanly; on a conflict, take the higher version, then rerun.
|
||||
- **Weekly history (W4b).** Weeks for quarantine expiry and drift are
|
||||
`evals-periodic.yml` runs on `main`: scheduled runs and main dispatches.
|
||||
|
||||
@@ -3789,7 +3789,9 @@ anchored pattern and a reason ([docs/pregate.md](pregate.md#allowlist));
|
||||
### `PREGATE_STRAYS`
|
||||
|
||||
**Meaning.** A file was added at the repository root, in a new top-level
|
||||
directory, or under a scratch name (`zz-*`, `tmp*`, `scratch*`).
|
||||
directory, or under a scratch name (`zz-*`, `tmp*`, `scratch*`) or an editor
|
||||
leftover name (`*.orig`, `*.rej`, `*.swp`, `*.swo`, `*.bak`, `*.tmp`, `*~`,
|
||||
`.DS_Store`, `Thumbs.db`).
|
||||
|
||||
**Fix.** Delete or move it. An intentional root file goes in `strays.allow`.
|
||||
|
||||
@@ -6470,7 +6472,7 @@ There is no ledger for that harness or id under `$GSTACK_STATE_ROOT/harness`. `g
|
||||
<a id="harness-hook-unsupported"></a>
|
||||
### `gstack-harness: <harness> has no start hook` / `the start hook goes into the <harness> profile <dir>, and this registration uses another skills folder` (HARNESS_HOOK_UNSUPPORTED)
|
||||
|
||||
The [start hook](hosts.md#start-hook) exists for OpenClaw and Hermes only, and it goes into the harness profile's own plugin folder, the one the harness scans. A registration with `--skills-dir` somewhere else has no profile gstack can trust, and with `OPENCLAW_CONFIG_PATH` set OpenClaw reads a config outside the profile, where gstack's settings change would not land, so `--start-hook` is refused and nothing is written. Register without `--skills-dir` (set `OPENCLAW_STATE_DIR` or `HERMES_HOME` to pick the profile), or drop `--start-hook`. When the choice comes from `GSTACK_HARNESS_START_HOOK` or `harness_start_hook` instead of the flag, such a registration just gets no hook, and its receipt says so.
|
||||
The [start hook](hosts.md#start-hook) exists for OpenClaw and Hermes only, and it goes into the harness profile's own plugin folder, the one the harness scans. A registration with `--skills-dir` somewhere else has no profile gstack can trust, and with `OPENCLAW_CONFIG_PATH` naming another config OpenClaw reads settings outside the profile, where gstack's settings change would not land, so `--start-hook` is refused and nothing is written. `OPENCLAW_CONFIG_PATH` naming the profile's own `openclaw.json` (compared after resolving symlinks) is not refused. Register without `--skills-dir` (set `OPENCLAW_STATE_DIR` or `HERMES_HOME` to pick the profile), unset `OPENCLAW_CONFIG_PATH` or point it at the profile's own `openclaw.json`, or drop `--start-hook`. When the choice comes from `GSTACK_HARNESS_START_HOOK` or `harness_start_hook` instead of the flag, the registration gets no hook, and `gstack-harness register` and `./setup --host` exit 4 with the reason (`HARNESS_HOOK_INACTIVE`, `start hook is off`); refreshes only print it.
|
||||
|
||||
<a id="harness-hook-choice-invalid"></a>
|
||||
### `gstack-harness: GSTACK_HARNESS_START_HOOK="<value>" is not on or off` (HARNESS_HOOK_CHOICE_INVALID)
|
||||
@@ -6485,7 +6487,9 @@ Each harness profile has one start hook, owned by the gstack registration that w
|
||||
<a id="harness-hook-inactive"></a>
|
||||
### `gstack-harness: <harness> start hook is written: <why>. Turn it on: <command>` (HARNESS_HOOK_INACTIVE, exit 4)
|
||||
|
||||
The skills are registered and the start hook's files are in place, but the hook is not active in the harness. gstack turns the hook on through the harness's own CLI and then reads it back, so this means the CLI was not on `PATH`, one of its commands failed or timed out (the message quotes its last output), or the harness's settings block the plugin: OpenClaw's `plugins.allow` set without `gstack-start`, `gstack-start` in `plugins.deny`, or `hooks.allowPromptInjection: false`. gstack never edits your allow or deny lists. Run the printed command, then `gstack-harness status`; `--start-hook` again retries the whole step. `./setup --host` exits 4 for the same reason. Refreshes and upgrades retry on their own without failing, and stop once you disable the plugin in the harness.
|
||||
The skills are registered and the start hook's files are in place, but the hook is not active in the harness. gstack turns the hook on through the harness's own CLI and then reads it back, so this means the harness has no config yet (`no OpenClaw config at <path> yet` or `no Hermes config at <path> yet`: run `openclaw onboard` or `hermes setup` first, then `gstack-harness refresh`; gstack runs no harness command until then, because the CLI would create a config holding only gstack's plugin setting), the CLI was not on `PATH`, one of its commands failed or timed out (the message quotes its last output), or the harness's settings block the plugin: OpenClaw's `plugins.allow` set without `gstack-start`, `gstack-start` in `plugins.deny`, or `hooks.allowPromptInjection: false`. gstack never edits your allow or deny lists. Run the printed command, then `gstack-harness status`; `--start-hook` again retries the whole step. `./setup --host` exits 4 for the same reason. Refreshes and upgrades retry on their own without failing, and stop once you disable the plugin in the harness.
|
||||
|
||||
The same code with `start hook is off: <why>` means you asked for the hook through `GSTACK_HARNESS_START_HOOK` or `harness_start_hook` but this registration cannot take it (a custom `--skills-dir`, `OPENCLAW_CONFIG_PATH` naming another config, or an unsafe plugin folder); the message names the fix.
|
||||
|
||||
<a id="harness-qm-credentials"></a>
|
||||
### `gstack-harness: no QM admin credential in the environment` / `... needs --yes` (HARNESS_QM_CREDENTIALS)
|
||||
|
||||
Reference in new issue
Block a user