v1.93.0.0 stamp: merge main 1a7f2578b7 into #3858 (land)

docs/TESTING_INTERNALS.md

@@ -195,15 +195,18 @@ fallback; unknown files get 99th-percentile pessimism, and both full-suite and
 the long pole. Packed
 shards get duration-aware walls (`max(base, predicted × 3, files × 5s)`). The
 legacy `--shards N --shard i` path keeps stable hash indices. Required CI uses
-one duration-packed `--ci-plan`, 20 ordinary `--ci-run` shards plus a separate
+one duration-packed `--ci-plan`, 10 ordinary `--ci-run` shards plus a separate
 exclusive-fixture shard, and a `--ci-verify` aggregate. CI shards still run on
-independent machines without ordering unrelated jobs. Two workflow-level
+independent machines without ordering unrelated jobs. Three workflow-level
 constants in `free-tests.yml` set that size for every event: `FREE_SUITE_RUNNER`
-(`ubicloud-standard-4`) and `FREE_SUITE_SHARDS` (`'20'`), which `free-plan`
-resolves into the shards' runner and shard count. A `workflow_dispatch` run may
-override them with `free_runner` (`ubicloud-standard-2`, `-4` or `-8`) and
-`free_shards` (1–40) for runner-sizing experiments only; its job summary says
-so, and it is never cited as a PR or train Free Tests result. Each shard prints
+(`ubicloud-standard-8`), `FREE_SUITE_SHARDS` (`'10'`) and `FREE_SUITE_JOBS`
+(`'4'`, the [sub-shards](#sub-shards-inside-one-ci-job) each shard runs side by
+side), which `free-plan` resolves into the shards' runner, shard count and plan.
+GSTA-203 chose them by a 3-run A/B against 20 × `ubicloud-standard-4` at k = 1
+(the PR that set them has the table). A `workflow_dispatch` run may
+override them with `free_runner` (`ubicloud-standard-2`, `-4` or `-8`),
+`free_shards` (1–40) and `free_jobs` (1–8) for runner-sizing experiments only;
+its job summary says so, and it is never cited as a PR or train Free Tests result. Each shard prints
 `RESOURCES: peak_used_mib=… min_free_disk_mib=… samples=…` from a memory and
 disk sample every 2 seconds, even when the suite fails. The public
 `TREE_MUTATING` map now classifies exclusive host-state fixtures; its sole
@@ -1553,7 +1556,7 @@ refresh to a maintainer.
   `scripts/ubicloud/setup-free-suite.sh` and the Windows seed through
   `windows-free-tests.yml`'s `record_durations` path. The recorder stays on
   `ubicloud-standard-8` while the free-suite shards run on `FREE_SUITE_RUNNER`
-  (`ubicloud-standard-4`); both are Ubicloud standard runners. When either recording adds
+  (`ubicloud-standard-8`); both are Ubicloud standard runners. When either recording adds
   or drops a test file, it force-updates the `bot/free-duration-seeds` branch,
   opens or refreshes one seed PR whose body states the ratchet result, and
   dispatches Free Tests, Windows Smoke and Quality gate on it. Those
@@ -1575,14 +1578,24 @@ k sub-shards side by side on the same machine (`scripts/lib/shard-jobs.ts`). Eac
 sub-shard is an ordinary free shard child: one `bun test` process, serial
 inside, with its own sandbox, process-group kill and log. Bun's worker
 `--parallel` mode is still not used (decision V3/D6 in
-`scripts/test-free-shards.ts`). The default is `k = 1`, which runs today's one
-child, so no workflow changes behavior until it passes `--jobs`.
+`scripts/test-free-shards.ts`). Without `--jobs`, a shard runs at the plan's
+per-shard k (below); a shard with no planned k runs one child (`k = 1`).
 
-- `--jobs` applies only to `--ci-run`. It takes a positive whole number up to the
-  shard's file count (an empty shard takes 1); anything else stops with
-  `FREE_JOBS_INVALID`. `GSTACK_FREE_JOBS` sets the local full-suite process count
-  and never changes `--ci-run`. The exclusive host-state shard has one file, so
-  a workflow passes it `--jobs 1` or no `--jobs`.
+- Workflows set k in the plan, per shard: `--ci-plan <plan> --shards N
+  --shard-jobs k` stores `jobs = min(k, the shard's files that may run in
+  parallel)` on each shard, and only when that is above 1. The exclusive
+  host-state shard holds only `TREE_MUTATING` files, so it stays one child, and
+  a plan made at k = 1 is byte-identical to one made without `--shard-jobs`.
+  `--ci-run` runs the shard at the plan's k, and `--ci-verify` rejects any
+  shard result that ran at a different k. `free-tests.yml` reads k from
+  `FREE_SUITE_JOBS` and `windows-free-tests.yml` from `WINDOWS_FREE_JOBS`; the
+  `free_jobs` and `windows_jobs` dispatch inputs override them for experiments
+  only.
+- `--jobs` applies only to `--ci-run` and overrides the plan's k, for local
+  reproduction. It takes a positive whole number up to the shard's file count
+  (an empty shard takes 1); anything else stops with `FREE_JOBS_INVALID`, as
+  does a bad `--shard-jobs` or one without `--ci-plan`. `GSTACK_FREE_JOBS` sets
+  the local full-suite process count and never changes `--ci-run`.
 - Files are split into disjoint sub-shards packed by the duration seed. Each
   sub-shard has an id (`<plan id prefix>:<shard>.<n>`) bound to its file list.
   Files in `TREE_MUTATING` or `SLOW_UNDER_LOAD` run after the parallel
@@ -1606,6 +1619,13 @@ reproduce command. If the file passes serially, add it to `SLOW_UNDER_LOAD` in
 `scripts/lib/shard-jobs.ts` with that run id and a reason. Never raise the
 timeout. `SLOW_UNDER_LOAD` is empty today.
 
+The nightly Windows lane stays at `WINDOWS_FREE_JOBS: '1'`. In GSTA-203's A/B, k = 2
+made the slowest Windows shard faster, but two of three runs needed the flaky
+retry for file-lock cleanup errors (`EBUSY` in
+`browse/test/stealth-webdriver.test.ts` and a native fixture cleanup in
+`browse/test/cookie-import-native-job.test.ts`), which k = 1 never showed. Fix
+those cleanups before raising it.
+
 Every `--ci-run` result, at any k, records per-file test and `expect()` counts
 in `outcome.counts` (`tests`, `pass`, `fail`, `skip`, `todo`, `expects` per
 file, plus shard totals). They come from bun's JUnit reporter, which the runner
@@ -1624,8 +1644,9 @@ test list in its last step, where each file's reason sits beside it.
 `test/windows-native-workflows.test.ts` keeps that list Windows-safe and fast.
 
 The curated suite below runs nightly on main and on dispatch, never on a PR.
-`windows-free-tests.yml` plans on Linux (`--windows-only --ci-plan --shards 6`),
-runs one strict serial shard per `windows-latest` job
+`windows-free-tests.yml` plans on Linux (`--windows-only --ci-plan --shards 6
+--shard-jobs $WINDOWS_FREE_JOBS`), runs one strict serial shard per
+`windows-latest` job
 (`bun run test:windows --ci-run`), and verifies every shard receipt on Linux
 (`--windows-only --ci-verify`). Packing uses Windows-measured durations from
 `scripts/free-test-durations-windows.json`; the weekly

docs/designs/PLAN_TUNING_V0.md

@@ -105,7 +105,7 @@ After weighing Codex's argument, we chose to roll back CEO EXPANSION and ship an
 }
 ```
 
-`signal_events` counts the logged answers that moved any dimension, and `contributing_events` counts, per dimension, the logged answers that moved it (an answer that writes two deltas to one dimension counts once), so a dimension with no evidence (0) is visible beside one whose evidence balances out; both print 0.5 (#2623). Each value is the sigmoid of its accumulated deltas, and each UTC day's sum per dimension (by the row's `ts`) is clamped to the largest single delta in the signal map (currently ±0.06), so one burst of answers can't pin a dimension; `--trace <dim>` lists every contributing answer, each clamped day and the total. The bound keys on the day, not on `session_id`: the PostToolUse hook and the agent log different ids for the same review (the host's conversation id and the preamble's run id), and an agent can leave the `SESSION_ID` placeholder unsubstituted, so no logged id reliably marks one review. The cost is that several reviews on one day count as one. `contributing_events` still counts an answer logged by both writers twice. Only an ISO 8601 date or date-time `ts` counts as dated (a time without a zone is read as UTC), every other row shares one undated bucket, and `diversity.days_span` counts the same UTC days. Nothing displays `contributing_events` yet (#3436).
+`signal_events` counts the logged answers that moved any dimension, and `contributing_events` counts, per dimension, the logged answers that moved it (an answer that writes two deltas to one dimension counts once), so a dimension with no evidence (0) is visible beside one whose evidence balances out; both print 0.5 (#2623). Each value is the sigmoid of its accumulated deltas, and each UTC day's sum per dimension (by the row's `ts`) is clamped to the largest single delta in the signal map (currently ±0.06), so one burst of answers can't pin a dimension; `--trace <dim>` lists every contributing answer, each clamped day and the total. The bound keys on the day, not on `session_id`: the PostToolUse hook and the agent log different ids for the same review (the host's conversation id and the preamble's run id), and an agent can leave the `SESSION_ID` placeholder unsubstituted, so no logged id reliably marks one review. The cost is that several reviews on one day count as one. `contributing_events` still counts an answer logged by both writers twice. Only an ISO 8601 date or date-time `ts` counts as dated (a time without a zone is read as UTC), every other row shares one undated bucket, and `diversity.days_span` counts the same UTC days. `--check-mismatch`, `--gap`, the stored `gap` and /plan-tune's profile view compare a dimension only when its `contributing_events` is at least 3; `--gap` lists the others under `low_evidence` and the stored `gap` omits them (#3436).
 
 **Diversity check** (Codex #13): `inferred` is considered "enough data" only when `sample_size >= 20 AND skills_covered >= 3 AND question_ids_covered >= 8 AND days_span >= 7`. Below this, `/plan-tune profile` shows "not enough observed data yet" instead of a potentially-misleading inferred value.
 

docs/pregate.md

@@ -1114,13 +1114,33 @@ publication whenever the managed pre-push hook (`gstack-redact
 install-prepush-hook`, no reinstall needed) or the standalone preflight runs:
 
 ```
-gstack-pregate sealed [--range <a..b>] [--remote origin] [--base <branch>] [--json]
+gstack-pregate sealed [--range <a..b>] [--remote origin] [--base <branch>] [--no-skip] [--json]
 ```
 
 - **Boundary.** Every commit the push publishes: `git rev-list <local sha>
   --not --remotes=<remote>` (and the remote tip when the ref exists); a
   destination with no refs means everything reachable. `--range` names the
   commits explicitly.
+- <a id="sealed-skip"></a>**Skipped commits.** When the base branch resolves, a
+  listed commit is not rescanned when the destination already holds it or its
+  patch already landed. Held: an ancestor of the destination's own
+  `refs/pull/<n>/head` for a squash commit `(#n)` on the base since the
+  merge-base; the subject is only a hint, the fetched ref is the proof. That
+  lookup lists and fetches at most 50 refs within 20 seconds (receipted, sink
+  `sealed-guard`) and deletes the fetched refs afterwards. Landed: the commit's
+  own `git patch-id --verbatim` equals a base commit's since the merge-base and
+  so does its diff, apart from blob ids and line numbers; a merge never matches,
+  and above 2,000 base or 400 listed commits this check is off. A landed commit
+  skips only the path and text rules: `sealed.label_join` judges the whole file
+  at that commit, which the base may never have held, so it still runs. Nothing
+  matches a combined diff, so a parent whose commits were never published is
+  scanned whole even when its squash landed. A lookup that fails, runs out of
+  time or exceeds a limit prints one line and scans everything. Each PR group
+  and landed commit is listed (`skipped <n> commit(s) the destination already
+  has as refs/pull/<n>/head`, `skipped commit <sha> (its patch landed on the
+  base as <sha>)`), the `SEALED:` line counts them as `skipped=<n>` (its
+  `commits=` counts the commits scanned), and `--no-skip` scans every listed
+  commit. The pre-push hook and `gstack-pregate sealed` skip the same way.
 - **Rules.** `sealed.path` and `sealed.transcript` check every path a commit
   adds, renames or copies, whatever its content (binary, empty, deleted again
   in a later commit); `sealed.answer`, `sealed.seed` and `sealed.local_path`

docs/review-policy.md

@@ -429,7 +429,14 @@ rules hash, base and registry commits, the gate scan bound to the head and the r
 the queue refuses a receipt below the depth its head needs
 ([ship-policy.md#ship-receipt](ship-policy.md#ship-receipt)). A stop the owner
 accepted after a fix-check edit goes in `review_loop_exception`, never in prose
-under `REVIEW_LOOP:`.
+under `REVIEW_LOOP:`. Step 16 takes the same record:
+`gstack-ship-release finish --review-loop-exception <json|@file>` routes the
+paths the branch touched at `fix_check.head` (narrowed to the record's
+`fix_check.files` when it lists them) and changed since as `accepted=` in
+`ROUTE:`, prints them on a `REVIEW_LOOP_EXCEPTION:` line, and carries them
+through the docs stage. A path outside that scope and a gate-edited test still
+route as behavior, with `--reviewed-base` (as /ship passes it) so does a file
+too large for the scoped review, and without the record nothing changes.
 
 The receipt's summary line prints `passes=<n> blocking_found=<k> deferred=<m>`,
 which is what a scorecard tracks (target: a median of 1 pass per PR). A
@@ -572,7 +579,18 @@ ones, and the arm lost blockers. `bin/gstack-review-dispatch` closes both ends:
    missing coverage leaves the review incomplete. A mismatched send re-sent
    byte-exact before anything was collected is void and counted (`voided=N` on the
    verdict line); one that produced an output stays `MISMATCH`. Then output hashes
-   against the merged lines they produced, and the judgment rerun against the
+   against the merged lines they produced, and each merged line against the finding collect
+   parses at that id's index from the unchanged output file, label and raw text byte for byte
+   (`merged_line_differs` on the OUTPUT and JUDGE rows otherwise, GST-541), so an edited
+   merged.jsonl cannot lower a finding or swap its text. A recorded output file that is missing
+   (`output_file_missing`), a judgment run on a merged file collect did not write
+   (`judge_merged_not_collected`) and merged rows collect never recorded
+   (`merged_rows_not_collected`) are dirty too, and so is an output whose re-parsed findings
+   outnumber or undercount the ids collect recorded (`merged_count_differs`). The one exception
+   is a reply collect refused for its tree: verify re-checks the reply's `REVIEWED_TREE` line
+   against the render's tree, and such an output stays `NOT_REVIEWED` until a re-sent brief is
+   answered by a reply that passes the same check. Deleting a whole output event with its rows is
+   not caught yet, because the ledger has no anchor for it. The judgment is then rerun against the
    recorded judged set, covering every collected id. A ledger from a run that kept no
    prompts prints `VERIFY: no brief records` (exit 3), never clean.
 6. `pass --base <b> --out <dir> --pass <id> -- <plan flags>` runs one pass from a