train S3: merge #3847 at 678c05a729 v1.93.10.0

docs/TESTING_INTERNALS.md

@@ -651,15 +651,37 @@ this checkout to read it)`.
 samples, rationale}` entry per failing dimension (since EVAL_POLICY v3 `mean`
 holds the panel's gating median), each part capped at 200
 characters. `sessions[]` (at most 32) summarizes the trial's session-ledger
-rows. `cost_known: false` marks a record or trial whose harness captured no
-billing: PTY and Codex sessions never bill, and an SDK session bills only with a
-terminal `total_cost_usd`. It also marks a trial no result record could be
-attributed to: a record named by the case id or carrying `case_id` is that
-case's, and a record named by a display label (`/qa fix loop`) is the shard's
-only case's when the shard ran one case, so a multi-case shard whose records
-carry display labels leaves those trials cost-unknown rather than guessing
-which case paid (`junitCensus` in `scripts/lib/paid-report.ts`). An absent
-field means known. Census and marathon cost lines print the known sum and
+rows. `cost_known: false` marks a record or trial whose cost is not fully
+known; an absent field means known. A record carries it when its harness
+captured no billing (a Codex session that completed no turn or whose model has
+no price row). A rule case's trial in the JUnit census (`caseCost` in
+`scripts/lib/paid-report.ts`) costs its records' sum plus what no record
+carries: each ledger session without a terminal result (`billed: false`) adds
+its `streamed_cost_usd`, and a trial with no record adds each billed session's
+own `cost_usd` (Codex). The trial is cost-unknown when any record says
+`cost_known: false`, when any of those sessions has no priced cost (a PTY
+session whose transcript does not price, a `claude -p` session that ended
+without a result, an aborted Agent SDK session with no usage or an unpriced
+model), or when it has neither records
+nor sessions. A case in `MODEL_FREE_CASES` (`timeline-event-flow`, which runs
+no model) with neither is a known $0. A panel trial (`behavior`, `judge`) from
+an isolated trial shard keeps the older rule (`trialCostKnown` in
+`test/helpers/eval-store.ts`): known only with at least one record and every
+session billed, and streamed cost is not added.
+
+Records are attributed to a case this way: a record named by the case id or
+carrying `case_id` is that case's, and a record named by a display label
+(`/qa fix loop`) is the shard's only case's when the shard ran one case. In a
+multi-case shard, a display-label record whose `runSkillTest` test name is a
+registered case id carries `cost_case_id` (`recordE2E`), and the census adds
+its cost to that case only: the failed record, failure class and model still
+come from the records named by case id, so no verdict changes. A display-label
+record without `cost_case_id` in a multi-case shard is not attributed rather
+than guessed, and its case is cost-unknown when nothing else prices it
+(`junitCensus`). The report's headline `cost $<x>` adds the streamed session
+cost no collector record carries, so a newly priced trial counts once.
+
+Census and marathon cost lines print the known sum and
 `N of M trial(s) cost unknown`, and a train verdict's `ATTEMPT:` line prints
 `cost=$<known> known + <n> of <m> case(s) cost unknown` when some but not
 every case in the train record has a cost (a record with no priced case prints
@@ -671,13 +693,33 @@ including `runSharedInteractive`, the PTY session and `runRecordedCodexEval`)
 appends one best-effort line per armed session to
 `$GSTACK_EVAL_DIR/session-ledger.jsonl` (`test/helpers/session-ledger.ts`):
 a stable key `<runner>:<label>#n`, the exact `budget_ms` it armed, monotonic
-`elapsed_ms`, the `end` reason, sanitized evidence, `billed`, and a compact
+`elapsed_ms`, the `end` reason, sanitized evidence, `billed`, the billed
+`model`, `cost_usd` and token classes of a Codex session, and a compact
 liveness summary (longest in-request silence, the last event kind, any tool open
 at the end). `claude -p` sessions stream partial messages for this summary;
 transcripts keep their shape and size (full stream diagnostics only with
 `publicStreamDiagnostics`). A failed ledger write never fails a trial; a trial
 without ledger rows has no `sessions` and its headroom is `unknown`, never
-scored. The ledger rides in each slice artifact beside the shard results.
+scored. An Agent SDK session that ends without a terminal result (aborted,
+thrown) also records `streamed_cost_usd`: its streamed assistant usage priced
+by `streamedUsageCostUsd` (`lib/pricing.ts`), each message id once at its
+largest output, subagent turns included, `<synthetic>` messages skipped,
+5-minute cache writes at 1.25x and 1-hour writes at 2x the input rate. The
+field is absent when no message carries usage or any message's model has no
+price row, so an unpriced session stays cost-unknown, never $0. A PTY session
+records `streamed_cost_usd` from its transcript the same way. Both PTY ledger
+writers price `<CLAUDE_CONFIG_DIR>/projects/*/<id>.jsonl` plus
+`<id>/subagents/*.jsonl` (`test/helpers/pty/transcript-cost.ts`), where the
+id is the caller's `--session-id`, else the one transcript that appeared
+during this launch for its cwd. The launcher adds no argument. Pricing never
+takes the whole directory, because concurrent PTY tests in one process share
+one hermetic config dir; two new transcripts for one cwd are ambiguous and stay
+unknown. A non-hermetic or resumed launch, a caller that opts out
+(`priceTranscript: false`, the native AskUserQuestion capture, which never
+reads model transcripts), a missing transcript, no usage, an unpriced model or
+a read error leaves the field absent (cost-unknown), and a pricing failure
+never fails the test. The row keeps `billed: false`. The ledger rides in each slice
+artifact beside the shard results.
 
 **Quarantine** (`CASE_QUARANTINE`, same file). An entry needs: a per-trial rate
 below 95% over at least 10 post-policy trials of the case's current input
@@ -1232,9 +1274,11 @@ observation never removes a command and a missing or empty artifact never erases
 one; removing a command is a reviewed edit. While the map is in shadow mode it
 changes no selection, so a PR that changes only the map selects no paid case.
 
-Dollar figures are lower bounds. Known cost excludes PTY trials, whose cost is
-unknown: about 39% of observed case runs in the committed map. The shadow block
-and `test:health` both say so.
+Dollar figures are lower bounds. Known cost excludes trials whose cost is
+unknown: about 39% of observed case runs in the committed map, mostly PTY
+trials recorded before PTY sessions were priced from their transcripts
+([Failure causes and details](#failure-causes)). The shadow block and
+`test:health` both say so.
 
 - **Shadow block:** `prCoverage.shadow` in the paid-plan manifest lists the gate
   cases the map would select, would drop and would add, a provenance line per
@@ -1750,6 +1794,32 @@ lower-case words in path components under `test/fixtures/`; a substring match
 separate change with its own touchfile updates; the list is here so that change
 can be scoped.
 
+## Agent SDK system prompt
+
+An Agent SDK caller that leaves out `systemPrompt` doesn't get Claude Code's
+prompt. It gets the SDK's bare stub, and the stub drops the line that tells the
+model to run `/skill` through the Skill tool. A skill arm then never loads its
+skill, so a "no difference" result says nothing about the skill (FOUN-32 found
+this in another crew's null result).
+
+**Every caller says which prompt it uses.** `runAgentSdkTest` and every direct
+SDK `query()` call pass the Claude Code preset,
+`{ type: 'preset', preset: 'claude_code' }` (also exported as
+`CLAUDE_CODE_PRESET` from `test/helpers/sdk-system-prompt.ts`; an overlay arm
+adds `append`), or name why not: `{ type: 'sdk-default', reason }` for a stub
+provider or a connectivity probe, `{ type: 'custom', text, reason }` for an arm
+that replaces the prompt. The runner refuses a missing, empty, bare-string or
+unnamed prompt before it starts, and
+`test/sdk-system-prompt-tripwire.test.ts` scans test/ and scripts/ for the same
+mistake in free CI. Arms that compare loadouts, the no-gstack arm included,
+keep the same preset and differ only in what the arm loads.
+
+**Record it, and treat a null on the stub as a method bug.** Every eval report
+and results table names the prompt each arm ran under (`claude_code`,
+`claude_code + append`, or the named reason). A null or negative result from an
+arm that ran on the SDK default is FIX-METHOD under R48 ("Why null?"): fix the
+prompt and rerun before reading it as KEEP or CUT-CANDIDATE.
+
 ## No-gstack arm
 
 Garry, 2026-10-31: "all of our head to head tests have to test no gstack from
@@ -1843,6 +1913,36 @@ it as the diagnostic. Candidate folders and git identity name a project, never
 the arm. The model panel (`test/skill-e2e-model-panel.test.ts`) runs a
 `<model>#no-gstack` arm beside each model (GST-468, GST-469).
 
+## Frontier models under test
+
+Garry, 2026-11-01 (R54): "I think you are spending too much time with Haiku and
+Sonnet, whereas GBrain and GStack are really designed to be used with frontier
+models like Opus and 6.1 Sol... just transfer from Haiku to Sonnet, who gives a
+shit about that, man?"
+
+**Models under test are frontier models.** Today that's `claude-opus-5-5` and
+`gpt-6.1-sol`, plus at most one low-cost primary the preregistration names. Haiku
+and Sonnet are not models under test. They aren't stand-ins for a frontier model,
+and an eval has no tier-transfer arms (Haiku against Sonnet, say) unless its
+preregistration says it is asking about that tier.
+
+**Verdicts need a frontier result.** A result measured only on Haiku or Sonnet
+can't carry a KEEP, FIX-METHOD or CUT-CANDIDATE verdict under R48. Rerun it on a
+frontier model first.
+
+**What the rule leaves alone.** Judges, cross-family review voices and a
+harness's shipped housekeeping model aren't models under test, so they keep the
+model they already use.
+
+**The check.** `gstack-eval-plan dry-run` prints `CHECK: frontier` whenever an arm
+is Haiku or Sonnet. When every arm is one, the dry run fails unless
+`--tier-question "<what the preregistration asks about that tier>"` is passed.
+When only some arms are, it warns. Either way it prints the whole plan's price
+on `claude-opus-5-5` and `gpt-6.1-sol`. The dry run doesn't enforce that number:
+compare it with the brief cap before any arm is swapped. `pilot start` refuses a
+pilot whose every arm is Haiku or Sonnet unless its dry run recorded a tier
+question.
+
 ## Running evals as an agent: detach
 
 Moved verbatim from CLAUDE.md (#2096 size limit). The short rule stays in

docs/captain.md

@@ -434,7 +434,9 @@ A scheduled automation starts one lane a day with the
 runs' flake-records artifacts and the free suites' retried cases, writes one ledger line
 per event, groups by case, and fixes the worst repeat first with a PR that shows the
 failure red before green. It never reruns a gate and never touches a threshold, trial
-count or case kind. On Capy, create it with the automation tool, scheduled in the owner's
+count or case kind. Where a repository runs `gstack-flaky`, the lane also reads
+`gstack-flaky quarantine report --ledger <downloaded ledgers>` and fixes expiring quarantine
+entries before anything else ([flaky.md](flaky.md#the-daily-flaky-pass)). On Capy, create it with the automation tool, scheduled in the owner's
 timezone, running as a thread in the crew so its report lands in `reports/`.
 
 The lane's side of the same rule: in CI the gate reruns a case that failed on its own

docs/evals/census-red.md

@@ -93,7 +93,7 @@ What incomplete data means:
 | headroom `censored` | a sample timed out, so its real duration is at least the budget |
 | cause `unrecorded` | the trial was written before `failure_cause` existed |
 | cause `unknown` | the trial failed with no evidence for any other cause |
-| `cost unknown` | the harness (PTY, Codex) records no billing, or the trial sits in a multi-case shard whose records are named by display label and cannot be attributed to it; only the known sum is shown |
+| `cost unknown` | a session's cost was not captured (a PTY session without a priced transcript, an unbilled Codex session, a `claude -p` session without a result, an aborted Agent SDK session with no priced streamed usage), or no record or session can be attributed to the trial (a multi-case shard's display-label record without `cost_case_id`); only the known sum is shown |
 | `history unavailable (...)` | the GitHub history could not be read; `--gate` fails closed |
 
 ## 4. Replay and repair (free)

docs/flaky.md

@@ -6,7 +6,9 @@ rerun is a **flaky pass**: the job stays green and the flake is recorded on a le
 the daily flaky pass can fix it at the source. A test that fails twice is red.
 This is CREW rule 16 ("run the failed test again and note that it is flaky")
 for any runner that writes JUnit: Playwright, `bun test`, jest, vitest and
-pytest. gstack's own free suite keeps its file-level retry
+pytest. A test that keeps flaking while it is fixed can be
+[quarantined](#quarantine): it still runs, with up to 3 attempts, and never counts
+as a pass. gstack's own free suite keeps its file-level retry
 (`GSTACK_FREE_RETRY_FLAKY`, [Flake ledger](TESTING_INTERNALS.md#flake-ledger)).
 
 ## Get started in GitHub Actions
@@ -74,6 +76,104 @@ It cannot see a coverage threshold configured without console output (for
 example `coverageReporter = ["lcov"]` in `bunfig.toml`), which is why this
 scope is opt-in.
 
+## Quarantine
+
+A test that keeps flaking while someone fixes it can be **quarantined** in
+`.gstack/quarantine.json` at the repository root. Quarantine is a larger
+attempt budget, never a skip: the test still runs on every run, is listed as
+quarantined in the output, and never counts as a pass.
+
+**How a quarantined test is judged.** The verdict rule from
+[Rerun scope](#rerun-scope) does not change: an attempt counts only when the
+runner itself reports a full green run. When every test an attempt failed is a
+live quarantined test, and nothing else is wrong with that attempt (no kill, no
+failure outside the report, no first-run test missing), the whole command may
+run again, up to 3 attempts in total. The run is green only when one of those
+attempts is fully green. A quarantined test that fails all 3 is a **hard
+break**, and the run is red. A test that is not quarantined keeps its one
+rerun. A red attempt never turns green because its only reported failure is a
+quarantined test, because the runner's exit status can come from things the
+report does not show (a coverage threshold, a fail-on-flaky policy, a global
+setup). With `--rerun-scope test` the quarantined test gets the same 3
+attempts on the targeted path, with the same fallback to the whole command.
+When the runner retries on its own (Playwright `retries`), its attempts are
+the budget and nothing is rerun again.
+
+**Missing results.** A quarantined id missing from a run that included its
+file (and, for Playwright, its project) is red. A run that did not include the
+file, such as a shard or a diff selection, is not.
+
+**Read from the base branch.** `run --base <ref>` reads the registry from
+that commit instead of the working tree. Pull-request CI passes its target
+branch, so a change cannot quarantine the test it breaks. `check --base`
+compares against the merge base, so check out full history for it:
+
+```yaml
+- uses: actions/checkout@v4
+  with: { fetch-depth: 0 }
+- name: Quarantine registry
+  run: '"$RUNNER_TEMP/gstack/bin/gstack-flaky" quarantine check --base "origin/${{ github.base_ref }}"'
+- name: E2E
+  run: |
+    "$RUNNER_TEMP/gstack/bin/gstack-flaky" run --base "origin/${{ github.base_ref }}" --junit results/junit.xml -- npx playwright test --reporter=line,junit
+```
+
+### Commands
+
+| Command | What it does |
+|---|---|
+| `gstack-flaky quarantine add <id> --owner <who> --ledger <url> --class <class> --reason <text> --evidence <url>… --junit <file> [--days <n>]` | Adds an entry that expires in `--days` (default 7, at most 14). `--junit` counts the suite for the cap and confirms the id exists; `gstack-flaky ids` prints the ids. |
+| `gstack-flaky quarantine renew <id> --owner <who> --evidence <url> [--days <n>]` | One renewal of up to 7 days from the later of now and the current expiry, with evidence not already on the entry and the owner named again. A second renewal is refused. |
+| `gstack-flaky quarantine release <id> --reason <text> --pr <url>` | Moves the entry to `released` with the date, the reason and the PR. |
+| `gstack-flaky quarantine check [--base <ref>] [--junit <file>]` | Validates every entry and the cap; with `--base`, also what this change did since the merge base. Exits 1 with one `QUARANTINE_REFUSED:` line per problem. |
+| `gstack-flaky quarantine report [--ledger <file\|dir>…] [--base <ref>]` | Lists expired entries, then live entries soonest-expiring first with days left. With downloaded ledgers it adds recent outcomes and marks entries with 10 consecutive clean passes `READY_TO_RELEASE`; without `--ledger` it says the exit rule was not checked. |
+
+### Rules
+
+- **Every entry is owned and expires.** An entry holds `owner`, `ledger`
+  (the ledger line or issue tracking the fix), `failureClass`, `reason`,
+  `evidence`, `enteredAt`, `expiresAt`, `renewals` and `suiteTests` (the
+  suite size `add` counted). It lasts 7 days by default and at most 14 from
+  entry, plus one renewal of at most 7 days.
+- **Expired means not quarantined.** An expired entry stops applying without
+  an edit: `run` treats the test as an ordinary test and prints
+  `FLAKY_QUARANTINE_EXPIRED` with the renew and release commands. An entry
+  that breaks a rule (for example one that lasts 30 days) is ignored the same
+  way, with `FLAKY_QUARANTINE_IGNORED`.
+- **Cap.** Live entries are capped at 10% of the suite's tests, at least 1,
+  and 5 for a suite under 50 tests. `add` refuses past the cap; `check`
+  enforces it against `--junit` when given, else against the smallest
+  `suiteTests` on a live entry.
+- **Classes.** `detector`, `harness`, `timing` and `model-latency`.
+  `product` is refused (a product defect is fixed, not quarantined), and so is
+  `infra` (an infrastructure red is rerun or fixed as infrastructure).
+- **`check --base` refuses** an entry added in the same change that edits its
+  test's file, an entry that disappears without a release record, a release
+  record that was removed, and a changed `enteredAt` or rewritten renewal
+  history.
+- **Exit rule.** 10 consecutive clean passes (`quarantined-pass` with no
+  failed attempt) on the ledgers since the entry was added. Release the entry
+  with the fix PR.
+- **Separate from `CASE_QUARANTINE`.** gstack's own paid-eval quarantine
+  (`CASE_QUARANTINE` and `EVAL_POLICY`,
+  [TESTING_INTERNALS.md](TESTING_INTERNALS.md)) is a different registry with
+  its own rules for eval cases. The two share no schema, and `gstack-flaky`
+  neither reads nor writes it.
+
+### The daily flaky pass
+
+Download the ledger artifacts of the last week's runs and read them with
+`report`; fix expired and `EXPIRING` entries first (fix the test, or renew
+once with new evidence), release `READY_TO_RELEASE` entries, then fix the
+worst repeat on the ledger:
+
+```bash
+for id in $(gh run list --workflow e2e.yml --limit 50 --json databaseId -q '.[].databaseId'); do
+  gh run download "$id" --pattern 'flaky-junit-ledger-*' --dir "ledgers/$id" || true
+done
+gstack-flaky quarantine report --ledger ledgers
+```
+
 ## Rules
 
 - **One rerun, small failures only.** A failed test gets one rerun when the
@@ -101,8 +201,9 @@ scope is opt-in.
 - **Ids.** A test is `<file>::<project>::<describe > title>`, with the project
   empty outside Playwright, so a test in two browser projects is two tests.
   `gstack-flaky ids --junit <file>` prints them.
-- **Ledger.** One JSON line per flaky pass (`kind: flaky-pass`) and per test
-  that failed its rerun (`rerun-fail`), with the tree, commit, branch, run URL,
+- **Ledger.** One JSON line per flaky pass (`kind: flaky-pass`), per test
+  that failed its rerun (`rerun-fail`) and per quarantined test on every run
+  (`quarantined-pass`, `quarantined-fail`, `hard-break`), with the tree, commit, branch, run URL,
   attempts, the rerun's scope (`test` or `command`) and the first lines of
   the error. Ids, paths and failure text go through gstack's redaction engine
   first; when the scan cannot run, they are withheld. The path is `--ledger`,
@@ -113,15 +214,17 @@ scope is opt-in.
 
 ## Output and exit codes
 
-One `FLAKY: <flaky-pass|fail|rerun-fail> <id> attempts=<…> (<reason>)` line per
-test that failed at least once, `FLAKY_RERUN: scope=command (<why>)` when the
-rerun repeated the whole command, then
-`FLAKY_RESULT: tests=<n> failed=<n> flaky=<n> reruns=<n> command_exit=<n> verdict=pass|fail ledger=<file|none>`.
+One `FLAKY: <flaky-pass|fail|rerun-fail|quarantined-pass|quarantined-fail|hard-break> <id> attempts=<…> (<reason>)`
+line per test that failed at least once and per quarantined test,
+`FLAKY_RERUN: scope=command (<why>)` when the rerun repeated the whole command, then
+`FLAKY_RESULT: tests=<n> failed=<n> flaky=<n> quarantined=<n> reruns=<n> command_exit=<n> verdict=pass|fail ledger=<file|none>`.
+`failed` counts every outcome but `flaky-pass` and `quarantined-pass`; `quarantined`
+counts the quarantined tests' outcomes, which are never counted as passes.
 
 | Exit | Meaning |
 |---|---|
 | 0 | Every test passed, flaky passes included |
-| 1 | A test failed (after its rerun, or with no rerun), including `FLAKY_UNREPORTED_FAILURES` |
+| 1 | A test failed (after its rerun, or with no rerun), including `FLAKY_UNREPORTED_FAILURES`, a quarantined test's hard break or missing result, and a refused `quarantine` command |
 | 2 | Usage error, including a `--junit` path that is not a `.xml` file, or an existing file there that is not a readable JUnit report |
 | 3 | No usable report, a crash, or a killed runner (a code below on stderr) |
 
@@ -135,3 +238,9 @@ rerun repeated the whole command, then
 | `FLAKY_COMMAND_NOT_STARTED` | 3 | The command could not be started | Check the argv after `--` |
 | `FLAKY_UNREPORTED_FAILURES` | 1 | The runner counted failures or errors its JUnit omits | Fix the load error or the error outside any test: a real red |
 | `FLAKY_LEDGER_WRITE_FAILED` | — | The ledger could not be appended (warning) | Check the path; the verdict stands |
+| `FLAKY_QUARANTINE_INVALID` | 3 | `.gstack/quarantine.json` is not valid JSON or not a schema 1 registry | Fix the file; `quarantine check` names the problem |
+| `FLAKY_QUARANTINE_BASE` | 3 | `--base` does not resolve to a commit (exit 2 for a value starting with `-`), or `check` finds no merge base | Fetch the base branch; `check --base` needs `fetch-depth: 0` |
+| `FLAKY_QUARANTINE_REFUSED` | 1 | `add`, `renew` or `release` broke a rule | Read the reason; [Quarantine](#quarantine) lists the rules |
+| `FLAKY_QUARANTINE_USAGE` | 2 | An unknown `quarantine` command or option, or a missing value | `gstack-flaky quarantine --help` |
+| `FLAKY_QUARANTINE_EXPIRED` | — | Notice: an expired entry no longer applies | Fix the test, renew once, or release; the line prints both commands |
+| `FLAKY_QUARANTINE_IGNORED` | — | Notice: an entry breaks a rule and does not apply | Fix the entry; `quarantine check` names the rule |

docs/skills.md

@@ -961,7 +961,10 @@ arms, decoys, stop rules, cap) and `--check` refuses to continue while a field i
 blank (`PREREG_INCOMPLETE`). `arms` resolves the `eval-arms` model role against
 `lib/pricing.ts` and stops on an unpriced model (`EVAL_PRICE_MISSING`); `dry-run`
 runs every row through the stub model for $0 and flags malformed rows
-(`EVAL_DRY_RUN_FAILED`); `pilot start|settle|release|report` runs three priced
+(`EVAL_DRY_RUN_FAILED`), and refuses a plan whose every arm is Haiku or Sonnet
+unless `--tier-question` names what it asks about that tier, re-costing it on
+frontier models ([frontier models under test](TESTING_INTERNALS.md#frontier-models-under-test));
+`pilot start|settle|release|report` runs three priced
 items, records actual spend, extrapolates each priced arm to the full set against
 the cap (`EVAL_BUDGET_EXHAUSTED`) and prints the spend table the approval question
 shows. An arm on a subscription with no extra charge is declared with

docs/standalone-binaries.md

@@ -19,7 +19,7 @@ Beyond the slash-command skills, gstack ships standalone CLIs for workflows that
 | `gstack-review-log` | **Review-pass receipts** — `--start <skill>` captures the working-tree fingerprint before a diff review; `'<JSON>' --finish <token>` consumes that single-use, repository/branch/skill-scoped receipt. Binding requires matching start/end content and reviewer-reported `completed:true` and `converged:true`; it is not independent proof that a model read the source. `--check-shared-libs <token>` reads a current finding as JSON on stdin and checks prior Skip decisions against the actual branch, capture and eligible raw source blobs. It returns `reusable:false` when proof is missing or unsafe, including older records without logger-versioned coverage. Final review logging computes shared-code fingerprints and coverage rather than trusting supplied proof. |
 | `gstack-review-read` | **Review freshness** — emits review records with computed `review_freshness.status` and `reason`: CURRENT, STALE, or UNVERIFIED for diff reviews. `/ship` and `/land-and-deploy` use the same grade; a matching commit alone never certifies a diff review. [Dashboard rules](skills.md#review-readiness-dashboard). |
 | `gstack-evidence` | **Verification-evidence ledger** — `run --label <lane> -- <cmd>` transparently wraps any test command (the child's exit code always passes through) and records what ran against which working-tree fingerprint (tracked and untracked files that are not gitignored, minus an untracked `.bun/` folder at the repo root, which Bun can create during a run); a run whose files changed while it ran is recorded without a fingerprint and with up to five of the changed paths, which `check` names in its STALE reason. `check` grades each label FRESH/STALE/MISSING with `--expect-cmd`, `--max-age`, and `--allow-paths` binding. /ship and /land-and-deploy cite fresh evidence instead of re-running suites. Per-run logs are 0600, capped at 2MB, pruned after 30 days; the ledger and logs stay machine-local by design. |
-| `gstack-flaky` | **Flaky-test rerun for any JUnit runner** — `run --junit <file.xml> -- <command>` runs your test command (Playwright, `bun test`, jest, vitest, pytest) and, when at most `--max-reruns` (3) tests failed and the runner made no retries of its own, reruns the whole command once (`--rerun-scope test` opts in to rerunning only the failed tests). A test that fails and then passes on that rerun is a flaky pass: the run stays green and one redacted line per flaky pass or failed rerun goes on a ledger (`GSTACK_FLAKE_LEDGER_JUNIT`, default `<state root>/flake-ledger-junit.jsonl`). A missing or malformed report, a crash or a killed runner never passes. `ids --junit <file>` prints a report's test ids. Exit 0 pass, 1 a test failed, 2 usage, 3 no usable report. [Guide](flaky.md). |
+| `gstack-flaky` | **Flaky-test rerun for any JUnit runner** — `run --junit <file.xml> -- <command>` runs your test command (Playwright, `bun test`, jest, vitest, pytest) and, when at most `--max-reruns` (3) tests failed and the runner made no retries of its own, reruns the whole command once (`--rerun-scope test` opts in to rerunning only the failed tests). A test that fails and then passes on that rerun is a flaky pass: the run stays green and one redacted line per flaky pass or failed rerun goes on a ledger (`GSTACK_FLAKE_LEDGER_JUNIT`, default `<state root>/flake-ledger-junit.jsonl`). A missing or malformed report, a crash or a killed runner never passes. `quarantine add|renew|release|check|report` keeps `.gstack/quarantine.json`: a quarantined test still runs, gets up to 3 attempts of the whole command, passes only on a full green attempt and is never counted as a pass (`run --base <ref>` reads the registry from the base branch). `ids --junit <file>` prints a report's test ids. Exit 0 pass, 1 a test failed, 2 usage, 3 no usable report. [Guide](flaky.md). |
 | `gstack-pr-wait` | **PR event watcher** — `gstack-pr-wait <pr> [--repo owner/name] [--until green\|ready\|any-change] [--interval 20] [--timeout 60] [--exclude CHECK,...]` polls one pull request through `gh` and prints one stdout line per real event (a check failed with its link, all checks finished, a comment or review from someone other than the author, a review bot finished on the head, the base moved and its new commits touch this PR's files, BEHIND or DIRTY, merged or closed), with progress on stderr, so a monitor wakes the agent once per event instead of on a timer. `green` gates on every check on the head (an advisory bot's review is never waited for, its checks gate like any other), as `gstack-ci-gate` does; a check the base branch requires (rulesets and classic branch protection) is pending until it appears, never green by omission; two identical polls confirm a check set is complete; `--exclude` leaves a named non-required check out of the verdict as `gstack-ci-gate --exclude` does (a required name is refused and gates). Exit 0 when the condition is met, 1 when it needs action or `--timeout` passes (`PR_WAIT_BLOCKED`), 2 on bad input, a second watcher for the same PR or a local error (`PR_WAIT_REFUSED`). State under `<state root>/pr-wait/`, baseline discarded after 6 hours. Review bots are configuration: `gstack-config set pr_wait_gating_bots greptile,macroscope`, `pr_wait_advisory_bots coderabbit` (advisory bots are reported, never waited on). `/land-and-deploy` Step 3 is one call of it. Design ported from tstack's pr-wait (MIT, Theo Browne; NOTICE.md). [Exit codes](troubleshooting.md#gstack-pr-wait). |
 | `gstack-issue-guard` | **Tracker-text trust envelope** — fetches GitHub issue/PR text (`issue <n>`, `pr-body`, `pr-comments`, or `--stdin`) and wraps it in a labeled envelope so agents treat it as data: injection-shaped lines get labeled even through fullwidth and invisible-character evasion, and forged envelope banners are defused. Every tracker-text ingress in gstack routes through it, enforced by a CI scanner. |
 | `gstack-ios-qa-daemon` | **iOS QA daemon** — Mac-side broker between an agent and a connected iPhone or iPad over USB CoreDevice. Loopback by default; `--tailnet` opens a Tailscale-facing listener with identity-gated capability tiers. Single-instance via flock on `~/.gstack/ios-qa-daemon.pid`. See [docs/howto-ios-testing-with-gstack.md](howto-ios-testing-with-gstack.md). |

docs/support-tiers.md

@@ -10,7 +10,7 @@ gstack has two support tiers. Full support gets the whole test suite, paid evals
 
 ## Minimal support
 
-- **Windows** (Git Bash or MSYS): with a default Git for Windows install, `./setup` installs, and the core skills start without errors. The core skills are `/ship`, `/review`, `/qa-only`, `/investigate` and `/browse` on gstack's own browser. Once the `windows-smoke` CI job lands, it checks this on every pull request (see Windows CI below).
+- **Windows** (Git Bash or MSYS): with a default Git for Windows install, `./setup` installs, and the core skills start without errors. The core skills are `/ship`, `/review`, `/qa-only`, `/investigate` and `/browse` on gstack's own browser. One required CI job, `windows-smoke`, checks this on every pull request (see Windows CI below).
 - **Gemini models:** the core workflows run to the end without errors when Gemini is the host agent or a reviewer. We check this by hand or with one cheap smoke run, not with an eval cell.
 
 That is the whole promise. Other skills, optional parts such as `/cso`'s native helper, non-default shell settings and speed are not covered.
@@ -23,8 +23,9 @@ That is the whole promise. Other skills, optional parts such as `/cso`'s native
 
 ## Windows CI
 
-- **Today:** the six-shard Windows free suite (the `Windows Free Tests` workflow, `windows-free-tests`) runs on every pull request and every push to main, and it is a required check.
-- **Once `windows-smoke` lands:** `windows-smoke` becomes the required Windows check on pull requests. The six-shard suite then runs nightly on main, or when someone starts it by hand, and never blocks a pull request. A red nightly run gets a fix only when it shows an install, startup or core-workflow break.
+- `windows-smoke` is the required Windows check on pull requests.
+- The six-shard Windows free suite (the `Windows Free Tests` workflow, `windows-free-tests`) runs nightly on main, or when someone starts it by hand, and never blocks a pull request.
+- A red nightly run gets a fix only when it shows an install, startup or core-workflow break.
 
 ## What doesn't change
 

docs/troubleshooting.md

@@ -2592,12 +2592,24 @@ case with `bun run eval:pass-rates --headroom --case <id>`.
 <a id="cost-unknown"></a>
 ### `cost unknown (no billing captured for any of <N> trial(s))` / `cost $<x> known + <N> of <M> trial(s) cost unknown`
 
-**Meaning.** PTY and Codex sessions record no billing, so their cost is
-unknown. So is a trial in a multi-case shard whose result records are named by
-a display label rather than the case id, because no record can be attributed
-to it (a one-case shard's records are its case's). Only the known sum is
-shown. A train verdict's `ATTEMPT:` line reports the same as `cost=$<x> known
-+ <n> of <m> case(s) cost unknown`, a bare `cost=$<x>` only when every case
+**Meaning.** Some trials' cost was not captured, so only the known sum is
+shown. A trial is cost-unknown when it ran a PTY session whose transcript
+could not be priced (a non-hermetic or resumed launch, the native
+AskUserQuestion capture, two new transcripts for one cwd, no transcript or
+usage, or a model with no price row), a
+Codex session that completed no turn or whose model has no price row, a
+`claude -p` session that ended without a result, or an aborted Agent SDK
+session whose streamed usage could not be priced (no usage, or a model with no
+price row). A PTY or aborted Agent SDK session that does price counts its
+`streamed_cost_usd` instead. A trial is also unknown when no record and no
+session can be attributed to it, for example a multi-case shard's
+display-label record without `cost_case_id` (a one-case shard's records are
+its case's); a model-free case (`timeline-event-flow`) is a known $0. A panel
+trial (`behavior`, `judge`) is unknown whenever any of its sessions billed
+nothing, even one with a priced `streamed_cost_usd`. The rules are in
+[Failure causes and details](TESTING_INTERNALS.md#failure-causes). A train
+verdict's `ATTEMPT:` line reports the same as `cost=$<x> known + <n> of <m>
+case(s) cost unknown`, a bare `cost=$<x>` only when every case
 in the train record is priced, and no cost at all when none is.
 
 **Fix.** None needed; this is informational.