Merge main into gsta147-squash-parent

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-10-29 06:04:50 +00:00
commit 14c4c5e263
33 files changed
+596 -117

No files matched your search

+45
View File
@@ -8,6 +8,51 @@
### Tests
- `test/ship-receipt-carry-squash.test.ts`: a child carries after its parent squash-merges and main is merged in, names main as its base, and carries again across a later main merge. A test-only child carries. A child edited after the merge, a parent that never landed, a parent `package.json` change that never reached main, and a merge that keeps the child's tree over the landed parent refuse. An older receipt falls back to `origin/<parent>` and refuses once that branch is deleted.
## [1.91.310.0] - 2026-10-29
**The cookie workflow judge now asks for structured output, so a quote the model forgets to escape can no longer fail the gate.** Train 42 went red because both of its `setup-browser-cookies/SKILL.md workflow` samples came back as free text with a bare quote inside `reasoning` ("one --domain per run"), and neither parsed. The GSTA-91 comparison had measured that failure: 2 of 144 free-text samples failed to parse, against 0 of 72 with the schema.
### Changed
- **Cookie workflow judge transport:** `setup-browser-cookies/SKILL.md workflow` now sends `WORKFLOW_JUDGE_RESPONSE_SCHEMA`, released on GStack Minder's ruling: "GStack Minder, 2026-10-29: release the GSTA-91 hold; a07 is matched by old-b; free-text parse errors cost train 42". Its model, thresholds (4/3/4), sample count and case kind are unchanged. The ruling is recorded in `test/fixtures/judge-calibration/cookie-workflow/release.json`, and the held comparison stays recorded as held.
### Tests
- `test/cookie-workflow-judge-input.test.ts` runs the registered cookie callback through the real `callJudge` and pins its request to the schema. It replays train 42's two replies (msg_011CgVnRtVj5vyLvb56m6KJH, msg_011CgVne5GStZ6f9pRoCvgWE): their free-text bytes do not parse, and the same judgment returned as schema output scores and passes. `test/judge-calibration.test.ts` reads the release record. Both fail without the transport change.
### Gate integrity
- Gate edit: the cookie judge's response transport moves from free text to the JSON schema, on the ruling above. No threshold, sample, trial, deadline or case-kind change. The Haiku free-text failure pin in `test/llm-judge-frontier.test.ts` is untouched.
## [1.91.309.0] - 2026-10-29
**The judged review and Eng prose each give one rule where the paid judges kept finding two, and a judge that runs out of time now says what it was waiting on.** Every sample of two red panels on #3527's screens (review 40617200857, plan-eng-review 40634787316), attempt and rerun, named the same contradictions; a third red (ship, train 35 gate 40660626349) hit the 120 s judge deadline with nothing logged but "No completed model response".
### Fixed
- **`/review` Step 5.8**: an unresolved non-gate defect (an unanswered question) is recorded as `open`, so every finding has a record action. A repeat pass's reruns of superseded probes are revalidation, so they run after smoke expiry, in the same probe directory; earlier provisional QA sections link checkpoints and only the final pass runs the once-only `materialize`.
- **`/plan-eng-review`**: the shared Voice now loads once. Eng no longer carries a near-copy whose only difference was its em-dash clause; the Scope gate's format precedence says required formats keep their em dashes.
- **`/benchmark` Phase 6 example**: the hero image recommendation no longer suggests lazy-loading (that delays the largest paint) and names one remedy, as the rule under it asks.
### Changed
- **Judge cancellations leave evidence.** When a judge request is cancelled (the workflow judge deadline), `callJudge` prints one `llm-judge-aborted` line: elapsed time, wrapper attempts and 429 waits, HTTP requests and statuses (so the SDK's own retries show), connection, first and last stream event times, the open block type, thinking and text character counts, message and request ids, and the reason (at most 200 characters). Counts only, never content; an observer that throws cannot change a judge result. Deadlines, effort, samples and thresholds are unchanged.
### Tests
- `test/review-workflow-clarity.test.ts` and `test/eng-scope-entry-ap.test.ts` pin the repaired rules; both fail with the prose change stashed.
- `test/llm-judge-stream.test.ts` and `test/llm-judge-abort.test.ts` replay a stream cancelled mid-thinking, a stream with no events, one cancelled after headers, an SDK-retried 429, a cancelled 429 wait, an event the observer cannot read and an overlong reason, with no-abort controls; each fails with the helper change stashed. `test/workflow-judge-cache.test.ts`'s SDK stream stand-ins gain the real stream's `on()`.
- Focused paid judge runs: review 4/4/4 at 1bdc4ce and again at 3463250, plan-eng-review 3/5/4 at 1bdc4ce ($0.18).
### Gate integrity
- No threshold, sample, trial, deadline or case-kind change. `scripts/harness-version.json` records the `llm-judge.ts` change as non-behavioral (no series reset). Generated SKILL.md size pins for review and plan-eng-review were lowered to their new sizes on every host.
## [1.91.308.0] - 2026-10-29
**`gstack-restamp` no longer cuts a four-part package.json version to three parts in a repo whose convention is four.** On garrytan/gbrain#7878, `/ship` restamped gbrain's `package.json` to `"0.60.483"` beside `VERSION` `0.60.483.0`; gbrain requires the two to match and its CI version gate fails when they drift, so the stamp had to be reverted by hand. The version stage called `gstack-version-bump write`, which always stores the npm-valid three-part form in the manifest (decision 11, which is why gstack's own `package.json` reads `1.91.298`).
### Fixed
- **`lib/restamp.ts`**: the version stage keeps the full version in the manifest when the policy lists it in `stamp_paths` or `stamp_files` and the base's copy holds four parts, and the stage detail says so (`wrote 0.60.483.0 (package.json keeps four parts: …)`). The form is read at the base, like the policy, so a branch an earlier restamp already cut to three parts goes back to four. Every other repo, gstack included, still gets the npm-valid form, and a four-part manifest the policy does not list still migrates to three parts as decision 11 says.
- **`bin/gstack-version-bump write`**: new `--manifest-form npm|full` (default `npm`); `full` writes the version as given to the manifest and its npm lockfiles. It has no effect when the version path is itself a JSON manifest.
### Tests
- `test/restamp-four-part-manifest.test.ts`: a gbrain-shaped fixture (four-part `VERSION` and `package.json`, the manifest in `stamp_files`, bun, no npm lockfile) restamped from an unstamped branch and from a branch already carrying its stamp (both failed before the fix with `"0.60.483"`), from a branch an older restamp cut to three parts, and with the manifest listed through `stamp_paths`. gstack's shape (three-part manifest and lockfile) and an unlisted four-part manifest keep their current results.
- `test/gstack-version-bump.test.ts`: `--manifest-form full` keeps four parts in the manifest and lockfile; an unknown form exits 2 before anything is written.
## [1.91.307.0] - 2026-10-29
**/ship now reviews in proportion to what the diff can break: light, standard or full depth.** A diff with 50 or fewer runtime lines gets one batched specialist plus the native adversarial pass, a larger runtime diff gets the class's specialists scoped to their hunks, and gate code or any classifier error gets the full review as before. The receipt records the depth and the queue refuses a receipt below what the head needs. Before this, a 12-line fix paid for the same reviewer fan-out as a change to the merge gate. The backtest that would have let light depth drop the adversarial pass was inconclusive (9 light-eligible PRs, 5 with attributable findings, below the 15 the plan required), so light keeps it.
+3 -3
View File
@@ -788,15 +788,15 @@ Each item was deferred with a reason during the `/claude-api prompt-audit` clean
format errors, so it stays on today's request), the default workflow judge
(landed, but one provider refusal on corpus item f03 under the new prompt,
matched by one under the old prompt) and the cookie workflow judge (GSTA-91:
held on one new false pass on a07, zero new-request format errors, so it stays
on today's request; today's request itself false-passes a07 in `old-b` and on
held on one new false pass on a07, zero new-request format errors; the hold was
released on GStack Minder's 2026-10-29 ruling and it now sends the schema; today's request itself false-passes a07 in `old-b` and on
both valid `old-a` samples, and a08 in `old-a`, so the prose step targets those). Reuse `test/fixtures/judge-calibration`
and `scripts/judge-calibration.ts`. Priority P3. Effort M.
- **Uncalibrated workflow-judge configurations** — build a corpus and calibrate
schema transport for the frontier configuration (review/SKILL.md workflow); it
stays on today's request. The cookie configuration
(setup-browser-cookies/SKILL.md workflow) has a corpus and a held comparison-1
(GSTA-91), so it also stays on today's request. After any transport change,
(GSTA-91); it moved to the schema on GStack Minder's 2026-10-29 release ruling. After any transport change,
re-run the review and cookie workflow judges. Priority P3. Effort M.
- **Claude model auto-detection for the skill overlay** — when no
`claude_overlay_model` is set, detect the Claude Code model (`ANTHROPIC_MODEL`,
+1 -1
View File
@@ -1 +1 @@
1.91.307.0
1.91.310.0
+1 -1
View File
@@ -1,4 +1,4 @@
# gstack digest v1.91.307.0 — regenerate/re-copy after upgrading gstack
# gstack digest v1.91.310.0 — regenerate/re-copy after upgrading gstack
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
for agent hosts without a full skill install. The full skills add workflows,
+1 -1
View File
@@ -412,7 +412,7 @@ TOP 10 SLOWEST RESOURCES
RECOMMENDATIONS:
- vendor.chunk.js: code-split — 320KB on initial load
- analytics.js: load async/defer — blocks rendering for 250ms
- hero-image.webp: add width/height (CLS), lazy-load
- hero-image.webp: add width/height (CLS)
```
Third-party = any host but the page's origin. Each recommendation names a resource here or in a Phase 5 regression, the metric it moves and one remedy (split, defer, compress, lazy-load, cache).
+1 -1
View File
@@ -201,7 +201,7 @@ TOP 10 SLOWEST RESOURCES
RECOMMENDATIONS:
- vendor.chunk.js: code-split — 320KB on initial load
- analytics.js: load async/defer — blocks rendering for 250ms
- hero-image.webp: add width/height (CLS), lazy-load
- hero-image.webp: add width/height (CLS)
```
Third-party = any host but the page's origin. Each recommendation names a resource here or in a Phase 5 regression, the metric it moves and one remedy (split, defer, compress, lazy-load, cache).
+11 -2
View File
@@ -25,8 +25,14 @@
// the caller must handle), exit 2 on bad args / unresolvable base /
// broken version source.
//
// write --version <X.Y.Z.W> [--version-path <p>] [--regen-digest] [--changelog <p>]
// write --version <X.Y.Z.W> [--version-path <p>] [--regen-digest] [--changelog <p>] [--manifest-form npm|full]
// Validates the 4-digit pattern, writes VERSION + package.json.version.
// --manifest-form full: the manifest and its npm lockfiles carry the
// version as written instead of the npm-valid translation (default
// npm). For repos whose declared convention is a four-part manifest
// (gbrain); gstack-restamp passes it only when the policy lists the
// manifest as a stamp file and the base's copy holds four parts. No
// effect when the version path is itself a JSON manifest.
// --changelog <p> (opt-in): re-head the branch's `## [Unreleased]`
// entry in <p> as `## [<version>] - <UTC date>` at the top, with
// lib/changelog-check's reheadChangelog — the same function
@@ -455,6 +461,8 @@ function cmdWrite(args: string[], cwd: string): void {
if (!VERSION_RE.test(version!)) {
fail(`NEW_VERSION (${version}) does not match MAJOR.MINOR.PATCH[.MICRO]. Aborting.`, 2);
}
const manifestForm = argVal(args, "--manifest-form") ?? "npm";
if (manifestForm !== "npm" && manifestForm !== "full") fail("write: --manifest-form needs npm|full", 2);
const src = versionSource(cwd, argVal(args, "--version-path"));
if (src.outcome === "broken") failBroken(src, "write");
if (src.outcome !== "valid") {
@@ -528,7 +536,8 @@ function cmdWrite(args: string[], cwd: string): void {
// 3-digit translation — npm rejects a fourth component, so mirroring the
// raw 4-digit form breaks `npm ci` in any repo npm actually manages.
// VERSION keeps the full 4-digit form; it stays the source of truth.
const manifestV = npmVersion(version!);
// --manifest-form full keeps a repo's declared four-part manifest as is.
const manifestV = manifestForm === "full" ? version! : npmVersion(version!);
if (hasPkg) {
let pkgWritten = false;
try {
+8
View File
@@ -783,6 +783,14 @@ once, plus cleanup. Each still has 120 seconds of model work. Its 17 workflow
judges own their deadline and abort signal, with five seconds for terminal
recording inside a ten-second Bun grace; the other 11 retain their existing
120-second Bun timeout. Late responses cannot create records or cache passes.
When a judge's abort signal cancels its request, `callJudge`
(`test/helpers/llm-judge.ts`) prints one `llm-judge-aborted` JSON line to
stderr: elapsed time, wrapper `attempts` and `retryWaitsMs` (429 waits),
`httpRequests` and `httpStatuses` (so SDK retries show), `connectedMs`,
`firstEventMs` and `lastEventMs`, the open `block` type, `thinkingChars` and
`textChars`, `messageId`, `requestId` and the `reason` (at most 200
characters). It holds counts and ids only, never content, and changes no
request, score or verdict.
The ship documentation file reserves 4,920 seconds for four 600-second cases and
eight 300-second fault cases, run once, plus cleanup; in CI each case runs as its
+1 -1
View File
@@ -1100,7 +1100,7 @@ or the order moved, the result is `QUEUE_STALE` (exit 3) with the exact
Stages, each journaled in `.gstack/tmp/restamp-journal.json`:
0. **restamp_command** (only when the policy sets it): the repo's own restamp, see [restamp_command](#restamp-command).
1. **version**: `gstack-version-bump write --version <X>` in the staging worktree (VERSION, the manifest, npm lockfiles); skipped when `restamp_command` owns the version.
1. **version**: `gstack-version-bump write --version <X>` in the staging worktree (VERSION, the manifest, npm lockfiles); skipped when `restamp_command` owns the version. The manifest gets the npm-valid three-part form, except when the policy lists it in `stamp_paths` or `stamp_files` and its copy at the base holds four parts (gbrain's convention): then it keeps the full version (`--manifest-form full`) and the stage detail says so.
2. **release_tool**: the policy command, under the contract below.
3. **changelog**: the PR's `[Unreleased]` or old-version section moves to the top under `<X>`, above its predecessor's entry. A duplicate heading the PR introduces (`CHANGELOG_DUPLICATE_HEADING`: the headings are counted against the base's CHANGELOG at `origin/<base>` and at the fork point, so a version the base already heads twice passes as long as the PR adds no further copy) and a top entry that is not `<X>` (`CHANGELOG_TOP_MISMATCH`) are errors; the section being re-headed itself may exist only once.
4. **stamp_paths**: exact `(was X)`, `since: X` and `vX` rewrites, plus every `stamp_patterns` template, within `stamp_paths` and `stamp_files`. A quote around X in the first two forms is part of the form and is kept as written: `since: 'X'`, `since: "X"`, `(was 'X')` and `(was "X")` move like the bare ones (the shape of a quoted source row such as gbrain's `{ since: '0.60.468.0', … }`), an opened quote must close right after X (`since: '1.2.3-beta'` and `since: '1.2.3"` are not the form, in the writer and in the tree receipt alike), and `vX` already accepts a quote before the `v`. Which X moves is decided by provenance. The branch's own tentative stamp (its VERSION when it moved past base) and `--was <X>`, when given, are each rewritten on every line of every stamp file. Beyond that, only lines the branch itself added are read, by their position in the file (`git diff --unified=0` from the fork point), so a base line with the same text, or a CRLF checkout whose diff is LF-normalized, is told apart: a stamp-form mention there of a version newer than the version at the branch's fork point is a tentative stamp and moves (this is what finds a quoted row after a lost slot and a merge of the base, since the base's release is newer than the fork point), and when the branch has no stamp of its own because its VERSION equals the base's and its fork point is the base itself (a branch rebased onto the base after it reached X, or one never stamped) a mention of the base version on a line it added is its stale stamp and moves too, while the same version on a line the base already carries is the base's release and stays. The stage detail names both counts per file (`VERSION equals base X, so X moved only on lines this branch added and stayed on the base's own lines (<file>: N moved, M kept)`), so the stamp commit shows which lines moved and why. A line the base added since the fork point is the base's even when that diff shows it (a merge of the base on the branch, committed or still staged, brings every line the base added since the fork into the diff from the fork point, main's own release keys included): it is left out, the detail says `N line(s) the base added since the fork point are the base's, not this branch's, and stay (<file>: N)`, and a base diff git cannot run refuses the stage (`POLICY_SOURCE_UNAVAILABLE`, `cannot diff <since>..<base> for the stamp_paths stage`) rather than reading as an empty set that would move the base's lines again. The rule reads line ownership, not meaning: a branch-added line inside `stamp_paths` that means the base's own release in a stamp form (`Requires vX or later` written on a branch forked at X) moves with it, which the `--dry-run` diff and the per-file counts show before anything is written; write such a reference outside a stamp form, or keep that file out of `stamp_paths`. `--was X` is the opposite tool: it rewrites X everywhere, the base's own lines included, and re-heads the CHANGELOG section `## [X]`. Only the repo's own version width counts, so a 3-digit dependency pin in a 4-digit repo is never a candidate.
+16 -2
View File
@@ -223,6 +223,19 @@ export function versionFiles(root: string, versionRel: string): string[] {
const dir = path.dirname(pkgRel) === '.' ? '' : path.dirname(pkgRel) + '/';
return [versionRel, pkgRel, ...NPM_LOCKFILES.map(l => dir + l)];
}
/**
* The manifest's path when the repo's convention is a four-part manifest: the policy lists it as a
* stamp file and the base's copy holds a four-part version (gbrain). Read at the base, like the
* policy, so a branch an older restamp cut to three parts heals. Otherwise null, and the writer
* stores the npm-valid three-part form (decision 11; gstack's own manifest).
*/
function manifestKeepsFourParts(plan: Plan, root: string): string | null {
const pkgRel = versionFiles(root, plan.versionRel)[1]!;
if (pkgRel === plan.versionRel || !matchesAny([...plan.policy.stamp_paths, ...plan.policy.stamp_files], pkgRel)) return null;
const atBase = git(root, ['show', `${plan.baseSha}:${pkgRel}`]);
const version = atBase.ok ? extractVersion(atBase.out, pkgRel) : '';
return version && versionWidth(version) === 4 ? pkgRel : null;
}
function allowedOutput(plan: Plan, root: string, rel: string): boolean {
const p = plan.policy;
if (versionFiles(root, plan.versionRel).includes(rel)) return true;
@@ -455,9 +468,10 @@ function runStages(o: RestampOptions, plan: Plan, staging: string, since: string
const v = stage('version');
if (command) { v.status = 'skipped'; v.detail = 'restamp_command owns the version'; }
else {
const w = spawnSync(process.execPath, [...bunIsolationArgs(), path.join(BIN_DIR, 'gstack-version-bump'), 'write', '--version', plan.newVersion], { cwd: staging, encoding: 'utf8', timeout: 120_000, env });
const full = manifestKeepsFourParts(plan, staging);
const w = spawnSync(process.execPath, [...bunIsolationArgs(), path.join(BIN_DIR, 'gstack-version-bump'), 'write', '--version', plan.newVersion, ...(full ? ['--manifest-form', 'full'] : [])], { cwd: staging, encoding: 'utf8', timeout: 120_000, env });
if (w.status !== 0) { v.status = 'failed'; v.detail = (w.stderr ?? '').trim().slice(0, 300); throw new Refusal('RESTAMP_VERSION_SOURCE', `gstack-version-bump write failed: ${v.detail}; nothing written`); }
v.status = 'done'; v.detail = `wrote ${plan.newVersion}`;
v.status = 'done'; v.detail = `wrote ${plan.newVersion}` + (full ? ` (${full} keeps four parts: a policy stamp file the base holds in four)` : '');
}
injected('version');
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "gstack",
"version": "1.91.307",
"version": "1.91.310",
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
"license": "MIT",
"type": "module",
+1 -23
View File
@@ -50,7 +50,7 @@ Recommendation: A when a branch diff exists, otherwise B. Reply with A, B, or C.
After target selection, use the preamble's full decision brief, transport and continuous D-numbering. Setup questions approve no engineering remedies.
**Format precedence:** Copy required command, output and question formats exactly. Apply Voice to newly composed prose.
**Format precedence:** Copy required command, output and question formats exactly, em dashes included. Apply Voice to newly composed prose.
**Startup sequence** (after target selection):
1. Run the Preamble command and its startup instructions (Context Recovery and the preamble's own one-time setup prompts; the Prior Learnings cross-project question waits for Step 0). Defer Operational Self-Improvement, Telemetry and Plan Status Footer to finish; format/transport rules apply throughout.
@@ -187,28 +187,6 @@ Before emitting a tool or prose decision brief, verify:
- [ ] Follow Tool resolution: tool call unless Conductor or documented prose fallback; prose includes the mandatory triad + explicit reply selectors, then STOP. Spawned sessions follow their auto-choice rule.
- [ ] Write non-ASCII directly, not \u-escaped. For 5+ options, split/batch into ≤4 without dropping; check dependencies and stop the chain immediately on Hold.
## Voice
GStack voice: Garry-shaped product and engineering judgment.
- Lead with the point. Say what it does, why it matters, and what changes for the builder.
- Be concrete. Name files, functions, line numbers, commands, outputs, evals, and real numbers.
- Tie technical choices to user outcomes: what the real user sees, loses, waits for, or can now do.
- Be direct about quality. Bugs matter. Edge cases matter. Fix the whole thing, not the demo path.
- Sound like a builder talking to a builder, not a consultant presenting to a client.
- Never corporate, academic, PR, or hype. Avoid filler, throat-clearing, generic optimism, and founder cosplay.
- Do not add em dashes in prose you compose during the review. Existing templates, quoted text, command output, and required copied labels may contain them. No AI vocabulary: delve, crucial, robust, comprehensive, nuanced, multifaceted, furthermore, moreover, additionally, pivotal, landscape, tapestry, underscore, foster, showcase, intricate, vibrant, fundamental, significant, load-bearing.
- Reply in the language of the user's latest message unless asked otherwise. Code, commands, paths, identifiers, quoted output and question markers (`D<N>`, option letters, `(recommended)`) stay verbatim.
- The user has context you do not: domain knowledge, timing, relationships, taste. Cross-model agreement is a recommendation, not a decision. The user decides.
Good: "auth.ts:47 returns undefined when the session cookie expires. Users hit a white screen. Fix: add a null check and redirect to /login. Two lines."
Bad: "I've identified a potential issue in the authentication flow that may cause problems under certain conditions."
**Bounded closer.** After completing work, report in at most a few short lines: what changed, what was skipped, what to watch. No feature tours or unrequested design notes. Exempt: decision briefs, completion-status blocks, requested explanations, and a skill's mandated report (/qa-only, /plan-*-review, /retro, /document-generate). The rule limits prose around the deliverable, never the deliverable.
Good closer: "Renamed the flag in 3 files, regenerated docs, tests green. Skipped the CLI alias (unused since v1.2); watch the Windows job."
Bad closer: a tour of every edit, a restatement of the plan, and three paragraphs justifying choices nobody questioned.
## Question Tuning (skip entirely if `QUESTION_TUNING: false`)
Before each decision brief (AskUserQuestion or Conductor/fallback prose), choose `question_id` from `~/.claude/skills/gstack/scripts/question-registry.ts` or `{skill}-{slug}`, then run `~/.claude/skills/gstack/bin/gstack-question-preference --check "<id>"`; for an unregistered id, write the question summary to `.gstack/tmp/qt.txt` (file-write tool) and append `--summary-file .gstack/tmp/qt.txt` (one-way keyword check). `AUTO_DECIDE` means choose the recommended option and say "Auto-decided [summary] → [option] (your preference). Change with /plan-tune." `ASK_NORMALLY` means ask.
+1 -1
View File
@@ -60,7 +60,7 @@ Recommendation: A when a branch diff exists, otherwise B. Reply with A, B, or C.
After target selection, use the preamble's full decision brief, transport and continuous D-numbering. Setup questions approve no engineering remedies.
**Format precedence:** Copy required command, output and question formats exactly. Apply Voice to newly composed prose.
**Format precedence:** Copy required command, output and question formats exactly, em dashes included. Apply Voice to newly composed prose.
**Startup sequence** (after target selection):
1. Run the Preamble command and its startup instructions (Context Recovery and the preamble's own one-time setup prompts; the Prior Learnings cross-project question waits for Step 0). Defer Operational Self-Improvement, Telemetry and Plan Status Footer to finish; format/transport rules apply throughout.
+14 -12
View File
@@ -650,9 +650,9 @@ findings and decides completion.
## Step 5: Fix-First Review
Before edits, confirm every dispatched reader has returned or is confirmed stopped.
For an active or unknown reader/writer, wait or confirm it is stopped; if that
cannot be confirmed, persist incomplete at Step 5.8 and STOP without edits.
Before edits, every dispatched reader/writer must have returned or be confirmed
stopped: wait, and if that cannot be confirmed, persist incomplete at Step 5.8 and
STOP without edits.
Terminal failure does not block fixes from independent evidence. Missing required
output still makes the pass incomplete, even after the reader is stopped.
@@ -806,7 +806,7 @@ After outputting your own findings, if Greptile comments were classified in Step
Before replying to any comment, run greptile-triage.md's **Escalation Detection** to choose Tier 1 (friendly) or Tier 2 (firm) reply templates.
1. **VALID & ACTIONABLE comments:** Use their Step 5a–5d disposition; do not ask a second fix question. Step 5c alone supplies A) Fix / B) Skip for ASK items. After a completed fix, use the **Fix reply template** with diff and explanation; cite the current diff if uncommitted, never invent a commit SHA. A Skip leaves the defect unresolved and grants no new fix permission. If evidence disproves the finding, reclassify it below.
1. **VALID & ACTIONABLE comments:** Use their Step 5a–5d disposition; do not ask a second fix question. After a completed fix, use the **Fix reply template** with diff and explanation; cite the current diff if uncommitted, never invent a commit SHA. A Skip leaves the defect unresolved and grants no new fix permission. If evidence disproves the finding, reclassify it below.
2. **FALSE POSITIVE comments:** These are reply decisions, not code approval. Show file:line (or [top-level]), summary, permalink and evidence, then ask:
- A) Reply explaining why this is incorrect (recommended if clearly wrong)
@@ -835,6 +835,9 @@ Before replying to any comment, run greptile-triage.md's **Escalation Detection*
final: repeat Steps 3–5 with a full start.
2. On a repeat, execute Steps 3–5 in order. At Step 4.7, an edited pass supersedes
every earlier QA capture (exploratory §4): rerun each probe the verdict relies on.
These reruns are revalidation, so they run even after smoke expiry; keep the
PROBE_DIR. Earlier provisional sections link checkpoints; only the final pass
runs its once-only `materialize`.
A zero-edit pass reuses only this invocation's unchanged-input QA evidence,
rerunning any probe whose inputs changed or are uncertain. Reusing never skips a review step.
3. **Verify completed actions.** On the final zero-edit pass, reconcile this
@@ -846,9 +849,8 @@ Before replying to any comment, run greptile-triage.md's **Escalation Detection*
defects and unresolved counts; earlier fixes do not suppress them.
4. **Recheck skipped advice.** Re-read its final-snapshot supporting source and
reconfirm the decision; otherwise report its history without a reusable skip.
The logger computes `snapshot_covered_paths` from eligible paths whose raw bytes
equal the bound snapshot blobs (`[]` if none). Never carry prior-cycle, supplied
or prior-record coverage forward or build this proof yourself. Fixed advice
The logger computes `snapshot_covered_paths` itself; never carry prior-cycle,
supplied or prior-record coverage forward or build this proof yourself. Fixed advice
needs no skip coverage.
### 2. Fill the record
@@ -878,14 +880,14 @@ pass: report it with its reason.
`helper_target`. Recheck source after fixes. The logger uses `sharedLibsFingerprint`,
never supplied/model hashes.
Actions: `auto-fixed` (Step 5b), `fixed` (approved **and completed** in Step 5d),
`skipped` (explicit Skip in Step 5c). Advice is never `auto-fixed`; pending
`skipped` (explicit Skip in Step 5c),
`open` (any other defect, e.g. unanswered). Advice is never `auto-fixed`; pending
advice stays in the response, not the record. Exclude prior Step 5.0
suppressions; include this invocation's revalidated decisions.
- Gate findings additionally carry `gate` (the RH tag, e.g. `"RH-15"`), `gate_id`
(the tool's hunk identity), `reason` (for `kept`) and one of the actions
`kept`, `restored` or `open`. Never store hunk text, test names or commit
messages in the record (it is brain-synced). `open` findings count in
`issues_found` and in `critical`/`informational` by severity. The logger stamps
messages in the record (it is brain-synced). The logger stamps
`actor` from `gstack-session-kind` itself and discards any `actor` you supply;
for a non-interactive session it also forces `action: "open"` on every gate
finding. Dispositions are per invocation: `/ship`'s own review pass asks again.
@@ -897,8 +899,8 @@ pass: report it with its reason.
`TIMESTAMP` is ISO 8601; `COMMIT` is `git rev-parse --short HEAD`.
`quality_score` is Step 4.6's specialist score (`10.0` when small-diff specialists
were skipped or this host omits Review Army). This default is not completion evidence;
unresolved non-advisory core defects still count in `issues_found`,
`critical`, `informational`. The logger builds trusted `review_binding` from the
unresolved defects, `open` ones included, still count in `issues_found` and
by severity in `critical`/`informational`. The logger builds trusted `review_binding` from the
validated captured branch digest, discarding caller bindings. Never invent a binding
or replace REVIEW_START at log time; finish only this pass's own token.
+14 -12
View File
@@ -251,9 +251,9 @@ critical finding or another writer during collection. Skip silently if no docs e
## Step 5: Fix-First Review
Before edits, confirm every dispatched reader has returned or is confirmed stopped.
For an active or unknown reader/writer, wait or confirm it is stopped; if that
cannot be confirmed, persist incomplete at Step 5.8 and STOP without edits.
Before edits, every dispatched reader/writer must have returned or be confirmed
stopped: wait, and if that cannot be confirmed, persist incomplete at Step 5.8 and
STOP without edits.
Terminal failure does not block fixes from independent evidence. Missing required
output still makes the pass incomplete, even after the reader is stopped.
@@ -357,7 +357,7 @@ After outputting your own findings, if Greptile comments were classified in Step
Before replying to any comment, run greptile-triage.md's **Escalation Detection** to choose Tier 1 (friendly) or Tier 2 (firm) reply templates.
1. **VALID & ACTIONABLE comments:** Use their Step 5a–5d disposition; do not ask a second fix question. Step 5c alone supplies A) Fix / B) Skip for ASK items. After a completed fix, use the **Fix reply template** with diff and explanation; cite the current diff if uncommitted, never invent a commit SHA. A Skip leaves the defect unresolved and grants no new fix permission. If evidence disproves the finding, reclassify it below.
1. **VALID & ACTIONABLE comments:** Use their Step 5a–5d disposition; do not ask a second fix question. After a completed fix, use the **Fix reply template** with diff and explanation; cite the current diff if uncommitted, never invent a commit SHA. A Skip leaves the defect unresolved and grants no new fix permission. If evidence disproves the finding, reclassify it below.
2. **FALSE POSITIVE comments:** These are reply decisions, not code approval. Show file:line (or [top-level]), summary, permalink and evidence, then ask:
- A) Reply explaining why this is incorrect (recommended if clearly wrong)
@@ -386,6 +386,9 @@ Before replying to any comment, run greptile-triage.md's **Escalation Detection*
final: repeat Steps 3–5 with a full start.
2. On a repeat, execute Steps 3–5 in order. At Step 4.7, an edited pass supersedes
every earlier QA capture (exploratory §4): rerun each probe the verdict relies on.
These reruns are revalidation, so they run even after smoke expiry; keep the
PROBE_DIR. Earlier provisional sections link checkpoints; only the final pass
runs its once-only `materialize`.
A zero-edit pass reuses only this invocation's unchanged-input QA evidence,
rerunning any probe whose inputs changed or are uncertain. Reusing never skips a review step.
3. **Verify completed actions.** On the final zero-edit pass, reconcile this
@@ -397,9 +400,8 @@ Before replying to any comment, run greptile-triage.md's **Escalation Detection*
defects and unresolved counts; earlier fixes do not suppress them.
4. **Recheck skipped advice.** Re-read its final-snapshot supporting source and
reconfirm the decision; otherwise report its history without a reusable skip.
The logger computes `snapshot_covered_paths` from eligible paths whose raw bytes
equal the bound snapshot blobs (`[]` if none). Never carry prior-cycle, supplied
or prior-record coverage forward or build this proof yourself. Fixed advice
The logger computes `snapshot_covered_paths` itself; never carry prior-cycle,
supplied or prior-record coverage forward or build this proof yourself. Fixed advice
needs no skip coverage.
### 2. Fill the record
@@ -429,14 +431,14 @@ pass: report it with its reason.
`helper_target`. Recheck source after fixes. The logger uses `sharedLibsFingerprint`,
never supplied/model hashes.
Actions: `auto-fixed` (Step 5b), `fixed` (approved **and completed** in Step 5d),
`skipped` (explicit Skip in Step 5c). Advice is never `auto-fixed`; pending
`skipped` (explicit Skip in Step 5c),
`open` (any other defect, e.g. unanswered). Advice is never `auto-fixed`; pending
advice stays in the response, not the record. Exclude prior Step 5.0
suppressions; include this invocation's revalidated decisions.
- Gate findings additionally carry `gate` (the RH tag, e.g. `"RH-15"`), `gate_id`
(the tool's hunk identity), `reason` (for `kept`) and one of the actions
`kept`, `restored` or `open`. Never store hunk text, test names or commit
messages in the record (it is brain-synced). `open` findings count in
`issues_found` and in `critical`/`informational` by severity. The logger stamps
messages in the record (it is brain-synced). The logger stamps
`actor` from `gstack-session-kind` itself and discards any `actor` you supply;
for a non-interactive session it also forces `action: "open"` on every gate
finding. Dispositions are per invocation: `/ship`'s own review pass asks again.
@@ -448,8 +450,8 @@ pass: report it with its reason.
`TIMESTAMP` is ISO 8601; `COMMIT` is `git rev-parse --short HEAD`.
`quality_score` is Step 4.6's specialist score (`10.0` when small-diff specialists
were skipped or this host omits Review Army). This default is not completion evidence;
unresolved non-advisory core defects still count in `issues_found`,
`critical`, `informational`. The logger builds trusted `review_binding` from the
unresolved defects, `open` ones included, still count in `issues_found` and
by severity in `critical`/`informational`. The logger builds trusted `review_binding` from the
validated captured branch digest, discarding caller bindings. Never invent a binding
or replace REVIEW_START at log time; finish only this pass's own token.
+1 -1
View File
@@ -29,7 +29,7 @@
"test/helpers/eval-budgets.ts": {"blob":"667d0e09b20fe5d7366f57b77b5fb9b04baa37ef","since":6},
"test/helpers/eval-store.ts": {"blob":"a50ea2c28f508ad9f35ea9353dd2f502ee0dcfc2","since":6,"nonBehavioral":"credential rejections are classed infra instead of assertion: no case's pass or fail changes; only failed trials whose provider rejected the run's key move class, so no rerun spends on a rejected key"},
"test/helpers/hermetic-env.ts": {"blob":"cc072895461dc52ea317d1beff72121342420d30","since":6},
"test/helpers/llm-judge.ts": {"blob":"a3177665b662932ce485ccc976c2b36a1531a0bb","since":6,"nonBehavioral":"callJudge rewrites only the message of a JSON parse failure (names the judge output, stop reason, length and the retained llm-judge-response-parse-error event); it still throws a SyntaxError before any score is parsed, the request is unchanged, and no case's pass or fail changes"},
"test/helpers/llm-judge.ts": {"blob":"095f9c85dd0ae29da8ba10fed91510313a4442a4","since":6,"nonBehavioral":"callJudge emits one llm-judge-aborted diagnostic line (wrapper attempts, HTTP request and status counts seen through a pass-through fetch, 429 waits, connect and event timings, block kind, character counts, message and request ids, a capped reason; no content) when its caller's signal aborts it; observers swallow their own errors, and the request, retries, parsing, scores and the thrown abort reason are unchanged, so no case's pass or fail changes"},
"test/helpers/paid-test-set.ts": {"blob":"ff9bfe43c43ec2bdd11114f7b8ec7c5736116f75","since":6},
"test/helpers/pty-screen.ts": {"blob":"4f77569f9ce11f9cee9f347216e468cd7c956820","since":6},
"test/helpers/pty/auq.ts": {"blob":"16e5ea2bb5b9f28f44808a1a09024dbd3e6f28c4","since":6},
@@ -1,17 +1,13 @@
import type { TemplateContext } from '../types';
export function generateVoiceDirective(ctx: TemplateContext, tier: number): string {
const emDashRule = ctx.skillName === 'plan-eng-review'
? 'Do not add em dashes in prose you compose during the review. Existing templates, quoted text, command output, and required copied labels may contain them.'
: 'No em dashes.';
export function generateVoiceDirective(_ctx: TemplateContext, tier: number): string {
if (tier <= 1) {
return `## Voice
Direct, concrete, builder-to-builder. Name the file, function, command, and user-visible impact. No filler.
${emDashRule} No AI vocabulary: delve, crucial, robust, comprehensive, nuanced, multifaceted, load-bearing. Never corporate or academic. Short paragraphs. End with what to do.
No em dashes. No AI vocabulary: delve, crucial, robust, comprehensive, nuanced, multifaceted, load-bearing. Never corporate or academic. Short paragraphs. End with what to do.
Reply in the language of the user's latest message unless asked otherwise. Code, commands, paths, identifiers and quoted output stay verbatim.
@@ -28,7 +24,7 @@ GStack voice: Garry-shaped product and engineering judgment.
- Be direct about quality. Bugs matter. Edge cases matter. Fix the whole thing, not the demo path.
- Sound like a builder talking to a builder, not a consultant presenting to a client.
- Never corporate, academic, PR, or hype. Avoid filler, throat-clearing, generic optimism, and founder cosplay.
- ${emDashRule} No AI vocabulary: delve, crucial, robust, comprehensive, nuanced, multifaceted, furthermore, moreover, additionally, pivotal, landscape, tapestry, underscore, foster, showcase, intricate, vibrant, fundamental, significant, load-bearing.
- No em dashes. No AI vocabulary: delve, crucial, robust, comprehensive, nuanced, multifaceted, furthermore, moreover, additionally, pivotal, landscape, tapestry, underscore, foster, showcase, intricate, vibrant, fundamental, significant, load-bearing.
- Reply in the language of the user's latest message unless asked otherwise. Code, commands, paths, identifiers, quoted output and question markers (\`D<N>\`, option letters, \`(recommended)\`) stay verbatim.
- The user has context you do not: domain knowledge, timing, relationships, taste. Cross-model agreement is a recommendation, not a decision. The user decides.
+49 -3
View File
@@ -1,4 +1,4 @@
import { afterEach, describe, expect, test } from 'bun:test';
import { afterEach, describe, expect, spyOn, test } from 'bun:test';
import { createHash } from 'node:crypto';
import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
@@ -11,7 +11,8 @@ import { selectTests } from './helpers/test-selection';
import { E2E_TOUCHFILES, LLM_JUDGE_TOUCHFILES } from './helpers/touchfiles-data';
import { selectPrProfile } from '../scripts/test-pr-profile';
import { JUDGE_MS } from './helpers/eval-budgets';
import { JudgeRefusalError, DEFAULT_JUDGE_MAX_TOKENS, judgePanel, judgePanelMean, judgePanelMedian, judgePanelReasoning, JUDGE_SCORE_DIMENSIONS, JUDGE_PANEL_SAMPLES } from './helpers/llm-judge';
import { Messages } from '@anthropic-ai/sdk/resources/messages';
import { callJudge, type CallJudgeOptions, JudgeRefusalError, DEFAULT_JUDGE_MAX_TOKENS, judgePanel, judgePanelMean, judgePanelMedian, judgePanelReasoning, JUDGE_SCORE_DIMENSIONS, JUDGE_PANEL_SAMPLES } from './helpers/llm-judge';
import { COOKIE_MANUAL_REVIEW_FILE, getCookieWorkflowManualReview, isManualReviewEntry } from './helpers/cookie-workflow-manual-review';
const ROOT = resolve(import.meta.dir, '..');
@@ -51,6 +52,7 @@ const passingScore: FixtureScore = { ...COOKIE_WORKFLOW_JUDGE.thresholds, reason
function actualCookieCallback(root: string, overrides: {
judge?: () => Promise<FixtureScore>;
callJudge?: (prompt: string, model: string | undefined, options: CallJudgeOptions) => Promise<FixtureScore>;
clock?: () => number;
budget?: number;
setTimer?: typeof setTimeout;
@@ -74,7 +76,11 @@ function actualCookieCallback(root: string, overrides: {
(_suite: string, names: string[], run: () => void) => { expect(names).toEqual([NAME]); run(); },
(name: string, run: () => Promise<void>, budget: number) => { expect(name).toBe(NAME); expect(budget).toBe(JUDGE_MS + 10_000); callback = run; },
root, buildCookieWorkflowJudgeInput, (_kind: string, explicit?: string) => explicit ?? 'fixture-model',
async (prompt: string, model: string | undefined, options: { signal: AbortSignal }) => { requests.push({ prompt, model, signal: options.signal }); return overrides.judge ? overrides.judge() : passingScore; },
async (prompt: string, model: string | undefined, options: CallJudgeOptions & { signal: AbortSignal }) => {
requests.push({ prompt, model, signal: options.signal });
if (overrides.callJudge) return overrides.callJudge(prompt, model, options);
return overrides.judge ? overrides.judge() : passingScore;
},
COOKIE_WORKFLOW_JUDGE, overrides.budget ?? JUDGE_MS, JUDGE_MS + 10_000, 5_000,
{ addTest: (record: EvalTestEntry) => records.push(record) }, expect, { log() {} },
readWorkflowJudgeInput, buildWorkflowJudgePrompt,
@@ -269,6 +275,46 @@ describe('cookie workflow judge input', () => {
expect(selectTests(['README.md'], LLM_JUDGE_TOUCHFILES).selected).not.toContain(NAME);
});
test('the registered cookie judge sends the response schema, so a free-text reply like train 42\'s is not what gets parsed', async () => {
const { replies } = JSON.parse(readFileSync(join(ROOT, 'test/fixtures/cookie-judge-train42-replies.json'), 'utf8')) as {
replies: Array<{ id: string; model: string; stop_reason: string; text: string }> };
expect(replies.map(reply => reply.id)).toEqual(['msg_011CgVnRtVj5vyLvb56m6KJH', 'msg_011CgVne5GStZ6f9pRoCvgWE']);
const original = process.env.ANTHROPIC_API_KEY;
process.env.ANTHROPIC_API_KEY = 'fake';
const create = spyOn(Messages.prototype, 'create');
try {
for (const reply of replies) {
expect(() => JSON.parse(reply.text)).toThrow();
const marker = '"reasoning": "';
const judgment: FixtureScore = {
clarity: Number(/"clarity": (\d)/.exec(reply.text)![1]),
completeness: Number(/"completeness": (\d)/.exec(reply.text)![1]),
actionability: Number(/"actionability": (\d)/.exec(reply.text)![1]),
reasoning: reply.text.slice(reply.text.indexOf(marker) + marker.length, reply.text.lastIndexOf('"}')),
};
expect(judgment.reasoning).toContain('"one --domain per run"');
create.mockReset();
create.mockResolvedValue({ id: reply.id, model: reply.model, stop_reason: 'end_turn',
content: [{ type: 'text', text: JSON.stringify(judgment) }], usage: { input_tokens: 1, output_tokens: 1 } } as never);
const root = fixture();
const h = actualCookieCallback(root, { callJudge: (prompt, model, options) => callJudge<FixtureScore>(prompt, model, options) });
await h.run();
expect(create).toHaveBeenCalledTimes(JUDGE_PANEL_SAMPLES);
for (const [request, options] of create.mock.calls as unknown as Array<[Record<string, unknown>, { signal: AbortSignal }]>) {
expect(request).toEqual({ model: COOKIE_WORKFLOW_JUDGE.model, max_tokens: DEFAULT_JUDGE_MAX_TOKENS,
output_config: { format: { type: 'json_schema', schema: WORKFLOW_JUDGE_RESPONSE_SCHEMA } },
messages: [{ role: 'user', content: buildCookieWorkflowJudgeInput(root).prompt }] });
expect(options.signal).toBeInstanceOf(AbortSignal);
}
expect(h.records[0]).toMatchObject({ name: NAME, passed: true, execution: 'executed',
judge_scores: { clarity: judgment.clarity, completeness: judgment.completeness, actionability: judgment.actionability } });
}
} finally {
create.mockRestore();
if (original === undefined) delete process.env.ANTHROPIC_API_KEY; else process.env.ANTHROPIC_API_KEY = original;
}
});
test('the managed callback sends and records exact custom input without borrowing a normal cache identity', async () => {
const root = fixture();
const input = buildCookieWorkflowJudgeInput(root);
+13
View File
@@ -163,3 +163,16 @@ test('calibration keeps its future hook but no shipped source enables the absent
}
expect(mentions).toEqual(['scripts/resolvers/gbrain.ts']);
});
test('the judged Eng bundle carries the shared Voice once and states its em-dash precedence in the Scope gate (paid screen 40634787316)', () => {
// Five of six samples (red panel and rerun) named Eng's near-copy of the shared Voice, whose only difference was
// its em-dash clause, as a rule the agent must reconcile; the Scope gate's Format precedence already owned it.
// fails_when=a second Voice section returns to the judged bundle or the precedence stops naming em dashes.
const input = readWorkflowJudgeInput({root: path.join(import.meta.dir, '..'), skillPath: 'plan-eng-review/SKILL.md',
startMarker: '# Plan Review Mode', endMarker: null});
expect(input.text.match(/^## Voice$/gm)).toHaveLength(1);
expect(input.text).not.toContain('Do not add em dashes in prose you compose');
expect(scope).toContain('**Format precedence:** Copy required command, output and question formats exactly, em dashes included. Apply Voice to newly composed prose.');
const startup = input.files[0]!.content;
expect(startup.indexOf('**Format precedence:**')).toBeLessThan(startup.indexOf('## Shared preamble (load first)'));
});
+17
View File
@@ -0,0 +1,17 @@
{
"source": "train 42 gate run 40809051510 (paid-logs-slice-5-a1, rerun-logs-1-a1): llm-judge-response-parse-error events",
"replies": [
{
"id": "msg_011CgVnRtVj5vyLvb56m6KJH",
"model": "claude-sonnet-5-5",
"stop_reason": "end_turn",
"text": "{\"clarity\": 4, \"completeness\": 4, \"actionability\": 4, \"reasoning\": \"The workflow is well ordered: check Aside, collect decisions up front, prepare the browser, import, and report. Consent for storage reset and --all is explicit, and the guidance separates copied cookies from verified sign-in. Error tables, platform limits, and the privacy rules are clear. Weaknesses: (1) the picker-versus-direct choice is underspecified, because the picker is used when browser, profile or domains are open and direct import when they are named, but the target-URL, `--all`, and verify-auth flows under each are not fully mapped. (2) The `--all` handling is thin. SKILL.md does not give direct-import syntax for it and does not say whether `$B goto` is still needed. (3) Verification setup has an ordering wrinkle: the env vars must be set before the daemon starts, while the Aside and `$B status` checks come first, and the instructions only partly say when to stop and restart the daemon. (4) A few edge cases are not defined, such as what to do when the reset or verification fails, how to count partial results, and how to proceed after the failure-reason handoff. (5) The multi-domain instruction is \"one --domain per run\", but it is unclear whether to repeat runs or how to aggregate the receipts. (6) The \"Windows App-Bound Encryption\" recovery depends on a display, with no alternative if none is available. These gaps are minor, and the workflow is mostly executable.\"}"
},
{
"id": "msg_011CgVne5GStZ6f9pRoCvgWE",
"model": "claude-sonnet-5-5",
"stop_reason": "end_turn",
"text": "{\"clarity\": 4, \"completeness\": 4, \"actionability\": 4, \"reasoning\": \"The workflow is well ordered: probe Aside, collect all decisions up front, set up the browser, import, then report three honest lines. Consent boundaries for verification, storage clearing and --all are explicit, the sign-in-versus-cookie-copy distinction is stated repeatedly, and the error and receipt tables give recovery paths. Gaps: (1) Step 2 asks for decisions before `$B` exists, yet the picker-versus-direct choice is tied to browser, profile and domains rather than to the user's wording, and the step 3.1 env vars come before the daemon starts. (2) The conditional step 3.3 (Mode: cdp) and the `$B stop` guidance could be cleaner. (3) The direct-import syntax for `--all` is not shown, and the \"one --domain per run\" rule is stated without saying what to do for several domains (loop, with the tab open on each). (4) The verification-missing return path says to return to step 3 but doesn't say how to handle the already-running daemon beyond `$B stop`. (5) Handling of partial and failed receipts, and the handoff for non-Windows cases, is thin. (6) How the agent learns the profile directory when the user only knows a display name is left to the picker. (7) The root-selection logic with \"never a generated host stub\" is a bit dense. These are minor; the workflow is mostly executable.\"}"
}
]
}
+5 -1
View File
@@ -51,7 +51,11 @@ reservation; its 14 cases now send `WORKFLOW_JUDGE_RESPONSE_SCHEMA` without the
compact-reasoning validator (still only on `ship`). The frontier (`review`) configuration has no corpus and stays on today's request. A $0.12 two-call pilot sized
output tokens before the manifest was priced. Total W2 spend: $90.47 of $115.
`cookie-workflow` (GSTA-91, separately funded) is held and stays on today's request. Its one
`cookie-workflow` (GSTA-91, separately funded) was held, and the hold was released on
GStack Minder's ruling of 2026-10-29: "GStack Minder, 2026-10-29: release the GSTA-91 hold; a07 is
matched by old-b; free-text parse errors cost train 42" (`cookie-workflow/release.json`). The
cookie workflow judge now sends `WORKFLOW_JUDGE_RESPONSE_SCHEMA`, with its thresholds, model and
samples unchanged; the comparison below stays recorded as held. Its one
new false pass, a07, scored 4/4/4 on all three new samples and on all three `old-b` samples;
`old-a`'s a07 panel is incomplete because one old sample returned invalid JSON (the defect the
schema transport removes), so the comparison counts it against the new request only. Both
@@ -0,0 +1,16 @@
{
"config": "cookie-workflow",
"released": true,
"date": "2026-10-29",
"by": "GStack Minder",
"ruling": "GStack Minder, 2026-10-29: release the GSTA-91 hold; a07 is matched by old-b; free-text parse errors cost train 42",
"scope": "cookie workflow judge transport only",
"recorded_result": "runs/comparison-1.result.json",
"train_red": {
"gate_run": "https://github.com/garrytan/gstack/actions/runs/40809051510",
"responses": [
"msg_011CgVnRtVj5vyLvb56m6KJH",
"msg_011CgVne5GStZ6f9pRoCvgWE"
]
}
}
+19 -19
View File
@@ -5,50 +5,50 @@
"agy/gstack-autoplan": 109810,
"agy/gstack-plan-design-review": 110044,
"agy/gstack-plan-devex-review": 116496,
"agy/gstack-plan-eng-review": 147241,
"agy/gstack-review": 103755,
"agy/gstack-plan-eng-review": 144907,
"agy/gstack-review": 103743,
"codex/gstack-autoplan": 108553,
"codex/gstack-land-and-deploy": 100284,
"codex/gstack-plan-design-review": 111116,
"codex/gstack-plan-devex-review": 118004,
"codex/gstack-plan-eng-review": 148406,
"codex/gstack-review": 103278,
"codex/gstack-plan-eng-review": 146072,
"codex/gstack-review": 103266,
"copilot/gstack-autoplan": 109253,
"copilot/gstack-plan-design-review": 110355,
"copilot/gstack-plan-devex-review": 115958,
"copilot/gstack-plan-eng-review": 147406,
"copilot/gstack-review": 103336,
"copilot/gstack-plan-eng-review": 145072,
"copilot/gstack-review": 103324,
"cursor/gstack-autoplan": 108485,
"cursor/gstack-plan-design-review": 108900,
"cursor/gstack-plan-devex-review": 115228,
"cursor/gstack-plan-eng-review": 145996,
"cursor/gstack-review": 121626,
"cursor/gstack-plan-eng-review": 143662,
"cursor/gstack-review": 121614,
"factory/gstack-autoplan": 110763,
"factory/gstack-land-and-deploy": 100105,
"factory/gstack-plan-design-review": 111675,
"factory/gstack-plan-devex-review": 118591,
"factory/gstack-plan-eng-review": 149104,
"factory/gstack-review": 125399,
"factory/gstack-plan-eng-review": 146770,
"factory/gstack-review": 125387,
"gbrain/gstack-autoplan": 108892,
"gbrain/gstack-plan-eng-review": 128623,
"gbrain/gstack-plan-eng-review": 126289,
"hermes/gstack-autoplan": 108982,
"hermes/gstack-plan-eng-review": 128708,
"hermes/gstack-plan-eng-review": 126374,
"kiro/gstack-autoplan": 108259,
"kiro/gstack-plan-design-review": 108752,
"kiro/gstack-plan-devex-review": 115080,
"kiro/gstack-plan-eng-review": 145858,
"kiro/gstack-review": 121450,
"kiro/gstack-plan-eng-review": 143524,
"kiro/gstack-review": 121438,
"openclaw/gstack-autoplan": 108715,
"openclaw/gstack-plan-eng-review": 127138,
"openclaw/gstack-plan-eng-review": 124804,
"opencode/gstack-autoplan": 109234,
"opencode/gstack-plan-design-review": 110187,
"opencode/gstack-plan-devex-review": 115836,
"opencode/gstack-plan-eng-review": 147265,
"opencode/gstack-review": 122248,
"opencode/gstack-plan-eng-review": 144931,
"opencode/gstack-review": 122236,
"slate/gstack-autoplan": 108372,
"slate/gstack-plan-design-review": 108826,
"slate/gstack-plan-devex-review": 115154,
"slate/gstack-plan-eng-review": 145927,
"slate/gstack-review": 121538
"slate/gstack-plan-eng-review": 143593,
"slate/gstack-review": 121526
}
}
+16
View File
@@ -84,6 +84,22 @@ describe('write (FRESH bump)', () => {
expect(code).toBe(2);
});
test('--manifest-form full keeps a four-part manifest and its lockfile as written', () => {
const d2 = fs.mkdtempSync(path.join(os.tmpdir(), 'vbump-full-'));
fs.writeFileSync(path.join(d2, 'VERSION'), '0.60.482.0\n');
fs.writeFileSync(path.join(d2, 'package.json'), JSON.stringify({ name: 'g', version: '0.60.482.0' }, null, 2) + '\n');
fs.writeFileSync(path.join(d2, 'package-lock.json'), JSON.stringify({ name: 'g', version: '0.60.482.0', lockfileVersion: 3, packages: { '': { version: '0.60.482.0' } } }, null, 2) + '\n');
const out = JSON.parse(execFileSync('bun', [BIN, 'write', '--version', '0.60.483.0', '--manifest-form', 'full'], { cwd: d2, timeout: 30_000 }).toString());
expect(out.packageJsonVersion).toBe('0.60.483.0');
expect(JSON.parse(fs.readFileSync(path.join(d2, 'package.json'), 'utf-8')).version).toBe('0.60.483.0');
const lock = JSON.parse(fs.readFileSync(path.join(d2, 'package-lock.json'), 'utf-8'));
expect([lock.version, lock.packages[''].version]).toEqual(['0.60.483.0', '0.60.483.0']);
const bad = spawnSync('bun', [BIN, 'write', '--version', '0.60.484.0', '--manifest-form', 'wide'], { cwd: d2, timeout: 30_000 });
expect(bad.status).toBe(2);
expect(fs.readFileSync(path.join(d2, 'VERSION'), 'utf-8').trim()).toBe('0.60.483.0');
fs.rmSync(d2, { recursive: true, force: true });
});
test('VERSION-only repo (no package.json) writes just VERSION', () => {
const d2 = fs.mkdtempSync(path.join(os.tmpdir(), 'vbump-noPkg-'));
fs.writeFileSync(path.join(d2, 'VERSION'), '0.1.0.0\n');
+1
View File
@@ -62,6 +62,7 @@ export const FREE_FIXTURES: Readonly<Record<string, readonly string[]>> = {
'test/fixtures/context-bill/**': ['test/context-bill.test.ts', 'test/context-budget-ratchet.test.ts'],
'test/fixtures/context-budget.json': ['test/helpers/capture-context-budget.ts', 'test/test-value-bar.test.ts'],
'test/fixtures/context-restore-fragment-match-captures.json': ['test/context-restore-fragment-verdict.test.ts'],
'test/fixtures/cookie-judge-train42-replies.json': ['test/cookie-workflow-judge-input.test.ts'],
'test/fixtures/coverage-audit-cd-chain.json': ['test/coverage-audit-evidence.test.ts'],
'test/fixtures/coverage-audit-parallel-column.json': ['test/coverage-audit-evidence.test.ts'],
'test/fixtures/coverage-audit-separator-collision.json': ['test/coverage-audit-evidence.test.ts'],
+63 -21
View File
@@ -135,7 +135,21 @@ export async function callJudge<T>(
// could be exhausted before a frontier judge emitted any JSON.
const resolvedModel = resolveEvalModel('judge', model);
const maxTokens = opts?.max_tokens ?? DEFAULT_JUDGE_MAX_TOKENS;
const client = new Anthropic();
// What a cancelled request had received, so a judge deadline names rate-limit
// waits, SDK retries, a stream with no events, or a long thinking phase. Counts
// only, never content; an observer that throws never changes the judge outcome.
const started = performance.now();
const elapsedMs = () => Math.round(performance.now() - started);
const progress = { httpRequests: 0, httpStatuses: [] as number[], retryWaitsMs: [] as number[], connectedMs: null as number | null,
firstEventMs: null as number | null, lastEventMs: null as number | null, block: null as string | null, thinkingChars: 0,
textChars: 0, messageId: null as string | null, requestId: null as string | null };
const observe = (record: () => void) => { try { record(); } catch {} };
const client = new Anthropic({ fetch: async (...args: Parameters<typeof fetch>) => {
progress.httpRequests += 1;
const response = await globalThis.fetch(...args);
progress.httpStatuses.push(response.status);
return response;
} });
const request = {
model: resolvedModel,
@@ -146,34 +160,62 @@ export async function callJudge<T>(
...(opts?.effort === undefined ? {} : { effort: opts.effort }) } }),
messages: [{ role: 'user' as const, content: prompt }],
};
const makeRequest = () => opts?.stream
? client.messages.stream(request, signal ? { signal } : undefined).finalMessage()
: client.messages.create(request, signal ? { signal } : undefined);
const makeRequest = () => {
if (!opts?.stream) return client.messages.create(request, signal ? { signal } : undefined);
const stream = client.messages.stream(request, signal ? { signal } : undefined);
stream.on('connect', () => observe(() => {
progress.connectedMs ??= elapsedMs();
progress.requestId ??= stream.request_id ?? null;
}));
stream.on('streamEvent', event => observe(() => {
progress.firstEventMs ??= elapsedMs();
progress.lastEventMs = elapsedMs();
progress.requestId ??= stream.request_id ?? null;
if (event.type === 'message_start') progress.messageId = event.message.id;
if (event.type === 'content_block_start') progress.block = event.content_block.type;
if (event.type === 'content_block_delta' && event.delta.type === 'thinking_delta') progress.thinkingChars += event.delta.thinking.length;
if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') progress.textChars += event.delta.text.length;
}));
return stream.finalMessage();
};
// 429s under CI concurrency: jittered exponential backoff over 3 retries
// (~1s/4s/16s + jitter), honoring the server's retry-after when present.
// The old single fixed 1s retry lost races reliably at 40-way concurrency.
let response;
let attempt = 0;
for (;;) {
try {
signal?.throwIfAborted();
response = await makeRequest();
signal?.throwIfAborted();
break;
} catch (err: any) {
signal?.throwIfAborted();
if (err?.status !== 429 || attempt >= 3) throw err;
const retryAfterSecs = Number(err?.headers?.['retry-after']);
const baseMs = Number.isFinite(retryAfterSecs) && retryAfterSecs > 0
? retryAfterSecs * 1000
: 1000 * 4 ** attempt;
await delay(baseMs + Math.random() * 500, undefined, { signal }).catch(error => {
try {
for (;;) {
try {
signal?.throwIfAborted();
throw error;
});
attempt += 1;
response = await makeRequest();
signal?.throwIfAborted();
break;
} catch (err: any) {
signal?.throwIfAborted();
if (err?.status !== 429 || attempt >= 3) throw err;
const retryAfterSecs = Number(err?.headers?.['retry-after']);
const baseMs = Number.isFinite(retryAfterSecs) && retryAfterSecs > 0
? retryAfterSecs * 1000
: 1000 * 4 ** attempt;
const waitMs = baseMs + Math.random() * 500;
progress.retryWaitsMs.push(Math.round(waitMs));
await delay(waitMs, undefined, { signal }).catch(error => {
signal?.throwIfAborted();
throw error;
});
attempt += 1;
}
}
} catch (error) {
if (signal?.aborted) {
console.error(JSON.stringify({
type: 'llm-judge-aborted', model: resolvedModel, stream: opts?.stream === true, elapsedMs: elapsedMs(),
attempts: attempt + 1, ...progress,
reason: (signal.reason instanceof Error ? signal.reason.message : String(signal.reason)).slice(0, 200),
}));
}
throw error;
}
opts?.onResponse?.({
+6 -2
View File
@@ -321,17 +321,21 @@ describe('landed schemas match the recorded calibration', () => {
expect(recorded.complete).toBe(true);
const cookie = result('cookie-workflow');
expect(cookie.complete).toBe(true);
expect(cookie.landing.lands).toBe(false);
const release = JSON.parse(fs.readFileSync(path.join(import.meta.dir, 'fixtures', 'judge-calibration', 'cookie-workflow', 'release.json'), 'utf8'));
expect(release).toMatchObject({ config: 'cookie-workflow', released: true, by: 'GStack Minder', scope: 'cookie workflow judge transport only',
ruling: 'GStack Minder, 2026-10-29: release the GSTA-91 hold; a07 is matched by old-b; free-text parse errors cost train 42' });
const registrations = [...evalSource.matchAll(/await runWorkflowJudge\(\{([\s\S]*?)\n \}\);/g)].map(match => match[1]!);
expect(registrations.length).toBeGreaterThan(14);
for (const body of registrations) {
const testName = /testName: '([^']+)'/.exec(body)![1];
const defaultConfig = !/^\s+(?:agentCapability|model|readInput|thresholds|maxTokens|stream|effort):|\.\.\.COOKIE_WORKFLOW_JUDGE/m.test(body);
const expected = testName === 'ship/SKILL.md workflow' || (defaultConfig && recorded.landing.lands)
|| (testName === 'setup-browser-cookies/SKILL.md workflow' && cookie.landing.lands);
|| testName === 'setup-browser-cookies/SKILL.md workflow';
expect(body.includes('schemaTransport: true'), testName).toBe(expected);
expect(body.includes('compactReasoning: true'), testName).toBe(testName === 'ship/SKILL.md workflow');
}
expect(registrations.filter(body => body.includes('schemaTransport: true'))).toHaveLength(cookie.landing.lands ? 16 : 15);
expect(registrations.filter(body => body.includes('schemaTransport: true'))).toHaveLength(16);
const cookieBody = registrations.find(body => body.includes("testName: 'setup-browser-cookies/SKILL.md workflow'"))!;
expect(cookieBody).toContain('...COOKIE_WORKFLOW_JUDGE');
expect(cookieBody).not.toMatch(/^\s+(?:agentCapability|model|thresholds|maxTokens|stream|effort):/m);
+19
View File
@@ -85,6 +85,25 @@ describe('callJudge cancellation', () => {
expect(create).toHaveBeenCalledTimes(1);
}, 3000);
test('an abort during a 429 wait retains the wait it interrupted', async () => {
const diagnostics = spyOn(console, 'error').mockImplementation(() => {});
try {
const controller = new AbortController();
create.mockRejectedValue({ status: 429, headers: { 'retry-after': '30' } });
const pending = callJudge('Rate-limited request.', 'test-judge-model', { signal: controller.signal });
await new Promise<void>(resolve => setImmediate(resolve));
controller.abort(new Error('Expired during rate-limit backoff.'));
await expect(pending).rejects.toThrow('Expired during rate-limit backoff.');
expect(diagnostics).toHaveBeenCalledTimes(1);
expect(JSON.parse(diagnostics.mock.calls[0]![0])).toMatchObject({ type: 'llm-judge-aborted', model: 'test-judge-model',
stream: false, attempts: 1, httpRequests: 0, retryWaitsMs: [30000], firstEventMs: null, reason: 'Expired during rate-limit backoff.' });
create.mockResolvedValue(response as any);
diagnostics.mockClear();
expect(await callJudge('Judge this.', undefined, { signal: new AbortController().signal })).toEqual({ passed: true });
expect(diagnostics).not.toHaveBeenCalled();
} finally { diagnostics.mockRestore(); }
}, 3000);
test('keeps three 429 retries for callers without a signal', async () => {
const rateLimit = { status: 429, headers: { 'retry-after': '0.001' } };
create.mockRejectedValue(rateLimit);
+120
View File
@@ -103,3 +103,123 @@ test('streaming retains the caller abort signal instead of extending its deadlin
expect(await rejected).toBe(failure);
expect(transport).toHaveBeenCalledTimes(1);
});
test('a judge aborted mid-stream retains what it had received, as counts without content', async () => {
const encoder = new TextEncoder();
const partial = [
{ type: 'message_start', message: { id: 'msg_stalled', type: 'message', role: 'assistant', model: 'claude-fable-5-1',
content: [], stop_reason: null, stop_sequence: null, usage: { input_tokens: 99023, output_tokens: 1 } } },
{ type: 'content_block_start', index: 0, content_block: { type: 'thinking', thinking: '', signature: '' } },
{ type: 'content_block_delta', index: 0, delta: { type: 'thinking_delta', thinking: 'PRIVATE_THINKING' } },
];
let delivered!: () => void;
const received = new Promise<void>(resolve => { delivered = resolve; });
transport.mockImplementation(async (_url: string, init: RequestInit) => {
const chunks = partial.map(event => encoder.encode(`event: ${event.type}\ndata: ${JSON.stringify(event)}\n\n`));
return new Response(new ReadableStream({
start(stream) {
init.signal!.addEventListener('abort', () => stream.error(init.signal!.reason), { once: true });
},
pull(stream) {
const chunk = chunks.shift();
if (chunk) stream.enqueue(chunk);
else delivered();
},
}, { highWaterMark: 0 }), { headers: { 'content-type': 'text/event-stream', 'request-id': 'req_stalled' } });
});
const controller = new AbortController();
const rejected = callJudge('score the whole bundle', 'claude-fable-5-1', {
max_tokens: 65_536, stream: true, jsonSchema: WORKFLOW_JUDGE_RESPONSE_SCHEMA, signal: controller.signal,
}).catch(error => error);
await received;
const failure = new Error('ship/SKILL.md workflow exceeded its workflow judge work deadline');
controller.abort(failure);
expect(await rejected).toBe(failure);
expect(diagnostics).toHaveBeenCalledTimes(1);
const diagnostic = diagnostics.mock.calls[0]![0];
expect(diagnostic).not.toContain('PRIVATE_');
const event = JSON.parse(diagnostic);
expect(event).toMatchObject({ type: 'llm-judge-aborted', model: 'claude-fable-5-1', stream: true, attempts: 1,
httpRequests: 1, httpStatuses: [200], retryWaitsMs: [], block: 'thinking', thinkingChars: 'PRIVATE_THINKING'.length, textChars: 0,
messageId: 'msg_stalled', requestId: 'req_stalled', reason: failure.message });
expect(event.firstEventMs).toBeGreaterThanOrEqual(0);
expect(event.lastEventMs).toBeGreaterThanOrEqual(event.firstEventMs); // gstack-timing-ok: ordering of two recorded offsets, not an elapsed-time bound
expect(event.elapsedMs).toBeGreaterThanOrEqual(event.lastEventMs); // gstack-timing-ok: ordering of two recorded offsets, not an elapsed-time bound
});
test('a stream that never sent an event is retained as such, and provider errors without an abort are not', async () => {
let sent!: () => void;
const requested = new Promise<void>(resolve => { sent = resolve; });
transport.mockImplementation((_url: string, init: RequestInit) => new Promise((_resolve, reject) => {
init.signal!.addEventListener('abort', () => reject(init.signal!.reason), { once: true });
sent();
}));
const controller = new AbortController();
const rejected = callJudge('score', 'claude-fable-5-1', { max_tokens: 65_536, stream: true, signal: controller.signal }).catch(error => error);
await requested;
controller.abort(new Error('deadline'));
await rejected;
expect(JSON.parse(diagnostics.mock.calls[0]![0])).toMatchObject({ type: 'llm-judge-aborted', httpRequests: 1,
httpStatuses: [], connectedMs: null, firstEventMs: null, lastEventMs: null, block: null, thinkingChars: 0, messageId: null, requestId: null, reason: 'deadline' });
diagnostics.mockClear();
transport.mockResolvedValue(new Response(JSON.stringify({ type: 'error', error: { type: 'api_error', message: 'boom' } }),
{ status: 500, headers: { 'content-type': 'application/json', 'x-should-retry': 'false' } }));
await expect(callJudge('score', 'claude-fable-5-1', { max_tokens: 65_536, stream: true, signal: new AbortController().signal })).rejects.toThrow();
expect(diagnostics).not.toHaveBeenCalled();
});
test('a deadline after headers but before any event retains the connection and its request id', async () => {
let reading!: () => void;
const connected = new Promise<void>(resolve => { reading = resolve; });
transport.mockImplementation(async (_url: string, init: RequestInit) => new Response(new ReadableStream({
start(stream) { init.signal!.addEventListener('abort', () => stream.error(init.signal!.reason), { once: true }); },
pull() { reading(); return new Promise<void>(() => {}); },
}, { highWaterMark: 0 }), { headers: { 'content-type': 'text/event-stream', 'request-id': 'req_quiet' } }));
const controller = new AbortController();
const rejected = callJudge('score', 'claude-fable-5-1', { max_tokens: 65_536, stream: true, signal: controller.signal }).catch(error => error);
await connected;
controller.abort(new Error('deadline'));
await rejected;
const event = JSON.parse(diagnostics.mock.calls[0]![0]);
expect(event).toMatchObject({ httpRequests: 1, httpStatuses: [200], requestId: 'req_quiet', firstEventMs: null, block: null });
expect(event.connectedMs).toBeGreaterThanOrEqual(0);
});
test('SDK-internal 429 retries are counted, so throttling never reads as a silent stream', async () => {
let resent!: () => void;
const retried = new Promise<void>(resolve => { resent = resolve; });
transport.mockResolvedValueOnce(new Response(JSON.stringify({ type: 'error', error: { type: 'rate_limit_error', message: 'slow down' } }),
{ status: 429, headers: { 'content-type': 'application/json', 'retry-after-ms': '1' } }));
transport.mockImplementationOnce((_url: string, init: RequestInit) => new Promise((_resolve, reject) => {
init.signal!.addEventListener('abort', () => reject(init.signal!.reason), { once: true });
resent();
}));
const controller = new AbortController();
const rejected = callJudge('score', 'claude-fable-5-1', { max_tokens: 65_536, stream: true, signal: controller.signal }).catch(error => error);
await retried;
controller.abort(new Error('deadline'));
await rejected;
expect(JSON.parse(diagnostics.mock.calls[0]![0])).toMatchObject({ attempts: 1, httpRequests: 2, httpStatuses: [429],
retryWaitsMs: [], connectedMs: null, firstEventMs: null });
});
test('an event the diagnostic cannot read never changes the judge result', async () => {
const unreadable = response();
const body = (await unreadable.text()).replace('"thinking":"PRIVATE_THINKING"', '"thinking":null');
transport.mockResolvedValue(new Response(body, { headers: { 'content-type': 'text/event-stream' } }));
expect(await callJudge('score the whole bundle', 'claude-fable-5-1', {
max_tokens: 65_536, stream: true, jsonSchema: WORKFLOW_JUDGE_RESPONSE_SCHEMA,
})).toEqual(scores);
});
test('an abort reason is retained at most 200 characters long', async () => {
transport.mockImplementation((_url: string, init: RequestInit) => new Promise((_resolve, reject) => {
init.signal!.addEventListener('abort', () => reject(init.signal!.reason), { once: true });
}));
const controller = new AbortController();
const rejected = callJudge('score', 'claude-fable-5-1', { max_tokens: 65_536, stream: true, signal: controller.signal }).catch(error => error);
controller.abort(new Error('x'.repeat(500)));
await rejected;
expect(JSON.parse(diagnostics.mock.calls[0]![0]).reason).toBe('x'.repeat(200));
});
+111
View File
@@ -0,0 +1,111 @@
/**
* A four-part package.json version survives gstack-restamp in a repo whose
* convention is a four-part manifest (gbrain: VERSION 0.60.483.0 and
* package.json "0.60.483.0", cross-checked by its CI version gate). The
* writer used to store the npm-valid three-part form there ("0.60.483").
* gstack's own shape (a three-part manifest outside the policy's stamp files)
* and the decision-11 migration of an unlisted four-part manifest are unchanged.
*/
import { afterEach, describe, expect, test } from 'bun:test';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { EXIT } from '../lib/headless-artifacts';
import { cleanup, makeFixtureRepo, makePr, readFile, restamp, type FixtureRepo } from './helpers/restamp-fixture';
const repos: FixtureRepo[] = [];
const fixture = (opts?: Parameters<typeof makeFixtureRepo>[0]) => { const r = makeFixtureRepo(opts); repos.push(r); return r; };
afterEach(() => { while (repos.length) cleanup(repos.pop()!); });
const GBRAIN_POLICY = {
$schema: 'https://github.com/garrytan/gstack/docs/ship-policy.md#schema-v1',
bump: 'patch',
queue_mode: 'stamp-at-merge',
changelog: 'CHANGELOG.md',
stamp_files: ['VERSION', 'package.json', 'CHANGELOG.md'],
stamp_patterns: ['"version": "{version}"'],
};
const pkgVersion = (repo: FixtureRepo) => JSON.parse(readFile(repo, 'package.json')).version;
const setPkgVersion = (repo: FixtureRepo, version: string) => {
const pkg = JSON.parse(readFile(repo, 'package.json'));
fs.writeFileSync(path.join(repo.work, 'package.json'), JSON.stringify({ ...pkg, version }, null, 2) + '\n');
};
/** Rewrite the fixture's base as gbrain ships it: a four-part manifest mirroring VERSION, and bun, so no npm lockfile. */
function fourPartManifest(repo: FixtureRepo, version: string): void {
setPkgVersion(repo, version);
fs.rmSync(path.join(repo.work, 'package-lock.json'));
repo.git('add', '-A');
repo.git('commit', '-q', '-m', 'four-part manifest');
repo.git('push', '-q', 'origin', 'main');
}
describe('gstack-restamp keeps the repo\'s manifest width', () => {
test('gbrain-shaped repo: package.json keeps four parts and agrees with VERSION', () => {
const repo = fixture({ policy: GBRAIN_POLICY, version: '0.60.482.0' });
fourPartManifest(repo, '0.60.482.0');
makePr(repo, 'feat-a');
const r = restamp(repo, ['--next']);
expect(r.status).toBe(EXIT.ok);
expect(r.out).toContain('STAMP_VERSION: 0.60.483.0');
expect(r.out).toContain('stage version: done (wrote 0.60.483.0 (package.json keeps four parts');
expect(readFile(repo, 'VERSION').trim()).toBe('0.60.483.0');
expect(pkgVersion(repo)).toBe('0.60.483.0');
});
test('gbrain-shaped branch that already carries its stamp: the restamp leaves the four-part manifest as is', () => {
const repo = fixture({ policy: GBRAIN_POLICY, version: '0.60.482.0' });
fourPartManifest(repo, '0.60.482.0');
makePr(repo, 'feat-a');
fs.writeFileSync(path.join(repo.work, 'VERSION'), '0.60.483.0\n');
setPkgVersion(repo, '0.60.483.0');
repo.git('commit', '-q', '-am', 'stamp 0.60.483.0');
const r = restamp(repo, ['--next']);
expect(r.status).toBe(EXIT.ok);
expect(r.out).toContain('STAMP_VERSION: 0.60.483.0');
expect(r.out).not.toContain('diff --git a/package.json');
expect(pkgVersion(repo)).toBe('0.60.483.0');
});
test('a branch an older restamp cut to three parts heals: the base\'s four-part manifest sets the form', () => {
const repo = fixture({ policy: GBRAIN_POLICY, version: '0.60.482.0' });
fourPartManifest(repo, '0.60.482.0');
makePr(repo, 'feat-a');
setPkgVersion(repo, '0.60.482');
repo.git('commit', '-q', '-am', 'old restamp cut the manifest');
const r = restamp(repo, ['--next']);
expect(r.status).toBe(EXIT.ok);
expect(pkgVersion(repo)).toBe('0.60.483.0');
});
test('a manifest listed through stamp_paths keeps four parts too', () => {
const repo = fixture({ policy: { ...GBRAIN_POLICY, stamp_files: [], stamp_paths: ['package.json'] }, version: '0.60.482.0' });
fourPartManifest(repo, '0.60.482.0');
makePr(repo, 'feat-a');
const r = restamp(repo, ['--next']);
expect(r.status).toBe(EXIT.ok);
expect(pkgVersion(repo)).toBe('0.60.483.0');
});
test('gstack-shaped repo: the three-part manifest and its lockfile still get the npm-valid form', () => {
const repo = fixture();
makePr(repo, 'feat-a');
const r = restamp(repo, ['--next']);
expect(r.status).toBe(EXIT.ok);
expect(r.out).toContain('stage version: done (wrote 1.2.4.0)');
expect(readFile(repo, 'VERSION').trim()).toBe('1.2.4.0');
expect(pkgVersion(repo)).toBe('1.2.4');
const lock = JSON.parse(readFile(repo, 'package-lock.json'));
expect([lock.version, lock.packages[''].version]).toEqual(['1.2.4', '1.2.4']);
});
test('a four-part manifest the policy does not list as a stamp file still migrates to the npm-valid form (decision 11)', () => {
const repo = fixture({ policy: { ...GBRAIN_POLICY, stamp_files: ['VERSION', 'CHANGELOG.md'] }, version: '0.60.482.0' });
fourPartManifest(repo, '0.60.482.0');
makePr(repo, 'feat-a');
const r = restamp(repo, ['--next']);
expect(r.status).toBe(EXIT.ok);
expect(r.out).toContain('stage version: done (wrote 0.60.483.0)');
expect(pkgVersion(repo)).toBe('0.60.483');
});
});
+15
View File
@@ -684,3 +684,18 @@ test('review suppressions allow a clean result and skip common false positives b
expectTokens(suppressions, ['"Function too long"', '"Possible null dereference"', '"N+1 query"', '"Hardcoded value"', '`Math.random()`'], 'suppressions');
expectAbsent(suppressions, [/missing input validation/i, /missing await/i], 'suppressions');
});
test('judged repeat-pass QA reruns and unresolved record actions each have one rule (paid screen 40617200857)', () => {
// Every sample of the red panel and its rerun named one of these: "post-expiry smoke rechecks are not-run" vs
// Step 5.8's rerun of superseded probes, materialize's once-per-PROBE_DIR limit vs reruns into the same report,
// and no record action for an unresolved non-gate defect (an unanswered question or an unproven repair).
// fails_when=the rerun stops being revalidation, materialize may run per pass, or such a defect has no action.
const flat = skill.replace(/\s+/g, ' ');
const repeat = flat.slice(flat.indexOf('2. On a repeat, execute Steps 3–5 in order.'), flat.indexOf('3. **Verify completed actions.**'));
expect(repeat).toContain('These reruns are revalidation, so they run even after smoke expiry; keep the PROBE_DIR. Earlier provisional sections link checkpoints; only the final pass runs its once-only `materialize`.');
expect(repeat.indexOf('rerun each probe the verdict relies on')).toBeLessThan(repeat.indexOf('These reruns are revalidation'));
const actions = flat.slice(flat.indexOf('Actions: `auto-fixed` (Step 5b)'), flat.indexOf('Advice is never `auto-fixed`'));
expect(actions).toContain('`fixed` (approved **and completed** in Step 5d)');
expect(actions).toContain('`skipped` (explicit Skip in Step 5c), `open` (any other defect, e.g. unanswered).');
expect(flat).toContain('Do not record an unanswered question as skipped');
});
+1
View File
@@ -792,6 +792,7 @@ describeIfSelected('Cookie setup workflow quality', ['setup-browser-cookies/SKIL
startMarker: '# Setup Browser Cookies',
endMarker: null,
...COOKIE_WORKFLOW_JUDGE,
schemaTransport: true,
readInput: () => buildCookieWorkflowJudgeInput(ROOT),
});
}, WORKFLOW_JUDGE_TEST_MS);
+2 -2
View File
@@ -410,7 +410,7 @@ test('Ship sends its authorized 64k cap and compact response contract through th
expect(WORKFLOW_JUDGE_RESPONSE_SCHEMA.properties.reasoning).not.toHaveProperty('maxLength');
expect(source.match(/compactReasoning: true/g)).toHaveLength(1);
const f = fixture();
const stream = spyOn(Messages.prototype, 'stream').mockReturnValue({ finalMessage: async () => ({
const stream = spyOn(Messages.prototype, 'stream').mockReturnValue({ on() { return this; }, finalMessage: async () => ({
stop_reason: 'end_turn', content: [{ type: 'text', text: JSON.stringify(scores) }],
}) } as any);
try {
@@ -516,7 +516,7 @@ test('the structured callback rejects incomplete, schema-invalid and below-thres
try {
for (const response of responses) {
const f = fixture();
stream.mockReturnValue({ finalMessage: async () => ({ stop_reason: response.stop_reason,
stream.mockReturnValue({ on() { return this; }, finalMessage: async () => ({ stop_reason: response.stop_reason,
content: [{ type: 'text', text: JSON.stringify(response.value) }] }) } as any);
const h = actualCallback(f, { judge: (prompt, model, request) => callJudge<typeof scores>(prompt, model, request) });
await expect(h.run({ ...h.options, schemaTransport: true, compactReasoning: true, maxTokens: 65_536, stream: true })).rejects.toThrow();