test: default-config workflow judges send their schema after calibration (W2 T8)

Comparison (1) on the 24-item workflow-default corpus (14 eval cases):
old-vs-old 2/23 flips, new-vs-old 3/23 (within old + 1), agreement 21/24
for old and new, zero false passes under the new prompt (old-a had a06),
p95 25.8 s new vs 29.5 s old; held-out 0/8 flips, 8/8 agreement. Item f03
(stubbed cookie-import workflow) drew one provider refusal under each
prompt, so its panels are incomplete and excluded from flips; that errored
new sample triggers the UC3 prose step, deferred to TODOS. The 14 cases
get schemaTransport without the compact-reasoning validator; review and
setup-browser-cookies stay on today's request. Funded by a $25 extension
of the W2 reservation. 216 calls, $19.66.

test/fixtures/judge-calibration/README.md

@@ -37,11 +37,15 @@ most 80% of `JUDGE_MS` (96 s).
 | `qa-anti-refusal` | 0/24 | 0/24 | 23 / 23 | 0 | 0 / 0 | 12.5 / 13.4 | 0/8, 8/8 | $7.02 | lands |
 | `cross-skill` | 0/24 | 0/24 | 23 / 23 | 0 | 0 / 0 | 19.7 / 22.6 | 0/8, 8/8 | $19.86 | lands |
 | `voice` | 0/24 | 0/24 | 22 / 22 | 0 | 0 / 0 | 13.3 / 13.7 | 0/8, 8/8 | $8.09 | lands |
+| `workflow-default` | 2/23 | 3/23 | 21 / 21 | 0 | 1 / 1 (refusal, f03) | 29.5 / 25.8 | 0/8, 8/8 | $19.66 | lands; UC3 prose step triggered by the errored sample |
 
-No sample errored (refusal, truncation, parse, schema or transport) under either prompt.
-`workflow-default` has a corpus but was not dispatched: its priced cost (about $20) exceeded
-what remained of the W2 reservation after the higher-priority configurations, so the shared
-workflow judge stays on today's request (only `ship` sends `WORKFLOW_JUDGE_RESPONSE_SCHEMA`,
-as before). The frontier (`review`) and cookie workflow configurations have no corpus yet.
-A $0.12 two-call pilot sized output tokens before the manifest was priced. Total W2 spend:
-$70.81 of $90.
+No sample errored under either prompt except one refusal each on `workflow-default` item
+`f03` (a stubbed cookie-import workflow): the old prompt refused once in `old-b` and the new
+prompt once, so that item's panels are incomplete and it is excluded from the flip counts
+(23 compared). The refusal is content-driven rather than format-driven, but under the eng
+trigger any errored new sample calls for a calibrated prose step, which is deferred to
+TODOS. `workflow-default` was funded after the first pass with a $25 extension of the W2
+reservation; its 14 cases now send `WORKFLOW_JUDGE_RESPONSE_SCHEMA` without the
+compact-reasoning validator (still only on `ship`). The frontier (`review`) and cookie workflow
+configurations have no corpus and stay on today's request. A $0.12 two-call pilot sized
+output tokens before the manifest was priced. Total W2 spend: $90.47 of $115.

test/fixtures/judge-calibration/workflow-default/runs/comparison-1.result.json

@@ -0,0 +1,93 @@
+{
+  "config": "workflow-default",
+  "model": "claude-fable-5-1",
+  "request_hash": {
+    "old": "e1a46ca4bfec06fb604cfeb71a71b7922c37c3f7647942c2a5aa503be7c5a35b",
+    "new": "d74c25c657358e50db2c6b181c5fbdf7cea3d17e77eb163e21616b39fd77b8e0"
+  },
+  "complete": true,
+  "stopped": null,
+  "calls": 216,
+  "cost_usd": 19.6554,
+  "errors": {
+    "old": {
+      "refusal": 1,
+      "truncation": 0,
+      "parse": 0,
+      "schema": 0,
+      "transport": 0,
+      "samples": 144,
+      "rate": 0.006944444444444444
+    },
+    "new": {
+      "refusal": 1,
+      "truncation": 0,
+      "parse": 0,
+      "schema": 0,
+      "transport": 0,
+      "samples": 72,
+      "rate": 0.013888888888888888
+    }
+  },
+  "latency_p95_ms": {
+    "old": 29479,
+    "new": 25782
+  },
+  "all": {
+    "items": 24,
+    "old_flips": {
+      "flips": 2,
+      "compared": 23
+    },
+    "new_flips": {
+      "flips": 3,
+      "compared": 23
+    },
+    "agreement": {
+      "old-a": 21,
+      "old-b": 20,
+      "new": 21
+    },
+    "false_passes": {
+      "old-a": [
+        "a06"
+      ],
+      "old-b": [],
+      "new": []
+    },
+    "new_false_passes": []
+  },
+  "heldout": {
+    "items": 8,
+    "old_flips": {
+      "flips": 0,
+      "compared": 8
+    },
+    "new_flips": {
+      "flips": 0,
+      "compared": 8
+    },
+    "agreement": {
+      "old-a": 8,
+      "old-b": 8,
+      "new": 8
+    },
+    "false_passes": {
+      "old-a": [],
+      "old-b": [],
+      "new": []
+    },
+    "new_false_passes": []
+  },
+  "landing": {
+    "lands": true,
+    "reasons": [],
+    "strict_no_flip_increase": false
+  },
+  "prose_step_triggered": {
+    "triggered": true,
+    "reasons": [
+      "new prompt has errored samples"
+    ]
+  }
+}

test/fixtures/judge-calibration/workflow-default/runs/comparison-1.samples.jsonl

@@ -0,0 +1,216 @@
+{"config":"workflow-default","item":"f03","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Section 1 is detailed but internally confused: it tells the agent to run the browse/SKILL.md probe and stop on READY (use Aside sessions), then embeds a different SETUP block where READY merely means the binary exists and the agent should continue to `$B status`, with another stop condition on `Mode: cdp`. The relationship between these two probes and their stop conditions is contradictory. Sections 2–4, which are the actual substance of the workflow (confirming import options, selecting source profile and domains with user consent, performing the import, reporting), are placeholders: 'TBD', 'Import whatever cookies seem relevant', 'Let the user know how it went'. There is no import command, no consent prompt, no profile/domain selection logic, no output/report format. An agent cannot execute the import or produce a defined deliverable from this bundle."},"latency_ms":10265,"stop_reason":"end_turn","usage":{"input_tokens":1630,"output_tokens":625},"cost_usd":0.04755}
+{"config":"workflow-default","item":"f01","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The remote flow (consent gate, ngrok install/auth paths, token-handling security, verbatim instruction-block output) is specific and executable, but the core workflow is broken. Step 3 is missing: Step 2 only asks which agent, yet Step 4 branches on 'same machine (option A)' vs 'different machine (option B)' that were never defined—and A/B collide with the relaunch/keep consent options. Worse, after offering 'Keep the live daemon (recommended)', the text says 'Always pass --force-restart to the commands below', directly contradicting the one-way-door consent and risking destroying tabs/logins; the commands shown don't even include the flag. Step 1's 'run Step 5 (Verify connection) first' before pairing is nonsensical. Verification criteria in Step 5 are vague."},"latency_ms":14017,"stop_reason":"end_turn","usage":{"input_tokens":4109,"output_tokens":968},"cost_usd":0.08949}
+{"config":"workflow-default","item":"f03","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Section 1 is internally contradictory: the prose says that a READY probe result means 'stop importing,' while the embedded SETUP block treats READY as the binary path to proceed with `$B status`, so the agent cannot tell when to continue. Step ordering is also muddled (run `./setup` before checking whether bun exists). Sections 2–4, which constitute the actual import workflow (confirming options, selecting source profile and domains with user consent, performing the import, reporting), are stubs: 'TBD', 'import whatever cookies seem relevant', 'let the user know how it went.' There is no import command, no consent prompt, no domain-selection rule, no success/failure criteria, and no report format. An agent can run the environment probe but cannot execute or report a cookie import from these instructions."},"latency_ms":15411,"stop_reason":"end_turn","usage":{"input_tokens":1630,"output_tokens":676},"cost_usd":0.0501}
+{"config":"workflow-default","item":"f01","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The workflow has structural breaks and direct contradictions. Step 3 is missing entirely (Step 2 jumps to Step 4), and Step 2 only asks which agent to pair with — it never asks local vs. remote. Yet Step 4 branches on 'If same machine (option A)' / 'If different machine (option B)', while the only A/B options actually defined in Step 4 are the live-daemon consent choices (Relaunch vs. Keep). So the agent has no defined input for the local/remote decision and the option labels collide with a different question. Worse, immediately after the carefully worded one-way-door consent gate (recommending option B: keep the live daemon, iron rule that only explicit --force-restart may kill it), the text says 'Always pass --force-restart to the commands below so pairing starts from a clean daemon' — a direct contradiction that would destroy user sessions, and none of the subsequent commands actually include the flag. Step 1's advice to 'run Step 5 (Verify connection) first and come back to pairing later' is nonsensical since there is nothing to verify before pairing. Step 5 also assumes the paste-into-other-agent flow even for local pairing. The remote branch itself (consent gate via gstack-config, ngrok detection, token-hygiene handling, verbatim instruction block output, scope options) is well specified, and the scope reference section is useful, which keeps this from being a 1. But the missing decision step, mislabeled branches, and the force-restart contradiction make the core flow ambiguous and risky to execute as written."},"latency_ms":15668,"stop_reason":"end_turn","usage":{"input_tokens":3712,"output_tokens":1211},"cost_usd":0.09767}
+{"config":"workflow-default","item":"f05","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Most steps (provenance guard, naming, synthesis template, fixture capture, test template, SDK resolution, staging, commit/discard, verify) are concrete and well-specified. However two central defects undermine the workflow: (1) Step 8 instructs the agent to proceed to commit even when the staged test fails ('a failing test can be fixed after the skill is committed'), which directly contradicts the Iron Contract that requires test pass + approval before writing and discards the temp dir on failure — an agent cannot resolve which rule governs. (2) Step 9 — the explicit user-approval gate — is entirely absent; Step 8 jumps to Step 10, yet Step 10 branches on 'if the user approved' / 'if the user rejected in step 9' without ever defining what is shown, how approval is asked, or what the decision options are. Minor inconsistency also: Step 3 says the SDK copy is 'written in step 6' when it is read in 6 and written in 7. The missing approval decision and contradictory test-failure handling leave the core gate of the workflow undefined."},"latency_ms":16243,"stop_reason":"end_turn","usage":{"input_tokens":5269,"output_tokens":1116},"cost_usd":0.10849}
+{"config":"workflow-default","item":"a01","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The workflow is well-structured with a clear Iron Law, five ordered phases, explicit fixture paths, and a failure-mode table that covers the main escalation paths (3-iteration cap, schema mismatch, disconnect, build failure). Most steps map to concrete endpoints or commands. Weaknesses: Phase 3 steps 3-4 are passive/vague ('Daemon detects the rebuild', 'Re-deploy. The same boot-token rotation flow runs') with no explicit agent action or verification; Phase 4's screenshot comparison doesn't say how to judge equivalence (pixel diff vs. agent inspection) or what to do if the bug is non-visual; there is no failure mode for 'cannot reproduce the bug in Phase 1'; bug-slug derivation and the exact assertion the regression test should make are left to the agent; and 'revert and try again' doesn't specify whether iterations should use a new hypothesis. These gaps are moderate rather than blocking, so the agent can likely produce the deliverables but with some judgment calls."},"latency_ms":17304,"stop_reason":"end_turn","usage":{"input_tokens":1716,"output_tokens":1102},"cost_usd":0.07226}
+{"config":"workflow-default","item":"a03","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"The bundle jumps from Step 2 to Step 4 with no Step 3; the local-vs-remote decision that Step 4 branches on ('If same machine (option A)' / 'If different machine (option B)') is never asked anywhere, and those A/B labels collide with Step 2's A/B (OpenClaw/Codex) and the daemon-consent A/B (relaunch/keep), creating real ambiguity about which answer drives which branch. The verification step promised by the workflow description is absent entirely. The live-daemon consent gate is well-motivated but under-specified: it says 'pairing headed requires relaunching' without ever defining when pairing is headed or which commands would need --force-restart, so an agent cannot tell when the gate actually applies. What is present is strong: ngrok detection/auth/install branches, the authtoken security rule, the verbatim-output requirement for the instruction block, the gstack-config consent gate, and the scope/--restrict/--control reference are concrete and executable. Net: individual branches are actionable, but the missing mode decision and missing verification leave the end-to-end flow incomplete."},"latency_ms":18555,"stop_reason":"end_turn","usage":{"input_tokens":3750,"output_tokens":894},"cost_usd":0.0822}
+{"config":"workflow-default","item":"p04","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The five steps are sequential and mostly unambiguous, with concrete commands, an exact render template, and an ordered decision list for the single suggestion. Gaps undermine completeness: the dashboard requires fields never sourced—`Repo: <owner/repo>` has no retrieval command, and the offline block cites `<offline reason from warnings>` though `.warnings` is not in the jq extraction list and the `{\"offline\":true}` fallback carries no reason. Step 2's `cat VERSION | tr ... || echo` fallback is broken (the `||` tests `tr`, so a missing file yields empty rather than 0.0.0.0). Sibling columns (last commit, PR) and collision detection are implied rather than specified. Still executable with modest inference."},"latency_ms":15000,"stop_reason":"end_turn","usage":{"input_tokens":2860,"output_tokens":881},"cost_usd":0.07265}
+{"config":"workflow-default","item":"f04","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Steps 0, 1, 7 and 9 are concrete, but the core of the workflow is hollow: Steps 2-6 collapse to 'write whatever documentation seems most useful, in whatever structure fits,' with no quadrant partitioning, templates, or per-quadrant guidance, yet Step 7 and the Important Rules assume quadrants were deliberately produced. Ordering is contradictory: Step 8's quality review is declared to occur after Step 9 has committed and pushed, is 'optional,' and failures are deferred to a later PR, while Step 9's summary requires pass/fail per gate before the review exists. The redaction scan and commit mechanics are precise, but an agent cannot reliably produce Diataxis deliverables from these instructions."},"latency_ms":9271,"stop_reason":"end_turn","usage":{"input_tokens":3892,"output_tokens":614},"cost_usd":0.06962}
+{"config":"workflow-default","item":"p01","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is linear and well-structured: binary lookup, prompt selection with three branches, provider discovery via dry-run, judge availability check, command construction, interpretation, and optional save. Each step has concrete commands and decision options with explicit recommendations. Minor gaps reduce the score: Step 1A's `find .` assumes an unspecified working directory; Step 6 doesn't define the `<date>` format or how to derive the slug, nor does it mention creating `~/.gstack/benchmarks/`; and saving requires a full re-run with `--output json` (doubling time/cost) without warning the user — capturing JSON once and rendering a table would be cleaner. The Step 3 shell one-liner relies on `||`/`&&` precedence that happens to work but is fragile. Otherwise the agent can execute this end-to-end and produce the expected deliverables."},"latency_ms":14289,"stop_reason":"end_turn","usage":{"input_tokens":2687,"output_tokens":922},"cost_usd":0.07297}
+{"config":"workflow-default","item":"p04","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The five steps are sequential, concrete, and mostly scriptable: base-branch detection, VERSION reads, per-level queue queries, a fixed-format dashboard, and a three-way next-action decision. Gaps weaken completeness: the render step lists five jq fields but the template needs data not sourced anywhere (Repo owner/repo, <workspace_root>, the 'Reason' from `.warnings`, per-level annotations like 'bumped past claimed', and the sibling row fields path/branch/VERSION/last-commit/PR whose schema is unstated). Collision detection within `.claimed` and the 'active' comparison against the user's branch are implied rather than specified, Step 5 doesn't state priority when multiple conditions hold, and the partial-offline case (some levels fail) is unaddressed. The `cat … | tr … || echo` fallbacks also won't fire as intended because the pipeline's exit status comes from `tr`. Still, an agent could produce a recognizable dashboard and a sensible recommendation from this."},"latency_ms":18636,"stop_reason":"end_turn","usage":{"input_tokens":2463,"output_tokens":1216},"cost_usd":0.08543}
+{"config":"workflow-default","item":"p02","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: mode detection, a six-part save flow (gather git state, summarize, duration, bash-computed collision-safe path with allowlist sanitizer, stamp, confirmation block), and a list flow with explicit table formats, --all variant, and empty-state message. Instructions are mostly unambiguous and directly executable. Minor flaws: Step 4 and List Step 1 contain redundant/duplicated preamble lines (GSTACK_STATE_ROOT computed twice, mkdir repeated); the collision comment references `${SLUG}` while the code uses `${TITLE_SLUG}`; the duration fallback uses BSD-only `date -jf` so Linux silently yields 'unknown' (acceptable because omission is specified); no bash snippet is given for producing the ISO-8601 timestamp with offset; `--all` is introduced only in the list display step rather than in command detection; and the list filter is by branch only, not by the project `remote:`/`project_root:` stamp despite noting multiple projects may share a checkpoints directory. None of these block execution, but they introduce small ambiguities and leave a few decisions to the agent."},"latency_ms":20033,"stop_reason":"end_turn","usage":{"input_tokens":4636,"output_tokens":1324},"cost_usd":0.11256}
+{"config":"workflow-default","item":"p05","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is concrete and well-sequenced: fixed state paths, a mechanical shortlist script with labeled output tags, an explicit retirement-card schema, a precise caller-check grep, a defined report outline plus JSON sidecar schema, a one-question-per-batch AskUserQuestion with fixed options and recommendation rule, and a gated apply step (scratch-worktree build check, runner + diff --check, numstat). Headless behavior and the 8-minute budget/resumable partial report are both specified. Weaknesses: Step 1 duplicates the GSTACK_STATE_ROOT line and references an undefined `$BRANCH` in the SEED_PLAN glob (DEFAULT_BRANCH is what's computed); the Step 1 test-file regex includes Java/Kotlin while Step 2's omits them, so TESTFILES and the shortlist disagree; `--since <ref>` is described but no command shows how to restrict FILES by changed-since; the per-candidate read requirement (test + owner + callers + overlapping tests) sits in tension with the `max-candidates × 3` file budget; the report asks for per-batch LOC while the sidecar's `loc_delta` is a single total; and Step 5 iterates a question over every batch while Step 6 implies apply-one-then-stop-until-/ship-lands, leaving the multi-approval flow underspecified. The large shared 'Test value bar'/'Regression proof' block is mostly aimed at new tests and adds noise to an audit of existing ones. None of these block execution, but they require the agent to resolve small inconsistencies on its own."},"latency_ms":36311,"stop_reason":"end_turn","usage":{"input_tokens":5101,"output_tokens":2654},"cost_usd":0.18371}
+{"config":"workflow-default","item":"p03","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Routing is explicit and each subcommand has a concrete command block plus presentation guidance, so an agent can execute most paths directly. Gaps undermine completeness: Prune tells the agent to edit learnings.jsonl but never resolves its path (only Stats computes GSTACK_STATE_ROOT/LEARN_FILE), and it simultaneously says to delete lines and that the store is 'append-only, latest entry wins', so the contradiction check will flag every legitimate update as a CONFLICT. Export's template omits the 'tool' type that Manual add permits, leaving those entries unformatted. SLUG is computed but unused in four blocks, and the search tool's output shape (needed for 'grouped by type') is assumed. Solid but with unresolved decisions in Prune."},"latency_ms":13193,"stop_reason":"end_turn","usage":{"input_tokens":3202,"output_tokens":787},"cost_usd":0.07137}
+{"config":"workflow-default","item":"a06","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":3,"reasoning":"Detection script, multi-platform disambiguation rule, per-platform branches, verification, and confirm-before-write gate are clear and sequenced. Gaps: Step 4 says \"Write the confirmed configuration into CLAUDE.md\" but never defines the section name, key names, or format, so the core deliverable's structure is left to the agent (and Custom Q4's post-merge hooks aren't in the write list). Several branches are under-specified: Render checks for RENDER_API_KEY but never uses it and defines no status command; Fly's \"/health if the app has one\" and \"merge method constrained to methods allowed by repo settings\" give no procedure to determine them. Workflow regex matching \"cd\" will over-detect. Executable but with judgment calls on the output itself."},"latency_ms":22281,"stop_reason":"end_turn","usage":{"input_tokens":3227,"output_tokens":1578},"cost_usd":0.11117}
+{"config":"workflow-default","item":"f05","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Steps 1–7 are detailed and well-specified (provenance guard, name/tier question, pure-parser template, staging via stageSkill). But the tail breaks the workflow. Step 8 says a failing test should 'continue to Step 10 anyway; a failing test can be fixed after the skill is committed,' directly contradicting the Iron contract that commits only on test pass and otherwise removes the temp dir. Step 9—the explicit user-approval gate that Step 10 depends on ('If the user approved', 'If the user rejected in step 9')—is absent entirely, so the agent has no defined approval question or decision criteria. Minor: Step 6's 'two dirname calls' arithmetic is wrong. The core gate is thus ambiguous and contradictory, undermining reliable execution."},"latency_ms":13540,"stop_reason":"end_turn","usage":{"input_tokens":5666,"output_tokens":892},"cost_usd":0.10126}
+{"config":"workflow-default","item":"f03","split":"dev","phase":"old-b","sample":0,"status":"error","error_kind":"refusal","error":"Judge provider refused the evaluation; no automated score","latency_ms":5475,"stop_reason":"refusal","usage":{"input_tokens":1630,"output_tokens":199},"cost_usd":0.02625}
+{"config":"workflow-default","item":"f04","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 0, 1, 7 and 9 are concrete, but the core of the workflow is missing: Steps 2-6 collapse to 'write whatever documentation seems most useful... in whatever structure fits,' with no partitioning rule, templates, or decision criteria for tutorial/how-to/reference/explanation, even though Step 7, Step 9's PR table and the Important Rules all assume quadrant-partitioned output. Ordering is also contradictory: Step 8 says self-review runs after Step 9 has committed and pushed, is optional, and failures 'can be fixed in a later PR,' yet Step 9's summary reports gate pass/fail and the rules call accuracy non-negotiable. An agent cannot reliably produce the expected Diataxis deliverables from this."},"latency_ms":9174,"stop_reason":"end_turn","usage":{"input_tokens":3892,"output_tokens":512},"cost_usd":0.06452}
+{"config":"workflow-default","item":"a01","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The workflow is well-structured with a clear Iron Law, numbered phases, explicit fixture paths, an iteration cap, and a failure-mode table. However, several steps are underspecified: Phase 3 steps 3-4 are passive/ambiguous ('Daemon detects...', 'Re-deploy' after `build install` already installed), Phase 4's screenshot comparison method is undefined (how is 'visibly persists' judged?), 'revert and try again' doesn't say whether to return to Phase 2 for a new hypothesis, the Phase 1 one-line description has no stated destination, and Phase 5 doesn't define how the test asserts post-fix behavior or detects device absence. Bug-slug derivation and scheme selection are left implicit. The device-disconnect row ('resume from Phase 4') is odd if disconnect happens mid-Phase 3. Still, an agent could execute this end-to-end with modest inference."},"latency_ms":19989,"stop_reason":"end_turn","usage":{"input_tokens":1716,"output_tokens":1202},"cost_usd":0.07726}
+{"config":"workflow-default","item":"f04","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 0, 1, 7, and 9 are concrete and well-specified (scoping prompt, archaeology commands, concept map, cross-linking rules, redaction scan, commit template, summary format). However, the core of the workflow is gutted: Steps 2-6, which should define quadrant partitioning, templates, and per-quadrant writing instructions, are replaced with 'write whatever documentation seems most useful... in whatever structure fits.' Downstream steps then assume quadrants exist (cross-link 'between quadrants', PR table by quadrant, summary coverage by quadrant, 'do not mix tutorial into reference'), so the agent has no defined way to produce the expected deliverables. Step 8 is internally contradictory: it says review happens after Step 9 commits and pushes, is 'optional,' and failures can be deferred to a later PR, while the Important Rules declare accuracy non-negotiable and Step 9's summary requires pass/fail per gate before output. These ordering and decision gaps make the workflow unreliable despite the strong peripheral steps."},"latency_ms":11393,"stop_reason":"end_turn","usage":{"input_tokens":3495,"output_tokens":859},"cost_usd":0.0779}
+{"config":"workflow-default","item":"p05","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":3,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: clear read-only boundaries, a concrete mechanical pre-filter, a fully enumerated retirement card with an exact caller-check grep, explicit verdicts, a defined report structure with a JSON sidecar schema, a per-batch AskUserQuestion with fixed options, and a gated apply step. However, several issues reduce clarity and execution confidence: the Step 1 script duplicates the GSTACK_STATE_ROOT line and references an undefined `$BRANCH` in the SEED_PLAN glob (only DEFAULT_BRANCH is set here), so seed discovery may silently never fire; the Step 1 test-file regex includes Java/Kotlin but Step 2's does not; `--since <ref>` is described but no command is given for restricting FILES to changed test files; the 'max-candidates × 3 files' read budget conflicts with the instruction to read each test plus its owner, callers, and overlapping tests; the sidecar verdict enum has no representation for 'not ready/incomplete card' candidates or the resumable partial-report state; and the value-card/regression-proof block is oriented to writing new tests and reads as noise in a retirement audit, leaving it unclear whether the agent must produce regression proofs here. The 8-minute budget is also not measurable by the agent. Overall executable with reasonable deliverables, but an agent will need to resolve several ambiguities and script bugs on its own."},"latency_ms":30243,"stop_reason":"end_turn","usage":{"input_tokens":5101,"output_tokens":2263},"cost_usd":0.16416}
+{"config":"workflow-default","item":"a06","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Steps 2–5 give concrete, runnable detection and verification commands, a clear multi-platform disambiguation rule, and a sensible non-deploying fallback. Gaps remain: Step 4 says 'write the confirmed configuration into CLAUDE.md' without specifying the section format, field names, or how to handle an existing deploy block, so the deliverable's shape is undefined. Minor ambiguities also exist, e.g., Fly's health check 'or /health if the app has one' with no way to determine that, and Step 3's post-merge hook question is never persisted in Step 4's field list. Step 6 is a one-line placeholder. Otherwise an agent could execute the flow and reach confirmation reliably."},"latency_ms":16319,"stop_reason":"end_turn","usage":{"input_tokens":3227,"output_tokens":1098},"cost_usd":0.08717}
+{"config":"workflow-default","item":"f01","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The workflow has a missing decision step and self-contradictory consent logic. Step 3 (local-vs-remote choice) is absent entirely, yet Step 4 branches on 'same machine (option A)' / 'different machine (option B)' — labels that collide with the just-defined A) Relaunch / B) Keep-daemon consent options, so an agent cannot tell which 'option B' governs. Worse, after presenting a one-way-door consent gate that recommends keeping the live daemon, the text says 'Always pass --force-restart to the commands below so pairing starts from a clean daemon,' directly nullifying the consent and the stated iron rule (and the commands shown omit the flag). Step 1's advice to run Step 5 verification first before pairing is nonsensical. The ngrok, tunnel-consent, and output-block sections are otherwise concrete."},"latency_ms":10179,"stop_reason":"end_turn","usage":{"input_tokens":4109,"output_tokens":670},"cost_usd":0.07459}
+{"config":"workflow-default","item":"f02","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 1-2 (detection and honest capture) are detailed and mostly unambiguous, with good rules for exit codes, SKIPPED vs ERROR, and capture failures. But the remaining four steps are one-line stubs: Step 3 gives no scoring rubric, weights, thresholds, or composite formula (despite Step 2 referencing 'that category's weight' and a 'numeric history row'); Step 4 specifies no dashboard format; Step 5 names no history file path or schema; Step 6 provides no trend comparison logic or recommendation rules. These are the skill's core deliverables, not external tools, so their absence leaves the agent unable to produce a consistent dashboard, composite score, or history. Minor issues: shell precedence bugs in detection (`[ a ] || [ b ] && echo`), zsh-only `setopt`, the GBRAIN dimension has no defined scoring, and the AskUserQuestion text is a hardcoded example rather than a template."},"latency_ms":13941,"stop_reason":"end_turn","usage":{"input_tokens":2948,"output_tokens":984},"cost_usd":0.07868}
+{"config":"workflow-default","item":"a01","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":3,"reasoning":"The five-phase structure and Iron Law are clear, but key decisions are underspecified. Phase 4 says to compare a fresh screenshot against the pre-bug PNG and judge whether the bug 'visibly persists' with no criterion, and 'revert and try again' doesn't say whether to loop back to Phase 2 or Phase 3. Phase 3 step 4 ('Re-deploy. The same boot-token rotation flow runs') is redundant with step 2's build install and references an undefined flow. Other gaps: Phase 1 step 5 gives no location for the persisted description; the 409 schema_mismatch remedy ('re-snapshot') after a fixed build contradicts the Iron Law since the fixed build can't reproduce the bug; 'resume from Phase 4' ignores disconnects during Phases 1-3; the regression test's assertion mechanism is unspecified."},"latency_ms":14120,"stop_reason":"end_turn","usage":{"input_tokens":2113,"output_tokens":1087},"cost_usd":0.07548}
+{"config":"workflow-default","item":"a04","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":4,"reasoning":"Steps 1–3 are concrete and the aside repl scripts are copy-pasteable with explicit quoting rules, so an agent can realistically run the look→extract loop or the fuzzy aside-exec path. However, the 'handle failed reads' and 'return one JSON document' parts of the workflow are underspecified: there is no explicit final-output step defining what the agent hands back (JSON_START/JSON_END for structured vs {answer, sources} for fuzzy, with no unified shape), no instruction for what to do when the three-or-four-attempt budget is exhausted or when a repl call errors/times out, and the sign-in-wall handling is the only failure mode fully addressed. Clarity is hurt by dangling cross-references: the untrusted-content block lists rules 1–4, yet the text cites 'Rule 5' for untrusted replies and 'Rule 4' for sign-in walls (which doesn't match rule 4 as listed), and '$B handoff' appears without context. The Capture Learnings section is tangential to the core flow."},"latency_ms":13982,"stop_reason":"end_turn","usage":{"input_tokens":3219,"output_tokens":941},"cost_usd":0.07924}
+{"config":"workflow-default","item":"f03","split":"dev","phase":"new","sample":0,"status":"error","error_kind":"refusal","error":"Judge provider refused the evaluation; no automated score","latency_ms":1663,"stop_reason":"refusal","usage":{"input_tokens":2027,"output_tokens":34},"cost_usd":0.02197}
+{"config":"workflow-default","item":"p03","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Routing is unambiguous and each subcommand has a concrete bash block plus a presentation instruction, so an agent can execute most paths directly. Weaknesses: Prune never defines the learnings.jsonl path (only inferable from the Stats block), and its 'same key, different insight = CONFLICT' rule clashes with the append-only/latest-wins semantics used elsewhere (a legitimate update would be flagged as a conflict); 'remove the line and write back' also sits awkwardly with 'append-only'. Export's template omits the 'tool' type that Manual add allows, uses an arbitrary --limit 50 for a full export, and leaves the 'separate file' destination undefined. Manual add always emits a files array even when files are optional, and USER_QUERY/INSIGHT substitution has no quoting guidance. No handling for unrecognized subcommands or for the tool returning empty output with exit 0. Overall a solid, mostly executable workflow with a few missing decisions and minor internal inconsistencies."},"latency_ms":24004,"stop_reason":"end_turn","usage":{"input_tokens":2805,"output_tokens":1009},"cost_usd":0.0785}
+{"config":"workflow-default","item":"p02","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The skill is well-structured: mode detection, a linear save flow (gather git state → summarize → duration → bash-computed collision-safe path → write file → stamp → confirm), and a list flow with explicit table formats and empty-state message. The bash blocks are concrete and the file template is fully specified, including security rationale for bash-side title sanitization. Minor issues reduce the scores: redundant duplicated GSTACK_STATE_ROOT/SLUG lines in several blocks; the duration script relies on macOS-only `date -jf` and an undefined `_TEL_START` (fallback behavior is covered, so not fatal); how TITLE_RAW is actually supplied when the agent runs the block is only hinted at via a comment; `files_modified` says 'staged and unstaged modified' leaving untracked files ambiguous; the ISO-8601 timestamp isn't computed in bash despite the emphasis on computing paths there; `status` is always written as `in-progress` yet the list example shows `completed` with no explanation of how that transition happens; and the list flow filters by branch via frontmatter reads but doesn't address the 'several projects share one checkpoints directory' case that the stamp step motivates, nor does the command-detection section mention `--all`. None of these block execution, but they introduce small gaps and ambiguities."},"latency_ms":20081,"stop_reason":"end_turn","usage":{"input_tokens":4636,"output_tokens":1237},"cost_usd":0.10821}
+{"config":"workflow-default","item":"a03","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":3,"reasoning":"The bundle jumps from Step 2 to Step 4, and the local-vs-remote decision that Step 4 depends on is never asked. Step 4 then branches on \"same machine (option A)\" / \"different machine (option B)\", but the only A/B options defined are agent types (OpenClaw/Codex) and the relaunch/keep-daemon consent, so the letters collide and an agent cannot tell which answer drives the branch. No verification step exists despite the stated goal; success is just a message to the user. Minor gaps: what to pass as TARGET_HOST for generic/Hermes in --local mode, and when pairing is 'headed' enough to need --force-restart. Commands, consent gates, and ngrok handling are concrete, which keeps actionability middling."},"latency_ms":11180,"stop_reason":"end_turn","usage":{"input_tokens":4147,"output_tokens":688},"cost_usd":0.07587}
+{"config":"workflow-default","item":"a05","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":3,"reasoning":"The capture scripts, cadence rules (max(0,60-elapsed), no overlapping rounds, stop at deadline), manifest schema, report format, status derivation, and history logging are concrete and followable. However, the core detection logic is essentially undefined: Phase 5 says only 'raise an alert at whatever severity seems right if a page looks noticeably worse' — no thresholds for load-time regression, no rule for new console messages, text disappearance, screenshot diffs, or new 404s, and no mapping to CRITICAL/HIGH/MEDIUM. 'Serious alert' triggering the AskUserQuestion is undefined. Phase 6 then relies on 'confirmed' vs 'unconfirmed transient' alerts (confirmed_at, observations array) without Phase 5 ever defining confirmation (e.g., persists across N checks, user choice). Minor gaps: no rule for deriving <page-name> from a URL, and 'resolved' semantics are unspecified. An agent can run the loop and emit the report, but the alert/verdict decisions will be subjective and inconsistent."},"latency_ms":17812,"stop_reason":"end_turn","usage":{"input_tokens":4530,"output_tokens":1193},"cost_usd":0.10495}
+{"config":"workflow-default","item":"p05","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: hard boundaries, a mechanical pre-filter with concrete grep commands, an explicit retirement card schema, a defined sidecar JSON, one AskUserQuestion per owner batch with A/B/C options and a recommendation rule, and scratch-worktree gating for production removal. Gaps are moderate. Example one: `--since <ref>` is declared as the default for >300 test files, but Step 2's `FILES=$(git ls-files -- <scope> ...)` never shows how to restrict to files changed since the ref, and Step 1 references an undefined `$BRANCH` for the seed plan while defining only `DEFAULT_BRANCH`. Example two: `retain` appears both as a `candidates[].verdict` and as a separate `retained[]` array, leaving it ambiguous where retained tests belong and whether rewrite/extend verdicts need full cards for Step 5 batching. The regression-proof block is irrelevant noise for a retirement sweep."},"latency_ms":25782,"stop_reason":"end_turn","usage":{"input_tokens":5498,"output_tokens":1904},"cost_usd":0.15018}
+{"config":"workflow-default","item":"p04","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":3,"reasoning":"The workflow is a linear five-step sequence with concrete bash and a fixed output template, so the overall shape is easy to follow. However, several rendering inputs are undefined: Step 4 lists only five jq fields but the dashboard needs owner/repo, <workspace_root>, sibling path/branch/VERSION/last-commit/PR shape, the per-level 'queue-advance'/'bumped past' annotations, and the offline 'Reason: <offline reason from warnings>' (`.warnings` is never extracted and the fallback JSON `{\"offline\":true}` has no warnings at all). Collision detection and 'last commit < 24h' computation are implied rather than specified. The 'unknown host' case is lumped with OFFLINE output even if the query succeeded, and Step 5 gives no next-action for the offline case nor explicit precedence when both collisions and active siblings exist. Step 2's `cat … | tr … || echo` fallbacks never fire because `tr` masks the failure, leaving empty versions. Step 1's heading promises platform detection but only the base branch is detected. These gaps mean an agent can run the queries but must invent data sources and logic to fill the dashboard."},"latency_ms":17571,"stop_reason":"end_turn","usage":{"input_tokens":2463,"output_tokens":1231},"cost_usd":0.08618}
+{"config":"workflow-default","item":"p05","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is concrete and mostly unambiguous: read-only discovery, mechanical shortlist commands, explicit retirement-card fields, defined verdicts, report+sidecar schema, one AskUserQuestion per batch, and gated application with scratch-worktree proof for production deletions. Weaknesses: the Step 1 and Step 2 test-file regexes disagree (Java/Kotlin only in Step 1); `--since` scoping is described but Step 2's FILES command never applies it; `$BRANCH` appears without local definition (likely preamble) and a GSTACK_STATE_ROOT line is duplicated; the sidecar puts `retain` both as a candidate verdict and in a separate `retained` array and reports only aggregate loc_delta while the report demands per-batch figures; Step 5 iterates over all batches while Step 6 implies one batch per run with re-discovery after landing, leaving the ask/apply sequencing ambiguous; the regression-proof block is irrelevant to an audit of existing tests and adds noise. No fallback when DEFAULT_BRANCH is unknown in the >300-file case. Despite these, an agent can execute it and produce the expected report, sidecar, and gated edits."},"latency_ms":27663,"stop_reason":"end_turn","usage":{"input_tokens":5101,"output_tokens":1996},"cost_usd":0.15081}
+{"config":"workflow-default","item":"p03","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Command routing is unambiguous and each subcommand has a concrete bash block plus a presentation instruction, so an agent can execute most paths directly. Weaknesses: Prune tells the agent to edit learnings.jsonl for removals but never gives the file path in that section (only Stats derives LEARN_FILE), and its 'same key, different insight = CONFLICT' rule collides with the append-only 'latest entry wins' update model, so legitimate updates will be flagged as conflicts with no guidance. Export's markdown template omits the 'tool' type that Manual add permits. No handling for unknown subcommands, `/learn search` with no query, optional files in Manual add (empty array vs omitted), or what to do after the user picks append-to-CLAUDE.md vs separate file. SLUG is computed but unused in several blocks (harmless but noisy)."},"latency_ms":13792,"stop_reason":"end_turn","usage":{"input_tokens":2805,"output_tokens":869},"cost_usd":0.0715}
+{"config":"workflow-default","item":"a04","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":4,"reasoning":"Steps 1–3 are concrete and the two extraction scripts are runnable with explicit quoting rules, JSON_START/JSON_END markers, and a retry budget, so an agent can execute the core flow. However, several gaps weaken it: there is no explicit final step defining how the 'one JSON document' is returned to the user (shape, whether structured and fuzzy outputs share a format, what to emit when the 3–4 attempt budget is exhausted or waitForSelector times out). 'Handle failed reads' is only partially covered via CONSOLE_ERRORS and the sign-in wall note. Internal rule references are inconsistent — the untrusted-content block lists rules 1–4, yet the text cites 'Rule 5' for the fuzzy reply and 'Rule 4' for the sign-in wall, where Rule 4 in-bundle is about prompt injection, not sign-in. Minor undefined tokens ($B handoff) and the lack of a terminal 'return result' step leave the deliverable under-specified."},"latency_ms":16503,"stop_reason":"end_turn","usage":{"input_tokens":3219,"output_tokens":1069},"cost_usd":0.08564}
+{"config":"workflow-default","item":"f05","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":2,"reasoning":"The individual steps (provenance guard, name/tier question, pure-parser script template, fixture capture, test template, SDK resolution, stageSkill/commitSkill/discardStaged calls, post-commit verification) are concrete and well-specified. However, the workflow has two critical defects at its core gate. First, Step 9 — the explicit user-approval step — is entirely absent: Step 8 jumps to Step 10, and Step 10 says 'If the user approved' / 'If the user rejected in step 9' with no definition of what question is asked, what the approval shows the user (test output, staged files), or what counts as approval. Second, Step 8 directly contradicts the 'Iron contract' (commit only on test pass + approval, discard on either failure) by instructing the agent to proceed to commit even when the test fails because it 'can be fixed after'. An agent following the literal instructions could write a half-broken skill to disk, exactly what the contract forbids. Minor issues: 'written in step 6' for _lib (actually staged in step 7), vague 'three turns past' heuristic, and no guidance on where the SKILL.md description text originates. The missing decision point and contradictory ordering make the deliverable gate undefined and unsafe to execute as written."},"latency_ms":16321,"stop_reason":"end_turn","usage":{"input_tokens":5269,"output_tokens":1089},"cost_usd":0.10714}
+{"config":"workflow-default","item":"f04","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 0, 1, 7, and 9 are reasonably concrete (scope options, research checklist with concept-map template, cross-linking rules, commit/PR/summary formats). But the core of the workflow is gutted: Steps 2-6 are collapsed into 'write whatever documentation seems most useful... in whatever structure fits', with no Diataxis partitioning logic, no per-quadrant templates, no file naming conventions, and no guidance on what goes into tutorial vs how-to vs reference vs explanation. Yet Step 7, Step 8, the commit message, the PR table, and the output summary all assume quadrant-structured output, so the agent has no defined path to the deliverables it is later asked to report on. Step 8 is self-contradictory: it says the review 'happens after Step 9 has committed and pushed', is 'optional', and failures 'can be fixed in a later PR', which directly conflicts with the Important Rules ('Accuracy is non-negotiable', 'Research before writing') and with Step 9's requirement to report pass/fail on each gate in the summary. The ordering of review-after-push also defeats the gates' purpose. Missing decisions: how to handle inline-vs-standalone option C concretely, how to map research output to doc files, and when/whether to block on gate failures. An agent could execute the periphery but would have to improvise the central content-generation step."},"latency_ms":12595,"stop_reason":"end_turn","usage":{"input_tokens":3495,"output_tokens":909},"cost_usd":0.0804}
+{"config":"workflow-default","item":"f05","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":2,"reasoning":"Steps 1–8 and 10–11 are mostly precise (name rules, script/test templates, stageSkill/commitSkill calls). But two decisive defects break the gate the workflow exists to enforce. First, Step 8 says 'If the test fails, continue to Step 10 anyway; a failing test can be fixed after the skill is committed,' directly contradicting the Iron contract that commits only on test pass and discards otherwise. Second, Step 9 — the explicit user-approval question — is entirely missing; Step 10 branches on 'If the user approved' / 'rejected in step 9' with no defined prompt, options, or decision ID. An agent following this literally would commit untested, unapproved skills or stall at an undefined decision point."},"latency_ms":10820,"stop_reason":"end_turn","usage":{"input_tokens":5666,"output_tokens":643},"cost_usd":0.08881}
+{"config":"workflow-default","item":"a04","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":4,"reasoning":"Steps 1–3 are concrete: intent capture, refusal wording, and runnable look/extract and fuzzy-agent scripts with quoting rules and a retry budget. But the workflow never defines the final deliverable step — the preamble promises 'one JSON document', yet there is no instruction on what to hand back after JSON_START/JSON_END or the agent reply, nor a unified shape between {items,count} and {answer,sources}. Failure handling is partial: the sign-in wall and CONSOLE_ERRORS cases are covered, but nothing says what to do when the three-or-four-attempt budget is exhausted or GSTACK_STEP_OK never prints. Cross-references are inconsistent ('Rule 5' and 'Rule 4' point to a four-rule list whose Rule 4 is about prompt injection, not sign-in), which muddles an otherwise followable flow."},"latency_ms":16126,"stop_reason":"end_turn","usage":{"input_tokens":3616,"output_tokens":999},"cost_usd":0.08611}
+{"config":"workflow-default","item":"f03","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Section 1 is detailed but internally confusing: it tells the agent to run the browse probe and stop on READY, then inlines a separate SETUP block whose READY means something different (binary exists), then adds a third stop condition (`Mode: cdp`), with an undefined reference to 'Aside's sessions'. The actual cookie-import workflow is essentially absent: Section 2 ('Confirm options') is literally 'TBD', Section 3 gives no profile selection, domain selection, consent prompt, or import command—just 'import whatever seems relevant'—and Section 4 has no report format or outcome criteria. The agent cannot determine how to select a source profile, obtain user consent, perform the import, or report results, so the stated deliverables are not achievable from these instructions."},"latency_ms":9566,"stop_reason":"end_turn","usage":{"input_tokens":1630,"output_tokens":505},"cost_usd":0.04155}
+{"config":"workflow-default","item":"a03","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"The bundle jumps from Step 2 to Step 4, omitting the step that decides local vs. remote mode; Step 4 then branches on 'option A (same machine)' and 'option B (different machine)', but those labels collide with Step 2's A–E agent choices and the live-daemon consent's A/B, so the agent has no defined decision point and must guess what the user chose. There is no verification step after pairing (the task calls for verifying the connection), no guidance on what to pass as TARGET_HOST/--client for the 'generic' choice, and the live-daemon consent gate is confusing — it says pairing 'can relaunch' the daemon and that 'headed' pairing requires it, yet none of the subsequent commands indicate when headed mode is triggered or how to pass --force-restart. On the plus side, the remote flow is well specified: standing-consent check for the tunnel, ngrok detection with three clear branches, strong token-hygiene rules, explicit verbatim-output requirement for the instruction block, and scope/--restrict/--control semantics. An agent could execute the remote and local commands, but the missing mode-selection step and mislabeled branches undermine reliable execution."},"latency_ms":17556,"stop_reason":"end_turn","usage":{"input_tokens":3750,"output_tokens":1156},"cost_usd":0.0953}
+{"config":"workflow-default","item":"a04","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Steps 1–3 are concrete: intent capture with a single question, a mutating-verb refusal script, and copy-ready look/extract and fuzzy-agent scripts with sound quoting guidance. Gaps undermine the stated deliverable: the preamble promises 'one JSON document' and the task says handle failed reads, but there is no final output step defining what is returned (JSON_START payload? wrapper with url/count?) nor what to do when the three-or-four-attempt budget is exhausted. Cross-references are also off — 'Rule 5' and 'Rule 4' (for sign-in) do not match the four untrusted-content rules shown, and the 'fallback browser only: /setup-browser-cookies or $B handoff' aside contradicts 'no cookie import.' Still executable for the main path."},"latency_ms":14300,"stop_reason":"end_turn","usage":{"input_tokens":3616,"output_tokens":842},"cost_usd":0.07826}
+{"config":"workflow-default","item":"p04","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The workflow is linear and well-sequenced: detect base branch, read current/base VERSION, loop the queue util over four bump levels, render a fixed-format dashboard (with an offline variant), then pick one of three prioritized suggestions. The exact output templates and the ★ active criteria are explicit. Gaps: the dashboard requires `<owner/repo>` and `<workspace_root>` but no step obtains them; the offline block's `Reason: <offline reason from warnings>` references a `.warnings` field not in the listed jq extractions, and the fallback `{\"offline\":true}` JSON has no warnings at all; the per-level annotations (`queue-advance: none`, `bumped past claimed ...`) have no stated derivation; and the unknown-host case is lumped with offline without saying whether a non-offline unknown host should suppress the full table. Minor bash issue: `cat VERSION | tr ... || echo` never triggers the fallback since the pipeline exit is tr's, yielding an empty version rather than 0.0.0.0. Step 5 priority when multiple conditions hold is only implied by ordering. Overall an agent can execute and produce the dashboard, but will have to improvise a few fields."},"latency_ms":21241,"stop_reason":"end_turn","usage":{"input_tokens":2463,"output_tokens":1493},"cost_usd":0.09928}
+{"config":"workflow-default","item":"f02","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 1 and 2 are detailed and largely unambiguous: detection commands, the AskUserQuestion flow, CLAUDE.md persistence format, honest exit-code capture, SKIPPED vs ERROR vs 127 semantics, and composite N/A on capture failure are all well specified. But Steps 3–6 — the scoring, dashboard, history, and trend core of the workflow — are one-sentence stubs. There are no category weights, no thresholds mapping tool output to scores, no handling of SKIPPED categories or the detected GBRAIN dimension in scoring, no dashboard layout, no history file path/format, and no trend comparison or recommendation logic. Step 2 even references 'that category's weight' and 'numeric history row' that are never defined. An agent would have to invent most of the deliverables, so outputs would be inconsistent across runs and not reproducible. Minor shell precedence issues in the detection snippet (`||` with `&&`) add small ambiguity."},"latency_ms":11774,"stop_reason":"end_turn","usage":{"input_tokens":2948,"output_tokens":704},"cost_usd":0.06468}
+{"config":"workflow-default","item":"p02","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The skill is well-structured with explicit mode detection, a four-step save flow (gather git state, summarize, compute duration, write collision-safe file + stamp), a clear file template, and a defined confirmation banner; the list flow has a concrete bash gather step and two table layouts. Minor clarity issues: the Step 4 block redundantly recomputes GSTACK_STATE_ROOT/PROJECT_DIR twice, a comment refers to `${SLUG}.md` while the code uses `TITLE_SLUG`, the TITLE_RAW hand-off mechanism is only sketched via an example, and the ISO-8601 timestamp is left to the LLM rather than computed in bash. Completeness gaps: the list flow never explicitly re-gathers the current branch needed for default filtering, `--all` isn't listed in the Detect section, no guidance if gstack-slug fails (empty SLUG yields a malformed path) or if not in a git repo, and the `completed` status shown in the example table is never produced by this skill. These are small, so the workflow remains readily executable."},"latency_ms":16366,"stop_reason":"end_turn","usage":{"input_tokens":4636,"output_tokens":1159},"cost_usd":0.10431}
+{"config":"workflow-default","item":"p02","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: mode detection, git state gathering, a bash-side collision-safe path computation with explicit allowlist sanitization, a precise file template, stamping, and a defined confirmation block. The list flow specifies ordering rationale, frontmatter parsing, branch filtering, and the empty-state message. Gaps: the `SLUG` lookup failure is unhandled, so an empty slug silently yields `projects//checkpoints`; and the list example shows a `completed` status while this skill only ever writes `in-progress`, leaving that transition undefined here. Minor noise includes duplicated `GSTACK_STATE_ROOT` lines and macOS-only `date -jf` (though it degrades to `unknown`). An agent can execute this and produce the deliverables with little ambiguity."},"latency_ms":17086,"stop_reason":"end_turn","usage":{"input_tokens":5033,"output_tokens":773},"cost_usd":0.08898}
+{"config":"workflow-default","item":"p03","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Command routing is unambiguous and each subcommand has concrete bash plus presentation guidance, so an agent can execute show/search/export/stats/add directly. Weaknesses cluster in Prune: the learnings.jsonl path is only defined in the Stats block (Prune says 'read the learnings.jsonl file' without a path), 'remove the matching line' is ambiguous when a key has multiple entries, and deleting lines contradicts the stated append-only/latest-wins model; the contradiction check also flags any same-key insight variation, which collides with normal latest-wins evolution. Smaller gaps: no fallback for unrecognized subcommands, the Export template omits the 'tool' type that Manual add permits, no filename decision for 'save as separate file', SLUG is computed but unused in several blocks, and no confirmation step after manual add. Overall followable with minor inference required."},"latency_ms":15923,"stop_reason":"end_turn","usage":{"input_tokens":2805,"output_tokens":1045},"cost_usd":0.0803}
+{"config":"workflow-default","item":"a05","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":3,"reasoning":"Baseline capture, page discovery, the 60s cadence rules, screenshot copy-out, report format, status rollup (BROKEN/DEGRADED/HEALTHY), JSON schema, and history logging are concrete and executable. However, the core decision in the monitoring loop is left undefined: 'Raise an alert at whatever severity seems right if a page looks noticeably worse than the baseline' gives no thresholds for console error deltas, load-time regression, broken links, or text disappearance, and never defines CRITICAL/HIGH/MEDIUM or what counts as a 'serious' alert that triggers AskUserQuestion. Phase 6 then depends on 'confirmed' alerts, 'resolved' incidents, and 'unconfirmed transients' in an observations array, but Phase 5 never specifies a confirmation rule (e.g., consecutive checks) or resolution criteria, so two agents would produce inconsistent alerts and verdicts. Phase 2 also awkwardly reaches forward into Phase 3's discovery script for per-page link checks, requiring adaptation. Overall the mechanics are solid but the judgment layer that determines alerts and the final verdict is underspecified."},"latency_ms":16469,"stop_reason":"end_turn","usage":{"input_tokens":4530,"output_tokens":1075},"cost_usd":0.09905}
+{"config":"workflow-default","item":"a05","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"Scripts, cadence rules, file paths, manifest schema, and report formats are concrete and executable. But the core detection logic is undefined: Phase 5 says 'Raise an alert at whatever severity seems right if a page looks noticeably worse,' giving no thresholds for load-time regression, new console messages, text-snapshot loss, or new 404s, and never mapping CRITICAL/HIGH/MEDIUM to conditions. Phase 6 then keys the verdict on 'confirmed' alerts and 'resolved' incidents, yet confirmation and resolution are never defined (user acknowledgment? repeated check?), so the HEALTHY/DEGRADED/BROKEN verdict is non-deterministic. 'Serious alert' triggering AskUserQuestion is similarly unspecified. Phase 7 only mentions copying screenshots, leaving metric refresh in baseline.json implicit."},"latency_ms":14380,"stop_reason":"end_turn","usage":{"input_tokens":4927,"output_tokens":950},"cost_usd":0.09677}
+{"config":"workflow-default","item":"a02","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is short and well-sequenced: log usage, ask the user for a path via AskUserQuestion (text input), call the shared freeze-state.sh helper, report success only on success, and emit a fixed confirmation message; the hook-enforcement section clearly explains fail-closed semantics, symlink resolution, and the hookSpecificOutput requirement. Weak spots: the failure branch is vague ('preserve it and ask the user to inspect recovery after any active writer finishes' doesn't specify what to tell the user or what 'unexpected state' looks like), jargon like 'serializes replacement with investigation cleanup' is unexplained, and it's ambiguous whether the confirmation should echo the user's raw path or the resolved absolute path. No guidance on a nonexistent or relative path beyond trusting the helper. Still, an agent can execute this end to end and produce the expected result."},"latency_ms":18394,"stop_reason":"end_turn","usage":{"input_tokens":1709,"output_tokens":1058},"cost_usd":0.06999}
+{"config":"workflow-default","item":"a04","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Steps 1–3 are concrete and well-specified: the intent prompt, the mutating-verb refusal text, and the look/extract and fuzzy `aside exec` scripts are copy-ready with explicit quoting rules, JSON_START/JSON_END markers, and a 3–4 attempt budget. Weaknesses: the 'handle failed reads' and 'return one JSON document' parts of the workflow lack a dedicated step — there is no instruction for what to return when the selector budget is exhausted, when CONSOLE_ERRORS shows a crashed app, or how the final JSON is presented to the user (shape, error field, sources). Cross-references to 'Rule 4' and 'Rule 5' do not match the four-rule untrusted-content block quoted inline (Rule 4 there is about prompt injection, not sign-in), which is a genuine internal inconsistency. The Capture Learnings section is boilerplate unrelated to the deliverable. Overall executable, but the terminal output/failure contract is under-defined."},"latency_ms":15551,"stop_reason":"end_turn","usage":{"input_tokens":3219,"output_tokens":988},"cost_usd":0.08159}
+{"config":"workflow-default","item":"a03","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"The individual sub-flows are well-specified: prerequisite check, target-agent question with TARGET_HOST mapping, a careful live-daemon consent gate, a standing tunnel-consent check via gstack-config, ngrok detection with three branches, strict token-handling security guidance, and explicit output requirements for the instruction block. However, the workflow jumps from Step 2 to Step 4 — the local-vs-remote decision that Step 4's '(option A)/(option B)' branches depend on is never asked, so an agent has no defined way to choose between `--local` and `--client`. Option letters are overloaded (Step 2 A–E, daemon consent A/B, machine A/B, tunnel consent A/B), and 'option A above' in the tunnel section is ambiguous. The live-daemon gate says 'pairing headed requires relaunching' but no command or flag indicates when headed mode is involved or when `--force-restart` would actually be needed, leaving the agent unsure whether the gate applies. The task calls for verifying the connection, but there is no verification step; the file ends with reference material on scopes rather than a success check. Local-flow failure handling ('suggest the generic remote flow') is vague. An agent could execute most of the remote flow but would stall on the missing mode decision and lack a defined completion state."},"latency_ms":16451,"stop_reason":"end_turn","usage":{"input_tokens":3750,"output_tokens":1082},"cost_usd":0.0916}
+{"config":"workflow-default","item":"a01","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The workflow is well-structured: a clear Iron Law, five ordered phases with concrete endpoints and fixture paths, a bounded retry (max 3), and a useful failure-mode table. Ambiguities remain: Phase 3 runs `build install` then separately says 'Re-deploy' with 'the same boot-token rotation flow' (undefined here, and unclear whether this is a distinct step); Phase 1 step 5 says to 'persist' a description without naming a location; Phase 4's 'revert and try again' doesn't say whether to loop back to Phase 2 (new root cause) or Phase 3 (new edit); the regression test's actual assertion is only 'asserts the post-fix behavior' with no guidance on what to compare (no post snapshot is captured, only a PNG); `<bug-slug>` derivation is unspecified. These gaps are mostly fillable by a capable agent, so it is executable, but several decision points are left implicit."},"latency_ms":15405,"stop_reason":"end_turn","usage":{"input_tokens":1716,"output_tokens":1058},"cost_usd":0.07006}
+{"config":"workflow-default","item":"a05","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"The browser scripts, file paths, manifest schemas, loop cadence, report format, JSON fields, and JSONL logging are concrete and followable, and Phases 2-4, 6, and 7 are well specified. However the core decision point of the workflow is essentially missing: Phase 5 says only to 'raise an alert at whatever severity seems right if a page looks noticeably worse than the baseline' — no thresholds for console-error deltas, load-time regression, text disappearance, or new 404s, and no mapping to CRITICAL/HIGH/MEDIUM. 'Serious alert' (which triggers the user prompt) is undefined, and Phase 6's status/verdict logic and JSON schema hinge on 'confirmed' alerts vs. 'unconfirmed transients' without ever defining what confirms an alert (e.g., consecutive checks). Minor issues: Phase 2 forward-references Phase 3's link check and requires adapting it per page, and the per-page STATUS determination in the report has no stated rule beyond the global alert-based one. The agent can run the loop and produce the artifacts, but the verdict will rest on improvised judgment."},"latency_ms":25683,"stop_reason":"end_turn","usage":{"input_tokens":4530,"output_tokens":1352},"cost_usd":0.1129}
+{"config":"workflow-default","item":"p05","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is tightly specified: concrete pre-filter commands, a fixed retirement-card schema, a caller-check grep with exact exclusions, explicit verdict set, read/search budgets, a JSON sidecar schema, defined AskUserQuestion options with a recommendation rule, and headless/spawned handling. Gaps and rough edges: the Step 1 block duplicates the GSTACK_STATE_ROOT line and references `$BRANCH` in the SEED_PLAN glob without defining it (likely preamble, but not shown); Step 2's test-file regex drops the `Test.(java|kt)` pattern that Step 1 counts, so TESTFILES and FILES can disagree; `--since <ref>` is described but no command is given to compute changed test files; the sidecar has a single `loc_delta` while the report requires per-batch LOC; Step 5 loops over batches but Step 6 implies only one batch is applied before handing to `/ship` and rerunning discovery, leaving multi-approval handling ambiguous; Step 6 is written for deletions and does not spell out how `rewrite`/`extend` verdicts are executed; and the Regression-proof paragraph is shared boilerplate with little bearing on an audit of existing tests. None of these block execution, but they create minor inconsistencies an agent must resolve itself."},"latency_ms":21460,"stop_reason":"end_turn","usage":{"input_tokens":5101,"output_tokens":1751},"cost_usd":0.13856}
+{"config":"workflow-default","item":"a05","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":3,"reasoning":"The capture scripts, manifest schema, report format, status rollup rules, and JSONL logging are concrete and executable. But the core monitoring decision is undefined: Phase 5 says 'Raise an alert at whatever severity seems right if a page looks noticeably worse,' giving no thresholds for load-time regression, new console errors, text loss, or screenshot diffs, and no mapping to CRITICAL/HIGH/MEDIUM or what counts as a 'serious alert.' Phase 6 then keys status and the alerts/observations split on 'confirmed' alerts and 'unconfirmed transients' without ever defining confirmation (e.g., consecutive rounds vs. user response). Also unaddressed: baseline pages diverging from discovered pages when a baseline.json exists but lacks them."},"latency_ms":23405,"stop_reason":"end_turn","usage":{"input_tokens":4927,"output_tokens":953},"cost_usd":0.09692}
+{"config":"workflow-default","item":"p02","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: mode detection, git state gathering, a bash-side sanitized collision-safe path, an explicit file template, a stamp step, and a list table with exact formats. Minor defects reduce confidence. Step 4's script redundantly recomputes GSTACK_STATE_ROOT/SLUG and its collision comment references `${SLUG}` while the code uses `${TITLE_SLUG}`, inviting confusion; no failure handling exists if `gstack-slug` returns empty, yielding `projects//checkpoints`. The `--all` flag appears only in List Step 2, not in Detect command, and the ISO-8601 frontmatter timestamp has no generation command (timezone unspecified). `files_modified` is ambiguous about untracked files. Still, an agent can execute both flows and produce the expected deliverables."},"latency_ms":14636,"stop_reason":"end_turn","usage":{"input_tokens":5033,"output_tokens":954},"cost_usd":0.09803}
+{"config":"workflow-default","item":"p04","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The five steps are sequential, concrete, and mostly runnable: base-branch fallback chain, per-level queue queries, canonical patch JSON, exact render templates, and a three-way next-action rule. Gaps undermine completeness: the dashboard requires `<owner/repo>`, `<workspace_root>`, and an offline `Reason: <offline reason from warnings>`, yet no step derives repo name or workspace root and the offline fallback writes `{\"offline\":true}` with no warnings field, leaving that line unfillable. The jq extraction list also omits `.version` and whatever field supplies annotations like \"(bumped past claimed 1.7.0.0)\". Minor shell bug: `cat VERSION | tr ... || echo` never triggers the default because `||` binds to `tr`. Still, an agent could produce a close approximation of the deliverable."},"latency_ms":14855,"stop_reason":"end_turn","usage":{"input_tokens":2860,"output_tokens":968},"cost_usd":0.077}
+{"config":"workflow-default","item":"p01","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The workflow is linearly ordered with concrete commands, explicit decision branches, and defined outputs at each step, so an agent can follow it. Weaknesses: Step 6 instructs a full re-run with --output json to save results, which costs another 30s-5min and may yield different numbers than the ones just interpreted (no guidance on capturing JSON up front or reconciling discrepancies); Step 1A's `find . -maxdepth 2` assumes cwd is the gstack repo while Step 0 locates the binary under ~/.claude, an implicit location mismatch; the Step 3 shell one-liner relies on fragile ||/&& chaining; and details like the <date> format, slug derivation, how to parse the dry-run 'Adapter availability' into a --models list, and the exact command for Step 2 option B/C are left to inference. Error/auth handling and result interpretation are reasonably specified."},"latency_ms":17332,"stop_reason":"end_turn","usage":{"input_tokens":2687,"output_tokens":915},"cost_usd":0.07262}
+{"config":"workflow-default","item":"f02","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 1 and 2 are detailed and mostly unambiguous: detection heuristics, a CLAUDE.md persistence format, an honest-capture subshell with exit-code preservation, and explicit SKIPPED/ERROR/N/A semantics. But Steps 3-6 are one-line stubs. Step 3 gives no category weights, scoring rubric, or composite formula even though Step 2 references weights and composite N/A handling; Step 4 gives no dashboard layout; Step 5 names no history file path, format, or fields; Step 6 defines no trend comparison or recommendation logic. The core deliverables (weighted composite, dashboard, persisted history, trends) are therefore undefined, forcing the agent to invent them. Minor issues in Step 1 (shell precedence in `||`/`&&` detection lines, hard-coded example tools in the AskUserQuestion prompt) add ambiguity. An agent can run the checkers correctly but cannot reliably produce the expected dashboard or history."},"latency_ms":17747,"stop_reason":"end_turn","usage":{"input_tokens":2948,"output_tokens":808},"cost_usd":0.06988}
+{"config":"workflow-default","item":"f02","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":2,"reasoning":"Steps 1-2 are precise: detection commands, AskUserQuestion options, CLAUDE.md persistence format, and a careful capture harness with explicit exit-code/ERROR/SKIPPED rules. But Steps 3-6 are one-sentence stubs. Step 3 says 'give each category a score... combine them' with no weights, scales, or formula, even though Step 2 forbids 'redistributing that category's weight' and persisting 'a numeric history row'—referencing definitions that never appear. Steps 5-6 give no history file path, schema, trend thresholds, or recommendation format, so two agents would produce incompatible histories. Minor: shell precedence bugs (`[ -f biome.json ] || [ -f biome.jsonc ] && echo`) can misfire. Half the required deliverables are undefined, so the dashboard cannot be reliably produced."},"latency_ms":10163,"stop_reason":"end_turn","usage":{"input_tokens":3345,"output_tokens":588},"cost_usd":0.06285}
+{"config":"workflow-default","item":"f03","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Only Step 1 (browser selection/setup) has concrete instructions, and even there the signals conflict: the prose says 'On READY, stop importing' (READY = probe found live sessions), while the SETUP block prints READY merely when the browse binary exists, then a separate `$B status` / `Mode: cdp` check is the real 'sessions exist' gate. The bun-install block also appears after `./setup` as step 3 despite being a prerequisite, so ordering is muddled. Steps 2-4, which are the core of the workflow, are stubs: 'Confirm options' is literally 'TBD', 'Select source and scope' says 'import whatever cookies seem relevant' with no profile selection, domain scoping, consent prompt, or actual import command, and 'Report honestly' gives no output format or content. An agent cannot import a single cookie or produce a defined deliverable from this bundle."},"latency_ms":11151,"stop_reason":"end_turn","usage":{"input_tokens":1630,"output_tokens":767},"cost_usd":0.05465}
+{"config":"workflow-default","item":"a05","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"The browser scripts, file paths, report template, JSON/JSONL schemas, and AskUserQuestion prompts are concrete and copy-pasteable, and the loop cadence rules (no overlap, max(0,60-elapsed), stop at deadline) are explicit. However, the core decision of the workflow is undefined: Phase 5 says to 'raise an alert at whatever severity seems right if a page looks noticeably worse' with no thresholds for console-error deltas, load-time regression, text disappearance, or new 404s, and no definitions of CRITICAL/HIGH/MEDIUM or what counts as a 'serious' alert. Phase 6 then keys the verdict on 'confirmed' alerts, 'unconfirmed transients', and 'resolved' incidents, none of which are defined anywhere in Phase 5 (no re-check/confirmation rule). There is also tension between Phase 2 ('homepage if no --pages') and Phase 3 ('auto-discover if no --pages'), and Phase 2 asks to reuse the Phase 3 link check per page and per round though the Phase 5 script omits it, leaving the agent to assemble it. An agent can execute the mechanics and produce a report, but alert and status outcomes will be inconsistent because the detection rules are missing."},"latency_ms":20044,"stop_reason":"end_turn","usage":{"input_tokens":4530,"output_tokens":1419},"cost_usd":0.11625}
+{"config":"workflow-default","item":"f04","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Steps 0, 1, 7, and 9 are concrete and executable (scope confirmation, research checklist with concept map, cross-linking rules, redaction scan, commit template, structured summary). However the core of the workflow is gutted: Steps 2-6, which should cover partitioning content into Diataxis quadrants and writing each from templates, are replaced with 'write whatever documentation seems most useful... in whatever structure fits.' Later steps and the Important Rules then assume quadrant-specific docs, templates, and tutorial step limits that were never defined, so the agent has no guidance for the main deliverable. Step 8 is internally contradictory: it says the self-review happens after Step 9 has committed and pushed, is optional, and failures can be deferred to a later PR, yet Step 9's summary requires reporting pass/fail on each gate and the rules declare accuracy 'non-negotiable.' These ordering and priority contradictions, combined with the missing writing/partitioning steps, make the workflow ambiguous and unlikely to reliably produce the expected Diataxis deliverables."},"latency_ms":9989,"stop_reason":"end_turn","usage":{"input_tokens":3495,"output_tokens":750},"cost_usd":0.07245}
+{"config":"workflow-default","item":"a06","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Steps 2–6 are logically ordered with no contradictions: a concrete detection script, per-platform branches that resolve ambiguity (multi-platform prompt, 'detection is a hint', explicit URL confirmation, Heroku/Railway deferring to the manual questions), a consolidated field-gathering step before writing, and a non-blocking verification step. Shell commands are specific and runnable. Gaps: the CLAUDE.md output format/section name/key names are never specified in this bundle (only implied by the 'set ... to none' field list), so the primary deliverable's shape is underdefined; 'constrained to methods allowed by repo settings' gives no way to query those settings; when a platform config and a GitHub Actions deploy workflow coexist, nothing says whether to record the workflow; and mapping the manual Q3 answers (e.g., 'check workflow status') to a concrete status command is left to the agent. These are fillable by a competent agent but make the output less deterministic."},"latency_ms":23278,"stop_reason":"end_turn","usage":{"input_tokens":2830,"output_tokens":1495},"cost_usd":0.10305}
+{"config":"workflow-default","item":"p02","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":5,"reasoning":"The skill is self-contained and well-sequenced: command detection, state gathering, summary structure, bash-side path computation with an allowlist sanitizer and collision suffix, an explicit frontmatter/body template, a stamp step, and a confirmation block are all spelled out. The list flow defines sorting rationale, default branch filtering, the --all variant, and both table layouts. Minor blemishes: the GSTACK_STATE_ROOT/SLUG lines are redundantly repeated in Step 4 and the list flow; the collision comment references `${SLUG}` when the code uses `${TITLE_SLUG}`; the session-duration fallback uses macOS-only `date -jf` (gracefully degrades to 'unknown' but is unstated); the Date column source (filename vs frontmatter timestamp) is slightly ambiguous; and the empty-list message doesn't distinguish 'no contexts for this branch' from 'no contexts at all'. None of these block execution—an agent can produce the saved file and table as specified."},"latency_ms":18851,"stop_reason":"end_turn","usage":{"input_tokens":4636,"output_tokens":1222},"cost_usd":0.10746}
+{"config":"workflow-default","item":"a03","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"The supplied workflow jumps from Step 2 to Step 4; the local-vs-remote decision that Step 4's 'option A/B' branches depend on is never asked, and those letters collide with both the Step 2 agent list and the relaunch-consent options, so an agent cannot tell which 'A' is meant. Example: Step 2 option E sets TARGET_HOST to 'generic (no host-specific config)', yet Step 4 requires `$B pair-agent --client TARGET_HOST` with no stated value for that case. Consent gates, ngrok handling, and the verbatim-output rule are clear and well-specified, but there is no connection-verification step beyond 'try asking it to navigate', and the --force-restart consent never says which command the flag attaches to."},"latency_ms":14942,"stop_reason":"end_turn","usage":{"input_tokens":4147,"output_tokens":992},"cost_usd":0.09107}
+{"config":"workflow-default","item":"f05","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":2,"reasoning":"Steps 1–7 are detailed and mostly unambiguous (name rules, pure-parser template, frontmatter contract, stageSkill call). But the gating logic is broken. Step 8 says a failing test should 'continue to Step 10 anyway; a failing test can be fixed after the skill is committed,' which directly contradicts the Iron contract that requires test pass before commit and discards the temp dir on failure. Step 9—the explicit user-approval question—is entirely missing, yet Step 10 branches on 'if the user approved' and 'if the user rejected in step 9,' so the central decision point is undefined. Minor gaps: Step 6 describes two resolution signals but the code implements one; fixture naming differs slightly between Steps 4 and 5. An agent could ship broken skills."},"latency_ms":11846,"stop_reason":"end_turn","usage":{"input_tokens":5666,"output_tokens":739},"cost_usd":0.09361}
+{"config":"workflow-default","item":"a01","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":3,"reasoning":"The five-phase structure, Iron Law, iteration cap, and failure-mode table are clearly laid out and give an agent a coherent spine to follow. However, several steps are underspecified: Phase 3 says `build install` then separately 'Re-deploy' with 'the same boot-token rotation flow' without clarifying whether these are distinct actions; Phase 4's 'compare against pre.png' gives no criterion for deciding the bug 'visibly persists' (pixel diff? visual inspection? accessibility tree?) and 'revert and try again' doesn't say whether to try a new hypothesis or the same edit; Phase 1 step 5 says to 'persist' a description without saying where; the bug-slug derivation is never defined. Phase 5 says the test 'asserts the post-fix behavior' but never specifies what artifact the assertion compares against (post.png? a post snapshot? an accessibility node?), leaving the main deliverable loosely defined. The 409 schema_mismatch remedy ('re-snapshot') conflicts with the Iron Law that the fixture must capture the pre-fix bug state, since re-snapshotting after a rebuilt fix would not reproduce the bug. Decision points exist (AskUserQuestion on multiple root causes, 3-iteration escalation) but the verification and test-assertion logic needs more precision for reliable autonomous execution."},"latency_ms":20288,"stop_reason":"end_turn","usage":{"input_tokens":1716,"output_tokens":1406},"cost_usd":0.08746}
+{"config":"workflow-default","item":"f01","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The workflow jumps from Step 2 to Step 4, and Step 4 branches on 'same machine (option A)' vs 'different machine (option B)' that were never defined—Step 2 only asks which agent, so the local-vs-remote decision is missing and the A/B labels collide with the relaunch-consent options. The consent gate presents 'Keep the live daemon' as the recommended choice, then immediately says 'Always pass --force-restart to the commands below,' directly contradicting the one-way-door consent (and no listed command includes the flag). Step 1 also tells a hurried agent to run Step 5 verification before pairing, which is nonsensical ordering. The ngrok auth/install flows, scope guidance, and verbatim-output rule are solid, but the core decision path is broken."},"latency_ms":13085,"stop_reason":"end_turn","usage":{"input_tokens":4109,"output_tokens":916},"cost_usd":0.08689}
+{"config":"workflow-default","item":"a03","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"Steps 1, 2, and 4 are present but Step 3 is absent, and Step 4 branches on 'same machine (option A)' vs 'different machine (option B)' — a decision that is never asked in the supplied text. Worse, those letters collide with Step 2's options A–E (where A = OpenClaw) and with the live-daemon consent's A/B, so an agent cannot unambiguously tell which 'option A/B' is meant. The live-daemon consent gate also says 'pairing headed requires relaunching' without ever defining when pairing is headed or which commands would relaunch, leaving unclear when --force-restart is even relevant. No step covers verifying the connection after pairing, which the workflow is supposed to include. On the positive side, the concrete commands ($B pair-agent --local/--client, --control, --restrict), the tunnel consent gate via gstack-config, the ngrok install/auth/not-authed branches with the explicit token-hygiene rule, and the 'print the full block verbatim' deliverable are well specified and runnable, so an agent can likely execute the core pairing by inferring local-vs-remote from Step 2's parentheticals — but it must guess at the missing decision point and has no defined verification output."},"latency_ms":17355,"stop_reason":"end_turn","usage":{"input_tokens":3750,"output_tokens":1111},"cost_usd":0.09305}
+{"config":"workflow-default","item":"p03","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Command routing and each subcommand's bash invocation are explicit, with clear presentation instructions and a hard gate against code changes. Weaknesses cluster in Prune: the learnings.jsonl path is only defined in the Stats section (prune relies on the agent inferring it), removal edits the file in place while updates are described as 'append-only, latest wins' — a semantic conflict that also means the contradiction check will flag legitimate updates as CONFLICTs unless deduped by latest ts (which prune never says to do, though the stats script does). Export's template omits the 'tool' type that Manual add allows and inconsistently drops confidence for preferences; Manual add shows a mandatory 'files' array for an optional field; unknown subcommands and tool/bun failures have no defined handling. Still, an agent can execute every path and produce the expected outputs with minor judgment calls."},"latency_ms":18034,"stop_reason":"end_turn","usage":{"input_tokens":2805,"output_tokens":1132},"cost_usd":0.08465}
+{"config":"workflow-default","item":"f04","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Step 0, Step 1, Step 7, and Step 9 are concrete and well-specified (scope question, research commands, concept map, redaction scan, commit format, summary table). But the heart of the workflow is hollow: 'Steps 2-6' is a single line saying 'write whatever documentation seems most useful... in whatever structure fits', so there is no partitioning into Diataxis quadrants, no templates, no per-quadrant writing guidance, and no decision rules for what goes where—despite later steps and the Important Rules assuming quadrant-specific outputs. Step 8 is self-contradictory and defanged: it says the review happens after Step 9 has committed and pushed, is 'optional', and failures 'can be fixed in a later PR', yet Step 9's summary requires reporting pass/fail on each gate and the rules say accuracy is non-negotiable. The result is that an agent can research and commit, but cannot reliably produce the expected structured Diataxis deliverables or honor the quality gates as written."},"latency_ms":10012,"stop_reason":"end_turn","usage":{"input_tokens":3495,"output_tokens":703},"cost_usd":0.0701}
+{"config":"workflow-default","item":"a05","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":3,"reasoning":"The capture scripts, loop cadence, file paths, manifest schema, report template, and JSONL logging are concrete and followable. However, the core decision point of the workflow — when to alert and at what severity — is left to 'whatever severity seems right if a page looks noticeably worse', with no thresholds for new console errors, load-time regression, 404 links, or text disappearance despite Phase 2 collecting exactly those signals. Phase 5 then gates user notification on an undefined 'serious alert', and Phase 6 computes BROKEN/DEGRADED/HEALTHY and the observations array from 'confirmed' alerts without ever defining how an alert becomes confirmed or resolved. This leaves the verdict logic underdetermined and inconsistent across runs. Minor gaps: `{date}` format unspecified, 'Avg Load' in the report vs 'latest metrics' in the JSON. The agent can execute the mechanics and emit the deliverables, but the alert/verdict judgments will be ad hoc."},"latency_ms":18777,"stop_reason":"end_turn","usage":{"input_tokens":4530,"output_tokens":1287},"cost_usd":0.10965}
+{"config":"workflow-default","item":"p01","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is linear and well-structured: binary lookup with fallback, prompt selection with three concrete branches mapped to CLI flags, a dry-run to surface authed providers, a judge-availability check, command construction, result interpretation, and optional JSON save. Each AskUserQuestion has explicit options and a recommendation, and Step 4 clearly explains how to assemble the command from prior decisions. Weaknesses: Step 1A's `find .` assumes the cwd is the gstack install dir, which Step 0 doesn't guarantee (the binary may be found under $HOME); Step 6 're-run with --output json' repeats the full, slow, paid benchmark (including judge cost) rather than capturing the first run's results, and produces a different sample than the one just summarized; the save path slug and `mkdir -p` for the directory are unspecified; Step 5 doesn't describe the table's columns or how errors appear. The judge-check one-liner works but relies on shell `||`/`&&` precedence that is easy to misread. None of these block execution, but they introduce wasted cost and minor ambiguity."},"latency_ms":16683,"stop_reason":"end_turn","usage":{"input_tokens":2687,"output_tokens":1109},"cost_usd":0.08232}
+{"config":"workflow-default","item":"p04","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The five steps are linear and mostly unambiguous: base-branch detection, VERSION reads, per-level queue queries, rendering, and a single next-action suggestion are each spelled out with concrete commands and exact output templates. Gaps weaken completeness: the render step lists five jq fields but the dashboard also needs `<owner/repo>`, `<workspace_root>`, per-level annotations like '(bumped past claimed 1.7.0.0)', sibling columns (last commit, PR), and the offline '<reason from warnings>' — none of these have a stated source or derivation, and collision detection in `.claimed` is implied rather than defined. Partial failure (some bump levels offline, others not) and the 'unknown host' case being folded into the OFFLINE block are not resolved. Step 2's `cat | tr || echo` fallback won't actually trigger on a missing VERSION (tr exits 0), yielding an empty string instead of 0.0.0.0. Step 5's priority among the three suggestions is only implied by numbering. Despite these, an agent can run the commands and produce a recognizable dashboard, so actionability remains fairly high."},"latency_ms":17578,"stop_reason":"end_turn","usage":{"input_tokens":2463,"output_tokens":1176},"cost_usd":0.08343}
+{"config":"workflow-default","item":"f03","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Step 1 is detailed but self-contradictory: 'On READY, stop importing' collides with the SETUP block where READY merely means the browse binary exists, and 'run ./setup' precedes the bun-install check it depends on. The remainder collapses: Step 2 is literally 'TBD', Step 3 says 'import whatever cookies seem relevant' with no source-profile selection, domain scoping, consent prompt, or actual import command, and Step 4 gives no report format or honesty criteria. The agent cannot determine which cookies to import, how, or what to deliver, so the workflow is not executable beyond the environment probe."},"latency_ms":8239,"stop_reason":"end_turn","usage":{"input_tokens":2027,"output_tokens":521},"cost_usd":0.04632}
+{"config":"workflow-default","item":"p04","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The five steps are sequential, with concrete bash, a canonical JSON source (patch-level), explicit ★-active criteria, and three mutually exclusive next-action rules. Gaps are in field mapping: the dashboard requires `<owner/repo>`, `<workspace_root>`, sibling `last commit`/PR columns and the offline `Reason: <from warnings>`, yet Step 4 only lists `.host/.offline/.claimed/.siblings/.active_siblings` to extract and the fallback `{\"offline\":true}` has no warnings at all. The per-level annotations (`queue-advance: none`, `bumped past claimed 1.7.0.0`) are shown in the template but no derivation rule is given. Step 5 rule 2 also never states which version (CURRENT or BASE) the sibling is compared against. An agent can still produce a plausible dashboard, but some cells require guessing."},"latency_ms":12682,"stop_reason":"end_turn","usage":{"input_tokens":2860,"output_tokens":863},"cost_usd":0.07175}
+{"config":"workflow-default","item":"p05","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is concrete and mostly unambiguous: boundaries, spawned/headless report-only mode, a low-value catalog with an explicit retention bar, a fully enumerated retirement card, concrete pre-filter scripts, budgets (8 minutes, max-candidates×3 reads/searches), a report outline, a JSON sidecar schema, and a fixed A/B/C AskUserQuestion with recommendation rules. Weaknesses: Step 1's script duplicates a line and references `$BRANCH` (not set in this excerpt) for seed discovery; Step 1 and Step 2 use slightly different test-file regexes (Java/Kotlin only in Step 1); `--since` scoping is described but Step 2 gives no command for turning a changed-files set into `<scope>`; how to estimate production/test LOC per batch before any edit is unspecified; and the multi-batch flow is ambiguous—Step 5 reads as asking one question per batch across all batches, while Step 6 implies a sequential ask→apply→/ship→'after it lands, rerun discovery' loop the agent cannot actually observe completing. The value-bar/regression-proof text is oriented to writing new tests and is partly noise for a retirement sweep, though not harmful. Overall an agent can execute this and produce the report, sidecar, and gated edits with only minor judgment calls."},"latency_ms":29479,"stop_reason":"end_turn","usage":{"input_tokens":5101,"output_tokens":2155},"cost_usd":0.15876}
+{"config":"workflow-default","item":"a02","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is short and linear: log analytics, prompt via AskUserQuestion with free-text input, call freeze-state.sh set with the path, confirm with a fixed message. The 'How it works' and Notes sections give useful context on fail-closed semantics, symlink handling, and scope (Edit/Write only). Weaknesses: the error path is underspecified — 'FREEZE_BUSY or unexpected state' is not enumerated, and 'ask the user to inspect recovery after any active writer finishes' is vague about what the agent should actually say or do. There is no guidance for a nonexistent or non-directory path, user cancellation of the prompt, or what to report on a non-BUSY helper failure. The confirmation text slightly glosses over the fact that the boundary persists across sessions (covered only in Notes). Otherwise an agent can execute this reliably."},"latency_ms":12711,"stop_reason":"end_turn","usage":{"input_tokens":1709,"output_tokens":861},"cost_usd":0.06014}
+{"config":"workflow-default","item":"f03","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Only the browser-selection step is specified, and even it conflicts: 'READY' from the probe means stop importing, while the SETUP block's 'READY: $B' means proceed, and bun installation is listed after the ./setup step that requires it. The core workflow is absent: section 2 is literally 'TBD', and section 3 says 'Import whatever cookies seem relevant' with no profile selection, domain scoping, consent prompt, or import command. Section 4 gives no report format or honesty criteria. An agent cannot execute the import or produce a defined deliverable from this bundle."},"latency_ms":11783,"stop_reason":"end_turn","usage":{"input_tokens":2027,"output_tokens":637},"cost_usd":0.05212}
+{"config":"workflow-default","item":"a02","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The happy path is clear and sequential: log analytics, ask for a directory via AskUserQuestion (text input, exact wording given), call the shared freeze-state.sh helper, and deliver a specified confirmation message. The 'How it works' and 'Notes' sections give good context on hook semantics (fail-closed, symlink resolution, trailing-slash matching). Gaps: the helper's success/failure contract is not defined (exit codes, output format, what 'unexpected state' looks like), so the agent must guess how to detect FREEZE_BUSY vs. success; the failure-branch instruction ('preserve it and ask the user to inspect recovery after any active writer finishes') is vague about what to actually tell the user; and there is no guidance for a nonexistent or relative path (beyond implying the helper resolves it) or for the user declining to answer. Minor tension between 'persists until explicitly removed via the state file' and 'never write or delete the state file directly' (resolved only by the /unfreeze pointer). Overall executable for the normal case, with underspecified error handling."},"latency_ms":13610,"stop_reason":"end_turn","usage":{"input_tokens":1709,"output_tokens":950},"cost_usd":0.06459}
+{"config":"workflow-default","item":"a04","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Steps 1–3 are crisply written with concrete, copy-ready scripts, an explicit refusal message, bash-quoting rules, a structured-vs-fuzzy fork, and a sign-in-wall branch. However, the workflow has no terminal step: there is no 'Step 4 — return the JSON document' defining the final deliverable shape or how the JSON_START/JSON_END payload (or fuzzy {answer, sources}) is handed back to the user. Failed-read handling is thin — CONSOLE_ERRORS and a '3–4 attempts' budget are mentioned, but nothing says what to do when the budget is exhausted, when waitForSelector times out, or when GSTACK_STEP_OK is absent. Cross-references are also inconsistent: the quoted untrusted-content block lists rules 1–4, yet the text cites 'Rule 5' for the agent reply and 'Rule 4' (the prompt-injection rule) for the sign-in wall, which doesn't match. The Capture Learnings section is generic boilerplate rather than part of the scrape flow. An agent can execute the happy path readily, but the missing exit/return decisions leave gaps."},"latency_ms":17875,"stop_reason":"end_turn","usage":{"input_tokens":3219,"output_tokens":1101},"cost_usd":0.08724}
+{"config":"workflow-default","item":"f02","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 1-2 are concrete and careful (detection script, AskUserQuestion flow, CLAUDE.md persistence format, a rigorous capture harness with explicit exit-code/ERROR/SKIPPED semantics), though the detection script has shell issues (&&/|| precedence, zsh `setopt` in a bash snippet, shellcheck line echoes no targets). However, Steps 3-6 — scoring with weights into a composite, dashboard presentation, history persistence, and trend/recommendation output — are one-sentence stubs with no weights, scale, thresholds, output format, file path/schema, or comparison logic. Step 2 even references 'category score', 'composite', and 'numeric history row' that are never defined anywhere in the bundle, and no Read pointers to on-demand files cover them. An agent cannot produce the expected deliverables (weighted composite, dashboard, history, trends) consistently from these instructions."},"latency_ms":14105,"stop_reason":"end_turn","usage":{"input_tokens":2948,"output_tokens":796},"cost_usd":0.06928}
+{"config":"workflow-default","item":"p01","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"Linear six-step flow with concrete bash for locating the binary, dry-run provider discovery, judge detection, command construction, and result interpretation; decision branches (prompt source A/B/C, provider choice, judge on/off, save/not) are explicit and map cleanly to CLI flags. Gaps are minor but real: Step 1A's `find .` assumes cwd is the gstack repo rather than the installed skills dir, so the skill list may come up empty; Step 6 doesn't define the slug or `mkdir -p` the output dir and silently re-runs the full (possibly judged) benchmark without warning the user of doubled cost/time; Step 5's interpretation presumes table column semantics not described here. Nothing contradictory, so an agent can execute end-to-end with small judgment calls."},"latency_ms":16088,"stop_reason":"end_turn","usage":{"input_tokens":2687,"output_tokens":1088},"cost_usd":0.08127}
+{"config":"workflow-default","item":"f01","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The core pairing mechanics (local --local flow, remote --client flow, ngrok install/auth branching, token hygiene, verbatim instruction-block output, scope options) are well specified, but the bundle has serious structural defects. Step 3 is missing entirely, yet Step 4 branches on 'same machine (option A)' vs 'different machine (option B)' — the local-vs-remote decision is never asked, and 'option A/B' is also reused for the relaunch/keep-daemon choice, making the references ambiguous. Step 1 bizarrely tells the agent to run Step 5 (verify connection) before pairing if short on time, which is nonsensical ordering. Most damaging: the live-daemon consent gate presents 'Keep the live daemon' as the recommended option under an 'iron rule,' then immediately says 'Always pass --force-restart to the commands below' — a direct contradiction that could lead an agent to destroy the user's sessions without consent, and the subsequent commands don't actually include the flag. Verification is only defined for the remote case. An agent could stumble through parts of this, but the missing decision step and contradictory destructive-action guidance make it unreliable to execute as written."},"latency_ms":13976,"stop_reason":"end_turn","usage":{"input_tokens":3712,"output_tokens":1022},"cost_usd":0.08822}
+{"config":"workflow-default","item":"a05","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":3,"reasoning":"The browser scripts for baseline, discovery, and monitoring checks are concrete and copy-pasteable, the loop cadence is explicitly defined, and the report format, status rollup rule, JSON fields, and history logging are well specified. However the core decision logic of the monitor is largely undefined: Phase 5 tells the agent to 'raise an alert at whatever severity seems right if a page looks noticeably worse' with no thresholds for console error deltas, load-time regression, missing text, or broken links, and no mapping to CRITICAL/HIGH/MEDIUM. 'Serious alert' and 'confirmed' alert (which drive the AskUserQuestion trigger and the HEALTHY/DEGRADED/BROKEN verdict and the alerts-vs-observations split) are never defined, so two agents would produce different verdicts from the same data. Smaller gaps: 'top 5 internal navigation links' has no ordering criterion, `{date}` format is unspecified, and the Phase 2 instruction to reuse Phase 3's link check per page is somewhat awkwardly cross-referenced. The workflow can be executed end to end, but the alerting/verdict step relies heavily on agent judgment where a monitoring skill should be deterministic."},"latency_ms":17343,"stop_reason":"end_turn","usage":{"input_tokens":4530,"output_tokens":1175},"cost_usd":0.10405}
+{"config":"workflow-default","item":"a03","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"The remote-pairing branch is well specified: tunnel consent gate with exact config commands, ngrok detection with three clearly separated branches, strong security guidance on never accepting the authtoken in chat, and an emphatic requirement to echo the full instruction block. However, the workflow jumps from Step 2 to Step 4 with no Step 3 — the local-vs-remote decision that the task calls for is never asked, yet Step 4 branches on 'same machine (option A)' vs 'different machine (option B)' as if it had been. Those letters also collide with Step 2's A–E agent options and the A/B letters of both consent prompts, so an agent cannot reliably map 'option A' to a prior answer. The live-daemon consent is confusing in trigger: Step 1 guarantees a daemon is running, the prompt talks about 'pairing headed' but no headed flag appears in any command, so when --force-restart would actually be needed is undefined. The local path gives no handling for the 'generic' (E) target and the 'verify the connection' deliverable is entirely absent — there is no post-pairing check beyond telling the user to try it. The trailing 'What the remote agent can do' section is informative but mostly reference material, not steps. An agent can execute the remote flow faithfully but must guess at the mode decision and skip verification."},"latency_ms":19301,"stop_reason":"end_turn","usage":{"input_tokens":3750,"output_tokens":1363},"cost_usd":0.10565}
+{"config":"workflow-default","item":"p03","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Routing and per-command scripts are explicit and runnable, and prune's user-approval loop is well-specified. Gaps undermine completeness: Prune tells the agent to \"read the learnings.jsonl file and remove the matching line\" but the file path is only defined inside the Stats block, and for a CONFLICT the instructions never say which of the two contradicting entries the Remove option targets. Minor inconsistencies also exist—Manual add offers a 'tool' type while the Export template has no Tool section, and dedup keys on key|type in Stats but key alone in Prune. Unknown subcommands and empty optional files in the add payload are unhandled. Overall executable but requires inference on several decisions."},"latency_ms":18181,"stop_reason":"end_turn","usage":{"input_tokens":3202,"output_tokens":775},"cost_usd":0.07077}
+{"config":"workflow-default","item":"f01","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The excerpt has serious internal contradictions and a missing decision step. Step 3 is absent entirely, yet Step 4 branches on 'same machine (option A)' vs 'different machine (option B)' that no prior step defines; Step 2 only asks which agent, not local-vs-remote, and the same A/B labels are reused for the live-daemon relaunch choice, inviting confusion. The daemon-consent block carefully establishes that --force-restart is only passed if the user explicitly chooses to relaunch (recommending they don't), but is immediately followed by 'Always pass --force-restart to the commands below so pairing starts from a clean daemon' — a direct contradiction on a one-way door that could destroy tabs/logins, and none of the subsequent commands actually include the flag. Step 1's advice to 'run Step 5 (Verify connection) first and come back to pairing later' is nonsensical since there is nothing to verify before pairing. Step 5 assumes an instruction block was pasted, which doesn't apply to the --local path, and 'look for the connected agent' is vague. The remote/ngrok sub-flow (consent gate, token-handling security, instruction-block output, restrict/control flags) is detailed and actionable, which keeps scores from the floor, but the missing mode-selection step and the force-restart contradiction mean an agent cannot reliably execute the workflow as written."},"latency_ms":18966,"stop_reason":"end_turn","usage":{"input_tokens":3712,"output_tokens":1422},"cost_usd":0.10822}
+{"config":"workflow-default","item":"p04","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The five steps are sequentially ordered, with concrete bash for base-branch detection, VERSION reads, and per-level queue queries, plus an exact dashboard template and a three-way next-action rule. Gaps weaken completeness: the `Repo: <owner/repo>` and `<workspace_root>` values are never derived; the `.siblings` object schema (path/branch/VERSION/last commit/PR) is not specified though the table requires it; the offline block references a `.warnings` reason that isn't in the jq extraction list and won't exist when the util fails (fallback JSON is just `{\"offline\":true}`); how to compute collision annotations and the 'queue-advance/bumped past' notes is left implicit. Minor inconsistencies: Step 1 title promises platform detection but only base branch is detected, 'unknown host' is told to use the offline block while the full template also allows `Host: unknown`, and the `|| echo` fallbacks in Step 2 won't fire due to the `tr` pipeline. Still, an agent can execute the core workflow and produce a recognizable dashboard with modest inference."},"latency_ms":19484,"stop_reason":"end_turn","usage":{"input_tokens":2463,"output_tokens":1309},"cost_usd":0.09008}
+{"config":"workflow-default","item":"p01","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is linear with explicit commands, decision gates, and deliverables (dry-run auth preview, judge gate, table summary, optional JSON baseline). Two gaps weaken it. First, Step 1A's `find . -maxdepth 2 -name SKILL.md` assumes the current directory is the gstack install, yet Step 0 locates the binary under `$HOME/.claude/skills/gstack`; the agent must guess which directory to search. Second, Step 6 instructs a full re-run with `--output json` to save results, duplicating the slow, paid benchmark (and judge cost) instead of capturing the first run, and leaves the `<skill-or-prompt-slug>` and directory creation undefined. Otherwise an agent can execute this end to end and produce the expected outputs."},"latency_ms":13106,"stop_reason":"end_turn","usage":{"input_tokens":3084,"output_tokens":876},"cost_usd":0.07464}
+{"config":"workflow-default","item":"a01","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":3,"reasoning":"The workflow has a solid phased skeleton, an explicit Iron Law, iteration cap, and a failure-mode table, but key decision points are underspecified. Example one: Phase 3 runs `build install` in step 2, then says \"Re-deploy. The same boot-token rotation flow runs\" in step 4 with no definition of what re-deploy means beyond the install already done, leaving the agent unsure whether a second action is required. Example two: Phase 4 tells the agent to compare a fresh screenshot against the *pre* (buggy) PNG with no pass criterion, and Phase 5 says the test must \"assert the post-fix behavior\" without specifying what is asserted or how `<bug-slug>-post.png` is used. Phase 1's \"persist a one-line description\" also names no destination. Executable, but requires guesswork at verification and test-writing."},"latency_ms":17653,"stop_reason":"end_turn","usage":{"input_tokens":2113,"output_tokens":1345},"cost_usd":0.08838}
+{"config":"workflow-default","item":"f02","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 1-2 (stack detection with CLAUDE.md persistence, honest capture with exit-code/ERROR/SKIPPED semantics) are detailed and mostly unambiguous, though the detection script has shaky shell precedence (e.g., the pylint/ruff line) and the Glob step is disconnected from the shellcheck detection. However, Steps 3-6 — the actual deliverables — are one-sentence stubs: no category weights, no score formula or thresholds, no composite calculation, no dashboard layout, no history file path/format, no trend comparison rules, no recommendation structure. Step 2 even references concepts ('do not redistribute that category's weight', 'persist a numeric history row') that are never defined. An agent could run tools faithfully but would have to invent the entire scoring, dashboard, persistence, and trend logic, so outputs would be inconsistent across runs and unusable for trend tracking."},"latency_ms":14328,"stop_reason":"end_turn","usage":{"input_tokens":2948,"output_tokens":829},"cost_usd":0.07093}
+{"config":"workflow-default","item":"f04","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 0, 1, 7 and 9 are concrete, but the workflow's core is hollowed out: Steps 2-6 reduce the entire Diataxis partitioning, templates, and per-quadrant writing to 'write whatever documentation seems most useful,' yet Step 7 and the final summary assume four defined quadrants exist. Ordering is also contradictory: Step 8's quality gates are declared to run 'after Step 9 has committed and pushed,' are 'optional,' and failures 'can be fixed in a later PR,' directly conflicting with the 'Accuracy is non-negotiable' rule and with Step 9's summary reporting gate pass/fail. An agent could research and commit something, but cannot reliably produce the specified Diataxis deliverables."},"latency_ms":9783,"stop_reason":"end_turn","usage":{"input_tokens":3892,"output_tokens":639},"cost_usd":0.07087}
+{"config":"workflow-default","item":"a04","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Steps 1–3 are concrete: intent capture, a verb list for refusing mutations, and copy-ready repl/exec scripts with quoting rules, bridge constraints, and a retry budget. Two gaps weaken it. First, the promised 'one JSON document' deliverable has no return step: the structured path emits JSON_START/JSON_END and the fuzzy path emits {answer, sources}, but nothing defines the final document shape or what to return when the three-or-four-attempt budget is exhausted. Second, failure handling is scattered (CONSOLE_ERRORS hint, sign-in wall, non-JSON wrap) rather than a decision path, and the sign-in text cites 'Rule 4' while the inline untrusted-content Rule 4 is about prompt injection, creating a mismatch. Executable, but output contract is underspecified."},"latency_ms":18060,"stop_reason":"end_turn","usage":{"input_tokens":3616,"output_tokens":1133},"cost_usd":0.09281}
+{"config":"workflow-default","item":"a02","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The happy path is clear and linear: log analytics, ask for a directory via AskUserQuestion (text input), call freeze-state.sh set, report success with a fixed message. The 'How it works' and Notes sections give useful context on fail-closed polarity, symlink handling, and scope. Weaknesses: the failure branch is underspecified—'FREEZE_BUSY or unexpected state' is not defined in terms of how the agent detects it (exit code vs. output), and 'ask the user to inspect recovery after any active writer finishes' is vague about what the agent should actually say or do. The confirmation message uses `<path>/` without saying whether to echo the user-typed path or the resolved physical absolute path. No guidance on nonexistent paths or empty input. Jargon like 'serializes replacement with investigation cleanup' adds noise. Overall executable but with a soft edge on error handling."},"latency_ms":12426,"stop_reason":"end_turn","usage":{"input_tokens":1709,"output_tokens":809},"cost_usd":0.05754}
+{"config":"workflow-default","item":"a06","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":3,"reasoning":"Steps 2-6 are mostly unambiguous: detection commands are concrete, per-platform branches are enumerated, multi-platform conflicts are resolved by asking the user, and Step 4 explicitly sweeps up missing fields across all paths with a confirm-before-write gate. Weaknesses: the CLAUDE.md output has no defined schema/template or section marker, so the persisted deliverable's shape (needed by downstream automation) is left to the agent; several sub-steps are underspecified (how to decide Fly '/health' exists, what to do with the Render API key after checking it, how to obtain Vercel's production URL from 'project settings', how to determine repo-allowed merge methods); the Step 5 status-command check is buggy (pipe to head masks the exit code so COMMAND_FAILED never fires); and Step 6 gives no summary structure. Workflow is executable end-to-end but produces an inconsistently formatted artifact and relies on agent judgment in several spots."},"latency_ms":24166,"stop_reason":"end_turn","usage":{"input_tokens":2830,"output_tokens":1504},"cost_usd":0.1035}
+{"config":"workflow-default","item":"f01","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The workflow has serious internal contradictions and gaps. Step 3 is missing entirely, and the local-vs-remote decision that Step 4 branches on ('If same machine (option A)' / 'If different machine (option B)') is never asked anywhere — Step 2 only asks which agent to pair. Worse, the 'option A/B' labels collide with the live-daemon consent options (A=Relaunch, B=Keep daemon), so an agent cannot tell which 'option A' is meant. The live-daemon consent gate recommends keeping the daemon, then immediately says 'Always pass --force-restart to the commands below so pairing starts from a clean daemon' — directly contradicting the consent gate and the stated iron rule, and none of the subsequent commands actually include the flag. Step 1's advice to 'run Step 5 (Verify connection) first and come back to pairing later' is nonsensical since there is nothing to verify before pairing. Step 5 assumes a remote paste flow and does not fit local pairing. The restrict section says read,write means 'no JS' while also noting 'eval works over the tunnel,' muddying the scope semantics. The remote-path ngrok handling, consent gate, and output-block requirement are well specified, and the commands are concrete, but an agent following this could unintentionally kill a live daemon or be unable to decide which branch to execute, so overall it is not reliably actionable."},"latency_ms":17203,"stop_reason":"end_turn","usage":{"input_tokens":3712,"output_tokens":1369},"cost_usd":0.10557}
+{"config":"workflow-default","item":"p02","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The skill is a single, well-structured entrypoint with explicit mode detection (save vs. list, redirecting resume/restore), concrete bash blocks for gathering git state, computing a collision-safe sanitized filename, a fully specified frontmatter/markdown template, a stamp step, and a defined confirmation banner; the list flow specifies ordering rationale, branch filtering default vs --all, table formats, and the empty case. Weaknesses: the Step 4 and List bash blocks redundantly re-derive GSTACK_STATE_ROOT/SLUG (sloppy copy-paste), the title must be injected via TITLE_RAW with only a loosely described invocation pattern, the session-duration block uses macOS-only `date -jf` and relies on an undefined `_TEL_START`/PPID that is unreliable in a tool subshell (though it degrades to 'unknown'), `files_modified` is ambiguous about untracked files, and there is no guidance for a non-git directory or a failed SLUG lookup. The 'completed' status in the example table has no origin in this skill. None of these block execution, so an agent can reliably produce the deliverables."},"latency_ms":17822,"stop_reason":"end_turn","usage":{"input_tokens":4636,"output_tokens":1137},"cost_usd":0.10321}
+{"config":"workflow-default","item":"p01","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The six steps are linear with clear decision branches, concrete commands, and defined AskUserQuestion options, so an agent can execute the flow end to end. Gaps undermine completeness: Step 6 instructs a full re-run with `--output json` to save a baseline, doubling cost/time and risking a baseline that differs from the table the user just saw, with no slug/date format or `mkdir -p` for `~/.gstack/benchmarks` specified. Step 1A's `find . -maxdepth 2 -name SKILL.md` assumes the current directory is the gstack install, while Step 0 locates the binary under `$HOME/.claude/skills/gstack`, leaving the SKILL.md path resolution ambiguous when run from a user project. Error-to-remediation mapping in Step 5 is also left to agent judgment."},"latency_ms":15901,"stop_reason":"end_turn","usage":{"input_tokens":3084,"output_tokens":1068},"cost_usd":0.08424}
+{"config":"workflow-default","item":"p01","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The workflow is linear and each step has a concrete command or AskUserQuestion with explicit options and a recommendation, so an agent can follow it with little ambiguity. Weak spots: Step 1A's `find . -maxdepth 2` assumes the CWD is the gstack repo without saying so; Step 2C and the mapping of user-chosen names to the `claude,gpt,gemini` identifiers is left implicit, and there is no guidance if the user picks an unauthed provider; Step 3's bash one-liner relies on shell operator precedence and could mislead; Step 5's 'best overall' weighting is undefined; Step 6 asks to re-run the entire benchmark for JSON (duplicating cost/time and producing a baseline that may not match the displayed table), leaves the date/slug format unspecified, and omits creating the output directory. No handling of a non-zero exit from the main run. Still, the deliverables (table, summary, optional JSON baseline) are clear and achievable."},"latency_ms":17174,"stop_reason":"end_turn","usage":{"input_tokens":2687,"output_tokens":1067},"cost_usd":0.08022}
+{"config":"workflow-default","item":"a06","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Steps 2–6 are well-sequenced and mostly unambiguous: detection script, per-platform branches, explicit handling for multi-platform detection, non-deploying projects, and Heroku/Railway falling through to the manual questions. Weaknesses: Step 4 says to write to CLAUDE.md but never defines the field names, section heading, or format of the persisted config (critical since it is meant for later automated consumption), nor how to handle pre-existing deploy config in CLAUDE.md; 'constrained to methods allowed by repo settings' gives no way to look those up; Fly's '/health if the app has one' has no detection method; Step 5's `cmd | head -5 || echo COMMAND_FAILED` won't fire because the pipeline exit status is head's. Step 1 is absent from the bundle, so prerequisites/entry context are unknown. Otherwise an agent could execute the flow and reach a deliverable, with the main risk being an inconsistent CLAUDE.md schema."},"latency_ms":26665,"stop_reason":"end_turn","usage":{"input_tokens":2830,"output_tokens":1180},"cost_usd":0.0873}
+{"config":"workflow-default","item":"p02","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The skill is self-contained and well-structured: mode detection, git state gathering, summary template, bash-side sanitized collision-safe path computation, explicit file format, stamp step, confirmation output, and a list flow with sort rationale and table formats. Minor issues: the Step 4 bash block redundantly re-fetches GSTACK_STATE_ROOT and its comment references `${SLUG}` where the code uses `${TITLE_SLUG}`; the duration computation uses BSD-only `date -jf` (gracefully degrades to unknown); the list flow filters by 'current branch' but never explicitly re-gathers the branch, and title parsing from filename doesn't address the random collision suffix; the `status: completed` value in the example table has no path to be set within this skill. None of these block execution, but they introduce small ambiguities."},"latency_ms":14218,"stop_reason":"end_turn","usage":{"input_tokens":4636,"output_tokens":941},"cost_usd":0.09341}
+{"config":"workflow-default","item":"a02","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The workflow is linear and mostly unambiguous: AskUserQuestion with text input, run freeze-state.sh set, confirm with a fixed message. The hook explanation and fail-closed semantics are well described. Gaps: the error path is underspecified — the agent isn't told how FREEZE_BUSY is signaled (exit code vs. stdout) or what 'ask the user to inspect recovery' concretely means, and non-BUSY failures (nonexistent path, helper nonzero exit) have no defined handling. Also unclear whether the confirmation message should echo the user's raw path or the resolved absolute path the helper computes. Despite these, an agent can execute the happy path and produce the expected deliverable."},"latency_ms":11008,"stop_reason":"end_turn","usage":{"input_tokens":2106,"output_tokens":678},"cost_usd":0.05496}
+{"config":"workflow-default","item":"p03","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":3,"reasoning":"Routing, show/search/stats, and manual add are concrete with runnable commands and a well-defined dedup script. Weaknesses cluster in Prune and Export. Prune tells the agent to remove lines from learnings.jsonl but only Stats defines that file's path (GSTACK_STATE_ROOT/projects/$SLUG/learnings.jsonl), and the contradiction check flags any same-key entries with different insights even though the model is explicitly append-only with latest-wins—so legitimate updates would be flagged as conflicts and removal contradicts the append-only rule. Export's template omits the 'tool' type that Manual add permits, and Manual add always emits a files array despite files being optional. SLUG is computed but unused in several blocks, and USER_QUERY quoting is unspecified."},"latency_ms":36138,"stop_reason":"end_turn","usage":{"input_tokens":3202,"output_tokens":792},"cost_usd":0.07162}
+{"config":"workflow-default","item":"a02","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The flow is short and linear: AskUserQuestion with text input, invoke freeze-state.sh set, report only on success, deliver a fixed confirmation string. The fail-closed hook explanation and symlink/trailing-slash notes clarify behavior well. Gaps: the error path is vague — \"preserve it and ask the user to inspect recovery after any active writer finishes\" doesn't say what to tell the user or how FREEZE_BUSY is signaled (exit code vs stdout), and non-busy failures like a nonexistent or relative path have no specified handling. The confirmation message's `<path>` is also ambiguous between the user's input and the resolved physical path. Overall executable with minor decision points left to the agent."},"latency_ms":12872,"stop_reason":"end_turn","usage":{"input_tokens":2106,"output_tokens":771},"cost_usd":0.05961}
+{"config":"workflow-default","item":"a04","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":3,"reasoning":"Steps 1–3 are well-written and concrete: intent capture with a single clarifying question, an explicit verb list for refusing mutations, and ready-to-run `aside repl`/`aside exec` scripts with sensible quoting and bridge-serialization rules. However, the workflow stops at the read step. There is no step defining what to do when a read fails (budget of 3–4 attempts exhausted, empty page from a crashed JS app, timeouts) and no step specifying the final deliverable — the intro promises 'one JSON document' but the structured path yields {items,count}, the fuzzy path {answer,sources}, and nothing unifies or describes what is actually handed back. Minor clarity issues: 'Rule 5' and 'Rule 4' are cited but the quoted untrusted-content block only has rules 1–4 and its Rule 4 is about prompt injection, not sign-in, so those references don't resolve within the bundle. An agent can execute the scripts but must improvise failure handling and output shape."},"latency_ms":19765,"stop_reason":"end_turn","usage":{"input_tokens":3219,"output_tokens":1250},"cost_usd":0.09469}
+{"config":"workflow-default","item":"a02","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is short and linear: log analytics, ask the user for a directory via AskUserQuestion (text input, exact question text given), invoke the shared freeze-state.sh helper with the path, report success only if the helper succeeds, and deliver a scripted confirmation message. The 'How it works' and Notes sections give useful context (fail-closed polarity, symlink resolution, trailing slash, Edit/Write-only scope, persistence until /unfreeze) so the agent can answer follow-up questions accurately. Weaknesses: failure handling is thin and somewhat vague — 'FREEZE_BUSY or unexpected state' is not fully specified, and 'ask the user to inspect recovery after any active writer finishes' does not say what message to give or what 'recovery' refers to; there is no guidance for non-BUSY helper errors (e.g., nonexistent path) beyond not reporting success. It is also slightly ambiguous whether the confirmation message should echo the user-typed path or the helper-resolved physical absolute path. Otherwise the agent can execute this and produce the expected result without difficulty."},"latency_ms":15191,"stop_reason":"end_turn","usage":{"input_tokens":1709,"output_tokens":1009},"cost_usd":0.06754}
+{"config":"workflow-default","item":"f05","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The bundle is well-structured through Step 8, but the core gate is broken: Step 8 says 'if the test fails, continue to Step 10 anyway; a failing test can be fixed after the skill is committed,' which directly contradicts the Iron contract ('only renames into the final tier path on (a) test pass + (b) explicit user approval... on either failure, the temp dir is removed'). Step 9 — the explicit user-approval question — is missing entirely, yet Step 10 branches on 'if the user approved' / 'if the user rejected in step 9,' so the agent has no defined decision point for the approval the whole workflow hinges on, and no defined discard path for test failure. Minor inconsistencies compound this (Step 3 says the SDK copy is 'written in step 6' when staging happens in step 7; Step 7 says to pass stagedDir to '$B skill test' while Step 8 says that command can't see staged dirs). An agent can synthesize, stage, and run the test, but cannot reliably execute the commit/discard decision as specified, and following Step 8 literally violates the stated safety guarantee."},"latency_ms":15185,"stop_reason":"end_turn","usage":{"input_tokens":5269,"output_tokens":1109},"cost_usd":0.10814}
+{"config":"workflow-default","item":"f05","split":"dev","phase":"old-a","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Steps 1-7 are detailed and mostly followable (good provenance guard, concrete templates, frontmatter contract, staging call). But the core gate is broken: Step 8 says 'If the test fails, continue to Step 10 anyway; a failing test can be fixed after the skill is committed', directly contradicting the Iron contract (commit only on test pass + explicit approval, otherwise discard). Step 9 — the explicit user-approval question — is entirely missing, yet Step 10 references 'If the user rejected in step 9', so the agent has no defined approval prompt or decision to make before commitSkill. Minor issues too: Step 6's 'two dirname calls above _lib/browse-client.ts' is arithmetically wrong for the stated path layout, and Step 4's /tmp fixture vs Step 7's fixtureHtml variable handoff is loosely specified. As written, an agent could commit a failing, unapproved skill, violating the workflow's stated purpose."},"latency_ms":16014,"stop_reason":"end_turn","usage":{"input_tokens":5269,"output_tokens":1114},"cost_usd":0.10839}
+{"config":"workflow-default","item":"a05","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":3,"reasoning":"Capture scripts, cadence rules, report template, JSON schema, and JSONL logging are concrete and reproducible. However, the core detection step is vague: Phase 5 says only 'raise an alert at whatever severity seems right if a page looks noticeably worse,' giving no thresholds for load regression, new console messages, text disappearance, or 404s, despite Phase 2 collecting exactly those signals. Phase 6 then keys verdicts to 'confirmed' CRITICAL/HIGH/MEDIUM alerts and an 'observations' array for 'unconfirmed transients,' but 'confirmed' and 'serious alert' are never defined, so BROKEN/DEGRADED status is left to agent judgment. Phase ordering (baseline before discovery, Phase 2 citing Phase 3's check) is workable but awkward."},"latency_ms":14086,"stop_reason":"end_turn","usage":{"input_tokens":4927,"output_tokens":935},"cost_usd":0.09602}
+{"config":"workflow-default","item":"a03","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"The supplied excerpt is strong in places: the prerequisite check, agent-selection question, live-daemon consent gate, tunnel consent gate, ngrok install/auth branches, and the scope/permission reference are detailed and well-reasoned, with explicit security handling of the authtoken. However, the bundle jumps from Step 2 to Step 4 — the local-vs-remote decision (Step 3) that the workflow depends on is absent, yet Step 4 branches on 'If same machine (option A)' / 'If different machine (option B)', labels that collide with Step 2's A–E agent options and the consent-gate A/B options, creating real ambiguity about which 'option A/B' is meant. Other gaps: no verification step after pairing (the task calls for verifying the connection, but the flow ends at 'tell the user'); the generic/E target has no defined TARGET_HOST value to pass to `pair-agent`; 'pairing headed' is referenced without any headed/headless mode decision; and `--force-restart` is said to be passed 'to the commands below' but no shown command demonstrates it. An agent could execute the ngrok and consent branches reliably but would have to guess at the local/remote routing and the completion criteria."},"latency_ms":15458,"stop_reason":"end_turn","usage":{"input_tokens":3750,"output_tokens":985},"cost_usd":0.08675}
+{"config":"workflow-default","item":"a03","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"The supplied workflow jumps from Step 2 to Step 4; the local-vs-remote decision that Step 4 depends on is never asked, yet Step 4 branches on \"option A\" (same machine) and \"option B\" (different machine) while Step 2's A–E letters mean agent choice and the daemon-consent prompt reuses A/B again—three conflicting meanings for the same labels. No verification step exists after pairing (no connection check, only a canned success message). Positives: concrete commands, well-specified consent gates for daemon relaunch and tunnel enablement, strong ngrok auth/security handling, and clear scope semantics. An agent can likely execute pairing, but must infer the missing mode decision and has no defined way to confirm the remote agent actually connected."},"latency_ms":11336,"stop_reason":"end_turn","usage":{"input_tokens":4147,"output_tokens":631},"cost_usd":0.07302}
+{"config":"workflow-default","item":"a06","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":3,"reasoning":"The detection script and per-platform branches (Fly, Render, Vercel, Netlify, Heroku/Railway→custom, GHA-only, custom) are mostly concrete and sequenced sensibly, with explicit confirm-before-write and non-blocking verification. Multi-platform disambiguation and the no-deploy case are handled. However, the key deliverable is underspecified: Step 4 says 'Write the confirmed configuration into CLAUDE.md' with no schema, section header, field names/format, or guidance on merging with existing CLAUDE.md content, so two agents could produce incompatible outputs that downstream automation may not parse. Several sub-steps are hand-wavy ('or /health if the app has one', 'production URL from vercel project settings', 'constrained to methods allowed by repo settings' with no command given). The GHA grep on 'cd' will over-match, and Step 6 gives no structure for the summary. Overall followable but with notable gaps in output definition and a few ambiguous decision points."},"latency_ms":18713,"stop_reason":"end_turn","usage":{"input_tokens":2830,"output_tokens":1285},"cost_usd":0.09255}
+{"config":"workflow-default","item":"p05","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The six-step flow is well ordered with concrete scripts, a defined retirement card, a retention bar, low-value catalog, caller-check command, report contents, a sidecar schema and an explicit A/B/C batch question with a recommendation rule, so an agent can execute it and produce the deliverables. Weaknesses: the Boundaries section mixes in material irrelevant to auditing existing tests (regression-proof recording, value-card byte clamping) and duplicates a setup line; Step 1 references an undefined `$BRANCH` in the SEED_PLAN lookup and gives no fallback when DEFAULT_BRANCH is unknown for the >300-file `--since` default; `--since` scoping is described but Step 2's `git ls-files` shortlist has no mechanism to apply it; the Step 1 and Step 2 test-file regexes differ (Java/Kotlin counted but never shortlisted); the sidecar puts a `retirement_card` on every candidate including `retain` verdicts while also having a separate `retained` array, leaving placement ambiguous; `loc_delta` is a single total though the report requires per-batch figures; and Step 5 loops questions over all batches while Step 6 applies 'an approved batch' and reruns discovery after landing, leaving multi-approval sequencing unclear."},"latency_ms":31263,"stop_reason":"end_turn","usage":{"input_tokens":5101,"output_tokens":2395},"cost_usd":0.17076}
+{"config":"workflow-default","item":"p03","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Command routing is unambiguous and each subcommand has concrete bash plus presentation guidance, so an agent can execute show/search/export/stats/add readily. Gaps: Prune tells the agent to edit learnings.jsonl for removals but only the Stats section defines LEARN_FILE, so the path must be inferred; removal is in-place while updates are append-only ('latest wins'), which is inconsistent and means deleting the latest line can resurface an older superseded entry; the contradiction check relies on gstack-learnings-search output, which per the Stats dedup logic likely collapses same-key entries, so conflicts may never surface. Export omits the 'tool' type that Manual add allows, and Manual add doesn't say how to handle the optional files field being empty or what to confirm afterward. Unrecognized subcommands and query escaping are unhandled."},"latency_ms":14562,"stop_reason":"end_turn","usage":{"input_tokens":2805,"output_tokens":979},"cost_usd":0.077}
+{"config":"workflow-default","item":"a06","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":3,"reasoning":"Steps 2–6 are sequenced sensibly, detection commands are concrete, and key decisions (multiple platforms, non-deploying projects, Heroku/Railway fallback) are explicitly resolved. The main gap is the deliverable itself: Step 4 says 'write the confirmed configuration into CLAUDE.md' but never defines the section name, key names, or format, so two agents would produce incompatible files that downstream automation cannot reliably parse. Several sub-steps are also under-specified or inert—e.g., Render checks `$RENDER_API_KEY` yet never uses it, and 'merge method constrained to methods allowed by repo settings' gives no command to discover those settings. Minor shell issue: the pipeline `| head -5 || echo COMMAND_FAILED` never triggers on command failure."},"latency_ms":15667,"stop_reason":"end_turn","usage":{"input_tokens":3227,"output_tokens":1022},"cost_usd":0.08337}
+{"config":"workflow-default","item":"p02","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: mode detection, git state capture, bash-side path/title sanitization, collision-safe filename, explicit file template, stamp step, and a list table with branch filtering. Minor gaps hold it back. The `--all` flag only appears in List Step 2, not in the Detect-command parsing, and the `status` field is always written as `in-progress` while the sample table shows `completed` with no defined transition. The duration script uses macOS-only `date -jf` (falls back gracefully), the `TITLE_RAW` hand-off is only shown as a comment, and `GSTACK_STATE_ROOT` is redundantly recomputed. None of these block execution; an agent can reliably produce the saved file and the list output."},"latency_ms":13627,"stop_reason":"end_turn","usage":{"input_tokens":5033,"output_tokens":908},"cost_usd":0.09573}
+{"config":"workflow-default","item":"f05","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The templates for script.ts, script.test.ts, SKILL.md frontmatter, and the stage/commit/discard calls are concrete and well-specified, but the core gating logic is broken. Step 9 (the explicit user-approval question) is absent entirely, yet Step 10 branches on 'if the user approved' and 'if the user rejected in step 9', leaving the approval decision undefined. Worse, Step 8 instructs the agent to proceed to commit even when the staged test fails ('a failing test can be fixed after the skill is committed'), which directly contradicts the Iron contract that requires test pass + approval and mandates discarding the temp dir on failure. Smaller defects: Step 3 says the SDK copy is 'written in step 6' when step 6 only reads it and step 7 writes it; the install-dir resolution in Step 6 gives an incorrect dirname count and the example code omits the first (primary) signal; decision labels (D<N>) are inconsistent. An agent following this literally could commit a half-broken skill with no approval step, defeating the stated purpose."},"latency_ms":16860,"stop_reason":"end_turn","usage":{"input_tokens":5269,"output_tokens":1147},"cost_usd":0.11004}
+{"config":"workflow-default","item":"p04","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":3,"reasoning":"The five steps are sequentially ordered and mostly unambiguous: base-branch detection, version reads, per-level queue queries, dashboard rendering with a fixed template, and a three-case next-action rule. However, several rendering inputs are undefined: how to obtain <owner/repo> and <workspace_root>, how to compute collision markers from `.claimed`, how to derive the per-level annotations ('queue-advance: none', 'bumped past claimed'), which sibling fields map to the table columns, and where the offline '<reason from warnings>' comes from (`.warnings` is not among the listed jq extractions). The offline/unknown-host branch is ambiguous (an unknown host may still be online; partial offline across levels is unaddressed). Step 5's 'outranks' comparison and priority when multiple conditions hold are implicit. The Step 2 shell has a latent bug: `|| echo` after a `tr` pipeline never triggers, so fallbacks yield empty strings rather than defaults. Workflow is executable but requires agent inference to produce the exact dashboard."},"latency_ms":17888,"stop_reason":"end_turn","usage":{"input_tokens":2463,"output_tokens":1204},"cost_usd":0.08483}
+{"config":"workflow-default","item":"a06","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The detection script and per-platform branches are concrete and mostly unambiguous, multi-platform detection is explicitly resolved by asking the user, non-deploying projects have a defined outcome, and the Custom/Manual AskUserQuestion set covers trigger, URL, status check, and hooks. Step 4 sensibly consolidates missing fields (merge method, pre-merge command) across all paths and requires confirmation before writing. Gaps: the actual CLAUDE.md output format/section structure for the configuration is never specified in this excerpt, so the primary deliverable's shape is left to the agent; 'constrained to methods allowed by repo settings' gives no command to discover those; Fly's '/health if the app has one' and Render's 'poll until it responds with the new version' are hand-wavy; the workflow grep pattern ('cd') will produce false positives; and Step 6 is a bare stub. Step 1 is outside the supplied range so not penalized."},"latency_ms":19633,"stop_reason":"end_turn","usage":{"input_tokens":2830,"output_tokens":1269},"cost_usd":0.09175}
+{"config":"workflow-default","item":"a01","split":"dev","phase":"old-b","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The phased structure, Iron Law, fixture paths, and failure-mode table are clear and concrete, and the deliverables (pre/post snapshot+screenshots, minimal Swift diff, regression test, commit) are well specified. Weak spots: Phase 3 is muddled — `xcodebuild ... build install` is followed by 'Re-deploy. The same boot-token rotation flow runs' with no indication of what re-deploy means beyond install or what the rotation flow is; Phase 4's 'revert and try again' doesn't say whether to loop to Phase 2 (new hypothesis) or Phase 3 (new edit), and the pass/fail criterion is a subjective screenshot comparison with no guidance; Phase 1 step 5 says to 'persist' a description without saying where; and there's no handling for the case where the bug can't be reproduced via snapshot-eligible state or where the pre-snapshot fails to reproduce after restore. Phase 2 defers almost entirely to /investigate. Still, an agent with the referenced external tools could execute this end to end and produce the expected artifacts."},"latency_ms":17235,"stop_reason":"end_turn","usage":{"input_tokens":1716,"output_tokens":1202},"cost_usd":0.07726}
+{"config":"workflow-default","item":"p01","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is linear, well-numbered, and gives concrete bash commands and AskUserQuestion scaffolds for every decision point (prompt source, providers, judge, save). Edge cases like all-providers-unauthed and missing binary are handled with explicit stop messages. Weaknesses: Step 1A's `find . -maxdepth 2` assumes the cwd is the gstack install dir without saying so; Step 2 option C doesn't specify how user-named models map to the `claude,gpt,gemini` identifiers; Step 6's 'save' path re-runs the entire (slow, paid) benchmark rather than capturing the first run's JSON, and `<date>`/slug formats are undefined; there's no guidance if the user picks 'Only Claude' when Claude is the unauthed one. These gaps are minor relative to an otherwise executable, well-specified flow."},"latency_ms":17438,"stop_reason":"end_turn","usage":{"input_tokens":2687,"output_tokens":1114},"cost_usd":0.08257}
+{"config":"workflow-default","item":"p03","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Command routing is explicit and each subcommand has a concrete script and presentation step, so an agent can execute most paths directly. Weaknesses: Prune tells the agent to edit learnings.jsonl but never defines its path (only Stats derives LEARN_FILE), and its 'remove the line' instruction sits awkwardly against the stated append-only/latest-wins model; the contradiction check (same key, different insight) will flag legitimate updates that Stats explicitly dedups by key|type latest-wins. Export's markdown template omits the 'tool' type that Manual add allows, and Manual add always emits a files array even when optional and gives no guidance on JSON escaping. Edge cases (search with no query, add with inline arguments) are unaddressed. Still, the core deliverables are well-defined and achievable."},"latency_ms":14091,"stop_reason":"end_turn","usage":{"input_tokens":2805,"output_tokens":891},"cost_usd":0.0726}
+{"config":"workflow-default","item":"a01","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The five phases are concrete and sequenced, with explicit commands, fixture paths, an iteration cap, and a failure-mode table. Gaps remain: Phase 1 step 5 says to 'persist' the bug description without specifying where, and Phase 4 step 2 says to compare the fresh screenshot against the pre.png without defining what counts as 'bug visibly persists' versus fixed. Phase 3 steps 3-4 ('Daemon detects... Re-deploy. The same boot-token rotation flow runs') are redundant with 'build install' and leave the agent unsure whether a separate deploy action is required. No branch covers failure to reproduce in Phase 1, and the regression test's assertion content is left open. Still largely executable to the stated deliverables."},"latency_ms":11221,"stop_reason":"end_turn","usage":{"input_tokens":2113,"output_tokens":818},"cost_usd":0.06203}
+{"config":"workflow-default","item":"a06","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"Detection script and per-platform branches are concrete and mostly unambiguous, with explicit handling for multi-platform detection, Heroku/Railway fallthrough, GitHub-Actions-only, and the no-deploy case. Step 4 sensibly gates on confirmation and defines 'none' semantics. Gaps: the actual CLAUDE.md output format/schema (section name, field keys) is never specified despite the config being meant for automated downstream use; no handling for an existing deploy config in CLAUDE.md (merge/overwrite); several extraction steps are hand-wavy (parse render.yaml service name, Vercel 'project settings' URL, deciding whether a Fly app 'has' /health, checking repo-allowed merge methods without a command). The 'cd' regex in workflow detection is noisy. Step 1 is absent from the excerpt but not penalized. An agent could execute this and produce a usable deliverable, but would have to invent the persisted format."},"latency_ms":25866,"stop_reason":"end_turn","usage":{"input_tokens":2830,"output_tokens":1339},"cost_usd":0.09525}
+{"config":"workflow-default","item":"p05","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: read-only discovery, mechanical shortlist, evidence-backed retirement cards, report+sidecar schema, one-question-per-batch gating, and strict apply rules are all explicit. Gaps are mostly wiring: `--since <ref>` is defined in Step 1 but Step 2's `git ls-files -- <scope>` never incorporates it, so the agent must improvise the changed-file filter; and the SEED_PLAN `ls` references `$BRANCH`, which this script never sets (only DEFAULT_BRANCH), so seeding may silently fail unless the preamble defines it. Minor noise: the duplicated GSTACK_STATE_ROOT line, Step 1/Step 2 test-file regexes differ (java/kt), and the 8-minute budget has no measurement mechanism. Sequencing of multiple approved batches versus /ship landing is slightly underspecified but workable."},"latency_ms":19124,"stop_reason":"end_turn","usage":{"input_tokens":5498,"output_tokens":1485},"cost_usd":0.12923}
+{"config":"workflow-default","item":"f04","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 0, 1, 7, and 9 are concrete and well-specified (scope confirmation, research checklist with concept-map outline, cross-linking rules, redaction scan, commit template, summary format). However the core of the workflow — Steps 2-6, which should partition content into Diataxis quadrants and write each from templates — is collapsed into a single sentence: 'Write whatever documentation seems most useful... in whatever structure fits.' This is not an on-demand reference; it is the absence of the central instructions. Later steps then depend on undefined concepts (quadrants, 'every reference doc should link to its how-to', PR table by quadrant, per-quadrant coverage counts), so the agent has no basis to produce the expected deliverables. Step 8 is internally contradictory: it says the self-review happens after Step 9 has committed and pushed, is 'optional,' and failures 'can be fixed in a later PR,' while Important Rules declare accuracy 'non-negotiable' and Step 9's summary requires reporting pass/fail on each gate before the review would have occurred. These ordering contradictions and the missing partition/template steps make the workflow ambiguous and unable to reliably yield structured Diataxis documentation."},"latency_ms":12118,"stop_reason":"end_turn","usage":{"input_tokens":3495,"output_tokens":964},"cost_usd":0.08315}
+{"config":"workflow-default","item":"f02","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 1 and 2 are detailed and mostly unambiguous (detection, honest exit-code capture, SKIPPED vs ERROR semantics), though the detection script has precedence bugs (`[ ] || [ ] && echo` chains, pylintrc/pyproject logic) and the template Health Stack list doesn't match detection output (no gbrain line). However, Steps 3–6 are one-sentence placeholders: no category weights, no per-category scoring formula, no composite calculation, no dashboard format, no history file path/schema, no trend comparison or recommendation rules. Step 2 even references composite/weights and a 'numeric history row' that are never defined anywhere in the bundle, and no Read instructions point to on-demand files for them. An agent could run the tools but could not produce the expected scored dashboard, persisted history, or trends without inventing most of the workflow."},"latency_ms":11219,"stop_reason":"end_turn","usage":{"input_tokens":2948,"output_tokens":694},"cost_usd":0.06418}
+{"config":"workflow-default","item":"p01","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The six steps are sequential, each with explicit commands, AskUserQuestion options, and clear branching (e.g., stop if all adapters NOT READY; omit --judge if credentials absent). Command construction in Step 4 cleanly maps Step 1/2/3 outputs. Weaknesses: Step 1A's `find . -maxdepth 2 -name SKILL.md` assumes cwd is the gstack install dir, which conflicts with the Step 0 fallback to `$HOME/.claude/skills/gstack`, so skill discovery may silently return nothing. Step 6 instructs a full re-run with `--output json` to save a baseline, doubling cost/time and producing results that differ from the table just interpreted, without specifying slug derivation or directory creation. Otherwise executable and deliverables are well-defined."},"latency_ms":13945,"stop_reason":"end_turn","usage":{"input_tokens":3084,"output_tokens":951},"cost_usd":0.07839}
+{"config":"workflow-default","item":"f02","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":2,"reasoning":"Steps 1-2 are precise: detection script, AskUserQuestion options, honest capture with exit-code preservation, SKIPPED/ERROR/CLEAN rules. But Steps 3-6 collapse to one-sentence placeholders. Step 3 says to score 'based on how healthy the output looks' with no weights, thresholds, or composite formula, even though Step 2's 'do not redistribute that category's weight' presupposes defined weights. Step 5 says 'save the results somewhere' with no file path, schema, or the numeric history row format Step 2 references. The dashboard format, trend comparison logic, and recommendation criteria are entirely undefined, so an agent can run tools but cannot produce a consistent, comparable deliverable."},"latency_ms":10070,"stop_reason":"end_turn","usage":{"input_tokens":3345,"output_tokens":596},"cost_usd":0.06325}
+{"config":"workflow-default","item":"a04","split":"dev","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":3,"reasoning":"Steps 1–3 are crisp and concrete: intent capture with a single clarifying prompt, an explicit verb list and canned refusal for mutating intents, and two well-specified read paths (look-then-extract with ready-to-adapt `aside repl` scripts, or `aside exec` for fuzzy intents) plus useful bash-quoting and JSON-bridge caveats. However, the workflow the agent is supposed to learn also includes handling failed reads and returning one JSON document, and neither is defined here: there is no step for what to do when the 3–4 attempt budget is exhausted, when `waitForSelector` times out, or when the page is empty beyond noting CONSOLE_ERRORS; and there is no final-output step unifying the structured `{items,count}` result and the fuzzy `{answer,sources}` result into the 'one JSON document' promised in the intro. Minor ambiguities: references to 'Rule 4' and 'Rule 5' don't match the four numbered untrusted-content rules in the bundle (Rule 4 is about prompt injection, not sign-in), so the agent can't resolve them without external context. The scripts are executable as written, but the missing failure/output steps leave the deliverable under-specified."},"latency_ms":21005,"stop_reason":"end_turn","usage":{"input_tokens":3219,"output_tokens":1167},"cost_usd":0.09054}
+{"config":"workflow-default","item":"f04","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Steps 0, 1, 7, and 9 are concrete and well-specified (scope confirmation, archaeology commands, concept map template, cross-linking rules, redaction scan, commit format, summary output). However, the core of the workflow—Steps 2-6, which should cover partitioning content into Diataxis quadrants, naming conventions, and writing each doc from templates—is collapsed into a single vague line ('write whatever documentation seems most useful... in whatever structure fits'). This leaves the primary deliverable undefined even though later steps (cross-links 'between quadrants', the summary's quadrant counts, the PR table) assume quadrant-specific files exist. Step 8 is internally contradictory: it states the self-review happens after Step 9 has committed and pushed, makes the review 'optional' with failures deferred to a later PR, yet Step 9's summary requires pass/fail on each gate and the Important Rules declare accuracy 'non-negotiable.' These ordering conflicts and the missing central decision logic mean an agent cannot reliably produce the expected Diataxis output despite solid bookends."},"latency_ms":10881,"stop_reason":"end_turn","usage":{"input_tokens":3495,"output_tokens":770},"cost_usd":0.07345}
+{"config":"workflow-default","item":"a02","split":"dev","phase":"new","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is linear and mostly unambiguous: ask via AskUserQuestion with text input, call freeze-state.sh set with the path, report success only on helper success, then deliver a fixed confirmation message. Enforcement semantics (fail-closed, symlink resolution, hookSpecificOutput nesting) are explained well. Gaps are in failure handling: 'On FREEZE_BUSY or unexpected state, preserve it and ask the user to inspect recovery' leaves 'unexpected state' undefined and gives no concrete user-facing wording. Also, non-busy helper failures (e.g., nonexistent or relative path the helper rejects) have no specified response, so the agent must improvise. Otherwise the deliverables and outputs are well-defined and executable."},"latency_ms":12069,"stop_reason":"end_turn","usage":{"input_tokens":2106,"output_tokens":733},"cost_usd":0.05771}
+{"config":"workflow-default","item":"p05","split":"dev","phase":"new","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":4,"reasoning":"The workflow is well-structured: boundaries, a concrete low-value catalog and retention bar, a fully enumerated retirement card, runnable pre-filter commands, hard evidence budgets, an explicit sidecar schema, and a fixed A/B/C question protocol. Gaps are mechanical rather than conceptual. Example: the Step 1 SEED_PLAN glob references `$BRANCH`, which is never defined (only `DEFAULT_BRANCH` is), so seed-plan detection silently fails; the two test-file regexes also disagree (Step 1 includes `Test.(java|kt)`, Step 2 omits it). Example: `--since <ref>` is a documented scope mode and the >300-file default, yet no command shows how to restrict `FILES` to files changed since the ref, and per-batch LOC deltas required by the report have no computation method or sidecar field. An agent can still produce the report and batches with modest interpretation."},"latency_ms":18850,"stop_reason":"end_turn","usage":{"input_tokens":5498,"output_tokens":1417},"cost_usd":0.12583}
+{"config":"workflow-default","item":"a02","split":"dev","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is short and linear: log analytics, prompt via AskUserQuestion (text input, exact question given), call the shared freeze-state.sh helper with the path, report success only on helper success, and deliver a prescribed confirmation message. The 'How it works' and Notes sections give the agent useful context (fail-closed polarity, symlink resolution, Edit/Write-only scope). Minor gaps: the confirmation template uses `<path>/` without saying whether to echo the user's raw input or the helper's resolved absolute path; the failure branch ('preserve it and ask the user to inspect recovery after any active writer finishes') is vague about what message to give and how FREEZE_BUSY vs other errors are detected; and there's no guidance for an empty/invalid user response. None of these block execution, but they leave small ambiguities in outputs and error handling."},"latency_ms":13492,"stop_reason":"end_turn","usage":{"input_tokens":1709,"output_tokens":874},"cost_usd":0.06079}
+{"config":"workflow-default","item":"f05","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The workflow is well-structured through Step 8 (provenance guard with exact refusal text, name/tier AskUserQuestion template, pure-parser script skeleton, fixture capture, test template, SDK resolution, stageSkill call with frontmatter contract). However, Step 8 directly contradicts the 'Iron contract': it says to continue to commit even if the staged test fails and 'fix it after', whereas the contract requires test pass + approval before any write and removal of the temp dir on either failure. There is no discard-on-test-failure branch at all. Step 9 — the explicit user-approval gate — is entirely missing (numbering jumps 8→10), yet Step 10 references 'if the user approved' and 'if the user rejected in step 9', so the agent has no defined question, options, or decision criteria for the approval step that the task description names as central. These gaps mean an agent could ship a half-broken skill or stall at an undefined decision point, undermining the core deliverable despite otherwise concrete code and commands."},"latency_ms":14820,"stop_reason":"end_turn","usage":{"input_tokens":5269,"output_tokens":952},"cost_usd":0.10029}
+{"config":"workflow-default","item":"f02","split":"dev","phase":"new","sample":0,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":2,"reasoning":"Steps 1–2 are precise: honest capture with separate exit-code recording, SKIPPED vs ERROR vs 127 semantics, and composite N/A on capture failure are all well specified. But Steps 3–6 collapse into one-sentence placeholders. Step 3 says 'give each category a score... combine them' with no weights, scale, or formula, even though Step 2 forbids 'redistributing that category's weight'—a weight that is never defined. Step 5 says 'save the results somewhere', giving no file path, schema, or 'numeric history row' format that Step 2 references. Dashboard layout, trend thresholds, and recommendation rules are absent, so two agents would produce incompatible deliverables and histories."},"latency_ms":10703,"stop_reason":"end_turn","usage":{"input_tokens":3345,"output_tokens":656},"cost_usd":0.06625}
+{"config":"workflow-default","item":"f03","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Only the browser-selection/setup step is fleshed out, and even it is muddled: 'READY' from the inline probe means the browse binary exists, which conflicts with the opening paragraph's 'On READY, stop importing' (which seems to refer to the browse/SKILL.md probe), the bun-install step comes after the ./setup step that needs bun, and <SKILL_DIR> is undefined. The actual substance of the workflow is absent: section 2 is literally 'TBD', section 3 says 'import whatever cookies seem relevant' with no command, no source-profile selection, no domain scoping, no user-consent gate, and section 4 gives no report format or honesty criteria (e.g., what to say on partial/failed import). An agent could run the setup probe but cannot import cookies or produce the expected deliverable from these instructions."},"latency_ms":12475,"stop_reason":"end_turn","usage":{"input_tokens":1630,"output_tokens":768},"cost_usd":0.0547}
+{"config":"workflow-default","item":"f01","split":"dev","phase":"old-a","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Several hard defects undermine the workflow. Step 3 is absent and Step 2 only asks which agent to pair with, yet Step 4 branches on 'same machine (option A)' vs 'different machine (option B)' — a local-vs-remote decision that is never asked, with option labels that refer to nothing (and collide with the A/B labels of the daemon-consent and tunnel-consent questions). The live-daemon consent gate recommends keeping the daemon (option B), but is immediately followed by 'Always pass --force-restart to the commands below so pairing starts from a clean daemon', which nullifies the consent and contradicts the iron rule; the subsequent commands don't actually include the flag either. Step 1 tells a time-pressed agent to run Step 5 (verify connection) before pairing, which is nonsensical ordering. Step 5 has no failure path if the agent doesn't appear. The remote branch itself (consent gate, ngrok detection, three ngrok states, token-handling security, verbatim output requirement, scope flags) is detailed and actionable, and the capability/scope section is useful, but the missing mode decision and the contradictory force-restart directive mean an agent cannot reliably follow this without guessing or risking destruction of a live daemon against user wishes."},"latency_ms":17812,"stop_reason":"end_turn","usage":{"input_tokens":3712,"output_tokens":1319},"cost_usd":0.10307}
+{"config":"workflow-default","item":"f01","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The remote/ngrok branch is detailed (consent gate, auth-token hygiene, instruction-block output, scope options), but the workflow has serious defects. Step 3 is absent: nothing asks whether the target agent is on the same or a different machine, yet Step 4 branches on 'option A (same machine)' vs 'option B (different machine)', and those labels collide with the A/B options of the live-daemon consent question immediately above, so an agent cannot tell which decision drives which branch. The daemon-consent block is directly contradicted by 'Always pass --force-restart to the commands below' — this nullifies the one-way-door consent and would destroy live tabs/logins even when the user chose 'keep the daemon' (and the commands shown don't actually include the flag). Step 1's advice to 'run Step 5 (Verify connection) first and come back to pairing later' is nonsensical since there is nothing to verify before pairing. Step 5 only describes verification for the remote paste flow, not local pairing. Together these ordering contradictions and the missing local-vs-remote decision make the workflow unreliable to execute despite the solid detail in the tunnel path."},"latency_ms":17804,"stop_reason":"end_turn","usage":{"input_tokens":3712,"output_tokens":1343},"cost_usd":0.10427}
+{"config":"workflow-default","item":"a01","split":"dev","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":3,"actionability":3,"reasoning":"The five-phase structure, Iron Law, and failure-mode table are well organized and mostly unambiguous, with concrete file paths and endpoints. However, several decision points are underspecified: Phase 4 says to 'compare' the fresh screenshot against the pre-bug PNG but never defines the pass/fail criterion (identical = bug persists? any diff = fixed?); Phase 5 says the test must 'assert the post-fix behavior' but gives no guidance on what to assert or how the captured post.png is used; Phase 3 step 2 (`build install`) and step 4 ('Re-deploy') appear redundant/contradictory, and steps 3-4 are observations rather than agent actions. Minor gaps: no instruction for when the bug cannot be reproduced in Phase 1, the 'persist a one-line description' step has no destination, the 409 're-snapshot' remedy conflicts with the Iron Law ordering (fix already applied), and 'resume from Phase 4' on disconnect is wrong if the disconnect happens earlier. An agent could execute most of this but would have to invent the verification and assertion criteria."},"latency_ms":24265,"stop_reason":"end_turn","usage":{"input_tokens":1716,"output_tokens":1690},"cost_usd":0.10166}
+{"config":"workflow-default","item":"a08","split":"heldout","phase":"new","sample":0,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"Steps 1-3 are concrete and runnable, but the deliverable steps are underspecified. Step 4 lists jq fields yet never defines the table's columns, how per-level `.version` values appear, or what to render when `.offline` is true—despite offline output being a stated requirement. Step 5 says only 'suggest a sensible next action' with no decision rules (e.g., base VERSION ahead of HEAD → rebase; offline → retry/manual check). Minor shell flaw: the `|| echo` fallbacks after `tr` pipes never trigger because `tr` succeeds, so defaults are not applied as intended. An agent can execute the queries but will improvise the dashboard and recommendation."},"latency_ms":11505,"stop_reason":"end_turn","usage":{"input_tokens":1814,"output_tokens":723},"cost_usd":0.05429}
+{"config":"workflow-default","item":"p08","split":"heldout","phase":"new","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is dense but internally consistent: scope, source establishment, recent-work survey, bounded older-PR scan, rubric, revalidation, and a three-part output are all specified with concrete commands and endpoints. Two decisive points: the safe-git probe (`rev-parse --is-inside-work-tree`) with explicit fallbacks (pinned-commit GET reads, or disclosed coverage gap) gives a clear decision tree; the page-budget rules (five metadata, fifty file-list pages, counters kept in conversation) are precise and enforceable. Weaknesses: 'relevant current-branch work' and 'strong candidates' rely on judgment without criteria, and the heavy prohibition lists make the text hard to scan, risking missed constraints under turn limits. Non-GitHub providers get only generic guidance."},"latency_ms":12609,"stop_reason":"end_turn","usage":{"input_tokens":5086,"output_tokens":747},"cost_usd":0.08821}
+{"config":"workflow-default","item":"f06","split":"heldout","phase":"new","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The Iron Law (\"FIX OBVIOUS BUGS IMMEDIATELY\") directly contradicts Phase 3's red flag that proposing a fix before tracing data flow is guessing and there is no \"quick fix for now,\" leaving the agent with no coherent starting rule. Phase 2 is a single vague sentence with no bug-pattern catalog or matching procedure, yet Phase 3 says to search \"through Aside, as in Phase 2,\" referencing content that isn't there. The advertised 3-strike hypothesis limit is absent—failed hypotheses loop back to Phase 1 indefinitely with no escalation—and Phase 5's \"structured debug report\" is reduced to \"summarize what you did.\" Phases 1, 3, and 4 are reasonably concrete, but core decision points and deliverables are undefined."},"latency_ms":12907,"stop_reason":"end_turn","usage":{"input_tokens":2162,"output_tokens":939},"cost_usd":0.06857}
+{"config":"workflow-default","item":"f08","split":"heldout","phase":"new","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The workflow has structural gaps and contradictions. The consent flow's steps 3–5 branch on options A/B/C that the AskUserQuestion text never defines, and step 3 touches the marker 'only if the user enabled' while the privacy note promises decliners are 'never re-asked'—a direct contradiction that makes the gate logic unfollowable. The 5-Q setup says 'ask about their preferences on each dimension' with no actual questions, answer scale, or mapping to the numeric <Qn_VALUE> placeholders, and instructs saving 'after each answer' while the script writes all five at once. Step 0 lists intents 4–12 but omits 1–3 and never enumerates which implicit gates run before routing; power-user shortcuts like 'stats' and 'audit' have no mapped action."},"latency_ms":13336,"stop_reason":"end_turn","usage":{"input_tokens":4092,"output_tokens":829},"cost_usd":0.08237}
+{"config":"workflow-default","item":"p07","split":"heldout","phase":"new","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: the Step 1 script emits tagged lines (NO_CHECKPOINTS, FOREIGN, UNVERIFIED, ROOT_DIFFERS, SHARED_BUCKET, LEGACY_BUCKET) and Step 2 maps each tag to explicit user-facing behavior, with a fixed output template and AskUserQuestion options. Branch-first ordering with cross-branch fallback is clearly explained and implemented. Gaps: `<title-fragment-or-number>` never defines what a 'number' indexes (position in the printed list? timestamp prefix?), and there is no decision for zero or multiple fragment matches. Minor: the GSTACK_STATE_ROOT line is duplicated, and the summary fields (title, duration) assume a frontmatter schema that is only implied. Otherwise an agent can execute this end-to-end and produce the expected summary."},"latency_ms":16685,"stop_reason":"end_turn","usage":{"input_tokens":5360,"output_tokens":1048},"cost_usd":0.106}
+{"config":"workflow-default","item":"p06","split":"heldout","phase":"new","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is concrete and well-sequenced: bundle discovery/staging script, a single render command with explicit error-line triage (ERROR vs benign PAGE_ERRORS), output-dir and slug rules, and delivery steps. Two gaps hold it back. First, the non-ASCII note says to use `decodeURIComponent(escape(atob(...)))` but the canonical command uses bare `atob`, leaving the agent to decide when and how to patch the template. Second, the edited-.excalidraw round-trip re-exports SVG/PNG while the `.mmd` remains 'single source of truth,' yet nothing says how to reconcile or flag the now-stale mermaid source (nor how the agent receives the edited scene). Minor: no explicit `mkdir -p` for `./diagrams/`. Otherwise an agent can execute this and produce the triplet reliably."},"latency_ms":17311,"stop_reason":"end_turn","usage":{"input_tokens":4101,"output_tokens":1167},"cost_usd":0.09936}
+{"config":"workflow-default","item":"p08","split":"heldout","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is unusually explicit: read-only boundaries, Git safety helper usage with a mandatory probe and defined fallbacks (pinned-commit GET API, then labeled raw source), a concrete PR-scan budget with unit accounting and exact endpoints, a five-point rubric with a defined savings formula, a pre-report revalidation step, and a three-part output spec with linking rules. Decision points (probe failure, API failure, overlap uncertainty, no surviving candidates) all have stated outcomes. Deductions: the prose is very dense and some terms are left to judgment ('relevant current-branch work', 'active language areas', 'strong candidates'); the recent-window PR paging is unbounded while the older-open-PR scan is budgeted, and whether cached recent-scan pages interact with that budget is only implied ('previously cached pages cost no new requests'); non-GitHub providers get only generic 'provider GET API' guidance. Overall an agent can follow and produce the deliverables, but must resolve a few fuzzy thresholds itself."},"latency_ms":17695,"stop_reason":"end_turn","usage":{"input_tokens":4689,"output_tokens":1153},"cost_usd":0.10454}
+{"config":"workflow-default","item":"a08","split":"heldout","phase":"new","sample":1,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"Steps 1-3 are concrete, copy-pasteable bash with explicit fallbacks (base branch detection, per-level queue queries writing /tmp/landing-$LEVEL.json). The workflow collapses at the deliverables: Step 4 says 'Build a single table' but gives no columns, layout, or how to show each level's `.version` (the stated reason for four queries), and never specifies offline rendering despite `{\"offline\":true}` being a designed outcome. Step 5 is just 'suggest a sensible next action' with no decision rules (e.g., when to recommend a bump level vs. wait on an active sibling). Minor: `cat ... | tr ... || echo` fallback never fires because `||` tests tr's status. Agent can gather data but must invent the dashboard and recommendation logic."},"latency_ms":11359,"stop_reason":"end_turn","usage":{"input_tokens":1814,"output_tokens":694},"cost_usd":0.05284}
+{"config":"workflow-default","item":"a08","split":"heldout","phase":"old-a","sample":2,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":2,"reasoning":"Steps 1-3 are concrete and runnable: base-branch detection has a clear fallback chain, VERSION reads are explicit, and the per-level queue loop with offline fallback JSON is well-defined (though the `cat ... | tr ... || echo` fallbacks never fire on a missing file since `tr` succeeds, so the documented defaults are misleading). The workflow collapses in the deliverable steps: Step 4 lists jq fields but gives no table layout, no columns, no instruction on how to present the four per-level `.version` values it just collected, and no rendering for the `offline:true` case despite that being a required output; Step 5 ('suggest a sensible next action') has no decision logic at all for choosing one action from the claimed/sibling/offline state. No Read instructions point to a template elsewhere, so the agent must invent the dashboard and the recommendation heuristic, making the final output unpredictable even though the data-gathering half is executable."},"latency_ms":15865,"stop_reason":"end_turn","usage":{"input_tokens":1417,"output_tokens":1133},"cost_usd":0.07082}
+{"config":"workflow-default","item":"a07","split":"heldout","phase":"new","sample":1,"status":"ok","value":{"clarity":3,"completeness":4,"actionability":4,"reasoning":"The workflow is thorough: honest capture with exit-code isolation, explicit SKIPPED/ERROR semantics, composite computed in code, like-for-like trend comparison, and ranked recommendations. But scoring is internally inconsistent: the rubric defines only 10/7/4/0 bands (e.g., Lint '<5 warnings' = 7), yet the dashboard example shows 'Lint 3 warnings → 8/10' and status labels span 7-9 and 4-6, leaving the agent to guess how intermediate scores arise. The GBrain row's '7' tier uses overlapping ORs (queue<100 also matches queue<10) with no precedence rule, and its JSON fields are unspecified. Step 4 ends mid-sentence ('list the top issues...:'), 'IMPROVING' thresholds are undefined, and Step 1's auto-detect shell uses `||`/`&&` precedence incorrectly. Still executable end-to-end."},"latency_ms":27934,"stop_reason":"end_turn","usage":{"input_tokens":6740,"output_tokens":1966},"cost_usd":0.1657}
+{"config":"workflow-default","item":"p07","split":"heldout","phase":"new","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: a runnable Step 1 script with labeled sentinel outputs (NO_CHECKPOINTS, NO_VERIFIED_CHECKPOINTS, IDENTITY_CHECK_FAILED, FOREIGN/UNVERIFIED/ROOT_DIFFERS), explicit mapping from each sentinel to agent behavior, a concrete summary template, and defined next-step options. Gaps remain: Step 2 allows `/context-restore <number>` but no numbering is ever produced by Step 1, so the agent cannot know what a number refers to; and title-fragment matching is undefined for multiple matches or zero matches (no fallback behavior specified). Minor script redundancy (duplicate GSTACK_STATE_ROOT line) is harmless. Otherwise the default-selection logic, branch-first ordering rationale, and identity-mismatch reporting are unambiguous and executable."},"latency_ms":13406,"stop_reason":"end_turn","usage":{"input_tokens":5360,"output_tokens":913},"cost_usd":0.09925}
+{"config":"workflow-default","item":"p07","split":"heldout","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: command parsing, a concrete discovery script with explicit sentinel outputs (NO_CHECKPOINTS, NO_VERIFIED_CHECKPOINTS, IDENTITY_CHECK_FAILED, FOREIGN/UNVERIFIED/ROOT_DIFFERS prefixes), clear default-selection rule (first plain path line = current-branch-first, newest), a fixed presentation template, identity-mismatch messaging ordered before the summary, a branch-mismatch note, an AskUserQuestion menu, and an explicit empty-state message. The HARD GATE and the list-redirect are unambiguous. Gaps: `/context-restore <number>` is supported but the script never numbers candidates and no numbering scheme is defined; title-fragment matching doesn't say whether to match filename or `title:` frontmatter, nor what to do on zero or multiple matches; Step 3 options B/C have no spelled-out behavior beyond their labels; and the script has a harmless duplicated GSTACK_STATE_ROOT line. The frontmatter fields (title, status, duration) and section names are assumed from the save-side format, which is acceptable given the shared-definitions caveat. Overall an agent can execute this reliably for the default and empty-state paths, with minor ambiguity on user-specified selection."},"latency_ms":19097,"stop_reason":"end_turn","usage":{"input_tokens":4963,"output_tokens":1329},"cost_usd":0.11608}
+{"config":"workflow-default","item":"p07","split":"heldout","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: command parsing, a self-contained bash step with explicit sentinel outputs (NO_CHECKPOINTS, NO_VERIFIED_CHECKPOINTS, IDENTITY_CHECK_FAILED, FOREIGN/UNVERIFIED/ROOT_DIFFERS, SHARED_BUCKET, LEGACY_BUCKET), clear selection rule (first plain path line = current-branch-first, newest), a fixed presentation template, a branch-mismatch note, an AskUserQuestion follow-up, and an explicit empty-state message. The branch-ordering rationale and 'do not hard-filter' guidance are unambiguous. Gaps: the `<number>` argument form is never defined (index into which list? the Step 1 output? `/context-save list`?), title-fragment matching is unspecified (filename vs. `title:` frontmatter, multiple matches, zero matches), and the Duration/timestamp fields reference frontmatter keys without naming them. Minor cosmetic issue: a duplicated GSTACK_STATE_ROOT line in the script (harmless). Otherwise an agent can execute this end to end and produce the expected summary and prompts."},"latency_ms":19834,"stop_reason":"end_turn","usage":{"input_tokens":4963,"output_tokens":1392},"cost_usd":0.11923}
+{"config":"workflow-default","item":"p06","split":"heldout","phase":"new","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":5,"reasoning":"The workflow is concrete end to end: artifact table, output-dir/slug decision, a copy-pasteable staging script with a sha-addressed path, one render command for the full triplet, and explicit interpretation of OK/ERROR/PAGE_ERRORS/NEEDS_ASIDE output with stop conditions. Minor gaps hurt clarity/completeness: the non-ASCII handling is a footnote (`decodeURIComponent(escape(atob(...)))`) the agent must splice into the command itself rather than a stated branch, and Step 4 declares the `.mmd` the single source of truth yet the edited-`.excalidraw` round-trip bypasses it without saying how the now-stale `.mmd` should be treated or where the user's edited scene should be placed. Otherwise an agent can execute this and produce the deliverables."},"latency_ms":18610,"stop_reason":"end_turn","usage":{"input_tokens":4101,"output_tokens":1194},"cost_usd":0.10071}
+{"config":"workflow-default","item":"f08","split":"heldout","phase":"old-a","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Step 0 says gates 'may run before or after intent routing, depending on the situation' with no decision rule, and the routing list starts at item 4 — the implicit gates (consent, setup) that later sections call 'Step 0's consent gate/setup gate' are never defined in the entry point. The consent AskUserQuestion gives framing text but never defines options A/B/C, yet steps 4-5 branch on them; 'Completeness: A=9/10' is unexplained. Step 3 touches the marker 'only if the user enabled tuning,' directly contradicting the privacy note that the marker ensures declined users are 'never re-asked.' The 5-Q setup says 'Ask about their preferences on each dimension' with no actual questions, answer options, or mapping to the numeric <Qn_VALUE>s the bun script expects; it says to save after each answer but the script writes all five at once, and partial-bailout handling of the write is undefined. Option B's post-consent path (enable without setup?) is unspecified, and shortcuts like `stats`/`audit` have no routing target. An agent could run the shell snippets but cannot reliably produce the consent decision or a valid declared profile."},"latency_ms":19214,"stop_reason":"end_turn","usage":{"input_tokens":3695,"output_tokens":1248},"cost_usd":0.09935}
+{"config":"workflow-default","item":"a08","split":"heldout","phase":"old-b","sample":1,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":2,"reasoning":"Steps 1-3 are concrete and executable (base-branch detection, VERSION reads, per-level queue queries), though Step 2's `|| echo` fallbacks sit after a `tr` pipe and never fire, so a missing VERSION yields empty strings rather than the documented defaults. The core deliverables are underspecified: Step 4 lists jq fields but gives no table layout, no column set, no instruction for how per-level `.version` values are shown, and no offline rendering path despite `{\"offline\":true}` being an explicit branch and the stated requirement to handle offline output. Step 5 ('suggest a sensible next action') provides no decision criteria, candidate actions, or mapping from queue/sibling state to a recommendation. `.active_siblings` is defined only as 'likely about to ship.' An agent can gather the data but must invent the dashboard format, offline handling, and next-action logic, so the expected deliverables are not reliably reproducible."},"latency_ms":11008,"stop_reason":"end_turn","usage":{"input_tokens":1417,"output_tokens":798},"cost_usd":0.05407}
+{"config":"workflow-default","item":"p07","split":"heldout","phase":"new","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-specified: a commented discovery script with distinct sentinel outputs (NO_CHECKPOINTS, NO_VERIFIED_CHECKPOINTS, IDENTITY_CHECK_FAILED, FOREIGN/UNVERIFIED/ROOT_DIFFERS), explicit selection rule (first plain path line), a concrete RESUMING CONTEXT template, branch-mismatch note, and AskUserQuestion next steps. Gaps remain in the user-specified path: `<title-fragment-or-number>` never defines what a 'number' indexes, and there is no instruction for zero or multiple fragment matches. Minor noise: GSTACK_STATE_ROOT is resolved twice and `mkdir -p` creates state in a read-only skill. Field sourcing (status, duration formatting) is implied rather than defined, but an agent can execute this and produce the deliverable reliably."},"latency_ms":13584,"stop_reason":"end_turn","usage":{"input_tokens":5360,"output_tokens":897},"cost_usd":0.09845}
+{"config":"workflow-default","item":"f07","split":"heldout","phase":"new","sample":0,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Step 1 (ask/auto-upgrade, snooze backoff) and Steps 5-7 are concrete, but the core upgrade path is hollow. Step 2 is a one-line stub ('figure out how gstack was installed') that never defines INSTALL_DIR or vendored-vs-git classification, yet Step 1 and 4.75 depend on install-specific recovery, backups, and './setup completes' that Step 4 never specifies ('whatever method fits... put it back'). Step 3 contradicts itself: 'Save old version' but instructs running it after Step 4, which would capture the new version and break migration gating and the What's New range. No backup, git fetch/reset, fresh clone, or setup commands exist, so an agent cannot execute the actual upgrade."},"latency_ms":11925,"stop_reason":"end_turn","usage":{"input_tokens":3626,"output_tokens":820},"cost_usd":0.07726}
+{"config":"workflow-default","item":"a08","split":"heldout","phase":"old-a","sample":0,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"Steps 1-3 are concrete and runnable (base-branch fallback chain, VERSION reads, per-level queue loop with an offline sentinel), though the shell has subtle bugs: `cat VERSION | tr ... || echo` and `git show ... | tr ... || echo` take the pipeline's (tr's) exit status, so the fallbacks never fire and an empty VERSION silently propagates. The core deliverables are underspecified: Step 4 says 'build a single table' and lists jq fields but gives no column layout, no template, no instruction to pull `.version` from each of the four per-level JSON files, and nothing about how to render the offline case despite that being an explicit requirement. Step 5 ('suggest a sensible next action') defines no decision logic at all—no mapping from queue state, sibling activity, or current-vs-base VERSION drift to a recommended action. An agent can execute the data-gathering commands but must invent the dashboard format, the offline output, and the next-action rule."},"latency_ms":13369,"stop_reason":"end_turn","usage":{"input_tokens":1417,"output_tokens":947},"cost_usd":0.06152}
+{"config":"workflow-default","item":"p06","split":"heldout","phase":"old-b","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":5,"reasoning":"The workflow is well-structured: four numbered steps with concrete, copy-pasteable shell commands, explicit placeholders (<staged>, <outdir>, <slug>), defined output-directory and slug rules, and clear interpretation of success/failure signals (OK lines, ERROR:, PAGE_ERRORS cases, BUNDLE_MISSING, NEEDS_ASIDE/no browser). Hard stops (no CDN fallback, never ship unrendered) are unambiguous. Minor clarity gaps: the UTF-8 handling (decodeURIComponent(escape(atob(...)))) is a trailing note rather than integrated into the main command, so an agent could miss it for non-ASCII labels; the WorkerInTheMainChunkError benign-case is only documented for the round-trip path. Completeness gaps: after a user edits the .excalidraw and the agent re-renders from the scene, the .mmd becomes stale, contradicting 'the source is the single source of truth' with no guidance on reconciling; no handling for creating ./diagrams/ or for slug collisions/overwrites. Otherwise all deliverables, decision points, and error paths are defined and directly executable."},"latency_ms":21244,"stop_reason":"end_turn","usage":{"input_tokens":3704,"output_tokens":1462},"cost_usd":0.11014}
+{"config":"workflow-default","item":"a07","split":"heldout","phase":"old-b","sample":0,"status":"ok","value":{"clarity":3,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured with a robust capture pattern (honest exit codes, private logs, ERROR vs SKIPPED vs CLEAN semantics), weights that sum to 100%, composite computed in code with proportional redistribution, explicit partial/zero-coverage handling, a defined JSONL history schema, and like-for-like trend comparison rules. However, the scoring rubric only defines anchors at 10/7/4/0 with no interpolation rule, yet the dashboard, history, and recommendation examples use 8, 9, 6, and 5 (e.g., 3 lint warnings shown as 8/10 when the rubric says <5 warnings = 7), a direct contradiction the agent must resolve on its own. The Shell lint and GBrain '0' columns are labeled N/A/skip, conflating a zero score with SKIPPED. Step 4's 'list the top issues' instruction ends in a colon with no format. HIGH/MED/LOW thresholds for recommendations are undefined; lint counting of errors vs warnings is ambiguous; the trend reads 10 entries but displays 5. The Step 1 detection shell has precedence bugs (`||`/`&&` chains, zsh-only setopt) and the SLUG chain silently no-ops on failure. Despite these gaps, an agent could execute the workflow end to end and produce the dashboard, history row, trend, and recommendations."},"latency_ms":29971,"stop_reason":"end_turn","usage":{"input_tokens":6343,"output_tokens":2182},"cost_usd":0.17253}
+{"config":"workflow-default","item":"f06","split":"heldout","phase":"old-b","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The entrypoint opens with an 'Iron Law' (fix obvious bugs immediately, investigate only if the quick fix fails) that directly contradicts Phase 3 ('Before writing ANY fix, verify your hypothesis') and the red flag against 'quick fix for now', leaving the agent with no clear rule on when to skip investigation. Phase 2 'Pattern Analysis' is a single vague sentence with no pattern catalog or matching procedure, yet Phase 3 refers back to 'searching for the error through Aside, as in Phase 2' — a dangling reference. The advertised 3-strike hypothesis limit is entirely absent; a wrong hypothesis simply loops back to Phase 1 with no escalation or exit condition. Phase 5's 'structured debug report' is reduced to 'Summarize what you did' with no defined fields, and there is no step to record learnings despite Phase 1 instructing the agent to search prior learnings. Phases 1, 3 (confirmation step), and 4 (minimal diff, failing-then-passing regression test, blast-radius prompt) are concrete and executable, which keeps scores above 1, but the contradictory ordering, missing decision points, and undefined deliverables make the workflow hard to follow reliably end to end."},"latency_ms":15352,"stop_reason":"end_turn","usage":{"input_tokens":1765,"output_tokens":1199},"cost_usd":0.0776}
+{"config":"workflow-default","item":"p07","split":"heldout","phase":"old-a","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The entrypoint is well-structured: role/hard gate, explicit default (current-branch-first, cross-branch fallback) with rationale, a clear command parser, a self-contained Step 1 bash script whose output tokens (NO_CHECKPOINTS, IDENTITY_CHECK_FAILED, NO_VERIFIED_CHECKPOINTS, FOREIGN/UNVERIFIED/ROOT_DIFFERS, SHARED_BUCKET, LEGACY_BUCKET) are each mapped to a concrete agent response in Step 2, a fixed output template, and a defined Step 3 AskUserQuestion. Branch mismatch and no-contexts cases are handled. Gaps keep it from a 5: 'number' in `<title-fragment-or-number>` is never defined (index into the list? timestamp prefix?); title-fragment matching doesn't say how to read titles or what to do on zero/multiple matches; 'Duration (if available)' and 'Saved' don't specify the frontmatter source; the script has a harmless duplicated GSTACK_STATE_ROOT line and no handling if SLUG resolution fails (yielding a malformed CHECKPOINT_DIR). These are minor relative to an otherwise executable, deterministic workflow."},"latency_ms":17512,"stop_reason":"end_turn","usage":{"input_tokens":4963,"output_tokens":1262},"cost_usd":0.11273}
+{"config":"workflow-default","item":"a07","split":"heldout","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured with a clear six-step pipeline, an explicit capture harness that separates checker exit codes from parser/display status, precise SKIPPED vs ERROR semantics, a code-based composite formula, a defined JSONL schema, and like-for-like coverage rules for trends. Weaknesses: the rubric only yields discrete scores (10/7/4/0) yet the dashboard/trend examples show 8, 9, 6 with no interpolation rule; the score for a non-zero exit with zero parsed matches is left undefined; the Shell/GBrain '0' column says 'N/A (skip)', conflating a zero score with skipping; the 'list the top issues' sentence in Step 4 is truncated; trend thresholds (IMPROVING/DECLINING/stable) are not defined, and reading tail -10 after persisting the current row leaves 'previous run' slightly ambiguous; detection script has minor quirks (pylint→ruff, duplicate knip lines, shellcheck arg mismatch) and the composite depends on bun being present. These are mostly edge-case gaps rather than blocking ambiguities, so an agent can execute and produce the deliverables."},"latency_ms":25663,"stop_reason":"end_turn","usage":{"input_tokens":6343,"output_tokens":1793},"cost_usd":0.15308}
+{"config":"workflow-default","item":"f06","split":"heldout","phase":"old-a","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The 'Iron Law' (fix obvious bugs immediately, only investigate if the quick fix fails) directly contradicts Phase 3's 'verify before writing ANY fix' and the red flag against 'quick fix for now', leaving the agent with no coherent entry rule. Phase 2 (Pattern Analysis) is a single vague sentence with no bug-pattern catalog or matching procedure, yet Phase 3 refers back to 'searching through Aside, as in Phase 2', a dangling reference. The advertised 3-strike hypothesis limit is absent—Phase 3 just says 'return to Phase 1' with no cap or escalation path. The 'structured debug report' is reduced to 'summarize what you did' with no template or required fields. Phase 1 step 5 ('search prior learnings') gives no location or method. Phases 1, 3 (sanitization guidance), and 4 (minimal diff, regression test fail/pass, blast-radius prompt) are reasonably concrete, so an agent could partially execute, but it cannot produce the expected pattern match, bounded hypothesis loop, or structured report."},"latency_ms":12758,"stop_reason":"end_turn","usage":{"input_tokens":1765,"output_tokens":1017},"cost_usd":0.0685}
+{"config":"workflow-default","item":"f08","split":"heldout","phase":"old-a","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The consent flow references options A/B/C (\"If A or B: enable\", \"If C: do nothing\", \"accepts option A\") but never defines them; the prompt text only hints at an 'A=9/10' recommendation. Step 3 says to touch the marker 'only if the user enabled tuning,' directly contradicting the privacy note that the answer is honored with a marker so the user is never re-asked — declining users would be re-prompted every time. Step 0 claims gates 'may run before or after intent routing, depending on the situation' with no decision rule, and the implicit gates (consent, setup) are not enumerated in the routing list (which starts at item 4). The 5-Q setup never states the five questions, the answer options, or how answers map to numeric values (<Q1_VALUE> placeholders with no scale), and it says to save after each answer while providing only a batch script that writes all five at once. Shortcuts like `stats` and `audit` have no mapped action. An agent could execute the shell commands, but cannot reliably produce correct consent handling or a correctly populated numeric profile."},"latency_ms":17008,"stop_reason":"end_turn","usage":{"input_tokens":3695,"output_tokens":1193},"cost_usd":0.0966}
+{"config":"workflow-default","item":"f08","split":"heldout","phase":"new","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The workflow has structural gaps that block unambiguous execution. The consent step references answers A/B/C and says 'If A or B: enable' but never defines the options presented, and Step 3 ('touch the marker only if the user enabled tuning') contradicts the privacy note promising decliners are never re-asked—a declining user will be re-prompted forever. The 5-Q setup says 'Ask about their preferences on each dimension' with no question text, no numeric scale, and no answer-to-value mapping, yet the script requires numeric Q1–Q5 values; it also says save after each answer while the script writes all five at once. Step 0 omits items 1–3 and gate ordering is hand-waved ('may run before or after')."},"latency_ms":10293,"stop_reason":"end_turn","usage":{"input_tokens":4092,"output_tokens":741},"cost_usd":0.07797}
+{"config":"workflow-default","item":"p08","split":"heldout","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The skill is dense but largely unambiguous: the read-only boundary, safe-git usage, API-vs-local fallbacks, the explicit paging budget with counter rules, the five-point rubric, and the three-part output are all spelled out with concrete commands and endpoints. Decision points are mostly covered (probe failure → pinned API reads; API failure → labeled raw source; budget exhaustion → disclose; candidate drift → revalidate or drop; covered-by-PR → set aside). Weaknesses: the heavy density and nested prohibitions make it easy to miss a rule; the choice of the 'selected review commit' when the remote tip is inaccessible is only implied; 'engineering-plan review' and 'active language areas' are referenced without definition in this bundle; and the opening emphasis on favoring actively-changing areas sits in mild tension with the rubric's 'recency only breaks ties', though the output section reconciles it by requiring explanation. An agent can execute the workflow and produce the deliverables, but the paging budget bookkeeping and strict isolation rules impose real cognitive overhead."},"latency_ms":23207,"stop_reason":"end_turn","usage":{"input_tokens":4689,"output_tokens":1523},"cost_usd":0.12304}
+{"config":"workflow-default","item":"f07","split":"heldout","phase":"old-a","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Step 1 (ask/auto-upgrade, snooze backoff), Step 4.75 (migrations), Step 5 (marker/cache), and Step 6 (What's New) are concrete and well-specified. However the core of the workflow is hollow or contradictory. Step 2 'Detect install type' gives no detection logic and never defines INSTALL_DIR, which every later step depends on. Step 4 'Upgrade' is a one-line placeholder with no backup commands, no git-pull vs fresh-clone branching, no `./setup` invocation, and no recovery logic—yet Step 1 points to 'Step 4's install-specific recovery' and Step 4.75 says 'After ./setup completes', referencing things that don't exist. Step 3 instructs running the OLD_VERSION capture *after* Step 4 finishes, which would read the new VERSION file, making OLD_VERSION equal the new version; this silently breaks the migration gate in Step 4.75, the just-upgraded-from marker, and the changelog range in Step 6. An agent cannot execute the backup/upgrade/setup phases from these instructions and would produce incorrect version records if it followed the stated ordering."},"latency_ms":17080,"stop_reason":"end_turn","usage":{"input_tokens":3229,"output_tokens":1317},"cost_usd":0.09814}
+{"config":"workflow-default","item":"a07","split":"heldout","phase":"new","sample":0,"status":"ok","value":{"clarity":3,"completeness":4,"actionability":4,"reasoning":"The workflow is thorough: honest capture with exit-code preservation, explicit SKIPPED/ERROR semantics, composite computed in code, like-for-like trend comparison, and ranked recommendations. Weaknesses are in rubric coherence and loose ends. The scoring table only yields 0/4/7/10, yet the dashboard, status labels (7-9 WARNING), and trend examples show 8s, 9s, and 6s, leaving the agent unsure whether interpolation is allowed; Shell lint's columns also make 0 unreachable and GBrain's OR-based tiers overlap. Step 4 ends mid-instruction ('list the top issues from that tool's output:') with no format, and 'Trend: IMPROVING' has no threshold rule for STABLE/DECLINING. Still, an agent can produce the dashboard, history row, and recommendations."},"latency_ms":41990,"stop_reason":"end_turn","usage":{"input_tokens":6740,"output_tokens":1391},"cost_usd":0.13695}
+{"config":"workflow-default","item":"p06","split":"heldout","phase":"old-a","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":5,"reasoning":"The single entrypoint file lays out a linear, concrete workflow: author mermaid with explicit type/size rules and output-dir/slug decisions, stage the bundle with an idempotent, copy-pasteable script and a clear BUNDLE_MISSING stop condition, render the triplet with one fully specified gstack-render invocation (base64 transport rationale included), and deliver with an explicit four-point checklist plus an edited-scene round-trip path. Error handling is well differentiated (hard ERROR → fix .mmd and retry; 'Error processing Mermaid diagram' PAGE_ERRORS → benign fallback; NEEDS_ASIDE/no browser → stop; WorkerInTheMainChunkError → ignore). Minor clarity/completeness gaps: the UTF-8 `decodeURIComponent(escape(atob()))` note is tacked on after the command rather than integrated into a decision ('if source has non-ASCII, use this form'), and it isn't mentioned for the scene-JSON round-trip; Step 2's heading says 'once per session' while the body says the directory is served 'for each render' (not contradictory in effect, but slightly muddled); splitting into multiple diagrams leaves per-diagram slug/loop handling implicit. Otherwise an agent can execute this end-to-end and produce the expected .mmd/.excalidraw/.svg/.png deliverables."},"latency_ms":22579,"stop_reason":"end_turn","usage":{"input_tokens":3704,"output_tokens":1546},"cost_usd":0.11434}
+{"config":"workflow-default","item":"a07","split":"heldout","phase":"old-b","sample":2,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":4,"reasoning":"The workflow is well-structured end-to-end (detect → run with honest capture → score → dashboard → persist → trend) and the capture block, SKIPPED/ERROR semantics, like-for-like trend comparison, and composite script are carefully specified. However several ambiguities and gaps undercut it: the scoring rubric only yields discrete 10/7/4/0 values yet the status bands (7-9, 4-6) and dashboard example (8/10, 9/10) imply intermediate scores with no interpolation rule; the Shell lint and GBrain rows put 'N/A (skip)' in the 0-score column, conflating skip with a score; GBrain's 10 and 7 conditions overlap (queue<10 also satisfies queue<100) with no precedence stated; the detection script has shell precedence bugs (`[ -f a ] || [ -f b ] && echo` never echoes when the first file exists) and a zsh-only `setopt`; the Glob shell-script search result is never used; 'If any category scored below 7, list the top issues…:' ends abruptly with no format; HIGH/MED/LOW thresholds for recommendations are undefined; and Step 6 reads 10 history entries but displays 5. An agent could still produce the dashboard, history row, and trend, but would have to make several judgment calls."},"latency_ms":27931,"stop_reason":"end_turn","usage":{"input_tokens":6343,"output_tokens":2122},"cost_usd":0.16953}
+{"config":"workflow-default","item":"f07","split":"heldout","phase":"old-a","sample":2,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Steps 1, 4.75, 5, 6, 7 are concrete and scriptable, but the core of the workflow is hollow or contradictory. Step 2 ('Figure out how gstack was installed and where') and Step 4 ('Upgrade gstack with whatever method fits the install... put it back the way it was') contain no actual instructions for detecting vendored vs git installs, backing up, git pull vs fresh clone, running ./setup, or restoring on failure—yet Step 1 explicitly points to 'Step 4's install-specific recovery' that doesn't exist. $INSTALL_DIR is never defined anywhere, though Steps 3, 4.75, and 6 depend on it. Step 3 instructs to capture OLD_VERSION *after* Step 4 finishes, which would record the new version and defeat the marker, migration gating, and changelog range—a direct logic contradiction. Step 4.75 says 'After ./setup completes' but setup is never invoked. No version comparison step exists. An agent cannot execute the upgrade from this text alone."},"latency_ms":12168,"stop_reason":"end_turn","usage":{"input_tokens":3229,"output_tokens":820},"cost_usd":0.07329}
+{"config":"workflow-default","item":"f06","split":"heldout","phase":"new","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The workflow contradicts itself: the Iron Law says 'FIX OBVIOUS BUGS IMMEDIATELY' and investigate only if the quick fix fails, while Phase 3 red-flags 'Quick fix for now' and forbids guessing. Phase 2 (Pattern Analysis) is a single vague sentence, yet Phase 3 says to search via Aside 'as in Phase 2', referencing procedure that doesn't exist. Key advertised elements are absent: no 3-strike hypothesis limit (the loop back to Phase 1 is unbounded), and Phase 5's 'Summarize what you did' defines no structured debug report. Phases 1, 4 are reasonably concrete (git log, failing-then-passing regression test, >5-file blast-radius prompt), but the missing decision points and contradictory stance make reliable execution unlikely."},"latency_ms":11407,"stop_reason":"end_turn","usage":{"input_tokens":2162,"output_tokens":790},"cost_usd":0.06112}
+{"config":"workflow-default","item":"p06","split":"heldout","phase":"old-a","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The SKILL.md is a tight, well-sequenced workflow: author mermaid with concrete sizing/type guidance, decide outdir and slug, stage the bundle with a fully specified script, render the triplet with one explicit gstack-render invocation, interpret output lines (OK, ERROR:, PAGE_ERRORS, NEEDS_ASIDE), and deliver with a scripted user-facing note. Error handling and stop conditions are explicit and the 'never improvise a CDN/renderer' contract is unambiguous. Minor weaknesses: the non-ASCII/UTF-8 atob caveat is placed after the canonical command rather than built into it, so an agent could render corrupted labels before noticing it; the round-trip path (re-render from an edited .excalidraw) silently leaves the .mmd stale while Step 4 insists the .mmd is the single source of truth, and no reconciliation step is given; the 'triplet' term actually denotes four files; and small operational details (creating ./diagrams/, detecting git repo, shasum vs sha256sum portability) are left implicit. Overall highly executable with a few decision gaps."},"latency_ms":21408,"stop_reason":"end_turn","usage":{"input_tokens":3704,"output_tokens":1430},"cost_usd":0.10854}
+{"config":"workflow-default","item":"p06","split":"heldout","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The single SKILL.md lays out a linear, well-ordered workflow (author → stage → render → deliver) with concrete bash, exact eval expressions, explicit output-dir and slug rules, and a clear taxonomy of render outcomes (OK lines, hard ERROR as parse failure, benign PAGE_ERRORS for the single-image fallback, NEEDS_ASIDE/no-browser stop condition). Decision points like editable vs. image-only diagram types and the offline-only contract are stated unambiguously. Weaknesses: the non-ASCII `decodeURIComponent(escape(atob()))` guidance is a trailing note after the canonical command rather than integrated into it, so the agent must retrofit the command; 'Any OTHER PAGE_ERRORS text: read it before trusting the output' gives no decision rule; and 'the .mmd is the single source of truth' sits in mild tension with the edited-.excalidraw round-trip path, which leaves the .mmd stale without saying how to reconcile subsequent edit requests. Minor gaps (creating ./diagrams/, partial-failure behavior when __svg is undefined for the PNG step) keep completeness and actionability just short of 5."},"latency_ms":21767,"stop_reason":"end_turn","usage":{"input_tokens":3704,"output_tokens":1470},"cost_usd":0.11054}
+{"config":"workflow-default","item":"a07","split":"heldout","phase":"new","sample":2,"status":"ok","value":{"clarity":3,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: hard gate against fixing, rigorous capture semantics (exit code before parser, ERROR vs SKIPPED vs CLEAN), code-computed composite, like-for-like trend comparison, and impact-ranked recommendations. Two decisive gaps: the rubric only defines scores 10/7/4/0, yet the dashboard example shows 8/10 for 3 lint warnings and status bands reference 8-9 and 5-6, leaving intermediate scoring undefined; and the GBrain row mixes OR conditions across the 7 and 4 columns (e.g., doctor=warnings but queue>=100) with no parsing guidance for `gbrain doctor --json`. Minor issues: shell `||`/`&&` precedence bugs in detection, and Step 4's 'list the top issues' ends in a colon with no format. Still executable with reasonable judgment."},"latency_ms":22566,"stop_reason":"end_turn","usage":{"input_tokens":6740,"output_tokens":1571},"cost_usd":0.14595}
+{"config":"workflow-default","item":"a08","split":"heldout","phase":"new","sample":2,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"Steps 1-3 are concrete, runnable bash with clear fallbacks. The deliverables are underspecified: Step 4 lists jq fields but never defines the table layout, how per-level `.version` values are shown, or any offline rendering despite `{\"offline\":true}` being produced—e.g., 'Render a dashboard from these fields' is the only instruction. Step 5 ('suggest a sensible next action') has no decision rules mapping queue state (claimed PRs, active siblings, offline) to a recommendation. Minor bug: `cat VERSION | tr ... || echo 0.0.0.0` only falls back on tr failure, so a missing VERSION yields an empty string. An agent can execute the queries but must invent the dashboard format and next-action logic."},"latency_ms":10580,"stop_reason":"end_turn","usage":{"input_tokens":1814,"output_tokens":734},"cost_usd":0.05484}
+{"config":"workflow-default","item":"p07","split":"heldout","phase":"old-b","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The restore flow is well-structured: a single runnable Step 1 script emits clear sentinel markers (NO_CHECKPOINTS, NO_VERIFIED_CHECKPOINTS, IDENTITY_CHECK_FAILED, FOREIGN/UNVERIFIED/ROOT_DIFFERS/SHARED_BUCKET/LEGACY_BUCKET) and Step 2 maps each marker to explicit user-facing behavior. Branch-preference-with-fallback rationale is stated consistently in prose and code, the presentation template is concrete, branch-mismatch messaging is specified, and the no-contexts path is covered. Weaknesses: 'load a specific saved context by number' is ambiguous since Step 1 prints an unnumbered list; there is no handling for a title fragment that matches zero or multiple files; 'first plain path line' requires the agent to mentally skip the PROJECT_DIR/PROJECT_IDENTITY/PROJECT_ROOT echo lines that precede the file list; the script duplicates the GSTACK_STATE_ROOT line and doesn't guard against gstack-slug failing to produce SLUG; and options B/C in Step 3 have no explicit follow-through (though obvious). These are minor gaps in an otherwise executable, well-specified workflow."},"latency_ms":20725,"stop_reason":"end_turn","usage":{"input_tokens":4963,"output_tokens":1445},"cost_usd":0.12188}
+{"config":"workflow-default","item":"f07","split":"heldout","phase":"old-a","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Step 1 (ask/auto-upgrade/snooze), Step 4.75 (migrations), Step 5 (marker/cache), and Step 6 (What's New) are concrete and followable. However the core of the workflow is hollow or self-contradictory: Step 2 'Detect install type' is a one-line placeholder with no commands and never defines $INSTALL_DIR, which Steps 3–6 depend on; Step 3 says to capture OLD_VERSION *after* Step 4 finishes, which would record the new version and break the migration gating and the 'upgraded from v{old}' message; Step 4 'Upgrade' gives no git-pull vs fresh-clone logic, no backup step, no ./setup invocation, and no recovery procedure even though Step 1 explicitly defers to 'Step 4's install-specific recovery' for vendored vs git installs. Step 4.75 also references './setup completes' with no setup step defined. An agent cannot execute the detect/backup/upgrade/setup portion as written and would be misled by the Step 3 ordering."},"latency_ms":14005,"stop_reason":"end_turn","usage":{"input_tokens":3229,"output_tokens":989},"cost_usd":0.08174}
+{"config":"workflow-default","item":"f06","split":"heldout","phase":"old-a","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The Iron Law ('FIX OBVIOUS BUGS IMMEDIATELY. Only investigate when the quick fix does not work') directly contradicts Phase 1 ('Gather context before forming any hypothesis') and Phase 3's red flags ('there is no \"for now\"', 'Proposing a fix before tracing data flow — you're guessing'), leaving the agent with no coherent entry rule. Phase 2 Pattern Analysis is a single vague sentence with no known-pattern catalog or matching procedure, yet Phase 3 says 'searching for the error through Aside, as in Phase 2' — a dangling reference to content that doesn't exist. The advertised 3-strike hypothesis limit is entirely absent: Phase 3 just loops back to Phase 1 indefinitely with no counter or escalation. Phase 5 says only 'Summarize what you did for the user' with no structured debug report template, fields, or format. Phases 1, 3, and 4 (symptom collection, git log, confirm-via-log, minimal diff, fail-then-pass regression test, >5 file blast-radius prompt) are concrete and executable, but the missing strike limit, pattern matching, and report structure mean the agent cannot produce several of the expected deliverables and faces a contradictory decision at the very start."},"latency_ms":15629,"stop_reason":"end_turn","usage":{"input_tokens":1765,"output_tokens":1271},"cost_usd":0.0812}
+{"config":"workflow-default","item":"f07","split":"heldout","phase":"new","sample":2,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Step 1 and the snooze/marker/migration/changelog steps are concrete, but the core of the workflow is hollow or contradictory. Step 2 (\"Figure out how gstack was installed and where\") and Step 4 (\"Upgrade gstack with whatever method fits the install. If something goes wrong, put it back the way it was\") give no commands, never define INSTALL_DIR, and omit backup, git-vs-fresh-clone logic, and running ./setup—yet Step 1 and Step 4.75 reference Step 4's install-specific recovery and ./setup completion as if they existed. Step 3 instructs saving OLD_VERSION after Step 4 finishes, which would capture the new version, breaking migrations and the What's New diff. An agent cannot execute the detection, backup, or upgrade deliverables from this bundle."},"latency_ms":9623,"stop_reason":"end_turn","usage":{"input_tokens":3626,"output_tokens":637},"cost_usd":0.06811}
+{"config":"workflow-default","item":"a08","split":"heldout","phase":"old-b","sample":2,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":2,"reasoning":"Steps 1-3 are concrete, runnable shell snippets (base-branch fallback chain, VERSION reads, per-level queue queries with an offline JSON fallback), though the `cat ... | tr || echo` and `git show ... | tr || echo` fallbacks never fire because the pipeline exit status comes from `tr`, and 'Detect platform' in the Step 1 heading is never actually done there. The deliverable-producing steps are badly underspecified: Step 4 says 'build a single table' and lists jq fields but gives no table layout, no columns, no guidance on showing the per-level `.version` values or the base-vs-HEAD VERSION drift, and no offline rendering format despite offline being an explicit case the workflow must handle. Step 5 ('suggest a sensible next action') contains zero decision logic—no mapping from queue state, siblings, offline status, or version drift to a recommended action. An agent can execute the data-gathering commands but must invent the dashboard and the next-action recommendation, so the expected outputs are not reliably reproducible."},"latency_ms":14866,"stop_reason":"end_turn","usage":{"input_tokens":1417,"output_tokens":1049},"cost_usd":0.06662}
+{"config":"workflow-default","item":"f07","split":"heldout","phase":"old-b","sample":2,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":2,"reasoning":"Step 1 (ask/auto-upgrade/snooze), Step 4.75 (migrations), Step 5 (marker + cache), Step 6 (What's New) and Step 7 are concrete and executable. But the core of the workflow is hollow or contradictory: Step 2 'Detect install type' is just 'Figure out how gstack was installed and where' with no commands, and INSTALL_DIR is never defined despite being used in Steps 3, 4.75, 5, and 6. Step 4 'Upgrade' is a single vague sentence ('whatever method fits... put it back the way it was') with no git pull / fresh-clone / backup / ./setup instructions, yet Step 1 points to 'Step 4's install-specific recovery' (vendored backup restore, git pre-upgrade commit) which does not exist, and Step 4.75 says 'After ./setup completes' though setup is never invoked. Step 3 explicitly says to capture OLD_VERSION after Step 4 finishes, which would record the new version and break both the migration gating and the What's New range—a direct ordering contradiction. Version comparison and backup procedures are absent. An agent could not reliably execute the upgrade itself from these instructions."},"latency_ms":14225,"stop_reason":"end_turn","usage":{"input_tokens":3229,"output_tokens":1057},"cost_usd":0.08514}
+{"config":"workflow-default","item":"f08","split":"heldout","phase":"old-b","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Step 0 says gates 'may run before or after intent routing, depending on the situation' with no decision rule, and the routing list starts at item 4 with items 1-3 (presumably the gates) absent, so the agent cannot tell when the consent/setup gates fire relative to routing. The consent prompt references options A, B, and C in steps 4-5 but never defines them (only 'A' appears in the recommendation line; B is entirely undefined). Step 3 says to touch the prompted-marker 'only if the user enabled tuning', contradicting the privacy note ('honored with a marker file so the user is never re-asked') and the setup section's 'touch it even if the user bails out'; it also sits before the branch that determines the answer. The 5-Q setup never supplies the five questions, their answer options, or the numeric scale for <Q1_VALUE>..<Q5_VALUE>, yet the deliverable is numeric values in developer-profile.json; it also says 'after each answer, save' while the script writes all five at once. Power-user shortcuts `stats`, `audit`, `setup` have no defined routes. The bash snippets are executable, but the core decisions and content needed to produce the expected output are missing or contradictory."},"latency_ms":25764,"stop_reason":"end_turn","usage":{"input_tokens":3695,"output_tokens":1822},"cost_usd":0.12805}
+{"config":"workflow-default","item":"p08","split":"heldout","phase":"old-b","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The skill is dense but largely unambiguous: scope and read-only boundary are explicit, Git access is pinned to a specific safe wrapper with a mandatory probe and clear fallbacks (pinned-commit GET API reads, then labeled raw source with disclosed gaps), the older-open-PR scan has concrete endpoints and a precisely counted page budget, and the rubric gives testable criteria (two verified first-party callers, removed/added/saved accounting split by implementation vs total). Output has three well-defined sections and an explicit stop condition. Minor gaps lower it from 5: the rule for choosing the 'selected review commit' when the default tip is not accessible is only implied, provider handling beyond GitHub is left generic, and the sheer volume of prohibitions and the wall-of-text formatting make it easy for an agent to miss a constraint (e.g., repeated page numbers costing budget, or not reusing cached responses). No contradictory ordering was found; the workflow is executable end to end."},"latency_ms":15027,"stop_reason":"end_turn","usage":{"input_tokens":4689,"output_tokens":911},"cost_usd":0.09244}
+{"config":"workflow-default","item":"p06","split":"heldout","phase":"new","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is concrete and sequenced: output-dir/slug rules, a runnable staging script, a single render command with explicit success/ERROR/PAGE_ERRORS handling, and a defined delivery checklist. Two gaps hold it back. First, the non-ASCII note tells the agent to swap `atob('…')` for `decodeURIComponent(escape(atob('…')))` but the render template uses plain `atob`, leaving the agent to decide when and how to rewrite the --eval strings. Second, Step 4 declares the `.mmd` the 'single source of truth' yet also supports re-rendering an edited `.excalidraw`, without saying how the user returns the edited scene or whether the now-divergent `.mmd` should be updated or flagged. Otherwise an agent can execute end to end and produce the triplet reliably."},"latency_ms":20338,"stop_reason":"end_turn","usage":{"input_tokens":4101,"output_tokens":1249},"cost_usd":0.10346}
+{"config":"workflow-default","item":"f07","split":"heldout","phase":"old-b","sample":1,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Step 1 (ask/auto-upgrade/snooze), Step 4.75 (migrations), Step 5 (marker), and Step 6 (changelog summary) are concrete and runnable, but the core of the workflow is hollow or self-contradictory. Step 2 'Detect install type' is a one-line placeholder with no commands and never defines $INSTALL_DIR, which every later step depends on. Step 4 'Upgrade' is likewise a placeholder ('whatever method fits the install... put it back the way it was') with no git pull / fresh clone / backup / ./setup instructions, even though Step 1 explicitly points to 'Step 4's install-specific recovery' (vendored backup restore vs. git pre-upgrade commit) and Step 4.75 assumes './setup completes'. Step 3 instructs capturing OLD_VERSION *after* Step 4 finishes, which would record the new version and break both the migration gating and the what's-new range — a direct ordering contradiction. Version comparison is never shown. An agent cannot reliably detect the install, back it up, upgrade, run setup, or recover on failure from these instructions."},"latency_ms":13917,"stop_reason":"end_turn","usage":{"input_tokens":3229,"output_tokens":968},"cost_usd":0.08069}
+{"config":"workflow-default","item":"a08","split":"heldout","phase":"old-b","sample":0,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"Steps 1-3 are concrete, runnable bash with clear fallbacks (base-branch detection, VERSION reads, per-level queue query to /tmp JSON), though the `cat ... | tr ... || echo` fallbacks are subtly wrong (the `||` binds to `tr`, so a missing VERSION yields empty rather than the intended default). The deliverables, however, are underspecified: Step 4 lists jq fields but gives no table template, no column layout, no instruction on how to show the per-level `.version` values the loop was built to surface, and nothing on how to render when `.offline` is true despite offline output being a required part of the dashboard. Step 5 ('suggest a sensible next action') has no decision logic at all—no mapping from queue state, siblings, offline status, or version drift to a recommended action. An agent can execute the data-gathering commands reliably but must invent the dashboard format and next-action heuristic, so the output will be inconsistent across runs."},"latency_ms":15606,"stop_reason":"end_turn","usage":{"input_tokens":1417,"output_tokens":1144},"cost_usd":0.07137}
+{"config":"workflow-default","item":"p07","split":"heldout","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: a clear hard gate, explicit default selection policy (current-branch-first with cross-branch fallback) that is consistently reflected in the Step 1 script, labeled output tokens (NO_CHECKPOINTS, NO_VERIFIED_CHECKPOINTS, IDENTITY_CHECK_FAILED, FOREIGN/UNVERIFIED/ROOT_DIFFERS, SHARED_BUCKET, LEGACY_BUCKET) each mapped to a specific agent response, a concrete presentation template, a branch-mismatch note, and a defined AskUserQuestion follow-up. Weaknesses: the user-specified path is underspecified — the '<number>' argument has no defined indexing (the candidate list is unnumbered and output lines are mixed with PROJECT_DIR/PROJECT_IDENTITY lines), matching a 'title fragment' isn't tied to filename vs. title frontmatter, and no-match or multiple-match cases aren't handled; the 'Duration' field's source isn't stated; 'first plain path line' requires the agent to mentally exclude several other labeled lines. The script also redundantly re-derives GSTACK_STATE_ROOT. These are minor gaps in an otherwise executable, non-contradictory workflow."},"latency_ms":26370,"stop_reason":"end_turn","usage":{"input_tokens":4963,"output_tokens":1853},"cost_usd":0.14228}
+{"config":"workflow-default","item":"a07","split":"heldout","phase":"old-a","sample":1,"status":"ok","value":{"clarity":3,"completeness":3,"actionability":4,"reasoning":"The workflow is well-structured with a clear persona, hard gate, detection→run→score→dashboard→persist→trend pipeline, and unusually careful capture semantics (private logs, true exit codes, parser guards, ERROR vs SKIPPED vs CLEAN distinctions, like-for-like trend comparison). However, several decision points are underspecified or contradictory: the rubric only yields scores of 10/7/4/0 yet the status labels (7-9 WARNING, 4-6 NEEDS WORK) and dashboard/history examples use 8s and 9s with no interpolation rule; the GBrain rubric's OR-conditions overlap across tiers; the score for a non-zero exit with zero parsed matches is left undefined; Step 4's 'list the top issues' section ends in a colon with no format; the trend header says 'last 5 runs' while the code reads 10; IMPROVING/STABLE/REGRESSING thresholds and HIGH/MED/LOW cutoffs are not defined; and the detection script has shell precedence bugs (`a || b && echo`). An agent can still execute the flow and produce the deliverables, but will have to make several unguided judgment calls on scoring and labeling."},"latency_ms":35044,"stop_reason":"end_turn","usage":{"input_tokens":6343,"output_tokens":1988},"cost_usd":0.16283}
+{"config":"workflow-default","item":"p06","split":"heldout","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":5,"reasoning":"The workflow is well-structured and sequential: authoring rules with concrete type-specific editability caveats, explicit outdir/slug decisions, a robust copy-pasteable staging script, a single render command with ordered --eval/--out pairs, and precise interpretation of ERROR/PAGE_ERRORS/NEEDS_ASIDE outputs with stop conditions. Delivery steps and the excalidraw round-trip are spelled out. Minor gaps: the non-ASCII `decodeURIComponent(escape(atob()))` fix is a footnote rather than integrated into both affected eval expressions, so the agent must infer how to modify the command; `./diagrams/` creation is not mentioned; and the 'the .mmd is the single source of truth' rule sits in tension with the edited-.excalidraw re-render path, which leaves the .mmd stale with no guidance on reconciling them. Otherwise the instructions are unambiguous and directly executable."},"latency_ms":20772,"stop_reason":"end_turn","usage":{"input_tokens":3704,"output_tokens":1368},"cost_usd":0.10544}
+{"config":"workflow-default","item":"f07","split":"heldout","phase":"new","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Step 1 and Steps 4.75–7 are concrete, but the core upgrade mechanics are hollowed out. Step 2 says only \"Figure out how gstack was installed and where\" with no commands or decision rules, yet `$INSTALL_DIR` is consumed by every later step and Step 1 promises install-specific recovery (vendored backup restore vs. git stop) that Step 4 never defines—Step 4 is just \"upgrade with whatever method fits.\" Step 3 is self-contradictory: it tells the agent to capture `OLD_VERSION` from `VERSION` *after* Step 4 finishes, which would record the new version and break migration gating, the marker file, and the What's New range. No backup, git pull/clone, or `./setup` invocation exists despite being referenced."},"latency_ms":11465,"stop_reason":"end_turn","usage":{"input_tokens":3626,"output_tokens":787},"cost_usd":0.07561}
+{"config":"workflow-default","item":"a07","split":"heldout","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured with a clear persona, hard gate (no fixing), six ordered steps, explicit capture semantics (SKIPPED vs ERROR vs failure, exit code handling, full-log parsing), a weighted composite computed in code with explicit redistribution rules, dashboard/history formats, and like-for-like trend comparison. Weaknesses: the scoring rubric only yields 10/7/4/0 yet the dashboard and trend examples show 8/10 and 9, leaving intermediate scores undefined; the GBrain rubric uses overlapping OR thresholds and never specifies how to parse `gbrain doctor --json`; lint/knip count parsing is vague ('error/warning patterns'); the Shell lint '0' column says 'N/A (skip)' rather than a score; the biome/pylint detection lines have a `||`/`&&` precedence bug that will misdetect tools; Step 4 ends on a dangling 'list the top issues...:' with no format; and Step 6 tails history after Step 5 appended the current row without stating to exclude it when picking 'previous run'. These are resolvable by a competent agent, so the workflow remains executable and produces the expected deliverables, but not without some judgment calls."},"latency_ms":36757,"stop_reason":"end_turn","usage":{"input_tokens":6343,"output_tokens":2093},"cost_usd":0.16808}
+{"config":"workflow-default","item":"p07","split":"heldout","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is well-structured: command parsing, a runnable discovery script with clearly named sentinel outputs (NO_CHECKPOINTS, NO_VERIFIED_CHECKPOINTS, IDENTITY_CHECK_FAILED, FOREIGN/UNVERIFIED/ROOT_DIFFERS, SHARED_BUCKET, LEGACY_BUCKET), each mapped to explicit user-facing behavior in Step 2, a concrete presentation template, a branch-mismatch note, an AskUserQuestion next-step menu, and a distinct no-contexts path. Branch-first ordering vs. fallback is explained consistently and the HARD GATE is respected by option A (suggest, don't implement). Gaps: the `<title-fragment-or-number>` path never defines what 'number' means (list index? timestamp?) or what field a title fragment matches against (filename vs. frontmatter title), nor how to handle zero or multiple matches; 'Duration ... (if available)' doesn't say where the value comes from; the script redundantly re-resolves GSTACK_STATE_ROOT and doesn't guard an empty SLUG. These are minor relative to an otherwise executable, unambiguous flow."},"latency_ms":18428,"stop_reason":"end_turn","usage":{"input_tokens":4963,"output_tokens":1465},"cost_usd":0.12288}
+{"config":"workflow-default","item":"a07","split":"heldout","phase":"old-a","sample":2,"status":"ok","value":{"clarity":3,"completeness":4,"actionability":4,"reasoning":"The workflow is well-sequenced (detect → run with honest capture → score → dashboard → persist → trend/recommend) with a strong capture script, explicit SKIPPED/ERROR/CLEAN semantics, like-for-like coverage comparison, and a code-computed composite. However, several inconsistencies undermine clarity: the rubric defines only 10/7/4/0 bands while status labels and examples assume intermediate scores (8, 9, 6) with no interpolation rule; the dashboard example's composite (9.1) doesn't match its own weights (≈9.3); Step 6 reads 'last 10 entries' but the example says 'last 5 runs'; the 'list the top issues' instruction ends in a dangling colon with no format; and the auto-detect script mixes zsh (`setopt`) with bash and has `||`/`&&` precedence bugs. The composite script also only models null (skipped), leaving ERROR→N/A handling to prose in Step 2. Still, an agent can execute every step and produce the dashboard, history row, trend, and ranked recommendations with minor judgment calls."},"latency_ms":35066,"stop_reason":"end_turn","usage":{"input_tokens":6343,"output_tokens":2058},"cost_usd":0.16633}
+{"config":"workflow-default","item":"f08","split":"heldout","phase":"new","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The workflow has structural gaps that block execution. The consent prompt never defines options A/B/C, yet steps 4–5 branch on them, and the marker is touched only if enabled while the privacy note promises decliners are never re-asked—a direct contradiction. The 5-Q setup omits the five questions entirely (\"Ask about their preferences on each dimension\") and gives no answer-to-number mapping for <Q1_VALUE>…<Q5_VALUE>, while instructing per-answer saves but supplying an all-at-once script. Step 0 says gates run \"before or after\" routing without defining them, and routing list starts at item 4. An agent could not reliably produce the numeric profile."},"latency_ms":9946,"stop_reason":"end_turn","usage":{"input_tokens":4092,"output_tokens":721},"cost_usd":0.07697}
+{"config":"workflow-default","item":"p08","split":"heldout","phase":"old-a","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow has a coherent sequence (establish source → survey recent commits/PRs → check older open PRs → evaluate via rubric → report) with explicit read-only boundaries, concrete API endpoints, a precise page-budget scheme with counters, a detailed scoring rubric, and a well-specified three-part output. Fallback paths (safe-git probe failure → pinned-commit GET reads → labeled raw source) and disclosure requirements are spelled out. Weaknesses: the prose is very dense and some decisions are implied rather than stated (how the 'selected review commit' is chosen when the default tip is inaccessible; no bound on the primary-window commit/PR scan while the older-open-PR scan is tightly budgeted; no method given for distinguishing generated from authored code beyond 'follow back to templates'). These are minor relative to the overall specificity, so an agent can execute and produce the deliverables with little ambiguity."},"latency_ms":24099,"stop_reason":"end_turn","usage":{"input_tokens":4689,"output_tokens":1261},"cost_usd":0.10994}
+{"config":"workflow-default","item":"a08","split":"heldout","phase":"old-a","sample":1,"status":"ok","value":{"clarity":3,"completeness":2,"actionability":3,"reasoning":"Steps 1-3 are concrete, copy-pasteable shell with clear fallbacks (though the `cat ... | tr ... || echo` fallback is broken since `||` tests tr's exit, so a missing VERSION yields an empty string rather than 0.0.0.0, and the 'detect platform' heading has no platform detection). Step 4 lists the jq fields to extract but never specifies the dashboard/table layout, how to show the per-level `.version` values the loop was built to collect, or how the offline case should be rendered despite the task calling for offline output. Step 5 is a single vague sentence with no decision logic for choosing a next action (e.g., based on version conflicts, active siblings, or offline state). An agent can execute the data-gathering portion reliably but must invent the actual deliverables, leaving the dashboard and recommendation underspecified."},"latency_ms":12275,"stop_reason":"end_turn","usage":{"input_tokens":1417,"output_tokens":894},"cost_usd":0.05887}
+{"config":"workflow-default","item":"f06","split":"heldout","phase":"old-a","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The 'Iron Law' (fix obvious bugs immediately, investigate only if the quick fix fails) directly contradicts Phase 1 ('gather context before forming any hypothesis') and the Phase 3 red flags ('there is no \"for now\"', 'proposing a fix before tracing data flow — you're guessing'), leaving an agent unsure whether to attempt a fix first or investigate first. Phase 2 (Pattern Analysis) is a single vague sentence with no known-pattern catalog, no matching procedure, and no Aside search instructions, yet Phase 3 says 'search through Aside, as in Phase 2' — a dangling reference. The advertised 3-strike hypothesis limit is absent entirely; the wrong-hypothesis branch loops back to Phase 1 with no cap or escalation. Phase 5 asks only to 'summarize what you did' with no structured debug report format, and nothing captures learnings despite Phase 1 step 5 depending on prior learnings existing. Phases 1 and 4 are concrete and actionable (git log, reproduce, failing-then-passing regression test, >5-file blast-radius prompt), which keeps scores above the floor, but the core loop (pattern match → bounded hypothesis testing → structured report) cannot be executed as described."},"latency_ms":12872,"stop_reason":"end_turn","usage":{"input_tokens":1765,"output_tokens":1051},"cost_usd":0.0702}
+{"config":"workflow-default","item":"f06","split":"heldout","phase":"new","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The workflow contradicts itself at the outset: the Iron Law says to fix obvious bugs immediately and investigate only if that fails, while Phase 3 forbids writing ANY fix before verifying a hypothesis and flags 'quick fix' as a red flag. Key promised elements are absent: there is no 3-strike limit on hypotheses, Phase 2 'Pattern Analysis' is a single vague sentence with no bug-pattern catalog or matching procedure, yet Phase 3 says 'search through Aside, as in Phase 2' — a dangling reference. Phase 5 asks only to 'summarize' with no structured report format. Phases 1 and 4 are concrete and executable (git log, failing/passing regression test, >5-file blast-radius prompt), but the missing decision loop and report leave the agent unable to deliver the full expected workflow."},"latency_ms":11458,"stop_reason":"end_turn","usage":{"input_tokens":2162,"output_tokens":840},"cost_usd":0.06362}
+{"config":"workflow-default","item":"f07","split":"heldout","phase":"old-b","sample":0,"status":"ok","value":{"clarity":2,"completeness":1,"actionability":1,"reasoning":"Step 1 (ask/auto-upgrade/snooze) and Steps 4.75–7 contain concrete, runnable code and clear outputs, but the core of the workflow is missing or broken. Step 2 ('Figure out how gstack was installed and where') gives no detection logic and never defines $INSTALL_DIR, which every later step depends on. Step 3 contradicts its own purpose: it instructs saving OLD_VERSION *after* Step 4 finishes, which would capture the new version and make migration gating, the just-upgraded-from marker, and the What's New range all wrong. Step 4 ('Upgrade with whatever method fits... put it back the way it was') has no backup procedure, no git-pull vs fresh-clone branches, no ./setup invocation, and no version comparison—yet Step 1 points to 'Step 4's install-specific recovery' (vendored backup restore, git pre-upgrade commit) that does not exist, and Step 4.75 assumes './setup completes' though nothing runs it. An agent cannot execute detect/backup/upgrade/setup from these instructions and would record incorrect version state if it followed the stated ordering."},"latency_ms":11569,"stop_reason":"end_turn","usage":{"input_tokens":3229,"output_tokens":773},"cost_usd":0.07094}
+{"config":"workflow-default","item":"p08","split":"heldout","phase":"new","sample":2,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is thorough and internally consistent: read-only boundaries, exact safe-git invocation and probe, explicit GitHub endpoints, fallback chains when local history or API access fails, a concrete rubric, and a three-part output spec. Decision points are mostly explicit (e.g., probe failure → pinned-commit API reads → labeled raw source). Weaknesses are density and some underdetermined choices: the 'selected review commit' is only defined by 'prefer that tip if accessible', leaving the alternative (local HEAD? stale tracking ref?) implicit; and the older-open-PR budget prose is convoluted while the recent-window PR paging is unbounded ('continue until... access limits'), creating tension in turn-limited sessions. Still, an agent can execute this and produce the expected deliverables."},"latency_ms":18475,"stop_reason":"end_turn","usage":{"input_tokens":5086,"output_tokens":1117},"cost_usd":0.10671}
+{"config":"workflow-default","item":"f08","split":"heldout","phase":"old-b","sample":2,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Step 0 says gates 'may run before or after intent routing, depending on the situation' with no rule, and the intent list begins at item 4 — the implicit gates (consent, setup) that later sections reference as 'Step 0's consent gate / setup gate' are never actually defined in Step 0. The consent AskUserQuestion prompts end with a recommendation but define no options, yet steps 4–5 branch on 'A or B' vs 'C', and 5-Q setup fires 'on option A' — the agent cannot know what A/B/C mean or how B differs. Step 3 says touch the marker only if the user enabled tuning, which directly contradicts the privacy note ('answer is honored with a marker file so the user is never re-asked') and means decliners are re-prompted. The 5-Q setup says 'Ask about their preferences on each dimension' without listing the five questions, their options, or the numeric scale that maps answers to <Q1_VALUE>..<Q5_VALUE>; dimension names appear only inside the bun script. It also says to save after each answer but the script writes all five at once. Power-user shortcuts `stats` and `audit` have no routing target. The bash snippets for config, markers, and atomic JSON write are concrete and usable, but the core decision points and deliverable (numeric declared profile) are underspecified."},"latency_ms":18954,"stop_reason":"end_turn","usage":{"input_tokens":3695,"output_tokens":1332},"cost_usd":0.10355}
+{"config":"workflow-default","item":"p06","split":"heldout","phase":"old-b","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The single SKILL.md is well-structured: four numbered steps, concrete bash for staging and rendering, explicit output-line parsing (OK/ERROR/PAGE_ERRORS/NEEDS_ASIDE), clear decision rules for output dir, slug, diagram-type editability, parse-error retry, and the no-browser stop condition. Delivery steps and the edited-.excalidraw round-trip are spelled out. Minor gaps and ambiguities keep it from a 5: the 'triplet' is actually four files (naming vs. table); `<outdir>` is never explicitly created; the bundle lookup checks a git-toplevel path but the render command hardcodes `~/.claude/skills/gstack/bin/gstack-render.ts`, so a repo-only install could stage but fail to render; the 'source of truth is the .mmd' rule sits awkwardly with the excalidraw round-trip path that bypasses the .mmd and leaves it stale; and the Latin-1/UTF-8 atob caveat is a footnote rather than integrated into the main command. Overall an agent could execute this and produce the expected deliverables with little guesswork."},"latency_ms":25235,"stop_reason":"end_turn","usage":{"input_tokens":3704,"output_tokens":1724},"cost_usd":0.12324}
+{"config":"workflow-default","item":"p08","split":"heldout","phase":"old-a","sample":2,"status":"ok","value":{"clarity":4,"completeness":5,"actionability":4,"reasoning":"The workflow is well-structured: read-only boundary, source establishment with explicit fallbacks (safe-git probe failure → pinned API reads → labeled raw source), recent-work survey, a precisely budgeted older-open-PR scan, a concrete evaluation rubric with line-accounting rules, a revalidation step, and a clearly specified three-part output. Decision points are mostly explicit (what to do on probe failure, failed API, changed source, uncertain overlap, no surviving candidates). Clarity is reduced by very dense prose—especially the Git/overlay safety paragraphs—and a few vague terms ('relevant current-branch work', 'handle deletions explicitly'). Actionability is high but the strict budget bookkeeping, byte-level overlay comparison, and separate implementation/total line estimates demand careful execution; the recent-work survey itself is unbounded while the older-PR scan is tightly capped, which could create pacing uncertainty within bounded sessions."},"latency_ms":22905,"stop_reason":"end_turn","usage":{"input_tokens":4689,"output_tokens":1216},"cost_usd":0.10769}
+{"config":"workflow-default","item":"f08","split":"heldout","phase":"old-b","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The routing list begins at item 4 with items 1-3 (the implicit gates) absent, and the gate-ordering sentence ('may run before or after intent routing') contradicts the stated requirement to run gates before routing, leaving the consent and setup gates referenced but never defined where they are said to live. The consent AskUserQuestion never enumerates options A/B/C, yet the follow-up steps branch on them and the recommendation cites 'A=9/10'; the difference between A and B is never stated. Step 3 touches the marker only when the user enables, directly contradicting the privacy note's promise that the answer is honored with a marker so the user is 'never re-asked' — a decline (C) will cause re-prompting. The 5-Q setup says 'ask about their preferences on each dimension' without supplying the five questions, the answer options, or the numeric scale that maps to <Q1_VALUE>…<Q5_VALUE>, and it says to save after each answer while the provided script writes all five at once. Power-user shortcuts like `stats` and `audit` have no route. An agent cannot reliably produce the expected deliverables from these instructions."},"latency_ms":18189,"stop_reason":"end_turn","usage":{"input_tokens":3695,"output_tokens":1360},"cost_usd":0.10495}
+{"config":"workflow-default","item":"p08","split":"heldout","phase":"new","sample":0,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The workflow is dense but largely unambiguous: scope, read-only boundaries, source establishment with explicit fallbacks, a bounded older-PR scan with concrete endpoints and page counters, a five-point rubric with defined savings math, and a three-part output. Decision points are mostly covered (e.g., failed probe → pinned API reads → labeled raw source). Gaps: the 'selected review commit' choice between default tip and current-branch work is only loosely resolved ('prefer that tip if accessible'), leaving how to combine branch overlay findings into one revision somewhat open; and the heavy prohibition detail buries the core survey/ranking steps, raising execution friction. Still, an agent can produce the expected deliverables without inventing procedure."},"latency_ms":14311,"stop_reason":"end_turn","usage":{"input_tokens":5086,"output_tokens":828},"cost_usd":0.09226}
+{"config":"workflow-default","item":"p08","split":"heldout","phase":"old-b","sample":1,"status":"ok","value":{"clarity":4,"completeness":4,"actionability":4,"reasoning":"The single-file skill lays out a coherent sequence (establish source → survey recent work → check older open PRs → evaluate via rubric → report) with concrete commands, exact API endpoints, explicit page budgets, and a detailed three-part output spec, so an agent can execute it and produce the deliverables. Read-only boundaries and fallback paths (safe-git probe failure → pinned API reads → labeled raw source) are spelled out. Weaknesses: the text is very dense with many prohibitions interleaved with steps, which raises the chance of missed rules; the choice of 'selected review commit' when the default-branch tip is inaccessible or differs from current-branch work is only implied ('prefer that tip'), not fully decided; non-GitHub provider handling is left generic; and the rubric's accounting rules (ranges, overlapping removals) require judgment without a worked format. These are minor gaps rather than contradictions."},"latency_ms":18369,"stop_reason":"end_turn","usage":{"input_tokens":4689,"output_tokens":897},"cost_usd":0.09174}
+{"config":"workflow-default","item":"f08","split":"heldout","phase":"old-a","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"Significant gaps and contradictions. Step 0's intent list begins at item 4, so the implicit gates (consent, setup) that later sections call 'Step 0's consent gate' / 'Step 0's setup gate' are never actually defined or ordered; the only guidance is 'some gates may run before or after intent routing, depending on the situation,' which is unactionable. The consent AskUserQuestion references options A/B/C in steps 4-5 but never defines them in the prompt text (only a stray 'Completeness: A=9/10'). Step 3 touches the marker 'only if the user enabled tuning,' directly contradicting the privacy note that the answer is 'honored with a marker file so the user is never re-asked' (decliners would be re-prompted). The 5-Q setup says 'ask about their preferences on each dimension' with no actual questions, no option wording, and no mapping of answers to the numeric <Qn_VALUE> placeholders; it also says to save 'after each answer' while the script writes all five at once. Option B's behavior relative to the 5-Q path is unspecified, and several power-user shortcuts (stats, audit) have no routing target. The bash snippets themselves are concrete, but the decision points and question content needed to produce the profile are missing."},"latency_ms":19267,"stop_reason":"end_turn","usage":{"input_tokens":3695,"output_tokens":1321},"cost_usd":0.103}
+{"config":"workflow-default","item":"f06","split":"heldout","phase":"old-b","sample":0,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The entry point contradicts itself: the 'Iron Law' instructs the agent to FIX OBVIOUS BUGS IMMEDIATELY and only investigate if a quick fix fails, while Phase 3's red flags explicitly forbid 'quick fix for now' and Phase 4 demands a confirmed root cause first. Phase 2 (Pattern Analysis) is a single vague sentence, yet Phase 3 refers back to 'searching for the error through Aside, as in Phase 2' and to an undefined 'Aside check' printing READY — a dangling reference with no defined procedure or matching of known bug patterns. The advertised 3-strike hypothesis limit is absent; a wrong hypothesis just loops back to Phase 1 indefinitely with no escalation rule. Phase 5 asks only to 'summarize what you did' with no structured debug report format or required fields. Phases 1 and 4 (symptom collection, git log, reproduction, minimal fix, fail-then-pass regression test, >5-file blast-radius prompt) are concrete and executable, but the inverted Iron Law, missing strike limit, empty pattern phase, and undefined report mean an agent cannot reliably produce the intended deliverables."},"latency_ms":15047,"stop_reason":"end_turn","usage":{"input_tokens":1765,"output_tokens":1165},"cost_usd":0.0759}
+{"config":"workflow-default","item":"f06","split":"heldout","phase":"old-b","sample":1,"status":"ok","value":{"clarity":2,"completeness":2,"actionability":2,"reasoning":"The opening 'Iron Law' (fix obvious bugs immediately, only investigate if the quick fix fails) directly contradicts the rest of the workflow ('Before writing ANY fix, verify your hypothesis', red flags against 'quick fix for now' and fixing before tracing data flow), leaving the agent with no coherent entry rule. Phase 2 'Pattern Analysis' is a single vague sentence with no known-pattern catalog or matching procedure, yet Phase 3 refers back to 'searching for the error through Aside, as in Phase 2' — a dangling reference to instructions that don't exist. The advertised 3-strike hypothesis limit is entirely absent: a wrong hypothesis simply loops back to Phase 1 with no cap or escalation path. Phase 5 asks only to 'summarize what you did' with no structured debug report format, so the key deliverable is undefined. 'Search prior learnings' names no location or tool, and the 'architectural smell' observation has no follow-on action. Phases 1, 3 (confirmation step), and 4 (minimal diff, failing-then-passing regression test, full-suite run, >5-file blast-radius prompt) are concrete and executable, which keeps scores from the floor, but the top-level contradiction, missing hypothesis limit, hollow pattern phase, and unspecified report make the workflow unreliable to follow end-to-end."},"latency_ms":17203,"stop_reason":"end_turn","usage":{"input_tokens":1765,"output_tokens":1382},"cost_usd":0.08675}

test/fixtures/judge-calibration/workflow-default/runs/comparison-1.stop.json

@@ -0,0 +1 @@
+{"stopped":null}

test/judge-calibration.test.ts

@@ -279,9 +279,18 @@ describe('landed schemas match the recorded calibration', () => {
     expect(judgeSource).toContain('buildArmJudgePrompt(task, diff), ARM_JUDGE_MODEL, { jsonSchema: ARM_JUDGE_SCHEMA })');
   });
 
-  test('uncalibrated workflow configurations stay on today\'s request; only ship keeps its existing schema', () => {
-    expect(fs.existsSync(path.join(import.meta.dir, 'fixtures', 'judge-calibration', 'workflow-default', 'runs'))).toBe(false);
-    expect(evalSource.match(/schemaTransport: true/g)).toHaveLength(1);
-    expect(evalSource.match(/compactReasoning: true/g)).toHaveLength(1);
+  test('workflow transport follows the workflow-default calibration; uncalibrated configurations stay on today\'s request', () => {
+    const recorded = result('workflow-default');
+    expect(recorded.complete).toBe(true);
+    const registrations = [...evalSource.matchAll(/await runWorkflowJudge\(\{([\s\S]*?)\n    \}\);/g)].map(match => match[1]!);
+    expect(registrations.length).toBeGreaterThan(14);
+    for (const body of registrations) {
+      const testName = /testName: '([^']+)'/.exec(body)![1];
+      const defaultConfig = !/^\s+(?:agentCapability|model|readInput|thresholds|maxTokens|stream|effort):|\.\.\.COOKIE_WORKFLOW_JUDGE/m.test(body);
+      const expected = testName === 'ship/SKILL.md workflow' || (defaultConfig && recorded.landing.lands);
+      expect(body.includes('schemaTransport: true'), testName).toBe(expected);
+      expect(body.includes('compactReasoning: true'), testName).toBe(testName === 'ship/SKILL.md workflow');
+    }
+    expect(registrations.filter(body => body.includes('schemaTransport: true'))).toHaveLength(15);
   });
 });

test/skill-llm-eval.test.ts

@@ -511,6 +511,7 @@ describeIfSelected('Ship & Release skill evals', ['ship/SKILL.md workflow', 'doc
   testIfSelected('document-release/SKILL.md workflow', async () => {
     await runWorkflowJudge({
       testName: 'document-release/SKILL.md workflow',
+      schemaTransport: true,
       suite: 'Ship & Release skill evals',
       skillPath: 'document-release/SKILL.md',
       startMarker: '# Document Release:',
@@ -528,6 +529,7 @@ describeIfSelected('Plan Review skill evals', [
   testIfSelected('plan-ceo-review/SKILL.md modes', async () => {
     await runWorkflowJudge({
       testName: 'plan-ceo-review/SKILL.md modes',
+      schemaTransport: true,
       suite: 'Plan Review skill evals',
       skillPath: 'plan-ceo-review/SKILL.md',
       startMarker: '## Step 0: Nuclear Scope Challenge',
@@ -540,6 +542,7 @@ describeIfSelected('Plan Review skill evals', [
   testIfSelected('plan-eng-review/SKILL.md sections', async () => {
     await runWorkflowJudge({
       testName: 'plan-eng-review/SKILL.md sections',
+      schemaTransport: true,
       suite: 'Plan Review skill evals',
       skillPath: ENG_REVIEW_EXCERPT.skillPath,
       startMarker: '# Plan Review Mode',
@@ -552,6 +555,7 @@ describeIfSelected('Plan Review skill evals', [
   testIfSelected('plan-design-review/SKILL.md passes', async () => {
     await runWorkflowJudge({
       testName: 'plan-design-review/SKILL.md passes',
+      schemaTransport: true,
       suite: 'Plan Review skill evals',
       skillPath: 'plan-design-review/SKILL.md',
       startMarker: '## Review Sections',
@@ -567,6 +571,7 @@ describeIfSelected('Design skill evals', ['design-review/SKILL.md fix loop', 'de
   testIfSelected('design-review/SKILL.md fix loop', async () => {
     await runWorkflowJudge({
       testName: 'design-review/SKILL.md fix loop',
+      schemaTransport: true,
       suite: 'Design skill evals',
       skillPath: 'design-review/SKILL.md',
       startMarker: '## Phase 7:',
@@ -579,6 +584,7 @@ describeIfSelected('Design skill evals', ['design-review/SKILL.md fix loop', 'de
   testIfSelected('design-consultation/SKILL.md research', async () => {
     await runWorkflowJudge({
       testName: 'design-consultation/SKILL.md research',
+      schemaTransport: true,
       suite: 'Design skill evals',
       skillPath: 'design-consultation/SKILL.md',
       startMarker: '## Phase 0:',
@@ -597,6 +603,7 @@ describeIfSelected('Deploy skill evals', [
   testIfSelected('land-and-deploy/SKILL.md workflow', async () => {
     await runWorkflowJudge({
       testName: 'land-and-deploy/SKILL.md workflow',
+      schemaTransport: true,
       suite: 'Deploy skill evals',
       skillPath: 'land-and-deploy/SKILL.md',
       startMarker: '## Step 1: Pre-flight',
@@ -609,6 +616,7 @@ describeIfSelected('Deploy skill evals', [
   testIfSelected('canary/SKILL.md monitoring loop', async () => {
     await runWorkflowJudge({
       testName: 'canary/SKILL.md monitoring loop',
+      schemaTransport: true,
       suite: 'Deploy skill evals',
       skillPath: 'canary/SKILL.md',
       startMarker: '### Phase 2: Baseline Capture',
@@ -621,6 +629,7 @@ describeIfSelected('Deploy skill evals', [
   testIfSelected('benchmark/SKILL.md perf collection', async () => {
     await runWorkflowJudge({
       testName: 'benchmark/SKILL.md perf collection',
+      schemaTransport: true,
       suite: 'Deploy skill evals',
       skillPath: 'benchmark/SKILL.md',
       startMarker: '### Phase 3: Performance Data Collection',
@@ -633,6 +642,7 @@ describeIfSelected('Deploy skill evals', [
   testIfSelected('setup-deploy/SKILL.md platform setup', async () => {
     await runWorkflowJudge({
       testName: 'setup-deploy/SKILL.md platform setup',
+      schemaTransport: true,
       suite: 'Deploy skill evals',
       skillPath: 'setup-deploy/SKILL.md',
       startMarker: '### Step 2: Detect platform',
@@ -665,6 +675,7 @@ describeIfSelected('Other skill evals', [
   testIfSelected('sync-gbrain/SKILL.md read-only readiness', async () => {
     await runWorkflowJudge({
       testName: 'sync-gbrain/SKILL.md read-only readiness',
+      schemaTransport: true,
       suite: 'Other skill evals',
       skillPath: 'sync-gbrain/SKILL.md',
       startMarker: '## Step 4: Refresh',
@@ -677,6 +688,7 @@ describeIfSelected('Other skill evals', [
   testIfSelected('retro/SKILL.md instructions', async () => {
     await runWorkflowJudge({
       testName: 'retro/SKILL.md instructions',
+      schemaTransport: true,
       suite: 'Other skill evals',
       skillPath: 'retro/SKILL.md',
       startMarker: '## Instructions',
@@ -689,6 +701,7 @@ describeIfSelected('Other skill evals', [
   testIfSelected('qa-only/SKILL.md workflow', async () => {
     await runWorkflowJudge({
       testName: 'qa-only/SKILL.md workflow',
+      schemaTransport: true,
       suite: 'Other skill evals',
       skillPath: 'qa-only/SKILL.md',
       startMarker: '# /qa-only:',
@@ -702,6 +715,7 @@ describeIfSelected('Other skill evals', [
   testIfSelected('gstack-upgrade/SKILL.md upgrade flow', async () => {
     await runWorkflowJudge({
       testName: 'gstack-upgrade/SKILL.md upgrade flow',
+      schemaTransport: true,
       suite: 'Other skill evals',
       skillPath: 'gstack-upgrade/SKILL.md',
       startMarker: '## Inline upgrade flow',