Files
gstack/scripts/paid-test-durations.json
Garry Tanandcapy-ai[bot] 8695938e74 v1.92.15.0 feat: plan reviews check facts and keep one task list; /office-hours hands its wireframe to design review (#340, #513, #973, #1048, #1791) (#3775)
* G1: gstack-plan-reality check opens every file:line receipt (#973)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G1: office-hours spec reviewer checks paths and flags through a bin-enforced probe (#973)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2: plan-review reruns reconcile the last task list (gstack-tasks prior/supersede) (#1791)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2: plan reviews write task lists through gstack-tasks start/add/close

The emit block shrinks to three calls: prior (now defaulting --dir and
--branch, and printing how to reconcile), start (creates the empty
branch-named list) and add (one validated row per task). close records a
done or dropped prior task. The plan-review result codes move to
lib/result-codes-plan.ts so lib/result-codes.ts stays under the module-size
ratchet; generated plan-review sections shrink, so their size pins go down.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* F2: office-hours saves the sketch beside the design doc; /plan-design-review reads it

Step 5 copies sketch.png/html to <PROJECT_DIR>/{user}-{branch}-sketch-{datetime}.*
and writes a Wireframe: line (state-root relative) or Wireframe: none (<reason>).
The design review section finds the design doc before Pass 1 and opens the
wireframe, text-only when the file is missing. G4's signals line rides along
in the same generated office-hours files; tests in planning-wave-prose.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G3: plan reviews tie storage to queries, not file count (#1048)

One bullet in each of plan-eng-review and plan-ceo-review: fields queried,
constrained or updated on their own get explicit columns or tables; JSON suits
a document read and written whole. Written fresh; the reporter's patch is not
ported.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* evals: GSTA-228 planning-wave cases with branch, main and no-gstack arms

planted facts (G1, behavior), data model (G3, behavior), task-list rerun (G2,
rule) and closing signals (G4, behavior). GSTACK_WAVE_ARM picks the arm; only
the branch arm asserts the preregistered bar, every arm writes a score row.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* changelog: GSTA-228 planning wave fragment

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G1: the probe takes subcommand words for fixed-subcommand CLIs

The planted-facts eval's one branch miss: the reviewer asked to probe
"git log", the probe refused it, and git's top-level help cannot show a log
flag, so the fabricated --since-commit stayed "unverified".
probe --help <command> [<subcommand> [<subcommand>]] now runs
<cli> <sub> --help for git, gh, docker, cargo, go, npm and uv (never node,
bun, python3, make, yarn, pnpm or a gstack bin, where the word could name a
script), in an empty temporary directory with pagers set to cat, falling back
to -h when --help has no usage line (a minimized system's missing man page).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2: task text goes through a rows file, never a command line

The free suite's generated-bash lint (INV-3) refused the add command's
<title>/<finding> placeholders: free text reaches a command only as a file the
agent writes with its file-write tool. start now also creates an empty rows
file in the repository's git-excluded .gstack/tmp and records the run; the
agent writes task and disposition rows there; add validates every row,
stamps phase, run, branch and commit, appends tasks to the list and
dispositions to task-dispositions-<phase>-<branch>.jsonl, then supersedes the
branch's earlier lists. close is gone (dispositions are rows).

The new planning-wave cases get measured periodic walls in
scripts/paid-test-durations.json so the periodic census still packs into 40
slices, the schema case declares the preamble sources its eng-template owners
need, and the paid file count is 114.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2: fold start into add so each plan review emits one bash fence

The rows file lives at .gstack/tmp/tasks-<phase>-rows.jsonl; add writes this
run's list (empty when there are no rows), appends dispositions, deletes the
rows file and supersedes the branch's earlier lists. Lowers 27 size pins.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2 tests: give gstack-tasks a private HOME and state root

CI shard 13 home containment caught gstack-slug writing ~/.gstack/slug-cache
entries from the tasks-prior fixtures.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* changelog: gstack-tasks has no start command

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G1/G2: tighten the probe and the rows file after ship review

The probe now accepts only each CLI's listed built-in subcommands (no uv run,
docker run, go test, npm exec or gh extensions), refuses gstack bins that do not
handle --help and never retries them with -h, and runs with a throwaway HOME and
an environment without keys. gstack-tasks resolves the slug through
runScriptSync (Windows) and prior deletes a rows file an unfinished run left.
Adds tests for the dropped disposition and the stale rows file.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G2: add refuses while an earlier task is neither carried nor closed

Ship review (state matrix): a rerun that skipped RECONCILE silently retired
open tasks. add now names each undecided <run_id>/<id> and writes nothing.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* G1/G2: fixes from the ship fix-check and Step 11 voices

Probe: find the repository root without running git, drop relative and
in-repository PATH entries for lookup and for the child, keep HOME and
toolchain locations (rustup, mise, asdf) but no keys, drop cargo fmt/clippy
(external cargo-* programs), restore planted env in the test.
gstack-tasks: resolve the exclude path against the cwd (subdirectories),
resolve --dir, stamp an empty branch on a detached HEAD like the aggregator,
name the file from --branch, and refuse a same-second second list instead of
appending. Receipts: line 0, reversed ranges, dangling links and paths
through files are unresolved instead of crashing.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* changelog: probe limits and the reconcile refusal

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: probe subcommand counts per CLI (ship docs audit)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-11-01 06:11:17 +00:00

227 lines
14 KiB
JSON

{
"version": 2,
"recordedAt": "2026-10-04T18:45:27.037Z",
"tiers": {
"gate": {
"test/llm-judge-recommendation.test.ts": 1000,
"test/skill-e2e-ask-user-question-format-compliance.test.ts": 54768,
"test/skill-e2e-autoplan-dual-voice.test.ts": 1000,
"test/skill-e2e-bws.test.ts": 74261,
"test/skill-e2e-context-skills.test.ts": 1000,
"test/skill-e2e-coverage-audit.test.ts": 65669,
"test/skill-e2e-cso.test.ts": 229620,
"test/skill-e2e-deploy.test.ts#benchmark-workflow": 120646,
"test/skill-e2e-deploy.test.ts#canary-workflow": 42804,
"test/skill-e2e-deploy.test.ts#land-and-deploy-first-run": 103709,
"test/skill-e2e-deploy.test.ts#land-and-deploy-review-gate": 147499,
"test/skill-e2e-deploy.test.ts#land-and-deploy-workflow": 148339,
"test/skill-e2e-deploy.test.ts#setup-deploy-workflow": 36798,
"test/skill-e2e-design.test.ts": 261000,
"test/skill-e2e-design.test.ts#design-review-detector-shim": 45224,
"test/skill-e2e-design.test.ts#design-review-detector-shim-dom": 87823,
"test/skill-e2e-design.test.ts#design-review-plugin-handoff": 74874,
"test/skill-e2e-design.test.ts#plan-design-review-no-ui-scope": 39970,
"test/skill-e2e-diagram.test.ts": 30124,
"test/skill-e2e-docsync-spawned.test.ts": 108078,
"test/skill-e2e-gbrain-roundtrip-local.test.ts": 1000,
"test/skill-e2e-hermetic-canary.test.ts": 4951,
"test/skill-e2e-investigate-owned-completion.test.ts": 36179,
"test/skill-e2e-investigate-owned-termination.test.ts": 48428,
"test/skill-e2e-ios-device.test.ts": 1000,
"test/skill-e2e-learnings.test.ts": 33590,
"test/skill-e2e-office-hours-auto-mode.test.ts": 78399,
"test/skill-e2e-office-hours-brain-writeback.test.ts": 1000,
"test/skill-e2e-office-hours-phase4.test.ts": 1000,
"test/skill-e2e-office-hours.test.ts": 1000,
"test/skill-e2e-plan-ceo-finding-floor.test.ts": 248087,
"test/skill-e2e-plan-ceo-plan-mode.test.ts": 112067,
"test/skill-e2e-plan-decision-classification.test.ts": 1000,
"test/skill-e2e-plan-design-with-ui.test.ts": 285788,
"test/skill-e2e-plan-devex-finding-floor.test.ts": 181446,
"test/skill-e2e-plan-devex-peer-comparison-classification.test.ts": 1000,
"test/skill-e2e-plan-devex-plan-mode.test.ts": 105953,
"test/skill-e2e-plan-format.test.ts": 1000,
"test/skill-e2e-plan-mode-no-op.test.ts": 271596,
"test/skill-e2e-plan-prosons.test.ts": 1000,
"test/skill-e2e-plan-tune.test.ts": 70702,
"test/skill-e2e-plan.test.ts": 251000,
"test/skill-e2e-plan.test.ts#plan-review-report": 197666,
"test/skill-e2e-qa-bugs.test.ts": 1000,
"test/skill-e2e-qa-callers.test.ts": 737164,
"test/skill-e2e-qa-callers.test.ts#review-exploratory-small-cli": 224426,
"test/skill-e2e-qa-callers.test.ts#ship-exploratory-late-input": 234381,
"test/skill-e2e-qa-callers.test.ts#ship-exploratory-plan-checks": 192841,
"test/skill-e2e-qa-callers.test.ts#ship-exploratory-small-cli": 147399,
"test/skill-e2e-qa-callers.test.ts#ship-exploratory-unavailable": 191231,
"test/skill-e2e-qa-functional-fix.test.ts": 200616,
"test/skill-e2e-qa-functional.test.ts": 188766,
"test/skill-e2e-qa-workflow.test.ts#qa-bootstrap": 35863,
"test/skill-e2e-qa-workflow.test.ts#qa-only-no-fix": 201333,
"test/skill-e2e-qa-workflow.test.ts#qa-quick": 169852,
"test/skill-e2e-retro.test.ts": 200434,
"test/skill-e2e-review-army.test.ts": 520000,
"test/skill-e2e-review-army.test.ts#review-army-delivery-audit": 39313,
"test/skill-e2e-review-army.test.ts#review-army-json-findings": 22950,
"test/skill-e2e-review-army.test.ts#review-army-migration-safety": 162617,
"test/skill-e2e-review-army.test.ts#review-army-perf-n-plus-one": 114005,
"test/skill-e2e-review-army.test.ts#review-army-quality-score": 55496,
"test/skill-e2e-review-attribution.test.ts": 80562,
"test/skill-e2e-review.test.ts": 132863,
"test/skill-e2e-safety-codex-boundary.test.ts": 1000,
"test/skill-e2e-safety-codex-consult.test.ts": 150155,
"test/skill-e2e-safety-design-risk.test.ts": 1000,
"test/skill-e2e-safety-ios-demo.test.ts": 1000,
"test/skill-e2e-safety-pair-agent.test.ts": 36028,
"test/skill-e2e-safety-ship-evidence.test.ts": 100833,
"test/skill-e2e-session-intelligence.test.ts": 53080,
"test/skill-e2e-shared-libs-paths.test.ts": 680000,
"test/skill-e2e-shared-libs-paths.test.ts#shared-libs-review-index-flags": 145972,
"test/skill-e2e-shared-libs-paths.test.ts#shared-libs-review-path-eligibility": 170775,
"test/skill-e2e-shared-libs-paths.test.ts#shared-libs-review-prior-coverage": 172422,
"test/skill-e2e-shared-libs.test.ts": 658000,
"test/skill-e2e-shared-libs.test.ts#shared-libs-read-only": 160556,
"test/skill-e2e-shared-libs.test.ts#shared-libs-review-lifecycle": 164017,
"test/skill-e2e-shared-libs.test.ts#shared-libs-review-revalidation": 281097,
"test/skill-e2e-shared-libs.test.ts#shared-libs-unsupported-git": 87581,
"test/skill-e2e-ship-docsync.test.ts": 129000,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-completion": 398349,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-current": 382630,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-failure": 238690,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-late-result": 132034,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-launch-failure": 53611,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-missing-asset": 106550,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-missing-marker": 59963,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-recovery": 132137,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-stale-after": 155419,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-stale-before": 164443,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-store": 392835,
"test/skill-e2e-ship-docsync.test.ts#ship-docsync-timeout-unsettled": 63645,
"test/skill-e2e-ship-hook-consent.test.ts": 49572,
"test/skill-e2e-ship-hook-refresh.test.ts": 40853,
"test/skill-e2e-ship-skip.test.ts": 81807,
"test/skill-e2e-skillify.test.ts": 177262,
"test/skill-e2e-test-value.test.ts": 216843,
"test/skill-e2e-third-party-actions.test.ts": 71961,
"test/skill-e2e-triage.test.ts": 64816,
"test/skill-e2e-workflow.test.ts": 265175,
"test/skill-llm-eval.test.ts": 354000,
"test/skill-routing-e2e.test.ts": 1000
},
"periodic": {
"test/carve-section-loading.test.ts#carve-section-loading-browse": 92911,
"test/carve-section-loading.test.ts#carve-section-loading-codex": 150172,
"test/carve-section-loading.test.ts#carve-section-loading-design-consultation": 318385,
"test/carve-section-loading.test.ts#carve-section-loading-design-html": 245660,
"test/carve-section-loading.test.ts#carve-section-loading-design-shotgun": 140298,
"test/carve-section-loading.test.ts#carve-section-loading-document-release": 97810,
"test/carve-section-loading.test.ts#carve-section-loading-land-and-deploy": 156293,
"test/carve-section-loading.test.ts#carve-section-loading-plan-design-review": 223606,
"test/carve-section-loading.test.ts#carve-section-loading-plan-devex-review": 384295,
"test/carve-section-loading.test.ts#carve-section-loading-plan-eng-review": 297627,
"test/carve-section-loading.test.ts#carve-section-loading-qa": 149637,
"test/carve-section-loading.test.ts#carve-section-loading-retro": 199931,
"test/carve-section-loading.test.ts#carve-section-loading-review": 247902,
"test/carve-section-loading.test.ts#carve-section-loading-setup-gbrain": 80994,
"test/carve-section-loading.test.ts#carve-section-loading-spec": 176639,
"test/codex-e2e-multiblock-live.test.ts": 180000,
"test/codex-e2e-recommendation-substance.test.ts": 14617,
"test/codex-e2e-shared-libs.test.ts": 49563,
"test/codex-e2e-sol-scope.test.ts": 32228,
"test/codex-e2e.test.ts": 110677,
"test/llm-judge-recommendation.test.ts": 21562,
"test/skill-e2e-arm-benchmark.test.ts": 72714,
"test/skill-e2e-aside.test.ts": 1000,
"test/skill-e2e-auq-consistency.test.ts": 63226,
"test/skill-e2e-auq-matrix.test.ts": 145925,
"test/skill-e2e-auq-verbose-vs-carved-ab.test.ts": 65480,
"test/skill-e2e-auto-decide-preserved.test.ts": 234151,
"test/skill-e2e-autoplan-dual-voice.test.ts": 154207,
"test/skill-e2e-benchmark-providers.test.ts": 8847,
"test/skill-e2e-bws.test.ts": 1000,
"test/skill-e2e-context-skills.test.ts": 144874,
"test/skill-e2e-coverage-audit.test.ts": 1000,
"test/skill-e2e-cso.test.ts": 303949,
"test/skill-e2e-design-model-smoke.test.ts": 90000,
"test/skill-e2e-design.test.ts": 817000,
"test/skill-e2e-design.test.ts#design-consultation-core": 173162,
"test/skill-e2e-design.test.ts#design-consultation-existing": 225306,
"test/skill-e2e-design.test.ts#design-consultation-preview": 107037,
"test/skill-e2e-design.test.ts#design-consultation-research": 99890,
"test/skill-e2e-design.test.ts#design-html-slop-gate": 139668,
"test/skill-e2e-design.test.ts#plan-design-review-plan-mode": 169183,
"test/skill-e2e-diagram.test.ts": 52819,
"test/skill-e2e-gbrain-roundtrip-local.test.ts": 1000,
"test/skill-e2e-health.test.ts": 164674,
"test/skill-e2e-hermetic-canary.test.ts": 1000,
"test/skill-e2e-ios-device.test.ts": 1000,
"test/skill-e2e-learnings.test.ts": 1000,
"test/skill-e2e-office-hours-brain-writeback.test.ts": 203044,
"test/skill-e2e-office-hours-design-draft.test.ts": 210162,
"test/skill-e2e-office-hours-phase4.test.ts": 41421,
"test/skill-e2e-office-hours-section-loading.test.ts": 1200000,
"test/skill-e2e-office-hours.test.ts": 136970,
"test/skill-e2e-outside-plan-disabled.test.ts": 72070,
"test/skill-e2e-outside-voice.test.ts": 195221,
"test/skill-e2e-overlay-harness-claude-dedicated-tools-vs-bash-opus-5-5.test.ts": 57374,
"test/skill-e2e-overlay-harness-claude-dedicated-tools-vs-bash-sonnet.test.ts": 77984,
"test/skill-e2e-overlay-harness-claude-dedicated-tools-vs-bash.test.ts": 55431,
"test/skill-e2e-overlay-harness-opus-4-7-effort-match-trivial.test.ts": 34905,
"test/skill-e2e-overlay-harness-opus-4-7-literal-interpretation.test.ts": 226920,
"test/skill-e2e-plan-ceo-mode-routing.test.ts": 378316,
"test/skill-e2e-plan-ceo-review-section-loading.test.ts": 302668,
"test/skill-e2e-plan-ceo-split-overflow.test.ts": 504266,
"test/skill-e2e-plan-decision-classification.test.ts": 88190,
"test/skill-e2e-plan-design-finding-floor.test.ts": 131042,
"test/skill-e2e-plan-design-plan-mode.test.ts": 111838,
"test/skill-e2e-plan-devex-peer-comparison-classification.test.ts": 77201,
"test/skill-e2e-plan-eng-artifact-full.test.ts": 595009,
"test/skill-e2e-plan-eng-artifact.test.ts": 406795,
"test/skill-e2e-plan-eng-finding-floor.test.ts": 234397,
"test/skill-e2e-plan-eng-multi-finding-batching.test.ts": 409161,
"test/skill-e2e-plan-eng-plan-mode.test.ts": 170914,
"test/skill-e2e-plan-format.test.ts": 219670,
"test/skill-e2e-plan-prosons.test.ts": 163659,
"test/skill-e2e-plan-tune.test.ts": 1000,
"test/skill-e2e-plan.test.ts": 824000,
"test/skill-e2e-plan.test.ts#plan-ceo-review": 274619,
"test/skill-e2e-plan.test.ts#plan-ceo-review-expansion-energy": 100000,
"test/skill-e2e-plan.test.ts#plan-ceo-review-selective": 265163,
"test/skill-e2e-plan.test.ts#plan-eng-review": 169400,
"test/skill-e2e-qa-bugs.test.ts": 355966,
"test/skill-e2e-qa-workflow.test.ts#qa-fix-loop": 411728,
"test/skill-e2e-retro.test.ts": 150139,
"test/skill-e2e-review-army.test.ts": 511000,
"test/skill-e2e-review-army.test.ts#review-army-consensus": 107375,
"test/skill-e2e-review-army.test.ts#review-army-red-team": 110743,
"test/skill-e2e-review-army.test.ts#review-army-simplification": 140434,
"test/skill-e2e-review-army.test.ts#review-army-simplification-precision": 25020,
"test/skill-e2e-review-attribution.test.ts": 1000,
"test/skill-e2e-review.test.ts": 179362,
"test/skill-e2e-safety-codex-boundary.test.ts": 102851,
"test/skill-e2e-safety-codex-consult.test.ts": 1000,
"test/skill-e2e-safety-design-risk.test.ts": 54889,
"test/skill-e2e-safety-ios-demo.test.ts": 100092,
"test/skill-e2e-safety-pair-agent.test.ts": 1000,
"test/skill-e2e-safety-ship-evidence.test.ts": 1000,
"test/skill-e2e-session-intelligence.test.ts": 1000,
"test/skill-e2e-setup-gbrain-bad-token.test.ts": 39292,
"test/skill-e2e-setup-gbrain-path4-local-pglite.test.ts": 128414,
"test/skill-e2e-setup-gbrain-remote.test.ts": 89701,
"test/skill-e2e-shared-libs-periodic.test.ts": 217221,
"test/skill-e2e-ship-section-loading.test.ts": 189463,
"test/skill-e2e-skillify.test.ts": 58522,
"test/skill-e2e-sync-gbrain-readiness.test.ts": 47502,
"test/skill-e2e-test-value.test.ts": 332000,
"test/skill-e2e-third-party-actions.test.ts": 1000,
"test/skill-e2e-triage.test.ts": 1000,
"test/skill-e2e-workflow.test.ts": 121594,
"test/skill-llm-eval.test.ts": 339704,
"test/skill-routing-e2e.test.ts": 81360,
"test/skill-e2e-planning-wave.test.ts#planning-wave-planted-facts": 74000,
"test/skill-e2e-planning-wave.test.ts#planning-wave-schema": 52000,
"test/skill-e2e-planning-wave.test.ts#planning-wave-tasks-rerun": 37000,
"test/skill-e2e-planning-wave.test.ts#planning-wave-closing-signals": 27000
}
}
}