CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-06-19 00:00:13 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
10495978e61e6fa51985d7687dcafe68dbe4ff8d
gstack/test/fixtures/parity-baseline-v1.47.0.0.json
T
Garry Tan a6fb31726c v1.48.0.0 feat: AskUserQuestion split rule + runtime AUTO_DECIDE carve-out (#1740) 
* feat(preamble): add "Handling 5+ options — split, never drop" rule

Agents repeatedly hit Conductor's 4-option AskUserQuestion cap and
silently drop one option to fit, shrinking the user's decision space.
This rule names the bug and gives two compliant shapes: batch into
≤4-groups (for coherent alternatives) or split into N sequential
per-option calls (for independent scope items, default).

Inline preamble subsection is ~15 lines (rule + buckets + pointer).
Full reference with worked examples, Hold/dependency semantics, and
final-summary validation lives in docs/askuserquestion-split.md.
The agent loads the docs file on demand when N>4.

Per-option call shape: D<N>.k header, ELI10, Recommendation, kind-note
(no completeness score — decision actions, not coverage), Include /
Defer / Cut / Hold buckets. Hold stops the chain immediately; the
final D<N>.final call validates dependencies and confirms the
assembled scope.

question_ids: <skill>-split-<option-slug> (kebab-case ASCII, ≤64
chars). Also fixes orphan "12. " prefix on the existing CJK rule.

Tier-2+ skills inherit via the existing resolver. SKILL.md regenerated
for all 41 affected skills + 3 golden fixtures. Net diff per SKILL.md:
~34 lines (vs ~110 for the full inline version).

6 tests pin the inline contract (4-option cap, buckets, D-numbering,
docs pointer, runtime AUTO_DECIDE gate reference, orphan 12 regression).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(question-pref): runtime AUTO_DECIDE carve-out for *-split-* ids

Split chains (per-option AskUserQuestion calls emitted by the new
"Handling 5+ options" rule) must never be silently auto-approved
via /plan-tune preferences. The user's option set is sacred.

Layer 1 (mechanism): unique <skill>-split-<option-slug> ids prevent
cross-option preference leakage. Layer 2 (this commit): the runtime
checker `gstack-question-preference --check` detects any id matching
*-split-* and forces ASK_NORMALLY even when never-ask or
ask-only-for-one-way preferences exist for that exact id. An
explanatory note tells the user their preference was bypassed and why.

7 tests pin the carve-out: no-pref baseline, never-ask override,
explanatory note text, ask-only-for-one-way override, always-ask
(no note), non-split id containing "split" word (negative case for
regex specificity), multi-skill split id formats.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(e2e): split-overflow regression for /plan-ceo-review

Periodic-tier E2E test that catches the original failure mode the
user complained about: 5+ options for ONE decision must split into
N sequential AskUserQuestion calls, not drop one to fit Conductor's
4-option cap.

Fixture: 5 independent chat-platform integration candidates
(Slack/Discord/Teams/Telegram/Mattermost), each carrying its own
include/defer/cut decision. Floor = 4 review-phase AUQs (standard
[N-1] tolerance band). Pre-fix "drop to 4 + 1 dropped" fails this
floor.

Wired into test/helpers/touchfiles.ts: tier periodic, depends on
plan-ceo-review/**, the new preamble subsection, the question-pref
binary (for the carve-out), and the runner helper. touchfiles.test.ts
expected count bumped 21 → 22 to account for the new entry.

Cost: ~$0.30/run when EVALS_TIER=periodic. Skips silently otherwise.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: post-merge regen + rebase size-budget baseline to v1.47.0.0

After merging origin/main (v1.45 → v1.47), three things needed cleanup:

1. spec/SKILL.md (main's new skill) regenerated to include our split-vs-drop
   preamble subsection — same mechanical regen as the other 41 tier-2+ skills.
2. Three golden ship fixtures refreshed to capture main's GSTACK_PLAN_MODE
   block + /spec routing entry + jargon-list.json refactor.
3. docs/skills.md — added /spec table row that main's PR (#1698/#1733) shipped
   without. Pre-existing failure on main; this PR catches and fixes.

Also rebased test/skill-size-budget.test.ts from v1.44.1 → v1.47.0.0 baseline.
Main's v1.46 (catalog tokens trim) + v1.47 (/spec skill) pushed the v1.44.1
anchor past the 5% ratchet to ×1.059 — pre-existing failure on main. This
PR captures a fresh parity-baseline-v1.47.0.0.json and re-anchors the test
there. Historical v1.44.1.json and v1.46.0.0.json retained in test/fixtures/
for reference. Our subsection contributes ~0.1% of the post-rebase corpus.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v1.48.0.0)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 23:43:07 -07:00

634 lines
15 KiB
JSON
Raw Blame History

{
"tag": "v1.47.0.0",
"capturedAt": "2026-05-27T05:50:57.656Z",
"capturedFromCommit": "e08e5fa8",
"capturedFromBranch": "garrytan/askuserquestion-split-on-overflow",
"totalSkills": 52,
"totalCorpusBytes": 3090887,
"estTotalCatalogTokens": 4116,
"topHeaviest": [
{
"skill": "ship",
"skillMdBytes": 166782,
"skillMdLines": 3099,
"estTokens": 41696,
"tmplBytes": 50495,
"descriptionLen": 291,
"hasGateEval": true,
"hasPeriodicEval": true
},
{
"skill": "plan-ceo-review",
"skillMdBytes": 132488,
"skillMdLines": 2197,
"estTokens": 33122,
"tmplBytes": 63393,
"descriptionLen": 794,
"hasGateEval": true,
"hasPeriodicEval": true
},
{
"skill": "office-hours",
"skillMdBytes": 112842,
"skillMdLines": 2066,
"estTokens": 28211,
"tmplBytes": 55466,
"descriptionLen": 860,
"hasGateEval": true,
"hasPeriodicEval": false
},
{
"skill": "plan-design-review",
"skillMdBytes": 107855,
"skillMdLines": 1928,
"estTokens": 26964,
"tmplBytes": 28624,
"descriptionLen": 218,
"hasGateEval": true,
"hasPeriodicEval": true
},
{
"skill": "plan-devex-review",
"skillMdBytes": 106167,
"skillMdLines": 2119,
"estTokens": 26542,
"tmplBytes": 35680,
"descriptionLen": 250,
"hasGateEval": true,
"hasPeriodicEval": true
},
{
"skill": "plan-eng-review",
"skillMdBytes": 103009,
"skillMdLines": 1762,
"estTokens": 25752,
"tmplBytes": 26234,
"descriptionLen": 231,
"hasGateEval": true,
"hasPeriodicEval": true
},
{
"skill": "spec",
"skillMdBytes": 102629,
"skillMdLines": 2141,
"estTokens": 25657,
"tmplBytes": 28429,
"descriptionLen": 282,
"hasGateEval": true,
"hasPeriodicEval": false
},
{
"skill": "design-review",
"skillMdBytes": 95654,
"skillMdLines": 1932,
"estTokens": 23914,
"tmplBytes": 11674,
"descriptionLen": 304,
"hasGateEval": true,
"hasPeriodicEval": false
},
{
"skill": "review",
"skillMdBytes": 94048,
"skillMdLines": 1762,
"estTokens": 23512,
"tmplBytes": 14099,
"descriptionLen": 205,
"hasGateEval": true,
"hasPeriodicEval": false
},
{
"skill": "land-and-deploy",
"skillMdBytes": 91886,
"skillMdLines": 1856,
"estTokens": 22972,
"tmplBytes": 48624,
"descriptionLen": 160,
"hasGateEval": true,
"hasPeriodicEval": false
}
],
"skills": {
"autoplan": {
"skill": "autoplan",
"skillMdBytes": 90870,
"skillMdLines": 1784,
"estTokens": 22718,
"tmplBytes": 45271,
"descriptionLen": 366,
"hasGateEval": true,
"hasPeriodicEval": true
},
"benchmark": {
"skill": "benchmark",
"skillMdBytes": 33266,
"skillMdLines": 747,
"estTokens": 8317,
"tmplBytes": 9378,
"descriptionLen": 213,
"hasGateEval": true,
"hasPeriodicEval": false
},
"benchmark-models": {
"skill": "benchmark-models",
"skillMdBytes": 29333,
"skillMdLines": 622,
"estTokens": 7333,
"tmplBytes": 6631,
"descriptionLen": 217,
"hasGateEval": false,
"hasPeriodicEval": false
},
"browse": {
"skill": "browse",
"skillMdBytes": 48018,
"skillMdLines": 929,
"estTokens": 12005,
"tmplBytes": 10805,
"descriptionLen": 181,
"hasGateEval": true,
"hasPeriodicEval": false
},
"canary": {
"skill": "canary",
"skillMdBytes": 47105,
"skillMdLines": 990,
"estTokens": 11776,
"tmplBytes": 8033,
"descriptionLen": 180,
"hasGateEval": true,
"hasPeriodicEval": false
},
"careful": {
"skill": "careful",
"skillMdBytes": 2551,
"skillMdLines": 68,
"estTokens": 638,
"tmplBytes": 2435,
"descriptionLen": 315,
"hasGateEval": false,
"hasPeriodicEval": false
},
"codex": {
"skill": "codex",
"skillMdBytes": 79620,
"skillMdLines": 1519,
"estTokens": 19905,
"tmplBytes": 34143,
"descriptionLen": 187,
"hasGateEval": true,
"hasPeriodicEval": false
},
"context-restore": {
"skill": "context-restore",
"skillMdBytes": 41493,
"skillMdLines": 848,
"estTokens": 10373,
"tmplBytes": 5255,
"descriptionLen": 238,
"hasGateEval": true,
"hasPeriodicEval": false
},
"context-save": {
"skill": "context-save",
"skillMdBytes": 45690,
"skillMdLines": 966,
"estTokens": 11423,
"tmplBytes": 9293,
"descriptionLen": 168,
"hasGateEval": true,
"hasPeriodicEval": false
},
"cso": {
"skill": "cso",
"skillMdBytes": 77397,
"skillMdLines": 1451,
"estTokens": 19349,
"tmplBytes": 35158,
"descriptionLen": 196,
"hasGateEval": true,
"hasPeriodicEval": false
},
"design-consultation": {
"skill": "design-consultation",
"skillMdBytes": 79222,
"skillMdLines": 1561,
"estTokens": 19806,
"tmplBytes": 25899,
"descriptionLen": 888,
"hasGateEval": true,
"hasPeriodicEval": false
},
"design-html": {
"skill": "design-html",
"skillMdBytes": 66547,
"skillMdLines": 1449,
"estTokens": 16637,
"tmplBytes": 22567,
"descriptionLen": 233,
"hasGateEval": false,
"hasPeriodicEval": false
},
"design-review": {
"skill": "design-review",
"skillMdBytes": 95654,
"skillMdLines": 1932,
"estTokens": 23914,
"tmplBytes": 11674,
"descriptionLen": 304,
"hasGateEval": true,
"hasPeriodicEval": false
},
"design-shotgun": {
"skill": "design-shotgun",
"skillMdBytes": 62836,
"skillMdLines": 1311,
"estTokens": 15709,
"tmplBytes": 13331,
"descriptionLen": 786,
"hasGateEval": false,
"hasPeriodicEval": false
},
"devex-review": {
"skill": "devex-review",
"skillMdBytes": 64413,
"skillMdLines": 1233,
"estTokens": 16103,
"tmplBytes": 7984,
"descriptionLen": 201,
"hasGateEval": false,
"hasPeriodicEval": false
},
"document-generate": {
"skill": "document-generate",
"skillMdBytes": 52987,
"skillMdLines": 1176,
"estTokens": 13247,
"tmplBytes": 15093,
"descriptionLen": 334,
"hasGateEval": false,
"hasPeriodicEval": false
},
"document-release": {
"skill": "document-release",
"skillMdBytes": 58251,
"skillMdLines": 1235,
"estTokens": 14563,
"tmplBytes": 20362,
"descriptionLen": 192,
"hasGateEval": true,
"hasPeriodicEval": false
},
"freeze": {
"skill": "freeze",
"skillMdBytes": 3154,
"skillMdLines": 92,
"estTokens": 789,
"tmplBytes": 3038,
"descriptionLen": 503,
"hasGateEval": false,
"hasPeriodicEval": false
},
"gstack-upgrade": {
"skill": "gstack-upgrade",
"skillMdBytes": 10817,
"skillMdLines": 285,
"estTokens": 2704,
"tmplBytes": 10667,
"descriptionLen": 163,
"hasGateEval": true,
"hasPeriodicEval": false
},
"guard": {
"skill": "guard",
"skillMdBytes": 3297,
"skillMdLines": 91,
"estTokens": 824,
"tmplBytes": 3181,
"descriptionLen": 686,
"hasGateEval": false,
"hasPeriodicEval": false
},
"health": {
"skill": "health",
"skillMdBytes": 47916,
"skillMdLines": 1014,
"estTokens": 11979,
"tmplBytes": 11617,
"descriptionLen": 184,
"hasGateEval": true,
"hasPeriodicEval": false
},
"investigate": {
"skill": "investigate",
"skillMdBytes": 50409,
"skillMdLines": 1012,
"estTokens": 12602,
"tmplBytes": 11561,
"descriptionLen": 1379,
"hasGateEval": true,
"hasPeriodicEval": false
},
"ios-clean": {
"skill": "ios-clean",
"skillMdBytes": 41045,
"skillMdLines": 813,
"estTokens": 10261,
"tmplBytes": 3851,
"descriptionLen": 252,
"hasGateEval": false,
"hasPeriodicEval": false
},
"ios-design-review": {
"skill": "ios-design-review",
"skillMdBytes": 41631,
"skillMdLines": 815,
"estTokens": 10408,
"tmplBytes": 4417,
"descriptionLen": 209,
"hasGateEval": false,
"hasPeriodicEval": false
},
"ios-fix": {
"skill": "ios-fix",
"skillMdBytes": 40760,
"skillMdLines": 811,
"estTokens": 10190,
"tmplBytes": 3574,
"descriptionLen": 187,
"hasGateEval": false,
"hasPeriodicEval": false
},
"ios-qa": {
"skill": "ios-qa",
"skillMdBytes": 47271,
"skillMdLines": 931,
"estTokens": 11818,
"tmplBytes": 10090,
"descriptionLen": 223,
"hasGateEval": true,
"hasPeriodicEval": false
},
"ios-sync": {
"skill": "ios-sync",
"skillMdBytes": 40737,
"skillMdLines": 804,
"estTokens": 10184,
"tmplBytes": 3544,
"descriptionLen": 269,
"hasGateEval": false,
"hasPeriodicEval": false
},
"land-and-deploy": {
"skill": "land-and-deploy",
"skillMdBytes": 91886,
"skillMdLines": 1856,
"estTokens": 22972,
"tmplBytes": 48624,
"descriptionLen": 160,
"hasGateEval": true,
"hasPeriodicEval": false
},
"landing-report": {
"skill": "landing-report",
"skillMdBytes": 43985,
"skillMdLines": 874,
"estTokens": 10996,
"tmplBytes": 6806,
"descriptionLen": 195,
"hasGateEval": false,
"hasPeriodicEval": false
},
"learn": {
"skill": "learn",
"skillMdBytes": 41722,
"skillMdLines": 891,
"estTokens": 10431,
"tmplBytes": 5594,
"descriptionLen": 178,
"hasGateEval": true,
"hasPeriodicEval": false
},
"make-pdf": {
"skill": "make-pdf",
"skillMdBytes": 29450,
"skillMdLines": 663,
"estTokens": 7363,
"tmplBytes": 5106,
"descriptionLen": 177,
"hasGateEval": false,
"hasPeriodicEval": false
},
"office-hours": {
"skill": "office-hours",
"skillMdBytes": 112842,
"skillMdLines": 2066,
"estTokens": 28211,
"tmplBytes": 55466,
"descriptionLen": 860,
"hasGateEval": true,
"hasPeriodicEval": false
},
"open-gstack-browser": {
"skill": "open-gstack-browser",
"skillMdBytes": 46131,
"skillMdLines": 954,
"estTokens": 11533,
"tmplBytes": 7702,
"descriptionLen": 204,
"hasGateEval": false,
"hasPeriodicEval": false
},
"pair-agent": {
"skill": "pair-agent",
"skillMdBytes": 46939,
"skillMdLines": 1010,
"estTokens": 11735,
"tmplBytes": 8548,
"descriptionLen": 167,
"hasGateEval": false,
"hasPeriodicEval": false
},
"plan-ceo-review": {
"skill": "plan-ceo-review",
"skillMdBytes": 132488,
"skillMdLines": 2197,
"estTokens": 33122,
"tmplBytes": 63393,
"descriptionLen": 794,
"hasGateEval": true,
"hasPeriodicEval": true
},
"plan-design-review": {
"skill": "plan-design-review",
"skillMdBytes": 107855,
"skillMdLines": 1928,
"estTokens": 26964,
"tmplBytes": 28624,
"descriptionLen": 218,
"hasGateEval": true,
"hasPeriodicEval": true
},
"plan-devex-review": {
"skill": "plan-devex-review",
"skillMdBytes": 106167,
"skillMdLines": 2119,
"estTokens": 26542,
"tmplBytes": 35680,
"descriptionLen": 250,
"hasGateEval": true,
"hasPeriodicEval": true
},
"plan-eng-review": {
"skill": "plan-eng-review",
"skillMdBytes": 103009,
"skillMdLines": 1762,
"estTokens": 25752,
"tmplBytes": 26234,
"descriptionLen": 231,
"hasGateEval": true,
"hasPeriodicEval": true
},
"plan-tune": {
"skill": "plan-tune",
"skillMdBytes": 51717,
"skillMdLines": 1077,
"estTokens": 12929,
"tmplBytes": 15586,
"descriptionLen": 325,
"hasGateEval": true,
"hasPeriodicEval": false
},
"qa": {
"skill": "qa",
"skillMdBytes": 73863,
"skillMdLines": 1622,
"estTokens": 18466,
"tmplBytes": 12701,
"descriptionLen": 218,
"hasGateEval": true,
"hasPeriodicEval": false
},
"qa-only": {
"skill": "qa-only",
"skillMdBytes": 56421,
"skillMdLines": 1194,
"estTokens": 14105,
"tmplBytes": 3851,
"descriptionLen": 165,
"hasGateEval": true,
"hasPeriodicEval": false
},
"retro": {
"skill": "retro",
"skillMdBytes": 82889,
"skillMdLines": 1750,
"estTokens": 20722,
"tmplBytes": 42427,
"descriptionLen": 648,
"hasGateEval": true,
"hasPeriodicEval": false
},
"review": {
"skill": "review",
"skillMdBytes": 94048,
"skillMdLines": 1762,
"estTokens": 23512,
"tmplBytes": 14099,
"descriptionLen": 205,
"hasGateEval": true,
"hasPeriodicEval": false
},
"scrape": {
"skill": "scrape",
"skillMdBytes": 43641,
"skillMdLines": 887,
"estTokens": 10910,
"tmplBytes": 5220,
"descriptionLen": 167,
"hasGateEval": true,
"hasPeriodicEval": false
},
"setup-browser-cookies": {
"skill": "setup-browser-cookies",
"skillMdBytes": 26618,
"skillMdLines": 594,
"estTokens": 6655,
"tmplBytes": 2724,
"descriptionLen": 222,
"hasGateEval": false,
"hasPeriodicEval": false
},
"setup-deploy": {
"skill": "setup-deploy",
"skillMdBytes": 43927,
"skillMdLines": 919,
"estTokens": 10982,
"tmplBytes": 7780,
"descriptionLen": 197,
"hasGateEval": true,
"hasPeriodicEval": false
},
"setup-gbrain": {
"skill": "setup-gbrain",
"skillMdBytes": 78394,
"skillMdLines": 1704,
"estTokens": 19599,
"tmplBytes": 42245,
"descriptionLen": 323,
"hasGateEval": true,
"hasPeriodicEval": false
},
"ship": {
"skill": "ship",
"skillMdBytes": 166782,
"skillMdLines": 3099,
"estTokens": 41696,
"tmplBytes": 50495,
"descriptionLen": 291,
"hasGateEval": true,
"hasPeriodicEval": true
},
"skillify": {
"skill": "skillify",
"skillMdBytes": 53534,
"skillMdLines": 1168,
"estTokens": 13384,
"tmplBytes": 15107,
"descriptionLen": 233,
"hasGateEval": true,
"hasPeriodicEval": false
},
"spec": {
"skill": "spec",
"skillMdBytes": 102629,
"skillMdLines": 2141,
"estTokens": 25657,
"tmplBytes": 28429,
"descriptionLen": 282,
"hasGateEval": true,
"hasPeriodicEval": false
},
"sync-gbrain": {
"skill": "sync-gbrain",
"skillMdBytes": 50156,
"skillMdLines": 1028,
"estTokens": 12539,
"tmplBytes": 13996,
"descriptionLen": 299,
"hasGateEval": false,
"hasPeriodicEval": false
},
"unfreeze": {
"skill": "unfreeze",
"skillMdBytes": 1504,
"skillMdLines": 49,
"estTokens": 376,
"tmplBytes": 1386,
"descriptionLen": 199,
"hasGateEval": false,
"hasPeriodicEval": false
}
}
}
Reference in New Issue View Git Blame Copy Permalink