CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-09-14 17:05:28 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat(skills): claimed limitations now require evidence, everywhere + wave follow-ups filed

Browse Source 
Every tier-2+ skill's preamble gains one directive distilled from nine live
release failures in two days on the fork: a claimed limitation or
requirement ('the API can't do this', 'X requires a credential',
'impossible on this platform') is a material claim, stated only with the
verbatim error, the documented statement, or a live probe in hand —
pattern-matching a failure to a familiar story is not evidence, and a cheap
probe runs BEFORE asking the user or declaring a step blocked. ONE directive
adapted into the preamble resolver; the fork's full judgment contract is
deliberately not imported. Full regen (46 files), ship goldens refreshed,
parity guards bumped with the measured ~0.45KB/skill (investigate, autoplan,
plan-design-review, office-hours), Step 0.9 registered as an intentional
sub-step.

Approved deferrals filed: persona-fleet hostile-user harness + answer-key
methodology in TODOS; the fork's question-budget ACCOUNTING judgment (never
its 5/8/12 constants) folded into the V1.1 pacing design doc; the Apple
adapter added to #1882's coverage note.

Ported from time-attack/gstack (GStack 2).

Co-authored-by: Sina Matian <sina@time-attack.dev>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-14 13:30:21 -07:00
co-authored by Sina Matian Claude Fable 5
parent 15d1144fea
commit 81a2d48592
55 changed files with 326 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/fixtures/golden/claude-ship-SKILL.md
Vendored
+4
View File
@@ -674,6 +674,10 @@ When options differ in coverage, include `Completeness: X/10` (10 = all edge cas
For high-stakes ambiguity (architecture, data model, destructive scope, missing context), STOP. Name it in one sentence, present 2-3 options with tradeoffs, and ask. Do not use for routine coding or obvious changes.
## Claimed Limitations Need Evidence
A claimed limitation or requirement ("the API can't do this", "X requires a credential", "that's impossible on this platform") is a material claim. State one only with the verbatim error, the documented statement, or a live probe in hand — pattern-matching a failure to a familiar story is not evidence. When a cheap probe settles the question, run it BEFORE asking the user anything or declaring a step blocked.
## Continuous Checkpoint Mode
If `CHECKPOINT_MODE` is `"continuous"`: auto-commit completed logical units with `WIP:` prefix.
test/fixtures/golden/codex-ship-SKILL.md
Vendored
+17 -6
View File
@@ -660,6 +660,10 @@ When options differ in coverage, include `Completeness: X/10` (10 = all edge cas
For high-stakes ambiguity (architecture, data model, destructive scope, missing context), STOP. Name it in one sentence, present 2-3 options with tradeoffs, and ask. Do not use for routine coding or obvious changes.
## Claimed Limitations Need Evidence
A claimed limitation or requirement ("the API can't do this", "X requires a credential", "that's impossible on this platform") is a material claim. State one only with the verbatim error, the documented statement, or a live probe in hand — pattern-matching a failure to a familiar story is not evidence. When a cheap probe settles the question, run it BEFORE asking the user anything or declaring a step blocked.
## Continuous Checkpoint Mode
If `CHECKPOINT_MODE` is `"continuous"`: auto-commit completed logical units with `WIP:` prefix.
@@ -1872,16 +1876,23 @@ Using the plan file already discovered in Step 8, look for a verification sectio
### 2. Check for running dev server
Before invoking browse-based verification, check if a dev server is reachable:
Before invoking browse-based verification, find the dev-server URL the way the
project declares it — never trust a hardcoded port list alone:
1. **CLAUDE.md first:** look for a documented dev URL or dev command (a
`## Development`/`## Testing` section naming a port or URL). Use it.
2. **The plan file:** if the plan's verification section names a URL, use it.
3. **Fallback probe** (common ports, only when 1-2 found nothing):
```bash
curl -s -o /dev/null -w '%{http_code}' http://localhost:3000 2>/dev/null || \
curl -s -o /dev/null -w '%{http_code}' http://localhost:8080 2>/dev/null || \
curl -s -o /dev/null -w '%{http_code}' http://localhost:5173 2>/dev/null || \
curl -s -o /dev/null -w '%{http_code}' http://localhost:4000 2>/dev/null || echo "NO_SERVER"
for _p in 3000 8080 5173 4000 4321 8000; do
_code=$(curl -s -o /dev/null -w '%{http_code}' "http://localhost:$_p" 2>/dev/null)
[ -n "$_code" ] && [ "$_code" != "000" ] && { echo "DEV_SERVER: http://localhost:$_p ($_code)"; break; }
done
[ -z "${_code:-}" ] || [ "${_code:-000}" = "000" ] && echo "NO_SERVER"
```
**If NO_SERVER:** Skip with "No dev server detected — skipping plan verification. Run /qa separately after deploying."
**If NO_SERVER:** Skip with "No dev server detected (checked CLAUDE.md, the plan, and common ports) — skipping plan verification. Run /qa separately after deploying, or document the dev URL in CLAUDE.md so this step finds it next time."
### 3. Invoke /qa-only inline
test/fixtures/golden/factory-ship-SKILL.md
Vendored
+17 -6
View File
@@ -662,6 +662,10 @@ When options differ in coverage, include `Completeness: X/10` (10 = all edge cas
For high-stakes ambiguity (architecture, data model, destructive scope, missing context), STOP. Name it in one sentence, present 2-3 options with tradeoffs, and ask. Do not use for routine coding or obvious changes.
## Claimed Limitations Need Evidence
A claimed limitation or requirement ("the API can't do this", "X requires a credential", "that's impossible on this platform") is a material claim. State one only with the verbatim error, the documented statement, or a live probe in hand — pattern-matching a failure to a familiar story is not evidence. When a cheap probe settles the question, run it BEFORE asking the user anything or declaring a step blocked.
## Continuous Checkpoint Mode
If `CHECKPOINT_MODE` is `"continuous"`: auto-commit completed logical units with `WIP:` prefix.
@@ -1874,16 +1878,23 @@ Using the plan file already discovered in Step 8, look for a verification sectio
### 2. Check for running dev server
Before invoking browse-based verification, check if a dev server is reachable:
Before invoking browse-based verification, find the dev-server URL the way the
project declares it — never trust a hardcoded port list alone:
1. **CLAUDE.md first:** look for a documented dev URL or dev command (a
`## Development`/`## Testing` section naming a port or URL). Use it.
2. **The plan file:** if the plan's verification section names a URL, use it.
3. **Fallback probe** (common ports, only when 1-2 found nothing):
```bash
curl -s -o /dev/null -w '%{http_code}' http://localhost:3000 2>/dev/null || \
curl -s -o /dev/null -w '%{http_code}' http://localhost:8080 2>/dev/null || \
curl -s -o /dev/null -w '%{http_code}' http://localhost:5173 2>/dev/null || \
curl -s -o /dev/null -w '%{http_code}' http://localhost:4000 2>/dev/null || echo "NO_SERVER"
for _p in 3000 8080 5173 4000 4321 8000; do
_code=$(curl -s -o /dev/null -w '%{http_code}' "http://localhost:$_p" 2>/dev/null)
[ -n "$_code" ] && [ "$_code" != "000" ] && { echo "DEV_SERVER: http://localhost:$_p ($_code)"; break; }
done
[ -z "${_code:-}" ] || [ "${_code:-000}" = "000" ] && echo "NO_SERVER"
```
**If NO_SERVER:** Skip with "No dev server detected — skipping plan verification. Run /qa separately after deploying."
**If NO_SERVER:** Skip with "No dev server detected (checked CLAUDE.md, the plan, and common ports) — skipping plan verification. Run /qa separately after deploying, or document the dev URL in CLAUDE.md so this step finds it next time."
### 3. Invoke /qa-only inline
test/gen-skill-docs.test.ts
+5 -2
View File
@@ -1050,8 +1050,11 @@ describe('PLAN_VERIFICATION_EXEC placeholder', () => {
expect(shipSkill).toContain('qa-only');
});
test('contains localhost reachability check', () => {
expect(shipSkill).toContain('localhost:3000');
test('contains dev-server discovery (CLAUDE.md first, then a port probe)', () => {
// Fork port wave 2: the hardcoded 4-port list became read-CLAUDE.md-or-
// probe; the probe loops common ports instead of naming each once.
expect(shipSkill).toContain('CLAUDE.md first');
expect(shipSkill).toContain('http://localhost:$_p');
expect(shipSkill).toContain('NO_SERVER');
});
test/helpers/carve-guards.ts
+7 -5
View File
@@ -208,10 +208,12 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
// always-loaded AskUserQuestion Format section.
// v1.2.0 activation lift (shared first-run-guidance preamble) + #2077 ask-first scope gate.
// +~1.3 KB: plan-mode auto-select-B scope-gate exceptions (2026-08).
maxSkeletonBytes: 89_000,
// Fork port wave 2 (D1): evidence directive adds ~0.45KB to every
// tier-2+ skeleton. Measured 89,184.
maxSkeletonBytes: 91_000,
minUnionBytes: 70_000,
mustContain: ['design', 'visual'],
maxSizeRatio: 1.07,
maxSizeRatio: 1.12, // D1 measured 1.104
},
'plan-devex-review': {
skill: 'plan-devex-review',
@@ -255,12 +257,12 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
// plus the P1 office-hours closing handoff (AUQ that launches the next skill).
// Fork port wave 2: the third-party web-actions contract sits inline
// (judgment must be visible before the workflow directs the user to a
// vendor site), plus the #703 dual-write + repo-doc-preference block —
// ratio 1.092 measured. Tight headroom.
// vendor site), plus the #703 dual-write + repo-doc-preference block and
// the #538 opt-out + D1 evidence directive — ratio 1.104 measured.
maxSkeletonBytes: 101_000,
minUnionBytes: 70_000,
mustContain: ['design doc', 'problem statement'],
maxSizeRatio: 1.10,
maxSizeRatio: 1.12,
},
'document-release': {
skill: 'document-release',
test/helpers/parity-harness.ts
+6 -2
View File
@@ -240,7 +240,9 @@ const MONOLITH_INVARIANTS: ParityInvariant[] = [
// 1.09 → 1.10: the plan-mode preamble reword (scope-gate auto-select-B
// change) adds ~250 B to every skill's shared preamble; investigate was
// the closest to its ceiling (landed 1.092).
maxSizeRatio: 1.10,
// Fork port wave 2 (D1): the evidence-before-claimed-limitations preamble
// directive adds ~0.45KB to every tier-2+ skill. Measured values noted.
maxSizeRatio: 1.12, // D1 measured
minBytes: 30_000,
},
{
@@ -248,7 +250,9 @@ const MONOLITH_INVARIANTS: ParityInvariant[] = [
mustContain: ['ceo', 'eng', 'design'],
mustHaveHeadings: ['## Preamble', '## When to invoke'],
// v1.2.0 activation lift: shared first-run-guidance preamble section.
maxSizeRatio: 1.07,
// Fork port wave 2 (D1): the evidence-before-claimed-limitations preamble
// directive adds ~0.45KB to every tier-2+ skill. Measured values noted.
maxSizeRatio: 1.09, // D1 measured
minBytes: 70_000,
},
];
test/skill-validation.test.ts
+5 -3
View File
@@ -1216,9 +1216,11 @@ describe('Step 3.4 test coverage audit', () => {
describe('ship step numbering', () => {
// Allowed sub-steps that are resolver-generated and intentionally nested:
// 8.1 (Plan Verification), 8.2 (Scope Drift), 9.1 (Review Army), 9.2 (Findings Merge),
// 9.3 (Cross-review dedup), 15.0 (WIP squash — continuous checkpoint), 15.1 (Bisectable commits).
const ALLOWED_SUBSTEPS = new Set(['8.1', '8.2', '9.1', '9.2', '9.3', '15.0', '15.1']);
// 0.9 (Apple target detection — MUST precede Step 1's branch gate, R2-pinned
// by test/ship-apple-gate.test.ts), 8.1 (Plan Verification), 8.2 (Scope
// Drift), 9.1 (Review Army), 9.2 (Findings Merge), 9.3 (Cross-review dedup),
// 15.0 (WIP squash — continuous checkpoint), 15.1 (Bisectable commits).
const ALLOWED_SUBSTEPS = new Set(['0.9', '8.1', '8.2', '9.1', '9.2', '9.3', '15.0', '15.1']);
test('ship/SKILL.md.tmpl contains no unexpected fractional step numbers', () => {
const tmpl = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md.tmpl'), 'utf-8');