CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-09-09 06:28:59 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix: red-team review fixes

Browse Source 
Five informational findings: TODOS shard-census arithmetic corrected (census
is 67 with one free ungated slot; the SECOND ungated file trips the floor)
and version pointer fixed (v0.18.2.0, not v0.18.1.0); the free tripwire now
pins the two dispatch-matcher marker strings so a pr-body prompt reword
fails the free suite instead of surfacing as a paid-tier mystery; the E2E
matcher gains a section-paste exclusion (scaffold strings disqualify) —
verified against all recorded runs; the E2E header documents the tierless
test:evals invisibility tradeoff.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-26 15:53:08 +00:00
co-authored by Claude Fable 5
parent 14ee421fb1
commit 595f6e60bd
3 changed files with 35 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/ship-document-release-dispatch.test.ts
+6
View File
@@ -59,6 +59,12 @@ describe('/ship Step 18 dispatches /document-release (carve visibility)', () =>
expect(content).toContain('"documentation_section"');
// Deliberate design: docs sync never holds a ship hostage.
expect(content).toContain('Do not block /ship on subagent failure');
// These two strings are the ship-docsync E2E's dispatch-matcher markers
// (test/skill-e2e-ship-docsync.test.ts) — they come verbatim from the
// dictated Step 18 subagent prompt. Rewording them in pr-body.md.tmpl
// silently decouples the paid matcher; update both in lockstep.
expect(content).toContain('You are executing the /document-release workflow');
expect(content).toContain('.claude/skills/gstack/document-release/SKILL.md');
}
});
test/skill-e2e-ship-docsync.test.ts
+18 -9
View File
@@ -8,8 +8,11 @@
*
* Gating: whole-file gate-tier self-gate (describeE2ETier) COMPOSED with
* diff-based selection (describeIfSelected). The self-gate keeps this file
* out of the periodic shard census (which sits at its ceiling) and under
* the hard tier-alignment invariant. Run locally with:
* out of the periodic shard census (near its ceiling) and under the hard
* tier-alignment invariant. DELIBERATE TRADEOFF: tierless runs (`bun run
* test:evals` / `test:e2e`) skip every tier-gated file, so this test does
* NOT run there even when ship/** changed — use the gate lane locally:
* EVALS_TIER=gate bun run test:evals # diff-selected gate lane
* EVALS=1 EVALS_TIER=gate EVALS_ALL=1 bun test test/skill-e2e-ship-docsync.test.ts
*
* Fixture layout (non-obvious — fake HOME + planted skill tree):
@@ -208,13 +211,19 @@ describeE2E('Ship doc-sync dispatch E2E (gate)', () => {
// (e.g. a PR-body drafter) must NOT count — that false-pass would mask
// the exact regression this test exists to catch. Verified against
// recorded burn-in transcripts: real dispatch inputs carry both markers.
const dispatchIdx = calls.findIndex(
(tc) =>
(tc.tool === 'Agent' || tc.tool === 'Task') &&
/document-release\/SKILL\.md|executing the \/document-release workflow/i.test(
JSON.stringify(tc.input ?? {})
)
);
// Section-paste exclusion: a subagent handed the WHOLE pr-body.md as
// context carries the markers too. The dictated Step 18 prompt never
// contains the section's scaffolding, so its presence disqualifies.
// Verified across all recorded runs: real dispatches match markers,
// zero contain scaffold strings.
const dispatchIdx = calls.findIndex((tc) => {
if (tc.tool !== 'Agent' && tc.tool !== 'Task') return false;
const input = JSON.stringify(tc.input ?? {});
return (
/document-release\/SKILL\.md|executing the \/document-release workflow/i.test(input) &&
!/## Step 19: Create PR\/MR|Parent processing:/.test(input)
);
});
const prCreateIdx = calls.findIndex(
(tc) =>
tc.tool === 'Bash' &&