CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-09-16 01:45:29 +02:00
Code Issues Packages Projects Releases Wiki Activity

test: touchfiles, tiers, shim E2E, real-engine fixture

Browse Source 
Touchfiles: the catalog, contract, detector bin, checklist resolver, review-
army resolver, and DESIGN.md lib join the dep lists of review-design-lite,
design-review-fix, the design-consultation cases, and plan-design-review-no-
ui-scope, so editing any of them re-selects the tests that read their output.
Three new E2E keys: design-review-detector-shim (gate; source mode on a
feature-branch diff), design-review-detector-shim-dom (gate; DOM mode: the
slop fixture served on loopback, dumped through the browse binary with
lib/dom-dump.js, persisted under a GSTACK_HOME-scoped REPORT_DIR, scanned once;
self-skips when browse/dist/browse is absent), and design-html-slop-gate
(periodic; one fix pass, at most two scans, remaining findings accepted with
reason). Every case reaches the engine through test/fixtures/fake-impeccable.ts
via IMPECCABLE_BIN from outside the temp repo, reads extracted skill sections
(never a whole SKILL.md) with the installed bin path rewritten to this
checkout, and asserts the probe ran, the right scan verb ran, `npx impeccable`
never did, and the output carries FINDING rows tagged [ai-color-palette] and
[low-contrast]. review-design-lite gets the fake engine and an eighth tally
signal for a detector row; its 4-hit threshold is unchanged.

test/gstack-design-detect.test.ts evaluates design-review's REPORT_DIR
expression with GSTACK_HOME set and proves a dump under it is accepted by the
wrapper's allow-list. The sample fixtures were real captures from commit 1
(engine 0.1.3), so there is nothing hand-written left to swap.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-09-08 16:38:46 +00:00
co-authored by Claude Fable 5.1
parent f1ce46eb8b
commit 9a03aa5e4a
6 changed files with 289 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/skill-e2e-review.test.ts
+19 -2
View File
@@ -172,6 +172,7 @@ The diff adds a new "returned" status to the Order model. Your job is to check i
describeIfSelected('Review design lite E2E', ['review-design-lite'], () => {
let designDir: string;
let fakeEngineDir: string;
beforeAll(() => {
designDir = fs.mkdtempSync(path.join(os.tmpdir(), 'skill-e2e-design-lite-'));
@@ -206,12 +207,22 @@ describeIfSelected('Review design lite E2E', ['review-design-lite'], () => {
extractSkillSections(path.join(ROOT, 'review'), REVIEW_E2E_SECTIONS),
);
fs.copyFileSync(path.join(ROOT, 'review', 'checklist.md'), path.join(designDir, 'review-checklist.md'));
fs.copyFileSync(path.join(ROOT, 'review', 'design-checklist.md'), path.join(designDir, 'review-design-checklist.md'));
// The checklist's mechanical pass (step 0) runs the design detector from the
// installed gstack bin; point it at THIS checkout so the test is hermetic.
fs.writeFileSync(
path.join(designDir, 'review-design-checklist.md'),
fs.readFileSync(path.join(ROOT, 'review', 'design-checklist.md'), 'utf-8').replaceAll('~/.claude/skills/gstack/bin', path.join(ROOT, 'bin')),
);
fs.copyFileSync(path.join(ROOT, 'review', 'greptile-triage.md'), path.join(designDir, 'review-greptile-triage.md'));
// Fake impeccable engine OUTSIDE the repo (the wrapper ignores an in-repo IMPECCABLE_BIN).
fakeEngineDir = fs.mkdtempSync(path.join(os.tmpdir(), 'skill-e2e-fake-impeccable-'));
fs.copyFileSync(path.join(ROOT, 'test', 'fixtures', 'fake-impeccable.ts'), path.join(fakeEngineDir, 'impeccable'));
fs.chmodSync(path.join(fakeEngineDir, 'impeccable'), 0o755);
});
afterAll(() => {
try { fs.rmSync(designDir, { recursive: true, force: true }); } catch {}
try { fs.rmSync(fakeEngineDir, { recursive: true, force: true }); } catch {}
});
testConcurrentIfSelected('review-design-lite', async () => {
@@ -233,6 +244,10 @@ Important: The design checklist should catch issues like blacklisted fonts, smal
timeout: CAPTURE_MS,
testName: 'review-design-lite',
runId,
env: {
IMPECCABLE_BIN: path.join(fakeEngineDir, 'impeccable'),
FAKE_IMPECCABLE_OUTPUT: path.join(ROOT, 'test', 'fixtures', 'impeccable-detect-sample.json'),
},
});
logCost('/review design lite', result);
@@ -259,8 +274,10 @@ Important: The design checklist should catch issues like blacklisted fonts, smal
if (review.includes('welcome to') || review.includes('all-in-one') || review.includes('generic') || review.includes('hero copy') || review.includes('ai slop')) detected++;
// Issue 7: 3-column feature grid — LOW
if (review.includes('3-column') || review.includes('three-column') || review.includes('feature grid') || review.includes('icon') || review.includes('circle')) detected++;
// Signal 8: the mechanical pass (fake impeccable engine via IMPECCABLE_BIN) surfaced a detector row
if (review.includes('detector') || review.includes('[ai-color-palette]') || review.includes('[low-contrast]') || review.includes('impeccable')) detected++;
console.log(`Design review detected ${detected}/7 planted issues`);
console.log(`Design review detected ${detected}/8 planted signals`);
expect(detected).toBeGreaterThanOrEqual(4);
}
}, CAPTURE_MS);