mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-13 08:29:04 +02:00
v1.81.0.0 feat: Aside is the browser gstack drives first; every browsing skill, the PDF/diagram renderer, and web research; the bundled browser stays the automatic fallback (#2810)
* feat(aside): browser-driver contract, cookbook, research and fallback resolvers
{{ASIDE_SETUP}} (readiness probe + ten rules for driving the user's real browser), {{ASIDE_COOKBOOK}} (script shapes verified live against Aside CLI 1.26: one flow per aside repl script, CDP console hook before navigation, evidence lines, session-directory artifact handoff, GSTACK_STEP_OK sentinel), {{ASIDE_RESEARCH}} (research through aside exec, WebSearch when Aside is absent, knowledge otherwise) and {{BROWSE_FALLBACK}} (the fifteen-row Aside-step to $B-command table plus the rules that differ, so every browsing skill keeps working on gstack's own headless browser). test/aside-driver.test.ts pins the sentences and asserts every browsing skill carries the Aside block followed by the fallback; test/helpers/aside-available.ts is the shared live-Aside probe.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(render): Aside-first local-HTML renderer with the bundled browser as fallback
lib/aside-render.ts serves the HTML's directory on loopback (Aside refuses file:// URLs), opens it with waitUntil load, prints through CDP Page.printToPDF so tagged output, outlines, header/footer templates and page numbers survive, emulates device metrics for sized screenshots, and writes in-page evaluations to files; when Aside is absent it runs the same spec through the browse daemon (newtab, load, js, pdf, screenshot, closetab) and reports ENGINE=aside|browse. bin/gstack-render.ts is the CLI skill templates call. lib/claude-bin.ts and lib/error-handling.ts become the canonical copies (browse/src re-exports them).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(browse): /browse drives Aside first, with the $B reference behind the fallback
Contract, cookbook, mode choice (aside repl by default, aside exec for reading), report format, the fallback section, and the full command reference carved on demand.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(qa): /qa and /qa-only drive Aside, fall back to $B
QA_METHODOLOGY runs every phase as Aside scripts (orient, explore, document, re-test, mobile viewport via CDP emulation, links via HEAD fetch); the authenticate phase is 'you are already signed in'; a 13th rule requires consent before mutating actions on non-local targets; the fallback section translates each step onto $B. The qa E2E tests run on whichever engine is present.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(design): design-review, design-consultation, design-shotgun, plan-design-review, design-html drive Aside
Design-system extraction is one script printing FONTS/COLORS/HEADINGS/TOUCH_TARGETS/NAV; competitor research confirms the exact URLs before opening them in the real browser and runs on the bundled browser when Aside is absent; design-html's viewport screenshots, sketches and comparison boards render through gstack-render.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(deploy): benchmark, canary, land-and-deploy Step 7, devex-review drive Aside
One aside repl script per page prints NAV/PAINT/LCP/RESOURCES/SCRIPTS/CSS/SUMMARY (benchmark), CONSOLE_ERRORS/NAV/TEXT + screenshot (canary, re-run every 60s), and the post-deploy check reads responseStatus from the navigation entry; each carries the $B fallback.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(third-party-actions): Aside is the recommended driver; gstack's visible browser stays the fallback
The readiness probe is lifted from {{ASIDE_SETUP}} at gen time (byte-identity pinned) and rule 3 points at browse/SKILL.md for how to drive; the consent question offers Aside first and gstack's own visible browser (handoff/resume for sign-in) as the fallback, as v1.72 framed it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(scrape): /scrape reads pages through Aside; the browser-skills runtime rides the fallback
Look-then-extract scripts build the JSON inside the page and print it between JSON_START/JSON_END; aside exec for fuzzy intents; on the $B fallback the browser-skills match/prototype flow and /skillify apply as before.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(make-pdf): print through Aside first, the bundled browser otherwise
asideClient.ts replaces the direct $B client with one render() call per PDF (the exact option mapping the browse pdf command had: paper, margins, header/footer/page numbers, tagged, outline, printBackground, preferCSSPageSize, Paged.js wait); the diagram pre-pass, oversized-image downscale and DOCX rasters each run as one render script with per-fence try/catch; exit 4 now means no browser is available and names both remedies; $P setup reports which engine it found. The e2e gates run on whichever engine is present, so the Linux lane exercises the fallback.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(diagram): the triplet is one gstack-render call
SVG, PNG and excalidraw from one invocation over the content-addressed bundle staged under /tmp/gstack-render; every diagram type gets an excalidraw export; gstack-render picks the engine and prints ENGINE=; the diagram E2E gates on either engine.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(research): web research runs in Aside first, WebSearch second
The planning, review, design, security and investigate skills research through {{ASIDE_RESEARCH}}; WebSearch stays in allowed-tools as the fallback; testing.ts's bootstrap step follows; skeleton ceilings ratcheted for the research block.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(setup,gen-skill-docs): prune renders of skills that no longer exist
setup gains _prune_stale_generated for every host tree and the doc generator removes gstack-* output dirs it did not write, so a skill removed from the source tree can never linger in an install.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test: registries, budgets and suite reconciled for Aside-first with the $B fallback
Touchfiles + E2E tiers gain the Aside keys, coverage matrix and eval baselines updated, size budget re-baselined to parity-baseline-v1.80.0.0.json (the contract plus fallback ride in every browsing skill), parity ceilings ratcheted with measured values, LLM-judge prompts and the E2E fixtures speak Aside-first, browse-fallback.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: Aside first, gstack browser fallback
README, BROWSER.md, docs/, CONTRIBUTING, CLAUDE.md, ARCHITECTURE, AGENTS.md, TODOS and the root router describe the one product story: Aside is the browser gstack drives first; the bundled headless browser is the automatic fallback (Linux, Windows, app closed) where cookie import, GStack Browser, pair-agent and browser-skills still apply.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore: regenerate SKILL.md docs, llms.txt, agents digest, ship goldens, context-budget fixture
bun run gen:skill-docs over the templates; goldens re-rendered; context-budget ceilings recaptured.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* v1.80.0.0: Aside is the browser gstack drives first; the bundled browser is the fallback
MINOR: new capability across ten skills, the renderer and research; nothing removed. CHANGELOG release summary + itemized changes; VERSION 1.80.0.0; package.json 1.80.0.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(todos): file non-Claude host ownership-gate and version-heading pin follow-ups
Two follow-ups from the /plan-ceo-review + /plan-eng-review pass on merging
PR #2804 with main's v1.80.0.0 ownership gate: bring the Codex/Factory/
OpenCode/Cursor/Kiro copy loops and the stale-render prune under the
.gstack-owned marker rule, and a free test pinning that the CHANGELOG top
heading equals VERSION (the collision that git cannot see).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix: pre-landing review fixes for the Aside-first branch
Review army + adversarial passes (Claude and Codex) on the merged branch:
setup
- _prune_stale_generated scans the host dirs too (the generator already
removed the render before setup ran, so the host branch was dead), skips
symlinks in the render tree (rm -rf on a slash-terminated link empties its
target), removes a host symlink only when it resolves into gstack, cleans a
bannered real dir through _cleanup_weak_dir, recognizes frontmatter-renamed
skills, and logs through log. The always-run codex render passes every host
dir that may link to it.
- NEEDS_BUILD checks all three binaries (with $_EXE) and lib/ sources; the
browser hint and the bootstrap summary honor GSTACK_SKIP_ASIDE, treat a
requested skip as a request, and derive one skill list.
lib/aside-render.ts + bin/gstack-render.ts
- The loopback server carries a per-render secret path, checks containment on
the real path (symlink escapes are 403), and rejects malformed encoding.
- Inline eval results are one base64 line, so page text cannot forge
ASIDE_DIR= or the sentinel; the last ASIDE_DIR wins.
- runProc escalates SIGTERM to SIGKILL, bounds every wait, and clears every
timer (an uncleared one kept gstack-render alive after printing OK).
- renderTmpDir refuses a shared /tmp name owned by someone else; the work dir
and server are created inside try; goto's budget follows the render budget.
- probeAside classifies a present-but-failing CLI as ASIDE_NOT_RUNNING like
the skills' bash probe; render() retries on gstack's own browser when Aside
could not start or its private CDP bridge is gone (never on a page error
or a timeout of a running script); the CLI reports the engine that actually
rendered, exits 0 on --help, rejects non-numeric flags, documents
--wait-timeout, fences EVAL/PAGE_ERRORS as untrusted content, and names the
daemon's cookie-import JS lock remedy.
- The browse path passes --scale only when asked (a scale change rebuilds
the daemon context) and restores the viewport after a sized screenshot.
resolvers / templates
- The bash probe honors GSTACK_SKIP_ASIDE and has a perl deadline on stock
macOS; .local is no longer LOCAL (mDNS); same-origin filters compare parsed
origins; link status is HEAD-checked only on LOCAL targets; every
aside exec goes through the receipted _aside_exec prelude
({{ASIDE_EXEC_PRELUDE}}), including nine template blocks that called it
bare; the design sketch and diagram staging use private directories.
- The generator prunes only bannered renders and never a host whose
generation failed.
Docs, stale comments and dead code cleaned; goldens re-rendered; tests
updated and added for every behavior above.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test: coverage for the render CLI, setup rebuild check, make-pdf exit codes, and prose $B spans
New free tests from the ship coverage audit: test/gstack-render-cli.test.ts
(argv guards, --help, output contract with a fake daemon, failure and
serve-root paths, no-browser case, prompt exit), test/setup-needs-build.test.ts
(every binary and source set flips NEEDS_BUILD, Windows suffixes),
make-pdf/test/cli-exit-codes.test.ts and setup-smoke.test.ts (error to exit
code mapping, runSetup stages, renderPdf's engine), and prose-span cases for
extractBrowseCommands in test/skill-parser.test.ts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: CHANGELOG and TODOS cover the review fixes (v1.81.0.0)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: sync project docs with the v1.81.0.0 review fixes
BROWSER.md, ARCHITECTURE.md, CONTRIBUTING.md, README.md, CLAUDE.md,
docs/TESTING_INTERNALS.md and docs/PROJECT_STRUCTURE.md now describe the
shipped renderer and setup: the loopback render server's per-render secret
path and real-path containment, ENGINE= naming the engine that actually
rendered (mid-run retry on gstack's own browser), EVAL/PAGE_ERRORS fenced as
untrusted content, --wait-timeout and the CLI's argv guards, the receipted
_aside_exec prelude ({{ASIDE_EXEC_PRELUDE}} in the placeholder table), the
LOCAL host rule without .local, LOCAL-only HEAD checks in the links script,
GSTACK_SKIP_ASIDE across probe/renderer/setup, the ownership-gated
retired-skill prune, the widened NEEDS_BUILD check, and the new free tests
(gstack-render-cli, setup-prune-stale-generated, setup-browser-hint,
setup-needs-build, make-pdf cli-exit-codes and setup-smoke).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: CHANGELOG states the precise mid-run retry rule
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(test): skill-e2e-bws slices the $B setup block from the Browser fallback section
browse/SKILL.md no longer has '## SETUP' / '## Core QA Patterns' (Aside is the
primary driver; the $B block moved under 'Browser fallback'), so the gate test
sliced an empty block and handed the agent nothing to run. Anchor on
'### Find the `$B` binary' up to the next heading. 7/7 pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(test): gate POSIX-only fixtures off Windows
windows-free-tests: the gstack-render CLI tests drive a shebang fake browse
that CreateProcess cannot exec, and two NEEDS_BUILD cases assert an execute
bit and a bare-name miss that MSYS bash does not have (test -x ignores mode
bits and resolves design -> design.exe). Those describes and cases now
self-skip on win32; argument guards, --help, the no-browser case, and every
other rebuild-check case still run there.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(render): runProc waits for the exit code until the kill deadline; newtab retries once on a cold daemon
A process whose pipes have reached EOF is exiting, but runProc gave the exit
code only five seconds to arrive and then returned null, which run() reports
as a failed command. Under CI's six-shard load one such render failed with the
artifact already written. The SIGTERM/SIGKILL timers already bound the wait,
so the exit race now runs to the kill deadline.
The first CLI call auto-starts the browse daemon; on a cold start it can
answer 'Unable to connect' once while the server is still coming up. That
single case is retried after 1.5s; every other newtab failure is not.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(aside-render): warm the daemon before live fallback cases; failures name the render error
- Live fallback cases run 'goto about:blank' up to twice before asserting and
skip (never fail) when the daemon cannot come up.
- expectOk() puts r.error and the browse transcript into the assertion so a
failed render is diagnosable from the CI log.
- The argv-contract cases dump the fake's log on a miss.
- File default timeout is 30s: the subject is the CLI contract, not latency.
- Two cases pin the cold-daemon newtab retry and that other errors are not
retried.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: CHANGELOG notes the cold-start tolerance of the bundled-browser renderer
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Sina <sdroid674+github@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Fable 5.1
Sina
Claude Fable 5.1
Sina
parent
c241216637
commit
0530392821
@@ -35,7 +35,7 @@ import {
|
||||
} from '@anthropic-ai/claude-agent-sdk';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import { resolveClaudeBinary as resolveClaudeBinaryShared } from '../../browse/src/claude-bin';
|
||||
import { resolveClaudeBinary as resolveClaudeBinaryShared } from '../../lib/claude-bin';
|
||||
import { hermeticChildEnv } from './hermetic-env';
|
||||
import type { SkillTestResult } from './session-runner';
|
||||
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
/**
|
||||
* Runtime probe for the Aside AI browser — the primary browser; gstack's own
|
||||
* headless browser is the fallback. E2E tests that need a live Aside call
|
||||
* `asideAvailable()` and self-skip when it is false (CI runners have no
|
||||
* Aside; the fallback path is exercised there instead). The probe mirrors the
|
||||
* bash one the skills run in BROWSER SETUP (scripts/resolvers/aside.ts) via
|
||||
* lib/aside-render.ts probeAside(); the two classify the same way.
|
||||
*/
|
||||
import { probeAside } from '../../lib/aside-render';
|
||||
|
||||
let cached: boolean | null = null;
|
||||
|
||||
export function asideAvailable(): boolean {
|
||||
if (cached !== null) return cached;
|
||||
if (process.env.GSTACK_SKIP_ASIDE === '1') return (cached = false);
|
||||
cached = probeAside().ok;
|
||||
return cached;
|
||||
}
|
||||
@@ -150,7 +150,7 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
},
|
||||
behavioral: 'external',
|
||||
externalTest: 'test/skill-e2e-ship-section-loading.test.ts',
|
||||
maxSkeletonBytes: 77_650, // + v1.78 AUQ spawned-trigger objectivity (explicit declaration + interactive fence); measured 77_236
|
||||
maxSkeletonBytes: 79_300, // + v2.0 {{ASIDE_RESEARCH}} (Aside first, WebSearch fallback); measured 78_275
|
||||
minUnionBytes: 181_000, // token-reduction Phases 1-2 (v1.69.x branch); measured union 201,464
|
||||
mustContain: ['VERSION', 'CHANGELOG', 'review', 'merge', 'PR'],
|
||||
// v1.58.5.0: pre-push-guard install (#2077) stacks on the shared first-run-guidance preamble.
|
||||
@@ -181,7 +181,7 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
// v1.65 merge: provisional larger-of-both-waves budget; re-measured below.
|
||||
// Fork port wave 2 (#703): the repo-doc-preference block in the design
|
||||
// check grew every plan-review skeleton ~0.7KB. Measured values noted.
|
||||
maxSkeletonBytes: 76_000, // + v1.78 AUQ objectivity + v1.79 foreground-dispatch sweep (merged); measured 75_586
|
||||
maxSkeletonBytes: 79_000, // + v2.0 {{ASIDE_RESEARCH}} (Aside first, WebSearch fallback); measured 77_657
|
||||
minUnionBytes: 123_600, // token-reduction Phases 1-2 (v1.69.x branch): preamble bash -> bin/gstack-skill-start, onboarding -> gated emission; measured union 137,346
|
||||
mustContain: ['SCOPE EXPANSION', 'SELECTIVE EXPANSION', 'HOLD SCOPE', 'SCOPE REDUCTION'],
|
||||
// Default-on Codex outside-voice (codexPreflight block + CODEX_MODE branch
|
||||
@@ -207,7 +207,7 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
// check grew every plan-review skeleton ~0.7KB. Measured values noted.
|
||||
// #2499 project-scope MCP jq in the brain-sync block grew every tier-2+
|
||||
// skeleton ~1.5KB (entry resolution emitted once per SKILL.md).
|
||||
maxSkeletonBytes: 54_200, // + v1.78 AUQ spawned-trigger objectivity (explicit declaration + interactive fence); measured 53_773
|
||||
maxSkeletonBytes: 56_500, // + v2.0 {{ASIDE_RESEARCH}} (Aside first, WebSearch fallback); measured 55_457
|
||||
minUnionBytes: 99_800, // token-reduction Phases 1-2 (v1.69.x branch); measured union 110,910
|
||||
mustContain: ['Architecture', 'Code Quality', 'Test', 'Performance'],
|
||||
// Cross-cutting preamble growth (v1.57.2.0 AUQ-failure prose fallback + the
|
||||
@@ -264,7 +264,7 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
// check grew every plan-review skeleton ~0.7KB. Measured values noted.
|
||||
// #2499 project-scope MCP jq in the brain-sync block grew every tier-2+
|
||||
// skeleton ~1.5KB (entry resolution emitted once per SKILL.md).
|
||||
maxSkeletonBytes: 65_900, // + v1.78 AUQ spawned-trigger objectivity (explicit declaration + interactive fence); measured 65_486
|
||||
maxSkeletonBytes: 68_500, // + v2.0 {{ASIDE_RESEARCH}} (Aside first, WebSearch fallback); measured 67_129
|
||||
minUnionBytes: 99_700, // token-reduction Phases 1-2 (v1.69.x branch); measured union 110,833
|
||||
mustContain: ['developer experience', 'Getting Started'],
|
||||
// Default-on Codex outside-voice (codexPreflight block + CODEX_MODE branch
|
||||
@@ -295,7 +295,7 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
// the #538 opt-out + D1 evidence directive — ratio 1.104 measured.
|
||||
// #2499 project-scope MCP jq in the brain-sync block grew every tier-2+
|
||||
// skeleton ~1.5KB (entry resolution emitted once per SKILL.md).
|
||||
maxSkeletonBytes: 73_450, // + v1.78 AUQ objectivity + v1.79 foreground-dispatch sweep (merged); measured 73_040
|
||||
maxSkeletonBytes: 76_800, // + v2.0 {{ASIDE_RESEARCH}} (Aside first, WebSearch fallback); measured 75_804
|
||||
minUnionBytes: 115_800, // Phase 4 wave 4; measured union 118,175
|
||||
mustContain: ['design doc', 'problem statement'],
|
||||
maxSizeRatio: 1.12,
|
||||
@@ -347,7 +347,7 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
// v1.65 merge: provisional larger-of-both-waves budget; re-measured below.
|
||||
// v1.64.1.0: shared-preamble prose from the two parallel v1.64 waves lands
|
||||
// the skeleton at 69,022 B; +~1 KB headroom.
|
||||
maxSkeletonBytes: 53_750, // + v1.78 AUQ objectivity + v1.79 foreground-dispatch sweep (merged); measured 53_342
|
||||
maxSkeletonBytes: 66_500, // + v2.0 {{ASIDE_SETUP}}/{{BROWSE_FALLBACK}} for the research phase; measured 65_506
|
||||
minUnionBytes: 65_000, // token-reduction Phases 1-2 (v1.69.x branch): preamble bash -> bin/gstack-skill-start, onboarding -> gated emission; measured union 72,252
|
||||
mustContain: ['Typography', 'Color', 'Aesthetic Direction'],
|
||||
// Cross-cutting preamble growth (v1.57.2.0 AUQ-failure prose fallback ~2KB +
|
||||
@@ -387,7 +387,7 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
// +Conductor AUQ-default-prose rule + one-way/continuation safety in the
|
||||
// always-loaded AskUserQuestion Format section.
|
||||
// v1.2.0 activation lift: first-run-guidance section in the shared preamble.
|
||||
maxSkeletonBytes: 58_800, // + v1.78 AUQ objectivity + v1.79 foreground-dispatch sweep (merged); measured 58_364
|
||||
maxSkeletonBytes: 61_800, // + v2.0 {{ASIDE_RESEARCH}} (Aside first, WebSearch fallback); measured 60_628
|
||||
minUnionBytes: 64_200, // token-reduction Phases 1-2 (v1.69.x branch); measured union 71,379
|
||||
mustContain: ['OWASP', 'STRIDE', 'daily', 'comprehensive', 'verif'],
|
||||
// cso keeps its mode-dispatch + FP-filtering phases always-loaded, so the
|
||||
@@ -424,7 +424,7 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
gateAfterStop: undefined, // operational multi-STOP skill, like ship
|
||||
},
|
||||
behavioral: 'plan',
|
||||
maxSkeletonBytes: 59_150, // + v1.78 AUQ spawned-trigger objectivity (explicit declaration + interactive fence); measured 58_722
|
||||
maxSkeletonBytes: 61_500, // + v2.0 {{ASIDE_RESEARCH}} (Aside first, WebSearch fallback); measured 60_309
|
||||
minUnionBytes: 89_000, // Phase 4 wave 1; measured union 93,357
|
||||
mustContain: ['confidence', 'P1', 'P2', 'Review Army', 'adversarial'],
|
||||
},
|
||||
@@ -477,7 +477,8 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
gateAfterStop: undefined, // operational skill
|
||||
},
|
||||
behavioral: 'prompt',
|
||||
maxSkeletonBytes: 62_450, // + v1.78 AUQ spawned-trigger objectivity (explicit declaration + interactive fence); measured 62_021
|
||||
maxSkeletonBytes: 74_500, // + Aside browser contract for Step 7 canary ({{ASIDE_SETUP}}); measured 73_523
|
||||
maxSizeRatio: 1.10, // + v1.81 Aside contract + gstack-browser fallback block; measured 1.077
|
||||
minUnionBytes: 91_000, // Phase 4 wave 1; estimated union ~94.9KB
|
||||
mustContain: ['readiness', 'merge', 'canary', 'revert', 'staging'],
|
||||
},
|
||||
@@ -541,7 +542,7 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
gateAfterStop: undefined,
|
||||
},
|
||||
behavioral: 'prompt',
|
||||
maxSkeletonBytes: 57_100, // + v1.78 AUQ spawned-trigger objectivity (explicit declaration + interactive fence); measured 56_697
|
||||
maxSkeletonBytes: 58_400, // + v2.0 {{ASIDE_RESEARCH}} (Aside first, WebSearch fallback); measured 57_356
|
||||
minUnionBytes: 64_500, // measured union 67,430
|
||||
mustContain: ['HARD GATE', 'dedupe', 'quality gate', 'acceptance criteria', 'archive'],
|
||||
},
|
||||
@@ -581,11 +582,11 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
expectedSections: ['test-bootstrap.md', 'qa-patterns.md'],
|
||||
requiredReads: ['qa-patterns.md'],
|
||||
scenario:
|
||||
'Walk /qa in SIMULATION — do not launch a browser, run any $B command, or execute bash; treat the working tree as clean, the tier as Quick, and the target app as http://localhost:3000 with a small feature-branch diff touching one page. Skip the test-framework bootstrap (assume CLAUDE.md documents the test command). Read each pointed section before doing its step, then produce the QA plan as the report: the mode you selected and why, the Phase 1-6 steps you would run, and a worked health-score computation from the rubric. Do NOT use AskUserQuestion.',
|
||||
'Walk /qa in SIMULATION — do not launch a browser, run any aside command, or execute bash; treat the working tree as clean, the tier as Quick, and the target app as http://localhost:3000 with a small feature-branch diff touching one page. Skip the test-framework bootstrap (assume CLAUDE.md documents the test command). Read each pointed section before doing its step, then produce the QA plan as the report: the mode you selected and why, the Phase 1-6 steps you would run, and a worked health-score computation from the rubric. Do NOT use AskUserQuestion.',
|
||||
staticInvariants: {
|
||||
mustStayInSkeleton: [
|
||||
'## Setup',
|
||||
'## SETUP (run this check BEFORE any browse command)',
|
||||
'## BROWSER SETUP (Aside',
|
||||
'## Phases 1-6: QA Baseline',
|
||||
'## Phase 7: Triage',
|
||||
'## Phase 8: Fix Loop',
|
||||
@@ -605,29 +606,32 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
gateAfterStop: undefined,
|
||||
},
|
||||
behavioral: 'prompt',
|
||||
maxSkeletonBytes: 52_550, // + v1.78 AUQ spawned-trigger objectivity (explicit declaration + interactive fence); measured 52_150
|
||||
maxSkeletonBytes: 63_500, // + v2.0 {{ASIDE_SETUP}}/{{BROWSE_FALLBACK}} (replaces the browse setup block); measured 61_253
|
||||
maxSizeRatio: 1.08, // + v1.81 Aside contract + gstack-browser fallback block; measured 1.063
|
||||
minUnionBytes: 69_500, // measured union 70,385
|
||||
mustContain: ['bug', 'browse', 'fix', 'Health Score Rubric', 'regression'],
|
||||
// 'aside repl' pins the Aside contract; '$B goto' pins the fallback block in the always-loaded skeleton.
|
||||
mustContain: ['bug', 'aside repl', '$B goto', 'fix', 'Health Score Rubric', 'regression'],
|
||||
},
|
||||
browse: {
|
||||
skill: 'browse',
|
||||
expectedSections: ['command-list.md'],
|
||||
requiredReads: ['command-list.md'],
|
||||
scenario:
|
||||
'QA a static page: before driving it, plan the full audit — enumerate which browse commands and snapshot flags you would use, including extraction/tab/dialog commands beyond the Most-Used table, reading the full command reference first. Do not launch the browser or run any $B command; produce the command plan as the report.',
|
||||
'QA a static page: before driving it, plan the full audit — enumerate which browser steps you would run (the aside repl shapes, and the $B commands and snapshot flags the fallback would need), including extraction/tab/dialog commands beyond the basics, reading the full command reference first. Do not launch any browser or run any aside/$B command; produce the plan as the report.',
|
||||
staticInvariants: {
|
||||
mustStayInSkeleton: ['## SETUP', '## Core QA Patterns', '## CSS Inspector', '## Most-Used Commands'],
|
||||
mustPrecedeStop: ['## SETUP'],
|
||||
// Aside is the primary browser: the skeleton carries the Aside contract;
|
||||
// the $B command reference + snapshot flags are the carved section.
|
||||
mustStayInSkeleton: ['## BROWSER SETUP (Aside'],
|
||||
mustPrecedeStop: ['## BROWSER SETUP (Aside'],
|
||||
mustMoveToSection: ['## Full Command List', '## Snapshot Flags', '### Navigation'],
|
||||
gateAfterStop: undefined,
|
||||
},
|
||||
behavioral: 'prompt',
|
||||
maxSkeletonBytes: 27_500, // Phase 4 wave 3; measured 26,875
|
||||
minUnionBytes: 39_500, // measured union 41,115
|
||||
// 'BEGIN/END UNTRUSTED EXTERNAL' pins the untrusted-content warning; the full
|
||||
// envelope phrase wraps across lines in the rendered blockquote, so the
|
||||
// contiguous-substring check needs the single-line prefix form.
|
||||
mustContain: ['BEGIN/END UNTRUSTED EXTERNAL', 'snapshot -i', '@e refs', 'deviceScaleFactor', 'handoff'],
|
||||
maxSkeletonBytes: 32_000, // + v2.0 {{ASIDE_SETUP}}/{{BROWSE_FALLBACK}} skeleton + the STOP-Read pointer; measured 29_986
|
||||
minUnionBytes: 36_000, // skeleton + command-list section (14,240); measured union ~38.5 KB
|
||||
// 'BEGIN/END UNTRUSTED EXTERNAL' pins the untrusted-content warning in the
|
||||
// command reference; 'aside repl' pins the Aside contract in the skeleton.
|
||||
mustContain: ['BEGIN/END UNTRUSTED EXTERNAL', 'snapshot -i', '@e refs', 'aside repl'],
|
||||
},
|
||||
retro: {
|
||||
skill: 'retro',
|
||||
@@ -701,7 +705,7 @@ export const CARVE_GUARDS: Record<string, CarveGuard> = {
|
||||
gateAfterStop: undefined,
|
||||
},
|
||||
behavioral: 'prompt',
|
||||
maxSkeletonBytes: 53_100, // + v1.78 AUQ objectivity + v1.79 foreground-dispatch sweep (merged); measured 52_685
|
||||
maxSkeletonBytes: 54_000, // + v2.0 preamble growth (merged); measured 52_962
|
||||
minUnionBytes: 53_200, // Phase 4 wave 4; measured union 54,290
|
||||
mustContain: ["Don't make me think", "Users scan, they don't read", 'trunk test', '44px minimum'],
|
||||
},
|
||||
|
||||
@@ -36,7 +36,7 @@ import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import { promotedEnv } from '../../lib/conductor-env-shim';
|
||||
import { isProcessAlive, safeUnlink } from '../../browse/src/error-handling';
|
||||
import { isProcessAlive, safeUnlink } from '../../lib/error-handling';
|
||||
import { skillCensus, frontmatterName } from './skill-census';
|
||||
|
||||
/** Exact env names a hermetic child keeps. Everything not listed (or matched
|
||||
|
||||
@@ -4,7 +4,7 @@ import { execFileSync, spawnSync } from 'child_process';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import { resolveClaudeCommand } from '../../../browse/src/claude-bin';
|
||||
import { resolveClaudeCommand } from '../../../lib/claude-bin';
|
||||
|
||||
/**
|
||||
* Claude adapter — wraps the `claude` CLI via claude -p.
|
||||
|
||||
@@ -17,7 +17,7 @@ import * as path from 'path';
|
||||
|
||||
/** CLI-only commands: valid $B invocations that are handled by the CLI, not the server */
|
||||
const CLI_COMMANDS = new Set([
|
||||
'status', 'pair-agent', 'tunnel',
|
||||
'status', 'pair-agent', 'tunnel', '--help',
|
||||
]);
|
||||
|
||||
export interface BrowseCommand {
|
||||
@@ -42,6 +42,7 @@ export function extractBrowseCommands(skillPath: string): BrowseCommand[] {
|
||||
const lines = content.split('\n');
|
||||
const commands: BrowseCommand[] = [];
|
||||
|
||||
let inCodeBlock = false;
|
||||
let inBashBlock = false;
|
||||
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
@@ -49,20 +50,28 @@ export function extractBrowseCommands(skillPath: string): BrowseCommand[] {
|
||||
|
||||
// Detect code block boundaries
|
||||
if (line.trimStart().startsWith('```')) {
|
||||
if (inBashBlock) {
|
||||
inBashBlock = false;
|
||||
} else if (line.trimStart().startsWith('```bash')) {
|
||||
inBashBlock = true;
|
||||
}
|
||||
// Non-bash code blocks (```json, ```, ```js, etc.) are skipped
|
||||
inCodeBlock = !inCodeBlock;
|
||||
inBashBlock = inCodeBlock && line.trimStart().startsWith('```bash');
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!inBashBlock) continue;
|
||||
let source: string;
|
||||
if (!inCodeBlock) {
|
||||
// Prose and table rows: the {{BROWSE_FALLBACK}} mapping table carries its
|
||||
// `$B` shapes in backticks — validate them like code-block commands.
|
||||
// `[flags]`-style placeholders are documentation, not arguments.
|
||||
const spans = [...line.matchAll(/`(\$B\s+[^`]+)`/g)].map(m => m[1].replace(/\[[^\]]*\]/g, ''));
|
||||
if (spans.length === 0) continue;
|
||||
source = spans.join(' ');
|
||||
} else if (inBashBlock) {
|
||||
source = line;
|
||||
} else {
|
||||
continue; // Non-bash code blocks (```json, ```, ```js, etc.) are skipped
|
||||
}
|
||||
|
||||
// Match lines with $B command invocations
|
||||
// Match $B command invocations
|
||||
// Handle multiple $B commands on one line (e.g., "$B click @e3 $B fill @e4 "value"")
|
||||
const matches = line.matchAll(/\$B\s+(\S+)(?:\s+([^\$]*))?/g);
|
||||
const matches = source.matchAll(/\$B\s+(\S+)(?:\s+([^\$]*))?/g);
|
||||
for (const match of matches) {
|
||||
const command = match[1];
|
||||
let argsStr = (match[2] || '').trim();
|
||||
@@ -144,7 +153,9 @@ export function validateSkill(skillPath: string): ValidationResult {
|
||||
*/
|
||||
export function extractRemoteSlugPatterns(rootDir: string, subdirs: string[]): Map<string, string[]> {
|
||||
const results = new Map<string, string[]>();
|
||||
const pattern = /^REMOTE_SLUG=\$\(.*\)$/;
|
||||
// Accepts both the bare `REMOTE_SLUG=$(...)` form and the gstack-slug form
|
||||
// (`eval "$(...gstack-slug)"; REMOTE_SLUG="${SLUG:-...}"`).
|
||||
const pattern = /^(?:eval\s[^;]*;\s*)?REMOTE_SLUG=\S/;
|
||||
|
||||
for (const subdir of subdirs) {
|
||||
const dir = path.join(rootDir, subdir);
|
||||
|
||||
@@ -36,7 +36,7 @@ export const TOOL_COMPATIBILITY: Record<'claude' | 'gpt' | 'gemini', Record<Tool
|
||||
Glob: true,
|
||||
Grep: true,
|
||||
AskUserQuestion: true,
|
||||
WebSearch: true,
|
||||
WebSearch: true, // research skills try Aside first ({{ASIDE_RESEARCH}}); WebSearch is the fallback
|
||||
WebFetch: true,
|
||||
},
|
||||
gpt: {
|
||||
@@ -64,7 +64,7 @@ export const TOOL_COMPATIBILITY: Record<'claude' | 'gpt' | 'gemini', Record<Tool
|
||||
Glob: false,
|
||||
Grep: false,
|
||||
AskUserQuestion: false,
|
||||
WebSearch: true,
|
||||
WebSearch: true, // research skills try Aside first ({{ASIDE_RESEARCH}}); WebSearch is the fallback
|
||||
WebFetch: false,
|
||||
},
|
||||
};
|
||||
|
||||
@@ -25,6 +25,14 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
|
||||
'browse-basic': ['browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-bws.test.ts'],
|
||||
'browse-snapshot': ['browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-bws.test.ts'],
|
||||
|
||||
// Aside-driven browsing skills — live E2E against the Aside AI browser, the
|
||||
// primary browser (test/skill-e2e-aside.test.ts self-skips without a running Aside)
|
||||
'aside-browse-basic': ['browse/**', 'scripts/resolvers/browse.ts', 'scripts/resolvers/aside.ts', 'browse/test/test-server.ts', 'browse/test/fixtures/basic.html', 'test/helpers/aside-available.ts', 'test/skill-e2e-aside.test.ts'],
|
||||
'aside-browse-flow': ['browse/**', 'scripts/resolvers/browse.ts', 'scripts/resolvers/aside.ts', 'browse/test/test-server.ts', 'browse/test/fixtures/forms.html', 'test/helpers/aside-available.ts', 'test/skill-e2e-aside.test.ts'],
|
||||
'aside-qa-quick': ['qa/**', 'scripts/resolvers/browse.ts', 'scripts/resolvers/aside.ts', 'browse/test/test-server.ts', 'browse/test/fixtures/basic.html', 'test/helpers/aside-available.ts', 'test/skill-e2e-aside.test.ts'],
|
||||
'aside-scrape-json': ['scrape/**', 'scripts/resolvers/aside.ts', 'browse/test/test-server.ts', 'browse/test/fixtures/basic.html', 'test/helpers/aside-available.ts', 'test/skill-e2e-aside.test.ts'],
|
||||
'aside-canary-quick': ['canary/**', 'scripts/resolvers/aside.ts', 'browse/test/test-server.ts', 'browse/test/fixtures/basic.html', 'test/helpers/aside-available.ts', 'test/skill-e2e-aside.test.ts'],
|
||||
|
||||
// Hermetic isolation canaries (hermetic-env.ts is also a GLOBAL touchfile;
|
||||
// these entries exist so the canaries themselves stay tier-classified)
|
||||
'hermetic-canary': ['test/helpers/hermetic-env.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-hermetic-canary.test.ts', 'lib/conductor-env-shim.ts'],
|
||||
@@ -44,13 +52,14 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
|
||||
'session-awareness': ['SKILL.md', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-bws.test.ts'],
|
||||
'operational-learning': ['scripts/resolvers/preamble.ts', 'bin/gstack-learnings-log', 'test/skill-e2e-bws.test.ts'],
|
||||
|
||||
// QA (+ test-server dependency)
|
||||
'qa-quick': ['qa/**', 'browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-qa-workflow.test.ts'],
|
||||
'qa-b6-static': ['qa/**', 'browse/src/**', 'browse/test/test-server.ts', 'test/helpers/llm-judge.ts', 'browse/test/fixtures/qa-eval.html', 'test/fixtures/qa-eval-ground-truth.json', 'test/skill-e2e-qa-bugs.test.ts'],
|
||||
'qa-b7-spa': ['qa/**', 'browse/src/**', 'browse/test/test-server.ts', 'test/helpers/llm-judge.ts', 'browse/test/fixtures/qa-eval-spa.html', 'test/fixtures/qa-eval-spa-ground-truth.json', 'test/skill-e2e-qa-bugs.test.ts'],
|
||||
'qa-b8-checkout': ['qa/**', 'browse/src/**', 'browse/test/test-server.ts', 'test/helpers/llm-judge.ts', 'browse/test/fixtures/qa-eval-checkout.html', 'test/fixtures/qa-eval-checkout-ground-truth.json', 'test/skill-e2e-qa-bugs.test.ts'],
|
||||
'qa-only-no-fix': ['qa-only/**', 'qa/templates/**', 'test/skill-e2e-qa-workflow.test.ts'],
|
||||
'qa-fix-loop': ['qa/**', 'browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-qa-workflow.test.ts'],
|
||||
// QA (+ test-server dependency). /qa drives Aside first (the resolver) and
|
||||
// the browse binary as fallback (browse/src), so both are deps.
|
||||
'qa-quick': ['qa/**', 'scripts/resolvers/browse.ts', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-qa-workflow.test.ts'],
|
||||
'qa-b6-static': ['qa/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/helpers/llm-judge.ts', 'browse/test/fixtures/qa-eval.html', 'test/fixtures/qa-eval-ground-truth.json', 'test/skill-e2e-qa-bugs.test.ts'],
|
||||
'qa-b7-spa': ['qa/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/helpers/llm-judge.ts', 'browse/test/fixtures/qa-eval-spa.html', 'test/fixtures/qa-eval-spa-ground-truth.json', 'test/skill-e2e-qa-bugs.test.ts'],
|
||||
'qa-b8-checkout': ['qa/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/helpers/llm-judge.ts', 'browse/test/fixtures/qa-eval-checkout.html', 'test/fixtures/qa-eval-checkout-ground-truth.json', 'test/skill-e2e-qa-bugs.test.ts'],
|
||||
'qa-only-no-fix': ['qa-only/**', 'qa/templates/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-qa-workflow.test.ts'],
|
||||
'qa-fix-loop': ['qa/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-qa-workflow.test.ts'],
|
||||
'qa-bootstrap': ['qa/**', 'ship/**', 'test/skill-e2e-qa-workflow.test.ts'],
|
||||
|
||||
// Review
|
||||
@@ -298,15 +307,19 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
|
||||
// Design
|
||||
'design-consultation-core': ['design-consultation/**', 'scripts/gen-skill-docs.ts', 'test/helpers/llm-judge.ts', 'test/skill-e2e-design.test.ts'],
|
||||
'design-consultation-existing': ['design-consultation/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
||||
'design-consultation-research': ['design-consultation/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
||||
'design-consultation-research': ['design-consultation/**', 'scripts/resolvers/aside.ts', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
||||
'design-consultation-preview': ['design-consultation/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
||||
'plan-design-review-no-ui-scope': ['plan-design-review/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
||||
'design-review-fix': ['design-review/**', 'browse/src/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
||||
'design-review-fix': ['design-review/**', 'scripts/resolvers/aside.ts', 'scripts/resolvers/design.ts', 'browse/src/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
||||
|
||||
// /diagram (diagram-render bundle consumers). Triplet = deterministic
|
||||
// functional (gate); authoring quality = LLM-judged benchmark (periodic).
|
||||
'diagram-triplet': ['diagram/**', 'lib/diagram-render/**', 'browse/src/write-commands.ts', 'browse/src/read-commands.ts', 'test/skill-e2e-diagram.test.ts'],
|
||||
'diagram-authoring-quality': ['diagram/**', 'lib/diagram-render/**', 'test/helpers/llm-judge.ts', 'test/skill-e2e-diagram.test.ts'],
|
||||
// Both render the triplet through gstack-render (lib/aside-render.ts +
|
||||
// bin/gstack-render.ts): Aside when it is running, the browse daemon
|
||||
// otherwise — so both engines are deps. Triplet = deterministic functional
|
||||
// (gate); authoring quality = LLM-judged benchmark (periodic).
|
||||
'diagram-triplet': ['diagram/**', 'lib/diagram-render/**', 'lib/aside-render.ts', 'bin/gstack-render.ts', 'test/helpers/aside-available.ts', 'browse/src/**', 'test/skill-e2e-diagram.test.ts'],
|
||||
'diagram-authoring-quality': ['diagram/**', 'lib/diagram-render/**', 'lib/aside-render.ts', 'bin/gstack-render.ts', 'test/helpers/aside-available.ts', 'browse/src/**', 'test/helpers/llm-judge.ts', 'test/skill-e2e-diagram.test.ts'],
|
||||
|
||||
// gstack-upgrade
|
||||
'gstack-upgrade-happy-path': ['gstack-upgrade/**', 'test/skill-e2e-workflow.test.ts'],
|
||||
@@ -315,8 +328,8 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
|
||||
'land-and-deploy-workflow': ['land-and-deploy/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-deploy.test.ts'],
|
||||
'land-and-deploy-first-run': ['land-and-deploy/**', 'scripts/gen-skill-docs.ts', 'bin/gstack-slug', 'test/skill-e2e-deploy.test.ts'],
|
||||
'land-and-deploy-review-gate': ['land-and-deploy/**', 'bin/gstack-review-read', 'test/skill-e2e-deploy.test.ts'],
|
||||
'canary-workflow': ['canary/**', 'browse/src/**', 'test/skill-e2e-deploy.test.ts'],
|
||||
'benchmark-workflow': ['benchmark/**', 'browse/src/**', 'test/skill-e2e-deploy.test.ts'],
|
||||
'canary-workflow': ['canary/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'test/skill-e2e-deploy.test.ts'],
|
||||
'benchmark-workflow': ['benchmark/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'test/skill-e2e-deploy.test.ts'],
|
||||
'setup-deploy-workflow': ['setup-deploy/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-deploy.test.ts'],
|
||||
|
||||
|
||||
@@ -483,6 +496,14 @@ export const E2E_TIERS: Record<string, 'gate' | 'periodic'> = {
|
||||
'browse-basic': 'gate',
|
||||
'browse-snapshot': 'gate',
|
||||
|
||||
// Aside-driven browsing — periodic (external app: the Aside browser; the
|
||||
// file self-gates on 'periodic' and skips without a live Aside)
|
||||
'aside-browse-basic': 'periodic',
|
||||
'aside-browse-flow': 'periodic',
|
||||
'aside-qa-quick': 'periodic',
|
||||
'aside-scrape-json': 'periodic',
|
||||
'aside-canary-quick': 'periodic',
|
||||
|
||||
// Hermetic isolation — gate (deterministic env/config assertions; if the
|
||||
// clean room breaks, every other eval's signal is contaminated)
|
||||
'hermetic-canary': 'gate',
|
||||
@@ -718,7 +739,8 @@ export const E2E_TIERS: Record<string, 'gate' | 'periodic'> = {
|
||||
'plan-design-review-no-ui-scope': 'gate',
|
||||
'design-review-fix': 'periodic',
|
||||
|
||||
// /diagram — triplet is deterministic functional, judge is a quality benchmark
|
||||
// /diagram — triplet is deterministic functional (gstack-render falls back
|
||||
// to the browse daemon, so CI runs it); judge is a quality benchmark
|
||||
'diagram-triplet': 'gate',
|
||||
'diagram-authoring-quality': 'periodic',
|
||||
|
||||
@@ -740,9 +762,14 @@ export const E2E_TIERS: Record<string, 'gate' | 'periodic'> = {
|
||||
// Multi-provider benchmark — periodic (requires external CLIs + auth, paid)
|
||||
'benchmark-providers-live': 'periodic',
|
||||
|
||||
// Browser-skills Phase 2a — gate (D1/D3 contracts must not silently break)
|
||||
'scrape-match-path': 'gate',
|
||||
'scrape-prototype-path': 'gate',
|
||||
// Browser-skills Phase 2a — skillify keys gate (D1/D3 contracts must not
|
||||
// silently break). The two scrape keys are periodic: /scrape is Aside-first
|
||||
// and its fallback no longer prescribes the `$B skill list` / `skill run`
|
||||
// match + prototype flow the tests assert, so a pass rides on prompt
|
||||
// compliance (non-deterministic). Flip back to gate when scrape's fallback
|
||||
// carries the browser-skills flow again.
|
||||
'scrape-match-path': 'periodic',
|
||||
'scrape-prototype-path': 'periodic',
|
||||
'skillify-happy-path': 'gate',
|
||||
'skillify-provenance-refusal': 'gate',
|
||||
'skillify-approval-reject': 'gate',
|
||||
@@ -796,7 +823,7 @@ export const LLM_JUDGE_TOUCHFILES: Record<string, string[]> = {
|
||||
'command reference table': ['browse/sections/**', 'SKILL.md', 'SKILL.md.tmpl', 'browse/src/commands.ts', 'test/skill-llm-eval.test.ts'],
|
||||
'snapshot flags reference': ['browse/sections/**', 'SKILL.md', 'SKILL.md.tmpl', 'browse/src/snapshot.ts', 'test/skill-llm-eval.test.ts'],
|
||||
'browse/SKILL.md reference': ['browse/sections/**', 'browse/SKILL.md', 'browse/SKILL.md.tmpl', 'browse/src/**', 'test/skill-llm-eval.test.ts'],
|
||||
'setup block': ['SKILL.md', 'SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
||||
'setup block': ['browse/SKILL.md', 'browse/SKILL.md.tmpl', 'scripts/resolvers/aside.ts', 'scripts/resolvers/browse.ts', 'test/skill-llm-eval.test.ts'],
|
||||
'regression vs baseline': ['browse/sections/**', 'SKILL.md', 'SKILL.md.tmpl', 'browse/src/commands.ts', 'test/fixtures/eval-baselines.json', 'test/skill-llm-eval.test.ts'],
|
||||
'qa/SKILL.md workflow': ['qa/sections/**', 'qa/SKILL.md', 'qa/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
||||
'qa/SKILL.md health rubric': ['qa/sections/**', 'qa/SKILL.md', 'qa/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
||||
|
||||
Reference in New Issue
Block a user