mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-20 11:52:20 +02:00
v1.81.0.0 feat: Aside is the browser gstack drives first; every browsing skill, the PDF/diagram renderer, and web research; the bundled browser stays the automatic fallback (#2810)
* feat(aside): browser-driver contract, cookbook, research and fallback resolvers
{{ASIDE_SETUP}} (readiness probe + ten rules for driving the user's real browser), {{ASIDE_COOKBOOK}} (script shapes verified live against Aside CLI 1.26: one flow per aside repl script, CDP console hook before navigation, evidence lines, session-directory artifact handoff, GSTACK_STEP_OK sentinel), {{ASIDE_RESEARCH}} (research through aside exec, WebSearch when Aside is absent, knowledge otherwise) and {{BROWSE_FALLBACK}} (the fifteen-row Aside-step to $B-command table plus the rules that differ, so every browsing skill keeps working on gstack's own headless browser). test/aside-driver.test.ts pins the sentences and asserts every browsing skill carries the Aside block followed by the fallback; test/helpers/aside-available.ts is the shared live-Aside probe.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(render): Aside-first local-HTML renderer with the bundled browser as fallback
lib/aside-render.ts serves the HTML's directory on loopback (Aside refuses file:// URLs), opens it with waitUntil load, prints through CDP Page.printToPDF so tagged output, outlines, header/footer templates and page numbers survive, emulates device metrics for sized screenshots, and writes in-page evaluations to files; when Aside is absent it runs the same spec through the browse daemon (newtab, load, js, pdf, screenshot, closetab) and reports ENGINE=aside|browse. bin/gstack-render.ts is the CLI skill templates call. lib/claude-bin.ts and lib/error-handling.ts become the canonical copies (browse/src re-exports them).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(browse): /browse drives Aside first, with the $B reference behind the fallback
Contract, cookbook, mode choice (aside repl by default, aside exec for reading), report format, the fallback section, and the full command reference carved on demand.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(qa): /qa and /qa-only drive Aside, fall back to $B
QA_METHODOLOGY runs every phase as Aside scripts (orient, explore, document, re-test, mobile viewport via CDP emulation, links via HEAD fetch); the authenticate phase is 'you are already signed in'; a 13th rule requires consent before mutating actions on non-local targets; the fallback section translates each step onto $B. The qa E2E tests run on whichever engine is present.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(design): design-review, design-consultation, design-shotgun, plan-design-review, design-html drive Aside
Design-system extraction is one script printing FONTS/COLORS/HEADINGS/TOUCH_TARGETS/NAV; competitor research confirms the exact URLs before opening them in the real browser and runs on the bundled browser when Aside is absent; design-html's viewport screenshots, sketches and comparison boards render through gstack-render.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(deploy): benchmark, canary, land-and-deploy Step 7, devex-review drive Aside
One aside repl script per page prints NAV/PAINT/LCP/RESOURCES/SCRIPTS/CSS/SUMMARY (benchmark), CONSOLE_ERRORS/NAV/TEXT + screenshot (canary, re-run every 60s), and the post-deploy check reads responseStatus from the navigation entry; each carries the $B fallback.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(third-party-actions): Aside is the recommended driver; gstack's visible browser stays the fallback
The readiness probe is lifted from {{ASIDE_SETUP}} at gen time (byte-identity pinned) and rule 3 points at browse/SKILL.md for how to drive; the consent question offers Aside first and gstack's own visible browser (handoff/resume for sign-in) as the fallback, as v1.72 framed it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(scrape): /scrape reads pages through Aside; the browser-skills runtime rides the fallback
Look-then-extract scripts build the JSON inside the page and print it between JSON_START/JSON_END; aside exec for fuzzy intents; on the $B fallback the browser-skills match/prototype flow and /skillify apply as before.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(make-pdf): print through Aside first, the bundled browser otherwise
asideClient.ts replaces the direct $B client with one render() call per PDF (the exact option mapping the browse pdf command had: paper, margins, header/footer/page numbers, tagged, outline, printBackground, preferCSSPageSize, Paged.js wait); the diagram pre-pass, oversized-image downscale and DOCX rasters each run as one render script with per-fence try/catch; exit 4 now means no browser is available and names both remedies; $P setup reports which engine it found. The e2e gates run on whichever engine is present, so the Linux lane exercises the fallback.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(diagram): the triplet is one gstack-render call
SVG, PNG and excalidraw from one invocation over the content-addressed bundle staged under /tmp/gstack-render; every diagram type gets an excalidraw export; gstack-render picks the engine and prints ENGINE=; the diagram E2E gates on either engine.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(research): web research runs in Aside first, WebSearch second
The planning, review, design, security and investigate skills research through {{ASIDE_RESEARCH}}; WebSearch stays in allowed-tools as the fallback; testing.ts's bootstrap step follows; skeleton ceilings ratcheted for the research block.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(setup,gen-skill-docs): prune renders of skills that no longer exist
setup gains _prune_stale_generated for every host tree and the doc generator removes gstack-* output dirs it did not write, so a skill removed from the source tree can never linger in an install.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test: registries, budgets and suite reconciled for Aside-first with the $B fallback
Touchfiles + E2E tiers gain the Aside keys, coverage matrix and eval baselines updated, size budget re-baselined to parity-baseline-v1.80.0.0.json (the contract plus fallback ride in every browsing skill), parity ceilings ratcheted with measured values, LLM-judge prompts and the E2E fixtures speak Aside-first, browse-fallback.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: Aside first, gstack browser fallback
README, BROWSER.md, docs/, CONTRIBUTING, CLAUDE.md, ARCHITECTURE, AGENTS.md, TODOS and the root router describe the one product story: Aside is the browser gstack drives first; the bundled headless browser is the automatic fallback (Linux, Windows, app closed) where cookie import, GStack Browser, pair-agent and browser-skills still apply.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore: regenerate SKILL.md docs, llms.txt, agents digest, ship goldens, context-budget fixture
bun run gen:skill-docs over the templates; goldens re-rendered; context-budget ceilings recaptured.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* v1.80.0.0: Aside is the browser gstack drives first; the bundled browser is the fallback
MINOR: new capability across ten skills, the renderer and research; nothing removed. CHANGELOG release summary + itemized changes; VERSION 1.80.0.0; package.json 1.80.0.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(todos): file non-Claude host ownership-gate and version-heading pin follow-ups
Two follow-ups from the /plan-ceo-review + /plan-eng-review pass on merging
PR #2804 with main's v1.80.0.0 ownership gate: bring the Codex/Factory/
OpenCode/Cursor/Kiro copy loops and the stale-render prune under the
.gstack-owned marker rule, and a free test pinning that the CHANGELOG top
heading equals VERSION (the collision that git cannot see).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix: pre-landing review fixes for the Aside-first branch
Review army + adversarial passes (Claude and Codex) on the merged branch:
setup
- _prune_stale_generated scans the host dirs too (the generator already
removed the render before setup ran, so the host branch was dead), skips
symlinks in the render tree (rm -rf on a slash-terminated link empties its
target), removes a host symlink only when it resolves into gstack, cleans a
bannered real dir through _cleanup_weak_dir, recognizes frontmatter-renamed
skills, and logs through log. The always-run codex render passes every host
dir that may link to it.
- NEEDS_BUILD checks all three binaries (with $_EXE) and lib/ sources; the
browser hint and the bootstrap summary honor GSTACK_SKIP_ASIDE, treat a
requested skip as a request, and derive one skill list.
lib/aside-render.ts + bin/gstack-render.ts
- The loopback server carries a per-render secret path, checks containment on
the real path (symlink escapes are 403), and rejects malformed encoding.
- Inline eval results are one base64 line, so page text cannot forge
ASIDE_DIR= or the sentinel; the last ASIDE_DIR wins.
- runProc escalates SIGTERM to SIGKILL, bounds every wait, and clears every
timer (an uncleared one kept gstack-render alive after printing OK).
- renderTmpDir refuses a shared /tmp name owned by someone else; the work dir
and server are created inside try; goto's budget follows the render budget.
- probeAside classifies a present-but-failing CLI as ASIDE_NOT_RUNNING like
the skills' bash probe; render() retries on gstack's own browser when Aside
could not start or its private CDP bridge is gone (never on a page error
or a timeout of a running script); the CLI reports the engine that actually
rendered, exits 0 on --help, rejects non-numeric flags, documents
--wait-timeout, fences EVAL/PAGE_ERRORS as untrusted content, and names the
daemon's cookie-import JS lock remedy.
- The browse path passes --scale only when asked (a scale change rebuilds
the daemon context) and restores the viewport after a sized screenshot.
resolvers / templates
- The bash probe honors GSTACK_SKIP_ASIDE and has a perl deadline on stock
macOS; .local is no longer LOCAL (mDNS); same-origin filters compare parsed
origins; link status is HEAD-checked only on LOCAL targets; every
aside exec goes through the receipted _aside_exec prelude
({{ASIDE_EXEC_PRELUDE}}), including nine template blocks that called it
bare; the design sketch and diagram staging use private directories.
- The generator prunes only bannered renders and never a host whose
generation failed.
Docs, stale comments and dead code cleaned; goldens re-rendered; tests
updated and added for every behavior above.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test: coverage for the render CLI, setup rebuild check, make-pdf exit codes, and prose $B spans
New free tests from the ship coverage audit: test/gstack-render-cli.test.ts
(argv guards, --help, output contract with a fake daemon, failure and
serve-root paths, no-browser case, prompt exit), test/setup-needs-build.test.ts
(every binary and source set flips NEEDS_BUILD, Windows suffixes),
make-pdf/test/cli-exit-codes.test.ts and setup-smoke.test.ts (error to exit
code mapping, runSetup stages, renderPdf's engine), and prose-span cases for
extractBrowseCommands in test/skill-parser.test.ts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: CHANGELOG and TODOS cover the review fixes (v1.81.0.0)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: sync project docs with the v1.81.0.0 review fixes
BROWSER.md, ARCHITECTURE.md, CONTRIBUTING.md, README.md, CLAUDE.md,
docs/TESTING_INTERNALS.md and docs/PROJECT_STRUCTURE.md now describe the
shipped renderer and setup: the loopback render server's per-render secret
path and real-path containment, ENGINE= naming the engine that actually
rendered (mid-run retry on gstack's own browser), EVAL/PAGE_ERRORS fenced as
untrusted content, --wait-timeout and the CLI's argv guards, the receipted
_aside_exec prelude ({{ASIDE_EXEC_PRELUDE}} in the placeholder table), the
LOCAL host rule without .local, LOCAL-only HEAD checks in the links script,
GSTACK_SKIP_ASIDE across probe/renderer/setup, the ownership-gated
retired-skill prune, the widened NEEDS_BUILD check, and the new free tests
(gstack-render-cli, setup-prune-stale-generated, setup-browser-hint,
setup-needs-build, make-pdf cli-exit-codes and setup-smoke).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: CHANGELOG states the precise mid-run retry rule
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(test): skill-e2e-bws slices the $B setup block from the Browser fallback section
browse/SKILL.md no longer has '## SETUP' / '## Core QA Patterns' (Aside is the
primary driver; the $B block moved under 'Browser fallback'), so the gate test
sliced an empty block and handed the agent nothing to run. Anchor on
'### Find the `$B` binary' up to the next heading. 7/7 pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(test): gate POSIX-only fixtures off Windows
windows-free-tests: the gstack-render CLI tests drive a shebang fake browse
that CreateProcess cannot exec, and two NEEDS_BUILD cases assert an execute
bit and a bare-name miss that MSYS bash does not have (test -x ignores mode
bits and resolves design -> design.exe). Those describes and cases now
self-skip on win32; argument guards, --help, the no-browser case, and every
other rebuild-check case still run there.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(render): runProc waits for the exit code until the kill deadline; newtab retries once on a cold daemon
A process whose pipes have reached EOF is exiting, but runProc gave the exit
code only five seconds to arrive and then returned null, which run() reports
as a failed command. Under CI's six-shard load one such render failed with the
artifact already written. The SIGTERM/SIGKILL timers already bound the wait,
so the exit race now runs to the kill deadline.
The first CLI call auto-starts the browse daemon; on a cold start it can
answer 'Unable to connect' once while the server is still coming up. That
single case is retried after 1.5s; every other newtab failure is not.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(aside-render): warm the daemon before live fallback cases; failures name the render error
- Live fallback cases run 'goto about:blank' up to twice before asserting and
skip (never fail) when the daemon cannot come up.
- expectOk() puts r.error and the browse transcript into the assertion so a
failed render is diagnosable from the CI log.
- The argv-contract cases dump the fake's log on a miss.
- File default timeout is 30s: the subject is the CLI contract, not latency.
- Two cases pin the cold-daemon newtab retry and that other errors are not
retried.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: CHANGELOG notes the cold-start tolerance of the bundled-browser renderer
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Sina <sdroid674+github@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Fable 5.1
Sina
Claude Fable 5.1
Sina
parent
c241216637
commit
0530392821
+207
-219
@@ -6,8 +6,8 @@
|
||||
* │ fences → placeholder tokens │
|
||||
* │ ▼
|
||||
* └─▶ renderFenceSlots() ───────────▶ substituteSlots(html, slots)
|
||||
* one browse render tab/run │
|
||||
* error ⇒ diagnostic block + page reload ▼
|
||||
* one browser render per batch │
|
||||
* error ⇒ diagnostic block ▼
|
||||
* inlineLocalImages(html)
|
||||
* data URIs, probe dims from bytes,
|
||||
* downscale >2x content box @300dpi,
|
||||
@@ -19,9 +19,11 @@
|
||||
* through the same sanitizer as user content before substitution (the bundle
|
||||
* renders with securityLevel strict — the sanitizer is the second layer).
|
||||
*
|
||||
* Reset contract (eng-review D6.2): each fence renders with a fresh
|
||||
* mermaid.render id; after ANY render error the bundle page is reloaded before
|
||||
* the next fence so a poisoned global can't corrupt diagram N+1.
|
||||
* Bundle calls are batched: one render per batch (Aside runs one script per `aside repl` process) and
|
||||
* nothing survives it, so every consumer collects its calls, runs them in one
|
||||
* script (`BundleRun`), and substitutes the results. A failed call is data
|
||||
* (diagnostic block / warning), never an abort. Each PDF run gets a fresh
|
||||
* bundle page and each fence a fresh mermaid.render id (eng-review D6.2).
|
||||
*/
|
||||
|
||||
import * as fs from "node:fs";
|
||||
@@ -30,7 +32,7 @@ import * as path from "node:path";
|
||||
import * as crypto from "node:crypto";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import * as browseClient from "./browseClient";
|
||||
import { render as renderHtml, renderTmpDir } from "../../lib/aside-render";
|
||||
import { escapeHtml, sanitizeUntrustedHtml } from "./render";
|
||||
import { imageDims } from "./image-size";
|
||||
|
||||
@@ -72,8 +74,8 @@ export interface PrepassImageOptions {
|
||||
/** Physical content-box width in inches (page width minus margins). */
|
||||
contentWidthIn: number;
|
||||
warn: (msg: string) => void;
|
||||
/** Lazily provides a ready bundle tab (only opened when needed). */
|
||||
getTab: () => RenderTab | null;
|
||||
/** Bundle runner for print-resolution downscaling; null = inline at full size. */
|
||||
run: BundleRun | null;
|
||||
}
|
||||
|
||||
/** Print-resolution policy (eng-review D4): downscale rasters wider than
|
||||
@@ -289,145 +291,89 @@ function diagramLabel(fence: DiagramFence): string {
|
||||
return fence.title ?? `diagram ${fence.ordinal}`;
|
||||
}
|
||||
|
||||
// ─── Render tab (bundle page lifecycle) ───────────────────────────────
|
||||
// ─── Bundle runner (diagram-render page, driven through Aside or gstack's browser) ────────
|
||||
|
||||
export type BundleCall = { fn: string; args: unknown[] };
|
||||
export type BundleResult =
|
||||
| { ok: true; value: string }
|
||||
| { ok: false; error: string };
|
||||
/** Run bundle calls in order. Every call gets a result; failures are data. */
|
||||
export type BundleRun = (calls: BundleCall[]) => Promise<BundleResult[]>;
|
||||
|
||||
const PAYLOAD_TMP_DIR = process.platform === "win32" ? os.tmpdir() : "/tmp";
|
||||
const READY_TIMEOUT_MS = 20_000;
|
||||
// Expressions bigger than this ship via `browse eval <file>` instead of argv.
|
||||
// 8KB is safe on every platform (Windows CreateProcess caps the WHOLE command
|
||||
// line at 32,767 chars; Linux MAX_ARG_STRLEN is ~128KiB) and the tmp-file
|
||||
// round-trip costs microseconds — one spawn regardless of payload size.
|
||||
const MAX_ARGV_EXPR_BYTES = 8_000;
|
||||
/** Aside caps a script at 120s (the browse path shares the budget); ~40 mermaid renders fit with room to spare. */
|
||||
const CALLS_PER_SCRIPT = 40;
|
||||
|
||||
export class RenderTab {
|
||||
private constructor(
|
||||
public readonly tabId: number,
|
||||
private readonly stagedBundlePath: string,
|
||||
) {}
|
||||
|
||||
/**
|
||||
* Open a tab and load the diagram-render bundle. The bundle HTML is staged
|
||||
* under /tmp (content-addressed, reused across runs — load-html only reads
|
||||
* inside its safe dirs) and loaded by PATH, not --from-file: a 9MB JSON
|
||||
* round-trip per run would be pure waste.
|
||||
*/
|
||||
static open(): RenderTab {
|
||||
const bundleSrc = resolveBundlePath();
|
||||
const html = fs.readFileSync(bundleSrc);
|
||||
const sha = crypto.createHash("sha256").update(html).digest("hex").slice(0, 16);
|
||||
const staged = path.join(PAYLOAD_TMP_DIR, `gstack-diagram-render-${sha}.html`);
|
||||
// Never trust an existing file at the predictable shared-/tmp name: verify
|
||||
// its content hash and re-stage on mismatch (a pre-planted file would
|
||||
// otherwise be loaded into the render tab as the bundle).
|
||||
let needsWrite = true;
|
||||
if (fs.existsSync(staged)) {
|
||||
try {
|
||||
const existing = crypto.createHash("sha256").update(fs.readFileSync(staged)).digest("hex").slice(0, 16);
|
||||
needsWrite = existing !== sha;
|
||||
} catch {
|
||||
needsWrite = true;
|
||||
}
|
||||
}
|
||||
if (needsWrite) {
|
||||
// Concurrent-safe: write to a unique temp name, then atomic rename.
|
||||
const tmp = `${staged}.${process.pid}.${crypto.randomBytes(4).toString("hex")}`;
|
||||
fs.writeFileSync(tmp, html);
|
||||
try {
|
||||
fs.renameSync(tmp, staged);
|
||||
} catch (renameErr) {
|
||||
try { fs.unlinkSync(tmp); } catch { /* best-effort tmp cleanup */ }
|
||||
// Only swallow the rename failure when the surviving file HASHES to
|
||||
// the expected bundle (a concurrent writer won an OS-level race).
|
||||
// Sticky-bit /tmp makes rename-over-foreign-file fail EPERM — if the
|
||||
// survivor were trusted on existence alone, a pre-planted file would
|
||||
// ride through the exact check added to stop it.
|
||||
let survivorOk = false;
|
||||
try {
|
||||
const survivor = crypto.createHash("sha256").update(fs.readFileSync(staged)).digest("hex").slice(0, 16);
|
||||
survivorOk = survivor === sha;
|
||||
} catch { /* unreadable survivor = not ok */ }
|
||||
if (!survivorOk) throw renameErr;
|
||||
}
|
||||
}
|
||||
const tabId = browseClient.newtab();
|
||||
const tab = new RenderTab(tabId, staged);
|
||||
tab.loadBundle();
|
||||
return tab;
|
||||
}
|
||||
|
||||
/** (Re)load the bundle page — also the reset path after a render error. */
|
||||
loadBundle(): void {
|
||||
browseClient.loadHtmlFile({ file: this.stagedBundlePath, tabId: this.tabId });
|
||||
const ready = browseClient.waitForExpression({
|
||||
expression: "document.getElementById('status') !== null && document.getElementById('status').textContent === 'ready'",
|
||||
tabId: this.tabId,
|
||||
timeoutMs: READY_TIMEOUT_MS,
|
||||
});
|
||||
if (!ready) {
|
||||
throw new Error(
|
||||
"diagram-render bundle did not become ready in the browse tab " +
|
||||
`(${READY_TIMEOUT_MS}ms). Check \`browse js "window.__errors"\` on tab ${this.tabId}.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Call one of the bundle's async window functions with JSON-safe string
|
||||
* args. Errors come back as a recognizable ERR: prefix so a render failure
|
||||
* is data, not a thrown browse exit.
|
||||
*/
|
||||
call(fn: string, ...args: Array<string | number>): string {
|
||||
const argList = args.map((a) => JSON.stringify(a)).join(",");
|
||||
const expression =
|
||||
`window.${fn}(${argList})` +
|
||||
`.then(r => "OK:" + r)` +
|
||||
`.catch(e => "ERR:" + String((e && e.message) || e))`;
|
||||
const result = this.js(expression);
|
||||
if (result.startsWith("OK:")) return result.slice(3);
|
||||
if (result.startsWith("ERR:")) throw new RenderCallError(result.slice(4));
|
||||
throw new RenderCallError(`unexpected bundle result: ${result.slice(0, 200)}`);
|
||||
}
|
||||
|
||||
private js(expression: string): string {
|
||||
// Large payloads (scene JSON, SVG text, data URIs) blow past argv limits —
|
||||
// browseClient.js shells out with the expression as an argv element. The
|
||||
// limit is BYTES, not chars (CJK content is 3x its char count in UTF-8),
|
||||
// and Windows caps the whole command line at 32,767 chars — so anything
|
||||
// big ships via `browse eval <file>` instead: one spawn, any size.
|
||||
if (Buffer.byteLength(expression, "utf8") <= MAX_ARGV_EXPR_BYTES) {
|
||||
return browseClient.js({ expression, tabId: this.tabId });
|
||||
}
|
||||
return this.jsViaFile(expression);
|
||||
}
|
||||
|
||||
/** argv-safe path for big expressions: stage to a tmp file under browse's
|
||||
* safe dirs and run `browse eval <file>` (one spawn regardless of size). */
|
||||
private jsViaFile(expression: string): string {
|
||||
const file = path.join(
|
||||
PAYLOAD_TMP_DIR,
|
||||
`gstack-diagram-expr-${process.pid}-${crypto.randomBytes(4).toString("hex")}.js`,
|
||||
);
|
||||
fs.writeFileSync(file, expression, "utf8");
|
||||
/**
|
||||
* Build the runner. The bundle path resolves lazily on the first call so an
|
||||
* image-only document never touches it; a missing bundle fails every call
|
||||
* with the resolver's message instead of throwing.
|
||||
*/
|
||||
export function bundleRunner(opts: { bundlePath?: string; render?: typeof renderHtml } = {}): BundleRun {
|
||||
const render = opts.render ?? renderHtml;
|
||||
let bundlePath = opts.bundlePath;
|
||||
return async (calls) => {
|
||||
const results: BundleResult[] = [];
|
||||
try {
|
||||
return browseClient.evalFile({ file, tabId: this.tabId });
|
||||
} finally {
|
||||
try { fs.unlinkSync(file); } catch { /* best-effort tmp cleanup */ }
|
||||
if (calls.length > 0) bundlePath ??= resolveBundlePath();
|
||||
for (let i = 0; i < calls.length; i += CALLS_PER_SCRIPT) {
|
||||
results.push(...await runScript(bundlePath!, calls.slice(i, i + CALLS_PER_SCRIPT), render));
|
||||
}
|
||||
} catch (err: any) {
|
||||
// Unresolvable/unreadable bundle, staging failure: fail what's left as data.
|
||||
const error = firstLine(err?.message ?? String(err));
|
||||
while (results.length < calls.length) results.push({ ok: false, error });
|
||||
}
|
||||
}
|
||||
|
||||
close(): void {
|
||||
try {
|
||||
browseClient.closetab(this.tabId);
|
||||
} catch {
|
||||
// best-effort: orchestrator finally path
|
||||
}
|
||||
}
|
||||
return results;
|
||||
};
|
||||
}
|
||||
|
||||
export class RenderCallError extends Error {
|
||||
constructor(msg: string) {
|
||||
super(msg);
|
||||
this.name = "RenderCallError";
|
||||
/**
|
||||
* One render (an Aside script, or a browse tab): open the bundle, wait for #done, evaluate one expression
|
||||
* per call, write each result to a file and read them back. The bundle copy
|
||||
* and one JSON args file per call sit in a private dir served over loopback,
|
||||
* so multi-MB payloads (data URIs, scene JSON) never ride argv; results are
|
||||
* files too (never inline stdout) so SVG/PNG text survives intact.
|
||||
*/
|
||||
async function runScript(
|
||||
bundlePath: string,
|
||||
calls: BundleCall[],
|
||||
render: typeof renderHtml,
|
||||
): Promise<BundleResult[]> {
|
||||
const dir = fs.mkdtempSync(path.join(renderTmpDir(), "make-pdf-diagrams-"));
|
||||
try {
|
||||
const bundle = path.join(dir, "diagram-render.html");
|
||||
fs.copyFileSync(bundlePath, bundle, fs.constants.COPYFILE_FICLONE);
|
||||
const steps = calls.map((call, i) => {
|
||||
fs.writeFileSync(path.join(dir, `call-${i}.json`), JSON.stringify(call.args));
|
||||
// try/catch INSIDE the expression: a throwing fence returns an ERR
|
||||
// marker and the script keeps going for the other fences. The URL is
|
||||
// resolved against location.href, not the document base: the bundle
|
||||
// sets <base href="https://gstack-render.localhost/"> for excalidraw.
|
||||
const expression =
|
||||
`(async () => { try { const a = await (await fetch(new URL("call-${i}.json", location.href).href)).json(); ` +
|
||||
`return "OK:" + await window[${JSON.stringify(call.fn)}](...a); } ` +
|
||||
`catch (e) { return "ERR:" + String((e && e.message) || e); } })()`;
|
||||
return { kind: "eval" as const, expression, out: path.join(dir, `result-${i}.txt`) };
|
||||
});
|
||||
const r = await render({
|
||||
file: bundle,
|
||||
serveRoot: dir,
|
||||
waitFor: { selector: "#done", timeoutMs: READY_TIMEOUT_MS },
|
||||
steps,
|
||||
});
|
||||
if (!r.ok) {
|
||||
const error = `diagram renderer: ${firstLine(r.error ?? "unknown error")}`;
|
||||
return calls.map(() => ({ ok: false, error }));
|
||||
}
|
||||
return calls.map((_, i) => {
|
||||
const text = fs.readFileSync(path.join(dir, `result-${i}.txt`), "utf8");
|
||||
if (text.startsWith("OK:")) return { ok: true, value: text.slice(3) };
|
||||
if (text.startsWith("ERR:")) return { ok: false, error: text.slice(4) };
|
||||
return { ok: false, error: `unexpected bundle result: ${text.slice(0, 200)}` };
|
||||
});
|
||||
} finally {
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -457,38 +403,39 @@ export function resolveBundlePath(env: NodeJS.ProcessEnv = process.env): string
|
||||
// ─── Fence rendering ──────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Render every extracted fence to its slot HTML. One bundle tab serves all
|
||||
* fences; a failed fence yields a diagnostic block and a bundle reload
|
||||
* (reset contract) before the next fence renders.
|
||||
* Render every extracted fence to its slot HTML in one batch. A failed fence
|
||||
* yields a visible diagnostic block; the others still render.
|
||||
*/
|
||||
export function renderFenceSlots(
|
||||
export async function renderFenceSlots(
|
||||
fences: DiagramFence[],
|
||||
tab: RenderTab,
|
||||
run: BundleRun,
|
||||
warn: (msg: string) => void,
|
||||
): Map<string, string> {
|
||||
): Promise<Map<string, string>> {
|
||||
const slots = new Map<string, string>();
|
||||
const fail = (fence: DiagramFence, msg: string) => {
|
||||
warn(`diagram ${fence.ordinal} (${fence.lang}) failed to render: ${firstLine(msg)}`);
|
||||
slots.set(fence.token, buildDiagnosticBlock(fence, msg));
|
||||
};
|
||||
const todo: DiagramFence[] = [];
|
||||
for (const fence of fences) {
|
||||
try {
|
||||
let svg: string;
|
||||
if (fence.lang === "mermaid") {
|
||||
svg = tab.call("__renderMermaid", `mermaid-fence-${fence.ordinal}`, fence.source);
|
||||
} else {
|
||||
JSON.parse(fence.source); // fail fast with a JSON diagnostic, not a bundle stack
|
||||
svg = tab.call("__excalidrawToSvg", fence.source);
|
||||
}
|
||||
slots.set(fence.token, buildDiagramFigure(fence, svg));
|
||||
} catch (err: any) {
|
||||
const msg = err?.message ?? String(err);
|
||||
warn(`diagram ${fence.ordinal} (${fence.lang}) failed to render: ${firstLine(msg)}`);
|
||||
slots.set(fence.token, buildDiagnosticBlock(fence, msg));
|
||||
// Reset contract: a poisoned page must not corrupt the next fence.
|
||||
if (fence.lang !== "mermaid") {
|
||||
try {
|
||||
tab.loadBundle();
|
||||
} catch (reloadErr: any) {
|
||||
warn(`bundle reload after render error failed: ${firstLine(reloadErr?.message ?? String(reloadErr))}`);
|
||||
JSON.parse(fence.source); // fail fast with a JSON diagnostic, not a bundle stack
|
||||
} catch (err: any) {
|
||||
fail(fence, err?.message ?? String(err));
|
||||
continue;
|
||||
}
|
||||
}
|
||||
todo.push(fence);
|
||||
}
|
||||
const results = await run(todo.map((f) => f.lang === "mermaid"
|
||||
? { fn: "__renderMermaid", args: [`mermaid-fence-${f.ordinal}`, f.source] }
|
||||
: { fn: "__excalidrawToSvg", args: [f.source] }));
|
||||
todo.forEach((fence, i) => {
|
||||
const r = results[i];
|
||||
if (r.ok) slots.set(fence.token, buildDiagramFigure(fence, r.value));
|
||||
else fail(fence, r.error);
|
||||
});
|
||||
return slots;
|
||||
}
|
||||
|
||||
@@ -498,15 +445,26 @@ export function renderFenceSlots(
|
||||
* Replace inline diagram SVGs (and svg data-URI images) with PNG <img> tags
|
||||
* for the DOCX export — Word's SVG support is unreliable, so the content-
|
||||
* fidelity contract embeds rasters at 300dpi of the placed width (the
|
||||
* content box). Diagnostic blocks keep their text form.
|
||||
* content box). Diagnostic blocks keep their text form. Two passes: swap
|
||||
* each target for a token while collecting its bundle call, run the batch,
|
||||
* substitute results.
|
||||
*/
|
||||
export function rasterizeDiagramFigures(
|
||||
export async function rasterizeDiagramFigures(
|
||||
html: string,
|
||||
tab: RenderTab,
|
||||
run: BundleRun,
|
||||
contentWidthIn: number,
|
||||
warn: (msg: string) => void,
|
||||
): string {
|
||||
): Promise<string> {
|
||||
const targetPx = Math.round(contentWidthIn * PRINT_DPI);
|
||||
const runId = crypto.randomBytes(4).toString("hex");
|
||||
const calls: BundleCall[] = [];
|
||||
const pending: Array<{ token: string; onOk: (png: string) => string; onErr: (reason: string) => string }> = [];
|
||||
const enqueue = (svgText: string, onOk: (png: string) => string, onErr: (reason: string) => string): string => {
|
||||
const token = `gstack-raster-slot-${runId}-${calls.length}`;
|
||||
calls.push({ fn: "__rasterize", args: [svgText, targetPx] });
|
||||
pending.push({ token, onOk, onErr });
|
||||
return token;
|
||||
};
|
||||
|
||||
// 1. Rendered diagram figures → <img> with the figure's aria-label as alt.
|
||||
let out = html.replace(
|
||||
@@ -515,21 +473,21 @@ export function rasterizeDiagramFigures(
|
||||
const svgMatch = figure.match(/<svg\b[\s\S]*<\/svg>/i);
|
||||
if (!svgMatch) return figure;
|
||||
const label = figure.match(/\baria-label\s*=\s*"([^"]*)"/i)?.[1] ?? "diagram";
|
||||
try {
|
||||
const png = tab.call("__rasterize", svgMatch[0], targetPx);
|
||||
return `<p><img src="${png}" alt="${label}"></p>`;
|
||||
} catch (err: any) {
|
||||
const reason = firstLine(err?.message ?? String(err));
|
||||
warn(`docx: diagram rasterization failed (${reason}); embedding source text instead`);
|
||||
// The converter drops <figure>/<svg> entirely, so returning the figure
|
||||
// would make the diagram vanish without a trace — the exact invisible
|
||||
// failure the diagnostic contract forbids. Surface the source.
|
||||
const source = decodeFigureSource(figure) ?? "(source unavailable)";
|
||||
return [
|
||||
`<p><strong>Diagram could not be rasterized for DOCX (${escapeHtml(reason)}) — source:</strong></p>`,
|
||||
`<pre>${escapeHtml(source)}</pre>`,
|
||||
].join("\n");
|
||||
}
|
||||
return enqueue(
|
||||
svgMatch[0],
|
||||
(png) => `<p><img src="${png}" alt="${label}"></p>`,
|
||||
(reason) => {
|
||||
warn(`docx: diagram rasterization failed (${reason}); embedding source text instead`);
|
||||
// The converter drops <figure>/<svg> entirely, so returning the figure
|
||||
// would make the diagram vanish without a trace — the exact invisible
|
||||
// failure the diagnostic contract forbids. Surface the source.
|
||||
const source = decodeFigureSource(figure) ?? "(source unavailable)";
|
||||
return [
|
||||
`<p><strong>Diagram could not be rasterized for DOCX (${escapeHtml(reason)}) — source:</strong></p>`,
|
||||
`<pre>${escapeHtml(source)}</pre>`,
|
||||
].join("\n");
|
||||
},
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
@@ -538,18 +496,25 @@ export function rasterizeDiagramFigures(
|
||||
const m = tag.match(SRC_RE);
|
||||
const src = m?.[2] ?? m?.[3] ?? "";
|
||||
if (!src.startsWith("data:image/svg+xml")) return tag;
|
||||
try {
|
||||
const b64 = src.slice(src.indexOf(",") + 1);
|
||||
const svgText = Buffer.from(b64, "base64").toString("utf8");
|
||||
const png = tab.call("__rasterize", svgText, targetPx);
|
||||
const svgText = Buffer.from(src.slice(src.indexOf(",") + 1), "base64").toString("utf8");
|
||||
return enqueue(
|
||||
svgText,
|
||||
// Function replacement: data URIs can contain $-patterns.
|
||||
return tag.replace(SRC_RE, () => `src="${png}"`);
|
||||
} catch (err: any) {
|
||||
warn(`docx: svg image rasterization failed (${firstLine(err?.message ?? String(err))})`);
|
||||
return tag;
|
||||
}
|
||||
(png) => tag.replace(SRC_RE, () => `src="${png}"`),
|
||||
(reason) => {
|
||||
warn(`docx: svg image rasterization failed (${reason})`);
|
||||
return tag;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
if (calls.length === 0) return out;
|
||||
const results = await run(calls);
|
||||
pending.forEach((p, i) => {
|
||||
const r = results[i];
|
||||
// split/join, not replace(): the replacement carries user content.
|
||||
out = out.split(p.token).join(r.ok ? p.onOk(r.value) : p.onErr(firstLine(r.error)));
|
||||
});
|
||||
return out;
|
||||
}
|
||||
|
||||
@@ -581,14 +546,21 @@ const SRC_RE = /\bsrc\s*=\s*("([^"]*)"|'([^']*)')/i;
|
||||
* tab. Missing files become visible placeholders (or throw under --strict);
|
||||
* remote URLs warn (offline posture) unless --allow-network.
|
||||
*/
|
||||
export function inlineLocalImages(html: string, opts: PrepassImageOptions): string {
|
||||
export async function inlineLocalImages(html: string, opts: PrepassImageOptions): Promise<string> {
|
||||
const maxPx = Math.round(opts.contentWidthIn * PRINT_DPI * DOWNSCALE_FACTOR);
|
||||
const targetPx = Math.round(opts.contentWidthIn * PRINT_DPI);
|
||||
// An image referenced N times is read/probed/downscaled once; the same data
|
||||
// URI string is reused (also dedupes memory until the final join).
|
||||
const memo = new Map<string, { dataUri: string; attrs: string }>();
|
||||
// Oversized rasters get a token src in this pass and their downscaled bytes
|
||||
// in the second — one bundle batch for the whole document.
|
||||
const runId = crypto.randomBytes(4).toString("hex");
|
||||
const downscales: Array<{
|
||||
token: string; src: string; name: string; buf: Buffer; mime: string;
|
||||
dims: { width: number; height: number };
|
||||
}> = [];
|
||||
|
||||
return html.replace(IMG_TAG_RE, (tag) => {
|
||||
const out = html.replace(IMG_TAG_RE, (tag) => {
|
||||
const srcMatch = tag.match(SRC_RE);
|
||||
if (!srcMatch) return tag;
|
||||
const src = srcMatch[2] ?? srcMatch[3] ?? "";
|
||||
@@ -675,38 +647,54 @@ export function inlineLocalImages(html: string, opts: PrepassImageOptions): stri
|
||||
return buildMissingImagePlaceholder(src);
|
||||
}
|
||||
|
||||
let buf = fs.readFileSync(filePath);
|
||||
let dims = imageDims(buf);
|
||||
let mime = dims?.mime ?? mimeFromExtension(filePath);
|
||||
const buf = fs.readFileSync(filePath);
|
||||
const dims = imageDims(buf);
|
||||
const mime = dims?.mime ?? mimeFromExtension(filePath);
|
||||
|
||||
// Print-resolution normalization (D4): rasters only — SVG scales free.
|
||||
if (dims && mime !== "image/svg+xml" && dims.width > maxPx) {
|
||||
const tab = opts.getTab();
|
||||
if (tab) {
|
||||
try {
|
||||
const dataUri = `data:${mime};base64,${buf.toString("base64")}`;
|
||||
const scaled = tab.call("__downscaleRaster", dataUri, targetPx, mime);
|
||||
const scaledB64 = scaled.replace(/^data:[^,]*,/, "");
|
||||
opts.warn(
|
||||
`downscaled ${path.basename(filePath)} ${dims.width}px → ${targetPx}px ` +
|
||||
`(print is ${PRINT_DPI}dpi; original exceeds ${maxPx}px content-box ceiling)`,
|
||||
);
|
||||
buf = Buffer.from(scaledB64, "base64");
|
||||
mime = scaled.slice(5, scaled.indexOf(";"));
|
||||
dims = { ...dims, height: Math.round((dims.height * targetPx) / dims.width), width: targetPx };
|
||||
} catch (err: any) {
|
||||
opts.warn(`downscale failed for ${src}, inlining at full size: ${firstLine(err?.message ?? String(err))}`);
|
||||
}
|
||||
}
|
||||
if (dims && mime !== "image/svg+xml" && dims.width > maxPx && opts.run) {
|
||||
const token = `gstack-downscale-slot-${runId}-${downscales.length}`;
|
||||
downscales.push({ token, src, name: path.basename(filePath), buf, mime, dims });
|
||||
memo.set(filePath, { dataUri: token, attrs: "" });
|
||||
return rewriteImgTag(tag, memo.get(filePath)!);
|
||||
}
|
||||
|
||||
const dataUri = `data:${mime};base64,${buf.toString("base64")}`;
|
||||
const attrs = dims
|
||||
? ` data-gstack-px-width="${Math.round(dims.width)}" data-gstack-px-height="${Math.round(dims.height)}"`
|
||||
: "";
|
||||
memo.set(filePath, { dataUri, attrs });
|
||||
memo.set(filePath, inlineEntry(buf, mime, dims));
|
||||
return rewriteImgTag(tag, memo.get(filePath)!);
|
||||
});
|
||||
|
||||
if (downscales.length === 0) return out;
|
||||
const results = await opts.run!(downscales.map((d) => ({
|
||||
fn: "__downscaleRaster",
|
||||
args: [`data:${d.mime};base64,${d.buf.toString("base64")}`, targetPx, d.mime],
|
||||
})));
|
||||
const byToken = new Map<string, { dataUri: string; attrs: string }>();
|
||||
downscales.forEach((d, i) => {
|
||||
const r = results[i];
|
||||
if (r.ok) {
|
||||
opts.warn(
|
||||
`downscaled ${d.name} ${d.dims.width}px → ${targetPx}px ` +
|
||||
`(print is ${PRINT_DPI}dpi; original exceeds ${maxPx}px content-box ceiling)`,
|
||||
);
|
||||
const height = Math.round((d.dims.height * targetPx) / d.dims.width);
|
||||
byToken.set(d.token, { dataUri: r.value, attrs: dimAttrs({ width: targetPx, height }) });
|
||||
} else {
|
||||
opts.warn(`downscale failed for ${d.src}, inlining at full size: ${firstLine(r.error)}`);
|
||||
byToken.set(d.token, inlineEntry(d.buf, d.mime, d.dims));
|
||||
}
|
||||
});
|
||||
return out.replace(IMG_TAG_RE, (tag) => {
|
||||
const entry = byToken.get(tag.match(SRC_RE)?.[2] ?? "");
|
||||
return entry ? rewriteImgTag(tag, entry) : tag;
|
||||
});
|
||||
}
|
||||
|
||||
function inlineEntry(buf: Buffer, mime: string, dims: { width: number; height: number } | null): { dataUri: string; attrs: string } {
|
||||
return { dataUri: `data:${mime};base64,${buf.toString("base64")}`, attrs: dims ? dimAttrs(dims) : "" };
|
||||
}
|
||||
|
||||
function dimAttrs(dims: { width: number; height: number }): string {
|
||||
return ` data-gstack-px-width="${Math.round(dims.width)}" data-gstack-px-height="${Math.round(dims.height)}"`;
|
||||
}
|
||||
|
||||
/** Apply a memoized inline result to an img tag. */
|
||||
|
||||
Reference in New Issue
Block a user