mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-27 23:21:53 +02:00
* fix(memory-ingest): --scan-secrets scans the rendered page and fails closed --scan-secrets ran gitleaks on the raw transcript .jsonl, then imported a page rendered from it. gitleaks' assignment rules don't match across a JSON-escaped quote (KEY=\"v\" on disk), so a secret the rendered page shows as KEY="v" was imported unflagged. And the gate skipped a file only on scanner "gitleaks" with findings, so a scan that errored (non-zero exit, 16MB maxBuffer overflow on a file with many findings, unparseable report) or could not run (gitleaks missing, slow-probe cooldown) imported the file unscanned. Scan the rendered page body, the exact bytes writeStaged() writes, via a new secretScanText() helper, and skip the file whenever the scan did not complete. Skipped files stay out of the state file, so the next run retries them. Reword the helper warnings and setup-gbrain/memory.md, which described the fail-open as intended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(test): reconcile Bun failure markers and footer counts * fix(sync-gbrain): verify source-scoped reads without mutation * fix(test): recognize grounded TTHW target choices structurally * fix(aside): make the readiness probe work under zsh and report why it failed The probe built its deadline into `_T` and expanded it unquoted, so `$_T aside repl …` only worked in a shell that word-splits. zsh does not: it looked for a command literally named "gtimeout 30", the probe answered ASIDE_NOT_RUNNING with Aside installed and ready, and every browsing skill fell back to the bundled Chromium in silence. zsh is the macOS default and Aside is macOS-only, so on a stock Mac the probe could never report READY. The deadline becomes a function, `_gs_d`. It receives the command as "$@", already split, so sh, bash and zsh all behave the same, and the gtimeout → timeout → perl alarm chain is unchanged. A 4th arm runs the call unbounded when none of the three is present, which is what the empty `_T` did before. Not `eval`: it re-parses the string, so the parens and `;` of the perl arm become syntax and that arm dies in bash *and* zsh — on a stock Mac, the arm that actually runs. On failure the probe now prints the CLI's reason after ASIDE_NOT_RUNNING:, the shape gstack-render already uses: the first line that starts with a capital letter, i.e. the CLI's own sentence or Node's `Error:` line below its loader frame. "Not running" covers states with different fixes — no window open for the profile, a NODE_OPTIONS preload that kills the CLI — and a bare verdict sent all of them to "open the Aside app". The BROWSER SETUP prose quotes that reason before asking the user to open the app. The text pin asserted the broken invocation verbatim, so it now pins the function and asserts neither `$_T aside repl` nor an eval form comes back. A second test executes the rendered probe in sh, bash and zsh on each of the four deadline arms with stubbed binaries on a narrowed PATH, plus two failing CLIs: one that prints its own sentence, one that crashes like Node with the useful line below the frame. The deadline function costs zero bytes against the lines it replaces; the reason costs 53 per copy of the probe (44 where the reworded BROWSER SETUP line gives 9 back). That moves four guards by the measured amount: plan-devex-review's skeleton cap to 68,550 (measured 68,544), plan-ceo-review's skeleton cap to 80,150 (measured 80,111) and union ratio to 1.081 (measured 1.0803), and plan-eng-review's union ratio to 1.151 (measured 1.1504). Fixes #2842, #2941. * Clarify engineering review startup and decision flow * Fix Windows readiness fixture PATH and command shim * fix(test): recognize grounded TTHW target choices structurally * Clarify engineering review startup and decision flow * fix(test): restrict QA-only fixture tools to its no-Edit contract * v1.90.0.0 fix(sync-gbrain): guard readiness verdicts and refresh metadata * fix(browse): validate canonical upload targets * fix(gbrain): classify structured PGLite busy response * fix(browse): preserve native extension runtime APIs * Fix displayless browser handoff ownership * Accept unique installed autoplan methodology aliases * fix(skills): preserve positional literals during installation * fix(browse): checksum installer contents through stdin * fix(test): normalize Windows checksum fixture paths * test: emulate unavailable shasum in Windows checksum fixture * fix(investigate): preserve owned freeze lifecycle * fix(review): preserve N+1 retry and Red Team completion * fix: bound Aside readiness and preserve safe fallback * test: exercise setup and Chromium on native ARM * fix: preserve install ownership and ARM browser selection * Fix gbrain ingest scan boundaries and seed observation * Refresh managed ship hooks and supervise expanded paid census * Reject resumed gbrain pages excluded by current policy * Recover zombie agent locks safely and enable CI Python venv * Repair paid actor declarations and Aside pitch assertions * Bump consolidated wave to next free minor release * Clarify CEO review admin choices and option tradeoffs * Preserve CEO mode handoff anchors in clarified workflow * Make Windows portability fixtures use shell-native paths * Restore ARM Bun alias and clarify ship review gates * Refresh ship workflow golden snapshots * Fix Windows DX documentation controls without piped stdin * Decode Codex child pipes without Bun's encoded-stream stall * Bound DX pre-review audit before product questions * Clarify trusted review-start read in paid revalidation * Bump consolidated wave to next free minor release * Clarify CEO review admin choices and option tradeoffs * Preserve CEO mode handoff anchors in clarified workflow * Make Windows portability fixtures use shell-native paths * Restore ARM Bun alias and clarify ship review gates * Refresh ship workflow golden snapshots * Fix Windows DX documentation controls without piped stdin * Decode Codex child pipes without Bun's encoded-stream stall * Bound DX pre-review audit before product questions * Clarify trusted review-start read in paid revalidation * Reconcile new main planning flow and paid judge census * fix: reconcile rebased planning and source-bound validation * test: pin cookie workflow judge to scored Sonnet model * fix: keep terminal agent boot out of module imports * fix: preserve pending-question uncertainty in engineering review * fix: stabilize Windows reliability-wave fixtures * fix: clarify design consultation research workflow * fix: preserve independent design consultation inputs * fix: resolve design taste scope and browser research guidance * fix: make consultation opt-in preflight unambiguous * test: await native Edge owner readiness or terminal result --------- Co-authored-by: Bruce Krysiak <brucek@alum.mit.edu> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Antonio Vitalic <antoninte99@gmail.com>
161 lines
8.8 KiB
Docker
161 lines
8.8 KiB
Docker
# gstack CI eval runner — pre-baked toolchain + deps
|
|
# Rebuild weekly via ci-image.yml, on Dockerfile changes, or on lockfile changes
|
|
FROM ubuntu:24.04
|
|
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
|
|
# Switch apt sources to Hetzner's public mirror.
|
|
# Ubicloud runners (Hetzner FSN1-DC21) hit reliable connection timeouts to
|
|
# archive.ubuntu.com:80 — observed 90+ second outages on multiple builds.
|
|
# Hetzner's mirror is publicly accessible from any cloud and route-local for
|
|
# Ubicloud, so this fixes both reliability and latency. Ubuntu 24.04 uses
|
|
# the deb822 sources format at /etc/apt/sources.list.d/ubuntu.sources.
|
|
#
|
|
# Using HTTP (not HTTPS) intentionally: the base ubuntu:24.04 image ships
|
|
# without ca-certificates, so HTTPS apt fails with "No system certificates
|
|
# available." Apt's security model verifies via GPG-signed Release files,
|
|
# not TLS, so HTTP here is no weaker than the upstream defaults.
|
|
RUN sed -i \
|
|
-e 's|http://archive.ubuntu.com/ubuntu|http://mirror.hetzner.com/ubuntu/packages|g' \
|
|
-e 's|http://security.ubuntu.com/ubuntu|http://mirror.hetzner.com/ubuntu/packages|g' \
|
|
/etc/apt/sources.list.d/ubuntu.sources
|
|
|
|
# Also make apt itself resilient — per-package retries + generous timeouts.
|
|
# Hetzner's mirror is reliable but individual packages can still blip; the
|
|
# retry config means a single failed fetch doesn't nuke the whole build.
|
|
RUN printf 'Acquire::Retries "5";\nAcquire::http::Timeout "30";\nAcquire::https::Timeout "30";\n' \
|
|
> /etc/apt/apt.conf.d/80-retries
|
|
|
|
# System deps (retry apt-get update + install as a unit — even Hetzner can blip).
|
|
# Includes xz-utils so the Node.js .tar.xz download below can decompress.
|
|
# python3: bin/gstack-jsonl-merge, gstack-brain-sync, gstack-detach, and other
|
|
# bash bins shell out to it (macOS ships python3; the base image doesn't).
|
|
# file: skill-validation's no-compiled-binaries-in-git check runs `file --mime-type`.
|
|
# poppler-utils: make-pdf's e2e gates hard-require pdftotext/pdffonts/pdfinfo in CI.
|
|
RUN for i in 1 2 3; do \
|
|
apt-get update && apt-get install -y --no-install-recommends \
|
|
git curl unzip xz-utils ca-certificates jq bc gpg python3 python3-venv file poppler-utils gcc libc6-dev && break || \
|
|
(echo "apt retry $i/3 after failure"; sleep 10); \
|
|
done \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
RUN python3 -m venv /tmp/gstack-ci-venv \
|
|
&& /tmp/gstack-ci-venv/bin/python -m pip --version \
|
|
&& rm -rf /tmp/gstack-ci-venv
|
|
|
|
# Direct builds produce the trusted CSO launcher and watchdog. Check the exact
|
|
# static-C capability here so the cached eval image cannot reach a slice without it.
|
|
RUN printf 'int main(void) { return 0; }\n' > /tmp/gstack-cso-cc-probe.c \
|
|
&& cc -std=c11 -static /tmp/gstack-cso-cc-probe.c -o /tmp/gstack-cso-cc-probe \
|
|
&& /tmp/gstack-cso-cc-probe \
|
|
&& rm -f /tmp/gstack-cso-cc-probe.c /tmp/gstack-cso-cc-probe
|
|
|
|
# GitHub CLI
|
|
RUN curl --retry 5 --retry-delay 5 --retry-connrefused -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \
|
|
| gpg --dearmor -o /usr/share/keyrings/githubcli-archive-keyring.gpg \
|
|
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \
|
|
| tee /etc/apt/sources.list.d/github-cli.list > /dev/null \
|
|
&& for i in 1 2 3; do \
|
|
apt-get update && apt-get install -y --no-install-recommends gh && break || \
|
|
(echo "gh install retry $i/3"; sleep 10); \
|
|
done \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Node.js 22 LTS (needed for claude CLI).
|
|
# Install from the official nodejs.org tarball instead of NodeSource's apt setup.
|
|
# NodeSource's setup_22.x script runs its own `apt-get update` + `apt-get install gnupg`,
|
|
# both of which depend on archive.ubuntu.com / security.ubuntu.com being reachable.
|
|
# Ubicloud CI runners frequently can't reach those mirrors (connection timeouts),
|
|
# and "gnupg" was renamed to "gpg" on Ubuntu 24.04 anyway, so NodeSource's script
|
|
# fails before it can add its own repo. Direct tarball download is network-simpler
|
|
# (one host: nodejs.org) and doesn't touch apt at all.
|
|
ENV NODE_VERSION=22.20.0
|
|
RUN curl --retry 5 --retry-delay 5 --retry-connrefused -fsSL "https://nodejs.org/dist/v${NODE_VERSION}/node-v${NODE_VERSION}-linux-x64.tar.xz" -o /tmp/node.tar.xz \
|
|
&& tar -xJ -C /usr/local --strip-components=1 --no-same-owner -f /tmp/node.tar.xz \
|
|
&& rm -f /tmp/node.tar.xz \
|
|
&& node --version \
|
|
&& npm --version
|
|
|
|
# Bun (install to /usr/local so non-root users can access it).
|
|
# The version MUST be passed as a positional arg — bun.sh/install ignores a
|
|
# BUN_VERSION env var, so the old `| BUN_VERSION=x.y.z bash` form silently
|
|
# installed latest on every image rebuild (observed: 1.3.13/1.3.14 drift vs
|
|
# the 1.3.10 devs ran locally).
|
|
ENV BUN_INSTALL="/usr/local"
|
|
RUN curl --retry 5 --retry-delay 5 --retry-connrefused -fsSL https://bun.sh/install \
|
|
| bash -s "bun-v1.4.0"
|
|
|
|
# Claude CLI — pinned to an EXACT version, same discipline as the bun pin
|
|
# above. The PTY harness (test/helpers/claude-pty-runner.ts) screen-scrapes
|
|
# this CLI's TUI (trust dialog, input prompt, spinner glyphs); an unpinned
|
|
# install rebuilt weekly rode the TUI wherever it drifted, and that drift
|
|
# broke the harness three separate times (welcome-screen wedge on 2.1.233,
|
|
# skillify HOME discovery on 2.1.237, guard/freeze hooks on 2.1.162).
|
|
# Bump deliberately, via a PR that runs the PTY gate against the new TUI.
|
|
# test/ci-image-cli-pin.test.ts fails the free suite if this pin is removed.
|
|
RUN npm i -g @anthropic-ai/claude-code@2.1.251
|
|
|
|
# Playwright system deps (Chromium) — needed for browse E2E tests
|
|
RUN npx playwright install-deps chromium
|
|
|
|
# Linux has neither Helvetica nor Arial. make-pdf's print CSS stacks fall back
|
|
# to Liberation Sans (metric-compatible Arial clone, SIL OFL 1.1) so PDFs don't
|
|
# render in DejaVu Sans. playwright install-deps happens to pull this in today,
|
|
# but the dep is implicit and could change — install explicitly so upgrades
|
|
# can't silently regress rendering.
|
|
#
|
|
# Xvfb is also installed here so the browse --headed integration tests
|
|
# (headed-xvfb, headed-orphan-cleanup) can exercise the Linux container
|
|
# auto-spawn path on every CI run. Without Xvfb in the image, the most
|
|
# common production --headed path goes untested.
|
|
# fonts-noto-color-emoji: the make-pdf emoji render gate needs a color-emoji
|
|
# fallback font (mirrors make-pdf-gate.yml's Ubuntu setup step).
|
|
RUN for i in 1 2 3; do \
|
|
apt-get update && apt-get install -y --no-install-recommends fonts-liberation fonts-noto-color-emoji fontconfig xvfb x11-utils && break || \
|
|
(echo "fonts-liberation install retry $i/3"; sleep 10); \
|
|
done \
|
|
&& fc-cache -f \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Pre-install dependencies (cached layer — only rebuilds when package.json or
|
|
# bun.lock changes). Copy BOTH so install is deterministic and matches local
|
|
# resolution. Without bun.lock here, bun install resolved transitive deps
|
|
# differently in CI vs local (observed on v1.28.0.0: socks landed but
|
|
# smart-buffer + ip-address didn't make it into the cached node_modules).
|
|
# patches/ rides along: bun.lock's patchedDependencies (playwright-core
|
|
# windowsHide, v1.67) makes install fail without the patch files present —
|
|
# and the workflows' image-tag hash includes patches/** so editing a patch
|
|
# rebuilds this layer.
|
|
COPY package.json bun.lock /workspace/
|
|
COPY patches /workspace/patches
|
|
WORKDIR /workspace
|
|
RUN bun install --frozen-lockfile && rm -rf /tmp/*
|
|
|
|
# Install Playwright Chromium to a shared location accessible by all users
|
|
ENV PLAYWRIGHT_BROWSERS_PATH=/opt/playwright-browsers
|
|
RUN npx playwright install chromium \
|
|
&& chmod -R a+rX /opt/playwright-browsers
|
|
|
|
# Verify everything works
|
|
RUN bun --version && node --version && claude --version && jq --version && gh --version \
|
|
&& python3 --version && command -v file && command -v pdftotext && command -v pdffonts && command -v pdfinfo \
|
|
&& npx playwright --version \
|
|
&& fc-match "Liberation Sans" | grep -qi "Liberation" \
|
|
|| (echo "ERROR: fonts-liberation not installed — make-pdf PDFs will render in DejaVu Sans" && exit 1)
|
|
|
|
# At runtime: checkout overwrites /workspace, but node_modules persists
|
|
# if we move it out of the way and symlink back
|
|
# Save node_modules + package.json snapshot for cache validation at runtime
|
|
RUN mv /workspace/node_modules /opt/node_modules_cache \
|
|
&& cp /workspace/package.json /opt/node_modules_cache/.package.json \
|
|
&& cp /workspace/bun.lock /opt/node_modules_cache/.bun.lock
|
|
|
|
# Claude CLI refuses --dangerously-skip-permissions as root.
|
|
# Create a non-root user for eval runs (GH Actions overrides USER, so
|
|
# the workflow must set options.user or use gosu/su-exec at runtime).
|
|
RUN useradd -m -s /bin/bash runner \
|
|
&& chmod -R a+rX /opt/node_modules_cache \
|
|
&& mkdir -p /home/runner/.gstack && chown -R runner:runner /home/runner/.gstack \
|
|
&& chmod 1777 /tmp \
|
|
&& mkdir -p /home/runner/.bun && chown -R runner:runner /home/runner/.bun
|