mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-31 22:14:25 +01:00
* fix(make-pdf,design): real TOC page numbers and anchors; design models move to gpt-5.5 + gpt-image-2 (#2903, #2807) * ci: pin every action to a SHA, Scorecard, checksummed Bun, scoped Codex CLI in the CI image, macOS named-regressions job (#1948, #1706, #1997) * fix(cso,browse): Windows/macOS lease identity, retention skips bad runs, truthful 'not assessed' reports, env.kv false positive, Windows browse spawn fixes (#2894, #2912, #2048, #2637, #2805) * fix(outside-voice): one Codex model precedence for probe and dispatch, bounded timeouts with KILL escalation, probe cache identity, guarded mktemp, skills hidden in nested Codex (#2776, #2914, #2787, #2881, #2847) * feat(setup): install registry, ./setup --status, staged activation, per-scope migrations, host tiers and conformance kit, OpenCode/Hermes fixes, Copilot CLI (experimental), disabled_skills, slimmer CLAUDE.md (#1925, #2879, #2347, #2338, #2626, #2629, #2825, #2015, #393, #1206, #2096) Co-authored-by: Andrey Esipov <61357035+andrey-esipov@users.noreply.github.com> Co-authored-by: Matthew Williams <5863886+ridermw@users.noreply.github.com> Co-authored-by: Rocky Wu <92764750+lolisaigao1234@users.noreply.github.com> * fix(preamble): skills start in worktree-isolated Claude Code sessions — one literal skill-start line, context-recovery helper, no eval in skill blocks, $N token tripwire, honest plan-mode precedence (#2763, #2896, #2851) * fix(setup): per-install renders for renamed, vendored and custom-config installs, space-safe alias roots, per-host /cso launcher path (#1882, #2906) * test: free tests never write the real HOME (home-write tripwire), no orphaned browse daemons, symlinked-TMPDIR (macOS) fixes, gbrain-on-PATH isolation (#2895, #2816, #2829, #2841, #2982, #2911, #2909, #2908, #2910) * chore: integrate wave follow-ups — regenerate all hosts, README model docs, TODOS, touchfiles, duration seeds, Bun 1.3 telemetry test fix, replay fixtures pinned to their recorded bytes * v1.91.14.0: release metadata * test(browse): group-kill cleanup falls back to the leader when macOS returns EPERM for a live group * ci: Codex login steps run under bash (the CI container's default sh rejects pipefail) * fix: full-suite repairs — private state roots for 7 real-HOME writers, truthful home-guard attribution with --attribute-home, gstack-paths shim in ship-hook fixtures, Copilot in team-init probes, test-of-test detector, cookie review fixture pinned to the approved excerpt * fix(windows): LF for Dockerfiles, normalized home-guard exclusions, path.relative fixture containment, ship push test spawn timeouts * test(design-detect): widen the whole-scan budget test's timing margins so a loaded macOS runner can't time out the single-batch call * fix(evals): qa protocol names defined for the judge, async PTY judge so overlapping sessions keep their cleanup deadline, bws setup cases use the standard JUDGE_MS tier * test(pty): the non-blocking judge case is POSIX-only (its fake judge is a shebang script, like the PTY harness it covers) * test(evals): ship-coverage-audit names absolute skill paths, uses Read, and skips the Step 7 artifact nothing consumes (removes the mechanism behind 3 of 4 historical timeouts) * test: deflake terminal-agent concurrent startup (write the placeholder record before spawning, like the product) and qa-evidence resumed pipe (barrier on a full high-water mark instead of a sleep) * fix(design): daemon log stream handles open errors instead of crashing the daemon; shutdown test awaits the stubbed exit * test(evals): skillify checks the product's resolved state root (no GSTACK_HOME bypass), negative cases check every root; tpa shims live outside the agent's cwd and answer consistently * test(evals): coverage-audit detector splits parallel columns at │ rails and [+] groups; coverage prompts name absolute skill paths; TODOs from the gate eval census * fix(plan-eng-review): scope gate always sends the menu (no headless 'Scope pending' exit); PTY harness ends idle turn-ends and API-error panels promptly with a precise diagnostic * test(evals): one position-bound coverage-read rule credits read-only && chains (labels, context ranges, git displays in either order) and still rejects forged output * fix(ship): documentation section has one saved source inserted by cat (never retyped); spawned-marker evidence is the child output; refreshed codex/factory ship goldens * test(docsync): the byte-exact insert positive is POSIX-only (the bash-command observer refuses backslash paths by design) * test(evals): plan-design-with-ui focus-title rule accepts the comma-less 'dimensions or focus' wording (replayed on the captured CI run, negative controls kept) * fix(qa-callers): exploratory preflight says the scope/method Reads come after exploratory.md; design-review scan check credits an executed scan proven by one engine run over the changed files --------- Co-authored-by: Andrey Esipov <61357035+andrey-esipov@users.noreply.github.com> Co-authored-by: Matthew Williams <5863886+ridermw@users.noreply.github.com> Co-authored-by: Rocky Wu <92764750+lolisaigao1234@users.noreply.github.com>
195 lines
11 KiB
Docker
195 lines
11 KiB
Docker
# gstack CI eval runner — pre-baked toolchain + deps
|
|
# Rebuild weekly via ci-image.yml, on Dockerfile changes, or on lockfile changes
|
|
FROM ubuntu:24.04
|
|
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
|
|
# Switch apt sources to Hetzner's public mirror.
|
|
# Ubicloud runners (Hetzner FSN1-DC21) hit reliable connection timeouts to
|
|
# archive.ubuntu.com:80 — observed 90+ second outages on multiple builds.
|
|
# Hetzner's mirror is publicly accessible from any cloud and route-local for
|
|
# Ubicloud, so this fixes both reliability and latency. Ubuntu 24.04 uses
|
|
# the deb822 sources format at /etc/apt/sources.list.d/ubuntu.sources.
|
|
#
|
|
# Using HTTP (not HTTPS) intentionally: the base ubuntu:24.04 image ships
|
|
# without ca-certificates, so HTTPS apt fails with "No system certificates
|
|
# available." Apt's security model verifies via GPG-signed Release files,
|
|
# not TLS, so HTTP here is no weaker than the upstream defaults.
|
|
RUN sed -i \
|
|
-e 's|http://archive.ubuntu.com/ubuntu|http://mirror.hetzner.com/ubuntu/packages|g' \
|
|
-e 's|http://security.ubuntu.com/ubuntu|http://mirror.hetzner.com/ubuntu/packages|g' \
|
|
/etc/apt/sources.list.d/ubuntu.sources
|
|
|
|
# Also make apt itself resilient — per-package retries + generous timeouts.
|
|
# Hetzner's mirror is reliable but individual packages can still blip; the
|
|
# retry config means a single failed fetch doesn't nuke the whole build.
|
|
RUN printf 'Acquire::Retries "5";\nAcquire::http::Timeout "30";\nAcquire::https::Timeout "30";\n' \
|
|
> /etc/apt/apt.conf.d/80-retries
|
|
|
|
# System deps (retry apt-get update + install as a unit — even Hetzner can blip).
|
|
# Includes xz-utils so the Node.js .tar.xz download below can decompress.
|
|
# python3: bin/gstack-jsonl-merge, gstack-brain-sync, gstack-detach, and other
|
|
# bash bins shell out to it (macOS ships python3; the base image doesn't).
|
|
# file: skill-validation's no-compiled-binaries-in-git check runs `file --mime-type`.
|
|
# poppler-utils: make-pdf's e2e gates hard-require pdftotext/pdffonts/pdfinfo in CI.
|
|
RUN for i in 1 2 3; do \
|
|
apt-get update && apt-get install -y --no-install-recommends \
|
|
git curl unzip xz-utils ca-certificates jq bc gpg python3 python3-venv file poppler-utils gcc libc6-dev && break || \
|
|
(echo "apt retry $i/3 after failure"; sleep 10); \
|
|
done \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
RUN python3 -m venv /tmp/gstack-ci-venv \
|
|
&& /tmp/gstack-ci-venv/bin/python -m pip --version \
|
|
&& rm -rf /tmp/gstack-ci-venv
|
|
|
|
# Direct builds produce the trusted CSO launcher and watchdog. Check the exact
|
|
# static-C capability here so the cached eval image cannot reach a slice without it.
|
|
RUN printf 'int main(void) { return 0; }\n' > /tmp/gstack-cso-cc-probe.c \
|
|
&& cc -std=c11 -static /tmp/gstack-cso-cc-probe.c -o /tmp/gstack-cso-cc-probe \
|
|
&& /tmp/gstack-cso-cc-probe \
|
|
&& rm -f /tmp/gstack-cso-cc-probe.c /tmp/gstack-cso-cc-probe
|
|
|
|
# GitHub CLI
|
|
RUN curl --retry 5 --retry-delay 5 --retry-connrefused -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \
|
|
| gpg --dearmor -o /usr/share/keyrings/githubcli-archive-keyring.gpg \
|
|
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \
|
|
| tee /etc/apt/sources.list.d/github-cli.list > /dev/null \
|
|
&& for i in 1 2 3; do \
|
|
apt-get update && apt-get install -y --no-install-recommends gh && break || \
|
|
(echo "gh install retry $i/3"; sleep 10); \
|
|
done \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Node.js 22 LTS (needed for claude CLI).
|
|
# Install from the official nodejs.org tarball instead of NodeSource's apt setup.
|
|
# NodeSource's setup_22.x script runs its own `apt-get update` + `apt-get install gnupg`,
|
|
# both of which depend on archive.ubuntu.com / security.ubuntu.com being reachable.
|
|
# Ubicloud CI runners frequently can't reach those mirrors (connection timeouts),
|
|
# and "gnupg" was renamed to "gpg" on Ubuntu 24.04 anyway, so NodeSource's script
|
|
# fails before it can add its own repo. Direct tarball download is network-simpler
|
|
# (one host: nodejs.org) and doesn't touch apt at all.
|
|
ENV NODE_VERSION=22.20.0
|
|
RUN curl --retry 5 --retry-delay 5 --retry-connrefused -fsSL "https://nodejs.org/dist/v${NODE_VERSION}/node-v${NODE_VERSION}-linux-x64.tar.xz" -o /tmp/node.tar.xz \
|
|
&& tar -xJ -C /usr/local --strip-components=1 --no-same-owner -f /tmp/node.tar.xz \
|
|
&& rm -f /tmp/node.tar.xz \
|
|
&& node --version \
|
|
&& npm --version
|
|
|
|
# Bun (install to /usr/local so non-root users can access it).
|
|
# The exact release archive, verified by SHA-256 before it is unpacked
|
|
# (#1706). The old `curl https://bun.sh/install | bash -s bun-vX.Y.Z` ran
|
|
# whatever script the network served at build time, and before that the
|
|
# `| BUN_VERSION=x.y.z bash` form silently installed latest on every rebuild.
|
|
# Hashes come from the release's SHASUMS256.txt. The image builds linux/amd64
|
|
# only (the Node.js tarball above is linux-x64); an architecture without a
|
|
# verified archive fails here. test/ci-image-cli-pin.test.ts pins this shape.
|
|
ARG BUN_VERSION=1.4.0
|
|
ARG TARGETARCH
|
|
ENV BUN_INSTALL="/usr/local"
|
|
RUN case "${TARGETARCH:-amd64}" in \
|
|
amd64) bun_target=linux-x64; bun_sha256=2d03fb5fb83ac8b567aca0a281b2ce1a1a19d488f56c2968d88c3f25e92fe452 ;; \
|
|
*) echo "Dockerfile.ci: no verified Bun ${BUN_VERSION} archive for ${TARGETARCH}" >&2; exit 1 ;; \
|
|
esac \
|
|
&& curl --retry 5 --retry-delay 5 --retry-connrefused -fsSL \
|
|
"https://github.com/oven-sh/bun/releases/download/bun-v${BUN_VERSION}/bun-${bun_target}.zip" -o /tmp/bun.zip \
|
|
&& echo "${bun_sha256} /tmp/bun.zip" | sha256sum -c - \
|
|
&& unzip -q /tmp/bun.zip -d /tmp/bun \
|
|
&& install -D -m 0755 "/tmp/bun/bun-${bun_target}/bun" "${BUN_INSTALL}/bin/bun" \
|
|
&& rm -rf /tmp/bun /tmp/bun.zip \
|
|
&& test "$(bun --version)" = "${BUN_VERSION}"
|
|
|
|
# Claude CLI — pinned to an EXACT version, same discipline as the bun pin
|
|
# above. The PTY harness (test/helpers/claude-pty-runner.ts) screen-scrapes
|
|
# this CLI's TUI (trust dialog, input prompt, spinner glyphs); an unpinned
|
|
# install rebuilt weekly rode the TUI wherever it drifted, and that drift
|
|
# broke the harness three separate times (welcome-screen wedge on 2.1.233,
|
|
# skillify HOME discovery on 2.1.237, guard/freeze hooks on 2.1.162).
|
|
# Bump deliberately, via a PR that runs the PTY gate against the new TUI.
|
|
# test/ci-image-cli-pin.test.ts fails the free suite if this pin is removed.
|
|
# 2.1.284 is the first pin that recognizes the eval model claude-fable-5-1.
|
|
# 2.1.251 already sent it effort "high", but ran it as an unknown model with
|
|
# a generic system prompt. Users on stable (2.1.280) and latest (2.1.285) get
|
|
# the fable-5-1 profile: its own system prompt, 64k max_tokens and per-turn
|
|
# effort, which only moves the same effort value into the conversation.
|
|
# Census 36626737820 on 2.1.284 looked slower mostly because the API was:
|
|
# its SDK-only judge evals, which never start this CLI, were 25% slower too.
|
|
RUN npm i -g @anthropic-ai/claude-code@2.1.284
|
|
|
|
# Codex CLI — exact pin, same discipline as the claude pin above, so the
|
|
# Codex eval files execute in CI instead of self-skipping. Installed OFF PATH:
|
|
# a skill's outside-voice probe (`command -v codex`) must keep reporting
|
|
# not_installed in every other paid case. The paid shard runner puts this
|
|
# directory on PATH only for the Codex files (scripts/lib/paid-cases.ts scopeCodexAccess).
|
|
# The image carries no credential: the eval workflows run `codex login
|
|
# --with-api-key` from the OPENAI_API_KEY secret in same-repo runs only
|
|
# (codex 0.160.0 does not read OPENAI_API_KEY from the environment).
|
|
ENV GSTACK_CI_CODEX_BIN_DIR=/opt/codex/bin
|
|
RUN npm i -g --prefix=/opt/codex @openai/codex@0.160.0 \
|
|
&& test -x /opt/codex/bin/codex \
|
|
&& ! command -v codex
|
|
|
|
# Playwright system deps (Chromium) — needed for browse E2E tests
|
|
RUN npx playwright install-deps chromium
|
|
|
|
# Linux has neither Helvetica nor Arial. make-pdf's print CSS stacks fall back
|
|
# to Liberation Sans (metric-compatible Arial clone, SIL OFL 1.1) so PDFs don't
|
|
# render in DejaVu Sans. playwright install-deps happens to pull this in today,
|
|
# but the dep is implicit and could change — install explicitly so upgrades
|
|
# can't silently regress rendering.
|
|
#
|
|
# Xvfb is also installed here so the browse --headed integration tests
|
|
# (headed-xvfb, headed-orphan-cleanup) can exercise the Linux container
|
|
# auto-spawn path on every CI run. Without Xvfb in the image, the most
|
|
# common production --headed path goes untested.
|
|
# fonts-noto-color-emoji: the make-pdf emoji render gate needs a color-emoji
|
|
# fallback font (mirrors make-pdf-gate.yml's Ubuntu setup step).
|
|
RUN for i in 1 2 3; do \
|
|
apt-get update && apt-get install -y --no-install-recommends fonts-liberation fonts-noto-color-emoji fontconfig xvfb x11-utils && break || \
|
|
(echo "fonts-liberation install retry $i/3"; sleep 10); \
|
|
done \
|
|
&& fc-cache -f \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Pre-install dependencies (cached layer — only rebuilds when package.json or
|
|
# bun.lock changes). Copy BOTH so install is deterministic and matches local
|
|
# resolution. Without bun.lock here, bun install resolved transitive deps
|
|
# differently in CI vs local (observed on v1.28.0.0: socks landed but
|
|
# smart-buffer + ip-address didn't make it into the cached node_modules).
|
|
# patches/ rides along: bun.lock's patchedDependencies (playwright-core
|
|
# windowsHide, v1.67) makes install fail without the patch files present —
|
|
# and the workflows' image-tag hash includes patches/** so editing a patch
|
|
# rebuilds this layer.
|
|
COPY package.json bun.lock /workspace/
|
|
COPY patches /workspace/patches
|
|
WORKDIR /workspace
|
|
RUN bun install --frozen-lockfile && rm -rf /tmp/*
|
|
|
|
# Install Playwright Chromium to a shared location accessible by all users
|
|
ENV PLAYWRIGHT_BROWSERS_PATH=/opt/playwright-browsers
|
|
RUN npx playwright install chromium \
|
|
&& chmod -R a+rX /opt/playwright-browsers
|
|
|
|
# Verify everything works
|
|
RUN bun --version && node --version && claude --version && /opt/codex/bin/codex --version && jq --version && gh --version \
|
|
&& python3 --version && command -v file && command -v pdftotext && command -v pdffonts && command -v pdfinfo \
|
|
&& npx playwright --version \
|
|
&& fc-match "Liberation Sans" | grep -qi "Liberation" \
|
|
|| (echo "ERROR: fonts-liberation not installed — make-pdf PDFs will render in DejaVu Sans" && exit 1)
|
|
|
|
# At runtime: checkout overwrites /workspace, but node_modules persists
|
|
# if we move it out of the way and symlink back
|
|
# Save node_modules + package.json snapshot for cache validation at runtime
|
|
RUN mv /workspace/node_modules /opt/node_modules_cache \
|
|
&& cp /workspace/package.json /opt/node_modules_cache/.package.json \
|
|
&& cp /workspace/bun.lock /opt/node_modules_cache/.bun.lock
|
|
|
|
# Claude CLI refuses --dangerously-skip-permissions as root.
|
|
# Create a non-root user for eval runs (GH Actions overrides USER, so
|
|
# the workflow must set options.user or use gosu/su-exec at runtime).
|
|
RUN useradd -m -s /bin/bash runner \
|
|
&& chmod -R a+rX /opt/node_modules_cache \
|
|
&& mkdir -p /home/runner/.gstack && chown -R runner:runner /home/runner/.gstack \
|
|
&& chmod 1777 /tmp \
|
|
&& mkdir -p /home/runner/.bun && chown -R runner:runner /home/runner/.bun
|