mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-31 22:14:25 +01:00
* docs: Oct 7 follow-up wave plan (pre-review) * docs: Oct 7 follow-up wave plan, reviewed through /autoplan * docs: Oct 7 wave plan, record final gate decisions (UC1 accepted, UC2 rejected) * docs: credit #1822 to @Mike-E-Log and #1743 to @habassa5 (v1.64.0.0) The v1.64.0.0 CHANGELOG entry credited the extension token fix (#1822) and the polyfill exited promise (#1743) to the wrong handle, and the extension-sender-auth test header repeated the first slip. Commit history is not rewritten. Fixes #3054. Co-authored-by: Mike Ilog <48659643+Mike-E-Log@users.noreply.github.com> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(context-restore): offer newer checkpoints from nested repos and task worktrees /context-restore could present an older checkpoint as the latest with no error in two layouts: 1. A session started in a non-git parent saved from a nested repo, so the checkpoint went to the nested repo's bucket. Restore in the parent now lists checkpoints of other buckets whose project_root lies below the cwd (BELOW_CWD; the 200 newest are read, and from $HOME only each project's newest). /context-save also leaves a JSON pointer (checkpoint path, bucket, project_root) in the starting directory's bucket when $CLAUDE_PROJECT_DIR is set and files elsewhere; restore lists those as POINTER, in or out of git. 2. A plan checkpoint on the main tree's branch shadowed a newer checkpoint saved in a ticket worktree. When another branch holds a newer checkpoint for the same task (a worktree below the main tree, read from the new worktree: stamp; equal normalized titles; or a shared #123 / ABC-123 token in titles or branch names), restore prints NEWER_TASK and the skill shows both and proposes the newer one. Generic titles (wip, checkpoint, save, notes, todo) never match on title alone, and the worktree rule applies only when restoring from the main tree, so sibling worktrees still never shadow each other. The branch-first ordering itself is unchanged. Refs #3065. Co-authored-by: Thomas Gernhardt <t.gernhardt@softwareprojekt.net> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(redact-prepush): list each MEDIUM finding with rule, file line and fix The pre-push hook printed only a MEDIUM count, which nobody could review without re-running the engine by hand. It now prints one line per finding: rule id, file and file line, and a fix, never the matched value. Line numbers come from the hunk headers (combined --cc headers included), so a finding in a later hunk or past a scan-chunk boundary points at the real file line. Findings from the per-commit catch-up path name the commit whose file line it is, so content removed before the pushed tip is still locatable. Plan items: CEO-7 (summary half), DX-9 (fix lines), ENG-8 (line mapping). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(redact-prepush): stop flagging your own email and already-public authors (#3060) The hook never passed the engine's selfEmail / repoPublicEmails, so every push that added the pusher's address, or a commit author's, raised a MEDIUM pii.email warning for an address the destination already shows. The hook now reads `git config user.email` and one bounded `git log` of author and committer addresses, raw and mailmapped (%ae %aE %ce %cE): the history of every pushed tip (those commits become public with the push), each remote tip being replaced, and, for a new branch or an unknown remote tip pushed to a configured remote, that remote's own tracking refs. A URL or unknown destination never borrows tracking refs, and another remote's history never counts. The read stops at 50,000 commits (the newest authors are used) and 5 seconds. An error or the timeout passes no history and, when an email finding is listed, prints one line saying suppression was limited for this push. The two inputs fail independently. HIGH findings and every other rule are untouched. Controls: a stranger's address, an address only another remote knows, a URL push next to a matching remote, and a first push of a repo with no history all still report. Plan items: A3, CEO-7, CEO-23 (A3 half), DX-9 (partial history), ENG-8 (history scope, mailmap). Co-authored-by: akmandhania <143462686+akmandhania@users.noreply.github.com> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * feat(redact-prepush): per-address allowlist for pii.email (gstack.redact.allowEmail) `git config --add gstack.redact.allowEmail <address>` allows that address for MEDIUM pii.email findings only, so a team alias or support address that has to be in the repo stops warning without reaching for --no-verify. It never affects HIGH findings or any other rule, and the pii.email fix line names it. Controls: a different address still reports, and a HIGH secret next to an allowed address still blocks. Plan item: DX-17. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(troubleshooting): pre-push pii.email findings, exemptions and allowlist Plan item: DX-12 (pre-push touchpoint). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(redact-prepush): assemble fixture addresses at runtime The email fixtures were literals, so this branch's own push listed 47 pii.email findings in its test file. Build them from a helper instead, the way the neighbouring tests build credential and IP fixtures. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(browse): offer the user's signed-in tab, name silent aside repl failures Rules (rendered into every browsing skill, kept short): - Rule 1: before the first openTab, offer the user's tabs on the target origin (title and origin only; the rest of listBrowserTabs() stays private) and attach only after the user confirms one. - Rule 4: a second sign-in wall after a confirmed sign-in means the session is bound to the tab or URL; offer that tab instead of another sign-in. - Rule 6 no longer suggests an early return; rule 7 says a fast [ok without GSTACK_STEP_OK is an abort. Cookbook (browse, devex-review): a "Silent failures" list (evaluate returns only serializable values, no top-level return, locator(sel).press(key), an empty DOM read is about the selector until the screenshot and the request's response say otherwise, read toggle state before clicking) and a signed-in tab recipe that prints only target-origin tabs and never closes the user's tab. Parity caps for design-consultation, land-and-deploy, qa and the browse skeleton are raised to the measured sizes, with notes. Refs #3063, #3064. Co-authored-by: Thomas Gernhardt <t.gernhardt@softwareprojekt.net> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * feat(codex): run gstack-codex-probe as a command, one subcommand per probe Generated skills now execute bin/gstack-codex-probe (select-model, check-auth, show-sandbox, check-sandbox, probe-model, check-version, show-first-use-notice, run-with-timeout, log-event, log-hang, help) and capture its KEY: value lines, so no helper state crosses into the calling shell and the preflight works from bash, zsh or sh. select-model keeps --model > GSTACK_CODEX_MODEL > config.toml precedence; run-with-timeout supervises with the existing gtimeout/timeout/ watchdog chain and returns 124 on a deadline; an unknown subcommand exits 64. Sourcing keeps working for skills rendered before the upgrade and prints one deprecation line naming /gstack-upgrade; it and the zsh self-locate block are removed in a release on or after 2026-10-21 (DX-18). Parity caps raised with measured notes for the longer capture lines. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * feat(gstack-post): one argv-only, redaction-scanned write path for PR and issue text bin/gstack-post posts a PR/MR or issue comment, a review-comment reply, a PR title or body, a new PR/MR or a new issue from agent-written files. Every value reaches gh/glab as its own argv element in --flag=<value> form or on stdin, never through a shell string, and a target is a number or a URL of this repository on the same host. The helper owns the redaction scan: it reads each file once, scans those exact bytes (visibility from config, then gh/glab; the pusher's own email suppressed) and sends the same bytes. HIGH is refused with exit 1 and no confirmation can post it. MEDIUM exits 2 with RULE:/LINE:/PART: lines and a TOKEN: digest of the bytes, destination, operation and findings; only --confirm <token> for those exact bytes posts, so any edit needs a new confirmation. gh/glab failures exit 3, usage errors 64. It also carries the projectCards REST fallback and the title read-back that /ship did by hand. The logic lives in lib/gstack-post.ts behind an injectable runner, so its tests run in the Windows free lane; the stub-gh executable test is POSIX. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(autoplan): guard verifies dispatches on Claude Code 2.1.29x (schema strip, unflushed call, named causes) Cause A: Claude Code journals the model's raw Agent input but hands PreToolUse the schema-parsed input; with the fork-subagent gate on (the interactive default in 2.1.292) run_in_background is dropped, so the journal record never deep-equaled the payload. nativeToolInput now drops only that documented strip for Agent. Agent input is bound on both paths: keys limited to prompt, description, subagent_type (general-purpose) and run_in_background; a model override or any other key is denied. Cause B: when a ready journal lacks the current call, the guard evaluates the payload placed after the last journaled record instead of polling for a record that may arrive only with the tool result. A phase transition counts a report only when a later journaled record follows it (and, on the journal path, it is not in the current call's message). One batch rule covers both paths; a background reviewer keeps the invocation armed until its completion notice. Every outcome now has a stable reason code, disposition and anchor (autoplan/bin/guard-reasons.ts). UC1: Claude-Code-caused states (unrecognized shape, journal lag, rewritten journal, oversized record, unchecked-version schema strip) allow with a visible warning; integrity failures stay denials. No denial advises a bare retry. Every guarded decision appends one content-free line to analytics/autoplan-guard.jsonl, trimmed to 1,000 lines past 256 KiB. Co-authored-by: Jared <182125291+yolo-jared@users.noreply.github.com> Co-authored-by: crblabs <293123359+crblabs@users.noreply.github.com> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(autoplan): publish each phase report with a no-op call; foreground requests are conditional Phase-close step 6 sends the report in a message whose only tool call is the Bash no-op ': autoplan-published <phase>', and step 7 makes the next guarded call in a later message, so the guard sees the report flushed before the next phase entry regardless of Claude Code's journal timing. Every site that asked for run_in_background: false now asks for it only when the Agent schema offers it and says how to recover when a dispatch runs in the background anyway (wait for its completion notice). Parity caps rise by the measured bytes. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * feat(ship,spec,review): PR and issue text goes out only through gstack-post /ship: Step 18 writes the title into an agent-written file. An existing PR's title is piped from gh/glab through gstack-pr-title-rewrite.sh --stdin (new), so neither the agent nor a shell string carries it; a new title is written with the file tool and its version prefix checked in the file. Step 19 composes the body as before and publishes with gstack-post pr-create, or pr-body then pr-title, which scan the exact bytes they send. The separate title and body scans, the hand-rolled REST fallback and the title self-check are gone (the helper does all three). Title preparation moved into the pr-body section, so the skeleton shrinks. /spec files its issue with gstack-post issue-create and drops the pre-issue re-scan; Greptile replies use gstack-post reply and pr-comment. scripts/resolvers/redact-doc.ts loses its PR and issue sinks, and a test keeps any template from rendering one again. Tests that pinned the old scan-then-gh wiring now pin the gstack-post wiring; the publication tests run the real compose and publish blocks against a stub gh with the real helper (HIGH in title or body refused, MEDIUM token flow). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ci: pin Claude Code 2.1.292 in the CI image 2.1.292 is the npm latest users run. Its interactive default turns on the fork-subagent gate, which strips run_in_background from the Agent schema and backgrounds reviewers; CI on 2.1.284 never saw the /autoplan guard break. The image tag is a content hash, so every eval workflow picks up the new image without a workflow edit. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(brain-sync): one drain lock for every writer to the artifacts repo gstack-artifacts-init, gstack-brain-restore, gstack-gbrain-source-wireup and the skill-start merge ran git in $GSTACK_HOME without the drain's lock, so the drain could never tell an abandoned .git/index.lock from a live one. They now take the same mkdir lock (bin/gstack-brain-lock.sh): the drain and the skill-start merge skip when it is held, user-run writers wait for it (GSTACK_BRAIN_LOCK_WAIT, default 120 s), and a child of the holder (the drain's daily worktree advance, restore's wireup) inherits it by token. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(brain-sync): hold back only the flagged file, keep coupled artifacts whole One file with secret-shaped text used to block every push, silently. The drain now attributes scanner hits to files in one pass over the staged diff (split on its diff --git headers), holds back each flagged file with the rest of its coupled group, unstages it, and requires the remaining staged diff to re-scan clean; anything short of that falls back to a full unstage and status blocked. Held files stay queued and are re-scanned every drain. - Status file gains held_count, drainable, last_drain_at, last_push_at and held[] (path, scanner rule, dependents, both fixes); new status held. - Coupled groups come from an explicit map (decisions.jsonl and decisions.active.json): a queued member stages the whole group, a group stages only when its snapshot matches its log in the index, a skipped member stops the whole group. - --unskip-file undoes --skip-file and queues the path again. - An index lock older than 10 minutes is removed under the drain lock; a younger one, a failed git add, a failed commit (hooks included) keep the queue and report status error. A verified empty staged diff is the only "nothing to commit". git add runs once, batched. - The scanner reads bytes, so undecodable content no longer crashes it into a false clean; matched text never reaches the status file. - An empty-queue run with unpushed sync commits stays push_failed. - Privacy-held duplicate records collapse to one per path; --status counts a spool too deep for a shell glob. - --attention prints skill start's fixed attention lines from the sync status table in the usage header. Co-authored-by: v639dragoon <matthewsbrickley@gmail.com> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * feat(skill-start): ARTIFACTS_SYNC attention lines for a stuck sync Skill start now prints one fixed attention line per sync-status-table row that applies (held, blocked, push_failed, error, unknown, stale-push, stale-drain), each naming gstack-brain-sync by absolute path. Values from the status file are clamped (status code to the table, counts to digits, timestamps to their shape), so the status message, file paths and scanner output never reach the line. The preamble tells the model to surface the line in one sentence and continue. Skill-start tests are generated from the table in bin/gstack-brain-sync, and docs/gbrain-sync-errors.md carries the same table. Co-authored-by: v639dragoon <matthewsbrickley@gmail.com> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(outside-review): severity words in label position count as findings The Oct 7 /autoplan run's Codex CEO and DX reviews labeled seven and eight findings High/Medium without [Pn] tags, and the classifier printed VERDICT: clean / FINDINGS: none for both. lib/outside-review-result.ts now reads severity words in label position (Severity: High, a leading High: / High — / [High], **High**, a table cell): critical/high block like P0/P1, medium/low are advisory. Words in prose (high-level, low-risk, no critical findings) never count. A review or structured response with neither tags, labels nor an explicit no-findings statement is unverified, never clean (DX-11, ENG-10). Design-direction proposals move to a new proposal gate so they keep needing only their Recommendation line, and review prompts ask for a severity per finding. Fixtures reproduce the CEO/DX label shapes, the eng [P1] output, a table and prose negative controls; the stored Codex panel keeps its verdicts. docs/troubleshooting.md explains how the verdict is read. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * feat: gstack-doctor readiness report and setup --status Codex row (B5) bin/gstack-doctor prints one row per check without starting a skill: install root (the registry rows behind ./setup --status), state root, Bun against the floor, hook parse check, Codex mode and self-locate via gstack-codex-probe subcommands, the cached model probe with its age (--live runs the paid probe-model), artifacts-sync health from the status file with the status code clamped, the browse bundle, Claude Code version, the largest session journal and the last five schema-1 guard reason codes. Rows are ok, warn, not configured or fail; only fail exits non-zero; fix lines name helpers by absolute install-root path. ./setup --status gains a read-only Codex row (bin/gstack-codex-status.sh, shared with the doctor) and ends with the doctor's absolute path. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: bug-report template asks for doctor output; README troubleshooting entry; doctor test in the Windows lane The template gives the Claude Code doctor path and a lookup for other hosts and names no bare gstack- command (asserted in test/gstack-doctor.test.ts). The doctor test runs every script through explicit bash argv, so it joins the curated Windows free subset. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(troubleshooting): readiness check section points at gstack-doctor Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(question-tuning,plan-tune,gbrain): free text reaches helpers only through agent-written files Question tuning no longer pipes the question summary through printf: a registered id needs no summary, and for an unregistered id the agent writes the summary to .gstack/tmp/qt.txt and passes --summary-file (new). The helper resolves a relative path against the git top level, removes scratch files under .gstack/tmp, and asks normally when the file is missing. /plan-tune tune events carry the user's own words again: they go into a file the agent writes and reach gstack-question-preference --write through --free-text-file (new), still behind the injection filter and the user-origin gate. docs/gbrain-write-surfaces.md drops its heredoc templates for the agent-written page file on stdin, matching the per-skill save block, and the generic save block stops showing a quoted --content argument. The tuning prose was tightened so every preamble stays within its size cap. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(evals): daily autoplan-schema-canary on the latest Claude Code A periodic rule case installs the latest published Claude Code with install scripts off, verifies registry signatures, and runs one headless session with the fork-subagent gate on whose PreToolUse hook records an Agent and a Read payload. Each payload must match its journal record under the guard's own comparison, keep the journal's key set apart from the documented run_in_background strip, and stay inside the Agent allowlist; a background Agent's completion notice must keep the shape the guard reads. evals-periodic.yml runs it daily on its own schedule and concurrency group (read-only token); the weekly census also includes it. A free test proves the comparison fails on an injected strip. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(doctor): read the artifacts sync held count from held_count, the field the drain writes * fix(artifacts sync): the init and restore pre-commit hooks catch Bearer/Basic/Token-prefixed JSON auth headers like the drain does * feat(greptile-early): detect Greptile, ask once on public repos, open the early draft, wait for the review bin/gstack-greptile-early backs /ship's early Greptile PR: - detect: a root .greptile/ folder (which takes precedence), greptile.json, or a Greptile comment on the repo in the last 90 days; triggerOnDrafts from the effective root config; repo visibility; public-repo consent. It prints the signal, that /ship will push and open a draft now, the 10-minute wait cap and the absolute gstack-config command that turns it off. - consent yes|no: the public-repo answer, remembered per repo in the project's state. - open: opens the draft through gstack-post with a body saying /ship opened it early, and posts one @greptileai comment through gstack-post when the config does not set triggerOnDrafts: true. An open PR is left alone. - wait: polls every 30 s for a completed Greptile check run, a Greptile review of the head commit or a summary naming it; each call returns within about 90 s and reports timeout 10 minutes after the PR opened. gstack-config gains ship_greptile_early (default true, invalid values rejected): on automatically whenever the repo uses Greptile, off per user. The logic is in lib/greptile-early.ts behind an injectable runner, so its fixture-repo tests run in the Windows lane; an end-to-end test drives the real helper and gstack-post against a stub gh. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * feat(ship): early Greptile PR at Step 6.5, bounded wait at Step 10 (#3020) When the repo uses Greptile, /ship pushes and opens a draft PR right after its free tests pass (new lazy section greptile-early, Step 6.5), so Greptile reviews while Steps 7-9 run. A public repo asks once, remembered per repo; declining keeps today's order. Step 10 waits for that review with gstack-greptile-early wait and takes the existing unavailable route on a timeout, never a claim of zero comments. Step 18 replaces the provisional title, Step 19 marks the draft ready unless a draft was requested, and a /ship that stops after Step 6.5 leaves the draft open with one comment saying why. ship's union cap rises to 1.5275 with the measured note (B4 +836 bytes, B2 +4,732 bytes); the step-numbering guard allows 6.5; ship goldens are refreshed for B4 and B2. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(autoplan): the publication no-op is 'true autoplan-published <phase>' Measured on Claude Code 2.1.292 in a default-settings interactive session: a bare ':' Bash command asks for approval ('This command requires approval'), and after a background reviewer's completion notice starts a new turn the skill's allowed tools no longer cover it, so the phase-close no-op prompted the user. 'true' with arguments runs without approval and prints nothing. The guard only needs a later journaled record, so the command changes everywhere it is named. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(evals): live autoplan-guard-pty case on the pinned Claude Code A gate-tier rule case (one trial, haiku) drives an interactive PTY session with default settings through a project skill that carries /autoplan's own hook block: init, Phase 1 entry, a CEO reviewer dispatch that sends run_in_background, a prepared close packet, the report published with the no-op, and the Phase 2 entry in a later message. It reads outcomes from the guard decision log (three verified allows) and fails if the no-op raises a permission card; other cards are approved and printed. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(autoplan): bounded, append-tolerant owned-journal read (A2) The publication guard no longer reads the whole parent journal or refuses it at 32 MiB. lib/claude-owned-journal.ts streams the bytes present at open to the last complete line (pass one), keeps a content-free index per record and decides ownership, ancestry and causal order on it with the same checks and reason codes as before. Pass two reloads in full only what the evaluation reads (autoplan/bin/guard-journal.ts): every literal init and its result, the current call, and inside the invocation window (from the latest human turn at or before the bound chain's first init) guarded and close-packet Read results, structured Bash results, error results, assistant text after the first close Read and active-plan mutations, plus the earlier Read a dedup reply cites even before the window (ENG-4). - lib/claude-journal-records.ts: per-record event extraction, record metadata and the causal-order checks, shared by readPlanCountTranscript and the bounded reader. The batch reader keeps its whole-file read and cap. - Appends after open never fail a read; same device and inode and no shrink are required (identity). A second read in one hook invocation must hash bytes [0, first size) the same, or it is `rewritten` (UC1 unverified); the unrecognized-shape double read uses that prefix hash. `changing` is removed (REMOVED_CODES; its anchor stays). - OWNED_RECORD_MAX_BYTES (32 MiB): only one record over it is too_large. OWNED_RETAINED_MAX_BYTES (32 MiB): over it is the new oversized_invocation fallback denial. GSTACK_TRANSCRIPT_TEST_MAX_BYTES and GSTACK_TRANSCRIPT_TEST_RETAINED_BYTES can only lower them. - ENG-5: evaluation groups uses and results by tool id once (no per-tool rescans). CEO-25 / ENG-5 spike (120 MiB synthetic, 31,242 records, 4 vCPU Linux, Bun 1.4.2): a full streaming rescan (parse every line, small index, sha256) took 254-326 ms at 147-161 MiB peak RSS; a pass-one index cache under the state root (load cached index, verify a 64 KiB tail hash, parse the tail) took 26-29 ms at 47 MiB. The rescan meets the 1 s budget, so the cache is not adopted: it would add trusted persistent state and invalidation for no budget need. A backward tail read cannot prove competing_root or cycle, so it was not a candidate. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(autoplan): journal-read benchmark and its dispatch-only workflow (CEO-2, CEO-6) scripts/measure-journal-read.ts builds two synthetic 120 MiB parent journals around a real /autoplan invocation (mixed: ~28k records, most bytes inside the invocation; large-invocation: ~89k records with a current invocation of many small records), keeps appending while the guard reads, and measures one hook call in a fresh process (wall ms, peak RSS). The workflow runs it on ubuntu-latest and macos-latest on workflow_dispatch only. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(evals): autoplan-long-session, a resumed >100 MiB session enters Phase 1 CEO-15 with ENG-6's observer: a real session is compacted, its journal padded past 100 MiB with owned records before the compact boundary, and the resumed interactive session runs the scripted phase boundary; the guard decision log must show a verified Phase 1 allow. A free test proves that observer on a padded journal through the lower-only bound seams (the PTY counting reader refuses the same journal), with a sidechain control that must deny. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * v1.91.34.0: CHANGELOG, TODOS, version; harness decision for test-pr-profile; ship title-step test reads its new section * docs(changelog): use an example.com address in the allowEmail example * test: assemble GitHub-token fixtures at runtime in the new brain-sync and skill-start tests (CI credential scan) * fix: integration repairs across the wave (Step 6.5 fence, printed helper paths on Windows, probe usage without a heredoc) and tests pinned to the moved title and gstack-post contracts * test(doctor): the healthy fixture builds browse.exe on Windows, the name setup uses there * test(shared-libs): recognize Claude Code 2.1.292's internal git probe prefix (format.pretty and per-hook disables), with its CI receipt * fix(ship-measure): report lists v1 measurements as an older format instead of crashing --------- Co-authored-by: Mike Ilog <48659643+Mike-E-Log@users.noreply.github.com> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> Co-authored-by: Thomas Gernhardt <t.gernhardt@softwareprojekt.net> Co-authored-by: akmandhania <143462686+akmandhania@users.noreply.github.com> Co-authored-by: Jared <182125291+yolo-jared@users.noreply.github.com> Co-authored-by: crblabs <293123359+crblabs@users.noreply.github.com> Co-authored-by: v639dragoon <matthewsbrickley@gmail.com>
556 lines
32 KiB
TypeScript
556 lines
32 KiB
TypeScript
import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
|
|
import * as fs from 'node:fs';
|
|
import * as os from 'node:os';
|
|
import * as path from 'node:path';
|
|
import { spawnSync } from 'node:child_process';
|
|
import { runGeneration } from '../scripts/gen-skill-docs';
|
|
import { docsDispatchIndex, extractDocsDispatch, parseDocsCompletion, vetDocsCompletion } from './helpers/docsync-contract';
|
|
import { fixtureDocs, repoSnapshot, changedFiles, gitAt, DOC_PATH, preserveDocsEvidence } from './helpers/docsync-fixture';
|
|
import type { SkillTestResult } from './helpers/session-runner';
|
|
import { observeDocsWrites, docsWriteFailures, docsCommandAllowed, docsPreambleCommands, docsCompletedRead } from './helpers/docsync-observer';
|
|
import { docsActorCommand, docsActorHook, installDocsActor, type DocsActorState, type DocsFault } from './helpers/docsync-fault-actor';
|
|
import { docsActorVerdict } from './helpers/docsync-fault-eval';
|
|
import { expectMentions, expectTokens } from './helpers/prompt-structure';
|
|
|
|
const ROOT = path.join(import.meta.dir, '..');
|
|
const read = (p: string) => fs.readFileSync(path.join(ROOT, p), 'utf8');
|
|
let generated: string;
|
|
|
|
beforeAll(async () => {
|
|
generated = fs.mkdtempSync(path.join(os.tmpdir(), 'docsync-render-'));
|
|
for (const host of ['claude', 'codex', 'factory'] as const) {
|
|
const result = await runGeneration({ host, outputRoot: generated, contentLinkRoot: null, log: () => {} });
|
|
expect(result.exitCode).toBe(0);
|
|
}
|
|
});
|
|
|
|
afterAll(() => fs.rmSync(generated, { recursive: true, force: true }));
|
|
|
|
describe('pre-publication documentation lifecycle', () => {
|
|
test('final verification orders build, bounded documentation refresh, freeze and evidence', () => {
|
|
const body = read('ship/SKILL.md.tmpl');
|
|
const gate = body.slice(body.indexOf('## Step 16:'), body.indexOf('## Step 17:'));
|
|
const steps = ['### 1. Finish writers and prepare outputs', '### 2. Choose the change route',
|
|
'### 3. Resolve documentation freshness', '### 4. Verify the frozen candidate', '### 5. Report, then push'];
|
|
const positions = steps.map(step => gate.indexOf(step));
|
|
expect(positions.every(position => position >= 0)).toBe(true);
|
|
expect(positions).toEqual([...positions].sort((a, b) => a - b));
|
|
const recovery = body.slice(body.indexOf('### 3. Resolve documentation freshness'), body.indexOf('### 4. Verify the frozen candidate')).replace(/\s+/g, ' ');
|
|
expect(recovery).toContain('Validate the outcome before Step 15');
|
|
expect(recovery).toContain('Never run a third audit');
|
|
expect(body.replace(/\s+/g, ' ')).toContain('its initial-plus-ONE limit never resets');
|
|
expect(gate.replace(/\s+/g, ' ')).toContain('Docs, TODO edits, new/generated tests and fixes make evidence STALE');
|
|
const docs = read('ship/sections/documentation.md.tmpl');
|
|
expectMentions(docs.replace(/\s+/g, ' '), [['never', 'attempt', 'changes']], 'docs.replace(/\s+/g, )');
|
|
expect(docs.replace(/\s+/g, ' ')).toContain('Otherwise STOP before commit/publication and do not launch another child');
|
|
});
|
|
|
|
test('dispatch is carved before commit and final verification on every host', () => {
|
|
const claude = fs.readFileSync(path.join(generated, 'ship/SKILL.md'), 'utf8');
|
|
const marker = '## Step 14.5: Documentation audit (every ship)';
|
|
expect(claude.indexOf(marker)).toBeGreaterThan(0);
|
|
expect(claude.indexOf('ship/sections/documentation.md', claude.indexOf(marker))).toBeLessThan(claude.indexOf('## Step 15:'));
|
|
expect(claude).not.toContain('Dispatch /document-release as a subagent');
|
|
// C4: ship is carved on external hosts too; the pointer is relative to the installed skill.
|
|
for (const dir of ['.agents/skills/gstack-ship', '.factory/skills/gstack-ship']) {
|
|
const body = fs.readFileSync(path.join(generated, dir, 'SKILL.md'), 'utf8');
|
|
const pointer = body.indexOf('`sections/documentation.md` relative to the installed `gstack-ship` SKILL.md directory', body.indexOf(marker));
|
|
expect(pointer).toBeGreaterThan(body.indexOf(marker));
|
|
expect(pointer).toBeLessThan(body.indexOf('## Step 15:'));
|
|
expect(body).not.toContain('Dispatch /document-release as a subagent');
|
|
expect(fs.readFileSync(path.join(generated, dir, 'sections/documentation.md'), 'utf8')).toContain('Dispatch /document-release as a subagent');
|
|
expect(body.indexOf('## Step 16:')).toBeLessThan(body.indexOf('## Step 17:'));
|
|
}
|
|
});
|
|
|
|
test('the actual dispatch has foreground, marker, typed result and ownership guards', () => {
|
|
const body = fs.readFileSync(path.join(generated, 'ship/sections/documentation.md'), 'utf8');
|
|
expect(body).toContain('run_in_background: false');
|
|
expect(body).toContain('subagent_type: "general-purpose"');
|
|
const prompt = extractDocsDispatch(body);
|
|
for (const marker of ['GSTACK_SESSION_KIND=spawned', 'LAST nonempty line', 'schema_version', 'read-only',
|
|
'no Git mutation', 'CHANGELOG or TODOS mutation', 'never auto-approve', 'staged, unstaged and selected new']) expect(prompt).toContain(marker);
|
|
expect(docsDispatchIndex([{ tool: 'Agent', input: { prompt } }])).toBe(0);
|
|
expect(docsDispatchIndex([{ tool: 'Agent', input: { prompt: body } }])).toBe(-1);
|
|
expect(() => extractDocsDispatch('no markers')).toThrow();
|
|
});
|
|
|
|
test('documentation preflight follows the installed host layout', () => {
|
|
const claude = fs.readFileSync(path.join(generated, 'ship/sections/documentation.md'), 'utf8');
|
|
for (const section of ['audit-scope', 'release-body']) {
|
|
expect(fs.existsSync(path.join(generated, `document-release/sections/${section}.md`))).toBe(true);
|
|
}
|
|
for (const host of ['.agents', '.factory']) {
|
|
const ship = fs.readFileSync(path.join(generated, host, 'skills/gstack-ship/sections/documentation.md'), 'utf8');
|
|
const directory = path.join(generated, host, 'skills/gstack-document-release');
|
|
const document = fs.readFileSync(path.join(directory, 'SKILL.md'), 'utf8');
|
|
expect(document).toContain('# Documentation scope and discovery');
|
|
expect(document).toContain('## Step 2: Per-File Documentation Audit');
|
|
expect(document).toContain('## Ship-owned documentation mode');
|
|
expect(fs.existsSync(path.join(directory, 'sections/audit-scope.md'))).toBe(false);
|
|
expect(fs.existsSync(path.join(directory, 'sections/release-body.md'))).toBe(false);
|
|
}
|
|
});
|
|
|
|
test('failure is visible, bounded and settled before another writer', () => {
|
|
const body = read('ship/sections/documentation.md.tmpl').replace(/\s+/g, ' ');
|
|
for (const text of ['Terminal completion or confirmed termination is sufficient', 'the request alone is insufficient',
|
|
'an initial audit plus ONE repair/re-audit', 'specific named documentation risk', 'Preserve partial',
|
|
'Compare actual changes against the candidate',
|
|
'enforcing prompt/audit-scope permissions and protected-file exclusions',
|
|
'HEAD and index must be unchanged, existing dirty/untracked user content preserved',
|
|
'changed paths exactly `files_updated`. Reject any read-only write',
|
|
'Only verified permitted child edits may differ. Other edits or base changes make the audit stale',
|
|
'Later changes require the remaining re-audit or a risk decision',
|
|
'never silently refreshed hashes']) expect(body).toContain(text);
|
|
expect(body).not.toContain('block /ship on subagent failure');
|
|
});
|
|
|
|
test('stale detection does not spend the remaining audit, but repair and inline takeover do', () => {
|
|
const body = read('ship/sections/documentation.md.tmpl').replace(/\s+/g, ' ');
|
|
for (const text of ['Increment before each launch or inline takeover',
|
|
'including failed launches',
|
|
'A stale snapshot is neither a new attempt nor a current audit',
|
|
'inline work follows the same validation gates',
|
|
'never a third attempt, even after Step 16 changes',
|
|
'Confirm the child stopped before any repair, retry, inline takeover or other writer',
|
|
'request stop and inspect its status; the request alone is insufficient',
|
|
'If an attempt remains and either the audited inputs changed or a concrete launch/input/permission correction or reviewed patch repair is available',
|
|
'using current inputs and a fresh id/snapshot, run the remaining attempt, then validate it through Parent processing',
|
|
'Otherwise STOP before commit/publication',
|
|
'do not launch another child']) expect(body).toContain(text);
|
|
});
|
|
|
|
test('PR creation and reruns keep current and blocked audits visible', () => {
|
|
const body = read('ship/sections/pr-body.md.tmpl');
|
|
// Oct 7 wave B4: the title travels in Step 18's agent-written file and both
|
|
// PR paths publish through gstack-post, which scans the exact bytes it sends.
|
|
expect(body).toContain('`grep -Eq \'^v<new-version>( |$)\' "$TITLE_FILE"` and never publish an unprefixed title');
|
|
expect(body).toContain('gstack-pr-title-rewrite.sh <new-version> --stdin > "$TITLE_FILE"');
|
|
expect(body).toContain('gstack-post pr-create --base <base> --title-file "$TITLE_FILE"');
|
|
expect(body).toContain('gstack-post pr-title <pr-number> --title-file "$TITLE_FILE"');
|
|
expect(body).toContain('`gstack-post` detects GitHub or GitLab from the remote');
|
|
expect(body).not.toContain('Dispatch /document-release');
|
|
expectMentions(body, [['never', 'section', 'another']], 'body');
|
|
expect(body).toContain('gstack-post pr-body <pr-number> --body-file "${PR_BODY_FILE:?restore the composed body path}"');
|
|
expect(body).toContain('gstack-redact --from-file "$PR_BODY_FILE" --auto-redact');
|
|
expect(read('ship/SKILL.md.tmpl')).toContain('existing PRs and docs-only changes');
|
|
expect(read('ship/sections/apple-release.md.tmpl')).toContain('read-only');
|
|
expect(read('ship/sections/apple-release.md.tmpl')).toContain('ship/sections/documentation.md');
|
|
});
|
|
|
|
test('ship-owned documentation_section carries its status so /ship embeds it unchanged', () => {
|
|
// ci-36641820398-1-gate-census-3 ship-docsync-completion: the section had scope, health and debt but no result,
|
|
// so the parent spliced a Status line into it and the report no longer contained the returned section.
|
|
const scope = read('document-release/sections/audit-scope.md.tmpl').replace(/\s+/g, ' ');
|
|
expectTokens(scope, ['`**Status:**`', '`status`'], 'scope');
|
|
expect(read('ship/sections/pr-body.md.tmpl')).toContain("Embed Step 14.5's vetted nonempty `documentation_section`");
|
|
});
|
|
|
|
test('a missing installed document-release section blocks before launch', () => {
|
|
// ci-36709485593-1-eval-slices-2 ship-docsync-missing-asset: audit-scope.md was absent, but the parent saw the
|
|
// SKILL.md "Ship-owned documentation mode" heading, read the gate as satisfied and dispatched.
|
|
const gate = read('ship/sections/documentation.md.tmpl').replace(/\s+/g, ' ');
|
|
expectTokens(gate, ['`Ship-owned documentation mode`'], 'gate');
|
|
expectMentions(gate, [['never', 'substitute', 'missing']], 'gate');
|
|
});
|
|
|
|
test('nested authored discovery and standalone protections survive', () => {
|
|
const skill = read('document-release/SKILL.md.tmpl') + read('document-release/sections/audit-scope.md.tmpl');
|
|
expect(skill).not.toContain('find . -maxdepth 2');
|
|
for (const text of ['declared documentation roots', '.tmpl', 'generated output', 'Read before editing',
|
|
'Ship-owned documentation mode', 'standalone branch gate']) expect(skill).toContain(text);
|
|
const body = read('document-release/sections/release-body.md.tmpl');
|
|
for (const text of ['never `git add -A`', 'Never regenerate a CHANGELOG entry',
|
|
'body-original.md', 'UNTRUSTED TRACKER CONTENT', 'gstack-redact --from-file']) expect(body).toContain(text);
|
|
const versionStep = body.slice(body.indexOf('## Step 8'), body.indexOf('\n## ', body.indexOf('## Step 8') + 1));
|
|
expect(versionStep).toMatch(/ask[^\n]*before[^\n]*VERSION/i);
|
|
expect(body).toMatch(/does not rewrite, replace, or regenerate CHANGELOG/i);
|
|
});
|
|
});
|
|
|
|
const current = {
|
|
schema_version: 1, audit_id: 'audit-1', status: 'current', files_updated: [],
|
|
files_reviewed: ['docs/reference/cli.md.tmpl'], documentation_section: 'Current — CLI reference reviewed.',
|
|
blockers: [], decisions: [],
|
|
};
|
|
const evidence = {
|
|
settled: true, markerSeen: true, headUnchanged: true, indexUnchanged: true,
|
|
candidateUnchanged: true, readOnly: false, changedPaths: [] as string[],
|
|
allowedDocs: ['docs/reference/cli.md.tmpl'],
|
|
};
|
|
|
|
describe('completion observer negative controls', () => {
|
|
test('updated, current, and partial blocked results retain their actual meaning', () => {
|
|
for (const value of [current,
|
|
{ ...current, status: 'updated', files_updated: evidence.allowedDocs },
|
|
{ ...current, status: 'blocked', files_updated: evidence.allowedDocs, blockers: ['Security narrative requires a decision'] }]) {
|
|
const parsed = parseDocsCompletion(`Audit complete\n${JSON.stringify(value)}\n`, 'audit-1');
|
|
vetDocsCompletion(parsed, { ...evidence, changedPaths: value.files_updated });
|
|
expect(parsed.status).toBe(value.status);
|
|
}
|
|
});
|
|
|
|
test.each([
|
|
['old shape', { files_updated: [], commit_sha: null, pushed: false, documentation_section: null }],
|
|
['old version', { ...current, schema_version: 0 }],
|
|
['late callback', { ...current, audit_id: 'abandoned' }],
|
|
['missing key', { ...current, decisions: undefined }],
|
|
['null summary', { ...current, documentation_section: null }],
|
|
['empty summary', { ...current, documentation_section: '' }],
|
|
['error field', { ...current, error: 'preamble failed' }],
|
|
['false current', { ...current, files_updated: evidence.allowedDocs }],
|
|
['empty updated', { ...current, status: 'updated' }],
|
|
['false blocked', { ...current, status: 'blocked' }],
|
|
['invalid array', { ...current, files_reviewed: [1] }],
|
|
['path escape', { ...current, files_reviewed: ['../README.md'] }],
|
|
['duplicate path', { ...current, files_reviewed: ['README.md', 'README.md'] }],
|
|
])('rejects %s', (_name, value) => {
|
|
expect(() => parseDocsCompletion(JSON.stringify(value), 'audit-1')).toThrow();
|
|
});
|
|
|
|
test.each(['{broken', 'agent_id: 123', `${JSON.stringify(current)}\nDone`, `${JSON.stringify(current)}\n\`\`\``])('rejects malformed or non-last-line completion %s', output => {
|
|
expect(() => parseDocsCompletion(output, 'audit-1')).toThrow();
|
|
});
|
|
|
|
test.each([
|
|
['running after timeout', { settled: false }], ['missing marker', { markerSeen: false }],
|
|
['unexpected commit', { headUnchanged: false }], ['staged edits', { indexUnchanged: false }],
|
|
['edit before callback', { candidateUnchanged: false }], ['edit after callback', { candidateUnchanged: false }],
|
|
['unreported partial edit', { changedPaths: evidence.allowedDocs }],
|
|
])('blocks %s', (_name, overrides) => {
|
|
expect(() => vetDocsCompletion(parseDocsCompletion(JSON.stringify(current), 'audit-1'), { ...evidence, ...overrides })).toThrow();
|
|
});
|
|
|
|
test.each(['VERSION', 'CHANGELOG.md', 'TODOS.md', 'package.json', 'nested/manifest.json', 'docs/generated.md', 'app.ts'])('rejects actual protected or unapproved change %s', changed => {
|
|
const result = parseDocsCompletion(JSON.stringify({ ...current, status: 'updated', files_updated: [changed] }), 'audit-1');
|
|
expect(() => vetDocsCompletion(result, { ...evidence, changedPaths: [changed] })).toThrow();
|
|
});
|
|
|
|
test('read-only store mode cannot accept an updated document', () => {
|
|
const result = parseDocsCompletion(JSON.stringify({ ...current, status: 'updated', files_updated: evidence.allowedDocs }), 'audit-1');
|
|
expect(() => vetDocsCompletion(result, { ...evidence, readOnly: true, changedPaths: evidence.allowedDocs })).toThrow();
|
|
});
|
|
});
|
|
|
|
describe('native docs fixture preflight', () => {
|
|
test('staged, unstaged and new content are distinct from HEAD and user content is excluded', () => {
|
|
const fixture = fixtureDocs('updated', generated);
|
|
try {
|
|
expect(gitAt(fixture.repo, 'diff', '--cached', '--name-only')).toBe('app.ts');
|
|
expect(gitAt(fixture.repo, 'diff', '--name-only')).toBe('README.md');
|
|
const candidate = JSON.parse(fs.readFileSync(fixture.candidate, 'utf8'));
|
|
expect(candidate.selected_paths).toContain('options.ts');
|
|
expect(candidate.selected_paths).toContain(DOC_PATH);
|
|
expect(candidate.selected_paths).not.toContain('personal-note.txt');
|
|
expect(gitAt(fixture.repo, 'show', 'HEAD:app.ts')).toContain('text');
|
|
expect(fs.readFileSync(path.join(fixture.repo, 'app.ts'), 'utf8')).toContain('json');
|
|
fs.writeFileSync(path.join(fixture.repo, 'VERSION'), '9.9.9.9\n');
|
|
expect(changedFiles(fixture.before, repoSnapshot(fixture.repo))).toEqual(['VERSION']);
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('docs-only repeat fixture is actually committed and already pushed', () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
try {
|
|
expect(gitAt(fixture.repo, 'rev-parse', 'HEAD')).toBe(gitAt(fixture.repo, 'rev-parse', '@{u}'));
|
|
expect(gitAt(fixture.repo, 'diff', 'main...HEAD', '--name-only')).toBe(DOC_PATH);
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('store fixture stays on base and supplies a read-only candidate', () => {
|
|
const fixture = fixtureDocs('store', generated);
|
|
try {
|
|
expect(gitAt(fixture.repo, 'branch', '--show-current')).toBe('main');
|
|
expect(JSON.parse(fs.readFileSync(fixture.candidate, 'utf8')).mode).toBe('read-only');
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test.each(['before callback', 'after callback'])('a real late content edit invalidates evidence %s', order => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
try {
|
|
const completion = () => parseDocsCompletion(JSON.stringify(current), 'audit-1');
|
|
let result = order === 'after callback' ? completion() : undefined;
|
|
fs.appendFileSync(path.join(fixture.repo, 'app.ts'), 'export const lateChange = true;\n');
|
|
result ??= completion();
|
|
const after = repoSnapshot(fixture.repo);
|
|
expect(() => vetDocsCompletion(result!, {
|
|
...evidence, candidateUnchanged: changedFiles(fixture.before, after).length === 0,
|
|
})).toThrow('stale candidate');
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('snapshotting rejects a documentation symlink outside the product root', () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
try {
|
|
fs.writeFileSync(path.join(fixture.home, 'outside.md'), 'outside');
|
|
fs.symlinkSync(path.join(fixture.home, 'outside.md'), path.join(fixture.repo, 'escape.md'));
|
|
expect(() => repoSnapshot(fixture.repo)).toThrow('fixture path escaped');
|
|
expect(fs.readFileSync(path.join(fixture.home, 'outside.md'), 'utf8')).toBe('outside');
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('the real preamble marker and private fixture evidence survive cleanup', () => {
|
|
const fixture = fixtureDocs('risky', generated);
|
|
const evidenceRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'docs-evidence-'));
|
|
let retained: string | undefined;
|
|
try {
|
|
const result = spawnSync('bash', [path.join(fixture.skills, 'bin/gstack-skill-start'), '--skill', 'document-release'], {
|
|
cwd: fixture.repo, encoding: 'utf8', timeout: 30000,
|
|
env: { ...process.env, ...fixture.env, GSTACK_SESSION_KIND: 'spawned' },
|
|
});
|
|
expect(result.status).toBe(0);
|
|
expect(result.stdout).toContain('SESSION_KIND: spawned');
|
|
retained = preserveDocsEvidence(fixture, { output: 'preflight', toolCalls: [] } as unknown as SkillTestResult,
|
|
`docs-free-${path.basename(fixture.home)}`, 'fixture', {}, evidenceRoot);
|
|
fixture.clean();
|
|
expect(fs.existsSync(retained)).toBe(true);
|
|
expect(fs.statSync(retained).mode & 0o777).toBe(0o600);
|
|
expect(JSON.parse(fs.readFileSync(retained, 'utf8')).before.contents['SECURITY.md']).toBeDefined();
|
|
} finally {
|
|
fixture.clean();
|
|
fs.rmSync(evidenceRoot, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|
|
|
|
(process.platform === 'linux' ? describe : describe.skip)('independent docs write observer', () => {
|
|
test('read-only native Git commands generate no forbidden writes', async () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
const observer = await observeDocsWrites(fixture);
|
|
try {
|
|
gitAt(fixture.repo, 'status', '--porcelain');
|
|
gitAt(fixture.repo, 'diff', 'main...HEAD');
|
|
expect(docsWriteFailures(observer.stop(), [])).toEqual([]);
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test.each(['write-revert', 'rename-revert', 'delete-recreate', 'git-config-revert', 'shell-write-revert'])('rejects transient %s despite restored bytes', async operation => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
const file = path.join(fixture.repo, operation === 'git-config-revert' ? '.git/config' : 'app.ts');
|
|
const before = fs.readFileSync(file);
|
|
const observer = await observeDocsWrites(fixture);
|
|
try {
|
|
if (operation === 'rename-revert') {
|
|
fs.renameSync(file, file + '.moved');
|
|
fs.renameSync(file + '.moved', file);
|
|
} else if (operation === 'delete-recreate') {
|
|
fs.unlinkSync(file);
|
|
fs.writeFileSync(file, before);
|
|
} else if (operation === 'git-config-revert') {
|
|
gitAt(fixture.repo, 'config', 'docs.fixture', 'transient');
|
|
fs.writeFileSync(file, before);
|
|
} else if (operation === 'shell-write-revert') {
|
|
const child = spawnSync(process.execPath, ['-e', 'const fs=require("fs");const p=process.argv[1];const b=fs.readFileSync(p);fs.writeFileSync(p,"transient");fs.writeFileSync(p,b)', file], { timeout: 10000, encoding: 'utf8' });
|
|
expect(child.status).toBe(0);
|
|
} else {
|
|
fs.writeFileSync(file, 'transient');
|
|
fs.writeFileSync(file, before);
|
|
}
|
|
const observation = observer.stop();
|
|
expect(fs.readFileSync(file).equals(before)).toBe(true);
|
|
expect(observation.events.some(e => e.path === path.relative(fixture.repo, file))).toBe(true);
|
|
expect(docsWriteFailures(observation, [])).not.toEqual([]);
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test.each(['overflow', 'truncated', 'unknown-watch'])('fails closed on %s observations', async fault => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
const observer = await observeDocsWrites(fixture);
|
|
try {
|
|
const record = Buffer.alloc(fault === 'truncated' ? 1 : 16);
|
|
if (record.length === 16) {
|
|
record.writeInt32LE(-1, 0);
|
|
record.writeUInt32LE(fault === 'overflow' ? 0x4000 : 0x2, 4);
|
|
}
|
|
observer.injectKernelRecordsForTest(record);
|
|
const observation = observer.stop();
|
|
expect(observation.complete).toBe(false);
|
|
expect(docsWriteFailures(observation, [DOC_PATH])).toContain('incomplete docs write observation');
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('allows named authored edits while retaining their full mutation trace', async () => {
|
|
const fixture = fixtureDocs('updated', generated);
|
|
const observer = await observeDocsWrites(fixture);
|
|
try {
|
|
const file = path.join(fixture.repo, DOC_PATH);
|
|
fs.writeFileSync(file, fs.readFileSync(file, 'utf8').replace('text.', 'JSON.'));
|
|
const observation = observer.stop();
|
|
expect(docsWriteFailures(observation, [DOC_PATH])).toEqual([]);
|
|
expect(docsWriteFailures(observation, [])).not.toEqual([]);
|
|
expect(observation.changed).toContain(DOC_PATH);
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('kernel evidence remains private and readable after fixture cleanup', async () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
const observer = await observeDocsWrites(fixture);
|
|
const evidenceRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'docs-evidence-'));
|
|
let evidence: string | undefined;
|
|
try {
|
|
const file = path.join(fixture.repo, 'app.ts');
|
|
const original = fs.readFileSync(file);
|
|
fs.writeFileSync(file, 'transient');
|
|
fs.writeFileSync(file, original);
|
|
const observation = observer.stop();
|
|
evidence = preserveDocsEvidence(fixture, { output: 'observer preflight', toolCalls: [] },
|
|
`docs-observer-${path.basename(fixture.home)}`, 'transient', { observation }, evidenceRoot);
|
|
fixture.clean();
|
|
expect(fs.statSync(evidence).mode & 0o777).toBe(0o600);
|
|
const retained = JSON.parse(fs.readFileSync(evidence, 'utf8')).observation;
|
|
expect(retained.complete).toBe(true);
|
|
expect(docsWriteFailures(retained, [])).toContain('forbidden docs write: app.ts');
|
|
} finally {
|
|
fixture.clean();
|
|
fs.rmSync(evidenceRoot, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
test('declared native interface rejects interpreters, shell composition and Git writes', () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
try {
|
|
for (const command of ['bun -e "new Uint8Array(1)"', 'python3 -c "pass"', 'git status && node attack.js',
|
|
'git hash-object -w app.ts', 'git diff --output=app.ts', 'git show --textconv HEAD:app.ts', 'git config x.y z']) {
|
|
expect(docsCommandAllowed(command, fixture)).toBe(false);
|
|
}
|
|
for (const command of ['git status --porcelain', 'git diff --cached', 'git hash-object app.ts', ...docsPreambleCommands(fixture)]) {
|
|
expect(docsCommandAllowed(command, fixture)).toBe(true);
|
|
}
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('source-read evidence resolves installed paths but rejects a partial excerpt', () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
try {
|
|
const file = path.join(fixture.skills, 'ship/sections/documentation.md');
|
|
const output = fs.readFileSync(file, 'utf8').split('\n').map((line, i) => `${i + 1}→${line}`).join('\n');
|
|
for (const file_path of [file, '~/.claude/skills/gstack/ship/sections/documentation.md']) {
|
|
const result = { toolCalls: [{ tool: 'Read', input: { file_path }, output }] } as SkillTestResult;
|
|
expect(docsCompletedRead(result, file, fixture)).toBe(true);
|
|
result.toolCalls[0].output = '# Documentation audit gate';
|
|
expect(docsCompletedRead(result, file, fixture)).toBe(false);
|
|
}
|
|
} finally { fixture.clean(); }
|
|
});
|
|
});
|
|
|
|
describe('deterministic native-parent fault actor preflight', () => {
|
|
function setup(scenario: DocsFault) {
|
|
const fixture = fixtureDocs('current', generated);
|
|
const stateFile = installDocsActor(fixture, scenario);
|
|
const prompt = path.join(fixture.home, 'prompt.md');
|
|
const dispatch = (id: string) => {
|
|
fs.writeFileSync(prompt, `document-release files_updated audit ${id}`);
|
|
return docsActorCommand(stateFile, 'dispatch', { audit_id: id, candidate: fixture.candidate, prompt, run_in_background: 'false' });
|
|
};
|
|
const state = () => JSON.parse(fs.readFileSync(stateFile, 'utf8')) as DocsActorState;
|
|
return { fixture, stateFile, dispatch, state };
|
|
}
|
|
|
|
test.each(['missing-marker', 'launch-failure'] as const)('%s cannot publish', fault => {
|
|
const x = setup(fault);
|
|
try {
|
|
const response = x.dispatch('a');
|
|
expect(response.exit).toBe(fault === 'launch-failure' ? 23 : 0);
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).toEqual([]);
|
|
expect(docsActorVerdict(x.state(), 'Documentation: current', true)).not.toEqual([]);
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test('missing asset is actually absent and requires zero dispatches', () => {
|
|
const x = setup('missing-asset');
|
|
try {
|
|
expect(fs.existsSync(path.join(x.fixture.skills, 'document-release/sections/audit-scope.md'))).toBe(false);
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).toEqual([]);
|
|
x.dispatch('a');
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).not.toEqual([]);
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test('unsettled cancellation is distinct from a stop acknowledgment', () => {
|
|
const x = setup('timeout-unsettled');
|
|
try {
|
|
const task = JSON.parse(x.dispatch('a').text).task_id;
|
|
expect(JSON.parse(docsActorCommand(x.stateFile, 'status', { task_id: task }).text).elapsed_ms).toBeGreaterThan(600000);
|
|
expect(JSON.parse(docsActorCommand(x.stateFile, 'stop', { task_id: task }).text).settled).toBe(false);
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).not.toEqual([]);
|
|
docsActorCommand(x.stateFile, 'status', { task_id: task });
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).toEqual([]);
|
|
expect(x.dispatch('b').exit).not.toBe(0);
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test('late callback after settlement and transport repair remains tied to the old audit', () => {
|
|
const x = setup('late-result');
|
|
try {
|
|
const task = JSON.parse(x.dispatch('a').text).task_id;
|
|
docsActorCommand(x.stateFile, 'status', { task_id: task });
|
|
docsActorCommand(x.stateFile, 'stop', { task_id: task });
|
|
docsActorCommand(x.stateFile, 'repair');
|
|
const late = x.dispatch('b').text;
|
|
expect(() => parseDocsCompletion(late, 'b')).toThrow('completion identity');
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).toEqual([]);
|
|
expect(docsActorVerdict(x.state(), 'Documentation: current', true)).not.toEqual([]);
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test.each(['stale-before', 'stale-after', 'recovery'] as const)('%s requires a fresh successful audit before publication', fault => {
|
|
const x = setup(fault);
|
|
try {
|
|
x.dispatch('a');
|
|
if (fault === 'recovery') docsActorCommand(x.stateFile, 'repair');
|
|
else if (fault === 'stale-after') {
|
|
expect(fs.readFileSync(path.join(x.fixture.repo, 'app.ts'), 'utf8')).toContain('text');
|
|
docsActorHook(x.stateFile, JSON.stringify({ hook_event_name: 'PreToolUse', cwd: x.fixture.repo,
|
|
tool_name: 'Bash', tool_input: { command: 'git diff' } }));
|
|
}
|
|
if (fault.startsWith('stale-')) expect(fs.readFileSync(path.join(x.fixture.repo, 'app.ts'), 'utf8')).toContain('json');
|
|
const result = parseDocsCompletion(x.dispatch('b').text, 'b');
|
|
const report = path.join(x.fixture.home, 'report.md');
|
|
fs.writeFileSync(report, result.documentation_section);
|
|
expect(docsActorCommand(x.stateFile, 'publish', { audit_id: 'b', report }).exit).toBe(0);
|
|
expect(docsActorVerdict(x.state(), result.documentation_section, true)).toEqual([]);
|
|
const bad = x.state();
|
|
bad.events = bad.events.filter(e => e.action !== (fault === 'recovery' ? 'repair' : 'scheduled-input-edit'));
|
|
expect(docsActorVerdict(bad, result.documentation_section, true)).not.toEqual([]);
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test('native command entrypoint returns the injected result and records real tool state', () => {
|
|
const x = setup('missing-marker');
|
|
try {
|
|
const prompt = path.join(x.fixture.home, 'native-prompt.md');
|
|
fs.writeFileSync(prompt, 'document-release files_updated audit native');
|
|
const result = spawnSync(process.execPath, [path.join(ROOT, 'test/helpers/docsync-fault-actor.ts'), 'dispatch', x.stateFile,
|
|
'audit_id=native', `candidate=${x.fixture.candidate}`, `prompt=${prompt}`, 'run_in_background=false'], {
|
|
cwd: x.fixture.repo, timeout: 10000, encoding: 'utf8',
|
|
});
|
|
expect(result.status).toBe(0);
|
|
expect(parseDocsCompletion(result.stdout, 'native').status).toBe('blocked');
|
|
expect(x.state().events[0].action).toBe('dispatch');
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test('the registered native callback delivers the after-result edit, not a guessed delay', () => {
|
|
const x = setup('stale-after');
|
|
try {
|
|
x.dispatch('a');
|
|
const config = JSON.parse(fs.readFileSync(path.join(x.fixture.env.CLAUDE_CONFIG_DIR, 'settings.json'), 'utf8'));
|
|
const hook = config.hooks.PreToolUse.at(-1).hooks[0];
|
|
const event = { hook_event_name: 'PreToolUse', cwd: x.fixture.repo, tool_name: 'Bash',
|
|
tool_input: { command: 'git diff' }, session_id: 'fixture-session', tool_use_id: 'fixture-call' };
|
|
const invoke = (cwd: string) => spawnSync('bash', ['-c', hook.command], {
|
|
cwd: x.fixture.repo, input: JSON.stringify({ ...event, cwd }), encoding: 'utf8', timeout: 10000,
|
|
});
|
|
expect(invoke(x.fixture.home).status).toBe(0);
|
|
expect(x.state().lateChanged).toBe(false);
|
|
expect(invoke(x.fixture.repo).status).toBe(0);
|
|
expect(x.state().lateChanged).toBe(true);
|
|
expect(fs.readFileSync(path.join(x.fixture.repo, 'app.ts'), 'utf8')).toContain('json');
|
|
const actions = x.state().events.map(e => e.action);
|
|
expect(actions.indexOf('scheduled-input-edit')).toBeGreaterThan(actions.indexOf('completion'));
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
});
|