mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-03 23:37:53 +01:00
v1.94.0.0 ship: CI is the one full test run (CI carry, tree cache, /ship beside CI) (#3901)
* fix(ship): base merges, adjacent base edits, test-only and audit-requested fixes no longer force a second review loop (GST-407, GST-384, GST-391) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ship driver: trim two phrases for headroom with #3662 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ship driver: keep the Next steps heading phrase the handoff check reads; trim elsewhere Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: regenerate ship goldens Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes comparePath returns same only when every hunk sits where the base change puts it (patch-ids ignore positions) and treats a file-mode change as changed; the scoped-review packet says when a fence is truncated; --docs-result must be the posted audit's result; the --context-reviewed record goes beside the packet, outside the worktree. Negative controls for each. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pass-2 review fixes A test edit routes as tests= only when it adds lines with no gate tag; a removed or changed line or an unreadable test diff goes through the full loop. Diffs skip textconv. A hunkless patch (binary, empty, mode-only) is same when its patch-id matches. Without --reviewed-base test files route as before. finish's packet path and attempt's state directory have fallbacks. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: Step 11 review fixes; a version-only stamp_files edit after review is release metadata (GBRA-251) gateEditedPaths reads removed lines only inside hunks; carry refuses when a path listing fails; docs freshness stops on a failed listing; attempt counts an unreadable state file. releasePaths includes stamp_files paths whose edit is a version swap under stamp_patterns. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: no test edit skips review; guard packet writes; exclusive attempt state A test-only edit with no gate tag gets the scoped fix-check of its test hunks instead of a lanes-only rerun. Packet write failures print a STOP or RECEIPT_CARRY_REFUSED. attempt's allowance uses an exclusive create. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: attempt prints gstack's install command; carry checks the product diff outside context paths The attempt STOP no longer echoes the audit child's fix text. A scoped-review carry now requires the product patch-id without the context paths to match at both heads. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: oversized packets take the full loop; exact --docs-requested match; name-only test globs only in test dirs A scoped-review packet section over 200 lines sends that path to the full loop instead of a truncated packet. The docs-result path match is exact. test_*.py, *Test.java and similar count as test code only under a test directory. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(unattended): carry's scoped-review refusal and what it keeps Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: carry's scoped-review route in ship-policy, review-policy and troubleshooting (docs-a-1) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(windows): curate ship-rereview-triggers out of the Windows lane like ship-release (run 41193336209) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review-log): record a notes-only /ship pass clean; name the cause and repair when a pass's tree moved (GST-430) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(receipt): carry and write handle PRs with thousands of files; tree receipt fails closed on a failed diff (GST-448) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): preflight routes RECEIPT_CARRY_SCOPED_REVIEW to the scoped review; carry --dry-run writes no packet (GST-442) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(ship): Step 1.4 and Step 16 route the scoped-review refusal and the --docs-requested fix; trims keep the cap (GST-442, GST-444) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: /ship review pass 1: notes-only floor, scoped carry order, named temp and diff refusals - A notes-only pass is recorded clean only when every finding is fixed, or noted or deferred below the blocking floor (never CRITICAL or blocking), by its logger-stamped action, and the findings account for the claimed issue and critical counts (GST-430). - The preflight scoped carry route commits and pushes the base-merge fix before writing the packet (GST-442). - patchId refuses by name on an unwritable TMPDIR; the tree receipt names why its diff failed (GST-448). - Step 16 stage 3 spells out --docs-requested with its --docs-result (GST-444). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: pin the Step 16 --docs-requested route with its --docs-result Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: a claimed clean review row with a logger-stamped open gate finding reads UNVERIFIED (red-team, pass 1) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: Step 11 findings: gate rows from --findings are stamped, one open rule, stale patch-id dirs swept - deriveFromFindings stamps gate rows with the session kind before deriving status, so a findings file cannot dispose of a gate finding the unattended logger forces open (outside-voice#1). - reviewFreshness reads a claimed row's findings with isOpenByAction, the rule the notes-only rewrite uses; noted and deferred clear a finding only when it is marked blocking: false and is not CRITICAL. - patchId sweeps gstack-patch-id-* directories over an hour old that a killed carry left in TMPDIR. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: Step 11 round 2: stamped-open gates win over status and disposition; resolved gate, skipped and below-floor actions stay CURRENT - isOpenFinding treats action 'open' (the logger's forced-open stamp) as open before a row's status or disposition. - reviewFreshness flags a claimed clean row only for an 'open' action or a CRITICAL or blocking deferral or note, so interactive kept and restored gates, skipped advisories and INFORMATIONAL deferrals read CURRENT as before. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: carry dry run offers the scoped route last; patch-id diff under the git dir; changelog matches the review-log floor Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: a claimed clean review row with any unresolved non-gate CRITICAL finding reads UNVERIFIED (pass 2 red-team) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: carry dry-run scoped route, patch-id directory, tree-receipt refusal (docs-d-1 audit) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): Step 3 names merge conflicts in its STOP sentence again (ship-workflow-clarity pin) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(tree-receipt): keep the gated-to-current diff at the file helper timeout (60 s); pregate literals check Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(carry): large-diff fixture sets core.longpaths so Git for Windows can add its long receipt paths (windows-free-shard 1, run 41210080055) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review): a stale local base branch yields to its origin twin in gate-diff and diff-scope (GST-462) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): Step 11 saves the targeted record and the receipt writes carry it; Step 9 clears it (GST-464) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * changelog: GST-464 and GST-462 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review): pass 1: ahead and diverged labels, diff-scope needs shared history; preflight refreshes origin/<base> and stops on a diff far beyond the PR (GST-462, HARN-15) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pass-2 review fixes for base twin, prSized stop and targeted record Resolve the tracking twin by refs/remotes/<remote>/<ref> and only for a bare branch name, keep a local base that is ahead or diverged, quote the prSized stop's base and name the fork case, and have every Step 9 entry create, exclude and clear .gstack/tmp/targeted.json. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: origin's tracking base wins whenever related; prSized lower bound The fix-check found that keeping a local base ahead of origin left unpushed base commits out of the gate scan while review-plan listed them. The tracking branch now wins whenever it shares history with HEAD, an explicit origin/main reads refs/remotes/origin/main, the preflight also stops when the diff lists fewer files than the PR, and the Step 9 exclude line survives a missing .git/info and an exclude file without a final newline. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * changelog: the tracking-ref claim holds while the tracking branch exists Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: repin the two ship section briefs under no-loss replay GST-464 GST-464 brief-identity check: every reviewer brief lib/review-brief.ts cuts from ship/sections/review-army.md and adversarial.md stays byte-identical to the bodies the L and GST-257 replays admitted; only lead instructions outside the cut spans changed. The check is a test CI runs. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ship: keep Step 9's PASS_HEAD sentence whole; repin review-army under GST-464 The targeted-record line follows the PASS_HEAD/RANGE_BASE sentence instead of splitting it (test/review-fix-range.test.ts pins that sentence). The brief-body check still holds: every reviewer brief is byte-identical. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: restore bin/gstack-ship-receipt after the main merge The merge commitb10e2dbwrote a truncated copy of the file. This is the three-way merge of both sides with the refused() union: main's codes plus RECEIPT_CARRY_SCOPED_REVIEW. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * review-collect: narrow the consumer scan and give it one 300 s deadline (GST-544) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * review-collect: keep the consumer scan deadline under the 180 s plan timeout (review fix) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * review-collect: plain paths under --literal-pathspecs; whole-millisecond timeout override (review fix) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * review-collect: whole-tree grep for non-UTF-8 names, no argument spread, name spawn errors (review fix) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * review-collect: cap the consumer grep output at 256 MB across calls; per-scan deadline wording (review fix, cycle 4) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * review-collect: read a missing spawn status as null (Bun 1.4.2 returns undefined on ENOENT) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: review-collect whole-tree helper reads a null status as -1 (typecheck:test ratchet) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * review-bots: /review and /ship triage every configured review bot, not only Greptile (#3122) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (gh before 2.48 pages without --slurp; driver routes Step 10; tests for guards, review-bot fence, REMOTE_SLUG owner) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: drop the review-bot fence test that could not fail (the tool-fence WARN degrade is unreachable: placeholder spans are skipped first) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: Step 11 review fixes (record by comment id so a filename never reaches a shell; lookup failures are errors; tojson pages; longest bot prefix wins) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pass-2 review fixes (record refuses * paths, colliding ids and a missing fetch dir; parent requires id and dir; botOwning doc) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * review: classify the PR type and always check repo conventions (#356) The checklist now opens with a PR-type line (application, CI/infra, scripts/tooling, docs-only or mixed). Application and mixed diffs run every category; other types always run five categories, including the new Repo Conventions & Platform Fit, and report the rest as not applicable in one line instead of skipping silently. No-loss replay: GSTA-224 H1 preregistered no-loss run (run id gsta224-h1-20261031, Opus 5.5, 81 calls, $9.53) against main's checklist with a no-gstack arm; every preregistered bar passed. The review/checklist.md pin in test/review-packets-removed.test.ts moves to a32963ef with replay 'GSTA-224 H1'. Refs GSTA-224 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * review-bots: allowlist the record placeholders and match Step 10's id rule in tests The record fence uses <comment-id>, <fix|fp|already-fixed> and <category>, each with an enum or numeric grammar; <greptile-dir> becomes <bot-dir>. review/greptile-triage.md also lists every REVIEW_BOTS skip reason. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: review-bot triage covers every configured bot (#3122) README, CONTRIBUTING, the contributor reference and standalone-binaries name gstack-review-bots and the review-bot fence; TODOS drops the #3122 item; the changelog fragment covers record-by-id and lookup errors. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: the PR-type test also pins the compose, Terraform and README/CHANGELOG phrases Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: follow the review-bot rename in the remote free tier's three reds The cross-skill consistency judge describes review-bot-history.md and its record helper; the tracker-guard exemption for the removed raw Greptile fetch in review/greptile-triage.md is dropped (the fetch now lives in gstack-review-bots, whose bodies still reach context only through gstack-issue-guard); the Step 10 PR-check test expects 'Review bots: not run'. HARNESS_VERSION stays 6: skill-fixture.ts and llm-judge.ts are recorded non-behavioral for every other case. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: keep the cross-skill judge's rubric to one renamed bullet Name review-bot-history.md beside the older greptile-history.md so the calibration corpus still matches, and keep the REMOTE_SLUG bullet. One live 3-sample panel at this tree: consistent 3/3, scores 4/4/4. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * changelog: H1 fragment passes prose lint Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * changelog: name the H1 fragment after its branch; TODOS drops #356 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: /review's PR type step in docs/skills.md (#356) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ship-policy: review_exclude_paths key and loadPolicyAt (GST-508) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * review: apply the base policy's review_exclude_paths and the reviewer diff ceiling (GST-508) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * receipt: bind review_exclude in write, carry and the queue (GST-508) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs, tests and changelog for review_exclude_paths (GST-508) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (binary by bytes, links kept, untracked files out of the selection) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: keep binary files at code paths under review_exclude_paths Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: name the reasons that keep a binary file under review_exclude_paths Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: keep every review_exclude_paths candidate when the gate scan is unknown; docs example names its version Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: keep the NUL fixture clear of the sync-spawn tripwire Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * changelog: split the ceiling sentence Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: keep a live binary file whose bytes cannot be read; tests render outside voices from a clean checkout and read the ship fixture text Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(review-exclude): explicit +x for the census script; skip the chmod-000 read case on Windows and root Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * refactor: move loadPolicyAt to lib/ship-policy-at.ts so lib/ship-policy.ts stays under the 800-line module cap Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: refuse a base that contains HEAD; skip a self-pointing origin/HEAD (GST-774) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * chore: regenerate gstack-doctor-components.sh for BASE_CONTAINS_HEAD Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: land resumes a merged merge-only PR instead of refusing BASE_CONTAINS_HEAD (GST-774 review) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: changelog fragment for GST-774 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: troubleshooting names diff-scope base_contains_head and the self-pointing no_base case (GST-774 docs audit) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: repin the reviewer briefs to GST-508's no-loss-tested bytes ship/sections/review-army.md, ship/sections/adversarial.md and lib/review-depth.ts now pin the bytes the GST-508 preregistered no-loss replay ran atef2760f(handoffs/GSTA-202/noloss-results.md in the GStack Minder drive): Arm V 33/41 vs R 33/41, delta 4, exact sign flip, p_NI 0.0279 (pass). The merge of main83ae96eleaves these three files byte-identical toef2760f, and an Arm P unit (3593) renders byte-identical briefs atef2760fand at the merge. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: GST-774 review round 1 — merged resume depth, aliased and detached PR checkouts, develop/trunk defaults Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ship: Step 20 says a metrics row, keeping DRIVER.md at 12 KB after the main merge Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: ship-parts resend case leaves room for the first phase-3 part after the Step 9 targeted-record sentence Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: /ship delivers review/checklist.md in two parts so checklist v1 fits the 24576-byte part cap (#3670) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * changelog: drop the GST-407 and GST-430 fragments, already released on main (stacked history) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * wave review-53cfb5bfe011: lib/ship-policy.ts back to 800 lines (module-size ratchet after merging #3744 and #3795) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * wave review-53cfb5bfe011: /ship pass 1 fixes - lib/ship-receipt-depth.ts: a merged resume (land after a true merge) no longer recomputes the review_exclude digest over the emptied range, so it lands instead of RECEIPT_INVALID (failing-first land test with review_exclude_paths active). - bin/gstack-review-dispatch render: a standalone targeted render under an active review_exclude_paths refuses and points at the pass manifest voices stage, instead of writing a brief with no selection that leaves verify dirty (failing-first test). - test/default-base.test.ts: #3795 expectations follow main R51 depth (light for an 80-line code diff; runtime_lines=80 still proves the real base). - test/review-dispatch.test.ts: the GST-897 fingerprint test renders from the clean checkout, like its neighbor, because CI rebuilds the tracked lib/diagram-render/dist/diagram-render.html and the reviewer diff ceiling refused the ROOT render. * wave review-53cfb5bfe011: /ship pass 2 fixes - lib/review-exclude.ts: binaryContent reads each blob's first 8,000 bytes (git cat-file blob, 8 KB buffer) instead of one --batch read capped at 256 MB, so head mode (receipt, carry, queue) and the plan agree past 256 MB of blobs; an unreadable blob is kept, as in live mode. Failing-first test with two 270 MB blobs. - lib/review-bots.ts: a file-level bot review comment (subject_type file, no line position) is triaged instead of read as outdated. Failing-first test. - test/ship-queue-land.test.ts: the merged-resume exclusion test runs its gate scan under the fixture state root, not the real home (CI free-suite 6 home containment). * fix: merge follow-ups for main3d99c22: origin/* stays qualified in diff-scope's twin step, retro skeleton back under its cap, combined no-loss pin GST-508+R51+GST-911 * fix(review): receipt write gate-scans whenever the base declares review_exclude_paths; collectLive never re-qualifies origin/<base> (#3852 /ship red-team#1, outside-voice#1) * test: the docs-only receipt digest test keeps its gate scan out of the real home (free-suite shard 5 FREE_SUBSHARD_REJECTED) * fix(mergefc292e22f): tentative REVIEW_EXCLUDE_KEY_VERSION, classify targeted.json, render main's new outside-voice test from a clean checkout - lib/ship-policy.ts: REVIEW_EXCLUDE_KEY_VERSION 1.93.0.0 -> 1.94.0.0, the next-minor tentative form stamp_paths moves (docs/ship-policy.md, #3512); 1.93.0.0 is a released heading that never names the key. - test/gstack-clean-producers.test.ts (new from main): KEPT gains targeted.json, the wave's --review-targeted input. - test/review-dispatch.test.ts: main's wrong-tree test rendered the outside voice from ROOT; a CI build that dirties ROOT pushes that diff past the wave's REVIEW_DIFF_MAX_BYTES ceiling, so render refused and verify saw no brief (exit 3). It now runs in cleanCheckout like the wave's other outside-voice tests. * test(merge68410828f): GST-967's brief-body pin carries the wave's GST-508 placeholders The merged sectionTexts differ from main's by exactly the wave's GST-508 exclude-pathspec lines (merged minus main equals wave minusfc292e22f); the 1,440-case plan grid still hashes to main's MAIN_PLANS. Checks: GStack Minder drive handoffs/GSTA-202/merge-6841-checks.md. * ship: CI carries mapped lanes, --reuse tree cache, early push, finish/accept split (GSTA-258 A) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ship: name the ship and pregate bins in prose so skill-bin-drift does not read a bin Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * size-ledger: re-record ship after the tests.md wording fix Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * pregate: a carried run clears the remote gate through remote_gate_runs, so a paid-only remainder is not a dead end Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * merge cleanup: drop wave fragments main already consumed and a duplicated KEPT entry Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship): record-ci verifies the CI run itself; finish's deferred preflight keeps pregate.json /ship pass 1 findings red-team#1 and specialist-batched#1 (GSTA-258 piece A). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * feat(pregate): CI carry is gstack's default remote gate route; the VM is opt-in (plan choice F1) The uncleared remote_gate obligation names the CI route first when the base declares remote_gate_runs. docs/pregate.md says how gstack clears it, including the pull_request merge-ref tree mismatch. GSTA-258 piece A. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (round 2) record-ci records the base policy that mapped the lane; the CI-first remote gate hint respects --no-ci-carry and reruns the whole pregate; changelog and docs say what the tool does. GSTA-258 piece A. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: CI carry in CONTRIBUTING, recommended-harness and the pregate usage line (ship docs audit) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
b42ad1e300
commit
4ec1722ecb
32 files changed
+1395
-61
No files matched your search
@@ -27,6 +27,9 @@
|
|||||||
"inventory": "bun --no-env-file --config=/dev/null run scripts/test-free-shards.ts --list | sed -n 's/^ //p'",
|
"inventory": "bun --no-env-file --config=/dev/null run scripts/test-free-shards.ts --list | sed -n 's/^ //p'",
|
||||||
"skipped": "bun --no-env-file --config=/dev/null -e 'import { isPaidTestFile } from \"./test/helpers/paid-test-set.ts\"; import * as fs from \"node:fs\"; for (const f of fs.readFileSync(process.env.PREGATE_FILES_FILE, \"utf8\").split(/\\r?\\n/)) if (f && isPaidTestFile(f)) console.log(f);'"
|
"skipped": "bun --no-env-file --config=/dev/null -e 'import { isPaidTestFile } from \"./test/helpers/paid-test-set.ts\"; import * as fs from \"node:fs\"; for (const f of fs.readFileSync(process.env.PREGATE_FILES_FILE, \"utf8\").split(/\\r?\\n/)) if (f && isPaidTestFile(f)) console.log(f);'"
|
||||||
},
|
},
|
||||||
|
"ship_lanes_from": {
|
||||||
|
"bun run test": { "profiles": ["ci-reuse/free-tests"], "inventory": "bun --no-env-file --config=/dev/null run scripts/test-free-shards.ts --list | sed -n 's/^ //p'" }
|
||||||
|
},
|
||||||
"warn": [],
|
"warn": [],
|
||||||
"strays": { "allow": ["modules/loadouts.json", "modules/loadouts.schema.json"] }
|
"strays": { "allow": ["modules/loadouts.json", "modules/loadouts.schema.json"] }
|
||||||
}
|
}
|
||||||
@@ -1,5 +1,14 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## [1.94.0.0] - 2026-11-03
|
||||||
|
|
||||||
|
- CI is now the one full test run. When a verified CI run already passed at the exact, clean tree, `gstack-pregate`'s tests stage and /ship's test lanes use it instead of running the suite again (GSTA-258, R65 item 1). The tests stage looks up the newest verified run at HEAD under the base branch's `run_profiles` by itself (`CI_CARRY: accepted …`; `--no-ci-carry` turns it off). A /ship lane carries only when the base branch's `.gstack/pregate.json` maps its exact command in the new `ship_lanes_from` key, and only when the run's `tree.json` covers the lane's inventory; gstack maps `bun run test` to its Free Tests run. A carried lane prints `LANE: <label> carried run=<id>/<attempt>` and is recorded as an explicit CI-sourced evidence row (`gstack-evidence record-ci`), never as a local run. Dirty trees, fork runs, stale runs and a newer red or running run never carry.
|
||||||
|
- A lane that already passed at the same tree, command and runtime is reused instead of rerun (`gstack-evidence run --reuse`, the default from `gstack-ship-preflight` and `finish`): a newer failure vetoes an older pass, and the reused row keeps its original timestamp, so the 24-hour age limit still measures the real run.
|
||||||
|
- /ship runs beside CI: on a repo that maps its lanes, Phase 2 pushes early (`gstack-ship-release push --early` runs the secrets scan and the sealed guard first), `finish` defers a mapped lane it cannot carry yet (`EVIDENCE: DEFERRED`), and the new `gstack-ship-release accept` carries it from the green run at the final pushed head and reruns the pregate there. `gstack-ci-carry find|wait` checks or waits for that run, and the ship receipt records `ci_carried`. See docs/pregate.md#ci-carry.
|
||||||
|
- On a small machine, the remote gate obligation clears from the same carried run (through `remote_gate_runs` and its inventory check) when the files left over are only ones the gate skips, such as gstack's paid evals. Before, that remainder made the gate report `tests=0` and `clear --run` refuse it as `paid_only`, so nothing could clear it.
|
||||||
|
- On gstack itself, a green Free Tests run at the same tree is now the default way the remote gate obligation clears, and the Ubicloud VM (`gstack-pregate --stage tests --remote`) is an opt-in. The uncleared obligation names the CI route first: push, `gstack-ci-carry wait --base main`, then rerun `gstack-pregate --allow-repo-commands`. On a branch behind main, dispatch Free Tests on the branch (`gh workflow run free-tests.yml --ref <branch>`), because a `pull_request` run tests the merge commit, another tree.
|
||||||
|
- `gstack-evidence record-ci` now finds and verifies the CI run itself (`record-ci --label <L> --base <base> -- <command>`), so no caller can hand it run ids or a test count. `finish` writes its preflight-only pregate run to its own file, so the full `.gstack/tmp/pregate.json` stays in place until `accept` replaces it.
|
||||||
|
|
||||||
## [1.93.42.0] - 2026-11-03
|
## [1.93.42.0] - 2026-11-03
|
||||||
|
|
||||||
**The stamp checks name the branch's lines by position, and the carry and finish accept a kept VERSION claim (GST-979, GST-980).** After GST-963, three gaps were left in the checks that accept a stamp commit, all found by the #3862 reviewers. A base line whose text duplicated a line the branch added got the branch's fork-version bound, so a hand edit of that base copy passed the tree receipt. When a train slot kept its claimed VERSION, a base line moved down to that version also read as a stamp. And the /ship receipt carry and finish refused the fork-version row move the restamp now makes: the carry kept the branch-added row in both product patch-ids (`RECEIPT_CARRY_PRODUCT_CHANGED`), and with a kept VERSION claim the carry and finish read every change as product or behavior.
|
**The stamp checks name the branch's lines by position, and the carry and finish accept a kept VERSION claim (GST-979, GST-980).** After GST-963, three gaps were left in the checks that accept a stamp commit, all found by the #3862 reviewers. A base line whose text duplicated a line the branch added got the branch's fork-version bound, so a hand edit of that base copy passed the tree receipt. When a train slot kept its claimed VERSION, a base line moved down to that version also read as a stamp. And the /ship receipt carry and finish refused the fork-version row move the restamp now makes: the carry kept the branch-added row in both product patch-ids (`RECEIPT_CARRY_PRODUCT_CHANGED`), and with a kept VERSION claim the carry and finish read every change as product or behavior.
|
||||||
|
|||||||
+2
-1
@@ -191,7 +191,8 @@ the new defaults.
|
|||||||
| Run one free test file while repairing | `bun test <file>` | Free, seconds | Never bare `bun test` for the suite |
|
| Run one free test file while repairing | `bun test <file>` | Free, seconds | Never bare `bun test` for the suite |
|
||||||
| Run full free acceptance before publishing | `bun run test` | Free, a few minutes | Bun 1.4.2 |
|
| Run full free acceptance before publishing | `bun run test` | Free, a few minutes | Bun 1.4.2 |
|
||||||
| Run the full free suite from a small machine | `bun run test:ubicloud` | Free suite on a billed 16-vCPU VM, about 5 minutes | `UBICLOUD_API_KEY`, `UBI_OWNER` |
|
| Run the full free suite from a small machine | `bun run test:ubicloud` | Free suite on a billed 16-vCPU VM, about 5 minutes | `UBICLOUD_API_KEY`, `UBI_OWNER` |
|
||||||
| Clear pregate's test tier on a machine under 8 cores ([remote gate](docs/pregate.md#remote-gate)) | `gstack-pregate --stage tests --remote --spend-cap 2 --timeout 2700 --allow-repo-commands` (same command keyed or keyless) | Keyed: one billed 16-vCPU VM, about 7 minutes and $0.06 for the whole free tier, worst case $0.38; keyless: free, local parallel shards, practical for small selections only (98 files took 103 s on 4 cores) | Keyed: `UBICLOUD_API_KEY`, `UBI_OWNER`; keyless: nothing |
|
| Clear pregate's test tier on a machine under 8 cores from CI, the default on gstack ([CI carry](docs/pregate.md#ci-carry)) | push, `gstack-ci-carry wait --base main`, then `gstack-pregate --allow-repo-commands` (a branch behind main first runs `gh workflow run free-tests.yml --ref <branch>`) | Free; reuses the green Free Tests run at your pushed tree | Pushed branch, `gh` |
|
||||||
|
| Clear pregate's test tier on a machine under 8 cores when no CI run covers your tree ([remote gate](docs/pregate.md#remote-gate), opt-in) | `gstack-pregate --stage tests --remote --spend-cap 2 --timeout 2700 --allow-repo-commands` (same command keyed or keyless) | Keyed: one billed 16-vCPU VM, about 7 minutes and $0.06 for the whole free tier, worst case $0.38; keyless: free, local parallel shards, practical for small selections only (98 files took 103 s on 4 cores) | Keyed: `UBICLOUD_API_KEY`, `UBI_OWNER`; keyless: nothing |
|
||||||
| Run the curated Windows-safe subset | `bun run test:windows` (CI: six `windows-latest` jobs, nightly on main; PRs run the shorter `windows-smoke` check) | Free | Windows, Git Bash |
|
| Run the curated Windows-safe subset | `bun run test:windows` (CI: six `windows-latest` jobs, nightly on main; PRs run the shorter `windows-smoke` check) | Free | Windows, Git Bash |
|
||||||
| Refresh the Windows duration seed | `gh workflow run windows-free-tests.yml --ref <branch> -f record_durations=true`, then commit the `free-test-durations-windows` artifact | Free CI runners | Pushed branch, `gh` with workflow rights |
|
| Refresh the Windows duration seed | `gh workflow run windows-free-tests.yml --ref <branch> -f record_durations=true`, then commit the `free-test-durations-windows` artifact | Free CI runners | Pushed branch, `gh` with workflow rights |
|
||||||
| Run a native Windows qualification campaign | `gh workflow run native-qualification.yml --ref <branch> -f mode=<mode>` ([modes](docs/TESTING_INTERNALS.md#windows-free-lane)) | Free CI runners | Pushed branch, `gh` with workflow rights |
|
| Run a native Windows qualification campaign | `gh workflow run native-qualification.yml --ref <branch> -f mode=<mode>` ([modes](docs/TESTING_INTERNALS.md#windows-free-lane)) | Free CI runners | Pushed branch, `gh` with workflow rights |
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
# gstack digest v1.93.42.0 — regenerate/re-copy after upgrading gstack
|
# gstack digest v1.94.0.0 — regenerate/re-copy after upgrading gstack
|
||||||
|
|
||||||
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
|
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
|
||||||
for agent hosts without a full skill install. The full skills add workflows,
|
for agent hosts without a full skill install. The full skills add workflows,
|
||||||
|
|||||||
Executable
+77
@@ -0,0 +1,77 @@
|
|||||||
|
#!/usr/bin/env -S bun --no-env-file --config=/dev/null
|
||||||
|
/**
|
||||||
|
* gstack-ci-carry — CI as the one full test run (GSTA-258; docs/pregate.md#ci-carry).
|
||||||
|
*
|
||||||
|
* gstack-ci-carry find [--base <branch>] [--profile <name>]... [--command <exact lane command> --allow-repo-commands] [--json]
|
||||||
|
* gstack-ci-carry wait [--base <branch>] [--profile <name>] [--timeout <s>] [--interval <s>]
|
||||||
|
*
|
||||||
|
* find: the newest verified run of each profile (default: every profile the base branch's
|
||||||
|
* ship_lanes_from names) at HEAD's clean tree. With --command, the base branch's mapping for that exact lane
|
||||||
|
* command supplies the profiles and the inventory the runs must cover (a pin-file command, so it needs
|
||||||
|
* --allow-repo-commands). Prints CI_CARRY: accepted <receipt> (exit 0) or CI_CARRY: none (<reason>) (exit 1).
|
||||||
|
* wait: polls the newest run of the profile's workflow at HEAD until it completes (default timeout 1800 s),
|
||||||
|
* prints CI_WAIT: green|red|running|none, and on green runs find. Exit 0 green and carried, 1 red or not
|
||||||
|
* carried, 3 still running or no run. Logic: lib/ci-carry.ts; the run checks are lib/run-evidence.ts.
|
||||||
|
*/
|
||||||
|
import { spawnSync } from 'node:child_process';
|
||||||
|
import { findCarry, waitForRun } from '../lib/ci-carry';
|
||||||
|
import { isConfigUnavailable, loadConfigFromBase } from '../lib/pregate/config';
|
||||||
|
import { buildRunCheck } from '../lib/pregate/run-check';
|
||||||
|
import { runInventory } from '../lib/run-evidence';
|
||||||
|
import { wtreeFingerprint } from '../lib/script-invocation';
|
||||||
|
import { resolveBaseBranch } from '../lib/ship-policy';
|
||||||
|
|
||||||
|
const HELP = `gstack-ci-carry — use a verified CI run at HEAD's exact tree instead of running the suite again (docs/pregate.md#ci-carry)
|
||||||
|
|
||||||
|
find [--base <branch>] [--profile <name>]... [--command <exact lane command> --allow-repo-commands] [--json]
|
||||||
|
CI_CARRY: accepted <receipt> (exit 0) or CI_CARRY: none (<reason>) (exit 1)
|
||||||
|
wait [--base <branch>] [--profile <name>] [--timeout <s>] [--interval <s>]
|
||||||
|
poll the newest run at HEAD (default 1800 s, every 20 s): CI_WAIT: green|red|running|none, then find on green;
|
||||||
|
exit 0 carried, 1 red or not carried, 3 still running or no run
|
||||||
|
|
||||||
|
Profiles and ship_lanes_from come from origin/<base>'s .gstack/pregate.json only. Only a clean, committed tree
|
||||||
|
carries; fork runs, stale runs, a newer red or running run on HEAD and a tree.json that misses an inventory
|
||||||
|
file never do. Exit 2 is usage.`;
|
||||||
|
|
||||||
|
const argv = process.argv.slice(2);
|
||||||
|
const cmd = argv[0];
|
||||||
|
const all = (n: string) => argv.flatMap((a, i) => (a === n && argv[i + 1] !== undefined ? [argv[i + 1]!] : []));
|
||||||
|
const flag = (n: string) => all(n).at(-1);
|
||||||
|
const out = (s: string) => process.stdout.write(`${s}\n`);
|
||||||
|
const usage = (m: string) => { process.stderr.write(`gstack-ci-carry: ${m}\n`); process.exit(2); };
|
||||||
|
if (!cmd || argv.includes('--help') || argv.includes('-h')) { out(HELP); process.exit(cmd ? 0 : 2); }
|
||||||
|
if (cmd !== 'find' && cmd !== 'wait') usage(`unknown command ${cmd}; run gstack-ci-carry --help`);
|
||||||
|
const git = (args: string[]) => { const r = spawnSync('git', args, { encoding: 'utf8', timeout: 30_000 }); return r.status === 0 ? r.stdout.trim() : ''; };
|
||||||
|
const root = git(['rev-parse', '--show-toplevel']);
|
||||||
|
if (!root) usage('not inside a git repository');
|
||||||
|
const base = resolveBaseBranch(root, flag('--base'));
|
||||||
|
const loaded = loadConfigFromBase(root, base);
|
||||||
|
if (isConfigUnavailable(loaded) || loaded.errors.length) { out(`CI_CARRY: none (profiles_unavailable: origin/${base}'s .gstack/pregate.json is unreadable or invalid)`); process.exit(1); }
|
||||||
|
const command = flag('--command');
|
||||||
|
const mapping = command !== undefined ? loaded.config.ship_lanes_from[command] : undefined;
|
||||||
|
if (command !== undefined && !mapping) { out(`CI_CARRY: none (not_mapped: ${JSON.stringify(command)} is not a key of origin/${base}'s ship_lanes_from)`); process.exit(1); }
|
||||||
|
if (mapping && !argv.includes('--allow-repo-commands')) usage('--command runs the mapping\'s inventory, a pin-file command: pass --allow-repo-commands');
|
||||||
|
const profiles = all('--profile').length ? all('--profile') : mapping ? mapping.profiles : [...new Set(Object.values(loaded.config.ship_lanes_from).flatMap(m => m.profiles))];
|
||||||
|
if (!profiles.length) { out(`CI_CARRY: none (no_profile: origin/${base} maps no lane in ship_lanes_from; pass --profile <name>)`); process.exit(1); }
|
||||||
|
const check = buildRunCheck(root, null, argv.includes('--allow-repo-commands'), base);
|
||||||
|
if ('error' in check) { out(`CI_CARRY: none (profiles_unavailable: ${check.error})`); process.exit(1); }
|
||||||
|
const head = git(['rev-parse', 'HEAD']);
|
||||||
|
const tree = git(['rev-parse', 'HEAD^{tree}']);
|
||||||
|
|
||||||
|
function find(): number {
|
||||||
|
const wtree = () => { const fp = wtreeFingerprint({ cwd: root }); return 'tree' in fp ? fp.tree : null; };
|
||||||
|
const v = findCarry(check as Exclude<typeof check, { error: string }>, { profiles, head, tree, wtree, ...(mapping ? { inventory: () => runInventory(mapping.inventory, root) } : {}) });
|
||||||
|
if (argv.includes('--json')) out(JSON.stringify(v));
|
||||||
|
else out(v.ok ? `CI_CARRY: accepted ${v.receipt}` : `CI_CARRY: none (${v.reason}: ${v.detail})`);
|
||||||
|
return v.ok ? 0 : 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (cmd === 'find') process.exit(find());
|
||||||
|
const timeout = Number(flag('--timeout') ?? '1800');
|
||||||
|
const interval = Number(flag('--interval') ?? '20');
|
||||||
|
if (!(timeout >= 0 && timeout <= 21_600) || !(interval >= 1 && interval <= 600)) usage('--timeout is 0-21600 seconds and --interval 1-600');
|
||||||
|
const profile = check.profiles[profiles[0]!];
|
||||||
|
if (!profile) usage(`${profiles[0]} is not a run_profiles name on origin/${base}`);
|
||||||
|
const w = await waitForRun(check.deps, { workflow: profile!.workflow, events: profile!.events, head, timeoutMs: timeout * 1000, intervalMs: interval * 1000, sleep: ms => Bun.sleep(ms), clock: Date.now });
|
||||||
|
out(`CI_WAIT: ${w.state} ${profiles[0]} ${'run' in w ? `run=${w.run.id}/${w.run.run_attempt}${w.state === 'red' ? ` failing=${w.failing.join(',') || '-'}` : ''}` : w.reason}`);
|
||||||
|
process.exit(w.state === 'green' ? find() : w.state === 'red' ? 1 : 3);
|
||||||
+162
-4
@@ -3,7 +3,9 @@
|
|||||||
* gstack-evidence — verification-evidence ledger: the mechanical arm of /ship's
|
* gstack-evidence — verification-evidence ledger: the mechanical arm of /ship's
|
||||||
* IRON LAW ("no completion claims without fresh verification evidence").
|
* IRON LAW ("no completion claims without fresh verification evidence").
|
||||||
*
|
*
|
||||||
* gstack-evidence run --label <L> -- <cmd...>
|
* gstack-evidence run [--reuse [--max-age <hours>]] --label <L> -- <cmd...>
|
||||||
|
* gstack-evidence record-ci --label <L> --base <base> -- <cmd>
|
||||||
|
* gstack-evidence carried [--json]
|
||||||
* gstack-evidence check [--label <L> [--expect-cmd <exact string>]]... | --all
|
* gstack-evidence check [--label <L> [--expect-cmd <exact string>]]... | --all
|
||||||
* [--max-age <hours>] [--allow-paths <csv>]
|
* [--max-age <hours>] [--allow-paths <csv>]
|
||||||
* gstack-evidence bundle --out <file> [--label <L>]... [--require <csv>] [--selection L=<text>]...
|
* gstack-evidence bundle --out <file> [--label <L>]... [--require <csv>] [--selection L=<text>]...
|
||||||
@@ -51,6 +53,21 @@
|
|||||||
* normalization — the same convention as bin/gstack-verify-gate (which hashes
|
* normalization — the same convention as bin/gstack-verify-gate (which hashes
|
||||||
* for TRUST; this ledger hashes for FRESHNESS).
|
* for TRUST; this ledger hashes for FRESHNESS).
|
||||||
*
|
*
|
||||||
|
* `run --reuse` (GSTA-258, what gstack-ship-preflight and finish pass): before
|
||||||
|
* running, the newest row for this label, command hash and working-tree
|
||||||
|
* fingerprint decides. It is reused only when it passed with a counted run
|
||||||
|
* (tests_ran > 0) under the same execution identity (bun and node versions,
|
||||||
|
* OS, architecture and runner image; lockfiles and pins are tree content) within --max-age
|
||||||
|
* hours (default 24); a newer failure on that tree vetoes an older pass. The
|
||||||
|
* reuse appends a copy that keeps the original timestamp, so check's max age
|
||||||
|
* still measures the real run.
|
||||||
|
*
|
||||||
|
* `record-ci` writes the explicit CI-sourced row for a lane CI carried: it
|
||||||
|
* finds and verifies the run itself (lib/ci-carry.ts) for a command the base's
|
||||||
|
* ship_lanes_from maps, then records source "ci", the runs and profiles, the
|
||||||
|
* run's finish time as ts, bound to HEAD's clean tree; `carried` lists those
|
||||||
|
* rows at the current tree for the ship receipt.
|
||||||
|
*
|
||||||
* MACHINE-LOCAL by design: neither the ledger nor the logs are brain-synced.
|
* MACHINE-LOCAL by design: neither the ledger nor the logs are brain-synced.
|
||||||
* A synced record citing an unsynced log would grade FRESH on a machine where
|
* A synced record citing an unsynced log would grade FRESH on a machine where
|
||||||
* the log doesn't exist.
|
* the log doesn't exist.
|
||||||
@@ -73,6 +90,10 @@ import { resultLine } from "../lib/result-codes";
|
|||||||
import { isUnavailable, loadPolicyFromBase, resolveBaseBranch } from "../lib/ship-policy";
|
import { isUnavailable, loadPolicyFromBase, resolveBaseBranch } from "../lib/ship-policy";
|
||||||
import { versionAtRef } from "../lib/restamp";
|
import { versionAtRef } from "../lib/restamp";
|
||||||
import { bashCommandInvocation, runScriptSync, wtreeFingerprint } from "../lib/script-invocation";
|
import { bashCommandInvocation, runScriptSync, wtreeFingerprint } from "../lib/script-invocation";
|
||||||
|
import { findCarry } from "../lib/ci-carry";
|
||||||
|
import { isConfigUnavailable, loadConfigFromBase } from "../lib/pregate/config";
|
||||||
|
import { buildRunCheck } from "../lib/pregate/run-check";
|
||||||
|
import { runInventory } from "../lib/run-evidence";
|
||||||
|
|
||||||
const BIN_DIR = dirname(Bun.fileURLToPath(import.meta.url));
|
const BIN_DIR = dirname(Bun.fileURLToPath(import.meta.url));
|
||||||
const LOG_MAX_BYTES = 2 * 1024 * 1024;
|
const LOG_MAX_BYTES = 2 * 1024 * 1024;
|
||||||
@@ -93,6 +114,16 @@ interface EvidenceRecord {
|
|||||||
wtree_changed?: string[];
|
wtree_changed?: string[];
|
||||||
log_path?: string;
|
log_path?: string;
|
||||||
redacted?: boolean;
|
redacted?: boolean;
|
||||||
|
/** sha256 of the runtime the run had (identityDigest). */
|
||||||
|
identity?: string;
|
||||||
|
/** The original ts of the row a `run --reuse` copied. */
|
||||||
|
reused_from?: string;
|
||||||
|
/** "ci" for a lane CI carried (record-ci); absent for a local run. */
|
||||||
|
source?: "ci";
|
||||||
|
runs?: string[];
|
||||||
|
profiles?: string[];
|
||||||
|
/** The base policy that mapped a carried lane (record-ci), as `origin/<base>@<sha>:.gstack/pregate.json`. */
|
||||||
|
policy?: string;
|
||||||
/** The runner's own test count (executed for bun/jest/vitest/pytest/rspec; ran with skips included for gstack's shard runner and the remote gate), 0 for ZERO-RUN, or "unknown" when no supported summary was seen. */
|
/** The runner's own test count (executed for bun/jest/vitest/pytest/rspec; ran with skips included for gstack's shard runner and the remote gate), 0 for ZERO-RUN, or "unknown" when no supported summary was seen. */
|
||||||
tests_ran?: number | "unknown";
|
tests_ran?: number | "unknown";
|
||||||
}
|
}
|
||||||
@@ -369,7 +400,15 @@ async function cmdRun(argv: string[]): Promise<number> {
|
|||||||
const sep = argv.indexOf("--");
|
const sep = argv.indexOf("--");
|
||||||
if (li >= 0 && li + 1 < argv.length && (sep < 0 || li < sep)) label = argv[li + 1];
|
if (li >= 0 && li + 1 < argv.length && (sep < 0 || li < sep)) label = argv[li + 1];
|
||||||
if (sep < 0 || sep + 1 >= argv.length) {
|
if (sep < 0 || sep + 1 >= argv.length) {
|
||||||
console.error("usage: gstack-evidence run --label <L> -- <cmd...>");
|
console.error("usage: gstack-evidence run [--reuse [--max-age <hours>]] --label <L> -- <cmd...>");
|
||||||
|
return 2;
|
||||||
|
}
|
||||||
|
const opts = argv.slice(0, sep);
|
||||||
|
const reuse = opts.includes("--reuse") && !opts.includes("--no-reuse");
|
||||||
|
const maxAgeAt = opts.indexOf("--max-age");
|
||||||
|
const reuseMaxAge = maxAgeAt >= 0 ? Number(opts[maxAgeAt + 1]) : 24;
|
||||||
|
if (!Number.isFinite(reuseMaxAge) || reuseMaxAge <= 0) {
|
||||||
|
console.error(`gstack-evidence: --max-age must be a positive number of hours, got: ${JSON.stringify(opts[maxAgeAt + 1])}`);
|
||||||
return 2;
|
return 2;
|
||||||
}
|
}
|
||||||
const cmdArgv = argv.slice(sep + 1);
|
const cmdArgv = argv.slice(sep + 1);
|
||||||
@@ -388,6 +427,7 @@ async function cmdRun(argv: string[]): Promise<number> {
|
|||||||
} catch (e: any) {
|
} catch (e: any) {
|
||||||
warn(`ledger setup failed (${e?.message ?? e}) — result will not be recorded`);
|
warn(`ledger setup failed (${e?.message ?? e}) — result will not be recorded`);
|
||||||
}
|
}
|
||||||
|
if (reuse && paths && reused(paths.file, label, cmdSha, reuseMaxAge)) return 0;
|
||||||
const log = paths ? openLog(paths.logsDir, label, cmdSha) : undefined;
|
const log = paths ? openLog(paths.logsDir, label, cmdSha) : undefined;
|
||||||
|
|
||||||
// Fingerprint the content BEFORE the child runs: a working-tree edit made
|
// Fingerprint the content BEFORE the child runs: a working-tree edit made
|
||||||
@@ -525,6 +565,7 @@ function record(
|
|||||||
const wtreeAfter = currentWtree();
|
const wtreeAfter = currentWtree();
|
||||||
if (wtreeBefore && wtreeAfter && wtreeBefore === wtreeAfter) {
|
if (wtreeBefore && wtreeAfter && wtreeBefore === wtreeAfter) {
|
||||||
rec.wtree = wtreeAfter;
|
rec.wtree = wtreeAfter;
|
||||||
|
rec.identity = identityDigest();
|
||||||
} else if (wtreeBefore || wtreeAfter) {
|
} else if (wtreeBefore || wtreeAfter) {
|
||||||
const changed = wtreeBefore && wtreeAfter ? (git(["diff", "--name-only", wtreeBefore, wtreeAfter]) ?? "").split("\n").map((s) => s.trim()).filter(Boolean) : [];
|
const changed = wtreeBefore && wtreeAfter ? (git(["diff", "--name-only", wtreeBefore, wtreeAfter]) ?? "").split("\n").map((s) => s.trim()).filter(Boolean) : [];
|
||||||
if (changed.length) rec.wtree_changed = changed.slice(0, 5);
|
if (changed.length) rec.wtree_changed = changed.slice(0, 5);
|
||||||
@@ -545,6 +586,118 @@ function record(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* sha256 of the runtime a lane ran under: the bun and node on PATH, OS, architecture and runner image. The lockfiles and
|
||||||
|
* pins the execution identity also names (lib/evidence-bundle.ts) are working-tree content, so the fingerprint binds them.
|
||||||
|
*/
|
||||||
|
function identityDigest(): string {
|
||||||
|
const version = (b: string) => {
|
||||||
|
const r = spawnSync(b, ["--version"], { encoding: "utf-8", timeout: 15000 });
|
||||||
|
return r.status === 0 ? (r.stdout || "").trim().replace(/^v/, "") || null : null;
|
||||||
|
};
|
||||||
|
return sha256(JSON.stringify({ bun: version("bun"), node: version("node"), os: process.platform, arch: process.arch, image: process.env.GSTACK_RUNNER_IMAGE || null }));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* `run --reuse`: the newest local row for this label, command and working tree decides. A counted pass under the same
|
||||||
|
* identity within the max age is copied (original ts kept) and printed as a run would be; anything else runs the command.
|
||||||
|
*/
|
||||||
|
function reused(file: string, label: string, cmdSha: string, maxAgeHours: number): boolean {
|
||||||
|
const wtree = currentWtree();
|
||||||
|
if (!wtree) return false;
|
||||||
|
let records: EvidenceRecord[] = [];
|
||||||
|
try {
|
||||||
|
records = readJsonl<EvidenceRecord>(file);
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
const newest = records.findLast((r) => r.label === label && r.cmd_sha256 === cmdSha && r.wtree === wtree && r.source === undefined);
|
||||||
|
if (!newest || newest.exit !== 0 || typeof newest.tests_ran !== "number" || newest.tests_ran <= 0) return false;
|
||||||
|
const age = Date.now() - Date.parse(newest.ts);
|
||||||
|
if (!(age >= 0 && age <= maxAgeHours * 3600_000)) return false;
|
||||||
|
if (newest.identity !== identityDigest()) return false;
|
||||||
|
try {
|
||||||
|
appendJsonl(file, { ...newest, reused_from: newest.reused_from ?? newest.ts }, { mode: 0o600 });
|
||||||
|
} catch (e: any) {
|
||||||
|
warn(`ledger append failed (${e?.message ?? e}) — running the command instead`);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
console.error(`gstack-evidence: reused label=${label} tree=${wtree.slice(0, 12)} from=${newest.ts}`);
|
||||||
|
console.error(`gstack-evidence: recorded label=${label} exit=0 tests_ran=${newest.tests_ran} log=${newest.log_path ?? "-"}`);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* `record-ci`: carry a lane from CI. Its command must be a key of the base branch's ship_lanes_from; the run is found and
|
||||||
|
* verified here (lib/ci-carry.ts) at HEAD's clean tree, so no caller supplies run ids, a test count or a tree. Never runs the lane.
|
||||||
|
*/
|
||||||
|
function cmdRecordCi(argv: string[]): number {
|
||||||
|
const sep = argv.indexOf("--");
|
||||||
|
const opts = sep < 0 ? argv : argv.slice(0, sep);
|
||||||
|
const get = (name: string) => { const i = opts.indexOf(name); return i >= 0 ? opts[i + 1] : undefined; };
|
||||||
|
const label = get("--label")?.replace(/[^a-zA-Z0-9._-]/g, "_");
|
||||||
|
const base = get("--base");
|
||||||
|
const command = sep < 0 ? "" : argv.slice(sep + 1).join(" ");
|
||||||
|
if (!label || !base || !command || opts.length !== 4) {
|
||||||
|
console.error("usage: gstack-evidence record-ci --label <L> --base <base> -- <command>");
|
||||||
|
return 2;
|
||||||
|
}
|
||||||
|
const none = (reason: string, detail: string) => {
|
||||||
|
console.error(`CARRY_NONE: ${reason}: ${detail}`);
|
||||||
|
console.error(`gstack-evidence record-ci: label=${label} not carried; nothing recorded`);
|
||||||
|
return 1;
|
||||||
|
};
|
||||||
|
const cwd = process.cwd();
|
||||||
|
const loaded = loadConfigFromBase(cwd, base);
|
||||||
|
if (isConfigUnavailable(loaded)) return none("config_unavailable", loaded.detail);
|
||||||
|
if (loaded.errors.length) return none("config_invalid", loaded.errors.map((x) => `${x.path}: ${x.message}`).join("; "));
|
||||||
|
const from = loaded.config.ship_lanes_from;
|
||||||
|
if (!Object.prototype.hasOwnProperty.call(from, command)) return none("not_mapped", `the command is not a key of origin/${base}'s ship_lanes_from`);
|
||||||
|
const m = from[command]!;
|
||||||
|
const check = buildRunCheck(cwd, null, true, base);
|
||||||
|
if ("error" in check) return none("profiles_unavailable", check.error);
|
||||||
|
const head = git(["rev-parse", "HEAD"]);
|
||||||
|
const tree = git(["rev-parse", "HEAD^{tree}"]);
|
||||||
|
if (!head || !tree) return none("no_head", "HEAD is unreadable");
|
||||||
|
const wtree = () => { const fp = wtreeFingerprint({ cwd, timeoutMs: 30000 }); return "tree" in fp ? fp.tree : null; };
|
||||||
|
const v = findCarry(check, { profiles: m.profiles, head, tree, wtree, inventory: () => runInventory(m.inventory, cwd) });
|
||||||
|
if (!v.ok) return none(v.reason, v.detail);
|
||||||
|
let paths: ReturnType<typeof ledgerPath>;
|
||||||
|
try {
|
||||||
|
paths = ledgerPath();
|
||||||
|
mkdirpSync(paths.dir);
|
||||||
|
} catch (e: any) {
|
||||||
|
console.error(`gstack-evidence record-ci: ledger setup failed (${e?.message ?? e}); nothing recorded`);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
const runs = v.runs.map((r) => `${r.run_id}/${r.run_attempt}`);
|
||||||
|
const rec: EvidenceRecord = {
|
||||||
|
ts: new Date(v.runs.map((r) => r.finished_at).sort()[0]!).toISOString(), label, command, cmd_sha256: sha256(command), exit: 0, duration_s: 0,
|
||||||
|
commit: head, tree, dirty: false, wtree: tree, tests_ran: v.tests, source: "ci", runs, profiles: m.profiles, policy: loaded.label,
|
||||||
|
};
|
||||||
|
appendJsonl(paths.file, rec, { mode: 0o600 });
|
||||||
|
console.error(`gstack-evidence: recorded label=${label} exit=0 tests_ran=${v.tests} log=- source=ci runs=${runs.join(",")} policy=${loaded.label}`);
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** `carried`: each label's newest row when it is a CI-sourced row at the current working tree, as the receipt's ci_carried entries. */
|
||||||
|
function cmdCarried(argv: string[]): number {
|
||||||
|
let records: EvidenceRecord[] = [];
|
||||||
|
try {
|
||||||
|
records = readJsonl<EvidenceRecord>(ledgerPath().file);
|
||||||
|
} catch {
|
||||||
|
records = [];
|
||||||
|
}
|
||||||
|
const wtree = currentWtree();
|
||||||
|
const latest = new Map<string, EvidenceRecord>();
|
||||||
|
for (const r of records) latest.set(r.label, r);
|
||||||
|
const carried = [...latest.values()].filter((r) => r.source === "ci" && !!wtree && r.wtree === wtree && Array.isArray(r.runs))
|
||||||
|
.flatMap((r) => r.runs!.map((ref) => { const [run, attempt] = ref.split("/").map(Number); return { lane: r.label, run, attempt, tree: r.wtree! }; }));
|
||||||
|
if (argv.includes("--json")) console.log(JSON.stringify(carried));
|
||||||
|
else for (const c of carried) console.log(`CI_CARRIED: ${c.lane} run=${c.run}/${c.attempt} tree=${c.tree.slice(0, 12)}`);
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
function cmdCheck(argv: string[]): number {
|
function cmdCheck(argv: string[]): number {
|
||||||
// Parse: repeated --label, each optionally followed (anywhere later) by its
|
// Parse: repeated --label, each optionally followed (anywhere later) by its
|
||||||
// own --expect-cmd; pairing is positional — an --expect-cmd binds to the most
|
// own --expect-cmd; pairing is positional — an --expect-cmd binds to the most
|
||||||
@@ -605,7 +758,8 @@ function cmdCheck(argv: string[]): number {
|
|||||||
}
|
}
|
||||||
const testsRan = typeof latest.tests_ran === "number" ? latest.tests_ran : "unknown";
|
const testsRan = typeof latest.tests_ran === "number" ? latest.tests_ran : "unknown";
|
||||||
const changedDuringRun = Array.isArray(latest.wtree_changed) ? latest.wtree_changed.filter((p): p is string => typeof p === "string" && p !== "" && !/[\u0000-\u001f\u007f]/.test(p)).slice(0, 5) : [];
|
const changedDuringRun = Array.isArray(latest.wtree_changed) ? latest.wtree_changed.filter((p): p is string => typeof p === "string" && p !== "" && !/[\u0000-\u001f\u007f]/.test(p)).slice(0, 5) : [];
|
||||||
const detail = `label=${label} exit=${latest.exit} tests_ran=${testsRan} ts=${latest.ts}${latest.log_path ? ` log=${latest.log_path}` : ""}`;
|
const ci = latest.source === "ci" && Array.isArray(latest.runs) ? ` source=ci runs=${latest.runs.filter((s) => /^\d+\/\d+$/.test(s)).join(",")}` : "";
|
||||||
|
const detail = `label=${label} exit=${latest.exit} tests_ran=${testsRan} ts=${latest.ts}${latest.log_path ? ` log=${latest.log_path}` : ""}${ci}`;
|
||||||
let verdict: "FRESH" | "STALE" | "ZERO-RUN" = "FRESH";
|
let verdict: "FRESH" | "STALE" | "ZERO-RUN" = "FRESH";
|
||||||
let reason = "";
|
let reason = "";
|
||||||
if (latest.exit !== 0) {
|
if (latest.exit !== 0) {
|
||||||
@@ -756,6 +910,10 @@ const [, , sub, ...rest] = process.argv;
|
|||||||
try {
|
try {
|
||||||
if (sub === "run") {
|
if (sub === "run") {
|
||||||
process.exit(await cmdRun(rest));
|
process.exit(await cmdRun(rest));
|
||||||
|
} else if (sub === "record-ci") {
|
||||||
|
process.exit(cmdRecordCi(rest));
|
||||||
|
} else if (sub === "carried") {
|
||||||
|
process.exit(cmdCarried(rest));
|
||||||
} else if (sub === "check") {
|
} else if (sub === "check") {
|
||||||
process.exit(cmdCheck(rest));
|
process.exit(cmdCheck(rest));
|
||||||
} else if (sub === "bundle") {
|
} else if (sub === "bundle") {
|
||||||
@@ -765,7 +923,7 @@ try {
|
|||||||
} else if (sub === "verify") {
|
} else if (sub === "verify") {
|
||||||
process.exit(cmdVerify(rest));
|
process.exit(cmdVerify(rest));
|
||||||
} else {
|
} else {
|
||||||
console.error("usage: gstack-evidence run|check|bundle|ancestor|verify ...");
|
console.error("usage: gstack-evidence run|record-ci|carried|check|bundle|ancestor|verify ...");
|
||||||
process.exit(2);
|
process.exit(2);
|
||||||
}
|
}
|
||||||
} catch (e: any) {
|
} catch (e: any) {
|
||||||
|
|||||||
+6
-2
@@ -4,7 +4,7 @@
|
|||||||
*
|
*
|
||||||
* gstack-pregate [run] [--stage preflight|tests|all] [--only <check,..>] [--base <branch>]
|
* gstack-pregate [run] [--stage preflight|tests|all] [--only <check,..>] [--base <branch>]
|
||||||
* [--allow-repo-commands] [--policy-from base|head] [--out <pregate.json>]
|
* [--allow-repo-commands] [--policy-from base|head] [--out <pregate.json>]
|
||||||
* [--timeout <s>] [--stall-timeout <s>] [--remote --spend-cap <usd>] [--run <id>] [--from-run <id|url>] [--explain] [--json]
|
* [--timeout <s>] [--stall-timeout <s>] [--remote --spend-cap <usd>] [--run <id>] [--from-run <id|url>] [--no-ci-carry] [--explain] [--json]
|
||||||
* gstack-pregate verify [--out <pregate.json>] [--json] publication check: current tree, no fail/incomplete, obligations cleared
|
* gstack-pregate verify [--out <pregate.json>] [--json] publication check: current tree, no fail/incomplete, obligations cleared
|
||||||
* gstack-pregate clear --remote <workflow.yml/job> --run-url <url> | --run <id|url> [--run …] [--allow-repo-commands] | --bundle-lane <label> | --receipt <file> [--out <pregate.json>]
|
* gstack-pregate clear --remote <workflow.yml/job> --run-url <url> | --run <id|url> [--run …] [--allow-repo-commands] | --bundle-lane <label> | --receipt <file> [--out <pregate.json>]
|
||||||
* gstack-pregate lanes [--explain] the lane list and platform tiers read from .github/workflows
|
* gstack-pregate lanes [--explain] the lane list and platform tiers read from .github/workflows
|
||||||
@@ -94,6 +94,9 @@ const HELP = `gstack-pregate — cheap checks before anything paid or remote (do
|
|||||||
--from-run <id|url> a Free Tests run on this exact tree, verified against the base branch's run profile for this OS
|
--from-run <id|url> a Free Tests run on this exact tree, verified against the base branch's run profile for this OS
|
||||||
whose workflow holds a local lane (docs/pregate.md#from-run): the selected test files its tree.json lists count as passed and
|
whose workflow holds a local lane (docs/pregate.md#from-run): the selected test files its tree.json lists count as passed and
|
||||||
are recorded with its receipt; the rest still run. A refused run prints REUSE: refused and runs everything
|
are recorded with its receipt; the rest still run. A refused run prints REUSE: refused and runs everything
|
||||||
|
--no-ci-carry without --from-run, the tests stage looks up the newest verified CI run at HEAD's clean tree under
|
||||||
|
those profiles itself and uses it (CI_CARRY: accepted <receipt> or CI_CARRY: none (<reason>);
|
||||||
|
docs/pregate.md#ci-carry); this flag skips that lookup
|
||||||
--explain print each selected test's provenance and each uncertain mapping
|
--explain print each selected test's provenance and each uncertain mapping
|
||||||
--json one JSON object on stdout; a refusal before any check runs (unavailable base, invalid pin
|
--json one JSON object on stdout; a refusal before any check runs (unavailable base, invalid pin
|
||||||
file, WTREE_UNAVAILABLE) is {ok:false,status:'refused',code,detail,fix,docs} with exit 3,
|
file, WTREE_UNAVAILABLE) is {ok:false,status:'refused',code,detail,fix,docs} with exit 3,
|
||||||
@@ -112,7 +115,7 @@ function flag(args: string[], name: string): string | undefined { const i = args
|
|||||||
|
|
||||||
/** Flags each verb accepts: `[flags that take a value, bare flags]`. Anything else is usage (exit 2), never a silent default run. */
|
/** Flags each verb accepts: `[flags that take a value, bare flags]`. Anything else is usage (exit 2), never a silent default run. */
|
||||||
const VERB_FLAGS: Record<string, [string[], string[]]> = {
|
const VERB_FLAGS: Record<string, [string[], string[]]> = {
|
||||||
run: [['--stage', '--only', '--base', '--policy-from', '--out', '--timeout', '--stall-timeout', '--spend-cap', '--run', '--from-run'], ['--allow-repo-commands', '--explain', '--json', '--remote']],
|
run: [['--stage', '--only', '--base', '--policy-from', '--out', '--timeout', '--stall-timeout', '--spend-cap', '--run', '--from-run'], ['--allow-repo-commands', '--explain', '--json', '--remote', '--no-ci-carry']],
|
||||||
verify: [['--out'], ['--json']],
|
verify: [['--out'], ['--json']],
|
||||||
clear: [['--remote', '--run-url', '--run', '--bundle-lane', '--receipt', '--out'], ['--allow-repo-commands']],
|
clear: [['--remote', '--run-url', '--run', '--bundle-lane', '--receipt', '--out'], ['--allow-repo-commands']],
|
||||||
lanes: [[], ['--explain', '--json']],
|
lanes: [[], ['--explain', '--json']],
|
||||||
@@ -179,6 +182,7 @@ async function cmdRun(root: string, rest: string[], json: boolean): Promise<numb
|
|||||||
}
|
}
|
||||||
const ctx = loaded.ctx;
|
const ctx = loaded.ctx;
|
||||||
if (fromRun !== undefined) ctx.fromRun = { ref: fromRun, check: buildRunCheck(root, null, allow, base) };
|
if (fromRun !== undefined) ctx.fromRun = { ref: fromRun, check: buildRunCheck(root, null, allow, base) };
|
||||||
|
if (rest.includes('--no-ci-carry')) ctx.ciCarry = false;
|
||||||
ensureTmpExcluded(root);
|
ensureTmpExcluded(root);
|
||||||
for (const signal of ['SIGINT', 'SIGTERM'] as const) {
|
for (const signal of ['SIGINT', 'SIGTERM'] as const) {
|
||||||
process.once(signal, () => {
|
process.once(signal, () => {
|
||||||
|
|||||||
@@ -172,6 +172,12 @@ function reviewedProduct(repoRoot: string, base: string, head: string, pregateFi
|
|||||||
return typeof id === 'string' ? { head_tree: tree, product_patch_id: id || 'none' } : { head_tree: tree };
|
return typeof id === 'string' ? { head_tree: tree, product_patch_id: id || 'none' } : { head_tree: tree };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** The lanes CI carried at the checked-out tree, from this machine's evidence ledger (gstack-evidence carried); undefined when none. */
|
||||||
|
function ciCarried(): ShipReceipt['ci_carried'] {
|
||||||
|
const r = runScriptSync(path.join(path.dirname(Bun.fileURLToPath(import.meta.url)), 'gstack-evidence'), ['carried', '--json'], { encoding: 'utf8', timeout: 60_000 });
|
||||||
|
try { const list = JSON.parse(r.stdout); return Array.isArray(list) && list.length ? list : undefined; } catch { return undefined; }
|
||||||
|
}
|
||||||
|
|
||||||
function cmdWrite(rest: string[], json: boolean): number {
|
function cmdWrite(rest: string[], json: boolean): number {
|
||||||
const repoRoot = git(['rev-parse', '--show-toplevel']) ?? process.cwd();
|
const repoRoot = git(['rev-parse', '--show-toplevel']) ?? process.cwd();
|
||||||
const checkedOut = git(['rev-parse', 'HEAD'], repoRoot);
|
const checkedOut = git(['rev-parse', 'HEAD'], repoRoot);
|
||||||
@@ -262,7 +268,7 @@ function cmdWrite(rest: string[], json: boolean): number {
|
|||||||
version: flag(rest, '--version') ?? receiptVersion(repoRoot, head, base), base,
|
version: flag(rest, '--version') ?? receiptVersion(repoRoot, head, base), base,
|
||||||
gated_tree: flag(rest, '--gated-tree') ?? tree?.gated_tree, predecessor: flag(rest, '--predecessor') ?? tree?.predecessor?.recorded_head ?? undefined,
|
gated_tree: flag(rest, '--gated-tree') ?? tree?.gated_tree, predecessor: flag(rest, '--predecessor') ?? tree?.predecessor?.recorded_head ?? undefined,
|
||||||
tree: tree?.tree, gate_reuse: tree ? `${tree.gate_reuse} (${tree.gate_reuse_reason})` : undefined,
|
tree: tree?.tree, gate_reuse: tree ? `${tree.gate_reuse} (${tree.gate_reuse_reason})` : undefined,
|
||||||
gate: jsonArg(flag(rest, '--gate')), ci: jsonArg(flag(rest, '--ci')),
|
gate: jsonArg(flag(rest, '--gate')), ci: jsonArg(flag(rest, '--ci')), ci_carried: ciCarried(),
|
||||||
spend_usd: flag(rest, '--spend-usd') !== undefined ? (Number.isFinite(Number(flag(rest, '--spend-usd'))) ? Number(flag(rest, '--spend-usd')) : flag(rest, '--spend-usd')) : undefined,
|
spend_usd: flag(rest, '--spend-usd') !== undefined ? (Number.isFinite(Number(flag(rest, '--spend-usd'))) ? Number(flag(rest, '--spend-usd')) : flag(rest, '--spend-usd')) : undefined,
|
||||||
session_kind: sessionKind, artifacts_consumed: flag(rest, '--artifacts-consumed'), run: flag(rest, '--run'), thread: flag(rest, '--thread'),
|
session_kind: sessionKind, artifacts_consumed: flag(rest, '--artifacts-consumed'), run: flag(rest, '--run'), thread: flag(rest, '--thread'),
|
||||||
policy: flag(rest, '--policy') ?? tree?.policy, queue_mode: flag(rest, '--queue-mode'), preregistration: flag(rest, '--preregistration'), history: flag(rest, '--history'),
|
policy: flag(rest, '--policy') ?? tree?.policy, queue_mode: flag(rest, '--queue-mode'), preregistration: flag(rest, '--preregistration'), history: flag(rest, '--history'),
|
||||||
|
|||||||
+117
-2
@@ -90,6 +90,7 @@ the byte-pinned `/ship` renders in `test/fixtures/golden/`).
|
|||||||
| `remote_gate` | the repo's remote test gate command; under 8 cores the local test tier becomes a `requires-remote` obligation naming it, and `--remote` runs it (see [remote gate](#remote-gate)) |
|
| `remote_gate` | the repo's remote test gate command; under 8 cores the local test tier becomes a `requires-remote` obligation naming it, and `--remote` runs it (see [remote gate](#remote-gate)) |
|
||||||
| `run_profiles` | name → `{ workflow, events, jobs, artifact, max_age_hours?, os? }`: the GitHub Actions runs `clear --run` verifies as evidence on the identical tree; a profile named after a lane (`free-tests.yml/macos-named-regressions`) clears that lane (see [run evidence](#run-evidence)). `--from-run` and the main-push reuse job verify runs against them too ([from-run](#from-run), [CI reuse](#ci-reuse)) |
|
| `run_profiles` | name → `{ workflow, events, jobs, artifact, max_age_hours?, os? }`: the GitHub Actions runs `clear --run` verifies as evidence on the identical tree; a profile named after a lane (`free-tests.yml/macos-named-regressions`) clears that lane (see [run evidence](#run-evidence)). `--from-run` and the main-push reuse job verify runs against them too ([from-run](#from-run), [CI reuse](#ci-reuse)) |
|
||||||
| `remote_gate_runs` | `{ profiles, inventory, skipped? }`: the run profiles that together clear `pregate.json/remote_gate`, the command that prints the remote gate's test files, one per line, which their runs must cover, and an optional command that prints which selected files the gate itself never runs (see [run evidence](#run-evidence)) |
|
| `remote_gate_runs` | `{ profiles, inventory, skipped? }`: the run profiles that together clear `pregate.json/remote_gate`, the command that prints the remote gate's test files, one per line, which their runs must cover, and an optional command that prints which selected files the gate itself never runs (see [run evidence](#run-evidence)) |
|
||||||
|
| `ship_lanes_from` | maps each exact /ship lane command to `{ profiles, inventory }`: the run profiles whose verified runs at HEAD's clean tree carry that lane instead of running it, and the command listing its test files, which their runs must cover (see [CI carry](#ci-carry)); read from the base branch only |
|
||||||
| `allowlist` | `[{ finding, paths?, pattern?, reason }]`: benign findings the `secrets` check accepts and names as allowlisted, by gstack-redact finding id or non-secret category, path globs and a pattern anchored at the finding; never `env.value` (see [allowlist](#allowlist)) |
|
| `allowlist` | `[{ finding, paths?, pattern?, reason }]`: benign findings the `secrets` check accepts and names as allowlisted, by gstack-redact finding id or non-secret category, path globs and a pattern anchored at the finding; never `env.value` (see [allowlist](#allowlist)) |
|
||||||
|
|
||||||
Only the `origin/<base>` copy or the user-owned file below is honoured, so a
|
Only the `origin/<base>` copy or the user-owned file below is honoured, so a
|
||||||
@@ -173,7 +174,7 @@ The list lives in its own file, not in `pregate.json`, because older pregate rea
|
|||||||
|
|
||||||
```
|
```
|
||||||
gstack-pregate [--stage preflight|tests|all] [--only <check,..>] [--allow-repo-commands] [--policy-from base|head]
|
gstack-pregate [--stage preflight|tests|all] [--only <check,..>] [--allow-repo-commands] [--policy-from base|head]
|
||||||
[--timeout <s>] [--stall-timeout <s>] [--remote --spend-cap <usd>] [--run <id>] [--from-run <id|url>] [--explain] [--json]
|
[--timeout <s>] [--stall-timeout <s>] [--remote --spend-cap <usd>] [--run <id>] [--from-run <id|url>] [--no-ci-carry] [--explain] [--json]
|
||||||
gstack-pregate verify | clear --remote <workflow.yml/job> --run-url <url> | lanes | validate | init
|
gstack-pregate verify | clear --remote <workflow.yml/job> --run-url <url> | lanes | validate | init
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -493,8 +494,15 @@ the selected local files as a `requires-remote` obligation with the command to
|
|||||||
run, `pregate.json/remote_gate` (the declared command) or
|
run, `pregate.json/remote_gate` (the declared command) or
|
||||||
`test_backend/<backend>` (gstack's `scripts/ubicloud/ubi-runner.sh run --src
|
`test_backend/<backend>` (gstack's `scripts/ubicloud/ubi-runner.sh run --src
|
||||||
<repo> -- '<runner> <files>'`). Without either, the tier runs locally under
|
<repo> -- '<runner> <files>'`). Without either, the tier runs locally under
|
||||||
the supervision above. Close the obligation one of three ways:
|
the supervision above. Close the obligation one of four ways:
|
||||||
|
|
||||||
|
- by default, when the base declares `remote_gate_runs`, let CI clear it: the
|
||||||
|
tests stage clears it on its own from a verified run at this exact tree
|
||||||
|
([CI carry](#ci-carry)), and the uncleared obligation's detail names that
|
||||||
|
route first (`push, then gstack-ci-carry wait --base <base> and rerun
|
||||||
|
gstack-pregate --allow-repo-commands`, the whole pregate, so the record keeps
|
||||||
|
its preflight checks) and the VM only as the opt-in (under `--no-ci-carry` it
|
||||||
|
names the VM route); or
|
||||||
- after running it, `gstack-pregate clear --remote pregate.json/remote_gate
|
- after running it, `gstack-pregate clear --remote pregate.json/remote_gate
|
||||||
--receipt <file>` with the gate's output, validated as `--remote` validates it
|
--receipt <file>` with the gate's output, validated as `--remote` validates it
|
||||||
(a run URL is refused, `PREGATE_RECEIPT_INVALID` for a bad receipt; a receipt
|
(a run URL is refused, `PREGATE_RECEIPT_INVALID` for a bad receipt; a receipt
|
||||||
@@ -907,6 +915,113 @@ as a usage error. It changes nothing in the preflight stage: with `--stage all`
|
|||||||
the preflight checks, the secrets scan included, run as before, and
|
the preflight checks, the secrets scan included, run as before, and
|
||||||
`--stage tests` runs no preflight check in that invocation, with or without it.
|
`--stage tests` runs no preflight check in that invocation, with or without it.
|
||||||
|
|
||||||
|
<a id="ci-carry"></a>
|
||||||
|
### CI carry: pregate and /ship use CI's run instead of running again (GSTA-258)
|
||||||
|
|
||||||
|
CI is the one full test run. When a verified run already passed at the exact
|
||||||
|
tree being tested, pregate's tests stage and /ship's test lanes use it instead
|
||||||
|
of running the suite a second time. `lib/ci-carry.ts` `findCarry` decides, and
|
||||||
|
it accepts a run only when all of these hold:
|
||||||
|
|
||||||
|
- The working tree is HEAD's committed tree (the `gstack-wtree` fingerprint
|
||||||
|
equals `HEAD^{tree}`), read before the lookup and again after it. A dirty
|
||||||
|
tree, an untracked file or an edit made during the lookup never carries.
|
||||||
|
- The newest of HEAD's runs (up to 10) under each named profile passes
|
||||||
|
`verifyRun` above: same-repository head (a fork run never counts), the
|
||||||
|
profile's workflow, event and jobs, the newest attempt, no newer red or
|
||||||
|
running run on HEAD, within `max_age_hours`, and a `tree.json` naming HEAD's
|
||||||
|
tree. A `pull_request` run counts only when the pull request already
|
||||||
|
contained its base.
|
||||||
|
- Every run executed tests, and with an inventory command, the runs'
|
||||||
|
`tree.json` files cover every file it prints. Each run is read once more
|
||||||
|
afterward (`run_changed`).
|
||||||
|
|
||||||
|
Security and advisory scans, the sealed guard, guards and every preflight-stage
|
||||||
|
check always run; only test lanes carry.
|
||||||
|
|
||||||
|
**The tests stage.** Without `--from-run`, when the base branch declares
|
||||||
|
`run_profiles`, `gstack-pregate` looks up the newest verified run at HEAD under
|
||||||
|
the profiles `--from-run` would use and applies it as `--from-run` does: the
|
||||||
|
selected local files its `tree.json` lists count as passed, and only the rest
|
||||||
|
run. On a small machine a selection the run covers records no remote gate
|
||||||
|
obligation. It prints `CI_CARRY: accepted <receipt>` or `CI_CARRY: none
|
||||||
|
(<reason>)` on stderr; `--no-ci-carry` skips the lookup.
|
||||||
|
|
||||||
|
When files are left over, the remote gate obligation still clears from the
|
||||||
|
same run when the base declares `remote_gate_runs` naming that run's profile:
|
||||||
|
the run must pass every `clear --run` check, with the carried files counted
|
||||||
|
as selected files. So a remainder of only files the gate's `skipped` command
|
||||||
|
names (gstack's paid evals, which the gate itself reports as
|
||||||
|
`PREGATE_REMOTE_PAID_ONLY` with `tests=0`) clears with `skipped=<n>` in the
|
||||||
|
receipt instead of becoming an obligation nothing can clear. The line is
|
||||||
|
`CI_CARRY: cleared pregate.json/remote_gate <receipt>`, or `CI_CARRY:
|
||||||
|
pregate.json/remote_gate stays uncleared (<reason>)` when a check refuses, for
|
||||||
|
example a gate inventory file missing from the run's `tree.json`.
|
||||||
|
|
||||||
|
**gstack's own gate.** gstack's `.gstack/pregate.json` declares
|
||||||
|
`remote_gate_runs` with the `ci-reuse/free-tests` profile, so on gstack a green
|
||||||
|
Free Tests run at the same tree is how the remote gate obligation clears, and
|
||||||
|
the Ubicloud VM (`gstack-pregate --stage tests --remote`) is an opt-in for when
|
||||||
|
no such run exists. The order after a push: wait for Free Tests
|
||||||
|
(`gstack-ci-carry wait --base main`), then `gstack-pregate
|
||||||
|
--allow-repo-commands`. A `pull_request` run tests GitHub's merge commit, so on
|
||||||
|
a branch that does not contain the base tip it tested another tree and
|
||||||
|
`CI_CARRY` says `none (no_verified_run: … tree_mismatch)`: dispatch Free Tests
|
||||||
|
on the branch
|
||||||
|
(`gh workflow run free-tests.yml --ref <branch>`), which tests HEAD's own tree,
|
||||||
|
and wait again. The preflight stage (generated-file freshness, secrets, strays
|
||||||
|
and the other mechanical checks) and the sealed guard are unchanged and always
|
||||||
|
run here. The macOS and Windows lane
|
||||||
|
obligations still clear with `gstack-pregate clear --remote <lane> --run-url
|
||||||
|
<url>` from their CI runs.
|
||||||
|
|
||||||
|
**/ship lanes: `ship_lanes_from`.** A lane carries only when the base branch's
|
||||||
|
`.gstack/pregate.json` maps its exact command (the evidence ledger binds the
|
||||||
|
command's sha256; a label alone never carries) to the profiles whose runs carry
|
||||||
|
it and the command that lists its test files:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"ship_lanes_from": {
|
||||||
|
"bun run test": { "profiles": ["ci-reuse/free-tests"], "inventory": "bun --no-env-file --config=/dev/null run scripts/test-free-shards.ts --list | sed -n 's/^ //p'" }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
A carried lane prints `LANE: <label> carried run=<id>/<attempt> tests_ran=<n>`
|
||||||
|
and writes an explicit CI-sourced ledger row (`gstack-evidence record-ci
|
||||||
|
--label <label> --base <base> -- <command>`, which finds and verifies the run
|
||||||
|
itself, so no caller hands it run ids or a test count: `source: "ci"`, its runs
|
||||||
|
and profiles, the run's finish time as `ts`, bound to the clean tree), never a
|
||||||
|
fictitious local run. `gstack-evidence check` grades it
|
||||||
|
like any row and names `source=ci runs=…`, so `finish`'s FRESH check and the
|
||||||
|
24-hour age limit apply to the CI run's own time. A mapped lane that cannot
|
||||||
|
carry prints `CI_CARRY: none label=<label> (<reason>)` and runs here. Every
|
||||||
|
other lane runs through `gstack-evidence run --reuse`: the newest row for its
|
||||||
|
label, command and working tree is reused when it passed with a counted run
|
||||||
|
under the same runtime (bun and node versions, OS, architecture, runner image)
|
||||||
|
within 24 hours; a newer failure on that tree vetoes an older pass, and the
|
||||||
|
reused copy keeps the original timestamp. The cache is this machine's ledger
|
||||||
|
only; across machines, CI is the trust boundary.
|
||||||
|
|
||||||
|
**Finish, push and accept.** `gstack-ship-release finish` commits the release
|
||||||
|
metadata, so the stamped tree has no CI run yet. A mapped lane that is not
|
||||||
|
already FRESH is deferred (`EVIDENCE: DEFERRED`), and the pregate runs only its
|
||||||
|
preflight stage on the stamped tree, written to its own file so the full
|
||||||
|
`.gstack/tmp/pregate.json` stays in place until `accept` replaces it. `gstack-ship-release push --early` (the
|
||||||
|
Phase 2 push that lets CI run beside the review) runs the secrets scan and the
|
||||||
|
sealed guard before it pushes. After the final push,
|
||||||
|
`gstack-ship-release accept --base <base> --lane …` waits (bounded by `--wait`)
|
||||||
|
for the run at the pushed HEAD, stops on a red or still-running run, carries
|
||||||
|
each mapped lane from the green run at that exact tree, reuses or runs the rest,
|
||||||
|
checks every lane FRESH and reruns the pregate there, whose tests stage carries
|
||||||
|
the same run. `gstack-ship-receipt write` records the carried runs as
|
||||||
|
`ci_carried: [{ lane, run, attempt, tree }]`, and the `SHIP_RECEIPT:` line names
|
||||||
|
them (`ci_carried=<lane>:<run>/<attempt>`).
|
||||||
|
|
||||||
|
**`gstack-ci-carry`.** `find [--profile <name>]… [--command <exact lane command>
|
||||||
|
--allow-repo-commands]` prints the verdict for HEAD; `wait [--timeout <s>]`
|
||||||
|
polls the newest run at HEAD until it completes (run it in the background) and
|
||||||
|
prints `CI_WAIT: green|red|running|none`, then `find`'s verdict on green.
|
||||||
|
|
||||||
<a id="ci-reuse"></a>
|
<a id="ci-reuse"></a>
|
||||||
### Main pushes reuse the merged pull request's run
|
### Main pushes reuse the merged pull request's run
|
||||||
|
|
||||||
|
|||||||
@@ -139,7 +139,9 @@ harness-generic; only the thread and the PR tool are Capy.
|
|||||||
default branch.
|
default branch.
|
||||||
4. **Run `/ship` unattended** with the `GSTACK_LAUNCH` assignments
|
4. **Run `/ship` unattended** with the `GSTACK_LAUNCH` assignments
|
||||||
exported: it runs the pregate (on a machine under 8 cores the test tier
|
exported: it runs the pregate (on a machine under 8 cores the test tier
|
||||||
needs `gstack-pregate --stage tests --remote`, about $0.06 on a Ubicloud VM
|
clears from a verified CI run at the pushed tree when the base declares
|
||||||
|
`remote_gate_runs` ([CI carry](pregate.md#ci-carry)); otherwise it needs
|
||||||
|
`gstack-pregate --stage tests --remote`, about $0.06 on a Ubicloud VM
|
||||||
or free locally for small selections), both reviews, the gate-integrity
|
or free locally for small selections), both reviews, the gate-integrity
|
||||||
list, the docs audit, and writes the ship receipt into the PR body
|
list, the docs audit, and writes the ship receipt into the PR body
|
||||||
(`gstack-ship-receipt write --pr <n>`). With `queue_mode:
|
(`gstack-ship-receipt write --pr <n>`). With `queue_mode:
|
||||||
|
|||||||
+132
@@ -0,0 +1,132 @@
|
|||||||
|
/**
|
||||||
|
* ci-carry: CI is the one full test run (GSTA-258, R65 item 1). When a verified
|
||||||
|
* CI run already passed at the exact tree /ship or pregate is about to test,
|
||||||
|
* they use it instead of running the suite again.
|
||||||
|
*
|
||||||
|
* `findCarry` looks up the newest verified run of each named run profile (the
|
||||||
|
* base branch's `.gstack/pregate.json` `run_profiles`) for HEAD and accepts it
|
||||||
|
* only when every rule holds:
|
||||||
|
*
|
||||||
|
* the working tree is HEAD's committed tree (gstack-wtree), before and after
|
||||||
|
* the lookup, so admitted dirty or untracked work never carries;
|
||||||
|
* each run passes verifyRun (same-repository head, the profile's workflow,
|
||||||
|
* event and jobs, the newest attempt, no newer red or running run on HEAD,
|
||||||
|
* max age, the run's own tree.json naming HEAD's tree);
|
||||||
|
* every run executed tests;
|
||||||
|
* with an inventory command, the runs' tree.json files cover every file it
|
||||||
|
* prints (a lane carries only what CI provably ran);
|
||||||
|
* every run still stands after the inventory ran (runChanged).
|
||||||
|
*
|
||||||
|
* Anything else returns a refusal with a reason code, and the caller runs the
|
||||||
|
* suite as before. Security and advisory scans and the sealed guard are never
|
||||||
|
* carried: callers carry test lanes only.
|
||||||
|
*
|
||||||
|
* waitForRun polls the newest run of one workflow on HEAD until it completes,
|
||||||
|
* bounded by a timeout, and returns green, red with the failing jobs, still
|
||||||
|
* running, or none.
|
||||||
|
*/
|
||||||
|
import { runChanged, runReceiptFields, uncoveredFiles, verifyRun, type ApiRun, type RunDeps, type RunEvidence, type RunProfile, type RunSource } from './run-evidence';
|
||||||
|
|
||||||
|
export type CarryRefusal = 'dirty_tree' | 'no_profile' | 'no_run' | 'no_verified_run' | 'tree_mismatch' | 'no_tests'
|
||||||
|
| 'inventory_failed' | 'inventory_empty' | 'inventory_uncovered' | 'run_changed' | 'tree_changed' | 'api_error';
|
||||||
|
export type CarryRefused = { ok: false; reason: CarryRefusal; detail: string };
|
||||||
|
export type CarryVerdict = { ok: true; runs: RunEvidence[]; tests: number; receipt: string } | CarryRefused;
|
||||||
|
|
||||||
|
/** How many of HEAD's runs per workflow are tried, newest first. */
|
||||||
|
const CANDIDATES = 10;
|
||||||
|
const no = (reason: CarryRefusal, detail: string): CarryRefused => ({ ok: false, reason, detail });
|
||||||
|
const newestFirst = (a: ApiRun, b: ApiRun) => (b.created_at ?? '').localeCompare(a.created_at ?? '') || b.id - a.id;
|
||||||
|
|
||||||
|
/** The newest run of `profile` on `head` that verifies; a refusal lists each candidate's reason, newest first. */
|
||||||
|
function newestVerified(src: RunSource, name: string, profile: RunProfile, head: string): { ok: true; evidence: RunEvidence } | CarryRefused {
|
||||||
|
const listed = src.deps.runsForHead(profile.workflow, head).filter(r => profile.events.includes(r.event)).sort(newestFirst).slice(0, CANDIDATES);
|
||||||
|
if (!listed.length) return no('no_run', `no ${profile.workflow} run (${profile.events.join(', ')}) on ${head.slice(0, 12)} yet`);
|
||||||
|
const refusals: string[] = [];
|
||||||
|
for (const run of listed) {
|
||||||
|
const v = verifyRun(src.deps, { repo: src.repo, runRef: String(run.id), profileName: name, profile, vetoHeads: [head], now: src.now });
|
||||||
|
if (v.ok) return v;
|
||||||
|
refusals.push(`run ${run.id}: ${v.reason}`);
|
||||||
|
}
|
||||||
|
return no('no_verified_run', `${name}: ${refusals.join('; ')}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The CI evidence for HEAD's tree under `profiles` (one verified run each), or why there is none. `wtree` reads the
|
||||||
|
* working-tree fingerprint (before the lookup and after it); `inventory` lists the files the runs must cover.
|
||||||
|
*/
|
||||||
|
export function findCarry(src: RunSource, o: { profiles: string[]; head: string; tree: string; wtree: () => string | null; inventory?: () => string[] | { error: string } }): CarryVerdict {
|
||||||
|
const before = o.wtree();
|
||||||
|
if (before !== o.tree) return no('dirty_tree', `the working tree (${before ? before.slice(0, 12) : 'fingerprint unavailable'}) is not HEAD's committed tree ${o.tree.slice(0, 12)}; only a clean, committed tree carries`);
|
||||||
|
if (!o.profiles.length) return no('no_profile', 'no run profile is named');
|
||||||
|
const runs: RunEvidence[] = [];
|
||||||
|
try {
|
||||||
|
for (const name of o.profiles) {
|
||||||
|
const profile = src.profiles[name];
|
||||||
|
if (!profile) return no('no_profile', `${name} is not a run_profiles name on the base branch`);
|
||||||
|
const v = newestVerified(src, name, profile, o.head);
|
||||||
|
if (!v.ok) return v;
|
||||||
|
runs.push(v.evidence);
|
||||||
|
}
|
||||||
|
const other = runs.find(e => e.tree !== o.tree);
|
||||||
|
if (other) return no('tree_mismatch', `run ${other.run_id} tested tree ${other.tree.slice(0, 12)}, not ${o.tree.slice(0, 12)}`);
|
||||||
|
const empty = runs.find(e => !e.files.length || e.tests === 0);
|
||||||
|
if (empty) return no('no_tests', `run ${empty.run_id}'s tree.json ${empty.files.length ? 'records 0 tests' : 'lists no test files'}`);
|
||||||
|
if (o.inventory) {
|
||||||
|
const listed = o.inventory();
|
||||||
|
if ('error' in listed) return no('inventory_failed', listed.error);
|
||||||
|
if (!listed.length) return no('inventory_empty', 'the inventory command printed no files');
|
||||||
|
const missing = uncoveredFiles(listed, runs);
|
||||||
|
if (missing.length) return no('inventory_uncovered', `${missing.length} of ${listed.length} inventory file(s) are in no run's tree.json: ${missing.slice(0, 5).join(', ')}${missing.length > 5 ? ', …' : ''}`);
|
||||||
|
}
|
||||||
|
for (const e of runs) {
|
||||||
|
const changed = runChanged(src.deps, e.run_id, e.run_attempt);
|
||||||
|
if (changed) return no('run_changed', changed);
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
return no('api_error', `reading runs failed: ${String((e as Error)?.message ?? e).slice(0, 300)}`);
|
||||||
|
}
|
||||||
|
const after = o.wtree();
|
||||||
|
if (after !== o.tree) return no('tree_changed', `the working tree changed while the runs were verified (now ${after ? after.slice(0, 12) : 'unreadable'})`);
|
||||||
|
const tests = runs.reduce((n, e) => n + (e.tests ?? e.files.length), 0);
|
||||||
|
return { ok: true, runs, tests, receipt: `ci-carry tree=${o.tree.slice(0, 12)} tests=${tests} ${runReceiptFields(runs, src.now)}` };
|
||||||
|
}
|
||||||
|
|
||||||
|
export type WaitVerdict = { state: 'green'; run: ApiRun } | { state: 'red'; run: ApiRun; failing: string[] } | { state: 'running'; run: ApiRun } | { state: 'none'; reason: string };
|
||||||
|
const PASSING = ['success', 'skipped', 'neutral'];
|
||||||
|
|
||||||
|
/** Poll the newest `workflow` run on `head` (among `events`) until it completes or `timeoutMs` passes. */
|
||||||
|
export async function waitForRun(deps: RunDeps, o: { workflow: string; events: string[]; head: string; timeoutMs: number; intervalMs: number; sleep: (ms: number) => Promise<void>; clock: () => number }): Promise<WaitVerdict> {
|
||||||
|
const end = o.clock() + o.timeoutMs;
|
||||||
|
for (;;) {
|
||||||
|
const newest = deps.runsForHead(o.workflow, o.head).filter(r => o.events.includes(r.event)).sort(newestFirst)[0];
|
||||||
|
if (newest?.status === 'completed') {
|
||||||
|
if (newest.conclusion === 'success') return { state: 'green', run: newest };
|
||||||
|
const failing = deps.jobs(newest.id, newest.run_attempt).filter(j => !PASSING.includes(j.conclusion ?? '')).map(j => j.name);
|
||||||
|
return { state: 'red', run: newest, failing };
|
||||||
|
}
|
||||||
|
if (o.clock() >= end) return newest ? { state: 'running', run: newest } : { state: 'none', reason: `no ${o.workflow} run on ${o.head.slice(0, 12)} after ${Math.round(o.timeoutMs / 1000)} s` };
|
||||||
|
await o.sleep(Math.max(1, Math.min(o.intervalMs, end - o.clock())));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** `ship_lanes_from` (base branch only): exact lane command → the profiles whose runs carry it and the command listing its test files. */
|
||||||
|
export interface ShipLaneMapping { profiles: string[]; inventory: string }
|
||||||
|
|
||||||
|
/** `<path>: <message>` errors for `ship_lanes_from`; empty when valid. Shared with lib/pregate/config.ts. */
|
||||||
|
export function shipLanesErrors(raw: unknown, profileNames: readonly string[]): Array<{ path: string; message: string }> {
|
||||||
|
if (raw === undefined) return [];
|
||||||
|
if (!raw || typeof raw !== 'object' || Array.isArray(raw)) return [{ path: 'ship_lanes_from', message: 'must be an object of exact lane command → { profiles: [run_profiles names], inventory: command }' }];
|
||||||
|
const errors: Array<{ path: string; message: string }> = [];
|
||||||
|
const oneLine = (s: unknown) => typeof s === 'string' && !!s.trim() && s.length <= 512 && !/[\0\r\n]/.test(s);
|
||||||
|
for (const [command, m] of Object.entries(raw as Record<string, unknown>)) {
|
||||||
|
const at = `ship_lanes_from[${JSON.stringify(command)}]`;
|
||||||
|
if (!oneLine(command)) errors.push({ path: at, message: 'the key must be the exact lane command: one line under 512 characters' });
|
||||||
|
if (!m || typeof m !== 'object' || Array.isArray(m)) { errors.push({ path: at, message: 'must be { profiles, inventory }' }); continue; }
|
||||||
|
const o = m as Record<string, unknown>;
|
||||||
|
for (const k of Object.keys(o)) if (k !== 'profiles' && k !== 'inventory') errors.push({ path: `${at}.${k}`, message: 'unknown key; valid keys: profiles, inventory' });
|
||||||
|
if (!Array.isArray(o.profiles) || !o.profiles.length || o.profiles.some(p => typeof p !== 'string')) errors.push({ path: `${at}.profiles`, message: 'must be a non-empty array of run_profiles names' });
|
||||||
|
else for (const p of o.profiles as string[]) if (!profileNames.includes(p)) errors.push({ path: `${at}.profiles`, message: `${JSON.stringify(p)} is not a run_profiles name` });
|
||||||
|
if (!oneLine(o.inventory)) errors.push({ path: `${at}.inventory`, message: 'must be one command line under 512 characters that prints the lane\'s test files, one per line' });
|
||||||
|
}
|
||||||
|
return errors;
|
||||||
|
}
|
||||||
@@ -97,6 +97,8 @@ export interface PregateFile {
|
|||||||
export interface ShipReceipt {
|
export interface ShipReceipt {
|
||||||
schema_version: 1; head: string; base: string; pr?: number; version?: string; gated_tree?: string;
|
schema_version: 1; head: string; base: string; pr?: number; version?: string; gated_tree?: string;
|
||||||
gate?: Record<string, unknown>; ci?: Record<string, unknown>; spend_usd?: number | string; predecessor?: string;
|
gate?: Record<string, unknown>; ci?: Record<string, unknown>; spend_usd?: number | string; predecessor?: string;
|
||||||
|
/** GSTA-258: the lanes CI carried at this head's tree, one entry per run (gstack-evidence carried). */
|
||||||
|
ci_carried?: Array<{ lane: string; run: number; attempt: number; tree: string }>;
|
||||||
session_kind?: string; artifacts_consumed?: string; run?: string;
|
session_kind?: string; artifacts_consumed?: string; run?: string;
|
||||||
// C-stamp (C3/C5/C7): the tree receipt's verdicts and the policy provenance, all optional.
|
// C-stamp (C3/C5/C7): the tree receipt's verdicts and the policy provenance, all optional.
|
||||||
thread?: string; tree?: string; gate_reuse?: string; policy?: string; queue_mode?: string; preregistration?: string; history?: string;
|
thread?: string; tree?: string; gate_reuse?: string; policy?: string; queue_mode?: string; preregistration?: string; history?: string;
|
||||||
@@ -239,6 +241,7 @@ export const ARTIFACT_SCHEMAS = {
|
|||||||
properties: {
|
properties: {
|
||||||
schema_version: { const: 1 }, pr: { type: 'integer' }, head: str, version: str, base: str, gated_tree: str,
|
schema_version: { const: 1 }, pr: { type: 'integer' }, head: str, version: str, base: str, gated_tree: str,
|
||||||
gate: { type: 'object' }, ci: { type: 'object' }, spend_usd: { type: ['number', 'string'] }, predecessor: str,
|
gate: { type: 'object' }, ci: { type: 'object' }, spend_usd: { type: ['number', 'string'] }, predecessor: str,
|
||||||
|
ci_carried: { type: 'array', items: { type: 'object', required: ['lane', 'run', 'attempt', 'tree'], properties: { lane: str, run: { type: 'integer', minimum: 1 }, attempt: { type: 'integer', minimum: 1 }, tree: str } } },
|
||||||
session_kind: str, artifacts_consumed: { ...str, description: '<n>/<m>' }, run: str,
|
session_kind: str, artifacts_consumed: { ...str, description: '<n>/<m>' }, run: str,
|
||||||
thread: str, tree: { enum: ['same-modulo-stamps', 'changed', 'not-compared'] }, gate_reuse: str, policy: str, queue_mode: str, preregistration: str, history: str,
|
thread: str, tree: { enum: ['same-modulo-stamps', 'changed', 'not-compared'] }, gate_reuse: str, policy: str, queue_mode: str, preregistration: str, history: str,
|
||||||
pregate: { ...str, description: 'pass|warn|requires-remote cleared … @<tree> from pregate.json, current for this head' }, requires_remote: strList,
|
pregate: { ...str, description: 'pass|warn|requires-remote cleared … @<tree> from pregate.json, current for this head' }, requires_remote: strList,
|
||||||
|
|||||||
@@ -40,7 +40,7 @@ import { selectLanes, type Selection } from '../lane-selection';
|
|||||||
import { launchRuntime } from '../runtime';
|
import { launchRuntime } from '../runtime';
|
||||||
import { cargoModel, type CargoModel } from '../rust';
|
import { cargoModel, type CargoModel } from '../rust';
|
||||||
import { adapterFor, readRange, STALL_IDLE, supervise, type Counter, type Supervised } from '../supervise';
|
import { adapterFor, readRange, STALL_IDLE, supervise, type Counter, type Supervised } from '../supervise';
|
||||||
import { verifyFromRun } from '../../run-evidence';
|
import { verifyFromRun, type RunProfile } from '../../run-evidence';
|
||||||
import { matchesAny } from '../../ship-policy';
|
import { matchesAny } from '../../ship-policy';
|
||||||
import type { CheckDef, CheckResult, PregateContext, RemoteObligation } from '../types';
|
import type { CheckDef, CheckResult, PregateContext, RemoteObligation } from '../types';
|
||||||
|
|
||||||
@@ -533,7 +533,10 @@ async function runSelected(ctx: PregateContext, sel: Selection, base: CheckResul
|
|||||||
const smallMachine = ctx.cores < SMALL_MACHINE_CORES && sel.localTests.length > 0 && (ctx.config.remote_gate || ctx.testBackend !== 'local');
|
const smallMachine = ctx.cores < SMALL_MACHINE_CORES && sel.localTests.length > 0 && (ctx.config.remote_gate || ctx.testBackend !== 'local');
|
||||||
if (smallMachine) {
|
if (smallMachine) {
|
||||||
const obligation = remoteObligation(ctx, sel);
|
const obligation = remoteObligation(ctx, sel);
|
||||||
return { ...base, status: 'requires-remote', remote: [...sel.remote, obligation], lines: [...lines, `requires-remote ${obligation.lane}: ${ctx.cores} cores (under ${SMALL_MACHINE_CORES}); ${sel.localTests.length} local test file(s) recorded for the remote gate, not run here`, `run: ${obligation.command}`], detail: `${sel.localTests.length} local test file(s) recorded as requires-remote ${obligation.lane} (${ctx.cores} cores, under ${SMALL_MACHINE_CORES}); run it, then ${clearHint(obligation.lane)}, or gstack-pregate --remote --spend-cap <usd> --allow-repo-commands` };
|
const route = obligation.lane === REMOTE_GATE_LANE && ctx.config.remote_gate_runs && ctx.ciCarry !== false
|
||||||
|
? `a verified CI run at this tree clears it: push, then gstack-ci-carry wait --base ${ctx.base} and rerun gstack-pregate --allow-repo-commands (a pull_request run of a branch behind its base tested the merge, another tree: dispatch the workflow on the branch); or opt in to the VM: gstack-pregate --remote --spend-cap <usd> --allow-repo-commands`
|
||||||
|
: `run it, then ${clearHint(obligation.lane)}, or gstack-pregate --remote --spend-cap <usd> --allow-repo-commands`;
|
||||||
|
return { ...base, status: 'requires-remote', remote: [...sel.remote, obligation], lines: [...lines, `requires-remote ${obligation.lane}: ${ctx.cores} cores (under ${SMALL_MACHINE_CORES}); ${sel.localTests.length} local test file(s) recorded for the remote gate, not run here`, `run: ${obligation.command}`], detail: `${sel.localTests.length} local test file(s) recorded as requires-remote ${obligation.lane} (${ctx.cores} cores, under ${SMALL_MACHINE_CORES}); ${route}` };
|
||||||
}
|
}
|
||||||
if (!ctx.allowRepoCommands) return { ...base, status: 'incomplete', code: 'REPO_COMMANDS_NOT_ALLOWED', detail: `${sel.localTests.length} local test file(s) selected but not run (pass --allow-repo-commands)`, fix: 'rerun with --allow-repo-commands (what /ship passes)' };
|
if (!ctx.allowRepoCommands) return { ...base, status: 'incomplete', code: 'REPO_COMMANDS_NOT_ALLOWED', detail: `${sel.localTests.length} local test file(s) selected but not run (pass --allow-repo-commands)`, fix: 'rerun with --allow-repo-commands (what /ship passes)' };
|
||||||
if (sel.localTests.length === 0) return { ...base, status: sel.remote.length ? 'requires-remote' : 'pass', detail: `no local lane; ${sel.remote.length} requires-remote obligation(s)` };
|
if (sel.localTests.length === 0) return { ...base, status: sel.remote.length ? 'requires-remote' : 'pass', detail: `no local lane; ${sel.remote.length} requires-remote obligation(s)` };
|
||||||
@@ -624,6 +627,12 @@ async function run(ctx: PregateContext): Promise<CheckResult> {
|
|||||||
return mode === 'no-policy' ? withStarter(ctx, sel, result) : withNoLaneWarning(ctx, sel, result);
|
return mode === 'no-policy' ? withStarter(ctx, sel, result) : withNoLaneWarning(ctx, sel, result);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** The base profiles a run may verify under here: this machine's OS, and a workflow that holds a lane this machine runs. */
|
||||||
|
export function fromRunProfiles(ctx: PregateContext, profiles: Record<string, RunProfile>): string[] {
|
||||||
|
const os = ({ linux: 'Linux', darwin: 'macOS', win32: 'Windows' } as Record<string, string>)[ctx.platform];
|
||||||
|
return Object.entries(profiles).filter(([, p]) => p.os === os && ctx.lanes.some(l => l.tier !== 'platform' && l.id.startsWith(`${p.workflow}/`))).map(([n]) => n);
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* `--from-run`: verify the run against the base branch's profiles for this machine's OS whose workflow holds a lane this
|
* `--from-run`: verify the run against the base branch's profiles for this machine's OS whose workflow holds a lane this
|
||||||
* machine runs (not a platform lane), on exactly this tree; the selected local test files its tree.json lists are the
|
* machine runs (not a platform lane), on exactly this tree; the selected local test files its tree.json lists are the
|
||||||
@@ -631,8 +640,7 @@ async function run(ctx: PregateContext): Promise<CheckResult> {
|
|||||||
*/
|
*/
|
||||||
function fromRun(ctx: PregateContext, sel: Selection): { line: string; passed: string[]; receipt?: string } {
|
function fromRun(ctx: PregateContext, sel: Selection): { line: string; passed: string[]; receipt?: string } {
|
||||||
const { ref, check } = ctx.fromRun!;
|
const { ref, check } = ctx.fromRun!;
|
||||||
const os = ({ linux: 'Linux', darwin: 'macOS', win32: 'Windows' } as Record<string, string>)[ctx.platform];
|
const names = 'error' in check ? [] : fromRunProfiles(ctx, check.profiles);
|
||||||
const names = 'error' in check ? [] : Object.entries(check.profiles).filter(([, p]) => p.os === os && ctx.lanes.some(l => l.tier !== 'platform' && l.id.startsWith(`${p.workflow}/`))).map(([n]) => n);
|
|
||||||
const v = 'error' in check ? { ok: false as const, reason: 'profiles_unavailable', detail: check.error } : verifyFromRun(check, ref, names, ctx.wtree, ctx.head);
|
const v = 'error' in check ? { ok: false as const, reason: 'profiles_unavailable', detail: check.error } : verifyFromRun(check, ref, names, ctx.wtree, ctx.head);
|
||||||
if (!v.ok) return { line: `REUSE: refused reason=${v.reason} — ${v.detail}; every selected test runs here`, passed: [] };
|
if (!v.ok) return { line: `REUSE: refused reason=${v.reason} — ${v.detail}; every selected test runs here`, passed: [] };
|
||||||
const ran = new Set(v.evidence.files);
|
const ran = new Set(v.evidence.files);
|
||||||
|
|||||||
+12
-4
@@ -46,6 +46,9 @@
|
|||||||
* tree, the command listing the gate's test files they must
|
* tree, the command listing the gate's test files they must
|
||||||
* cover, and the command naming the selected files the gate
|
* cover, and the command naming the selected files the gate
|
||||||
* itself never runs (gstack: paid files)
|
* itself never runs (gstack: paid files)
|
||||||
|
* ship_lanes_from exact /ship lane command → `{ profiles, inventory }`: the
|
||||||
|
* run profiles whose verified runs at HEAD's tree carry that
|
||||||
|
* lane, and the command listing its test files (lib/ci-carry.ts)
|
||||||
* allowlist `{ finding, paths?, pattern?, reason }` entries the `secrets`
|
* allowlist `{ finding, paths?, pattern?, reason }` entries the `secrets`
|
||||||
* check accepts as benign: a gstack-redact finding id or a
|
* check accepts as benign: a gstack-redact finding id or a
|
||||||
* non-secret category, scoped by path globs and/or a regex
|
* non-secret category, scoped by path globs and/or a regex
|
||||||
@@ -67,6 +70,7 @@ import * as fs from 'node:fs';
|
|||||||
import * as path from 'node:path';
|
import * as path from 'node:path';
|
||||||
import { spawnSync } from 'node:child_process';
|
import { spawnSync } from 'node:child_process';
|
||||||
import { createHash } from 'node:crypto';
|
import { createHash } from 'node:crypto';
|
||||||
|
import { shipLanesErrors, type ShipLaneMapping } from '../ci-carry';
|
||||||
import type { ResultCodeName } from '../result-codes';
|
import type { ResultCodeName } from '../result-codes';
|
||||||
import { runProfileErrors, type RemoteGateRuns, type RunProfile } from '../run-evidence';
|
import { runProfileErrors, type RemoteGateRuns, type RunProfile } from '../run-evidence';
|
||||||
import { sealedErrors } from '../sealed-guard';
|
import { sealedErrors } from '../sealed-guard';
|
||||||
@@ -102,15 +106,16 @@ export interface PregateConfig {
|
|||||||
allowlist: AllowlistEntry[];
|
allowlist: AllowlistEntry[];
|
||||||
run_profiles: Record<string, RunProfile>;
|
run_profiles: Record<string, RunProfile>;
|
||||||
remote_gate_runs: RemoteGateRuns | null;
|
remote_gate_runs: RemoteGateRuns | null;
|
||||||
|
ship_lanes_from: Record<string, ShipLaneMapping>;
|
||||||
}
|
}
|
||||||
export interface Guard { id: string; when: string[]; command: string; fix?: string; timeout_s?: number }
|
export interface Guard { id: string; when: string[]; command: string; fix?: string; timeout_s?: number }
|
||||||
export interface AllowlistEntry { finding: string; paths?: string[]; pattern?: string; reason: string }
|
export interface AllowlistEntry { finding: string; paths?: string[]; pattern?: string; reason: string }
|
||||||
export const DEFAULT_CONFIG: PregateConfig = { runner: null, dependencies: {}, empty_lanes: [], selection: {}, warn: [], strays: { allow: [] }, local: [], guards: [], serial_patterns: DEFAULT_SERIAL_PATTERNS, price_file: null, stall_timeout_s: null, timeout_s: null, remote_gate: null, allowlist: [], run_profiles: {}, remote_gate_runs: null };
|
export const DEFAULT_CONFIG: PregateConfig = { runner: null, dependencies: {}, empty_lanes: [], selection: {}, warn: [], strays: { allow: [] }, local: [], guards: [], serial_patterns: DEFAULT_SERIAL_PATTERNS, price_file: null, stall_timeout_s: null, timeout_s: null, remote_gate: null, allowlist: [], run_profiles: {}, remote_gate_runs: null, ship_lanes_from: {} };
|
||||||
export interface ConfigError { code: ResultCodeName; path: string; message: string }
|
export interface ConfigError { code: ResultCodeName; path: string; message: string }
|
||||||
|
|
||||||
export const MAX_LIST = 256;
|
export const MAX_LIST = 256;
|
||||||
const MAX_COMMAND = 512;
|
const MAX_COMMAND = 512;
|
||||||
const KEYS = ['runner', 'dependencies', 'empty_lanes', 'selection', 'warn', 'strays', 'local', 'guards', 'serial_patterns', 'price_file', 'sealed', 'stall_timeout_s', 'timeout_s', 'remote_gate', 'allowlist', 'run_profiles', 'remote_gate_runs'];
|
const KEYS = ['runner', 'dependencies', 'empty_lanes', 'selection', 'warn', 'strays', 'local', 'guards', 'serial_patterns', 'price_file', 'sealed', 'stall_timeout_s', 'timeout_s', 'remote_gate', 'allowlist', 'run_profiles', 'remote_gate_runs', 'ship_lanes_from'];
|
||||||
const LANE_ID = /^[A-Za-z0-9_.-]+\.ya?ml\/[A-Za-z0-9_-]+$/;
|
const LANE_ID = /^[A-Za-z0-9_.-]+\.ya?ml\/[A-Za-z0-9_-]+$/;
|
||||||
const MAX_GUARDS = 32;
|
const MAX_GUARDS = 32;
|
||||||
const GUARD_KEYS = ['id', 'when', 'command', 'fix', 'timeout_s'];
|
const GUARD_KEYS = ['id', 'when', 'command', 'fix', 'timeout_s'];
|
||||||
@@ -202,6 +207,8 @@ export function validateConfig(raw: unknown, knownChecks: readonly string[] = []
|
|||||||
if (why) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'remote_gate', message: why });
|
if (why) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'remote_gate', message: why });
|
||||||
}
|
}
|
||||||
for (const e of runProfileErrors(obj.run_profiles, obj.remote_gate_runs)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: e.path, message: e.message });
|
for (const e of runProfileErrors(obj.run_profiles, obj.remote_gate_runs)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: e.path, message: e.message });
|
||||||
|
const profileNames = obj.run_profiles && typeof obj.run_profiles === 'object' ? Object.keys(obj.run_profiles) : [];
|
||||||
|
for (const e of shipLanesErrors(obj.ship_lanes_from, profileNames)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: e.path, message: e.message });
|
||||||
if ('guards' in obj) guardErrors(obj.guards, errors);
|
if ('guards' in obj) guardErrors(obj.guards, errors);
|
||||||
if ('allowlist' in obj) allowlistErrors(obj.allowlist, errors);
|
if ('allowlist' in obj) allowlistErrors(obj.allowlist, errors);
|
||||||
if ('sealed' in obj) for (const e of sealedErrors(obj.sealed)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: e.path, message: e.message });
|
if ('sealed' in obj) for (const e of sealedErrors(obj.sealed)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: e.path, message: e.message });
|
||||||
@@ -273,7 +280,7 @@ export function parseConfig(text: string, knownChecks: readonly string[] = []):
|
|||||||
const errors = validateConfig(raw, knownChecks);
|
const errors = validateConfig(raw, knownChecks);
|
||||||
if (errors.length) return { config: DEFAULT_CONFIG, errors, declaresLanes: false };
|
if (errors.length) return { config: DEFAULT_CONFIG, errors, declaresLanes: false };
|
||||||
const o = raw as Partial<PregateConfig>;
|
const o = raw as Partial<PregateConfig>;
|
||||||
return { declaresLanes: LANE_KEYS.some(k => k in o), config: { $schema: o.$schema, runner: o.runner ?? null, dependencies: o.dependencies ?? {}, empty_lanes: o.empty_lanes ?? [], selection: o.selection ?? {}, warn: o.warn ?? [], strays: { allow: o.strays?.allow ?? [] }, local: o.local ?? [], guards: o.guards ?? [], serial_patterns: o.serial_patterns ?? DEFAULT_SERIAL_PATTERNS, price_file: o.price_file ?? null, stall_timeout_s: o.stall_timeout_s ?? null, timeout_s: o.timeout_s ?? null, remote_gate: o.remote_gate ?? null, allowlist: o.allowlist ?? [], run_profiles: o.run_profiles ?? {}, remote_gate_runs: o.remote_gate_runs ?? null }, errors: [] };
|
return { declaresLanes: LANE_KEYS.some(k => k in o), config: { $schema: o.$schema, runner: o.runner ?? null, dependencies: o.dependencies ?? {}, empty_lanes: o.empty_lanes ?? [], selection: o.selection ?? {}, warn: o.warn ?? [], strays: { allow: o.strays?.allow ?? [] }, local: o.local ?? [], guards: o.guards ?? [], serial_patterns: o.serial_patterns ?? DEFAULT_SERIAL_PATTERNS, price_file: o.price_file ?? null, stall_timeout_s: o.stall_timeout_s ?? null, timeout_s: o.timeout_s ?? null, remote_gate: o.remote_gate ?? null, allowlist: o.allowlist ?? [], run_profiles: o.run_profiles ?? {}, remote_gate_runs: o.remote_gate_runs ?? null, ship_lanes_from: o.ship_lanes_from ?? {} }, errors: [] };
|
||||||
}
|
}
|
||||||
|
|
||||||
export type ConfigLoad = { config: PregateConfig; errors: ConfigError[]; present: boolean; label: string; kind: 'base' | 'head' | 'user' | 'none'; /** sha256 of the user file's bytes (kind 'user' only). */ digest?: string; /** The policy has a LANE_KEYS key; absent under kind 'none'. */ declaresLanes?: boolean } | { unavailable: true; base: string; detail: string };
|
export type ConfigLoad = { config: PregateConfig; errors: ConfigError[]; present: boolean; label: string; kind: 'base' | 'head' | 'user' | 'none'; /** sha256 of the user file's bytes (kind 'user' only). */ digest?: string; /** The policy has a LANE_KEYS key; absent under kind 'none'. */ declaresLanes?: boolean } | { unavailable: true; base: string; detail: string };
|
||||||
@@ -329,7 +336,7 @@ export function loadConfigFromUser(file: string, knownChecks: readonly string[]
|
|||||||
// `sealed` (the pre-push guard), `price_file` (gstack-eval-plan) and the run profiles (`clear --run`) read only the git copies, so a non-null value in the user file would look active and do nothing.
|
// `sealed` (the pre-push guard), `price_file` (gstack-eval-plan) and the run profiles (`clear --run`) read only the git copies, so a non-null value in the user file would look active and do nothing.
|
||||||
let raw: unknown = null;
|
let raw: unknown = null;
|
||||||
try { raw = JSON.parse(text); } catch { /* parseConfig reported it */ }
|
try { raw = JSON.parse(text); } catch { /* parseConfig reported it */ }
|
||||||
const ignored = raw && typeof raw === 'object' && !Array.isArray(raw) ? ['sealed', 'price_file', 'run_profiles', 'remote_gate_runs'].filter(k => { const v = (raw as Record<string, unknown>)[k]; return v != null && !(k === 'run_profiles' && typeof v === 'object' && Object.keys(v).length === 0); }) : [];
|
const ignored = raw && typeof raw === 'object' && !Array.isArray(raw) ? ['sealed', 'price_file', 'run_profiles', 'remote_gate_runs', 'ship_lanes_from'].filter(k => { const v = (raw as Record<string, unknown>)[k]; return v != null && !(k === 'run_profiles' && typeof v === 'object' && Object.keys(v).length === 0); }) : [];
|
||||||
// parseConfig names the repo copy for a JSON error; this file is not it.
|
// parseConfig names the repo copy for a JSON error; this file is not it.
|
||||||
const errors = [...load.errors.map(e => e.path === PREGATE_CONFIG_PATH ? { ...e, path: file } : e), ...ignored.map(k => ({ code: 'PREGATE_CONFIG_INVALID' as const, path: k, message: `the user-owned policy covers gstack-pregate run only; ${k} is read from ${PREGATE_CONFIG_PATH} on the base branch` }))];
|
const errors = [...load.errors.map(e => e.path === PREGATE_CONFIG_PATH ? { ...e, path: file } : e), ...ignored.map(k => ({ code: 'PREGATE_CONFIG_INVALID' as const, path: k, message: `the user-owned policy covers gstack-pregate run only; ${k} is read from ${PREGATE_CONFIG_PATH} on the base branch` }))];
|
||||||
return { ...load, errors, present: true, label: `user ${file}`, kind: 'user', digest: createHash('sha256').update(text).digest('hex') };
|
return { ...load, errors, present: true, label: `user ${file}`, kind: 'user', digest: createHash('sha256').update(text).digest('hex') };
|
||||||
@@ -358,6 +365,7 @@ export function explainConfig(): string {
|
|||||||
'remote_gate: the repo\'s remote test gate command; under 8 cores the test tier becomes a requires-remote obligation naming it, and gstack-pregate --remote runs it (it must print PREGATE_REMOTE: tree=<tree> tests=<n> failed=<n>) (docs/pregate.md#remote-gate)',
|
'remote_gate: the repo\'s remote test gate command; under 8 cores the test tier becomes a requires-remote obligation naming it, and gstack-pregate --remote runs it (it must print PREGATE_REMOTE: tree=<tree> tests=<n> failed=<n>) (docs/pregate.md#remote-gate)',
|
||||||
'run_profiles: name → { workflow, events, jobs, artifact, max_age_hours?, os? }; the GitHub Actions runs gstack-pregate clear --run verifies as evidence on the identical tree (docs/pregate.md#run-evidence)',
|
'run_profiles: name → { workflow, events, jobs, artifact, max_age_hours?, os? }; the GitHub Actions runs gstack-pregate clear --run verifies as evidence on the identical tree (docs/pregate.md#run-evidence)',
|
||||||
'remote_gate_runs: { profiles, inventory, skipped? }; the run profiles that together clear the remote gate obligation, the command listing the gate\'s test files they must cover, and an optional command printing which selected files (PREGATE_FILES_FILE) the gate itself does not run, which then need no run (docs/pregate.md#run-evidence)',
|
'remote_gate_runs: { profiles, inventory, skipped? }; the run profiles that together clear the remote gate obligation, the command listing the gate\'s test files they must cover, and an optional command printing which selected files (PREGATE_FILES_FILE) the gate itself does not run, which then need no run (docs/pregate.md#run-evidence)',
|
||||||
|
'ship_lanes_from: exact /ship lane command → { profiles, inventory }; a verified run of each profile at HEAD\'s clean tree whose tree.json covers the inventory carries the lane instead of running it again (docs/pregate.md#ci-carry)',
|
||||||
`allowlist: [{ finding, paths?, pattern?, reason }]: benign findings the secrets check accepts and names as allowlisted. finding is a gstack-redact id (env.kv, google.api_key, pii.phone.e164, …) or a category (${ALLOWLISTABLE_CATEGORIES.join(', ')}); paths are globs; pattern is a regex matched at the finding's position in the added line; one of the two is required, reason always. Never ${NON_ALLOWLISTABLE.join(' or ')} or engine.* (docs/pregate.md#allowlist)`,
|
`allowlist: [{ finding, paths?, pattern?, reason }]: benign findings the secrets check accepts and names as allowlisted. finding is a gstack-redact id (env.kv, google.api_key, pii.phone.e164, …) or a category (${ALLOWLISTABLE_CATEGORIES.join(', ')}); paths are globs; pattern is a regex matched at the finding's position in the added line; one of the two is required, reason always. Never ${NON_ALLOWLISTABLE.join(' or ')} or engine.* (docs/pregate.md#allowlist)`,
|
||||||
'no .gstack/pregate.json on the base: <state root>/projects/<slug>/pregate.json is read instead (state root: GSTACK_STATE_ROOT, else GSTACK_HOME, else GSTACK_STATE_DIR, else a gstack plugin data dir, else ~/.gstack, per docs/state-root.md; run --explain prints the resolved path; outside the repo, so a PR cannot reach it; a path inside the checkout is ignored)',
|
'no .gstack/pregate.json on the base: <state root>/projects/<slug>/pregate.json is read instead (state root: GSTACK_STATE_ROOT, else GSTACK_HOME, else GSTACK_STATE_DIR, else a gstack plugin data dir, else ~/.gstack, per docs/state-root.md; run --explain prints the resolved path; outside the repo, so a PR cannot reach it; a path inside the checkout is ignored)',
|
||||||
].join('\n') + '\n';
|
].join('\n') + '\n';
|
||||||
|
|||||||
+57
-1
@@ -10,6 +10,7 @@
|
|||||||
* land: `guards` (field wave CD1); ratchets, lint, links, hermetic, patches
|
* land: `guards` (field wave CD1); ratchets, lint, links, hermetic, patches
|
||||||
* and integrator later.
|
* and integrator later.
|
||||||
*/
|
*/
|
||||||
|
import { spawnSync } from 'node:child_process';
|
||||||
import { createHash } from 'node:crypto';
|
import { createHash } from 'node:crypto';
|
||||||
import * as fs from 'node:fs';
|
import * as fs from 'node:fs';
|
||||||
import * as path from 'node:path';
|
import * as path from 'node:path';
|
||||||
@@ -17,7 +18,8 @@ import { ARTIFACT_SCHEMAS, checkSchema, type PregateFile, type PregateReceiptSub
|
|||||||
import { wtreeFingerprint, type WtreeResult } from '../script-invocation';
|
import { wtreeFingerprint, type WtreeResult } from '../script-invocation';
|
||||||
import { resolveStateRoot } from '../state-root';
|
import { resolveStateRoot } from '../state-root';
|
||||||
import { guardsCheck } from './checks/guards';
|
import { guardsCheck } from './checks/guards';
|
||||||
import { clearHint, lanesCheck, parseRemoteReport, REMOTE_GATE_LANE, remoteReportProblem, treeMismatch } from './checks/lanes';
|
import { findCarry } from '../ci-carry';
|
||||||
|
import { clearHint, fromRunProfiles, lanesCheck, parseRemoteReport, REMOTE_GATE_LANE, remoteReportProblem, treeMismatch } from './checks/lanes';
|
||||||
import { literalsCheck } from './checks/literals';
|
import { literalsCheck } from './checks/literals';
|
||||||
import { regenCheck } from './checks/regen';
|
import { regenCheck } from './checks/regen';
|
||||||
import { secretsCheck } from './checks/secrets';
|
import { secretsCheck } from './checks/secrets';
|
||||||
@@ -26,6 +28,7 @@ import { timingCheck } from './checks/timing';
|
|||||||
import { NON_DOWNGRADABLE, PREGATE_CONFIG_PATH, userPolicyAbsent } from './config';
|
import { NON_DOWNGRADABLE, PREGATE_CONFIG_PATH, userPolicyAbsent } from './config';
|
||||||
import { sha256, withFingerprint } from './context';
|
import { sha256, withFingerprint } from './context';
|
||||||
import { baseBranchOf, baseMismatchFix, describeSubstitution, substituteStampOnlyTree } from './remote-receipt';
|
import { baseBranchOf, baseMismatchFix, describeSubstitution, substituteStampOnlyTree } from './remote-receipt';
|
||||||
|
import { buildRunCheck } from './run-check';
|
||||||
import { resolveBaseBranch } from '../ship-policy';
|
import { resolveBaseBranch } from '../ship-policy';
|
||||||
import { parseRunReceipt, parseRunRef, runChanged, runReceiptFields, uncoveredFiles, verifyRun, type ApiJob, type ApiRun, type RemoteGateRuns, type RunDeps, type RunEvidence, type RunProfile } from '../run-evidence';
|
import { parseRunReceipt, parseRunRef, runChanged, runReceiptFields, uncoveredFiles, verifyRun, type ApiJob, type ApiRun, type RemoteGateRuns, type RunDeps, type RunEvidence, type RunProfile } from '../run-evidence';
|
||||||
import type { CheckDef, CheckResult, PregateContext, Stage } from './types';
|
import type { CheckDef, CheckResult, PregateContext, Stage } from './types';
|
||||||
@@ -47,8 +50,40 @@ export function applyDowngrades(ctx: PregateContext, result: CheckResult, def: C
|
|||||||
return { ...result, status: 'warn', detail: `${result.detail} (downgraded to warn by ${ctx.configLabel})` };
|
return { ...result, status: 'warn', detail: `${result.detail} (downgraded to warn by ${ctx.configLabel})` };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The tests stage's automatic `--from-run` (GSTA-258): when the base branch declares run profiles and no run was named,
|
||||||
|
* the newest verified CI run at HEAD's clean tree under a profile for this OS becomes `ctx.fromRun`, so its tree.json
|
||||||
|
* files count as passed and only the rest run (on a small machine, an all-covered selection records no remote gate
|
||||||
|
* obligation). Returns the CI_CARRY line, or null when the base declares no profiles. Security scans, guards and the
|
||||||
|
* preflight stage are never carried.
|
||||||
|
*/
|
||||||
|
export function autoCarry(ctx: PregateContext): string | null {
|
||||||
|
if (ctx.fromRun || ctx.ciCarry === false || !Object.keys(ctx.config.run_profiles).length) return null;
|
||||||
|
const none = (why: string) => `CI_CARRY: none (${why.length > 300 ? `${why.slice(0, 300)}…` : why})`;
|
||||||
|
const tree = git(ctx.repoRoot, ['rev-parse', 'HEAD^{tree}']);
|
||||||
|
if (!tree || ctx.wtree !== tree) return none(`dirty_tree: the working tree ${ctx.wtree.slice(0, 12)} is not HEAD's committed tree; only a clean, committed tree carries`);
|
||||||
|
const check = buildRunCheck(ctx.repoRoot, null, ctx.allowRepoCommands, ctx.base);
|
||||||
|
if ('error' in check) return none(`profiles_unavailable: ${check.error}`);
|
||||||
|
const names = fromRunProfiles(ctx, check.profiles);
|
||||||
|
if (!names.length) return none('no_profile: no base run profile for this OS covers a lane this machine runs');
|
||||||
|
const wtree = () => { const fp = wtreeFingerprint({ cwd: ctx.repoRoot }); return 'tree' in fp ? fp.tree : null; };
|
||||||
|
const refusals: string[] = [];
|
||||||
|
for (const name of names) {
|
||||||
|
const v = findCarry(check, { profiles: [name], head: ctx.head, tree, wtree });
|
||||||
|
if (v.ok) { ctx.fromRun = { ref: String(v.runs[0]!.run_id), check }; return `CI_CARRY: accepted ${v.receipt}`; }
|
||||||
|
refusals.push(`${v.reason}: ${v.detail}`);
|
||||||
|
}
|
||||||
|
return none(refusals.join(' | '));
|
||||||
|
}
|
||||||
|
|
||||||
|
const git = (cwd: string, args: string[]): string | null => {
|
||||||
|
const r = spawnSync('git', args, { cwd, encoding: 'utf8', timeout: 30_000 });
|
||||||
|
return r.status === 0 ? r.stdout.trim() || null : null;
|
||||||
|
};
|
||||||
|
|
||||||
export async function runPregate(ctx: PregateContext, o: RunOptions): Promise<PregateFile> {
|
export async function runPregate(ctx: PregateContext, o: RunOptions): Promise<PregateFile> {
|
||||||
const selected = CHECKS.filter(c => (o.stage === 'all' || c.stage === o.stage) && (!o.only || o.only.includes(c.id)));
|
const selected = CHECKS.filter(c => (o.stage === 'all' || c.stage === o.stage) && (!o.only || o.only.includes(c.id)));
|
||||||
|
if (selected.some(c => c.id === 'lanes')) { const line = autoCarry(ctx); if (line) ctx.progress(line); }
|
||||||
const checks: CheckResult[] = [];
|
const checks: CheckResult[] = [];
|
||||||
const clocks: Record<Stage, number> = { preflight: 0, tests: 0 };
|
const clocks: Record<Stage, number> = { preflight: 0, tests: 0 };
|
||||||
for (const def of selected) {
|
for (const def of selected) {
|
||||||
@@ -68,9 +103,30 @@ export async function runPregate(ctx: PregateContext, o: RunOptions): Promise<Pr
|
|||||||
requires_remote: remote,
|
requires_remote: remote,
|
||||||
};
|
};
|
||||||
file.verdict = verdictOf(file);
|
file.verdict = verdictOf(file);
|
||||||
|
const gateLine = carryGate(ctx, file);
|
||||||
|
if (gateLine) ctx.progress(gateLine);
|
||||||
return file;
|
return file;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* After a verified run carried part of the tests stage, that run clears the remote gate obligation under `clear --run`'s
|
||||||
|
* own checks (remote_gate_runs profiles, inventory coverage, skipped files). The obligation is checked as the carried
|
||||||
|
* files plus the rest: without that, a remainder of only gate-skipped files (gstack's paid evals) is refused paid_only,
|
||||||
|
* and the gate itself reports tests=0 for it, so nothing could ever clear it.
|
||||||
|
*/
|
||||||
|
function carryGate(ctx: PregateContext, file: PregateFile): string | null {
|
||||||
|
const gate = file.requires_remote?.find(r => r.lane === REMOTE_GATE_LANE && !r.receipt);
|
||||||
|
const carried = file.checks.find(c => c.id === 'lanes')?.from_run?.files;
|
||||||
|
if (!gate || !carried?.length || !ctx.fromRun || !('inventory' in ctx.fromRun.check)) return null;
|
||||||
|
const check = ctx.fromRun.check as RunCheck;
|
||||||
|
if (!check.gateRuns) return null;
|
||||||
|
const view: PregateFile = { ...file, requires_remote: file.requires_remote!.map(r => (r === gate ? { ...r, tests: [...new Set([...carried, ...r.tests])] } : r)) };
|
||||||
|
const d = clearByRuns(view, REMOTE_GATE_LANE, [ctx.fromRun.ref], check, ctx.repoRoot);
|
||||||
|
if (!('receipt' in d)) return `CI_CARRY: ${REMOTE_GATE_LANE} stays uncleared (${d.message})`;
|
||||||
|
clearObligation(file, REMOTE_GATE_LANE, d.receipt, 'ci-carry', d.substitution);
|
||||||
|
return `CI_CARRY: cleared ${REMOTE_GATE_LANE} ${d.receipt}`;
|
||||||
|
}
|
||||||
|
|
||||||
/** `pass`, `warn`, `requires-remote`, `incomplete` or `fail`: the worst status across checks, with uncleared obligations counted. */
|
/** `pass`, `warn`, `requires-remote`, `incomplete` or `fail`: the worst status across checks, with uncleared obligations counted. */
|
||||||
export function verdictOf(file: PregateFile): string {
|
export function verdictOf(file: PregateFile): string {
|
||||||
const statuses = new Set(file.checks.map(c => c.status));
|
const statuses = new Set(file.checks.map(c => c.status));
|
||||||
|
|||||||
@@ -64,6 +64,8 @@ export interface PregateContext {
|
|||||||
policyFrom: 'base' | 'head';
|
policyFrom: 'base' | 'head';
|
||||||
/** `--from-run`: a Free Tests run whose tree.json files count as passed for this tree (lanes, lib/run-evidence.ts verifyFromRun), with the base branch's run check or why it could not be built. */
|
/** `--from-run`: a Free Tests run whose tree.json files count as passed for this tree (lanes, lib/run-evidence.ts verifyFromRun), with the base branch's run check or why it could not be built. */
|
||||||
fromRun?: { ref: string; check: RunSource | { error: string } };
|
fromRun?: { ref: string; check: RunSource | { error: string } };
|
||||||
|
/** `--no-ci-carry`: false skips the tests stage's automatic --from-run (lib/ci-carry.ts findCarry at HEAD's clean tree). */
|
||||||
|
ciCarry?: boolean;
|
||||||
/** `--remote`: run the declared remote gate now, under this spend cap. */
|
/** `--remote`: run the declared remote gate now, under this spend cap. */
|
||||||
remote: { spendCapUsd: number } | null;
|
remote: { spendCapUsd: number } | null;
|
||||||
env: NodeJS.ProcessEnv;
|
env: NodeJS.ProcessEnv;
|
||||||
|
|||||||
+46
-9
@@ -10,6 +10,7 @@ import * as fs from 'node:fs';
|
|||||||
import * as os from 'node:os';
|
import * as os from 'node:os';
|
||||||
import * as path from 'node:path';
|
import * as path from 'node:path';
|
||||||
import { fragmentSlug } from './changelog-fragments';
|
import { fragmentSlug } from './changelog-fragments';
|
||||||
|
import { isConfigUnavailable, loadConfigFromBase } from './pregate/config';
|
||||||
import { parseUnifiedDiff, versionAtRef } from './restamp';
|
import { parseUnifiedDiff, versionAtRef } from './restamp';
|
||||||
import { branchSides, stampOnlyLine } from './stamp-only';
|
import { branchSides, stampOnlyLine } from './stamp-only';
|
||||||
import { matchesAny } from './ship-policy';
|
import { matchesAny } from './ship-policy';
|
||||||
@@ -170,26 +171,62 @@ export function laneConcurrency(mode: string | undefined, cpus = os.availablePar
|
|||||||
return cpus <= 4 ? 1 : Infinity;
|
return cpus <= 4 ? 1 : Infinity;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Run each lane through gstack-evidence run, at most `concurrency` at a time, and read its recorded line; results keep the lanes' order. */
|
export interface LaneResult { label: string; exit: number | null; testsRan: string; log: string; carried?: string; reused?: string; carryNone?: string }
|
||||||
export async function runLanes(cwd: string, list: { label: string; command: string }[], concurrency = Infinity): Promise<{ label: string; exit: number | null; testsRan: string; log: string }[]> {
|
|
||||||
const out: { label: string; exit: number | null; testsRan: string; log: string }[] = [];
|
/** The base branch's `ship_lanes_from` entry for each lane whose command is a key, byte for byte (the ledger binds its sha256). */
|
||||||
|
export function mappedLanes(cwd: string, base: string, list: { label: string; command: string }[]): Map<string, { profiles: string[]; inventory: string }> {
|
||||||
|
const loaded = loadConfigFromBase(cwd, base);
|
||||||
|
const map = new Map<string, { profiles: string[]; inventory: string }>();
|
||||||
|
if (isConfigUnavailable(loaded) || loaded.errors.length) return map;
|
||||||
|
const from = loaded.config.ship_lanes_from;
|
||||||
|
for (const l of list) if (Object.prototype.hasOwnProperty.call(from, l.command)) map.set(l.label, from[l.command]!);
|
||||||
|
return map;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A mapped lane carried from CI at HEAD's clean tree: gstack-evidence record-ci finds and verifies the run (lib/ci-carry.ts)
|
||||||
|
* and records the CI-sourced ledger row; or why it was not (`carryNone`), so the caller runs it.
|
||||||
|
*/
|
||||||
|
export function carryLane(cwd: string, base: string, l: { label: string; command: string }): LaneResult {
|
||||||
|
const rec = bin(cwd, 'gstack-evidence', ['record-ci', '--label', l.label, '--base', base, '--', l.command], 600_000);
|
||||||
|
const got = / tests_ran=(\d+) log=- source=ci runs=(\S+)/.exec(rec.stderr);
|
||||||
|
if (rec.status === 0 && got) return { label: l.label, exit: 0, testsRan: got[1]!, log: '-', carried: got[2]! };
|
||||||
|
return { label: l.label, exit: null, testsRan: 'unknown', log: '-', carryNone: /^CARRY_NONE: (.+)$/m.exec(rec.stderr)?.[1] ?? `record_failed: ${firstLine(rec.stderr)}` };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Run each lane, at most `concurrency` at a time; results keep the lanes' order. With `base`, a lane the base branch maps in
|
||||||
|
* `ship_lanes_from` is carried from a verified CI run at this exact tree when one exists. Every other lane runs through
|
||||||
|
* gstack-evidence run --reuse (a pass at this tree, command and runtime is reused; `reuse: false` forces a run).
|
||||||
|
*/
|
||||||
|
export async function runLanes(cwd: string, list: { label: string; command: string }[], concurrency = Infinity, opts: { base?: string; reuse?: boolean } = {}): Promise<LaneResult[]> {
|
||||||
|
const out: LaneResult[] = [];
|
||||||
|
const mapped = opts.base ? mappedLanes(cwd, opts.base, list) : new Map<string, { profiles: string[]; inventory: string }>();
|
||||||
let next = 0;
|
let next = 0;
|
||||||
const worker = async () => {
|
const worker = async () => {
|
||||||
while (next < list.length) {
|
while (next < list.length) {
|
||||||
const i = next++;
|
const i = next++;
|
||||||
const l = list[i]!;
|
const l = list[i]!;
|
||||||
const r = await binAsync(cwd, 'gstack-evidence', ['run', '--label', l.label, '--', l.command]);
|
const m = mapped.get(l.label);
|
||||||
const m = /gstack-evidence: recorded label=\S+ exit=(\S+) tests_ran=(\S+) log=(\S+)/.exec(r.stderr);
|
const carry = m ? carryLane(cwd, opts.base!, l) : null;
|
||||||
out[i] = { label: l.label, exit: r.status, testsRan: m?.[2] ?? 'unknown', log: m?.[3] ?? '-' };
|
if (carry?.carried) { out[i] = carry; continue; }
|
||||||
|
const r = await binAsync(cwd, 'gstack-evidence', ['run', ...(opts.reuse === false ? [] : ['--reuse']), '--label', l.label, '--', l.command]);
|
||||||
|
const rec = /gstack-evidence: recorded label=\S+ exit=(\S+) tests_ran=(\S+) log=(\S+)/.exec(r.stderr);
|
||||||
|
const reused = /gstack-evidence: reused label=\S+ tree=\S+ from=(\S+)/.exec(r.stderr)?.[1];
|
||||||
|
out[i] = { label: l.label, exit: r.status, testsRan: rec?.[2] ?? 'unknown', log: rec?.[3] ?? '-', ...(reused ? { reused } : {}), ...(carry?.carryNone ? { carryNone: carry.carryNone } : {}) };
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
await Promise.all(Array.from({ length: Math.max(1, Math.min(concurrency, list.length)) }, worker));
|
await Promise.all(Array.from({ length: Math.max(1, Math.min(concurrency, list.length)) }, worker));
|
||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Step 5's lane verdict: a lane passes on its runner count, never exit 0 alone. */
|
/** Step 5's lane verdict: a lane passes on its runner count, never exit 0 alone; a carried lane names its CI runs. */
|
||||||
export function laneLine(l: { label: string; exit: number | null; testsRan: string; log: string }): { line: string; ok: boolean } {
|
export function laneLine(l: LaneResult): { line: string; ok: boolean } {
|
||||||
|
if (l.carried) return { line: `LANE: ${l.label} carried run=${l.carried} tests_ran=${l.testsRan}`, ok: true };
|
||||||
const zero = l.exit === 0 && l.testsRan === '0';
|
const zero = l.exit === 0 && l.testsRan === '0';
|
||||||
const state = l.exit !== 0 ? 'FAIL' : zero ? 'ZERO-RUN' : l.testsRan === 'unknown' ? 'pass (count unavailable)' : 'pass';
|
const state = l.exit !== 0 ? 'FAIL' : zero ? 'ZERO-RUN' : l.testsRan === 'unknown' ? 'pass (count unavailable)' : 'pass';
|
||||||
return { line: `LANE: ${l.label} ${state} exit=${l.exit} tests_ran=${l.testsRan} log=${l.log}`, ok: l.exit === 0 && !zero };
|
return { line: `LANE: ${l.label} ${state} exit=${l.exit} tests_ran=${l.testsRan} log=${l.log}${l.reused ? ` reused=${l.reused}` : ''}`, ok: l.exit === 0 && !zero };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** The CI_CARRY line for a mapped lane that ran here instead. */
|
||||||
|
export const carryNoneLine = (l: LaneResult) => (l.carryNone ? `CI_CARRY: none label=${l.label} (${l.carryNone.length > 220 ? `${l.carryNone.slice(0, 220)}…` : l.carryNone})` : null);
|
||||||
@@ -10,7 +10,7 @@ import * as os from 'node:os';
|
|||||||
import * as path from 'node:path';
|
import * as path from 'node:path';
|
||||||
import { classifyDiff } from './review-depth';
|
import { classifyDiff } from './review-depth';
|
||||||
import { audit } from './ship-audit';
|
import { audit } from './ship-audit';
|
||||||
import { bin, firstLine, flagReader, gh, git, hostedOrigin, laneConcurrency, laneLine, lanes, Report, runLanes, shipPolicy, short } from './ship-bin-common';
|
import { bin, carryNoneLine, firstLine, flagReader, gh, git, hostedOrigin, laneConcurrency, laneLine, lanes, Report, runLanes, shipPolicy, short } from './ship-bin-common';
|
||||||
|
|
||||||
export const PREFLIGHT_HELP = `gstack-ship-preflight — /ship Steps 1-3 and Step 5's pregate and lanes in one call (ship/SKILL.md)
|
export const PREFLIGHT_HELP = `gstack-ship-preflight — /ship Steps 1-3 and Step 5's pregate and lanes in one call (ship/SKILL.md)
|
||||||
|
|
||||||
@@ -23,8 +23,10 @@ export const PREFLIGHT_HELP = `gstack-ship-preflight — /ship Steps 1-3 and Ste
|
|||||||
(CONTEXT, TODOS, COVERAGE_MAP, Coverage, PLAN_*, READER: coverage=needed|none plan=needed|none)
|
(CONTEXT, TODOS, COVERAGE_MAP, Coverage, PLAN_*, READER: coverage=needed|none plan=needed|none)
|
||||||
|
|
||||||
Prints at most 30 lines: BRANCH, STATUS, DIFF, PR/CARRY, APPLE, SCOPE, MERGE, DISTRIBUTION, POLICY,
|
Prints at most 30 lines: BRANCH, STATUS, DIFF, PR/CARRY, APPLE, SCOPE, MERGE, DISTRIBUTION, POLICY,
|
||||||
RUNTIME_LINES, PREGATE (with up to three PREGATE_DETAIL lines when it does not pass), LANES, one LANE line per lane,
|
RUNTIME_LINES, PREGATE (with up to three PREGATE_DETAIL lines when it does not pass), LANES, one LANE line per lane
|
||||||
then NEXT: lines. STOP: names what the agent must
|
(a lane the base maps in .gstack/pregate.json ship_lanes_from is carried from a verified CI run at this exact clean tree:
|
||||||
|
LANE: <label> carried run=<id>/<attempt> tests_ran=<n>, else CI_CARRY: none (<reason>) and it runs; any other lane that
|
||||||
|
passed at this tree, command and runtime is reused: reused=<original time>), then NEXT: lines. STOP: names what the agent must
|
||||||
decide or repair; nothing after it ran. Exit codes: 0 ok · 2 usage · 3 STOP`;
|
decide or repair; nothing after it ran. Exit codes: 0 ok · 2 usage · 3 STOP`;
|
||||||
|
|
||||||
const DIST = /(^|\/)(cmd\/[^/]+\/main\.go|bin\/[^/]+|Cargo\.toml|setup\.py|package\.json)$/;
|
const DIST = /(^|\/)(cmd\/[^/]+\/main\.go|bin\/[^/]+|Cargo\.toml|setup\.py|package\.json)$/;
|
||||||
@@ -151,8 +153,9 @@ async function tests(cwd: string, base: string, list: { label: string; command:
|
|||||||
}
|
}
|
||||||
if (/uncleared/.test(verdict)) r.next('pregate requires-remote: gstack-pregate clear, or --remote --spend-cap <usd> under a user cap, before Step 18');
|
if (/uncleared/.test(verdict)) r.next('pregate requires-remote: gstack-pregate clear, or --remote --spend-cap <usd> under a user cap, before Step 18');
|
||||||
r.add(run.line);
|
r.add(run.line);
|
||||||
const results = (await runLanes(cwd, list, run.concurrency)).map(laneLine);
|
const ran = await runLanes(cwd, list, run.concurrency, { base });
|
||||||
for (const l of results) r.add(l.line);
|
const results = ran.map(laneLine);
|
||||||
|
for (const [i, l] of results.entries()) { const none = carryNoneLine(ran[i]!); if (none) r.add(none); r.add(l.line); }
|
||||||
const bad = results.filter(l => !l.ok);
|
const bad = results.filter(l => !l.ok);
|
||||||
if (bad.length) { r.stop(`${bad.length} lane(s) failed or ran zero tests: Step 5 triage (read each log; a free-suite shard failure reruns its file list per measure.md); in-branch failures stop the ship`); return; }
|
if (bad.length) { r.stop(`${bad.length} lane(s) failed or ran zero tests: Step 5 triage (read each log; a free-suite shard failure reruns its file list per measure.md); in-branch failures stop the ship`); return; }
|
||||||
r.next('Step 6: select eval suites for prompt-related changes with the project\'s selector, then Step 7');
|
r.next('Step 6: select eval suites for prompt-related changes with the project\'s selector, then Step 7');
|
||||||
|
|||||||
+2
-1
@@ -16,7 +16,7 @@ import { stampOnlyLine, type BranchSide } from './stamp-only';
|
|||||||
|
|
||||||
export const RECEIPT_FENCE = 'gstack-ship-receipt';
|
export const RECEIPT_FENCE = 'gstack-ship-receipt';
|
||||||
const KEY_ORDER: Array<keyof ShipReceipt | 'docs'> = [
|
const KEY_ORDER: Array<keyof ShipReceipt | 'docs'> = [
|
||||||
'schema_version', 'pr', 'head', 'version', 'base', 'gated_tree', 'predecessor', 'tree', 'gate_reuse', 'gate', 'ci', 'spend_usd',
|
'schema_version', 'pr', 'head', 'version', 'base', 'gated_tree', 'predecessor', 'tree', 'gate_reuse', 'gate', 'ci', 'ci_carried', 'spend_usd',
|
||||||
'session_kind', 'artifacts_consumed', 'run', 'thread', 'policy', 'queue_mode', 'preregistration', 'history',
|
'session_kind', 'artifacts_consumed', 'run', 'thread', 'policy', 'queue_mode', 'preregistration', 'history',
|
||||||
'head_tree', 'product_patch_id', 'gstack_version', 'review_depth', 'pregate', 'requires_remote', 'issues', 'approval', 'docs', 'review_loop',
|
'head_tree', 'product_patch_id', 'gstack_version', 'review_depth', 'pregate', 'requires_remote', 'issues', 'approval', 'docs', 'review_loop',
|
||||||
'review_loop_exception', 'review_targeted', 'review_exclude', 'carry', 'base_red',
|
'review_loop_exception', 'review_targeted', 'review_exclude', 'carry', 'base_red',
|
||||||
@@ -131,6 +131,7 @@ export function receiptSummaryLine(r: ShipReceipt): string {
|
|||||||
if (loop) parts.push(`passes=${loop.passes} blocking_found=${loop.blocking_found} deferred=${loop.deferred}`
|
if (loop) parts.push(`passes=${loop.passes} blocking_found=${loop.blocking_found} deferred=${loop.deferred}`
|
||||||
+ (loop.fix_checks ? ` fix_checks=${loop.fix_checks}` : '') + (loop.rounds !== undefined ? ` rounds=${loop.rounds}` : '') + (loop.dispatches ? ` dispatches=${Object.values(loop.dispatches).reduce((a, c) => a + c, 0)}` : ''));
|
+ (loop.fix_checks ? ` fix_checks=${loop.fix_checks}` : '') + (loop.rounds !== undefined ? ` rounds=${loop.rounds}` : '') + (loop.dispatches ? ` dispatches=${Object.values(loop.dispatches).reduce((a, c) => a + c, 0)}` : ''));
|
||||||
if (r.carry && typeof r.carry.from === 'string') parts.push(`carried_from=${r.carry.from.slice(0, 12)}`);
|
if (r.carry && typeof r.carry.from === 'string') parts.push(`carried_from=${r.carry.from.slice(0, 12)}`);
|
||||||
|
if (Array.isArray(r.ci_carried) && r.ci_carried.length) parts.push(`ci_carried=${r.ci_carried.map(c => `${c.lane}:${c.run}/${c.attempt}`).join(',')}`);
|
||||||
if (r.base_red && typeof r.base_red.base === 'string') parts.push(`base_red=${r.base_red.base.slice(0, 12)}`);
|
if (r.base_red && typeof r.base_red.base === 'string') parts.push(`base_red=${r.base_red.base.slice(0, 12)}`);
|
||||||
return `SHIP_RECEIPT: ${oneLine(parts.join(' '))}`;
|
return `SHIP_RECEIPT: ${oneLine(parts.join(' '))}`;
|
||||||
}
|
}
|
||||||
|
|||||||
+91
-5
@@ -19,7 +19,9 @@ import { parseReviewLoopException, type ReviewLoopException } from './ship-recei
|
|||||||
import { parseReceiptDocs, type ReceiptDocs } from './ship-receipt-docs';
|
import { parseReceiptDocs, type ReceiptDocs } from './ship-receipt-docs';
|
||||||
import { resolveStateRoot } from './state-root';
|
import { resolveStateRoot } from './state-root';
|
||||||
import { bumpVersion, cmpVersion, fmtVersion, parseVersion, versionWidth, type Bump } from './version-source';
|
import { bumpVersion, cmpVersion, fmtVersion, parseVersion, versionWidth, type Bump } from './version-source';
|
||||||
import { bin, evidenceExempt, firstLine, flagReader, fragmentPath, gh, git, hostedOrigin, laneLine, lanes, releasePaths, Report, runLanes, sh, shipPolicy, short, stampedFiles } from './ship-bin-common';
|
import { waitForRun } from './ci-carry';
|
||||||
|
import { buildRunCheck } from './pregate/run-check';
|
||||||
|
import { bin, carryLane, carryNoneLine, evidenceExempt, firstLine, flagReader, fragmentPath, gh, git, hostedOrigin, laneLine, lanes, mappedLanes, releasePaths, Report, runLanes, sh, shipPolicy, short, stampedFiles } from './ship-bin-common';
|
||||||
|
|
||||||
export const RELEASE_HELP = `gstack-ship-release — /ship Steps 12 and 15-17 (ship/SKILL.md)
|
export const RELEASE_HELP = `gstack-ship-release — /ship Steps 12 and 15-17 (ship/SKILL.md)
|
||||||
|
|
||||||
@@ -33,8 +35,15 @@ export const RELEASE_HELP = `gstack-ship-release — /ship Steps 12 and 15-17 (s
|
|||||||
each lane's receipt run once if stale) and the pregate on the stamped tree; --reviewed-base (the merge base
|
each lane's receipt run once if stale) and the pregate on the stamped tree; --reviewed-base (the merge base
|
||||||
saved with the reviewed tree) routes paths only the base moved as base=, and an adjacent base edit as context=;
|
saved with the reviewed tree) routes paths only the base moved as base=, and an adjacent base edit as context=;
|
||||||
--review-loop-exception (the record write takes) routes paths the branch touched at fix_check.head (narrowed to fix_check.files when listed) and changed since as accepted=
|
--review-loop-exception (the record write takes) routes paths the branch touched at fix_check.head (narrowed to fix_check.files when listed) and changed since as accepted=
|
||||||
push --branch <branch-name> [--gate-open <n>] [--gate-open-approved]
|
A lane the base branch maps in .gstack/pregate.json ship_lanes_from is carried from a verified CI run at this exact
|
||||||
Step 17 after its guard block: the saved-branch check, the open-gate check, the sealed guard, the idempotency check, git push -u
|
tree, else deferred (EVIDENCE: DEFERRED): the pregate then runs its preflight stage here and its tests stage in accept
|
||||||
|
push --branch <branch-name> [--gate-open <n>] [--gate-open-approved] [--early [--base <base>]]
|
||||||
|
Step 17 after its guard block: the saved-branch check, the open-gate check, the sealed guard, the idempotency check, git push -u;
|
||||||
|
--early (the Phase 2 push before review, so CI runs beside it) also runs pregate's secrets scan on the commits first
|
||||||
|
accept --base <base> --lane <label>=<command>... [--wait <s>]
|
||||||
|
after push: final-tree CI acceptance. Waits up to --wait seconds (default 170) for the CI run of each mapped profile at
|
||||||
|
the pushed HEAD, carries each mapped lane from that run (a clean tree equal to the run's), reuses or runs the rest,
|
||||||
|
checks every lane FRESH and reruns the pregate on the final tree (its tests stage carries the same run)
|
||||||
|
|
||||||
At most 30 lines each; STOP: names what the agent must decide or repair, and nothing after it ran.
|
At most 30 lines each; STOP: names what the agent must decide or repair, and nothing after it ran.
|
||||||
Exit codes: 0 ok · 2 usage · 3 STOP`;
|
Exit codes: 0 ok · 2 usage · 3 STOP`;
|
||||||
@@ -366,10 +375,19 @@ export async function finish(argv: readonly string[], cwd: string): Promise<Repo
|
|||||||
if (!list.length && !has('--no-lanes-approved')) return r.stop('no test lanes: require Step 5\'s explicit untested-scope approval (rerun with --no-lanes-approved) or run Steps 5-15');
|
if (!list.length && !has('--no-lanes-approved')) return r.stop('no test lanes: require Step 5\'s explicit untested-scope approval (rerun with --no-lanes-approved) or run Steps 5-15');
|
||||||
const allow = evidenceExempt(release).join(',');
|
const allow = evidenceExempt(release).join(',');
|
||||||
const check = (l: { label: string; command: string }) => bin(cwd, 'gstack-evidence', ['check', '--label', l.label, '--expect-cmd', l.command, '--max-age', '24', '--allow-paths', allow], 60_000).stdout;
|
const check = (l: { label: string; command: string }) => bin(cwd, 'gstack-evidence', ['check', '--label', l.label, '--expect-cmd', l.command, '--max-age', '24', '--allow-paths', allow], 60_000).stdout;
|
||||||
|
const mapped = mappedLanes(cwd, base, list);
|
||||||
|
const deferred: string[] = [];
|
||||||
for (const l of list) {
|
for (const l of list) {
|
||||||
let e = check(l);
|
let e = check(l);
|
||||||
if (!/EVIDENCE: FRESH/.test(e)) {
|
if (!/EVIDENCE: FRESH/.test(e)) {
|
||||||
const ran = laneLine((await runLanes(cwd, [l]))[0]!);
|
const m = mapped.get(l.label);
|
||||||
|
const res = m ? carryLane(cwd, base, l) : (await runLanes(cwd, [l], Infinity, { base }))[0]!;
|
||||||
|
if (m && !res.carried) {
|
||||||
|
r.add(`EVIDENCE: DEFERRED label=${l.label} (${short([res.carryNone ?? 'not carried'], 1).slice(0, 160)}): CI at the pushed head carries it in accept`);
|
||||||
|
deferred.push(l.label);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const ran = laneLine(res);
|
||||||
r.add(ran.line);
|
r.add(ran.line);
|
||||||
if (!ran.ok) return r.stop(`lane ${l.label} failed or ran zero tests: STOP publication; run Steps 5-15 from Step 5's triage, then Step 16 stage 1`);
|
if (!ran.ok) return r.stop(`lane ${l.label} failed or ran zero tests: STOP publication; run Steps 5-15 from Step 5's triage, then Step 16 stage 1`);
|
||||||
e = check(l);
|
e = check(l);
|
||||||
@@ -378,6 +396,15 @@ export async function finish(argv: readonly string[], cwd: string): Promise<Repo
|
|||||||
r.add(line.replace(/ log=\S+/, ''));
|
r.add(line.replace(/ log=\S+/, ''));
|
||||||
if (!/EVIDENCE: FRESH/.test(line)) return r.stop(`lane ${l.label} is not FRESH after one rerun: Step 16 stage 4 recovery`);
|
if (!/EVIDENCE: FRESH/.test(line)) return r.stop(`lane ${l.label} is not FRESH after one rerun: Step 16 stage 4 recovery`);
|
||||||
}
|
}
|
||||||
|
const laneArgs = list.map(l => `--lane '${l.label}=${l.command.replace(/'/g, `'\\''`)}'`).join(' ');
|
||||||
|
if (deferred.length) {
|
||||||
|
const out = path.resolve(cwd, git(cwd, ['rev-parse', '--git-path', 'gstack/pregate-finish-preflight.json']).stdout.trim() || '.git/gstack/pregate-finish-preflight.json');
|
||||||
|
const pg = bin(cwd, 'gstack-pregate', ['--allow-repo-commands', '--base', base, '--stage', 'preflight', '--out', out], 3_600_000);
|
||||||
|
const verdict = /^PREGATE: (.+?) tree=/m.exec(pg.stdout + pg.stderr)?.[1] ?? `unreadable (exit ${pg.status})`;
|
||||||
|
r.add(`PREGATE: preflight ${verdict} on the stamped tree (the tests stage runs in accept, carried from CI)`);
|
||||||
|
if (!/^(pass|warn)/.test(verdict)) return r.stop(`pregate preflight ${verdict} on the stamped tree: fix it, then rerun finish`);
|
||||||
|
return r.next(`Step 16 stage 5, then Step 17's guard block and gstack-ship-release push; then gstack-ship-release accept --base ${base} ${laneArgs} carries ${deferred.join(', ')} from CI at the pushed head${loop ? '; Step 18 records the same exception with gstack-ship-receipt write --review-loop-exception' : ''}`);
|
||||||
|
}
|
||||||
const v = bin(cwd, 'gstack-pregate', ['verify'], 60_000);
|
const v = bin(cwd, 'gstack-pregate', ['verify'], 60_000);
|
||||||
if (v.status === 0) r.add('PREGATE: verify ok on the stamped tree');
|
if (v.status === 0) r.add('PREGATE: verify ok on the stamped tree');
|
||||||
else {
|
else {
|
||||||
@@ -389,6 +416,57 @@ export async function finish(argv: readonly string[], cwd: string): Promise<Repo
|
|||||||
return r.next(`Step 16 stage 5: paste these results and the two-line honesty inventory, then run Step 17's guard block and gstack-ship-release push${loop ? '; Step 18 records the same exception with gstack-ship-receipt write --review-loop-exception' : ''}`);
|
return r.next(`Step 16 stage 5: paste these results and the two-line honesty inventory, then run Step 17's guard block and gstack-ship-release push${loop ? '; Step 18 records the same exception with gstack-ship-receipt write --review-loop-exception' : ''}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Final-tree CI acceptance (design fix A1): after push, each mapped lane is carried only from a green run at the pushed
|
||||||
|
* HEAD's exact, clean tree; the rest are reused or run; every lane must be FRESH, and the pregate reruns on that tree.
|
||||||
|
*/
|
||||||
|
export async function accept(argv: readonly string[], cwd: string): Promise<Report> {
|
||||||
|
const { flag } = flagReader(argv);
|
||||||
|
const base = flag('--base')!;
|
||||||
|
const list = lanes(argv) as { label: string; command: string }[];
|
||||||
|
const r = new Report();
|
||||||
|
const branch = git(cwd, ['branch', '--show-current']).stdout.trim();
|
||||||
|
if (!branch) return r.stop('detached HEAD: accept needs the feature branch');
|
||||||
|
const head = git(cwd, ['rev-parse', 'HEAD']).stdout.trim();
|
||||||
|
const remote = git(cwd, ['ls-remote', '--heads', 'origin', `refs/heads/${branch}`], 120_000);
|
||||||
|
if (remote.status !== 0) return r.stop(`cannot read origin/${branch} (${firstLine(remote.stderr)}): restore access, then rerun accept`);
|
||||||
|
if (remote.stdout.split(/\s/)[0] !== head) return r.stop(`HEAD ${head.slice(0, 12)} is not origin/${branch}: run gstack-ship-release push first; accept carries only a run at the pushed head`);
|
||||||
|
const mapped = mappedLanes(cwd, base, list);
|
||||||
|
if (!mapped.size) return r.stop(`no --lane command is a key of origin/${base}'s .gstack/pregate.json ship_lanes_from: nothing comes from CI, so finish runs every lane here`);
|
||||||
|
const runCheck = buildRunCheck(cwd, null, true, base);
|
||||||
|
if ('error' in runCheck) return r.stop(`run profiles unavailable (${runCheck.error}): carry needs them; fix it, or run the lanes here with gstack-ship-preflight --tests-only`);
|
||||||
|
const waitS = Number(flag('--wait') ?? '170');
|
||||||
|
if (!Number.isFinite(waitS) || waitS < 0 || waitS > 3600) return r.stop(`--wait ${flag('--wait')} is not 0-3600 seconds`);
|
||||||
|
for (const name of [...new Set([...mapped.values()].flatMap(m => m.profiles))]) {
|
||||||
|
const p = runCheck.profiles[name];
|
||||||
|
if (!p) return r.stop(`ship_lanes_from names ${name}, which is not a run_profiles entry on origin/${base}`);
|
||||||
|
const w = await waitForRun(runCheck.deps, { workflow: p.workflow, events: p.events, head, timeoutMs: waitS * 1000, intervalMs: 15_000, sleep: ms => new Promise(res => setTimeout(res, ms)), clock: Date.now });
|
||||||
|
r.add(`CI_WAIT: ${w.state} ${name} ${'run' in w ? `run=${w.run.id}/${w.run.run_attempt}` : w.reason}`);
|
||||||
|
if (w.state === 'running') return r.stop(`${p.workflow} run ${w.run.id} at ${head.slice(0, 12)} is still running: rerun accept (or gstack-ci-carry wait --base ${base} in the background, then accept)`);
|
||||||
|
if (w.state === 'red') return r.stop(`${p.workflow} run ${w.run.id} failed (${short(w.failing, 4) || 'no failing job named'}): Step 5 triage on its logs; fix, commit, push, then accept again`);
|
||||||
|
if (w.state === 'none') return r.stop(`${w.reason}: push the branch (a pull_request or workflow_dispatch run must exist at HEAD), then accept again`);
|
||||||
|
}
|
||||||
|
for (const l of list) {
|
||||||
|
const m = mapped.get(l.label);
|
||||||
|
const res = m ? carryLane(cwd, base, l) : (await runLanes(cwd, [l], Infinity, { base }))[0]!;
|
||||||
|
if (m && !res.carried) { r.add(carryNoneLine(res)!); return r.stop(`lane ${l.label} was not carried from the green run (${short([res.carryNone ?? ''], 1).slice(0, 120)}): fix the cause, or run it here with gstack-ship-preflight --tests-only`); }
|
||||||
|
const ran = laneLine(res);
|
||||||
|
r.add(ran.line);
|
||||||
|
if (!ran.ok) return r.stop(`lane ${l.label} failed or ran zero tests: Step 5 triage, then push and accept again`);
|
||||||
|
const e = bin(cwd, 'gstack-evidence', ['check', '--label', l.label, '--expect-cmd', l.command, '--max-age', '24'], 60_000).stdout;
|
||||||
|
const line = /^EVIDENCE: .*$/m.exec(e)?.[0] ?? `EVIDENCE: unreadable label=${l.label}`;
|
||||||
|
r.add(line.replace(/ log=\S+/, ''));
|
||||||
|
if (!/EVIDENCE: FRESH/.test(line)) return r.stop(`lane ${l.label} is not FRESH at the pushed head: Step 16 stage 4 recovery`);
|
||||||
|
}
|
||||||
|
const pg = bin(cwd, 'gstack-pregate', ['--allow-repo-commands', '--base', base], 3_600_000);
|
||||||
|
const verdict = /^PREGATE: (.+?) tree=/m.exec(pg.stdout + pg.stderr)?.[1] ?? `unreadable (exit ${pg.status})`;
|
||||||
|
const carried = /^CI_CARRY: .*$/m.exec(pg.stdout + pg.stderr)?.[0];
|
||||||
|
if (carried) r.add(carried.length > 240 ? `${carried.slice(0, 240)}…` : carried);
|
||||||
|
r.add(`PREGATE: ${verdict} (final tree)`);
|
||||||
|
if (!/^(pass|warn|requires-remote cleared)/.test(verdict)) return r.stop(`pregate ${verdict} on the final tree: fix it, then rerun accept`);
|
||||||
|
return r.next('Step 18: the receipt records ci_carried from the ledger');
|
||||||
|
}
|
||||||
|
|
||||||
export function push(argv: readonly string[], cwd: string): Report {
|
export function push(argv: readonly string[], cwd: string): Report {
|
||||||
const { flag, has } = flagReader(argv);
|
const { flag, has } = flagReader(argv);
|
||||||
const r = new Report();
|
const r = new Report();
|
||||||
@@ -411,6 +489,13 @@ export function push(argv: readonly string[], cwd: string): Report {
|
|||||||
r.add(`SEALED: ${sealed.status === 0 ? 'clear' : sealed.status === 1 ? 'refused' : `not checked (exit ${sealed.status})`}`);
|
r.add(`SEALED: ${sealed.status === 0 ? 'clear' : sealed.status === 1 ? 'refused' : `not checked (exit ${sealed.status})`}`);
|
||||||
if (sealed.status === 1) return r.stop(`sealed output would be published: ${firstLine(sealed.stderr)}; remove it from the commits, then repeat Step 16`);
|
if (sealed.status === 1) return r.stop(`sealed output would be published: ${firstLine(sealed.stderr)}; remove it from the commits, then repeat Step 16`);
|
||||||
if (sealed.status !== 0) return r.stop(`the sealed-output guard did not run (exit ${sealed.status}: ${firstLine(sealed.stderr || sealed.stdout)}); repair it, then rerun push`);
|
if (sealed.status !== 0) return r.stop(`the sealed-output guard did not run (exit ${sealed.status}: ${firstLine(sealed.stderr || sealed.stdout)}); repair it, then rerun push`);
|
||||||
|
if (has('--early')) {
|
||||||
|
const out = path.resolve(cwd, git(cwd, ['rev-parse', '--git-path', 'gstack/pregate-early-push.json']).stdout.trim() || '.git/gstack/pregate-early-push.json');
|
||||||
|
const scan = bin(cwd, 'gstack-pregate', ['--stage', 'preflight', '--only', 'secrets', '--out', out, ...(flag('--base') ? ['--base', flag('--base')!] : [])], 600_000);
|
||||||
|
const verdict = /^PREGATE: (.+?) tree=/m.exec(scan.stdout + scan.stderr)?.[1] ?? `unreadable (exit ${scan.status})`;
|
||||||
|
r.add(`SECRETS: ${verdict} (early push)`);
|
||||||
|
if (scan.status !== 0 || !/^(pass|warn)/.test(verdict)) return r.stop(`the secrets scan did not pass before the early push (${firstLine(scan.stderr) || verdict}): remove the finding from the commits, then rerun push --early`);
|
||||||
|
}
|
||||||
const local = git(cwd, ['rev-parse', 'HEAD']).stdout.trim();
|
const local = git(cwd, ['rev-parse', 'HEAD']).stdout.trim();
|
||||||
const remote = git(cwd, ['ls-remote', '--heads', 'origin', `refs/heads/${branch}`], 120_000);
|
const remote = git(cwd, ['ls-remote', '--heads', 'origin', `refs/heads/${branch}`], 120_000);
|
||||||
if (remote.status !== 0) return r.stop(`cannot verify the remote branch (${firstLine(remote.stderr)}): restore access before pushing`);
|
if (remote.status !== 0) return r.stop(`cannot verify the remote branch (${firstLine(remote.stderr)}): restore access before pushing`);
|
||||||
@@ -431,11 +516,12 @@ export async function release(argv: readonly string[], cwd: string): Promise<{ t
|
|||||||
const usage = (m: string) => ({ text: `gstack-ship-release: ${m}\n`, exit: 2 });
|
const usage = (m: string) => ({ text: `gstack-ship-release: ${m}\n`, exit: 2 });
|
||||||
if (cmd !== 'push' && !flag('--base')) return usage(`${cmd} needs --base <base>`);
|
if (cmd !== 'push' && !flag('--base')) return usage(`${cmd} needs --base <base>`);
|
||||||
if (cmd === 'push' && !flag('--branch')) return usage('push needs --branch <the branch Step 1 saved>');
|
if (cmd === 'push' && !flag('--branch')) return usage('push needs --branch <the branch Step 1 saved>');
|
||||||
|
if (cmd === 'accept' && !(lanes(rest) as unknown[]).length) return usage('accept needs at least one --lane <label>=<command> (the Step 5 lanes)');
|
||||||
if (typeof lanes(rest) === 'string') return usage(lanes(rest) as string);
|
if (typeof lanes(rest) === 'string') return usage(lanes(rest) as string);
|
||||||
const answer = flag('--answer');
|
const answer = flag('--answer');
|
||||||
if (has('--answer') && (!answer?.trim() || answer.startsWith('--') || /[\r\n]/.test(answer) || answer.length > 200)) return usage('--answer needs the owner\'s words: one non-empty line of at most 200 characters');
|
if (has('--answer') && (!answer?.trim() || answer.startsWith('--') || /[\r\n]/.test(answer) || answer.length > 200)) return usage('--answer needs the owner\'s words: one non-empty line of at most 200 characters');
|
||||||
if (cmd === 'finish' && (!flag('--reviewed-tree') || !flag('--docs-post'))) return usage('finish needs --reviewed-tree <the tree saved before Step 12> and --docs-post <Step 14.5\'s post-audit snapshot>');
|
if (cmd === 'finish' && (!flag('--reviewed-tree') || !flag('--docs-post'))) return usage('finish needs --reviewed-tree <the tree saved before Step 12> and --docs-post <Step 14.5\'s post-audit snapshot>');
|
||||||
const r = cmd === 'version' ? version(rest, cwd) : cmd === 'finish' ? await finish(rest, cwd) : cmd === 'push' ? push(rest, cwd) : null;
|
const r = cmd === 'version' ? version(rest, cwd) : cmd === 'finish' ? await finish(rest, cwd) : cmd === 'push' ? push(rest, cwd) : cmd === 'accept' ? await accept(rest, cwd) : null;
|
||||||
if (!r) return usage(`unknown command ${cmd}; run gstack-ship-release --help`);
|
if (!r) return usage(`unknown command ${cmd}; run gstack-ship-release --help`);
|
||||||
return { text: r.render(), exit: r.exit };
|
return { text: r.render(), exit: r.exit };
|
||||||
}
|
}
|
||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "gstack",
|
"name": "gstack",
|
||||||
"version": "1.93.42",
|
"version": "1.94.0",
|
||||||
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
|
|||||||
+8
-9
@@ -7,8 +7,8 @@ Read this, not `ship/SKILL.md`, when a parent agent runs /ship
|
|||||||
the `GSTACK_LAUNCH` assignments your installer printed (`. <its GSTACK_LAUNCH_FILE>`), never
|
the `GSTACK_LAUNCH` assignments your installer printed (`. <its GSTACK_LAUNCH_FILE>`), never
|
||||||
`export $(...)`, `env`, `printenv` or a bare `set` in a tool fence.
|
`export $(...)`, `env`, `printenv` or a bare `set` in a tool fence.
|
||||||
|
|
||||||
**Rules.** Each phase below is one call: one program (Capy: an execute program; else a
|
**Rules.** Each phase below is one call: one program (Capy: an execute program; else a script)
|
||||||
script) that runs the bins, reads, dispatches and waits, returning only result lines and reader
|
that runs the bins, reads, dispatches and waits, returning only result lines and reader
|
||||||
outputs, printed verbatim. The model keeps only lead judgment, fixes, CHANGELOG, PR prose,
|
outputs, printed verbatim. The model keeps only lead judgment, fixes, CHANGELOG, PR prose,
|
||||||
gates, the report. `STOP:` (exit 3) means nothing after it ran; `NEXT:` names
|
gates, the report. `STOP:` (exit 3) means nothing after it ran; `NEXT:` names
|
||||||
the next act. Read a section only when a phase or the list at the end sends you, never
|
the next act. Read a section only when a phase or the list at the end sends you, never
|
||||||
@@ -21,7 +21,7 @@ start tokens, `Pre-Landing Review:`, `Gate edits:`, the reviewed tree), **Releas
|
|||||||
version, CHANGELOG, commits), **Documentation** (Step 14.5) and **Next steps** (every phase: the next act).
|
version, CHANGELOG, commits), **Documentation** (Step 14.5) and **Next steps** (every phase: the next act).
|
||||||
|
|
||||||
**Reviewers are the sections' own.** Every reviewer brief, its severity and confidence rules
|
**Reviewers are the sections' own.** Every reviewer brief, its severity and confidence rules
|
||||||
and gates come from its section verbatim (placeholders filled); this file
|
and gates come from its section verbatim; this file
|
||||||
never shortens a reviewer's brief. A blocking reader's prompt is its `gstack-review-dispatch render`
|
never shortens a reviewer's brief. A blocking reader's prompt is its `gstack-review-dispatch render`
|
||||||
file, unchanged; its findings reach `merged.jsonl` only via `collect`; `judge` refuses a drop or downgrade. Audit readers (Steps 7, 8, 14.5) cannot block, so each runs at low effort on a small model,
|
file, unchanged; its findings reach `merged.jsonl` only via `collect`; `judge` refuses a drop or downgrade. Audit readers (Steps 7, 8, 14.5) cannot block, so each runs at low effort on a small model,
|
||||||
Step 14.5 once on Sonnet 5.5; findings go to the PR body.
|
Step 14.5 once on Sonnet 5.5; findings go to the PR body.
|
||||||
@@ -60,17 +60,15 @@ Install check, this file.
|
|||||||
## 2. Preflight, tests, evals and reads: Steps 0.9, 1, 2, 3, 4, 5, 6, 6.5, 10
|
## 2. Preflight, tests, evals and reads: Steps 0.9, 1, 2, 3, 4, 5, 6, 6.5, 10
|
||||||
|
|
||||||
`gstack-ship-budget start --record <rec> --mode observe [--token-cap <n>] --reading R --credits C`, then
|
`gstack-ship-budget start --record <rec> --mode observe [--token-cap <n>] --reading R --credits C`, then
|
||||||
`gstack-ship-preflight --base <base> --lane <label>=<command>...` (AGENTS.md or CLAUDE.md test commands). `CARRY: carried` ends the run on its `GSTACK_RESULT:`
|
`gstack-ship-preflight --base <base> --lane <label>=<command>...` (repo test commands). `CARRY: carried` ends the run on its `GSTACK_RESULT:`
|
||||||
line. A lane passes on `tests_ran: N` with exit 0; `tests_ran: 0` is ZERO-RUN, a failed lane; `unknown`
|
line. `ship_lanes_from`: push early (`sections/tests.md#ci-carry`). A lane passes on `tests_ran: N` with exit 0; `0` is ZERO-RUN (failed); `unknown`
|
||||||
is `count unavailable`. In-branch failures stop; after a repair rerun `--tests-only`.
|
is `count unavailable`. In-branch failures stop; after a repair rerun `--tests-only`.
|
||||||
`requires-remote` needs `gstack-pregate clear` before Step 18. When prompts, templates, judges or eval
|
`requires-remote` needs `gstack-pregate clear` before Step 18. When prompts, templates, judges or eval
|
||||||
harness changed, start the eval selector and pre-merge command (detached past 30 min); else
|
harness changed, start the eval selector and pre-merge command (detached past 30 min); else
|
||||||
record `evals: none affected`. Step 6.5 runs only if `gstack-greptile-early detect` finds
|
record `evals: none affected`. Step 6.5 runs only if `gstack-greptile-early detect` finds
|
||||||
Greptile. Then `gstack-ship-preflight --audit --base <base> --out <run dir>` writes the diff once
|
Greptile. Then `gstack-ship-preflight --audit --base <base> --out <run dir>` writes the diff once
|
||||||
(`CONTEXT:`) and the TODOS.md items naming touched paths (`TODOS:`), and prints the coverage map,
|
(`CONTEXT:`) and the TODOS.md items naming touched paths (`TODOS:`), and prints the coverage map,
|
||||||
plan candidates and `READER: coverage=<needed|none> plan=<needed|none>`. Programs read the todos and
|
plan candidates and `READER: coverage=<needed|none> plan=<needed|none>`; read a section only when its reader is `needed`.
|
||||||
commits files and `sections/test-coverage.md` or `sections/plan-completion.md` only when its reader is
|
|
||||||
`needed`.
|
|
||||||
Never read TODOS.md or the full diff into the parent. A `PERSIST:` hint never replaces Step 9.
|
Never read TODOS.md or the full diff into the parent. A `PERSIST:` hint never replaces Step 9.
|
||||||
|
|
||||||
## 3. Audits and review: Steps 7, 8, 9, 9.1, 9.2, 9.2.1
|
## 3. Audits and review: Steps 7, 8, 9, 9.1, 9.2, 9.2.1
|
||||||
@@ -112,7 +110,8 @@ Commit in bisectable chunks; `gstack-ship-release finish --base <base> --reviewe
|
|||||||
--docs-post <post> --lane ... [--build <cmd>] --trailer '<t>'`: `ROUTE:`, `DOCS:`, `EVIDENCE:` (FRESH;
|
--docs-post <post> --lane ... [--build <cmd>] --trailer '<t>'`: `ROUTE:`, `DOCS:`, `EVIDENCE:` (FRESH;
|
||||||
STALE or MISSING reruns once), `PREGATE:`; print them, the last `Gate edits:` and each `tests_ran`. Then
|
STALE or MISSING reruns once), `PREGATE:`; print them, the last `Gate edits:` and each `tests_ran`. Then
|
||||||
`gstack-ship-release push --branch <branch> [--gate-open <n>]`: `REDACT_PREPUSH:`, `SEALED:`, `PUSH:`
|
`gstack-ship-release push --branch <branch> [--gate-open <n>]`: `REDACT_PREPUSH:`, `SEALED:`, `PUSH:`
|
||||||
(pushed, ALREADY_PUSHED). Unattended, push defers the pre-push offer; else its STOP runs Step 17's guard block (`HOOK_STATE:`, `HOOKS_IN_GIT_DIR:`, `PREPUSH_PROMPTED:`). Never force-push.
|
(pushed, ALREADY_PUSHED). `EVIDENCE: DEFERRED`: the `accept` its `NEXT:` names; final-head
|
||||||
|
repo gates run beside CI. Unattended, push defers the pre-push offer; else its STOP runs Step 17's guard block (`HOOK_STATE:`, `HOOKS_IN_GIT_DIR:`, `PREPUSH_PROMPTED:`). Never force-push.
|
||||||
|
|
||||||
## 8. Publish: Steps 18, 19, 19-C, 20, 21, 22
|
## 8. Publish: Steps 18, 19, 19-C, 20, 21, 22
|
||||||
|
|
||||||
|
|||||||
+8
-9
@@ -5,8 +5,8 @@ Read this, not `ship/SKILL.md`, when a parent agent runs /ship
|
|||||||
the `GSTACK_LAUNCH` assignments your installer printed (`. <its GSTACK_LAUNCH_FILE>`), never
|
the `GSTACK_LAUNCH` assignments your installer printed (`. <its GSTACK_LAUNCH_FILE>`), never
|
||||||
`export $(...)`, `env`, `printenv` or a bare `set` in a tool fence.
|
`export $(...)`, `env`, `printenv` or a bare `set` in a tool fence.
|
||||||
|
|
||||||
**Rules.** Each phase below is one call: one program (Capy: an execute program; else a
|
**Rules.** Each phase below is one call: one program (Capy: an execute program; else a script)
|
||||||
script) that runs the bins, reads, dispatches and waits, returning only result lines and reader
|
that runs the bins, reads, dispatches and waits, returning only result lines and reader
|
||||||
outputs, printed verbatim. The model keeps only lead judgment, fixes, CHANGELOG, PR prose,
|
outputs, printed verbatim. The model keeps only lead judgment, fixes, CHANGELOG, PR prose,
|
||||||
gates, the report. `STOP:` (exit 3) means nothing after it ran; `NEXT:` names
|
gates, the report. `STOP:` (exit 3) means nothing after it ran; `NEXT:` names
|
||||||
the next act. Read a section only when a phase or the list at the end sends you, never
|
the next act. Read a section only when a phase or the list at the end sends you, never
|
||||||
@@ -19,7 +19,7 @@ start tokens, `Pre-Landing Review:`, `Gate edits:`, the reviewed tree), **Releas
|
|||||||
version, CHANGELOG, commits), **Documentation** (Step 14.5) and **Next steps** (every phase: the next act).
|
version, CHANGELOG, commits), **Documentation** (Step 14.5) and **Next steps** (every phase: the next act).
|
||||||
|
|
||||||
**Reviewers are the sections' own.** Every reviewer brief, its severity and confidence rules
|
**Reviewers are the sections' own.** Every reviewer brief, its severity and confidence rules
|
||||||
and gates come from its section verbatim (placeholders filled); this file
|
and gates come from its section verbatim; this file
|
||||||
never shortens a reviewer's brief. A blocking reader's prompt is its `gstack-review-dispatch render`
|
never shortens a reviewer's brief. A blocking reader's prompt is its `gstack-review-dispatch render`
|
||||||
file, unchanged; its findings reach `merged.jsonl` only via `collect`; `judge` refuses a drop or downgrade. Audit readers (Steps 7, 8, 14.5) cannot block, so each runs at low effort on a small model,
|
file, unchanged; its findings reach `merged.jsonl` only via `collect`; `judge` refuses a drop or downgrade. Audit readers (Steps 7, 8, 14.5) cannot block, so each runs at low effort on a small model,
|
||||||
Step 14.5 once on Sonnet 5.5; findings go to the PR body.
|
Step 14.5 once on Sonnet 5.5; findings go to the PR body.
|
||||||
@@ -58,17 +58,15 @@ Install check, this file.
|
|||||||
## 2. Preflight, tests, evals and reads: Steps 0.9, 1, 2, 3, 4, 5, 6, 6.5, 10
|
## 2. Preflight, tests, evals and reads: Steps 0.9, 1, 2, 3, 4, 5, 6, 6.5, 10
|
||||||
|
|
||||||
`gstack-ship-budget start --record <rec> --mode observe [--token-cap <n>] --reading R --credits C`, then
|
`gstack-ship-budget start --record <rec> --mode observe [--token-cap <n>] --reading R --credits C`, then
|
||||||
`gstack-ship-preflight --base <base> --lane <label>=<command>...` (AGENTS.md or CLAUDE.md test commands). `CARRY: carried` ends the run on its `GSTACK_RESULT:`
|
`gstack-ship-preflight --base <base> --lane <label>=<command>...` (repo test commands). `CARRY: carried` ends the run on its `GSTACK_RESULT:`
|
||||||
line. A lane passes on `tests_ran: N` with exit 0; `tests_ran: 0` is ZERO-RUN, a failed lane; `unknown`
|
line. `ship_lanes_from`: push early (`sections/tests.md#ci-carry`). A lane passes on `tests_ran: N` with exit 0; `0` is ZERO-RUN (failed); `unknown`
|
||||||
is `count unavailable`. In-branch failures stop; after a repair rerun `--tests-only`.
|
is `count unavailable`. In-branch failures stop; after a repair rerun `--tests-only`.
|
||||||
`requires-remote` needs `gstack-pregate clear` before Step 18. When prompts, templates, judges or eval
|
`requires-remote` needs `gstack-pregate clear` before Step 18. When prompts, templates, judges or eval
|
||||||
harness changed, start the eval selector and pre-merge command (detached past 30 min); else
|
harness changed, start the eval selector and pre-merge command (detached past 30 min); else
|
||||||
record `evals: none affected`. Step 6.5 runs only if `gstack-greptile-early detect` finds
|
record `evals: none affected`. Step 6.5 runs only if `gstack-greptile-early detect` finds
|
||||||
Greptile. Then `gstack-ship-preflight --audit --base <base> --out <run dir>` writes the diff once
|
Greptile. Then `gstack-ship-preflight --audit --base <base> --out <run dir>` writes the diff once
|
||||||
(`CONTEXT:`) and the TODOS.md items naming touched paths (`TODOS:`), and prints the coverage map,
|
(`CONTEXT:`) and the TODOS.md items naming touched paths (`TODOS:`), and prints the coverage map,
|
||||||
plan candidates and `READER: coverage=<needed|none> plan=<needed|none>`. Programs read the todos and
|
plan candidates and `READER: coverage=<needed|none> plan=<needed|none>`; read a section only when its reader is `needed`.
|
||||||
commits files and `sections/test-coverage.md` or `sections/plan-completion.md` only when its reader is
|
|
||||||
`needed`.
|
|
||||||
Never read TODOS.md or the full diff into the parent. A `PERSIST:` hint never replaces Step 9.
|
Never read TODOS.md or the full diff into the parent. A `PERSIST:` hint never replaces Step 9.
|
||||||
|
|
||||||
## 3. Audits and review: Steps 7, 8, 9, 9.1, 9.2, 9.2.1
|
## 3. Audits and review: Steps 7, 8, 9, 9.1, 9.2, 9.2.1
|
||||||
@@ -110,7 +108,8 @@ Commit in bisectable chunks; `gstack-ship-release finish --base <base> --reviewe
|
|||||||
--docs-post <post> --lane ... [--build <cmd>] --trailer '<t>'`: `ROUTE:`, `DOCS:`, `EVIDENCE:` (FRESH;
|
--docs-post <post> --lane ... [--build <cmd>] --trailer '<t>'`: `ROUTE:`, `DOCS:`, `EVIDENCE:` (FRESH;
|
||||||
STALE or MISSING reruns once), `PREGATE:`; print them, the last `Gate edits:` and each `tests_ran`. Then
|
STALE or MISSING reruns once), `PREGATE:`; print them, the last `Gate edits:` and each `tests_ran`. Then
|
||||||
`gstack-ship-release push --branch <branch> [--gate-open <n>]`: `REDACT_PREPUSH:`, `SEALED:`, `PUSH:`
|
`gstack-ship-release push --branch <branch> [--gate-open <n>]`: `REDACT_PREPUSH:`, `SEALED:`, `PUSH:`
|
||||||
(pushed, ALREADY_PUSHED). Unattended, push defers the pre-push offer; else its STOP runs Step 17's guard block (`HOOK_STATE:`, `HOOKS_IN_GIT_DIR:`, `PREPUSH_PROMPTED:`). Never force-push.
|
(pushed, ALREADY_PUSHED). `EVIDENCE: DEFERRED`: the `accept` its `NEXT:` names; final-head
|
||||||
|
repo gates run beside CI. Unattended, push defers the pre-push offer; else its STOP runs Step 17's guard block (`HOOK_STATE:`, `HOOKS_IN_GIT_DIR:`, `PREPUSH_PROMPTED:`). Never force-push.
|
||||||
|
|
||||||
## 8. Publish: Steps 18, 19, 19-C, 20, 21, 22
|
## 8. Publish: Steps 18, 19, 19-C, 20, 21, 22
|
||||||
|
|
||||||
|
|||||||
@@ -222,6 +222,22 @@ after the final stamp (`gstack-ship-release finish` does); Step 18 refuses a sta
|
|||||||
|
|
||||||
Use the project's test commands discovered in Step 4 or documented in CLAUDE.md/AGENTS.md. Run every applicable suite; do not assume Rails or Vitest. Pass each as `--lane <label>=<command>`; use the same lane labels and exact commands again in Step 16.
|
Use the project's test commands discovered in Step 4 or documented in CLAUDE.md/AGENTS.md. Run every applicable suite; do not assume Rails or Vitest. Pass each as `--lane <label>=<command>`; use the same lane labels and exact commands again in Step 16.
|
||||||
|
|
||||||
|
<a id="ci-carry"></a>
|
||||||
|
**CI is the one full test run.** When `origin/<base>`'s `.gstack/pregate.json` maps a lane's exact
|
||||||
|
command in `ship_lanes_from` (gstack maps `bun run test`), that lane is carried from a verified CI run
|
||||||
|
at this exact, clean tree instead of running here: `LANE: <label> carried run=<id>/<attempt> tests_ran=<n>`.
|
||||||
|
Before the first push run only targeted tests and the repro (the full suite too when the diff touches
|
||||||
|
merge-check code: `lib/ship-*`, `lib/pregate/*`, the ship and pregate bins,
|
||||||
|
`scripts/lib/shard-engine.ts`). Push early so CI runs beside the review: commit, `gstack-ship-release push
|
||||||
|
--branch <branch> --early` (sealed and secrets scans first), open a draft PR, then `gstack-ci-carry wait` in
|
||||||
|
the background; once it prints `CI_WAIT: green`, rerun `--tests-only` and the mapped lane is carried. A red
|
||||||
|
run is a failed lane: triage it below. Batch review fixes into one commit and one push. `CI_CARRY: none
|
||||||
|
label=<l> (<reason>)` names why a mapped lane ran here (a dirty tree, no green run at HEAD yet, a fork or stale
|
||||||
|
run, a newer red run, a tree.json that misses an inventory file). Any other lane that already passed at this
|
||||||
|
tree, command and runtime is reused (`reused=<time>`), never run twice. In Step 16, `finish` defers a mapped
|
||||||
|
lane it cannot carry (`EVIDENCE: DEFERRED`); after the push, `gstack-ship-release accept --base <base>
|
||||||
|
--lane …` carries it from the green run at the final head and reruns the pregate there.
|
||||||
|
|
||||||
**If no applicable test suite exists:** Name the untested scope. AskUserQuestion:
|
**If no applicable test suite exists:** Name the untested scope. AskUserQuestion:
|
||||||
A) Add tests (recommended), B) Ship with this named testing
|
A) Add tests (recommended), B) Ship with this named testing
|
||||||
gap, or C) Stop. Reuse an actual prior B answer only for the same scope and
|
gap, or C) Stop. Reuse an actual prior B answer only for the same scope and
|
||||||
|
|||||||
@@ -18,6 +18,22 @@ after the final stamp (`gstack-ship-release finish` does); Step 18 refuses a sta
|
|||||||
|
|
||||||
Use the project's test commands discovered in Step 4 or documented in CLAUDE.md/AGENTS.md. Run every applicable suite; do not assume Rails or Vitest. Pass each as `--lane <label>=<command>`; use the same lane labels and exact commands again in Step 16.
|
Use the project's test commands discovered in Step 4 or documented in CLAUDE.md/AGENTS.md. Run every applicable suite; do not assume Rails or Vitest. Pass each as `--lane <label>=<command>`; use the same lane labels and exact commands again in Step 16.
|
||||||
|
|
||||||
|
<a id="ci-carry"></a>
|
||||||
|
**CI is the one full test run.** When `origin/<base>`'s `.gstack/pregate.json` maps a lane's exact
|
||||||
|
command in `ship_lanes_from` (gstack maps `bun run test`), that lane is carried from a verified CI run
|
||||||
|
at this exact, clean tree instead of running here: `LANE: <label> carried run=<id>/<attempt> tests_ran=<n>`.
|
||||||
|
Before the first push run only targeted tests and the repro (the full suite too when the diff touches
|
||||||
|
merge-check code: `lib/ship-*`, `lib/pregate/*`, the ship and pregate bins,
|
||||||
|
`scripts/lib/shard-engine.ts`). Push early so CI runs beside the review: commit, `gstack-ship-release push
|
||||||
|
--branch <branch> --early` (sealed and secrets scans first), open a draft PR, then `gstack-ci-carry wait` in
|
||||||
|
the background; once it prints `CI_WAIT: green`, rerun `--tests-only` and the mapped lane is carried. A red
|
||||||
|
run is a failed lane: triage it below. Batch review fixes into one commit and one push. `CI_CARRY: none
|
||||||
|
label=<l> (<reason>)` names why a mapped lane ran here (a dirty tree, no green run at HEAD yet, a fork or stale
|
||||||
|
run, a newer red run, a tree.json that misses an inventory file). Any other lane that already passed at this
|
||||||
|
tree, command and runtime is reused (`reused=<time>`), never run twice. In Step 16, `finish` defers a mapped
|
||||||
|
lane it cannot carry (`EVIDENCE: DEFERRED`); after the push, `gstack-ship-release accept --base <base>
|
||||||
|
--lane …` carries it from the green run at the final head and reruns the pregate there.
|
||||||
|
|
||||||
**If no applicable test suite exists:** Name the untested scope. AskUserQuestion:
|
**If no applicable test suite exists:** Name the untested scope. AskUserQuestion:
|
||||||
A) Add tests (recommended), B) Ship with this named testing
|
A) Add tests (recommended), B) Ship with this named testing
|
||||||
gap, or C) Stop. Reuse an actual prior B answer only for the same scope and
|
gap, or C) Stop. Reuse an actual prior B answer only for the same scope and
|
||||||
|
|||||||
@@ -0,0 +1,111 @@
|
|||||||
|
/**
|
||||||
|
* lib/ci-carry.ts (GSTA-258 piece 1; design fixes A2, A3): CI's verified run at HEAD's exact, clean tree stands in
|
||||||
|
* for running a lane again. The dependencies are fakes, so nothing here reaches GitHub.
|
||||||
|
*/
|
||||||
|
import { describe, expect, test } from 'bun:test';
|
||||||
|
import { findCarry, shipLanesErrors, waitForRun } from '../lib/ci-carry';
|
||||||
|
import { validateConfig } from '../lib/pregate/config';
|
||||||
|
import { receiptSummaryLine, validateReceipt } from '../lib/ship-receipt';
|
||||||
|
import type { ApiJob, ApiRun, RunDeps, RunProfile, RunSource, TreeRecord } from '../lib/run-evidence';
|
||||||
|
|
||||||
|
const REPO = 'garrytan/gstack';
|
||||||
|
const HEAD = 'a'.repeat(40);
|
||||||
|
const TREE = 'b'.repeat(40);
|
||||||
|
const NOW = new Date('2026-11-02T12:00:00Z');
|
||||||
|
const PROFILE: RunProfile = { workflow: 'free-tests.yml', events: ['pull_request', 'workflow_dispatch'], jobs: ['free-tests'], artifact: 'tested-tree', os: 'Linux' };
|
||||||
|
|
||||||
|
const run = (id: number, over: Partial<ApiRun> = {}): ApiRun => ({
|
||||||
|
id, run_attempt: 1, status: 'completed', conclusion: 'success', event: 'workflow_dispatch', head_sha: HEAD, path: '.github/workflows/free-tests.yml',
|
||||||
|
created_at: `2026-11-02T1${id % 10}:00:00Z`.replace('T1', 'T0'), updated_at: '2026-11-02T11:30:00Z', repository: { full_name: REPO }, head_repository: { full_name: REPO }, ...over,
|
||||||
|
});
|
||||||
|
const record = (id: number, over: Partial<TreeRecord> = {}): TreeRecord => ({ run_id: id, run_attempt: 1, event: 'workflow_dispatch', sha: HEAD, tree: TREE, files: ['test/a.test.ts', 'test/b.test.ts'], tests: 40, os: 'Linux', ...over });
|
||||||
|
function source(runs: ApiRun[], records: Record<number, TreeRecord> = {}, jobs: ApiJob[] = [{ name: 'free-tests', conclusion: 'success', completed_at: '2026-11-02T11:30:00Z' }]): RunSource {
|
||||||
|
const deps: RunDeps = {
|
||||||
|
run: id => { const r = runs.find(x => x.id === id); if (!r) throw new Error('HTTP 404'); return r; },
|
||||||
|
runsForHead: (_wf, sha) => runs.filter(r => r.head_sha === sha),
|
||||||
|
jobs: () => jobs,
|
||||||
|
artifact: id => { if (!records[id]) throw new Error('no artifact'); return records[id]!; },
|
||||||
|
commitTree: sha => (sha === HEAD ? TREE : null),
|
||||||
|
};
|
||||||
|
return { repo: REPO, deps, profiles: { 'ci-reuse/free-tests': PROFILE }, now: NOW };
|
||||||
|
}
|
||||||
|
const clean = () => TREE;
|
||||||
|
const carry = (src: RunSource, over: Partial<Parameters<typeof findCarry>[1]> = {}) => findCarry(src, { profiles: ['ci-reuse/free-tests'], head: HEAD, tree: TREE, wtree: clean, ...over });
|
||||||
|
const why = (v: ReturnType<typeof findCarry>) => (v.ok ? 'ok' : v.reason);
|
||||||
|
|
||||||
|
describe('findCarry', () => {
|
||||||
|
// Value: protects=a green verified run at this exact tree carries, with a receipt naming it; fails_when=a verified run is ignored or its count is lost;
|
||||||
|
// why_new=GSTA-258 piece 1; seam=lib/run-evidence.ts verifyRun; no_claim=GitHub's API shape (ghRunDeps)
|
||||||
|
test('the newest verified run at HEAD carries, and the receipt names its run and tree', () => {
|
||||||
|
const v = findCarry(source([run(4)], { 4: record(4) }), { profiles: ['ci-reuse/free-tests'], head: HEAD, tree: TREE, wtree: clean, inventory: () => ['test/a.test.ts', 'test/b.test.ts'] });
|
||||||
|
expect(v.ok).toBe(true);
|
||||||
|
if (!v.ok) return;
|
||||||
|
expect(v.tests).toBe(40);
|
||||||
|
expect(v.runs.map(e => e.run_id)).toEqual([4]);
|
||||||
|
expect(v.receipt).toMatch(/^ci-carry tree=bbbbbbbbbbbb tests=40 runs=4\/1 profiles=ci-reuse\/free-tests heads=aaaaaaaaaaaa verified=/);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=A2, a carry binds to the working-tree fingerprint before and after; fails_when=a dirty tree or a tree edited during the lookup carries
|
||||||
|
test('a dirty working tree, or one that changes while runs are read, never carries', () => {
|
||||||
|
expect(why(carry(source([run(4)], { 4: record(4) }), { wtree: () => 'c'.repeat(40) }))).toBe('dirty_tree');
|
||||||
|
expect(why(carry(source([run(4)], { 4: record(4) }), { wtree: () => null }))).toBe('dirty_tree');
|
||||||
|
let reads = 0;
|
||||||
|
expect(why(carry(source([run(4)], { 4: record(4) }), { wtree: () => (reads++ ? 'c'.repeat(40) : TREE) }))).toBe('tree_changed');
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=A3, a lane carries only what CI provably ran; fails_when=a tree.json missing an inventory file still carries
|
||||||
|
test('the inventory must be covered by the runs\' tree.json files', () => {
|
||||||
|
const src = source([run(4)], { 4: record(4) });
|
||||||
|
const v = carry(src, { inventory: () => ['test/a.test.ts', 'test/c.test.ts'] });
|
||||||
|
expect(why(v)).toBe('inventory_uncovered');
|
||||||
|
expect(v.ok ? '' : v.detail).toContain('test/c.test.ts');
|
||||||
|
expect(why(carry(src, { inventory: () => ({ error: 'exited 1' }) }))).toBe('inventory_failed');
|
||||||
|
expect(why(carry(src, { inventory: () => [] }))).toBe('inventory_empty');
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=fork runs, newer red or running runs, stale runs and empty runs never carry; fails_when=any of them is accepted
|
||||||
|
test('no run, a fork run, a newer red or running run, a stale run and a run with no tests are refused', () => {
|
||||||
|
expect(why(carry(source([])))).toBe('no_run');
|
||||||
|
expect(why(carry(source([run(4, { head_repository: { full_name: 'evil/gstack' } })], { 4: record(4) })))).toBe('no_verified_run');
|
||||||
|
const red = carry(source([run(4), run(6, { conclusion: 'failure' })], { 4: record(4), 6: record(6) }));
|
||||||
|
expect(why(red)).toBe('no_verified_run');
|
||||||
|
expect(red.ok ? '' : red.detail).toContain('run 6: run_not_success; run 4: newer_run_vetoes');
|
||||||
|
expect(why(carry(source([run(4), run(6, { status: 'in_progress', conclusion: null })], { 4: record(4) })))).toBe('no_verified_run');
|
||||||
|
expect(why(carry(source([run(4)], { 4: record(4) }, [{ name: 'free-tests', conclusion: 'success', completed_at: '2026-10-30T00:00:00Z' }])))).toBe('no_verified_run');
|
||||||
|
expect(why(carry(source([run(4)], { 4: record(4, { files: [] }) })))).toBe('no_tests');
|
||||||
|
expect(why(carry(source([run(4)], { 4: record(4) }), { profiles: ['nope'] }))).toBe('no_profile');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('waitForRun', () => {
|
||||||
|
const clockAt = () => { let t = 0; return { clock: () => t, sleep: async (ms: number) => { t += ms; } }; };
|
||||||
|
// Value: protects=the bounded wait reports green, red with failing jobs, still running and none; fails_when=it waits forever or calls a red run green
|
||||||
|
test('green, red with its failing jobs, still running at the deadline, and none', async () => {
|
||||||
|
const o = (runs: ApiRun[], jobs?: ApiJob[]) => waitForRun(source(runs, {}, jobs).deps, { workflow: 'free-tests.yml', events: PROFILE.events, head: HEAD, timeoutMs: 60_000, intervalMs: 20_000, ...clockAt() });
|
||||||
|
expect((await o([run(4)])).state).toBe('green');
|
||||||
|
const red = await o([run(4, { conclusion: 'failure' })], [{ name: 'free-tests', conclusion: 'failure' }, { name: 'typecheck', conclusion: 'success' }]);
|
||||||
|
expect(red).toMatchObject({ state: 'red', failing: ['free-tests'] });
|
||||||
|
expect((await o([run(4, { status: 'in_progress', conclusion: null })])).state).toBe('running');
|
||||||
|
expect((await o([])).state).toBe('none');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('ship_lanes_from config and the ci_carried receipt field', () => {
|
||||||
|
// Value: protects=A3, the mapping names real profiles and one inventory command; fails_when=a typo or a label-shaped entry validates
|
||||||
|
test('ship_lanes_from validates its profiles and inventory', () => {
|
||||||
|
const profiles = { 'ci-reuse/free-tests': PROFILE };
|
||||||
|
expect(validateConfig({ run_profiles: profiles, ship_lanes_from: { 'bun run test': { profiles: ['ci-reuse/free-tests'], inventory: 'bun run list' } } })).toEqual([]);
|
||||||
|
expect(shipLanesErrors({ 'bun run test': { profiles: ['nope'], inventory: 'x' } }, ['ci-reuse/free-tests']).map(e => e.message)).toEqual(['"nope" is not a run_profiles name']);
|
||||||
|
expect(shipLanesErrors({ 'bun run test': { profiles: ['ci-reuse/free-tests'], inventory: 'a\nb' } }, ['ci-reuse/free-tests'])[0]!.path).toBe('ship_lanes_from["bun run test"].inventory');
|
||||||
|
expect(shipLanesErrors({ 'bun run test': { profiles: ['ci-reuse/free-tests'] , inventory: 'x', label: 'free' } }, ['ci-reuse/free-tests'])[0]!.message).toContain('unknown key');
|
||||||
|
expect(shipLanesErrors(['bun run test'], [])[0]!.path).toBe('ship_lanes_from');
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=the receipt and its SHIP_RECEIPT line name the carried runs; fails_when=ci_carried is dropped or malformed entries validate
|
||||||
|
test('the receipt records ci_carried and its summary line names each run', () => {
|
||||||
|
const receipt = { schema_version: 1 as const, head: HEAD, base: 'main', ci_carried: [{ lane: 'free', run: 41, attempt: 2, tree: TREE }] };
|
||||||
|
expect(validateReceipt(receipt)).toEqual([]);
|
||||||
|
expect(receiptSummaryLine(receipt)).toContain('ci_carried=free:41/2');
|
||||||
|
expect(validateReceipt({ ...receipt, ci_carried: [{ lane: 'free', run: '41' }] }).length).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,141 @@
|
|||||||
|
/**
|
||||||
|
* gstack-evidence reuse and CI-sourced rows (GSTA-258 pieces 1-2, design fixes A4 and A5).
|
||||||
|
*
|
||||||
|
* run --reuse: a lane that passed at this exact working tree, command and execution identity never runs again;
|
||||||
|
* the newest matching row decides (a newer failure vetoes), the reused row keeps its original timestamp and the
|
||||||
|
* max age applies to it. record-ci: a lane CI carried is an explicit CI-sourced row for a run it verified itself, never a fictitious local run.
|
||||||
|
*/
|
||||||
|
import { afterEach, beforeEach, describe, expect, setDefaultTimeout, test } from 'bun:test';
|
||||||
|
import { createHash } from 'node:crypto';
|
||||||
|
import * as fs from 'node:fs';
|
||||||
|
import * as os from 'node:os';
|
||||||
|
import * as path from 'node:path';
|
||||||
|
import { findFilesBySuffix, gitIn } from './helpers/scratch-repo';
|
||||||
|
import { runBin } from './helpers/run-bin';
|
||||||
|
|
||||||
|
setDefaultTimeout(60_000);
|
||||||
|
const EVIDENCE = path.join(import.meta.dir, '..', 'bin', 'gstack-evidence');
|
||||||
|
let home: string;
|
||||||
|
let repo: string;
|
||||||
|
let counter: string;
|
||||||
|
|
||||||
|
const run = (args: string[]) => runBin(EVIDENCE, args, { cwd: repo, env: { GSTACK_HOME: home } });
|
||||||
|
const ledger = () => findFilesBySuffix(path.join(home, 'projects'), '-evidence.jsonl')[0]!;
|
||||||
|
const rows = (): any[] => fs.readFileSync(ledger(), 'utf8').trim().split('\n').map(l => JSON.parse(l));
|
||||||
|
const rewrite = (edit: (r: any[]) => any[]) => fs.writeFileSync(ledger(), edit(rows()).map(r => JSON.stringify(r)).join('\n') + '\n');
|
||||||
|
const executions = () => (fs.existsSync(counter) ? fs.readFileSync(counter, 'utf8').length : 0);
|
||||||
|
const lane = (pass: number, fail = 0) => `printf x >> '${counter}'; printf ' ${pass} pass\\n ${fail} fail\\nRan ${pass + fail} tests across 1 file. [1.00ms]\\n' >&2; exit ${fail ? 1 : 0}`;
|
||||||
|
const wtree = () => runBin(path.join(import.meta.dir, '..', 'bin', 'gstack-wtree'), [], { cwd: repo }).stdout.trim();
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
home = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-reuse-home-'));
|
||||||
|
repo = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-reuse-repo-'));
|
||||||
|
counter = path.join(home, 'executions');
|
||||||
|
gitIn(repo, 'init -q -b main');
|
||||||
|
fs.writeFileSync(path.join(repo, 'src.txt'), 'v1\n');
|
||||||
|
gitIn(repo, 'add src.txt');
|
||||||
|
gitIn(repo, 'commit -q -m init');
|
||||||
|
});
|
||||||
|
afterEach(() => {
|
||||||
|
fs.rmSync(home, { recursive: true, force: true });
|
||||||
|
fs.rmSync(repo, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('gstack-evidence run --reuse (A5)', () => {
|
||||||
|
// Value: protects=a lane that passed at this tree never reruns, and the reuse is visible; fails_when=--reuse runs the suite again, or reports a pass it never recorded;
|
||||||
|
// why_new=GSTA-258 piece 2; seam=the ledger and the wtree fingerprint; no_claim=cross-machine reuse (CI carries that)
|
||||||
|
test('a pass at the same tree, command and identity is reused with its original timestamp and count', () => {
|
||||||
|
expect(run(['run', '--label', 'unit', '--', lane(4)]).status).toBe(0);
|
||||||
|
const first = rows().at(-1);
|
||||||
|
const again = run(['run', '--reuse', '--label', 'unit', '--', lane(4)]);
|
||||||
|
expect(again.status).toBe(0);
|
||||||
|
expect(executions()).toBe(1);
|
||||||
|
expect(again.stderr).toContain(`gstack-evidence: reused label=unit tree=${wtree().slice(0, 12)} from=${first.ts}`);
|
||||||
|
expect(again.stderr).toMatch(/gstack-evidence: recorded label=unit exit=0 tests_ran=4 log=\S+/);
|
||||||
|
const copy = rows().at(-1);
|
||||||
|
expect(copy.ts).toBe(first.ts);
|
||||||
|
expect(copy.reused_from).toBe(first.ts);
|
||||||
|
expect(copy.identity).toBe(first.identity);
|
||||||
|
expect(typeof first.identity).toBe('string');
|
||||||
|
expect(run(['check', '--label', 'unit']).stdout).toContain('EVIDENCE: FRESH label=unit exit=0 tests_ran=4');
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=only the newest matching row decides; fails_when=an older pass is reused under a newer failure on the same tree
|
||||||
|
test('a newer failure on the same tree vetoes an older pass; without --reuse nothing is reused', () => {
|
||||||
|
expect(run(['run', '--label', 'unit', '--', lane(4)]).status).toBe(0);
|
||||||
|
expect(run(['run', '--label', 'unit', '--', lane(3, 1)]).status).toBe(1);
|
||||||
|
expect(run(['run', '--reuse', '--label', 'unit', '--', lane(3, 1)]).status).toBe(1);
|
||||||
|
expect(executions()).toBe(3);
|
||||||
|
expect(run(['run', '--label', 'unit', '--', lane(4)]).status).toBe(0);
|
||||||
|
expect(run(['run', '--label', 'unit', '--', lane(4)]).status).toBe(0);
|
||||||
|
expect(executions()).toBe(5);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=reuse binds to the exact command, tree, count and identity; fails_when=a changed command, an edited file, a zero-run or unknown row, or another runtime reuses
|
||||||
|
test('another command, an edited tree, a zero-run or unknown count, or another identity runs again', () => {
|
||||||
|
expect(run(['run', '--label', 'unit', '--', lane(4)]).status).toBe(0);
|
||||||
|
expect(run(['run', '--reuse', '--label', 'unit', '--', lane(5)]).status).toBe(0);
|
||||||
|
expect(executions()).toBe(2);
|
||||||
|
fs.writeFileSync(path.join(repo, 'src.txt'), 'v2\n');
|
||||||
|
expect(run(['run', '--reuse', '--label', 'unit', '--', lane(5)]).status).toBe(0);
|
||||||
|
expect(executions()).toBe(3);
|
||||||
|
rewrite(r => r.map(x => ({ ...x, tests_ran: 0 })));
|
||||||
|
expect(run(['run', '--reuse', '--label', 'unit', '--', lane(5)]).status).toBe(0);
|
||||||
|
expect(executions()).toBe(4);
|
||||||
|
rewrite(r => r.map(x => ({ ...x, tests_ran: 'unknown' })));
|
||||||
|
expect(run(['run', '--reuse', '--label', 'unit', '--', lane(5)]).status).toBe(0);
|
||||||
|
expect(executions()).toBe(5);
|
||||||
|
rewrite(r => r.map(x => ({ ...x, identity: 'f'.repeat(64) })));
|
||||||
|
expect(run(['run', '--reuse', '--label', 'unit', '--', lane(5)]).status).toBe(0);
|
||||||
|
expect(executions()).toBe(6);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=the max age applies to the original pass, never to a refreshed copy; fails_when=reuse makes an old pass look new
|
||||||
|
test('a pass older than --max-age is not reused, and a reused copy keeps the age check honest', () => {
|
||||||
|
expect(run(['run', '--label', 'unit', '--', lane(4)]).status).toBe(0);
|
||||||
|
rewrite(r => r.map(x => ({ ...x, ts: new Date(Date.now() - 30 * 3600_000).toISOString() })));
|
||||||
|
expect(run(['run', '--reuse', '--label', 'unit', '--', lane(4)]).status).toBe(0);
|
||||||
|
expect(executions()).toBe(2);
|
||||||
|
rewrite(r => r.map(x => ({ ...x, ts: new Date(Date.now() - 2 * 3600_000).toISOString() })));
|
||||||
|
expect(run(['run', '--reuse', '--max-age', '1', '--label', 'unit', '--', lane(4)]).status).toBe(0);
|
||||||
|
expect(executions()).toBe(3);
|
||||||
|
rewrite(r => r.map(x => ({ ...x, ts: new Date(Date.now() - 3 * 3600_000).toISOString() })));
|
||||||
|
expect(run(['run', '--reuse', '--label', 'unit', '--', lane(4)]).status).toBe(0);
|
||||||
|
expect(executions()).toBe(3);
|
||||||
|
expect(run(['check', '--label', 'unit', '--max-age', '2']).stdout).toContain('EVIDENCE: STALE label=unit');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('gstack-evidence record-ci and carried (A4)', () => {
|
||||||
|
const ciRow = () => {
|
||||||
|
const tree = wtree();
|
||||||
|
return { ts: new Date(Date.now() - 600_000).toISOString(), label: 'free', command: 'bun run test', cmd_sha256: createHash('sha256').update('bun run test').digest('hex'), exit: 0, duration_s: 0, commit: gitIn(repo, 'rev-parse HEAD').trim(), tree, dirty: false, wtree: tree, tests_ran: 900, source: 'ci', runs: ['41/2'], profiles: ['ci-reuse/free-tests'] };
|
||||||
|
};
|
||||||
|
|
||||||
|
// Value: protects=a carried lane is an explicit CI row the FRESH check accepts by source, and the receipt lists it only at its own tree;
|
||||||
|
// fails_when=a check cannot tell a CI row from a local run, or a stale carry from an earlier head reaches ci_carried
|
||||||
|
test('a CI-sourced row is FRESH for its exact command, and carried lists it only at its tree', () => {
|
||||||
|
expect(run(['run', '--label', 'unit', '--', lane(2)]).status).toBe(0);
|
||||||
|
fs.appendFileSync(ledger(), JSON.stringify(ciRow()) + '\n');
|
||||||
|
const chk = run(['check', '--label', 'free', '--expect-cmd', 'bun run test', '--max-age', '24']);
|
||||||
|
expect(chk.status).toBe(0);
|
||||||
|
expect(chk.stdout).toContain('EVIDENCE: FRESH label=free exit=0 tests_ran=900');
|
||||||
|
expect(chk.stdout).toContain('source=ci runs=41/2');
|
||||||
|
expect(JSON.parse(run(['carried', '--json']).stdout)).toEqual([{ lane: 'free', run: 41, attempt: 2, tree: wtree() }]);
|
||||||
|
fs.writeFileSync(path.join(repo, 'src.txt'), 'v2\n');
|
||||||
|
gitIn(repo, 'commit -qam two');
|
||||||
|
expect(JSON.parse(run(['carried', '--json']).stdout)).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=record-ci finds and verifies the CI run itself, so no caller supplies run ids, a test count or a tree (/ship pass 1 red-team#1);
|
||||||
|
// fails_when=the forged call from that finding writes a FRESH carried row; seam=no origin, so the base's ship_lanes_from is unreadable
|
||||||
|
test('record-ci takes no run facts from its caller and records nothing without a verified run', () => {
|
||||||
|
const forged = run(['record-ci', '--label', 'free', '--runs', '1/1', '--profiles', 'anything', '--tree', wtree(), '--tests', '999', '--finished', new Date().toISOString(), '--', 'bun run test']);
|
||||||
|
expect(forged.status).toBe(2);
|
||||||
|
expect(forged.stderr).toContain('usage: gstack-evidence record-ci --label <L> --base <base> -- <command>');
|
||||||
|
const unverified = run(['record-ci', '--label', 'free', '--base', 'main', '--', 'bun run test']);
|
||||||
|
expect(unverified.status).toBe(1);
|
||||||
|
expect(unverified.stderr).toMatch(/^CARRY_NONE: config_unavailable: /m);
|
||||||
|
expect(findFilesBySuffix(path.join(home, 'projects'), '-evidence.jsonl')).toEqual([]);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
/**
|
||||||
|
* A fake gh for the CI-carry tests (GSTA-258): serves the Actions run, jobs, artifact and download reads
|
||||||
|
* lib/run-evidence.ts ghRunDeps makes from a world file, and answers every other gh call with nothing.
|
||||||
|
*/
|
||||||
|
import * as fs from 'node:fs';
|
||||||
|
import * as path from 'node:path';
|
||||||
|
|
||||||
|
export interface FakeWorld {
|
||||||
|
runs: Array<Record<string, unknown> & { id: number; run_attempt: number; head_sha: string; path: string }>;
|
||||||
|
jobs: Record<string, Array<{ name: string; conclusion: string | null; completed_at?: string }>>;
|
||||||
|
trees: Record<string, Record<string, unknown>>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The stub body for shipFixture's gh(): run this file with the world beside it. */
|
||||||
|
export const fakeGhScript = (worldFile: string) => `exec bun --no-env-file --config=/dev/null '${import.meta.path}' '${worldFile}' "$@"`;
|
||||||
|
|
||||||
|
if (import.meta.main) {
|
||||||
|
const [worldFile, ...a] = process.argv.slice(2);
|
||||||
|
const world: FakeWorld = fs.existsSync(worldFile!) ? JSON.parse(fs.readFileSync(worldFile!, 'utf8')) : { runs: [], jobs: {}, trees: {} };
|
||||||
|
const lines = (list: unknown[]) => process.stdout.write(list.map(x => JSON.stringify(x)).join('\n') + (list.length ? '\n' : ''));
|
||||||
|
if (a[0] === 'api') {
|
||||||
|
const jq = a.indexOf('--jq');
|
||||||
|
const p = a.filter((x, i) => i > 0 && x !== '--paginate' && i !== jq && i !== jq + 1)[0] ?? '';
|
||||||
|
const [route, query = ''] = p.split('?');
|
||||||
|
const q = new URLSearchParams(query);
|
||||||
|
let m: RegExpExecArray | null;
|
||||||
|
if ((m = /\/actions\/runs\/(\d+)$/.exec(route!))) {
|
||||||
|
const run = world.runs.find(r => r.id === Number(m![1]));
|
||||||
|
if (!run) { process.stderr.write('HTTP 404'); process.exit(1); }
|
||||||
|
process.stdout.write(JSON.stringify(run));
|
||||||
|
} else if ((m = /\/actions\/workflows\/([^/]+)\/runs$/.exec(route!))) {
|
||||||
|
lines(world.runs.filter(r => r.head_sha === q.get('head_sha') && r.path === `.github/workflows/${m![1]}`));
|
||||||
|
} else if ((m = /\/actions\/runs\/(\d+)\/attempts\/\d+\/jobs$/.exec(route!))) {
|
||||||
|
lines(world.jobs[m[1]!] ?? []);
|
||||||
|
} else if ((m = /\/actions\/runs\/(\d+)\/artifacts$/.exec(route!))) {
|
||||||
|
lines(world.trees[m[1]!] ? [{ name: q.get('name'), size_in_bytes: 200, expired: false }] : []);
|
||||||
|
} else { process.stderr.write(`fake gh: no route for ${p}`); process.exit(1); }
|
||||||
|
} else if (a[0] === 'run' && a[1] === 'download') {
|
||||||
|
const tree = world.trees[a[2]!];
|
||||||
|
if (!tree) process.exit(1);
|
||||||
|
const dir = a[a.indexOf('--dir') + 1]!;
|
||||||
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
|
fs.writeFileSync(path.join(dir, 'tree.json'), JSON.stringify(tree));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
/**
|
||||||
|
* CI carry and the remote gate (GSTA-258 A): after a verified CI run carries part of the tests stage on a small
|
||||||
|
* machine, the files left over can all be ones the remote gate never runs (gstack's paid evals). The gate itself then
|
||||||
|
* reports tests=0 (PREGATE_REMOTE_PAID_ONLY) and `clear --run` refuses paid_only, so the obligation could never clear.
|
||||||
|
* The carried run clears it instead, through remote_gate_runs and its inventory check. The run API is a fake.
|
||||||
|
*/
|
||||||
|
import { afterEach, describe, expect, test } from 'bun:test';
|
||||||
|
import type { PregateFile } from '../lib/headless-artifacts';
|
||||||
|
import { buildContext } from '../lib/pregate/context';
|
||||||
|
import { clearDecision, runPregate, type RunCheck } from '../lib/pregate/index';
|
||||||
|
import type { ApiRun, RunProfile } from '../lib/run-evidence';
|
||||||
|
import { BASE_PREGATE, cleanup, commit, makeFixtureRepo, type FixtureRepo } from './helpers/pregate-fixture';
|
||||||
|
|
||||||
|
const repos: FixtureRepo[] = [];
|
||||||
|
afterEach(() => { while (repos.length) cleanup(repos.pop()!); });
|
||||||
|
|
||||||
|
const LANE = 'pregate.json/remote_gate';
|
||||||
|
const NOW = new Date('2026-11-02T12:00:00Z');
|
||||||
|
const LOCAL_OS = process.platform === 'darwin' ? 'macOS' : 'Linux';
|
||||||
|
const PROFILE: RunProfile = { workflow: 'free-tests.yml', events: ['pull_request', 'workflow_dispatch'], jobs: ['unit'], artifact: 'tested-tree', os: LOCAL_OS };
|
||||||
|
const PAID = 'test/nightly/retry-heavy.test.js';
|
||||||
|
const FREE = ['test/retry.test.js', 'test/util.test.js'];
|
||||||
|
|
||||||
|
function fixture(): FixtureRepo {
|
||||||
|
const repo = makeFixtureRepo({ pregate: { ...BASE_PREGATE, remote_gate: 'node scripts/remote-gate.js' } });
|
||||||
|
repos.push(repo);
|
||||||
|
commit(repo, { 'lib/util.js': 'exports.add = (a, b) => a + b + 0;\n', 'lib/retry.js': 'exports.MAX_RETRIES = 3;\nexports.TIMEOUT_MS = 5000;\nexports.retry = (fn) => fn();\nexports.touched = true;\n' }, 'touch two subjects');
|
||||||
|
return repo;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Run 100 of free-tests.yml at HEAD, green, its tree.json listing `files`; the gate's inventory is FREE and its skipped command names PAID. */
|
||||||
|
function runCheck(repo: FixtureRepo, files: string[]): RunCheck {
|
||||||
|
const head = repo.git('rev-parse', 'HEAD');
|
||||||
|
const api: ApiRun = { id: 100, run_attempt: 1, status: 'completed', conclusion: 'success', event: 'workflow_dispatch', head_sha: head, path: '.github/workflows/free-tests.yml', created_at: '2026-11-02T10:00:00Z', updated_at: '2026-11-02T10:30:00Z', repository: { full_name: 'fx/fx' }, head_repository: { full_name: 'fx/fx' } };
|
||||||
|
return {
|
||||||
|
repo: 'fx/fx', profiles: { 'ci-reuse/free-tests': PROFILE }, now: NOW,
|
||||||
|
gateRuns: { profiles: ['ci-reuse/free-tests'], inventory: 'INVENTORY', skipped: 'SKIPPED' },
|
||||||
|
inventory: (command, _tree, selected) => (command === 'SKIPPED' ? (selected ?? []).filter(f => f === PAID) : FREE),
|
||||||
|
deps: {
|
||||||
|
run: () => api, runsForHead: (_wf, sha) => (sha === head ? [api] : []), jobs: () => [{ name: 'unit', conclusion: 'success' }],
|
||||||
|
artifact: () => ({ run_id: 100, run_attempt: 1, event: 'workflow_dispatch', sha: head, head_sha: head, tree: repo.git('rev-parse', 'HEAD^{tree}'), os: [LOCAL_OS], files, tests: files.length * 4 }),
|
||||||
|
commitTree: sha => repo.git('rev-parse', `${sha}^{tree}`),
|
||||||
|
},
|
||||||
|
} as RunCheck;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function carried(repo: FixtureRepo, check: RunCheck): Promise<{ file: PregateFile; progress: string[] }> {
|
||||||
|
const progress: string[] = [];
|
||||||
|
const loaded = buildContext({ repoRoot: repo.work, base: 'main', policyFrom: 'base', allowRepoCommands: true, env: { ...process.env, ...repo.env }, progress: l => progress.push(l), cores: 4, testBackend: 'local' });
|
||||||
|
if (!('ctx' in loaded)) throw new Error('fixture context refused');
|
||||||
|
loaded.ctx.fromRun = { ref: '100', check };
|
||||||
|
return { file: await runPregate(loaded.ctx, { stage: 'tests' }), progress };
|
||||||
|
}
|
||||||
|
const gate = (file: PregateFile) => file.requires_remote?.find(r => r.lane === LANE);
|
||||||
|
|
||||||
|
describe.skipIf(process.platform === 'win32')('CI carry clears the remote gate on a small machine', () => {
|
||||||
|
// Value: protects=a carried run whose remainder is only gate-skipped (paid) files leaves no obligation nothing can clear;
|
||||||
|
// fails_when=the remainder stays a remote_gate obligation that the gate reports tests=0 for and clear --run refuses paid_only;
|
||||||
|
// why_new=GSTA-258 A, the PREGATE_REMOTE_PAID_ONLY dead end seen at 0fba5533 (112 paid files after run 41554380563); seam=runPregate with a fake run API
|
||||||
|
test('the carried run clears the gate when only skipped files remain, and the receipt counts them', async () => {
|
||||||
|
const repo = fixture();
|
||||||
|
const check = runCheck(repo, FREE);
|
||||||
|
const { file, progress } = await carried(repo, check);
|
||||||
|
const obligation = gate(file)!;
|
||||||
|
expect(obligation.tests).toEqual([PAID]);
|
||||||
|
expect(clearDecision({ ...file, requires_remote: [{ ...obligation, receipt: null }] }, LANE, { runs: ['100'], evidence: check }, repo.work)).toMatchObject({ refused: 'PREGATE_RECEIPT_INVALID', message: expect.stringContaining('reason=paid_only') });
|
||||||
|
expect([obligation.receipt, obligation.cleared_by]).toEqual([expect.stringMatching(new RegExp(`^remote-gate tree=${file.tree.slice(0, 12)} tests=8 skipped=1 runs=100/1 `)), 'ci-carry']);
|
||||||
|
expect(progress).toContain(`CI_CARRY: cleared ${LANE} ${obligation.receipt}`);
|
||||||
|
expect(file.verdict).toBe('requires-remote cleared');
|
||||||
|
}, 120_000);
|
||||||
|
|
||||||
|
// Value: protects=the carry clears the gate only under clear --run's own checks; fails_when=a run that misses an inventory file clears it
|
||||||
|
test('a run that misses a gate inventory file leaves the gate uncleared and says why', async () => {
|
||||||
|
const repo = fixture();
|
||||||
|
const { file, progress } = await carried(repo, runCheck(repo, ['test/util.test.js']));
|
||||||
|
expect(gate(file)?.receipt).toBeNull();
|
||||||
|
expect(progress).toContainEqual(expect.stringMatching(new RegExp(`^CI_CARRY: ${LANE} stays uncleared \\(REUSE: refused reason=inventory_uncovered`)));
|
||||||
|
}, 120_000);
|
||||||
|
|
||||||
|
// Value: protects=when the base declares remote_gate_runs, the uncleared gate names the CI route first and the VM only as an opt-in;
|
||||||
|
// fails_when=the obligation sends an agent to the paid VM while a CI run at the same tree would clear it; why_new=GSTA-258 A, plan choice F1
|
||||||
|
test('with remote_gate_runs on the base, an uncleared gate names the CI route first and the VM as the opt-in', async () => {
|
||||||
|
const repo = makeFixtureRepo({ pregate: { ...BASE_PREGATE, remote_gate: 'node scripts/remote-gate.js', run_profiles: { 'ci-reuse/free-tests': PROFILE }, remote_gate_runs: { profiles: ['ci-reuse/free-tests'], inventory: 'echo test/retry.test.js' } } });
|
||||||
|
repos.push(repo);
|
||||||
|
commit(repo, { 'lib/retry.js': 'exports.MAX_RETRIES = 3;\nexports.TIMEOUT_MS = 5000;\nexports.retry = (fn) => fn();\nexports.touched = 2;\n' }, 'touch retry');
|
||||||
|
const loaded = buildContext({ repoRoot: repo.work, base: 'main', policyFrom: 'base', allowRepoCommands: true, env: { ...process.env, ...repo.env }, progress: () => {}, cores: 4, testBackend: 'local' });
|
||||||
|
if (!('ctx' in loaded)) throw new Error('fixture context refused');
|
||||||
|
const file = await runPregate(loaded.ctx, { stage: 'tests' });
|
||||||
|
expect(gate(file)?.receipt).toBeNull();
|
||||||
|
const detail = file.checks.find(c => c.id === 'lanes')?.detail ?? '';
|
||||||
|
expect(detail).toContain(`a verified CI run at this tree clears it: push, then gstack-ci-carry wait --base main and rerun gstack-pregate --allow-repo-commands`);
|
||||||
|
expect(detail.indexOf('gstack-ci-carry wait')).toBeLessThan(detail.indexOf('opt in to the VM: gstack-pregate --remote --spend-cap <usd> --allow-repo-commands'));
|
||||||
|
loaded.ctx.ciCarry = false;
|
||||||
|
const off = (await runPregate(loaded.ctx, { stage: 'tests' })).checks.find(c => c.id === 'lanes')?.detail ?? '';
|
||||||
|
expect(off).not.toContain('gstack-ci-carry wait');
|
||||||
|
expect(off).toContain('run it, then gstack-pregate clear --remote pregate.json/remote_gate');
|
||||||
|
}, 120_000);
|
||||||
|
});
|
||||||
@@ -0,0 +1,194 @@
|
|||||||
|
/**
|
||||||
|
* CI-carried lanes end to end (GSTA-258 pieces 1, 2 and 4; design fixes A1-A4): gstack-ship-preflight, gstack-ship-release
|
||||||
|
* finish, push --early and accept, gstack-pregate's automatic --from-run and gstack-ci-carry, against a fixture repository
|
||||||
|
* whose gh is a fake serving one Actions world, so nothing reaches GitHub.
|
||||||
|
*/
|
||||||
|
import { afterEach, describe, expect, setDefaultTimeout, test } from 'bun:test';
|
||||||
|
import * as fs from 'node:fs';
|
||||||
|
import * as path from 'node:path';
|
||||||
|
import { fakeGhScript, type FakeWorld } from './helpers/fake-gh-runs';
|
||||||
|
import { findFilesBySuffix } from './helpers/scratch-repo';
|
||||||
|
import { lineOf, PREFLIGHT, RELEASE, shipFixture, type ShipFx } from './helpers/ship-bin-fixture';
|
||||||
|
|
||||||
|
setDefaultTimeout(120_000);
|
||||||
|
const BIN = path.dirname(RELEASE);
|
||||||
|
const fixtures: ShipFx[] = [];
|
||||||
|
afterEach(() => { while (fixtures.length) fixtures.pop()!.cleanup(); });
|
||||||
|
|
||||||
|
const FREE = "printf x >> ../free-runs; printf ' 3 pass\\n 0 fail\\nRan 3 tests across 1 file. [1.00ms]\\n' >&2";
|
||||||
|
const UNIT = "printf x >> ../unit-runs; printf ' 2 pass\\n 0 fail\\nRan 2 tests across 1 file. [1.00ms]\\n' >&2";
|
||||||
|
const PROFILE = { workflow: 'free.yml', events: ['pull_request', 'workflow_dispatch'], jobs: ['suite'], artifact: 'tested-tree', os: 'Linux' };
|
||||||
|
const BASE = {
|
||||||
|
'.gstack/pregate.json': JSON.stringify({ run_profiles: { 'ci/free': PROFILE }, ship_lanes_from: { [FREE]: { profiles: ['ci/free'], inventory: "printf 'test/a.test.ts\\ntest/b.test.ts\\n'" } } }, null, 2) + '\n',
|
||||||
|
'.github/workflows/free.yml': 'name: Free\non: [pull_request, workflow_dispatch]\njobs:\n suite:\n runs-on: ubuntu-latest\n steps:\n - run: bun test\n',
|
||||||
|
};
|
||||||
|
const fx = () => { const f = shipFixture(BASE); fixtures.push(f); return f; };
|
||||||
|
const runs = (f: ShipFx, name: string) => { const p = path.join(f.root, name); return fs.existsSync(p) ? fs.readFileSync(p, 'utf8').length : 0; };
|
||||||
|
const at = (minutesAgo: number) => new Date(Date.now() - minutesAgo * 60_000).toISOString();
|
||||||
|
|
||||||
|
/** One run of free.yml at HEAD (id 7 unless named), green with a tree.json naming HEAD's tree, unless overridden. */
|
||||||
|
function world(f: ShipFx, over: { run?: Record<string, unknown>; tree?: Record<string, unknown>; others?: Array<Record<string, unknown>>; jobs?: Array<{ name: string; conclusion: string | null }> } = {}): void {
|
||||||
|
const head = f.git('rev-parse', 'HEAD').stdout.trim();
|
||||||
|
const tree = f.git('rev-parse', 'HEAD^{tree}').stdout.trim();
|
||||||
|
const run = { id: 7, run_attempt: 1, status: 'completed', conclusion: 'success', event: 'workflow_dispatch', head_sha: head, path: '.github/workflows/free.yml', created_at: at(20), run_started_at: at(20), updated_at: at(5), repository: { full_name: 'fx/fx' }, head_repository: { full_name: 'fx/fx' }, ...over.run };
|
||||||
|
const w: FakeWorld = {
|
||||||
|
runs: [run, ...((over.others ?? []).map(o => ({ ...run, ...o })))] as FakeWorld['runs'],
|
||||||
|
jobs: { 7: (over.jobs ?? [{ name: 'suite', conclusion: 'success' }]).map(j => ({ ...j, completed_at: at(5) })), 9: [{ name: 'suite', conclusion: 'failure', completed_at: at(1) }] },
|
||||||
|
trees: { 7: { run_id: 7, run_attempt: 1, event: 'workflow_dispatch', sha: head, tree, files: ['test/a.test.ts', 'test/b.test.ts'], tests: 12, os: 'Linux', ...over.tree } },
|
||||||
|
};
|
||||||
|
const file = path.join(f.root, 'world.json');
|
||||||
|
fs.writeFileSync(file, JSON.stringify(w));
|
||||||
|
f.gh(fakeGhScript(file));
|
||||||
|
}
|
||||||
|
const preflight = (f: ShipFx, lanes = [`free=${FREE}`, `unit=${UNIT}`]) => f.run(PREFLIGHT, ['--tests-only', '--base', 'main', ...lanes.flatMap(l => ['--lane', l])]);
|
||||||
|
const lane = (out: string, label: string) => out.split('\n').find(l => l.startsWith(`LANE: ${label} `)) ?? '';
|
||||||
|
|
||||||
|
describe('Step 5 lanes: CI carry and the tree cache', () => {
|
||||||
|
// Value: protects=a lane the base maps is carried from CI at this exact tree and never runs; any other lane runs once and is reused at the same tree;
|
||||||
|
// fails_when=the free suite reruns locally beside a green CI run, or a passed lane reruns at an unchanged tree; why_new=GSTA-258 pieces 1-2; seam=fake gh, gstack-evidence ledger
|
||||||
|
test('the mapped lane is carried from the green run; the other lane runs once, then is reused', () => {
|
||||||
|
const f = fx();
|
||||||
|
world(f);
|
||||||
|
const first = preflight(f);
|
||||||
|
expect(first.status).toBe(0);
|
||||||
|
expect(lane(first.stdout, 'free')).toBe('LANE: free carried run=7/1 tests_ran=12');
|
||||||
|
expect(lane(first.stdout, 'unit')).toMatch(/^LANE: unit pass exit=0 tests_ran=2 log=\S+$/);
|
||||||
|
expect([runs(f, 'free-runs'), runs(f, 'unit-runs')]).toEqual([0, 1]);
|
||||||
|
const again = preflight(f);
|
||||||
|
expect(lane(again.stdout, 'unit')).toMatch(/^LANE: unit pass exit=0 tests_ran=2 log=\S+ reused=\S+$/);
|
||||||
|
expect([runs(f, 'free-runs'), runs(f, 'unit-runs')]).toEqual([0, 1]);
|
||||||
|
const chk = f.run(path.join(BIN, 'gstack-evidence'), ['check', '--label', 'free', '--expect-cmd', FREE]);
|
||||||
|
expect(chk.stdout).toContain('source=ci runs=7/1');
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=A2/A3 and verifyRun's refusals: a dirty tree, an uncovered inventory, a fork run and a newer red run never carry, and a label alone never maps;
|
||||||
|
// fails_when=any of them carries; seam=lib/ci-carry.ts findCarry through runLanes
|
||||||
|
test('a dirty tree, an uncovered inventory, a fork run, a newer red run or a changed command runs the lane here', () => {
|
||||||
|
const f = fx();
|
||||||
|
const cases: Array<[string, () => void, string[]?]> = [
|
||||||
|
['dirty_tree', () => { world(f); f.write({ 'src/b.ts': 'export const b = 3;\n' }); }],
|
||||||
|
['inventory_uncovered', () => { f.git('checkout', '--', 'src/b.ts'); world(f, { tree: { files: ['test/a.test.ts'] } }); }],
|
||||||
|
['fork_run', () => world(f, { run: { head_repository: { full_name: 'evil/fx' } } })],
|
||||||
|
['newer_run_vetoes', () => world(f, { others: [{ id: 9, conclusion: 'failure', created_at: at(2), run_started_at: at(2) }] })],
|
||||||
|
];
|
||||||
|
let expected = 0;
|
||||||
|
for (const [reason, setup] of cases) {
|
||||||
|
setup();
|
||||||
|
const out = preflight(f, [`free=${FREE}`]).stdout;
|
||||||
|
expect([reason, out]).toEqual([reason, expect.stringContaining('CI_CARRY: none label=free (')]);
|
||||||
|
expect(out).toContain(reason);
|
||||||
|
expect(lane(out, 'free')).toMatch(/^LANE: free pass exit=0 tests_ran=3 /);
|
||||||
|
expected += lane(out, 'free').includes('reused=') ? 0 : 1;
|
||||||
|
expect(runs(f, 'free-runs')).toBe(expected);
|
||||||
|
}
|
||||||
|
world(f);
|
||||||
|
const other = preflight(f, [`free=${FREE} `]).stdout;
|
||||||
|
expect(other).not.toContain('CI_CARRY');
|
||||||
|
expect(lane(other, 'free')).not.toContain('carried');
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=a CI-sourced ledger row exists only for a run gstack-evidence record-ci found and verified itself (/ship pass 1 red-team#1);
|
||||||
|
// fails_when=caller-supplied run ids, test counts or a tree make a FRESH carried row, or an unmapped command or a missing run records one; seam=fake gh
|
||||||
|
test('record-ci verifies the run itself: forged facts, an unmapped command or no run at HEAD record nothing', () => {
|
||||||
|
const f = fx();
|
||||||
|
const ev = (args: string[]) => f.run(path.join(BIN, 'gstack-evidence'), args);
|
||||||
|
const forged = ev(['record-ci', '--label', 'free', '--runs', '1/1', '--profiles', 'anything', '--tree', f.git('rev-parse', 'HEAD^{tree}').stdout.trim(), '--tests', '999', '--finished', at(1), '--', FREE]);
|
||||||
|
expect(forged.status).toBe(2);
|
||||||
|
world(f, { run: { head_sha: '0'.repeat(40) } });
|
||||||
|
const noRun = ev(['record-ci', '--label', 'free', '--base', 'main', '--', FREE]);
|
||||||
|
expect([noRun.status, noRun.stderr]).toEqual([1, expect.stringMatching(/^CARRY_NONE: no_run: /m)]);
|
||||||
|
world(f);
|
||||||
|
const unmapped = ev(['record-ci', '--label', 'free', '--base', 'main', '--', 'bun run test']);
|
||||||
|
expect([unmapped.status, unmapped.stderr]).toEqual([1, expect.stringMatching(/^CARRY_NONE: not_mapped: /m)]);
|
||||||
|
expect(ev(['check', '--label', 'free', '--expect-cmd', FREE]).stdout).not.toContain('EVIDENCE: FRESH');
|
||||||
|
expect(JSON.parse(ev(['carried', '--json']).stdout)).toEqual([]);
|
||||||
|
const ok = ev(['record-ci', '--label', 'free', '--base', 'main', '--', FREE]);
|
||||||
|
expect([ok.status, ok.stderr]).toEqual([0, expect.stringContaining('tests_ran=12 log=- source=ci runs=7/1 policy=origin/main@')]);
|
||||||
|
const row = JSON.parse(fs.readFileSync(findFilesBySuffix(path.join(f.state, 'projects'), '-evidence.jsonl')[0]!, 'utf8').trim().split('\n').at(-1)!);
|
||||||
|
expect(row.policy).toMatch(/^origin\/main@[0-9a-f]{12}:\.gstack\/pregate\.json$/);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('finish, push --early and accept (A1)', () => {
|
||||||
|
const finishArgs = (f: ShipFx, reviewed: string, post: string) => ['finish', '--base', 'main', '--reviewed-tree', reviewed, '--docs-post', post, '--lane', `free=${FREE}`];
|
||||||
|
const snapshot = (f: ShipFx) => {
|
||||||
|
const out = path.join(f.root, 'post.json');
|
||||||
|
expect(f.run(path.join(BIN, 'gstack-docs-candidate'), ['snapshot', '--out', out, '--audit-id', 'a1', '--mode', 'edit', '--base', 'main', '--docs', 'README.md']).status).toBe(0);
|
||||||
|
return out;
|
||||||
|
};
|
||||||
|
const unattended = { GSTACK_SESSION_KIND: 'unattended' };
|
||||||
|
|
||||||
|
// Value: protects=finish never runs a mapped lane locally on the stamped tree: it defers it to accept and runs only pregate's preflight stage; accept carries
|
||||||
|
// it from the green run at the pushed head and reruns the pregate there; fails_when=the stamped tree reruns the suite, or accept trusts a run at another tree;
|
||||||
|
// why_new=design fix A1; seam=fake gh, gstack-evidence, gstack-pregate
|
||||||
|
test('finish defers the mapped lane; accept carries it from the green run at the pushed head', () => {
|
||||||
|
const f = fx();
|
||||||
|
world(f);
|
||||||
|
const reviewed = f.run(path.join(BIN, 'gstack-wtree'), []).stdout.trim();
|
||||||
|
expect(f.run(RELEASE, ['version', '--base', 'main', '--bump', 'micro']).status).toBe(0);
|
||||||
|
const post = snapshot(f);
|
||||||
|
const pregateFile = path.join(f.work, '.gstack/tmp/pregate.json');
|
||||||
|
const before = fs.existsSync(pregateFile) ? fs.readFileSync(pregateFile, 'utf8') : null;
|
||||||
|
const fin = f.run(RELEASE, finishArgs(f, reviewed, post));
|
||||||
|
expect(fin.stdout).toMatch(/^EVIDENCE: DEFERRED label=free \(/m);
|
||||||
|
expect(lineOf(fin.stdout, 'PREGATE')).toMatch(/^PREGATE: preflight (pass|warn) on the stamped tree/);
|
||||||
|
expect(fs.existsSync(pregateFile) ? fs.readFileSync(pregateFile, 'utf8') : null).toBe(before);
|
||||||
|
expect(fin.stdout).toContain('gstack-ship-release accept --base main --lane');
|
||||||
|
expect(fin.status).toBe(0);
|
||||||
|
expect(runs(f, 'free-runs')).toBe(0);
|
||||||
|
const notPushed = f.run(RELEASE, ['accept', '--base', 'main', '--lane', `free=${FREE}`, '--wait', '0']);
|
||||||
|
expect(notPushed.stdout).toContain('is not origin/feat: run gstack-ship-release push first');
|
||||||
|
const push = f.run(RELEASE, ['push', '--branch', 'feat', '--early', '--base', 'main'], unattended);
|
||||||
|
expect(lineOf(push.stdout, 'SECRETS')).toMatch(/^SECRETS: (pass|warn) \(early push\)$/);
|
||||||
|
expect(lineOf(push.stdout, 'PUSH')).toMatch(/^PUSH: pushed /);
|
||||||
|
world(f, { run: { status: 'in_progress', conclusion: null } });
|
||||||
|
const running = f.run(RELEASE, ['accept', '--base', 'main', '--lane', `free=${FREE}`, '--wait', '0']);
|
||||||
|
expect(running.status).toBe(3);
|
||||||
|
expect(running.stdout).toContain('CI_WAIT: running ci/free run=7/1');
|
||||||
|
world(f, { run: { conclusion: 'failure' }, jobs: [{ name: 'suite', conclusion: 'failure' }] });
|
||||||
|
const red = f.run(RELEASE, ['accept', '--base', 'main', '--lane', `free=${FREE}`, '--wait', '0']);
|
||||||
|
expect(red.stdout).toContain('STOP: free.yml run 7 failed (suite)');
|
||||||
|
world(f);
|
||||||
|
const ok = f.run(RELEASE, ['accept', '--base', 'main', '--lane', `free=${FREE}`, '--wait', '0']);
|
||||||
|
expect(ok.stdout).toContain('CI_WAIT: green ci/free run=7/1');
|
||||||
|
expect(lane(ok.stdout, 'free')).toBe('LANE: free carried run=7/1 tests_ran=12');
|
||||||
|
expect(lineOf(ok.stdout, 'EVIDENCE')).toMatch(/^EVIDENCE: FRESH label=free exit=0 tests_ran=12 .* source=ci runs=7\/1$/);
|
||||||
|
expect(ok.stdout).toContain('CI_CARRY: accepted ci-carry tree=');
|
||||||
|
expect(ok.stdout).toContain('NEXT: Step 18');
|
||||||
|
expect(ok.status).toBe(0);
|
||||||
|
expect(runs(f, 'free-runs')).toBe(0);
|
||||||
|
const carried = JSON.parse(f.run(path.join(BIN, 'gstack-evidence'), ['carried', '--json']).stdout);
|
||||||
|
expect(carried).toEqual([{ lane: 'free', run: 7, attempt: 1, tree: f.git('rev-parse', 'HEAD^{tree}').stdout.trim() }]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('gstack-pregate tests stage and gstack-ci-carry', () => {
|
||||||
|
// Value: protects=the tests stage carries CI at HEAD's clean tree on its own, and --no-ci-carry or a dirty tree turns it off; fails_when=pregate reruns what CI ran, or carries a dirty tree;
|
||||||
|
// why_new=GSTA-258 piece 1; seam=fake gh; no_claim=a small machine's remote gate obligation (covered by from-run's tests)
|
||||||
|
test('the tests stage accepts the run on its own, and --no-ci-carry or a dirty tree does not', () => {
|
||||||
|
const f = fx();
|
||||||
|
world(f);
|
||||||
|
const pregate = (extra: string[] = []) => f.run(path.join(BIN, 'gstack-pregate'), ['--stage', 'tests', '--allow-repo-commands', '--base', 'main', ...extra]);
|
||||||
|
expect(pregate().stderr).toMatch(/^CI_CARRY: accepted ci-carry tree=[0-9a-f]{12} tests=12 runs=7\/1 profiles=ci\/free /m);
|
||||||
|
expect(pregate(['--no-ci-carry']).stderr).not.toContain('CI_CARRY');
|
||||||
|
f.write({ 'scratch.txt': 'x\n' });
|
||||||
|
expect(pregate().stderr).toContain('CI_CARRY: none (dirty_tree');
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=the CLI finds and waits on the same evidence; fails_when=find or wait accepts a red run or prints no verdict
|
||||||
|
test('gstack-ci-carry find and wait', () => {
|
||||||
|
const f = fx();
|
||||||
|
world(f);
|
||||||
|
const cli = (args: string[]) => f.run(path.join(BIN, 'gstack-ci-carry'), args);
|
||||||
|
const found = cli(['find', '--base', 'main']);
|
||||||
|
expect([found.status, found.stdout]).toEqual([0, expect.stringMatching(/^CI_CARRY: accepted ci-carry tree=/)]);
|
||||||
|
const mapped = cli(['find', '--base', 'main', '--command', FREE, '--allow-repo-commands']);
|
||||||
|
expect(mapped.status).toBe(0);
|
||||||
|
expect(cli(['find', '--base', 'main', '--command', 'bun run test', '--allow-repo-commands']).stdout).toContain('CI_CARRY: none (not_mapped');
|
||||||
|
expect(cli(['wait', '--base', 'main', '--timeout', '0']).stdout).toContain('CI_WAIT: green ci/free run=7/1');
|
||||||
|
world(f, { run: { conclusion: 'failure' }, jobs: [{ name: 'suite', conclusion: 'failure' }] });
|
||||||
|
const red = cli(['wait', '--base', 'main', '--timeout', '0']);
|
||||||
|
expect([red.status, red.stdout.trim()]).toEqual([1, 'CI_WAIT: red ci/free run=7/1 failing=suite']);
|
||||||
|
expect(cli(['bogus']).status).toBe(2);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
{
|
||||||
|
"bytes": 1463,
|
||||||
|
"why": "ship: CI-carry, finish/accept split and early push (GSTA-258 piece A)"
|
||||||
|
}
|
||||||
Reference in new issue
Block a user