gstack/test/fixtures/multi-agent-wave/reference-run/eng-outside.md

1. [Severity: High] Gate reuse still confuses permitted edits with equivalent behavior. C3 permits version substitutions without establishing that the affected gates are insensitive to those substitutions; existing digest checks explicitly consume VERSION and generated bytes (test/agents-digest.test.ts:22, test/agents-digest.test.ts:44). Requiring an unchanged raw lockfile hash also conflicts with permitting lockfile version-metadata changes, which the writer performs (bin/gstack-version-bump:486). Define equivalence separately for each reusable gate, retain raw and normalized identities, and rerun stamp-sensitive checks on every final stamped tree. Replace the impossible “changed tracked lockfile with an identical tree” fixture with distinct tests for metadata-only lockfile changes and environment changes under an identical tree.

2. [Severity: High] The portable bundle lacks the data and provenance needed to justify reuse. Working-tree fingerprints can reference unreachable Git objects, including uncommitted content; copying their hashes does not preserve those objects after the VM disappears (bin/gstack-wtree:24). Existing evidence deliberately distinguishes command freshness from trust and warns that existing labels cannot prove all required lanes ran (bin/gstack-evidence:23). Export the comparison material, complete required-lane manifest, executed selections, and producer-bound run metadata; distinguish imported assertions from independently verified evidence. Unknown execution identity or missing comparison objects must prevent reuse.

3. [Severity: High] Queue validation remains vulnerable to a check-to-merge race. C2 rechecks base and order “immediately before merge,” but another coordinator can advance either after that check. Gate-ahead also needs an explicit identity for the complete integration tree, including the ordered predecessor chain, rather than only one predecessor head. Bind allocation and receipts to repository, base SHA, ordered predecessors, PR head, and synthetic merge tree, then require a serialized merge operation with server-enforced head/base conditions or recompute when those conditions cannot be enforced.

4. [Severity: High] Journaling cannot make arbitrary release commands reversible. C2 promises dry-run purity and rollback across release_tool, generators, and a mirror, although those commands can modify undeclared files or perform external actions. The existing version writer already exposes partial completion, including successful VERSION writes followed by failed manifest or lockfile writes (bin/gstack-version-bump:475). Define a release-tool contract: execute in an isolated staging tree, enumerate allowed outputs, record preimages and postimages, and apply only after validation. Rollback must refuse to overwrite subsequent user edits, and mirror updates need an explicit partial-commit recovery state.

5. [Severity: High] The unattended runner needs a durable state machine, not just snapshots. B2’s promise never to repeat completed paid work fails when the provider finishes but the process dies before recording completion; snapshot hashes establish input identity, not whether execution finished (bin/gstack-autoplan-snapshot.ts:659). Specify attempt IDs, exclusive run ownership, atomic manifest transitions, cancellation, and an execution_unknown recovery state. Persist dispatch intent before launching and provider results before phase completion; ambiguous attempts must require reconciliation rather than automatic redispatch. Keep acknowledgments separate from immutable completion evidence.

6. [Severity: High] Gate answers can finalize a plan that reviewers never evaluated. B2 finalizes after applying choices and promises never to rerun completed paid phases, but a nonrecommended answer can materially change scope, architecture, and downstream reviews. The current close packet explicitly invalidates itself after implementation or accepted-decision edits (bin/gstack-autoplan-snapshot.ts:777). Require the caller to supply the expected gate_rev—the proposed answer signature currently has no such input—and atomically reject stale, conflicting, or partially parsed replies. Recompute affected review-input identities and reopen dependent phases when an answer changes reviewed content.

7. [Severity: High] Hashing reviewer prose cannot detect omitted findings. B0 claims that binding an immutable output file prevents omitted findings, but a matching hash proves only which file was retained; it does not prove that findings.jsonl contains everything that prose reported. Existing snapshot code explicitly limits its guarantees to explained bytes, not completeness or approval (bin/gstack-autoplan-snapshot.ts:677). Require canonical structured findings from each reviewer and reconcile their exact IDs into the aggregate, preserving disagreements and resolutions separately. Validation must also reject duplicate IDs, dependency cycles, malformed JSONL, and artifact paths escaping the run directory.

8. [Severity: High] The runner adapters do not yet define equivalent review capabilities. B3 treats a direct API response, CLI review, and supplied subagent file as interchangeable without specifying repository access, complete input delivery, terminal execution evidence, or model-family verification. Existing outside-review handling separates transport execution from review verdict, while native dispatch requires successful full-file reads (lib/outside-review-result.ts:4, bin/gstack-autoplan-snapshot.ts:661). Define one adapter interface covering capabilities, input digest, actual model metadata, cancellation, terminal result, and usage. An API adapter needs either a bounded repository tool loop or an explicitly limited supplied-input review; label coverage accordingly.

9. [Severity: High] PR B promises spending controls before their implementation arrives in PR E. --spend-cap and concurrent native/outside work require durable admission control in B, not a later ledger extraction. Current measurement admission operates on in-memory batch accounting, and its pricing source explicitly excludes billing categories (scripts/ship-measure.ts:409, test/helpers/pricing.ts:4). Move the minimal shared ledger into B, atomically check spent + reserved + next_estimate, and retain reservations for unknown-charge attempts after crashes. Specify whether deadlines cancel every child and how externally dispatched subagent spending is bounded or reported as unknown.

10. [Severity: High] Suppressing prompts does not enforce unattended privacy semantics. Skill startup already performs analytics work and invokes artifact synchronization outside the spawned-only onboarding guard (bin/gstack-skill-start:179, bin/gstack-skill-start:310). Review logging also independently enqueues records for synchronization (bin/gstack-review-log:99). Enforce unattended restrictions at the relevant writers and synchronization entrypoints, including inherited enabled settings and pending work from earlier sessions. Test actual subprocess/network-call absence, not merely absence of telemetry prompts.

11. [Severity: High] The installer has unresolved bootstrap and false-success paths. A1 moves component probes into TypeScript even though doctor deliberately works without Bun, and A2’s upgrade command cannot bootstrap a missing Bun executable (bin/gstack-doctor:22, setup:296). Checking only the security floor also allows Bun 1.3.14 when gstack’s supported minimum is 1.4.2 (bin/gstack-bun-version.sh:15). Preserve a dependency-free diagnostic/bootstrap path, distinguish the security floor from the supported runtime, and define missing-tool installation or refusal explicitly. An empty installed-host set and an unmet --revision request must not produce the same success certification as a satisfied installation.

12. [Severity: High] CI parsing and import closure cannot establish complete lane coverage. C8’s import graph misses subprocess targets, runtime file reads, generated inputs, and computed imports; doctor tests themselves exercise the binary through subprocesses and filesystem fixtures (test/gstack-doctor.test.ts:25, test/gstack-doctor.test.ts:107). Windows selection is computed by a planner, and serial classification belongs to runner code (.github/workflows/windows-free-tests.yml:62, scripts/test-free-shards.ts:235). Define supported workflow constructs and explicit dependency declarations, with conservative selection when mappings are uncertain. Persist remote-lane obligations and require their matching receipts at merge; printing requires-remote is insufficient enforcement.

13. [Severity: High] The pre-gate lacks a precise late-change invalidation rule. C8 runs before paid evaluations, but later fixes, stamping, regeneration, and integration can invalidate its findings and executed selections. Existing evidence captures fingerprints before and after execution specifically to reject changing inputs (bin/gstack-evidence:394). Give every pre-gate check an input identity and prerequisite set, then require current results after final generation and before publication. Define how timeout, unavailable tooling, and repository-requested warn downgrades affect acceptance; mandatory credential and required-lane checks must not silently become advisory.

14. [Severity: High] The ratchet assumptions are inaccurate, and --add-rationale weakens the enforced convention. The size test checks explicitly registered modules and residuals; it does not automatically cap every new library, and the three oversized files cited by the plan are absent from the current residual list (test/helpers/module-size.ts:202, test/fixtures/module-size-ratchet.json:51). Its documented exception is for data tables or generated literals, not growing interface logic (test/helpers/module-size.ts:225). Register every new owner module and intended residual limit, remove automatic rationale-based exemptions, and split artifact schemas, validation, storage, and orchestration into cohesive modules. Preserve both the 800-line module and 150-line function limits.

15. [Severity: High] Artifact and review-log changes require an explicit migration. B2 calls its task format the existing schema, but current tasks contain source_finding and local task IDs, without the proposed dependency, acceptance, and finding-reference fields (scripts/resolvers/tasks-section.ts:77). B7 also changes findings semantics while freshness still checks critical and legacy Codex counters, and binding consumes the start token before completing validation (lib/review-evidence.ts:188, lib/review-evidence.ts:238). Version the formats, enumerate all readers/writers, preserve conservative legacy behavior, and validate findings before consuming a token. Include resolved critical findings and rejected-write retries in migration tests.

16. [Severity: Medium] --next bypasses the existing version-source contract. C2 hard-codes origin/<base>:VERSION, although the repository already supports explicit version paths, pinned package manifests, and absent/ambiguous/broken outcomes (lib/version-source.ts:135). Read the configured version source from the selected base revision through the shared resolver. Make no-version repositories, monorepos, and three- versus four-component versions explicit supported cases rather than silently inventing a new allocation path.

17. [Severity: Medium] The proposed hermetic runner does not provide the isolation it advertises. buildHermeticEnv() retains HOME, supports a nonhermetic escape hatch, and accepts overrides after scrubbing (test/helpers/hermetic-env.ts:49, test/helpers/hermetic-env.ts:118, test/helpers/hermetic-env.ts:147). Wrapping it alone does not remove home dotfiles or Git identity. Extract a production-owned environment helper, allocate an isolated home, disable global/system Git configuration, and distinguish isolation failures from fixtures legitimately requiring a configured identity.

18. [Severity: Medium] Checklist reduction and the extra spec pass lack a trustworthy acceptance criterion. A 200-line cap and scope enum prove structure, not preservation of review obligations; B4 can silently exclude a required methodology section after a false-negative scope decision. B8 also adds a fourth launch after the existing explicit three-launch cap, then checks only previous issues, potentially missing regressions introduced by their fixes (scripts/resolvers/spec-review.ts:188). Maintain an obligation-to-section map with conservative scope fallback and behavioral evaluations for omitted requirements. Distinguish deterministic issue verification from another model review, preregister its budget, and never promote issue-only verification to an unchanged full-review verdict.

19. [Severity: Medium] P7 cannot measure adoption from the proposed events alone. Run and acknowledgment events cannot establish the denominator of the first ten Capy threads, especially threads that never invoke gstack, and ephemeral local analytics disappear unless a parent retains them. The state-root resolver selects a location; it does not establish durability (lib/state-root.ts:22). Define an owner-maintained thread cohort, deduplicate consumption by run/thread identity, and let the parent collect consent-compatible receipts durably. Assign a tier to every item—the current omissions make the adoption gate and funded scope ambiguous.

20. [Severity: Medium] Automatic issue closure exceeds the available evidence. C6 turns every issue mentioned in commit messages into Fixes #N, including context references, partial fixes, and explicitly deferred asks. E3 also treats CODEOWNERS as a maintainer list, although file ownership and repository maintenance authority are different concepts. Require explicit closure intent plus satisfaction of the linked issue’s remaining obligations, and use explicit maintainer configuration or repository permissions for contributor classification.

21. [Severity: Medium] The shared command contract is not yet internally consistent. B0’s universal terminal-line rule conflicts with commands whose primary output is schemas, parsed JSON, or doctor trailers; B7 assigns both exit 2 and exit 1 to status mismatch. C1 proposes commented examples in .json files and omits the queue-mode key that C2 reads, while B0 conflates structural validity with completed execution. Publish the schemas and command/status/exit-code matrix first, including gate_pending, partial answers, interrupted runs, and JSON-only errors. Keep wire-contract assertions exact while testing workflow prose behaviorally, as required by docs/test-value-bar.md:23.

22. [Severity: Medium] The delivery estimate and validation sequence omit substantial required work. PR B contains a runner, crash recovery, provider adapters, schema migration, gate parsing, and prompt reduction; PR C contains transactional mutation, queue coordination, evidence portability, and dependency analysis, so neither is supported by the stated small library counts. Validation also omits the test-code type ratchet that CI explicitly runs (.github/workflows/free-tests.yml:223). Re-estimate B and C by these components, move optional breadth behind explicit tiers, and add build, credential scanning, applicable formatting, and selected quality judges before the final acceptance sequence. Preserve the final frozen-tree free-suite run and report incomplete platform/provider coverage separately.

Architecture:

Parent / coordinator
    |
    +--> gstack-capy-install --> dependency-free bootstrap --> setup
    |                                  |
    |                                  v
    +--> gstack-doctor --> doctor-components + runtime-pins
    |
    +--> gstack-autoplan run/status/answer/resume
    |          |
    |          v
    |     autoplan-run [exclusive owner + durable transitions]
    |          +--> snapshot / prompt preparation
    |          +--> runner adapters ----> CLI / API / host-subagent
    |          |        |
    |          |        +--> shared spend ledger + result classifier
    |          +--> gate-list <---- reply + expected gate_rev
    |          |
    |          v
    |     Parent-selected run directory
    |       immutable inputs + reviewer outputs + attempt records
    |       plan.md / review-record.md
    |       tasks.jsonl / decisions.jsonl / findings.jsonl / timing.json
    |       run.json [atomic manifest]
    |          |
    |          +--> artifact schema registry + strict validator
    |          +--> separate consumption acknowledgments
    |
    +--> /ship or explicit --allow-repo-commands
               |
Committed base pin files + policy revision
  ship-policy.json / generated.json / pregate.json
               |
               v
       policy validation + authorization
               |
       +-------+------------------+
       |                          |
       v                          v
  restamp + regen              pregate registry
       |                          |
  isolated staged tree       selection manifests
  preimages / postimages     local + remote lane obligations
  restamp journal                |
       |                         v
       +--> final stamped tree --> gate execution
                                      |
                                      v
                            portable evidence bundle
                            [content + execution identity]
                                      |
                            tree-receipt eligibility
                                      |
                            ship-receipt in PR body
                                      |
                         serialized conditional merge

Persistent analytics / learnings:
  all callers --> state-root.ts / gstack-paths
              --> explicit durability and session policy

Test plan gaps:

• Paid completion followed by process death: add test/autoplan-run-recovery.test.ts, killing the runner before and after result persistence and asserting no automatic duplicate charge.
• Concurrent resume, answer, and ack: add test/autoplan-run-concurrency.test.ts, checking exclusive ownership, revision conflicts, and exactly-once acknowledgment accounting.
• Answer changes reviewed architecture: add test/autoplan-answer-invalidation.test.ts, proving only affected phases reopen and stale replies perform no writes.
• Structured output omits a reviewer finding: add test/headless-artifact-integrity.test.ts, including duplicate IDs, cycles, malformed tails, traversal, symlinks, and unknown schema versions.
• Imported evidence survives VM destruction: add test/evidence-bundle-portability.test.ts, verifying from a fresh clone without the original object database or logs.
• Version-only edits affect a gate: add test/tree-receipt-stamp-sensitive.test.ts, including version-dependent generated output and lockfile metadata versus dependency changes.
• Base moves after the final queue check: extend test/gstack-restamp-queue.test.ts with concurrent coordinators and a move between validation and merge.
• Release command modifies undeclared state: add test/restamp-command-boundary.test.ts, including dry-run side effects, rollback conflicts, and mirror partial completion.
• API/host-subagent claims completion without usable coverage: add test/outside-runner-contract.test.ts, exercising missing input, truncation, refusal, unknown family, and missing terminal evidence.
• Unattended inherits enabled synchronization: extend test/gstack-skill-start.test.ts with instrumented egress sinks and pending telemetry work.
• Missing Bun and unmet revision pin: extend test/gstack-doctor.test.ts and add test/gstack-capy-install.test.ts, covering zero registered hosts and unsupported bootstrap runtimes.
• Dependencies bypass static imports: add test/pregate-selection.test.ts, covering subprocess targets, computed reads, deleted/renamed files, and unsupported workflow expressions.
• Late generation invalidates preflight: add test/pregate-invalidation.test.ts, proving publication rejects stale checks and uncleared remote obligations.
• Concurrent spending and unknown charges: add test/spend-ledger.test.ts, including a worker dying after dispatch and before settlement.
• Legacy review rows and rejected writes: extend review-evidence tests with resolved critical findings, old counters, and reusable start tokens after validation rejection.
• Custom version sources: extend queue tests with package-manifest pins, monorepos, and absent/ambiguous version sources.
• Hermetic mode still sees operator configuration: add test/gstack-hermetic-run.test.ts with a planted home file and global Git identity.
• Reduced checklist misses an applicable obligation: add behavioral scope-negative controls and mutation cases; a section-count assertion is insufficient.
• Adoption denominator includes nonusers: add test/adoption-cohort.test.ts, covering zero-run threads, duplicate acknowledgments, and ephemeral worker loss.
• Issue references are mistaken for completed work: add test/ship-issue-intent.test.ts, covering references, partial fixes, cross-repository IDs, and deferred asks.

Failure modes:

Failure mode	Detection	Handling	Test
Provider finished; completion record missing	Dispatched attempt lacks terminal receipt	Mark execution unknown; reconcile	autoplan-run-recovery
Two processes resume one run	Run ownership/revision conflict	Refuse second writer	autoplan-run-concurrency
Reply targets an old gate	Expected revision mismatch	Reject without mutation	autoplan-answer-invalidation
Artifact aggregate drops findings	Reviewer-ID reconciliation	Mark incomplete	headless-artifact-integrity
Receipt references vanished Git objects	Bundle completeness check	Refuse reuse	evidence-bundle-portability
Version transform changes gate behavior	Gate-specific equivalence check	Run affected checks on stamped tree	tree-receipt-stamp-sensitive
Queue changes during merge	Conditional merge rejection	Recompute allocation and integration tree	gstack-restamp-queue
Release hook partially writes or mirror fails	Journal and output comparison	Resume or conflict-aware rollback	restamp-command-boundary
Pre-gate misses a dynamic dependency	Unsupported mapping/dependency declaration	Select conservative coverage	pregate-selection
Required remote lane never runs	Required-lane receipt reconciliation	Block publication	pregate-invalidation
Worker dies with reserved spending	Unsettled attempt record	Retain reservation/unknown charge	spend-ledger
Unattended session inherits sync consent	Session policy at egress entrypoint	Skip synchronization explicitly	gstack-skill-start
Requested installation revision differs	Resolved commit comparison	Report unmet pin without updating	gstack-capy-install
Rollback encounters subsequent user edits	Current bytes differ from postimage	Stop with conflict; preserve edits	restamp-command-boundary

Scores:

Dimension	Score
Architecture soundness	5/10
Edge-case coverage	4/10
Test plan	5/10
Right-sized diff	3/10
Reversibility	4/10
Maintainability under the repo’s ratchets	4/10

Recommendation: Revise before approval because gate reuse, unattended recovery, and transactional restamping lack enforceable contracts, while several proposed integrations bypass existing repository guarantees.