gstack/test/fixtures/multi-agent-wave/reference-run/dx-outside.md

1. [Severity: Critical] Gate reuse can certify changed behavior. C3 excludes entire stamp and generated-file paths, allowing a changed generated program or prompt to qualify as same-modulo-stamps; the existing evidence implementation likewise compares allowed filenames, not permitted byte changes (bin/gstack-evidence:505). Conversely, requiring identical raw lockfile hashes rejects ordinary npm restamps because the version writer updates lockfile version metadata (bin/gstack-version-bump:486). Define narrowly permitted version transformations, retain all other generated bytes in the comparison, and capture actual runtime/container identity and executed test selection alongside declared pins. Test changed generated behavior, metadata-only lockfile changes, and dependency changes separately.

2. [Severity: High] The fresh-machine installer fails before its repair step. A2 upgrades Bun after setup and doctor, but setup exits immediately when Bun is absent or below its floor (setup:296). Skipping Chromium also leaves the dependency installation and full build on the cold path (setup:1808), so planning-only TTHW remains unmeasured. Preflight and provision the runtime before setup, explicitly check jq, and measure a clean planning installation separately from browser bootstrap. Add a requested revision and an explicit stale-checkout remedy: “reuse, never pull” otherwise gives different subagents different command contracts.

3. [Severity: High] Readiness requirements contradict the supported outside runners. A1 requires both host profiles and the Astra patch for planning, while B3 promises a complete workflow using an API or another-family host subagent without Codex CLI. A1 also makes the native helper required for --for cso, while B2 describes its absence as optional SKIP. Define readiness against the selected host, runner, model and workflow capabilities, with required, optional and not_applicable represented explicitly. Test a healthy Capy-plus-Claude-subagent installation with no Codex installation and require an honest workflow-specific verdict.

4. [Severity: High] Runtime-pin checks can repair the wrong environment. A2 enters the gstack checkout for setup without specifying how doctor identifies the target project, and A1 combines project constraints with every CI runtime declaration. Those declarations are platform-specific: the Windows workflow pins Bun and Node separately (windows-free-tests.yml:83), so treating them as universal local requirements can block a Linux planning run. Require explicit --project and workflow/lane selection, show each effective constraint’s source, and distinguish compatibility ranges from tested configurations. Resolve conflicts before changing Bun, and separate gstack’s runtime from a target project’s incompatible runtime.

5. [Severity: High] There is no complete unattended invocation or continuation contract. B2–B4 introduce export, prompt printing and flags such as --outside-model, but never identify one executable that owns launch, phase dispatch, termination and resumption. The current snapshot CLI exposes preparation operations, not an unattended runner (bin/gstack-autoplan-snapshot.ts:830), and package binaries expose only browse and make-pdf (package.json:7). Specify a runnable parent-agent sequence with explicit project, input, output directory, runner, deadline and spend limit, plus status, decision application and resume operations. Acceptance must include answering a pending gate and resuming without repeating completed paid reviews.

6. [Severity: High] The promised task file cannot replace reading the plan. B2 preserves today’s task schema, which contains titles, files and free-text source findings but omits verification commands, dependencies and acceptance criteria—even though verification appears in the human task format (tasks-section.ts:42, tasks-section.ts:77). Its IDs are phase-local T1, T2, while B2’s finding fields do not explicitly include the finding IDs later required by B12. Define globally unique run-bound IDs, task dependencies, acceptance checks, finding references and blocking decision references. Validate referential integrity and bind each reviewer’s immutable result to the manifest; matching a model-written count cannot establish that findings were not omitted.

7. [Severity: High] “Machine-readable first” has no common transport contract. A1 mixes JSON rows with an always-required text trailer, B2 prints whitespace-delimited paths, C5 provides a Markdown fence without specifying its payload encoding, and C3 supplies text verdicts without a JSON interface. A parent must invent separate parsers, and paths containing spaces immediately break naïve token extraction. Require a shared result envelope and exit-code table, JSON-only stdout under --json, diagnostics on stderr, explicit gate_pending versus incomplete states, and machine-readable recovery actions. Give every new command—including gate, restamp, regen, pregate, receipt, lane-check, hermetic-run and banned-terms tools—discoverable --help, argument ownership and a tested jq example.

8. [Severity: High] Durable state does not yet survive the parent–child boundary. A3 says the installer “sets” GSTACK_STATE_ROOT, but a child shell’s export cannot configure later parent-launched commands; the resolver only consults the environment supplied to each process (lib/state-root.ts:22). B2 additionally writes deliverables beside ACTIVE_PLAN, which may remain on ephemeral storage despite a durable state root. Return a machine-readable launch configuration that the parent explicitly supplies, require a durable run output directory, and report durability separately for each artifact class. Add an idempotent consumption-acknowledgment command tied to the manifest hash; otherwise P7’s adoption threshold depends on undocumented reporting.

9. [Severity: High] Queue reordering lacks an authoritative version-selection rule. C2 delegates --next to an allocator that deliberately advances beyond the highest version claimed by any queued PR (bin/gstack-next-version:97), conflicting with “main’s version plus one when this PR is next.” Checking a predecessor’s head does not establish that it remains the predecessor after reordering, nor that the base stayed unchanged during stamping and gating. Introduce an explicit stamp-at-merge allocation mode with expected base SHA, queue order, predecessor identity and requested-version semantics. Revalidate those expectations immediately before merge and return a structured stale-queue result with the exact recomputation command.

10. [Severity: High] Cross-machine gate reuse arrives before portable evidence. C3 promises reuse of gates run on ephemeral machines, but portable bundles appear only in E2. Existing evidence is explicitly machine-local, and checking every recorded label does not prove every required lane ran (bin/gstack-evidence:27); C5’s aggregate counts do not close that gap. Move the minimum portable evidence package into C, including named required lanes, executed selections, source run URLs, outcomes and artifact hashes. Verify it from a fresh coordinator machine after deleting the gate VM, without rerunning the gate.

11. [Severity: High] Restamp has incompatible check/apply semantics and no recovery transaction. C2 requires generators to produce changed stamp outputs, while C4 immediately uses git diff --exit-code to classify changed generated outputs as stale. The underlying version writer already has partial-write failure states (bin/gstack-version-bump:489); adding release hooks, changelog rewrites and a mirror creates more failure boundaries than “same version is a no-op” addresses. Separate regen --check from regen --write, define one authoritative release-tool interface, and journal each restamp’s intended inputs and completed writes. Provide resume/rollback behavior that preserves unrelated edits, with injected failures after every stage.

12. [Severity: High] Base-only pin files have no first-use workflow. C1 reads executable policy from the base branch, so a PR introducing .gstack/generated.json or its first release command cannot use that declaration to validate itself—even with execution permission. Moreover, a base-declared bun run release:restamp still executes the PR’s package script, so pinning the command string does not freeze its implementation; the existing version writer explicitly documents execution of target-repository code (bin/gstack-version-bump:374). Add policy initialization, validation and effective-configuration inspection, showing source revision, command, working directory and trust scope. Specify an explicit reviewed bootstrap path for new policies and test standalone, ship-invoked and changed-policy behavior.

13. [Severity: High] The two-minute pre-gate assumes CI information that YAML does not contain. C8 promises lane discovery from workflows, but this repository computes its Windows matrix by executing a planner (windows-free-tests.yml:62) and defines exclusive test classification in TypeScript (scripts/test-free-shards.ts:243). Running all locally affected tests can also exceed two minutes; printing an overrun does not preserve the promised fast feedback. Separate bounded mechanical preflight from required test execution, consume runner-owned selection manifests, and expose --explain for selection provenance and uncertainty. Remote requirements must name the workflow/job to dispatch and the receipt needed to clear them.

14. [Severity: High] The owner gate can conceal decisions or misinterpret approval. E3 caps the brief at seven decisions, while this plan presents twelve, and provides no overflow behavior; it also promises defaults for unanswered items despite B1’s pending approval semantics and UC1’s instruction that the owner’s direction stands. B9 leaves all except d9b ambiguous between exclusion and override, while bare yes/no cannot identify a choice across multiple open or multi-option decisions. Preserve every stable decision ID through pagination, explicitly distinguish automatic choices from required approvals, and bind replies to a gate revision. Define the grammar with expected resolved JSON, reject ambiguous aliases, and keep unanswered approvals pending.

15. [Severity: Medium] Findings-derived status will collide with existing freshness and error handling. B7 derives both clean and issues_found, but the current freshness reader rejects a review whenever issues_found > 0, even if its status is clean (lib/review-evidence.ts:238). Its proposed detailed mismatch error would also be swallowed by the shell wrapper, which suppresses stderr and converts binding failures into “invalid JSON” with exit 1 (bin/gstack-review-log:88). Define separate total/open/resolved counts and update both reader and writer together. Preserve typed errors through the wrapper and test a review with many findings, all resolved.

16. [Severity: Medium] Tiering does not define what each merge gate requires. C2 marks mirror and gate-ahead support tier 2, but Validation 3 requires both in PR C’s queue replay; delivery also alternates between moving tier-2 work to “the next PR” and a separate follow-up. E2 and E3 lack explicit tiers despite the assertion that every item has one. Publish a feature-to-PR-to-tier acceptance matrix, including commands, artifact capabilities and deferred asks. The owner’s gate should approve that concrete matrix, and each installed revision should expose which capabilities actually shipped.

Persona traces:

• Parent Capy agent: (1) Bootstrap stops before Bun repair or performs an unexpected full build (#2); (2) readiness requires irrelevant hosts or runtime pins (#3–4); (3) the parent invents launch and continuation glue (#5); (4) it resolves artifact references, storage and parser differences before acknowledging consumption (#6–8). Estimated TTHW: 45–90 minutes with manual integration to obtain and consume the first valid gate_pending bundle; unattended completion has no bounded path as specified.
• Queue coordinator, three PRs reordered twice: (1) Establish effective policy (#12); (2) choose the next PR and reconcile requested versus allocated version (#9); (3) restamp and recover generator/mirror failures (#11); (4) retrieve gate evidence from the old machine and establish reuse after each reorder (#1, #10). Estimated TTHW: 30–60 minutes of orchestration plus one gate’s runtime, with roughly 15–30 additional minutes per reorder and possible full reruns when evidence is unavailable. These are review estimates, not measured runs.

Scores:

Dimension	Score
Onboarding/TTHW	3/10
Discoverability	3/10
Error messages	5/10
Consistency of CLI/flags	3/10
Machine-readability of outputs	4/10
Docs/examples	3/10
Owner gate experience	3/10
Recoverability after failure	2/10

Recommendation: Revise before implementation because neither persona has a complete executable path through installation, approval, reordering and recovery, and gate reuse can incorrectly certify changed generated behavior.