gstack/test/fixtures/multi-agent-wave/reference-run/ceo-outside.md

1. [Severity: High] One operating environment is being treated as a market. The 27 threads describe two related repositories, one coordinator pattern, and one cloud environment—not 27 independent customer validations. Thirteen threads stopping after installation does not establish whether installation, workflow relevance, latency, or preference for existing tooling caused abandonment. Before funding B–D, test a minimal headless workflow across unrelated repositories and measure repeat usage, operator interventions, completion time, and cost against their existing workflows.

2. [Severity: Critical] “A tree that passed never gets another gate” is an invalid release promise. B3 treats equality modulo stamp paths as sufficient to refuse another gate, although runtime, dependencies, test selection, model configuration, and external state can change without changing that tree. Existing reuse explicitly requires complete input and environment identity, trusted receipt provenance, and bounded age; unknown inputs disable reuse (scripts/eval-input-cache.ts:4, scripts/eval-input-cache.ts:63). Stamp-path globs also do not prove that every change within those files is behaviorally irrelevant. Replace P3 with per-check reuse under the existing identity contract, retaining release-specific freshness requirements and checks of the final stamped artifacts.

3. [Severity: High] Machine-readable output is being confused with independently verifiable work. Six files and derived counters can make an incomplete review internally consistent: an omitted finding disappears from both the findings file and its totals. The snapshot implementation explicitly distinguishes input identity from evidence that a reviewer consumed it, while plan-review records bypass the stronger diff-review binding path (bin/gstack-autoplan-snapshot.ts:659, lib/review-evidence.ts:183). A4 lacks a versioned run manifest binding required phases, reviewer outcomes, decisions, input hashes, and artifact completeness. Make that manifest and a deterministic completion validator the primary deliverable; test interrupted runs, missing reviewers, stale artifacts, and omitted findings before promising that parents need only jq.

4. [Severity: High] Session mode does not solve host compatibility. D2 dismisses a host adapter by examining question capabilities, but host identity already determines which outside reviewer executes: Codex renders select Claude Code, while other renders select Codex (scripts/resolvers/outside-voice.ts:8). Consequently, the proposed Codex login helper and GSTACK_CODEX_MODEL override cannot supply outside coverage for every installed profile. The existing native dispatch also requires a child’s native file-reading tool, which a session-kind flag cannot create (bin/gstack-autoplan-snapshot.ts:661). Define host capabilities, reviewer transport, and approval policy separately, then prove the exact Capy dispatch path before rejecting an adapter.

5. [Severity: High] The urgent repair is hostage to twelve initiatives. PR A combines installation fixes with artifact export, checklist extraction, interactive decision redesign, score-loop changes, timing prediction, and reviewer reconciliation. That contradicts the stated reason for splitting the wave—landing P0 earlier—and makes the 1.5-day estimate unsupported by any demonstrated throughput or validation budget. Ship readiness reporting independently, then one headless review-to-pending-gate flow; require adoption evidence before adding the remaining A features or starting B–D.

6. [Severity: High] The queue design assumes version churn must be managed rather than eliminated. B1–B7 build a release coordinator spanning predecessor merges, version allocation, historical text rewriting, generated files, and a second repository. The existing allocator already solves collision avoidance by advancing beyond claimed versions; that is different from stamping the next item against main at merge time (bin/gstack-next-version:92). The plan does not compare this machinery with assigning versions after merge, separating release metadata from feature PRs, or delegating stamping entirely to the repository’s release tool. Prototype those alternatives first, and remove generic historical-text rewriting and cross-repository mutation unless their necessity and recovery semantics are demonstrated.

7. [Severity: High] The pre-gate can prevent the gate needed to satisfy it. B8 blocks remote gates until relevant platform lanes have run, while the exclusions explicitly leave Windows execution to CI; a Linux machine with a Windows-affecting change therefore has no specified forward path. Its proposed dependency foundation is also narrower than claimed: paidTestClosure follows test helpers and fixtures and explicitly calls its result a lower bound (test/helpers/touchfile-closure.ts:1, test/helpers/touchfile-closure.ts:25). Running affected tests plus hermetic reruns is not substantiated as a two-minute operation. Separate local deterministic preflight, remote coverage execution, and final evidence acceptance; scope blockers to prerequisites that can actually exist at that stage.

8. [Severity: High] Repository policy files acquire authority the plan never defines. B1, B2, and B4 introduce executable commands and cross-repository writes under a workflow advertised as unattended, but containment of a JSON file does not establish trust in its commands or mirror target. The existing version writer deliberately requires opt-in before executing a target repository’s generator, specifically because file presence must not silently authorize code execution (bin/gstack-version-bump:374). Define which trusted actor authorizes command execution, allowed write roots, and policy changes, preserving existing host permissions. Without that boundary, “headless never approves” is a prose promise sitting above tools with unspecified authority.

9. [Severity: High] /eval-plan is a second product with an overstated budget guarantee. D1 expands the wave into experimental design, scorer calibration, financial admission, data isolation, remote persistence, and long-running workload supervision without establishing demand for that product. The proposed source machinery explicitly uses estimated admission costs reconciled after execution, and its pricing helper excludes several billing categories (scripts/ship-measure.ts:22, test/helpers/pricing.ts:4). Extracting those functions does not establish a hard cap across independent workers or machines. Defer D1; if pursued separately, distinguish estimated admission limits from enforceable spending limits and validate concurrent reservations, unknown charges, crash recovery, and held-out-data isolation.

10. [Severity: High] Review compliance is replacing evidence of review quality. C1 makes more rows mandatory, A6 removes methodology based on detected scope, and validation mainly asks whether known incidents produce expected artifacts; none establishes whether the resulting reviews catch unfamiliar failures at an acceptable false-positive rate. A8 additionally assumes a 7/10 stopping criterion that the cited implementation does not use: it stops on PASS, repeated issues, or three launches, and explicitly says a missing score does not require another review (scripts/resolvers/spec-review.ts:182). “Rerun until 7+” risks selecting a favorable reviewer result instead of confirming repairs. Compare current and condensed workflows on a fixed, held-out incident panel, and gate on unresolved consequential findings rather than row presence or a resampled overall score.

11. [Severity: Medium] The install verdict still lacks a definition of intended readiness. A1 defaults requirements to installed hosts, which cannot reveal a missing intended host unless the caller already knows to request it; checking a patch heading establishes presence, not behavior. Runtime discovery also mixes compatibility requirements with CI selections: this repository declares Bun >=1.4.2, while its runtime helper separately distinguishes a security floor and supported minimum (package.json:80, bin/gstack-bun-version.sh:10). Make readiness explicitly relative to a requested workflow, host, and capability set, distinguishing gstack’s runtime from the target project’s requirements. Test wrong-host, missing-authentication, and incompatible-runtime cases, not just missing files.

12. [Severity: High] Default contributor rewriting undermines the maintenance model. D3’s prescribed output offers supersede or decline, omitting acceptance of a correct external implementation; D10 makes this treatment the default based on authorship. Reimplementing working contributions adds maintainer load precisely while this wave creates more tools, schemas, and platform obligations to maintain. Evaluate contribution behavior and evidence under the same acceptance criteria as maintainer changes, preserve a direct accept path, and reserve rewriting for demonstrated architectural or quality problems.

Premise verdicts: The 27-thread sample establishes broad demand for this wave — UNVERIFIED; supplied context lacks independent adoption evidence or a causal comparison. Doctor currently lacks --check and treats below-tested Bun as a warning — VALID; bin/gstack-doctor:34, bin/gstack-doctor:138. setup --status exits successfully after reporting installation checks — VALID; setup:136. Headless question failure blocks, and onboarding suppression currently targets spawned sessions — VALID; scripts/resolvers/preamble/generate-ask-user-format.ts:29, bin/gstack-skill-start:394. Raw plans need no literal implementation heading — VALID; normalization wraps raw input, subject to boundary validation (bin/gstack-autoplan-snapshot.ts:409). Review-log counters are not derived from findings — VALID; supplied records are bound and stamped, although diff freshness already rejects nonzero issue counts (lib/review-evidence.ts:178, lib/review-evidence.ts:238). Codex authentication alone supplies every host’s outside reviewer — FALSE; Codex-host reviews select Claude Code (scripts/resolvers/outside-voice.ts:8). The existing spec-review loop requires a score of at least seven — FALSE; its stopping rules use PASS, repeated findings, and launch count (scripts/resolvers/spec-review.ts:188). The existing touchfile closure provides general source-to-test dependency coverage — FALSE; its traversal is restricted to helpers and fixtures and is explicitly incomplete (test/helpers/touchfile-closure.ts:1). Tree equality alone establishes reusable gate evidence — FALSE; current reuse requires complete consumed inputs, runtime identity, provenance, and freshness (scripts/eval-input-cache.ts:4). Extracting ship-measure admission logic establishes a hard spending cap — FALSE; admission uses estimates and reconciles actual costs afterward (scripts/ship-measure.ts:22). The full pre-gate takes two minutes and PR A takes 1.5 days — UNVERIFIED; no supporting measurements are supplied.

Scores: premises valid: 5/10
right problem: 6/10
scope calibration: 2/10
alternatives explored: 2/10
risks covered: 3/10
6-month trajectory: 3/10

Recommendation: Re-scope to capability-specific readiness and one verifiable headless review flow, deferring B–D because broad adoption demand is unproven and the proposed gate-reuse guarantee is unsound.