mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-06 00:44:07 +01:00
capy/gsta-75-diff-scope-python
1
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Garry Tanandcapy-ai[bot]
|
77d125e0bc |
v1.91.127.0 fix(evals): paid-gate reliability round 2: docsync candidate protocol, scope-block detection, qa fix-loop budgets, lettered consent (#3260)
* test(qa-callers): bind QA section names in the caller prompt instead of teaching helper forms (W3-4)
The caller session prompt taught full capture/checkpoint syntax but only
mentioned materialize, so two ship-exploratory-small-cli sessions (paid
gate runs 38973329985, 38993076767) skipped qa/sections/exploratory.md and
system-functional.md and guessed a bare materialize. The prompt now
declares only fixture bindings (DEADLINE_TOOL, EVIDENCE_TOOL, PROBE_DIR,
DEADLINE_FILE, --public approval, --timeout-ms 10000 for plan probes) and
points to the section for every form. The read requirement is unchanged;
both captured transcripts are kept as negative controls.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(review-army): run the state-machine detector in the scope block and check its exit status (W3-5)
Specialist 9 named gstack-state-matrix detect only in selection prose, and
small diffs keep only State Machines, so agents ran an undeclared command
(paid gate run 39002269281). The generated scope block now runs the
detector with its exit status checked: on failure it keeps the detector's
diagnostics and sets SCOPE_STATE_MACHINE=unknown, and selection dispatches
State Machines when the signal is true or unknown. The check step becomes a
declared heredoc block. The clarity test requires every non-diff-scope
signal to come from a detect helper the scope block runs; a free test runs
the block against a failing detector stub; the captured standalone call
stays outside the caller interface.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(third-party-actions): letter the absent-Aside consent options B/C/D (W3-7)
The contract fixed letters only when Aside is detected, so a tpa-absent-linux
agent numbered its consent question 1/2/3 (paid run 39011101113), which the
lettered-offer inspection cannot read. The absent branch now offers
B) gstack's own visible browser, C) manual instructions, D) defer and omits
A). The paid test's assertion is unchanged; the captured numbered answer is
a documented fail example in the unit test.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(plan-reality): say tier 1 rows are never n/a (W3-8)
The plan-review reality block offered pass|finding|n/a without saying tier 1
rows may never be n/a, so a plan-mode-no-op session wrote
`REALITY: deferred-asks n/a ...`, check returned repair, and the repair loop
ran past the 300 s capture (paid run 39004466422). The block now says tier
1 rows are never n/a and that with no linked issue deferred-asks is pass
with a receipt showing none. The captured line is pinned through check as
missing=deferred-asks verdict=repair.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(qa-functional): webhook fix runs re-verify the repaired invariant and rerun superseded captures (W3-6, W3-14)
W3-6: stage 2 of the fix-run prompt asked only for the original, happy,
cancel and dependency probes while verifyQANativeRegression reruns all
eight, so a status-only repair that left partial broken reached the verifier
(paid run 38986120559). The webhook fix prompt now states the capture budget
up front (at most 5 pre-repair captures, then the 7 post-repair scenarios:
duplicate, partial, concurrent-ab, concurrent-ba, happy, cancel, dependency;
12 in total; report incomplete if 5 cannot establish the defect). That list
feeds the capture nudge (GSTACK_QA_REQUIRED_PROBES);
QA_WEBHOOK_REQUIRED_SCENARIOS and the verdict are unchanged.
W3-14: the fix prompt now says to rerun every pre-repair capture the
verdict relies on after the final repair, and that a superseded row stays
open until its command reran on the repaired source (paid run 39053431488
labelled a pre-repair duplicate superseded and never reran it). The
detector and its negative test are unchanged; the captured shape joins them
as a negative control.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(qa): tie the exploratory stop rule to per-rule input probes; fix runs close superseded captures (W3-9, W3-14)
W3-9: two qa-functional-cli-report sessions stopped at 5-7 of 12 probes
without probing a non-digit amount (paid runs 38983306684, 39001107574),
though system-functional.md requires one probe per documented input rule.
The single stop rule shared by both exploratory templates now says each
documented input rule needs its own probe breaking only that rule, and
rules left uncovered at a probe or time cap go in limits.
W3-14: for fix runs (not qa-only), every capture taken before a fix is
superseded even when its client prints no snapshot; before completion the
agent reruns each one its verdict relies on or reports it as an open
remainder. Production superseded/open semantics are unchanged.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(qa-functional): keep the captured webhook-report cat attempts as denied negative controls (Q3)
Paid run 39006896632 (qa-functional-webhook-report) attempted
`cat package.json probe.ts .gitignore` and `cat src/*.ts test/*.ts`; the
PreToolUse hook denied both. This is a model miss, not a detector defect, so
both commands are pinned as denied by the registered hook and outside
qaCommandAllowed. No detector change.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(docs-candidate): per-subcommand --help and a compare reject that names the fix (W3-2)
snapshot --help and compare --help print the usage and exit 0 before any side
effect; the docsync observer admits exactly those two forms. compare names the
record path, each missing or wrong field (schema_version, base, hashes) and
the snapshot command that writes a valid record, keeping exit 1. The captured
ship-docsync-stale-after commands from PR gate run 38990111230 replay inside
the interface; help with any other argument stays rejected.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(docsync-fault): prepare writes the production candidate through the installed snapshot (W3-1)
The fault actor prepare wrote its own {base_sha, content_hashes, ...} record,
which the installed gstack-docs-candidate compare rejects, while
ship/sections/documentation.md requires running compare; 79 of 198 docsync
captures (and the red in PR gate run 38990111230) were pushed off the
documented path by it.
prepare now runs the installed snapshot under a 30 s timeout for the same
selection, returns its exact command as snapshot_command so the post-child
rerun is literal, publishes prompt and event only after a valid record
exists, and removes a record its own failed snapshot left. dispatch observes
the candidate with the same installed snapshot; inspect returns base, index
and per-file git blob ids in the record representation; the legacy re-audit
check reads selected/hashes. The fault prompt names snapshot_command and the
new field names.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* docs(todos): file the qa-quick capture headroom follow-up (fix wave 3, T)
qa-quick timed out once at 300 s (paid run 38980005793) against passing
runs of 164-272 s. Trimming what the Quick tier loads is a product call, so
it is filed, not repaired; no budget change.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(docsync): keep the two docsync model misses as negative controls (W3-3)
The Bash grep of ship-docsync-missing-asset (PR gate run 38983306684,
bash#13) and the composed cat; cat of ship-docsync-failure (38984977679,
bash#12) were correct rejections. They join the stored known_bad commands
(with a {home} placeholder for the fixture home); no detector change.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(golden): refresh the ship goldens for the W3-5 scope block and W3-7 consent letters
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(third-party-actions): state the absent-Aside letters in fewer characters (W3-7)
Same contract (B), C), D) when Aside is not detected; the next sentence
already omits the Aside option), 67 fewer characters on the pinned
land-and-deploy and office-hours skills.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(skill-md-size): re-pin 31 over-limit skills grown by fix wave 3 prompt repairs (owner sign-off needed)
CREW rule 11: these skills were already over 100,000 characters and grow
by the minimum text each repair needs; none crosses the limit for the
first time.
- gstack-review on cursor, factory, kiro, opencode, slate (+657..+676):
W3-5 runs gstack-state-matrix detect inside the scope block with its exit
status checked and declares the check step as a heredoc block; the new
fence carries the env-var-host runtime prelude (about 360 characters).
- gstack-plan-eng-review on 12 hosts (+104): W3-8 says tier 1 reality rows
are never n/a (paid run 39004466422 looped on a rejected n/a row).
- gstack-land-and-deploy on 12 hosts and gstack-office-hours on 4 hosts
(+21): W3-7 letters the absent-Aside consent options B), C), D).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(skills): fit the W3-5, W3-7 and W3-8 prompt repairs inside the parity budgets
The parity suite left 2-42 bytes of headroom on ship, review,
plan-eng-review, plan-ceo-review and land-and-deploy, so each repair now
pays for itself in the same resolver:
- W3-5: the scope block runs `source <(gstack-state-matrix detect --base
<base> || echo SCOPE_STATE_MACHINE=unknown)` (a failed detector's
stdout is overridden by the later unknown assignment; its stderr still
shows), the check block uses a short heredoc, and DIFF_LINES comes from one
`git diff --numstat | awk` sum instead of two --stat greps (same count:
insertions plus deletions, binary files zero).
- W3-7: the absent branch reads "offer only options B), C) and D)."
- W3-8: "never n/a" joins the tier 1 parenthesis, the no-linked-issue rule
joins the receipt sentence, and the deep header says the same in fewer
words; the redundant "(it prints the PLAN_REALITY: line)" is dropped.
Size pins: land-and-deploy (12 hosts, -20), office-hours (4 hosts, -20)
and plan-eng-review (12 hosts, -10) are lowered to their new sizes;
gstack-review on cursor, factory, kiro, opencode and slate stays above its
main pin (+355..+374) because the declared check block is a new fence that
carries the env-var-host runtime prelude. Raising those five pins needs
owner sign-off.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(design-e2e): the shim-dom scan check resolves the command's literal _REPORT/_RUN (W3-10)
PR gate run 38980005793 went red because the agent ran the generated DOM
block, scanning "$_REPORT/dom/$_RUN" with _RUN="run1" assigned in the same
command, and the check looked for the literal dom/run1 anywhere in a command.
domScanOfRun substitutes that command's own literal _REPORT=/_RUN=
assignments and credits a scan only when one of its own arguments is the run
directory. scan --changed, another directory, quoted heredoc report text and
non-literal or conflicting assignments stay negative controls. All 15
captured shim-dom runs in the window pass the new check (14 passed the old).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(pty): keep the last non-blank viewport for failure dumps (W3-11)
autoplan-guard-pty (PR gate run 38976253774) stalled at 489 s and its screen
dump, the tail of visibleText(), was only spinner redraws. Live PTY sessions
now remember the last non-blank viewport a screen read returned
(session.lastScreen()) and the harness wait-timeout dumps append it.
Diagnostic only: visibleText(), matching and verdicts are unchanged.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(autoplan): explicit 30 s timeout for each answer-invalidation test (W3-12)
The eng-only override test ran 4.1-4.4 s on a 4-core machine against Bun's
5 s default (7.8-9.4 s per file). Each test now names a 30 s budget.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(review-army): size the nested bun test budgets for windows-latest (W3-15)
windows-free-tests run 39053431430 (shard 5): the nested bun test exited 1 in
5.58 s because Bun's default 5 s hook timeout fired on the real Red Team
beforeAll (8 git spawns plus skill-section copies). Both nested argvs pass
--timeout 30000, which raises only the nested default; the registrations
under test keep their explicit timeouts. Spawn timeouts move from 8 s to
40 s and outer test timeouts to 45 s, including the consensus test that had
none. With a 6 s beforeAll injected, the lifecycle still matches; the old
argv reproduces the Windows hook timeout.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* docs(todos): close the createQAFunctionalFixture entry and file D6 (W3-13)
The git auto-maintenance entry was fixed by
|