mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-06 08:54:04 +01:00
capy/gsta-75-diff-scope-python
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Garry Tanandcapy-ai[bot]
|
fc046b5c83 |
v1.91.168.0 fix(evals): QA deadline joins, QA-callers handoff order and plan-check probes (GSTA-57) (#3321)
* fix(evals): QA deadline joins, QA-callers handoff order and plan-check probes (GSTA-57) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(fixtures): register the QA-callers handoff-order capture as a free fixture Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * v1.91.168.0 stamp Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> |
||
|
Garry Tanandcapy-ai[bot]
|
a09b3b8532 |
v1.91.145.0 fix(evals): qa-only opts in to the read-only grammar; stderr-to-null forms accepted (#3295)
* fix(evals): qa-only opts in to the read-only grammar; stderr-to-null forms accepted Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(evals): qa-only checkpoint rule counts guarded "$B" stop as browser cleanup, like closetab PR #3272's Opus gate 39217387697 (slice 8) failed qa-only-no-fix when the agent stopped the browser inside the guard; the rule counted stop as a third probe that needed a checkpoint. The prompt now names both cleanup children. A free replay of the captured three guarded calls passes (and fails on the old rule); stop before a probe, a disguised probe, stop --all and stop outside the guard still fail. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(evals): register the qa-only stop-cleanup replay fixture in qa-only-no-fix touchfiles Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(evals): docs-sync observer ignores an unexecuted candidate call that exited 127 at a missing path PR #3295's gate (run 39223031058, slice 2) failed ship-docsync-current after Opus called gstack-docs-candidate compare at a path missing the fixture-home segment, got exit 127 (no such file or directory), and reran it at the declared path. The observer now ignores such a call only when the absolute binary path does not exist, the recorded result is exactly the shell's exit-127 not-found message for that path, and a later call runs the same command at the declared path without failing. A free replay of all 43 captured tool calls passes and fails on the previous observer; negative controls cover an existing wrong-path binary, other exits and messages, a missing/earlier/failing/changed retry, and non-candidate or relative binaries. GStack Minder-approved. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * v1.91.145.0 chore(release): stamp qa-only read-only opt-in, stderr forms, stop cleanup and docs-sync exit-127 exception Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> |
||
|
Garry Tanandcapy-ai[bot]
|
3766edac8d |
v1.91.144.0 feat(evals): Opus 5.5 is the paid gate's model under test; judge stays Fable (GSTA-49 TM1) (#3275)
* feat(evals): Opus 5.5 is the paid gate's model under test; judge stays Fable (GSTA-49 TM1) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * v1.91.140.0 chore: stamp TM1 (Opus 5.5 model under test) and CHANGELOG Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(evals): repair the five Opus gate reds from run 39161502626 - coverage audit: owned cd; followed by one && chain, wc -l on in-cwd files - shared-libs: Claude Code 2.1.292 git --version probe, exact argv only - ship hook: accept 'integrate the hook yourself' / 'combine ... by hand' - qa-only: deadline starts only after the charter Write completes - plan floor: tool-only parent session identifies the owned Create grant; floor runner fails fast with PERMISSION_DIALOG_UNANSWERED Each repair replays its captured run as a free regression with negative controls. HARNESS_VERSION stays 6 (re-recorded from main's 5). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(evals): ship hook manual-integration sentence rule; qa-only inspection uses Read/Glob; tool-only permission tests join their owner file Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(evals): tpa-absent-linux accepts instructions the user follows themselves as the manual option Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(evals): owned plan Edit menu binds to exactly one pending Edit, including unpublished ones Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(evals): shared-libs read evidence counts a literal path loop whose output names each file Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * chore: restamp to v1.91.142.0 (queue slot moved) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(evals): investigate boundary fixture identifies the owner question by its continue/abort options Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(evals): plan-design-with-ui focus check accepts the skill's design-completeness rating in the question body Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(evals): phrasing sweep across the 108 gate rule cases; replay captured Opus paraphrases with negative controls Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(evals): command-detector sweep — exact-count anchor, label slot, 2.1.292 worktree probe - coverage audit (gate 39207508651 plan-eng-coverage-audit): output of exactly known length (wc -l over literal in-cwd files) anchors the next owned read at that line; a label's slot counts from the end and a later complete copy voids the credit (review/specialists/testing.md prints '---') - shared-libs-unsupported-git (39161502626): Claude Code 2.1.292's askPass-first 'worktree list --porcelain -z' cwd probe is Claude's own; the read-only Git check is shared (repoGitRequests, isReadOnlyRepoGitRequest) and the free regression replays every Git request of that capture Each fix replays its captured command with negative controls. No threshold, trial count, case kind or judge changed. HARNESS_VERSION stays 6. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> |
||
|
Garry Tanandcapy-ai[bot]
|
f592d67826 |
v1.91.111.0 fix(evals): four PR-gate background reds were a detector, an interface, a helper and a workflow-block defect (#3231)
* fix(evals): three PR-gate background reds were a detector, an interface and a helper defect - docsync-fault-eval: a conditional mention (until/unless/once/when/before/if) of 'Documentation is current' is not a false current claim; replay corpus for ship-docsync-launch-failure from PR run 38336400566 with derived refusals. - gstack-skill-start/gstack-skill-end: --help prints usage and exits before any side effect (it used to start a session); the docsync interface admits the lifecycle helpers' own --help; captured run 38969924945 replays clean. - qa deadline policy: listing the owned qa-reports/screenshots directory is bookkeeping (qa-only-no-fix, run 38969924945). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(evals): the launch-failure corpus is a row in the docsync corpus test that owns docsync-fault-eval Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(evals): register the launch-failure corpus and docsync-store capture in their contracts The B1 corpus contract wants its own detector-corpus-<case> replay test, so the launch-failure corpus keeps one, listed in the test-of-test baseline beside its sibling corpora; FREE_FIXTURES names both new fixtures' free consumers. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(review-army): the scope block prints the SCOPE_* signals specialist selection reads The Detect stack and scope block sourced gstack-diff-scope silently, so an agent that needed SCOPE_FRONTEND to pick specialists appended its own echo and left the declared caller interface (ship-exploratory-plan-checks red in run 38981523623). The block now lists the sourced signals itself. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * v1.91.111.0 fix(evals): four PR-gate background reds were a detector, an interface, a helper and a workflow-block defect Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> |
||
|
Garry Tan
|
857466ff8b |
v1.91.25.0 feat(cso): private two-architecture release scoring + self-checking runtime release gates (#3036)
* v1.91.23.0 feat(cso): private two-architecture release scoring and self-checking runtime release gates
* Keep the Linux-only native-evidence and requalification tests out of the Windows lane
* Skip the two Linux-release-only checks on Windows in-file instead of growing the shard list
* fix(cso): opaque producer inputs and receipts; requalification triggers independent of cwd
Producer inputs are {schemaVersion: 2, cellRef, skill, source, execution} and
receipts bind cellRef + inputHash; collection rebuilds the cell from the
matrix, so a cell VM never learns its case or variant. requalificationTriggers
resolves Bun's cwd-relative metafile inputs against the repository root.
* test(evals): bound the review judge's thinking; guarded browser cleanup is not a probe
review/SKILL.md workflow: the ~64k-token bundle let default-effort thinking
spend all 8192 output tokens without an answer in CI. Medium effort with a
64k streaming cap kept 24/24 measured samples under 5k tokens and 63 s with
the same gating scores (as ship already does).
qa-only-no-fix: the fixture prompt routes browser cleanup through the deadline
guard, but the checkpoint detector treated every guarded run as a probe that
needs a checkpoint. A child that only closes the tab with the supplied browse
binary, after the last probe, is cleanup; anything else stays a probe. The
captured CI transcript (run 37252022510) now passes and new scenarios pin
the negative controls.
* test(evals): keep the review judge at default effort; only lift its output cap
Truncation was the defect: thinking and the answer share max_tokens, and the
~64k-token review bundle once spent all 8192. The registration keeps default
effort and response format and gets ship's 64k streaming cap; the guard test
pins exactly that.
|
||
|
Garry Tan
|
dcaea52800 |
v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)
* feat: add surface-aware exploratory QA and ship documentation gates * test: preserve delegated QA setup authority after main integration * fix(qa): clarify exploration order and preserve report artifacts * test(qa): follow the shared setup reference directly * refactor(ship): make verification and recovery routes explicit * test(ship): align evidence and review guards with explicit routes * fix(workflows): clarify ship recovery and functional QA evidence * fix(workflows): clarify approval recovery and full QA coverage * refactor(workflows): order review transactions and clarify ship state * fix(ship): clarify final verification and fail closed at publication * fix(evals): attribute native atomic documentation writes * fix(ship): clarify recovery and documentation lifecycle guidance * fix(test): preserve observed native placeholder styling in CI * fix(codex): report watchdog timeouts without a process-exit race * Checkpoint functional QA implementation and workflow validation repairs * Fix documentation and shared-review fixture contracts * docs: clarify judge reuse and evaluation supervision * test: align review evidence and selected case contracts * test: verify append-only documentation checkpoints and recovery * fix: qualify QA workflows and CI validation repairs * fix: launch shared-libs fixture scripts on Windows * fix: qualify QA deadlines, fixture isolation, and shard cleanup * fix: preserve qualified QA and cancellation repairs * fix: enforce functional fixture authority and share strict event decoding * fix: retain free-test evidence and explain recovery * fix: reject malformed native evidence after decoder consolidation * test: use reliable capture for telemetry privacy filters * test: refresh measured quick coverage and document validation costs * Fix native fixture receipts and preserve VM validation evidence * Align negative judge controls with upstream clarity policy * Fix report-only QA preparation and public evidence handling * Clarify QA-only preparation and current-report preservation * Stream Ship quality judgments with an explicit 64k response contract * Validate compact judge reasoning locally with supported wire schema * Align functional QA fixture instructions with evidence acceptance * Bind native browser diagnostics to execution evidence and align review verdicts * Preserve native diagnostic line boundaries * Serialize functional QA evidence from native captures * Keep large QA evidence fixture payload out of Windows argv |