mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-01 14:34:42 +01:00
Garry Tanandcapy-ai[bot]
ab79e55119
* v1.91.70.0 feat(unattended): artifact contract, unattended session kind, findings-derived review log, gate list Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * v1.91.72.0 feat(autoplan): unattended runner next|submit, pluggable outside voice, review checklists, spec-loop cap Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(b-runner): atomic lock creation in spend-ledger, judge bundle skips runner-only sections, fixture env spelling - lib/spend-ledger.ts: create the lock by linking a complete temp file so a contender never reads an empty lock and breaks a live owner (CI admitted 20 of a cap of 8 under contention); unreadable locks break only by mtime age. - test/helpers/workflow-judge-input.ts: exclude manifest runner_only sections (the B4 checklists) from the judge bundle; the judge rates the interactive workflow. test/workflow-judge-input.test.ts expects the same set. - test/helpers/autoplan-run-fixture.ts: spell the runner's session kind without the literal the free-suite environment scan rejects. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * v1.91.74.0 feat(review): plan reality rows, sideways review and cso sweep, investigate recipes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: register the investigate carve case and map Reality rows to real headings in the review checklists Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: pin the corrected Reality rows obligation rows Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(investigate): use allowlisted placeholders in the flake and polluter fences; select the investigate carve case on PTY changes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
1.4 KiB
1.4 KiB
Eval bisect recipe
A judged or scored metric regressed between two revisions. Bisect inputs, not commits, and spend the paid judge last.
- Distinct revisions only.
git log --oneline <good>..<bad> -- <prompt and resolver paths>; collapse commits that leave the expanded prompt bytes identical (build the prompt with the actual builder at each revision and compare hashes; a template edit that renders the same bytes is not a candidate). - A cached cheap scorer first. Score every distinct revision with the
deterministic or cheap check (the structural judge, a token count, the
quality judge at its lowest tier) and cache results per
(revision, case)under.gstack/tmp/eval-bisect/. Bisect on the cheap score; spend the paid judge only on the two revisions where the cheap scorer disagrees with the reported regression. - Detached runs. Launch paid cases through the documented detached runner
(
bun run eval:bg:pr), poll the log for### gstack-detach EXIT=<code> ###, and keep the eval lock. Never retry a paid case; a failed launch is a launch failure in the report, not a second sample. - Report. The first bad revision with its prompt diff, the cheap and paid scores per revision (cached hits marked), the pre-registered panel size and every trial, and the cases that were not run.