Commit Graph
4 Commits
Author SHA1 Message Date
Garry Tanandcapy-ai[bot] 83ae96e086 v1.92.30.0 test(free): split files over the 60 s seed ceiling and the Windows doctor file, then reseed (GSTA-151 A) (#3585)
* test(free): split files over the 60 s seed ceiling and the Windows doctor file

Eight free test files the fresh recordings put over the seed ratchet's 60 s ceiling, and test/gstack-doctor.test.ts (343 s on Windows), are split by describe block or test with shared helpers moved to test/helpers/. Test bodies are unchanged.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(todos): note the GSTA-151 seed-ceiling splits

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(free): reseed both duration seeds at this head

Recorded by free-durations-record.yml (open_pr=false) run 40818261447 at ec6faf1:
1,416 Linux files on ubicloud-standard-8 and the 795 curated Windows files on
windows-latest. test/setup-codex-scope.test.ts fell to 49 s and leaves the
seed allowlist.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore: changelog entry for the GSTA-151 seed-ceiling splits (unreleased)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(free): split test/ubicloud-runner.test.ts over the 60 s seed ceiling; keep setup-codex-scope allowlisted

The reseed at the main merge (free-durations-record run 41226501374, 221320e) put
test/ubicloud-runner.test.ts, which main added after this branch's first reseed,
at 127 s and test/setup-codex-scope.test.ts back at 66 s. The fake-API cases move
into four parts by describe topic with their shared fake VM in
test/helpers/ubi-runner-fake-vm.ts (46 tests before and after). The allowlist
keeps main's setup-codex-scope entry.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(free): reseed both duration seeds at the main merge; changelog fragment

Recorded by free-durations-record.yml (open_pr=false) run 41229924108 at 4f5bd2d:
1,470 Linux files on ubicloud-standard-8 and 822 curated Windows files. The
Windows recorder was red on browse/test/cookie-import-native-job.test.ts (known
Edge leak) and test/hook-scripts.test.ts, plus test/free-seed-ratchet.test.ts on
both OSes against the pre-reseed seed; their durations are kept. The CHANGELOG
entry moves to changelog.d/ now that main runs on fragments.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: correct the seed-ceiling TODO and the ubicloud-runner test paths after the split

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-11-01 14:23:49 +00:00
Garry Tanandcapy-ai[bot] 29f8ca053e v1.91.301.0 fix(flaky): four paid rule cases stop failing correct runs (2026-10-28 pass) (#3547)
* fix(flaky): repair four paid rule fixtures and detectors from the 2026-10-28 pass (GSTA-139 S2)

- plan-design-with-ui-scope: the focus detector accepts the all-7 offer in the
  rated title line; issue-numbered options never count as Step 0D (gate edit).
- plan-review-report: the fixture state uses codex_reviews disabled so the
  default-on outside voice no longer spends the 20-turn budget (gate edit).
- shared-libs-review-path-eligibility: an errored multi-command Bash read counts
  when its output holds the exact evidence source (gate edit).
- office-hours-brain-writeback: the fake gbrain accepts the page on stdin, the
  SAVE_RESULTS template form.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(changelog): note the 2026-10-28 S2 flaky repairs under Unreleased

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(evals): re-pin two detector-inventory line ranges moved by the S2 repairs

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes (Git Bash path for the preflight probe; empty-source control)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(fixtures): register the two S2 replay fixtures as free fixtures

The remote pregate at f28fbdf failed FREE_FIXTURES inventory: the new
captured-failure fixtures are read only by free replay tests and were in
neither a paid touchfile nor FREE_FIXTURES.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-10-29 00:16:09 +00:00
Garry Tanandcapy-ai[bot] 3766edac8d v1.91.144.0 feat(evals): Opus 5.5 is the paid gate's model under test; judge stays Fable (GSTA-49 TM1) (#3275)
* feat(evals): Opus 5.5 is the paid gate's model under test; judge stays Fable (GSTA-49 TM1)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* v1.91.140.0 chore: stamp TM1 (Opus 5.5 model under test) and CHANGELOG

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(evals): repair the five Opus gate reds from run 39161502626

- coverage audit: owned cd; followed by one && chain, wc -l on in-cwd files
- shared-libs: Claude Code 2.1.292 git --version probe, exact argv only
- ship hook: accept 'integrate the hook yourself' / 'combine ... by hand'
- qa-only: deadline starts only after the charter Write completes
- plan floor: tool-only parent session identifies the owned Create grant;
  floor runner fails fast with PERMISSION_DIALOG_UNANSWERED

Each repair replays its captured run as a free regression with negative
controls. HARNESS_VERSION stays 6 (re-recorded from main's 5).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(evals): ship hook manual-integration sentence rule; qa-only inspection uses Read/Glob; tool-only permission tests join their owner file

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(evals): tpa-absent-linux accepts instructions the user follows themselves as the manual option

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(evals): owned plan Edit menu binds to exactly one pending Edit, including unpublished ones

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(evals): shared-libs read evidence counts a literal path loop whose output names each file

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore: restamp to v1.91.142.0 (queue slot moved)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(evals): investigate boundary fixture identifies the owner question by its continue/abort options

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(evals): plan-design-with-ui focus check accepts the skill's design-completeness rating in the question body

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(evals): phrasing sweep across the 108 gate rule cases; replay captured Opus paraphrases with negative controls

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(evals): command-detector sweep — exact-count anchor, label slot, 2.1.292 worktree probe

- coverage audit (gate 39207508651 plan-eng-coverage-audit): output of exactly
  known length (wc -l over literal in-cwd files) anchors the next owned read
  at that line; a label's slot counts from the end and a later complete copy
  voids the credit (review/specialists/testing.md prints '---')
- shared-libs-unsupported-git (39161502626): Claude Code 2.1.292's askPass-first
  'worktree list --porcelain -z' cwd probe is Claude's own; the read-only Git
  check is shared (repoGitRequests, isReadOnlyRepoGitRequest) and the free
  regression replays every Git request of that capture

Each fix replays its captured command with negative controls. No threshold,
trial count, case kind or judge changed. HARNESS_VERSION stays 6.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-10-19 07:26:49 +00:00
Garry Tan dcaea52800 v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)
* feat: add surface-aware exploratory QA and ship documentation gates

* test: preserve delegated QA setup authority after main integration

* fix(qa): clarify exploration order and preserve report artifacts

* test(qa): follow the shared setup reference directly

* refactor(ship): make verification and recovery routes explicit

* test(ship): align evidence and review guards with explicit routes

* fix(workflows): clarify ship recovery and functional QA evidence

* fix(workflows): clarify approval recovery and full QA coverage

* refactor(workflows): order review transactions and clarify ship state

* fix(ship): clarify final verification and fail closed at publication

* fix(evals): attribute native atomic documentation writes

* fix(ship): clarify recovery and documentation lifecycle guidance

* fix(test): preserve observed native placeholder styling in CI

* fix(codex): report watchdog timeouts without a process-exit race

* Checkpoint functional QA implementation and workflow validation repairs

* Fix documentation and shared-review fixture contracts

* docs: clarify judge reuse and evaluation supervision

* test: align review evidence and selected case contracts

* test: verify append-only documentation checkpoints and recovery

* fix: qualify QA workflows and CI validation repairs

* fix: launch shared-libs fixture scripts on Windows

* fix: qualify QA deadlines, fixture isolation, and shard cleanup

* fix: preserve qualified QA and cancellation repairs

* fix: enforce functional fixture authority and share strict event decoding

* fix: retain free-test evidence and explain recovery

* fix: reject malformed native evidence after decoder consolidation

* test: use reliable capture for telemetry privacy filters

* test: refresh measured quick coverage and document validation costs

* Fix native fixture receipts and preserve VM validation evidence

* Align negative judge controls with upstream clarity policy

* Fix report-only QA preparation and public evidence handling

* Clarify QA-only preparation and current-report preservation

* Stream Ship quality judgments with an explicit 64k response contract

* Validate compact judge reasoning locally with supported wire schema

* Align functional QA fixture instructions with evidence acceptance

* Bind native browser diagnostics to execution evidence and align review verdicts

* Preserve native diagnostic line boundaries

* Serialize functional QA evidence from native captures

* Keep large QA evidence fixture payload out of Windows argv
2026-09-29 06:07:35 -07:00