* feat(evals): Opus 5.5 is the paid gate's model under test; judge stays Fable (GSTA-49 TM1)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* v1.91.140.0 chore: stamp TM1 (Opus 5.5 model under test) and CHANGELOG
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(evals): repair the five Opus gate reds from run 39161502626
- coverage audit: owned cd; followed by one && chain, wc -l on in-cwd files
- shared-libs: Claude Code 2.1.292 git --version probe, exact argv only
- ship hook: accept 'integrate the hook yourself' / 'combine ... by hand'
- qa-only: deadline starts only after the charter Write completes
- plan floor: tool-only parent session identifies the owned Create grant;
floor runner fails fast with PERMISSION_DIALOG_UNANSWERED
Each repair replays its captured run as a free regression with negative
controls. HARNESS_VERSION stays 6 (re-recorded from main's 5).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(evals): ship hook manual-integration sentence rule; qa-only inspection uses Read/Glob; tool-only permission tests join their owner file
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(evals): tpa-absent-linux accepts instructions the user follows themselves as the manual option
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(evals): owned plan Edit menu binds to exactly one pending Edit, including unpublished ones
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(evals): shared-libs read evidence counts a literal path loop whose output names each file
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* chore: restamp to v1.91.142.0 (queue slot moved)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(evals): investigate boundary fixture identifies the owner question by its continue/abort options
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(evals): plan-design-with-ui focus check accepts the skill's design-completeness rating in the question body
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(evals): phrasing sweep across the 108 gate rule cases; replay captured Opus paraphrases with negative controls
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(evals): command-detector sweep — exact-count anchor, label slot, 2.1.292 worktree probe
- coverage audit (gate 39207508651 plan-eng-coverage-audit): output of exactly
known length (wc -l over literal in-cwd files) anchors the next owned read
at that line; a label's slot counts from the end and a later complete copy
voids the credit (review/specialists/testing.md prints '---')
- shared-libs-unsupported-git (39161502626): Claude Code 2.1.292's askPass-first
'worktree list --porcelain -z' cwd probe is Claude's own; the read-only Git
check is shared (repoGitRequests, isReadOnlyRepoGitRequest) and the free
regression replays every Git request of that capture
Each fix replays its captured command with negative controls. No threshold,
trial count, case kind or judge changed. HARNESS_VERSION stays 6.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
---------
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
The paid CSO producer gives `/cso` the Agent tool users have, so
independent review and repair verification can run in the evaluation. The
producer's receipt now counts subagent tokens.
- every helper artifact path tripped the secret redactor: the run ID's
13-digit epoch reads as a phone number, some 32-hex IDs as wallets.
Validate paths against the helper layout; schema-shaped ID components
are accepted by shape, every other component still goes through redact
- the provider was killed at exactly --budget after cell start, before
the helper deadline (start + budget) and before finish; give it the
budget plus a 300 s reporting grace (evaluator graceSeconds)
- Claude producers stream events (stream-json --verbose
--include-partial-messages) and add usage per message, so a timed-out
receipt keeps tokens, cost, turns and the last text
- failed post-run checks name the check in the receipt reason
* v1.91.68.0 fix(cso-eval): failed receipts with usage for post-run integrity failures; Claude producers stop mutating the sealed source
Claude Code adds <dir>/.mcp.json as a sandbox deny path for every --add-dir.
The producer passed the sealed source as one, so bubblewrap left an empty
0444 .mcp.json mount point there that Claude Code cannot unlink from the 0555
directory, and the post-run source check failed with no receipt.
- claudeExecArgs no longer adds the source directory
- post-run integrity failures write status:failed receipts carrying the
check code/reason plus usage, tokens and estimated cost (no output or
artifacts)
- INVALID_PRODUCER_SOURCE / PRODUCER_CHANGED_SOURCE_MODE name the first
offending relative path and the rule it broke
* fix(cso-eval): run the trusted launcher outside Claude's sandbox; keep every other command sandboxed
Claude Code 2.1.263 forces every Bash command into its bubblewrap sandbox
while CLAUDE_CODE_SUBPROCESS_ENV_SCRUB is set, excludedCommands
notwithstanding. Inside it the launcher's owner check sees root as nobody
and the Docker socket is unreachable, so comprehensive cells degraded to
static-only results.
- drop CLAUDE_CODE_SUBPROCESS_ENV_SCRUB for producers; pass --settings with
sandbox enabled, failIfUnavailable, no unsandboxed fallback, no
auto-allow, and excludedCommands naming only the exact launcher path
- ClaudeAdapter.run refuses to start the agent
(CLAUDE_PRODUCER_SANDBOX_POLICY_MISMATCH) unless args and env carry
exactly that policy
- README containment note and CHANGELOG updated
* test(doctor): split the two multi-run doctor tests so each fits Windows' 30s per-test budget
On windows-latest each bin/gstack-doctor run costs ~4-5s. The cached/live probe test made 7 runs and the artifacts-sync test 5; both hit the 30s per-test timeout, which killed doctor mid-output ("no row codex probe") and then EBUSY on cleanup. Same failure on #3093 and the fix/runtime-root-source-guard PR. Every assertion is kept; each test now makes at most 4 runs.
