mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-31 14:06:50 +01:00
TODOS/CHANGELOG: close plan-eng-review-artifact (real PTY review, 498 s pass)
This commit is contained in:
1 parent
289672264a
commit
364f0f3492
2 files changed
+4
-9
No files matched your search
+1
-1
@@ -98,7 +98,7 @@ The first two rows are the ones to care about: a review that read nothing, or fo
|
||||
- Eval children no longer receive credential env vars that carry a suffix (`GITHUB_TOKEN_1`, `GITHUB_APP_PRIVATE_KEY_BASE64`); admit one explicitly with `extraAllow` (#2949). Chunked large-push coverage for the pre-push guard (#2915).
|
||||
- The test typecheck ratchet ignores the order tsc prints an object type's members in.
|
||||
- The weekly eval report counts only cases that ran: tests deselected for another tier or case shard are listed by reason with no credit, every result is attributed to its case (0 unattributed, was 72), and no PR-lane eval shard runs over 10 minutes (the deploy eval is split per case).
|
||||
- Codex evals run on their own CI job where Codex's sandbox can start, and pass only when Codex completed a command; a review whose sandbox failed now fails instead of passing. The `/codex review` eval passes only when Codex actually reviewed the diff. The AUQ matrix checks the `(recommended)` marker on option labels, where the auto-decide hook reads it.
|
||||
- Codex evals run on their own CI job where Codex's sandbox can start, and pass only when Codex completed a command; a review whose sandbox failed now fails instead of passing. The `/codex review` eval passes only when Codex actually reviewed the diff. The AUQ matrix checks the `(recommended)` marker on option labels, where the auto-decide hook reads it. The `/plan-eng-review` QA test-plan eval now drives the real interactive review instead of a shortcut prompt that skipped the workflow.
|
||||
|
||||
Contributed by @CermakM (#2843), @Haijie-Lee (#2192), @SomSamantray (#2965), @Vaughan-g-aus (#2329), @BharadwajMittapelli (#2729), @maxpetrusenkoagent (#1931, #1932, #1981), @Infiniteyieldai (#2884), @afshaker (#2778), @time-attack (#2232, #2229), @jzeisweiss (#2670), @liutiming (#2783), @agile-operators (#1437), @benjaminberes-bp (#2369), @lewispeel (#2885), @Yugz29 (#2899), @fcmerle (#2962), @cdecook23 (#2784), @mb1810 (#2913), @ntdatt812 (#2949), @pneumorea-ai (#2915), @gabrielrondon (#2811), @kevingrasso32 (#2876), @xwang4-svg (#2354), @DizzyWesterwald (#2936), @oli548 (#2771), @shofel (#1968), @ryanlaiwy (#1633), @yinanli1917-cloud (#2281), @Spooks444 (#2260), @punksterlabs (#2439), @himiaocc (#2492), @jbetala7 (#1419, #1777), @mamedov (#2891), @Lockyer228 (#2651), @topcoder1 (#2263), @jonahberg (#2707), @AlinValentin7 (#3015), @philhie (#2208), @Alchemist-DevAI (#2334), @merit-blake (#2343), @day-of-davon (#2952), @loulanyue (#2797, #2792, #2794, #2796), @Bit0ps (#2840), @szsunyuan (#2799), @frosimanuel (#2818), @JiayuuWang (#1096), @RyanAlberts (#1728), @exGeni (#2704), @Jino00 (#2812), @snig-17 (#2809) and @SholtoMc (#2037).
|
||||
|
||||
|
||||
@@ -75,19 +75,14 @@ Deferred from the approved severe fix wave (docs/designs/SEVERE_FIX_WAVE_2026_10
|
||||
(`GIT_OBJECT_DIRECTORY` + `GIT_ALTERNATE_OBJECT_DIRECTORIES` under
|
||||
`$TMPDIR`) for the fingerprint, and report the fetch as stale-base coverage.
|
||||
**Effort:** S. **Priority:** P2.
|
||||
- **plan-eng-review-artifact skips the QA test plan (0/4 since #3017)** —
|
||||
red in 37151477069, 37158847998, 37162480720 and one local CI-image run.
|
||||
Every attempt reads SKILL.md, sections, plan and sources, writes the whole
|
||||
review into review-output.md in one Write, runs no Bash, and never runs the
|
||||
Test Plan Artifact block. A prompt destination rewording and a self-check
|
||||
rewording each got one paid rerun and did not move it. Options: put the
|
||||
artifact instruction back in the prompt, move the assertion into a PTY
|
||||
full-workflow case, or quarantine with this evidence. **Priority:** P2.
|
||||
- **Closed by the wave's evals2 lane (run 37162480720):** Codex paid evals now
|
||||
run commands in CI (host-run job; all 8 Codex cases executed real commands),
|
||||
ship-exploratory-late-input no longer ends `inconclusive` (capture names the
|
||||
probes to rerun after an input change), plan-eng-multi-finding-batching's
|
||||
checker and office-hours-brain-writeback's prompt are fixed (3/3).
|
||||
plan-eng-review-artifact (0/4 since #3017) measured a shortcut prompt that
|
||||
skipped the workflow; it now drives the real interactive /plan-eng-review in
|
||||
a PTY and wrote its QA test plan at 498 s in one CI-image paid run.
|
||||
- **Gate-outcome telemetry** — count `not_run`/`unavailable` outcomes per
|
||||
reason code from `lib/gate-outcomes.ts` so the most common gaps get fixed
|
||||
first. **Effort:** S. **Priority:** P3.
|
||||
|
||||
Reference in new issue
Block a user