From 973d7e3ba9e41dbd09365f3c438e6951172684e4 Mon Sep 17 00:00:00 2001 From: garrytan Date: Fri, 30 Oct 2026 21:30:37 +0000 Subject: [PATCH] chore(changelog): move the GSTA-178 entry to changelog.d/capy-gsta178-flaky.md (fragments, #3654) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 22 ---------------------- changelog.d/capy-gsta178-flaky.md | 19 +++++++++++++++++++ 2 files changed, 19 insertions(+), 22 deletions(-) create mode 100644 changelog.d/capy-gsta178-flaky.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 2b7251bf3..92354d630 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,27 +1,5 @@ # Changelog -## [Unreleased] - -**Nine flaky tests from the 2026-10-30 flaky pass stop failing correct runs.** Two paid detectors missed correct replies, one paid case lost the evidence of its own failure, five free tests raced their own fixtures or paid a cold start inside a timed call, and two Windows failures came from real defects: a lost IPC receipt in the QA deadline worker and a Chromium feature that fails loopback connections. - -### Fixed -- **`lib/qa-deadline.ts`**: the receipt worker waits for the parent to acknowledge each receipt before it exits. Bun can drop an IPC message the parent has not read yet when the child exits, so on Windows `gstack-qa-deadline run` and `gstack-qa-evidence` could return the exit code without the `finished` receipt (Windows Free Tests 40991662945, `test/qa-deadline.test.ts:431` received one of two receipts). A receipt still unacknowledged after the existing 5 s write wait makes the worker exit 2, as an unwritten receipt did before. -- **`patches/playwright-core@1.62.1.patch`**: backports microsoft/playwright#43605, which disables Chromium's `TcpPortRandomizationWin`. On Windows it fails loopback page loads with `net::ERR_NO_BUFFER_SPACE` even when ports are free (`browse/test/cookie-picker-ui.test.ts`, Windows Free Tests 40656299153 and 41056150287). gstack's own browser launches on Windows had the same exposure. -- **`test/helpers/qa-only-cleanup.ts`**: the cleanup check reads the daemon state file once and treats a file deleted during shutdown as absent, instead of checking that it exists and then reading it (Free Tests 40964422549: ENOENT on `browse.json`). -- **`test/claude-code-runner.test.ts`**: the liveness helper counts a process in the `X` (being reaped) state as dead, like `Z`, and the cleanup no longer kills a pid that is already gone (Free Tests 41034844588: ESRCH). -- **`test/helpers/free-owned-browser-settlement.ts`**: in the `cancel-vanishing-record` case the fake daemon waits for the observer to remove its record before it exits, so the race the case injects always happens (Free Tests 40924799742). -- **`test/investigate-freeze-path.test.ts`**: the E1 cases pay the first bash and JSON interpreter start once, in an unasserted warm-up, before the timed hook calls. In four Windows runs (40824965988, 40917695070, 40985163846, 40996480693) only the file's first hook call hit the 15 s kill, the second took 1.9 to 7.9 s and the rest 0.3 to 1 s. Each timed call keeps its 15 s kill. - -### Gate integrity -- **ship-unmanaged-hook-consent** (`test/helpers/ship-hook-actor.ts`): the decline check also credits "no hook/change/file was installed, changed, modified, overwritten or replaced". Gate 41005113772 said "No hook was installed or changed" and "nothing was overwritten" after a correct decline; its installs, hook bytes and question checks all passed. The four negative controls still fail. -- **tpa-absent-darwin** (`test/helpers/third-party-actions.ts`): the Aside download pitch also counts in the passive voice at a clause start ("Aside ... can be downloaded at aside.com", gate 40987391676). Negated, reported and mid-clause forms still count zero. - -### Changed -- **autoplan-guard-pty** (`test/helpers/autoplan-guard-pty.ts`, `test/skill-e2e-autoplan-guard-pty.test.ts`): a `dispatch_prompt` deny prints each Agent dispatch's length, first differing offset and context next to the prompt the actor had to copy, read before cleanup deletes it. Gate 40992022983 lost those bytes because the journal summary cuts inputs at 160 characters. - -### Tests -- New captures: `test/fixtures/ship-hook-consent-2026-10-30.json` and a passive-voice capture in `test/fixtures/third-party-actions-pitch-captures.json`; both tests fail with their detector change reverted. `browse/test/playwright-core-patch.test.ts` checks the backport in the patch and the installed tree, and fails without it. `test/autoplan-guard-pty-fixture.test.ts` checks the dispatch delta on a synthetic journal. `test/qa-deadline.test.ts` plays the parent, so it now acknowledges each receipt; its assertions are unchanged. - ## [1.91.333.0] - 2026-10-30 **`gstack-factory-scorecard --now` now ends the window, so a past week no longer counts what happened after it.** Every line counts only items whose timestamps fall in `[now - since-days, now)`. Before, `--now` moved only the start, so `--now 2026-10-15T00:00:00Z --since-days 14` counted 290 merged PRs (every merge from 2026-10-01 through the day of the run) where the window holds 65, and the gate-run and release lines also counted through the day of the run. Rerun with the fix, the two weeks before 2026-10-15 show 65 merged PRs and the two weeks before 2026-10-29 show 197, matching GitHub search for both windows. diff --git a/changelog.d/capy-gsta178-flaky.md b/changelog.d/capy-gsta178-flaky.md new file mode 100644 index 000000000..943b9e03a --- /dev/null +++ b/changelog.d/capy-gsta178-flaky.md @@ -0,0 +1,19 @@ +**Nine flaky tests from the 2026-10-30 flaky pass stop failing correct runs.** Two paid detectors missed correct replies, one paid case lost the evidence of its own failure, five free tests raced their own fixtures or paid a cold start inside a timed call, and two Windows failures came from real defects: a lost IPC receipt in the QA deadline worker and a Chromium feature that fails loopback connections. + +### Fixed +- **`lib/qa-deadline.ts`**: the receipt worker waits for the parent to acknowledge each receipt before it exits. Bun can drop an IPC message the parent has not read yet when the child exits, so on Windows `gstack-qa-deadline run` and `gstack-qa-evidence` could return the exit code without the `finished` receipt (Windows Free Tests 40991662945, `test/qa-deadline.test.ts:431` received one of two receipts). A receipt still unacknowledged after the existing 5 s write wait makes the worker exit 2, as an unwritten receipt did before. +- **`patches/playwright-core@1.62.1.patch`**: backports microsoft/playwright#43605, which disables Chromium's `TcpPortRandomizationWin`. On Windows it fails loopback page loads with `net::ERR_NO_BUFFER_SPACE` even when ports are free (`browse/test/cookie-picker-ui.test.ts`, Windows Free Tests 40656299153 and 41056150287). gstack's own browser launches on Windows had the same exposure. +- **`test/helpers/qa-only-cleanup.ts`**: the cleanup check reads the daemon state file once and treats a file deleted during shutdown as absent, instead of checking that it exists and then reading it (Free Tests 40964422549: ENOENT on `browse.json`). +- **`test/claude-code-runner.test.ts`**: the liveness helper counts a process in the `X` (being reaped) state as dead, like `Z`, and the cleanup no longer kills a pid that is already gone (Free Tests 41034844588: ESRCH). +- **`test/helpers/free-owned-browser-settlement.ts`**: in the `cancel-vanishing-record` case the fake daemon waits for the observer to remove its record before it exits, so the race the case injects always happens (Free Tests 40924799742). +- **`test/investigate-freeze-path.test.ts`**: the E1 cases pay the first bash and JSON interpreter start once, in an unasserted warm-up, before the timed hook calls. In four Windows runs (40824965988, 40917695070, 40985163846, 40996480693) only the file's first hook call hit the 15 s kill, the second took 1.9 to 7.9 s and the rest 0.3 to 1 s. Each timed call keeps its 15 s kill. + +### Gate integrity +- **ship-unmanaged-hook-consent** (`test/helpers/ship-hook-actor.ts`): the decline check also credits "no hook/change/file was installed, changed, modified, overwritten or replaced". Gate 41005113772 said "No hook was installed or changed" and "nothing was overwritten" after a correct decline; its installs, hook bytes and question checks all passed. The four negative controls still fail. +- **tpa-absent-darwin** (`test/helpers/third-party-actions.ts`): the Aside download pitch also counts in the passive voice at a clause start ("Aside ... can be downloaded at aside.com", gate 40987391676). Negated, reported and mid-clause forms still count zero. + +### Changed +- **autoplan-guard-pty** (`test/helpers/autoplan-guard-pty.ts`, `test/skill-e2e-autoplan-guard-pty.test.ts`): a `dispatch_prompt` deny prints each Agent dispatch's length, first differing offset and context next to the prompt the actor had to copy, read before cleanup deletes it. Gate 40992022983 lost those bytes because the journal summary cuts inputs at 160 characters. + +### Tests +- New captures: `test/fixtures/ship-hook-consent-2026-10-30.json` and a passive-voice capture in `test/fixtures/third-party-actions-pitch-captures.json`; both tests fail with their detector change reverted. `browse/test/playwright-core-patch.test.ts` checks the backport in the patch and the installed tree, and fails without it. `test/autoplan-guard-pty-fixture.test.ts` checks the dispatch delta on a synthetic journal. `test/qa-deadline.test.ts` plays the parent, so it now acknowledges each receipt; its assertions are unchanged.