From a509613f453d4579fc2f01c7d2b4df7efcf3dde5 Mon Sep 17 00:00:00 2001 From: garrytan Date: Mon, 2 Nov 2026 16:23:31 +0000 Subject: [PATCH] docs: free-suite job timeout follows the plan (#3850) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- changelog.d/gsta203-3850-job-timeout.md | 1 + docs/TESTING_INTERNALS.md | 11 ++++++++++- 2 files changed, 11 insertions(+), 1 deletion(-) create mode 100644 changelog.d/gsta203-3850-job-timeout.md diff --git a/changelog.d/gsta203-3850-job-timeout.md b/changelog.d/gsta203-3850-job-timeout.md new file mode 100644 index 000000000..4c67c470f --- /dev/null +++ b/changelog.d/gsta203-3850-job-timeout.md @@ -0,0 +1 @@ +Free Tests no longer goes red with no named file when one test hangs. Each `free-suite` job's GitHub timeout used to be a fixed 20 minutes, shorter than the runner's own wall for a 10-shard plan (about 37 minutes), so GitHub killed the job before the runner could name the hung file or retry it, and no shard result was uploaded. The plan now emits each shard's job timeout as its runner wall plus the flaky-retry pass plus the job's measured setup (`--ci-plan --job-setup-seconds`), 49 minutes per shard today, and fails loudly above 90 minutes instead of truncating. See docs/TESTING_INTERNALS.md#runners-how-the-suites-execute-2026-08-overhaul. Closes #3850. diff --git a/docs/TESTING_INTERNALS.md b/docs/TESTING_INTERNALS.md index 441525762..068adaae0 100644 --- a/docs/TESTING_INTERNALS.md +++ b/docs/TESTING_INTERNALS.md @@ -206,7 +206,16 @@ GSTA-203 chose them by a 3-run A/B against 20 × `ubicloud-standard-4` at k = 1 (the PR that set them has the table). A `workflow_dispatch` run may override them with `free_runner` (`ubicloud-standard-2`, `-4` or `-8`), `free_shards` (1–40) and `free_jobs` (1–8) for runner-sizing experiments only; -its job summary says so, and it is never cited as a PR or train Free Tests result. Each shard prints +its job summary says so, and it is never cited as a PR or train Free Tests result. +Each `free-suite` job's GitHub timeout comes from the plan, not a fixed number +(#3850): `--ci-plan --job-setup-seconds $FREE_SUITE_SETUP_SECONDS` emits every +shard's `timeout_minutes` as the shard's runner wall (the packed wall above), +plus the flaky-retry pass's wall (at most 5 files, so the 6-minute base), plus +the job's setup and teardown (`ciShardJobTimeoutMinutes`). So GitHub never kills +a hung file before the runner names it and retries it. A shard that would need +more than 90 minutes fails the plan with `FREE_JOB_TIMEOUT_OVER_CAP`; plan more +shards or a higher k instead. The plan step prints the emitted timeouts and adds +them to its job summary. Each shard prints `RESOURCES: peak_used_mib=… min_free_disk_mib=… samples=…` from a memory and disk sample every 2 seconds, even when the suite fails. The public `TREE_MUTATING` map now classifies exclusive host-state fixtures; its sole