Garry Tan and Claude Fable 5
bd11416d80
fix(evals): absorb codex/gemini CLI drift; external-service tests go periodic-tier
...
- codex exec gains --skip-git-repo-check: newer CLIs refuse exec in an
untrusted non-git dir (our temp skill dirs) — empirically verified.
- gemini: --skip-trust was removed in gemini-cli 0.34 (argv parse error);
dropped from the session runner and the benchmark adapter. A present-
but-unusable CLI (deprecated individual code-assist auth path) now
classifies as SKIP, not a false adapter failure; the benchmark live
smoke skips on auth/rate_limit error codes (environmental) while still
failing on timeout/unknown (the drift classes it exists to catch).
- codex-e2e, gemini-e2e, and benchmark-providers gain the canonical
whole-file EVALS_TIER === 'periodic' guard per CLAUDE.md tiering rule 3
(external service -> periodic) — the sharded gate runner now excludes
all three (gate: 45 -> 42 shards).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-08-13 10:34:25 -07:00
..
2026-08-12 15:31:49 -07:00
2026-08-13 10:34:25 -07:00
2026-06-14 11:40:57 -07:00
2026-03-18 23:57:59 -05:00
2026-05-17 08:26:36 -07:00
2026-06-25 09:42:45 -07:00
2026-06-07 21:38:21 -07:00
2026-06-07 21:38:21 -07:00
2026-07-10 13:58:07 -07:00
2026-08-07 15:08:12 -07:00
2026-04-19 17:50:31 +08:00
2026-06-21 07:15:19 -07:00
2026-06-08 06:39:38 -07:00
2026-05-29 08:35:00 -07:00
2026-05-29 08:35:00 -07:00
2026-05-25 10:57:15 -07:00
2026-08-12 16:02:11 -07:00
2026-06-21 07:15:19 -07:00
2026-05-20 07:35:01 -07:00
2026-04-08 22:21:28 -10:00
2026-05-29 08:35:00 -07:00
2026-06-07 19:13:24 -07:00
2026-06-07 19:13:24 -07:00
2026-06-07 19:13:24 -07:00
2026-06-07 19:13:24 -07:00
2026-08-12 15:31:49 -07:00
2026-05-30 14:57:07 -07:00
2026-05-30 14:57:07 -07:00
2026-04-23 07:25:20 -07:00
2026-06-04 11:14:43 -07:00
2026-08-13 10:34:25 -07:00
2026-05-14 11:11:52 -04:00
2026-05-09 08:06:47 -07:00
2026-05-16 12:32:33 -07:00
2026-08-12 15:31:49 -07:00
2026-08-08 09:28:45 -07:00
2026-06-07 19:13:24 -07:00
2026-06-07 19:13:24 -07:00
2026-05-28 18:21:09 -07:00
2026-08-08 09:28:45 -07:00
2026-06-09 22:29:23 -07:00
2026-06-12 15:38:53 -07:00
2026-06-08 06:39:38 -07:00
2026-05-30 12:09:10 -07:00
2026-05-28 18:21:09 -07:00
2026-05-28 18:21:09 -07:00
2026-05-21 21:21:07 -07:00
2026-06-07 19:13:24 -07:00
2026-04-26 13:55:13 -07:00
2026-08-12 11:12:28 -07:00
2026-08-12 15:31:48 -07:00
2026-08-12 16:02:11 -07:00
2026-08-12 15:31:47 -07:00
2026-08-07 15:08:12 -07:00
2026-05-24 01:43:51 -07:00
2026-05-20 07:35:01 -07:00
2026-06-08 06:20:58 -07:00
2026-05-30 14:57:07 -07:00
2026-06-21 07:15:19 -07:00
2026-06-21 07:15:19 -07:00
2026-06-08 06:20:58 -07:00
2026-05-17 08:26:36 -07:00
2026-06-08 06:20:58 -07:00
2026-05-14 17:20:48 -07:00
2026-05-21 18:55:55 -07:00
2026-05-21 21:21:07 -07:00
2026-04-24 01:38:21 -07:00
2026-08-08 09:28:45 -07:00
2026-08-13 10:34:25 -07:00
2026-04-24 01:38:21 -07:00
2026-05-17 08:26:36 -07:00
2026-05-30 14:57:07 -07:00
2026-08-08 09:28:45 -07:00
2026-05-30 14:57:07 -07:00
2026-05-21 21:21:07 -07:00
2026-07-10 16:51:16 -07:00
2026-05-21 18:55:55 -07:00
2026-08-13 10:34:25 -07:00
2026-05-26 16:50:03 -07:00
2026-06-09 22:29:23 -07:00
2026-08-12 11:12:28 -07:00
2026-05-20 06:56:41 -07:00
2026-05-14 11:11:52 -04:00
2026-05-21 21:21:07 -07:00
2026-05-20 07:35:01 -07:00
2026-05-28 18:21:09 -07:00
2026-05-30 08:54:46 -07:00
2026-06-08 06:20:58 -07:00
2026-06-08 06:20:58 -07:00
2026-06-08 06:20:58 -07:00
2026-06-14 11:40:57 -07:00
2026-08-08 09:28:45 -07:00
2026-08-12 15:31:49 -07:00
2026-08-08 09:28:45 -07:00
2026-08-12 15:31:48 -07:00
2026-04-28 01:17:54 -07:00
2026-05-20 07:35:01 -07:00
2026-06-08 06:39:38 -07:00
2026-07-14 12:56:09 -07:00
2026-05-17 08:26:36 -07:00
2026-05-25 10:57:15 -07:00
2026-05-20 07:35:01 -07:00
2026-06-21 07:15:19 -07:00
2026-08-08 09:28:45 -07:00
2026-05-30 08:54:46 -07:00
2026-07-10 03:09:13 -07:00
2026-05-29 08:35:00 -07:00
2026-06-07 21:38:21 -07:00
2026-05-28 18:21:09 -07:00
2026-06-02 22:36:34 -07:00
2026-05-28 18:21:09 -07:00
2026-04-28 01:17:54 -07:00
2026-05-14 17:20:48 -07:00
2026-05-25 10:57:15 -07:00
2026-05-30 12:09:10 -07:00
2026-04-26 13:55:13 -07:00
2026-08-12 15:31:47 -07:00
2026-08-12 15:31:47 -07:00
2026-08-08 09:28:45 -07:00
2026-07-14 12:56:10 -07:00
2026-05-25 10:57:15 -07:00
2026-07-14 16:35:30 -07:00
2026-07-14 16:35:30 -07:00
2026-06-14 11:40:57 -07:00
2026-04-18 15:05:42 +08:00
2026-05-30 14:57:07 -07:00
2026-06-08 06:20:58 -07:00
2026-05-21 21:21:07 -07:00
2026-05-21 21:21:07 -07:00
2026-06-21 07:15:19 -07:00
2026-05-01 19:51:51 -07:00
2026-05-07 20:14:59 -07:00
2026-08-08 09:28:45 -07:00
2026-05-20 07:35:01 -07:00
2026-05-30 14:57:07 -07:00
2026-04-19 08:38:19 +08:00
2026-05-06 19:37:53 -07:00
2026-04-25 11:52:48 -07:00
2026-05-14 21:19:58 -07:00
2026-08-08 09:28:45 -07:00
2026-04-17 00:45:13 -07:00
2026-08-12 15:31:47 -07:00
2026-05-26 16:50:03 -07:00
2026-05-30 12:09:10 -07:00
2026-06-08 21:17:18 -07:00
2026-05-28 18:21:09 -07:00
2026-05-30 11:42:13 -07:00
2026-05-06 19:37:53 -07:00
2026-05-01 07:06:37 -07:00
2026-06-07 22:04:18 -07:00
2026-06-14 11:40:57 -07:00
2026-06-25 09:42:45 -07:00
2026-08-12 15:31:47 -07:00
2026-05-28 18:21:09 -07:00
2026-08-08 09:28:45 -07:00
2026-04-18 15:05:42 +08:00
2026-05-30 08:54:46 -07:00
2026-05-30 08:54:46 -07:00
2026-05-30 08:54:46 -07:00
2026-06-21 07:15:19 -07:00
2026-05-30 08:54:46 -07:00
2026-06-21 07:15:19 -07:00
2026-05-30 08:54:46 -07:00
2026-05-30 12:09:10 -07:00
2026-06-07 06:51:10 -07:00
2026-05-21 21:21:07 -07:00
2026-05-20 06:56:41 -07:00
2026-05-20 06:56:41 -07:00
2026-05-21 21:21:07 -07:00
2026-05-30 12:09:10 -07:00
2026-06-14 11:40:57 -07:00
2026-05-26 16:50:03 -07:00
2026-05-29 08:35:00 -07:00
2026-05-29 08:35:00 -07:00
2026-03-26 23:21:27 -06:00
2026-05-29 08:35:00 -07:00
2026-05-29 08:35:00 -07:00
2026-04-24 01:38:21 -07:00
2026-06-04 11:14:43 -07:00
2026-06-21 07:15:19 -07:00
2026-07-09 23:49:09 -07:00
2026-05-24 01:43:51 -07:00
2026-05-14 21:19:58 -07:00
2026-05-29 18:06:19 -07:00
2026-05-06 19:37:53 -07:00
2026-05-30 11:42:13 -07:00
2026-05-30 12:09:10 -07:00
2026-05-14 21:19:58 -07:00
2026-05-30 12:09:10 -07:00
2026-05-30 12:09:10 -07:00
2026-04-18 23:58:59 +08:00
2026-08-12 15:31:47 -07:00
2026-08-12 15:31:46 -07:00
2026-06-04 11:14:43 -07:00
2026-04-19 08:38:19 +08:00
2026-05-26 16:50:03 -07:00
2026-05-26 16:50:03 -07:00
2026-06-12 15:38:53 -07:00
2026-05-01 19:51:51 -07:00
2026-06-04 11:14:43 -07:00
2026-06-04 11:14:43 -07:00
2026-06-04 11:14:43 -07:00
2026-06-04 11:14:43 -07:00
2026-06-14 11:40:57 -07:00
2026-08-12 15:31:47 -07:00
2026-07-09 23:49:09 -07:00
2026-08-13 10:34:25 -07:00
2026-05-06 19:37:53 -07:00
2026-06-25 09:42:45 -07:00
2026-06-14 11:40:57 -07:00
2026-04-19 08:38:19 +08:00
2026-03-23 06:57:22 -07:00
2026-03-26 11:08:31 -07:00
2026-08-12 11:12:28 -07:00
2026-06-12 15:38:53 -07:00
2026-06-25 09:42:45 -07:00
2026-05-29 08:35:00 -07:00
2026-08-13 10:34:25 -07:00
2026-07-14 16:35:30 -07:00
2026-07-14 16:35:30 -07:00
2026-07-14 16:35:30 -07:00
2026-03-31 23:08:22 -06:00
2026-05-04 09:29:48 -07:00
2026-05-01 08:45:36 -07:00
2026-06-04 11:14:43 -07:00
2026-05-01 19:51:51 -07:00
2026-06-04 11:14:43 -07:00
2026-04-22 01:06:22 -07:00
2026-04-23 18:42:58 -07:00
2026-04-30 02:50:09 -07:00
2026-05-06 20:27:20 -07:00
2026-08-12 15:31:47 -07:00
2026-08-12 11:12:28 -07:00
2026-06-07 19:13:24 -07:00
2026-05-26 23:43:07 -07:00
2026-04-30 02:50:09 -07:00
2026-08-12 11:12:28 -07:00
2026-08-12 11:12:28 -07:00
2026-08-12 15:31:47 -07:00
2026-04-30 02:50:09 -07:00
2026-05-06 20:27:20 -07:00
2026-05-03 20:26:59 -07:00
2026-04-30 02:50:09 -07:00
2026-08-12 11:12:28 -07:00
2026-05-09 17:01:13 -07:00
2026-08-12 11:12:28 -07:00
2026-05-01 19:51:51 -07:00
2026-08-12 11:12:28 -07:00
2026-04-23 18:25:34 -07:00
2026-08-08 09:28:45 -07:00
2026-04-18 15:05:42 +08:00
2026-08-08 09:28:45 -07:00
2026-05-11 12:16:26 -07:00
2026-03-23 10:17:33 -07:00
2026-03-30 22:07:50 -06:00
2026-08-08 09:28:45 -07:00
2026-04-19 08:38:19 +08:00
2026-05-06 19:37:53 -07:00
2026-05-14 17:20:48 -07:00
2026-05-06 19:37:53 -07:00
2026-08-12 15:31:47 -07:00
2026-06-07 19:13:24 -07:00
2026-04-04 15:32:20 -07:00
2026-05-11 12:16:26 -07:00
2026-05-26 21:36:53 -07:00
2026-05-11 12:16:26 -07:00
2026-06-04 11:14:43 -07:00
2026-05-26 21:36:53 -07:00
2026-06-25 09:42:45 -07:00
2026-03-13 21:08:12 -07:00
2026-05-29 08:35:00 -07:00
2026-04-16 10:41:38 -07:00
2026-06-07 19:13:24 -07:00
2026-08-12 11:12:28 -07:00
2026-06-07 19:13:24 -07:00
2026-05-26 21:36:53 -07:00
2026-05-25 10:57:15 -07:00
2026-08-12 16:02:11 -07:00
2026-05-29 08:35:00 -07:00
2026-04-19 17:50:31 +08:00
2026-04-22 01:06:22 -07:00
2026-06-02 22:36:34 -07:00
2026-06-21 07:15:19 -07:00
2026-05-30 12:09:10 -07:00
2026-06-08 05:41:07 -07:00
2026-05-01 07:21:28 -07:00
2026-05-25 10:57:15 -07:00
2026-06-04 11:14:43 -07:00
2026-05-30 12:09:10 -07:00
2026-03-27 00:44:37 -06:00
2026-05-24 01:43:51 -07:00
2026-07-10 11:21:19 -07:00
2026-04-18 15:05:42 +08:00
2026-04-06 14:41:06 -07:00
2026-05-26 16:50:03 -07:00