fix: retire decided A/B experiments; vendor the pre-cut fixture; ban raw-SHA fixtures

Three one-shot decision experiments kept re-running weekly as N=1
stochastic comparisons — flaky by construction with near-zero remaining
information: skill-e2e-auq-repetition-cut-ab (its own header: gate "passed
pre-landing, approved 2026-08-25"), skill-e2e-preamble-script-ab ("demoted
post-Phase-3"), and opus-47's fanout arm-vs-arm (parA >= parB across two
SINGLE stochastic runs — a coin flip). Deleted, with their selection keys;
the SDK overlay-harness stays as the maintained instrument for the next
experiment, and opus-47 keeps its routing-precision cases.

verboseSkill() now reads the VENDORED test/fixtures/auq-pre-cut-...-SKILL.md
instead of `git show ab66193e^:...` — a branch-local ref that dies on
branch prune and already failed on shallow clones. New free tripwire
(test/git-ref-fixture-tripwire.test.ts) bans the raw-SHA fixture class
outright: quoted SHA:path rev-specs and gitRef-style hex defaults in the
test trees fail the suite with the vendor-instead instruction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-31 05:01:46 +00:00
co-authored by Claude Fable 5
parent 99b5fa2e46
commit d6df2af7dd
7 changed files with 88 additions and 340 deletions
+11 -3
View File
@@ -281,9 +281,17 @@ export function carvedSkill(): { skillMd: string; sectionsFrom: string | null }
};
}
/** Read the pre-carve verbose monolith plan-ceo SKILL.md from git. */
export function verboseSkill(gitRef = 'ab66193e^'): string {
return execGit(['show', `${gitRef}:plan-ceo-review/SKILL.md`]);
/** Read the pre-carve verbose monolith plan-ceo SKILL.md.
* VENDORED fixture (v1.75 precedent), not a git ref: the old default
* `git show ab66193e^:...` pinned a BRANCH-LOCAL commit — it dies the day
* that branch is pruned and already fails on shallow clones. The fixture
* is the frozen pre-cut render; test/git-ref-fixture-tripwire.test.ts
* keeps this class from coming back. */
export function verboseSkill(): string {
return fs.readFileSync(
path.join(ROOT, 'test', 'fixtures', 'auq-pre-cut-plan-ceo-review-SKILL.md'),
'utf-8',
);
}
function execGit(args: string[]): string {