mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-12 19:44:48 +01:00
test(W7h): retire the paid SKILL.md-summary cases in skill-e2e-plan
W7h retirement (t14 F6-F8): office-hours-spec-review, plan-ceo-review-benefits
and codex-offered-{office-hours,ceo-review,design-review,eng-review}. Each
asked a model to read a generated SKILL.md, write a summary, and grepped the
summary for facts the generated file states directly.
Registration cleanup in the same commit: E2E_TOUCHFILES, E2E_TIERS (all six
gate) and E2E_KINDS (rule) in test/helpers/touchfiles-data.ts; six
scripts/paid-test-durations.json case-shard entries; no
test/helpers/periodic-exclude-data.ts entries existed. The fast PR profile
(scripts/test-pr-profile.ts) drops plan-ceo-review-benefits and
office-hours-spec-review from PR_PROFILE_CASE_IDS and PR_PROFILE_FILES. Tests
that named the retired ids as live examples move to existing ones:
paid-free-boundary and paid-pr-profile (a CEO template change now plans
auq-format-gate plus the CEO judge; the case-shard fixture uses
plan-review-report), test-pr-profile, touchfiles (CEO selection count
21 -> 19), the touchfile-moved-code w5 golden (retired ids removed), the
behavior-lane case-shard census (78 -> 72) in cookie-validation-phases, and
the skill-e2e-plan timeout list and full-gate PR floor (78,425 -> 72,755 s)
in paid-retry-supervision. e2e-tier-alignment, eval-kinds,
eval-detach-timeout-floor and paid-shards stay green.
Coverage consequence: per the Eng Retirements item, plan-ceo-review-benefits
was the only gate-tier CEO E2E in the fast PR profile, so CEO invariants are
treated as periodic-only under the W7c gate (every CEO conversion in this
branch keeps one free check per invariant); the gate judge
'plan-ceo-review/SKILL.md modes' still runs for CEO prompt changes. An
office-hours prompt change now has no fast PR E2E and is reported as
deferred to broad coverage rather than run.
Retirement cards:
- office-hours-spec-review: detects the Spec Review Loop's stated dimensions,
dispatch tool, iteration cap and metrics. Replacement: gen-skill-docs.test.ts
'SPEC_REVIEW_LOOP resolver' (round-3 cap, prepare/check/finalize commands,
verdict and round-file wiring) and test/office-hours-review.test.ts (verdict
parsing). Control: changing the round limit in
scripts/resolvers/spec-review.ts turns gen-skill-docs red.
- plan-ceo-review-benefits: detects the CEO /office-hours prerequisite and
mid-session offer. Replacement: gen-skill-docs.test.ts 'plan-ceo-review
offers /office-hours when no design doc exists and detects a lost user
mid-session' (previous commit). Control: 4/4 red mutations there.
- codex-offered-* (x4): detects the `command -v codex` probe, a fallback and
optionality. Replacement: gen-skill-docs.test.ts 'Codex outside voice is
offered behind an executed availability probe' (probe + not_installed
fallback for all four skills) and skill-validation.test.ts 'plan reviews run
the Codex outside voice default-on'. Control: 4/4 red there.
- non_test_callers: none; history: c118e24 v1.64.1.0 (#2572).
- validation: the registration tests named above.
Controls log: .context/mutation-skill-e2e-plan-retirements.md.
This commit is contained in:
1 parent
51f1d947ee
commit
fc9049a8b5
11 files changed
+16
-297
No files matched your search
@@ -40,12 +40,6 @@
|
||||
"test/skill-e2e-plan-prosons.test.ts": 1000,
|
||||
"test/skill-e2e-plan-tune.test.ts": 63925,
|
||||
"test/skill-e2e-plan.test.ts": 251000,
|
||||
"test/skill-e2e-plan.test.ts#codex-offered-ceo-review": 49880,
|
||||
"test/skill-e2e-plan.test.ts#codex-offered-design-review": 54895,
|
||||
"test/skill-e2e-plan.test.ts#codex-offered-eng-review": 59818,
|
||||
"test/skill-e2e-plan.test.ts#codex-offered-office-hours": 50442,
|
||||
"test/skill-e2e-plan.test.ts#office-hours-spec-review": 33866,
|
||||
"test/skill-e2e-plan.test.ts#plan-ceo-review-benefits": 48811,
|
||||
"test/skill-e2e-plan.test.ts#plan-review-report": 76406,
|
||||
"test/skill-e2e-qa-bugs.test.ts": 1000,
|
||||
"test/skill-e2e-qa-callers.test.ts": 737164,
|
||||
|
||||
@@ -12,8 +12,8 @@ export const PR_PROFILE_CASE_IDS = [
|
||||
'qa-functional-cli-fix', 'qa-functional-webhook-fix',
|
||||
'review-exploratory-small-cli', 'ship-exploratory-small-cli', 'ship-exploratory-unavailable',
|
||||
'ship-exploratory-plan-checks', 'ship-exploratory-late-input',
|
||||
'plan-ceo-review-benefits', 'plan-eng-coverage-audit', 'plan-review-report',
|
||||
'auq-format-gate', 'plan-design-review-no-ui-scope', 'office-hours-spec-review',
|
||||
'plan-eng-coverage-audit', 'plan-review-report',
|
||||
'auq-format-gate', 'plan-design-review-no-ui-scope',
|
||||
'tpa-present', 'tpa-absent-linux',
|
||||
'ship-local-workflow', 'ship-coverage-audit', 'docsync-spawned',
|
||||
'ship-docsync-completion', 'ship-docsync-current', 'ship-docsync-failure', 'ship-docsync-store',
|
||||
@@ -41,7 +41,7 @@ export const PR_PROFILE_FILES: Record<string, readonly string[]> = {
|
||||
'test/skill-e2e-review.test.ts': ['review-sql-injection'],
|
||||
'test/skill-e2e-coverage-audit.test.ts': ['review-coverage-audit', 'plan-eng-coverage-audit'],
|
||||
'test/skill-e2e-test-value.test.ts': ['ship-coverage-value', 'review-test-value', 'test-audit-report-only'],
|
||||
'test/skill-e2e-plan.test.ts': ['plan-ceo-review-benefits', 'plan-review-report', 'office-hours-spec-review'],
|
||||
'test/skill-e2e-plan.test.ts': ['plan-review-report'],
|
||||
'test/skill-e2e-ask-user-question-format-compliance.test.ts': ['auq-format-gate'],
|
||||
'test/skill-e2e-design.test.ts': ['plan-design-review-no-ui-scope'],
|
||||
'test/skill-e2e-third-party-actions.test.ts': ['tpa-present', 'tpa-absent-linux'],
|
||||
|
||||
Reference in new issue
Block a user