test: derive paid touchfiles from each eval's static closure (E)

touchfiles.test.ts now checks, per key, that the paid file's static
test/helpers and test/fixtures closure (plus fixture paths it names in string
literals) is covered, and names the file, path, chain and key to fix when it is
not. Free *.test.ts files are no longer touchfiles, so editing a free replay
test stops selecting paid evals: 950 entries removed, 653 real closure paths
added. The hand-copied inventories go: periodic-fixture-selection,
fake-impeccable-touchfiles and 45 per-file selection examples. Selection for
the sample edits (plan-eng-review template, claude-pty-runner,
plan-count-fixture, gstack-config) loses no case under either profile.
CONTRIBUTING documents the rule and its lower bound.
This commit is contained in:
garrytan committed 2026-09-29 07:00:28 +00:00
1 parent 689ef3094b
commit 7cbb5e04e4
53 files changed
+627 -1873

No files matched your search

-9
View File
@@ -10,8 +10,6 @@ import { runRecordedOfficeHoursAttempt, OFFICE_HOURS_BUN_GRACE_MS } from './help
import { buildHermeticEnv } from './helpers/hermetic-env';
import { CAPTURE_MS } from './helpers/eval-budgets';
import { redactFindingSpans } from '../lib/redact-engine';
import { E2E_TOUCHFILES } from './helpers/touchfiles-data';
// Assemble synthetic credentials at runtime, as in gate-secret-scan.test.ts.
const CREDENTIAL = ['ghp_', 'aB3dE5fG7hI9jK1lM3nO5pQ7rS9tU1vW3xY5'].join('');
const source = fs.readFileSync(path.join(import.meta.dir, 'skill-e2e-setup-gbrain-path4-local-pglite.test.ts'), 'utf8');
@@ -133,13 +131,6 @@ test('Path4 public diagnostics exclude private fields and redact output, tools a
expect(serialized).not.toContain('apiKeySource');
expect(row.transcript).toHaveLength(3);
});
test('Path4 caller controls select only their registered paid owner', () => {
const owners = Object.entries(E2E_TOUCHFILES).filter(([, paths]) => paths.includes('test/setup-gbrain-path4-caller.test.ts')).map(([name]) => name);
expect(owners).toEqual(['setup-gbrain-path4-local-pglite']);
});
test('Path4 redacted credential URLs cannot erase success or mask a failed assertion', async () => {
// Text redaction can consume JSON punctuation after an unspaced URL host.
// These synthetic credentials exercise the real redactor and actual callback.