fix(pregate): keep test-read markdown on its test lane

Unmarked Markdown that no test depends on is a docs/no-tests empty lane.
A file a test reads, including a fixture, still selects that test through
the import graph. Unmapped source and unreferenced test-tree files still
fail closed.

Co-authored-by: Nick <surlymo@apache.org>
This commit is contained in:
Cursor AgentandNick authored and garrytan committed 2026-10-16 23:27:22 +00:00
1 parent e8793e59e8
commit 7aabbddf3d
9 files changed
+105 -51

No files matched your search

+6 -5
View File
@@ -59,7 +59,7 @@ the byte-pinned `/ship` renders in `test/fixtures/golden/`).
|---|---|
| `runner` | the local test command with `{files}` (gstack: `bun test {files}`); without it `lanes` is `incomplete` |
| `dependencies` | glob → test files for subprocess targets, runtime reads and generated inputs the import graph cannot see; `[]` declares "no lane needed" (docs, licences) |
| `empty_lanes` | extra globs for the docs/no-tests empty lane. Planning and evidence Markdown already qualifies with no pin file (a `docs/` / `doc/` / `documentation/` tree, a `.work` plan, `plans/` / `evidence/` / `redproof/`, and markdown named plan, planning, redproof or evidence, plus README, CHANGELOG, CONTRIBUTING, LICENSE, AGENTS, CLAUDE). Source code and files under a test directory never qualify. A test that names the file still selects that test |
| `empty_lanes` | extra globs for the docs/no-tests empty lane. Any Markdown file already qualifies with no pin file when no test depends on it (the same import graph as other files: a quoted path, a `join()` of path segments, an import, or a `dependencies` entry). README, CHANGELOG, CONTRIBUTING, LICENSE, AGENTS and CLAUDE qualify even without `.md`. Source code and files under a test directory never qualify. A test that reads the file still selects that test |
| `selection` | `workflow.yml/job` → a command printing the lane's file list, for matrices a planner computes (gstack: `scripts/test-free-shards.ts --list`) |
| `warn` | check ids downgraded to warn; `secrets` and required lanes never |
| `strays.allow` | intentional root files or new top-level directories |
@@ -120,11 +120,12 @@ Two stages with separate clocks, both printed:
`test/<name>*.test.ts` convention ∪ `dependencies`), transitively through
non-test files; deleted and renamed files select the tests that named them.
A touched source file with no lane is `FAIL lanes — <file> has no test lane`,
never a pass. Planning and evidence Markdown is a docs/no-tests empty lane
never a pass. Markdown that no test depends on is a docs/no-tests empty lane
instead (`docs/no-tests: N documentation file(s)`), including when the base
branch has no `.gstack/pregate.json`. `empty_lanes` adds globs for other
documentation; it cannot clear a source file or a file under a test
directory. A test that names a Markdown file still selects that test.
branch has no `.gstack/pregate.json`. The dependency is the same import graph
as every other file, so a test that reads the Markdown (a fixture or a doc)
still selects that test. `empty_lanes` adds globs for other documentation;
it cannot clear a source file or a file under a test directory.
Lanes and platform tiers come from `.github/workflows/*.yml`
(`gstack-pregate lanes` prints them): local ones run through the runner under
`set -o pipefail`, judged on the runner's summary line (zero tests ran is a
+8 -7
View File
@@ -2743,16 +2743,17 @@ and Heavy-only files included.
**Meaning.** A touched source file maps to no test through static imports,
repo-path string literals, the `bin/<name>` → `test/<name>*.test.ts` convention
or a `dependencies` declaration. That is a failure, not a pass. Planning and
evidence Markdown is a docs/no-tests empty lane and is not this error: a `docs/`
tree, a `.work` plan, a redproof or evidence note, and README/CHANGELOG/LICENSE.
A test that names one of those files still selects that test. An unmapped
source or code file still fails.
or a `dependencies` declaration. That is a failure, not a pass. Markdown no
test depends on is a docs/no-tests empty lane and is not this error, whether
or not the name says plan, evidence or redproof. A test that reads one of
those files (a quoted path or a `join()` of path segments) still selects that
test. An unmapped source or code file still fails. A file under a test
directory is not this empty lane.
**Fix.** Add a test that names the file, or declare its lane (an explicit `[]`
for files with no behaviour) under `dependencies` in `.gstack/pregate.json`.
Other documentation goes in `empty_lanes` on the base branch. `--explain`
prints every mapping and why. For a file with no lane it also prints the import
Other non-Markdown documentation goes in `empty_lanes` on the base branch.
`--explain` prints every mapping and why. For a file with no lane it also prints the import
specifiers in test files that resolved to no repo file (for example a
`tsconfig.json` `paths` alias such as `@/lib/x` whose target is missing) and any
`extends` hop that was not followed (only relative `extends`, at most 4 hops). If
+2 -1
View File
@@ -5,8 +5,9 @@
* (zero tests ran is a failure, exit 0 or not); platform lanes this machine
* cannot run become persisted `requires-remote` obligations. A touched file
* with no lane is `FAIL lanes — <file> has no test lane`, never a pass.
* Planning and evidence Markdown is the docs/no-tests empty lane instead
* Markdown no test depends on is the docs/no-tests empty lane instead
* (lib/pregate/docs-lane.ts): it does not fail, and unmapped code still does.
* A test that reads the file still selects that test.
* Incidents: p0-4-pipe-hidden-failure (`false | tail` read as green),
* p0-4-windows-lane-unrun (a Windows-only change merged with its lane unrun).
*/
+5 -5
View File
@@ -8,9 +8,9 @@
* dependencies glob → test files for subprocess targets, runtime reads and
* generated inputs the import graph cannot see; `[]` declares
* "no lane needed" for docs and the like
* empty_lanes extra globs for the docs/no-tests empty lane (planning and
* evidence Markdown already qualifies without a pin file;
* source code and test trees never do)
* empty_lanes extra globs for the docs/no-tests empty lane (any Markdown
* file already qualifies without a pin file when no test
* depends on it; source code and test trees never do)
* selection `<workflow.yml>/<job>` → the runner-owned command that lists
* the lane's test files when the matrix is planner-computed
* warn check ids downgraded to warn (never `secrets`; `lanes` keeps
@@ -47,7 +47,7 @@ export interface PregateConfig {
$schema?: string;
runner: string | null;
dependencies: Record<string, string[]>;
/** Extra docs/no-tests globs. Built-in planning and evidence Markdown always qualifies. */
/** Extra docs/no-tests globs. Built-in Markdown always qualifies when no test depends on it. */
empty_lanes: string[];
selection: Record<string, string>;
warn: string[];
@@ -232,7 +232,7 @@ export function explainConfig(): string {
return [
'runner: the test command with a {files} placeholder; null detects bun test / vitest / jest / pytest from the repo',
'dependencies: glob → test files the import graph cannot see (subprocess targets, runtime reads, generated inputs); [] declares no lane needed',
'empty_lanes: extra globs treated as docs/no-tests when nothing else maps them. Built-in, with or without this file: a docs/doc/documentation tree, a .work plan, plans/evidence/redproof directories, markdown whose name says plan, planning, redproof or evidence, and README/CHANGELOG/CONTRIBUTING/LICENSE/AGENTS/CLAUDE. Source code and files under a test directory never qualify. A test that names the file still selects that test',
'empty_lanes: extra globs treated as docs/no-tests when nothing else maps them. Built-in, with or without this file: any Markdown file the import graph maps to no test (a quoted path, a join() of path segments, an import, or a dependencies entry), plus README/CHANGELOG/CONTRIBUTING/LICENSE/AGENTS/CLAUDE. Source code and files under a test directory never qualify. A test that reads the file still selects that test',
'selection: <workflow.yml>/<job> → the runner-owned command that lists that lane’s test files when its matrix is planner-computed',
`warn: check ids downgraded to warn; never ${NON_DOWNGRADABLE.join(', ')}; lanes keeps its required-lane and requires-remote verdicts`,
'strays.allow: root files or new top-level directories that are intentional',
+17 -21
View File
@@ -1,29 +1,23 @@
/**
* pregate docs/no-tests — planning and evidence Markdown is an empty lane.
* pregate docs/no-tests — Markdown no test depends on is an empty lane.
*
* A repo with no `.gstack/pregate.json` used to fail `lanes` on a tracked
* `.work` plan or a redproof note (`PREGATE_NO_LANE`). Those files document
* a change; they are not test subjects. `isDocsEmptyLane` is that class.
* `lane-selection` consults it only when nothing else maps the file, so a
* test that names the file still runs. Source code and anything under a
* test directory never qualify, including when `empty_lanes` globs match
* them. Unmapped code stays `PREGATE_NO_LANE`.
* `.work` plan, a redproof note, or any other Markdown file (`PREGATE_NO_LANE`).
* Those files document a change; they are not test subjects. `isDocsEmptyLane`
* is that class. `lane-selection` consults it only when the import graph
* mapped nothing (`testsFor`: imports, repo-path literals including `join()`,
* the bin convention, declarations), so a test that reads the file — a
* fixture or a doc — still runs. Source code and anything under a test
* directory never qualify, including when `empty_lanes` globs match them.
* Unmapped code stays `PREGATE_NO_LANE`.
*/
import { SOURCE_EXT } from './import-graph';
import { matchesAny } from '../ship-policy';
const TEST_TREE = /(?:^|\/)(?:test|tests|spec|__tests__)(?:\/|$)/i;
const MARKDOWN = /\.(?:md|markdown)$/i;
/** Root-level prose names, with or without `.md`. */
/** Prose names, with or without `.md` (LICENSE has no extension). */
const PROSE_NAME = /^(?:README|CHANGELOG|CONTRIBUTING|AGENTS|CLAUDE|ETHOS|LICENSE|LICENCE|COPYING|NOTICE)(?:\.md)?$/i;
/** A documentation tree. `doc` does not match `docstring`; `.work` is the plan workspace. */
const DOC_DIR = /(?:^|\/)(?:\.work|docs|doc|documentation|plans|evidence|redproof)(?:\/|$)/i;
/**
* A planning or evidence basename: `plan.md`, `implementation-plan.md`,
* `planning.md`, `redproof.md`, `red-proof.md`, `evidence.md`.
* `planet.md` and `explanation.md` do not match.
*/
const EVIDENCE_NAME = /(?:^|[^a-z0-9])(?:planning|plans|plan|red-?proofs?|evidence)(?:[^a-z0-9]|$)/i;
function isMarkdown(rel: string): boolean {
return MARKDOWN.test(rel);
@@ -44,16 +38,18 @@ function isSourceCode(rel: string): boolean {
function isBuiltinDocsPath(rel: string): boolean {
const base = rel.split('/').pop() ?? rel;
if (PROSE_NAME.test(base)) return true;
if (!isMarkdown(rel)) return false;
if (DOC_DIR.test(rel)) return true;
return EVIDENCE_NAME.test(base);
// Any Markdown file. Name and directory do not matter: `planet.md` and
// `prompts/system.md` qualify the same way a `.work` plan does. A test
// that reads the file is handled by the caller, before this check.
return isMarkdown(rel);
}
/**
* True when `rel` is documentation or evidence that needs no test lane.
* True when `rel` is documentation that needs no test lane.
* `extraGlobs` are the repo's `empty_lanes` (`.gstack/pregate.json` on the
* base branch). They add paths; they cannot classify source code or a file
* under a test directory.
* under a test directory. Callers apply this only after the import graph
* found no test.
*/
export function isDocsEmptyLane(rel: string, extraGlobs: readonly string[] = []): boolean {
const norm = rel.replace(/\\/g, '/').replace(/^\.\//, '');
+2 -1
View File
@@ -19,7 +19,7 @@ import type { Lane } from './workflows';
export interface Selection {
/** Touched files with no lane and no `[]` declaration. */
noLane: string[];
/** Documentation and evidence files with no mapped test: the docs/no-tests empty lane. */
/** Markdown (and `empty_lanes` paths) with no import-graph test: the docs/no-tests empty lane. */
emptyLane: string[];
/** Every test the touched files select, sorted. */
tests: string[];
@@ -104,6 +104,7 @@ export function selectLanes(ctx: PregateContext): Selection {
if (generator) { provenance.push(`${file}: covered by regen ${generator.id} (registry input)`); continue; }
// The pin files are validated by pregate itself (validateConfig / validateRegistry) on every run.
if (file === PREGATE_CONFIG_PATH || file === GENERATED_PATH) { provenance.push(`${file}: pin file validated by pregate`); continue; }
// Reached only when testsFor mapped nothing, so a Markdown file a test reads never lands here.
if (isDocsEmptyLane(file, ctx.config.empty_lanes ?? [])) {
emptyLane.push(file);
provenance.push(`${file}: docs/no-tests (empty lane)`);
+1 -1
View File
@@ -382,7 +382,7 @@ export const RESULT_CODES = {
PREGATE_NO_LANE: {
anchor: 'pregate-no-lane',
summary: 'a touched source file maps to no test lane through imports, repo-path literals, the bin/<name> convention or a declaration',
fix: 'add a test that names the file, or declare its lane (or an explicit empty lane) under `dependencies` in .gstack/pregate.json. Planning and evidence Markdown is already a docs/no-tests empty lane; other documentation goes in `empty_lanes`',
fix: 'add a test that names the file, or declare its lane (or an explicit empty lane) under `dependencies` in .gstack/pregate.json. Markdown no test depends on is already a docs/no-tests empty lane; other documentation goes in `empty_lanes`',
},
PREGATE_LANE_FAILED: {
anchor: 'pregate-lane-failed',
+1 -1
View File
@@ -102,7 +102,7 @@
{ "file": "lib/pregate/import-graph.ts", "movedFrom": "(new: imports ∪ repo-path literals ∪ bin convention ∪ declarations, C8)" },
{ "file": "lib/pregate/context.ts", "movedFrom": "(new: touched files, base inputs and before/after fingerprinting, C8)" },
{ "file": "lib/pregate/lane-selection.ts", "movedFrom": "(new: touched files to lanes, local vs requires-remote, C8)" },
{ "file": "lib/pregate/docs-lane.ts", "movedFrom": "(new: docs/no-tests empty lane for planning and evidence markdown, #3210)" },
{ "file": "lib/pregate/docs-lane.ts", "movedFrom": "(new: docs/no-tests empty lane for markdown no test depends on, #3210)" },
{ "file": "lib/pregate/index.ts", "movedFrom": "(new: the check registry, two-stage runner, pregate.json writer and publication verifier, C8)" },
{ "file": "lib/pregate/checks/regen.ts", "movedFrom": "(new: pre-gate check, C8)" },
{ "file": "lib/pregate/checks/secrets.ts", "movedFrom": "(new: pre-gate check, C8)" },
+63 -9
View File
@@ -64,6 +64,23 @@ describe('import graph (static imports ∪ repo-path literals ∪ bin convention
expect(testsFor(g, 'docs/README.md', BASE_PREGATE.dependencies)).toMatchObject({ tests: [], declaredNone: true });
expect(testsFor(g, 'test/util.test.js').tests).toEqual(['test/util.test.js']);
});
test('markdown a test reads maps through the same literal and join edges; an unreferenced markdown file does not', () => {
const repo = fixture();
commit(repo, {
'prompts/system.md': '# system\n',
'notes/scratch.md': '# scratch\n',
'test/fixtures/sample.md': '# sample\n',
'test/prompt-doc.test.js': "const test = require('node:test');\nconst fs = require('node:fs');\ntest('reads the prompt', () => { fs.readFileSync('prompts/system.md', 'utf8'); });\n",
'test/fixture-doc.test.js': "const test = require('node:test');\nconst fs = require('node:fs');\nconst path = require('node:path');\ntest('reads the fixture', () => { fs.readFileSync(path.join('test', 'fixtures', 'sample.md'), 'utf8'); });\n",
});
const g = buildGraph(repo.work);
expect(testsFor(g, 'prompts/system.md').tests).toEqual(['test/prompt-doc.test.js']);
expect(testsFor(g, 'prompts/system.md').provenance[0]).toContain('(literal)');
expect(testsFor(g, 'test/fixtures/sample.md').tests).toEqual(['test/fixture-doc.test.js']);
expect(testsFor(g, 'test/fixtures/sample.md').provenance[0]).toContain('(literal)');
expect(testsFor(g, 'notes/scratch.md')).toMatchObject({ tests: [], declaredNone: false });
});
});
describe('literals (pure)', () => {
@@ -103,19 +120,19 @@ describe('docs/no-tests empty lane', () => {
'notes/round-3.red-proof.md', 'docs/evidence.md', 'docs/guide.md', 'Docs/Guide.MD',
'documentation/index.md', 'plans/2026-10-16.md', 'README.md', 'lib/README.md',
'CHANGELOG.md', 'LICENSE', 'notes/implementation-plan.md', 'notes/planning.md', 'src/plan.md',
'prompts/system.md', 'src/notes.md', 'src/explanation.md', 'planet.md', 'my-docs/guide.md', 'notes/scratch.md',
];
const blocked = [
'lib/orphan.js', 'src/app.ts', 'bin/tool', 'scripts/gen.sh', 'prompts/system.md', 'src/notes.md',
'src/explanation.md', 'planet.md', 'docs/example.ts', 'test/fixtures/plan.md', 'test/plans/plan.md',
'__tests__/redproof.md', 'my-docs/guide.md',
'lib/orphan.js', 'src/app.ts', 'bin/tool', 'scripts/gen.sh', 'notes/scratch.txt',
'docs/example.ts', 'test/fixtures/plan.md', 'test/plans/plan.md', '__tests__/redproof.md',
];
test('built-in planning and evidence markdown qualifies; source, test trees and lookalikes do not', () => {
test('markdown qualifies, including unmarked names; source, test trees and non-markdown do not', () => {
for (const file of docs) expect(isDocsEmptyLane(file)).toBe(true);
for (const file of blocked) expect(isDocsEmptyLane(file)).toBe(false);
});
test('empty_lanes adds documentation globs and cannot clear source code or a test-tree file', () => {
expect(isDocsEmptyLane('notes/scratch.md', ['notes/**/*.md'])).toBe(true);
expect(isDocsEmptyLane('prompts/system.md', ['prompts/**'])).toBe(true);
expect(isDocsEmptyLane('notes/scratch.txt')).toBe(false);
expect(isDocsEmptyLane('notes/scratch.txt', ['notes/**/*.txt'])).toBe(true);
expect(isDocsEmptyLane('lib/orphan.js', ['**', 'lib/**', '**/*.js'])).toBe(false);
expect(isDocsEmptyLane('docs/example.ts', ['docs/**'])).toBe(false);
expect(isDocsEmptyLane('bin/tool', ['bin/**'])).toBe(false);
@@ -246,12 +263,49 @@ describe('gstack-pregate (bin)', () => {
expect(bad.out).toContain('docs/no-tests (empty lane): .work/round-3-plan.md');
expect(bad.out).not.toContain('.work/round-3-plan.md has no test lane');
const extra = fixture({ pregate: { empty_lanes: ['notes/**/*.md'], runner: 'node --test {files}' } });
commit(extra, { 'notes/scratch.md': '# a planning note outside the built-in names\n' });
const extra = fixture({ pregate: { empty_lanes: ['notes/**/*.txt'], runner: 'node --test {files}' } });
commit(extra, { 'notes/sketch.txt': 'a note that is not markdown\n' });
const declared = pregate(extra, ['--only', 'lanes']);
expect(declared.status).toBe(EXIT.ok);
expect(declared.err).not.toContain('PREGATE_NO_LANE');
expect(declared.out).toContain('docs/no-tests (empty lane): notes/scratch.md');
expect(declared.out).toContain('docs/no-tests (empty lane): notes/sketch.txt');
});
test('lanes: unreferenced markdown is docs/no-tests; markdown a test reads selects that test', () => {
const unread = fixture({ pregate: null });
commit(unread, {
'prompts/system.md': '# system\n\nAn unmarked prompt, not a plan or a redproof.\n',
'notes/scratch.md': '# scratch\n',
});
const idle = pregate(unread, ['--only', 'lanes', '--explain']);
expect(idle.status).toBe(EXIT.ok);
expect(idle.err).not.toContain('PREGATE_NO_LANE');
expect(idle.out).toMatch(/docs\/no-tests: 2 documentation file\(s\)/);
expect(idle.out).toContain('prompts/system.md: docs/no-tests (empty lane)');
expect(idle.out).toContain('notes/scratch.md: docs/no-tests (empty lane)');
expect(idle.out).not.toContain('has no test lane');
const named = fixture({ pregate: null });
named.git('checkout', '-q', 'main');
write(named.work, 'prompts/system.md', '# system\n');
write(named.work, 'test/fixtures/sample.md', '# sample\n');
write(named.work, 'test/prompt-doc.test.js', "const test = require('node:test');\nconst fs = require('node:fs');\ntest('reads the prompt', () => { fs.readFileSync('prompts/system.md', 'utf8'); });\n");
write(named.work, 'test/fixture-doc.test.js', "const test = require('node:test');\nconst fs = require('node:fs');\nconst path = require('node:path');\ntest('reads the fixture', () => { fs.readFileSync(path.join('test', 'fixtures', 'sample.md'), 'utf8'); });\n");
named.git('add', '-A');
named.git('commit', '-q', '-m', 'tests read markdown');
named.git('push', '-q', 'origin', 'main');
named.git('checkout', '-q', 'feature');
named.git('reset', '--hard', 'main');
commit(named, {
'prompts/system.md': '# system\n\nupdated\n',
'test/fixtures/sample.md': '# sample\n\nupdated\n',
});
const r = pregate(named, ['--only', 'lanes', '--explain']);
expect(r.status).toBe(EXIT.ok);
expect(r.out).toContain('test/prompt-doc.test.js ← prompts/system.md');
expect(r.out).toContain('test/fixture-doc.test.js ← test/fixtures/sample.md');
expect(r.err).not.toContain('PREGATE_NO_LANE');
expect(r.out).not.toContain('docs/no-tests');
});
test('lanes: a test that names a markdown file still runs that test', () => {