Let a repository name its own docs-only paths in its ship policy (GST-1974) (#4185)

* Add failing tests for repo-declared docs-only paths (GST-1974)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Let a repository name its own docs-only paths in its base ship policy (GST-1974)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Catch relative directory readers, missing docs tests and readers the base gained in the docs-only class

Review-Finding: review/specialist-batched#1 reader=security
Review-Finding: review/red-team#1 reader=red-team
Review-Finding: review/red-team#2 reader=red-team
Review-Finding: voices/adversarial-native#1 reader=adversarial-native
Review-Finding: voices/adversarial-native#3 reader=adversarial-native
Review-Finding: voices/adversarial-native#5 reader=adversarial-native
Review-Finding: voices/outside-voice#1 reader=outside-voice
Review-Finding: voices/outside-voice#2 reader=outside-voice
Review-Finding: voices/outside-voice#3 reader=outside-voice
Review-Finding: voices/outside-voice#4 reader=outside-voice
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Say in the changelog that a missing docs test or a new base reader keeps the full checks

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Describe the new directory-reader forms, the missing docs test refusal and land's base-reader check

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

docs/pregate.md

@@ -1445,12 +1445,15 @@ pregate, `gstack-ci-carry`, /ship's lanes and land. A path is docs only when eve
 these holds:
 
 - It is a plain file (mode 100644). Executables, symlinks, submodules and type changes refuse.
-- It is on the allowlist, matched case-sensitively: root `README.md`, `docs/**/*.md`, `changelog.d/*.md`.
+- It is on the allowlist, matched case-sensitively: root `README.md`, `docs/**/*.md`, `changelog.d/*.md`, plus the
+  repository's own globs from the base branch's `docs_only.allow` ([below](#repo-docs-paths)).
 - review-depth's `classifyPath` calls it docs, so agent prose, skill directories and policy docs stay code.
 - It matches no denylist entry. The denylist covers agent-instruction names matched case-insensitively
   (`SKILL.md`, `AGENTS.md`, `CLAUDE.md`, `ETHOS.md`, `CREW.md`, `captain.md`, `factory.md`, `pregate.md`,
-  `unattended.md` and others), `*-policy.md`, preferences, `sections/`, `templates/`, `*.tmpl`, pinned
-  results (`docs/evals/`, `docs/inventory/`), preregistrations, and every output in `.gstack/generated.json`.
+  `unattended.md` and others), `*-policy.md`, preferences, `sections/`, `templates/`, `skills/`, `prompts/` and
+  `*prompt.md`, `*.tmpl`, hidden directories (`.github/`, `.gstack/`), pinned results (any `evals/` or `inventory/`
+  directory), preregistrations, every output in `.gstack/generated.json`, and the repository's `docs_only.deny`.
+  The list lives in `lib/ship-policy-docs-only.ts`, and no repository setting can override it.
 - No tracked file outside human prose names its path. A script, test, workflow or skill that reads a doc pins it.
   `CHANGELOG.md`, `TODOS.md` and other allowlisted docs don't pin. The match is a substring match, so it errs
   toward pinning.
@@ -1485,9 +1488,57 @@ also calls the head docs-only. Without that check, the run is never evidence (`a
 
 **Land.** `land.docs_only_skip` in `.gstack/ship-policy.json` names the `merged_checks` commands land
 skips for a docs-only pin, such as the test list, keeping typecheck and the freshness check. Land keeps every
-check when a non-docs file on the base tip names a changed doc, because the pin merges onto that tip. A gstack
-release older than this change refuses the key as unknown, so main's policy adds it only with `min_gstack_version`
-raised to the first release that reads it; until then land runs every merged check on a docs-only pin.
+check when a non-docs file on the base tip names a changed doc, or reads the directory of a repo-admitted one
+([below](#repo-docs-paths)) where the pin had no such reader, because the pin merges onto that tip. Releases up
+to 1.98.0.0 refuse the key as unknown, so `validate` refuses it until `min_gstack_version` reaches
+`LAND_DOCS_ONLY_SKIP_KEY_VERSION` (lib/ship-policy-keys.ts). To turn it on (GST-1968), use one pull request, after a
+stable release carries the key: add `docs_only_skip`, raise `min_gstack_version` to that release, and check that
+`gstack-ship-policy validate` prints `POLICY_VALID` and a `MIN_READER` no newer than the stable release. Until
+then land runs every merged check on a docs-only pin. gstack's own policy leaves the key off.
+
+<a id="repo-docs-paths"></a>
+### Your repository's own docs paths (`docs_only`)
+
+A repository whose human docs live outside `docs/` names them in `.gstack/ship-policy.json`:
+
+```json
+"docs_only": { "allow": ["wiki/board/**.md"], "deny": ["wiki/board/drafts/**"], "tests": ["tests/wiki.test.ts"] }
+```
+
+Every check above still applies to a repo-allowed path: its mode, the built-in deny list, review-depth's
+`classifyPath` (a top-level directory holding a `SKILL.md` stays agent prose), generated outputs, renames on both sides, and
+the reference scan (a script or test naming the path pins it; a repo-allowed doc naming another does not).
+
+- **Base only.** The classifier reads the block with git from the base branch tip (CI: the merge commit's first
+  parent; pregate and `gstack-ci-carry`: `origin/<base>`; land: the tip it merges onto), while the diff still starts
+  at the merge base, so an allow the base revoked after a branch forked no longer applies. It never reads the head or
+  the working tree, and a pull request that edits `.gstack/ship-policy.json` is never docs-only,
+  so a branch can't allowlist itself.
+- **Narrow patterns.** Each `allow` entry ends in `.md` (not `.mdx`, which can import code), starts with a
+  literal directory or file name, uses only `*` and `**` as wildcards, and passes through no hidden directory and no `skills`, `templates`,
+  `sections`, `specialists`, `prompts`, `test`, `tests`, `spec`, `fixtures`, `lib`, `bin`, `scripts`, `src`,
+  `vendor`, `dist` or `build` segment. `**`, `*`, `*.md`, `**/*.md` and a pattern naming an agent file
+  (`wiki/SKILL.md`, `wiki/**/templates/**.md`) are `POLICY_INVALID` when the policy is read, with an example of a
+  narrow pattern. A path the pattern admits that still passes through one of those directories (`wiki/lib/x.md`)
+  refuses too. `deny` entries only remove paths, built-in ones included.
+- **Fail closed.** A base policy that is not valid JSON, or a `docs_only` block that fails those rules, makes every
+  change not docs-only, with the error as the refusal. An unknown key inside `docs_only` is a warning to
+  `gstack-ship-policy validate`, and the classifier then ignores `allow` (keeping `deny`), so a gstack that
+  doesn't understand a newer key never widens on it.
+- **Readers by directory.** A script that lists a directory reads every post in it, which a literal-path scan can't
+  see. So for a path only `allow` admits, any non-prose tracked file that names one of its directories (`wiki/`,
+  `'wiki'`, `"wiki/board"`, `'./wiki'`, `'../wiki'`, `root + '/wiki'`, `` `${root}/wiki` ``) refuses the change unless it is tied to `tests`: it is one of them, names one, or one
+  names it (with or without its extension). `tests` lists up to 16 test files (`*.test.ts` and the like, never a glob);
+  the docs lane runs them, in pregate and in CI's `tests` output, whenever a repo-admitted path changed. A listed test missing from the classified tree refuses
+  the change, so the docs lane can't pass without it. A prose file
+  that names a directory only describes it; one that names a path still pins that path.
+
+**Turning it on.** Releases up to 1.98.0.0 refuse `docs_only` as an unknown key, and `validate` refuses the block
+until `min_gstack_version` reaches `DOCS_ONLY_KEY_VERSION`. After a stable release carries it, add the block and
+raise `min_gstack_version` to that release in the same pull request; that pull request edits the policy, so it runs
+the full suites. Pushes to the base branch always run the full suites. A repository whose CI runs an older pinned
+gstack, or has no `docs-scope` job, gets the class in pregate, /ship and land only: bump the pin first, then add the
+`docs-scope` job from [CI](#docs-only) (never gating its own docs checks), then add the block.
 
 <a id="flaky-exception"></a>
 ## Owner-granted flaky exception

docs/ship-policy.md

@@ -28,9 +28,14 @@ code from [troubleshooting.md](troubleshooting.md#queue-aware-ship-policy-restam
 
 `gstack-ship-policy init` writes every key at its default; `init --explain`
 prints this table. JSON carries no comments, so examples live in
-`.gstack/ship-policy.example.json`. Unknown keys are errors, and there is no
-extension mechanism: a key from a newer gstack, or a misspelled one, fails
-closed (`POLICY_INVALID`) instead of being ignored. Every key has an
+`.gstack/ship-policy.example.json`. Unknown top-level keys are errors: a key
+from a newer gstack, or a misspelled one, fails closed (`POLICY_INVALID`)
+instead of being ignored. The one extension point is a key inside `land` or
+`docs_only`: this gstack and later ones ignore it with a `WARN` line
+(`validate --json` lists it under `warnings`), so a later release can add a
+nested key without breaking readers that carry this rule. A nested key that
+must be enforced still names its own introducing version, as
+`land.docs_only_skip` does, so `validate` asks for `min_gstack_version`. Every key has an
 introducing version (`init --explain` and `validate` print it). A key newer
 than 1.91.98.0 set to a non-default value requires `min_gstack_version` at
 least at its introducing version, and a gstack older than `min_gstack_version`
@@ -75,7 +80,8 @@ branch that changes the policy.
 | `docs` | `{ caps: [{ path, max_bytes }] }` or `null` | `null` | Documentation files with a byte cap. `/ship` Step 14.5 runs `gstack-ship-receipt docs-caps --base <base>` (the caps come from `origin/<base>`'s policy, never the branch's copy; `--policy <file>` is for a repository bootstrapping its policy) before the verification gate and prints one `DOCS_CAP: <path> <bytes>/<max> ok\|over` line per cap; any `over` (an absent file counts) exits 1 and blocks the ship until the file is trimmed or the cap raised on the base branch (`DOCS_AUDIT_BLOCKED`). At most 64 caps, each path contained. See [the ship receipt](#ship-receipt). Needs `min_gstack_version`. |
 | `train` | `{ gate_workflow, free_workflow, full_gate_check, record_artifact, max_size, fill_hours, screen_selection, trial_reservation_usd, land? }`, `{ gates, land, max_size, fill_hours }` or `null` | `null` | Turns on [merge trains](#run-a-merge-train), read from the base revision only. Every field is required and an unknown field is `POLICY_INVALID`: `max_size` 1 to 6, `fill_hours` above 0 and at most 24, `screen_selection` `gstack-paid` \| `none`, `trial_reservation_usd` above 0 and at most 100. A block with `gates` ([any repository](#train-gates)) takes `max_size` 1 to 5 and needs `min_gstack_version` at least `TRAIN_GATES_KEY_VERSION`. `null` keeps the per-PR queue exactly as it is. Needs `min_gstack_version`. |
 | `merge` | `train` \| `direct` | `train` | `direct` lands each READY PR at its own green head with [`land --pr <n> --pin <sha>`](#land-direct): no train and no stamp commit, and `train build` refuses. A PR then leaves VERSION, the `package.json` version and the CHANGELOG at its merge base, and the release sets them. Needs `changelog_fragments`, and the stamping `land --pr <n>` refuses under it. `train` keeps merge trains. Read from the base revision only. Needs `min_gstack_version`. |
-| `land` | `{ merged_checks, docs_only_skip? }` or `null` | `null` | Under `merge: direct`, the shell commands (1 to 16, one line of at most 512 characters each, 15 minutes each) [`land --pin`](#land-direct) runs on the pin merged onto the base tip when the base moved past the pin's merge base; it lands only when every one exits 0. They are repo commands, so land runs them only with `--allow-repo-commands`, never under `--dry-run`. `null` lands a stale base untested with a `STALE_BASE: warn` line. `docs_only_skip` lists `merged_checks` commands to skip for a docs-only pin ([pregate.md](pregate.md#docs-only)). Read from the base revision only. Needs `min_gstack_version`. |
+| `land` | `{ merged_checks, docs_only_skip? }` or `null` | `null` | Under `merge: direct`, the shell commands (1 to 16, one line of at most 512 characters each, 15 minutes each) [`land --pin`](#land-direct) runs on the pin merged onto the base tip when the base moved past the pin's merge base; it lands only when every one exits 0. They are repo commands, so land runs them only with `--allow-repo-commands`, never under `--dry-run`. `null` lands a stale base untested with a `STALE_BASE: warn` line. `docs_only_skip` lists `merged_checks` commands to skip for a docs-only pin ([pregate.md](pregate.md#docs-only)) and needs `min_gstack_version` at its own release (`LAND_DOCS_ONLY_SKIP_KEY_VERSION`), because older releases refuse it. Any other key under `land` is a warning, ignored. Read from the base revision only. Needs `min_gstack_version`. |
+| `docs_only` | `{ allow, deny?, tests? }` or `null` | `null` | Human-docs paths beyond `README.md`, `docs/**.md` and `changelog.d/*.md` for the [docs-only class](pregate.md#repo-docs-paths) (`{ "allow": ["wiki/board/**.md"] }`). `allow` (1 to 16) takes narrow Markdown globs: each ends in `.md`, starts with a literal directory or file name, and passes through no hidden, code, test or agent directory; `**`, `*`, `*.md` and patterns naming agent files are `POLICY_INVALID` with an example of a narrow one. `deny` (up to 64 globs) removes paths, built-in ones included. `tests` (up to 16 test files) are the repository's checks for those directories: the docs lane runs them, and a non-prose file that reads an allowed directory refuses the class unless it is tied to one; a listed test missing from the branch refuses it too. The built-in deny list always wins. Any other key under `docs_only` is a warning, and the classifier then ignores `allow`. Read from the base revision only; a pull request that edits this file is never docs-only. Needs `min_gstack_version`. |
 | `release_gate` | `{ cadence_hours, gate_workflow, branch, stable_ref, check_name, on_red, bisect_spend_cap_usd }` or `null` | `null` | One full paid gate on main every `cadence_hours` (8 to 168) instead of the per-push main backstop, feeding the stable release; see [release_gate](#release-gate). Every field is required: `on_red` is `hold-release`, `bisect_spend_cap_usd` 0 to 500 (0 skips the bisect). `null` keeps the per-push backstop. Needs `min_gstack_version`. |
 | `bookkeeping` | glob list | `[]` | <a id="bookkeeping"></a>Root-anchored globs of bookkeeping tables a verify or ratchet step rewrites (size baselines, serial lists, limits), each ending in `.tsv`, `.csv` or `.txt` and never under `.gstack/` or `.github/` (`["scripts/*-baseline.tsv"]`). A matching file changed at most 20 lines after review binds, first touched or not, takes finish's [one scoped reader](review-policy.md#bounded-post-review-edits) instead of Steps 5–11.5; the reader still reads every changed row. Read from the base revision only. Needs `min_gstack_version`. |
 | `mechanical_retake` | `[{ path, template, fields, regen }]` | `[]` | A version or rules constant a branch may have to retake after a base merge, next to `bookkeeping`'s bounded kind: `path` is one literal file, `template` the whole line with one `{n}` (`const RULE_NUMBERS := {n}`), `fields` the `{ glob, field }` fixture fields that carry the same value, and `regen` the [`.gstack/generated.json`](pregate.md#generated) entry ids whose command regenerates its goldens. A post-review edit finish proves to be only that retake routes as `mechanical=`, with no reader and no review round; see [mechanical retakes](#mechanical-retake). At most 8 constants and 16 fields or ids each; no glob, `..`, `.gstack/` or `.github/` path. Read from the base revision only. Needs `min_gstack_version`. |