Never let a later docs-audit line override an earlier blocked result (GST-2349) (#4434)

* Never let a later docs-audit line override an earlier blocked line (GST-2349)

A blocked documentation-audit result line followed by an unfenced "current"
JSON line for the same audit as the very last line was accepted as current,
and a last "current" line whose only fault was its format was repaired back to
current. The blocked verdict now carries: a last line that drops an earlier
blocked status or blocker for the audit is never accepted and gets the
one-way re-ask, whose reply must stay blocked and keep those blockers.

Step 2 of the docs audit section no longer says it parses only the last
line, and the changelog entry now claims only what the parser checks.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Leave the landed changelog entry alone and correct its claim in a new entry (GST-2349)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Split a run-on sentence in the changelog entry for the docs-audit fix

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
Garry Tanandcapy-ai[bot] authored and GitHub committed 2026-11-17 11:12:52 +00:00
1 parent 67f1f82fbb
commit eb54692427
7 files changed
+72 -10

No files matched your search

@@ -0,0 +1,4 @@
---
important: blocker
---
- **/ship's documentation audit no longer accepts a passing result when an earlier line from the same audit said "blocked".** A blocked result line followed by an unfenced "current" JSON line for the same audit as the very last line was accepted as current, and a last "current" line whose only fault was its format could be repaired back to current. Now a last line that drops an earlier blocked status or blocker is never accepted. It gets the free one-way re-ask, whose reply must stay blocked and keep those blockers. This corrects the previous entry's claim. A later JSON line, fenced or not, and a re-ask reply can no longer turn a blocked audit into a passing one. A blocker written only in prose after a passing line still depends on the re-ask counting it, which the parser can't check (GST-2349).
+4 -1
View File
@@ -7294,7 +7294,10 @@ the audit id (`text follows the JSON result line`) gets the same free re-ask for
the whole line, but only when every such line is free of content problems (a
content problem counts as an attempt). A blocked status and its blockers on any
such line must survive the reply, which may add blockers; nothing else from the
earlier line is kept or accepted. Otherwise run Step 14.5's one
earlier line is kept or accepted. The same holds when the last line is a JSON
result that drops an earlier blocked status or blocker for the audit (`the last
JSON line drops the blocked result an earlier line gave for this audit`): it is
never accepted, and its re-ask must stay blocked. Otherwise run Step 14.5's one
remaining re-audit with the printed `RETRY_NOTE:` text appended to its prompt,
word for word; it quotes the exact keys. With no attempt left, Step 14.5 stops
and reports this code and the keys it names
+9 -5
View File
@@ -11,6 +11,8 @@
* repair re-asks for the whole line (`whole`), and only one way: every JSON line carrying the audit id must be content-clean, or
* the failure counts, and a blocked status or blocker on any of them must survive the reply (round-2 review of #4285: a blocked
* line, then a fenced "current" example, then a "current" reply). Nothing else carries over; the reply is judged on its own.
* The same carry holds when the last line parses: a last line that drops an earlier blocked status or blocker for this audit is
* never accepted, and gets that one-way re-ask (a blocked line, then an unfenced "current" line last).
* A failure of shape alone (the line is one object; only keys, types or the section are wrong) is `repairable`: the parent
* re-asks for the line once (docsRepairPrompt) without spending an attempt, and checkDocsRepair accepts the reply only when
* every value the first line already gave correctly is unchanged (GBEV-3: a missing section, then a misnamed key set to "x").
@@ -36,14 +38,14 @@ export function checkDocsResult(output: string, auditId: string, isFile: IsFile
const lines = output.replace(/\r\n/g, '\n').split('\n').map(l => l.trim()).filter(Boolean);
const object = (l: string) => { try { const x: unknown = JSON.parse(l); return x && typeof x === 'object' && !Array.isArray(x) ? x : undefined; } catch { return undefined; } };
const parsed = object(lines.at(-1) ?? '');
const earlier = lines.slice(0, -1).filter(l => Object.entries(object(l) ?? {}).some(([key, value]) => key.trim() === 'audit_id' && value === auditId)).map(l => checkDocsResult(l, auditId, isFile));
const said = earlier.map(c => (c.ok ? c.result : c.kept) as Record<string, unknown>);
const carried = [...new Set(said.flatMap(s => Array.isArray(s.blockers) ? s.blockers as string[] : []))];
const carry = carried.length || said.some(s => s.status === 'blocked') ? { status: 'blocked', blockers: carried } : {};
if (!parsed) {
const earlier = lines.slice(0, -1).filter(l => Object.entries(object(l) ?? {}).some(([key, value]) => key.trim() === 'audit_id' && value === auditId)).map(l => checkDocsResult(l, auditId, isFile));
if (!earlier.length) return { ok: false, problems: ['the last nonempty line is not one JSON object'], trimmed: [], repairable: false, kept: {} };
const content = earlier.flatMap(c => c.ok || c.repairable ? [] : c.problems);
const said = earlier.map(c => (c.ok ? c.result : c.kept) as Record<string, unknown>);
const blockers = [...new Set(said.flatMap(s => Array.isArray(s.blockers) ? s.blockers as string[] : []))];
const kept = blockers.length || said.some(s => s.status === 'blocked') ? { status: 'blocked', blockers } : {};
return { ok: false, problems: ['text follows the JSON result line', ...content], trimmed: [], repairable: !content.length, kept, whole: true };
return { ok: false, problems: ['text follows the JSON result line', ...content], trimmed: [], repairable: !content.length, kept: carry, whole: true };
}
const problems: string[] = [];
const trimmed: string[] = [];
@@ -89,6 +91,8 @@ export function checkDocsResult(output: string, auditId: string, isFile: IsFile
}
if (problems.length > invariants) content = true;
const kept = Object.fromEntries(DOCS_RESULT_KEYS.filter(key => Object.hasOwn(v, key) && !bad.has(key)).map(key => [key, v[key]]));
const keeps = v.status === 'blocked' && Array.isArray(v.blockers) && carried.every(b => (v.blockers as unknown[]).includes(b));
if (carry.status === 'blocked' && !keeps && !(problems.length && content)) return { ok: false, problems: ['the last JSON line drops the blocked result an earlier line gave for this audit', ...problems], trimmed, repairable: true, kept: carry, whole: true };
if (problems.length) return { ok: false, problems, trimmed, repairable: !content, kept };
return { ok: true, result: v as unknown as DocsResult, trimmed };
}
+2 -2
View File
@@ -71,11 +71,11 @@ Reentry never resets the count or authorizes a launch.
use recovery before another writer.
2. **Check output.** Save the output, then run
`~/.claude/skills/gstack/bin/gstack-docs-candidate result <output> --audit-id <audit-id> --out-dir <record dir>`.
It parses only the LAST nonempty line, trims whitespace around keys and requires
It takes the LAST nonempty line as the result, trims whitespace around keys and requires
every field/type, exact audit id, schema and status invariant. `DOCS_RESULT: ok`
saves `<audit-id>-result.json` and the section unchanged as `<audit-id>-documentation.md`;
`DOCS_RESULT_INVALID` names each bad key and prints a `RETRY_NOTE:`, or `FORMAT_REPAIR:` when
only the line's shape is wrong or text follows it (a blocked verdict carries over): then, once and using no attempt, send `gstack-docs-candidate
only the line's shape is wrong, text follows it, or it drops an earlier blocked line (blocked carries over): then, once and using no attempt, send `gstack-docs-candidate
repair-prompt <output> --audit-id <audit-id>`'s output to one fresh agent with no tools (never the writer child),
and rerun `result` on its reply adding `--repair-of <output>`; a refusal prints `RETRY_NOTE:`. Also require
the actual spawned marker above, as echoed in the child output; state files are
+2 -2
View File
@@ -69,11 +69,11 @@ Reentry never resets the count or authorizes a launch.
use recovery before another writer.
2. **Check output.** Save the output, then run
`~/.claude/skills/gstack/bin/gstack-docs-candidate result <output> --audit-id <audit-id> --out-dir <record dir>`.
It parses only the LAST nonempty line, trims whitespace around keys and requires
It takes the LAST nonempty line as the result, trims whitespace around keys and requires
every field/type, exact audit id, schema and status invariant. `DOCS_RESULT: ok`
saves `<audit-id>-result.json` and the section unchanged as `<audit-id>-documentation.md`;
`DOCS_RESULT_INVALID` names each bad key and prints a `RETRY_NOTE:`, or `FORMAT_REPAIR:` when
only the line's shape is wrong or text follows it (a blocked verdict carries over): then, once and using no attempt, send `gstack-docs-candidate
only the line's shape is wrong, text follows it, or it drops an earlier blocked line (blocked carries over): then, once and using no attempt, send `gstack-docs-candidate
repair-prompt <output> --audit-id <audit-id>`'s output to one fresh agent with no tools (never the writer child),
and rerun `result` on its reply adding `--repair-of <output>`; a refusal prints `RETRY_NOTE:`. Also require
the actual spawned marker above, as echoed in the child output; state files are
+47
View File
@@ -499,3 +499,50 @@ describe('the whole-line re-ask is one-way: a blocked verdict carries over', ()
expect(repairPrompt(r.file, ID).status).not.toBe(0);
});
});
// Value: protects=a later JSON line for the same audit never overrides an earlier blocked line (open note on #4285 at 67375f4: a
// blocked result line followed by an unfenced "current" line as the last line was accepted as current, and a "current" last line
// with only a format slip was repaired back to current); fails_when=the last line drops an earlier blocked status or blocker and
// is accepted or repaired without the blocked verdict; seam=gstack-docs-candidate result --repair-of; no_claim=a blocker written
// only in prose
describe('an earlier blocked line for the same audit carries into the last line', () => {
const ID = 'docs-carry-1';
const C = { schema_version: 1, audit_id: ID, status: 'current', files_updated: [], files_reviewed: ['README.md'], blockers: [], decisions: [],
documentation_section: '**Status:** current. No edits needed.\n\nScope: README.md; no coverage debt.' };
const blocker = 'README.md has stale security instructions and needs owner approval.';
const blocked = { ...C, status: 'blocked', blockers: [blocker], documentation_section: '**Status:** blocked. README.md needs an owner decision.\n\nScope: README.md.' };
const { documentation_section: _s, ...slip } = C;
test.each([
['an unfenced current line', C],
['a current line whose only fault is a missing section', slip],
])('a blocked line followed by %s as the last line is never accepted as current', (_name, last) => {
const first = result([JSON.stringify(blocked), 'A clean result looks like this:', JSON.stringify(last)].join('\n') + '\n', ID);
expect(first.status).toBe(1);
expect(first.stdout).not.toContain('DOCS_RESULT: ok');
expect(first.stdout).toContain('the last JSON line drops the blocked result an earlier line gave for this audit');
expect(first.stdout).toMatch(/^FORMAT_REPAIR: format-only; not an attempt/m);
expect(first.written).toEqual([]);
expect(repairPrompt(first.file, ID).stdout).toContain('must keep "status": "blocked" and every blocker');
const current = result(JSON.stringify(C) + '\n', ID, { file: first.file, out: first.out });
expect(current.status).toBe(1);
expect(current.stdout).toContain('the format repair changed "status"');
expect(current.stdout).toMatch(/^RETRY_NOTE: /m);
});
test('the re-ask reply that keeps the blocked verdict is accepted', () => {
const first = result([JSON.stringify(blocked), JSON.stringify(C)].join('\n') + '\n', ID);
const answer = result(JSON.stringify(blocked) + '\n', ID, { file: first.file, out: first.out });
expect([answer.status, answer.stderr]).toEqual([0, '']);
expect(answer.stdout).toContain('DOCS_RESULT: ok status=blocked');
});
test.each([
['a current line, then a blocked last line', [C, blocked]],
['a blocked line, then a blocked last line that adds a blocker', [blocked, { ...blocked, blockers: [blocker, 'docs/setup.md repeats the stale steps.'] }]],
])('%s is accepted as before', (_name, objects) => {
const r = result(objects.map(o => JSON.stringify(o)).join('\n') + '\n', ID);
expect([r.status, r.stderr]).toEqual([0, '']);
expect(r.stdout).toContain('DOCS_RESULT: ok status=blocked');
});
});
@@ -0,0 +1,4 @@
{
"bytes": 33,
"why": "Step 14.5 Step 2 says the last line is the result and an earlier blocked line carries over (GST-2349 follow-up)"
}