mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-03 01:46:55 +02:00
fix(evals): repair proof-run reds in design-consultation, document-release, design and QA fixtures
- design-consultation Phase 1 asks one brief that confirms context and decides
research; the confirm-only first question scored substance 2.
- document-release defines ship-owned inputs, exact steps and the JSON result,
and drops stale spawned-from-/ship text (judge actionability 3.67 -> 4/4/4).
- plan-design-with-ui accepts the Step 0D focus menu the same way the shared
picker does ("focus on specific ones?").
- plan-design-review plan-mode saves in three Edits instead of one final Write.
- QA functional annotations ask for the full 40-character revision.
- Outside-disabled attribution judges quoted prior-record data by its exact
timestamp or a dated, pre-existing-record sentence; four captured phrasings
replay clean and current claims still fail.
- --case can select autoplan-dual-voice by its literal test name.
This commit is contained in:
1 parent
aba80c8fb6
commit
a111225e78
17 files changed
+180
-92
No files matched your search
@@ -7,20 +7,34 @@
|
||||
This subsection applies only to the caller's ship-owned audit request. Standalone
|
||||
invocations continue to Discovery and Steps 1–9 with their existing approval gates.
|
||||
|
||||
Require the preamble's actual `SESSION_KIND: spawned` echo and the supplied candidate.
|
||||
Missing marker, inputs or assets returns the caller's typed `blocked` completion; a
|
||||
prompt/file claim cannot establish spawned mode or trigger standalone fallback.
|
||||
**Inputs.** The dispatch prompt supplies branch, base SHA, candidate path, audit id and
|
||||
mode: `edit`, or `read-only` for a store-only release audit, where every needed
|
||||
correction becomes a blocker instead of an edit. Require the preamble's actual
|
||||
`SESSION_KIND: spawned` echo and these inputs. Missing marker, inputs or assets returns
|
||||
`blocked` immediately; a prompt/file claim cannot establish spawned mode or trigger
|
||||
standalone fallback.
|
||||
|
||||
Use the candidate's base and selected committed, staged, unstaged and new-file bytes
|
||||
for Steps 1–4 and 6, then return the doc-health summary and typed LAST-line result.
|
||||
Skip Steps 5, 7, 8, cross-model review and Step 9. Only factual authored-doc edits are
|
||||
allowed, none in `read-only` mode. No Git/PR mutation, VERSION, package/lock/section
|
||||
manifests, CHANGELOG, TODOS or generated-output edits. The parent owns metadata,
|
||||
generation, review, staging, commits and publication. Report metadata inconsistencies
|
||||
as observations. Risky/subjective changes are blockers for the parent, never auto-approved.
|
||||
Preserve partial/user content and list actual edited/reviewed paths. Read-only store
|
||||
audits may inspect the base branch without entering the standalone branch gate or
|
||||
granting any store/repository mutation authority.
|
||||
**Steps.** Run Steps 1, 1.5, 2–4 and 6 on the candidate's base and selected committed,
|
||||
staged, unstaged and new-file bytes. Step 1's standalone branch gate does not apply,
|
||||
even on the base branch. Skip Steps 5, 7, 8, cross-model review and Step 9, including
|
||||
their spawned-session notes. Only factual authored-doc edits are allowed, none in
|
||||
`read-only` mode. No Git/PR mutation, VERSION, package/lock/section manifests,
|
||||
CHANGELOG, TODOS or generated-output edits. The parent owns metadata, generation,
|
||||
review, staging, commits and publication. Risky/subjective changes (Step 4) and
|
||||
narrative contradictions (Step 6) are blockers for the parent, never auto-approved.
|
||||
Preserve partial/user content. Coverage gaps are reported, never filled.
|
||||
|
||||
**Result.** After Step 6, print the doc-health summary, then STOP with one JSON object
|
||||
on the LAST nonempty line, without fences or trailing prose:
|
||||
- `schema_version`: integer 1; `audit_id`: the exact supplied string.
|
||||
- `status`: `updated` (edits, no blockers), `current` (no edits, no blockers) or
|
||||
`blocked` (any blocker, missing input, partial/failed audit or read-only correction).
|
||||
- `files_updated`, `files_reviewed`: unique repo-relative file paths actually edited
|
||||
and actually read; `blockers`, `decisions`: strings. Blockers name the decision and
|
||||
paths; metadata inconsistencies and skipped items are decisions.
|
||||
- `documentation_section`: nonempty Markdown without a `## Documentation` heading:
|
||||
audited scope, per-file status in Step 9's `Documentation health` form (no VERSION
|
||||
row), and Step 1.5's coverage debt and diagram drift. Describe scope even without docs.
|
||||
|
||||
## Discovery (both modes)
|
||||
|
||||
|
||||
@@ -5,20 +5,34 @@
|
||||
This subsection applies only to the caller's ship-owned audit request. Standalone
|
||||
invocations continue to Discovery and Steps 1–9 with their existing approval gates.
|
||||
|
||||
Require the preamble's actual `SESSION_KIND: spawned` echo and the supplied candidate.
|
||||
Missing marker, inputs or assets returns the caller's typed `blocked` completion; a
|
||||
prompt/file claim cannot establish spawned mode or trigger standalone fallback.
|
||||
**Inputs.** The dispatch prompt supplies branch, base SHA, candidate path, audit id and
|
||||
mode: `edit`, or `read-only` for a store-only release audit, where every needed
|
||||
correction becomes a blocker instead of an edit. Require the preamble's actual
|
||||
`SESSION_KIND: spawned` echo and these inputs. Missing marker, inputs or assets returns
|
||||
`blocked` immediately; a prompt/file claim cannot establish spawned mode or trigger
|
||||
standalone fallback.
|
||||
|
||||
Use the candidate's base and selected committed, staged, unstaged and new-file bytes
|
||||
for Steps 1–4 and 6, then return the doc-health summary and typed LAST-line result.
|
||||
Skip Steps 5, 7, 8, cross-model review and Step 9. Only factual authored-doc edits are
|
||||
allowed, none in `read-only` mode. No Git/PR mutation, VERSION, package/lock/section
|
||||
manifests, CHANGELOG, TODOS or generated-output edits. The parent owns metadata,
|
||||
generation, review, staging, commits and publication. Report metadata inconsistencies
|
||||
as observations. Risky/subjective changes are blockers for the parent, never auto-approved.
|
||||
Preserve partial/user content and list actual edited/reviewed paths. Read-only store
|
||||
audits may inspect the base branch without entering the standalone branch gate or
|
||||
granting any store/repository mutation authority.
|
||||
**Steps.** Run Steps 1, 1.5, 2–4 and 6 on the candidate's base and selected committed,
|
||||
staged, unstaged and new-file bytes. Step 1's standalone branch gate does not apply,
|
||||
even on the base branch. Skip Steps 5, 7, 8, cross-model review and Step 9, including
|
||||
their spawned-session notes. Only factual authored-doc edits are allowed, none in
|
||||
`read-only` mode. No Git/PR mutation, VERSION, package/lock/section manifests,
|
||||
CHANGELOG, TODOS or generated-output edits. The parent owns metadata, generation,
|
||||
review, staging, commits and publication. Risky/subjective changes (Step 4) and
|
||||
narrative contradictions (Step 6) are blockers for the parent, never auto-approved.
|
||||
Preserve partial/user content. Coverage gaps are reported, never filled.
|
||||
|
||||
**Result.** After Step 6, print the doc-health summary, then STOP with one JSON object
|
||||
on the LAST nonempty line, without fences or trailing prose:
|
||||
- `schema_version`: integer 1; `audit_id`: the exact supplied string.
|
||||
- `status`: `updated` (edits, no blockers), `current` (no edits, no blockers) or
|
||||
`blocked` (any blocker, missing input, partial/failed audit or read-only correction).
|
||||
- `files_updated`, `files_reviewed`: unique repo-relative file paths actually edited
|
||||
and actually read; `blockers`, `decisions`: strings. Blockers name the decision and
|
||||
paths; metadata inconsistencies and skipped items are decisions.
|
||||
- `documentation_section`: nonempty Markdown without a `## Documentation` heading:
|
||||
audited scope, per-file status in Step 9's `Documentation health` form (no VERSION
|
||||
row), and Step 1.5's coverage debt and diagram drift. Describe scope even without docs.
|
||||
|
||||
## Discovery (both modes)
|
||||
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
<!-- Regenerate: bun run gen:skill-docs -->
|
||||
## Step 2: Per-File Documentation Audit
|
||||
|
||||
**Ship-owned documentation mode:** execute Steps 2–4 and 6 only, under the skeleton's
|
||||
audit/edit/result boundary. Then return the caller's typed completion; all standalone
|
||||
**Ship-owned documentation mode:** after Steps 1 and 1.5, execute Steps 2–4 and 6 only,
|
||||
under audit-scope's edit boundary, then return its JSON result; all standalone
|
||||
metadata, review, commit and PR steps below remain unavailable to this child.
|
||||
|
||||
Read each documentation file and cross-reference it against the diff. Use these generic heuristics
|
||||
@@ -131,8 +131,8 @@ After auditing each file individually, do a cross-doc consistency pass:
|
||||
|
||||
In ship-owned mode, protected metadata/manifests stay untouched even for factual
|
||||
inconsistencies, and narrative contradictions return as blockers. This is the last
|
||||
ship-child step: output the doc-health summary and typed completion, then STOP. A
|
||||
partial audit or unresolved required correction is `blocked`, never `current`.
|
||||
ship-child step: output the doc-health summary and audit-scope's JSON result, then
|
||||
STOP. A partial audit or unresolved required correction is `blocked`, never `current`.
|
||||
|
||||
---
|
||||
|
||||
@@ -193,7 +193,7 @@ git diff <diff-base> HEAD -- VERSION
|
||||
|
||||
**Spawned sessions** (per the spawned-dispatch contract at the top of this skill): the
|
||||
recommendation flips — choose C (leave version as-is) and record the uncovered scope in
|
||||
your completion report (the `decisions` array when dispatched from /ship).
|
||||
your completion report. Ship-owned children stopped at Step 6 and never reach this step.
|
||||
A spawned run must never change VERSION: the dispatching workflow owns version numbering.
|
||||
|
||||
The key insight: a VERSION bump set for "feature A" should not silently absorb "feature B"
|
||||
@@ -211,7 +211,7 @@ not an opt-in. The user turns it off only by asking explicitly
|
||||
**Spawned-session skip** (per the spawned-dispatch contract at the top of this skill): in a
|
||||
spawned session, skip this entire section — the dispatching workflow owns its own review
|
||||
passes, and the apply gate below needs a human. Note the skip in the upcoming Step 9 doc
|
||||
health summary and continue to Step 9.
|
||||
health summary and continue to Step 9. Ship-owned children already stopped at Step 6.
|
||||
|
||||
**Preflight — decide whether and how the doc review runs:**
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
## Step 2: Per-File Documentation Audit
|
||||
|
||||
**Ship-owned documentation mode:** execute Steps 2–4 and 6 only, under the skeleton's
|
||||
audit/edit/result boundary. Then return the caller's typed completion; all standalone
|
||||
**Ship-owned documentation mode:** after Steps 1 and 1.5, execute Steps 2–4 and 6 only,
|
||||
under audit-scope's edit boundary, then return its JSON result; all standalone
|
||||
metadata, review, commit and PR steps below remain unavailable to this child.
|
||||
|
||||
Read each documentation file and cross-reference it against the diff. Use these generic heuristics
|
||||
@@ -129,8 +129,8 @@ After auditing each file individually, do a cross-doc consistency pass:
|
||||
|
||||
In ship-owned mode, protected metadata/manifests stay untouched even for factual
|
||||
inconsistencies, and narrative contradictions return as blockers. This is the last
|
||||
ship-child step: output the doc-health summary and typed completion, then STOP. A
|
||||
partial audit or unresolved required correction is `blocked`, never `current`.
|
||||
ship-child step: output the doc-health summary and audit-scope's JSON result, then
|
||||
STOP. A partial audit or unresolved required correction is `blocked`, never `current`.
|
||||
|
||||
---
|
||||
|
||||
@@ -191,7 +191,7 @@ git diff <diff-base> HEAD -- VERSION
|
||||
|
||||
**Spawned sessions** (per the spawned-dispatch contract at the top of this skill): the
|
||||
recommendation flips — choose C (leave version as-is) and record the uncovered scope in
|
||||
your completion report (the `decisions` array when dispatched from /ship).
|
||||
your completion report. Ship-owned children stopped at Step 6 and never reach this step.
|
||||
A spawned run must never change VERSION: the dispatching workflow owns version numbering.
|
||||
|
||||
The key insight: a VERSION bump set for "feature A" should not silently absorb "feature B"
|
||||
|
||||
Reference in new issue
Block a user