mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-15 01:15:29 +02:00
feat(evidence): verification-evidence ledger mechanizes /ship's IRON LAW
New bin/gstack-evidence: a transparent wrapper that records every verification
run as {ts, label, command, cmd_sha256, exit, duration_s, commit, tree, dirty,
wtree, log_path} in ~/.gstack/projects/<slug>/<branch>-evidence.jsonl, plus a
read-only `check` that grades FRESH/STALE/MISSING per label. "Tests passed"
now binds to the exact working-tree content it ran on (bin/gstack-wtree
fingerprint), so evidence recorded on uncommitted code stays FRESH after the
exact tested content is committed — the /ship Step 5 -> Step 16 case — while
an untracked new source file or any content change invalidates it.
Check semantics: every named label's latest record must be green, within
--max-age, matching --expect-cmd's hash when given, and fingerprint-identical
(or diff confined to --allow-paths — mechanizing Step 16's existing "CHANGELOG
edits don't count" carve-out). No --any mode: a green lane can never mask a
red sibling. Any git failure inside check (gc'd tree object, not a repo)
degrades to STALE/MISSING, never an error into the calling skill flow.
Transparency invariant (load-bearing, test-pinned): the child's exit code is
ALWAYS the wrapper's exit code; ledger/log/redact failures are stderr
warnings. Logs are per-run (0600, exclusive-open, 2MB truncation marker,
30-day opportunistic prune) — no more shared /tmp collisions between
concurrent ships. Command strings are redact-scanned before recording (HIGH
credential -> stored redacted). Machine-local by design: neither ledger nor
logs brain-sync.
Wired: ship Step 5 lanes run wrapped (per-lane labels), ship Step 16 and
land-and-deploy 3.5b check the ledger first and cite FRESH evidence instead of
re-running; a failed CHECK never blocks (run live), a failed RUN does.
test/evidence.test.ts: 21 tests incl. the keystone dirty-record -> commit ->
FRESH case.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
8e4a4f7c3d
commit
4836f0d1e3
+16
-1
@@ -1280,9 +1280,24 @@ EOF
|
||||
|
||||
**IRON LAW: NO COMPLETION CLAIMS WITHOUT FRESH VERIFICATION EVIDENCE.**
|
||||
|
||||
The evidence ledger is the mechanical arm of this law. Check it FIRST:
|
||||
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-evidence check --label tests --label vitest --max-age 24 --allow-paths CHANGELOG.md,VERSION,package.json
|
||||
```
|
||||
|
||||
- **Every line FRESH (exit 0):** the recorded runs were green and the working-tree
|
||||
content is identical to what was tested, modulo the allow-listed release files
|
||||
(this mechanizes the "CHANGELOG edits don't count" rule — VERSION/CHANGELOG
|
||||
commits between Step 5 and here don't invalidate the run). Cite the evidence
|
||||
lines (label, exit, ts, log path) as the verification evidence and continue.
|
||||
- **Any STALE/MISSING (exit non-zero):** run live, wrapped, so the fresh run is
|
||||
recorded: `~/.claude/skills/gstack/bin/gstack-evidence run --label <lane> -- '<command>'`.
|
||||
The check is an advisory guardrail — a failed CHECK never blocks; a failed RUN does.
|
||||
|
||||
Before pushing, re-verify if code changed during Steps 4-6:
|
||||
|
||||
1. **Test verification:** If ANY code changed after Step 5's test run (fixes from review findings, CHANGELOG edits don't count), re-run the test suite. Paste fresh output. Stale output from Step 5 is NOT acceptable.
|
||||
1. **Test verification:** If ANY code changed after Step 5's test run (fixes from review findings, CHANGELOG edits don't count), re-run the test suite. The evidence check above IS this rule, mechanized — trust FRESH, re-run on STALE. Paste fresh output when you re-run. Stale output from Step 5 with changed content is NOT acceptable.
|
||||
|
||||
2. **Build verification:** If the project has a build step, run it. Paste output.
|
||||
|
||||
|
||||
+16
-1
@@ -375,9 +375,24 @@ EOF
|
||||
|
||||
**IRON LAW: NO COMPLETION CLAIMS WITHOUT FRESH VERIFICATION EVIDENCE.**
|
||||
|
||||
The evidence ledger is the mechanical arm of this law. Check it FIRST:
|
||||
|
||||
```bash
|
||||
~/.claude/skills/gstack/bin/gstack-evidence check --label tests --label vitest --max-age 24 --allow-paths CHANGELOG.md,VERSION,package.json
|
||||
```
|
||||
|
||||
- **Every line FRESH (exit 0):** the recorded runs were green and the working-tree
|
||||
content is identical to what was tested, modulo the allow-listed release files
|
||||
(this mechanizes the "CHANGELOG edits don't count" rule — VERSION/CHANGELOG
|
||||
commits between Step 5 and here don't invalidate the run). Cite the evidence
|
||||
lines (label, exit, ts, log path) as the verification evidence and continue.
|
||||
- **Any STALE/MISSING (exit non-zero):** run live, wrapped, so the fresh run is
|
||||
recorded: `~/.claude/skills/gstack/bin/gstack-evidence run --label <lane> -- '<command>'`.
|
||||
The check is an advisory guardrail — a failed CHECK never blocks; a failed RUN does.
|
||||
|
||||
Before pushing, re-verify if code changed during Steps 4-6:
|
||||
|
||||
1. **Test verification:** If ANY code changed after Step 5's test run (fixes from review findings, CHANGELOG edits don't count), re-run the test suite. Paste fresh output. Stale output from Step 5 is NOT acceptable.
|
||||
1. **Test verification:** If ANY code changed after Step 5's test run (fixes from review findings, CHANGELOG edits don't count), re-run the test suite. The evidence check above IS this rule, mechanized — trust FRESH, re-run on STALE. Paste fresh output when you re-run. Stale output from Step 5 with changed content is NOT acceptable.
|
||||
|
||||
2. **Build verification:** If the project has a build step, run it. Paste output.
|
||||
|
||||
|
||||
+11
-4
@@ -195,15 +195,22 @@ Only commit if there are changes. Stage all bootstrap files (config, test direct
|
||||
`db:test:prepare` internally, which loads the schema into the correct lane database.
|
||||
Running bare test migrations without INSTANCE hits an orphan DB and corrupts structure.sql.
|
||||
|
||||
Run both test suites in parallel:
|
||||
Run both test suites in parallel, each wrapped in the evidence ledger. The
|
||||
wrapper is transparent (streams output live, exit code passes through) and
|
||||
records `{command, exit, working-tree fingerprint, log path}` to
|
||||
`~/.gstack/projects/<slug>/<branch>-evidence.jsonl` — Step 16 cites this
|
||||
record instead of re-running when the content hasn't changed:
|
||||
|
||||
```bash
|
||||
bin/test-lane 2>&1 | tee /tmp/ship_tests.txt &
|
||||
npm run test 2>&1 | tee /tmp/ship_vitest.txt &
|
||||
~/.claude/skills/gstack/bin/gstack-evidence run --label tests -- 'bin/test-lane 2>&1' &
|
||||
~/.claude/skills/gstack/bin/gstack-evidence run --label vitest -- 'npm run test 2>&1' &
|
||||
wait
|
||||
```
|
||||
|
||||
After both complete, read the output files and check pass/fail.
|
||||
After both complete, check the `gstack-evidence: recorded label=... exit=...
|
||||
log=...` summary lines — each carries the lane's exit code and a per-run log
|
||||
file (no shared /tmp collisions between concurrent ships). Read the log files
|
||||
for failure detail.
|
||||
|
||||
**If any test fails:** Do NOT immediately stop. Apply the Test Failure Ownership Triage:
|
||||
|
||||
|
||||
@@ -10,15 +10,22 @@
|
||||
`db:test:prepare` internally, which loads the schema into the correct lane database.
|
||||
Running bare test migrations without INSTANCE hits an orphan DB and corrupts structure.sql.
|
||||
|
||||
Run both test suites in parallel:
|
||||
Run both test suites in parallel, each wrapped in the evidence ledger. The
|
||||
wrapper is transparent (streams output live, exit code passes through) and
|
||||
records `{command, exit, working-tree fingerprint, log path}` to
|
||||
`~/.gstack/projects/<slug>/<branch>-evidence.jsonl` — Step 16 cites this
|
||||
record instead of re-running when the content hasn't changed:
|
||||
|
||||
```bash
|
||||
bin/test-lane 2>&1 | tee /tmp/ship_tests.txt &
|
||||
npm run test 2>&1 | tee /tmp/ship_vitest.txt &
|
||||
~/.claude/skills/gstack/bin/gstack-evidence run --label tests -- 'bin/test-lane 2>&1' &
|
||||
~/.claude/skills/gstack/bin/gstack-evidence run --label vitest -- 'npm run test 2>&1' &
|
||||
wait
|
||||
```
|
||||
|
||||
After both complete, read the output files and check pass/fail.
|
||||
After both complete, check the `gstack-evidence: recorded label=... exit=...
|
||||
log=...` summary lines — each carries the lane's exit code and a per-run log
|
||||
file (no shared /tmp collisions between concurrent ships). Read the log files
|
||||
for failure detail.
|
||||
|
||||
**If any test fails:** Do NOT immediately stop. Apply the Test Failure Ownership Triage:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user