mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-03 15:27:56 +01:00
Garry Tanandcapy-ai[bot]
c1900088d5
* feat(skills): absorb pstack's blast radius, transcript recall, vet-the-number and design-twice as lazy sections
/investigate gains sections/blast-radius.md (the one safety fact, the five-rung
evidence ladder, look where grep stops, confirmed and cleared risks apart) with a
Phase 4 pointer, and the shared-premise red flag before the 3-strike rule.
/context-restore is carved for the first time: on NO_CHECKPOINTS, sections/transcripts.md
mines this workspace's own transcripts (host stores declared in hosts/*.ts and rendered
by TRANSCRIPT_SOURCES / TRANSCRIPT_DISCOVERY), a fixed 7-day window, never another
project's folder, subagents return only findings, redaction before anything leaves,
git and gh verify every artifact, a status-tagged brief. /benchmark is carved for the
first time: sections/vet-the-number.md asks the seven questions before Phase 5 reports
a comparison. /plan-eng-review gains sections/design-twice.md (a second structurally
distinct shape compared on interface depth, the eight design red flags), bound by the
autoplan methodology snapshot alongside review-sections.md.
Ideas from pstack (Lauren Tan, MIT), rewritten in gstack's words; NOTICE.md. Garry,
2026-10-26: "Do it, send". Guards: carve-guards entries and measured lines, eleven
plan-eng-review pins re-measured, periodic carve-section-loading cases for the two new
carves, free tests with a fixture transcript.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(pstack-c1): fold design-twice into Section 1 of the eng review section, per-host transcript table, fixture and slice-cap repairs
The remote free run reported nine failures in seven files, all from this
branch: four tests encode plan-eng-review's one companion section
(eng-scope-entry-ap, autoplan-methodology-names, plan-review-cases,
review-checklists), so the design-twice pass now lives inside Section 1 of
sections/review-sections.md with the skeleton, manifest and
bin/gstack-autoplan-snapshot.ts back at main; the Codex render of
context-restore named ~/.claude/ through the all-hosts transcript table, so
{{TRANSCRIPT_SOURCES}} renders this host's row only; the transcript fixture
is declared in FREE_FIXTURES; and the two new periodic cases took the census
to 39 slices, so evals-periodic.yml's eval-slices max-parallel goes 38 -> 40.
Pins: 11 plan-eng-review re-pins at +2,896; maxSizeRatio 1.2400 (measured
1.2389); Codex gpt-5.4 overlay 159,588 bytes.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix: pre-landing review fixes
The discovery block takes the none branch from hosts/*.ts's transcripts
declaration instead of a host-name fallthrough, and the resolver header says
where each half lives; the T2 subagent brief frames the transcript as data,
not instructions; the test asserts table/block agreement per host, imports
CAPY_NO_TRANSCRIPTS, and drops the fixture-shape case nothing reads.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* docs: ADDING_A_HOST.md lists the optional transcripts HostConfig field
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
---------
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
3.7 KiB
3.7 KiB
Vet the number
A performance number is a claim, and these seven questions test the claim before anyone acts on it. Answer each from a run, never from a reading of the code. A one-run ballpark the user asked for is allowed: say it is one run, still answer questions 4 and 7, and skip the rest unless the run looks wrong. A choice between two options is never a ballpark.
Before running anything, write the sentence you expect to ship ("LCP 1.6 s →
1.1 s at the median on the dashboard page") and read what the measurement
script times, counts and ignores. Check uptime and nproc; on a busy
machine, interleave the two sides so both see the same noise, and say so.
- Why not double? Name the limiter: the one resource or stage the time is spent in (one core at 100%, a 3 MB script parse, a serialized request chain, link bandwidth). Profile in a run you do not report, because profilers slow the work. When a change did not move the number, the limiter is the reason; find it before calling the change useless.
- Was it tuned? Both sides run the way production runs: production build and flags, the same data, caches as warm or cold as users see them, the same versions. A side on defaults (a debug build, a cold cache, a missing index) makes this a comparison of configurations, not of implementations. Tune it and measure again, or pick no winner.
- Did it break a limit? Do the arithmetic. Bytes per second against the link, operations per second times cost against the cores, time saved against the time the removed piece took. Removing a piece that took 10% of the run makes the run at most about 11% faster; a result past a limit measured something other than the work (a cache, a no-op, a bug).
- Did it error? Count failures, timeouts, retries and non-success responses, and check that the output is right, not only present. Rejections are fast and retries are slow. If the script does not count errors, add the count before trusting the time.
- Does it reproduce? At least 5 runs per side, alternating A, B, A, B, so warmup, lazy initialization, caches and drift favor neither side. Report the median and the range. A gap smaller than the run-to-run variation is no difference; when the call is close, use the harness's own statistics or a rank-sum test.
- Does it matter? Put the micro number next to the end-to-end path a user waits on, with realistic data and concurrency, and report it as a share of the whole. A step that is 1% of the page can make the page at most 1% faster.
- Did it even happen? Confirm the work ran inside the timed region: the request reached the server, the bytes were fetched, the frame was painted, the result was used. Lazy code (a promise nobody awaits, a generator nobody drains) and timeouts produce numbers for work that never ran.
Report
Lead with the verdict: faster, slower, no measurable difference, or inconclusive. Give the number with its unit, the run count, the range and the limiter ("LCP 1.6 s → 1.1 s, median of 7 alternating runs per side, range 1.0 to 1.2 s after, bound by the hero image decode"). Call the verdict inconclusive when you claim a difference but cannot name the limiter, when a side ran untuned, or when questions 4 and 7 went unanswered, and say which. A PR body carries one primary number; the runs, the range and the limiter evidence go in the report file this skill writes in Phase 9.