mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-20 20:00:45 +02:00
record gstack 2 baseline and backlog map
This commit is contained in:
Sinabina
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,101 @@
|
||||
# GStack 2.0 baseline
|
||||
|
||||
This is the measured starting point, not a completion claim. The audit was run
|
||||
from detached base worktree `/tmp/gstack2-baseline.e2qk7F` at commit
|
||||
`bb57306d98c97011b0919c6132705a15b1579781` on 2026-07-16.
|
||||
|
||||
## Skill surface and size
|
||||
|
||||
| Measure | Baseline |
|
||||
|---|---:|
|
||||
| Source `SKILL.md.tmpl` templates | 55 |
|
||||
| Primary generated skill outputs | 54 |
|
||||
| Generated section files | 16 |
|
||||
| Primary surface | 61,093 lines / 3,301,287 bytes |
|
||||
| Audited union | 67,996 lines / 3,695,876 bytes |
|
||||
| Repeated preambles | 49 |
|
||||
| Repeated-preamble footprint | 35,802 lines / 2,132,374 bytes |
|
||||
| Generated hosts | 10 |
|
||||
|
||||
The 10 generated hosts observed in the generator/check logs are Claude Code,
|
||||
OpenAI Codex CLI, Factory Droid, Kiro, OpenCode, Slate, Cursor, OpenClaw,
|
||||
Hermes, and GBrain.
|
||||
|
||||
The catalog-token audit has two materially different numbers:
|
||||
|
||||
- Correct YAML parsing of the names and descriptions for 54 skills is about
|
||||
**1,100 token-equivalents**.
|
||||
- The official buggy capture reported
|
||||
`captureBaseline().estTotalCatalogTokens = 4,214`. It applied the same
|
||||
four-characters-per-token estimate to **16,857 incorrectly parsed
|
||||
description bytes**. The 4,214 figure is therefore an instrument bug, not the
|
||||
real catalog cost.
|
||||
|
||||
These counts establish the consolidation problem: they do not by themselves
|
||||
set a target or prove that any repeated instruction is safe to remove.
|
||||
|
||||
## Installer/discovery baseline
|
||||
|
||||
The old standard-container path has a concrete root-shadow failure. From the
|
||||
detached base worktree, this command:
|
||||
|
||||
```bash
|
||||
DISABLE_TELEMETRY=1 npx --yes skills add . --list
|
||||
```
|
||||
|
||||
reported `Found 1 skill` and listed only the root `gstack` skill. Under the
|
||||
skills CLI discovery rule, the shallower root `SKILL.md` is discovered first
|
||||
and shadows discovery of the nested standard-container skills. Separately,
|
||||
legacy `./setup` installs 54 host-specific skills. Thus “the legacy installer
|
||||
works” is not evidence that the standard installer exposes the same surface;
|
||||
both install roots and collision behavior need migration tests.
|
||||
|
||||
## Command baseline
|
||||
|
||||
The baseline runner recorded these commands:
|
||||
|
||||
```bash
|
||||
bun install
|
||||
bun run gen:skill-docs --host all
|
||||
bun run skill:check
|
||||
bun run build
|
||||
bun test
|
||||
bun run test:windows
|
||||
bun test design/test
|
||||
bun test ios-qa/daemon/test
|
||||
```
|
||||
|
||||
Results are intentionally not summarized as “green”:
|
||||
|
||||
- dependency installation completed with 297 packages;
|
||||
- all-host generation reached its final `llms.txt` summary;
|
||||
- the build reached its final Node server bundle step;
|
||||
- `skill:check` exited 1 because `claude/SKILL.md` was missing;
|
||||
- the broad `bun test` capture contains seven explicit failed tests and one
|
||||
unhandled between-tests error, then ends without Bun's terminal summary or
|
||||
exit record;
|
||||
- the Windows-safe run exited 1 on shard 5 of 20;
|
||||
- the targeted design suite contains one failed timing assertion and no
|
||||
terminal summary;
|
||||
- the targeted iOS daemon suite completed with 91 pass / 0 fail.
|
||||
|
||||
The precise assertions and log paths are in
|
||||
[TEST-EVIDENCE.md](./TEST-EVIDENCE.md). An empty
|
||||
`/tmp/gstack2-baseline-logs/git-status-after.txt` records no tracked/untracked
|
||||
status output after the baseline runner, but it is not a substitute for test
|
||||
success.
|
||||
|
||||
## Backlog baseline
|
||||
|
||||
The frozen API inputs contain 755 unique open GitHub items after reconciling
|
||||
the issues endpoint (which also contains PRs) with the PR endpoint:
|
||||
|
||||
- `garrytan/gstack`: 748 items;
|
||||
- `time-attack/gstack`: 7 items;
|
||||
- 326 issues and 429 pull requests in total;
|
||||
- 1,184 raw open-endpoint records, including 429 reconciled duplicates;
|
||||
- 16 required upstream PRs with separate detail and changed-file snapshots.
|
||||
|
||||
See [BACKLOG-MAP.json](./BACKLOG-MAP.json) for the complete mapping and
|
||||
`scripts/gstack2/generate-backlog-map.ts` for the deterministic, offline
|
||||
generator.
|
||||
@@ -0,0 +1,56 @@
|
||||
# GStack 2.0 status
|
||||
|
||||
**Current state: BASELINED AND MAPPED — NOT IMPLEMENTED, NOT VERIFIED, NOT
|
||||
DONE.**
|
||||
|
||||
This checklist separates evidence capture from delivery so generated plans or
|
||||
passing narrow tests cannot be mistaken for a completed rewrite.
|
||||
|
||||
## Phase checklist
|
||||
|
||||
- [x] Freeze the audit base at
|
||||
`bb57306d98c97011b0919c6132705a15b1579781`.
|
||||
- [x] Record template/output/section, line, byte, repeated-preamble, catalog,
|
||||
host, and installer-discovery baselines.
|
||||
- [x] Capture the baseline command logs without relabeling pre-existing
|
||||
failures as passes.
|
||||
- [x] Flatten and reconcile every frozen open issue/PR snapshot into one
|
||||
deterministic map (755 unique items from 1,184 endpoint records).
|
||||
- [x] Trace all 16 required PRs to their detail snapshots, changed-file
|
||||
snapshots, component, judgment module, and replacement-test contract.
|
||||
- [ ] Review and approve the GStack 2.0 architecture and migration contract.
|
||||
- [ ] Convert heuristic backlog dispositions into accepted product/engineering
|
||||
decisions. `NEEDS_EVIDENCE` is intentionally the majority disposition.
|
||||
- [ ] Implement the consolidated judgment modules and runtime components.
|
||||
- [ ] Implement the replacement contract tests named in `BACKLOG-MAP.json`.
|
||||
- [ ] Prove legacy setup migration and standard-installer discovery without
|
||||
root/nested skill shadowing.
|
||||
- [ ] Prove behavior and safety parity across all 10 generated hosts.
|
||||
- [ ] Re-run the broad, Windows-safe, design, and iOS suites with complete
|
||||
terminal summaries; separate repaired baseline failures from regressions.
|
||||
- [ ] Produce release evidence and make an explicit ship/no-ship decision.
|
||||
|
||||
## Evidence index
|
||||
|
||||
| Evidence | Path | State |
|
||||
|---|---|---|
|
||||
| Measured baseline | [`BASELINE.md`](./BASELINE.md) | Recorded |
|
||||
| Baseline command results and failures | [`TEST-EVIDENCE.md`](./TEST-EVIDENCE.md) | Recorded; baseline is not green |
|
||||
| Deterministic backlog generator | [`../../scripts/gstack2/generate-backlog-map.ts`](../../scripts/gstack2/generate-backlog-map.ts) | Implemented and locally validated |
|
||||
| Complete mapped backlog | [`BACKLOG-MAP.json`](./BACKLOG-MAP.json) | Generated from frozen snapshots |
|
||||
| Baseline logs | `/tmp/gstack2-baseline-logs/` | External/ephemeral audit evidence |
|
||||
| time-attack snapshots | `/tmp/gstack2-{open-items,open-prs,label}-pages.json` | External/ephemeral frozen input |
|
||||
| garrytan snapshots | `/tmp/gstack2-upstream-{open-items,open-prs,label}-pages.json` | External/ephemeral frozen input |
|
||||
| Required PR evidence | `/tmp/gstack2-port-prs/{610,...,2189}{,-files}.json` | External/ephemeral frozen input |
|
||||
|
||||
## Interpretation rules
|
||||
|
||||
- `PORT_JUDGMENT` means “port the decision/policy with a replacement test,”
|
||||
not “apply the upstream patch verbatim.”
|
||||
- `FIX_IN_GSTACK_2` still requires a reproduced defect.
|
||||
- `NEEDS_EVIDENCE`, `DEFER_COMMUNITY`, and
|
||||
`SUPERSEDED_BY_CONSOLIDATION` remain review decisions, not closed GitHub
|
||||
items. The generator performs no external mutation.
|
||||
- The heuristic map is deterministic triage. Title-first and narrow body rules
|
||||
make it auditable, but human approval is still required before implementation
|
||||
or upstream disposition changes.
|
||||
@@ -0,0 +1,70 @@
|
||||
# GStack 2.0 baseline test evidence
|
||||
|
||||
Scope: detached base worktree `/tmp/gstack2-baseline.e2qk7F`, commit
|
||||
`bb57306d98c97011b0919c6132705a15b1579781`, captured 2026-07-16. These are
|
||||
baseline observations only; no result below was produced by the GStack 2.0
|
||||
implementation.
|
||||
|
||||
## Command ledger
|
||||
|
||||
| Command | Evidence | Observed result |
|
||||
|---|---|---|
|
||||
| `bun install` | `/tmp/gstack2-baseline-logs/bun-install.log` | Completed; Bun 1.3.14 reported 297 packages installed. |
|
||||
| `bun run gen:skill-docs --host all` | `/tmp/gstack2-baseline-logs/gen-all.log` | Reached the final GBrain token table and `llms.txt` summary (`55 skills, 76 browse commands`); no error marker is present. |
|
||||
| `bun run skill:check` | `/tmp/gstack2-baseline-logs/skill-check.log` | **Exit 1**: `claude/SKILL.md — generated file missing`. The same run reported 54/54 outputs for each of the 10 hosts and fresh generated files, but the missing primary output keeps the command red. |
|
||||
| `bun run build` | `/tmp/gstack2-baseline-logs/build.log` | Reached the final Node-compatible server bundle (`server-node.mjs`, 0.83 MB); no error marker is present. |
|
||||
| `bun test` | `/tmp/gstack2-baseline-logs/bun-test.log` | **Not green and incompletely terminated**: 4,292 `(pass)` markers, 488 `(skip)` markers, seven explicit `(fail)` markers, and one unhandled between-tests error. The capture ends after `browse/test/findport.test.ts` without Bun's aggregate summary or exit record. |
|
||||
| `bun run test:windows` | `/tmp/gstack2-baseline-logs/test-windows.log` | **Exit 1** on shard 5/20. The runner selected 189 Windows-safe files and excluded 155. Completed shard summaries contain 593 passes, 14 skips, and one failed test before the run stopped. |
|
||||
| `bun test design/test` | `/tmp/gstack2-baseline-logs/design-test.log` | **Not green**: 64 pass markers and one explicit failure. The log ends on daemon shutdown without an aggregate summary or exit record. |
|
||||
| `bun test ios-qa/daemon/test` | `/tmp/gstack2-baseline-logs/ios-daemon-test.log` | **Pass**: 91 pass, 0 fail, 217 assertions across 10 files. |
|
||||
|
||||
`/tmp/gstack2-baseline-logs/git-status-after.txt` is empty, recording no status
|
||||
entries after the runner. The logs are external audit artifacts and are not
|
||||
vendored into the repository.
|
||||
|
||||
## Pre-existing `bun test` failures
|
||||
|
||||
The broad-suite log records these seven explicit failures:
|
||||
|
||||
1. `gstack-decision-search --recent / --scope / datamark > --scope filters by scope`
|
||||
expected output containing `branch-call` and received an empty string.
|
||||
2. `gstack-gbrain-detect > emits valid JSON even when nothing is configured`
|
||||
expected status 0 and received 127.
|
||||
3. `gstack-gbrain-detect > reports gstack_brain_git: true when GSTACK_HOME has a .git dir`
|
||||
attempted to parse empty output and raised `Unexpected EOF`.
|
||||
4. `gstack-gbrain-detect > reports gbrain_config + engine when ~/.gbrain/config.json exists`
|
||||
attempted to parse empty output and raised `Unexpected EOF`.
|
||||
5. `gstack-gbrain-detect > malformed config returns null engine, does not crash`
|
||||
expected status 0 and received 127.
|
||||
6. `gstack-gbrain-detect > detects a mocked gbrain binary on PATH and reports its version`
|
||||
expected status 0 and received 127.
|
||||
7. `resolve-user-slug fallback chain > persists resolution to user_slug_at_<hash> on first call`
|
||||
expected a hashed key and observed `user_slug_at_local: persisttest`.
|
||||
|
||||
Separately, `test/gbrain-refresh-install-render.test.ts` raised an unhandled
|
||||
between-tests error: `Could not locate gbrain-refresh ok) branch`. Because the
|
||||
log has no terminal Bun summary, the marker counts are evidence of what ran,
|
||||
not a claim that all files were reached or that the command had only those
|
||||
failures.
|
||||
|
||||
## Other pre-existing failures
|
||||
|
||||
- `skill:check`: the primary `claude/SKILL.md` generated file was absent; the
|
||||
script explicitly exited with code 1.
|
||||
- Windows-safe shard 5:
|
||||
`Source-level guard: terminal-agent > lazy spawn: claude PTY is spawned in message handler, not on upgrade`
|
||||
still looked for `spawnClaude(`, while the source excerpt used
|
||||
`maybeSpawnPty(...)`. The shard reported 82 pass / 1 skip / 1 fail and the
|
||||
remaining 15 shards did not run.
|
||||
- Design target:
|
||||
`generateVariant Retry-After handling > HTTP-date: honors a future date with no extra leading exponential`
|
||||
expected a delay of at least 2,500 ms and observed 2,246 ms.
|
||||
|
||||
## What this evidence does not establish
|
||||
|
||||
- It does not establish a green baseline.
|
||||
- It does not establish that GStack 2.0 preserves behavior; replacement
|
||||
contract tests have not yet been implemented.
|
||||
- It does not turn incomplete logs into passes.
|
||||
- It does not attribute these failures to later changes; every failure above
|
||||
was captured from the stated base SHA before GStack 2.0 work.
|
||||
Reference in New Issue
Block a user