mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-21 04:10:47 +02:00
document gstack 2 evidence and blocked gates
This commit is contained in:
Sinabina
@@ -0,0 +1,266 @@
|
||||
# GStack 2 architecture
|
||||
|
||||
Status: implemented in the candidate branch, with release evidence still
|
||||
pending. This document describes the candidate; it is not a 2.0 release claim.
|
||||
The authoritative gate state is [STATUS.md](./STATUS.md).
|
||||
|
||||
## Product boundary
|
||||
|
||||
GStack 2 is an engineering judgment layer with exactly six public skills:
|
||||
|
||||
```text
|
||||
plan -> design -> qa -> debug -> review -> ship
|
||||
```
|
||||
|
||||
This is a navigation surface, not a linear workflow and not a compression of
|
||||
specialists into generic prompts. Each dispatcher selects a mode from
|
||||
structured stage, surface, artifact, mutation-authority, evidence, and deploy
|
||||
signals. It prints the target, mode, depth, mutation boundary, active modules,
|
||||
skipped modules, and web-context choice before work. It then loads only the
|
||||
selected preserved module.
|
||||
|
||||
The canonical tree is:
|
||||
|
||||
```text
|
||||
skills/
|
||||
plan|design|qa|debug|review|ship/
|
||||
SKILL.md thin dispatcher
|
||||
agents/openai.yaml optional host presentation metadata
|
||||
references/
|
||||
legacy/*.md lazily loaded preserved specialists
|
||||
ASSETS.md relocated-asset index
|
||||
COMPATIBILITY.md old-name routes within this tree
|
||||
assets/ copied assets with pinned blob evidence
|
||||
compat/*.md internal aliases, never default skills
|
||||
evals/parity/ contracts, scenarios, regressions, manifest
|
||||
```
|
||||
|
||||
`scripts/gstack2/generate-skill-tree.ts` mechanically renders 55 pinned legacy
|
||||
modules, inlines 16 carved sections, and carries 78 assets. The generator
|
||||
records source Git blob IDs and normalized render hashes. It adds only reviewed
|
||||
upstream bug-fix overlays, each with a PR link, stable anchor, and regression fixture. The parity
|
||||
runner checks nine behavioral dimensions: question order, pressure, smart
|
||||
skips, STOP/approval gates, evidence, artifacts, mutation, exit behavior, and
|
||||
voice.
|
||||
|
||||
The public dispatcher cannot substitute for its selected module. Its
|
||||
completeness invariant explicitly requires reading that module in full. See
|
||||
[JUDGMENT-PROVENANCE.json](./JUDGMENT-PROVENANCE.json),
|
||||
[JUDGMENT-PARITY.md](./JUDGMENT-PARITY.md), and
|
||||
[SKILL-MIGRATION.md](./SKILL-MIGRATION.md).
|
||||
|
||||
The reproducible semantic layer adds 295 checks across 14 suites, 15
|
||||
executions, 15 dimensions, 16 sections, and nine authority-policy unit cases.
|
||||
Exact source preservation remains the primary oracle. These are deterministic
|
||||
policy checks, not behavioral-adversarial proof. Paid live-model comparisons
|
||||
are supplemental; the currently retained semantic samples are regressions and
|
||||
do not support a release-pass claim.
|
||||
|
||||
The separate installed-host adversarial lane also has no passing live result:
|
||||
v1 failed; immutable v2 failed despite QA passing because its classifier
|
||||
produced false negatives for debug, review, and ship; and v3 has 18 pass / 0
|
||||
fail / 111 assertions offline but has not run live and has no artifact. See the
|
||||
[installed-host evidence overview](../../evals/host-adversarial/README.md).
|
||||
|
||||
## Specialist modes
|
||||
|
||||
The primary modes are summarized here; internal compatibility modes are in the
|
||||
migration map.
|
||||
|
||||
| Skill | Public top-level modes and preserved refinements |
|
||||
|---|---|
|
||||
| `/plan` | exactly **Discovery, Product, Engineering, DX, Specification, Full chain**, refined to office-hours, CEO, engineering, DX, spec, or autoplan judgment |
|
||||
| `/design` | **Explore, Generate, Critique, Implement**, refined to consultation, alternatives, HTML/CSS, plan/live review, physical-iOS HIG, diagram, or PDF |
|
||||
| `/qa` | **Report** or **Fix**, refined by web, physical-iOS, DX, performance, or canary surface |
|
||||
| `/debug` | **Diagnose-only** or **Fix**, refined to general investigation or the physical-iOS fix loop |
|
||||
| `/review` | **Normal, Security, Performance, Deep**, with health and genuinely independent outside voices selected only when applicable |
|
||||
| `/ship` | **Prepare, Land, Deploy, Monitor, Resume**, refined to PR, queue, docs, deploy setup, land/deploy, canary, or context restoration modules |
|
||||
|
||||
Old commands are opt-in aliases for the compatibility window. An alias points
|
||||
to one exact module, prints the replacement invocation, and carries no copied
|
||||
judgment. This prevents an alias from drifting away from the canonical source.
|
||||
|
||||
## Installation boundary
|
||||
|
||||
The standard Agent Skills tree is the unit of distribution:
|
||||
|
||||
```bash
|
||||
npx skills add time-attack/gstack
|
||||
```
|
||||
|
||||
The installer—not GStack—owns host detection, project/global scope, destination
|
||||
paths, selected-skill installation, copies versus symlinks, updates, and
|
||||
removal. GStack does not silently enroll a detected host. The former
|
||||
host-placement behavior of `./setup` and ten-host generated trees is historical
|
||||
compatibility/development machinery, not the 2.0 installation architecture;
|
||||
the current `./setup` installs only the optional runtime described below.
|
||||
|
||||
The standards installer matrix passed 470/470 checks with CLI 1.5.19. It
|
||||
verified project/global copies for Claude Code, Codex, Cursor, Pi, OpenClaw, and
|
||||
GitHub Copilot plus selected-skill and opt-in compatibility-alias cases,
|
||||
removal, spaces, source symlink, copy mode, and canonical hashes across 16
|
||||
installs and two removals. The committed artifact is
|
||||
[`evals/installation/install-matrix.json`](../../evals/installation/install-matrix.json).
|
||||
This is installer-layer verification; it is Markdown-only placement and does
|
||||
not prove a passing live installed-host judgment run.
|
||||
|
||||
Pure judgment is Markdown and works when the optional runtime is absent. A
|
||||
capability-dependent module performs one runtime check and must degrade with an
|
||||
actionable setup choice rather than make the six skills undiscoverable.
|
||||
|
||||
## Optional host-neutral runtime
|
||||
|
||||
`bin/gstack` calls a Node-built-ins runtime under `runtime/`. It does not own
|
||||
skill placement and is installed once per user, not once per host. The
|
||||
host-neutral `./setup` wrapper installs only this runtime/capability bundle. It
|
||||
stages an explicit allowlist, rejects internal symlinks/path escapes,
|
||||
hash-validates files, smoke-tests the CLI, atomically activates the version,
|
||||
and writes stable POSIX/Windows launchers plus an uninstall manifest under
|
||||
`$GSTACK_HOME`. Runtime capability builds use a dedicated target that does not
|
||||
regenerate skills or require repository history; Windows targets use `.exe`
|
||||
suffixes and Darwin-only iOS artifacts are omitted elsewhere. Its public
|
||||
operations are setup, doctor, config, state
|
||||
inspection/resume, Context.dev status/options/select/setup/smoke, cleanup,
|
||||
upgrade/rollback, and uninstall.
|
||||
|
||||
The current managed-bundle audit records 107 components, 1,830 files,
|
||||
459,056,031 bytes, and 50 launchers. Setup installs frozen production-only
|
||||
dependencies; the development-only Claude Agent SDK is excluded. The
|
||||
Sharp/ngrok dependency closure is included. The Hugging Face sidecar is
|
||||
excluded and its package is development-only, so setup installs neither its
|
||||
inference runtime nor model weights and reports the L4 capability unavailable.
|
||||
|
||||
A clean Linux arm64 runtime smoke installed only production dependencies with
|
||||
the development SDK absent, completed a local-browser journey and Sharp
|
||||
full-page screenshot, and uninstalled while preserving state. This is Linux
|
||||
container evidence, not native Windows evidence.
|
||||
|
||||
The authoritative root is `$GSTACK_HOME` or `~/.gstack`; host-specific variables
|
||||
do not redirect it. Paths are passed as values and never emitted for shell
|
||||
evaluation. The state shape is human-readable:
|
||||
|
||||
```text
|
||||
~/.gstack/
|
||||
config.json
|
||||
secrets.json mode 0600 where supported
|
||||
migration.json
|
||||
locks/
|
||||
tmp/
|
||||
versions/current.json
|
||||
projects/<repo+worktree-id>/
|
||||
state.json
|
||||
timeline.jsonl
|
||||
decisions.jsonl
|
||||
evidence/
|
||||
artifacts/
|
||||
reviews/
|
||||
checkpoints/
|
||||
```
|
||||
|
||||
Repository identity is shared across linked worktrees; worktree identity is
|
||||
stable and distinct. Their combination selects project state, preventing one
|
||||
worktree from resuming another's run. Updates use atomic replace plus lock
|
||||
leases. External actions are durably claimed before execution. A process death
|
||||
after a claim marks the action uncertain on resume and refuses automatic
|
||||
repetition until explicitly reconciled.
|
||||
|
||||
Migrations are forward-only and idempotent. Runtime upgrades copy into a staged
|
||||
version, verify, switch an atomic pointer, and retain the last known good
|
||||
version. A failed health check or interrupted pending pointer restores the
|
||||
previous active version. This is bounded release plumbing, not a distributed
|
||||
workflow engine.
|
||||
|
||||
## Context.dev boundary
|
||||
|
||||
Context.dev is the only new external service and handles only public web
|
||||
content. `network.mode` and `network.consent` both start off. The client checks
|
||||
both before DNS or fetch. It rejects credentials in URLs, localhost, local and
|
||||
private names, private/link-local IP literals, cloud-metadata targets, and a
|
||||
public hostname that resolves to a private address.
|
||||
|
||||
Supported candidate operations use the documented scrape-Markdown,
|
||||
scrape-HTML, crawl, sitemap, and screenshot endpoints. The current general
|
||||
search endpoint is deprecated, so `search()` returns a typed unsupported error
|
||||
without a network request. A workflow must instead use an explicitly selected
|
||||
host-native public search, the local browser, or no web research and label the
|
||||
result unverified. It must not invent a replacement API.
|
||||
|
||||
`gstack context options` presents those choices. `gstack context select
|
||||
host|local-browser|none` persists the explicit fallback while keeping
|
||||
Context.dev consent false; `context setup` is the only path that selects
|
||||
Context.dev and persists export consent with the protected key.
|
||||
|
||||
The deterministic Context contract is green at 22 pass / 0 fail and 139
|
||||
assertions. No verified key was available for a live provider smoke, so provider
|
||||
behavior remains blocked rather than inferred from the contract suite.
|
||||
|
||||
See [CONTEXT-DEV.md](./CONTEXT-DEV.md) and [PRIVACY.md](./PRIVACY.md).
|
||||
|
||||
## Local browser and physical iOS
|
||||
|
||||
The existing Chromium/Playwright daemon remains the sole browser-automation
|
||||
backend. It binds loopback, requires bearer authorization for mutations,
|
||||
preserves authenticated sessions locally, denies tunnel commands by default,
|
||||
and treats page/console/network content as untrusted. Context.dev complements
|
||||
public research; it does not replace browser QA. No cloud browser was added.
|
||||
|
||||
The existing DebugBridge/CoreDevice harness remains the sole physical-iOS
|
||||
backend. The candidate distinguishes hardware UDID from CoreDevice UUID,
|
||||
returns bounded 504 responses for a suspended app, asserts the expected bundle
|
||||
around coordinate mutations, preserves typed snapshot/mutation/restoration,
|
||||
and keeps bridge symbols debug-only. Device signing and provisioning are
|
||||
remediable setup gates, not product failures. A live signed-device pass is
|
||||
still required before release. Current preflight evidence is 9 pass / 0 fail /
|
||||
1 deploy skip and 29 assertions. The direct smoke returned
|
||||
`signing_unavailable` / `setup_gate`; no app was installed and no pass artifact
|
||||
was written.
|
||||
|
||||
PDF rendering and Mermaid/Excalidraw remain internal capabilities. GStack does
|
||||
not add Typst, hosted document rendering, alternate diagram services, local
|
||||
image weights, ComfyUI, or a GPU runtime.
|
||||
|
||||
## Infrastructure-defect disposition
|
||||
|
||||
“Contained” means the defect is outside the canonical GStack 2 path, not that
|
||||
legacy code was proven bug-free. “Implemented” names candidate code. “Pending”
|
||||
means the replacement still lacks its required release evidence.
|
||||
|
||||
| # | Legacy defect | GStack 2 disposition / replacement | Replacement evidence |
|
||||
|---:|---|---|---|
|
||||
| 1 | Ten registered hosts; setup fully installs five | **Contained:** placement is delegated to the standard installer. | 470/470 checks across six hosts, 16 installs, two removals, project/global scopes, and selections. Passing live v3/host UI launch remains separate. |
|
||||
| 2 | Kiro rewrites Codex output | **Contained:** one canonical standards tree; no Kiro rewrite in the 2.0 path. | Six-host matrix installs byte-matching canonical copies without host rewrites. |
|
||||
| 3 | Gitignored external trees defeat freshness CI | **Implemented:** canonical `skills/`, `compat/`, and parity fixtures are committed. | 4,681 parity checks plus installed-file hash equality. |
|
||||
| 4 | External `--dry-run` mutates files | **Contained:** external host generation/dry-run is not used for 2.0 distribution. | Canonical regeneration/parity check exists; a non-mutating canonical check mode is not yet present. |
|
||||
| 5 | Single-host generation failures only warn | **Contained:** no per-host generation in the canonical path; canonical generation throws on failure. | Generator/parity suite and final build rerun are green. |
|
||||
| 6 | Setup continues after failed generation | **Implemented/contained:** standard installer handles skills; runtime setup fails before activation and preserves last known good. | Failure tests plus the real two-version default lifecycle and rollback pass. |
|
||||
| 7 | Removed/renamed generated skills are not pruned | **Implemented for canonical references:** regeneration removes each reference/asset tree and parity corpus before writing the fixed six. | Parity inventory test; stale-public-directory fixture pending. |
|
||||
| 8 | Freshness misses generator/design/PDF changes | **Implemented:** build invokes canonical generation; parity hashes source, sections, assets, and fixtures. | Design is green at 101/0/381, PDF at 189/0/398, and the uninterrupted broad singleton run is green across all 383 files. |
|
||||
| 9 | State paths bypass canonical resolver | **Implemented:** `runtime/paths.js` is authoritative for 2.0 runtime state. | `gstack2-runtime-core.test.ts`. |
|
||||
| 10 | Shell-evaluated path assignments are unsafe | **Implemented:** runtime paths are JavaScript values; shell-looking input remains literal. | Core test covers spaces, `$()`, semicolon, and `$HOME` text. |
|
||||
| 11 | Relinking omits carved section links | **Contained:** no per-section host relinking; sections are inlined into pinned modules. | Parity checks all 16 sections. |
|
||||
| 12 | External preambles omit PDF paths | **Contained:** PDF is indexed as an internal design capability/assets route, not copied host preamble prose. | Asset parity plus 189/0/398 PDF strict tests and a visually checked four-page live render pass. |
|
||||
| 13 | Production model benchmark imports test helpers | **Implemented:** the runner, pricing, providers, and optional judge live under `lib/model-benchmark/`; the production CLI imports only production modules. | `benchmark-production-boundary.test.ts` rejects imports from `test/` across `bin/` and `lib/`; focused runner and CLI tests exercise the relocated implementation. |
|
||||
| 14 | Default tests omit `design/test` | **Implemented:** package default and free-test roots include `design/test`. | Design is green at 101/0/381 and is included in the uninterrupted 383-file broad pass. |
|
||||
| 15 | Default tests omit `ios-qa/daemon/test` | **Implemented:** package default and free-test roots include the daemon tests. | Focused daemon run: 95 pass / 0 fail / 229 assertions; daemon tests are also included in the uninterrupted broad pass. |
|
||||
| 16 | Host setup contradicts config-driven claim | **Contained:** host setup is no longer a 2.0 responsibility. | Standard installer CLI 1.5.19 passed all six configured host targets. |
|
||||
| 17 | Host-generated judgment copies drift | **Implemented:** one canonical module corpus with source-blob/render hashes. | 4,681 parity checks and installed-copy hashes pass. |
|
||||
| 18 | Updating one host leaves another stale | **Contained:** one tree is installed by each host's standard installer. | Project/global copies across six hosts matched canonical hashes; remote update flow remains installer-owned. |
|
||||
| 19 | State identity crosses worktrees | **Implemented:** repo plus stable worktree identity selects state. | Linked-worktree core test passes. |
|
||||
| 20 | Partial ship failures are not reliably idempotent | **Implemented at runtime primitive:** claimed effects become uncertain and are not automatically repeated. | Crash/resume and completed-effect tests pass. End-to-end ship resume remains pending. |
|
||||
| 21 | Parser failures become empty success | **Implemented in iOS device discovery:** parse/tool failures are typed errors. | `tunnel-bootstrap.test.ts` malformed-JSON regression passes in the focused daemon suite. |
|
||||
| 22 | Setup failures become product failures | **Partly implemented:** iOS discovery/setup categories and runtime doctor return actionable setup state. | Automated categories exist; live signing/provisioning gate pending. |
|
||||
| 23 | Runtime network activity is not obvious | **Implemented for Context runtime:** selection, mode, and consent are explicit; status/doctor report them; zero lookup/fetch before Context selection+consent. | Context contract: 22 pass / 0 fail / 139 assertions, including persisted non-export fallbacks. Verified-key live smoke remains blocked. |
|
||||
| 24 | Context restore selects another worktree | **Implemented for canonical state resume:** current repo+worktree project ID scopes inspection/resume. | Linked-worktree identity test passes; compatibility end-to-end restore test pending. |
|
||||
| 25 | Preambles repeat large sections in every skill | **Implemented structurally:** six thin lazy dispatchers share infrastructure and load one preserved module on demand. | Current generated six-name/description catalog is 982 characters, about 246 token-equivalents versus the correctly parsed baseline of about 1,100 (77.6% lower). Re-measure if frontmatter changes. |
|
||||
|
||||
No defect in this table should be closed from prose alone. The final claim must
|
||||
link its reproduction and passing test in [TEST-EVIDENCE.md](./TEST-EVIDENCE.md).
|
||||
|
||||
## Explicit non-architecture
|
||||
|
||||
GStack 2 does not contain a generic provider or plugin marketplace, distributed
|
||||
workflow engine, mandatory cloud service, cloud browser/device farm,
|
||||
Browserbase/Browser Use/Agent Device/Appium/XCUITest driver layer, mandatory
|
||||
telemetry, mandatory GBrain/Docker/image generation, local model weights,
|
||||
ComfyUI, or a new runtime database. Existing development-only dependencies are
|
||||
not promoted into user setup or the GStack 2 architecture.
|
||||
@@ -0,0 +1,162 @@
|
||||
# Context.dev setup and operation
|
||||
|
||||
Context.dev is the only newly authorized external service in GStack 2. It is
|
||||
optional and restricted to public web context. The current automated contract
|
||||
is green at 22 pass / 0 fail and 139 assertions. No verified account key was
|
||||
available, so the live smoke has not run and provider behavior remains
|
||||
unverified; see [TEST-EVIDENCE.md](./TEST-EVIDENCE.md).
|
||||
|
||||
## Choose a public-web mode
|
||||
|
||||
When a workflow needs public research and no choice is stored, it must offer:
|
||||
|
||||
```text
|
||||
A) Set up Context.dev free (recommended)
|
||||
B) Use this host's built-in public web search, if available
|
||||
C) Use GStack's local browser
|
||||
D) Continue without web research
|
||||
```
|
||||
|
||||
Declining Context.dev is not a workflow failure. Persist only the explicit
|
||||
choice. Never infer consent from a previous browser navigation, host login,
|
||||
environment variable, or legacy configuration.
|
||||
|
||||
The runtime exposes the same choice without requiring a prompt:
|
||||
|
||||
```bash
|
||||
gstack context options
|
||||
gstack context select host
|
||||
gstack context select local-browser
|
||||
gstack context select none
|
||||
```
|
||||
|
||||
`select` persists exactly one choice in `config.json`. The three fallback
|
||||
choices keep `network.consent` false and never configure Context.dev export.
|
||||
`context status` reports the stored selection. If `$GSTACK_HOME/bin` is not on
|
||||
`PATH`, invoke the default launcher as `~/.gstack/bin/gstack`.
|
||||
|
||||
## Free-tier caveat
|
||||
|
||||
Provider limits can change. At the 2026-07-16 implementation check, the
|
||||
[Context.dev pricing page](https://www.context.dev/pricing) described 500
|
||||
monthly credits for a work-email signup, 250 for a personal/free-email signup,
|
||||
no credit card, and ordinary page scraping at one credit. Email verification
|
||||
is required, and provider responses for unverified or exhausted accounts may
|
||||
not use intuitive HTTP status codes. Verify current terms before promising a
|
||||
quota or cost; GStack maps the response body as well as status.
|
||||
|
||||
GStack never opts into paid usage or surprise spending. When credits are
|
||||
exhausted it explains the condition and offers a fallback.
|
||||
|
||||
## Setup
|
||||
|
||||
1. Open [Context.dev agent authentication](https://www.context.dev/auth.md) or
|
||||
[signup](https://www.context.dev/signup).
|
||||
2. Create the account and complete email verification.
|
||||
3. Install the optional host-neutral runtime if the active mode needs it.
|
||||
4. Review the options, then run the interactive Context.dev setup:
|
||||
|
||||
```bash
|
||||
gstack context options
|
||||
gstack context setup
|
||||
```
|
||||
|
||||
5. Read the egress prompt. Type `yes` only if public-page requests to
|
||||
Context.dev are acceptable.
|
||||
6. Enter the API key at the hidden prompt. Do not paste it into chat, a command
|
||||
argument, shell history, a project `.env`, or repository file.
|
||||
7. Check status:
|
||||
|
||||
```bash
|
||||
gstack context status
|
||||
gstack doctor
|
||||
```
|
||||
|
||||
8. Run one public-page smoke test:
|
||||
|
||||
```bash
|
||||
gstack context smoke --url https://www.context.dev
|
||||
```
|
||||
|
||||
For a non-interactive controlled environment, `--consent` is required and the
|
||||
key must come from protected stdin or `CONTEXT_DEV_API_KEY`; the CLI rejects
|
||||
key-looking command-line arguments. Interactive hidden input is preferred.
|
||||
|
||||
The runtime stores the explicit selection and public choices in
|
||||
`~/.gstack/config.json` and the key in
|
||||
`~/.gstack/secrets.json`, mode `0600` where the platform supports POSIX modes.
|
||||
`$GSTACK_HOME` changes that root. Secrets cannot be written through
|
||||
`gstack config set`.
|
||||
|
||||
## Data-egress contract
|
||||
|
||||
Context.dev may receive only the public target URL and operation parameters
|
||||
needed for the selected scrape/crawl/sitemap/screenshot operation. It must not
|
||||
receive:
|
||||
|
||||
- authenticated pages or private dashboards;
|
||||
- localhost, intranet, local/private/link-local addresses, or cloud metadata;
|
||||
- a public hostname whose DNS result is non-public;
|
||||
- URLs containing usernames or passwords;
|
||||
- private repository content, diffs, prompts, or user files;
|
||||
- cookies, session tokens, browser profiles, or unrelated credentials.
|
||||
|
||||
The client validates the lexical URL and DNS resolution before a provider
|
||||
request. `network.mode` must equal `context` and `network.consent` must be true
|
||||
before it performs even the DNS lookup. The base endpoint is locked to the
|
||||
credential-free official `https://api.context.dev/v1` origin.
|
||||
|
||||
This is not permission to upload public-looking content from an authenticated
|
||||
session. If provenance is ambiguous, use the local browser and keep the data on
|
||||
the machine.
|
||||
|
||||
## Supported operations and search
|
||||
|
||||
The candidate client implements the documented public scrape-Markdown,
|
||||
scrape-HTML, crawl, sitemap, and screenshot calls. General Context.dev search
|
||||
is currently deprecated by the provider: the official
|
||||
[Context.dev changelog](https://docs.context.dev/changelog) says the endpoint
|
||||
returns `410 Gone`. The current pricing page still lists Web Search credit
|
||||
costs, so the public provider pages conflict. The runtime follows the explicit
|
||||
deprecation notice and returns a typed unsupported `CONTEXT_BAD_RESPONSE`
|
||||
without a network call; it does not guess an endpoint or claim search evidence.
|
||||
|
||||
For public search, use the host-native facility only when the user selects it
|
||||
and it is available. Otherwise use the local browser. If neither is appropriate,
|
||||
continue without web research and label conclusions unverified.
|
||||
|
||||
## Exact failure taxonomy
|
||||
|
||||
| Code | Meaning | Recovery |
|
||||
|---|---|---|
|
||||
| `CONTEXT_KEY_MISSING` | No key is available. | Offer setup, host-native public search, local browser, or no-web continuation. |
|
||||
| `CONTEXT_KEY_INVALID` | Key format or provider authentication failed. | Re-enter/reissue the key; never print it. |
|
||||
| `CONTEXT_EMAIL_UNVERIFIED` | Provider account needs email verification. | Ask the user to verify, then retry the smoke test. |
|
||||
| `CONTEXT_CREDITS_EXHAUSTED` | Free/paid allowance is depleted. | Explain the quota and offer fallbacks; never purchase automatically. |
|
||||
| `CONTEXT_RATE_LIMITED` | Provider is throttling requests. | Honor `Retry-After` or fall back; avoid a retry storm. |
|
||||
| `CONTEXT_TIMEOUT` | Request reached its bounded timeout. | Retry only when safe, or use local/host-native context. |
|
||||
| `CONTEXT_BLOCKED` | URL/privacy policy rejected the target or provider could not access it safely. | Do not weaken the private-data gate; use local browser when appropriate. |
|
||||
| `CONTEXT_BAD_RESPONSE` | Provider/server/schema error or an explicitly unsupported operation. | Preserve details without secrets; use a fallback. |
|
||||
|
||||
Do not collapse these into “401” or “Context.dev failed.”
|
||||
|
||||
## Disable or rotate
|
||||
|
||||
Disable network use without deleting the stored key:
|
||||
|
||||
```bash
|
||||
gstack context select none
|
||||
```
|
||||
|
||||
This persists selection `none`, mode `off`, and consent `false`.
|
||||
|
||||
Run `gstack context setup` again to replace a rotated key. To remove all runtime
|
||||
state including secrets, use the separately confirmed destructive operation:
|
||||
|
||||
```bash
|
||||
gstack uninstall --purge --yes
|
||||
```
|
||||
|
||||
Standard skill installation and pure judgment do not require a Context.dev
|
||||
account. The 22-test deterministic contract is not a substitute for the
|
||||
blocked verified-key live smoke.
|
||||
@@ -0,0 +1,179 @@
|
||||
# GStack 2 host compatibility
|
||||
|
||||
This matrix uses evidence tiers. It does not call a host “supported” merely
|
||||
because its directory name exists in legacy setup code.
|
||||
|
||||
## Tiers
|
||||
|
||||
| Tier | Meaning |
|
||||
|---|---|
|
||||
| **Portable** | The canonical `skills/<name>/SKILL.md` tree follows the Agent Skills specification and uses no required host-private placement logic. This is a source-format claim. |
|
||||
| **Verified** | The named layer and matrix cells passed against a recorded host/installer version. A scoped “Verified — installer” claim does not imply the host UI launched or executed judgment. |
|
||||
| **Native** | A necessary host API adds behavior unavailable through portable skills while consuming the same canonical judgment source. Native is not “better”; it carries an additional adapter maintenance contract. |
|
||||
|
||||
Tiers are cumulative only when evidence says so. A portable host is not
|
||||
automatically verified. A legacy generated host output is not a native GStack 2
|
||||
bundle.
|
||||
|
||||
## Canonical installation
|
||||
|
||||
```bash
|
||||
npx skills add time-attack/gstack
|
||||
```
|
||||
|
||||
The standards installer owns host detection, destination paths, project/global
|
||||
scope, copy versus symlink, updates, removal, and selected-skill installation.
|
||||
Review its detected-host prompt; detection must never silently enroll a host.
|
||||
|
||||
Examples supported by the installer interface:
|
||||
|
||||
```bash
|
||||
# One public skill only
|
||||
npx skills add time-attack/gstack --skill plan
|
||||
|
||||
# Installer-managed global scope
|
||||
npx skills add time-attack/gstack -g
|
||||
```
|
||||
|
||||
Run `npx skills add --help` for the installed CLI version before scripting
|
||||
agent-selection flags. GStack deliberately does not reproduce those flags or
|
||||
host paths in `./setup`.
|
||||
|
||||
The expected discoverable names are exactly:
|
||||
|
||||
```text
|
||||
plan
|
||||
design
|
||||
qa
|
||||
debug
|
||||
review
|
||||
ship
|
||||
```
|
||||
|
||||
`compat/*.md` and `references/legacy/*.md` are not `SKILL.md` files and must not
|
||||
appear as additional skills. Installing a subset must not pull the other five
|
||||
public entries unless the user selected them.
|
||||
|
||||
## Candidate installer matrix
|
||||
|
||||
The isolated standard-installer matrix passed 470/470 checks with `skills` CLI
|
||||
1.5.19: 16 install cases and two removal cases. Default discovery projected
|
||||
only canonical `skills/`; separate explicit-selection cases covered a single
|
||||
canonical skill and an opt-in legacy alias. The matrix used symlinked and spaced
|
||||
source paths matching a clean checkout where ignored legacy host trees are
|
||||
absent. Every installed file was a physical copy and its hash matched the
|
||||
selected source. The standard skill installation remains Markdown-only and
|
||||
does not install the optional runtime. The committed evidence artifact is
|
||||
[`evals/installation/install-matrix.json`](../../evals/installation/install-matrix.json).
|
||||
|
||||
| Host | Portable | Project all-six | Global all-six | Selected-skill coverage | Installer tier | Host UI/process |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Claude Code | yes | pass | pass | no separate subset case | **Verified — installer** | pending |
|
||||
| OpenAI Codex | yes | pass | pass | global `qa`, `review`, `ship` pass + removal pass; opt-in alias covered | **Verified — installer** | v1/v2 failed; live v3 pending |
|
||||
| Cursor | yes | pass | pass | project `qa`, `review`, `ship` pass + removal pass | **Verified — installer** | pending |
|
||||
| Pi | yes | pass | pass | no separate subset case | **Verified — installer** | pending |
|
||||
| OpenClaw | yes | pass | pass | project `ship` single-skill pass | **Verified — installer** | pending |
|
||||
| GitHub Copilot | yes | pass | pass | no separate subset case | **Verified — installer** | pending |
|
||||
|
||||
The representative selection cases installed exactly `qa`, `review`, and
|
||||
`ship`, independently at project scope for Cursor and global scope for Codex,
|
||||
then removed them noninteractively. Separate cases installed only `ship` for
|
||||
OpenClaw and explicitly selected the `office-hours` compatibility alias for
|
||||
Codex; neither alias nor unselected canonical skill was silently enrolled.
|
||||
`--copy` was advertised and used.
|
||||
|
||||
This is filesystem/installer verification, not a claim that six host UIs loaded
|
||||
or executed the skills. The Codex adversarial lane has no passing live result:
|
||||
v1 and immutable v2 failed, while v3 is green only in its offline 18-test /
|
||||
111-assertion harness and has not run live. The installer matrix used the
|
||||
current local canonical projection through the published `npx skills` CLI, not
|
||||
the still-unpushed GitHub branch URL.
|
||||
|
||||
Legacy generators currently know ten host names. That is historical breadth,
|
||||
not proof that all ten install correctly. Kiro's old Codex-rewrite behavior and
|
||||
other host-output transforms are outside the canonical 2.0 path.
|
||||
|
||||
## Verification procedure and evidence
|
||||
|
||||
Use a clean temporary home and project for every cell; never test against an
|
||||
operator's live skill directory.
|
||||
|
||||
1. Record OS, host version, Node/npm version, and `skills` CLI version.
|
||||
2. List the source and assert exactly six default entries.
|
||||
3. Install all six at project scope and verify each host discovers only those
|
||||
six GStack public skills.
|
||||
4. Remove them through the standard installer.
|
||||
5. Repeat at global scope.
|
||||
6. Install a selected subset; verify unselected skills were not enrolled.
|
||||
7. Reinstall/update without re-detecting or enrolling an unselected host.
|
||||
8. Invoke a pure judgment mode with `gstack` absent from `PATH`.
|
||||
9. Invoke a capability-dependent mode and verify one actionable runtime offer,
|
||||
without breaking judgment.
|
||||
10. Exercise a path containing spaces and, where supported, symlink and
|
||||
read-only failure behavior.
|
||||
11. Record exact command, exit status, output artifact, and cleanup result in
|
||||
[TEST-EVIDENCE.md](./TEST-EVIDENCE.md).
|
||||
|
||||
The automated filesystem portion is:
|
||||
|
||||
```bash
|
||||
bun test test/gstack2-installation.test.ts
|
||||
bun run scripts/gstack2/test-install-matrix.ts --full \
|
||||
--output /tmp/gstack2-install-matrix.json
|
||||
```
|
||||
|
||||
The current matrix passed 470/470 installer CLI checks across 16 installs and two
|
||||
removals; its JSON artifact is committed at
|
||||
[`evals/installation/install-matrix.json`](../../evals/installation/install-matrix.json).
|
||||
Steps 8–9, a passing live v3 adversarial run, and actual host UI loading remain
|
||||
separate behavioral gates.
|
||||
|
||||
## Optional runtime/platform matrix
|
||||
|
||||
The optional runtime is separate from skill placement. From a repository
|
||||
checkout, the one host-neutral setup entrypoint is:
|
||||
|
||||
```bash
|
||||
./setup
|
||||
~/.gstack/bin/gstack doctor --json
|
||||
```
|
||||
|
||||
`./setup` resolves a symlinked checkout, installs dependencies only when absent,
|
||||
builds missing allowlisted capabilities through the runtime-only build target,
|
||||
validates and hashes every staged file, smoke-tests the CLI, atomically
|
||||
activates the version, and writes stable POSIX and Windows launchers under
|
||||
`$GSTACK_HOME/bin` (default `~/.gstack/bin`). The bundle uses `.exe` targets on
|
||||
Windows and includes the CoreDevice/iOS bundle only on Darwin. Add the bin
|
||||
directory to `PATH` if the short `gstack` command is desired.
|
||||
|
||||
Twenty-one focused installer tests pass with 307 assertions. They cover
|
||||
manifests, paths with spaces,
|
||||
source-root symlinks, internal-link/path-escape rejection, failed build/
|
||||
validation/smoke rollback, interrupted-pointer recovery, stable POSIX/Windows
|
||||
launchers, runtime-only builder selection, managed uninstall, and the
|
||||
host-neutral wrapper.
|
||||
|
||||
The current managed-bundle audit records 107 components, 1,830 files,
|
||||
459,056,031 bytes, and 50 launchers. Setup installs frozen production-only
|
||||
dependencies and excludes the development-only Claude Agent SDK. The
|
||||
Sharp/ngrok closure is included. The Hugging Face sidecar is excluded and its
|
||||
package is development-only, so setup installs neither its inference runtime
|
||||
nor model weights and reports the L4 capability unavailable.
|
||||
|
||||
| Platform | Source-level target | Candidate evidence |
|
||||
|---|---|---|
|
||||
| macOS | Node runtime + local browser + physical iOS where applicable | Runtime installer 21/307 and current bundle audit pass. The uninterrupted broad singleton run is green at 6,234 pass / 226 expected skips / 0 fail across 383 files. The signed-device gate remains pending. |
|
||||
| Linux | Node runtime + local browser | Declared Dev Container focused suite passed at 43/0 with 265 assertions. A clean Linux arm64 container used production-only install with the development SDK absent, passed a local-browser journey and Sharp full-page screenshot, and uninstalled while preserving state. Native-host broad Linux remains pending. |
|
||||
| Native Windows | Node runtime; curated free tests; browser fallback where retained | **Blocked:** native Windows CI pending. The local Windows-safe singleton lane is green at 2,813 pass / 57 expected skips / 0 fail across 213 files, but it is not native evidence. |
|
||||
| Dev Container | Pure skills and optional runtime; browser only when container supports it | Declared image builds; focused GStack 2 suite and clean runtime install smoke pass. Full broad/browser coverage remains pending. |
|
||||
|
||||
The six portable skills remain useful when runtime installation fails. A
|
||||
runtime failure must not remove or corrupt their standard-installer placement.
|
||||
|
||||
## Native adapters
|
||||
|
||||
No GStack 2 native tier is currently required or awarded. A future native
|
||||
bundle needs an accepted issue showing that a host API is necessary, must load
|
||||
the same canonical judgment modules, and must pass the portable parity suite.
|
||||
Host-specific presentation metadata such as `agents/openai.yaml` does not by
|
||||
itself create a second judgment source or a Native claim.
|
||||
@@ -0,0 +1,196 @@
|
||||
# iOS physical-device harness
|
||||
|
||||
`ios-qa/scripts/physical-device-smoke.ts` is the real-iPhone deployment lane for the existing DebugBridge backend. It uses Xcode, `devicectl`, and the daemon's CoreDevice tunnel bootstrap. It does not add an Appium, XCUITest, WebDriverAgent, simulator, or cloud-device backend.
|
||||
|
||||
The harness is intentionally fail-closed. A setup problem is not a product failure, and neither one is a pass.
|
||||
|
||||
## Current validation status
|
||||
|
||||
Validated locally at the July 17, 2026 checkpoint with Xcode 26.6 and the wired iPhone supplied for this work:
|
||||
|
||||
| Identifier kind | Example form | Used for |
|
||||
|---|---|---|
|
||||
| Hardware UDID | `<hardware-udid>` | Xcode build destination |
|
||||
| CoreDevice UUID | `<coredevice-uuid>` | `devicectl`, tunnel bootstrap, install, launch |
|
||||
|
||||
Exact local identifiers are deliberately not committed. Successful evidence stores only a SHA-256 fingerprint derived from both identifiers and omits the user-assigned device name.
|
||||
|
||||
The daemon suite is green at 95 pass / 0 fail and 229 assertions. The physical
|
||||
E2E preflight records 9 pass / 0 fail / 1 deploy skip and 29 assertions. The
|
||||
host, pairing/trust, Developer Mode, wired transport, `devicectl`, `xcodegen`,
|
||||
and DevToolsSecurity gates pass. The unsigned Release build also passes and
|
||||
contains no DebugBridge module symbols or artifacts.
|
||||
|
||||
The direct physical-device smoke is externally blocked at automatic signing.
|
||||
It returned typed code `signing_unavailable`, category `setup_gate`. The
|
||||
underlying Xcode diagnostic is:
|
||||
|
||||
```text
|
||||
Signing for "FixtureApp" requires a development team.
|
||||
```
|
||||
|
||||
That is a setup gate, not a DebugBridge failure. No app was installed or
|
||||
launched, and no pass artifact was written. The deploy skip and typed smoke
|
||||
failure must not be represented as a physical-device pass.
|
||||
|
||||
## Hardware UDID versus CoreDevice UUID
|
||||
|
||||
An iPhone has two identifiers relevant to this flow:
|
||||
|
||||
- The hardware UDID is the stable device identifier shown by Xcode. `xcodebuild -destination 'platform=iOS,id=…'` uses it.
|
||||
- The CoreDevice UUID is the session-facing identifier in `devicectl` JSON. Device inspection, install, launch, app-container copy, and tunnel keepalive use it.
|
||||
|
||||
The harness accepts either value through `--device` or `GSTACK_IOS_TARGET_UDID`. It matches the supplied value against both fields, then uses the correct identifier for each tool. It never assumes they are interchangeable internally.
|
||||
|
||||
Inspect both values with:
|
||||
|
||||
```bash
|
||||
tmp=$(mktemp)
|
||||
xcrun devicectl list devices --json-output "$tmp"
|
||||
jq '.result.devices[] | {
|
||||
name: .deviceProperties.name,
|
||||
coreDeviceUUID: .identifier,
|
||||
hardwareUDID: .hardwareProperties.udid,
|
||||
transport: .connectionProperties.transportType,
|
||||
pairing: .connectionProperties.pairingState,
|
||||
developerMode: .deviceProperties.developerModeStatus
|
||||
}' "$tmp"
|
||||
rm "$tmp"
|
||||
```
|
||||
|
||||
If exactly one wired iPhone is visible, the harness selects it. If selection is ambiguous, it refuses to guess and prints both identifier forms.
|
||||
|
||||
## Setup gates
|
||||
|
||||
Run the non-deploying preflight first:
|
||||
|
||||
```bash
|
||||
GSTACK_IOS_TARGET_UDID=<hardware-UDID-or-CoreDevice-UUID> \
|
||||
bun run ios-qa/scripts/physical-device-smoke.ts --preflight-only --json
|
||||
```
|
||||
|
||||
The preflight checks these gates before any build or install:
|
||||
|
||||
| Gate | Exact remediation |
|
||||
|---|---|
|
||||
| Full Xcode selected | `sudo xcode-select -s /Applications/Xcode.app/Contents/Developer` |
|
||||
| First-launch components/license | `sudo xcodebuild -runFirstLaunch` |
|
||||
| `devicectl` available | Verify `xcrun --find devicectl`; repair or update Xcode if missing |
|
||||
| `xcodegen` available | `brew install xcodegen` |
|
||||
| macOS DevToolsSecurity | `sudo DevToolsSecurity -enable`, then `DevToolsSecurity -status` |
|
||||
| Wired iPhone | Connect directly over USB, unlock it, and accept the accessory prompt |
|
||||
| Pairing and trust | `xcrun devicectl manage pair --device <CoreDevice-UUID>`, tap Trust, enter the iPhone passcode |
|
||||
| iPhone Developer Mode | Settings > Privacy & Security > Developer Mode; enable, restart, unlock, and confirm Enable |
|
||||
| CoreDevice management surface | Keep the phone unlocked and verify `xcrun devicectl device info processes --device <CoreDevice-UUID>` |
|
||||
|
||||
DevToolsSecurity and iPhone Developer Mode are separate gates. DevToolsSecurity authorizes developer tools on the Mac. Developer Mode authorizes development services on the iPhone.
|
||||
|
||||
### Signing and provisioning
|
||||
|
||||
The temporary Xcode project contains no hardcoded account or development team. The Debug build always requests automatic signing and provisioning updates.
|
||||
|
||||
To unblock signing:
|
||||
|
||||
1. Open Xcode > Settings > Accounts.
|
||||
2. Add the Apple ID that owns the development team.
|
||||
3. Create or download an Apple Development certificate.
|
||||
4. Leave the iPhone connected and unlocked so Xcode can register it.
|
||||
5. Optionally select a team explicitly for the harness:
|
||||
|
||||
```bash
|
||||
export GSTACK_IOS_DEVELOPMENT_TEAM=<10-character-team-id>
|
||||
```
|
||||
|
||||
`GSTACK_IOS_TEAM_ID` is accepted as a compatibility alias. If both variables are present and disagree, the harness stops. It never discovers a team and silently hardcodes it.
|
||||
|
||||
## Install safety
|
||||
|
||||
The deployment fixture uses the reserved bundle ID:
|
||||
|
||||
```text
|
||||
com.gstack.iosqa.fixture.gstack2
|
||||
```
|
||||
|
||||
Before building, the harness asks `devicectl` whether that exact bundle ID is already installed.
|
||||
|
||||
- No match: installation may proceed.
|
||||
- A clearly identified prior `FixtureApp`: an in-place fixture update may proceed; app data is preserved.
|
||||
- An app with the same bundle ID that does not identify as the fixture: the harness refuses to replace it.
|
||||
|
||||
Only after inspecting a conflict may an operator explicitly allow an in-place replacement:
|
||||
|
||||
```bash
|
||||
export GSTACK_IOS_ALLOW_REPLACE_FIXTURE=1
|
||||
```
|
||||
|
||||
The harness never uninstalls an app and never deletes app data. It uses `devicectl device install app` only after the conflict check.
|
||||
|
||||
## Build, deploy, and verify
|
||||
|
||||
The full lane is opt-in in the Bun E2E test:
|
||||
|
||||
```bash
|
||||
GSTACK_HAS_IOS_DEVICE=1 \
|
||||
GSTACK_IOS_DEVICE_DEPLOY=1 \
|
||||
GSTACK_IOS_TARGET_UDID=<hardware-UDID-or-CoreDevice-UUID> \
|
||||
bun test test/skill-e2e-ios-device.test.ts
|
||||
```
|
||||
|
||||
It can also run directly:
|
||||
|
||||
```bash
|
||||
GSTACK_IOS_TARGET_UDID=<hardware-UDID-or-CoreDevice-UUID> \
|
||||
bun run ios-qa/scripts/physical-device-smoke.ts --json
|
||||
```
|
||||
|
||||
The harness performs these phases:
|
||||
|
||||
1. Copies `test/fixtures/ios-qa/FixtureApp` into a new temporary directory, excluding previous build output and the fixture's unrelated signing spec.
|
||||
2. Generates a team-neutral Release project with `xcodegen`, builds it unsigned for iPhoneOS, and scans the app executable and bundle for DebugBridge module names/artifacts.
|
||||
3. Regenerates the Debug project with local DebugBridge package products, then asks Xcode for automatic signing/provisioning. An optional team comes only from the environment.
|
||||
4. Checks the reserved bundle ID on the selected phone, installs without uninstalling or erasing data, and launches a fresh foreground fixture process.
|
||||
5. Captures the short-lived boot token, calls the existing `bootstrapTunnel`, rotates the credential, and starts the existing CoreDevice tunnel keepalive.
|
||||
6. Runs all five live iterations. A partial run is never promoted to pass evidence.
|
||||
|
||||
## The five checks in every iteration
|
||||
|
||||
Each of the five iterations executes the same five real-device checks:
|
||||
|
||||
| Check | Required evidence |
|
||||
|---|---|
|
||||
| Health and bundle | `/healthz` returns `com.gstack.iosqa.fixture.gstack2` before and after the tap |
|
||||
| Token rotation | Reusing the captured original boot token returns `401 boot_token_invalid` while the rotated credential remains usable |
|
||||
| Session acquire | `/session/acquire` returns a session ID; the session is released even on failure |
|
||||
| Screenshot and elements | `/screenshot` returns a valid PNG and `/elements` returns a live accessibility tree containing `tap-button` |
|
||||
| Coordinate tap and cleanup | A center-coordinate `/tap` reports the expected active bundle before/after, the button count and screenshot both change, state is snapshotted/restored if needed, and the session is released |
|
||||
|
||||
The loop records no bearer tokens or session IDs. It continues through all five iteration slots to make a 5/5 claim meaningful, then fails the run if any iteration failed.
|
||||
|
||||
## Evidence policy
|
||||
|
||||
JSON evidence is written atomically under `docs/gstack-2/evidence/` only after:
|
||||
|
||||
- the Release guard passes;
|
||||
- Debug signing, build, install, launch, and daemon bootstrap pass;
|
||||
- all five iterations pass all five checks;
|
||||
- the final session is released;
|
||||
- the keepalive is stopped; and
|
||||
- the temporary workspace is removed.
|
||||
|
||||
Setup-gate failures, safety refusals, product failures, and partial live runs create no evidence file. A signing error must never be represented as a live pass.
|
||||
|
||||
## Failure categories
|
||||
|
||||
The CLI emits a typed `GSTACK_IOS_PHYSICAL_DEVICE_ERROR` JSON object and uses these categories:
|
||||
|
||||
| Category | Exit code | Meaning |
|
||||
|---|---:|---|
|
||||
| `setup_gate` | 2 | Host, cable, trust, Developer Mode, account, certificate, provisioning, or CoreDevice setup prevents the product from being exercised |
|
||||
| `safety_refusal` | 3 | The harness cannot prove an install is safe, usually because of an unrelated bundle-ID conflict |
|
||||
| `product_failure` | 1 | The fixture or DebugBridge compiled incorrectly, leaked into Release, failed install/launch/bootstrap, or failed a live assertion |
|
||||
|
||||
Fix setup gates without filing them as DebugBridge regressions. Treat a product failure as actionable only after all setup gates pass.
|
||||
|
||||
## Cleanup guarantees
|
||||
|
||||
Every session release is in a `finally` path. The outer cleanup stops the CoreDevice keepalive and removes the harness-owned temporary workspace whether the run passes or fails. The fixture remains installed and its data remains intact; uninstalling or deleting its data requires separate, explicit operator approval.
|
||||
@@ -0,0 +1,18 @@
|
||||
# Judgment parity
|
||||
|
||||
Parity is executable, not a prose claim. Run `bun run scripts/gstack2/run-parity.ts` or the dedicated Bun tests.
|
||||
|
||||
The pinned release inventory passes **4,681 checks** across 55 specialist sources, 16 carved sections, 25 routing scenarios, 16 regression ports, and **78 assets**.
|
||||
|
||||
The suite verifies:
|
||||
|
||||
- exactly six discoverable public skills and 55 internal legacy modules;
|
||||
- 55 canonical templates plus 16 carved section templates at base `bb57306d98c97011b0919c6132705a15b1579781`;
|
||||
- normalized legacy-body SHA-256 equality between source rendering and generated references;
|
||||
- preservation of nine behavioral contract dimensions per module;
|
||||
- 25 structured non-keyword routing fixtures with active/skipped modules, depth, mutation, and web context;
|
||||
- 16 upstream judgment-port regression fixtures and anchors;
|
||||
- all linked asset copies against their pinned Git blobs;
|
||||
- frontmatter and `agents/openai.yaml` schema for each public skill.
|
||||
|
||||
Golden normalization removes only generated provenance wrappers, bug-fix overlays, and irrelevant whitespace. It never removes legacy workflow prose, gates, questions, evidence requirements, artifacts, or exit behavior.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,145 @@
|
||||
# GStack 2 privacy boundary
|
||||
|
||||
GStack 2 defaults to local judgment and local state. Optional network and
|
||||
device capabilities are separated by explicit purpose and consent. Passing a
|
||||
unit test is not a substitute for the remaining live egress audit listed in
|
||||
[STATUS.md](./STATUS.md).
|
||||
|
||||
## Data-flow summary
|
||||
|
||||
| Capability | Default | May leave the machine | Never send |
|
||||
|---|---|---|---|
|
||||
| Six judgment skills | available | nothing by the skill text alone | project data to a new service without separate user/host authority |
|
||||
| Standard installer | user-invoked, Markdown-only | repository/registry requests needed to fetch the skills | runtime dependencies, Context key, project content, browser credentials |
|
||||
| Optional runtime/state | local, network off | nothing until a separately selected operation requires it | secrets in config/log output; one worktree's state to another |
|
||||
| Context.dev | off, explicit consent required | public target URL and public extraction options | authenticated/private/local content, cookies, tokens, repo content, user files |
|
||||
| Local browser | local daemon | navigation requests to sites the user directs it to; explicit legacy tunnels if separately enabled | browser profile/cookies to Context.dev or a cloud-browser provider |
|
||||
| Physical iPhone | local Mac/device bridge | optional pre-existing Tailscale path only when explicitly configured | device session to a cloud-device farm or alternate driver provider |
|
||||
| Telemetry | off in the GStack 2 contract | minimal legacy telemetry only after its independent opt-in | code, prompts, paths, repo/branch names, user content |
|
||||
|
||||
Network installation is not Context.dev consent. Browser navigation is not
|
||||
Context.dev consent. A key present in the environment is not consent. The
|
||||
Context client requires persisted selection `context`, mode `context`, and
|
||||
`network.consent: true` before DNS or fetch. Explicit selections `host`,
|
||||
`local-browser`, and `none` persist with consent false.
|
||||
|
||||
The deterministic Context contract is green at 22 pass / 0 fail and 139
|
||||
assertions. No verified key was available for a live provider smoke; those unit
|
||||
results are not live-egress evidence.
|
||||
|
||||
## Public Context.dev gate
|
||||
|
||||
Before any provider request, the runtime rejects:
|
||||
|
||||
- non-HTTP(S) schemes;
|
||||
- URL usernames/passwords;
|
||||
- localhost and `.localhost`;
|
||||
- private/intranet/test/local suffixes and single-label hosts;
|
||||
- loopback, private, link-local, unspecified, multicast, and mapped-private IPs;
|
||||
- cloud metadata names/addresses; and
|
||||
- a nominally public hostname if any resolved address is non-public.
|
||||
|
||||
Allowed operations still receive only the public URL and necessary operation
|
||||
parameters. Do not paste or synthesize cookies, authorization headers, private
|
||||
page HTML, repository snippets, diffs, or prompts into a public-web request.
|
||||
When a page requires login or its provenance is uncertain, use the local
|
||||
browser and keep the content local.
|
||||
|
||||
Context.dev credentials are read from protected runtime secrets or a deliberate
|
||||
environment variable. Interactive setup uses hidden input. The CLI rejects
|
||||
key-looking arguments, redacts known key formats from errors, and prevents
|
||||
secret-looking config keys from entering public `config.json`. See
|
||||
[CONTEXT-DEV.md](./CONTEXT-DEV.md).
|
||||
|
||||
## Local browser boundary
|
||||
|
||||
GStack retains its own loopback Chromium/Playwright daemon; no cloud-browser
|
||||
backend has been added. Mutating commands require bearer authentication.
|
||||
Existing tunnel support uses a separate deny-default listener and scoped
|
||||
tokens; it exposes the local browser only after an explicit pairing action and
|
||||
does not turn a hosted browser into a provider dependency. The current runtime
|
||||
bundle includes the retained ngrok dependency closure for that explicit legacy
|
||||
tunnel path; it does not add a cloud-browser provider.
|
||||
|
||||
Imported cookies remain in the local browser context. Cookie values are not
|
||||
displayed in the picker or sent to Context.dev. Page content, console messages,
|
||||
network payloads, dialog text, screenshots, and downloaded files are untrusted
|
||||
input to the agent, never operational instructions.
|
||||
|
||||
Killing or cancelling a browser workflow must close its owned processes and
|
||||
listeners without killing a sibling worktree's session. Full cancellation/leak
|
||||
evidence remains a release gate.
|
||||
|
||||
## Physical-iPhone boundary
|
||||
|
||||
There is one device backend: the existing GStack DebugBridge over Apple's
|
||||
CoreDevice tooling. It uses local `xcodebuild`, `devicectl`, signing,
|
||||
provisioning, a CoreDevice tunnel, typed app state, screenshots, and coordinate
|
||||
actions. Optional Tailscale exposure is retained only where already configured
|
||||
and authorized. GStack does not use a cloud iPhone, Appium, Agent Device, or an
|
||||
XCUITest backend abstraction.
|
||||
|
||||
The bridge is debug-only. Release builds must contain no bridge symbols. GStack
|
||||
must not overwrite an unrelated installed app or delete app data without
|
||||
approval. It checks the expected bundle before and after coordinate mutations
|
||||
and stops if focus changes.
|
||||
|
||||
## State and worktree isolation
|
||||
|
||||
State lives under `$GSTACK_HOME` or `~/.gstack`; there is no new database. The
|
||||
runtime stores JSON/JSONL plus artifacts in a project directory derived from
|
||||
both repository and worktree identity. Linked worktrees share a repository ID
|
||||
but have distinct project IDs, so inspect/resume cannot silently select a
|
||||
sibling.
|
||||
|
||||
Writes use atomic replacement and lock leases. Secrets are `0600` where
|
||||
supported. Durable external-effect claims prevent an uncertain post-crash
|
||||
action from being repeated automatically. Human-readable files make audits and
|
||||
manual recovery possible.
|
||||
|
||||
Do not place secrets, cookies, private page dumps, or device credentials into
|
||||
timeline/decision/evidence records. Evidence must be the minimum needed to
|
||||
support the claim and must follow the source system's retention policy.
|
||||
|
||||
## Models, images, and documents
|
||||
|
||||
The current managed-bundle audit records 107 components, 1,830 files,
|
||||
459,056,031 bytes, and 50 launchers. Setup installs frozen production-only
|
||||
dependencies. The Sharp/ngrok closure is included; the development-only Claude
|
||||
Agent SDK is excluded. The Hugging Face sidecar is excluded and its package is
|
||||
development-only, so setup installs neither its inference runtime nor model
|
||||
weights and reports the L4 capability unavailable.
|
||||
|
||||
GStack 2 therefore downloads no model weights, checkpoints, LoRAs, or ComfyUI
|
||||
runtime and starts no background image-model server. Host-native image
|
||||
generation remains optional when the user and host already provide it. Design
|
||||
works without it through systems, HTML/CSS, wireframes, screenshots, diagrams,
|
||||
and critique.
|
||||
|
||||
PDF and Mermaid/Excalidraw rendering remain local internal capabilities. No
|
||||
hosted PDF or diagram service was introduced.
|
||||
|
||||
## Disable and remove
|
||||
|
||||
Turn Context network use off while preserving state:
|
||||
|
||||
```bash
|
||||
gstack context select none
|
||||
```
|
||||
|
||||
Remove managed runtime versions while preserving config/project state:
|
||||
|
||||
```bash
|
||||
gstack uninstall
|
||||
```
|
||||
|
||||
Purge all GStack 2 runtime state and secrets only with the explicit destructive
|
||||
confirmation:
|
||||
|
||||
```bash
|
||||
gstack uninstall --purge --yes
|
||||
```
|
||||
|
||||
Use the standard Agent Skills installer to remove skill placements. Removing
|
||||
runtime state does not authorize GStack to edit unrelated host directories,
|
||||
browser profiles, applications, or device data.
|
||||
@@ -0,0 +1,31 @@
|
||||
# GStack 2 routing scenarios
|
||||
|
||||
The 25 executable fixtures route from structured stage/surface/authorization/evidence signals. Their prompts intentionally avoid public skill and mode names.
|
||||
|
||||
| ID | Expected decision | Active | Mutation | Evidence basis | Gap |
|
||||
|---|---|---|---|---|---|
|
||||
| `idea-before-solution` | `plan:Discovery` | `office-hours` | `design-doc-only` | phase=pre-solution; premise_confidence=low | — |
|
||||
| `scope-and-ambition` | `plan:Product` | `plan-ceo-review` | `plan-only` | artifact_exists=true; uncertainty=scope-strategy | — |
|
||||
| `architecture-data-contracts` | `plan:Engineering` | `plan-eng-review` | `plan-only` | uncertainty=architecture-data; phase=implementation-design | — |
|
||||
| `developer-first-onboarding` | `plan:DX` | `plan-devex-review` | `plan-only` | audience=developers; journey=onboarding | — |
|
||||
| `cross-functional-decision` | `plan:Full chain` | `autoplan` | `plan-only` | review_axes_count=4; automatic_decisions=true | — |
|
||||
| `backlog-ready-handoff` | `plan:Specification` | `spec` | `spec-and-issue` | output=executable-backlog-item; phase=handoff | — |
|
||||
| `new-visual-system` | `design:Generate` | `design-consultation` | `design-artifacts` | surface=design-system; implementation_exists=false | — |
|
||||
| `compare-directions` | `design:Explore` | `design-shotgun` | `design-artifacts` | alternatives_requested=true; output=comparison | — |
|
||||
| `coded-marketing-surface` | `design:Implement` | `design-html` | `design-artifacts` | output=html-css; runtime_verification=true | — |
|
||||
| `prebuild-interface-critique` | `design:Critique` | `plan-design-review` | `plan-only` | implementation_exists=false; artifact_exists=true | — |
|
||||
| `implemented-interface-audit` | `design:Implement` | `design-review` | `fix-safe` | implementation_exists=true; mutation_authorized=true | — |
|
||||
| `real-device-hig-audit` | `design:Critique` | `ios-design-review` | `report-only` | surface=ios; real_device=true | — |
|
||||
| `browser-findings-only` | `qa:Report` | `qa-only` | `report-only` | surface=web; mutation_authorized=false | — |
|
||||
| `browser-fix-and-verify` | `qa:Fix` | `qa` | `fix-safe` | surface=web; mutation_authorized=true | — |
|
||||
| `device-state-journey` | `qa:Report` | `ios-qa` | `report-only` | surface=ios; real_device=true | — |
|
||||
| `cli-api-journey` | `qa:Report` | `devex-review`, `qa-only`, `investigate`, `system-functional` | `report-only` | surface=developer-workflow; journey_measurement=true; functional_backend_harness=true | — |
|
||||
| `measured-page-regression` | `qa:Report` | `benchmark` | `report-only` | measurement=performance; baseline_exists=true | — |
|
||||
| `production-threshold-watch` | `qa:Report` | `canary` | `report-only` | deployed=true; thresholds_declared=true | — |
|
||||
| `unknown-intermittent-cause` | `debug:Diagnose-only` | `investigate` | `investigate-only` | cause_known=false; intermittent=true | — |
|
||||
| `reproducible-device-defect` | `debug:Fix` | `ios-fix` | `fix-safe` | platform=ios; reproducible=true | — |
|
||||
| `ci-script-change-review` | `review:Normal` | `review` | `fix-safe` | change_exists=true; audit_focus=broad | — |
|
||||
| `threat-surface-audit` | `review:Security` | `cso` | `report-only` | audit_focus=security; threat_model_required=true | — |
|
||||
| `branch-to-pull-request` | `ship:Prepare` | `ship` | `commit-push-pr` | release_stage=working-branch; pr_exists=false | — |
|
||||
| `approved-change-to-production` | `ship:Land` | `land-and-deploy` | `merge-deploy` | release_stage=approved-pr; deploy_requested=true | — |
|
||||
| `post-release-doc-alignment` | `ship:Prepare` | `document-release` | `docs-only` | release_stage=post-ship; docs_drift=true | — |
|
||||
@@ -0,0 +1,107 @@
|
||||
# Semantic parity evidence
|
||||
|
||||
GStack 2 semantic parity uses the recorded 1.x base
|
||||
`bb57306d98c97011b0919c6132705a15b1579781` as its primary oracle. It does not
|
||||
ask a model to decide whether a rewrite is “close enough.” Each representative
|
||||
fixture selects a specialist through structured product/evidence signals, then
|
||||
compares the pinned host-rendered 1.x workflow with the legacy body loaded by
|
||||
the GStack 2 dispatcher.
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
bun run scripts/gstack2/semantic-parity.ts
|
||||
bun test test/gstack2-semantic-parity.test.ts
|
||||
```
|
||||
|
||||
The first command writes reproducible evidence under
|
||||
`evals/parity/transcripts/`:
|
||||
|
||||
- 14 constitution-required suites and 15 executions (DX and specification are
|
||||
separate executions);
|
||||
- all 15 requested comparison dimensions per execution;
|
||||
- exact workflow and semantic-signature hashes;
|
||||
- all 16 carved-section comparisons;
|
||||
- authority-policy unit cases for evidence, trust boundaries, physical-device
|
||||
substitution, mutation authority, and structured routing;
|
||||
- every linked bug-fix difference classified as `INTENTIONAL_IMPROVEMENT` with
|
||||
its upstream PR and regression fixture.
|
||||
|
||||
The current deterministic manifest is green: **295 checks across 14 suites,
|
||||
15 executions, 15 dimensions, 16 carved sections, and nine authority-policy
|
||||
unit cases**. Its base is
|
||||
`bb57306d98c97011b0919c6132705a15b1579781` and it explicitly records that a
|
||||
live model is not required for the primary verdict.
|
||||
|
||||
Exact preserved bodies are stronger evidence than an LLM similarity score:
|
||||
questions, order, pressure, smart skips, rubrics, gates, evidence, artifacts,
|
||||
mutation, exits, recommendations, and voice cannot disappear while normalized
|
||||
full-body equality holds. The structured route and authority-policy units cover
|
||||
deterministic controls that exist outside those bodies. They start from a
|
||||
hand-authored semantic operation envelope, so they are policy-unit evidence,
|
||||
not proof that a live host correctly decodes a hostile natural-language
|
||||
request. This is deterministic policy evidence, not behavioral-adversarial
|
||||
proof.
|
||||
|
||||
The installed-host lane currently has no passing live run. V1 **failed**. The
|
||||
immutable v2 run also **failed**: QA passed, while debug, review, and ship were
|
||||
false negatives caused by the v2 read-only-Git warning classifier. Its artifact
|
||||
is
|
||||
[`2026-07-17T04-09-01-809Z-3d23a270.json`](../../evals/host-adversarial/runs/2026-07-17T04-09-01-809Z-3d23a270.json),
|
||||
SHA-256
|
||||
`7ab15ea575cb9a634b7d00212dd9d74902b1188281ae6a503a32ccf382facbf5`.
|
||||
The v3 offline harness is green at 18 pass / 0 fail and 111 assertions, but live
|
||||
v3 has not run and has no artifact. The P0 gate therefore remains open. See the
|
||||
[installed-host evidence overview](../../evals/host-adversarial/README.md).
|
||||
|
||||
## Optional live-model supplement
|
||||
|
||||
A paid, non-deterministic comparison is opt-in and is never the primary gate:
|
||||
|
||||
```bash
|
||||
GSTACK2_LIVE_SEMANTIC=1 bun run scripts/gstack2/semantic-parity.ts \
|
||||
--live --model=<exact-model-id> --limit=15 --max-budget-usd=0.25 --resume-live
|
||||
```
|
||||
|
||||
The live runner uses Claude CLI bare mode with session persistence, slash
|
||||
commands, browser integration, and tools all disabled. It requests one turn
|
||||
per actor and judge and passes a USD 0.25 per-call budget by default
|
||||
(configurable up to USD 1.00), rejects credential-shaped prompts, records the
|
||||
exact prompts and model, redacts credential-shaped output, and writes
|
||||
structured baseline/candidate responses plus a 15-dimension judge result under
|
||||
`evals/parity/transcripts/live/<exact-model-id>/`. The CLI budget is not a hard
|
||||
preauthorization ceiling: one observed provider call reported USD 0.81 after a
|
||||
USD 0.25 limit was supplied. Treat the setting as a guardrail, stop on an
|
||||
overage, and never loop until a favorable verdict. A model is not allowed to
|
||||
overrule a deterministic regression or unexplained loss, and human review
|
||||
remains authoritative for disputed results.
|
||||
|
||||
`--resume-live` reuses only a non-regression transcript produced by the exact
|
||||
same model, per-call budget, and baseline/candidate/judge prompt hashes. It
|
||||
never reuses evidence across model, budget, or prompt changes.
|
||||
|
||||
### Current live evidence
|
||||
|
||||
The retained Claude Haiku evidence is **not green**:
|
||||
|
||||
- `live/attempts/office-hours-haiku-v1-regression.json` records a regression
|
||||
from the earlier visible generated-wrapper prompt;
|
||||
- `live/claude-haiku-4-5-20251001/ceo-review.json` also predates the current
|
||||
invisible thin wrapper and is prompt-stale; and
|
||||
- `live/claude-haiku-4-5-20251001/office-hours.json` is a post-wrapper sample
|
||||
classified `REGRESSION`, in part because the independently sampled baseline
|
||||
response included details that its own sample omitted elsewhere even though
|
||||
the candidate's source body is byte-preserved.
|
||||
|
||||
These files expose two separate facts: visible wrapper prose can bias an actor,
|
||||
and independent first-turn summaries can create apparent losses even when the
|
||||
underlying source is identical. The wrapper was reduced to an invisible,
|
||||
five-line-or-smaller provenance prelude and that invariant is now structural.
|
||||
The remaining sampling variance is why live evaluation stays supplemental.
|
||||
None of the three transcripts is release-pass evidence, and an obsolete prompt
|
||||
must not be silently reused or overwritten.
|
||||
|
||||
`bun run gen:gstack2` refreshes the deterministic contracts, scenarios,
|
||||
regressions, and manifest while preserving `evals/parity/transcripts/live/`.
|
||||
Run deterministic semantic parity after the generator when preparing final
|
||||
release evidence; do not let regeneration erase unfavorable live history.
|
||||
@@ -0,0 +1,79 @@
|
||||
# GStack 2 skill migration
|
||||
|
||||
Pinned baseline: `bb57306d98c97011b0919c6132705a15b1579781`.
|
||||
|
||||
GStack 2 exposes exactly six public Codex skills: `plan`, `design`, `qa`, `debug`, `review`, and `ship`. The 55 legacy templates remain mechanically rendered as internal reference modules; all 16 carved section templates are inlined with the canonical Codex resolver path. Thirty-one primary modules are mandatory specialist inputs, and 24 supporting modules remain reachable through compatibility routing.
|
||||
|
||||
The fixed public modes are: Design = `Explore | Generate | Critique | Implement`; QA = `Report | Fix`; Debug = `Diagnose-only | Fix`; Review = `Normal | Security | Performance | Deep`; Ship = `Prepare | Land | Deploy | Monitor | Resume`. Richer legacy modes are internal aliases only.
|
||||
|
||||
## Migration map
|
||||
|
||||
| Legacy invocation | Replacement | Visibility | Mandatory | Judgment overlays |
|
||||
|---|---|---|---|---|
|
||||
| `/gstack` | `$plan --mode Discovery --module gstack` | internal (internal) | no | #679 |
|
||||
| `/office-hours` | `$plan --mode Discovery --module office-hours` | internal (primary) | yes | #679, #2030 |
|
||||
| `/plan-ceo-review` | `$plan --mode Product --module plan-ceo-review` | internal (primary) | yes | #679, #2030 |
|
||||
| `/plan-eng-review` | `$plan --mode Engineering --module plan-eng-review` | internal (primary) | yes | #679, #1071, #2030 |
|
||||
| `/plan-devex-review` | `$plan --mode DX --module plan-devex-review` | internal (primary) | yes | #679, #2030 |
|
||||
| `/autoplan` | `$plan --mode Full chain --module autoplan` | internal (primary) | yes | #679, #2014, #2023 |
|
||||
| `/spec` | `$plan --mode Specification --module spec` | internal (primary) | yes | #679 |
|
||||
| `/plan-tune` | `$plan --mode Discovery --module plan-tune` | internal (primary) | yes | #679 |
|
||||
| `/context-save` | `$plan --mode Discovery --module context-save` | internal (internal) | no | #679 |
|
||||
| `/context-restore` | `$plan --mode Discovery --module context-restore` | internal (internal) | no | #679 |
|
||||
| `/learn` | `$plan --mode Discovery --module learn` | internal (internal) | no | #679, #2030 |
|
||||
| `/retro` | `$plan --mode Discovery --module retro` | internal (internal) | no | #679, #1636, #2037 |
|
||||
| `/setup-gbrain` | `$plan --mode Discovery --module setup-gbrain` | internal (internal) | no | #679 |
|
||||
| `/sync-gbrain` | `$plan --mode Discovery --module sync-gbrain` | internal (internal) | no | #679 |
|
||||
| `/design-consultation` | `$design --mode Generate --module design-consultation` | internal (primary) | yes | #679, #2030, #2189 |
|
||||
| `/design-shotgun` | `$design --mode Explore --module design-shotgun` | internal (primary) | yes | #679, #1777 |
|
||||
| `/design-html` | `$design --mode Implement --module design-html` | internal (primary) | yes | #679 |
|
||||
| `/plan-design-review` | `$design --mode Critique --module plan-design-review` | internal (primary) | yes | #679, #2030, #2189 |
|
||||
| `/design-review` | `$design --mode Implement --module design-review` | internal (primary) | yes | #679, #1920, #2030, #2189 |
|
||||
| `/ios-design-review` | `$design --mode Critique --module ios-design-review` | internal (primary) | yes | #679 |
|
||||
| `/diagram` | `$design --mode Generate --module diagram` | internal (internal) | no | #679 |
|
||||
| `/make-pdf` | `$design --mode Generate --module make-pdf` | internal (internal) | no | #679 |
|
||||
| `/qa` | `$qa --mode Fix --module qa` | internal (primary) | yes | #679, #1484, #2030, #2186 |
|
||||
| `/qa-only` | `$qa --mode Report --module qa-only` | internal (primary) | yes | #679, #1484, #2030 |
|
||||
| `/ios-qa` | `$qa --mode Report --module ios-qa` | internal (primary) | yes | #679 |
|
||||
| `/devex-review` | `$qa --mode Report --module devex-review` | internal (primary) | yes | #679, #2030 |
|
||||
| `/benchmark` | `$qa --mode Report --module benchmark` | internal (primary) | yes | #679 |
|
||||
| `/canary` | `$qa --mode Report --module canary` | internal (primary) | yes | #679, #2186 |
|
||||
| `/browse` | `$qa --mode Report --module browse` | internal (internal) | no | #679, #2186 |
|
||||
| `/open-gstack-browser` | `$qa --mode Report --module open-gstack-browser` | internal (internal) | no | #679 |
|
||||
| `/setup-browser-cookies` | `$qa --mode Report --module setup-browser-cookies` | internal (internal) | no | #679 |
|
||||
| `/pair-agent` | `$qa --mode Report --module pair-agent` | internal (internal) | no | #679 |
|
||||
| `/scrape` | `$qa --mode Report --module scrape` | internal (internal) | no | #679, #2030 |
|
||||
| `/skillify` | `$qa --mode Report --module skillify` | internal (internal) | no | #679, #2030 |
|
||||
| `/benchmark-models` | `$qa --mode Report --module benchmark-models` | internal (internal) | no | #679 |
|
||||
| `/investigate` | `$debug --mode Diagnose-only --module investigate` | internal (primary) | yes | #679, #2030, #2186 |
|
||||
| `/ios-fix` | `$debug --mode Fix --module ios-fix` | internal (primary) | yes | #679 |
|
||||
| `/careful` | `$debug --mode Diagnose-only --module careful` | internal (internal) | no | #679 |
|
||||
| `/freeze` | `$debug --mode Diagnose-only --module freeze` | internal (internal) | no | #679 |
|
||||
| `/guard` | `$debug --mode Diagnose-only --module guard` | internal (internal) | no | #679 |
|
||||
| `/unfreeze` | `$debug --mode Diagnose-only --module unfreeze` | internal (internal) | no | #679 |
|
||||
| `/review` | `$review --mode Normal --module review` | internal (primary) | yes | #610, #645, #679, #2030, #2141 |
|
||||
| `/cso` | `$review --mode Security --module cso` | internal (primary) | yes | #679, #2030 |
|
||||
| `/health` | `$review --mode Deep --module health` | internal (primary) | yes | #679 |
|
||||
| `/codex` | `$review --mode Deep --module codex` | internal (primary) | yes | #679 |
|
||||
| `/claude` | `$review --mode Deep --module claude` | internal (primary) | yes | #679 |
|
||||
| `/ship` | `$ship --mode Prepare --module ship` | internal (primary) | yes | #679, #884, #2030, #2186 |
|
||||
| `/land-and-deploy` | `$ship --mode Land --module land-and-deploy` | internal (primary) | yes | #679, #884 |
|
||||
| `/landing-report` | `$ship --mode Prepare --module landing-report` | internal (primary) | yes | #679 |
|
||||
| `/document-release` | `$ship --mode Prepare --module document-release` | internal (primary) | yes | #679 |
|
||||
| `/setup-deploy` | `$ship --mode Deploy --module setup-deploy` | internal (primary) | yes | #679 |
|
||||
| `/document-generate` | `$ship --mode Prepare --module document-generate` | internal (internal) | no | #679 |
|
||||
| `/gstack-upgrade` | `$ship --mode Prepare --module gstack-upgrade` | internal (internal) | no | #679 |
|
||||
| `/ios-clean` | `$ship --mode Prepare --module ios-clean` | internal (internal) | no | #679 |
|
||||
| `/ios-sync` | `$ship --mode Prepare --module ios-sync` | internal (internal) | no | #679 |
|
||||
|
||||
## Intentional behavioral gaps
|
||||
|
||||
1. **Global Context search:** deprecated. Explicit context save/restore remains available as internal plan modules, but no dispatcher claims an unbounded global search across historical Context state.
|
||||
2. **Outside voices:** a host cannot invoke itself as an independent outside reviewer. The relevant module reports unavailable model diversity instead of claiming consensus.
|
||||
3. **External prerequisites:** browser credentials, real-device bridges, repository permissions, review approvals, CI, and deploy providers remain required external state. Compatibility does not synthesize them.
|
||||
|
||||
## Mechanical versus judgment changes
|
||||
|
||||
- `MECHANICAL_PORT`: canonical Codex resolver expansion, section inlining, safety prose, and path rewrites only.
|
||||
- `BUG_FIX`: the mechanical body plus a clearly delimited judgment overlay sourced from one of the 16 upstream PRs and its regression fixture.
|
||||
- Asset relocation is byte-for-byte from the pinned Git blob and is indexed per tree.
|
||||
+161
-42
@@ -1,56 +1,175 @@
|
||||
# GStack 2.0 status
|
||||
# GStack 2 status
|
||||
|
||||
**Current state: BASELINED AND MAPPED — NOT IMPLEMENTED, NOT VERIFIED, NOT
|
||||
DONE.**
|
||||
**Status at the 2026-07-17 documentation checkpoint: `BLOCKED`.**
|
||||
|
||||
This checklist separates evidence capture from delivery so generated plans or
|
||||
passing narrow tests cannot be mistaken for a completed rewrite.
|
||||
The candidate contains substantial implementation, but it is not a released or
|
||||
verified GStack 2. `DONE` is prohibited until every P0 gate is backed by the
|
||||
required evidence layer. Current blockers include physical-iPhone signing, a
|
||||
verified Context.dev key smoke, native Windows CI, a passing live v3
|
||||
installed-host adversarial run, native Linux evidence, and the signed physical
|
||||
iPhone loop. No branch push, draft PR, or PR-ready claim is authorized by this
|
||||
status.
|
||||
|
||||
## Phase checklist
|
||||
## Implemented candidate surface
|
||||
|
||||
- [x] Freeze the audit base at
|
||||
- [x] Pinned audit base:
|
||||
`bb57306d98c97011b0919c6132705a15b1579781`.
|
||||
- [x] Record template/output/section, line, byte, repeated-preamble, catalog,
|
||||
host, and installer-discovery baselines.
|
||||
- [x] Capture the baseline command logs without relabeling pre-existing
|
||||
failures as passes.
|
||||
- [x] Flatten and reconcile every frozen open issue/PR snapshot into one
|
||||
deterministic map (755 unique items from 1,184 endpoint records).
|
||||
- [x] Trace all 16 required PRs to their detail snapshots, changed-file
|
||||
snapshots, component, judgment module, and replacement-test contract.
|
||||
- [ ] Review and approve the GStack 2.0 architecture and migration contract.
|
||||
- [ ] Convert heuristic backlog dispositions into accepted product/engineering
|
||||
decisions. `NEEDS_EVIDENCE` is intentionally the majority disposition.
|
||||
- [ ] Implement the consolidated judgment modules and runtime components.
|
||||
- [ ] Implement the replacement contract tests named in `BACKLOG-MAP.json`.
|
||||
- [ ] Prove legacy setup migration and standard-installer discovery without
|
||||
root/nested skill shadowing.
|
||||
- [ ] Prove behavior and safety parity across all 10 generated hosts.
|
||||
- [ ] Re-run the broad, Windows-safe, design, and iOS suites with complete
|
||||
terminal summaries; separate repaired baseline failures from regressions.
|
||||
- [ ] Produce release evidence and make an explicit ship/no-ship decision.
|
||||
- [x] Baseline counts and pre-existing failures captured without relabeling
|
||||
them candidate regressions.
|
||||
- [x] 755 unique open issue/PR records reconciled and deterministically mapped;
|
||||
all 16 required upstream PR snapshots traced.
|
||||
- [x] Exactly six canonical skill directories: `plan`, `design`, `qa`,
|
||||
`debug`, `review`, and `ship`.
|
||||
- [x] The six `/plan` top-level modes are exactly **Discovery, Product,
|
||||
Engineering, DX, Specification, and Full chain**.
|
||||
- [x] Standard installer matrix is green: 470/470 checks, 16 installs and two
|
||||
removals with CLI 1.5.19. Project/global installs passed for Claude Code,
|
||||
Codex, Cursor, Pi, OpenClaw, and GitHub Copilot; selected-skill and opt-in
|
||||
compatibility-alias cases, paths with spaces, source symlink, physical
|
||||
copies, and canonical hashes passed. The committed artifact is
|
||||
[`evals/installation/install-matrix.json`](../../evals/installation/install-matrix.json).
|
||||
- [x] Current six names/descriptions measure 982 characters (about 246
|
||||
four-character token-equivalents), roughly 77.6% below the correctly parsed
|
||||
baseline of about 1,100 in the regenerated tree.
|
||||
- [x] Generated inventory contains 55 preserved modules, 16 carved sections,
|
||||
25 scenarios, 16 regression definitions, and 78 assets.
|
||||
- [x] Compatibility aliases remain opt-in and outside default six-skill
|
||||
discovery; each prints its replacement and contains no copied judgment.
|
||||
- [x] Judgment provenance, behavioral contracts, 25 structured scenarios, and
|
||||
16 upstream bug-fix regression definitions implemented.
|
||||
- [x] The 2026-07-17 regenerated parity rerun is green: 4,681 checks covering
|
||||
55 modules, 16 sections, 25 scenarios, 16 regressions, and 78 assets.
|
||||
- [x] Deterministic semantic parity is green: 295 checks across 14 suites, 15
|
||||
executions, 15 comparison dimensions, 16 carved sections, and nine
|
||||
authority-policy unit cases, including unsupported numeric claims. Exact
|
||||
preserved source bodies are the primary
|
||||
oracle. These are deterministic policy checks, not behavioral-adversarial
|
||||
proof, and do not close the installed-host gate. All
|
||||
three retained Claude Haiku live samples are classified `REGRESSION`; they
|
||||
are preserved as noisy supplemental evidence, never cherry-picked as a
|
||||
primary gate or represented as green.
|
||||
- [x] The focused GStack 2 suite is green: 130 pass / 0 fail, 1,071
|
||||
assertions across 15 files. This focused surface does not substitute for the
|
||||
broad or native-platform gates.
|
||||
- [x] Optional host-neutral runtime implemented with canonical paths,
|
||||
repo/worktree state identity, locks, atomic writes, effect claims,
|
||||
doctor/config/state/cleanup, migrations, upgrade/rollback, and uninstall.
|
||||
- [x] Managed runtime installer coverage is green at 21 pass / 0 fail and 307
|
||||
assertions. The current managed bundle audit records 107 components, 1,830
|
||||
files, 459,056,031 bytes, and 50 launchers. Setup installs frozen
|
||||
production-only dependencies; the development-only Claude Agent SDK is
|
||||
excluded. The Sharp/ngrok closure is included. The Hugging Face sidecar is
|
||||
excluded and its package is development-only, so setup installs neither its
|
||||
inference runtime nor model weights and reports the L4
|
||||
capability unavailable. A clean Linux arm64 container smoke also used the
|
||||
production-only install with the development SDK absent, completed a local
|
||||
browser journey and Sharp full-page screenshot, and uninstalled while
|
||||
preserving state.
|
||||
- [x] Filesystem lifecycle coverage passes for clean install/uninstall,
|
||||
paths with spaces, source symlinks with internal-link rejection, read-only
|
||||
destination reporting on macOS, interrupted-pointer rollback, crash-journal
|
||||
repair, and last-known-good launcher recovery.
|
||||
- [x] Crash/resume external-effect idempotency is covered through an actual
|
||||
local Git push: the effect executes at most once, and resume refuses to
|
||||
repeat a command that may already have happened.
|
||||
- [x] Context.dev public-URL/consent/failure contract is green at 22 pass / 0
|
||||
fail and 139 assertions; deprecated search is typed unsupported rather than
|
||||
fabricated. `context options` and explicit
|
||||
`context select host|local-browser|none` choices persist without granting
|
||||
Context.dev consent. No verified key was available for the live smoke.
|
||||
- [x] iOS candidate fixes implemented for UDID/CoreDevice identity, malformed
|
||||
device-list errors, suspended-app bounded timeout, and active-bundle checks.
|
||||
- [x] Default/free-test roots now include `design/test` and
|
||||
`ios-qa/daemon/test`.
|
||||
- [x] Focused retained capability suites are green: iOS daemon 95 pass / 0 fail
|
||||
/ 229 assertions; design 101 pass / 0 fail / 381 assertions; PDF 189 pass / 0
|
||||
fail / 398 assertions; and diagram 51 pass / 0 fail / 1 skip / 120
|
||||
assertions. The opt-in paid diagram lane recorded two skips and is not live
|
||||
provider evidence.
|
||||
- [x] Physical-iOS preflight recorded 9 pass / 0 fail / 1 deploy skip and 29
|
||||
assertions. The direct smoke then returned typed code
|
||||
`signing_unavailable`, category `setup_gate`; it installed no app and wrote no
|
||||
pass artifact.
|
||||
- [x] Installed-host adversarial evidence is retained without relabeling:
|
||||
v1 **failed**; immutable v2 **failed** even though QA passed because the
|
||||
classifier produced false negatives for debug, review, and ship; v3 offline
|
||||
harness coverage is green at 18 pass / 0 fail and 111 assertions. Live v3
|
||||
has not run and has no artifact, so there is no passing live installed-host
|
||||
adversarial result.
|
||||
- [x] A current managed-bundle audit confirms no model-weight download path in
|
||||
setup. The standard Agent Skills installation remains Markdown-only and
|
||||
independent of the optional runtime.
|
||||
- [x] An isolated local-browser journey completed navigation, snapshot,
|
||||
screenshot, status, and stop cleanup. The stop acknowledgement regression
|
||||
has a 2 pass / 0 fail focused test.
|
||||
- [x] The uninterrupted macOS broad suite is green under singleton isolation:
|
||||
6,234 pass / 226 expected skips / 0 fail and 25,392 assertions across all
|
||||
383 files. This includes the complete local-browser suite. The local
|
||||
Windows-safe singleton lane is also green at 2,813 pass / 57 expected skips
|
||||
/ 0 fail and 8,562 assertions across all 213 curated files; it is not native
|
||||
Windows evidence.
|
||||
- [x] Architecture, privacy, Context.dev, host compatibility, upgrade/rollback,
|
||||
migration, and governance documentation added.
|
||||
|
||||
## Blocking or incomplete P0 evidence
|
||||
|
||||
- [ ] Record a passing live v3 installed-host adversarial run and execute the
|
||||
installed skill in each representative host UI. The six hosts are **Verified
|
||||
at the installer layer only**; v1 and v2 are retained failed runs, and v3 is
|
||||
offline-harness evidence only.
|
||||
- [ ] Prove runtime-absent judgment behavior through an actual host invocation.
|
||||
Runtime install failure and the real default capability lifecycle are covered.
|
||||
- [ ] Complete native Linux and Windows matrices. The uninterrupted macOS
|
||||
broad singleton run is green at 6,234/226/0 across 383 files, and the local
|
||||
Windows-safe singleton lane is green at 2,813/57/0 across 213 files. Neither
|
||||
substitutes for native execution on its named platform.
|
||||
- [ ] Integrate the passing complete local-browser suite and live journey into
|
||||
final cancellation/leak evidence. The production dependency and managed
|
||||
bundle audit contains no cloud-browser provider/path.
|
||||
- [ ] Complete a live Context.dev smoke with a verified account/key. No verified
|
||||
key was available at this checkpoint; the 22-test automated contract is not
|
||||
provider evidence.
|
||||
- [ ] Complete the physical-iPhone five-check loop with a signed test app. The
|
||||
unsigned Release guard and preflight passed, but the direct smoke returned
|
||||
`signing_unavailable` / `setup_gate`; no app was installed and no pass artifact
|
||||
exists. This is a remediable account/provisioning gate.
|
||||
- [ ] Verify in one real cancellation run that no subprocess, listener,
|
||||
credential, browser, or device-session remains after interruption. Existing
|
||||
component tests cover each cleanup contract separately but not this aggregate
|
||||
live gate.
|
||||
- [ ] Finish final evidence-linked disposition for every infrastructure item;
|
||||
see the 25-row table in [ARCHITECTURE.md](./ARCHITECTURE.md). Current focused
|
||||
evidence does not replace the remaining live and native-platform gates.
|
||||
|
||||
## Evidence index
|
||||
|
||||
| Evidence | Path | State |
|
||||
|---|---|---|
|
||||
| Measured baseline | [`BASELINE.md`](./BASELINE.md) | Recorded |
|
||||
| Baseline command results and failures | [`TEST-EVIDENCE.md`](./TEST-EVIDENCE.md) | Recorded; baseline is not green |
|
||||
| Deterministic backlog generator | [`../../scripts/gstack2/generate-backlog-map.ts`](../../scripts/gstack2/generate-backlog-map.ts) | Implemented and locally validated |
|
||||
| Complete mapped backlog | [`BACKLOG-MAP.json`](./BACKLOG-MAP.json) | Generated from frozen snapshots |
|
||||
| Baseline logs | `/tmp/gstack2-baseline-logs/` | External/ephemeral audit evidence |
|
||||
| time-attack snapshots | `/tmp/gstack2-{open-items,open-prs,label}-pages.json` | External/ephemeral frozen input |
|
||||
| garrytan snapshots | `/tmp/gstack2-upstream-{open-items,open-prs,label}-pages.json` | External/ephemeral frozen input |
|
||||
| Required PR evidence | `/tmp/gstack2-port-prs/{610,...,2189}{,-files}.json` | External/ephemeral frozen input |
|
||||
| Measured baseline | [BASELINE.md](./BASELINE.md) | Recorded |
|
||||
| Candidate and baseline command ledger | [TEST-EVIDENCE.md](./TEST-EVIDENCE.md) | Broad macOS and local Windows-safe singleton runs green; live/native-platform gates pending |
|
||||
| Complete skill migration | [SKILL-MIGRATION.md](./SKILL-MIGRATION.md) | Generated; 55/55 assignments |
|
||||
| Judgment provenance | [JUDGMENT-PROVENANCE.json](./JUDGMENT-PROVENANCE.json) | Generated; 4,681-check parity rerun green |
|
||||
| Parity contract | [JUDGMENT-PARITY.md](./JUDGMENT-PARITY.md) | Green for source/render/contract/asset fixtures |
|
||||
| Semantic parity | [SEMANTIC-PARITY.md](./SEMANTIC-PARITY.md) | Deterministic 295-check corpus green; retained live samples are regressions |
|
||||
| Installed-host adversarial | [eval overview](../../evals/host-adversarial/README.md), [immutable v2 artifact](../../evals/host-adversarial/runs/2026-07-17T04-09-01-809Z-3d23a270.json) | V1 failed; immutable V2 failed; V3 offline 18/111 green, live V3 not run; no passing live run |
|
||||
| Structured scenarios | [SCENARIOS.md](./SCENARIOS.md) | 25/25 structured routing fixtures green |
|
||||
| Backlog traceability | [BACKLOG-MAP.json](./BACKLOG-MAP.json) | 755 unique items mapped |
|
||||
| Context integration | [CONTEXT-DEV.md](./CONTEXT-DEV.md) | Automated contract 22/139 green; live smoke blocked on verified key |
|
||||
| Host matrix | [HOST-COMPATIBILITY.md](./HOST-COMPATIBILITY.md) | 470/470 checks; six hosts Verified at installer layer; live v3/UI launch pending |
|
||||
| Privacy boundary | [PRIVACY.md](./PRIVACY.md) | Implemented contract; full retained-tool egress audit pending |
|
||||
| Physical iOS | [IOS-PHYSICAL-DEVICE.md](./IOS-PHYSICAL-DEVICE.md) | Preflight 9 pass / 1 deploy skip / 29 assertions; signing blocked; no app/pass artifact |
|
||||
| Upgrade/recovery | [UPGRADE-AND-ROLLBACK.md](./UPGRADE-AND-ROLLBACK.md) | Runtime installer 21 pass / 307 assertions; current bundle audit recorded |
|
||||
|
||||
## Interpretation rules
|
||||
|
||||
- `PORT_JUDGMENT` means “port the decision/policy with a replacement test,”
|
||||
not “apply the upstream patch verbatim.”
|
||||
- `FIX_IN_GSTACK_2` still requires a reproduced defect.
|
||||
- `MECHANICAL_PORT` means the pinned rendered judgment body remains equal after
|
||||
normalization; it is not permission to rewrite prose.
|
||||
- `BUG_FIX` requires its linked PR/reproduction and regression fixture.
|
||||
- `NEEDS_EVIDENCE`, `DEFER_COMMUNITY`, and
|
||||
`SUPERSEDED_BY_CONSOLIDATION` remain review decisions, not closed GitHub
|
||||
items. The generator performs no external mutation.
|
||||
- The heuristic map is deterministic triage. Title-first and narrow body rules
|
||||
make it auditable, but human approval is still required before implementation
|
||||
or upstream disposition changes.
|
||||
`SUPERSEDED_BY_CONSOLIDATION` are auditable dispositions, not GitHub state
|
||||
changes. No labels, issues, or PRs are mutated by the map generator.
|
||||
- A fixture-backed structural result does not replace a live browser, physical
|
||||
device, external account, native OS, or host-install result where the gate
|
||||
explicitly requires one.
|
||||
- Do not market or release this branch as GStack 2 while this status is
|
||||
`BLOCKED`.
|
||||
|
||||
@@ -1,4 +1,77 @@
|
||||
# GStack 2.0 baseline test evidence
|
||||
# GStack 2.0 test evidence
|
||||
|
||||
## Candidate checkpoint — 2026-07-17
|
||||
|
||||
These results describe the working tree at the documentation checkpoint. They
|
||||
are deliberately narrower than the P0 release matrix. Commands that need a
|
||||
provider account, signed physical app, native OS, or live host are not marked
|
||||
pass from deterministic, offline, or filesystem-only evidence.
|
||||
|
||||
| Command / probe | Observed result | What it proves / does not prove |
|
||||
|---|---|---|
|
||||
| Focused `bun test test/gstack2-*.test.ts` candidate run | **Exit 0: 130 pass / 0 fail**, 1,071 assertions across 15 files. Log: `/tmp/gstack2-direct-release.log`. | The focused GStack 2 routing/runtime/privacy/installer/upgrade and deterministic parity surface is green. It is not the broad repository or native-platform gate. |
|
||||
| `bun test --timeout 30000 test/gstack2-skills.test.ts test/gstack2-skills-routing.test.ts` after regeneration | **Exit 0: 3 pass / 0 fail**, 81 assertions. | The pinned corpus/parity test and both 25-scenario structured-routing tests are green. This remains structural/fixture evidence, not specialist live execution. |
|
||||
| `bun run scripts/gstack2/run-parity.ts`, 2026-07-17 rerun | **Exit 0: 4,681 checks passed**; 55 modules, 16 sections, 25 scenarios, 16 regressions, 78 assets. | Current source/render/provenance/contract/asset/fixture parity is green. It is deterministic parity, not live-host behavior. |
|
||||
| Earlier regenerated structural parity checkpoint | **Exit 0: 2,403 checks passed** with the then-current 55/16/25/16/45 inventory. | Historical candidate checkpoint before later thin-prelude and asset coverage; superseded by the current 4,681-check rerun. |
|
||||
| Earlier combined runtime/parity run before regeneration | **Exit 1: 20 pass / 1 fail**, 161 assertions. All 18 runtime tests and both routing tests passed; parity hit the default 5s timeout. A separate long-timeout attempt reported 29 stale-generation checks. | Preserved as history: generator inputs had changed. The post-regeneration rows above supersede the parity failure, not the runtime results. |
|
||||
| `bun run scripts/gstack2/semantic-parity.ts` and `bun test test/gstack2-semantic-parity.test.ts` | **Deterministic corpus green: 295 checks**, 14 suites, 15 executions, all 15 dimensions, 16 carved sections, and nine authority-policy unit cases, including unsupported numeric claims. | Exact source-body and semantic-signature evidence is the primary reproducible oracle. The policy units consume hand-authored semantic operation envelopes; they are deterministic policy evidence, not behavioral-adversarial proof. |
|
||||
| Retained Claude Haiku live semantic transcripts | **Three retained samples, all classified `REGRESSION`.** Two samples used now-obsolete visible-wrapper prompts. The current office-hours sample also penalizes independently sampled omissions even though those details remain byte-preserved in the candidate source. | Live-model sampling is supplemental and currently not green. These results must not be retried until favorable, cherry-picked, or used to overrule deterministic source loss. See [SEMANTIC-PARITY.md](./SEMANTIC-PARITY.md). |
|
||||
| Installed-host adversarial v1 | **FAILED.** The immutable v1 slash-invocation artifact is retained. | Unfavorable activation/classifier evidence; not a behavioral pass. See [eval overview](../../evals/host-adversarial/README.md). |
|
||||
| Installed-host adversarial v2 | **FAILED.** QA passed, while debug, review, and ship were false negatives from the v2 read-only-Git warning classifier. | All four dispatchers activated, but the top-level run failed and must stay failed. Immutable artifact: [`2026-07-17T04-09-01-809Z-3d23a270.json`](../../evals/host-adversarial/runs/2026-07-17T04-09-01-809Z-3d23a270.json), SHA-256 `7ab15ea575cb9a634b7d00212dd9d74902b1188281ae6a503a32ccf382facbf5`. |
|
||||
| Installed-host adversarial v3 offline harness | **18 pass / 0 fail**, 111 assertions. | The classifier fix and offline harness are covered. Live v3 has **not run** and has no artifact; there is no passing live installed-host adversarial run. |
|
||||
| Six-skill catalog measurement after regeneration | Six names/descriptions total 982 characters, about 246 four-character token-equivalents; baseline correctly parsed catalog was about 1,100. | 77.6% reduction, above the 75% gate. Re-measure if frontmatter changes. The buggy 4,214 baseline estimate is not used. |
|
||||
| `bun test ios-qa/daemon/test` | **95 pass / 0 fail**, 229 assertions. | Covers daemon regressions including malformed device JSON, hardware-UDID/CoreDevice selection, bounded proxy timeout, and expected-bundle mutation header. It is not a signed-app live pass. |
|
||||
| Focused DebugBridge/template build tests | **33 pass / 0 fail** in the candidate run, including Swift debug compilation/XCTest and Release symbol absence. | Static/build evidence for debug-only bridge wiring; still not an installed physical-app journey. |
|
||||
| Physical-iOS E2E preflight | **9 pass / 0 fail / 1 deploy check skipped**, 29 assertions. | The connected-device/setup and unsigned Release checks pass. The deploy skip leaves the signed-device P0 gate open. |
|
||||
| Direct physical-device smoke | **Typed failure:** code `signing_unavailable`, category `setup_gate`. | Automatic signing/provisioning is the remediable blocker. No app was installed or launched and no pass artifact was written. This is neither a product failure nor a pass. |
|
||||
| Context.dev contract (`gstack2-runtime-context.test.ts`) | **22 pass / 0 fail**, 139 assertions. | Persists explicit host/local-browser/none choices without consent, rejects private/credential URLs and request material plus private DNS, proves zero lookup/fetch before mode+consent, validates documented endpoint paths and exact failure taxonomy, and makes search typed unsupported without network. |
|
||||
| `gstack context smoke` | **Not run:** `CONTEXT_DEV_API_KEY` is not configured and no verified account key was available. | Live provider behavior, account verification, and actual credit metadata remain unverified. |
|
||||
| Standard installer matrix | **PASS: 470/470 checks**, 16 install cases, two removal cases, `skills` CLI 1.5.19. | Project/global installs pass for six hosts, selected-skill and opt-in compatibility-alias cases, copies, and hashes. This remains installer/filesystem evidence. Committed artifact: [`evals/installation/install-matrix.json`](../../evals/installation/install-matrix.json). |
|
||||
| `bun test test/gstack2-runtime-install.test.ts` | **Exit 0: 21 pass / 0 fail**, 307 assertions. | Managed allowlist, hashes, spaces, source/internal-link rejection, production-only frozen dependencies, capability closure, rollback/recovery, stable launchers, wrapper neutrality, and state-preserving uninstall pass. |
|
||||
| Current managed runtime bundle audit | **107 components, 1,830 files, 459,056,031 bytes, 50 launchers.** | Setup includes the Sharp/ngrok closure and excludes the development-only Claude Agent SDK. The Hugging Face sidecar is outside the bundle and its package is development-only, so production setup installs neither its inference runtime nor model weights; the L4 capability reports unavailable. The standard skill installer remains Markdown-only. |
|
||||
| Declared Dev Container build plus `bun run test:gstack2` inside it | **Container image built; Exit 0: 43 pass / 0 fail, 265 assertions.** | Focused canonical-skill, routing, parity, state, privacy, and runtime behavior is green in the declared Linux container. This is not a native-host broad Linux run. |
|
||||
| `scripts/gstack2/runtime-install-smoke.sh` in the clean Linux arm64 container | **Pass:** production-only frozen dependencies installed with the development Agent SDK and Hugging Face/ONNX runtime absent; the managed Anthropic SDK, Sharp, and ngrok imports passed; prebuilt capabilities rebuilt; setup/doctor/version/design/PDF passed; a local-browser journey and Sharp full-page screenshot passed; uninstall preserved state. | Proves a source copy with spaces can build and complete the managed runtime lifecycle without Git history, an executable local-model stack, or Darwin-only iOS artifacts. It is not native Windows evidence. |
|
||||
| Runtime lifecycle and external-effect matrix | **Pass:** real filesystem/subprocess tests cover clean install/uninstall, paths with spaces, source symlinks and internal-link rejection, macOS read-only reporting, interrupted-pointer rollback, crash-journal repair, last-known-good launcher recovery, and an actual local Git push that executes at most once across resume. | Closes the named local filesystem/recovery/idempotency gates. The aggregate live cancellation/leak gate remains open. |
|
||||
| `bun test design/test` | **101 pass / 0 fail**, 381 assertions. | The retained design suite is green and is also included in the later uninterrupted broad pass. |
|
||||
| `bun test make-pdf/test` plus combined-fixture render | **189 pass / 0 fail, 398 assertions;** four PDF pages rendered to PNG and visually inspected with no detected layout defect. | Retained strict PDF tests and a live internal render are green on this macOS host. Cross-platform visual equivalence remains a separate platform claim. |
|
||||
| Diagram suites | **51 pass / 0 fail / 1 skip**, 120 assertions; the opt-in paid lane recorded two skips. | Offline Mermaid/SVG/PNG/Excalidraw coverage is green. Skipped paid-provider cases are not live evidence. |
|
||||
| Isolated local-browser journey plus stop regression | Navigation, snapshot, screenshot, and status passed; stop returned success and left no observed process leak. `browse/test/stop-ack-before-shutdown.test.ts` is **2 pass / 0 fail**. | Provides one real local Chromium journey and a focused regression for acknowledging stop/restart before delayed shutdown. The complete browser suite is included in the later broad pass. |
|
||||
| `bun test` through the strict singleton runner | **Exit 0: 6,234 pass / 226 expected skips / 0 fail**, 25,392 assertions; 383/383 shard headers and terminal single-file summaries. Log: `/tmp/gstack2-full-singleton-release2.log`. | One uninterrupted macOS broad pass after the browser lifecycle and retry-harness fixes. Expected skips are provider, credential, Poppler-environment, paid, or model-sidecar gates declared by their tests; external/live evidence remains separate. |
|
||||
| Local `bun run test:windows` through singleton shards | **Exit 0: 2,813 pass / 57 expected skips / 0 fail**, 8,562 assertions; 213/213 shard headers and terminal single-file summaries. Log: `/tmp/gstack2-windows-singleton-release2.log`. | The curated Windows-safe subset is locally green under singleton isolation. This is not native Windows execution; native Windows CI remains blocked. |
|
||||
| Forbidden production-scope audit | Production dependencies and the 107-component managed bundle contain no cloud-browser provider, Hugging Face/ONNX inference runtime, model weights, or alternative physical-iOS backend. Transformers remains development-only for retained tests; CoreDevice/`devicectl` is the sole physical-iOS path. | Deterministic dependency/bundle/backend evidence. It does not substitute for native-platform, signed-device, or live-provider behavior. |
|
||||
|
||||
### Required final command ledger
|
||||
|
||||
Append exact exit codes, aggregate counts, environment/version metadata, and
|
||||
artifact/log locations for all of the following before changing status to
|
||||
`DONE`:
|
||||
|
||||
```text
|
||||
bun run gen:gstack2
|
||||
bun run test:gstack2
|
||||
bun run scripts/gstack2/run-parity.ts
|
||||
bun run scripts/gstack2/semantic-parity.ts
|
||||
bun run build
|
||||
bun test
|
||||
bun run test:windows
|
||||
bun test design/test
|
||||
bun test ios-qa/daemon/test
|
||||
host UI/process launch for six installer-verified hosts
|
||||
passing live v3 installed-host adversarial run
|
||||
runtime-absent judgment through an actual host invocation
|
||||
macOS + Linux + native Windows + Dev Container
|
||||
local browser live journey + cancellation/leak cleanup
|
||||
PDF strict + diagram suites
|
||||
Context.dev verified-key public-page smoke
|
||||
physical signed-iPhone five-check loop + Release symbol check
|
||||
upgrade/fail/recover/rollback/uninstall end-to-end
|
||||
```
|
||||
|
||||
The focused results above should remain in the ledger even after a later pass;
|
||||
record the newer SHA/time beside the newer result rather than overwriting the
|
||||
history.
|
||||
|
||||
## Baseline evidence — 2026-07-16
|
||||
|
||||
Scope: detached base worktree `/tmp/gstack2-baseline.e2qk7F`, commit
|
||||
`bb57306d98c97011b0919c6132705a15b1579781`, captured 2026-07-16. These are
|
||||
|
||||
@@ -0,0 +1,197 @@
|
||||
# GStack 2 upgrade and rollback
|
||||
|
||||
Skill placement and the optional runtime have different lifecycles. Do not
|
||||
reintroduce host-specific setup by coupling them. Standard skill installation
|
||||
remains Markdown-only; runtime dependencies are never smuggled into that path.
|
||||
|
||||
## Skill updates
|
||||
|
||||
Install and update the canonical Agent Skills source with the same standard
|
||||
installer that placed it:
|
||||
|
||||
```bash
|
||||
npx skills add time-attack/gstack
|
||||
```
|
||||
|
||||
Use that installer's update/remove commands and scope. It owns project/global
|
||||
placement, host destination paths, and selected-skill choices. GStack must not
|
||||
re-detect hosts during an update or enroll a host/skill the user did not
|
||||
previously select.
|
||||
|
||||
Before accepting an update, list the source and confirm that the defaults are
|
||||
still exactly `plan`, `design`, `qa`, `debug`, `review`, and `ship`. Pure
|
||||
judgment must remain usable even if the optional runtime update fails.
|
||||
|
||||
## 1.x command migration
|
||||
|
||||
The complete old-to-new map is [SKILL-MIGRATION.md](./SKILL-MIGRATION.md).
|
||||
Opt-in compatibility aliases are retained for two minor releases or 90 days
|
||||
from the 2.0 release, whichever policy the release notes identify. Each alias:
|
||||
|
||||
1. prints the canonical replacement;
|
||||
2. routes to the exact preserved specialist module;
|
||||
3. contains no copied judgment; and
|
||||
4. stays outside default skill discovery.
|
||||
|
||||
Examples:
|
||||
|
||||
```text
|
||||
/office-hours -> /plan --mode product
|
||||
/plan-eng-review -> /plan --mode eng
|
||||
/design-review -> /design --mode live-review
|
||||
/qa-only -> /qa --mode report
|
||||
/investigate -> /debug --mode investigate
|
||||
/cso -> /review --mode security
|
||||
/land-and-deploy -> /ship --mode land
|
||||
```
|
||||
|
||||
Existing safe preferences may migrate. Context.dev choice and data-egress
|
||||
consent must never be inferred from legacy browser, telemetry, service, or
|
||||
update-check settings. Leave Context network mode off until the user explicitly
|
||||
selects it.
|
||||
|
||||
## Initial managed runtime install
|
||||
|
||||
From a reviewed repository checkout, install only the optional runtime and
|
||||
local capability bundle with:
|
||||
|
||||
```bash
|
||||
./setup
|
||||
```
|
||||
|
||||
Skill placement remains a separate standard-installer operation. `./setup`
|
||||
resolves a symlinked source checkout, installs frozen production-only
|
||||
dependencies, and invokes `runtime/install.js`. The installer copies an explicit allowlist
|
||||
into an immutable version, rejects internal symlinks and path escapes, records
|
||||
size/mode/SHA-256 for every file, validates capability targets, smoke-tests the
|
||||
CLI, then atomically activates it. Stable POSIX and Windows launchers are
|
||||
written under `$GSTACK_HOME/bin` (default `~/.gstack/bin`), alongside
|
||||
`runtime-install.json`, which distinguishes managed paths from preserved
|
||||
config, secrets, projects, and plans.
|
||||
|
||||
Twenty-one focused installer tests pass with 307 assertions. They cover paths
|
||||
with spaces, a symlinked source root, internal-link rejection, runtime-only
|
||||
builder selection, production dependency closure, failed build/validation/
|
||||
smoke rollback, interrupted recovery, launchers, manifests, wrapper neutrality,
|
||||
and state-preserving uninstall.
|
||||
|
||||
The current managed-bundle audit records 107 components, 1,830 files,
|
||||
459,056,031 bytes, and 50 launchers. The Sharp/ngrok closure is included. The
|
||||
development-only Claude Agent SDK and Hugging Face sidecar are excluded; the
|
||||
Hugging Face package is development-only. Setup therefore installs neither its
|
||||
inference runtime nor model weights and reports the L4 capability unavailable.
|
||||
|
||||
A separate clean Linux arm64 container smoke copied the source through a path
|
||||
with spaces, installed only frozen production dependencies with the development
|
||||
SDK absent, rebuilt runtime capabilities without Git history or skill
|
||||
regeneration, passed setup/doctor/version/design/PDF checks, completed a local
|
||||
browser journey and Sharp full-page screenshot, and uninstalled while
|
||||
preserving state. Non-Darwin bundles omit the physical-iOS capability. Native
|
||||
Windows execution remains a separate gate; source-level Windows launchers are
|
||||
not native evidence.
|
||||
|
||||
## Runtime upgrade transaction
|
||||
|
||||
The candidate runtime intentionally does not combine “download arbitrary code”
|
||||
with activation. Give it a reviewed staged directory and an explicit version:
|
||||
|
||||
```bash
|
||||
gstack upgrade --source /path/to/verified-runtime --version 2.0.1
|
||||
```
|
||||
|
||||
The transaction is:
|
||||
|
||||
```text
|
||||
validate version and source
|
||||
-> lock upgrade state
|
||||
-> recover any interrupted pending transaction
|
||||
-> copy to a unique stage directory
|
||||
-> write stage metadata
|
||||
-> run verification when supplied
|
||||
-> atomically rename stage to immutable version directory
|
||||
-> write pending pointer with last-known-good
|
||||
-> run health check
|
||||
-> atomically mark pointer active
|
||||
```
|
||||
|
||||
If copy/verification fails, the stage directory is removed and the active
|
||||
pointer is unchanged. If health fails, the previous pointer is restored and
|
||||
the command returns `UPGRADE_ROLLED_BACK`. If the process dies after writing a
|
||||
pending pointer, the next upgrade/cleanup/doctor path recovers the last known
|
||||
good version before selection.
|
||||
|
||||
The runtime uses one per-user version store, not one copy per AI host. Skill
|
||||
install failure and runtime install failure are independent; neither may erase
|
||||
the other.
|
||||
|
||||
## Manual rollback
|
||||
|
||||
Inspect health first:
|
||||
|
||||
```bash
|
||||
gstack doctor
|
||||
gstack doctor --json
|
||||
```
|
||||
|
||||
Roll back to the retained last-known-good version:
|
||||
|
||||
```bash
|
||||
gstack upgrade --rollback
|
||||
```
|
||||
|
||||
Rollback validates that the retained directory exists, optionally health-checks
|
||||
it in the library call, atomically switches the pointer, and retains the version
|
||||
rolled back from as the next fallback. If no fallback exists it fails with an
|
||||
actionable error rather than choosing an arbitrary directory.
|
||||
|
||||
Runtime state schemas migrate forward only. If a state or migration marker is
|
||||
newer than the running runtime, stop with `STATE_NEWER_THAN_RUNTIME` or
|
||||
`MIGRATION_NEWER_THAN_RUNTIME`; never downgrade or rewrite the newer data.
|
||||
|
||||
## Recovery after interruption
|
||||
|
||||
1. Do not delete `~/.gstack/versions/current.json` while diagnosing.
|
||||
2. Run `gstack doctor --json` and preserve the output with secrets removed.
|
||||
3. Run `gstack cleanup --dry-run` to preview only stale managed temporaries.
|
||||
4. Retry the upgrade command. It recovers a pending pointer before staging.
|
||||
5. If the candidate remains unhealthy, use `gstack upgrade --rollback`.
|
||||
6. Inspect the current worktree's run state with `gstack state inspect`.
|
||||
7. Resume only the named/current-worktree run. An external effect left
|
||||
`uncertain` must be reconciled; do not clear the marker and repeat it blindly.
|
||||
|
||||
Cleanup skips symlinks and user-named data, and removes only recognized stale
|
||||
runtime temporaries/locks. `--dry-run` is non-mutating.
|
||||
|
||||
## Uninstall
|
||||
|
||||
Use the standard installer to remove skills. Separately, remove managed runtime
|
||||
versions while preserving configuration, secrets, and project history:
|
||||
|
||||
```bash
|
||||
gstack uninstall
|
||||
```
|
||||
|
||||
Purge all runtime state only after reviewing the path and accepting data loss:
|
||||
|
||||
```bash
|
||||
gstack uninstall --purge --yes
|
||||
```
|
||||
|
||||
The runtime refuses unsafe roots such as the filesystem root or home directory.
|
||||
It does not remove host skills, browser profiles, unrelated apps, repositories,
|
||||
or iPhone data.
|
||||
|
||||
## Evidence state
|
||||
|
||||
Focused candidate tests cover the managed installer plus atomic activation,
|
||||
failed-health rollback, interrupted-pointer recovery, manual rollback,
|
||||
newer-schema refusal, and non-mutating cleanup preview. The standard installer
|
||||
matrix separately passed 470/470 checks across six hosts, 16 installs, scopes,
|
||||
selections, and two removals; its artifact is
|
||||
[`evals/installation/install-matrix.json`](../../evals/installation/install-matrix.json).
|
||||
The runtime installer is green at 21 pass / 0 fail and 307 assertions, and the
|
||||
current 107-component bundle audit is recorded. The clean Linux Dev Container
|
||||
install/uninstall smoke passed. Interrupted network acquisition/stage at OS
|
||||
level, a passing live v3 host run, actual host UI execution, native-host Linux,
|
||||
and native Windows runs remain gates. See
|
||||
[TEST-EVIDENCE.md](./TEST-EVIDENCE.md).
|
||||
Reference in New Issue
Block a user