mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-21 04:10:47 +02:00
record live Context and device evidence
This commit is contained in:
@@ -22,10 +22,10 @@ pass from deterministic, offline, or filesystem-only evidence.
|
||||
| Six-skill catalog measurement after regeneration | Six names/descriptions total 982 characters, about 246 four-character token-equivalents; baseline correctly parsed catalog was about 1,100. | 77.6% reduction, above the 75% gate. Re-measure if frontmatter changes. The buggy 4,214 baseline estimate is not used. |
|
||||
| `bun test ios-qa/daemon/test` | **95 pass / 0 fail**, 229 assertions. | Covers daemon regressions including malformed device JSON, hardware-UDID/CoreDevice selection, bounded proxy timeout, and expected-bundle mutation header. It is not a signed-app live pass. |
|
||||
| Focused DebugBridge/template build tests | **33 pass / 0 fail** in the candidate run, including Swift debug compilation/XCTest and Release symbol absence. | Static/build evidence for debug-only bridge wiring; still not an installed physical-app journey. |
|
||||
| Physical-iOS E2E preflight | **9 pass / 0 fail / 1 deploy check skipped**, 29 assertions. | The connected-device/setup and unsigned Release checks pass. The deploy skip leaves the signed-device P0 gate open. |
|
||||
| Direct physical-device smoke | **Typed failure:** code `signing_unavailable`, category `setup_gate`. | Automatic signing/provisioning is the remediable blocker. No app was installed or launched and no pass artifact was written. This is neither a product failure nor a pass. |
|
||||
| Physical-iOS E2E preflight | **9 pass / 0 fail / 1 deploy check skipped**, 29 assertions. | The earlier connected-device/setup and unsigned Release checks pass. The deploy skip leaves the signed-device P0 gate open. |
|
||||
| Direct physical-device smoke | **Typed setup-gate failures, no product failure:** the earlier target returned `signing_unavailable`; the latest explicitly selected target returned `device_not_wired`, and CoreDevice pairing returned error 1011. | The latest target is an iPhone10,6 on iOS 16.7.10: legacy USB pairing validates, but Xcode reports CoreDevice pairing unsupported and exposes no wired CoreDevice service. The other connected phone was not used. No app was installed or launched and no pass artifact was written. |
|
||||
| Context.dev contract (`gstack2-runtime-context.test.ts`) | **22 pass / 0 fail**, 139 assertions. | Persists explicit host/local-browser/none choices without consent, rejects private/credential URLs and request material plus private DNS, proves zero lookup/fetch before mode+consent, validates documented endpoint paths and exact failure taxonomy, and makes search typed unsupported without network. |
|
||||
| `gstack context smoke` | **Not run:** `CONTEXT_DEV_API_KEY` is not configured and no verified account key was available. | Live provider behavior, account verification, and actual credit metadata remain unverified. |
|
||||
| `gstack context smoke --url https://www.context.dev --json` | **PASS:** the official `/web/scrape/markdown` endpoint returned `ok: true` and credit metadata. | The verified key entered through protected stdin/environment worked. The temporary secrets file was mode `0600`, the isolated home was removed, and the safe output did not contain the key. Committed redacted artifact: [`evals/context-dev/live-smoke-2026-07-17.json`](../../evals/context-dev/live-smoke-2026-07-17.json). |
|
||||
| Standard installer matrix | **PASS: 470/470 checks**, 16 install cases, two removal cases, `skills` CLI 1.5.19. | Project/global installs pass for six hosts, selected-skill and opt-in compatibility-alias cases, copies, and hashes. This remains installer/filesystem evidence. Committed artifact: [`evals/installation/install-matrix.json`](../../evals/installation/install-matrix.json). |
|
||||
| `bun test test/gstack2-runtime-install.test.ts` | **Exit 0: 24 pass / 0 fail**, 336 assertions. | Managed allowlist, hashes, spaces, source/internal-link rejection, production-only frozen dependencies, deterministic exact Sharp/ngrok platform closure, native-load rollback smoke, rollback/recovery, stable launchers, wrapper neutrality, and state-preserving uninstall pass. |
|
||||
| Deterministic clean macOS arm64 managed runtime bundle audit | **110 components, 1,829 files, 450,044,315 bytes, 50 capability launchers.** | This is a platform-specific bundle measurement, not a universal byte count; platform-native package payloads differ. Setup includes the Sharp/ngrok closure and excludes the development-only Claude Agent SDK. The Hugging Face sidecar is outside the bundle and its package is development-only, so production setup installs neither its inference runtime nor model weights; the L4 capability reports unavailable. The standard skill installer remains Markdown-only. Committed artifact: [`evals/runtime-bundle/darwin-arm64.json`](../../evals/runtime-bundle/darwin-arm64.json); reproduce with `bun run scripts/gstack2/audit-runtime-bundle.ts --output evals/runtime-bundle/darwin-arm64.json`. |
|
||||
|
||||
Reference in New Issue
Block a user