feat: add provider-aware browser QA setup

Detect host-native browser tools before offering the isolated local Chromium fallback, add a common readiness fixture, harden managed browser startup, and verify standards installs expose one canonical QA skill.
2026-09-21 04:10:47 +02:00 · 2026-07-20 16:01:24 -07:00
parent f14445bb00
commit d6ef673e4d
163 changed files with 5272 additions and 990 deletions
@@ -28,8 +28,8 @@ pass from deterministic, offline, or filesystem-only evidence.
 | Physical-device smoke, 2026-07-20 | **PASS:** 12/12 harness checks and all five required live iterations. Release symbols were absent; signing, safe in-place install, launch, CoreDevice bootstrap, boot-token rotation, five session acquire/release cycles, ten screenshots, accessibility reads, coordinate taps, bundle checks, state cleanup, tunnel shutdown, and temporary-workspace cleanup passed. | Closes the physical-iPhone evidence layer for the tested wired paired `iPhone17,1`; it is not a claim about every iPhone or iOS release. Redacted artifact: [`ios-physical-device-2026-07-20T17-49-19-302Z.json`](./evidence/ios-physical-device-2026-07-20T17-49-19-302Z.json). |
 | Context.dev contract (`gstack2-runtime-context.test.ts`) | **22 pass / 0 fail**, 139 assertions. | Persists explicit host/local-browser/none choices without consent, rejects private/credential URLs and request material plus private DNS, proves zero lookup/fetch before mode+consent, validates documented endpoint paths and exact failure taxonomy, and makes search typed unsupported without network. |
 | `gstack context smoke --url https://www.context.dev --json` | **PASS:** the official `/web/scrape/markdown` endpoint returned `ok: true` and credit metadata. | The verified key entered through protected stdin/environment worked. The temporary secrets file was mode `0600`, the isolated home was removed, and the safe output did not contain the key. Committed redacted artifact: [`evals/context-dev/live-smoke-2026-07-17.json`](../../evals/context-dev/live-smoke-2026-07-17.json). |
-| Standard installer matrix | **PASS: 470/470 checks**, 16 install cases, two removal cases, `skills` CLI 1.5.19. | Project/global installs pass for six hosts, selected-skill and opt-in compatibility-alias cases, copies, and hashes. This remains installer/filesystem evidence. Committed artifact: [`evals/installation/install-matrix.json`](../../evals/installation/install-matrix.json). |
-| Standard Agent Skills install + actual runtime-absent Codex invocation | **PASS.** Root `--list` returned exactly six. The selected source was `time-attack/gstack/skills --skill qa`; despite `skills` 1.5.19's pre-filter display counting hidden aliases, exactly one skill (`qa`) installed byte-identically. Codex exited 0 with `gstack` absent, reported `NEEDS_SETUP` and one approval prompt, changed no files, and created no runtime/browser/external-service activity. | Closes the actual-host runtime-absent judgment gate for Codex without overstating six-host UI coverage. Artifact: [`standard-codex-runtime-absent-2026-07-17.json`](../../evals/installation/standard-codex-runtime-absent-2026-07-17.json). |
+| Standard installer matrix | **PASS: 510/510 checks**, 18 install cases, two removal cases, `skills` CLI 1.5.19. | Project/global installs pass for seven hosts, including Kimi Code CLI through the standard `.agents/skills` path; selected-skill and opt-in compatibility-alias cases, copies, and hashes pass. This remains installer/filesystem evidence. Committed artifact: [`evals/installation/install-matrix.json`](../../evals/installation/install-matrix.json). |
+| Standard Agent Skills install + actual runtime-absent Codex invocation | **PASS.** Root `--list` returned exactly six. The selected source was `time-attack/gstack/skills --skill qa`; despite `skills` 1.5.19's pre-filter display counting hidden aliases, exactly one skill (`qa`) installed byte-identically. Codex exited 0 with `gstack` absent, reported `NEEDS_SETUP` and one approval prompt, changed no files, and created no runtime/browser/external-service activity. | Closes the actual-host runtime-absent judgment gate for Codex without overstating seven-host UI coverage. Artifact: [`standard-codex-runtime-absent-2026-07-17.json`](../../evals/installation/standard-codex-runtime-absent-2026-07-17.json). |
 | `bun test test/gstack2-runtime-install.test.ts` | **Exit 0: 25 pass / 0 fail**, 341 assertions. | Managed allowlist, hashes, spaces, source/internal-link rejection, production-only frozen dependencies, deterministic exact Sharp/ngrok platform closure, native-load rollback smoke, rollback/recovery, stable launchers, wrapper neutrality, and state-preserving uninstall pass. |
 | Deterministic clean macOS arm64 managed runtime bundle audit | **110 components, 1,829 files, 450,044,315 bytes, 50 capability launchers.** | This is a platform-specific bundle measurement, not a universal byte count; platform-native package payloads differ. Setup includes the Sharp/ngrok closure and excludes the development-only Claude Agent SDK. The Hugging Face sidecar is outside the bundle and its package is development-only, so production setup installs neither its inference runtime nor model weights; the L4 capability reports unavailable. The standard skill installer remains Markdown-only. Committed artifact: [`evals/runtime-bundle/darwin-arm64.json`](../../evals/runtime-bundle/darwin-arm64.json); reproduce with `bun run scripts/gstack2/audit-runtime-bundle.ts --output evals/runtime-bundle/darwin-arm64.json`. |
 | Earlier declared Linux Dev Container plus `bun run test:gstack2` inside it | **Exit 0: 136 pass / 0 fail, 1,127 assertions across 15 files.** Log: `/tmp/gstack2-devcontainer-gate-candidate-final4.log`. | Historical container checkpoint retained rather than overwritten; the current 150-test container result is recorded in the native-CI row below. |