Files
gstack/ios-qa/SKILL.md.tmpl
c7554824cb v1.91.38.0 fix(ios-qa): iPads work, a dropped route no longer relaunches the app, loud boot-token write (#3076)
* fix(ios-qa): allow physical iPad devices in tunnel bootstrap (#2743)

(cherry picked from commit d6d43934ef)

* fix(ios-qa): iPhone + iPad on USB names each UDID instead of picking one

With GSTACK_IOS_TARGET_UDID unset, tied default ranks used to pick the first
device devicectl listed, so a QA session could drive the wrong device. Tied
candidates now fail with multiple_devices, list each device and UDID, and
print a ready export line. Device selection moves into selectDevice() so the
route-drop recovery can re-resolve with the same rules.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(ios-qa): a dropped CoreDevice route no longer relaunches the app (#1975)

Rotation deletes the app's one-shot boot-token file, so the daemon's old
503/504 path (invalidate + full bootstrap) could only get a new token by
running devicectl launch --terminate-existing, wiping the in-app QA state at
every route blip.

A route failure now goes through recoverTunnel(): re-select the device with
the bootstrap rules, re-resolve the tunnel address for the pinned UDID, check
the unauthenticated /healthz owner, then probe /state/snapshot with the cached
bearer. Outcomes:
- bearer accepted: keep the session (new address if it moved)
- 401: full bootstrap from the fresh boot token (app was relaunched)
- app confirmed absent: bootstrap, which launches it normally once
- different device or an address devicectl cannot tie to the UDID:
  bootstrap; the old bearer is never sent there
- route still down with the app running: surface the error, keep the session

Concurrent failures on one tunnel share a single refresh. Mutations whose
response was lost are still never replayed. The CLI wiring moves into
deviceTunnelSource() so tests drive it against a simulated device.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(ios-qa): a failed boot-token write is loud instead of silent (#1837)

StateServer.start() wrote the boot-token file with try?, so a failed write
left nothing on disk and no trace anywhere, and the daemon then relaunched the
app trying to mint a token that could never be written. The write is now a
do/catch that logs the path and error (never the token), logs a NOT READY
bootstrap line, and reports boot_token_error on /healthz. bootstrapTunnel
reads that field and returns boot_token_unavailable naming the device-side
cause instead of running --terminate-existing.

Also drops the stale comments claiming the token was written to os_log.
Template and fixture stay byte-identical.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(ios-qa): type the iPad acceptance test's mDNS stub

ResolveImpl returns Promise<string[]>; resolving null tripped the test-code
typecheck ratchet. A rejected lookup exercises the same resolve_failed path.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(ios-qa): iPad support, GSTACK_IOS_TARGET_UDID, route-drop recovery, known limits

- iPhone-only description, triggers, and prerequisites now cover iPads, and
  say how to pick one device with GSTACK_IOS_TARGET_UDID.
- Phase 2 no longer tells you to capture a boot token from os_log (it has
  not been logged for many releases); it describes the 0600 tmp/ file and
  the NOT READY signal (#1837, #1735).
- Source-control guidance for the generated DebugBridge/ package (#1735).
- Failure table: route drops keep the session, multiple_devices, unwritable
  token file, and the one relaunch a daemon restart still costs.
- Known limits: SwiftUI DragGesture and /swipe on iOS 26 (#1975), the
  iOS 26.3.1 /elements data from PR #1755, unverified iPad Stage Manager.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: iOS QA how-to and skill index cover iPads and route-drop recovery

The how-to still said the boot token was readable from os_log and that a
route drop re-bootstraps; both are now accurate, with iPad selection,
GSTACK_IOS_TARGET_UDID, and three new failure rows.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* v1.91.37.0 fix(ios-qa): iPads work, a route drop no longer relaunches the app, loud boot-token write

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(ios-qa): keep issue numbers out of model-read skill text

The archaeology lint rejects issue references in generated SKILL.md text;
the Known limits entry keeps the iOS version and drops the number.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore: re-slot to v1.91.38.0 (v1.91.37.0 claimed by #3075)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(ios-qa): device-change recovery test without an IPv4-mapped address

The Windows lane could not reach a fake StateServer through
::ffff:127.0.0.1, so the iPad half of the device-change test timed out. The
fake device now swaps identity on its one ::1 listener, which is also closer
to production: every StateServer answers on the same port.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: loulanyue <260355617@qq.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-10-08 07:02:02 +00:00

298 lines
15 KiB
Cheetah

---
name: ios-qa
preamble-tier: 3
version: 1.0.0
description: |
Live-device iOS QA for SwiftUI apps. Connects to a real iPhone or iPad via
USB CoreDevice IPv6 tunnel, reads Swift source to understand every screen, then
runs a vision-driven agent loop: screenshot → analyze → decide → act →
verify → repeat. All interaction happens via HTTP to an embedded
StateServer in the app under test. Optionally exposes the device over
Tailscale so remote agents (OpenClaw, Codex, any HTTP-capable agent) can
run iOS QA from anywhere without touching the hardware.
Use when asked to "ios qa", "test my iPhone app", "test my iPad app",
"find bugs on the device", or "qa the iOS app". (gstack)
voice-triggers:
- "iOS quality check"
- "test the iPhone app"
- "test the iPad app"
- "run iOS QA"
allowed-tools:
- Bash
- Read
- Write
- Edit
- Grep
- Glob
- AskUserQuestion
triggers:
- ios qa
- test the iphone app
- test the ipad app
- test my ios app
- find bugs on the device
- qa the ios app
---
{{PREAMBLE}}
# Live-device iOS QA
This skill drives a real iPhone or iPad via USB. The agent reads your Swift source,
generates typed state accessors, deploys a debug bridge, and runs a closed
find→fix→verify loop. No simulator, no XCTest, no WebDriverAgent.
## Architecture
```
┌──────────────────────┐ USB CoreDevice (IPv6) ┌──────────────────┐
│ gstack-ios-qa daemon │ ────────────────────────▶ │ iOS app │
│ (Mac, bun/TS) │ bearer + X-Session-Id │ StateServer │
│ │ │ (loopback only) │
│ - boot token rotate │ │ - /tap /swipe │
│ - session minting │ │ - /type /state │
│ - audit + redact │ │ - /snapshot │
└──────────────────────┘ └──────────────────┘
▲
│ Tailscale (optional, --tailnet)
│
┌──────────────────────┐
│ Remote agent │
│ (OpenClaw, etc.) │
└──────────────────────┘
```
The iOS app's `StateServer` binds loopback only (`::1` + `127.0.0.1`). Tailnet
ingress is exclusively the Mac daemon's job. The daemon validates Tailscale
identities via the local `tailscaled` socket and mints short-lived session
tokens (default 1h) for remote agents.
## Prerequisites
- macOS (the daemon uses `devicectl` from Xcode).
- iPhone or iPad connected via USB, paired and trusted. With more than one
connected, pick one before starting the daemon:
`export GSTACK_IOS_TARGET_UDID=<udid>` (`xcrun devicectl list devices` shows
UDIDs). Otherwise the daemon refuses to guess, lists each device with its
UDID, and prints that export line.
- Xcode + Swift toolchain installed (`swift --version` reports >= 5.9).
- App source available on disk, with at least one `@Observable` class.
- For remote-control mode: Tailscale installed and the user logged in.
## Phase 0: Session warm-start (optional)
If `~/.gstack/ios-qa-session.json` exists and the device is still connected,
skip Phase 1-2 and jump to Phase 3. The session cache holds the rotated token,
UDID, tunnel address, and accessor hash. Invalidate the cache when:
- The user passes `--cold` to force a full bootstrap.
- The accessor hash mismatch is detected on first state query.
- The daemon reports the cached UDID is no longer connected.
```bash
GSTACK_STATE_ROOT=$(~/.claude/skills/gstack/bin/gstack-paths --get GSTACK_STATE_ROOT); : "${GSTACK_STATE_ROOT:?gstack-paths failed; reinstall with ./setup or /gstack-upgrade}"
SESSION="$GSTACK_STATE_ROOT/ios-qa-session.json"
if [ -f "$SESSION" ] && [ "$COLD" != "1" ]; then
CACHED_UDID=$(python3 -c "import json,os; d=json.load(open(os.path.expanduser('$SESSION'))); print(d['udid'])")
CACHED_PORT=$(python3 -c "import json,os; d=json.load(open(os.path.expanduser('$SESSION'))); print(d['daemon_port'])")
if curl -sf "http://127.0.0.1:$CACHED_PORT/healthz" > /dev/null; then
echo "Warm start: daemon alive, device $CACHED_UDID connected"
fi
fi
```
## Phase 1: Read source, plan codegen
1. Before changing the app or replacing an installed build, verify that the
bridge is compatible with the project:
- The generator currently supports file-scope `@Observable` classes only;
`ObservableObject`, `@StateObject`, and other observation models do not
produce accessors.
- The documented dependency wiring assumes a SwiftPM app manifest. For an
`.xcodeproj` or `.xcworkspace`, do not invent package or target wiring.
If either requirement is unmet, stop the bridge bootstrap without modifying
the app. Preserve any installed production or TestFlight build. Prefer an
existing real-device XCUITest harness; when a separate QA build is needed,
use an isolated bundle identifier and non-production entitlements so it can
coexist with the production app. Report fixture-driven state, provider UI,
and actual external-provider success as distinct evidence tiers.
2. Walk the app source (passed as `--source <dir>`) and identify all `@Observable`
classes. Note any property immediately preceded by the generator marker
comment `// @Snapshotable` — those are the snapshot-eligible fields. The
marker is a comment so it composes with the `@Observable` macro. Each
marked field must belong to a file-scope observable class and be a writable
instance `var` with an explicit type and an internal or public setter.
Snapshot types are JSON-native scalars (`String`, `Bool`, integer widths,
`Float`, `Double`, `CGFloat`), arrays, String-keyed dictionaries, and their
Optional compositions. Keys must be unique across observable classes.
Codegen stops with a source diagnostic instead of emitting a broken or
lossy harness when any of these constraints is violated.
3. Show the user the accessor list and ask whether to install the DebugBridge
SPM dependency into their `Package.swift` (one AskUserQuestion).
## Phase 2: Bootstrap the device bridge
1. Generate the canonical local bridge package, typed accessors, and installed
version marker with one deterministic command:
```bash
~/.claude/skills/gstack/bin/gstack-ios-qa-regen \
--app-source "<source-dir>" \
--bridge-dir "<source-dir>/DebugBridge"
```
The regenerator also removes the explicit obsolete flat-file set created by
older ios-sync versions, preventing a stale second harness from remaining
in the app target.
Source control: `DebugBridge/` is generated; never hand-edit it. Commit it
when teammates or CI build the Debug configuration without gstack (re-run
the regenerator after a gstack upgrade); otherwise add `DebugBridge/` to
`.gitignore` and have each developer run the regenerator. Tell the user
which one you picked.
2. Add the generated `DebugBridge` local SPM dependency to the app's
`Package.swift`. The package
ships three Debug-config-only library products:
- `DebugBridgeCore` (Swift, cross-platform) — StateServer + bridge protocols.
- `DebugBridgeTouch` (Objective-C, iOS-only) — KIF-derived in-process touch
synthesis with iOS 18+ `_UIHitTestContext` SwiftUI hit-testing.
- `DebugBridgeUI` (Swift, iOS-only) — Screenshot / Elements / Mutation
bridge implementations.
The app target depends on `DebugBridgeUI` with `.when(configuration: .debug)`
(transitively pulls in Core + Touch). Release builds refuse to link these
targets.
3. Wire the bridges from the `@main` App init, gated on `#if DEBUG`:
```swift
#if DEBUG
import DebugBridgeCore
#if canImport(UIKit)
import DebugBridgeUI
// Install resolvers before StateServer opens its listener.
DebugBridgeUIWiring.installAll()
#endif
// Replace AppState/AppStateAccessor with the type discovered in Phase 1.
DebugBridgeManager.shared.start(
appState: appState,
register: AppStateAccessor.register
)
#endif
```
4. Build + deploy to the device with `xcodebuild -scheme <SchemeName>
-destination 'platform=iOS,id=<UDID>' build install`.
5. Launch via `devicectl device process launch --device <UDID> <bundle-id>`.
On launch the StateServer writes a one-use boot token to a 0600 file in the
app's `tmp/`; the daemon copies it out with `devicectl`. The token is never
printed to `os_log`. If the app cannot write that file, it logs `NOT READY`
and the daemon reports `boot_token_unavailable` with the cause.
6. Spawn the Mac-side daemon (on-demand) — `gstack-ios-qa-daemon`. Daemon
acquires an exclusive flock on `~/.gstack/ios-qa-daemon.pid`. If another
daemon is alive, the second invocation discovers its port and connects.
7. Daemon immediately calls `POST /auth/rotate` on the iOS StateServer with a
fresh in-memory-only token. Rotation deletes the boot-token file, so a copy
taken after this point is a dead credential.
If a fresh daemon finds the app running after another daemon consumed that
one-use token, it verifies the bundle owner, relaunches the target once,
waits for the new token, verifies ownership again, and then rotates.
## Phase 3: Vision-driven agent loop
Each iteration:
1. `GET /screenshot` (via daemon) → save PNG.
2. `GET /elements` → accessibility tree.
3. `GET /state/snapshot` (only `// @Snapshotable` fields) → current state.
4. Decide next action based on what's on the screen vs the test goal.
5. `POST /session/acquire` to grab the device lock.
6. Execute `POST /tap`, `/swipe`, `/type`, or `POST /state/<key>` write.
7. Re-screenshot; compare; record finding if buggy.
8. `POST /session/release` once the iteration is done.
Each authenticated mutating request through the tailnet listener (if remote
mode is active) writes an audit row to
`~/.gstack/security/ios-qa-audit.jsonl`.
## Modes
**Local-USB mode (default).** Daemon binds loopback only; no Tailscale
required. The spawning skill gets full-surface access. Best for solo
development.
**Tailnet mode (`--tailnet`).** Daemon additionally binds the Tailscale
interface (never `0.0.0.0`). Requires `tailscaled` to be running locally and
the daemon to be able to read `/var/run/tailscale.sock`. Fails closed if the
socket is missing, permission-denied, or returns an unparseable WhoIs
response. Remote agents hit `POST /auth/mint` over tailnet, daemon
canonicalizes identity via WhoIs, checks the allowlist file, mints a
session token. See `ios-qa/docs/tailscale-acl-example.md`.
**Capability tiers (tailnet mode).** Minted tokens default to `interact`
(taps, swipes, types). Higher tiers require explicit owner mint:
- **observe:** `/screenshot`, `/elements`, `GET /state/*`, `/healthz`,
`/session/heartbeat`.
- **interact:** observe + `/tap`, `/swipe`, `/type`.
- **mutate:** interact + `POST /state/<key>`.
- **restore:** mutate + `POST /state/restore`.
Owner mints via `gstack-ios-qa-mint --remote <identity> --capability <tier>`
on the Mac. Self-service mint over tailnet only succeeds for already-allowlisted
identities.
**Recording mode (`--recording`).** DebugOverlay renders a small diagonal
"AGENT DEMO" watermark in a corner so screencasts are unambiguous about the
device being agent-driven.
## Demo mode
If the user says "demo", "demo mode", "show me", or "I want to see it
working", run in **DEMO MODE**. This changes how the agent interacts with
the app:
**DEMO MODE OVERRIDES ALL OTHER RULES.** When demo mode is active, the
agent MUST drive every action through visible UI (`/tap`, `/swipe`, `/type`)
and NEVER use `POST /state/*` writes to skip steps. Viewers see the agent
type every key, tap every button. The on-device DebugOverlay attribution
chip shows "Driven by Claude Code (demo)" or the remote agent identity.
In demo mode, the screencap rate is bumped to 4fps so the recording feels
live.
## Failure modes + recovery
| Symptom | Likely cause | Action |
|---|---|---|
| `curl: connection refused` to daemon | daemon crashed | Re-run `/ios-qa`; spawn-race lock will fail closed |
| `403 identity_not_allowed` from `/auth/mint` | identity missing from allowlist | Run `gstack-ios-qa-mint --remote <identity>` on the Mac |
| `409 schema_mismatch` on `/state/restore` | snapshot from older app build | Discard the snapshot; re-capture |
| `503 device_disconnected` / `504 upstream_timeout` from proxy | USB route dropped, app stopped, or app relaunched | Daemon probes the running app with its session bearer and keeps the session (no relaunch, app state intact). It bootstraps only when the app rejected the bearer (401), is not running, or a different device is now selected. A lost `/tap`/`/swipe`/`/type` response is never replayed: check the screen before retrying. If it persists, reconnect/unlock the device |
| `multiple_devices` at bootstrap | iPhone and iPad (or two devices) connected, no target set | Run the printed `export GSTACK_IOS_TARGET_UDID=<udid>`, then restart the daemon |
| `boot_token_unavailable ... could not write` | app's `tmp/` not writable | Fix the app container, relaunch the app |
| App relaunched after the daemon restarted | a new daemon has no session bearer and the one-use boot token is gone | Expected: the first bootstrap relaunches the app once; keep one daemon alive for a session |
| `429 rate_limited` from `/auth/mint` | >10 mints/min from one identity | Wait 60s; check audit log for anomalies |
| `413 body_too_large` on `/state/restore` | snapshot >1MB | Increase `--max-body` or trim snapshot |
## Known limits
Device-verified by users, not fixable in the bridge today. Plan around them:
- **SwiftUI gestures on iOS 26.** In-process synthesized touches report success
but never reach a SwiftUI `DragGesture` (for example a `Canvas` driven by
drag input), even with phase-separated touches (seen on iOS 26.5). Buttons and
UIKit controls still respond. For gesture-driven views, have the app expose
its input handlers to the bridge under `#if DEBUG` and drive them through a
state write, or cover the flow with an XCUITest harness.
- **`/swipe` scrolls only.** It moves the nearest enclosing `UIScrollView` and
returns `false` when there is none; it is not a drag. Custom pan or drag
views need the input-routing approach above.
- **`/elements` on iOS 26.** The in-process SwiftUI accessibility tree is often
not materialized: an iPhone 12 Pro on iOS 26.3.1 returned only the three
hosting views, with no identifiers or labels. Locate controls from the
screenshot and tap by coordinate.
- **iPad windows.** iPad sessions work like iPhone sessions, but the overlay
and window selection have not been verified with Stage Manager or multiple
scenes; report what you see.
## Cleanup
Use `/ios-clean` to remove the DebugBridge SPM dependency and all `#if DEBUG`
wiring before a Release build. This is a convenience flow; the structural
Release-build guard (Package.swift `.when(configuration: .debug)` + CI
`swift build -c release` check) is the safety-critical path.