mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-01 19:25:10 +02:00
a94a64f821
* fix: snapshot -i auto-detects dropdown/popover interactive elements - Auto-enable cursor-interactive scan (-C) when -i flag is used - Add floating container detection (portals, popovers, dropdowns) - Detects position:fixed/absolute with high z-index - Recognizes data-floating-ui-portal, data-radix-* attributes - Recognizes role=listbox, role=menu containers - Elements inside floating containers bypass the hasRole skip - Catches dropdown items missed by the accessibility tree - Role=option/menuitem elements in floating containers captured even without cursor:pointer/onclick - Tag floating container items with 'popover-child' reason - Include role name in @c ref reasons when present - Add dropdown.html test fixture - Add dropdown/popover detection test suite (6 tests) - Add test: -i alone includes cursor-interactive elements Fixes: Bookface autocomplete, Radix UI combobox, React portals, and similar dynamic dropdown patterns where ariaSnapshot() misses the floating content. * chore: bump version and changelog (v0.15.12.0) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: update snapshot -i/-C flag descriptions to mention auto-enable behavior * test: strengthen clickability test guard assertions The @c ref clickability test previously used if-guards that would silently pass when no Alice line was found in the snapshot output. Both Claude and Codex adversarial review flagged this as a test that could regress without CI noticing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: regenerate top-level SKILL.md with updated flag descriptions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: root <root@localhost> Co-authored-by: gstack <ship@gstack.dev> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
103 lines
4.5 KiB
Markdown
103 lines
4.5 KiB
Markdown
# Plan: Snapshot Dropdown/Autocomplete Interactive Element Detection
|
|
|
|
## Problem
|
|
|
|
`snapshot -i` misses dropdown/autocomplete items on modern web apps. These elements:
|
|
1. Are often `<div>`/`<li>` with click handlers but no semantic ARIA roles
|
|
2. Live inside dynamically-created portals/popovers (floating containers)
|
|
3. Don't appear in Playwright's accessibility tree (`ariaSnapshot()`)
|
|
|
|
The `-C` flag (cursor-interactive scan) was designed for this but:
|
|
- Requires separate flag — agents using `-i` don't get it automatically
|
|
- Skips elements that HAVE an ARIA role (even if the ARIA tree missed them)
|
|
- Doesn't prioritize popover/portal containers where dropdown items live
|
|
|
|
## Root Cause
|
|
|
|
Playwright's `ariaSnapshot()` builds from the browser's accessibility tree. Dynamically-rendered popovers (React portals, Radix Popover, etc.) may not be in the accessibility tree if:
|
|
- The component doesn't set ARIA roles
|
|
- The portal renders outside the scoped `body` locator's subtree timing
|
|
- The browser hasn't updated the accessibility tree yet after DOM mutation
|
|
|
|
## Changes
|
|
|
|
### 1. Auto-enable cursor-interactive scan with `-i` flag
|
|
|
|
**File:** `browse/src/snapshot.ts`
|
|
|
|
When `-i` (interactive) is passed, automatically include the cursor-interactive scan. This means agents always see clickable non-ARIA elements when they ask for interactive elements.
|
|
|
|
The `-C` flag remains as a standalone option for non-interactive snapshots.
|
|
|
|
```
|
|
if (opts.interactive) {
|
|
opts.cursorInteractive = true;
|
|
}
|
|
```
|
|
|
|
### 2. Add popover/portal priority scanning
|
|
|
|
**File:** `browse/src/snapshot.ts` (inside cursor-interactive evaluate block)
|
|
|
|
Before the general cursor:pointer scan, specifically scan for visible floating containers (popovers, dropdowns, menus) and include ALL their direct children as interactive:
|
|
|
|
Detection heuristics for floating containers:
|
|
- `position: fixed` or `position: absolute` with `z-index >= 10`
|
|
- Has `role="listbox"`, `role="menu"`, `role="dialog"`, `role="tooltip"`, `[data-radix-popper-content-wrapper]`, `[data-floating-ui-portal]`, etc.
|
|
- Appeared recently in the DOM (not in initial page load)
|
|
- Is visible (`offsetParent !== null` or `position: fixed`)
|
|
|
|
For each floating container, include child elements that:
|
|
- Have text content
|
|
- Are visible
|
|
- Have cursor:pointer OR onclick OR role="option" OR role="menuitem"
|
|
- Tag with reason `popover-child` for clarity
|
|
|
|
### 3. Remove the `hasRole` skip in cursor-interactive scan
|
|
|
|
**File:** `browse/src/snapshot.ts`
|
|
|
|
Currently: `if (hasRole) continue;` — skips any element with an ARIA role, assuming the ARIA tree already captured it.
|
|
|
|
Problem: if the ARIA tree MISSED the element (timing, portal, bad DOM structure), it falls through both systems.
|
|
|
|
Fix: Only skip if the element's role is in `INTERACTIVE_ROLES` AND it was actually captured in the main refMap. Otherwise include it.
|
|
|
|
Since we can't easily check the refMap from inside `page.evaluate()`, the simpler fix: remove the `hasRole` skip entirely for elements inside detected floating containers. For elements outside floating containers, keep the `hasRole` skip as-is (to avoid duplicates in normal page content).
|
|
|
|
### 4. Add dropdown test fixture and tests
|
|
|
|
**File:** `browse/test/fixtures/dropdown.html`
|
|
|
|
HTML page with:
|
|
- A combobox input that shows a dropdown on focus/type
|
|
- Dropdown items as `<div>` with click handlers (no ARIA roles)
|
|
- Dropdown items as `<li>` with `role="option"`
|
|
- A React-portal-style container (`position: fixed`, high z-index)
|
|
|
|
**File:** `browse/test/snapshot.test.ts`
|
|
|
|
New test cases:
|
|
- `snapshot -i` on dropdown page finds dropdown items via cursor scan
|
|
- `snapshot -i` on dropdown page includes popover-child elements
|
|
- `@c` refs from dropdown scan are clickable
|
|
- Elements inside floating containers with ARIA roles are captured even when ARIA tree misses them
|
|
|
|
## Rollout Risk
|
|
|
|
**Low.** The `-C` scan is additive — it only adds `@c` refs, never removes `@e` refs. The change to auto-enable it with `-i` increases output size but agents already handle mixed ref types.
|
|
|
|
**One concern:** The `-C` scan queries ALL elements (`document.querySelectorAll('*')`) which can be slow on heavy pages. For the popover-specific scan, we limit to elements inside detected floating containers, which is fast (small subtree).
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
cd /data/gstack/browse && bun test snapshot
|
|
```
|
|
|
|
## Files Changed
|
|
|
|
1. `browse/src/snapshot.ts` — auto-enable -C with -i, popover scanning, remove hasRole skip in floating containers
|
|
2. `browse/test/fixtures/dropdown.html` — new test fixture
|
|
3. `browse/test/snapshot.test.ts` — new dropdown/popover test cases
|