gstack/skills/qa/references/legacy/scrape.md

<!-- GENERATED by scripts/gstack2/generate-skill-tree.ts; do not edit. -->
<!-- GSTACK2_PROVENANCE source=scrape/SKILL.md.tmpl base=bb57306d98c97011b0919c6132705a15b1579781 blob=4cb4f17c074edcdce0bc8d133f19a6a739964851 baseline_render_sha256=fc2f222a75825e92458b3503bc122edda87cfe9d92a41c4e87e89fc68d5bdcd5 ported_render_sha256=82b7214021c997000822b2b4adcae19443490198556a7e6af54d8937ae504943 disposition=BUG_FIX -->
<!-- GSTACK2_ROUTING replacement=$qa --mode Report --module scrape visibility=internal depth=standard mutation=report-only web=production -->

<!-- GSTACK2_LEGACY_BODY_START source=scrape -->
## Host-neutral runtime bindings

These assignments select stable paths only; they do not install anything or grant consent:

```bash
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
GSTACK_ROOT="$GSTACK_HOME"
GSTACK_STATE_ROOT="$GSTACK_HOME"
GSTACK_BIN="$GSTACK_HOME/bin"
BUN_CMD="$GSTACK_BIN/bun"
B="$GSTACK_BIN/browse"
D="$GSTACK_BIN/gstack-design"
P="$GSTACK_BIN/make-pdf"
```
# $qa --mode Report --module scrape — pull data from a page

One entry point for getting data off the web. Two paths under the hood:

1. **Match path** (~200ms) — if the user's intent matches an existing
   browser-skill's triggers, run it via `$B skill run <name>` and emit
   the JSON.
2. **Prototype path** (~30s) — no matching skill yet, so drive the page
   with `$B` primitives, return the JSON, and suggest `$qa --mode Report --module skillify` so the
   next call lands on the match path.

Read-only by contract. If the intent implies writing (submitting forms,
clicking buttons that mutate state), refuse and route to `/automate`.

## Step 1 — Determine intent

The user's request after `$qa --mode Report --module scrape` is the intent. If they did not include
one, ask once:

> "What do you want to scrape? Describe it in one line, e.g. 'top stories
> on Hacker News' or 'product names + prices on example.com/products'."

Do not ask multiple clarifying questions up front. Any further questions
go in the prototype path where they're cheaper.

## Step 2 — Refuse mutating intents

If the intent implies writes — verbs like *submit*, *post*, *send*, *log
in*, *click X*, *fill the form*, *delete*, *create*, *order*, *book* —
respond:

> "$qa --mode Report --module scrape is read-only. For mutating flows, use /automate (browser-skills
> Phase 2 P0 in TODOS.md — not yet shipped). Until then, use $B click /
> $B fill / $B type directly."

Stop. Do not enter the match or prototype path.

## Step 3 — Match phase

List existing browser-skills:

```bash
$B skill list
```

For each skill, `$B skill show <name>` exposes the full SKILL.md including
`triggers:`, `description:`, and `host:`. Read these and judge whether the
user's intent semantically matches one of them.

A confident match means **all three** are true:

- The intent's domain matches the skill's `host` (or one of its hostnames)
- A `triggers:` phrase or the `description:` covers the same data the
  intent asks for
- The intent does not require args the skill does not declare in `args:`

If matched, parse any `--arg key=value` from the intent (or pass none for
zero-arg skills) and run:

```bash
$B skill run <name> [--arg key=value ...]
```

Emit the JSON the skill prints to stdout. Stop.

If matching is ambiguous (two skills could plausibly fit), pick the
narrower-tier one (project > global > bundled — `$B skill list` shows the
tier). If still ambiguous, fall through to the prototype path rather than
guess wrong.

## Step 4 — Prototype phase

No match. Drive the page using `$B` primitives:

1. `$B goto <url>` — navigate to the target. The user's intent usually
   names a host or a URL; use it directly.
2. `$B snapshot --text` (or `$B text`) — get a clean text view of the
   page to find selectors.
3. `$B html` — pull the raw HTML when you need to parse structured data
   (lists, tables, repeated rows).
4. `$B links` — when the intent is to gather URLs.
5. Iterate: try a selector, check the output, refine.

Emit the result as JSON on stdout (one document, not pretty-printed).
Use a stable shape — typically `{ "items": [...], "count": N }` or
similar — so downstream consumers can treat it as data.

## Step 5 — Skillify nudge

After a successful prototype, append exactly one line:

> "Say $qa --mode Report --module skillify to make this a permanent skill (200ms on next call)."

That is the entire nudge. Do not nag, do not list pros, do not push.
Proactive surfacing is a Phase 3 knob (`gstack-config browser_skillify_prompts`),
not this skill's job.

## When the prototype fails

If the page loads but data extraction does not yield a sensible JSON shape
after 3-4 selector attempts:

- Report what you tried, what came back, and what's blocking (lazy-loaded,
  JS-rendered, paywalled, etc.).
- Do NOT write a partial result and call it done.
- Do NOT suggest $qa --mode Report --module skillify on a broken prototype.
- Ask the user whether they want to (a) try a different selector, (b)
  switch to a different page, or (c) stop.

## What this skill does NOT do

- Mutating actions (use /automate when shipped, or $B primitives directly)
- Auth flows / cookie import (use $qa --mode Report --module setup-browser-cookies first)
- Multi-page crawls (this is one-shot per call)
- Anything that requires the daemon to not be running

## Output discipline

The match path returns whatever JSON the matched skill emits. The
prototype path returns whatever JSON you construct. In both cases:

- One JSON document, on stdout.
- Stderr (or chat) is for logs and the skillify nudge.
- Do not embed prose around the JSON in the chat reply unless the user
  asked for an explanation — many `$qa --mode Report --module scrape` callers pipe the output to
  `jq`.

## Capture Learnings

If you discovered a non-obvious pattern, pitfall, or architectural insight during
this session, log it for future sessions:

```bash
$GSTACK_BIN/gstack-learnings-log '{"skill":"scrape","type":"TYPE","key":"SHORT_KEY","insight":"DESCRIPTION","confidence":N,"source":"SOURCE","files":["path/to/relevant/file"]}'
```

**Types:** `pattern` (reusable approach), `pitfall` (what NOT to do), `preference`
(user stated), `architecture` (structural decision), `tool` (library/framework insight),
`operational` (project environment/CLI/workflow knowledge).

**Sources:** `observed` (you found this in the code), `user-stated` (user told you),
`inferred` (AI deduction), `cross-model` (both Claude and Codex agree).

**Confidence:** 1-10. Be honest. An observed pattern you verified in the code is 8-9.
An inference you're not sure about is 4-5. A user preference they explicitly stated is 10.

**files:** Include the specific file paths this learning references. This enables
staleness detection: if those files are later deleted, the learning can be flagged.

**Only log genuine discoveries.** Don't log obvious things. Don't log things the user
already knows. A good test: would this insight save time in a future session? If yes, log it.
<!-- GSTACK2_LEGACY_BODY_END source=scrape -->

<!-- GSTACK2_BUG_FIX_START pr=679 anchor=GSTACK2_FIX_679_MATCH_USER_LANGUAGE -->
## Upstream judgment port: PR #679

[Match the user language](https://github.com/garrytan/gstack/pull/679)

### User-language rule

Write questions, progress updates, reports, and artifacts in the language used by the user. Source material, code identifiers, commands, and quotations may remain in their original language when translating them would reduce accuracy.
<!-- GSTACK2_BUG_FIX_END pr=679 -->

<!-- GSTACK2_BUG_FIX_START pr=2030 anchor=GSTACK2_FIX_2030_SIGNAL_GATED_LEARNING -->
## Upstream judgment port: PR #2030

[Record only signal-bearing learnings](https://github.com/garrytan/gstack/pull/2030)

### Signal-gated learning

Persist a learning only when the interaction contains a useful, reusable signal such as an explicit preference, correction, accepted recommendation, or rejected direction. Track helpful and harmful outcomes separately. Do not manufacture a learning merely because a workflow completed.
<!-- GSTACK2_BUG_FIX_END pr=2030 -->