mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-05 00:03:15 +01:00
* test(qa-callers): bind QA section names in the caller prompt instead of teaching helper forms (W3-4)
The caller session prompt taught full capture/checkpoint syntax but only
mentioned materialize, so two ship-exploratory-small-cli sessions (paid
gate runs 38973329985, 38993076767) skipped qa/sections/exploratory.md and
system-functional.md and guessed a bare materialize. The prompt now
declares only fixture bindings (DEADLINE_TOOL, EVIDENCE_TOOL, PROBE_DIR,
DEADLINE_FILE, --public approval, --timeout-ms 10000 for plan probes) and
points to the section for every form. The read requirement is unchanged;
both captured transcripts are kept as negative controls.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(review-army): run the state-machine detector in the scope block and check its exit status (W3-5)
Specialist 9 named gstack-state-matrix detect only in selection prose, and
small diffs keep only State Machines, so agents ran an undeclared command
(paid gate run 39002269281). The generated scope block now runs the
detector with its exit status checked: on failure it keeps the detector's
diagnostics and sets SCOPE_STATE_MACHINE=unknown, and selection dispatches
State Machines when the signal is true or unknown. The check step becomes a
declared heredoc block. The clarity test requires every non-diff-scope
signal to come from a detect helper the scope block runs; a free test runs
the block against a failing detector stub; the captured standalone call
stays outside the caller interface.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(third-party-actions): letter the absent-Aside consent options B/C/D (W3-7)
The contract fixed letters only when Aside is detected, so a tpa-absent-linux
agent numbered its consent question 1/2/3 (paid run 39011101113), which the
lettered-offer inspection cannot read. The absent branch now offers
B) gstack's own visible browser, C) manual instructions, D) defer and omits
A). The paid test's assertion is unchanged; the captured numbered answer is
a documented fail example in the unit test.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(plan-reality): say tier 1 rows are never n/a (W3-8)
The plan-review reality block offered pass|finding|n/a without saying tier 1
rows may never be n/a, so a plan-mode-no-op session wrote
`REALITY: deferred-asks n/a ...`, check returned repair, and the repair loop
ran past the 300 s capture (paid run 39004466422). The block now says tier
1 rows are never n/a and that with no linked issue deferred-asks is pass
with a receipt showing none. The captured line is pinned through check as
missing=deferred-asks verdict=repair.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(qa-functional): webhook fix runs re-verify the repaired invariant and rerun superseded captures (W3-6, W3-14)
W3-6: stage 2 of the fix-run prompt asked only for the original, happy,
cancel and dependency probes while verifyQANativeRegression reruns all
eight, so a status-only repair that left partial broken reached the verifier
(paid run 38986120559). The webhook fix prompt now states the capture budget
up front (at most 5 pre-repair captures, then the 7 post-repair scenarios:
duplicate, partial, concurrent-ab, concurrent-ba, happy, cancel, dependency;
12 in total; report incomplete if 5 cannot establish the defect). That list
feeds the capture nudge (GSTACK_QA_REQUIRED_PROBES);
QA_WEBHOOK_REQUIRED_SCENARIOS and the verdict are unchanged.
W3-14: the fix prompt now says to rerun every pre-repair capture the
verdict relies on after the final repair, and that a superseded row stays
open until its command reran on the repaired source (paid run 39053431488
labelled a pre-repair duplicate superseded and never reran it). The
detector and its negative test are unchanged; the captured shape joins them
as a negative control.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(qa): tie the exploratory stop rule to per-rule input probes; fix runs close superseded captures (W3-9, W3-14)
W3-9: two qa-functional-cli-report sessions stopped at 5-7 of 12 probes
without probing a non-digit amount (paid runs 38983306684, 39001107574),
though system-functional.md requires one probe per documented input rule.
The single stop rule shared by both exploratory templates now says each
documented input rule needs its own probe breaking only that rule, and
rules left uncovered at a probe or time cap go in limits.
W3-14: for fix runs (not qa-only), every capture taken before a fix is
superseded even when its client prints no snapshot; before completion the
agent reruns each one its verdict relies on or reports it as an open
remainder. Production superseded/open semantics are unchanged.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(qa-functional): keep the captured webhook-report cat attempts as denied negative controls (Q3)
Paid run 39006896632 (qa-functional-webhook-report) attempted
`cat package.json probe.ts .gitignore` and `cat src/*.ts test/*.ts`; the
PreToolUse hook denied both. This is a model miss, not a detector defect, so
both commands are pinned as denied by the registered hook and outside
qaCommandAllowed. No detector change.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(docs-candidate): per-subcommand --help and a compare reject that names the fix (W3-2)
snapshot --help and compare --help print the usage and exit 0 before any side
effect; the docsync observer admits exactly those two forms. compare names the
record path, each missing or wrong field (schema_version, base, hashes) and
the snapshot command that writes a valid record, keeping exit 1. The captured
ship-docsync-stale-after commands from PR gate run 38990111230 replay inside
the interface; help with any other argument stays rejected.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(docsync-fault): prepare writes the production candidate through the installed snapshot (W3-1)
The fault actor prepare wrote its own {base_sha, content_hashes, ...} record,
which the installed gstack-docs-candidate compare rejects, while
ship/sections/documentation.md requires running compare; 79 of 198 docsync
captures (and the red in PR gate run 38990111230) were pushed off the
documented path by it.
prepare now runs the installed snapshot under a 30 s timeout for the same
selection, returns its exact command as snapshot_command so the post-child
rerun is literal, publishes prompt and event only after a valid record
exists, and removes a record its own failed snapshot left. dispatch observes
the candidate with the same installed snapshot; inspect returns base, index
and per-file git blob ids in the record representation; the legacy re-audit
check reads selected/hashes. The fault prompt names snapshot_command and the
new field names.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* docs(todos): file the qa-quick capture headroom follow-up (fix wave 3, T)
qa-quick timed out once at 300 s (paid run 38980005793) against passing
runs of 164-272 s. Trimming what the Quick tier loads is a product call, so
it is filed, not repaired; no budget change.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(docsync): keep the two docsync model misses as negative controls (W3-3)
The Bash grep of ship-docsync-missing-asset (PR gate run 38983306684,
bash#13) and the composed cat; cat of ship-docsync-failure (38984977679,
bash#12) were correct rejections. They join the stored known_bad commands
(with a {home} placeholder for the fixture home); no detector change.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(golden): refresh the ship goldens for the W3-5 scope block and W3-7 consent letters
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(third-party-actions): state the absent-Aside letters in fewer characters (W3-7)
Same contract (B), C), D) when Aside is not detected; the next sentence
already omits the Aside option), 67 fewer characters on the pinned
land-and-deploy and office-hours skills.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(skill-md-size): re-pin 31 over-limit skills grown by fix wave 3 prompt repairs (owner sign-off needed)
CREW rule 11: these skills were already over 100,000 characters and grow
by the minimum text each repair needs; none crosses the limit for the
first time.
- gstack-review on cursor, factory, kiro, opencode, slate (+657..+676):
W3-5 runs gstack-state-matrix detect inside the scope block with its exit
status checked and declares the check step as a heredoc block; the new
fence carries the env-var-host runtime prelude (about 360 characters).
- gstack-plan-eng-review on 12 hosts (+104): W3-8 says tier 1 reality rows
are never n/a (paid run 39004466422 looped on a rejected n/a row).
- gstack-land-and-deploy on 12 hosts and gstack-office-hours on 4 hosts
(+21): W3-7 letters the absent-Aside consent options B), C), D).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(skills): fit the W3-5, W3-7 and W3-8 prompt repairs inside the parity budgets
The parity suite left 2-42 bytes of headroom on ship, review,
plan-eng-review, plan-ceo-review and land-and-deploy, so each repair now
pays for itself in the same resolver:
- W3-5: the scope block runs `source <(gstack-state-matrix detect --base
<base> || echo SCOPE_STATE_MACHINE=unknown)` (a failed detector's
stdout is overridden by the later unknown assignment; its stderr still
shows), the check block uses a short heredoc, and DIFF_LINES comes from one
`git diff --numstat | awk` sum instead of two --stat greps (same count:
insertions plus deletions, binary files zero).
- W3-7: the absent branch reads "offer only options B), C) and D)."
- W3-8: "never n/a" joins the tier 1 parenthesis, the no-linked-issue rule
joins the receipt sentence, and the deep header says the same in fewer
words; the redundant "(it prints the PLAN_REALITY: line)" is dropped.
Size pins: land-and-deploy (12 hosts, -20), office-hours (4 hosts, -20)
and plan-eng-review (12 hosts, -10) are lowered to their new sizes;
gstack-review on cursor, factory, kiro, opencode and slate stays above its
main pin (+355..+374) because the declared check block is a new fence that
carries the env-var-host runtime prelude. Raising those five pins needs
owner sign-off.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(design-e2e): the shim-dom scan check resolves the command's literal _REPORT/_RUN (W3-10)
PR gate run 38980005793 went red because the agent ran the generated DOM
block, scanning "$_REPORT/dom/$_RUN" with _RUN="run1" assigned in the same
command, and the check looked for the literal dom/run1 anywhere in a command.
domScanOfRun substitutes that command's own literal _REPORT=/_RUN=
assignments and credits a scan only when one of its own arguments is the run
directory. scan --changed, another directory, quoted heredoc report text and
non-literal or conflicting assignments stay negative controls. All 15
captured shim-dom runs in the window pass the new check (14 passed the old).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(pty): keep the last non-blank viewport for failure dumps (W3-11)
autoplan-guard-pty (PR gate run 38976253774) stalled at 489 s and its screen
dump, the tail of visibleText(), was only spinner redraws. Live PTY sessions
now remember the last non-blank viewport a screen read returned
(session.lastScreen()) and the harness wait-timeout dumps append it.
Diagnostic only: visibleText(), matching and verdicts are unchanged.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(autoplan): explicit 30 s timeout for each answer-invalidation test (W3-12)
The eng-only override test ran 4.1-4.4 s on a 4-core machine against Bun's
5 s default (7.8-9.4 s per file). Each test now names a 30 s budget.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(review-army): size the nested bun test budgets for windows-latest (W3-15)
windows-free-tests run 39053431430 (shard 5): the nested bun test exited 1 in
5.58 s because Bun's default 5 s hook timeout fired on the real Red Team
beforeAll (8 git spawns plus skill-section copies). Both nested argvs pass
--timeout 30000, which raises only the nested default; the registrations
under test keep their explicit timeouts. Spawn timeouts move from 8 s to
40 s and outer test timeouts to 45 s, including the consensus test that had
none. With a 6 s beforeAll injected, the lifecycle still matches; the old
argv reproduces the Windows hook timeout.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* docs(todos): close the createQAFunctionalFixture entry and file D6 (W3-13)
The git auto-maintenance entry was fixed by 0bf9d50 (#3190). D6: nested bun
test output pollutes free-runner failure attribution in
scripts/lib/shard-engine.ts (windows-free-tests 39053431430).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(free-fixtures): declare the W3-4 captured caller transcript as a free-only fixture
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(pty): strip cursor-visibility escapes and print the last screen on autoplan-guard-pty failure (W3-11)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(review-army): keep the state-machine detector's output only on exit 0 (W3-5 confirm round)
A failing gstack-state-matrix detect prints SCOPE_STATE_MACHINE=false before
exiting non-zero. The scope block now captures its output and sources it only
when the detector succeeds; otherwise it discards the output, sets
SCOPE_STATE_MACHINE=unknown (dispatching the State Machines specialist) and
leaves the detector's stderr visible. Free tests cover false-output/non-zero,
empty-output/non-zero and a late failure after printing true. The stack
comment is shortened so gstack-review stays 4 chars under its pin; the five
host pins are lowered to match.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(qa-functional): reserve webhook post-repair reruns first inside the unchanged 12-capture ceiling (W3-6/W3-14 confirm round)
The fix prompt now states precedence: post-repair reruns (the 7 stage-2
scenarios plus each distinct pre-repair command the verdict relies on, such as
a reject probe) are reserved first, pre-repair exploration gets what remains,
and reruns that cannot fit are reported as open. A free test fits a pre-repair
reject plus the reproduced defect and its replay within 12 (11 captures).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(docs-candidate): the compare reject never sends an agent to re-snapshot after the child audit (W3-2)
A post-audit snapshot over the candidate replaces the before-audit baseline, so
the ownership check would compare current files with themselves. The fix line
now says the candidate must be the record the Prepare snapshot wrote, that the
Prepare snapshot is rerun only before dispatching the child audit, and that the
post-child snapshot uses a different --out (<audit-id>-post.json). The helper
test pins the text.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(docsync-fault): diagnosable prepare snapshot failures with one fresh-id retry (W3-1)
A failed installed snapshot now returns exit 25 with a stable reason code
(snapshot_exit, snapshot_timeout, snapshot_spawn_error, snapshot_invalid_record),
the helper's stderr tail redacted through lib/redact-engine with the fixture
root as <HOME> and bounded to 800 characters, and keeps the attempt's files: the
candidate the helper left, if any, and prepare-<audit-id>-failure.json. The
failure is a prepare-failed event, not a rejected interaction. One retry is
allowed under a fresh audit id; reusing the failed id, dispatching it or a third
prepare is rejected. Tested before and after candidate creation. The two
snapshot tests get an explicit 30 s timeout: the compare/inspect agreement
test runs 4.3-4.8 s alone and timed out at Bun's 5 s default under load.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(qa-evidence): count required post-repair probes only after the fixture-declared final source edit (W3-6/W3-14 confirm round)
GSTACK_QA_REQUIRED_PROBES reminders counted any complete capture, so a
pre-repair duplicate or partial capture looked like post-repair coverage. The
caller may now declare GSTACK_QA_REQUIRED_AFTER_EDIT with absolute paths and a
clean-fixture time: only captures started after the last edit under those
paths count, none count before the first edit, and a malformed declaration
counts none. Without the declaration every capture counts as before. The
webhook fix fixture declares its src/ directory. A free test covers
pre-repair-only coverage (nudges), complete post-repair coverage (no nudge)
and a second edit (nudges again).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(harness): bump HARNESS_VERSION to 5 for the W3-11 stripAnsi change
stripAnsi now removes cursor-visibility escapes (ESC[?25h/l). Every PTY case
matches patterns against its output, so a pattern spanning one of those escapes
can now match where it did not before: a verdict can change, so this is a
behavioral bump, not a non-behavioral entry. The launch.ts change (last
non-blank screen in failure messages) is diagnostic only.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(review-army): pass the state-machines output to check as a written file, not a heredoc placeholder (W3-5)
The generated-bash lint (env-host-fence-conformance INV-3) rejects free text in
a heredoc body: <its complete output> reached the shell. The agent now writes
the specialist's full output to .gstack/tmp/state-machines.md and the fence
runs gstack-state-matrix check on that file. Two stack-detection comments and
the parenthetical are trimmed to stay inside the review/ship parity ratios;
gstack-review shrinks by 33 on the five pinned hosts and their pins are
lowered to match.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(autoplan-guard-pty): the scripted actor ends its turn after a background CEO dispatch before reading the close packet
PR gate 39070276914 (dc739be) failed at the close-packet check while gate
39069171159 (b789898) passed with byte-identical autoplan inputs. The evidence
differs only in order: the failing actor read the close packet in the dispatch
turn, so the first tool call after the CEO notice was the publication. The
product guard allowed all four guarded calls in both runs. The fixture step 5
told the actor to end its turn only conditionally; step 5 now ends the turn as
soon as the result says the agent runs in the background, and step 6 reads the
close packet only after the reviewer finished. The close-packet check moves to
closePacketReadAfterNotice, replayed in a free test against both captured
orders; a second free test pins the actor's steps 5 and 6.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* chore(release): v1.91.127.0 fix wave 3 paid-gate reliability round 2
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(qa): input-rule probes are per field, and a value breaking two rules covers neither (W3-9)
PR gate 39073934326 (93449bc) failed qa-functional-cli-report with no observed
unannounced defect, the same class as 38983306684 and 39001107574. The agent
read 'allowed characters' as the ID rule only, counted -5 as its sign probe
and never probed the amount's digits-only rule with an otherwise valid value,
so the seeded parseInt defect stayed hidden. The stop rule now says each
field's documented input rule needs a probe breaking only it, and that a
signed amount breaks sign and digits-only and covers neither; the functional
table row names input rules per field. The qa union stays inside its parity
ratio.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(qa): state the input-rule principle instead of a worked example (W3-9)
The signed-amount example in 0fe616e was misread in the focused rerun as 'a
signed amount breaks only the digits-only rule', and the run again missed
the seeded non-digit defect. The stop rule now states the principle: each
field's documented input rule needs a probe that breaks only it and satisfies
every other rule.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(tpa): the Darwin pitch detector accepts the canonical phrase anywhere, short forms only at a sentence start
PR gate 39075977549 (d6d82fb) failed tpa-absent-darwin on a correct pitch:
'One-time note: you can download Aside (macOS 15+) at aside.com, then open it,
sign in, and re-run'. The contract is to deliver the pitch once; the
sentence-start anchor exists only so narration is not counted. The full
canonical phrase now counts anywhere unless a negation precedes it, and the
short forms (download it / download Aside at aside.com) keep the
sentence-start rule. The detector moves to test/helpers/third-party-actions.ts
with a count helper. Free tests replay the captured red (counts once), the
four captured passing Darwin pitches (once each), five Linux and five READY
narrations plus both recovery-public responses (zero), a mid-sentence short
form and negated phrases (zero), and a duplicated phrase (two).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(qa-callers): only a required plan check may swap the deadline for --timeout-ms; reruns keep the deadline
PR gate 39077524802 (0f71a55) failed ship-exploratory-late-input: after the
late input change the actor reran captures 001 and 002 with --timeout-ms
10000, reading the fixture's 'explicit plan probes outside the smoke budget'
clause as covering revalidation reruns, and the verifier reported both as
smoke probes missing a trusted deadline run. The binding now says only an
explicit plan check this phase requires may use --timeout-ms, and every other
capture, including a rerun or revalidation, uses the deadline.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(third-party-actions): count the Darwin pitch only at a recommendation boundary
/review delta on 51f1e02 (Claude and GPT-6.1 Sol voices): the 0f71a55
detector accepted the canonical phrase anywhere unless the adjacent word was
a negation, so a Darwin refusal or a quoted policy mention ('I will not ask
you to download Aside ...', 'The Darwin policy says to "download Aside ..."',
'You cannot download ...') satisfied the paid case's positive match, and a
free assertion even expected "You don't need to download ..." to count.
The canonical phrase now counts only where a recommendation can start: a
sentence, clause (, : ; dash), bullet, line or bold start, optionally led by
then/please/just/so/first/now and you can/could/may/should/need to, with no
negation directly before it across a line break. Quoted, parenthesised,
reported and clause-negated mentions fall outside that boundary. Every probe
from both voices is a free test; the five captured Darwin pitches (including
the 39075977549 red) still count 1 each and the Linux/READY captures 0.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(review-army): write the state-machines output to a fresh private file from gstack-state-matrix outfile
/review delta on 51f1e02 (Claude F2): dc739be wrote the specialist output to
a fixed .gstack/tmp/state-machines.md with no exclude. On a second review
Claude Code's Write refuses an existing unread file, so check could validate
the previous run's output, and git add -A could commit it.
gstack-state-matrix outfile creates .gstack/tmp (0700) at the repo top,
adds /.gstack/tmp/ to info/exclude once, and prints a new empty 0600 file
per call; the selection prose has the agent Read that file, write the full
output there and check that path. The prose moved the check command inline,
so gstack-review shrinks and its pins are lowered (cursor 141010->140637,
factory 144792->144416, kiro 140816->140449, opencode 143258->142872,
slate 140913->140543).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(qa-evidence): pin the post-repair edit's mtime to capture 002's completion
The GPT voice's /review run on 51f1e02 failed qa-evidence.test.ts:547 once:
capture 003 reported no required probe remaining, so capture 002 (started
before the edit) counted as post-edit. The test wrote the edit right after
002 returned, a gap of one child run (7-27 ms measured here; the voice's
machine ran this test in 59 ms against 183 ms here, so a few ms there),
while an ext4 mtime comes from the kernel's coarse clock and lags
Date.now() by up to a tick (0.5-4.5 ms measured here).
The edit's mtime is now set to 002's recorded completedAt, so 002 started
strictly before the boundary and 003 starts after it on any machine. The
product boundary is unchanged: tick-scale imprecision only matters when a
capture starts within milliseconds before an edit, and a stricter margin
would refuse a real edit && capture chain.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
---------
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
496 lines
27 KiB
TypeScript
496 lines
27 KiB
TypeScript
/**
|
||
* Third-party web actions contract pins (Aside is the RECOMMENDED driver,
|
||
* gstack's own stack — `$B` headed mode + handoff/resume, GStack Browser —
|
||
* the universal fallback; CEO review D2-D9 + eng review E1-E10 pins carried
|
||
* forward).
|
||
*
|
||
* The contract's load-bearing sentences are pinned here so no future edit can
|
||
* quietly strip the consent gate, the install ban, the credential boundaries,
|
||
* or the failure path — the fork this contract was adapted from carried +24
|
||
* parity checks for exactly this reason, and lost its credential ban once to
|
||
* a "compression" that a release run promptly exploited.
|
||
*
|
||
* Two scopes:
|
||
* - resolver output (the section itself): consent, boundaries, failure path,
|
||
* the Aside-first option set with the gstack drive as fallback.
|
||
* - repo-wide generated markdown: Aside command allowlist (the probe +
|
||
* cookbook verbs only — no `aside mcp`, no invented subcommands) and no
|
||
* Aside-specific installer invocation anywhere.
|
||
*/
|
||
import { describe, test, expect } from "bun:test";
|
||
import * as fs from "fs";
|
||
import * as path from "path";
|
||
import { Glob } from "bun";
|
||
import { marked } from "marked";
|
||
import { generateThirdPartyActions } from "../scripts/resolvers/third-party-actions";
|
||
import { generateAsideSetup } from "../scripts/resolvers/aside";
|
||
import { HOST_PATHS } from "../scripts/resolvers/types";
|
||
import { asideDownloadPitch, asideDownloadPitchCount, asideDriveOptions } from './helpers/third-party-actions';
|
||
import recoveryFixture from './fixtures/third-party-actions-recovery-public.json';
|
||
import pitchCaptures from './fixtures/third-party-actions-pitch-captures.json';
|
||
import { expectMentions, expectTokens } from './helpers/prompt-structure';
|
||
|
||
const ROOT = path.resolve(import.meta.dir, "..");
|
||
|
||
const ctx = {
|
||
skillName: "ship",
|
||
tmplPath: "",
|
||
host: "claude" as const,
|
||
paths: HOST_PATHS["claude"],
|
||
};
|
||
|
||
const section = generateThirdPartyActions(ctx);
|
||
|
||
describe('consent offer extraction', () => {
|
||
test('an unavailable-option explanation is not an offer', () => {
|
||
expect(asideDriveOptions(`B) I drive it in gstack's own visible browser
|
||
C) Manual instructions
|
||
D) Defer
|
||
|
||
(Option A, driving in your Aside browser, is unavailable because Aside was not detected.)`)).toEqual([]);
|
||
});
|
||
|
||
test('detects plain, bulleted, bold, and multiline drive offers', () => {
|
||
for (const option of [
|
||
'A) I drive it in your Aside browser',
|
||
'- **A)** I drive it in your Aside browser',
|
||
'**A)** I drive it in your Aside browser',
|
||
'A. I drive it in your Aside browser',
|
||
'A) Open the Aside app first\n then I drive the token creation.',
|
||
]) {
|
||
expect(asideDriveOptions(option), option).toHaveLength(1);
|
||
}
|
||
});
|
||
|
||
test('allows recovery questions but rejects conditional drive consent', () => {
|
||
expect(asideDriveOptions('A) Open the Aside app so I can re-run the probe.')).toEqual([]);
|
||
// Captured tpa-absent-linux answer (paid run 38988927989): the English idiom is not the Aside browser.
|
||
expect(asideDriveOptions('D) Defer. I set this aside and mark it blocked on you, with no browsing now.')).toEqual([]);
|
||
expect(asideDriveOptions('D) Defer, or I drive it in Aside later.')).toHaveLength(1);
|
||
expect(asideDriveOptions('A) Open the Aside app; if READY, I drive the dashboard.')).toHaveLength(1);
|
||
});
|
||
|
||
test.each(recoveryFixture.responses)('native recovery response $attempt defers drive consent to a new question', ({ text }) => {
|
||
expect(asideDriveOptions(text)).toEqual([]);
|
||
expect(asideDriveOptions(text.replace('option included.', 'option included; then I drive in your Aside browser.'))).toHaveLength(1);
|
||
});
|
||
|
||
test('a future question can name its option without offering the drive now', () => {
|
||
for (const subject of ["I'll", 'I will', 'We’ll', 'we will']) {
|
||
for (const question of ['re-ask', 'ask again', 're-ask this question']) {
|
||
for (const label of ['the Aside drive', 'the "drive it in your Aside browser"', 'the “drive it in your Aside browser”']) {
|
||
const reference = `${subject} ${question} with ${label} option included`;
|
||
expect(asideDriveOptions(`A) Open Aside and re-probe; if READY, ${reference}.`)).toEqual([]);
|
||
expect(asideDriveOptions(`A) I drive in Aside first; ${reference}.`)).toHaveLength(1);
|
||
expect(asideDriveOptions(`A) Open Aside; ${reference}, then I drive the dashboard.`)).toHaveLength(1);
|
||
expect(asideDriveOptions(`A) Open Aside; ${reference} and navigate the dashboard before that question.`)).toHaveLength(1);
|
||
}
|
||
}
|
||
}
|
||
expect(asideDriveOptions('A) I will re-ask after I drive in Aside with the drive option included.')).toHaveLength(1);
|
||
expect(asideDriveOptions('A) Open Aside; I will re-ask with the Aside drive option, then I browse using that option.')).toHaveLength(1);
|
||
});
|
||
});
|
||
|
||
/** Generated skill markdown: every SKILL.md + carved sections at repo root. */
|
||
function generatedSkillDocs(): string[] {
|
||
const files: string[] = [];
|
||
for (const pattern of ["*/SKILL.md", "*/sections/*.md", "openclaw/skills/*/SKILL.md"]) {
|
||
for (const f of new Glob(pattern).scanSync({ cwd: ROOT })) {
|
||
files.push(path.join(ROOT, f));
|
||
}
|
||
}
|
||
return files;
|
||
}
|
||
|
||
/**
|
||
* Extract `aside <token>` command usages from inline code spans and fenced
|
||
* blocks, plus prose-form imperatives naming a known subcommand (exec, repl,
|
||
* mcp) anywhere in the text. Requires whitespace after `aside`, so prose
|
||
* ("aside from"), CSS selectors (`aside[class*=...]`), and domains
|
||
* (aside.com) never match.
|
||
*/
|
||
function asideCommandTokens(text: string): string[] {
|
||
if (!/\baside\s+(?:--?[A-Za-z]|[a-z][\w-]*)/.test(text)) return [];
|
||
const tokens: string[] = [];
|
||
const codeChunks: string[] = [];
|
||
marked.walkTokens(marked.lexer(text, { gfm: false }), token => {
|
||
if (token.type === 'code' || token.type === 'codespan') codeChunks.push(token.text);
|
||
});
|
||
for (const chunk of codeChunks) {
|
||
for (const m of chunk.matchAll(/(?:^|[\s;&|(])aside\s+(skills[ \t]+[a-z][\w-]*|--?[A-Za-z][\w-]*|[a-z][\w-]*)/g)) {
|
||
tokens.push(m[1].replace(/[ \t]+/g, ' '));
|
||
}
|
||
}
|
||
// Prose-form drift: an instruction like "then run aside mcp against the
|
||
// dashboard" never appears in a code span, so scan the whole text for the
|
||
// vendor's known subcommand names too.
|
||
for (const m of text.matchAll(/\baside\s+(exec|repl|mcp)\b/g)) {
|
||
tokens.push(m[1]);
|
||
}
|
||
return tokens;
|
||
}
|
||
|
||
describe('Aside command extraction boundaries', () => {
|
||
test.each(['```bash', '````bash', '~~~bash'])('prose after a %s fence is not inline code', fence => {
|
||
const closing = fence.replace('bash', '');
|
||
const text = [fence, 'ls DESIGN.md', closing, '',
|
||
'Set aside prior visual choices; put aside old assumptions.', '',
|
||
'Continue with `DESIGN.md`.'].join('\n');
|
||
expect(asideCommandTokens(text)).toEqual([]);
|
||
});
|
||
|
||
test.each([
|
||
'`aside invented`',
|
||
'``aside invented `literal` ``',
|
||
'```bash\naside invented\n```',
|
||
'````bash\naside invented\n```\n````',
|
||
'~~~bash\naside invented\n~~~',
|
||
'- Run:\n\n ```bash\n aside invented\n ```',
|
||
'> ```bash\n> aside invented\n> ```',
|
||
' aside invented',
|
||
'| Command |\n| --- |\n| `aside invented` |',
|
||
])('still detects unsupported commands in %s', text => {
|
||
expect(asideCommandTokens(text)).toContain('invented');
|
||
});
|
||
|
||
test('retains prose-form drift detection without treating ordinary aside prose as a command', () => {
|
||
expect(asideCommandTokens('Then run aside mcp against the dashboard.')).toContain('mcp');
|
||
expect(asideCommandTokens('Set aside prior choices, aside from constraints; visit aside.com.')).toEqual([]);
|
||
});
|
||
|
||
test('the documented read-only skill listing does not allow installation or invented skill actions', () => {
|
||
for (const command of ['aside skills list', 'aside skills list']) {
|
||
expect(asideCommandTokens('`' + command + '`')).toEqual(['skills list']);
|
||
expect(ASIDE_ALLOWLIST).toContain('skills list');
|
||
}
|
||
for (const command of ['aside skills install', 'aside skills invented', 'aside skills']) {
|
||
expect(asideCommandTokens('`' + command + '`')).toEqual([command.slice('aside '.length)]);
|
||
expect(ASIDE_ALLOWLIST).not.toContain(command.slice('aside '.length));
|
||
}
|
||
});
|
||
});
|
||
|
||
const ASIDE_ALLOWLIST = ["--version", "--help", "repl", "exec", "skills list"];
|
||
|
||
describe("THIRD_PARTY_ACTIONS contract pins", () => {
|
||
// (a) Aside is named as the RECOMMENDED driver, with the download pointer +
|
||
// macOS floor, and gstack's own stack as the fallback on every platform.
|
||
test("names Aside as the recommended driver, gstack's stack as the fallback", () => {
|
||
expect(section).toContain("aside.com");
|
||
expect(section).toContain("macOS 15+");
|
||
expect(section).toContain("GStack Browser when installed");
|
||
});
|
||
|
||
// Detection probe: the same readiness probe as {{ASIDE_SETUP}} — portable
|
||
// timeout guard, three named outcomes, explicit Darwin gate on the pitch.
|
||
test("runtime probe is the BROWSER SETUP probe with a Darwin-gated pitch", () => {
|
||
expect(section).toContain("command -v aside");
|
||
expect(section).not.toContain("aside --version");
|
||
expect(section).toContain('echo "READY: $_A"'); // E7: "aside" on PATH, else the ~/.local/bin path
|
||
expect(section).toContain("ASIDE_UNAVAILABLE");
|
||
expectMentions(section, [['never', 'diagnostics', 'report']], 'section');
|
||
expect(section).toContain("NEEDS_ASIDE");
|
||
expect(section).toContain("ASIDE_NOT_RUNNING");
|
||
expect(section).toContain("ASIDE_READY");
|
||
// Stock macOS ships neither gtimeout nor timeout(1) — the guard must be
|
||
// conditional, never a bare `timeout N aside` invocation.
|
||
expect(section).toContain("command -v gtimeout");
|
||
expect(section).not.toMatch(/\btimeout \d+ aside/);
|
||
expect(section).toContain("`NEEDS_ASIDE: Darwin` (trust it; don't re-probe)");
|
||
expectMentions(section, [['do not', 'macos', 'pitch']], 'section');
|
||
});
|
||
|
||
// The probe is LIFTED from {{ASIDE_SETUP}}, not copied: a probe fix after an
|
||
// Aside release lands in both places or the render fails loudly.
|
||
test("probe is byte-identical to the {{ASIDE_SETUP}} probe", () => {
|
||
const asideProbe = generateAsideSetup(ctx).match(/```bash\n([\s\S]*?)```/)![1].trimEnd();
|
||
const tpaProbe = section.match(/```bash\n([\s\S]*?)```/)![1].trimEnd()
|
||
.split("\n").map((l) => l.replace(/^ {3}/, "")).join("\n");
|
||
expect(tpaProbe).toBe(asideProbe);
|
||
});
|
||
|
||
// Rule 3 sends the agent to the /browse skill doc for HOW to drive. That
|
||
// keeps the ~10KB Aside contract out of every planning skill that embeds
|
||
// this section (ship, spec, setup-deploy, office-hours) while still never
|
||
// letting an agent write `aside repl` from memory.
|
||
test("rule 3 points at browse/SKILL.md for HOW to drive; planning skills do not embed {{ASIDE_SETUP}}", () => {
|
||
expect(section).toContain("Read the /browse skill (`browse/SKILL.md`");
|
||
expect(section).toContain("one flow per script");
|
||
for (const f of ["ship/SKILL.md.tmpl", "spec/SKILL.md.tmpl", "setup-deploy/SKILL.md.tmpl", "office-hours/SKILL.md.tmpl"]) {
|
||
const tmpl = fs.readFileSync(path.join(ROOT, f), "utf-8");
|
||
expect({ f, tpa: tmpl.includes("{{THIRD_PARTY_ACTIONS}}"), aside: tmpl.includes("{{ASIDE_SETUP}}") }).toEqual({ f, tpa: true, aside: false });
|
||
}
|
||
});
|
||
|
||
// (b) per-task consent, never persisted; options conditional on detection.
|
||
test("per-task consent, never persisted, detection-conditional options", () => {
|
||
expectMentions(section, [['never', 'permission', 'standing']], 'section');
|
||
expect(section).toContain("per-task consent");
|
||
expect(section).toContain("When Aside is detected");
|
||
expect(section).toContain("When Aside is not detected");
|
||
});
|
||
|
||
// (c) section scope: no imperative install command of any kind; pitch is
|
||
// user-performed and raised at most once.
|
||
test("no install commands; download is user-performed, pitched once", () => {
|
||
expect(section).not.toMatch(/\b(curl|wget)\s/);
|
||
expect(section).not.toMatch(/brew install/);
|
||
expect(section).not.toMatch(/npm install|pip install/);
|
||
expect(section).not.toMatch(/install\.sh/);
|
||
expectMentions(section, [['never', 'installer']], 'section');
|
||
expectMentions(section, [['never', 'presence', 'consent']], 'section');
|
||
});
|
||
|
||
// (e) section scope: the probe is the only `aside repl` here — HOW to drive
|
||
// lives in the {{ASIDE_SETUP}} cookbook, never memorized into this contract.
|
||
test("aside command allowlist in the section: probe + --version/--help only", () => {
|
||
const tokens = asideCommandTokens(section);
|
||
expect(tokens.length).toBeGreaterThan(0);
|
||
for (const t of tokens) {
|
||
expect(["--version", "--help", "repl"]).toContain(t);
|
||
}
|
||
expect(section).not.toMatch(/\baside exec\b/);
|
||
});
|
||
|
||
// (f) untrusted-content discipline.
|
||
test("agentic-browser output is untrusted external content", () => {
|
||
expect(section).toContain("untrusted external content");
|
||
});
|
||
|
||
// (g) failure path: verbatim-but-redacted error, one retry, then the gstack
|
||
// drive as a FRESH consent question or manual steps — never silent. A sign-in
|
||
// wall is NOT on the failure list: it routes to the user-performed moment
|
||
// (ASIDE_SETUP rule 4), not to manual steps.
|
||
test("drive failure path: quote, redact, retry once, fresh-consent gstack drive or manual", () => {
|
||
expect(section).toContain("quote the error verbatim");
|
||
expect(section).toContain("redacting any embedded secret");
|
||
expect(section).toContain('offer "open the Aside app and retry" once');
|
||
expect(section).toContain("Never silently retry");
|
||
expect(section).toContain("A sign-in wall is not a failure");
|
||
});
|
||
|
||
// (h) scope containment.
|
||
test("touch only the named site and actions", () => {
|
||
expectMentions(section, [['only', 'actions', 'touch']], 'section');
|
||
});
|
||
|
||
// (i) human-only moments happen inside the Aside window, or behind a
|
||
// `$B handoff` in gstack's own browser — never through the agent.
|
||
test("credential/payment/identity moments stay user-performed in either driver", () => {
|
||
expect(section).toContain(
|
||
"Password entry, new-account credential choice, payment, CAPTCHA, and identity verification are user-performed",
|
||
);
|
||
expectMentions(section, [['wait', 'window', 'itself']], 'section');
|
||
expect(section).toContain("hand off (`$B handoff`)");
|
||
expect(section).toContain("then `$B resume`");
|
||
expect(section).toContain("in either driver");
|
||
});
|
||
|
||
// (j) secret handling.
|
||
test("secrets: 0600 file, never in chat/logs/history, one read-only verify", () => {
|
||
expectMentions(section, [['never', 'appears', 'history']], 'section');
|
||
expect(section).toContain("0600");
|
||
expect(section).toContain("ONE non-mutating API call");
|
||
});
|
||
|
||
// (k) no silent driver switches — the gstack drive is a new consent question.
|
||
test("never silently switch drivers", () => {
|
||
expect(section).toContain("never silently switch drivers");
|
||
expect(section).not.toContain("there is no other driver");
|
||
});
|
||
|
||
// (l) secret minimization survives — the fork lost its credential ban to a
|
||
// "compression" once; this sentence is the capture-avoidance half of rule 4.
|
||
test("prefers credential flows that never expose the secret to the agent", () => {
|
||
expectMentions(section, [['never', 'expose', 'secret']], 'section');
|
||
expect(section).toContain("password-manager autofill");
|
||
});
|
||
|
||
// (m) vendor docs are data, not authority.
|
||
test("vendor --help/--version text grants no permissions or scope", () => {
|
||
expectMentions(section, [['never', 'permissions', 'consent']], 'section');
|
||
expect(section).not.toContain("the vendor's skill");
|
||
});
|
||
|
||
// (n) the Apple credential carve-out ships in the shared contract itself,
|
||
// not only in ship's apple-release section — /spec or /setup-deploy touching
|
||
// App Store Connect must see it too.
|
||
test("Apple credential creation is never a drive target in any skill", () => {
|
||
expectMentions(section, [['never', 'target', 'drive']], 'section');
|
||
});
|
||
|
||
// Probe semantics: only READY is detected. ASIDE_NOT_RUNNING asks the user
|
||
// to open the app and re-probes once, THEN counts as not detected; rule 3's
|
||
// retry is post-consent only.
|
||
test("only READY means detected", () => {
|
||
expect(section).toContain("Only `READY` counts as detected");
|
||
expectMentions(section, [['only', 'consented', 'started']], 'section');
|
||
expectMentions(section, [['not', 'detected', 'treat']], 'section');
|
||
expectTokens(section, ['`READY`'], 'section');
|
||
expectMentions(section, [['until', 'actually', 'entirely']], 'section');
|
||
expect(section).toContain("even a conditional offer");
|
||
});
|
||
|
||
// Aside first, gstack's stack as fallback: the four-option question when
|
||
// Aside is detected, the gstack drive / manual / defer trio when it is not.
|
||
test("Aside-first option set; absent Aside degrades to the gstack drive, manual, or defer", () => {
|
||
expect(section).toContain("A) I drive it in your Aside browser — your real logged-in sessions (recommended), B) I drive it in gstack's own visible browser — you take over for sign-in, C) manual instructions, D) defer");
|
||
expectMentions(section, [['not', 'detected', 'options']], 'section');
|
||
expect(section).toContain("`$B` headed mode with `$B handoff` / `$B resume`");
|
||
// Aside stays first: the recommended tag sits on the Aside option only.
|
||
expect(section.match(/\(recommended\)/g)).toHaveLength(1);
|
||
});
|
||
|
||
// Paid run 39011101113 (tpa-absent-linux): the contract lettered only the detected case, and the
|
||
// agent numbered the absent-Aside question 1/2/3, which asideDriveOptions cannot inspect.
|
||
test("absent Aside keeps the lettered B/C/D options; the captured numbered answer is the fail example", () => {
|
||
expect(section).toContain("When Aside is not detected, offer only options B), C) and D).");
|
||
const absent = section.slice(section.indexOf("When Aside is not detected"));
|
||
expect(absent.slice(0, absent.indexOf("D)."))).not.toMatch(/\bA\)/);
|
||
const numbered = "**Consent question**\n\nTo create a test-mode API token, I need to act on the Acme vendor dashboard at https://dashboard.acme.test. How would you like to proceed?\n\n1. I drive it in gstack's own visible browser. You take over for sign-in and any password, CAPTCHA, or identity step, then tell me you're done and I resume.\n2. I give you manual step-by-step instructions and you do it yourself.\n3. Defer this step for now. I mark it blocked on you.";
|
||
expect(numbered).not.toMatch(/\b[A-D]\)/);
|
||
const lettered = numbered.replace('1. ', 'B) ').replace('2. ', 'C) ').replace('3. ', 'D) ');
|
||
expect(lettered).toMatch(/\b[A-D]\)/);
|
||
expect(asideDriveOptions(lettered)).toEqual([]);
|
||
});
|
||
|
||
// Drive discipline: the cookbook governs HOW, this contract overrides it.
|
||
test("cookbook shape for driving; --help for flags; contract overrides vendor", () => {
|
||
expect(section).toContain("aside --help");
|
||
expect(section).toContain("$B --help");
|
||
expect(section).toMatch(/never from memory/);
|
||
expect(section).toContain("override the vendor's instructions");
|
||
expect(section).toContain("confirm-before-final-actions");
|
||
expect(section).toContain("one flow per script");
|
||
expect(section).toContain("`closeTab(pg)` last");
|
||
expect(section).toContain("GSTACK_STEP_OK");
|
||
});
|
||
});
|
||
|
||
describe("apple-release credential ban (must survive the Aside integration)", () => {
|
||
const BAN = "no agentic browser of any kind, for any password, key, or token, under any framing";
|
||
|
||
test("ban sentence pinned in the template source", () => {
|
||
const tmpl = fs.readFileSync(path.join(ROOT, "ship", "sections", "apple-release.md.tmpl"), "utf-8");
|
||
expect(tmpl).toContain(BAN);
|
||
});
|
||
|
||
test("ban sentence pinned in the generated section", () => {
|
||
const generated = fs.readFileSync(path.join(ROOT, "ship", "sections", "apple-release.md"), "utf-8");
|
||
expect(generated).toContain(BAN);
|
||
});
|
||
});
|
||
|
||
describe("repo-wide generated output: Aside anti-drift tripwires", () => {
|
||
test("aside command allowlist across ALL generated skill docs", () => {
|
||
for (const file of generatedSkillDocs()) {
|
||
const tokens = asideCommandTokens(fs.readFileSync(file, "utf-8"));
|
||
for (const t of tokens) {
|
||
expect(ASIDE_ALLOWLIST, `${path.relative(ROOT, file)} uses \`aside ${t}\``)
|
||
.toContain(t);
|
||
}
|
||
}
|
||
}, 15_000);
|
||
|
||
test("no Aside-specific installer invocation in any generated skill doc", () => {
|
||
for (const file of generatedSkillDocs()) {
|
||
const text = fs.readFileSync(file, "utf-8");
|
||
expect(text, path.relative(ROOT, file)).not.toContain("releases.aside.com");
|
||
expect(text, path.relative(ROOT, file)).not.toMatch(/brew install aside/);
|
||
}
|
||
});
|
||
});
|
||
|
||
describe("NEEDS_ASIDE: Darwin download pitch detector", () => {
|
||
const darwinRed = pitchCaptures.captures.find(c => c.case === 'tpa-absent-darwin' && c.run === '39075977549')!;
|
||
|
||
test("the captured d6d82fb pitch (gate 39075977549) counts: the canonical phrase mid-sentence", () => {
|
||
expect(darwinRed.passed).toBe(false);
|
||
expect(darwinRed.text).toContain('you can download Aside (macOS 15+) at aside.com');
|
||
expect(darwinRed.text).toMatch(asideDownloadPitch);
|
||
expect(asideDownloadPitchCount(darwinRed.text)).toBe(1);
|
||
});
|
||
|
||
test("every captured passing Darwin pitch counts exactly once", () => {
|
||
const passing = pitchCaptures.captures.filter(c => c.case === 'tpa-absent-darwin' && c.passed);
|
||
expect(passing.length).toBeGreaterThanOrEqual(4);
|
||
for (const c of passing) expect(asideDownloadPitchCount(c.text), c.run).toBe(1);
|
||
});
|
||
|
||
test("captured Linux and READY narrations and both recovery-public responses never count", () => {
|
||
const narrations = pitchCaptures.captures.filter(c => c.case !== 'tpa-absent-darwin');
|
||
expect(narrations.map(c => c.case).sort()).toEqual([...Array(5).fill('tpa-absent-linux'), ...Array(5).fill('tpa-present')]);
|
||
for (const c of narrations) {
|
||
expect(c.text, `${c.case} ${c.run}`).not.toMatch(asideDownloadPitch);
|
||
expect(asideDownloadPitchCount(c.text)).toBe(0);
|
||
}
|
||
expect(recoveryFixture.responses).toHaveLength(2);
|
||
for (const response of recoveryFixture.responses) {
|
||
expect(response.text).toMatch(/no (?:Aside )?download pitch/);
|
||
expect(response.text).not.toMatch(asideDownloadPitch);
|
||
}
|
||
});
|
||
|
||
test("short forms count only at a sentence start; negated canonical phrases never count", () => {
|
||
expect('you can download it at aside.com').not.toMatch(asideDownloadPitch);
|
||
expect('you can download Aside at aside.com').not.toMatch(asideDownloadPitch);
|
||
expect('Download it at aside.com').toMatch(asideDownloadPitch);
|
||
expect('Note: download Aside at aside.com.').toMatch(asideDownloadPitch);
|
||
expect('Do not Download Aside (macOS 15+) at aside.com').not.toMatch(asideDownloadPitch);
|
||
expect("You don't need to download Aside (macOS 15+) at aside.com").not.toMatch(asideDownloadPitch);
|
||
expect("I won't download Aside (macOS 15+) at aside.com").not.toMatch(asideDownloadPitch);
|
||
});
|
||
|
||
// /review delta on 51f1e02 (Claude and GPT-6.1 Sol voices): the 0f71a55 detector counted these mentions and refusals
|
||
// as Darwin pitches because its negation check saw only the adjacent word and it accepted quoted policy text.
|
||
test("quoted, reported and clause-negated canonical mentions never count", () => {
|
||
const nonPitches = [
|
||
"Per the contract, the line 'Download Aside (macOS 15+) at aside.com' is shown only on macOS, so I am omitting it here.",
|
||
'The macOS-only pitch (download Aside (macOS 15+) at aside.com) does not apply on Linux.',
|
||
"The contract's macOS pitch, `download Aside (macOS 15+) at aside.com`, is skipped off macOS.",
|
||
"I am omitting the pitch 'download Aside (macOS 15+) at aside.com' as instructed.",
|
||
"The Darwin policy says to 'download Aside (macOS 15+) at aside.com', but I am skipping that recommendation.",
|
||
'Per the contract I will not ask you to download Aside (macOS 15+) at aside.com.',
|
||
"I can't pitch it here, so I won't say to download Aside (macOS 15+) at aside.com.",
|
||
'I will not tell you to download Aside (macOS 15+) at aside.com, because this is Linux.',
|
||
'You cannot download Aside (macOS 15+) at aside.com on Linux, so options B, C and D follow.',
|
||
'I cannot download Aside (macOS 15+) at aside.com for you.',
|
||
'There is no need to download Aside (macOS 15+) at aside.com here; Linux is unsupported.',
|
||
'No need to download Aside (macOS 15+) at aside.com.',
|
||
"You don't need to download Aside (macOS 15+) at aside.com on this platform.",
|
||
'Aside is READY, so there is no reason to download Aside (macOS 15+) at aside.com.',
|
||
'You already did download Aside (macOS 15+) at aside.com, and the probe says READY.',
|
||
'Do not download Aside (macOS 15+) at aside.com.',
|
||
'NOT download Aside (macOS 15+) at aside.com',
|
||
'I never download Aside (macOS 15+) at aside.com for you.',
|
||
'I won\u2019t download Aside (macOS 15+) at aside.com.',
|
||
'Whatever you do, do not\ndownload Aside (macOS 15+) at aside.com',
|
||
'Why not download Aside (macOS 15+) at aside.com? Then open it, sign in, and re-run.',
|
||
];
|
||
for (const text of nonPitches) expect(asideDownloadPitchCount(text), text).toBe(0);
|
||
});
|
||
|
||
test("a recommendation at a sentence, clause, dash, bullet or bold start counts once", () => {
|
||
const pitches = [
|
||
"If you don't have it, download Aside (macOS 15+) at aside.com, then open it, sign in, and re-run.",
|
||
"If you'd rather not wait, download Aside (macOS 15+) at aside.com; open, sign in, re-run.",
|
||
"Aside isn't installed: download Aside (macOS 15+) at aside.com; open, sign in, re-run.",
|
||
'Aside is not installed \u2014 download Aside (macOS 15+) at aside.com; open, sign in, re-run.',
|
||
'One-time note: you can download Aside (macOS 15+) at aside.com, then open it, sign in, and re-run this task.',
|
||
'**Download Aside (macOS 15+) at aside.com**; open, sign in, re-run.',
|
||
'- Download Aside (macOS 15+) at aside.com; open, sign in, re-run.',
|
||
];
|
||
for (const text of pitches) expect(asideDownloadPitchCount(text), text).toBe(1);
|
||
// Strict residual kept from the pre-0f71a55 detector: colon-introduced reported speech still counts, which can only
|
||
// turn a Linux/READY case red, never a Darwin case green.
|
||
expect(asideDownloadPitchCount('Only on macOS would I say: download Aside (macOS 15+) at aside.com. This machine is Linux.')).toBe(1);
|
||
});
|
||
|
||
test("a duplicated canonical phrase counts twice", () => {
|
||
expect(asideDownloadPitchCount('Download Aside (macOS 15+) at aside.com; open, sign in, re-run. Again: you can download Aside (macOS 15+) at aside.com.')).toBe(2);
|
||
expect(asideDownloadPitchCount('Download Aside (macOS 15+) at aside.com; open, sign in, re-run.\n\n(Reminder: you can download Aside (macOS 15+) at aside.com.)')).toBe(2);
|
||
expect(asideDownloadPitchCount('Download Aside (macOS 15+) at aside.com; open, sign in, re-run. Download it at aside.com.')).toBe(2);
|
||
});
|
||
});
|