mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-31 22:14:25 +01:00
82530ac143
* fix(pregate): recognise bats test files so lanes does not fail in bats repos (#3161) gstack-pregate never treated a .bats file as a test, so in a repository whose tests are bats files the lanes check failed every run with PREGATE_NO_LANE: a .gstack/pregate.json dependency pointing at tests/foo.bats selected nothing, a selection manifest printing .bats paths yielded 0 files, and a bats run that executed no tests could not trip the zero-run rule. Recognise .bats in the test-file pattern and in the manifest-output parser, and read the bats TAP plan and pretty summary (as a last resort, after every existing runner's summary) so a run that reports zero tests is a failure. Fixes #3161. Cherry-picked from #3177 (d355c116b). Conflicts resolved as in the author's own merges (b09476beb,444f4e768): lane-selection.ts keeps main's isTestFile-based manifest parser, and lanes.ts carries the bats summary on main's supervised runner. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(pregate): a nested .bats file is selected and run end to end (#3161) Reviewer test for #3177: touched itself with no declaration, and through a declared dependency; a failing nested bats run reports the bats counts. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(pregate): a wtree failure is WTREE_UNAVAILABLE, not an unresolvable base (#3197) The pregate consumers of the working-tree fingerprint (buildContext, withFingerprint, verifyPregate) move to lib/script-invocation.ts's {tree}|{error} form, and lib/regen.ts drops its string-returning wrapper. - A fingerprint failure was reported as "policy source unavailable: origin/<base> is not a resolvable commit" with git fetch as the fix. It is now WTREE_UNAVAILABLE: a launch failure lists each Windows bash candidate with its rejection reason and names GSTACK_GIT_BASH; a gstack-wtree exit keeps the repair it printed (set TMPDIR to a writable directory). - A fingerprint failing after a check is incomplete with that cause, not "working tree changed"; verify says the tree cannot be fingerprinted. - #3190's gstack-config read (test_backend) goes through runScriptSync, so Windows reaches it through Git Bash. The tripwire learns the bin-constant shape that let it through. - Under --json every context refusal is one envelope on stdout ({ok:false,status:'refused',code,detail,fix,docs}, exit 3); the run object carries ok and status too, and GSTACK_RESULT goes to stderr. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ci(pregate): gstack-pregate joins the Windows pregate job; pregate tests return to the Windows lane (#3197) - windows-pregate-e2e runs gstack-pregate run (preflight: regen, secrets through the gstack-redact bun bin, literals, guards) in both journeys, and asserts a missing GSTACK_GIT_BASH refuses with one WTREE_UNAVAILABLE --json envelope. - test/helpers/pregate-fixture.ts and restamp-fixture.ts runBin launch bins through lib/script-invocation.ts (bun bins as Bun argv, bash bins through Git Bash on Windows), which removes the "shebang via CreateProcess" reason for the pregate tests; they move from KNOWN_WINDOWS_INCOMPATIBLE to KNOWN_WINDOWS_SAFE. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * doctor: list rescue refs and warn about CLAUDE.md skill lists gstack-doctor's plain report gains two rows. 'rescue refs' lists each refs/gstack/rescue/* commit in the install checkout, newest first, with its date, a needs-manual-resolution label when the rescue held unmerged files, and the shell-quoted inspect and delete commands bin/gstack-ff-update's banner prints. 'CLAUDE.md' warns when a gstack section in ~/.claude/CLAUDE.md or the project's CLAUDE.md enumerates skills (#3169) and prints the short block to paste instead; docs/troubleshooting.md#claude-md-skill-list explains it. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * doctor: name the Windows bash gstack resolves and what chose it On Windows (OSTYPE msys/cygwin) gstack-doctor adds a 'windows bash' row from lib/script-invocation.ts resolveWindowsBash, run through the new thin bin/gstack-windows-bash.ts with the bun on PATH: the resolved bash and the override or discovery step that chose it, a warn when GSTACK_GIT_BASH and GSTACK_BASH differ, and a fail carrying the BASH_UNAVAILABLE line (every candidate with its rejection reason) when none qualifies. Tests inject the platform (GSTACK_DOCTOR_PLATFORM) and the discovery inputs (GSTACK_DOCTOR_BASH_PROBE) so the row runs on every OS. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * doctor --check: name the held-install recovery on Windows, not bin/gstack-upgrade bin/gstack-upgrade compares a pwd -P path (/c/...) with git's toplevel (C:/...) and refuses on every Git Bash install, so on Windows the revision row's upgrade fix (REVISION_BELOW_MINIMUM, the Capy-wrapper variant and REVISION_UNMET) names the held-install recovery forms from docs/troubleshooting.md#auto-update-held-install-recovery instead: the Git Bash form and the Windows PowerShell 5.1 form, each labelled by shell with the install path quoted for that shell. Other platforms keep bin/gstack-upgrade. A test pins both printed forms to the documented ones. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(skills): omit the plan-mode section on hosts without plan mode (#3168) generatePlanModeInfo now honours capabilities.planMode. Hosts that declare planMode:false (codex, agy, hermes, openclaw, and define-host's default: factory, kiro, slate, cursor, gbrain) no longer render the '## Plan Mode Safe Operations' and '## Skill Invocation During Plan Mode' preamble sections. Claude, copilot and opencode are unchanged. The codex, factory and agy ship goldens are regenerated. The clarity test that pinned the plan-mode entry-order prose on every host now pins it on planMode hosts and asserts its absence elsewhere (intentional contract change). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(skills): guard generated SKILL.md size at 100,000 chars on every host (#3199) A free test renders every host and fails when a generated SKILL.md exceeds 100,000 characters, naming host, skill, actual size, limit and the fix. The 89 current offenders are pinned at their exact size in test/fixtures/skill-md-size-allowlist.json: growth past a pin fails, and a shrink fails until the pin is lowered. The shared-preamble hoist is deferred (owner Garry, revisit 2026-11-15). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(pregate): keep #3177's bats tests beside the nested-directory test Moves #3177's bats describe block verbatim from test/pregate-selection.test.ts into test/pregate-bats-nested.test.ts, so pregate-selection.test.ts is unchanged on this branch and the in-flight field-list-2 pregate lane (capy/fl2-L6-pregate, which edits that file's imports and adds cases) merges without a conflict. No assertion changes. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * doctor --check: keep bin/gstack-upgrade as the Windows fix (path spelling fixed on main) Reverts37bac07f8. Plan item D named the held-install recovery forms on Windows only until bin/gstack-upgrade stopped refusing Git Bash installs. bin/gstack-upgrade:63-67 now compares with gstack_path_same from bin/gstack-path-spelling.sh, pinned by test/gstack-upgrade-path-spelling.test.ts, so bin/gstack-upgrade is the right fix on every platform again. * test(skills): measure the size-guard pins on v1.91.109.0 #3225 (merged into this branch) grew 53 pinned skills: 3 characters each from the Aside driver resolver, and about 340 more in plan-design-review and plan-devex-review from the #3219 summary sections. The pins are this PR's starting measurement, so they are taken on the merged base; no skill this PR renders grew. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ci(windows): only the pregate tests that pass on windows-latest join the lane windows-free-tests run 38984824731 (this branch at8ed6da402) ran the moved files. gstack-regen and the new pregate-wtree-refusal passed. pregate-guards, -remote, -regate, -invalidation, -selection and -bats-nested launched gstack-pregate fine but fail on Linux-host expectations: their fixture lanes are Linux workflow jobs, which a Windows host records as requires-remote obligations. They go back to KNOWN_WINDOWS_INCOMPATIBLE with that reason instead of "shebang via CreateProcess". Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * windows curation: keep CI run ids out of reason prose (pii.phone false positive) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * skill-md size pins: follow #3231's review-army line (+21 chars on five inlining hosts) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * doctor test: drop the reverted Windows upgrade-fix wording from item D's describe Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * skill-md size pins: follow #3226 (plan-devex-review +3324) and #3234 (autoplan +59) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * WTREE_UNAVAILABLE moves to lib/result-codes-release.ts (#3235 split; result-codes.ts stays at 796 of 800 lines); doctor fixture copies the release table; size pins follow #3235's review +1 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): fix wave PR 5 release notes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * v1.91.117.0 fix: fix wave PR 5 (pregate bats + WTREE_UNAVAILABLE, doctor rows, SKILL.md size guard, planMode hosts) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(qa): an invalid-input probe breaks only the rule it tests The paid gate's qa-functional-cli-report failed again on #3236 (run 39001107574): the agent probed the digits-only amount rule with -5, which the positivity check rejected first, so the planted parseInt defect stayed hidden and it reported no defect. The contract map's invalid-input row now asks for each input to break only the rule it tests, so another check cannot reject it first. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: harjothkhara <48686985+harjothkhara@users.noreply.github.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
3.9 KiB
3.9 KiB
Functional QA with repository-native tools
Use documented repository commands, CLI/API clients and job/queue tools, not a new harness or browser substitution.
Functional modes
For /qa and /qa-only, within the selected scope:
- Full (default): cover every applicable documented contract below.
- Quick (
--quick): check success and the highest-risk changed edge; mark other contracts not run. - Regression (
--regression <previous-report>): before probes, read the supplied functional report and linked replay evidence. A missing, unreadable or wrong-target baseline blocks regression mode. A browser-onlybaseline.jsonis not a functional baseline. Re-establish owned setup; replay prior failed probes against the documented expectation, never recorded buggy output, then check changed adjacent behavior. Preserve the prior report; report fixed, still failing and new findings separately. Missing safe replay inputs block affected probes, never count as passes.
Mixed runs apply each surface's mode separately. /review and /ship retain their caller's bounded smoke and explicit plan checks, not Full exploration.
Contract map
Record each contract/source, isolated setup, exact probe, expectation and outcome: pass/fail/blocked/not run/inconclusive/not applicable (reason).
| Contract | Observe |
|---|---|
| Successful execution | Expected return/output and final business effect, not just launch/acceptance |
| Invalid/missing input | One probe per documented input rule (allowed characters, format, sign, range, arity), each input breaking only that rule so no other check rejects it first: declared rejection, correct status and no forbidden state change |
| Authentication/authorization | Valid identity, missing/invalid identity, wrong owner/role and durable no-effect boundary |
| CLI process contract | Exact exit code, stdout and stderr separately; resulting file/state changes |
| State transitions | Initial, intermediate and completed/failed states and their permitted transitions |
| Timeout/cancellation | Deadline, partial state, termination of owned work and recovery |
| Retry | Attempts/backoff/terminal state promised by the repository; no unbounded retry |
| Duplicates/idempotency | Repeated request/event and number of durable effects under the documented guarantee |
| Concurrency/order | Controlled competing operations in both relevant completion orders; final invariant |
| Partial-failure recovery | Interrupt after an effect, restart/replay, inspect completion/dead-letter state and duplicates |
Do not impose universal exactly-once delivery. Separate acceptance, enqueue, processing, retry/dead-letter and final effect; 2xx is not completion. Expected rejection/injected failure may pass; a missing service preventing execution blocks coverage.
Execute and retain evidence
- Apply the shared isolation/permission preflight. Verify cwd, command, environment NAMES and safe reset; use synthetic data/credentials.
- Follow the shared exploratory loop's order and written checkpoints. For every probe, inspect initial/final durable state and retain exit/status and stdout/stderr separately without masking failure.
- On timeout, retain partial output/state and stop only owned work. Record setup errors and untested contracts; never patch product code to hide missing prerequisites.
- Record exact command or method/path/headers/body, setup/reset, expected contract/source, observed output/state, revision/runtime, evidence paths and limits. Secrets are referenced only by environment name. Disclose replay limits caused by redaction.
- Use
templates/functional-report-template.mdrelative to the installed QA SKILL.md. Preserve evidence before owned cleanup and disclose leftovers. Return to the caller without expanding discovery authority.