From d41ae9c20dc9cd12be15357d758216bd0abaa83e Mon Sep 17 00:00:00 2001 From: ezl-keygraph Date: Tue, 18 Aug 2026 15:46:25 +0530 Subject: [PATCH] feat(cli): overhaul commands and add live scan status (#424) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * refactor(cli): list workspaces natively instead of via the worker image * feat(cli): preflight that Docker is installed and running * feat(cli): stop scans by workspace or --all, terminating their Temporal workflows * fix(worker): abort the running agent on cancellation so Temporal cancel takes effect * refactor(cli): split destructive teardown out of stop into a reset command * refactor(cli): centralise flag parsing and confirmation across commands * fix(cli): pass provider credentials to docker by name to keep secrets out of argv * feat(cli): add per-command help via --help/-h and help * feat(cli): replace raw docker output with clack spinners for infra and scan teardown * fix(cli): verify scan stop by re-querying container and workflow state instead of assuming success * fix(cli): resolve running state before prompting on stop and report no-op stops honestly * refactor(cli): show splash first and drive start with one spinner resolving to a clean line * fix(cli): validate --url up front so a bad value fails cleanly instead of a late crash * refactor(cli): centralize error reporting with fail() for expected errors and a crash handler that logs the stack and links the issue tracker * feat(cli): add --json/--plain machine-readable output to workspaces and status * refactor(cli): remove the workspaces command * refactor(cli): remove the status command * feat(cli): add 'progress ' — live scan progress from Temporal * fix(cli): mark metric-less agents as skipped in progress, not done * feat(cli): animate running agents in progress with a clack-style spinner * feat(cli): rename progress->status, reveal agents as they run, show live per-agent elapsed * fix(cli): mark passed-over phases as skipped live, not pending * style(cli): rename status footer 'Wall-clock' to 'Time Taken', drop the parenthetical * style(cli): drop '(sum of agents)' from status total cost line * style(cli): green filled circle for completed, Shannon gold for running * style(cli): use Shannon gold in place of green in status * feat(cli): suggest closest command or flag on typo * refactor(cli): single-source start help and drop ./repos bare-name shortcut * feat(cli): name providers and fix in multi-provider credential error * feat(cli): support --flag=value syntax and expand leading ~ in paths * refactor(cli): centralize ANSI color codes in colors.ts * feat(cli): add scans command listing completed scans with cost and duration * fix(cli): keep stdout clean off-TTY for logs and start * feat(cli): add repo link to top-level help * feat(worker): record auth-validation metrics and register resume attempts early * refactor(cli): share resume-aware workflow-id resolution and surface root-cause failures * feat(cli): add status --json, auth phase, dashboard link, and stable live redraw * refactor(cli): drop cost from status and scans output * feat(worker): surface both PDF and markdown report at run root * refactor(cli): normalize error/warning prefixing through fail and warn * feat(cli): add version --json for machine-readable output * refactor(cli): rename start --debug to --keep-container * refactor(cli): point start's progress hint at status instead of the Temporal dashboard * refactor(cli): centralize the mode-aware command prefix * refactor(cli): trim start and logs output to durable facts off-TTY * feat(cli): require typed confirmation for reset instead of --yes reset permanently wipes all Temporal data and volumes — a severe, irreversible action. Replace its default y/N confirm (bypassable with --yes) with a typed-word confirmation that has no bypass, so the wipe can only be triggered by a deliberate interactive answer. * feat(cli): surface logs and status hints after start on a TTY * feat(cli): exit 2 on usage errors, distinct from operational failures * feat(cli): add start --follow to stream logs and exit on scan outcome * refactor(cli): redesign splash with sunset-gradient wordmark and truecolor * refactor(cli): remove the uninstall command * docs: sync CLI docs with removed uninstall/workspaces, new scans and --follow * docs: fix reset confirmation — typed confirm, not --yes/-y * style(cli): restructure status footer with divider, aligned Logs/Temporal rows * feat(cli): show splash in the status command * fix(worker): validate auth-state shape, not entry count * docs: correct reset confirmation and add markdown report to run-root docs --- CLAUDE.md | 35 +- apps/cli/package.json | 1 + apps/cli/src/args.ts | 106 ++++++ apps/cli/src/colors.ts | 34 ++ apps/cli/src/commands/build.ts | 12 +- apps/cli/src/commands/logs.ts | 118 ++++--- apps/cli/src/commands/reset.ts | 26 ++ apps/cli/src/commands/scans.ts | 197 +++++++++++ apps/cli/src/commands/start.ts | 209 ++++++----- apps/cli/src/commands/status.ts | 204 ++++++++++- apps/cli/src/commands/stop.ts | 132 ++++++- apps/cli/src/commands/uninstall.ts | 55 --- apps/cli/src/commands/workspaces.ts | 35 -- apps/cli/src/config/resolver.ts | 21 +- apps/cli/src/confirm.ts | 43 +++ apps/cli/src/docker.ts | 186 +++++++--- apps/cli/src/env.ts | 34 +- apps/cli/src/errors.ts | 70 ++++ apps/cli/src/help.ts | 145 ++++++++ apps/cli/src/index.ts | 332 ++++++++++-------- apps/cli/src/mode.ts | 5 + apps/cli/src/paths.ts | 56 ++- apps/cli/src/scan/derive.ts | 155 ++++++++ apps/cli/src/scan/pipeline.ts | 123 +++++++ apps/cli/src/scan/render.ts | 248 +++++++++++++ apps/cli/src/scan/status-json.ts | 68 ++++ apps/cli/src/session.ts | 26 ++ apps/cli/src/splash.ts | 95 +++-- apps/cli/src/suggest.ts | 58 +++ apps/cli/src/temporal-client.ts | 126 +++++++ apps/cli/src/tty.ts | 6 +- apps/cli/src/ui.ts | 60 ++++ apps/worker/src/ai/pi/pi-executor.ts | 17 + apps/worker/src/paths.ts | 3 + apps/worker/src/services/reporting.ts | 14 +- .../src/services/validate-authentication.ts | 48 ++- apps/worker/src/temporal/activities.ts | 25 +- apps/worker/src/temporal/workflows.ts | 19 +- apps/worker/src/temporal/workspaces.ts | 174 --------- docs/development.md | 32 +- docs/workspaces.md | 8 +- llms-full.txt | 40 ++- llms.txt | 2 +- pnpm-lock.yaml | 3 + 44 files changed, 2605 insertions(+), 801 deletions(-) create mode 100644 apps/cli/src/args.ts create mode 100644 apps/cli/src/colors.ts create mode 100644 apps/cli/src/commands/reset.ts create mode 100644 apps/cli/src/commands/scans.ts delete mode 100644 apps/cli/src/commands/uninstall.ts delete mode 100644 apps/cli/src/commands/workspaces.ts create mode 100644 apps/cli/src/confirm.ts create mode 100644 apps/cli/src/errors.ts create mode 100644 apps/cli/src/help.ts create mode 100644 apps/cli/src/scan/derive.ts create mode 100644 apps/cli/src/scan/pipeline.ts create mode 100644 apps/cli/src/scan/render.ts create mode 100644 apps/cli/src/scan/status-json.ts create mode 100644 apps/cli/src/session.ts create mode 100644 apps/cli/src/suggest.ts create mode 100644 apps/cli/src/temporal-client.ts create mode 100644 apps/cli/src/ui.ts delete mode 100644 apps/worker/src/temporal/workspaces.ts diff --git a/CLAUDE.md b/CLAUDE.md index 0b53db7..f20d6f0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -44,8 +44,8 @@ echo "ANTHROPIC_API_KEY=your-key" > .env ./shannon build # Run -./shannon start -u -r my-repo -./shannon start -u -r my-repo -c ./apps/worker/configs/my-config.yaml +./shannon start -u -r ./my-repo +./shannon start -u -r ./my-repo -c ./apps/worker/configs/my-config.yaml ./shannon start -u -r /any/path/to/repo ``` @@ -56,25 +56,24 @@ echo "ANTHROPIC_API_KEY=your-key" > .env npx @keygraph/shannon setup # Workspaces & Resume -./shannon start -u -r my-repo -w my-audit # New named workspace -./shannon start -u -r my-repo -w my-audit # Resume (same command) -./shannon workspaces # List all workspaces +./shannon start -u -r ./my-repo -w my-audit # New named workspace +./shannon start -u -r ./my-repo -w my-audit # Resume (same command) # Monitor ./shannon logs # Show a scan's live log -./shannon status # Show running scans +./shannon status # Live phase/agent progress of one scan, read from Temporal (redraws, then exits) # Dashboard: http://localhost:8233 # Stop -./shannon stop # Preserves scan data -./shannon stop --clean # Full cleanup including volumes (confirms first; --yes/-y to skip) +./shannon stop # Stop one scan (confirms first; --yes/-y to skip) +./shannon stop --all # Stop all running scans (Temporal stays up; confirms first) +./shannon reset # Stop everything and wipe all Temporal data + volumes (type 'confirm' to proceed; cannot be skipped) # Version ./shannon version # npx: package version; local: git SHA # Image management ./shannon build [--no-cache] # Local mode: build worker image -npx @keygraph/shannon uninstall # npx mode: remove ~/.shannon/ (confirms first; --yes/-y to skip) # Build TypeScript (development) pnpm run build # Build all packages via Turborepo @@ -85,7 +84,7 @@ pnpm biome:fix # Auto-fix lint, format, and import sorting **Monorepo tooling:** pnpm workspaces, Turborepo for task orchestration, Biome for linting/formatting. TypeScript compiler options shared via `tsconfig.base.json` at the root. All packages extend it, overriding only `rootDir` and `outDir`. Shared devDependencies (`typescript`, `@types/node`, `turbo`, `@biomejs/biome`) are hoisted to the root workspace. -**Options:** `-c ` (YAML config), `-o ` (output directory), `-w ` (named workspace; auto-resumes if exists), `--pipeline-testing` (minimal prompts, 10s retries), `--debug` (preserve worker container after exit for log inspection), `--yes`/`-y` (skip the confirmation prompt on `stop --clean`/`uninstall`; required for non-interactive use) +**Options:** `-c ` (YAML config), `-o ` (output directory), `-w ` (named workspace; auto-resumes if exists), `--pipeline-testing` (minimal prompts, 10s retries), `--keep-container` (preserve worker container after exit for log inspection), `--yes`/`-y` (skip the confirmation prompt on `stop`; required for non-interactive use; `reset` requires a typed `confirm` and cannot be skipped) ## Architecture @@ -97,9 +96,11 @@ apps/worker/ — @shannon/worker (private, Temporal worker + pipeline logic) ``` ### CLI Package (`apps/cli/`) -Published as `@keygraph/shannon` on npm. Contains only Docker orchestration logic — no Temporal SDK, business logic, or prompts. Bundled with tsdown for single-file ESM output. +Published as `@keygraph/shannon` on npm. Contains Docker orchestration logic plus a read-only `@temporalio/client` reader (for `status`); no worker/pipeline business logic or prompts. Bundled with tsdown for single-file ESM output (deps stay external). -- `apps/cli/src/index.ts` — CLI dispatcher (`setup`, `start`, `stop`, `logs`, `workspaces`, `status`, `build`, `uninstall`, `version`) +- `apps/cli/src/index.ts` — CLI dispatcher (`setup`, `start`, `stop`, `reset`, `logs`, `status`, `build`, `version`) +- `apps/cli/src/temporal-client.ts` — `@temporalio/client` reader for `status`: connects to the frontend on `127.0.0.1:7233` (published by compose), `describeScan` (status + `pendingActivities` → running agents), `queryProgress` (live `getProgress` query → `PipelineState`), `getTerminalOutcome` (workflow `result()`). No worker of its own; scans are visible only within Temporal's ~24h retention (namespace default, unset in compose) +- `apps/cli/src/scan/` — `status` rendering: `pipeline.ts` (static phase/agent plan + `run*Agent` activity-type→agent map + mirrored `PipelineState`/`AgentMetrics` types; keep in sync with the worker), `render.ts` (one renderer for both the live query state and the terminal result) - `apps/cli/src/mode.ts` — Auto-detection: local mode if `SHANNON_LOCAL=1` env var is set - `apps/cli/src/docker.ts` — Compose lifecycle, image pull/build, ephemeral `docker run` worker spawning - `apps/cli/src/home.ts` — State directory management (`~/.shannon/` for npx, `./` for local) @@ -108,7 +109,7 @@ Published as `@keygraph/shannon` on npm. Contains only Docker orchestration logi - `apps/cli/src/config/resolver.ts` — Cascading config (npx only): env vars → `~/.shannon/config.toml` (parsed with `smol-toml`) - `apps/cli/src/config/writer.ts` — TOML serialization and secure file persistence (0o600) - `apps/cli/src/commands/setup.ts` — Interactive TUI wizard (`@clack/prompts`) for provider credential setup (npx only) -- `apps/cli/src/paths.ts` — Repo/config path resolution (bare name → `./repos/`, or any absolute/relative path) +- `apps/cli/src/paths.ts` — Repo/config path resolution (any absolute or relative path) - `apps/cli/src/version.ts` — Version reporting (npx: `package.json` version; local: `git-`) - `apps/cli/src/tty.ts` — Terminal capability detection: `requireInteractive` guard (fails fast off-TTY instead of hanging on a prompt), `supportsColor` color gating (`NO_COLOR`/`FORCE_COLOR`), and `stdoutIsTerminal` for spinner/cursor output - `apps/cli/src/commands/` — Command handlers @@ -155,9 +156,9 @@ Durable workflow orchestration with crash recovery, queryable progress, intellig - **Prompts** — Per-phase templates in `apps/worker/prompts/` with variable substitution (`{{TARGET_URL}}`, `{{CONFIG_CONTEXT}}`). Shared partials in `apps/worker/prompts/shared/` via `apps/worker/src/services/prompt-manager.ts`, including `_code-path-rules.txt` (focus/avoid `[FILE]`/`[GLOB]` routing) and `_rules-of-engagement.txt` (free-text engagement rules). When `exploit: false`, `apps/worker/src/services/findings-renderer.ts` deterministically converts each `*_exploitation_queue.json` into a `*_findings.md` for report assembly — no LLM in the loop - **Agent Harness (pi)** — Uses the **pi harness** (`@earendil-works/pi-coding-agent`, requires Node ≥ 22.19) via `apps/worker/src/ai/pi/pi-executor.ts` (`runPiPrompt` → `createAgentSession`). Retry is split in `apps/worker/src/ai/pi/retry-settings.ts`: pi's agent-level loop is off so Temporal owns agent restarts, while `provider.maxRetries` stays on — pi reads the `provider` block independently of the `enabled` flag — so transport faults are absorbed in-session rather than costing a full agent re-run. `maxRetryDelayMs` is left at pi's 60s default. One model runs every phase, named by `SHANNON_AI_MODEL=:` (default `anthropic:claude-sonnet-4-6`). `apps/worker/src/ai/models.ts` parses the spec — splitting on the **first** colon only, so Bedrock IDs keep theirs — and resolves it through pi's `ModelRuntime`. pi ships the `CredentialStore` interface but no in-memory implementation (its own reads `auth.json` from disk), so `RuntimeCredentialStore` in that file supplies one: credentials arrive as env vars in an ephemeral container and must never touch disk. `createModelRuntime(providerId, apiKey)` builds the runtime; `allowModelNetwork` stays at its default `false` so a scan never blocks on a catalog refresh. `resolveModelSelection()` is **async** because `ModelRuntime.create()` is. Any pi-ai provider id is accepted — `parseModelSpec` no longer rejects against a hardcoded list, so pi's registry is the authority (an unknown provider/model surfaces as a clear "not found in pi registry" error at preflight, which points to the browsable catalogue at `pi.dev/models` — `PI_CATALOG_URL` in `apps/worker/src/ai/models.ts`, appended to the not-found errors and shown in the setup wizard's "Other provider" hint). Four providers are **curated** (`CURATED_PROVIDERS`: `anthropic`, `openai`, `xai`, `amazon-bedrock`) with their own credential variables, config sections, and setup flows; each provider's API key env var is declared once in `PROVIDER_API_KEY_ENV` — Shannon uses each vendor's own variable name (`OPENAI_API_KEY`, `XAI_API_KEY`, …), never an invented one; Bedrock's entry is `AWS_BEARER_TOKEN_BEDROCK`, paired with `AWS_REGION`, which preflight requires separately as provider config rather than a credential. Any other provider uses the **generic** credential path: `SHANNON_AI_API_KEY` (`GENERIC_API_KEY_ENV`) supplies the key for any provider whose credential is a plain API key. Curated providers' own variables take precedence over it, and it also works as a fallback for them — Bedrock is the sole exception (it authenticates through its AWS_ variables, so the generic key never stands in for it). The CLI forwards `SHANNON_AI_API_KEY` in `COMMON_FORWARD_VARS` (it is provider-neutral, binding to whatever `SHANNON_AI_MODEL` names, so the "only one provider configured" guard counts only named credentials), and stores it under a generic `[provider]` config.toml section (`provider.api_key`). `npx @keygraph/shannon setup` exposes this as the "Other provider" option: free-text provider id + model id + key (a curated provider id is rejected there, since it has its own option). `SHANNON_AI_BASE_URL` overrides the endpoint for any provider (proxies/gateways); the credential is unchanged. `pointAtGateway` (`apps/worker/src/ai/models.ts`) applies the one dialect change: behind a base URL, `openai` follows `SHANNON_AI_OPENAI_FORMAT` (`chat-completions` default, or `responses`). On `chat-completions` it switches the API to `openai-completions` and drops the catalogue's Responses-shaped `compat` block so pi's `detectCompat` derives completions settings; on `responses` the descriptor is unchanged but for the endpoint. `resolveGatewayFormat` rejects the variable when the provider is not `openai` or no base URL is set, since it cannot take effect there. All other providers keep their API. The CLI mirrors the accepted values in `apps/cli/src/model-spec.ts`, forwards the variable in `COMMON_FORWARD_VARS`, and maps it to `openai.format` in config.toml. `buildEnvFlags` forwards only the selected provider's credential into the worker container. The CLI mirrors the parse rule and the provider/credential tables in `apps/cli/src/model-spec.ts` (it cannot import from the worker package); the two must stay in sync. pi ships no JSON-schema output or `Task`/`TodoWrite` built-ins, so structured queues are captured via a `submit_exploitation_queue` custom tool (`apps/worker/src/ai/queue-schemas.ts`), and `task` (child sessions scoped to `read`, `grep`, `find`, `ls`, `write`, and `bash` — no nested `task` or collector tools; `CHILD_TOOLS` in `apps/worker/src/ai/pi/task-tool.ts`) + `todo_write` (`apps/worker/src/ai/pi/session-tools.ts`) are provided as custom tools; the per-phase collectors are pi custom tools (TypeBox `defineTool` in `apps/worker/src/collectors/`). Shannon sets no thinking configuration at all — no `thinkingLevel` is passed to any `createAgentSession` call, so pi's own default applies. There is no adaptive-thinking support and no `CLAUDE_ADAPTIVE_THINKING` / `core.adaptive_thinking` setting. Browser automation via `playwright-cli` with session isolation (`-s=`). TOTP generation via `generate-totp` CLI tool. Login flow template at `apps/worker/prompts/shared/login-instructions.txt` supports form, SSO, API, and basic auth. On authenticated whitebox scans, the `validate-authentication` preflight performs the single real login and saves the browser session to `auth-state.json` in the per-session audit directory (path from `authStateFile()` in `apps/worker/src/audit/utils.ts`, derived from `generateAuditPath()`). The validation activity (`apps/worker/src/services/validate-authentication.ts`) removes any stale file from a prior run before the agent runs and verifies the file parses and contains cookies or storage before the preflight is marked complete; `logWorkflowComplete` deletes it when the workflow ends so authenticated cookies don't sit on disk between scans. Agent prompts opt in to session reuse by `@include(shared/_shared-session.txt)` before their `` block — the partial restores the session and falls through to the full login flow if verification fails. `vuln-auth`/`exploit-auth` omit the include and own their own login - **Pi Credential Reuse** — `SHANNON_USE_PI_AUTH=1` opts into reusing the host's Pi login, including an `openai-codex` ChatGPT Plus/Pro subscription selected with `SHANNON_AI_MODEL=openai-codex:`. `apps/cli/src/env.ts` requires `~/.pi/agent/auth.json`; `start.ts` passes its path to `spawnWorker`, which mounts only that file read-write at `/tmp/.pi/agent/auth.json`. The flag itself is not forwarded: the worker detects the file with `piAuthPresent()` and passes its path to `ModelRuntime.create`. CLI and worker API-key presence checks are skipped on this path, but the normal preflight model probe still validates the credential. The image and UID-remapping entrypoint keep `/tmp/.pi/agent` owned by `pentest` so adjacent Pi/Shannon configuration remains writable. Refreshed OAuth state is persisted to the host for subsequent scans. -- **Audit System** — Crash-safe append-only logging in `workspaces/{hostname}_{sessionId}/`. The run directory's top level holds only the human-facing PDF report (`Security-Assessment-Report.pdf`, `FINAL_REPORT_PDF_FILENAME` in `apps/worker/src/paths.ts`); everything else — deliverables, per-agent logs, prompts, `session.json`, `workflow.log`, and browser artifacts — is nested under a hidden `.shannon/` internals dir (`INTERNAL_DIR`) so a customer sees only the report. Audit path helpers route through `generateInternalPath` (`apps/worker/src/audit/utils.ts`); the CLI nests the overlay backing dirs under the same `.shannon/` (`apps/cli/src/docker.ts`, `start.ts`). `session.json`/`workflow.log` reads use dual-read resolvers (`resolveSessionJsonPath`, `resolveRunFile`) that prefer `.shannon/` and fall back to the legacy run-root layout, so pre-restructure workspaces stay listable (`workspaces`/`logs`) without migration. Resuming a pre-restructure workspace upgrades it in place first: `migrateLegacyWorkspaceLayout` (`apps/cli/src/commands/start.ts`) renames the flat deliverables/logs/session entries into `.shannon/` (carrying the deliverables `.git` along) before the overlay dirs are mounted, so resume finds the old checkpoints instead of re-running every agent. The report agent writes structured findings to `report.json`, from which `report-renderer.ts` renders the assembled markdown and `report-json-adapter.ts` produces the Typst-shaped JSON that `pdf-renderer.ts` compiles into `comprehensive_security_assessment_report.pdf` using the bundled `apps/worker/templates/typst/report.typ` template (the `typst` binary is installed in the worker image). `copyReportToRunRoot` (`apps/worker/src/services/reporting.ts`) surfaces the PDF to the run root as `Security-Assessment-Report.pdf`; the markdown stays in the deliverables dir and is not surfaced. PDF compilation is best-effort — a failure is logged and the run still completes. WorkflowLogger (`apps/worker/src/audit/workflow-logger.ts`) provides unified human-readable per-workflow logs, backed by LogStream (`apps/worker/src/audit/log-stream.ts`) shared stream primitive +- **Audit System** — Crash-safe append-only logging in `workspaces/{hostname}_{sessionId}/`. The run directory's top level holds the human-facing report in both formats (`Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`, `FINAL_REPORT_PDF_FILENAME`/`FINAL_REPORT_MD_FILENAME` in `apps/worker/src/paths.ts`); everything else — deliverables, per-agent logs, prompts, `session.json`, `workflow.log`, and browser artifacts — is nested under a hidden `.shannon/` internals dir (`INTERNAL_DIR`) so a customer sees only the report. Audit path helpers route through `generateInternalPath` (`apps/worker/src/audit/utils.ts`); the CLI nests the overlay backing dirs under the same `.shannon/` (`apps/cli/src/docker.ts`, `start.ts`). `session.json`/`workflow.log` reads use dual-read resolvers (`resolveSessionJsonPath`, `resolveRunFile`) that prefer `.shannon/` and fall back to the legacy run-root layout, so pre-restructure workspaces stay listable (`workspaces`/`logs`) without migration. Resuming a pre-restructure workspace upgrades it in place first: `migrateLegacyWorkspaceLayout` (`apps/cli/src/commands/start.ts`) renames the flat deliverables/logs/session entries into `.shannon/` (carrying the deliverables `.git` along) before the overlay dirs are mounted, so resume finds the old checkpoints instead of re-running every agent. The report agent writes structured findings to `report.json`, from which `report-renderer.ts` renders the assembled markdown and `report-json-adapter.ts` produces the Typst-shaped JSON that `pdf-renderer.ts` compiles into `comprehensive_security_assessment_report.pdf` using the bundled `apps/worker/templates/typst/report.typ` template (the `typst` binary is installed in the worker image). `copyReportToRunRoot` (`apps/worker/src/services/reporting.ts`) surfaces both the PDF and the markdown to the run root as `Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`; the deliverables-dir copies remain as the git-checkpointed sources. PDF compilation is best-effort — a failure is logged and the run still completes. WorkflowLogger (`apps/worker/src/audit/workflow-logger.ts`) provides unified human-readable per-workflow logs, backed by LogStream (`apps/worker/src/audit/log-stream.ts`) shared stream primitive - **Deliverables** — Saved to `.shannon/deliverables/` in the target repo via the `save-deliverable` CLI script (`apps/worker/src/scripts/save-deliverable.ts`) -- **Workspaces & Resume** — Named workspaces via `-w ` or auto-named from URL+timestamp. Resume detects completed agents via `session.json`. `loadResumeState()` in `apps/worker/src/temporal/activities.ts` validates deliverable existence, restores git checkpoints, and cleans up incomplete deliverables. Workspace listing via `apps/worker/src/temporal/workspaces.ts` +- **Workspaces & Resume** — Named workspaces via `-w ` or auto-named from URL+timestamp. Resume detects completed agents via `session.json`. `loadResumeState()` in `apps/worker/src/temporal/activities.ts` validates deliverable existence, restores git checkpoints, and cleans up incomplete deliverables ## Development Notes @@ -247,9 +248,9 @@ Package managers are configured with a minimum release age (7 days). Requires pn ## Troubleshooting -- **"Repository not found"** — Pass a bare name (`-r my-repo`) for `./repos/my-repo`, or a path (`-r /path/to/repo`) for any directory +- **"Repository not found"** — Pass a path to the target repo (`-r /path/to/repo` or `-r ./my-repo`) - **"Temporal not ready"** — Wait for health check or `docker compose logs temporal` - **Worker not processing** — Check `docker ps --filter "name=shannon-worker-"` -- **Reset state** — `./shannon stop --clean` +- **Reset state** — `./shannon reset` - **Local apps unreachable** — Use `host.docker.internal` instead of `localhost` - **Container permissions** — On Linux, may need `sudo` for docker commands diff --git a/apps/cli/package.json b/apps/cli/package.json index 63ee6a7..0510dc6 100644 --- a/apps/cli/package.json +++ b/apps/cli/package.json @@ -18,6 +18,7 @@ }, "dependencies": { "@clack/prompts": "^1.1.0", + "@temporalio/client": "^1.11.0", "chokidar": "^5.0.0", "dotenv": "^17.3.1", "smol-toml": "^1.6.1" diff --git a/apps/cli/src/args.ts b/apps/cli/src/args.ts new file mode 100644 index 0000000..620e021 --- /dev/null +++ b/apps/cli/src/args.ts @@ -0,0 +1,106 @@ +/** + * Shared argument parsing for CLI commands. + * + * Every command declares which boolean flags, value options, and positionals it + * accepts; `parseArgs` resolves aliases, rejects anything unrecognized, and hands + * back a typed result. This centralizes the common flags (notably `--yes`/`-y`) so + * each command no longer re-hardcodes `args.includes('--yes')`, and it makes + * unknown flags and stray arguments fail loudly instead of being silently ignored. + */ + +import { closestMatch } from './suggest.js'; + +/** Thrown when argv does not match a command's schema. The dispatcher formats it. */ +export class ArgError extends Error {} + +/** Tokens that set the "skip confirmation" flag, declared once for every command. */ +export const YES_FLAGS = ['--yes', '-y'] as const; + +export interface ArgSchema { + /** Boolean flags: result key -> accepted tokens (canonical plus any aliases). */ + readonly booleans?: Record; + /** Value-taking options: result key -> accepted tokens. */ + readonly values?: Record; + /** Maximum positional arguments allowed. Defaults to 0. */ + readonly maxPositionals?: number; + /** Extra guidance appended to the error when too many positionals are given. */ + readonly positionalHint?: string; +} + +export interface ParsedArgs { + readonly flags: Record; + readonly values: Record; + readonly positionals: readonly string[]; +} + +/** Build a token -> result-key lookup from a schema section. */ +function indexTokens(section: Record): Map { + const byToken = new Map(); + for (const [key, tokens] of Object.entries(section)) { + for (const token of tokens) { + byToken.set(token, key); + } + } + return byToken; +} + +export function parseArgs(argv: readonly string[], schema: ArgSchema): ParsedArgs { + const booleanByToken = indexTokens(schema.booleans ?? {}); + const valueByToken = indexTokens(schema.values ?? {}); + const maxPositionals = schema.maxPositionals ?? 0; + + const flags: Record = {}; + const values: Record = {}; + const positionals: string[] = []; + + for (let i = 0; i < argv.length; i++) { + const arg = argv[i]; + if (arg === undefined) { + continue; + } + + const equalsIndex = arg.startsWith('--') ? arg.indexOf('=') : -1; + const token = equalsIndex === -1 ? arg : arg.slice(0, equalsIndex); + const inlineValue = equalsIndex === -1 ? undefined : arg.slice(equalsIndex + 1); + + const booleanKey = booleanByToken.get(token); + if (booleanKey !== undefined) { + if (inlineValue !== undefined) { + throw new ArgError(`Flag ${token} does not take a value`); + } + flags[booleanKey] = true; + continue; + } + + const valueKey = valueByToken.get(token); + if (valueKey !== undefined) { + if (inlineValue !== undefined) { + values[valueKey] = inlineValue; + continue; + } + const next = argv[i + 1]; + if (next === undefined || next.startsWith('-')) { + throw new ArgError(`Option ${token} requires a value`); + } + values[valueKey] = next; + i++; + continue; + } + + if (arg.startsWith('-')) { + const suggestion = closestMatch(token, [...booleanByToken.keys(), ...valueByToken.keys()]); + const hint = suggestion ? `\nDid you mean '${suggestion}'?` : ''; + throw new ArgError(`Unknown option: ${token}${hint}`); + } + + positionals.push(arg); + } + + if (positionals.length > maxPositionals) { + const extra = positionals[maxPositionals]; + const hint = schema.positionalHint ? `\n${schema.positionalHint}` : ''; + throw new ArgError(`Unexpected argument: ${extra}${hint}`); + } + + return { flags, values, positionals }; +} diff --git a/apps/cli/src/colors.ts b/apps/cli/src/colors.ts new file mode 100644 index 0000000..cbc1801 --- /dev/null +++ b/apps/cli/src/colors.ts @@ -0,0 +1,34 @@ +/** + * ANSI color and style escapes — the single source for the CLI's palette. + * + * Codes are plain constants; callers decide whether to emit them via `paint` + * (wrap-and-reset) or `gate` (prefix-or-empty), gating on `supportsColor()` from + * `tty.ts`. Cursor-control escapes live with their sole consumer, not here — this + * module is color only. + */ + +export const RESET = '\x1b[0m'; + +/** Shannon brand gold — the running/completed accent, shared with the splash logo. */ +export const GOLD = '\x1b[38;2;244;197;66m'; + +export const BOLD = '\x1b[1m'; +export const RED = '\x1b[31m'; +export const YELLOW = '\x1b[33m'; +export const DIM = '\x1b[90m'; + +// The splash logo uses bolder variants of cyan/white/yellow than the progress tree. +export const CYAN = '\x1b[36;1m'; +export const WHITE = '\x1b[1;37m'; +export const GRAY = '\x1b[0;37m'; +export const BOLD_YELLOW = '\x1b[1;33m'; + +/** Wrap `text` in `code` and reset, or return it unchanged when color is off. */ +export function paint(text: string, code: string, enabled: boolean): string { + return enabled ? `${code}${text}${RESET}` : text; +} + +/** A style code when color is on, or an empty string when off — for templates that interleave prefixes directly. */ +export function gate(code: string, enabled: boolean): string { + return enabled ? code : ''; +} diff --git a/apps/cli/src/commands/build.ts b/apps/cli/src/commands/build.ts index 80992e4..1c17cea 100644 --- a/apps/cli/src/commands/build.ts +++ b/apps/cli/src/commands/build.ts @@ -3,13 +3,17 @@ * Requires a clone (Dockerfile in the working directory). */ -import { buildImage, canBuildImage } from '../docker.js'; +import { buildImage, canBuildImage, ensureDocker } from '../docker.js'; +import { fail } from '../errors.js'; export function build(noCache: boolean, version: string): void { + ensureDocker(); + if (!canBuildImage()) { - console.error('ERROR: Build is only available when running from the Shannon repository'); - console.error(' (Dockerfile not found in current directory)'); - process.exit(1); + fail( + 'Build is only available when running from the Shannon repository', + ' (Dockerfile not found in current directory)', + ); } buildImage(noCache, version); diff --git a/apps/cli/src/commands/logs.ts b/apps/cli/src/commands/logs.ts index 37d464a..ed6bcec 100644 --- a/apps/cli/src/commands/logs.ts +++ b/apps/cli/src/commands/logs.ts @@ -8,8 +8,10 @@ import fs from 'node:fs'; import path from 'node:path'; import { watch } from 'chokidar'; +import { fail } from '../errors.js'; import { getWorkspacesDir } from '../home.js'; import { resolveRunFile } from '../paths.js'; +import { stdoutIsTerminal } from '../tty.js'; // Match the exact line the worker writes — anchored to prevent false positives from agent output const COMPLETION_PATTERN = /^Scan (COMPLETED|FAILED)$/m; @@ -28,7 +30,7 @@ function readRange(filePath: string, start: number, end: number): string { } /** Resolve a workspace ID to its workflow.log path, or exit with an error. */ -function resolveLogFile(workspaceId: string): string { +export function resolveLogFile(workspaceId: string): string { const workspacesDir = getWorkspacesDir(); // 1. Direct match @@ -49,59 +51,71 @@ function resolveLogFile(workspaceId: string): string { if (fs.existsSync(namedPath)) return namedPath; } - console.error(`ERROR: No scan found named: ${workspaceId}`); - console.error(''); - console.error('Possible causes:'); - console.error(" - The scan hasn't started yet"); - console.error(' - The workspace name is incorrect'); - console.error(''); - console.error('Check the dashboard at http://localhost:8233 for scan details'); - process.exit(1); + fail( + `No scan found named: ${workspaceId}`, + '', + 'Possible causes:', + " - The scan hasn't started yet", + ' - The workspace name is incorrect', + '', + 'Check the dashboard at http://localhost:8233 for scan details', + ); +} + +/** + * Tail a scan's log until it reports completion, resolving when the completion marker appears + * (or the file is gone, or Ctrl-C stops it). Never exits the process, so the caller decides what + * happens next: plain `logs` exits 0; `start --follow` reads the workflow outcome first. + */ +export function tailUntilComplete(logFile: string): Promise { + return new Promise((resolve) => { + let position = 0; + + /** + * Output any new content appended since the last read. + * Returns true when the workflow completion marker is detected. + */ + function flush(): boolean { + try { + const { size } = fs.statSync(logFile); + if (size <= position) return false; + + const data = readRange(logFile, position, size); + process.stdout.write(data); + position = size; + + return COMPLETION_PATTERN.test(data); + } catch { + // File deleted or unreadable — treat as done + return true; + } + } + + // 1. Output existing content + if (flush()) { + resolve(); + return; + } + + // 2. Watch for appended content via chokidar + const watcher = watch(logFile, { persistent: true }); + + const stop = (): void => { + watcher.close().finally(() => resolve()); + // Safety net — resolve anyway if watcher.close() stalls + setTimeout(() => resolve(), 1000).unref(); + }; + + watcher.on('change', () => { + if (flush()) stop(); + }); + + process.on('SIGINT', stop); + }); } export function logs(workspaceId: string): void { const logFile = resolveLogFile(workspaceId); - let position = 0; - - /** - * Output any new content appended since the last read. - * Returns true when the workflow completion marker is detected. - */ - function flush(): boolean { - try { - const { size } = fs.statSync(logFile); - if (size <= position) return false; - - const data = readRange(logFile, position, size); - process.stdout.write(data); - position = size; - - return COMPLETION_PATTERN.test(data); - } catch { - // File deleted or unreadable — treat as done - return true; - } - } - - console.log(`Tailing scan log: ${logFile}`); - - // 1. Output existing content - if (flush()) { - process.exit(0); - } - - // 2. Watch for appended content via chokidar - const watcher = watch(logFile, { persistent: true }); - - const shutdown = (): void => { - watcher.close().finally(() => process.exit(0)); - // Safety net — force exit if watcher.close() stalls - setTimeout(() => process.exit(0), 1000).unref(); - }; - - watcher.on('change', () => { - if (flush()) shutdown(); - }); - - process.on('SIGINT', shutdown); + console.error(stdoutIsTerminal() ? `Tailing scan log: ${logFile}` : 'Tailing scan log'); + tailUntilComplete(logFile).finally(() => process.exit(0)); } diff --git a/apps/cli/src/commands/reset.ts b/apps/cli/src/commands/reset.ts new file mode 100644 index 0000000..3675915 --- /dev/null +++ b/apps/cli/src/commands/reset.ts @@ -0,0 +1,26 @@ +/** + * `shannon reset` command — stop everything and wipe all Temporal data and volumes, + * returning the machine to a clean slate. The destructive counterpart to `stop`. + */ + +import * as p from '@clack/prompts'; +import { confirmByTyping } from '../confirm.js'; +import { ensureDocker, runningContainers, stopContainers, stopInfra, WORKER_FILTER } from '../docker.js'; + +export async function reset(): Promise { + ensureDocker(); + + console.log('This will stop all running scans and permanently remove all Temporal data and volumes.'); + await confirmByTyping('reset', 'confirm'); + + const spinner = p.spinner(); + spinner.start('Stopping scans'); + const running = runningContainers(WORKER_FILTER); + await stopContainers(running); + spinner.stop( + running.length > 0 ? `Stopped ${running.length} scan${running.length === 1 ? '' : 's'}` : 'No scans running', + ); + + await stopInfra(true); + console.log('Reset complete.'); +} diff --git a/apps/cli/src/commands/scans.ts b/apps/cli/src/commands/scans.ts new file mode 100644 index 0000000..d184eef --- /dev/null +++ b/apps/cli/src/commands/scans.ts @@ -0,0 +1,197 @@ +/** + * `shannon scans` command — list completed scans and where each report lives. + * + * A scan counts as completed when it produced a report. The report can live in any of a + * few locations depending on the version that ran it, so `findReport` probes them in order + * and the first hit is both the completion signal and the link target behind the workspace + * name. The date and wall-clock duration come from the run's session.json + * (createdAt/completedAt), with the report file's mtime as the date fallback for + * runs that lack a recorded time. + * + * Human-readable by default; `--json` emits the same rows as raw machine values on stdout. + * + * Filesystem-only (local ./workspaces/ or npx ~/.shannon/workspaces/ via getWorkspacesDir); + * no Temporal dependency. + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import { pathToFileURL } from 'node:url'; +import { BOLD, GOLD, paint } from '../colors.js'; +import { getWorkspacesDir } from '../home.js'; +import { commandPrefix } from '../mode.js'; +import { FINAL_REPORT_PDF_FILENAME, INTERNAL_DIR, resolveRunFile } from '../paths.js'; +import { stdoutIsTerminal, supportsColor } from '../tty.js'; + +/** Assembled report in the deliverables dir. Must match ASSEMBLED_REPORT_FILENAME in the worker package. */ +const ASSEMBLED_REPORT_FILENAME = 'comprehensive_security_assessment_report.md'; + +/** Run-root markdown surfaced by older versions, before the PDF. Kept so those runs still list. */ +const FINAL_REPORT_MD_FILENAME = 'Security-Assessment-Report.md'; + +const DELIVERABLES_SUBDIR = 'deliverables'; + +/** One completed scan; raw values so the table and --json render from one source. */ +interface ScanRow { + readonly workspace: string; + /** Completion time in ms — sort key and date source. */ + readonly finishedMs: number; + /** Wall-clock duration (completedAt − createdAt) in ms, or null when unknown. */ + readonly durationMs: number | null; + /** Absolute path to the report file — the link target behind the workspace name. */ + readonly report: string; +} + +/** The --json row shape: raw machine values, one per completed scan. */ +interface JsonRow { + readonly workspace: string; + readonly finishedAt: string; + readonly durationMs: number | null; + readonly reportPath: string; +} + +/** Compact wall-clock duration from milliseconds: "47s", "1m 32s", "1h 47m". */ +function formatDuration(ms: number): string { + const totalSeconds = Math.round(ms / 1000); + if (totalSeconds < 60) { + return `${totalSeconds}s`; + } + const totalMinutes = Math.floor(totalSeconds / 60); + if (totalMinutes < 60) { + return `${totalMinutes}m ${totalSeconds % 60}s`; + } + return `${Math.floor(totalMinutes / 60)}h ${totalMinutes % 60}m`; +} + +/** + * Wrap `text` in an OSC 8 hyperlink to `url` so a supporting terminal opens it on click, + * or return `text` unchanged. Terminals without OSC 8 simply show the text. + */ +function hyperlink(text: string, url: string): string { + return `\x1b]8;;${url}\x1b\\${text}\x1b]8;;\x1b\\`; +} + +/** First existing report path for a run (newest-surfaced first), or null if it has none. */ +function findReport(runDir: string): string | null { + const candidates = [ + path.join(runDir, FINAL_REPORT_PDF_FILENAME), + path.join(runDir, FINAL_REPORT_MD_FILENAME), + path.join(runDir, INTERNAL_DIR, DELIVERABLES_SUBDIR, ASSEMBLED_REPORT_FILENAME), + path.join(runDir, DELIVERABLES_SUBDIR, ASSEMBLED_REPORT_FILENAME), + ]; + + for (const candidate of candidates) { + if (fs.existsSync(candidate)) { + return candidate; + } + } + return null; +} + +interface SessionData { + readonly session: { readonly createdAt?: string; readonly completedAt?: string }; +} + +/** Read a run's session.json (dual-read across layouts). Missing or unreadable → empty shape. */ +function readSession(runDir: string): SessionData { + try { + const parsed = JSON.parse(fs.readFileSync(resolveRunFile(runDir, 'session.json'), 'utf8')); + return { session: parsed?.session ?? {} }; + } catch { + return { session: {} }; + } +} + +/** Gather every workspace that has a report, one row each. */ +function collectCompletedScans(workspacesDir: string): ScanRow[] { + let entries: fs.Dirent[]; + try { + entries = fs.readdirSync(workspacesDir, { withFileTypes: true }); + } catch { + // Workspaces directory does not exist yet — no scans have ever run. + return []; + } + + const rows: ScanRow[] = []; + for (const entry of entries) { + if (!entry.isDirectory()) { + continue; + } + + const runDir = path.join(workspacesDir, entry.name); + const reportPath = findReport(runDir); + if (!reportPath) { + continue; + } + + const { session } = readSession(runDir); + const completedMs = Date.parse(session.completedAt ?? ''); + const createdMs = Date.parse(session.createdAt ?? ''); + const finishedMs = Number.isNaN(completedMs) ? fs.statSync(reportPath).mtimeMs : completedMs; + const durationMs = Number.isNaN(completedMs) || Number.isNaN(createdMs) ? null : completedMs - createdMs; + + rows.push({ workspace: entry.name, finishedMs, durationMs, report: reportPath }); + } + return rows; +} + +function toJsonRow(row: ScanRow): JsonRow { + return { + workspace: row.workspace, + finishedAt: new Date(row.finishedMs).toISOString(), + durationMs: row.durationMs, + reportPath: row.report, + }; +} + +/** Print the completed scans as an aligned table with the workspace name linked to its report. */ +function printTable(workspacesDir: string, rows: readonly ScanRow[]): void { + if (rows.length === 0) { + const prefix = commandPrefix(); + console.log(`No completed scans yet. Run '${prefix} start -u -r ' to begin.`); + return; + } + + const color = supportsColor(); + // On a terminal the workspace name is an OSC 8 hyperlink that opens its report; when + // piped there is nothing to click, so it prints as plain text. + const linkable = stdoutIsTerminal(); + + const table = rows.map((row) => ({ + finished: new Date(row.finishedMs).toISOString().slice(0, 10), + duration: row.durationMs === null ? '—' : formatDuration(row.durationMs), + workspace: row.workspace, + report: row.report, + })); + + const dateWidth = Math.max('FINISHED'.length, 'YYYY-MM-DD'.length); + const durationWidth = Math.max('DURATION'.length, ...table.map((row) => row.duration.length)); + + console.log(`\nCompleted scans in ${workspacesDir}:\n`); + const header = `${'FINISHED'.padEnd(dateWidth)} ${'DURATION'.padEnd(durationWidth)} WORKSPACE`; + console.log(paint(header, BOLD, color)); + + for (const row of table) { + const finished = row.finished.padEnd(dateWidth); + const duration = row.duration.padEnd(durationWidth); + const name = paint(row.workspace, GOLD, color); + const workspace = linkable ? hyperlink(name, pathToFileURL(row.report).href) : name; + console.log(`${finished} ${duration} ${workspace}`); + } + console.log(''); +} + +export function scans(opts: { readonly json: boolean }): void { + const workspacesDir = getWorkspacesDir(); + const rows = collectCompletedScans(workspacesDir); + + // Latest on top. + rows.sort((a, b) => b.finishedMs - a.finishedMs); + + if (opts.json) { + console.log(JSON.stringify(rows.map(toJsonRow), null, 2)); + return; + } + + printTable(workspacesDir, rows); +} diff --git a/apps/cli/src/commands/start.ts b/apps/cli/src/commands/start.ts index dbc36f2..97de116 100644 --- a/apps/cli/src/commands/start.ts +++ b/apps/cli/src/commands/start.ts @@ -8,14 +8,27 @@ import { execFileSync } from 'node:child_process'; import fs from 'node:fs'; import path from 'node:path'; -import { ensureImage, ensureInfra, randomSuffix, spawnWorker } from '../docker.js'; +import { setTimeout as sleep } from 'node:timers/promises'; +import * as p from '@clack/prompts'; +import { ensureDocker, ensureImage, ensureInfra, randomSuffix, spawnWorker } from '../docker.js'; import { buildEnvFlags, loadEnv, resolveHostPiAuthPath, shouldUsePiAuth, validateCredentials } from '../env.js'; +import { fail } from '../errors.js'; import { getWorkspacesDir, initHome } from '../home.js'; -import { isLocal } from '../mode.js'; +import { commandPrefix, isLocal } from '../mode.js'; import { resolveModelSpec } from '../model-spec.js'; -import { FINAL_REPORT_PDF_FILENAME, INTERNAL_DIR, resolveConfig, resolveRepo, resolveRunFile } from '../paths.js'; +import { + expandHome, + FINAL_REPORT_PDF_FILENAME, + INTERNAL_DIR, + resolveConfig, + resolveRepo, + resolveRunFile, +} from '../paths.js'; +import { resolveWorkflowId } from '../session.js'; import { displaySplash } from '../splash.js'; +import { getTerminalOutcome } from '../temporal-client.js'; import { stdoutIsTerminal } from '../tty.js'; +import { tailUntilComplete } from './logs.js'; export interface StartArgs { url: string; @@ -24,7 +37,8 @@ export interface StartArgs { workspace?: string; output?: string; pipelineTesting: boolean; - debug: boolean; + keepContainer: boolean; + follow: boolean; version: string; } @@ -59,22 +73,29 @@ export async function start(args: StartArgs): Promise { // 2. Validate credentials const creds = validateCredentials(); if (!creds.valid) { - console.error(`ERROR: ${creds.error}`); - process.exit(1); + fail(creds.error ?? 'Invalid credentials'); } // 3. Resolve paths const repo = resolveRepo(args.repo); const config = args.config ? resolveConfig(args.config) : undefined; + // Inputs are valid — show the splash before the Docker/Temporal setup work. + displaySplash(isLocal() ? undefined : args.version); + // 4. Ensure workspaces dir is writable by container user (UID 1001) const workspacesDir = getWorkspacesDir(); fs.mkdirSync(workspacesDir, { recursive: true }); fs.chmodSync(workspacesDir, 0o777); - // 5. Ensure image (auto-build in dev, pull in npx) and start infra + // 5. Ensure Docker and the worker image are available (pull/build prints its own progress). + ensureDocker(); ensureImage(args.version); - await ensureInfra(); + + // One spinner spans the whole launch: bringing up Temporal and registering the worker. + const spinner = p.spinner(); + spinner.start('Starting scan'); + await ensureInfra(spinner); // 6. Generate unique task queue and container name const suffix = randomSuffix(); @@ -109,7 +130,7 @@ export async function start(args: StartArgs): Promise { fs.mkdirSync(path.join(repo.hostPath, '.playwright'), { recursive: true }); // 10. Resolve output directory - const outputDir = args.output ? path.resolve(args.output) : undefined; + const outputDir = args.output ? path.resolve(expandHome(args.output)) : undefined; if (outputDir) { fs.mkdirSync(outputDir, { recursive: true }); } @@ -117,10 +138,7 @@ export async function start(args: StartArgs): Promise { // 11. Resolve prompts directory (local mode only) const promptsDir = isLocal() ? path.resolve('apps/worker/prompts') : undefined; - // 12. Display splash screen - displaySplash(isLocal() ? undefined : args.version); - - // 13. Spawn worker container + // 12. Spawn worker container const proc = spawnWorker({ version: args.version, url: args.url, @@ -134,20 +152,18 @@ export async function start(args: StartArgs): Promise { ...(outputDir && { outputDir }), workspace, ...(args.pipelineTesting && { pipelineTesting: true }), - ...(args.debug && { debug: true }), + ...(args.keepContainer && { keepContainer: true }), ...(shouldUsePiAuth() && { piAuthHostPath: resolveHostPiAuthPath() }), }); - // 14. Bail if `docker run -d` itself fails (mount error, image missing, etc.) + // Bail if `docker run -d` itself fails (mount error, image missing, etc.) const dockerExitCode = await new Promise((resolve) => { proc.once('exit', (code) => resolve(code ?? 1)); - proc.once('error', (err) => { - console.error(`Failed to start the scan: ${err.message}`); - resolve(1); - }); + proc.once('error', () => resolve(1)); }); if (dockerExitCode !== 0) { + spinner.error('Could not start the scan'); process.exit(1); } @@ -164,64 +180,23 @@ export async function start(args: StartArgs): Promise { } } - // Poll for workflow to register in session.json. Off-TTY, skip the dots and - // clear-line escape so redirected logs stay clean. - const animate = stdoutIsTerminal(); - process.stdout.write('Waiting for the scan to start...'); - let workflowId = ''; let started = false; - let attempts = 0; - const pollInterval = setInterval(() => { - attempts++; - if (attempts > 60) { - clearInterval(pollInterval); - process.stdout.write('\n'); - console.error('Timed out waiting for the scan to start'); - process.exit(1); - } - try { - const session = JSON.parse(fs.readFileSync(sessionJson, 'utf-8')); - const resumeAttempts: { workflowId: string }[] = session.session?.resumeAttempts ?? []; - - // Fresh: session.json appears with originalWorkflowId. Resume: new resumeAttempts entry. - const ready = isResume ? resumeAttempts.length > initialResumeCount : !!session.session?.originalWorkflowId; - - if (ready) { - clearInterval(pollInterval); - started = true; - - // Latest workflow ID: last resume attempt, or originalWorkflowId for fresh scans - workflowId = resumeAttempts.at(-1)?.workflowId ?? session.session?.originalWorkflowId ?? ''; - - // Clear the waiting line, or just break it off-TTY - process.stdout.write(animate ? '\r\x1b[K' : '\n'); - printInfo(args, workspace, workflowId, repo.hostPath, workspacesDir); - return; - } - } catch { - // File doesn't exist yet - } - if (animate) process.stdout.write('.'); - }, 2000); - - // Stop the worker container only if it hasn't started yet + // Stop the worker only if the scan hasn't registered yet (e.g. Ctrl-C mid-startup). let cleaned = false; const cleanup = (): void => { if (cleaned || started) return; cleaned = true; - clearInterval(pollInterval); - console.log('\nStopping scan...'); + spinner.stop('Stopping scan'); try { execFileSync('docker', ['stop', containerName], { stdio: 'pipe' }); } catch { // Container may have already exited } - if (args.debug) { - printDebugHint(containerName); + if (args.keepContainer) { + printPreservedContainerHint(containerName); } }; - process.on('SIGINT', () => { cleanup(); process.exit(0); @@ -231,9 +206,69 @@ export async function start(args: StartArgs): Promise { process.exit(0); }); process.on('exit', cleanup); + + // Poll for the workflow to register in session.json; the spinner resolves once it does. + spinner.message('Waiting for the scan to start'); + for (let attempts = 0; attempts < 60; attempts++) { + try { + const session = JSON.parse(fs.readFileSync(sessionJson, 'utf-8')); + const resumeAttempts: { workflowId: string }[] = session.session?.resumeAttempts ?? []; + + // Fresh: session.json appears with originalWorkflowId. Resume: new resumeAttempts entry. + const ready = isResume ? resumeAttempts.length > initialResumeCount : !!session.session?.originalWorkflowId; + + if (ready) { + started = true; + spinner.stop(`Scan started — ${workspace}`); + printInfo(args, workspace, repo.hostPath, workspacesDir); + if (args.follow) { + await followScan(workspace, workspacesDir); + } + return; + } + } catch { + // File doesn't exist yet + } + await sleep(2000); + } + + spinner.error('Timed out waiting for the scan to start'); + process.exit(1); } -function printDebugHint(containerName: string): void { +/** + * Follow a just-started scan (for `--follow`, aimed at CI): stream its log to completion, then + * exit on the workflow outcome — 0 if the assessment ran, 1 if the scan failed. That tracks + * whether the pipeline ran, not whether vulnerabilities were found. + */ +async function followScan(workspace: string, workspacesDir: string): Promise { + const logFile = resolveRunFile(path.join(workspacesDir, workspace), 'workflow.log'); + + // The worker creates workflow.log as it starts; wait briefly so the first read doesn't + // mistake a not-yet-created file for an already-finished scan. + for (let attempts = 0; attempts < 30 && !fs.existsSync(logFile); attempts++) { + await sleep(1000); + } + + if (stdoutIsTerminal()) { + console.error('\n Following scan log (Ctrl-C to stop watching):\n'); + } + await tailUntilComplete(logFile); + + const workflowId = resolveWorkflowId(workspace); + if (!workflowId) { + fail('Scan finished but its workflow id could not be resolved from session.json.'); + } + + try { + const outcome = await getTerminalOutcome(workflowId); + process.exit(outcome.kind === 'success' ? 0 : 1); + } catch { + fail('Could not reach Temporal at 127.0.0.1:7233 to read the scan outcome.'); + } +} + +function printPreservedContainerHint(containerName: string): void { console.log(''); console.log(` Worker container preserved: ${containerName}`); console.log(` Inspect logs: docker logs ${containerName}`); @@ -241,23 +276,19 @@ function printDebugHint(containerName: string): void { console.log(''); } -function printInfo( - args: StartArgs, - workspace: string, - workflowId: string, - repoPath: string, - workspacesDir: string, -): void { - const logsCmd = isLocal() ? `./shannon logs ${workspace}` : `npx @keygraph/shannon logs ${workspace}`; - const reportPath = path.join(workspacesDir, workspace, FINAL_REPORT_PDF_FILENAME); +function printInfo(args: StartArgs, workspace: string, repoPath: string, workspacesDir: string): void { + const interactive = stdoutIsTerminal(); + + if (interactive && !args.follow) { + console.log(' It runs in the background — you can close this terminal.'); + console.log(''); + } - console.log(' Scan started — it runs in the background, so you can close this terminal.'); - console.log(''); console.log(` Target: ${args.url}`); - console.log(` Repository: ${repoPath}`); + console.log(` Repository: ${interactive ? repoPath : path.basename(repoPath)}`); console.log(` Workspace: ${workspace}`); if (args.config) { - console.log(` Config: ${path.resolve(args.config)}`); + console.log(` Config: ${interactive ? path.resolve(args.config) : path.basename(args.config)}`); } if (args.pipelineTesting) { console.log(' Mode: Pipeline Testing'); @@ -268,14 +299,22 @@ function printInfo( console.log(` Model: ${spec.providerId}:${spec.modelId}`); } - console.log(''); - console.log(' Watch scan progress:'); - console.log(` Live logs: ${logsCmd}`); - if (workflowId) { - console.log(` Dashboard: http://localhost:8233/namespaces/default/workflows/${workflowId}`); - } else { - console.log(' Dashboard: http://localhost:8233'); + if (!interactive) { + return; } + + const reportPath = path.join(workspacesDir, workspace, FINAL_REPORT_PDF_FILENAME); + + // When following, the scan log streams inline next, so the "run these to watch it" hints + // would only contradict that. + if (!args.follow) { + const prefix = commandPrefix(); + console.log(''); + console.log(' Watch scan progress:'); + console.log(` Live logs: ${prefix} logs ${workspace}`); + console.log(` Progress: ${prefix} status ${workspace}`); + } + console.log(''); console.log(' Report (when the scan finishes):'); console.log(` ${reportPath}`); diff --git a/apps/cli/src/commands/status.ts b/apps/cli/src/commands/status.ts index 4bc982f..5ef3a04 100644 --- a/apps/cli/src/commands/status.ts +++ b/apps/cli/src/commands/status.ts @@ -1,24 +1,196 @@ /** - * `shannon status` command — show running scans and Temporal health. + * `shannon status ` — one scan's live progress from Temporal. + * + * While the scan runs, polls Temporal and redraws the phase/agent tree on a + * terminal (a pipe or a finished scan gets a single frame). When the scan reaches + * a terminal state, prints the overall result and exits. Reads Temporal directly — + * no worker, no session files — so it needs Temporal up and shows scans within its + * ~24h retention window. */ -import { isTemporalReady, listRunningWorkers } from '../docker.js'; +import { setTimeout as sleep } from 'node:timers/promises'; +import { fail } from '../errors.js'; +import { isLocal } from '../mode.js'; +import { type RenderInput, renderScan } from '../scan/render.js'; +import { toStatusJson } from '../scan/status-json.js'; +import { resolveWorkflowId } from '../session.js'; +import { displaySplash } from '../splash.js'; +import { describeScan, getTerminalOutcome, queryProgress, type ScanDescription } from '../temporal-client.js'; +import { stdoutIsTerminal, supportsColor } from '../tty.js'; +import { getVersion } from '../version.js'; -export function status(): void { - // 1. Temporal health - const temporalUp = isTemporalReady(); - console.log(`Temporal: ${temporalUp ? 'running' : 'not running'}`); - if (temporalUp) { - console.log(' Dashboard: http://localhost:8233'); +const HIDE_CURSOR = '\x1b[?25l'; +const SHOW_CURSOR = '\x1b[?25h'; +/** Redraw cadence for the spinner animation; data is refreshed on the slower poll. */ +const RENDER_MS = 120; +const POLL_MS = 1200; + +/** Terminal = anything other than an open, running execution. */ +function isTerminalStatus(status: string): boolean { + return status !== 'RUNNING' && status !== 'UNSPECIFIED'; +} + +// Match SGR color escapes (ESC[…m) so a line's on-screen width excludes them. Built from the ESC +// char code so the source carries no literal control character. +const ANSI_PATTERN = new RegExp(`${String.fromCharCode(27)}\\[[0-9;]*m`, 'g'); + +/** + * Physical terminal rows a frame occupies, so the live redraw moves the cursor up by the right + * amount. A line wider than the terminal wraps onto extra rows, so counting logical lines alone + * undercounts and the redraw drifts downward. Color escapes don't take screen columns, so strip them. + */ +function physicalRows(frame: string): number { + const columns = process.stdout.columns || 80; + return frame.split('\n').reduce((rows, line) => { + const width = line.replace(ANSI_PATTERN, '').length; + return rows + Math.max(1, Math.ceil(width / columns)); + }, 0); +} + +function exitCodeFor(input: RenderInput): number { + if (input.temporalStatus === 'FAILED' || input.temporalStatus === 'TIMED_OUT') return 1; + if (input.state?.status === 'failed') return 1; + return 0; +} + +/** Live view of a running scan: its progress query plus the in-flight agents from describe. */ +async function buildRunningInput(workspace: string, workflowId: string, desc: ScanDescription): Promise { + const state = await queryProgress(workflowId); + return { + workspace, + workflowId, + temporalStatus: desc.status, + state, + running: desc.runningAgents, + ...(desc.startedAt !== undefined && { startedAt: desc.startedAt }), + }; +} + +/** Final view of a closed scan: its result (or the failure) plus timing from describe. */ +async function buildTerminalInput(workspace: string, workflowId: string, desc: ScanDescription): Promise { + const outcome = await getTerminalOutcome(workflowId); + const timing = { + ...(desc.startedAt !== undefined && { startedAt: desc.startedAt }), + ...(desc.closedAt !== undefined && { endedAt: desc.closedAt }), + }; + if (outcome.kind === 'success') { + return { workspace, workflowId, temporalStatus: desc.status, state: outcome.state, running: [], ...timing }; } - console.log(''); + return { + workspace, + workflowId, + temporalStatus: desc.status, + state: null, + running: [], + failureMessage: outcome.message, + ...timing, + }; +} - // 2. Running scans - const workers = listRunningWorkers(); - if (workers) { - console.log('Running scans:'); - console.log(workers); - } else { - console.log('No scans running'); +function printFrame(input: RenderInput): void { + const frame = renderScan(input, { + now: Date.now(), + color: supportsColor(), + unicode: stdoutIsTerminal(), + live: false, + frame: 0, + }); + process.stdout.write(`${frame}\n`); +} + +/** + * Poll Temporal and redraw until the scan reaches a terminal state, then print the + * final frame and exit. A fast ticker animates the running spinner off the cached + * snapshot; the network poll refreshes that snapshot on a slower cadence. + */ +async function watch(workspace: string, workflowId: string): Promise { + let prevRows = 0; + let frame = 0; + let cached: RenderInput | null = null; + + const draw = (input: RenderInput, live: boolean): void => { + const out = renderScan(input, { now: Date.now(), color: supportsColor(), unicode: true, live, frame }); + if (prevRows > 0) process.stdout.write(`\x1b[${prevRows}A\x1b[0J`); + process.stdout.write(`${out}\n`); + prevRows = physicalRows(out); + }; + + process.on('exit', () => process.stdout.write(SHOW_CURSOR)); + process.on('SIGINT', () => { + process.stdout.write('\n'); + process.exit(0); + }); + process.stdout.write(HIDE_CURSOR); + + const ticker = setInterval(() => { + frame++; + if (cached) draw(cached, true); + }, RENDER_MS); + + for (;;) { + const desc = await describeScan(workflowId); + if (!desc) { + clearInterval(ticker); + fail(`Scan "${workspace}" is no longer in Temporal.`); + } + + if (isTerminalStatus(desc.status)) { + clearInterval(ticker); + const input = await buildTerminalInput(workspace, workflowId, desc); + draw(input, false); + process.exit(exitCodeFor(input)); + } + + cached = await buildRunningInput(workspace, workflowId, desc); + await sleep(POLL_MS); } } + +/** Read one point-in-time snapshot from Temporal: the terminal result if closed, else live progress. */ +async function snapshot(workspace: string, workflowId: string, desc: ScanDescription): Promise { + return isTerminalStatus(desc.status) + ? buildTerminalInput(workspace, workflowId, desc) + : buildRunningInput(workspace, workflowId, desc); +} + +export async function status(workspace: string, opts: { readonly json: boolean }): Promise { + // A resume spawns a new workflow id (recorded in session.json); resolve through there so status + // follows the current resume, not the superseded original. Fresh scans: the name is the id. + const workflowId = resolveWorkflowId(workspace) ?? workspace; + + let desc: ScanDescription | null; + try { + desc = await describeScan(workflowId); + } catch { + fail('Could not reach Temporal at 127.0.0.1:7233.', 'Start Temporal (it comes up with a scan) and try again.'); + } + + if (!desc) { + fail( + `No scan found for "${workspace}".`, + '', + 'Scans are visible while running and for ~24h after they finish (Temporal retention).', + ); + } + + // --json is always a single snapshot then exit, even on a TTY — it never enters the live watch loop. + if (opts.json) { + const input = await snapshot(workspace, workflowId, desc); + process.stdout.write(`${JSON.stringify(toStatusJson(input, Date.now()), null, 2)}\n`); + process.exit(exitCodeFor(input)); + } + + // Human-facing views open with the splash; skip it off a real terminal so piped output stays clean. + if (stdoutIsTerminal()) { + displaySplash(isLocal() ? undefined : getVersion()); + } + + // A finished scan, or output that isn't a live terminal, gets a single frame. + if (isTerminalStatus(desc.status) || !stdoutIsTerminal()) { + const input = await snapshot(workspace, workflowId, desc); + printFrame(input); + process.exit(exitCodeFor(input)); + } + + await watch(workspace, workflowId); +} diff --git a/apps/cli/src/commands/stop.ts b/apps/cli/src/commands/stop.ts index 2b805aa..9096823 100644 --- a/apps/cli/src/commands/stop.ts +++ b/apps/cli/src/commands/stop.ts @@ -1,23 +1,127 @@ /** - * `shannon stop` command — stop workers and infrastructure. + * `shannon stop` command — stop one scan by workspace, or every scan with --all. + * Never touches infra or data; to wipe Temporal state entirely, use `shannon reset`. */ import * as p from '@clack/prompts'; -import { stopInfra, stopWorkers } from '../docker.js'; -import { requireInteractive } from '../tty.js'; +import { confirmOrExit } from '../confirm.js'; +import { + anyRunningScanWorkflow, + ensureDocker, + isTemporalReady, + isWorkflowRunning, + runningContainers, + scanFilter, + stopContainers, + terminateAllWorkflows, + terminateWorkflow, + WORKER_FILTER, +} from '../docker.js'; +import { fail, failUsage, warn } from '../errors.js'; +import { commandPrefix } from '../mode.js'; +import { resolveWorkflowId } from '../session.js'; -export async function stop(clean: boolean, yes: boolean): Promise { - if (clean && !yes) { - requireInteractive('stop --clean', 'Re-run with --yes to skip this confirmation.'); - const confirmed = await p.confirm({ - message: 'This will stop all running scans and remove the Temporal data. Continue?', - }); - if (p.isCancel(confirmed) || !confirmed) { - p.cancel('Aborted.'); - process.exit(0); +export interface StopOptions { + all: boolean; + yes: boolean; + workspace?: string; +} + +/** + * Stop a single scan. Terminating the workflow both clears Temporal's record and + * brings the container down (the worker waits on the workflow result), so that runs + * first; `docker stop` is the fallback for the pre-registration window and an + * unreachable Temporal. The stop is then verified rather than assumed. + */ +async function stopSingleScan(workspace: string, yes: boolean): Promise { + const workflowId = resolveWorkflowId(workspace); + const filter = scanFilter(workspace); + const temporalUp = isTemporalReady(); + + const initialContainers = runningContainers(filter); + const workflowRunning = Boolean(workflowId && temporalUp && isWorkflowRunning(workflowId)); + + // Resolve what is running before prompting, so we never confirm a no-op. + if (initialContainers.length === 0 && !workflowRunning) { + if (!workflowId) { + fail(`No scan found for workspace: ${workspace}`); } + console.log(`Nothing was running for ${workspace}.`); + return; } - stopWorkers(); - stopInfra(clean); + await confirmOrExit('stop', `Stop the scan "${workspace}"?`, yes); + + const spinner = p.spinner(); + spinner.start(`Stopping scan ${workspace}`); + + if (workflowId && workflowRunning) { + terminateWorkflow(workflowId, `Stopped via shannon stop ${workspace}`); + } + await stopContainers(runningContainers(filter)); + + const stillRunning = runningContainers(filter); + if (stillRunning.length > 0) { + spinner.error(`Scan ${workspace} may still be running`); + console.error(`${stillRunning.length} container(s) did not stop. Retry: ${commandPrefix()} stop ${workspace}`); + process.exit(1); + } + + spinner.stop(`Stopped scan ${workspace}`); + + if (workflowId && temporalUp && isWorkflowRunning(workflowId)) { + warn(`scan ${workspace} stopped, but its workflow is still Running in Temporal.`); + } +} + +async function stopAllScans(yes: boolean): Promise { + const temporalUp = isTemporalReady(); + const initial = runningContainers(WORKER_FILTER); + + // Resolve what is running before prompting, so we never confirm a no-op. + if (initial.length === 0) { + console.log('No running scans to stop.'); + return; + } + + await confirmOrExit('stop', 'This will stop all running scans. Continue?', yes); + + const spinner = p.spinner(); + spinner.start('Stopping all scans'); + + if (temporalUp) { + terminateAllWorkflows('Stopped via shannon stop --all'); + } + await stopContainers(runningContainers(WORKER_FILTER)); + + const stillRunning = runningContainers(WORKER_FILTER); + if (stillRunning.length > 0) { + spinner.error(`Stopped ${initial.length - stillRunning.length} of ${initial.length} scans`); + console.error(`${stillRunning.length} container(s) did not stop. Retry: ${commandPrefix()} stop --all`); + process.exit(1); + } + + spinner.stop(`Stopped ${initial.length} scan${initial.length === 1 ? '' : 's'}`); + + if (temporalUp && anyRunningScanWorkflow()) { + warn('some scan workflows are still Running in Temporal — check http://localhost:8233'); + } +} + +export async function stop(opts: StopOptions): Promise { + ensureDocker(); + + // Validate the target: exactly one of or --all. + if (opts.all && opts.workspace) { + failUsage('Pass a workspace name or --all, not both.'); + } + if (!opts.all && !opts.workspace) { + failUsage('Specify which scan to stop: `stop `, or `stop --all` to stop every scan.'); + } + + if (opts.workspace) { + await stopSingleScan(opts.workspace, opts.yes); + } else { + await stopAllScans(opts.yes); + } } diff --git a/apps/cli/src/commands/uninstall.ts b/apps/cli/src/commands/uninstall.ts deleted file mode 100644 index 1010736..0000000 --- a/apps/cli/src/commands/uninstall.ts +++ /dev/null @@ -1,55 +0,0 @@ -/** - * `npx @keygraph/shannon uninstall` command — remove ~/.shannon/ after confirmation (npx only). - */ - -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import * as p from '@clack/prompts'; -import { stopInfra, stopWorkers } from '../docker.js'; -import { requireInteractive } from '../tty.js'; - -const SHANNON_HOME = path.join(os.homedir(), '.shannon'); - -export async function uninstall(yes: boolean): Promise { - const interactive = !yes; - if (interactive) p.intro('Shannon Uninstall'); - - if (!fs.existsSync(SHANNON_HOME)) { - const message = 'Nothing to remove. Shannon is not configured on this machine.'; - if (interactive) { - p.log.info(message); - p.outro('Done.'); - } else { - console.log(message); - } - return; - } - - if (interactive) { - requireInteractive('uninstall', 'Re-run with --yes to skip this confirmation.'); - const confirmed = await p.confirm({ - message: 'This will permanently remove all past scan data, saved configurations, and API keys. Continue?', - }); - if (p.isCancel(confirmed) || !confirmed) { - p.cancel('Aborted.'); - process.exit(0); - } - } - - // Stop any running containers first - stopWorkers(); - stopInfra(false); - - fs.rmSync(SHANNON_HOME, { recursive: true, force: true }); - - const done = 'All Shannon data has been removed.'; - const hint = 'Shannon has been uninstalled. Run `npx @keygraph/shannon setup` to start fresh.'; - if (interactive) { - p.log.success(done); - p.outro(hint); - } else { - console.log(done); - console.log(hint); - } -} diff --git a/apps/cli/src/commands/workspaces.ts b/apps/cli/src/commands/workspaces.ts deleted file mode 100644 index 3a9aa33..0000000 --- a/apps/cli/src/commands/workspaces.ts +++ /dev/null @@ -1,35 +0,0 @@ -/** - * `shannon workspaces` command — list all workspaces. - */ - -import { execFileSync } from 'node:child_process'; -import os from 'node:os'; -import { getWorkerImage } from '../docker.js'; -import { getWorkspacesDir } from '../home.js'; - -export function workspaces(version: string): void { - const workspacesDir = getWorkspacesDir(); - const image = getWorkerImage(version); - - try { - execFileSync( - 'docker', - [ - 'run', - '--rm', - '-v', - `${workspacesDir}:/app/workspaces`, - '-e', - 'WORKSPACES_DIR=/app/workspaces', - image, - 'node', - 'apps/worker/dist/temporal/workspaces.js', - ], - { stdio: 'inherit', ...(os.platform() === 'win32' && { env: { ...process.env, MSYS_NO_PATHCONV: '1' } }) }, - ); - } catch { - console.error('ERROR: Failed to list workspaces. Is the Docker image available?'); - console.error(` Run: docker pull ${image}`); - process.exit(1); - } -} diff --git a/apps/cli/src/config/resolver.ts b/apps/cli/src/config/resolver.ts index 6bf218a..a58cd0c 100644 --- a/apps/cli/src/config/resolver.ts +++ b/apps/cli/src/config/resolver.ts @@ -7,6 +7,7 @@ import fs from 'node:fs'; import { parse as parseTOML } from 'smol-toml'; +import { fail } from '../errors.js'; import { getConfigFile } from '../home.js'; import { getMode } from '../mode.js'; import { @@ -100,10 +101,9 @@ function loadTOML(): TOMLConfig | null { const mode = fs.statSync(configPath).mode; if (mode & 0o077) { const actual = (mode & 0o777).toString(8).padStart(3, '0'); - console.error( - `\nYour config file is readable by other users on this machine (${actual}). Lock it down: chmod 600 ${configPath}\n`, + fail( + `Your config file is readable by other users on this machine (${actual}). Lock it down: chmod 600 ${configPath}`, ); - process.exit(1); } } @@ -112,9 +112,7 @@ function loadTOML(): TOMLConfig | null { return parseTOML(content) as TOMLConfig; } catch (err) { const message = err instanceof Error ? err.message : String(err); - console.error(`\nFailed to parse ${configPath}: ${message}`); - console.error(`\nRun 'npx @keygraph/shannon setup' to reconfigure.\n`); - process.exit(1); + fail(`Failed to parse ${configPath}: ${message}`, `Run 'npx @keygraph/shannon setup' to reconfigure.`); } } @@ -256,12 +254,11 @@ export function resolveConfig(): void { // Validate before injecting const errors = validateConfig(toml); if (errors.length > 0) { - console.error('\nInvalid configuration:'); - for (const err of errors) { - console.error(` - ${err}`); - } - console.error(`\nRun 'npx @keygraph/shannon setup' to reconfigure.\n`); - process.exit(1); + fail( + 'Invalid configuration:', + ...errors.map((err) => ` - ${err}`), + `Run 'npx @keygraph/shannon setup' to reconfigure.`, + ); } for (const mapping of CONFIG_MAP) { diff --git a/apps/cli/src/confirm.ts b/apps/cli/src/confirm.ts new file mode 100644 index 0000000..32ab908 --- /dev/null +++ b/apps/cli/src/confirm.ts @@ -0,0 +1,43 @@ +/** + * Shared confirmation prompt for destructive or batch commands. + * + * `stop` and `reset` gate their action behind the same "confirm unless --yes" + * flow. Centralizing it here keeps the behavior identical across commands and + * impossible to change in only one place by accident. + */ + +import * as p from '@clack/prompts'; +import { requireInteractive } from './tty.js'; + +/** + * Ask the user to confirm an action, unless `yes` was passed. Off a TTY without + * `--yes`, fails fast rather than hanging on a prompt. Exits 0 if the user declines. + */ +export async function confirmOrExit(command: string, message: string, yes: boolean): Promise { + if (yes) { + return; + } + + requireInteractive(command, 'Re-run with --yes to skip this confirmation.'); + const confirmed = await p.confirm({ message }); + if (p.isCancel(confirmed) || !confirmed) { + p.cancel('Aborted.'); + process.exit(0); + } +} + +/** + * Severe-tier confirmation: the user must type `word` exactly to proceed. Unlike + * `confirmOrExit` there is no `--yes` bypass. Off a TTY it fails fast; exits 0 if declined. + */ +export async function confirmByTyping(command: string, word: string): Promise { + requireInteractive(command, `'${command}' cannot be run non-interactively.`); + const typed = await p.text({ + message: `Type ${word} to confirm — this cannot be undone:`, + validate: (value) => (value === word ? undefined : `Type ${word} to proceed, or press Ctrl-C to abort.`), + }); + if (p.isCancel(typed) || typed !== word) { + p.cancel('Aborted.'); + process.exit(0); + } +} diff --git a/apps/cli/src/docker.ts b/apps/cli/src/docker.ts index fbb6d80..421bca0 100644 --- a/apps/cli/src/docker.ts +++ b/apps/cli/src/docker.ts @@ -12,15 +12,21 @@ import os from 'node:os'; import path from 'node:path'; import { setTimeout as sleep } from 'node:timers/promises'; import { fileURLToPath } from 'node:url'; +import type { SpinnerResult } from '@clack/prompts'; import { envBool, PI_AUTH_CONTAINER_PATH } from './env.js'; +import { fail } from './errors.js'; import { getMode, isDevMode } from './mode.js'; import { INTERNAL_DIR } from './paths.js'; +import { runStep, spawnCaptured, surfaceOutput } from './ui.js'; const __dirname = path.dirname(fileURLToPath(import.meta.url)); const NPX_IMAGE_REPO = 'keygraph/shannon'; const DEV_IMAGE = 'shannon-worker'; +/** Docker label stamped on each worker container, mapping it back to its workspace so a single scan can be stopped by name. */ +const WORKSPACE_LABEL = 'shannon.workspace'; + export function getWorkerImage(version: string): string { return getMode() === 'local' ? DEV_IMAGE : `${NPX_IMAGE_REPO}:${version}`; } @@ -66,44 +72,77 @@ function runOutput(cmd: string, args: string[]): string { } } +/** Run a command asynchronously, resolving true on success. Never rejects. */ +function spawnQuiet(cmd: string, args: string[]): Promise { + return new Promise((resolve) => { + const child = spawn(cmd, args, { stdio: 'ignore' }); + child.on('close', (code) => resolve(code === 0)); + child.on('error', () => resolve(false)); + }); +} + +const TEMPORAL_CONTAINER = 'shannon-temporal'; +const TEMPORAL_ADDRESS = 'localhost:7233'; + +/** Query matching every running pentest scan workflow. */ +const RUNNING_SCAN_QUERY = "ExecutionStatus = 'Running' AND WorkflowType = 'pentestPipelineWorkflow'"; + +/** Build `docker exec` args for a `temporal` CLI command run inside the Temporal container. */ +function temporalCmd(...args: string[]): string[] { + return ['exec', TEMPORAL_CONTAINER, 'temporal', ...args, '--address', TEMPORAL_ADDRESS]; +} + +/** + * Verify Docker is installed and its daemon is running, exiting otherwise. + * `docker info` succeeds only when both are true. Call this before any command + * that shells out to Docker. + */ +export function ensureDocker(): void { + try { + execFileSync('docker', ['info'], { stdio: 'pipe' }); + } catch { + fail( + 'Docker must be installed and running. Start Docker and try again.', + 'Install Docker: https://docs.docker.com/get-docker/', + ); + } +} + /** * Check if Temporal is running and healthy. */ export function isTemporalReady(): boolean { - const output = runOutput('docker', [ - 'exec', - 'shannon-temporal', - 'temporal', - 'operator', - 'cluster', - 'health', - '--address', - 'localhost:7233', - ]); + const output = runOutput('docker', temporalCmd('operator', 'cluster', 'health')); return output.includes('SERVING'); } /** * Ensure Temporal is running via compose. */ -export async function ensureInfra(): Promise { +export async function ensureInfra(spinner: SpinnerResult): Promise { if (isTemporalReady()) { return; } + // Drive the caller's spinner — the whole "start" flow is one spinner, not several. + spinner.message('Starting Temporal'); const composeFile = getComposeFile(); - console.log('Starting Shannon infrastructure...'); - execFileSync('docker', ['compose', '-f', composeFile, 'up', '-d'], { stdio: 'inherit' }); + const result = await spawnCaptured('docker', ['compose', '-f', composeFile, 'up', '-d']); + if (!result.ok) { + spinner.error('Could not start Temporal'); + surfaceOutput(result.output); + process.exit(1); + } - console.log('Waiting for Temporal to be ready...'); + spinner.message('Waiting for Temporal to be ready'); for (let i = 0; i < 30; i++) { if (isTemporalReady()) { - console.log('Temporal is ready!'); return; } await sleep(2000); } - console.error('Timeout waiting for Temporal'); + + spinner.error('Temporal did not become ready in time'); process.exit(1); } @@ -138,10 +177,11 @@ export function ensureImage(version: string): void { try { execFileSync('docker', ['pull', image], { stdio: 'inherit' }); } catch { - console.error(`\nERROR: Failed to pull ${image}`); - console.error('The image may not be available for your platform yet.'); - console.error('Check https://hub.docker.com/r/keygraph/shannon for available tags.'); - process.exit(1); + fail( + `Failed to pull ${image}`, + 'The image may not be available for your platform yet.', + 'Check https://hub.docker.com/r/keygraph/shannon for available tags.', + ); } pruneOldImages(version); } @@ -255,21 +295,24 @@ export interface WorkerOptions { outputDir?: string; workspace: string; pipelineTesting?: boolean; - debug?: boolean; + keepContainer?: boolean; piAuthHostPath?: string; } /** * Spawn the worker container in detached mode and return the process. - * When `opts.debug` is true, omits `--rm` so the container persists for log inspection. + * When `opts.keepContainer` is true, omits `--rm` so the container persists for log inspection. */ export function spawnWorker(opts: WorkerOptions): ChildProcess { const args = ['run', '-d']; - if (!opts.debug) { + if (!opts.keepContainer) { args.push('--rm'); } args.push('--name', opts.containerName, '--network', 'shannon-net'); + // Tag with the workspace so `stop ` can target this scan's container + args.push('--label', `${WORKSPACE_LABEL}=${opts.workspace}`); + // Add host flag for Linux args.push(...addHostFlag()); @@ -344,26 +387,86 @@ export function spawnWorker(opts: WorkerOptions): ChildProcess { }); } -/** - * Stop all running shannon-worker-* containers. - */ -export function stopWorkers(): void { - const workers = runOutput('docker', ['ps', '-q', '--filter', 'name=shannon-worker-']); - if (!workers) return; +/** `docker ps --filter` args matching every running worker container. */ +export const WORKER_FILTER: readonly string[] = ['--filter', 'name=shannon-worker-']; - const ids = workers.split('\n').filter(Boolean); - console.log('Stopping running scans...'); - execFileSync('docker', ['stop', ...ids], { stdio: 'inherit' }); +/** `docker ps --filter` args matching one scan's worker container(s), by workspace label. */ +export function scanFilter(workspace: string): readonly string[] { + return ['--filter', `label=${WORKSPACE_LABEL}=${workspace}`]; } /** - * Tear down the compose stack. + * IDs of running containers matching the filter. Re-querying this after a stop is + * the authoritative check for whether containers actually stopped — `docker stop`'s + * exit code can't distinguish "already gone" from "failed to stop". */ -export function stopInfra(clean: boolean): void { +export function runningContainers(filter: readonly string[]): string[] { + const output = runOutput('docker', ['ps', '-q', ...filter]); + return output.split('\n').filter(Boolean); +} + +/** + * Stop containers by ID, tolerating any that vanished between being listed and + * stopped (a `--rm` worker exiting is success, not an error). Async so a spinner + * can animate during docker's graceful-shutdown wait. + */ +export async function stopContainers(ids: string[]): Promise { + await Promise.all(ids.map((id) => spawnQuiet('docker', ['stop', id]))); +} + +/** + * Terminate a Temporal workflow so a stopped scan doesn't linger as a running + * workflow with no worker. Best-effort: returns false if Temporal is unreachable + * or the workflow already closed. Requires Temporal to be up (guard with isTemporalReady). + */ +export function terminateWorkflow(workflowId: string, reason: string): boolean { + return runQuiet('docker', temporalCmd('workflow', 'terminate', '--workflow-id', workflowId, '--reason', reason)); +} + +/** + * Terminate every running pentest workflow in one batch, so `stop --all` doesn't + * leave workflows running with no worker. Best-effort: returns false if Temporal + * is unreachable. Requires Temporal to be up (guard with isTemporalReady). + */ +export function terminateAllWorkflows(reason: string): boolean { + return runQuiet( + 'docker', + temporalCmd('workflow', 'terminate', '--query', RUNNING_SCAN_QUERY, '--reason', reason, '--yes'), + ); +} + +/** + * Whether a specific workflow is still in the Running state. Re-querying this after + * a terminate verifies it actually took effect, rather than trusting the terminate + * command's exit code. Requires Temporal to be up (guard with isTemporalReady). + */ +export function isWorkflowRunning(workflowId: string): boolean { + const query = `WorkflowId = '${workflowId}' AND ExecutionStatus = 'Running'`; + const output = runOutput('docker', temporalCmd('workflow', 'list', '--query', query)); + return output.includes(workflowId); +} + +/** + * Whether any pentest scan workflow is still Running — the `stop --all` counterpart + * to isWorkflowRunning. Requires Temporal to be up (guard with isTemporalReady). + */ +export function anyRunningScanWorkflow(): boolean { + const output = runOutput('docker', temporalCmd('workflow', 'list', '--query', RUNNING_SCAN_QUERY)); + return output.includes('pentestPipelineWorkflow'); +} + +/** + * Tear down the compose stack. When `clean` is set, volumes are removed too. + */ +export async function stopInfra(clean: boolean): Promise { const composeFile = getComposeFile(); const args = ['compose', '-f', composeFile, 'down']; if (clean) args.push('-v'); - execFileSync('docker', args, { stdio: 'inherit' }); + const label = clean ? 'Removing Temporal data and volumes' : 'Stopping Temporal'; + const step = await runStep(label, 'docker', args); + if (!step.ok) { + fail(`${label} failed. See the output above.`); + } } /** @@ -379,16 +482,3 @@ function pruneOldImages(currentVersion: string): void { runQuiet('docker', ['rmi', `${NPX_IMAGE_REPO}:${tag}`]); } } - -/** - * List running worker containers. - */ -export function listRunningWorkers(): string { - return runOutput('docker', [ - 'ps', - '--filter', - 'name=shannon-worker-', - '--format', - 'table {{.Names}}\t{{.Status}}\t{{.RunningFor}}', - ]); -} diff --git a/apps/cli/src/env.ts b/apps/cli/src/env.ts index 3071d59..98feb23 100644 --- a/apps/cli/src/env.ts +++ b/apps/cli/src/env.ts @@ -87,8 +87,9 @@ export function loadEnv(): void { } /** - * Build `-e KEY=VALUE` flags for docker run. Forwards the common vars plus only - * the selected provider's credentials. + * Build `-e` flags for docker run. Forwards the common vars plus only the + * selected provider's credentials, passed by name (`-e KEY`) so secret values + * stay out of the `docker run` argv; docker inherits them from this process's env. */ export function buildEnvFlags(): string[] { const flags: string[] = ['-e', 'TEMPORAL_ADDRESS=shannon-temporal:7233']; @@ -97,9 +98,8 @@ export function buildEnvFlags(): string[] { const providerVars = typeof spec === 'string' ? [] : providerForwardVars(spec.providerId); for (const key of [...COMMON_FORWARD_VARS, ...providerVars]) { - const value = process.env[key]; - if (value) { - flags.push('-e', `${key}=${value}`); + if (process.env[key]) { + flags.push('-e', key); } } @@ -171,8 +171,28 @@ export function validateCredentials(): CredentialValidation { // 3. Exactly one provider may be configured. Several complete credentials make // the scan's provider depend on SHANNON_AI_MODEL alone, which is too easy to // misread as "both are in play" and too easy to redirect by editing one line. - if (configuredProviders().length > 1) { - return { valid: false, error: 'Credentials for more than one provider are set.' }; + const configured = configuredProviders(); + if (configured.length > 1) { + const setKeys = (id: CuratedProviderId): string[] => + PROVIDER_API_KEY_ENV[id].filter((name) => Boolean(process.env[name])); + const list = configured.map((id) => `${id} (${setKeys(id).join(', ')})`).join(' and '); + const others = configured.filter((id) => id !== spec.providerId); + const extraVars = others.flatMap(setKeys); + + const dropHint = + getMode() === 'local' + ? 'remove them from .env or unset them in your shell:' + : "unset them in your shell, or reconfigure with 'npx @keygraph/shannon setup':"; + + const lines = [`Credentials for more than one provider are set: ${list}.`]; + if (extraVars.length > 0) { + lines.push( + `Shannon runs one provider per scan, selected by SHANNON_AI_MODEL ("${spec.providerId}:...").`, + `Keep ${spec.providerId} and drop the rest — ${dropHint}`, + ` unset ${extraVars.join(' ')}`, + ); + } + return { valid: false, error: lines.join('\n') }; } return { valid: true }; diff --git a/apps/cli/src/errors.ts b/apps/cli/src/errors.ts new file mode 100644 index 0000000..b067deb --- /dev/null +++ b/apps/cli/src/errors.ts @@ -0,0 +1,70 @@ +/** + * Centralized error reporting. + * + * `fail` — an expected, user-fixable error (bad input, missing prerequisite): + * a clean message on stderr and a non-zero exit, never a stack trace. + * `failUsage` — a malformed invocation (unknown command, bad or missing + * arguments): the same clean message, but a distinct exit code so callers can + * tell a usage mistake from an operational failure. + * `crash` — an unexpected error (a bug): a brief message, the full stack written + * to a log file for a bug report, and a pointer to the issue tracker. + */ + +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; + +const ISSUES_URL = 'https://github.com/KeygraphHQ/shannon/issues'; + +/** Report an expected, user-fixable error (with optional extra lines) and exit non-zero. */ +export function fail(message: string, ...hints: string[]): never { + console.error(`ERROR: ${message}`); + for (const hint of hints) { + console.error(hint); + } + process.exit(1); +} + +/** Report a usage/argument error (with optional extra lines) and exit 2. */ +export function failUsage(message: string, ...hints: string[]): never { + console.error(`ERROR: ${message}`); + for (const hint of hints) { + console.error(hint); + } + process.exit(2); +} + +/** Report a non-fatal warning on stderr (with optional extra lines) without exiting. */ +export function warn(message: string, ...hints: string[]): void { + console.error(`WARNING: ${message}`); + for (const hint of hints) { + console.error(hint); + } +} + +/** Report an unexpected error: brief message, full stack to a log file, plus the issue link. */ +export function crash(error: unknown): never { + console.error(`ERROR: ${error instanceof Error ? error.message : String(error)}`); + if (process.env.DEBUG) { + console.error(error instanceof Error ? error.stack : String(error)); + } + + const logPath = writeCrashLog(error); + if (logPath) { + console.error(`Details written to ${logPath}`); + } + console.error(`If this looks like a bug, please report it: ${ISSUES_URL}`); + process.exit(1); +} + +/** Write the full error and stack to a log file; return its path, or null if it can't be written. */ +function writeCrashLog(error: unknown): string | null { + try { + const logPath = path.join(os.tmpdir(), 'shannon-error.log'); + const detail = error instanceof Error && error.stack ? error.stack : String(error); + fs.writeFileSync(logPath, `${new Date().toISOString()}\n${detail}\n`); + return logPath; + } catch { + return null; + } +} diff --git a/apps/cli/src/help.ts b/apps/cli/src/help.ts new file mode 100644 index 0000000..f9da880 --- /dev/null +++ b/apps/cli/src/help.ts @@ -0,0 +1,145 @@ +/** + * Per-command help text. + * + * `shannon --help`, `shannon -h`, and `shannon help ` + * all render the matching command's usage, so a user can discover a command's + * flags without scanning the global help. The global help lives in index.ts. + */ + +import { commandPrefix, getMode } from './mode.js'; + +interface CommandHelp { + readonly usage: readonly string[]; + readonly description: string; + readonly options?: readonly (readonly [string, string])[]; + readonly examples?: readonly string[]; +} + +const YES_OPTION: readonly [string, string] = [ + '-y, --yes', + 'Skip the confirmation prompt (required for non-interactive use)', +]; +const HELP_OPTION: readonly [string, string] = ['-h, --help', 'Show this help']; + +/** + * `start`'s flags, the single source rendered by both the per-command help here + * and the global help in index.ts, so the two can never drift. + */ +export const START_OPTIONS: readonly (readonly [string, string])[] = [ + ['-u, --url ', 'Target URL (required)'], + ['-r, --repo ', 'Repository path (required)'], + ['-c, --config ', 'Configuration file (YAML)'], + ['-o, --output ', 'Copy deliverables to this directory after the run'], + ['-w, --workspace ', 'Named workspace (auto-resumes if it exists)'], + ['-f, --follow', 'Stream the scan log until it finishes'], + ['--pipeline-testing', 'Use minimal prompts for fast testing'], + ['--keep-container', 'Preserve the worker container after exit for log inspection'], +]; + +const COMMAND_HELP: Readonly> = { + start: { + usage: ['start -u -r [options]'], + description: 'Start a pentest scan.', + examples: [ + 'start -u https://example.com -r ./my-repo', + 'start -u https://example.com -r /path/to/repo -c config.yaml -w q1-audit', + 'start -u https://example.com -r ./my-repo --follow', + ], + }, + stop: { + usage: ['stop [--yes]', 'stop --all [--yes]'], + description: 'Stop one scan by workspace, or every scan with --all (Temporal stays up).', + options: [['--all', 'Stop all running scans'], YES_OPTION], + examples: ['stop q1-audit', 'stop --all'], + }, + reset: { + usage: ['reset'], + description: 'Stop everything and permanently remove all Temporal data and volumes.', + }, + logs: { + usage: ['logs '], + description: "Tail a scan's live log until it completes.", + examples: ['logs q1-audit'], + }, + status: { + usage: ['status [--json]'], + description: + "Show one scan's phase-by-phase progress, read live from Temporal. Watches and redraws until the scan finishes on a terminal; prints one frame when piped or already finished. With --json, prints a single machine-readable snapshot and exits.", + options: [['--json', 'Output a point-in-time snapshot as JSON, then exit']], + examples: ['status q1-audit', 'status q1-audit --json'], + }, + scans: { + usage: ['scans [--json]'], + description: 'List completed scans and where each report lives.', + options: [['--json', 'Output the scan list as JSON']], + examples: ['scans', 'scans --json'], + }, + build: { + usage: ['build [--no-cache]'], + description: 'Build the worker Docker image (local mode only).', + options: [['--no-cache', 'Build without using the Docker layer cache']], + }, + setup: { + usage: ['setup'], + description: 'Configure provider credentials interactively (npx mode only).', + }, + version: { + usage: ['version [--json]'], + description: 'Show the version. With --json, prints the version and mode as a machine-readable object.', + options: [['--json', 'Output the version and mode as JSON']], + examples: ['version', 'version --json'], + }, +}; + +/** Commands that only exist in one mode; everything else is available in both. */ +const MODE_ONLY: Readonly> = { + build: 'local', + setup: 'npx', +}; + +/** Whether a command has its own help page (and so responds to `--help`/`-h`). */ +export function isHelpableCommand(command: string): boolean { + return command in COMMAND_HELP; +} + +/** + * User-facing command names available in the current mode, for "did you mean?" + * suggestions. Derived from the same table that backs per-command help, so the + * suggestion set can never drift from the commands that actually exist. + */ +export function availableCommands(): readonly string[] { + const mode = getMode(); + const commands = Object.keys(COMMAND_HELP).filter((command) => (MODE_ONLY[command] ?? mode) === mode); + return [...commands, 'help']; +} + +/** Print the help page for one command. No-op if the command has no page. */ +export function printCommandHelp(command: string): void { + const help = COMMAND_HELP[command]; + if (!help) return; + + const prefix = commandPrefix(); + const baseOptions = command === 'start' ? START_OPTIONS : (help.options ?? []); + const options = [...baseOptions, HELP_OPTION]; + const flagWidth = Math.max(...options.map(([flag]) => flag.length)); + + const lines: string[] = ['', help.description, '', 'USAGE']; + for (const line of help.usage) { + lines.push(` ${prefix} ${line}`); + } + + lines.push('', 'OPTIONS'); + for (const [flag, desc] of options) { + lines.push(` ${flag.padEnd(flagWidth)} ${desc}`); + } + + if (help.examples && help.examples.length > 0) { + lines.push('', 'EXAMPLES'); + for (const example of help.examples) { + lines.push(` ${prefix} ${example}`); + } + } + + lines.push(''); + console.log(lines.join('\n')); +} diff --git a/apps/cli/src/index.ts b/apps/cli/src/index.ts index c03d4ac..aa29560 100644 --- a/apps/cli/src/index.ts +++ b/apps/cli/src/index.ts @@ -9,15 +9,19 @@ * in the current working directory. */ +import { ArgError, parseArgs, YES_FLAGS } from './args.js'; import { build } from './commands/build.js'; import { logs } from './commands/logs.js'; +import { reset } from './commands/reset.js'; +import { scans } from './commands/scans.js'; import { setup } from './commands/setup.js'; import { start } from './commands/start.js'; import { status } from './commands/status.js'; import { stop } from './commands/stop.js'; -import { uninstall } from './commands/uninstall.js'; -import { workspaces } from './commands/workspaces.js'; -import { getMode } from './mode.js'; +import { crash, fail, failUsage } from './errors.js'; +import { availableCommands, isHelpableCommand, printCommandHelp, START_OPTIONS } from './help.js'; +import { commandPrefix, getMode } from './mode.js'; +import { closestMatch } from './suggest.js'; import { getVersion, getVersionLine } from './version.js'; function blockSudo(): void { @@ -25,23 +29,30 @@ function blockSudo(): void { const isRoot = process.geteuid?.() === 0; if (!isSudo && !isRoot) return; + const linuxHints = + process.platform === 'linux' + ? ['Configure Docker to run without sudo first:', 'https://docs.docker.com/engine/install/linux-postinstall'] + : []; + if (isSudo) { - console.error('ERROR: Shannon must not be run with sudo.'); - console.error('Re-run this command as your normal user.'); - } else { - console.error('ERROR: Shannon must not be run as the root user.'); - console.error('Switch to a regular user account and re-run this command.'); + fail('Shannon must not be run with sudo.', 'Re-run this command as your normal user.', ...linuxHints); } - if (process.platform === 'linux') { - console.error('Configure Docker to run without sudo first:'); - console.error('https://docs.docker.com/engine/install/linux-postinstall'); - } - process.exit(1); + fail( + 'Shannon must not be run as the root user.', + 'Switch to a regular user account and re-run this command.', + ...linuxHints, + ); +} + +/** Render `start`'s flags for the global help, from the same source as `start --help`. */ +function renderStartOptions(): string { + const flagWidth = Math.max(...START_OPTIONS.map(([flag]) => flag.length)); + return START_OPTIONS.map(([flag, desc]) => ` ${flag.padEnd(flagWidth)} ${desc}`).join('\n'); } function showHelp(): void { const mode = getMode(); - const prefix = mode === 'local' ? './shannon' : 'npx @keygraph/shannon'; + const prefix = commandPrefix(); console.log(` Shannon - AI Penetration Testing Framework @@ -53,33 +64,31 @@ Usage:${ ${prefix} setup Configure credentials` } ${prefix} start --url --repo [options] Start a pentest scan - ${prefix} stop [--clean] [--yes] Stop all running scans - ${prefix} workspaces List all workspaces + ${prefix} stop [--yes] Stop one scan + ${prefix} stop --all [--yes] Stop all scans (Temporal stays up) + ${prefix} reset Stop everything and wipe all Temporal data ${prefix} logs Show a scan's live log - ${prefix} status Show running scans${ + ${prefix} status [--json] Live phase/agent progress of one scan + ${prefix} scans [--json] List completed scans and their reports${ mode === 'local' ? ` ${prefix} build [--no-cache] Build worker image` - : ` - ${prefix} uninstall [--yes] Remove ~/.shannon/ and all data` + : '' } - ${prefix} version Show version + ${prefix} version [--json] Show version ${prefix} help Show this help Options for 'start': - -u, --url Target URL (required) - -r, --repo Repository path${mode === 'local' ? ' or bare name' : ''} (required) - -c, --config Configuration file (YAML) - -o, --output Copy deliverables to this directory after run - -w, --workspace Named workspace (auto-resumes if exists) - --pipeline-testing Use minimal prompts for fast testing - --debug Preserve worker container after exit for log inspection +${renderStartOptions()} Examples: - ${prefix} start -u https://example.com -r ${mode === 'local' ? 'my-repo' : './my-repo'} + ${prefix} start -u https://example.com -r ./my-repo ${prefix} start -u https://example.com -r /path/to/repo -c config.yaml -w q1-audit ${prefix} logs q1-audit - ${prefix} stop --clean + ${prefix} stop q1-audit + ${prefix} reset + +Run '${prefix} --help' for help on a specific command. ${ mode === 'local' ? ` @@ -88,6 +97,7 @@ State directory: ./workspaces/` State directory: ~/.shannon/` } Monitor scans at http://localhost:8233 +Docs & source: https://github.com/KeygraphHQ/shannon `); } @@ -98,150 +108,166 @@ interface ParsedStartArgs { workspace?: string; output?: string; pipelineTesting: boolean; - debug: boolean; + keepContainer: boolean; + follow: boolean; } function parseStartArgs(argv: string[]): ParsedStartArgs { - let url = ''; - let repo = ''; - let config: string | undefined; - let workspace: string | undefined; - let output: string | undefined; - let pipelineTesting = false; - let debug = false; + const { flags, values } = parseArgs(argv, { + values: { + url: ['-u', '--url'], + repo: ['-r', '--repo'], + config: ['-c', '--config'], + output: ['-o', '--output'], + workspace: ['-w', '--workspace'], + }, + booleans: { + pipelineTesting: ['--pipeline-testing'], + keepContainer: ['--keep-container'], + follow: ['-f', '--follow'], + }, + }); - for (let i = 0; i < argv.length; i++) { - const arg = argv[i]; - const next = argv[i + 1]; - - switch (arg) { - case '-u': - case '--url': - if (next && !next.startsWith('-')) { - url = next; - i++; - } - break; - case '-r': - case '--repo': - if (next && !next.startsWith('-')) { - repo = next; - i++; - } - break; - case '-c': - case '--config': - if (next && !next.startsWith('-')) { - config = next; - i++; - } - break; - case '-w': - case '--workspace': - if (next && !next.startsWith('-')) { - workspace = next; - i++; - } - break; - case '-o': - case '--output': - if (next && !next.startsWith('-')) { - output = next; - i++; - } - break; - case '--pipeline-testing': - pipelineTesting = true; - break; - case '--debug': - debug = true; - break; - default: - console.error(`Unknown option: ${arg}`); - console.error(`Run "${getMode() === 'local' ? './shannon' : 'npx @keygraph/shannon'} help" for usage`); - process.exit(1); - } + const url = values.url ?? ''; + const repo = values.repo ?? ''; + if (!url || !repo) { + failUsage('--url and --repo are required', `Usage: ${commandPrefix()} start -u -r `); } - if (!url || !repo) { - console.error('ERROR: --url and --repo are required'); - console.error(`Usage: ${getMode() === 'local' ? './shannon' : 'npx @keygraph/shannon'} start -u -r `); - process.exit(1); + try { + new URL(url); + } catch { + failUsage(`invalid --url: ${url}`); } return { url, repo, - pipelineTesting, - debug, - ...(config && { config }), - ...(workspace && { workspace }), - ...(output && { output }), + pipelineTesting: !!flags.pipelineTesting, + keepContainer: !!flags.keepContainer, + follow: !!flags.follow, + ...(values.config && { config: values.config }), + ...(values.workspace && { workspace: values.workspace }), + ...(values.output && { output: values.output }), }; } // === Main Dispatch === -blockSudo(); +async function main(): Promise { + // A reader that closes early (e.g. `shannon logs my-scan | head`) makes writes + // to stdout raise EPIPE. That's normal for a piped CLI, not a crash — exit quietly + // instead of letting Node dump an unhandled-error stack trace. + process.stdout.on('error', (err: NodeJS.ErrnoException) => { + if (err.code === 'EPIPE') process.exit(0); + throw err; + }); -const args = process.argv.slice(2); -const command = args[0]; + blockSudo(); -switch (command) { - case 'start': { - const parsed = parseStartArgs(args.slice(1)); - await start({ ...parsed, version: getVersion() }); - break; + const args = process.argv.slice(2); + const command = args[0]; + const rest = args.slice(1); + + if (command === undefined || command === 'help' || command === '--help' || command === '-h') { + const topic = rest[0]; + if (topic && isHelpableCommand(topic)) { + printCommandHelp(topic); + } else { + showHelp(); + } + return; } - case 'stop': - stop(args.includes('--clean'), args.includes('--yes') || args.includes('-y')); - break; - case 'logs': { - const workspaceId = args[1]; - if (!workspaceId) { - console.error('ERROR: Workspace ID is required'); - console.error(`Usage: ${getMode() === 'local' ? './shannon' : 'npx @keygraph/shannon'} logs `); - process.exit(1); - } - logs(workspaceId); - break; + + // Reachable from any invocation: `-h`/`--help` anywhere wins over the rest of the line. + if (isHelpableCommand(command) && (rest.includes('-h') || rest.includes('--help'))) { + printCommandHelp(command); + return; } - case 'workspaces': - workspaces(getVersion()); - break; - case 'status': - status(); - break; - case 'setup': - if (getMode() === 'local') { - console.error('ERROR: setup is only available in npx mode. In local mode, use .env'); - process.exit(1); + + switch (command) { + case 'start': { + const parsed = parseStartArgs(rest); + await start({ ...parsed, version: getVersion() }); + break; } - setup(); - break; - case 'build': - build(args.includes('--no-cache'), getVersion()); - break; - case 'uninstall': - if (getMode() === 'local') { - console.error('ERROR: uninstall is only available in npx mode.'); - process.exit(1); + case 'stop': { + const { flags, positionals } = parseArgs(rest, { + booleans: { all: ['--all'], yes: YES_FLAGS }, + maxPositionals: 1, + }); + await stop({ all: !!flags.all, yes: !!flags.yes, ...(positionals[0] && { workspace: positionals[0] }) }); + break; } - uninstall(args.includes('--yes') || args.includes('-y')); - break; - case 'version': - case '--version': - case '-v': - console.log(getVersionLine()); - break; - case 'help': - case '--help': - case '-h': - case undefined: - showHelp(); - break; - default: - console.error(`Unknown command: ${command}`); - showHelp(); - process.exit(1); + case 'reset': { + // reset is all-or-nothing; a stray name likely means the user wanted `stop `. + parseArgs(rest, { + positionalHint: 'reset takes no workspace argument. To stop one scan, use: stop ', + }); + await reset(); + break; + } + case 'logs': { + const { positionals } = parseArgs(rest, { maxPositionals: 1 }); + const workspaceId = positionals[0]; + if (!workspaceId) { + failUsage('Workspace ID is required', `Usage: ${commandPrefix()} logs `); + } + logs(workspaceId); + break; + } + case 'status': { + const { flags, positionals } = parseArgs(rest, { booleans: { json: ['--json'] }, maxPositionals: 1 }); + const workspaceId = positionals[0]; + if (!workspaceId) { + failUsage('Workspace is required', `Usage: ${commandPrefix()} status [--json]`); + } + await status(workspaceId, { json: !!flags.json }); + break; + } + case 'scans': { + const { flags } = parseArgs(rest, { booleans: { json: ['--json'] } }); + scans({ json: !!flags.json }); + break; + } + case 'setup': + if (getMode() === 'local') { + fail('setup is only available in npx mode. In local mode, use .env'); + } + parseArgs(rest, {}); + await setup(); + break; + case 'build': { + const { flags } = parseArgs(rest, { booleans: { noCache: ['--no-cache'] } }); + build(!!flags.noCache, getVersion()); + break; + } + case 'version': + case '--version': + case '-v': { + const { flags } = parseArgs(rest, { booleans: { json: ['--json'] } }); + if (flags.json) { + console.log(JSON.stringify({ version: getVersion(), mode: getMode() }, null, 2)); + } else { + console.log(getVersionLine()); + } + break; + } + default: { + const prefix = commandPrefix(); + const suggestion = closestMatch(command, availableCommands()); + const hints = [ + ...(suggestion ? [`Did you mean '${suggestion}'?`] : []), + `Run '${prefix} help' to see available commands.`, + ]; + failUsage(`Unknown command: ${command}`, ...hints); + } + } } + +main().catch((err) => { + if (err instanceof ArgError) { + failUsage(err.message, `Run "${commandPrefix()} help" for usage`); + } + crash(err); +}); diff --git a/apps/cli/src/mode.ts b/apps/cli/src/mode.ts index 6cb2043..da0f3c9 100644 --- a/apps/cli/src/mode.ts +++ b/apps/cli/src/mode.ts @@ -24,6 +24,11 @@ export function isLocal(): boolean { return getMode() === 'local'; } +/** The invocation prefix for the current mode, so help and hints point at a runnable command. */ +export function commandPrefix(): string { + return getMode() === 'local' ? './shannon' : 'npx @keygraph/shannon'; +} + export function isDevMode(): boolean { return process.env.SHANNON_DEV === '1'; } diff --git a/apps/cli/src/paths.ts b/apps/cli/src/paths.ts index c26ec85..a1314fc 100644 --- a/apps/cli/src/paths.ts +++ b/apps/cli/src/paths.ts @@ -1,13 +1,27 @@ /** * Path resolution for --repo and --config arguments. * - * Local mode supports bare repo names (e.g. "my-repo" → ./repos/my-repo). - * Both modes resolve relative paths against CWD. + * Both --repo and --config are filesystem paths, absolute or relative to CWD. */ import fs from 'node:fs'; +import os from 'node:os'; import path from 'node:path'; -import { isLocal } from './mode.js'; +import { fail } from './errors.js'; + +/** + * Expand a leading `~` or `~/` to the home directory. The shell skips this in the + * `--flag=~/x` form (the tilde is not at the word start), so it must be done here. + */ +export function expandHome(inputPath: string): string { + if (inputPath === '~') { + return os.homedir(); + } + if (inputPath.startsWith('~/')) { + return path.join(os.homedir(), inputPath.slice(2)); + } + return inputPath; +} export interface MountPair { hostPath: string; @@ -47,36 +61,18 @@ export function resolveRunFile(runDir: string, filename: string): string { } /** - * Resolve --repo to absolute path and container mount. - * Dev mode: bare names (no / or . prefix) check ./repos/ first. + * Resolve --repo to an absolute path and container mount. The argument is a + * filesystem path, absolute or relative to CWD. */ export function resolveRepo(repoArg: string): MountPair { - let hostPath: string; - - if (isLocal() && !repoArg.startsWith('/') && !repoArg.startsWith('.')) { - // Bare name — check ./repos/ for backward compatibility - const barePath = path.resolve('repos', repoArg); - if (fs.existsSync(barePath)) { - hostPath = barePath; - } else { - console.error(`ERROR: Repository not found at ./repos/${repoArg}`); - console.error(''); - console.error('Place your target repository under the ./repos/ directory,'); - console.error('or pass an absolute/relative path: -r /path/to/repo'); - process.exit(1); - } - } else { - hostPath = path.resolve(repoArg); - } + const hostPath = path.resolve(expandHome(repoArg)); if (!fs.existsSync(hostPath)) { - console.error(`ERROR: Repository not found: ${hostPath}`); - process.exit(1); + fail(`Repository not found: ${hostPath}`); } if (!fs.statSync(hostPath).isDirectory()) { - console.error(`ERROR: Not a directory: ${hostPath}`); - process.exit(1); + fail(`Not a directory: ${hostPath}`); } const basename = path.basename(hostPath); @@ -90,16 +86,14 @@ export function resolveRepo(repoArg: string): MountPair { * Resolve --config to absolute path and container mount. */ export function resolveConfig(configArg: string): MountPair { - const hostPath = path.resolve(configArg); + const hostPath = path.resolve(expandHome(configArg)); if (!fs.existsSync(hostPath)) { - console.error(`ERROR: Config file not found: ${hostPath}`); - process.exit(1); + fail(`Config file not found: ${hostPath}`); } if (!fs.statSync(hostPath).isFile()) { - console.error(`ERROR: Not a file: ${hostPath}`); - process.exit(1); + fail(`Not a file: ${hostPath}`); } const basename = path.basename(hostPath); diff --git a/apps/cli/src/scan/derive.ts b/apps/cli/src/scan/derive.ts new file mode 100644 index 0000000..c3de3f2 --- /dev/null +++ b/apps/cli/src/scan/derive.ts @@ -0,0 +1,155 @@ +/** + * Pure derivation of a scan's per-agent and per-phase state from its Temporal snapshot. + * + * This is the single source of truth for "what state is each agent in" — both the + * human progress tree (render.ts) and the machine-readable snapshot (status-json.ts) + * consume it, so the two views can never disagree about whether an agent is running, + * skipped, or still pending. No glyphs, no color, no formatting live here. + */ + +import type { RunningAgent } from '../temporal-client.js'; +import { agentClass, PIPELINE, type PipelineState } from './pipeline.js'; +import type { RenderInput } from './render.js'; + +export type RunState = 'pending' | 'running' | 'completed' | 'failed' | 'skipped'; + +/** One agent's resolved state plus the raw metrics/timing a consumer needs to present it. Null metrics + * mean the value doesn't apply to the current state (e.g. duration only for completed agents). */ +export interface DerivedAgent { + readonly name: string; + readonly label: string; + readonly state: RunState; + readonly durationMs: number | null; + readonly runningElapsedMs: number | null; + readonly attempt: number | null; + readonly error?: string; +} + +export interface DerivedPhase { + readonly key: string; + readonly label: string; + readonly parallel: boolean; + readonly state: RunState; + readonly agents: readonly DerivedAgent[]; +} + +/** Terminal = anything other than an open, running execution. */ +export function isTerminal(status: string): boolean { + return status !== 'RUNNING' && status !== 'UNSPECIFIED'; +} + +function isFailedAgent(name: string, state: PipelineState | null): boolean { + return !!state && (state.failedAgent === name || state.failedPipelines.some((f) => f.vulnType === agentClass(name))); +} + +/** An agent has entered play once it is running, has metrics, or has failed. */ +function isAgentActive(name: string, state: PipelineState | null, running: Set): boolean { + return running.has(name) || !!state?.agentMetrics[name] || isFailedAgent(name, state); +} + +/** + * Resolve one agent's state. "Ran" is signalled by a metrics entry, not by + * completedAgents — the workflow lists conditionally-skipped agents (e.g. exploit + * agents when there is nothing to exploit) as completed but records no metrics for + * them. `resolved` is true once we've moved past this agent's phase (the scan is + * terminal, or a later phase is already active), at which point a metric-less, + * non-running agent is skipped rather than still pending. + */ +function agentState(name: string, state: PipelineState | null, running: Set, resolved: boolean): RunState { + if (running.has(name)) return 'running'; + if (isFailedAgent(name, state)) return 'failed'; + if (state?.agentMetrics[name]) return 'completed'; + return resolved ? 'skipped' : 'pending'; +} + +function agentError(name: string, state: PipelineState | null, byAgent: Map): string | undefined { + const failed = state?.failedPipelines.find((f) => f.vulnType === agentClass(name)); + return ( + failed?.error ?? + byAgent.get(name)?.lastFailure ?? + (state?.failedAgent === name ? (state.error ?? undefined) : undefined) + ); +} + +/** Scan wall-clock elapsed ms: recorded duration for a closed scan, live elapsed for a running one. */ +export function scanElapsedMs(input: RenderInput, now: number): number | undefined { + if (isTerminal(input.temporalStatus)) { + if (input.state?.summary) return input.state.summary.totalDurationMs; + if (input.endedAt !== undefined && input.startedAt !== undefined) return input.endedAt - input.startedAt; + return undefined; + } + return input.startedAt !== undefined ? now - input.startedAt : undefined; +} + +/** Collapse a phase's agent states into a single state for the phase line. */ +export function phaseGlyphState(states: readonly RunState[]): RunState { + if (states.some((s) => s === 'running')) return 'running'; + if (states.some((s) => s === 'failed')) return 'failed'; + if (states.every((s) => s === 'skipped')) return 'skipped'; + if (states.every((s) => s === 'completed' || s === 'skipped')) return 'completed'; + if (states.some((s) => s === 'completed')) return 'running'; + return 'pending'; +} + +/** + * Compute each agent's RunState. This is the drift-prone part shared by every view. + * + * The pipeline is sequential across phases: the last phase with any active agent is the + * frontier. Earlier phases with nothing active were skipped (e.g. exploitation when no + * class had anything to exploit), not still pending. + */ +export function deriveAgentStates(input: RenderInput): Map { + const runningSet = new Set(input.running.map((r) => r.agent)); + const terminal = isTerminal(input.temporalStatus); + + let frontier = -1; + PIPELINE.forEach((phase, idx) => { + if (phase.agents.some((a) => isAgentActive(a.name, input.state, runningSet))) frontier = idx; + }); + + const states = new Map(); + for (const [phaseIdx, phase] of PIPELINE.entries()) { + const resolved = terminal || phaseIdx < frontier; + for (const agent of phase.agents) { + states.set(agent.name, agentState(agent.name, input.state, runningSet, resolved)); + } + } + return states; +} + +/** + * Full structured view of the pipeline: every agent's state plus the raw + * metrics/timing needed to present it, and each phase's collapsed state. + */ +export function derivePipeline(input: RenderInput, now: number): DerivedPhase[] { + const states = deriveAgentStates(input); + const byAgent = new Map(input.running.map((r) => [r.agent, r])); + + return PIPELINE.map((phase) => { + const agents = phase.agents.map((a): DerivedAgent => { + const state = states.get(a.name) ?? 'pending'; + const metrics = input.state?.agentMetrics[a.name]; + const runner = byAgent.get(a.name); + const error = agentError(a.name, input.state, byAgent); + return { + name: a.name, + label: a.label, + state, + durationMs: state === 'completed' && metrics ? metrics.durationMs : null, + runningElapsedMs: state === 'running' && runner?.startedAt !== undefined ? now - runner.startedAt : null, + attempt: state === 'running' && runner ? runner.attempt : null, + ...(error !== undefined && { error }), + }; + }); + + return { + key: phase.key, + label: phase.label, + parallel: phase.parallel, + state: phaseGlyphState(agents.map((ag) => ag.state)), + agents, + }; + }); +} + +export { agentError }; diff --git a/apps/cli/src/scan/pipeline.ts b/apps/cli/src/scan/pipeline.ts new file mode 100644 index 0000000..fe877ed --- /dev/null +++ b/apps/cli/src/scan/pipeline.ts @@ -0,0 +1,123 @@ +/** + * Static description of the Shannon scan pipeline, plus the worker types the CLI + * reads back from Temporal. + * + * The CLI cannot import from the worker package, so this mirrors it. Keep in sync with: + * - apps/worker/src/types/agents.ts (agent names / ordering) + * - apps/worker/src/session-manager.ts (phase membership) + * - apps/worker/src/temporal/activities.ts (the run*Agent activity names → `activityType`) + * - apps/worker/src/temporal/shared.ts (PipelineState / PipelineSummary) + * - apps/worker/src/types/metrics.ts (AgentMetrics) + */ + +export interface AgentSpec { + /** Canonical agent name as it appears in PipelineState.completedAgents / agentMetrics. */ + readonly name: string; + /** Short label for the progress tree. */ + readonly label: string; + /** Temporal activity type name — how a running agent shows up in pendingActivities. */ + readonly activityType: string; +} + +export interface PhaseSpec { + readonly key: string; + readonly label: string; + readonly parallel: boolean; + readonly agents: readonly AgentSpec[]; +} + +/** The pipeline phases in execution order, each with its agents. */ +export const PIPELINE: readonly PhaseSpec[] = [ + { + // Preflight login check. Only authenticated scans record metrics here; a non-auth scan + // records none, so it renders as skipped — like Exploitation when nothing is exploitable. + key: 'auth-validation', + label: 'Authentication', + parallel: false, + agents: [{ name: 'validate-authentication', label: 'auth', activityType: 'runAuthenticationValidation' }], + }, + { + key: 'pre-recon', + label: 'Pre-Recon', + parallel: false, + agents: [{ name: 'pre-recon', label: 'pre-recon', activityType: 'runPreReconAgent' }], + }, + { + key: 'recon', + label: 'Recon', + parallel: false, + agents: [{ name: 'recon', label: 'recon', activityType: 'runReconAgent' }], + }, + { + key: 'vulnerability-analysis', + label: 'Vulnerability Analysis', + parallel: true, + agents: [ + { name: 'injection-vuln', label: 'injection', activityType: 'runInjectionVulnAgent' }, + { name: 'xss-vuln', label: 'xss', activityType: 'runXssVulnAgent' }, + { name: 'auth-vuln', label: 'auth', activityType: 'runAuthVulnAgent' }, + { name: 'ssrf-vuln', label: 'ssrf', activityType: 'runSsrfVulnAgent' }, + { name: 'authz-vuln', label: 'authz', activityType: 'runAuthzVulnAgent' }, + ], + }, + { + key: 'exploitation', + label: 'Exploitation', + parallel: true, + agents: [ + { name: 'injection-exploit', label: 'injection', activityType: 'runInjectionExploitAgent' }, + { name: 'xss-exploit', label: 'xss', activityType: 'runXssExploitAgent' }, + { name: 'auth-exploit', label: 'auth', activityType: 'runAuthExploitAgent' }, + { name: 'ssrf-exploit', label: 'ssrf', activityType: 'runSsrfExploitAgent' }, + { name: 'authz-exploit', label: 'authz', activityType: 'runAuthzExploitAgent' }, + ], + }, + { + key: 'reporting', + label: 'Reporting', + parallel: false, + agents: [{ name: 'report', label: 'report', activityType: 'runReportAgent' }], + }, +]; + +/** Temporal activity type name → canonical agent name, for mapping pendingActivities. */ +export const ACTIVITY_TO_AGENT: Readonly> = Object.fromEntries( + PIPELINE.flatMap((phase) => phase.agents.map((agent) => [agent.activityType, agent.name])), +); + +/** The vuln/exploit class of an agent (e.g. "authz-vuln" → "authz"), for failedPipelines matching. */ +export function agentClass(name: string): string { + return name.replace(/-(vuln|exploit)$/, ''); +} + +// === Worker types read back from Temporal (mirror of shared.ts / metrics.ts) === + +export interface AgentMetrics { + readonly durationMs: number; + readonly costUsd: number | null; + readonly numTurns: number | null; + readonly model?: string; + readonly skipped?: boolean; +} + +export interface PipelineSummary { + readonly totalCostUsd: number; + readonly totalDurationMs: number; // Wall-clock (end - start) + readonly totalTurns: number; + readonly agentCount: number; +} + +export type PipelineStatus = 'running' | 'completed' | 'failed' | 'cancelled' | 'partial'; + +export interface PipelineState { + readonly status: PipelineStatus; + readonly currentPhase: string | null; + readonly currentAgent: string | null; + readonly completedAgents: string[]; + readonly failedPipelines: { vulnType: string; error: string }[]; + readonly failedAgent: string | null; + readonly error: string | null; + readonly startTime: number; + readonly agentMetrics: Record; + readonly summary: PipelineSummary | null; +} diff --git a/apps/cli/src/scan/render.ts b/apps/cli/src/scan/render.ts new file mode 100644 index 0000000..605b874 --- /dev/null +++ b/apps/cli/src/scan/render.ts @@ -0,0 +1,248 @@ +/** + * Renders a scan's Temporal state into the terminal progress tree. + * + * The same PipelineState drives both the live view (from the getProgress query) and + * the final view (from the workflow result); the running-agents overlay (from + * pendingActivities) supplies the in-flight set and retry counts the state lacks. + * Colors and Unicode glyphs are gated by the caller so the frame degrades off a TTY. + */ + +import { BOLD, DIM, GOLD, paint, RED, YELLOW } from '../colors.js'; +import { commandPrefix } from '../mode.js'; +import type { RunningAgent } from '../temporal-client.js'; +import { agentError, deriveAgentStates, isTerminal, phaseGlyphState, type RunState, scanElapsedMs } from './derive.js'; +import { PIPELINE, type PipelineState } from './pipeline.js'; + +export interface RenderInput { + readonly workspace: string; + /** Temporal workflow id backing this scan (differs from workspace on a resume); used for the dashboard link. */ + readonly workflowId?: string; + /** Temporal WorkflowExecutionStatusName: RUNNING | COMPLETED | FAILED | CANCELLED | TERMINATED | … */ + readonly temporalStatus: string; + /** Progress (live) or result (terminal). Null when unavailable, e.g. a hard failure with no result. */ + readonly state: PipelineState | null; + readonly running: readonly RunningAgent[]; + readonly startedAt?: number; + readonly endedAt?: number; + /** Failure text when a failed scan has no readable state. */ + readonly failureMessage?: string; +} + +export interface RenderOptions { + readonly now: number; + readonly color: boolean; + readonly unicode: boolean; + /** True for the live view (adds a watch footer); false for the final/one-shot frame. */ + readonly live: boolean; + /** Animation tick — advances the running-agent spinner. Ignored for static frames. */ + readonly frame: number; +} + +const COLORS = { + red: RED, + gold: GOLD, + yellow: YELLOW, + dim: DIM, + bold: BOLD, +} as const; + +// === Formatting === + +function formatDuration(ms: number): string { + const seconds = Math.max(0, Math.floor(ms / 1000)); + const hours = Math.floor(seconds / 3600); + const minutes = Math.floor((seconds % 3600) / 60); + const secs = seconds % 60; + + if (hours > 0) return `${hours}h ${minutes}m`; + if (minutes > 0) return `${minutes}m ${secs}s`; + return `${secs}s`; +} + +function truncate(text: string, max: number): string { + const flat = text.replace(/\s+/g, ' ').trim(); + return flat.length <= max ? flat : `${flat.slice(0, max - 1)}…`; +} + +/** Temporal Web UI, published by compose on 8233; deep-links to the workflow when its id is known. */ +function temporalDashboardUrl(workflowId: string | undefined): string { + const base = 'http://localhost:8233'; + return workflowId ? `${base}/namespaces/default/workflows/${workflowId}` : base; +} + +// === Glyphs & status === + +const GLYPH_UNICODE: Record = { + pending: '○', + running: '⟳', + completed: '●', + failed: '✗', + skipped: '·', +}; +const GLYPH_ASCII: Record = { + pending: '.', + running: '>', + completed: '+', + failed: 'x', + skipped: '-', +}; +const STATE_COLOR: Record = { + pending: COLORS.dim, + running: COLORS.gold, + completed: COLORS.gold, + failed: COLORS.red, + skipped: COLORS.dim, +}; + +/** Braille spinner frames for running agents — the clack loader style. */ +const SPINNER_FRAMES = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'] as const; + +function glyph(state: RunState, opts: RenderOptions): string { + if (state === 'running' && opts.unicode) { + const spin = SPINNER_FRAMES[opts.frame % SPINNER_FRAMES.length] ?? SPINNER_FRAMES[0]; + return paint(spin, STATE_COLOR.running, opts.color); + } + const symbol = opts.unicode ? GLYPH_UNICODE[state] : GLYPH_ASCII[state]; + return paint(symbol, STATE_COLOR[state], opts.color); +} + +/** Badge text + color for the scan as a whole, preferring the workflow's own status when known. */ +function statusBadge(input: RenderInput, opts: RenderOptions): string { + const workflowStatus = input.state?.status; + if (!isTerminal(input.temporalStatus)) return paint('running', COLORS.gold, opts.color); + if (workflowStatus === 'partial') return paint('partial', COLORS.yellow, opts.color); + if (input.temporalStatus === 'COMPLETED') return paint('completed', COLORS.gold, opts.color); + if (input.temporalStatus === 'TERMINATED') return paint('stopped', COLORS.yellow, opts.color); + if (input.temporalStatus === 'CANCELLED' || input.temporalStatus === 'CANCELED') { + return paint('cancelled', COLORS.yellow, opts.color); + } + if (input.temporalStatus === 'TIMED_OUT') return paint('timed out', COLORS.red, opts.color); + return paint('FAILED', COLORS.red, opts.color); +} + +// === Line builders === + +function agentMeta( + state: RunState, + metrics: { durationMs: number } | undefined, + runner: RunningAgent | undefined, + error: string | undefined, + opts: RenderOptions, +): string { + if (state === 'completed') { + const duration = metrics?.durationMs != null ? formatDuration(metrics.durationMs) : 'done'; + return paint(duration, COLORS.dim, opts.color); + } + if (state === 'running') { + const parts = ['running']; + if (runner?.startedAt !== undefined) parts.push(formatDuration(opts.now - runner.startedAt)); + if (runner && runner.attempt > 1) parts.push(`retry ${runner.attempt}`); + return paint(parts.join(' · '), COLORS.gold, opts.color); + } + if (state === 'failed') { + const detail = error ? ` · ${truncate(error, 46)}` : ''; + return paint(`failed${detail}`, COLORS.red, opts.color); + } + if (state === 'skipped') return paint('skipped', COLORS.dim, opts.color); + return paint('queued', COLORS.dim, opts.color); +} + +function phaseMeta(states: readonly RunState[], inPlay: number, parallel: boolean, opts: RenderOptions): string { + if (states.every((s) => s === 'pending')) return paint('pending', COLORS.dim, opts.color); + if (states.every((s) => s === 'skipped')) return paint('skipped', COLORS.dim, opts.color); + if (states.some((s) => s === 'failed') && !states.some((s) => s === 'running')) { + return paint('failed', COLORS.red, opts.color); + } + if (!parallel) return ''; + const done = states.filter((s) => s === 'completed').length; + const allDone = states.every((s) => s === 'completed' || s === 'skipped'); + return paint(`${done}/${inPlay} done`, allDone ? COLORS.gold : COLORS.dim, opts.color); +} + +/** Render the full progress frame as one string (no trailing newline). */ +export function renderScan(input: RenderInput, opts: RenderOptions): string { + const byAgent = new Map(input.running.map((r) => [r.agent, r])); + const stateMap = deriveAgentStates(input); + const lines: string[] = ['', ...headerLines(input, opts), '']; + + const metaFor = (name: string, state: RunState): string => + agentMeta(state, input.state?.agentMetrics[name], byAgent.get(name), agentError(name, input.state, byAgent), opts); + // Only agents that have actually entered play are shown; pending/skipped ones stay hidden. + const inPlay = (s: RunState): boolean => s === 'running' || s === 'completed' || s === 'failed'; + + for (const phase of PIPELINE) { + const states = phase.agents.map((a) => stateMap.get(a.name) ?? 'pending'); + const playing = states.filter(inPlay).length; + const phaseRunState: RunState = phaseGlyphState(states); + + // A single-agent phase carries that agent's own duration/cost on the phase line once it + // starts; a parallel phase gets a "k/N done" summary over the agents in play. + const first = phase.agents[0]; + const firstState = states[0]; + const phaseMetaStr = + !phase.parallel && first && firstState && inPlay(firstState) + ? metaFor(first.name, firstState) + : phaseMeta(states, playing, phase.parallel, opts); + lines.push(` ${glyph(phaseRunState, opts)} ${phase.label.padEnd(26)}${phaseMetaStr}`); + + if (!phase.parallel) continue; + for (let i = 0; i < phase.agents.length; i++) { + const agent = phase.agents[i]; + const state = states[i]; + if (!agent || !state || !inPlay(state)) continue; + lines.push(` ${glyph(state, opts)} ${agent.label.padEnd(18)}${metaFor(agent.name, state)}`); + } + } + + lines.push(...footerLines(input, opts)); + return lines.join('\n'); +} + +function headerLines(input: RenderInput, opts: RenderOptions): string[] { + const elapsedMs = scanElapsedMs(input, opts.now); + const meta = [statusBadge(input, opts), elapsedMs !== undefined ? formatDuration(elapsedMs) : '—'].join(' · '); + return [` ${paint('Scan:', COLORS.bold, opts.color)} ${input.workspace.padEnd(22)} ${meta}`]; +} + +/** Aligned label column for the footer's Logs / Temporal rows. */ +const FOOTER_LABEL_WIDTH = 12; + +/** A thin rule that sets the footer apart from the phase list above it. */ +function footerDivider(opts: RenderOptions): string { + return paint(` ${(opts.unicode ? '─' : '-').repeat(60)}`, COLORS.dim, opts.color); +} + +/** One footer row: an accent-colored label in a fixed column, then its value in the default color. */ +function footerRow(label: string, value: string, opts: RenderOptions): string { + return ` ${paint(label.padEnd(FOOTER_LABEL_WIDTH), COLORS.gold, opts.color)}${value}`; +} + +function footerLines(input: RenderInput, opts: RenderOptions): string[] { + const prefix = commandPrefix(); + + if (isTerminal(input.temporalStatus) && input.state?.summary) { + const wall = formatDuration(input.state.summary.totalDurationMs); + return ['', ` Time Taken ${wall}`]; + } + + const logsValue = `${prefix} logs ${input.workspace}`; + const temporalValue = temporalDashboardUrl(input.workflowId); + + if (isTerminal(input.temporalStatus)) { + const reason = input.failureMessage ?? input.state?.error ?? 'no result recorded'; + return [ + footerDivider(opts), + paint( + ` ${input.temporalStatus === 'TERMINATED' ? 'Stopped' : 'Ended'} — ${truncate(reason, 240)}`, + COLORS.dim, + opts.color, + ), + footerRow('Logs', logsValue, opts), + footerRow('Temporal', temporalValue, opts), + ]; + } + + const lines = [footerDivider(opts), footerRow('Logs', logsValue, opts), footerRow('Temporal', temporalValue, opts)]; + if (opts.live) lines.push('', paint(' Ctrl-C stops watching — the scan keeps running.', COLORS.dim, opts.color)); + return lines; +} diff --git a/apps/cli/src/scan/status-json.ts b/apps/cli/src/scan/status-json.ts new file mode 100644 index 0000000..4229758 --- /dev/null +++ b/apps/cli/src/scan/status-json.ts @@ -0,0 +1,68 @@ +/** + * Machine-readable snapshot of one scan, for `shannon status --json`. + * + * A point-in-time view built from the same derivation the human progress tree uses + * (derive.ts), so the JSON and the rendered tree can never disagree about an agent's + * state. One invocation is one snapshot — callers that want to track progress poll it. + */ + +import type { DerivedPhase } from './derive.js'; +import { derivePipeline, isTerminal, scanElapsedMs } from './derive.js'; +import type { RenderInput } from './render.js'; + +/** Coarse scan status token, mirroring the human status badge in machine-friendly form. */ +export type ScanStatus = 'running' | 'completed' | 'partial' | 'failed' | 'stopped' | 'cancelled' | 'timed_out'; + +export interface StatusJson { + readonly workspace: string; + /** Temporal workflow id backing this scan (differs from workspace on a resume). */ + readonly workflowId?: string; + /** Coarse outcome: `running` until the scan closes, then its terminal status. */ + readonly status: ScanStatus; + /** Raw Temporal WorkflowExecutionStatusName, for callers that need the source status. */ + readonly temporalStatus: string; + /** Wall-clock elapsed ms (live for a running scan, final for a closed one), or null when unknown. */ + readonly elapsedMs: number | null; + readonly startedAt?: string; + readonly endedAt?: string; + /** Failure text when a failed scan left no readable state. */ + readonly failureMessage?: string; + readonly phases: readonly DerivedPhase[]; +} + +/** Map the raw Temporal status (and workflow status) onto the coarse machine token. */ +function deriveStatus(input: RenderInput): ScanStatus { + if (!isTerminal(input.temporalStatus)) return 'running'; + if (input.state?.status === 'partial') return 'partial'; + + switch (input.temporalStatus) { + case 'COMPLETED': + return 'completed'; + case 'TERMINATED': + return 'stopped'; + case 'CANCELLED': + case 'CANCELED': + return 'cancelled'; + case 'TIMED_OUT': + return 'timed_out'; + default: + return 'failed'; + } +} + +/** Build the JSON snapshot for a scan at instant `now`. */ +export function toStatusJson(input: RenderInput, now: number): StatusJson { + const elapsedMs = scanElapsedMs(input, now); + + return { + workspace: input.workspace, + ...(input.workflowId !== undefined && { workflowId: input.workflowId }), + status: deriveStatus(input), + temporalStatus: input.temporalStatus, + elapsedMs: elapsedMs ?? null, + ...(input.startedAt !== undefined && { startedAt: new Date(input.startedAt).toISOString() }), + ...(input.endedAt !== undefined && { endedAt: new Date(input.endedAt).toISOString() }), + ...(input.failureMessage !== undefined && { failureMessage: input.failureMessage }), + phases: derivePipeline(input, now), + }; +} diff --git a/apps/cli/src/session.ts b/apps/cli/src/session.ts new file mode 100644 index 0000000..cbee128 --- /dev/null +++ b/apps/cli/src/session.ts @@ -0,0 +1,26 @@ +/** + * Workspace → Temporal workflow-id resolution. + * + * A workspace name is not always its workflow id: a fresh scan's id equals the + * workspace name, but each resume spawns a new workflow (`_resume_`). + * The workspace's session.json records the authoritative id — the latest resume + * attempt, or the original — so commands that query Temporal (status, stop) resolve + * through here instead of assuming the name is the id. + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import { getWorkspacesDir } from './home.js'; +import { resolveRunFile } from './paths.js'; + +/** Latest workflow id recorded for a workspace: last resume attempt, else the original. */ +export function resolveWorkflowId(workspace: string): string | undefined { + const sessionPath = resolveRunFile(path.join(getWorkspacesDir(), workspace), 'session.json'); + try { + const session = JSON.parse(fs.readFileSync(sessionPath, 'utf-8')); + const resumeAttempts: { workflowId?: string }[] = session.session?.resumeAttempts ?? []; + return resumeAttempts.at(-1)?.workflowId ?? session.session?.originalWorkflowId ?? undefined; + } catch { + return undefined; + } +} diff --git a/apps/cli/src/splash.ts b/apps/cli/src/splash.ts index 1b41b61..6d3955a 100644 --- a/apps/cli/src/splash.ts +++ b/apps/cli/src/splash.ts @@ -5,50 +5,73 @@ import { supportsColor } from './tty.js'; +/** SHANNON wordmark. Block glyphs take the row fill; box-drawing strokes take the deeper edge shade. */ +const SHANNON = [ + '███████╗██╗ ██╗ █████╗ ███╗ ██╗███╗ ██╗ ██████╗ ███╗ ██╗', + '██╔════╝██║ ██║██╔══██╗████╗ ██║████╗ ██║██╔═══██╗████╗ ██║', + '███████╗███████║███████║██╔██╗ ██║██╔██╗ ██║██║ ██║██╔██╗ ██║', + '╚════██║██╔══██║██╔══██║██║╚██╗██║██║╚██╗██║██║ ██║██║╚██╗██║', + '███████║██║ ██║██║ ██║██║ ╚████║██║ ╚████║╚██████╔╝██║ ╚████║', + '╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═══╝╚═╝ ╚═══╝ ╚═════╝ ╚═╝ ╚═══╝', +]; + +/** + * Sunset ramp, yellow at the top row down to burnt orange at the base. + * Wordmark row i is filled with stop i and edged with stop i + 1, so the + * box-drawing strokes read as a shadow one shade deeper than their row. + * `xterm` is the 256-color approximation for terminals without 24-bit color. + */ +const SUNSET: ReadonlyArray<{ rgb: readonly [number, number, number]; xterm: number }> = [ + { rgb: [247, 203, 45], xterm: 220 }, + { rgb: [246, 182, 38], xterm: 220 }, + { rgb: [245, 160, 32], xterm: 214 }, + { rgb: [242, 141, 28], xterm: 214 }, + { rgb: [238, 121, 24], xterm: 208 }, + { rgb: [231, 100, 21], xterm: 208 }, + { rgb: [222, 82, 19], xterm: 202 }, +]; + export function displaySplash(version?: string): void { const color = supportsColor(); - const GOLD = color ? '\x1b[38;2;244;197;66m' : ''; - const CYAN = color ? '\x1b[36;1m' : ''; - const WHITE = color ? '\x1b[1;37m' : ''; - const GRAY = color ? '\x1b[0;37m' : ''; - const YELLOW = color ? '\x1b[1;33m' : ''; + const truecolor = color && /truecolor|24bit/i.test(process.env.COLORTERM ?? ''); const RESET = color ? '\x1b[0m' : ''; + const WHITE = color ? '\x1b[1;97m' : ''; + const GRAY = color ? '\x1b[0;37m' : ''; + const DIM = color ? '\x1b[90m' : ''; - const B = `${CYAN}\u2551${RESET}`; - const S67 = ' '.repeat(67); - const HR = '\u2550'.repeat(67); + const ramp = SUNSET.map(({ rgb: [r, g, b], xterm }) => { + if (!color) return ''; + return truecolor ? `\x1b[38;2;${r};${g};${b}m` : `\x1b[38;5;${xterm}m`; + }); + + /** Color one wordmark row, emitting an escape only where the run changes. Spaces stay unpainted. */ + const paint = (row: string, fill: string, edge: string): string => { + if (!color) return row; + let out = ''; + let open = ''; + for (const ch of row) { + const want = ch === ' ' ? '' : ch === '█' ? fill : edge; + if (want !== open) { + if (open) out += RESET; + out += want; + open = want; + } + out += ch; + } + return open ? out + RESET : out; + }; const lines = [ '', - ` ${CYAN}\u2554${HR}\u2557${RESET}`, - ` ${B}${S67}${B}`, - ` ${B} ${GOLD}\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2557\u2588\u2588\u2557 \u2588\u2588\u2557 \u2588\u2588\u2588\u2588\u2588\u2557 \u2588\u2588\u2588\u2557 \u2588\u2588\u2557\u2588\u2588\u2588\u2557 \u2588\u2588\u2557 \u2588\u2588\u2588\u2588\u2588\u2588\u2557 \u2588\u2588\u2588\u2557 \u2588\u2588\u2557${RESET} ${B}`, - ` ${B} ${GOLD}\u2588\u2588\u2554\u2550\u2550\u2550\u2550\u255D\u2588\u2588\u2551 \u2588\u2588\u2551\u2588\u2588\u2554\u2550\u2550\u2588\u2588\u2557\u2588\u2588\u2588\u2588\u2557 \u2588\u2588\u2551\u2588\u2588\u2588\u2588\u2557 \u2588\u2588\u2551\u2588\u2588\u2554\u2550\u2550\u2550\u2588\u2588\u2557\u2588\u2588\u2588\u2588\u2557 \u2588\u2588\u2551${RESET} ${B}`, - ` ${B} ${GOLD}\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2557\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2551\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2551\u2588\u2588\u2554\u2588\u2588\u2557 \u2588\u2588\u2551\u2588\u2588\u2554\u2588\u2588\u2557 \u2588\u2588\u2551\u2588\u2588\u2551 \u2588\u2588\u2551\u2588\u2588\u2554\u2588\u2588\u2557 \u2588\u2588\u2551${RESET} ${B}`, - ` ${B} ${GOLD}\u255A\u2550\u2550\u2550\u2550\u2588\u2588\u2551\u2588\u2588\u2554\u2550\u2550\u2588\u2588\u2551\u2588\u2588\u2554\u2550\u2550\u2588\u2588\u2551\u2588\u2588\u2551\u255A\u2588\u2588\u2557\u2588\u2588\u2551\u2588\u2588\u2551\u255A\u2588\u2588\u2557\u2588\u2588\u2551\u2588\u2588\u2551 \u2588\u2588\u2551\u2588\u2588\u2551\u255A\u2588\u2588\u2557\u2588\u2588\u2551${RESET} ${B}`, - ` ${B} ${GOLD}\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2551\u2588\u2588\u2551 \u2588\u2588\u2551\u2588\u2588\u2551 \u2588\u2588\u2551\u2588\u2588\u2551 \u255A\u2588\u2588\u2588\u2588\u2551\u2588\u2588\u2551 \u255A\u2588\u2588\u2588\u2588\u2551\u255A\u2588\u2588\u2588\u2588\u2588\u2588\u2554\u255D\u2588\u2588\u2551 \u255A\u2588\u2588\u2588\u2588\u2551${RESET} ${B}`, - ` ${B} ${GOLD}\u255A\u2550\u2550\u2550\u2550\u2550\u2550\u255D\u255A\u2550\u255D \u255A\u2550\u255D\u255A\u2550\u255D \u255A\u2550\u255D\u255A\u2550\u255D \u255A\u2550\u2550\u2550\u255D\u255A\u2550\u255D \u255A\u2550\u2550\u2550\u255D \u255A\u2550\u2550\u2550\u2550\u2550\u255D \u255A\u2550\u255D \u255A\u2550\u2550\u2550\u255D${RESET} ${B}`, - ` ${B}${S67}${B}`, - ` ${B} ${CYAN}\u2554\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2557${RESET} ${B}`, - ` ${B} ${CYAN}\u2551${RESET} ${WHITE}AI Penetration Testing Framework${RESET} ${CYAN}\u2551${RESET} ${B}`, - ` ${B} ${CYAN}\u255A\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255D${RESET} ${B}`, - ` ${B}${S67}${B}`, - ]; - - if (version) { - const verStr = `v${version}`; - const verPadLeft = Math.floor((67 - verStr.length) / 2); - const verPadRight = 67 - verStr.length - verPadLeft; - lines.push(` ${B}${' '.repeat(verPadLeft)}${GRAY}${verStr}${RESET}${' '.repeat(verPadRight)}${B}`); - } - - lines.push( - ` ${B}${S67}${B}`, - ` ${B} ${YELLOW}\uD83D\uDD10 DEFENSIVE SECURITY ONLY \uD83D\uDD10${RESET} ${B}`, - ` ${B}${S67}${B}`, - ` ${CYAN}\u255A${HR}\u255D${RESET}`, + ` ${WHITE}Keygraph${RESET}${version ? ` ${DIM}v${version}${RESET}` : ''}`, '', - ); + ...SHANNON.map((row, i) => ` ${paint(row, ramp[i] ?? '', ramp[i + 1] ?? '')}`), + '', + ` ${WHITE}AI Pentester for Web Apps and APIs${RESET}`, + '', + ` ${GRAY}-Authorized Security Testing Only-${RESET}`, + '', + ]; console.log(lines.join('\n')); } diff --git a/apps/cli/src/suggest.ts b/apps/cli/src/suggest.ts new file mode 100644 index 0000000..d80f17c --- /dev/null +++ b/apps/cli/src/suggest.ts @@ -0,0 +1,58 @@ +/** + * "Did you mean?" suggestions for mistyped commands and flags. + * + * A single Levenshtein-based matcher powers both the unknown-command path in the + * dispatcher and the unknown-option path in `parseArgs`, so a typo like `statsu` + * or `--workspce` points the user at the closest real name instead of just failing. + */ + +/** Levenshtein edit distance between two strings (insertions, deletions, substitutions). */ +export function editDistance(a: string, b: string): number { + if (a.length === 0) return b.length; + if (b.length === 0) return a.length; + + // Rolling single row; `diagonal` and `above` carry the two neighbours a full grid would. + const row = Array.from({ length: b.length + 1 }, (_, j) => j); + + for (let i = 1; i <= a.length; i++) { + let diagonal = row[0] as number; + row[0] = i; + for (let j = 1; j <= b.length; j++) { + const above = row[j] as number; + const cost = a[i - 1] === b[j - 1] ? 0 : 1; + row[j] = Math.min(above + 1, (row[j - 1] as number) + 1, diagonal + cost); + diagonal = above; + } + } + return row[b.length] as number; +} + +/** + * The candidate closest to `input`, or undefined if none is near enough. + * + * A prefix match ("stat" -> "status") wins first; otherwise the lowest edit + * distance within a length-scaled threshold, so unrelated words don't match. + */ +export function closestMatch(input: string, candidates: readonly string[]): string | undefined { + if (input.length >= 2) { + const prefix = candidates.find((candidate) => candidate.startsWith(input)); + if (prefix) return prefix; + } + + let best: string | undefined; + let bestDistance = Number.POSITIVE_INFINITY; + for (const candidate of candidates) { + if (candidate.length <= 3) continue; + + const distance = editDistance(input, candidate); + if (distance < bestDistance) { + bestDistance = distance; + best = candidate; + } + } + + if (best === undefined) return undefined; + + const threshold = Math.max(2, Math.floor(best.length / 3)); + return bestDistance <= threshold ? best : undefined; +} diff --git a/apps/cli/src/temporal-client.ts b/apps/cli/src/temporal-client.ts new file mode 100644 index 0000000..c7f3fdc --- /dev/null +++ b/apps/cli/src/temporal-client.ts @@ -0,0 +1,126 @@ +/** + * Thin Temporal client for reading one scan's state. + * + * A running scan is queried live (getProgress) and read via pendingActivities for + * the in-flight agents; a closed scan is read once from its result. Everything goes + * straight to the frontend on 127.0.0.1:7233 — the gRPC port the compose file + * publishes — so this needs Temporal up, but no worker of its own. + */ + +import { Client, Connection, WorkflowFailedError, WorkflowNotFoundError } from '@temporalio/client'; +import { ACTIVITY_TO_AGENT, type PipelineState } from './scan/pipeline.js'; + +const ADDRESS = '127.0.0.1:7233'; +const NAMESPACE = 'default'; + +export interface RunningAgent { + readonly agent: string; + readonly attempt: number; + readonly startedAt?: number; + readonly lastFailure?: string; +} + +/** Convert a proto ITimestamp (seconds is a Long) to epoch millis. */ +function timestampMs( + ts: { seconds?: { toString(): string } | number | null; nanos?: number | null } | null, +): number | undefined { + const seconds = ts?.seconds; + if (seconds == null) return undefined; + const secNum = typeof seconds === 'number' ? seconds : Number(seconds.toString()); + return secNum * 1000 + (ts?.nanos ?? 0) / 1e6; +} + +export interface ScanDescription { + /** WorkflowExecutionStatusName: RUNNING | COMPLETED | FAILED | CANCELLED | TERMINATED | TIMED_OUT | … */ + readonly status: string; + readonly startedAt?: number; + readonly closedAt?: number; + readonly runningAgents: readonly RunningAgent[]; +} + +export type TerminalOutcome = + | { readonly kind: 'success'; readonly state: PipelineState } + | { readonly kind: 'failed'; readonly message: string }; + +let clientPromise: Promise | null = null; + +function getClient(): Promise { + if (!clientPromise) { + clientPromise = Connection.connect({ address: ADDRESS }).then( + (connection) => new Client({ connection, namespace: NAMESPACE }), + ); + } + return clientPromise; +} + +/** Describe a scan: status, timing, and the agents currently running (from pendingActivities). Null if not found. */ +export async function describeScan(workflowId: string): Promise { + const client = await getClient(); + try { + const desc = await client.workflow.getHandle(workflowId).describe(); + + const runningAgents: RunningAgent[] = []; + for (const pending of desc.raw.pendingActivities ?? []) { + const agent = ACTIVITY_TO_AGENT[pending.activityType?.name ?? '']; + if (!agent) continue; + const lastFailure = pending.lastFailure?.message; + const startedAt = timestampMs(pending.scheduledTime ?? pending.lastStartedTime ?? null); + runningAgents.push({ + agent, + attempt: pending.attempt ?? 1, + ...(startedAt !== undefined ? { startedAt } : {}), + ...(lastFailure ? { lastFailure } : {}), + }); + } + + return { + status: desc.status.name, + runningAgents, + ...(desc.startTime ? { startedAt: desc.startTime.getTime() } : {}), + ...(desc.closeTime ? { closedAt: desc.closeTime.getTime() } : {}), + }; + } catch (err) { + if (err instanceof WorkflowNotFoundError) return null; + throw err; + } +} + +/** Live progress of a running scan via the getProgress query. Null if the query can't be served (no worker). */ +export async function queryProgress(workflowId: string): Promise { + const client = await getClient(); + try { + return await client.workflow.getHandle(workflowId).query('getProgress'); + } catch { + // The query needs a live worker; a just-closed scan may have none. Caller falls back to the result. + return null; + } +} + +/** + * Deepest message in a Temporal failure's cause chain — the real reason nested under generic + * wrappers (WorkflowFailedError → ActivityFailure → ApplicationFailure). Covers failed, cancelled, + * and terminated alike. Mirrors the SDK's `rootCause` (only exported from @temporalio/common). + */ +function rootFailureMessage(err: WorkflowFailedError): string { + let message = err.message; + let cause: unknown = err.cause; + while (cause instanceof Error && cause.message) { + message = cause.message; + cause = cause.cause; + } + return message; +} + +/** Final state of a closed scan: success carries the full PipelineState, failure carries the message. */ +export async function getTerminalOutcome(workflowId: string): Promise { + const client = await getClient(); + try { + const state = (await client.workflow.getHandle(workflowId).result()) as PipelineState; + return { kind: 'success', state }; + } catch (err) { + if (err instanceof WorkflowFailedError) { + return { kind: 'failed', message: rootFailureMessage(err) }; + } + throw err; + } +} diff --git a/apps/cli/src/tty.ts b/apps/cli/src/tty.ts index 7c48c79..8bc73d8 100644 --- a/apps/cli/src/tty.ts +++ b/apps/cli/src/tty.ts @@ -3,6 +3,8 @@ * whether the user can be prompted interactively. */ +import { fail } from './errors.js'; + /** True when stdout is a real terminal — safe for color, cursor moves, and spinners. */ export function stdoutIsTerminal(): boolean { return !!process.stdout.isTTY; @@ -28,7 +30,5 @@ export function supportsColor(): boolean { /** Exit with a clear error when an interactive-only command has no terminal, instead of hanging on a prompt. */ export function requireInteractive(command: string, alternative: string): void { if (isInteractive()) return; - console.error(`ERROR: '${command}' needs an interactive terminal.`); - console.error(alternative); - process.exit(1); + fail(`'${command}' needs an interactive terminal.`, alternative); } diff --git a/apps/cli/src/ui.ts b/apps/cli/src/ui.ts new file mode 100644 index 0000000..7a24087 --- /dev/null +++ b/apps/cli/src/ui.ts @@ -0,0 +1,60 @@ +/** + * Terminal status output for long-running steps. + * + * Commands are run with their output captured rather than inherited, so raw docker + * plumbing never floods the terminal. Progress is shown with a `@clack/prompts` + * spinner. On failure the captured output is printed so the error stays visible + * instead of being swallowed. + */ + +import { spawn } from 'node:child_process'; +import * as p from '@clack/prompts'; + +export interface StepResult { + ok: boolean; + output: string; +} + +/** + * Run a command capturing stdout and stderr. Resolves the exit result and combined + * output; never rejects. Callers that want a spinner wrap this in one themselves. + */ +export function spawnCaptured(cmd: string, args: string[]): Promise { + return new Promise((resolve) => { + let output = ''; + const child = spawn(cmd, args, { stdio: ['ignore', 'pipe', 'pipe'] }); + child.stdout?.on('data', (chunk) => { + output += chunk.toString(); + }); + child.stderr?.on('data', (chunk) => { + output += chunk.toString(); + }); + child.on('close', (code) => resolve({ ok: code === 0, output })); + child.on('error', () => resolve({ ok: false, output })); + }); +} + +/** Print captured command output to stderr, so a failure is never swallowed. */ +export function surfaceOutput(output: string): void { + const trimmed = output.trim(); + if (trimmed) process.stderr.write(`${trimmed}\n`); +} + +/** + * Run a command as a labeled step, with a spinner over it. On failure the captured + * output is surfaced. Returns the exit result and captured output. + */ +export async function runStep(label: string, cmd: string, args: string[]): Promise { + const spinner = p.spinner(); + spinner.start(label); + + const result = await spawnCaptured(cmd, args); + if (result.ok) { + spinner.stop(label); + } else { + spinner.error(label); + surfaceOutput(result.output); + } + + return result; +} diff --git a/apps/worker/src/ai/pi/pi-executor.ts b/apps/worker/src/ai/pi/pi-executor.ts index ce63d00..f38c68c 100644 --- a/apps/worker/src/ai/pi/pi-executor.ts +++ b/apps/worker/src/ai/pi/pi-executor.ts @@ -290,6 +290,14 @@ export async function runPiPrompt( // Declared out here so the catch can bill spend accrued before a failure. let session: AgentSession | undefined; + // Abort the in-flight agent when the Temporal activity is cancelled (UI/CLI cancel). + // Without this the top-level session runs to startToCloseTimeout despite the cancel. + const onCancellation = (): void => { + void session?.abort().catch(() => { + // Best-effort — the session is torn down regardless once the prompt unwinds. + }); + }; + progress.start(); try { @@ -307,6 +315,13 @@ export async function runPiPrompt( resourceLoader, })); + // Wire activity cancellation to the session now that it exists. + if (cancellationSignal?.aborted) { + onCancellation(); + } else { + cancellationSignal?.addEventListener('abort', onCancellation, { once: true }); + } + // 5. Map pi events to audit logging + progress + error capture. session.subscribe((event: AgentSessionEvent) => { switch (event.type) { @@ -414,5 +429,7 @@ export async function runPiPrompt( cacheWriteTokens: usage.cacheWriteTokens, retryable: isRetryableFailure(err), }; + } finally { + cancellationSignal?.removeEventListener('abort', onCancellation); } } diff --git a/apps/worker/src/paths.ts b/apps/worker/src/paths.ts index 8d53a70..595c76d 100644 --- a/apps/worker/src/paths.ts +++ b/apps/worker/src/paths.ts @@ -37,6 +37,9 @@ export const ASSEMBLED_REPORT_PDF_FILENAME = 'comprehensive_security_assessment_ /** Filename of the human-facing PDF report surfaced at the run directory root */ export const FINAL_REPORT_PDF_FILENAME = 'Security-Assessment-Report.pdf'; +/** Filename of the human-facing markdown report surfaced at the run directory root, alongside the PDF */ +export const FINAL_REPORT_MD_FILENAME = 'Security-Assessment-Report.md'; + /** Structured findings the report agent emits; the markdown report is rendered from it. */ export const REPORT_JSON_FILENAME = 'report.json'; diff --git a/apps/worker/src/services/reporting.ts b/apps/worker/src/services/reporting.ts index 8d98e5e..fb1cfa3 100644 --- a/apps/worker/src/services/reporting.ts +++ b/apps/worker/src/services/reporting.ts @@ -9,6 +9,7 @@ import { ASSEMBLED_REPORT_FILENAME, ASSEMBLED_REPORT_PDF_FILENAME, deliverablesDir, + FINAL_REPORT_MD_FILENAME, FINAL_REPORT_PDF_FILENAME, resolveSessionJsonPath, SARIF_FILENAME, @@ -175,8 +176,8 @@ export async function injectModelIntoReport( /** * Surface the run's deliverables at the run directory's top level, so a customer opening the run * folder sees the report without digging through internals. Sources stay in the deliverables dir - * (git-checkpointed, used by resume). The PDF is the customer-facing report surfaced here; the - * markdown remains in the deliverables dir but is not surfaced. + * (git-checkpointed, used by resume). Both the PDF and the markdown report are surfaced here as the + * customer-facing copies. * * The SARIF log is surfaced beside it when present, since a CI step consuming it needs a stable * path and cannot be expected to reach into the internals directory. It is absent whenever the @@ -199,6 +200,15 @@ export async function copyReportToRunRoot( logger.warn(`PDF report not found, skipping ${FINAL_REPORT_PDF_FILENAME}`); } + const markdownSource = path.join(dir, ASSEMBLED_REPORT_FILENAME); + if (await fs.pathExists(markdownSource)) { + const destination = path.join(runDir, FINAL_REPORT_MD_FILENAME); + await fs.copy(markdownSource, destination, { overwrite: true }); + logger.info(`Surfaced markdown report at ${destination}`); + } else { + logger.warn(`Markdown report not found, skipping ${FINAL_REPORT_MD_FILENAME}`); + } + const sarifSource = path.join(dir, SARIF_FILENAME); if (await fs.pathExists(sarifSource)) { const sarifDestination = path.join(runDir, SARIF_FILENAME); diff --git a/apps/worker/src/services/validate-authentication.ts b/apps/worker/src/services/validate-authentication.ts index 0359a4c..f0f8127 100644 --- a/apps/worker/src/services/validate-authentication.ts +++ b/apps/worker/src/services/validate-authentication.ts @@ -23,6 +23,7 @@ import type { ActivityLogger } from '../types/activity-logger.js'; import type { AgentEndResult } from '../types/audit.js'; import type { DistributedConfig } from '../types/config.js'; import { ErrorCode } from '../types/errors.js'; +import type { AgentMetrics } from '../types/metrics.js'; import { err, ok, type Result } from '../types/result.js'; import { PentestError } from './error-handling.js'; import { loadPrompt } from './prompt-manager.js'; @@ -97,7 +98,9 @@ export interface ValidateAuthInput { readonly cancellationSignal?: AbortSignal; } -export async function validateAuthentication(input: ValidateAuthInput): Promise> { +export async function validateAuthentication( + input: ValidateAuthInput, +): Promise> { const { distributedConfig, repoPath, @@ -113,7 +116,7 @@ export async function validateAuthentication(input: ValidateAuthInput): Promise< const authentication = distributedConfig.authentication; if (!authentication) { - return ok(undefined); + return ok(null); } logger.info('Validating authentication credentials with live browser...', { @@ -160,9 +163,10 @@ export async function validateAuthentication(input: ValidateAuthInput): Promise< } } + const durationMs = Date.now() - startTime; const endResult: AgentEndResult = { attemptNumber, - duration_ms: Date.now() - startTime, + duration_ms: durationMs, cost_usd: result.cost || 0, success: classification.ok, ...(result.model !== undefined && { model: result.model }), @@ -170,7 +174,21 @@ export async function validateAuthentication(input: ValidateAuthInput): Promise< }; await auditSession.endAgent(AGENT_NAME, endResult); - return classification; + if (!classification.ok) { + return err(classification.error); + } + + const metrics: AgentMetrics = { + durationMs, + inputTokens: result.inputTokens ?? null, + outputTokens: result.outputTokens ?? null, + cacheReadTokens: result.cacheReadTokens ?? null, + cacheWriteTokens: result.cacheWriteTokens ?? null, + costUsd: result.cost ?? null, + numTurns: result.turns ?? null, + ...(result.model !== undefined && { model: result.model }), + }; + return ok(metrics); } async function verifySavedAuthState(stateFile: string, logger: ActivityLogger): Promise> { @@ -205,28 +223,32 @@ async function verifySavedAuthState(stateFile: string, logger: ActivityLogger): ); } - const cookieCount = countStorageEntries(parsed, 'cookies'); - const originCount = countStorageEntries(parsed, 'origins'); - if (cookieCount === 0 && originCount === 0) { + const cookies = storageEntries(parsed, 'cookies'); + const origins = storageEntries(parsed, 'origins'); + if (!cookies || !origins) { return err( new PentestError( - `Preflight saved an authenticated session to ${stateFile}, but it contains no cookies or origins — the browser was not actually logged in.`, + `Preflight saved an authenticated session to ${stateFile}, but it is not a storage state — cookies and origins arrays are missing.`, 'validation', true, - { stateFile, cookieCount, originCount }, + { stateFile, hasCookies: !!cookies, hasOrigins: !!origins }, ErrorCode.AGENT_EXECUTION_FAILED, ), ); } - logger.info('Preflight authenticated session saved', { stateFile, cookieCount, originCount }); + logger.info('Preflight authenticated session saved', { + stateFile, + cookieCount: cookies.length, + originCount: origins.length, + }); return ok(undefined); } -function countStorageEntries(parsed: unknown, key: 'cookies' | 'origins'): number { - if (typeof parsed !== 'object' || parsed === null) return 0; +function storageEntries(parsed: unknown, key: 'cookies' | 'origins'): unknown[] | null { + if (typeof parsed !== 'object' || parsed === null) return null; const value = (parsed as Record)[key]; - return Array.isArray(value) ? value.length : 0; + return Array.isArray(value) ? value : null; } function classifyResult( diff --git a/apps/worker/src/temporal/activities.ts b/apps/worker/src/temporal/activities.ts index f121093..7cc9621 100644 --- a/apps/worker/src/temporal/activities.ts +++ b/apps/worker/src/temporal/activities.ts @@ -637,7 +637,7 @@ export async function runPreflightValidation(input: ActivityInput): Promise { +export async function runAuthenticationValidation(input: ActivityInput): Promise { const startTime = Date.now(); const attemptNumber = Context.current().info.attempt; @@ -655,13 +655,13 @@ export async function runAuthenticationValidation(input: ActivityInput): Promise if (isErr(configResult)) { // runPreflightValidation already validated parsing, so this is unexpected. logger.warn(`runAuthenticationValidation: config load failed unexpectedly: ${configResult.error.message}`); - return; + return null; } const distributedConfig = configResult.value; if (!distributedConfig?.authentication) { logger.info('No authentication configured — skipping credential validation'); - return; + return null; } const auditSession = new AuditSession(sessionMetadata); @@ -700,6 +700,8 @@ export async function runAuthenticationValidation(input: ActivityInput): Promise truncateStackTrace(failure); throw failure; } + + return result.value; } catch (error) { if (error instanceof ApplicationFailure) { throw error; @@ -1138,9 +1140,19 @@ export async function restoreGitCheckpoint( /** * Record a resume attempt in session.json and write resume header to workflow.log. */ +/** + * Register this resume's workflow id in session.json before loadResumeState (which can throw), + * so the CLI can resolve and follow the resume even when validation fails instead of timing out. + */ +export async function registerResumeAttempt(input: ActivityInput, terminatedWorkflows: string[]): Promise { + const sessionMetadata = buildSessionMetadata(input); + const auditSession = new AuditSession(sessionMetadata); + await auditSession.initialize(); + await auditSession.addResumeAttempt(input.workflowId, terminatedWorkflows); +} + export async function recordResumeAttempt( input: ActivityInput, - terminatedWorkflows: string[], checkpointHash: string, previousWorkflowId: string, completedAgents: string[], @@ -1149,10 +1161,7 @@ export async function recordResumeAttempt( const auditSession = new AuditSession(sessionMetadata); await auditSession.initialize(); - // Update session.json with resume attempt - await auditSession.addResumeAttempt(input.workflowId, terminatedWorkflows, checkpointHash); - - // Write resume header to workflow.log + // session.json entry already added by registerResumeAttempt; here we only write the workflow.log header. await auditSession.logResumeHeader({ previousWorkflowId, newWorkflowId: input.workflowId, diff --git a/apps/worker/src/temporal/workflows.ts b/apps/worker/src/temporal/workflows.ts index bee2156..b04e83a 100644 --- a/apps/worker/src/temporal/workflows.ts +++ b/apps/worker/src/temporal/workflows.ts @@ -24,6 +24,7 @@ */ import { + ActivityCancellationType, ApplicationFailure, CancellationScope, isCancellation, @@ -96,6 +97,8 @@ const acts = proxyActivities({ startToCloseTimeout: '2 hours', heartbeatTimeout: '60 minutes', // Extended for nested pi task execution retry: PRODUCTION_RETRY, + // Cancel promptly instead of waiting out startToCloseTimeout; the agent aborts on the signal. + cancellationType: ActivityCancellationType.TRY_CANCEL, }); // Activity proxy with testing retry configuration (fast) @@ -103,6 +106,7 @@ const testActs = proxyActivities({ startToCloseTimeout: '30 minutes', heartbeatTimeout: '30 minutes', // Extended for sub-agent execution in testing retry: TESTING_RETRY, + cancellationType: ActivityCancellationType.TRY_CANCEL, }); // Retry configuration for preflight validation (short timeout, few retries) @@ -119,6 +123,7 @@ const preflightActs = proxyActivities({ startToCloseTimeout: '2 minutes', heartbeatTimeout: '2 minutes', retry: PREFLIGHT_RETRY, + cancellationType: ActivityCancellationType.TRY_CANCEL, }); // Credential rejection is not retryable; transient provider errors get 3 attempts. @@ -135,6 +140,7 @@ const authValidationActs = proxyActivities({ startToCloseTimeout: '10 minutes', heartbeatTimeout: '10 minutes', retry: AUTH_VALIDATION_RETRY, + cancellationType: ActivityCancellationType.TRY_CANCEL, }); /** @@ -246,6 +252,10 @@ export async function pentestPipeline(input: PipelineInput): Promise 0) { - return `${hours}h ${minutes % 60}m`; - } - if (minutes > 0) { - return `${minutes}m`; - } - return `${seconds}s`; -} - -function getStatusDisplay(status: string): string { - return status; -} - -function truncate(str: string, maxLen: number): string { - if (str.length <= maxLen) return str; - return `${str.slice(0, maxLen - 1)}\u2026`; -} - -async function listWorkspaces(): Promise { - const workspacesDir = process.env.WORKSPACES_DIR || DEFAULT_WORKSPACES_DIR; - - let entries: string[]; - try { - entries = await fs.readdir(workspacesDir); - } catch { - console.log('No workspaces directory found.'); - console.log(`Expected: ${workspacesDir}`); - return; - } - - const workspaces: WorkspaceInfo[] = []; - - for (const entry of entries) { - const sessionPath = resolveSessionJsonPath(path.join(workspacesDir, entry)); - try { - const content = await fs.readFile(sessionPath, 'utf8'); - const data = JSON.parse(content) as SessionJson; - - workspaces.push({ - name: entry, - url: data.session.webUrl, - status: data.session.status, - createdAt: new Date(data.session.createdAt), - completedAt: data.session.completedAt ? new Date(data.session.completedAt) : null, - costUsd: data.metrics.total_cost_usd, - }); - } catch { - // Skip directories without valid session.json - } - } - - if (workspaces.length === 0) { - console.log('\nNo workspaces found.'); - console.log('Run a pipeline first: ./shannon start -u -r '); - return; - } - - // Sort by creation date (most recent first) - workspaces.sort((a, b) => b.createdAt.getTime() - a.createdAt.getTime()); - - console.log('\n=== Shannon Workspaces ===\n'); - - // Column widths - const nameWidth = 30; - const urlWidth = 30; - const statusWidth = 14; - const durationWidth = 10; - const costWidth = 10; - - // Header - console.log( - ' ' + - 'WORKSPACE'.padEnd(nameWidth) + - 'URL'.padEnd(urlWidth) + - 'STATUS'.padEnd(statusWidth) + - 'DURATION'.padEnd(durationWidth) + - 'COST'.padEnd(costWidth), - ); - console.log(` ${'\u2500'.repeat(nameWidth + urlWidth + statusWidth + durationWidth + costWidth)}`); - - let resumableCount = 0; - - for (const ws of workspaces) { - const now = new Date(); - const endTime = ws.completedAt || now; - const durationMs = endTime.getTime() - ws.createdAt.getTime(); - const duration = formatDuration(durationMs); - const cost = `$${ws.costUsd.toFixed(2)}`; - const isResumable = ws.status !== 'completed'; - - if (isResumable) { - resumableCount++; - } - - const resumeTag = isResumable ? ' (resumable)' : ''; - - console.log( - ' ' + - truncate(ws.name, nameWidth - 2).padEnd(nameWidth) + - truncate(ws.url, urlWidth - 2).padEnd(urlWidth) + - getStatusDisplay(ws.status).padEnd(statusWidth) + - duration.padEnd(durationWidth) + - cost.padEnd(costWidth) + - resumeTag, - ); - } - - console.log(); - const summary = `${workspaces.length} workspace${workspaces.length === 1 ? '' : 's'} found`; - const resumeSummary = resumableCount > 0 ? ` (${resumableCount} resumable)` : ''; - console.log(`${summary}${resumeSummary}`); - - if (resumableCount > 0) { - console.log('\nResume with: ./shannon start -u -r -w '); - } - - console.log(); -} - -listWorkspaces().catch((err) => { - console.error('Error listing workspaces:', err); - process.exit(1); -}); diff --git a/docs/development.md b/docs/development.md index a4d8e38..43f8f55 100644 --- a/docs/development.md +++ b/docs/development.md @@ -58,7 +58,8 @@ Monitor progress: ```bash npx @keygraph/shannon logs -npx @keygraph/shannon status +npx @keygraph/shannon status +npx @keygraph/shannon scans npx @keygraph/shannon version ``` @@ -66,7 +67,8 @@ Source-build equivalents: ```bash ./shannon logs -./shannon status +./shannon status +./shannon scans ./shannon version ``` @@ -79,16 +81,17 @@ open http://localhost:8233 Stop Shannon: ```bash -npx @keygraph/shannon stop -npx @keygraph/shannon stop --clean # confirms first; add --yes (or -y) to skip -npx @keygraph/shannon uninstall # confirms first; add --yes (or -y) to skip +npx @keygraph/shannon stop # stop one scan (confirms first; add --yes/-y to skip) +npx @keygraph/shannon stop --all # stop all scans (Temporal stays up) +npx @keygraph/shannon reset # stop everything and wipe all Temporal data (type 'confirm' to proceed; cannot be skipped) ``` Source-build equivalents: ```bash -./shannon stop -./shannon stop --clean # add --yes (or -y) to skip the confirmation +./shannon stop # stop one scan (confirms first; add --yes/-y to skip) +./shannon stop --all # stop all scans (Temporal stays up) +./shannon reset # stop everything and wipe all Temporal data (type 'confirm' to proceed; cannot be skipped) ``` Usage examples: @@ -106,8 +109,11 @@ npx @keygraph/shannon start -u https://example.com -r /path/to/repo -o ./my-repo # Named workspace. npx @keygraph/shannon start -u https://example.com -r /path/to/repo -w q1-audit -# List all workspaces. -npx @keygraph/shannon workspaces +# Stream the log until the scan finishes, then exit on its outcome (useful in CI). +npx @keygraph/shannon start -u https://example.com -r /path/to/repo --follow + +# List completed scans. +npx @keygraph/shannon scans ``` Source-build examples: @@ -117,7 +123,8 @@ Source-build examples: ./shannon start -u https://example.com -r /path/to/repo -c /path/to/my-config.yaml ./shannon start -u https://example.com -r /path/to/repo -o ./my-reports ./shannon start -u https://example.com -r /path/to/repo -w q1-audit -./shannon workspaces +./shannon start -u https://example.com -r /path/to/repo --follow +./shannon scans # Rebuild the worker image. ./shannon build --no-cache @@ -132,11 +139,12 @@ Results are saved to the workspaces directory: Use `-o ` to copy deliverables to a custom output directory after a run completes. -Output structure — the run directory's top level holds only the final report; everything else is nested under a hidden `.shannon/` directory: +Output structure — the run directory's top level holds the final report, in PDF and Markdown; everything else is nested under a hidden `.shannon/` directory: ```text workspaces/{hostname}_{sessionId}/ -|-- Security-Assessment-Report.pdf # the final report (the deliverable) +|-- Security-Assessment-Report.pdf # the final report (PDF) +|-- Security-Assessment-Report.md # the final report (Markdown) `-- .shannon/ # internals |-- deliverables/ # report source, per-phase analysis, queues |-- agents/ # per-agent logs diff --git a/docs/workspaces.md b/docs/workspaces.md index 10625cf..d9b5682 100644 --- a/docs/workspaces.md +++ b/docs/workspaces.md @@ -11,7 +11,7 @@ Shannon uses workspaces to store scan state, logs, prompts, and deliverables. Wo - Use `-w ` to give a run a custom name. - To resume a run, pass the same workspace name with `-w`. - Each agent's progress is checkpointed so resumed runs can skip completed work. -- The final report is surfaced at the workspace root as `Security-Assessment-Report.pdf`. Run internals — deliverables, logs, prompts, and session state — live under a hidden `.shannon/` directory. +- The final report is surfaced at the workspace root as `Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`. Run internals — deliverables, logs, prompts, and session state — live under a hidden `.shannon/` directory. > [!NOTE] > The URL must match the original workspace URL when resuming. Shannon rejects mismatched URLs to prevent cross-target contamination. @@ -36,10 +36,10 @@ Resume an auto-named workspace: npx @keygraph/shannon start -u https://example.com -r /path/to/repo -w example-com_shannon-1771007534808 ``` -List all workspaces: +List completed scans: ```bash -npx @keygraph/shannon workspaces +npx @keygraph/shannon scans ``` Source-build equivalents: @@ -47,5 +47,5 @@ Source-build equivalents: ```bash ./shannon start -u https://example.com -r /path/to/repo -w my-audit ./shannon start -u https://example.com -r /path/to/repo -w example-com_shannon-1771007534808 -./shannon workspaces +./shannon scans ``` diff --git a/llms-full.txt b/llms-full.txt index 2f88070..72bcc84 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -323,7 +323,8 @@ Monitor progress: ```bash npx @keygraph/shannon logs -npx @keygraph/shannon status +npx @keygraph/shannon status +npx @keygraph/shannon scans npx @keygraph/shannon version ``` @@ -331,7 +332,8 @@ Source-build equivalents: ```bash ./shannon logs -./shannon status +./shannon status +./shannon scans ./shannon version ``` @@ -344,16 +346,17 @@ open http://localhost:8233 Stop Shannon: ```bash -npx @keygraph/shannon stop -npx @keygraph/shannon stop --clean # confirms first; add --yes (or -y) to skip -npx @keygraph/shannon uninstall # confirms first; add --yes (or -y) to skip +npx @keygraph/shannon stop # stop one scan (confirms first; add --yes/-y to skip) +npx @keygraph/shannon stop --all # stop all scans (Temporal stays up) +npx @keygraph/shannon reset # stop everything and wipe all Temporal data (type 'confirm' to proceed; cannot be skipped) ``` Source-build equivalents: ```bash -./shannon stop -./shannon stop --clean # add --yes (or -y) to skip the confirmation +./shannon stop # stop one scan (confirms first; add --yes/-y to skip) +./shannon stop --all # stop all scans (Temporal stays up) +./shannon reset # stop everything and wipe all Temporal data (type 'confirm' to proceed; cannot be skipped) ``` Usage examples: @@ -371,8 +374,11 @@ npx @keygraph/shannon start -u https://example.com -r /path/to/repo -o ./my-repo # Named workspace. npx @keygraph/shannon start -u https://example.com -r /path/to/repo -w q1-audit -# List all workspaces. -npx @keygraph/shannon workspaces +# Stream the log until the scan finishes, then exit on its outcome (useful in CI). +npx @keygraph/shannon start -u https://example.com -r /path/to/repo --follow + +# List completed scans. +npx @keygraph/shannon scans ``` Source-build examples: @@ -382,7 +388,8 @@ Source-build examples: ./shannon start -u https://example.com -r /path/to/repo -c /path/to/my-config.yaml ./shannon start -u https://example.com -r /path/to/repo -o ./my-reports ./shannon start -u https://example.com -r /path/to/repo -w q1-audit -./shannon workspaces +./shannon start -u https://example.com -r /path/to/repo --follow +./shannon scans # Rebuild the worker image. ./shannon build --no-cache @@ -397,11 +404,12 @@ Results are saved to the workspaces directory: Use `-o ` to copy deliverables to a custom output directory after a run completes. -Output structure — the run directory's top level holds only the final report; everything else is nested under a hidden `.shannon/` directory: +Output structure — the run directory's top level holds the final report, in PDF and Markdown; everything else is nested under a hidden `.shannon/` directory: ```text workspaces/{hostname}_{sessionId}/ -|-- Security-Assessment-Report.pdf # the final report (the deliverable) +|-- Security-Assessment-Report.pdf # the final report (PDF) +|-- Security-Assessment-Report.md # the final report (Markdown) `-- .shannon/ # internals |-- deliverables/ # report source, per-phase analysis, queues |-- agents/ # per-agent logs @@ -861,7 +869,7 @@ Shannon uses workspaces to store scan state, logs, prompts, and deliverables. Wo - Use `-w ` to give a run a custom name. - To resume a run, pass the same workspace name with `-w`. - Each agent's progress is checkpointed so resumed runs can skip completed work. -- The final report is surfaced at the workspace root as `Security-Assessment-Report.pdf`. Run internals — deliverables, logs, prompts, and session state — live under a hidden `.shannon/` directory. +- The final report is surfaced at the workspace root as `Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`. Run internals — deliverables, logs, prompts, and session state — live under a hidden `.shannon/` directory. > [!NOTE] > The URL must match the original workspace URL when resuming. Shannon rejects mismatched URLs to prevent cross-target contamination. @@ -886,10 +894,10 @@ Resume an auto-named workspace: npx @keygraph/shannon start -u https://example.com -r /path/to/repo -w example-com_shannon-1771007534808 ``` -List all workspaces: +List completed scans: ```bash -npx @keygraph/shannon workspaces +npx @keygraph/shannon scans ``` Source-build equivalents: @@ -897,7 +905,7 @@ Source-build equivalents: ```bash ./shannon start -u https://example.com -r /path/to/repo -w my-audit ./shannon start -u https://example.com -r /path/to/repo -w example-com_shannon-1771007534808 -./shannon workspaces +./shannon scans ``` --- diff --git a/llms.txt b/llms.txt index 772dbcd..4abddef 100644 --- a/llms.txt +++ b/llms.txt @@ -12,7 +12,7 @@ Use this file as the concise entry point for AI agents and LLMs reading this rep ## Shannon - [Development](docs/development.md): Source-build workflow, common CLI commands, repository paths, and output locations. -- [Configuration](docs/configuration.md): Authenticated testing, login flows, rules of engagement, report filters, credential precedence, adaptive thinking, and rate-limit settings. +- [Configuration](docs/configuration.md): Authenticated testing, login flows, rules of engagement, report filters, credential precedence, and rate-limit settings. - [AI Providers](docs/ai-providers.md): Anthropic, OpenAI, xAI, AWS Bedrock, any other Pi-supported provider, and custom gateway setup. - [Platforms and Networking](docs/platforms.md): Windows/WSL2, Linux, macOS, Docker networking, local applications, and custom hostnames. - [Workspaces and Resuming](docs/workspaces.md): Workspace storage, naming, resuming interrupted scans, and examples. diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index fcd0b95..6b7be47 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -26,6 +26,9 @@ importers: '@clack/prompts': specifier: ^1.1.0 version: 1.1.0 + '@temporalio/client': + specifier: ^1.11.0 + version: 1.15.0 chokidar: specifier: ^5.0.0 version: 5.0.0