diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index fe7437e..484a8e1 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -69,9 +69,9 @@ body: Issues without this information may be difficult to triage. - - Check the workflow log: - - **npx mode:** `~/.shannon/workspaces//workflow.log` - - **Local mode:** `./workspaces//workflow.log` + - Check the scan log: + - **npx mode:** `~/.shannon/workspaces//.shannon/workflow.log` + - **Local mode:** `./workspaces//.shannon/workflow.log` Use `grep` or search to identify errors. Paste the relevant error output below. - Temporal: diff --git a/CLAUDE.md b/CLAUDE.md index 2af618a..f0b9017 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -61,17 +61,20 @@ npx @keygraph/shannon setup ./shannon workspaces # List all workspaces # Monitor -./shannon logs # Tail workflow log -./shannon status # Show running workers -# Temporal Web UI: http://localhost:8233 +./shannon logs # Show a scan's live log +./shannon status # Show running scans +# Dashboard: http://localhost:8233 # Stop -./shannon stop # Preserves workflow data -./shannon stop --clean # Full cleanup including volumes (confirms first) +./shannon stop # Preserves scan data +./shannon stop --clean # Full cleanup including volumes (confirms first; --yes/-y to skip) + +# Version +./shannon version # npx: package version; local: git SHA # Image management ./shannon build [--no-cache] # Local mode: build worker image -npx @keygraph/shannon uninstall # npx mode: remove ~/.shannon/ (confirms first) +npx @keygraph/shannon uninstall # npx mode: remove ~/.shannon/ (confirms first; --yes/-y to skip) # Build TypeScript (development) pnpm run build # Build all packages via Turborepo @@ -82,7 +85,7 @@ pnpm biome:fix # Auto-fix lint, format, and import sorting **Monorepo tooling:** pnpm workspaces, Turborepo for task orchestration, Biome for linting/formatting. TypeScript compiler options shared via `tsconfig.base.json` at the root. All packages extend it, overriding only `rootDir` and `outDir`. Shared devDependencies (`typescript`, `@types/node`, `turbo`, `@biomejs/biome`) are hoisted to the root workspace. -**Options:** `-c ` (YAML config), `-o ` (output directory), `-w ` (named workspace; auto-resumes if exists), `--pipeline-testing` (minimal prompts, 10s retries), `--debug` (preserve worker container after exit for log inspection) +**Options:** `-c ` (YAML config), `-o ` (output directory), `-w ` (named workspace; auto-resumes if exists), `--pipeline-testing` (minimal prompts, 10s retries), `--debug` (preserve worker container after exit for log inspection), `--yes`/`-y` (skip the confirmation prompt on `stop --clean`/`uninstall`; required for non-interactive use) ## Architecture @@ -96,7 +99,7 @@ apps/worker/ — @shannon/worker (private, Temporal worker + pipeline logic) ### CLI Package (`apps/cli/`) Published as `@keygraph/shannon` on npm. Contains only Docker orchestration logic — no Temporal SDK, business logic, or prompts. Bundled with tsdown for single-file ESM output. -- `apps/cli/src/index.ts` — CLI dispatcher (`setup`, `start`, `stop`, `logs`, `workspaces`, `status`, `build`, `uninstall`, `info`) +- `apps/cli/src/index.ts` — CLI dispatcher (`setup`, `start`, `stop`, `logs`, `workspaces`, `status`, `build`, `uninstall`, `version`) - `apps/cli/src/mode.ts` — Auto-detection: local mode if `SHANNON_LOCAL=1` env var is set - `apps/cli/src/docker.ts` — Compose lifecycle, image pull/build, ephemeral `docker run` worker spawning - `apps/cli/src/home.ts` — State directory management (`~/.shannon/` for npx, `./` for local) @@ -105,6 +108,8 @@ Published as `@keygraph/shannon` on npm. Contains only Docker orchestration logi - `apps/cli/src/config/writer.ts` — TOML serialization and secure file persistence (0o600) - `apps/cli/src/commands/setup.ts` — Interactive TUI wizard (`@clack/prompts`) for provider credential setup (npx only) - `apps/cli/src/paths.ts` — Repo/config path resolution (bare name → `./repos/`, or any absolute/relative path) +- `apps/cli/src/version.ts` — Version reporting (npx: `package.json` version; local: `git-`) +- `apps/cli/src/tty.ts` — Terminal capability detection: `requireInteractive` guard (fails fast off-TTY instead of hanging on a prompt), `supportsColor` color gating (`NO_COLOR`/`FORCE_COLOR`), and `stdoutIsTerminal` for spinner/cursor output - `apps/cli/src/commands/` — Command handlers - `apps/cli/infra/compose.yml` — Bundled Temporal compose file for npx mode - `apps/cli/tsdown.config.ts` — tsdown bundler config @@ -148,8 +153,8 @@ Durable workflow orchestration with crash recovery, queryable progress, intellig - **Configuration** — YAML configs in `apps/worker/configs/` with JSON Schema validation (`config-schema.json`). Supports auth settings (MFA/TOTP), URL/code rule scoping (`rules.avoid`/`rules.focus`), run-scope steering (`vuln_classes`, `exploit`), free-form `rules_of_engagement`, and post-hoc `report` filters (`min_severity`, `min_confidence`, `guidance`). `code_path` avoid rules are written into `~/.claude/settings.json` `permissions.deny` (`Read`/`Edit`) once per workflow by `apps/worker/src/temporal/activities.ts:syncCodePathDenyRules` so the SDK enforces them at the tool layer even in `bypassPermissions` mode. `vuln_classes`/`exploit` scope is locked into `session.json` on first run; resumes with a different scope fail fast (`persistOrValidateRunScope`). Credential resolution — local mode: env vars → `./.env`; npx mode: env vars → `~/.shannon/config.toml` (via `npx @keygraph/shannon setup`) - **Prompts** — Per-phase templates in `apps/worker/prompts/` with variable substitution (`{{TARGET_URL}}`, `{{CONFIG_CONTEXT}}`). Shared partials in `apps/worker/prompts/shared/` via `apps/worker/src/services/prompt-manager.ts`, including `_code-path-rules.txt` (focus/avoid `[FILE]`/`[GLOB]` routing) and `_rules-of-engagement.txt` (free-text engagement rules). When `exploit: false`, `apps/worker/src/services/findings-renderer.ts` deterministically converts each `*_exploitation_queue.json` into a `*_findings.md` for report assembly — no LLM in the loop - **SDK Integration** — Uses `@anthropic-ai/claude-agent-sdk` with `maxTurns: 10_000` and `bypassPermissions` mode. Adaptive thinking is enabled by default on Opus 4.6/4.7/4.8 (`supportsAdaptiveThinking` in `apps/worker/src/ai/models.ts`); disable per-scan via `CLAUDE_ADAPTIVE_THINKING=false` (env) or `core.adaptive_thinking = false` (npx TOML). Browser automation via `playwright-cli` with session isolation (`-s=`). TOTP generation via `generate-totp` CLI tool. Login flow template at `apps/worker/prompts/shared/login-instructions.txt` supports form, SSO, API, and basic auth. On authenticated whitebox scans, the `validate-authentication` preflight performs the single real login and saves the browser session to `auth-state.json` in the per-session audit directory (path from `authStateFile()` in `apps/worker/src/audit/utils.ts`, derived from `generateAuditPath()`). The validation activity (`apps/worker/src/services/validate-authentication.ts`) removes any stale file from a prior run before the agent runs and verifies the file parses and contains cookies or storage before the preflight is marked complete; `logWorkflowComplete` deletes it when the workflow ends so authenticated cookies don't sit on disk between scans. Agent prompts opt in to session reuse by `@include(shared/_shared-session.txt)` before their `` block — the partial restores the session and falls through to the full login flow if verification fails. `vuln-auth`/`exploit-auth` omit the include and own their own login -- **Audit System** — Crash-safe append-only logging in `workspaces/{hostname}_{sessionId}/`. Tracks session metrics, per-agent logs, prompts, and deliverables. WorkflowLogger (`apps/worker/src/audit/workflow-logger.ts`) provides unified human-readable per-workflow logs, backed by LogStream (`apps/worker/src/audit/log-stream.ts`) shared stream primitive -- **Deliverables** — Saved to `deliverables/` in the target repo via the `save-deliverable` CLI script (`apps/worker/src/scripts/save-deliverable.ts`) +- **Audit System** — Crash-safe append-only logging in `workspaces/{hostname}_{sessionId}/`. The run directory's top level holds only the human-facing report (`Security-Assessment-Report.md`, `FINAL_REPORT_FILENAME` in `apps/worker/src/paths.ts`); everything else — deliverables, per-agent logs, prompts, `session.json`, `workflow.log`, and browser artifacts — is nested under a hidden `.shannon/` internals dir (`INTERNAL_DIR`) so a customer sees only the report. Audit path helpers route through `generateInternalPath` (`apps/worker/src/audit/utils.ts`); the CLI nests the overlay backing dirs under the same `.shannon/` (`apps/cli/src/docker.ts`, `start.ts`). `session.json`/`workflow.log` reads use dual-read resolvers (`resolveSessionJsonPath`, `resolveRunFile`) that prefer `.shannon/` and fall back to the legacy run-root layout, so pre-restructure workspaces stay listable (`workspaces`/`logs`) without migration. Resuming a pre-restructure workspace upgrades it in place first: `migrateLegacyWorkspaceLayout` (`apps/cli/src/commands/start.ts`) renames the flat deliverables/logs/session entries into `.shannon/` (carrying the deliverables `.git` along) before the overlay dirs are mounted, so resume finds the old checkpoints instead of re-running every agent. The report is surfaced by copying the assembled `comprehensive_security_assessment_report.md` from the deliverables dir to the run root (`copyReportToRunRoot` in `apps/worker/src/services/reporting.ts`). WorkflowLogger (`apps/worker/src/audit/workflow-logger.ts`) provides unified human-readable per-workflow logs, backed by LogStream (`apps/worker/src/audit/log-stream.ts`) shared stream primitive +- **Deliverables** — Saved to `.shannon/deliverables/` in the target repo via the `save-deliverable` CLI script (`apps/worker/src/scripts/save-deliverable.ts`) - **Workspaces & Resume** — Named workspaces via `-w ` or auto-named from URL+timestamp. Resume detects completed agents via `session.json`. `loadResumeState()` in `apps/worker/src/temporal/activities.ts` validates deliverable existence, restores git checkpoints, and cleans up incomplete deliverables. Workspace listing via `apps/worker/src/temporal/workspaces.ts` ## Development Notes diff --git a/apps/cli/src/commands/logs.ts b/apps/cli/src/commands/logs.ts index 4bee242..37d464a 100644 --- a/apps/cli/src/commands/logs.ts +++ b/apps/cli/src/commands/logs.ts @@ -1,5 +1,5 @@ /** - * `shannon logs` command — tail a workspace's workflow log. + * `shannon logs` command — tail a scan's live log. * * Uses chokidar for reliable cross-platform file watching and * bounded synchronous reads to prevent duplicate output. @@ -9,9 +9,10 @@ import fs from 'node:fs'; import path from 'node:path'; import { watch } from 'chokidar'; import { getWorkspacesDir } from '../home.js'; +import { resolveRunFile } from '../paths.js'; // Match the exact line the worker writes — anchored to prevent false positives from agent output -const COMPLETION_PATTERN = /^Workflow (COMPLETED|FAILED)$/m; +const COMPLETION_PATTERN = /^Scan (COMPLETED|FAILED)$/m; /** Read a byte range from a file and return it as a UTF-8 string. */ function readRange(filePath: string, start: number, end: number): string { @@ -31,30 +32,30 @@ function resolveLogFile(workspaceId: string): string { const workspacesDir = getWorkspacesDir(); // 1. Direct match - const directPath = path.join(workspacesDir, workspaceId, 'workflow.log'); + const directPath = resolveRunFile(path.join(workspacesDir, workspaceId), 'workflow.log'); if (fs.existsSync(directPath)) return directPath; // 2. Resume workflow ID (e.g. workspace_resume_123) const resumeBase = workspaceId.replace(/_resume_\d+$/, ''); if (resumeBase !== workspaceId) { - const resumePath = path.join(workspacesDir, resumeBase, 'workflow.log'); + const resumePath = resolveRunFile(path.join(workspacesDir, resumeBase), 'workflow.log'); if (fs.existsSync(resumePath)) return resumePath; } // 3. Named workspace ID (e.g. workspace_shannon-123) const namedBase = workspaceId.replace(/_shannon-\d+$/, ''); if (namedBase !== workspaceId) { - const namedPath = path.join(workspacesDir, namedBase, 'workflow.log'); + const namedPath = resolveRunFile(path.join(workspacesDir, namedBase), 'workflow.log'); if (fs.existsSync(namedPath)) return namedPath; } - console.error(`ERROR: Workflow log not found for: ${workspaceId}`); + console.error(`ERROR: No scan found named: ${workspaceId}`); console.error(''); console.error('Possible causes:'); - console.error(" - Workflow hasn't started yet"); - console.error(' - Workspace ID is incorrect'); + console.error(" - The scan hasn't started yet"); + console.error(' - The workspace name is incorrect'); console.error(''); - console.error('Check the Temporal Web UI at http://localhost:8233 for workflow details'); + console.error('Check the dashboard at http://localhost:8233 for scan details'); process.exit(1); } @@ -82,7 +83,7 @@ export function logs(workspaceId: string): void { } } - console.log(`Tailing workflow log: ${logFile}`); + console.log(`Tailing scan log: ${logFile}`); // 1. Output existing content if (flush()) { diff --git a/apps/cli/src/commands/setup.ts b/apps/cli/src/commands/setup.ts index ed03cdc..fe1dd44 100644 --- a/apps/cli/src/commands/setup.ts +++ b/apps/cli/src/commands/setup.ts @@ -10,12 +10,14 @@ import os from 'node:os'; import path from 'node:path'; import * as p from '@clack/prompts'; import { type ShannonConfig, saveConfig } from '../config/writer.js'; +import { requireInteractive } from '../tty.js'; const SHANNON_HOME = path.join(os.homedir(), '.shannon'); type Provider = 'anthropic' | 'custom_base_url' | 'bedrock' | 'vertex'; export async function setup(): Promise { + requireInteractive('setup', 'For non-interactive use, export credentials as env vars (e.g. ANTHROPIC_API_KEY).'); p.intro('Shannon Setup'); // 1. Select provider diff --git a/apps/cli/src/commands/start.ts b/apps/cli/src/commands/start.ts index 6a78a5e..13c47fc 100644 --- a/apps/cli/src/commands/start.ts +++ b/apps/cli/src/commands/start.ts @@ -12,8 +12,9 @@ import { ensureImage, ensureInfra, randomSuffix, spawnWorker } from '../docker.j import { buildEnvFlags, loadEnv, validateCredentials } from '../env.js'; import { getCredentialsPath, getWorkspacesDir, initHome } from '../home.js'; import { isLocal } from '../mode.js'; -import { resolveConfig, resolveRepo } from '../paths.js'; +import { FINAL_REPORT_FILENAME, INTERNAL_DIR, resolveConfig, resolveRepo, resolveRunFile } from '../paths.js'; import { displaySplash } from '../splash.js'; +import { stdoutIsTerminal } from '../tty.js'; export interface StartArgs { url: string; @@ -26,6 +27,29 @@ export interface StartArgs { version: string; } +/** + * Upgrade a pre-restructure workspace (flat layout, no INTERNAL_DIR) before it is mounted, + * so resume finds the old deliverables and their git checkpoints instead of re-running every + * agent. For a legacy run every top-level entry is internal, so move them all into INTERNAL_DIR + * (a same-filesystem rename carries the deliverables .git along). + */ +function migrateLegacyWorkspaceLayout(workspacePath: string): void { + const legacySessionJson = path.join(workspacePath, 'session.json'); + const internalPath = path.join(workspacePath, INTERNAL_DIR); + if (!fs.existsSync(legacySessionJson) || fs.existsSync(internalPath)) { + return; + } + + fs.mkdirSync(internalPath, { recursive: true }); + for (const entry of fs.readdirSync(workspacePath)) { + if (entry === INTERNAL_DIR) { + continue; + } + fs.renameSync(path.join(workspacePath, entry), path.join(internalPath, entry)); + } + console.log(`Migrated workspace to ${INTERNAL_DIR}/ layout: ${workspacePath}`); +} + export async function start(args: StartArgs): Promise { // 1. Initialize state directories and load env initHome(); @@ -61,12 +85,17 @@ export async function start(args: StartArgs): Promise { args.workspace ?? `${new URL(args.url).hostname.replace(/[^a-zA-Z0-9-]/g, '-')}_shannon-${Date.now()}`; // 8. Create writable overlay directories (mounted over :ro repo paths inside container) - // Workspace dir must be 0o777 so the container user (UID 1001) can create audit subdirs + // The run dir and its INTERNAL_DIR must be 0o777 so the container user can create audit + // subdirs and the overlay backing dirs. const workspacePath = path.join(workspacesDir, workspace); + const internalPath = path.join(workspacePath, INTERNAL_DIR); fs.mkdirSync(workspacePath, { recursive: true }); fs.chmodSync(workspacePath, 0o777); + migrateLegacyWorkspaceLayout(workspacePath); + fs.mkdirSync(internalPath, { recursive: true }); + fs.chmodSync(internalPath, 0o777); for (const dir of ['deliverables', 'scratchpad', '.playwright-cli', '.playwright']) { - const dirPath = path.join(workspacePath, dir); + const dirPath = path.join(internalPath, dir); fs.mkdirSync(dirPath, { recursive: true }); fs.chmodSync(dirPath, 0o777); } @@ -119,7 +148,7 @@ export async function start(args: StartArgs): Promise { const dockerExitCode = await new Promise((resolve) => { proc.once('exit', (code) => resolve(code ?? 1)); proc.once('error', (err) => { - console.error(`Failed to start worker: ${err.message}`); + console.error(`Failed to start the scan: ${err.message}`); resolve(1); }); }); @@ -129,7 +158,7 @@ export async function start(args: StartArgs): Promise { } // Detect whether this is a fresh workspace or a resume by checking session.json existence - const sessionJson = path.join(workspacesDir, workspace, 'session.json'); + const sessionJson = resolveRunFile(path.join(workspacesDir, workspace), 'session.json'); const isResume = fs.existsSync(sessionJson); let initialResumeCount = 0; if (isResume) { @@ -141,8 +170,10 @@ export async function start(args: StartArgs): Promise { } } - // Poll for workflow to register in session.json - process.stdout.write('Waiting for workflow to start...'); + // Poll for workflow to register in session.json. Off-TTY, skip the dots and + // clear-line escape so redirected logs stay clean. + const animate = stdoutIsTerminal(); + process.stdout.write('Waiting for the scan to start...'); let workflowId = ''; let started = false; let attempts = 0; @@ -151,7 +182,7 @@ export async function start(args: StartArgs): Promise { if (attempts > 60) { clearInterval(pollInterval); process.stdout.write('\n'); - console.error('Timeout waiting for workflow to start'); + console.error('Timed out waiting for the scan to start'); process.exit(1); } @@ -169,15 +200,15 @@ export async function start(args: StartArgs): Promise { // Latest workflow ID: last resume attempt, or originalWorkflowId for fresh scans workflowId = resumeAttempts.at(-1)?.workflowId ?? session.session?.originalWorkflowId ?? ''; - // Clear waiting line and show info - process.stdout.write('\r\x1b[K'); + // Clear the waiting line, or just break it off-TTY + process.stdout.write(animate ? '\r\x1b[K' : '\n'); printInfo(args, workspace, workflowId, repo.hostPath, workspacesDir); return; } } catch { // File doesn't exist yet } - process.stdout.write('.'); + if (animate) process.stdout.write('.'); }, 2000); // Stop the worker container only if it hasn't started yet @@ -186,7 +217,7 @@ export async function start(args: StartArgs): Promise { if (cleaned || started) return; cleaned = true; clearInterval(pollInterval); - console.log(`\nStopping worker ${containerName}...`); + console.log('\nStopping scan...'); try { execFileSync('docker', ['stop', containerName], { stdio: 'pipe' }); } catch { @@ -224,8 +255,10 @@ function printInfo( workspacesDir: string, ): void { const logsCmd = isLocal() ? `./shannon logs ${workspace}` : `npx @keygraph/shannon logs ${workspace}`; - const reportsPath = path.join(workspacesDir, workspace); + const reportPath = path.join(workspacesDir, workspace, FINAL_REPORT_FILENAME); + console.log(' Scan started — it runs in the background, so you can close this terminal.'); + console.log(''); console.log(` Target: ${args.url}`); console.log(` Repository: ${repoPath}`); console.log(` Workspace: ${workspace}`); @@ -252,15 +285,15 @@ function printInfo( } console.log(''); - console.log(' Monitor:'); + console.log(' Watch scan progress:'); + console.log(` Live logs: ${logsCmd}`); if (workflowId) { - console.log(` Web UI: http://localhost:8233/namespaces/default/workflows/${workflowId}`); + console.log(` Dashboard: http://localhost:8233/namespaces/default/workflows/${workflowId}`); } else { - console.log(' Web UI: http://localhost:8233'); + console.log(' Dashboard: http://localhost:8233'); } - console.log(` Logs: ${logsCmd}`); console.log(''); - console.log(' Output:'); - console.log(` Reports: ${reportsPath}/`); + console.log(' Report (when the scan finishes):'); + console.log(` ${reportPath}`); console.log(''); } diff --git a/apps/cli/src/commands/status.ts b/apps/cli/src/commands/status.ts index 724ae49..4bc982f 100644 --- a/apps/cli/src/commands/status.ts +++ b/apps/cli/src/commands/status.ts @@ -1,5 +1,5 @@ /** - * `shannon status` command — show running workers and Temporal health. + * `shannon status` command — show running scans and Temporal health. */ import { isTemporalReady, listRunningWorkers } from '../docker.js'; @@ -9,16 +9,16 @@ export function status(): void { const temporalUp = isTemporalReady(); console.log(`Temporal: ${temporalUp ? 'running' : 'not running'}`); if (temporalUp) { - console.log(' Web UI: http://localhost:8233'); + console.log(' Dashboard: http://localhost:8233'); } console.log(''); - // 2. Running workers + // 2. Running scans const workers = listRunningWorkers(); if (workers) { - console.log('Workers:'); + console.log('Running scans:'); console.log(workers); } else { - console.log('Workers: none running'); + console.log('No scans running'); } } diff --git a/apps/cli/src/commands/stop.ts b/apps/cli/src/commands/stop.ts index f123013..2b805aa 100644 --- a/apps/cli/src/commands/stop.ts +++ b/apps/cli/src/commands/stop.ts @@ -4,9 +4,11 @@ import * as p from '@clack/prompts'; import { stopInfra, stopWorkers } from '../docker.js'; +import { requireInteractive } from '../tty.js'; -export async function stop(clean: boolean): Promise { - if (clean) { +export async function stop(clean: boolean, yes: boolean): Promise { + if (clean && !yes) { + requireInteractive('stop --clean', 'Re-run with --yes to skip this confirmation.'); const confirmed = await p.confirm({ message: 'This will stop all running scans and remove the Temporal data. Continue?', }); diff --git a/apps/cli/src/commands/uninstall.ts b/apps/cli/src/commands/uninstall.ts index b2d7f36..1010736 100644 --- a/apps/cli/src/commands/uninstall.ts +++ b/apps/cli/src/commands/uninstall.ts @@ -7,24 +7,34 @@ import os from 'node:os'; import path from 'node:path'; import * as p from '@clack/prompts'; import { stopInfra, stopWorkers } from '../docker.js'; +import { requireInteractive } from '../tty.js'; const SHANNON_HOME = path.join(os.homedir(), '.shannon'); -export async function uninstall(): Promise { - p.intro('Shannon Uninstall'); +export async function uninstall(yes: boolean): Promise { + const interactive = !yes; + if (interactive) p.intro('Shannon Uninstall'); if (!fs.existsSync(SHANNON_HOME)) { - p.log.info('Nothing to remove. Shannon is not configured on this machine.'); - p.outro('Done.'); + const message = 'Nothing to remove. Shannon is not configured on this machine.'; + if (interactive) { + p.log.info(message); + p.outro('Done.'); + } else { + console.log(message); + } return; } - const confirmed = await p.confirm({ - message: 'This will permanently remove all past scan data, saved configurations, and API keys. Continue?', - }); - if (p.isCancel(confirmed) || !confirmed) { - p.cancel('Aborted.'); - process.exit(0); + if (interactive) { + requireInteractive('uninstall', 'Re-run with --yes to skip this confirmation.'); + const confirmed = await p.confirm({ + message: 'This will permanently remove all past scan data, saved configurations, and API keys. Continue?', + }); + if (p.isCancel(confirmed) || !confirmed) { + p.cancel('Aborted.'); + process.exit(0); + } } // Stop any running containers first @@ -32,6 +42,14 @@ export async function uninstall(): Promise { stopInfra(false); fs.rmSync(SHANNON_HOME, { recursive: true, force: true }); - p.log.success('All Shannon data has been removed.'); - p.outro('Shannon has been uninstalled. Run `npx @keygraph/shannon setup` to start fresh.'); + + const done = 'All Shannon data has been removed.'; + const hint = 'Shannon has been uninstalled. Run `npx @keygraph/shannon setup` to start fresh.'; + if (interactive) { + p.log.success(done); + p.outro(hint); + } else { + console.log(done); + console.log(hint); + } } diff --git a/apps/cli/src/config/resolver.ts b/apps/cli/src/config/resolver.ts index c2125b2..677a63a 100644 --- a/apps/cli/src/config/resolver.ts +++ b/apps/cli/src/config/resolver.ts @@ -88,7 +88,9 @@ function loadTOML(): TOMLConfig | null { const mode = fs.statSync(configPath).mode; if (mode & 0o077) { const actual = (mode & 0o777).toString(8).padStart(3, '0'); - console.error(`\nInsecure permissions (${actual}) on ${configPath}. Run: chmod 600 ${configPath}\n`); + console.error( + `\nYour config file is readable by other users on this machine (${actual}). Lock it down: chmod 600 ${configPath}\n`, + ); process.exit(1); } } diff --git a/apps/cli/src/docker.ts b/apps/cli/src/docker.ts index caef82c..16e4963 100644 --- a/apps/cli/src/docker.ts +++ b/apps/cli/src/docker.ts @@ -13,6 +13,7 @@ import path from 'node:path'; import { setTimeout as sleep } from 'node:timers/promises'; import { fileURLToPath } from 'node:url'; import { getMode } from './mode.js'; +import { INTERNAL_DIR } from './paths.js'; const __dirname = path.dirname(fileURLToPath(import.meta.url)); @@ -116,7 +117,7 @@ export function ensureImage(version: string): void { if (exists) return; if (getMode() === 'local') { - console.log('Worker image not found, building...'); + console.log('Shannon image not found, building...'); buildImage(false); } else { console.log(`Pulling ${image}...`); @@ -270,12 +271,13 @@ export function spawnWorker(opts: WorkerOptions): ChildProcess { args.push('-v', `${opts.workspacesDir}:/app/workspaces`); args.push('-v', `${opts.repo.hostPath}:${opts.repo.containerPath}:ro`); - // Writable overlays: shadow .shannon/ and .playwright/ inside the :ro repo with workspace-backed dirs - const workspacePath = path.join(opts.workspacesDir, opts.workspace); - args.push('-v', `${path.join(workspacePath, 'deliverables')}:${opts.repo.containerPath}/.shannon/deliverables`); - args.push('-v', `${path.join(workspacePath, 'scratchpad')}:${opts.repo.containerPath}/.shannon/scratchpad`); - args.push('-v', `${path.join(workspacePath, '.playwright-cli')}:${opts.repo.containerPath}/.shannon/.playwright-cli`); - args.push('-v', `${path.join(workspacePath, '.playwright')}:${opts.repo.containerPath}/.playwright`); + // Writable overlays: shadow .shannon/ and .playwright/ inside the :ro repo with workspace-backed + // dirs, nested under the run's INTERNAL_DIR. Container paths are unchanged. + const internalPath = path.join(opts.workspacesDir, opts.workspace, INTERNAL_DIR); + args.push('-v', `${path.join(internalPath, 'deliverables')}:${opts.repo.containerPath}/.shannon/deliverables`); + args.push('-v', `${path.join(internalPath, 'scratchpad')}:${opts.repo.containerPath}/.shannon/scratchpad`); + args.push('-v', `${path.join(internalPath, '.playwright-cli')}:${opts.repo.containerPath}/.shannon/.playwright-cli`); + args.push('-v', `${path.join(internalPath, '.playwright')}:${opts.repo.containerPath}/.playwright`); // Local mode: mount prompts for live editing if (opts.promptsDir) { @@ -336,7 +338,7 @@ export function stopWorkers(): void { if (!workers) return; const ids = workers.split('\n').filter(Boolean); - console.log('Stopping worker containers...'); + console.log('Stopping running scans...'); execFileSync('docker', ['stop', ...ids], { stdio: 'inherit' }); } diff --git a/apps/cli/src/index.ts b/apps/cli/src/index.ts index 64c5299..4a38901 100644 --- a/apps/cli/src/index.ts +++ b/apps/cli/src/index.ts @@ -9,9 +9,6 @@ * in the current working directory. */ -import fs from 'node:fs'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; import { build } from './commands/build.js'; import { logs } from './commands/logs.js'; import { setup } from './commands/setup.js'; @@ -21,9 +18,7 @@ import { stop } from './commands/stop.js'; import { uninstall } from './commands/uninstall.js'; import { workspaces } from './commands/workspaces.js'; import { getMode } from './mode.js'; -import { displaySplash } from './splash.js'; - -const __dirname = path.dirname(fileURLToPath(import.meta.url)); +import { getVersion, getVersionLine } from './version.js'; function blockSudo(): void { const isSudo = !!process.env.SUDO_USER; @@ -44,16 +39,6 @@ function blockSudo(): void { process.exit(1); } -function getVersion(): string { - try { - const pkgPath = path.join(__dirname, '..', 'package.json'); - const pkg = JSON.parse(fs.readFileSync(pkgPath, 'utf-8')) as { version?: string }; - return pkg.version || '1.0.0'; - } catch { - return '1.0.0'; - } -} - function showHelp(): void { const mode = getMode(); const prefix = mode === 'local' ? './shannon' : 'npx @keygraph/shannon'; @@ -68,17 +53,17 @@ Usage:${ ${prefix} setup Configure credentials` } ${prefix} start --url --repo [options] Start a pentest scan - ${prefix} stop [--clean] Stop all containers + ${prefix} stop [--clean] [--yes] Stop all running scans ${prefix} workspaces List all workspaces - ${prefix} logs Tail workflow log - ${prefix} status Show running workers${ + ${prefix} logs Show a scan's live log + ${prefix} status Show running scans${ mode === 'local' ? ` ${prefix} build [--no-cache] Build worker image` : ` - ${prefix} uninstall Remove ~/.shannon/ and all data` + ${prefix} uninstall [--yes] Remove ~/.shannon/ and all data` } - ${prefix} info Show splash screen + ${prefix} version Show version ${prefix} help Show this help Options for 'start': @@ -102,7 +87,7 @@ State directory: ./workspaces/` : ` State directory: ~/.shannon/` } -Monitor workflows at http://localhost:8233 +Monitor scans at http://localhost:8233 `); } @@ -209,7 +194,7 @@ switch (command) { break; } case 'stop': - stop(args.includes('--clean')); + stop(args.includes('--clean'), args.includes('--yes') || args.includes('-y')); break; case 'logs': { const workspaceId = args[1]; @@ -242,10 +227,12 @@ switch (command) { console.error('ERROR: uninstall is only available in npx mode.'); process.exit(1); } - uninstall(); + uninstall(args.includes('--yes') || args.includes('-y')); break; - case 'info': - displaySplash(getMode() === 'local' ? undefined : getVersion()); + case 'version': + case '--version': + case '-v': + console.log(getVersionLine()); break; case 'help': case '--help': diff --git a/apps/cli/src/paths.ts b/apps/cli/src/paths.ts index 5ab460a..dcbdc49 100644 --- a/apps/cli/src/paths.ts +++ b/apps/cli/src/paths.ts @@ -14,6 +14,38 @@ export interface MountPair { containerPath: string; } +/** + * Hidden subdirectory inside each run directory that holds all internals + * (deliverables, logs, prompts, session state, browser artifacts). Keeps the + * run folder's top level clean so only the final report is visible. Must match + * INTERNAL_DIR in the worker package. + */ +export const INTERNAL_DIR = '.shannon'; + +/** + * Filename of the human-facing final report surfaced at the run directory root. + * Must match FINAL_REPORT_FILENAME in the worker package. + */ +export const FINAL_REPORT_FILENAME = 'Security-Assessment-Report.md'; + +/** + * Resolve a run-directory file (e.g. session.json, workflow.log), preferring the + * current INTERNAL_DIR location and falling back to the legacy run-root location + * so pre-restructure workspaces keep working. Returns the INTERNAL_DIR path when + * neither exists — the right default for new runs and error messages. + */ +export function resolveRunFile(runDir: string, filename: string): string { + const current = path.join(runDir, INTERNAL_DIR, filename); + if (fs.existsSync(current)) { + return current; + } + const legacy = path.join(runDir, filename); + if (fs.existsSync(legacy)) { + return legacy; + } + return current; +} + /** * Resolve --repo to absolute path and container mount. * Dev mode: bare names (no / or . prefix) check ./repos/ first. diff --git a/apps/cli/src/splash.ts b/apps/cli/src/splash.ts index 004421d..1b41b61 100644 --- a/apps/cli/src/splash.ts +++ b/apps/cli/src/splash.ts @@ -1,14 +1,18 @@ /** * Splash screen display — pure terminal output, no npm dependencies. + * Color escapes are gated on terminal support; the Unicode art is always kept. */ +import { supportsColor } from './tty.js'; + export function displaySplash(version?: string): void { - const GOLD = '\x1b[38;2;244;197;66m'; - const CYAN = '\x1b[36;1m'; - const WHITE = '\x1b[1;37m'; - const GRAY = '\x1b[0;37m'; - const YELLOW = '\x1b[1;33m'; - const RESET = '\x1b[0m'; + const color = supportsColor(); + const GOLD = color ? '\x1b[38;2;244;197;66m' : ''; + const CYAN = color ? '\x1b[36;1m' : ''; + const WHITE = color ? '\x1b[1;37m' : ''; + const GRAY = color ? '\x1b[0;37m' : ''; + const YELLOW = color ? '\x1b[1;33m' : ''; + const RESET = color ? '\x1b[0m' : ''; const B = `${CYAN}\u2551${RESET}`; const S67 = ' '.repeat(67); diff --git a/apps/cli/src/tty.ts b/apps/cli/src/tty.ts new file mode 100644 index 0000000..7c48c79 --- /dev/null +++ b/apps/cli/src/tty.ts @@ -0,0 +1,34 @@ +/** + * Terminal capability detection — output coloring, cursor animation, and + * whether the user can be prompted interactively. + */ + +/** True when stdout is a real terminal — safe for color, cursor moves, and spinners. */ +export function stdoutIsTerminal(): boolean { + return !!process.stdout.isTTY; +} + +/** True when both stdin and stdout are terminals, so interactive prompts can run. */ +function isInteractive(): boolean { + return !!process.stdin.isTTY && !!process.stdout.isTTY; +} + +/** True when color escapes should be emitted. NO_COLOR disables; FORCE_COLOR overrides (0/false/empty = off). */ +export function supportsColor(): boolean { + if (process.env.NO_COLOR !== undefined) return false; + + const force = process.env.FORCE_COLOR; + if (force !== undefined) { + return force !== '0' && force !== 'false' && force !== ''; + } + + return stdoutIsTerminal(); +} + +/** Exit with a clear error when an interactive-only command has no terminal, instead of hanging on a prompt. */ +export function requireInteractive(command: string, alternative: string): void { + if (isInteractive()) return; + console.error(`ERROR: '${command}' needs an interactive terminal.`); + console.error(alternative); + process.exit(1); +} diff --git a/apps/cli/src/version.ts b/apps/cli/src/version.ts new file mode 100644 index 0000000..f21038d --- /dev/null +++ b/apps/cli/src/version.ts @@ -0,0 +1,59 @@ +/** + * Version reporting — mode-aware. + * + * NPX mode: the published package.json version (stamped by CI at release). + * Local mode: the git commit SHA of the checked-out clone (`git-`). + * A clone has no meaningful semver, so the commit is the honest identifier. + */ + +import { execFileSync } from 'node:child_process'; +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { getMode } from './mode.js'; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); + +function readPackageVersion(): string { + try { + const pkgPath = path.join(__dirname, '..', 'package.json'); + const pkg = JSON.parse(fs.readFileSync(pkgPath, 'utf-8')) as { version?: string }; + return pkg.version || '1.0.0'; + } catch { + return '1.0.0'; + } +} + +/** Run a git command in the CLI's own repo; returns trimmed stdout or null on any failure. */ +function git(...args: string[]): string | null { + try { + return execFileSync('git', args, { cwd: __dirname, encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'] }).trim(); + } catch { + return null; + } +} + +function readGitSha(): string | null { + return git('rev-parse', 'HEAD'); +} + +/** + * Version identifier. NPX: package.json version. Local: `git-`, + * falling back to the package version if git is unavailable. + */ +export function getVersion(): string { + if (getMode() !== 'local') return readPackageVersion(); + + const sha = readGitSha(); + if (!sha) return readPackageVersion(); + + return `git-${sha}`; +} + +/** + * Human-facing version line printed by `--version`. + * NPX: `shannon `. Local: `shannon git-`. + */ +export function getVersionLine(): string { + return `shannon ${getVersion()}`; +} diff --git a/apps/worker/src/audit/utils.ts b/apps/worker/src/audit/utils.ts index 1b80e64..97de3de 100644 --- a/apps/worker/src/audit/utils.ts +++ b/apps/worker/src/audit/utils.ts @@ -12,7 +12,7 @@ */ import path from 'node:path'; -import { WORKSPACES_DIR } from '../paths.js'; +import { INTERNAL_DIR, WORKSPACES_DIR } from '../paths.js'; import { ensureDirectory } from '../utils/file-io.js'; export type { SessionMetadata } from '../types/audit.js'; @@ -35,8 +35,9 @@ export function generateSessionIdentifier(sessionMetadata: SessionMetadata): str } /** - * Generate path to audit log directory for a session - * Uses custom outputPath if provided, otherwise defaults to WORKSPACES_DIR + * Generate path to a run directory for a session (its top level). + * Uses custom outputPath if provided, otherwise defaults to WORKSPACES_DIR. + * Only the final report lives here; all internals live under INTERNAL_DIR. */ export function generateAuditPath(sessionMetadata: SessionMetadata): string { const sessionIdentifier = generateSessionIdentifier(sessionMetadata); @@ -44,6 +45,14 @@ export function generateAuditPath(sessionMetadata: SessionMetadata): string { return path.join(baseDir, sessionIdentifier); } +/** + * Generate path to the hidden internals directory inside a run directory. + * Holds logs, prompts, session state, deliverables, and browser artifacts. + */ +export function generateInternalPath(sessionMetadata: SessionMetadata): string { + return path.join(generateAuditPath(sessionMetadata), INTERNAL_DIR); +} + /** * Generate path to agent log file */ @@ -53,25 +62,25 @@ export function generateLogPath( timestamp: number, attemptNumber: number, ): string { - const auditPath = generateAuditPath(sessionMetadata); + const internalPath = generateInternalPath(sessionMetadata); const filename = `${timestamp}_${agentName}_attempt-${attemptNumber}.log`; - return path.join(auditPath, 'agents', filename); + return path.join(internalPath, 'agents', filename); } /** * Generate path to prompt snapshot file */ export function generatePromptPath(sessionMetadata: SessionMetadata, agentName: string): string { - const auditPath = generateAuditPath(sessionMetadata); - return path.join(auditPath, 'prompts', `${agentName}.md`); + const internalPath = generateInternalPath(sessionMetadata); + return path.join(internalPath, 'prompts', `${agentName}.md`); } /** * Generate path to session.json file */ export function generateSessionJsonPath(sessionMetadata: SessionMetadata): string { - const auditPath = generateAuditPath(sessionMetadata); - return path.join(auditPath, 'session.json'); + const internalPath = generateInternalPath(sessionMetadata); + return path.join(internalPath, 'session.json'); } /** @@ -79,29 +88,28 @@ export function generateSessionJsonPath(sessionMetadata: SessionMetadata): strin * validator and consumed by downstream agents via `_shared-session.txt`. */ export function authStateFile(sessionMetadata: SessionMetadata): string { - return path.join(generateAuditPath(sessionMetadata), 'auth-state.json'); + return path.join(generateInternalPath(sessionMetadata), 'auth-state.json'); } /** * Generate path to workflow.log file */ export function generateWorkflowLogPath(sessionMetadata: SessionMetadata): string { - const auditPath = generateAuditPath(sessionMetadata); - return path.join(auditPath, 'workflow.log'); + const internalPath = generateInternalPath(sessionMetadata); + return path.join(internalPath, 'workflow.log'); } /** - * Initialize audit directory structure for a session - * Creates: workspaces/{sessionId}/, agents/, prompts/, deliverables/ + * Initialize audit directory structure for a session. + * Creates: workspaces/{sessionId}/.shannon/{agents,prompts}. The deliverables, + * scratchpad, and browser dirs are created host-side and bind-mounted in. */ export async function initializeAuditStructure(sessionMetadata: SessionMetadata): Promise { - const auditPath = generateAuditPath(sessionMetadata); - const agentsPath = path.join(auditPath, 'agents'); - const promptsPath = path.join(auditPath, 'prompts'); - const deliverablesPath = path.join(auditPath, 'deliverables'); + const internalPath = generateInternalPath(sessionMetadata); + const agentsPath = path.join(internalPath, 'agents'); + const promptsPath = path.join(internalPath, 'prompts'); - await ensureDirectory(auditPath); + await ensureDirectory(internalPath); await ensureDirectory(agentsPath); await ensureDirectory(promptsPath); - await ensureDirectory(deliverablesPath); } diff --git a/apps/worker/src/audit/workflow-logger.ts b/apps/worker/src/audit/workflow-logger.ts index 9637695..a228cb6 100644 --- a/apps/worker/src/audit/workflow-logger.ts +++ b/apps/worker/src/audit/workflow-logger.ts @@ -80,7 +80,7 @@ export class WorkflowLogger { private async writeHeader(): Promise { const lines = [ `================================================================================`, - `Shannon Pentest - Workflow Log`, + `Shannon Pentest - Scan Log`, `================================================================================`, `Workflow ID: ${this.workflowId ?? this.sessionMetadata.id}`, `Target URL: ${this.sessionMetadata.webUrl}`, @@ -337,7 +337,7 @@ export class WorkflowLogger { const lines: string[] = [ '', '================================================================================', - `Workflow ${status}`, + `Scan ${status}`, '────────────────────────────────────────', `Workflow ID: ${this.workflowId ?? this.sessionMetadata.id}`, `Status: ${summary.status}`, diff --git a/apps/worker/src/paths.ts b/apps/worker/src/paths.ts index 70380ae..fb25a70 100644 --- a/apps/worker/src/paths.ts +++ b/apps/worker/src/paths.ts @@ -15,6 +15,36 @@ export const DEFAULT_DELIVERABLES_SUBDIR = '.shannon/deliverables'; /** Default audit log directory */ export const DEFAULT_AUDIT_DIR = './workspaces'; +/** + * Hidden subdirectory inside each run directory that holds all internals + * (logs, prompts, session state, deliverables, browser artifacts). Keeps the + * run folder's top level clean so only the final report is visible. + */ +export const INTERNAL_DIR = '.shannon'; + +/** Filename of the assembled report inside the deliverables dir (internal, source of the surfaced copy) */ +export const ASSEMBLED_REPORT_FILENAME = 'comprehensive_security_assessment_report.md'; + +/** Filename of the human-facing final report surfaced at the run directory root */ +export const FINAL_REPORT_FILENAME = 'Security-Assessment-Report.md'; + +/** + * Resolve the session.json path for a run directory, preferring the current + * `.shannon/` location and falling back to the legacy run-root location so + * pre-restructure workspaces remain listable and resumable. + */ +export function resolveSessionJsonPath(runDir: string): string { + const current = path.join(runDir, INTERNAL_DIR, 'session.json'); + if (fs.existsSync(current)) { + return current; + } + const legacy = path.join(runDir, 'session.json'); + if (fs.existsSync(legacy)) { + return legacy; + } + return current; +} + /** * Resolve the deliverables directory for a given repoPath and optional subdir override. * @param repoPath - Absolute path to the target repository diff --git a/apps/worker/src/services/index.ts b/apps/worker/src/services/index.ts index 48a0019..2d1ef40 100644 --- a/apps/worker/src/services/index.ts +++ b/apps/worker/src/services/index.ts @@ -20,4 +20,4 @@ export type { ContainerDependencies } from './container.js'; export { Container, getContainer, getOrCreateContainer, removeContainer, setContainerFactory } from './container.js'; export { ExploitationCheckerService } from './exploitation-checker.js'; export { loadPrompt } from './prompt-manager.js'; -export { assembleFinalReport, injectModelIntoReport } from './reporting.js'; +export { assembleFinalReport, copyReportToRunRoot, injectModelIntoReport } from './reporting.js'; diff --git a/apps/worker/src/services/reporting.ts b/apps/worker/src/services/reporting.ts index 8919a0d..c298c03 100644 --- a/apps/worker/src/services/reporting.ts +++ b/apps/worker/src/services/reporting.ts @@ -5,7 +5,7 @@ // as published by the Free Software Foundation. import { fs, path } from 'zx'; -import { deliverablesDir } from '../paths.js'; +import { ASSEMBLED_REPORT_FILENAME, deliverablesDir, FINAL_REPORT_FILENAME, resolveSessionJsonPath } from '../paths.js'; import type { ActivityLogger } from '../types/activity-logger.js'; import { ErrorCode } from '../types/errors.js'; import { PentestError } from './error-handling.js'; @@ -68,7 +68,7 @@ export async function assembleFinalReport( } const finalContent = sections.join('\n\n'); - const finalReportPath = path.join(dir, 'comprehensive_security_assessment_report.md'); + const finalReportPath = path.join(dir, ASSEMBLED_REPORT_FILENAME); try { await fs.ensureDir(dir); @@ -97,7 +97,7 @@ export async function injectModelIntoReport( logger: ActivityLogger, ): Promise { // 1. Read session.json to get model information - const sessionJsonPath = path.join(outputPath, 'session.json'); + const sessionJsonPath = resolveSessionJsonPath(outputPath); if (!(await fs.pathExists(sessionJsonPath))) { logger.warn('session.json not found, skipping model injection'); @@ -129,10 +129,7 @@ export async function injectModelIntoReport( logger.info(`Injecting model info into report: ${modelStr}`); // 3. Read the final report - const reportPath = path.join( - deliverablesDir(repoPath, deliverablesSubdir), - 'comprehensive_security_assessment_report.md', - ); + const reportPath = path.join(deliverablesDir(repoPath, deliverablesSubdir), ASSEMBLED_REPORT_FILENAME); if (!(await fs.pathExists(reportPath))) { logger.warn('Final report not found, skipping model injection'); @@ -167,3 +164,26 @@ export async function injectModelIntoReport( // 5. Write modified report back await fs.writeFile(reportPath, reportContent); } + +/** + * Surface the assembled report at the run directory's top level as the single + * human-facing deliverable, so a customer opening the run folder sees only the + * report. The source stays in the deliverables dir (git-checkpointed, used by resume). + */ +export async function copyReportToRunRoot( + repoPath: string, + deliverablesSubdir: string | undefined, + runDir: string, + logger: ActivityLogger, +): Promise { + const source = path.join(deliverablesDir(repoPath, deliverablesSubdir), ASSEMBLED_REPORT_FILENAME); + + if (!(await fs.pathExists(source))) { + logger.warn(`Final report not found, skipping ${FINAL_REPORT_FILENAME}`); + return; + } + + const destination = path.join(runDir, FINAL_REPORT_FILENAME); + await fs.copy(source, destination, { overwrite: true }); + logger.info(`Surfaced report at ${destination}`); +} diff --git a/apps/worker/src/temporal/activities.ts b/apps/worker/src/temporal/activities.ts index 70ceba5..7963f5f 100644 --- a/apps/worker/src/temporal/activities.ts +++ b/apps/worker/src/temporal/activities.ts @@ -22,10 +22,10 @@ import { writePlaywrightStealthConfig } from '../ai/playwright-config-writer.js' import { writeUserSettingsForCodePathAvoids } from '../ai/settings-writer.js'; import { AuditSession } from '../audit/index.js'; import type { ResumeAttempt } from '../audit/metrics-tracker.js'; -import { authStateFile, generateSessionJsonPath, type SessionMetadata } from '../audit/utils.js'; +import { authStateFile, generateAuditPath, generateSessionJsonPath, type SessionMetadata } from '../audit/utils.js'; import type { WorkflowSummary } from '../audit/workflow-logger.js'; import type { CheckpointContext } from '../interfaces/checkpoint-provider.js'; -import { DEFAULT_DELIVERABLES_SUBDIR, deliverablesDir } from '../paths.js'; +import { DEFAULT_DELIVERABLES_SUBDIR, deliverablesDir, resolveSessionJsonPath } from '../paths.js'; import { getContainer, getOrCreateContainer, removeContainer } from '../services/container.js'; import { classifyErrorForTemporal, PentestError } from '../services/error-handling.js'; import { ExploitationCheckerService } from '../services/exploitation-checker.js'; @@ -33,7 +33,7 @@ import { renderFindingsFromQueues } from '../services/findings-renderer.js'; import { executeGitCommandWithRetry } from '../services/git-manager.js'; import { runPreflightChecks } from '../services/preflight.js'; import type { ExploitationDecision, VulnType } from '../services/queue-validation.js'; -import { assembleFinalReport, injectModelIntoReport } from '../services/reporting.js'; +import { assembleFinalReport, copyReportToRunRoot, injectModelIntoReport } from '../services/reporting.js'; import { validateAuthentication } from '../services/validate-authentication.js'; import { AGENTS } from '../session-manager.js'; import type { AgentName } from '../types/agents.js'; @@ -756,8 +756,8 @@ export async function loadResumeState( expectedRepoPath: string, deliverablesSubdir?: string, ): Promise { - // 1. Validate workspace exists - const sessionPath = path.join('./workspaces', workspaceName, 'session.json'); + // 1. Validate workspace exists (prefers .shannon/, falls back to legacy run-root layout) + const sessionPath = resolveSessionJsonPath(path.join('./workspaces', workspaceName)); const exists = await fileExists(sessionPath); if (!exists) { @@ -1055,7 +1055,23 @@ export async function logWorkflowComplete(input: ActivityInput, summary: Workflo // 5. Write completion entry to workflow.log await auditSession.logWorkflowComplete(cumulativeSummary); - // 6. Drop the authenticated browser session + // 6. Surface the final report at the run root. Done here (not in the report phase) + // so it also runs when a resume skips an already-complete report phase. + if (summary.status === 'completed') { + try { + await copyReportToRunRoot( + input.repoPath, + input.deliverablesSubdir, + generateAuditPath(sessionMetadata), + createActivityLogger(), + ); + } catch (error) { + const detail = error instanceof Error ? error.message : String(error); + console.warn(`Failed to surface report at run root: ${detail}`); + } + } + + // 7. Drop the authenticated browser session try { await fs.rm(authStateFile(sessionMetadata), { force: true }); } catch (error) { @@ -1063,7 +1079,7 @@ export async function logWorkflowComplete(input: ActivityInput, summary: Workflo console.warn(`Failed to clean up auth-state.json: ${detail}`); } - // 7. Clean up container + // 8. Clean up container removeContainer(workflowId); } diff --git a/apps/worker/src/temporal/worker.ts b/apps/worker/src/temporal/worker.ts index 5ca35ec..1eadb0a 100644 --- a/apps/worker/src/temporal/worker.ts +++ b/apps/worker/src/temporal/worker.ts @@ -35,7 +35,7 @@ import { bundleWorkflowCode, NativeConnection, Worker } from '@temporalio/worker import dotenv from 'dotenv'; import { sanitizeHostname } from '../audit/utils.js'; import { parseConfig } from '../config-parser.js'; -import { deliverablesDir } from '../paths.js'; +import { ASSEMBLED_REPORT_FILENAME, deliverablesDir, FINAL_REPORT_FILENAME, resolveSessionJsonPath } from '../paths.js'; import type { PipelineConfig, VulnClass } from '../types/config.js'; import { fileExists, readJson } from '../utils/file-io.js'; import * as activities from './activities.js'; @@ -171,7 +171,7 @@ interface WorkspaceResolution { } async function terminateExistingWorkflows(client: Client, workspaceName: string): Promise { - const sessionPath = path.join('./workspaces', workspaceName, 'session.json'); + const sessionPath = resolveSessionJsonPath(path.join('./workspaces', workspaceName)); if (!(await fileExists(sessionPath))) { throw new Error(`Workspace not found: ${workspaceName}\n` + `Expected path: ${sessionPath}`); @@ -192,16 +192,16 @@ async function terminateExistingWorkflows(client: Client, workspaceName: string) const description = await handle.describe(); if (description.status.name === 'RUNNING') { - console.log(`Terminating running workflow: ${wfId}`); + console.log(`Terminating running scan: ${wfId}`); await handle.terminate('Superseded by resume workflow'); terminated.push(wfId); console.log(`Terminated: ${wfId}`); } else { - console.log(`Workflow already ${description.status.name}: ${wfId}`); + console.log(`Scan already ${description.status.name}: ${wfId}`); } } catch (error) { if (error instanceof WorkflowNotFoundError) { - console.log(`Workflow not found (already cleaned up): ${wfId}`); + console.log(`Scan not found (already cleaned up): ${wfId}`); } else { console.log(`Failed to terminate ${wfId}: ${error}`); } @@ -224,7 +224,7 @@ async function resolveWorkspace(client: Client, args: CliArgs): Promise 0) { - console.log(`Terminated ${terminatedWorkflows.length} previous workflow(s)\n`); + console.log(`Terminated ${terminatedWorkflows.length} previous scan(s)\n`); } const session = await readJson(sessionPath); @@ -355,7 +355,9 @@ async function waitForWorkflowResult( if (workspace.isResume) { try { - const session = await readJson(path.join('./workspaces', workspace.sessionId, 'session.json')); + const session = await readJson( + resolveSessionJsonPath(path.join('./workspaces', workspace.sessionId)), + ); console.log(`Cumulative cost: $${session.metrics.total_cost_usd.toFixed(4)}`); } catch { // Non-fatal @@ -393,6 +395,12 @@ function copyDeliverables(repoPath: string, outputPath: string): void { fs.cpSync(src, dest, { recursive: true }); } + // Surface the report under its human-facing name alongside the raw deliverables + const assembledReport = path.join(outputDir, ASSEMBLED_REPORT_FILENAME); + if (fs.existsSync(assembledReport)) { + fs.copyFileSync(assembledReport, path.join(outputPath, FINAL_REPORT_FILENAME)); + } + console.log(`Copied ${files.length} deliverable(s) to ${outputPath}`); } @@ -412,7 +420,7 @@ async function run(): Promise { try { // 3. Bundle workflows and create worker on per-invocation task queue - console.log('Bundling workflows...'); + console.log('Preparing scan...'); const workflowBundle = await bundleWorkflowCode({ workflowsPath: path.join(__dirname, 'workflows.js'), }); diff --git a/apps/worker/src/temporal/workspaces.ts b/apps/worker/src/temporal/workspaces.ts index ab3f39f..4003055 100644 --- a/apps/worker/src/temporal/workspaces.ts +++ b/apps/worker/src/temporal/workspaces.ts @@ -20,7 +20,7 @@ import fs from 'node:fs/promises'; import path from 'node:path'; -import { WORKSPACES_DIR as DEFAULT_WORKSPACES_DIR } from '../paths.js'; +import { WORKSPACES_DIR as DEFAULT_WORKSPACES_DIR, resolveSessionJsonPath } from '../paths.js'; interface SessionJson { session: { @@ -82,7 +82,7 @@ async function listWorkspaces(): Promise { const workspaces: WorkspaceInfo[] = []; for (const entry of entries) { - const sessionPath = path.join(workspacesDir, entry, 'session.json'); + const sessionPath = resolveSessionJsonPath(path.join(workspacesDir, entry)); try { const content = await fs.readFile(sessionPath, 'utf8'); const data = JSON.parse(content) as SessionJson; diff --git a/docs/ai-providers.md b/docs/ai-providers.md index db97b7e..d7b3eca 100644 --- a/docs/ai-providers.md +++ b/docs/ai-providers.md @@ -98,7 +98,7 @@ Shannon supports pointing the SDK at an Anthropic-compatible endpoint with `ANTH > [!IMPORTANT] > Only Claude models are officially supported. Shannon's evaluations, internal testing, and agent harness are optimized for Claude. Smaller or alternative models, including non-Claude models routed through a proxy, may not reliably follow Shannon's instructions or tool-use constraints. Use them at your own risk. -The experimental `claude-code-router` integration is being removed. If you rely on it, migrate to an Anthropic-compatible proxy such as LiteLLM before upgrading. +The experimental `claude-code-router` integration has been removed. If you previously relied on it, migrate to an Anthropic-compatible proxy such as LiteLLM. Run `npx @keygraph/shannon setup` and select **Custom Base URL**, or export variables directly: diff --git a/docs/development.md b/docs/development.md index 01d108e..c2b5af7 100644 --- a/docs/development.md +++ b/docs/development.md @@ -61,6 +61,7 @@ Monitor progress: ```bash npx @keygraph/shannon logs npx @keygraph/shannon status +npx @keygraph/shannon version ``` Source-build equivalents: @@ -68,6 +69,7 @@ Source-build equivalents: ```bash ./shannon logs ./shannon status +./shannon version ``` Open the Temporal Web UI for detailed monitoring: @@ -80,15 +82,15 @@ Stop Shannon: ```bash npx @keygraph/shannon stop -npx @keygraph/shannon stop --clean -npx @keygraph/shannon uninstall +npx @keygraph/shannon stop --clean # confirms first; add --yes (or -y) to skip +npx @keygraph/shannon uninstall # confirms first; add --yes (or -y) to skip ``` Source-build equivalents: ```bash ./shannon stop -./shannon stop --clean +./shannon stop --clean # add --yes (or -y) to skip the confirmation ``` Usage examples: @@ -132,14 +134,16 @@ Results are saved to the workspaces directory: Use `-o ` to copy deliverables to a custom output directory after a run completes. -Output structure: +Output structure — the run directory's top level holds only the final report; everything else is nested under a hidden `.shannon/` directory: ```text workspaces/{hostname}_{sessionId}/ -|-- session.json -|-- workflow.log -|-- agents/ -|-- prompts/ -`-- deliverables/ - `-- comprehensive_security_assessment_report.md +|-- Security-Assessment-Report.md # the final report (the deliverable) +`-- .shannon/ # internals + |-- deliverables/ # report source, per-phase analysis, queues + |-- agents/ # per-agent logs + |-- prompts/ # rendered prompts + |-- scratchpad/ # screenshots, scripts + |-- session.json # resume state + `-- workflow.log ``` diff --git a/docs/workspaces.md b/docs/workspaces.md index c688cfd..a350c1a 100644 --- a/docs/workspaces.md +++ b/docs/workspaces.md @@ -11,6 +11,7 @@ Shannon uses workspaces to store scan state, logs, prompts, and deliverables. Wo - Use `-w ` to give a run a custom name. - To resume a run, pass the same workspace name with `-w`. - Each agent's progress is checkpointed so resumed runs can skip completed work. +- The final report is surfaced at the workspace root as `Security-Assessment-Report.md`. Run internals — deliverables, logs, prompts, and session state — live under a hidden `.shannon/` directory. > [!NOTE] > The URL must match the original workspace URL when resuming. Shannon rejects mismatched URLs to prevent cross-target contamination. diff --git a/llms-full.txt b/llms-full.txt index f5e29c7..142c862 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -322,6 +322,7 @@ Monitor progress: ```bash npx @keygraph/shannon logs npx @keygraph/shannon status +npx @keygraph/shannon version ``` Source-build equivalents: @@ -329,6 +330,7 @@ Source-build equivalents: ```bash ./shannon logs ./shannon status +./shannon version ``` Open the Temporal Web UI for detailed monitoring: @@ -341,15 +343,15 @@ Stop Shannon: ```bash npx @keygraph/shannon stop -npx @keygraph/shannon stop --clean -npx @keygraph/shannon uninstall +npx @keygraph/shannon stop --clean # confirms first; add --yes (or -y) to skip +npx @keygraph/shannon uninstall # confirms first; add --yes (or -y) to skip ``` Source-build equivalents: ```bash ./shannon stop -./shannon stop --clean +./shannon stop --clean # add --yes (or -y) to skip the confirmation ``` Usage examples: @@ -393,16 +395,18 @@ Results are saved to the workspaces directory: Use `-o ` to copy deliverables to a custom output directory after a run completes. -Output structure: +Output structure — the run directory's top level holds only the final report; everything else is nested under a hidden `.shannon/` directory: ```text workspaces/{hostname}_{sessionId}/ -|-- session.json -|-- workflow.log -|-- agents/ -|-- prompts/ -`-- deliverables/ - `-- comprehensive_security_assessment_report.md +|-- Security-Assessment-Report.md # the final report (the deliverable) +`-- .shannon/ # internals + |-- deliverables/ # report source, per-phase analysis, queues + |-- agents/ # per-agent logs + |-- prompts/ # rendered prompts + |-- scratchpad/ # screenshots, scripts + |-- session.json # resume state + `-- workflow.log ``` --- @@ -665,7 +669,7 @@ Shannon supports pointing the SDK at an Anthropic-compatible endpoint with `ANTH > [!IMPORTANT] > Only Claude models are officially supported. Shannon's evaluations, internal testing, and agent harness are optimized for Claude. Smaller or alternative models, including non-Claude models routed through a proxy, may not reliably follow Shannon's instructions or tool-use constraints. Use them at your own risk. -The experimental `claude-code-router` integration is being removed. If you rely on it, migrate to an Anthropic-compatible proxy such as LiteLLM before upgrading. +The experimental `claude-code-router` integration has been removed. If you previously relied on it, migrate to an Anthropic-compatible proxy such as LiteLLM. Run `npx @keygraph/shannon setup` and select **Custom Base URL**, or export variables directly: @@ -793,6 +797,7 @@ Shannon uses workspaces to store scan state, logs, prompts, and deliverables. Wo - Use `-w ` to give a run a custom name. - To resume a run, pass the same workspace name with `-w`. - Each agent's progress is checkpointed so resumed runs can skip completed work. +- The final report is surfaced at the workspace root as `Security-Assessment-Report.md`. Run internals — deliverables, logs, prompts, and session state — live under a hidden `.shannon/` directory. > [!NOTE] > The URL must match the original workspace URL when resuming. Shannon rejects mismatched URLs to prevent cross-target contamination.