From 0e997339b741591992bab304174e2cdac39f2e08 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=85=AC=E6=98=8E?= <83812544+Ed1s0nZ@users.noreply.github.com> Date: Thu, 20 Nov 2025 23:40:36 +0800 Subject: [PATCH] Add files via upload --- README.md | 1572 +++++-------------------------------------------- README_CN.md | 1574 +++++--------------------------------------------- 2 files changed, 266 insertions(+), 2880 deletions(-) diff --git a/README.md b/README.md index 0e6d2217..2eba2d2e 100644 --- a/README.md +++ b/README.md @@ -6,1497 +6,191 @@ [中文](README_CN.md) | [English](README.md) -🚀 **AI-Powered Autonomous Penetration Testing Platform** - Built with Golang, featuring hundreds of built-in security tools, flexible custom tool extensions, and intelligent AI decision-making through MCP protocol, making security testing as simple as a conversation. +CyberStrikeAI is an **AI-native penetration-testing copilot** built in Go. It combines hundreds of security tools, MCP-native orchestration, and an agent that reasons over findings so that a full engagement can be run from a single conversation. -- Web Mode - Preview - -- MCP Stdio Mode - Preview +- Web console + Preview +- MCP stdio mode + Preview +- External MCP servers & attack-chain view + Preview -- External MCP Server Integration (supports stdio and HTTP transport modes) - Preview +## Highlights -- Attack Chain Visualization - Preview +- 🤖 AI decision engine with OpenAI-compatible models (GPT, Claude, DeepSeek, etc.) +- 🔌 Native MCP implementation with HTTP/stdio transports and external MCP federation +- 🧰 100+ prebuilt tool recipes + YAML-based extension system +- 📄 Large-result pagination, compression, and searchable archives +- 🔗 Attack-chain graph, risk scoring, and step-by-step replay +- 🔒 Password-protected web UI, audit logs, and SQLite persistence -## Changelog -- 2025.11.20 oversized tool logs, MCP transcripts, and attachments can be losslessly compressed or summary-first compressed before being sent back to the UI or stored in SQLite. This keeps archives small without losing critical context. -- 2025.11.17 Added attack chain visualization feature: automatically build attack chains from conversations using AI analysis, visualize tool execution flows, vulnerability discovery paths, and relationships between nodes, support interactive graph exploration with risk scoring -- 2025.11.15 Added large result pagination feature: when tool execution results exceed the threshold (default 200KB), automatically save to file and return execution ID, support paginated queries, keyword search, conditional filtering, and regex matching through query_execution_result tool, effectively solving the problem of overly long single responses and improving large file processing capabilities -- 2025.11.15 Added external MCP integration feature: support for integrating external MCP servers to extend tool capabilities, supports both stdio and HTTP transport modes, tool-level enable/disable control, complete configuration guide and management APIs -- 2025.11.14 Performance optimizations: optimized tool lookup from O(n) to O(1) using index map, added automatic cleanup mechanism for execution records to prevent memory leaks, and added pagination support for database queries -- 2025.11.13 Added authentication for the web mode, including automatic password generation and in-app password change -- 2025.11.13 Added `Settings` feature in the frontend -- 2025.11.13 Added MCP Stdio mode support, now seamlessly integrated and usable in code editors, CLI, and automation scripts -- 2025.11.12 Added task stop functionality, optimized frontend +## Basic Usage -## ✨ Features +### Quick Start +1. **Clone & install** + ```bash + git clone https://github.com/Ed1s0nZ/CyberStrikeAI.git + cd CyberStrikeAI-main + go mod download + ``` +2. **Configure OpenAI-compatible access** + Either open the in-app `Settings` panel after launch or edit `config.yaml`: + ```yaml + openai: + api_key: "sk-your-key" + base_url: "https://api.openai.com/v1" + model: "gpt-4o" + auth: + password: "" # empty = auto-generate & log once + session_duration_hours: 12 + security: + tools_dir: "tools" + ``` +3. **Install the tooling you need (optional)** + ```bash + # macOS + brew install nmap sqlmap nuclei httpx gobuster feroxbuster subfinder amass + # Ubuntu/Debian + sudo apt-get install nmap sqlmap nuclei httpx gobuster feroxbuster + ``` + AI automatically falls back to alternatives when a tool is missing. +4. **Launch** + ```bash + chmod +x run.sh && ./run.sh + # or + go run cmd/server/main.go + ``` +5. **Open the console** at http://localhost:8080, log in with the generated password, and start chatting. -### Core Features -- 🤖 **AI Intelligent Agent** - Integrated OpenAI-compatible API (supports GPT, Claude, DeepSeek, etc.), AI autonomously makes decisions and executes security tests -- 🧠 **Intelligent Decision Engine** - AI analyzes targets and automatically selects optimal testing strategies and tool combinations -- ⚡ **Autonomous Execution** - AI agent automatically invokes security tools without human intervention -- 🔄 **Adaptive Adjustment** - AI automatically adjusts testing strategies based on tool execution results and discovered vulnerabilities -- 📝 **Intelligent Summary** - When maximum iterations are reached, AI automatically summarizes test results and provides next-step execution plans -- 💬 **Conversational Interface** - Natural language conversation interface with streaming output (SSE), real-time execution viewing -- 📊 **Conversation History Management** - Complete conversation history records, supports viewing, deletion, and management -- ⚙️ **Visual Configuration Management** - Web interface for system settings, supports real-time loading and saving configurations with required field validation -- 📄 **Large Result Pagination** - When tool execution results exceed the threshold, automatically save to file, support paginated queries, keyword search, conditional filtering, and regex matching, effectively solving the problem of overly long single responses, with examples for various tools (head, tail, grep, sed, etc.) for segmented reading -- 🔗 **Attack Chain Visualization** - Automatically build and visualize attack chains from conversations, showing tool execution flows, vulnerability discovery paths, and relationships between targets, tools, vulnerabilities, and discoveries, with AI-powered analysis and interactive graph exploration +### Core Workflows +- **Conversation testing** – Natural-language prompts trigger toolchains with streaming SSE output. +- **Tool monitor** – Inspect running jobs, execution logs, and large-result attachments. +- **History & audit** – Every conversation and tool invocation is stored in SQLite with replay. +- **Settings** – Tweak provider keys, MCP enablement, tool toggles, and agent iteration limits. -### Tool Integration -- 🔌 **MCP Protocol Support** - Complete MCP protocol implementation, supports tool registration, invocation, and monitoring -- 📡 **Dual Transport Modes** - Supports both HTTP and stdio transport methods, seamlessly usable in web applications and IDEs -- 🛠️ **Flexible Tool Configuration** - Supports loading tool configurations from directories (YAML), easy to extend and maintain -- 📈 **Real-time Monitoring** - Monitors execution status, results, call counts, and statistics of all tools -- 🔍 **Automatic Vulnerability Analysis** - Automatically analyzes tool output, extracts and categorizes discovered vulnerabilities +### Built-in Safeguards +- Required-field validation prevents accidental blank API credentials. +- Auto-generated strong passwords when `auth.password` is empty. +- Unified auth middleware for every web/API call (Bearer token flow). +- Timeout and sandbox guards per tool, plus structured logging for triage. -### Technical Features -- 🚀 **Streaming Output** - Supports Server-Sent Events (SSE) for real-time streaming output, enhancing user experience -- 💾 **Data Persistence** - SQLite database stores conversation history and process details -- 📝 **Detailed Logging** - Structured logging for easy debugging and troubleshooting -- 🔒 **Secure Execution** - Tool execution isolation, error handling, and timeout control -- 🔐 **Password-Protected Web Interface** - Unified authentication middleware secures every API call with configurable session duration +## Advanced Usage -## 📁 Project Structure +### Tool Orchestration & Extensions +- **YAML recipes** in `tools/*.yaml` describe commands, arguments, prompts, and metadata. +- **Directory hot-reload** – pointing `security.tools_dir` to a folder is usually enough; inline definitions in `config.yaml` remain supported for quick experiments. +- **Large-result pagination** – outputs beyond 200 KB are stored as artifacts retrievable through the `query_execution_result` tool with paging, filters, and regex search. +- **Result compression** – multi-megabyte logs can be summarized or losslessly compressed before persisting to keep SQLite lean. -``` -CyberStrikeAI/ -├── cmd/ -│ ├── server/ -│ │ └── main.go # Program entry point, starts HTTP server -│ ├── mcp-stdio/ -│ │ └── main.go # MCP stdio mode entry (for Cursor and other IDE integration) -│ └── test-config/ -│ └── main.go # Configuration testing tool -├── internal/ -│ ├── agent/ # AI agent module -│ │ └── agent.go # Agent Loop implementation, handles AI conversations and tool calls -│ ├── app/ # Application initialization -│ │ └── app.go # Main application logic, route setup -│ ├── config/ # Configuration management -│ │ └── config.go # Configuration loading and tool configuration management -│ ├── database/ # Database module -│ │ ├── database.go # Database connection and table structure -│ │ └── conversation.go # Conversation and message data access -│ ├── handler/ # HTTP handlers -│ │ ├── agent.go # Agent Loop API handling -│ │ ├── conversation.go # Conversation history API handling -│ │ └── monitor.go # Monitoring API handling -│ ├── logger/ # Logging system -│ │ └── logger.go # Structured logging wrapper -│ ├── mcp/ # MCP protocol implementation -│ │ ├── server.go # MCP server core logic -│ │ └── types.go # MCP protocol type definitions -│ └── security/ # Security tool executor -│ └── executor.go # Tool execution and parameter building -├── tools/ # Tool configuration directory -│ ├── nmap.yaml # nmap tool configuration -│ ├── sqlmap.yaml # sqlmap tool configuration -│ ├── nikto.yaml # nikto tool configuration -│ ├── dirb.yaml # dirb tool configuration -│ ├── exec.yaml # System command execution tool configuration -│ └── README.md # Tool configuration documentation -├── web/ # Web frontend -│ ├── static/ # Static resources -│ │ ├── css/ -│ │ │ └── style.css # Stylesheet -│ │ └── js/ -│ │ └── app.js # Frontend JavaScript logic -│ └── templates/ # HTML templates -│ └── index.html # Main page template -├── data/ # Data directory (auto-created) -│ └── conversations.db # SQLite database file -├── config.yaml # Main configuration file -├── go.mod # Go module dependencies -├── go.sum # Go dependency checksums -├── run.sh # Startup script -└── README.md # Project documentation -``` +**Creating a custom tool (typical flow)** +1. Copy an existing YAML file from `tools/` (for example `tools/sample.yaml`). +2. Update `name`, `command`, `args`, and `short_description`. +3. Describe positional or flag parameters in `parameters[]` so the agent knows how to build CLI arguments. +4. Provide a longer `description`/`notes` block if the agent needs extra context or post-processing tips. +5. Restart the server or reload configuration; the new tool becomes available immediately and can be enabled/disabled from the Settings panel. -## Quick Start +### Attack-Chain Intelligence +- AI parses each conversation to assemble targets, tools, vulnerabilities, and relationships. +- The web UI renders the chain as an interactive graph with severity scoring and step replay. +- Export the chain or raw findings to external reporting pipelines. -### Prerequisites +### MCP Everywhere +- **Web mode** – ships with HTTP MCP server automatically consumed by the UI. +- **MCP stdio mode** – `go run cmd/mcp-stdio/main.go` exposes the agent to Cursor/CLI. +- **External MCP federation** – register third-party MCP servers (HTTP or stdio) from the UI, toggle them per engagement, and monitor their health and call volume in real time. -- Go 1.21 or higher -- OpenAI API Key (or other OpenAI-compatible API, such as DeepSeek, Claude, etc.) -- Security tools (optional): Install corresponding security tools based on your needs, the system supports hundreds of tools +### Automation Hooks +- **REST APIs** – everything the UI uses (auth, conversations, tool runs, monitor) is available over JSON. +- **Task control** – pause/resume/stop long scans, re-run steps with new params, or stream transcripts. +- **Audit & security** – rotate passwords via `/api/auth/change-password`, enforce short-lived sessions, and restrict MCP ports at the network layer when exposing the service. -### Installation Steps - -1. **Clone the repository** -```bash -git clone https://github.com/Ed1s0nZ/CyberStrikeAI.git -cd CyberStrikeAI-main -``` - -2. **Install dependencies** -```bash -go mod download -``` - -3. **Configuration** - -#### Method 1: Configure via Web Interface (Recommended) - -After starting the server, click the "Settings" button in the top-right corner of the web interface to configure: -- **OpenAI Configuration**: API Key, Base URL, Model (required fields marked with *) -- **MCP Tool Configuration**: Enable/disable tools -- **Agent Configuration**: Maximum iterations, etc. - -Configuration is automatically saved to the `config.yaml` file. Opening settings automatically loads values from the current configuration file. - -#### Method 2: Edit Configuration File Directly - -Edit the `config.yaml` file and set your API configuration: +## Configuration Reference ```yaml -# OpenAI-compatible API configuration (supports OpenAI, DeepSeek, Claude, etc.) -openai: - api_key: "sk-your-api-key-here" # Replace with your API Key - base_url: "https://api.openai.com/v1" # Or use other compatible API addresses - model: "gpt-4" # Or "deepseek-chat", "gpt-3.5-turbo", etc. - -# Authentication configuration auth: - password: "" # Leave empty to auto-generate a strong password on first launch - session_duration_hours: 12 # Login validity (hours) - -# Server configuration + password: "change-me" + session_duration_hours: 12 server: host: "0.0.0.0" port: 8080 - -# Database configuration +log: + level: "info" + output: "stdout" +mcp: + enabled: true + host: "0.0.0.0" + port: 8081 +openai: + api_key: "sk-xxx" + base_url: "https://api.deepseek.com/v1" + model: "deepseek-chat" database: path: "data/conversations.db" - -# Security tool configuration security: - tools_dir: "tools" # Tool configuration file directory + tools_dir: "tools" ``` -**Supported API Providers:** -- OpenAI: `https://api.openai.com/v1` -- DeepSeek: `https://api.deepseek.com/v1` -- Other OpenAI-compatible API services - -**Note**: API Key, Base URL, and Model are required fields and must be configured for the system to run properly. When configuring in the web interface, these fields are validated, and error prompts are displayed if not filled. - -4. **Install Security Tools (Optional)** - -Install corresponding security tools based on your needs. The system supports hundreds of tools, and you can selectively install based on actual requirements: - -```bash -# macOS (using Homebrew) -brew install nmap sqlmap nuclei httpx gobuster feroxbuster subfinder amass - -# Ubuntu/Debian -sudo apt-get install nmap sqlmap nuclei httpx gobuster feroxbuster - -# Or use Docker to run tools -# Or use official installation methods for each tool -``` - -Ubuntu security tools batch installation script: `https://github.com/Ed1s0nZ/sec_tools/blob/main/install_tools_ubuntu.sh` - -**Note**: Not all tools need to be installed. AI will automatically select available tools based on your testing needs. If a tool is not installed, AI will try to use alternative tools. - -5. **Start the Server** - -#### Method 1: Using Startup Script (Recommended) -```bash -chmod +x run.sh -./run.sh -``` - -#### Method 2: Direct Run -```bash -go run cmd/server/main.go -``` - -#### Method 3: Build and Run -```bash -go build -o cyberstrike-ai cmd/server/main.go -./cyberstrike-ai -``` - -#### Method 4: Specify Configuration File -```bash -go run cmd/server/main.go -config /path/to/config.yaml -``` - -6. **Access the Application** -Open your browser and visit: http://localhost:8080 - -You will see: -- **Conversation Testing** - Chat with AI for penetration testing -- **Tool Monitoring** - View tool execution status and results -- **Conversation History** - Manage historical conversation records -- **System Settings** - Configure API keys, tool enable status, etc. (click the settings button in the top-right corner) - -**First-time Usage Tips**: -- Before starting, please click the "Settings" button in the top-right corner to configure your API Key -- API Key, Base URL, and Model are required fields (marked with *), must be filled for normal use -- Configuration is automatically saved to the `config.yaml` file -- Opening settings automatically loads the latest configuration from the current configuration file -- If `auth.password` is empty, the server generates a random strong password on first launch, writes it back to `config.yaml`, and prints it in the terminal with a security warning -- The web UI prompts for this password when you first open it; you can change it anytime in **Settings → Security** - -## ⚙️ Configuration - -### Web Interface Configuration Management - -The system provides a visual configuration management interface. You can access it as follows: - -1. **Open Settings**: Click the "Settings" button in the top-right corner of the web interface -2. **Load Configuration**: Opening settings automatically loads current configuration from `config.yaml` -3. **Modify Configuration**: - - **OpenAI Configuration**: Modify API Key, Base URL, Model (required fields marked with *) - - **MCP Tool Configuration**: Enable or disable tools, supports search and batch operations - - **Agent Configuration**: Set maximum iterations and other parameters -4. **Save Configuration**: Click the "Apply Configuration" button, configuration is saved to `config.yaml` and takes effect immediately -5. **Validation Prompts**: Error prompts are displayed when required fields are not filled, and error fields are highlighted - -**Configuration Validation Rules**: -- API Key, Base URL, and Model are required fields -- Validation is performed automatically when saving, and saving is blocked with error prompts if required fields are not filled - -### Complete Configuration Example - -```yaml -# Authentication -auth: - password: "change-me" # Web login password - session_duration_hours: 12 # Session validity (hours) - -# Server configuration -server: - host: "0.0.0.0" # Listen address - port: 8080 # HTTP service port - -# Log configuration -log: - level: "info" # Log level: debug, info, warn, error - output: "stdout" # Output location: stdout, stderr, or file path - -# MCP protocol configuration -mcp: - enabled: true # Whether to enable MCP server - host: "0.0.0.0" # MCP server listen address - port: 8081 # MCP server port - -# AI model configuration (supports OpenAI-compatible API) -openai: - api_key: "sk-xxx" # API key - base_url: "https://api.deepseek.com/v1" # API base URL - model: "deepseek-chat" # Model name - -# Database configuration -database: - path: "data/conversations.db" # SQLite database path - -# Security tool configuration -security: - # Recommended: Load tool configurations from directory - tools_dir: "tools" # Tool configuration file directory (relative to config file location) - - # Backward compatibility: Can also define tools directly in main config file - # tools: - # - name: "nmap" - # command: "nmap" - # args: ["-sT", "-sV", "-sC"] - # description: "Network scanning tool" - # enabled: true -``` - -### Tool Configuration Methods - -**Method 1: Using Tool Directory (Recommended)** - -Create independent YAML configuration files for each tool in the `tools/` directory, for example `tools/nmap.yaml`: +### Tool Definition Example (`tools/nmap.yaml`) ```yaml name: "nmap" command: "nmap" args: ["-sT", "-sV", "-sC"] enabled: true - -short_description: "Network scanning tool for discovering network hosts, open ports, and services" - -description: | - Network mapping and port scanning tool for discovering hosts, services, and open ports in a network. - +short_description: "Network mapping & service fingerprinting" parameters: - name: "target" type: "string" - description: "Target IP address or domain name" + description: "IP or domain" required: true position: 0 - format: "positional" - - name: "ports" type: "string" - description: "Port range, e.g.: 1-1000" - required: false flag: "-p" - format: "flag" + description: "Range, e.g. 1-1000" ``` -**Method 2: Define in Main Configuration File** +## Project Layout -Define tool configurations directly in `config.yaml` under `security.tools`. +``` +CyberStrikeAI/ +├── cmd/ # Server, MCP stdio entrypoints, tooling +├── internal/ # Agent, MCP core, handlers, security executor +├── web/ # Static SPA + templates +├── tools/ # YAML tool recipes (100+ examples provided) +├── img/ # Docs screenshots & diagrams +├── config.yaml # Runtime configuration +├── run.sh # Convenience launcher +└── README*.md +``` -**Note**: If both `tools_dir` and `tools` are configured, tools in `tools_dir` take priority. +## Basic Usage Examples -### Authentication & Security - -- **Login Workflow**: Every web/API request (except `/api/auth/login`) is protected by a unified middleware. Obtain a token through `/api/auth/login` with the configured password, then include `Authorization: Bearer ` in subsequent requests. -- **Automatic Password Generation**: When `auth.password` is empty, the server generates a 24-character strong password on startup, writes it back to `config.yaml`, and prints the password with bilingual security warnings in the terminal. -- **Session Control**: Sessions expire according to `auth.session_duration_hours`. After expiration or password change, clients must log in again. -- **Password Rotation**: Use **Settings → Security** in the web UI (or call `/api/auth/change-password`) to update the password. The change revokes all existing sessions instantly. -- **MCP Port**: The standalone MCP server (default `8081`) remains authentication-free for IDE integrations. Restrict network access to this port if required. - -## 🚀 Usage Examples - -### Conversational Penetration Testing - -In the "Conversation Testing" tab of the web interface, you can use natural language to chat with AI: - -#### 1. Network Scanning ``` Scan open ports on 192.168.1.1 -``` -Or more detailed instructions: -``` -Perform a comprehensive port scan on 192.168.1.1, focusing on ports 80, 443, 22, 21 +Perform a comprehensive port scan on 192.168.1.1 focusing on 80,443,22 +Check if https://example.com/page?id=1 is vulnerable to SQL injection +Scan https://example.com for hidden directories and outdated software +Enumerate subdomains for example.com, then run nuclei against the results ``` -#### 2. SQL Injection Detection -``` -Check if https://example.com/page?id=1 has SQL injection vulnerabilities -``` - -#### 3. Web Vulnerability Scanning -``` -Scan https://example.com for web server vulnerabilities, including common security issues -``` - -#### 4. Directory Scanning -``` -Scan https://example.com for hidden directories and files -``` - -#### 5. Comprehensive Security Testing -``` -Perform a comprehensive security assessment on example.com, including port scanning, web vulnerability detection, and directory enumeration -``` - -#### 6. Multi-step Testing -``` -First scan open ports on 192.168.1.1, then perform vulnerability scanning on discovered web services -``` - -### Post-Exploitation Testing - -After gaining initial access, you can use post-exploitation tools for privilege escalation, lateral movement, and persistence: - -#### 1. Linux Privilege Escalation Enumeration -``` -Use linpeas to perform privilege escalation checks on the target Linux system -``` - -#### 2. Windows Privilege Escalation Enumeration -``` -Use winpeas to perform privilege escalation checks on the target Windows system -``` - -#### 3. Active Directory Attack Path Analysis -``` -Use bloodhound to analyze Active Directory attack paths -``` - -#### 4. Credential Extraction -``` -Use mimikatz to extract credential information from Windows systems -``` - -#### 5. Lateral Movement -``` -Use impacket toolset for network protocol attacks and lateral movement -``` - -#### 6. Backdoor Generation -``` -Use msfvenom to generate reverse shell payloads -``` - -### CTF Competition Support - -The system has built-in rich CTF tools supporting various CTF problem types: - -#### 1. Steganography Analysis -``` -Use stegsolve to analyze image steganography -Use zsteg to detect LSB steganography -``` - -#### 2. Password Cracking -``` -Use hashcat to crack hash values -Use john to crack password files -Use fcrackzip to crack ZIP file passwords -Use pdfcrack to crack PDF file passwords -``` - -#### 3. Binary Analysis -``` -Use gdb to debug binary files -Use radare2 for reverse engineering analysis -Use strings to extract strings from binary files -``` - -#### 4. Hash Identification -``` -Use hash-identifier to identify hash types -``` - -#### 5. Data Conversion and Analysis -``` -Use cyberchef for various data conversions and analysis -Use xxd to view file hexadecimal content -``` - -#### 6. Comprehensive CTF Problem Solving -``` -Analyze this CTF problem: Given a file containing steganography and encryption, find the flag -``` - -### Monitor Tool Execution - -In the "Tool Monitoring" tab, you can: - -- 📊 **Execution Statistics** - View call counts, success/failure statistics for all tools -- 📝 **Execution Records** - View detailed tool execution history, including parameters, results, and duration -- 🔍 **Vulnerability List** - Automatically extracted and categorized discovered vulnerabilities -- ⏱️ **Real-time Status** - Real-time viewing of currently executing tool status - -### Conversation History Management - -- 📚 **View History** - Browse all historical conversation records -- 🔍 **Search Conversations** - Search conversations by title -- 🗑️ **Delete Conversations** - Clean up unwanted conversation records -- 📄 **View Details** - View complete messages and tool execution processes of conversations - -## Results - -### Conversation Results - ![Preview](./img/效果1.png) - ![Preview](./img/效果2.png) - -### MCP Calls - ![Preview](./img/MCP.png) - -### Call Chain - ![Preview](./img/调用链1.png) - -## 📡 API Endpoints - -### Agent Loop API - -#### Standard Request (Synchronous) - -**POST** `/api/agent-loop` - -Request body: -```json -{ - "message": "Scan 192.168.1.1", - "conversationId": "optional-conversation-id" // Optional, for continuing conversations -} -``` - -Response: -```json -{ - "response": "AI response content", - "mcpExecutionIds": ["exec-id-1", "exec-id-2"], - "conversationId": "conversation-id", - "time": "2024-01-01T00:00:00Z" -} -``` - -Usage example: -```bash -curl -X POST http://localhost:8080/api/agent-loop \ - -H "Content-Type: application/json" \ - -d '{"message": "Scan 192.168.1.1"}' -``` - -#### Streaming Request (Recommended, Real-time Output) - -**POST** `/api/agent-loop/stream` - -Uses Server-Sent Events (SSE) to return execution process in real-time. - -Request body: -```json -{ - "message": "Scan 192.168.1.1", - "conversationId": "optional-conversation-id" -} -``` - -Event types: -- `progress` - Progress update -- `iteration` - Iteration start -- `thinking` - AI thinking content -- `tool_call` - Tool call start -- `tool_result` - Tool execution result -- `response` - Final response -- `error` - Error information -- `done` - Complete - -Usage example (JavaScript): -```javascript -const eventSource = new EventSource('/api/agent-loop/stream', { - method: 'POST', - body: JSON.stringify({ message: 'Scan 192.168.1.1' }) -}); - -eventSource.onmessage = (event) => { - const data = JSON.parse(event.data); - console.log(data.type, data.message, data.data); -}; -``` - -### Conversation History API - -#### Create Conversation - -**POST** `/api/conversations` - -Request body: -```json -{ - "title": "Conversation title" -} -``` - -#### Get Conversation List - -**GET** `/api/conversations` - -Query parameters: -- `limit` - Limit return count (optional) -- `offset` - Offset (optional) - -#### Get Single Conversation - -**GET** `/api/conversations/:id` - -Returns complete conversation information, including all messages. - -#### Delete Conversation - -**DELETE** `/api/conversations/:id` - -### Monitoring API - -#### Get All Monitoring Information - -**GET** `/api/monitor` - -Returns all execution records, statistics, and vulnerability lists. - -#### Get Specific Execution Record - -**GET** `/api/monitor/execution/:id` - -Returns tool execution details for the specified ID. - -#### Get Statistics - -**GET** `/api/monitor/stats` - -Returns call statistics for all tools: -```json -{ - "nmap": { - "toolName": "nmap", - "totalCalls": 10, - "successCalls": 9, - "failedCalls": 1, - "lastCallTime": "2024-01-01T00:00:00Z" - } -} -``` - -#### Get Vulnerability List - -**GET** `/api/monitor/vulnerabilities` - -Returns all discovered vulnerabilities: -```json -{ - "total": 5, - "severityCount": { - "critical": 0, - "high": 2, - "medium": 2, - "low": 1 - }, - "vulnerabilities": [...] -} -``` - -### MCP Protocol Endpoint - -**POST** `/api/mcp` - -MCP protocol endpoint, supports JSON-RPC 2.0 format requests. - -## 🔌 MCP Protocol - -This project fully implements the MCP (Model Context Protocol) protocol, supporting the following features: - -### Transport Modes - -CyberStrikeAI supports two MCP transport modes: - -#### 1. HTTP Mode (Default) -- Communication via HTTP POST requests -- Suitable for web applications and other HTTP clients -- Default listen address: `0.0.0.0:8081/mcp` -- Accessible via `/api/mcp` endpoint -- 🌐 Remote-friendly: expose a single endpoint that IDEs, web apps, or automation running on other machines can reach over the network. -- 🧩 Easy reuse: no extra binaries—just point any HTTP-capable client (curl, Postman, cloud automations) to the service. -- 🔁 Always-on workflow: runs together with the main web server, so the same deployment handles UI, API, and MCP traffic. - -#### MCP HTTP Mode (IDE Integration) - -You can connect IDEs such as Cursor or Claude Desktop directly to the built-in HTTP MCP server: - -1. Ensure `mcp.enabled: true` in `config.yaml`, adjust `host`/`port` if you need a different bind address. -2. Start the main server (`./run.sh` or `go run cmd/server/main.go`). The MCP endpoint will be available at `http://:/mcp` (default `http://127.0.0.1:8081/mcp` when running locally). -3. In Cursor, open **Settings → Tools & MCP → Add Custom MCP**, choose HTTP, and set: - - `Base URL`: `http://127.0.0.1:8081/mcp` - - Optional headers (e.g., `Authorization`) if you enforce authentication in front of MCP. -4. Alternatively create `.cursor/mcp.json` in your project: - ```json - { - "mcpServers": { - "cyberstrike-ai-http": { - "transport": "http", - "url": "http://127.0.0.1:8081/mcp" - } - } - } - ``` -5. Restart the IDE; CyberStrikeAI’s tools will appear under the MCP tool list. - -> 🔐 **Security tip**: if you expose the MCP HTTP port beyond localhost, protect it with firewalls or authentication to prevent misuse. - -#### 2. stdio Mode (New) -- Communication via standard input/output (stdio) -- Suitable for Cursor, Claude Desktop, and other IDE integrations -- Fully compliant with JSON-RPC 2.0 specification -- Supports string, number, and null types for id field -- Properly handles notification messages -- 🔒 Isolated execution: the stdio binary is built and launched separately, so you can run it with least-privilege policies and tighter filesystem/network permissions. -- 🪟 No network exposure: data stays inside the local process boundary—perfect when you do not want an HTTP port listening on your machine. -- 🧰 Editor-first experience: Cursor, Claude Desktop, and other IDEs expect stdio transports for local tooling, enabling plug-and-play integration with minimal setup. -- 🧱 Defense in depth: using both transports in parallel lets you pick the safest option per workflow—stdio for local, HTTP for remote or shared deployments. - -#### Mode comparison: pick what fits your workflow - -| Aspect | `mcp-http` | `mcp-stdio` | -|---------------------|-----------------------------------------------|------------------------------------------------------------------| -| Transport | HTTP/HTTPS over the network | Standard input/output streams | -| Deployment | Runs inside the main server process | Compiled as a standalone binary | -| Isolation & safety | Depends on server hardening (firewall, auth) | Sandboxed by OS process boundaries, no socket exposure | -| Remote access | ✅ Accessible across machines | ❌ Local only (unless tunneled manually) | -| IDE integration | Works with HTTP-capable clients | Native fit for Cursor/Claude Desktop stdio connectors | -| Best use case | Remote automations, shared services | Local development, high-trust / locked-down environments | - -### Supported Methods - -- `initialize` - Initialize connection, negotiate protocol version and capabilities -- `tools/list` - List all available tools -- `tools/call` - Call specified tool and execute -- `prompts/list` - List available prompt templates -- `prompts/get` - Get prompt template content -- `resources/list` - List available resources -- `resources/read` - Read resource content -- `sampling/request` - Sampling request (placeholder implementation) -- `notifications/initialized` - Initialization complete notification (stdio mode) - -### Tool Execution Mechanism - -- Tool calls are executed synchronously, ensuring errors are correctly returned -- Each tool call creates an execution record containing: - - Execution ID (unique identifier) - - Tool name and parameters - - Execution status (running, completed, failed) - - Start and end time - - Execution result or error information -- System automatically tracks execution statistics for all tools - -### MCP stdio Mode (Cursor IDE Integration) - -stdio mode allows you to directly use all CyberStrikeAI security tools in Cursor IDE. - -#### Compile stdio Mode Program - -```bash -# Execute in project root directory -go build -o cyberstrike-ai-mcp cmd/mcp-stdio/main.go -``` - -#### Configure in Cursor - -**Method 1: Via UI Configuration** - -1. Open Cursor Settings → **Tools & MCP** -2. Click **Add Custom MCP** -3. Configure as follows (replace with your actual path): - -```json -{ - "mcpServers": { - "cyberstrike-ai": { - "command": "/absolute/path/to/cyberstrike-ai-mcp", - "args": [ - "--config", - "/absolute/path/to/config.yaml" - ] - } - } -} -``` - -**Method 2: Via Project Configuration File** - -Create `.cursor/mcp.json` file in project root directory: - -```json -{ - "mcpServers": { - "cyberstrike-ai": { - "command": "/Users/yourname/Downloads/CyberStrikeAI-main/cyberstrike-ai-mcp", - "args": [ - "--config", - "/Users/yourname/Downloads/CyberStrikeAI-main/config.yaml" - ] - } - } -} -``` - -**Important Notes:** -- ✅ Use absolute paths: `command` and config file paths must use absolute paths -- ✅ Executable permissions: Ensure compiled program has execute permissions (Linux/macOS) -- ✅ Restart Cursor: Need to restart Cursor after configuration for it to take effect - -After configuration, restart Cursor, and you can directly use all security tools in chat! - -#### stdio Mode Features - -- ✅ Fully compliant with JSON-RPC 2.0 specification -- ✅ Supports string, number, and null types for id field -- ✅ Properly handles notification messages -- ✅ Log output to stderr, doesn't interfere with JSON-RPC communication -- ✅ Completely independent from HTTP mode, can be used simultaneously - -### MCP HTTP Mode Usage Examples - -#### Initialize Connection - -```bash -curl -X POST http://localhost:8080/api/mcp \ - -H "Content-Type: application/json" \ - -d '{ - "jsonrpc": "2.0", - "id": "1", - "method": "initialize", - "params": { - "protocolVersion": "2024-11-05", - "capabilities": {}, - "clientInfo": { - "name": "test-client", - "version": "1.0.0" - } - } - }' -``` - -#### List Tools - -```bash -curl -X POST http://localhost:8080/api/mcp \ - -H "Content-Type: application/json" \ - -d '{ - "jsonrpc": "2.0", - "id": "2", - "method": "tools/list" - }' -``` - -#### Call Tool - -```bash -curl -X POST http://localhost:8080/api/mcp \ - -H "Content-Type: application/json" \ - -d '{ - "jsonrpc": "2.0", - "id": "3", - "method": "tools/call", - "params": { - "name": "nmap", - "arguments": { - "target": "192.168.1.1", - "ports": "1-1000" - } - } - }' -``` - -## 🔗 External MCP Integration - -CyberStrikeAI supports integrating external MCP servers to extend tool capabilities. External MCP tools are automatically registered in the system, and AI can call them just like built-in tools. - -### Configuration Methods - -#### Method 1: Configure via Web Interface (Recommended) - -1. After starting the server, access the web interface -2. Click the "Settings" button in the top-right corner -3. Find the "External MCP" configuration section in the settings -4. Click the "Add External MCP" button -5. Fill in the configuration information: - - **Name**: Unique identifier for the external MCP server (e.g., `hexstrike-ai`) - - **Transport Mode**: Choose `stdio` or `http` - - **Description**: Optional, used to describe the MCP server's functionality - - **Timeout**: Tool execution timeout in seconds, default 300 seconds - - **Enable Status**: Whether to enable this external MCP immediately -6. Fill in the corresponding configuration based on transport mode: - - **stdio Mode**: - - **Command**: Startup command for the MCP server (e.g., `python3`) - - **Args**: Startup arguments array (e.g., `["/path/to/mcp_server.py", "--arg", "value"]`) - - **HTTP Mode**: - - **URL**: HTTP endpoint address of the MCP server (e.g., `http://127.0.0.1:8888`) -7. Click the "Save" button, configuration will be automatically saved to `config.yaml` -8. The system will automatically connect to the external MCP server and load its tools - -#### Method 2: Edit Configuration File Directly - -Add `external_mcp` configuration in `config.yaml`: - -```yaml -# External MCP Configuration -external_mcp: - servers: - # External MCP server name (unique identifier) - hexstrike-ai: - # stdio mode configuration - command: python3 - args: - - /path/to/hexstrike_mcp.py - - --server - - 'http://127.0.0.1:8888' - - # Or HTTP mode configuration (choose one) - # transport: http - # url: http://127.0.0.1:8888 - - # Common configuration - description: HexStrike AI v6.0 - Advanced Cybersecurity Automation Platform - timeout: 300 # Timeout in seconds - external_mcp_enable: true # Whether to enable - - # Tool-level control (optional) - tool_enabled: - nmap_scan: true - sqlmap_scan: true - # ... other tools -``` - -### Transport Mode Description - -#### stdio Mode - -Communicates with external MCP servers via standard input/output (stdio), suitable for locally running MCP servers. - -**Configuration Example:** -```yaml -external_mcp: - servers: - my-mcp-server: - command: python3 - args: - - /path/to/mcp_server.py - - --config - - /path/to/config.json - description: My Custom MCP Server - timeout: 300 - external_mcp_enable: true -``` - -**Features:** -- ✅ Local process communication, no network port required -- ✅ High security, data doesn't traverse network -- ✅ Suitable for local development and testing - -#### HTTP Mode - -Communicates with external MCP servers via HTTP requests, suitable for remote MCP servers or scenarios requiring cross-network access. - -**Configuration Example:** -```yaml -external_mcp: - servers: - remote-mcp-server: - transport: http - url: http://192.168.1.100:8888 - description: Remote MCP Server - timeout: 300 - external_mcp_enable: true -``` - -**Features:** -- ✅ Supports remote access -- ✅ Suitable for distributed deployment -- ✅ Easy to integrate with existing HTTP services - -### Tool Naming Convention - -External MCP tools in the system use the naming format: `{mcp-server-name}::{tool-name}` - -For example: -- External MCP server name: `hexstrike-ai` -- Tool name: `nmap_scan` -- Full tool name in system: `hexstrike-ai::nmap_scan` - -AI will automatically recognize and use the full tool name when calling. - -### Tool Enable/Disable Control - -#### Global Enable/Disable - -Control the enable status of the entire external MCP server via the `external_mcp_enable` field: -- `true`: Enabled, system will automatically connect and load tools -- `false`: Disabled, system will not connect to this server - -#### Tool-Level Control - -Precisely control the enable status of each tool via the `tool_enabled` field: - -```yaml -external_mcp: - servers: - hexstrike-ai: - # ... other configuration - tool_enabled: - nmap_scan: true # Enable this tool - sqlmap_scan: false # Disable this tool - nuclei_scan: true # Enable this tool -``` - -- If a tool is not listed in `tool_enabled`, it is enabled by default -- If a tool is set to `false`, it won't appear in the tool list and AI cannot call it - -### Managing External MCP - -#### Via Web Interface - -1. **View External MCP List**: View all configured external MCP servers in the settings interface -2. **Start/Stop**: Start or stop external MCP server connections at any time -3. **Edit Configuration**: Modify external MCP configuration information -4. **Delete Configuration**: Remove unnecessary external MCP servers - -#### Via API - -- **Get External MCP List**: `GET /api/external-mcp` -- **Add External MCP**: `POST /api/external-mcp` -- **Update External MCP**: `PUT /api/external-mcp/:name` -- **Delete External MCP**: `DELETE /api/external-mcp/:name` -- **Start External MCP**: `POST /api/external-mcp/:name/start` -- **Stop External MCP**: `POST /api/external-mcp/:name/stop` - -### Monitoring and Statistics - -External MCP tool execution records and statistics are automatically recorded in the system: - -- **Execution Records**: View execution history of all external MCP tools in the "Tool Monitoring" page -- **Execution Statistics**: The execution statistics panel displays call counts and success/failure statistics for external MCP tools -- **Real-time Monitoring**: Real-time viewing of external MCP tool execution status - -### Troubleshooting - -**Issue: External MCP Cannot Connect** - -- ✅ Check if `command` and `args` configuration are correct (stdio mode) -- ✅ Check if `url` configuration is correct and accessible (HTTP mode) -- ✅ Check if the external MCP server is running normally -- ✅ View server logs for detailed error information -- ✅ Check network connection (HTTP mode) -- ✅ Check firewall settings (HTTP mode) - -**Issue: External MCP Tools Not Displayed** - -- ✅ Confirm `external_mcp_enable: true` -- ✅ Check `tool_enabled` configuration to ensure tools are not disabled -- ✅ Confirm external MCP server has successfully connected -- ✅ View server logs to confirm if tools have been loaded - -**Issue: External MCP Tool Execution Failed** - -- ✅ Check if external MCP server is running normally -- ✅ View tool execution logs for detailed error information -- ✅ Check if timeout setting is reasonable -- ✅ Confirm external MCP server supports the tool call format - -### Best Practices - -1. **Naming Convention**: Use meaningful names to identify external MCP servers, avoid conflicts -2. **Timeout Settings**: Set timeout reasonably based on tool execution time -3. **Tool Control**: Use `tool_enabled` to precisely control needed tools, avoid loading too many unnecessary tools -4. **Security Considerations**: HTTP mode is recommended to use intranet addresses or configure appropriate access control -5. **Monitoring Management**: Regularly check external MCP connection status and execution statistics - -## 🛠️ Security Tool Support - -### Tool Overview - -The system currently integrates **hundreds of security tools**, covering the following categories: - -- **Network Scanning Tools** - nmap, masscan, rustscan, arp-scan, nbtscan, etc. -- **Web Application Scanning** - sqlmap, nikto, dirb, gobuster, feroxbuster, ffuf, httpx, etc. -- **Vulnerability Scanning** - nuclei, wpscan, wafw00f, dalfox, xsser, etc. -- **Subdomain Enumeration** - subfinder, amass, findomain, dnsenum, fierce, etc. -- **API Security** - graphql-scanner, arjun, api-fuzzer, api-schema-analyzer, etc. -- **Container Security** - trivy, clair, docker-bench-security, kube-bench, kube-hunter, etc. -- **Cloud Security** - prowler, scout-suite, cloudmapper, pacu, terrascan, checkov, etc. -- **Binary Analysis** - gdb, radare2, ghidra, objdump, strings, binwalk, etc. -- **Exploitation** - metasploit, msfvenom, pwntools, ropper, ropgadget, etc. -- **Password Cracking** - hashcat, john, hashpump, etc. -- **Forensics** - volatility, volatility3, foremost, steghide, exiftool, etc. -- **Post-Exploitation Tools** - linpeas, winpeas, mimikatz, bloodhound, impacket, responder, etc. -- **CTF Tools** - stegsolve, zsteg, hash-identifier, fcrackzip, pdfcrack, cyberchef, etc. -- **System Tools** - exec, create-file, delete-file, list-files, modify-file, etc. - -### Main Tool Examples - -- **nmap** - Network port scanning and service identification - - Features: Host discovery, port scanning, service version detection, OS identification - - Configuration: `tools/nmap.yaml` - -- **sqlmap** - Automated SQL injection detection and exploitation tool - - Features: Automatic SQL injection detection, database fingerprinting, data extraction - - Configuration: `tools/sqlmap.yaml` - -- **nuclei** - Fast vulnerability scanner - - Features: Template-based vulnerability scanning, large-scale scanning support - - Configuration: `tools/nuclei.yaml` - -- **httpx** - HTTP probing tool - - Features: HTTP/HTTPS probing, status code detection, title extraction - - Configuration: `tools/httpx.yaml` - -- **exec** - System command execution tool - - Features: Execute arbitrary system commands (use with caution) - - Configuration: `tools/exec.yaml` - - ⚠️ Warning: This tool can execute arbitrary commands, please ensure secure use - -For the complete tool list, please check the `tools/TOOLS_LIST.md` file. - -### Adding New Tools - -To help the AI reason about custom tools without ambiguity, each `tools/*.yaml` file must describe the tool metadata, default behaviour, and runtime parameters in detail. The loader validates every field on startup—tools with missing required fields are skipped and reported in the logs. - -> Need the full checklist? See `tools/README.md` (Chinese) or `tools/README_EN.md` (English) for the complete field reference, templates, and best practices. - -#### Top-level fields (tool scope) - -| Field | Required | Type | Description | -|-------|----------|------|-------------| -| `name` | ✅ | string | Unique identifier. Use lowercase letters, numbers, and hyphens for easy reference in conversations. | -| `command` | ✅ | string | Executable or script to run. Must be resolvable from `$PATH` or provided as an absolute path. | -| `enabled` | ✅ | bool | Registers the tool with MCP. When `false`, the tool is hidden from the UI and AI runtime. | -| `description` | ✅ | string | Full Markdown description. Powers MCP `resources/read` and gives the AI rich context. | -| `short_description` | Optional | string | 20–50 character summary shown in tool lists. If omitted, the loader extracts the start of `description`. | -| `args` | Optional | string[] | Static arguments prepended to every invocation, often used for default scan profiles. | -| `parameters` | Optional | array | List of user-supplied parameters. See the next section for field definitions. | -| `arg_mapping` | Optional | string | Mapping strategy (`auto`/`manual`/`template`). Defaults to `auto`; usually no change required. | - -> Tip: prefer plain ASCII/UTF-8 content and keep YAML indentation consistent to avoid parse errors. - -#### Parameter objects (`parameters[]`) - -Each parameter definition may include: - -- `name` (required): Key used in both command construction and MCP JSON Schema. -- `type` (required): `string`, `int`/`integer`, `bool`/`boolean`, `array`, etc. -- `description` (required): Use Markdown to explain purpose, accepted formats, example values, and safety notes. -- `required`: Boolean flag. When `true`, the executor will stop with an error if the caller omits the value. -- `default`: Default value applied when the caller does not provide one. -- `flag`: CLI switch such as `-u` or `--url`. Combined with `format` to decide how the argument is rendered. -- `position`: Zero-based integer for positional arguments. Positional arguments are appended after flag arguments in ascending order. -- `format`: Controls rendering strategy: - - `flag` (default): Emits `--flag value` or `-f value`. - - `combined`: Emits `--flag=value`. - - `positional`: Inserts the value directly according to `position`. - - `template`: Uses the `template` string with placeholders. -- `template`: Active when `format: "template"`. Supports `{flag}`, `{value}`, and `{name}` placeholders. -- `options`: Array of allowed values. Populates MCP schema `enum` entries and drives AI suggestions. - -#### Reserved parameter names - -- `additional_args`: Lets users append raw CLI fragments in one field. The executor tokenises the string and appends it to the command. -- `scan_type`: Primarily for tools like `nmap` to replace default scan switches (e.g., `-sV -sC`). -- `action`: Consumed exclusively inside server-side logic for tools requiring sub-commands. Not passed to the CLI. - -These names are handled specially by the executor—reuse them instead of creating near-duplicates. - -#### Recommended authoring workflow - -1. Create a new YAML file under `tools/`, for example `tools/mytool.yaml`. -2. Fill out the top-level fields, list common parameters with helpful defaults, and document examples inside `description`. -3. Run `go run cmd/test-config/main.go` to lint configurations locally. -4. Restart the server (or trigger a reload) so the frontend settings panel and MCP registry pick up the new tool. - -#### Method 2: Add to main configuration file - -If you prefer inline definitions, replicate the exact structure under `security.tools` in `config.yaml`. When both `tools_dir` and inline `tools` are present, files from the directory take precedence. - -### Tool Parameter Configuration - -Tool parameters support the following formats: - -- **positional** - Positional parameters, added to command in order -- **flag** - Flag parameters, format: `--flag value` or `-f value` -- **combined** - Combined format, format: `--flag=value` -- **template** - Template format, uses custom template string - -Supported parameter types: -- `string` - String -- `int` / `integer` - Integer -- `bool` / `boolean` - Boolean -- `array` - Array (automatically converted to comma-separated string) - -## 🔧 Troubleshooting - -### API Connection Issues - -**Problem: Cannot connect to OpenAI API** - -- ✅ Check if API Key is correctly configured in `config.yaml` -- ✅ Check if network connection is normal -- ✅ Verify `base_url` configuration is correct -- ✅ Confirm API provider supports the model you're using -- ✅ Check API quota and balance are sufficient - -**Problem: API returns 401 or 403 error** - -- ✅ Verify API Key is valid -- ✅ Check if API Key has permission to access specified model -- ✅ Confirm API provider access restrictions - -### Tool Execution Issues - -**Problem: Tool execution fails or command not found** - -- ✅ Ensure corresponding security tools are installed: - ```bash - # Check if tools are installed - which nmap sqlmap nikto dirb - ``` -- ✅ Check if tools are in system PATH -- ✅ Some tools may require root privileges (e.g., nmap SYN scan) -- ✅ Check server logs for detailed error information - -**Problem: Tool execution timeout** - -- ✅ Some tools (e.g., nmap full port scan) may take a long time -- ✅ Check network connection and target response -- ✅ Consider using smaller scan ranges - -**Problem: Tool parameter errors** - -- ✅ Check parameter definitions in tool configuration files -- ✅ View actual commands in tool execution logs -- ✅ Refer to tool official documentation to verify parameter format - -### Server Issues - -**Problem: Frontend cannot load** - -- ✅ Check if server is running normally: - ```bash - curl http://localhost:8080 - ``` -- ✅ Check if port 8080 is occupied: - ```bash - lsof -i :8080 - ``` -- ✅ Check browser console error messages -- ✅ Check firewall settings - -**Problem: Database errors** - -- ✅ Ensure `data/` directory has write permissions -- ✅ Check if database file is corrupted -- ✅ Delete database file to let system recreate (will lose historical data) - -### Configuration Issues - -**Problem: Tools not loaded** - -- ✅ Check if `tools_dir` configuration is correct -- ✅ Verify tool configuration file format is correct (YAML syntax) -- ✅ Check tool loading information in startup logs -- ✅ Ensure `enabled: true` in tool configuration - -**Problem: MCP server cannot start** - -- ✅ Check if MCP port (default 8081) is occupied -- ✅ Verify `enabled: true` in MCP configuration -- ✅ Check MCP server startup information in logs - -**Problem: MCP stdio mode cannot connect in Cursor** - -- ✅ Check if `cyberstrike-ai-mcp` program path is correct (use absolute path) -- ✅ Check if program has execute permissions (Linux/macOS): `chmod +x cyberstrike-ai-mcp` -- ✅ Check if `config.yaml` configuration file path is correct -- ✅ Check Cursor log output (usually in Cursor developer tools) -- ✅ Ensure `security.tools_dir` configuration in config file is correct - -**Problem: Tool list is empty in Cursor** - -- ✅ Ensure `security.tools_dir` configuration in `config.yaml` is correct -- ✅ Ensure tool configuration files are in specified directory -- ✅ Check tool configuration file format is correct (YAML syntax) -- ✅ Check program logs (stderr output) - -**Problem: Tool execution fails in Cursor** - -- ✅ Ensure corresponding security tools are installed in the system -- ✅ Check if tools are in system PATH -- ✅ Check program logs (stderr output) -- ✅ Try running tool commands directly in terminal to confirm tools are available - -### Log Debugging - -Enable detailed logging: -```yaml -log: - level: "debug" # Change to debug level - output: "stdout" -``` - -View real-time logs: -```bash -# If using run.sh -./run.sh | tee cyberstrike.log - -# Or run directly -go run cmd/server/main.go 2>&1 | tee cyberstrike.log -``` - -## Security Considerations - -⚠️ **Important Notes**: - -- Only test systems you own or have authorization for -- Comply with relevant laws and regulations -- Recommended to use in isolated test environments -- Do not use in production environments -- Some security tools may require root privileges - -## 💻 Development Guide - -### Project Architecture +## Advanced Playbooks ``` -┌─────────────┐ -│ Web UI │ ← User interface (HTML/CSS/JS) -└──────┬──────┘ - │ HTTP -┌──────▼─────────────────────────────────────┐ -│ Gin HTTP Server │ -│ ┌──────────────────────────────────────┐ │ -│ │ Handlers (agent, conversation, etc) │ │ -│ └──────┬───────────────────────────────┘ │ -│ │ │ -│ ┌──────▼──────────┐ ┌─────────────────┐ │ -│ │ Agent Module │ │ MCP Server │ │ -│ │ (AI Loop) │◄─┤ (Tool Manager) │ │ -│ └──────┬──────────┘ └────────┬────────┘ │ -│ │ │ │ -│ ┌──────▼──────────┐ ┌────────▼────────┐ │ -│ │ OpenAI API │ │ Security Executor│ │ -│ └─────────────────┘ └────────┬────────┘ │ -│ │ │ -│ ┌────────▼────────┐ │ -│ │ Security Tools │ │ -│ │ (nmap, sqlmap) │ │ -│ └─────────────────┘ │ -└─────────────────────────────────────────────┘ - │ -┌────────▼────────┐ -│ SQLite Database│ ← Conversation history and message storage -└─────────────────┘ +Load the recon-engagement template, run amass/subfinder, then brute-force dirs on every live host. +Use external Burp-based MCP server for authenticated traffic replay, then pass findings back for graphing. +Compress the 5 MB nuclei report, summarize critical CVEs, and attach the artifact to the conversation. +Build an attack chain for the latest engagement and export the node list with severity >= high. ``` -### Core Module Descriptions +## Changelog (Recent) -- **Agent Module** (`internal/agent/agent.go`) - - Implements Agent Loop, handles AI conversations and tool call decisions - - Supports multi-turn conversations and tool call chains - - Handles tool execution errors and retry logic +- 2025-11-20 – Added automatic compression/summarization for oversized tool logs and MCP transcripts. +- 2025-11-17 – Introduced AI-built attack-chain visualization with interactive graph and risk scoring. +- 2025-11-15 – Delivered large-result pagination, advanced filtering, and external MCP federation. +- 2025-11-14 – Optimized tool lookups (O(1)), execution record cleanup, and DB pagination. +- 2025-11-13 – Added web authentication, settings UI, and MCP stdio mode integration. -- **MCP Server** (`internal/mcp/server.go`) - - Implements MCP protocol server - - Manages tool registration and invocation - - Tracks tool execution status and statistics +--- -- **Security Executor** (`internal/security/executor.go`) - - Executes security tool commands - - Builds tool parameters - - Parses tool output - -- **Database** (`internal/database/`) - - SQLite database operations - - Conversation and message management - - Process detail storage - -### Adding New Tools - -#### Recommended: Use Tool Configuration Files - -1. Create tool configuration file in `tools/` directory (e.g., `tools/mytool.yaml`) -2. Define tool parameters and descriptions -3. Restart server, tool automatically loads - -#### Advanced: Custom Parameter Building Logic - -If a tool requires special parameter handling, you can add it in the `buildCommandArgs` method of `internal/security/executor.go`: - -```go -case "mytool": - // Custom parameter building logic - if target, ok := args["target"].(string); ok { - cmdArgs = append(cmdArgs, "--target", target) - } -``` - -### Build and Deployment - -#### Local Build - -```bash -go build -o cyberstrike-ai cmd/server/main.go -``` - -#### Cross-compilation - -```bash -# Linux -GOOS=linux GOARCH=amd64 go build -o cyberstrike-ai-linux cmd/server/main.go - -# macOS -GOOS=darwin GOARCH=amd64 go build -o cyberstrike-ai-macos cmd/server/main.go - -# Windows -GOOS=windows GOARCH=amd64 go build -o cyberstrike-ai.exe cmd/server/main.go -``` - -#### Docker Deployment (Example) - -```dockerfile -FROM golang:1.21-alpine AS builder -WORKDIR /app -COPY . . -RUN go build -o cyberstrike-ai cmd/server/main.go - -FROM alpine:latest -RUN apk --no-cache add ca-certificates nmap sqlmap nikto dirb -WORKDIR /root/ -COPY --from=builder /app/cyberstrike-ai . -COPY --from=builder /app/config.yaml . -COPY --from=builder /app/tools ./tools -COPY --from=builder /app/web ./web -EXPOSE 8080 -CMD ["./cyberstrike-ai"] -``` - -### Code Contribution - -Welcome to submit Issues and Pull Requests! - -Contribution Guidelines: -1. Fork this project -2. Create a feature branch (`git checkout -b feature/AmazingFeature`) -3. Commit your changes (`git commit -m 'Add some AmazingFeature'`) -4. Push to the branch (`git push origin feature/AmazingFeature`) -5. Open a Pull Request - -## 📋 Tech Stack - -- **Backend Framework**: Gin (Go Web Framework) -- **Database**: SQLite3 -- **Logging**: Zap (Uber's structured logging) -- **Configuration**: YAML -- **Protocol**: MCP (Model Context Protocol) -- **Frontend**: HTML/CSS/JavaScript (native, no framework dependencies) - -## 🔐 Security Considerations - -⚠️ **Important Notes**: - -- ⚠️ **Only test systems you own or have authorization for** -- ⚠️ **Comply with relevant laws, regulations, and ethical guidelines** -- ⚠️ **Recommended to use in isolated test environments** -- ⚠️ **Do not use in production environments** -- ⚠️ **Some security tools may require root privileges, use with caution** -- ⚠️ **exec tool can execute arbitrary system commands, security risk exists** -- ⚠️ **Properly store API keys, do not commit to code repositories** - -## ⚙️ Advanced Features - -### AI Iteration Mechanism - -- **Maximum Iterations**: System supports multiple rounds of AI iterations, ensuring complex testing tasks can be completed -- **Intelligent Summary**: When maximum iterations are reached, AI automatically summarizes all test results, discovered issues, and completed work -- **Next Steps Plan**: If testing is incomplete, AI provides detailed next-step execution plans to guide subsequent testing - -### Tool Execution Optimization - -- **Error Handling**: When tool execution fails, AI automatically analyzes error causes and tries alternative solutions -- **Parameter Optimization**: AI automatically optimizes tool parameters based on target characteristics, improving testing efficiency -- **Result Analysis**: Automatically analyzes tool output, extracts key information and vulnerabilities - -## 📄 License - -This project is for learning and research purposes only. - -## 🤝 Contributing - -Welcome to submit Issues and Pull Requests! - -If you find bugs or have feature suggestions, please: -1. Check existing Issues to avoid duplicates -2. Create detailed Issue descriptions of problems or suggestions -3. When submitting Pull Requests, please include clear descriptions - -## 📞 Support - -For questions or help, please: -- Check the troubleshooting section of this documentation -- Check project Issues -- Submit new Issues describing your problems - -## 🙏 Acknowledgments - -Thanks to all contributors and the open-source community for support! +Need help or want to contribute? Open an issue or PR—community tooling additions are welcome! diff --git a/README_CN.md b/README_CN.md index 7f1bd64f..b5639f48 100644 --- a/README_CN.md +++ b/README_CN.md @@ -4,1500 +4,192 @@ # CyberStrikeAI -[English](README.md) | [中文](README_CN.md) +[中文](README_CN.md) | [English](README.md) -🚀 **AI自主渗透测试平台** - 基于Golang构建，内置上百个安全工具，支持灵活扩展自定义工具，通过MCP协议实现AI智能决策与自动化执行，让安全测试像对话一样简单。 -- web模式 - Preview - -- mcp-stdio / mcp-http 模式 - Preview - -- 外部MCP服务器集成（支持 stdio 和 HTTP 传输模式） - Preview +CyberStrikeAI 是一款 **AI 原生渗透测试协同体**，以 Go 编写，内置上百款安全工具，完整支持 MCP 协议，能够让智能体按照对话指令自主规划、执行并总结一次完整的安全测试流程。 -- 攻击链可视化 - Preview - -## 更新日志 -- 2025.11.20 对超大日志、MCP 对话和附件自动执行无损压缩或“摘要+原文”分层压缩，显著降低数据库与共享文件体积，同时保留定位漏洞所需的上下文 -- 2025.11.17 新增攻击链可视化功能：基于AI分析自动从对话中构建攻击链，可视化展示工具执行流程、漏洞发现路径和节点间关联关系，支持交互式图谱探索和风险评分 -- 2025.11.15 新增大结果分段读取功能：当工具执行结果超过阈值（默认200KB）时，自动保存到文件并返回执行ID，支持通过 query_execution_result 工具进行分页查询、关键词搜索、条件过滤和正则表达式匹配，有效解决单次返回过长的问题，提升大文件处理能力 -- 2025.11.15 新增外部 MCP 接入功能：支持接入外部 MCP 服务器扩展工具能力，支持 stdio 和 HTTP 两种传输模式，支持工具级别的启用/禁用控制，提供完整的配置指南和管理接口 -- 2025.11.14 性能优化：工具查找从 O(n) 优化为 O(1)（使用索引映射），添加执行记录自动清理机制防止内存泄漏，数据库查询支持分页加载 -- 2025.11.13 Web 端新增统一鉴权，支持自动生成强密码与前端修改密码 -- 2025.11.13 在前端新增`设置`功能 -- 2025.11.13 新增 MCP Stdio 模式支持，现可在代码编辑器、CLI 及自动化脚本等多种场景下，无缝集成并使用全套安全工具 -- 2025.11.12 增加了任务停止功能，优化前端 +- Web 控制台 + Preview +- MCP stdio 模式 + Preview +- 外部 MCP 联邦 & 攻击链视图 + Preview +## 特性速览 -## ✨ 功能特性 +- 🤖 兼容 OpenAI/DeepSeek/Claude 等模型的智能决策引擎 +- 🔌 原生 MCP 协议，支持 HTTP / stdio 以及外部 MCP 接入 +- 🧰 100+ 现成工具模版 + YAML 扩展能力 +- 📄 大结果分页、压缩与全文检索 +- 🔗 攻击链可视化、风险打分与步骤回放 +- 🔒 Web 登录保护、审计日志、SQLite 持久化 -### 核心功能 -- 🤖 **AI智能代理** - 集成OpenAI兼容API（支持GPT、Claude、DeepSeek等），AI自主决策和执行安全测试 -- 🧠 **智能决策引擎** - AI分析目标并自动选择最佳测试策略和工具组合 -- ⚡ **自主执行** - AI代理自动调用安全工具，无需人工干预 -- 🔄 **自适应调整** - 根据工具执行结果和发现的漏洞，AI自动调整测试策略 -- 📝 **智能总结** - 达到最大迭代次数时，AI自动总结测试结果并提供下一步执行计划 -- 💬 **对话式交互** - 自然语言对话界面，支持流式输出（SSE），实时查看执行过程 -- 📊 **对话历史管理** - 完整的对话历史记录，支持查看、删除和管理 -- ⚙️ **可视化配置管理** - Web界面配置系统设置，支持实时加载和保存配置，必填项验证 -- 📄 **大结果分段读取** - 当工具执行结果超过阈值时自动保存，支持分页查询、关键词搜索、条件过滤和正则表达式匹配，有效解决单次返回过长问题，提供多种工具（head、tail、grep、sed等）的分段读取示例 -- 🔗 **攻击链可视化** - 基于AI分析自动从对话中构建攻击链，可视化展示工具执行流程、漏洞发现路径以及目标、工具、漏洞、发现之间的关联关系，支持交互式图谱探索和风险评分 +## 基础使用 -### 工具集成 -- 🔌 **MCP协议支持** - 完整实现MCP协议，支持工具注册、调用、监控 -- 📡 **双传输模式** - 支持HTTP和stdio两种传输方式，可在Web应用和IDE中无缝使用 -- 🛠️ **灵活工具配置** - 支持从目录加载工具配置（YAML），易于扩展和维护 -- 📈 **实时监控** - 监控所有工具的执行状态、结果、调用次数和统计信息 -- 🔍 **漏洞自动分析** - 自动分析工具输出，提取和分类发现的漏洞 +### 快速上手 +1. **获取代码并安装依赖** + ```bash + git clone https://github.com/Ed1s0nZ/CyberStrikeAI.git + cd CyberStrikeAI-main + go mod download + ``` +2. **配置模型与鉴权** + 启动后在 Web 端 `Settings` 填写，或直接编辑 `config.yaml`： + ```yaml + openai: + api_key: "sk-your-key" + base_url: "https://api.openai.com/v1" + model: "gpt-4o" + auth: + password: "" # 为空则首次启动自动生成强口令 + session_duration_hours: 12 + security: + tools_dir: "tools" + ``` +3. **按需安装安全工具（可选）** + ```bash + # macOS + brew install nmap sqlmap nuclei httpx gobuster feroxbuster subfinder amass + # Ubuntu/Debian + sudo apt-get install nmap sqlmap nuclei httpx gobuster feroxbuster + ``` + 未安装的工具会自动跳过或改用替代方案。 +4. **启动服务** + ```bash + chmod +x run.sh && ./run.sh + # 或 + go run cmd/server/main.go + ``` +5. **浏览器访问** http://localhost:8080 ，使用日志中提示的密码登录并开始对话。 -### 技术特性 -- 🚀 **流式输出** - 支持Server-Sent Events (SSE)实时流式输出，提升用户体验 -- 💾 **数据持久化** - SQLite数据库存储对话历史和过程详情 -- 📝 **详细日志** - 结构化日志记录，便于调试和问题排查 -- 🔒 **安全执行** - 工具执行隔离，错误处理和超时控制 -- 🔐 **登录鉴权保护** - Web 端与 API 统一鉴权，中间件校验会话，并支持可配置会话有效期 +### 常用流程 +- **对话测试**：自然语言触发多步工具编排，SSE 实时输出。 +- **工具监控**：查看任务队列、执行日志、大文件附件。 +- **会话历史**：所有对话与工具调用保存在 SQLite，可随时重放。 +- **可视化配置**：在界面中切换模型、启停工具、设置迭代次数等。 -## 📁 项目结构 +### 默认安全措施 +- 设置面板内置必填校验，防止漏配 API Key/Base URL/模型。 +- `auth.password` 为空时自动生成 24 位强口令并写回 `config.yaml`。 +- 所有 API（除登录外）都需携带 Bearer Token，统一鉴权中间件拦截。 +- 每个工具执行都带有超时、日志和错误隔离。 -``` -CyberStrikeAI/ -├── cmd/ -│ ├── server/ -│ │ └── main.go # 程序入口，启动HTTP服务器 -│ ├── mcp-stdio/ -│ │ └── main.go # MCP stdio模式入口（用于Cursor等IDE集成） -│ └── test-config/ -│ └── main.go # 配置测试工具 -├── internal/ -│ ├── agent/ # AI代理模块 -│ │ └── agent.go # Agent Loop实现，处理AI对话和工具调用 -│ ├── app/ # 应用初始化 -│ │ └── app.go # 应用主逻辑，路由设置 -│ ├── config/ # 配置管理 -│ │ └── config.go # 配置加载和工具配置管理 -│ ├── database/ # 数据库模块 -│ │ ├── database.go # 数据库连接和表结构 -│ │ └── conversation.go # 对话和消息数据访问 -│ ├── handler/ # HTTP处理器 -│ │ ├── agent.go # Agent Loop API处理 -│ │ ├── conversation.go # 对话历史API处理 -│ │ └── monitor.go # 监控API处理 -│ ├── logger/ # 日志系统 -│ │ └── logger.go # 结构化日志封装 -│ ├── mcp/ # MCP协议实现 -│ │ ├── server.go # MCP服务器核心逻辑 -│ │ └── types.go # MCP协议类型定义 -│ └── security/ # 安全工具执行器 -│ └── executor.go # 工具执行和参数构建 -├── tools/ # 工具配置文件目录 -│ ├── nmap.yaml # nmap工具配置 -│ ├── sqlmap.yaml # sqlmap工具配置 -│ ├── nikto.yaml # nikto工具配置 -│ ├── dirb.yaml # dirb工具配置 -│ ├── exec.yaml # 系统命令执行工具配置 -│ └── README.md # 工具配置说明 -├── web/ # Web前端 -│ ├── static/ # 静态资源 -│ │ ├── css/ -│ │ │ └── style.css # 样式文件 -│ │ └── js/ -│ │ └── app.js # 前端JavaScript逻辑 -│ └── templates/ # HTML模板 -│ └── index.html # 主页面模板 -├── data/ # 数据目录（自动创建） -│ └── conversations.db # SQLite数据库文件 -├── config.yaml # 主配置文件 -├── go.mod # Go模块依赖 -├── go.sum # Go依赖校验和 -├── run.sh # 启动脚本 -└── README.md # 项目说明文档 -``` +## 进阶使用 -## 快速开始 +### 工具编排与扩展 +- `tools/*.yaml` 定义命令、参数、提示词与元数据，可热加载。 +- `security.tools_dir` 指向目录即可批量启用；仍支持在主配置里内联定义。 +- **大结果分页**：超过 200KB 的输出会保存为附件，可通过 `query_execution_result` 工具分页、过滤、正则检索。 +- **结果压缩/摘要**：多兆字节日志可先压缩或生成摘要再写入 SQLite，减小档案体积。 -### 前置要求 +**自定义工具的一般步骤** +1. 复制 `tools/` 下现有示例（如 `tools/sample.yaml`）。 +2. 修改 `name`、`command`、`args`、`short_description` 等基础信息。 +3. 在 `parameters[]` 中声明位置参数或带 flag 的参数，方便智能体自动拼装命令。 +4. 视需要补充 `description` 或 `notes`，给 AI 额外上下文或结果解读提示。 +5. 重启服务或在界面中重新加载配置，新工具即可在 Settings 面板中启用/禁用。 -- Go 1.21 或更高版本 -- OpenAI API Key（或其他兼容OpenAI协议的API，如DeepSeek、Claude等） -- 安全工具（可选）：根据您的需求安装相应的安全工具，系统支持上百个工具 +### 攻击链分析 +- 智能体解析每次对话，抽取目标、工具、漏洞与因果关系。 +- Web 端可交互式查看链路节点、风险级别及时间轴，支持导出报告。 -### 安装步骤 +### MCP 全场景 +- **Web 模式**：自带 HTTP MCP 服务供前端调用。 +- **MCP stdio 模式**：`go run cmd/mcp-stdio/main.go` 可接入 Cursor/命令行。 +- **外部 MCP 联邦**：在设置中注册第三方 MCP（HTTP/stdio），按需启停并实时查看调用统计与健康度。 -1. **克隆项目** -```bash -git clone https://github.com/Ed1s0nZ/CyberStrikeAI.git -cd CyberStrikeAI-main -``` +### 自动化与安全 +- **REST API**：认证、会话、任务、监控等接口全部开放，可与 CI/CD 集成。 +- **任务控制**：支持暂停/终止长任务、修改参数后重跑、流式获取日志。 +- **安全管理**：`/api/auth/change-password` 可即时轮换口令；建议在暴露 MCP 端口时配合网络层 ACL。 -2. **安装依赖** -```bash -go mod download -``` - -3. **配置** - -#### 方式一：通过Web界面配置（推荐） - -启动服务器后，在Web界面点击右上角的"设置"按钮，可以可视化配置： -- **OpenAI配置**：API Key、Base URL、模型（必填项，标记为 *） -- **MCP工具配置**：启用/禁用工具 -- **Agent配置**：最大迭代次数等 - -配置会自动保存到 `config.yaml` 文件中。打开设置时会自动加载当前配置文件中的值。 - -#### 方式二：直接编辑配置文件 - -编辑 `config.yaml` 文件，设置您的API配置： +## 配置参考 ```yaml -# 身份认证配置 auth: - password: "" # 可留空，首次启动自动生成强密码 - session_duration_hours: 12 # 登录有效期（小时） - -# OpenAI兼容API配置（支持OpenAI、DeepSeek、Claude等） -openai: - api_key: "sk-your-api-key-here" # 替换为您的API Key - base_url: "https://api.openai.com/v1" # 或使用其他兼容API的地址 - model: "gpt-4" # 或 "deepseek-chat", "gpt-3.5-turbo" 等 - -# 服务器配置 + password: "change-me" + session_duration_hours: 12 server: host: "0.0.0.0" port: 8080 - -# 数据库配置 +log: + level: "info" + output: "stdout" +mcp: + enabled: true + host: "0.0.0.0" + port: 8081 +openai: + api_key: "sk-xxx" + base_url: "https://api.deepseek.com/v1" + model: "deepseek-chat" database: path: "data/conversations.db" - -# 安全工具配置 security: - tools_dir: "tools" # 工具配置文件目录 + tools_dir: "tools" ``` -**支持的API服务商：** -- OpenAI: `https://api.openai.com/v1` -- DeepSeek: `https://api.deepseek.com/v1` -- 其他兼容OpenAI协议的API服务 - -**注意**：API Key、Base URL 和模型是必填项，必须配置才能正常运行系统。在Web界面配置时，这些字段会进行验证，未填写时会显示错误提示。 - -4. **安装安全工具（可选）** - -根据您的需求安装相应的安全工具。系统支持上百个工具，您可以根据实际需要选择性安装： - -```bash -# macOS (使用Homebrew) -brew install nmap sqlmap nuclei httpx gobuster feroxbuster subfinder amass - -# Ubuntu/Debian -sudo apt-get install nmap sqlmap nuclei httpx gobuster feroxbuster - -# 或使用Docker运行工具 -# 或使用各工具的官方安装方法 -``` - -Ubuntu安全工具批量安装脚本: `https://github.com/Ed1s0nZ/sec_tools/blob/main/install_tools_ubuntu.sh` - -**注意**：不是所有工具都需要安装，AI会根据您的测试需求自动选择可用的工具。如果某个工具未安装，AI会尝试使用替代工具。 - -5. **启动服务器** - -#### 方式一：使用启动脚本（推荐） -```bash -chmod +x run.sh -./run.sh -``` - -#### 方式二：直接运行 -```bash -go run cmd/server/main.go -``` - -#### 方式三：编译后运行 -```bash -go build -o cyberstrike-ai cmd/server/main.go -./cyberstrike-ai -``` - -#### 方式四：指定配置文件 -```bash -go run cmd/server/main.go -config /path/to/config.yaml -``` - -6. **访问应用** -打开浏览器访问：http://localhost:8080 - -您将看到： -- **对话测试** - 与AI对话进行渗透测试 -- **工具监控** - 查看工具执行状态和结果 -- **对话历史** - 管理历史对话记录 -- **系统设置** - 配置API密钥、工具启用状态等（点击右上角设置按钮） - -**首次使用提示**： -- 在开始使用前，请先点击右上角的"设置"按钮配置您的API Key -- API Key、Base URL 和模型是必填项（标记为 *），必须填写才能正常使用 -- 配置会自动保存到 `config.yaml` 文件中 -- 打开设置时会自动加载当前配置文件中的最新配置 -- 如果 `auth.password` 留空，程序首次启动会自动生成 24 位强密码，写回 `config.yaml` 并在终端输出中英文安全提示 -- Web 首次访问会弹出登录框，请使用该密码登录；可在 **设置 → 安全设置** 中随时修改密码 - -## ⚙️ 配置说明 - -### Web界面配置管理 - -系统提供了可视化的配置管理界面，您可以通过以下方式访问： - -1. **打开设置**：点击Web界面右上角的"设置"按钮 -2. **加载配置**：打开设置时会自动从 `config.yaml` 加载当前配置 -3. **修改配置**： - - **OpenAI配置**：修改API Key、Base URL、模型（必填项标记为 *） - - **MCP工具配置**：启用或禁用工具，支持搜索和批量操作 - - **Agent配置**：设置最大迭代次数等参数 -4. **保存配置**：点击"应用配置"按钮，配置会保存到 `config.yaml` 并立即生效 -5. **验证提示**：必填项未填写时会显示错误提示，并高亮显示错误字段 - -**配置验证规则**： -- API Key、Base URL、模型为必填项 -- 保存时会自动验证，未填写必填项会阻止保存并提示错误 - -### 完整配置示例 - -```yaml -# 身份认证 -auth: - password: "change-me" # Web 登录密码 - session_duration_hours: 12 # 会话有效期（小时） - -# 服务器配置 -server: - host: "0.0.0.0" # 监听地址 - port: 8080 # HTTP服务端口 - -# 日志配置 -log: - level: "info" # 日志级别: debug, info, warn, error - output: "stdout" # 输出位置: stdout, stderr, 或文件路径 - -# MCP协议配置 -mcp: - enabled: true # 是否启用MCP服务器 - host: "0.0.0.0" # MCP服务器监听地址 - port: 8081 # MCP服务器端口 - -# AI模型配置（支持OpenAI兼容API） -openai: - api_key: "sk-xxx" # API密钥 - base_url: "https://api.deepseek.com/v1" # API基础URL - model: "deepseek-chat" # 模型名称 - -# 数据库配置 -database: - path: "data/conversations.db" # SQLite数据库路径 - -# 安全工具配置 -security: - # 推荐方式：从目录加载工具配置 - tools_dir: "tools" # 工具配置文件目录（相对于配置文件所在目录） - - # 向后兼容：也可以在主配置文件中直接定义工具 - # tools: - # - name: "nmap" - # command: "nmap" - # args: ["-sT", "-sV", "-sC"] - # description: "网络扫描工具" - # enabled: true -``` - -### 工具配置方式 - -**方式一：使用工具目录（推荐）** - -在 `tools/` 目录下为每个工具创建独立的YAML配置文件，例如 `tools/nmap.yaml`： +### 工具模版示例（`tools/nmap.yaml`） ```yaml name: "nmap" command: "nmap" args: ["-sT", "-sV", "-sC"] enabled: true - -short_description: "网络扫描工具，用于发现网络主机、开放端口和服务" - -description: | - 网络映射和端口扫描工具，用于发现网络中的主机、服务和开放端口。 - +short_description: "网络资产扫描与服务指纹识别" parameters: - name: "target" type: "string" - description: "目标IP地址或域名" + description: "IP 或域名" required: true position: 0 - format: "positional" - - name: "ports" type: "string" - description: "端口范围，例如: 1-1000" - required: false flag: "-p" - format: "flag" + description: "端口范围，如 1-1000" ``` -**方式二：在主配置文件中定义** +## 项目结构 -直接在 `config.yaml` 的 `security.tools` 中定义工具配置。 +``` +CyberStrikeAI/ +├── cmd/ # Web 服务、MCP stdio 入口及辅助工具 +├── internal/ # Agent、MCP 核心、路由与执行器 +├── web/ # 前端静态资源与模板 +├── tools/ # YAML 工具目录（含 100+ 示例） +├── img/ # 文档配图 +├── config.yaml # 运行配置 +├── run.sh # 启动脚本 +└── README*.md +``` -**注意：** 如果同时配置了 `tools_dir` 和 `tools`，`tools_dir` 中的工具优先。 +## 基础体验示例 -### 身份认证与安全 - -- **登录流程**：除 `/api/auth/login` 外，所有 `/api` 接口均需携带 `Authorization: Bearer ` 请求头。登录成功后返回的 token 会由前端自动缓存。 -- **自动生成密码**：`auth.password` 为空时，启动会生成 24 位随机强密码，写回配置文件并在终端输出中英文提示，请务必妥善保管。 -- **会话控制**：会话有效期由 `auth.session_duration_hours` 控制。过期或修改密码后需重新登录。 -- **密码修改**：在 **设置 → 安全设置** 中即可修改密码，或直接调用 `/api/auth/change-password` 接口；修改会立即使所有旧会话失效。 -- **MCP 端口**：独立 MCP 服务器（默认 `8081`）为了兼容 IDE 插件暂未启用鉴权，建议通过网络层限制访问范围。 - -## 🚀 使用示例 - -### 对话式渗透测试 - -在Web界面的"对话测试"标签页中，您可以使用自然语言与AI对话： - -#### 1. 网络扫描 ``` 扫描 192.168.1.1 的开放端口 -``` -或更详细的指令： -``` -对 192.168.1.1 进行全面的端口扫描，重点关注80、443、22、21端口 +对 192.168.1.1 做 80/443/22 重点扫描 +检查 https://example.com/page?id=1 是否存在 SQL 注入 +枚举 https://example.com 的隐藏目录与组件漏洞 +获取 example.com 的子域并批量执行 nuclei ``` -#### 2. SQL注入检测 -``` -检测 https://example.com/page?id=1 是否存在SQL注入漏洞 -``` - -#### 3. Web漏洞扫描 -``` -扫描 https://example.com 的Web服务器漏洞，包括常见的安全问题 -``` - -#### 4. 目录扫描 -``` -扫描 https://example.com 的隐藏目录和文件 -``` - -#### 5. 综合安全测试 -``` -对 example.com 进行全面的安全评估，包括端口扫描、Web漏洞检测和目录枚举 -``` - -#### 6. 多步骤测试 -``` -首先扫描 192.168.1.1 的开放端口，然后对发现的Web服务进行漏洞扫描 -``` - -### 后渗透测试 - -在获得初始访问权限后，可以使用后渗透工具进行权限提升、横向移动和持久化： - -#### 1. Linux 权限提升枚举 -``` -使用 linpeas 对目标 Linux 系统进行权限提升检查 -``` - -#### 2. Windows 权限提升枚举 -``` -使用 winpeas 对目标 Windows 系统进行权限提升检查 -``` - -#### 3. Active Directory 攻击路径分析 -``` -使用 bloodhound 分析 Active Directory 的攻击路径 -``` - -#### 4. 凭证提取 -``` -使用 mimikatz 提取 Windows 系统的凭证信息 -``` - -#### 5. 横向移动 -``` -使用 impacket 工具集进行网络协议攻击和横向移动 -``` - -#### 6. 后门生成 -``` -使用 msfvenom 生成反向 shell 载荷 -``` - -### CTF 竞赛支持 - -系统内置了丰富的 CTF 工具，支持各种 CTF 题型的解题： - -#### 1. 隐写分析 -``` -使用 stegsolve 分析图片隐写 -使用 zsteg 检测 LSB 隐写 -``` - -#### 2. 密码破解 -``` -使用 hashcat 破解哈希值 -使用 john 破解密码文件 -使用 fcrackzip 破解 ZIP 文件密码 -使用 pdfcrack 破解 PDF 文件密码 -``` - -#### 3. 二进制分析 -``` -使用 gdb 调试二进制文件 -使用 radare2 进行逆向分析 -使用 strings 提取二进制文件中的字符串 -``` - -#### 4. 哈希识别 -``` -使用 hash-identifier 识别哈希类型 -``` - -#### 5. 数据转换和分析 -``` -使用 cyberchef 进行各种数据转换和分析 -使用 xxd 查看文件十六进制内容 -``` - -#### 6. 综合 CTF 解题 -``` -分析这个 CTF 题目：给定一个包含隐写和加密的文件，找出 flag -``` - -### 监控工具执行 - -在"工具监控"标签页中，您可以： - -- 📊 **执行统计** - 查看所有工具的调用次数、成功/失败统计 -- 📝 **执行记录** - 查看详细的工具执行历史，包括参数、结果、耗时 -- 🔍 **漏洞列表** - 自动提取和分类发现的漏洞 -- ⏱️ **实时状态** - 实时查看正在执行的工具状态 - -### 对话历史管理 - -- 📚 **查看历史** - 浏览所有历史对话记录 -- 🔍 **搜索对话** - 根据标题搜索对话 -- 🗑️ **删除对话** - 清理不需要的对话记录 -- 📄 **查看详情** - 查看对话的完整消息和工具执行过程 - -## 效果 - -### 对话效果 - ![详情预览](./img/效果1.png) - ![详情预览](./img/效果2.png) - -### MCP调用 - ![详情预览](./img/MCP.png) - -### 调用链 - ![详情预览](./img/调用链1.png) - - -## 📡 API接口 - -### Agent Loop API - -#### 标准请求（同步） - -**POST** `/api/agent-loop` - -请求体： -```json -{ - "message": "扫描 192.168.1.1", - "conversationId": "optional-conversation-id" // 可选，用于继续对话 -} -``` - -响应： -```json -{ - "response": "AI的回复内容", - "mcpExecutionIds": ["exec-id-1", "exec-id-2"], - "conversationId": "conversation-id", - "time": "2024-01-01T00:00:00Z" -} -``` - -使用示例： -```bash -curl -X POST http://localhost:8080/api/agent-loop \ - -H "Content-Type: application/json" \ - -d '{"message": "扫描 192.168.1.1"}' -``` - -#### 流式请求（推荐，实时输出） - -**POST** `/api/agent-loop/stream` - -使用Server-Sent Events (SSE)实时返回执行过程。 - -请求体： -```json -{ - "message": "扫描 192.168.1.1", - "conversationId": "optional-conversation-id" -} -``` - -事件类型： -- `progress` - 进度更新 -- `iteration` - 迭代开始 -- `thinking` - AI思考内容 -- `tool_call` - 工具调用开始 -- `tool_result` - 工具执行结果 -- `response` - 最终回复 -- `error` - 错误信息 -- `done` - 完成 - -使用示例（JavaScript）： -```javascript -const eventSource = new EventSource('/api/agent-loop/stream', { - method: 'POST', - body: JSON.stringify({ message: '扫描 192.168.1.1' }) -}); - -eventSource.onmessage = (event) => { - const data = JSON.parse(event.data); - console.log(data.type, data.message, data.data); -}; -``` - -### 对话历史API - -#### 创建对话 - -**POST** `/api/conversations` - -请求体： -```json -{ - "title": "对话标题" -} -``` - -#### 获取对话列表 - -**GET** `/api/conversations` - -查询参数： -- `limit` - 限制返回数量（可选） -- `offset` - 偏移量（可选） - -#### 获取单个对话 - -**GET** `/api/conversations/:id` - -返回对话的完整信息，包括所有消息。 - -#### 删除对话 - -**DELETE** `/api/conversations/:id` - -### 监控API - -#### 获取所有监控信息 - -**GET** `/api/monitor` - -返回所有执行记录、统计信息和漏洞列表。 - -#### 获取特定执行记录 - -**GET** `/api/monitor/execution/:id` - -返回指定ID的工具执行详情。 - -#### 获取统计信息 - -**GET** `/api/monitor/stats` - -返回所有工具的调用统计： -```json -{ - "nmap": { - "toolName": "nmap", - "totalCalls": 10, - "successCalls": 9, - "failedCalls": 1, - "lastCallTime": "2024-01-01T00:00:00Z" - } -} -``` - -#### 获取漏洞列表 - -**GET** `/api/monitor/vulnerabilities` - -返回所有发现的漏洞： -```json -{ - "total": 5, - "severityCount": { - "critical": 0, - "high": 2, - "medium": 2, - "low": 1 - }, - "vulnerabilities": [...] -} -``` - -### MCP协议接口 - -**POST** `/api/mcp` - -MCP协议端点，支持JSON-RPC 2.0格式的请求。 - -## 🔌 MCP协议 - -本项目完整实现了MCP（Model Context Protocol）协议，支持以下功能： - -### 传输模式 - -CyberStrikeAI 支持两种 MCP 传输模式： - -#### 1. HTTP 模式（默认） -- 通过 HTTP POST 请求进行通信 -- 适用于 Web 应用和其他 HTTP 客户端 -- 默认监听地址：`0.0.0.0:8081/mcp` -- 可通过 `/api/mcp` 端点访问 -- 🌐 便于远程：可对外暴露单个 HTTP 端口，IDE、Web 应用或其他机器上的自动化流程都能直接访问。 -- 🧩 易于复用：无需额外二进制，只要支持 HTTP 的客户端（例如 curl、Postman、云端任务）都能复用同一个服务。 -- 🔁 持续服务：与主 Web 服务同进程运行，部署一次即可同时提供 UI、API 和 MCP 能力。 - -#### MCP HTTP 模式（IDE 集成） - -可将 Cursor、Claude Desktop 等 IDE 直接连接到内置的 HTTP MCP 服务： - -1. 确认 `config.yaml` 中 `mcp.enabled: true`，根据需要调整 `host` / `port`。 -2. 启动主服务（`./run.sh` 或 `go run cmd/server/main.go`），MCP 端点将暴露在 `http://:/mcp`（本地运行时推荐使用 `http://127.0.0.1:8081/mcp`）。 -3. 在 Cursor 中打开 **Settings → Tools & MCP → Add Custom MCP**，选择 HTTP，填写： - - `Base URL`: `http://127.0.0.1:8081/mcp` - - 如启用了鉴权，可在 Headers 中添加 `Authorization` 等字段。 -4. 或者在项目根目录创建 `.cursor/mcp.json`： - ```json - { - "mcpServers": { - "cyberstrike-ai-http": { - "transport": "http", - "url": "http://127.0.0.1:8081/mcp" - } - } - } - ``` -5. 重启 IDE 后即可在 MCP 工具列表中看到 CyberStrikeAI 的工具集。 - -> 🔐 **安全提示**：若将 MCP HTTP 端口开放给外部，请结合防火墙或认证机制，避免被未授权访问。 - -#### 2. stdio 模式（新增） -- 通过标准输入输出（stdio）进行通信 -- 适用于 Cursor、Claude Desktop 等 IDE 集成 -- 完全符合 JSON-RPC 2.0 规范 -- 支持字符串、数字和 null 类型的 id 字段 -- 正确处理通知（notification）消息 -- 🔒 更强隔离：以独立二进制方式运行，可结合最小权限策略、独立运行账号来限制文件/网络访问，安全性更高。 -- 🪟 无需暴露端口：所有通信都在本地进程内完成，适合不希望在本机开启额外 HTTP 监听端口的场景。 -- 🧰 IDE 优先体验：Cursor、Claude Desktop 等 IDE 的自定义 MCP 首选 stdio 传输，配置简单即插即用。 -- 🧱 多层防护：HTTP 适合远程共享场景，stdio 适合本地高安全场景，同时保留可根据工作流自由选择。 - -#### 模式对比：按需选择 - -| 对比维度 | `mcp-http`（HTTP 模式） | `mcp-stdio`（stdio 模式） | -|------------------|-----------------------------------------------|-------------------------------------------------------------------| -| 传输协议 | 基于网络的 HTTP/HTTPS | 标准输入输出流 | -| 部署方式 | 与主服务器同进程运行 | 独立编译为单独可执行文件 | -| 隔离与安全 | 依赖服务端加固（防火墙、认证、网络策略） | 借助操作系统进程隔离，无需暴露监听端口 | -| 远程访问 | ✅ 可跨机器访问 | ❌ 仅限本地（除非手动隧道转发） | -| IDE 集成 | 适用于支持 HTTP 的客户端 | 原生适配 Cursor / Claude Desktop 等 stdio 连接器 | -| 最佳使用场景 | 远程自动化、共享服务、云端部署 | 本地开发、对安全隔离要求较高的环境 | - -### 支持的方法 - -- `initialize` - 初始化连接，协商协议版本和功能 -- `tools/list` - 列出所有可用的工具 -- `tools/call` - 调用指定工具并执行 -- `prompts/list` - 列出可用的提示词模板 -- `prompts/get` - 获取提示词模板内容 -- `resources/list` - 列出可用资源 -- `resources/read` - 读取资源内容 -- `sampling/request` - 采样请求（占位实现） -- `notifications/initialized` - 初始化完成通知（stdio 模式） - -### 工具执行机制 - -- 工具调用是同步执行的，确保错误能正确返回 -- 每个工具调用都会创建执行记录，包含： - - 执行ID（唯一标识） - - 工具名称和参数 - - 执行状态（running, completed, failed） - - 开始和结束时间 - - 执行结果或错误信息 -- 系统自动跟踪所有工具的执行统计信息 - -### MCP stdio 模式（Cursor IDE 集成） - -stdio 模式允许你在 Cursor IDE 中直接使用 CyberStrikeAI 的所有安全工具。 - -#### 编译 stdio 模式程序 - -```bash -# 在项目根目录执行 -go build -o cyberstrike-ai-mcp cmd/mcp-stdio/main.go -``` - -#### 在 Cursor 中配置 - -**方法一：通过 UI 配置** - -1. 打开 Cursor 设置 → **Tools & MCP** -2. 点击 **Add Custom MCP** -3. 配置如下（请替换为你的实际路径）： - -```json -{ - "mcpServers": { - "cyberstrike-ai": { - "command": "/absolute/path/to/cyberstrike-ai-mcp", - "args": [ - "--config", - "/absolute/path/to/config.yaml" - ] - } - } -} -``` - -**方法二：通过项目配置文件** - -在项目根目录创建 `.cursor/mcp.json` 文件： - -```json -{ - "mcpServers": { - "cyberstrike-ai": { - "command": "/Users/yourname/Downloads/CyberStrikeAI-main/cyberstrike-ai-mcp", - "args": [ - "--config", - "/Users/yourname/Downloads/CyberStrikeAI-main/config.yaml" - ] - } - } -} -``` - -**重要提示：** -- ✅ 使用绝对路径：`command` 和配置文件路径必须使用绝对路径 -- ✅ 可执行权限：确保编译后的程序有执行权限（Linux/macOS） -- ✅ 重启 Cursor：配置后需要重启 Cursor 才能生效 - -配置完成后，重启 Cursor，你就可以在聊天中直接使用所有安全工具了！ - -#### stdio 模式特性 - -- ✅ 完全符合 JSON-RPC 2.0 规范 -- ✅ 支持字符串、数字和 null 类型的 id 字段 -- ✅ 正确处理通知（notification）消息 -- ✅ 日志输出到 stderr，不干扰 JSON-RPC 通信 -- ✅ 与 HTTP 模式完全独立，可同时使用 - -### MCP HTTP 模式使用示例 - -#### 初始化连接 - -```bash -curl -X POST http://localhost:8080/api/mcp \ - -H "Content-Type: application/json" \ - -d '{ - "jsonrpc": "2.0", - "id": "1", - "method": "initialize", - "params": { - "protocolVersion": "2024-11-05", - "capabilities": {}, - "clientInfo": { - "name": "test-client", - "version": "1.0.0" - } - } - }' -``` - -#### 列出工具 - -```bash -curl -X POST http://localhost:8080/api/mcp \ - -H "Content-Type: application/json" \ - -d '{ - "jsonrpc": "2.0", - "id": "2", - "method": "tools/list" - }' -``` - -#### 调用工具 - -```bash -curl -X POST http://localhost:8080/api/mcp \ - -H "Content-Type: application/json" \ - -d '{ - "jsonrpc": "2.0", - "id": "3", - "method": "tools/call", - "params": { - "name": "nmap", - "arguments": { - "target": "192.168.1.1", - "ports": "1-1000" - } - } - }' -``` - -## 🔗 外部 MCP 接入 - -CyberStrikeAI 支持接入外部 MCP 服务器，扩展工具能力。外部 MCP 工具会自动注册到系统中，AI 可以像使用内置工具一样调用它们。 - -### 配置方式 - -#### 方式一：通过 Web 界面配置（推荐） - -1. 启动服务器后，访问 Web 界面 -2. 点击右上角的"设置"按钮 -3. 在设置界面中找到"外部 MCP"配置部分 -4. 点击"添加外部 MCP"按钮 -5. 填写配置信息： - - **名称**：外部 MCP 服务器的唯一标识（如：`hexstrike-ai`） - - **传输模式**：选择 `stdio` 或 `http` - - **描述**：可选，用于说明该 MCP 服务器的功能 - - **超时时间**：工具执行超时时间（秒），默认 300 秒 - - **启用状态**：是否立即启用该外部 MCP -6. 根据传输模式填写相应配置： - - **stdio 模式**： - - **命令**：MCP 服务器的启动命令（如：`python3`） - - **参数**：启动参数数组（如：`["/path/to/mcp_server.py", "--arg", "value"]`） - - **HTTP 模式**： - - **URL**：MCP 服务器的 HTTP 端点地址（如：`http://127.0.0.1:8888`） -7. 点击"保存"按钮，配置会自动保存到 `config.yaml` -8. 系统会自动连接外部 MCP 服务器并加载其工具 - -#### 方式二：直接编辑配置文件 - -在 `config.yaml` 中添加 `external_mcp` 配置： - -```yaml -# 外部MCP配置 -external_mcp: - servers: - # 外部MCP服务器名称（唯一标识） - hexstrike-ai: - # stdio 模式配置 - command: python3 - args: - - /path/to/hexstrike_mcp.py - - --server - - 'http://127.0.0.1:8888' - - # 或 HTTP 模式配置（二选一） - # transport: http - # url: http://127.0.0.1:8888 - - # 通用配置 - description: HexStrike AI v6.0 - Advanced Cybersecurity Automation Platform - timeout: 300 # 超时时间（秒） - external_mcp_enable: true # 是否启用 - - # 工具级别控制（可选） - tool_enabled: - nmap_scan: true - sqlmap_scan: true - # ... 其他工具 -``` - -### 传输模式说明 - -#### stdio 模式 - -通过标准输入输出（stdio）与外部 MCP 服务器通信，适合本地运行的 MCP 服务器。 - -**配置示例：** -```yaml -external_mcp: - servers: - my-mcp-server: - command: python3 - args: - - /path/to/mcp_server.py - - --config - - /path/to/config.json - description: My Custom MCP Server - timeout: 300 - external_mcp_enable: true -``` - -**特点：** -- ✅ 本地进程通信，无需网络端口 -- ✅ 安全性高，数据不经过网络 -- ✅ 适合本地开发和测试 - -#### HTTP 模式 - -通过 HTTP 请求与外部 MCP 服务器通信，适合远程 MCP 服务器或需要跨网络访问的场景。 - -**配置示例：** -```yaml -external_mcp: - servers: - remote-mcp-server: - transport: http - url: http://192.168.1.100:8888 - description: Remote MCP Server - timeout: 300 - external_mcp_enable: true -``` - -**特点：** -- ✅ 支持远程访问 -- ✅ 适合分布式部署 -- ✅ 易于集成现有 HTTP 服务 - -### 工具命名规则 - -外部 MCP 工具在系统中的名称格式为：`{mcp-server-name}::{tool-name}` - -例如： -- 外部 MCP 服务器名称：`hexstrike-ai` -- 工具名称：`nmap_scan` -- 系统内完整工具名：`hexstrike-ai::nmap_scan` - -AI 在调用时会自动识别并使用完整工具名。 - -### 工具启用控制 - -#### 全局启用/禁用 - -通过 `external_mcp_enable` 字段控制整个外部 MCP 服务器的启用状态： -- `true`：启用，系统会自动连接并加载工具 -- `false`：禁用，系统不会连接该服务器 - -#### 工具级别控制 - -通过 `tool_enabled` 字段精确控制每个工具的启用状态： - -```yaml -external_mcp: - servers: - hexstrike-ai: - # ... 其他配置 - tool_enabled: - nmap_scan: true # 启用此工具 - sqlmap_scan: false # 禁用此工具 - nuclei_scan: true # 启用此工具 -``` - -- 如果 `tool_enabled` 中未列出某个工具，默认启用 -- 如果某个工具设置为 `false`，该工具不会出现在工具列表中，AI 也无法调用 - -### 管理外部 MCP - -#### 通过 Web 界面管理 - -1. **查看外部 MCP 列表**：在设置界面中查看所有已配置的外部 MCP 服务器 -2. **启动/停止**：可以随时启动或停止外部 MCP 服务器连接 -3. **编辑配置**：修改外部 MCP 的配置信息 -4. **删除配置**：移除不需要的外部 MCP 服务器 - -#### 通过 API 管理 - -- **获取外部 MCP 列表**：`GET /api/external-mcp` -- **添加外部 MCP**：`POST /api/external-mcp` -- **更新外部 MCP**：`PUT /api/external-mcp/:name` -- **删除外部 MCP**：`DELETE /api/external-mcp/:name` -- **启动外部 MCP**：`POST /api/external-mcp/:name/start` -- **停止外部 MCP**：`POST /api/external-mcp/:name/stop` - -### 监控和统计 - -外部 MCP 工具的执行记录和统计信息会自动记录到系统中： - -- **执行记录**：在"工具监控"页面可以查看所有外部 MCP 工具的执行历史 -- **执行统计**：执行统计面板会显示外部 MCP 工具的调用次数、成功/失败统计 -- **实时监控**：可以实时查看外部 MCP 工具的执行状态 - -### 故障排除 - -**问题：外部 MCP 无法连接** - -- ✅ 检查 `command` 和 `args` 配置是否正确（stdio 模式） -- ✅ 检查 `url` 配置是否正确且可访问（HTTP 模式） -- ✅ 检查外部 MCP 服务器是否正常运行 -- ✅ 查看服务器日志获取详细错误信息 -- ✅ 检查网络连接（HTTP 模式） -- ✅ 检查防火墙设置（HTTP 模式） - -**问题：外部 MCP 工具未显示** - -- ✅ 确认 `external_mcp_enable: true` -- ✅ 检查 `tool_enabled` 配置，确保工具未被禁用 -- ✅ 确认外部 MCP 服务器已成功连接 -- ✅ 查看服务器日志确认工具是否已加载 - -**问题：外部 MCP 工具执行失败** - -- ✅ 检查外部 MCP 服务器是否正常运行 -- ✅ 查看工具执行日志获取详细错误信息 -- ✅ 检查超时时间设置是否合理 -- ✅ 确认外部 MCP 服务器支持的工具调用格式 - -### 最佳实践 - -1. **命名规范**：使用有意义的名称标识外部 MCP 服务器，避免冲突 -2. **超时设置**：根据工具执行时间合理设置超时时间 -3. **工具控制**：使用 `tool_enabled` 精确控制需要的工具，避免加载过多无用工具 -4. **安全考虑**：HTTP 模式建议使用内网地址或配置适当的访问控制 -5. **监控管理**：定期检查外部 MCP 的连接状态和执行统计 - -## 🛠️ 安全工具支持 - -### 工具概览 - -当前系统集成了 **上百个安全工具**，涵盖以下类别： - -- **网络扫描工具** - nmap, masscan, rustscan, arp-scan, nbtscan 等 -- **Web应用扫描** - sqlmap, nikto, dirb, gobuster, feroxbuster, ffuf, httpx 等 -- **漏洞扫描** - nuclei, wpscan, wafw00f, dalfox, xsser 等 -- **子域名枚举** - subfinder, amass, findomain, dnsenum, fierce 等 -- **API安全** - graphql-scanner, arjun, api-fuzzer, api-schema-analyzer 等 -- **容器安全** - trivy, clair, docker-bench-security, kube-bench, kube-hunter 等 -- **云安全** - prowler, scout-suite, cloudmapper, pacu, terrascan, checkov 等 -- **二进制分析** - gdb, radare2, ghidra, objdump, strings, binwalk 等 -- **漏洞利用** - metasploit, msfvenom, pwntools, ropper, ropgadget 等 -- **密码破解** - hashcat, john, hashpump 等 -- **取证分析** - volatility, volatility3, foremost, steghide, exiftool 等 -- **后渗透工具** - linpeas, winpeas, mimikatz, bloodhound, impacket, responder 等 -- **CTF工具** - stegsolve, zsteg, hash-identifier, fcrackzip, pdfcrack, cyberchef 等 -- **系统工具** - exec, create-file, delete-file, list-files, modify-file 等 - -### 主要工具示例 - -- **nmap** - 网络端口扫描和服务识别 - - 功能：主机发现、端口扫描、服务版本检测、操作系统识别 - - 配置：`tools/nmap.yaml` - -- **sqlmap** - SQL注入自动化检测和利用工具 - - 功能：自动检测SQL注入漏洞、数据库指纹识别、数据提取 - - 配置：`tools/sqlmap.yaml` - -- **nuclei** - 快速漏洞扫描器 - - 功能：基于模板的漏洞扫描、大规模扫描支持 - - 配置：`tools/nuclei.yaml` - -- **httpx** - HTTP探测工具 - - 功能：HTTP/HTTPS探测、状态码检测、标题提取 - - 配置：`tools/httpx.yaml` - -- **exec** - 系统命令执行工具 - - 功能：执行任意系统命令（需谨慎使用） - - 配置：`tools/exec.yaml` - - ⚠️ 警告：此工具可以执行任意命令，请确保安全使用 - -完整的工具列表请查看 `tools/TOOLS_LIST.md` 文件。 - -### 添加新工具 - -为确保 AI 能够无歧义地调用自定义工具，请按照下述字段规范编写 `tools/*.yaml` 配置文件。系统在加载时会逐项校验，缺失必填字段的工具会被跳过并在日志中提示。 - -> 需要完整的字段对照表和模板示例？请查看 `tools/README.md`（中文）或 `tools/README_EN.md`（英文）获取更详细的配置指南。 - -#### 顶层字段（工具级） - -| 字段 | 必填 | 类型 | 说明 | -|------|------|------|------| -| `name` | ✅ | string | 工具唯一标识，建议使用小写字母＋数字＋短横线组合，便于在对话中引用。 | -| `command` | ✅ | string | 实际执行的命令或脚本名称，需在 PATH 中或写绝对路径。 | -| `enabled` | ✅ | bool | 是否注册到 MCP。为 `false` 时，前端和 AI 都不会看到该工具。 | -| `description` | ✅ | string | 工具详细说明，支持多行 Markdown，供 AI 深度理解以及 `resources/read` 查询。 | -| `short_description` | 可选 | string | 适合 20-50 字的摘要，优先用于工具列表和对话上下文，缺失时系统会自动截取 `description` 片段。 | -| `args` | 可选 | string[] | 固定参数，按顺序写入命令行，常用于定义默认扫描模式或全局开关。 | -| `parameters` | 可选 | 参数对象数组 | 定义调用时可覆盖的动态参数，详见下节。 | -| `arg_mapping` | 可选 | string | 参数映射模式（`auto`/`manual`/`template`），当前版本默认 `auto`，不填写即可。 | - -> **提示**：所有文本字段推荐使用 UTF-8/ASCII，无需额外引号的内容需保持 YAML 缩进正确。 - -#### 参数字段（`parameters[]`） - -每个参数对象支持以下属性，请根据工具特性选择： - -- `name`（必填）：参数键名，同时用于构建命令行和生成 MCP JSON-Schema。 -- `type`（必填）：`string`、`int`/`integer`、`bool`/`boolean`、`array` 等。 -- `description`（必填）：建议使用 Markdown 描述用途、格式、示例值及安全提示。 -- `required`：布尔值，标记是否必须由调用方提供。缺失必填参数时执行器会返回错误信息。 -- `default`：默认值。调用方未传值时将使用该值，便于提供合理的开箱体验。 -- `flag`：命令行标志（如 `-u`、`--url`），结合 `format` 决定拼接方式。 -- `position`：整数（从 0 开始），指定位置参数顺序。所有位置参数会在标志参数之后依次插入。 -- `format`：拼接策略，支持： - - `flag`（默认）：输出为 `--flag value` 或 `-f value`； - - `combined`：输出为 `--flag=value`； - - `positional`：按 `position` 顺序直接写入值； - - `template`：配合 `template` 字段自定义渲染。 -- `template`：当 `format: "template"` 时生效，支持 `{flag}`、`{value}`、`{name}` 占位符。 -- `options`：枚举值数组。定义后 AI 会提示可选项，MCP schema 也会自动带上 `enum` 信息。 - -#### 约定的特殊参数名 - -- `additional_args`：允许用户在一次调用中追加任意命令行片段（内部会智能拆分并追加到命令末尾）。 -- `scan_type`：主要用于 `nmap` 等网络扫描工具，用来覆盖默认扫描参数（如 `-sV -sC`）。 -- `action`：供少数需要内部状态分支的工具使用，仅在服务端逻辑消费，不会拼接到命令行。 - -这些名称在执行器中有特殊处理，建议复用而不要自定义同义字段。 - -#### 推荐配置流程 - -1. 在 `tools/` 目录新建 YAML 文件，例如 `tools/mytool.yaml`。 -2. 填写上述顶层字段及参数列表，为常用参数设置 `default`，并在 `description` 中提供示例。 -3. 如需快速校验，执行 `go run cmd/test-config/main.go`，确认加载无误。 -4. 重启服务或触发热更新后，前端设置面板与 MCP 工具列表会自动展示新工具。 - -#### 方法二：在主配置文件中添加 - -如果暂时不想拆分文件，也可以直接在 `config.yaml` 的 `security.tools` 下编写相同结构的对象。若同时存在 `tools_dir` 与内联 `tools`，目录中的配置优先生效。 - -### 工具参数配置 - -工具参数支持以下格式： - -- **positional** - 位置参数，按顺序添加到命令中 -- **flag** - 标志参数，格式为 `--flag value` 或 `-f value` -- **combined** - 组合格式，格式为 `--flag=value` -- **template** - 模板格式，使用自定义模板字符串 - -参数类型支持： -- `string` - 字符串 -- `int` / `integer` - 整数 -- `bool` / `boolean` - 布尔值 -- `array` - 数组（自动转换为逗号分隔的字符串） - -## 🔧 故障排除 - -### API连接问题 - -**问题：无法连接到OpenAI API** - -- ✅ 检查API Key是否正确配置在 `config.yaml` 中 -- ✅ 检查网络连接是否正常 -- ✅ 验证 `base_url` 配置是否正确 -- ✅ 确认API服务商是否支持您使用的模型 -- ✅ 检查API配额和余额是否充足 - -**问题：API返回401或403错误** - -- ✅ 验证API Key是否有效 -- ✅ 检查API Key是否有权限访问指定模型 -- ✅ 确认API服务商的访问限制 - -### 工具执行问题 - -**问题：工具执行失败或找不到命令** - -- ✅ 确保已安装相应的安全工具： - ```bash - # 检查工具是否安装 - which nmap sqlmap nikto dirb - ``` -- ✅ 检查工具是否在系统PATH中 -- ✅ 某些工具可能需要root权限（如nmap的SYN扫描） -- ✅ 查看服务器日志获取详细错误信息 - -**问题：工具执行超时** - -- ✅ 某些工具（如nmap全端口扫描）可能需要较长时间 -- ✅ 检查网络连接和目标响应 -- ✅ 考虑使用更小的扫描范围 - -**问题：工具参数错误** - -- ✅ 检查工具配置文件中的参数定义 -- ✅ 查看工具执行日志中的实际命令 -- ✅ 参考工具的官方文档验证参数格式 - -### 服务器问题 - -**问题：前端无法加载** - -- ✅ 检查服务器是否正常运行： - ```bash - curl http://localhost:8080 - ``` -- ✅ 检查端口8080是否被占用： - ```bash - lsof -i :8080 - ``` -- ✅ 查看浏览器控制台错误信息 -- ✅ 检查防火墙设置 - -**问题：数据库错误** - -- ✅ 确保 `data/` 目录有写权限 -- ✅ 检查数据库文件是否损坏 -- ✅ 删除数据库文件让系统重新创建（会丢失历史数据） - -### 配置问题 - -**问题：工具未加载** - -- ✅ 检查 `tools_dir` 配置是否正确 -- ✅ 验证工具配置文件格式是否正确（YAML语法） -- ✅ 查看启动日志中的工具加载信息 -- ✅ 确保工具配置中的 `enabled: true` - -**问题：MCP服务器无法启动** - -- ✅ 检查MCP端口（默认8081）是否被占用 -- ✅ 验证MCP配置中的 `enabled: true` -- ✅ 查看日志中的MCP服务器启动信息 - -**问题：Cursor 中 MCP stdio 模式无法连接** - -- ✅ 检查 `cyberstrike-ai-mcp` 程序路径是否正确（使用绝对路径） -- ✅ 检查程序是否有执行权限（Linux/macOS）：`chmod +x cyberstrike-ai-mcp` -- ✅ 检查 `config.yaml` 配置文件路径是否正确 -- ✅ 查看 Cursor 的日志输出（通常在 Cursor 的开发者工具中） -- ✅ 确保配置文件中的 `security.tools_dir` 配置正确 - -**问题：Cursor 中工具列表为空** - -- ✅ 确保 `config.yaml` 中的 `security.tools_dir` 配置正确 -- ✅ 确保工具配置文件在指定目录中 -- ✅ 检查工具配置文件格式是否正确（YAML 语法） -- ✅ 查看程序日志（stderr 输出） - -**问题：Cursor 中工具执行失败** - -- ✅ 确保相应的安全工具已安装在系统中 -- ✅ 检查工具是否在系统 PATH 中 -- ✅ 查看程序日志（stderr 输出） -- ✅ 尝试在终端中直接运行工具命令，确认工具可用 - -### 日志调试 - -启用详细日志： -```yaml -log: - level: "debug" # 改为debug级别 - output: "stdout" -``` - -查看实时日志： -```bash -# 如果使用run.sh -./run.sh | tee cyberstrike.log - -# 或直接运行 -go run cmd/server/main.go 2>&1 | tee cyberstrike.log -``` - -## 安全注意事项 - -⚠️ **重要提示**： - -- 仅对您拥有或已获得授权的系统进行测试 -- 遵守相关法律法规 -- 建议在隔离的测试环境中使用 -- 不要在生产环境中使用 -- 某些安全工具可能需要root权限 - -## 💻 开发指南 - -### 项目架构 +## 进阶剧本示例 ``` -┌─────────────┐ -│ Web UI │ ← 用户界面（HTML/CSS/JS） -└──────┬──────┘ - │ HTTP -┌──────▼─────────────────────────────────────┐ -│ Gin HTTP Server │ -│ ┌──────────────────────────────────────┐ │ -│ │ Handlers (agent, conversation, etc) │ │ -│ └──────┬───────────────────────────────┘ │ -│ │ │ -│ ┌──────▼──────────┐ ┌─────────────────┐ │ -│ │ Agent Module │ │ MCP Server │ │ -│ │ (AI Loop) │◄─┤ (Tool Manager) │ │ -│ └──────┬──────────┘ └────────┬────────┘ │ -│ │ │ │ -│ ┌──────▼──────────┐ ┌────────▼────────┐ │ -│ │ OpenAI API │ │ Security Executor│ │ -│ └─────────────────┘ └────────┬────────┘ │ -│ │ │ -│ ┌────────▼────────┐ │ -│ │ Security Tools │ │ -│ │ (nmap, sqlmap) │ │ -│ └─────────────────┘ │ -└─────────────────────────────────────────────┘ - │ -┌────────▼────────┐ -│ SQLite Database│ ← 对话历史和消息存储 -└─────────────────┘ +加载侦察剧本：先 amass/subfinder，再对存活主机进行目录爆破。 +挂载基于 Burp 的外部 MCP，完成认证流量回放并回传到攻击链。 +将 5MB nuclei 报告压缩并生成摘要，附加到对话记录。 +构建最新一次测试的攻击链，只导出风险 >= 高的节点列表。 ``` -### 核心模块说明 +## Changelog（近期） -- **Agent模块** (`internal/agent/agent.go`) - - 实现Agent Loop，处理AI对话和工具调用决策 - - 支持多轮对话和工具调用链 - - 处理工具执行错误和重试逻辑 +- 2025-11-20 —— 支持超大日志/MCP 记录的自动压缩与摘要回写。 +- 2025-11-17 —— 上线 AI 驱动的攻击链图谱与风险评分。 +- 2025-11-15 —— 提供大结果分页检索与外部 MCP 挂载能力。 +- 2025-11-14 —— 工具检索 O(1)、执行记录清理、数据库分页优化。 +- 2025-11-13 —— Web 鉴权、Settings 面板与 MCP stdio 模式发布。 -- **MCP Server** (`internal/mcp/server.go`) - - 实现MCP协议服务器 - - 管理工具注册和调用 - - 跟踪工具执行状态和统计 +--- -- **Security Executor** (`internal/security/executor.go`) - - 执行安全工具命令 - - 构建工具参数 - - 解析工具输出 - -- **Database** (`internal/database/`) - - SQLite数据库操作 - - 对话和消息管理 - - 过程详情存储 - -### 添加新工具 - -#### 推荐方式：使用工具配置文件 - -1. 在 `tools/` 目录创建工具配置文件（如 `tools/mytool.yaml`） -2. 定义工具参数和描述 -3. 重启服务器，工具自动加载 - -#### 高级方式：自定义参数构建逻辑 - -如果工具需要特殊的参数处理，可以在 `internal/security/executor.go` 的 `buildCommandArgs` 方法中添加： - -```go -case "mytool": - // 自定义参数构建逻辑 - if target, ok := args["target"].(string); ok { - cmdArgs = append(cmdArgs, "--target", target) - } -``` - -### 构建和部署 - -#### 本地构建 - -```bash -go build -o cyberstrike-ai cmd/server/main.go -``` - -#### 交叉编译 - -```bash -# Linux -GOOS=linux GOARCH=amd64 go build -o cyberstrike-ai-linux cmd/server/main.go - -# macOS -GOOS=darwin GOARCH=amd64 go build -o cyberstrike-ai-macos cmd/server/main.go - -# Windows -GOOS=windows GOARCH=amd64 go build -o cyberstrike-ai.exe cmd/server/main.go -``` - -#### Docker部署（示例） - -```dockerfile -FROM golang:1.21-alpine AS builder -WORKDIR /app -COPY . . -RUN go build -o cyberstrike-ai cmd/server/main.go - -FROM alpine:latest -RUN apk --no-cache add ca-certificates nmap sqlmap nikto dirb -WORKDIR /root/ -COPY --from=builder /app/cyberstrike-ai . -COPY --from=builder /app/config.yaml . -COPY --from=builder /app/tools ./tools -COPY --from=builder /app/web ./web -EXPOSE 8080 -CMD ["./cyberstrike-ai"] -``` - -### 代码贡献 - -欢迎提交Issue和Pull Request！ - -贡献指南： -1. Fork本项目 -2. 创建特性分支 (`git checkout -b feature/AmazingFeature`) -3. 提交更改 (`git commit -m 'Add some AmazingFeature'`) -4. 推送到分支 (`git push origin feature/AmazingFeature`) -5. 开启Pull Request - -## 📋 技术栈 - -- **后端框架**: Gin (Go Web Framework) -- **数据库**: SQLite3 -- **日志**: Zap (Uber's structured logging) -- **配置**: YAML -- **协议**: MCP (Model Context Protocol) -- **前端**: HTML/CSS/JavaScript (原生，无框架依赖) - -## 🔐 安全注意事项 - -⚠️ **重要提示**： - -- ⚠️ **仅对您拥有或已获得授权的系统进行测试** -- ⚠️ **遵守相关法律法规和道德准则** -- ⚠️ **建议在隔离的测试环境中使用** -- ⚠️ **不要在生产环境中使用** -- ⚠️ **某些安全工具可能需要root权限，请谨慎使用** -- ⚠️ **exec工具可以执行任意系统命令，存在安全风险** -- ⚠️ **妥善保管API密钥，不要提交到代码仓库** - -## ⚙️ 高级特性 - -### AI迭代机制 - -- **最大迭代次数**：系统支持多轮AI迭代，确保复杂测试任务能够完成 -- **智能总结**：当达到最大迭代次数时，AI会自动总结所有测试结果、发现的问题和已完成的工作 -- **下一步计划**：如果测试未完成，AI会提供详细的下一步执行计划，指导后续测试 - -### 工具执行优化 - -- **错误处理**：工具执行失败时，AI会自动分析错误原因并尝试替代方案 -- **参数优化**：AI会根据目标特征自动优化工具参数，提高测试效率 -- **结果分析**：自动分析工具输出，提取关键信息和漏洞 - -## 📄 许可证 - -本项目仅供学习和研究使用。 - -## 🤝 贡献 - -欢迎提交Issue和Pull Request！ - -如果您发现bug或有功能建议，请： -1. 查看现有的Issues，避免重复 -2. 创建详细的Issue描述问题或建议 -3. 提交Pull Request时请包含清晰的说明 - -## 📞 支持 - -如有问题或需要帮助，请： -- 查看本文档的故障排除部分 -- 查看项目的Issues -- 提交新的Issue描述您的问题 - -## 🙏 致谢 - -感谢所有贡献者和开源社区的支持！ +欢迎提交 Issue/PR 贡献新的工具模版或优化建议！