Add files via upload

2026-07-10 14:28:43 +02:00 · 2026-07-07 14:04:18 +08:00
parent 3dfcde7c88
commit 00802a275c
61 changed files with 9460 additions and 24 deletions
@@ -0,0 +1,66 @@
+# Eino Multi-Agent Notes
+
+[中文](../zh-CN/MULTI_AGENT_EINO.md)
+
+CyberStrikeAI uses CloudWeGo Eino ADK for the current single-agent and multi-agent execution paths. The native legacy ReAct path has been removed.
+
+## Entrypoints
+
+- Single-agent: `/api/eino-agent` and `/api/eino-agent/stream`
+- Multi-agent: `/api/multi-agent` and `/api/multi-agent/stream`
+
+Multi-agent orchestration is selected by request body:
+
+- `deep`
+- `plan_execute`
+- `supervisor`
+
+Robots default to `robot_default_agent_mode`, and batch tasks can opt into multi-agent through config.
+
+## Agent Definitions
+
+Markdown agents live under `agents/`.
+
+Typical files:
+
+```text
+agents/orchestrator.md
+agents/orchestrator-plan-execute.md
+agents/orchestrator-supervisor.md
+agents/*.md
+```
+
+Front matter controls name, id, description, tools, bound role, max iterations, and optional orchestrator kind.
+
+## Middleware
+
+Important Eino middleware:
+
+- tool search: exposes a small visible tool set and unlocks others on demand;
+- patch tool calls: repairs interrupted histories;
+- plan task: structured task board;
+- reduction: truncates or persists large tool outputs;
+- summarization: compresses long contexts;
+- checkpoint: resume after crash/OOM.
+
+These settings live under `multi_agent.eino_middleware`.
+
+## Skills
+
+Eino Skills support progressive disclosure. The Agent initially sees names and descriptions; details are loaded only when needed through the configured skill tool.
+
+## Operational Notes
+
+- Tool visibility is not the same as tool availability in the UI.
+- Running streams keep their startup context even if config changes mid-run.
+- Summarization can write transcripts under `data/conversation_artifacts/...`.
+- High-risk tools should still be constrained by roles and HITL.
+
+## Source Anchors
+
+- Multi-agent handler: `internal/handler/multi_agent.go`
+- Preparation: `internal/handler/multi_agent_prepare.go`
+- Orchestration: `internal/multiagent/eino_orchestration.go`
+- Run loop: `internal/multiagent/eino_adk_run_loop.go`
+- Skills: `internal/multiagent/eino_skills.go`
+- Middleware: `internal/multiagent/eino_middleware.go`
@@ -0,0 +1,29 @@
+# English Docs
+
+- [Deployment Guide](deployment.md): deployment modes, HTTPS, reverse proxy, systemd, backup, upgrade, and acceptance checks.
+- [Runbooks](runbooks.md): operational steps for production setup, external MCP, KB, Web testing, C2 cleanup, and tool debugging.
+- [Configuration Profiles](configuration-profiles.md): recommended profiles for dev, internal team, knowledge-only, production, C2, and MCP automation.
+- [Security Hardening](security-hardening.md): pre-launch baseline, reverse proxy, HITL allowlist, file permissions, and periodic review.
+- [API Recipes](api-recipes.md): examples for login, Agent, streaming, multi-agent, uploads, vulnerabilities, KB, MCP, and audit export.
+- [Contributing Guide](contributing-guide.md): checklists for APIs, config, tools, frontend, DB, high-risk features, and docs.
+- [Configuration Reference](configuration.md): `config.yaml` fields, hot-apply boundaries, recommended values, and source anchors.
+- [Security Model](security-model.md): trust boundaries, HITL, tool execution, C2/WebShell, and data safety.
+- [Architecture](architecture.md): request flow, module relationships, complexity hotspots, and design trade-offs.
+- [API Reference](api-reference.md): authentication, OpenAPI, SSE, stability tiers, and common endpoints.
+- [Troubleshooting](troubleshooting.md): diagnostic order, minimal commands, common misdiagnoses, and issue template.
+- [Audit and Monitoring](audit-and-monitoring.md): platform audit, tool monitoring, HITL logs, and retention.
+- [Knowledge Base](knowledge-base.md): indexing pipeline, retrieval tuning, log analysis, and content writing.
+- [C2 Guide](c2.md): lifecycle, task classification, event review, and safety guidance.
+- [WebShell Management](webshell.md): operation tiers, naming, AI guardrails, and troubleshooting.
+- [MCP Federation](mcp-federation.md): built-in MCP, external MCP, lifecycle, and tool naming.
+- [Agent and Role Guide](agent-and-role-guide.md): roles, sub-agents, Skills, orchestration modes, and tool visibility.
+- [Skills Guide](skills-guide.md): Skill structure, progressive disclosure, anti-patterns, and local-tool risk.
+- [Plugin Development](plugin-development.md): API plugins, MCP plugins, resource-pack plugins, and security boundaries.
+- [Release Process](release-process.md): release risk, config compatibility, DB migrations, and acceptance checks.
+- [Testing Guide](testing.md): test layers, regression focus, test data, and failure cases.
+- [Graph Orchestration Guide](workflow-graph.md)
+- [HITL Best Practices](hitl-best-practices.md)
+- [Robot / Chatbot Guide](robot.md)
+- [Vision Analysis](VISION.md)
+- [Frontend i18n](frontend-i18n.md)
+- [Eino Multi-Agent Notes](MULTI_AGENT_EINO.md)
@@ -0,0 +1,61 @@
+# Vision Analysis
+
+[中文](../zh-CN/VISION.md)
+
+Vision analysis registers the `analyze_image` MCP tool when enabled. It is intended for screenshots, captchas, UI states, and image evidence in authorized workflows.
+
+## Config
+
+```yaml
+vision:
+  enabled: true
+  model: qwen-vl
+  api_key: ""
+  base_url: ""
+  provider: ""
+  max_image_bytes: 5242880
+  max_dimension: 2048
+  jpeg_quality: 82
+  max_payload_bytes: 524288
+  detail: auto
+  timeout_seconds: 60
+```
+
+Empty `api_key`, `base_url`, or `provider` inherits from `openai`.
+
+## Data Handling
+
+Image bytes are sent only to the vision model call. Agent history keeps text summaries, not raw image bytes. This reduces context size and accidental image propagation.
+
+## Preprocessing
+
+The runtime can resize and recompress large images based on:
+
+- maximum file size;
+- maximum dimension;
+- JPEG quality;
+- encoded payload size.
+
+If small images are already under limits, preprocessing may be skipped.
+
+## Usage Guidance
+
+Use vision for:
+
+- UI screenshots;
+- visual vulnerability evidence;
+- captcha or image-based prompts in authorized tests;
+- interpreting tool screenshots.
+
+Do not use it for:
+
+- unrelated personal images;
+- sensitive screenshots without authorization;
+- long-term storage of raw evidence when a text summary is enough.
+
+## Source Anchors
+
+- Tool registration: `internal/app/vision_tools.go`
+- Client: `internal/vision/client.go`
+- Preprocess: `internal/vision/preprocess.go`
+- Config: `internal/config/vision.go`
@@ -0,0 +1,79 @@
+# Agent and Role Guide
+
+[中文](../zh-CN/agent-and-role-guide.md)
+
+Agent behavior is shaped by roles, Markdown sub-agents, Skills, tool visibility, and HITL policy.
+
+## Responsibility Boundaries
+
+| Resource | Purpose | Not for |
+| --- | --- | --- |
+| Role | identity, tone, tool boundary, authorization rules | large reference material |
+| Agent Markdown | multi-agent specialization, handoff format, local strategy | one-off facts |
+| Skill | reusable procedures, checklists, templates, references | permission control |
+
+Authorization boundaries belong in roles and HITL first, not only in Skills.
+
+## Modes
+
+| Mode | Good for | Poor fit |
+| --- | --- | --- |
+| `eino_single` | short tasks, interactive analysis | large multi-stage work |
+| `deep` | dynamic task decomposition | strict sequential workflows |
+| `plan_execute` | plan, execute, replan loops | frequent user interruption |
+| `supervisor` | expert routing | vague or too many sub-agents |
+
+Start with `eino_single`; use `plan_execute` for structured projects; use `deep` or `supervisor` when specialist agents matter.
+
+## Markdown Sub-Agent
+
+Example:
+
+```yaml
+---
+name: Vulnerability Triage
+id: vulnerability-triage
+description: Validate, classify, and summarize vulnerability evidence
+tools:
+  - nmap
+  - nuclei
+bind_role: 综合漏洞扫描
+max_iterations: 200
+---
+```
+
+The body should define scope, tool order, output format, and prohibited actions.
+
+## Tool Visibility
+
+With `tool_search`, the model initially sees only a subset of tools:
+
+- visible in UI does not mean visible in current model context;
+- `tool_search_always_visible_tools` are easier to call;
+- clear tool descriptions improve search hits;
+- sub-agent tool constraints still matter.
+
+When a tool is not used, check role tools, sub-agent tools, tool_search config, and description.
+
+## Output Format
+
+Sub-agents should return structured results:
+
+```markdown
+## Conclusion
+## Evidence
+- Tool:
+- Key output:
+- Confidence:
+## Risks
+## Suggested next step
+```
+
+This helps the orchestrator continue and supports reporting.
+
+## Source Anchors
+
+- Markdown Agent parser: `internal/agents/markdown.go`
+- Multi-agent preparation: `internal/handler/multi_agent_prepare.go`
+- Orchestration: `internal/multiagent/eino_orchestration.go`
+- Tool search middleware: `internal/multiagent/eino_middleware.go`
@@ -0,0 +1,153 @@
+# API Recipes
+
+[中文](../zh-CN/api-recipes.md)
+
+Common API workflows for scripts and plugins. Use `/api-docs` and `/api/openapi/spec` for complete schemas.
+
+## Recipe 1: Login and Validate
+
+```bash
+curl -k https://127.0.0.1:8080/api/auth/login \
+  -H "Content-Type: application/json" \
+  -d '{"password":"<password>"}'
+```
+
+Use:
+
+```text
+Authorization: Bearer <token>
+```
+
+Validate:
+
+```bash
+curl -k https://127.0.0.1:8080/api/auth/validate \
+  -H "Authorization: Bearer <token>"
+```
+
+## Recipe 2: Create Conversation and Send Message
+
+Simplest path: call Agent without pre-creating an empty conversation.
+
+```bash
+curl -k https://127.0.0.1:8080/api/eino-agent \
+  -H "Authorization: Bearer <token>" \
+  -H "Content-Type: application/json" \
+  -d '{"message":"Run authorized basic read-only recon against 127.0.0.1"}'
+```
+
+If you need an empty conversation first:
+
+```bash
+curl -k https://127.0.0.1:8080/api/conversations \
+  -H "Authorization: Bearer <token>" \
+  -H "Content-Type: application/json" \
+  -d '{"title":"Web Test"}'
+```
+
+Then pass `conversationId` to the Agent request.
+
+## Recipe 3: Stream Agent Output
+
+```bash
+curl -k -N https://127.0.0.1:8080/api/eino-agent/stream \
+  -H "Authorization: Bearer <token>" \
+  -H "Content-Type: application/json" \
+  -d '{"message":"Summarize current project facts and propose read-only next steps"}'
+```
+
+Notes:
+
+- `-N` disables curl buffering.
+- reverse proxy buffering must also be disabled.
+- wait for `done`.
+
+## Recipe 4: Multi-Agent
+
+```bash
+curl -k -N https://127.0.0.1:8080/api/multi-agent/stream \
+  -H "Authorization: Bearer <token>" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "message":"Run a staged authorized Web security test; plan first, execute read-only steps",
+    "orchestration":"plan_execute"
+  }'
+```
+
+Options:
+
+- `deep`
+- `plan_execute`
+- `supervisor`
+
+## Recipe 5: Upload Attachment
+
+```bash
+curl -k https://127.0.0.1:8080/api/chat-uploads \
+  -H "Authorization: Bearer <token>" \
+  -F "file=@./request.txt"
+```
+
+Upload large files and reference them in messages instead of pasting raw content.
+
+## Recipe 6: Create Vulnerability
+
+```bash
+curl -k https://127.0.0.1:8080/api/vulnerabilities \
+  -H "Authorization: Bearer <token>" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "title":"Example SQL Injection",
+    "severity":"high",
+    "target":"https://example.com/item?id=1",
+    "description":"Parameter id has verified SQL injection",
+    "evidence":"read-only validation output...",
+    "remediation":"Use parameterized queries"
+  }'
+```
+
+Check OpenAPI for exact fields.
+
+## Recipe 7: Search Knowledge Base
+
+```bash
+curl -k https://127.0.0.1:8080/api/knowledge/search \
+  -H "Authorization: Bearer <token>" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "query":"How to infer SQL injection column count",
+    "riskType":"SQL Injection",
+    "topK":5,
+    "threshold":0.4
+  }'
+```
+
+If empty, check categories first.
+
+## Recipe 8: External MCP Status
+
+```bash
+curl -k https://127.0.0.1:8080/api/external-mcp/stats \
+  -H "Authorization: Bearer <token>"
+```
+
+If service is running but Agent cannot find tools, check role constraints and `tool_search`.
+
+## Recipe 9: Tool Schema
+
+```bash
+curl -k https://127.0.0.1:8080/api/config/tools/nmap/schema \
+  -H "Authorization: Bearer <token>"
+```
+
+Scripts should build args from schema rather than guessing field names.
+
+## Recipe 10: Export Audit Logs
+
+```bash
+curl -k "https://127.0.0.1:8080/api/audit/logs/export" \
+  -H "Authorization: Bearer <token>" \
+  -o audit.csv
+```
+
+Exported logs may contain sensitive operational data. Store encrypted.
@@ -0,0 +1,103 @@
+# API Reference
+
+[中文](../zh-CN/api-reference.md)
+
+CyberStrikeAI exposes built-in OpenAPI docs:
+
+```text
+/api-docs
+GET /api/openapi/spec
+```
+
+The OpenAPI spec is protected to avoid exposing the API surface to unauthenticated users.
+
+## Authentication
+
+Login:
+
+```http
+POST /api/auth/login
+Content-Type: application/json
+
+{"password":"your-password"}
+```
+
+The auth middleware accepts token from:
+
+1. `Authorization: Bearer <token>`
+2. `Authorization: <token>`
+3. `?token=<token>`
+4. `auth_token` cookie
+
+Prefer `Authorization: Bearer` for scripts. Query tokens can leak through logs.
+
+## Agent APIs
+
+Single-agent:
+
+- `POST /api/eino-agent`
+- `POST /api/eino-agent/stream`
+
+Multi-agent:
+
+- `POST /api/multi-agent`
+- `POST /api/multi-agent/stream`
+
+`orchestration` may be `deep`, `plan_execute`, or `supervisor`.
+
+## SSE Notes
+
+Streaming endpoints are long-lived. Clients should:
+
+- handle `error` events;
+- wait for `done`;
+- avoid blindly replaying destructive requests;
+- disable proxy buffering;
+- pass `conversationId` when continuing a conversation.
+
+## Stability Tiers
+
+| API type | Stability | Recommendation |
+| --- | --- | --- |
+| `/api/auth/*` | high | safe to integrate |
+| `/api/eino-agent*` | high | preferred chat entry |
+| `/api/openapi/spec` | high | client generation |
+| `/api/config*` | medium | admin automation only |
+| `/api/c2/*`, `/api/webshell/*` | medium | high-risk, restrict access |
+| frontend private calls | low | avoid plugin dependency |
+
+## Common Areas
+
+- Conversations: `/api/conversations`
+- Projects/facts: `/api/projects`
+- Vulnerabilities: `/api/vulnerabilities`
+- Knowledge: `/api/knowledge/*`
+- Roles: `/api/roles`
+- Skills: `/api/skills`
+- External MCP: `/api/external-mcp`
+- Monitoring: `/api/monitor`
+- Audit: `/api/audit`
+- C2: `/api/c2`
+- WebShell: `/api/webshell`
+
+## Curl Example
+
+```bash
+curl -k https://127.0.0.1:8080/api/conversations \
+  -H "Authorization: Bearer <token>"
+```
+
+```bash
+curl -k https://127.0.0.1:8080/api/eino-agent \
+  -H "Authorization: Bearer <token>" \
+  -H "Content-Type: application/json" \
+  -d '{"message":"Run authorized basic recon against 127.0.0.1; avoid high-risk actions."}'
+```
+
+## Source Anchors
+
+- Routes: `internal/app/app.go`
+- Auth middleware: `internal/security/auth_middleware.go`
+- OpenAPI: `internal/handler/openapi.go`
+- Single-agent: `internal/handler/eino_single_agent.go`
+- Multi-agent: `internal/handler/multi_agent.go`
@@ -0,0 +1,74 @@
+# Architecture
+
+[中文](../zh-CN/architecture.md)
+
+CyberStrikeAI is a single Go Web application with a static frontend, SQLite persistence, Agent orchestration, MCP tooling, workflow graphs, knowledge retrieval, and optional C2/WebShell subsystems.
+
+## Overview
+
+```mermaid
+flowchart LR
+    U["Web / Robot / API"] --> R["Gin Router"]
+    R --> H["Handlers"]
+    H --> DB["SQLite"]
+    H --> A["Agent / Multi-Agent"]
+    A --> M["MCP Server"]
+    M --> T["Built-in / YAML / Skill tools"]
+    M --> EM["External MCP"]
+    A --> K["Knowledge Retrieval"]
+    H --> W["Workflow Runtime"]
+    H --> C2["C2"]
+    H --> WS["WebShell"]
+    H --> AU["Audit / Monitor"]
+```
+
+## Request Path
+
+For `/api/eino-agent/stream`:
+
+1. Gin route enters auth middleware.
+2. Handler parses message, conversation, role, uploads, and WebShell context.
+3. Agent builds model input: history, role prompt, project facts, tools.
+4. Eino Runner calls the model.
+5. Tool requests go through MCP.
+6. HITL may interrupt before execution.
+7. Tool results are saved to process details and monitoring.
+8. Model continues and produces final text.
+9. SSE streams progress and deltas to the browser.
+10. Conversation and process details persist to SQLite.
+
+This explains why a failure may live in auth, config, model, MCP, HITL, DB, SSE, or frontend rendering.
+
+## Cross-Cutting Modules
+
+- Project facts are injected into Agent context.
+- HITL sits before tool execution.
+- Monitor records tool execution and supports cancellation/review.
+- Audit records platform management actions.
+- Tool search controls what tools the model can currently see.
+
+These are not just pages; they affect many runtime paths.
+
+## Complexity Hotspots
+
+- `internal/app/app.go`: service construction and route wiring.
+- `internal/handler/config.go`: hot application of config across model, KB, C2, robot, MCP.
+- `internal/multiagent/`: streaming, retry, summarization, middleware, tools.
+- `internal/security/`: auth and shell execution boundary.
+- `internal/database/`: SQLite schema compatibility.
+
+## Design Trade-Offs
+
+The project uses a single Go service, static frontend, and SQLite to keep deployment simple. The trade-offs:
+
+- multi-instance scale is not automatic;
+- runtime files must be backed up carefully;
+- high-privilege tools and admin UI live in one process, so deployment isolation matters.
+
+## Source Anchors
+
+- App wiring: `internal/app/app.go`
+- Handlers: `internal/handler/`
+- Multi-agent: `internal/multiagent/`
+- MCP: `internal/mcp/`
+- DB: `internal/database/`
@@ -0,0 +1,87 @@
+# Audit and Monitoring
+
+[中文](../zh-CN/audit-and-monitoring.md)
+
+CyberStrikeAI has separate observability streams:
+
+- Audit: who performed platform management actions.
+- Monitor: how tool executions ran.
+- HITL logs: why a tool call was approved, edited, or rejected.
+- Process details: how an Agent chained reasoning, tools, and outputs.
+
+Use them together during review.
+
+## Audit
+
+Config:
+
+```yaml
+audit:
+  enabled: true
+  retention_days: 15
+  max_detail_bytes: 8192
+```
+
+Endpoints:
+
+- `GET /api/audit/meta`
+- `GET /api/audit/summary`
+- `GET /api/audit/logs`
+- `GET /api/audit/logs/:id`
+- `GET /api/audit/logs/export`
+
+Watch for login failures, password changes, config updates, external MCP changes, WebShell/C2 actions, and HITL rejections.
+
+## Tool Monitoring
+
+Config:
+
+```yaml
+monitor:
+  retention_days: 90
+```
+
+Endpoints:
+
+- `GET /api/monitor`
+- `GET /api/monitor/execution/:id`
+- `POST /api/monitor/execution/:id/cancel`
+- `GET /api/monitor/stats`
+- `GET /api/monitor/calls-timeline`
+
+Monitoring is for execution state, duration, cancellation, and result review. It is not a substitute for platform audit.
+
+## Retention Guidance
+
+Security-tool logs can include targets, paths, commands, and sensitive outputs. Longer retention is not always safer.
+
+- Short engagements: 15-30 days.
+- Continuous red-team platform: 90-180 days.
+- Compliance archive: export and encrypt.
+
+## Review Checklist
+
+Weekly:
+
+- failed logins and unusual IPs;
+- config changes;
+- long-running or frequently failing tools;
+- external MCP state;
+- DB size and disk.
+
+After engagement:
+
+- export required evidence;
+- delete stale WebShell/C2 resources;
+- clean uploads and temporary workspaces;
+- archive reports, vulnerabilities, and project facts.
+
+## Source Anchors
+
+- Audit service: `internal/audit/service.go`
+- Sanitization: `internal/audit/sanitize.go`
+- Retention: `internal/audit/retention.go`
+- Audit handler: `internal/handler/audit.go`
+- Monitor: `internal/monitor/reconcile.go`
+- Monitor handler: `internal/handler/monitor.go`
+- HITL logs: `internal/handler/hitl_logs.go`
@@ -0,0 +1,68 @@
+# C2 Guide
+
+[中文](../zh-CN/c2.md)
+
+The built-in C2 subsystem is for authorized environments only. Disable it when not needed:
+
+```yaml
+c2:
+  enabled: false
+```
+
+## Objects
+
+- Listener: receives sessions.
+- Session: connected implant/session.
+- Task: command or operation assigned to a session.
+- Payload: generated binary or one-liner.
+- Profile: communication configuration.
+- Event: runtime event stream.
+- File: upload/download channel.
+
+APIs live under `/api/c2`; disabled C2 returns `503 c2_disabled`.
+
+## Lifecycle
+
+Correct C2 operation is a lifecycle:
+
+1. Authorization: project, targets, time window, allowed actions.
+2. Profile design: transport, sleep, callback address.
+3. Listener start: port, network path, logs.
+4. Payload generation: hash, purpose, delivery method.
+5. Session intake: confirm host identity and privilege.
+6. Tasking: only authorized tasks.
+7. Result archival: project facts or report.
+8. Cleanup: stop listeners, delete payloads, clear stale sessions/events.
+
+Skipping authorization and profile design makes the rest hard to audit.
+
+## Task Classification
+
+| Level | Example | Approval |
+| --- | --- | --- |
+| L1 read-only identity | `whoami`, hostname | audit agent may approve |
+| L2 environment enum | interfaces, processes | strict review |
+| L3 file access | read config, download result | human confirms path |
+| L4 change execution | upload, run script, sleep change | human approval |
+| L5 persistence/lateral/destructive | startup, creds, delete, spread | reject unless explicit authorization |
+
+Put this classification into HITL prompts for practical decisions.
+
+## Review Questions
+
+- Which listener received which session?
+- Who generated the payload and when?
+- Which authorized target does the session represent?
+- Which tasks were issued?
+- Were outputs saved into facts or reports?
+- Were listener and payload cleaned up?
+
+## Source Anchors
+
+- Manager: `internal/c2/manager.go`
+- Listener: `internal/c2/listener.go`
+- HTTP listener: `internal/c2/listener_http.go`
+- TCP listener: `internal/c2/listener_tcp.go`
+- Payload: `internal/c2/payload_builder.go`
+- Handler: `internal/handler/c2.go`
+- MCP tools: `internal/app/c2_tools.go`
@@ -0,0 +1,159 @@
+# Configuration Profiles
+
+[中文](../zh-CN/configuration-profiles.md)
+
+These profiles are not full `config.yaml` files. They show the key sections that most affect safety and operability.
+
+## Local Development
+
+Goal: easy debugging with local capabilities.
+
+Common startup:
+
+```bash
+chmod +x run.sh && ./run.sh
+```
+
+```yaml
+server:
+  host: 127.0.0.1
+  port: 8080
+  tls_enabled: true
+  tls_auto_self_sign: true
+auth:
+  password: "dev-only-change-me"
+audit:
+  enabled: true
+  retention_days: 7
+c2:
+  enabled: false
+multi_agent:
+  enabled: true
+  eino_skills:
+    filesystem_tools: true
+```
+
+Not for shared or public use.
+
+## Internal Team
+
+Goal: shared team instance with audit and limited high-risk surface.
+
+```yaml
+server:
+  host: 127.0.0.1
+  port: 8080
+  tls_enabled: false
+auth:
+  password: "<long-random-password>"
+audit:
+  enabled: true
+  retention_days: 30
+monitor:
+  retention_days: 90
+c2:
+  enabled: false
+mcp:
+  enabled: false
+hitl:
+  default_reviewer: human
+  tool_whitelist: [read_file, glob, grep, tool_search]
+```
+
+Pair with reverse-proxy HTTPS, IP allowlist, and backups.
+
+## Knowledge-Only Assistant
+
+Goal: use CyberStrikeAI as a knowledge-augmented assistant with minimal attack surface.
+
+```yaml
+c2:
+  enabled: false
+mcp:
+  enabled: false
+knowledge:
+  enabled: true
+  base_path: knowledge_base
+  retrieval:
+    top_k: 5
+    similarity_threshold: 0.4
+multi_agent:
+  eino_skills:
+    filesystem_tools: false
+```
+
+Use read-only roles and avoid storing sensitive customer data.
+
+## High-Audit Production
+
+Goal: long-running production red-team or security platform.
+
+```yaml
+auth:
+  password: "<managed-secret>"
+  session_duration_hours: 8
+audit:
+  enabled: true
+  retention_days: 90
+monitor:
+  retention_days: 180
+hitl:
+  default_reviewer: human
+  retention_days: 180
+  tool_whitelist: [read_file, glob, grep, tool_search]
+c2:
+  enabled: false
+multi_agent:
+  eino_callbacks:
+    enabled: true
+    mode: log_only
+    sse_trace_to_client: false
+```
+
+Pair with proxy auth, dedicated OS user, log collection, encrypted backups, and project closeout cleanup.
+
+## C2 Exercise Window
+
+Goal: temporarily enable C2 only during authorized exercise.
+
+```yaml
+c2:
+  enabled: true
+hitl:
+  default_reviewer: human
+  tool_whitelist: [read_file, glob, grep, tool_search]
+audit:
+  enabled: true
+monitor:
+  retention_days: 180
+```
+
+Requirements:
+
+- confirm scope before exercise;
+- separate listener ports from admin UI;
+- run C2 cleanup afterward;
+- restore `c2.enabled: false`.
+
+## External MCP Automation
+
+Goal: connect trusted internal tool services.
+
+```yaml
+external_mcp:
+  servers: {}
+multi_agent:
+  eino_middleware:
+    tool_search_enable: true
+    tool_search_min_tools: 20
+hitl:
+  default_reviewer: audit_agent
+  tool_whitelist: [read_file, glob, grep, tool_search]
+```
+
+Guidance:
+
+- every MCP tool needs clear schema;
+- high-risk MCP tools stay out of allowlist;
+- stdio MCP gets its own working directory;
+- HTTP MCP must authenticate.
@@ -0,0 +1,86 @@
+# Configuration Reference
+
+[中文](../zh-CN/configuration.md)
+
+The main configuration file is `config.yaml`. Many fields are editable through the Web settings page, but not every field has the same hot-apply behavior.
+
+## Core Sections
+
+```yaml
+server:
+  host: 0.0.0.0
+  port: 8080
+  tls_enabled: true
+auth:
+  password: "change-me"
+  session_duration_hours: 12
+openai:
+  provider: openai
+  base_url: https://api.openai.com/v1
+  api_key: sk-...
+  model: gpt-4.1
+agent:
+  max_iterations: 12000
+  tool_timeout_minutes: 60
+```
+
+Change the default password immediately. Use HTTPS or a trusted reverse proxy in any shared environment.
+
+## Hot-Apply Boundaries
+
+`POST /api/config/apply` coordinates model config, tool description mode, MCP tool registration, knowledge components, robot restarts, and C2 runtime reconciliation. It does not make every field instantly effective.
+
+| Section | Usually hot-applies | Extra action |
+| --- | --- | --- |
+| `openai` | new requests use new model settings | running streams keep their current state |
+| `agent.max_iterations` | new tasks | existing tasks continue |
+| `hitl.tool_whitelist` | new approval checks | pending approvals are not re-decided |
+| `knowledge.enabled` | initializes/updates components | scan and index are still required |
+| `knowledge.embedding` | updates retriever/indexer config | rebuild index for existing vectors |
+| `robots` | restarts long-lived connections | platform callback settings must still match |
+| `c2.enabled` | reconciles C2 runtime | verify existing listeners/sessions manually |
+| `server.port/tls` | usually needs process restart | listener settings are not ordinary hot state |
+
+## Fallback Relationships
+
+- `vision.api_key/base_url/provider` can inherit from `openai`.
+- `hitl.audit_model` can inherit from `openai`.
+- `knowledge.embedding.base_url/api_key` can inherit from model settings.
+- rerank config can inherit from embedding/openai.
+- `database.knowledge_db_path` can be separate or reuse the main DB.
+
+When debugging, inspect both the child config and the fallback parent.
+
+## Recommended Values
+
+| Field | Conservative | Aggressive | Decide by |
+| --- | --- | --- | --- |
+| `agent.tool_timeout_minutes` | 10-30 | 60+ | long scanners |
+| `shell_no_output_timeout_seconds` | 300-600 | 1200+ | quiet tools |
+| `knowledge.indexing.batch_size` | 5-10 | 20+ | embedding API limits |
+| `knowledge.indexing.rate_limit_delay_ms` | 300-800 | 0-100 | 429 frequency |
+| `retrieval.top_k` | 3-5 | 8-12 | context budget |
+| `similarity_threshold` | 0.35-0.45 | 0.5+ | recall vs precision |
+| `audit.retention_days` | 15-30 | 90+ | compliance and disk |
+
+## Change Template
+
+Before changing config, write down:
+
+```text
+Purpose:
+Sections:
+Expected impact:
+Rollback:
+Validation endpoints:
+```
+
+After changing, validate the specific subsystem rather than trusting the save message.
+
+## Source Anchors
+
+- Config structs: `internal/config/config.go`
+- Env expansion: `internal/config/envexpand.go`
+- Config API and apply: `internal/handler/config.go`
+- Route registration: `internal/app/app.go`
+- C2 reconciliation: `internal/app/c2_lifecycle.go`
@@ -0,0 +1,115 @@
+# Contributing Guide
+
+[中文](../zh-CN/contributing-guide.md)
+
+This guide defines baseline expectations when adding features, APIs, tools, frontend pages, or docs.
+
+## Principles
+
+- New features need documentation.
+- New APIs need OpenAPI updates.
+- New frontend text needs zh-CN and en-US i18n.
+- New config must state hot-apply behavior.
+- New high-risk tools must define HITL policy.
+- New DB fields must be compatible with old databases.
+- New long-running tasks need state, cancellation, or recovery strategy.
+
+## New API Checklist
+
+- Handler validates parameters.
+- Error response has stable `error` and readable `message`.
+- Endpoint is authenticated unless it is an explicit platform callback.
+- Mutations write audit events.
+- Long tasks write monitoring/task state.
+- `internal/handler/openapi.go` updated.
+- API docs or recipes updated.
+- Handler tests added.
+
+## New Config Checklist
+
+- Field exists in `config.Config`.
+- `config.yaml` sample has comments.
+- Safe default when omitted.
+- Old configs still start.
+- Hot-apply behavior documented.
+- Web settings do not delete unknown fields.
+- Security docs updated if high-risk capability is affected.
+
+## New Tool Checklist
+
+For YAML tools and Go MCP tools:
+
+- stable and specific tool name;
+- searchable `short_description`;
+- explicit input schema, not one raw `cmd`;
+- readable and stable output;
+- controlled timeout and error path;
+- high-risk operation not globally allowlisted;
+- docs explain use case and risk.
+
+## New Frontend Page Checklist
+
+- Reuse `apiFetch`, modal, notifications, and existing state patterns.
+- Add all visible text to `zh-CN.json` and `en-US.json`.
+- Include loading, empty, and error states.
+- Confirm destructive/high-risk actions.
+- Avoid overflow in long English labels.
+- Browser console clean.
+
+## DB Change Checklist
+
+- Migration is idempotent.
+- Old DB upgrades.
+- Defaults are safe.
+- Large indexes are deliberate.
+- Empty DB and old DB tested.
+- Release notes mention backup.
+
+## High-Risk Capability Checklist
+
+High-risk includes Shell, WebShell, C2, external MCP write/execute, credential access, and bulk scanning.
+
+Answer:
+
+- Who can call it?
+- Does it require HITL?
+- What is audited?
+- How can it be cancelled?
+- How is cleanup done?
+- How can it be disabled?
+- Is it off by default?
+
+## Documentation Requirements
+
+Each important feature should document:
+
+- purpose;
+- config;
+- workflow;
+- risk boundary;
+- troubleshooting;
+- source anchors.
+
+Chinese and English docs must have matching filenames:
+
+```text
+docs/zh-CN/
+docs/en-US/
+```
+
+Update:
+
+- `docs/README.md`
+- `docs/zh-CN/README.md`
+- `docs/en-US/README.md`
+
+## Review Focus
+
+Prioritize:
+
+- behavior regressions;
+- security boundaries;
+- old data compatibility;
+- error handling;
+- test gaps;
+- docs and OpenAPI sync.
@@ -0,0 +1,113 @@
+# Deployment Guide
+
+[中文](../zh-CN/deployment.md)
+
+CyberStrikeAI can run as a local testing tool, an internal team service, or a production red-team platform. Treat it as a high-privilege security system: it can execute commands, call MCP tools, manage WebShell connections, and optionally run C2 listeners.
+
+## Prerequisites
+
+- Go for source runs and binary builds.
+- Python for some MCP servers and tool scripts.
+- SQLite files under `data/`; no external DB is required by default.
+- Actual security tools installed in PATH. YAML files under `tools/` only describe commands.
+- An OpenAI-compatible model endpoint, or `openai.provider: claude` for the Claude bridge.
+
+Important persistent paths:
+
+```text
+config.yaml
+data/
+tools/
+roles/
+skills/
+agents/
+knowledge_base/
+chat_uploads/
+```
+
+Back these up before upgrades.
+
+## Startup Modes
+
+Local quick start:
+
+```bash
+chmod +x run.sh && ./run.sh
+```
+
+`run.sh` is the most common startup path for local use, development, small temporary internal deployments, and quick post-upgrade verification.
+
+For long-running service, boot-time startup, managed logs, and crash recovery, prefer a binary managed by systemd.
+
+Source run:
+
+```bash
+go run ./cmd/server --config config.yaml
+```
+
+Binary build:
+
+```bash
+go build -o cyberstrike-ai ./cmd/server
+./cyberstrike-ai --config config.yaml
+```
+
+The binary still needs `web/templates`, `web/static`, and the runtime resource directories.
+
+## HTTPS and Reverse Proxy
+
+For local testing, self-signed HTTPS is acceptable:
+
+```yaml
+server:
+  tls_enabled: true
+  tls_auto_self_sign: true
+```
+
+For production, use real certificates or terminate TLS at a reverse proxy. If the proxy terminates TLS and forwards HTTP to the app, avoid enabling app-side TLS on the same upstream unless `proxy_pass` uses HTTPS.
+
+Nginx must not buffer SSE:
+
+```nginx
+proxy_buffering off;
+proxy_http_version 1.1;
+proxy_set_header Upgrade $http_upgrade;
+proxy_set_header Connection "upgrade";
+```
+
+## Deployment Decision Table
+
+| Scenario | Recommended setup | Key settings | Avoid |
+| --- | --- | --- | --- |
+| Personal testing | `./run.sh` + self-signed HTTPS | `tls_auto_self_sign: true` | Public exposure |
+| Internal team | Binary + systemd + internal HTTPS | strong password, audit, backup, IP restrictions | Shared weak password |
+| Production red-team platform | Reverse proxy + dedicated OS user + log collection | real certs, proxy auth, C2 only when needed | Direct public admin UI |
+| Chat/KB only | Disable C2 and unnecessary MCP | `c2.enabled: false` | All tools enabled by default |
+| Tool automation | Isolated workspace + HITL | `workspace_root_dir`, `hitl`, `monitor` | Shell tools globally allowlisted |
+
+## Acceptance Checklist
+
+After startup:
+
+1. Open `/` and verify no HTTP/HTTPS redirect loop.
+2. Login and validate `/api/auth/validate`.
+3. Run model test in settings.
+4. Check tool list and schemas.
+5. If KB is enabled, check index status.
+6. If external MCP is enabled, verify connection and tool visibility.
+7. If C2 is enabled, start and stop a test listener only in an authorized network.
+8. Check audit logs for login and config activity.
+
+## Runtime File Layers
+
+- Replaceable: binary, `web/`, default docs/resources.
+- Preserve: `config.yaml`, `data/`, custom tools/roles/skills/agents, `knowledge_base`, uploads.
+- Cleanup candidates: checkpoints, temporary workspaces, stale payloads, old tool execution records.
+
+## Source Anchors
+
+- App wiring and routes: `internal/app/app.go`
+- TLS bootstrap: `internal/app/main_server_tls.go`
+- HTTP to HTTPS redirect: `internal/app/main_server_http_redirect.go`
+- Config structs: `internal/config/config.go`
+- Config apply: `internal/handler/config.go`
@@ -0,0 +1,111 @@
+# Developer Guide
+
+[中文](../zh-CN/developer-guide.md)
+
+This guide is for contributors extending CyberStrikeAI. The project is a Go single-service application with a static frontend, SQLite persistence, Agent/MCP orchestration, and optional high-risk security subsystems.
+
+## Project Layout
+
+```text
+cmd/server/              service entrypoint
+internal/app/            app wiring, routes, MCP tool registration
+internal/handler/        HTTP handlers
+internal/database/       SQLite access
+internal/security/       auth, rate limits, shell execution
+internal/mcp/            MCP server and external MCP manager
+internal/multiagent/     Eino single-agent, multi-agent, middleware
+internal/workflow/       graph orchestration runtime
+internal/knowledge/      indexing and retrieval
+internal/c2/             built-in C2
+internal/project/        project fact blackboard
+web/static/              frontend JS/CSS/assets
+web/templates/           HTML templates
+tools/                   YAML command tools
+roles/                   role YAML
+agents/                  multi-agent Markdown definitions
+skills/                  Agent Skills
+docs/                    documentation
+```
+
+## Development Startup
+
+```bash
+go run ./cmd/server --config config.yaml
+```
+
+The frontend is static. Most JS/CSS/template changes only require a browser refresh.
+
+## Adding a Business Module
+
+Do not add only a handler. A complete module usually needs:
+
+1. Data model and SQLite migration.
+2. Handler: parameters, errors, pagination/filtering.
+3. Audit: management actions.
+4. Monitor: long-running execution state.
+5. MCP: whether Agents should call it.
+6. HITL: approval boundary for MCP tools.
+7. OpenAPI: update `/api/openapi/spec`.
+8. Frontend: i18n, states, empty/error UI.
+9. Tests: DB, handler, edge cases.
+10. Docs: config, usage, troubleshooting, safety impact.
+
+Missing one of these usually becomes a later usability or safety bug.
+
+## Error Response Design
+
+Prefer stable JSON:
+
+```json
+{
+  "error": "machine_readable_code",
+  "message": "human-readable explanation"
+}
+```
+
+Frontend needs stable fields, users need actionable messages, and logs need detailed internal errors.
+
+## Long-Running Tasks
+
+For scanning, indexing, batch tasks, C2, or external operations, answer:
+
+- Can it be cancelled?
+- Can progress be queried?
+- Can it be retried?
+- Where is the result stored?
+- Does state survive page refresh?
+- Does it block the HTTP request?
+
+If not, use task tables, event streams, or monitoring.
+
+## Extending Tools
+
+Prefer `tools/*.yaml` for command tools. Use Go built-in tools when the tool needs internal state or structured integration.
+
+Built-in tools should define clear input schemas, handle timeouts and errors, and respect HITL for risky actions.
+
+## Frontend Changes
+
+Use existing helpers such as `apiFetch`, modal utilities, notifications, and i18n. Update both `web/static/i18n/zh-CN.json` and `web/static/i18n/en-US.json` for new visible text.
+
+Avoid putting secrets or provider keys in frontend code.
+
+## Test Priority
+
+High-value tests:
+
+- config hot-apply;
+- HITL branches;
+- shell timeout/no-output;
+- external MCP recovery;
+- KB indexing and post-processing;
+- WebShell OS/encoding detection;
+- SQLite migration compatibility.
+
+## Source Anchors
+
+- App wiring: `internal/app/app.go`
+- Config apply: `internal/handler/config.go`
+- OpenAPI: `internal/handler/openapi.go`
+- Tool executor: `internal/security/executor.go`
+- Skill package: `internal/skillpackage/`
@@ -0,0 +1,54 @@
+# Frontend i18n
+
+[中文](../zh-CN/frontend-i18n.md)
+
+CyberStrikeAI frontend i18n is static and lightweight. Text is organized in JSON files and applied through `data-i18n` attributes plus JavaScript helper functions.
+
+## Files
+
+```text
+web/static/i18n/zh-CN.json
+web/static/i18n/en-US.json
+web/static/js/i18n.js
+```
+
+## Key Principles
+
+- Keep keys stable and semantic.
+- Update Chinese and English together.
+- Do not hardcode new visible text in JS when it should be localized.
+- Preserve default HTML text as fallback before JS initialization.
+
+## HTML Usage
+
+```html
+<button data-i18n="common.save">保存</button>
+```
+
+For attributes, follow the existing `i18n.js` conventions.
+
+## JavaScript Usage
+
+Use the global translation helper where available:
+
+```javascript
+const label = t('common.save');
+```
+
+When adding dynamic UI, make sure language switching refreshes the text or re-renders the component.
+
+## Migration Workflow
+
+1. Add or update UI text.
+2. Add keys to `zh-CN.json`.
+3. Add matching keys to `en-US.json`.
+4. Replace hardcoded text with `data-i18n` or `t()`.
+5. Test both languages and browser console.
+
+## Common Pitfalls
+
+- Missing keys only in one language.
+- Dynamic text built from hardcoded fragments.
+- Button labels too long in English.
+- HTML fallback text diverges from JSON text.
+- Adding new page text without updating language switch behavior.
@@ -0,0 +1,122 @@
+# Human-in-the-loop (HITL) Best Practices
+
+[中文](../zh-CN/hitl-best-practices.md)
+
+HITL reviews tool calls before an Agent executes them. Use it to control high-risk operations, keep an audit trail, and let an Audit Agent take over routine approvals when human reviewers cannot keep up.
+
+## Where To Configure
+
+Open **System Settings → Human-in-the-loop** in the web UI. You can configure:
+
+- Global default reviewer: `human` or `audit_agent`
+- Dedicated Audit Agent model: `hitl.audit_model`
+- Resolved audit log retention days
+- No-approval tool allowlist: `hitl.tool_whitelist`
+- Audit prompts for approval mode and review-edit mode
+
+Example `config.yaml`:
+
+```yaml
+hitl:
+  default_reviewer: human
+  audit_model:
+    provider: ""
+    base_url: ""
+    api_key: ""
+    model: "" # set a small model here; blank reuses openai.model
+  retention_days: 90
+  tool_whitelist: [read_file, list_dir, glob, grep, tool_search]
+```
+
+`audit_model` supports partial configuration. Empty fields inherit from the main `openai` config, so the common setup is to fill only `model` and run approvals on a cheaper small model.
+
+## Recommended Approval Strategy
+
+### 1. Start With Humans, Then Delegate Gradually
+
+At the beginning, prefer:
+
+- `default_reviewer: human`
+- Only clearly read-only tools in `tool_whitelist`
+- Human approval for file writes, command execution, C2 tasks, and WebShell operations
+
+After observing audit logs, move repeated low-risk operations into the allowlist.
+
+### 2. Use A Small Model When Humans Cannot Keep Up
+
+When pending approvals start piling up, switch routine review to the Audit Agent:
+
+```yaml
+hitl:
+  default_reviewer: audit_agent
+  audit_model:
+    model: "your-small-reviewer-model"
+```
+
+Good candidates for small-model review:
+
+- Read-only queries
+- Reconnaissance
+- Port and service scans
+- Directory enumeration
+- Non-destructive validation commands
+
+Keep human review for:
+
+- Deleting, overwriting, or clearing data
+- Modifying permissions, passwords, or accounts
+- Persistence, lateral movement, and high-risk C2 tasks
+- Writes against production targets
+
+### 3. Encode Your Policy In The Prompt
+
+The Audit Agent prompt should describe an operational policy, not just say “be careful.” Make it explicit:
+
+- Which low-risk actions are normally approved
+- Which destructive actions must be rejected
+- Which cases require escalation to a human
+- How review-edit mode may narrow arguments
+
+Example policy snippet:
+
+```text
+Approve routine reconnaissance, read-only queries, and port scans by default.
+Reject file deletion, database clearing, account or permission changes, persistence, and stopping critical services.
+Reject actions outside the user-authorized target scope.
+In review-edit mode, you may narrow paths, targets, or command arguments before approving, but must not expand the attack surface.
+```
+
+### 4. Keep The Allowlist Conservative
+
+Allowlisted tools skip approval, so keep the list stable and low-risk. Recommended examples:
+
+- `read_file`
+- `list_dir`
+- `glob`
+- `grep`
+- `tool_search`
+
+Avoid globally allowlisting:
+
+- Arbitrary shell execution tools
+- File write/delete tools
+- C2 task tools
+- WebShell command execution tools
+
+## Mode Selection
+
+| Mode | Best for |
+|------|----------|
+| Off | Local labs or fully trusted toolchains |
+| Approval | Approve/reject only |
+| Review-edit | Let the Audit Agent narrow arguments before approval |
+
+If you configured a small audit model, start with **Approval** mode. Use **Review-edit** only when you want the AI to safely narrow paths, target ranges, or command arguments.
+
+## Operations Tips
+
+- Review **Human-in-the-loop → Audit logs** regularly and tune allowlists/prompts.
+- In high-risk environments, keep `default_reviewer: human` and use the Audit Agent only for recommendations.
+- If the small-model reviewer fails, CyberStrikeAI rejects conservatively by default.
+- After changing `hitl.audit_model`, click **Test audit model** in the settings page.
+- For production, customer, or real business systems, keep a human as the final approver.
@@ -0,0 +1,107 @@
+# Knowledge Base
+
+[中文](../zh-CN/knowledge-base.md)
+
+The knowledge base turns local security notes, playbooks, vulnerability guides, and organizational standards into retrievable context for Agents.
+
+## Enable
+
+```yaml
+knowledge:
+  enabled: true
+  base_path: knowledge_base
+  embedding:
+    provider: openai
+    model: text-embedding-v4
+database:
+  knowledge_db_path: data/knowledge.db
+```
+
+Keep the knowledge DB separate when you want portable reusable indexes.
+
+## Internal Pipeline
+
+```mermaid
+flowchart LR
+    F["Markdown / Web item"] --> M["Manager"]
+    M --> C["Chunker"]
+    C --> E["Embedding"]
+    E --> V["SQLite Vector Index"]
+    Q["Agent query"] --> MQ["MultiQuery"]
+    MQ --> V
+    V --> R["Rerank"]
+    R --> P["Post-process"]
+    P --> A["Agent context"]
+```
+
+Quality depends on source structure, chunk size, embedding quality, and rerank behavior.
+
+## Content Writing
+
+Bad:
+
+```text
+SQL injection is dangerous. Use sqlmap. Filter input.
+```
+
+Better:
+
+```markdown
+# MySQL UNION Injection Verification
+
+## Preconditions
+- Parameter is concatenated into SELECT.
+
+## Steps
+1. Use `order by` to infer column count.
+2. Use `union select null,...` to find reflection.
+3. Use read-only functions to confirm DB type.
+
+## False Positives
+- WAF error page.
+- Generic error page.
+
+## Fix
+- Parameterized queries.
+- Least DB privilege.
+```
+
+Structured headings and concrete steps improve chunking and retrieval.
+
+## Tuning
+
+Use a fixed test query set, then change one variable at a time:
+
+- empty results: lower `similarity_threshold`, verify indexing;
+- wrong topic: improve titles and category/risk type;
+- broken context: tune `chunk_size` and `chunk_overlap`;
+- noisy results: raise threshold or fix rerank;
+- high cost: lower `multi_query.max_queries`, `prefetch_top_k`, or `top_k`.
+
+## MCP Tools
+
+Enabled KB registers tools such as:
+
+- list risk types;
+- search knowledge base.
+
+Prompt roles to query the KB before giving vulnerability validation or remediation advice when unsure.
+
+## Retrieval Logs
+
+Use logs to improve content:
+
+- frequent no-results queries: missing content or synonyms;
+- low scores: titles/terms mismatch;
+- duplicate hits: merge or categorize docs;
+- Agent ignores results: output may be too long or not actionable.
+
+## Source Anchors
+
+- Manager: `internal/knowledge/manager.go`
+- Index pipeline: `internal/knowledge/index_pipeline.go`
+- Chunking: `internal/knowledge/chunk_eino.go`
+- Retriever: `internal/knowledge/retriever.go`
+- Eino chain: `internal/knowledge/eino_retrieve_chain.go`
+- Rerank: `internal/knowledge/rerank_http.go`
+- MCP tools: `internal/knowledge/tool.go`
@@ -0,0 +1,86 @@
+# MCP Federation
+
+[中文](../zh-CN/mcp-federation.md)
+
+CyberStrikeAI uses MCP as the primary tool protocol. Tools can be built-in, YAML-backed, Skill-local, or provided by external MCP servers.
+
+## Built-In MCP
+
+The internal MCP server registers:
+
+- YAML command tools;
+- security execution tools;
+- knowledge tools;
+- project fact tools;
+- C2 tools;
+- WebShell tools;
+- batch task tools;
+- vision analysis.
+
+Agents usually call these internally without extra setup.
+
+## HTTP MCP
+
+```yaml
+mcp:
+  enabled: true
+  host: 0.0.0.0
+  port: 8081
+  auth_header: "X-MCP-Token"
+  auth_header_value: "random-secret"
+```
+
+Always set an auth value and restrict network access.
+
+## External MCP Lifecycle
+
+1. Register config: name, type, command/URL, environment.
+2. Start connection: stdio process or HTTP/SSE client.
+3. Pull tool list: names, descriptions, schemas.
+4. Expose to Agent: affected by role, tool_search, HITL.
+5. Execute: validate args, call, monitor.
+6. Recover: handle process/network failure.
+7. Stop/delete: remove runtime and config.
+
+Debug by locating the failed step.
+
+## Tool Naming
+
+Good names are stable, specific, and action-object oriented:
+
+```text
+burp_send_to_repeater
+asset_lookup_domain
+cloud_list_public_buckets
+```
+
+Avoid:
+
+```text
+run
+execute
+scan
+tool1
+```
+
+Specific names improve tool_search and reduce misuse.
+
+## Security Review
+
+Before connecting an external MCP, ask:
+
+- Can it read/write local files?
+- Can it execute commands?
+- What network does it access?
+- Does it send data to third parties?
+- Are tool descriptions trustworthy?
+- Can output contain prompt injection?
+- Should it run under a separate OS user or container?
+
+## Source Anchors
+
+- External manager: `internal/mcp/external_manager.go`
+- Recovery: `internal/mcp/connection_recovery.go`
+- Tool adapter: `internal/einomcp/mcp_tools.go`
+- Handler: `internal/handler/external_mcp.go`
+- Invoke notification: `internal/einomcp/tool_invoke_notify.go`
@@ -0,0 +1,62 @@
+# Plugin Development
+
+[中文](../zh-CN/plugin-development.md)
+
+Plugins can integrate with CyberStrikeAI through HTTP APIs, MCP servers, or resource packs such as tools, roles, Skills, and agents.
+
+## Plugin Layers
+
+| Layer | Example | Benefit | Cost |
+| --- | --- | --- | --- |
+| API plugin | Burp extension calls `/api/eino-agent` | simple UI integration | depends on API/auth |
+| MCP plugin | exposes tools to Agent | Agent can call it | needs schema and safety design |
+| Resource pack | ships tools/roles/skills/agents | simple and versionable | less interactive |
+
+Do not start with MCP unless the Agent must actively call your capability.
+
+## API Plugin Payload
+
+Include:
+
+- source tool and context;
+- target URL, method, key headers;
+- truncation policy for request/response bodies;
+- user intent;
+- authorization boundary.
+
+Large responses should be uploaded or summarized, not pasted whole into the prompt.
+
+## MCP Schema Design
+
+Bad:
+
+```json
+{"cmd":{"type":"string"}}
+```
+
+Better:
+
+```json
+{
+  "target_url": {"type":"string","description":"authorized target URL"},
+  "scan_profile": {"type":"string","enum":["passive","active-safe"]},
+  "max_requests": {"type":"integer","description":"request limit"}
+}
+```
+
+Specific schemas make HITL and Agent behavior safer.
+
+## Security Boundaries
+
+Plugins should not bypass platform controls:
+
+- no hidden destructive local commands;
+- no plaintext long-lived credentials;
+- no default third-party data exfiltration;
+- no dependency on browser state to bypass login.
+
+## Source Anchors
+
+- Burp plugin: `plugins/burp-suite/cyberstrikeai-burp-extension/src/main/java/burp/`
+- OpenAPI: `internal/handler/openapi.go`
+- External MCP: `internal/handler/external_mcp.go`
@@ -0,0 +1,67 @@
+# Release Process
+
+[中文](../zh-CN/release-process.md)
+
+Use this guide for maintainers and operators preparing upgrades or releases.
+
+## Pre-Release Checklist
+
+- README and docs updated.
+- `config.yaml` sample includes new fields.
+- OpenAPI includes new endpoints.
+- i18n updated when frontend text changed.
+- Security docs updated for high-risk capabilities.
+
+## Release Risk Tiers
+
+| Change | Risk | Must test |
+| --- | --- | --- |
+| Docs/assets | low | links/rendering |
+| Frontend | medium | login, page states, API errors |
+| Handler/API | medium | OpenAPI, auth, errors |
+| Config struct | high | old config compatibility, ApplyConfig |
+| DB schema | high | old DB migration, rollback |
+| Agent/MCP/HITL | high | tools, approvals, streaming |
+| C2/WebShell/Terminal | critical | authorized lab, audit, disable switch |
+
+Release notes should call out risk, not just features.
+
+## Config Compatibility
+
+New fields should:
+
+- have safe defaults;
+- allow old configs to start;
+- be documented in sample `config.yaml`;
+- not cause Web settings to delete unknown fields;
+- be tested via restart and hot-apply paths.
+
+Avoid default-enabling high-risk capabilities.
+
+## Database Changes
+
+SQLite migrations must be:
+
+- compatible with old versions;
+- idempotent after interruption;
+- careful with nullable/default fields;
+- mindful of large indexes and locks;
+- documented with backup instructions.
+
+## Build and Test
+
+```bash
+go test ./internal/...
+go test ./cmd/...
+go build -o cyberstrike-ai ./cmd/server
+```
+
+Manual smoke:
+
+```text
+login -> model test -> new chat -> tools -> HITL -> KB -> external MCP -> C2 enable/disable
+```
+
+## Rollback
+
+Restore binary/code, `config.yaml`, and `data/` together. If a new version changed DB schema, replacing only the binary is not a reliable rollback.
@@ -0,0 +1,463 @@
+# CyberStrikeAI Robot / Chatbot Guide
+
+[中文](../zh-CN/robot.md)
+
+This document explains how to chat with CyberStrikeAI from **personal WeChat**, **DingTalk**, **Lark (Feishu)**, and **WeCom (Enterprise WeChat)** using long-lived connections or HTTP callbacks—no need to open a browser on the server. Following the steps below helps avoid common mistakes.
+
+---
+
+## 1. Where to configure in CyberStrikeAI
+
+1. Log in to the CyberStrikeAI web UI.
+2. Open **System Settings** in the left sidebar.
+3. Click **Robot settings** (between “Basic” and “Security”).
+4. Configure per platform:
+   - **Personal WeChat**: Open **WeChat / iLink** → **Generate QR code and bind**, then scan with WeChat (see [Section 3.4](#34-personal-wechat-wechat--ilink))
+   - **DingTalk**: Enable and fill in Client ID / Client Secret
+   - **Lark**: Enable and fill in App ID / App Secret
+5. Click **Apply configuration** to save (WeChat binding saves and enables automatically on success—usually no extra click needed)
+6. **Restart the CyberStrikeAI process** (DingTalk/Lark: saving alone does not establish the connection; WeChat auto-restarts the iLink poll after binding—usually no manual restart needed)
+
+Settings are written to the `robots` section of `config.yaml`; you can also edit the file directly. **After changing DingTalk or Lark config, you must restart for the long-lived connection to take effect.** Personal WeChat binding automatically writes `robots.wechat` and restarts the iLink long poll.
+
+---
+
+## 2. Supported platforms (long-lived / callback)
+
+| Platform       | Description |
+|----------------|-------------|
+| Personal WeChat| WeChat iLink protocol; scan QR in the web UI to bind, then long-poll for messages—**no public callback URL needed** |
+| DingTalk       | Stream long-lived connection; the app connects to DingTalk to receive messages |
+| Lark (Feishu)  | Long-lived connection; the app connects to Lark to receive messages |
+| WeCom (Qiye WX)| HTTP callback to receive messages; CyberStrikeAI replies via WeCom’s message sending API |
+| Telegram       | Bot API long polling (`getUpdates`); **no public callback URL needed** |
+| Slack          | Socket Mode (outbound WebSocket); **no public callback URL needed** |
+| Discord        | Gateway WebSocket; **no public callback URL needed** |
+| QQ Bot         | QQ Open Platform WebSocket (C2C / group @); **no public callback URL needed** |
+
+Section 3 below describes, per platform, what to do in the developer console and which fields to copy into CyberStrikeAI.
+
+---
+
+## 3. Configuration and step-by-step setup
+
+### 3.1 DingTalk
+
+**Important: two types of DingTalk bots**
+
+| Type | Where it’s created | Can do “user sends message → bot replies”? | Supported here? |
+|------|-------------------|-------------------------------------------|------------------|
+| **Custom bot (Webhook)** | In a DingTalk group: Group settings → Add robot → Custom (Webhook) | No; you can only post to the group | No |
+| **Enterprise internal app bot** | [DingTalk Open Platform](https://open.dingtalk.com): create an app and enable the bot | Yes | Yes |
+
+If you only have a **custom bot** Webhook URL (`oapi.dingtalk.com/robot/send?access_token=...`) and sign secret (`SEC...`), **do not** put them into CyberStrikeAI. You must create an **enterprise internal app** in the open platform and obtain **Client ID** and **Client Secret** as below.
+
+---
+
+**DingTalk setup (in order)**
+
+1. **Open DingTalk Open Platform**  
+   Go to [https://open.dingtalk.com](https://open.dingtalk.com) and log in with an **enterprise admin** account.
+
+2. **Create or select an app**  
+   In the left menu: **Application development** → **Enterprise internal development** → **Create application** (or choose an existing app). Fill in the app name and create.
+
+3. **Get Client ID and Client Secret**  
+   - In the left menu open **Credentials and basic info** (under “Basic information”).  
+   - Copy **Client ID (formerly AppKey)** and **Client Secret (formerly AppSecret)**.  
+   - Use copy/paste; avoid typing by hand. Watch for **0** vs **o** and **1** vs **l** (e.g. `ding9gf9tiozuc504aer` has the digits **504**, not 5o4).
+
+4. **Enable the bot and choose Stream mode**  
+   - Left menu: **Application capabilities** → **Robot**.  
+   - Turn on “Robot configuration”.  
+   - Fill in robot name, description, etc. as required.  
+   - **Critical**: set message reception to **“Stream mode”** (流式接入). If you only enable “HTTP callback” or do not select Stream, CyberStrikeAI will not receive messages.  
+   - Save.
+
+5. **Permissions and release**  
+   - Left menu: **Permission management** — search for “robot”, “message”, etc., and enable **receive message**, **send message**, and other bot-related permissions; confirm.  
+   - Left menu: **Version management and release** — if there are unpublished changes, click **Release new version** / **Publish**; otherwise changes do not take effect.
+
+6. **Fill in CyberStrikeAI**  
+   - In CyberStrikeAI: System settings → Robot settings → DingTalk.  
+   - Enable “Enable DingTalk robot”.  
+   - Paste the Client ID and Client Secret from step 3.  
+   - Click **Apply configuration**, then **restart CyberStrikeAI**.
+
+---
+
+**Field mapping (DingTalk)**
+
+| Field in CyberStrikeAI | Source in DingTalk Open Platform |
+|------------------------|----------------------------------|
+| Enable DingTalk robot | Check to enable |
+| Client ID (AppKey) | Credentials and basic info → **Client ID (formerly AppKey)** |
+| Client Secret | Credentials and basic info → **Client Secret (formerly AppSecret)** |
+
+---
+
+### 3.2 Lark (Feishu)
+
+| Field | Description |
+|-------|-------------|
+| Enable Lark robot | Check to start the Lark long-lived connection |
+| App ID | From Lark open platform app credentials |
+| App Secret | From Lark open platform app credentials |
+| Verify Token | Optional; for event subscription |
+
+**Lark setup in short**: Log in to [Lark Open Platform](https://open.feishu.cn) → Create an enterprise app → In “Credentials and basic info” get **App ID** and **App Secret** → In “Application capabilities” enable **Robot** and the right permissions → Add **event subscription** and **permissions** below → Publish the app → Enter App ID and App Secret in CyberStrikeAI robot settings → Save and **restart** the app.
+
+**Event subscription**  
+The long-lived connection only receives message events if you subscribe to them. In the app’s **Events and callbacks** (事件与回调) → **Event subscription** (事件订阅), add the event **Receive message** (**im.message.receive_v1**). Without it, the connection succeeds but no message events are delivered (no logs when users send messages).
+
+**Lark permissions (required)**  
+In **Permission management** (权限管理), enable the following (names and identifiers match the Lark console). After changes, **publish a new version** in Version management and release so they take effect.
+
+| Permission name (as shown in console) | Identifier | Notes |
+|--------------------------------------|------------|-------|
+| 获取与发送单聊、群组消息 (Get and send direct & group messages) | `im:message` | Base permission for sending and receiving; **required**. |
+| 接收群聊中@机器人消息事件 (Receive @bot messages in group chat) | `im:message.group_at_msg:readonly` | Required for group chat when users @ the bot. |
+| 读取用户发给机器人的单聊消息 (Read direct messages from users to bot) | `im:message.p2p_msg:readonly` | **Required** for 1:1 chat; otherwise no response in private chat. |
+| 获取单聊、群组消息 (Get direct & group messages) | `im:message:readonly` | **Required** to read message content. |
+
+**Event subscription** (configured separately): In **Event subscription** (事件订阅), add **Receive message** (**im.message.receive_v1**). Without it, the long-lived connection will not receive message events.
+
+- **1:1 chat**: Open the bot’s private chat in Lark and send e.g. “帮助” or “help”; no @ needed.  
+- **Group chat**: Only messages that **@ the bot** are received and replied to.
+
+---
+
+### 3.3 WeCom (Enterprise WeChat)
+
+> WeCom uses a **“HTTP callback + active message send API”** model:  
+> - User sends a message → WeCom sends an **encrypted XML callback** to your server (CyberStrikeAI’s `/api/robot/wecom`).  
+> - CyberStrikeAI decrypts it, calls the AI, then uses WeCom’s `message/send` API to **actively push the reply** to the user.
+
+**Configuration overview:**
+
+- In the WeCom admin console, create or select a **custom app** (自建应用).
+- In that app’s settings, configure the message **callback URL**, **Token**, and **EncodingAESKey**.
+- In CyberStrikeAI’s `config.yaml`, fill in:
+  - `robots.wecom.corp_id`: your CorpID (企业 ID)
+  - `robots.wecom.agent_id`: the app’s AgentId
+  - `robots.wecom.token`: the Token used for message callbacks
+  - `robots.wecom.encoding_aes_key`: the EncodingAESKey used for callbacks
+  - `robots.wecom.secret`: the app’s Secret (used when calling WeCom APIs to send messages)
+
+> **Important: IP allowlist (errcode 60020)**  
+> CyberStrikeAI calls `https://qyapi.weixin.qq.com/cgi-bin/message/send` to actively send AI replies.  
+> If logs show `errcode 60020 not allow to access from your ip`:
+>
+> - Your server’s outbound IP is **not in WeCom’s IP allowlist**.  
+> - In the WeCom admin console, open the custom app’s **Security / IP allowlist** settings (name may vary slightly), and add the public IP of the machine running CyberStrikeAI (e.g. `110.xxx.xxx.xxx`).  
+> - Save and wait for it to take effect, then test again.
+>
+> If the IP is not whitelisted, WeCom will reject active message sending. You will see that `/api/robot/wecom` receives and processes callbacks, but users **never see AI replies**, and logs contain `not allow to access from your ip`.
+
+---
+
+### 3.4 Personal WeChat (WeChat / iLink)
+
+> Personal WeChat uses **“web QR binding + iLink long polling”**:  
+> - Generate a QR code in the CyberStrikeAI web UI → scan and confirm with **WeChat on your phone**;  
+> - On success, `robots.wechat` in `config.yaml` is updated automatically and iLink long polling starts (the app connects outbound to `ilinkai.weixin.qq.com`);  
+> - **No** public callback URL on your server and **no** WeChat Open Platform app registration required.
+
+**Personal WeChat vs WeCom**
+
+| Item | Personal WeChat (iLink) | WeCom (Enterprise WeChat) |
+|------|-------------------------|---------------------------|
+| Use case | Private chat in personal WeChat | Custom app in WeCom |
+| Setup | QR scan in web UI | Admin console callback URL + Token |
+| Public IP needed? | No (outbound long poll only) | Yes (HTTPS callback reachable by WeCom) |
+| Config key | `robots.wechat` | `robots.wecom` |
+
+**Binding steps (in order)**
+
+1. **Log in to CyberStrikeAI web UI**  
+   **System settings** → **Robot settings** → click the **WeChat / iLink** card.
+
+2. **(Optional) Enable “Enable WeChat robot”**  
+   You can skip this on first bind; it is checked automatically after a successful bind.
+
+3. **Generate QR code**  
+   Click **“Generate QR code and bind”**. The QR code is valid for about **5 minutes**; regenerate if it expires.
+
+4. **Scan and confirm in WeChat**  
+   - Scan the QR code with WeChat on your phone;  
+   - Complete confirmation on the phone;  
+   - If WeChat shows a **pairing code**, enter it on the web page and click **Submit** (only some accounts need this).
+
+5. **Wait for binding to complete**  
+   When the page shows “Binding successful, WeChat robot enabled”, you’re done. `bot_token`, `ilink_bot_id`, etc. are saved to `config.yaml` and the iLink poll restarts automatically—**usually no manual service restart**.
+
+6. **Test in WeChat**  
+   Open the **private chat** with the CyberStrikeAI bot in WeChat and send “帮助” (help) or any text.
+
+**Field reference (WeChat)**
+
+| Field | Description |
+|-------|-------------|
+| Enable WeChat robot | Starts iLink long polling when checked; auto-enabled after bind |
+| Generate QR code and bind | Starts the scan-to-bind flow |
+| **Advanced** (defaults are fine) | |
+| API Base URL | Default `https://ilinkai.weixin.qq.com` |
+| Bot Type | Default `3` |
+| Bot Agent | Default `CyberStrikeAI/1.0` |
+| iLink Bot ID | Filled automatically after bind (read-only) |
+
+**How to use**
+
+- **Private chat only**—send text directly; **no @ needed**.  
+- Group @-bot is **not** supported (unlike DingTalk/Lark groups).  
+- **Text messages only**; images, voice, etc. are ignored or not supported.
+
+**Re-bind**
+
+- To bind a different WeChat account, click **“Re-bind”** on the robot settings page and scan again.  
+- If you see “This WeChat account is already bound”, that account was bound before.
+
+**Common issues**
+
+| Symptom | What to do |
+|---------|------------|
+| QR code expired | Click “Generate QR code and bind” again (~5 min TTL) |
+| Phone asks for a pairing code | Enter the digits shown in WeChat on the web page |
+| Bound but no replies | Check logs for `微信 iLink 长轮询已启动` and `微信收到消息`; ensure “Enable WeChat robot” is on |
+| No reply after sleep / network drop | Auto-reconnect in ~5–60 s; restart CyberStrikeAI if still stuck |
+| Cannot generate QR code | Ensure outbound HTTPS to `https://ilinkai.weixin.qq.com` |
+
+---
+
+### 3.5 Telegram
+
+> Telegram uses **Bot API long polling** (`getUpdates`): the app connects outbound to `api.telegram.org`—**no public callback URL needed**.
+
+1. Create a bot via **@BotFather** (`/newbot`) and copy the **Bot Token**.  
+2. CyberStrikeAI → **System settings** → **Robot settings** → **Telegram**.  
+3. Enable, paste the token, optionally allow group @ mentions → **Apply configuration**.
+
+---
+
+### 3.6 Slack
+
+> Slack uses **Socket Mode** (outbound WebSocket): requires **Bot Token (xoxb-)** and **App-Level Token (xapp-)** with `connections:write`.
+
+1. Create an app at [api.slack.com](https://api.slack.com/apps) → enable **Socket Mode**.  
+2. Create an App-Level Token; install the app to get a Bot Token.  
+3. Subscribe to `message.im` and `app_mention` events.  
+4. Paste both tokens in CyberStrikeAI → **Apply configuration**.
+
+---
+
+### 3.7 Discord
+
+> Discord uses **Gateway WebSocket**—**no public callback URL needed**.
+
+1. [Discord Developer Portal](https://discord.com/developers/applications) → create app → **Bot** → copy **Token**.  
+2. Enable **Message Content Intent** under Privileged Gateway Intents.  
+3. Invite the bot with `Send Messages` permission.  
+4. Paste token in CyberStrikeAI; optionally allow guild @ mentions → **Apply configuration**.
+
+---
+
+### 3.8 QQ Bot
+
+> QQ Bot uses **QQ Open Platform WebSocket** (official `botgo` SDK) for C2C and group @—**no public callback URL needed**.
+
+1. Create a bot at [q.qq.com](https://q.qq.com) → get **App ID** and **Client Secret**.  
+2. Add sandbox testers before going live.  
+3. Subscribe to C2C and group @ events (WebSocket).  
+4. Fill in CyberStrikeAI; use **Sandbox** for testing → **Apply configuration**.
+
+---
+
+## 4. Bot commands
+
+Send these **text commands** to the bot on any connected platform (text only):
+
+| Command | Description |
+|---------|-------------|
+| **帮助** (help) | Show command help |
+| **列表** or **对话列表** (list) | List all conversation titles and IDs |
+| **切换 \<conversationID\>** or **继续 \<conversationID\>** | Continue in the given conversation |
+| **新对话** (new) | Start a new conversation |
+| **清空** (clear) | Clear current context (same effect as new conversation) |
+| **当前** (current) | Show current conversation ID and title |
+| **停止** (stop) | Abort the currently running task |
+| **角色** or **角色列表** (roles) | List all available roles (penetration testing, CTF, Web scan, etc.) |
+| **角色 \<roleName\>** or **切换角色 \<roleName\>** | Switch to the specified role |
+| **删除 \<conversationID\>** | Delete the specified conversation |
+| **版本** (version) | Show current CyberStrikeAI version |
+
+Any other text is sent to the AI as a user message, same as in the web UI (e.g. penetration testing, security analysis).
+
+---
+
+## 5. How to use (do I need to @ the bot?)
+
+- **Personal WeChat**: Send directly in the **private chat** with the bot; **no @ needed** (group chat not supported).  
+- **DingTalk / Lark direct chat (recommended)**: **Search for the bot and open a direct chat**. Type “帮助” or any message; **no @ needed**.  
+- **DingTalk / Lark group chat**: If the bot is in a group, only messages that **@ the bot** are received and answered; other group messages are ignored.
+
+Summary: **Personal WeChat and direct chat**—just send; **DingTalk/Lark in a group**—@ the bot first, then send.
+
+---
+
+## 6. Recommended flow (so you don’t skip steps)
+
+**Personal WeChat (simplest—no open platform)**
+
+1. CyberStrikeAI web UI → System settings → Robot settings → **WeChat / iLink** → **Generate QR code and bind**.  
+2. Scan with WeChat and confirm (enter pairing code on the web page if prompted).  
+3. After binding, send “帮助” in the WeChat private chat to test.
+
+**DingTalk / Lark**
+
+1. **In the open platform**: Complete app creation, copy credentials, enable the bot (DingTalk: **Stream mode**), set permissions, and publish (Section 3).  
+2. **In CyberStrikeAI**: System settings → Robot settings → Enable the platform, paste Client ID/App ID and Client Secret/App Secret → **Apply configuration**.  
+3. **Restart the CyberStrikeAI process** (otherwise the long-lived connection is not established).  
+4. **On your phone**: Open DingTalk or Lark, find the bot (direct chat or @ in a group), send “帮助” or any message to test.
+
+If the bot does not respond, see **Section 9 (troubleshooting)** and **Section 10 (common pitfalls)**.
+
+---
+
+## 7. Config file example
+
+Example `robots` section in `config.yaml`:
+
+```yaml
+robots:
+  wechat: # Personal WeChat iLink (auto-filled after QR bind; usually no manual edit)
+    enabled: true
+    bot_token: "your_bot_token@im.bot:..."
+    ilink_bot_id: "your_bot_id@im.bot"
+    ilink_user_id: "your_user_id@im.wechat"
+    base_url: "https://ilinkai.weixin.qq.com"
+    bot_type: "3"
+    bot_agent: "CyberStrikeAI/1.0"
+  dingtalk:
+    enabled: true
+    client_id: "your_dingtalk_app_key"
+    client_secret: "your_dingtalk_app_secret"
+  lark:
+    enabled: true
+    app_id: "your_lark_app_id"
+    app_secret: "your_lark_app_secret"
+    verify_token: ""
+  wecom:
+    enabled: false
+    corp_id: ""
+    agent_id: 0
+    token: ""
+    encoding_aes_key: ""
+    secret: ""
+  telegram:
+    enabled: false
+    bot_token: ""
+    allow_group_messages: false
+  slack:
+    enabled: false
+    bot_token: ""
+    app_token: ""
+  discord:
+    enabled: false
+    bot_token: ""
+    allow_guild_messages: false
+  qq:
+    enabled: false
+    app_id: ""
+    client_secret: ""
+    sandbox: true
+```
+
+After changing DingTalk/Lark/WeCom/Telegram/Slack/Discord/QQ settings, **Apply configuration** restarts the corresponding connections. Personal WeChat QR binding saves and restarts automatically.
+
+---
+
+## 8. Testing without DingTalk/Lark installed
+
+You can verify bot logic with the **test API** (no DingTalk/Lark client needed):
+
+1. Log in to the CyberStrikeAI web UI (so you have a session).
+2. Call the test endpoint with curl (include your session Cookie):
+
+```bash
+# Replace YOUR_COOKIE with the Cookie from your browser (F12 → Network → any request → Request headers → Cookie)
+curl -X POST "http://localhost:8080/api/robot/test" \
+  -H "Content-Type: application/json" \
+  -H "Cookie: YOUR_COOKIE" \
+  -d '{"platform":"dingtalk","user_id":"test_user","text":"帮助"}'
+```
+
+If the JSON response contains `"reply":"【CyberStrikeAI 机器人命令】..."`, command handling works. You can also try `"text":"列表"` or `"text":"当前"`.
+
+API: `POST /api/robot/test` (requires login). Body: `{"platform":"optional","user_id":"optional","text":"required"}`. Response: `{"reply":"..."}`.
+
+---
+
+## 9. Troubleshooting: no response when sending messages
+
+### 9.1 Personal WeChat
+
+Check in this order:
+
+1. **Binding completed?**  
+   Robot settings should show “Connected” or a bound Bot ID; `robots.wechat.bot_token` in `config.yaml` must not be empty.
+
+2. **Enabled?**  
+   Confirm “Enable WeChat robot” is checked; restart CyberStrikeAI if you just changed settings.
+
+3. **Application logs**  
+   - On startup: `微信 iLink 长轮询已启动`;  
+   - After sending a message: `微信收到消息`; if missing, binding may have failed or `bot_token` is invalid—try **Re-bind**.  
+   - `微信 iLink 长轮询异常，将自动重连`: wait for auto-reconnect or restart.
+
+4. **Network**  
+   The server must reach `https://ilinkai.weixin.qq.com` (outbound HTTPS). If QR generation fails, check this first.
+
+5. **After sleep or network drop**  
+   Same as DingTalk/Lark: **auto-reconnect** in ~5–60 s; restart if still no response.
+
+### 9.2 DingTalk
+
+Check in this order:
+
+0. **After laptop sleep or network drop**  
+   DingTalk and Lark both use long-lived connections; they break when the machine sleeps or the network drops. The app **auto-reconnects** (retries within about 5–60 seconds). After wake or network recovery, wait a moment before sending; if there is still no response, restart the CyberStrikeAI process.
+
+1. **Client ID / Client Secret match the open platform exactly**  
+   Copy from “Credentials and basic info”; avoid typing. Watch **0** vs **o** and **1** vs **l** (e.g. `ding9gf9tiozuc504aer` has **504**, not 5o4).
+
+2. **Did you restart after saving?**  
+   The long-lived connection is created at **startup**. “Apply configuration” only updates the config file; you **must restart the CyberStrikeAI process** for the DingTalk connection to start.
+
+3. **Application logs**  
+   - On startup you should see: `钉钉 Stream 正在连接…`, `钉钉 Stream 已启动（无需公网），等待收消息`.  
+   - If you see `钉钉 Stream 长连接退出` with an error, it’s usually wrong **Client ID / Client Secret** or **Stream not enabled** in the open platform.  
+   - After sending a message in DingTalk, you should see `钉钉收到消息` in the logs; if not, the platform is not pushing to this app (check that the bot is enabled and **Stream mode** is selected).
+
+4. **Open platform**  
+   The app must be **published**. Under “Robot” you must enable **Stream** for receiving messages (HTTP callback only is not enough). Permission management must include robot receive/send message permissions.
+
+---
+
+## 10. Common pitfalls
+
+- **Personal WeChat vs WeCom**: Personal WeChat uses `robots.wechat` + web QR bind; WeCom uses `robots.wecom` + admin callback URL—they are completely different.  
+- **WeChat QR expired**: QR codes last ~5 minutes; regenerate instead of reusing an old one.  
+- **Wrong bot type**: The “Custom” bot added in a DingTalk **group** (Webhook + sign secret) **cannot** be used for two-way chat. Only the **enterprise internal app** bot from the open platform is supported.  
+- **Saved but not restarted**: After changing DingTalk/Lark robot settings you **must restart** the app (WeChat QR bind restarts the connection automatically).  
+- **Client ID typo**: If the platform shows `504`, use `504` (not `5o4`); prefer copy/paste.  
+- **DingTalk: only HTTP callback, no Stream**: This app receives messages via **Stream**. In the open platform, message reception must be **Stream mode**.  
+- **App not published**: After changing the bot or permissions in the open platform, **publish a new version** under “Version management and release”, or changes won’t apply.
+
+---
+
+## 11. Notes
+
+- All platforms: **text messages only**; other types (e.g. image, voice) are not supported and may be ignored.  
+- Personal WeChat: **private chat only**—group @-bot is not supported.  
+- Conversations are shared with the web UI: conversations created from the bot appear in the web “Conversations” list and vice versa.  
+- Bot execution uses the same **Eino single/multi-agent** path as the web UI (`ProcessMessageForRobot`, with progress callbacks and process details stored in the DB); only the final reply is sent back to personal WeChat/DingTalk/Lark/WeCom in one message (no SSE). Default: `robot_default_agent_mode: eino_single`.
@@ -0,0 +1,188 @@
+# Runbooks
+
+[中文](../zh-CN/runbooks.md)
+
+Runbooks are task-oriented procedures you can follow during real operations.
+
+## Runbook 1: Production Instance from Zero to Ready
+
+Use for first-time internal or production red-team deployment.
+
+For local or temporary verification, start with the bundled script:
+
+```bash
+chmod +x run.sh && ./run.sh
+```
+
+After it is verified, decide whether to move to systemd plus reverse proxy for long-running deployment.
+
+### Preconditions
+
+- Host is managed as an asset.
+- Access path is decided: internal network, VPN, bastion, or reverse proxy.
+- Model API key and model are available.
+- C2, WebShell, and external MCP policy is decided.
+
+### Steps
+
+1. Prepare directory:
+
+```bash
+mkdir -p /opt/CyberStrikeAI
+```
+
+2. Place binary and resources:
+
+```text
+cyberstrike-ai
+web/
+tools/
+roles/
+skills/
+agents/
+docs/
+config.yaml
+```
+
+3. Set baseline config:
+
+```yaml
+auth:
+  password: "<long-random-password>"
+server:
+  host: 127.0.0.1
+  port: 8080
+  tls_enabled: false
+audit:
+  enabled: true
+c2:
+  enabled: false
+```
+
+4. Configure HTTPS at the reverse proxy and restrict source IPs.
+5. Run with systemd.
+6. Login and test the model.
+7. Check tools and audit logs.
+8. Create backup policy.
+
+### Acceptance
+
+- `/api/auth/validate` succeeds after login.
+- Model test passes.
+- Tools load.
+- Audit shows login.
+- C2 is disabled when not needed.
+
+## Runbook 2: Connect External MCP
+
+### Preconditions
+
+- MCP service is trusted.
+- You know whether it can read/write files, execute commands, or access networks.
+- Transport is chosen: stdio, HTTP, or SSE.
+
+### Steps
+
+1. Add service in External MCP page.
+2. For stdio, configure command, args, cwd, and env.
+3. For HTTP/SSE, configure URL and auth.
+4. Start service.
+5. Check `/api/external-mcp/stats`.
+6. Confirm tools and schemas.
+7. Execute one low-risk tool call.
+8. Keep high-risk tools out of global allowlist.
+
+### Acceptance
+
+- MCP status is running.
+- Tool schemas are visible.
+- Agent can find tools through `tool_search`.
+- Monitor records tool execution.
+- Audit records config change.
+
+## Runbook 3: Enable and Tune Knowledge Base
+
+### Steps
+
+1. Enable config:
+
+```yaml
+knowledge:
+  enabled: true
+  base_path: knowledge_base
+  retrieval:
+    top_k: 5
+    similarity_threshold: 0.4
+```
+
+2. Put Markdown files under `knowledge_base/`.
+3. Scan directory.
+4. Rebuild index.
+5. Prepare 5-10 fixed test queries.
+6. Search and record hits.
+7. Tune threshold, top_k, chunking, and document titles.
+
+### Acceptance
+
+- Index status is complete.
+- Common queries hit correct docs.
+- Agent consults KB when uncertain.
+- Retrieval logs show query and hit docs.
+
+## Runbook 4: Authorized Web Test Workflow
+
+1. Create project and record scope.
+2. Start conversation and bind project.
+3. Choose minimal role.
+4. State target, time window, and prohibited actions.
+5. Start with read-only recon.
+6. Record useful leads as project facts.
+7. Use HITL for risky validation.
+8. Save confirmed issues to vulnerability management.
+9. Generate attack-chain/report material.
+10. Clean uploads, workspace, and unnecessary execution logs.
+
+Acceptance:
+
+- Each vulnerability has evidence, impact, reproduction, and fix.
+- Risky actions have HITL records.
+- Project facts reconstruct the path.
+- Report excludes unrelated sensitive data.
+
+## Runbook 5: C2 Cleanup After Exercise
+
+1. Stop all listeners.
+2. List sessions and confirm no authorized session remains active.
+3. Export required task results.
+4. Delete or archive payloads.
+5. Delete stale tasks, events, and files.
+6. Review C2 audit trail.
+7. Write key results to project facts or report.
+8. Set `c2.enabled: false` unless continuously needed.
+
+Acceptance:
+
+- No running listener.
+- No pending task.
+- Payloads are not publicly downloadable.
+- Audit/report explains the lifecycle.
+
+## Runbook 6: Agent Does Not Call a Tool
+
+Check in order:
+
+1. Role includes the tool.
+2. Tool appears in `/api/config/tools`.
+3. `tool_search` is not hiding it.
+4. Tool name and description are clear.
+5. HITL is not pending.
+6. Agent is not in final summarization phase.
+7. Sub-agent does not have a narrower tool list.
+
+Fix:
+
+- add tool to role;
+- improve `short_description`;
+- add to `tool_search_always_visible_tools`;
+- prompt when to use it;
+- inspect process details and monitor records.
@@ -0,0 +1,130 @@
+# Security Hardening
+
+[中文](../zh-CN/security-hardening.md)
+
+This checklist covers pre-production and continuous hardening for CyberStrikeAI.
+
+## Before Going Live
+
+- Change `auth.password` to a long random secret.
+- Use HTTPS or a trusted reverse proxy.
+- Restrict access by IP, VPN, or bastion.
+- Enable `audit.enabled`.
+- Set `c2.enabled: false` when C2 is not required.
+- Do not expose standalone HTTP MCP without strong auth and network isolation.
+- Connect only trusted external MCP services.
+- Back up `config.yaml`, `data/`, and custom resource directories.
+
+## Reverse Proxy Baseline
+
+```nginx
+client_max_body_size 200m;
+proxy_buffering off;
+proxy_http_version 1.1;
+proxy_set_header Host $host;
+proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+proxy_set_header X-Forwarded-Proto https;
+proxy_set_header Upgrade $http_upgrade;
+proxy_set_header Connection "upgrade";
+```
+
+Recommended security headers:
+
+```nginx
+add_header X-Content-Type-Options nosniff;
+add_header Referrer-Policy no-referrer;
+add_header X-Frame-Options DENY;
+```
+
+## HITL Allowlist Baseline
+
+Minimal allowlist:
+
+```yaml
+hitl:
+  tool_whitelist:
+    - read_file
+    - glob
+    - grep
+    - tool_search
+```
+
+Do not globally allowlist:
+
+- `execute`;
+- WebShell write/execute tools;
+- C2 task/payload tools;
+- high-risk external MCP tools;
+- delete, write, upload, persistence tools.
+
+## File Permissions
+
+```bash
+chmod 600 config.yaml
+chmod 700 data
+```
+
+Run under a dedicated OS user. Avoid root unless explicitly required.
+
+## External MCP Review
+
+Before connecting:
+
+- Can it execute commands?
+- Can it read/write local files?
+- Does it send data to third parties?
+- Does it authenticate?
+- Can output contain untrusted model/web content?
+- Should it run in a container or separate user?
+
+After connecting:
+
+- keep high-risk tools out of allowlist;
+- review tool list changes;
+- audit config changes.
+
+## C2 and WebShell
+
+C2:
+
+- disabled by default;
+- enabled only during authorized window;
+- listener ports separated from admin UI;
+- cleanup payloads, sessions, tasks, and events.
+
+WebShell:
+
+- authorized targets only;
+- clear naming;
+- write/delete/execute requires approval;
+- delete connections after project end.
+
+## Retention
+
+Suggested:
+
+- audit: 30-90 days;
+- monitor: 90-180 days;
+- uploads: clean after project;
+- C2/WebShell outputs: keep only report evidence;
+- knowledge base: no real credentials or customer secrets.
+
+## Periodic Review
+
+Weekly:
+
+- failed logins and unusual IPs;
+- config changes;
+- external MCP changes;
+- long-running tools;
+- unexpected C2 enablement;
+- stale WebShell connections;
+- disk and DB size.
+
+Project closeout:
+
+- clean temp workspaces;
+- delete unnecessary uploads;
+- archive evidence;
+- delete stale WebShell/C2 resources;
+- export audit records.
@@ -0,0 +1,72 @@
+# Security Model
+
+[中文](../zh-CN/security-model.md)
+
+CyberStrikeAI is not a generic chatbot. It is a high-privilege security automation system with command execution, MCP tools, WebShell management, optional C2, batch tasks, and multi-agent orchestration.
+
+## Trust Boundaries
+
+Main actors:
+
+- Web user: can chat, change settings, manage resources, and trigger tools.
+- Agent: selects tools based on role, context, and middleware.
+- MCP tools: may access files, run commands, call services, or touch targets.
+- External MCP: third-party local or remote tool providers.
+- Robot callbacks: platform-authenticated message ingress outside Web login.
+
+Anyone who can log into the Web UI should be treated as an operator of the instance.
+
+## Threat Model
+
+| Threat | Path | Impact | Controls |
+| --- | --- | --- | --- |
+| Password leak | login, then use terminal/WebShell/C2 | platform takeover | strong password, HTTPS, internal network, audit |
+| Prompt injection | target content instructs Agent to misuse tools | unauthorized actions | role boundaries, HITL, least tools |
+| Malicious MCP | external tool lies or has side effects | host/target impact | trusted MCP only, isolation |
+| Tool YAML tampering | command template changed | malicious execution | file permissions, review |
+| C2 misuse | payload or task against unauthorized target | legal and business risk | disabled by default, approvals |
+| WebShell misuse | destructive command on business host | outage/data loss | naming, read-only first, HITL |
+| DB leak | copy `data/*.db` or uploads | sensitive target data | permissions, encrypted backups |
+
+## HITL Is Not Magic
+
+HITL sees a tool name, arguments, and context. It does not always see real-world impact. Be conservative when:
+
+- a harmless-looking command wraps `bash -c` or base64;
+- the MCP tool description is untrusted;
+- WebShell target identity is vague;
+- C2 payload delivery happens outside the platform;
+- a read-only tool can still create traffic or side effects.
+
+Audit Agent is useful for routine checks, not for replacing humans on destructive operations.
+
+## Data Minimization
+
+Avoid long-term storage of:
+
+- real customer credentials;
+- raw production data;
+- long-lived cookies;
+- unrelated scan output;
+- stale WebShell or C2 sessions.
+
+Project closeout should include cleanup of uploads, WebShell connections, C2 payloads, temporary workspaces, and bulky execution logs.
+
+## Production Baseline
+
+- Strong password and HTTPS.
+- Internal/VPN/proxy restricted access.
+- `audit.enabled: true`.
+- Random `mcp.auth_header_value` when HTTP MCP is exposed.
+- `c2.enabled: false` unless required.
+- Minimal external MCP.
+- No high-risk tools in global allowlist.
+
+## Source Anchors
+
+- Sessions: `internal/security/auth_manager.go`
+- Auth middleware: `internal/security/auth_middleware.go`
+- Rate limiting: `internal/security/ratelimit.go`
+- Shell execution: `internal/security/executor.go`
+- HITL execution: `internal/handler/hitl_execution.go`
+- Audit service: `internal/audit/service.go`
@@ -0,0 +1,70 @@
+# Skills Guide
+
+[中文](../zh-CN/skills-guide.md)
+
+Skills provide reusable procedures, checklists, templates, and references that Agents can load when needed. A Skill should be an executable procedure, not an encyclopedia page.
+
+## Structure
+
+```text
+skills/
+  ssrf-testing/
+    SKILL.md
+    REFERENCE.md
+```
+
+`SKILL.md` front matter:
+
+```markdown
+---
+name: ssrf-testing
+description: SSRF identification, validation, bypass, and remediation workflow
+---
+```
+
+The description determines when the Agent loads it.
+
+## Recommended Sections
+
+```markdown
+## When to use
+## Preconditions
+## Procedure
+## Stop conditions
+## Output
+```
+
+Stop conditions matter: they tell the Agent when to escalate, ask for approval, or stop expanding scope.
+
+## Anti-Patterns
+
+| Anti-pattern | Result | Fix |
+| --- | --- | --- |
+| Description too broad | triggers too often | make it scenario-specific |
+| Encyclopedia content | Agent lacks next step | write procedures and decisions |
+| Secrets in Skill | leakage/misuse | use runtime config or user input |
+| One huge Skill | costly and noisy | split by task/vulnerability |
+| No stop condition | scope creep | define approval/stop rules |
+
+## Skill vs Knowledge Base
+
+- Skill: how to do something.
+- Knowledge base: facts, references, cases.
+
+For SSRF, a Skill describes the test procedure; the KB stores metadata addresses, bypass cases, and remediation references.
+
+## Local Tool Risk
+
+`filesystem_tools: true` exposes local read/write/execute capability. In production:
+
+- constrain workspace;
+- require HITL for write/execute;
+- do not globally allowlist `execute`;
+- make Skills explicitly avoid out-of-scope files.
+
+## Source Anchors
+
+- Validation: `internal/skillpackage/validate.go`
+- Service: `internal/skillpackage/service.go`
+- Eino Skills: `internal/multiagent/eino_skills.go`
+- Handler: `internal/handler/skills.go`
@@ -0,0 +1,78 @@
+# Testing Guide
+
+[中文](../zh-CN/testing.md)
+
+Testing CyberStrikeAI means more than running Go tests. Agent, MCP, HITL, C2, WebShell, and frontend streaming all have different failure modes.
+
+## Commands
+
+```bash
+go test ./internal/...
+go test ./cmd/...
+go build -o cyberstrike-ai ./cmd/server
+```
+
+Run focused packages when working locally:
+
+```bash
+go test ./internal/multiagent
+go test ./internal/handler
+go test ./internal/security
+```
+
+## Test Pyramid
+
+| Layer | Goal | Example |
+| --- | --- | --- |
+| Unit | pure logic | expressions, chunking, sanitization |
+| Handler | HTTP behavior | validation, auth, status codes |
+| Integration | module cooperation | external MCP, KB indexing, HITL |
+| Smoke | user path | login, chat, tools, settings |
+| Authorized lab | high-risk features | C2, WebShell, terminal |
+
+Do not use end-to-end manual testing as a substitute for unit tests, or unit tests as a substitute for high-risk lab validation.
+
+## Regression Focus
+
+Expand testing when changing:
+
+- `internal/handler/config.go`: model, KB, MCP, C2, robot apply paths;
+- `internal/multiagent/`: streaming, tool calls, summarization, retry, HITL;
+- `internal/security/`: auth, shell, timeout, no-output;
+- `internal/database/`: old data compatibility;
+- `web/static/js/chat.js`: chat, process details, attack chain, groups.
+
+## Test Data
+
+Avoid real customer data. Prepare:
+
+- small Markdown KB sample;
+- fake local MCP server;
+- controlled local HTTP target;
+- harmless WebShell simulator;
+- temporary SQLite DB.
+
+## Failure Cases
+
+Cover:
+
+- model API 401/429/500;
+- MCP startup failure;
+- tool timeout;
+- HITL rejection;
+- interrupted KB indexing;
+- unwritable database;
+- WebShell non-200 response;
+- C2 disabled endpoint access.
+
+## Source Anchors
+
+Existing tests live across:
+
+- `internal/handler/*_test.go`
+- `internal/multiagent/*_test.go`
+- `internal/workflow/*_test.go`
+- `internal/knowledge/*_test.go`
+- `internal/security/*_test.go`
+- `internal/mcp/*_test.go`
+- `internal/c2/*_test.go`
@@ -0,0 +1,98 @@
+# Troubleshooting
+
+[中文](../zh-CN/troubleshooting.md)
+
+Debug by layer. Do not change random config before locating the failing layer.
+
+## Diagnostic Order
+
+1. Process: is the service alive, any panic?
+2. Network: port, HTTPS, reverse proxy, browser console.
+3. Auth: does `/api/auth/validate` return 200?
+4. Config: can `/api/config` be read and applied?
+5. Model: does model test pass?
+6. Tools: do tool list and schemas look right?
+7. Database: is `data/` writable, any lock?
+8. Subsystem: KB, MCP, C2, WebShell minimal action.
+
+## Minimal Commands
+
+```bash
+lsof -i :8080
+curl -k -I https://127.0.0.1:8080/
+curl -k -I https://127.0.0.1:8080/static/logo.png
+ls -lh data/
+```
+
+If a reverse proxy is involved, test both proxy address and upstream address.
+
+## Common Issues
+
+Page inaccessible:
+
+- wrong protocol, especially HTTPS vs HTTP;
+- self-signed cert warning;
+- port occupied;
+- reverse proxy loop.
+
+Login fails:
+
+- wrong `auth.password`;
+- config not applied/restarted;
+- stale cookie;
+- audit throttling repeated failures.
+
+Model fails:
+
+- wrong `base_url` path;
+- invalid API key;
+- model unavailable;
+- reasoning fields unsupported by gateway. Try `openai.reasoning.mode: off`.
+
+Streaming stalls:
+
+- proxy buffers SSE;
+- model gateway timeout;
+- context too large;
+- browser/network interruption.
+
+Tool fails:
+
+- real command not installed;
+- YAML schema wrong;
+- HITL rejected or pending;
+- timeout or no-output timeout.
+
+Knowledge base empty:
+
+- `knowledge.enabled` false;
+- scan/index not run;
+- embedding API failed;
+- threshold or risk type too strict.
+
+C2 returns 503:
+
+- expected when `c2.enabled: false`.
+
+## Common Misdiagnoses
+
+- "Model is broken": HITL is waiting.
+- "Tool missing": tool_search hides it from current context.
+- "Knowledge base useless": index not rebuilt or risk type too narrow.
+- "Config saved but ineffective": listener/TLS changes need restart.
+- "Robot silent": platform callback or signature config wrong.
+
+## Issue Template
+
+```text
+Version:
+Startup method:
+Access path:
+Relevant config:
+Steps:
+Expected:
+Actual:
+Server logs:
+Browser console:
+API response:
+```
@@ -0,0 +1,68 @@
+# WebShell Management
+
+[中文](../zh-CN/webshell.md)
+
+WebShell management stores authorized WebShell connections and allows command/file operations through the UI and Agent tools.
+
+## Workflow
+
+1. Add a connection.
+2. Fill URL, parameter/password, and metadata.
+3. Test connectivity.
+4. Run read-only identification commands first.
+5. Let AI assist only after selecting the correct connection.
+
+Connections are stored in SQLite.
+
+## Operation Tiers
+
+| Tier | Operation | Risk | Guidance |
+| --- | --- | --- | --- |
+| Identify | `whoami`, `pwd`, OS version | low | may automate |
+| Enumerate | dirs, processes, env vars | medium | constrain path/command |
+| Read | config, logs, source | medium-high | human confirms sensitivity |
+| Write/execute | write, run script, delete | high | human approval and rollback |
+
+Having a WebShell does not make follow-up operations low risk.
+
+## Naming
+
+Use:
+
+```text
+<project>-<environment>-<target>-<privilege>-<date>
+```
+
+Example:
+
+```text
+acme-staging-web01-www-20260707
+```
+
+Avoid vague names like `test`, `shell1`, or `customer machine`.
+
+## AI Guardrail Prompt
+
+```text
+Before using WebShell, confirm connection_id, target name, current directory, and privilege. Default to read-only commands. Any write, delete, upload, permission change, persistence, credential access, or internal probing requires purpose, impact, rollback plan, and approval.
+```
+
+## MCP Tools
+
+Typical tools:
+
+- `webshell_exec`
+- `webshell_file_list`
+- `webshell_file_read`
+- `webshell_file_write`
+- connection management tools
+
+Do not put write/execute tools in a global allowlist.
+
+## Source Anchors
+
+- Handler: `internal/handler/webshell.go`
+- Context: `internal/handler/webshell_context.go`
+- Probe: `internal/handler/webshell_probe.go`
+- Encoding/OS tests: `internal/handler/webshell_encoding_test.go`, `internal/handler/webshell_os_test.go`
+- Tool registration: `internal/app/app.go`
@@ -0,0 +1,553 @@
+# CyberStrikeAI Graph Orchestration Guide
+
+[中文](../zh-CN/workflow-graph.md)
+
+This document explains how to use **Graph Orchestration**: building workflows on the canvas, configuring node types, passing data between nodes, and binding a graph to a role for automatic execution.
+
+---
+
+## 1. Where to find Graph Orchestration
+
+1. Log in to the CyberStrikeAI web UI.  
+2. Open **Graph Orchestration** in the left sidebar.  
+3. Select an existing workflow from the list, or create a new one.  
+4. Drag nodes, draw edges, and configure properties on the canvas.  
+5. Fill in **ID**, **Name**, and **Description**, then click **Save**.
+
+Saved workflows can be bound to a role under **Role Management**. When `workflow_policy` is `auto`, chatting with that role runs the bound graph automatically.
+
+---
+
+## 2. Canvas basics
+
+| Action | Description |
+|--------|-------------|
+| Add node | Click a node type button above the canvas (Start, Tool, Agent, Condition, HITL, Output, End) |
+| Connect | Click **Connect**, then click source and target nodes; click **Connect** again to exit connect mode |
+| Select | Click a node or edge; properties appear in the right panel |
+| Delete selected | Remove the current node or edge |
+| Auto layout | Rearrange node positions |
+| Dry run | Safely simulate data flow; Tool, Agent, and HITL nodes are not executed for real |
+| Delete workflow | Remove the entire workflow definition |
+
+**Hard requirements:** Every workflow needs at least **one Start node** and **one Output node**. Start nodes must not have incoming edges; Output / End nodes must not have outgoing edges. Both frontend and backend run strict validation before save.
+
+---
+
+## 3. Execution model (read this before configuring)
+
+The engine executes the workflow as a **directed graph**, starting from the **Start** node and following edges to downstream nodes.
+
+During a run, the engine keeps internal state. Template expressions `{{...}}` read from that state:
+
+| Internal state | Template prefix | Meaning |
+|----------------|-----------------|---------|
+| `inputs` | `{{inputs.xxx}}` | Workflow inputs at start (user message, conversation ID, etc.) |
+| `lastOutput` | `{{previous.xxx}}` | Output of the **most recently executed** node |
+| `outputs` | `{{outputs.xxx}}` | Global **named variable pool** (written by nodes with an output key) |
+| `nodeOutputs` | `{{nodeId.xxx}}` | Full output object of a specific node ID |
+| `metrics` | available in run details | Node duration, tool call count, and usage/cost metrics when reported |
+
+### 3.1 What is `previous`?
+
+`{{previous.output}}` is the `output` field of the **immediately preceding executed node**.
+
+- After every node finishes, the engine updates `lastOutput`.
+- It is **not** “the node drawn upstream on the canvas”; it is **the previous step in actual execution order**.
+
+Example:
+
+```text
+Start → Agent A → Agent B
+```
+
+For Agent B, `{{previous.output}}` = Agent A’s output.
+
+With a condition in between:
+
+```text
+Start → Agent A → Condition → Agent B
+```
+
+For Agent B, `{{previous.output}}` = the **condition node** output (`true` / `false`), **not** Agent A’s result.
+
+If a node has **multiple upstream nodes**, `previous` is built by that node’s **join strategy** first:
+
+| Join strategy | Meaning | Use case |
+|---------------|---------|----------|
+| `all_merge` | Merge all upstream outputs; `previous.output` is an array | Default; aggregate multiple results |
+| `last_by_canvas` | Use the last upstream output by canvas order | Explicitly use one branch |
+| `first_non_empty` | Use the first non-empty output | Fallback chains |
+| `fail_fast` | Stop the node if any upstream failed | Critical gates, approval prechecks, safety checks |
+
+### 3.2 What is `outputs`?
+
+`outputs` is a **named variable registry** maintained by the engine during execution.
+
+When an Agent, Tool, or Output node sets an **Output variable name** (`output_key`), the result is stored as:
+
+```text
+outputs["your_variable_name"] = node_output
+```
+
+Any downstream node can then reference it via `{{outputs.variable_name}}`, even if other nodes sit in between.
+
+Example:
+
+- Agent A **Output variable name**: `agent_result1`
+- Agent B **Input source**: `{{outputs.agent_result1}}`
+
+Agent B still receives Agent A’s output even when a condition node lies between them.
+
+### 3.3 When to use `previous` vs `outputs`
+
+| Scenario | Recommended |
+|----------|-------------|
+| Two nodes are **directly connected**; you only need the last step | `{{previous.output}}` |
+| Other nodes sit in between (condition, tool, HITL, etc.) | `{{outputs.variable_name}}` |
+| Reference output from an **earlier** node | `{{outputs.variable_name}}` or `{{nodeId.output}}` |
+| Condition should test an Agent’s output | `{{outputs.variable_name}} != ""` |
+| Read the original user input | `{{inputs.message}}` |
+
+**Rule of thumb:**
+
+- `previous` = last step (chained, adjacent)
+- `outputs` = by name (cross-node, look back)
+
+---
+
+## 4. Template syntax
+
+### 4.1 Basic format
+
+```text
+{{path.to.value}}
+```
+
+Allowed characters in paths: letters, digits, underscore, dot, hyphen. Examples:
+
+```text
+{{previous.output}}
+{{outputs.agent_result1}}
+{{inputs.message}}
+{{inputs.conversationId}}
+{{previous.matched}}
+{{node-abc123.output}}
+```
+
+### 4.2 Available paths
+
+| Path | Description |
+|------|-------------|
+| `{{inputs.message}}` | User message (Start node input) |
+| `{{inputs.conversationId}}` | Conversation ID |
+| `{{inputs.projectId}}` | Project ID |
+| `{{previous.output}}` | Primary output of the previous node |
+| `{{previous.matched}}` | Match result of the previous condition node (`true` / `false`) |
+| `{{outputs.variable_name}}` | Named output registered by a node |
+| `{{nodeId.output}}` | `output` field of the node with that ID |
+| `{{previous.kind}}` | Previous node output kind, e.g. `agent` / `tool` / `condition` |
+| `{{previous.status}}` | Previous node status, e.g. `completed` / `failed` / `simulated` |
+
+Node outputs keep compatibility fields such as `output` and `matched`, and also include a structured envelope:
+
+```json
+{
+  "kind": "agent",
+  "node_id": "node-2",
+  "node_type": "agent",
+  "status": "completed",
+  "output": "..."
+}
+```
+
+### 4.3 Condition expressions
+
+Condition nodes and edge conditions support comparisons, text matching, regex, logical operators, and safe JSONPath/JQ path reads:
+
+```text
+{{outputs.agent_result1}} != ""
+{{previous.output}} == "ok"
+{{outputs.count}} >= 100
+{{previous.output}} contains "success"
+{{previous.output}} matches "^ok"
+{{outputs.risk_score}} >= 8 && {{previous.output}} != ""
+jsonpath({{previous.output}}, "$.status") == "ok"
+jq({{outputs.scan}}, ".severity") == "high"
+```
+
+Rules:
+
+- Operators: `==`, `!=`, `>`, `>=`, `<`, `<=`
+- `contains` checks substrings; `matches` checks regular expressions
+- Simple `&&` / `||` is supported
+- `jsonpath(value, "$.path")` and `jq(value, ".path")` support a **safe path-only subset**; no arbitrary script execution
+- Leading/trailing spaces and quotes are trimmed before comparison
+- Without a comparator, non-empty values that are not `false`, `0`, or `null` are treated as true
+- Expressions, regexes, and JSONPath/JQ paths are statically validated before save
+
+### 4.4 Nested field binding
+
+Field bindings can read ordinary fields such as `output` or `message`, and also JSONPath/JQ-style paths:
+
+| Binding | Meaning |
+|---------|---------|
+| `from=previous, field=$.status` | Read `status` from previous output |
+| `from=outputs, field=$.scan.severity` | Read a nested field from named outputs |
+| `from=node-1, field=.output.items[0]` | Read an array element from a specific node output |
+
+---
+
+## 5. Node types and configuration
+
+### 5.1 Start
+
+Workflow entry point; injects user input into `inputs`.
+
+| Field | Description | Default |
+|-------|-------------|---------|
+| Input keys | Comma-separated input key names | `message, conversationId, projectId` |
+
+Start node output includes: `output`, `message`, `conversationId`, `projectId`.
+
+### 5.2 Agent
+
+Runs an LLM Agent task. Supports multiple modes.
+
+| Field | Description | Default |
+|-------|-------------|---------|
+| Agent mode | `eino_single` / `deep` / `plan_execute` / `supervisor` | `eino_single` |
+| Input source | Template for upstream data | `{{previous.output}}` |
+| Node instruction | Task description for this node | empty |
+| Output variable name | Key written into `outputs` | `agent_result` |
+| Join strategy | How to build `previous` when multiple upstreams enter this node | `all_merge` |
+
+**Message assembly:**
+
+- Instruction only → send instruction to the Agent  
+- Input source only → “Continue based on upstream output: …”  
+- Both → combined “upstream input + node instruction”
+
+After execution:
+
+- `previous.output` becomes this node’s response text  
+- If **Output variable name** is set, the value is also stored in `outputs[variable_name]`
+- In the Eino graph, the Agent node is split into `prepare → execute → finalize` for clearer trace and future checkpointing
+
+### 5.3 Tool
+
+Calls an enabled MCP tool.
+
+| Field | Description | Default |
+|-------|-------------|---------|
+| MCP tool | Tool name (required) | — |
+| Argument template | JSON with `{{...}}` templates | `{}` |
+| Timeout (seconds) | Optional | empty |
+| Join strategy | How to build `previous` when multiple upstreams enter this node | `all_merge` |
+
+Example argument template:
+
+```json
+{"target": "{{inputs.message}}", "port": "443"}
+```
+
+If an output variable name is configured, the tool result is written to `outputs`.
+
+### 5.4 Condition
+
+Evaluates an expression and outputs `matched` (`true` / `false`).
+
+| Field | Description | Default |
+|-------|-------------|---------|
+| Expression | Supports `{{...}}` and `==` / `!=` | `{{previous.output}} != ""` |
+| Join strategy | How to build `previous` when multiple upstreams enter this node | `all_merge` |
+
+**Branching rules:**
+
+- The **first outgoing edge** defaults to the **“yes”** branch (`matched == true`)
+- The **second outgoing edge** defaults to the **“no”** branch (`matched == false`)
+- Edge labels such as `是` / `否` (or `yes` / `no`, `true` / `false`) help identify branches
+- A third or later edge needs a custom **edge condition**
+
+Edge condition examples (select an edge, configure in the right panel):
+
+```text
+{{previous.matched}} == "true"
+{{previous.matched}} == "false"
+```
+
+### 5.5 HITL (human-in-the-loop)
+
+Human approval checkpoint. The run pauses before this node through Eino interrupt/checkpoint and resumes after approval via API or the monitor panel.
+
+| Field | Description | Default |
+|-------|-------------|---------|
+| Prompt | Supports templates | `Please approve before continuing` |
+| Prompt binding | If prompt text is empty, read approval text from a bound field | `previous.output` |
+| Reviewer | `human` / `audit_agent` | `human` |
+| Join strategy | How to build `previous` when multiple upstreams enter this node | `all_merge` |
+
+Pending HITL metadata records:
+
+- `checkpointId`
+- interrupt `beforeNodes`
+- resume target / address / path
+- resume payload schema (`approved`, `comment`)
+
+### 5.6 Output
+
+Writes the final workflow result into `outputs` for summary and chat display.
+
+| Field | Description | Default |
+|-------|-------------|---------|
+| Output variable name | Required key for the final result | `result` |
+| Variable source | Template deciding what to write | `{{previous.output}}` |
+| Static output value | Optional; overrides variable source when set | empty |
+| Join strategy | How to build `previous` when multiple upstreams enter this node | `all_merge` |
+
+**Note:** Output nodes are workflow exits and must not have outgoing edges.
+
+### 5.7 End
+
+Optional node for an end summary template (less common in role-bound flows).
+
+| Field | Description | Default |
+|-------|-------------|---------|
+| Result template | Supports `{{outputs.xxx}}` | `{{outputs.result}}` |
+| Join strategy | How to build `previous` when multiple upstreams enter this node | `all_merge` |
+
+---
+
+## 6. Edge configuration
+
+Select an **edge** to configure its **condition** in the right panel.
+
+| Scenario | Example |
+|----------|---------|
+| Filter after a normal node | `{{previous.output}} == "ok"` |
+| “Yes” branch from a condition | `{{previous.matched}} == "true"` |
+| “No” branch from a condition | `{{previous.matched}} == "false"` |
+
+If no edge condition is set:
+
+- Non-condition nodes: edge is always allowed  
+- Condition nodes: yes/no branches are assigned by edge order automatically
+
+---
+
+## 7. Full example: passing Agent output across a condition
+
+### 7.1 Graph structure
+
+```text
+Start → Agent (initial value) → Condition → Agent (transform) → Output
+                                    ↘ no → Output
+```
+
+### 7.2 Node configuration
+
+**Agent 1**
+
+| Field | Value |
+|-------|-------|
+| Node instruction | Output only `123333333` |
+| Output variable name | `agent_result1` |
+
+**Condition**
+
+| Field | Value |
+|-------|-------|
+| Expression | `{{outputs.agent_result1}} != ""` |
+
+**Agent 2**
+
+| Field | Value |
+|-------|-------|
+| Input source | `{{outputs.agent_result1}}` |
+| Node instruction | Add 100 to the input, then output |
+| Output variable name | `agent_result` |
+
+**Output**
+
+| Field | Value |
+|-------|-------|
+| Output variable name | `result` |
+| Variable source | `{{outputs.agent_result}}` |
+
+### 7.3 Common mistakes
+
+| Wrong config | Why it fails |
+|--------------|--------------|
+| Agent 2 input source = `{{previous.output}}` | `previous` points to the condition node → `true`/`false`, not Agent 1’s text |
+| Agent 1 has no output variable name | `outputs.agent_result1` does not exist → empty downstream |
+| Condition uses `{{previous.output}}` | Tests the wrong upstream value instead of Agent 1’s named output |
+
+---
+
+## 8. Bind to a role and run
+
+### 8.1 Bind in Role Management
+
+1. Open **Role Management**, edit or create a role.  
+2. Select the workflow / graph ID to bind.  
+3. Set policy to `auto` (default when `workflow_id` is set).  
+4. Save the role.
+
+You can also configure this in role YAML:
+
+```yaml
+name: workflow-test
+workflow_id: "1233"
+workflow_version: latest
+workflow_policy: auto
+```
+
+### 8.2 Runtime behavior
+
+When a user chats with that role:
+
+1. The engine loads `graph_json` and executes the graph.  
+2. The chat UI shows progress events (`workflow_start`, `workflow_node_start`, Agent reasoning, etc.).  
+3. When finished, a summary lists all named entries in `outputs`.
+
+If no Output node is reached or no branch matches, `outputs` may be empty and the summary will suggest checking the Output node and branches.
+
+---
+
+## 9. Debugging, dry-run, and replay
+
+### 9.1 Safe dry-run
+
+Click **Dry run** on the canvas toolbar and enter a test message to simulate the workflow.
+
+Dry-run safety rules:
+
+- `start` / `condition` / `output` / `end` use real logic
+- `tool` does not call MCP; it returns `[dry-run] tool call skipped`
+- `agent` does not call the model; it returns `[dry-run] agent execution skipped`
+- `hitl` does not pause; it simulates approval
+
+API:
+
+```http
+POST /api/workflows/dry-run
+```
+
+Request:
+
+```json
+{
+  "graph": { "nodes": [], "edges": [], "config": {} },
+  "inputs": { "message": "ping" }
+}
+```
+
+Response includes:
+
+- `outputs`
+- `nodeOutputs`
+- `trace`
+- `metrics`
+- `replayScript`
+
+### 9.2 Run details and replay
+
+Query full node execution traces after a run:
+
+```http
+GET /api/workflows/runs/{runId}
+```
+
+The response contains `run` and `nodeRuns`. Each node run records:
+
+- input snapshot
+- output snapshot
+- status / error
+- started_at / finished_at
+- `duration_ms`
+
+Replay API:
+
+```http
+GET /api/workflows/runs/{runId}/replay
+```
+
+This generates replay steps from saved `nodeRuns`; it does not re-execute tools or Agents.
+
+### 9.3 Metrics
+
+The workflow accumulates, when available:
+
+- `node_count`
+- `duration_ms`
+- `tool_call_count`
+- Agent progress usage such as `prompt_tokens` / `completion_tokens` / `total_tokens` / `cost`
+
+Token and cost metrics depend on whether the underlying model/Agent events report usage.
+
+---
+
+## 10. Validation before save
+
+On save, the system checks:
+
+| Rule | Description |
+|------|-------------|
+| Start node required | At least one `start` node |
+| Output node required | At least one `output` node with an output variable name |
+| Valid edges | Source and target exist; no self-loops |
+| Start has no incoming edges | Start must not be targeted |
+| Output / End has no outgoing edges | Nothing after Output / End |
+| Non-start nodes must have incoming edges | Prevent orphan nodes |
+| Non-output/end nodes must have outgoing edges | Prevent dead ends |
+| No cycles | Workflow orchestration must be a DAG |
+| Reachability | Every node must be reachable from Start and eventually reach output/end |
+| Tool nodes | MCP tool required; argument JSON must be valid; timeout must be a positive integer |
+| Agent nodes | Must have node instruction or input binding; output variable name required |
+| Condition nodes | Expression required; 1–2 outgoing edges; branches must be yes/no and unique |
+| Edge conditions | Expressions, regexes, and JSONPath/JQ paths must pass static validation |
+| Join strategy | Must be `all_merge` / `last_by_canvas` / `first_non_empty` / `fail_fast` |
+
+---
+
+## 11. Troubleshooting
+
+| Symptom | Likely cause | Fix |
+|---------|--------------|-----|
+| Downstream gets empty value | Upstream has no output variable name | Set **Output variable name** on upstream; use `{{outputs.xxx}}` downstream |
+| Downstream gets `true`/`false` | Used `{{previous.output}}` while previous node is a condition | Use `{{outputs.xxx}}` instead |
+| Condition always takes “no” | Expression does not match actual output format | Check Agent output for quotes/newlines; try `!= ""` first |
+| No final output | Output node branch not reached | Verify condition wiring; ensure every path reaches an **Output** node |
+| Role chat does not run workflow | Role not bound or disabled | Check `workflow_id`, `workflow_policy: auto`, workflow `enabled: true` |
+| Tool node fails | Invalid JSON in arguments or tool disabled | Fix argument template; enable the tool in MCP settings |
+| Save fails with invalid branch | Condition outgoing edges are not marked yes/no, or are duplicated | Select the edge and set branch to `true` or `false` |
+| Multi-upstream result is unexpected | Join strategy does not match the workflow | Switch between `all_merge`, `first_non_empty`, `last_by_canvas`, and `fail_fast` |
+| Nested field is empty | JSONPath/JQ path is outside the safe subset | Use `$.a.b[0]` or `.a.b[0]`; avoid wildcards, recursion, or expressions |
+
+---
+
+## 12. Best practices
+
+1. **Meaningful names**: Use descriptive output variable names (`scan_result`, `parsed_targets`) instead of reusing `agent_result` everywhere.  
+2. **Prefer `outputs` for cross-node data**: If a condition, tool, or HITL node might sit in between, use named variables.  
+3. **Use `previous` only for direct links**: `A → B` with nothing in between is the ideal case for `{{previous.output}}`.  
+4. **Conditions should reference source data**: When testing Agent output, use `{{outputs.xxx}}` unless the condition immediately follows that Agent.  
+5. **Every path needs an exit**: Ensure both yes and no branches eventually reach an **Output** node (or your intended end).  
+6. **Choose join strategy explicitly for multi-upstream nodes**: Use `all_merge` for aggregation, `first_non_empty` for fallback, and `fail_fast` for critical gates.  
+7. **Use JSONPath/JQ safe paths for nested JSON**: e.g. `jsonpath({{previous.output}}, "$.status") == "ok"`.  
+8. **Dry-run before real execution**: Validate data flow and branches with a simple message before binding the workflow to a role.
+
+---
+
+## 13. Code references (for developers)
+
+| Module | Path |
+|--------|------|
+| Execution engine | `internal/workflow/runner.go` |
+| Eino compile / checkpoint / HITL | `internal/workflow/eino_compile.go` |
+| Graph validation | `internal/workflow/validation.go` |
+| Expressions / JSONPath / joins | `internal/workflow/expression.go`, `jsonpath.go`, `join.go` |
+| Dry-run / replay data | `internal/workflow/dry_run.go`, `internal/handler/workflow_run.go` |
+| Canvas UI | `web/static/js/workflows.js` |
+| Workflow API | `internal/handler/workflow.go` |
+| Role binding | `internal/config/config.go` (`workflow_id` field) |