mirror of
https://github.com/Ed1s0nZ/CyberStrikeAI.git
synced 2026-05-17 21:44:43 +02:00
Compare commits
70 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 0b950f95db | |||
| d36984a1c1 | |||
| da2109a970 | |||
| 1866aa8089 | |||
| 5af06e539d | |||
| 7493e70686 | |||
| 81f7a601b7 | |||
| 27830d1399 | |||
| d9a0178f80 | |||
| 1dd8cc7f50 | |||
| 55045dd4e0 | |||
| 90508c9084 | |||
| 361480f2d1 | |||
| 538565117b | |||
| 1c8742b7b6 | |||
| 2fb6a1d1ef | |||
| 6e390acb3d | |||
| d6236e285d | |||
| ad8efffbb4 | |||
| 352d9b712c | |||
| acadbe19c6 | |||
| c265e66afb | |||
| 647bb4b5e4 | |||
| dd311f7a3b | |||
| 2e482a3baf | |||
| 67d5e7f11e | |||
| 7e0198a64c | |||
| 1e50272229 | |||
| 39b47a86fb | |||
| 74738ee555 | |||
| 90bc3f4b61 | |||
| ad96be3c64 | |||
| 8866ff4cdd | |||
| 3534a956b2 | |||
| 691793cb38 | |||
| 7270e3c3d1 | |||
| 5e28782b1f | |||
| 3e61b77b9c | |||
| 64f9053061 | |||
| 426b0e282e | |||
| 78c6bd0b6a | |||
| e54815e018 | |||
| 9baa99ea40 | |||
| 83a8c46db1 | |||
| 4b2619e1fe | |||
| 3fffee80f4 | |||
| 41d7afcf99 | |||
| 6431dcb240 | |||
| 665b1d553a | |||
| fd3a52af01 | |||
| 8368ee7712 | |||
| dd883677b8 | |||
| 2edd5ffe95 | |||
| ae588dbfe4 | |||
| 93be113a79 | |||
| d3fb14f72d | |||
| af715e23cb | |||
| 3aecdc275f | |||
| 660d95a787 | |||
| 01271fd8eb | |||
| 8c6e044f84 | |||
| cb2defd0cc | |||
| 88ab73e422 | |||
| 5404d95db7 | |||
| 32d0e98cfb | |||
| e4b1e10a42 | |||
| 870715fc8f | |||
| 772a04b715 | |||
| 2455bde7ab | |||
| dbdfc18d57 |
@@ -14,6 +14,12 @@ CyberStrikeAI is an **AI-native security testing platform** built in Go. It inte
|
||||
|
||||
<div align="center">
|
||||
|
||||
### System Dashboard Overview
|
||||
|
||||
<img src="./images/dashboard.png" alt="System Dashboard" width="100%">
|
||||
|
||||
*The dashboard provides a comprehensive overview of system runtime status, security vulnerabilities, tool usage, and knowledge base, helping users quickly understand the platform's core features and current state.*
|
||||
|
||||
### Core Features Overview
|
||||
|
||||
<table>
|
||||
@@ -77,6 +83,7 @@ CyberStrikeAI is an **AI-native security testing platform** built in Go. It inte
|
||||
- 📋 Batch task management: create task queues, add multiple tasks, and execute them sequentially
|
||||
- 🎭 Role-based testing: predefined security testing roles (Penetration Testing, CTF, Web App Scanning, etc.) with custom prompts and tool restrictions
|
||||
- 🎯 Skills system: 20+ predefined security testing skills (SQL injection, XSS, API security, etc.) that can be attached to roles or called on-demand by AI agents
|
||||
- 📱 **Chatbot**: DingTalk and Lark (Feishu) long-lived connections so you can talk to CyberStrikeAI from mobile (see [Robot / Chatbot guide](docs/robot_en.md) for setup and commands)
|
||||
|
||||
## Tool Overview
|
||||
|
||||
@@ -454,6 +461,10 @@ tools:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
## Related documentation
|
||||
|
||||
- [Robot / Chatbot guide (DingTalk & Lark)](docs/robot_en.md): Full setup, commands, and troubleshooting for using CyberStrikeAI from DingTalk or Lark on your phone. **Follow this doc to avoid common pitfalls.**
|
||||
|
||||
## Project Layout
|
||||
|
||||
```
|
||||
@@ -464,6 +475,7 @@ CyberStrikeAI/
|
||||
├── tools/ # YAML tool recipes (100+ examples provided)
|
||||
├── roles/ # Role configurations (12+ predefined security testing roles)
|
||||
├── skills/ # Skills directory (20+ predefined security testing skills)
|
||||
├── docs/ # Documentation (e.g. robot/chbot guide)
|
||||
├── images/ # Docs screenshots & diagrams
|
||||
├── config.yaml # Runtime configuration
|
||||
├── run.sh # Convenience launcher
|
||||
@@ -489,20 +501,6 @@ Compress the 5 MB nuclei report, summarize critical CVEs, and attach the artifac
|
||||
Build an attack chain for the latest engagement and export the node list with severity >= high.
|
||||
```
|
||||
|
||||
## Changelog
|
||||
|
||||
### Recent Highlights
|
||||
|
||||
- **2026-01-27** – OpenAPI documentation with interactive testing interface, supporting conversation management, message interaction, and result querying
|
||||
- **2026-01-15** – Skills system with 20+ predefined security testing skills
|
||||
- **2026-01-11** – Role-based testing with predefined security testing roles
|
||||
- **2026-01-08** – SSE transport mode support for external MCP servers
|
||||
- **2026-01-01** – Batch task management with queue-based execution
|
||||
- **2025-12-25** – Vulnerability management and conversation grouping features
|
||||
- **2025-12-20** – Knowledge base with vector search and hybrid retrieval
|
||||
|
||||
|
||||
|
||||
## 404Starlink
|
||||
|
||||
<img src="./images/404StarLinkLogo.png" width="30%">
|
||||
@@ -516,8 +514,26 @@ CyberStrikeAI has joined [404Starlink](https://github.com/knownsec/404StarLink)
|
||||
</a>
|
||||
</div>
|
||||
|
||||
## Stargazers over time
|
||||

|
||||
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Disclaimer
|
||||
|
||||
**This tool is for educational and authorized testing purposes only!**
|
||||
|
||||
CyberStrikeAI is a professional security testing platform designed to assist security researchers, penetration testers, and IT professionals in conducting security assessments and vulnerability research **with explicit authorization**.
|
||||
|
||||
**By using this tool, you agree to:**
|
||||
- Use this tool only on systems where you have clear written authorization
|
||||
- Comply with all applicable laws, regulations, and ethical standards
|
||||
- Take full responsibility for any unauthorized use or misuse
|
||||
- Not use this tool for any illegal or malicious purposes
|
||||
|
||||
**The developers are not responsible for any misuse!** Please ensure your usage complies with local laws and regulations, and that you have obtained explicit authorization from the target system owner.
|
||||
|
||||
---
|
||||
|
||||
Need help or want to contribute? Open an issue or PR—community tooling additions are welcome!
|
||||
|
||||
+31
-13
@@ -13,6 +13,12 @@ CyberStrikeAI 是一款 **AI 原生安全测试平台**,基于 Go 构建,集
|
||||
|
||||
<div align="center">
|
||||
|
||||
### 系统仪表盘概览
|
||||
|
||||
<img src="./images/dashboard.png" alt="系统仪表盘" width="100%">
|
||||
|
||||
*仪表盘提供系统运行状态、安全漏洞、工具使用情况和知识库的全面概览,帮助用户快速了解平台核心功能和当前状态。*
|
||||
|
||||
### 核心功能概览
|
||||
|
||||
<table>
|
||||
@@ -76,6 +82,7 @@ CyberStrikeAI 是一款 **AI 原生安全测试平台**,基于 Go 构建,集
|
||||
- 📋 批量任务管理:创建任务队列,批量添加任务,依次顺序执行,支持任务编辑与状态跟踪
|
||||
- 🎭 角色化测试:预设安全测试角色(渗透测试、CTF、Web 应用扫描等),支持自定义提示词和工具限制
|
||||
- 🎯 Skills 技能系统:20+ 预设安全测试技能(SQL 注入、XSS、API 安全等),可附加到角色或由 AI 按需调用
|
||||
- 📱 **机器人**:支持钉钉、飞书长连接,在手机端与 CyberStrikeAI 对话(配置与命令详见 [机器人使用说明](docs/robot.md))
|
||||
|
||||
## 工具概览
|
||||
|
||||
@@ -453,6 +460,10 @@ tools:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
## 相关文档
|
||||
|
||||
- [机器人使用说明(钉钉 / 飞书)](docs/robot.md):在手机端通过钉钉、飞书与 CyberStrikeAI 对话的完整配置步骤、命令与排查说明,**建议按该文档操作以避免走弯路**。
|
||||
|
||||
## 项目结构
|
||||
|
||||
```
|
||||
@@ -463,6 +474,7 @@ CyberStrikeAI/
|
||||
├── tools/ # YAML 工具目录(含 100+ 示例)
|
||||
├── roles/ # 角色配置文件目录(含 12+ 预设安全测试角色)
|
||||
├── skills/ # Skills 目录(含 20+ 预设安全测试技能)
|
||||
├── docs/ # 说明文档(如机器人使用说明)
|
||||
├── images/ # 文档配图
|
||||
├── config.yaml # 运行配置
|
||||
├── run.sh # 启动脚本
|
||||
@@ -488,19 +500,6 @@ CyberStrikeAI/
|
||||
构建最新一次测试的攻击链,只导出风险 >= 高的节点列表。
|
||||
```
|
||||
|
||||
## 更新日志
|
||||
|
||||
### 近期亮点
|
||||
|
||||
- **2026-01-27** – 新增 OpenAPI 文档,提供交互式测试界面,支持对话管理、消息交互和结果查询
|
||||
- **2026-01-15** – 新增 Skills 技能系统,内置 20+ 预设安全测试技能
|
||||
- **2026-01-11** – 新增角色化测试功能,支持预设安全测试角色
|
||||
- **2026-01-08** – 新增 SSE 传输模式支持,外部 MCP 联邦支持三种模式
|
||||
- **2026-01-01** – 新增批量任务管理功能,支持队列式任务执行
|
||||
- **2025-12-25** – 新增漏洞管理和对话分组功能
|
||||
- **2025-12-20** – 新增知识库功能,支持向量检索和混合搜索
|
||||
|
||||
|
||||
## 404星链计划
|
||||
<img src="./images/404StarLinkLogo.png" width="30%">
|
||||
|
||||
@@ -513,6 +512,25 @@ CyberStrikeAI 现已加入 [404星链计划](https://github.com/knownsec/404Star
|
||||
</a>
|
||||
</div>
|
||||
|
||||
## Stargazers over time
|
||||

|
||||
|
||||
---
|
||||
|
||||
## ⚠️ 免责声明
|
||||
|
||||
**本工具仅供教育和授权测试使用!**
|
||||
|
||||
CyberStrikeAI 是一个专业的安全测试平台,旨在帮助安全研究人员、渗透测试人员和IT专业人员在**获得明确授权**的情况下进行安全评估和漏洞研究。
|
||||
|
||||
**使用本工具即表示您同意:**
|
||||
- 仅在您拥有明确书面授权的系统上使用此工具
|
||||
- 遵守所有适用的法律法规和道德准则
|
||||
- 对任何未经授权的使用或滥用行为承担全部责任
|
||||
- 不会将本工具用于任何非法或恶意目的
|
||||
|
||||
**开发者不对任何滥用行为负责!** 请确保您的使用符合当地法律法规,并获得目标系统所有者的明确授权。
|
||||
|
||||
---
|
||||
|
||||
欢迎提交 Issue/PR 贡献新的工具模版或优化建议!
|
||||
|
||||
+50
-1
@@ -10,7 +10,7 @@
|
||||
# ============================================
|
||||
|
||||
# 前端显示的版本号(可选,不填则显示默认版本)
|
||||
version: "v1.3.5"
|
||||
version: "v1.3.18"
|
||||
|
||||
# 服务器配置
|
||||
server:
|
||||
@@ -44,6 +44,16 @@ openai:
|
||||
model: deepseek-chat # 模型名称(必填)
|
||||
max_total_tokens: 120000 # LLM 相关上下文的最大 Token 数限制(内存压缩和攻击链构建会共用此配置)
|
||||
|
||||
# ============================================
|
||||
# 信息收集(FOFA)配置(可选)
|
||||
# ============================================
|
||||
# 用于「信息收集」页面调用 FOFA API(后端代理,避免前端暴露 key)
|
||||
# 也可通过环境变量配置:FOFA_EMAIL / FOFA_API_KEY(优先级更高)
|
||||
fofa:
|
||||
base_url: "https://fofa.info/api/v1/search/all" # 可选,留空则使用默认
|
||||
email: "" # FOFA 账号邮箱(可选,建议在系统设置中填写)
|
||||
api_key: "" # FOFA API Key(可选,建议在系统设置中填写)
|
||||
|
||||
# Agent 配置
|
||||
# 达到最大迭代次数时,AI 会自动总结测试结果
|
||||
agent:
|
||||
@@ -106,6 +116,45 @@ knowledge:
|
||||
top_k: 5 # 检索返回的Top-K结果数量
|
||||
similarity_threshold: 0.7 # 相似度阈值(0-1),低于此值的结果将被过滤
|
||||
hybrid_weight: 0.7 # 混合检索权重(0-1),向量检索的权重,1.0表示纯向量检索,0.0表示纯关键词检索
|
||||
# ============================================
|
||||
# 索引配置(用于解决 API 限制问题)
|
||||
# ============================================
|
||||
indexing:
|
||||
# 分块配置
|
||||
chunk_size: 512 # 每个块的最大 token 数(默认 512),长文本会被分割成多个块
|
||||
chunk_overlap: 50 # 块之间的重叠 token 数(默认 50),保持上下文连贯性
|
||||
max_chunks_per_item: 0 # 单个知识项的最大块数量(0 表示不限制),防止单个文件消耗过多 API 配额
|
||||
# 速率限制配置(解决 429 错误)
|
||||
max_rpm: 0 # 每分钟最大请求数(默认 0 表示不限制),如 OpenAI 默认 200 RPM
|
||||
rate_limit_delay_ms: 300 # 请求间隔毫秒数(默认 300),用于避免 API 速率限制,设为 0 不限制
|
||||
# 建议值:200 次/分钟≈300ms, 100 次/分钟≈600ms
|
||||
|
||||
# 重试配置
|
||||
max_retries: 3 # 最大重试次数(默认 3),遇到速率限制或服务器错误时自动重试
|
||||
retry_delay_ms: 1000 # 重试间隔毫秒数(默认 1000),每次重试会递增延迟
|
||||
|
||||
# ============================================
|
||||
# 机器人配置(企业微信、钉钉、飞书)
|
||||
# ============================================
|
||||
# 用于在手机端通过企业微信/钉钉/飞书与 CyberStrikeAI 对话,无需部署在服务器上也可使用
|
||||
# 在系统设置 -> 机器人设置 中可配置
|
||||
robots:
|
||||
wecom: # 企业微信
|
||||
enabled: false
|
||||
token: ""
|
||||
encoding_aes_key: ""
|
||||
corp_id: ""
|
||||
secret: ""
|
||||
agent_id: 0
|
||||
dingtalk: # 钉钉
|
||||
enabled: false
|
||||
client_id:
|
||||
client_secret:
|
||||
lark: # 飞书
|
||||
enabled: false
|
||||
app_id: ""
|
||||
app_secret: ""
|
||||
verify_token: ""
|
||||
|
||||
# ============================================
|
||||
# Skills 相关配置
|
||||
|
||||
+225
@@ -0,0 +1,225 @@
|
||||
# CyberStrikeAI 机器人使用说明
|
||||
|
||||
[English](robot_en.md)
|
||||
|
||||
本文档说明如何通过**钉钉**、**飞书**与 CyberStrikeAI 对话(长连接模式),在手机端即可使用,无需在服务器上打开网页。按下面步骤操作可避免常见弯路。
|
||||
|
||||
---
|
||||
|
||||
## 一、在 CyberStrikeAI 里从哪里配置
|
||||
|
||||
1. 登录 CyberStrikeAI Web 端
|
||||
2. 左侧导航进入 **系统设置**
|
||||
3. 在左侧设置分类中点击 **机器人设置**(位于「基本设置」与「安全设置」之间)
|
||||
4. 按平台勾选并填写(钉钉填 Client ID / Client Secret,飞书填 App ID / App Secret)
|
||||
5. 点击 **应用配置** 保存
|
||||
6. **重启 CyberStrikeAI 应用**(只保存不重启,机器人不会连上)
|
||||
|
||||
配置会写入 `config.yaml` 的 `robots` 段,也可在配置文件中直接编辑。**修改钉钉/飞书配置后必须重启,长连接才会生效。**
|
||||
|
||||
---
|
||||
|
||||
## 二、支持的平台(长连接)
|
||||
|
||||
| 平台 | 说明 |
|
||||
|------|------|
|
||||
| 钉钉 | 使用 Stream 长连接,程序主动连接钉钉接收消息 |
|
||||
| 飞书 | 使用长连接,程序主动连接飞书接收消息 |
|
||||
|
||||
下面第三节会按平台写清:在开放平台要做什么、要复制哪些字段、填到 CyberStrikeAI 的哪一栏。
|
||||
|
||||
---
|
||||
|
||||
## 三、各平台配置项与详细步骤
|
||||
|
||||
### 3.1 钉钉
|
||||
|
||||
**先搞清楚:两种钉钉机器人不一样**
|
||||
|
||||
| 类型 | 从哪里创建 | 能否做「用户发消息→机器人回复」 | 本程序是否支持 |
|
||||
|------|------------|----------------------------------|----------------|
|
||||
| **自定义机器人** | 钉钉群里:群设置 → 添加机器人 → 自定义(Webhook) | ❌ 不能,只能你往群里发消息 | ❌ 不支持 |
|
||||
| **企业内部应用机器人** | [钉钉开放平台](https://open.dingtalk.com) 创建应用并开通机器人 | ✅ 能 | ✅ 支持 |
|
||||
|
||||
如果你手里是「自定义机器人」的 Webhook 地址(`oapi.dingtalk.com/robot/send?access_token=xxx`)和加签密钥(`SEC...`),**不能直接填到本程序**,必须按下面步骤在开放平台创建「企业内部应用」并拿到 **Client ID**、**Client Secret**。
|
||||
|
||||
---
|
||||
|
||||
**钉钉配置完整步骤(按顺序做)**
|
||||
|
||||
1. **打开钉钉开放平台**
|
||||
浏览器访问 [https://open.dingtalk.com](https://open.dingtalk.com),用**企业管理员**账号登录。
|
||||
|
||||
2. **进入应用开发**
|
||||
左侧选 **应用开发** → **企业内部开发** → 点击 **创建应用**(或选择已有应用)。填写应用名称等基本信息后创建。
|
||||
|
||||
3. **拿到 Client ID 和 Client Secret**
|
||||
- 左侧点 **凭证与基础信息**(在「基础信息」下)。
|
||||
- 页面上有 **Client ID(原 AppKey)** 和 **Client Secret(原 AppSecret)**。
|
||||
- 点击复制,**不要手打**,注意:数字 **0** 和字母 **o**、数字 **1** 和字母 **l** 容易抄错(例如 `ding9gf9tiozuc504aer` 中间是数字 **504** 不是 5o4)。
|
||||
|
||||
4. **开通机器人并选 Stream 模式**
|
||||
- 左侧 **应用能力** → **机器人**。
|
||||
- 打开「机器人配置」开关。
|
||||
- 填写机器人名称、简介等(必填项按提示填)。
|
||||
- **关键**:消息接收方式要选 **「Stream 模式」**(流式接入)。若只有「HTTP 回调」或未选 Stream,本程序收不到消息。
|
||||
- 保存。
|
||||
|
||||
5. **权限与发布**
|
||||
- 左侧 **权限管理**:搜索「机器人」「消息」等,勾选**接收消息**、**发送消息**等机器人相关权限,并确认授权。
|
||||
- 左侧 **版本管理与发布**:若有未发布配置,点击 **发布新版本** / **上线**,否则修改不生效。
|
||||
|
||||
6. **填回 CyberStrikeAI**
|
||||
- 回到 CyberStrikeAI → 系统设置 → 机器人设置 → 钉钉。
|
||||
- 勾选「启用钉钉机器人」。
|
||||
- **Client ID (AppKey)** 粘贴第 3 步复制的 Client ID。
|
||||
- **Client Secret** 粘贴第 3 步复制的 Client Secret。
|
||||
- 点击 **应用配置**,然后**重启 CyberStrikeAI**。
|
||||
|
||||
---
|
||||
|
||||
**CyberStrikeAI 钉钉栏位对照**
|
||||
|
||||
| CyberStrikeAI 中填写项 | 在钉钉开放平台的来源 |
|
||||
|------------------------|------------------------|
|
||||
| 启用钉钉机器人 | 勾选即启用 |
|
||||
| Client ID (AppKey) | 凭证与基础信息 → **Client ID(原 AppKey)** |
|
||||
| Client Secret | 凭证与基础信息 → **Client Secret(原 AppSecret)** |
|
||||
|
||||
---
|
||||
|
||||
### 3.2 飞书 (Lark)
|
||||
|
||||
| 配置项 | 说明 |
|
||||
|--------|------|
|
||||
| 启用飞书机器人 | 勾选后启动飞书长连接 |
|
||||
| App ID | 飞书开放平台应用凭证中的 App ID |
|
||||
| App Secret | 飞书开放平台应用凭证中的 App Secret |
|
||||
| Verify Token | 事件订阅用(可选) |
|
||||
|
||||
**飞书配置简要步骤**:登录 [飞书开放平台](https://open.feishu.cn) → 创建企业自建应用 → 在「凭证与基础信息」中获取 **App ID**、**App Secret** → 在「应用能力」中开通**机器人**并启用相应权限 → 发布应用 → 将 App ID、App Secret 填到 CyberStrikeAI 机器人设置 → 保存并**重启应用**。
|
||||
|
||||
---
|
||||
|
||||
## 四、机器人命令
|
||||
|
||||
在钉钉/飞书中向机器人发送以下**文本命令**(仅支持文本):
|
||||
|
||||
| 命令 | 说明 |
|
||||
|------|------|
|
||||
| **帮助** | 显示命令帮助与说明 |
|
||||
| **列表** 或 **对话列表** | 列出所有对话的标题与对话 ID |
|
||||
| **切换 \<对话ID\>** 或 **继续 \<对话ID\>** | 指定对话 ID,后续消息在该对话中继续 |
|
||||
| **新对话** | 开启一个新对话,后续消息在新对话中 |
|
||||
| **清空** | 清空当前对话上下文(效果等同「新对话」) |
|
||||
| **当前** | 显示当前对话 ID 与标题 |
|
||||
| **停止** | 中断当前正在执行的任务 |
|
||||
| **角色** 或 **角色列表** | 列出所有可用角色(渗透测试、CTF、Web 应用扫描等) |
|
||||
| **角色 \<角色名\>** 或 **切换角色 \<角色名\>** | 切换当前使用的角色 |
|
||||
| **删除 \<对话ID\>** | 删除指定对话 |
|
||||
| **版本** | 显示当前 CyberStrikeAI 版本号 |
|
||||
|
||||
除以上命令外,**直接输入任意文字**会作为用户消息发给 AI,与 Web 端对话逻辑一致(渗透测试/安全分析等)。
|
||||
|
||||
---
|
||||
|
||||
## 五、如何使用(要 @ 机器人吗?)
|
||||
|
||||
- **单聊(推荐)**:在钉钉/飞书里**搜索并打开该机器人**,进入与机器人的**私聊**,直接输入「帮助」或任意文字即可,**不需要 @**。
|
||||
- **群聊**:若机器人被添加到群里,在群内只有 **@机器人** 后发送的消息才会被机器人收到并回复;不 @ 的群消息不会触发机器人。
|
||||
|
||||
总结:和机器人**单聊时直接发**;在**群里用时需要 @机器人** 再发内容。
|
||||
|
||||
---
|
||||
|
||||
## 六、推荐使用流程(避免漏步骤)
|
||||
|
||||
1. **在开放平台**:按第三节完成钉钉或飞书应用创建、凭证复制、机器人开通(钉钉务必选 **Stream 模式**)、权限与发布。
|
||||
2. **在 CyberStrikeAI**:系统设置 → 机器人设置 → 勾选对应平台,粘贴 Client ID/App ID、Client Secret/App Secret → 点击 **应用配置**。
|
||||
3. **重启 CyberStrikeAI 进程**(否则长连接不会建立)。
|
||||
4. **在手机钉钉/飞书**:找到该机器人(单聊直接发,群聊需 @机器人),发「帮助」或任意内容测试。
|
||||
|
||||
若发消息没反应,先看 **第九节排查** 和 **第十节常见弯路**。
|
||||
|
||||
---
|
||||
|
||||
## 七、配置文件示例
|
||||
|
||||
`config.yaml` 中机器人相关片段示例:
|
||||
|
||||
```yaml
|
||||
robots:
|
||||
dingtalk:
|
||||
enabled: true
|
||||
client_id: "your_dingtalk_app_key"
|
||||
client_secret: "your_dingtalk_app_secret"
|
||||
lark:
|
||||
enabled: true
|
||||
app_id: "your_lark_app_id"
|
||||
app_secret: "your_lark_app_secret"
|
||||
verify_token: ""
|
||||
```
|
||||
|
||||
修改后需**重启应用**,长连接在应用启动时建立。
|
||||
|
||||
---
|
||||
|
||||
## 八、如何验证是否可用(无需钉钉/飞书客户端)
|
||||
|
||||
在未安装钉钉或飞书时,可用**测试接口**验证机器人逻辑是否正常:
|
||||
|
||||
1. 先登录 CyberStrikeAI Web 端(保证有登录态)。
|
||||
2. 使用 curl 调用测试接口(需携带登录后的 Cookie):
|
||||
|
||||
```bash
|
||||
# 将 YOUR_COOKIE 替换为登录后获得的 Cookie(浏览器 F12 → 网络 → 任意请求 → 请求头中的 Cookie)
|
||||
curl -X POST "http://localhost:8080/api/robot/test" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Cookie: YOUR_COOKIE" \
|
||||
-d '{"platform":"dingtalk","user_id":"test_user","text":"帮助"}'
|
||||
```
|
||||
|
||||
若返回 JSON 中含有 `"reply":"【CyberStrikeAI 机器人命令】..."`,说明命令处理正常。可再试 `"text":"列表"`、`"text":"当前"` 等。
|
||||
|
||||
接口说明:`POST /api/robot/test`(需登录),请求体 `{"platform":"可选","user_id":"可选","text":"必填"}`,响应 `{"reply":"回复内容"}`。
|
||||
|
||||
---
|
||||
|
||||
## 九、钉钉发消息没反应时排查
|
||||
|
||||
按顺序检查:
|
||||
|
||||
0. **笔记本合盖睡眠 / 断网后**
|
||||
钉钉、飞书均使用长连接收消息,睡眠或断网后连接会断开。程序会**自动重连**(约 5 秒~60 秒内重试)。唤醒或恢复网络后稍等一会儿再发消息;若仍无反应,可重启 CyberStrikeAI 进程。
|
||||
|
||||
1. **Client ID / Client Secret 是否与开放平台完全一致**
|
||||
从「凭证与基础信息」里**复制粘贴**,不要手打。注意数字 **0** 与字母 **o**、数字 **1** 与字母 **l**(例如 `ding9gf9tiozuc504aer` 中间是 **504** 不是 5o4)。
|
||||
|
||||
2. **是否在保存配置后重启了应用**
|
||||
机器人长连接在**应用启动时**建立。在 Web 端点击「应用配置」只写入配置文件,**必须重启 CyberStrikeAI 进程**后钉钉连接才会生效。
|
||||
|
||||
3. **看程序日志**
|
||||
- 启动后应看到:`钉钉 Stream 正在连接…`、`钉钉 Stream 已启动(无需公网),等待收消息`。
|
||||
- 若出现 `钉钉 Stream 长连接退出` 且带错误信息,多为 **Client ID / Client Secret 错误**或**开放平台未开通流式接入**。
|
||||
- 在钉钉里发一条消息后,若有收到,应有日志:`钉钉收到消息`;若没有,说明钉钉未把消息推到本程序(回头检查开放平台「机器人」是否开通、是否选用 **Stream 模式**)。
|
||||
|
||||
4. **开放平台侧**
|
||||
应用需已**发布**;在「机器人」能力中需开启**流式接入(Stream)** 用于接收消息(仅 HTTP 回调不够);权限管理里需有机器人接收、发送消息等权限。
|
||||
|
||||
---
|
||||
|
||||
## 十、常见弯路(避免踩坑)
|
||||
|
||||
- **用错了机器人类型**:在钉钉**群里**添加的「自定义」机器人(Webhook + 加签)**不能**用来做对话,本程序只支持**开放平台「企业内部应用」**里的机器人。
|
||||
- **只保存没重启**:在 CyberStrikeAI 里改完机器人配置后必须**重启应用**,否则长连接不会建立。
|
||||
- **Client ID 抄错**:开放平台是 `504` 就填 `504`,不要填成 `5o4`;尽量用复制粘贴。
|
||||
- **钉钉只开了 HTTP 回调没开 Stream**:本程序通过 **Stream 长连接**收消息,开放平台里机器人的消息接收方式必须选 **Stream 模式**。
|
||||
- **应用没发布**:开放平台里修改了机器人或权限后,要在「版本管理与发布」里**发布新版本**,否则不生效。
|
||||
|
||||
---
|
||||
|
||||
## 十一、注意事项
|
||||
|
||||
- 钉钉、飞书均**仅处理文本消息**;其他类型(如图片、语音)会提示暂不支持或忽略。
|
||||
- 会话与 Web 端共用同一套对话数据:在机器人里创建的对话会在 Web 端「对话」列表中看到,反之亦然。
|
||||
- 机器人执行逻辑与 **`/api/agent-loop/stream`** 一致(含进度回调、过程详情写入数据库),仅不向客户端推送 SSE,最后将完整回复一次性发回钉钉/飞书/企业微信。
|
||||
@@ -0,0 +1,224 @@
|
||||
# CyberStrikeAI Robot / Chatbot Guide
|
||||
|
||||
[中文](robot.md)
|
||||
|
||||
This document explains how to chat with CyberStrikeAI from **DingTalk** and **Lark (Feishu)** using long-lived connections—no need to open a browser on the server. Following the steps below helps avoid common mistakes.
|
||||
|
||||
---
|
||||
|
||||
## 1. Where to configure in CyberStrikeAI
|
||||
|
||||
1. Log in to the CyberStrikeAI web UI.
|
||||
2. Open **System Settings** in the left sidebar.
|
||||
3. Click **Robot settings** (between “Basic” and “Security”).
|
||||
4. Enable the platform and fill in credentials (DingTalk: Client ID / Client Secret; Lark: App ID / App Secret).
|
||||
5. Click **Apply configuration** to save.
|
||||
6. **Restart the CyberStrikeAI process** (saving alone does not establish the connection).
|
||||
|
||||
Settings are written to the `robots` section of `config.yaml`; you can also edit the file directly. **After changing DingTalk or Lark config, you must restart for the long-lived connection to take effect.**
|
||||
|
||||
---
|
||||
|
||||
## 2. Supported platforms (long-lived connection)
|
||||
|
||||
| Platform | Description |
|
||||
|----------|-------------|
|
||||
| DingTalk | Stream long-lived connection; the app connects to DingTalk to receive messages |
|
||||
| Lark (Feishu) | Long-lived connection; the app connects to Lark to receive messages |
|
||||
|
||||
Section 3 below describes, per platform, what to do in the developer console and which fields to copy into CyberStrikeAI.
|
||||
|
||||
---
|
||||
|
||||
## 3. Configuration and step-by-step setup
|
||||
|
||||
### 3.1 DingTalk
|
||||
|
||||
**Important: two types of DingTalk bots**
|
||||
|
||||
| Type | Where it’s created | Can do “user sends message → bot replies”? | Supported here? |
|
||||
|------|-------------------|-------------------------------------------|------------------|
|
||||
| **Custom bot (Webhook)** | In a DingTalk group: Group settings → Add robot → Custom (Webhook) | No; you can only post to the group | No |
|
||||
| **Enterprise internal app bot** | [DingTalk Open Platform](https://open.dingtalk.com): create an app and enable the bot | Yes | Yes |
|
||||
|
||||
If you only have a **custom bot** Webhook URL (`oapi.dingtalk.com/robot/send?access_token=...`) and sign secret (`SEC...`), **do not** put them into CyberStrikeAI. You must create an **enterprise internal app** in the open platform and obtain **Client ID** and **Client Secret** as below.
|
||||
|
||||
---
|
||||
|
||||
**DingTalk setup (in order)**
|
||||
|
||||
1. **Open DingTalk Open Platform**
|
||||
Go to [https://open.dingtalk.com](https://open.dingtalk.com) and log in with an **enterprise admin** account.
|
||||
|
||||
2. **Create or select an app**
|
||||
In the left menu: **Application development** → **Enterprise internal development** → **Create application** (or choose an existing app). Fill in the app name and create.
|
||||
|
||||
3. **Get Client ID and Client Secret**
|
||||
- In the left menu open **Credentials and basic info** (under “Basic information”).
|
||||
- Copy **Client ID (formerly AppKey)** and **Client Secret (formerly AppSecret)**.
|
||||
- Use copy/paste; avoid typing by hand. Watch for **0** vs **o** and **1** vs **l** (e.g. `ding9gf9tiozuc504aer` has the digits **504**, not 5o4).
|
||||
|
||||
4. **Enable the bot and choose Stream mode**
|
||||
- Left menu: **Application capabilities** → **Robot**.
|
||||
- Turn on “Robot configuration”.
|
||||
- Fill in robot name, description, etc. as required.
|
||||
- **Critical**: set message reception to **“Stream mode”** (流式接入). If you only enable “HTTP callback” or do not select Stream, CyberStrikeAI will not receive messages.
|
||||
- Save.
|
||||
|
||||
5. **Permissions and release**
|
||||
- Left menu: **Permission management** — search for “robot”, “message”, etc., and enable **receive message**, **send message**, and other bot-related permissions; confirm.
|
||||
- Left menu: **Version management and release** — if there are unpublished changes, click **Release new version** / **Publish**; otherwise changes do not take effect.
|
||||
|
||||
6. **Fill in CyberStrikeAI**
|
||||
- In CyberStrikeAI: System settings → Robot settings → DingTalk.
|
||||
- Enable “Enable DingTalk robot”.
|
||||
- Paste the Client ID and Client Secret from step 3.
|
||||
- Click **Apply configuration**, then **restart CyberStrikeAI**.
|
||||
|
||||
---
|
||||
|
||||
**Field mapping (DingTalk)**
|
||||
|
||||
| Field in CyberStrikeAI | Source in DingTalk Open Platform |
|
||||
|------------------------|----------------------------------|
|
||||
| Enable DingTalk robot | Check to enable |
|
||||
| Client ID (AppKey) | Credentials and basic info → **Client ID (formerly AppKey)** |
|
||||
| Client Secret | Credentials and basic info → **Client Secret (formerly AppSecret)** |
|
||||
|
||||
---
|
||||
|
||||
### 3.2 Lark (Feishu)
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| Enable Lark robot | Check to start the Lark long-lived connection |
|
||||
| App ID | From Lark open platform app credentials |
|
||||
| App Secret | From Lark open platform app credentials |
|
||||
| Verify Token | Optional; for event subscription |
|
||||
|
||||
**Lark setup in short**: Log in to [Lark Open Platform](https://open.feishu.cn) → Create an enterprise app → In “Credentials and basic info” get **App ID** and **App Secret** → In “Application capabilities” enable **Robot** and the right permissions → Publish the app → Enter App ID and App Secret in CyberStrikeAI robot settings → Save and **restart** the app.
|
||||
|
||||
---
|
||||
|
||||
## 4. Bot commands
|
||||
|
||||
Send these **text commands** to the bot in DingTalk or Lark (text only):
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| **帮助** (help) | Show command help |
|
||||
| **列表** or **对话列表** (list) | List all conversation titles and IDs |
|
||||
| **切换 \<conversationID\>** or **继续 \<conversationID\>** | Continue in the given conversation |
|
||||
| **新对话** (new) | Start a new conversation |
|
||||
| **清空** (clear) | Clear current context (same effect as new conversation) |
|
||||
| **当前** (current) | Show current conversation ID and title |
|
||||
| **停止** (stop) | Abort the currently running task |
|
||||
| **角色** or **角色列表** (roles) | List all available roles (penetration testing, CTF, Web scan, etc.) |
|
||||
| **角色 \<roleName\>** or **切换角色 \<roleName\>** | Switch to the specified role |
|
||||
| **删除 \<conversationID\>** | Delete the specified conversation |
|
||||
| **版本** (version) | Show current CyberStrikeAI version |
|
||||
|
||||
Any other text is sent to the AI as a user message, same as in the web UI (e.g. penetration testing, security analysis).
|
||||
|
||||
---
|
||||
|
||||
## 5. How to use (do I need to @ the bot?)
|
||||
|
||||
- **Direct chat (recommended)**: In DingTalk or Lark, **search for the bot and open a direct chat**. Type “帮助” or any message; **no @ needed**.
|
||||
- **Group chat**: If the bot is in a group, only messages that **@ the bot** are received and answered; other group messages are ignored.
|
||||
|
||||
Summary: **Direct chat** — just send; **in a group** — @ the bot first, then send.
|
||||
|
||||
---
|
||||
|
||||
## 6. Recommended flow (so you don’t skip steps)
|
||||
|
||||
1. **In the open platform**: Complete app creation, copy credentials, enable the bot (DingTalk: **Stream mode**), set permissions, and publish (Section 3).
|
||||
2. **In CyberStrikeAI**: System settings → Robot settings → Enable the platform, paste Client ID/App ID and Client Secret/App Secret → **Apply configuration**.
|
||||
3. **Restart the CyberStrikeAI process** (otherwise the long-lived connection is not established).
|
||||
4. **On your phone**: Open DingTalk or Lark, find the bot (direct chat or @ in a group), send “帮助” or any message to test.
|
||||
|
||||
If the bot does not respond, see **Section 9 (troubleshooting)** and **Section 10 (common pitfalls)**.
|
||||
|
||||
---
|
||||
|
||||
## 7. Config file example
|
||||
|
||||
Example `robots` section in `config.yaml`:
|
||||
|
||||
```yaml
|
||||
robots:
|
||||
dingtalk:
|
||||
enabled: true
|
||||
client_id: "your_dingtalk_app_key"
|
||||
client_secret: "your_dingtalk_app_secret"
|
||||
lark:
|
||||
enabled: true
|
||||
app_id: "your_lark_app_id"
|
||||
app_secret: "your_lark_app_secret"
|
||||
verify_token: ""
|
||||
```
|
||||
|
||||
**Restart the app** after changes; the long-lived connection is created at startup.
|
||||
|
||||
---
|
||||
|
||||
## 8. Testing without DingTalk/Lark installed
|
||||
|
||||
You can verify bot logic with the **test API** (no DingTalk/Lark client needed):
|
||||
|
||||
1. Log in to the CyberStrikeAI web UI (so you have a session).
|
||||
2. Call the test endpoint with curl (include your session Cookie):
|
||||
|
||||
```bash
|
||||
# Replace YOUR_COOKIE with the Cookie from your browser (F12 → Network → any request → Request headers → Cookie)
|
||||
curl -X POST "http://localhost:8080/api/robot/test" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Cookie: YOUR_COOKIE" \
|
||||
-d '{"platform":"dingtalk","user_id":"test_user","text":"帮助"}'
|
||||
```
|
||||
|
||||
If the JSON response contains `"reply":"【CyberStrikeAI 机器人命令】..."`, command handling works. You can also try `"text":"列表"` or `"text":"当前"`.
|
||||
|
||||
API: `POST /api/robot/test` (requires login). Body: `{"platform":"optional","user_id":"optional","text":"required"}`. Response: `{"reply":"..."}`.
|
||||
|
||||
---
|
||||
|
||||
## 9. DingTalk: no response when sending messages
|
||||
|
||||
Check in this order:
|
||||
|
||||
0. **After laptop sleep or network drop**
|
||||
DingTalk and Lark both use long-lived connections; they break when the machine sleeps or the network drops. The app **auto-reconnects** (retries within about 5–60 seconds). After wake or network recovery, wait a moment before sending; if there is still no response, restart the CyberStrikeAI process.
|
||||
|
||||
1. **Client ID / Client Secret match the open platform exactly**
|
||||
Copy from “Credentials and basic info”; avoid typing. Watch **0** vs **o** and **1** vs **l** (e.g. `ding9gf9tiozuc504aer` has **504**, not 5o4).
|
||||
|
||||
2. **Did you restart after saving?**
|
||||
The long-lived connection is created at **startup**. “Apply configuration” only updates the config file; you **must restart the CyberStrikeAI process** for the DingTalk connection to start.
|
||||
|
||||
3. **Application logs**
|
||||
- On startup you should see: `钉钉 Stream 正在连接…`, `钉钉 Stream 已启动(无需公网),等待收消息`.
|
||||
- If you see `钉钉 Stream 长连接退出` with an error, it’s usually wrong **Client ID / Client Secret** or **Stream not enabled** in the open platform.
|
||||
- After sending a message in DingTalk, you should see `钉钉收到消息` in the logs; if not, the platform is not pushing to this app (check that the bot is enabled and **Stream mode** is selected).
|
||||
|
||||
4. **Open platform**
|
||||
The app must be **published**. Under “Robot” you must enable **Stream** for receiving messages (HTTP callback only is not enough). Permission management must include robot receive/send message permissions.
|
||||
|
||||
---
|
||||
|
||||
## 10. Common pitfalls
|
||||
|
||||
- **Wrong bot type**: The “Custom” bot added in a DingTalk **group** (Webhook + sign secret) **cannot** be used for two-way chat. Only the **enterprise internal app** bot from the open platform is supported.
|
||||
- **Saved but not restarted**: After changing robot settings in CyberStrikeAI you **must restart** the app, or the long-lived connection will not be established.
|
||||
- **Client ID typo**: If the platform shows `504`, use `504` (not `5o4`); prefer copy/paste.
|
||||
- **DingTalk: only HTTP callback, no Stream**: This app receives messages via **Stream**. In the open platform, message reception must be **Stream mode**.
|
||||
- **App not published**: After changing the bot or permissions in the open platform, **publish a new version** under “Version management and release”, or changes won’t apply.
|
||||
|
||||
---
|
||||
|
||||
## 11. Notes
|
||||
|
||||
- DingTalk and Lark: **text messages only**; other types (e.g. image, voice) are not supported and may be ignored.
|
||||
- Conversations are shared with the web UI: conversations created from the bot appear in the web “Conversations” list and vice versa.
|
||||
- Bot execution uses the same logic as **`/api/agent-loop/stream`** (progress callbacks, process details stored in the DB); only the final reply is sent back to DingTalk/Lark in one message (no SSE to the client).
|
||||
@@ -1,16 +1,21 @@
|
||||
module cyberstrike-ai
|
||||
|
||||
go 1.23.0
|
||||
go 1.24.0
|
||||
|
||||
toolchain go1.24.4
|
||||
|
||||
require (
|
||||
github.com/creack/pty v1.1.24
|
||||
github.com/gin-gonic/gin v1.9.1
|
||||
github.com/google/uuid v1.5.0
|
||||
github.com/gorilla/websocket v1.5.0
|
||||
github.com/larksuite/oapi-sdk-go/v3 v3.4.22
|
||||
github.com/mattn/go-sqlite3 v1.14.18
|
||||
github.com/modelcontextprotocol/go-sdk v1.2.0
|
||||
github.com/open-dingtalk/dingtalk-stream-sdk-go v0.9.1
|
||||
github.com/pkoukk/tiktoken-go v0.1.8
|
||||
go.uber.org/zap v1.26.0
|
||||
golang.org/x/time v0.14.0
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
)
|
||||
|
||||
@@ -24,6 +29,7 @@ require (
|
||||
github.com/go-playground/universal-translator v0.18.1 // indirect
|
||||
github.com/go-playground/validator/v10 v10.14.0 // indirect
|
||||
github.com/goccy/go-json v0.10.2 // indirect
|
||||
github.com/gogo/protobuf v1.3.2 // indirect
|
||||
github.com/google/jsonschema-go v0.3.0 // indirect
|
||||
github.com/json-iterator/go v1.1.12 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.2.4 // indirect
|
||||
@@ -44,3 +50,7 @@ require (
|
||||
golang.org/x/text v0.13.0 // indirect
|
||||
google.golang.org/protobuf v1.30.0 // indirect
|
||||
)
|
||||
|
||||
// 修复钉钉 Stream SDK 在长连接断开(熄屏/网络中断)后 "panic: send on closed channel" 问题
|
||||
// 详见: https://github.com/open-dingtalk/dingtalk-stream-sdk-go/issues/28
|
||||
replace github.com/open-dingtalk/dingtalk-stream-sdk-go => github.com/uouuou/dingtalk-stream-sdk-go v0.0.0-20250626025113-079132acc406
|
||||
|
||||
@@ -4,6 +4,8 @@ github.com/bytedance/sonic v1.9.1/go.mod h1:i736AoUSYt75HyZLoJW9ERYxcy6eaN6h4BZX
|
||||
github.com/chenzhuoyu/base64x v0.0.0-20211019084208-fb5309c8db06/go.mod h1:DH46F32mSOjUmXrMHnKwZdA8wcEefY7UVqBKYGjpdQY=
|
||||
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 h1:qSGYFH7+jGhDF8vLC+iwCD4WpbV1EBDSzWkJODFLams=
|
||||
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311/go.mod h1:b583jCggY9gE99b6G5LEC39OIiVsWj+R97kbl5odCEk=
|
||||
github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s=
|
||||
github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
@@ -25,6 +27,8 @@ github.com/go-playground/validator/v10 v10.14.0 h1:vgvQWe3XCz3gIeFDm/HnTIbj6UGmg
|
||||
github.com/go-playground/validator/v10 v10.14.0/go.mod h1:9iXMNT7sEkjXb0I+enO7QXmzG6QCsPWY4zveKFVRSyU=
|
||||
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
|
||||
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
|
||||
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
|
||||
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
|
||||
github.com/golang-jwt/jwt/v5 v5.2.2 h1:Rl4B7itRWVtYIHFrSNd7vhTiz9UpLdi6gZhZ3wEeDy8=
|
||||
github.com/golang-jwt/jwt/v5 v5.2.2/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
|
||||
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
|
||||
@@ -36,11 +40,17 @@ github.com/google/jsonschema-go v0.3.0 h1:6AH2TxVNtk3IlvkkhjrtbUc4S8AvO0Xii0DxIy
|
||||
github.com/google/jsonschema-go v0.3.0/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE=
|
||||
github.com/google/uuid v1.5.0 h1:1p67kYwdtXjb0gL0BPiP1Av9wiZPo5A8z2cWkTZ+eyU=
|
||||
github.com/google/uuid v1.5.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/gorilla/websocket v1.5.0 h1:PPwGk2jz7EePpoHN/+ClbZu8SPxiqlu12wZP/3sWmnc=
|
||||
github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
|
||||
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
|
||||
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
|
||||
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
|
||||
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
|
||||
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
|
||||
github.com/klauspost/cpuid/v2 v2.2.4 h1:acbojRNwl3o09bUq+yDCtZFc1aiwaAAxtcn8YkZXnvk=
|
||||
github.com/klauspost/cpuid/v2 v2.2.4/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY=
|
||||
github.com/larksuite/oapi-sdk-go/v3 v3.4.22 h1:57daKuslQPX9X3hC2idc5bu8bl2krfsBGWGJ6b5FlD8=
|
||||
github.com/larksuite/oapi-sdk-go/v3 v3.4.22/go.mod h1:ZEplY+kwuIrj/nqw5uSCINNATcH3KdxSN7y+UxYY5fI=
|
||||
github.com/leodido/go-urn v1.2.4 h1:XlAE/cm/ms7TE/VMVoduSpNBoyc2dOxHs5MZSwAN63Q=
|
||||
github.com/leodido/go-urn v1.2.4/go.mod h1:7ZrI8mTSeBSHl/UaRyKQW1qZeMgak41ANeCNaVckg+4=
|
||||
github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA=
|
||||
@@ -75,8 +85,12 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS
|
||||
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
|
||||
github.com/ugorji/go/codec v1.2.11 h1:BMaWp1Bb6fHwEtbplGBGJ498wD+LKlNSl25MjdZY4dU=
|
||||
github.com/ugorji/go/codec v1.2.11/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
|
||||
github.com/uouuou/dingtalk-stream-sdk-go v0.0.0-20250626025113-079132acc406 h1:b72HNsEnmTRn7vhWGOfbWHAkA5RbRCk0Pbc56V2WAuY=
|
||||
github.com/uouuou/dingtalk-stream-sdk-go v0.0.0-20250626025113-079132acc406/go.mod h1:ln3IqPYYocZbYvl9TAOrG/cxGR9xcn4pnZRLdCTEGEU=
|
||||
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
|
||||
github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
|
||||
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
|
||||
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
|
||||
go.uber.org/goleak v1.2.0 h1:xqgm/S+aQvhWFTtR0XK3Jvg7z8kGV8P4X14IzwN3Eqk=
|
||||
go.uber.org/goleak v1.2.0/go.mod h1:XJYK+MuIchqpmGmUSAzotztawfKvYLUIgg7guXrwVUo=
|
||||
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
|
||||
@@ -86,21 +100,47 @@ go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
|
||||
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
|
||||
golang.org/x/arch v0.3.0 h1:02VY4/ZcO/gBOH6PUaoiptASxtXU10jazRCP865E97k=
|
||||
golang.org/x/arch v0.3.0/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
|
||||
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
|
||||
golang.org/x/crypto v0.14.0 h1:wBqGXzWJW6m1XrIKlAH0Hs1JJ7+9KBwnIO8v66Q9cHc=
|
||||
golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4=
|
||||
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
|
||||
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
|
||||
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
|
||||
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
|
||||
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
|
||||
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
|
||||
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
|
||||
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
|
||||
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE=
|
||||
golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
|
||||
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
|
||||
golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=
|
||||
golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
|
||||
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
|
||||
golang.org/x/tools v0.34.0 h1:qIpSLOxeCYGg9TrcJokLBG4KFA6d795g0xkBkiESGlo=
|
||||
golang.org/x/tools v0.34.0/go.mod h1:pAP9OwEaY1CAW3HOmg3hLZC5Z0CCmzjAF2UQMSqNARg=
|
||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
|
||||
google.golang.org/protobuf v1.30.0 h1:kPPoIgf3TsEvrm0PFe15JQ+570QVxYzEvvHqChK+cng=
|
||||
google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 832 KiB |
+100
-2
@@ -7,6 +7,7 @@ import (
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"cyberstrike-ai/internal/agent"
|
||||
@@ -14,6 +15,7 @@ import (
|
||||
"cyberstrike-ai/internal/database"
|
||||
"cyberstrike-ai/internal/handler"
|
||||
"cyberstrike-ai/internal/knowledge"
|
||||
"cyberstrike-ai/internal/robot"
|
||||
"cyberstrike-ai/internal/logger"
|
||||
"cyberstrike-ai/internal/mcp"
|
||||
"cyberstrike-ai/internal/mcp/builtin"
|
||||
@@ -43,6 +45,10 @@ type App struct {
|
||||
knowledgeIndexer *knowledge.Indexer // 知识库索引器(用于动态初始化)
|
||||
knowledgeHandler *handler.KnowledgeHandler // 知识库处理器(用于动态初始化)
|
||||
agentHandler *handler.AgentHandler // Agent处理器(用于更新知识库管理器)
|
||||
robotHandler *handler.RobotHandler // 机器人处理器(钉钉/飞书/企业微信)
|
||||
robotMu sync.Mutex // 保护钉钉/飞书长连接的 cancel
|
||||
dingCancel context.CancelFunc // 钉钉 Stream 取消函数,用于配置变更时重启
|
||||
larkCancel context.CancelFunc // 飞书长连接取消函数,用于配置变更时重启
|
||||
}
|
||||
|
||||
// New 创建新应用
|
||||
@@ -192,7 +198,7 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
|
||||
knowledgeRetriever = knowledge.NewRetriever(knowledgeDB, embedder, retrievalConfig, log.Logger)
|
||||
|
||||
// 创建索引器
|
||||
knowledgeIndexer = knowledge.NewIndexer(knowledgeDB, embedder, log.Logger)
|
||||
knowledgeIndexer = knowledge.NewIndexer(knowledgeDB, embedder, log.Logger, &cfg.Knowledge.Indexing)
|
||||
|
||||
// 注册知识检索工具到MCP服务器
|
||||
knowledge.RegisterKnowledgeTool(mcpServer, knowledgeRetriever, knowledgeManager, log.Logger)
|
||||
@@ -318,12 +324,15 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
|
||||
roleHandler := handler.NewRoleHandler(cfg, configPath, log.Logger)
|
||||
roleHandler.SetSkillsManager(skillsManager) // 设置Skills管理器到RoleHandler
|
||||
skillsHandler := handler.NewSkillsHandler(skillsManager, cfg, configPath, log.Logger)
|
||||
fofaHandler := handler.NewFofaHandler(cfg, log.Logger)
|
||||
terminalHandler := handler.NewTerminalHandler(log.Logger)
|
||||
if db != nil {
|
||||
skillsHandler.SetDB(db) // 设置数据库连接以便获取调用统计
|
||||
}
|
||||
|
||||
// 创建OpenAPI处理器
|
||||
conversationHandler := handler.NewConversationHandler(db, log.Logger)
|
||||
robotHandler := handler.NewRobotHandler(cfg, db, agentHandler, log.Logger)
|
||||
openAPIHandler := handler.NewOpenAPIHandler(db, log.Logger, resultStorage, conversationHandler, agentHandler)
|
||||
|
||||
// 创建 App 实例(部分字段稍后填充)
|
||||
@@ -343,7 +352,10 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
|
||||
knowledgeIndexer: knowledgeIndexer,
|
||||
knowledgeHandler: knowledgeHandler,
|
||||
agentHandler: agentHandler,
|
||||
robotHandler: robotHandler,
|
||||
}
|
||||
// 飞书/钉钉长连接(无需公网),启用时在后台启动;后续前端应用配置时会通过 RestartRobotConnections 重启
|
||||
app.startRobotConnections()
|
||||
|
||||
// 设置漏洞工具注册器(内置工具,必须设置)
|
||||
vulnerabilityRegistrar := func() error {
|
||||
@@ -400,6 +412,9 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
|
||||
configHandler.SetRetrieverUpdater(knowledgeRetriever)
|
||||
}
|
||||
|
||||
// 设置机器人连接重启器,前端应用配置后无需重启服务即可使钉钉/飞书新配置生效
|
||||
configHandler.SetRobotRestarter(app)
|
||||
|
||||
// 设置路由(使用 App 实例以便动态获取 handler)
|
||||
setupRoutes(
|
||||
router,
|
||||
@@ -407,6 +422,7 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
|
||||
agentHandler,
|
||||
monitorHandler,
|
||||
conversationHandler,
|
||||
robotHandler,
|
||||
groupHandler,
|
||||
configHandler,
|
||||
externalMCPHandler,
|
||||
@@ -415,6 +431,8 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
|
||||
vulnerabilityHandler,
|
||||
roleHandler,
|
||||
skillsHandler,
|
||||
fofaHandler,
|
||||
terminalHandler,
|
||||
mcpServer,
|
||||
authManager,
|
||||
openAPIHandler,
|
||||
@@ -450,6 +468,18 @@ func (a *App) Run() error {
|
||||
|
||||
// Shutdown 关闭应用
|
||||
func (a *App) Shutdown() {
|
||||
// 停止钉钉/飞书长连接
|
||||
a.robotMu.Lock()
|
||||
if a.dingCancel != nil {
|
||||
a.dingCancel()
|
||||
a.dingCancel = nil
|
||||
}
|
||||
if a.larkCancel != nil {
|
||||
a.larkCancel()
|
||||
a.larkCancel = nil
|
||||
}
|
||||
a.robotMu.Unlock()
|
||||
|
||||
// 停止所有外部MCP客户端
|
||||
if a.externalMCPMgr != nil {
|
||||
a.externalMCPMgr.StopAll()
|
||||
@@ -463,6 +493,40 @@ func (a *App) Shutdown() {
|
||||
}
|
||||
}
|
||||
|
||||
// startRobotConnections 根据当前配置启动钉钉/飞书长连接(不先关闭已有连接,仅用于首次启动)
|
||||
func (a *App) startRobotConnections() {
|
||||
a.robotMu.Lock()
|
||||
defer a.robotMu.Unlock()
|
||||
cfg := a.config
|
||||
if cfg.Robots.Lark.Enabled && cfg.Robots.Lark.AppID != "" && cfg.Robots.Lark.AppSecret != "" {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
a.larkCancel = cancel
|
||||
go robot.StartLark(ctx, cfg.Robots.Lark, a.robotHandler, a.logger.Logger)
|
||||
}
|
||||
if cfg.Robots.Dingtalk.Enabled && cfg.Robots.Dingtalk.ClientID != "" && cfg.Robots.Dingtalk.ClientSecret != "" {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
a.dingCancel = cancel
|
||||
go robot.StartDing(ctx, cfg.Robots.Dingtalk, a.robotHandler, a.logger.Logger)
|
||||
}
|
||||
}
|
||||
|
||||
// RestartRobotConnections 重启钉钉/飞书长连接,使前端应用配置后立即生效(实现 handler.RobotRestarter)
|
||||
func (a *App) RestartRobotConnections() {
|
||||
a.robotMu.Lock()
|
||||
if a.dingCancel != nil {
|
||||
a.dingCancel()
|
||||
a.dingCancel = nil
|
||||
}
|
||||
if a.larkCancel != nil {
|
||||
a.larkCancel()
|
||||
a.larkCancel = nil
|
||||
}
|
||||
a.robotMu.Unlock()
|
||||
// 给旧 goroutine 一点时间退出
|
||||
time.Sleep(200 * time.Millisecond)
|
||||
a.startRobotConnections()
|
||||
}
|
||||
|
||||
// setupRoutes 设置路由
|
||||
func setupRoutes(
|
||||
router *gin.Engine,
|
||||
@@ -470,6 +534,7 @@ func setupRoutes(
|
||||
agentHandler *handler.AgentHandler,
|
||||
monitorHandler *handler.MonitorHandler,
|
||||
conversationHandler *handler.ConversationHandler,
|
||||
robotHandler *handler.RobotHandler,
|
||||
groupHandler *handler.GroupHandler,
|
||||
configHandler *handler.ConfigHandler,
|
||||
externalMCPHandler *handler.ExternalMCPHandler,
|
||||
@@ -478,6 +543,8 @@ func setupRoutes(
|
||||
vulnerabilityHandler *handler.VulnerabilityHandler,
|
||||
roleHandler *handler.RoleHandler,
|
||||
skillsHandler *handler.SkillsHandler,
|
||||
fofaHandler *handler.FofaHandler,
|
||||
terminalHandler *handler.TerminalHandler,
|
||||
mcpServer *mcp.Server,
|
||||
authManager *security.AuthManager,
|
||||
openAPIHandler *handler.OpenAPIHandler,
|
||||
@@ -494,9 +561,18 @@ func setupRoutes(
|
||||
authRoutes.GET("/validate", security.AuthMiddleware(authManager), authHandler.Validate)
|
||||
}
|
||||
|
||||
// 机器人回调(无需登录,供企业微信/钉钉/飞书服务器调用)
|
||||
api.GET("/robot/wecom", robotHandler.HandleWecomGET)
|
||||
api.POST("/robot/wecom", robotHandler.HandleWecomPOST)
|
||||
api.POST("/robot/dingtalk", robotHandler.HandleDingtalkPOST)
|
||||
api.POST("/robot/lark", robotHandler.HandleLarkPOST)
|
||||
|
||||
protected := api.Group("")
|
||||
protected.Use(security.AuthMiddleware(authManager))
|
||||
{
|
||||
// 机器人测试(需登录):POST /api/robot/test,body: {"platform":"dingtalk","user_id":"test","text":"帮助"},用于验证机器人逻辑
|
||||
protected.POST("/robot/test", robotHandler.HandleRobotTest)
|
||||
|
||||
// Agent Loop
|
||||
protected.POST("/agent-loop", agentHandler.AgentLoop)
|
||||
// Agent Loop 流式输出
|
||||
@@ -506,6 +582,11 @@ func setupRoutes(
|
||||
protected.GET("/agent-loop/tasks", agentHandler.ListAgentTasks)
|
||||
protected.GET("/agent-loop/tasks/completed", agentHandler.ListCompletedTasks)
|
||||
|
||||
// 信息收集 - FOFA 查询(后端代理)
|
||||
protected.POST("/fofa/search", fofaHandler.Search)
|
||||
// 信息收集 - 自然语言解析为 FOFA 语法(需人工确认后再查询)
|
||||
protected.POST("/fofa/parse", fofaHandler.ParseNaturalLanguage)
|
||||
|
||||
// 批量任务管理
|
||||
protected.POST("/batch-tasks", agentHandler.CreateBatchQueue)
|
||||
protected.GET("/batch-tasks", agentHandler.ListBatchQueues)
|
||||
@@ -550,6 +631,11 @@ func setupRoutes(
|
||||
protected.PUT("/config", configHandler.UpdateConfig)
|
||||
protected.POST("/config/apply", configHandler.ApplyConfig)
|
||||
|
||||
// 系统设置 - 终端(执行命令,提高运维效率)
|
||||
protected.POST("/terminal/run", terminalHandler.RunCommand)
|
||||
protected.POST("/terminal/run/stream", terminalHandler.RunCommandStream)
|
||||
protected.GET("/terminal/ws", terminalHandler.RunCommandWS)
|
||||
|
||||
// 外部MCP管理
|
||||
protected.GET("/external-mcp", externalMCPHandler.GetExternalMCPs)
|
||||
protected.GET("/external-mcp/stats", externalMCPHandler.GetExternalMCPStats)
|
||||
@@ -694,6 +780,18 @@ func setupRoutes(
|
||||
}
|
||||
app.knowledgeHandler.Search(c)
|
||||
})
|
||||
knowledgeRoutes.GET("/stats", func(c *gin.Context) {
|
||||
if app.knowledgeHandler == nil {
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"enabled": false,
|
||||
"total_categories": 0,
|
||||
"total_items": 0,
|
||||
"message": "知识库功能未启用,请前往系统设置启用知识检索功能",
|
||||
})
|
||||
return
|
||||
}
|
||||
app.knowledgeHandler.GetStats(c)
|
||||
})
|
||||
}
|
||||
|
||||
// 漏洞管理
|
||||
@@ -1004,7 +1102,7 @@ func initializeKnowledge(
|
||||
knowledgeRetriever := knowledge.NewRetriever(knowledgeDB, embedder, retrievalConfig, logger)
|
||||
|
||||
// 创建索引器
|
||||
knowledgeIndexer := knowledge.NewIndexer(knowledgeDB, embedder, logger)
|
||||
knowledgeIndexer := knowledge.NewIndexer(knowledgeDB, embedder, logger, &cfg.Knowledge.Indexing)
|
||||
|
||||
// 注册知识检索工具到MCP服务器
|
||||
knowledge.RegisterKnowledgeTool(mcpServer, knowledgeRetriever, knowledgeManager, logger)
|
||||
|
||||
@@ -18,17 +18,51 @@ type Config struct {
|
||||
Log LogConfig `yaml:"log"`
|
||||
MCP MCPConfig `yaml:"mcp"`
|
||||
OpenAI OpenAIConfig `yaml:"openai"`
|
||||
FOFA FofaConfig `yaml:"fofa,omitempty" json:"fofa,omitempty"`
|
||||
Agent AgentConfig `yaml:"agent"`
|
||||
Security SecurityConfig `yaml:"security"`
|
||||
Database DatabaseConfig `yaml:"database"`
|
||||
Auth AuthConfig `yaml:"auth"`
|
||||
ExternalMCP ExternalMCPConfig `yaml:"external_mcp,omitempty"`
|
||||
Knowledge KnowledgeConfig `yaml:"knowledge,omitempty"`
|
||||
Robots RobotsConfig `yaml:"robots,omitempty" json:"robots,omitempty"` // 企业微信/钉钉/飞书等机器人配置
|
||||
RolesDir string `yaml:"roles_dir,omitempty" json:"roles_dir,omitempty"` // 角色配置文件目录(新方式)
|
||||
Roles map[string]RoleConfig `yaml:"roles,omitempty" json:"roles,omitempty"` // 向后兼容:支持在主配置文件中定义角色
|
||||
SkillsDir string `yaml:"skills_dir,omitempty" json:"skills_dir,omitempty"` // Skills配置文件目录
|
||||
}
|
||||
|
||||
// RobotsConfig 机器人配置(企业微信、钉钉、飞书等)
|
||||
type RobotsConfig struct {
|
||||
Wecom RobotWecomConfig `yaml:"wecom,omitempty" json:"wecom,omitempty"` // 企业微信
|
||||
Dingtalk RobotDingtalkConfig `yaml:"dingtalk,omitempty" json:"dingtalk,omitempty"` // 钉钉
|
||||
Lark RobotLarkConfig `yaml:"lark,omitempty" json:"lark,omitempty"` // 飞书
|
||||
}
|
||||
|
||||
// RobotWecomConfig 企业微信机器人配置
|
||||
type RobotWecomConfig struct {
|
||||
Enabled bool `yaml:"enabled" json:"enabled"`
|
||||
Token string `yaml:"token" json:"token"` // 回调 URL 校验 Token
|
||||
EncodingAESKey string `yaml:"encoding_aes_key" json:"encoding_aes_key"` // EncodingAESKey
|
||||
CorpID string `yaml:"corp_id" json:"corp_id"` // 企业 ID
|
||||
Secret string `yaml:"secret" json:"secret"` // 应用 Secret
|
||||
AgentID int64 `yaml:"agent_id" json:"agent_id"` // 应用 AgentId
|
||||
}
|
||||
|
||||
// RobotDingtalkConfig 钉钉机器人配置
|
||||
type RobotDingtalkConfig struct {
|
||||
Enabled bool `yaml:"enabled" json:"enabled"`
|
||||
ClientID string `yaml:"client_id" json:"client_id"` // 应用 Key (AppKey)
|
||||
ClientSecret string `yaml:"client_secret" json:"client_secret"` // 应用 Secret
|
||||
}
|
||||
|
||||
// RobotLarkConfig 飞书机器人配置
|
||||
type RobotLarkConfig struct {
|
||||
Enabled bool `yaml:"enabled" json:"enabled"`
|
||||
AppID string `yaml:"app_id" json:"app_id"` // 应用 App ID
|
||||
AppSecret string `yaml:"app_secret" json:"app_secret"` // 应用 App Secret
|
||||
VerifyToken string `yaml:"verify_token" json:"verify_token"` // 事件订阅 Verification Token(可选)
|
||||
}
|
||||
|
||||
type ServerConfig struct {
|
||||
Host string `yaml:"host"`
|
||||
Port int `yaml:"port"`
|
||||
@@ -52,6 +86,13 @@ type OpenAIConfig struct {
|
||||
MaxTotalTokens int `yaml:"max_total_tokens,omitempty" json:"max_total_tokens,omitempty"`
|
||||
}
|
||||
|
||||
type FofaConfig struct {
|
||||
// Email 为 FOFA 账号邮箱;APIKey 为 FOFA API Key(建议使用只读权限的 Key)
|
||||
Email string `yaml:"email,omitempty" json:"email,omitempty"`
|
||||
APIKey string `yaml:"api_key,omitempty" json:"api_key,omitempty"`
|
||||
BaseURL string `yaml:"base_url,omitempty" json:"base_url,omitempty"` // 默认 https://fofa.info/api/v1/search/all
|
||||
}
|
||||
|
||||
type SecurityConfig struct {
|
||||
Tools []ToolConfig `yaml:"tools,omitempty"` // 向后兼容:支持在主配置文件中定义工具
|
||||
ToolsDir string `yaml:"tools_dir,omitempty"` // 工具配置文件目录(新方式)
|
||||
@@ -554,6 +595,26 @@ type KnowledgeConfig struct {
|
||||
BasePath string `yaml:"base_path" json:"base_path"` // 知识库路径
|
||||
Embedding EmbeddingConfig `yaml:"embedding" json:"embedding"`
|
||||
Retrieval RetrievalConfig `yaml:"retrieval" json:"retrieval"`
|
||||
Indexing IndexingConfig `yaml:"indexing,omitempty" json:"indexing,omitempty"` // 索引构建配置
|
||||
}
|
||||
|
||||
// IndexingConfig 索引构建配置(用于控制知识库索引构建时的行为)
|
||||
type IndexingConfig struct {
|
||||
// 分块配置
|
||||
ChunkSize int `yaml:"chunk_size,omitempty" json:"chunk_size,omitempty"` // 每个块的最大 token 数(估算),默认 512
|
||||
ChunkOverlap int `yaml:"chunk_overlap,omitempty" json:"chunk_overlap,omitempty"` // 块之间的重叠 token 数,默认 50
|
||||
MaxChunksPerItem int `yaml:"max_chunks_per_item,omitempty" json:"max_chunks_per_item,omitempty"` // 单个知识项的最大块数量,0 表示不限制
|
||||
|
||||
// 速率限制配置(用于避免 API 速率限制)
|
||||
RateLimitDelayMs int `yaml:"rate_limit_delay_ms,omitempty" json:"rate_limit_delay_ms,omitempty"` // 请求间隔时间(毫秒),0 表示不使用固定延迟
|
||||
MaxRPM int `yaml:"max_rpm,omitempty" json:"max_rpm,omitempty"` // 每分钟最大请求数,0 表示不限制
|
||||
|
||||
// 重试配置(用于处理临时错误)
|
||||
MaxRetries int `yaml:"max_retries,omitempty" json:"max_retries,omitempty"` // 最大重试次数,默认 3
|
||||
RetryDelayMs int `yaml:"retry_delay_ms,omitempty" json:"retry_delay_ms,omitempty"` // 重试间隔(毫秒),默认 1000
|
||||
|
||||
// 批处理配置(用于批量嵌入,当前未使用,保留扩展)
|
||||
BatchSize int `yaml:"batch_size,omitempty" json:"batch_size,omitempty"` // 批量处理大小,0 表示逐个处理
|
||||
}
|
||||
|
||||
// EmbeddingConfig 嵌入配置
|
||||
|
||||
+259
-8
@@ -2,10 +2,14 @@ package handler
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -108,11 +112,132 @@ func (h *AgentHandler) SetSkillsManager(manager *skills.Manager) {
|
||||
h.skillsManager = manager
|
||||
}
|
||||
|
||||
// ChatAttachment 聊天附件(用户上传的文件)
|
||||
type ChatAttachment struct {
|
||||
FileName string `json:"fileName"` // 文件名
|
||||
Content string `json:"content"` // 文本内容或 base64(由 MimeType 决定是否解码)
|
||||
MimeType string `json:"mimeType,omitempty"`
|
||||
}
|
||||
|
||||
// ChatRequest 聊天请求
|
||||
type ChatRequest struct {
|
||||
Message string `json:"message" binding:"required"`
|
||||
ConversationID string `json:"conversationId,omitempty"`
|
||||
Role string `json:"role,omitempty"` // 角色名称
|
||||
Message string `json:"message" binding:"required"`
|
||||
ConversationID string `json:"conversationId,omitempty"`
|
||||
Role string `json:"role,omitempty"` // 角色名称
|
||||
Attachments []ChatAttachment `json:"attachments,omitempty"`
|
||||
}
|
||||
|
||||
const (
|
||||
maxAttachments = 10
|
||||
chatUploadsDirName = "chat_uploads" // 对话附件保存的根目录(相对当前工作目录)
|
||||
)
|
||||
|
||||
// saveAttachmentsToDateAndConversationDir 将附件保存到 chat_uploads/YYYY-MM-DD/{conversationID}/,返回每个文件的保存路径(与 attachments 顺序一致)
|
||||
// conversationID 为空时使用 "_new" 作为目录名(新对话尚未有 ID)
|
||||
func saveAttachmentsToDateAndConversationDir(attachments []ChatAttachment, conversationID string, logger *zap.Logger) (savedPaths []string, err error) {
|
||||
if len(attachments) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
cwd, err := os.Getwd()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("获取当前工作目录失败: %w", err)
|
||||
}
|
||||
dateDir := filepath.Join(cwd, chatUploadsDirName, time.Now().Format("2006-01-02"))
|
||||
convDirName := strings.TrimSpace(conversationID)
|
||||
if convDirName == "" {
|
||||
convDirName = "_new"
|
||||
} else {
|
||||
convDirName = strings.ReplaceAll(convDirName, string(filepath.Separator), "_")
|
||||
}
|
||||
targetDir := filepath.Join(dateDir, convDirName)
|
||||
if err = os.MkdirAll(targetDir, 0755); err != nil {
|
||||
return nil, fmt.Errorf("创建上传目录失败: %w", err)
|
||||
}
|
||||
savedPaths = make([]string, 0, len(attachments))
|
||||
for i, a := range attachments {
|
||||
raw, decErr := attachmentContentToBytes(a)
|
||||
if decErr != nil {
|
||||
return nil, fmt.Errorf("附件 %s 解码失败: %w", a.FileName, decErr)
|
||||
}
|
||||
baseName := filepath.Base(a.FileName)
|
||||
if baseName == "" || baseName == "." {
|
||||
baseName = "file"
|
||||
}
|
||||
baseName = strings.ReplaceAll(baseName, string(filepath.Separator), "_")
|
||||
ext := filepath.Ext(baseName)
|
||||
nameNoExt := strings.TrimSuffix(baseName, ext)
|
||||
suffix := fmt.Sprintf("_%s_%s", time.Now().Format("150405"), shortRand(6))
|
||||
var unique string
|
||||
if ext != "" {
|
||||
unique = nameNoExt + suffix + ext
|
||||
} else {
|
||||
unique = baseName + suffix
|
||||
}
|
||||
fullPath := filepath.Join(targetDir, unique)
|
||||
if err = os.WriteFile(fullPath, raw, 0644); err != nil {
|
||||
return nil, fmt.Errorf("写入文件 %s 失败: %w", a.FileName, err)
|
||||
}
|
||||
absPath, _ := filepath.Abs(fullPath)
|
||||
savedPaths = append(savedPaths, absPath)
|
||||
if logger != nil {
|
||||
logger.Debug("对话附件已保存", zap.Int("index", i+1), zap.String("fileName", a.FileName), zap.String("path", absPath))
|
||||
}
|
||||
}
|
||||
return savedPaths, nil
|
||||
}
|
||||
|
||||
func shortRand(n int) string {
|
||||
const letters = "0123456789abcdef"
|
||||
b := make([]byte, n)
|
||||
_, _ = rand.Read(b)
|
||||
for i := range b {
|
||||
b[i] = letters[int(b[i])%len(letters)]
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
||||
func attachmentContentToBytes(a ChatAttachment) ([]byte, error) {
|
||||
content := a.Content
|
||||
if decoded, err := base64.StdEncoding.DecodeString(content); err == nil && len(decoded) > 0 {
|
||||
return decoded, nil
|
||||
}
|
||||
return []byte(content), nil
|
||||
}
|
||||
|
||||
// userMessageContentForStorage 返回要存入数据库的用户消息内容:有附件时在正文后追加附件名(及路径),刷新后仍能显示,继续对话时大模型也能从历史中拿到路径
|
||||
func userMessageContentForStorage(message string, attachments []ChatAttachment, savedPaths []string) string {
|
||||
if len(attachments) == 0 {
|
||||
return message
|
||||
}
|
||||
var b strings.Builder
|
||||
b.WriteString(message)
|
||||
for i, a := range attachments {
|
||||
b.WriteString("\n📎 ")
|
||||
b.WriteString(a.FileName)
|
||||
if i < len(savedPaths) && savedPaths[i] != "" {
|
||||
b.WriteString(": ")
|
||||
b.WriteString(savedPaths[i])
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// appendAttachmentsToMessage 仅将附件的保存路径追加到用户消息末尾,不再内联附件内容,避免上下文过长
|
||||
func appendAttachmentsToMessage(msg string, attachments []ChatAttachment, savedPaths []string) string {
|
||||
if len(attachments) == 0 {
|
||||
return msg
|
||||
}
|
||||
var b strings.Builder
|
||||
b.WriteString(msg)
|
||||
b.WriteString("\n\n[用户上传的文件已保存到以下路径(请按需读取文件内容,而不是依赖内联内容)]\n")
|
||||
for i, a := range attachments {
|
||||
if i < len(savedPaths) && savedPaths[i] != "" {
|
||||
b.WriteString(fmt.Sprintf("- %s: %s\n", a.FileName, savedPaths[i]))
|
||||
} else {
|
||||
b.WriteString(fmt.Sprintf("- %s: (路径未知,可能保存失败)\n", a.FileName))
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// ChatResponse 聊天响应
|
||||
@@ -181,6 +306,12 @@ func (h *AgentHandler) AgentLoop(c *gin.Context) {
|
||||
h.logger.Info("从ReAct数据恢复历史上下文", zap.Int("count", len(agentHistoryMessages)))
|
||||
}
|
||||
|
||||
// 校验附件数量(非流式)
|
||||
if len(req.Attachments) > maxAttachments {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": fmt.Sprintf("附件最多 %d 个", maxAttachments)})
|
||||
return
|
||||
}
|
||||
|
||||
// 应用角色用户提示词和工具配置
|
||||
finalMessage := req.Message
|
||||
var roleTools []string // 角色配置的工具列表
|
||||
@@ -206,9 +337,20 @@ func (h *AgentHandler) AgentLoop(c *gin.Context) {
|
||||
}
|
||||
}
|
||||
}
|
||||
var savedPaths []string
|
||||
if len(req.Attachments) > 0 {
|
||||
savedPaths, err = saveAttachmentsToDateAndConversationDir(req.Attachments, conversationID, h.logger)
|
||||
if err != nil {
|
||||
h.logger.Error("保存对话附件失败", zap.Error(err))
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "保存上传文件失败: " + err.Error()})
|
||||
return
|
||||
}
|
||||
}
|
||||
finalMessage = appendAttachmentsToMessage(finalMessage, req.Attachments, savedPaths)
|
||||
|
||||
// 保存用户消息(保存原始消息,不包含角色提示词)
|
||||
_, err = h.db.AddMessage(conversationID, "user", req.Message, nil)
|
||||
// 保存用户消息:有附件时一并保存附件名与路径,刷新后显示、继续对话时大模型也能从历史中拿到路径
|
||||
userContent := userMessageContentForStorage(req.Message, req.Attachments, savedPaths)
|
||||
_, err = h.db.AddMessage(conversationID, "user", userContent, nil)
|
||||
if err != nil {
|
||||
h.logger.Error("保存用户消息失败", zap.Error(err))
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "保存用户消息失败: " + err.Error()})
|
||||
@@ -259,6 +401,96 @@ func (h *AgentHandler) AgentLoop(c *gin.Context) {
|
||||
})
|
||||
}
|
||||
|
||||
// ProcessMessageForRobot 供机器人(企业微信/钉钉/飞书)调用:与 /api/agent-loop/stream 相同执行路径(含 progressCallback、过程详情),仅不发送 SSE,最后返回完整回复
|
||||
func (h *AgentHandler) ProcessMessageForRobot(ctx context.Context, conversationID, message, role string) (response string, convID string, err error) {
|
||||
if conversationID == "" {
|
||||
title := safeTruncateString(message, 50)
|
||||
conv, createErr := h.db.CreateConversation(title)
|
||||
if createErr != nil {
|
||||
return "", "", fmt.Errorf("创建对话失败: %w", createErr)
|
||||
}
|
||||
conversationID = conv.ID
|
||||
} else {
|
||||
if _, getErr := h.db.GetConversation(conversationID); getErr != nil {
|
||||
return "", "", fmt.Errorf("对话不存在")
|
||||
}
|
||||
}
|
||||
|
||||
agentHistoryMessages, err := h.loadHistoryFromReActData(conversationID)
|
||||
if err != nil {
|
||||
historyMessages, getErr := h.db.GetMessages(conversationID)
|
||||
if getErr != nil {
|
||||
agentHistoryMessages = []agent.ChatMessage{}
|
||||
} else {
|
||||
agentHistoryMessages = make([]agent.ChatMessage, 0, len(historyMessages))
|
||||
for _, msg := range historyMessages {
|
||||
agentHistoryMessages = append(agentHistoryMessages, agent.ChatMessage{Role: msg.Role, Content: msg.Content})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
finalMessage := message
|
||||
var roleTools, roleSkills []string
|
||||
if role != "" && role != "默认" && h.config.Roles != nil {
|
||||
if r, exists := h.config.Roles[role]; exists && r.Enabled {
|
||||
if r.UserPrompt != "" {
|
||||
finalMessage = r.UserPrompt + "\n\n" + message
|
||||
}
|
||||
roleTools = r.Tools
|
||||
roleSkills = r.Skills
|
||||
}
|
||||
}
|
||||
|
||||
if _, err = h.db.AddMessage(conversationID, "user", message, nil); err != nil {
|
||||
return "", "", fmt.Errorf("保存用户消息失败: %w", err)
|
||||
}
|
||||
|
||||
// 与 agent-loop/stream 一致:先创建助手消息占位,用 progressCallback 写过程详情(不发送 SSE)
|
||||
assistantMsg, err := h.db.AddMessage(conversationID, "assistant", "处理中...", nil)
|
||||
if err != nil {
|
||||
h.logger.Warn("机器人:创建助手消息占位失败", zap.Error(err))
|
||||
}
|
||||
var assistantMessageID string
|
||||
if assistantMsg != nil {
|
||||
assistantMessageID = assistantMsg.ID
|
||||
}
|
||||
progressCallback := h.createProgressCallback(conversationID, assistantMessageID, nil)
|
||||
|
||||
result, err := h.agent.AgentLoopWithProgress(ctx, finalMessage, agentHistoryMessages, conversationID, progressCallback, roleTools, roleSkills)
|
||||
if err != nil {
|
||||
errMsg := "执行失败: " + err.Error()
|
||||
if assistantMessageID != "" {
|
||||
_, _ = h.db.Exec("UPDATE messages SET content = ? WHERE id = ?", errMsg, assistantMessageID)
|
||||
_ = h.db.AddProcessDetail(assistantMessageID, conversationID, "error", errMsg, nil)
|
||||
}
|
||||
return "", conversationID, err
|
||||
}
|
||||
|
||||
// 更新助手消息内容与 MCP 执行 ID(与 stream 一致)
|
||||
if assistantMessageID != "" {
|
||||
mcpIDsJSON := ""
|
||||
if len(result.MCPExecutionIDs) > 0 {
|
||||
jsonData, _ := json.Marshal(result.MCPExecutionIDs)
|
||||
mcpIDsJSON = string(jsonData)
|
||||
}
|
||||
_, err = h.db.Exec(
|
||||
"UPDATE messages SET content = ?, mcp_execution_ids = ? WHERE id = ?",
|
||||
result.Response, mcpIDsJSON, assistantMessageID,
|
||||
)
|
||||
if err != nil {
|
||||
h.logger.Warn("机器人:更新助手消息失败", zap.Error(err))
|
||||
}
|
||||
} else {
|
||||
if _, err = h.db.AddMessage(conversationID, "assistant", result.Response, result.MCPExecutionIDs); err != nil {
|
||||
h.logger.Warn("机器人:保存助手消息失败", zap.Error(err))
|
||||
}
|
||||
}
|
||||
if result.LastReActInput != "" || result.LastReActOutput != "" {
|
||||
_ = h.db.SaveReActData(conversationID, result.LastReActInput, result.LastReActOutput)
|
||||
}
|
||||
return result.Response, conversationID, nil
|
||||
}
|
||||
|
||||
// StreamEvent 流式事件
|
||||
type StreamEvent struct {
|
||||
Type string `json:"type"` // conversation, progress, tool_call, tool_result, response, error, cancelled, done
|
||||
@@ -528,6 +760,12 @@ func (h *AgentHandler) AgentLoopStream(c *gin.Context) {
|
||||
h.logger.Info("从ReAct数据恢复历史上下文", zap.Int("count", len(agentHistoryMessages)))
|
||||
}
|
||||
|
||||
// 校验附件数量
|
||||
if len(req.Attachments) > maxAttachments {
|
||||
sendEvent("error", fmt.Sprintf("附件最多 %d 个", maxAttachments), nil)
|
||||
return
|
||||
}
|
||||
|
||||
// 应用角色用户提示词和工具配置
|
||||
finalMessage := req.Message
|
||||
var roleTools []string // 角色配置的工具列表
|
||||
@@ -555,10 +793,22 @@ func (h *AgentHandler) AgentLoopStream(c *gin.Context) {
|
||||
}
|
||||
}
|
||||
}
|
||||
var savedPaths []string
|
||||
if len(req.Attachments) > 0 {
|
||||
savedPaths, err = saveAttachmentsToDateAndConversationDir(req.Attachments, conversationID, h.logger)
|
||||
if err != nil {
|
||||
h.logger.Error("保存对话附件失败", zap.Error(err))
|
||||
sendEvent("error", "保存上传文件失败: "+err.Error(), nil)
|
||||
return
|
||||
}
|
||||
}
|
||||
// 仅将附件保存路径追加到 finalMessage,避免将文件内容内联到大模型上下文中
|
||||
finalMessage = appendAttachmentsToMessage(finalMessage, req.Attachments, savedPaths)
|
||||
// 如果roleTools为空,表示使用所有工具(默认角色或未配置工具的角色)
|
||||
|
||||
// 保存用户消息(保存原始消息,不包含角色提示词)
|
||||
_, err = h.db.AddMessage(conversationID, "user", req.Message, nil)
|
||||
// 保存用户消息:有附件时一并保存附件名与路径,刷新后显示、继续对话时大模型也能从历史中拿到路径
|
||||
userContent := userMessageContentForStorage(req.Message, req.Attachments, savedPaths)
|
||||
_, err = h.db.AddMessage(conversationID, "user", userContent, nil)
|
||||
if err != nil {
|
||||
h.logger.Error("保存用户消息失败", zap.Error(err))
|
||||
}
|
||||
@@ -1194,7 +1444,8 @@ func (h *AgentHandler) executeBatchQueue(queueID string) {
|
||||
// 执行任务(使用包含角色提示词的finalMessage和角色工具列表)
|
||||
h.logger.Info("执行批量任务", zap.String("queueId", queueID), zap.String("taskId", task.ID), zap.String("message", task.Message), zap.String("role", queue.Role), zap.String("conversationId", conversationID))
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
|
||||
// 单个子任务超时时间:从30分钟调整为6小时,适配长时间渗透/扫描任务
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 6*time.Hour)
|
||||
// 存储取消函数,以便在取消队列时能够取消当前任务
|
||||
h.batchTaskManager.SetTaskCancel(queueID, cancel)
|
||||
// 使用队列配置的角色工具列表(如果为空,表示使用所有工具)
|
||||
|
||||
@@ -44,6 +44,11 @@ type AppUpdater interface {
|
||||
UpdateKnowledgeComponents(handler *KnowledgeHandler, manager interface{}, retriever interface{}, indexer interface{})
|
||||
}
|
||||
|
||||
// RobotRestarter 机器人连接重启器(用于配置应用后重启钉钉/飞书长连接)
|
||||
type RobotRestarter interface {
|
||||
RestartRobotConnections()
|
||||
}
|
||||
|
||||
// ConfigHandler 配置处理器
|
||||
type ConfigHandler struct {
|
||||
configPath string
|
||||
@@ -59,6 +64,7 @@ type ConfigHandler struct {
|
||||
retrieverUpdater RetrieverUpdater // 检索器更新器(可选)
|
||||
knowledgeInitializer KnowledgeInitializer // 知识库初始化器(可选)
|
||||
appUpdater AppUpdater // App更新器(可选)
|
||||
robotRestarter RobotRestarter // 机器人连接重启器(可选),ApplyConfig 时重启钉钉/飞书
|
||||
logger *zap.Logger
|
||||
mu sync.RWMutex
|
||||
lastEmbeddingConfig *config.EmbeddingConfig // 上一次的嵌入模型配置(用于检测变更)
|
||||
@@ -142,13 +148,22 @@ func (h *ConfigHandler) SetAppUpdater(updater AppUpdater) {
|
||||
h.appUpdater = updater
|
||||
}
|
||||
|
||||
// SetRobotRestarter 设置机器人连接重启器(ApplyConfig 时用于重启钉钉/飞书长连接)
|
||||
func (h *ConfigHandler) SetRobotRestarter(restarter RobotRestarter) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.robotRestarter = restarter
|
||||
}
|
||||
|
||||
// GetConfigResponse 获取配置响应
|
||||
type GetConfigResponse struct {
|
||||
OpenAI config.OpenAIConfig `json:"openai"`
|
||||
FOFA config.FofaConfig `json:"fofa"`
|
||||
MCP config.MCPConfig `json:"mcp"`
|
||||
Tools []ToolConfigInfo `json:"tools"`
|
||||
Agent config.AgentConfig `json:"agent"`
|
||||
Knowledge config.KnowledgeConfig `json:"knowledge"`
|
||||
Robots config.RobotsConfig `json:"robots,omitempty"`
|
||||
}
|
||||
|
||||
// ToolConfigInfo 工具配置信息
|
||||
@@ -216,10 +231,12 @@ func (h *ConfigHandler) GetConfig(c *gin.Context) {
|
||||
|
||||
c.JSON(http.StatusOK, GetConfigResponse{
|
||||
OpenAI: h.config.OpenAI,
|
||||
FOFA: h.config.FOFA,
|
||||
MCP: h.config.MCP,
|
||||
Tools: tools,
|
||||
Agent: h.config.Agent,
|
||||
Knowledge: h.config.Knowledge,
|
||||
Robots: h.config.Robots,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -472,10 +489,12 @@ func (h *ConfigHandler) GetTools(c *gin.Context) {
|
||||
// UpdateConfigRequest 更新配置请求
|
||||
type UpdateConfigRequest struct {
|
||||
OpenAI *config.OpenAIConfig `json:"openai,omitempty"`
|
||||
FOFA *config.FofaConfig `json:"fofa,omitempty"`
|
||||
MCP *config.MCPConfig `json:"mcp,omitempty"`
|
||||
Tools []ToolEnableStatus `json:"tools,omitempty"`
|
||||
Agent *config.AgentConfig `json:"agent,omitempty"`
|
||||
Knowledge *config.KnowledgeConfig `json:"knowledge,omitempty"`
|
||||
Robots *config.RobotsConfig `json:"robots,omitempty"`
|
||||
}
|
||||
|
||||
// ToolEnableStatus 工具启用状态
|
||||
@@ -506,6 +525,12 @@ func (h *ConfigHandler) UpdateConfig(c *gin.Context) {
|
||||
)
|
||||
}
|
||||
|
||||
// 更新FOFA配置
|
||||
if req.FOFA != nil {
|
||||
h.config.FOFA = *req.FOFA
|
||||
h.logger.Info("更新FOFA配置", zap.String("email", h.config.FOFA.Email))
|
||||
}
|
||||
|
||||
// 更新MCP配置
|
||||
if req.MCP != nil {
|
||||
h.config.MCP = *req.MCP
|
||||
@@ -546,6 +571,16 @@ func (h *ConfigHandler) UpdateConfig(c *gin.Context) {
|
||||
)
|
||||
}
|
||||
|
||||
// 更新机器人配置
|
||||
if req.Robots != nil {
|
||||
h.config.Robots = *req.Robots
|
||||
h.logger.Info("更新机器人配置",
|
||||
zap.Bool("wecom_enabled", h.config.Robots.Wecom.Enabled),
|
||||
zap.Bool("dingtalk_enabled", h.config.Robots.Dingtalk.Enabled),
|
||||
zap.Bool("lark_enabled", h.config.Robots.Lark.Enabled),
|
||||
)
|
||||
}
|
||||
|
||||
// 更新工具启用状态
|
||||
if req.Tools != nil {
|
||||
// 分离内部工具和外部工具
|
||||
@@ -815,6 +850,12 @@ func (h *ConfigHandler) ApplyConfig(c *gin.Context) {
|
||||
}
|
||||
}
|
||||
|
||||
// 重启钉钉/飞书长连接,使前端修改的机器人配置立即生效(无需重启服务)
|
||||
if h.robotRestarter != nil {
|
||||
h.robotRestarter.RestartRobotConnections()
|
||||
h.logger.Info("已触发机器人连接重启(钉钉/飞书)")
|
||||
}
|
||||
|
||||
h.logger.Info("配置已应用",
|
||||
zap.Int("tools_count", len(h.config.Security.Tools)),
|
||||
)
|
||||
@@ -845,7 +886,9 @@ func (h *ConfigHandler) saveConfig() error {
|
||||
updateAgentConfig(root, h.config.Agent.MaxIterations)
|
||||
updateMCPConfig(root, h.config.MCP)
|
||||
updateOpenAIConfig(root, h.config.OpenAI)
|
||||
updateFOFAConfig(root, h.config.FOFA)
|
||||
updateKnowledgeConfig(root, h.config.Knowledge)
|
||||
updateRobotsConfig(root, h.config.Robots)
|
||||
// 更新外部MCP配置(使用external_mcp.go中的函数,同一包中可直接调用)
|
||||
// 读取原始配置以保持向后兼容
|
||||
originalConfigs := make(map[string]map[string]bool)
|
||||
@@ -989,6 +1032,14 @@ func updateOpenAIConfig(doc *yaml.Node, cfg config.OpenAIConfig) {
|
||||
setStringInMap(openaiNode, "model", cfg.Model)
|
||||
}
|
||||
|
||||
func updateFOFAConfig(doc *yaml.Node, cfg config.FofaConfig) {
|
||||
root := doc.Content[0]
|
||||
fofaNode := ensureMap(root, "fofa")
|
||||
setStringInMap(fofaNode, "base_url", cfg.BaseURL)
|
||||
setStringInMap(fofaNode, "email", cfg.Email)
|
||||
setStringInMap(fofaNode, "api_key", cfg.APIKey)
|
||||
}
|
||||
|
||||
func updateKnowledgeConfig(doc *yaml.Node, cfg config.KnowledgeConfig) {
|
||||
root := doc.Content[0]
|
||||
knowledgeNode := ensureMap(root, "knowledge")
|
||||
@@ -1011,6 +1062,40 @@ func updateKnowledgeConfig(doc *yaml.Node, cfg config.KnowledgeConfig) {
|
||||
setIntInMap(retrievalNode, "top_k", cfg.Retrieval.TopK)
|
||||
setFloatInMap(retrievalNode, "similarity_threshold", cfg.Retrieval.SimilarityThreshold)
|
||||
setFloatInMap(retrievalNode, "hybrid_weight", cfg.Retrieval.HybridWeight)
|
||||
|
||||
// 更新索引配置
|
||||
indexingNode := ensureMap(knowledgeNode, "indexing")
|
||||
setIntInMap(indexingNode, "chunk_size", cfg.Indexing.ChunkSize)
|
||||
setIntInMap(indexingNode, "chunk_overlap", cfg.Indexing.ChunkOverlap)
|
||||
setIntInMap(indexingNode, "max_chunks_per_item", cfg.Indexing.MaxChunksPerItem)
|
||||
setIntInMap(indexingNode, "max_rpm", cfg.Indexing.MaxRPM)
|
||||
setIntInMap(indexingNode, "rate_limit_delay_ms", cfg.Indexing.RateLimitDelayMs)
|
||||
setIntInMap(indexingNode, "max_retries", cfg.Indexing.MaxRetries)
|
||||
setIntInMap(indexingNode, "retry_delay_ms", cfg.Indexing.RetryDelayMs)
|
||||
}
|
||||
|
||||
func updateRobotsConfig(doc *yaml.Node, cfg config.RobotsConfig) {
|
||||
root := doc.Content[0]
|
||||
robotsNode := ensureMap(root, "robots")
|
||||
|
||||
wecomNode := ensureMap(robotsNode, "wecom")
|
||||
setBoolInMap(wecomNode, "enabled", cfg.Wecom.Enabled)
|
||||
setStringInMap(wecomNode, "token", cfg.Wecom.Token)
|
||||
setStringInMap(wecomNode, "encoding_aes_key", cfg.Wecom.EncodingAESKey)
|
||||
setStringInMap(wecomNode, "corp_id", cfg.Wecom.CorpID)
|
||||
setStringInMap(wecomNode, "secret", cfg.Wecom.Secret)
|
||||
setIntInMap(wecomNode, "agent_id", int(cfg.Wecom.AgentID))
|
||||
|
||||
dingtalkNode := ensureMap(robotsNode, "dingtalk")
|
||||
setBoolInMap(dingtalkNode, "enabled", cfg.Dingtalk.Enabled)
|
||||
setStringInMap(dingtalkNode, "client_id", cfg.Dingtalk.ClientID)
|
||||
setStringInMap(dingtalkNode, "client_secret", cfg.Dingtalk.ClientSecret)
|
||||
|
||||
larkNode := ensureMap(robotsNode, "lark")
|
||||
setBoolInMap(larkNode, "enabled", cfg.Lark.Enabled)
|
||||
setStringInMap(larkNode, "app_id", cfg.Lark.AppID)
|
||||
setStringInMap(larkNode, "app_secret", cfg.Lark.AppSecret)
|
||||
setStringInMap(larkNode, "verify_token", cfg.Lark.VerifyToken)
|
||||
}
|
||||
|
||||
func ensureMap(parent *yaml.Node, path ...string) *yaml.Node {
|
||||
|
||||
@@ -0,0 +1,467 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"cyberstrike-ai/internal/config"
|
||||
openaiClient "cyberstrike-ai/internal/openai"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"go.uber.org/zap"
|
||||
)
|
||||
|
||||
type FofaHandler struct {
|
||||
cfg *config.Config
|
||||
logger *zap.Logger
|
||||
client *http.Client
|
||||
openAIClient *openaiClient.Client
|
||||
}
|
||||
|
||||
func NewFofaHandler(cfg *config.Config, logger *zap.Logger) *FofaHandler {
|
||||
// LLM 请求通常比 FOFA 查询更慢一点,单独给一个更宽松的超时。
|
||||
llmHTTPClient := &http.Client{Timeout: 2 * time.Minute}
|
||||
var llmCfg *config.OpenAIConfig
|
||||
if cfg != nil {
|
||||
llmCfg = &cfg.OpenAI
|
||||
}
|
||||
return &FofaHandler{
|
||||
cfg: cfg,
|
||||
logger: logger,
|
||||
client: &http.Client{Timeout: 30 * time.Second},
|
||||
openAIClient: openaiClient.NewClient(llmCfg, llmHTTPClient, logger),
|
||||
}
|
||||
}
|
||||
|
||||
type fofaSearchRequest struct {
|
||||
Query string `json:"query" binding:"required"`
|
||||
Size int `json:"size,omitempty"`
|
||||
Page int `json:"page,omitempty"`
|
||||
Fields string `json:"fields,omitempty"`
|
||||
Full bool `json:"full,omitempty"`
|
||||
}
|
||||
|
||||
type fofaParseRequest struct {
|
||||
Text string `json:"text" binding:"required"`
|
||||
}
|
||||
|
||||
type fofaParseResponse struct {
|
||||
Query string `json:"query"`
|
||||
Explanation string `json:"explanation,omitempty"`
|
||||
Warnings []string `json:"warnings,omitempty"`
|
||||
}
|
||||
|
||||
type fofaAPIResponse struct {
|
||||
Error bool `json:"error"`
|
||||
ErrMsg string `json:"errmsg"`
|
||||
Size int `json:"size"`
|
||||
Page int `json:"page"`
|
||||
Total int `json:"total"`
|
||||
Mode string `json:"mode"`
|
||||
Query string `json:"query"`
|
||||
Results [][]interface{} `json:"results"`
|
||||
}
|
||||
|
||||
type fofaSearchResponse struct {
|
||||
Query string `json:"query"`
|
||||
Size int `json:"size"`
|
||||
Page int `json:"page"`
|
||||
Total int `json:"total"`
|
||||
Fields []string `json:"fields"`
|
||||
ResultsCount int `json:"results_count"`
|
||||
Results []map[string]interface{} `json:"results"`
|
||||
}
|
||||
|
||||
func (h *FofaHandler) resolveCredentials() (email, apiKey string) {
|
||||
// 优先环境变量(便于容器部署),其次配置文件
|
||||
email = strings.TrimSpace(os.Getenv("FOFA_EMAIL"))
|
||||
apiKey = strings.TrimSpace(os.Getenv("FOFA_API_KEY"))
|
||||
if email != "" && apiKey != "" {
|
||||
return email, apiKey
|
||||
}
|
||||
if h.cfg != nil {
|
||||
if email == "" {
|
||||
email = strings.TrimSpace(h.cfg.FOFA.Email)
|
||||
}
|
||||
if apiKey == "" {
|
||||
apiKey = strings.TrimSpace(h.cfg.FOFA.APIKey)
|
||||
}
|
||||
}
|
||||
return email, apiKey
|
||||
}
|
||||
|
||||
func (h *FofaHandler) resolveBaseURL() string {
|
||||
if h.cfg != nil {
|
||||
if v := strings.TrimSpace(h.cfg.FOFA.BaseURL); v != "" {
|
||||
return v
|
||||
}
|
||||
}
|
||||
return "https://fofa.info/api/v1/search/all"
|
||||
}
|
||||
|
||||
// ParseNaturalLanguage 将自然语言解析为 FOFA 查询语法(仅生成,不执行查询)
|
||||
func (h *FofaHandler) ParseNaturalLanguage(c *gin.Context) {
|
||||
var req fofaParseRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "无效的请求参数: " + err.Error()})
|
||||
return
|
||||
}
|
||||
req.Text = strings.TrimSpace(req.Text)
|
||||
if req.Text == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "text 不能为空"})
|
||||
return
|
||||
}
|
||||
|
||||
if h.cfg == nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "系统配置未初始化"})
|
||||
return
|
||||
}
|
||||
if strings.TrimSpace(h.cfg.OpenAI.APIKey) == "" || strings.TrimSpace(h.cfg.OpenAI.Model) == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{
|
||||
"error": "未配置 AI 模型:请在系统设置中填写 openai.api_key 与 openai.model(支持 OpenAI 兼容 API,如 DeepSeek)",
|
||||
"need": []string{"openai.api_key", "openai.model"},
|
||||
})
|
||||
return
|
||||
}
|
||||
if h.openAIClient == nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "AI 客户端未初始化"})
|
||||
return
|
||||
}
|
||||
|
||||
systemPrompt := strings.TrimSpace(`
|
||||
你是“FOFA 查询语法生成器”。任务:把用户输入的自然语言搜索意图,转换成 FOFA 查询语法。
|
||||
|
||||
输出要求(非常重要):
|
||||
1) 只输出 JSON(不要 markdown、不要代码块、不要额外解释文本)
|
||||
2) JSON 结构必须是:
|
||||
{
|
||||
"query": "string,FOFA查询语法(可直接粘贴到 FOFA 或本系统查询框)",
|
||||
"explanation": "string,可选,解释你如何映射字段/逻辑",
|
||||
"warnings": ["string"...] 可选,列出歧义/风险/需要人工确认的点
|
||||
}
|
||||
3) 如果用户输入本身已经是 FOFA 查询语法(或非常接近 FOFA 语法的表达式),应当“原样返回”为 query:
|
||||
- 不要擅自改写字段名、操作符、括号结构
|
||||
- 不要改写任何字符串值(尤其是地理位置类值),不要做缩写/同义词替换/翻译/音译
|
||||
|
||||
查询语法要点(来自 FOFA 语法参考):
|
||||
- 逻辑连接符:&&(与)、||(或),必要时用 () 包住子表达式以确认优先级(括号优先级最高)
|
||||
- 当同一层级同时出现 && 与 ||(混用)时,用 () 明确优先级(避免歧义)
|
||||
- 比较/匹配:
|
||||
- = 匹配;当字段="" 时,可查询“不存在该字段”或“值为空”的情况
|
||||
- == 完全匹配;当字段=="" 时,可查询“字段存在且值为空”的情况
|
||||
- != 不匹配;当字段!="" 时,可查询“值不为空”的情况
|
||||
- *= 模糊匹配;可使用 * 或 ? 进行搜索
|
||||
- 直接输入关键词(不带字段)会在标题、HTML内容、HTTP头、URL字段中搜索;但当意图明确时优先用字段表达(更可控、更准确)
|
||||
|
||||
字段示例速查(来自用户提供的案例,可直接套用/拼接):
|
||||
- 高级搜索操作符示例:
|
||||
- title="beijing" (= 匹配)
|
||||
- title=="" (== 完全匹配,字段存在且值为空)
|
||||
- title="" (= 匹配,可能表示字段不存在或值为空)
|
||||
- title!="" (!= 不匹配,可用于值不为空)
|
||||
- title*="*Home*" (*= 模糊匹配,用 * 或 ?)
|
||||
- (app="Apache" || app="Nginx") && country="CN" (混用 && / || 时用括号)
|
||||
- 基础类(General):
|
||||
- ip="1.1.1.1"
|
||||
- ip="220.181.111.1/24"
|
||||
- ip="2600:9000:202a:2600:18:4ab7:f600:93a1"
|
||||
- port="6379"
|
||||
- domain="qq.com"
|
||||
- host=".fofa.info"
|
||||
- os="centos"
|
||||
- server="Microsoft-IIS/10"
|
||||
- asn="19551"
|
||||
- org="LLC Baxet"
|
||||
- is_domain=true / is_domain=false
|
||||
- is_ipv6=true / is_ipv6=false
|
||||
- 标记类(Special Label):
|
||||
- app="Microsoft-Exchange"
|
||||
- fid="sSXXGNUO2FefBTcCLIT/2Q=="
|
||||
- product="NGINX"
|
||||
- product="Roundcube-Webmail" && product.version="1.6.10"
|
||||
- category="服务"
|
||||
- type="service" / type="subdomain"
|
||||
- cloud_name="Aliyundun"
|
||||
- is_cloud=true / is_cloud=false
|
||||
- is_fraud=true / is_fraud=false
|
||||
- is_honeypot=true / is_honeypot=false
|
||||
- 协议类(type=service):
|
||||
- protocol="quic"
|
||||
- banner="users"
|
||||
- banner_hash="7330105010150477363"
|
||||
- banner_fid="zRpqmn0FXQRjZpH8MjMX55zpMy9SgsW8"
|
||||
- base_protocol="udp" / base_protocol="tcp"
|
||||
- 网站类(type=subdomain):
|
||||
- title="beijing"
|
||||
- header="elastic"
|
||||
- header_hash="1258854265"
|
||||
- body="网络空间测绘"
|
||||
- body_hash="-2090962452"
|
||||
- js_name="js/jquery.js"
|
||||
- js_md5="82ac3f14327a8b7ba49baa208d4eaa15"
|
||||
- cname="customers.spektrix.com"
|
||||
- cname_domain="siteforce.com"
|
||||
- icon_hash="-247388890"
|
||||
- status_code="402"
|
||||
- icp="京ICP证030173号"
|
||||
- sdk_hash="Are3qNnP2Eqn7q5kAoUO3l+w3mgVIytO"
|
||||
- 地理位置(Location):
|
||||
- country="CN" 或 country="中国"
|
||||
- region="Zhejiang" 或 region="浙江"(仅支持中国地区中文)
|
||||
- city="Hangzhou"
|
||||
- 证书类(Certificate):
|
||||
- cert="baidu"
|
||||
- cert.subject="Oracle Corporation"
|
||||
- cert.issuer="DigiCert"
|
||||
- cert.subject.org="Oracle Corporation"
|
||||
- cert.subject.cn="baidu.com"
|
||||
- cert.issuer.org="cPanel, Inc."
|
||||
- cert.issuer.cn="Synology Inc. CA"
|
||||
- cert.domain="huawei.com"
|
||||
- cert.is_equal=true / cert.is_equal=false
|
||||
- cert.is_valid=true / cert.is_valid=false
|
||||
- cert.is_match=true / cert.is_match=false
|
||||
- cert.is_expired=true / cert.is_expired=false
|
||||
- jarm="2ad2ad0002ad2ad22c2ad2ad2ad2ad2eac92ec34bcc0cf7520e97547f83e81"
|
||||
- tls.version="TLS 1.3"
|
||||
- tls.ja3s="15af977ce25de452b96affa2addb1036"
|
||||
- cert.sn="356078156165546797850343536942784588840297"
|
||||
- cert.not_after.after="2025-03-01" / cert.not_after.before="2025-03-01"
|
||||
- cert.not_before.after="2025-03-01" / cert.not_before.before="2025-03-01"
|
||||
- 时间类(Last update time):
|
||||
- after="2023-01-01"
|
||||
- before="2023-12-01"
|
||||
- after="2023-01-01" && before="2023-12-01"
|
||||
- 独立IP语法(需配合 ip_filter / ip_exclude):
|
||||
- ip_filter(banner="SSH-2.0-OpenSSH_6.7p2") && ip_filter(icon_hash="-1057022626")
|
||||
- ip_filter(banner="SSH-2.0-OpenSSH_6.7p2" && asn="3462") && ip_exclude(title="EdgeOS")
|
||||
- port_size="6" / port_size_gt="6" / port_size_lt="12"
|
||||
- ip_ports="80,161"
|
||||
- ip_country="CN"
|
||||
- ip_region="Zhejiang"
|
||||
- ip_city="Hangzhou"
|
||||
- ip_after="2021-03-18"
|
||||
- ip_before="2019-09-09"
|
||||
|
||||
生成约束与注意事项:
|
||||
- 字符串值一律用英文双引号包裹,例如 title="登录"、country="CN"
|
||||
- 字符串值保持字面一致:不要缩写(例如 city="beijing" 不要变成 city="BJ"),不要用别名(例如 Beijing/Peking),不要擅自翻译/音译/改写大小写
|
||||
- 地理位置字段(country/region/city)更倾向于“按用户给定值输出”;不确定合法取值时,不要猜测,把备选写进 warnings
|
||||
- 不要捏造不存在的 FOFA 字段;不确定时把不确定点写进 warnings,并输出一个保守的 query
|
||||
- 当用户描述里有“多个与/或条件”,优先加 () 明确优先级,例如:(app="Apache" || app="Nginx") && country="CN"
|
||||
- 当用户缺少关键条件导致范围过大或歧义(如地点/协议/端口/服务类型未说明),允许 query 为空字符串,并在 warnings 里明确需要补充的信息
|
||||
`)
|
||||
|
||||
userPrompt := fmt.Sprintf("自然语言意图:%s", req.Text)
|
||||
|
||||
requestBody := map[string]interface{}{
|
||||
"model": h.cfg.OpenAI.Model,
|
||||
"messages": []map[string]interface{}{
|
||||
{"role": "system", "content": systemPrompt},
|
||||
{"role": "user", "content": userPrompt},
|
||||
},
|
||||
"temperature": 0.1,
|
||||
"max_tokens": 1200,
|
||||
}
|
||||
|
||||
// OpenAI 返回结构:只需要 choices[0].message.content
|
||||
var apiResponse struct {
|
||||
Choices []struct {
|
||||
Message struct {
|
||||
Content string `json:"content"`
|
||||
} `json:"message"`
|
||||
} `json:"choices"`
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(c.Request.Context(), 90*time.Second)
|
||||
defer cancel()
|
||||
|
||||
if err := h.openAIClient.ChatCompletion(ctx, requestBody, &apiResponse); err != nil {
|
||||
var apiErr *openaiClient.APIError
|
||||
if errors.As(err, &apiErr) {
|
||||
h.logger.Warn("FOFA自然语言解析:LLM返回错误", zap.Int("status", apiErr.StatusCode))
|
||||
c.JSON(http.StatusBadGateway, gin.H{"error": "AI 解析失败(上游返回非 200),请检查模型配置或稍后重试"})
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusBadGateway, gin.H{"error": "AI 解析失败: " + err.Error()})
|
||||
return
|
||||
}
|
||||
if len(apiResponse.Choices) == 0 {
|
||||
c.JSON(http.StatusBadGateway, gin.H{"error": "AI 未返回有效结果"})
|
||||
return
|
||||
}
|
||||
|
||||
content := strings.TrimSpace(apiResponse.Choices[0].Message.Content)
|
||||
// 兼容模型偶尔返回 ```json ... ``` 的情况
|
||||
content = strings.TrimPrefix(content, "```json")
|
||||
content = strings.TrimPrefix(content, "```")
|
||||
content = strings.TrimSuffix(content, "```")
|
||||
content = strings.TrimSpace(content)
|
||||
|
||||
var parsed fofaParseResponse
|
||||
if err := json.Unmarshal([]byte(content), &parsed); err != nil {
|
||||
// 直接回传一部分原文,方便排查,但避免太大
|
||||
snippet := content
|
||||
if len(snippet) > 1200 {
|
||||
snippet = snippet[:1200]
|
||||
}
|
||||
c.JSON(http.StatusBadGateway, gin.H{
|
||||
"error": "AI 返回内容无法解析为 JSON,请稍后重试或换个描述方式",
|
||||
"snippet": snippet,
|
||||
})
|
||||
return
|
||||
}
|
||||
parsed.Query = strings.TrimSpace(parsed.Query)
|
||||
if parsed.Query == "" {
|
||||
// query 允许为空(表示需求不明确),但前端需要明确提示
|
||||
if len(parsed.Warnings) == 0 {
|
||||
parsed.Warnings = []string{"需求信息不足,未能生成可用的 FOFA 查询语法,请补充关键条件(如国家/端口/产品/域名等)。"}
|
||||
}
|
||||
}
|
||||
|
||||
c.JSON(http.StatusOK, parsed)
|
||||
}
|
||||
|
||||
// Search FOFA 查询(后端代理,避免前端暴露 key)
|
||||
func (h *FofaHandler) Search(c *gin.Context) {
|
||||
var req fofaSearchRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "无效的请求参数: " + err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
req.Query = strings.TrimSpace(req.Query)
|
||||
if req.Query == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "query 不能为空"})
|
||||
return
|
||||
}
|
||||
if req.Size <= 0 {
|
||||
req.Size = 100
|
||||
}
|
||||
if req.Page <= 0 {
|
||||
req.Page = 1
|
||||
}
|
||||
// FOFA 接口 size 上限和账户权限相关,这里只做一个合理的保护
|
||||
if req.Size > 10000 {
|
||||
req.Size = 10000
|
||||
}
|
||||
if req.Fields == "" {
|
||||
req.Fields = "host,ip,port,domain,title,protocol,country,province,city,server"
|
||||
}
|
||||
|
||||
email, apiKey := h.resolveCredentials()
|
||||
if email == "" || apiKey == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{
|
||||
"error": "FOFA 未配置:请在系统设置中填写 FOFA Email/API Key,或设置环境变量 FOFA_EMAIL/FOFA_API_KEY",
|
||||
"need": []string{"fofa.email", "fofa.api_key"},
|
||||
"env_key": []string{"FOFA_EMAIL", "FOFA_API_KEY"},
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
baseURL := h.resolveBaseURL()
|
||||
qb64 := base64.StdEncoding.EncodeToString([]byte(req.Query))
|
||||
|
||||
u, err := url.Parse(baseURL)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "FOFA base_url 无效: " + err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
params := u.Query()
|
||||
params.Set("email", email)
|
||||
params.Set("key", apiKey)
|
||||
params.Set("qbase64", qb64)
|
||||
params.Set("size", fmt.Sprintf("%d", req.Size))
|
||||
params.Set("page", fmt.Sprintf("%d", req.Page))
|
||||
params.Set("fields", strings.TrimSpace(req.Fields))
|
||||
if req.Full {
|
||||
params.Set("full", "true")
|
||||
} else {
|
||||
// 明确传 false,便于排查
|
||||
params.Set("full", "false")
|
||||
}
|
||||
u.RawQuery = params.Encode()
|
||||
|
||||
httpReq, err := http.NewRequestWithContext(c.Request.Context(), http.MethodGet, u.String(), nil)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "创建请求失败: " + err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
resp, err := h.client.Do(httpReq)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadGateway, gin.H{"error": "请求 FOFA 失败: " + err.Error()})
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
c.JSON(http.StatusBadGateway, gin.H{"error": fmt.Sprintf("FOFA 返回非 2xx: %d", resp.StatusCode)})
|
||||
return
|
||||
}
|
||||
|
||||
var apiResp fofaAPIResponse
|
||||
if err := json.NewDecoder(resp.Body).Decode(&apiResp); err != nil {
|
||||
c.JSON(http.StatusBadGateway, gin.H{"error": "解析 FOFA 响应失败: " + err.Error()})
|
||||
return
|
||||
}
|
||||
if apiResp.Error {
|
||||
msg := strings.TrimSpace(apiResp.ErrMsg)
|
||||
if msg == "" {
|
||||
msg = "FOFA 返回错误"
|
||||
}
|
||||
c.JSON(http.StatusBadGateway, gin.H{"error": msg})
|
||||
return
|
||||
}
|
||||
|
||||
fields := splitAndCleanCSV(req.Fields)
|
||||
results := make([]map[string]interface{}, 0, len(apiResp.Results))
|
||||
for _, row := range apiResp.Results {
|
||||
item := make(map[string]interface{}, len(fields))
|
||||
for i, f := range fields {
|
||||
if i < len(row) {
|
||||
item[f] = row[i]
|
||||
} else {
|
||||
item[f] = nil
|
||||
}
|
||||
}
|
||||
results = append(results, item)
|
||||
}
|
||||
|
||||
c.JSON(http.StatusOK, fofaSearchResponse{
|
||||
Query: req.Query,
|
||||
Size: apiResp.Size,
|
||||
Page: apiResp.Page,
|
||||
Total: apiResp.Total,
|
||||
Fields: fields,
|
||||
ResultsCount: len(results),
|
||||
Results: results,
|
||||
})
|
||||
}
|
||||
|
||||
func splitAndCleanCSV(s string) []string {
|
||||
parts := strings.Split(s, ",")
|
||||
out := make([]string, 0, len(parts))
|
||||
seen := make(map[string]struct{}, len(parts))
|
||||
for _, p := range parts {
|
||||
v := strings.TrimSpace(p)
|
||||
if v == "" {
|
||||
continue
|
||||
}
|
||||
if _, ok := seen[v]; ok {
|
||||
continue
|
||||
}
|
||||
seen[v] = struct{}{}
|
||||
out = append(out, v)
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -15,11 +15,11 @@ import (
|
||||
|
||||
// KnowledgeHandler 知识库处理器
|
||||
type KnowledgeHandler struct {
|
||||
manager *knowledge.Manager
|
||||
manager *knowledge.Manager
|
||||
retriever *knowledge.Retriever
|
||||
indexer *knowledge.Indexer
|
||||
db *database.DB
|
||||
logger *zap.Logger
|
||||
indexer *knowledge.Indexer
|
||||
db *database.DB
|
||||
logger *zap.Logger
|
||||
}
|
||||
|
||||
// NewKnowledgeHandler 创建新的知识库处理器
|
||||
@@ -55,7 +55,7 @@ func (h *KnowledgeHandler) GetCategories(c *gin.Context) {
|
||||
func (h *KnowledgeHandler) GetItems(c *gin.Context) {
|
||||
category := c.Query("category")
|
||||
searchKeyword := c.Query("search") // 搜索关键字
|
||||
|
||||
|
||||
// 如果提供了搜索关键字,执行关键字搜索(在所有数据中搜索)
|
||||
if searchKeyword != "" {
|
||||
items, err := h.manager.SearchItemsByKeyword(searchKeyword, category)
|
||||
@@ -75,7 +75,7 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
|
||||
groupedByCategory[cat] = append(groupedByCategory[cat], item)
|
||||
}
|
||||
|
||||
// 转换为CategoryWithItems格式
|
||||
// 转换为 CategoryWithItems 格式
|
||||
categoriesWithItems := make([]*knowledge.CategoryWithItems, 0, len(groupedByCategory))
|
||||
for cat, catItems := range groupedByCategory {
|
||||
categoriesWithItems = append(categoriesWithItems, &knowledge.CategoryWithItems{
|
||||
@@ -102,12 +102,12 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
|
||||
// 分页模式:categoryPage=true 表示按分类分页,否则按项分页(向后兼容)
|
||||
categoryPageMode := c.Query("categoryPage") != "false" // 默认使用分类分页
|
||||
|
||||
|
||||
// 分页参数
|
||||
limit := 50 // 默认每页50条(分类分页时为分类数,项分页时为项数)
|
||||
limit := 50 // 默认每页 50 条(分类分页时为分类数,项分页时为项数)
|
||||
offset := 0
|
||||
if limitStr := c.Query("limit"); limitStr != "" {
|
||||
if parsed, err := parseInt(limitStr); err == nil && parsed > 0 && parsed <= 500 {
|
||||
@@ -120,7 +120,7 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
|
||||
}
|
||||
}
|
||||
|
||||
// 如果指定了category参数,且使用分类分页模式,则只返回该分类
|
||||
// 如果指定了 category 参数,且使用分类分页模式,则只返回该分类
|
||||
if category != "" && categoryPageMode {
|
||||
// 单分类模式:返回该分类的所有知识项(不分页)
|
||||
items, total, err := h.manager.GetItemsSummary(category, 0, 0)
|
||||
@@ -150,9 +150,9 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
|
||||
|
||||
if categoryPageMode {
|
||||
// 按分类分页模式(默认)
|
||||
// limit表示每页分类数,推荐5-10个分类
|
||||
// limit 表示每页分类数,推荐 5-10 个分类
|
||||
if limit <= 0 || limit > 100 {
|
||||
limit = 10 // 默认每页10个分类
|
||||
limit = 10 // 默认每页 10 个分类
|
||||
}
|
||||
|
||||
categoriesWithItems, totalCategories, err := h.manager.GetCategoriesWithItems(limit, offset)
|
||||
@@ -172,7 +172,7 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
|
||||
}
|
||||
|
||||
// 按项分页模式(向后兼容)
|
||||
// 是否包含完整内容(默认false,只返回摘要)
|
||||
// 是否包含完整内容(默认 false,只返回摘要)
|
||||
includeContent := c.Query("includeContent") == "true"
|
||||
|
||||
if includeContent {
|
||||
@@ -192,9 +192,9 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
|
||||
}
|
||||
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"items": items,
|
||||
"total": total,
|
||||
"limit": limit,
|
||||
"items": items,
|
||||
"total": total,
|
||||
"limit": limit,
|
||||
"offset": offset,
|
||||
})
|
||||
} else {
|
||||
@@ -207,9 +207,9 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
|
||||
}
|
||||
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"items": items,
|
||||
"total": total,
|
||||
"limit": limit,
|
||||
"items": items,
|
||||
"total": total,
|
||||
"limit": limit,
|
||||
"offset": offset,
|
||||
})
|
||||
}
|
||||
@@ -341,12 +341,12 @@ func (h *KnowledgeHandler) ScanKnowledgeBase(c *gin.Context) {
|
||||
consecutiveFailures := 0
|
||||
var firstFailureItemID string
|
||||
var firstFailureError error
|
||||
|
||||
|
||||
for i, itemID := range itemsToIndex {
|
||||
if err := h.indexer.IndexItem(ctx, itemID); err != nil {
|
||||
failedCount++
|
||||
consecutiveFailures++
|
||||
|
||||
|
||||
// 只在第一个失败时记录详细日志
|
||||
if consecutiveFailures == 1 {
|
||||
firstFailureItemID = itemID
|
||||
@@ -357,8 +357,8 @@ func (h *KnowledgeHandler) ScanKnowledgeBase(c *gin.Context) {
|
||||
zap.Error(err),
|
||||
)
|
||||
}
|
||||
|
||||
// 如果连续失败2次,立即停止增量索引
|
||||
|
||||
// 如果连续失败 2 次,立即停止增量索引
|
||||
if consecutiveFailures >= 2 {
|
||||
h.logger.Error("连续索引失败次数过多,立即停止增量索引",
|
||||
zap.Int("consecutiveFailures", consecutiveFailures),
|
||||
@@ -371,14 +371,14 @@ func (h *KnowledgeHandler) ScanKnowledgeBase(c *gin.Context) {
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
|
||||
// 成功时重置连续失败计数
|
||||
if consecutiveFailures > 0 {
|
||||
consecutiveFailures = 0
|
||||
firstFailureItemID = ""
|
||||
firstFailureError = nil
|
||||
}
|
||||
|
||||
|
||||
// 减少进度日志频率
|
||||
if (i+1)%10 == 0 || i+1 == len(itemsToIndex) {
|
||||
h.logger.Info("索引进度", zap.Int("current", i+1), zap.Int("total", len(itemsToIndex)), zap.Int("failed", failedCount))
|
||||
@@ -388,7 +388,7 @@ func (h *KnowledgeHandler) ScanKnowledgeBase(c *gin.Context) {
|
||||
}()
|
||||
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"message": fmt.Sprintf("扫描完成,开始索引 %d 个新添加或更新的知识项", len(itemsToIndex)),
|
||||
"message": fmt.Sprintf("扫描完成,开始索引 %d 个新添加或更新的知识项", len(itemsToIndex)),
|
||||
"items_to_index": len(itemsToIndex),
|
||||
})
|
||||
}
|
||||
@@ -397,7 +397,7 @@ func (h *KnowledgeHandler) ScanKnowledgeBase(c *gin.Context) {
|
||||
func (h *KnowledgeHandler) GetRetrievalLogs(c *gin.Context) {
|
||||
conversationID := c.Query("conversationId")
|
||||
messageID := c.Query("messageId")
|
||||
limit := 50 // 默认50条
|
||||
limit := 50 // 默认 50 条
|
||||
|
||||
if limitStr := c.Query("limit"); limitStr != "" {
|
||||
if parsed, err := parseInt(limitStr); err == nil && parsed > 0 {
|
||||
@@ -441,18 +441,40 @@ func (h *KnowledgeHandler) GetIndexStatus(c *gin.Context) {
|
||||
if h.indexer != nil {
|
||||
lastError, lastErrorTime := h.indexer.GetLastError()
|
||||
if lastError != "" {
|
||||
// 如果错误是最近发生的(5分钟内),则返回错误信息
|
||||
// 如果错误是最近发生的(5 分钟内),则返回错误信息
|
||||
if time.Since(lastErrorTime) < 5*time.Minute {
|
||||
status["last_error"] = lastError
|
||||
status["last_error_time"] = lastErrorTime.Format(time.RFC3339)
|
||||
}
|
||||
}
|
||||
|
||||
// 获取重建索引状态
|
||||
isRebuilding, totalItems, current, failed, lastItemID, lastChunks, startTime := h.indexer.GetRebuildStatus()
|
||||
if isRebuilding {
|
||||
status["is_rebuilding"] = true
|
||||
status["rebuild_total"] = totalItems
|
||||
status["rebuild_current"] = current
|
||||
status["rebuild_failed"] = failed
|
||||
status["rebuild_start_time"] = startTime.Format(time.RFC3339)
|
||||
if lastItemID != "" {
|
||||
status["rebuild_last_item_id"] = lastItemID
|
||||
}
|
||||
if lastChunks > 0 {
|
||||
status["rebuild_last_chunks"] = lastChunks
|
||||
}
|
||||
// 重建中时,is_complete 为 false
|
||||
status["is_complete"] = false
|
||||
// 计算重建进度百分比
|
||||
if totalItems > 0 {
|
||||
status["progress_percent"] = float64(current) / float64(totalItems) * 100
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
c.JSON(http.StatusOK, status)
|
||||
}
|
||||
|
||||
// Search 搜索知识库(用于API调用,Agent内部使用Retriever)
|
||||
// Search 搜索知识库(用于 API 调用,Agent 内部使用 Retriever)
|
||||
func (h *KnowledgeHandler) Search(c *gin.Context) {
|
||||
var req knowledge.SearchRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
@@ -470,10 +492,25 @@ func (h *KnowledgeHandler) Search(c *gin.Context) {
|
||||
c.JSON(http.StatusOK, gin.H{"results": results})
|
||||
}
|
||||
|
||||
// GetStats 获取知识库统计信息
|
||||
func (h *KnowledgeHandler) GetStats(c *gin.Context) {
|
||||
totalCategories, totalItems, err := h.manager.GetStats()
|
||||
if err != nil {
|
||||
h.logger.Error("获取知识库统计信息失败", zap.Error(err))
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"enabled": true,
|
||||
"total_categories": totalCategories,
|
||||
"total_items": totalItems,
|
||||
})
|
||||
}
|
||||
|
||||
// 辅助函数:解析整数
|
||||
func parseInt(s string) (int, error) {
|
||||
var result int
|
||||
_, err := fmt.Sscanf(s, "%d", &result)
|
||||
return result, err
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,593 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/aes"
|
||||
"crypto/cipher"
|
||||
"encoding/base64"
|
||||
"encoding/binary"
|
||||
"encoding/xml"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"cyberstrike-ai/internal/config"
|
||||
"cyberstrike-ai/internal/database"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"go.uber.org/zap"
|
||||
)
|
||||
|
||||
const (
|
||||
robotCmdHelp = "帮助"
|
||||
robotCmdList = "列表"
|
||||
robotCmdListAlt = "对话列表"
|
||||
robotCmdSwitch = "切换"
|
||||
robotCmdContinue = "继续"
|
||||
robotCmdNew = "新对话"
|
||||
robotCmdClear = "清空"
|
||||
robotCmdCurrent = "当前"
|
||||
robotCmdStop = "停止"
|
||||
robotCmdRoles = "角色"
|
||||
robotCmdRolesList = "角色列表"
|
||||
robotCmdSwitchRole = "切换角色"
|
||||
robotCmdDelete = "删除"
|
||||
robotCmdVersion = "版本"
|
||||
)
|
||||
|
||||
// RobotHandler 企业微信/钉钉/飞书等机器人回调处理
|
||||
type RobotHandler struct {
|
||||
config *config.Config
|
||||
db *database.DB
|
||||
agentHandler *AgentHandler
|
||||
logger *zap.Logger
|
||||
mu sync.RWMutex
|
||||
sessions map[string]string // key: "platform_userID", value: conversationID
|
||||
sessionRoles map[string]string // key: "platform_userID", value: roleName(默认"默认")
|
||||
cancelMu sync.Mutex // 保护 runningCancels
|
||||
runningCancels map[string]context.CancelFunc // key: "platform_userID", 用于停止命令中断任务
|
||||
}
|
||||
|
||||
// NewRobotHandler 创建机器人处理器
|
||||
func NewRobotHandler(cfg *config.Config, db *database.DB, agentHandler *AgentHandler, logger *zap.Logger) *RobotHandler {
|
||||
return &RobotHandler{
|
||||
config: cfg,
|
||||
db: db,
|
||||
agentHandler: agentHandler,
|
||||
logger: logger,
|
||||
sessions: make(map[string]string),
|
||||
sessionRoles: make(map[string]string),
|
||||
runningCancels: make(map[string]context.CancelFunc),
|
||||
}
|
||||
}
|
||||
|
||||
// sessionKey 生成会话 key
|
||||
func (h *RobotHandler) sessionKey(platform, userID string) string {
|
||||
return platform + "_" + userID
|
||||
}
|
||||
|
||||
// getOrCreateConversation 获取或创建当前会话,title 用于新对话的标题(取用户首条消息前50字)
|
||||
func (h *RobotHandler) getOrCreateConversation(platform, userID, title string) (convID string, isNew bool) {
|
||||
h.mu.RLock()
|
||||
convID = h.sessions[h.sessionKey(platform, userID)]
|
||||
h.mu.RUnlock()
|
||||
if convID != "" {
|
||||
return convID, false
|
||||
}
|
||||
t := strings.TrimSpace(title)
|
||||
if t == "" {
|
||||
t = "新对话 " + time.Now().Format("01-02 15:04")
|
||||
} else {
|
||||
t = safeTruncateString(t, 50)
|
||||
}
|
||||
conv, err := h.db.CreateConversation(t)
|
||||
if err != nil {
|
||||
h.logger.Warn("创建机器人会话失败", zap.Error(err))
|
||||
return "", false
|
||||
}
|
||||
convID = conv.ID
|
||||
h.mu.Lock()
|
||||
h.sessions[h.sessionKey(platform, userID)] = convID
|
||||
h.mu.Unlock()
|
||||
return convID, true
|
||||
}
|
||||
|
||||
// setConversation 切换当前会话
|
||||
func (h *RobotHandler) setConversation(platform, userID, convID string) {
|
||||
h.mu.Lock()
|
||||
h.sessions[h.sessionKey(platform, userID)] = convID
|
||||
h.mu.Unlock()
|
||||
}
|
||||
|
||||
// getRole 获取当前用户使用的角色,未设置时返回"默认"
|
||||
func (h *RobotHandler) getRole(platform, userID string) string {
|
||||
h.mu.RLock()
|
||||
role := h.sessionRoles[h.sessionKey(platform, userID)]
|
||||
h.mu.RUnlock()
|
||||
if role == "" {
|
||||
return "默认"
|
||||
}
|
||||
return role
|
||||
}
|
||||
|
||||
// setRole 设置当前用户使用的角色
|
||||
func (h *RobotHandler) setRole(platform, userID, roleName string) {
|
||||
h.mu.Lock()
|
||||
h.sessionRoles[h.sessionKey(platform, userID)] = roleName
|
||||
h.mu.Unlock()
|
||||
}
|
||||
|
||||
// clearConversation 清空当前会话(切换到新对话)
|
||||
func (h *RobotHandler) clearConversation(platform, userID string) (newConvID string) {
|
||||
title := "新对话 " + time.Now().Format("01-02 15:04")
|
||||
conv, err := h.db.CreateConversation(title)
|
||||
if err != nil {
|
||||
h.logger.Warn("创建新对话失败", zap.Error(err))
|
||||
return ""
|
||||
}
|
||||
h.setConversation(platform, userID, conv.ID)
|
||||
return conv.ID
|
||||
}
|
||||
|
||||
// HandleMessage 处理用户输入,返回回复文本(供各平台 webhook 调用)
|
||||
func (h *RobotHandler) HandleMessage(platform, userID, text string) (reply string) {
|
||||
text = strings.TrimSpace(text)
|
||||
if text == "" {
|
||||
return "请输入内容或发送「帮助」/ help 查看命令。"
|
||||
}
|
||||
|
||||
// 命令分发(支持中英文)
|
||||
switch {
|
||||
case text == robotCmdHelp || text == "help" || text == "?" || text == "?":
|
||||
return h.cmdHelp()
|
||||
case text == robotCmdList || text == robotCmdListAlt || text == "list":
|
||||
return h.cmdList()
|
||||
case strings.HasPrefix(text, robotCmdSwitch+" ") || strings.HasPrefix(text, robotCmdContinue+" ") || strings.HasPrefix(text, "switch ") || strings.HasPrefix(text, "continue "):
|
||||
var id string
|
||||
switch {
|
||||
case strings.HasPrefix(text, robotCmdSwitch+" "):
|
||||
id = strings.TrimSpace(text[len(robotCmdSwitch)+1:])
|
||||
case strings.HasPrefix(text, robotCmdContinue+" "):
|
||||
id = strings.TrimSpace(text[len(robotCmdContinue)+1:])
|
||||
case strings.HasPrefix(text, "switch "):
|
||||
id = strings.TrimSpace(text[7:])
|
||||
default:
|
||||
id = strings.TrimSpace(text[9:])
|
||||
}
|
||||
return h.cmdSwitch(platform, userID, id)
|
||||
case text == robotCmdNew || text == "new":
|
||||
return h.cmdNew(platform, userID)
|
||||
case text == robotCmdClear || text == "clear":
|
||||
return h.cmdClear(platform, userID)
|
||||
case text == robotCmdCurrent || text == "current":
|
||||
return h.cmdCurrent(platform, userID)
|
||||
case text == robotCmdStop || text == "stop":
|
||||
return h.cmdStop(platform, userID)
|
||||
case text == robotCmdRoles || text == robotCmdRolesList || text == "roles":
|
||||
return h.cmdRoles()
|
||||
case strings.HasPrefix(text, robotCmdRoles+" ") || strings.HasPrefix(text, robotCmdSwitchRole+" ") || strings.HasPrefix(text, "role "):
|
||||
var roleName string
|
||||
switch {
|
||||
case strings.HasPrefix(text, robotCmdRoles+" "):
|
||||
roleName = strings.TrimSpace(text[len(robotCmdRoles)+1:])
|
||||
case strings.HasPrefix(text, robotCmdSwitchRole+" "):
|
||||
roleName = strings.TrimSpace(text[len(robotCmdSwitchRole)+1:])
|
||||
default:
|
||||
roleName = strings.TrimSpace(text[5:])
|
||||
}
|
||||
return h.cmdSwitchRole(platform, userID, roleName)
|
||||
case strings.HasPrefix(text, robotCmdDelete+" ") || strings.HasPrefix(text, "delete "):
|
||||
var convID string
|
||||
if strings.HasPrefix(text, robotCmdDelete+" ") {
|
||||
convID = strings.TrimSpace(text[len(robotCmdDelete)+1:])
|
||||
} else {
|
||||
convID = strings.TrimSpace(text[7:])
|
||||
}
|
||||
return h.cmdDelete(platform, userID, convID)
|
||||
case text == robotCmdVersion || text == "version":
|
||||
return h.cmdVersion()
|
||||
}
|
||||
|
||||
// 普通消息:走 Agent
|
||||
convID, _ := h.getOrCreateConversation(platform, userID, text)
|
||||
if convID == "" {
|
||||
return "无法创建或获取对话,请稍后再试。"
|
||||
}
|
||||
// 若对话标题为「新对话 xx:xx」格式(由「新对话」命令创建),将标题更新为首条消息内容,与 Web 端体验一致
|
||||
if conv, err := h.db.GetConversation(convID); err == nil && strings.HasPrefix(conv.Title, "新对话 ") {
|
||||
newTitle := safeTruncateString(text, 50)
|
||||
if newTitle != "" {
|
||||
_ = h.db.UpdateConversationTitle(convID, newTitle)
|
||||
}
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
|
||||
sk := h.sessionKey(platform, userID)
|
||||
h.cancelMu.Lock()
|
||||
h.runningCancels[sk] = cancel
|
||||
h.cancelMu.Unlock()
|
||||
defer func() {
|
||||
cancel()
|
||||
h.cancelMu.Lock()
|
||||
delete(h.runningCancels, sk)
|
||||
h.cancelMu.Unlock()
|
||||
}()
|
||||
role := h.getRole(platform, userID)
|
||||
resp, newConvID, err := h.agentHandler.ProcessMessageForRobot(ctx, convID, text, role)
|
||||
if err != nil {
|
||||
h.logger.Warn("机器人 Agent 执行失败", zap.String("platform", platform), zap.String("userID", userID), zap.Error(err))
|
||||
if errors.Is(err, context.Canceled) {
|
||||
return "任务已取消。"
|
||||
}
|
||||
return "处理失败: " + err.Error()
|
||||
}
|
||||
if newConvID != convID {
|
||||
h.setConversation(platform, userID, newConvID)
|
||||
}
|
||||
return resp
|
||||
}
|
||||
|
||||
func (h *RobotHandler) cmdHelp() string {
|
||||
return "**【CyberStrikeAI 机器人命令】**\n\n" +
|
||||
"- `帮助` `help` — 显示本帮助 | Show this help\n" +
|
||||
"- `列表` `list` — 列出所有对话标题与 ID | List conversations\n" +
|
||||
"- `切换 <ID>` `switch <ID>` — 指定对话继续 | Switch to conversation\n" +
|
||||
"- `新对话` `new` — 开启新对话 | Start new conversation\n" +
|
||||
"- `清空` `clear` — 清空当前上下文 | Clear context\n" +
|
||||
"- `当前` `current` — 显示当前对话 ID 与标题 | Show current conversation\n" +
|
||||
"- `停止` `stop` — 中断当前任务 | Stop running task\n" +
|
||||
"- `角色` `roles` — 列出所有可用角色 | List roles\n" +
|
||||
"- `角色 <名>` `role <name>` — 切换当前角色 | Switch role\n" +
|
||||
"- `删除 <ID>` `delete <ID>` — 删除指定对话 | Delete conversation\n" +
|
||||
"- `版本` `version` — 显示当前版本号 | Show version\n\n" +
|
||||
"---\n" +
|
||||
"除以上命令外,直接输入内容将发送给 AI 进行渗透测试/安全分析。\n" +
|
||||
"Otherwise, send any text for AI penetration testing / security analysis."
|
||||
}
|
||||
|
||||
func (h *RobotHandler) cmdList() string {
|
||||
convs, err := h.db.ListConversations(50, 0, "")
|
||||
if err != nil {
|
||||
return "获取对话列表失败: " + err.Error()
|
||||
}
|
||||
if len(convs) == 0 {
|
||||
return "暂无对话。发送任意内容将自动创建新对话。"
|
||||
}
|
||||
var b strings.Builder
|
||||
b.WriteString("【对话列表】\n")
|
||||
for i, c := range convs {
|
||||
if i >= 20 {
|
||||
b.WriteString("… 仅显示前 20 条\n")
|
||||
break
|
||||
}
|
||||
b.WriteString(fmt.Sprintf("· %s\n ID: %s\n", c.Title, c.ID))
|
||||
}
|
||||
return strings.TrimSuffix(b.String(), "\n")
|
||||
}
|
||||
|
||||
func (h *RobotHandler) cmdSwitch(platform, userID, convID string) string {
|
||||
if convID == "" {
|
||||
return "请指定对话 ID,例如:切换 xxx-xxx-xxx"
|
||||
}
|
||||
conv, err := h.db.GetConversation(convID)
|
||||
if err != nil {
|
||||
return "对话不存在或 ID 错误。"
|
||||
}
|
||||
h.setConversation(platform, userID, conv.ID)
|
||||
return fmt.Sprintf("已切换到对话:「%s」\nID: %s", conv.Title, conv.ID)
|
||||
}
|
||||
|
||||
func (h *RobotHandler) cmdNew(platform, userID string) string {
|
||||
newID := h.clearConversation(platform, userID)
|
||||
if newID == "" {
|
||||
return "创建新对话失败,请重试。"
|
||||
}
|
||||
return "已开启新对话,可直接发送内容。"
|
||||
}
|
||||
|
||||
func (h *RobotHandler) cmdClear(platform, userID string) string {
|
||||
return h.cmdNew(platform, userID)
|
||||
}
|
||||
|
||||
func (h *RobotHandler) cmdStop(platform, userID string) string {
|
||||
sk := h.sessionKey(platform, userID)
|
||||
h.cancelMu.Lock()
|
||||
cancel, ok := h.runningCancels[sk]
|
||||
if ok {
|
||||
delete(h.runningCancels, sk)
|
||||
cancel()
|
||||
}
|
||||
h.cancelMu.Unlock()
|
||||
if !ok {
|
||||
return "当前没有正在执行的任务。"
|
||||
}
|
||||
return "已停止当前任务。"
|
||||
}
|
||||
|
||||
func (h *RobotHandler) cmdCurrent(platform, userID string) string {
|
||||
h.mu.RLock()
|
||||
convID := h.sessions[h.sessionKey(platform, userID)]
|
||||
h.mu.RUnlock()
|
||||
if convID == "" {
|
||||
return "当前没有进行中的对话。发送任意内容将创建新对话。"
|
||||
}
|
||||
conv, err := h.db.GetConversation(convID)
|
||||
if err != nil {
|
||||
return "当前对话 ID: " + convID + "(获取标题失败)"
|
||||
}
|
||||
role := h.getRole(platform, userID)
|
||||
return fmt.Sprintf("当前对话:「%s」\nID: %s\n当前角色: %s", conv.Title, conv.ID, role)
|
||||
}
|
||||
|
||||
func (h *RobotHandler) cmdRoles() string {
|
||||
if h.config.Roles == nil || len(h.config.Roles) == 0 {
|
||||
return "暂无可用角色。"
|
||||
}
|
||||
names := make([]string, 0, len(h.config.Roles))
|
||||
for name, role := range h.config.Roles {
|
||||
if role.Enabled {
|
||||
names = append(names, name)
|
||||
}
|
||||
}
|
||||
if len(names) == 0 {
|
||||
return "暂无可用角色。"
|
||||
}
|
||||
sort.Slice(names, func(i, j int) bool {
|
||||
if names[i] == "默认" {
|
||||
return true
|
||||
}
|
||||
if names[j] == "默认" {
|
||||
return false
|
||||
}
|
||||
return names[i] < names[j]
|
||||
})
|
||||
var b strings.Builder
|
||||
b.WriteString("【角色列表】\n")
|
||||
for _, name := range names {
|
||||
role := h.config.Roles[name]
|
||||
desc := role.Description
|
||||
if desc == "" {
|
||||
desc = "无描述"
|
||||
}
|
||||
b.WriteString(fmt.Sprintf("· %s — %s\n", name, desc))
|
||||
}
|
||||
return strings.TrimSuffix(b.String(), "\n")
|
||||
}
|
||||
|
||||
func (h *RobotHandler) cmdSwitchRole(platform, userID, roleName string) string {
|
||||
if roleName == "" {
|
||||
return "请指定角色名称,例如:角色 渗透测试"
|
||||
}
|
||||
if h.config.Roles == nil {
|
||||
return "暂无可用角色。"
|
||||
}
|
||||
role, exists := h.config.Roles[roleName]
|
||||
if !exists {
|
||||
return fmt.Sprintf("角色「%s」不存在。发送「角色」查看可用角色。", roleName)
|
||||
}
|
||||
if !role.Enabled {
|
||||
return fmt.Sprintf("角色「%s」已禁用。", roleName)
|
||||
}
|
||||
h.setRole(platform, userID, roleName)
|
||||
return fmt.Sprintf("已切换到角色:「%s」\n%s", roleName, role.Description)
|
||||
}
|
||||
|
||||
func (h *RobotHandler) cmdDelete(platform, userID, convID string) string {
|
||||
if convID == "" {
|
||||
return "请指定对话 ID,例如:删除 xxx-xxx-xxx"
|
||||
}
|
||||
sk := h.sessionKey(platform, userID)
|
||||
h.mu.RLock()
|
||||
currentConvID := h.sessions[sk]
|
||||
h.mu.RUnlock()
|
||||
if convID == currentConvID {
|
||||
// 删除当前对话时,先清空会话绑定
|
||||
h.mu.Lock()
|
||||
delete(h.sessions, sk)
|
||||
h.mu.Unlock()
|
||||
}
|
||||
if err := h.db.DeleteConversation(convID); err != nil {
|
||||
return "删除失败: " + err.Error()
|
||||
}
|
||||
return fmt.Sprintf("已删除对话 ID: %s", convID)
|
||||
}
|
||||
|
||||
func (h *RobotHandler) cmdVersion() string {
|
||||
v := h.config.Version
|
||||
if v == "" {
|
||||
v = "未知"
|
||||
}
|
||||
return "CyberStrikeAI " + v
|
||||
}
|
||||
|
||||
// —————— 企业微信 ——————
|
||||
|
||||
// wecomXML 企业微信回调 XML(明文模式下的简化结构;加密模式需先解密再解析)
|
||||
type wecomXML struct {
|
||||
ToUserName string `xml:"ToUserName"`
|
||||
FromUserName string `xml:"FromUserName"`
|
||||
CreateTime int64 `xml:"CreateTime"`
|
||||
MsgType string `xml:"MsgType"`
|
||||
Content string `xml:"Content"`
|
||||
MsgID string `xml:"MsgId"`
|
||||
AgentID int64 `xml:"AgentID"`
|
||||
Encrypt string `xml:"Encrypt"` // 加密模式下消息在此
|
||||
}
|
||||
|
||||
// wecomReplyXML 被动回复 XML
|
||||
type wecomReplyXML struct {
|
||||
XMLName xml.Name `xml:"xml"`
|
||||
ToUserName string `xml:"ToUserName"`
|
||||
FromUserName string `xml:"FromUserName"`
|
||||
CreateTime int64 `xml:"CreateTime"`
|
||||
MsgType string `xml:"MsgType"`
|
||||
Content string `xml:"Content"`
|
||||
}
|
||||
|
||||
// HandleWecomGET 企业微信 URL 校验(GET)
|
||||
func (h *RobotHandler) HandleWecomGET(c *gin.Context) {
|
||||
if !h.config.Robots.Wecom.Enabled {
|
||||
c.String(http.StatusNotFound, "")
|
||||
return
|
||||
}
|
||||
echostr := c.Query("echostr")
|
||||
if echostr == "" {
|
||||
c.String(http.StatusBadRequest, "missing echostr")
|
||||
return
|
||||
}
|
||||
// 明文模式时企业微信可能直接传 echostr,先直接返回以通过校验
|
||||
c.String(http.StatusOK, echostr)
|
||||
}
|
||||
|
||||
// wecomDecrypt 企业微信消息解密(AES-256-CBC,PKCS7,明文格式:16字节随机+4字节长度+消息+corpID)
|
||||
func wecomDecrypt(encodingAESKey, encryptedB64 string) ([]byte, error) {
|
||||
key, err := base64.StdEncoding.DecodeString(encodingAESKey + "=")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(key) != 32 {
|
||||
return nil, fmt.Errorf("encoding_aes_key 解码后应为 32 字节")
|
||||
}
|
||||
ciphertext, err := base64.StdEncoding.DecodeString(encryptedB64)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
block, err := aes.NewCipher(key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
iv := key[:16]
|
||||
mode := cipher.NewCBCDecrypter(block, iv)
|
||||
if len(ciphertext)%aes.BlockSize != 0 {
|
||||
return nil, fmt.Errorf("密文长度不是块大小的倍数")
|
||||
}
|
||||
plain := make([]byte, len(ciphertext))
|
||||
mode.CryptBlocks(plain, ciphertext)
|
||||
// 去除 PKCS7 填充
|
||||
n := int(plain[len(plain)-1])
|
||||
if n < 1 || n > 32 {
|
||||
return nil, fmt.Errorf("无效的 PKCS7 填充")
|
||||
}
|
||||
plain = plain[:len(plain)-n]
|
||||
// 企业微信格式:16 字节随机 + 4 字节长度(大端) + 消息 + corpID
|
||||
if len(plain) < 20 {
|
||||
return nil, fmt.Errorf("明文过短")
|
||||
}
|
||||
msgLen := binary.BigEndian.Uint32(plain[16:20])
|
||||
if int(20+msgLen) > len(plain) {
|
||||
return nil, fmt.Errorf("消息长度越界")
|
||||
}
|
||||
return plain[20 : 20+msgLen], nil
|
||||
}
|
||||
|
||||
// HandleWecomPOST 企业微信消息回调(POST),支持明文与加密模式
|
||||
func (h *RobotHandler) HandleWecomPOST(c *gin.Context) {
|
||||
if !h.config.Robots.Wecom.Enabled {
|
||||
c.String(http.StatusOK, "")
|
||||
return
|
||||
}
|
||||
bodyRaw, _ := io.ReadAll(c.Request.Body)
|
||||
var body wecomXML
|
||||
if err := xml.Unmarshal(bodyRaw, &body); err != nil {
|
||||
h.logger.Debug("企业微信 POST 解析 XML 失败", zap.Error(err))
|
||||
c.String(http.StatusOK, "")
|
||||
return
|
||||
}
|
||||
// 加密模式:先解密再解析内层 XML
|
||||
if body.Encrypt != "" && h.config.Robots.Wecom.EncodingAESKey != "" {
|
||||
decrypted, err := wecomDecrypt(h.config.Robots.Wecom.EncodingAESKey, body.Encrypt)
|
||||
if err != nil {
|
||||
h.logger.Warn("企业微信消息解密失败", zap.Error(err))
|
||||
c.String(http.StatusOK, "")
|
||||
return
|
||||
}
|
||||
if err := xml.Unmarshal(decrypted, &body); err != nil {
|
||||
h.logger.Warn("企业微信解密后 XML 解析失败", zap.Error(err))
|
||||
c.String(http.StatusOK, "")
|
||||
return
|
||||
}
|
||||
}
|
||||
if body.MsgType != "text" {
|
||||
c.XML(http.StatusOK, wecomReplyXML{
|
||||
ToUserName: body.FromUserName,
|
||||
FromUserName: body.ToUserName,
|
||||
CreateTime: time.Now().Unix(),
|
||||
MsgType: "text",
|
||||
Content: "暂仅支持文本消息,请发送文字。",
|
||||
})
|
||||
return
|
||||
}
|
||||
userID := body.FromUserName
|
||||
text := strings.TrimSpace(body.Content)
|
||||
reply := h.HandleMessage("wecom", userID, text)
|
||||
// 加密模式需加密回复(此处简化为明文回复;若企业要求加密需再实现加密)
|
||||
c.XML(http.StatusOK, wecomReplyXML{
|
||||
ToUserName: body.FromUserName,
|
||||
FromUserName: body.ToUserName,
|
||||
CreateTime: time.Now().Unix(),
|
||||
MsgType: "text",
|
||||
Content: reply,
|
||||
})
|
||||
}
|
||||
|
||||
// —————— 测试接口(需登录,用于验证机器人逻辑,无需钉钉/飞书客户端) ——————
|
||||
|
||||
// RobotTestRequest 模拟机器人消息请求
|
||||
type RobotTestRequest struct {
|
||||
Platform string `json:"platform"` // 如 "dingtalk"、"lark"、"wecom"
|
||||
UserID string `json:"user_id"`
|
||||
Text string `json:"text"`
|
||||
}
|
||||
|
||||
// HandleRobotTest 供本地验证:POST JSON { "platform", "user_id", "text" },返回 { "reply": "..." }
|
||||
func (h *RobotHandler) HandleRobotTest(c *gin.Context) {
|
||||
var req RobotTestRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "请求体需为 JSON,包含 platform、user_id、text"})
|
||||
return
|
||||
}
|
||||
platform := strings.TrimSpace(req.Platform)
|
||||
if platform == "" {
|
||||
platform = "test"
|
||||
}
|
||||
userID := strings.TrimSpace(req.UserID)
|
||||
if userID == "" {
|
||||
userID = "test_user"
|
||||
}
|
||||
reply := h.HandleMessage(platform, userID, req.Text)
|
||||
c.JSON(http.StatusOK, gin.H{"reply": reply})
|
||||
}
|
||||
|
||||
// —————— 钉钉 ——————
|
||||
|
||||
// HandleDingtalkPOST 钉钉事件回调(流式接入等);当前为占位,返回 200
|
||||
func (h *RobotHandler) HandleDingtalkPOST(c *gin.Context) {
|
||||
if !h.config.Robots.Dingtalk.Enabled {
|
||||
c.JSON(http.StatusOK, gin.H{})
|
||||
return
|
||||
}
|
||||
// 钉钉流式/事件回调格式需按官方文档解析并异步回复,此处仅返回 200
|
||||
c.JSON(http.StatusOK, gin.H{"message": "ok"})
|
||||
}
|
||||
|
||||
// —————— 飞书 ——————
|
||||
|
||||
// HandleLarkPOST 飞书事件回调;当前为占位,返回 200;验证时需返回 challenge
|
||||
func (h *RobotHandler) HandleLarkPOST(c *gin.Context) {
|
||||
if !h.config.Robots.Lark.Enabled {
|
||||
c.JSON(http.StatusOK, gin.H{})
|
||||
return
|
||||
}
|
||||
var body struct {
|
||||
Challenge string `json:"challenge"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&body); err == nil && body.Challenge != "" {
|
||||
c.JSON(http.StatusOK, gin.H{"challenge": body.Challenge})
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{})
|
||||
}
|
||||
@@ -0,0 +1,257 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"go.uber.org/zap"
|
||||
)
|
||||
|
||||
const (
|
||||
terminalMaxCommandLen = 4096
|
||||
terminalMaxOutputLen = 256 * 1024 // 256KB
|
||||
terminalTimeout = 120 * time.Second
|
||||
)
|
||||
|
||||
// TerminalHandler 处理系统设置中的终端命令执行
|
||||
type TerminalHandler struct {
|
||||
logger *zap.Logger
|
||||
}
|
||||
|
||||
// maskTerminalCommand 对可能包含敏感信息的终端命令做脱敏,避免在日志中直接记录密码等内容
|
||||
func maskTerminalCommand(cmd string) string {
|
||||
trimmed := strings.TrimSpace(cmd)
|
||||
lower := strings.ToLower(trimmed)
|
||||
if strings.Contains(lower, "sudo") || strings.Contains(lower, "password") {
|
||||
return "[masked sensitive terminal command]"
|
||||
}
|
||||
if len(trimmed) > 256 {
|
||||
return trimmed[:256] + "..."
|
||||
}
|
||||
return trimmed
|
||||
}
|
||||
|
||||
// NewTerminalHandler 创建终端处理器
|
||||
func NewTerminalHandler(logger *zap.Logger) *TerminalHandler {
|
||||
return &TerminalHandler{logger: logger}
|
||||
}
|
||||
|
||||
// RunCommandRequest 执行命令请求
|
||||
type RunCommandRequest struct {
|
||||
Command string `json:"command"`
|
||||
Shell string `json:"shell,omitempty"`
|
||||
Cwd string `json:"cwd,omitempty"`
|
||||
}
|
||||
|
||||
// RunCommandResponse 执行命令响应
|
||||
type RunCommandResponse struct {
|
||||
Stdout string `json:"stdout"`
|
||||
Stderr string `json:"stderr"`
|
||||
ExitCode int `json:"exit_code"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// RunCommand 执行终端命令(需登录)
|
||||
func (h *TerminalHandler) RunCommand(c *gin.Context) {
|
||||
var req RunCommandRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "请求体无效,需要 command 字段"})
|
||||
return
|
||||
}
|
||||
|
||||
cmdStr := strings.TrimSpace(req.Command)
|
||||
if cmdStr == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "command 不能为空"})
|
||||
return
|
||||
}
|
||||
if len(cmdStr) > terminalMaxCommandLen {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "命令过长"})
|
||||
return
|
||||
}
|
||||
|
||||
shell := req.Shell
|
||||
if shell == "" {
|
||||
if runtime.GOOS == "windows" {
|
||||
shell = "cmd"
|
||||
} else {
|
||||
shell = "sh"
|
||||
}
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(c.Request.Context(), terminalTimeout)
|
||||
defer cancel()
|
||||
|
||||
var cmd *exec.Cmd
|
||||
if runtime.GOOS == "windows" {
|
||||
cmd = exec.CommandContext(ctx, "cmd", "/c", cmdStr)
|
||||
} else {
|
||||
cmd = exec.CommandContext(ctx, shell, "-c", cmdStr)
|
||||
// 无 TTY 时设置 COLUMNS/TERM,使 ping 等工具的 usage 排版与真实终端一致
|
||||
cmd.Env = append(os.Environ(), "COLUMNS=256", "LINES=40", "TERM=xterm-256color")
|
||||
}
|
||||
|
||||
if req.Cwd != "" {
|
||||
absCwd, err := filepath.Abs(req.Cwd)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "工作目录无效"})
|
||||
return
|
||||
}
|
||||
cur, _ := os.Getwd()
|
||||
curAbs, _ := filepath.Abs(cur)
|
||||
rel, err := filepath.Rel(curAbs, absCwd)
|
||||
if err != nil || strings.HasPrefix(rel, "..") || rel == ".." {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "工作目录必须在当前进程目录下"})
|
||||
return
|
||||
}
|
||||
cmd.Dir = absCwd
|
||||
}
|
||||
|
||||
var stdout, stderr bytes.Buffer
|
||||
cmd.Stdout = &stdout
|
||||
cmd.Stderr = &stderr
|
||||
|
||||
err := cmd.Run()
|
||||
stdoutBytes := stdout.Bytes()
|
||||
stderrBytes := stderr.Bytes()
|
||||
|
||||
// 限制输出长度,防止内存占用过大(复制后截断,避免修改原 buffer)
|
||||
truncSuffix := []byte("\n...(输出已截断)\n")
|
||||
if len(stdoutBytes) > terminalMaxOutputLen {
|
||||
tmp := make([]byte, terminalMaxOutputLen+len(truncSuffix))
|
||||
n := copy(tmp, stdoutBytes[:terminalMaxOutputLen])
|
||||
copy(tmp[n:], truncSuffix)
|
||||
stdoutBytes = tmp
|
||||
}
|
||||
if len(stderrBytes) > terminalMaxOutputLen {
|
||||
tmp := make([]byte, terminalMaxOutputLen+len(truncSuffix))
|
||||
n := copy(tmp, stderrBytes[:terminalMaxOutputLen])
|
||||
copy(tmp[n:], truncSuffix)
|
||||
stderrBytes = tmp
|
||||
}
|
||||
|
||||
exitCode := 0
|
||||
if err != nil {
|
||||
if exitErr, ok := err.(*exec.ExitError); ok {
|
||||
exitCode = exitErr.ExitCode()
|
||||
} else {
|
||||
exitCode = -1
|
||||
}
|
||||
if ctx.Err() == context.DeadlineExceeded {
|
||||
so := strings.ReplaceAll(string(stdoutBytes), "\r\n", "\n")
|
||||
so = strings.ReplaceAll(so, "\r", "\n")
|
||||
se := strings.ReplaceAll(string(stderrBytes), "\r\n", "\n")
|
||||
se = strings.ReplaceAll(se, "\r", "\n")
|
||||
resp := RunCommandResponse{
|
||||
Stdout: so,
|
||||
Stderr: se,
|
||||
ExitCode: -1,
|
||||
Error: "命令执行超时(" + terminalTimeout.String() + ")",
|
||||
}
|
||||
c.JSON(http.StatusOK, resp)
|
||||
return
|
||||
}
|
||||
h.logger.Debug("终端命令执行异常", zap.String("command", maskTerminalCommand(cmdStr)), zap.Error(err))
|
||||
}
|
||||
|
||||
// 统一为 \n,避免前端因 \r 出现错位/对角线排版
|
||||
stdoutStr := strings.ReplaceAll(string(stdoutBytes), "\r\n", "\n")
|
||||
stdoutStr = strings.ReplaceAll(stdoutStr, "\r", "\n")
|
||||
stderrStr := strings.ReplaceAll(string(stderrBytes), "\r\n", "\n")
|
||||
stderrStr = strings.ReplaceAll(stderrStr, "\r", "\n")
|
||||
|
||||
resp := RunCommandResponse{
|
||||
Stdout: stdoutStr,
|
||||
Stderr: stderrStr,
|
||||
ExitCode: exitCode,
|
||||
}
|
||||
if err != nil && exitCode != 0 {
|
||||
resp.Error = err.Error()
|
||||
}
|
||||
c.JSON(http.StatusOK, resp)
|
||||
}
|
||||
|
||||
// streamEvent SSE 事件
|
||||
type streamEvent struct {
|
||||
T string `json:"t"` // "out" | "err" | "exit"
|
||||
D string `json:"d,omitempty"`
|
||||
C int `json:"c"` // exit code(不用 omitempty,否则 0 不序列化导致前端显示 [exit undefined])
|
||||
}
|
||||
|
||||
// RunCommandStream 流式执行命令,输出实时推送到前端(SSE)
|
||||
func (h *TerminalHandler) RunCommandStream(c *gin.Context) {
|
||||
var req RunCommandRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "请求体无效,需要 command 字段"})
|
||||
return
|
||||
}
|
||||
cmdStr := strings.TrimSpace(req.Command)
|
||||
if cmdStr == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "command 不能为空"})
|
||||
return
|
||||
}
|
||||
if len(cmdStr) > terminalMaxCommandLen {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "命令过长"})
|
||||
return
|
||||
}
|
||||
shell := req.Shell
|
||||
if shell == "" {
|
||||
if runtime.GOOS == "windows" {
|
||||
shell = "cmd"
|
||||
} else {
|
||||
shell = "sh"
|
||||
}
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(c.Request.Context(), terminalTimeout)
|
||||
defer cancel()
|
||||
|
||||
var cmd *exec.Cmd
|
||||
if runtime.GOOS == "windows" {
|
||||
cmd = exec.CommandContext(ctx, "cmd", "/c", cmdStr)
|
||||
} else {
|
||||
cmd = exec.CommandContext(ctx, shell, "-c", cmdStr)
|
||||
cmd.Env = append(os.Environ(), "COLUMNS=256", "LINES=40", "TERM=xterm-256color")
|
||||
}
|
||||
if req.Cwd != "" {
|
||||
absCwd, err := filepath.Abs(req.Cwd)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "工作目录无效"})
|
||||
return
|
||||
}
|
||||
cur, _ := os.Getwd()
|
||||
curAbs, _ := filepath.Abs(cur)
|
||||
rel, err := filepath.Rel(curAbs, absCwd)
|
||||
if err != nil || strings.HasPrefix(rel, "..") || rel == ".." {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "工作目录必须在当前进程目录下"})
|
||||
return
|
||||
}
|
||||
cmd.Dir = absCwd
|
||||
}
|
||||
|
||||
c.Header("Content-Type", "text/event-stream")
|
||||
c.Header("Cache-Control", "no-cache")
|
||||
c.Header("Connection", "keep-alive")
|
||||
c.Header("X-Accel-Buffering", "no")
|
||||
c.Writer.WriteHeader(http.StatusOK)
|
||||
flusher, ok := c.Writer.(http.Flusher)
|
||||
if !ok {
|
||||
cancel()
|
||||
return
|
||||
}
|
||||
|
||||
sendEvent := func(ev streamEvent) {
|
||||
body, _ := json.Marshal(ev)
|
||||
c.SSEvent("", string(body))
|
||||
flusher.Flush()
|
||||
}
|
||||
|
||||
runCommandStreamImpl(cmd, sendEvent, ctx)
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
//go:build !windows
|
||||
|
||||
package handler
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"os/exec"
|
||||
"strings"
|
||||
|
||||
"github.com/creack/pty"
|
||||
)
|
||||
|
||||
const ptyCols = 256
|
||||
const ptyRows = 40
|
||||
|
||||
// runCommandStreamImpl 在 Unix 下用 PTY 执行,使 ping 等命令按终端宽度排版(isatty 为真)
|
||||
func runCommandStreamImpl(cmd *exec.Cmd, sendEvent func(streamEvent), ctx context.Context) {
|
||||
ptmx, err := pty.StartWithSize(cmd, &pty.Winsize{Cols: ptyCols, Rows: ptyRows})
|
||||
if err != nil {
|
||||
sendEvent(streamEvent{T: "exit", C: -1})
|
||||
return
|
||||
}
|
||||
defer ptmx.Close()
|
||||
|
||||
normalize := func(s string) string {
|
||||
s = strings.ReplaceAll(s, "\r\n", "\n")
|
||||
return strings.ReplaceAll(s, "\r", "\n")
|
||||
}
|
||||
sc := bufio.NewScanner(ptmx)
|
||||
for sc.Scan() {
|
||||
sendEvent(streamEvent{T: "out", D: normalize(sc.Text())})
|
||||
}
|
||||
exitCode := 0
|
||||
if err := cmd.Wait(); err != nil {
|
||||
if exitErr, ok := err.(*exec.ExitError); ok {
|
||||
exitCode = exitErr.ExitCode()
|
||||
} else {
|
||||
exitCode = -1
|
||||
}
|
||||
}
|
||||
if ctx.Err() == context.DeadlineExceeded {
|
||||
exitCode = -1
|
||||
}
|
||||
sendEvent(streamEvent{T: "exit", C: exitCode})
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
//go:build windows
|
||||
|
||||
package handler
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// runCommandStreamImpl 在 Windows 下用 stdout/stderr 管道执行
|
||||
func runCommandStreamImpl(cmd *exec.Cmd, sendEvent func(streamEvent), ctx context.Context) {
|
||||
stdoutPipe, err := cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
sendEvent(streamEvent{T: "exit", C: -1})
|
||||
return
|
||||
}
|
||||
stderrPipe, err := cmd.StderrPipe()
|
||||
if err != nil {
|
||||
sendEvent(streamEvent{T: "exit", C: -1})
|
||||
return
|
||||
}
|
||||
if err := cmd.Start(); err != nil {
|
||||
sendEvent(streamEvent{T: "exit", C: -1})
|
||||
return
|
||||
}
|
||||
|
||||
normalize := func(s string) string {
|
||||
s = strings.ReplaceAll(s, "\r\n", "\n")
|
||||
return strings.ReplaceAll(s, "\r", "\n")
|
||||
}
|
||||
|
||||
var wg sync.WaitGroup
|
||||
wg.Add(2)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
sc := bufio.NewScanner(stdoutPipe)
|
||||
for sc.Scan() {
|
||||
sendEvent(streamEvent{T: "out", D: normalize(sc.Text())})
|
||||
}
|
||||
}()
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
sc := bufio.NewScanner(stderrPipe)
|
||||
for sc.Scan() {
|
||||
sendEvent(streamEvent{T: "err", D: normalize(sc.Text())})
|
||||
}
|
||||
}()
|
||||
|
||||
wg.Wait()
|
||||
exitCode := 0
|
||||
if err := cmd.Wait(); err != nil {
|
||||
if exitErr, ok := err.(*exec.ExitError); ok {
|
||||
exitCode = exitErr.ExitCode()
|
||||
} else {
|
||||
exitCode = -1
|
||||
}
|
||||
}
|
||||
if ctx.Err() == context.DeadlineExceeded {
|
||||
exitCode = -1
|
||||
}
|
||||
sendEvent(streamEvent{T: "exit", C: exitCode})
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
//go:build !windows
|
||||
|
||||
package handler
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"time"
|
||||
|
||||
"github.com/creack/pty"
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/gorilla/websocket"
|
||||
)
|
||||
|
||||
// wsUpgrader 仅用于系统设置中的终端 WebSocket,会复用已有的登录保护(JWT 中间件在上层路由组)
|
||||
var wsUpgrader = websocket.Upgrader{
|
||||
CheckOrigin: func(r *http.Request) bool {
|
||||
// 由于已在 Gin 路由层做了认证,这里放宽 Origin,方便在同一域名下通过 HTTPS/WSS 访问
|
||||
return true
|
||||
},
|
||||
}
|
||||
|
||||
// RunCommandWS 提供真正交互式 Shell:基于 WebSocket + PTY 的长会话
|
||||
// 前端建立 WebSocket 连接后,所有键盘输入都会透传到 Shell,Shell 的输出也会实时写回前端。
|
||||
func (h *TerminalHandler) RunCommandWS(c *gin.Context) {
|
||||
conn, err := wsUpgrader.Upgrade(c.Writer, c.Request, nil)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
// 启动交互式 Shell,这里优先使用 bash,找不到则退回 sh
|
||||
shell := "bash"
|
||||
if _, err := exec.LookPath(shell); err != nil {
|
||||
shell = "sh"
|
||||
}
|
||||
cmd := exec.Command(shell)
|
||||
cmd.Env = append(os.Environ(),
|
||||
"COLUMNS=256",
|
||||
"LINES=40",
|
||||
"TERM=xterm-256color",
|
||||
)
|
||||
|
||||
ptmx, err := pty.StartWithSize(cmd, &pty.Winsize{Cols: ptyCols, Rows: ptyRows})
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
defer ptmx.Close()
|
||||
|
||||
// Shell -> WebSocket:将 PTY 输出实时发给前端
|
||||
doneChan := make(chan struct{})
|
||||
go func() {
|
||||
buf := make([]byte, 4096)
|
||||
for {
|
||||
n, err := ptmx.Read(buf)
|
||||
if n > 0 {
|
||||
_ = conn.WriteMessage(websocket.BinaryMessage, buf[:n])
|
||||
}
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
close(doneChan)
|
||||
}()
|
||||
|
||||
// WebSocket -> Shell:将前端输入写入 PTY(包括 sudo 密码、Ctrl+C 等)
|
||||
conn.SetReadLimit(64 * 1024)
|
||||
_ = conn.SetReadDeadline(time.Now().Add(terminalTimeout))
|
||||
conn.SetPongHandler(func(string) error {
|
||||
_ = conn.SetReadDeadline(time.Now().Add(terminalTimeout))
|
||||
return nil
|
||||
})
|
||||
|
||||
for {
|
||||
msgType, data, err := conn.ReadMessage()
|
||||
if err != nil {
|
||||
_ = cmd.Process.Kill()
|
||||
break
|
||||
}
|
||||
if msgType != websocket.TextMessage && msgType != websocket.BinaryMessage {
|
||||
continue
|
||||
}
|
||||
if len(data) == 0 {
|
||||
continue
|
||||
}
|
||||
if _, err := ptmx.Write(data); err != nil {
|
||||
_ = cmd.Process.Kill()
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
<-doneChan
|
||||
}
|
||||
|
||||
+149
-31
@@ -6,39 +6,75 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"cyberstrike-ai/internal/config"
|
||||
"cyberstrike-ai/internal/openai"
|
||||
|
||||
"go.uber.org/zap"
|
||||
"golang.org/x/time/rate"
|
||||
)
|
||||
|
||||
// Embedder 文本嵌入器
|
||||
type Embedder struct {
|
||||
openAIClient *openai.Client
|
||||
config *config.KnowledgeConfig
|
||||
openAIConfig *config.OpenAIConfig // 用于获取API Key
|
||||
logger *zap.Logger
|
||||
openAIClient *openai.Client
|
||||
config *config.KnowledgeConfig
|
||||
openAIConfig *config.OpenAIConfig // 用于获取 API Key
|
||||
logger *zap.Logger
|
||||
rateLimiter *rate.Limiter // 速率限制器
|
||||
rateLimitDelay time.Duration // 请求间隔时间
|
||||
maxRetries int // 最大重试次数
|
||||
retryDelay time.Duration // 重试间隔
|
||||
mu sync.Mutex // 保护 rateLimiter
|
||||
}
|
||||
|
||||
// NewEmbedder 创建新的嵌入器
|
||||
func NewEmbedder(cfg *config.KnowledgeConfig, openAIConfig *config.OpenAIConfig, openAIClient *openai.Client, logger *zap.Logger) *Embedder {
|
||||
// 初始化速率限制器
|
||||
var rateLimiter *rate.Limiter
|
||||
var rateLimitDelay time.Duration
|
||||
|
||||
// 如果配置了 MaxRPM,根据 RPM 计算速率限制
|
||||
if cfg.Indexing.MaxRPM > 0 {
|
||||
rpm := cfg.Indexing.MaxRPM
|
||||
rateLimiter = rate.NewLimiter(rate.Every(time.Minute/time.Duration(rpm)), rpm)
|
||||
logger.Info("知识库索引速率限制已启用", zap.Int("maxRPM", rpm))
|
||||
} else if cfg.Indexing.RateLimitDelayMs > 0 {
|
||||
// 如果没有配置 MaxRPM 但配置了固定延迟,使用固定延迟模式
|
||||
rateLimitDelay = time.Duration(cfg.Indexing.RateLimitDelayMs) * time.Millisecond
|
||||
logger.Info("知识库索引固定延迟已启用", zap.Duration("delay", rateLimitDelay))
|
||||
}
|
||||
|
||||
// 重试配置
|
||||
maxRetries := 3
|
||||
retryDelay := 1000 * time.Millisecond
|
||||
if cfg.Indexing.MaxRetries > 0 {
|
||||
maxRetries = cfg.Indexing.MaxRetries
|
||||
}
|
||||
if cfg.Indexing.RetryDelayMs > 0 {
|
||||
retryDelay = time.Duration(cfg.Indexing.RetryDelayMs) * time.Millisecond
|
||||
}
|
||||
|
||||
return &Embedder{
|
||||
openAIClient: openAIClient,
|
||||
config: cfg,
|
||||
openAIConfig: openAIConfig,
|
||||
logger: logger,
|
||||
openAIClient: openAIClient,
|
||||
config: cfg,
|
||||
openAIConfig: openAIConfig,
|
||||
logger: logger,
|
||||
rateLimiter: rateLimiter,
|
||||
rateLimitDelay: rateLimitDelay,
|
||||
maxRetries: maxRetries,
|
||||
retryDelay: retryDelay,
|
||||
}
|
||||
}
|
||||
|
||||
// EmbeddingRequest OpenAI嵌入请求
|
||||
// EmbeddingRequest OpenAI 嵌入请求
|
||||
type EmbeddingRequest struct {
|
||||
Model string `json:"model"`
|
||||
Input []string `json:"input"`
|
||||
}
|
||||
|
||||
// EmbeddingResponse OpenAI嵌入响应
|
||||
// EmbeddingResponse OpenAI 嵌入响应
|
||||
type EmbeddingResponse struct {
|
||||
Data []EmbeddingData `json:"data"`
|
||||
Error *EmbeddingError `json:"error,omitempty"`
|
||||
@@ -56,12 +92,69 @@ type EmbeddingError struct {
|
||||
Type string `json:"type"`
|
||||
}
|
||||
|
||||
// EmbedText 对文本进行嵌入
|
||||
func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error) {
|
||||
if e.openAIClient == nil {
|
||||
return nil, fmt.Errorf("OpenAI客户端未初始化")
|
||||
// waitRateLimiter 等待速率限制器
|
||||
func (e *Embedder) waitRateLimiter() {
|
||||
e.mu.Lock()
|
||||
defer e.mu.Unlock()
|
||||
|
||||
if e.rateLimiter != nil {
|
||||
// 等待令牌
|
||||
ctx := context.Background()
|
||||
if err := e.rateLimiter.Wait(ctx); err != nil {
|
||||
e.logger.Warn("速率限制器等待失败", zap.Error(err))
|
||||
}
|
||||
}
|
||||
|
||||
if e.rateLimitDelay > 0 {
|
||||
time.Sleep(e.rateLimitDelay)
|
||||
}
|
||||
}
|
||||
|
||||
// EmbedText 对文本进行嵌入(带重试和速率限制)
|
||||
func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error) {
|
||||
if e.openAIClient == nil {
|
||||
return nil, fmt.Errorf("OpenAI 客户端未初始化")
|
||||
}
|
||||
|
||||
var lastErr error
|
||||
for attempt := 0; attempt < e.maxRetries; attempt++ {
|
||||
// 速率限制
|
||||
if attempt > 0 {
|
||||
// 重试时等待更长时间
|
||||
waitTime := e.retryDelay * time.Duration(attempt)
|
||||
e.logger.Debug("重试前等待", zap.Int("attempt", attempt+1), zap.Duration("waitTime", waitTime))
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
case <-time.After(waitTime):
|
||||
}
|
||||
} else {
|
||||
e.waitRateLimiter()
|
||||
}
|
||||
|
||||
result, err := e.doEmbedText(ctx, text)
|
||||
if err == nil {
|
||||
return result, nil
|
||||
}
|
||||
|
||||
lastErr = err
|
||||
|
||||
// 检查是否是可重试的错误(429 速率限制、5xx 服务器错误、网络错误)
|
||||
if !e.isRetryableError(err) {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
e.logger.Debug("嵌入请求失败,准备重试",
|
||||
zap.Int("attempt", attempt+1),
|
||||
zap.Int("maxRetries", e.maxRetries),
|
||||
zap.Error(err))
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("达到最大重试次数 (%d): %v", e.maxRetries, lastErr)
|
||||
}
|
||||
|
||||
// doEmbedText 执行实际的嵌入请求(内部方法)
|
||||
func (e *Embedder) doEmbedText(ctx context.Context, text string) ([]float32, error) {
|
||||
// 使用配置的嵌入模型
|
||||
model := e.config.Embedding.Model
|
||||
if model == "" {
|
||||
@@ -73,7 +166,7 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
|
||||
Input: []string{text},
|
||||
}
|
||||
|
||||
// 清理baseURL:去除前后空格和尾部斜杠
|
||||
// 清理 baseURL:去除前后空格和尾部斜杠
|
||||
baseURL := strings.TrimSpace(e.config.Embedding.BaseURL)
|
||||
baseURL = strings.TrimSuffix(baseURL, "/")
|
||||
if baseURL == "" {
|
||||
@@ -83,24 +176,24 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
|
||||
// 构建请求
|
||||
body, err := json.Marshal(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("序列化请求失败: %w", err)
|
||||
return nil, fmt.Errorf("序列化请求失败:%w", err)
|
||||
}
|
||||
|
||||
requestURL := baseURL + "/embeddings"
|
||||
httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost, requestURL, strings.NewReader(string(body)))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("创建请求失败: %w", err)
|
||||
return nil, fmt.Errorf("创建请求失败:%w", err)
|
||||
}
|
||||
|
||||
httpReq.Header.Set("Content-Type", "application/json")
|
||||
|
||||
// 使用配置的API Key,如果没有则使用OpenAI配置的
|
||||
|
||||
// 使用配置的 API Key,如果没有则使用 OpenAI 配置的
|
||||
apiKey := strings.TrimSpace(e.config.Embedding.APIKey)
|
||||
if apiKey == "" && e.openAIConfig != nil {
|
||||
apiKey = e.openAIConfig.APIKey
|
||||
}
|
||||
if apiKey == "" {
|
||||
return nil, fmt.Errorf("API Key未配置")
|
||||
return nil, fmt.Errorf("API Key 未配置")
|
||||
}
|
||||
httpReq.Header.Set("Authorization", "Bearer "+apiKey)
|
||||
|
||||
@@ -110,7 +203,7 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
|
||||
}
|
||||
resp, err := httpClient.Do(httpReq)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("发送请求失败: %w", err)
|
||||
return nil, fmt.Errorf("发送请求失败:%w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
@@ -132,7 +225,7 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
|
||||
if len(requestBodyPreview) > 200 {
|
||||
requestBodyPreview = requestBodyPreview[:200] + "..."
|
||||
}
|
||||
e.logger.Debug("嵌入API请求",
|
||||
e.logger.Debug("嵌入 API 请求",
|
||||
zap.String("url", httpReq.URL.String()),
|
||||
zap.String("model", model),
|
||||
zap.String("requestBody", requestBodyPreview),
|
||||
@@ -148,12 +241,12 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
|
||||
if len(bodyPreview) > 500 {
|
||||
bodyPreview = bodyPreview[:500] + "..."
|
||||
}
|
||||
return nil, fmt.Errorf("解析响应失败 (URL: %s, 状态码: %d, 响应长度: %d字节): %w\n请求体: %s\n响应内容预览: %s",
|
||||
return nil, fmt.Errorf("解析响应失败 (URL: %s, 状态码:%d, 响应长度:%d字节): %w\n请求体:%s\n响应内容预览:%s",
|
||||
requestURL, resp.StatusCode, len(bodyBytes), err, requestBodyPreview, bodyPreview)
|
||||
}
|
||||
|
||||
if embeddingResp.Error != nil {
|
||||
return nil, fmt.Errorf("OpenAI API错误 (状态码: %d): 类型=%s, 消息=%s",
|
||||
return nil, fmt.Errorf("OpenAI API 错误 (状态码:%d): 类型=%s, 消息=%s",
|
||||
resp.StatusCode, embeddingResp.Error.Type, embeddingResp.Error.Message)
|
||||
}
|
||||
|
||||
@@ -162,7 +255,7 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
|
||||
if len(bodyPreview) > 500 {
|
||||
bodyPreview = bodyPreview[:500] + "..."
|
||||
}
|
||||
return nil, fmt.Errorf("HTTP请求失败 (URL: %s, 状态码: %d): 响应内容=%s", requestURL, resp.StatusCode, bodyPreview)
|
||||
return nil, fmt.Errorf("HTTP 请求失败 (URL: %s, 状态码:%d): 响应内容=%s", requestURL, resp.StatusCode, bodyPreview)
|
||||
}
|
||||
|
||||
if len(embeddingResp.Data) == 0 {
|
||||
@@ -170,11 +263,11 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
|
||||
if len(bodyPreview) > 500 {
|
||||
bodyPreview = bodyPreview[:500] + "..."
|
||||
}
|
||||
return nil, fmt.Errorf("未收到嵌入数据 (状态码: %d, 响应长度: %d字节)\n响应内容: %s",
|
||||
return nil, fmt.Errorf("未收到嵌入数据 (状态码:%d, 响应长度:%d字节)\n响应内容:%s",
|
||||
resp.StatusCode, len(bodyBytes), bodyPreview)
|
||||
}
|
||||
|
||||
// 转换为float32
|
||||
// 转换为 float32
|
||||
embedding := make([]float32, len(embeddingResp.Data[0].Embedding))
|
||||
for i, v := range embeddingResp.Data[0].Embedding {
|
||||
embedding[i] = float32(v)
|
||||
@@ -183,23 +276,48 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
|
||||
return embedding, nil
|
||||
}
|
||||
|
||||
// isRetryableError 判断是否是可重试的错误
|
||||
func (e *Embedder) isRetryableError(err error) bool {
|
||||
if err == nil {
|
||||
return false
|
||||
}
|
||||
|
||||
errStr := err.Error()
|
||||
|
||||
// 429 速率限制错误
|
||||
if strings.Contains(errStr, "429") || strings.Contains(errStr, "rate limit") {
|
||||
return true
|
||||
}
|
||||
|
||||
// 5xx 服务器错误
|
||||
if strings.Contains(errStr, "500") || strings.Contains(errStr, "502") ||
|
||||
strings.Contains(errStr, "503") || strings.Contains(errStr, "504") {
|
||||
return true
|
||||
}
|
||||
|
||||
// 网络错误
|
||||
if strings.Contains(errStr, "timeout") || strings.Contains(errStr, "connection") ||
|
||||
strings.Contains(errStr, "network") || strings.Contains(errStr, "EOF") {
|
||||
return true
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
// EmbedTexts 批量嵌入文本
|
||||
func (e *Embedder) EmbedTexts(ctx context.Context, texts []string) ([][]float32, error) {
|
||||
if len(texts) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// OpenAI API支持批量,但为了简单起见,我们逐个处理
|
||||
// 实际可以使用批量API以提高效率
|
||||
embeddings := make([][]float32, len(texts))
|
||||
for i, text := range texts {
|
||||
embedding, err := e.EmbedText(ctx, text)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("嵌入文本[%d]失败: %w", i, err)
|
||||
return nil, fmt.Errorf("嵌入文本 [%d] 失败:%w", i, err)
|
||||
}
|
||||
embeddings[i] = embedding
|
||||
}
|
||||
|
||||
return embeddings, nil
|
||||
}
|
||||
|
||||
|
||||
+369
-97
@@ -10,56 +10,133 @@ import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"cyberstrike-ai/internal/config"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"go.uber.org/zap"
|
||||
)
|
||||
|
||||
// Indexer 索引器,负责将知识项分块并向量化
|
||||
type Indexer struct {
|
||||
db *sql.DB
|
||||
embedder *Embedder
|
||||
logger *zap.Logger
|
||||
chunkSize int // 每个块的最大token数(估算)
|
||||
overlap int // 块之间的重叠token数
|
||||
|
||||
db *sql.DB
|
||||
embedder *Embedder
|
||||
logger *zap.Logger
|
||||
chunkSize int // 每个块的最大 token 数(估算)
|
||||
overlap int // 块之间的重叠 token 数
|
||||
maxChunks int // 单个知识项的最大块数量(0 表示不限制)
|
||||
|
||||
// 错误跟踪
|
||||
mu sync.RWMutex
|
||||
lastError string // 最近一次错误信息
|
||||
mu sync.RWMutex
|
||||
lastError string // 最近一次错误信息
|
||||
lastErrorTime time.Time // 最近一次错误时间
|
||||
errorCount int // 连续错误计数
|
||||
errorCount int // 连续错误计数
|
||||
|
||||
// 重建索引状态跟踪
|
||||
rebuildMu sync.RWMutex
|
||||
isRebuilding bool // 是否正在重建索引
|
||||
rebuildTotalItems int // 重建总项数
|
||||
rebuildCurrent int // 当前已处理项数
|
||||
rebuildFailed int // 重建失败项数
|
||||
rebuildStartTime time.Time // 重建开始时间
|
||||
rebuildLastItemID string // 最近处理的项 ID
|
||||
rebuildLastChunks int // 最近处理的项的分块数
|
||||
}
|
||||
|
||||
// NewIndexer 创建新的索引器
|
||||
func NewIndexer(db *sql.DB, embedder *Embedder, logger *zap.Logger) *Indexer {
|
||||
func NewIndexer(db *sql.DB, embedder *Embedder, logger *zap.Logger, indexingCfg *config.IndexingConfig) *Indexer {
|
||||
chunkSize := 512
|
||||
overlap := 50
|
||||
maxChunks := 0
|
||||
if indexingCfg != nil {
|
||||
if indexingCfg.ChunkSize > 0 {
|
||||
chunkSize = indexingCfg.ChunkSize
|
||||
}
|
||||
if indexingCfg.ChunkOverlap >= 0 {
|
||||
overlap = indexingCfg.ChunkOverlap
|
||||
}
|
||||
if indexingCfg.MaxChunksPerItem > 0 {
|
||||
maxChunks = indexingCfg.MaxChunksPerItem
|
||||
}
|
||||
}
|
||||
return &Indexer{
|
||||
db: db,
|
||||
embedder: embedder,
|
||||
logger: logger,
|
||||
chunkSize: 512, // 默认512 tokens
|
||||
overlap: 50, // 默认50 tokens重叠
|
||||
chunkSize: chunkSize,
|
||||
overlap: overlap,
|
||||
maxChunks: maxChunks,
|
||||
}
|
||||
}
|
||||
|
||||
// ChunkText 将文本分块(支持重叠)
|
||||
// ChunkText 将文本分块(支持重叠,保留标题上下文)
|
||||
func (idx *Indexer) ChunkText(text string) []string {
|
||||
// 按Markdown标题分割
|
||||
chunks := idx.splitByMarkdownHeaders(text)
|
||||
// 按 Markdown 标题分割,获取带标题的块
|
||||
sections := idx.splitByMarkdownHeadersWithContent(text)
|
||||
|
||||
// 如果块太大,进一步分割
|
||||
// 处理每个块
|
||||
result := make([]string, 0)
|
||||
for _, chunk := range chunks {
|
||||
if idx.estimateTokens(chunk) <= idx.chunkSize {
|
||||
result = append(result, chunk)
|
||||
for _, section := range sections {
|
||||
// 构建父级标题路径(不包含最后一级标题,因为内容中已经包含)
|
||||
// 例如:["# A", "## B", "### C"] -> "[# A > ## B]"
|
||||
var parentHeaderPath string
|
||||
if len(section.HeaderPath) > 1 {
|
||||
parentHeaderPath = strings.Join(section.HeaderPath[:len(section.HeaderPath)-1], " > ")
|
||||
}
|
||||
|
||||
// 提取内容的第一行作为标题(如 "# Prompt Injection")
|
||||
firstLine, remainingContent := extractFirstLine(section.Content)
|
||||
|
||||
// 如果剩余内容为空或只有空白,说明这个块只有标题没有正文,跳过
|
||||
if strings.TrimSpace(remainingContent) == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
// 如果块太大,进一步分割
|
||||
if idx.estimateTokens(section.Content) <= idx.chunkSize {
|
||||
// 块大小合适,添加父级标题前缀
|
||||
if parentHeaderPath != "" {
|
||||
result = append(result, fmt.Sprintf("[%s] %s", parentHeaderPath, section.Content))
|
||||
} else {
|
||||
result = append(result, section.Content)
|
||||
}
|
||||
} else {
|
||||
// 按段落分割
|
||||
subChunks := idx.splitByParagraphs(chunk)
|
||||
for _, subChunk := range subChunks {
|
||||
if idx.estimateTokens(subChunk) <= idx.chunkSize {
|
||||
result = append(result, subChunk)
|
||||
} else {
|
||||
// 按句子分割(支持重叠)
|
||||
chunksWithOverlap := idx.splitBySentencesWithOverlap(subChunk)
|
||||
result = append(result, chunksWithOverlap...)
|
||||
// 块太大,按子标题或段落分割,保持标题上下文
|
||||
// 首先尝试按子标题分割(保留子标题结构)
|
||||
subSections := idx.splitBySubHeaders(section.Content, firstLine, parentHeaderPath)
|
||||
if len(subSections) > 1 {
|
||||
// 成功按子标题分割,递归处理每个子块
|
||||
for _, sub := range subSections {
|
||||
if idx.estimateTokens(sub) <= idx.chunkSize {
|
||||
result = append(result, sub)
|
||||
} else {
|
||||
// 子块仍然太大,按段落分割(保留标题前缀)
|
||||
paragraphs := idx.splitByParagraphsWithHeader(sub, parentHeaderPath)
|
||||
for _, para := range paragraphs {
|
||||
if idx.estimateTokens(para) <= idx.chunkSize {
|
||||
result = append(result, para)
|
||||
} else {
|
||||
// 段落仍太大,按句子分割
|
||||
sentenceChunks := idx.splitBySentencesWithOverlap(para)
|
||||
for _, chunk := range sentenceChunks {
|
||||
result = append(result, chunk)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// 没有子标题,按段落分割(保留标题前缀)
|
||||
paragraphs := idx.splitByParagraphsWithHeader(section.Content, parentHeaderPath)
|
||||
for _, para := range paragraphs {
|
||||
if idx.estimateTokens(para) <= idx.chunkSize {
|
||||
result = append(result, para)
|
||||
} else {
|
||||
// 段落仍太大,按句子分割
|
||||
sentenceChunks := idx.splitBySentencesWithOverlap(para)
|
||||
for _, chunk := range sentenceChunks {
|
||||
result = append(result, chunk)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -68,43 +145,183 @@ func (idx *Indexer) ChunkText(text string) []string {
|
||||
return result
|
||||
}
|
||||
|
||||
// splitByMarkdownHeaders 按Markdown标题分割
|
||||
func (idx *Indexer) splitByMarkdownHeaders(text string) []string {
|
||||
// 匹配Markdown标题 (# ## ### 等)
|
||||
// extractFirstLine 提取第一行内容和剩余内容
|
||||
func extractFirstLine(content string) (firstLine, remaining string) {
|
||||
lines := strings.SplitN(content, "\n", 2)
|
||||
if len(lines) == 0 {
|
||||
return "", ""
|
||||
}
|
||||
if len(lines) == 1 {
|
||||
return lines[0], ""
|
||||
}
|
||||
return lines[0], lines[1]
|
||||
}
|
||||
|
||||
// splitBySubHeaders 尝试按子标题分割内容(用于处理大块内容)
|
||||
// headerPrefix 是父级标题路径,用于添加到每个子块
|
||||
func (idx *Indexer) splitBySubHeaders(content, headerPrefix, parentPath string) []string {
|
||||
// 匹配 Markdown 子标题(## 及以上)
|
||||
subHeaderRegex := regexp.MustCompile(`(?m)^#{2,6}\s+.+$`)
|
||||
matches := subHeaderRegex.FindAllStringIndex(content, -1)
|
||||
|
||||
if len(matches) == 0 {
|
||||
// 没有子标题,返回原始内容
|
||||
return []string{content}
|
||||
}
|
||||
|
||||
result := make([]string, 0, len(matches))
|
||||
for i, match := range matches {
|
||||
start := match[0]
|
||||
nextStart := len(content)
|
||||
if i+1 < len(matches) {
|
||||
nextStart = matches[i+1][0]
|
||||
}
|
||||
|
||||
subContent := strings.TrimSpace(content[start:nextStart])
|
||||
|
||||
// 添加父级路径前缀
|
||||
if parentPath != "" {
|
||||
result = append(result, fmt.Sprintf("[%s] %s", parentPath, subContent))
|
||||
} else {
|
||||
result = append(result, subContent)
|
||||
}
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// splitByParagraphsWithHeader 按段落分割,每个段落添加标题前缀(用于保持上下文)
|
||||
func (idx *Indexer) splitByParagraphsWithHeader(content, parentPath string) []string {
|
||||
// 提取第一行作为标题
|
||||
firstLine, _ := extractFirstLine(content)
|
||||
|
||||
paragraphs := strings.Split(content, "\n\n")
|
||||
result := make([]string, 0)
|
||||
|
||||
for i, p := range paragraphs {
|
||||
trimmed := strings.TrimSpace(p)
|
||||
if trimmed == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
// 过滤掉只有标题的段落(没有实际内容)
|
||||
if strings.TrimSpace(trimmed) == strings.TrimSpace(firstLine) {
|
||||
continue
|
||||
}
|
||||
|
||||
// 第一个段落已经包含标题,不需要重复添加
|
||||
if i == 0 && strings.Contains(trimmed, firstLine) {
|
||||
if parentPath != "" {
|
||||
result = append(result, fmt.Sprintf("[%s] %s", parentPath, trimmed))
|
||||
} else {
|
||||
result = append(result, trimmed)
|
||||
}
|
||||
} else {
|
||||
// 其他段落添加标题前缀以保持上下文
|
||||
if parentPath != "" {
|
||||
result = append(result, fmt.Sprintf("[%s] %s\n%s", parentPath, firstLine, trimmed))
|
||||
} else {
|
||||
result = append(result, fmt.Sprintf("%s\n%s", firstLine, trimmed))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// Section 表示一个带标题路径的文本块
|
||||
type Section struct {
|
||||
HeaderPath []string // 标题路径(如 ["# SQL 注入", "## 检测方法"])
|
||||
Content string // 块内容
|
||||
}
|
||||
|
||||
// splitByMarkdownHeadersWithContent 按 Markdown 标题分割,返回带标题路径的块
|
||||
// 每个块的内容包含自己的标题,用于向量化检索
|
||||
//
|
||||
// 例如,对于以下 Markdown:
|
||||
// # Prompt Injection
|
||||
// 引言内容
|
||||
// ## Summary
|
||||
// 目录内容
|
||||
//
|
||||
// 返回:
|
||||
// [{HeaderPath: ["# Prompt Injection"], Content: "# Prompt Injection\n引言内容"},
|
||||
// {HeaderPath: ["# Prompt Injection", "## Summary"], Content: "## Summary\n目录内容"}]
|
||||
func (idx *Indexer) splitByMarkdownHeadersWithContent(text string) []Section {
|
||||
// 匹配 Markdown 标题 (# ## ### 等)
|
||||
headerRegex := regexp.MustCompile(`(?m)^#{1,6}\s+.+$`)
|
||||
|
||||
// 找到所有标题位置
|
||||
matches := headerRegex.FindAllStringIndex(text, -1)
|
||||
if len(matches) == 0 {
|
||||
return []string{text}
|
||||
// 没有标题,返回整个文本
|
||||
return []Section{{HeaderPath: []string{}, Content: text}}
|
||||
}
|
||||
|
||||
chunks := make([]string, 0)
|
||||
lastPos := 0
|
||||
sections := make([]Section, 0, len(matches))
|
||||
currentHeaderPath := []string{}
|
||||
|
||||
for _, match := range matches {
|
||||
for i, match := range matches {
|
||||
start := match[0]
|
||||
if start > lastPos {
|
||||
chunks = append(chunks, strings.TrimSpace(text[lastPos:start]))
|
||||
}
|
||||
lastPos = start
|
||||
}
|
||||
end := match[1]
|
||||
nextStart := len(text)
|
||||
|
||||
// 添加最后一部分
|
||||
if lastPos < len(text) {
|
||||
chunks = append(chunks, strings.TrimSpace(text[lastPos:]))
|
||||
// 找到下一个标题的位置
|
||||
if i+1 < len(matches) {
|
||||
nextStart = matches[i+1][0]
|
||||
}
|
||||
|
||||
// 提取当前标题
|
||||
headerLine := strings.TrimSpace(text[start:end])
|
||||
|
||||
// 计算标题层级(# 的数量)
|
||||
level := 0
|
||||
for _, ch := range headerLine {
|
||||
if ch == '#' {
|
||||
level++
|
||||
} else {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
// 更新标题路径:移除比当前层级深或等于的子标题,然后添加当前标题
|
||||
newPath := make([]string, 0, len(currentHeaderPath)+1)
|
||||
for _, h := range currentHeaderPath {
|
||||
hLevel := 0
|
||||
for _, ch := range h {
|
||||
if ch == '#' {
|
||||
hLevel++
|
||||
} else {
|
||||
break
|
||||
}
|
||||
}
|
||||
if hLevel < level {
|
||||
newPath = append(newPath, h)
|
||||
}
|
||||
}
|
||||
newPath = append(newPath, headerLine)
|
||||
currentHeaderPath = newPath
|
||||
|
||||
// 提取当前标题到下一个标题之间的内容(包含当前标题)
|
||||
content := strings.TrimSpace(text[start:nextStart])
|
||||
|
||||
// 创建块,使用当前标题路径(包含当前标题)
|
||||
sections = append(sections, Section{
|
||||
HeaderPath: append([]string(nil), currentHeaderPath...),
|
||||
Content: content,
|
||||
})
|
||||
}
|
||||
|
||||
// 过滤空块
|
||||
result := make([]string, 0)
|
||||
for _, chunk := range chunks {
|
||||
if strings.TrimSpace(chunk) != "" {
|
||||
result = append(result, chunk)
|
||||
result := make([]Section, 0, len(sections))
|
||||
for _, section := range sections {
|
||||
if strings.TrimSpace(section.Content) != "" {
|
||||
result = append(result, section)
|
||||
}
|
||||
}
|
||||
|
||||
if len(result) == 0 {
|
||||
return []string{text}
|
||||
return []Section{{HeaderPath: []string{}, Content: text}}
|
||||
}
|
||||
|
||||
return result
|
||||
@@ -124,8 +341,12 @@ func (idx *Indexer) splitByParagraphs(text string) []string {
|
||||
|
||||
// splitBySentences 按句子分割(用于内部,不包含重叠逻辑)
|
||||
func (idx *Indexer) splitBySentences(text string) []string {
|
||||
// 简单的句子分割(按句号、问号、感叹号)
|
||||
sentenceRegex := regexp.MustCompile(`[.!?]+\s+`)
|
||||
// 简单的句子分割(按句号、问号、感叹号,支持中英文)
|
||||
// . ! ? = 英文标点
|
||||
// \u3002 = 。(中文句号)
|
||||
// \uFF01 = !(中文叹号)
|
||||
// \uFF1F = ?(中文问号)
|
||||
sentenceRegex := regexp.MustCompile(`[.!?\x{3002}\x{FF01}\x{FF1F}]+`)
|
||||
sentences := sentenceRegex.Split(text, -1)
|
||||
result := make([]string, 0)
|
||||
for _, s := range sentences {
|
||||
@@ -221,13 +442,13 @@ func (idx *Indexer) splitBySentencesSimple(text string) []string {
|
||||
return result
|
||||
}
|
||||
|
||||
// extractLastTokens 从文本末尾提取指定token数量的内容
|
||||
// extractLastTokens 从文本末尾提取指定 token 数量的内容
|
||||
func (idx *Indexer) extractLastTokens(text string, tokenCount int) string {
|
||||
if tokenCount <= 0 || text == "" {
|
||||
return ""
|
||||
}
|
||||
|
||||
// 估算字符数(1 token ≈ 4字符)
|
||||
// 估算字符数(1 token ≈ 4 字符)
|
||||
charCount := tokenCount * 4
|
||||
runes := []rune(text)
|
||||
|
||||
@@ -236,12 +457,11 @@ func (idx *Indexer) extractLastTokens(text string, tokenCount int) string {
|
||||
}
|
||||
|
||||
// 从末尾提取指定数量的字符
|
||||
// 尝试在句子边界处截断,避免截断句子中间
|
||||
startPos := len(runes) - charCount
|
||||
extracted := string(runes[startPos:])
|
||||
|
||||
// 尝试找到第一个句子边界(句号、问号、感叹号后的空格)
|
||||
sentenceBoundary := regexp.MustCompile(`[.!?]+\s+`)
|
||||
// 尝试找到第一个句子边界(支持中英文标点)
|
||||
sentenceBoundary := regexp.MustCompile(`[.!?\x{3002}\x{FF01}\x{FF1F}]+`)
|
||||
matches := sentenceBoundary.FindStringIndex(extracted)
|
||||
if len(matches) > 0 && matches[0] > 0 {
|
||||
// 在句子边界处截断,保留完整句子
|
||||
@@ -251,41 +471,51 @@ func (idx *Indexer) extractLastTokens(text string, tokenCount int) string {
|
||||
return strings.TrimSpace(extracted)
|
||||
}
|
||||
|
||||
// estimateTokens 估算token数(简单估算:1 token ≈ 4字符)
|
||||
// estimateTokens 估算 token 数(简单估算:1 token ≈ 4 字符)
|
||||
func (idx *Indexer) estimateTokens(text string) int {
|
||||
return len([]rune(text)) / 4
|
||||
}
|
||||
|
||||
// IndexItem 索引知识项(分块并向量化)
|
||||
func (idx *Indexer) IndexItem(ctx context.Context, itemID string) error {
|
||||
// 获取知识项(包含category和title,用于向量化)
|
||||
// 获取知识项(包含 category 和 title,用于向量化)
|
||||
var content, category, title string
|
||||
err := idx.db.QueryRow("SELECT content, category, title FROM knowledge_base_items WHERE id = ?", itemID).Scan(&content, &category, &title)
|
||||
if err != nil {
|
||||
return fmt.Errorf("获取知识项失败: %w", err)
|
||||
return fmt.Errorf("获取知识项失败:%w", err)
|
||||
}
|
||||
|
||||
// 删除旧的向量(在 RebuildIndex 中已经统一清空,这里保留是为了单独调用 IndexItem 时的兼容性)
|
||||
_, err = idx.db.Exec("DELETE FROM knowledge_embeddings WHERE item_id = ?", itemID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("删除旧向量失败: %w", err)
|
||||
return fmt.Errorf("删除旧向量失败:%w", err)
|
||||
}
|
||||
|
||||
// 分块
|
||||
chunks := idx.ChunkText(content)
|
||||
|
||||
// 应用最大块数限制
|
||||
if idx.maxChunks > 0 && len(chunks) > idx.maxChunks {
|
||||
idx.logger.Info("知识项块数量超过限制,已截断",
|
||||
zap.String("itemId", itemID),
|
||||
zap.Int("originalChunks", len(chunks)),
|
||||
zap.Int("maxChunks", idx.maxChunks))
|
||||
chunks = chunks[:idx.maxChunks]
|
||||
}
|
||||
|
||||
idx.logger.Info("知识项分块完成", zap.String("itemId", itemID), zap.Int("chunks", len(chunks)))
|
||||
|
||||
// 跟踪该知识项的错误
|
||||
itemErrorCount := 0
|
||||
var firstError error
|
||||
firstErrorChunkIndex := -1
|
||||
|
||||
// 向量化每个块(包含category和title信息,以便向量检索时能匹配到风险类型)
|
||||
|
||||
// 向量化每个块(包含 category 和 title 信息,以便向量检索时能匹配到风险类型)
|
||||
for i, chunk := range chunks {
|
||||
// 将category和title信息包含到向量化的文本中
|
||||
// 格式:"[风险类型: {category}] [标题: {title}]\n{chunk内容}"
|
||||
// 这样向量嵌入就会包含风险类型信息,即使SQL过滤失败,向量相似度也能帮助匹配
|
||||
textForEmbedding := fmt.Sprintf("[风险类型: %s] [标题: %s]\n%s", category, title, chunk)
|
||||
// 将 category 和 title 信息包含到向量化的文本中
|
||||
// 格式:"[风险类型:{category}] [标题:{title}]\n{chunk 内容}"
|
||||
// 这样向量嵌入就会包含风险类型信息,即使 SQL 过滤失败,向量相似度也能帮助匹配
|
||||
textForEmbedding := fmt.Sprintf("[风险类型:%s] [标题:%s]\n%s", category, title, chunk)
|
||||
|
||||
embedding, err := idx.embedder.EmbedText(ctx, textForEmbedding)
|
||||
if err != nil {
|
||||
@@ -305,17 +535,17 @@ func (idx *Indexer) IndexItem(ctx context.Context, itemID string) error {
|
||||
zap.String("chunkPreview", chunkPreview),
|
||||
zap.Error(err),
|
||||
)
|
||||
|
||||
|
||||
// 更新全局错误跟踪
|
||||
errorMsg := fmt.Sprintf("向量化失败 (知识项: %s): %v", itemID, err)
|
||||
errorMsg := fmt.Sprintf("向量化失败 (知识项:%s): %v", itemID, err)
|
||||
idx.mu.Lock()
|
||||
idx.lastError = errorMsg
|
||||
idx.lastErrorTime = time.Now()
|
||||
idx.mu.Unlock()
|
||||
}
|
||||
|
||||
// 如果连续失败2个块,立即停止处理该知识项(降低阈值,更快停止)
|
||||
// 这样可以避免继续浪费API调用,同时也能更快地检测到配置问题
|
||||
|
||||
// 如果连续失败 2 个块,立即停止处理该知识项(降低阈值,更快停止)
|
||||
// 这样可以避免继续浪费 API 调用,同时也能更快地检测到配置问题
|
||||
if itemErrorCount >= 2 {
|
||||
idx.logger.Error("知识项连续向量化失败,停止处理",
|
||||
zap.String("itemId", itemID),
|
||||
@@ -344,6 +574,13 @@ func (idx *Indexer) IndexItem(ctx context.Context, itemID string) error {
|
||||
}
|
||||
|
||||
idx.logger.Info("知识项索引完成", zap.String("itemId", itemID), zap.Int("chunks", len(chunks)))
|
||||
|
||||
// 更新重建状态中的最近处理信息
|
||||
idx.rebuildMu.Lock()
|
||||
idx.rebuildLastItemID = itemID
|
||||
idx.rebuildLastChunks = len(chunks)
|
||||
idx.rebuildMu.Unlock()
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -352,23 +589,38 @@ func (idx *Indexer) HasIndex() (bool, error) {
|
||||
var count int
|
||||
err := idx.db.QueryRow("SELECT COUNT(*) FROM knowledge_embeddings").Scan(&count)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("检查索引失败: %w", err)
|
||||
return false, fmt.Errorf("检查索引失败:%w", err)
|
||||
}
|
||||
return count > 0, nil
|
||||
}
|
||||
|
||||
// RebuildIndex 重建所有索引
|
||||
func (idx *Indexer) RebuildIndex(ctx context.Context) error {
|
||||
// 设置重建状态
|
||||
idx.rebuildMu.Lock()
|
||||
idx.isRebuilding = true
|
||||
idx.rebuildTotalItems = 0
|
||||
idx.rebuildCurrent = 0
|
||||
idx.rebuildFailed = 0
|
||||
idx.rebuildStartTime = time.Now()
|
||||
idx.rebuildLastItemID = ""
|
||||
idx.rebuildLastChunks = 0
|
||||
idx.rebuildMu.Unlock()
|
||||
|
||||
// 重置错误跟踪
|
||||
idx.mu.Lock()
|
||||
idx.lastError = ""
|
||||
idx.lastErrorTime = time.Time{}
|
||||
idx.errorCount = 0
|
||||
idx.mu.Unlock()
|
||||
|
||||
|
||||
rows, err := idx.db.Query("SELECT id FROM knowledge_base_items")
|
||||
if err != nil {
|
||||
return fmt.Errorf("查询知识项失败: %w", err)
|
||||
// 重置重建状态
|
||||
idx.rebuildMu.Lock()
|
||||
idx.isRebuilding = false
|
||||
idx.rebuildMu.Unlock()
|
||||
return fmt.Errorf("查询知识项失败:%w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
@@ -376,34 +628,36 @@ func (idx *Indexer) RebuildIndex(ctx context.Context) error {
|
||||
for rows.Next() {
|
||||
var id string
|
||||
if err := rows.Scan(&id); err != nil {
|
||||
return fmt.Errorf("扫描知识项ID失败: %w", err)
|
||||
// 重置重建状态
|
||||
idx.rebuildMu.Lock()
|
||||
idx.isRebuilding = false
|
||||
idx.rebuildMu.Unlock()
|
||||
return fmt.Errorf("扫描知识项 ID 失败:%w", err)
|
||||
}
|
||||
itemIDs = append(itemIDs, id)
|
||||
}
|
||||
|
||||
idx.rebuildMu.Lock()
|
||||
idx.rebuildTotalItems = len(itemIDs)
|
||||
idx.rebuildMu.Unlock()
|
||||
|
||||
idx.logger.Info("开始重建索引", zap.Int("totalItems", len(itemIDs)))
|
||||
|
||||
// 在开始重建前,先清空所有旧的向量,确保进度从0开始
|
||||
// 这样 GetIndexStatus 可以准确反映重建进度
|
||||
_, err = idx.db.Exec("DELETE FROM knowledge_embeddings")
|
||||
if err != nil {
|
||||
idx.logger.Warn("清空旧索引失败", zap.Error(err))
|
||||
// 继续执行,即使清空失败也尝试重建
|
||||
} else {
|
||||
idx.logger.Info("已清空旧索引,开始重建")
|
||||
}
|
||||
// 注意:不再清空所有旧索引,而是按增量方式更新
|
||||
// 每个知识项在 IndexItem 中会先删除自己的旧向量,然后插入新向量
|
||||
// 这样配置更新后只重新索引变化的知识项,保留其他知识项的索引
|
||||
|
||||
failedCount := 0
|
||||
consecutiveFailures := 0
|
||||
maxConsecutiveFailures := 2 // 连续失败2次后立即停止(降低阈值,更快停止)
|
||||
maxConsecutiveFailures := 2 // 连续失败 2 次后立即停止(降低阈值,更快停止)
|
||||
firstFailureItemID := ""
|
||||
var firstFailureError error
|
||||
|
||||
|
||||
for i, itemID := range itemIDs {
|
||||
if err := idx.IndexItem(ctx, itemID); err != nil {
|
||||
failedCount++
|
||||
consecutiveFailures++
|
||||
|
||||
|
||||
// 只在第一个失败时记录详细日志
|
||||
if consecutiveFailures == 1 {
|
||||
firstFailureItemID = itemID
|
||||
@@ -414,15 +668,15 @@ func (idx *Indexer) RebuildIndex(ctx context.Context) error {
|
||||
zap.Error(err),
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
// 如果连续失败过多,可能是配置问题,立即停止索引
|
||||
if consecutiveFailures >= maxConsecutiveFailures {
|
||||
errorMsg := fmt.Sprintf("连续 %d 个知识项索引失败,可能存在配置问题(如嵌入模型配置错误、API密钥无效、余额不足等)。第一个失败项: %s, 错误: %v", consecutiveFailures, firstFailureItemID, firstFailureError)
|
||||
errorMsg := fmt.Sprintf("连续 %d 个知识项索引失败,可能存在配置问题(如嵌入模型配置错误、API 密钥无效、余额不足等)。第一个失败项:%s, 错误:%v", consecutiveFailures, firstFailureItemID, firstFailureError)
|
||||
idx.mu.Lock()
|
||||
idx.lastError = errorMsg
|
||||
idx.lastErrorTime = time.Now()
|
||||
idx.mu.Unlock()
|
||||
|
||||
|
||||
idx.logger.Error("连续索引失败次数过多,立即停止索引",
|
||||
zap.Int("consecutiveFailures", consecutiveFailures),
|
||||
zap.Int("totalItems", len(itemIDs)),
|
||||
@@ -430,17 +684,17 @@ func (idx *Indexer) RebuildIndex(ctx context.Context) error {
|
||||
zap.String("firstFailureItemId", firstFailureItemID),
|
||||
zap.Error(firstFailureError),
|
||||
)
|
||||
return fmt.Errorf("连续索引失败次数过多: %v", firstFailureError)
|
||||
return fmt.Errorf("连续索引失败次数过多:%v", firstFailureError)
|
||||
}
|
||||
|
||||
// 如果失败的知识项过多,记录警告但继续处理(降低阈值到30%)
|
||||
|
||||
// 如果失败的知识项过多,记录警告但继续处理(降低阈值到 30%)
|
||||
if failedCount > len(itemIDs)*3/10 && failedCount == len(itemIDs)*3/10+1 {
|
||||
errorMsg := fmt.Sprintf("索引失败的知识项过多 (%d/%d),可能存在配置问题。第一个失败项: %s, 错误: %v", failedCount, len(itemIDs), firstFailureItemID, firstFailureError)
|
||||
errorMsg := fmt.Sprintf("索引失败的知识项过多 (%d/%d),可能存在配置问题。第一个失败项:%s, 错误:%v", failedCount, len(itemIDs), firstFailureItemID, firstFailureError)
|
||||
idx.mu.Lock()
|
||||
idx.lastError = errorMsg
|
||||
idx.lastErrorTime = time.Now()
|
||||
idx.mu.Unlock()
|
||||
|
||||
|
||||
idx.logger.Error("索引失败的知识项过多,可能存在配置问题",
|
||||
zap.Int("failedCount", failedCount),
|
||||
zap.Int("totalItems", len(itemIDs)),
|
||||
@@ -450,20 +704,31 @@ func (idx *Indexer) RebuildIndex(ctx context.Context) error {
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
|
||||
// 成功时重置连续失败计数和第一个失败信息
|
||||
if consecutiveFailures > 0 {
|
||||
consecutiveFailures = 0
|
||||
firstFailureItemID = ""
|
||||
firstFailureError = nil
|
||||
}
|
||||
|
||||
// 减少进度日志频率(每10个或每10%记录一次)
|
||||
|
||||
// 更新重建进度
|
||||
idx.rebuildMu.Lock()
|
||||
idx.rebuildCurrent = i + 1
|
||||
idx.rebuildFailed = failedCount
|
||||
idx.rebuildMu.Unlock()
|
||||
|
||||
// 减少进度日志频率(每 10 个或每 10% 记录一次)
|
||||
if (i+1)%10 == 0 || (len(itemIDs) > 0 && (i+1)*100/len(itemIDs)%10 == 0 && (i+1)*100/len(itemIDs) > 0) {
|
||||
idx.logger.Info("索引进度", zap.Int("current", i+1), zap.Int("total", len(itemIDs)), zap.Int("failed", failedCount))
|
||||
}
|
||||
}
|
||||
|
||||
// 重置重建状态
|
||||
idx.rebuildMu.Lock()
|
||||
idx.isRebuilding = false
|
||||
idx.rebuildMu.Unlock()
|
||||
|
||||
idx.logger.Info("索引重建完成", zap.Int("totalItems", len(itemIDs)), zap.Int("failedCount", failedCount))
|
||||
return nil
|
||||
}
|
||||
@@ -474,3 +739,10 @@ func (idx *Indexer) GetLastError() (string, time.Time) {
|
||||
defer idx.mu.RUnlock()
|
||||
return idx.lastError, idx.lastErrorTime
|
||||
}
|
||||
|
||||
// GetRebuildStatus 获取重建索引状态
|
||||
func (idx *Indexer) GetRebuildStatus() (isRebuilding bool, totalItems int, current int, failed int, lastItemID string, lastChunks int, startTime time.Time) {
|
||||
idx.rebuildMu.RLock()
|
||||
defer idx.rebuildMu.RUnlock()
|
||||
return idx.isRebuilding, idx.rebuildTotalItems, idx.rebuildCurrent, idx.rebuildFailed, idx.rebuildLastItemID, idx.rebuildLastChunks, idx.rebuildStartTime
|
||||
}
|
||||
|
||||
@@ -153,6 +153,25 @@ func (m *Manager) GetCategories() ([]string, error) {
|
||||
return categories, nil
|
||||
}
|
||||
|
||||
// GetStats 获取知识库统计信息
|
||||
func (m *Manager) GetStats() (int, int, error) {
|
||||
// 获取分类总数
|
||||
categories, err := m.GetCategories()
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("获取分类失败: %w", err)
|
||||
}
|
||||
totalCategories := len(categories)
|
||||
|
||||
// 获取知识项总数
|
||||
var totalItems int
|
||||
err = m.db.QueryRow("SELECT COUNT(*) FROM knowledge_base_items").Scan(&totalItems)
|
||||
if err != nil {
|
||||
return totalCategories, 0, fmt.Errorf("获取知识项总数失败: %w", err)
|
||||
}
|
||||
|
||||
return totalCategories, totalItems, nil
|
||||
}
|
||||
|
||||
// GetCategoriesWithItems 按分类分页获取知识项(每个分类包含其下的所有知识项)
|
||||
// limit: 每页分类数量(0表示不限制)
|
||||
// offset: 偏移量(按分类偏移)
|
||||
@@ -359,7 +378,7 @@ func (m *Manager) SearchItemsByKeyword(keyword string, category string) ([]*Know
|
||||
// SQLite的LIKE不区分大小写,使用COLLATE NOCASE或LOWER()函数
|
||||
// 使用%keyword%进行模糊匹配
|
||||
searchPattern := "%" + keyword + "%"
|
||||
|
||||
|
||||
query = `
|
||||
SELECT id, category, title, file_path, created_at, updated_at
|
||||
FROM knowledge_base_items
|
||||
|
||||
@@ -55,6 +55,14 @@ func New(level, output string) *Logger {
|
||||
}
|
||||
|
||||
func (l *Logger) Fatal(msg string, fields ...interface{}) {
|
||||
l.Logger.Fatal(msg, zap.Any("fields", fields))
|
||||
zapFields := make([]zap.Field, 0, len(fields))
|
||||
for _, f := range fields {
|
||||
switch v := f.(type) {
|
||||
case error:
|
||||
zapFields = append(zapFields, zap.Error(v))
|
||||
default:
|
||||
zapFields = append(zapFields, zap.Any("field", v))
|
||||
}
|
||||
}
|
||||
l.Logger.Fatal(msg, zapFields...)
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
package robot
|
||||
|
||||
// MessageHandler 供飞书/钉钉长连接调用的消息处理接口(由 handler.RobotHandler 实现)
|
||||
type MessageHandler interface {
|
||||
HandleMessage(platform, userID, text string) string
|
||||
}
|
||||
@@ -0,0 +1,137 @@
|
||||
package robot
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"cyberstrike-ai/internal/config"
|
||||
|
||||
"github.com/open-dingtalk/dingtalk-stream-sdk-go/chatbot"
|
||||
"github.com/open-dingtalk/dingtalk-stream-sdk-go/client"
|
||||
dingutils "github.com/open-dingtalk/dingtalk-stream-sdk-go/utils"
|
||||
"go.uber.org/zap"
|
||||
)
|
||||
|
||||
const (
|
||||
dingReconnectInitial = 5 * time.Second // 首次重连间隔
|
||||
dingReconnectMax = 60 * time.Second // 最大重连间隔
|
||||
)
|
||||
|
||||
// StartDing 启动钉钉 Stream 长连接(无需公网),收到消息后调用 handler 并通过 SessionWebhook 回复。
|
||||
// 断线(如笔记本睡眠、网络中断)后会自动重连;ctx 被取消时退出,便于配置变更时重启。
|
||||
func StartDing(ctx context.Context, cfg config.RobotDingtalkConfig, h MessageHandler, logger *zap.Logger) {
|
||||
if !cfg.Enabled || cfg.ClientID == "" || cfg.ClientSecret == "" {
|
||||
return
|
||||
}
|
||||
go runDingLoop(ctx, cfg, h, logger)
|
||||
}
|
||||
|
||||
// runDingLoop 循环维持钉钉长连接:断开且 ctx 未取消时按退避间隔重连。
|
||||
func runDingLoop(ctx context.Context, cfg config.RobotDingtalkConfig, h MessageHandler, logger *zap.Logger) {
|
||||
backoff := dingReconnectInitial
|
||||
for {
|
||||
streamClient := client.NewStreamClient(
|
||||
client.WithAppCredential(client.NewAppCredentialConfig(cfg.ClientID, cfg.ClientSecret)),
|
||||
client.WithSubscription(dingutils.SubscriptionTypeKCallback, "/v1.0/im/bot/messages/get",
|
||||
chatbot.NewDefaultChatBotFrameHandler(func(ctx context.Context, msg *chatbot.BotCallbackDataModel) ([]byte, error) {
|
||||
go handleDingMessage(ctx, msg, h, logger)
|
||||
return nil, nil
|
||||
}).OnEventReceived),
|
||||
)
|
||||
logger.Info("钉钉 Stream 正在连接…", zap.String("client_id", cfg.ClientID))
|
||||
err := streamClient.Start(ctx)
|
||||
if ctx.Err() != nil {
|
||||
logger.Info("钉钉 Stream 已按配置重启关闭")
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
logger.Warn("钉钉 Stream 长连接断开(如睡眠/断网),将自动重连", zap.Error(err), zap.Duration("retry_after", backoff))
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(backoff):
|
||||
// 下次重连间隔递增,上限 60 秒,避免频繁重试
|
||||
if backoff < dingReconnectMax {
|
||||
backoff *= 2
|
||||
if backoff > dingReconnectMax {
|
||||
backoff = dingReconnectMax
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func handleDingMessage(ctx context.Context, msg *chatbot.BotCallbackDataModel, h MessageHandler, logger *zap.Logger) {
|
||||
if msg == nil || msg.SessionWebhook == "" {
|
||||
return
|
||||
}
|
||||
content := ""
|
||||
if msg.Text.Content != "" {
|
||||
content = strings.TrimSpace(msg.Text.Content)
|
||||
}
|
||||
if content == "" && msg.Msgtype == "richText" {
|
||||
if cMap, ok := msg.Content.(map[string]interface{}); ok {
|
||||
if rich, ok := cMap["richText"].([]interface{}); ok {
|
||||
for _, c := range rich {
|
||||
if m, ok := c.(map[string]interface{}); ok {
|
||||
if txt, ok := m["text"].(string); ok {
|
||||
content = strings.TrimSpace(txt)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if content == "" {
|
||||
logger.Debug("钉钉消息内容为空,已忽略", zap.String("msgtype", msg.Msgtype))
|
||||
return
|
||||
}
|
||||
logger.Info("钉钉收到消息", zap.String("sender", msg.SenderId), zap.String("content", content))
|
||||
userID := msg.SenderId
|
||||
if userID == "" {
|
||||
userID = msg.ConversationId
|
||||
}
|
||||
reply := h.HandleMessage("dingtalk", userID, content)
|
||||
// 使用 markdown 类型以便正确展示标题、列表、代码块等格式
|
||||
title := reply
|
||||
if idx := strings.IndexAny(reply, "\n"); idx > 0 {
|
||||
title = strings.TrimSpace(reply[:idx])
|
||||
}
|
||||
if len(title) > 50 {
|
||||
title = title[:50] + "…"
|
||||
}
|
||||
if title == "" {
|
||||
title = "回复"
|
||||
}
|
||||
body := map[string]interface{}{
|
||||
"msgtype": "markdown",
|
||||
"markdown": map[string]string{
|
||||
"title": title,
|
||||
"text": reply,
|
||||
},
|
||||
}
|
||||
bodyBytes, _ := json.Marshal(body)
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, msg.SessionWebhook, bytes.NewReader(bodyBytes))
|
||||
if err != nil {
|
||||
logger.Warn("钉钉构造回复请求失败", zap.Error(err))
|
||||
return
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
logger.Warn("钉钉回复请求失败", zap.Error(err))
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
logger.Warn("钉钉回复非 200", zap.Int("status", resp.StatusCode))
|
||||
return
|
||||
}
|
||||
logger.Debug("钉钉回复成功", zap.String("content_preview", reply))
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
package robot
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"cyberstrike-ai/internal/config"
|
||||
|
||||
lark "github.com/larksuite/oapi-sdk-go/v3"
|
||||
larkcore "github.com/larksuite/oapi-sdk-go/v3/core"
|
||||
"github.com/larksuite/oapi-sdk-go/v3/event/dispatcher"
|
||||
larkim "github.com/larksuite/oapi-sdk-go/v3/service/im/v1"
|
||||
larkws "github.com/larksuite/oapi-sdk-go/v3/ws"
|
||||
"go.uber.org/zap"
|
||||
)
|
||||
|
||||
const (
|
||||
larkReconnectInitial = 5 * time.Second // 首次重连间隔
|
||||
larkReconnectMax = 60 * time.Second // 最大重连间隔
|
||||
)
|
||||
|
||||
type larkTextContent struct {
|
||||
Text string `json:"text"`
|
||||
}
|
||||
|
||||
// StartLark 启动飞书长连接(无需公网),收到消息后调用 handler 并回复。
|
||||
// 断线(如笔记本睡眠、网络中断)后会自动重连;ctx 被取消时退出,便于配置变更时重启。
|
||||
func StartLark(ctx context.Context, cfg config.RobotLarkConfig, h MessageHandler, logger *zap.Logger) {
|
||||
if !cfg.Enabled || cfg.AppID == "" || cfg.AppSecret == "" {
|
||||
return
|
||||
}
|
||||
go runLarkLoop(ctx, cfg, h, logger)
|
||||
}
|
||||
|
||||
// runLarkLoop 循环维持飞书长连接:断开且 ctx 未取消时按退避间隔重连。
|
||||
func runLarkLoop(ctx context.Context, cfg config.RobotLarkConfig, h MessageHandler, logger *zap.Logger) {
|
||||
backoff := larkReconnectInitial
|
||||
for {
|
||||
larkClient := lark.NewClient(cfg.AppID, cfg.AppSecret)
|
||||
eventHandler := dispatcher.NewEventDispatcher("", "").OnP2MessageReceiveV1(func(ctx context.Context, event *larkim.P2MessageReceiveV1) error {
|
||||
go handleLarkMessage(ctx, event, h, larkClient, logger)
|
||||
return nil
|
||||
})
|
||||
wsClient := larkws.NewClient(cfg.AppID, cfg.AppSecret,
|
||||
larkws.WithEventHandler(eventHandler),
|
||||
larkws.WithLogLevel(larkcore.LogLevelInfo),
|
||||
)
|
||||
logger.Info("飞书长连接正在连接…", zap.String("app_id", cfg.AppID))
|
||||
err := wsClient.Start(ctx)
|
||||
if ctx.Err() != nil {
|
||||
logger.Info("飞书长连接已按配置重启关闭")
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
logger.Warn("飞书长连接断开(如睡眠/断网),将自动重连", zap.Error(err), zap.Duration("retry_after", backoff))
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(backoff):
|
||||
if backoff < larkReconnectMax {
|
||||
backoff *= 2
|
||||
if backoff > larkReconnectMax {
|
||||
backoff = larkReconnectMax
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func handleLarkMessage(ctx context.Context, event *larkim.P2MessageReceiveV1, h MessageHandler, client *lark.Client, logger *zap.Logger) {
|
||||
if event == nil || event.Event == nil || event.Event.Message == nil || event.Event.Sender == nil || event.Event.Sender.SenderId == nil {
|
||||
return
|
||||
}
|
||||
msg := event.Event.Message
|
||||
msgType := larkcore.StringValue(msg.MessageType)
|
||||
if msgType != larkim.MsgTypeText {
|
||||
logger.Debug("飞书暂仅处理文本消息", zap.String("msg_type", msgType))
|
||||
return
|
||||
}
|
||||
var textBody larkTextContent
|
||||
if err := json.Unmarshal([]byte(larkcore.StringValue(msg.Content)), &textBody); err != nil {
|
||||
logger.Warn("飞书消息 Content 解析失败", zap.Error(err))
|
||||
return
|
||||
}
|
||||
text := strings.TrimSpace(textBody.Text)
|
||||
if text == "" {
|
||||
return
|
||||
}
|
||||
userID := ""
|
||||
if event.Event.Sender.SenderId.UserId != nil {
|
||||
userID = *event.Event.Sender.SenderId.UserId
|
||||
}
|
||||
messageID := larkcore.StringValue(msg.MessageId)
|
||||
reply := h.HandleMessage("lark", userID, text)
|
||||
contentBytes, _ := json.Marshal(larkTextContent{Text: reply})
|
||||
_, err := client.Im.Message.Reply(ctx, larkim.NewReplyMessageReqBuilder().
|
||||
MessageId(messageID).
|
||||
Body(larkim.NewReplyMessageReqBodyBuilder().
|
||||
MsgType(larkim.MsgTypeText).
|
||||
Content(string(contentBytes)).
|
||||
Build()).
|
||||
Build())
|
||||
if err != nil {
|
||||
logger.Warn("飞书回复失败", zap.String("message_id", messageID), zap.Error(err))
|
||||
return
|
||||
}
|
||||
logger.Debug("飞书已回复", zap.String("message_id", messageID))
|
||||
}
|
||||
+6
-3
@@ -1,8 +1,8 @@
|
||||
# Python HTTP helpers leveraged by tools like api-fuzzer, dnslog, http-intruder, http-framework-test
|
||||
requests>=2.32.3
|
||||
|
||||
# dirsearch:用 python3 -m dirsearch 时由本依赖提供(含 defusedxml 等)
|
||||
dirsearch>=0.4.3
|
||||
httpx>=0.27.0
|
||||
charset-normalizer>=3.3.2
|
||||
chardet>=5.2.0
|
||||
|
||||
# Python exploitation / analysis frameworks referenced by tool recipes
|
||||
# angr>=9.2.96
|
||||
@@ -12,3 +12,6 @@ uro>=1.0.2
|
||||
|
||||
bloodhound>=1.6.1
|
||||
impacket>=0.11.0
|
||||
|
||||
# MCP (Model Context Protocol) SDK
|
||||
mcp>=1.0.0
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
name: "list-files"
|
||||
command: "ls"
|
||||
enabled: true
|
||||
short_description: "列出目录文件工具"
|
||||
description: |
|
||||
列出服务器上指定目录中的文件。
|
||||
|
||||
**主要功能:**
|
||||
- 列出文件
|
||||
- 显示详细信息
|
||||
- 递归列出
|
||||
|
||||
**使用场景:**
|
||||
- 目录浏览
|
||||
- 文件查找
|
||||
- 系统检查
|
||||
parameters:
|
||||
- name: "directory"
|
||||
type: "string"
|
||||
description: "要列出的目录(相对于服务器基础目录)"
|
||||
required: false
|
||||
default: "."
|
||||
position: 0
|
||||
format: "positional"
|
||||
- name: "long_format"
|
||||
type: "bool"
|
||||
description: "显示详细信息(长格式)"
|
||||
required: false
|
||||
flag: "-l"
|
||||
format: "flag"
|
||||
default: false
|
||||
- name: "recursive"
|
||||
type: "bool"
|
||||
description: "递归列出"
|
||||
required: false
|
||||
flag: "-R"
|
||||
format: "flag"
|
||||
default: false
|
||||
- name: "additional_args"
|
||||
type: "string"
|
||||
description: |
|
||||
额外的list-files参数。用于传递未在参数列表中定义的list-files选项。
|
||||
|
||||
**示例值:**
|
||||
- 根据工具特性添加常用参数示例
|
||||
|
||||
**注意事项:**
|
||||
- 多个参数用空格分隔
|
||||
- 确保参数格式正确,避免命令注入
|
||||
- 此参数会直接追加到命令末尾
|
||||
required: false
|
||||
format: "positional"
|
||||
+2
-1
@@ -46,8 +46,9 @@ parameters:
|
||||
**注意事项:**
|
||||
- 必需参数,不能为空
|
||||
- 如果指定进程ID,需要配合 -d 参数使用
|
||||
- 注意:radare2 要求文件路径必须是最后一个参数,因此 target 使用 position 1
|
||||
required: true
|
||||
position: 0
|
||||
position: 1
|
||||
format: "positional"
|
||||
- name: "commands"
|
||||
type: "string"
|
||||
|
||||
+1115
-16
File diff suppressed because it is too large
Load Diff
+165
-15
@@ -22,6 +22,12 @@ const DRAFT_STORAGE_KEY = 'cyberstrike-chat-draft';
|
||||
let draftSaveTimer = null;
|
||||
const DRAFT_SAVE_DELAY = 500; // 500ms防抖延迟
|
||||
|
||||
// 对话文件上传相关(后端会拼接路径与内容发给大模型,前端不再重复发文件列表)
|
||||
const MAX_CHAT_FILES = 10;
|
||||
const CHAT_FILE_DEFAULT_PROMPT = '请根据上传的文件内容进行分析。';
|
||||
/** @type {{ fileName: string, content: string, mimeType: string }[]} */
|
||||
let chatAttachments = [];
|
||||
|
||||
// 保存输入框草稿到localStorage(防抖版本)
|
||||
function saveChatDraftDebounced(content) {
|
||||
// 清除之前的定时器
|
||||
@@ -107,14 +113,22 @@ function adjustTextareaHeight(textarea) {
|
||||
// 发送消息
|
||||
async function sendMessage() {
|
||||
const input = document.getElementById('chat-input');
|
||||
const message = input.value.trim();
|
||||
|
||||
if (!message) {
|
||||
let message = input.value.trim();
|
||||
const hasAttachments = chatAttachments && chatAttachments.length > 0;
|
||||
|
||||
if (!message && !hasAttachments) {
|
||||
return;
|
||||
}
|
||||
|
||||
// 显示用户消息
|
||||
addMessage('user', message);
|
||||
// 有附件且用户未输入时,发一句简短默认提示即可(后端会拼接路径和文件内容给大模型)
|
||||
if (hasAttachments && !message) {
|
||||
message = CHAT_FILE_DEFAULT_PROMPT;
|
||||
}
|
||||
|
||||
// 显示用户消息(含附件名,便于用户确认)
|
||||
const displayMessage = hasAttachments
|
||||
? message + '\n' + chatAttachments.map(a => '📎 ' + a.fileName).join('\n')
|
||||
: message;
|
||||
addMessage('user', displayMessage);
|
||||
|
||||
// 清除防抖定时器,防止在清空输入框后重新保存草稿
|
||||
if (draftSaveTimer) {
|
||||
@@ -135,7 +149,24 @@ async function sendMessage() {
|
||||
input.value = '';
|
||||
// 强制重置输入框高度为初始高度(40px)
|
||||
input.style.height = '40px';
|
||||
|
||||
|
||||
// 构建请求体(含附件)
|
||||
const body = {
|
||||
message: message,
|
||||
conversationId: currentConversationId,
|
||||
role: typeof getCurrentRole === 'function' ? getCurrentRole() : ''
|
||||
};
|
||||
if (hasAttachments) {
|
||||
body.attachments = chatAttachments.map(a => ({
|
||||
fileName: a.fileName,
|
||||
content: a.content,
|
||||
mimeType: a.mimeType || ''
|
||||
}));
|
||||
}
|
||||
// 发送后清空附件列表
|
||||
chatAttachments = [];
|
||||
renderChatFileChips();
|
||||
|
||||
// 创建进度消息容器(使用详细的进度展示)
|
||||
const progressId = addProgressMessage();
|
||||
const progressElement = document.getElementById(progressId);
|
||||
@@ -145,19 +176,12 @@ async function sendMessage() {
|
||||
let mcpExecutionIds = [];
|
||||
|
||||
try {
|
||||
// 获取当前选中的角色(从 roles.js 的函数获取)
|
||||
const roleName = typeof getCurrentRole === 'function' ? getCurrentRole() : '';
|
||||
|
||||
const response = await apiFetch('/api/agent-loop/stream', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
message: message,
|
||||
conversationId: currentConversationId,
|
||||
role: roleName || undefined
|
||||
}),
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
@@ -222,6 +246,130 @@ async function sendMessage() {
|
||||
}
|
||||
}
|
||||
|
||||
// ---------- 对话文件上传 ----------
|
||||
function renderChatFileChips() {
|
||||
const list = document.getElementById('chat-file-list');
|
||||
if (!list) return;
|
||||
list.innerHTML = '';
|
||||
if (!chatAttachments.length) return;
|
||||
chatAttachments.forEach((a, i) => {
|
||||
const chip = document.createElement('div');
|
||||
chip.className = 'chat-file-chip';
|
||||
chip.setAttribute('role', 'listitem');
|
||||
const name = document.createElement('span');
|
||||
name.className = 'chat-file-chip-name';
|
||||
name.title = a.fileName;
|
||||
name.textContent = a.fileName;
|
||||
const remove = document.createElement('button');
|
||||
remove.type = 'button';
|
||||
remove.className = 'chat-file-chip-remove';
|
||||
remove.title = '移除';
|
||||
remove.innerHTML = '×';
|
||||
remove.setAttribute('aria-label', '移除 ' + a.fileName);
|
||||
remove.addEventListener('click', () => removeChatAttachment(i));
|
||||
chip.appendChild(name);
|
||||
chip.appendChild(remove);
|
||||
list.appendChild(chip);
|
||||
});
|
||||
}
|
||||
|
||||
function removeChatAttachment(index) {
|
||||
chatAttachments.splice(index, 1);
|
||||
renderChatFileChips();
|
||||
}
|
||||
|
||||
// 有附件且输入框为空时,填入一句默认提示(可编辑);后端会单独拼接路径与内容给大模型
|
||||
function appendChatFilePrompt() {
|
||||
const input = document.getElementById('chat-input');
|
||||
if (!input || !chatAttachments.length) return;
|
||||
if (!input.value.trim()) {
|
||||
input.value = CHAT_FILE_DEFAULT_PROMPT;
|
||||
adjustTextareaHeight(input);
|
||||
}
|
||||
}
|
||||
|
||||
function readFileAsAttachment(file) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const mimeType = file.type || '';
|
||||
const isTextLike = /^text\//i.test(mimeType) || /^(application\/(json|xml|javascript)|image\/svg\+xml)/i.test(mimeType);
|
||||
const reader = new FileReader();
|
||||
reader.onload = () => {
|
||||
let content = reader.result;
|
||||
if (typeof content === 'string' && content.startsWith('data:')) {
|
||||
content = content.replace(/^data:[^;]+;base64,/, '');
|
||||
}
|
||||
resolve({ fileName: file.name, content: content, mimeType: mimeType });
|
||||
};
|
||||
reader.onerror = () => reject(reader.error);
|
||||
if (isTextLike) {
|
||||
reader.readAsText(file, 'UTF-8');
|
||||
} else {
|
||||
reader.readAsDataURL(file);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
function addFilesToChat(files) {
|
||||
if (!files || !files.length) return;
|
||||
const next = Array.from(files);
|
||||
if (chatAttachments.length + next.length > MAX_CHAT_FILES) {
|
||||
alert('最多同时上传 ' + MAX_CHAT_FILES + ' 个文件,当前已选 ' + chatAttachments.length + ' 个。');
|
||||
return;
|
||||
}
|
||||
const addOne = (file) => {
|
||||
return readFileAsAttachment(file).then((a) => {
|
||||
chatAttachments.push(a);
|
||||
renderChatFileChips();
|
||||
appendChatFilePrompt();
|
||||
}).catch(() => {
|
||||
alert('读取文件失败:' + file.name);
|
||||
});
|
||||
};
|
||||
let p = Promise.resolve();
|
||||
next.forEach((file) => { p = p.then(() => addOne(file)); });
|
||||
p.then(() => {});
|
||||
}
|
||||
|
||||
function setupChatFileUpload() {
|
||||
const inputEl = document.getElementById('chat-file-input');
|
||||
const container = document.getElementById('chat-input-container') || document.querySelector('.chat-input-container');
|
||||
if (!inputEl || !container) return;
|
||||
|
||||
inputEl.addEventListener('change', function () {
|
||||
const files = this.files;
|
||||
if (files && files.length) {
|
||||
addFilesToChat(files);
|
||||
}
|
||||
this.value = '';
|
||||
});
|
||||
|
||||
container.addEventListener('dragover', function (e) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
this.classList.add('drag-over');
|
||||
});
|
||||
container.addEventListener('dragleave', function (e) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (!this.contains(e.relatedTarget)) {
|
||||
this.classList.remove('drag-over');
|
||||
}
|
||||
});
|
||||
container.addEventListener('drop', function (e) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
this.classList.remove('drag-over');
|
||||
const files = e.dataTransfer && e.dataTransfer.files;
|
||||
if (files && files.length) addFilesToChat(files);
|
||||
});
|
||||
}
|
||||
|
||||
// 确保 chat-input-container 有 id(若模板未写)
|
||||
function ensureChatInputContainerId() {
|
||||
const c = document.querySelector('.chat-input-container');
|
||||
if (c && !c.id) c.id = 'chat-input-container';
|
||||
}
|
||||
|
||||
function setupMentionSupport() {
|
||||
mentionSuggestionsEl = document.getElementById('mention-suggestions');
|
||||
if (mentionSuggestionsEl) {
|
||||
@@ -799,6 +947,8 @@ function initializeChatUI() {
|
||||
}
|
||||
activeTaskInterval = setInterval(() => loadActiveTasks(), ACTIVE_TASK_REFRESH_INTERVAL);
|
||||
setupMentionSupport();
|
||||
ensureChatInputContainerId();
|
||||
setupChatFileUpload();
|
||||
}
|
||||
|
||||
// 消息计数器,确保ID唯一
|
||||
|
||||
+165
-15
@@ -17,7 +17,7 @@ async function refreshDashboard() {
|
||||
setEl('dashboard-kpi-tools-calls', '…');
|
||||
setEl('dashboard-kpi-success-rate', '…');
|
||||
var chartPlaceholder = document.getElementById('dashboard-tools-pie-placeholder');
|
||||
if (chartPlaceholder) { chartPlaceholder.style.display = 'block'; chartPlaceholder.textContent = '加载中…'; }
|
||||
if (chartPlaceholder) { chartPlaceholder.style.removeProperty('display'); chartPlaceholder.textContent = '加载中…'; }
|
||||
var barChartEl = document.getElementById('dashboard-tools-bar-chart');
|
||||
if (barChartEl) { barChartEl.style.display = 'none'; barChartEl.innerHTML = ''; }
|
||||
|
||||
@@ -29,11 +29,12 @@ async function refreshDashboard() {
|
||||
}
|
||||
|
||||
try {
|
||||
const [tasksRes, vulnRes, batchRes, monitorRes, skillsRes] = await Promise.all([
|
||||
const [tasksRes, vulnRes, batchRes, monitorRes, knowledgeRes, skillsRes] = await Promise.all([
|
||||
apiFetch('/api/agent-loop/tasks').then(r => r.ok ? r.json() : null).catch(() => null),
|
||||
apiFetch('/api/vulnerabilities/stats').then(r => r.ok ? r.json() : null).catch(() => null),
|
||||
apiFetch('/api/batch-tasks?limit=500&page=1').then(r => r.ok ? r.json() : null).catch(() => null),
|
||||
apiFetch('/api/monitor/stats').then(r => r.ok ? r.json() : null).catch(() => null),
|
||||
apiFetch('/api/knowledge/stats').then(r => r.ok ? r.json() : null).catch(() => null),
|
||||
apiFetch('/api/skills/stats').then(r => r.ok ? r.json() : null).catch(() => null)
|
||||
]);
|
||||
|
||||
@@ -62,7 +63,7 @@ async function refreshDashboard() {
|
||||
});
|
||||
}
|
||||
|
||||
// 批量任务队列:按状态统计
|
||||
// 批量任务队列:按状态统计(优化版)
|
||||
if (batchRes && Array.isArray(batchRes.queues)) {
|
||||
const queues = batchRes.queues;
|
||||
let pending = 0, running = 0, done = 0;
|
||||
@@ -72,16 +73,36 @@ async function refreshDashboard() {
|
||||
else if (s === 'running') running++;
|
||||
else if (s === 'completed' || s === 'cancelled') done++;
|
||||
});
|
||||
const total = pending + running + done;
|
||||
setEl('dashboard-batch-pending', String(pending));
|
||||
setEl('dashboard-batch-running', String(running));
|
||||
setEl('dashboard-batch-done', String(done));
|
||||
setEl('dashboard-batch-total', total > 0 ? `共 ${total} 个` : '暂无任务');
|
||||
|
||||
// 更新进度条
|
||||
if (total > 0) {
|
||||
const pendingPct = (pending / total * 100).toFixed(1);
|
||||
const runningPct = (running / total * 100).toFixed(1);
|
||||
const donePct = (done / total * 100).toFixed(1);
|
||||
updateProgressBar('dashboard-batch-progress-pending', pendingPct);
|
||||
updateProgressBar('dashboard-batch-progress-running', runningPct);
|
||||
updateProgressBar('dashboard-batch-progress-done', donePct);
|
||||
} else {
|
||||
updateProgressBar('dashboard-batch-progress-pending', '0');
|
||||
updateProgressBar('dashboard-batch-progress-running', '0');
|
||||
updateProgressBar('dashboard-batch-progress-done', '0');
|
||||
}
|
||||
} else {
|
||||
setEl('dashboard-batch-pending', '-');
|
||||
setEl('dashboard-batch-running', '-');
|
||||
setEl('dashboard-batch-done', '-');
|
||||
setEl('dashboard-batch-total', '-');
|
||||
updateProgressBar('dashboard-batch-progress-pending', '0');
|
||||
updateProgressBar('dashboard-batch-progress-running', '0');
|
||||
updateProgressBar('dashboard-batch-progress-done', '0');
|
||||
}
|
||||
|
||||
// 工具调用:monitor/stats 为 { toolName: { totalCalls, successCalls, failedCalls, ... } }
|
||||
// 工具调用:monitor/stats 为 { toolName: { totalCalls, successCalls, failedCalls, ... } }(优化版)
|
||||
if (monitorRes && typeof monitorRes === 'object') {
|
||||
const names = Object.keys(monitorRes);
|
||||
let totalCalls = 0, totalSuccess = 0, totalFailed = 0;
|
||||
@@ -95,26 +116,80 @@ async function refreshDashboard() {
|
||||
if (typeof f === 'number') totalFailed += f;
|
||||
});
|
||||
setEl('dashboard-tools-count', String(names.length));
|
||||
setEl('dashboard-tools-calls', String(totalCalls));
|
||||
setEl('dashboard-tools-calls', formatNumber(totalCalls));
|
||||
setEl('dashboard-kpi-tools-calls', String(totalCalls));
|
||||
var rateStr = totalCalls > 0 ? ((totalSuccess / totalCalls) * 100).toFixed(1) + '%' : '-';
|
||||
setEl('dashboard-kpi-success-rate', rateStr);
|
||||
setEl('dashboard-tools-success-rate', rateStr !== '-' ? `成功率 ${rateStr}` : '-');
|
||||
renderDashboardToolsBar(monitorRes);
|
||||
} else {
|
||||
setEl('dashboard-tools-count', '-');
|
||||
setEl('dashboard-tools-calls', '-');
|
||||
setEl('dashboard-kpi-tools-calls', '-');
|
||||
setEl('dashboard-kpi-success-rate', '-');
|
||||
setEl('dashboard-tools-success-rate', '-');
|
||||
renderDashboardToolsBar(null);
|
||||
}
|
||||
|
||||
// Skills:{ total_skills, total_calls, ... }
|
||||
// 知识:{ enabled, total_categories, total_items, ... }(优化版)
|
||||
const knowledgeItemsEl = document.getElementById('dashboard-knowledge-items');
|
||||
const knowledgeCategoriesEl = document.getElementById('dashboard-knowledge-categories');
|
||||
const knowledgeStatusEl = document.getElementById('dashboard-knowledge-status');
|
||||
if (knowledgeRes && typeof knowledgeRes === 'object') {
|
||||
if (knowledgeRes.enabled === false) {
|
||||
// 功能未启用:用状态标签展示,数值保持为 "-"
|
||||
if (knowledgeStatusEl) knowledgeStatusEl.textContent = '未启用';
|
||||
if (knowledgeItemsEl) knowledgeItemsEl.textContent = '-';
|
||||
if (knowledgeCategoriesEl) knowledgeCategoriesEl.textContent = '-';
|
||||
} else {
|
||||
const categories = knowledgeRes.total_categories ?? 0;
|
||||
const items = knowledgeRes.total_items ?? 0;
|
||||
if (knowledgeItemsEl) knowledgeItemsEl.textContent = formatNumber(items);
|
||||
if (knowledgeCategoriesEl) knowledgeCategoriesEl.textContent = formatNumber(categories);
|
||||
// 根据数据量给个轻量状态文案
|
||||
if (knowledgeStatusEl) {
|
||||
if (items > 0 || categories > 0) {
|
||||
knowledgeStatusEl.textContent = '已启用';
|
||||
} else {
|
||||
knowledgeStatusEl.textContent = '待配置';
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if (knowledgeItemsEl) knowledgeItemsEl.textContent = '-';
|
||||
if (knowledgeCategoriesEl) knowledgeCategoriesEl.textContent = '-';
|
||||
if (knowledgeStatusEl) knowledgeStatusEl.textContent = '-';
|
||||
}
|
||||
|
||||
// Skills:{ total_skills, total_calls, ... }(优化版)
|
||||
if (skillsRes && typeof skillsRes === 'object') {
|
||||
setEl('dashboard-skills-count', String(skillsRes.total_skills ?? '-'));
|
||||
setEl('dashboard-skills-calls', String(skillsRes.total_calls ?? '-'));
|
||||
const totalSkills = skillsRes.total_skills ?? 0;
|
||||
const totalCalls = skillsRes.total_calls ?? 0;
|
||||
setEl('dashboard-skills-count', formatNumber(totalSkills));
|
||||
setEl('dashboard-skills-calls', formatNumber(totalCalls));
|
||||
|
||||
// 设置状态标签
|
||||
const statusEl = document.getElementById('dashboard-skills-status');
|
||||
if (statusEl) {
|
||||
if (totalCalls === 0) {
|
||||
statusEl.textContent = '待使用';
|
||||
statusEl.style.background = 'rgba(0, 0, 0, 0.05)';
|
||||
statusEl.style.color = 'var(--text-secondary)';
|
||||
} else if (totalCalls < 10) {
|
||||
statusEl.textContent = '活跃';
|
||||
statusEl.style.background = 'rgba(16, 185, 129, 0.1)';
|
||||
statusEl.style.color = '#10b981';
|
||||
} else {
|
||||
statusEl.textContent = '高频';
|
||||
statusEl.style.background = 'rgba(59, 130, 246, 0.1)';
|
||||
statusEl.style.color = '#3b82f6';
|
||||
}
|
||||
}
|
||||
} else {
|
||||
setEl('dashboard-skills-count', '-');
|
||||
setEl('dashboard-skills-calls', '-');
|
||||
const statusEl = document.getElementById('dashboard-skills-status');
|
||||
if (statusEl) statusEl.textContent = '-';
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn('仪表盘拉取统计失败', e);
|
||||
@@ -125,7 +200,7 @@ async function refreshDashboard() {
|
||||
setEl('dashboard-kpi-tools-calls', '-');
|
||||
renderDashboardToolsBar(null);
|
||||
var ph = document.getElementById('dashboard-tools-pie-placeholder');
|
||||
if (ph) { ph.style.display = 'block'; ph.textContent = '暂无调用数据'; }
|
||||
if (ph) { ph.style.removeProperty('display'); ph.textContent = '暂无调用数据'; }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -135,8 +210,29 @@ function setEl(id, text) {
|
||||
}
|
||||
|
||||
function setDashboardOverviewPlaceholder(t) {
|
||||
['dashboard-batch-pending', 'dashboard-batch-running', 'dashboard-batch-done',
|
||||
'dashboard-tools-count', 'dashboard-tools-calls', 'dashboard-skills-count', 'dashboard-skills-calls'].forEach(id => setEl(id, t));
|
||||
['dashboard-batch-pending', 'dashboard-batch-running', 'dashboard-batch-done', 'dashboard-batch-total',
|
||||
'dashboard-tools-count', 'dashboard-tools-calls', 'dashboard-tools-success-rate',
|
||||
'dashboard-skills-count', 'dashboard-skills-calls', 'dashboard-skills-status',
|
||||
'dashboard-knowledge-items', 'dashboard-knowledge-categories', 'dashboard-knowledge-status'].forEach(id => setEl(id, t));
|
||||
updateProgressBar('dashboard-batch-progress-pending', '0');
|
||||
updateProgressBar('dashboard-batch-progress-running', '0');
|
||||
updateProgressBar('dashboard-batch-progress-done', '0');
|
||||
}
|
||||
|
||||
// 格式化数字,添加千位分隔符
|
||||
function formatNumber(num) {
|
||||
if (typeof num !== 'number' || isNaN(num)) return '-';
|
||||
if (num === 0) return '0';
|
||||
return num.toLocaleString('zh-CN');
|
||||
}
|
||||
|
||||
// 更新进度条宽度
|
||||
function updateProgressBar(id, percentage) {
|
||||
const el = document.getElementById(id);
|
||||
if (el) {
|
||||
const pct = parseFloat(percentage) || 0;
|
||||
el.style.width = Math.max(0, Math.min(100, pct)) + '%';
|
||||
}
|
||||
}
|
||||
|
||||
// Top 30 工具执行次数柱状图颜色(30 色不重复,柔和、易区分)
|
||||
@@ -160,7 +256,8 @@ function renderDashboardToolsBar(monitorRes) {
|
||||
if (!placeholder || !barChartEl) return;
|
||||
|
||||
if (!monitorRes || typeof monitorRes !== 'object') {
|
||||
placeholder.style.display = 'block';
|
||||
placeholder.style.removeProperty('display');
|
||||
placeholder.textContent = '暂无调用数据';
|
||||
barChartEl.style.display = 'none';
|
||||
barChartEl.innerHTML = '';
|
||||
return;
|
||||
@@ -175,7 +272,8 @@ function renderDashboardToolsBar(monitorRes) {
|
||||
.slice(0, 30);
|
||||
|
||||
if (entries.length === 0) {
|
||||
placeholder.style.display = 'block';
|
||||
placeholder.style.removeProperty('display');
|
||||
placeholder.textContent = '暂无调用数据';
|
||||
barChartEl.style.display = 'none';
|
||||
barChartEl.innerHTML = '';
|
||||
return;
|
||||
@@ -190,11 +288,63 @@ function renderDashboardToolsBar(monitorRes) {
|
||||
var pct = maxCalls > 0 ? (e.totalCalls / maxCalls) * 100 : 0;
|
||||
var label = e.name.length > 12 ? e.name.slice(0, 10) + '…' : e.name;
|
||||
var color = DASHBOARD_BAR_COLORS[i % DASHBOARD_BAR_COLORS.length];
|
||||
html += '<div class="dashboard-tools-bar-item">';
|
||||
html += '<span class="dashboard-tools-bar-label" title="' + esc(e.name) + '">' + esc(label) + '</span>';
|
||||
var fullName = esc(e.name);
|
||||
html += '<div class="dashboard-tools-bar-item" data-tooltip="' + fullName + '">';
|
||||
html += '<span class="dashboard-tools-bar-label">' + esc(label) + '</span>';
|
||||
html += '<div class="dashboard-tools-bar-track"><div class="dashboard-tools-bar-fill" style="width:' + pct + '%;background:' + color + '"></div></div>';
|
||||
html += '<span class="dashboard-tools-bar-value">' + e.totalCalls + '</span>';
|
||||
html += '</div>';
|
||||
});
|
||||
barChartEl.innerHTML = html;
|
||||
attachDashboardBarTooltips(barChartEl);
|
||||
}
|
||||
|
||||
var dashboardBarTooltipEl = null;
|
||||
var dashboardBarTooltipTimer = null;
|
||||
|
||||
function attachDashboardBarTooltips(barChartEl) {
|
||||
if (!barChartEl) return;
|
||||
if (!dashboardBarTooltipEl) {
|
||||
dashboardBarTooltipEl = document.createElement('div');
|
||||
dashboardBarTooltipEl.className = 'dashboard-tools-bar-tooltip';
|
||||
dashboardBarTooltipEl.setAttribute('role', 'tooltip');
|
||||
document.body.appendChild(dashboardBarTooltipEl);
|
||||
}
|
||||
barChartEl.removeEventListener('mouseover', dashboardBarTooltipOnOver);
|
||||
barChartEl.removeEventListener('mouseout', dashboardBarTooltipOnOut);
|
||||
barChartEl.addEventListener('mouseover', dashboardBarTooltipOnOver);
|
||||
barChartEl.addEventListener('mouseout', dashboardBarTooltipOnOut);
|
||||
}
|
||||
|
||||
function dashboardBarTooltipOnOver(ev) {
|
||||
var item = ev.target && ev.target.closest && ev.target.closest('.dashboard-tools-bar-item');
|
||||
if (!item || !dashboardBarTooltipEl) return;
|
||||
var text = item.getAttribute('data-tooltip');
|
||||
if (!text) return;
|
||||
clearTimeout(dashboardBarTooltipTimer);
|
||||
dashboardBarTooltipTimer = setTimeout(function () {
|
||||
dashboardBarTooltipEl.textContent = text;
|
||||
dashboardBarTooltipEl.style.display = 'block';
|
||||
requestAnimationFrame(function () {
|
||||
var rect = item.getBoundingClientRect();
|
||||
var ttRect = dashboardBarTooltipEl.getBoundingClientRect();
|
||||
var x = rect.left + (rect.width / 2) - (ttRect.width / 2);
|
||||
var y = rect.top - ttRect.height - 6;
|
||||
if (y < 8) y = rect.bottom + 6;
|
||||
var pad = 8;
|
||||
if (x < pad) x = pad;
|
||||
if (x + ttRect.width > window.innerWidth - pad) x = window.innerWidth - ttRect.width - pad;
|
||||
dashboardBarTooltipEl.style.left = x + 'px';
|
||||
dashboardBarTooltipEl.style.top = y + 'px';
|
||||
});
|
||||
}, 180);
|
||||
}
|
||||
|
||||
function dashboardBarTooltipOnOut(ev) {
|
||||
var item = ev.target && ev.target.closest && ev.target.closest('.dashboard-tools-bar-item');
|
||||
var related = ev.relatedTarget && ev.relatedTarget.closest && ev.relatedTarget.closest('.dashboard-tools-bar-item');
|
||||
if (item && item === related) return;
|
||||
clearTimeout(dashboardBarTooltipTimer);
|
||||
dashboardBarTooltipTimer = null;
|
||||
if (dashboardBarTooltipEl) dashboardBarTooltipEl.style.display = 'none';
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -459,6 +459,9 @@ async function updateIndexProgress() {
|
||||
const isComplete = status.is_complete || false;
|
||||
const lastError = status.last_error || '';
|
||||
|
||||
// 检查是否正在重建索引(优先使用重建状态)
|
||||
const isRebuilding = status.is_rebuilding || false;
|
||||
|
||||
if (totalItems === 0) {
|
||||
// 没有知识项,隐藏进度条
|
||||
progressContainer.style.display = 'none';
|
||||
@@ -524,6 +527,45 @@ async function updateIndexProgress() {
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
// 优先处理重建状态
|
||||
if (isRebuilding) {
|
||||
const rebuildTotal = status.rebuild_total || totalItems;
|
||||
const rebuildCurrent = status.rebuild_current || 0;
|
||||
const rebuildFailed = status.rebuild_failed || 0;
|
||||
const rebuildLastItemID = status.rebuild_last_item_id || '';
|
||||
const rebuildLastChunks = status.rebuild_last_chunks || 0;
|
||||
const rebuildStartTime = status.rebuild_start_time || '';
|
||||
|
||||
// 计算进度百分比(使用重建进度)
|
||||
let rebuildProgress = progressPercent;
|
||||
if (rebuildTotal > 0) {
|
||||
rebuildProgress = (rebuildCurrent / rebuildTotal) * 100;
|
||||
}
|
||||
|
||||
progressContainer.innerHTML = `
|
||||
<div class="knowledge-index-progress">
|
||||
<div class="progress-header">
|
||||
<span class="progress-icon">🔨</span>
|
||||
<span class="progress-text">正在重建索引:${rebuildCurrent}/${rebuildTotal} (${rebuildProgress.toFixed(1)}%) - 失败:${rebuildFailed}</span>
|
||||
</div>
|
||||
<div class="progress-bar-container">
|
||||
<div class="progress-bar" style="width: ${rebuildProgress}%"></div>
|
||||
</div>
|
||||
<div class="progress-hint">
|
||||
${rebuildLastItemID ? `正在处理:${escapeHtml(rebuildLastItemID.substring(0, 36))}... (${rebuildLastChunks} chunks)` : '正在处理...'}
|
||||
${rebuildStartTime ? `<br>开始时间:${new Date(rebuildStartTime).toLocaleString()}` : ''}
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
// 重建中时继续轮询
|
||||
if (!indexProgressInterval) {
|
||||
indexProgressInterval = setInterval(updateIndexProgress, 2000);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (isComplete) {
|
||||
progressContainer.innerHTML = `
|
||||
<div class="knowledge-index-progress-complete">
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -8,7 +8,7 @@ function initRouter() {
|
||||
if (hash) {
|
||||
const hashParts = hash.split('?');
|
||||
const pageId = hashParts[0];
|
||||
if (pageId && ['dashboard', 'chat', 'vulnerabilities', 'mcp-monitor', 'mcp-management', 'knowledge-management', 'knowledge-retrieval-logs', 'roles-management', 'skills-monitor', 'skills-management', 'settings', 'tasks'].includes(pageId)) {
|
||||
if (pageId && ['dashboard', 'chat', 'info-collect', 'vulnerabilities', 'mcp-monitor', 'mcp-management', 'knowledge-management', 'knowledge-retrieval-logs', 'roles-management', 'skills-monitor', 'skills-management', 'settings', 'tasks'].includes(pageId)) {
|
||||
switchPage(pageId);
|
||||
|
||||
// 如果是chat页面且带有conversation参数,加载对应对话
|
||||
@@ -245,6 +245,12 @@ function initPage(pageId) {
|
||||
case 'chat':
|
||||
// 对话页面已由chat.js初始化
|
||||
break;
|
||||
case 'info-collect':
|
||||
// 信息收集页面
|
||||
if (typeof initInfoCollectPage === 'function') {
|
||||
initInfoCollectPage();
|
||||
}
|
||||
break;
|
||||
case 'tasks':
|
||||
// 初始化任务管理页面
|
||||
if (typeof initTasksPage === 'function') {
|
||||
@@ -355,7 +361,7 @@ document.addEventListener('DOMContentLoaded', function() {
|
||||
const hashParts = hash.split('?');
|
||||
const pageId = hashParts[0];
|
||||
|
||||
if (pageId && ['chat', 'tasks', 'vulnerabilities', 'mcp-monitor', 'mcp-management', 'knowledge-management', 'knowledge-retrieval-logs', 'roles-management', 'skills-monitor', 'skills-management', 'settings'].includes(pageId)) {
|
||||
if (pageId && ['chat', 'info-collect', 'tasks', 'vulnerabilities', 'mcp-monitor', 'mcp-management', 'knowledge-management', 'knowledge-retrieval-logs', 'roles-management', 'skills-monitor', 'skills-management', 'settings'].includes(pageId)) {
|
||||
switchPage(pageId);
|
||||
|
||||
// 如果是chat页面且带有conversation参数,加载对应对话
|
||||
|
||||
@@ -46,6 +46,9 @@ function switchSettingsSection(section) {
|
||||
if (activeContent) {
|
||||
activeContent.classList.add('active');
|
||||
}
|
||||
if (section === 'terminal' && typeof initTerminal === 'function') {
|
||||
setTimeout(initTerminal, 0);
|
||||
}
|
||||
}
|
||||
|
||||
// 打开设置
|
||||
@@ -102,6 +105,15 @@ async function loadConfig(loadTools = true) {
|
||||
document.getElementById('openai-api-key').value = currentConfig.openai.api_key || '';
|
||||
document.getElementById('openai-base-url').value = currentConfig.openai.base_url || '';
|
||||
document.getElementById('openai-model').value = currentConfig.openai.model || '';
|
||||
|
||||
// 填充FOFA配置
|
||||
const fofa = currentConfig.fofa || {};
|
||||
const fofaEmailEl = document.getElementById('fofa-email');
|
||||
const fofaKeyEl = document.getElementById('fofa-api-key');
|
||||
const fofaBaseUrlEl = document.getElementById('fofa-base-url');
|
||||
if (fofaEmailEl) fofaEmailEl.value = fofa.email || '';
|
||||
if (fofaKeyEl) fofaKeyEl.value = fofa.api_key || '';
|
||||
if (fofaBaseUrlEl) fofaBaseUrlEl.value = fofa.base_url || '';
|
||||
|
||||
// 填充Agent配置
|
||||
document.getElementById('agent-max-iterations').value = currentConfig.agent.max_iterations || 30;
|
||||
@@ -160,7 +172,76 @@ async function loadConfig(loadTools = true) {
|
||||
// 允许0.0值,只有undefined/null时才使用默认值
|
||||
retrievalWeightInput.value = (hybridWeight !== undefined && hybridWeight !== null) ? hybridWeight : 0.7;
|
||||
}
|
||||
|
||||
// 索引配置
|
||||
const indexing = knowledge.indexing || {};
|
||||
const chunkSizeInput = document.getElementById('knowledge-indexing-chunk-size');
|
||||
if (chunkSizeInput) {
|
||||
chunkSizeInput.value = indexing.chunk_size || 512;
|
||||
}
|
||||
|
||||
const chunkOverlapInput = document.getElementById('knowledge-indexing-chunk-overlap');
|
||||
if (chunkOverlapInput) {
|
||||
chunkOverlapInput.value = indexing.chunk_overlap ?? 50;
|
||||
}
|
||||
|
||||
const maxChunksPerItemInput = document.getElementById('knowledge-indexing-max-chunks-per-item');
|
||||
if (maxChunksPerItemInput) {
|
||||
maxChunksPerItemInput.value = indexing.max_chunks_per_item ?? 0;
|
||||
}
|
||||
|
||||
const maxRpmInput = document.getElementById('knowledge-indexing-max-rpm');
|
||||
if (maxRpmInput) {
|
||||
maxRpmInput.value = indexing.max_rpm ?? 0;
|
||||
}
|
||||
|
||||
const rateLimitDelayInput = document.getElementById('knowledge-indexing-rate-limit-delay-ms');
|
||||
if (rateLimitDelayInput) {
|
||||
rateLimitDelayInput.value = indexing.rate_limit_delay_ms ?? 300;
|
||||
}
|
||||
|
||||
const maxRetriesInput = document.getElementById('knowledge-indexing-max-retries');
|
||||
if (maxRetriesInput) {
|
||||
maxRetriesInput.value = indexing.max_retries ?? 3;
|
||||
}
|
||||
|
||||
const retryDelayInput = document.getElementById('knowledge-indexing-retry-delay-ms');
|
||||
if (retryDelayInput) {
|
||||
retryDelayInput.value = indexing.retry_delay_ms ?? 1000;
|
||||
}
|
||||
}
|
||||
|
||||
// 填充机器人配置
|
||||
const robots = currentConfig.robots || {};
|
||||
const wecom = robots.wecom || {};
|
||||
const dingtalk = robots.dingtalk || {};
|
||||
const lark = robots.lark || {};
|
||||
const wecomEnabled = document.getElementById('robot-wecom-enabled');
|
||||
if (wecomEnabled) wecomEnabled.checked = wecom.enabled === true;
|
||||
const wecomToken = document.getElementById('robot-wecom-token');
|
||||
if (wecomToken) wecomToken.value = wecom.token || '';
|
||||
const wecomAes = document.getElementById('robot-wecom-encoding-aes-key');
|
||||
if (wecomAes) wecomAes.value = wecom.encoding_aes_key || '';
|
||||
const wecomCorp = document.getElementById('robot-wecom-corp-id');
|
||||
if (wecomCorp) wecomCorp.value = wecom.corp_id || '';
|
||||
const wecomSecret = document.getElementById('robot-wecom-secret');
|
||||
if (wecomSecret) wecomSecret.value = wecom.secret || '';
|
||||
const wecomAgentId = document.getElementById('robot-wecom-agent-id');
|
||||
if (wecomAgentId) wecomAgentId.value = wecom.agent_id || '0';
|
||||
const dingtalkEnabled = document.getElementById('robot-dingtalk-enabled');
|
||||
if (dingtalkEnabled) dingtalkEnabled.checked = dingtalk.enabled === true;
|
||||
const dingtalkClientId = document.getElementById('robot-dingtalk-client-id');
|
||||
if (dingtalkClientId) dingtalkClientId.value = dingtalk.client_id || '';
|
||||
const dingtalkClientSecret = document.getElementById('robot-dingtalk-client-secret');
|
||||
if (dingtalkClientSecret) dingtalkClientSecret.value = dingtalk.client_secret || '';
|
||||
const larkEnabled = document.getElementById('robot-lark-enabled');
|
||||
if (larkEnabled) larkEnabled.checked = lark.enabled === true;
|
||||
const larkAppId = document.getElementById('robot-lark-app-id');
|
||||
if (larkAppId) larkAppId.value = lark.app_id || '';
|
||||
const larkAppSecret = document.getElementById('robot-lark-app-secret');
|
||||
if (larkAppSecret) larkAppSecret.value = lark.app_secret || '';
|
||||
const larkVerify = document.getElementById('robot-lark-verify-token');
|
||||
if (larkVerify) larkVerify.value = lark.verify_token || '';
|
||||
|
||||
// 只有在需要时才加载工具列表(MCP管理页面需要,系统设置页面不需要)
|
||||
if (loadTools) {
|
||||
@@ -684,19 +765,55 @@ async function applySettings() {
|
||||
const val = parseFloat(document.getElementById('knowledge-retrieval-hybrid-weight')?.value);
|
||||
return isNaN(val) ? 0.7 : val; // 允许0.0值,只有NaN时才使用默认值
|
||||
})()
|
||||
},
|
||||
indexing: {
|
||||
chunk_size: parseInt(document.getElementById("knowledge-indexing-chunk-size")?.value) || 512,
|
||||
chunk_overlap: parseInt(document.getElementById("knowledge-indexing-chunk-overlap")?.value) ?? 50,
|
||||
max_chunks_per_item: parseInt(document.getElementById("knowledge-indexing-max-chunks-per-item")?.value) ?? 0,
|
||||
max_rpm: parseInt(document.getElementById("knowledge-indexing-max-rpm")?.value) ?? 0,
|
||||
rate_limit_delay_ms: parseInt(document.getElementById("knowledge-indexing-rate-limit-delay-ms")?.value) ?? 300,
|
||||
max_retries: parseInt(document.getElementById("knowledge-indexing-max-retries")?.value) ?? 3,
|
||||
retry_delay_ms: parseInt(document.getElementById("knowledge-indexing-retry-delay-ms")?.value) ?? 1000
|
||||
}
|
||||
};
|
||||
|
||||
const wecomAgentIdVal = document.getElementById('robot-wecom-agent-id')?.value.trim();
|
||||
const config = {
|
||||
openai: {
|
||||
api_key: apiKey,
|
||||
base_url: baseUrl,
|
||||
model: model
|
||||
},
|
||||
fofa: {
|
||||
email: document.getElementById('fofa-email')?.value.trim() || '',
|
||||
api_key: document.getElementById('fofa-api-key')?.value.trim() || '',
|
||||
base_url: document.getElementById('fofa-base-url')?.value.trim() || ''
|
||||
},
|
||||
agent: {
|
||||
max_iterations: parseInt(document.getElementById('agent-max-iterations').value) || 30
|
||||
},
|
||||
knowledge: knowledgeConfig,
|
||||
robots: {
|
||||
wecom: {
|
||||
enabled: document.getElementById('robot-wecom-enabled')?.checked === true,
|
||||
token: document.getElementById('robot-wecom-token')?.value.trim() || '',
|
||||
encoding_aes_key: document.getElementById('robot-wecom-encoding-aes-key')?.value.trim() || '',
|
||||
corp_id: document.getElementById('robot-wecom-corp-id')?.value.trim() || '',
|
||||
secret: document.getElementById('robot-wecom-secret')?.value.trim() || '',
|
||||
agent_id: parseInt(wecomAgentIdVal, 10) || 0
|
||||
},
|
||||
dingtalk: {
|
||||
enabled: document.getElementById('robot-dingtalk-enabled')?.checked === true,
|
||||
client_id: document.getElementById('robot-dingtalk-client-id')?.value.trim() || '',
|
||||
client_secret: document.getElementById('robot-dingtalk-client-secret')?.value.trim() || ''
|
||||
},
|
||||
lark: {
|
||||
enabled: document.getElementById('robot-lark-enabled')?.checked === true,
|
||||
app_id: document.getElementById('robot-lark-app-id')?.value.trim() || '',
|
||||
app_secret: document.getElementById('robot-lark-app-secret')?.value.trim() || '',
|
||||
verify_token: document.getElementById('robot-lark-verify-token')?.value.trim() || ''
|
||||
}
|
||||
},
|
||||
tools: []
|
||||
};
|
||||
|
||||
|
||||
@@ -1197,7 +1197,8 @@ async function showBatchQueueDetail(queueId) {
|
||||
batchQueuesState.currentQueueId = queueId;
|
||||
|
||||
if (title) {
|
||||
title.textContent = queue.title ? `批量任务队列 - ${escapeHtml(queue.title)}` : '批量任务队列';
|
||||
// textContent 本身会做转义;这里不要再 escapeHtml,否则会把 && 显示成 &...(看起来像“变形/乱码”)
|
||||
title.textContent = queue.title ? `批量任务队列 - ${String(queue.title)}` : '批量任务队列';
|
||||
}
|
||||
|
||||
// 更新按钮显示
|
||||
|
||||
@@ -0,0 +1,419 @@
|
||||
/**
|
||||
* 系统设置 - 终端:多标签、流式输出、命令历史、Ctrl+L 清屏、长时间可取消
|
||||
*/
|
||||
(function () {
|
||||
var getContext = HTMLCanvasElement.prototype.getContext;
|
||||
HTMLCanvasElement.prototype.getContext = function (type, attrs) {
|
||||
if (type === '2d') {
|
||||
attrs = (attrs && typeof attrs === 'object') ? Object.assign({ willReadFrequently: true }, attrs) : { willReadFrequently: true };
|
||||
return getContext.call(this, type, attrs);
|
||||
}
|
||||
return getContext.apply(this, arguments);
|
||||
};
|
||||
|
||||
var terminals = [];
|
||||
var currentTabId = 1;
|
||||
var inited = false;
|
||||
var tabIdCounter = 1;
|
||||
var PROMPT = ''; // 真实 Shell 自己输出提示符,这里不再自定义
|
||||
var HISTORY_MAX = 100;
|
||||
var CANCEL_AFTER_MS = 125000;
|
||||
|
||||
function getCurrent() {
|
||||
for (var i = 0; i < terminals.length; i++) {
|
||||
if (terminals[i].id === currentTabId) return terminals[i];
|
||||
}
|
||||
return terminals[0] || null;
|
||||
}
|
||||
|
||||
var WELCOME_LINE = 'CyberStrikeAI 终端 - 真实 Shell 会话,直接输入命令;Ctrl+L 清屏\r\n';
|
||||
|
||||
function writePrompt(tab) {
|
||||
// 提示符交由后端 Shell 自行输出,这里仅保留占位函数,避免旧代码报错
|
||||
}
|
||||
|
||||
function redrawTabDisplay(t) {
|
||||
if (!t || !t.term) return;
|
||||
t.term.clear();
|
||||
t.term.write(WELCOME_LINE);
|
||||
}
|
||||
|
||||
function writeln(tabOrS, s) {
|
||||
var t, text;
|
||||
if (arguments.length === 1) { text = tabOrS; t = getCurrent(); } else { t = tabOrS; text = s; }
|
||||
if (!t || !t.term) return;
|
||||
if (text) t.term.writeln(text);
|
||||
else t.term.writeln('');
|
||||
}
|
||||
|
||||
function writeOutput(tab, text, isError) {
|
||||
var t = tab || getCurrent();
|
||||
if (!t || !t.term || !text) return;
|
||||
var s = String(text).replace(/\r\n/g, '\n').replace(/\r/g, '\n');
|
||||
var lines = s.split('\n');
|
||||
var prefix = isError ? '\x1b[31m' : '';
|
||||
var suffix = isError ? '\x1b[0m' : '';
|
||||
t.term.write(prefix);
|
||||
for (var i = 0; i < lines.length; i++) {
|
||||
var line = lines[i].replace(/\r/g, '');
|
||||
t.term.writeln(line);
|
||||
}
|
||||
t.term.write(suffix);
|
||||
}
|
||||
|
||||
// 从本地存储中获取当前登录 token(与 auth.js 使用的结构保持一致)
|
||||
function getStoredAuthToken() {
|
||||
try {
|
||||
var raw = localStorage.getItem('cyberstrike-auth');
|
||||
if (!raw) return null;
|
||||
var o = JSON.parse(raw);
|
||||
if (o && o.token) return o.token;
|
||||
} catch (e) {}
|
||||
return null;
|
||||
}
|
||||
|
||||
// WebSocket 地址构造(兼容 http/https,并通过 query 传递 token 以通过后端鉴权)
|
||||
function buildTerminalWSURL() {
|
||||
var proto = (window.location.protocol === 'https:') ? 'wss://' : 'ws://';
|
||||
var url = proto + window.location.host + '/api/terminal/ws';
|
||||
var token = getStoredAuthToken();
|
||||
if (token) {
|
||||
url += '?token=' + encodeURIComponent(token);
|
||||
}
|
||||
return url;
|
||||
}
|
||||
|
||||
function ensureTerminalWS(tab) {
|
||||
if (tab.ws && (tab.ws.readyState === WebSocket.OPEN || tab.ws.readyState === WebSocket.CONNECTING)) {
|
||||
return;
|
||||
}
|
||||
try {
|
||||
var ws = new WebSocket(buildTerminalWSURL());
|
||||
tab.ws = ws;
|
||||
tab.running = true;
|
||||
|
||||
ws.onopen = function () {
|
||||
if (tab.term) {
|
||||
tab.term.focus();
|
||||
}
|
||||
};
|
||||
|
||||
ws.onmessage = function (ev) {
|
||||
if (!tab.term) return;
|
||||
// 处理二进制消息和文本消息
|
||||
if (ev.data instanceof ArrayBuffer) {
|
||||
var decoder = new TextDecoder('utf-8');
|
||||
tab.term.write(decoder.decode(ev.data));
|
||||
} else if (ev.data instanceof Blob) {
|
||||
// Blob 类型,需要异步读取
|
||||
var reader = new FileReader();
|
||||
reader.onload = function () {
|
||||
var decoder = new TextDecoder('utf-8');
|
||||
tab.term.write(decoder.decode(reader.result));
|
||||
};
|
||||
reader.readAsArrayBuffer(ev.data);
|
||||
} else {
|
||||
// 字符串类型
|
||||
tab.term.write(ev.data);
|
||||
}
|
||||
};
|
||||
|
||||
ws.onclose = function () {
|
||||
tab.running = false;
|
||||
if (tab.term) {
|
||||
tab.term.writeln('\r\n\x1b[2m[会话已关闭]\x1b[0m');
|
||||
}
|
||||
};
|
||||
|
||||
ws.onerror = function () {
|
||||
tab.running = false;
|
||||
if (tab.term) {
|
||||
tab.term.writeln('\r\n\x1b[31m[终端连接出错]\x1b[0m');
|
||||
}
|
||||
};
|
||||
} catch (e) {
|
||||
if (tab.term) {
|
||||
tab.term.writeln('\r\n\x1b[31m[无法连接终端服务: ' + String(e) + ']\x1b[0m');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function createTerminalInContainer(container, tab) {
|
||||
if (typeof Terminal === 'undefined') return null;
|
||||
if (!tab.history) tab.history = [];
|
||||
if (tab.historyIndex === undefined) tab.historyIndex = -1;
|
||||
if (tab.cursorIndex === undefined) tab.cursorIndex = 0;
|
||||
|
||||
var term = new Terminal({
|
||||
cursorBlink: true,
|
||||
cursorStyle: 'bar',
|
||||
fontSize: 13,
|
||||
fontFamily: 'Menlo, Monaco, "Courier New", monospace',
|
||||
lineHeight: 1.2,
|
||||
scrollback: 1000,
|
||||
theme: {
|
||||
background: '#0d1117',
|
||||
foreground: '#e6edf3',
|
||||
cursor: '#58a6ff',
|
||||
cursorAccent: '#0d1117',
|
||||
selection: 'rgba(88, 166, 255, 0.3)',
|
||||
black: '#484f58',
|
||||
red: '#ff7b72',
|
||||
green: '#3fb950',
|
||||
yellow: '#d29922',
|
||||
blue: '#58a6ff',
|
||||
magenta: '#bc8cff',
|
||||
cyan: '#39c5cf',
|
||||
white: '#e6edf3',
|
||||
brightBlack: '#6e7681',
|
||||
brightRed: '#ffa198',
|
||||
brightGreen: '#56d364',
|
||||
brightYellow: '#e3b341',
|
||||
brightBlue: '#79c0ff',
|
||||
brightMagenta: '#d2a8ff',
|
||||
brightCyan: '#56d4dd',
|
||||
brightWhite: '#f0f6fc'
|
||||
}
|
||||
});
|
||||
var fitAddon = null;
|
||||
if (typeof FitAddon !== 'undefined') {
|
||||
var FitCtor = (FitAddon.FitAddon || FitAddon);
|
||||
fitAddon = new FitCtor();
|
||||
term.loadAddon(fitAddon);
|
||||
}
|
||||
term.open(container);
|
||||
term.write(WELCOME_LINE);
|
||||
container.addEventListener('click', function () {
|
||||
switchTerminalTab(tab.id);
|
||||
if (term) term.focus();
|
||||
});
|
||||
container.setAttribute('tabindex', '0');
|
||||
container.title = '点击此处后输入命令';
|
||||
|
||||
function sendToWS(data) {
|
||||
ensureTerminalWS(tab);
|
||||
if (tab.ws && tab.ws.readyState === WebSocket.OPEN) {
|
||||
try {
|
||||
tab.ws.send(data);
|
||||
} catch (e) {}
|
||||
}
|
||||
}
|
||||
|
||||
term.onData(function (data) {
|
||||
// Ctrl+L:本地清屏,同时把 ^L 也发给后端
|
||||
if (data === '\x0c') {
|
||||
term.clear();
|
||||
sendToWS(data);
|
||||
return;
|
||||
}
|
||||
sendToWS(data);
|
||||
});
|
||||
|
||||
tab.term = term;
|
||||
tab.fitAddon = fitAddon;
|
||||
return term;
|
||||
}
|
||||
|
||||
function switchTerminalTab(id) {
|
||||
var prevId = currentTabId;
|
||||
currentTabId = id;
|
||||
document.querySelectorAll('.terminal-tab').forEach(function (el) {
|
||||
el.classList.toggle('active', parseInt(el.getAttribute('data-tab-id'), 10) === id);
|
||||
});
|
||||
document.querySelectorAll('.terminal-pane').forEach(function (el) {
|
||||
var paneId = el.getAttribute('id');
|
||||
var match = paneId && paneId.match(/terminal-pane-(\d+)/);
|
||||
var paneTabId = match ? parseInt(match[1], 10) : 0;
|
||||
el.classList.toggle('active', paneTabId === id);
|
||||
});
|
||||
var t = getCurrent();
|
||||
if (t && t.term) {
|
||||
if (prevId !== id) {
|
||||
requestAnimationFrame(function () {
|
||||
if (currentTabId === id && t.term) t.term.focus();
|
||||
});
|
||||
} else {
|
||||
t.term.focus();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function addTerminalTab() {
|
||||
if (typeof Terminal === 'undefined') return;
|
||||
tabIdCounter += 1;
|
||||
var id = tabIdCounter;
|
||||
var paneId = 'terminal-pane-' + id;
|
||||
var containerId = 'terminal-container-' + id;
|
||||
var tabsEl = document.querySelector('.terminal-tabs');
|
||||
var panesEl = document.querySelector('.terminal-panes');
|
||||
if (!tabsEl || !panesEl) return;
|
||||
|
||||
var tabDiv = document.createElement('div');
|
||||
tabDiv.className = 'terminal-tab';
|
||||
tabDiv.setAttribute('data-tab-id', String(id));
|
||||
var label = document.createElement('span');
|
||||
label.className = 'terminal-tab-label';
|
||||
label.textContent = '终端 ' + id;
|
||||
label.onclick = function () { switchTerminalTab(id); };
|
||||
var closeBtn = document.createElement('button');
|
||||
closeBtn.type = 'button';
|
||||
closeBtn.className = 'terminal-tab-close';
|
||||
closeBtn.title = '关闭';
|
||||
closeBtn.textContent = '×';
|
||||
closeBtn.onclick = function (e) { e.stopPropagation(); removeTerminalTab(id); };
|
||||
tabDiv.appendChild(label);
|
||||
tabDiv.appendChild(closeBtn);
|
||||
var plusBtn = tabsEl.querySelector('.terminal-tab-new');
|
||||
tabsEl.insertBefore(tabDiv, plusBtn);
|
||||
|
||||
var paneDiv = document.createElement('div');
|
||||
paneDiv.id = paneId;
|
||||
paneDiv.className = 'terminal-pane';
|
||||
var containerDiv = document.createElement('div');
|
||||
containerDiv.id = containerId;
|
||||
containerDiv.className = 'terminal-container';
|
||||
paneDiv.appendChild(containerDiv);
|
||||
panesEl.appendChild(paneDiv);
|
||||
|
||||
var tab = { id: id, paneId: paneId, containerId: containerId, lineBuffer: '', cursorIndex: 0, running: false, term: null, fitAddon: null, history: [], historyIndex: -1 };
|
||||
terminals.push(tab);
|
||||
createTerminalInContainer(containerDiv, tab);
|
||||
switchTerminalTab(id);
|
||||
updateTerminalTabCloseVisibility();
|
||||
setTimeout(function () {
|
||||
try { if (tab.fitAddon) tab.fitAddon.fit(); if (tab.term) tab.term.focus(); } catch (e) {}
|
||||
}, 50);
|
||||
}
|
||||
|
||||
function updateTerminalTabCloseVisibility() {
|
||||
var tabsEl = document.querySelector('.terminal-tabs');
|
||||
if (!tabsEl) return;
|
||||
var tabDivs = tabsEl.querySelectorAll('.terminal-tab');
|
||||
var showClose = terminals.length > 1;
|
||||
for (var i = 0; i < tabDivs.length; i++) {
|
||||
var btn = tabDivs[i].querySelector('.terminal-tab-close');
|
||||
if (btn) btn.style.display = showClose ? '' : 'none';
|
||||
}
|
||||
}
|
||||
|
||||
function removeTerminalTab(id) {
|
||||
if (terminals.length <= 1) return;
|
||||
var idx = -1;
|
||||
for (var i = 0; i < terminals.length; i++) { if (terminals[i].id === id) { idx = i; break; } }
|
||||
if (idx < 0) return;
|
||||
|
||||
var deletingCurrent = (currentTabId === id);
|
||||
var switchToIndex = deletingCurrent ? (idx > 0 ? idx - 1 : 0) : -1;
|
||||
|
||||
var tab = terminals[idx];
|
||||
if (tab.term && tab.term.dispose) tab.term.dispose();
|
||||
tab.term = null;
|
||||
tab.fitAddon = null;
|
||||
terminals.splice(idx, 1);
|
||||
|
||||
var tabDiv = document.querySelector('.terminal-tab[data-tab-id="' + id + '"]');
|
||||
var paneDiv = document.getElementById('terminal-pane-' + id);
|
||||
if (tabDiv && tabDiv.parentNode) tabDiv.parentNode.removeChild(tabDiv);
|
||||
if (paneDiv && paneDiv.parentNode) paneDiv.parentNode.removeChild(paneDiv);
|
||||
|
||||
var curIdxBeforeRenumber = -1;
|
||||
if (!deletingCurrent) {
|
||||
for (var i = 0; i < terminals.length; i++) {
|
||||
if (terminals[i].id === currentTabId) { curIdxBeforeRenumber = i; break; }
|
||||
}
|
||||
}
|
||||
|
||||
for (var i = 0; i < terminals.length; i++) {
|
||||
var t = terminals[i];
|
||||
t.id = i + 1;
|
||||
t.paneId = 'terminal-pane-' + (i + 1);
|
||||
t.containerId = 'terminal-container-' + (i + 1);
|
||||
}
|
||||
tabIdCounter = terminals.length;
|
||||
if (curIdxBeforeRenumber >= 0) currentTabId = terminals[curIdxBeforeRenumber].id;
|
||||
|
||||
var tabsEl = document.querySelector('.terminal-tabs');
|
||||
var panesEl = document.querySelector('.terminal-panes');
|
||||
if (tabsEl) {
|
||||
var tabDivs = tabsEl.querySelectorAll('.terminal-tab');
|
||||
for (var i = 0; i < tabDivs.length; i++) {
|
||||
var t = terminals[i];
|
||||
tabDivs[i].setAttribute('data-tab-id', String(t.id));
|
||||
var lbl = tabDivs[i].querySelector('.terminal-tab-label');
|
||||
if (lbl) lbl.textContent = '终端 ' + t.id;
|
||||
if (lbl) lbl.onclick = (function (tid) { return function () { switchTerminalTab(tid); }; })(t.id);
|
||||
var cb = tabDivs[i].querySelector('.terminal-tab-close');
|
||||
if (cb) cb.onclick = (function (tid) { return function (e) { e.stopPropagation(); removeTerminalTab(tid); }; })(t.id);
|
||||
}
|
||||
}
|
||||
if (panesEl) {
|
||||
var paneDivs = panesEl.querySelectorAll('.terminal-pane');
|
||||
for (var i = 0; i < paneDivs.length; i++) {
|
||||
var t = terminals[i];
|
||||
paneDivs[i].id = t.paneId;
|
||||
var cont = paneDivs[i].querySelector('.terminal-container');
|
||||
if (cont) cont.id = t.containerId;
|
||||
}
|
||||
}
|
||||
|
||||
updateTerminalTabCloseVisibility();
|
||||
|
||||
if (deletingCurrent && terminals.length > 0) {
|
||||
currentTabId = terminals[switchToIndex].id;
|
||||
switchTerminalTab(currentTabId);
|
||||
}
|
||||
}
|
||||
|
||||
function initTerminal() {
|
||||
var pane1 = document.getElementById('terminal-pane-1');
|
||||
var container1 = document.getElementById('terminal-container-1');
|
||||
if (!pane1 || !container1) return;
|
||||
if (inited) {
|
||||
var t = getCurrent();
|
||||
if (t && t.term) t.term.focus();
|
||||
terminals.forEach(function (tab) { try { if (tab.fitAddon) tab.fitAddon.fit(); } catch (e) {} });
|
||||
return;
|
||||
}
|
||||
inited = true;
|
||||
|
||||
if (typeof Terminal === 'undefined') {
|
||||
container1.innerHTML = '<p class="terminal-error">未加载 xterm.js,请刷新页面或检查网络。</p>';
|
||||
return;
|
||||
}
|
||||
|
||||
currentTabId = 1;
|
||||
var tab = { id: 1, paneId: 'terminal-pane-1', containerId: 'terminal-container-1', lineBuffer: '', cursorIndex: 0, running: false, term: null, fitAddon: null, history: [], historyIndex: -1 };
|
||||
terminals.push(tab);
|
||||
createTerminalInContainer(container1, tab);
|
||||
|
||||
updateTerminalTabCloseVisibility();
|
||||
|
||||
setTimeout(function () {
|
||||
try { if (tab.fitAddon) tab.fitAddon.fit(); if (tab.term) tab.term.focus(); } catch (e) {}
|
||||
}, 100);
|
||||
|
||||
var resizeTimer;
|
||||
window.addEventListener('resize', function () {
|
||||
clearTimeout(resizeTimer);
|
||||
resizeTimer = setTimeout(function () {
|
||||
terminals.forEach(function (t) { try { if (t.fitAddon) t.fitAddon.fit(); } catch (e) {} });
|
||||
}, 150);
|
||||
});
|
||||
}
|
||||
|
||||
function terminalClear() {
|
||||
var t = getCurrent();
|
||||
if (!t || !t.term) return;
|
||||
t.term.clear();
|
||||
t.lineBuffer = '';
|
||||
if (t.cursorIndex !== undefined) t.cursorIndex = 0;
|
||||
writePrompt(t);
|
||||
t.term.focus();
|
||||
}
|
||||
|
||||
window.initTerminal = initTerminal;
|
||||
window.terminalClear = terminalClear;
|
||||
window.switchTerminalTab = switchTerminalTab;
|
||||
window.addTerminalTab = addTerminalTab;
|
||||
window.removeTerminalTab = removeTerminalTab;
|
||||
})();
|
||||
+418
-23
@@ -7,11 +7,12 @@
|
||||
<link rel="icon" type="image/png" href="/static/logo.png">
|
||||
<link rel="shortcut icon" type="image/png" href="/static/favicon.ico">
|
||||
<link rel="stylesheet" href="/static/css/style.css">
|
||||
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/xterm@4.19.0/css/xterm.css">
|
||||
</head>
|
||||
<body>
|
||||
<div id="login-overlay" class="login-overlay" style="display: none;">
|
||||
<div class="login-card">
|
||||
<div class="login-header">
|
||||
<div class="login-brand">
|
||||
<h2>登录 CyberStrikeAI</h2>
|
||||
<p class="login-subtitle">请输入配置中的访问密码</p>
|
||||
</div>
|
||||
@@ -21,7 +22,9 @@
|
||||
<input type="password" id="login-password" placeholder="输入登录密码" required autocomplete="current-password" />
|
||||
</div>
|
||||
<div id="login-error" class="login-error" role="alert" style="display: none;"></div>
|
||||
<button type="submit" class="btn-primary login-submit">登录</button>
|
||||
<button type="submit" class="btn-primary login-submit">
|
||||
<span>登录</span>
|
||||
</button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
@@ -90,6 +93,15 @@
|
||||
<span>对话</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="nav-item" data-page="info-collect">
|
||||
<div class="nav-item-content" data-title="信息收集" onclick="switchPage('info-collect')">
|
||||
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
||||
<circle cx="11" cy="11" r="8"></circle>
|
||||
<line x1="21" y1="21" x2="16.65" y2="16.65"></line>
|
||||
</svg>
|
||||
<span>信息收集</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="nav-item" data-page="tasks">
|
||||
<div class="nav-item-content" data-title="任务管理" onclick="switchPage('tasks')">
|
||||
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
||||
@@ -245,28 +257,99 @@
|
||||
<section class="dashboard-section dashboard-section-overview">
|
||||
<h3 class="dashboard-section-title">运行概览</h3>
|
||||
<div class="dashboard-overview-list">
|
||||
<div class="dashboard-overview-item" role="button" tabindex="0" onclick="switchPage('tasks')" onkeydown="if(event.key==='Enter'||event.key===' ') { event.preventDefault(); switchPage('tasks'); }">
|
||||
<span class="dashboard-overview-icon" aria-hidden="true"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><rect x="3" y="3" width="7" height="7"/><rect x="14" y="3" width="7" height="7"/><rect x="14" y="14" width="7" height="7"/><rect x="3" y="14" width="7" height="7"/></svg></span>
|
||||
<div><span class="dashboard-overview-label">批量任务队列</span><span class="dashboard-overview-value"><span id="dashboard-batch-pending">-</span> 待执行 / <span id="dashboard-batch-running">-</span> 执行中 / <span id="dashboard-batch-done">-</span> 已完成</span></div>
|
||||
<div class="dashboard-overview-item dashboard-overview-item-batch" role="button" tabindex="0" onclick="switchPage('tasks')" onkeydown="if(event.key==='Enter'||event.key===' ') { event.preventDefault(); switchPage('tasks'); }">
|
||||
<span class="dashboard-overview-icon dashboard-overview-icon-batch" aria-hidden="true"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><rect x="3" y="3" width="7" height="7"/><rect x="14" y="3" width="7" height="7"/><rect x="14" y="14" width="7" height="7"/><rect x="3" y="14" width="7" height="7"/></svg></span>
|
||||
<div class="dashboard-overview-content">
|
||||
<div class="dashboard-overview-header">
|
||||
<span class="dashboard-overview-label">批量任务队列</span>
|
||||
<span class="dashboard-overview-total" id="dashboard-batch-total">-</span>
|
||||
</div>
|
||||
<div class="dashboard-overview-stats">
|
||||
<span class="dashboard-overview-stat dashboard-overview-stat-pending">
|
||||
<span class="dashboard-overview-stat-badge badge-pending"></span>
|
||||
<span class="dashboard-overview-stat-value" id="dashboard-batch-pending">-</span>
|
||||
<span class="dashboard-overview-stat-label">待执行</span>
|
||||
</span>
|
||||
<span class="dashboard-overview-stat dashboard-overview-stat-running">
|
||||
<span class="dashboard-overview-stat-badge badge-running"></span>
|
||||
<span class="dashboard-overview-stat-value" id="dashboard-batch-running">-</span>
|
||||
<span class="dashboard-overview-stat-label">执行中</span>
|
||||
</span>
|
||||
<span class="dashboard-overview-stat dashboard-overview-stat-done">
|
||||
<span class="dashboard-overview-stat-badge badge-done"></span>
|
||||
<span class="dashboard-overview-stat-value" id="dashboard-batch-done">-</span>
|
||||
<span class="dashboard-overview-stat-label">已完成</span>
|
||||
</span>
|
||||
</div>
|
||||
<div class="dashboard-overview-progress">
|
||||
<div class="dashboard-overview-progress-bar">
|
||||
<div class="dashboard-overview-progress-segment dashboard-overview-progress-pending" id="dashboard-batch-progress-pending" style="width: 0%"></div>
|
||||
<div class="dashboard-overview-progress-segment dashboard-overview-progress-running" id="dashboard-batch-progress-running" style="width: 0%"></div>
|
||||
<div class="dashboard-overview-progress-segment dashboard-overview-progress-done" id="dashboard-batch-progress-done" style="width: 0%"></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="dashboard-overview-item" role="button" tabindex="0" onclick="switchPage('mcp-monitor')" onkeydown="if(event.key==='Enter'||event.key===' ') { event.preventDefault(); switchPage('mcp-monitor'); }">
|
||||
<span class="dashboard-overview-icon" aria-hidden="true"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/></svg></span>
|
||||
<div><span class="dashboard-overview-label">工具调用</span><span class="dashboard-overview-value"><span id="dashboard-tools-count">-</span> 个工具,共 <span id="dashboard-tools-calls">-</span> 次</span></div>
|
||||
<div class="dashboard-overview-item dashboard-overview-item-tools" role="button" tabindex="0" onclick="switchPage('mcp-monitor')" onkeydown="if(event.key==='Enter'||event.key===' ') { event.preventDefault(); switchPage('mcp-monitor'); }">
|
||||
<span class="dashboard-overview-icon dashboard-overview-icon-tools" aria-hidden="true"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/></svg></span>
|
||||
<div class="dashboard-overview-content">
|
||||
<div class="dashboard-overview-header">
|
||||
<span class="dashboard-overview-label">工具调用</span>
|
||||
<span class="dashboard-overview-success-rate" id="dashboard-tools-success-rate">-</span>
|
||||
</div>
|
||||
<div class="dashboard-overview-value-group">
|
||||
<span class="dashboard-overview-value-large" id="dashboard-tools-calls">-</span>
|
||||
<span class="dashboard-overview-value-unit">次调用</span>
|
||||
<span class="dashboard-overview-value-separator">·</span>
|
||||
<span class="dashboard-overview-value-normal" id="dashboard-tools-count">-</span>
|
||||
<span class="dashboard-overview-value-unit">个工具</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="dashboard-overview-item" role="button" tabindex="0" onclick="switchPage('skills-monitor')" onkeydown="if(event.key==='Enter'||event.key===' ') { event.preventDefault(); switchPage('skills-monitor'); }">
|
||||
<span class="dashboard-overview-icon" aria-hidden="true"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/><polyline points="14 2 14 8 20 8"/><line x1="16" y1="13" x2="8" y2="13"/><line x1="16" y1="17" x2="8" y2="17"/></svg></span>
|
||||
<div><span class="dashboard-overview-label">Skills</span><span class="dashboard-overview-value"><span id="dashboard-skills-count">-</span> 个 Skill,共 <span id="dashboard-skills-calls">-</span> 次调用</span></div>
|
||||
<div class="dashboard-overview-item dashboard-overview-item-knowledge" role="button" tabindex="0" onclick="switchPage('knowledge-management')" onkeydown="if(event.key==='Enter'||event.key===' ') { event.preventDefault(); switchPage('knowledge-management'); }">
|
||||
<span class="dashboard-overview-icon dashboard-overview-icon-knowledge" aria-hidden="true"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M4 19.5A2.5 2.5 0 0 1 6.5 17H20"></path><path d="M6.5 2H20v20H6.5A2.5 2.5 0 0 1 4 19.5v-15A2.5 2.5 0 0 1 6.5 2z"></path></svg></span>
|
||||
<div class="dashboard-overview-content">
|
||||
<div class="dashboard-overview-header">
|
||||
<span class="dashboard-overview-label">知识</span>
|
||||
<span class="dashboard-overview-status" id="dashboard-knowledge-status">-</span>
|
||||
</div>
|
||||
<div class="dashboard-overview-value-group">
|
||||
<span class="dashboard-overview-value-large" id="dashboard-knowledge-items">-</span>
|
||||
<span class="dashboard-overview-value-unit">项知识</span>
|
||||
<span class="dashboard-overview-value-separator">·</span>
|
||||
<span class="dashboard-overview-value-normal" id="dashboard-knowledge-categories">-</span>
|
||||
<span class="dashboard-overview-value-unit">个分类</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="dashboard-overview-item dashboard-overview-item-skills" role="button" tabindex="0" onclick="switchPage('skills-monitor')" onkeydown="if(event.key==='Enter'||event.key===' ') { event.preventDefault(); switchPage('skills-monitor'); }">
|
||||
<span class="dashboard-overview-icon dashboard-overview-icon-skills" aria-hidden="true"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/><polyline points="14 2 14 8 20 8"/><line x1="16" y1="13" x2="8" y2="13"/><line x1="16" y1="17" x2="8" y2="17"/></svg></span>
|
||||
<div class="dashboard-overview-content">
|
||||
<div class="dashboard-overview-header">
|
||||
<span class="dashboard-overview-label">Skills</span>
|
||||
<span class="dashboard-overview-status" id="dashboard-skills-status">-</span>
|
||||
</div>
|
||||
<div class="dashboard-overview-value-group">
|
||||
<span class="dashboard-overview-value-large" id="dashboard-skills-calls">-</span>
|
||||
<span class="dashboard-overview-value-unit">次调用</span>
|
||||
<span class="dashboard-overview-value-separator">·</span>
|
||||
<span class="dashboard-overview-value-normal" id="dashboard-skills-count">-</span>
|
||||
<span class="dashboard-overview-value-unit">个 Skill</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<section class="dashboard-section dashboard-section-quick dashboard-quick-inline">
|
||||
<h3 class="dashboard-section-title">快捷入口</h3>
|
||||
<div class="dashboard-quick-links dashboard-quick-links-row">
|
||||
<a class="dashboard-quick-link" onclick="switchPage('chat')"><span class="dashboard-quick-icon"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"></path></svg></span><span>对话</span></a>
|
||||
<a class="dashboard-quick-link" onclick="switchPage('tasks')"><span class="dashboard-quick-icon"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M9 11l3 3L22 4"></path><path d="M21 12v7a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h11"></path></svg></span><span>任务管理</span></a>
|
||||
<a class="dashboard-quick-link" onclick="switchPage('vulnerabilities')"><span class="dashboard-quick-icon"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M12 22s8-4 8-10V5l-8-3-8 3v7c0 6 8 10 8 10z"></path></svg></span><span>漏洞管理</span></a>
|
||||
<a class="dashboard-quick-link" onclick="switchPage('chat')"><span class="dashboard-quick-icon"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"></path></svg></span><span>对话</span></a>
|
||||
<a class="dashboard-quick-link" onclick="switchPage('mcp-monitor')"><span class="dashboard-quick-icon"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M13 2L3 14h9l-1 8 10-12h-9l1-8z"></path></svg></span><span>MCP 监控</span></a>
|
||||
<a class="dashboard-quick-link" onclick="switchPage('skills-monitor')"><span class="dashboard-quick-icon"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M14.5 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7.5L14.5 2z"></path><polyline points="14 2 14 8 20 8"></polyline></svg></span><span>Skills 监控</span></a>
|
||||
<a class="dashboard-quick-link" onclick="switchPage('mcp-management')"><span class="dashboard-quick-icon"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M13 2L3 14h9l-1 8 10-12h-9l1-8z"></path></svg></span><span>MCP 管理</span></a>
|
||||
<a class="dashboard-quick-link" onclick="switchPage('knowledge-management')"><span class="dashboard-quick-icon"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M2 3h6a4 4 0 0 1 4 4v14a3 3 0 0 0-3-3H2z"></path><path d="M22 3h-6a4 4 0 0 0-4 4v14a3 3 0 0 1 3-3h7z"></path></svg></span><span>知识管理</span></a>
|
||||
<a class="dashboard-quick-link" onclick="switchPage('skills-management')"><span class="dashboard-quick-icon"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M14.5 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7.5L14.5 2z"></path><polyline points="14 2 14 8 20 8"></polyline></svg></span><span>Skills 管理</span></a>
|
||||
<a class="dashboard-quick-link" onclick="switchPage('roles-management')"><span class="dashboard-quick-icon"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M17 21v-2a4 4 0 0 0-4-4H5a4 4 0 0 0-4 4v2"></path><circle cx="9" cy="7" r="4"></circle><path d="M23 21v-2a4 4 0 0 0-3-3.87"></path><path d="M16 3.13a4 4 0 0 1 0 7.75"></path></svg></span><span>角色管理</span></a>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
@@ -286,7 +369,7 @@
|
||||
<svg width="28" height="28" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"></path></svg>
|
||||
</div>
|
||||
<div class="dashboard-cta-copy">
|
||||
<p class="dashboard-cta-text">准备好开始安全测试?</p>
|
||||
<p class="dashboard-cta-text">开始你的安全之旅</p>
|
||||
<p class="dashboard-cta-sub">在对话中描述目标,AI 将协助执行扫描与漏洞分析</p>
|
||||
</div>
|
||||
</div>
|
||||
@@ -417,7 +500,7 @@
|
||||
</div>
|
||||
<div id="active-tasks-bar" class="active-tasks-bar"></div>
|
||||
<div id="chat-messages" class="chat-messages"></div>
|
||||
<div class="chat-input-container">
|
||||
<div id="chat-input-container" class="chat-input-container">
|
||||
<div class="role-selector-wrapper">
|
||||
<button id="role-selector-btn" class="role-selector-btn" onclick="toggleRoleSelectionPanel()" title="选择角色">
|
||||
<span id="role-selector-icon" class="role-selector-icon">🔵</span>
|
||||
@@ -439,10 +522,19 @@
|
||||
<div id="role-selection-list" class="role-selection-list-main"></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="chat-input-field">
|
||||
<textarea id="chat-input" placeholder="输入测试目标或命令... (输入 @ 选择工具 | Shift+Enter 换行,Enter 发送)" rows="1"></textarea>
|
||||
<div id="mention-suggestions" class="mention-suggestions" role="listbox" aria-label="工具提及候选"></div>
|
||||
<div class="chat-input-with-files">
|
||||
<div id="chat-file-list" class="chat-file-list" aria-label="已选文件列表"></div>
|
||||
<div class="chat-input-field">
|
||||
<textarea id="chat-input" placeholder="输入测试目标或命令... (输入 @ 选择工具 | Shift+Enter 换行,Enter 发送)" rows="1"></textarea>
|
||||
<div id="mention-suggestions" class="mention-suggestions" role="listbox" aria-label="工具提及候选"></div>
|
||||
</div>
|
||||
</div>
|
||||
<input type="file" id="chat-file-input" class="chat-file-input-hidden" multiple accept="*" title="选择文件">
|
||||
<button type="button" class="chat-upload-btn" onclick="document.getElementById('chat-file-input').click()" title="上传文件(可多选或拖拽到此处)">
|
||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4M17 8l-5-5-5 5M12 3v12" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
</svg>
|
||||
</button>
|
||||
<button class="send-btn" onclick="sendMessage()">
|
||||
<span>发送</span>
|
||||
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
@@ -651,6 +743,106 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 信息收集页面 -->
|
||||
<div id="page-info-collect" class="page">
|
||||
<div class="page-header">
|
||||
<h2>信息收集</h2>
|
||||
<div class="page-header-actions">
|
||||
<button class="btn-secondary" onclick="resetFofaForm()">重置</button>
|
||||
<button class="btn-primary" onclick="submitFofaSearch()">确定</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="page-content">
|
||||
<div class="info-collect-panel">
|
||||
<div class="info-collect-form">
|
||||
<div class="form-group">
|
||||
<label for="fofa-query">FOFA 查询语法</label>
|
||||
<textarea id="fofa-query" class="info-collect-query-input" rows="1" placeholder='例如:app="Apache" && country="CN"'></textarea>
|
||||
<small class="form-hint">查询语法参考 FOFA 文档,支持 && / || / () 等。</small>
|
||||
<div class="info-collect-presets" aria-label="FOFA 查询示例">
|
||||
<button class="preset-chip" type="button" onclick="applyFofaQueryPreset('app="Apache" && country="CN"')" title="填入示例">Apache + 中国</button>
|
||||
<button class="preset-chip" type="button" onclick="applyFofaQueryPreset('title="登录" && country="CN"')" title="填入示例">登录页 + 中国</button>
|
||||
<button class="preset-chip" type="button" onclick="applyFofaQueryPreset('domain="example.com"')" title="填入示例">指定域名</button>
|
||||
<button class="preset-chip" type="button" onclick="applyFofaQueryPreset('ip="1.1.1.1"')" title="填入示例">指定 IP</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="fofa-nl">自然语言(AI 解析为 FOFA 语法)</label>
|
||||
<div class="info-collect-nl-row">
|
||||
<textarea id="fofa-nl" class="info-collect-query-input" rows="1" placeholder="例如:找美国 Missouri 的 Apache 站点,标题包含 Home"></textarea>
|
||||
<button id="fofa-nl-parse-btn" class="btn-secondary" type="button" onclick="parseFofaNaturalLanguage()" title="将自然语言解析为 FOFA 查询语法">AI 解析</button>
|
||||
</div>
|
||||
<div id="fofa-nl-status" class="fofa-nl-status muted" style="display: none;" aria-live="polite"></div>
|
||||
<small class="form-hint">解析后会弹窗展示 FOFA 语法(可编辑),确认无误后再填入查询框并执行查询。</small>
|
||||
</div>
|
||||
<div class="info-collect-form-row">
|
||||
<div class="form-group">
|
||||
<label for="fofa-size">返回数量</label>
|
||||
<input type="number" id="fofa-size" min="1" max="10000" value="100" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="fofa-page">页码</label>
|
||||
<input type="number" id="fofa-page" min="1" value="1" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label class="checkbox-label" style="margin-top: 24px;">
|
||||
<input type="checkbox" id="fofa-full" class="modern-checkbox" />
|
||||
<span class="checkbox-custom"></span>
|
||||
<span class="checkbox-text">full</span>
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="fofa-fields">返回字段名(逗号分隔)</label>
|
||||
<input type="text" id="fofa-fields" value="host,ip,port,domain,title,protocol,country,province,city,server" />
|
||||
<div class="info-collect-presets" aria-label="FOFA 字段模板">
|
||||
<button class="preset-chip" type="button" onclick="applyFofaFieldsPreset('host,ip,port,domain')" title="适合快速导出目标">最小字段</button>
|
||||
<button class="preset-chip" type="button" onclick="applyFofaFieldsPreset('host,title,ip,port,domain,protocol,server,icp,country,province,city')" title="适合浏览和筛选">Web 常用</button>
|
||||
<button class="preset-chip" type="button" onclick="applyFofaFieldsPreset('host,ip,port,domain,title,protocol,country,province,city,server,as_number,as_organization,icp,header,banner')" title="更偏指纹/情报">情报增强</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="info-collect-results">
|
||||
<div class="info-collect-results-header">
|
||||
<div class="info-collect-results-header-left">
|
||||
<div class="info-collect-results-title">查询结果</div>
|
||||
<div class="info-collect-results-meta" id="fofa-results-meta">-</div>
|
||||
</div>
|
||||
<div class="info-collect-results-toolbar" aria-label="结果工具条">
|
||||
<div class="info-collect-selected" id="fofa-selected-meta">已选择 0 条</div>
|
||||
<button class="btn-secondary btn-small" type="button" onclick="toggleFofaColumnsPanel()" title="显示/隐藏字段">列</button>
|
||||
<button class="btn-secondary btn-small" type="button" onclick="exportFofaResults('csv')" title="导出当前结果为 CSV(UTF-8,兼容中文)">导出 CSV</button>
|
||||
<button class="btn-secondary btn-small" type="button" onclick="exportFofaResults('json')" title="导出当前结果为 JSON">导出 JSON</button>
|
||||
<button class="btn-secondary btn-small" type="button" onclick="exportFofaResults('xlsx')" title="导出当前结果为 Excel">导出 XLSX</button>
|
||||
<button class="btn-primary btn-small" type="button" onclick="batchScanSelectedFofaRows()" title="将所选行创建为批量任务队列">批量扫描</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="info-collect-results-table-wrap">
|
||||
<!-- 字段显示/隐藏面板 -->
|
||||
<div id="fofa-columns-panel" class="info-collect-columns-panel" style="display: none;">
|
||||
<div class="info-collect-columns-panel-header">
|
||||
<div class="info-collect-columns-title">显示字段</div>
|
||||
<div class="info-collect-columns-actions">
|
||||
<button class="btn-secondary btn-small" type="button" onclick="showAllFofaColumns()">全选</button>
|
||||
<button class="btn-secondary btn-small" type="button" onclick="hideAllFofaColumns()">全不选</button>
|
||||
<button class="btn-secondary btn-small" type="button" onclick="closeFofaColumnsPanel()">关闭</button>
|
||||
</div>
|
||||
</div>
|
||||
<div id="fofa-columns-list" class="info-collect-columns-list"></div>
|
||||
</div>
|
||||
<table class="info-collect-table">
|
||||
<thead id="fofa-results-thead"></thead>
|
||||
<tbody id="fofa-results-tbody">
|
||||
<tr><td class="muted" style="padding: 16px;">暂无数据</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 漏洞管理页面 -->
|
||||
<div id="page-vulnerabilities" class="page">
|
||||
<div class="page-header">
|
||||
@@ -880,6 +1072,12 @@
|
||||
<div class="settings-nav-item active" data-section="basic" onclick="switchSettingsSection('basic')">
|
||||
<span>基本设置</span>
|
||||
</div>
|
||||
<div class="settings-nav-item" data-section="robots" onclick="switchSettingsSection('robots')">
|
||||
<span>机器人设置</span>
|
||||
</div>
|
||||
<div class="settings-nav-item" data-section="terminal" onclick="switchSettingsSection('terminal')">
|
||||
<span>终端</span>
|
||||
</div>
|
||||
<div class="settings-nav-item" data-section="security" onclick="switchSettingsSection('security')">
|
||||
<span>安全设置</span>
|
||||
</div>
|
||||
@@ -913,6 +1111,27 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- FOFA配置 -->
|
||||
<div class="settings-subsection">
|
||||
<h4>FOFA 配置</h4>
|
||||
<div class="settings-form">
|
||||
<div class="form-group">
|
||||
<label for="fofa-base-url">Base URL</label>
|
||||
<input type="text" id="fofa-base-url" placeholder="https://fofa.info/api/v1/search/all(可选)" />
|
||||
<small class="form-hint">留空则使用默认地址。</small>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="fofa-email">Email</label>
|
||||
<input type="text" id="fofa-email" placeholder="输入 FOFA 账号邮箱" autocomplete="off" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="fofa-api-key">API Key</label>
|
||||
<input type="password" id="fofa-api-key" placeholder="输入 FOFA API Key" autocomplete="off" />
|
||||
<small class="form-hint">仅保存在服务器配置中(`config.yaml`)。</small>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Agent配置 -->
|
||||
<div class="settings-subsection">
|
||||
<h4>Agent 配置</h4>
|
||||
@@ -984,6 +1203,157 @@
|
||||
<small class="form-hint">向量检索的权重(0-1),1.0表示纯向量检索,0.0表示纯关键词检索</small>
|
||||
</div>
|
||||
</div>
|
||||
<div class="settings-subsection-header">
|
||||
<h5>索引配置</h5>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="knowledge-indexing-chunk-size">分块大小(Chunk Size)</label>
|
||||
<input type="number" id="knowledge-indexing-chunk-size" min="128" max="4096" placeholder="512" />
|
||||
<small class="form-hint">每个块的最大 token 数(默认 512),长文本会被分割成多个块</small>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="knowledge-indexing-chunk-overlap">分块重叠(Chunk Overlap)</label>
|
||||
<input type="number" id="knowledge-indexing-chunk-overlap" min="0" max="512" placeholder="50" />
|
||||
<small class="form-hint">块之间的重叠 token 数(默认 50),保持上下文连贯性</small>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="knowledge-indexing-max-chunks-per-item">单个知识项最大块数</label>
|
||||
<input type="number" id="knowledge-indexing-max-chunks-per-item" min="0" max="1000" placeholder="0" />
|
||||
<small class="form-hint">单个知识项的最大块数量(0 表示不限制),防止单个文件消耗过多 API 配额</small>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="knowledge-indexing-max-rpm">每分钟最大请求数(Max RPM)</label>
|
||||
<input type="number" id="knowledge-indexing-max-rpm" min="0" max="1000" placeholder="0" />
|
||||
<small class="form-hint">每分钟最大请求数(默认 0 表示不限制),如 OpenAI 默认 200 RPM</small>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="knowledge-indexing-rate-limit-delay-ms">请求间隔延迟(毫秒)</label>
|
||||
<input type="number" id="knowledge-indexing-rate-limit-delay-ms" min="0" max="10000" placeholder="300" />
|
||||
<small class="form-hint">请求间隔毫秒数(默认 300),用于避免 API 速率限制,设为 0 不限制</small>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="knowledge-indexing-max-retries">最大重试次数</label>
|
||||
<input type="number" id="knowledge-indexing-max-retries" min="0" max="10" placeholder="3" />
|
||||
<small class="form-hint">最大重试次数(默认 3),遇到速率限制或服务器错误时自动重试</small>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="knowledge-indexing-retry-delay-ms">重试间隔(毫秒)</label>
|
||||
<input type="number" id="knowledge-indexing-retry-delay-ms" min="0" max="10000" placeholder="1000" />
|
||||
<small class="form-hint">重试间隔毫秒数(默认 1000),每次重试会递增延迟</small>
|
||||
</div> </div>
|
||||
|
||||
<div class="settings-actions">
|
||||
<button class="btn-primary" onclick="applySettings()">应用配置</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 机器人设置 -->
|
||||
<div id="settings-section-robots" class="settings-section-content">
|
||||
<div class="settings-section-header">
|
||||
<h3>机器人设置</h3>
|
||||
<p class="settings-description">配置企业微信、钉钉、飞书等机器人,在手机端直接与 CyberStrikeAI 对话,无需在服务器上打开网页。</p>
|
||||
</div>
|
||||
|
||||
<!-- 企业微信 -->
|
||||
<div class="settings-subsection">
|
||||
<h4>企业微信</h4>
|
||||
<div class="settings-form">
|
||||
<div class="form-group">
|
||||
<label class="checkbox-label">
|
||||
<input type="checkbox" id="robot-wecom-enabled" class="modern-checkbox" />
|
||||
<span class="checkbox-custom"></span>
|
||||
<span class="checkbox-text">启用企业微信机器人</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="robot-wecom-token">Token</label>
|
||||
<input type="text" id="robot-wecom-token" placeholder="Token" autocomplete="off" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="robot-wecom-encoding-aes-key">EncodingAESKey</label>
|
||||
<input type="text" id="robot-wecom-encoding-aes-key" placeholder="EncodingAESKey(明文模式可留空)" autocomplete="off" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="robot-wecom-corp-id">CorpID</label>
|
||||
<input type="text" id="robot-wecom-corp-id" placeholder="企业 ID" autocomplete="off" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="robot-wecom-secret">Secret</label>
|
||||
<input type="password" id="robot-wecom-secret" placeholder="应用 Secret" autocomplete="off" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="robot-wecom-agent-id">AgentID</label>
|
||||
<input type="number" id="robot-wecom-agent-id" placeholder="应用 AgentId" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 钉钉 -->
|
||||
<div class="settings-subsection">
|
||||
<h4>钉钉</h4>
|
||||
<div class="settings-form">
|
||||
<div class="form-group">
|
||||
<label class="checkbox-label">
|
||||
<input type="checkbox" id="robot-dingtalk-enabled" class="modern-checkbox" />
|
||||
<span class="checkbox-custom"></span>
|
||||
<span class="checkbox-text">启用钉钉机器人</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="robot-dingtalk-client-id">Client ID (AppKey)</label>
|
||||
<input type="text" id="robot-dingtalk-client-id" placeholder="钉钉应用 AppKey" autocomplete="off" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="robot-dingtalk-client-secret">Client Secret</label>
|
||||
<input type="password" id="robot-dingtalk-client-secret" placeholder="钉钉应用 Secret" autocomplete="off" />
|
||||
<small class="form-hint">需开启机器人能力并配置流式接入</small>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 飞书 -->
|
||||
<div class="settings-subsection">
|
||||
<h4>飞书 (Lark)</h4>
|
||||
<div class="settings-form">
|
||||
<div class="form-group">
|
||||
<label class="checkbox-label">
|
||||
<input type="checkbox" id="robot-lark-enabled" class="modern-checkbox" />
|
||||
<span class="checkbox-custom"></span>
|
||||
<span class="checkbox-text">启用飞书机器人</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="robot-lark-app-id">App ID</label>
|
||||
<input type="text" id="robot-lark-app-id" placeholder="飞书应用 App ID" autocomplete="off" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="robot-lark-app-secret">App Secret</label>
|
||||
<input type="password" id="robot-lark-app-secret" placeholder="飞书应用 App Secret" autocomplete="off" />
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="robot-lark-verify-token">Verify Token(可选)</label>
|
||||
<input type="text" id="robot-lark-verify-token" placeholder="事件订阅 Verification Token" autocomplete="off" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="settings-subsection">
|
||||
<h4>机器人命令说明</h4>
|
||||
<p class="settings-description">在对话中可发送以下命令(支持中英文):</p>
|
||||
<ul style="color: var(--text-muted); font-size: 13px; line-height: 1.8; margin: 8px 0 0 16px;">
|
||||
<li><code>帮助</code> <code>help</code> — 显示本帮助 | Show this help</li>
|
||||
<li><code>列表</code> <code>list</code> — 列出所有对话标题与 ID | List conversations</li>
|
||||
<li><code>切换 <ID></code> <code>switch <ID></code> — 指定对话继续 | Switch to conversation</li>
|
||||
<li><code>新对话</code> <code>new</code> — 开启新对话 | Start new conversation</li>
|
||||
<li><code>清空</code> <code>clear</code> — 清空当前上下文 | Clear context</li>
|
||||
<li><code>当前</code> <code>current</code> — 显示当前对话 ID 与标题 | Show current conversation</li>
|
||||
<li><code>停止</code> <code>stop</code> — 中断当前任务 | Stop running task</li>
|
||||
<li><code>角色</code> <code>roles</code> — 列出所有可用角色 | List roles</li>
|
||||
<li><code>角色 <名></code> <code>role <name></code> — 切换当前角色 | Switch role</li>
|
||||
<li><code>删除 <ID></code> <code>delete <ID></code> — 删除指定对话 | Delete conversation</li>
|
||||
<li><code>版本</code> <code>version</code> — 显示当前版本号 | Show version</li>
|
||||
</ul>
|
||||
<p class="settings-description" style="margin-top: 8px;">除以上命令外,直接输入内容将发送给 AI 进行渗透测试/安全分析。Otherwise, send any text for AI penetration testing / security analysis.</p>
|
||||
</div>
|
||||
|
||||
<div class="settings-actions">
|
||||
@@ -991,6 +1361,25 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 终端 -->
|
||||
<div id="settings-section-terminal" class="settings-section-content">
|
||||
<div class="settings-section-header">
|
||||
<h3>终端</h3>
|
||||
<p class="settings-description">在服务器上执行命令,便于运维与调试。命令在服务端执行,请勿执行敏感或破坏性操作。</p>
|
||||
</div>
|
||||
<div class="terminal-wrapper">
|
||||
<div class="terminal-tabs">
|
||||
<div class="terminal-tab active" data-tab-id="1"><span class="terminal-tab-label" onclick="switchTerminalTab(1)">终端 1</span><button type="button" class="terminal-tab-close" onclick="removeTerminalTab(1); event.stopPropagation();" title="关闭">×</button></div>
|
||||
<button type="button" class="terminal-tab-new" onclick="addTerminalTab()" title="新终端">+</button>
|
||||
</div>
|
||||
<div class="terminal-panes">
|
||||
<div id="terminal-pane-1" class="terminal-pane active">
|
||||
<div id="terminal-container-1" class="terminal-container"></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 安全设置 -->
|
||||
<div id="settings-section-security" class="settings-section-content">
|
||||
<div class="settings-section-header">
|
||||
@@ -1270,6 +1659,8 @@
|
||||
<script src="https://cdn.jsdelivr.net/npm/cytoscape@3.27.0/dist/cytoscape.min.js"></script>
|
||||
<!-- ELK.js for high-quality DAG layout (reduces edge crossings) -->
|
||||
<script src="https://cdn.jsdelivr.net/npm/elkjs@0.9.2/lib/elk.bundled.js"></script>
|
||||
<!-- SheetJS for XLSX export (info-collect) -->
|
||||
<script src="https://cdn.jsdelivr.net/npm/xlsx@0.18.5/dist/xlsx.full.min.js"></script>
|
||||
<script>
|
||||
// 确保ELK对象全局可用
|
||||
if (typeof ELK === 'undefined' && typeof elk !== 'undefined') {
|
||||
@@ -1568,10 +1959,10 @@ version: 1.0.0<br>
|
||||
<h2 id="batch-queue-detail-title">批量任务队列详情</h2>
|
||||
<div style="display: flex; align-items: center; gap: 12px;">
|
||||
<div class="modal-header-actions">
|
||||
<button class="btn-secondary" id="batch-queue-add-task-btn" onclick="showAddBatchTaskModal()" style="display: none;">添加任务</button>
|
||||
<button class="btn-primary" id="batch-queue-start-btn" onclick="startBatchQueue()" style="display: none;">开始执行</button>
|
||||
<button class="btn-secondary" id="batch-queue-pause-btn" onclick="pauseBatchQueue()" style="display: none;">暂停队列</button>
|
||||
<button class="btn-secondary btn-danger" id="batch-queue-delete-btn" onclick="deleteBatchQueue()" style="display: none;">删除队列</button>
|
||||
<button class="btn-secondary btn-small" id="batch-queue-add-task-btn" onclick="showAddBatchTaskModal()" style="display: none;">添加任务</button>
|
||||
<button class="btn-primary btn-small" id="batch-queue-start-btn" onclick="startBatchQueue()" style="display: none;">开始执行</button>
|
||||
<button class="btn-secondary btn-small" id="batch-queue-pause-btn" onclick="pauseBatchQueue()" style="display: none;">暂停队列</button>
|
||||
<button class="btn-secondary btn-small btn-danger" id="batch-queue-delete-btn" onclick="deleteBatchQueue()" style="display: none;">删除队列</button>
|
||||
</div>
|
||||
<span class="modal-close" onclick="closeBatchQueueDetailModal()">×</span>
|
||||
</div>
|
||||
@@ -1807,11 +2198,15 @@ version: 1.0.0<br>
|
||||
|
||||
<script src="/static/js/builtin-tools.js"></script>
|
||||
<script src="/static/js/auth.js"></script>
|
||||
<script src="/static/js/info-collect.js"></script>
|
||||
<script src="/static/js/router.js"></script>
|
||||
<script src="/static/js/dashboard.js"></script>
|
||||
<script src="/static/js/monitor.js"></script>
|
||||
<script src="/static/js/chat.js"></script>
|
||||
<script src="/static/js/settings.js"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/xterm@4.19.0/lib/xterm.js"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/xterm-addon-fit@0.5.0/lib/xterm-addon-fit.js"></script>
|
||||
<script src="/static/js/terminal.js"></script>
|
||||
<script src="/static/js/knowledge.js"></script>
|
||||
<script src="/static/js/skills.js"></script>
|
||||
<script src="/static/js/vulnerability.js?v=4"></script>
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user