Compare commits

..

48 Commits

Author SHA1 Message Date
公明 0b950f95db Add files via upload 2026-03-07 00:17:02 +08:00
公明 d36984a1c1 Add files via upload 2026-03-06 23:21:16 +08:00
公明 da2109a970 Update version number to v1.3.18 2026-03-06 23:18:49 +08:00
公明 1866aa8089 Add files via upload 2026-03-06 22:51:18 +08:00
公明 5af06e539d Update config.yaml 2026-03-06 22:42:19 +08:00
公明 7493e70686 Add files via upload 2026-03-06 22:39:30 +08:00
公明 81f7a601b7 Update config.yaml 2026-03-06 21:06:42 +08:00
公明 27830d1399 Add files via upload 2026-03-06 20:11:22 +08:00
公明 d9a0178f80 Merge pull request #47 from chhs1129/fix-bug-logger-missing-error
Fix: logger shows empty error msg
2026-03-06 10:20:44 +08:00
chhs1129 1dd8cc7f50 Fix: logger shows empty error msg 2026-03-05 09:40:47 -08:00
公明 55045dd4e0 Add files via upload 2026-03-04 00:18:29 +08:00
公明 90508c9084 Update version to v1.3.16 in config.yaml 2026-03-03 20:03:56 +08:00
公明 361480f2d1 Add files via upload 2026-03-03 19:55:24 +08:00
公明 538565117b Add files via upload 2026-03-03 19:36:56 +08:00
公明 1c8742b7b6 Update README_CN.md 2026-03-03 13:52:50 +08:00
公明 2fb6a1d1ef Add disclaimer for ethical use of CyberStrikeAI
Added a disclaimer section emphasizing the ethical use of the tool.
2026-03-03 10:07:43 +08:00
公明 6e390acb3d Update README.md 2026-03-03 10:06:31 +08:00
公明 d6236e285d Update version to v1.3.15 in config.yaml 2026-03-03 01:34:14 +08:00
公明 ad8efffbb4 Add files via upload 2026-03-03 01:31:05 +08:00
公明 352d9b712c Add files via upload 2026-03-03 01:30:23 +08:00
公明 acadbe19c6 Add files via upload 2026-03-03 01:28:30 +08:00
公明 c265e66afb Update config.yaml 2026-03-02 20:41:41 +08:00
公明 647bb4b5e4 Add files via upload 2026-03-02 20:37:27 +08:00
公明 dd311f7a3b Add files via upload 2026-03-02 20:13:16 +08:00
公明 2e482a3baf Update version number to v1.3.13 2026-03-02 01:11:08 +08:00
公明 67d5e7f11e Add files via upload 2026-03-02 01:10:44 +08:00
公明 7e0198a64c Add files via upload 2026-03-02 00:58:40 +08:00
公明 1e50272229 Update version number to v1.3.12 2026-03-02 00:52:38 +08:00
公明 39b47a86fb Add files via upload 2026-03-02 00:49:21 +08:00
公明 74738ee555 Add files via upload 2026-03-01 13:35:11 +08:00
公明 90bc3f4b61 Update config.yaml 2026-02-28 23:34:07 +08:00
公明 ad96be3c64 Add files via upload 2026-02-28 23:31:17 +08:00
公明 8866ff4cdd Add files via upload 2026-02-28 23:09:48 +08:00
公明 3534a956b2 Add files via upload 2026-02-28 23:03:09 +08:00
公明 691793cb38 Update config.yaml 2026-02-28 00:52:51 +08:00
公明 7270e3c3d1 Add files via upload 2026-02-28 00:52:17 +08:00
公明 5e28782b1f Add files via upload 2026-02-28 00:33:35 +08:00
公明 3e61b77b9c Add files via upload 2026-02-28 00:30:53 +08:00
公明 64f9053061 Update config.yaml 2026-02-28 00:29:04 +08:00
公明 426b0e282e Add files via upload 2026-02-28 00:27:09 +08:00
公明 78c6bd0b6a Add files via upload 2026-02-25 19:58:28 +08:00
公明 e54815e018 Update config.yaml 2026-02-21 01:02:38 +08:00
公明 9baa99ea40 Add files via upload 2026-02-21 01:01:51 +08:00
公明 83a8c46db1 Add files via upload 2026-02-21 00:50:59 +08:00
公明 4b2619e1fe Update config.yaml 2026-02-20 18:48:07 +08:00
公明 3fffee80f4 Add files via upload 2026-02-20 18:43:50 +08:00
公明 41d7afcf99 Add files via upload 2026-02-20 18:10:29 +08:00
公明 6431dcb240 Add files via upload 2026-02-20 17:53:38 +08:00
35 changed files with 9226 additions and 241 deletions
+22 -14
View File
@@ -83,6 +83,7 @@ CyberStrikeAI is an **AI-native security testing platform** built in Go. It inte
- 📋 Batch task management: create task queues, add multiple tasks, and execute them sequentially
- 🎭 Role-based testing: predefined security testing roles (Penetration Testing, CTF, Web App Scanning, etc.) with custom prompts and tool restrictions
- 🎯 Skills system: 20+ predefined security testing skills (SQL injection, XSS, API security, etc.) that can be attached to roles or called on-demand by AI agents
- 📱 **Chatbot**: DingTalk and Lark (Feishu) long-lived connections so you can talk to CyberStrikeAI from mobile (see [Robot / Chatbot guide](docs/robot_en.md) for setup and commands)
## Tool Overview
@@ -460,6 +461,10 @@ tools:
enabled: true
```
## Related documentation
- [Robot / Chatbot guide (DingTalk & Lark)](docs/robot_en.md): Full setup, commands, and troubleshooting for using CyberStrikeAI from DingTalk or Lark on your phone. **Follow this doc to avoid common pitfalls.**
## Project Layout
```
@@ -470,6 +475,7 @@ CyberStrikeAI/
├── tools/ # YAML tool recipes (100+ examples provided)
├── roles/ # Role configurations (12+ predefined security testing roles)
├── skills/ # Skills directory (20+ predefined security testing skills)
├── docs/ # Documentation (e.g. robot/chbot guide)
├── images/ # Docs screenshots & diagrams
├── config.yaml # Runtime configuration
├── run.sh # Convenience launcher
@@ -495,20 +501,6 @@ Compress the 5 MB nuclei report, summarize critical CVEs, and attach the artifac
Build an attack chain for the latest engagement and export the node list with severity >= high.
```
## Changelog
### Recent Highlights
- **2026-01-27** OpenAPI documentation with interactive testing interface, supporting conversation management, message interaction, and result querying
- **2026-01-15** Skills system with 20+ predefined security testing skills
- **2026-01-11** Role-based testing with predefined security testing roles
- **2026-01-08** SSE transport mode support for external MCP servers
- **2026-01-01** Batch task management with queue-based execution
- **2025-12-25** Vulnerability management and conversation grouping features
- **2025-12-20** Knowledge base with vector search and hybrid retrieval
## 404Starlink
<img src="./images/404StarLinkLogo.png" width="30%">
@@ -526,6 +518,22 @@ CyberStrikeAI has joined [404Starlink](https://github.com/knownsec/404StarLink)
![Stargazers over time](https://starchart.cc/Ed1s0nZ/CyberStrikeAI.svg)
---
## ⚠️ Disclaimer
**This tool is for educational and authorized testing purposes only!**
CyberStrikeAI is a professional security testing platform designed to assist security researchers, penetration testers, and IT professionals in conducting security assessments and vulnerability research **with explicit authorization**.
**By using this tool, you agree to:**
- Use this tool only on systems where you have clear written authorization
- Comply with all applicable laws, regulations, and ethical standards
- Take full responsibility for any unauthorized use or misuse
- Not use this tool for any illegal or malicious purposes
**The developers are not responsible for any misuse!** Please ensure your usage complies with local laws and regulations, and that you have obtained explicit authorization from the target system owner.
---
Need help or want to contribute? Open an issue or PR—community tooling additions are welcome!
+22 -13
View File
@@ -82,6 +82,7 @@ CyberStrikeAI 是一款 **AI 原生安全测试平台**,基于 Go 构建,集
- 📋 批量任务管理:创建任务队列,批量添加任务,依次顺序执行,支持任务编辑与状态跟踪
- 🎭 角色化测试:预设安全测试角色(渗透测试、CTF、Web 应用扫描等),支持自定义提示词和工具限制
- 🎯 Skills 技能系统:20+ 预设安全测试技能(SQL 注入、XSS、API 安全等),可附加到角色或由 AI 按需调用
- 📱 **机器人**:支持钉钉、飞书长连接,在手机端与 CyberStrikeAI 对话(配置与命令详见 [机器人使用说明](docs/robot.md)
## 工具概览
@@ -459,6 +460,10 @@ tools:
enabled: true
```
## 相关文档
- [机器人使用说明(钉钉 / 飞书)](docs/robot.md):在手机端通过钉钉、飞书与 CyberStrikeAI 对话的完整配置步骤、命令与排查说明,**建议按该文档操作以避免走弯路**。
## 项目结构
```
@@ -469,6 +474,7 @@ CyberStrikeAI/
├── tools/ # YAML 工具目录(含 100+ 示例)
├── roles/ # 角色配置文件目录(含 12+ 预设安全测试角色)
├── skills/ # Skills 目录(含 20+ 预设安全测试技能)
├── docs/ # 说明文档(如机器人使用说明)
├── images/ # 文档配图
├── config.yaml # 运行配置
├── run.sh # 启动脚本
@@ -494,19 +500,6 @@ CyberStrikeAI/
构建最新一次测试的攻击链,只导出风险 >= 高的节点列表。
```
## 更新日志
### 近期亮点
- **2026-01-27** 新增 OpenAPI 文档,提供交互式测试界面,支持对话管理、消息交互和结果查询
- **2026-01-15** 新增 Skills 技能系统,内置 20+ 预设安全测试技能
- **2026-01-11** – 新增角色化测试功能,支持预设安全测试角色
- **2026-01-08** 新增 SSE 传输模式支持,外部 MCP 联邦支持三种模式
- **2026-01-01** – 新增批量任务管理功能,支持队列式任务执行
- **2025-12-25** 新增漏洞管理和对话分组功能
- **2025-12-20** – 新增知识库功能,支持向量检索和混合搜索
## 404星链计划
<img src="./images/404StarLinkLogo.png" width="30%">
@@ -524,4 +517,20 @@ CyberStrikeAI 现已加入 [404星链计划](https://github.com/knownsec/404Star
---
## ⚠️ 免责声明
**本工具仅供教育和授权测试使用!**
CyberStrikeAI 是一个专业的安全测试平台,旨在帮助安全研究人员、渗透测试人员和IT专业人员在**获得明确授权**的情况下进行安全评估和漏洞研究。
**使用本工具即表示您同意:**
- 仅在您拥有明确书面授权的系统上使用此工具
- 遵守所有适用的法律法规和道德准则
- 对任何未经授权的使用或滥用行为承担全部责任
- 不会将本工具用于任何非法或恶意目的
**开发者不对任何滥用行为负责!** 请确保您的使用符合当地法律法规,并获得目标系统所有者的明确授权。
---
欢迎提交 Issue/PR 贡献新的工具模版或优化建议!
+40 -1
View File
@@ -10,7 +10,7 @@
# ============================================
# 前端显示的版本号(可选,不填则显示默认版本)
version: "v1.3.7"
version: "v1.3.18"
# 服务器配置
server:
@@ -116,6 +116,45 @@ knowledge:
top_k: 5 # 检索返回的Top-K结果数量
similarity_threshold: 0.7 # 相似度阈值(0-1),低于此值的结果将被过滤
hybrid_weight: 0.7 # 混合检索权重(0-1),向量检索的权重,1.0表示纯向量检索,0.0表示纯关键词检索
# ============================================
# 索引配置(用于解决 API 限制问题)
# ============================================
indexing:
# 分块配置
chunk_size: 512 # 每个块的最大 token 数(默认 512),长文本会被分割成多个块
chunk_overlap: 50 # 块之间的重叠 token 数(默认 50),保持上下文连贯性
max_chunks_per_item: 0 # 单个知识项的最大块数量(0 表示不限制),防止单个文件消耗过多 API 配额
# 速率限制配置(解决 429 错误)
max_rpm: 0 # 每分钟最大请求数(默认 0 表示不限制),如 OpenAI 默认 200 RPM
rate_limit_delay_ms: 300 # 请求间隔毫秒数(默认 300),用于避免 API 速率限制,设为 0 不限制
# 建议值:200 次/分钟≈300ms, 100 次/分钟≈600ms
# 重试配置
max_retries: 3 # 最大重试次数(默认 3),遇到速率限制或服务器错误时自动重试
retry_delay_ms: 1000 # 重试间隔毫秒数(默认 1000),每次重试会递增延迟
# ============================================
# 机器人配置(企业微信、钉钉、飞书)
# ============================================
# 用于在手机端通过企业微信/钉钉/飞书与 CyberStrikeAI 对话,无需部署在服务器上也可使用
# 在系统设置 -> 机器人设置 中可配置
robots:
wecom: # 企业微信
enabled: false
token: ""
encoding_aes_key: ""
corp_id: ""
secret: ""
agent_id: 0
dingtalk: # 钉钉
enabled: false
client_id:
client_secret:
lark: # 飞书
enabled: false
app_id: ""
app_secret: ""
verify_token: ""
# ============================================
# Skills 相关配置
+225
View File
@@ -0,0 +1,225 @@
# CyberStrikeAI 机器人使用说明
[English](robot_en.md)
本文档说明如何通过**钉钉**、**飞书**与 CyberStrikeAI 对话(长连接模式),在手机端即可使用,无需在服务器上打开网页。按下面步骤操作可避免常见弯路。
---
## 一、在 CyberStrikeAI 里从哪里配置
1. 登录 CyberStrikeAI Web 端
2. 左侧导航进入 **系统设置**
3. 在左侧设置分类中点击 **机器人设置**(位于「基本设置」与「安全设置」之间)
4. 按平台勾选并填写(钉钉填 Client ID / Client Secret,飞书填 App ID / App Secret
5. 点击 **应用配置** 保存
6. **重启 CyberStrikeAI 应用**(只保存不重启,机器人不会连上)
配置会写入 `config.yaml``robots` 段,也可在配置文件中直接编辑。**修改钉钉/飞书配置后必须重启,长连接才会生效。**
---
## 二、支持的平台(长连接)
| 平台 | 说明 |
|------|------|
| 钉钉 | 使用 Stream 长连接,程序主动连接钉钉接收消息 |
| 飞书 | 使用长连接,程序主动连接飞书接收消息 |
下面第三节会按平台写清:在开放平台要做什么、要复制哪些字段、填到 CyberStrikeAI 的哪一栏。
---
## 三、各平台配置项与详细步骤
### 3.1 钉钉
**先搞清楚:两种钉钉机器人不一样**
| 类型 | 从哪里创建 | 能否做「用户发消息→机器人回复」 | 本程序是否支持 |
|------|------------|----------------------------------|----------------|
| **自定义机器人** | 钉钉群里:群设置 → 添加机器人 → 自定义(Webhook) | ❌ 不能,只能你往群里发消息 | ❌ 不支持 |
| **企业内部应用机器人** | [钉钉开放平台](https://open.dingtalk.com) 创建应用并开通机器人 | ✅ 能 | ✅ 支持 |
如果你手里是「自定义机器人」的 Webhook 地址(`oapi.dingtalk.com/robot/send?access_token=xxx`)和加签密钥(`SEC...`),**不能直接填到本程序**,必须按下面步骤在开放平台创建「企业内部应用」并拿到 **Client ID**、**Client Secret**。
---
**钉钉配置完整步骤(按顺序做)**
1. **打开钉钉开放平台**
浏览器访问 [https://open.dingtalk.com](https://open.dingtalk.com),用**企业管理员**账号登录。
2. **进入应用开发**
左侧选 **应用开发****企业内部开发** → 点击 **创建应用**(或选择已有应用)。填写应用名称等基本信息后创建。
3. **拿到 Client ID 和 Client Secret**
- 左侧点 **凭证与基础信息**(在「基础信息」下)。
- 页面上有 **Client ID(原 AppKey****Client Secret(原 AppSecret**
- 点击复制,**不要手打**,注意:数字 **0** 和字母 **o**、数字 **1** 和字母 **l** 容易抄错(例如 `ding9gf9tiozuc504aer` 中间是数字 **504** 不是 5o4)。
4. **开通机器人并选 Stream 模式**
- 左侧 **应用能力****机器人**
- 打开「机器人配置」开关。
- 填写机器人名称、简介等(必填项按提示填)。
- **关键**:消息接收方式要选 **「Stream 模式」**(流式接入)。若只有「HTTP 回调」或未选 Stream,本程序收不到消息。
- 保存。
5. **权限与发布**
- 左侧 **权限管理**:搜索「机器人」「消息」等,勾选**接收消息**、**发送消息**等机器人相关权限,并确认授权。
- 左侧 **版本管理与发布**:若有未发布配置,点击 **发布新版本** / **上线**,否则修改不生效。
6. **填回 CyberStrikeAI**
- 回到 CyberStrikeAI → 系统设置 → 机器人设置 → 钉钉。
- 勾选「启用钉钉机器人」。
- **Client ID (AppKey)** 粘贴第 3 步复制的 Client ID。
- **Client Secret** 粘贴第 3 步复制的 Client Secret。
- 点击 **应用配置**,然后**重启 CyberStrikeAI**。
---
**CyberStrikeAI 钉钉栏位对照**
| CyberStrikeAI 中填写项 | 在钉钉开放平台的来源 |
|------------------------|------------------------|
| 启用钉钉机器人 | 勾选即启用 |
| Client ID (AppKey) | 凭证与基础信息 → **Client ID(原 AppKey** |
| Client Secret | 凭证与基础信息 → **Client Secret(原 AppSecret** |
---
### 3.2 飞书 (Lark)
| 配置项 | 说明 |
|--------|------|
| 启用飞书机器人 | 勾选后启动飞书长连接 |
| App ID | 飞书开放平台应用凭证中的 App ID |
| App Secret | 飞书开放平台应用凭证中的 App Secret |
| Verify Token | 事件订阅用(可选) |
**飞书配置简要步骤**:登录 [飞书开放平台](https://open.feishu.cn) → 创建企业自建应用 → 在「凭证与基础信息」中获取 **App ID**、**App Secret** → 在「应用能力」中开通**机器人**并启用相应权限 → 发布应用 → 将 App ID、App Secret 填到 CyberStrikeAI 机器人设置 → 保存并**重启应用**。
---
## 四、机器人命令
在钉钉/飞书中向机器人发送以下**文本命令**(仅支持文本):
| 命令 | 说明 |
|------|------|
| **帮助** | 显示命令帮助与说明 |
| **列表****对话列表** | 列出所有对话的标题与对话 ID |
| **切换 \<对话ID\>****继续 \<对话ID\>** | 指定对话 ID,后续消息在该对话中继续 |
| **新对话** | 开启一个新对话,后续消息在新对话中 |
| **清空** | 清空当前对话上下文(效果等同「新对话」) |
| **当前** | 显示当前对话 ID 与标题 |
| **停止** | 中断当前正在执行的任务 |
| **角色****角色列表** | 列出所有可用角色(渗透测试、CTF、Web 应用扫描等) |
| **角色 \<角色名\>****切换角色 \<角色名\>** | 切换当前使用的角色 |
| **删除 \<对话ID\>** | 删除指定对话 |
| **版本** | 显示当前 CyberStrikeAI 版本号 |
除以上命令外,**直接输入任意文字**会作为用户消息发给 AI,与 Web 端对话逻辑一致(渗透测试/安全分析等)。
---
## 五、如何使用(要 @ 机器人吗?)
- **单聊(推荐)**:在钉钉/飞书里**搜索并打开该机器人**,进入与机器人的**私聊**,直接输入「帮助」或任意文字即可,**不需要 @**。
- **群聊**:若机器人被添加到群里,在群内只有 **@机器人** 后发送的消息才会被机器人收到并回复;不 @ 的群消息不会触发机器人。
总结:和机器人**单聊时直接发**;在**群里用时需要 @机器人** 再发内容。
---
## 六、推荐使用流程(避免漏步骤)
1. **在开放平台**:按第三节完成钉钉或飞书应用创建、凭证复制、机器人开通(钉钉务必选 **Stream 模式**)、权限与发布。
2. **在 CyberStrikeAI**:系统设置 → 机器人设置 → 勾选对应平台,粘贴 Client ID/App ID、Client Secret/App Secret → 点击 **应用配置**
3. **重启 CyberStrikeAI 进程**(否则长连接不会建立)。
4. **在手机钉钉/飞书**:找到该机器人(单聊直接发,群聊需 @机器人),发「帮助」或任意内容测试。
若发消息没反应,先看 **第九节排查****第十节常见弯路**
---
## 七、配置文件示例
`config.yaml` 中机器人相关片段示例:
```yaml
robots:
dingtalk:
enabled: true
client_id: "your_dingtalk_app_key"
client_secret: "your_dingtalk_app_secret"
lark:
enabled: true
app_id: "your_lark_app_id"
app_secret: "your_lark_app_secret"
verify_token: ""
```
修改后需**重启应用**,长连接在应用启动时建立。
---
## 八、如何验证是否可用(无需钉钉/飞书客户端)
在未安装钉钉或飞书时,可用**测试接口**验证机器人逻辑是否正常:
1. 先登录 CyberStrikeAI Web 端(保证有登录态)。
2. 使用 curl 调用测试接口(需携带登录后的 Cookie):
```bash
# 将 YOUR_COOKIE 替换为登录后获得的 Cookie(浏览器 F12 → 网络 → 任意请求 → 请求头中的 Cookie)
curl -X POST "http://localhost:8080/api/robot/test" \
-H "Content-Type: application/json" \
-H "Cookie: YOUR_COOKIE" \
-d '{"platform":"dingtalk","user_id":"test_user","text":"帮助"}'
```
若返回 JSON 中含有 `"reply":"【CyberStrikeAI 机器人命令】..."`,说明命令处理正常。可再试 `"text":"列表"``"text":"当前"` 等。
接口说明:`POST /api/robot/test`(需登录),请求体 `{"platform":"可选","user_id":"可选","text":"必填"}`,响应 `{"reply":"回复内容"}`
---
## 九、钉钉发消息没反应时排查
按顺序检查:
0. **笔记本合盖睡眠 / 断网后**
钉钉、飞书均使用长连接收消息,睡眠或断网后连接会断开。程序会**自动重连**(约 5 秒~60 秒内重试)。唤醒或恢复网络后稍等一会儿再发消息;若仍无反应,可重启 CyberStrikeAI 进程。
1. **Client ID / Client Secret 是否与开放平台完全一致**
从「凭证与基础信息」里**复制粘贴**,不要手打。注意数字 **0** 与字母 **o**、数字 **1** 与字母 **l**(例如 `ding9gf9tiozuc504aer` 中间是 **504** 不是 5o4)。
2. **是否在保存配置后重启了应用**
机器人长连接在**应用启动时**建立。在 Web 端点击「应用配置」只写入配置文件,**必须重启 CyberStrikeAI 进程**后钉钉连接才会生效。
3. **看程序日志**
- 启动后应看到:`钉钉 Stream 正在连接…``钉钉 Stream 已启动(无需公网),等待收消息`
- 若出现 `钉钉 Stream 长连接退出` 且带错误信息,多为 **Client ID / Client Secret 错误**或**开放平台未开通流式接入**
- 在钉钉里发一条消息后,若有收到,应有日志:`钉钉收到消息`;若没有,说明钉钉未把消息推到本程序(回头检查开放平台「机器人」是否开通、是否选用 **Stream 模式**)。
4. **开放平台侧**
应用需已**发布**;在「机器人」能力中需开启**流式接入(Stream)** 用于接收消息(仅 HTTP 回调不够);权限管理里需有机器人接收、发送消息等权限。
---
## 十、常见弯路(避免踩坑)
- **用错了机器人类型**:在钉钉**群里**添加的「自定义」机器人(Webhook + 加签)**不能**用来做对话,本程序只支持**开放平台「企业内部应用」**里的机器人。
- **只保存没重启**:在 CyberStrikeAI 里改完机器人配置后必须**重启应用**,否则长连接不会建立。
- **Client ID 抄错**:开放平台是 `504` 就填 `504`,不要填成 `5o4`;尽量用复制粘贴。
- **钉钉只开了 HTTP 回调没开 Stream**:本程序通过 **Stream 长连接**收消息,开放平台里机器人的消息接收方式必须选 **Stream 模式**
- **应用没发布**:开放平台里修改了机器人或权限后,要在「版本管理与发布」里**发布新版本**,否则不生效。
---
## 十一、注意事项
- 钉钉、飞书均**仅处理文本消息**;其他类型(如图片、语音)会提示暂不支持或忽略。
- 会话与 Web 端共用同一套对话数据:在机器人里创建的对话会在 Web 端「对话」列表中看到,反之亦然。
- 机器人执行逻辑与 **`/api/agent-loop/stream`** 一致(含进度回调、过程详情写入数据库),仅不向客户端推送 SSE,最后将完整回复一次性发回钉钉/飞书/企业微信。
+224
View File
@@ -0,0 +1,224 @@
# CyberStrikeAI Robot / Chatbot Guide
[中文](robot.md)
This document explains how to chat with CyberStrikeAI from **DingTalk** and **Lark (Feishu)** using long-lived connections—no need to open a browser on the server. Following the steps below helps avoid common mistakes.
---
## 1. Where to configure in CyberStrikeAI
1. Log in to the CyberStrikeAI web UI.
2. Open **System Settings** in the left sidebar.
3. Click **Robot settings** (between “Basic” and “Security”).
4. Enable the platform and fill in credentials (DingTalk: Client ID / Client Secret; Lark: App ID / App Secret).
5. Click **Apply configuration** to save.
6. **Restart the CyberStrikeAI process** (saving alone does not establish the connection).
Settings are written to the `robots` section of `config.yaml`; you can also edit the file directly. **After changing DingTalk or Lark config, you must restart for the long-lived connection to take effect.**
---
## 2. Supported platforms (long-lived connection)
| Platform | Description |
|----------|-------------|
| DingTalk | Stream long-lived connection; the app connects to DingTalk to receive messages |
| Lark (Feishu) | Long-lived connection; the app connects to Lark to receive messages |
Section 3 below describes, per platform, what to do in the developer console and which fields to copy into CyberStrikeAI.
---
## 3. Configuration and step-by-step setup
### 3.1 DingTalk
**Important: two types of DingTalk bots**
| Type | Where its created | Can do “user sends message → bot replies”? | Supported here? |
|------|-------------------|-------------------------------------------|------------------|
| **Custom bot (Webhook)** | In a DingTalk group: Group settings → Add robot → Custom (Webhook) | No; you can only post to the group | No |
| **Enterprise internal app bot** | [DingTalk Open Platform](https://open.dingtalk.com): create an app and enable the bot | Yes | Yes |
If you only have a **custom bot** Webhook URL (`oapi.dingtalk.com/robot/send?access_token=...`) and sign secret (`SEC...`), **do not** put them into CyberStrikeAI. You must create an **enterprise internal app** in the open platform and obtain **Client ID** and **Client Secret** as below.
---
**DingTalk setup (in order)**
1. **Open DingTalk Open Platform**
Go to [https://open.dingtalk.com](https://open.dingtalk.com) and log in with an **enterprise admin** account.
2. **Create or select an app**
In the left menu: **Application development****Enterprise internal development****Create application** (or choose an existing app). Fill in the app name and create.
3. **Get Client ID and Client Secret**
- In the left menu open **Credentials and basic info** (under “Basic information”).
- Copy **Client ID (formerly AppKey)** and **Client Secret (formerly AppSecret)**.
- Use copy/paste; avoid typing by hand. Watch for **0** vs **o** and **1** vs **l** (e.g. `ding9gf9tiozuc504aer` has the digits **504**, not 5o4).
4. **Enable the bot and choose Stream mode**
- Left menu: **Application capabilities****Robot**.
- Turn on “Robot configuration”.
- Fill in robot name, description, etc. as required.
- **Critical**: set message reception to **“Stream mode”** (流式接入). If you only enable “HTTP callback” or do not select Stream, CyberStrikeAI will not receive messages.
- Save.
5. **Permissions and release**
- Left menu: **Permission management** — search for “robot”, “message”, etc., and enable **receive message**, **send message**, and other bot-related permissions; confirm.
- Left menu: **Version management and release** — if there are unpublished changes, click **Release new version** / **Publish**; otherwise changes do not take effect.
6. **Fill in CyberStrikeAI**
- In CyberStrikeAI: System settings → Robot settings → DingTalk.
- Enable “Enable DingTalk robot”.
- Paste the Client ID and Client Secret from step 3.
- Click **Apply configuration**, then **restart CyberStrikeAI**.
---
**Field mapping (DingTalk)**
| Field in CyberStrikeAI | Source in DingTalk Open Platform |
|------------------------|----------------------------------|
| Enable DingTalk robot | Check to enable |
| Client ID (AppKey) | Credentials and basic info → **Client ID (formerly AppKey)** |
| Client Secret | Credentials and basic info → **Client Secret (formerly AppSecret)** |
---
### 3.2 Lark (Feishu)
| Field | Description |
|-------|-------------|
| Enable Lark robot | Check to start the Lark long-lived connection |
| App ID | From Lark open platform app credentials |
| App Secret | From Lark open platform app credentials |
| Verify Token | Optional; for event subscription |
**Lark setup in short**: Log in to [Lark Open Platform](https://open.feishu.cn) → Create an enterprise app → In “Credentials and basic info” get **App ID** and **App Secret** → In “Application capabilities” enable **Robot** and the right permissions → Publish the app → Enter App ID and App Secret in CyberStrikeAI robot settings → Save and **restart** the app.
---
## 4. Bot commands
Send these **text commands** to the bot in DingTalk or Lark (text only):
| Command | Description |
|---------|-------------|
| **帮助** (help) | Show command help |
| **列表** or **对话列表** (list) | List all conversation titles and IDs |
| **切换 \<conversationID\>** or **继续 \<conversationID\>** | Continue in the given conversation |
| **新对话** (new) | Start a new conversation |
| **清空** (clear) | Clear current context (same effect as new conversation) |
| **当前** (current) | Show current conversation ID and title |
| **停止** (stop) | Abort the currently running task |
| **角色** or **角色列表** (roles) | List all available roles (penetration testing, CTF, Web scan, etc.) |
| **角色 \<roleName\>** or **切换角色 \<roleName\>** | Switch to the specified role |
| **删除 \<conversationID\>** | Delete the specified conversation |
| **版本** (version) | Show current CyberStrikeAI version |
Any other text is sent to the AI as a user message, same as in the web UI (e.g. penetration testing, security analysis).
---
## 5. How to use (do I need to @ the bot?)
- **Direct chat (recommended)**: In DingTalk or Lark, **search for the bot and open a direct chat**. Type “帮助” or any message; **no @ needed**.
- **Group chat**: If the bot is in a group, only messages that **@ the bot** are received and answered; other group messages are ignored.
Summary: **Direct chat** — just send; **in a group** — @ the bot first, then send.
---
## 6. Recommended flow (so you dont skip steps)
1. **In the open platform**: Complete app creation, copy credentials, enable the bot (DingTalk: **Stream mode**), set permissions, and publish (Section 3).
2. **In CyberStrikeAI**: System settings → Robot settings → Enable the platform, paste Client ID/App ID and Client Secret/App Secret → **Apply configuration**.
3. **Restart the CyberStrikeAI process** (otherwise the long-lived connection is not established).
4. **On your phone**: Open DingTalk or Lark, find the bot (direct chat or @ in a group), send “帮助” or any message to test.
If the bot does not respond, see **Section 9 (troubleshooting)** and **Section 10 (common pitfalls)**.
---
## 7. Config file example
Example `robots` section in `config.yaml`:
```yaml
robots:
dingtalk:
enabled: true
client_id: "your_dingtalk_app_key"
client_secret: "your_dingtalk_app_secret"
lark:
enabled: true
app_id: "your_lark_app_id"
app_secret: "your_lark_app_secret"
verify_token: ""
```
**Restart the app** after changes; the long-lived connection is created at startup.
---
## 8. Testing without DingTalk/Lark installed
You can verify bot logic with the **test API** (no DingTalk/Lark client needed):
1. Log in to the CyberStrikeAI web UI (so you have a session).
2. Call the test endpoint with curl (include your session Cookie):
```bash
# Replace YOUR_COOKIE with the Cookie from your browser (F12 → Network → any request → Request headers → Cookie)
curl -X POST "http://localhost:8080/api/robot/test" \
-H "Content-Type: application/json" \
-H "Cookie: YOUR_COOKIE" \
-d '{"platform":"dingtalk","user_id":"test_user","text":"帮助"}'
```
If the JSON response contains `"reply":"【CyberStrikeAI 机器人命令】..."`, command handling works. You can also try `"text":"列表"` or `"text":"当前"`.
API: `POST /api/robot/test` (requires login). Body: `{"platform":"optional","user_id":"optional","text":"required"}`. Response: `{"reply":"..."}`.
---
## 9. DingTalk: no response when sending messages
Check in this order:
0. **After laptop sleep or network drop**
DingTalk and Lark both use long-lived connections; they break when the machine sleeps or the network drops. The app **auto-reconnects** (retries within about 560 seconds). After wake or network recovery, wait a moment before sending; if there is still no response, restart the CyberStrikeAI process.
1. **Client ID / Client Secret match the open platform exactly**
Copy from “Credentials and basic info”; avoid typing. Watch **0** vs **o** and **1** vs **l** (e.g. `ding9gf9tiozuc504aer` has **504**, not 5o4).
2. **Did you restart after saving?**
The long-lived connection is created at **startup**. “Apply configuration” only updates the config file; you **must restart the CyberStrikeAI process** for the DingTalk connection to start.
3. **Application logs**
- On startup you should see: `钉钉 Stream 正在连接…`, `钉钉 Stream 已启动(无需公网),等待收消息`.
- If you see `钉钉 Stream 长连接退出` with an error, its usually wrong **Client ID / Client Secret** or **Stream not enabled** in the open platform.
- After sending a message in DingTalk, you should see `钉钉收到消息` in the logs; if not, the platform is not pushing to this app (check that the bot is enabled and **Stream mode** is selected).
4. **Open platform**
The app must be **published**. Under “Robot” you must enable **Stream** for receiving messages (HTTP callback only is not enough). Permission management must include robot receive/send message permissions.
---
## 10. Common pitfalls
- **Wrong bot type**: The “Custom” bot added in a DingTalk **group** (Webhook + sign secret) **cannot** be used for two-way chat. Only the **enterprise internal app** bot from the open platform is supported.
- **Saved but not restarted**: After changing robot settings in CyberStrikeAI you **must restart** the app, or the long-lived connection will not be established.
- **Client ID typo**: If the platform shows `504`, use `504` (not `5o4`); prefer copy/paste.
- **DingTalk: only HTTP callback, no Stream**: This app receives messages via **Stream**. In the open platform, message reception must be **Stream mode**.
- **App not published**: After changing the bot or permissions in the open platform, **publish a new version** under “Version management and release”, or changes wont apply.
---
## 11. Notes
- DingTalk and Lark: **text messages only**; other types (e.g. image, voice) are not supported and may be ignored.
- Conversations are shared with the web UI: conversations created from the bot appear in the web “Conversations” list and vice versa.
- Bot execution uses the same logic as **`/api/agent-loop/stream`** (progress callbacks, process details stored in the DB); only the final reply is sent back to DingTalk/Lark in one message (no SSE to the client).
+11 -1
View File
@@ -1,16 +1,21 @@
module cyberstrike-ai
go 1.23.0
go 1.24.0
toolchain go1.24.4
require (
github.com/creack/pty v1.1.24
github.com/gin-gonic/gin v1.9.1
github.com/google/uuid v1.5.0
github.com/gorilla/websocket v1.5.0
github.com/larksuite/oapi-sdk-go/v3 v3.4.22
github.com/mattn/go-sqlite3 v1.14.18
github.com/modelcontextprotocol/go-sdk v1.2.0
github.com/open-dingtalk/dingtalk-stream-sdk-go v0.9.1
github.com/pkoukk/tiktoken-go v0.1.8
go.uber.org/zap v1.26.0
golang.org/x/time v0.14.0
gopkg.in/yaml.v3 v3.0.1
)
@@ -24,6 +29,7 @@ require (
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.14.0 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/google/jsonschema-go v0.3.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.2.4 // indirect
@@ -44,3 +50,7 @@ require (
golang.org/x/text v0.13.0 // indirect
google.golang.org/protobuf v1.30.0 // indirect
)
// 修复钉钉 Stream SDK 在长连接断开(熄屏/网络中断)后 "panic: send on closed channel" 问题
// 详见: https://github.com/open-dingtalk/dingtalk-stream-sdk-go/issues/28
replace github.com/open-dingtalk/dingtalk-stream-sdk-go => github.com/uouuou/dingtalk-stream-sdk-go v0.0.0-20250626025113-079132acc406
+40
View File
@@ -4,6 +4,8 @@ github.com/bytedance/sonic v1.9.1/go.mod h1:i736AoUSYt75HyZLoJW9ERYxcy6eaN6h4BZX
github.com/chenzhuoyu/base64x v0.0.0-20211019084208-fb5309c8db06/go.mod h1:DH46F32mSOjUmXrMHnKwZdA8wcEefY7UVqBKYGjpdQY=
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 h1:qSGYFH7+jGhDF8vLC+iwCD4WpbV1EBDSzWkJODFLams=
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311/go.mod h1:b583jCggY9gE99b6G5LEC39OIiVsWj+R97kbl5odCEk=
github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s=
github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -25,6 +27,8 @@ github.com/go-playground/validator/v10 v10.14.0 h1:vgvQWe3XCz3gIeFDm/HnTIbj6UGmg
github.com/go-playground/validator/v10 v10.14.0/go.mod h1:9iXMNT7sEkjXb0I+enO7QXmzG6QCsPWY4zveKFVRSyU=
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang-jwt/jwt/v5 v5.2.2 h1:Rl4B7itRWVtYIHFrSNd7vhTiz9UpLdi6gZhZ3wEeDy8=
github.com/golang-jwt/jwt/v5 v5.2.2/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
@@ -36,11 +40,17 @@ github.com/google/jsonschema-go v0.3.0 h1:6AH2TxVNtk3IlvkkhjrtbUc4S8AvO0Xii0DxIy
github.com/google/jsonschema-go v0.3.0/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE=
github.com/google/uuid v1.5.0 h1:1p67kYwdtXjb0gL0BPiP1Av9wiZPo5A8z2cWkTZ+eyU=
github.com/google/uuid v1.5.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.5.0 h1:PPwGk2jz7EePpoHN/+ClbZu8SPxiqlu12wZP/3sWmnc=
github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.2.4 h1:acbojRNwl3o09bUq+yDCtZFc1aiwaAAxtcn8YkZXnvk=
github.com/klauspost/cpuid/v2 v2.2.4/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY=
github.com/larksuite/oapi-sdk-go/v3 v3.4.22 h1:57daKuslQPX9X3hC2idc5bu8bl2krfsBGWGJ6b5FlD8=
github.com/larksuite/oapi-sdk-go/v3 v3.4.22/go.mod h1:ZEplY+kwuIrj/nqw5uSCINNATcH3KdxSN7y+UxYY5fI=
github.com/leodido/go-urn v1.2.4 h1:XlAE/cm/ms7TE/VMVoduSpNBoyc2dOxHs5MZSwAN63Q=
github.com/leodido/go-urn v1.2.4/go.mod h1:7ZrI8mTSeBSHl/UaRyKQW1qZeMgak41ANeCNaVckg+4=
github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA=
@@ -75,8 +85,12 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.11 h1:BMaWp1Bb6fHwEtbplGBGJ498wD+LKlNSl25MjdZY4dU=
github.com/ugorji/go/codec v1.2.11/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
github.com/uouuou/dingtalk-stream-sdk-go v0.0.0-20250626025113-079132acc406 h1:b72HNsEnmTRn7vhWGOfbWHAkA5RbRCk0Pbc56V2WAuY=
github.com/uouuou/dingtalk-stream-sdk-go v0.0.0-20250626025113-079132acc406/go.mod h1:ln3IqPYYocZbYvl9TAOrG/cxGR9xcn4pnZRLdCTEGEU=
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
go.uber.org/goleak v1.2.0 h1:xqgm/S+aQvhWFTtR0XK3Jvg7z8kGV8P4X14IzwN3Eqk=
go.uber.org/goleak v1.2.0/go.mod h1:XJYK+MuIchqpmGmUSAzotztawfKvYLUIgg7guXrwVUo=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
@@ -86,21 +100,47 @@ go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/arch v0.3.0 h1:02VY4/ZcO/gBOH6PUaoiptASxtXU10jazRCP865E97k=
golang.org/x/arch v0.3.0/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.14.0 h1:wBqGXzWJW6m1XrIKlAH0Hs1JJ7+9KBwnIO8v66Q9cHc=
golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE=
golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=
golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.34.0 h1:qIpSLOxeCYGg9TrcJokLBG4KFA6d795g0xkBkiESGlo=
golang.org/x/tools v0.34.0/go.mod h1:pAP9OwEaY1CAW3HOmg3hLZC5Z0CCmzjAF2UQMSqNARg=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.30.0 h1:kPPoIgf3TsEvrm0PFe15JQ+570QVxYzEvvHqChK+cng=
google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
+82 -2
View File
@@ -7,6 +7,7 @@ import (
"net/http"
"os"
"path/filepath"
"sync"
"time"
"cyberstrike-ai/internal/agent"
@@ -14,6 +15,7 @@ import (
"cyberstrike-ai/internal/database"
"cyberstrike-ai/internal/handler"
"cyberstrike-ai/internal/knowledge"
"cyberstrike-ai/internal/robot"
"cyberstrike-ai/internal/logger"
"cyberstrike-ai/internal/mcp"
"cyberstrike-ai/internal/mcp/builtin"
@@ -43,6 +45,10 @@ type App struct {
knowledgeIndexer *knowledge.Indexer // 知识库索引器(用于动态初始化)
knowledgeHandler *handler.KnowledgeHandler // 知识库处理器(用于动态初始化)
agentHandler *handler.AgentHandler // Agent处理器(用于更新知识库管理器)
robotHandler *handler.RobotHandler // 机器人处理器(钉钉/飞书/企业微信)
robotMu sync.Mutex // 保护钉钉/飞书长连接的 cancel
dingCancel context.CancelFunc // 钉钉 Stream 取消函数,用于配置变更时重启
larkCancel context.CancelFunc // 飞书长连接取消函数,用于配置变更时重启
}
// New 创建新应用
@@ -192,7 +198,7 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
knowledgeRetriever = knowledge.NewRetriever(knowledgeDB, embedder, retrievalConfig, log.Logger)
// 创建索引器
knowledgeIndexer = knowledge.NewIndexer(knowledgeDB, embedder, log.Logger)
knowledgeIndexer = knowledge.NewIndexer(knowledgeDB, embedder, log.Logger, &cfg.Knowledge.Indexing)
// 注册知识检索工具到MCP服务器
knowledge.RegisterKnowledgeTool(mcpServer, knowledgeRetriever, knowledgeManager, log.Logger)
@@ -319,12 +325,14 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
roleHandler.SetSkillsManager(skillsManager) // 设置Skills管理器到RoleHandler
skillsHandler := handler.NewSkillsHandler(skillsManager, cfg, configPath, log.Logger)
fofaHandler := handler.NewFofaHandler(cfg, log.Logger)
terminalHandler := handler.NewTerminalHandler(log.Logger)
if db != nil {
skillsHandler.SetDB(db) // 设置数据库连接以便获取调用统计
}
// 创建OpenAPI处理器
conversationHandler := handler.NewConversationHandler(db, log.Logger)
robotHandler := handler.NewRobotHandler(cfg, db, agentHandler, log.Logger)
openAPIHandler := handler.NewOpenAPIHandler(db, log.Logger, resultStorage, conversationHandler, agentHandler)
// 创建 App 实例(部分字段稍后填充)
@@ -344,7 +352,10 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
knowledgeIndexer: knowledgeIndexer,
knowledgeHandler: knowledgeHandler,
agentHandler: agentHandler,
robotHandler: robotHandler,
}
// 飞书/钉钉长连接(无需公网),启用时在后台启动;后续前端应用配置时会通过 RestartRobotConnections 重启
app.startRobotConnections()
// 设置漏洞工具注册器(内置工具,必须设置)
vulnerabilityRegistrar := func() error {
@@ -401,6 +412,9 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
configHandler.SetRetrieverUpdater(knowledgeRetriever)
}
// 设置机器人连接重启器,前端应用配置后无需重启服务即可使钉钉/飞书新配置生效
configHandler.SetRobotRestarter(app)
// 设置路由(使用 App 实例以便动态获取 handler
setupRoutes(
router,
@@ -408,6 +422,7 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
agentHandler,
monitorHandler,
conversationHandler,
robotHandler,
groupHandler,
configHandler,
externalMCPHandler,
@@ -417,6 +432,7 @@ func New(cfg *config.Config, log *logger.Logger) (*App, error) {
roleHandler,
skillsHandler,
fofaHandler,
terminalHandler,
mcpServer,
authManager,
openAPIHandler,
@@ -452,6 +468,18 @@ func (a *App) Run() error {
// Shutdown 关闭应用
func (a *App) Shutdown() {
// 停止钉钉/飞书长连接
a.robotMu.Lock()
if a.dingCancel != nil {
a.dingCancel()
a.dingCancel = nil
}
if a.larkCancel != nil {
a.larkCancel()
a.larkCancel = nil
}
a.robotMu.Unlock()
// 停止所有外部MCP客户端
if a.externalMCPMgr != nil {
a.externalMCPMgr.StopAll()
@@ -465,6 +493,40 @@ func (a *App) Shutdown() {
}
}
// startRobotConnections 根据当前配置启动钉钉/飞书长连接(不先关闭已有连接,仅用于首次启动)
func (a *App) startRobotConnections() {
a.robotMu.Lock()
defer a.robotMu.Unlock()
cfg := a.config
if cfg.Robots.Lark.Enabled && cfg.Robots.Lark.AppID != "" && cfg.Robots.Lark.AppSecret != "" {
ctx, cancel := context.WithCancel(context.Background())
a.larkCancel = cancel
go robot.StartLark(ctx, cfg.Robots.Lark, a.robotHandler, a.logger.Logger)
}
if cfg.Robots.Dingtalk.Enabled && cfg.Robots.Dingtalk.ClientID != "" && cfg.Robots.Dingtalk.ClientSecret != "" {
ctx, cancel := context.WithCancel(context.Background())
a.dingCancel = cancel
go robot.StartDing(ctx, cfg.Robots.Dingtalk, a.robotHandler, a.logger.Logger)
}
}
// RestartRobotConnections 重启钉钉/飞书长连接,使前端应用配置后立即生效(实现 handler.RobotRestarter
func (a *App) RestartRobotConnections() {
a.robotMu.Lock()
if a.dingCancel != nil {
a.dingCancel()
a.dingCancel = nil
}
if a.larkCancel != nil {
a.larkCancel()
a.larkCancel = nil
}
a.robotMu.Unlock()
// 给旧 goroutine 一点时间退出
time.Sleep(200 * time.Millisecond)
a.startRobotConnections()
}
// setupRoutes 设置路由
func setupRoutes(
router *gin.Engine,
@@ -472,6 +534,7 @@ func setupRoutes(
agentHandler *handler.AgentHandler,
monitorHandler *handler.MonitorHandler,
conversationHandler *handler.ConversationHandler,
robotHandler *handler.RobotHandler,
groupHandler *handler.GroupHandler,
configHandler *handler.ConfigHandler,
externalMCPHandler *handler.ExternalMCPHandler,
@@ -481,6 +544,7 @@ func setupRoutes(
roleHandler *handler.RoleHandler,
skillsHandler *handler.SkillsHandler,
fofaHandler *handler.FofaHandler,
terminalHandler *handler.TerminalHandler,
mcpServer *mcp.Server,
authManager *security.AuthManager,
openAPIHandler *handler.OpenAPIHandler,
@@ -497,9 +561,18 @@ func setupRoutes(
authRoutes.GET("/validate", security.AuthMiddleware(authManager), authHandler.Validate)
}
// 机器人回调(无需登录,供企业微信/钉钉/飞书服务器调用)
api.GET("/robot/wecom", robotHandler.HandleWecomGET)
api.POST("/robot/wecom", robotHandler.HandleWecomPOST)
api.POST("/robot/dingtalk", robotHandler.HandleDingtalkPOST)
api.POST("/robot/lark", robotHandler.HandleLarkPOST)
protected := api.Group("")
protected.Use(security.AuthMiddleware(authManager))
{
// 机器人测试(需登录):POST /api/robot/testbody: {"platform":"dingtalk","user_id":"test","text":"帮助"},用于验证机器人逻辑
protected.POST("/robot/test", robotHandler.HandleRobotTest)
// Agent Loop
protected.POST("/agent-loop", agentHandler.AgentLoop)
// Agent Loop 流式输出
@@ -511,6 +584,8 @@ func setupRoutes(
// 信息收集 - FOFA 查询(后端代理)
protected.POST("/fofa/search", fofaHandler.Search)
// 信息收集 - 自然语言解析为 FOFA 语法(需人工确认后再查询)
protected.POST("/fofa/parse", fofaHandler.ParseNaturalLanguage)
// 批量任务管理
protected.POST("/batch-tasks", agentHandler.CreateBatchQueue)
@@ -556,6 +631,11 @@ func setupRoutes(
protected.PUT("/config", configHandler.UpdateConfig)
protected.POST("/config/apply", configHandler.ApplyConfig)
// 系统设置 - 终端(执行命令,提高运维效率)
protected.POST("/terminal/run", terminalHandler.RunCommand)
protected.POST("/terminal/run/stream", terminalHandler.RunCommandStream)
protected.GET("/terminal/ws", terminalHandler.RunCommandWS)
// 外部MCP管理
protected.GET("/external-mcp", externalMCPHandler.GetExternalMCPs)
protected.GET("/external-mcp/stats", externalMCPHandler.GetExternalMCPStats)
@@ -1022,7 +1102,7 @@ func initializeKnowledge(
knowledgeRetriever := knowledge.NewRetriever(knowledgeDB, embedder, retrievalConfig, logger)
// 创建索引器
knowledgeIndexer := knowledge.NewIndexer(knowledgeDB, embedder, logger)
knowledgeIndexer := knowledge.NewIndexer(knowledgeDB, embedder, logger, &cfg.Knowledge.Indexing)
// 注册知识检索工具到MCP服务器
knowledge.RegisterKnowledgeTool(mcpServer, knowledgeRetriever, knowledgeManager, logger)
+53
View File
@@ -25,11 +25,44 @@ type Config struct {
Auth AuthConfig `yaml:"auth"`
ExternalMCP ExternalMCPConfig `yaml:"external_mcp,omitempty"`
Knowledge KnowledgeConfig `yaml:"knowledge,omitempty"`
Robots RobotsConfig `yaml:"robots,omitempty" json:"robots,omitempty"` // 企业微信/钉钉/飞书等机器人配置
RolesDir string `yaml:"roles_dir,omitempty" json:"roles_dir,omitempty"` // 角色配置文件目录(新方式)
Roles map[string]RoleConfig `yaml:"roles,omitempty" json:"roles,omitempty"` // 向后兼容:支持在主配置文件中定义角色
SkillsDir string `yaml:"skills_dir,omitempty" json:"skills_dir,omitempty"` // Skills配置文件目录
}
// RobotsConfig 机器人配置(企业微信、钉钉、飞书等)
type RobotsConfig struct {
Wecom RobotWecomConfig `yaml:"wecom,omitempty" json:"wecom,omitempty"` // 企业微信
Dingtalk RobotDingtalkConfig `yaml:"dingtalk,omitempty" json:"dingtalk,omitempty"` // 钉钉
Lark RobotLarkConfig `yaml:"lark,omitempty" json:"lark,omitempty"` // 飞书
}
// RobotWecomConfig 企业微信机器人配置
type RobotWecomConfig struct {
Enabled bool `yaml:"enabled" json:"enabled"`
Token string `yaml:"token" json:"token"` // 回调 URL 校验 Token
EncodingAESKey string `yaml:"encoding_aes_key" json:"encoding_aes_key"` // EncodingAESKey
CorpID string `yaml:"corp_id" json:"corp_id"` // 企业 ID
Secret string `yaml:"secret" json:"secret"` // 应用 Secret
AgentID int64 `yaml:"agent_id" json:"agent_id"` // 应用 AgentId
}
// RobotDingtalkConfig 钉钉机器人配置
type RobotDingtalkConfig struct {
Enabled bool `yaml:"enabled" json:"enabled"`
ClientID string `yaml:"client_id" json:"client_id"` // 应用 Key (AppKey)
ClientSecret string `yaml:"client_secret" json:"client_secret"` // 应用 Secret
}
// RobotLarkConfig 飞书机器人配置
type RobotLarkConfig struct {
Enabled bool `yaml:"enabled" json:"enabled"`
AppID string `yaml:"app_id" json:"app_id"` // 应用 App ID
AppSecret string `yaml:"app_secret" json:"app_secret"` // 应用 App Secret
VerifyToken string `yaml:"verify_token" json:"verify_token"` // 事件订阅 Verification Token(可选)
}
type ServerConfig struct {
Host string `yaml:"host"`
Port int `yaml:"port"`
@@ -562,6 +595,26 @@ type KnowledgeConfig struct {
BasePath string `yaml:"base_path" json:"base_path"` // 知识库路径
Embedding EmbeddingConfig `yaml:"embedding" json:"embedding"`
Retrieval RetrievalConfig `yaml:"retrieval" json:"retrieval"`
Indexing IndexingConfig `yaml:"indexing,omitempty" json:"indexing,omitempty"` // 索引构建配置
}
// IndexingConfig 索引构建配置(用于控制知识库索引构建时的行为)
type IndexingConfig struct {
// 分块配置
ChunkSize int `yaml:"chunk_size,omitempty" json:"chunk_size,omitempty"` // 每个块的最大 token 数(估算),默认 512
ChunkOverlap int `yaml:"chunk_overlap,omitempty" json:"chunk_overlap,omitempty"` // 块之间的重叠 token 数,默认 50
MaxChunksPerItem int `yaml:"max_chunks_per_item,omitempty" json:"max_chunks_per_item,omitempty"` // 单个知识项的最大块数量,0 表示不限制
// 速率限制配置(用于避免 API 速率限制)
RateLimitDelayMs int `yaml:"rate_limit_delay_ms,omitempty" json:"rate_limit_delay_ms,omitempty"` // 请求间隔时间(毫秒),0 表示不使用固定延迟
MaxRPM int `yaml:"max_rpm,omitempty" json:"max_rpm,omitempty"` // 每分钟最大请求数,0 表示不限制
// 重试配置(用于处理临时错误)
MaxRetries int `yaml:"max_retries,omitempty" json:"max_retries,omitempty"` // 最大重试次数,默认 3
RetryDelayMs int `yaml:"retry_delay_ms,omitempty" json:"retry_delay_ms,omitempty"` // 重试间隔(毫秒),默认 1000
// 批处理配置(用于批量嵌入,当前未使用,保留扩展)
BatchSize int `yaml:"batch_size,omitempty" json:"batch_size,omitempty"` // 批量处理大小,0 表示逐个处理
}
// EmbeddingConfig 嵌入配置
+259 -8
View File
@@ -2,10 +2,14 @@ package handler
import (
"context"
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"time"
@@ -108,11 +112,132 @@ func (h *AgentHandler) SetSkillsManager(manager *skills.Manager) {
h.skillsManager = manager
}
// ChatAttachment 聊天附件(用户上传的文件)
type ChatAttachment struct {
FileName string `json:"fileName"` // 文件名
Content string `json:"content"` // 文本内容或 base64(由 MimeType 决定是否解码)
MimeType string `json:"mimeType,omitempty"`
}
// ChatRequest 聊天请求
type ChatRequest struct {
Message string `json:"message" binding:"required"`
ConversationID string `json:"conversationId,omitempty"`
Role string `json:"role,omitempty"` // 角色名称
Message string `json:"message" binding:"required"`
ConversationID string `json:"conversationId,omitempty"`
Role string `json:"role,omitempty"` // 角色名称
Attachments []ChatAttachment `json:"attachments,omitempty"`
}
const (
maxAttachments = 10
chatUploadsDirName = "chat_uploads" // 对话附件保存的根目录(相对当前工作目录)
)
// saveAttachmentsToDateAndConversationDir 将附件保存到 chat_uploads/YYYY-MM-DD/{conversationID}/,返回每个文件的保存路径(与 attachments 顺序一致)
// conversationID 为空时使用 "_new" 作为目录名(新对话尚未有 ID)
func saveAttachmentsToDateAndConversationDir(attachments []ChatAttachment, conversationID string, logger *zap.Logger) (savedPaths []string, err error) {
if len(attachments) == 0 {
return nil, nil
}
cwd, err := os.Getwd()
if err != nil {
return nil, fmt.Errorf("获取当前工作目录失败: %w", err)
}
dateDir := filepath.Join(cwd, chatUploadsDirName, time.Now().Format("2006-01-02"))
convDirName := strings.TrimSpace(conversationID)
if convDirName == "" {
convDirName = "_new"
} else {
convDirName = strings.ReplaceAll(convDirName, string(filepath.Separator), "_")
}
targetDir := filepath.Join(dateDir, convDirName)
if err = os.MkdirAll(targetDir, 0755); err != nil {
return nil, fmt.Errorf("创建上传目录失败: %w", err)
}
savedPaths = make([]string, 0, len(attachments))
for i, a := range attachments {
raw, decErr := attachmentContentToBytes(a)
if decErr != nil {
return nil, fmt.Errorf("附件 %s 解码失败: %w", a.FileName, decErr)
}
baseName := filepath.Base(a.FileName)
if baseName == "" || baseName == "." {
baseName = "file"
}
baseName = strings.ReplaceAll(baseName, string(filepath.Separator), "_")
ext := filepath.Ext(baseName)
nameNoExt := strings.TrimSuffix(baseName, ext)
suffix := fmt.Sprintf("_%s_%s", time.Now().Format("150405"), shortRand(6))
var unique string
if ext != "" {
unique = nameNoExt + suffix + ext
} else {
unique = baseName + suffix
}
fullPath := filepath.Join(targetDir, unique)
if err = os.WriteFile(fullPath, raw, 0644); err != nil {
return nil, fmt.Errorf("写入文件 %s 失败: %w", a.FileName, err)
}
absPath, _ := filepath.Abs(fullPath)
savedPaths = append(savedPaths, absPath)
if logger != nil {
logger.Debug("对话附件已保存", zap.Int("index", i+1), zap.String("fileName", a.FileName), zap.String("path", absPath))
}
}
return savedPaths, nil
}
func shortRand(n int) string {
const letters = "0123456789abcdef"
b := make([]byte, n)
_, _ = rand.Read(b)
for i := range b {
b[i] = letters[int(b[i])%len(letters)]
}
return string(b)
}
func attachmentContentToBytes(a ChatAttachment) ([]byte, error) {
content := a.Content
if decoded, err := base64.StdEncoding.DecodeString(content); err == nil && len(decoded) > 0 {
return decoded, nil
}
return []byte(content), nil
}
// userMessageContentForStorage 返回要存入数据库的用户消息内容:有附件时在正文后追加附件名(及路径),刷新后仍能显示,继续对话时大模型也能从历史中拿到路径
func userMessageContentForStorage(message string, attachments []ChatAttachment, savedPaths []string) string {
if len(attachments) == 0 {
return message
}
var b strings.Builder
b.WriteString(message)
for i, a := range attachments {
b.WriteString("\n📎 ")
b.WriteString(a.FileName)
if i < len(savedPaths) && savedPaths[i] != "" {
b.WriteString(": ")
b.WriteString(savedPaths[i])
}
}
return b.String()
}
// appendAttachmentsToMessage 仅将附件的保存路径追加到用户消息末尾,不再内联附件内容,避免上下文过长
func appendAttachmentsToMessage(msg string, attachments []ChatAttachment, savedPaths []string) string {
if len(attachments) == 0 {
return msg
}
var b strings.Builder
b.WriteString(msg)
b.WriteString("\n\n[用户上传的文件已保存到以下路径(请按需读取文件内容,而不是依赖内联内容)]\n")
for i, a := range attachments {
if i < len(savedPaths) && savedPaths[i] != "" {
b.WriteString(fmt.Sprintf("- %s: %s\n", a.FileName, savedPaths[i]))
} else {
b.WriteString(fmt.Sprintf("- %s: (路径未知,可能保存失败)\n", a.FileName))
}
}
return b.String()
}
// ChatResponse 聊天响应
@@ -181,6 +306,12 @@ func (h *AgentHandler) AgentLoop(c *gin.Context) {
h.logger.Info("从ReAct数据恢复历史上下文", zap.Int("count", len(agentHistoryMessages)))
}
// 校验附件数量(非流式)
if len(req.Attachments) > maxAttachments {
c.JSON(http.StatusBadRequest, gin.H{"error": fmt.Sprintf("附件最多 %d 个", maxAttachments)})
return
}
// 应用角色用户提示词和工具配置
finalMessage := req.Message
var roleTools []string // 角色配置的工具列表
@@ -206,9 +337,20 @@ func (h *AgentHandler) AgentLoop(c *gin.Context) {
}
}
}
var savedPaths []string
if len(req.Attachments) > 0 {
savedPaths, err = saveAttachmentsToDateAndConversationDir(req.Attachments, conversationID, h.logger)
if err != nil {
h.logger.Error("保存对话附件失败", zap.Error(err))
c.JSON(http.StatusInternalServerError, gin.H{"error": "保存上传文件失败: " + err.Error()})
return
}
}
finalMessage = appendAttachmentsToMessage(finalMessage, req.Attachments, savedPaths)
// 保存用户消息(保存原始消息,不包含角色提示词)
_, err = h.db.AddMessage(conversationID, "user", req.Message, nil)
// 保存用户消息:有附件时一并保存附件名与路径,刷新后显示、继续对话时大模型也能从历史中拿到路径
userContent := userMessageContentForStorage(req.Message, req.Attachments, savedPaths)
_, err = h.db.AddMessage(conversationID, "user", userContent, nil)
if err != nil {
h.logger.Error("保存用户消息失败", zap.Error(err))
c.JSON(http.StatusInternalServerError, gin.H{"error": "保存用户消息失败: " + err.Error()})
@@ -259,6 +401,96 @@ func (h *AgentHandler) AgentLoop(c *gin.Context) {
})
}
// ProcessMessageForRobot 供机器人(企业微信/钉钉/飞书)调用:与 /api/agent-loop/stream 相同执行路径(含 progressCallback、过程详情),仅不发送 SSE,最后返回完整回复
func (h *AgentHandler) ProcessMessageForRobot(ctx context.Context, conversationID, message, role string) (response string, convID string, err error) {
if conversationID == "" {
title := safeTruncateString(message, 50)
conv, createErr := h.db.CreateConversation(title)
if createErr != nil {
return "", "", fmt.Errorf("创建对话失败: %w", createErr)
}
conversationID = conv.ID
} else {
if _, getErr := h.db.GetConversation(conversationID); getErr != nil {
return "", "", fmt.Errorf("对话不存在")
}
}
agentHistoryMessages, err := h.loadHistoryFromReActData(conversationID)
if err != nil {
historyMessages, getErr := h.db.GetMessages(conversationID)
if getErr != nil {
agentHistoryMessages = []agent.ChatMessage{}
} else {
agentHistoryMessages = make([]agent.ChatMessage, 0, len(historyMessages))
for _, msg := range historyMessages {
agentHistoryMessages = append(agentHistoryMessages, agent.ChatMessage{Role: msg.Role, Content: msg.Content})
}
}
}
finalMessage := message
var roleTools, roleSkills []string
if role != "" && role != "默认" && h.config.Roles != nil {
if r, exists := h.config.Roles[role]; exists && r.Enabled {
if r.UserPrompt != "" {
finalMessage = r.UserPrompt + "\n\n" + message
}
roleTools = r.Tools
roleSkills = r.Skills
}
}
if _, err = h.db.AddMessage(conversationID, "user", message, nil); err != nil {
return "", "", fmt.Errorf("保存用户消息失败: %w", err)
}
// 与 agent-loop/stream 一致:先创建助手消息占位,用 progressCallback 写过程详情(不发送 SSE)
assistantMsg, err := h.db.AddMessage(conversationID, "assistant", "处理中...", nil)
if err != nil {
h.logger.Warn("机器人:创建助手消息占位失败", zap.Error(err))
}
var assistantMessageID string
if assistantMsg != nil {
assistantMessageID = assistantMsg.ID
}
progressCallback := h.createProgressCallback(conversationID, assistantMessageID, nil)
result, err := h.agent.AgentLoopWithProgress(ctx, finalMessage, agentHistoryMessages, conversationID, progressCallback, roleTools, roleSkills)
if err != nil {
errMsg := "执行失败: " + err.Error()
if assistantMessageID != "" {
_, _ = h.db.Exec("UPDATE messages SET content = ? WHERE id = ?", errMsg, assistantMessageID)
_ = h.db.AddProcessDetail(assistantMessageID, conversationID, "error", errMsg, nil)
}
return "", conversationID, err
}
// 更新助手消息内容与 MCP 执行 ID(与 stream 一致)
if assistantMessageID != "" {
mcpIDsJSON := ""
if len(result.MCPExecutionIDs) > 0 {
jsonData, _ := json.Marshal(result.MCPExecutionIDs)
mcpIDsJSON = string(jsonData)
}
_, err = h.db.Exec(
"UPDATE messages SET content = ?, mcp_execution_ids = ? WHERE id = ?",
result.Response, mcpIDsJSON, assistantMessageID,
)
if err != nil {
h.logger.Warn("机器人:更新助手消息失败", zap.Error(err))
}
} else {
if _, err = h.db.AddMessage(conversationID, "assistant", result.Response, result.MCPExecutionIDs); err != nil {
h.logger.Warn("机器人:保存助手消息失败", zap.Error(err))
}
}
if result.LastReActInput != "" || result.LastReActOutput != "" {
_ = h.db.SaveReActData(conversationID, result.LastReActInput, result.LastReActOutput)
}
return result.Response, conversationID, nil
}
// StreamEvent 流式事件
type StreamEvent struct {
Type string `json:"type"` // conversation, progress, tool_call, tool_result, response, error, cancelled, done
@@ -528,6 +760,12 @@ func (h *AgentHandler) AgentLoopStream(c *gin.Context) {
h.logger.Info("从ReAct数据恢复历史上下文", zap.Int("count", len(agentHistoryMessages)))
}
// 校验附件数量
if len(req.Attachments) > maxAttachments {
sendEvent("error", fmt.Sprintf("附件最多 %d 个", maxAttachments), nil)
return
}
// 应用角色用户提示词和工具配置
finalMessage := req.Message
var roleTools []string // 角色配置的工具列表
@@ -555,10 +793,22 @@ func (h *AgentHandler) AgentLoopStream(c *gin.Context) {
}
}
}
var savedPaths []string
if len(req.Attachments) > 0 {
savedPaths, err = saveAttachmentsToDateAndConversationDir(req.Attachments, conversationID, h.logger)
if err != nil {
h.logger.Error("保存对话附件失败", zap.Error(err))
sendEvent("error", "保存上传文件失败: "+err.Error(), nil)
return
}
}
// 仅将附件保存路径追加到 finalMessage,避免将文件内容内联到大模型上下文中
finalMessage = appendAttachmentsToMessage(finalMessage, req.Attachments, savedPaths)
// 如果roleTools为空,表示使用所有工具(默认角色或未配置工具的角色)
// 保存用户消息(保存原始消息,不包含角色提示词)
_, err = h.db.AddMessage(conversationID, "user", req.Message, nil)
// 保存用户消息:有附件时一并保存附件名与路径,刷新后显示、继续对话时大模型也能从历史中拿到路径
userContent := userMessageContentForStorage(req.Message, req.Attachments, savedPaths)
_, err = h.db.AddMessage(conversationID, "user", userContent, nil)
if err != nil {
h.logger.Error("保存用户消息失败", zap.Error(err))
}
@@ -1194,7 +1444,8 @@ func (h *AgentHandler) executeBatchQueue(queueID string) {
// 执行任务(使用包含角色提示词的finalMessage和角色工具列表)
h.logger.Info("执行批量任务", zap.String("queueId", queueID), zap.String("taskId", task.ID), zap.String("message", task.Message), zap.String("role", queue.Role), zap.String("conversationId", conversationID))
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
// 单个子任务超时时间:从30分钟调整为6小时,适配长时间渗透/扫描任务
ctx, cancel := context.WithTimeout(context.Background(), 6*time.Hour)
// 存储取消函数,以便在取消队列时能够取消当前任务
h.batchTaskManager.SetTaskCancel(queueID, cancel)
// 使用队列配置的角色工具列表(如果为空,表示使用所有工具)
+76
View File
@@ -44,6 +44,11 @@ type AppUpdater interface {
UpdateKnowledgeComponents(handler *KnowledgeHandler, manager interface{}, retriever interface{}, indexer interface{})
}
// RobotRestarter 机器人连接重启器(用于配置应用后重启钉钉/飞书长连接)
type RobotRestarter interface {
RestartRobotConnections()
}
// ConfigHandler 配置处理器
type ConfigHandler struct {
configPath string
@@ -59,6 +64,7 @@ type ConfigHandler struct {
retrieverUpdater RetrieverUpdater // 检索器更新器(可选)
knowledgeInitializer KnowledgeInitializer // 知识库初始化器(可选)
appUpdater AppUpdater // App更新器(可选)
robotRestarter RobotRestarter // 机器人连接重启器(可选),ApplyConfig 时重启钉钉/飞书
logger *zap.Logger
mu sync.RWMutex
lastEmbeddingConfig *config.EmbeddingConfig // 上一次的嵌入模型配置(用于检测变更)
@@ -142,6 +148,13 @@ func (h *ConfigHandler) SetAppUpdater(updater AppUpdater) {
h.appUpdater = updater
}
// SetRobotRestarter 设置机器人连接重启器(ApplyConfig 时用于重启钉钉/飞书长连接)
func (h *ConfigHandler) SetRobotRestarter(restarter RobotRestarter) {
h.mu.Lock()
defer h.mu.Unlock()
h.robotRestarter = restarter
}
// GetConfigResponse 获取配置响应
type GetConfigResponse struct {
OpenAI config.OpenAIConfig `json:"openai"`
@@ -150,6 +163,7 @@ type GetConfigResponse struct {
Tools []ToolConfigInfo `json:"tools"`
Agent config.AgentConfig `json:"agent"`
Knowledge config.KnowledgeConfig `json:"knowledge"`
Robots config.RobotsConfig `json:"robots,omitempty"`
}
// ToolConfigInfo 工具配置信息
@@ -222,6 +236,7 @@ func (h *ConfigHandler) GetConfig(c *gin.Context) {
Tools: tools,
Agent: h.config.Agent,
Knowledge: h.config.Knowledge,
Robots: h.config.Robots,
})
}
@@ -479,6 +494,7 @@ type UpdateConfigRequest struct {
Tools []ToolEnableStatus `json:"tools,omitempty"`
Agent *config.AgentConfig `json:"agent,omitempty"`
Knowledge *config.KnowledgeConfig `json:"knowledge,omitempty"`
Robots *config.RobotsConfig `json:"robots,omitempty"`
}
// ToolEnableStatus 工具启用状态
@@ -555,6 +571,16 @@ func (h *ConfigHandler) UpdateConfig(c *gin.Context) {
)
}
// 更新机器人配置
if req.Robots != nil {
h.config.Robots = *req.Robots
h.logger.Info("更新机器人配置",
zap.Bool("wecom_enabled", h.config.Robots.Wecom.Enabled),
zap.Bool("dingtalk_enabled", h.config.Robots.Dingtalk.Enabled),
zap.Bool("lark_enabled", h.config.Robots.Lark.Enabled),
)
}
// 更新工具启用状态
if req.Tools != nil {
// 分离内部工具和外部工具
@@ -824,6 +850,12 @@ func (h *ConfigHandler) ApplyConfig(c *gin.Context) {
}
}
// 重启钉钉/飞书长连接,使前端修改的机器人配置立即生效(无需重启服务)
if h.robotRestarter != nil {
h.robotRestarter.RestartRobotConnections()
h.logger.Info("已触发机器人连接重启(钉钉/飞书)")
}
h.logger.Info("配置已应用",
zap.Int("tools_count", len(h.config.Security.Tools)),
)
@@ -854,7 +886,9 @@ func (h *ConfigHandler) saveConfig() error {
updateAgentConfig(root, h.config.Agent.MaxIterations)
updateMCPConfig(root, h.config.MCP)
updateOpenAIConfig(root, h.config.OpenAI)
updateFOFAConfig(root, h.config.FOFA)
updateKnowledgeConfig(root, h.config.Knowledge)
updateRobotsConfig(root, h.config.Robots)
// 更新外部MCP配置(使用external_mcp.go中的函数,同一包中可直接调用)
// 读取原始配置以保持向后兼容
originalConfigs := make(map[string]map[string]bool)
@@ -998,6 +1032,14 @@ func updateOpenAIConfig(doc *yaml.Node, cfg config.OpenAIConfig) {
setStringInMap(openaiNode, "model", cfg.Model)
}
func updateFOFAConfig(doc *yaml.Node, cfg config.FofaConfig) {
root := doc.Content[0]
fofaNode := ensureMap(root, "fofa")
setStringInMap(fofaNode, "base_url", cfg.BaseURL)
setStringInMap(fofaNode, "email", cfg.Email)
setStringInMap(fofaNode, "api_key", cfg.APIKey)
}
func updateKnowledgeConfig(doc *yaml.Node, cfg config.KnowledgeConfig) {
root := doc.Content[0]
knowledgeNode := ensureMap(root, "knowledge")
@@ -1020,6 +1062,40 @@ func updateKnowledgeConfig(doc *yaml.Node, cfg config.KnowledgeConfig) {
setIntInMap(retrievalNode, "top_k", cfg.Retrieval.TopK)
setFloatInMap(retrievalNode, "similarity_threshold", cfg.Retrieval.SimilarityThreshold)
setFloatInMap(retrievalNode, "hybrid_weight", cfg.Retrieval.HybridWeight)
// 更新索引配置
indexingNode := ensureMap(knowledgeNode, "indexing")
setIntInMap(indexingNode, "chunk_size", cfg.Indexing.ChunkSize)
setIntInMap(indexingNode, "chunk_overlap", cfg.Indexing.ChunkOverlap)
setIntInMap(indexingNode, "max_chunks_per_item", cfg.Indexing.MaxChunksPerItem)
setIntInMap(indexingNode, "max_rpm", cfg.Indexing.MaxRPM)
setIntInMap(indexingNode, "rate_limit_delay_ms", cfg.Indexing.RateLimitDelayMs)
setIntInMap(indexingNode, "max_retries", cfg.Indexing.MaxRetries)
setIntInMap(indexingNode, "retry_delay_ms", cfg.Indexing.RetryDelayMs)
}
func updateRobotsConfig(doc *yaml.Node, cfg config.RobotsConfig) {
root := doc.Content[0]
robotsNode := ensureMap(root, "robots")
wecomNode := ensureMap(robotsNode, "wecom")
setBoolInMap(wecomNode, "enabled", cfg.Wecom.Enabled)
setStringInMap(wecomNode, "token", cfg.Wecom.Token)
setStringInMap(wecomNode, "encoding_aes_key", cfg.Wecom.EncodingAESKey)
setStringInMap(wecomNode, "corp_id", cfg.Wecom.CorpID)
setStringInMap(wecomNode, "secret", cfg.Wecom.Secret)
setIntInMap(wecomNode, "agent_id", int(cfg.Wecom.AgentID))
dingtalkNode := ensureMap(robotsNode, "dingtalk")
setBoolInMap(dingtalkNode, "enabled", cfg.Dingtalk.Enabled)
setStringInMap(dingtalkNode, "client_id", cfg.Dingtalk.ClientID)
setStringInMap(dingtalkNode, "client_secret", cfg.Dingtalk.ClientSecret)
larkNode := ensureMap(robotsNode, "lark")
setBoolInMap(larkNode, "enabled", cfg.Lark.Enabled)
setStringInMap(larkNode, "app_id", cfg.Lark.AppID)
setStringInMap(larkNode, "app_secret", cfg.Lark.AppSecret)
setStringInMap(larkNode, "verify_token", cfg.Lark.VerifyToken)
}
func ensureMap(parent *yaml.Node, path ...string) *yaml.Node {
+250 -6
View File
@@ -1,8 +1,10 @@
package handler
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
@@ -11,22 +13,31 @@ import (
"time"
"cyberstrike-ai/internal/config"
openaiClient "cyberstrike-ai/internal/openai"
"github.com/gin-gonic/gin"
"go.uber.org/zap"
)
type FofaHandler struct {
cfg *config.Config
logger *zap.Logger
client *http.Client
cfg *config.Config
logger *zap.Logger
client *http.Client
openAIClient *openaiClient.Client
}
func NewFofaHandler(cfg *config.Config, logger *zap.Logger) *FofaHandler {
// LLM 请求通常比 FOFA 查询更慢一点,单独给一个更宽松的超时。
llmHTTPClient := &http.Client{Timeout: 2 * time.Minute}
var llmCfg *config.OpenAIConfig
if cfg != nil {
llmCfg = &cfg.OpenAI
}
return &FofaHandler{
cfg: cfg,
logger: logger,
client: &http.Client{Timeout: 30 * time.Second},
cfg: cfg,
logger: logger,
client: &http.Client{Timeout: 30 * time.Second},
openAIClient: openaiClient.NewClient(llmCfg, llmHTTPClient, logger),
}
}
@@ -38,6 +49,16 @@ type fofaSearchRequest struct {
Full bool `json:"full,omitempty"`
}
type fofaParseRequest struct {
Text string `json:"text" binding:"required"`
}
type fofaParseResponse struct {
Query string `json:"query"`
Explanation string `json:"explanation,omitempty"`
Warnings []string `json:"warnings,omitempty"`
}
type fofaAPIResponse struct {
Error bool `json:"error"`
ErrMsg string `json:"errmsg"`
@@ -86,6 +107,229 @@ func (h *FofaHandler) resolveBaseURL() string {
return "https://fofa.info/api/v1/search/all"
}
// ParseNaturalLanguage 将自然语言解析为 FOFA 查询语法(仅生成,不执行查询)
func (h *FofaHandler) ParseNaturalLanguage(c *gin.Context) {
var req fofaParseRequest
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "无效的请求参数: " + err.Error()})
return
}
req.Text = strings.TrimSpace(req.Text)
if req.Text == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "text 不能为空"})
return
}
if h.cfg == nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "系统配置未初始化"})
return
}
if strings.TrimSpace(h.cfg.OpenAI.APIKey) == "" || strings.TrimSpace(h.cfg.OpenAI.Model) == "" {
c.JSON(http.StatusBadRequest, gin.H{
"error": "未配置 AI 模型:请在系统设置中填写 openai.api_key 与 openai.model(支持 OpenAI 兼容 API,如 DeepSeek",
"need": []string{"openai.api_key", "openai.model"},
})
return
}
if h.openAIClient == nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "AI 客户端未初始化"})
return
}
systemPrompt := strings.TrimSpace(`
你是FOFA 查询语法生成器任务把用户输入的自然语言搜索意图转换成 FOFA 查询语法
输出要求非常重要
1) 只输出 JSON不要 markdown不要代码块不要额外解释文本
2) JSON 结构必须是
{
"query": "stringFOFA查询语法(可直接粘贴到 FOFA 或本系统查询框)",
"explanation": "string,可选,解释你如何映射字段/逻辑",
"warnings": ["string"...] 可选列出歧义/风险/需要人工确认的点
}
3) 如果用户输入本身已经是 FOFA 查询语法或非常接近 FOFA 语法的表达式应当原样返回 query
- 不要擅自改写字段名操作符括号结构
- 不要改写任何字符串值尤其是地理位置类值不要做缩写/同义词替换/翻译/音译
查询语法要点来自 FOFA 语法参考
- 逻辑连接符&&||必要时用 () 包住子表达式以确认优先级括号优先级最高
- 当同一层级同时出现 && ||混用 () 明确优先级避免歧义
- 比较/匹配
- = 匹配当字段="" 可查询不存在该字段值为空的情况
- == 完全匹配当字段=="" 可查询字段存在且值为空的情况
- != 不匹配当字段!="" 可查询值不为空的情况
- *= 模糊匹配可使用 * ? 进行搜索
- 直接输入关键词不带字段会在标题HTML内容HTTP头URL字段中搜索但当意图明确时优先用字段表达更可控更准确
字段示例速查来自用户提供的案例可直接套用/拼接
- 高级搜索操作符示例
- title="beijing" = 匹配
- title=="" == 完全匹配字段存在且值为空
- title="" = 匹配可能表示字段不存在或值为空
- title!="" != 不匹配可用于值不为空
- title*="*Home*" *= 模糊匹配 * ?
- (app="Apache" || app="Nginx") && country="CN" 混用 && / || 时用括号
- 基础类General
- ip="1.1.1.1"
- ip="220.181.111.1/24"
- ip="2600:9000:202a:2600:18:4ab7:f600:93a1"
- port="6379"
- domain="qq.com"
- host=".fofa.info"
- os="centos"
- server="Microsoft-IIS/10"
- asn="19551"
- org="LLC Baxet"
- is_domain=true / is_domain=false
- is_ipv6=true / is_ipv6=false
- 标记类Special Label
- app="Microsoft-Exchange"
- fid="sSXXGNUO2FefBTcCLIT/2Q=="
- product="NGINX"
- product="Roundcube-Webmail" && product.version="1.6.10"
- category="服务"
- type="service" / type="subdomain"
- cloud_name="Aliyundun"
- is_cloud=true / is_cloud=false
- is_fraud=true / is_fraud=false
- is_honeypot=true / is_honeypot=false
- 协议类type=service
- protocol="quic"
- banner="users"
- banner_hash="7330105010150477363"
- banner_fid="zRpqmn0FXQRjZpH8MjMX55zpMy9SgsW8"
- base_protocol="udp" / base_protocol="tcp"
- 网站类type=subdomain
- title="beijing"
- header="elastic"
- header_hash="1258854265"
- body="网络空间测绘"
- body_hash="-2090962452"
- js_name="js/jquery.js"
- js_md5="82ac3f14327a8b7ba49baa208d4eaa15"
- cname="customers.spektrix.com"
- cname_domain="siteforce.com"
- icon_hash="-247388890"
- status_code="402"
- icp="京ICP证030173号"
- sdk_hash="Are3qNnP2Eqn7q5kAoUO3l+w3mgVIytO"
- 地理位置Location
- country="CN" country="中国"
- region="Zhejiang" region="浙江"仅支持中国地区中文
- city="Hangzhou"
- 证书类Certificate
- cert="baidu"
- cert.subject="Oracle Corporation"
- cert.issuer="DigiCert"
- cert.subject.org="Oracle Corporation"
- cert.subject.cn="baidu.com"
- cert.issuer.org="cPanel, Inc."
- cert.issuer.cn="Synology Inc. CA"
- cert.domain="huawei.com"
- cert.is_equal=true / cert.is_equal=false
- cert.is_valid=true / cert.is_valid=false
- cert.is_match=true / cert.is_match=false
- cert.is_expired=true / cert.is_expired=false
- jarm="2ad2ad0002ad2ad22c2ad2ad2ad2ad2eac92ec34bcc0cf7520e97547f83e81"
- tls.version="TLS 1.3"
- tls.ja3s="15af977ce25de452b96affa2addb1036"
- cert.sn="356078156165546797850343536942784588840297"
- cert.not_after.after="2025-03-01" / cert.not_after.before="2025-03-01"
- cert.not_before.after="2025-03-01" / cert.not_before.before="2025-03-01"
- 时间类Last update time
- after="2023-01-01"
- before="2023-12-01"
- after="2023-01-01" && before="2023-12-01"
- 独立IP语法需配合 ip_filter / ip_exclude
- ip_filter(banner="SSH-2.0-OpenSSH_6.7p2") && ip_filter(icon_hash="-1057022626")
- ip_filter(banner="SSH-2.0-OpenSSH_6.7p2" && asn="3462") && ip_exclude(title="EdgeOS")
- port_size="6" / port_size_gt="6" / port_size_lt="12"
- ip_ports="80,161"
- ip_country="CN"
- ip_region="Zhejiang"
- ip_city="Hangzhou"
- ip_after="2021-03-18"
- ip_before="2019-09-09"
生成约束与注意事项
- 字符串值一律用英文双引号包裹例如 title="登录"country="CN"
- 字符串值保持字面一致不要缩写例如 city="beijing" 不要变成 city="BJ"不要用别名例如 Beijing/Peking不要擅自翻译/音译/改写大小写
- 地理位置字段country/region/city更倾向于按用户给定值输出不确定合法取值时不要猜测把备选写进 warnings
- 不要捏造不存在的 FOFA 字段不确定时把不确定点写进 warnings并输出一个保守的 query
- 当用户描述里有多个与/或条件优先加 () 明确优先级例如(app="Apache" || app="Nginx") && country="CN"
- 当用户缺少关键条件导致范围过大或歧义如地点/协议/端口/服务类型未说明允许 query 为空字符串并在 warnings 里明确需要补充的信息
`)
userPrompt := fmt.Sprintf("自然语言意图:%s", req.Text)
requestBody := map[string]interface{}{
"model": h.cfg.OpenAI.Model,
"messages": []map[string]interface{}{
{"role": "system", "content": systemPrompt},
{"role": "user", "content": userPrompt},
},
"temperature": 0.1,
"max_tokens": 1200,
}
// OpenAI 返回结构:只需要 choices[0].message.content
var apiResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
}
ctx, cancel := context.WithTimeout(c.Request.Context(), 90*time.Second)
defer cancel()
if err := h.openAIClient.ChatCompletion(ctx, requestBody, &apiResponse); err != nil {
var apiErr *openaiClient.APIError
if errors.As(err, &apiErr) {
h.logger.Warn("FOFA自然语言解析:LLM返回错误", zap.Int("status", apiErr.StatusCode))
c.JSON(http.StatusBadGateway, gin.H{"error": "AI 解析失败(上游返回非 200),请检查模型配置或稍后重试"})
return
}
c.JSON(http.StatusBadGateway, gin.H{"error": "AI 解析失败: " + err.Error()})
return
}
if len(apiResponse.Choices) == 0 {
c.JSON(http.StatusBadGateway, gin.H{"error": "AI 未返回有效结果"})
return
}
content := strings.TrimSpace(apiResponse.Choices[0].Message.Content)
// 兼容模型偶尔返回 ```json ... ``` 的情况
content = strings.TrimPrefix(content, "```json")
content = strings.TrimPrefix(content, "```")
content = strings.TrimSuffix(content, "```")
content = strings.TrimSpace(content)
var parsed fofaParseResponse
if err := json.Unmarshal([]byte(content), &parsed); err != nil {
// 直接回传一部分原文,方便排查,但避免太大
snippet := content
if len(snippet) > 1200 {
snippet = snippet[:1200]
}
c.JSON(http.StatusBadGateway, gin.H{
"error": "AI 返回内容无法解析为 JSON,请稍后重试或换个描述方式",
"snippet": snippet,
})
return
}
parsed.Query = strings.TrimSpace(parsed.Query)
if parsed.Query == "" {
// query 允许为空(表示需求不明确),但前端需要明确提示
if len(parsed.Warnings) == 0 {
parsed.Warnings = []string{"需求信息不足,未能生成可用的 FOFA 查询语法,请补充关键条件(如国家/端口/产品/域名等)。"}
}
}
c.JSON(http.StatusOK, parsed)
}
// Search FOFA 查询(后端代理,避免前端暴露 key)
func (h *FofaHandler) Search(c *gin.Context) {
var req fofaSearchRequest
+32 -10
View File
@@ -75,7 +75,7 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
groupedByCategory[cat] = append(groupedByCategory[cat], item)
}
// 转换为CategoryWithItems格式
// 转换为 CategoryWithItems 格式
categoriesWithItems := make([]*knowledge.CategoryWithItems, 0, len(groupedByCategory))
for cat, catItems := range groupedByCategory {
categoriesWithItems = append(categoriesWithItems, &knowledge.CategoryWithItems{
@@ -107,7 +107,7 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
categoryPageMode := c.Query("categoryPage") != "false" // 默认使用分类分页
// 分页参数
limit := 50 // 默认每页50条(分类分页时为分类数,项分页时为项数)
limit := 50 // 默认每页 50 条(分类分页时为分类数,项分页时为项数)
offset := 0
if limitStr := c.Query("limit"); limitStr != "" {
if parsed, err := parseInt(limitStr); err == nil && parsed > 0 && parsed <= 500 {
@@ -120,7 +120,7 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
}
}
// 如果指定了category参数,且使用分类分页模式,则只返回该分类
// 如果指定了 category 参数,且使用分类分页模式,则只返回该分类
if category != "" && categoryPageMode {
// 单分类模式:返回该分类的所有知识项(不分页)
items, total, err := h.manager.GetItemsSummary(category, 0, 0)
@@ -150,9 +150,9 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
if categoryPageMode {
// 按分类分页模式(默认)
// limit表示每页分类数,推荐5-10个分类
// limit 表示每页分类数,推荐 5-10 个分类
if limit <= 0 || limit > 100 {
limit = 10 // 默认每页10个分类
limit = 10 // 默认每页 10 个分类
}
categoriesWithItems, totalCategories, err := h.manager.GetCategoriesWithItems(limit, offset)
@@ -172,7 +172,7 @@ func (h *KnowledgeHandler) GetItems(c *gin.Context) {
}
// 按项分页模式(向后兼容)
// 是否包含完整内容(默认false,只返回摘要)
// 是否包含完整内容(默认 false,只返回摘要)
includeContent := c.Query("includeContent") == "true"
if includeContent {
@@ -358,7 +358,7 @@ func (h *KnowledgeHandler) ScanKnowledgeBase(c *gin.Context) {
)
}
// 如果连续失败2次,立即停止增量索引
// 如果连续失败 2 次,立即停止增量索引
if consecutiveFailures >= 2 {
h.logger.Error("连续索引失败次数过多,立即停止增量索引",
zap.Int("consecutiveFailures", consecutiveFailures),
@@ -397,7 +397,7 @@ func (h *KnowledgeHandler) ScanKnowledgeBase(c *gin.Context) {
func (h *KnowledgeHandler) GetRetrievalLogs(c *gin.Context) {
conversationID := c.Query("conversationId")
messageID := c.Query("messageId")
limit := 50 // 默认50条
limit := 50 // 默认 50
if limitStr := c.Query("limit"); limitStr != "" {
if parsed, err := parseInt(limitStr); err == nil && parsed > 0 {
@@ -441,18 +441,40 @@ func (h *KnowledgeHandler) GetIndexStatus(c *gin.Context) {
if h.indexer != nil {
lastError, lastErrorTime := h.indexer.GetLastError()
if lastError != "" {
// 如果错误是最近发生的(5分钟内),则返回错误信息
// 如果错误是最近发生的(5 分钟内),则返回错误信息
if time.Since(lastErrorTime) < 5*time.Minute {
status["last_error"] = lastError
status["last_error_time"] = lastErrorTime.Format(time.RFC3339)
}
}
// 获取重建索引状态
isRebuilding, totalItems, current, failed, lastItemID, lastChunks, startTime := h.indexer.GetRebuildStatus()
if isRebuilding {
status["is_rebuilding"] = true
status["rebuild_total"] = totalItems
status["rebuild_current"] = current
status["rebuild_failed"] = failed
status["rebuild_start_time"] = startTime.Format(time.RFC3339)
if lastItemID != "" {
status["rebuild_last_item_id"] = lastItemID
}
if lastChunks > 0 {
status["rebuild_last_chunks"] = lastChunks
}
// 重建中时,is_complete 为 false
status["is_complete"] = false
// 计算重建进度百分比
if totalItems > 0 {
status["progress_percent"] = float64(current) / float64(totalItems) * 100
}
}
}
c.JSON(http.StatusOK, status)
}
// Search 搜索知识库(用于API调用,Agent内部使用Retriever
// Search 搜索知识库(用于 API 调用,Agent 内部使用 Retriever
func (h *KnowledgeHandler) Search(c *gin.Context) {
var req knowledge.SearchRequest
if err := c.ShouldBindJSON(&req); err != nil {
+593
View File
@@ -0,0 +1,593 @@
package handler
import (
"context"
"crypto/aes"
"crypto/cipher"
"encoding/base64"
"encoding/binary"
"encoding/xml"
"errors"
"fmt"
"io"
"net/http"
"sort"
"strings"
"sync"
"time"
"cyberstrike-ai/internal/config"
"cyberstrike-ai/internal/database"
"github.com/gin-gonic/gin"
"go.uber.org/zap"
)
const (
robotCmdHelp = "帮助"
robotCmdList = "列表"
robotCmdListAlt = "对话列表"
robotCmdSwitch = "切换"
robotCmdContinue = "继续"
robotCmdNew = "新对话"
robotCmdClear = "清空"
robotCmdCurrent = "当前"
robotCmdStop = "停止"
robotCmdRoles = "角色"
robotCmdRolesList = "角色列表"
robotCmdSwitchRole = "切换角色"
robotCmdDelete = "删除"
robotCmdVersion = "版本"
)
// RobotHandler 企业微信/钉钉/飞书等机器人回调处理
type RobotHandler struct {
config *config.Config
db *database.DB
agentHandler *AgentHandler
logger *zap.Logger
mu sync.RWMutex
sessions map[string]string // key: "platform_userID", value: conversationID
sessionRoles map[string]string // key: "platform_userID", value: roleName(默认"默认"
cancelMu sync.Mutex // 保护 runningCancels
runningCancels map[string]context.CancelFunc // key: "platform_userID", 用于停止命令中断任务
}
// NewRobotHandler 创建机器人处理器
func NewRobotHandler(cfg *config.Config, db *database.DB, agentHandler *AgentHandler, logger *zap.Logger) *RobotHandler {
return &RobotHandler{
config: cfg,
db: db,
agentHandler: agentHandler,
logger: logger,
sessions: make(map[string]string),
sessionRoles: make(map[string]string),
runningCancels: make(map[string]context.CancelFunc),
}
}
// sessionKey 生成会话 key
func (h *RobotHandler) sessionKey(platform, userID string) string {
return platform + "_" + userID
}
// getOrCreateConversation 获取或创建当前会话,title 用于新对话的标题(取用户首条消息前50字)
func (h *RobotHandler) getOrCreateConversation(platform, userID, title string) (convID string, isNew bool) {
h.mu.RLock()
convID = h.sessions[h.sessionKey(platform, userID)]
h.mu.RUnlock()
if convID != "" {
return convID, false
}
t := strings.TrimSpace(title)
if t == "" {
t = "新对话 " + time.Now().Format("01-02 15:04")
} else {
t = safeTruncateString(t, 50)
}
conv, err := h.db.CreateConversation(t)
if err != nil {
h.logger.Warn("创建机器人会话失败", zap.Error(err))
return "", false
}
convID = conv.ID
h.mu.Lock()
h.sessions[h.sessionKey(platform, userID)] = convID
h.mu.Unlock()
return convID, true
}
// setConversation 切换当前会话
func (h *RobotHandler) setConversation(platform, userID, convID string) {
h.mu.Lock()
h.sessions[h.sessionKey(platform, userID)] = convID
h.mu.Unlock()
}
// getRole 获取当前用户使用的角色,未设置时返回"默认"
func (h *RobotHandler) getRole(platform, userID string) string {
h.mu.RLock()
role := h.sessionRoles[h.sessionKey(platform, userID)]
h.mu.RUnlock()
if role == "" {
return "默认"
}
return role
}
// setRole 设置当前用户使用的角色
func (h *RobotHandler) setRole(platform, userID, roleName string) {
h.mu.Lock()
h.sessionRoles[h.sessionKey(platform, userID)] = roleName
h.mu.Unlock()
}
// clearConversation 清空当前会话(切换到新对话)
func (h *RobotHandler) clearConversation(platform, userID string) (newConvID string) {
title := "新对话 " + time.Now().Format("01-02 15:04")
conv, err := h.db.CreateConversation(title)
if err != nil {
h.logger.Warn("创建新对话失败", zap.Error(err))
return ""
}
h.setConversation(platform, userID, conv.ID)
return conv.ID
}
// HandleMessage 处理用户输入,返回回复文本(供各平台 webhook 调用)
func (h *RobotHandler) HandleMessage(platform, userID, text string) (reply string) {
text = strings.TrimSpace(text)
if text == "" {
return "请输入内容或发送「帮助」/ help 查看命令。"
}
// 命令分发(支持中英文)
switch {
case text == robotCmdHelp || text == "help" || text == "" || text == "?":
return h.cmdHelp()
case text == robotCmdList || text == robotCmdListAlt || text == "list":
return h.cmdList()
case strings.HasPrefix(text, robotCmdSwitch+" ") || strings.HasPrefix(text, robotCmdContinue+" ") || strings.HasPrefix(text, "switch ") || strings.HasPrefix(text, "continue "):
var id string
switch {
case strings.HasPrefix(text, robotCmdSwitch+" "):
id = strings.TrimSpace(text[len(robotCmdSwitch)+1:])
case strings.HasPrefix(text, robotCmdContinue+" "):
id = strings.TrimSpace(text[len(robotCmdContinue)+1:])
case strings.HasPrefix(text, "switch "):
id = strings.TrimSpace(text[7:])
default:
id = strings.TrimSpace(text[9:])
}
return h.cmdSwitch(platform, userID, id)
case text == robotCmdNew || text == "new":
return h.cmdNew(platform, userID)
case text == robotCmdClear || text == "clear":
return h.cmdClear(platform, userID)
case text == robotCmdCurrent || text == "current":
return h.cmdCurrent(platform, userID)
case text == robotCmdStop || text == "stop":
return h.cmdStop(platform, userID)
case text == robotCmdRoles || text == robotCmdRolesList || text == "roles":
return h.cmdRoles()
case strings.HasPrefix(text, robotCmdRoles+" ") || strings.HasPrefix(text, robotCmdSwitchRole+" ") || strings.HasPrefix(text, "role "):
var roleName string
switch {
case strings.HasPrefix(text, robotCmdRoles+" "):
roleName = strings.TrimSpace(text[len(robotCmdRoles)+1:])
case strings.HasPrefix(text, robotCmdSwitchRole+" "):
roleName = strings.TrimSpace(text[len(robotCmdSwitchRole)+1:])
default:
roleName = strings.TrimSpace(text[5:])
}
return h.cmdSwitchRole(platform, userID, roleName)
case strings.HasPrefix(text, robotCmdDelete+" ") || strings.HasPrefix(text, "delete "):
var convID string
if strings.HasPrefix(text, robotCmdDelete+" ") {
convID = strings.TrimSpace(text[len(robotCmdDelete)+1:])
} else {
convID = strings.TrimSpace(text[7:])
}
return h.cmdDelete(platform, userID, convID)
case text == robotCmdVersion || text == "version":
return h.cmdVersion()
}
// 普通消息:走 Agent
convID, _ := h.getOrCreateConversation(platform, userID, text)
if convID == "" {
return "无法创建或获取对话,请稍后再试。"
}
// 若对话标题为「新对话 xx:xx」格式(由「新对话」命令创建),将标题更新为首条消息内容,与 Web 端体验一致
if conv, err := h.db.GetConversation(convID); err == nil && strings.HasPrefix(conv.Title, "新对话 ") {
newTitle := safeTruncateString(text, 50)
if newTitle != "" {
_ = h.db.UpdateConversationTitle(convID, newTitle)
}
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
sk := h.sessionKey(platform, userID)
h.cancelMu.Lock()
h.runningCancels[sk] = cancel
h.cancelMu.Unlock()
defer func() {
cancel()
h.cancelMu.Lock()
delete(h.runningCancels, sk)
h.cancelMu.Unlock()
}()
role := h.getRole(platform, userID)
resp, newConvID, err := h.agentHandler.ProcessMessageForRobot(ctx, convID, text, role)
if err != nil {
h.logger.Warn("机器人 Agent 执行失败", zap.String("platform", platform), zap.String("userID", userID), zap.Error(err))
if errors.Is(err, context.Canceled) {
return "任务已取消。"
}
return "处理失败: " + err.Error()
}
if newConvID != convID {
h.setConversation(platform, userID, newConvID)
}
return resp
}
func (h *RobotHandler) cmdHelp() string {
return "**【CyberStrikeAI 机器人命令】**\n\n" +
"- `帮助` `help` — 显示本帮助 | Show this help\n" +
"- `列表` `list` — 列出所有对话标题与 ID | List conversations\n" +
"- `切换 <ID>` `switch <ID>` — 指定对话继续 | Switch to conversation\n" +
"- `新对话` `new` — 开启新对话 | Start new conversation\n" +
"- `清空` `clear` — 清空当前上下文 | Clear context\n" +
"- `当前` `current` — 显示当前对话 ID 与标题 | Show current conversation\n" +
"- `停止` `stop` — 中断当前任务 | Stop running task\n" +
"- `角色` `roles` — 列出所有可用角色 | List roles\n" +
"- `角色 <名>` `role <name>` — 切换当前角色 | Switch role\n" +
"- `删除 <ID>` `delete <ID>` — 删除指定对话 | Delete conversation\n" +
"- `版本` `version` — 显示当前版本号 | Show version\n\n" +
"---\n" +
"除以上命令外,直接输入内容将发送给 AI 进行渗透测试/安全分析。\n" +
"Otherwise, send any text for AI penetration testing / security analysis."
}
func (h *RobotHandler) cmdList() string {
convs, err := h.db.ListConversations(50, 0, "")
if err != nil {
return "获取对话列表失败: " + err.Error()
}
if len(convs) == 0 {
return "暂无对话。发送任意内容将自动创建新对话。"
}
var b strings.Builder
b.WriteString("【对话列表】\n")
for i, c := range convs {
if i >= 20 {
b.WriteString("… 仅显示前 20 条\n")
break
}
b.WriteString(fmt.Sprintf("· %s\n ID: %s\n", c.Title, c.ID))
}
return strings.TrimSuffix(b.String(), "\n")
}
func (h *RobotHandler) cmdSwitch(platform, userID, convID string) string {
if convID == "" {
return "请指定对话 ID,例如:切换 xxx-xxx-xxx"
}
conv, err := h.db.GetConversation(convID)
if err != nil {
return "对话不存在或 ID 错误。"
}
h.setConversation(platform, userID, conv.ID)
return fmt.Sprintf("已切换到对话:「%s」\nID: %s", conv.Title, conv.ID)
}
func (h *RobotHandler) cmdNew(platform, userID string) string {
newID := h.clearConversation(platform, userID)
if newID == "" {
return "创建新对话失败,请重试。"
}
return "已开启新对话,可直接发送内容。"
}
func (h *RobotHandler) cmdClear(platform, userID string) string {
return h.cmdNew(platform, userID)
}
func (h *RobotHandler) cmdStop(platform, userID string) string {
sk := h.sessionKey(platform, userID)
h.cancelMu.Lock()
cancel, ok := h.runningCancels[sk]
if ok {
delete(h.runningCancels, sk)
cancel()
}
h.cancelMu.Unlock()
if !ok {
return "当前没有正在执行的任务。"
}
return "已停止当前任务。"
}
func (h *RobotHandler) cmdCurrent(platform, userID string) string {
h.mu.RLock()
convID := h.sessions[h.sessionKey(platform, userID)]
h.mu.RUnlock()
if convID == "" {
return "当前没有进行中的对话。发送任意内容将创建新对话。"
}
conv, err := h.db.GetConversation(convID)
if err != nil {
return "当前对话 ID: " + convID + "(获取标题失败)"
}
role := h.getRole(platform, userID)
return fmt.Sprintf("当前对话:「%s」\nID: %s\n当前角色: %s", conv.Title, conv.ID, role)
}
func (h *RobotHandler) cmdRoles() string {
if h.config.Roles == nil || len(h.config.Roles) == 0 {
return "暂无可用角色。"
}
names := make([]string, 0, len(h.config.Roles))
for name, role := range h.config.Roles {
if role.Enabled {
names = append(names, name)
}
}
if len(names) == 0 {
return "暂无可用角色。"
}
sort.Slice(names, func(i, j int) bool {
if names[i] == "默认" {
return true
}
if names[j] == "默认" {
return false
}
return names[i] < names[j]
})
var b strings.Builder
b.WriteString("【角色列表】\n")
for _, name := range names {
role := h.config.Roles[name]
desc := role.Description
if desc == "" {
desc = "无描述"
}
b.WriteString(fmt.Sprintf("· %s — %s\n", name, desc))
}
return strings.TrimSuffix(b.String(), "\n")
}
func (h *RobotHandler) cmdSwitchRole(platform, userID, roleName string) string {
if roleName == "" {
return "请指定角色名称,例如:角色 渗透测试"
}
if h.config.Roles == nil {
return "暂无可用角色。"
}
role, exists := h.config.Roles[roleName]
if !exists {
return fmt.Sprintf("角色「%s」不存在。发送「角色」查看可用角色。", roleName)
}
if !role.Enabled {
return fmt.Sprintf("角色「%s」已禁用。", roleName)
}
h.setRole(platform, userID, roleName)
return fmt.Sprintf("已切换到角色:「%s」\n%s", roleName, role.Description)
}
func (h *RobotHandler) cmdDelete(platform, userID, convID string) string {
if convID == "" {
return "请指定对话 ID,例如:删除 xxx-xxx-xxx"
}
sk := h.sessionKey(platform, userID)
h.mu.RLock()
currentConvID := h.sessions[sk]
h.mu.RUnlock()
if convID == currentConvID {
// 删除当前对话时,先清空会话绑定
h.mu.Lock()
delete(h.sessions, sk)
h.mu.Unlock()
}
if err := h.db.DeleteConversation(convID); err != nil {
return "删除失败: " + err.Error()
}
return fmt.Sprintf("已删除对话 ID: %s", convID)
}
func (h *RobotHandler) cmdVersion() string {
v := h.config.Version
if v == "" {
v = "未知"
}
return "CyberStrikeAI " + v
}
// —————— 企业微信 ——————
// wecomXML 企业微信回调 XML(明文模式下的简化结构;加密模式需先解密再解析)
type wecomXML struct {
ToUserName string `xml:"ToUserName"`
FromUserName string `xml:"FromUserName"`
CreateTime int64 `xml:"CreateTime"`
MsgType string `xml:"MsgType"`
Content string `xml:"Content"`
MsgID string `xml:"MsgId"`
AgentID int64 `xml:"AgentID"`
Encrypt string `xml:"Encrypt"` // 加密模式下消息在此
}
// wecomReplyXML 被动回复 XML
type wecomReplyXML struct {
XMLName xml.Name `xml:"xml"`
ToUserName string `xml:"ToUserName"`
FromUserName string `xml:"FromUserName"`
CreateTime int64 `xml:"CreateTime"`
MsgType string `xml:"MsgType"`
Content string `xml:"Content"`
}
// HandleWecomGET 企业微信 URL 校验(GET
func (h *RobotHandler) HandleWecomGET(c *gin.Context) {
if !h.config.Robots.Wecom.Enabled {
c.String(http.StatusNotFound, "")
return
}
echostr := c.Query("echostr")
if echostr == "" {
c.String(http.StatusBadRequest, "missing echostr")
return
}
// 明文模式时企业微信可能直接传 echostr,先直接返回以通过校验
c.String(http.StatusOK, echostr)
}
// wecomDecrypt 企业微信消息解密(AES-256-CBC,PKCS7,明文格式:16字节随机+4字节长度+消息+corpID)
func wecomDecrypt(encodingAESKey, encryptedB64 string) ([]byte, error) {
key, err := base64.StdEncoding.DecodeString(encodingAESKey + "=")
if err != nil {
return nil, err
}
if len(key) != 32 {
return nil, fmt.Errorf("encoding_aes_key 解码后应为 32 字节")
}
ciphertext, err := base64.StdEncoding.DecodeString(encryptedB64)
if err != nil {
return nil, err
}
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
iv := key[:16]
mode := cipher.NewCBCDecrypter(block, iv)
if len(ciphertext)%aes.BlockSize != 0 {
return nil, fmt.Errorf("密文长度不是块大小的倍数")
}
plain := make([]byte, len(ciphertext))
mode.CryptBlocks(plain, ciphertext)
// 去除 PKCS7 填充
n := int(plain[len(plain)-1])
if n < 1 || n > 32 {
return nil, fmt.Errorf("无效的 PKCS7 填充")
}
plain = plain[:len(plain)-n]
// 企业微信格式:16 字节随机 + 4 字节长度(大端) + 消息 + corpID
if len(plain) < 20 {
return nil, fmt.Errorf("明文过短")
}
msgLen := binary.BigEndian.Uint32(plain[16:20])
if int(20+msgLen) > len(plain) {
return nil, fmt.Errorf("消息长度越界")
}
return plain[20 : 20+msgLen], nil
}
// HandleWecomPOST 企业微信消息回调(POST),支持明文与加密模式
func (h *RobotHandler) HandleWecomPOST(c *gin.Context) {
if !h.config.Robots.Wecom.Enabled {
c.String(http.StatusOK, "")
return
}
bodyRaw, _ := io.ReadAll(c.Request.Body)
var body wecomXML
if err := xml.Unmarshal(bodyRaw, &body); err != nil {
h.logger.Debug("企业微信 POST 解析 XML 失败", zap.Error(err))
c.String(http.StatusOK, "")
return
}
// 加密模式:先解密再解析内层 XML
if body.Encrypt != "" && h.config.Robots.Wecom.EncodingAESKey != "" {
decrypted, err := wecomDecrypt(h.config.Robots.Wecom.EncodingAESKey, body.Encrypt)
if err != nil {
h.logger.Warn("企业微信消息解密失败", zap.Error(err))
c.String(http.StatusOK, "")
return
}
if err := xml.Unmarshal(decrypted, &body); err != nil {
h.logger.Warn("企业微信解密后 XML 解析失败", zap.Error(err))
c.String(http.StatusOK, "")
return
}
}
if body.MsgType != "text" {
c.XML(http.StatusOK, wecomReplyXML{
ToUserName: body.FromUserName,
FromUserName: body.ToUserName,
CreateTime: time.Now().Unix(),
MsgType: "text",
Content: "暂仅支持文本消息,请发送文字。",
})
return
}
userID := body.FromUserName
text := strings.TrimSpace(body.Content)
reply := h.HandleMessage("wecom", userID, text)
// 加密模式需加密回复(此处简化为明文回复;若企业要求加密需再实现加密)
c.XML(http.StatusOK, wecomReplyXML{
ToUserName: body.FromUserName,
FromUserName: body.ToUserName,
CreateTime: time.Now().Unix(),
MsgType: "text",
Content: reply,
})
}
// —————— 测试接口(需登录,用于验证机器人逻辑,无需钉钉/飞书客户端) ——————
// RobotTestRequest 模拟机器人消息请求
type RobotTestRequest struct {
Platform string `json:"platform"` // 如 "dingtalk"、"lark"、"wecom"
UserID string `json:"user_id"`
Text string `json:"text"`
}
// HandleRobotTest 供本地验证:POST JSON { "platform", "user_id", "text" },返回 { "reply": "..." }
func (h *RobotHandler) HandleRobotTest(c *gin.Context) {
var req RobotTestRequest
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "请求体需为 JSON,包含 platform、user_id、text"})
return
}
platform := strings.TrimSpace(req.Platform)
if platform == "" {
platform = "test"
}
userID := strings.TrimSpace(req.UserID)
if userID == "" {
userID = "test_user"
}
reply := h.HandleMessage(platform, userID, req.Text)
c.JSON(http.StatusOK, gin.H{"reply": reply})
}
// —————— 钉钉 ——————
// HandleDingtalkPOST 钉钉事件回调(流式接入等);当前为占位,返回 200
func (h *RobotHandler) HandleDingtalkPOST(c *gin.Context) {
if !h.config.Robots.Dingtalk.Enabled {
c.JSON(http.StatusOK, gin.H{})
return
}
// 钉钉流式/事件回调格式需按官方文档解析并异步回复,此处仅返回 200
c.JSON(http.StatusOK, gin.H{"message": "ok"})
}
// —————— 飞书 ——————
// HandleLarkPOST 飞书事件回调;当前为占位,返回 200;验证时需返回 challenge
func (h *RobotHandler) HandleLarkPOST(c *gin.Context) {
if !h.config.Robots.Lark.Enabled {
c.JSON(http.StatusOK, gin.H{})
return
}
var body struct {
Challenge string `json:"challenge"`
}
if err := c.ShouldBindJSON(&body); err == nil && body.Challenge != "" {
c.JSON(http.StatusOK, gin.H{"challenge": body.Challenge})
return
}
c.JSON(http.StatusOK, gin.H{})
}
+257
View File
@@ -0,0 +1,257 @@
package handler
import (
"bytes"
"context"
"encoding/json"
"net/http"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"time"
"github.com/gin-gonic/gin"
"go.uber.org/zap"
)
const (
terminalMaxCommandLen = 4096
terminalMaxOutputLen = 256 * 1024 // 256KB
terminalTimeout = 120 * time.Second
)
// TerminalHandler 处理系统设置中的终端命令执行
type TerminalHandler struct {
logger *zap.Logger
}
// maskTerminalCommand 对可能包含敏感信息的终端命令做脱敏,避免在日志中直接记录密码等内容
func maskTerminalCommand(cmd string) string {
trimmed := strings.TrimSpace(cmd)
lower := strings.ToLower(trimmed)
if strings.Contains(lower, "sudo") || strings.Contains(lower, "password") {
return "[masked sensitive terminal command]"
}
if len(trimmed) > 256 {
return trimmed[:256] + "..."
}
return trimmed
}
// NewTerminalHandler 创建终端处理器
func NewTerminalHandler(logger *zap.Logger) *TerminalHandler {
return &TerminalHandler{logger: logger}
}
// RunCommandRequest 执行命令请求
type RunCommandRequest struct {
Command string `json:"command"`
Shell string `json:"shell,omitempty"`
Cwd string `json:"cwd,omitempty"`
}
// RunCommandResponse 执行命令响应
type RunCommandResponse struct {
Stdout string `json:"stdout"`
Stderr string `json:"stderr"`
ExitCode int `json:"exit_code"`
Error string `json:"error,omitempty"`
}
// RunCommand 执行终端命令(需登录)
func (h *TerminalHandler) RunCommand(c *gin.Context) {
var req RunCommandRequest
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "请求体无效,需要 command 字段"})
return
}
cmdStr := strings.TrimSpace(req.Command)
if cmdStr == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "command 不能为空"})
return
}
if len(cmdStr) > terminalMaxCommandLen {
c.JSON(http.StatusBadRequest, gin.H{"error": "命令过长"})
return
}
shell := req.Shell
if shell == "" {
if runtime.GOOS == "windows" {
shell = "cmd"
} else {
shell = "sh"
}
}
ctx, cancel := context.WithTimeout(c.Request.Context(), terminalTimeout)
defer cancel()
var cmd *exec.Cmd
if runtime.GOOS == "windows" {
cmd = exec.CommandContext(ctx, "cmd", "/c", cmdStr)
} else {
cmd = exec.CommandContext(ctx, shell, "-c", cmdStr)
// 无 TTY 时设置 COLUMNS/TERM,使 ping 等工具的 usage 排版与真实终端一致
cmd.Env = append(os.Environ(), "COLUMNS=256", "LINES=40", "TERM=xterm-256color")
}
if req.Cwd != "" {
absCwd, err := filepath.Abs(req.Cwd)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "工作目录无效"})
return
}
cur, _ := os.Getwd()
curAbs, _ := filepath.Abs(cur)
rel, err := filepath.Rel(curAbs, absCwd)
if err != nil || strings.HasPrefix(rel, "..") || rel == ".." {
c.JSON(http.StatusBadRequest, gin.H{"error": "工作目录必须在当前进程目录下"})
return
}
cmd.Dir = absCwd
}
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
err := cmd.Run()
stdoutBytes := stdout.Bytes()
stderrBytes := stderr.Bytes()
// 限制输出长度,防止内存占用过大(复制后截断,避免修改原 buffer)
truncSuffix := []byte("\n...(输出已截断)\n")
if len(stdoutBytes) > terminalMaxOutputLen {
tmp := make([]byte, terminalMaxOutputLen+len(truncSuffix))
n := copy(tmp, stdoutBytes[:terminalMaxOutputLen])
copy(tmp[n:], truncSuffix)
stdoutBytes = tmp
}
if len(stderrBytes) > terminalMaxOutputLen {
tmp := make([]byte, terminalMaxOutputLen+len(truncSuffix))
n := copy(tmp, stderrBytes[:terminalMaxOutputLen])
copy(tmp[n:], truncSuffix)
stderrBytes = tmp
}
exitCode := 0
if err != nil {
if exitErr, ok := err.(*exec.ExitError); ok {
exitCode = exitErr.ExitCode()
} else {
exitCode = -1
}
if ctx.Err() == context.DeadlineExceeded {
so := strings.ReplaceAll(string(stdoutBytes), "\r\n", "\n")
so = strings.ReplaceAll(so, "\r", "\n")
se := strings.ReplaceAll(string(stderrBytes), "\r\n", "\n")
se = strings.ReplaceAll(se, "\r", "\n")
resp := RunCommandResponse{
Stdout: so,
Stderr: se,
ExitCode: -1,
Error: "命令执行超时(" + terminalTimeout.String() + "",
}
c.JSON(http.StatusOK, resp)
return
}
h.logger.Debug("终端命令执行异常", zap.String("command", maskTerminalCommand(cmdStr)), zap.Error(err))
}
// 统一为 \n,避免前端因 \r 出现错位/对角线排版
stdoutStr := strings.ReplaceAll(string(stdoutBytes), "\r\n", "\n")
stdoutStr = strings.ReplaceAll(stdoutStr, "\r", "\n")
stderrStr := strings.ReplaceAll(string(stderrBytes), "\r\n", "\n")
stderrStr = strings.ReplaceAll(stderrStr, "\r", "\n")
resp := RunCommandResponse{
Stdout: stdoutStr,
Stderr: stderrStr,
ExitCode: exitCode,
}
if err != nil && exitCode != 0 {
resp.Error = err.Error()
}
c.JSON(http.StatusOK, resp)
}
// streamEvent SSE 事件
type streamEvent struct {
T string `json:"t"` // "out" | "err" | "exit"
D string `json:"d,omitempty"`
C int `json:"c"` // exit code(不用 omitempty,否则 0 不序列化导致前端显示 [exit undefined]
}
// RunCommandStream 流式执行命令,输出实时推送到前端(SSE)
func (h *TerminalHandler) RunCommandStream(c *gin.Context) {
var req RunCommandRequest
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "请求体无效,需要 command 字段"})
return
}
cmdStr := strings.TrimSpace(req.Command)
if cmdStr == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "command 不能为空"})
return
}
if len(cmdStr) > terminalMaxCommandLen {
c.JSON(http.StatusBadRequest, gin.H{"error": "命令过长"})
return
}
shell := req.Shell
if shell == "" {
if runtime.GOOS == "windows" {
shell = "cmd"
} else {
shell = "sh"
}
}
ctx, cancel := context.WithTimeout(c.Request.Context(), terminalTimeout)
defer cancel()
var cmd *exec.Cmd
if runtime.GOOS == "windows" {
cmd = exec.CommandContext(ctx, "cmd", "/c", cmdStr)
} else {
cmd = exec.CommandContext(ctx, shell, "-c", cmdStr)
cmd.Env = append(os.Environ(), "COLUMNS=256", "LINES=40", "TERM=xterm-256color")
}
if req.Cwd != "" {
absCwd, err := filepath.Abs(req.Cwd)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "工作目录无效"})
return
}
cur, _ := os.Getwd()
curAbs, _ := filepath.Abs(cur)
rel, err := filepath.Rel(curAbs, absCwd)
if err != nil || strings.HasPrefix(rel, "..") || rel == ".." {
c.JSON(http.StatusBadRequest, gin.H{"error": "工作目录必须在当前进程目录下"})
return
}
cmd.Dir = absCwd
}
c.Header("Content-Type", "text/event-stream")
c.Header("Cache-Control", "no-cache")
c.Header("Connection", "keep-alive")
c.Header("X-Accel-Buffering", "no")
c.Writer.WriteHeader(http.StatusOK)
flusher, ok := c.Writer.(http.Flusher)
if !ok {
cancel()
return
}
sendEvent := func(ev streamEvent) {
body, _ := json.Marshal(ev)
c.SSEvent("", string(body))
flusher.Flush()
}
runCommandStreamImpl(cmd, sendEvent, ctx)
}
+46
View File
@@ -0,0 +1,46 @@
//go:build !windows
package handler
import (
"bufio"
"context"
"os/exec"
"strings"
"github.com/creack/pty"
)
const ptyCols = 256
const ptyRows = 40
// runCommandStreamImpl 在 Unix 下用 PTY 执行,使 ping 等命令按终端宽度排版(isatty 为真)
func runCommandStreamImpl(cmd *exec.Cmd, sendEvent func(streamEvent), ctx context.Context) {
ptmx, err := pty.StartWithSize(cmd, &pty.Winsize{Cols: ptyCols, Rows: ptyRows})
if err != nil {
sendEvent(streamEvent{T: "exit", C: -1})
return
}
defer ptmx.Close()
normalize := func(s string) string {
s = strings.ReplaceAll(s, "\r\n", "\n")
return strings.ReplaceAll(s, "\r", "\n")
}
sc := bufio.NewScanner(ptmx)
for sc.Scan() {
sendEvent(streamEvent{T: "out", D: normalize(sc.Text())})
}
exitCode := 0
if err := cmd.Wait(); err != nil {
if exitErr, ok := err.(*exec.ExitError); ok {
exitCode = exitErr.ExitCode()
} else {
exitCode = -1
}
}
if ctx.Err() == context.DeadlineExceeded {
exitCode = -1
}
sendEvent(streamEvent{T: "exit", C: exitCode})
}
@@ -0,0 +1,65 @@
//go:build windows
package handler
import (
"bufio"
"context"
"os/exec"
"strings"
"sync"
)
// runCommandStreamImpl 在 Windows 下用 stdout/stderr 管道执行
func runCommandStreamImpl(cmd *exec.Cmd, sendEvent func(streamEvent), ctx context.Context) {
stdoutPipe, err := cmd.StdoutPipe()
if err != nil {
sendEvent(streamEvent{T: "exit", C: -1})
return
}
stderrPipe, err := cmd.StderrPipe()
if err != nil {
sendEvent(streamEvent{T: "exit", C: -1})
return
}
if err := cmd.Start(); err != nil {
sendEvent(streamEvent{T: "exit", C: -1})
return
}
normalize := func(s string) string {
s = strings.ReplaceAll(s, "\r\n", "\n")
return strings.ReplaceAll(s, "\r", "\n")
}
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
sc := bufio.NewScanner(stdoutPipe)
for sc.Scan() {
sendEvent(streamEvent{T: "out", D: normalize(sc.Text())})
}
}()
go func() {
defer wg.Done()
sc := bufio.NewScanner(stderrPipe)
for sc.Scan() {
sendEvent(streamEvent{T: "err", D: normalize(sc.Text())})
}
}()
wg.Wait()
exitCode := 0
if err := cmd.Wait(); err != nil {
if exitErr, ok := err.(*exec.ExitError); ok {
exitCode = exitErr.ExitCode()
} else {
exitCode = -1
}
}
if ctx.Err() == context.DeadlineExceeded {
exitCode = -1
}
sendEvent(streamEvent{T: "exit", C: exitCode})
}
+95
View File
@@ -0,0 +1,95 @@
//go:build !windows
package handler
import (
"net/http"
"os"
"os/exec"
"time"
"github.com/creack/pty"
"github.com/gin-gonic/gin"
"github.com/gorilla/websocket"
)
// wsUpgrader 仅用于系统设置中的终端 WebSocket,会复用已有的登录保护(JWT 中间件在上层路由组)
var wsUpgrader = websocket.Upgrader{
CheckOrigin: func(r *http.Request) bool {
// 由于已在 Gin 路由层做了认证,这里放宽 Origin,方便在同一域名下通过 HTTPS/WSS 访问
return true
},
}
// RunCommandWS 提供真正交互式 Shell:基于 WebSocket + PTY 的长会话
// 前端建立 WebSocket 连接后,所有键盘输入都会透传到 Shell,Shell 的输出也会实时写回前端。
func (h *TerminalHandler) RunCommandWS(c *gin.Context) {
conn, err := wsUpgrader.Upgrade(c.Writer, c.Request, nil)
if err != nil {
return
}
defer conn.Close()
// 启动交互式 Shell,这里优先使用 bash,找不到则退回 sh
shell := "bash"
if _, err := exec.LookPath(shell); err != nil {
shell = "sh"
}
cmd := exec.Command(shell)
cmd.Env = append(os.Environ(),
"COLUMNS=256",
"LINES=40",
"TERM=xterm-256color",
)
ptmx, err := pty.StartWithSize(cmd, &pty.Winsize{Cols: ptyCols, Rows: ptyRows})
if err != nil {
return
}
defer ptmx.Close()
// Shell -> WebSocket:将 PTY 输出实时发给前端
doneChan := make(chan struct{})
go func() {
buf := make([]byte, 4096)
for {
n, err := ptmx.Read(buf)
if n > 0 {
_ = conn.WriteMessage(websocket.BinaryMessage, buf[:n])
}
if err != nil {
break
}
}
close(doneChan)
}()
// WebSocket -> Shell:将前端输入写入 PTY(包括 sudo 密码、Ctrl+C 等)
conn.SetReadLimit(64 * 1024)
_ = conn.SetReadDeadline(time.Now().Add(terminalTimeout))
conn.SetPongHandler(func(string) error {
_ = conn.SetReadDeadline(time.Now().Add(terminalTimeout))
return nil
})
for {
msgType, data, err := conn.ReadMessage()
if err != nil {
_ = cmd.Process.Kill()
break
}
if msgType != websocket.TextMessage && msgType != websocket.BinaryMessage {
continue
}
if len(data) == 0 {
continue
}
if _, err := ptmx.Write(data); err != nil {
_ = cmd.Process.Kill()
break
}
}
<-doneChan
}
+149 -31
View File
@@ -6,39 +6,75 @@ import (
"fmt"
"net/http"
"strings"
"sync"
"time"
"cyberstrike-ai/internal/config"
"cyberstrike-ai/internal/openai"
"go.uber.org/zap"
"golang.org/x/time/rate"
)
// Embedder 文本嵌入器
type Embedder struct {
openAIClient *openai.Client
config *config.KnowledgeConfig
openAIConfig *config.OpenAIConfig // 用于获取API Key
logger *zap.Logger
openAIClient *openai.Client
config *config.KnowledgeConfig
openAIConfig *config.OpenAIConfig // 用于获取 API Key
logger *zap.Logger
rateLimiter *rate.Limiter // 速率限制器
rateLimitDelay time.Duration // 请求间隔时间
maxRetries int // 最大重试次数
retryDelay time.Duration // 重试间隔
mu sync.Mutex // 保护 rateLimiter
}
// NewEmbedder 创建新的嵌入器
func NewEmbedder(cfg *config.KnowledgeConfig, openAIConfig *config.OpenAIConfig, openAIClient *openai.Client, logger *zap.Logger) *Embedder {
// 初始化速率限制器
var rateLimiter *rate.Limiter
var rateLimitDelay time.Duration
// 如果配置了 MaxRPM,根据 RPM 计算速率限制
if cfg.Indexing.MaxRPM > 0 {
rpm := cfg.Indexing.MaxRPM
rateLimiter = rate.NewLimiter(rate.Every(time.Minute/time.Duration(rpm)), rpm)
logger.Info("知识库索引速率限制已启用", zap.Int("maxRPM", rpm))
} else if cfg.Indexing.RateLimitDelayMs > 0 {
// 如果没有配置 MaxRPM 但配置了固定延迟,使用固定延迟模式
rateLimitDelay = time.Duration(cfg.Indexing.RateLimitDelayMs) * time.Millisecond
logger.Info("知识库索引固定延迟已启用", zap.Duration("delay", rateLimitDelay))
}
// 重试配置
maxRetries := 3
retryDelay := 1000 * time.Millisecond
if cfg.Indexing.MaxRetries > 0 {
maxRetries = cfg.Indexing.MaxRetries
}
if cfg.Indexing.RetryDelayMs > 0 {
retryDelay = time.Duration(cfg.Indexing.RetryDelayMs) * time.Millisecond
}
return &Embedder{
openAIClient: openAIClient,
config: cfg,
openAIConfig: openAIConfig,
logger: logger,
openAIClient: openAIClient,
config: cfg,
openAIConfig: openAIConfig,
logger: logger,
rateLimiter: rateLimiter,
rateLimitDelay: rateLimitDelay,
maxRetries: maxRetries,
retryDelay: retryDelay,
}
}
// EmbeddingRequest OpenAI嵌入请求
// EmbeddingRequest OpenAI 嵌入请求
type EmbeddingRequest struct {
Model string `json:"model"`
Input []string `json:"input"`
}
// EmbeddingResponse OpenAI嵌入响应
// EmbeddingResponse OpenAI 嵌入响应
type EmbeddingResponse struct {
Data []EmbeddingData `json:"data"`
Error *EmbeddingError `json:"error,omitempty"`
@@ -56,12 +92,69 @@ type EmbeddingError struct {
Type string `json:"type"`
}
// EmbedText 对文本进行嵌入
func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error) {
if e.openAIClient == nil {
return nil, fmt.Errorf("OpenAI客户端未初始化")
// waitRateLimiter 等待速率限制器
func (e *Embedder) waitRateLimiter() {
e.mu.Lock()
defer e.mu.Unlock()
if e.rateLimiter != nil {
// 等待令牌
ctx := context.Background()
if err := e.rateLimiter.Wait(ctx); err != nil {
e.logger.Warn("速率限制器等待失败", zap.Error(err))
}
}
if e.rateLimitDelay > 0 {
time.Sleep(e.rateLimitDelay)
}
}
// EmbedText 对文本进行嵌入(带重试和速率限制)
func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error) {
if e.openAIClient == nil {
return nil, fmt.Errorf("OpenAI 客户端未初始化")
}
var lastErr error
for attempt := 0; attempt < e.maxRetries; attempt++ {
// 速率限制
if attempt > 0 {
// 重试时等待更长时间
waitTime := e.retryDelay * time.Duration(attempt)
e.logger.Debug("重试前等待", zap.Int("attempt", attempt+1), zap.Duration("waitTime", waitTime))
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(waitTime):
}
} else {
e.waitRateLimiter()
}
result, err := e.doEmbedText(ctx, text)
if err == nil {
return result, nil
}
lastErr = err
// 检查是否是可重试的错误(429 速率限制、5xx 服务器错误、网络错误)
if !e.isRetryableError(err) {
return nil, err
}
e.logger.Debug("嵌入请求失败,准备重试",
zap.Int("attempt", attempt+1),
zap.Int("maxRetries", e.maxRetries),
zap.Error(err))
}
return nil, fmt.Errorf("达到最大重试次数 (%d): %v", e.maxRetries, lastErr)
}
// doEmbedText 执行实际的嵌入请求(内部方法)
func (e *Embedder) doEmbedText(ctx context.Context, text string) ([]float32, error) {
// 使用配置的嵌入模型
model := e.config.Embedding.Model
if model == "" {
@@ -73,7 +166,7 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
Input: []string{text},
}
// 清理baseURL:去除前后空格和尾部斜杠
// 清理 baseURL:去除前后空格和尾部斜杠
baseURL := strings.TrimSpace(e.config.Embedding.BaseURL)
baseURL = strings.TrimSuffix(baseURL, "/")
if baseURL == "" {
@@ -83,24 +176,24 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
// 构建请求
body, err := json.Marshal(req)
if err != nil {
return nil, fmt.Errorf("序列化请求失败: %w", err)
return nil, fmt.Errorf("序列化请求失败%w", err)
}
requestURL := baseURL + "/embeddings"
httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost, requestURL, strings.NewReader(string(body)))
if err != nil {
return nil, fmt.Errorf("创建请求失败: %w", err)
return nil, fmt.Errorf("创建请求失败%w", err)
}
httpReq.Header.Set("Content-Type", "application/json")
// 使用配置的API Key,如果没有则使用OpenAI配置的
// 使用配置的 API Key,如果没有则使用 OpenAI 配置的
apiKey := strings.TrimSpace(e.config.Embedding.APIKey)
if apiKey == "" && e.openAIConfig != nil {
apiKey = e.openAIConfig.APIKey
}
if apiKey == "" {
return nil, fmt.Errorf("API Key未配置")
return nil, fmt.Errorf("API Key 未配置")
}
httpReq.Header.Set("Authorization", "Bearer "+apiKey)
@@ -110,7 +203,7 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
}
resp, err := httpClient.Do(httpReq)
if err != nil {
return nil, fmt.Errorf("发送请求失败: %w", err)
return nil, fmt.Errorf("发送请求失败%w", err)
}
defer resp.Body.Close()
@@ -132,7 +225,7 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
if len(requestBodyPreview) > 200 {
requestBodyPreview = requestBodyPreview[:200] + "..."
}
e.logger.Debug("嵌入API请求",
e.logger.Debug("嵌入 API 请求",
zap.String("url", httpReq.URL.String()),
zap.String("model", model),
zap.String("requestBody", requestBodyPreview),
@@ -148,12 +241,12 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
if len(bodyPreview) > 500 {
bodyPreview = bodyPreview[:500] + "..."
}
return nil, fmt.Errorf("解析响应失败 (URL: %s, 状态码: %d, 响应长度: %d字节): %w\n请求体: %s\n响应内容预览: %s",
return nil, fmt.Errorf("解析响应失败 (URL: %s, 状态码%d, 响应长度%d字节): %w\n请求体%s\n响应内容预览%s",
requestURL, resp.StatusCode, len(bodyBytes), err, requestBodyPreview, bodyPreview)
}
if embeddingResp.Error != nil {
return nil, fmt.Errorf("OpenAI API错误 (状态码: %d): 类型=%s, 消息=%s",
return nil, fmt.Errorf("OpenAI API 错误 (状态码%d): 类型=%s, 消息=%s",
resp.StatusCode, embeddingResp.Error.Type, embeddingResp.Error.Message)
}
@@ -162,7 +255,7 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
if len(bodyPreview) > 500 {
bodyPreview = bodyPreview[:500] + "..."
}
return nil, fmt.Errorf("HTTP请求失败 (URL: %s, 状态码: %d): 响应内容=%s", requestURL, resp.StatusCode, bodyPreview)
return nil, fmt.Errorf("HTTP 请求失败 (URL: %s, 状态码%d): 响应内容=%s", requestURL, resp.StatusCode, bodyPreview)
}
if len(embeddingResp.Data) == 0 {
@@ -170,11 +263,11 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
if len(bodyPreview) > 500 {
bodyPreview = bodyPreview[:500] + "..."
}
return nil, fmt.Errorf("未收到嵌入数据 (状态码: %d, 响应长度: %d字节)\n响应内容: %s",
return nil, fmt.Errorf("未收到嵌入数据 (状态码%d, 响应长度%d字节)\n响应内容%s",
resp.StatusCode, len(bodyBytes), bodyPreview)
}
// 转换为float32
// 转换为 float32
embedding := make([]float32, len(embeddingResp.Data[0].Embedding))
for i, v := range embeddingResp.Data[0].Embedding {
embedding[i] = float32(v)
@@ -183,23 +276,48 @@ func (e *Embedder) EmbedText(ctx context.Context, text string) ([]float32, error
return embedding, nil
}
// isRetryableError 判断是否是可重试的错误
func (e *Embedder) isRetryableError(err error) bool {
if err == nil {
return false
}
errStr := err.Error()
// 429 速率限制错误
if strings.Contains(errStr, "429") || strings.Contains(errStr, "rate limit") {
return true
}
// 5xx 服务器错误
if strings.Contains(errStr, "500") || strings.Contains(errStr, "502") ||
strings.Contains(errStr, "503") || strings.Contains(errStr, "504") {
return true
}
// 网络错误
if strings.Contains(errStr, "timeout") || strings.Contains(errStr, "connection") ||
strings.Contains(errStr, "network") || strings.Contains(errStr, "EOF") {
return true
}
return false
}
// EmbedTexts 批量嵌入文本
func (e *Embedder) EmbedTexts(ctx context.Context, texts []string) ([][]float32, error) {
if len(texts) == 0 {
return nil, nil
}
// OpenAI API支持批量,但为了简单起见,我们逐个处理
// 实际可以使用批量API以提高效率
embeddings := make([][]float32, len(texts))
for i, text := range texts {
embedding, err := e.EmbedText(ctx, text)
if err != nil {
return nil, fmt.Errorf("嵌入文本[%d]失败: %w", i, err)
return nil, fmt.Errorf("嵌入文本 [%d] 失败%w", i, err)
}
embeddings[i] = embedding
}
return embeddings, nil
}
+369 -97
View File
@@ -10,56 +10,133 @@ import (
"sync"
"time"
"cyberstrike-ai/internal/config"
"github.com/google/uuid"
"go.uber.org/zap"
)
// Indexer 索引器,负责将知识项分块并向量化
type Indexer struct {
db *sql.DB
embedder *Embedder
logger *zap.Logger
chunkSize int // 每个块的最大token数(估算)
overlap int // 块之间的重叠token数
db *sql.DB
embedder *Embedder
logger *zap.Logger
chunkSize int // 每个块的最大 token 数(估算)
overlap int // 块之间的重叠 token
maxChunks int // 单个知识项的最大块数量(0 表示不限制)
// 错误跟踪
mu sync.RWMutex
lastError string // 最近一次错误信息
mu sync.RWMutex
lastError string // 最近一次错误信息
lastErrorTime time.Time // 最近一次错误时间
errorCount int // 连续错误计数
errorCount int // 连续错误计数
// 重建索引状态跟踪
rebuildMu sync.RWMutex
isRebuilding bool // 是否正在重建索引
rebuildTotalItems int // 重建总项数
rebuildCurrent int // 当前已处理项数
rebuildFailed int // 重建失败项数
rebuildStartTime time.Time // 重建开始时间
rebuildLastItemID string // 最近处理的项 ID
rebuildLastChunks int // 最近处理的项的分块数
}
// NewIndexer 创建新的索引器
func NewIndexer(db *sql.DB, embedder *Embedder, logger *zap.Logger) *Indexer {
func NewIndexer(db *sql.DB, embedder *Embedder, logger *zap.Logger, indexingCfg *config.IndexingConfig) *Indexer {
chunkSize := 512
overlap := 50
maxChunks := 0
if indexingCfg != nil {
if indexingCfg.ChunkSize > 0 {
chunkSize = indexingCfg.ChunkSize
}
if indexingCfg.ChunkOverlap >= 0 {
overlap = indexingCfg.ChunkOverlap
}
if indexingCfg.MaxChunksPerItem > 0 {
maxChunks = indexingCfg.MaxChunksPerItem
}
}
return &Indexer{
db: db,
embedder: embedder,
logger: logger,
chunkSize: 512, // 默认512 tokens
overlap: 50, // 默认50 tokens重叠
chunkSize: chunkSize,
overlap: overlap,
maxChunks: maxChunks,
}
}
// ChunkText 将文本分块(支持重叠)
// ChunkText 将文本分块(支持重叠,保留标题上下文
func (idx *Indexer) ChunkText(text string) []string {
// 按Markdown标题分割
chunks := idx.splitByMarkdownHeaders(text)
// 按 Markdown 标题分割,获取带标题的块
sections := idx.splitByMarkdownHeadersWithContent(text)
// 如果块太大,进一步分割
// 处理每个块
result := make([]string, 0)
for _, chunk := range chunks {
if idx.estimateTokens(chunk) <= idx.chunkSize {
result = append(result, chunk)
for _, section := range sections {
// 构建父级标题路径(不包含最后一级标题,因为内容中已经包含)
// 例如:["# A", "## B", "### C"] -> "[# A > ## B]"
var parentHeaderPath string
if len(section.HeaderPath) > 1 {
parentHeaderPath = strings.Join(section.HeaderPath[:len(section.HeaderPath)-1], " > ")
}
// 提取内容的第一行作为标题(如 "# Prompt Injection"
firstLine, remainingContent := extractFirstLine(section.Content)
// 如果剩余内容为空或只有空白,说明这个块只有标题没有正文,跳过
if strings.TrimSpace(remainingContent) == "" {
continue
}
// 如果块太大,进一步分割
if idx.estimateTokens(section.Content) <= idx.chunkSize {
// 块大小合适,添加父级标题前缀
if parentHeaderPath != "" {
result = append(result, fmt.Sprintf("[%s] %s", parentHeaderPath, section.Content))
} else {
result = append(result, section.Content)
}
} else {
// 按段落分割
subChunks := idx.splitByParagraphs(chunk)
for _, subChunk := range subChunks {
if idx.estimateTokens(subChunk) <= idx.chunkSize {
result = append(result, subChunk)
} else {
// 按句子分割(支持重叠)
chunksWithOverlap := idx.splitBySentencesWithOverlap(subChunk)
result = append(result, chunksWithOverlap...)
// 块太大,按子标题或段落分割,保持标题上下文
// 首先尝试按子标题分割(保留子标题结构)
subSections := idx.splitBySubHeaders(section.Content, firstLine, parentHeaderPath)
if len(subSections) > 1 {
// 成功按子标题分割,递归处理每个子块
for _, sub := range subSections {
if idx.estimateTokens(sub) <= idx.chunkSize {
result = append(result, sub)
} else {
// 子块仍然太大,按段落分割(保留标题前缀)
paragraphs := idx.splitByParagraphsWithHeader(sub, parentHeaderPath)
for _, para := range paragraphs {
if idx.estimateTokens(para) <= idx.chunkSize {
result = append(result, para)
} else {
// 段落仍太大,按句子分割
sentenceChunks := idx.splitBySentencesWithOverlap(para)
for _, chunk := range sentenceChunks {
result = append(result, chunk)
}
}
}
}
}
} else {
// 没有子标题,按段落分割(保留标题前缀)
paragraphs := idx.splitByParagraphsWithHeader(section.Content, parentHeaderPath)
for _, para := range paragraphs {
if idx.estimateTokens(para) <= idx.chunkSize {
result = append(result, para)
} else {
// 段落仍太大,按句子分割
sentenceChunks := idx.splitBySentencesWithOverlap(para)
for _, chunk := range sentenceChunks {
result = append(result, chunk)
}
}
}
}
}
@@ -68,43 +145,183 @@ func (idx *Indexer) ChunkText(text string) []string {
return result
}
// splitByMarkdownHeaders 按Markdown标题分割
func (idx *Indexer) splitByMarkdownHeaders(text string) []string {
// 匹配Markdown标题 (# ## ### 等)
// extractFirstLine 提取第一行内容和剩余内容
func extractFirstLine(content string) (firstLine, remaining string) {
lines := strings.SplitN(content, "\n", 2)
if len(lines) == 0 {
return "", ""
}
if len(lines) == 1 {
return lines[0], ""
}
return lines[0], lines[1]
}
// splitBySubHeaders 尝试按子标题分割内容(用于处理大块内容)
// headerPrefix 是父级标题路径,用于添加到每个子块
func (idx *Indexer) splitBySubHeaders(content, headerPrefix, parentPath string) []string {
// 匹配 Markdown 子标题(## 及以上)
subHeaderRegex := regexp.MustCompile(`(?m)^#{2,6}\s+.+$`)
matches := subHeaderRegex.FindAllStringIndex(content, -1)
if len(matches) == 0 {
// 没有子标题,返回原始内容
return []string{content}
}
result := make([]string, 0, len(matches))
for i, match := range matches {
start := match[0]
nextStart := len(content)
if i+1 < len(matches) {
nextStart = matches[i+1][0]
}
subContent := strings.TrimSpace(content[start:nextStart])
// 添加父级路径前缀
if parentPath != "" {
result = append(result, fmt.Sprintf("[%s] %s", parentPath, subContent))
} else {
result = append(result, subContent)
}
}
return result
}
// splitByParagraphsWithHeader 按段落分割,每个段落添加标题前缀(用于保持上下文)
func (idx *Indexer) splitByParagraphsWithHeader(content, parentPath string) []string {
// 提取第一行作为标题
firstLine, _ := extractFirstLine(content)
paragraphs := strings.Split(content, "\n\n")
result := make([]string, 0)
for i, p := range paragraphs {
trimmed := strings.TrimSpace(p)
if trimmed == "" {
continue
}
// 过滤掉只有标题的段落(没有实际内容)
if strings.TrimSpace(trimmed) == strings.TrimSpace(firstLine) {
continue
}
// 第一个段落已经包含标题,不需要重复添加
if i == 0 && strings.Contains(trimmed, firstLine) {
if parentPath != "" {
result = append(result, fmt.Sprintf("[%s] %s", parentPath, trimmed))
} else {
result = append(result, trimmed)
}
} else {
// 其他段落添加标题前缀以保持上下文
if parentPath != "" {
result = append(result, fmt.Sprintf("[%s] %s\n%s", parentPath, firstLine, trimmed))
} else {
result = append(result, fmt.Sprintf("%s\n%s", firstLine, trimmed))
}
}
}
return result
}
// Section 表示一个带标题路径的文本块
type Section struct {
HeaderPath []string // 标题路径(如 ["# SQL 注入", "## 检测方法"]
Content string // 块内容
}
// splitByMarkdownHeadersWithContent 按 Markdown 标题分割,返回带标题路径的块
// 每个块的内容包含自己的标题,用于向量化检索
//
// 例如,对于以下 Markdown:
// # Prompt Injection
// 引言内容
// ## Summary
// 目录内容
//
// 返回:
// [{HeaderPath: ["# Prompt Injection"], Content: "# Prompt Injection\n引言内容"},
// {HeaderPath: ["# Prompt Injection", "## Summary"], Content: "## Summary\n目录内容"}]
func (idx *Indexer) splitByMarkdownHeadersWithContent(text string) []Section {
// 匹配 Markdown 标题 (# ## ### 等)
headerRegex := regexp.MustCompile(`(?m)^#{1,6}\s+.+$`)
// 找到所有标题位置
matches := headerRegex.FindAllStringIndex(text, -1)
if len(matches) == 0 {
return []string{text}
// 没有标题,返回整个文本
return []Section{{HeaderPath: []string{}, Content: text}}
}
chunks := make([]string, 0)
lastPos := 0
sections := make([]Section, 0, len(matches))
currentHeaderPath := []string{}
for _, match := range matches {
for i, match := range matches {
start := match[0]
if start > lastPos {
chunks = append(chunks, strings.TrimSpace(text[lastPos:start]))
}
lastPos = start
}
end := match[1]
nextStart := len(text)
// 添加最后一部分
if lastPos < len(text) {
chunks = append(chunks, strings.TrimSpace(text[lastPos:]))
// 找到下一个标题的位置
if i+1 < len(matches) {
nextStart = matches[i+1][0]
}
// 提取当前标题
headerLine := strings.TrimSpace(text[start:end])
// 计算标题层级(# 的数量)
level := 0
for _, ch := range headerLine {
if ch == '#' {
level++
} else {
break
}
}
// 更新标题路径:移除比当前层级深或等于的子标题,然后添加当前标题
newPath := make([]string, 0, len(currentHeaderPath)+1)
for _, h := range currentHeaderPath {
hLevel := 0
for _, ch := range h {
if ch == '#' {
hLevel++
} else {
break
}
}
if hLevel < level {
newPath = append(newPath, h)
}
}
newPath = append(newPath, headerLine)
currentHeaderPath = newPath
// 提取当前标题到下一个标题之间的内容(包含当前标题)
content := strings.TrimSpace(text[start:nextStart])
// 创建块,使用当前标题路径(包含当前标题)
sections = append(sections, Section{
HeaderPath: append([]string(nil), currentHeaderPath...),
Content: content,
})
}
// 过滤空块
result := make([]string, 0)
for _, chunk := range chunks {
if strings.TrimSpace(chunk) != "" {
result = append(result, chunk)
result := make([]Section, 0, len(sections))
for _, section := range sections {
if strings.TrimSpace(section.Content) != "" {
result = append(result, section)
}
}
if len(result) == 0 {
return []string{text}
return []Section{{HeaderPath: []string{}, Content: text}}
}
return result
@@ -124,8 +341,12 @@ func (idx *Indexer) splitByParagraphs(text string) []string {
// splitBySentences 按句子分割(用于内部,不包含重叠逻辑)
func (idx *Indexer) splitBySentences(text string) []string {
// 简单的句子分割(按句号、问号、感叹号)
sentenceRegex := regexp.MustCompile(`[.!?]+\s+`)
// 简单的句子分割(按句号、问号、感叹号,支持中英文
// . ! ? = 英文标点
// \u3002 = 。(中文句号)
// \uFF01 = (中文叹号)
// \uFF1F = (中文问号)
sentenceRegex := regexp.MustCompile(`[.!?\x{3002}\x{FF01}\x{FF1F}]+`)
sentences := sentenceRegex.Split(text, -1)
result := make([]string, 0)
for _, s := range sentences {
@@ -221,13 +442,13 @@ func (idx *Indexer) splitBySentencesSimple(text string) []string {
return result
}
// extractLastTokens 从文本末尾提取指定token数量的内容
// extractLastTokens 从文本末尾提取指定 token 数量的内容
func (idx *Indexer) extractLastTokens(text string, tokenCount int) string {
if tokenCount <= 0 || text == "" {
return ""
}
// 估算字符数(1 token ≈ 4字符)
// 估算字符数(1 token ≈ 4 字符)
charCount := tokenCount * 4
runes := []rune(text)
@@ -236,12 +457,11 @@ func (idx *Indexer) extractLastTokens(text string, tokenCount int) string {
}
// 从末尾提取指定数量的字符
// 尝试在句子边界处截断,避免截断句子中间
startPos := len(runes) - charCount
extracted := string(runes[startPos:])
// 尝试找到第一个句子边界(句号、问号、感叹号后的空格
sentenceBoundary := regexp.MustCompile(`[.!?]+\s+`)
// 尝试找到第一个句子边界(支持中英文标点
sentenceBoundary := regexp.MustCompile(`[.!?\x{3002}\x{FF01}\x{FF1F}]+`)
matches := sentenceBoundary.FindStringIndex(extracted)
if len(matches) > 0 && matches[0] > 0 {
// 在句子边界处截断,保留完整句子
@@ -251,41 +471,51 @@ func (idx *Indexer) extractLastTokens(text string, tokenCount int) string {
return strings.TrimSpace(extracted)
}
// estimateTokens 估算token数(简单估算:1 token ≈ 4字符)
// estimateTokens 估算 token 数(简单估算:1 token ≈ 4 字符)
func (idx *Indexer) estimateTokens(text string) int {
return len([]rune(text)) / 4
}
// IndexItem 索引知识项(分块并向量化)
func (idx *Indexer) IndexItem(ctx context.Context, itemID string) error {
// 获取知识项(包含categorytitle,用于向量化)
// 获取知识项(包含 categorytitle,用于向量化)
var content, category, title string
err := idx.db.QueryRow("SELECT content, category, title FROM knowledge_base_items WHERE id = ?", itemID).Scan(&content, &category, &title)
if err != nil {
return fmt.Errorf("获取知识项失败: %w", err)
return fmt.Errorf("获取知识项失败%w", err)
}
// 删除旧的向量(在 RebuildIndex 中已经统一清空,这里保留是为了单独调用 IndexItem 时的兼容性)
_, err = idx.db.Exec("DELETE FROM knowledge_embeddings WHERE item_id = ?", itemID)
if err != nil {
return fmt.Errorf("删除旧向量失败: %w", err)
return fmt.Errorf("删除旧向量失败%w", err)
}
// 分块
chunks := idx.ChunkText(content)
// 应用最大块数限制
if idx.maxChunks > 0 && len(chunks) > idx.maxChunks {
idx.logger.Info("知识项块数量超过限制,已截断",
zap.String("itemId", itemID),
zap.Int("originalChunks", len(chunks)),
zap.Int("maxChunks", idx.maxChunks))
chunks = chunks[:idx.maxChunks]
}
idx.logger.Info("知识项分块完成", zap.String("itemId", itemID), zap.Int("chunks", len(chunks)))
// 跟踪该知识项的错误
itemErrorCount := 0
var firstError error
firstErrorChunkIndex := -1
// 向量化每个块(包含categorytitle信息,以便向量检索时能匹配到风险类型)
// 向量化每个块(包含 categorytitle 信息,以便向量检索时能匹配到风险类型)
for i, chunk := range chunks {
// 将categorytitle信息包含到向量化的文本中
// 格式:"[风险类型: {category}] [标题: {title}]\n{chunk内容}"
// 这样向量嵌入就会包含风险类型信息,即使SQL过滤失败,向量相似度也能帮助匹配
textForEmbedding := fmt.Sprintf("[风险类型: %s] [标题: %s]\n%s", category, title, chunk)
// 将 categorytitle 信息包含到向量化的文本中
// 格式:"[风险类型{category}] [标题{title}]\n{chunk 内容}"
// 这样向量嵌入就会包含风险类型信息,即使 SQL 过滤失败,向量相似度也能帮助匹配
textForEmbedding := fmt.Sprintf("[风险类型%s] [标题%s]\n%s", category, title, chunk)
embedding, err := idx.embedder.EmbedText(ctx, textForEmbedding)
if err != nil {
@@ -305,17 +535,17 @@ func (idx *Indexer) IndexItem(ctx context.Context, itemID string) error {
zap.String("chunkPreview", chunkPreview),
zap.Error(err),
)
// 更新全局错误跟踪
errorMsg := fmt.Sprintf("向量化失败 (知识项: %s): %v", itemID, err)
errorMsg := fmt.Sprintf("向量化失败 (知识项%s): %v", itemID, err)
idx.mu.Lock()
idx.lastError = errorMsg
idx.lastErrorTime = time.Now()
idx.mu.Unlock()
}
// 如果连续失败2个块,立即停止处理该知识项(降低阈值,更快停止)
// 这样可以避免继续浪费API调用,同时也能更快地检测到配置问题
// 如果连续失败 2 个块,立即停止处理该知识项(降低阈值,更快停止)
// 这样可以避免继续浪费 API 调用,同时也能更快地检测到配置问题
if itemErrorCount >= 2 {
idx.logger.Error("知识项连续向量化失败,停止处理",
zap.String("itemId", itemID),
@@ -344,6 +574,13 @@ func (idx *Indexer) IndexItem(ctx context.Context, itemID string) error {
}
idx.logger.Info("知识项索引完成", zap.String("itemId", itemID), zap.Int("chunks", len(chunks)))
// 更新重建状态中的最近处理信息
idx.rebuildMu.Lock()
idx.rebuildLastItemID = itemID
idx.rebuildLastChunks = len(chunks)
idx.rebuildMu.Unlock()
return nil
}
@@ -352,23 +589,38 @@ func (idx *Indexer) HasIndex() (bool, error) {
var count int
err := idx.db.QueryRow("SELECT COUNT(*) FROM knowledge_embeddings").Scan(&count)
if err != nil {
return false, fmt.Errorf("检查索引失败: %w", err)
return false, fmt.Errorf("检查索引失败%w", err)
}
return count > 0, nil
}
// RebuildIndex 重建所有索引
func (idx *Indexer) RebuildIndex(ctx context.Context) error {
// 设置重建状态
idx.rebuildMu.Lock()
idx.isRebuilding = true
idx.rebuildTotalItems = 0
idx.rebuildCurrent = 0
idx.rebuildFailed = 0
idx.rebuildStartTime = time.Now()
idx.rebuildLastItemID = ""
idx.rebuildLastChunks = 0
idx.rebuildMu.Unlock()
// 重置错误跟踪
idx.mu.Lock()
idx.lastError = ""
idx.lastErrorTime = time.Time{}
idx.errorCount = 0
idx.mu.Unlock()
rows, err := idx.db.Query("SELECT id FROM knowledge_base_items")
if err != nil {
return fmt.Errorf("查询知识项失败: %w", err)
// 重置重建状态
idx.rebuildMu.Lock()
idx.isRebuilding = false
idx.rebuildMu.Unlock()
return fmt.Errorf("查询知识项失败:%w", err)
}
defer rows.Close()
@@ -376,34 +628,36 @@ func (idx *Indexer) RebuildIndex(ctx context.Context) error {
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return fmt.Errorf("扫描知识项ID失败: %w", err)
// 重置重建状态
idx.rebuildMu.Lock()
idx.isRebuilding = false
idx.rebuildMu.Unlock()
return fmt.Errorf("扫描知识项 ID 失败:%w", err)
}
itemIDs = append(itemIDs, id)
}
idx.rebuildMu.Lock()
idx.rebuildTotalItems = len(itemIDs)
idx.rebuildMu.Unlock()
idx.logger.Info("开始重建索引", zap.Int("totalItems", len(itemIDs)))
// 在开始重建前,先清空所有旧的向量,确保进度从0开始
// 这样 GetIndexStatus 可以准确反映重建进度
_, err = idx.db.Exec("DELETE FROM knowledge_embeddings")
if err != nil {
idx.logger.Warn("清空旧索引失败", zap.Error(err))
// 继续执行,即使清空失败也尝试重建
} else {
idx.logger.Info("已清空旧索引,开始重建")
}
// 注意:不再清空所有旧索引,而是按增量方式更新
// 每个知识项在 IndexItem 中会先删除自己的旧向量,然后插入新向量
// 这样配置更新后只重新索引变化的知识项,保留其他知识项的索引
failedCount := 0
consecutiveFailures := 0
maxConsecutiveFailures := 2 // 连续失败2次后立即停止(降低阈值,更快停止)
maxConsecutiveFailures := 2 // 连续失败 2 次后立即停止(降低阈值,更快停止)
firstFailureItemID := ""
var firstFailureError error
for i, itemID := range itemIDs {
if err := idx.IndexItem(ctx, itemID); err != nil {
failedCount++
consecutiveFailures++
// 只在第一个失败时记录详细日志
if consecutiveFailures == 1 {
firstFailureItemID = itemID
@@ -414,15 +668,15 @@ func (idx *Indexer) RebuildIndex(ctx context.Context) error {
zap.Error(err),
)
}
// 如果连续失败过多,可能是配置问题,立即停止索引
if consecutiveFailures >= maxConsecutiveFailures {
errorMsg := fmt.Sprintf("连续 %d 个知识项索引失败,可能存在配置问题(如嵌入模型配置错误、API密钥无效、余额不足等)。第一个失败项: %s, 错误: %v", consecutiveFailures, firstFailureItemID, firstFailureError)
errorMsg := fmt.Sprintf("连续 %d 个知识项索引失败,可能存在配置问题(如嵌入模型配置错误、API 密钥无效、余额不足等)。第一个失败项%s, 错误%v", consecutiveFailures, firstFailureItemID, firstFailureError)
idx.mu.Lock()
idx.lastError = errorMsg
idx.lastErrorTime = time.Now()
idx.mu.Unlock()
idx.logger.Error("连续索引失败次数过多,立即停止索引",
zap.Int("consecutiveFailures", consecutiveFailures),
zap.Int("totalItems", len(itemIDs)),
@@ -430,17 +684,17 @@ func (idx *Indexer) RebuildIndex(ctx context.Context) error {
zap.String("firstFailureItemId", firstFailureItemID),
zap.Error(firstFailureError),
)
return fmt.Errorf("连续索引失败次数过多: %v", firstFailureError)
return fmt.Errorf("连续索引失败次数过多%v", firstFailureError)
}
// 如果失败的知识项过多,记录警告但继续处理(降低阈值到30%)
// 如果失败的知识项过多,记录警告但继续处理(降低阈值到 30%
if failedCount > len(itemIDs)*3/10 && failedCount == len(itemIDs)*3/10+1 {
errorMsg := fmt.Sprintf("索引失败的知识项过多 (%d/%d),可能存在配置问题。第一个失败项: %s, 错误: %v", failedCount, len(itemIDs), firstFailureItemID, firstFailureError)
errorMsg := fmt.Sprintf("索引失败的知识项过多 (%d/%d),可能存在配置问题。第一个失败项%s, 错误%v", failedCount, len(itemIDs), firstFailureItemID, firstFailureError)
idx.mu.Lock()
idx.lastError = errorMsg
idx.lastErrorTime = time.Now()
idx.mu.Unlock()
idx.logger.Error("索引失败的知识项过多,可能存在配置问题",
zap.Int("failedCount", failedCount),
zap.Int("totalItems", len(itemIDs)),
@@ -450,20 +704,31 @@ func (idx *Indexer) RebuildIndex(ctx context.Context) error {
}
continue
}
// 成功时重置连续失败计数和第一个失败信息
if consecutiveFailures > 0 {
consecutiveFailures = 0
firstFailureItemID = ""
firstFailureError = nil
}
// 减少进度日志频率(每10个或每10%记录一次)
// 更新重建进度
idx.rebuildMu.Lock()
idx.rebuildCurrent = i + 1
idx.rebuildFailed = failedCount
idx.rebuildMu.Unlock()
// 减少进度日志频率(每 10 个或每 10% 记录一次)
if (i+1)%10 == 0 || (len(itemIDs) > 0 && (i+1)*100/len(itemIDs)%10 == 0 && (i+1)*100/len(itemIDs) > 0) {
idx.logger.Info("索引进度", zap.Int("current", i+1), zap.Int("total", len(itemIDs)), zap.Int("failed", failedCount))
}
}
// 重置重建状态
idx.rebuildMu.Lock()
idx.isRebuilding = false
idx.rebuildMu.Unlock()
idx.logger.Info("索引重建完成", zap.Int("totalItems", len(itemIDs)), zap.Int("failedCount", failedCount))
return nil
}
@@ -474,3 +739,10 @@ func (idx *Indexer) GetLastError() (string, time.Time) {
defer idx.mu.RUnlock()
return idx.lastError, idx.lastErrorTime
}
// GetRebuildStatus 获取重建索引状态
func (idx *Indexer) GetRebuildStatus() (isRebuilding bool, totalItems int, current int, failed int, lastItemID string, lastChunks int, startTime time.Time) {
idx.rebuildMu.RLock()
defer idx.rebuildMu.RUnlock()
return idx.isRebuilding, idx.rebuildTotalItems, idx.rebuildCurrent, idx.rebuildFailed, idx.rebuildLastItemID, idx.rebuildLastChunks, idx.rebuildStartTime
}
+10 -2
View File
@@ -55,6 +55,14 @@ func New(level, output string) *Logger {
}
func (l *Logger) Fatal(msg string, fields ...interface{}) {
l.Logger.Fatal(msg, zap.Any("fields", fields))
zapFields := make([]zap.Field, 0, len(fields))
for _, f := range fields {
switch v := f.(type) {
case error:
zapFields = append(zapFields, zap.Error(v))
default:
zapFields = append(zapFields, zap.Any("field", v))
}
}
l.Logger.Fatal(msg, zapFields...)
}
+6
View File
@@ -0,0 +1,6 @@
package robot
// MessageHandler 供飞书/钉钉长连接调用的消息处理接口(由 handler.RobotHandler 实现)
type MessageHandler interface {
HandleMessage(platform, userID, text string) string
}
+137
View File
@@ -0,0 +1,137 @@
package robot
import (
"bytes"
"context"
"encoding/json"
"net/http"
"strings"
"time"
"cyberstrike-ai/internal/config"
"github.com/open-dingtalk/dingtalk-stream-sdk-go/chatbot"
"github.com/open-dingtalk/dingtalk-stream-sdk-go/client"
dingutils "github.com/open-dingtalk/dingtalk-stream-sdk-go/utils"
"go.uber.org/zap"
)
const (
dingReconnectInitial = 5 * time.Second // 首次重连间隔
dingReconnectMax = 60 * time.Second // 最大重连间隔
)
// StartDing 启动钉钉 Stream 长连接(无需公网),收到消息后调用 handler 并通过 SessionWebhook 回复。
// 断线(如笔记本睡眠、网络中断)后会自动重连;ctx 被取消时退出,便于配置变更时重启。
func StartDing(ctx context.Context, cfg config.RobotDingtalkConfig, h MessageHandler, logger *zap.Logger) {
if !cfg.Enabled || cfg.ClientID == "" || cfg.ClientSecret == "" {
return
}
go runDingLoop(ctx, cfg, h, logger)
}
// runDingLoop 循环维持钉钉长连接:断开且 ctx 未取消时按退避间隔重连。
func runDingLoop(ctx context.Context, cfg config.RobotDingtalkConfig, h MessageHandler, logger *zap.Logger) {
backoff := dingReconnectInitial
for {
streamClient := client.NewStreamClient(
client.WithAppCredential(client.NewAppCredentialConfig(cfg.ClientID, cfg.ClientSecret)),
client.WithSubscription(dingutils.SubscriptionTypeKCallback, "/v1.0/im/bot/messages/get",
chatbot.NewDefaultChatBotFrameHandler(func(ctx context.Context, msg *chatbot.BotCallbackDataModel) ([]byte, error) {
go handleDingMessage(ctx, msg, h, logger)
return nil, nil
}).OnEventReceived),
)
logger.Info("钉钉 Stream 正在连接…", zap.String("client_id", cfg.ClientID))
err := streamClient.Start(ctx)
if ctx.Err() != nil {
logger.Info("钉钉 Stream 已按配置重启关闭")
return
}
if err != nil {
logger.Warn("钉钉 Stream 长连接断开(如睡眠/断网),将自动重连", zap.Error(err), zap.Duration("retry_after", backoff))
}
select {
case <-ctx.Done():
return
case <-time.After(backoff):
// 下次重连间隔递增,上限 60 秒,避免频繁重试
if backoff < dingReconnectMax {
backoff *= 2
if backoff > dingReconnectMax {
backoff = dingReconnectMax
}
}
}
}
}
func handleDingMessage(ctx context.Context, msg *chatbot.BotCallbackDataModel, h MessageHandler, logger *zap.Logger) {
if msg == nil || msg.SessionWebhook == "" {
return
}
content := ""
if msg.Text.Content != "" {
content = strings.TrimSpace(msg.Text.Content)
}
if content == "" && msg.Msgtype == "richText" {
if cMap, ok := msg.Content.(map[string]interface{}); ok {
if rich, ok := cMap["richText"].([]interface{}); ok {
for _, c := range rich {
if m, ok := c.(map[string]interface{}); ok {
if txt, ok := m["text"].(string); ok {
content = strings.TrimSpace(txt)
break
}
}
}
}
}
}
if content == "" {
logger.Debug("钉钉消息内容为空,已忽略", zap.String("msgtype", msg.Msgtype))
return
}
logger.Info("钉钉收到消息", zap.String("sender", msg.SenderId), zap.String("content", content))
userID := msg.SenderId
if userID == "" {
userID = msg.ConversationId
}
reply := h.HandleMessage("dingtalk", userID, content)
// 使用 markdown 类型以便正确展示标题、列表、代码块等格式
title := reply
if idx := strings.IndexAny(reply, "\n"); idx > 0 {
title = strings.TrimSpace(reply[:idx])
}
if len(title) > 50 {
title = title[:50] + "…"
}
if title == "" {
title = "回复"
}
body := map[string]interface{}{
"msgtype": "markdown",
"markdown": map[string]string{
"title": title,
"text": reply,
},
}
bodyBytes, _ := json.Marshal(body)
req, err := http.NewRequestWithContext(ctx, http.MethodPost, msg.SessionWebhook, bytes.NewReader(bodyBytes))
if err != nil {
logger.Warn("钉钉构造回复请求失败", zap.Error(err))
return
}
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
logger.Warn("钉钉回复请求失败", zap.Error(err))
return
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
logger.Warn("钉钉回复非 200", zap.Int("status", resp.StatusCode))
return
}
logger.Debug("钉钉回复成功", zap.String("content_preview", reply))
}
+111
View File
@@ -0,0 +1,111 @@
package robot
import (
"context"
"encoding/json"
"strings"
"time"
"cyberstrike-ai/internal/config"
lark "github.com/larksuite/oapi-sdk-go/v3"
larkcore "github.com/larksuite/oapi-sdk-go/v3/core"
"github.com/larksuite/oapi-sdk-go/v3/event/dispatcher"
larkim "github.com/larksuite/oapi-sdk-go/v3/service/im/v1"
larkws "github.com/larksuite/oapi-sdk-go/v3/ws"
"go.uber.org/zap"
)
const (
larkReconnectInitial = 5 * time.Second // 首次重连间隔
larkReconnectMax = 60 * time.Second // 最大重连间隔
)
type larkTextContent struct {
Text string `json:"text"`
}
// StartLark 启动飞书长连接(无需公网),收到消息后调用 handler 并回复。
// 断线(如笔记本睡眠、网络中断)后会自动重连;ctx 被取消时退出,便于配置变更时重启。
func StartLark(ctx context.Context, cfg config.RobotLarkConfig, h MessageHandler, logger *zap.Logger) {
if !cfg.Enabled || cfg.AppID == "" || cfg.AppSecret == "" {
return
}
go runLarkLoop(ctx, cfg, h, logger)
}
// runLarkLoop 循环维持飞书长连接:断开且 ctx 未取消时按退避间隔重连。
func runLarkLoop(ctx context.Context, cfg config.RobotLarkConfig, h MessageHandler, logger *zap.Logger) {
backoff := larkReconnectInitial
for {
larkClient := lark.NewClient(cfg.AppID, cfg.AppSecret)
eventHandler := dispatcher.NewEventDispatcher("", "").OnP2MessageReceiveV1(func(ctx context.Context, event *larkim.P2MessageReceiveV1) error {
go handleLarkMessage(ctx, event, h, larkClient, logger)
return nil
})
wsClient := larkws.NewClient(cfg.AppID, cfg.AppSecret,
larkws.WithEventHandler(eventHandler),
larkws.WithLogLevel(larkcore.LogLevelInfo),
)
logger.Info("飞书长连接正在连接…", zap.String("app_id", cfg.AppID))
err := wsClient.Start(ctx)
if ctx.Err() != nil {
logger.Info("飞书长连接已按配置重启关闭")
return
}
if err != nil {
logger.Warn("飞书长连接断开(如睡眠/断网),将自动重连", zap.Error(err), zap.Duration("retry_after", backoff))
}
select {
case <-ctx.Done():
return
case <-time.After(backoff):
if backoff < larkReconnectMax {
backoff *= 2
if backoff > larkReconnectMax {
backoff = larkReconnectMax
}
}
}
}
}
func handleLarkMessage(ctx context.Context, event *larkim.P2MessageReceiveV1, h MessageHandler, client *lark.Client, logger *zap.Logger) {
if event == nil || event.Event == nil || event.Event.Message == nil || event.Event.Sender == nil || event.Event.Sender.SenderId == nil {
return
}
msg := event.Event.Message
msgType := larkcore.StringValue(msg.MessageType)
if msgType != larkim.MsgTypeText {
logger.Debug("飞书暂仅处理文本消息", zap.String("msg_type", msgType))
return
}
var textBody larkTextContent
if err := json.Unmarshal([]byte(larkcore.StringValue(msg.Content)), &textBody); err != nil {
logger.Warn("飞书消息 Content 解析失败", zap.Error(err))
return
}
text := strings.TrimSpace(textBody.Text)
if text == "" {
return
}
userID := ""
if event.Event.Sender.SenderId.UserId != nil {
userID = *event.Event.Sender.SenderId.UserId
}
messageID := larkcore.StringValue(msg.MessageId)
reply := h.HandleMessage("lark", userID, text)
contentBytes, _ := json.Marshal(larkTextContent{Text: reply})
_, err := client.Im.Message.Reply(ctx, larkim.NewReplyMessageReqBuilder().
MessageId(messageID).
Body(larkim.NewReplyMessageReqBodyBuilder().
MsgType(larkim.MsgTypeText).
Content(string(contentBytes)).
Build()).
Build())
if err != nil {
logger.Warn("飞书回复失败", zap.String("message_id", messageID), zap.Error(err))
return
}
logger.Debug("飞书已回复", zap.String("message_id", messageID))
}
+2 -1
View File
@@ -46,8 +46,9 @@ parameters:
**注意事项:**
- 必需参数,不能为空
- 如果指定进程ID,需要配合 -d 参数使用
- 注意:radare2 要求文件路径必须是最后一个参数,因此 target 使用 position 1
required: true
position: 0
position: 1
format: "positional"
- name: "commands"
type: "string"
+366 -5
View File
@@ -1582,12 +1582,106 @@ header {
box-sizing: border-box;
}
.chat-input-container > .chat-input-with-files {
flex: 1;
display: flex;
flex-direction: column;
min-width: 0;
gap: 6px;
}
.chat-input-container > .chat-input-field {
flex: 1;
display: flex;
min-width: 0;
}
.chat-file-list {
display: flex;
flex-wrap: wrap;
gap: 6px;
align-items: center;
min-height: 0;
}
.chat-file-chip {
display: inline-flex;
align-items: center;
gap: 6px;
padding: 4px 10px;
background: rgba(0, 102, 255, 0.08);
border: 1px solid rgba(0, 102, 255, 0.2);
border-radius: 8px;
font-size: 12px;
color: var(--text-primary);
max-width: 200px;
}
.chat-file-chip-name {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.chat-file-chip-remove {
flex-shrink: 0;
width: 18px;
height: 18px;
padding: 0;
border: none;
background: rgba(0, 0, 0, 0.08);
border-radius: 50%;
cursor: pointer;
color: var(--text-muted);
display: flex;
align-items: center;
justify-content: center;
transition: background 0.2s, color 0.2s;
}
.chat-file-chip-remove:hover {
background: rgba(0, 0, 0, 0.12);
color: var(--text-primary);
}
.chat-file-input-hidden {
position: absolute;
width: 0;
height: 0;
opacity: 0;
overflow: hidden;
pointer-events: none;
}
.chat-upload-btn {
display: flex;
align-items: center;
justify-content: center;
width: 40px;
height: 40px;
padding: 0;
border: 1px solid rgba(0, 0, 0, 0.1);
border-radius: 12px;
background: #ffffff;
color: var(--text-muted);
cursor: pointer;
transition: border-color 0.2s, color 0.2s, background 0.2s;
flex-shrink: 0;
}
.chat-upload-btn:hover {
border-color: var(--accent-color);
color: var(--accent-color);
background: rgba(0, 102, 255, 0.04);
}
.chat-input-container.drag-over {
background: rgba(0, 102, 255, 0.06);
border-radius: 12px;
outline: 2px dashed rgba(0, 102, 255, 0.35);
outline-offset: -2px;
}
.chat-input-field {
flex: 1;
position: relative;
@@ -2142,6 +2236,10 @@ header {
-webkit-background-clip: text;
-webkit-text-fill-color: transparent;
background-clip: text;
/* 允许标题在 flex 中收缩/换行,避免把右侧按钮挤压变形 */
flex: 1 1 auto;
min-width: 0;
overflow-wrap: anywhere;
}
.modal-close {
@@ -3034,6 +3132,164 @@ header {
line-height: 1.6;
}
/* 系统设置 - 终端 */
.terminal-wrapper {
border: 1px solid var(--border-color);
border-radius: 6px;
overflow: hidden;
background: #0d1117;
}
.terminal-tabs {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 0;
padding: 6px 10px 0;
background: #161b22;
border-bottom: 1px solid var(--border-color);
min-height: 36px;
}
.terminal-tab {
display: flex;
align-items: center;
gap: 6px;
padding: 5px 8px 5px 12px;
font-size: 0.8125rem;
color: #8b949e;
background: transparent;
border: none;
border-radius: 4px 4px 0 0;
cursor: pointer;
margin-right: 2px;
}
.terminal-tab-label {
cursor: pointer;
}
.terminal-tab-close {
padding: 0;
width: 18px;
height: 18px;
font-size: 1.1rem;
line-height: 1;
color: #8b949e;
background: transparent;
border: none;
border-radius: 4px;
cursor: pointer;
display: flex;
align-items: center;
justify-content: center;
}
.terminal-tab-close:hover {
color: #ff7b72;
background: rgba(255, 123, 114, 0.15);
}
.terminal-tab:hover {
color: #e6edf3;
background: rgba(255, 255, 255, 0.06);
}
.terminal-tab.active {
color: #e6edf3;
background: #0d1117;
font-weight: 500;
}
.terminal-tab-new {
margin-left: 4px;
width: 28px;
height: 28px;
font-size: 1.125rem;
line-height: 1;
color: #8b949e;
background: transparent;
border: 1px solid transparent;
border-radius: 6px;
cursor: pointer;
display: flex;
align-items: center;
justify-content: center;
}
.terminal-tab-new:hover {
color: #58a6ff;
background: rgba(88, 166, 255, 0.1);
border-color: rgba(88, 166, 255, 0.3);
}
.terminal-toolbar {
display: flex;
align-items: center;
justify-content: space-between;
padding: 10px 16px;
background: #161b22;
border-bottom: 1px solid var(--border-color);
}
.terminal-toolbar-title {
font-size: 0.8125rem;
font-weight: 600;
color: #8b949e;
letter-spacing: 0.02em;
}
.terminal-btn {
padding: 6px 12px;
font-size: 0.8125rem;
color: var(--text-secondary);
background: transparent;
border: 1px solid var(--border-color);
border-radius: 6px;
cursor: pointer;
transition: color 0.15s, background 0.15s, border-color 0.15s;
}
.terminal-btn:hover {
color: var(--text-primary);
background: rgba(255, 255, 255, 0.06);
border-color: #8b949e;
}
.terminal-panes {
position: relative;
min-height: 400px;
}
.terminal-pane {
display: none;
min-height: 400px;
}
.terminal-pane.active {
display: block;
}
.terminal-container {
min-height: 400px;
padding: 8px;
box-sizing: border-box;
}
.terminal-container .xterm {
padding: 0;
}
.terminal-container .xterm-viewport {
border-radius: 0;
}
.terminal-error {
color: #ff7b72;
padding: 16px;
font-size: 0.875rem;
}
.settings-section {
margin-bottom: 32px;
}
@@ -3854,6 +4110,36 @@ header {
border-color: var(--accent-color);
}
/* 通用按钮加载态(用于耗时请求:AI 解析等) */
.btn-loading {
opacity: 0.9;
cursor: progress;
display: inline-flex;
align-items: center;
justify-content: center;
gap: 6px;
}
.btn-loading::before {
content: '';
width: 12px;
height: 12px;
border: 2px solid rgba(0, 0, 0, 0.18);
border-top-color: currentColor;
border-radius: 50%;
display: inline-block;
animation: btnSpin 0.8s linear infinite;
}
@keyframes btnSpin {
to { transform: rotate(360deg); }
}
.fofa-nl-status {
margin-top: 6px;
font-size: 0.8125rem;
}
.btn-danger {
padding: 10px 20px;
background: rgba(220, 53, 69, 0.08);
@@ -7808,6 +8094,27 @@ header {
color: var(--accent-color);
}
/* 批量队列详情弹窗:标题很长时避免挤压右侧按钮 */
#batch-queue-detail-modal .modal-header {
align-items: flex-start;
gap: 12px;
}
#batch-queue-detail-modal .modal-header > div {
flex-shrink: 0;
}
#batch-queue-detail-modal .modal-header-actions {
flex-direction: row;
width: auto;
flex-wrap: wrap;
justify-content: flex-end;
}
#batch-queue-detail-modal .modal-header-actions button {
white-space: nowrap;
}
.batch-queue-tasks-list {
max-height: 500px;
overflow-y: auto;
@@ -7847,12 +8154,13 @@ header {
}
.batch-task-header .btn-small {
margin-left: auto;
margin-left: 0;
flex-shrink: 0;
}
.batch-task-header .batch-task-edit-btn {
margin-left: 8px;
/* 仅把“编辑”(首个操作按钮)推到最右,避免多个按钮都 margin-left:auto 导致挤压/错位 */
margin-left: auto;
}
.batch-task-header .batch-task-delete-btn {
@@ -9303,6 +9611,13 @@ header {
width: 100%;
gap: 6px;
}
/* 批量队列详情弹窗:即使在窄屏也保持按钮不“变形” */
#batch-queue-detail-modal .modal-header-actions {
flex-direction: row;
width: auto;
gap: 8px;
}
.attack-chain-action-btn {
width: 100%;
@@ -10952,10 +11267,55 @@ header {
box-shadow: 0 2px 10px rgba(0, 0, 0, 0.04);
}
/* 表单整体增加纵向留白,避免“挤在一起” */
.info-collect-form {
display: flex;
flex-direction: column;
gap: 14px;
}
/* 将每个表单块做成轻量卡片分组(只作用于 info-collect 顶层 form-group */
.info-collect-form > .form-group,
.info-collect-form > .info-collect-form-row {
padding: 14px 14px;
border: 1px solid rgba(0, 0, 0, 0.06);
border-radius: 12px;
background: linear-gradient(135deg, #ffffff 0%, #fafbfc 100%);
}
/* 覆盖全局 .form-group textarea(min-height:200px) 的默认大留白 */
.form-group textarea.info-collect-query-input {
min-height: 36px;
max-height: 96px;
overflow: hidden; /* 配合 JS 自动增高 */
resize: none;
padding: 10px 12px;
}
.info-collect-nl-row {
display: flex;
align-items: flex-start;
gap: 10px;
}
.info-collect-nl-row .info-collect-query-input {
flex: 1 1 auto;
min-width: 0; /* 允许在 flex 中收缩,避免撑破布局 */
}
.info-collect-nl-row button {
flex: 0 0 auto;
white-space: nowrap;
}
@media (max-width: 980px) {
.info-collect-form-row {
grid-template-columns: 1fr;
}
.info-collect-nl-row {
flex-direction: column;
align-items: stretch;
}
.info-collect-col-actions {
width: 140px;
}
@@ -10964,7 +11324,8 @@ header {
.info-collect-form-row {
display: grid;
grid-template-columns: repeat(3, minmax(180px, 1fr));
gap: 12px;
gap: 14px;
align-items: end;
}
.info-collect-form-row .form-group {
@@ -11106,7 +11467,7 @@ header {
.info-collect-presets {
display: flex;
gap: 8px;
gap: 10px;
flex-wrap: wrap;
margin-top: 8px;
}
@@ -11123,7 +11484,7 @@ header {
border: 1px solid var(--border-color);
background: var(--bg-secondary);
color: var(--text-secondary);
padding: 6px 10px;
padding: 7px 12px;
border-radius: 999px;
cursor: pointer;
font-size: 0.8125rem;
+165 -15
View File
@@ -22,6 +22,12 @@ const DRAFT_STORAGE_KEY = 'cyberstrike-chat-draft';
let draftSaveTimer = null;
const DRAFT_SAVE_DELAY = 500; // 500ms防抖延迟
// 对话文件上传相关(后端会拼接路径与内容发给大模型,前端不再重复发文件列表)
const MAX_CHAT_FILES = 10;
const CHAT_FILE_DEFAULT_PROMPT = '请根据上传的文件内容进行分析。';
/** @type {{ fileName: string, content: string, mimeType: string }[]} */
let chatAttachments = [];
// 保存输入框草稿到localStorage(防抖版本)
function saveChatDraftDebounced(content) {
// 清除之前的定时器
@@ -107,14 +113,22 @@ function adjustTextareaHeight(textarea) {
// 发送消息
async function sendMessage() {
const input = document.getElementById('chat-input');
const message = input.value.trim();
if (!message) {
let message = input.value.trim();
const hasAttachments = chatAttachments && chatAttachments.length > 0;
if (!message && !hasAttachments) {
return;
}
// 显示用户消息
addMessage('user', message);
// 有附件且用户未输入时,发一句简短默认提示即可(后端会拼接路径和文件内容给大模型)
if (hasAttachments && !message) {
message = CHAT_FILE_DEFAULT_PROMPT;
}
// 显示用户消息(含附件名,便于用户确认)
const displayMessage = hasAttachments
? message + '\n' + chatAttachments.map(a => '📎 ' + a.fileName).join('\n')
: message;
addMessage('user', displayMessage);
// 清除防抖定时器,防止在清空输入框后重新保存草稿
if (draftSaveTimer) {
@@ -135,7 +149,24 @@ async function sendMessage() {
input.value = '';
// 强制重置输入框高度为初始高度(40px)
input.style.height = '40px';
// 构建请求体(含附件)
const body = {
message: message,
conversationId: currentConversationId,
role: typeof getCurrentRole === 'function' ? getCurrentRole() : ''
};
if (hasAttachments) {
body.attachments = chatAttachments.map(a => ({
fileName: a.fileName,
content: a.content,
mimeType: a.mimeType || ''
}));
}
// 发送后清空附件列表
chatAttachments = [];
renderChatFileChips();
// 创建进度消息容器(使用详细的进度展示)
const progressId = addProgressMessage();
const progressElement = document.getElementById(progressId);
@@ -145,19 +176,12 @@ async function sendMessage() {
let mcpExecutionIds = [];
try {
// 获取当前选中的角色(从 roles.js 的函数获取)
const roleName = typeof getCurrentRole === 'function' ? getCurrentRole() : '';
const response = await apiFetch('/api/agent-loop/stream', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
message: message,
conversationId: currentConversationId,
role: roleName || undefined
}),
body: JSON.stringify(body),
});
if (!response.ok) {
@@ -222,6 +246,130 @@ async function sendMessage() {
}
}
// ---------- 对话文件上传 ----------
function renderChatFileChips() {
const list = document.getElementById('chat-file-list');
if (!list) return;
list.innerHTML = '';
if (!chatAttachments.length) return;
chatAttachments.forEach((a, i) => {
const chip = document.createElement('div');
chip.className = 'chat-file-chip';
chip.setAttribute('role', 'listitem');
const name = document.createElement('span');
name.className = 'chat-file-chip-name';
name.title = a.fileName;
name.textContent = a.fileName;
const remove = document.createElement('button');
remove.type = 'button';
remove.className = 'chat-file-chip-remove';
remove.title = '移除';
remove.innerHTML = '×';
remove.setAttribute('aria-label', '移除 ' + a.fileName);
remove.addEventListener('click', () => removeChatAttachment(i));
chip.appendChild(name);
chip.appendChild(remove);
list.appendChild(chip);
});
}
function removeChatAttachment(index) {
chatAttachments.splice(index, 1);
renderChatFileChips();
}
// 有附件且输入框为空时,填入一句默认提示(可编辑);后端会单独拼接路径与内容给大模型
function appendChatFilePrompt() {
const input = document.getElementById('chat-input');
if (!input || !chatAttachments.length) return;
if (!input.value.trim()) {
input.value = CHAT_FILE_DEFAULT_PROMPT;
adjustTextareaHeight(input);
}
}
function readFileAsAttachment(file) {
return new Promise((resolve, reject) => {
const mimeType = file.type || '';
const isTextLike = /^text\//i.test(mimeType) || /^(application\/(json|xml|javascript)|image\/svg\+xml)/i.test(mimeType);
const reader = new FileReader();
reader.onload = () => {
let content = reader.result;
if (typeof content === 'string' && content.startsWith('data:')) {
content = content.replace(/^data:[^;]+;base64,/, '');
}
resolve({ fileName: file.name, content: content, mimeType: mimeType });
};
reader.onerror = () => reject(reader.error);
if (isTextLike) {
reader.readAsText(file, 'UTF-8');
} else {
reader.readAsDataURL(file);
}
});
}
function addFilesToChat(files) {
if (!files || !files.length) return;
const next = Array.from(files);
if (chatAttachments.length + next.length > MAX_CHAT_FILES) {
alert('最多同时上传 ' + MAX_CHAT_FILES + ' 个文件,当前已选 ' + chatAttachments.length + ' 个。');
return;
}
const addOne = (file) => {
return readFileAsAttachment(file).then((a) => {
chatAttachments.push(a);
renderChatFileChips();
appendChatFilePrompt();
}).catch(() => {
alert('读取文件失败:' + file.name);
});
};
let p = Promise.resolve();
next.forEach((file) => { p = p.then(() => addOne(file)); });
p.then(() => {});
}
function setupChatFileUpload() {
const inputEl = document.getElementById('chat-file-input');
const container = document.getElementById('chat-input-container') || document.querySelector('.chat-input-container');
if (!inputEl || !container) return;
inputEl.addEventListener('change', function () {
const files = this.files;
if (files && files.length) {
addFilesToChat(files);
}
this.value = '';
});
container.addEventListener('dragover', function (e) {
e.preventDefault();
e.stopPropagation();
this.classList.add('drag-over');
});
container.addEventListener('dragleave', function (e) {
e.preventDefault();
e.stopPropagation();
if (!this.contains(e.relatedTarget)) {
this.classList.remove('drag-over');
}
});
container.addEventListener('drop', function (e) {
e.preventDefault();
e.stopPropagation();
this.classList.remove('drag-over');
const files = e.dataTransfer && e.dataTransfer.files;
if (files && files.length) addFilesToChat(files);
});
}
// 确保 chat-input-container 有 id(若模板未写)
function ensureChatInputContainerId() {
const c = document.querySelector('.chat-input-container');
if (c && !c.id) c.id = 'chat-input-container';
}
function setupMentionSupport() {
mentionSuggestionsEl = document.getElementById('mention-suggestions');
if (mentionSuggestionsEl) {
@@ -799,6 +947,8 @@ function initializeChatUI() {
}
activeTaskInterval = setInterval(() => loadActiveTasks(), ACTIVE_TASK_REFRESH_INTERVAL);
setupMentionSupport();
ensureChatInputContainerId();
setupChatFileUpload();
}
// 消息计数器,确保ID唯一
+347 -25
View File
@@ -10,6 +10,11 @@ const infoCollectState = {
tableBound: false
};
// AI 解析(自然语言 -> FOFA)交互状态
let fofaParseAbortController = null;
let fofaParseSlowTimer = null;
let fofaParseToastHandle = null;
// HTML转义(如果未定义)
if (typeof escapeHtml === 'undefined') {
function escapeHtml(text) {
@@ -23,6 +28,7 @@ if (typeof escapeHtml === 'undefined') {
function getFofaFormElements() {
return {
query: document.getElementById('fofa-query'),
nl: document.getElementById('fofa-nl'),
size: document.getElementById('fofa-size'),
page: document.getElementById('fofa-page'),
fields: document.getElementById('fofa-fields'),
@@ -101,20 +107,35 @@ function initInfoCollectPage() {
}
});
// 单行输入:按内容自动增高(避免默认留空白行)
const autoGrow = () => {
// 自然语言输入:Ctrl/Cmd+Enter 触发解析
if (els.nl) {
els.nl.addEventListener('keydown', (e) => {
if ((e.ctrlKey || e.metaKey) && e.key === 'Enter') {
e.preventDefault();
parseFofaNaturalLanguage();
}
});
}
// textarea:按内容自动增高(避免默认留空白行)
const autoGrowTextarea = (el) => {
if (!el) return;
try {
els.query.style.height = '40px';
const max = 110;
const h = Math.min(max, els.query.scrollHeight);
els.query.style.height = `${h}px`;
el.style.height = '36px';
const max = 96;
const h = Math.min(max, el.scrollHeight);
el.style.height = `${h}px`;
} catch (e) {
// ignore
}
};
els.query.addEventListener('input', autoGrow);
els.query.addEventListener('input', () => autoGrowTextarea(els.query));
if (els.nl) els.nl.addEventListener('input', () => autoGrowTextarea(els.nl));
// 初始化时也执行一次
setTimeout(autoGrow, 0);
setTimeout(() => {
autoGrowTextarea(els.query);
autoGrowTextarea(els.nl);
}, 0);
// 绑定表格事件(事件委托,只绑定一次)
bindFofaTableEvents();
@@ -206,6 +227,213 @@ async function submitFofaSearch() {
}
}
async function parseFofaNaturalLanguage() {
const els = getFofaFormElements();
const text = (els.nl?.value || '').trim();
if (!text) {
alert('请输入自然语言描述');
return;
}
// 二次点击:取消进行中的解析(避免“以为卡死/失败”)
if (fofaParseAbortController) {
try { fofaParseAbortController.abort(); } catch (e) { /* ignore */ }
return;
}
// 先创建 controller,避免极快的重复点击触发并发请求
fofaParseAbortController = new AbortController();
setFofaParseLoading(true, 'AI 解析中...');
// 持续提示:直到请求完成/取消/失败才消失
fofaParseToastHandle = showInlineToast('AI 解析中...(点击按钮可取消)', { duration: 0, id: 'fofa-parse-pending' });
// 如果超过一小段时间还没返回,再强调“仍在进行中”,降低误判为失败的概率
fofaParseSlowTimer = setTimeout(() => {
const status = document.getElementById('fofa-nl-status');
if (status) {
status.textContent = 'AI 解析耗时较长,仍在处理中…';
status.style.display = 'block';
}
}, 1800);
try {
const resp = await apiFetch('/api/fofa/parse', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ text }),
signal: fofaParseAbortController.signal
});
const result = await resp.json().catch(() => ({}));
if (!resp.ok) {
throw new Error(result.error || `请求失败: ${resp.status}`);
}
showFofaParseModal(text, result);
showInlineToast('AI 解析完成');
} catch (e) {
// AbortController 取消:不视为失败
if (e && (e.name === 'AbortError' || String(e).includes('AbortError'))) {
showInlineToast('已取消 AI 解析');
return;
}
console.error('FOFA 自然语言解析失败:', e);
showInlineToast('AI 解析失败:' + (e && e.message ? e.message : String(e)), { duration: 2800 });
}
finally {
fofaParseAbortController = null;
if (fofaParseSlowTimer) {
clearTimeout(fofaParseSlowTimer);
fofaParseSlowTimer = null;
}
if (fofaParseToastHandle && typeof fofaParseToastHandle.remove === 'function') {
fofaParseToastHandle.remove();
}
fofaParseToastHandle = null;
setFofaParseLoading(false, '');
}
}
function setFofaParseLoading(loading, statusText) {
const btn = document.getElementById('fofa-nl-parse-btn');
const status = document.getElementById('fofa-nl-status');
if (btn) {
if (loading) {
if (!btn.dataset.originalText) btn.dataset.originalText = btn.textContent || 'AI 解析';
btn.classList.add('btn-loading');
btn.textContent = '取消解析';
btn.title = '点击取消 AI 解析';
btn.dataset.loading = '1';
btn.setAttribute('aria-busy', 'true');
btn.disabled = false;
} else {
btn.classList.remove('btn-loading');
btn.textContent = btn.dataset.originalText || 'AI 解析';
btn.title = '将自然语言解析为 FOFA 查询语法';
btn.disabled = false;
delete btn.dataset.loading;
btn.removeAttribute('aria-busy');
}
}
if (status) {
const text = (statusText || '').trim();
if (loading && text) {
status.textContent = text;
status.style.display = 'block';
} else {
status.textContent = '';
status.style.display = 'none';
}
}
}
function showFofaParseModal(nlText, parsed) {
const existing = document.getElementById('fofa-parse-modal');
if (existing) existing.remove();
const safeNL = escapeHtml((nlText || '').trim());
const warnings = Array.isArray(parsed?.warnings) ? parsed.warnings.filter(Boolean).map(x => String(x)) : [];
const explanation = parsed?.explanation != null ? String(parsed.explanation) : '';
const warningsHtml = warnings.length
? `<ul style="margin: 8px 0 0 18px;">${warnings.map(w => `<li>${escapeHtml(w)}</li>`).join('')}</ul>`
: `<div class="muted" style="margin-top: 8px;">无</div>`;
const modal = document.createElement('div');
modal.id = 'fofa-parse-modal';
modal.className = 'modal';
modal.style.display = 'block';
modal.innerHTML = `
<div class="modal-content" style="max-width: 900px;">
<div class="modal-header">
<h2>AI 解析结果</h2>
<span class="modal-close" id="fofa-parse-modal-close" title="关闭">&times;</span>
</div>
<div style="padding: 18px 28px; overflow: auto;">
<div class="form-group">
<label>自然语言</label>
<div class="muted" style="margin-top: 6px; white-space: pre-wrap;">${safeNL || '-'}</div>
</div>
<div class="form-group" style="margin-top: 14px;">
<label for="fofa-parse-query">FOFA 查询语法可编辑</label>
<textarea id="fofa-parse-query" class="info-collect-query-input" rows="2" placeholder='例如:app="Apache" && country="CN"'></textarea>
<small class="form-hint">请人工确认语法与范围无误后再执行查询</small>
</div>
<div class="form-group" style="margin-top: 14px;">
<label>提醒</label>
<div style="background: #fff8e1; border: 1px solid #ffe8a3; border-radius: 10px; padding: 10px 12px;">
${warningsHtml}
</div>
</div>
${explanation ? `
<div class="form-group" style="margin-top: 14px;">
<label>解析说明</label>
<pre style="margin-top: 8px; white-space: pre-wrap; background: var(--bg-tertiary); border: 1px solid var(--border-color); border-radius: 10px; padding: 10px 12px; font-size: 13px;">${escapeHtml(explanation)}</pre>
</div>` : ''}
</div>
<div class="modal-footer" style="padding: 18px 28px;">
<button class="btn-secondary" type="button" id="fofa-parse-cancel">取消</button>
<button class="btn-secondary" type="button" id="fofa-parse-apply">填入查询框</button>
<button class="btn-primary" type="button" id="fofa-parse-apply-run">填入并查询</button>
</div>
</div>
`;
document.body.appendChild(modal);
const queryTextarea = document.getElementById('fofa-parse-query');
if (queryTextarea) {
queryTextarea.value = (parsed?.query || '').trim();
setTimeout(() => {
try { queryTextarea.focus(); } catch (e) { /* ignore */ }
}, 0);
}
const close = () => modal.remove();
modal.addEventListener('click', (e) => {
if (e.target === modal) close();
});
document.getElementById('fofa-parse-modal-close')?.addEventListener('click', close);
document.getElementById('fofa-parse-cancel')?.addEventListener('click', close);
const applyToQuery = (run) => {
const els = getFofaFormElements();
const q = (queryTextarea?.value || '').trim();
if (!q) {
showInlineToast('解析结果为空:请在弹窗中补充/修改 FOFA 查询语法', { duration: 2600 });
return;
}
if (els.query) {
els.query.value = q;
try { els.query.focus(); } catch (e) { /* ignore */ }
}
// 写入表单缓存(与现有“直接查询”一致)
saveFofaFormToStorage({
query: q,
size: parseInt(els.size?.value, 10) || 100,
page: parseInt(els.page?.value, 10) || 1,
fields: (els.fields?.value || '').trim(),
full: !!els.full?.checked
});
close();
if (run) submitFofaSearch();
};
document.getElementById('fofa-parse-apply')?.addEventListener('click', () => applyToQuery(false));
document.getElementById('fofa-parse-apply-run')?.addEventListener('click', () => applyToQuery(true));
// Esc 关闭
const onKey = (e) => {
if (e.key === 'Escape') {
close();
document.removeEventListener('keydown', onKey);
}
};
document.addEventListener('keydown', onKey);
}
function setFofaMeta(text) {
const els = getFofaFormElements();
if (els.meta) {
@@ -393,12 +621,79 @@ function copyFofaTargetEncoded(encodedTarget) {
}
}
function showInlineToast(text) {
// showInlineToast('xxx');也支持 showInlineToast('xxx', { duration: 0, id: '...' })
function showInlineToast(text, options) {
const opts = options && typeof options === 'object' ? options : {};
const duration = typeof opts.duration === 'number' ? opts.duration : 1200;
const id = typeof opts.id === 'string' && opts.id.trim() ? opts.id.trim() : '';
const replace = opts.replace !== false;
if (id && replace) {
document.getElementById(id)?.remove();
}
const toast = document.createElement('div');
toast.textContent = text;
toast.style.cssText = 'position: fixed; top: 24px; right: 24px; background: rgba(0,0,0,0.85); color: #fff; padding: 10px 12px; border-radius: 8px; z-index: 10000; font-size: 13px;';
if (id) toast.id = id;
toast.textContent = String(text == null ? '' : text);
toast.style.cssText = 'position: fixed; top: 24px; right: 24px; background: rgba(0,0,0,0.85); color: #fff; padding: 10px 12px; border-radius: 8px; z-index: 10000; font-size: 13px; max-width: 420px; line-height: 1.4; box-shadow: 0 6px 18px rgba(0,0,0,0.22);';
document.body.appendChild(toast);
setTimeout(() => toast.remove(), 1200);
let timer = null;
const remove = () => {
try { if (timer) clearTimeout(timer); } catch (e) { /* ignore */ }
timer = null;
try { toast.remove(); } catch (e) { /* ignore */ }
};
if (duration > 0) {
timer = setTimeout(remove, duration);
}
return { el: toast, remove };
}
function truncateForPreview(value, maxLen) {
const s = value == null ? '' : String(value);
if (maxLen <= 0 || s.length <= maxLen) return s;
return s.slice(0, maxLen) + '...(已截断)';
}
function formatFofaRowSummary(row, fields) {
const r = row && typeof row === 'object' ? row : {};
const order = [];
const seen = new Set();
const preferred = Array.isArray(fields) ? fields : [];
preferred.forEach(k => {
const key = String(k || '').trim();
if (!key || seen.has(key)) return;
seen.add(key);
order.push(key);
});
Object.keys(r).sort().forEach(k => {
if (seen.has(k)) return;
seen.add(k);
order.push(k);
});
if (order.length === 0) return '-';
const lines = order.map((k) => {
const v = r[k];
let text = '';
if (v === null) text = 'null';
else if (v === undefined) text = '';
else if (typeof v === 'string') text = v === '' ? '""' : v;
else if (typeof v === 'number' || typeof v === 'boolean') text = String(v);
else {
try { text = JSON.stringify(v); } catch (e) { text = String(v); }
}
text = truncateForPreview(text, 800);
return `- ${k}: ${text}`;
});
return lines.join('\n');
}
function scanFofaRow(encodedRowJson, clickEvent) {
@@ -423,7 +718,7 @@ function scanFofaRow(encodedRowJson, clickEvent) {
window.location.hash = 'chat';
}
const message = buildScanMessage(target, row);
const message = buildScanMessage(target, row, { fields });
const autoSend = !!(clickEvent && (clickEvent.ctrlKey || clickEvent.metaKey));
setTimeout(async () => {
@@ -458,15 +753,12 @@ function scanFofaRow(encodedRowJson, clickEvent) {
}, 250);
}
function buildScanMessage(target, row) {
const title = row && row.title != null ? String(row.title).trim() : '';
const server = row && row.server != null ? String(row.server).trim() : '';
const hintParts = [];
if (title) hintParts.push(`title="${title}"`);
if (server) hintParts.push(`server="${server}"`);
const hint = hintParts.length ? `FOFA提示:${hintParts.join(', ')}` : '';
function buildScanMessage(target, row, options) {
const opts = options && typeof options === 'object' ? options : {};
const fields = Array.isArray(opts.fields) ? opts.fields : [];
return `对以下目标做信息收集与基础扫描:\n${target}\n\n要求:\n1) 识别服务/框架与关键指纹\n2) 枚举开放端口与常见管理入口\n3) 用 httpx/指纹/目录探测等方式快速确认可访问面\n4) 输出可复现的命令与结论\n\n${hint}`.trim();
const summary = formatFofaRowSummary(row || {}, fields);
return `对以下目标做信息收集与基础扫描:\n${target}\n\n要求:\n1) 识别服务/框架与关键指纹\n2) 枚举开放端口与常见管理入口\n3) 用 httpx/指纹/目录探测等方式快速确认可访问面\n4) 输出可复现的命令与结论\n\n已知信息(来自 FOFA 该行全部字段):\n${summary}`.trim();
}
function bindFofaTableEvents() {
@@ -641,14 +933,32 @@ function exportFofaResults(format) {
return;
}
// csv:默认导出可见字段(更符合“列隐藏”直觉)
if (format === 'xlsx') {
// 使用 SheetJS 生成 XLSX(需在页面中引入 xlsx 库)
if (typeof XLSX === 'undefined') {
alert('未加载 XLSX 库,请刷新页面后重试');
return;
}
const aoa = [visibleFields].concat(p.results.map(row => {
const r = row && typeof row === 'object' ? row : {};
return visibleFields.map(f => r[f] != null ? r[f] : '');
}));
const ws = XLSX.utils.aoa_to_sheet(aoa);
const wb = XLSX.utils.book_new();
XLSX.utils.book_append_sheet(wb, ws, 'FOFA结果');
XLSX.writeFile(wb, `fofa_results_${ts}.xlsx`);
return;
}
// csv:默认导出可见字段,带 UTF-8 BOM 以兼容 Excel 中文
const header = visibleFields;
const rows = p.results.map(row => {
const r = row && typeof row === 'object' ? row : {};
return header.map(f => csvEscape(r[f]));
});
const csv = [header.map(csvEscape).join(','), ...rows.map(cols => cols.join(','))].join('\n');
downloadBlob(csv, `fofa_results_${ts}.csv`, 'text/csv;charset=utf-8');
const csvWithBom = '\uFEFF' + csv;
downloadBlob(csvWithBom, `fofa_results_${ts}.csv`, 'text/csv;charset=utf-8');
}
function csvEscape(value) {
@@ -684,6 +994,7 @@ async function batchScanSelectedFofaRows() {
const fields = p.fields || [];
const tasks = [];
const skipped = [];
selected.forEach(idx => {
const row = p.results[idx];
const target = inferTargetFromRow(row || {}, fields);
@@ -691,7 +1002,10 @@ async function batchScanSelectedFofaRows() {
skipped.push(idx + 1);
return;
}
tasks.push(buildScanMessage(target, row || {}));
// 批量任务:与单条一致,只带“该行全部字段”的摘要(避免重复与超长)
tasks.push(buildScanMessage(target, row || {}, {
fields
}));
});
if (tasks.length === 0) {
@@ -701,10 +1015,17 @@ async function batchScanSelectedFofaRows() {
const title = (p.query ? `FOFA 批量扫描:${p.query}` : 'FOFA 批量扫描').slice(0, 80);
try {
// 不强制切换到“信息收集”角色:沿用当前已选角色;若为默认则传空字符串交给后端走默认逻辑
let role = '';
if (typeof getCurrentRole === 'function') {
try { role = getCurrentRole() || ''; } catch (e) { /* ignore */ }
}
if (role === '默认') role = '';
const resp = await apiFetch('/api/batch-tasks', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ title, tasks, role: '信息收集' })
body: JSON.stringify({ title, tasks, role })
});
const result = await resp.json().catch(() => ({}));
if (!resp.ok) {
@@ -787,6 +1108,7 @@ function showCellDetailModal(field, fullText) {
window.initInfoCollectPage = initInfoCollectPage;
window.resetFofaForm = resetFofaForm;
window.submitFofaSearch = submitFofaSearch;
window.parseFofaNaturalLanguage = parseFofaNaturalLanguage;
window.scanFofaRow = scanFofaRow;
window.copyFofaTarget = copyFofaTarget;
window.copyFofaTargetEncoded = copyFofaTargetEncoded;
+42
View File
@@ -459,6 +459,9 @@ async function updateIndexProgress() {
const isComplete = status.is_complete || false;
const lastError = status.last_error || '';
// 检查是否正在重建索引(优先使用重建状态)
const isRebuilding = status.is_rebuilding || false;
if (totalItems === 0) {
// 没有知识项,隐藏进度条
progressContainer.style.display = 'none';
@@ -524,6 +527,45 @@ async function updateIndexProgress() {
return;
}
// 优先处理重建状态
if (isRebuilding) {
const rebuildTotal = status.rebuild_total || totalItems;
const rebuildCurrent = status.rebuild_current || 0;
const rebuildFailed = status.rebuild_failed || 0;
const rebuildLastItemID = status.rebuild_last_item_id || '';
const rebuildLastChunks = status.rebuild_last_chunks || 0;
const rebuildStartTime = status.rebuild_start_time || '';
// 计算进度百分比(使用重建进度)
let rebuildProgress = progressPercent;
if (rebuildTotal > 0) {
rebuildProgress = (rebuildCurrent / rebuildTotal) * 100;
}
progressContainer.innerHTML = `
<div class="knowledge-index-progress">
<div class="progress-header">
<span class="progress-icon">🔨</span>
<span class="progress-text">正在重建索引${rebuildCurrent}/${rebuildTotal} (${rebuildProgress.toFixed(1)}%) - 失败${rebuildFailed}</span>
</div>
<div class="progress-bar-container">
<div class="progress-bar" style="width: ${rebuildProgress}%"></div>
</div>
<div class="progress-hint">
${rebuildLastItemID ? `正在处理:${escapeHtml(rebuildLastItemID.substring(0, 36))}... (${rebuildLastChunks} chunks)` : '正在处理...'}
${rebuildStartTime ? `<br>开始时间:${new Date(rebuildStartTime).toLocaleString()}` : ''}
</div>
</div>
`;
// 重建中时继续轮询
if (!indexProgressInterval) {
indexProgressInterval = setInterval(updateIndexProgress, 2000);
}
return;
}
if (isComplete) {
progressContainer.innerHTML = `
<div class="knowledge-index-progress-complete">
File diff suppressed because it is too large Load Diff
+103
View File
@@ -46,6 +46,9 @@ function switchSettingsSection(section) {
if (activeContent) {
activeContent.classList.add('active');
}
if (section === 'terminal' && typeof initTerminal === 'function') {
setTimeout(initTerminal, 0);
}
}
// 打开设置
@@ -169,7 +172,76 @@ async function loadConfig(loadTools = true) {
// 允许0.0值,只有undefined/null时才使用默认值
retrievalWeightInput.value = (hybridWeight !== undefined && hybridWeight !== null) ? hybridWeight : 0.7;
}
// 索引配置
const indexing = knowledge.indexing || {};
const chunkSizeInput = document.getElementById('knowledge-indexing-chunk-size');
if (chunkSizeInput) {
chunkSizeInput.value = indexing.chunk_size || 512;
}
const chunkOverlapInput = document.getElementById('knowledge-indexing-chunk-overlap');
if (chunkOverlapInput) {
chunkOverlapInput.value = indexing.chunk_overlap ?? 50;
}
const maxChunksPerItemInput = document.getElementById('knowledge-indexing-max-chunks-per-item');
if (maxChunksPerItemInput) {
maxChunksPerItemInput.value = indexing.max_chunks_per_item ?? 0;
}
const maxRpmInput = document.getElementById('knowledge-indexing-max-rpm');
if (maxRpmInput) {
maxRpmInput.value = indexing.max_rpm ?? 0;
}
const rateLimitDelayInput = document.getElementById('knowledge-indexing-rate-limit-delay-ms');
if (rateLimitDelayInput) {
rateLimitDelayInput.value = indexing.rate_limit_delay_ms ?? 300;
}
const maxRetriesInput = document.getElementById('knowledge-indexing-max-retries');
if (maxRetriesInput) {
maxRetriesInput.value = indexing.max_retries ?? 3;
}
const retryDelayInput = document.getElementById('knowledge-indexing-retry-delay-ms');
if (retryDelayInput) {
retryDelayInput.value = indexing.retry_delay_ms ?? 1000;
}
}
// 填充机器人配置
const robots = currentConfig.robots || {};
const wecom = robots.wecom || {};
const dingtalk = robots.dingtalk || {};
const lark = robots.lark || {};
const wecomEnabled = document.getElementById('robot-wecom-enabled');
if (wecomEnabled) wecomEnabled.checked = wecom.enabled === true;
const wecomToken = document.getElementById('robot-wecom-token');
if (wecomToken) wecomToken.value = wecom.token || '';
const wecomAes = document.getElementById('robot-wecom-encoding-aes-key');
if (wecomAes) wecomAes.value = wecom.encoding_aes_key || '';
const wecomCorp = document.getElementById('robot-wecom-corp-id');
if (wecomCorp) wecomCorp.value = wecom.corp_id || '';
const wecomSecret = document.getElementById('robot-wecom-secret');
if (wecomSecret) wecomSecret.value = wecom.secret || '';
const wecomAgentId = document.getElementById('robot-wecom-agent-id');
if (wecomAgentId) wecomAgentId.value = wecom.agent_id || '0';
const dingtalkEnabled = document.getElementById('robot-dingtalk-enabled');
if (dingtalkEnabled) dingtalkEnabled.checked = dingtalk.enabled === true;
const dingtalkClientId = document.getElementById('robot-dingtalk-client-id');
if (dingtalkClientId) dingtalkClientId.value = dingtalk.client_id || '';
const dingtalkClientSecret = document.getElementById('robot-dingtalk-client-secret');
if (dingtalkClientSecret) dingtalkClientSecret.value = dingtalk.client_secret || '';
const larkEnabled = document.getElementById('robot-lark-enabled');
if (larkEnabled) larkEnabled.checked = lark.enabled === true;
const larkAppId = document.getElementById('robot-lark-app-id');
if (larkAppId) larkAppId.value = lark.app_id || '';
const larkAppSecret = document.getElementById('robot-lark-app-secret');
if (larkAppSecret) larkAppSecret.value = lark.app_secret || '';
const larkVerify = document.getElementById('robot-lark-verify-token');
if (larkVerify) larkVerify.value = lark.verify_token || '';
// 只有在需要时才加载工具列表(MCP管理页面需要,系统设置页面不需要)
if (loadTools) {
@@ -693,9 +765,19 @@ async function applySettings() {
const val = parseFloat(document.getElementById('knowledge-retrieval-hybrid-weight')?.value);
return isNaN(val) ? 0.7 : val; // 允许0.0值,只有NaN时才使用默认值
})()
},
indexing: {
chunk_size: parseInt(document.getElementById("knowledge-indexing-chunk-size")?.value) || 512,
chunk_overlap: parseInt(document.getElementById("knowledge-indexing-chunk-overlap")?.value) ?? 50,
max_chunks_per_item: parseInt(document.getElementById("knowledge-indexing-max-chunks-per-item")?.value) ?? 0,
max_rpm: parseInt(document.getElementById("knowledge-indexing-max-rpm")?.value) ?? 0,
rate_limit_delay_ms: parseInt(document.getElementById("knowledge-indexing-rate-limit-delay-ms")?.value) ?? 300,
max_retries: parseInt(document.getElementById("knowledge-indexing-max-retries")?.value) ?? 3,
retry_delay_ms: parseInt(document.getElementById("knowledge-indexing-retry-delay-ms")?.value) ?? 1000
}
};
const wecomAgentIdVal = document.getElementById('robot-wecom-agent-id')?.value.trim();
const config = {
openai: {
api_key: apiKey,
@@ -711,6 +793,27 @@ async function applySettings() {
max_iterations: parseInt(document.getElementById('agent-max-iterations').value) || 30
},
knowledge: knowledgeConfig,
robots: {
wecom: {
enabled: document.getElementById('robot-wecom-enabled')?.checked === true,
token: document.getElementById('robot-wecom-token')?.value.trim() || '',
encoding_aes_key: document.getElementById('robot-wecom-encoding-aes-key')?.value.trim() || '',
corp_id: document.getElementById('robot-wecom-corp-id')?.value.trim() || '',
secret: document.getElementById('robot-wecom-secret')?.value.trim() || '',
agent_id: parseInt(wecomAgentIdVal, 10) || 0
},
dingtalk: {
enabled: document.getElementById('robot-dingtalk-enabled')?.checked === true,
client_id: document.getElementById('robot-dingtalk-client-id')?.value.trim() || '',
client_secret: document.getElementById('robot-dingtalk-client-secret')?.value.trim() || ''
},
lark: {
enabled: document.getElementById('robot-lark-enabled')?.checked === true,
app_id: document.getElementById('robot-lark-app-id')?.value.trim() || '',
app_secret: document.getElementById('robot-lark-app-secret')?.value.trim() || '',
verify_token: document.getElementById('robot-lark-verify-token')?.value.trim() || ''
}
},
tools: []
};
+2 -1
View File
@@ -1197,7 +1197,8 @@ async function showBatchQueueDetail(queueId) {
batchQueuesState.currentQueueId = queueId;
if (title) {
title.textContent = queue.title ? `批量任务队列 - ${escapeHtml(queue.title)}` : '批量任务队列';
// textContent 本身会做转义;这里不要再 escapeHtml,否则会把 && 显示成 &amp;...(看起来像“变形/乱码”)
title.textContent = queue.title ? `批量任务队列 - ${String(queue.title)}` : '批量任务队列';
}
// 更新按钮显示
+419
View File
@@ -0,0 +1,419 @@
/**
* 系统设置 - 终端多标签流式输出命令历史Ctrl+L 清屏长时间可取消
*/
(function () {
var getContext = HTMLCanvasElement.prototype.getContext;
HTMLCanvasElement.prototype.getContext = function (type, attrs) {
if (type === '2d') {
attrs = (attrs && typeof attrs === 'object') ? Object.assign({ willReadFrequently: true }, attrs) : { willReadFrequently: true };
return getContext.call(this, type, attrs);
}
return getContext.apply(this, arguments);
};
var terminals = [];
var currentTabId = 1;
var inited = false;
var tabIdCounter = 1;
var PROMPT = ''; // 真实 Shell 自己输出提示符,这里不再自定义
var HISTORY_MAX = 100;
var CANCEL_AFTER_MS = 125000;
function getCurrent() {
for (var i = 0; i < terminals.length; i++) {
if (terminals[i].id === currentTabId) return terminals[i];
}
return terminals[0] || null;
}
var WELCOME_LINE = 'CyberStrikeAI 终端 - 真实 Shell 会话,直接输入命令;Ctrl+L 清屏\r\n';
function writePrompt(tab) {
// 提示符交由后端 Shell 自行输出,这里仅保留占位函数,避免旧代码报错
}
function redrawTabDisplay(t) {
if (!t || !t.term) return;
t.term.clear();
t.term.write(WELCOME_LINE);
}
function writeln(tabOrS, s) {
var t, text;
if (arguments.length === 1) { text = tabOrS; t = getCurrent(); } else { t = tabOrS; text = s; }
if (!t || !t.term) return;
if (text) t.term.writeln(text);
else t.term.writeln('');
}
function writeOutput(tab, text, isError) {
var t = tab || getCurrent();
if (!t || !t.term || !text) return;
var s = String(text).replace(/\r\n/g, '\n').replace(/\r/g, '\n');
var lines = s.split('\n');
var prefix = isError ? '\x1b[31m' : '';
var suffix = isError ? '\x1b[0m' : '';
t.term.write(prefix);
for (var i = 0; i < lines.length; i++) {
var line = lines[i].replace(/\r/g, '');
t.term.writeln(line);
}
t.term.write(suffix);
}
// 从本地存储中获取当前登录 token(与 auth.js 使用的结构保持一致)
function getStoredAuthToken() {
try {
var raw = localStorage.getItem('cyberstrike-auth');
if (!raw) return null;
var o = JSON.parse(raw);
if (o && o.token) return o.token;
} catch (e) {}
return null;
}
// WebSocket 地址构造(兼容 http/https,并通过 query 传递 token 以通过后端鉴权)
function buildTerminalWSURL() {
var proto = (window.location.protocol === 'https:') ? 'wss://' : 'ws://';
var url = proto + window.location.host + '/api/terminal/ws';
var token = getStoredAuthToken();
if (token) {
url += '?token=' + encodeURIComponent(token);
}
return url;
}
function ensureTerminalWS(tab) {
if (tab.ws && (tab.ws.readyState === WebSocket.OPEN || tab.ws.readyState === WebSocket.CONNECTING)) {
return;
}
try {
var ws = new WebSocket(buildTerminalWSURL());
tab.ws = ws;
tab.running = true;
ws.onopen = function () {
if (tab.term) {
tab.term.focus();
}
};
ws.onmessage = function (ev) {
if (!tab.term) return;
// 处理二进制消息和文本消息
if (ev.data instanceof ArrayBuffer) {
var decoder = new TextDecoder('utf-8');
tab.term.write(decoder.decode(ev.data));
} else if (ev.data instanceof Blob) {
// Blob 类型,需要异步读取
var reader = new FileReader();
reader.onload = function () {
var decoder = new TextDecoder('utf-8');
tab.term.write(decoder.decode(reader.result));
};
reader.readAsArrayBuffer(ev.data);
} else {
// 字符串类型
tab.term.write(ev.data);
}
};
ws.onclose = function () {
tab.running = false;
if (tab.term) {
tab.term.writeln('\r\n\x1b[2m[会话已关闭]\x1b[0m');
}
};
ws.onerror = function () {
tab.running = false;
if (tab.term) {
tab.term.writeln('\r\n\x1b[31m[终端连接出错]\x1b[0m');
}
};
} catch (e) {
if (tab.term) {
tab.term.writeln('\r\n\x1b[31m[无法连接终端服务: ' + String(e) + ']\x1b[0m');
}
}
}
function createTerminalInContainer(container, tab) {
if (typeof Terminal === 'undefined') return null;
if (!tab.history) tab.history = [];
if (tab.historyIndex === undefined) tab.historyIndex = -1;
if (tab.cursorIndex === undefined) tab.cursorIndex = 0;
var term = new Terminal({
cursorBlink: true,
cursorStyle: 'bar',
fontSize: 13,
fontFamily: 'Menlo, Monaco, "Courier New", monospace',
lineHeight: 1.2,
scrollback: 1000,
theme: {
background: '#0d1117',
foreground: '#e6edf3',
cursor: '#58a6ff',
cursorAccent: '#0d1117',
selection: 'rgba(88, 166, 255, 0.3)',
black: '#484f58',
red: '#ff7b72',
green: '#3fb950',
yellow: '#d29922',
blue: '#58a6ff',
magenta: '#bc8cff',
cyan: '#39c5cf',
white: '#e6edf3',
brightBlack: '#6e7681',
brightRed: '#ffa198',
brightGreen: '#56d364',
brightYellow: '#e3b341',
brightBlue: '#79c0ff',
brightMagenta: '#d2a8ff',
brightCyan: '#56d4dd',
brightWhite: '#f0f6fc'
}
});
var fitAddon = null;
if (typeof FitAddon !== 'undefined') {
var FitCtor = (FitAddon.FitAddon || FitAddon);
fitAddon = new FitCtor();
term.loadAddon(fitAddon);
}
term.open(container);
term.write(WELCOME_LINE);
container.addEventListener('click', function () {
switchTerminalTab(tab.id);
if (term) term.focus();
});
container.setAttribute('tabindex', '0');
container.title = '点击此处后输入命令';
function sendToWS(data) {
ensureTerminalWS(tab);
if (tab.ws && tab.ws.readyState === WebSocket.OPEN) {
try {
tab.ws.send(data);
} catch (e) {}
}
}
term.onData(function (data) {
// Ctrl+L:本地清屏,同时把 ^L 也发给后端
if (data === '\x0c') {
term.clear();
sendToWS(data);
return;
}
sendToWS(data);
});
tab.term = term;
tab.fitAddon = fitAddon;
return term;
}
function switchTerminalTab(id) {
var prevId = currentTabId;
currentTabId = id;
document.querySelectorAll('.terminal-tab').forEach(function (el) {
el.classList.toggle('active', parseInt(el.getAttribute('data-tab-id'), 10) === id);
});
document.querySelectorAll('.terminal-pane').forEach(function (el) {
var paneId = el.getAttribute('id');
var match = paneId && paneId.match(/terminal-pane-(\d+)/);
var paneTabId = match ? parseInt(match[1], 10) : 0;
el.classList.toggle('active', paneTabId === id);
});
var t = getCurrent();
if (t && t.term) {
if (prevId !== id) {
requestAnimationFrame(function () {
if (currentTabId === id && t.term) t.term.focus();
});
} else {
t.term.focus();
}
}
}
function addTerminalTab() {
if (typeof Terminal === 'undefined') return;
tabIdCounter += 1;
var id = tabIdCounter;
var paneId = 'terminal-pane-' + id;
var containerId = 'terminal-container-' + id;
var tabsEl = document.querySelector('.terminal-tabs');
var panesEl = document.querySelector('.terminal-panes');
if (!tabsEl || !panesEl) return;
var tabDiv = document.createElement('div');
tabDiv.className = 'terminal-tab';
tabDiv.setAttribute('data-tab-id', String(id));
var label = document.createElement('span');
label.className = 'terminal-tab-label';
label.textContent = '终端 ' + id;
label.onclick = function () { switchTerminalTab(id); };
var closeBtn = document.createElement('button');
closeBtn.type = 'button';
closeBtn.className = 'terminal-tab-close';
closeBtn.title = '关闭';
closeBtn.textContent = '×';
closeBtn.onclick = function (e) { e.stopPropagation(); removeTerminalTab(id); };
tabDiv.appendChild(label);
tabDiv.appendChild(closeBtn);
var plusBtn = tabsEl.querySelector('.terminal-tab-new');
tabsEl.insertBefore(tabDiv, plusBtn);
var paneDiv = document.createElement('div');
paneDiv.id = paneId;
paneDiv.className = 'terminal-pane';
var containerDiv = document.createElement('div');
containerDiv.id = containerId;
containerDiv.className = 'terminal-container';
paneDiv.appendChild(containerDiv);
panesEl.appendChild(paneDiv);
var tab = { id: id, paneId: paneId, containerId: containerId, lineBuffer: '', cursorIndex: 0, running: false, term: null, fitAddon: null, history: [], historyIndex: -1 };
terminals.push(tab);
createTerminalInContainer(containerDiv, tab);
switchTerminalTab(id);
updateTerminalTabCloseVisibility();
setTimeout(function () {
try { if (tab.fitAddon) tab.fitAddon.fit(); if (tab.term) tab.term.focus(); } catch (e) {}
}, 50);
}
function updateTerminalTabCloseVisibility() {
var tabsEl = document.querySelector('.terminal-tabs');
if (!tabsEl) return;
var tabDivs = tabsEl.querySelectorAll('.terminal-tab');
var showClose = terminals.length > 1;
for (var i = 0; i < tabDivs.length; i++) {
var btn = tabDivs[i].querySelector('.terminal-tab-close');
if (btn) btn.style.display = showClose ? '' : 'none';
}
}
function removeTerminalTab(id) {
if (terminals.length <= 1) return;
var idx = -1;
for (var i = 0; i < terminals.length; i++) { if (terminals[i].id === id) { idx = i; break; } }
if (idx < 0) return;
var deletingCurrent = (currentTabId === id);
var switchToIndex = deletingCurrent ? (idx > 0 ? idx - 1 : 0) : -1;
var tab = terminals[idx];
if (tab.term && tab.term.dispose) tab.term.dispose();
tab.term = null;
tab.fitAddon = null;
terminals.splice(idx, 1);
var tabDiv = document.querySelector('.terminal-tab[data-tab-id="' + id + '"]');
var paneDiv = document.getElementById('terminal-pane-' + id);
if (tabDiv && tabDiv.parentNode) tabDiv.parentNode.removeChild(tabDiv);
if (paneDiv && paneDiv.parentNode) paneDiv.parentNode.removeChild(paneDiv);
var curIdxBeforeRenumber = -1;
if (!deletingCurrent) {
for (var i = 0; i < terminals.length; i++) {
if (terminals[i].id === currentTabId) { curIdxBeforeRenumber = i; break; }
}
}
for (var i = 0; i < terminals.length; i++) {
var t = terminals[i];
t.id = i + 1;
t.paneId = 'terminal-pane-' + (i + 1);
t.containerId = 'terminal-container-' + (i + 1);
}
tabIdCounter = terminals.length;
if (curIdxBeforeRenumber >= 0) currentTabId = terminals[curIdxBeforeRenumber].id;
var tabsEl = document.querySelector('.terminal-tabs');
var panesEl = document.querySelector('.terminal-panes');
if (tabsEl) {
var tabDivs = tabsEl.querySelectorAll('.terminal-tab');
for (var i = 0; i < tabDivs.length; i++) {
var t = terminals[i];
tabDivs[i].setAttribute('data-tab-id', String(t.id));
var lbl = tabDivs[i].querySelector('.terminal-tab-label');
if (lbl) lbl.textContent = '终端 ' + t.id;
if (lbl) lbl.onclick = (function (tid) { return function () { switchTerminalTab(tid); }; })(t.id);
var cb = tabDivs[i].querySelector('.terminal-tab-close');
if (cb) cb.onclick = (function (tid) { return function (e) { e.stopPropagation(); removeTerminalTab(tid); }; })(t.id);
}
}
if (panesEl) {
var paneDivs = panesEl.querySelectorAll('.terminal-pane');
for (var i = 0; i < paneDivs.length; i++) {
var t = terminals[i];
paneDivs[i].id = t.paneId;
var cont = paneDivs[i].querySelector('.terminal-container');
if (cont) cont.id = t.containerId;
}
}
updateTerminalTabCloseVisibility();
if (deletingCurrent && terminals.length > 0) {
currentTabId = terminals[switchToIndex].id;
switchTerminalTab(currentTabId);
}
}
function initTerminal() {
var pane1 = document.getElementById('terminal-pane-1');
var container1 = document.getElementById('terminal-container-1');
if (!pane1 || !container1) return;
if (inited) {
var t = getCurrent();
if (t && t.term) t.term.focus();
terminals.forEach(function (tab) { try { if (tab.fitAddon) tab.fitAddon.fit(); } catch (e) {} });
return;
}
inited = true;
if (typeof Terminal === 'undefined') {
container1.innerHTML = '<p class="terminal-error">未加载 xterm.js,请刷新页面或检查网络。</p>';
return;
}
currentTabId = 1;
var tab = { id: 1, paneId: 'terminal-pane-1', containerId: 'terminal-container-1', lineBuffer: '', cursorIndex: 0, running: false, term: null, fitAddon: null, history: [], historyIndex: -1 };
terminals.push(tab);
createTerminalInContainer(container1, tab);
updateTerminalTabCloseVisibility();
setTimeout(function () {
try { if (tab.fitAddon) tab.fitAddon.fit(); if (tab.term) tab.term.focus(); } catch (e) {}
}, 100);
var resizeTimer;
window.addEventListener('resize', function () {
clearTimeout(resizeTimer);
resizeTimer = setTimeout(function () {
terminals.forEach(function (t) { try { if (t.fitAddon) t.fitAddon.fit(); } catch (e) {} });
}, 150);
});
}
function terminalClear() {
var t = getCurrent();
if (!t || !t.term) return;
t.term.clear();
t.lineBuffer = '';
if (t.cursorIndex !== undefined) t.cursorIndex = 0;
writePrompt(t);
t.term.focus();
}
window.initTerminal = initTerminal;
window.terminalClear = terminalClear;
window.switchTerminalTab = switchTerminalTab;
window.addTerminalTab = addTerminalTab;
window.removeTerminalTab = removeTerminalTab;
})();
+210 -9
View File
@@ -7,6 +7,7 @@
<link rel="icon" type="image/png" href="/static/logo.png">
<link rel="shortcut icon" type="image/png" href="/static/favicon.ico">
<link rel="stylesheet" href="/static/css/style.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/xterm@4.19.0/css/xterm.css">
</head>
<body>
<div id="login-overlay" class="login-overlay" style="display: none;">
@@ -499,7 +500,7 @@
</div>
<div id="active-tasks-bar" class="active-tasks-bar"></div>
<div id="chat-messages" class="chat-messages"></div>
<div class="chat-input-container">
<div id="chat-input-container" class="chat-input-container">
<div class="role-selector-wrapper">
<button id="role-selector-btn" class="role-selector-btn" onclick="toggleRoleSelectionPanel()" title="选择角色">
<span id="role-selector-icon" class="role-selector-icon">🔵</span>
@@ -521,10 +522,19 @@
<div id="role-selection-list" class="role-selection-list-main"></div>
</div>
</div>
<div class="chat-input-field">
<textarea id="chat-input" placeholder="输入测试目标或命令... (输入 @ 选择工具 | Shift+Enter 换行,Enter 发送)" rows="1"></textarea>
<div id="mention-suggestions" class="mention-suggestions" role="listbox" aria-label="工具提及候选"></div>
<div class="chat-input-with-files">
<div id="chat-file-list" class="chat-file-list" aria-label="已选文件列表"></div>
<div class="chat-input-field">
<textarea id="chat-input" placeholder="输入测试目标或命令... (输入 @ 选择工具 | Shift+Enter 换行,Enter 发送)" rows="1"></textarea>
<div id="mention-suggestions" class="mention-suggestions" role="listbox" aria-label="工具提及候选"></div>
</div>
</div>
<input type="file" id="chat-file-input" class="chat-file-input-hidden" multiple accept="*" title="选择文件">
<button type="button" class="chat-upload-btn" onclick="document.getElementById('chat-file-input').click()" title="上传文件(可多选或拖拽到此处)">
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4M17 8l-5-5-5 5M12 3v12" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
</svg>
</button>
<button class="send-btn" onclick="sendMessage()">
<span>发送</span>
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
@@ -756,6 +766,15 @@
<button class="preset-chip" type="button" onclick="applyFofaQueryPreset('ip=&quot;1.1.1.1&quot;')" title="填入示例">指定 IP</button>
</div>
</div>
<div class="form-group">
<label for="fofa-nl">自然语言(AI 解析为 FOFA 语法)</label>
<div class="info-collect-nl-row">
<textarea id="fofa-nl" class="info-collect-query-input" rows="1" placeholder="例如:找美国 Missouri 的 Apache 站点,标题包含 Home"></textarea>
<button id="fofa-nl-parse-btn" class="btn-secondary" type="button" onclick="parseFofaNaturalLanguage()" title="将自然语言解析为 FOFA 查询语法">AI 解析</button>
</div>
<div id="fofa-nl-status" class="fofa-nl-status muted" style="display: none;" aria-live="polite"></div>
<small class="form-hint">解析后会弹窗展示 FOFA 语法(可编辑),确认无误后再填入查询框并执行查询。</small>
</div>
<div class="info-collect-form-row">
<div class="form-group">
<label for="fofa-size">返回数量</label>
@@ -794,8 +813,9 @@
<div class="info-collect-results-toolbar" aria-label="结果工具条">
<div class="info-collect-selected" id="fofa-selected-meta">已选择 0 条</div>
<button class="btn-secondary btn-small" type="button" onclick="toggleFofaColumnsPanel()" title="显示/隐藏字段"></button>
<button class="btn-secondary btn-small" type="button" onclick="exportFofaResults('csv')" title="导出当前结果为 CSV">导出 CSV</button>
<button class="btn-secondary btn-small" type="button" onclick="exportFofaResults('csv')" title="导出当前结果为 CSVUTF-8,兼容中文)">导出 CSV</button>
<button class="btn-secondary btn-small" type="button" onclick="exportFofaResults('json')" title="导出当前结果为 JSON">导出 JSON</button>
<button class="btn-secondary btn-small" type="button" onclick="exportFofaResults('xlsx')" title="导出当前结果为 Excel">导出 XLSX</button>
<button class="btn-primary btn-small" type="button" onclick="batchScanSelectedFofaRows()" title="将所选行创建为批量任务队列">批量扫描</button>
</div>
</div>
@@ -1052,6 +1072,12 @@
<div class="settings-nav-item active" data-section="basic" onclick="switchSettingsSection('basic')">
<span>基本设置</span>
</div>
<div class="settings-nav-item" data-section="robots" onclick="switchSettingsSection('robots')">
<span>机器人设置</span>
</div>
<div class="settings-nav-item" data-section="terminal" onclick="switchSettingsSection('terminal')">
<span>终端</span>
</div>
<div class="settings-nav-item" data-section="security" onclick="switchSettingsSection('security')">
<span>安全设置</span>
</div>
@@ -1177,6 +1203,157 @@
<small class="form-hint">向量检索的权重(0-1),1.0表示纯向量检索,0.0表示纯关键词检索</small>
</div>
</div>
<div class="settings-subsection-header">
<h5>索引配置</h5>
</div>
<div class="form-group">
<label for="knowledge-indexing-chunk-size">分块大小(Chunk Size</label>
<input type="number" id="knowledge-indexing-chunk-size" min="128" max="4096" placeholder="512" />
<small class="form-hint">每个块的最大 token 数(默认 512),长文本会被分割成多个块</small>
</div>
<div class="form-group">
<label for="knowledge-indexing-chunk-overlap">分块重叠(Chunk Overlap</label>
<input type="number" id="knowledge-indexing-chunk-overlap" min="0" max="512" placeholder="50" />
<small class="form-hint">块之间的重叠 token 数(默认 50),保持上下文连贯性</small>
</div>
<div class="form-group">
<label for="knowledge-indexing-max-chunks-per-item">单个知识项最大块数</label>
<input type="number" id="knowledge-indexing-max-chunks-per-item" min="0" max="1000" placeholder="0" />
<small class="form-hint">单个知识项的最大块数量(0 表示不限制),防止单个文件消耗过多 API 配额</small>
</div>
<div class="form-group">
<label for="knowledge-indexing-max-rpm">每分钟最大请求数(Max RPM</label>
<input type="number" id="knowledge-indexing-max-rpm" min="0" max="1000" placeholder="0" />
<small class="form-hint">每分钟最大请求数(默认 0 表示不限制),如 OpenAI 默认 200 RPM</small>
</div>
<div class="form-group">
<label for="knowledge-indexing-rate-limit-delay-ms">请求间隔延迟(毫秒)</label>
<input type="number" id="knowledge-indexing-rate-limit-delay-ms" min="0" max="10000" placeholder="300" />
<small class="form-hint">请求间隔毫秒数(默认 300),用于避免 API 速率限制,设为 0 不限制</small>
</div>
<div class="form-group">
<label for="knowledge-indexing-max-retries">最大重试次数</label>
<input type="number" id="knowledge-indexing-max-retries" min="0" max="10" placeholder="3" />
<small class="form-hint">最大重试次数(默认 3),遇到速率限制或服务器错误时自动重试</small>
</div>
<div class="form-group">
<label for="knowledge-indexing-retry-delay-ms">重试间隔(毫秒)</label>
<input type="number" id="knowledge-indexing-retry-delay-ms" min="0" max="10000" placeholder="1000" />
<small class="form-hint">重试间隔毫秒数(默认 1000),每次重试会递增延迟</small>
</div> </div>
<div class="settings-actions">
<button class="btn-primary" onclick="applySettings()">应用配置</button>
</div>
</div>
<!-- 机器人设置 -->
<div id="settings-section-robots" class="settings-section-content">
<div class="settings-section-header">
<h3>机器人设置</h3>
<p class="settings-description">配置企业微信、钉钉、飞书等机器人,在手机端直接与 CyberStrikeAI 对话,无需在服务器上打开网页。</p>
</div>
<!-- 企业微信 -->
<div class="settings-subsection">
<h4>企业微信</h4>
<div class="settings-form">
<div class="form-group">
<label class="checkbox-label">
<input type="checkbox" id="robot-wecom-enabled" class="modern-checkbox" />
<span class="checkbox-custom"></span>
<span class="checkbox-text">启用企业微信机器人</span>
</label>
</div>
<div class="form-group">
<label for="robot-wecom-token">Token</label>
<input type="text" id="robot-wecom-token" placeholder="Token" autocomplete="off" />
</div>
<div class="form-group">
<label for="robot-wecom-encoding-aes-key">EncodingAESKey</label>
<input type="text" id="robot-wecom-encoding-aes-key" placeholder="EncodingAESKey(明文模式可留空)" autocomplete="off" />
</div>
<div class="form-group">
<label for="robot-wecom-corp-id">CorpID</label>
<input type="text" id="robot-wecom-corp-id" placeholder="企业 ID" autocomplete="off" />
</div>
<div class="form-group">
<label for="robot-wecom-secret">Secret</label>
<input type="password" id="robot-wecom-secret" placeholder="应用 Secret" autocomplete="off" />
</div>
<div class="form-group">
<label for="robot-wecom-agent-id">AgentID</label>
<input type="number" id="robot-wecom-agent-id" placeholder="应用 AgentId" />
</div>
</div>
</div>
<!-- 钉钉 -->
<div class="settings-subsection">
<h4>钉钉</h4>
<div class="settings-form">
<div class="form-group">
<label class="checkbox-label">
<input type="checkbox" id="robot-dingtalk-enabled" class="modern-checkbox" />
<span class="checkbox-custom"></span>
<span class="checkbox-text">启用钉钉机器人</span>
</label>
</div>
<div class="form-group">
<label for="robot-dingtalk-client-id">Client ID (AppKey)</label>
<input type="text" id="robot-dingtalk-client-id" placeholder="钉钉应用 AppKey" autocomplete="off" />
</div>
<div class="form-group">
<label for="robot-dingtalk-client-secret">Client Secret</label>
<input type="password" id="robot-dingtalk-client-secret" placeholder="钉钉应用 Secret" autocomplete="off" />
<small class="form-hint">需开启机器人能力并配置流式接入</small>
</div>
</div>
</div>
<!-- 飞书 -->
<div class="settings-subsection">
<h4>飞书 (Lark)</h4>
<div class="settings-form">
<div class="form-group">
<label class="checkbox-label">
<input type="checkbox" id="robot-lark-enabled" class="modern-checkbox" />
<span class="checkbox-custom"></span>
<span class="checkbox-text">启用飞书机器人</span>
</label>
</div>
<div class="form-group">
<label for="robot-lark-app-id">App ID</label>
<input type="text" id="robot-lark-app-id" placeholder="飞书应用 App ID" autocomplete="off" />
</div>
<div class="form-group">
<label for="robot-lark-app-secret">App Secret</label>
<input type="password" id="robot-lark-app-secret" placeholder="飞书应用 App Secret" autocomplete="off" />
</div>
<div class="form-group">
<label for="robot-lark-verify-token">Verify Token(可选)</label>
<input type="text" id="robot-lark-verify-token" placeholder="事件订阅 Verification Token" autocomplete="off" />
</div>
</div>
</div>
<div class="settings-subsection">
<h4>机器人命令说明</h4>
<p class="settings-description">在对话中可发送以下命令(支持中英文):</p>
<ul style="color: var(--text-muted); font-size: 13px; line-height: 1.8; margin: 8px 0 0 16px;">
<li><code>帮助</code> <code>help</code> — 显示本帮助 | Show this help</li>
<li><code>列表</code> <code>list</code> — 列出所有对话标题与 ID | List conversations</li>
<li><code>切换 &lt;ID&gt;</code> <code>switch &lt;ID&gt;</code> — 指定对话继续 | Switch to conversation</li>
<li><code>新对话</code> <code>new</code> — 开启新对话 | Start new conversation</li>
<li><code>清空</code> <code>clear</code> — 清空当前上下文 | Clear context</li>
<li><code>当前</code> <code>current</code> — 显示当前对话 ID 与标题 | Show current conversation</li>
<li><code>停止</code> <code>stop</code> — 中断当前任务 | Stop running task</li>
<li><code>角色</code> <code>roles</code> — 列出所有可用角色 | List roles</li>
<li><code>角色 &lt;&gt;</code> <code>role &lt;name&gt;</code> — 切换当前角色 | Switch role</li>
<li><code>删除 &lt;ID&gt;</code> <code>delete &lt;ID&gt;</code> — 删除指定对话 | Delete conversation</li>
<li><code>版本</code> <code>version</code> — 显示当前版本号 | Show version</li>
</ul>
<p class="settings-description" style="margin-top: 8px;">除以上命令外,直接输入内容将发送给 AI 进行渗透测试/安全分析。Otherwise, send any text for AI penetration testing / security analysis.</p>
</div>
<div class="settings-actions">
@@ -1184,6 +1361,25 @@
</div>
</div>
<!-- 终端 -->
<div id="settings-section-terminal" class="settings-section-content">
<div class="settings-section-header">
<h3>终端</h3>
<p class="settings-description">在服务器上执行命令,便于运维与调试。命令在服务端执行,请勿执行敏感或破坏性操作。</p>
</div>
<div class="terminal-wrapper">
<div class="terminal-tabs">
<div class="terminal-tab active" data-tab-id="1"><span class="terminal-tab-label" onclick="switchTerminalTab(1)">终端 1</span><button type="button" class="terminal-tab-close" onclick="removeTerminalTab(1); event.stopPropagation();" title="关闭">×</button></div>
<button type="button" class="terminal-tab-new" onclick="addTerminalTab()" title="新终端">+</button>
</div>
<div class="terminal-panes">
<div id="terminal-pane-1" class="terminal-pane active">
<div id="terminal-container-1" class="terminal-container"></div>
</div>
</div>
</div>
</div>
<!-- 安全设置 -->
<div id="settings-section-security" class="settings-section-content">
<div class="settings-section-header">
@@ -1463,6 +1659,8 @@
<script src="https://cdn.jsdelivr.net/npm/cytoscape@3.27.0/dist/cytoscape.min.js"></script>
<!-- ELK.js for high-quality DAG layout (reduces edge crossings) -->
<script src="https://cdn.jsdelivr.net/npm/elkjs@0.9.2/lib/elk.bundled.js"></script>
<!-- SheetJS for XLSX export (info-collect) -->
<script src="https://cdn.jsdelivr.net/npm/xlsx@0.18.5/dist/xlsx.full.min.js"></script>
<script>
// 确保ELK对象全局可用
if (typeof ELK === 'undefined' && typeof elk !== 'undefined') {
@@ -1761,10 +1959,10 @@ version: 1.0.0<br>
<h2 id="batch-queue-detail-title">批量任务队列详情</h2>
<div style="display: flex; align-items: center; gap: 12px;">
<div class="modal-header-actions">
<button class="btn-secondary" id="batch-queue-add-task-btn" onclick="showAddBatchTaskModal()" style="display: none;">添加任务</button>
<button class="btn-primary" id="batch-queue-start-btn" onclick="startBatchQueue()" style="display: none;">开始执行</button>
<button class="btn-secondary" id="batch-queue-pause-btn" onclick="pauseBatchQueue()" style="display: none;">暂停队列</button>
<button class="btn-secondary btn-danger" id="batch-queue-delete-btn" onclick="deleteBatchQueue()" style="display: none;">删除队列</button>
<button class="btn-secondary btn-small" id="batch-queue-add-task-btn" onclick="showAddBatchTaskModal()" style="display: none;">添加任务</button>
<button class="btn-primary btn-small" id="batch-queue-start-btn" onclick="startBatchQueue()" style="display: none;">开始执行</button>
<button class="btn-secondary btn-small" id="batch-queue-pause-btn" onclick="pauseBatchQueue()" style="display: none;">暂停队列</button>
<button class="btn-secondary btn-small btn-danger" id="batch-queue-delete-btn" onclick="deleteBatchQueue()" style="display: none;">删除队列</button>
</div>
<span class="modal-close" onclick="closeBatchQueueDetailModal()">&times;</span>
</div>
@@ -2006,6 +2204,9 @@ version: 1.0.0<br>
<script src="/static/js/monitor.js"></script>
<script src="/static/js/chat.js"></script>
<script src="/static/js/settings.js"></script>
<script src="https://cdn.jsdelivr.net/npm/xterm@4.19.0/lib/xterm.js"></script>
<script src="https://cdn.jsdelivr.net/npm/xterm-addon-fit@0.5.0/lib/xterm-addon-fit.js"></script>
<script src="/static/js/terminal.js"></script>
<script src="/static/js/knowledge.js"></script>
<script src="/static/js/skills.js"></script>
<script src="/static/js/vulnerability.js?v=4"></script>
File diff suppressed because it is too large Load Diff