feat: add runtime storage cleanup with per-category retention (#310) (#313)

Runtime artifacts (agent workspaces, tool-output spill, C2 payloads, chat
uploads, workflow checkpoints, diagnostic logs) previously accumulated
without bound: most were only removed when a conversation or project was
deleted, and tmp/c2 plus workflow checkpoints were never removed at all.

Add a storage cleaner with named per-category tasks (Gitea-style), a
settings page tab, and a background sweep that is off by default so
upgrading never deletes existing data.

Safety properties, since mis-deleting live task data costs far more than
the disk saved:
- dry-run is the default; a real cleanup requires dry_run=false together
  with confirm=true at the API layer, not just a frontend dialog
- sessions active within active_grace_hours are always skipped, and a
  failed activity lookup skips conservatively (fail closed)
- directories whose conversation/project no longer exists are reclaimed
  as orphans after orphan_grace_days
- scanners never follow symlinks and every candidate path is confined to
  its category root; deletion renames to a .tmp-for-deletion marker first
  so a crash leaves recoverable residue instead of a half-deleted dir
- storage:* permissions are admin-only; without the grantSystemRolePermissions
  skip the default branch would have given operators an irreversible
  file-deletion right

Also fix two confirmed leaks: DeleteConversation left chat_uploads files
on disk (their rows already vanished via ON DELETE CASCADE), and workflow
checkpoints had no deletion path at all.

Co-authored-by: Parallels <parallels@kali-linux-2025-2.localdomain>
This commit is contained in:
SycunandParallels authored and GitHub committed 2026-09-25 14:46:23 +08:00
1 parent e9b6e0d86e
commit 470eb5ead1
28 files changed
+3356 -1

No files matched your search

+97
View File
@@ -3468,6 +3468,103 @@ func (h *OpenAPIHandler) GetOpenAPISpec(c *gin.Context) {
},
},
},
"/api/storage/meta": map[string]interface{}{
"get": map[string]interface{}{
"tags": []string{"存储清理"},
"summary": "获取存储清理策略",
"description": "返回自动清理开关、间隔、宽限窗口与全部清理类别的启用状态及保留天数",
"operationId": "getStorageMeta",
"responses": map[string]interface{}{
"200": map[string]interface{}{
"description": "获取成功",
},
"401": map[string]interface{}{
"description": "未授权",
},
"403": map[string]interface{}{
"description": "缺少 storage:read 权限",
},
},
},
},
"/api/storage/status": map[string]interface{}{
"get": map[string]interface{}{
"tags": []string{"存储清理"},
"summary": "获取运行空间占用",
"description": "返回文件系统容量(含 inode)与各类别的占用、可回收量;结果按短 TTL 缓存",
"operationId": "getStorageStatus",
"parameters": []interface{}{
map[string]interface{}{
"name": "refresh",
"in": "query",
"description": "为 true 时强制重新遍历目录,绕过缓存",
"required": false,
"schema": map[string]interface{}{"type": "boolean"},
},
},
"responses": map[string]interface{}{
"200": map[string]interface{}{
"description": "获取成功",
},
"401": map[string]interface{}{
"description": "未授权",
},
"403": map[string]interface{}{
"description": "缺少 storage:read 权限",
},
},
},
},
"/api/storage/cleanup": map[string]interface{}{
"post": map[string]interface{}{
"tags": []string{"存储清理"},
"summary": "预览或执行运行空间清理",
"description": "默认 dry_run=true 只统计不删除;真实删除必须同时传 dry_run=false 与 confirm=true。仅处理已启用的类别,同一时刻只允许一轮执行",
"operationId": "runStorageCleanup",
"requestBody": map[string]interface{}{
"required": false,
"content": map[string]interface{}{
"application/json": map[string]interface{}{
"schema": map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"dry_run": map[string]interface{}{
"type": "boolean",
"description": "省略时按 true 处理",
},
"confirm": map[string]interface{}{
"type": "boolean",
"description": "dry_run=false 时必须为 true",
},
"categories": map[string]interface{}{
"type": "array",
"description": "限定类别;省略表示全部已启用类别",
"items": map[string]interface{}{"type": "string"},
},
},
},
},
},
},
"responses": map[string]interface{}{
"200": map[string]interface{}{
"description": "执行完成(或预览完成)",
},
"400": map[string]interface{}{
"description": "缺少 confirm、或类别键未注册",
},
"401": map[string]interface{}{
"description": "未授权",
},
"403": map[string]interface{}{
"description": "缺少 storage:write 权限",
},
"409": map[string]interface{}{
"description": "已有一轮清理在执行",
},
},
},
},
"/api/config/tools": map[string]interface{}{
"get": map[string]interface{}{
"tags": []string{"配置管理"},