feat: add runtime storage cleanup with per-category retention (#310) (#313)

Runtime artifacts (agent workspaces, tool-output spill, C2 payloads, chat
uploads, workflow checkpoints, diagnostic logs) previously accumulated
without bound: most were only removed when a conversation or project was
deleted, and tmp/c2 plus workflow checkpoints were never removed at all.

Add a storage cleaner with named per-category tasks (Gitea-style), a
settings page tab, and a background sweep that is off by default so
upgrading never deletes existing data.

Safety properties, since mis-deleting live task data costs far more than
the disk saved:
- dry-run is the default; a real cleanup requires dry_run=false together
  with confirm=true at the API layer, not just a frontend dialog
- sessions active within active_grace_hours are always skipped, and a
  failed activity lookup skips conservatively (fail closed)
- directories whose conversation/project no longer exists are reclaimed
  as orphans after orphan_grace_days
- scanners never follow symlinks and every candidate path is confined to
  its category root; deletion renames to a .tmp-for-deletion marker first
  so a crash leaves recoverable residue instead of a half-deleted dir
- storage:* permissions are admin-only; without the grantSystemRolePermissions
  skip the default branch would have given operators an irreversible
  file-deletion right

Also fix two confirmed leaks: DeleteConversation left chat_uploads files
on disk (their rows already vanished via ON DELETE CASCADE), and workflow
checkpoints had no deletion path at all.

Co-authored-by: Parallels <parallels@kali-linux-2025-2.localdomain>
This commit is contained in:
SycunandParallels authored and GitHub committed 2026-09-25 14:46:23 +08:00
1 parent e9b6e0d86e
commit 470eb5ead1
28 files changed
+3356 -1

No files matched your search

+35
View File
@@ -48,6 +48,41 @@ audit:
# MCP 状态监控执行记录保留(tool_executions 表)
monitor:
retention_days: 90 # 省略时默认 90;0 表示不自动清理
# 运行空间垃圾清理(系统设置 -> 存储清理)
# 覆盖 Agent 工作区、工具输出缓存、C2 产物、对话上传件等磁盘产物的保留策略。
# auto_clean 默认关闭:升级后不会在管理员不知情的情况下删除既有数据;
# 关闭时仍可在「存储清理」页手动预览并执行清理。
storage:
auto_clean: false # true 开启后台定时清理
interval_minutes: 60 # 后台清理间隔(分钟);最小 5
orphan_grace_days: 1 # 会话/项目已删除但目录残留时,闲置多少天后回收
active_grace_hours: 24 # 最近有活动的会话一律跳过,避免误删正在跑的任务数据
# 各类别 retention_days:省略时使用下方默认值;0 表示不按保留期清理(孤儿目录仍会回收)
categories:
workspace: # Agent 工作区 tmp/workspace;默认 30 天
enabled: true
retention_days: 30
reduction: # 超长工具输出落盘 tmp/reduction;默认 7 天(纯派生数据)
enabled: true
retention_days: 7
conversation_artifacts: # 摘要与超长输入台账 data/conversation_artifacts;默认 30 天
enabled: true
retention_days: 30
plantask: # 多代理计划看板 skills/.eino/plantask;默认 30 天(纯派生数据)
enabled: true
retention_days: 30
c2_artifacts: # C2 回传截图/上传件/下发文件/payload tmp/c2;默认 30 天
enabled: true # 注意:清理 payload 后对应下载链接会失效
retention_days: 30
chat_uploads: # 对话上传附件 chat_uploads;默认 90 天
enabled: true
retention_days: 90
workflow_checkpoints: # 工作流断点 data/workflow-checkpoints;默认 7 天
enabled: true
retention_days: 7
diagnostic_logs: # 诊断日志 log/diagnostic-*.log;默认 14 天
enabled: true # 与 log.diagnostic_retention_days 各自独立生效,取先到者
retention_days: 14
# ============================================
# 对话相关配置
# ============================================