diff --git a/docs/README.md b/docs/README.md index 1dce7639..69137466 100644 --- a/docs/README.md +++ b/docs/README.md @@ -64,6 +64,8 @@ CyberStrikeAI documentation is organized by user journey. Start with deployment, - [Agents and Roles](en-US/agent-and-role-guide.md) - [Skills](en-US/skills-guide.md) - [Eino Multi-Agent](en-US/MULTI_AGENT_EINO.md) +- [Blackboard and Context Memory](en-US/blackboard-context-memory.md) +- [Agent Finalization](en-US/agent-finalization-best-practices.md) - [Workflows](en-US/workflow-graph.md) - [Tool Call Guard](en-US/tool-call-guard.md) - [Tool Execution Governance](en-US/tool-execution-governance.md) @@ -72,6 +74,8 @@ CyberStrikeAI documentation is organized by user journey. Start with deployment, ### Feature guides - [Asset Management](en-US/asset-management.md) +- [Resuming Project Tasks](en-US/project-resume.md) +- [Token Usage Analysis](en-US/token-usage-monitoring.md) - [Knowledge Base](en-US/knowledge-base.md) - [RBAC Administration](en-US/rbac.md) - [Robot / Chatbot](en-US/robot.md) @@ -98,11 +102,5 @@ CyberStrikeAI documentation is organized by user journey. Start with deployment, - 公开文档面向用户和贡献者,保留使用指南、稳定接口与架构说明、兼容性说明和发布记录。内部任务阶段记录、对话确认、一次性核查表和本地验证流水账应保存在仓库外,不作为产品文档提交。 - Public docs serve users and contributors: usage guides, stable API and architecture references, compatibility notes, and release records. Keep internal task journals, conversation approvals, one-off review sheets, and local verification logs outside the repository. -- 中英文目录以各语言索引为准。仅有中文版本的指南会在下方明确标注,英文译文尚未提供。 - -### Additional guides (Chinese only) - -- [Agent finalization](zh-CN/agent-finalization-best-practices.md) -- [Blackboard and context memory](zh-CN/blackboard-context-memory.md) -- [Resuming project tasks](zh-CN/project-resume.md) -- [Token usage analysis](zh-CN/token-usage-monitoring.md) +- 中英文指南使用相同文件名,并提供语言切换链接;各语言索引覆盖对应的全部指南。 +- Chinese and English guides use matching filenames and language-switch links; each locale index covers all guides in that language. diff --git a/docs/en-US/README.md b/docs/en-US/README.md index ccd09749..237d12eb 100644 --- a/docs/en-US/README.md +++ b/docs/en-US/README.md @@ -12,12 +12,14 @@ ## Concepts and orchestration - [Architecture](architecture.md) · [Security Model](security-model.md) · [RBAC](rbac.md) -- [Agents and Roles](agent-and-role-guide.md) · [Skills](skills-guide.md) · [Eino Multi-Agent](MULTI_AGENT_EINO.md) +- [Agents and Roles](agent-and-role-guide.md) · [Skills](skills-guide.md) · [Eino Multi-Agent](MULTI_AGENT_EINO.md) · [Agent Finalization](agent-finalization-best-practices.md) +- [Blackboard and Context Memory](blackboard-context-memory.md) - [Tool Call Guard](tool-call-guard.md) - [Workflows](workflow-graph.md) · [Tool Execution Governance](tool-execution-governance.md) · [HITL Best Practices](hitl-best-practices.md) ## Feature guides +- [Resuming Project Tasks](project-resume.md) · [Token Usage Analysis](token-usage-monitoring.md) - [Asset Management](asset-management.md) · [Knowledge Base](knowledge-base.md) · [Robot / Chatbot](robot.md) · [Vision](VISION.md) - [WebShell](webshell.md) · [C2](c2.md) · [MCP Federation](mcp-federation.md) @@ -31,12 +33,3 @@ - [Developer Guide](developer-guide.md) · [Plugin Development](plugin-development.md) · [Frontend i18n](frontend-i18n.md) - [Testing](testing.md) · [Contributing](contributing-guide.md) · [Release Process](release-process.md) - -## Additional guides (Chinese only) - -English translations of these guides are not yet available: - -- [Agent finalization](../zh-CN/agent-finalization-best-practices.md) -- [Blackboard and context memory](../zh-CN/blackboard-context-memory.md) -- [Resuming project tasks](../zh-CN/project-resume.md) -- [Token usage analysis](../zh-CN/token-usage-monitoring.md) diff --git a/docs/en-US/agent-finalization-best-practices.md b/docs/en-US/agent-finalization-best-practices.md new file mode 100644 index 00000000..5313b5b6 --- /dev/null +++ b/docs/en-US/agent-finalization-best-practices.md @@ -0,0 +1,335 @@ +# Agent final-response governance best practices + +[English documentation](README.md) | [中文](../zh-CN/agent-finalization-best-practices.md) + +Research date: 2026-07-28. + +This guide addresses a specific failure: an Agent produces natural language that sounds conclusive before tool calls, reasoning, planning, or sub-agent collaboration have actually finished, and the frontend or orchestrator displays it as the final response. Task completion should be determined by runtime state, tool state, verification results, and explicit terminal events together, rather than by the most recent assistant text. + +## 1. Core conclusions + +1. **A final response is a runtime event, not natural-language content.** + Phrases such as “obtained,” “next step,” or “Huge breakthrough” are candidate observations or progress, not completion signals. + +2. **Separate execution details from delivery.** + `thinking`, `reasoning_chain`, `planning`, `response_delta`, sub-agent replies, and tool output belong to execution details. Only `response` / `final` events that pass the final gate may populate the main message bubble and `messages.content`. + +3. **Complex tasks need a verifier, not a longer prompt.** + Prompts can encourage caution, but code must decide whether tools or background executions are pending, required plan steps remain incomplete, evidence is unverified, facts/vulnerabilities are unrecorded, or cleanup is unfinished without explanation. + +4. **Agent modes differ, but finalization principles are shared.** + Single-agent, Deep, Plan-Execute, and Supervisor modes all need a final gate. Evidence sources differ: single-agent mode checks tool traces; Deep also checks sub-agent results; Plan-Execute checks the Replanner's termination decision; Supervisor checks `exit` and the supervisor's synthesis. + +## 2. Public practices in established Agent systems + +The following summarizes the practices surveyed on the research date, not a new verification of external products. + +| System | Public practice | Implication for finalization | +| --- | --- | --- | +| Codex | OpenAI's Codex prompting guide advises against forcing upfront plans, preambles, or status updates because these can stop a rollout before completion | Model-written plans/status are not completion evidence; the harness owns the execution loop and finalization | +| Claude Code | Provides `PreToolUse`, `PostToolUse`, and `Stop` hooks; `PostToolUse` occurs after successful tool execution | Lifecycle events are more reliable than prose; attach verification, auditing, and blocking to deterministic boundaries | +| Claude Code Subagents | Sub-agents have separate contexts, custom system prompts, specific tool access, and permissions; suitable for isolating extensive retrieval/log/file reads | Sub-agent output is evidence material, not the parent task's final conclusion; the parent must synthesize and verify it | +| Claude Code Plan Mode | Reads files and produces a plan without editing before approval | Planning and execution are distinct states; a completed plan is not a completed task | +| Cursor Plan Mode | Studies the codebase, asks clarifying questions, generates a reviewable plan, and waits for confirmation before building | Separating plan/review/build helps users distinguish plans from delivered results | +| OpenCode | Separates agents such as Build, Plan, Review, Debug, and Docs by tools and purpose; Plan analyzes without making changes | Capability boundaries reduce accidental transitions: planning capability does not imply completed execution | +| Eino ADK | Provides event-driven output, Runner callbacks, interrupts, checkpoints, and collaboration primitives including Supervisor and Plan-Execute; the latter uses Planner, Executor, and Replanner | These primitives fit the project; event-driven behavior needs an explicit finalization contract | + +Primary references from the source guide: + +- [OpenAI Codex Prompting Guide](https://developers.openai.com/cookbook/examples/gpt-5/codex_prompting_guide) +- [Claude Code Hooks](https://docs.anthropic.com/en/docs/claude-code/hooks) +- [Claude Code Subagents](https://docs.anthropic.com/en/docs/claude-code/sub-agents) +- [Claude Code Common Workflows](https://docs.anthropic.com/en/docs/claude-code/common-workflows) +- [Cursor Agent Best Practices](https://cursor.com/blog/agent-best-practices) +- [OpenCode Agents](https://opencode.ai/docs/agents/) +- [CloudWeGo Eino ADK](https://www.cloudwego.io/docs/eino/core_modules/eino_adk/) +- [CloudWeGo Eino ADK Patterns](https://www.cloudwego.io/docs/eino/overview/eino_adk0_1/) + +## 3. General best practices + +### 1. Establish a Finalization Contract + +Every execution entry point should produce one structured finalization object. Only that object may authorize a final response. The following is a conceptual contract: + +```go +type FinalizationDecision struct { + Status string // in_progress | completed | blocked | failed | cancelled + Finalizable bool + CompletionReason string // verified | user_cancelled | timeout | blocked | failed + FinalText string + EvidenceVerified bool + EvidenceRefs []string + PendingToolRuns []string + PendingPlanSteps []string + PendingApprovals []string + MissingChecks []string +} +``` + +Hard rules: + +- When `Finalizable=false`, do not send a `response` as a successful terminal event. The project's blocked-response envelope is described below. +- When `Status=in_progress`, use process events such as `progress`, `planning`, `tool_*`, and `reasoning_chain`. +- `FinalText` must not be empty, but nonempty text alone does not authorize finalization. +- Any nonempty `PendingToolRuns`, `PendingPlanSteps`, or `PendingApprovals` prevents `completed`. +- When `EvidenceVerified=false`, do not present candidate output as a verified conclusion. + +### 2. Define SSE event semantics + +Recommended event layers: + +| Event | Display location | May write `messages.content`? | Meaning | +| --- | --- | --- | --- | +| `progress` | Task state/timeline | No | Short progress update | +| `planning` | Execution details | No | Main-agent plan or interim judgment | +| `reasoning_chain` / `thinking` | Execution details | No | Reasoning/thinking summary | +| `tool_call` / `tool_result` | Execution details | No | Tool events | +| `eino_agent_reply` | Execution details | No | Material returned by a sub-agent | +| `finalization_check` | Execution details | No | Verifier result | +| `finalization_auto_continue` | Execution details | No | Verifier-triggered continuation, `contextInjection=false` | +| `response` | Main message bubble | Yes | Final-response treatment requires `data.finalized=true` | +| `done` | Stream closure | No | The stream ended; it does not establish task success | +| `error` / `cancelled` | Main message bubble or system notice | Yes, as terminal failure | Must include a reason | + +### 3. Separate final candidates from final responses + +The model can produce candidate conclusions. First route them into `final_candidate` or `planning`; the verifier then decides whether to promote them: + +```text +assistant text + -> candidate + -> finalization gate + -> response(finalized=true) +``` + +Avoid direct promotion: + +```text +assistant text + -> response +``` + +### 4. Verify at stop time + +Following the lifecycle-hook approach, run deterministic checks when an agent run stops: + +- Every tool call has a corresponding tool result. +- Background executions are terminal, or explicitly registered as still running with task state `in_progress` / `blocked`. +- Plan-Execute has no unexecuted required steps. +- Supervisor has no unsynthesized sub-agent results. +- An evidence-required policy has at least one queryable completed tool execution. + +### 5. Treat sub-agent output as evidence + +A sub-agent reply cannot directly become the user-facing final response. The main agent must: + +- Deduplicate and reconcile conflicts. +- Rank evidence strength. +- Identify uncertainty. +- Check scope boundaries. +- Produce a readable delivery for the user. + +### 6. Use prompts for guidance and code for enforcement + +A prompt may say: + +```text +Interim observations must be marked as progress, not final. +Do not produce a final answer until verification is complete. +``` + +Backend fields and state machines must still decide finalization. Otherwise, any conclusive-sounding model text could mislead the UI. + +## 4. Current CyberStrikeAI implementation + +The project has an explicit final gate: + +- [internal/agentfinalizer/decision.go](../../internal/agentfinalizer/decision.go) is the single final-response decision contract. +- [internal/handler/finalization_helpers.go](../../internal/handler/finalization_helpers.go) writes decisions to `process_details` and calls `UpdateAssistantMessageFinalize` only when `Finalizable=true`. +- [internal/handler/eino_single_agent.go](../../internal/handler/eino_single_agent.go), [internal/handler/multi_agent.go](../../internal/handler/multi_agent.go), [internal/handler/workflow_integration.go](../../internal/handler/workflow_integration.go), and [internal/handler/batch_queue_executor.go](../../internal/handler/batch_queue_executor.go) invoke the finalizer at completion. +- [web/static/js/monitor.js](../../web/static/js/monitor.js) treats a `response` as final only when `data.finalized === true`. Nonfinalized text is shown as a failed finalization check. +- [web/static/js/webshell.js](../../web/static/js/webshell.js) marks streaming text as candidate output and switches to completed only on `response(finalized=true)`. +- [internal/agentfinalizer/decision_test.go](../../internal/agentfinalizer/decision_test.go) covers pending tools, HITL, empty output, missing required execution evidence, failed evidence that cannot justify finalization, and completed evidence that can. +- [internal/handler/finalization_auto_continue.go](../../internal/handler/finalization_auto_continue.go) automatically continues up to two segments when completed execution evidence is missing. It restores existing model traces without injecting new user/system text. + +Core rules of the implemented contract: + +1. **Model prose is only a candidate.** + `RunResult.Response` must pass through `agentfinalizer.Decide` before becoming a final response. + +2. **Every `response` carries terminal-state fields.** + At minimum: `finalized`, `finalizable`, `status`, `completionReason`, `evidenceVerified`, `evidenceRefs`, `pendingExecutionIds`, and `missingChecks`. + +3. **Unfinished tools block finalization.** + Remaining `queued/running` executions produce `in_progress/pending_tool_executions`. + +4. **Execution evidence is declared through structured policy.** + The backend does not infer execution intent from user prose, assistant replies, or agent-mode names. Chat requests explicitly declare `finalization.requireExecutionEvidence`; execution entry points such as WebShell, Workflow, batch tasks, and robots pass policy at the call site. When required, at least one queryable `completed` tool execution must exist. Failed/cancelled records alone cannot justify finalization. + +5. **Missing evidence triggers continuation before blocking.** + Eino single-agent and main multi-agent paths automatically continue from existing traces on `missing_execution_evidence`, without injecting extra context. They record blocked only if evidence is still missing at the continuation limit. + +6. **HITL and empty output do not finalize.** + Workflows awaiting human approval, empty assistant text, and Eino empty-output placeholders produce blocked messages instead of success summaries. + +## 5. Architecture for this project + +The implemented flow is: + +```text +Agent / Eino ADK events + -> event normalizer + -> process_details + -> finalization verifier + -> response(finalized=true) + -> messages.content +``` + +### 1. A shared backend Finalizer + +Responsibilities: + +- Accept `RunResult` / candidate text, `mcpExecutionIds`, conversation and assistant-message IDs, HITL state, and orchestration mode. +- Query tool execution states in the database, distinguishing pending, completed, failed, and cancelled evidence. +- Return `FinalizationDecision`. +- Check state and evidence without invoking high-risk tools. + +### 2. Terminal fields in RunResult + +[internal/multiagent/runner.go](../../internal/multiagent/runner.go) includes terminal-state fields: + +```go +type RunResult struct { + Response string + MCPExecutionIDs []string + LastAgentTraceInput string + LastAgentTraceOutput string + + Finalized bool + Status string + CompletionReason string + EvidenceVerified bool + EvidenceRefs []string + PendingExecutionIDs []string + MissingChecks []string +} +``` + +### 3. Conditions for sending `response` + +Single-agent, multi-agent, workflow, and batch completion paths use the same pattern. These snippets illustrate control flow rather than standalone compilable code: + +```go +decision := h.finalizeAgentRunForDelivery(...) +if !decision.Finalizable { + sendEvent("finalization_check", "Final-response requirements have not been met", decision) + sendEvent("response", finalizationBlockedMessage(decision), finalizationResponsePayload(decision, extra)) + return +} + +sendEvent("response", decision.FinalText, finalizationResponsePayload(decision, extra)) +``` + +### 4. The frontend trusts only `finalized=true` + +The `case 'response'` branch in [web/static/js/monitor.js](../../web/static/js/monitor.js) enforces the check: + +```js +const responseFinalized = isFinalizedResponseData(responseData); +const bubbleText = responseFinalized + ? resolvedResponseText + : (event.message || 'Final-response requirements have not been met; no success conclusion is available.'); +markAssistantFinalizationState(assistantIdFinal, responseData); +``` + +The same principle applies to WebShell. `response_delta` can preview live text labeled as output from an ongoing execution; only the final `response(finalized=true)` shows completion. + +### 5. Final gates by mode + +| Mode | Final candidate producer | Finalization decision | Required checks | +| --- | --- | --- | --- | +| Eino single-agent | Last assistant text from the agent | Finalizer | No pending tools; complete evidence references; terminal task state | +| Deep | Main-agent synthesis | Finalizer | Sub-agent results synthesized; sub-agent text cannot directly finalize; terminal tool states | +| Plan-Execute | Synthesis after Replanner ends | Replanner + Finalizer | Executor step output cannot finalize; plan steps completed or explicitly blocked | +| Supervisor | Supervisor `exit` / synthesis | Supervisor + Finalizer | Transfers returned; no unhandled specialist results; supervisor owns the final synthesis | + +### 6. Evidence gates for security testing + +For evidence-required scenarios such as security testing, WebShell, batch validation, Workflow, and multi-agent execution, final delivery should satisfy at least these governance requirements: + +- Identify the target and authorization scope. +- Provide reviewable evidence references, such as tool execution IDs, request/response summaries, screenshot paths, command-output summaries, and fact/vulnerability record IDs. +- Validate identity or impact instead of relying only on marker text. +- Record results in the project blackboard or vulnerability store, or explicitly state that no project is bound and recording is unavailable. +- Clean up, roll back, or cancel high-risk actions, or explain why cleanup was not performed. +- Do not implicitly treat still-running scans, commands, WebShell, or C2 tasks as completed. + +These are governance rules, not a requirement to expose sensitive exploitation details in final reports. Evidence summaries and internal references are sufficient. + +## 6. Implementation status and future improvements + +### P0: Prevent false finalization (implemented) + +1. `FinalizationDecision` is in place. +2. Main Agent SSE `response` events carry fields such as `data.finalized/finalizable/status/completionReason`. +3. `monitor.js` and `webshell.js` distinguish candidate output from final responses using `finalized=true`. +4. `RunResult.Response` retains its compatible field name, but promotion is controlled by the finalizer. A future split into `CandidateResponse` / `FinalResponse` could reduce misuse. +5. Plan-Execute, Deep, Supervisor, and Eino Single use the unified handler completion gate. + +### P1: Strengthen the evidence chain (partly implemented) + +1. `mcp_execution:` is the basic evidence reference. +2. `finalization_check` events expose pending executions and missing checks. +3. Execution entry points enable explicit evidence policy. When policy requires evidence but completed tool evidence is missing, the Eino main path continues without context injection and blocks only after reaching the limit. +4. Future work: finer-grained evidence references for `record_vulnerability`, `upsert_project_fact`, and blackboard records. +5. Future work: final report templates with conclusions, evidence, risks/uncertainty, and next actions. + +### P2: Experience and observability (future work) + +1. Display `in_progress / verifying / finalizing / completed / blocked` on task cards. +2. Add finalizer logs and metrics: false-block rate, missing-evidence types, and pending-tool counts. +3. Offer a “Continue verification” button that builds the next input from `FinalizationDecision.MissingChecks`. + +## 7. Suggested acceptance tests + +Include at least these regressions: + +1. **Reasoning text cannot finalize.** + Simulate a conclusive-sounding candidate in `reasoning_chain` without completed tool evidence in the current turn. The main bubble should show ongoing or blocked state, not success. + +2. **Interim main-agent output cannot finalize.** + Simulate “continue verification next” in `response_start/delta`. It should enter only timeline `planning`. + +3. **Unfinished background tools cannot finalize.** + A tool returns `execution_id` with state `running`. Even with a model summary, the run remains `in_progress`. + +4. **Plan-Execute Executor output cannot finalize.** + The Executor reports a successful breakthrough before the Replanner ends. `messages.content` must not finalize. + +5. **Supervisor sub-agent output cannot finalize.** + A sub-agent returns a definite conclusion before Supervisor `exit`. It should enter only `eino_agent_reply`. + +6. **Final events require finalized metadata.** + A legacy `response` without `finalized=true` should place candidate content only in details/warnings and show blocked state in the main message, without a successful final bubble. + +7. **Failure and cancellation may be terminal.** + `error` / `cancelled` may update the assistant message, but `completionReason` must be `failed` / `user_cancelled`, never successful completion. + +## 8. Recommended defaults + +Recommended policies for CyberStrikeAI: + +```text +eino_single: suitable for lightweight tasks, with the final gate enabled +deep: recommended default for complex security testing +plan_execute: recommended for clear goals needing strict plan-execute-replan control +supervisor: use for routing among specialists, not as the general default +``` + +The governing rule is: + +```text +messages.content may come only from FinalizationDecision.FinalText; +process_details may show the entire process; +the frontend treats only response(finalized=true) as a final response. +``` diff --git a/docs/en-US/blackboard-context-memory.md b/docs/en-US/blackboard-context-memory.md new file mode 100644 index 00000000..6e9f3326 --- /dev/null +++ b/docs/en-US/blackboard-context-memory.md @@ -0,0 +1,138 @@ +# Retaining blackboard checkpoints after long-context compaction + +[中文](../zh-CN/blackboard-context-memory.md) + +When a conversation is bound to a project and the project blackboard is enabled, every successful compaction summary automatically saves a continuation record. Retention no longer relies only on the Agent calling fact tools before compaction. + +## Saved content and retrieval + +Continuation records use `note/context-*` keys. They contain important findings, authorization constraints, progress, unsuccessful attempts, and next steps from the summary, together with the source conversation, agent scope, and snapshot path of the original pre-compaction transcript. The main agent, each sub-agent, and the planner and executor in plan-execute mode maintain separate records. Repeated compactions for the same conversation/agent update the same key. + +Records use `category=note` and `confidence=tentative`. They are not automatically pinned and do not overwrite confirmed facts such as `finding/*`. A record is a model-generated working summary, not an independently verified vulnerability or new authorization. Maintain formal facts and vulnerabilities with `upsert_project_fact` and `record_vulnerability`. + +The compacted context includes a `get_project_fact(key)` entry point and refreshes the system's blackboard index. Read records on demand, then use `read_file` to inspect the original evidence in segments; the full history need not be injected into every model request. + +The new context and blackboard record retain the same continuation summary, incorporating the absolute snapshot path, historical snapshot directory, and retrieval hints. The summary preserves the source target, purpose, and discovery context of key evidence. If a task needs a Cookie, authentication information, exact parameters, or original error text, the Agent can search the relevant snapshot and inspect nearby original text with `read_file` instead of guessing values. + +Existing summary sections and the original-user-input ledger remain unchanged. The same summary adds an “original evidence retrieval hints” section with sources and keywords for tool results or key progress. It adds no new summary of user input and does not ask the model to invent filenames or line numbers. + +No separate `.index.md` file is generated, no regex or keyword classification scans snapshots, and no additional model call is made. The summary itself provides the evidence retrieval entry point. Details may still be omitted; snapshots remain authoritative for original content. + +## User input and system summaries have different provenance + +Eino transmits its default continuation summary with the `user` message role, but also marks it with `_eino_summarization_content_type=summary`; retained skill context uses the `skills` marker. The original-user-input ledger excludes internal summaries and skill messages using this metadata. `role=user` alone cannot establish that generated content is original user input. Simulated user ledgers within summaries are not imported as original input, preventing repeated compaction from promoting model-written goals or actions into user instructions. Snapshot and summarization-model text records also identify internal messages as system-generated, not user input or new authorization. + +Ledger continuation preferentially reads structured provenance metadata in system messages, rather than extracting similarly named text labels from ordinary assistant or tool responses. JSON checkpoint restoration, classic/native conversion, and system-message merging preserve this provenance. Legacy compatibility accepts only a complete host ledger block at the end of a system message, not fragments embedded in a blackboard or response. + +Compacted summaries remain in the model's working context; distinguishing provenance does not discard progress. Main/sub-agent input for the current turn still comes from the original request and chat records. Text keywords such as `` are not used to filter genuine user input. Regression tests invoke the real `DefaultFinalize`, perform three compaction rounds and classic/native conversions, and check that original authorized targets and the prohibition on testing the local machine remain; generated targets and simulated instructions do not enter the user ledger; summary progress is retained; and user quotations of summary text remain genuine user input. + +## Read boundaries for conversation work records and shared project facts + +The design distinguishes conversation state from cross-conversation memory rather than treating every automatic summary as a shared fact. The [official LangChain memory concepts](https://docs.langchain.com/oss/python/concepts/memory) describe working state as thread-scoped and cross-conversation memory as using a separate namespace. This project retains its existing fact structure and uses the reserved `note/context-*` prefix plus source conversation identity for the corresponding read scope, without a database migration or additional model extraction. + +| Content | Automatic injection and default fact tools | Explicit historical queries | +| --- | --- | --- | +| Ordinary project facts, including confirmed findings and manual notes | Existing project-sharing behavior | Existing behavior | +| Automatic continuation records from the current conversation, including its sub-agents/planner/executor | Visible and labeled as current conversation work records | Visible | +| Automatic continuation records from other conversations in the same project | Not injected; absent from default lists/searches; direct get requires the history flag | Readable with `include_history=true`, labeled as historical reference | +| Legacy continuation records without a source conversation | Not returned as current work records | Readable through explicit historical queries, labeled as unknown origin | +| Records in other projects | Unreadable through current-project tools | The history flag cannot expand project scope or bypass `project:read` | + +`list_project_facts`, `search_project_facts`, and `get_project_fact` default to the current conversation scope. The first two support `source_conversation_id` to narrow queries. Filtering occurs before database pagination, preventing pinned historical records from filling a page only to be discarded later. Initial main-agent injection, sub-agent blackboard supplements, and classic/native post-compaction index refreshes explicitly pass the current conversation ID. Missing conversation identity does not fall back to injecting every automatic continuation record. + +Relationship overviews, inline relationship hints, and relation edges returned by get/upsert also filter automatic continuation keys from other conversations to prevent a scope bypass through edges. The mirrored “related” section in a fact body is returned unchanged by `get_project_fact` and is not reader-filtered, so no automatic continuation keys are written into that section. Structured relation edges still respect caller scope; the current conversation's own continuation records remain available. Historical bodies include source conversation and update time. Reading them does not promote them into confirmed facts or new task authorization. `note/context-*` is a system-reserved prefix: fact-writing tools and console create/update APIs cannot create, overwrite, or rename ordinary facts into it. Only compaction writes continuation records. Deprecation/restoration tools cannot modify other conversations' automatic records. Save verified project knowledge under separate fact keys with its evidence. + +These are default context-injection and fact-retrieval scopes, not a new filesystem permission boundary. They do not delete snapshots, change existing records, or remove content already present in old conversation history. Authorized users can still view all project data through project management and existing database queries without scope parameters. Existing project permissions remain enforced by tool authorization. + +Validation uses two conversations in one project, another project, and legacy records of unknown origin. It covers default/explicit queries, source filters, pagination, pinned history, relation edges, reserved-key write protection, permission denial, initial main-agent injection, and index refresh after actual classic/native compaction saves. Existing blackboard-budget and original-user-ledger tests ensure scope markers do not exceed length limits or alter user input. + +## Original evidence survives later compactions + +Previously, every compaction wrote the same `transcript.txt`, overwriting earlier records. Each pre-compaction history is now saved to `snapshots/transcript--.txt`, for example `transcript-a1b2c-20261012-010203.000000004+0800.txt`. The timestamp includes nanoseconds and timezone and records the first save. Retries with identical content compare the complete file and reuse the snapshot without changing its time or path. The short MD5 is a label, not the uniqueness check; prefix or timestamp collisions do not overwrite files. Older SHA-256 filenames remain readable and reusable without renaming, preserving references in older summaries. `transcript.txt` remains a compatibility entry point for the latest record, while continuation summaries reference specific snapshots. Snapshots use the existing conversation transcript format, retaining user, assistant, and tool records while omitting unchanged system prompts and tool/skill indexes. + +Snapshots remain under `conversation_artifacts//summarization/` and follow the conversation-artifact lifecycle. Sub-agents use separate scope subdirectories. Deleting a conversation removes its artifacts through existing cleanup; snapshots are not independent post-deletion backups. Files are written through atomic replacement using temporary files, with mode 0600. + +With the default database at `data/conversations.db`, the main-agent layout is as follows. Summaries provide absolute paths to the Agent: + +```text +data/conversation_artifacts//summarization/ +├── transcript.txt +└── snapshots/ + └── transcript--.txt +``` + +Filename timestamps can order snapshots. A snapshot may be saved before summary generation, including failed attempts, and retries with identical history reuse files. File order therefore does not strictly correspond to the number of successful compactions. + +When the database path is empty, temporary snapshots use `/cyberstrike-summarization//summarization/snapshots/` instead of sharing one directory across conversations. + +Sub-agents, planners, and executors use `summarization/scopes//snapshots/`. Summaries provide historical snapshot directories; the Agent can locate `transcript-*.txt` with `glob` and read older evidence on demand. Legacy index files are unused by the new process and are not automatically deleted on upgrade. + +The system reuses the compaction summary already generated, without an additional model extraction call. Original history is snapshotted before summary generation, preserving a source even when the summary omits an exact detail. Blackboard or snapshot save failures log warnings and continue with the available summary without claiming that saving succeeded. + +## Configuration and boundaries + +```yaml +project: + enabled: true + context_checkpoints: true +``` + +`context_checkpoints` defaults to enabled when omitted. Setting it to false disables automatic blackboard continuation records without affecting original-evidence snapshots. Unbound conversations still retain local snapshots and never write their contents into another project. + +Before saving automatically, the system checks the conversation's current project binding. The actual SQLite upsert checks the binding again in the same write statement, preventing a project switch between checking and writing from saving an old run into the old project. A mismatch inserts no row and changes no existing record. This adds no cross-project search. Existing blackboard permissions and fact-tool conversation scopes still apply. + +Automatic records are persisted only when compaction occurs; they do not replace immediate registration of new findings. Summaries may omit details; snapshots remain the source for original text. Snapshots consume local storage until their conversation artifacts are cleaned up. + +## Index budgets and provenance + +Automatic injection still includes only keys, short summaries, and relation edges; complete bodies stay in the database for on-demand reads. Index queries no longer load bodies that will not be injected. The character budget includes headings, boundary markers, retrieval hints, omission notices, and relationship overviews. Very small budgets use a short retrieval hint or omit the index rather than truncating HTML boundaries. + +The blackboard needs no additional model compaction. Each conversation/agent's automatic continuation body is updated to the current summary; short labels are capped at 200 characters. Neither full bodies nor original snapshots are automatically injected. Retrieval entry points are attached only to continuation summaries and blackboard bodies; no full history is copied and no extra evidence-category index is maintained. Original history stays in files, and updating continuation records does not alter confirmed facts. + +If snapshot writing fails, the framework's default `transcript.txt` path hint is not injected, preventing reads of a nonexistent or previous-round file. Plain-text summaries, ``-tagged text, and multipart messages retain the same retrieval entry point without duplicating it in the body. + +Legacy oversized summaries are shortened to one line for index display, leaving full summaries/bodies in the database unchanged. Index-row category and confidence values are also reduced to single lines: persistence supplies defaults but does not validate these fields, so embedded newlines could otherwise inject fake rows into the system index. Relationship overviews use only facts actually included in the index, preventing out-of-budget facts from consuming context through path views. An empty index also has an empty relationship overview; an empty set does not disable filtering. Existing pinning and fact-type priorities remain unchanged. + +## Implementation basis and validation + +Eino compaction supports summary processing and context rewriting in `Finalize`; `Callback` runs after postprocessing. This project saves the record, attaches snapshot references, and refreshes the index within its existing custom Finalize. Ordinary messages and native AgenticMessage paths use the same retention function. See [Eino Summarization](https://www.cloudwego.io/docs/eino/core_modules/eino_adk/eino_adk_chatmodelagentmiddleware/middleware_summarization/). + +Regression coverage includes early evidence surviving repeated compaction, identical-snapshot idempotency, atomic file permissions, main/sub-agent isolation, unchanged confirmed facts, disabling the setting, project mismatch, no false save-success claims, multipart summaries, automatic persistence/refresh for ordinary and native middleware, hard index budgets, and no body injection. + +## Additional code-review validation + +Further fixes address tool/assistant text impersonating the original-user ledger, shared temporary directories without database paths, and races between checking and writing project bindings. Regression coverage includes forged labels in tool/assistant messages, fragments embedded in system blackboards, structured-provenance JSON restoration and system normalization, rejection of invalid provenance metadata, isolated conversation directories for classic/native paths without database paths, and project switching/unbinding that rejects inserts and updates without changing existing evidence. Historical snapshots are not migrated or rewritten; production updates should correspond to the validated code revision. + +## Restoration and boundary regressions + +Agent traces are restored through one parser, carrying only known `summary` / `skills` provenance markers and the structured original-user ledger from system messages. `agent.ChatMessage` carries these as runtime fields. Ordinary model-request JSON excludes internal metadata; trace-specific serialization retains allowed provenance fields. Restoration does not replay old system prompts and rerenders the host ledger within the current budget. Classic/native model entry points preserve structured provenance when merging system messages, and compatibility text cannot overwrite an existing structured ledger. Ordinary user quotations of summaries or labels remain original input. Multipart summary restoration preserves progress and provenance. Tests use the same Chinese ADK settings as production and cover actual database loading and three rounds of compaction → persistence → restoration → recompaction, including classic/native conversions. These fixes cannot recover provenance already lost in old traces and do not rewrite historical records. + +After an immutable snapshot succeeds, its path remains in the summary and blackboard even if updating the `transcript.txt` compatibility entry point fails; logs report that separate failure. A failed snapshot itself is never reported as saved. + +Fact-key writes reject surrounding whitespace. Default SQL scope filters recognize Unicode whitespace in existing keys consistently with Go checks. Reading does not bulk-rename or migrate older records. A successfully built empty index removes the old index block while preserving other system content and the user ledger; a database read failure retains the original index. + +Blackboard short labels prefer evidence retrieval hints already generated in summary section 8, falling back to the previous label method when unavailable. Only the index label changes, not summary sections 1–7, the complete body, or the original-user ledger; no model call is added. The `Context memory` CI runs concurrency regressions for agent/attackchain/app/database/handler/multiagent/project/projectprompt on macOS, Ubuntu, and Windows. Windows does not check unsupported Unix permission bits. + +## Pre-release scenario acceptance + +User-role messages for planning and replanning are system-generated templates containing plans and model/tool results, not new user input. Templates use a separate allowlisted provenance field. Before construction, a bounded user ledger is saved from the original uncombined messages, followed by rewriting/compaction. Role and provenance remain visible in input text and snapshots. Trace restoration and framework user-message blocks do not import templates as original user input, while their working progress remains available to the model. Ordinary user quotations of plans or summaries remain genuine input. + +Restarts after context overflow or transient failure remove old instructions and indexes but preserve and rebuild the structured original-user ledger within the current budget. Budgeting the first long input includes numbering, separators, and ellipses, so available space is not wasted by dropping the whole entry. Only bounded excerpts of original text are kept, without model-generated summaries of user input. Budgets cannot retain unlimited original text; retrieve exact details from snapshots when needed. + +Facts with existing surrounding-whitespace keys can have summaries/bodies updated using their original ID, project, and complete key, without renaming or changing historical relation references. Creating or renaming to noncanonical keys remains rejected. Actual HTTP partial-update regression tests cover this compatibility path. + +Acceptance also covers saving long tool output as an original-text file, obtaining its file reference from snapshots after compaction/restoration, and reading the exact Cookie through the `read_file` backend in segments. A synthetic Cookie sits in the middle of 2,001 output lines and appears in neither the truncated preview nor the summary; both classic/native paths recover it completely. This guarantees evidence and retrieval entry points are available, not that a model will choose to read them in every task. + +Chat input, approval interactions, and API routes retain their existing flow. New visible content consists of automatic continuation records, short retrieval hints, and a summary entry point in the attack-chain sidebar. Automatic work records from other conversations in the project are not injected by default; ordinary shared facts and manual notes retain their existing read behavior. Local snapshots accumulate for distinct compacted content and follow the conversation-artifact deletion lifecycle. + +## Summaries and evidence hints in attack chains + +Attack-chain analysis input uses persisted provenance metadata to distinguish actual user input, the host-retained original-user ledger, system-generated summaries, skill context, and execution templates. Generated summaries and templates are working hints, not new authorization or independent proof of vulnerabilities or tool execution. Nodes still require actual tool records. User quotations of summary text remain genuine user input; provenance is not inferred from body keywords. + +Actual tool-record grounding is enforced in code, not only in prompts. `action` and `vulnerability` nodes may include `tool_call_id`, copied exactly from a host-issued call ID in the trace. Referencing an ID absent from the trace is treated as fabrication; the node and its incident edges are dropped. `target` nodes are exempt. Whether references are mandatory depends on ID visibility to the model: if all actual IDs remain visible after budget trimming, a missing reference is also dropped. If some IDs were trimmed, missing references are unmatched rather than fabricated. Multi-agent supplemental process details are dumped as raw event JSON, whose IDs are absent from the parsed trace; validation is skipped in that case, as in the `messages` table fallback, to avoid discarding real nodes. Valid references are saved in node `metadata.tool_call_id`, not `attack_chain_nodes.tool_execution_id`: that column references `tool_executions`, whereas model references are call IDs. + +The attack-chain sidebar offers an expandable “Compaction summaries and evidence hints” section showing complete bodies with summary provenance from the current conversation trace. It retains immutable snapshot paths and supports copying text for source checks. Reading an existing cached attack chain also returns summaries from the current trace without regenerating nodes. Summaries are displayed separately and do not become attack nodes or affect node counts, filters, or graph exports. A conversation without tool execution can show summaries with an empty graph. + +This entry point reads only the current conversation's agent trace, not a shared-blackboard aggregation of other conversations. It adds no snapshot-download or arbitrary-file-reading API. Older traces without provenance metadata are not guessed to be summaries. Older summaries removed by subsequent compaction remain retrievable through snapshot directories. Updating code does not rewrite cached nodes; choose **Regenerate** to reanalyze a chain using the new provenance labels. + +Attack-chain requests discard late responses based on conversation and request order. Switching, closing, and errors clear old summaries, preventing stale content from replacing a new view. A busy concurrent regeneration returns 409 without first deleting the cached graph. Cached empty nodes and edges are always arrays, and a single node with no edges displays correctly. Regressions cover real model HTTP request input, current-conversation versus same-project isolation, cached-summary update/clearing, summary HTML rendered as text, copying and failure fallbacks, and out-of-order requests within and across conversations. diff --git a/docs/en-US/project-resume.md b/docs/en-US/project-resume.md new file mode 100644 index 00000000..d7c26a5e --- /dev/null +++ b/docs/en-US/project-resume.md @@ -0,0 +1,48 @@ +# Resuming project conversations and reusing results + +[中文](../zh-CN/project-resume.md) + +When continuing work on the same project the next day, restore existing results before starting new work. A chat summary alone cannot recover script parameters, dependencies, success evidence, or previously tested unsuccessful approaches. + +## What is restored automatically + +Context is rebuilt from the current database and disk for every user task. Single-agent and multi-agent runs share this entry point: + +- The project fact index and test scope. Read the entire index first. If entries are omitted, paginate with `list_project_facts` to complete the index, then read the fact bodies relevant to the task. +- Script and test records: `artifact/*`, `test/*`, and `negative/*`, with backward compatibility for `exploit/*` and `poc/*`. Script and test records each receive up to three body previews so that one category cannot crowd out the other. Previews do not replace the full body returned by `get_project_fact`. +- IDs, titles, targets, and statuses of project-linked vulnerabilities, including fixed and false-positive records. Use `get_vulnerability` for details and paginate or filter with `list_vulnerabilities` for additional records. Existing vulnerabilities are restored even when a project has no facts. +- Paths, sizes, and modification times of candidate scripts in the working directory. The inventory does not read or execute scripts, and file existence is not evidence of successful validation. It checks `artifacts/` first, then other locations. Read each file and check its fact records before executing it. + +The project continuation supplement is placed inside the fact-index refresh boundary. Index refreshes after compaction and multi-agent context supplements therefore include current database records. The on-disk file inventory is rebuilt for each user task. + +## Saving results + +Conversations bound to a project share `/projects//`; unbound conversations use their own `conversations//` directories. A new conversation in the same project, or a reopened conversation, can access the project's existing working files. + +Store validated scripts and dependencies under `artifacts/` in the project workspace where possible. Immediately after validation, register them with the existing `upsert_project_fact` tool: + +- Use `category=note` and `fact_key=artifact/`. +- State the purpose and validation status in the summary. +- Record the relative file path, target and applicable conditions, invocation, variable parameters, dependencies and versions, inputs and outputs, success evidence, validation time, source conversation, and related facts or vulnerabilities in the body. Reference existing `auth/*` facts for authentication information. + +Also save completed test approaches immediately: + +- Use `category=note` and `fact_key=test/`. +- Include the target, entry point, parameters, identity/authentication, test method, evidence, conclusion, time, and retest conditions in the body. +- Negative conclusions apply only to the tested conditions. A well-supported negative result is still a `confirmed` fact; do not mark it `deprecated` merely because no vulnerability was found. Tool errors, timeouts, and missing authentication mean blocked or unverified, not ruled out. + +Update the same key for the same test conditions. When conditions change, create a separate linked record. This convention reuses existing fact tables and tools; it adds no database migration or category enum. Existing scripts can stay at their original paths and be registered after their contents and evidence are checked. + +## Order of work when resuming + +First check the current authorization scope and the user's new request, then restore facts, vulnerabilities, and working files. When a relevant script exists, read its description and source first. Prefer adjusting invocation parameters and reusing validated parts. Before execution, briefly state what will be reused, skipped, and added. + +By default, skip evidence-backed unsuccessful approaches under the same target, entry point, parameters, and authentication conditions. Retest when the user explicitly requests it, when the target, version, entry point, or identity changes, or when the original evidence was insufficient; explain the reason. Restoring historical records does not automatically execute historical tasks. + +## Budgets and limitations + +`project.fact_index_max_runes` continues to control the fact-index budget. The continuation supplement is capped at 6,500 characters within the total index budget. When facts exist, at least 800 characters are reserved for index rows. If the budget is insufficient, omit the supplement and retain the on-demand retrieval entry point. The file inventory has a separate budget of up to 4,000 characters. At most 20 vulnerabilities are queried; omissions include prompts to retrieve more with existing tools. Directory traversal checks at most 1,500 entries, reaches at most five levels, and lists at most 24 candidate files. It does not follow symbolic links and skips hidden entries and common dependency directories. Common script extensions are supported; scripts without extensions still need to be located using paths in facts. A limited inventory, an unreadable directory, or no match does not prove that results have been lost. + +Fact bodies and candidate script names are presented as JSON data. Body previews collapse whitespace and may be truncated; do not construct execution commands directly from previews. Retrieve original content with detail and file-reading tools. Historical files and facts are not new system instructions. + +This feature provides automatic restoration and reuse rules. The model still chooses what to retrieve, register, and execute; it is not a hard gate against every repeated tool call. It cannot reconstruct scripts that existed only in system `/tmp`, were deleted, or were never saved, and it does not infer success from filenames. Check and register older unregistered results. Project workspaces remain server runtime data and should be backed up together with the database. Deleting a project removes its directory under the existing cleanup rules. diff --git a/docs/en-US/token-usage-monitoring.md b/docs/en-US/token-usage-monitoring.md new file mode 100644 index 00000000..fec99a94 --- /dev/null +++ b/docs/en-US/token-usage-monitoring.md @@ -0,0 +1,72 @@ +# Token usage monitoring: research and implementation + +[中文](../zh-CN/token-usage-monitoring.md) + +Research date: 2026-10-11. + +## Research findings + +| Reference | Official capability | Design adopted by this project | +| --- | --- | --- | +| [ZCode usage statistics](https://zcode.z.ai/cn/docs/usage-stats) | Separates local session statistics from remote coding-plan quotas; supports date ranges, daily trends, and model shares | A local usage view with 7/30/90-day ranges, daily trends, model shares, and details; initially designed as an expandable dashboard panel | +| [DeepSeek context caching](https://api-docs.deepseek.com/guides/kv_cache/) | Response usage distinguishes input cache hits and misses with `prompt_cache_hit_tokens` / `prompt_cache_miss_tokens` | Map cache hits to the standard OpenAI cache-read field; do not map misses to cache writes | +| [Eino v0.9.21](https://github.com/cloudwego/eino/releases/tag/v0.9.21), [schema PR #1309](https://github.com/cloudwego/eino/pull/1309) | The stable release adds `PromptTokenDetails.CacheWriteTokens`, retaining cache reads, input, output, and reasoning fields | Carry the stable standard fields through streaming merges, run summaries, the database, and UI | +| [Eino-ext PR #1015](https://github.com/cloudwego/eino-ext/pull/1015) | Model adapters forward cache-write counts | Upgrade agenticclaude to 0.1.7, agenticopenai to 0.2.4, and the corresponding OpenAI ACL, rather than upgrading only the framework | +| [Eino callbacks and tracing](https://www.cloudwego.io/docs/eino/quick_start/chapter_06_callback_and_trace/) | Component callbacks can collect model lifecycle, usage, and duration data | Retain existing run-event collection; covering every internal call would require a separate unified model-call ledger | + +Versions were checked through Go module resolution and local source inspection; the 0.10 alpha was not adopted. Raw DeepSeek fields do not automatically enter Eino's standard cache-read count, so the project's existing HTTP compatibility layer supplies the mapping for ordinary JSON and SSE streaming responses. Existing standard `cached_tokens` takes precedence. Unknown fields are preserved as RawMessage to avoid loss of numeric precision through floating-point conversion. + +## Metric definitions + +- `promptTokens`: all input reported by the model. +- `completionTokens`: all output, including reasoning counted as output by the provider. +- `totalTokens`: the provider's total; falls back to input plus output when absent. +- `cachedTokens`: the cache-read portion of input. +- `cacheWriteTokens`: the cache-write portion of input. +- `reasoningTokens`: the reasoning portion of output. +- Cache reads, cache writes, and reasoning are breakdowns and must not be added to the total again. The cache-read share of input is `cachedTokens / promptTokens`; a dash is shown when there is no input. +- A streaming model output may report cumulative counts multiple times. The merge retains the maximum for each field and adds the output to the run summary only once when it ends. Usage already received can be retained even if the output subsequently fails. + +DeepSeek cache misses are not equivalent to Claude cache writes. Text length cannot establish billing-grade counts, and an absent field does not prove that the provider used no cache. New fields on older records default to 0, meaning that the breakdown was not recorded. + +## Entry points and data flow + +Click the dashboard **Token Usage** card to open **System Settings → Token Usage**, or open the section directly from the settings submenu. The dashboard card shows total usage for the last seven days; its subtitle shows only the cache-hit rate (cache-read tokens / input tokens). Call counts and daily usage remain on the detail page. The detail page supports the current project scope, 7/30/90-day ranges, eight metrics, up to 20 model-ranking entries grouped by the run's configured model, daily trends and full daily details, and up to 20 recent runs. It loads independently, supports manual refresh and request cancellation, and does not refresh the whole dashboard when the range changes. Reloading the page retains the settings section. + +The detail page separates four primary metrics (total, input, output, and cache-read share of input) from four secondary metrics (calls, cache reads, cache writes, and reasoning). Trends switch between total tokens, model calls, and cache-hit rate. Hover over or keyboard-focus a bar to see the date and exact value; switching chart metrics does not request the API again. The cache donut separates cache reads from remaining input instead of adding cache or reasoning to the total. Model details include share bars; daily numeric details are collapsed by default; recent runs distinguish complete and partial records. Metric definitions can be expanded. + +Daily charts use the server dates returned by the API and fill only gaps between returned dates; they do not guess the query endpoint from the browser timezone. Charts and wide tables scroll horizontally within their own areas on small screens. No records and unavailable statistics have distinct empty and unavailable states. + +### UI research basis (2026-10-12) + +| Reference | Design adopted | +| --- | --- | +| [DeepSeek Context Caching](https://api-docs.deepseek.com/guides/kv_cache/) | Its explicit cache-hit/miss input definitions inform the cache composition view; the ratio uses tokens, not request counts | +| [LiteLLM Usage](https://docs.litellm.ai/docs/proxy/ui/chat_ui) | Separate range selection, core metrics, and usage logs; retain the existing data scope without adding costs or success rates that cannot be measured accurately | +| [Langfuse Custom Dashboards](https://langfuse.com/docs/metrics/features/custom-dashboards) | Separate summaries, time trends, and model distribution into readable areas, with switchable trend metrics | + +Full DeepSeek and GLM console pages were not publicly readable during that research. The visual design therefore used verifiable open-source dashboards and does not claim to reproduce their current authenticated interfaces. The project adds no provider billing, budget, or savings estimates. + +Run events → streaming/non-streaming usage accumulation → `eino_usage_summary` → `process_details` and `model_token_usage` → existing `/api/usage/tokens` and conversation usage endpoints → dashboard and chat summaries. + +`model_token_usage.cache_write_tokens INTEGER NOT NULL DEFAULT 0` was added. Existing declarative SQLite schema reconciliation adds the column at startup; older databases need no manual migration. Existing event-ID idempotency, foreign-key deletion relationships, and historical backfills remain in effect. Daily grouping uses the server's local timezone, matching the dashboard's “Today” boundary. + +Provider model names are rendered as text nodes. Tables scroll locally on mobile, and labels support Chinese, English, and Russian. Chat-turn tooltips, the main conversation timeline, and WebShell timelines also include cache writes. + +## Current scope and future work + +Statistics use recorded Agent output events, including sub-agents in the existing collection path. They are **not a site-wide API request ledger**. Compaction, retrieval embeddings, hidden model calls, and requests without usage data are not fully covered. Consumption whose final usage was lost through request cancellation cannot be reconstructed. + +Model rankings group by the model name configured for the run. A failover run may mix multiple models, so rankings cannot be used for per-model billing. Each recent entry is a run summary, not an individual HTTP request. Model call counts include only outputs with nonempty usage. + +Costs and remaining plan quotas are not displayed. Custom gateways and subscription plans have different pricing and billing rules; multiplying tokens by a fixed price cannot reliably represent cost. Complete cost management would first require a per-request ledger covering call origin, actual provider and model, success/error/cancellation, duration, TTFT, whether usage was returned, price version, and currency. It should cover main/sub-agents, compaction, retries, and embeddings, with separate provider quota integrations. + +## Validation + +The Chinese guide records these implementation-time checks; this translation does not represent a new execution of them: + +- `go test ./...`: the full repository passed. +- `node --test web/static/js/*.test.cjs`: 295 tests passed. +- Regression coverage was added for cache-write persistence, grouping, recent entries, adding the column to older schemas, user visibility, streaming deduplication, and total-token fallback. +- DeepSeek JSON/SSE cache-field conversion, standard-field precedence, preservation of error/termination chunks, and very large number precision passed validation. +- Browser checks used synthetic data for desktop, 390px mobile, dark mode, model-name escaping, empty data, and unavailable API states. No real run database or paid model was accessed. diff --git a/docs/zh-CN/agent-finalization-best-practices.md b/docs/zh-CN/agent-finalization-best-practices.md index 97febaf6..15903f66 100644 --- a/docs/zh-CN/agent-finalization-best-practices.md +++ b/docs/zh-CN/agent-finalization-best-practices.md @@ -1,6 +1,6 @@ # Agent 最终回复治理最佳实践 -[返回中文文档](README.md) +[返回中文文档](README.md) | [English](../en-US/agent-finalization-best-practices.md) 调研日期:2026-07-28 diff --git a/docs/zh-CN/blackboard-context-memory.md b/docs/zh-CN/blackboard-context-memory.md index 39d0cdd6..feadf317 100644 --- a/docs/zh-CN/blackboard-context-memory.md +++ b/docs/zh-CN/blackboard-context-memory.md @@ -1,5 +1,7 @@ # 长上下文压缩后的黑板续接留存 +[English](../en-US/blackboard-context-memory.md) + 已绑定项目且启用项目黑板时,每次成功生成压缩摘要,系统会自动保存一条续接记录。不再仅依靠 Agent 在压缩前主动调用事实工具。 ## 保存内容与读取方式 diff --git a/docs/zh-CN/project-resume.md b/docs/zh-CN/project-resume.md index 84d2cc22..2b89b432 100644 --- a/docs/zh-CN/project-resume.md +++ b/docs/zh-CN/project-resume.md @@ -1,5 +1,7 @@ # 项目续聊与成果复用 +[English](../en-US/project-resume.md) + 同一项目第二天继续任务时,应先恢复已有成果,再执行新的工作。仅保留聊天摘要不足以恢复脚本参数、依赖、成功证据和已经验证过的失败路线。 ## 自动恢复的内容 diff --git a/docs/zh-CN/token-usage-monitoring.md b/docs/zh-CN/token-usage-monitoring.md index 9a60fd48..b2fb7f75 100644 --- a/docs/zh-CN/token-usage-monitoring.md +++ b/docs/zh-CN/token-usage-monitoring.md @@ -1,5 +1,7 @@ # Token 用量监控:调研与实现 +[English](../en-US/token-usage-monitoring.md) + 调研时间:2026-10-11。 ## 调研结论