CyberStrikeAI/docs/en-US/developer-guide.md

Developer Guide

中文

This guide is for contributors extending CyberStrikeAI. The project is a Go single-service application with a static frontend, SQLite persistence, Agent/MCP orchestration, and optional high-risk security subsystems.

Project Layout

cmd/server/              service entrypoint
internal/app/            app wiring, routes, MCP tool registration
internal/handler/        HTTP handlers
internal/database/       SQLite access
internal/security/       auth, rate limits, shell execution
internal/mcp/            MCP server and external MCP manager
internal/multiagent/     Eino single-agent, multi-agent, middleware
internal/workflow/       graph orchestration runtime
internal/knowledge/      indexing and retrieval
internal/c2/             built-in C2
internal/project/        project fact blackboard
web/static/              frontend JS/CSS/assets
web/templates/           HTML templates
tools/                   YAML command tools
roles/                   role YAML
agents/                  multi-agent Markdown definitions
skills/                  Agent Skills
docs/                    documentation

Development Startup

go run ./cmd/server --config config.yaml

The frontend is static. Most JS/CSS/template changes only require a browser refresh.

One-Click Update: Letting an Installation Update Its Own Source

On a deployed machine the routine is "my tree moved forward by a few dozen commits -> bring this installation up -> rebuild -> swap the binary -> restart". This used to be only upgrade.sh, which pointed at one fixed upstream repository, so for anyone running a fork "upgrade" meant "overwrite yourself with somebody else's code". The capability now lives in the platform (internal/update) and the console page, the REST API and the CLI share one implementation, so the three surfaces cannot disagree about why an update was refused.

Three entry points

entry	how it is used
console	System settings -> One-click update (#system-update, the old deep link still works). Opening this section reads the local tree only and does not go online; Check is what performs the fetch; Update starts a job the page polls; the page also carries the "exit after updating" option and a rollback button
REST	GET /api/system/update (state on disk, no network), POST /api/system/update/check (fetch, then report the gap), POST /api/system/update/apply (202 with job_id, polled through GET /api/system/update/job), POST /api/system/update/rollback
CLI	./cyberstrike-ai -check-update, ./cyberstrike-ai -update, ./cyberstrike-ai -update-rollback. The install root is the directory holding --config, or the current directory when --config was not given

Permissions are update:read (GET) and update:apply (the four mutating endpoints). update:apply additionally requires a global (all scope) session: one machine has one source tree, and an assigned/own session must not be able to move the code everybody else is running.

Where the code comes from

From the remote this directory already tracks; no third-party repository is hardcoded. The remote is picked from the ones this tree has, in the order mine, origin, upstream (falling back to the first remote when none of those names exists), and the branch defaults to what the current branch's @{upstream} names (the current branch itself when there is no upstream). upgrade.sh is now a thin shell over the same implementation: when this directory is a git work tree it just calls ./cyberstrike-ai -update.

Choosing the source (the update section of config.yaml)

Unset, the rules above apply; set, the explicit choice wins:

update:
  remote: origin        # an existing remote name (mine/origin/upstream/anything), or remote_url instead
  # remote_url: https://github.com/AIPentest/CyberStrikeAI.git
  branch: main          # optional; defaults to @{upstream} / the current branch

The console page edits and saves all three. Saving validates: remote and branch names go through the same allowlist an update uses, and the address allowlist is https/http/ssh/git/file plus local absolute paths - git's ext:: transport executes commands and is rejected outright. Following the official repository, your own fork, or somebody else's second-development repository is exactly the choice this section expresses.

Connecting a non-git installation (the tarball kind)

An installation that started as an unpacked Release archive has no .git; once a source is saved it can be connected from the same page: the preview fetches in a throwaway repository and first lists the local files the target would replace and the operator content that will be kept; on confirm the directory becomes a git work tree (git init, origin added, the target branch checked out), every replaced file is kept under .update-backup/<timestamp>/overwritten/, operator content is stashed and restored as usual, and the binary is rebuilt. From then on it is a normal installation with one-click update and rollback - but note there is no git history before the connection, so the connection itself has no earlier commit to roll back to; the first real update writes the first rollback point.

The four refusals

situation	reason	meaning and what to do
product source modified locally (a path outside the protected list)	local_source_edits	An update is a download, not a conflict decision. The file names are listed; commit or restore them first
branch diverged from its remote (ahead and behind at once)	diverged	That is a merge decision; the counts are reported so you can resolve it by hand
no Go toolchain on this box	no_toolchain	The source still updates; only the binary is not swapped. Install go and press update again to finish
this directory is not a git work tree	not_a_repo (installed: false in the status)	A tarball installation has no remote to fast-forward; update it through the release package

apply is asynchronous, so it always answers 202 first and the refusals above arrive in the polled job.failure (reason / message / items). rollback is synchronous and maps refusals to status codes (local_source_edits, diverged, no_state, moved_since_update, no_binary -> 409, anything else -> 400). Other failures (merge_failed, build_failed, swap_failed) carry git's or the compiler's own output; a failed build leaves the previous binary in place while the source has already moved and the state file is written, so it stays rollback-able.

How operator content survives

roles/ skills/ tools/ agents/ knowledge_base/ data/ log/ venv/ config.yaml .env belong to the operator (update.Protected is the judge). Before the merge, only those of those paths the update actually writes to are copied into .update-backup/<timestamp>/ and removed from the tree - your extra directories that upstream never touched are not disturbed - and they are put back afterwards, which is what "update the code, keep my work" means. The result names every file in keptContent instead of quietly dropping it: where both you and upstream changed the same file, the version that lands is yours, and the result says so.

Rollback

./cyberstrike-ai -update-rollback, or the button on the page: git reset --hard back to the commit the update came from, the binary kept as cyberstrike-ai.prev is put back, and .update-state.json is removed. It refuses when there is no update record (no_state), no kept binary (no_binary), the recorded commit is not in this repository (bad_state), HEAD has moved since that update (moved_since_update - a rollback undoes the update, not the work done after it), or source is modified locally (local_source_edits).

The two restart cases

The process only stands down (graceful Shutdown, then exit 0) when the request explicitly says restart: true and a restart hook was wired at startup. Whether it actually comes back is up to a supervisor (systemd, run.sh), and the page says exactly that rather than promising a boot that may not happen. Otherwise the API only reports needsRestart and the old binary keeps running. Asking for a restart with no hook wired is a 400, instead of stopping the service and claiming it restarted.

Adding a Business Module

Do not add only a handler. A complete module usually needs:

1. Data model and SQLite migration.
2. Handler: parameters, errors, pagination/filtering.
3. Audit: management actions.
4. Monitor: long-running execution state.
5. MCP: whether Agents should call it.
6. HITL: approval boundary for MCP tools.
7. OpenAPI: update /api/openapi/spec.
8. Frontend: i18n, states, empty/error UI.
9. Tests: DB, handler, edge cases.
10. Docs: config, usage, troubleshooting, safety impact.

Missing one of these usually becomes a later usability or safety bug.

Error Response Design

Prefer stable JSON:

{
  "error": "machine_readable_code",
  "message": "human-readable explanation"
}

Frontend needs stable fields, users need actionable messages, and logs need detailed internal errors.

Long-Running Tasks

For scanning, indexing, batch tasks, C2, or external operations, answer:

• Can it be cancelled?
• Can progress be queried?
• Can it be retried?
• Where is the result stored?
• Does state survive page refresh?
• Does it block the HTTP request?

If not, use task tables, event streams, or monitoring.

Extending Tools

Prefer tools/*.yaml for command tools. Use Go built-in tools when the tool needs internal state or structured integration.

Built-in tools should define clear input schemas, handle timeouts and errors, and respect HITL for risky actions.

Frontend Changes

Use existing helpers such as apiFetch, modal utilities, notifications, and i18n. Update both web/static/i18n/zh-CN.json and web/static/i18n/en-US.json for new visible text.

Avoid putting secrets or provider keys in frontend code.

Test Priority

High-value tests:

• config hot-apply;
• HITL branches;
• shell timeout/no-output;
• external MCP recovery;
• KB indexing and post-processing;
• WebShell OS/encoding detection;
• SQLite migration compatibility.

Source Anchors

• App wiring: internal/app/app.go
• Config apply: internal/handler/config.go
• OpenAPI: internal/handler/openapi.go
• Tool executor: internal/security/executor.go
• Skill package: internal/skillpackage/
• One-click update: internal/update/ (status, apply, rollback), HTTP surface in internal/handler/update.go, CLI switches in cmd/server/update_cli.go, thin shell in upgrade.sh