mirror of
https://github.com/BigBodyCobain/Shadowbroker.git
synced 2026-05-28 10:01:31 +02:00
Shadowbroker
5e6bb8511a
Replace the dated editorial fallback positions baked into the registry with a one-shot seed file + persistent observation cache. The user's runtime cache now reflects what THIS install has actually observed, not what USNI published on March 9, 2026. A year from now, the cache holds a year of observations and the seed is irrelevant. == #244: dated editorial coordinates out of the registry == CARRIER_REGISTRY no longer carries fallback_lat/lng/heading/desc. Those fields are deleted. The registry is now identity + homeport only. New file: backend/data/carrier_seed.json - Read-only, shipped with every release. - Used ONCE on first-ever startup to bootstrap carrier_cache.json. - Each entry stamped with position_confidence="seed" and the actual as-of date (2026-03-09), NOT now(). == #245: approximate confidence for headline-derived positions == _parse_carrier_positions_from_news() now stamps every GDELT-derived entry with position_confidence="approximate" so the UI knows the coordinate is a region-centroid match, not a precise observation. After the freshness window the label rolls over to "stale_approximate" so old-and-imprecise is distinguishable from recent-and-imprecise. The article's actual seendate is used as position_source_at instead of now(), so the "last reported X days ago" badge is honest. == #246: freshness is labelling, not eviction == The cache always preserves the last position the system observed, forever. What changes is the position_confidence label: - within configurable window (default 14d, env-overridable via SHADOWBROKER_CARRIER_FRESHNESS_DAYS) -> "recent" - older -> "stale" - seed-bootstrap entries that were never refreshed -> "seed" - homeport defaults (carrier added post-install) -> "homeport_default" - headline-derived (any age, fresh) -> "approximate" - headline-derived (older than window) -> "stale_approximate" The position itself never reverts to the seed or the registry. The user always sees the last position the system observed. Per the user's explicit guidance: "from there have it be the last position the user has logged the carriers that way a year from now it doesnt revert to where the ships are today". == Other improvements == - CACHE_FILE moved to backend/data/carrier_cache.json so it lives in the volume-mounted dir under Docker compose. Previously it was at /app/carrier_cache.json which got wiped on every container restart (pre-existing bug). - Atomic cache write (temp + os.replace) so a crash mid-write does not leave a truncated cache file. == Public API shape == Every carrier object the API emits now includes: - position_confidence: seed | recent | stale | approximate | stale_approximate | homeport_default - position_source_at: ISO timestamp of when the underlying source was observed (NOT now()) - is_fallback: convenience boolean for the UI; true when the confidence is seed/stale/stale_approximate/ homeport_default Existing fields (estimated, source, source_url, last_osint_update, name, type, lat, lng, country, desc, wiki) are preserved exactly so the current ShipPopup frontend renders unchanged. last_osint_update now reflects position_source_at instead of now(), which is what the existing "last reported MM/DD" badge always meant to show. Tests: backend/tests/test_carrier_tracker_quality.py — 17 tests covering seed bootstrap, subsequent-startup ignoring seed, no-seed/ no-cache homeport fallback, registry no longer has fallback fields, freshness window labelling + env override, "year-old cache entry keeps its position, only the label flips" regression, approximate confidence for headline matches, GDELT seendate ISO parser, public response shape backward compat. Credit: tg12 (external security audit, three P1/P2 issues).
264 lines
5.7 KiB
Plaintext
264 lines
5.7 KiB
Plaintext
# shadowbroker .gitignore
|
|
# ----------------------
|
|
|
|
# Dependencies
|
|
node_modules/
|
|
venv/
|
|
env/
|
|
.venv/
|
|
backend/.venv-dir
|
|
backend/venv-repair*/
|
|
backend/.venv-repair*/
|
|
|
|
# Environment Variables & Secrets
|
|
.env
|
|
.envrc
|
|
.env.local
|
|
.env.development.local
|
|
.env.test.local
|
|
.env.production.local
|
|
.npmrc
|
|
.pypirc
|
|
.netrc
|
|
*.pem
|
|
*.key
|
|
*.crt
|
|
*.csr
|
|
*.p12
|
|
*.pfx
|
|
id_rsa
|
|
id_rsa.*
|
|
id_ed25519
|
|
id_ed25519.*
|
|
known_hosts
|
|
authorized_keys
|
|
|
|
# Python caches & compiled files
|
|
__pycache__/
|
|
*.py[cod]
|
|
*$py.class
|
|
*.so
|
|
.Python
|
|
.ruff_cache/
|
|
.pytest_cache/
|
|
.mypy_cache/
|
|
.hypothesis/
|
|
.tox/
|
|
|
|
# Next.js build output
|
|
.next/
|
|
out/
|
|
build/
|
|
*.tsbuildinfo
|
|
|
|
# Deprecated standalone Infonet Terminal skeleton (migrated into frontend/src/components/InfonetTerminal/)
|
|
frontend/infonet-terminal/
|
|
|
|
# Rust build artifacts (privacy-core)
|
|
target/
|
|
target-test/
|
|
|
|
# ========================
|
|
# LOCAL-ONLY: extra/ folder
|
|
# ========================
|
|
# All internal docs, planning files, raw data, backups, and dev scratch
|
|
# live here. NEVER commit this folder.
|
|
extra/
|
|
|
|
# ========================
|
|
# Application caches & runtime DBs (regenerate on startup)
|
|
# ========================
|
|
backend/ais_cache.json
|
|
backend/carrier_cache.json
|
|
backend/cctv.db
|
|
cctv.db
|
|
*.db
|
|
*.sqlite
|
|
*.sqlite3
|
|
|
|
# ========================
|
|
# backend/data/ — blanket ignore, whitelist static reference files
|
|
# ========================
|
|
# Everything in data/ is runtime-generated state (encrypted keys,
|
|
# MLS bindings, relay spools, caches) and MUST NOT be committed.
|
|
# Only static reference datasets that ship with the repo are whitelisted.
|
|
backend/data/*
|
|
!backend/data/datacenters.json
|
|
!backend/data/datacenters_geocoded.json
|
|
!backend/data/military_bases.json
|
|
!backend/data/plan_ccg_vessels.json
|
|
!backend/data/plane_alert_db.json
|
|
!backend/data/power_plants.json
|
|
!backend/data/tracked_names.json
|
|
!backend/data/yacht_alert_db.json
|
|
# Issue #206: bundled KiwiSDR receiver directory used as last-resort
|
|
# fallback when rx.linkfanel.net (HTTP-only upstream) is unreachable
|
|
# or returns content that fails our integrity validation.
|
|
!backend/data/kiwisdr_directory.json
|
|
# Issue #201: pinned SHA-256 digests for known Tor Expert Bundle URLs.
|
|
# Used as a second verification source when upstream .sha256sum fails.
|
|
!backend/data/tor_bundle_digests.json
|
|
# Issue #258: SPKI pins for stream.aisstream.io so we can survive upstream
|
|
# Let's Encrypt renewal failures without disabling TLS validation entirely.
|
|
!backend/data/aisstream_spki_pins.json
|
|
# Issue #231: pinned SHA-256 digests for known release archives. Used by
|
|
# the self-updater as a second-line integrity check when the release's
|
|
# SHA256SUMS.txt asset can't be fetched.
|
|
!backend/data/release_digests.json
|
|
# Issue #244/#245/#246: one-shot carrier-position seed shipped with each
|
|
# release. Used ONLY on first-ever startup to bootstrap carrier_cache.json;
|
|
# after that the cache reflects this install's own GDELT observations.
|
|
!backend/data/carrier_seed.json
|
|
|
|
# OS generated files
|
|
.DS_Store
|
|
.DS_Store?
|
|
._*
|
|
.Spotlight-V100
|
|
.Trashes
|
|
ehthumbs.db
|
|
Thumbs.db
|
|
|
|
# IDEs and Editors
|
|
.vscode/
|
|
.idea/
|
|
*.suo
|
|
*.ntvs*
|
|
*.njsproj
|
|
*.sln
|
|
*.sw?
|
|
|
|
# Vercel / Deployment
|
|
.vercel
|
|
|
|
# ========================
|
|
# Temp / scratch / debug files
|
|
# ========================
|
|
tmp/
|
|
*.log
|
|
*.tmp
|
|
*.bak
|
|
*.swp
|
|
*.swo
|
|
out.txt
|
|
out_sys.txt
|
|
rss_output.txt
|
|
merged.txt
|
|
tmp_fast.json
|
|
diff.txt
|
|
local_diff.txt
|
|
map_diff.txt
|
|
TERMINAL
|
|
|
|
# Debug dumps & release artifacts
|
|
backend/dump.json
|
|
backend/debug_fast.json
|
|
backend/nyc_sample.json
|
|
backend/nyc_full.json
|
|
backend/liveua_test.html
|
|
backend/out_liveua.json
|
|
backend/out.json
|
|
backend/temp.json
|
|
backend/seattle_sample.json
|
|
backend/sgp_sample.json
|
|
backend/wsdot_sample.json
|
|
backend/xlsx_analysis.txt
|
|
frontend/server_logs*.txt
|
|
frontend/cctv.db
|
|
frontend/eslint-report.json
|
|
*.zip
|
|
*.tar.gz
|
|
*.xlsx
|
|
|
|
# Old backups & repo clones
|
|
.git_backup/
|
|
local-artifacts/
|
|
release-secrets/
|
|
shadowbroker_repo/
|
|
frontend/src/components.bak/
|
|
frontend/src/components/map/icons/backups/
|
|
|
|
# Coverage
|
|
coverage/
|
|
.coverage
|
|
.coverage.*
|
|
dist/
|
|
|
|
# Test scratch files (not in tests/ folder)
|
|
backend/test_*.py
|
|
backend/services/test_*.py
|
|
|
|
# Local analysis & dev tools
|
|
backend/analyze_xlsx.py
|
|
backend/services/ais_cache.json
|
|
graphify/
|
|
graphify-out/
|
|
|
|
# ========================
|
|
# Internal docs & brainstorming (never commit)
|
|
# ========================
|
|
docs/*
|
|
!docs/mesh/
|
|
docs/mesh/*
|
|
!docs/mesh/threat-model.md
|
|
!docs/mesh/claims-reconciliation.md
|
|
!docs/mesh/mesh-canonical-fixtures.json
|
|
!docs/mesh/mesh-merkle-fixtures.json
|
|
!docs/mesh/wormhole-dm-root-operations-runbook.md
|
|
.local-docs/
|
|
infonet-economy/
|
|
updatestuff.md
|
|
ROADMAP.md
|
|
UPDATEPROTOCOL.md
|
|
CLAUDE.md
|
|
DOCKER_SECRETS.md
|
|
|
|
# Misc dev artifacts
|
|
clean_zip.py
|
|
zip_repo.py
|
|
refactor_cesium.py
|
|
jobs.json
|
|
|
|
# Claude / AI
|
|
.claude
|
|
.mise.local.toml
|
|
.codex-tmp/
|
|
prototype/
|
|
.runtime/
|
|
|
|
# ========================
|
|
# Runtime state & operator-local data (never commit)
|
|
# ========================
|
|
# TimeMachine snapshot cache — regenerated at runtime, can be 100 MB+
|
|
backend/timemachine/
|
|
# Operator witness keys, identity material, transparency ledgers (machine-local)
|
|
ops/
|
|
# Runtime DM relay state
|
|
dm_relay.json
|
|
# Dev scratch notes
|
|
improvements.txt
|
|
|
|
# ========================
|
|
# Custody verification temp dirs (runtime test artifacts with private keys!)
|
|
# ========================
|
|
backend/sb-custody-verify-*/
|
|
|
|
# Python egg-info (build artifact, regenerated by pip install -e)
|
|
*.egg-info/
|
|
|
|
# Privacy-core debug build (Windows DLL, 3.6 MB, not shipped)
|
|
privacy-core/debug/
|
|
|
|
# Desktop-shell export stash dirs (empty temp dirs from Tauri build)
|
|
frontend/.desktop-export-stash-*/
|
|
|
|
# Wormhole logs (can be 30 MB+ each, runtime-generated)
|
|
backend/data/wormhole_stderr.log
|
|
backend/data/wormhole_stdout.log
|
|
|
|
# Runtime caches that already slip through the backend/data/* blanket
|
|
# (these are caught by the wildcard but listing for clarity)
|
|
|
|
# Compressed snapshot archives (can be 100 MB+)
|
|
*.json.gz
|