Background
==========
PR #285 set up the seed -> cache -> GDELT model for the carrier tracker
to address audit issues #244/#245/#246. The GDELT half of that pipeline
hits api.gdeltproject.org's doc API for headline-region keyword
matching -- low precision (false centroid positions per #245) AND
unreliable (the host times out from some networks, including Docker
Desktop on Windows).
USNI publishes a weekly Fleet & Marine Tracker with explicit prose like:
"The Gerald R. Ford Carrier Strike Group is operating in the Red Sea"
"Aircraft carrier USS George Washington (CVN-73) is in port in
Yokosuka, Japan"
That is a strictly better source for U.S. Navy carrier positions:
authoritative, deterministically parseable, weekly cadence.
What this PR does
=================
New module: backend/services/fetchers/usni_fleet_tracker.py
- Pulls USNI's WordPress RSS feeds (site-wide + category, unioned).
- Picks the most recent fleet-tracker post by parsed pubDate.
- For each carrier in the registry, scans the article body for
"is operating in / is in port in / returned to / transiting" near
the carrier's name, hull code, or "<name> Carrier Strike Group"
variant. Captures the region/port phrase that follows.
- Maps the region phrase to coordinates via the existing
REGION_COORDS table, with a USNI-phrase alias table for the
specific wording USNI uses ("Yokosuka, Japan", "Norfolk, Va.",
"Naval Station San Diego", "5th Fleet AOR", etc.).
- Returns {hull: position_entry} with position_confidence="recent"
and position_source_at = the article's actual publication
timestamp (not now()).
Politeness
----------
Uses outbound_user_agent("usni-fleet-tracker") so USNI sees a
per-install Shadowbroker identifier (Round 7a / PR #292). The
article body pages return 403 to non-browser UAs; the WordPress RSS
feed serves the full <content:encoded> body and is the supported
aggregator path. No browser UA spoofing.
carrier_tracker.update_carrier_positions() now runs three phases:
1. Bootstrap from cache (or seed on first run).
2. USNI fleet tracker -- PRIMARY high-confidence source.
3. GDELT -- SECONDARY backfill; can NOT demote a "recent" USNI
position to an "approximate" GDELT headline match.
Verified live: 6 of 11 carriers picked up real May 18, 2026 positions
on first refresh (Eisenhower, Ford, Bush, Roosevelt, Lincoln,
Washington). The other 5 weren't mentioned in this week's article
(they're in port at homeports with no deployment changes) and kept
their cache entries -- which is the correct seed/cache contract from
PR #285.
Other small fixes bundled in
============================
docker-compose.yml: add the 6 third-party-fetcher opt-in env vars
(PREDICTION_MARKETS_ENABLED, FINANCIAL_ENABLED, FIMI_ENABLED,
NUFORC_ENABLED, NEWS_ENABLED, CROWDTHREAT_ENABLED). They were
documented in .env.example but never wired through compose, so setting
them in .env had no effect.
frontend/src/components/TopRightControls.tsx: fix 6 broken i18n keys
that were showing as raw "terminal.term1" / "terminal.cleanupDetail" /
"node.soloReady" placeholders in the INFONET TERMINAL modal. The
translation files have these strings under different key names; the
component now calls the right ones. Full-file sweep confirmed every
other t('...') key in the whole frontend resolves cleanly.
Replace the dated editorial fallback positions baked into the registry
with a one-shot seed file + persistent observation cache. The user's
runtime cache now reflects what THIS install has actually observed,
not what USNI published on March 9, 2026. A year from now, the cache
holds a year of observations and the seed is irrelevant.
== #244: dated editorial coordinates out of the registry ==
CARRIER_REGISTRY no longer carries fallback_lat/lng/heading/desc.
Those fields are deleted. The registry is now identity + homeport
only.
New file: backend/data/carrier_seed.json
- Read-only, shipped with every release.
- Used ONCE on first-ever startup to bootstrap carrier_cache.json.
- Each entry stamped with position_confidence="seed" and the actual
as-of date (2026-03-09), NOT now().
== #245: approximate confidence for headline-derived positions ==
_parse_carrier_positions_from_news() now stamps every GDELT-derived
entry with position_confidence="approximate" so the UI knows the
coordinate is a region-centroid match, not a precise observation.
After the freshness window the label rolls over to
"stale_approximate" so old-and-imprecise is distinguishable from
recent-and-imprecise.
The article's actual seendate is used as position_source_at instead
of now(), so the "last reported X days ago" badge is honest.
== #246: freshness is labelling, not eviction ==
The cache always preserves the last position the system observed,
forever. What changes is the position_confidence label:
- within configurable window (default 14d, env-overridable via
SHADOWBROKER_CARRIER_FRESHNESS_DAYS) -> "recent"
- older -> "stale"
- seed-bootstrap entries that were never refreshed -> "seed"
- homeport defaults (carrier added post-install) -> "homeport_default"
- headline-derived (any age, fresh) -> "approximate"
- headline-derived (older than window) -> "stale_approximate"
The position itself never reverts to the seed or the registry. The
user always sees the last position the system observed. Per the
user's explicit guidance: "from there have it be the last position
the user has logged the carriers that way a year from now it doesnt
revert to where the ships are today".
== Other improvements ==
- CACHE_FILE moved to backend/data/carrier_cache.json so it lives in
the volume-mounted dir under Docker compose. Previously it was at
/app/carrier_cache.json which got wiped on every container restart
(pre-existing bug).
- Atomic cache write (temp + os.replace) so a crash mid-write does
not leave a truncated cache file.
== Public API shape ==
Every carrier object the API emits now includes:
- position_confidence: seed | recent | stale | approximate |
stale_approximate | homeport_default
- position_source_at: ISO timestamp of when the underlying source
was observed (NOT now())
- is_fallback: convenience boolean for the UI; true when the
confidence is seed/stale/stale_approximate/
homeport_default
Existing fields (estimated, source, source_url, last_osint_update,
name, type, lat, lng, country, desc, wiki) are preserved exactly so
the current ShipPopup frontend renders unchanged. last_osint_update
now reflects position_source_at instead of now(), which is what the
existing "last reported MM/DD" badge always meant to show.
Tests: backend/tests/test_carrier_tracker_quality.py — 17 tests
covering seed bootstrap, subsequent-startup ignoring seed, no-seed/
no-cache homeport fallback, registry no longer has fallback fields,
freshness window labelling + env override, "year-old cache entry keeps
its position, only the label flips" regression, approximate
confidence for headline matches, GDELT seendate ISO parser, public
response shape backward compat.
Credit: tg12 (external security audit, three P1/P2 issues).
Gate messages now propagate via the Infonet hashchain as encrypted blobs — every node syncs them
through normal chain sync while only Gate members with MLS keys can decrypt. Added mesh reputation
system, peer push workers, voluntary Wormhole opt-in for node participation, fork recovery,
killwormhole scripts, obfuscated terminology, and hardened the self-updater to protect encryption
keys and chain state during updates.
New features: Shodan search, train tracking, Sentinel Hub imagery, 8 new intelligence layers,
CCTV expansion to 11,000+ cameras across 6 countries, Mesh Terminal CLI, prediction markets,
desktop-shell scaffold, and comprehensive mesh test suite (215 frontend + backend tests passing).
Community contributors: @wa1id, @AlborzNazari, @adust09, @Xpirix, @imqdcr, @csysp, @suranyami,
@chr0n1x, @johan-martensson, @singularfailure, @smithbh, @OrfeoTerkuci, @deuza, @tm-const,
@Elhard1, @ttulttul
New features:
- POTUS fleet (AF1, AF2, Marine One) with hot-pink icons + gold halo ring
- 9-color aircraft system: military, medical, police, VIP, privacy, dictators
- Sentinel-2 fullscreen overlay with download/copy/open buttons (green themed)
- Carrier homeport deconfliction — distinct pier positions instead of stacking
- Toggle all data layers button (cyan when active, excludes MODIS Terra)
- Version badge + update checker + Discussions shortcut in UI
- Overhauled MapLegend with POTUS fleet, wildfires, infrastructure sections
- Data center map layer with ~700 global DCs from curated dataset
Fixes:
- All Air Force Two ICAO hex codes now correctly identified
- POTUS icon priority over grounded state
- Sentinel-2 no longer overlaps bottom coordinate bar
- Region dossier Nominatim 429 rate-limit retry/backoff
- Docker ENV legacy format warnings resolved
- UI buttons cyan in dark mode, grey in light mode
- Circuit breaker for flaky upstream APIs
Community: @suranyami — parallel multi-arch Docker builds + runtime BACKEND_URL fix (PR #35, #44)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Former-commit-id: 7c523df70a2d26f675603166e3513d29230592cd
Shadowbroker