Shadowbroker/UPDATEPROTOCOL.md

ShadowBroker Release Protocol

This document exists because API keys were leaked in release zips v0.5.0, v0.6.0, and briefly v0.8.0. Follow this exactly. No shortcuts.

---

Pre-Release Checklist

1. Bump the Version

• frontend/package.json — update "version" field
• frontend/src/components/ChangelogModal.tsx — update CURRENT_VERSION and STORAGE_KEY
• Update NEW_FEATURES, BUG_FIXES, and CONTRIBUTORS arrays in the changelog modal

2. Pull Remote Changes First

git pull --rebase origin main

If there are merge conflicts, resolve them carefully. Do not blindly delete files during rebase — this is how the API proxy route (frontend/src/app/api/[...path]/route.ts) was accidentally deleted and broke the entire app.

After resolving conflicts, verify critical files still exist:

ls frontend/src/app/api/\[...path\]/route.ts   # API proxy — app is dead without this
ls backend/main.py
ls frontend/src/app/page.tsx

3. Test Before Committing

# Backend
cd backend && python -c "import main; print('Backend OK')"

# Frontend
cd frontend && npm run build

If the backend fails with a missing module, install it:

pip install -r requirements.txt

---

Building the Release Zip

The Command

Run from the project root (live-risk-dashboard/):

7z a -tzip ../ShadowBroker_vX.Y.Z.zip \
    -xr!node_modules -xr!.next -xr!__pycache__ -xr!venv -xr!.git -xr!.git_backup \
    -xr!*.pyc -xr!*.db -xr!*.sqlite -xr!*.xlsx \
    -xr!.env -xr!.env.local -xr!.env.production -xr!.env.development \
    -xr!carrier_cache.json -xr!ais_cache.json \
    -xr!tmp_fast.json -xr!dump.json -xr!debug_fast.json \
    -xr!nyc_sample.json -xr!nyc_full.json \
    -xr!server_logs.txt -xr!server_logs2.txt -xr!xlsx_analysis.txt -xr!liveua_test.html \
    -xr!merged.txt -xr!recent_commits.txt \
    -xr!build_error.txt -xr!build_logs*.txt -xr!build_output.txt -xr!errors.txt \
    -xr!geocode_log.txt -xr!tsconfig.tsbuildinfo \
    -xr!ShadowBroker_v*.zip \
    .

Critical Exclusions (NEVER ship these)

Pattern	Why
.env	Contains real API keys (OpenSky, AIS Stream)
.env.local	Contains real API keys (TomTom, etc.)
.env.production / .env.development	May contain secrets
carrier_cache.json / ais_cache.json	Runtime cache, not source
node_modules/ / __pycache__/ / .next/	Build artifacts
*.db / *.sqlite / *.xlsx	Data files, not source
ShadowBroker_v*.zip	Previous release zips sitting in the project dir

What SHOULD Be in the Zip

File	Required
frontend/src/app/api/[...path]/route.ts	YES — API proxy, app is dead without it
backend/.env.example	YES — template for users
.env.example	YES — template for users
backend/data/plane_alert_db.json	YES — aircraft database
backend/data/datacenters*.json	YES — data center layer
backend/data/tracked_names.json	YES — tracked aircraft names
frontend/src/lib/airlines.json	YES — airline codes
start.bat / start.sh	YES — launcher scripts

Do NOT Use

• git archive — includes tracked junk, misses untracked essential files
• Compress-Archive (PowerShell) — has lock file issues, no exclusion control
• Gemini's zip script — included test files, debug outputs, .env with real keys, and 30+ unnecessary files

---

Post-Build Audit (MANDATORY)

Before uploading, always scan the zip for leaks:

# Check for .env files (should only show .env.example files)
7z l ShadowBroker_vX.Y.Z.zip | grep -i "\.env" | grep "....A"

# Check for anything with "secret", "key", "token", "credential" in the filename
7z l ShadowBroker_vX.Y.Z.zip | grep -iE "secret|api.key|credential|token" | grep "....A"

# Check the largest files (look for unexpected blobs)
7z l ShadowBroker_vX.Y.Z.zip | grep "....A" | awk '{print $4, $NF}' | sort -rn | head -15

# Verify the API proxy route exists
7z l ShadowBroker_vX.Y.Z.zip | grep "route.ts"

Expected results:

• .env files: ONLY .env.example and next-env.d.ts
• No files with "secret"/"credential" in the name
• Largest files: plane_alert_db.json (~4.6MB), datacenters_geocoded.json (~1.2MB), airlines.json (~800KB)
• route.ts exists under frontend/src/app/api/[...path]/
• Total zip size: ~1.7MB (as of v0.8.0). If it's 5MB+ something leaked.

---

Commit, Tag, and Push

# Stage specific files (NEVER use git add -A)
git add <specific files>

# Commit
git commit -m "v0.X.0: brief description of release"

# Tag
git tag v0.X.0

# Push (pull first if remote has new commits)
git pull --rebase origin main
git push origin main --tags

# If the tag was created before rebase, re-tag on the new HEAD:
git tag -f v0.X.0
git push origin v0.X.0 --force

---

Creating the GitHub Release

Via GitHub API (when gh CLI is unavailable)

# 1. Create the release
import urllib.request, json

body = {
    "tag_name": "v0.X.0",
    "name": "v0.X.0 — Title Here",
    "body": "Release notes here...",
    "draft": False,
    "prerelease": False
}

# Write to a temp file to avoid JSON escaping hell in bash
with open("release_body.json", "w") as f:
    json.dump(body, f)

# POST to GitHub API...

# 2. Upload the zip asset to the release
# Use the upload_url from the release response

Via gh CLI (if installed)

gh release create v0.X.0 ../ShadowBroker_v0.X.0.zip \
    --title "v0.X.0 — Title" \
    --notes-file RELEASE_NOTES.md

---

Post-Release Verification

After uploading, download the release zip from GitHub and verify it:

# Download what GitHub is actually serving
curl -L -o /tmp/verify.zip "https://github.com/BigBodyCobain/Shadowbroker/releases/download/v0.X.0/ShadowBroker_v0.X.0.zip"

# Scan for leaks (same audit as above)
7z l /tmp/verify.zip | grep -i "\.env" | grep "....A"

# Compare hash to your local copy
md5sum /tmp/verify.zip ../ShadowBroker_v0.X.0.zip

---

If You Discover a Leak

Immediate Actions

1. Rebuild the zip without the leaked file
2. Delete the old asset from the GitHub release via API
3. Upload the clean zip as a replacement
4. Rotate ALL leaked keys immediately:
   • OpenSky: https://opensky-network.org/
   • AIS Stream: https://aisstream.io/
   • Any other keys found in the leak
5. Audit ALL other releases — leaks tend to exist in multiple versions

Audit All Releases Script

import urllib.request, json

TOKEN = "your_token"
headers = {"Authorization": f"token {TOKEN}", "Accept": "application/vnd.github+json"}

# Get all releases
req = urllib.request.Request(
    "https://api.github.com/repos/BigBodyCobain/Shadowbroker/releases",
    headers=headers
)
releases = json.loads(urllib.request.urlopen(req).read())

for r in releases:
    for asset in r.get("assets", []):
        # Download via API
        req2 = urllib.request.Request(
            asset["url"],
            headers={**headers, "Accept": "application/octet-stream"}
        )
        data = urllib.request.urlopen(req2).read()
        filename = f"/tmp/{r['tag_name']}.zip"
        with open(filename, "wb") as f:
            f.write(data)
        print(f"Downloaded {r['tag_name']}: {len(data)} bytes")
        # Then run 7z l on each to check for .env files

---

Lessons Learned (v0.8.0 Incident)

1. Rebasing can silently delete files. After git pull --rebase, always verify that critical files like the API proxy route still exist.
2. The zip command must explicitly exclude .env and .env.local. These files are not in .gitignore patterns that 7z understands — you must pass -xr!.env -xr!.env.local every time.
3. Always audit the zip before uploading. A 10-second grep saves a key rotation.
4. Never trust another tool's zip output. Gemini's zip included .env with real keys, 30+ test files, debug outputs, and sample JSON dumps.
5. 2,000+ stars means 2,000+ potential eyes on every release. Treat every zip as if it will be decompiled line by line.