GitHub credential storage and migration

GitHub tokens live in credentials, with a unique (ownerId, provider) index. Repositories, gists, and pull requests use their existing owner field to resolve credentials. Users and resources have no credential reference fields. API login tokens remain hashed in users.apiTokens.

Each credential contains ownerId, provider, updatedAt, and encryptedToken: { version, keyId, nonce, ciphertext, tag }. AES-256-GCM uses a fresh 12-byte nonce and a 16-byte authentication tag. Additional authenticated data binds the ciphertext to the owner, provider, field, and format version. Plaintext exists only in application memory and requests to GitHub or the internal streamer. Use TLS for transport across untrusted networks.

Configuration

Set these secrets on the API, workers, and migration process:

CREDENTIAL_KEYS='{"2026-09":"<base64-encoded 32-byte random key>"}'
CREDENTIAL_ACTIVE_KEY_ID=2026-09
CREDENTIAL_LEGACY_READS=false

Generate the key with openssl rand -base64 32 and store it in your deployment's secret manager or protected environment file. Keep it out of Git, MongoDB, logs, and database backups. Keep a separately protected recovery copy. The application validates the keyring before connecting to the database; there is no temporary or default encryption key, including in development. The streamer consumes tokens from the API and does not need the key unless it also accesses the database.

CREDENTIAL_LEGACY_READS=true temporarily permits reads from legacy user/resource fields when a credential does not exist. All new writes are encrypted regardless of this setting. A corrupt envelope or missing decryption key fails explicitly; it never falls back to a plaintext token. Legacy tokens are not automatically refreshed: migrate them or log in to create an encrypted credential first.

Initial deployment

This release requires a maintenance window. Do not run old and new writers against the same database during migration. The --maintenance flag records the operator's acknowledgement; it does not stop processes or acquire a distributed lock.

Build the release and provision the keyring. Back up MongoDB and verify that the database and separately stored keys can be recovered. Rehearse on a protected copy first.
Stop all API instances, queue workers, scheduled tasks, and other database writers. Keep MongoDB and Redis running. Do not use the normal rolling deploy script for this initial upgrade. Run the migration as a one-off process using the new image, the existing service environment, and database network access.
Run the read-only inventory:
```
node build/scripts/migrate-credentials.js
```
From a source checkout with development dependencies, the equivalent is npm run migrate:credentials -- <flags>.
Resolve reported conflicting_token, malformed_token, and missing_owner records. Output contains collection names, record IDs, and counts, never token values. Conflicting owners retain all legacy fields and receive no new credential. An existing encrypted credential is never overwritten.

If differing resource tokens are obsolete, --prefer-owner-token explicitly chooses the existing encrypted credential, or otherwise the user's token, over conflicting copies. This can remove access supplied by a different token, so review the inventory first. It never arbitrarily chooses between distinct resource-only tokens. Resource-only tokens migrate when they all agree and their owner exists. Removed accounts do not receive credentials.
Backfill without deleting the legacy fields:
```
node build/scripts/migrate-credentials.js --apply --maintenance
```
Add --prefer-owner-token only for the conflict resolution described above. Investigate every nonzero exit status. The cursor processes bounded batches; reruns rescan owners and skip existing credentials, so no checkpoint file or exported plaintext is needed.
Remove legacy copies after successful backfill:
```
node build/scripts/migrate-credentials.js --apply --maintenance --remove-legacy
node build/scripts/migrate-credentials.js --verify
```
Use the same conflict-resolution flag if needed. Every owner's envelope is authenticated before their legacy fields are removed. Cleanup spans multiple collections and is not a transaction; keep writers stopped. An interruption is safe to resume with the same command. Verification must report legacy: 0 and exit successfully; it also authenticates all stored credentials and checks their owners.
Enforce the storage boundary in MongoDB and purge old Redis sessions:
```
node build/scripts/migrate-credentials.js --apply --maintenance --enforce
node build/scripts/purge-legacy-sessions.js
node build/scripts/purge-legacy-sessions.js --apply
```
Enforcement preserves existing collection validators and rejects legacy token fields on users and resources. Session cleanup scans only anoGH_session:*, removes the old object-based Passport session format, and leaves queues and ID-only sessions intact. Rerun its dry run to confirm found: 0. The application also rejects old session objects. Existing users must log in again.
Start only the new release with CREDENTIAL_LEGACY_READS=false. Test OAuth login, API-token login, private repositories, gists, pull requests, downloads, and account removal. Confirm startup creates the unique credential index.

A database-only dump cannot decrypt the credentials. Historical dumps, replica oplogs, Redis snapshots/AOF files, and existing logs may still contain plaintext until their retention expires. Migration does not erase those historical copies. If tokens were exposed previously, revoke/reissue them; encryption cannot undo an exposure.

Key rotation and rollback

Add a new random key to CREDENTIAL_KEYS on every reader before switching CREDENTIAL_ACTIVE_KEY_ID. New logins and refreshes use the active key; existing records remain readable through their keyId. Retain old keys until all records using them have been replaced and any backups needing them have expired. This migration command does not bulk re-encrypt existing credentials.

After encrypted writes begin, rollback must stay on an encryption-capable release with the same keyring. A pre-encryption release cannot read the new collection and will be rejected by the post-migration validators. Do not decrypt production data as a rollback procedure. If maintenance must be aborted before cutover, keep writers stopped and fix/retry the migration, or restore the pre-upgrade database under the original release as a coordinated recovery.

Verification tests

The regular suite tests cryptography, redaction, session contents, and the credential service. Run MongoDB integration tests against a disposable instance:

TEST_MONGODB_URI=mongodb://127.0.0.1:27028 npm test

The tests create randomly named databases and delete only those test databases. They cover stored ciphertext, hidden projections, concurrent credential writes, refresh, OAuth login, resource lookup, migration conflicts/reruns, missing owners, corruption, and MongoDB validators. Without TEST_MONGODB_URI, these integration tests are skipped.

Docker Compose command sequence

Run these blocks in a Bash session on the production host, from the existing Compose project directory, after checking out the reviewed release. Keep the same Compose project name and override files used by production. These commands assume the repository's local mongodb service, with its existing root credentials in MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD. If MONGODB_URI points elsewhere, back up that database instead. The host needs Python 3 and GPG.

Set the Compose command, preserve the running streamer count, and build without restarting production:

set -euo pipefail
set +x
umask 077
dc=(docker compose)
# If production uses the replica override, use instead:
# dc=(docker compose -f docker-compose.yml -f docker-compose.replica-primary.yml)
streamer_replicas=$("${dc[@]}" ps -q streamer | wc -l | tr -d ' ')
test "$streamer_replicas" -gt 0
"${dc[@]}" build anonymous_github

Generate the persistent key directly into .env, without printing it or putting it in shell history. This deliberately refuses to replace any existing key configuration. On a resumed migration, keep the existing keys and skip this generation step.

python3 <<'PY'
import base64, os, re, secrets
from pathlib import Path
env = Path('.env')
content = env.read_text()
if re.search(r'^\s*(?:export\s+)?CREDENTIAL_(?:KEYS|ACTIVE_KEY_ID|LEGACY_READS)\s*=', content, re.M):
    raise SystemExit('Credential configuration already exists; preserve it and review before continuing.')
key = base64.b64encode(secrets.token_bytes(32)).decode('ascii')
os.chmod(env, 0o600)
with env.open('a') as output:
    output.write('\nCREDENTIAL_KEYS=\'{"v1":"' + key + '"}\'\n')
    output.write('CREDENTIAL_ACTIVE_KEY_ID=v1\nCREDENTIAL_LEGACY_READS=false\n')
PY
"${dc[@]}" run --rm --no-deps -T --entrypoint node anonymous_github \
  -e 'require("./build/core/credentials").credentialCipher(); console.log("Credential keyring valid")'

Save the keyring in your secret manager or a separately protected recovery location before proceeding. Do not use docker compose config or cat .env in a recorded terminal: those can disclose secrets.

Enter maintenance and stop all application writers. In this Compose file the API starts the queue workers and scheduler. Stop any additional instances or external writers too; do not leave the rolling deploy script running.
```
"${dc[@]}" stop -t 120 anonymous_github streamer
test -z "$("${dc[@]}" ps --status running -q anonymous_github streamer)"
"${dc[@]}" ps mongodb redis
```

Take a fresh encrypted dump while writers are stopped. The password is read from the database container's environment into a temporary mode-0600 config in /dev/shm; it is never placed in command arguments. GPG prompts for a backup passphrase, which must be stored separately from the archive.

backup_dir=$(mktemp -d /var/tmp/anonymous-gh-migration.XXXXXXXX)
export GPG_TTY=$(tty)
"${dc[@]}" exec -T mongodb bash -se <<'SH' | gpg --symmetric --cipher-algo AES256 --output "$backup_dir/mongo.archive.gz.gpg"
set -euo pipefail
umask 077
export CREDENTIAL_DUMP_CONFIG=$(mktemp /dev/shm/credential-dump.XXXXXXXX)
trap 'rm -f "$CREDENTIAL_DUMP_CONFIG"' EXIT
mongosh --nodb --quiet --eval 'require("fs").writeFileSync(process.env.CREDENTIAL_DUMP_CONFIG, JSON.stringify({password: process.env.MONGO_INITDB_ROOT_PASSWORD}), {mode: 0o600})' >/dev/null
mongodump --host 127.0.0.1 --port 27017 \
  --username "$MONGO_INITDB_ROOT_USERNAME" --authenticationDatabase admin \
  --config "$CREDENTIAL_DUMP_CONFIG" --archive --gzip
SH
test -s "$backup_dir/mongo.archive.gz.gpg"
gpg --decrypt "$backup_dir/mongo.archive.gz.gpg" >/dev/null
printf 'Encrypted backup: %s\n' "$backup_dir/mongo.archive.gz.gpg"

The decryption check verifies the encrypted file, not MongoDB restoreability. Confirm your restore rehearsal succeeded on an isolated database before deleting legacy fields. Move this archive to your protected backup storage.

Inventory, backfill, then inventory again:
```
migrate() {
  "${dc[@]}" run --rm --no-deps -T --entrypoint node anonymous_github \
    build/scripts/migrate-credentials.js "$@"
}
migrate
migrate --apply --maintenance
migrate
```
Stop on any nonzero exit status or issues count. Do not add --prefer-owner-token automatically: review the conflict policy above first. After backfill, the second inventory should report created: 0, issues: 0.

After the backup/restore check and inventory pass, clean and enforce:

migrate --apply --maintenance --remove-legacy
migrate --verify
migrate --apply --maintenance --enforce
"${dc[@]}" run --rm --no-deps -T --entrypoint node anonymous_github \
  build/scripts/purge-legacy-sessions.js --apply
"${dc[@]}" run --rm --no-deps -T --entrypoint node anonymous_github \
  build/scripts/purge-legacy-sessions.js

Require legacy: 0 from MongoDB verification and found: 0 from the final session scan. Do not resume traffic on an unresolved error.

Recreate only the application services with the new image and environment:
```
"${dc[@]}" up -d --no-deps --force-recreate --wait \
  --scale "streamer=$streamer_replicas" streamer anonymous_github
"${dc[@]}" ps anonymous_github streamer
migrate --verify
```
Test login and a private repository/gist/pull-request download before ending maintenance. Existing sessions have been invalidated. If a command fails, keep the application stopped and fix/retry; do not launch the old release against the migrated database.

The one-off invocation follows Docker's Compose run documentation. The backup password handling uses MongoDB's mongodump configuration-file support.

13 KiB Raw Blame History