* api: purge reset data in the background
Resetting a large account deleted every sync collection in one Mongo
transaction, which outlived Mongo's time limit and the request timeout.
A reset now deletes the small collections (inbox keys and items,
monographs, devices) inline and then, in one settings update, clears the
keys, increments the user's Generation and marks a purge as pending.
Sync items are stamped with the generation they were pushed in, and
fetches and the encryption verifier only see the current one, so the old
data disappears as soon as the request returns. UserPurgeJob deletes the
older generations one collection at a time and clears the pending mark
only if no newer reset happened meanwhile.
Each push stamps its items inside its transaction by incrementing the
user's WriteRevision, so a reset committed during the push conflicts
with it and the retry gets the new generation. Items stored before this
change have no generation and count as generation 0.
* api: reject pushes that cross a reset
A push retried after a conflicting reset was stamped with the new
generation, although its items are still encrypted with the keys the
reset cleared. The push now keeps the generation it first saw across
retries and is rejected if a reset changed it.
Items are stamped through the captured replacements, so a lazily
enumerated input isn't left at generation 0.
Client errors thrown by a hub method now reach the client as hub
errors, since SignalR hides other exceptions' messages outside
development.
* api: reset under a lockout and clear token caches over RPC
Sessions are revoked while the user is locked out, retrying and then
aborting the reset if that fails. Hub connections, devices, inbox keys
and monographs from before a reset are fenced by the user's generation.
Identity now calls each server to clear cached tokens instead of
publishing a message, and the API and subscription server keep their
introspection cache in Redis so every replica sees the clear. Servers
without the call still get the message until they're upgraded.
Logins during a reset say so, and failed purges are counted for alerting.
* api: close the gaps the reset review found
- the old devices are purged one at a time in a transaction, so a device
registering again keeps the ids recorded for it
- the reset purge waits 15 minutes for requests still writing to the old
generation, then runs once
- republishing a monograph replaces a copy the purge hasn't removed yet
- inbox keys from before a reset no longer show or count toward the limit
- the reset holds a 10 minute lock through the attachments' removal, refuses
to overlap another reset, and keeps a lockout set while it ran
- the purge failure counter is exported from the first run
* api: only tag sync items with a generation
The user's monographs, inbox keys, inbox items and devices are few enough
to delete during the reset, before the generation changes, so only the
sync items need a generation and a background purge. UserPurgeJob now
deletes them itself, and the purge failure counter has no labels.
* api: reset in one transaction and keep devices that fetch across it unsynced
The generation bump, the user check and the deletes of monographs, inbox
keys, inbox items and devices run in one transaction, so a failed reset,
or one for a user without settings, leaves nothing deleted.
A fetch already under way when the reset commits no longer marks its
re-registered device as synced, so it still does a full sync after
logging in again.
* api: log out the other devices before resetting
They're told as soon as their sessions end, so their open sync
connections close before the reset rather than after it.
* api: log out the other devices after a password change even if their sessions weren't ended
* identity: keep ended sessions' cache keys until a clear succeeds
Ending sessions deletes them before their cached tokens are cleared, so
a retry after a failed clear found nothing to clear and succeeded while
the revoked tokens still worked from the cache.
* api: delete users in the background
Deleting a large account ran every collection's delete in one Mongo
transaction, which outlived Mongo's time limit and the request timeout.
The delete now marks the user's settings with DeletedAt and unregisters
the user from Identity, and UserPurgeJob purges the data after a grace
period, one idempotent step at a time, retrying until it's done. A user
that still exists in Identity is unmarked instead of purged.
A deleted user's open sync connections are cut off by a hub filter, which
also takes over the sync access check, and unregistering clears the
user's cached token introspection.
Identity drops delete_user and adds user_exists, so it must be deployed
before this API.
* api: simplify the background user deletion
* api: address review of the background user deletion
Covers uploads admitted by a cached token in the grace period, keeps a mark set by a retried delete, tests that open connections are cut off, and keeps delete_user for API instances that predate this change.
MinioFixture starts a MinIO-compatible server in Docker with Testcontainers (Silo, since MinIO no longer publishes images) and points S3Clients at it. S3ServiceTests cover signed uploads and object sizes, deleting an object and a user's directory, and multipart uploads within and past the storage limit. The interface stub moves out of ApiErrorResponseTests so both can use it. Running the tests now needs Docker.
Controllers no longer catch everything and return ex.Message as a 400, which leaked internal messages and made outages look like client errors. User-facing errors are now ClientExceptions; everything else is a generic 500 or a 503. S3 failover throws S3StorageUnavailableException when every endpoint fails transiently.
Deleting an account checks the password first, since Identity's "Wrong password." only arrived as a WAMP runtime error. Inbox API key checks no longer turn database errors into 401s. Monograph sanitization fails closed instead of storing unsanitized content, and an unreachable WebRisk counts as safe, like an error response already did.
Abdullah Atta