In a versioned bucket, HEAD on a key whose latest version is a delete
marker returns 405 MethodNotAllowed instead of 404. GetObjectSizeAsync
threw on it, so re-adding a previously deleted attachment (same hash,
same key) failed with a 500 before the upload started.
* sync: keep pushed ciphers as UTF-8 from the hub to MongoDB
Pushed ciphers were parsed into strings, then encoded back to UTF-8 by
the driver into a new buffer for each large one: the two largest
sources of large object heap allocations on prod.
PushItems now reads each cipher straight into a pooled UTF-8 buffer and
UpsertMany writes the item's BSON itself, from that buffer, into another
pooled one. The stored documents are unchanged. Validation moves to
SyncItemValidator, and the base64 check is vectorized.
* sync: hand back pushed buffers when a push is rejected
The hub disposed the pushed items only after its own checks, and the
authorization filter rejects a call after SignalR has bound it, so those
rejections left their pooled buffers to the GC. A push with null items
also made the cleanup throw over the real error.
* api: drop the MessagePack hub protocol
* sync: fetch items in size-bounded chunks and send them as stored
* sync: label fetch failures by reason and log rejected connections at info
* sync: count rejected connections and cover fetch failure reasons and mid-fetch resets
Rejections are logged at Information, which production filters out, so a
counter by stage is what shows them. Also drops a catch around the no-op
Hub.OnDisconnectedAsync.
* api: purge reset data in the background
Resetting a large account deleted every sync collection in one Mongo
transaction, which outlived Mongo's time limit and the request timeout.
A reset now deletes the small collections (inbox keys and items,
monographs, devices) inline and then, in one settings update, clears the
keys, increments the user's Generation and marks a purge as pending.
Sync items are stamped with the generation they were pushed in, and
fetches and the encryption verifier only see the current one, so the old
data disappears as soon as the request returns. UserPurgeJob deletes the
older generations one collection at a time and clears the pending mark
only if no newer reset happened meanwhile.
Each push stamps its items inside its transaction by incrementing the
user's WriteRevision, so a reset committed during the push conflicts
with it and the retry gets the new generation. Items stored before this
change have no generation and count as generation 0.
* api: reject pushes that cross a reset
A push retried after a conflicting reset was stamped with the new
generation, although its items are still encrypted with the keys the
reset cleared. The push now keeps the generation it first saw across
retries and is rejected if a reset changed it.
Items are stamped through the captured replacements, so a lazily
enumerated input isn't left at generation 0.
Client errors thrown by a hub method now reach the client as hub
errors, since SignalR hides other exceptions' messages outside
development.
* api: reset under a lockout and clear token caches over RPC
Sessions are revoked while the user is locked out, retrying and then
aborting the reset if that fails. Hub connections, devices, inbox keys
and monographs from before a reset are fenced by the user's generation.
Identity now calls each server to clear cached tokens instead of
publishing a message, and the API and subscription server keep their
introspection cache in Redis so every replica sees the clear. Servers
without the call still get the message until they're upgraded.
Logins during a reset say so, and failed purges are counted for alerting.
* api: close the gaps the reset review found
- the old devices are purged one at a time in a transaction, so a device
registering again keeps the ids recorded for it
- the reset purge waits 15 minutes for requests still writing to the old
generation, then runs once
- republishing a monograph replaces a copy the purge hasn't removed yet
- inbox keys from before a reset no longer show or count toward the limit
- the reset holds a 10 minute lock through the attachments' removal, refuses
to overlap another reset, and keeps a lockout set while it ran
- the purge failure counter is exported from the first run
* api: only tag sync items with a generation
The user's monographs, inbox keys, inbox items and devices are few enough
to delete during the reset, before the generation changes, so only the
sync items need a generation and a background purge. UserPurgeJob now
deletes them itself, and the purge failure counter has no labels.
* api: reset in one transaction and keep devices that fetch across it unsynced
The generation bump, the user check and the deletes of monographs, inbox
keys, inbox items and devices run in one transaction, so a failed reset,
or one for a user without settings, leaves nothing deleted.
A fetch already under way when the reset commits no longer marks its
re-registered device as synced, so it still does a full sync after
logging in again.
* api: log out the other devices before resetting
They're told as soon as their sessions end, so their open sync
connections close before the reset rather than after it.
* api: log out the other devices after a password change even if their sessions weren't ended
* identity: keep ended sessions' cache keys until a clear succeeds
Ending sessions deletes them before their cached tokens are cleared, so
a retry after a failed clear found nothing to clear and succeeded while
the revoked tokens still worked from the cache.
* api: delete users in the background
Deleting a large account ran every collection's delete in one Mongo
transaction, which outlived Mongo's time limit and the request timeout.
The delete now marks the user's settings with DeletedAt and unregisters
the user from Identity, and UserPurgeJob purges the data after a grace
period, one idempotent step at a time, retrying until it's done. A user
that still exists in Identity is unmarked instead of purged.
A deleted user's open sync connections are cut off by a hub filter, which
also takes over the sync access check, and unregistering clears the
user's cached token introspection.
Identity drops delete_user and adds user_exists, so it must be deployed
before this API.
* api: simplify the background user deletion
* api: address review of the background user deletion
Covers uploads admitted by a cached token in the grace period, keeps a mark set by a retried delete, tests that open connections are cut off, and keeps delete_user for API instances that predate this change.
- revoke tokens through one OpenIddictSessionStore.RevokeTokenAsync
- strip the legacy lineage prefix in one place (LegacyGrants.LineageKey)
- reuse the user AccessTokens loads when introspecting
- drop support for unencrypted token keys, which no deployment has
The password reset's access token comes without a refresh token, so OpenIddict
links it to no session, and logging out with it ended nothing: it stayed valid
until it expired. IdentityServer4 revoked it. Logout now revokes the token
itself, and removes its IdentityServer4 copy.
"Streetwriters.Identity keys replace", run in the Identity container, makes
a new signing and encryption key and retires every other one, so the next
start uses only the new ones, ending every session. "keys list" shows them
all. Identity loads the newest key that isn't retired, and refuses to start
if every key is, rather than make new ones.
Release builds link to IDENTITY_SERVER_URL in the emails they send, which the tests didn't set, and email the password reset link that debug builds answer with.
IdentityServer4 accepted "bearer" and "BEARER" on the account and MFA APIs, where
OpenIddict answered 401, which the apps take as being signed out. Records what a browser
sees from another origin (preflights, success and error responses) on both servers.
They were stored as is, so every backup held the signing key, with which anyone could
mint the tokens of the sign in steps and skip 2FA for accounts whose password they
have. They're now encrypted with a key derived from NOTESNOOK_API_SECRET. Keys that
can't be decrypted stop the start instead of being replaced, which would end every
session; NOTESNOOK_API_SECRET_PREVIOUS re-encrypts them after the secret changes, and
keys stored by earlier builds are encrypted on start.
Every instance registered the apps' client on startup, so on a new database all but
one failed with a duplicate key, and on an existing one they all updated it at once.
The ones that lose the race now retry and find it registered, and it's only updated
when its definition changed.
OpenIddict's pruning only removes tokens older than a threshold, which had to be the 45
days a redeemed refresh token stays usable, so every refresh's tokens stayed that long:
a session refreshed for a month kept 244 tokens. Pruning now removes expired, revoked
and rejected tokens and their records, and keeps valid ones and the replaced refresh
tokens that still work. The same session keeps 4.
Replays each call packages/core makes to Identity, encoded as utils/http.ts encodes it
(form bodies without falsy values, bearer tokens), signing in through the apps' email,
code and password steps, and records what IdentityServer4 answers for OpenIddict to
match. Both give the same answers to all of them.
Also records three requests the server doesn't serve as the apps expect, on either:
GET /account/sessions (404), DELETE /mfa to disable 2FA (405), and /mfa/send with a
session token (403).
The apps refresh with the scope of the token they have, and sessions started at signup
hold "mfa" and "openid", which OpenIddict rejected, logging those users out.
IdentityServer4 ignored the scope of refresh requests and kept the session's, and so
does OpenIddict now; signup sessions also get IdentityServer4's scopes, and refreshed
sessions the client's six hour access tokens rather than signup's hour.
The tests now refresh as the apps do, including restored IdentityServer4 sessions, and
compare scopes as sets: IdentityServer4 ordered the same scopes differently in
different responses.
Runs the API servers' introspection client, with their options, against IdentityServer4
and OpenIddict: same claims, same authorization, and OpenIddict's issuer is accepted. The
API servers need no changes.
Exposes Prometheus metrics on port 5068, as the API does on 5067, including the count of
IdentityServer4 tokens still in use.
Every session OpenIddict starts is also written to IdentityServer4's grants, as
IdentityServer4 would have stored it, and removed from them when it ends, so a rollback
to IdentityServer4 logs nobody out. The rollback drill runs both servers on one database,
back and forth, and checks the API servers see the same as with IdentityServer4's own
tokens.
Replaces Quartz with a background service for IdentityServer4's grant cleanup: it ran
just that job, and its static logger broke a second server in the same process.
Sessions IdentityServer4 started are read from its grants until they expire. A legacy
refresh token is refreshed into an OpenIddict session, and still works until the token
that replaced it is used; legacy access tokens introspect and authenticate as before;
logout, clearing sessions, account deletion and revocation end them as IdentityServer4
did. Legacy handles are told apart by their format, so new tokens never touch the old
grants. Counts their use in identity_legacy_token_hits, to tell when nothing needs them.
The restored session tests run on both servers, with new ones for sessions that carry on
across the switch.
Setting IDENTITY_ENGINE=openiddict runs OpenIddict 7 instead of IdentityServer4, which
stays the default. OpenIddict issues and stores sessions; the token endpoint answers
everything else in IdentityServer4's exact shape, and introspection, revocation and the
account API's authentication are our own, so clients and the API servers see no change.
Token records keep the jti format, the API servers' cache keys and the refresh token
lineage that makes a replaced refresh token work until its replacement is used.
The contract tests now run against both servers, from the snapshots recorded against
IdentityServer4. Pins the Microsoft.IdentityModel packages to 8.19.2, since OpenIddict's
versions broke IdentityServer4's JWTs.
Moves the password, email, mfa and mfa_password grant logic into GrantService,
which returns engine-neutral results; the IdentityServer4 validators are now thin
adapters. Sessions go through ISessionStore and tokens issued outside the token
endpoint through ITokenIssuer, so the controllers and account service no longer
depend on IdentityServer4.
The email grant answered unregistered emails with invalid_grant, because
ProfileService rejected the stand-in user. They now get the same step token and
the same mfa errors as a registered user with email MFA, and step tokens no
longer carry user claims.
Golden snapshots of every grant, refresh, introspection and session flow, run against the whole Identity server in-process. Legacy sessions exported as IdentityServer4 stores them, with tests that restore and exercise them.
Re-check inactivity inside the transaction so a device that syncs in the
meantime keeps its ids, and scope the id deletion by user so another
user's device with the same id is left alone.
Identity already ran on driver 3.x through AspNetCore.Identity.Mongo 10.1.0 and Streetwriters.IdentityServer4.Contrib.MongoDB 5.0.0, while Streetwriters.Data and Streetwriters.Common were built against 2.30, so shared code could hit APIs removed in 3.0 at runtime in Identity only. Every project now uses 3.12.
- MongoDB.Driver.Core is part of MongoDB.Driver in 3.x
- AspNetCore.Identity.Mongo 10.1.0 in Common, matching Identity, which needs Microsoft.Extensions.DependencyInjection 9.0.2
- AspNetCore.HealthChecks.MongoDb 9.0.0, built against driver 3.x
- tests use EphemeralMongo 3.2, as EphemeralMongo8 2.0 is built against driver 2.x; it no longer stops mongod on exit, so the Identity tests now dispose their runner like the API tests do
Committing or discarding the keys of a password change that failed midway trusted the client to have just logged in. An older session could discard keys for a password that had been set, leaving them unwrappable. Both now require the token's auth_time to be after the keys were staged, and return login_required otherwise. Refreshed tokens keep the auth_time of the original login.
Discarding still waits until the password change can no longer be running.
The fetch filter compared pending monograph IDs with _id as strings, but _id is stored as an ObjectId, so monographs without an ItemId were only ever sent during a full sync. Valid IDs are now parsed into ObjectIds first, as MonographsController already does; malformed ones are only matched against ItemId.
DeleteDirectoryAsync sent every key in one DeleteObjects request, which S3 caps at 1000, and ignored keys S3 failed to delete. It now uses the same batching as DeleteObjectsAsync, and lists objects under "{userId}/" so the prefix can't match another user whose ID starts with the same characters.
Account deletion and reset delete attachments last, after the user's data is already gone, so a failure there is now logged instead of turning a completed deletion into an error.
Identity publishes ClearCache to a service's WAMP address, which reaches only one replica, so a revoked token stayed valid on the others for up to 30 minutes. With CACHE_REDIS_CONNECTION_STRING set, the replica that receives it publishes the keys on Redis and every replica removes them from its in-memory cache. The cache itself stays in memory so a Redis outage doesn't affect authentication; without Redis, only the receiving instance is cleared, as before.
MinioFixture starts a MinIO-compatible server in Docker with Testcontainers (Silo, since MinIO no longer publishes images) and points S3Clients at it. S3ServiceTests cover signed uploads and object sizes, deleting an object and a user's directory, and multipart uploads within and past the storage limit. The interface stub moves out of ApiErrorResponseTests so both can use it. Running the tests now needs Docker.
Fetches and pushes with a null or empty device ID were accepted: a fetch registered a device with no ID, which every such request from that user then shared. Both now fail with a HubException before anything is registered or written.
Concurrent signups for the same email can all pass Identity's duplicate check, which left duplicate users. With unique indexes on NormalizedEmail and NormalizedUserName, all but one now fail with a duplicate-key error, which signup reports as "Unable to create an account on this email." instead of a generic error. Adds an Identity test project.
Self-hosted and new deployments got none of the API's indexes, and without unique ones concurrent upserts can duplicate sync items. A background service creates any missing index in the notesnook database, including unique UserId + ItemId on sync collections and unique sync_devices and user_settings keys. Existing indexes are never changed: one with the same name but a different definition is logged. Each index is created on its own, so existing duplicates only skip that index, and no failure stops the host.
Registering a device inserted a new document every time, so concurrent registrations (two first fetches at once) left duplicate devices, and one stuck with IsSyncReset=true full-synced on every fetch. Registration is now an upsert on UserId + DeviceId; registering an existing device resets it to a full sync.
Signup ignores a duplicate-key error when inserting user settings: until Identity's users are unique, a double-submitted signup can resolve to the same user twice.
Controllers no longer catch everything and return ex.Message as a 400, which leaked internal messages and made outages look like client errors. User-facing errors are now ClientExceptions; everything else is a generic 500 or a 503. S3 failover throws S3StorageUnavailableException when every endpoint fails transiently.
Deleting an account checks the password first, since Identity's "Wrong password." only arrived as a WAMP runtime error. Inbox API key checks no longer turn database errors into 401s. Monograph sanitization fails closed instead of storing unsanitized content, and an unreachable WebRisk counts as safe, like an error response already did.
The hub's two fetch lists, its push map, the encryption verifier lookup and the account delete/reset lists each named every collection. They now all come from SyncItemCollections on the repository accessor, which also marks the legacy settings collection as deleted but not synced. A test checks every repository is registered.
The quota was checked, then the file uploaded, then usage incremented, so concurrent uploads could exceed it. Reserve the size with a conditional increment before uploading and release it if the upload fails. Also only roll over the usage that was read in GET /users, and make Limit.Value a plain property so setting it doesn't change UpdatedAt.
Group clients by user on connect so per-user events don't scan every client, and time out each send after 10s so a stuck client can't stall the heartbeat.
Publishing now waits at most 10s for a connection. Opening the channel
is guarded by a lock so concurrent callers share one connection, and
disconnect handlers are attached once per channel so they no longer
multiply on every reconnect.
Abdullah Atta