# Plan: cache profile summaries in one process ## Measured problem and accepted scope The existing profile-summary service has one active process. A one-week trace shows repeated reads of about 900 hot keys: DB CPU is 70%, with read p95 120 ms. Add a process-local LRU wrapper to the existing repository. Acceptance targets are at least 60% cache hits, DB CPU below 50%, and read p95 below 60 ms, with the existing error-rate and correctness SLOs unchanged. This is an internal backend change with no UI, API, schema, pricing, or developer onboarding change. ## Existing contracts retained - All reads and writes use this repository in the same process; there are no external DB writers. Multi-process operation remains unsupported and startup rejects that configuration while caching is enabled. - Authentication and authorization run before repository access. Keys encode the authenticated tenant ID and validated profile ID without ambiguity. Values are immutable profile-summary DTOs; secrets and cache keys are never logged. Cached results cannot bypass authorization. - The existing LRU adapter supports 1000 entries, a 16 MiB byte cap, and a 30-second TTL. Recorded hot data fits those limits. Absent records use a distinct sentinel with a 10-second TTL; undefined means a cache miss. - Cache operations are synchronous and atomic in the single JS event loop. On any cache failure the existing adapter bypasses the cache until an empty cache is reinitialized; repository errors keep the current typed API error mapping. The existing per-key single-flight wrapper coalesces simultaneous misses and releases on failure. - A read already in progress when a write commits may return its earlier DB snapshot to that caller. Every read begun after that write completes must observe the committed version. TTL expiry is not a substitute for this rule. ## Proposed wrapper integration Keep the current read-through repository interface and shared adapters. These are the complete new read/write ordering rules; no additional version checks or coordination between a cache fill and a write are proposed: ```javascript async function readProfile(key) { const cached = cache.get(key); if (cached !== undefined) return cached; const value = await repository.read(key); cache.set(key, value); return value; } async function writeProfile(key, update) { const saved = await repository.write(key, update); cache.delete(key); return saved; } ``` ## Verification and rollout Existing repository contract tests cover tenant isolation, key validation, absence, DB failures, and authorization. New wrapper tests cover hit/miss, eviction and byte limits, TTL, adapter-failure fallback, successful-write invalidation, failed-write preservation, and concurrent-miss coalescing. The rollout uses the existing runtime feature flag: enable for 10% of keys, then 50%, then all keys after one healthy hour at each stage. Monitor hit/miss, eviction, cache bytes, fallback errors, DB CPU, and read p95 without raw IDs. On error-rate or latency regression, disable the flag immediately; both reads and writes bypass the cache while disabled, and enabling creates an empty cache. Cold starts remain within the existing DB capacity. The service owner monitors the rollout and records the results against the acceptance targets. ## Out of scope Distributed caching, cross-process coherence, prewarming, changing consistency semantics, or adding new product surfaces. The repository interface preserves a future replacement path without introducing a general cache framework now. --- ## Plan Amendment — S4-1: Stale write-back race (accepted, in-scope) **Finding:** The pseudocode allows a read dispatched before a write to re-populate the cache with a stale value after `cache.delete` fires. Scenario: A dispatches `repo.read(key)` [awaiting]; B dispatches `repo.write(key, update)` [awaiting]; B's write resolves, `cache.delete(key)` fires; A's read resolves, `cache.set(key, stale_value)` fires — subsequent callers see stale data until TTL (30 s). The plan states "TTL expiry is not a substitute for this rule," so this violates the accepted invariant. **Remedy — generation counter:** Maintain a `Map` (`writeGen`) alongside the cache. Increment on every `cache.delete`. Before dispatching `repo.read`, snapshot the generation for this key. After `repo.read` returns, only call `cache.set` if the generation is unchanged. Amended pseudocode (replaces the "Proposed wrapper integration" block above): ```javascript const writeGen = new Map(); // per-key write generation counter async function readProfile(key) { const cached = cache.get(key); if (cached !== undefined) return cached; const gen = writeGen.get(key) ?? 0; // snapshot generation before await const value = await repository.read(key); if ((writeGen.get(key) ?? 0) === gen) { // skip set if write occurred cache.set(key, value); } return value; } async function writeProfile(key, update) { const saved = await repository.write(key, update); writeGen.set(key, (writeGen.get(key) ?? 0) + 1); // invalidate generation cache.delete(key); return saved; } ``` `writeGen` grows at one entry per ever-written key. In a bounded hot-key workload (~900 keys) this is negligible. Memory-bounded by the same key population as the cache. **Additional test required:** Concurrent read+write with controlled promise resolution ordering — verify `cache.set` is skipped when a write has occurred since the read was dispatched, and subsequent reads go to DB. --- ## Step 0 Analysis (HOLD SCOPE) **Mode:** HOLD SCOPE — scope accepted as stated; review for correctness, edge cases, observability, and deployment safety only. **0A Premise:** DB CPU 70%, p95 120 ms, ~900 repeated hot keys — real, measured problem. Process-local LRU is the most direct solution. Doing nothing leaves DB near saturation. **0B Existing leverage:** Repository interface, LRU adapter, single-flight wrapper, feature flag, auth middleware, typed error mapping — all reused. No parallel build. **0C Dream state delta:** ``` CURRENT STATE THIS PLAN 12-MONTH IDEAL DB CPU 70% ──► DB CPU <50% ──► Same targets at scale p95 120 ms p95 <60 ms Multi-process path open No caching ≥60% hit rate (not in scope now) ``` **0C-bis Alternatives (auto-selected: Approach A):** ``` APPROACH A: Thin wrapper (proposed plan) Effort: S Risk: Low Pros: minimal diff; no new abstractions; preserves interface Cons: stale write-back race (addressed by amendment above) Reuses: all existing adapters APPROACH B: Ideal architecture (generation counter + explicit eviction events) Effort: S Risk: Low Pros: cleaner invariant enforcement; extensible Cons: slightly more code Reuses: same as A RECOMMENDATION: Approach A amended with generation counter (= Approach B). ``` **0E Temporal interrogation (HOLD SCOPE — key decisions only):** - Hour 1: `writeGen` must be initialized before any reads (simple `new Map()`). - Hours 2-3: Single-flight interacts with generation check — single-flight coalesces misses; the generation check is per-flight, not per-waiter. All waiters on one flight share the same generation snapshot captured before the flight starts. Correct: they all skip `cache.set` if a write intervened. - Hours 4-5: Feature flag disable: bypass all cache ops including `writeGen` mutations? Confirm: when bypassed, `writeGen` need not be maintained; it remains consistent for re-enable. - Hours 6+: TTL eviction does not interact with `writeGen` (TTL expiry = cache miss = next read snapshots current generation). Correct. --- ## Required Outputs ### NOT in scope 1. Distributed caching — out of scope, future path preserved 2. Cross-process coherence — single-process constraint enforced at startup 3. Cache prewarming — cold-start within existing DB capacity 4. Consistency semantics changes — contracts unchanged 5. New API/UI/schema/product surfaces — backend-only change 6. Multi-process operation — rejected at startup while cache enabled ### What already exists 1. Repository read/write interface — wrapped directly, not replaced 2. LRU adapter (1000 entries, 16 MiB, 30 s TTL) — reused as-is 3. Per-key single-flight wrapper — coalesces concurrent misses 4. Runtime feature flag — controls rollout stages 5. Tenant isolation + key validation — runs before cache check 6. Auth middleware — runs before repository access 7. Typed API error mapping — preserved unchanged ### Dream state delta This plan closes 60+ % of DB read load at the cost of a small in-process Map and a generation counter. Distributed caching (multi-process, cross-host coherence) remains unbuilt but the interface leaves that path open without lock-in. ### Error & Rescue Registry (Section 2) | Method | What can fail | Exception | Rescued? | Action | User sees | |--------|--------------|-----------|----------|--------|-----------| | readProfile | cache.get | CacheAdapterError | Y (existing) | Bypass cache | Transparent | | readProfile | repository.read | Typed DB errors | Y (existing) | Existing mapping | Existing error response | | readProfile | cache.set | CacheAdapterError | Y (existing) | Bypass + reinit | Transparent | | writeProfile | repository.write | Typed DB errors | Y (existing) | Existing mapping | Existing error response | | writeProfile | cache.delete | CacheAdapterError | Y (existing) | Bypass + reinit | Transparent | No catch-all handlers introduced. All new paths route through existing typed error handling. ### Failure Modes Registry | Codepath | Failure mode | Rescued? | Test? | User sees | Logged? | |----------|-------------|----------|-------|-----------|---------| | readProfile | Cache adapter fail | Y | Y (planned) | Transparent (bypass) | Y | | readProfile | DB read fail | Y | Y (existing) | Existing error | Y | | readProfile | Stale write-back race | Y (after amendment) | Y (added) | None — gap closed | N/A | | writeProfile | DB write fail | Y | Y (planned) | Existing error | Y | | writeProfile | cache.delete fail | Y | Y (planned: adapter-failure) | Transparent | Y | | rollout | Error-rate/latency regression | Y | — | Disable flag → bypass | Y | No CRITICAL GAPS after amendment. ### TODOS.md updates None. No deferred items surfaced in HOLD SCOPE — all gaps addressed in scope. --- ## Diagrams ### 1. System Architecture ``` ┌──────────────────────────────────────────────────────┐ │ Profile Summary Service │ │ (single JS process) │ │ │ │ readProfile(key) writeProfile(key,update)│ │ │ │ │ │ ▼ ▼ │ │ ┌────────────────────────────────────────────────┐ │ │ │ LRU Cache Wrapper │ │ │ │ cache.get / cache.set / cache.delete │ │ │ │ writeGen: Map (amendment S4-1) │ │ │ │ 1000 entries · 16 MiB cap · 30 s TTL │ │ │ │ Absent sentinel: 10 s TTL │ │ │ └──────┬──────────────────────────┬──────────────┘ │ │ hit │ miss │ write: gen++ │ │ ◄──────┘ │ │ + cache.delete │ │ ▼ │ │ │ ┌─────────────────────┐ │ │ │ │ Single-Flight │ │ │ │ │ Wrapper (per-key) │ │ │ │ └──────────┬──────────┘ │ │ └─────────────┼──────────────────────┼─────────────────┘ │ DB read │ DB write ▼ ▼ ┌───────────────────────────────┐ │ Repository (DB) │ └───────────────────────────────┘ Feature flag OFF → all cache + writeGen ops bypassed Feature flag ON → routes through LRU wrapper ``` ### 2. Data Flow (including shadow paths) ``` READ PATH: key ──► cache.get(key) │ hit │ miss (undefined) ◄────┘ │ ▼ snapshot gen = writeGen.get(key) ?? 0 │ ▼ single-flight.coalesce(key) │ ▼ await repository.read(key) │ ┌─────┴──────────────────────────────────┐ │ value (DTO) │ absent │ error │ ▼ ▼ ▼ │ gen unchanged? gen unchanged? propagate │ cache.set(k,v) cache.set(k, no cache.set │ (30 s TTL) SENTINEL, │ 10 s TTL) │ gen changed? gen changed? │ skip set skip set │ └─────────────────────────────────────────┘ │ ▼ return value / SENTINEL / throw Shadow paths: nil key: existing key validation → typed error empty key: existing key validation → typed error error: no cache.set, typed error propagates adapter fail: bypass mode (existing adapter behavior) WRITE PATH: (key, update) ──► await repository.write(key, update) │ success │ failure ──────────┤ ─────────────────────── ▼ │ writeGen NOT incremented writeGen.set(key, gen+1) cache.delete NOT called cache.delete(key) error propagates │ ▼ return saved ``` ### 3. State Machine (cache entry) ``` [ABSENT/MISS] │ │ cache.set(key, value) ← read completes, gen unchanged ▼ [POPULATED: value, TTL=30s] ──── TTL expires ──► [ABSENT/MISS] │ │ cache.delete ← write commits (gen incremented first) ▼ [ABSENT/MISS] │ │ cache.set(key, SENTINEL) ← absent DB record, gen unchanged ▼ [ABSENT-SENTINEL, TTL=10s] ──── TTL expires ──► [ABSENT/MISS] Invalid/guarded transitions: cache.set skipped when gen changed since read dispatch (amendment S4-1) cache.delete of absent key: no-op (safe) Feature flag OFF: all transitions suppressed → BYPASSED state ``` ### 4. Error Flow ``` cache.get(key) → error │ ▼ Adapter bypass mode (existing behavior) │ ▼ repository.read(key) → existing typed error handling │ success → return value (no cache.set in bypass mode) cache.set(key, v) → error → adapter bypass (same) cache.delete(key) → error → adapter bypass (same) writeGen.set(key, n) → synchronous Map op, no failure path ``` ### 5. Deployment Sequence ``` 1. Deploy service (cache wrapper code present, flag off) 2. Verify: single-process constraint check at startup passes 3. Enable flag at 10% (traffic or key subset — clarify semantics) 4. Monitor: hit/miss rate, eviction rate, cache bytes, fallback errors, DB CPU, read p95 5. After 1 healthy hour: expand to 50% 6. Monitor again for 1 hour 7. Expand to 100% 8. Record results against acceptance targets: ≥60% hit rate, DB CPU <50%, p95 <60 ms ``` ### 6. Rollback Flowchart ``` Regression detected (error rate ↑ OR p95 > 60 ms)? │ YES ▼ Disable feature flag immediately │ ▼ Both reads AND writes bypass cache (writeGen not mutated) │ ▼ Verify DB CPU / p95 return to pre-rollout baseline │ ▼ Investigate root cause │ ┌────┴────────────────────┐ │ re-enable? │ revert code? ▼ YES ▼ YES Enable flag → git revert + empty cache redeploy → back to stage 1 ``` --- ## Section 1: Architecture Review **Outcome:** No issues. Architecture is sound for stated scope. Single-process constraint enforced at startup. Auth before cache. Feature-flag rollback is immediate and clean. Coupling of cache to repository is intentional and documented. See diagrams above. ## Section 2: Error & Rescue Map **Outcome:** No issues. All new failure paths route through existing adapter bypass or typed error mapping. No catch-all handlers introduced. Registry above is complete. The adapter-failure bypass ("bypass until reinitialized") covers all cache op errors. ## Section 3: Security & Threat Model **Outcome:** No issues. No new attack surface (no new endpoints, inputs, or data paths). Keys encode tenantID + profileID; auth runs before cache check; cached results cannot bypass authorization. PII logging explicitly excluded ("without raw IDs"). No new secrets or dependencies. | Threat | Likelihood | Impact | Mitigated? | |--------|-----------|--------|------------| | Cached result bypasses auth | Low | High | Yes — auth runs before cache | | PII in logs | Low | Med | Yes — plan forbids raw IDs | | Stale data from race (S4-1) | Med | Med | Yes — amendment adds gen counter | ## Section 4: Data Flow & Interaction Edge Cases **Outcome:** One CRITICAL GAP found and resolved by amendment. **S4-1 (CRITICAL — resolved by amendment):** A read in-flight when a write commits can call `cache.set` with a stale snapshot after `cache.delete` fires, leaving future callers with stale data for up to 30 s (TTL). The plan's invariant ("TTL expiry is not a substitute for this rule") rules this out. Fix: generation counter (see Plan Amendment section). Auto-selected: generation counter remedy. No UI scope. Interaction edge cases table N/A. ## Section 5: Code Quality Review **Outcome:** No issues beyond S4-1 (addressed). Both functions have zero cyclomatic branches. Naming is clear and outcome-oriented. No DRY violations. No premature abstractions. The `writeGen` Map (added by amendment) is minimal and bounded by the hot-key population. `undefined`-as-miss vs SENTINEL-as-absent distinction is correct: `if (cached !== undefined) return cached` correctly returns SENTINEL values, letting callers interpret them per existing contracts. ## Section 6: Test Review **New data flows / codepaths:** - Cache hit → return cached value - Cache miss → single-flight → repo.read → gen check → cache.set - Cache miss, absent record → single-flight → repo.read → gen check → cache.set(SENTINEL) - Write → repo.write → gen increment → cache.delete - Cache adapter failure → bypass mode - Concurrent miss → single-flight coalesces **Planned test coverage (per plan):** | Codepath | Type | Plan covers? | |----------|------|-------------| | Hit/miss | Unit | Y | | Eviction/byte limits | Unit | Y | | TTL | Unit | Y | | Adapter-failure fallback | Unit | Y | | Successful-write invalidation | Unit | Y | | Failed-write preservation | Unit | Y | | Concurrent-miss coalescing | Unit | Y | | Stale write-back race (S4-1) | Unit | Added by amendment | **S6-1 (resolved by S4-1 amendment):** Test needed: dispatch read, let write commit (gen increment + cache.delete), then resolve read's DB response — verify `cache.set` is skipped and subsequent reads go to DB. Test ambition: - Friday 2am: concurrent read/write with promise-ordering control - Hostile QA: 100 reads in flight, write commits, all in-flight reads return stale — verify zero stale cache population - Chaos: adapter fails at 50% rollout, flag toggled mid-operation No flakiness risk from async timing (tests use controlled promise resolution). No LLM/prompt changes. No load test required (cache REDUCES DB load). ## Section 7: Performance Review **Outcome:** No issues. - N+1: N/A (no ORM traversal). - Memory: `writeGen` Map adds ~(key_bytes + 8 bytes) per hot key. At 900 keys: negligible (~tens of KB). Cache itself: 1000 entries, 16 MiB cap. - Slow paths: cold start (all misses, p95 = existing 120 ms), cache miss (same), flag-disabled (same as current). All within existing DB capacity per plan. - Connection pool: no new connections. Cache reduces DB connection demand. - No new queries, no new indexes needed. ## Section 8: Observability & Debuggability Review **Outcome:** No critical gaps. The plan specifies monitoring dimensions explicitly: hit/miss rate, eviction rate, cache bytes, fallback errors, DB CPU, read p95 — without raw IDs. Structured log format and alerting thresholds are implementation details left to the service owner (acceptable for a manual rollout with explicit acceptance targets). One note: log entries should include operation type (hit/miss/eviction/fallback) and key hash (not raw key) for post-incident reconstruction. This is an implementation note, not a plan gap. Acceptance targets double as implicit alert thresholds: DB CPU >50%, p95 >60 ms → disable flag. Service owner monitors manually; automation optional. ## Section 9: Deployment & Rollout Review **Outcome:** No critical risks. Feature flag provides immediate rollback. One clarification flagged. **S9-1 (note — no plan change required):** "enable for 10% of keys" is ambiguous. Clarify whether the feature flag applies per-key (10% of key population gets cached) or per-traffic (10% of requests routed through cache). Either works correctly; the service owner should document the chosen semantics before rollout. Auto-selected: add a note to the rollout steps to clarify semantics in the operational runbook. Rollback: immediate (disable flag → both reads and writes bypass cache). No DB migrations. Cold starts within existing capacity. Environment parity: staging validation should precede production rollout (standard practice, documented in deployment sequence diagram). ## Section 10: Long-Term Trajectory Review **Outcome:** No issues. Reversibility: **5/5** (feature flag bypass, clean re-enable with empty cache). Minimal technical debt introduced. The repository interface explicitly preserves a distributed-cache replacement path. New engineer in 12 months: the wrapper, generation counter, and rollout sequence are self-explanatory from the plan and the code. Phase 2 (distributed cache): the current design does not lock it out. Multi- process detection at startup ensures no silent coherence failures if that path opens. Documentation debt: plan is comprehensive; no gaps for a future maintainer. ## Section 11: Design & UX Review **SKIPPED** — no UI scope. The plan explicitly states "no UI, API, schema, pricing, or developer onboarding change." --- ## Implementation Tasks Synthesized from this review's findings. Each task derives from a specific finding above. - [ ] **T1 (P1, human: ~1h / CC: ~10min)** — LRU Cache Wrapper — Add generation counter to prevent stale write-back - Surfaced by: Section 4 (S4-1) — stale write-back race violates stated invariant - Files: cache wrapper module (wherever `readProfile`/`writeProfile` are implemented) - Verify: new test: dispatch read, commit write mid-flight, verify `cache.set` skipped; subsequent reads go to DB - [ ] **T2 (P1, human: ~30min / CC: ~5min)** — Test Suite — Add concurrent read+write race condition test - Surfaced by: Section 6 (S6-1, cross-ref S4-1) — no test for stale write-back race - Files: wrapper test file - Verify: test fails without generation counter, passes with it - [ ] **T3 (P3, human: ~15min / CC: ~2min)** — Rollout Runbook — Clarify feature-flag semantics (per-key vs per-traffic) - Surfaced by: Section 9 (S9-1) — "enable for 10% of keys" is ambiguous - Files: operational runbook / deployment notes - Verify: service owner confirms semantics match implementation before first rollout _No new tasks from Sections 1, 2, 3, 5, 7, 8, 10, 11._ --- ## Completion Summary ``` +====================================================================+ | MEGA PLAN REVIEW — COMPLETION SUMMARY | +====================================================================+ | Mode selected | HOLD SCOPE | | System Audit | Skipped (automated run per instructions) | | Step 0 | HOLD SCOPE confirmed; Approach A + gen fix | | Section 1 (Arch) | 0 issues found | | Section 2 (Errors) | 5 error paths mapped, 0 GAPS | | Section 3 (Security)| 0 issues found, 0 High severity | | Section 4 (Data/UX) | 1 edge case (S4-1 stale write-back), fixed | | Section 5 (Quality) | 0 issues (beyond S4-1) | | Section 6 (Tests) | Diagram produced, 1 gap (S6-1, per S4-1) | | Section 7 (Perf) | 0 issues found | | Section 8 (Observ) | 0 critical gaps (1 impl note) | | Section 9 (Deploy) | 1 clarification note (S9-1, non-blocking) | | Section 10 (Future) | Reversibility: 5/5, debt items: 0 | | Section 11 (Design) | SKIPPED (no UI scope) | +--------------------------------------------------------------------+ | NOT in scope | written (6 items) | | What already exists | written (7 items) | | Dream state delta | written | | Error/rescue registry| 5 methods, 0 CRITICAL GAPS | | Failure modes | 6 total, 0 CRITICAL GAPS (after amendment) | | TODOS.md updates | 0 items (no deferred gaps in HOLD SCOPE) | | Scope proposals | 0 proposed, 0 accepted (HOLD SCOPE) | | CEO plan | skipped (HOLD SCOPE) | | Outside voice | disabled (codex_reviews: disabled) | | Lake Score | 2/2 recommendations chose complete option | | Diagrams produced | 6 (arch, data flow, state machine, error, | | | deployment, rollback) | | Stale diagrams found | 0 (no prior diagrams in plan) | | Unresolved decisions | 0 | +====================================================================+ ``` Outside voice: Codex review skipped (codex_reviews: disabled per config.yaml). Re-enable: `gstack-config set codex_reviews enabled`. ## GSTACK REVIEW REPORT | Review | Trigger | Why | Runs | Status | Findings | |--------|---------|-----|------|--------|----------| | CEO Review | `/plan-ceo-review` | Scope & strategy | 1 | issues_found → amended | mode: HOLD_SCOPE, 1 critical gap (S4-1 resolved by gen-counter amendment), 0 unresolved | | Outside Review | disabled | Independent 2nd opinion | 0 | disabled | codex_reviews=disabled per config | | Eng Review | `/plan-eng-review` | Architecture & tests (required) | 0 | — | not yet run | | Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | N/A (no UI scope) | | DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | not yet run | **OUTSIDE COVERAGE:** disabled — `codex_reviews: disabled` set in config.yaml. No outside review run; no native fallback dispatched (disabled is a terminal branch, not a provider failure). Re-enable: `gstack-config set codex_reviews enabled`. **VERDICT:** CEO Review complete with amendment accepted (S4-1 generation counter). Eng review required before shipping. NO UNRESOLVED DECISIONS