docs: consolidate cache guidance and keep generated reports local

docs/shared-cache.md

@@ -0,0 +1,188 @@
+# Shared blob cache
+
+The optional shared filesystem cache stores immutable source content across projects.
+It uses Node's built-in SQLite module and requires Node 22.15 or newer. All API and
+streamer processes on one host must mount the same local cache directory. SQLite
+WAL is not a distributed lock service: do not put this directory on NFS or share
+it across hosts. S3 storage continues to use the existing storage implementation.
+
+Enable the integrated implementation with `SHARED_CACHE_ENABLED=true`. The default
+is false so a code deployment does not silently migrate production data. The other
+settings are independent rollback controls:
+
+- `SHARED_CACHE_PATH`: defaults to `.shared-cache-v1` inside `FOLDER`.
+- `SHARED_CACHE_BYTES`: source pool budget, default 10 GiB.
+- `SHARED_CACHE_RESULT_BYTES`: transformed output budget, default 512 MiB.
+- `SHARED_CACHE_MAX_FILE_BYTES`: maximum source admission size, default 32 MiB.
+- `SHARED_CACHE_FREE_BYTES`: free-device-space reserve, default 1 GiB.
+- `SHARED_CACHE_LOW_WATER`: eviction target fraction, default 0.9.
+- `SHARED_CACHE_GZIP`: store gzip when beneficial, default true.
+- `SHARED_CACHE_GZIP_MIN_SAVING`: minimum fractional saving, default 0.10.
+- `SHARED_CACHE_DECISIONS`: enable unchanged decisions and changed-output reuse,
+  default true.
+- `CACHE_MONITORING_ENABLED`: collect cache observations, default true. Monitoring
+  is independent of shared storage and also records legacy cache/inspection events.
+
+Logical user quotas remain based on repository file sizes. Shared-pool deduplication,
+compression and eviction do not change user quota usage.
+
+These are initial limits, not capacity recommendations from the production sample.
+Restart processes together when changing pool limits. Reserve headroom for SQLite,
+filesystem directories, existing caches and other application temporary files.
+The budget accounts for blob allocation with a conservative 4 KiB minimum plus
+in-flight reservations of three times the maximum object size and a small overhead.
+Keep each pool budget larger than this reservation. Admission can bypass the cache
+while other fills/readers occupy the budget. Oversized source objects are streamed
+without admission, still subject to the application's file-size limit.
+
+Git blobs are verified against the requested Git SHA before publication. Resolved
+LFS content is admitted only after verifying both the Git pointer and its SHA-256
+payload digest and size. Gzip is a representation of the same uncompressed content.
+Files are written privately, synced and atomically renamed before index publication.
+Fills and readers have renewable leases. Eviction skips readers, and stale fills
+are recovered before their reservations are released. LRU access timestamps are
+coarsened to 30 seconds. Orphan reconciliation runs at most hourly across processes.
+Alias and reuse-history indexes are bounded; decisions expire after seven days and
+are capped at 100,000 entries.
+
+Existing generation caches are read as a migration fallback and verified before
+promotion. They are not deleted during promotion, so rollback remains possible.
+The new pool budget does not include legacy copies: monitor total filesystem usage
+and retire legacy data only after validating rollout. Uploaded ZIP originals and
+other durable source material are untouched. Repository removal invalidates access
+and project mappings, while shared blobs remain subject to global eviction.
+
+The HTTP fast path applies to individual file responses backed by the shared source
+pool, including small text files. Decisions include content identity, path, project
+context, all transformation options, generation/revision, mask, hostname and engine
+version. Unchanged decisions serve the original representation. Changed output has
+its own bounded pool. Gzip negotiation honors quality values, uses `Vary`, and
+preserves MIME type. File validators are weak when shared delivery is enabled so
+identity and gzip representations can revalidate the same logical content. HEAD
+continues to use metadata only. Range requests receive a complete response; this
+implementation does not add random access into gzip files. Website rendering keeps
+its renderer cache; ZIP scans do not admit bulk entries into the source blob pool.
+
+The admin overview's Cache section reads `/api/admin/cache?minutes=60` behind the
+existing admin authorization. Windows are 15 minutes, one hour, 24 hours and seven
+days. SQLite telemetry batches at most 512 groups every five seconds and retains
+minute aggregates for 48 hours and hourly aggregates for 30 days. No paths, terms,
+user IDs or blob hashes appear in metric labels. The UI distinguishes unavailable
+telemetry, dropped events, disabled storage and empty windows. Rows separate file,
+website, ZIP, background and known-monitor traffic. Monitor classification is an
+operational heuristic, not a complete bot detector.
+
+Hit counts and byte hit rates measure source lookups; misses classify first observed
+keys, known misses and eviction refetches within bounded seven-day history. Reuse
+intervals are consecutive blob accesses, not distinct people or browser visits.
+Eviction time measures idle age; codec and inspection timings are elapsed work time,
+not CPU samples. Cached decisions are separate from actual inspections. Minute and
+hour boundaries make the displayed windows approximate.
+
+Run the reproducible acceptance workload with:
+
+```sh
+mkdir -p benchmark-results/cache
+node scripts/cache-workload.js benchmark-results/cache/workload.json
+npx mocha test/shared-blob-cache.test.js test/shared-cache-http.test.js
+```
+
+The JSON report asserts cross-project source reuse, unchanged-decision reuse,
+changed-output reuse, gzip delivery and budget enforcement. Its compression ratio
+comes from synthetic text and must not be presented as a production projection.
+Production rollout should compare request/byte hit rates, eviction refetches, p95
+latency, upstream errors and free disk over a representative week, with large files,
+ZIP work and known monitors visible separately.
+
+### Benefits and costs
+
+| Choice | Benefit | Cost or limit |
+| --- | --- | --- |
+| Global content identity | Reuses identical files across paths and projects; renames do not duplicate content | Requires verified hashes and authorization on every request; uses digest-prefix directories to avoid one huge directory |
+| Byte budget with approximate LRU | Predictable blob footprint and automatic reclamation | Evicted files incur upstream fetches; active readers can temporarily prevent admission |
+| Selective gzip | Reduces payload storage and can serve compressed bytes directly | First-fill compression costs work; identity consumers and uncached anonymization require decoding |
+| Rule-aware decisions and results | Repeated unchanged reads skip regex inspection; changed output is reusable | More metadata and a separate output budget; settings changes produce new decisions |
+| SQLite coordination and telemetry | Cross-process coordination without another service; admin can measure actual reuse and rewrite work | Local-filesystem scope; short synchronous transactions; metadata/headroom are outside blob budgets |
+
+### Rollout sequence
+
+1. Deploy with shared storage disabled and collect legacy hits, inspections and codec
+   timings. Keep known-monitor traffic separate from file readers.
+2. Enable the shared cache on the single-host filesystem deployment with both byte
+   budgets configured. Keep legacy copies for rollback.
+3. Compare a representative week of byte/request hits, rewrite decisions, eviction
+   refetches, request latency, upstream errors and whole-device free space. Adjust
+   admission size and budgets from those observations.
+4. Retire legacy cache copies only after the rollback window. Uploaded originals
+   remain durable and are never candidates for shared-pool eviction.
+
+The implementation and acceptance workload are local changes; this work does not
+enable or migrate production storage.
+
+### Production observations used for this design
+
+The read-only audit on 2026-11-02 found 425,312 source files across 31,522 projects,
+containing 144.32 GB of payload. Deduplication within projects could remove 1.87 GB;
+global deduplication could remove 2.93 GB total (about 2.03%), including another
+1.06 GB across projects. One file could not be read. These are decimal GB and
+payload savings, not a compression estimate or the expected new working-set size.
+
+The preceding 24-hour access sample contained 29,980 successful file-body responses,
+including 9,249 repeat responses (30.9%). URL repetitions are not the same as blob
+cache hits. A separate approximately two-hour metric window recorded 2,302 source
+hits out of 3,600 lookups (63.9%). This supports retaining a bounded source cache;
+it does not establish the optimal byte budget.
+
+In the uniform anonymization replay, 937 resident files were measured: 764 text,
+173 binary, and 21 with changed output. Thirteen resolved files exceeded the 4 MiB
+sample limit, and missing files/mappings were excluded. The sample's mean gzip-6
+work was 1.46 ms per file and mean decompression was 0.24 ms; these replay timings
+are not live-request latency or a forecast for large files. Continuous telemetry is
+needed to replace these samples with representative workload measurements.
+
+### Metadata durability and accounting
+
+Reader leases, access history, decisions and telemetry use NORMAL WAL commits.
+Reservations, publication intent and final blob publication use FULL commits;
+payloads are synced before rename. Recent disposable metadata may be lost after
+system failure. Recovery retains reservations until orphan files are removed or
+accounted for. SIGKILL recovery is tested; physical power failure is not simulated.
+
+Admin statistics use transactional counters and an indexed oldest-access lookup.
+History and decisions are capped at 100,000 rows each, aliases at 200,000. Expired
+history/decision cleanup removes at most 256 rows per pass. Access history has
+30-second resolution. Stop all cache processes before upgrading an existing index
+and restart them on the same version; do not mix old and new writers.
+
+### Performance and validation
+
+Local synthetic measurements on Node 24.14.1, dated 2026-11-02, compared the initial
+FULL-commit implementation with optimized bookkeeping. Warm serial changed-text
+medians fell from 44.9 to 7.3 ms at 16 KiB, 45.9 to 9.0 ms at 256 KiB, and 60.5 to
+19.0 ms at 2 MiB. At 100,000 synthetic blob rows, admin stats p95 fell from 78.7 to
+0.50 ms and decision-write p95 from 45.5 to 1.11 ms.
+
+The optimized runs verified 2,500 HTTP responses. The 4 MiB eviction probe peaked
+at 3,997,696 allocated bytes. Its oversized sequential working set had no hits:
+a bounded LRU cache does not guarantee useful reuse. Measurements used loopback
+HTTP, warm OS caches and a shared host; they exclude production traffic, TLS,
+authorization overhead and multi-process throughput. They are not capacity estimates.
+
+On the PR checkout based on main at d0b217a, the production build, scoped lint and
+199 cache, HTTP, admin access, HEAD/revalidation and Vue UI tests passed. Coverage
+includes process crash recovery, concurrent fills, active readers, LFS verification,
+and public submodule origins without parent credentials. Admin layout was also
+checked at 1280 px and 390 px during implementation.
+
+Generate fresh reports locally; generated output is ignored by Git:
+
+```sh
+npx tsc
+node scripts/cache-index-performance.js
+CACHE_BENCH_COMPILED=1 node scripts/cache-performance.js
+CACHE_BENCH_COMPILED=1 node scripts/cache-performance.js benchmark-results/cache/repeat.json --shared
+```
+
+Default reports go to `benchmark-results/cache/`. Scripts also accept an explicit
+output path. Keep raw measurements with the evaluation that produced them rather
+than committing generated reports to the documentation directory.