Merge pull request #915 from tdurieux/feat/shared-blob-cache

perf: deduplicate cached files across projects and track cache reuse
This commit is contained in:
Thomas Durieux authored and GitHub committed 2026-11-03 09:23:07 +01:00
commit 88cd1b4cdb
31 files changed
+2107 -42

No files matched your search

+3
View File
@@ -124,3 +124,6 @@ temp/
# GitHub App signing key
/secrets/github-app.pem
# Generated local benchmark reports
/benchmark-results/
+5
View File
@@ -125,3 +125,8 @@ an unavailable file as a successful cache fill. Persistent 404s still require
checking the repository's source snapshot. Application rate limits remain in
force; 429 replies now include `rate_limited`, `resetAt`, and `Retry-After` so
clients can wait for the actual store reset.
## Shared blob cache
See [Shared blob cache](shared-cache.md) for configuration, storage and quota
accounting, admin metrics, production observations, rollout and benchmark commands.
+206
View File
@@ -0,0 +1,206 @@
# Shared blob cache
The optional shared filesystem cache stores immutable source content across projects.
It uses Node's built-in SQLite module and requires Node 22.15 or newer. All API and
streamer processes on one host must mount the same local cache directory. SQLite
WAL is not a distributed lock service: do not put this directory on NFS or share
it across hosts. S3 storage continues to use the existing storage implementation.
Enable the integrated implementation with `SHARED_CACHE_ENABLED=true`. The default
is false so a code deployment does not silently migrate production data. The other
settings are independent rollback controls:
- `SHARED_CACHE_PATH`: defaults to `.shared-cache-v1` inside `FOLDER`.
- `SHARED_CACHE_BYTES`: source pool budget, default 10 GiB.
- `SHARED_CACHE_RESULT_BYTES`: transformed output budget, default 512 MiB.
- `SHARED_CACHE_MAX_FILE_BYTES`: maximum source admission size, default 32 MiB.
- `SHARED_CACHE_FREE_BYTES`: free-device-space reserve, default 1 GiB.
- `SHARED_CACHE_LOW_WATER`: eviction target fraction, default 0.9.
- `SHARED_CACHE_GZIP`: store gzip when beneficial, default true.
- `SHARED_CACHE_GZIP_MIN_SAVING`: minimum fractional saving, default 0.10.
- `SHARED_CACHE_DECISIONS`: enable unchanged decisions and changed-output reuse,
default true.
- `CACHE_MONITORING_ENABLED`: collect cache observations, default true. Monitoring
is independent of shared storage and also records legacy cache/inspection events.
Logical user quotas remain based on repository file sizes. Shared-pool deduplication,
compression and eviction do not change user quota usage.
These are initial limits, not capacity recommendations from the production sample.
Restart processes together when changing pool limits. Reserve headroom for SQLite,
filesystem directories, existing caches and other application temporary files.
The budget accounts for blob allocation with a conservative 4 KiB minimum plus
in-flight reservations of three times the maximum object size and a small overhead.
A shared SQLite ledger reserves physical headroom across source and result pools
before producers write bytes. Its check uses a fresh free-space sample under a
cross-process lock and includes all outstanding reservations. Reservations remain
until cleanup or crash recovery completes. This is conservative while fills have
already written some bytes, and cannot reserve space against unrelated applications.
Keep each pool budget larger than this reservation. Admission can bypass the cache
while other fills/readers occupy the budget. Oversized source objects are streamed
without admission, still subject to the application's file-size limit.
Git blobs are verified against the requested Git SHA before publication. Resolved
LFS content is admitted only after verifying both the Git pointer and its SHA-256
payload digest and size. Gzip is a representation of the same uncompressed content.
Files are written privately, synced and atomically renamed before index publication.
Fills and readers have renewable leases. Eviction skips readers, and stale fills
are recovered before their reservations are released. LRU access timestamps are
coarsened to 30 seconds. Orphan reconciliation runs at most hourly across processes.
Alias and reuse-history indexes are bounded; decisions expire after seven days and
are capped at 100,000 entries.
Existing generation caches are read as a migration fallback and verified before
promotion. They are not deleted during promotion, so rollback remains possible.
The new pool budget does not include legacy copies: monitor total filesystem usage
and retire legacy data only after validating rollout. Uploaded ZIP originals and
other durable source material are untouched. Repository removal invalidates access
and project mappings, while shared blobs remain subject to global eviction.
Transient SQLite write contention bypasses caching instead of failing file requests.
Reader heartbeats retry contention within their last confirmed lease window and
terminate only if renewal cannot succeed before expiry or the lease is removed.
Admission-bypass streams retain the minimum expected-size check.
Publication installs a reader lease before releasing its transaction, including
duplicate blobs. Decisions retain sniffed text classification for consistent MIME
headers on repeat reads; binary responses omit the range-denial header.
Shared result production and delivery use the same repository lifecycle checks as
the legacy transformed cache. Client disconnects cancel the active transformer and
its worker signal, releasing spool reservations. Completed evictions are committed
even when pinned readers still prevent a new admission.
The HTTP fast path applies to individual file responses backed by the shared source
pool, including small text files. Decisions include content identity, path, project
context, all transformation options, generation/revision, mask, hostname and engine
version. Unchanged decisions serve the original representation. Changed output has
its own bounded pool. Gzip negotiation honors quality values, uses `Vary`, and
preserves MIME type. File validators are weak when shared delivery is enabled so
identity and gzip representations can revalidate the same logical content. HEAD
continues to use metadata only. Range requests receive a complete response; this
implementation does not add random access into gzip files. Website rendering keeps
its renderer cache; ZIP scans do not admit bulk entries into the source blob pool.
The admin overview's Cache section reads `/api/admin/cache?minutes=60` behind the
existing admin authorization. Windows are 15 minutes, one hour, 24 hours and seven
days. SQLite telemetry batches at most 512 groups every five seconds and retains
minute aggregates for 48 hours and hourly aggregates for 30 days. No paths, terms,
user IDs or blob hashes appear in metric labels. The UI distinguishes unavailable
telemetry, dropped events, disabled storage and empty windows. Rows separate file,
website, ZIP, background and known-monitor traffic. Monitor classification is an
operational heuristic, not a complete bot detector.
Hit counts and byte hit rates measure source lookups; misses classify first observed
keys, known misses and eviction refetches within bounded seven-day history. Reuse
intervals are consecutive blob accesses, not distinct people or browser visits.
Eviction time measures idle age; codec and inspection timings are elapsed work time,
not CPU samples. Cached decisions are separate from actual inspections. Minute and
hour boundaries make the displayed windows approximate.
Run the reproducible acceptance workload with:
```sh
mkdir -p benchmark-results/cache
node scripts/cache-workload.js benchmark-results/cache/workload.json
npx mocha test/shared-blob-cache.test.js test/shared-cache-http.test.js
```
The JSON report asserts cross-project source reuse, unchanged-decision reuse,
changed-output reuse, gzip delivery and budget enforcement. Its compression ratio
comes from synthetic text and must not be presented as a production projection.
Production rollout should compare request/byte hit rates, eviction refetches, p95
latency, upstream errors and free disk over a representative week, with large files,
ZIP work and known monitors visible separately.
### Benefits and costs
| Choice | Benefit | Cost or limit |
| --- | --- | --- |
| Global content identity | Reuses identical files across paths and projects; renames do not duplicate content | Requires verified hashes and authorization on every request; uses digest-prefix directories to avoid one huge directory |
| Byte budget with approximate LRU | Predictable blob footprint and automatic reclamation | Evicted files incur upstream fetches; active readers can temporarily prevent admission |
| Selective gzip | Reduces payload storage and can serve compressed bytes directly | First-fill compression costs work; identity consumers and uncached anonymization require decoding |
| Rule-aware decisions and results | Repeated unchanged reads skip regex inspection; changed output is reusable | More metadata and a separate output budget; settings changes produce new decisions |
| SQLite coordination and telemetry | Cross-process coordination without another service; admin can measure actual reuse and rewrite work | Local-filesystem scope; short synchronous transactions; metadata/headroom are outside blob budgets |
### Rollout sequence
1. Deploy with shared storage disabled and collect legacy hits, inspections and codec
timings. Keep known-monitor traffic separate from file readers.
2. Enable the shared cache on the single-host filesystem deployment with both byte
budgets configured. Keep legacy copies for rollback.
3. Compare a representative week of byte/request hits, rewrite decisions, eviction
refetches, request latency, upstream errors and whole-device free space. Adjust
admission size and budgets from those observations.
4. Retire legacy cache copies only after the rollback window. Uploaded originals
remain durable and are never candidates for shared-pool eviction.
The implementation and acceptance workload are local changes; this work does not
enable or migrate production storage.
### Production observations used for this design
The read-only audit on 2026-11-02 found 425,312 source files across 31,522 projects,
containing 144.32 GB of payload. Deduplication within projects could remove 1.87 GB;
global deduplication could remove 2.93 GB total (about 2.03%), including another
1.06 GB across projects. One file could not be read. These are decimal GB and
payload savings, not a compression estimate or the expected new working-set size.
The preceding 24-hour access sample contained 29,980 successful file-body responses,
including 9,249 repeat responses (30.9%). URL repetitions are not the same as blob
cache hits. A separate approximately two-hour metric window recorded 2,302 source
hits out of 3,600 lookups (63.9%). This supports retaining a bounded source cache;
it does not establish the optimal byte budget.
In the uniform anonymization replay, 937 resident files were measured: 764 text,
173 binary, and 21 with changed output. Thirteen resolved files exceeded the 4 MiB
sample limit, and missing files/mappings were excluded. The sample's mean gzip-6
work was 1.46 ms per file and mean decompression was 0.24 ms; these replay timings
are not live-request latency or a forecast for large files. Continuous telemetry is
needed to replace these samples with representative workload measurements.
### Metadata durability and accounting
Reader leases, access history, decisions and telemetry use NORMAL WAL commits.
Reservations, publication intent and final blob publication use FULL commits;
payloads are synced before rename. Recent disposable metadata may be lost after
system failure. Recovery retains reservations until orphan files are removed or
accounted for. SIGKILL recovery is tested; physical power failure is not simulated.
Admin statistics use transactional counters and an indexed oldest-access lookup.
History and decisions are capped at 100,000 rows each, aliases at 200,000. Expired
history/decision cleanup removes at most 256 rows per pass. Access history has
30-second resolution. Stop all cache processes before upgrading an existing index
and restart them on the same version; do not mix old and new writers.
### Performance and validation
Local synthetic measurements on Node 24.14.1, dated 2026-11-02, compared the initial
FULL-commit implementation with optimized bookkeeping. Warm serial changed-text
medians fell from 44.9 to 7.3 ms at 16 KiB, 45.9 to 9.0 ms at 256 KiB, and 60.5 to
19.0 ms at 2 MiB. At 100,000 synthetic blob rows, admin stats p95 fell from 78.7 to
0.50 ms and decision-write p95 from 45.5 to 1.11 ms.
The optimized runs verified 2,500 HTTP responses. The 4 MiB eviction probe peaked
at 3,997,696 allocated bytes. Its oversized sequential working set had no hits:
a bounded LRU cache does not guarantee useful reuse. Measurements used loopback
HTTP, warm OS caches and a shared host; they exclude production traffic, TLS,
authorization overhead and multi-process throughput. They are not capacity estimates.
On the PR checkout based on main at d0b217a, the production build, scoped lint and
199 cache, HTTP, admin access, HEAD/revalidation and Vue UI tests passed. Coverage
includes process crash recovery, concurrent fills, active readers, LFS verification,
and public submodule origins without parent credentials. Admin layout was also
checked at 1280 px and 390 px during implementation.
Generate fresh reports locally; generated output is ignored by Git:
```sh
npx tsc
node scripts/cache-index-performance.js
CACHE_BENCH_COMPILED=1 node scripts/cache-performance.js
CACHE_BENCH_COMPILED=1 node scripts/cache-performance.js benchmark-results/cache/repeat.json --shared
```
Default reports go to `benchmark-results/cache/`. Scripts also accept an explicit
output path. Keep raw measurements with the evaluation that produced them rather
than committing generated reports to the documentation directory.
+2 -2
View File
@@ -1,8 +1,8 @@
{
"core.min.js": "core.4a77c8ac41.min.js",
"vendor.min.js": "vendor.906b828b05.min.js",
"vendor.min.js": "vendor.208b4c16f4.min.js",
"mermaid.min.js": "mermaid.a7356d5ed1.min.js",
"all.min.css": "all.7b7927afcc.min.css",
"all.min.css": "all.83492de3a4.min.css",
"markdown.min.js": "markdown.6c8dfb7b1d.min.js",
"pdf.min.js": "pdf.4d9cc6ff49.min.js",
"notebook.min.js": "notebook.c3144c18ae.min.js",
+1 -1
View File
@@ -6,4 +6,4 @@
*/:root{--blue:#007bff;--indigo:#6610f2;--purple:#6f42c1;--pink:#e83e8c;--red:#dc3545;--orange:#fd7e14;--yellow:#ffc107;--green:#28a745;--teal:#20c997;--cyan:#17a2b8;--white:#fff;--gray:#6c757d;--gray-dark:#343a40;--primary:#007bff;--secondary:#6c757d;--success:#28a745;--info:#17a2b8;--warning:#ffc107;--danger:#dc3545;--light:#f8f9fa;--dark:#343a40;--breakpoint-xs:0;--breakpoint-sm:576px;--breakpoint-md:768px;--breakpoint-lg:992px;--breakpoint-xl:1200px;--font-family-sans-serif:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";--font-family-monospace:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace}*,::after,::before{box-sizing:border-box}html{font-family:sans-serif;line-height:1.15;-webkit-text-size-adjust:100%;-webkit-tap-highlight-color:transparent}article,aside,figcaption,figure,footer,header,hgroup,main,nav,section{display:block}body{margin:0;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:1rem;font-weight:400;line-height:1.5;color:#212529;text-align:left;background-color:#fff}[tabindex="-1"]:focus:not(:focus-visible){outline:0!important}hr{box-sizing:content-box;height:0;overflow:visible}h1,h2,h3,h4,h5,h6{margin-top:0;margin-bottom:.5rem}p{margin-top:0;margin-bottom:1rem}abbr[data-original-title],abbr[title]{text-decoration:underline;-webkit-text-decoration:underline dotted;text-decoration:underline dotted;cursor:help;border-bottom:0;-webkit-text-decoration-skip-ink:none;text-decoration-skip-ink:none}address{margin-bottom:1rem;font-style:normal;line-height:inherit}dl,ol,ul{margin-top:0;margin-bottom:1rem}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem}b,strong{font-weight:bolder}small{font-size:80%}sub,sup{position:relative;font-size:75%;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}a{color:#007bff;text-decoration:none;background-color:transparent}a:hover{color:#0056b3;text-decoration:underline}a:not([href]){color:inherit;text-decoration:none}a:not([href]):hover{color:inherit;text-decoration:none}code,kbd,pre,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em}pre{margin-top:0;margin-bottom:1rem;overflow:auto}figure{margin:0 0 1rem}img{vertical-align:middle;border-style:none}svg{overflow:hidden;vertical-align:middle}table{border-collapse:collapse}caption{padding-top:.75rem;padding-bottom:.75rem;color:#6c757d;text-align:left;caption-side:bottom}th{text-align:inherit}label{display:inline-block;margin-bottom:.5rem}button{border-radius:0}button:focus{outline:1px dotted;outline:5px auto -webkit-focus-ring-color}button,input,optgroup,select,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,input{overflow:visible}button,select{text-transform:none}select{word-wrap:normal}[type=button],[type=reset],[type=submit],button{-webkit-appearance:button}[type=button]:not(:disabled),[type=reset]:not(:disabled),[type=submit]:not(:disabled),button:not(:disabled){cursor:pointer}[type=button]::-moz-focus-inner,[type=reset]::-moz-focus-inner,[type=submit]::-moz-focus-inner,button::-moz-focus-inner{padding:0;border-style:none}input[type=checkbox],input[type=radio]{box-sizing:border-box;padding:0}input[type=date],input[type=datetime-local],input[type=month],input[type=time]{-webkit-appearance:listbox}textarea{overflow:auto;resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{display:block;width:100%;max-width:100%;padding:0;margin-bottom:.5rem;font-size:1.5rem;line-height:inherit;color:inherit;white-space:normal}progress{vertical-align:baseline}[type=number]::-webkit-inner-spin-button,[type=number]::-webkit-outer-spin-button{height:auto}[type=search]{outline-offset:-2px;-webkit-appearance:none}[type=search]::-webkit-search-decoration{-webkit-appearance:none}::-webkit-file-upload-button{font:inherit;-webkit-appearance:button}output{display:inline-block}summary{display:list-item;cursor:pointer}template{display:none}[hidden]{display:none!important}.h1,.h2,.h3,.h4,.h5,.h6,h1,h2,h3,h4,h5,h6{margin-bottom:.5rem;font-weight:500;line-height:1.2}.h1,h1{font-size:2.5rem}.h2,h2{font-size:2rem}.h3,h3{font-size:1.75rem}.h4,h4{font-size:1.5rem}.h5,h5{font-size:1.25rem}.h6,h6{font-size:1rem}.lead{font-size:1.25rem;font-weight:300}.display-1{font-size:6rem;font-weight:300;line-height:1.2}.display-2{font-size:5.5rem;font-weight:300;line-height:1.2}.display-3{font-size:4.5rem;font-weight:300;line-height:1.2}.display-4{font-size:3.5rem;font-weight:300;line-height:1.2}hr{margin-top:1rem;margin-bottom:1rem;border:0;border-top:1px solid rgba(0,0,0,.1)}.small,small{font-size:80%;font-weight:400}.mark,mark{padding:.2em;background-color:#fcf8e3}.list-unsLine truncated
* Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com
* License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License)
*/.fa,.fab,.fad,.fal,.far,.fas{-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased;display:inline-block;font-style:normal;font-variant:normal;text-rendering:auto;line-height:1}.fa-lg{font-size:1.33333em;line-height:.75em;vertical-align:-.0667em}.fa-xs{font-size:.75em}.fa-sm{font-size:.875em}.fa-1x{font-size:1em}.fa-2x{font-size:2em}.fa-3x{font-size:3em}.fa-4x{font-size:4em}.fa-5x{font-size:5em}.fa-6x{font-size:6em}.fa-7x{font-size:7em}.fa-8x{font-size:8em}.fa-9x{font-size:9em}.fa-10x{font-size:10em}.fa-fw{text-align:center;width:1.25em}.fa-ul{list-style-type:none;margin-left:2.5em;padding-left:0}.fa-ul>li{position:relative}.fa-li{left:-2em;position:absolute;text-align:center;width:2em;line-height:inherit}.fa-border{border:solid .08em #eee;border-radius:.1em;padding:.2em .25em .15em}.fa-pull-left{float:left}.fa-pull-right{float:right}.fa.fa-pull-left,.fab.fa-pull-left,.fal.fa-pull-left,.far.fa-pull-left,.fas.fa-pull-left{margin-right:.3em}.fa.fa-pull-right,.fab.fa-pull-right,.fal.fa-pull-right,.far.fa-pull-right,.fas.fa-pull-right{margin-left:.3em}.fa-spin{-webkit-animation:fa-spin 2s infinite linear;animation:fa-spin 2s infinite linear}.fa-pulse{-webkit-animation:fa-spin 1s infinite steps(8);animation:fa-spin 1s infinite steps(8)}@-webkit-keyframes fa-spin{0%{-webkit-transform:rotate(0);transform:rotate(0)}100%{-webkit-transform:rotate(360deg);transform:rotate(360deg)}}@keyframes fa-spin{0%{-webkit-transform:rotate(0);transform:rotate(0)}100%{-webkit-transform:rotate(360deg);transform:rotate(360deg)}}.fa-rotate-90{-webkit-transform:rotate(90deg);transform:rotate(90deg)}.fa-rotate-180{-webkit-transform:rotate(180deg);transform:rotate(180deg)}.fa-rotate-270{-webkit-transform:rotate(270deg);transform:rotate(270deg)}.fa-flip-horizontal{-webkit-transform:scale(-1,1);transform:scale(-1,1)}.fa-flip-vertical{-webkit-transform:scale(1,-1);transform:scale(1,-1)}.fa-flip-both,.fa-flip-horizontal.fa-flip-vertical{-webkit-transform:scale(-1,-1);transform:scale(-1,-1)}:root .fa-flip-both,:root .fa-flip-horizontal,:root .fa-flip-vertical,:root .fa-rotate-180,:root .fa-rotate-270,:root .fa-rotate-90{-webkit-filter:none;filter:none}.fa-stack{display:inline-block;height:2em;line-height:2em;position:relative;vertical-align:middle;width:2.5em}.fa-stack-1x,.fa-stack-2x{left:0;position:absolute;text-align:center;width:100%}.fa-stack-1x{line-height:inherit}.fa-stack-2x{font-size:2em}.fa-inverse{color:#fff}.fa-500px:before{content:"\f26e"}.fa-accessible-icon:before{content:"\f368"}.fa-accusoft:before{content:"\f369"}.fa-acquisitions-incorporated:before{content:"\f6af"}.fa-ad:before{content:"\f641"}.fa-address-book:before{content:"\f2b9"}.fa-address-card:before{content:"\f2bb"}.fa-adjust:before{content:"\f042"}.fa-adn:before{content:"\f170"}.fa-adversal:before{content:"\f36a"}.fa-affiliatetheme:before{content:"\f36b"}.fa-air-freshener:before{content:"\f5d0"}.fa-airbnb:before{content:"\f834"}.fa-algolia:before{content:"\f36c"}.fa-align-center:before{content:"\f037"}.fa-align-justify:before{content:"\f039"}.fa-align-left:before{content:"\f036"}.fa-align-right:before{content:"\f038"}.fa-alipay:before{content:"\f642"}.fa-allergies:before{content:"\f461"}.fa-amazon:before{content:"\f270"}.fa-amazon-pay:before{content:"\f42c"}.fa-ambulance:before{content:"\f0f9"}.fa-american-sign-language-interpreting:before{content:"\f2a3"}.fa-amilia:before{content:"\f36d"}.fa-anchor:before{content:"\f13d"}.fa-android:before{content:"\f17b"}.fa-angellist:before{content:"\f209"}.fa-angle-double-down:before{content:"\f103"}.fa-angle-double-left:before{content:"\f100"}.fa-angle-double-right:before{content:"\f101"}.fa-angle-double-up:before{content:"\f102"}.fa-angle-down:before{content:"\f107"}.fa-angle-left:before{content:"\f104"}.fa-angle-right:before{content:"\f105"}.fa-angle-up:before{content:"\f106"}.fa-angry:before{content:"\f556"}.fa-angrycreative:before{content:"\f36e"}.fa-angular:before{content:"\f420"}.fa-ankh:before{content:"\f644"}.fa-app-store:before{content:"\f36f"}.fa-app-store-ios:before{content:"\f370"}.fa-apper:before{content:"\f371"}.fa-apple:before{content:"\f179"}.fa-apple-alt:before{content:"\f5d1"}.fa-apple-pay:before{content:"\f415"}.fa-archive:before{content:"\f187"}.fa-archway:before{content:"\f557"}.fa-arrow-alt-circle-down:before{content:"\f358"}.fa-arrow-alt-circle-left:before{content:"\f359"}.fa-arrow-alt-circle-right:before{content:"\f35a"}.fa-arrow-alt-circle-up:before{content:"\f35b"}.fa-arrow-circle-down:before{content:"\f0ab"}.fa-arrow-circle-left:before{content:"\f0a8"}.fa-arrow-circle-right:before{content:"\f0a9"}.fa-arrow-circle-up:before{content:"\f0aa"}.fa-arrow-down:before{content:"\f063"}.fa-arrow-left:before{content:"\f060"}.fa-arrow-right:before{content:"\f061"}.fa-arrow-up:before{content:"\f062"}.fa-arrows-alt:before{content:"\f0b2"}.fa-arrows-alt-h:before{content:"\f337"}.fa-arrows-alt-v:before{content:"\f338"}.fa-artstation:before{content:"\f77a"}.fa-assistive-listening-systems:before{contentLine truncated
*/.fa,.fab,.fad,.fal,.far,.fas{-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased;display:inline-block;font-style:normal;font-variant:normal;text-rendering:auto;line-height:1}.fa-lg{font-size:1.33333em;line-height:.75em;vertical-align:-.0667em}.fa-xs{font-size:.75em}.fa-sm{font-size:.875em}.fa-1x{font-size:1em}.fa-2x{font-size:2em}.fa-3x{font-size:3em}.fa-4x{font-size:4em}.fa-5x{font-size:5em}.fa-6x{font-size:6em}.fa-7x{font-size:7em}.fa-8x{font-size:8em}.fa-9x{font-size:9em}.fa-10x{font-size:10em}.fa-fw{text-align:center;width:1.25em}.fa-ul{list-style-type:none;margin-left:2.5em;padding-left:0}.fa-ul>li{position:relative}.fa-li{left:-2em;position:absolute;text-align:center;width:2em;line-height:inherit}.fa-border{border:solid .08em #eee;border-radius:.1em;padding:.2em .25em .15em}.fa-pull-left{float:left}.fa-pull-right{float:right}.fa.fa-pull-left,.fab.fa-pull-left,.fal.fa-pull-left,.far.fa-pull-left,.fas.fa-pull-left{margin-right:.3em}.fa.fa-pull-right,.fab.fa-pull-right,.fal.fa-pull-right,.far.fa-pull-right,.fas.fa-pull-right{margin-left:.3em}.fa-spin{-webkit-animation:fa-spin 2s infinite linear;animation:fa-spin 2s infinite linear}.fa-pulse{-webkit-animation:fa-spin 1s infinite steps(8);animation:fa-spin 1s infinite steps(8)}@-webkit-keyframes fa-spin{0%{-webkit-transform:rotate(0);transform:rotate(0)}100%{-webkit-transform:rotate(360deg);transform:rotate(360deg)}}@keyframes fa-spin{0%{-webkit-transform:rotate(0);transform:rotate(0)}100%{-webkit-transform:rotate(360deg);transform:rotate(360deg)}}.fa-rotate-90{-webkit-transform:rotate(90deg);transform:rotate(90deg)}.fa-rotate-180{-webkit-transform:rotate(180deg);transform:rotate(180deg)}.fa-rotate-270{-webkit-transform:rotate(270deg);transform:rotate(270deg)}.fa-flip-horizontal{-webkit-transform:scale(-1,1);transform:scale(-1,1)}.fa-flip-vertical{-webkit-transform:scale(1,-1);transform:scale(1,-1)}.fa-flip-both,.fa-flip-horizontal.fa-flip-vertical{-webkit-transform:scale(-1,-1);transform:scale(-1,-1)}:root .fa-flip-both,:root .fa-flip-horizontal,:root .fa-flip-vertical,:root .fa-rotate-180,:root .fa-rotate-270,:root .fa-rotate-90{-webkit-filter:none;filter:none}.fa-stack{display:inline-block;height:2em;line-height:2em;position:relative;vertical-align:middle;width:2.5em}.fa-stack-1x,.fa-stack-2x{left:0;position:absolute;text-align:center;width:100%}.fa-stack-1x{line-height:inherit}.fa-stack-2x{font-size:2em}.fa-inverse{color:#fff}.fa-500px:before{content:"\f26e"}.fa-accessible-icon:before{content:"\f368"}.fa-accusoft:before{content:"\f369"}.fa-acquisitions-incorporated:before{content:"\f6af"}.fa-ad:before{content:"\f641"}.fa-address-book:before{content:"\f2b9"}.fa-address-card:before{content:"\f2bb"}.fa-adjust:before{content:"\f042"}.fa-adn:before{content:"\f170"}.fa-adversal:before{content:"\f36a"}.fa-affiliatetheme:before{content:"\f36b"}.fa-air-freshener:before{content:"\f5d0"}.fa-airbnb:before{content:"\f834"}.fa-algolia:before{content:"\f36c"}.fa-align-center:before{content:"\f037"}.fa-align-justify:before{content:"\f039"}.fa-align-left:before{content:"\f036"}.fa-align-right:before{content:"\f038"}.fa-alipay:before{content:"\f642"}.fa-allergies:before{content:"\f461"}.fa-amazon:before{content:"\f270"}.fa-amazon-pay:before{content:"\f42c"}.fa-ambulance:before{content:"\f0f9"}.fa-american-sign-language-interpreting:before{content:"\f2a3"}.fa-amilia:before{content:"\f36d"}.fa-anchor:before{content:"\f13d"}.fa-android:before{content:"\f17b"}.fa-angellist:before{content:"\f209"}.fa-angle-double-down:before{content:"\f103"}.fa-angle-double-left:before{content:"\f100"}.fa-angle-double-right:before{content:"\f101"}.fa-angle-double-up:before{content:"\f102"}.fa-angle-down:before{content:"\f107"}.fa-angle-left:before{content:"\f104"}.fa-angle-right:before{content:"\f105"}.fa-angle-up:before{content:"\f106"}.fa-angry:before{content:"\f556"}.fa-angrycreative:before{content:"\f36e"}.fa-angular:before{content:"\f420"}.fa-ankh:before{content:"\f644"}.fa-app-store:before{content:"\f36f"}.fa-app-store-ios:before{content:"\f370"}.fa-apper:before{content:"\f371"}.fa-apple:before{content:"\f179"}.fa-apple-alt:before{content:"\f5d1"}.fa-apple-pay:before{content:"\f415"}.fa-archive:before{content:"\f187"}.fa-archway:before{content:"\f557"}.fa-arrow-alt-circle-down:before{content:"\f358"}.fa-arrow-alt-circle-left:before{content:"\f359"}.fa-arrow-alt-circle-right:before{content:"\f35a"}.fa-arrow-alt-circle-up:before{content:"\f35b"}.fa-arrow-circle-down:before{content:"\f0ab"}.fa-arrow-circle-left:before{content:"\f0a8"}.fa-arrow-circle-right:before{content:"\f0a9"}.fa-arrow-circle-up:before{content:"\f0aa"}.fa-arrow-down:before{content:"\f063"}.fa-arrow-left:before{content:"\f060"}.fa-arrow-right:before{content:"\f061"}.fa-arrow-up:before{content:"\f062"}.fa-arrows-alt:before{content:"\f0b2"}.fa-arrows-alt-h:before{content:"\f337"}.fa-arrows-alt-v:before{content:"\f338"}.fa-artstation:before{content:"\f77a"}.fa-assistive-listening-systems:before{contentLine truncated
+9
View File
@@ -7953,3 +7953,12 @@ body.dark-mode .markdown-body .hljs-comment { color: #b1bac4; }
body:not(.dark-mode) .markdown-body .gh-dark-mode-only,
body.dark-mode .markdown-body .gh-light-mode-only { display: none; }
@media (max-width: 600px) {
.ov-cache .ov-performance-window {
display: grid;
grid-template-columns: auto minmax(0, 1fr);
width: 100%;
}
.ov-cache .ov-performance-window select { width: 100%; min-width: 0; }
}
+51
View File
@@ -276,6 +276,57 @@
</div>
<section class="ov-panel-card ov-cache" aria-labelledby="cache-heading">
<div class="ov-panel-head ov-performance-head">
<div><h2 class="ov-panel-title" id="cache-heading">Cache</h2>
<p class="ov-performance-note">Shared storage, repeated reads and anonymization work · refreshes every 30 seconds</p></div>
<div class="ov-performance-window">
<label for="cache-window">Time range</label>
<select id="cache-window" class="form-control form-control-sm" v-model.number="cacheMinutes" @change="loadCache()">
<option :value="15">Last 15 minutes</option><option :value="60">Last hour</option>
<option :value="1440">Last 24 hours</option><option :value="10080">Last 7 days</option>
</select>
<label for="cache-traffic">Traffic</label>
<select id="cache-traffic" class="form-control form-control-sm" v-model="cacheTraffic">
<option value="all">All traffic</option><option value="file">File reads</option><option value="website">Website rendering</option>
<option value="zip">ZIP downloads</option><option value="monitor">Known monitors</option><option value="background">Background work</option>
</select>
</div>
</div>
<div v-if="cacheError" class="ov-performance-message ov-performance-warning" role="alert">
<span>{{ cacheError }}</span><button type="button" class="btn btn-sm" @click="loadCache()">Retry</button>
</div>
<p v-if="!cacheReport &amp;&amp; !cacheError" class="ov-performance-empty" role="status">Loading cache monitoring…</p>
<template v-if="cacheReport?.available">
<p v-if="!cacheReport.enabled" class="ov-performance-note">Shared blob storage is disabled. Inspection and legacy-cache activity are still measured.</p>
<p v-if="!cacheReport.filesystemSupported" class="ov-performance-note">Shared blobs require a local filesystem. The configured storage continues to use its existing cache.</p>
<p v-if="cacheReport.dropped" class="ov-performance-message ov-performance-warning" role="status">{{ cacheReport.dropped }} telemetry events were dropped. Totals may be incomplete.</p>
<div class="ov-metrics-scroll" tabindex="0" role="region" aria-label="Cache storage budgets" v-if="cacheReport.source">
<table class="ov-metrics-table">
<caption class="sr-only">Current shared-cache storage and byte budgets</caption>
<thead><tr><th scope="col">Pool</th><th scope="col">Allocated / budget</th><th scope="col">Reserved for fills</th><th scope="col">Original bytes</th><th scope="col">Blobs</th><th scope="col">Active readers</th></tr></thead>
<tbody><tr v-for="pool in [{name: 'Source blobs', data: cacheReport.source}, {name: 'Transformed output', data: cacheReport.results}]" :key="pool.name" v-show="pool.data">
<th scope="row">{{ pool.name }}</th><td>{{ humanBytes(pool.data?.allocatedBytes) }} / {{ humanBytes(pool.data?.budgetBytes) }}</td>
<td>{{ humanBytes(pool.data?.reservedBytes) }}</td><td>{{ humanBytes(pool.data?.originalBytes) }}</td>
<td>{{ pool.data?.blobs }}</td><td>{{ pool.data?.activeReaders }}</td></tr></tbody>
</table>
</div>
<p class="ov-performance-note" v-if="cacheReport.source">Source request hit rate: {{ cacheRate('source', 'count') }} · Source byte hit rate: {{ cacheRate('source', 'bytes') }} · Decision hit rate: {{ cacheRate('decision', 'count') }}</p>
<p class="ov-performance-note">Reuse counts measure consecutive reads of the same blob within each interval. Codec timings are elapsed stream time, not CPU time; eviction times measure how long a blob was idle. Known monitors are identified by request metadata; other automation may remain in file reads.</p>
<div v-if="cacheRows().length" class="ov-metrics-scroll" tabindex="0" role="region" aria-label="Cache events and processing durations">
<table class="ov-metrics-table">
<caption class="sr-only">Cache events for the selected time range and traffic</caption>
<thead><tr><th scope="col">Event</th><th scope="col">Traffic</th><th scope="col">Count</th><th scope="col">Bytes</th><th scope="col">Mean measured time</th><th scope="col">p95 bound</th></tr></thead>
<tbody><tr v-for="row in cacheRows()" :key="row.name + ':' + row.traffic">
<th scope="row">{{ cacheEventName(row.name) }}</th><td>{{ row.traffic }}</td><td>{{ row.count }}</td><td>{{ humanBytes(row.bytes) }}</td>
<td>{{ row.ms ? (row.ms / row.count).toFixed(2) + " ms" : "Not timed" }}</td><td>{{ cacheP95(row) }}</td></tr></tbody>
</table>
</div>
<p v-else class="ov-performance-empty" role="status">No cache events in this time range for the selected traffic.</p>
<p class="ov-performance-note">Updated {{ sampleAge(cacheReport) }} seconds ago. Minute history is retained for 48 hours; hourly history for 30 days.</p>
</template>
</section>
<section class="ov-panel-card ov-performance" aria-labelledby="performance-heading">
<div class="ov-panel-head ov-performance-head">
<div>
+43
View File
@@ -1737,6 +1737,48 @@ export const overviewAdminController = function (state, http, location, interval
state.performance = null;
state.performanceMinutes = 15;
state.performanceTab = "routes";
state.cacheReport = null;
state.cacheError = null;
state.cacheMinutes = 60;
state.cacheTraffic = "all";
var cacheLoading = false;
state.loadCache = function () {
if (cacheLoading) return;
cacheLoading = true;
var minutes = state.cacheMinutes;
http.get("/api/admin/cache", { params: { minutes: minutes } }).then(function (r) {
if (minutes !== state.cacheMinutes) return;
state.cacheReport = r.data;
state.cacheError = r.data.available ? null : (r.data.reason || "Cache monitoring is unavailable.");
}, function () {
state.cacheError = "Could not refresh cache monitoring. Displayed values may be stale.";
}).finally(function () {
cacheLoading = false;
if (minutes !== state.cacheMinutes) state.loadCache();
});
};
state.cacheRows = function () {
return (state.cacheReport?.rows || []).filter(function (row) {
return state.cacheTraffic === "all" || row.traffic === state.cacheTraffic;
});
};
state.cacheRate = function (prefix, field) {
var rows = state.cacheRows();
var sum = function (name) { return rows.filter(function (r) { return r.name === name; }).reduce(function (n, r) { return n + r[field]; }, 0); };
var hits = sum(prefix + "_hit"), misses = sum(prefix + "_miss");
return hits + misses ? (100 * hits / (hits + misses)).toFixed(1) + "%" : "No samples";
};
state.cacheEventName = function (name) { return name.replaceAll("_", " "); };
state.cacheP95 = function (row) {
var count = 0, bounds = state.cacheReport?.latencyBucketsMs || [];
if (!row.count) return "No samples";
if (!row.ms) return "Not timed";
for (var i = 0; i <= bounds.length; i++) {
count += row["b" + i] || 0;
if (count >= Math.ceil(row.count * 0.95)) return i < bounds.length ? "≤ " + bounds[i] + " ms" : "> 10 s";
}
return "No samples";
};
state.performanceError = null;
// 18400 ms reads better as 18.4 s once a value passes one second.
state.formatMs = function (ms) {
@@ -1891,6 +1933,7 @@ export const overviewAdminController = function (state, http, location, interval
function load() {
state.loadPerformance();
state.loadCache();
http.get("/api/admin/overview").then(function (r) {
r.data.history = computeDailyHistory(r.data.history);
r.data.daily = {
+23 -23
View File
@@ -1,30 +1,30 @@
"use strict";(()=>{var kd=Object.defineProperty;var Ed=(e,t)=>{for(var n in t)kd(e,n,{get:t[n],enumerable:!0})};var pe=["abap","abc","actionscript","ada","alda","apache_conf","apex","applescript","aql","asciidoc","asl","assembly_x86","autohotkey","batchfile","c9search","c_cpp","cirru","clojure","cobol","coffee","coldfusion","crystal","csharp","csound_document","csound_orchestra","csound_score","csp","css","curly","d","dart","diff","django","dockerfile","dot","drools","edifact","eiffel","ejs","elixir","elm","erlang","forth","fortran","fsharp","fsl","ftl","gcode","gherkin","gitignore","glsl","gobstones","golang","graphqlschema","groovy","haml","handlebars","haskell","haskell_cabal","haxe","hjson","html","html_elixir","html_ruby","ini","io","jack","jade","java","javascript","json","json5","jsoniq","jsp","jssm","jsx","julia","kotlin","latex","less","liquid","lisp","livescript","logiql","logtalk","lsl","lua","luapage","lucene","makefile","markdown","mask","matlab","maze","mediawiki","mel","mixal","mushcode","mysql","nginx","nim","nix","nsis","nunjucks","objectivec","ocaml","pascal","perl","perl6","pgsql","php","php_laravel_blade","pig","plain_text","powershell","praat","prisma","prolog","properties","protobuf","puppet","python","qml","r","razor","rdoc","red","redshift","rhtml","rst","ruby","rust","sass","scad","scala","scheme","scss","sh","sjs","slim","smarty","snippets","soy_template","space","sparql","sql","sqlserver","stylus","svg","swift","tcl","terraform","tex","text","textile","toml","tsx","turtle","twig","typescript","vala","vbscript","velocity","verilog","vhdl","visualforce","wollok","xml","xquery","yaml","zeek"];var fe={markdown:"/script/markdown.6c8dfb7b1d.min.js",notebook:"/script/notebook.c3144c18ae.min.js",org:"/script/org.f61f262915.min.js",editor:"/script/editor.4f142d3adc.min.js",pdf:"/script/pdf.4d9cc6ff49.min.js"};function br(e){let t=e.location.hash.slice(1);if(e.history.replaceState(null,"","/review-link"),t.length>2048)return null;let n=new URLSearchParams(t);if([...n.keys()].length!==3||[...n.keys()].some(a=>!["clientId","intentId","token"].includes(a)))return null;let r=n.get("clientId"),s=n.get("intentId"),i=n.get("token");return!/^[a-f0-9]{32}$/.test(r||"")||!/^[a-f0-9]{32}$/.test(s||"")||!/^[a-f0-9]{64}$/.test(i||"")?null:{contract:"4open.artifacts/1",clientId:r,intentId:s,token:i}}function wr(e,t){let{http:n,promises:r,window:s}=t,i=t.takeReviewHandoff(),a=i?{clientId:i.clientId,intentId:i.intentId}:null,u,m,k,y,E,h=!0,C,g=r.defer(),b=()=>[...s.crypto.getRandomValues(new Uint8Array(16))].map(M=>M.toString(16).padStart(2,"0")).join("");Object.assign(e,{repositoryId:"",handoffEnabled:!1,busy:!1,preview:null,acceptAccess:!1,acceptRetention:!1,uncertain:!1,saved:!1,expired:!1,error:i?"":"Open a new artifact link from your submission. This link is missing or invalid."});let S=()=>h&&e.user?.username===E;function R(){h=!1,i=u=m=void 0,g.resolve(),s.clearTimeout(C)}e.on("dispose",R),e.on("routeLeave",R),e.watch(()=>e.user?.username,M=>{!E&&M?E=M:E&&M!==E&&(R(),e.preview=null,e.repositoryId="",e.busy=!1,e.saved=!1,e.handoffEnabled=!1,e.error="Your account changed. Open a new artifact link from your submission.")});let T=()=>({timeout:g.promise,headers:{"Content-Type":"application/json","X-Review-CSRF":m}}),q=M=>M?.status===404?"Artifact linking is not available on this server.":M?.status===401?"Sign in, then open a new artifact link from your submission.":M?.status===403?"This account cannot approve this repository or this link.":"The request could not be completed. You can retry from this page.";e.loadPreview=async()=>{if(!h||!i||e.busy||e.uncertain||e.saved||!e.user?.username)return;E=e.user.username;let M=e.repositoryId.trim();if(!/^[A-Za-z0-9_-]{3,128}$/.test(M)){e.error="Enter the anonymous repository ID from its URL.";return}e.busy=!0,e.error="",e.preview=null,u=void 0,e.acceptAccess=e.acceptRetention=!1,e.expired=!1,s.clearTimeout(C),y||(y=b());try{let V=(await n.get("/api/review-consent/csrf",{timeout:g.promise})).data;if(!S())return;if(m=V.csrf,e.handoffEnabled=V.handoffEnabled===!0,!/^[a-f0-9]{64}$/.test(m||""))throw new Error("invalid_csrf");let z=(await n.post("/api/review-consent/preview",{repositoryId:M,intent:{...i,requestId:y}},T())).data;if(!S())return;if(!z||typeof z.ticket!="string"||!z.ticket||z.ticket.length>8192||typeof z.repositoryName!="string"||!["restricted-review","anonymous-link"].includes(z.policy?.access)||!Number.isSafeInteger(z.policy.version)||!Number.isFinite(Date.parse(z.policy.retainUntil))||!Number.isFinite(Date.parse(z.expiresAt))||Date.parse(z.expiresAt)<=Date.now()||Date.parse(z.expiresAt)>Date.now()+6e5)throw new Error("invalid_preview");u=z.ticket,k=b(),e.preview={repositoryName:z.repositoryName,policy:{...z.policy},expiresAt:z.expiresAt},C=s.setTimeout(()=>{h&&(e.expired=!0)},Math.max(0,Date.parse(z.expiresAt)-Date.now()))}catch(V){S()&&(e.error=q(V))}finally{S()&&(e.busy=!1)}},e.confirmConsent=async()=>{if(!(!S()||e.busy||e.saved||!u||!e.acceptAccesLine truncated
`))),e[Ft]=Ro(s);let i=r||s.props&&s.props.type==="number";to(e,t?"change":"input",a=>{a.target.composing||e[Ft](Js(e.value,n,i))}),(n||i)&&to(e,"change",()=>{e.value=Js(e.value,n,i)}),t||(to(e,"compositionstart",Sc),to(e,"compositionend",aa),to(e,"change",aa))},mounted(e,{value:t,modifiers:{trim:n,number:r}}){let s=t??"",i=e[Jn];delete e[Jn],i!==void 0&&(e.type==="text"||e.type==="textarea")&&e.value!==i?e[Ft](Js(e.value,n,r)):e.value=s},beforeUpdate(e,{value:t,oldValue:n,modifiers:{lazy:r,trim:s,number:i}},a){if(e[Ft]=Ro(a),e.composing)return;let u=(i||e.type==="number")&&!/^0\d/.test(e.value)?ko(e.value):e.value,m=t??"";if(u===m)return;let k=e.getRootNode();(k instanceof Document||k instanceof ShadowRoot)&&k.activeElement===e&&e.type!=="range"&&(r&&t===n||s&&e.value.trim()===m)||(e.value=m)}},Xs={deep:!0,created(e,t,n){e[Ft]=Ro(n),to(e,"change",()=>{let r=e._modelValue,s=mn(e),i=e.checked,a=e[Ft];if(me(r)){let u=Cn(r,s),m=u!==-1;if(i&&!m)a(r.concat(s));else if(!i&&m){let k=[...r];k.splice(u,1),a(k)}}else if(Et(r)){let u=new Set(r);i?u.add(s):u.delete(s),a(u)}else a(fa(e,i))})},mounted:la,beforeUpdate(e,t,n){e[Ft]=Ro(n),la(e,t,n)}};function la(e,{value:t,oldValue:n},r){e._modelValue=t;let s;if(me(t))s=Cn(t,r.props.value)>-1;else if(Et(t))s=t.has(r.props.value);else{if(t===n)return;s=Ct(t,fa(e,!0))}e.checked!==s&&(e.checked=s)}var Xn={deep:!0,created(e,{value:t,modifiers:{number:n}},r){e._modelValue=t,to(e,"change",()=>{let s=Array.prototype.filter.call(e.options,m=>m.selected).map(m=>n?ko(mn(m)):mn(m)),i=e.multiple,a=i?Et(e._modelValue)?new Set(s):s:s[0],u=e._pendingValue=[i,i?me(a)?s.slice():s:a];try{e[Ft](a)}finally{ot(()=>{e._pendingValue===u&&(e._pendingValue=void 0)})}}),e[Ft]=Ro(r)},mounted(e,{value:t}){da(e,t)},beforeUpdate(e,{value:t},n){e._modelValue=t,e[Ft]=Ro(n)},updated(e,{value:t}){let n=e._pendingValue;e._pendingValue=void 0,(!n||n[0]!==e.multiple||!Nc(t,n[1],n[0]))&&da(e,t)}};function Nc(e,t,n){if(!n)return Ct(e,t);if(me(e))return Ct(e,t);if(Et(e)){if(e.size!==t.length)return!1;for(let r of t)if(!e.has(r))return!1;return!0}return!1}function da(e,t){let n=e.multiple,r=me(t);if(!(n&&!r&&!Et(t))){for(let s=0,i=e.options.length;s<i;s++){let a=e.options[s],u=mn(a);if(n)if(r){let m=typeof u;m==="string"||m==="number"?a.selected=t.some(k=>String(k)===String(u)):a.selected=Cn(t,u)>-1}else a.selected=t.has(u);else if(Ct(mn(a),t)){e.selectedIndex!==s&&(e.selectedIndex=s);return}}!n&&e.selectedIndex!==-1&&(e.selectedIndex=-1)}}function mn(e){return"_value"in e?e._value:e.value}function fa(e,t){let n=t?"_trueValue":"_falseValue";return n in e?e[n]:t}var Dc=["ctrl","shift","alt","meta"],Rc={stop:e=>e.stopPropagation(),prevent:e=>e.preventDefault(),self:e=>e.target!==e.currentTarget,ctrl:e=>!e.ctrlKey,shift:e=>!e.shiftKey,alt:e=>!e.altKey,meta:e=>!e.metaKey,left:e=>"button"in e&&e.button!==0,middle:e=>"button"in e&&e.button!==1,right:e=>"button"in e&&e.button!==2,exact:(e,t)=>Dc.some(n=>e[`${n}Key`]&&!t.includes(n))},ie=(e,t)=>{if(!e)return e;let n=e._withMods||(e._withMods={}),r=t.join(".");return n[r]||(n[r]=((s,...i)=>{for(let a=0;a<t.length;a++){let u=Rc[t[a]];if(u&&u(s,t))return}return e(s,...i)}))},Tc={esc:"escape",space:" ",up:"arrow-up",left:"arrow-left",right:"arrow-right",down:"arrow-down",delete:"backspace"},To=(e,t)=>{let n=e._withKeys||(e._withKeys={}),r=t.join(".");return n[r]||(n[r]=(s=>{if(!("key"in s))return;let i=At(s.key);if(t.some(a=>a===i||Tc[a]===i))return e(s)}))},Ac=je({patchProp:Ec},ic),ua;function Oc(){return ua||(ua=Bi(Ac))}var ma=((...e)=>{let t=Oc().createApp(...e),{mount:n}=t;return t.mount=r=>{let s=Pc(r);if(!s)return;let i=t._component;!ge(i)&&!i.render&&!i.template&&(i.template=s.innerHTML),s.nodeType===1&&(s.textContent="");let a=n(s,!1,Ic(s));return s instanceof Element&&(s.removeAttribute("v-cloak"),s.setAttribute("data-v-app","")),a},t});function Ic(e){if(e instanceof SVGElement)return"svg";if(typeof MathMLElement=="function"&&e instanceof MathMLElement)return"mathml"}function Pc(e){return Le(e)?document.querySelector(e):e}var mo=typeof document<"u";function va(e){return typeof e=="object"||"displayName"in e||"props"in e||"__vccOpts"in e}function Vc(e){return e.__esModule||e[Symbol.toStringTag]==="Module"||e.default&&va(e.default)}var Te=Object.assign;function Zn(e,t){let n={};for(let r in t){let s=t[r];n[r]=wt(s)?s.map(e):e(s)}return n}var Oo=()=>{},wt=Array.isArray;function er(e,t){let n={};for(let r in e)n[r]=r in t?t[r]:e[r];return n}var ya=/#/g,qc=/&/g,Mc=/\//g,Uc=/=/g,$c=/\?/g,ga=/\+/g,Lc=/%5B/g,Fc=/%5D/g,ba=/%5E/g,zc=/%60/g,wa=/%7B/g,Hc=/%7C/g,ka=/%7D/g,Bc=/%20/g;function tr(e){return e==null?"":encodeURI(""+e).replace(Hc,"|").replace(Lc,"[").replace(Fc,"]")}function Ea(e){return tr(e).replace(wa,"{").replace(ka,"}").replace(ba,"^")}function Zs(e){return tr(e).replace(ga,"%2B").replace(Bc,"+").replace(ya,"%23").replace(qc,"%26").replace(zc,"`").replace(wa,"{").replace(ka,"}").replace(ba,"^")}function jc(e){return Zs(e).replace(Uc,"%3D")}function Gc(e){return tLine truncated
"use strict";(()=>{var kd=Object.defineProperty;var Ed=(e,t)=>{for(var n in t)kd(e,n,{get:t[n],enumerable:!0})};var pe=["abap","abc","actionscript","ada","alda","apache_conf","apex","applescript","aql","asciidoc","asl","assembly_x86","autohotkey","batchfile","c9search","c_cpp","cirru","clojure","cobol","coffee","coldfusion","crystal","csharp","csound_document","csound_orchestra","csound_score","csp","css","curly","d","dart","diff","django","dockerfile","dot","drools","edifact","eiffel","ejs","elixir","elm","erlang","forth","fortran","fsharp","fsl","ftl","gcode","gherkin","gitignore","glsl","gobstones","golang","graphqlschema","groovy","haml","handlebars","haskell","haskell_cabal","haxe","hjson","html","html_elixir","html_ruby","ini","io","jack","jade","java","javascript","json","json5","jsoniq","jsp","jssm","jsx","julia","kotlin","latex","less","liquid","lisp","livescript","logiql","logtalk","lsl","lua","luapage","lucene","makefile","markdown","mask","matlab","maze","mediawiki","mel","mixal","mushcode","mysql","nginx","nim","nix","nsis","nunjucks","objectivec","ocaml","pascal","perl","perl6","pgsql","php","php_laravel_blade","pig","plain_text","powershell","praat","prisma","prolog","properties","protobuf","puppet","python","qml","r","razor","rdoc","red","redshift","rhtml","rst","ruby","rust","sass","scad","scala","scheme","scss","sh","sjs","slim","smarty","snippets","soy_template","space","sparql","sql","sqlserver","stylus","svg","swift","tcl","terraform","tex","text","textile","toml","tsx","turtle","twig","typescript","vala","vbscript","velocity","verilog","vhdl","visualforce","wollok","xml","xquery","yaml","zeek"];var fe={markdown:"/script/markdown.6c8dfb7b1d.min.js",org:"/script/org.f61f262915.min.js",notebook:"/script/notebook.c3144c18ae.min.js",editor:"/script/editor.4f142d3adc.min.js",pdf:"/script/pdf.4d9cc6ff49.min.js"};function br(e){let t=e.location.hash.slice(1);if(e.history.replaceState(null,"","/review-link"),t.length>2048)return null;let n=new URLSearchParams(t);if([...n.keys()].length!==3||[...n.keys()].some(a=>!["clientId","intentId","token"].includes(a)))return null;let r=n.get("clientId"),s=n.get("intentId"),i=n.get("token");return!/^[a-f0-9]{32}$/.test(r||"")||!/^[a-f0-9]{32}$/.test(s||"")||!/^[a-f0-9]{64}$/.test(i||"")?null:{contract:"4open.artifacts/1",clientId:r,intentId:s,token:i}}function wr(e,t){let{http:n,promises:r,window:s}=t,i=t.takeReviewHandoff(),a=i?{clientId:i.clientId,intentId:i.intentId}:null,u,m,w,g,k,h=!0,E,S=r.defer(),v=()=>[...s.crypto.getRandomValues(new Uint8Array(16))].map(M=>M.toString(16).padStart(2,"0")).join("");Object.assign(e,{repositoryId:"",handoffEnabled:!1,busy:!1,preview:null,acceptAccess:!1,acceptRetention:!1,uncertain:!1,saved:!1,expired:!1,error:i?"":"Open a new artifact link from your submission. This link is missing or invalid."});let _=()=>h&&e.user?.username===k;function N(){h=!1,i=u=m=void 0,S.resolve(),s.clearTimeout(E)}e.on("dispose",N),e.on("routeLeave",N),e.watch(()=>e.user?.username,M=>{!k&&M?k=M:k&&M!==k&&(N(),e.preview=null,e.repositoryId="",e.busy=!1,e.saved=!1,e.handoffEnabled=!1,e.error="Your account changed. Open a new artifact link from your submission.")});let T=()=>({timeout:S.promise,headers:{"Content-Type":"application/json","X-Review-CSRF":m}}),V=M=>M?.status===404?"Artifact linking is not available on this server.":M?.status===401?"Sign in, then open a new artifact link from your submission.":M?.status===403?"This account cannot approve this repository or this link.":"The request could not be completed. You can retry from this page.";e.loadPreview=async()=>{if(!h||!i||e.busy||e.uncertain||e.saved||!e.user?.username)return;k=e.user.username;let M=e.repositoryId.trim();if(!/^[A-Za-z0-9_-]{3,128}$/.test(M)){e.error="Enter the anonymous repository ID from its URL.";return}e.busy=!0,e.error="",e.preview=null,u=void 0,e.acceptAccess=e.acceptRetention=!1,e.expired=!1,s.clearTimeout(E),g||(g=v());try{let q=(await n.get("/api/review-consent/csrf",{timeout:S.promise})).data;if(!_())return;if(m=q.csrf,e.handoffEnabled=q.handoffEnabled===!0,!/^[a-f0-9]{64}$/.test(m||""))throw new Error("invalid_csrf");let z=(await n.post("/api/review-consent/preview",{repositoryId:M,intent:{...i,requestId:g}},T())).data;if(!_())return;if(!z||typeof z.ticket!="string"||!z.ticket||z.ticket.length>8192||typeof z.repositoryName!="string"||!["restricted-review","anonymous-link"].includes(z.policy?.access)||!Number.isSafeInteger(z.policy.version)||!Number.isFinite(Date.parse(z.policy.retainUntil))||!Number.isFinite(Date.parse(z.expiresAt))||Date.parse(z.expiresAt)<=Date.now()||Date.parse(z.expiresAt)>Date.now()+6e5)throw new Error("invalid_preview");u=z.ticket,w=v(),e.preview={repositoryName:z.repositoryName,policy:{...z.policy},expiresAt:z.expiresAt},E=s.setTimeout(()=>{h&&(e.expired=!0)},Math.max(0,Date.parse(z.expiresAt)-Date.now()))}catch(q){_()&&(e.error=V(q))}finally{_()&&(e.busy=!1)}},e.confirmConsent=async()=>{if(!(!_()||e.busy||e.saved||!u||!e.acceptAccesLine truncated
`))),e[Ft]=Do(s);let i=r||s.props&&s.props.type==="number";to(e,t?"change":"input",a=>{a.target.composing||e[Ft](Js(e.value,n,i))}),(n||i)&&to(e,"change",()=>{e.value=Js(e.value,n,i)}),t||(to(e,"compositionstart",Sc),to(e,"compositionend",aa),to(e,"change",aa))},mounted(e,{value:t,modifiers:{trim:n,number:r}}){let s=t??"",i=e[Qn];delete e[Qn],i!==void 0&&(e.type==="text"||e.type==="textarea")&&e.value!==i?e[Ft](Js(e.value,n,r)):e.value=s},beforeUpdate(e,{value:t,oldValue:n,modifiers:{lazy:r,trim:s,number:i}},a){if(e[Ft]=Do(a),e.composing)return;let u=(i||e.type==="number")&&!/^0\d/.test(e.value)?ko(e.value):e.value,m=t??"";if(u===m)return;let w=e.getRootNode();(w instanceof Document||w instanceof ShadowRoot)&&w.activeElement===e&&e.type!=="range"&&(r&&t===n||s&&e.value.trim()===m)||(e.value=m)}},Xs={deep:!0,created(e,t,n){e[Ft]=Do(n),to(e,"change",()=>{let r=e._modelValue,s=hn(e),i=e.checked,a=e[Ft];if(me(r)){let u=_n(r,s),m=u!==-1;if(i&&!m)a(r.concat(s));else if(!i&&m){let w=[...r];w.splice(u,1),a(w)}}else if(Et(r)){let u=new Set(r);i?u.add(s):u.delete(s),a(u)}else a(fa(e,i))})},mounted:la,beforeUpdate(e,t,n){e[Ft]=Do(n),la(e,t,n)}};function la(e,{value:t,oldValue:n},r){e._modelValue=t;let s;if(me(t))s=_n(t,r.props.value)>-1;else if(Et(t))s=t.has(r.props.value);else{if(t===n)return;s=Ct(t,fa(e,!0))}e.checked!==s&&(e.checked=s)}var To={deep:!0,created(e,{value:t,modifiers:{number:n}},r){e._modelValue=t,to(e,"change",()=>{let s=Array.prototype.filter.call(e.options,m=>m.selected).map(m=>n?ko(hn(m)):hn(m)),i=e.multiple,a=i?Et(e._modelValue)?new Set(s):s:s[0],u=e._pendingValue=[i,i?me(a)?s.slice():s:a];try{e[Ft](a)}finally{ot(()=>{e._pendingValue===u&&(e._pendingValue=void 0)})}}),e[Ft]=Do(r)},mounted(e,{value:t}){da(e,t)},beforeUpdate(e,{value:t},n){e._modelValue=t,e[Ft]=Do(n)},updated(e,{value:t}){let n=e._pendingValue;e._pendingValue=void 0,(!n||n[0]!==e.multiple||!Nc(t,n[1],n[0]))&&da(e,t)}};function Nc(e,t,n){if(!n)return Ct(e,t);if(me(e))return Ct(e,t);if(Et(e)){if(e.size!==t.length)return!1;for(let r of t)if(!e.has(r))return!1;return!0}return!1}function da(e,t){let n=e.multiple,r=me(t);if(!(n&&!r&&!Et(t))){for(let s=0,i=e.options.length;s<i;s++){let a=e.options[s],u=hn(a);if(n)if(r){let m=typeof u;m==="string"||m==="number"?a.selected=t.some(w=>String(w)===String(u)):a.selected=_n(t,u)>-1}else a.selected=t.has(u);else if(Ct(hn(a),t)){e.selectedIndex!==s&&(e.selectedIndex=s);return}}!n&&e.selectedIndex!==-1&&(e.selectedIndex=-1)}}function hn(e){return"_value"in e?e._value:e.value}function fa(e,t){let n=t?"_trueValue":"_falseValue";return n in e?e[n]:t}var Rc=["ctrl","shift","alt","meta"],Dc={stop:e=>e.stopPropagation(),prevent:e=>e.preventDefault(),self:e=>e.target!==e.currentTarget,ctrl:e=>!e.ctrlKey,shift:e=>!e.shiftKey,alt:e=>!e.altKey,meta:e=>!e.metaKey,left:e=>"button"in e&&e.button!==0,middle:e=>"button"in e&&e.button!==1,right:e=>"button"in e&&e.button!==2,exact:(e,t)=>Rc.some(n=>e[`${n}Key`]&&!t.includes(n))},ie=(e,t)=>{if(!e)return e;let n=e._withMods||(e._withMods={}),r=t.join(".");return n[r]||(n[r]=((s,...i)=>{for(let a=0;a<t.length;a++){let u=Dc[t[a]];if(u&&u(s,t))return}return e(s,...i)}))},Tc={esc:"escape",space:" ",up:"arrow-up",left:"arrow-left",right:"arrow-right",down:"arrow-down",delete:"backspace"},Ao=(e,t)=>{let n=e._withKeys||(e._withKeys={}),r=t.join(".");return n[r]||(n[r]=(s=>{if(!("key"in s))return;let i=At(s.key);if(t.some(a=>a===i||Tc[a]===i))return e(s)}))},Ac=je({patchProp:Ec},ic),ua;function Oc(){return ua||(ua=Bi(Ac))}var ma=((...e)=>{let t=Oc().createApp(...e),{mount:n}=t;return t.mount=r=>{let s=Pc(r);if(!s)return;let i=t._component;!ge(i)&&!i.render&&!i.template&&(i.template=s.innerHTML),s.nodeType===1&&(s.textContent="");let a=n(s,!1,Ic(s));return s instanceof Element&&(s.removeAttribute("v-cloak"),s.setAttribute("data-v-app","")),a},t});function Ic(e){if(e instanceof SVGElement)return"svg";if(typeof MathMLElement=="function"&&e instanceof MathMLElement)return"mathml"}function Pc(e){return Le(e)?document.querySelector(e):e}var mo=typeof document<"u";function va(e){return typeof e=="object"||"displayName"in e||"props"in e||"__vccOpts"in e}function Vc(e){return e.__esModule||e[Symbol.toStringTag]==="Module"||e.default&&va(e.default)}var Te=Object.assign;function Zn(e,t){let n={};for(let r in t){let s=t[r];n[r]=wt(s)?s.map(e):e(s)}return n}var Io=()=>{},wt=Array.isArray;function er(e,t){let n={};for(let r in e)n[r]=r in t?t[r]:e[r];return n}var ya=/#/g,qc=/&/g,Mc=/\//g,Uc=/=/g,$c=/\?/g,ga=/\+/g,Lc=/%5B/g,Fc=/%5D/g,ba=/%5E/g,zc=/%60/g,wa=/%7B/g,Hc=/%7C/g,ka=/%7D/g,Bc=/%20/g;function tr(e){return e==null?"":encodeURI(""+e).replace(Hc,"|").replace(Lc,"[").replace(Fc,"]")}function Ea(e){return tr(e).replace(wa,"{").replace(ka,"}").replace(ba,"^")}function Zs(e){return tr(e).replace(ga,"%2B").replace(Bc,"+").replace(ya,"%23").replace(qc,"%26").replace(zc,"`").replace(wa,"{").replace(ka,"}").replace(ba,"^")}function jc(e){return Zs(e).replace(Uc,"%3D")}function Gc(e){return tLine truncated
${JSON.stringify(e)}${t?`
while being at
`+JSON.stringify(t):""}`},[Be.NAVIGATION_GUARD_REDIRECT]({from:e,to:t}){return`Redirected from "${e.fullPath}" to "${ep(t)}" via a navigation guard.`},[Be.NAVIGATION_ABORTED]({from:e,to:t}){return`Navigation aborted from "${e.fullPath}" to "${t.fullPath}" via a navigation guard.`},[Be.NAVIGATION_CANCELLED]({from:e,to:t}){return`Navigation cancelled from "${e.fullPath}" to "${t.fullPath}" with a new navigation.`},[Be.NAVIGATION_DUPLICATED]({from:e,to:t}){return`Avoided redundant navigation to current location: "${e.fullPath}".`}};function fo(e,t){return Te(new Error,{type:e,[Oa]:!0},t)}function Ht(e,t){return e instanceof Error&&Oa in e&&(t==null||!!(e.type&t))}var xc=["params","query","hash"];function ep(e){if(typeof e=="string")return e;if(e.path!=null)return e.path;let t={};for(let n of xc)n in e&&(t[n]=e[n]);return JSON.stringify(t,null,2)}function Ia(e){let t={};if(e===""||e==="?")return t;let n=(e[0]==="?"?e.slice(1):e).split("&");for(let r=0;r<n.length;++r){let s=n[r].replace(ga," "),i=s.indexOf("="),a=Ao(i<0?s:s.slice(0,i)),u=i<0?null:Ao(s.slice(i+1));if(a in t){let m=t[a];wt(m)||(m=t[a]=[m]),m.push(u)}else t[a]=u}return t}function ir(e){let t="";for(let n in e){let r=e[n];if(n=jc(n),r==null){r!==void 0&&(t+=(t.length?"&":"")+n);continue}(wt(r)?r.map(s=>s&&Zs(s)):[r&&Zs(r)]).forEach(s=>{s!==void 0&&(t+=(t.length?"&":"")+n,s!=null&&(t+="="+s))})}return t}function Pa(e){let t={};for(let n in e){let r=e[n];r!==void 0&&(t[n]=wt(r)?r.map(s=>s==null?null:""+s):r==null?r:""+r)}return t}var Va=Symbol(""),ar=Symbol(""),os=Symbol(""),ns=Symbol(""),ss=Symbol("");function Io(){let e=[];function t(r){return e.push(r),()=>{let s=e.indexOf(r);s>-1&&e.splice(s,1)}}function n(){e=[]}return{add:t,list:()=>e.slice(),reset:n}}function Qt(e,t,n,r,s,i=a=>a()){let a=r&&(r.enterCallbacks[s]=r.enterCallbacks[s]||[]);return()=>new Promise((u,m)=>{let k=h=>{h===!1?m(fo(Be.NAVIGATION_ABORTED,{from:n,to:t})):h instanceof Error?m(h):Zc(h)?m(fo(Be.NAVIGATION_GUARD_REDIRECT,{from:t,to:h})):(a&&r.enterCallbacks[s]===a&&typeof h=="function"&&a.push(h),u())},y=i(()=>e.call(r&&r.instances[s],t,n,k)),E=Promise.resolve(y);e.length<3&&(E=E.then(k)),E.catch(h=>m(h))})}function rs(e,t,n,r,s=i=>i()){let i=[];for(let a of e)for(let u in a.components){let m=a.components[u];if(!(t!=="beforeRouteEnter"&&!a.instances[u]))if(va(m)){let k=(m.__vccOpts||m)[t];k&&i.push(Qt(k,n,r,a,u,s))}else{let k=m();i.push(()=>k.then(y=>{if(!y)throw new Error(`Couldn't resolve component "${u}" at "${a.path}"`);let E=Vc(y)?y.default:y;a.mods[u]=y,a.components[u]=E;let h=(E.__vccOpts||E)[t];return h&&Qt(h,n,r,a,u,s)()}))}}return i}function qa(e,t){let n=[],r=[],s=[],i=Math.max(t.matched.length,e.matched.length);for(let a=0;a<i;a++){let u=t.matched[a];u&&(e.matched.find(k=>po(k,u))?r.push(u):n.push(u));let m=e.matched[a];m&&(t.matched.find(k=>po(k,m))||s.push(m))}return[n,r,s]}var tp=()=>location.protocol+"//"+location.host;function Wa(e,t){let{pathname:n,search:r,hash:s}=t,i=e.indexOf("#");if(i>-1){let a=s.includes(e.slice(i))?e.slice(i).length:1,u=s.slice(a);return u[0]!=="/"&&(u="/"+u),or(u,"")}return or(n,e)+r+s}function op(e,t,n,r){let s=[],i=[],a=null,u=({state:h})=>{let C=Wa(e,location),g=n.value,b=t.value,S=0;if(h){if(n.value=C,t.value=h,a&&a===g){a=null;return}S=b?h.position-b.position:0}else r(C);s.forEach(R=>{R(n.value,g,{delta:S,type:es.pop,direction:S?S>0?ts.forward:ts.back:ts.unknown})})};function m(){a=n.value}function k(h){s.push(h);let C=()=>{let g=s.indexOf(h);g>-1&&s.splice(g,1)};return i.push(C),C}function y(){if(document.visibilityState==="hidden"){let{history:h}=window;if(!h.state)return;h.replaceState(Te({},h.state,{scroll:hn()}),"")}}function E(){for(let h of i)h();i=[],window.removeEventListener("popstate",u),window.removeEventListener("pagehide",y),document.removeEventListener("visibilitychange",y)}return window.addEventListener("popstate",u),window.addEventListener("pagehide",y),document.addEventListener("visibilitychange",y),{pauseListeners:m,listen:k,destroy:E}}function Ma(e,t,n,r=!1,s=!1){return{back:e,current:t,forward:n,replaced:r,position:window.history.length,scroll:s?hn():null}}function np(e){let{history:t,location:n}=window,r={value:Wa(e,n)},s={value:t.state};s.value||i(r.value,{back:null,current:r.value,forward:null,position:t.length-1,replaced:!0,scroll:null},!0);function i(m,k,y){let E=e.indexOf("#"),h=E>-1?(n.host&&document.querySelector("base")?e:e.slice(E))+m:tp()+e+m;try{t[y?"replaceState":"pushState"](k,"",h),s.value=k}catch(C){console.error(C),n[y?"replace":"assign"](h)}}function a(m,k){i(m,Te({},t.state,Ma(s.value.back,m,s.value.forward,!0),k,{position:s.value.position}),!0),r.value=m}function u(m,k){let y=Te({},s.value,t.state,{forward:m,scroll:hn()});i(y.current,y,!0),i(m,Te({},Ma(r.value,m,null),{position:y.position+1},k),!1),r.value=m}return{location:r,state:s,push:u,replace:a}}function Ka(e){e=Na(e);let t=np(e),n=op(e,t.state,t.location,t.replace);function r(i,a=!0){a||n.pauseListeners(),history.go(i)}let s=Te({locatLine truncated
`)),e.options=Object.assign({},e.options,y.options),a.restore()})}u();let m=null;e.saveDefault=k=>{if(k&&k.preventDefault&&k.preventDefault(),e.saving)return;let y=a.checkpoint(),E={terms:e.terms.split(`
`).map(h=>h.trim()).filter(h=>h.length>0),options:e.options};e.saving=!0,e.error=null,t.post("/api/user/default",E).then(()=>{a.accept(y),e.saving=!1,e.message=e.unsavedChanges?null:"Saved",m&&r.cancel(m),m=r(()=>{e.message=null},2500)},h=>{e.saving=!1;let C=h&&h.data&&h.data.error,g="Unable to save your defaults. Please try again.";Promise.all([n("ERRORS."+C),n("ERRORS.unknown_error")]).then(([b,S])=>{e.error=C&&b!==S?b:g},()=>{e.error=g})})},e.deleteAccount=()=>{confirm("Delete your account? All your anonymized repositories, gists, and pull requests will be removed, and your personal data will be erased. This cannot be undone.")&&(e.deletingAccount=!0,t.delete("/api/user").then(()=>{window.location.href="/"},()=>{e.deletingAccount=!1,e.deleteError="Unable to delete the account. Please try again."}))}},il=function(e,t,n){e.repoId=null,e.repoUrl=null,e.claim=()=>{t.post("/api/repo/claim",{repoId:e.repoId,repoUrl:e.repoUrl}).then(r=>{n.url("/dashboard")},r=>{e.error=r.data,e.claimForm.repoUrl.setValidity("not_found",!1),e.claimForm.repoId.setValidity("not_found",!1)})}},al=function(e,t,n,r,s){e.user&&!e.user.status&&n.url("/dashboard"),e.watch("user.status",()=>{e.user&&!e.user.status&&n.url("/dashboard")}),e.features=[{key:"anonymize",num:"01",eyebrow:"Anonymize",title:"Double-anonymous,",accent:"your rules.",text:"Choose what reviewers may see: links, images, PDFs, notebooks, GitHub Pages. Add your own terms, with regex if you need it, and pick the expiration date.",cta:"Start an anonymization",href:"/anonymize",url:"anonymous.4open.science/anonymize",img:"/imgs/anonymize.png",alt:"The anonymize form: source repository, terms to redact, options and a live README preview"},{key:"review",num:"02",eyebrow:"Review",title:"Reviewers browse",accent:"the real thing.",text:"Highlighted source code, rendered PDFs, images, and notebooks, in a familiar file explorer. GitHub Pages is also supported.",cta:"Open the example",href:"https://anonymous.4open.science/r/840c8c57-3c32-451e-bf12-0e20be300389/",target:"_self",url:"anonymous.4open.science/r/840c8c57-\u2026",img:"/imgs/explorer.png",alt:"The repository explorer with a file tree and a rendered README"},{key:"manage",num:"03",eyebrow:"Manage",title:"One dashboard,",accent:"until the decision.",text:"Monitor views, edit configuration, remove or update your repository. Program chairs can group submissions under a conference with one shared expiry.",cta:"Open the dashboard",href:"/dashboard",needsUser:!0,url:"anonymous.4open.science/dashboard",img:"/imgs/dashboard.png",alt:"The dashboard listing anonymized repositories with status, views and expiry"}],e.feature=e.features[0].key,e.selectFeature=function(u){e.feature=u},e.featureHref=function(u){return u.needsUser&&!e.user?"/signin":u.href},e.featureTarget=function(u){return u.needsUser&&!e.user?"_self":u.target||void 0},e.featureKeydown=function(u,m){let k={ArrowDown:1,ArrowRight:1,ArrowUp:-1,ArrowLeft:-1,Home:"first",End:"last"}[u.key];if(k===void 0)return;u.preventDefault();let y=e.features.length,E=k==="first"?0:k==="last"?y-1:(m+k+y)%y;e.feature=e.features[E].key,s(()=>{let h=r.document.getElementById("feature-tab-"+e.feature);h&&h.focus()})},e.cards=[{key:"repositories",total:0,label:"repositories anonymized"},{key:"users",total:0,label:"researchers"},{key:"pageViews",total:0,label:"page views"},{key:"pullRequests",total:0,label:"pull requests"}];function i(){t.get("/api/stat/").then(u=>{e.stat=u.data,e.cards[0].total=u.data.nbRepositories,e.cards[1].total=u.data.nbUsers,e.cards[2].total=u.data.nbPageViews,e.cards[3].total=u.data.nbPullRequests})}i();function a(u){let m={series:u,bars:[],viewW:100,deltaToday:0,pctChange:0,pctAbs:0,isUp:!0};if(!u||u.length<2)return m;let k=new Array(u.length-1);for(let b=1;b<u.length;b++)k[b-1]=u[b]-u[b-1];let y=k.length,E=Math.max.apply(null,k),h=Math.min.apply(null,k),C=Math.min(0,h),g=E-C||1;m.viewW=y*2,m.bars=new Array(y);for(let b=0;b<y;b++){let S=(k[b]-C)/g,R=Math.max(1.5,S*34);m.bars[b]={x:(b*2+.25).toFixed(2),y:(36-R).toFixed(2),w:"1.5",h:R.toFixed(2)}}if(m.deltaToday=k[y-1],y>=2){let b=k[y-2];b&&(m.pctChange=(m.deltaToday-b)/b*100)}return m.pctAbs=Math.round(Math.abs(m.pctChange)),m.isUp=m.pctChange>=0,m}e.history={repositories:a([]),users:a([]),pageViews:a([]),pullRequests:a([])},t.get("/api/stat/history?days=60").then(u=>{let m=u.data||[];e.history={repositories:a(m.map(k=>k.nbRepositories||0)),users:a(m.map(k=>k.nbUsers||0)),pageViews:a(m.map(k=>k.nbPageViews||0)),pullRequests:a(m.map(k=>k.nbPullRequests||0))}})},ll=function(e,t,n,r,s,i){let a=ut();e.on("routeLeave",function(){$('[data-toggle="tooltip"]').tooltip("dispose")}),e.watch("user.status",()=>{e.user==null&&n.url("/")}),e.user==null&&n.url("/"),a.timeout(()=>{$('[data-toggle="tooltip"]').tooltip()},250),e.quotaExpanded=!1,e.closeQuota=()=>{e.quotaExpanded=!1,ot(()=>document.querySelector(".dashboard-quota > summary")?.focus())},Zt()(document,"click",_=>{e.quotaExpanded&&!_.target.closest?.(".dashboard-quota")& Line truncated
`)),e.options=Object.assign({},e.options,D.options),e.options.expirationDate=D.options&&D.options.expirationDate?new Date(D.options.expirationDate):T(),!r.repoId&&!r.pullRequestId&&!r.gistId&&!e.anonymize?.terms?.dirty&&(e.terms=e.defaultTerms),p&&p()})}function _(p,v,D){e.anonymize&&e.anonymize[p]&&e.anonymize[p].setValidity(v,D)}function P(p){try{let v=parseGithubUrl(p);if(v&&v.owner&&v.repo)return v.owner+"/"+v.repo}catch{}return null}e.accessRepairAllowed=!1,e.repairingGitHubAccess=!1,e.accessRepairMessage="",e.repairGitHubAccess=async()=>{if(!e.accessRepairAllowed||e.repairingGitHubAccess||!e.githubConnections?.appEnabled||!r.repoId||e.role!=="owner"||e.githubConnection!=="github-app"||P(e.sourceUrl)!==e._originalFullName)return;e.repairingGitHubAccess=!0,e.accessRepairMessage="";let p=h();try{let v=await t.get("/github/connections");if(e.githubConnections=v.data,!v.data.appEnabled)return;let D=await t.post("/github/connections/migrate",{type:"repository",id:r.repoId,connection:"github-app"},{headers:{"X-CSRF-Token":v.data.csrf}});if(!E(p))return;e.error=null,D.data.fullName&&(e._originalFullName=D.data.fullName,e.source.fullName=D.data.fullName,e.sourceUrl="https://github.com/"+D.data.fullName),e.accessRepairMessage="GitHub access repaired. Your anonymous URL and published commit are unchanged.",await e.urlSelected(!0)}catch{E(p)&&(e.accessRepairMessage="Unable to repair GitHub access. Check the App's repository permissions in Manage access, then retry. Your published snapshot has not been replaced.")}finally{e.repairingGitHubAccess=!1,C(p)}},e.reconnectingSource=!1,e.reconnectRepositoryId=void 0,e.reconnectSource=async()=>{if(confirm("Reconnect to the repository currently at this GitHub URL? Review its branch and commit before saving. Saving will replace the published snapshot.")){e.reconnectingSource=!0,e.reconnectRepositoryId=void 0,e.source.commit="",e.source.branch="",e.readme="",e.branches=[];try{await W()}catch{}}};function G(){return e.reconnectingSource||!e.isUpdate||!e._originalRepositoryID?void 0:P(e.sourceUrl)===e._originalFullName?e._originalRepositoryID:void 0}async function J(){e._preservingDraft=!0;try{await e.urlSelected(!0)}finally{e._preservingDraft=!1}}async function X(){let p;try{p=JSON.parse(sessionStorage.getItem("github-access-draft")||"null")}catch{return!1}if(!p||p.path!==s.path()||Date.now()-p.savedAt>=30*6e4)return S.restore()?(e.sourceUrl&&await J(),!0):!1;S.restore();for(let v of["sourceUrl","terms","repoId","pullRequestId","gistId","source","options","conference","githubConnection"])Object.prototype.hasOwnProperty.call(p.draft,v)&&(e[v]=p.draft[v]);return e.options.expirationDate&&(e.options.expirationDate=new Date(e.options.expirationDate)),await J(),sessionStorage.removeItem("github-access-draft"),!0}ee(()=>{!r.repoId&&!r.pullRequestId&&!r.gistId&&a(X,0),r.repoId&&r.repoId!=""&&(e.isUpdate=!0,e.detectedType="repo",e.repoId=r.repoId,t.get("/api/repo/"+e.repoId).then(async p=>{e.accessRepairAllowed=!["removed","archived","preparing","queue","download","removing","expiring"].includes(p.data.status),e.githubConnection=p.data.connection||"oauth",e.sourceUrl="https://github.com/"+p.data.source.fullName,e._originalFullName=p.data.source.fullName,e.terms=p.data.options.terms.filter(v=>v).join(`
`),e.source=p.data.source,e.role=p.data.role||"owner",e.coauthors=p.data.coauthors||[],e._originalBranch=p.data.source.branch,e.options=Object.assign({},e.options,p.data.options),e.conference=p.data.conference,e._originalConference=p.data.conference,e.repositoryID=p.data.source.repositoryID,e._originalRepositoryID=p.data.source.repositoryID,p.data.options.expirationDate&&(e.options.expirationDate=new Date(p.data.options.expirationDate)),g=b(parseGithubUrl(e.sourceUrl)),!await X()&&(await Promise.all([W(),se()]),_e())},()=>{s.url("/404")})),r.pullRequestId&&r.pullRequestId!=""&&(e.isUpdate=!0,e.detectedType="pr",e.pullRequestId=r.pullRequestId,t.get("/api/pr/"+e.pullRequestId).then(async p=>{if(e.githubConnection=p.data.connection||"oauth",e.sourceUrl="https://github.com/"+p.data.source.repositoryFullName+"/pull/"+p.data.source.pullRequestId,e.terms=p.data.options.terms.filter(v=>v).join(`
`),e.source=p.data.source,e.options=Object.assign({},e.options,p.data.options),e.conference=p.data.conference,e._originalConference=p.data.conference,p.data.options.expirationDate&&(e.options.expirationDate=new Date(p.data.options.expirationDate)),g=b(parseGithubUrl(e.sourceUrl)),!await X())try{await qe()}catch{}},()=>{s.url("/404")})),r.gistId&&r.gistId!=""&&(e.isUpdate=!0,e.detectedType="gist",e.gistId=r.gistId,t.get("/api/gist/"+e.gistId).then(async p=>{e.sourceUrl="https://gist.github.com/"+p.data.source.gistId,e.terms=p.data.options.terms.filter(v=>v).join(`
`),e.source=p.data.source,e.options=Object.assign({},e.options,p.data.options),e.conference=p.data.conference,e._originalConference=p.data.conference,p.data.options.expirationDate&&(e.options.expirationDate=new Date(p.data.options.expirationDate)),g=b(parseGithubUrl(e.sourceUrl)),!await X()&&await ze()},()=>{s.url("/404")}))}),t.get("/github/connections").then(p=>{e.githubConnections=p.data,!r.repoId&&!r.pullRequestId&&!r.gistId&&e.githubConnection===void 0&&p.data.appEnabled&&p.data.appConnected&&(e.githubConnection="github-app",e.sourceUrl&&e.workspaceOpen&&J())}).catch(()=>{}),e.urlSelected=async(p=!1)=>{let v=++m;e.previousConfigurations=[],e.previousConfigurationId="";let D;try{D=parseGithubUrl(e.sourceUrl.trim())}catch{e.sourceLoading=!1,re.cancel(),Se.cancel(),F.cancel(),e.previewStatus="idle",e.sourceUrlError=e.sourceUrl?"Please provide a valid GitHub URL.":"",_("sourceUrl","github",!1);return}let O=b(D),I=p||g===O;g=O,e.sourceUrlError="",_("sourceUrl","github",!0);let U=document.activeElement?.id==="sourceUrl-landing";e.workspaceOpen=!0,U&&a(()=>document.getElementById("sourceUrl")?.focus(),0),e.reconnectingSource=!1,e.reconnectRepositoryId=void 0,!I&&!e.isUpdate&&(e.repoId="",e.pullRequestId="",e.gistId=""),e.details=null,e.branches=[],I||(e.source={type:"GitHubStream",branch:"",commit:""}),re.cancel(),Se.cancel(),F.cancel(),Me.clear(),Ue.clear(),e.previewStatus="idle",e.anonymize_readme="",e.readme="",e.html_readme="",e.detectedType=null,e.gistOAuthRequired=!1,e.gistAppRequired=!1,e._preservingDraft=I;try{D.gistId&&!D.repo?(e.detectedType="gist",e.source={gistId:D.gistId},await ze()):D.pullRequestId?(e.detectedType="pr",e.source={repositoryFullName:D.owner+"/"+D.repo,pullRequestId:D.pullRequestId},await qe()):(e.detectedType="repo",ne(),await Promise.all([W(),se()]),_e())}catch{}finally{v===m&&(e._preservingDraft=!1)}$('[data-toggle="tooltip"]').tooltip()},$('[data-toggle="tooltip"]').tooltip(),e.watch("source.branch",async()=>{if(e.detectedType!=="repo")return;let p=e.branches.filter(D=>D.name==e.source.branch)[0];if(!p)return;!(!e.reconnectingSource&&e.isUpdate&&e._originalBranch===e.source.branch&&e.source.commit)&&!(e._preservingDraft&&e.source.commit)&&(e.source.commit=p.commit),e.readme=p.readme,await se(),_e()}),e.getBranches=async p=>{let v=parseGithubUrl(e.sourceUrl),D=h();try{let O=await t.get(`/api/repo/${v.owner}/${v.repo}/branches`,{params:{reconnect:e.reconnectingSource?"1":void 0,anonymizedRepoId:e.isUpdate&&r.repoId&&P(e.sourceUrl)===e._originalFullName?r.repoId:void 0,connection:e.githubConnection,force:p===!0||e.reconnectingSource?"1":"0",repositoryID:G()}});if(!E(D))return;e.branches=O.data,e.sourceUnreachable=!1,e.source.branch||(e.source.branch=e.details.defaultBranch);let I=e.branches.filter(U=>U.name==e.source.branch);I.length>0&&(!(!e.reconnectingSource&&!p&&e.isUpdate&&!e.options.update&&e._originalBranch===e.source.branch&&e.source.commit)&&!(e._preservingDraft&&e.source.commit)&&(e.source.commit=I[0].commit),e.readme=I[0].readme,await se(p))}catch(O){if(!E(D))return;e.branches=[],e.sourceUnreachable=O&&(O.status===404||O.data&&O.data.error==="repo_not_found");let I=O&&O.data&&O.data.error||(O&&O.status===404?"repo_not_found":"unknown_error");i("ERRORS."+I).then(U=>{e.toasts=e.toasts||[],e.addToast({title:"Error",date:new Date,body:U}),e.error=U},console.error),typeof _=="function"&&_("sourceUrl","missing",!1)}finally{C(D)}};async function W(){let p=parseGithubUrl(e.sourceUrl),v=h();try{te();let D=await t.get(`/api/repo/${p.owner}/${p.repo}/`,{params:{reconnect:e.reconnectingSource?"1":void 0,anonymizedRepoId:e.isUpdate&&r.repoId&&P(e.sourceUrl)===e._originalFullName?r.repoId:void 0,connection:e.githubConnection,repositoryID:G(),force:"1"}});if(!E(v))return;e.details=D.data,e.reconnectingSource&&(e.reconnectRepositoryId=e.details.externalId),e.details&&e.details.id&&(e.repositoryID=e.details.id),e.repoId||(e.repoId=e.details.repo+"-"+generateRandomId(4)),await e.getBranches(e.reconnectingSource)}catch(D){if(!E(v))return;throw D.data&&(i("ERRORS."+D.data.error).then(O=>{e.addToast({title:"Error",date:new Date,body:O}),e.error=O},console.error),Ee(D.data.error)),_("sourceUrl","missing",!1),D}finally{C(v)}}async function se(p){if(e.readme&&!p)return e.readme;let v=parseGithubUrl(e.sourceUrl),D=h();try{let O=await t.get(`/api/repo/${v.owner}/${v.repo}/readme`,{params:{reconnect:e.reconnectingSource?"1":void 0,anonymizedRepoId:e.isUpdate&&r.repoId&&P(e.sourceUrl)===e._originalFullName?r.repoId:void 0,connection:e.githubConnection,force:p===!0?"1":"0",branch:e.source.branch,repositoryID:G()}});E(D,!0)&&(e.readme=O.data)}catch{E(D,!0)&&(e.readme="")}finally{C(D)}}function ce(){let p={terms:e.terms?e.terms.split(`
`):[],image:!!e.options.image,link:!!e.options.link,repoId:e.repoId};e.source&&e.source.branch&&(p.branchName=e.source.branch);try{let v=parseGithubUrl(e.sourceUrl);p.repoName=`${v.owner}/${v.repo}`}catch{}return p}function de(p,v,D){let O=null,I=0,U=()=>{if(e.detectedType!==p)return;O&&a.cancel(O);let L=++I;e.previewStatus="loading",O=a(()=>{O=null;let H=v();if(!H){e.previewStatus="idle";return}t.post("/api/anonymize-preview",H).then(Y=>{L===I&&(D(Y.data,H),e.previewStatus="ready")}).catch(()=>{L===I&&(e.previewStatus="error")})},200)};return U.cancel=()=>{I++,O&&a.cancel(O),O=null},e.on("dispose",U.cancel),U}e.retryPreview=()=>{e.detectedType==="repo"&&re(),e.detectedType==="pr"&&Se(),e.detectedType==="gist"&&F()};let re=de("repo",()=>e.readme?{content:e.readme,options:ce()}:null,p=>{if(typeof p?.content!="string")throw Error("Invalid preview response");e.anonymize_readme=p.content;let v="";try{let O=parseGithubUrl(e.sourceUrl),I=e.source.branch||e.details&&e.details.defaultBranch||"main";v=`https://github.com/${O.owner}/${O.repo}/raw/${I}/`}catch{}let D=renderMD(e.anonymize_readme,v);e.html_readme=D,a(Prism.highlightAll,150)});function _e(){!e.anonymize||!e.anonymize.terms||(e.termsRegexWarning=!!e.terms&&!!e.terms.match(/[-[\]{}()*+?.,\\^$|#]/g),re())}async function qe(){let p=parseGithubUrl(e.sourceUrl),v=h();try{te();let D=await t.get(`/api/pr/${p.owner}/${p.repo}/${p.pullRequestId}`,{params:{connection:e.githubConnection}});if(!E(v))return;e.details=D.data,e.pullRequestId||(e.pullRequestId=p.repo+"-PR"+p.pullRequestId+"-"+generateRandomId(4))}catch(D){if(!E(v))return;throw D.data&&(i("ERRORS."+D.data.error).then(O=>{e.addToast({title:"Error",date:new Date,body:O}),e.error=O},console.error),Ee(D.data.error)),_("sourceUrl","missing",!1),D}finally{C(v)}}let Me=Fe(new Map);function Pe(){let p=new Set,v=e.details&&e.details.pullRequest;if(!v)return p;typeof v.title=="string"&&p.add(v.title),typeof v.body=="string"&&p.add(v.body),typeof v.diff=="string"&&p.add(v.diff);let D=v.comments||[];for(let O of D)typeof O.author=="string"&&p.add(O.author),typeof O.body=="string"&&p.add(O.body);return p}let Se=de("pr",()=>{let p=Pe(),v=Array.from(p);return v.length===0?null:{contents:v,options:ce()}},(p,v)=>{if(!Array.isArray(p?.contents)||p.contents.length!==v.contents.length||p.contents.some(I=>typeof I!="string"))throw Error("Invalid preview response");let D=v.contents,O=new Map;for(let I=0;I<D.length&&I<p.contents.length;I++)O.set(D[I],p.contents[I]);Me.clear(),O.forEach((I,U)=>Me.set(U,I))});e.anonymizePrContent=function(p){return p&&(Me.has(p)?Me.get(p):"")};async function ze(){let p=parseGithubUrl(e.sourceUrl),v=h();try{te(),e.gistOAuthRequired=!1,e.gistAppRequired=!1;let D=await t.get(`/api/gist/source/${p.gistId}`);if(!E(v))return;e.details=D.data,e.gistId||(e.gistId="gist-"+p.gistId.substring(0,6)+"-"+generateRandomId(4))}catch(D){if(!E(v))return;if(D.data?.error==="github_app_reconnect_required"){e.gistAppRequired=!0,_("sourceUrl","missing",!1);return}if(D.data?.error==="github_oauth_required"){e.gistOAuthRequired=!0,_("sourceUrl","missing",!1);return}throw D.data&&(i("ERRORS."+D.data.error).then(O=>{e.addToast({title:"Error",date:new Date,body:O}),e.error=O},console.error),Ee(D.data.error)),_("sourceUrl","missing",!1),D}finally{C(v)}}let Ue=Fe(new Map);function ke(){let p=new Set,v=e.details&&e.details.gist;if(!v)return p;typeof v.description=="string"&&p.add(v.description),typeof v.ownerLogin=="string"&&p.add(v.ownerLogin);let D=v.files||[];for(let I of D)typeof I.filename=="string"&&p.add(I.filename),typeof I.content=="string"&&p.add(I.content);let O=v.comments||[];for(let I of O)typeof I.author=="string"&&p.add(I.author),typeof I.body=="string"&&p.add(I.body);return p}let F=de("gist",()=>{let p=ke(),v=Array.from(p);return v.length===0?null:{contents:v,options:ce()}},(p,v)=>{if(!Array.isArray(p?.contents)||p.contents.length!==v.contents.length||p.contents.some(I=>typeof I!="string"))throw Error("Invalid preview response");let D=v.contents,O=new Map;for(let I=0;I<D.length&&I<p.contents.length;I++)O.set(D[I],p.contents[I]);Ue.clear(),O.forEach((I,U)=>Ue.set(U,I)),Z()});e.anonymizeGistContent=function(p){return p&&(Ue.has(p)?Ue.get(p):"")},e.previewGistFiles=[];function Z(){let p=e.details&&e.details.gist&&e.details.gist.files||[];e.previewGistFiles=p.map(v=>({filename:e.anonymizeGistContent(v.filename),content:e.anonymizeGistContent(v.content),language:v.language}))}e.watch("details.gist.files",Z,!0),e.watch("terms",Z);function Q(){let p=e.conference;if(e.conference_data=null,!p)return;let v=e.isUpdate&&p===e._originalConference;t.get("/api/conferences/"+p).then(D=>{e.conference===p&&(e.conference_data=D.data,e.conference_data.startDate=new Date(e.conference_data.startDate),e.conference_data.endDate=new Date(e.conference_data.endDate),!v&&(e.options.expirationDate=new Date(e.conference_data.endDate),e.options.expirationMode="remove",e.options.update=e.conference_data.options.update,e.options.image=e.conf Line truncated
`).filter(U=>U),connection:e.githubConnection,fullName:`${D.owner}/${D.repo}`,repository:e.sourceUrl,reconnectRepositoryId:e.reconnectRepositoryId,options:e.options,source:e.source,conference:e.conference};e.details&&(O.options.pageSource=e.details.pageSource),te();let I=e.isUpdate?"/api/repo/"+e.repoId:"/api/repo/";t.post(I,O,{headers:{"Content-Type":"application/json"}}).then(()=>{S.clear(),window.location.href="/status/"+e.repoId},U=>{U.data&&(i("ERRORS."+U.data.error).then(L=>{e.error=L},console.error),Ee(U.data.error))}).finally(()=>{v.disabled=!1})},e.anonymizeGist=p=>{if(e.sourceLoading){e.error="Wait for the source to finish loading before saving.";return}if(ye())return;if(e.anonymize?.invalid||e.sourceUrlError){e.error="Please correct the highlighted fields before saving.";return}let v=p.currentTarget||p.target.closest?.("button")||p.target;v.disabled=!0;let D=parseGithubUrl(e.sourceUrl),O={gistId:e.gistId,terms:e.terms.trim().split(`
`).filter(U=>U),source:{gistId:D.gistId},options:e.options,conference:e.conference};te();let I=e.isUpdate?"/api/gist/"+e.gistId:"/api/gist/";t.post(I,O,{headers:{"Content-Type":"application/json"}}).then(()=>{S.clear(),window.location.href="/gist/"+e.gistId},U=>{U.data&&(i("ERRORS."+U.data.error).then(L=>{e.error=L},console.error),Ee(U.data.error))}).finally(()=>{v.disabled=!1})},e.anonymizePullRequest=p=>{if(e.sourceLoading){e.error="Wait for the source to finish loading before saving.";return}if(ye())return;if(e.anonymize?.invalid||e.sourceUrlError){e.error="Please correct the highlighted fields before saving.";return}let v=p.currentTarget||p.target.closest?.("button")||p.target;v.disabled=!0;let D=parseGithubUrl(e.sourceUrl),O={pullRequestId:e.pullRequestId,connection:e.githubConnection,terms:e.terms.trim().split(`
`).filter(U=>U),source:{repositoryFullName:`${D.owner}/${D.repo}`,pullRequestId:D.pullRequestId},options:e.options,conference:e.conference};te();let I=e.isUpdate?"/api/pr/"+e.pullRequestId:"/api/pr/";t.post(I,O,{headers:{"Content-Type":"application/json"}}).then(()=>{S.clear(),window.location.href="/pr/"+e.pullRequestId},U=>{U.data&&(i("ERRORS."+U.data.error).then(L=>{e.error=L},console.error),Ee(U.data.error))}).finally(()=>{v.disabled=!1})},e.watch("conference",()=>{Q()}),e.watch("terms",()=>{e.detectedType==="repo"&&_e(),e.detectedType==="pr"&&Se(),e.detectedType==="gist"&&F()}),e.watch("options.image",()=>{e.detectedType==="repo"&&_e(),e.detectedType==="pr"&&Se(),e.detectedType==="gist"&&F()}),e.watch("options.link",()=>{e.detectedType==="repo"&&_e(),e.detectedType==="pr"&&Se(),e.detectedType==="gist"&&F()}),e.watch("details",()=>{e.detectedType==="pr"&&Se(),e.detectedType==="gist"&&F()},!0)},pr=function(e,t,n,r,s,i){let a=ut();window.Prism?.hooks&&!window.Prism.redactionHook&&(window.Prism.hooks.add("complete",P=>el(P.element)),window.Prism.redactionHook=!0);let u=Zt();e.on("dark-mode",(P,G)=>{e.aceOption&&(G?e.aceOption.theme="nord_dark":e.aceOption.theme="chrome")});let m=0,k=!1;e.on("dispose",()=>{k=!0,m++,y&&y.resolve()}),e.files=[],e.isMac=/Mac|iPhone|iPad|iPod/.test(navigator.platform||navigator.userAgent),e.fileSearchQuery="",e.fileSearchResults=null,e.fileSearchLoading=!1,u(document,"keydown",function(P){if((P.metaKey||P.ctrlKey)&&P.key==="k"){P.preventDefault();var G=document.querySelector(".tree-search-input");G&&(G.focus(),G.select())}});var y=null;e.onFileSearchChange=function(){y&&(y.resolve(),y=null);let P=e.fileSearchQuery;if(!P||P.length<2){e.fileSearchResults=null,e.fileSearchLoading=!1;return}e.fileSearchLoading=!0;let G=i.defer();y=G,t.get(`/api/repo/${e.repoId}/files/search?q=${encodeURIComponent(P)}`,{timeout:G.promise}).then(function(J){if(!(k||y!==G)){y=null,e.fileSearchLoading=!1;var X={};e.files.forEach(function(F){X[(F.path||"")+"/"+F.name]=!0});for(var W=[],se={},ce=0;ce<J.data.length;ce++){var de=J.data[ce];if(de.path)for(var re=de.path.split("/"),_e="",qe=0;qe<re.length;qe++){var Me=_e;_e=_e?_e+"/"+re[qe]:re[qe];var Pe=Me+"/"+re[qe];!X[Pe]&&!se[Pe]&&(se[Pe]=!0,W.push({name:re[qe],path:Me}))}}W.sort(function(F,Z){return(F.path||"").split("/").length-(Z.path||"").split("/").length}),W.length>0&&e.files.push.apply(e.files,W);for(var Se=[],ze=0;ze<J.data.length;ze++){var Ue=J.data[ze],ke=(Ue.path||"")+"/"+Ue.name;!X[ke]&&Ue.size!=null&&(Se.push(Ue),X[ke]=!0)}Se.length>0&&e.files.push.apply(e.files,Se),e.fileSearchResults=J.data}},function(){!k&&y===G&&(y=null,e.fileSearchLoading=!1,e.fileSearchResults=[])})};let E={yml:"yaml",txt:"text",py:"python",js:"javascript",ts:"typescript"},h=["license","txt"],C=["png","jpg","jpeg","gif","svg","ico","bmp","tiff","tif","webp","avif","heif","heic"],g=["wav","mp3","ogg","wma","flac","aac","m4a"],b=["mp4","avi","webm","mov","mpg","mpeg","mkv","flv","wmv","3gp","3g2","m4v","f4v","f4p","f4a","f4b"];e.on("routeUpdate",function(P,G){if(e.repoId!=r.repoId)return _();if((r.path||"")!=e.filePath){e.filePath=r.path||"",e.paths=e.filePath.split("/").filter(J=>J&&J.trim().length>0),ee();for(let J=0;J<e.paths.length-1;J++){let X=J>0?e.paths.slice(0,J).join("/"):"";e.files.some(se=>se.path===X)||e.getFiles(X)}}});function S(){if(e.paths[0]!="")return;let P=["readme.md","readme.txt","readme.org","readme.1st","readme"],G={};for(let X of e.files)X.name.toLowerCase().indexOf("readme")>-1&&(G[X.name.toLowerCase()]=X.name);let J=null;for(let X of P)if(G[X]){J=X;break}if(!J&&Object.keys(G).length>0&&(J=Object.keys(G)[0]),J){let X=n.url();X[X.length-1]!="/"&&(X+="/"),n.url(X+encodePathForUrl(G[J]))}}e.fileCounts=null,e.getFiles=function(P){let G=e.repoId;return t.get(`/api/repo/${e.repoId}/files/?path=${encodeURIComponent(P)}&v=${e.options.lastUpdateDate}`).then(function(J){if(k||G!==e.repoId)return[];let X=P||"";return e.files=e.files.filter(W=>W.path!==X),e.files.push(...J.data),J.data},function(J){if(k||G!==e.repoId)return[];e.type="error",e.content=cr(J),e.files=[]})};function R(){let P=e.repoId;t.get(`/api/repo/${e.repoId}/files/counts`).then(function(G){k||P!==e.repoId||(e.fileCounts=G.data)},function(){e.fileCounts={}})}function T(){return e.files.filter(P=>P.name==e.paths[e.paths.length-1]&&P.path==e.paths.slice(0,e.paths.length-1).join("/"))[0]}var q=null;e.on("dispose",function(){q&&a.interval.cancel(q)});function M(P){if(k)return;let G=e.repoId;t.get(`/api/repo/${e.repoId}/options`).then(J=>{if(!(k||G!==e.repoId)){if(e.options=J.data,e.options.url){window.location=e.options.url;return}P&&P(J.data)}},J=>{if(!(k||G!==e.repoId)){var X=J.data||{};if(X.error==="rate_limited"&&X.resetAt){let se=function(){var ce=Math.max(0,Math.ceil((e.rateLimitResetAt-Date.now())/1e3));if(ce<=0)e.rateLimitCountdown="",e.rateLimitResetAt=0,q&&(a.interval.cancel(q),q=null),M(P);else{var de=Math.floor(ce/60),re=ce%60;e.rateLimitCountdown=de>0?de+"m "+re+"s":re+"s"}};var W=s Line truncated
`+b.join(`
`)],{type:"text/csv"}),R=document.createElement("a");R.href=URL.createObjectURL(S),R.download=r.username+"-repositories.csv",R.click()},e.showStatusMessage=h=>{let C=h.statusMessage||"(no message)";window.prompt(`Status message for ${h.repoId} (${h.status}):`,C)},e.fetchGithubInfo=h=>{let C=window.open("","_blank");C&&C.document.write("<pre>Loading GitHub info for "+h.repoId+"...</pre>"),t.get("/api/admin/repos/"+h.repoId+"/github").then(g=>{C&&(C.document.open(),C.document.write('<pre style="font:13px monospace;padding:16px;white-space:pre-wrap">'+JSON.stringify(g.data,null,2).replace(/[<>]/g,b=>b==="<"?"&lt;":"&gt;")+"</pre>"),C.document.close())},g=>{let b=g&&g.data?JSON.stringify(g.data,null,2):String(g);C&&(C.document.body.innerHTML='<pre style="color:#B42318;padding:16px">'+b+"</pre>")})};function m(h){t.get("/api/admin/users/"+h+"/repos",{}).then(C=>{e.repositories=C.data},C=>{console.error(C)})}function k(h){t.get("/api/admin/users/"+h,{}).then(C=>{e.userInfo=C.data},C=>{console.error(C)})}k(r.username),m(r.username),e.banUser=()=>{confirm(`Ban user ${r.username}?`)&&t.post(`/api/admin/users/${r.username}/ban`).then(()=>k(r.username),h=>console.error(h))},e.activateUser=()=>{t.post(`/api/admin/users/${r.username}/activate`).then(()=>k(r.username),h=>console.error(h))},e.promoteUser=()=>{confirm(`Promote ${r.username} to admin?`)&&t.post(`/api/admin/users/${r.username}/promote`).then(()=>k(r.username),h=>console.error(h))},e.demoteUser=()=>{confirm(`Remove admin privileges from ${r.username}?`)&&t.post(`/api/admin/users/${r.username}/demote`).then(()=>k(r.username),h=>console.error(h))},e.tokens=[],e.tokenForm={name:"",plaintext:null};function y(){t.get("/api/admin/tokens").then(h=>{e.tokens=h.data||[]},h=>{h.status!==401&&h.status!==403&&console.error(h)})}y(),e.createToken=()=>{e.tokenForm.name&&t.post("/api/admin/tokens",{name:e.tokenForm.name}).then(h=>{e.tokenForm.plaintext=h.data.token,e.tokenForm.name="",y()},h=>console.error(h))},e.revokeToken=h=>{confirm(`Revoke token "${h.name}"?`)&&t.delete("/api/admin/tokens/"+h.id).then(()=>y(),C=>console.error(C))},e.removeCache=h=>{confirm("Remove cached files for "+h.repoId+"?")&&t.delete("/api/admin/repos/"+h.repoId).then(()=>m(r.username),C=>console.error(C))},e.removeRepository=h=>{confirm("Remove repository "+h.repoId+"?")&&t.delete("/api/repo/"+h.repoId+"/").then(()=>m(r.username),C=>console.error(C))},e.updateRepository=h=>{let C=Fe({title:`Refreshing ${h.repoId}...`,date:new Date,body:`The repository ${h.repoId} is going to be refreshed.`});e.toasts.push(C),t.post(`/api/repo/${h.repoId}/refresh`).then(g=>{g.data.status=="ready"?C.title=`${h.repoId} is refreshed.`:C.title=`Refreshing of ${h.repoId}.`},g=>{C.title=`Error during the refresh of ${h.repoId}.`,C.body=g.body})},e.getGitHubRepositories=h=>{t.get(`/api/user/${e.userInfo.username}/all_repositories`,{params:{force:"1"}}).then(C=>{e.userInfo.repositories=C.data})};let E=null;e.watch("query",()=>{s.timeout.cancel(E),E=s.timeout(()=>{m(r.username)},500)},!0)},bl=function(e,t,n){let r=ut(),s=Zt();e.Math=Math,e.watch("user.status",()=>{e.user==null&&n.url("/")}),e.user==null&&n.url("/"),e.conferences=[],e.total=-1,e.totalPage=0,e.statusCounts=[];let i=g=>{if(g.key==="/"&&!["INPUT","TEXTAREA","SELECT"].includes(document.activeElement?.tagName)){g.preventDefault();let b=document.querySelector('.admin-filter-toolbar input[type="search"]');b&&b.focus()}};s(document,"keydown",i),e.on("dispose",()=>document.removeEventListener("keydown",i)),e.clearFilter=g=>{g==="dateRange"?(e.query.dateFrom="",e.query.dateTo=""):e.query[g]="",e.query.page=1},e.chips=[];let a=()=>{let g=[];(e.query.dateFrom||e.query.dateTo)&&g.push({key:"dateRange",label:"Date",value:(e.query.dateFrom||"\u2026")+" \u2013 "+(e.query.dateTo||"\u2026")}),e.chips=g};e.statusCountFor=g=>{let b=(e.statusCounts||[]).find(S=>S._id===g);return b?b.count:0},e.toggleSortDirection=()=>{e.query.direction=e.query.direction==="asc"?"desc":"asc"},e.sortBy=g=>{e.query.sort===g?e.query.direction=e.query.direction==="asc"?"desc":"asc":(e.query.sort=g,e.query.direction="desc"),e.query.page=1},e.sortIcon=g=>e.query.sort===g?e.query.direction==="asc"?"fa-arrow-up":"fa-arrow-down":"";let u="admin.conferences.filterPrefs",m={page:1,limit:25,sort:"name",direction:"asc",search:"",dateFrom:"",dateTo:"",ready:!1,expired:!1,removed:!1,error:!0,preparing:!0},k=loadFilterPrefs(u)||{};e.query=Object.assign({},m,k,{page:1,search:""});let y=n.search();y.search&&(e.query.search=y.search);let E="admin.conferences.presets";e.presets=JSON.parse(localStorage.getItem(E)||"[]"),e.savePreset=()=>{let g=window.prompt("Preset name:");if(!g)return;let b=Object.assign({},e.query);delete b.page,e.presets=(e.presets||[]).filter(S=>S.name!==g),e.presets.push({name:g,query:b}),localStorage.setItem(E,JSON.stringify(e.presets))},e.applyPreset=g=>{Object.assign(e.query,g.query,{page:1})},e.deletePreset=g=>{e.presets=(e.presets||[]).filter(b=>b.name!==g.name),localStorage.setItem(E,JS Line truncated
`).map((ce,de)=>de===0?ce:se+ce).join(`
`)}else typeof _=="number"||typeof _=="boolean"?W=String(_):W=JSON.stringify(_);ne.push(`${J}${W}${X}`)}),ne.push("}"),ne.join(`
`)}function k(R){let T=[],q="",M=/(\w+):(>=|<=|!=|>|<|=)?([^\s]+)/g,V=0,z;for(;z=M.exec(R);)q+=R.slice(V,z.index),V=M.lastIndex,T.push({key:z[1],op:z[2]||"=",val:z[3]});return q+=R.slice(V),{filters:T,free:q.trim().toLowerCase()}}function y(R,T){for(let q of T.filters){let M=(z,ne,ee)=>{let _=parseFloat(z),P=parseFloat(ne);return ee==="="?String(z)===String(ne):ee==="!="?String(z)!==String(ne):ee===">="?_>=P:ee==="<="?_<=P:ee===">"?_>P:ee==="<"?_<P:!0},V;if(q.key==="code")V=R.displayMessage;else if(q.key==="module")V=R.module;else if(q.key==="status")V=R._status;else if(q.key==="url")V=R._url;else if(q.key==="repo")V=R._repoId;else if(q.key==="level")V=R.level;else continue;if(V==null||!M(V,q.val,q.op))return!1}return!(T.free&&((R.displayMessage||"")+" "+(R.module||"")+" "+(R._url||"")+" "+JSON.stringify(R.raw||[])).toLowerCase().indexOf(T.free)===-1)}function E(){let R=k(e.query.search||"");e.parsedFilterCount=R.filters.length;let T=e.query.bucket,q=e.entries.filter(V=>T&&V._bucket!==T?!1:y(V,R)),M=e.query.group;if(M){let V=ee=>M==="module"?ee.module:ee.displayMessage||ee.message||"_",z=new Map;for(let ee of q){let _=V(ee);if(z.has(_)){let P=z.get(_);P.count++,P._related.push(ee),new Date(ee.ts)>new Date(P.ts)&&(P.ts=ee.ts,P._url=ee._url,P._status=ee._status),new Date(ee.ts)<new Date(P._firstSeen)&&(P._firstSeen=ee.ts)}else{let P=Object.assign({},ee);P._key=`${M}:${_}`,P._related=[ee],P._firstSeen=ee.ts,P._lastHourCount=0,P.count=1,z.set(_,P)}}let ne=Date.now()-3600*1e3;for(let ee of z.values())ee._lastHourCount=ee._related.filter(_=>new Date(_.ts).getTime()>=ne).length;q=Array.from(z.values())}else q=q.map((V,z)=>(V._key="row:"+z+":"+V.ts,V._related=[V],V._firstSeen=V.ts,V._lastHourCount=0,V.count=1,V));e.query.sort==="count"?q.sort((V,z)=>z.count-V.count||new Date(z.ts)-new Date(V.ts)):q.sort((V,z)=>new Date(z.ts)-new Date(V.ts)),e.visible=q}function h(R){let T=R?e.entries.length:0,q=R?e.pageSize:Math.max(e.pageSize,e.entries.length||e.pageSize);t.get("/api/admin/errors",{params:{offset:T,limit:q}}).then(M=>{let V=(M.data.entries||[]).map(u);e.entries=R?e.entries.concat(V):V,e.available=!!M.data.available,e.cap=M.data.max||e.cap,e.total=M.data.total||e.entries.length,E()},M=>console.error(M))}e.loadMore=()=>h(!0),e.canLoadMore=()=>e.entries.length<e.total;function C(){t.get("/api/admin/errors/stats").then(R=>{let T=R.data||{},q=T.prev24h?Math.round((T.last24h-T.prev24h)/T.prev24h*100):0;e.stats={last24h:T.last24h||0,prev24h:T.prev24h||0,delta:q,severity:T.severity||{error:0,warn:0,info:0},unique:T.unique||{error:0,warn:0,info:0},buckets:T.buckets||[],dropped:T.dropped||0}},R=>console.error(R))}function g(){h(),C()}e.barPx=(R,T)=>{let q=e.stats.buckets||[],M=0;for(let ee of q)M=Math.max(M,(ee.error||0)+(ee.warn||0)+(ee.info||0));if(!M)return 0;let V=(R.error||0)+(R.warn||0)+(R.info||0);if(!V)return 0;let z=Math.round(V/M*60),ne=R[T]||0;return Math.round(ne/V*z)},e.bucketTitle=R=>`${new Date(R.hour).toLocaleTimeString([],{hour:"2-digit",minute:"2-digit"})} \xB7 ${R.error||0} err \xB7 ${R.warn||0} warn \xB7 ${R.info||0} info`,e.toggle=R=>{e.expanded[R._key]=!e.expanded[R._key]},e.setBucket=R=>{e.query.bucket=R},e.refreshNow=g,e.clearAll=()=>{confirm("Clear all captured errors?")&&t.delete("/api/admin/errors").then(g,R=>console.error(R))},e.exportCsv=()=>{let R=["ts","level","module","displayMessage","_status","_url","_repoId"],T=[R.join(",")];for(let z of e.visible)T.push(R.map(ne=>{let ee=z[ne]==null?"":String(z[ne]);return/[",\n]/.test(ee)?`"${ee.replace(/"/g,'""')}"`:ee}).join(","));let q=new Blob([T.join(`
`)],{type:"text/csv;charset=utf-8"}),M=URL.createObjectURL(q),V=document.createElement("a");V.href=M,V.download=`errors-${new Date().toISOString().slice(0,19)}.csv`,document.body.appendChild(V),V.click(),document.body.removeChild(V),URL.revokeObjectURL(M)};function b(R){e.copyHint=`${R} copied`,s.timeout(()=>{e.copyHint=""},1500)}e.copyJson=R=>{navigator.clipboard.writeText(R._detailJson).then(()=>b("JSON"))},e.copyCurl=R=>{if(!R._url)return;let q=`curl -X ${R._method||"GET"} '${window.location.origin}${R._url}'`;navigator.clipboard.writeText(q).then(()=>b("curl"))},g();let S=r(()=>{e.query.autoRefresh&&g()},15e3);e.on("dispose",()=>r.cancel(S)),e.watch("query.search",E),e.watch("query.bucket",E),e.watch("query.sort",E),e.watch("query.group",E)},El=function(e,t,n,r){if(e.Math=Math,e.watch("user.status",()=>{e.user==null&&n.url("/")}),e.user==null){n.url("/");return}e.data=null,e.loading=!0,e.error=null,e.performance=null,e.performanceMinutes=15,e.performanceTab="routes",e.performanceError=null,e.formatMs=function(g){return g==null?"":g<1e3?g+" ms":+(g/1e3).toFixed(1)+" s"},e.latencyBound=function(g,b){return g.count?g[b]==null?"> 60 s":"\u2264 "+e.formatMs(g[b]):"No samples"},e.firstByteBound=function(g){return e.latencyBound({count:g.firstByteCount,p95UpperMs:g.firstByteP95UpperMs},"p95UpperMs")},e.sampleAge=function(g){return Math.max(0,Math.round((Date.now()-g.sampledAt)/1e3))},e.peakRss=function(g){return(e.performance.runtimeSeries||[]).filter(function(b){return b.instance===g.instance}).reduce(function(b,S){return Math.max(b,S.rss)},g.memory.rss)},e.performanceWarning=function(g){return e.sampleAge(g)>45||g.droppedBatches||g.droppedMetrics||g.sockets?.closeWait>0||g.eventLoop.p95Ms>100||g.memoryLimitBytes&&g.memory.rss>g.memoryLimitBytes*.8};var s=!1;e.loadPerformance=function(){var g=e.performanceMinutes;e.performance?.windowMinutes&&e.performance.windowMinutes!==g&&(e.performance=null),!s&&(s=!0,t.get("/api/admin/performance",{params:{minutes:g}}).then(function(b){g===e.performanceMinutes&&(e.performance=b.data,e.performanceError=b.data.available?null:"Performance monitoring is unavailable. Request history may be incomplete.")},function(){g===e.performanceMinutes&&(e.performanceError="Could not refresh performance monitoring. Displayed samples may be stale.")}).finally(function(){s=!1,g!==e.performanceMinutes&&e.loadPerformance()}))};function i(g){if(g==null)return"\u2014";for(var b=["B","KB","MB","GB","TB"],S=0,R=g;R>=1024&&S<b.length-1;)R/=1024,S++;return R.toFixed(S>0?1:0)+" "+b[S]}e.humanBytes=i;function a(g){if(!g)return"\u2014";var b=Math.floor(g/86400),S=Math.floor(g%86400/3600),R=Math.floor(g%3600/60);return b>0?b+"d "+(S<10?"0":"")+S+"h":S>0?S+"h "+(R<10?"0":"")+R+"m":R+"m"}e.humanDuration=a;function u(g){return g==null?"\u2014":g>=1e6?(g/1e6).toFixed(1)+"M":g>=1e3?(g/1e3).toFixed(1)+"K":String(g)}e.humanNum=u,e.queueTotal=function(g){return g?(g.waiting||0)+(g.active||0)+(g.delayed||0)+(g.failed||0):0},e.statusCount=function(g){if(!e.data||!e.data.repos)return 0;for(var b=e.data.repos.statusBreakdown||[],S=0;S<b.length;S++)if(b[S]._id===g)return b[S].count;return 0},e.barPct=function(g){var b=e.data&&e.data.repos?e.data.repos.total:0;if(!b)return 0;var S=[g];g==="expired"&&S.push("expiring"),g==="removed"&&S.push("removing"),g==="preparing"&&S.push("download");var R=0;return S.forEach(function(T){R+=e.statusCount(T)}),Math.max(.4,R/b*100)},e.errPct=function(g){if(!e.data||!e.data.errors)return 0;var b=Math.max(e.data.errors.severity.error,e.data.errors.severity.warn,e.data.errors.severity.info,1);return e.data.errors.severity[g]/b*100};function m(g){var b=g||[];return b.map(function(S,R){var T=b[R-1]||{},q=Object.assign({},S);return q.dailyRepositories=R?Math.max(0,(S.nbRepositories||0)-(T.nbRepositories||0)):0,q.dailyUsers=R?Math.max(0,(S.nbUsers||0)-(T.nbUsers||0)):0,q.dailyPageViews=R?Math.max(0,(S.nbPageViews||0)-(T.nbPageViews||0)):0,q})}function k(g){var b=g&&g.length?g[g.length-1]:{};return{repositories:b.dailyRepositories||0,users:b.dailyUsers||0,pageViews:b.dailyPageViews||0}}var y={},E={};e.historyBarH=function(g,b){return!g||!y[b]?0:Math.max(1,Math.round(g[b]/y[b]*140))},e.historyRangeH=function(g,b){if(!g||y[b]==null)return 0;var S=y[b]-E[b];return S?Math.round(8+(g[b]-E[b])/S*132):70},e.historyRange=function(g){return{min:E[g],max:y[g]}},e.historyLabel=function(g){if(!g||!g.date)return"";var b=new Date(g.date);return b.getUTCMonth()+1+"/"+b.getUTCDate()};function h(){e.loadPerformance(),t.get("/api/admin/overview").then(function(g){g.data.history=m(g.data.history),g.data.daily={today:k(g.data.history)},e.data=g.data,e.loading=!1,e.error=null,y={},E={},(g.data.history||[]).forEach(function(b){["dailyPageViews","dailyRepositories","dailyUsers","nbUsers"].forEach(function(S){(!y[S]||b[S]>y[S])&&(y[S]=b[S]),(E[S]==null||b[S]<E[S])&&(E[S]=b[S])})})},function(g){e.loading=!1,e.error=g.data&&g.data.error||"Failed to load overview"})}h();var C=r(h,3e4);e.on("dispose",function(){r.can Line truncated
anonymous_github`)])])])])]),f(" Limitations "),o("h4",{class:"faq-cat"},[o("svg",{class:"ui-icon mr-2","aria-hidden":"true"},[o("use",{href:"#ui-icon-warning"})]),w("Limitations ")]),o("div",{class:"panel-group",id:"faq-limitations",role:"tablist","aria-multiselectable":"true"},[o("div",{class:"panel panel-default mb-3"},[o("div",{class:"panel-heading p-3",role:"tab",id:"headingLimitation"},[o("h3",{class:"panel-title"},[o("a",{class:"collapsed",role:"button","data-toggle":"collapse","data-parent":"#faq-limitations",href:"#limitation","aria-expanded":"false","aria-controls":"limitation"}," What are the limitations of Anonymous GitHub? ")])]),o("div",{id:"limitation",class:"panel-collapse collapse",role:"tabpanel","aria-labelledby":"headingLimitation"},[o("div",{class:"panel-body p-3"},[o("ul",null,[o("li",null," Anonymous GitHub only anonymizes text-based files. Binary files (compiled executables, archives, etc.) are served as-is without anonymization. "),o("li",null," Files larger than 8 MB are not supported. "),o("li",null," Static site generators (such as Jekyll) used with GitHub Pages are not fully supported, although Markdown files are converted to HTML. "),o("li",null," The anonymization of terms within source code may change the behavior of the program (e.g., if a replaced term appears in a string literal or identifier). ")])])])])]),f(" Privacy & Security "),o("h4",{class:"faq-cat"},[o("svg",{class:"ui-icon mr-2","aria-hidden":"true"},[o("use",{href:"#ui-icon-shield"})]),w("Privacy & Security ")]),o("div",{class:"panel-group",id:"faq-privacy",role:"tablist","aria-multiselectable":"true"},[o("div",{class:"panel panel-default mb-3"},[o("div",{class:"panel-heading p-3",role:"tab",id:"headingData"},[o("h3",{class:"panel-title"},[o("a",{class:"collapsed",role:"button","data-toggle":"collapse","data-parent":"#faq-privacy",href:"#data","aria-expanded":"false","aria-controls":"data"}," How is my data handled? ")])]),o("div",{id:"data",class:"panel-collapse collapse",role:"tabpanel","aria-labelledby":"headingData"},[o("div",{class:"panel-body p-3"},[o("p",null," Data stored on Anonymous GitHub is never shared or used for any purpose beyond providing the anonymization service. When a repository is removed or expires, only its configuration is retained \u2014 this makes it easy to restore the repository if needed and ensures that no future repository reuses the same ID. ")])])]),o("div",{class:"panel panel-default mb-3"},[o("div",{class:"panel-heading p-3",role:"tab",id:"headingPermissions"},[o("h3",{class:"panel-title"},[o("a",{class:"collapsed",role:"button","data-toggle":"collapse","data-parent":"#faq-privacy",href:"#permissions","aria-expanded":"false","aria-controls":"permissions"}," Why does GitHub say Anonymous GitHub asks for write access? ")])]),o("div",{id:"permissions",class:"panel-collapse collapse",role:"tabpanel","aria-labelledby":"headingPermissions"},[o("div",{class:"panel-body p-3"},[o("p",null,[w("The GitHub App has read-only access to the repositories you choose. You can change repository access on your "),o("a",{href:"/connections"},"GitHub connections"),w(" page.")]),o("p",null,"Legacy GitHub OAuth includes write access to private repositories because GitHub's OAuth scopes do not offer a read-only option for them. Existing OAuth anonymizations and gists can still use this connection."),o("p",null,"Anonymous GitHub reads repository content. It never pushes, modifies, or deletes files on GitHub.")])])]),o("div",{class:"panel panel-default mb-3"},[o("div",{class:"panel-heading p-3",role:"tab",id:"headingViewer"},[o("h3",{class:"panel-title"},[o("a",{class:"collapsed",role:"button","data-toggle":"collapse","data-parent":"#faq-privacy",href:"#viewer","aria-expanded":"false","aria-controls":"viewer"}," Can repository owners see who viewed their anonymized repository? ")])]),o("div",{id:"viewer",class:"panel-collapse collapse",role:"tabpanel","aria-labelledby":"headingViewer"},[o("div",{class:"panel-body p-3"},[o("p",null," No. Only the total number of views is tracked as an incremental counter. It is not possible for the repository owner or for Anonymous GitHub to identify individual viewers. Reviewer anonymity is fully preserved. ")])])])]),f(" Self-Hosting "),o("h4",{class:"faq-cat"},[o("svg",{class:"ui-icon mr-2","aria-hidden":"true"},[o("use",{href:"#project-icon-storage"})]),w("Self-Hosting ")]),o("div",{class:"panel-group",id:"faq-hosting",role:"tablist","aria-multiselectable":"true"},[o("div",{class:"panel panel-default mb-3"},[o("div",{class:"panel-heading p-3",role:"tab",id:"headingDeploy"},[o("h3",{class:"panel-title"},[o("a",{class:"collapsed",role:"button","data-toggle":"collapse","data-parent":"#faq-hosting",href:"#deploy","aria-expanded":"false","aria-controls":"deploy"}," Can I deploy my own instance? ")])]),o("div",{id:"deploy",class:"panel-collapse collapse",role:"tabpanel","aria-labelledby":"headingDeploy"},[o("div",{class:"panel-body p-3"},[o("p",null,[w(" Yes. Anonymous Git Line truncated
`+JSON.stringify(t):""}`},[Be.NAVIGATION_GUARD_REDIRECT]({from:e,to:t}){return`Redirected from "${e.fullPath}" to "${ep(t)}" via a navigation guard.`},[Be.NAVIGATION_ABORTED]({from:e,to:t}){return`Navigation aborted from "${e.fullPath}" to "${t.fullPath}" via a navigation guard.`},[Be.NAVIGATION_CANCELLED]({from:e,to:t}){return`Navigation cancelled from "${e.fullPath}" to "${t.fullPath}" with a new navigation.`},[Be.NAVIGATION_DUPLICATED]({from:e,to:t}){return`Avoided redundant navigation to current location: "${e.fullPath}".`}};function fo(e,t){return Te(new Error,{type:e,[Oa]:!0},t)}function Ht(e,t){return e instanceof Error&&Oa in e&&(t==null||!!(e.type&t))}var xc=["params","query","hash"];function ep(e){if(typeof e=="string")return e;if(e.path!=null)return e.path;let t={};for(let n of xc)n in e&&(t[n]=e[n]);return JSON.stringify(t,null,2)}function Ia(e){let t={};if(e===""||e==="?")return t;let n=(e[0]==="?"?e.slice(1):e).split("&");for(let r=0;r<n.length;++r){let s=n[r].replace(ga," "),i=s.indexOf("="),a=Oo(i<0?s:s.slice(0,i)),u=i<0?null:Oo(s.slice(i+1));if(a in t){let m=t[a];wt(m)||(m=t[a]=[m]),m.push(u)}else t[a]=u}return t}function ir(e){let t="";for(let n in e){let r=e[n];if(n=jc(n),r==null){r!==void 0&&(t+=(t.length?"&":"")+n);continue}(wt(r)?r.map(s=>s&&Zs(s)):[r&&Zs(r)]).forEach(s=>{s!==void 0&&(t+=(t.length?"&":"")+n,s!=null&&(t+="="+s))})}return t}function Pa(e){let t={};for(let n in e){let r=e[n];r!==void 0&&(t[n]=wt(r)?r.map(s=>s==null?null:""+s):r==null?r:""+r)}return t}var Va=Symbol(""),ar=Symbol(""),os=Symbol(""),ns=Symbol(""),ss=Symbol("");function Po(){let e=[];function t(r){return e.push(r),()=>{let s=e.indexOf(r);s>-1&&e.splice(s,1)}}function n(){e=[]}return{add:t,list:()=>e.slice(),reset:n}}function Qt(e,t,n,r,s,i=a=>a()){let a=r&&(r.enterCallbacks[s]=r.enterCallbacks[s]||[]);return()=>new Promise((u,m)=>{let w=h=>{h===!1?m(fo(Be.NAVIGATION_ABORTED,{from:n,to:t})):h instanceof Error?m(h):Zc(h)?m(fo(Be.NAVIGATION_GUARD_REDIRECT,{from:t,to:h})):(a&&r.enterCallbacks[s]===a&&typeof h=="function"&&a.push(h),u())},g=i(()=>e.call(r&&r.instances[s],t,n,w)),k=Promise.resolve(g);e.length<3&&(k=k.then(w)),k.catch(h=>m(h))})}function rs(e,t,n,r,s=i=>i()){let i=[];for(let a of e)for(let u in a.components){let m=a.components[u];if(!(t!=="beforeRouteEnter"&&!a.instances[u]))if(va(m)){let w=(m.__vccOpts||m)[t];w&&i.push(Qt(w,n,r,a,u,s))}else{let w=m();i.push(()=>w.then(g=>{if(!g)throw new Error(`Couldn't resolve component "${u}" at "${a.path}"`);let k=Vc(g)?g.default:g;a.mods[u]=g,a.components[u]=k;let h=(k.__vccOpts||k)[t];return h&&Qt(h,n,r,a,u,s)()}))}}return i}function qa(e,t){let n=[],r=[],s=[],i=Math.max(t.matched.length,e.matched.length);for(let a=0;a<i;a++){let u=t.matched[a];u&&(e.matched.find(w=>po(w,u))?r.push(u):n.push(u));let m=e.matched[a];m&&(t.matched.find(w=>po(w,m))||s.push(m))}return[n,r,s]}var tp=()=>location.protocol+"//"+location.host;function Wa(e,t){let{pathname:n,search:r,hash:s}=t,i=e.indexOf("#");if(i>-1){let a=s.includes(e.slice(i))?e.slice(i).length:1,u=s.slice(a);return u[0]!=="/"&&(u="/"+u),or(u,"")}return or(n,e)+r+s}function op(e,t,n,r){let s=[],i=[],a=null,u=({state:h})=>{let E=Wa(e,location),S=n.value,v=t.value,_=0;if(h){if(n.value=E,t.value=h,a&&a===S){a=null;return}_=v?h.position-v.position:0}else r(E);s.forEach(N=>{N(n.value,S,{delta:_,type:es.pop,direction:_?_>0?ts.forward:ts.back:ts.unknown})})};function m(){a=n.value}function w(h){s.push(h);let E=()=>{let S=s.indexOf(h);S>-1&&s.splice(S,1)};return i.push(E),E}function g(){if(document.visibilityState==="hidden"){let{history:h}=window;if(!h.state)return;h.replaceState(Te({},h.state,{scroll:vn()}),"")}}function k(){for(let h of i)h();i=[],window.removeEventListener("popstate",u),window.removeEventListener("pagehide",g),document.removeEventListener("visibilitychange",g)}return window.addEventListener("popstate",u),window.addEventListener("pagehide",g),document.addEventListener("visibilitychange",g),{pauseListeners:m,listen:w,destroy:k}}function Ma(e,t,n,r=!1,s=!1){return{back:e,current:t,forward:n,replaced:r,position:window.history.length,scroll:s?vn():null}}function np(e){let{history:t,location:n}=window,r={value:Wa(e,n)},s={value:t.state};s.value||i(r.value,{back:null,current:r.value,forward:null,position:t.length-1,replaced:!0,scroll:null},!0);function i(m,w,g){let k=e.indexOf("#"),h=k>-1?(n.host&&document.querySelector("base")?e:e.slice(k))+m:tp()+e+m;try{t[g?"replaceState":"pushState"](w,"",h),s.value=w}catch(E){console.error(E),n[g?"replace":"assign"](h)}}function a(m,w){i(m,Te({},t.state,Ma(s.value.back,m,s.value.forward,!0),w,{position:s.value.position}),!0),r.value=m}function u(m,w){let g=Te({},s.value,t.state,{forward:m,scroll:vn()});i(g.current,g,!0),i(m,Te({},Ma(r.value,m,null),{position:g.position+1},w),!1),r.value=m}return{location:r,state:s,push:u,replace:a}}function Ka(e){e=Na(e);let t=np(e),n=op(e,t.state,t.location,t.replace);function r(i,a=!0){a||n.pauseListeners(),history.go(i)}let s=Te({locatLine truncated
`)),e.options=Object.assign({},e.options,g.options),a.restore()})}u();let m=null;e.saveDefault=w=>{if(w&&w.preventDefault&&w.preventDefault(),e.saving)return;let g=a.checkpoint(),k={terms:e.terms.split(`
`).map(h=>h.trim()).filter(h=>h.length>0),options:e.options};e.saving=!0,e.error=null,t.post("/api/user/default",k).then(()=>{a.accept(g),e.saving=!1,e.message=e.unsavedChanges?null:"Saved",m&&r.cancel(m),m=r(()=>{e.message=null},2500)},h=>{e.saving=!1;let E=h&&h.data&&h.data.error,S="Unable to save your defaults. Please try again.";Promise.all([n("ERRORS."+E),n("ERRORS.unknown_error")]).then(([v,_])=>{e.error=E&&v!==_?v:S},()=>{e.error=S})})},e.deleteAccount=()=>{confirm("Delete your account? All your anonymized repositories, gists, and pull requests will be removed, and your personal data will be erased. This cannot be undone.")&&(e.deletingAccount=!0,t.delete("/api/user").then(()=>{window.location.href="/"},()=>{e.deletingAccount=!1,e.deleteError="Unable to delete the account. Please try again."}))}},il=function(e,t,n){e.repoId=null,e.repoUrl=null,e.claim=()=>{t.post("/api/repo/claim",{repoId:e.repoId,repoUrl:e.repoUrl}).then(r=>{n.url("/dashboard")},r=>{e.error=r.data,e.claimForm.repoUrl.setValidity("not_found",!1),e.claimForm.repoId.setValidity("not_found",!1)})}},al=function(e,t,n,r,s){e.user&&!e.user.status&&n.url("/dashboard"),e.watch("user.status",()=>{e.user&&!e.user.status&&n.url("/dashboard")}),e.features=[{key:"anonymize",num:"01",eyebrow:"Anonymize",title:"Double-anonymous,",accent:"your rules.",text:"Choose what reviewers may see: links, images, PDFs, notebooks, GitHub Pages. Add your own terms, with regex if you need it, and pick the expiration date.",cta:"Start an anonymization",href:"/anonymize",url:"anonymous.4open.science/anonymize",img:"/imgs/anonymize.png",alt:"The anonymize form: source repository, terms to redact, options and a live README preview"},{key:"review",num:"02",eyebrow:"Review",title:"Reviewers browse",accent:"the real thing.",text:"Highlighted source code, rendered PDFs, images, and notebooks, in a familiar file explorer. GitHub Pages is also supported.",cta:"Open the example",href:"https://anonymous.4open.science/r/840c8c57-3c32-451e-bf12-0e20be300389/",target:"_self",url:"anonymous.4open.science/r/840c8c57-\u2026",img:"/imgs/explorer.png",alt:"The repository explorer with a file tree and a rendered README"},{key:"manage",num:"03",eyebrow:"Manage",title:"One dashboard,",accent:"until the decision.",text:"Monitor views, edit configuration, remove or update your repository. Program chairs can group submissions under a conference with one shared expiry.",cta:"Open the dashboard",href:"/dashboard",needsUser:!0,url:"anonymous.4open.science/dashboard",img:"/imgs/dashboard.png",alt:"The dashboard listing anonymized repositories with status, views and expiry"}],e.feature=e.features[0].key,e.selectFeature=function(u){e.feature=u},e.featureHref=function(u){return u.needsUser&&!e.user?"/signin":u.href},e.featureTarget=function(u){return u.needsUser&&!e.user?"_self":u.target||void 0},e.featureKeydown=function(u,m){let w={ArrowDown:1,ArrowRight:1,ArrowUp:-1,ArrowLeft:-1,Home:"first",End:"last"}[u.key];if(w===void 0)return;u.preventDefault();let g=e.features.length,k=w==="first"?0:w==="last"?g-1:(m+w+g)%g;e.feature=e.features[k].key,s(()=>{let h=r.document.getElementById("feature-tab-"+e.feature);h&&h.focus()})},e.cards=[{key:"repositories",total:0,label:"repositories anonymized"},{key:"users",total:0,label:"researchers"},{key:"pageViews",total:0,label:"page views"},{key:"pullRequests",total:0,label:"pull requests"}];function i(){t.get("/api/stat/").then(u=>{e.stat=u.data,e.cards[0].total=u.data.nbRepositories,e.cards[1].total=u.data.nbUsers,e.cards[2].total=u.data.nbPageViews,e.cards[3].total=u.data.nbPullRequests})}i();function a(u){let m={series:u,bars:[],viewW:100,deltaToday:0,pctChange:0,pctAbs:0,isUp:!0};if(!u||u.length<2)return m;let w=new Array(u.length-1);for(let v=1;v<u.length;v++)w[v-1]=u[v]-u[v-1];let g=w.length,k=Math.max.apply(null,w),h=Math.min.apply(null,w),E=Math.min(0,h),S=k-E||1;m.viewW=g*2,m.bars=new Array(g);for(let v=0;v<g;v++){let _=(w[v]-E)/S,N=Math.max(1.5,_*34);m.bars[v]={x:(v*2+.25).toFixed(2),y:(36-N).toFixed(2),w:"1.5",h:N.toFixed(2)}}if(m.deltaToday=w[g-1],g>=2){let v=w[g-2];v&&(m.pctChange=(m.deltaToday-v)/v*100)}return m.pctAbs=Math.round(Math.abs(m.pctChange)),m.isUp=m.pctChange>=0,m}e.history={repositories:a([]),users:a([]),pageViews:a([]),pullRequests:a([])},t.get("/api/stat/history?days=60").then(u=>{let m=u.data||[];e.history={repositories:a(m.map(w=>w.nbRepositories||0)),users:a(m.map(w=>w.nbUsers||0)),pageViews:a(m.map(w=>w.nbPageViews||0)),pullRequests:a(m.map(w=>w.nbPullRequests||0))}})},ll=function(e,t,n,r,s,i){let a=ut();e.on("routeLeave",function(){$('[data-toggle="tooltip"]').tooltip("dispose")}),e.watch("user.status",()=>{e.user==null&&n.url("/")}),e.user==null&&n.url("/"),a.timeout(()=>{$('[data-toggle="tooltip"]').tooltip()},250),e.quotaExpanded=!1,e.closeQuota=()=>{e.quotaExpanded=!1,ot(()=>document.querySelector(".dashboard-quota > summary")?.focus())},Zt()(document,"click",C=>{e.quotaExpanded&&!C.target.closest?.(".dashboard-quota")& Line truncated
`)),e.options=Object.assign({},e.options,D.options),e.options.expirationDate=D.options&&D.options.expirationDate?new Date(D.options.expirationDate):T(),!r.repoId&&!r.pullRequestId&&!r.gistId&&!e.anonymize?.terms?.dirty&&(e.terms=e.defaultTerms),p&&p()})}function C(p,y,D){e.anonymize&&e.anonymize[p]&&e.anonymize[p].setValidity(y,D)}function P(p){try{let y=parseGithubUrl(p);if(y&&y.owner&&y.repo)return y.owner+"/"+y.repo}catch{}return null}e.accessRepairAllowed=!1,e.repairingGitHubAccess=!1,e.accessRepairMessage="",e.repairGitHubAccess=async()=>{if(!e.accessRepairAllowed||e.repairingGitHubAccess||!e.githubConnections?.appEnabled||!r.repoId||e.role!=="owner"||e.githubConnection!=="github-app"||P(e.sourceUrl)!==e._originalFullName)return;e.repairingGitHubAccess=!0,e.accessRepairMessage="";let p=h();try{let y=await t.get("/github/connections");if(e.githubConnections=y.data,!y.data.appEnabled)return;let D=await t.post("/github/connections/migrate",{type:"repository",id:r.repoId,connection:"github-app"},{headers:{"X-CSRF-Token":y.data.csrf}});if(!k(p))return;e.error=null,D.data.fullName&&(e._originalFullName=D.data.fullName,e.source.fullName=D.data.fullName,e.sourceUrl="https://github.com/"+D.data.fullName),e.accessRepairMessage="GitHub access repaired. Your anonymous URL and published commit are unchanged.",await e.urlSelected(!0)}catch{k(p)&&(e.accessRepairMessage="Unable to repair GitHub access. Check the App's repository permissions in Manage access, then retry. Your published snapshot has not been replaced.")}finally{e.repairingGitHubAccess=!1,E(p)}},e.reconnectingSource=!1,e.reconnectRepositoryId=void 0,e.reconnectSource=async()=>{if(confirm("Reconnect to the repository currently at this GitHub URL? Review its branch and commit before saving. Saving will replace the published snapshot.")){e.reconnectingSource=!0,e.reconnectRepositoryId=void 0,e.source.commit="",e.source.branch="",e.readme="",e.branches=[];try{await K()}catch{}}};function G(){return e.reconnectingSource||!e.isUpdate||!e._originalRepositoryID?void 0:P(e.sourceUrl)===e._originalFullName?e._originalRepositoryID:void 0}async function J(){e._preservingDraft=!0;try{await e.urlSelected(!0)}finally{e._preservingDraft=!1}}async function X(){let p;try{p=JSON.parse(sessionStorage.getItem("github-access-draft")||"null")}catch{return!1}if(!p||p.path!==s.path()||Date.now()-p.savedAt>=30*6e4)return _.restore()?(e.sourceUrl&&await J(),!0):!1;_.restore();for(let y of["sourceUrl","terms","repoId","pullRequestId","gistId","source","options","conference","githubConnection"])Object.prototype.hasOwnProperty.call(p.draft,y)&&(e[y]=p.draft[y]);return e.options.expirationDate&&(e.options.expirationDate=new Date(e.options.expirationDate)),await J(),sessionStorage.removeItem("github-access-draft"),!0}ee(()=>{!r.repoId&&!r.pullRequestId&&!r.gistId&&a(X,0),r.repoId&&r.repoId!=""&&(e.isUpdate=!0,e.detectedType="repo",e.repoId=r.repoId,t.get("/api/repo/"+e.repoId).then(async p=>{e.accessRepairAllowed=!["removed","archived","preparing","queue","download","removing","expiring"].includes(p.data.status),e.githubConnection=p.data.connection||"oauth",e.sourceUrl="https://github.com/"+p.data.source.fullName,e._originalFullName=p.data.source.fullName,e.terms=p.data.options.terms.filter(y=>y).join(`
`),e.source=p.data.source,e.role=p.data.role||"owner",e.coauthors=p.data.coauthors||[],e._originalBranch=p.data.source.branch,e.options=Object.assign({},e.options,p.data.options),e.conference=p.data.conference,e._originalConference=p.data.conference,e.repositoryID=p.data.source.repositoryID,e._originalRepositoryID=p.data.source.repositoryID,p.data.options.expirationDate&&(e.options.expirationDate=new Date(p.data.options.expirationDate)),S=v(parseGithubUrl(e.sourceUrl)),!await X()&&(await Promise.all([K(),se()]),_e())},()=>{s.url("/404")})),r.pullRequestId&&r.pullRequestId!=""&&(e.isUpdate=!0,e.detectedType="pr",e.pullRequestId=r.pullRequestId,t.get("/api/pr/"+e.pullRequestId).then(async p=>{if(e.githubConnection=p.data.connection||"oauth",e.sourceUrl="https://github.com/"+p.data.source.repositoryFullName+"/pull/"+p.data.source.pullRequestId,e.terms=p.data.options.terms.filter(y=>y).join(`
`),e.source=p.data.source,e.options=Object.assign({},e.options,p.data.options),e.conference=p.data.conference,e._originalConference=p.data.conference,p.data.options.expirationDate&&(e.options.expirationDate=new Date(p.data.options.expirationDate)),S=v(parseGithubUrl(e.sourceUrl)),!await X())try{await qe()}catch{}},()=>{s.url("/404")})),r.gistId&&r.gistId!=""&&(e.isUpdate=!0,e.detectedType="gist",e.gistId=r.gistId,t.get("/api/gist/"+e.gistId).then(async p=>{e.sourceUrl="https://gist.github.com/"+p.data.source.gistId,e.terms=p.data.options.terms.filter(y=>y).join(`
`),e.source=p.data.source,e.options=Object.assign({},e.options,p.data.options),e.conference=p.data.conference,e._originalConference=p.data.conference,p.data.options.expirationDate&&(e.options.expirationDate=new Date(p.data.options.expirationDate)),S=v(parseGithubUrl(e.sourceUrl)),!await X()&&await ze()},()=>{s.url("/404")}))}),t.get("/github/connections").then(p=>{e.githubConnections=p.data,!r.repoId&&!r.pullRequestId&&!r.gistId&&e.githubConnection===void 0&&p.data.appEnabled&&p.data.appConnected&&(e.githubConnection="github-app",e.sourceUrl&&e.workspaceOpen&&J())}).catch(()=>{}),e.urlSelected=async(p=!1)=>{let y=++m;e.previousConfigurations=[],e.previousConfigurationId="";let D;try{D=parseGithubUrl(e.sourceUrl.trim())}catch{e.sourceLoading=!1,re.cancel(),Se.cancel(),F.cancel(),e.previewStatus="idle",e.sourceUrlError=e.sourceUrl?"Please provide a valid GitHub URL.":"",C("sourceUrl","github",!1);return}let O=v(D),I=p||S===O;S=O,e.sourceUrlError="",C("sourceUrl","github",!0);let U=document.activeElement?.id==="sourceUrl-landing";e.workspaceOpen=!0,U&&a(()=>document.getElementById("sourceUrl")?.focus(),0),e.reconnectingSource=!1,e.reconnectRepositoryId=void 0,!I&&!e.isUpdate&&(e.repoId="",e.pullRequestId="",e.gistId=""),e.details=null,e.branches=[],I||(e.source={type:"GitHubStream",branch:"",commit:""}),re.cancel(),Se.cancel(),F.cancel(),Me.clear(),Ue.clear(),e.previewStatus="idle",e.anonymize_readme="",e.readme="",e.html_readme="",e.detectedType=null,e.gistOAuthRequired=!1,e.gistAppRequired=!1,e._preservingDraft=I;try{D.gistId&&!D.repo?(e.detectedType="gist",e.source={gistId:D.gistId},await ze()):D.pullRequestId?(e.detectedType="pr",e.source={repositoryFullName:D.owner+"/"+D.repo,pullRequestId:D.pullRequestId},await qe()):(e.detectedType="repo",ne(),await Promise.all([K(),se()]),_e())}catch{}finally{y===m&&(e._preservingDraft=!1)}$('[data-toggle="tooltip"]').tooltip()},$('[data-toggle="tooltip"]').tooltip(),e.watch("source.branch",async()=>{if(e.detectedType!=="repo")return;let p=e.branches.filter(D=>D.name==e.source.branch)[0];if(!p)return;!(!e.reconnectingSource&&e.isUpdate&&e._originalBranch===e.source.branch&&e.source.commit)&&!(e._preservingDraft&&e.source.commit)&&(e.source.commit=p.commit),e.readme=p.readme,await se(),_e()}),e.getBranches=async p=>{let y=parseGithubUrl(e.sourceUrl),D=h();try{let O=await t.get(`/api/repo/${y.owner}/${y.repo}/branches`,{params:{reconnect:e.reconnectingSource?"1":void 0,anonymizedRepoId:e.isUpdate&&r.repoId&&P(e.sourceUrl)===e._originalFullName?r.repoId:void 0,connection:e.githubConnection,force:p===!0||e.reconnectingSource?"1":"0",repositoryID:G()}});if(!k(D))return;e.branches=O.data,e.sourceUnreachable=!1,e.source.branch||(e.source.branch=e.details.defaultBranch);let I=e.branches.filter(U=>U.name==e.source.branch);I.length>0&&(!(!e.reconnectingSource&&!p&&e.isUpdate&&!e.options.update&&e._originalBranch===e.source.branch&&e.source.commit)&&!(e._preservingDraft&&e.source.commit)&&(e.source.commit=I[0].commit),e.readme=I[0].readme,await se(p))}catch(O){if(!k(D))return;e.branches=[],e.sourceUnreachable=O&&(O.status===404||O.data&&O.data.error==="repo_not_found");let I=O&&O.data&&O.data.error||(O&&O.status===404?"repo_not_found":"unknown_error");i("ERRORS."+I).then(U=>{e.toasts=e.toasts||[],e.addToast({title:"Error",date:new Date,body:U}),e.error=U},console.error),typeof C=="function"&&C("sourceUrl","missing",!1)}finally{E(D)}};async function K(){let p=parseGithubUrl(e.sourceUrl),y=h();try{te();let D=await t.get(`/api/repo/${p.owner}/${p.repo}/`,{params:{reconnect:e.reconnectingSource?"1":void 0,anonymizedRepoId:e.isUpdate&&r.repoId&&P(e.sourceUrl)===e._originalFullName?r.repoId:void 0,connection:e.githubConnection,repositoryID:G(),force:"1"}});if(!k(y))return;e.details=D.data,e.reconnectingSource&&(e.reconnectRepositoryId=e.details.externalId),e.details&&e.details.id&&(e.repositoryID=e.details.id),e.repoId||(e.repoId=e.details.repo+"-"+generateRandomId(4)),await e.getBranches(e.reconnectingSource)}catch(D){if(!k(y))return;throw D.data&&(i("ERRORS."+D.data.error).then(O=>{e.addToast({title:"Error",date:new Date,body:O}),e.error=O},console.error),Ee(D.data.error)),C("sourceUrl","missing",!1),D}finally{E(y)}}async function se(p){if(e.readme&&!p)return e.readme;let y=parseGithubUrl(e.sourceUrl),D=h();try{let O=await t.get(`/api/repo/${y.owner}/${y.repo}/readme`,{params:{reconnect:e.reconnectingSource?"1":void 0,anonymizedRepoId:e.isUpdate&&r.repoId&&P(e.sourceUrl)===e._originalFullName?r.repoId:void 0,connection:e.githubConnection,force:p===!0?"1":"0",branch:e.source.branch,repositoryID:G()}});k(D,!0)&&(e.readme=O.data)}catch{k(D,!0)&&(e.readme="")}finally{E(D)}}function ce(){let p={terms:e.terms?e.terms.split(`
`):[],image:!!e.options.image,link:!!e.options.link,repoId:e.repoId};e.source&&e.source.branch&&(p.branchName=e.source.branch);try{let y=parseGithubUrl(e.sourceUrl);p.repoName=`${y.owner}/${y.repo}`}catch{}return p}function ue(p,y,D){let O=null,I=0,U=()=>{if(e.detectedType!==p)return;O&&a.cancel(O);let L=++I;e.previewStatus="loading",O=a(()=>{O=null;let H=y();if(!H){e.previewStatus="idle";return}t.post("/api/anonymize-preview",H).then(Y=>{L===I&&(D(Y.data,H),e.previewStatus="ready")}).catch(()=>{L===I&&(e.previewStatus="error")})},200)};return U.cancel=()=>{I++,O&&a.cancel(O),O=null},e.on("dispose",U.cancel),U}e.retryPreview=()=>{e.detectedType==="repo"&&re(),e.detectedType==="pr"&&Se(),e.detectedType==="gist"&&F()};let re=ue("repo",()=>e.readme?{content:e.readme,options:ce()}:null,p=>{if(typeof p?.content!="string")throw Error("Invalid preview response");e.anonymize_readme=p.content;let y="";try{let O=parseGithubUrl(e.sourceUrl),I=e.source.branch||e.details&&e.details.defaultBranch||"main";y=`https://github.com/${O.owner}/${O.repo}/raw/${I}/`}catch{}let D=renderMD(e.anonymize_readme,y);e.html_readme=D,a(Prism.highlightAll,150)});function _e(){!e.anonymize||!e.anonymize.terms||(e.termsRegexWarning=!!e.terms&&!!e.terms.match(/[-[\]{}()*+?.,\\^$|#]/g),re())}async function qe(){let p=parseGithubUrl(e.sourceUrl),y=h();try{te();let D=await t.get(`/api/pr/${p.owner}/${p.repo}/${p.pullRequestId}`,{params:{connection:e.githubConnection}});if(!k(y))return;e.details=D.data,e.pullRequestId||(e.pullRequestId=p.repo+"-PR"+p.pullRequestId+"-"+generateRandomId(4))}catch(D){if(!k(y))return;throw D.data&&(i("ERRORS."+D.data.error).then(O=>{e.addToast({title:"Error",date:new Date,body:O}),e.error=O},console.error),Ee(D.data.error)),C("sourceUrl","missing",!1),D}finally{E(y)}}let Me=Fe(new Map);function Pe(){let p=new Set,y=e.details&&e.details.pullRequest;if(!y)return p;typeof y.title=="string"&&p.add(y.title),typeof y.body=="string"&&p.add(y.body),typeof y.diff=="string"&&p.add(y.diff);let D=y.comments||[];for(let O of D)typeof O.author=="string"&&p.add(O.author),typeof O.body=="string"&&p.add(O.body);return p}let Se=ue("pr",()=>{let p=Pe(),y=Array.from(p);return y.length===0?null:{contents:y,options:ce()}},(p,y)=>{if(!Array.isArray(p?.contents)||p.contents.length!==y.contents.length||p.contents.some(I=>typeof I!="string"))throw Error("Invalid preview response");let D=y.contents,O=new Map;for(let I=0;I<D.length&&I<p.contents.length;I++)O.set(D[I],p.contents[I]);Me.clear(),O.forEach((I,U)=>Me.set(U,I))});e.anonymizePrContent=function(p){return p&&(Me.has(p)?Me.get(p):"")};async function ze(){let p=parseGithubUrl(e.sourceUrl),y=h();try{te(),e.gistOAuthRequired=!1,e.gistAppRequired=!1;let D=await t.get(`/api/gist/source/${p.gistId}`);if(!k(y))return;e.details=D.data,e.gistId||(e.gistId="gist-"+p.gistId.substring(0,6)+"-"+generateRandomId(4))}catch(D){if(!k(y))return;if(D.data?.error==="github_app_reconnect_required"){e.gistAppRequired=!0,C("sourceUrl","missing",!1);return}if(D.data?.error==="github_oauth_required"){e.gistOAuthRequired=!0,C("sourceUrl","missing",!1);return}throw D.data&&(i("ERRORS."+D.data.error).then(O=>{e.addToast({title:"Error",date:new Date,body:O}),e.error=O},console.error),Ee(D.data.error)),C("sourceUrl","missing",!1),D}finally{E(y)}}let Ue=Fe(new Map);function ke(){let p=new Set,y=e.details&&e.details.gist;if(!y)return p;typeof y.description=="string"&&p.add(y.description),typeof y.ownerLogin=="string"&&p.add(y.ownerLogin);let D=y.files||[];for(let I of D)typeof I.filename=="string"&&p.add(I.filename),typeof I.content=="string"&&p.add(I.content);let O=y.comments||[];for(let I of O)typeof I.author=="string"&&p.add(I.author),typeof I.body=="string"&&p.add(I.body);return p}let F=ue("gist",()=>{let p=ke(),y=Array.from(p);return y.length===0?null:{contents:y,options:ce()}},(p,y)=>{if(!Array.isArray(p?.contents)||p.contents.length!==y.contents.length||p.contents.some(I=>typeof I!="string"))throw Error("Invalid preview response");let D=y.contents,O=new Map;for(let I=0;I<D.length&&I<p.contents.length;I++)O.set(D[I],p.contents[I]);Ue.clear(),O.forEach((I,U)=>Ue.set(U,I)),Z()});e.anonymizeGistContent=function(p){return p&&(Ue.has(p)?Ue.get(p):"")},e.previewGistFiles=[];function Z(){let p=e.details&&e.details.gist&&e.details.gist.files||[];e.previewGistFiles=p.map(y=>({filename:e.anonymizeGistContent(y.filename),content:e.anonymizeGistContent(y.content),language:y.language}))}e.watch("details.gist.files",Z,!0),e.watch("terms",Z);function Q(){let p=e.conference;if(e.conference_data=null,!p)return;let y=e.isUpdate&&p===e._originalConference;t.get("/api/conferences/"+p).then(D=>{e.conference===p&&(e.conference_data=D.data,e.conference_data.startDate=new Date(e.conference_data.startDate),e.conference_data.endDate=new Date(e.conference_data.endDate),!y&&(e.options.expirationDate=new Date(e.conference_data.endDate),e.options.expirationMode="remove",e.options.update=e.conference_data.options.update,e.options.image=e.conf Line truncated
`).filter(U=>U),connection:e.githubConnection,fullName:`${D.owner}/${D.repo}`,repository:e.sourceUrl,reconnectRepositoryId:e.reconnectRepositoryId,options:e.options,source:e.source,conference:e.conference};e.details&&(O.options.pageSource=e.details.pageSource),te();let I=e.isUpdate?"/api/repo/"+e.repoId:"/api/repo/";t.post(I,O,{headers:{"Content-Type":"application/json"}}).then(()=>{_.clear(),window.location.href="/status/"+e.repoId},U=>{U.data&&(i("ERRORS."+U.data.error).then(L=>{e.error=L},console.error),Ee(U.data.error))}).finally(()=>{y.disabled=!1})},e.anonymizeGist=p=>{if(e.sourceLoading){e.error="Wait for the source to finish loading before saving.";return}if(ye())return;if(e.anonymize?.invalid||e.sourceUrlError){e.error="Please correct the highlighted fields before saving.";return}let y=p.currentTarget||p.target.closest?.("button")||p.target;y.disabled=!0;let D=parseGithubUrl(e.sourceUrl),O={gistId:e.gistId,terms:e.terms.trim().split(`
`).filter(U=>U),source:{gistId:D.gistId},options:e.options,conference:e.conference};te();let I=e.isUpdate?"/api/gist/"+e.gistId:"/api/gist/";t.post(I,O,{headers:{"Content-Type":"application/json"}}).then(()=>{_.clear(),window.location.href="/gist/"+e.gistId},U=>{U.data&&(i("ERRORS."+U.data.error).then(L=>{e.error=L},console.error),Ee(U.data.error))}).finally(()=>{y.disabled=!1})},e.anonymizePullRequest=p=>{if(e.sourceLoading){e.error="Wait for the source to finish loading before saving.";return}if(ye())return;if(e.anonymize?.invalid||e.sourceUrlError){e.error="Please correct the highlighted fields before saving.";return}let y=p.currentTarget||p.target.closest?.("button")||p.target;y.disabled=!0;let D=parseGithubUrl(e.sourceUrl),O={pullRequestId:e.pullRequestId,connection:e.githubConnection,terms:e.terms.trim().split(`
`).filter(U=>U),source:{repositoryFullName:`${D.owner}/${D.repo}`,pullRequestId:D.pullRequestId},options:e.options,conference:e.conference};te();let I=e.isUpdate?"/api/pr/"+e.pullRequestId:"/api/pr/";t.post(I,O,{headers:{"Content-Type":"application/json"}}).then(()=>{_.clear(),window.location.href="/pr/"+e.pullRequestId},U=>{U.data&&(i("ERRORS."+U.data.error).then(L=>{e.error=L},console.error),Ee(U.data.error))}).finally(()=>{y.disabled=!1})},e.watch("conference",()=>{Q()}),e.watch("terms",()=>{e.detectedType==="repo"&&_e(),e.detectedType==="pr"&&Se(),e.detectedType==="gist"&&F()}),e.watch("options.image",()=>{e.detectedType==="repo"&&_e(),e.detectedType==="pr"&&Se(),e.detectedType==="gist"&&F()}),e.watch("options.link",()=>{e.detectedType==="repo"&&_e(),e.detectedType==="pr"&&Se(),e.detectedType==="gist"&&F()}),e.watch("details",()=>{e.detectedType==="pr"&&Se(),e.detectedType==="gist"&&F()},!0)},pr=function(e,t,n,r,s,i){let a=ut();window.Prism?.hooks&&!window.Prism.redactionHook&&(window.Prism.hooks.add("complete",P=>el(P.element)),window.Prism.redactionHook=!0);let u=Zt();e.on("dark-mode",(P,G)=>{e.aceOption&&(G?e.aceOption.theme="nord_dark":e.aceOption.theme="chrome")});let m=0,w=!1;e.on("dispose",()=>{w=!0,m++,g&&g.resolve()}),e.files=[],e.isMac=/Mac|iPhone|iPad|iPod/.test(navigator.platform||navigator.userAgent),e.fileSearchQuery="",e.fileSearchResults=null,e.fileSearchLoading=!1,u(document,"keydown",function(P){if((P.metaKey||P.ctrlKey)&&P.key==="k"){P.preventDefault();var G=document.querySelector(".tree-search-input");G&&(G.focus(),G.select())}});var g=null;e.onFileSearchChange=function(){g&&(g.resolve(),g=null);let P=e.fileSearchQuery;if(!P||P.length<2){e.fileSearchResults=null,e.fileSearchLoading=!1;return}e.fileSearchLoading=!0;let G=i.defer();g=G,t.get(`/api/repo/${e.repoId}/files/search?q=${encodeURIComponent(P)}`,{timeout:G.promise}).then(function(J){if(!(w||g!==G)){g=null,e.fileSearchLoading=!1;var X={};e.files.forEach(function(F){X[(F.path||"")+"/"+F.name]=!0});for(var K=[],se={},ce=0;ce<J.data.length;ce++){var ue=J.data[ce];if(ue.path)for(var re=ue.path.split("/"),_e="",qe=0;qe<re.length;qe++){var Me=_e;_e=_e?_e+"/"+re[qe]:re[qe];var Pe=Me+"/"+re[qe];!X[Pe]&&!se[Pe]&&(se[Pe]=!0,K.push({name:re[qe],path:Me}))}}K.sort(function(F,Z){return(F.path||"").split("/").length-(Z.path||"").split("/").length}),K.length>0&&e.files.push.apply(e.files,K);for(var Se=[],ze=0;ze<J.data.length;ze++){var Ue=J.data[ze],ke=(Ue.path||"")+"/"+Ue.name;!X[ke]&&Ue.size!=null&&(Se.push(Ue),X[ke]=!0)}Se.length>0&&e.files.push.apply(e.files,Se),e.fileSearchResults=J.data}},function(){!w&&g===G&&(g=null,e.fileSearchLoading=!1,e.fileSearchResults=[])})};let k={yml:"yaml",txt:"text",py:"python",js:"javascript",ts:"typescript"},h=["license","txt"],E=["png","jpg","jpeg","gif","svg","ico","bmp","tiff","tif","webp","avif","heif","heic"],S=["wav","mp3","ogg","wma","flac","aac","m4a"],v=["mp4","avi","webm","mov","mpg","mpeg","mkv","flv","wmv","3gp","3g2","m4v","f4v","f4p","f4a","f4b"];e.on("routeUpdate",function(P,G){if(e.repoId!=r.repoId)return C();if((r.path||"")!=e.filePath){e.filePath=r.path||"",e.paths=e.filePath.split("/").filter(J=>J&&J.trim().length>0),ee();for(let J=0;J<e.paths.length-1;J++){let X=J>0?e.paths.slice(0,J).join("/"):"";e.files.some(se=>se.path===X)||e.getFiles(X)}}});function _(){if(e.paths[0]!="")return;let P=["readme.md","readme.txt","readme.org","readme.1st","readme"],G={};for(let X of e.files)X.name.toLowerCase().indexOf("readme")>-1&&(G[X.name.toLowerCase()]=X.name);let J=null;for(let X of P)if(G[X]){J=X;break}if(!J&&Object.keys(G).length>0&&(J=Object.keys(G)[0]),J){let X=n.url();X[X.length-1]!="/"&&(X+="/"),n.url(X+encodePathForUrl(G[J]))}}e.fileCounts=null,e.getFiles=function(P){let G=e.repoId;return t.get(`/api/repo/${e.repoId}/files/?path=${encodeURIComponent(P)}&v=${e.options.lastUpdateDate}`).then(function(J){if(w||G!==e.repoId)return[];let X=P||"";return e.files=e.files.filter(K=>K.path!==X),e.files.push(...J.data),J.data},function(J){if(w||G!==e.repoId)return[];e.type="error",e.content=cr(J),e.files=[]})};function N(){let P=e.repoId;t.get(`/api/repo/${e.repoId}/files/counts`).then(function(G){w||P!==e.repoId||(e.fileCounts=G.data)},function(){e.fileCounts={}})}function T(){return e.files.filter(P=>P.name==e.paths[e.paths.length-1]&&P.path==e.paths.slice(0,e.paths.length-1).join("/"))[0]}var V=null;e.on("dispose",function(){V&&a.interval.cancel(V)});function M(P){if(w)return;let G=e.repoId;t.get(`/api/repo/${e.repoId}/options`).then(J=>{if(!(w||G!==e.repoId)){if(e.options=J.data,e.options.url){window.location=e.options.url;return}P&&P(J.data)}},J=>{if(!(w||G!==e.repoId)){var X=J.data||{};if(X.error==="rate_limited"&&X.resetAt){let se=function(){var ce=Math.max(0,Math.ceil((e.rateLimitResetAt-Date.now())/1e3));if(ce<=0)e.rateLimitCountdown="",e.rateLimitResetAt=0,V&&(a.interval.cancel(V),V=null),M(P);else{var ue=Math.floor(ce/60),re=ce%60;e.rateLimitCountdown=ue>0?ue+"m "+re+"s":re+"s"}};var K=s Line truncated
`+v.join(`
`)],{type:"text/csv"}),N=document.createElement("a");N.href=URL.createObjectURL(_),N.download=r.username+"-repositories.csv",N.click()},e.showStatusMessage=h=>{let E=h.statusMessage||"(no message)";window.prompt(`Status message for ${h.repoId} (${h.status}):`,E)},e.fetchGithubInfo=h=>{let E=window.open("","_blank");E&&E.document.write("<pre>Loading GitHub info for "+h.repoId+"...</pre>"),t.get("/api/admin/repos/"+h.repoId+"/github").then(S=>{E&&(E.document.open(),E.document.write('<pre style="font:13px monospace;padding:16px;white-space:pre-wrap">'+JSON.stringify(S.data,null,2).replace(/[<>]/g,v=>v==="<"?"&lt;":"&gt;")+"</pre>"),E.document.close())},S=>{let v=S&&S.data?JSON.stringify(S.data,null,2):String(S);E&&(E.document.body.innerHTML='<pre style="color:#B42318;padding:16px">'+v+"</pre>")})};function m(h){t.get("/api/admin/users/"+h+"/repos",{}).then(E=>{e.repositories=E.data},E=>{console.error(E)})}function w(h){t.get("/api/admin/users/"+h,{}).then(E=>{e.userInfo=E.data},E=>{console.error(E)})}w(r.username),m(r.username),e.banUser=()=>{confirm(`Ban user ${r.username}?`)&&t.post(`/api/admin/users/${r.username}/ban`).then(()=>w(r.username),h=>console.error(h))},e.activateUser=()=>{t.post(`/api/admin/users/${r.username}/activate`).then(()=>w(r.username),h=>console.error(h))},e.promoteUser=()=>{confirm(`Promote ${r.username} to admin?`)&&t.post(`/api/admin/users/${r.username}/promote`).then(()=>w(r.username),h=>console.error(h))},e.demoteUser=()=>{confirm(`Remove admin privileges from ${r.username}?`)&&t.post(`/api/admin/users/${r.username}/demote`).then(()=>w(r.username),h=>console.error(h))},e.tokens=[],e.tokenForm={name:"",plaintext:null};function g(){t.get("/api/admin/tokens").then(h=>{e.tokens=h.data||[]},h=>{h.status!==401&&h.status!==403&&console.error(h)})}g(),e.createToken=()=>{e.tokenForm.name&&t.post("/api/admin/tokens",{name:e.tokenForm.name}).then(h=>{e.tokenForm.plaintext=h.data.token,e.tokenForm.name="",g()},h=>console.error(h))},e.revokeToken=h=>{confirm(`Revoke token "${h.name}"?`)&&t.delete("/api/admin/tokens/"+h.id).then(()=>g(),E=>console.error(E))},e.removeCache=h=>{confirm("Remove cached files for "+h.repoId+"?")&&t.delete("/api/admin/repos/"+h.repoId).then(()=>m(r.username),E=>console.error(E))},e.removeRepository=h=>{confirm("Remove repository "+h.repoId+"?")&&t.delete("/api/repo/"+h.repoId+"/").then(()=>m(r.username),E=>console.error(E))},e.updateRepository=h=>{let E=Fe({title:`Refreshing ${h.repoId}...`,date:new Date,body:`The repository ${h.repoId} is going to be refreshed.`});e.toasts.push(E),t.post(`/api/repo/${h.repoId}/refresh`).then(S=>{S.data.status=="ready"?E.title=`${h.repoId} is refreshed.`:E.title=`Refreshing of ${h.repoId}.`},S=>{E.title=`Error during the refresh of ${h.repoId}.`,E.body=S.body})},e.getGitHubRepositories=h=>{t.get(`/api/user/${e.userInfo.username}/all_repositories`,{params:{force:"1"}}).then(E=>{e.userInfo.repositories=E.data})};let k=null;e.watch("query",()=>{s.timeout.cancel(k),k=s.timeout(()=>{m(r.username)},500)},!0)},bl=function(e,t,n){let r=ut(),s=Zt();e.Math=Math,e.watch("user.status",()=>{e.user==null&&n.url("/")}),e.user==null&&n.url("/"),e.conferences=[],e.total=-1,e.totalPage=0,e.statusCounts=[];let i=S=>{if(S.key==="/"&&!["INPUT","TEXTAREA","SELECT"].includes(document.activeElement?.tagName)){S.preventDefault();let v=document.querySelector('.admin-filter-toolbar input[type="search"]');v&&v.focus()}};s(document,"keydown",i),e.on("dispose",()=>document.removeEventListener("keydown",i)),e.clearFilter=S=>{S==="dateRange"?(e.query.dateFrom="",e.query.dateTo=""):e.query[S]="",e.query.page=1},e.chips=[];let a=()=>{let S=[];(e.query.dateFrom||e.query.dateTo)&&S.push({key:"dateRange",label:"Date",value:(e.query.dateFrom||"\u2026")+" \u2013 "+(e.query.dateTo||"\u2026")}),e.chips=S};e.statusCountFor=S=>{let v=(e.statusCounts||[]).find(_=>_._id===S);return v?v.count:0},e.toggleSortDirection=()=>{e.query.direction=e.query.direction==="asc"?"desc":"asc"},e.sortBy=S=>{e.query.sort===S?e.query.direction=e.query.direction==="asc"?"desc":"asc":(e.query.sort=S,e.query.direction="desc"),e.query.page=1},e.sortIcon=S=>e.query.sort===S?e.query.direction==="asc"?"fa-arrow-up":"fa-arrow-down":"";let u="admin.conferences.filterPrefs",m={page:1,limit:25,sort:"name",direction:"asc",search:"",dateFrom:"",dateTo:"",ready:!1,expired:!1,removed:!1,error:!0,preparing:!0},w=loadFilterPrefs(u)||{};e.query=Object.assign({},m,w,{page:1,search:""});let g=n.search();g.search&&(e.query.search=g.search);let k="admin.conferences.presets";e.presets=JSON.parse(localStorage.getItem(k)||"[]"),e.savePreset=()=>{let S=window.prompt("Preset name:");if(!S)return;let v=Object.assign({},e.query);delete v.page,e.presets=(e.presets||[]).filter(_=>_.name!==S),e.presets.push({name:S,query:v}),localStorage.setItem(k,JSON.stringify(e.presets))},e.applyPreset=S=>{Object.assign(e.query,S.query,{page:1})},e.deletePreset=S=>{e.presets=(e.presets||[]).filter(v=>v.name!==S.name),localStorage.setItem(k,JS Line truncated
`).map((ce,ue)=>ue===0?ce:se+ce).join(`
`)}else typeof C=="number"||typeof C=="boolean"?K=String(C):K=JSON.stringify(C);ne.push(`${J}${K}${X}`)}),ne.push("}"),ne.join(`
`)}function w(N){let T=[],V="",M=/(\w+):(>=|<=|!=|>|<|=)?([^\s]+)/g,q=0,z;for(;z=M.exec(N);)V+=N.slice(q,z.index),q=M.lastIndex,T.push({key:z[1],op:z[2]||"=",val:z[3]});return V+=N.slice(q),{filters:T,free:V.trim().toLowerCase()}}function g(N,T){for(let V of T.filters){let M=(z,ne,ee)=>{let C=parseFloat(z),P=parseFloat(ne);return ee==="="?String(z)===String(ne):ee==="!="?String(z)!==String(ne):ee===">="?C>=P:ee==="<="?C<=P:ee===">"?C>P:ee==="<"?C<P:!0},q;if(V.key==="code")q=N.displayMessage;else if(V.key==="module")q=N.module;else if(V.key==="status")q=N._status;else if(V.key==="url")q=N._url;else if(V.key==="repo")q=N._repoId;else if(V.key==="level")q=N.level;else continue;if(q==null||!M(q,V.val,V.op))return!1}return!(T.free&&((N.displayMessage||"")+" "+(N.module||"")+" "+(N._url||"")+" "+JSON.stringify(N.raw||[])).toLowerCase().indexOf(T.free)===-1)}function k(){let N=w(e.query.search||"");e.parsedFilterCount=N.filters.length;let T=e.query.bucket,V=e.entries.filter(q=>T&&q._bucket!==T?!1:g(q,N)),M=e.query.group;if(M){let q=ee=>M==="module"?ee.module:ee.displayMessage||ee.message||"_",z=new Map;for(let ee of V){let C=q(ee);if(z.has(C)){let P=z.get(C);P.count++,P._related.push(ee),new Date(ee.ts)>new Date(P.ts)&&(P.ts=ee.ts,P._url=ee._url,P._status=ee._status),new Date(ee.ts)<new Date(P._firstSeen)&&(P._firstSeen=ee.ts)}else{let P=Object.assign({},ee);P._key=`${M}:${C}`,P._related=[ee],P._firstSeen=ee.ts,P._lastHourCount=0,P.count=1,z.set(C,P)}}let ne=Date.now()-3600*1e3;for(let ee of z.values())ee._lastHourCount=ee._related.filter(C=>new Date(C.ts).getTime()>=ne).length;V=Array.from(z.values())}else V=V.map((q,z)=>(q._key="row:"+z+":"+q.ts,q._related=[q],q._firstSeen=q.ts,q._lastHourCount=0,q.count=1,q));e.query.sort==="count"?V.sort((q,z)=>z.count-q.count||new Date(z.ts)-new Date(q.ts)):V.sort((q,z)=>new Date(z.ts)-new Date(q.ts)),e.visible=V}function h(N){let T=N?e.entries.length:0,V=N?e.pageSize:Math.max(e.pageSize,e.entries.length||e.pageSize);t.get("/api/admin/errors",{params:{offset:T,limit:V}}).then(M=>{let q=(M.data.entries||[]).map(u);e.entries=N?e.entries.concat(q):q,e.available=!!M.data.available,e.cap=M.data.max||e.cap,e.total=M.data.total||e.entries.length,k()},M=>console.error(M))}e.loadMore=()=>h(!0),e.canLoadMore=()=>e.entries.length<e.total;function E(){t.get("/api/admin/errors/stats").then(N=>{let T=N.data||{},V=T.prev24h?Math.round((T.last24h-T.prev24h)/T.prev24h*100):0;e.stats={last24h:T.last24h||0,prev24h:T.prev24h||0,delta:V,severity:T.severity||{error:0,warn:0,info:0},unique:T.unique||{error:0,warn:0,info:0},buckets:T.buckets||[],dropped:T.dropped||0}},N=>console.error(N))}function S(){h(),E()}e.barPx=(N,T)=>{let V=e.stats.buckets||[],M=0;for(let ee of V)M=Math.max(M,(ee.error||0)+(ee.warn||0)+(ee.info||0));if(!M)return 0;let q=(N.error||0)+(N.warn||0)+(N.info||0);if(!q)return 0;let z=Math.round(q/M*60),ne=N[T]||0;return Math.round(ne/q*z)},e.bucketTitle=N=>`${new Date(N.hour).toLocaleTimeString([],{hour:"2-digit",minute:"2-digit"})} \xB7 ${N.error||0} err \xB7 ${N.warn||0} warn \xB7 ${N.info||0} info`,e.toggle=N=>{e.expanded[N._key]=!e.expanded[N._key]},e.setBucket=N=>{e.query.bucket=N},e.refreshNow=S,e.clearAll=()=>{confirm("Clear all captured errors?")&&t.delete("/api/admin/errors").then(S,N=>console.error(N))},e.exportCsv=()=>{let N=["ts","level","module","displayMessage","_status","_url","_repoId"],T=[N.join(",")];for(let z of e.visible)T.push(N.map(ne=>{let ee=z[ne]==null?"":String(z[ne]);return/[",\n]/.test(ee)?`"${ee.replace(/"/g,'""')}"`:ee}).join(","));let V=new Blob([T.join(`
`)],{type:"text/csv;charset=utf-8"}),M=URL.createObjectURL(V),q=document.createElement("a");q.href=M,q.download=`errors-${new Date().toISOString().slice(0,19)}.csv`,document.body.appendChild(q),q.click(),document.body.removeChild(q),URL.revokeObjectURL(M)};function v(N){e.copyHint=`${N} copied`,s.timeout(()=>{e.copyHint=""},1500)}e.copyJson=N=>{navigator.clipboard.writeText(N._detailJson).then(()=>v("JSON"))},e.copyCurl=N=>{if(!N._url)return;let V=`curl -X ${N._method||"GET"} '${window.location.origin}${N._url}'`;navigator.clipboard.writeText(V).then(()=>v("curl"))},S();let _=r(()=>{e.query.autoRefresh&&S()},15e3);e.on("dispose",()=>r.cancel(_)),e.watch("query.search",k),e.watch("query.bucket",k),e.watch("query.sort",k),e.watch("query.group",k)},El=function(e,t,n,r){if(e.Math=Math,e.watch("user.status",()=>{e.user==null&&n.url("/")}),e.user==null){n.url("/");return}e.data=null,e.loading=!0,e.error=null,e.performance=null,e.performanceMinutes=15,e.performanceTab="routes",e.cacheReport=null,e.cacheError=null,e.cacheMinutes=60,e.cacheTraffic="all";var s=!1;e.loadCache=function(){if(!s){s=!0;var v=e.cacheMinutes;t.get("/api/admin/cache",{params:{minutes:v}}).then(function(_){v===e.cacheMinutes&&(e.cacheReport=_.data,e.cacheError=_.data.available?null:_.data.reason||"Cache monitoring is unavailable.")},function(){e.cacheError="Could not refresh cache monitoring. Displayed values may be stale."}).finally(function(){s=!1,v!==e.cacheMinutes&&e.loadCache()})}},e.cacheRows=function(){return(e.cacheReport?.rows||[]).filter(function(v){return e.cacheTraffic==="all"||v.traffic===e.cacheTraffic})},e.cacheRate=function(v,_){var N=e.cacheRows(),T=function(q){return N.filter(function(z){return z.name===q}).reduce(function(z,ne){return z+ne[_]},0)},V=T(v+"_hit"),M=T(v+"_miss");return V+M?(100*V/(V+M)).toFixed(1)+"%":"No samples"},e.cacheEventName=function(v){return v.replaceAll("_"," ")},e.cacheP95=function(v){var _=0,N=e.cacheReport?.latencyBucketsMs||[];if(!v.count)return"No samples";if(!v.ms)return"Not timed";for(var T=0;T<=N.length;T++)if(_+=v["b"+T]||0,_>=Math.ceil(v.count*.95))return T<N.length?"\u2264 "+N[T]+" ms":"> 10 s";return"No samples"},e.performanceError=null,e.formatMs=function(v){return v==null?"":v<1e3?v+" ms":+(v/1e3).toFixed(1)+" s"},e.latencyBound=function(v,_){return v.count?v[_]==null?"> 60 s":"\u2264 "+e.formatMs(v[_]):"No samples"},e.firstByteBound=function(v){return e.latencyBound({count:v.firstByteCount,p95UpperMs:v.firstByteP95UpperMs},"p95UpperMs")},e.sampleAge=function(v){return Math.max(0,Math.round((Date.now()-v.sampledAt)/1e3))},e.peakRss=function(v){return(e.performance.runtimeSeries||[]).filter(function(_){return _.instance===v.instance}).reduce(function(_,N){return Math.max(_,N.rss)},v.memory.rss)},e.performanceWarning=function(v){return e.sampleAge(v)>45||v.droppedBatches||v.droppedMetrics||v.sockets?.closeWait>0||v.eventLoop.p95Ms>100||v.memoryLimitBytes&&v.memory.rss>v.memoryLimitBytes*.8};var i=!1;e.loadPerformance=function(){var v=e.performanceMinutes;e.performance?.windowMinutes&&e.performance.windowMinutes!==v&&(e.performance=null),!i&&(i=!0,t.get("/api/admin/performance",{params:{minutes:v}}).then(function(_){v===e.performanceMinutes&&(e.performance=_.data,e.performanceError=_.data.available?null:"Performance monitoring is unavailable. Request history may be incomplete.")},function(){v===e.performanceMinutes&&(e.performanceError="Could not refresh performance monitoring. Displayed samples may be stale.")}).finally(function(){i=!1,v!==e.performanceMinutes&&e.loadPerformance()}))};function a(v){if(v==null)return"\u2014";for(var _=["B","KB","MB","GB","TB"],N=0,T=v;T>=1024&&N<_.length-1;)T/=1024,N++;return T.toFixed(N>0?1:0)+" "+_[N]}e.humanBytes=a;function u(v){if(!v)return"\u2014";var _=Math.floor(v/86400),N=Math.floor(v%86400/3600),T=Math.floor(v%3600/60);return _>0?_+"d "+(N<10?"0":"")+N+"h":N>0?N+"h "+(T<10?"0":"")+T+"m":T+"m"}e.humanDuration=u;function m(v){return v==null?"\u2014":v>=1e6?(v/1e6).toFixed(1)+"M":v>=1e3?(v/1e3).toFixed(1)+"K":String(v)}e.humanNum=m,e.queueTotal=function(v){return v?(v.waiting||0)+(v.active||0)+(v.delayed||0)+(v.failed||0):0},e.statusCount=function(v){if(!e.data||!e.data.repos)return 0;for(var _=e.data.repos.statusBreakdown||[],N=0;N<_.length;N++)if(_[N]._id===v)return _[N].count;return 0},e.barPct=function(v){var _=e.data&&e.data.repos?e.data.repos.total:0;if(!_)return 0;var N=[v];v==="expired"&&N.push("expiring"),v==="removed"&&N.push("removing"),v==="preparing"&&N.push("download");var T=0;return N.forEach(function(V){T+=e.statusCount(V)}),Math.max(.4,T/_*100)},e.errPct=function(v){if(!e.data||!e.data.errors)return 0;var _=Math.max(e.data.errors.severity.error,e.data.errors.severity.warn,e.data.errors.severity.info,1);return e.data.errors.severity[v]/_*100};function w(v){var _=v||[];return _.map(function(N,T){var V=_[T-1]||{},M=Object.assign({},N);return M.dailyRepositories=T?Math.max(0,(N.nbRepositories||0)-(V.nbRepositories||0)):0,M.dailyUsers=T? Line truncated
anonymous_github`)])])])])]),f(" Limitations "),o("h4",{class:"faq-cat"},[o("svg",{class:"ui-icon mr-2","aria-hidden":"true"},[o("use",{href:"#ui-icon-warning"})]),b("Limitations ")]),o("div",{class:"panel-group",id:"faq-limitations",role:"tablist","aria-multiselectable":"true"},[o("div",{class:"panel panel-default mb-3"},[o("div",{class:"panel-heading p-3",role:"tab",id:"headingLimitation"},[o("h3",{class:"panel-title"},[o("a",{class:"collapsed",role:"button","data-toggle":"collapse","data-parent":"#faq-limitations",href:"#limitation","aria-expanded":"false","aria-controls":"limitation"}," What are the limitations of Anonymous GitHub? ")])]),o("div",{id:"limitation",class:"panel-collapse collapse",role:"tabpanel","aria-labelledby":"headingLimitation"},[o("div",{class:"panel-body p-3"},[o("ul",null,[o("li",null," Anonymous GitHub only anonymizes text-based files. Binary files (compiled executables, archives, etc.) are served as-is without anonymization. "),o("li",null," Files larger than 8 MB are not supported. "),o("li",null," Static site generators (such as Jekyll) used with GitHub Pages are not fully supported, although Markdown files are converted to HTML. "),o("li",null," The anonymization of terms within source code may change the behavior of the program (e.g., if a replaced term appears in a string literal or identifier). ")])])])])]),f(" Privacy & Security "),o("h4",{class:"faq-cat"},[o("svg",{class:"ui-icon mr-2","aria-hidden":"true"},[o("use",{href:"#ui-icon-shield"})]),b("Privacy & Security ")]),o("div",{class:"panel-group",id:"faq-privacy",role:"tablist","aria-multiselectable":"true"},[o("div",{class:"panel panel-default mb-3"},[o("div",{class:"panel-heading p-3",role:"tab",id:"headingData"},[o("h3",{class:"panel-title"},[o("a",{class:"collapsed",role:"button","data-toggle":"collapse","data-parent":"#faq-privacy",href:"#data","aria-expanded":"false","aria-controls":"data"}," How is my data handled? ")])]),o("div",{id:"data",class:"panel-collapse collapse",role:"tabpanel","aria-labelledby":"headingData"},[o("div",{class:"panel-body p-3"},[o("p",null," Data stored on Anonymous GitHub is never shared or used for any purpose beyond providing the anonymization service. When a repository is removed or expires, only its configuration is retained \u2014 this makes it easy to restore the repository if needed and ensures that no future repository reuses the same ID. ")])])]),o("div",{class:"panel panel-default mb-3"},[o("div",{class:"panel-heading p-3",role:"tab",id:"headingPermissions"},[o("h3",{class:"panel-title"},[o("a",{class:"collapsed",role:"button","data-toggle":"collapse","data-parent":"#faq-privacy",href:"#permissions","aria-expanded":"false","aria-controls":"permissions"}," Why does GitHub say Anonymous GitHub asks for write access? ")])]),o("div",{id:"permissions",class:"panel-collapse collapse",role:"tabpanel","aria-labelledby":"headingPermissions"},[o("div",{class:"panel-body p-3"},[o("p",null,[b("The GitHub App has read-only access to the repositories you choose. You can change repository access on your "),o("a",{href:"/connections"},"GitHub connections"),b(" page.")]),o("p",null,"Legacy GitHub OAuth includes write access to private repositories because GitHub's OAuth scopes do not offer a read-only option for them. Existing OAuth anonymizations and gists can still use this connection."),o("p",null,"Anonymous GitHub reads repository content. It never pushes, modifies, or deletes files on GitHub.")])])]),o("div",{class:"panel panel-default mb-3"},[o("div",{class:"panel-heading p-3",role:"tab",id:"headingViewer"},[o("h3",{class:"panel-title"},[o("a",{class:"collapsed",role:"button","data-toggle":"collapse","data-parent":"#faq-privacy",href:"#viewer","aria-expanded":"false","aria-controls":"viewer"}," Can repository owners see who viewed their anonymized repository? ")])]),o("div",{id:"viewer",class:"panel-collapse collapse",role:"tabpanel","aria-labelledby":"headingViewer"},[o("div",{class:"panel-body p-3"},[o("p",null," No. Only the total number of views is tracked as an incremental counter. It is not possible for the repository owner or for Anonymous GitHub to identify individual viewers. Reviewer anonymity is fully preserved. ")])])])]),f(" Self-Hosting "),o("h4",{class:"faq-cat"},[o("svg",{class:"ui-icon mr-2","aria-hidden":"true"},[o("use",{href:"#project-icon-storage"})]),b("Self-Hosting ")]),o("div",{class:"panel-group",id:"faq-hosting",role:"tablist","aria-multiselectable":"true"},[o("div",{class:"panel panel-default mb-3"},[o("div",{class:"panel-heading p-3",role:"tab",id:"headingDeploy"},[o("h3",{class:"panel-title"},[o("a",{class:"collapsed",role:"button","data-toggle":"collapse","data-parent":"#faq-hosting",href:"#deploy","aria-expanded":"false","aria-controls":"deploy"}," Can I deploy my own instance? ")])]),o("div",{id:"deploy",class:"panel-collapse collapse",role:"tabpanel","aria-labelledby":"headingDeploy"},[o("div",{class:"panel-body p-3"},[o("p",null,[b(" Yes. Anonymous Git Line truncated
anonymized repository often think it is about the paper, so the
label names the service and the tooltip says who to contact. `),t[17]||(t[17]=o("li",{class:"nav-item d-none d-lg-block"},[o("a",{class:"nav-link",target:"_blank",rel:"noopener",href:"https://github.com/tdurieux/anonymous_github/issues/new?template=issue_report.yml",title:"Report a problem with the Anonymous GitHub service. Questions about an anonymized repository or paper go to its authors.","data-offset":"30"},[o("svg",{class:"ui-icon d-lg-none mr-1","aria-hidden":"true"},[o("use",{href:"#ui-icon-bug"})]),w("Report a bug ")])],-1)),e.isDarkMode?f("v-if",!0):(l(),d("li",K3,[o("a",{class:"nav-link nav-icon",href:"#",title:"Switch to dark mode","aria-label":"Switch to dark mode",role:"button",onClick:t[0]||(t[0]=ie(n=>{e.darkMode(!0)},["prevent"]))},[...t[13]||(t[13]=[o("svg",{class:"ui-icon","aria-hidden":"true"},[o("use",{href:"#ui-icon-moon"})],-1)])])])),e.isDarkMode?(l(),d("li",Y3,[o("a",{class:"nav-link nav-icon",href:"#",title:"Switch to light mode","aria-label":"Switch to light mode",role:"button",onClick:t[1]||(t[1]=ie(n=>{e.darkMode(!1)},["prevent"]))},[...t[14]||(t[14]=[o("svg",{class:"ui-icon","aria-hidden":"true"},[o("use",{href:"#ui-icon-sun"})],-1)])])])):f("v-if",!0),!e.user&&!e.isReviewerRoute?(l(),d("li",J3,[...t[15]||(t[15]=[o("a",{class:"nav-link btn-signin",target:"_self",href:"/signin","data-offset":"30"},[o("i",{class:"fab fa-github mr-1"}),w(" Sign in ")],-1)])])):f("v-if",!0),e.user?(l(),d("li",Q3,[o("a",X3,[e.user?.photo?(l(),d("img",{key:0,width:"22",height:"22",class:"rounded-circle",src:e.safeUrl(e.user?.photo)},null,8,Z3)):f("v-if",!0),o("span",x3,c(e.user?.username),1)]),o("div",eP,[o("h6",tP,"Signed in as @"+c(e.user?.username),1),t[16]||(t[16]=ue('<a class="dropdown-item" href="/profile"><svg class="ui-icon" aria-hidden="true"><use href="#project-icon-filter"></use></svg><span>Settings<small>Defaults, quotas, account</small></span></a><a class="dropdown-item" href="/connections"><i class="fab fa-github" aria-hidden="true"></i><span>GitHub connections<small>Repository access and OAuth</small></span></a><a class="dropdown-item" href="/claim"><svg class="ui-icon" aria-hidden="true"><use href="#ui-icon-link"></use></svg><span>Claim an anonymization<small>Attach an existing mirror to your account</small></span></a><div class="dropdown-divider"></div><a class="dropdown-item" href="https://github.com/tdurieux/anonymous_github/issues/new?template=issue_report.yml" target="_blank" rel="noopener"><svg class="ui-icon" aria-hidden="true"><use href="#ui-icon-bug"></use></svg><span>Report a bug<small>About this service, not a paper</small></span></a><a class="dropdown-item" href="https://github.com/tdurieux/anonymous_github/" target="_blank" rel="noopener"><i class="fab fa-github" aria-hidden="true"></i><span>Source code</span></a><a class="dropdown-item" href="https://ko-fi.com/tdurieux" target="_blank" rel="noopener"><svg class="ui-icon" aria-hidden="true"><use href="#ui-icon-heart"></use></svg><span>Support the project</span></a><div class="dropdown-divider"></div><a class="dropdown-item" href="/api/user/logout" target="_self"><svg class="ui-icon" aria-hidden="true"><use href="#ui-icon-logout"></use></svg><span>Sign out</span></a>',9))])])):f("v-if",!0)])])],2),e.generalMessage?(l(),d("div",{key:0,class:"navbar shadow generalMessage",innerHTML:e.sanitize(e.generalMessage)},null,8,oP)):f("v-if",!0),e.emailPromptOpen?(l(),d("dialog",{key:1,id:"email-prompt",class:"email-prompt","aria-labelledby":"email-prompt-title","aria-describedby":"email-prompt-reason",onCancel:t[6]||(t[6]=ie(n=>e.respondToEmailPrompt("later"),["prevent"]))},[o("form",{onSubmit:t[5]||(t[5]=ie(n=>e.respondToEmailPrompt("save"),["prevent"]))},[t[19]||(t[19]=o("h2",{id:"email-prompt-title"},"Stay informed about access problems",-1)),t[20]||(t[20]=o("p",{id:"email-prompt-reason"},"Get email alerts when your GitHub token or repository access fails.",-1)),t[21]||(t[21]=o("label",{for:"email-prompt-address"},"Email address",-1)),N(o("input",{id:"email-prompt-address",class:"form-control",type:"email",autocomplete:"email",maxlength:"254",required:"","onUpdate:modelValue":t[2]||(t[2]=n=>e.emailPromptAddress=n),disabled:e.emailPromptBusy},null,8,nP),[[zt,e.emailPromptAddress]]),e.emailPromptError?(l(),d("p",sP,c(e.emailPromptError),1)):f("v-if",!0),o("div",rP,[o("button",{type:"button",class:"btn btn-outline-ink",disabled:e.emailPromptBusy,onClick:t[3]||(t[3]=n=>e.respondToEmailPrompt("later"))},"Remind me later",8,iP),o("button",{type:"button",class:"btn btn-outline-ink",disabled:e.emailPromptBusy,onClick:t[4]||(t[4]=n=>e.respondToEmailPrompt("never"))},"Never",8,aP),o("button",{type:"submit",class:"btn btn-ink",disabled:e.emailPromptBusy},c(e.emailPromptBusy?"Saving\u2026":"Save"),9,lP)])],32)],32)):f("v-if",!0)],64)}var dP={class:"paper-landing"},uP={class:"paper-hero"},cP={class:"paper-hero-copy"},pP={class:"paper-cta-row"},fP={key:0,href:"/signin",target:"_self",class:"btn-hero"},mP={key:1,href:"/anonymize",cla Line truncated
label names the service and the tooltip says who to contact. `),t[17]||(t[17]=o("li",{class:"nav-item d-none d-lg-block"},[o("a",{class:"nav-link",target:"_blank",rel:"noopener",href:"https://github.com/tdurieux/anonymous_github/issues/new?template=issue_report.yml",title:"Report a problem with the Anonymous GitHub service. Questions about an anonymized repository or paper go to its authors.","data-offset":"30"},[o("svg",{class:"ui-icon d-lg-none mr-1","aria-hidden":"true"},[o("use",{href:"#ui-icon-bug"})]),b("Report a bug ")])],-1)),e.isDarkMode?f("v-if",!0):(l(),d("li",uP,[o("a",{class:"nav-link nav-icon",href:"#",title:"Switch to dark mode","aria-label":"Switch to dark mode",role:"button",onClick:t[0]||(t[0]=ie(n=>{e.darkMode(!0)},["prevent"]))},[...t[13]||(t[13]=[o("svg",{class:"ui-icon","aria-hidden":"true"},[o("use",{href:"#ui-icon-moon"})],-1)])])])),e.isDarkMode?(l(),d("li",cP,[o("a",{class:"nav-link nav-icon",href:"#",title:"Switch to light mode","aria-label":"Switch to light mode",role:"button",onClick:t[1]||(t[1]=ie(n=>{e.darkMode(!1)},["prevent"]))},[...t[14]||(t[14]=[o("svg",{class:"ui-icon","aria-hidden":"true"},[o("use",{href:"#ui-icon-sun"})],-1)])])])):f("v-if",!0),!e.user&&!e.isReviewerRoute?(l(),d("li",pP,[...t[15]||(t[15]=[o("a",{class:"nav-link btn-signin",target:"_self",href:"/signin","data-offset":"30"},[o("i",{class:"fab fa-github mr-1"}),b(" Sign in ")],-1)])])):f("v-if",!0),e.user?(l(),d("li",fP,[o("a",mP,[e.user?.photo?(l(),d("img",{key:0,width:"22",height:"22",class:"rounded-circle",src:e.safeUrl(e.user?.photo)},null,8,hP)):f("v-if",!0),o("span",vP,c(e.user?.username),1)]),o("div",yP,[o("h6",gP,"Signed in as @"+c(e.user?.username),1),t[16]||(t[16]=de('<a class="dropdown-item" href="/profile"><svg class="ui-icon" aria-hidden="true"><use href="#project-icon-filter"></use></svg><span>Settings<small>Defaults, quotas, account</small></span></a><a class="dropdown-item" href="/connections"><i class="fab fa-github" aria-hidden="true"></i><span>GitHub connections<small>Repository access and OAuth</small></span></a><a class="dropdown-item" href="/claim"><svg class="ui-icon" aria-hidden="true"><use href="#ui-icon-link"></use></svg><span>Claim an anonymization<small>Attach an existing mirror to your account</small></span></a><div class="dropdown-divider"></div><a class="dropdown-item" href="https://github.com/tdurieux/anonymous_github/issues/new?template=issue_report.yml" target="_blank" rel="noopener"><svg class="ui-icon" aria-hidden="true"><use href="#ui-icon-bug"></use></svg><span>Report a bug<small>About this service, not a paper</small></span></a><a class="dropdown-item" href="https://github.com/tdurieux/anonymous_github/" target="_blank" rel="noopener"><i class="fab fa-github" aria-hidden="true"></i><span>Source code</span></a><a class="dropdown-item" href="https://ko-fi.com/tdurieux" target="_blank" rel="noopener"><svg class="ui-icon" aria-hidden="true"><use href="#ui-icon-heart"></use></svg><span>Support the project</span></a><div class="dropdown-divider"></div><a class="dropdown-item" href="/api/user/logout" target="_self"><svg class="ui-icon" aria-hidden="true"><use href="#ui-icon-logout"></use></svg><span>Sign out</span></a>',9))])])):f("v-if",!0)])])],2),e.generalMessage?(l(),d("div",{key:0,class:"navbar shadow generalMessage",innerHTML:e.sanitize(e.generalMessage)},null,8,bP)):f("v-if",!0),e.emailPromptOpen?(l(),d("dialog",{key:1,id:"email-prompt",class:"email-prompt","aria-labelledby":"email-prompt-title","aria-describedby":"email-prompt-reason",onCancel:t[6]||(t[6]=ie(n=>e.respondToEmailPrompt("later"),["prevent"]))},[o("form",{onSubmit:t[5]||(t[5]=ie(n=>e.respondToEmailPrompt("save"),["prevent"]))},[t[19]||(t[19]=o("h2",{id:"email-prompt-title"},"Stay informed about access problems",-1)),t[20]||(t[20]=o("p",{id:"email-prompt-reason"},"Get email alerts when your GitHub token or repository access fails.",-1)),t[21]||(t[21]=o("label",{for:"email-prompt-address"},"Email address",-1)),R(o("input",{id:"email-prompt-address",class:"form-control",type:"email",autocomplete:"email",maxlength:"254",required:"","onUpdate:modelValue":t[2]||(t[2]=n=>e.emailPromptAddress=n),disabled:e.emailPromptBusy},null,8,wP),[[zt,e.emailPromptAddress]]),e.emailPromptError?(l(),d("p",kP,c(e.emailPromptError),1)):f("v-if",!0),o("div",EP,[o("button",{type:"button",class:"btn btn-outline-ink",disabled:e.emailPromptBusy,onClick:t[3]||(t[3]=n=>e.respondToEmailPrompt("later"))},"Remind me later",8,CP),o("button",{type:"button",class:"btn btn-outline-ink",disabled:e.emailPromptBusy,onClick:t[4]||(t[4]=n=>e.respondToEmailPrompt("never"))},"Never",8,_P),o("button",{type:"submit",class:"btn btn-ink",disabled:e.emailPromptBusy},c(e.emailPromptBusy?"Saving\u2026":"Save"),9,SP)])],32)],32)):f("v-if",!0)],64)}var NP={class:"paper-landing"},RP={class:"paper-hero"},DP={class:"paper-hero-copy"},TP={class:"paper-cta-row"},AP={key:0,href:"/signin",target:"_self",class:"btn-hero"},OP={key:1,href:"/anonymize",cla Line truncated
MIT CSAIL
jane.smith@example.org`},Mq={id:"termsHelp",class:"form-text text-muted"},Uq={class:"invalid-feedback"},$q={id:"settings-display",class:"paper-settings-section"},Lq={class:"form-group paper-check-list"},Fq={class:"form-check"},zq={class:"form-check-input",type:"checkbox",id:"link",name:"link"},Hq={class:"form-check"},Bq={class:"form-check-input",type:"checkbox",id:"image",name:"image"},jq={class:"form-check"},Gq={class:"form-check-input",type:"checkbox",id:"pdf",name:"pdf"},Wq={class:"form-check"},Kq={class:"form-check-input",type:"checkbox",id:"notebook",name:"notebook"},Yq={id:"settings-features",class:"paper-settings-section"},Jq={class:"form-group paper-check-list"},Qq={class:"form-check"},Xq={class:"form-check-input",type:"checkbox",id:"page",name:"page"},Zq={class:"form-check"},xq={class:"form-check-input",type:"checkbox",id:"loc",name:"loc"},eM={class:"form-check"},tM={class:"form-check-input",type:"checkbox",id:"update",name:"update"},oM={class:"form-group"},nM={class:"form-control",id:"mode",name:"mode"},sM={class:"form-text text-muted"},rM={id:"settings-expiration",class:"paper-settings-section"},iM={class:"form-group"},aM={class:"form-control",id:"expiration",name:"expiration"},lM={key:0,class:"draft-status",role:"status"},dM={key:1,class:"draft-status"},uM={class:"paper-settings-footer"},cM={key:0,class:"paper-inline-alert paper-inline-alert-error",role:"alert"},pM=["textContent"],fM={class:"paper-settings-actions"},mM=["disabled"],hM={key:0,class:"ui-icon mr-1","aria-hidden":"true"},vM={id:"settings-account",class:"paper-settings-section"},yM={class:"paper-account-card"},gM=["src"],bM={class:"paper-account-identity"},wM={class:"paper-account-name"},kM=["disabled"],EM={key:0,role:"status"},CM={key:1,class:"paper-inline-alert paper-inline-alert-error",role:"alert"},_M=["disabled"],SM={key:3,role:"status"},NM={class:"paper-danger-zone"},DM={key:0,class:"paper-inline-alert paper-inline-alert-error",role:"alert"},RM=["textContent"],TM=["disabled"];function Ql(e,t){let n=Ge("quota-usage"),r=ve("paper-scrollspy"),s=ve("field"),i=ve("form");return l(),d("div",Nq,[t[58]||(t[58]=ue('<div class="paper-crumbs"><a href="/dashboard">My work</a> \xA0/\xA0 <span class="here">Settings</span></div><h1 class="paper-page-title">Your <em>settings</em></h1><p class="paper-page-lede">Defaults applied to every new anonymization, your quotas, and your account.</p>',3)),o("div",Dq,[N((l(),d("aside",Rq,[...t[8]||(t[8]=[ue('<div class="paper-settings-toc-head">On this page</div><nav><a href="#settings-quota">Quota</a><a href="#settings-terms">Terms to redact</a><a href="#settings-display">Display</a><a href="#settings-features">Features</a><a href="#settings-expiration">Expiration</a><a href="#settings-account">Account</a></nav>',2)])])),[[r]]),o("div",Tq,[f(" Account usage shares the dashboard component. "),o("section",Aq,[t[10]||(t[10]=o("div",{class:"paper-section-eyebrow"},"Quota",-1)),t[11]||(t[11]=o("p",{class:"paper-section-copy"},"What you are using across all your anonymizations. Conferences can lift these limits during a review window.",-1)),e.quota?(l(),Lt(n,{key:0,quota:e.quota,fmt:e.fmt},null,8,["quota","fmt"])):f("v-if",!0),e.quota?f("v-if",!0):(l(),d("div",Oq,[(l(),d(K,null,oe([1,2,3],(a,u)=>o("div",Iq,[...t[9]||(t[9]=[o("div",{class:"quota-header"},[o("span",{class:"skeleton skeleton-line",style:{width:"40%"}}),o("span",{class:"skeleton skeleton-line",style:{width:"25%"}})],-1),o("div",{class:"quota-track"},null,-1)])])),64))]))]),N((l(),d("form",{class:"form needs-validation",name:"defaultsForm",novalidate:"",onSubmit:t[2]||(t[2]=ie(a=>e.submitForm(a,()=>{e.saveDefault(a)}),["prevent"]))},[o("section",Pq,[t[17]||(t[17]=o("div",{class:"paper-section-eyebrow"},"Terms to redact",-1)),t[18]||(t[18]=o("p",{class:"paper-section-copy"},"Pre-filled in every new anonymization. You can still edit the list per anonymization.",-1)),o("div",Vq,[t[16]||(t[16]=o("label",{class:"paper-field-label",for:"terms"},"Default terms",-1)),N(o("textarea",qq,null,512),[[s,{state:e.viewState,set:a=>{e.terms=a},value:e.terms,form:"defaultsForm",options:{debounce:250}}]]),o("small",Mq,[t[12]||(t[12]=w(" One term per line (regex allowed). Each match is replaced by ",-1)),o("code",null,c(e.site_options?.ANONYMIZATION_MASK||"XXX")+"-[N]",1),t[13]||(t[13]=w(", or use ",-1)),t[14]||(t[14]=o("code",null,"term=>replacement",-1)),t[15]||(t[15]=w(" to pick your own. ",-1))]),N(o("div",Uq," Terms are in an invalid format ",512),[[j,e.defaultsForm?.terms?.errors?.format]])])]),o("section",$q,[t[27]||(t[27]=o("div",{class:"paper-section-eyebrow"},"Display",-1)),t[28]||(t[28]=o("p",{class:"paper-section-copy"},"What reviewers can see inside an anonymized repository.",-1)),o("div",Lq,[o("div",Fq,[N(o("input",zq,null,512),[[s,{state:e.viewState,set:a=>{e.options.link=a},value:e.options?.link,form:"defaultsForm",options:{}}]]),t[19]||(t[19]=o("label",{class:"form-check-label",for:"link"},"Keep links",-1)),t[20]||(t[20]=o("small",{class:"form-text t Line truncated
`);for(let E=0;E<y.length;E++){let h=y[E];if(h.startsWith("diff --git")){k(),m();continue}if(h.startsWith("--- ")){i&&i.lines.length&&k(),m().oldPath=h.replace(/^--- (a\/)?/,"").trim();continue}if(h.startsWith("+++ ")){m().newPath=h.replace(/^\+\+\+ (b\/)?/,"").trim();continue}if(!(h.startsWith("index ")||h.startsWith("similarity index")||h.startsWith("rename ")||h.startsWith("new file mode")||h.startsWith("deleted file mode")||h.startsWith("Binary files"))){if(h.startsWith("@@")){let C=h.match(/@@\s+-(\d+)(?:,\d+)?\s+\+(\d+)(?:,\d+)?\s+@@/);C&&(a=parseInt(C[1],10),u=parseInt(C[2],10)),m().lines.push({kind:"hunk",oldNo:"",newNo:"",text:h});continue}i&&(h.startsWith("+")?(i.lines.push({kind:"add",oldNo:"",newNo:u,text:h.slice(1)}),u++):h.startsWith("-")?(i.lines.push({kind:"remove",oldNo:a,newNo:"",text:h.slice(1)}),a++):(i.lines.push({kind:"ctx",oldNo:a,newNo:u,text:h.startsWith(" ")?h.slice(1):h}),a++,u++))}}return n(s,i),s.join("")}})({trustAsHtml:e=>e});function W8(e,t){return e==null?"":Number(e).toLocaleString(void 0,t==null?{}:{minimumFractionDigits:t,maximumFractionDigits:t})}function K8(e){return String(e??"").toUpperCase()}function Y8(e,t,n=0){return(e||[]).slice(n,n+t)}function J8(e,t){return Array.isArray(e)?typeof t=="function"?e.filter(t):e:[]}function Q8(e,t){if(!t)return e||[];let n=Array.isArray(t)?t:[t];return[...e||[]].sort((r,s)=>{for(let i of n){let a=i[0]==="-"?-1:1;i=i.replace(/^[+-]/,"");let u=i.split(".").reduce((k,y)=>k?.[y],r),m=i.split(".").reduce((k,y)=>k?.[y],s);if(typeof u=="string"&&(u=u.toLowerCase()),typeof m=="string"&&(m=m.toLowerCase()),u!==m)return(u==null?-1:m==null?1:u<m?-1:1)*a}return 0})}function X8(e,t){if(!e)return"";let n=new Date(e);if(isNaN(n))return String(e);if(!t||t==="mediumDate")return n.toLocaleDateString(void 0,{year:"numeric",month:"short",day:"numeric"});let r={yyyy:n.getFullYear(),MMM:n.toLocaleDateString(void 0,{month:"short"}),MM:String(n.getMonth()+1).padStart(2,"0"),dd:String(n.getDate()).padStart(2,"0"),d:n.getDate(),HH:String(n.getHours()).padStart(2,"0"),mm:String(n.getMinutes()).padStart(2,"0"),ss:String(n.getSeconds()).padStart(2,"0")};return t.replace(/yyyy|MMM|MM|dd|HH|mm|ss|d/g,s=>r[s])}function Z8(e,t){return(t?.[e]??t?.[e===1?"one":"other"]??"").replace(/\{\}/g,e??0)}var sd={name:"htmlDoc",props:["content","baseUrl","allowScripts"],setup(e){let t=xe(null);return $t(()=>{let r=[t.value][0];r.classList.add("html-doc");function s(){r.innerHTML="";let i=e.content;if(typeof i!="string")return;let a=document.createElement("iframe");a.className="html-doc-frame",a.setAttribute("title","Rendered HTML document");let u=["allow-popups","allow-popups-to-escape-sandbox"];e.allowScripts&&u.push("allow-scripts","allow-forms","allow-modals"),a.setAttribute("sandbox",u.join(" ")),a.setAttribute("referrerpolicy","no-referrer"),r.appendChild(a);let m="";e.baseUrl&&(m='<base href="'+e.baseUrl.replace(/&/g,"&amp;").replace(/"/g,"&quot;")+'">'),a.srcdoc=m+i}He(()=>e.content,s,{immediate:!0}),He(()=>e.baseUrl,s,{immediate:!0}),He(()=>e.allowScripts,s,{immediate:!0})}),()=>be("html-doc",{ref:t})}};var rd={name:"pdfviewer",props:["src"],setup(e){let t=xe(null);return $t(()=>{let n=[t.value],r=400,s=[.5,.75,1,1.25,1.5,2,3],i=1e3,a=n[0];a.classList.add("pdf-viewer");let u=null,m=[],k=!1,y=null,E=1,h=1,C=0,g=0,b=612/792,S=0,R=null,T=new Map,q=0;function M(p){let v=T.get(p);v&&(v.cancelled=!0,v.task?.cancel?.(),T.delete(p));let D=p.querySelector("canvas");D&&(D.width=0,D.height=0),p.innerHTML="",delete p.dataset.state}let V=document.createElement("div");V.className="pdf-toolbar";let z=document.createElement("div");z.className="pdf-pages",z.tabIndex=0,z.setAttribute("aria-label","PDF pages"),a.appendChild(V),a.appendChild(z);function ne(p,v,D){let O=document.createElement("button");return O.type="button",O.className="pdf-toolbar-btn",O.title=v,O.setAttribute("aria-label",v),O.innerHTML='<i class="fas '+p+'"></i>',O.addEventListener("click",D),O}let ee=ne("fa-chevron-up","Previous page",function(){qe(h-1)}),_=ne("fa-chevron-down","Next page",function(){qe(h+1)}),P=document.createElement("input");P.type="text",P.className="pdf-toolbar-page",P.setAttribute("aria-label","Page number"),P.title="Page number \u2014 type a page and press Enter",P.addEventListener("keydown",function(p){p.key==="Enter"&&(qe(parseInt(P.value,10)),P.blur())}),P.addEventListener("blur",function(){P.value=String(h)});let G=document.createElement("span");G.className="pdf-toolbar-count";let J=ne("fa-search-minus","Zoom out",function(){te(s[Math.max(0,de()-1)])}),X=ne("fa-search-plus","Zoom in",function(){te(s[Math.min(s.length-1,de()+1)])}),W=document.createElement("button");W.type="button",W.className="pdf-toolbar-btn pdf-toolbar-zoom",W.title="Reset zoom to fit the width",W.setAttribute("aria-label","Reset zoom to fit the width"),W.addEventListener("click",function(){te(1)}),V.appendChild(ee),V.appendChild(_);let se=document.createElement("span");se.className="pdf-toolbar-group",se.appendChild(P),s Line truncated
jane.smith@example.org`},xq={id:"termsHelp",class:"form-text text-muted"},eM={class:"invalid-feedback"},tM={id:"settings-display",class:"paper-settings-section"},oM={class:"form-group paper-check-list"},nM={class:"form-check"},sM={class:"form-check-input",type:"checkbox",id:"link",name:"link"},rM={class:"form-check"},iM={class:"form-check-input",type:"checkbox",id:"image",name:"image"},aM={class:"form-check"},lM={class:"form-check-input",type:"checkbox",id:"pdf",name:"pdf"},dM={class:"form-check"},uM={class:"form-check-input",type:"checkbox",id:"notebook",name:"notebook"},cM={id:"settings-features",class:"paper-settings-section"},pM={class:"form-group paper-check-list"},fM={class:"form-check"},mM={class:"form-check-input",type:"checkbox",id:"page",name:"page"},hM={class:"form-check"},vM={class:"form-check-input",type:"checkbox",id:"loc",name:"loc"},yM={class:"form-check"},gM={class:"form-check-input",type:"checkbox",id:"update",name:"update"},bM={class:"form-group"},wM={class:"form-control",id:"mode",name:"mode"},kM={class:"form-text text-muted"},EM={id:"settings-expiration",class:"paper-settings-section"},CM={class:"form-group"},_M={class:"form-control",id:"expiration",name:"expiration"},SM={key:0,class:"draft-status",role:"status"},NM={key:1,class:"draft-status"},RM={class:"paper-settings-footer"},DM={key:0,class:"paper-inline-alert paper-inline-alert-error",role:"alert"},TM=["textContent"],AM={class:"paper-settings-actions"},OM=["disabled"],IM={key:0,class:"ui-icon mr-1","aria-hidden":"true"},PM={id:"settings-account",class:"paper-settings-section"},VM={class:"paper-account-card"},qM=["src"],MM={class:"paper-account-identity"},UM={class:"paper-account-name"},$M=["disabled"],LM={key:0,role:"status"},FM={key:1,class:"paper-inline-alert paper-inline-alert-error",role:"alert"},zM=["disabled"],HM={key:3,role:"status"},BM={class:"paper-danger-zone"},jM={key:0,class:"paper-inline-alert paper-inline-alert-error",role:"alert"},GM=["textContent"],WM=["disabled"];function Ql(e,t){let n=Ge("quota-usage"),r=ve("paper-scrollspy"),s=ve("field"),i=ve("form");return l(),d("div",Bq,[t[58]||(t[58]=de('<div class="paper-crumbs"><a href="/dashboard">My work</a> \xA0/\xA0 <span class="here">Settings</span></div><h1 class="paper-page-title">Your <em>settings</em></h1><p class="paper-page-lede">Defaults applied to every new anonymization, your quotas, and your account.</p>',3)),o("div",jq,[R((l(),d("aside",Gq,[...t[8]||(t[8]=[de('<div class="paper-settings-toc-head">On this page</div><nav><a href="#settings-quota">Quota</a><a href="#settings-terms">Terms to redact</a><a href="#settings-display">Display</a><a href="#settings-features">Features</a><a href="#settings-expiration">Expiration</a><a href="#settings-account">Account</a></nav>',2)])])),[[r]]),o("div",Wq,[f(" Account usage shares the dashboard component. "),o("section",Kq,[t[10]||(t[10]=o("div",{class:"paper-section-eyebrow"},"Quota",-1)),t[11]||(t[11]=o("p",{class:"paper-section-copy"},"What you are using across all your anonymizations. Conferences can lift these limits during a review window.",-1)),e.quota?(l(),Lt(n,{key:0,quota:e.quota,fmt:e.fmt},null,8,["quota","fmt"])):f("v-if",!0),e.quota?f("v-if",!0):(l(),d("div",Yq,[(l(),d(W,null,oe([1,2,3],(a,u)=>o("div",Jq,[...t[9]||(t[9]=[o("div",{class:"quota-header"},[o("span",{class:"skeleton skeleton-line",style:{width:"40%"}}),o("span",{class:"skeleton skeleton-line",style:{width:"25%"}})],-1),o("div",{class:"quota-track"},null,-1)])])),64))]))]),R((l(),d("form",{class:"form needs-validation",name:"defaultsForm",novalidate:"",onSubmit:t[2]||(t[2]=ie(a=>e.submitForm(a,()=>{e.saveDefault(a)}),["prevent"]))},[o("section",Qq,[t[17]||(t[17]=o("div",{class:"paper-section-eyebrow"},"Terms to redact",-1)),t[18]||(t[18]=o("p",{class:"paper-section-copy"},"Pre-filled in every new anonymization. You can still edit the list per anonymization.",-1)),o("div",Xq,[t[16]||(t[16]=o("label",{class:"paper-field-label",for:"terms"},"Default terms",-1)),R(o("textarea",Zq,null,512),[[s,{state:e.viewState,set:a=>{e.terms=a},value:e.terms,form:"defaultsForm",options:{debounce:250}}]]),o("small",xq,[t[12]||(t[12]=b(" One term per line (regex allowed). Each match is replaced by ",-1)),o("code",null,c(e.site_options?.ANONYMIZATION_MASK||"XXX")+"-[N]",1),t[13]||(t[13]=b(", or use ",-1)),t[14]||(t[14]=o("code",null,"term=>replacement",-1)),t[15]||(t[15]=b(" to pick your own. ",-1))]),R(o("div",eM," Terms are in an invalid format ",512),[[j,e.defaultsForm?.terms?.errors?.format]])])]),o("section",tM,[t[27]||(t[27]=o("div",{class:"paper-section-eyebrow"},"Display",-1)),t[28]||(t[28]=o("p",{class:"paper-section-copy"},"What reviewers can see inside an anonymized repository.",-1)),o("div",oM,[o("div",nM,[R(o("input",sM,null,512),[[s,{state:e.viewState,set:a=>{e.options.link=a},value:e.options?.link,form:"defaultsForm",options:{}}]]),t[19]||(t[19]=o("label",{class:"form-check-label",for:"link"},"Keep links",-1)),t[20]||(t[20]=o("small",{class:"form-text t Line truncated
`);for(let k=0;k<g.length;k++){let h=g[k];if(h.startsWith("diff --git")){w(),m();continue}if(h.startsWith("--- ")){i&&i.lines.length&&w(),m().oldPath=h.replace(/^--- (a\/)?/,"").trim();continue}if(h.startsWith("+++ ")){m().newPath=h.replace(/^\+\+\+ (b\/)?/,"").trim();continue}if(!(h.startsWith("index ")||h.startsWith("similarity index")||h.startsWith("rename ")||h.startsWith("new file mode")||h.startsWith("deleted file mode")||h.startsWith("Binary files"))){if(h.startsWith("@@")){let E=h.match(/@@\s+-(\d+)(?:,\d+)?\s+\+(\d+)(?:,\d+)?\s+@@/);E&&(a=parseInt(E[1],10),u=parseInt(E[2],10)),m().lines.push({kind:"hunk",oldNo:"",newNo:"",text:h});continue}i&&(h.startsWith("+")?(i.lines.push({kind:"add",oldNo:"",newNo:u,text:h.slice(1)}),u++):h.startsWith("-")?(i.lines.push({kind:"remove",oldNo:a,newNo:"",text:h.slice(1)}),a++):(i.lines.push({kind:"ctx",oldNo:a,newNo:u,text:h.startsWith(" ")?h.slice(1):h}),a++,u++))}}return n(s,i),s.join("")}})({trustAsHtml:e=>e});function d6(e,t){return e==null?"":Number(e).toLocaleString(void 0,t==null?{}:{minimumFractionDigits:t,maximumFractionDigits:t})}function u6(e){return String(e??"").toUpperCase()}function c6(e,t,n=0){return(e||[]).slice(n,n+t)}function p6(e,t){return Array.isArray(e)?typeof t=="function"?e.filter(t):e:[]}function f6(e,t){if(!t)return e||[];let n=Array.isArray(t)?t:[t];return[...e||[]].sort((r,s)=>{for(let i of n){let a=i[0]==="-"?-1:1;i=i.replace(/^[+-]/,"");let u=i.split(".").reduce((w,g)=>w?.[g],r),m=i.split(".").reduce((w,g)=>w?.[g],s);if(typeof u=="string"&&(u=u.toLowerCase()),typeof m=="string"&&(m=m.toLowerCase()),u!==m)return(u==null?-1:m==null?1:u<m?-1:1)*a}return 0})}function m6(e,t){if(!e)return"";let n=new Date(e);if(isNaN(n))return String(e);if(!t||t==="mediumDate")return n.toLocaleDateString(void 0,{year:"numeric",month:"short",day:"numeric"});let r={yyyy:n.getFullYear(),MMM:n.toLocaleDateString(void 0,{month:"short"}),MM:String(n.getMonth()+1).padStart(2,"0"),dd:String(n.getDate()).padStart(2,"0"),d:n.getDate(),HH:String(n.getHours()).padStart(2,"0"),mm:String(n.getMinutes()).padStart(2,"0"),ss:String(n.getSeconds()).padStart(2,"0")};return t.replace(/yyyy|MMM|MM|dd|HH|mm|ss|d/g,s=>r[s])}function h6(e,t){return(t?.[e]??t?.[e===1?"one":"other"]??"").replace(/\{\}/g,e??0)}var sd={name:"htmlDoc",props:["content","baseUrl","allowScripts"],setup(e){let t=xe(null);return $t(()=>{let r=[t.value][0];r.classList.add("html-doc");function s(){r.innerHTML="";let i=e.content;if(typeof i!="string")return;let a=document.createElement("iframe");a.className="html-doc-frame",a.setAttribute("title","Rendered HTML document");let u=["allow-popups","allow-popups-to-escape-sandbox"];e.allowScripts&&u.push("allow-scripts","allow-forms","allow-modals"),a.setAttribute("sandbox",u.join(" ")),a.setAttribute("referrerpolicy","no-referrer"),r.appendChild(a);let m="";e.baseUrl&&(m='<base href="'+e.baseUrl.replace(/&/g,"&amp;").replace(/"/g,"&quot;")+'">'),a.srcdoc=m+i}He(()=>e.content,s,{immediate:!0}),He(()=>e.baseUrl,s,{immediate:!0}),He(()=>e.allowScripts,s,{immediate:!0})}),()=>be("html-doc",{ref:t})}};var rd={name:"pdfviewer",props:["src"],setup(e){let t=xe(null);return $t(()=>{let n=[t.value],r=400,s=[.5,.75,1,1.25,1.5,2,3],i=1e3,a=n[0];a.classList.add("pdf-viewer");let u=null,m=[],w=!1,g=null,k=1,h=1,E=0,S=0,v=612/792,_=0,N=null,T=new Map,V=0;function M(p){let y=T.get(p);y&&(y.cancelled=!0,y.task?.cancel?.(),T.delete(p));let D=p.querySelector("canvas");D&&(D.width=0,D.height=0),p.innerHTML="",delete p.dataset.state}let q=document.createElement("div");q.className="pdf-toolbar";let z=document.createElement("div");z.className="pdf-pages",z.tabIndex=0,z.setAttribute("aria-label","PDF pages"),a.appendChild(q),a.appendChild(z);function ne(p,y,D){let O=document.createElement("button");return O.type="button",O.className="pdf-toolbar-btn",O.title=y,O.setAttribute("aria-label",y),O.innerHTML='<i class="fas '+p+'"></i>',O.addEventListener("click",D),O}let ee=ne("fa-chevron-up","Previous page",function(){qe(h-1)}),C=ne("fa-chevron-down","Next page",function(){qe(h+1)}),P=document.createElement("input");P.type="text",P.className="pdf-toolbar-page",P.setAttribute("aria-label","Page number"),P.title="Page number \u2014 type a page and press Enter",P.addEventListener("keydown",function(p){p.key==="Enter"&&(qe(parseInt(P.value,10)),P.blur())}),P.addEventListener("blur",function(){P.value=String(h)});let G=document.createElement("span");G.className="pdf-toolbar-count";let J=ne("fa-search-minus","Zoom out",function(){te(s[Math.max(0,ue()-1)])}),X=ne("fa-search-plus","Zoom in",function(){te(s[Math.min(s.length-1,ue()+1)])}),K=document.createElement("button");K.type="button",K.className="pdf-toolbar-btn pdf-toolbar-zoom",K.title="Reset zoom to fit the width",K.setAttribute("aria-label","Reset zoom to fit the width"),K.addEventListener("click",function(){te(1)}),q.appendChild(ee),q.appendChild(C);let se=document.createElement("span");se.className="pdf-toolbar-group",se.appendChild(P),s Line truncated
/*! Bundled license information:
@vue/shared/dist/shared.esm-bundler.js:
+60
View File
@@ -0,0 +1,60 @@
/* Synthetic metadata cardinality and bounded eviction probes. No production access. */
const { strict: assert } = require('node:assert');
const { promises: fs } = require('node:fs');
const { tmpdir } = require('node:os');
const { join, dirname } = require('node:path');
const { Readable } = require('node:stream');
const { performance } = require('node:perf_hooks');
require('ts-node/register/transpile-only');
const config = require('../src/config').default;
const { BlobCache } = require('../src/core/blob-cache');
const percentile=(values,p)=>[...values].sort((a,b)=>a-b)[Math.min(values.length-1,Math.ceil(values.length*p)-1)];
const consume=async stream=>{let size=0;for await(const chunk of stream)size+=chunk.length;return size;};
async function main(){
const root=await fs.mkdtemp(join(tmpdir(),'cache-index-performance-'));config.CACHE_MONITORING_ENABLED=false;
const results=[];let cache;
try{
for(const cardinality of [0,10000,100000]){
cache=new BlobCache({root:join(root,'index-'+cardinality),budget:1024**3,maxObject:4096,gzip:false});
const real=await cache.getOrFill('real',async()=>Readable.from('verified real content'));const digest=real.cachedBlob.record.digest;await consume(real);
const now=Date.now();cache.db.exec('BEGIN');
const blob=cache.db.prepare('INSERT INTO blobs VALUES(?,?,?,?,?,?,?)');
const alias=cache.db.prepare('INSERT INTO aliases VALUES(?,?)');
const decision=cache.db.prepare('INSERT INTO decisions(key,digest,changed,accessed,expires) VALUES(?,?,?,?,?)');
const history=cache.db.prepare('INSERT INTO history VALUES(?,?,0)');
for(let i=0;i<cardinality;i++){
// Metadata-only dummy rows, never opened. Real lookups/fills use the separate verified content above.
const key='sha256-'+i.toString(16).padStart(64,'0');blob.run(key,'dead.raw',4096,4096,'identity',now,4096);
alias.run('seed-'+i,digest);alias.run('alias-'+i,digest);decision.run('rule-'+i,digest,0,now,now+86400000);history.run('history-'+i,now);
}
cache.db.exec('COMMIT');
const stats=[],hits=[],fills=[],decisions=[];
for(let i=0;i<30;i++){
let start=performance.now();cache.stats();stats.push(performance.now()-start);
start=performance.now();assert.equal(await consume(await cache.getOrFill(digest,async()=>{throw Error('unexpected miss');})),21);hits.push(performance.now()-start);
start=performance.now();cache.saveDecision('measured-'+i,digest,false);decisions.push(performance.now()-start);
start=performance.now();await consume(await cache.getOrFill('new-'+i,async()=>Readable.from('verified real content')));fills.push(performance.now()-start);
}
const summarize=x=>({p50Ms:percentile(x,.5),p95Ms:percentile(x,.95),maxMs:Math.max(...x)});
results.push({seedBlobs:cardinality,seedAliases:cardinality*2,seedDecisions:cardinality,seedHistory:cardinality,
adminStats:summarize(stats),warmSourceHit:summarize(hits),decisionWrite:summarize(decisions),smallFill:summarize(fills)});
cache.close();cache=undefined;
}
cache=new BlobCache({root:join(root,'eviction'),budget:4*1024**2,maxObject:65536,gzip:false});
const times=[];let upstream=0,peakAllocated=0;const started=performance.now();
for(let i=0;i<300;i++){
// 100 distinct objects, three sequential traversals: deliberately larger than the pool.
const id=i%100,data=Buffer.alloc(65536,id);const begin=performance.now();
await consume(await cache.getOrFill('object-'+id,async()=>{upstream++;return Readable.from(data);}));times.push(performance.now()-begin);
const stats=cache.stats();peakAllocated=Math.max(peakAllocated,stats.allocatedBytes);assert.ok(stats.allocatedBytes+stats.reservedBytes<=stats.budgetBytes);
}
const report={measuredAt:new Date().toISOString(),scope:'Synthetic metadata-only scale probe. Dummy blob rows have no payload files; excludes directory scanning and disk reads for those rows. Real reads use one verified blob. Eviction probe uses real 64 KiB files.',
node:process.version,cardinality:results,eviction:{requests:300,distinctObjects:100,upstreamFills:upstream,hitRate:1-upstream/300,
elapsedMs:performance.now()-started,p50Ms:percentile(times,.5),p95Ms:percentile(times,.95),peakAllocated,final:cache.stats()},assertions:'passed'};
const output = process.argv[2] || 'benchmark-results/cache/index.json';
await fs.mkdir(dirname(output), { recursive: true });
await fs.writeFile(output,JSON.stringify(report,null,2)+'\n');
process.stdout.write(JSON.stringify(report,null,2)+'\n');
}finally{cache?.close();await fs.rm(root,{recursive:true,force:true});}
}
main().catch(error=>{console.error(error);process.exitCode=1;});
+142
View File
@@ -0,0 +1,142 @@
/* Local HTTP benchmark. Uses temporary files and deterministic synthetic content, never production. */
const { strict: assert } = require('node:assert');
const { promises: fs, createReadStream } = require('node:fs');
const { tmpdir, cpus, totalmem, loadavg } = require('node:os');
const { join, dirname } = require('node:path');
const { createHash } = require('node:crypto');
const { Readable } = require('node:stream');
const { pipeline } = require('node:stream/promises');
const { createGzip, gunzipSync } = require('node:zlib');
const { performance, monitorEventLoopDelay } = require('node:perf_hooks');
const { setTimeout: delay } = require('node:timers/promises');
const { setInterval, clearInterval } = require('node:timers');
const { fork } = require('node:child_process');
const http = require('node:http');
const percentile = (values, p) => [...values].sort((a,b) => a-b)[Math.min(values.length-1, Math.ceil(values.length*p)-1)];
function fixture(size, binary) {
const chunks = []; let length = 0;
for (let i=0; length<size; i++) {
const digest = createHash('sha256').update('cache-evaluation-' + i).digest();
const chunk = binary ? digest : Buffer.from(`${i}: ResearcherAlice record ${digest.toString('hex')} module/path/${i % 137}.ts\n`);
chunks.push(chunk); length += chunk.length;
}
return Buffer.concat(chunks).subarray(0,size);
}
async function worker(spec) {
let privateTmp;
if(spec.mode==='legacy-transform'){privateTmp=await fs.mkdtemp(join(tmpdir(),'cache-legacy-bench-'));process.env.TMPDIR=privateTmp;}
if(process.env.CACHE_BENCH_COMPILED !== '1') require('ts-node/register/transpile-only');
const code=process.env.CACHE_BENCH_COMPILED === '1' ? '../build' : '../src';
const express = require('express');
const config = require(code+'/config').default;
const { sharedBlobCache, resultBlobCache, closeBlobCaches } = require(code+'/core/blob-cache');
const { sendSharedContent } = require(code+'/core/cached-response');
const { AnonymizeTransformer } = require(code+'/core/anonymize-utils');
const { closeCacheMetrics, cacheMetricsReport } = require(code+'/core/cache-observability');
const root = await fs.mkdtemp(join(tmpdir(), 'cache-performance-'));
const body = fixture(spec.size, spec.binary);
const expected = spec.changed ? Buffer.from(body.toString().replaceAll('ResearcherAlice', 'ResearcherHidden')) : body;
const digest = 'git-' + createHash('sha1').update(`blob ${body.length}\0`).update(body).digest('hex');
const raw = join(root, 'source.raw'); let sourceReady = false, upstreamFills = 0, server;
Object.assign(config, { FOLDER:root, SHARED_CACHE_PATH:join(root,'pool'), SHARED_CACHE_ENABLED:spec.mode==='shared',
SHARED_CACHE_BYTES:64*1024**2, SHARED_CACHE_RESULT_BYTES:32*1024**2, SHARED_CACHE_MAX_FILE_BYTES:4*1024**2,
SHARED_CACHE_FREE_BYTES:0, CACHE_MONITORING_ENABLED:true, REDIS_HOSTNAME:'127.0.0.1', REDIS_PORT:1 });
if (spec.sqliteNormal) { sharedBlobCache().db.exec('PRAGMA synchronous=NORMAL'); resultBlobCache().db.exec('PRAGMA synchronous=NORMAL'); }
const options = { filePath:spec.binary?'fixture.bin':'fixture.txt', repoId:'performance-project',
terms:spec.changed?['ResearcherAlice=>ResearcherHidden']:['AbsentResearcher'], decisionManaged:spec.mode!=='legacy-transform' };
const obtain = async () => {
if (spec.mode === 'uncached') { upstreamFills++; await delay(20); return Readable.from(body); }
if (spec.mode === 'raw-source' || spec.mode === 'legacy-transform') {
if (!sourceReady) { upstreamFills++; await delay(20); await fs.writeFile(raw,body); sourceReady=true; }
return createReadStream(raw);
}
return sharedBlobCache().getOrFill(digest, async()=>{upstreamFills++;await delay(20);return Readable.from(body);},
async proof=>{assert.equal('git-'+proof.gitSha,digest);return digest;},body.length);
};
const app=express(); app.get('/file',async(_req,res)=>{
res.type(spec.binary?'application/octet-stream':'text/plain');
try {
const source=await obtain();
if (spec.mode==='shared' && await sendSharedContent(source,res,options)) return;
const stages=[source,new AnonymizeTransformer(options)];
if(spec.encoding==='gzip'){res.setHeader('Content-Encoding','gzip');stages.push(createGzip({level:6}));}
await pipeline(...stages,res);
}catch(error){res.destroy(error);}
});
const agent=new http.Agent({keepAlive:true,maxSockets:8});
let wireHash;
try {
await new Promise(resolve=>{server=app.listen(0,'127.0.0.1',resolve);});
const request=()=>new Promise((resolve,reject)=>{
const start=performance.now();
http.get({host:'127.0.0.1',port:server.address().port,path:'/file',agent,headers:{'accept-encoding':spec.encoding}},res=>{
const ttfb=performance.now()-start, chunks=[]; let bytes=0;const sha=createHash('sha256');
res.on('data',b=>{bytes+=b.length;sha.update(b);if(!wireHash)chunks.push(b);});
res.on('error',reject);res.on('end',()=>{
try {
assert.equal(res.statusCode,200); const digest=sha.digest('hex');
const elapsed=performance.now()-start;
if(!wireHash){const data=Buffer.concat(chunks);assert.deepEqual(res.headers['content-encoding']==='gzip'?gunzipSync(data):data,expected);wireHash=digest;}
else assert.equal(digest,wireHash);
resolve({ms:elapsed,ttfb,bytes});
}catch(error){reject(error);}
});
}).on('error',reject);
});
const coldCpu=process.cpuUsage(),coldStart=performance.now();const cold=await request();
const coldResources={...cold,wallMs:performance.now()-coldStart,cpuMs:Object.values(process.cpuUsage(coldCpu)).reduce((a,b)=>a+b,0)/1000};
// JIT and worker warmup is excluded from warm samples.
for(let i=0;i<3;i++) await request();
const batches=[];
for(const concurrency of [1,8]) for(let repeat=0;repeat<2;repeat++) {
const count=concurrency===1?16:32;let next=0;const rows=[];
let peakRss=process.memoryUsage().rss;
const timer=setInterval(()=>{peakRss=Math.max(peakRss,process.memoryUsage().rss);},5);
const eventLoop=monitorEventLoopDelay({resolution:10});eventLoop.enable();await delay(20);
const cpu=process.cpuUsage(), start=performance.now();
await Promise.all(Array.from({length:concurrency},async()=>{for(;;){const i=next++;if(i>=count)return;rows.push(await request());}}));
const elapsed=performance.now()-start,cpuUsed=process.cpuUsage(cpu);eventLoop.disable();clearInterval(timer);
batches.push({concurrency,repeat,requests:count,elapsedMs:elapsed,requestsPerSecond:count*1000/elapsed,
p50Ms:percentile(rows.map(r=>r.ms),.5),p95Ms:percentile(rows.map(r=>r.ms),.95),maxMs:Math.max(...rows.map(r=>r.ms)),
ttfbP95Ms:percentile(rows.map(r=>r.ttfb),.95),cpuMs:(cpuUsed.user+cpuUsed.system)/1000,
cpuMsPerRequest:(cpuUsed.user+cpuUsed.system)/1000/count,peakRssMiB:Math.max(peakRss,process.memoryUsage().rss)/1048576,
eventLoopP95Ms:eventLoop.count?eventLoop.percentile(95)/1e6:null,wireBytes:rows.reduce((n,r)=>n+r.bytes,0),latenciesMs:rows.map(r=>r.ms)});
}
const metrics=cacheMetricsReport(60);
const report={...spec,cold:coldResources,batches,upstreamFills,source:sharedBlobCache()?.stats()||null,results:resultBlobCache()?.stats()||null,
metrics:metrics.rows,wireBytesPerResponse:cold.bytes,uncompressedBytes:expected.length,verifiedResponses:100};
process.send({report});
} finally {agent.destroy();if(server)await new Promise(resolve=>server.close(resolve));closeBlobCaches();closeCacheMetrics();await fs.rm(root,{recursive:true,force:true});if(privateTmp)await fs.rm(privateTmp,{recursive:true,force:true});}
}
async function main() {
let scenarios=[];
for(const size of [16384,262144,2097152])for(const changed of [false,true])for(const mode of ['uncached','raw-source','shared'])scenarios.push({size,changed,mode,encoding:'gzip',binary:false});
for(const size of [262144,2097152])scenarios.push({size,changed:true,mode:'shared',encoding:'identity',binary:false});
for(const mode of ['raw-source','shared'])scenarios.push({size:262144,changed:false,mode,encoding:'gzip',binary:true});
if (process.argv.includes('--shared')) scenarios=scenarios.filter(s=>s.mode==='shared' && s.changed && s.encoding==='gzip');
if (process.argv.includes('--normal')) scenarios=[16384,262144,2097152].map(size=>({size,changed:true,mode:'shared',encoding:'gzip',binary:false,sqliteNormal:true}));
if (process.argv.includes('--legacy')) scenarios=[false,true].map(changed=>({size:2097152,changed,mode:'legacy-transform',encoding:'gzip',binary:false}));
const report={measuredAt:new Date().toISOString(),environment:{execution:process.env.CACHE_BENCH_COMPILED === '1' ? 'compiled JavaScript' : 'ts-node',node:process.version,platform:process.platform,arch:process.arch,cpu:cpus()[0]?.model,logicalCpus:cpus().length,totalMemoryGiB:totalmem()/1024**3,loadAverage:loadavg()},
methodology:{scope:'Local loopback HTTP, deterministic synthetic content, isolated process per scenario; no production traffic',
upstreamDelayMs:20,rawSourceBaseline:'Raw local file; anonymization and gzip repeated every request; no transformed-result cache. Not a complete legacy deployment comparison.',
measurements:'Cold request then 3 warmups, 2 batches each at concurrency 1 (16 requests) and 8 (32 requests). CPU and RSS include server, loopback client, hashing and Node runtime; worker-thread CPU included. First cold request also includes worker startup when applicable.',
limitations:'Warm OS page cache; no TLS, proxy, GitHub bandwidth, database auth or real-world RTT. Small sample p95 is descriptive, not a capacity SLA. Fresh metadata indexes; not a full-size index test.'},scenarios:[]};
for(const spec of scenarios){
process.stderr.write(`Measuring ${JSON.stringify(spec)}\n`);
report.scenarios.push(await new Promise((resolve,reject)=>{
const child=fork(__filename,['--worker',JSON.stringify(spec)],{env:{...process.env,NODE_ENV:'test'},stdio:['ignore','ignore','pipe','ipc']});
let result,stderr='';const timeout=setInterval(()=>{child.kill();reject(Error('Scenario exceeded 120 seconds'));},120000);
child.stderr.on('data',b=>{stderr=(stderr+b).slice(-4000);});child.on('message',m=>{result=m.report;});child.on('error',reject);
child.on('exit',code=>{clearInterval(timeout);if(code===0&&result)resolve(result);else reject(Error(`Scenario failed ${code}: ${stderr}`));});
}));
}
report.completedAt=new Date().toISOString();report.assertions=`All ${scenarios.length*100} response bodies verified against expected output or the validated wire digest`;
if (process.argv.includes('--normal')) report.methodology.experimentalOverride='PRAGMA synchronous=NORMAL in both temporary pool connections only. Application defaults are unchanged. WAL commits may be lost after power failure.';
if (process.argv.includes('--legacy')) report.methodology.legacyComparison='Existing large-file transformedFile cache enabled, isolated TMPDIR, local source file, no Redis coordination or database authorization.';
const output = process.argv[2] || 'benchmark-results/cache/http.json';
await fs.mkdir(dirname(output), { recursive: true });
await fs.writeFile(output,JSON.stringify(report,null,2)+'\n');
process.stdout.write(JSON.stringify({scenarios:report.scenarios.length,assertions:report.assertions})+'\n');
}
if(process.argv[2]==='--worker')worker(JSON.parse(process.argv[3])).then(()=>process.exit(0),e=>{console.error(e);process.exit(1);});
else main().catch(e=>{console.error(e);process.exitCode=1;});
+70
View File
@@ -0,0 +1,70 @@
/* Reproducible local acceptance workload. Never connects to production or downloads upstream files. */
const { strict: assert } = require("node:assert");
const { promises: fs } = require("node:fs");
const { tmpdir } = require("node:os");
const { join } = require("node:path");
const { Readable } = require("node:stream");
const { createHash } = require("node:crypto");
const http = require("node:http");
const express = require("express");
require("ts-node/register/transpile-only");
const config = require("../src/config").default;
const { BlobCache, sharedBlobCache, resultBlobCache, closeBlobCaches } = require("../src/core/blob-cache");
const { sendSharedContent } = require("../src/core/cached-response");
const { cacheMetricsReport, closeCacheMetrics } = require("../src/core/cache-observability");
const { monitorRequests } = require("../src/core/request-monitoring");
const consume = async s => { const chunks = []; for await (const b of s) chunks.push(Buffer.from(b)); return Buffer.concat(chunks); };
async function main() {
const root = await fs.mkdtemp(join(tmpdir(), "cache-workload-")); let server, eviction;
const old = { ...config };
Object.assign(config, { SHARED_CACHE_ENABLED: true, SHARED_CACHE_PATH: root, SHARED_CACHE_BYTES: 1024 * 1024,
SHARED_CACHE_RESULT_BYTES: 1024 * 1024, SHARED_CACHE_MAX_FILE_BYTES: 32768, SHARED_CACHE_FREE_BYTES: 0, CACHE_MONITORING_ENABLED: true });
try {
const source = sharedBlobCache();
const content = Buffer.from("A public example with repeatable documentation.\n".repeat(400));
const key = "git-" + createHash("sha1").update(`blob ${content.length}\0`).update(content).digest("hex");
let fills = 0;
const obtain = () => source.getOrFill(key, async () => { fills++; return Readable.from(content); }, async p => {
assert.equal(key, "git-" + p.gitSha); return key;
}, content.length);
const app = express(); app.use(monitorRequests("api", undefined, { info() {}, warn() {} }));
app.get("/:project/:rule", async (req, res) => {
res.type("text/plain");
try { assert.equal(await sendSharedContent(await obtain(), res, { filePath: "README.md", repoId: req.params.project,
terms: req.params.rule === "changed" ? ["public=>private"] : ["Alice"] }), true); }
catch (e) { res.destroy(e); }
});
await new Promise(resolve => { server = app.listen(0, "127.0.0.1", resolve); });
const responses = [];
for (const path of ["/a/plain", "/b/plain", "/a/plain", "/a/changed", "/a/changed"]) {
responses.push(await new Promise((resolve, reject) => {
http.get({ host: "127.0.0.1", port: server.address().port, path, headers: { "accept-encoding": "gzip" } }, res => {
consume(res).then(b => resolve({ status: res.statusCode, encoding: res.headers["content-encoding"], bytes: b.length }), reject);
}).on("error", reject);
}));
}
assert.equal(fills, 1); assert.ok(responses.every(r => r.status === 200 && r.encoding === "gzip"));
eviction = new BlobCache({ root: join(root, "eviction"), budget: 65536, maxObject: 4096, gzip: false, name: "workload" });
for (let i = 0; i < 30; i++) {
const data = Buffer.alloc(4000, i);
await consume(await eviction.getOrFill(`item-${i}`, async () => Readable.from(data)));
const stats = eviction.stats(); assert.ok(stats.allocatedBytes + stats.reservedBytes <= stats.budgetBytes);
}
const stats = eviction.stats(); assert.ok(stats.blobs < 30);
const metrics = cacheMetricsReport(60);
const count = name => metrics.rows.filter(r => r.name === name).reduce((n, r) => n + r.count, 0);
assert.equal(count("decision_hit"), 2); assert.equal(count("rewrite_changed"), 1);
const result = { node: process.version, measuredAt: new Date().toISOString(), scope: "Synthetic local acceptance workload; not a production projection",
sourceRequests: 5, upstreamFills: fills, source: source.stats(), results: resultBlobCache().stats(), eviction: stats,
decisionsReused: count("decision_hit"), unchangedDecisionsComputed: count("rewrite_unchanged"), changedOutputsComputed: count("rewrite_changed"),
compressionSavingPercent: 100 * (1 - source.stats().storedBytes / source.stats().originalBytes), responses, metrics: metrics.rows,
assertions: "passed" };
const json = JSON.stringify(result, null, 2) + "\n";
if (process.argv[2]) await fs.writeFile(process.argv[2], json);
process.stdout.write(json);
} finally {
if (server) await new Promise(resolve => server.close(resolve));
eviction?.close(); closeBlobCaches(); closeCacheMetrics(); Object.assign(config, old); await fs.rm(root, { recursive: true, force: true });
}
}
main().catch(e => { console.error(e); process.exitCode = 1; });
+26
View File
@@ -4,6 +4,19 @@ import { randomBytes } from "crypto";
interface Config {
RESEND_API_KEY: string;
EMAIL_FROM: string;
SHARED_CACHE_ENABLED: boolean;
SHARED_CACHE_PATH: string;
SHARED_CACHE_BYTES: number;
SHARED_CACHE_MAX_FILE_BYTES: number;
SHARED_CACHE_GZIP: boolean;
SHARED_CACHE_GZIP_MIN_SAVING: number;
SHARED_CACHE_FREE_BYTES: number;
SHARED_CACHE_LOW_WATER: number;
SHARED_CACHE_DECISIONS: boolean;
SHARED_CACHE_RESULT_BYTES: number;
CACHE_MONITORING_ENABLED: boolean;
GITHUB_APP_ENABLED: boolean;
GITHUB_APP_NEW_CONNECTIONS: boolean;
GITHUB_OAUTH_ENABLED: boolean;
@@ -71,6 +84,19 @@ interface Config {
const config: Config = {
RESEND_API_KEY: "",
EMAIL_FROM: "",
SHARED_CACHE_ENABLED: false,
SHARED_CACHE_PATH: resolve(process.env.FOLDER || resolve(__dirname, "..", "repositories"), ".shared-cache-v1"),
SHARED_CACHE_BYTES: 10 * 1024 ** 3,
SHARED_CACHE_MAX_FILE_BYTES: 32 * 1024 ** 2,
SHARED_CACHE_GZIP: true,
SHARED_CACHE_GZIP_MIN_SAVING: 0.10,
SHARED_CACHE_FREE_BYTES: 1024 ** 3,
SHARED_CACHE_LOW_WATER: 0.9,
SHARED_CACHE_DECISIONS: true,
SHARED_CACHE_RESULT_BYTES: 512 * 1024 ** 2,
CACHE_MONITORING_ENABLED: true,
GITHUB_APP_ENABLED: false,
GITHUB_APP_NEW_CONNECTIONS: true,
GITHUB_OAUTH_ENABLED: true,
+8 -1
View File
@@ -1,4 +1,5 @@
import { notifyOwnerAccessProblem } from "./owner-notifications";
import { sendSharedContent } from "./cached-response";
import { streamResponse } from "./response-stream";
import { join, basename, dirname } from "path";
import { Response } from "express";
@@ -454,7 +455,8 @@ export default class AnonymizedFile {
const resStream = got
.stream(join(config.STREAMER_ENTRYPOINT, "api"), {
method: "POST",
headers: requestHeaders(),
headers: { ...requestHeaders(), "accept-encoding": res.req?.get?.("accept-encoding") || "identity" },
decompress: false,
json: {
sha,
size,
@@ -477,6 +479,10 @@ export default class AnonymizedFile {
resStream.on("response", (upstream: { headers: Record<string, string | string[] | undefined> }) => {
headersDone();
if (res.headersSent) return;
for (const header of ["content-encoding", "content-length", "vary", "accept-ranges"]) {
const value = upstream.headers[header];
if (typeof value === "string") res.setHeader(header, value);
}
const ct = upstream.headers["content-type"];
if (typeof ct === "string") {
res.contentType(ct);
@@ -530,6 +536,7 @@ export default class AnonymizedFile {
});
const content = await this.content();
if (res.destroyed || res.writableEnded) { content.destroy(); anonymizer.destroy(); resolve(); return; }
if (await sendSharedContent(content, res, anonymizer.opt)) { anonymizer.destroy(); resolve(); return; }
function handleStreamError(error: Error) {
if (!content.closed && !content.destroyed) {
content.destroy();
+14 -3
View File
@@ -1,3 +1,5 @@
import { observeCache } from "./cache-observability";
import { performance } from "perf_hooks";
import { createHash } from "crypto";
import { transformedFile } from "./transformed-cache";
import { promises as fs, createReadStream, ReadStream } from "fs";
@@ -105,7 +107,7 @@ function isTextualMime(mime: string): boolean {
// Name-only classification: returns true (known text), false (known binary),
// or null when the name alone is inconclusive. The streaming transformer
// resolves null by sniffing the first chunk with isbinaryfile.
function classifyByName(filePath: string): boolean | null {
export function classifyByName(filePath: string): boolean | null {
const name = basename(filePath);
const extension = name.split(".").reverse()[0].toLowerCase();
if (config.additionalExtensions.includes(extension)) return true;
@@ -143,6 +145,7 @@ export class AnonymizeTransformer extends Transform {
anonimizer: ContentAnonimizer;
private wholeChunks: Buffer[] = [];
private wholeBytes = 0;
private passthroughBytes = 0;
private digest = createHash("sha256");
private spool?: string;
private reservedSpool = 0;
@@ -157,6 +160,7 @@ export class AnonymizeTransformer extends Transform {
filePath: string;
cacheGeneration?: string;
cacheRevision?: string;
decisionManaged?: boolean;
} & ConstructorParameters<typeof ContentAnonimizer>[0]
) {
super();
@@ -200,6 +204,7 @@ export class AnonymizeTransformer extends Transform {
return;
}
} else {
this.passthroughBytes += chunk.length;
this.emit("transform", { isText: this.isText, wasAnonimized: false, chunk });
this.push(chunk);
}
@@ -237,13 +242,17 @@ export class AnonymizeTransformer extends Transform {
}
_flush(callback: (error?: Error) => void) {
if (this.passthroughBytes) observeCache(this.isText ? "inspected_passthrough" : "inspected_binary", this.passthroughBytes);
if (this.spool) {
const output = join(this.spool, "output");
void transformedFile(this.digest.digest("hex"), this.opt, output, () => anonymizeOnWorker({ input: join(this.spool!, "input"), output,
options: this.opt, maxOutput: config.MAX_FILE_SIZE * 4, context: { mask: config.ANONYMIZATION_MASK, hostname: config.APP_HOSTNAME } }, this.wholeBytes, this.cancelled.signal))
const started = performance.now();
const produce = () => anonymizeOnWorker({ input: join(this.spool!, "input"), output,
options: this.opt, maxOutput: config.MAX_FILE_SIZE * 4, context: { mask: config.ANONYMIZATION_MASK, hostname: config.APP_HOSTNAME } }, this.wholeBytes, this.cancelled.signal);
void (this.opt.decisionManaged ? produce() : transformedFile(this.digest.digest("hex"), this.opt, output, produce))
.then(changed => {
if (this.destroyed) return;
this.anonimizer.wasAnonymized = changed;
observeCache(changed ? "inspected_changed" : "inspected_unchanged", this.wholeBytes, performance.now() - started);
this.output = createReadStream(output);
this.output.on("data", chunk => {
this.emit("transform", { isText: true, wasAnonimized: changed, chunk });
@@ -260,7 +269,9 @@ export class AnonymizeTransformer extends Transform {
const original = Buffer.concat(this.wholeChunks, this.wholeBytes);
this.wholeChunks = [];
const text = original.toString("utf8");
const started = performance.now();
const out = this.anonimizer.anonymize(text);
observeCache(out === text ? "inspected_unchanged" : "inspected_changed", original.length, performance.now() - started);
// Preserve the original encoding when no replacement was needed.
const chunk = out === text ? original : Buffer.from(out, "utf8");
this.emit("transform", { isText: true, wasAnonimized: this.wasAnonimized, chunk });
+423
View File
@@ -0,0 +1,423 @@
import { DatabaseSync } from "node:sqlite";
import { createHash, randomUUID } from "crypto";
import { createReadStream, createWriteStream, mkdirSync, unlinkSync, rmdirSync, statfsSync, promises as fs } from "fs";
import { join } from "path";
import { Readable, Transform } from "stream";
import { pipeline } from "stream/promises";
import { createGzip, createGunzip } from "zlib";
import { performance } from "perf_hooks";
import config from "../config";
import { observeCache } from "./cache-observability";
export class CacheAdmissionError extends Error {}
export function isCacheContention(error: unknown) {
const code = Number((error as { errcode?: number })?.errcode) & 255;
return code === 5 || code === 6;
}
function cacheError(error: unknown): never {
if (isCacheContention(error)) throw new CacheAdmissionError("cache_index_busy");
throw error;
}
export interface BlobRecord { digest: string; file: string; size: number; stored: number; allocated: number; encoding: "gzip" | "identity"; accessed: number }
export interface BlobProof { gitSha: string; sha256: string; size: number }
export interface BlobOptions { root: string; budget: number; maxObject: number; gzip?: boolean; saving?: number; freeBytes?: number; lowWater?: number; name?: string; reservationRoot?: string }
export type BlobStream = Readable & { cachedBlob?: { store: BlobCache; record: BlobRecord } };
const validDigest = (s: string) => /^(git-[a-f0-9]{40}|sha256-[a-f0-9]{64})$/.test(s);
/** One local filesystem shared by all processes. SQLite WAL coordinates metadata, fills and reader leases. */
export class BlobCache {
readonly db: DatabaseSync;
readonly name: string;
private disk?: DatabaseSync;
constructor(readonly options: BlobOptions) {
if (!(options.budget > 0 && options.maxObject > 0)) throw new Error("invalid_cache_budget");
this.name = options.name || "source";
mkdirSync(join(options.root, "tmp"), { recursive: true, mode: 0o700 });
this.db = new DatabaseSync(join(options.root, "index.sqlite"));
try {
this.db.exec(`PRAGMA busy_timeout=5000; PRAGMA journal_mode=WAL;
CREATE TABLE IF NOT EXISTS blobs(digest TEXT PRIMARY KEY,file TEXT,size INTEGER,stored INTEGER,encoding TEXT,accessed INTEGER,allocated INTEGER);
CREATE INDEX IF NOT EXISTS blobs_lru ON blobs(accessed);
CREATE TABLE IF NOT EXISTS aliases(key TEXT PRIMARY KEY,digest TEXT REFERENCES blobs(digest) ON DELETE CASCADE);
CREATE INDEX IF NOT EXISTS aliases_digest ON aliases(digest);
CREATE TABLE IF NOT EXISTS fills(key TEXT PRIMARY KEY,token TEXT,expires INTEGER,reserved INTEGER);
CREATE TABLE IF NOT EXISTS publications(token TEXT PRIMARY KEY,digest TEXT,file TEXT);
CREATE TABLE IF NOT EXISTS readers(token TEXT PRIMARY KEY,digest TEXT,expires INTEGER);
CREATE INDEX IF NOT EXISTS readers_digest ON readers(digest);
CREATE INDEX IF NOT EXISTS readers_expiry ON readers(expires);
CREATE TABLE IF NOT EXISTS decisions(key TEXT PRIMARY KEY,digest TEXT,changed INTEGER,accessed INTEGER,expires INTEGER);
CREATE INDEX IF NOT EXISTS decisions_age ON decisions(accessed);
CREATE INDEX IF NOT EXISTS decisions_expiry ON decisions(expires);
CREATE TABLE IF NOT EXISTS maintenance(id INTEGER PRIMARY KEY,nextScan INTEGER);
CREATE TABLE IF NOT EXISTS history(key TEXT PRIMARY KEY,seen INTEGER,evicted INTEGER);
CREATE INDEX IF NOT EXISTS history_seen ON history(seen);
CREATE INDEX IF NOT EXISTS blobs_file ON blobs(file);
CREATE TABLE IF NOT EXISTS totals(id INTEGER PRIMARY KEY, bytes INTEGER);
CREATE TRIGGER IF NOT EXISTS blob_added AFTER INSERT ON blobs BEGIN UPDATE totals SET bytes=bytes+NEW.allocated WHERE id=1; END;
CREATE TRIGGER IF NOT EXISTS blob_removed AFTER DELETE ON blobs BEGIN UPDATE totals SET bytes=bytes-OLD.allocated WHERE id=1; END;
PRAGMA foreign_keys=ON; PRAGMA recursive_triggers=ON;`);
// Initialize once under a write lock; all connections see data and counters atomically.
this.transaction(() => {
// Classification is nullable for decisions written by older versions.
if (!this.db.prepare("PRAGMA table_info(decisions)").all().some(row => row.name === "isText"))
this.db.exec("ALTER TABLE decisions ADD COLUMN isText INTEGER");
if (!this.db.prepare("SELECT 1 FROM totals WHERE id=1").get()) this.db.exec("INSERT INTO totals SELECT 1,COALESCE(SUM(allocated),0) FROM blobs");
this.db.exec(`CREATE TABLE IF NOT EXISTS summary(id INTEGER PRIMARY KEY,blobs INTEGER,originalBytes INTEGER,storedBytes INTEGER,allocatedBytes INTEGER,activeReaders INTEGER,decisions INTEGER,aliases INTEGER,history INTEGER,reservedBytes INTEGER)`);
if (!this.db.prepare("SELECT 1 FROM summary WHERE id=1").get()) {
this.db.exec(`INSERT INTO summary SELECT 1,COUNT(*),COALESCE(SUM(size),0),COALESCE(SUM(stored),0),COALESCE(SUM(allocated),0),
(SELECT COUNT(*) FROM readers),(SELECT COUNT(*) FROM decisions),(SELECT COUNT(*) FROM aliases),(SELECT COUNT(*) FROM history),(SELECT COALESCE(SUM(reserved),0) FROM fills) FROM blobs`);
}
for (const [table, columns] of [
["blobs", { blobs: "1", originalBytes: "size", storedBytes: "stored", allocatedBytes: "allocated" }],
["readers", { activeReaders: "1" }], ["decisions", { decisions: "1" }],
["aliases", { aliases: "1" }], ["history", { history: "1" }], ["fills", { reservedBytes: "reserved" }]
] as [string, Record<string, string>][]) {
const changes = (prefix: string, sign: string) => Object.entries(columns).map(([column, value]) =>
`${column}=${column}${sign}${value === "1" ? "1" : prefix + "." + value}`).join(",");
this.db.exec(`CREATE TRIGGER IF NOT EXISTS summary_${table}_insert AFTER INSERT ON ${table} BEGIN UPDATE summary SET ${changes("NEW", "+")} WHERE id=1; END;
CREATE TRIGGER IF NOT EXISTS summary_${table}_delete AFTER DELETE ON ${table} BEGIN UPDATE summary SET ${changes("OLD", "-")} WHERE id=1; END;`);
const values = Object.entries(columns).filter(([, value]) => value !== "1");
if (values.length) this.db.exec(`CREATE TRIGGER IF NOT EXISTS summary_${table}_update AFTER UPDATE OF ${values.map(([, value]) => value).join(",")} ON ${table}
BEGIN UPDATE summary SET ${values.map(([column, value]) => `${column}=${column}-OLD.${value}+NEW.${value}`).join(",")} WHERE id=1; END;`);
}
});
const reservationRoot = options.reservationRoot || options.root;
mkdirSync(reservationRoot, { recursive: true, mode: 0o700 });
this.disk = new DatabaseSync(join(reservationRoot, "disk-reservations.sqlite"));
this.disk.exec(`PRAGMA busy_timeout=5000; PRAGMA journal_mode=WAL; PRAGMA synchronous=FULL;
CREATE TABLE IF NOT EXISTS reservations(token TEXT PRIMARY KEY,bytes INTEGER);
PRAGMA busy_timeout=50;`);
this.db.exec("PRAGMA synchronous=NORMAL; PRAGMA busy_timeout=50;");
} catch (error) { this.disk?.close(); this.db.close(); throw error; }
}
// Reader leases, history and decisions are disposable after a system failure.
// Reservations and publication records use FULL before creating/renaming payloads,
// so recovery never releases disk accounting before removing an unindexed file.
private transaction<T>(fn: () => T, durable = false): T {
if (durable) this.db.exec("PRAGMA synchronous=FULL");
try {
this.db.exec("BEGIN IMMEDIATE");
try { const value = fn(); this.db.exec("COMMIT"); return value; }
catch (error) { this.db.exec("ROLLBACK"); throw error; }
} finally { if (durable) this.db.exec("PRAGMA synchronous=NORMAL"); }
}
private trim(table: "aliases" | "history" | "decisions", limit: number, order: string) {
const excess = Number(this.db.prepare(`SELECT ${table} n FROM summary WHERE id=1`).get()!.n) - limit;
if (excess > 0) this.db.prepare(`DELETE FROM ${table} WHERE rowid IN (SELECT rowid FROM ${table} ORDER BY ${order} LIMIT ?)`).run(excess);
}
private reserveDisk(token: string, bytes: number) {
const disk = this.disk!;
disk.exec("BEGIN IMMEDIATE");
try {
// Take a fresh filesystem sample while holding the shared admission lock.
const space = statfsSync(this.options.root);
const pending = Number(disk.prepare("SELECT COALESCE(SUM(bytes),0) bytes FROM reservations").get()!.bytes);
if (space.bavail * space.bsize < pending + bytes + (this.options.freeBytes || 0))
throw new CacheAdmissionError("disk_headroom");
disk.prepare("INSERT INTO reservations VALUES(?,?)").run(token, bytes);
disk.exec("COMMIT");
} catch (error) { disk.exec("ROLLBACK"); throw error; }
}
private releaseDisk(token: string) { this.disk!.prepare("DELETE FROM reservations WHERE token=?").run(token); }
private metric(event: string, bytes = 0, ms = 0) { observeCache(`${this.name}_${event}`, bytes, ms); }
private path(record: BlobRecord) {
if (!validDigest(record.digest) || !/^[a-f0-9-]+\.(gz|raw)$/.test(record.file)) throw new Error("invalid_blob_record");
return join(this.options.root, "blobs", record.digest.substring(record.digest.indexOf("-") + 1, record.digest.indexOf("-") + 3), record.digest, record.file);
}
private usage() { return Number(this.db.prepare("SELECT allocatedBytes bytes FROM summary WHERE id=1").get()!.bytes); }
private reservations() { return Number(this.db.prepare("SELECT reservedBytes bytes FROM summary WHERE id=1").get()!.bytes); }
private expire() {
const now = Date.now();
// A crashed producer's files are removed BEFORE its reservation is released.
const dead = this.db.prepare("SELECT token FROM fills WHERE expires<?").all(now);
for (const row of dead) {
const pending = this.db.prepare("SELECT digest,file FROM publications WHERE token=?").get(row.token);
if (pending && !this.db.prepare("SELECT 1 FROM blobs WHERE file=?").get(pending.file)) {
try { unlinkSync(this.path(pending as unknown as BlobRecord)); }
catch (error) { if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; }
}
this.db.prepare("DELETE FROM publications WHERE token=?").run(row.token);
for (const suffix of ["raw", "gz"]) {
try { unlinkSync(join(this.options.root, "tmp", `${row.token}.${suffix}`)); }
catch (error) { if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; }
}
this.releaseDisk(String(row.token));
}
this.db.prepare("DELETE FROM fills WHERE expires<?").run(now);
this.db.prepare("DELETE FROM readers WHERE expires<?").run(now);
this.db.prepare("DELETE FROM decisions WHERE rowid IN (SELECT rowid FROM decisions WHERE expires<? ORDER BY expires LIMIT 256)").run(now);
this.db.prepare("DELETE FROM history WHERE rowid IN (SELECT rowid FROM history WHERE seen<? ORDER BY seen LIMIT 256)").run(now - 7 * 86400000);
}
private evictTo(target: number) {
let used = this.usage();
const rows = this.db.prepare(`SELECT * FROM blobs WHERE digest NOT IN (SELECT digest FROM readers)
ORDER BY accessed LIMIT 512`).all() as unknown as BlobRecord[];
for (const record of rows) {
if (used <= target) break;
try { unlinkSync(this.path(record)); } catch (error) { if ((error as NodeJS.ErrnoException).code !== "ENOENT") continue; }
this.db.prepare("UPDATE history SET evicted=1 WHERE key IN (SELECT key FROM aliases WHERE digest=?)").run(record.digest);
this.db.prepare("DELETE FROM blobs WHERE digest=?").run(record.digest);
try { rmdirSync(join(this.path(record), "..")); } catch { /* another representation remains */ }
used -= record.allocated; this.metric("eviction", record.stored, Date.now() - record.accessed);
}
}
async maintain() {
this.transaction(() => { this.expire(); this.evictTo(Math.max(0, this.options.budget - this.reservations())); });
const scan = this.transaction(() => {
const previous = this.db.prepare("SELECT nextScan FROM maintenance WHERE id=1").get();
if (previous && Number(previous.nextScan) > Date.now()) return false;
this.db.prepare("INSERT OR REPLACE INTO maintenance VALUES(1,?)").run(Date.now() + 3600000);
return true;
});
if (!scan) return;
// Recover publication orphans after crashes. Never remove a recent publisher's file.
const walk = async (directory: string): Promise<void> => {
let entries; try { entries = await fs.opendir(directory); } catch { return; }
for await (const entry of entries) {
const p = join(directory, entry.name);
if (entry.isDirectory()) { await walk(p); continue; }
const stat = await fs.stat(p).catch(() => undefined);
if (!stat || Date.now() - stat.mtimeMs < 3600000) continue;
const found = this.db.prepare("SELECT 1 FROM blobs WHERE file=?").get(entry.name);
if (!found) await fs.unlink(p).catch(() => {});
}
};
await walk(join(this.options.root, "blobs"));
}
stats() {
const data = this.db.prepare("SELECT blobs,originalBytes,storedBytes,allocatedBytes,activeReaders,decisions,reservedBytes FROM summary WHERE id=1").get();
return { ...data, budgetBytes: this.options.budget, maxObjectBytes: this.options.maxObject,
oldestAccess: this.db.prepare("SELECT accessed FROM blobs ORDER BY accessed LIMIT 1").get()?.accessed ?? null };
}
async open(digest: string, encoded = false): Promise<BlobStream | undefined> {
try { return await this.openReader(digest, encoded); } catch (error) { return cacheError(error); }
}
private async openReader(digest: string, encoded = false): Promise<BlobStream | undefined> {
if (!validDigest(digest)) return;
const token = randomUUID();
let leaseUntil = Date.now() + 120000;
const record = this.transaction(() => {
const r = this.db.prepare("SELECT * FROM blobs WHERE digest=?").get(digest) as unknown as BlobRecord | undefined;
if (!r) return;
this.db.prepare("INSERT INTO readers VALUES(?,?,?)").run(token, digest, leaseUntil);
if (Date.now() - r.accessed > 30000) this.db.prepare("UPDATE blobs SET accessed=? WHERE digest=?").run(Date.now(), digest);
return r;
});
if (!record) return;
let handle;
try {
handle = await fs.open(this.path(record), "r");
if ((await handle.stat()).size !== record.stored) throw new Error("corrupt_blob_size");
} catch {
await handle?.close();
await fs.unlink(this.path(record)).catch(() => {});
this.db.prepare("DELETE FROM readers WHERE token=?").run(token);
this.transaction(() => { this.db.prepare("DELETE FROM blobs WHERE digest=?").run(digest); });
this.metric("corrupt"); return;
}
const source = handle.createReadStream();
const needsDecode = record.encoding === "gzip" && !encoded;
const output = (needsDecode ? createGunzip() : source) as BlobStream;
output.cachedBlob = { store: this, record };
let start = performance.now();
let renewAt = Date.now() + 30000;
const heartbeat = setInterval(() => {
const now = Date.now();
if (now < renewAt) return;
if (now >= leaseUntil) { output.destroy(new Error("cache_reader_lease_lost")); return; }
try {
const renewedUntil = now + 120000;
if (!this.db.prepare("UPDATE readers SET expires=? WHERE token=? AND expires>?").run(renewedUntil, token, now).changes) {
output.destroy(new Error("cache_reader_lease_lost")); return;
}
leaseUntil = renewedUntil; renewAt = now + 30000;
} catch (error) {
if (!isCacheContention(error)) output.destroy(new Error("cache_reader_lease_lost"));
// Retry on the next tick while the previously confirmed lease is valid.
}
}, 1000); heartbeat.unref();
let released = false;
const release = () => {
if (released) return; released = true; clearInterval(heartbeat); source.destroy();
try { this.db.prepare("DELETE FROM readers WHERE token=?").run(token); } catch { /* expires */ }
};
output.once("close", release);
if (needsDecode) {
// Delay decompression until a consumer asks for data; fast-path callers can close unused streams.
const originalRead = output._read.bind(output); let begun = false;
output._read = size => { if (!begun) { begun = true; start = performance.now(); source.on("error", e => output.destroy(e)); source.pipe(output as Transform); } originalRead(size); };
output.once("end", () => observeCache("decompress", record.size, performance.now() - start));
}
return output;
}
async lookup(key: string): Promise<BlobStream | undefined> {
const alias = this.db.prepare("SELECT digest FROM aliases WHERE key=?").get(key);
return this.open(String(alias?.digest || key));
}
private accessed(key: string, hit: boolean, bytes = 0) {
const previous = this.db.prepare("SELECT * FROM history WHERE key=? AND seen>=?").get(key, Date.now() - 7 * 86400000);
this.metric(hit ? "hit" : "miss", bytes);
if (!hit) this.metric(previous?.evicted ? "refetch" : previous ? "known_miss" : "first_seen_miss");
if (previous) {
const age = Date.now() - Number(previous.seen);
for (const [label, limit] of [["1h", 3600000], ["1d", 86400000], ["7d", 604800000]] as const) if (age <= limit) this.metric(`reuse_${label}`, bytes);
}
if (!previous || previous.evicted || Date.now() - Number(previous.seen) >= 30000) this.transaction(() => {
this.db.prepare("INSERT INTO history VALUES(?,?,0) ON CONFLICT(key) DO UPDATE SET seen=excluded.seen,evicted=0").run(key, Date.now());
this.trim("history", 100000, "seen");
});
}
async getOrFill(key: string, produce: () => Promise<Readable>, verify?: (proof: BlobProof) => Promise<string>, sizeHint?: number): Promise<BlobStream> {
try { return await this.fill(key, produce, verify, sizeHint); } catch (error) { return cacheError(error); }
}
private async fill(key: string, produce: () => Promise<Readable>, verify?: (proof: BlobProof) => Promise<string>, sizeHint?: number): Promise<BlobStream> {
if (!/^[a-z0-9-]{1,100}$/.test(key)) throw new Error("invalid_cache_key");
let hit = await this.lookup(key);
try { this.accessed(key, !!hit, hit?.cachedBlob?.record.size || sizeHint || 0); }
catch (error) { hit?.destroy(); throw error; }
if (hit) return hit;
if (sizeHint != null && sizeHint > this.options.maxObject) { this.metric("bypass", sizeHint); throw new CacheAdmissionError("object_too_large"); }
const token = randomUUID(), reservation = this.options.maxObject * 3 + 1024;
const deadline = Date.now() + 120000;
let leaseUntil = 0;
for (;;) {
const owned = this.transaction(() => {
this.expire();
if (this.db.prepare("SELECT 1 FROM fills WHERE key=?").get(key)) return false;
const reserved = this.reservations();
if (this.usage() + reserved + reservation > this.options.budget) {
this.evictTo(Math.max(0, Math.min(this.options.budget * (this.options.lowWater ?? 0.9), this.options.budget - reserved - reservation)));
}
// Eviction unlinks files: commit its metadata even if admission still fails.
if (this.usage() + reserved + reservation > this.options.budget) return null;
leaseUntil = Date.now() + 120000;
this.db.prepare("INSERT INTO fills VALUES(?,?,?,?)").run(key, token, leaseUntil, reservation); return true;
}, true);
if (owned === null) throw new CacheAdmissionError("cache_budget_busy");
if (owned) break;
this.metric("fill_wait");
if (Date.now() >= deadline) throw new CacheAdmissionError("fill_timeout");
await new Promise(resolve => setTimeout(resolve, 50));
hit = await this.lookup(key); if (hit) return hit;
}
const raw = join(this.options.root, "tmp", `${token}.raw`), gz = join(this.options.root, "tmp", `${token}.gz`);
let input: Readable | undefined, file: string | undefined;
let renewAt = Date.now() + 30000, leaseLost = false;
const loseLease = () => { leaseLost = true; input?.destroy(new Error("cache_fill_lease_lost")); };
const heartbeat = setInterval(() => {
const now = Date.now();
if (leaseLost || now < renewAt) return;
if (now >= leaseUntil) { loseLease(); return; }
try {
const renewedUntil = now + 120000;
if (!this.db.prepare("UPDATE fills SET expires=? WHERE key=? AND token=? AND expires>?").run(renewedUntil, key, token, now).changes) {
loseLease(); return;
}
leaseUntil = renewedUntil; renewAt = now + 30000;
} catch (error) {
if (!isCacheContention(error)) loseLease();
// Retry contention on the next tick within the last confirmed lease.
}
}, 1000); heartbeat.unref();
const timeout = setTimeout(() => input?.destroy(new Error("cache_fill_timeout")), 120000); timeout.unref();
try {
// The pool reservation is already durable before acquiring physical headroom.
this.transaction(() => {
if (!this.db.prepare("SELECT 1 FROM fills WHERE token=? AND expires>?").get(token, Date.now())) throw new CacheAdmissionError("cache_fill_lease_lost");
this.reserveDisk(token, reservation);
});
// Recheck after ownership; another producer may have published between the initial lookup and lease acquisition.
hit = await this.lookup(key); if (hit) return hit;
input = await produce();
if (leaseLost || Date.now() >= leaseUntil) throw new Error("cache_fill_lease_lost");
let size = 0; const sha256 = createHash("sha256");
await pipeline(input, new Transform({ transform: (chunk, _encoding, done) => {
size += chunk.length; sha256.update(chunk);
done(size > this.options.maxObject ? new CacheAdmissionError("object_too_large") : null, chunk);
} }), createWriteStream(raw, { flags: "wx", mode: 0o600 }));
const git = createHash("sha1").update(`blob ${size}\0`);
for await (const chunk of createReadStream(raw)) git.update(chunk);
const proof = { gitSha: git.digest("hex"), sha256: sha256.digest("hex"), size };
const digest = verify ? await verify(proof) : `sha256-${proof.sha256}`;
if (!validDigest(digest)) throw new Error("invalid_verified_digest");
let encoding: "gzip" | "identity" = "identity", stored = size, chosen = raw;
if (this.options.gzip && size >= 1024) {
const started = performance.now();
await pipeline(createReadStream(raw), createGzip({ level: 6 }), createWriteStream(gz, { flags: "wx", mode: 0o600 }));
const compressed = (await fs.stat(gz)).size;
observeCache("compress", size, performance.now() - started);
if (compressed <= size * (1 - (this.options.saving ?? 0.1))) { encoding = "gzip"; stored = compressed; chosen = gz; }
}
const record: BlobRecord = { digest, file: `${token}.${encoding === "gzip" ? "gz" : "raw"}`, size, stored, allocated: 0, encoding, accessed: Date.now() };
file = this.path(record); await fs.mkdir(join(file, ".."), { recursive: true, mode: 0o700 });
const durable = await fs.open(chosen, "r");
try { await durable.sync(); } finally { await durable.close(); }
this.transaction(() => { this.db.prepare("INSERT INTO publications VALUES(?,?,?)").run(token, digest, record.file); }, true);
await fs.rename(chosen, file);
record.allocated = Math.max(4096, (await fs.stat(file)).blocks * 512);
let duplicate = false;
this.transaction(() => {
if (!this.db.prepare("SELECT 1 FROM fills WHERE key=? AND token=? AND expires>?").get(key, token, Date.now())) throw new Error("cache_fill_lease_lost");
duplicate = !!this.db.prepare("SELECT 1 FROM blobs WHERE digest=?").get(digest);
if (!duplicate) this.db.prepare("INSERT INTO blobs VALUES(?,?,?,?,?,?,?)").run(digest, record.file, size, stored, encoding, Date.now(), record.allocated);
this.db.prepare("INSERT INTO aliases VALUES(?,?) ON CONFLICT(key) DO UPDATE SET digest=excluded.digest").run(key, digest);
this.trim("aliases", 200000, "rowid");
// Bridge publication to open(), including asynchronous duplicate cleanup.
this.db.prepare("INSERT INTO readers VALUES(?,?,?)").run(token, digest, Date.now() + 120000);
}, true);
if (duplicate) { await fs.unlink(file); this.metric("deduplicated", stored); }
file = undefined;
this.metric("fill", size); this.metric("stored", duplicate ? 0 : stored);
const result = await this.open(digest); if (!result) throw new Error("published_blob_missing"); return result;
} finally {
clearTimeout(timeout); clearInterval(heartbeat); input?.destroy();
await Promise.all([raw, gz, ...(file ? [file] : [])].map(p => fs.unlink(p).catch(error => {
if (error.code !== "ENOENT") throw error;
})));
this.releaseFill(key, token);
}
}
private releaseFill(key: string, token: string) {
try {
this.transaction(() => {
this.releaseDisk(token);
this.db.prepare("DELETE FROM readers WHERE token=?").run(token);
this.db.prepare("DELETE FROM publications WHERE token=?").run(token);
this.db.prepare("DELETE FROM fills WHERE key=? AND token=?").run(key, token);
});
} catch (error) { if (!isCacheContention(error)) throw error; /* Leases expire conservatively. */ }
}
decision(key: string) {
try {
const row = this.db.prepare("SELECT * FROM decisions WHERE key=? AND expires>?").get(key, Date.now());
if (row) this.db.prepare("UPDATE decisions SET accessed=? WHERE key=? AND accessed<?").run(Date.now(), key, Date.now() - 30000);
return row ? { digest: String(row.digest), changed: !!row.changed, isText: row.isText == null ? undefined : !!row.isText } : undefined;
} catch (error) { return cacheError(error); }
}
saveDecision(key: string, digest: string, changed: boolean, isText?: boolean) {
try {
this.transaction(() => {
this.db.prepare("INSERT INTO decisions(key,digest,changed,accessed,expires,isText) VALUES(?,?,?,?,?,?) ON CONFLICT(key) DO UPDATE SET digest=excluded.digest,changed=excluded.changed,accessed=excluded.accessed,expires=excluded.expires,isText=excluded.isText")
.run(key, digest, +changed, Date.now(), Date.now() + 7 * 86400000, isText == null ? null : +isText);
this.trim("decisions", 100000, "accessed");
});
} catch (error) { if (!isCacheContention(error)) throw error; /* Optional decision can be recomputed. */ }
}
close() { this.disk?.close(); this.db.close(); }
}
let source: BlobCache | undefined, results: BlobCache | undefined;
export function sharedBlobCache() {
if (!config.SHARED_CACHE_ENABLED || config.STORAGE !== "filesystem") return;
try { return source ||= new BlobCache({ reservationRoot: config.SHARED_CACHE_PATH, root: join(config.SHARED_CACHE_PATH, "source"), budget: config.SHARED_CACHE_BYTES,
maxObject: config.SHARED_CACHE_MAX_FILE_BYTES, gzip: config.SHARED_CACHE_GZIP, saving: config.SHARED_CACHE_GZIP_MIN_SAVING,
freeBytes: config.SHARED_CACHE_FREE_BYTES, lowWater: config.SHARED_CACHE_LOW_WATER });
} catch (error) { if (!isCacheContention(error)) throw error; }
}
export function resultBlobCache() {
if (!sharedBlobCache() || !config.SHARED_CACHE_DECISIONS) return;
try { return results ||= new BlobCache({ reservationRoot: config.SHARED_CACHE_PATH, root: join(config.SHARED_CACHE_PATH, "results"), budget: config.SHARED_CACHE_RESULT_BYTES,
maxObject: Math.min(config.SHARED_CACHE_MAX_FILE_BYTES, Math.floor(config.SHARED_CACHE_RESULT_BYTES / 8)), gzip: config.SHARED_CACHE_GZIP,
freeBytes: config.SHARED_CACHE_FREE_BYTES, name: "result" });
} catch (error) { if (!isCacheContention(error)) throw error; }
}
export function closeBlobCaches() { source?.close(); results?.close(); source = results = undefined; }
+76
View File
@@ -0,0 +1,76 @@
import { hostname } from "os";
import { DatabaseSync } from "node:sqlite";
import { mkdirSync } from "fs";
import { join } from "path";
import config from "../config";
import { cacheTraffic } from "./request-monitoring";
export const CACHE_BUCKETS = [0.1, 1, 10, 100, 1000, 10000];
type Sample = { count: number; bytes: number; ms: number; buckets: number[] };
let db: DatabaseSync | undefined;
const pending = new Map<string, Sample>();
let dropped = 0;
let timer: NodeJS.Timeout | undefined;
let lastFlush = 0;
function connection() {
if (!db) {
mkdirSync(config.SHARED_CACHE_PATH, { recursive: true, mode: 0o700 });
db = new DatabaseSync(join(config.SHARED_CACHE_PATH, "telemetry.sqlite"));
db.exec(`PRAGMA journal_mode=WAL; PRAGMA synchronous=NORMAL; PRAGMA busy_timeout=50;
CREATE TABLE IF NOT EXISTS metrics (period INTEGER, bucket INTEGER, name TEXT, traffic TEXT,
count INTEGER, bytes INTEGER, ms REAL, b0 INTEGER, b1 INTEGER, b2 INTEGER, b3 INTEGER, b4 INTEGER, b5 INTEGER, b6 INTEGER,
PRIMARY KEY(period,bucket,name,traffic));
CREATE TABLE IF NOT EXISTS health (id TEXT PRIMARY KEY, updated INTEGER, dropped INTEGER);
CREATE INDEX IF NOT EXISTS metric_time ON metrics(period,bucket);`);
}
return db;
}
/** Fixed event names only. No paths, terms, user identifiers or blob hashes in metrics. */
export function observeCache(name: string, bytes = 0, ms = 0, traffic = cacheTraffic()) {
if (!config.CACHE_MONITORING_ENABLED) return;
const key = JSON.stringify([Math.floor(Date.now() / 60000) * 60000, name, traffic]);
if (!pending.has(key) && pending.size >= 512) { dropped++; return; }
const value = pending.get(key) || { count: 0, bytes: 0, ms: 0, buckets: Array(7).fill(0) };
value.count++; value.bytes += Math.max(0, bytes); value.ms += Math.max(0, ms);
const bucket = CACHE_BUCKETS.findIndex(bound => ms <= bound);
value.buckets[bucket < 0 ? 6 : bucket]++;
pending.set(key, value);
if (!timer) { timer = setInterval(flushCacheMetrics, 5000); timer.unref(); }
}
export function flushCacheMetrics() {
if (!config.CACHE_MONITORING_ENABLED) return;
try {
const sql = connection();
const insert = sql.prepare(`INSERT INTO metrics VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?)
ON CONFLICT(period,bucket,name,traffic) DO UPDATE SET count=count+excluded.count,bytes=bytes+excluded.bytes,ms=ms+excluded.ms,
b0=b0+excluded.b0,b1=b1+excluded.b1,b2=b2+excluded.b2,b3=b3+excluded.b3,b4=b4+excluded.b4,b5=b5+excluded.b5,b6=b6+excluded.b6`);
sql.exec("BEGIN IMMEDIATE");
try {
for (const [key, value] of pending) {
const [at, name, traffic] = JSON.parse(key);
for (const period of [60000, 3600000]) insert.run(period, Math.floor(at / period) * period, name, traffic,
value.count, value.bytes, value.ms, ...value.buckets);
}
sql.prepare("DELETE FROM metrics WHERE (period=60000 AND bucket<?) OR (period=3600000 AND bucket<?)")
.run(Date.now() - 48 * 3600000, Date.now() - 30 * 86400000);
sql.prepare("INSERT OR REPLACE INTO health VALUES(?,?,?)").run(`${hostname()}:${process.pid}`, Date.now(), dropped);
sql.prepare("DELETE FROM health WHERE updated<?").run(Date.now() - 30 * 86400000);
sql.exec("COMMIT"); pending.clear(); lastFlush = Date.now();
} catch (error) { sql.exec("ROLLBACK"); throw error; }
} catch { dropped += [...pending.values()].reduce((sum, v) => sum + v.count, 0); pending.clear(); }
}
export function cacheMetricsReport(minutes = 60) {
if (!config.CACHE_MONITORING_ENABLED) return { available: false, reason: "Monitoring is disabled", rows: [] };
flushCacheMetrics();
const windowMinutes = [15, 60, 1440, 10080].includes(minutes) ? minutes : 60;
try {
const sql = connection();
const rows = sql.prepare(`SELECT name,traffic,SUM(count) count,SUM(bytes) bytes,SUM(ms) ms,
SUM(b0) b0,SUM(b1) b1,SUM(b2) b2,SUM(b3) b3,SUM(b4) b4,SUM(b5) b5,SUM(b6) b6
FROM metrics WHERE period=? AND bucket>=? GROUP BY name,traffic ORDER BY name,traffic`)
.all(windowMinutes > 2880 ? 3600000 : 60000, Date.now() - windowMinutes * 60000);
const health = sql.prepare("SELECT COALESCE(SUM(dropped),0) dropped FROM health").get();
return { available: true, windowMinutes, sampledAt: Date.now(), lastFlush, dropped: health?.dropped, rows, latencyBucketsMs: CACHE_BUCKETS };
} catch { return { available: false, reason: "Cache telemetry could not be read", rows: [], dropped }; }
}
export function closeCacheMetrics() { clearInterval(timer); timer = undefined; flushCacheMetrics(); db?.close(); db = undefined; }
+111
View File
@@ -0,0 +1,111 @@
import { createGzip } from "zlib";
import { createHash } from "crypto";
import { Readable, PassThrough } from "stream";
import { pipeline } from "stream/promises";
import { Response } from "express";
import { performance } from "perf_hooks";
import config from "../config";
import { assertTransformScopeActive } from "./transformed-cache";
import { BlobStream, CacheAdmissionError, resultBlobCache } from "./blob-cache";
import { AnonymizeTransformer, classifyByName } from "./anonymize-utils";
import { observeCache } from "./cache-observability";
class Unchanged extends Error {}
export function decisionKey(digest: string, options: unknown) {
return createHash("sha256").update(JSON.stringify(["anonymizer-v2", digest, options,
config.ANONYMIZATION_MASK, config.APP_HOSTNAME, config.additionalExtensions])).digest("hex");
}
/** Returns false when the caller should use the existing streaming pipeline. Authorization precedes this call. */
export async function sendSharedContent(content: Readable, res: Response,
options: ConstructorParameters<typeof AnonymizeTransformer>[0]): Promise<boolean> {
try { return await sendCached(content, res, options); }
catch (error) { if (error instanceof CacheAdmissionError) return false; content.destroy(); throw error; }
}
async function sendCached(content: Readable, res: Response,
options: ConstructorParameters<typeof AnonymizeTransformer>[0]): Promise<boolean> {
const source = (content as BlobStream).cachedBlob;
const results = resultBlobCache();
if (!source || !results) return false;
const assertActive = async () => {
if (res.destroyed) throw new Error("response_cancelled");
await assertTransformScopeActive(options);
if (res.destroyed) throw new Error("response_cancelled");
};
const encoding = res.req.acceptsEncodings("gzip", "identity");
res.vary("Accept-Encoding");
if (!encoding) { content.destroy(); res.status(406).end(); return true; }
const key = decisionKey(source.record.digest, options);
const namedType = classifyByName(options.filePath);
const send = async (store: typeof results, digest: string, isText: boolean, onHit?: () => void) => {
const stream = await store.open(digest, encoding === "gzip");
if (!stream) return false;
try { await assertActive(); } catch (error) { stream.destroy(); throw error; }
const record = stream.cachedBlob!.record;
onHit?.();
content.destroy();
if (record.encoding === "gzip" && encoding === "gzip") {
res.setHeader("Content-Encoding", "gzip"); res.setHeader("Content-Length", record.stored);
} else { res.removeHeader("Content-Encoding"); res.setHeader("Content-Length", record.size); }
if (isText && !res.getHeader("Content-Type")) res.type("text/plain");
if (isText) res.setHeader("Accept-Ranges", "none");
else res.removeHeader("Accept-Ranges");
if (record.encoding === "identity" && !res.req.acceptsEncodings("identity")) {
res.setHeader("Content-Encoding", "gzip"); res.removeHeader("Content-Length");
await pipeline(stream, createGzip(), res);
} else await pipeline(stream, res);
return true;
};
const decision = results.decision(key);
if (decision && (decision.isText !== undefined || namedType !== null) && await send(decision.changed ? results : source.store, decision.digest, decision.isText ?? namedType!, () => {
observeCache("decision_hit", source.record.size);
observeCache(decision.changed ? "rewrite_changed_cached" : "rewrite_unchanged_cached", source.record.size);
})) return true;
observeCache("decision_miss", source.record.size);
if (classifyByName(options.filePath) === false) {
results.saveDecision(key, source.record.digest, false, false);
observeCache("rewrite_binary", source.record.size);
return send(source.store, source.record.digest, false);
}
// Preserve the original source reader as a fallback if result admission fails.
let processing: Readable | undefined;
let transformer: AnonymizeTransformer | undefined;
const started = performance.now();
const abort = () => {
if (!res.writableFinished) {
const error = new Error("response_cancelled");
transformer?.destroy(error);
processing?.destroy(error);
}
};
res.once("close", abort);
try {
const output = await results.getOrFill(`render-${key}`, async () => {
await assertActive();
processing = await source.store.open(source.record.digest);
if (!processing) throw new CacheAdmissionError("source_evicted");
if (res.destroyed) throw new Error("response_cancelled");
const transform = transformer = new AnonymizeTransformer({ ...options, decisionManaged: true });
const stream = new PassThrough();
void pipeline(processing, transform, stream).catch(error => stream.destroy(error));
return stream;
}, async proof => {
await assertActive();
const unchanged = source.record.digest === `git-${proof.gitSha}` || source.record.digest === `sha256-${proof.sha256}`;
observeCache(unchanged ? "rewrite_unchanged" : "rewrite_changed", source.record.size, performance.now() - started);
if (unchanged) { results.saveDecision(key, source.record.digest, false, !!transformer?.isText); throw new Unchanged(); }
return `sha256-${proof.sha256}`;
}, source.record.size);
const digest = output.cachedBlob!.record.digest;
// Keep the result pinned until its HTTP reader has been opened.
try {
await assertActive();
results.saveDecision(key, digest, true, true);
if (await send(results, digest, true)) return true;
} finally { output.destroy(); }
return false;
} catch (error) {
if (error instanceof Unchanged) return send(source.store, source.record.digest, !!transformer?.isText);
if (error instanceof CacheAdmissionError) { observeCache("decision_bypass", source.record.size); return false; }
content.destroy(); throw error;
} finally { res.removeListener("close", abort); transformer?.destroy(); processing?.destroy(); }
}
+9 -3
View File
@@ -12,7 +12,7 @@ export interface Metric {
bytes: number; aborted: number; errors: number; slow: number; buckets: number[];
}
const logger = createLogger("requests");
const context = new AsyncLocalStorage<{ id: string; service: Service; stages: Partial<Record<Stage, number>> }>();
const context = new AsyncLocalStorage<{ id: string; service: Service; traffic?: string; stages: Partial<Record<Stage, number>> }>();
const methods = new Set(["GET", "HEAD", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"]);
const staticRoutes = new Set([
"/", "/dashboard", "/anonymize", "/healthcheck", "/api/healthcheck", "/api/stat", "/api/stat/history",
@@ -77,8 +77,9 @@ let activeRequests = 0;
export function activeRequestCount() { return activeRequests; }
export function requestHeaders(): Record<string, string> {
const id = context.getStore()?.id;
return id ? { "x-request-id": id } : {};
return id ? { "x-request-id": id, "x-cache-traffic": cacheTraffic() } : {};
}
export function cacheTraffic() { return context.getStore()?.traffic || "background"; }
export function recordCacheHit(cache: "source" | "transformed", hit: boolean) {
if (context.getStore()) requestMetrics.observe(`${cache}_${hit ? "hit" : "miss"}`, "all", "all", 0);
}
@@ -107,7 +108,12 @@ export function monitorRequests(service: Service, metrics = requestMetrics,
return (req, res, next) => {
const forwarded = service === "streamer" ? req.get("x-request-id") : undefined;
const id = forwarded && /^[a-f0-9-]{36}$/.test(forwarded) ? forwarded : randomUUID();
const trace = { id, service, stages: {} };
const forwardedTraffic = service === "streamer" ? req.get("x-cache-traffic") : undefined;
const traffic = forwardedTraffic && ["file", "website", "zip", "monitor", "background"].includes(forwardedTraffic)
? forwardedTraffic : /uptime|healthcheck|monitor/i.test(req.get("user-agent") || "") ? "monitor"
: /\/zip(?:[/?]|$)|\/download(?:[/?]|$)/.test(req.originalUrl) ? "zip"
: req.originalUrl.startsWith("/w/") ? "website" : "file";
const trace = { id, service, traffic, stages: {} };
const route = requestRoute(req.originalUrl, service);
const method = methods.has(req.method) ? req.method : "OTHER";
const started = performance.now();
+86
View File
@@ -1,5 +1,7 @@
import { withOwnerAccessRecovery } from "../owner-notifications";
import { ExpiringMap } from "../expiring-map";
import { sharedBlobCache, CacheAdmissionError, BlobProof } from "../blob-cache";
import { observeCache } from "../cache-observability";
import { contentGenerationPrefix, contentRetirementMarker } from "../content-generation";
import { createHash } from "crypto";
import { coordinatedFill } from "../cache-coordination";
@@ -384,8 +386,92 @@ export default class GitHubStream extends GitHubBase {
return undefined;
}
};
const shared = sharedBlobCache();
if (shared && /^[a-f0-9]{40}$/.test(expected.sha)) {
await assertActive();
if (expected.size != null && expected.size > config.MAX_FILE_SIZE) throw new AnonymousError("file_too_big", { httpStatus: 413 });
let legacy = false, allowLegacy = true;
const download = async (checkMinimum = false) => {
const parentToken = await this.data.getToken();
const token = expected.upstream ? "" : parentToken;
const sourceMiss = cachedContentMiss(this.data, parentToken);
if (sourceMiss && sourceMiss !== "file_not_found") throw new AnonymousError(sourceMiss, { httpStatus: 404 });
const missKey = createHash("sha256").update(JSON.stringify([
key, this.data.organization, this.data.repoName, token,
])).digest("hex");
const missed = missingContent.get(missKey);
if (missed) throw new AnonymousError(missed, { httpStatus: 404 });
const normalize = async (error: Error) => {
const normalized = downloadError(error, filePath);
if (expected.upstream && ["file_not_found", "file_not_accessible"].includes(normalized.message))
return new AnonymousError("submodule_not_available", { httpStatus: 404, cause: normalized });
if (!expected.upstream && normalized.message === "file_not_found" && normalized.httpStatus === 404) {
const code = await classifyContentMiss(this.data, token);
missingContent.set(missKey, code, 30_000);
if (code !== "file_not_found") return new AnonymousError(code, { httpStatus: 404, cause: normalized });
}
return normalized;
};
const input = await origin.downloadWithFallback(token, expected.sha, sourcePath)
.catch(async error => { throw await normalize(error); });
input.on("error", () => {}); // The iterator below propagates normalized failures.
const deadline = setTimeout(() => input.destroy(new AnonymousError("upstream_error", { httpStatus: 502 })), 120000);
deadline.unref();
const limited = stream.Readable.from((async function* () {
let bytes = 0;
try {
for await (const chunk of input) {
bytes += chunk.length;
if (bytes > config.MAX_FILE_SIZE) throw new AnonymousError("file_too_big", { httpStatus: 413 });
yield chunk;
}
if (checkMinimum && expected.size != null && expected.size > 0 && bytes < expected.size)
throw new AnonymousError("storage_write_size_mismatch", { httpStatus: 502 });
} catch (error) { throw await normalize(error as Error); }
finally { clearTimeout(deadline); input.destroy(); }
})());
stopSourceOnDestroy(limited, () => { clearTimeout(deadline); input.destroy(); });
return limited;
};
const produce = async () => {
if (allowLegacy && await cached()) { legacy = true; observeCache("source_legacy_promotion"); return storage.read(repoId, cachePath); }
legacy = false; return download();
};
const verify = async (proof: BlobProof) => {
if (proof.gitSha === expected.sha) return `git-${proof.gitSha}`;
// Resolved LFS bytes do not hash to the Git pointer SHA. Verify both the pointer and its payload.
const parentToken = await this.data.getToken();
const pointer = origin.downloadFile(expected.upstream ? "" : parentToken, expected.sha);
const timer = setTimeout(() => pointer.destroy(new Error("lfs_pointer_timeout")), 30000); timer.unref();
let bytes = 0; const chunks: Buffer[] = [];
try {
for await (const chunk of pointer) { bytes += chunk.length; if (bytes > 1024) throw new Error("invalid_lfs_pointer"); chunks.push(Buffer.from(chunk)); }
const raw = Buffer.concat(chunks), text = raw.toString("utf8");
const sha = createHash("sha1").update(`blob ${raw.length}\0`).update(raw).digest("hex");
if (sha !== expected.sha || !text.startsWith("version https://git-lfs.github.com/spec/v1\n") ||
text.match(/^oid sha256:([a-f0-9]{64})$/m)?.[1] !== proof.sha256 ||
Number(text.match(/^size (\d+)$/m)?.[1]) !== proof.size) throw new Error("invalid_blob_hash");
return `sha256-${proof.sha256}`;
} finally { clearTimeout(timer); pointer.destroy(); }
};
try {
let content;
try { content = await shared.getOrFill(`git-${expected.sha}`, produce, verify, expected.size); }
catch (error) {
if (!legacy || error instanceof CacheAdmissionError) throw error;
allowLegacy = false;
content = await shared.getOrFill(`git-${expected.sha}`, produce, verify, expected.size);
}
try { await assertActive(); } catch (error) { content.destroy(); throw error; }
return content;
} catch (error) {
if (!(error instanceof CacheAdmissionError)) throw error;
observeCache("source_bypass"); await assertActive(); return download(true);
}
}
const hit = await measureStage("cache_lookup", cached);
recordCacheHit("source", !!hit);
observeCache(hit ? "legacy_source_hit" : "legacy_source_miss", expected.size || 0);
if (hit) return storage.read(repoId, cachePath);
if (expected.size != null && expected.size > config.MAX_FILE_SIZE) {
throw new AnonymousError("file_too_big", { httpStatus: 413, object: filePath });
+2 -1
View File
@@ -1,3 +1,4 @@
import { sharedBlobCache, resultBlobCache } from "./blob-cache";
import { promises as fs } from "fs";
import { join } from "path";
import { tmpdir } from "os";
@@ -34,7 +35,7 @@ export async function startTemporaryStorageMaintenance(intervalMs = 60_000) {
if (running) return;
running = true;
try {
const results = await Promise.allSettled([maintainTextSpools(true), pruneTransformedCache()]);
const results = await Promise.allSettled([maintainTextSpools(true), pruneTransformedCache(), sharedBlobCache()?.maintain(), resultBlobCache()?.maintain()]);
for (const result of results) if (result.status === "rejected") {
logger.warn("temporary storage cleanup deferred", { message: (result.reason as Error).message });
}
+12 -5
View File
@@ -1,3 +1,4 @@
import { observeCache } from "./cache-observability";
import { promises as fs } from "fs";
import { join } from "path";
import { tmpdir } from "os";
@@ -25,6 +26,14 @@ async function scopeActive(scope?: CacheScope) {
&& scope.revision === current.contentCacheRevision;
}
/** Shared and legacy result caches must enforce the same repository lifecycle. */
export async function assertTransformScopeActive(options: unknown) {
const settings = options as { repoId?: string; cacheGeneration?: string; cacheRevision?: string } | undefined;
const scope = typeof settings?.repoId === "string"
? { repoId: settings.repoId, generation: settings.cacheGeneration, revision: settings.cacheRevision } : undefined;
if (!await scopeActive(scope)) throw new AnonymousError("repository_changed", { httpStatus: 409 });
}
async function removeEntry(metadata: string) {
await Promise.all([metadata, metadata.replace(/\.json$/, ".data")].map(path => fs.rm(path, { force: true })));
}
@@ -35,9 +44,7 @@ export async function transformedFile(digest: string, options: unknown, output:
const settings = options as { repoId?: string; cacheGeneration?: string; cacheRevision?: string } | undefined;
const scope: CacheScope | undefined = typeof settings?.repoId === "string"
? { repoId: settings.repoId, generation: settings.cacheGeneration, revision: settings.cacheRevision } : undefined;
const assertActive = async () => {
if (!await scopeActive(scope)) throw new AnonymousError("repository_changed", { httpStatus: 409 });
};
const assertActive = () => assertTransformScopeActive(options);
const produceActive = async () => {
await assertActive();
const changed = await produce();
@@ -63,9 +70,9 @@ export async function transformedFile(digest: string, options: unknown, output:
};
const hit = await read();
if (hit !== undefined) {
try { const copied = await copyCached(hit); recordCacheHit("transformed", true); return copied; } catch { /* eviction: retry the transformation */ }
try { const copied = await copyCached(hit); recordCacheHit("transformed", true); observeCache("legacy_result_hit"); return copied; } catch { /* eviction: retry the transformation */ }
}
recordCacheHit("transformed", false);
recordCacheHit("transformed", false); observeCache("legacy_result_miss");
const pending = fills.get(key);
if (pending) {
await pending.catch(() => {});
+11
View File
@@ -1,3 +1,5 @@
import { cacheMetricsReport } from "../../core/cache-observability";
import { sharedBlobCache, resultBlobCache } from "../../core/blob-cache";
import { AsyncCache } from "../../core/async-cache";
import * as os from "os";
import { execFile } from "child_process";
@@ -88,6 +90,15 @@ router.get("/performance", async (req, res) => {
catch { res.json({ available: false, instances: [], routes: [], stages: [] }); }
});
router.get("/cache", async (req, res) => {
res.set("Cache-Control", "no-store");
try {
res.json({ ...cacheMetricsReport(Number(req.query.minutes)), enabled: config.SHARED_CACHE_ENABLED,
filesystemSupported: config.STORAGE === "filesystem", source: sharedBlobCache()?.stats() || null,
results: resultBlobCache()?.stats() || null });
} catch { res.json({ available: false, reason: "The cache index is unavailable", rows: [] }); }
});
function dashboardCache(
_req: express.Request,
res: express.Response,
+2 -1
View File
@@ -146,7 +146,8 @@ router.get(
anonymizedPath,
repo.model.options
);
res.header("ETag", etag);
if (config.SHARED_CACHE_ENABLED) res.vary("Accept-Encoding");
res.header("ETag", config.SHARED_CACHE_ENABLED ? `W/${etag}` : etag);
// Force the browser to revalidate every time. The previous 210-day
// max-age was keyed only on the upstream sha, so editing the
// anonymization term list left old anonymizations cached under the
+2
View File
@@ -1,3 +1,4 @@
import { sendSharedContent } from "../core/cached-response";
import * as express from "express";
import GitHubStream from "../core/source/GitHubStream";
import {
@@ -99,6 +100,7 @@ router.post("/", async (req: express.Request, res: express.Response) => {
} else if (isTextFile(filePath)) {
res.contentType("text/plain");
}
if (await sendSharedContent(content, res, anonymizer.opt)) { anonymizer.destroy(); return; }
// Only declare Accept-Ranges: none for text files — they get rewritten on
// the fly so byte ranges aren't meaningful. For binary entries the
// transformer is a passthrough; let <video>/<audio> fall back to a full
+11
View File
@@ -72,6 +72,17 @@ describe("download headers and website revalidation", function () {
expect(response.headers["content-disposition"]).to.include("attachment");
expect(response.headers["x-content-type-options"]).to.equal("nosniff");
});
it("uses representation-safe validators for shared-cache HEAD and revalidation without reading bytes", async function () {
const previous = config.SHARED_CACHE_ENABLED; config.SHARED_CACHE_ENABLED = true;
try {
const path = "/api/repo/fixture/file/data.txt";
const head = await fetch(path, "HEAD", { "accept-encoding": "gzip" });
expect(head.status).to.equal(200); expect(head.headers.etag).to.match(/^W\/"f-/);
expect(head.headers.vary).to.include("Accept-Encoding"); expect(head.body).to.equal("");
const fresh = await fetch(path, "GET", { "if-none-match": head.headers.etag, "accept-encoding": "identity" });
expect(fresh.status).to.equal(304); expect(transfers).to.equal(0);
} finally { config.SHARED_CACHE_ENABLED = previous; }
});
it("preserves content gates for HEAD", async function () {
repo.options.pdf = false;
expect((await fetch("/api/repo/fixture/file/private.pdf", "HEAD")).status).to.equal(403);
+28
View File
@@ -0,0 +1,28 @@
require("ts-node/register/transpile-only");
const { setTimeout } = require("node:timers");
const { Readable } = require("node:stream");
const { BlobCache } = require("../../src/core/blob-cache");
require("../../src/config").default.CACHE_MONITORING_ENABLED = false;
const cache = new BlobCache(JSON.parse(process.env.BLOB_TEST_OPTIONS));
if (process.env.BLOB_TEST_CRASH_AFTER_RENAME) {
const fs = require("node:fs").promises, rename = fs.rename.bind(fs);
fs.rename = async (...args) => {
await rename(...args);
process.send({ renamed: args[1] });
await new Promise(() => {}); // Parent kills the process before index publication.
};
}
if (process.env.BLOB_TEST_DISK_BYTES) require("node:fs").statfsSync = () => ({ bavail: Number(process.env.BLOB_TEST_DISK_BYTES), bsize: 1 });
process.send("ready");
process.once("message", async () => {
try {
const stream = await cache.getOrFill("multiprocess", async () => {
process.send("fill");
if (process.env.BLOB_TEST_HOLD_FILL) await new Promise(resolve => process.once("message", resolve));
else await new Promise(resolve => setTimeout(resolve, 100));
return Readable.from("shared across processes");
});
let body = ""; for await (const chunk of stream) body += chunk;
cache.close(); process.send({ body }); process.disconnect();
} catch (error) { process.send({ error: error.message }); process.exitCode = 1; process.disconnect(); }
});
+10 -2
View File
@@ -16,9 +16,9 @@ describe("performance report access", function () {
server = await new Promise(resolve => { const s = app.listen(0, "127.0.0.1", () => resolve(s)); });
});
after(async function () { server.closeAllConnections(); await new Promise(resolve => server.close(resolve)); utils.getUser = original; });
async function request(user) {
async function request(user, path = "/api/admin/performance") {
return new Promise((resolve, reject) => {
http.get({ host: "127.0.0.1", port: server.address().port, path: "/api/admin/performance", headers: user ? { "x-fixture-user": user } : {} }, res => {
http.get({ host: "127.0.0.1", port: server.address().port, path, headers: user ? { "x-fixture-user": user } : {} }, res => {
let body = ""; res.on("data", chunk => { body += chunk; }); res.on("end", () => resolve({ status: res.statusCode, body, headers: res.headers }));
}).on("error", reject);
});
@@ -29,4 +29,12 @@ describe("performance report access", function () {
const result = await request("admin"); expect(result.status).to.equal(200);
expect(JSON.parse(result.body).available).to.equal(false); expect(result.headers["cache-control"]).to.equal("no-store");
});
it("restricts cache telemetry to admins and disables response caching", async function () {
expect((await request(undefined, "/api/admin/cache")).status).to.equal(401);
expect((await request("regular", "/api/admin/cache")).status).to.equal(401);
const result = await request("admin", "/api/admin/cache");
expect(result.status).to.equal(200); expect(result.headers["cache-control"]).to.equal("no-store");
expect(JSON.parse(result.body)).to.have.property("available");
});
});
+363
View File
@@ -0,0 +1,363 @@
const { strict: assert } = require("node:assert");
const { promises: fs } = require("node:fs");
const { tmpdir } = require("node:os");
const { join } = require("node:path");
const { Readable } = require("node:stream");
const { createHash, randomBytes } = require("node:crypto");
const { setTimeout: delay } = require("node:timers/promises");
const { gunzipSync } = require("node:zlib");
require("ts-node/register/transpile-only");
const { BlobCache, CacheAdmissionError } = require("../src/core/blob-cache");
const config = require("../src/config").default;
const read = async stream => { const chunks = []; for await (const b of stream) chunks.push(Buffer.from(b)); return Buffer.concat(chunks); };
const digest = b => "git-" + createHash("sha1").update(`blob ${b.length}\0`).update(b).digest("hex");
describe("shared content-addressed blob pool", function () {
this.timeout(10000);
let root, cache, oldMonitoring;
beforeEach(async () => { oldMonitoring = config.CACHE_MONITORING_ENABLED; config.CACHE_MONITORING_ENABLED = false;
root = await fs.mkdtemp(join(tmpdir(), "shared-blob-test-"));
cache = new BlobCache({ root, budget: 65536, maxObject: 4096, gzip: true }); });
afterEach(async () => { cache.close(); config.CACHE_MONITORING_ENABLED = oldMonitoring; await fs.rm(root, { recursive: true, force: true }); });
it("stores one verified blob across project aliases and independently reads gzip and identity", async () => {
const b = Buffer.from("shared text\n".repeat(300));
const verify = async p => { assert.equal("git-" + p.gitSha, digest(b)); return digest(b); };
assert.deepEqual(await read(await cache.getOrFill("project-a", async () => Readable.from(b), verify)), b);
assert.deepEqual(await read(await cache.getOrFill("project-b", async () => Readable.from(b), verify)), b);
assert.equal(cache.stats().blobs, 1); assert.ok(cache.stats().storedBytes < b.length / 2);
assert.deepEqual(gunzipSync(await read(await cache.open(digest(b), true))), b);
assert.deepEqual(await read(await cache.lookup("project-a")), b);
});
it("keeps summary counters exact across connections, replacement, rollback and migration", async () => {
const check = store => {
const expected = store.db.prepare(`SELECT COUNT(*) blobs,COALESCE(SUM(size),0) originalBytes,
COALESCE(SUM(stored),0) storedBytes,COALESCE(SUM(allocated),0) allocatedBytes,
(SELECT COUNT(*) FROM readers) activeReaders,(SELECT COUNT(*) FROM decisions) decisions,
(SELECT COUNT(*) FROM aliases) aliases,(SELECT COUNT(*) FROM history) history,
(SELECT COALESCE(SUM(reserved),0) FROM fills) reservedBytes FROM blobs`).get();
const actual = store.db.prepare("SELECT * FROM summary WHERE id=1").get();
for (const [key, value] of Object.entries(expected)) assert.equal(actual[key], value, key);
};
const stream = await cache.getOrFill("counter-a", async () => Readable.from("same content"));
check(cache);
const other = new BlobCache(cache.options);
try {
await read(await other.getOrFill("counter-b", async () => Readable.from("same content")));
other.saveDecision("rule", stream.cachedBlob.record.digest, false);
other.saveDecision("rule", stream.cachedBlob.record.digest, true);
check(cache); check(other);
other.db.exec("BEGIN; INSERT OR REPLACE INTO history VALUES('counter-a',0,1); DELETE FROM decisions; ROLLBACK;");
check(cache);
await read(stream); await delay(20); check(other);
other.db.prepare("DELETE FROM blobs").run(); check(cache);
} finally { other.close(); }
// Simulate an index created before the summary migration.
for (const row of cache.db.prepare("SELECT name FROM sqlite_master WHERE type='trigger' AND name LIKE 'summary_%'").all())
cache.db.exec('DROP TRIGGER "' + row.name + '"');
cache.db.exec("DROP TABLE summary");
cache.close(); cache = new BlobCache(cache.options); check(cache);
});
it("upgrades old decisions without inventing a sniffed content classification", () => {
cache.saveDecision("old", "git-" + "a".repeat(40), false);
cache.db.exec("ALTER TABLE decisions DROP COLUMN isText");
cache.close(); cache = new BlobCache(cache.options);
assert.equal(cache.decision("old").isText, undefined);
cache.saveDecision("old", "git-" + "a".repeat(40), false, true);
assert.equal(cache.decision("old").isText, true);
assert.equal(cache.stats().decisions, 1);
});
it("uses durable commits for reservations and publication while keeping hit bookkeeping disposable", async () => {
assert.equal(cache.db.prepare("PRAGMA synchronous").get().synchronous, 1);
const original = cache.db.exec.bind(cache.db), commits = [];
cache.db.exec = sql => {
if (sql === "COMMIT") commits.push(Number(cache.db.prepare("PRAGMA synchronous").get().synchronous));
return original(sql);
};
await read(await cache.getOrFill("durability", async () => Readable.from("durable bytes")));
assert.equal(commits.filter(x => x === 2).length, 3);
commits.length = 0;
await read(await cache.getOrFill("durability", async () => { throw Error("unexpected miss"); }));
assert.ok(commits.length > 0); assert.ok(commits.every(x => x === 1));
cache.options.budget = 1;
await assert.rejects(cache.getOrFill("no-space", async () => Readable.from("x")), CacheAdmissionError);
assert.equal(cache.db.prepare("PRAGMA synchronous").get().synchronous, 1);
});
it("coalesces fills across independent index connections", async () => {
const other = new BlobCache(cache.options); let calls = 0;
try {
const produce = async () => { calls++; await delay(50); return Readable.from("complete output"); };
const streams = await Promise.all(Array.from({ length: 8 }, (_, i) => (i % 2 ? other : cache).getOrFill("same", produce)));
const values = await Promise.all(streams.map(read)); assert.equal(calls, 1);
assert.ok(values.every(b => b.toString() === "complete output"));
} finally { other.close(); }
});
it("coalesces fills across separate operating-system processes", async function () {
this.timeout(45000);
const { fork } = require("node:child_process");
const workers = [0, 1].map(() => fork(join(__dirname, "fixtures/shared-cache-worker.js"), [], {
env: { ...process.env, NODE_ENV: "test", BLOB_TEST_OPTIONS: JSON.stringify(cache.options) }, stdio: ["ignore", "ignore", "pipe", "ipc"]
}));
let fills = 0;
const readiness = workers.map(worker => new Promise((resolve, reject) => {
let stderr = "";
worker.stderr.on("data", chunk => { stderr = (stderr + chunk).slice(-4000); });
worker.on("message", message => { if (message === "ready") resolve(); });
worker.on("error", reject);
worker.on("exit", code => reject(Error(`worker exited ${code}: ${stderr}`)));
}));
try {
const results = workers.map(worker => new Promise((resolve, reject) => {
worker.on("message", message => {
if (message === "fill") fills++;
if (message.error) reject(Error(message.error));
if (message.body) resolve(message.body);
});
worker.on("error", reject);
worker.on("exit", code => { if (code) reject(Error("worker exited " + code)); });
}));
await Promise.all(readiness);
workers.forEach(worker => worker.send("start"));
assert.deepEqual(await Promise.all(results), ["shared across processes", "shared across processes"]);
assert.equal(fills, 1); assert.equal(cache.stats().blobs, 1);
} finally { workers.forEach(worker => worker.kill()); }
});
it("recovers a producer killed after rename without losing its disk reservation", async function () {
this.timeout(45000);
const { fork } = require("node:child_process");
const worker = fork(join(__dirname, "fixtures/shared-cache-worker.js"), [], {
env: { ...process.env, NODE_ENV: "test", BLOB_TEST_OPTIONS: JSON.stringify(cache.options), BLOB_TEST_CRASH_AFTER_RENAME: "1" },
stdio: ["ignore", "ignore", "pipe", "ipc"]
});
try {
const renamed = await new Promise((resolve, reject) => {
let stderr = "";
worker.stderr.on("data", chunk => { stderr = (stderr + chunk).slice(-4000); });
worker.on("error", reject);
worker.on("exit", code => reject(Error("premature exit " + code + ": " + stderr)));
worker.on("message", message => {
if (message === "ready") worker.send("start");
if (message.renamed) resolve(message.renamed);
if (message.error) reject(Error(message.error));
});
});
const exited = new Promise(resolve => worker.once("exit", resolve));
worker.kill("SIGKILL"); await exited;
assert.ok(cache.stats().reservedBytes > 0);
assert.equal(cache.disk.prepare("SELECT COUNT(*) n FROM reservations").get().n, 1);
assert.equal(cache.stats().blobs, 0);
assert.equal(cache.db.prepare("SELECT COUNT(*) n FROM publications").get().n, 1);
await fs.access(renamed);
cache.db.exec("UPDATE fills SET expires=1");
await cache.maintain();
await assert.rejects(fs.access(renamed), { code: "ENOENT" });
assert.equal(cache.stats().reservedBytes, 0);
assert.equal(cache.disk.prepare("SELECT COUNT(*) n FROM reservations").get().n, 0);
assert.equal(cache.db.prepare("SELECT COUNT(*) n FROM publications").get().n, 0);
assert.equal((await read(await cache.getOrFill("multiprocess", async () => Readable.from("recovered")))).toString(), "recovered");
} finally { worker.kill(); }
});
it("pins a duplicate publication while another connection tries to evict it", async () => {
await read(await cache.getOrFill("original", async () => Readable.from("identical")));
const other = new BlobCache({ ...cache.options, budget: 8192, maxObject: 1024 });
const unlink = fs.unlink.bind(fs); let competed = false;
fs.unlink = async path => {
if (!competed && path.includes("/blobs/") && !cache.db.prepare("SELECT 1 FROM blobs WHERE file=?").get(require("node:path").basename(path))) {
competed = true;
await assert.rejects(other.getOrFill("competitor", async () => Readable.from("different")), CacheAdmissionError);
}
return unlink(path);
};
try {
assert.equal((await read(await cache.getOrFill("duplicate", async () => Readable.from("identical")))).toString(), "identical");
assert.ok(competed);
await delay(20); assert.equal(cache.stats().activeReaders, 0);
} finally { fs.unlink = unlink; other.close(); }
});
it("turns SQLite write contention into a cache bypass and preserves existing data", async () => {
await read(await cache.getOrFill("locked", async () => Readable.from("available")));
const other = new BlobCache(cache.options);
other.db.exec("BEGIN IMMEDIATE");
try {
await assert.rejects(cache.getOrFill("locked", async () => { throw Error("unexpected fill"); }),
error => error instanceof CacheAdmissionError && error.message === "cache_index_busy");
} finally { other.db.exec("ROLLBACK"); other.close(); }
assert.equal((await read(await cache.lookup("locked"))).toString(), "available");
await delay(20); assert.equal(cache.stats().activeReaders, 0);
});
it("retries a contended reader heartbeat while its confirmed lease is valid", async () => {
await read(await cache.getOrFill("heartbeat", async () => Readable.from("still readable")));
const other = new BlobCache(cache.options);
const interval = global.setInterval, now = Date.now; let tick, reader;
global.setInterval = fn => { tick = fn; return interval(fn, 10000000); };
try { reader = await cache.lookup("heartbeat"); } finally { global.setInterval = interval; }
const base = now();
try {
other.db.exec("BEGIN IMMEDIATE"); Date.now = () => base + 31000; tick();
assert.equal(reader.destroyed, false);
other.db.exec("ROLLBACK"); Date.now = () => base + 32000; tick();
assert.ok(Number(cache.db.prepare("SELECT MAX(expires) n FROM readers").get().n) >= base + 152000);
Date.now = now;
assert.equal((await read(reader)).toString(), "still readable");
} finally { Date.now = now; reader.destroy(); if (other.db.isTransaction) other.db.exec("ROLLBACK"); other.close(); }
});
it("ends a contended reader once its confirmed lease expires", async () => {
await read(await cache.getOrFill("expiry", async () => Readable.from("data")));
const other = new BlobCache(cache.options), interval = global.setInterval, now = Date.now;
let tick, reader;
global.setInterval = fn => { tick = fn; return interval(fn, 10000000); };
try { reader = await cache.lookup("expiry"); } finally { global.setInterval = interval; }
const error = new Promise(resolve => reader.once("error", resolve)), base = now();
try {
other.db.exec("BEGIN IMMEDIATE"); Date.now = () => base + 121000; tick();
assert.equal((await error).message, "cache_reader_lease_lost");
} finally { Date.now = now; other.db.exec("ROLLBACK"); reader.destroy(); other.close(); }
});
for (const outcome of ["recover", "expire", "replaced"]) {
it(`handles fill heartbeat contention and ${outcome}`, async () => {
const other = new BlobCache(cache.options), interval = global.setInterval, now = Date.now;
let tick, started;
const ready = new Promise(resolve => { started = resolve; });
const input = new Readable({ read() {} });
global.setInterval = fn => { tick = fn; return interval(fn, 10000000); };
const fill = cache.getOrFill("fill-heartbeat", async () => { started(); return input; });
// Attach rejection handling before deliberately terminating the producer.
const result = fill.then(read);
const checked = outcome === "recover" ? result : assert.rejects(result, /cache_fill_lease_lost/);
try {
await ready; global.setInterval = interval; await delay(10);
const base = now();
other.db.exec("BEGIN IMMEDIATE"); Date.now = () => base + 31000; tick();
assert.equal(input.destroyed, false);
other.db.exec("ROLLBACK");
if (outcome === "recover") {
Date.now = () => base + 32000; tick();
assert.ok(Number(cache.db.prepare("SELECT expires FROM fills WHERE key=?").get("fill-heartbeat").expires) >= base + 152000);
Date.now = now; input.push("completed after contention"); input.push(null);
assert.equal((await checked).toString(), "completed after contention");
} else {
if (outcome === "replaced") other.db.prepare("UPDATE fills SET token=? WHERE key=?").run("replacement", "fill-heartbeat");
Date.now = () => base + (outcome === "expire" ? 121000 : 32000); tick();
assert.equal(input.destroyed, true);
Date.now = now; await checked;
assert.equal(cache.stats().blobs, 0);
}
} finally {
global.setInterval = interval; Date.now = now;
if (other.db.isTransaction) other.db.exec("ROLLBACK");
input.destroy(); await checked.catch(() => {}); other.close();
}
});
}
it("coordinates physical headroom between source and result pools", async () => {
const native = require("node:fs"), statfs = native.statfsSync;
const other = new BlobCache({ ...cache.options, root: join(root, "results"), reservationRoot: root, freeBytes: 1000 });
cache.options.freeBytes = 1000;
native.statfsSync = () => ({ bavail: 18000, bsize: 1 });
let release, started;
const pending = new Promise(resolve => { started = resolve; });
const fill = cache.getOrFill("source-pending", async () => {
started(); await new Promise(resolve => { release = resolve; }); return Readable.from("source");
});
try {
await pending;
await assert.rejects(other.getOrFill("result-pending", async () => Readable.from("result")),
error => error instanceof CacheAdmissionError && error.message === "disk_headroom");
release(); await read(await fill);
assert.equal((await read(await other.getOrFill("result-pending", async () => Readable.from("result")))).toString(), "result");
assert.equal(cache.disk.prepare("SELECT COUNT(*) n FROM reservations").get().n, 0);
} finally { release?.(); await fill.catch(() => {}); native.statfsSync = statfs; other.close(); }
});
it("shares physical headroom reservations across separate producer processes", async function () {
this.timeout(45000);
const { fork } = require("node:child_process");
const workers = ["source-process", "result-process"].map(name => fork(join(__dirname, "fixtures/shared-cache-worker.js"), [], {
env: { ...process.env, NODE_ENV: "test", BLOB_TEST_DISK_BYTES: "18000", BLOB_TEST_HOLD_FILL: "1",
BLOB_TEST_OPTIONS: JSON.stringify({ ...cache.options, root: join(root, name), reservationRoot: root, freeBytes: 1000 }) },
stdio: ["ignore", "ignore", "pipe", "ipc"]
}));
try {
const ready = [], outcomes = [], bodies = [];
for (const worker of workers) {
let stderr = ""; worker.stderr.on("data", b => { stderr = (stderr + b).slice(-4000); });
ready.push(new Promise((resolve, reject) => {
worker.on("message", m => { if (m === "ready") resolve(); });
worker.on("error", reject); worker.on("exit", code => reject(Error("worker exited " + code + ": " + stderr)));
}));
outcomes.push(new Promise((resolve, reject) => {
worker.on("message", m => { if (m === "fill" || m.error) resolve(m); }); worker.on("error", reject);
}));
bodies.push(new Promise(resolve => worker.on("message", m => { if (m.body || m.error) resolve(m); })));
}
await Promise.all(ready); workers.forEach(w => w.send("start"));
const values = await Promise.all(outcomes);
assert.equal(values.filter(v => v === "fill").length, 1);
assert.equal(values.find(v => v.error)?.error, "disk_headroom");
workers[values.indexOf("fill")].send("release");
const completed = await Promise.all(bodies);
assert.equal(completed.find(v => v.body)?.body, "shared across processes");
assert.equal(cache.disk.prepare("SELECT COUNT(*) n FROM reservations").get().n, 0);
} finally { workers.forEach(w => w.kill()); }
});
it("commits completed evictions when pinned readers still prevent admission", async () => {
for (let i = 0; i < 11; i++) await read(await cache.getOrFill("evict-" + i, async () => Readable.from("content-" + i)));
const pinned = await Promise.all(Array.from({ length: 10 }, (_, i) => cache.lookup("evict-" + i)));
let release, started;
const began = new Promise(resolve => { started = resolve; });
const pending = cache.getOrFill("pending", async () => {
started(); await new Promise(resolve => { release = resolve; }); return Readable.from("pending content");
});
try {
await began;
await assert.rejects(cache.getOrFill("denied", async () => Readable.from("new")),
error => error instanceof CacheAdmissionError && error.message === "cache_budget_busy");
assert.equal(cache.stats().blobs, 10); assert.equal(cache.stats().allocatedBytes, 10 * 4096);
assert.equal(cache.db.prepare("SELECT 1 FROM aliases WHERE key='evict-10'").get(), undefined);
for (const row of cache.db.prepare("SELECT * FROM blobs").all())
await fs.access(join(root, "blobs", row.digest.split("-")[1].slice(0, 2), row.digest, row.file));
release(); await read(await pending);
} finally { release?.(); await pending.catch(() => {}); pinned.forEach(stream => stream.destroy()); }
});
it("does not publish failed or hash-invalid fills and releases reservations", async () => {
await assert.rejects(cache.getOrFill("invalid", async () => Readable.from("bad"), async () => { throw Error("invalid_hash"); }), /invalid_hash/);
assert.equal(cache.stats().blobs, 0); assert.equal(cache.stats().reservedBytes, 0);
assert.deepEqual(await fs.readdir(join(root, "tmp")), []);
await assert.rejects(cache.getOrFill("large", async () => Readable.from(Buffer.alloc(5000))), CacheAdmissionError);
assert.equal(cache.stats().reservedBytes, 0);
});
it("evicts least-recently-read data while retaining active readers and respecting the budget", async () => {
const data = randomBytes(4000);
for (let i = 0; i < 12; i++) { const b = Buffer.concat([Buffer.from([i]), data]); await read(await cache.getOrFill("file" + i, async () => Readable.from(b))); }
const pinned = await cache.lookup("file0"); assert.ok(pinned);
const pinnedDigest = pinned.cachedBlob.record.digest;
cache.db.prepare("UPDATE blobs SET accessed=1 WHERE digest<>?").run(pinnedDigest);
for (let i = 12; i < 20; i++) { const b = Buffer.concat([Buffer.from([i]), data]); await read(await cache.getOrFill("file" + i, async () => Readable.from(b))); }
assert.ok(cache.stats().storedBytes + cache.stats().reservedBytes <= cache.options.budget);
assert.ok(cache.db.prepare("SELECT 1 FROM blobs WHERE digest=?").get(pinnedDigest));
assert.equal((await read(pinned)).length, 4001);
assert.equal(await cache.lookup("file1"), undefined);
assert.deepEqual(await read(await cache.getOrFill("file1", async () => Readable.from("refetched"))), Buffer.from("refetched"));
});
it("recovers an expired producer and removes its temporary files before admitting new work", async () => {
cache.db.prepare("INSERT INTO fills VALUES(?,?,?,?)").run("dead", "dead", 1, 60000);
await fs.writeFile(join(root, "tmp", "dead.raw"), "partial");
const orphanDigest = "sha256-" + "a".repeat(64);
const orphanDir = join(root, "blobs", "aa", orphanDigest);
await fs.mkdir(orphanDir, { recursive: true }); await fs.writeFile(join(orphanDir, "dead.raw"), "unindexed");
cache.db.prepare("INSERT INTO publications VALUES(?,?,?)").run("dead", orphanDigest, "dead.raw");
await read(await cache.getOrFill("new", async () => Readable.from("new")));
assert.equal(cache.stats().reservedBytes, 0); assert.deepEqual(await fs.readdir(join(root, "tmp")), []);
assert.deepEqual(await fs.readdir(orphanDir), []);
});
it("invalidates a missing or truncated file and refetches it", async () => {
const stream = await cache.getOrFill("missing", async () => Readable.from("hello"));
const record = stream.cachedBlob.record; await read(stream);
const p = join(root, "blobs", record.digest.split("-")[1].slice(0, 2), record.digest, record.file);
await fs.writeFile(p, "x"); assert.equal(await cache.lookup("missing"), undefined);
assert.deepEqual(await read(await cache.getOrFill("missing", async () => Readable.from("hello"))), Buffer.from("hello"));
});
it("bounds decision retention and expires old transformation settings", () => {
cache.saveDecision("rule-v1", "git-" + "a".repeat(40), false);
assert.equal(cache.decision("rule-v1").changed, false); assert.equal(cache.decision("rule-v2"), undefined);
cache.db.prepare("UPDATE decisions SET expires=1").run(); assert.equal(cache.decision("rule-v1"), undefined);
});
});
+267
View File
@@ -0,0 +1,267 @@
const { strict: assert } = require("node:assert");
const { promises: fs } = require("node:fs");
const { tmpdir } = require("node:os");
const { join } = require("node:path");
const http = require("node:http");
const { setTimeout } = require("node:timers");
const { Readable } = require("node:stream");
const { createHash } = require("node:crypto");
const { gunzipSync } = require("node:zlib");
const express = require("express");
require("ts-node/register/transpile-only");
const config = require("../src/config").default;
const { sharedBlobCache, resultBlobCache, closeBlobCaches } = require("../src/core/blob-cache");
const { sendSharedContent } = require("../src/core/cached-response");
const { closeCacheMetrics, cacheMetricsReport, observeCache } = require("../src/core/cache-observability");
const { monitorRequests } = require("../src/core/request-monitoring");
const GitHubStream = require("../src/core/source/GitHubStream").default;
const sha = b => createHash("sha1").update(`blob ${b.length}\0`).update(b).digest("hex");
const consume = async s => { const chunks = []; for await (const b of s) chunks.push(Buffer.from(b)); return Buffer.concat(chunks); };
function request(port, path, encoding = "gzip", extra = {}) {
return new Promise((resolve, reject) => {
const req = http.get({ host: "127.0.0.1", port, path, headers: { "accept-encoding": encoding, ...extra } }, res => {
consume(res).then(body => resolve({ status: res.statusCode, headers: res.headers, body }), reject);
}); req.on("error", reject);
});
}
describe("shared cache HTTP and telemetry integration", function () {
this.timeout(15000);
let root, old, server, port, fills, body;
beforeEach(async () => {
old = { ...config }; closeBlobCaches(); closeCacheMetrics();
root = await fs.mkdtemp(join(tmpdir(), "shared-http-"));
Object.assign(config, { FOLDER: root, SHARED_CACHE_PATH: join(root, "cache"), SHARED_CACHE_ENABLED: true,
SHARED_CACHE_BYTES: 16 * 1024 ** 2, SHARED_CACHE_RESULT_BYTES: 8 * 1024 ** 2, SHARED_CACHE_MAX_FILE_BYTES: 1024 ** 2,
SHARED_CACHE_FREE_BYTES: 0, CACHE_MONITORING_ENABLED: true });
fills = 0; body = Buffer.from("Public documentation with no identifying names.\n".repeat(100));
const app = express(); app.use(monitorRequests("api", undefined, { info() {}, warn() {} }));
app.get("/:project/:mode", async (req, res) => {
if (req.headers.authorization !== "test-access") return res.status(403).end();
const source = new GitHubStream({ repoId: req.params.project, organization: "org", repoName: "repo", commit: "commit", getToken: async () => "" });
source.downloadWithFallback = async () => { fills++; return Readable.from(body); };
source.downloadFile = () => Readable.from("invalid pointer");
const filePath = String(req.query.name || "README.md");
const mime = require("mime-types").lookup(filePath);
if (mime) res.type(mime);
try {
const input = await source.getFileContentCache(filePath, req.params.project, () => ({ sha: sha(body), size: body.length }));
const handled = await sendSharedContent(input, res, { filePath, repoId: req.params.project,
terms: req.params.mode === "changed" ? ["Public=>Private"] : ["Alice"] });
if (!handled) res.send(await consume(input));
} catch (e) { if (!res.headersSent) res.status(e.httpStatus || 500).end(e.message); else res.destroy(e); }
});
await new Promise(resolve => { server = app.listen(0, "127.0.0.1", resolve); }); port = server.address().port;
});
afterEach(async () => {
await new Promise(resolve => server.close(resolve)); closeBlobCaches(); closeCacheMetrics(); Object.assign(config, old);
await fs.rm(root, { recursive: true, force: true });
});
const headers = { authorization: "test-access" };
it("shares source content across projects without bypassing authorization", async () => {
assert.equal((await request(port, "/a/plain")).status, 403); assert.equal(fills, 0);
const a = await request(port, "/a/plain", "gzip", headers);
assert.equal(a.status, 200); assert.equal(a.headers["content-encoding"], "gzip"); assert.deepEqual(gunzipSync(a.body), body);
const b = await request(port, "/b/plain", "identity", headers);
assert.deepEqual(b.body, body); assert.equal(b.headers["content-encoding"], undefined); assert.equal(fills, 1);
assert.equal(sharedBlobCache().stats().blobs, 1); assert.equal(resultBlobCache().stats().blobs, 0);
assert.equal((await request(port, "/b/plain", "gzip")).status, 403);
});
it("fetches public submodule content from its origin without forwarding parent credentials", async () => {
const original = GitHubStream.prototype.downloadWithFallback;
const upstream = { organization: "public-org", repoName: "dependency", commit: "a".repeat(40), path: "src/data.txt" };
let calls = 0;
GitHubStream.prototype.downloadWithFallback = async function (token, digest, path) {
calls++;
assert.equal(token, ""); assert.equal(digest, sha(body)); assert.equal(path, upstream.path);
assert.equal(this.data.organization, upstream.organization); assert.equal(this.data.repoName, upstream.repoName);
assert.equal(this.data.commit, upstream.commit);
return Readable.from(body);
};
try {
const source = new GitHubStream({ repoId: "parent", organization: "private-org", repoName: "parent",
commit: "parent-commit", getToken: async () => "parent-secret" });
const meta = () => ({ sha: sha(body), size: body.length, upstream });
assert.deepEqual(await consume(await source.getFileContentCache("vendor/src/data.txt", "parent", meta)), body);
assert.deepEqual(await consume(await source.getFileContentCache("vendor/src/data.txt", "parent", meta)), body);
assert.equal(calls, 1);
} finally { GitHubStream.prototype.downloadWithFallback = original; }
});
for (const change of ["expired", "removed", "revision"]) it("rejects a large rewrite retired during worker execution: " + change, async () => {
const db = require("../src/server/database"), model = require("../src/core/model/anonymizedRepositories/anonymizedRepositories.model").default;
const pool = require("../src/core/anonymization-pool"), worker = pool.anonymizeOnWorker, find = model.findOne, connected = db.isConnected;
const current = { status: "ready" }; let release, started;
const began = new Promise(resolve => { started = resolve; }), held = new Promise(resolve => { release = resolve; });
body = Buffer.from("Public documentation.\n".repeat(20000));
db.isConnected = true;
model.findOne = () => ({ select() { return this; }, lean() { return this; }, exec: async () => current });
pool.anonymizeOnWorker = async message => {
started(); await held;
await fs.writeFile(message.output, (await fs.readFile(message.input, "utf8")).replaceAll("Public", "Private"));
return true;
};
let pending;
try {
pending = request(port, "/a/changed", "identity", headers); await began;
if (change === "revision") current.contentCacheRevision = "new"; else current.status = change;
release(); const r = await pending;
assert.equal(r.status, 409); assert.equal(r.body.toString(), "repository_changed");
assert.equal(resultBlobCache().stats().blobs, 0); assert.equal(resultBlobCache().stats().decisions, 0);
} finally { release(); await pending?.catch(() => {}); pool.anonymizeOnWorker = worker; model.findOne = find; db.isConnected = connected; }
});
it("rechecks repository lifecycle on a decision hit before sending bytes", async () => {
await request(port, "/a/changed", "identity", headers);
const db = require("../src/server/database"), model = require("../src/core/model/anonymizedRepositories/anonymizedRepositories.model").default;
const results = resultBlobCache(), open = results.open.bind(results), find = model.findOne, connected = db.isConnected;
const current = { status: "ready" }; db.isConnected = true;
model.findOne = () => ({ select() { return this; }, lean() { return this; }, exec: async () => current });
results.open = async (...args) => { const stream = await open(...args); current.status = "removed"; return stream; };
try {
const r = await request(port, "/a/changed", "identity", headers);
assert.equal(r.status, 409); assert.equal(r.body.toString(), "repository_changed");
} finally { results.open = open; model.findOne = find; db.isConnected = connected; }
});
it("cancels an active worker and releases spools when the client disconnects", async () => {
const pool = require("../src/core/anonymization-pool"), worker = pool.anonymizeOnWorker;
const baseline = pool.getAnonymizationPoolStats().spoolBytes;
let started, aborted, rejectWork, client;
const began = new Promise(resolve => { started = resolve; }), cancelled = new Promise(resolve => { aborted = resolve; });
body = Buffer.from("Public documentation.\n".repeat(20000));
pool.anonymizeOnWorker = (_message, _size, signal) => new Promise((_resolve, reject) => {
rejectWork = reject; started();
signal.addEventListener("abort", () => { aborted(); reject(Error("anonymization_cancelled")); }, { once: true });
});
try {
client = http.get({ host: "127.0.0.1", port, path: "/a/changed", headers }); client.on("error", () => {});
await began; assert.ok(pool.getAnonymizationPoolStats().spoolBytes > baseline);
client.destroy();
await Promise.race([cancelled, new Promise((_, reject) => setTimeout(() => reject(Error("worker was not cancelled")), 2000))]);
for (let i = 0; i < 100 && (pool.getAnonymizationPoolStats().spoolBytes !== baseline || resultBlobCache().stats().reservedBytes); i++)
await new Promise(resolve => setTimeout(resolve, 10));
assert.equal(pool.getAnonymizationPoolStats().spoolBytes, baseline);
assert.equal(resultBlobCache().stats().reservedBytes, 0);
assert.equal(resultBlobCache().stats().blobs, 0);
} finally { client?.destroy(); rejectWork?.(Error("test cleanup")); pool.anonymizeOnWorker = worker; }
});
for (const changed of [false, true]) it("preserves sniffed text MIME on initial and cached responses: " + changed, async () => {
for (const encoding of ["gzip", "identity"]) {
const r = await request(port, "/a/" + (changed ? "changed" : "plain") + "?name=data.unknownext", encoding, headers);
assert.equal(r.status, 200); assert.match(r.headers["content-type"], /^text\/plain/);
const decoded = r.headers["content-encoding"] === "gzip" ? gunzipSync(r.body) : r.body;
assert.equal(decoded.toString(), changed ? body.toString().replaceAll("Public", "Private") : body.toString());
}
});
for (const name of ["video.mp4", "audio.mp3", "binary.unknownext"]) it("omits the range-denial header for binary " + name, async () => {
body = Buffer.from([0, 1, 2, 0, 255]);
for (let i = 0; i < 2; i++) {
const r = await request(port, "/a/plain?name=" + name, "identity", headers);
assert.equal(r.status, 200); assert.equal(r.headers["accept-ranges"], undefined);
assert.notEqual(r.headers["content-type"], "text/plain; charset=utf-8"); assert.deepEqual(r.body, body);
}
});
it("bypasses a contended source index without failing a download", async () => {
await request(port, "/a/plain", "identity", headers);
const { BlobCache } = require("../src/core/blob-cache");
const other = new BlobCache(sharedBlobCache().options); other.db.exec("BEGIN IMMEDIATE");
try {
const r = await request(port, "/a/plain", "identity", headers);
assert.equal(r.status, 200); assert.deepEqual(r.body, body); assert.equal(fills, 2);
} finally { other.db.exec("ROLLBACK"); other.close(); }
});
it("bypasses a contended result index while retaining the source fallback", async () => {
await request(port, "/a/plain", "identity", headers);
const { BlobCache } = require("../src/core/blob-cache");
const other = new BlobCache(resultBlobCache().options); other.db.exec("BEGIN IMMEDIATE");
try {
const r = await request(port, "/a/plain", "identity", headers);
assert.equal(r.status, 200); assert.deepEqual(r.body, body); assert.equal(fills, 1);
} finally { other.db.exec("ROLLBACK"); other.close(); }
});
for (const bypass of [false, true]) for (const streamed of [false, true]) {
it("normalizes upstream permission errors for " + JSON.stringify({ bypass, streamed }), async () => {
if (bypass) sharedBlobCache().options.budget = 1;
const source = new GitHubStream({ repoId: "errors", organization: "org", repoName: "repo", commit: "commit", getToken: async () => "" });
source.downloadWithFallback = async () => {
const error = Object.assign(Error("forbidden"), { response: { statusCode: 403 } });
if (!streamed) throw error;
return Readable.from((async function* () { yield Buffer.alloc(0); throw error; })());
};
await assert.rejects(async () => consume(await source.getFileContentCache("data.txt", "errors", () => ({ sha: sha(body), size: body.length }))),
error => error.message === "file_not_accessible" && error.httpStatus === 403);
});
}
it("rejects truncated admission-bypass downloads before successful completion", async () => {
sharedBlobCache().options.budget = 1;
const source = new GitHubStream({ repoId: "short", organization: "org", repoName: "repo", commit: "commit", getToken: async () => "" });
source.downloadWithFallback = async () => Readable.from("short");
await assert.rejects(async () => consume(await source.getFileContentCache("data.txt", "short", () => ({ sha: sha(body), size: 20 }))),
error => error.message === "storage_write_size_mismatch" && error.httpStatus === 502);
});
it("classifies parent repository misses before returning a shared-download error", async () => {
const misses = require("../src/core/source/github-content-miss"), original = misses.classifyContentMiss;
let classified = 0;
misses.classifyContentMiss = async () => { classified++; return "repo_not_found"; };
try {
const source = new GitHubStream({ repoId: "missing-parent", organization: "org", repoName: "repo", commit: "missing-parent", getToken: async () => "" });
source.downloadWithFallback = async () => { throw Object.assign(Error("missing"), { status: 404 }); };
await assert.rejects(source.getFileContentCache("data.txt", "missing-parent", () => ({ sha: sha(body), size: body.length })),
error => error.message === "repo_not_found" && error.httpStatus === 404);
assert.equal(classified, 1);
} finally { misses.classifyContentMiss = original; }
});
it("classifies unavailable submodule streams on cache bypass", async () => {
sharedBlobCache().options.budget = 1;
const original = GitHubStream.prototype.downloadWithFallback;
GitHubStream.prototype.downloadWithFallback = async () => Readable.from((async function* () {
yield Buffer.alloc(0);
throw Object.assign(Error("missing"), { response: { statusCode: 404 } });
})());
try {
const source = new GitHubStream({ repoId: "errors", organization: "org", repoName: "repo", commit: "commit", getToken: async () => "" });
await assert.rejects(async () => consume(await source.getFileContentCache("vendor/data.txt", "errors", () => ({ sha: sha(body), size: body.length,
upstream: { organization: "public", repoName: "dependency", commit: "a".repeat(40), path: "data.txt" } }))),
error => error.message === "submodule_not_available" && error.httpStatus === 404);
} finally { GitHubStream.prototype.downloadWithFallback = original; }
});
it("reuses unchanged decisions, separates changed rules, and caches small changed output", async () => {
const first = await request(port, "/a/plain", "gzip", headers);
const second = await request(port, "/a/plain", "gzip", headers);
assert.deepEqual(second.body, first.body);
const changed = await request(port, "/a/changed", "gzip", headers);
assert.equal(gunzipSync(changed.body).toString(), body.toString().replaceAll("Public", "Private"));
const repeated = await request(port, "/a/changed", "identity", headers);
assert.equal(repeated.body.toString(), body.toString().replaceAll("Public", "Private"));
assert.equal(resultBlobCache().stats().blobs, 1);
const report = cacheMetricsReport(60); assert.equal(report.available, true);
const count = name => report.rows.filter(r => r.name === name).reduce((n, r) => n + r.count, 0);
assert.equal(count("rewrite_unchanged"), 1); assert.equal(count("rewrite_changed"), 1); assert.equal(count("decision_hit"), 2);
});
it("respects gzip q=0, rejects all forbidden encodings, and gzip-encodes small identity blobs when required", async () => {
let r = await request(port, "/a/plain", "gzip;q=0, identity;q=1", headers);
assert.equal(r.headers["content-encoding"], undefined); assert.deepEqual(r.body, body);
r = await request(port, "/a/plain", "gzip;q=0, identity;q=0", headers); assert.equal(r.status, 406);
body = Buffer.from("tiny text");
r = await request(port, "/a/plain", "gzip, identity;q=0", headers);
assert.equal(r.headers["content-encoding"], "gzip"); assert.deepEqual(gunzipSync(r.body), body);
assert.ok(r.headers.vary.includes("Accept-Encoding"));
});
it("verifies the Git LFS pointer and resolved payload before sharing the payload", async () => {
const payload = Buffer.from("large file payload");
const pointer = Buffer.from(`version https://git-lfs.github.com/spec/v1\noid sha256:${createHash("sha256").update(payload).digest("hex")}\nsize ${payload.length}\n`);
const source = new GitHubStream({ repoId: "lfs", organization: "org", repoName: "repo", commit: "commit", getToken: async () => "" });
source.downloadWithFallback = async () => Readable.from(payload);
source.downloadFile = () => Readable.from(pointer);
assert.deepEqual(await consume(await source.getFileContentCache("data.bin", "lfs", () => ({ sha: sha(pointer), size: pointer.length }))), payload);
const badPointer = Buffer.from(pointer.toString().replace(`size ${payload.length}`, "size 999"));
source.downloadFile = () => Readable.from(badPointer);
await assert.rejects(source.getFileContentCache("bad.bin", "lfs", () => ({ sha: sha(badPointer), size: badPointer.length })), /invalid_blob_hash/);
assert.equal(sharedBlobCache().stats().blobs, 1);
});
it("uses bounded persistent metrics with separate monitoring traffic and no file identifiers", async () => {
await request(port, "/a/plain", "identity", { ...headers, "user-agent": "Uptime-Kuma/1" });
observeCache("compress", 100, 1, "zip");
const report = cacheMetricsReport(60);
assert.ok(report.rows.some(r => r.traffic === "monitor")); assert.ok(report.rows.some(r => r.traffic === "zip"));
assert.ok(!JSON.stringify(report).includes("README")); assert.ok(!JSON.stringify(report).includes(sha(body)));
config.CACHE_MONITORING_ENABLED = false; assert.equal(cacheMetricsReport().available, false);
});
});
+31
View File
@@ -704,6 +704,37 @@ describe("Vue 3 UI", function () {
expect(ui.errors).to.deep.equal([]);
});
it("shows cache budgets and filters monitoring traffic without confusing unavailable data with zero", async function () {
ui = await browser("/", { "/api/user": { username: "admin", isAdmin: true },
"/api/admin/overview": { history: [] },
"/api/admin/cache": { available: true, enabled: true, filesystemSupported: true, sampledAt: Date.now(),
source: { allocatedBytes: 4096, budgetBytes: 65536, reservedBytes: 0, originalBytes: 19000, blobs: 1, activeReaders: 0 },
latencyBucketsMs: [0.1, 1, 10, 100, 1000, 10000],
rows: [{ name: "source_hit", traffic: "file", count: 4, bytes: 400, ms: 0 },
{ name: "source_miss", traffic: "file", count: 1, bytes: 100, ms: 0 },
{ name: "source_hit", traffic: "monitor", count: 10, bytes: 1000, ms: 0 }] } });
await ui.go("/admin/");
const panel = ui.window.document.querySelector(".ov-cache");
expect(panel.textContent).to.include("4.0 KB / 64.0 KB");
await ui.input("#cache-traffic", "file", "change");
expect(panel.textContent).to.include("80.0%");
await ui.input("#cache-traffic", "zip", "change");
expect(panel.textContent).to.include("No cache events");
await ui.input("#cache-window", "1440", "change");
expect(ui.requests.filter(r => r.url.pathname === "/api/admin/cache").at(-1).url.searchParams.get("minutes")).to.equal("1440");
expect(ui.errors).to.deep.equal([]);
});
it("shows unavailable cache telemetry with a retry action", async function () {
ui = await browser("/", { "/api/user": { username: "admin", isAdmin: true },
"/api/admin/overview": { history: [] }, "/api/admin/cache": { available: false, reason: "Cache index unavailable" } });
await ui.go("/admin/");
const panel = ui.window.document.querySelector(".ov-cache");
expect(panel.querySelector('[role="alert"]').textContent).to.include("Cache index unavailable");
expect(panel.querySelector("button").textContent).to.equal("Retry");
expect(panel.textContent).not.to.include("Source request hit rate"); expect(ui.errors).to.deep.equal([]);
});
it("shows latency bounds, interruptions and resource samples on the admin overview", async function () {
const sample = { instance: "streamer:fixture", service: "streamer", sampledAt: Date.now(), cpuPercent: 2,
memory: { rss: 16 * 1024 * 1024 }, memoryLimitBytes: 128 * 1024 * 1024,