Commit Graph
136 Commits
Author SHA1 Message Date
Thomas Durieux 4fd31a59c8 fix: avoid timeouts for safe ZIP anonymization (#873) 2026-10-08 08:20:26 +01:00
Thomas Durieux 40c369641b fix: keep ZIP downloads flowing through compression (#872) 2026-10-08 04:31:52 +01:00
Thomas Durieux 6434337923 perf: bound request work and resource usage (#866)
* perf: bound request work and resource usage

* fix: address performance review regressions

* fix: guard content expiration and dashboard state

* fix: tolerate cache outages and retire derived paths

* fix: revalidate expiration claims and status pagination

* fix: retry interrupted repository metadata cleanup

* fix: retire obsolete content cache storage

* fix: activate tree metadata and recheck expiration claims

* fix: fence cache publication across expiration

* fix: recover staged trees and enforce cache lifecycle cleanup

* fix: prune expired transforms during database outages

* fix: finish conference removals and reload expiration candidates
2026-10-08 03:38:19 +01:00
Thomas Durieux 282efdc16e fix: make dashboard UI consistent and compact on mobile 2026-10-07 14:54:49 +00:00
Thomas Durieux e5dac99598 fix: distinguish upstream stream failures from storage errors 2026-10-07 12:24:23 +00:00
Thomas Durieux e6073941b9 fix: verify reconnects through supported GitHub routes and protect active jobs 2026-10-07 11:24:59 +00:00
Thomas Durieux a7ae790640 fix: recover renamed repositories by ID and hide archived actions 2026-10-07 11:10:30 +00:00
Thomas Durieux b50e6c4565 perf: index anonymization terms before scanning large files 2026-10-07 10:57:09 +00:00
Thomas Durieux 083544bf68 fix: preserve active downloads and atomically prepare refreshed snapshots 2026-09-27 06:41:39 +00:00
Thomas Durieux 0002e8d1a1 fix: serialize repository refreshes and recover stale downloads 2026-09-27 06:34:05 +00:00
tdurieux 0e7a8899b5 fix: prevent large notebook download timeouts 2026-09-23 11:27:17 +08:00
Thomas Durieux 2e6b79b30d fix: address streamer review edge cases 2026-09-14 15:23:13 +00:00
Thomas Durieux c9c729c2c5 fix: restore public file streaming and contain ZIP failures 2026-09-14 15:06:29 +00:00
tdurieux 2d153dd353 fix: read public repositories without scoped App token minting 2026-09-14 16:50:18 +02:00
Thomas Durieux 8abdaf7119 Merge pull request #828 from tdurieux/fix/public-repository-app-access
fix: allow GitHub App access to public repositories without installation
2026-09-13 19:06:25 +02:00
Thomas Durieux dc026e2ee8 fix: revalidate scoped public tokens throughout repository reads 2026-09-13 15:16:45 +00:00
Thomas Durieux 5e5cc6e0c2 fix: renew public grants and verify source identity and visibility 2026-09-13 15:05:43 +00:00
Thomas Durieux 8efb9bd506 fix: allow GitHub App access to public repositories without installation 2026-09-13 08:42:55 +00:00
Thomas Durieux 7cdf0f5e8d fix: distinguish repository publication and settings dates 2026-09-13 08:42:54 +00:00
Thomas Durieux 4e52f42009 Merge pull request #814 from tdurieux/feat/github-app-dual-auth
Add read-only GitHub App access alongside OAuth
2026-09-10 06:27:36 -07:00
tdurieux 19f819949e perf: avoid anonymization timeouts for absent literal terms 2026-09-10 14:49:01 +02:00
Thomas Durieux a60ad6a921 feat: add read-only GitHub App access alongside OAuth 2026-09-09 22:08:12 +02:00
Thomas Durieux a05f42ad9f Merge pull request #801 from tdurieux/t3code/archive-legacy-spaced-ids
fix: archive legacy repository IDs containing spaces
2026-09-08 02:46:38 -10:00
tdurieux c7eed1a6b4 perf: parallelize credential migration owner analysis 2026-09-08 14:28:46 +02:00
tdurieux 0c808a8675 fix: recover conflicting credentials by verifying the recorded owner 2026-09-08 14:21:16 +02:00
tdurieux 90f14b19b7 fix: archive legacy repository IDs containing spaces 2026-09-08 14:08:48 +02:00
Thomas Durieux 530e388e46 fix: archive ownerless repositories without assigning ownership (#800) 2026-09-08 14:03:09 +02:00
Thomas Durieux 8de4a536cc fix: recover legacy repository owners through GitHub (#799) 2026-09-08 13:39:39 +02:00
Thomas Durieux 3fb376dd53 fix: encrypt stored GitHub credentials (#797) 2026-09-08 13:04:28 +02:00
tdurieux b070d7e281 fix: evaluate document creation timestamps per instance 2026-09-06 09:19:33 +02:00
tdurieux f312105ec3 fix: fetch private pull request diffs through authenticated API 2026-09-06 09:19:28 +02:00
tdurieux 4aecc1d92f fix: fetch complete content for truncated Gist files 2026-09-06 09:19:23 +02:00
tdurieux 38a09188ad fix: expire conference repositories before completing expiration 2026-09-06 09:18:42 +02:00
tdurieux 6777537d44 fix: restrict S3 deletion to exact keys and descendants 2026-09-06 09:18:37 +02:00
tdurieux 5e84bfaa5b fix: surface partial S3 object deletion failures 2026-09-06 09:18:32 +02:00
tdurieux 4643e9f838 fix: preserve and anonymize truncated folder warnings 2026-09-06 09:18:28 +02:00
tdurieux 8d2921b406 fix: respect backpressure when resolving Git LFS pointers 2026-09-06 09:18:24 +02:00
tdurieux c14a548cb9 fix: enforce content policies using original file types 2026-09-06 09:17:42 +02:00
tdurieux 11eb160d2b fix: prevent stale downloads from reviving removed repositories 2026-09-06 09:17:33 +02:00
tdurieux 2af66cd1b6 fix: authorize coauthors by stable GitHub identity 2026-09-06 09:17:01 +02:00
tdurieux 8ec656c201 fix: authorize access before returning cached GitHub metadata 2026-09-06 09:16:57 +02:00
tdurieux 50d606d0c1 fix: bound regex execution during anonymization 2026-09-06 09:16:48 +02:00
tdurieux b3f86513dd fix: prevent anonymization leaks across stream boundaries 2026-09-06 09:16:44 +02:00
tdurieux 900516f225 fix: omit source content length for rewritten file responses 2026-09-06 09:16:23 +02:00
Thomas Durieux 35c3b1804b Fix repository cleanup after removal and expiration (#782) 2026-08-20 12:52:19 +02:00
Thomas Durieux 7aa9557905 perf: cut MongoDB scans in repository workflows (#780)
* Add repository name index

* Optimize repository file counts

* Optimize MongoDB query workloads
2026-08-20 12:29:26 +02:00
Thomas Durieux aab6ccf7fc fix: harden anonymization privacy paths (#754) 2026-07-22 04:28:01 +02:00
tdurieux 1c1993f972 fix: try to fix gist 2026-07-02 14:53:26 +03:00
Thomas Durieux 839582c657 Fix .bat anonymization, truncated-tree misses, submodule warning, account deletion (#742)
* fix: anonymize Windows batch scripts (#735)

mime-types maps .bat to application/x-msdownload, the same MIME type as
.exe/.dll, so batch scripts were classified as binary and streamed
through without any anonymization. Special-case .bat/.cmd as text before
the MIME lookup, keeping .exe/.dll binary.

* fix: recover files missing from truncated tree listings (#738)

GitHub truncates tree listings of very large repositories. Folders whose
listing was truncated are recorded in truncatedFolders, but files that
fell outside the listing never reached the database, so requesting them
returned 404 file_not_found even though they exist on GitHub — and a
force refresh could not help.

When a file lookup misses and its directory is under a truncated folder,
fetch the file metadata directly from GitHub's contents API (object
media type, so it works past the 1MB inline limit), cache it in the
database, and serve it normally.

* feat: warn when a repository uses git submodules (#737)

GitHub archives and tree listings never include submodule contents, so
submodules end up as empty folders in the anonymized repository, which
surprises users. Detect a root .gitmodules file and show a warning
banner in the explorer explaining that submodule contents are not
included.

* feat: allow users to delete their account (#741)

Add DELETE /api/user: removes all anonymized repositories, gists, and
pull requests owned by the user, best-effort revokes the GitHub OAuth
grant, and scrubs personal data (username, emails, tokens, GitHub id,
photo) from the user record. The record itself is kept with a
placeholder username so removed repoIds stay reserved and owner
references remain resolvable.

The settings page gains an Account section with a confirmed delete
button.

* fix: add missing error translations for token_expired and job_is_active

The error-code coverage test failed because both backend codes had no
frontend translation.
2026-07-02 13:35:48 +02:00
Thomas DurieuxandClaude Opus 4.8 e4ffd74068 Security hardening + gist UI fixes (#731)
* security: harden against XSS, ReDoS, path traversal, and injection

Defensive fixes across the server, storage, and viewer:

- XSS (CWE-79): sanitise rendered notebooks with DOMPurify, escape file
  names interpolated into AngularJS expressions (escapeNgString), set
  Mermaid securityLevel to 'strict', and stop urlRel2abs from returning
  javascript:/vbscript:/data:text/html URLs.
- Path traversal / zip-slip (CWE-22/23/24): validate URL-derived path
  components before they reach the storage layer (file/webview routes +
  StorageBase.assertSafePath) and sanitise zip entry names on extract for
  both the filesystem and S3 backends.
- ReDoS (CWE-1333): escape anonymization terms with catastrophic
  backtracking shapes to literals instead of compiling them as regexes.
- Secret hardening (CWE-798): require SESSION_SECRET / OAuth creds / DB
  password in production, random dev SESSION_SECRET fallback.
- Rate-limit spoofing (CWE-290): derive request.ip via trust-proxy hop
  count instead of the client-settable cf-connecting-ip header.
- NoSQL injection (CWE-943): allow only plain field paths as admin sort keys.
- Reject malformed streamer requests missing required string fields.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(ui): make gists reachable/visible and clarify the ZIP button

- Gist & PR routes now accept a trailing slash (/gist/:id/:path*?), so the
  dashboard links (which end in "/") resolve to the gist/PR page instead of
  falling through to the 404 route (#725).
- Gist viewer picks the default tab after content loads, defaulting to
  "files" when files exist; previously the ng-init ran before the async
  load and a files-only gist rendered blank under the hidden comments tab.
- Explorer toolbar: relabel ZIP to "Full repo ZIP" with a tooltip, and add
  tooltips to Raw/Download clarifying they apply to the current file (#721).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: report SAML-enforced orgs clearly instead of "token expired"

When a repo's organization enforces SAML SSO, GitHub returns a 403 whose
message differs from the OAuth-App-restriction case. That 403 fell through
to the generic handler and surfaced as "token_expired", pushing users to
re-login when the real fix is authorizing their token for the org. Detect
the "SAML enforcement" message and raise a dedicated, actionable error
instead (#379, #550).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* security: catch nested quantified groups in ReDoS guard and backslash path traversal

- hasCatastrophicBacktracking now scans across nested parens ([\s\S]*?)
  so shapes like ((a+))+ are detected; comment reframed as a heuristic
  backstop rather than a proof.
- file route path-traversal check now rejects backslash separators and a
  leading backslash, covering Windows-style "..\" payloads (CWE-22/25).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(dev): track dev-proxy script, ignore .DS_Store and .claude/

scripts/dev-proxy.js is referenced by the "dev:ui" npm script but was
never committed, breaking the command on a fresh clone. Add it and
ignore local-only macOS/Claude Code files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 13:50:55 +02:00