Commit Graph

61 Commits

Author SHA1 Message Date
tdurieux b8cfe293ea Fix BullMQ "Custom Id cannot be integers" error by prefixing jobId 2026-05-07 05:53:26 +03:00
tdurieux 8fc7ac5175 Add user ban/activate feature
Add admin endpoints to ban and activate users, block banned users
from all auth flows (OAuth, token login, bearer auth), and invalidate
existing sessions on next request. Includes frontend translation and
user detail page ban/activate buttons.
2026-05-07 05:41:12 +03:00
tdurieux d9104c2ec2 Update commit on branch refresh and validate commit exists on save
Refresh button now always updates the commit to the latest SHA instead
of preserving the stale one in edit mode. Both create and update routes
verify the commit still exists on GitHub before persisting.
2026-05-06 21:14:53 +03:00
tdurieux bd8656206a fix persistance bugs 2026-05-06 20:00:59 +03:00
tdurieux 67cb2538b1 fix old github download repos 2026-05-06 19:37:16 +03:00
tdurieux da78708b7b Improve error handling 2026-05-06 18:43:36 +03:00
tdurieux 6bad6c2f09 fix bugs and report better errors 2026-05-06 17:26:47 +03:00
tdurieux 804bbffb7a Improve error handling 2026-05-06 17:03:19 +03:00
tdurieux 48e782946a Improve error handling 2026-05-06 16:56:07 +03:00
tdurieux dcb524c8c1 Improve error handling 2026-05-06 16:45:22 +03:00
tdurieux 3613c895c8 improve logging 2026-05-06 16:31:10 +03:00
tdurieux 873c910dd3 Improve error dashboard 2026-05-06 16:12:37 +03:00
tdurieux 06a098fba7 perf(deploy): faster builds and zero-downtime streamer rollover
- Multi-stage Dockerfile with BuildKit npm cache mounts and a separate
  prod-deps stage so source edits don't reinstall or prune.
- Tighter .dockerignore to shrink build context.
- Healthchecks: add start_period and tighten interval/retries so
  containers report healthy as soon as the process is actually ready
  instead of after a full polling interval.
- Move recoverStuckPreparing() off the startup critical path; the
  recovery sweep now runs in the background after app.listen.
- depends_on uses condition: service_healthy and the obsolete
  compose 'version' key is gone.
- New scripts/build.sh + scripts/deploy.sh: deploy.sh builds, exits
  early if the image is unchanged, runs a blue/green streamer swap
  (scale to 2N, wait healthy in parallel, drop olds), then recreates
  the API with --no-deps to avoid compose's depends_on re-poll.
2026-05-06 13:38:19 +03:00
tdurieux c2d43164d0 error logging improvement, regex fix 2026-05-06 11:16:12 +03:00
dependabot[bot] e34f45522f chore(deps): bump uuid and bullmq (#691)
* chore(deps): bump uuid and bullmq

Removes [uuid](https://github.com/uuidjs/uuid). It's no longer used after updating ancestor dependency [bullmq](https://github.com/taskforcesh/bullmq). These dependencies need to be updated together.


Removes `uuid`

Updates `bullmq` from 2.4.0 to 5.76.5
- [Release notes](https://github.com/taskforcesh/bullmq/releases)
- [Commits](https://github.com/taskforcesh/bullmq/compare/v2.4.0...v5.76.5)

---
updated-dependencies:
- dependency-name: bullmq
  dependency-version: 5.76.5
  dependency-type: direct:production
- dependency-name: uuid
  dependency-version: 
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix(bullmq): adapt isRunning + getJobs typing for v5 API

Worker.isRunning became a method (was a property in v2), and
Queue.getJobs now requires a mutable JobType[] (was string[]).

* clean up

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: tdurieux <durieuxthomas@hotmail.com>
Co-authored-by: Thomas Durieux <5577568+tdurieux@users.noreply.github.com>
2026-05-06 08:40:12 +02:00
tdurieux 7e0f900c7a fix: resolve eslint unused-var and useless-assignment warnings
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-06 08:22:45 +03:00
tdurieux c903627da0 fix build 2026-05-05 10:35:08 +03:00
tdurieux f8c91ca0af multiple fixes 2026-05-05 10:32:31 +03:00
tdurieux 5b72b630c4 fix: silent-truncation, token-refresh, and content-type bugs across hot paths
Follow-up review pass after the cache fixes turned up several bugs in
the same family — silent failures that look like success to the client,
plus content-correctness issues in the ZIP and per-file delivery paths.

- zipStream: stop calling archive.finalize() on upstream/parser errors.
  That produced a valid-looking ZIP (200 OK, archive opens) silently
  missing entries — same class as #694, but worse because the user has
  no signal anything went wrong. Destroy the response on failure
  instead so the client sees a connection drop.
- zipStream: apply per-repo image/pdf gates inside the entry handler.
  The single-file /file/... endpoint refuses to serve those types
  via AnonymizedFile.isFileSupported when image=false / pdf=false, but
  the ZIP shipped them anyway — privacy-relevant for maintainers who
  toggle image=false to suppress identifying screenshots. Threaded
  contentOptions through both ZIP entry points (direct and streamer).
- GitHubUtils.getToken: validate the OAuth token-refresh response
  before persisting. On a non-2xx response or a body without a string
  token, we used to overwrite the stored token with `undefined`, which
  then propagated as `Authorization: token undefined` to every API
  call — 401 even on public repos, with the config.GITHUB_TOKEN
  fallback unreachable because the field was no longer falsy.
- AnonymizedFile.send (streamer branch): forward Content-Type from the
  upstream streamer response. got.stream(...).pipe(res) carries body
  bytes only, so the parent response had no Content-Type and browsers
  guessed (text rendered as download, etc.). Also resolve on
  res.on("finish") in addition to "close" — keep-alive sockets stay
  open long after the response is delivered, delaying countView().
- Repository.updateIfNeeded: persist a renamed source.repositoryName
  even when the commit hasn't changed. Previously the new value lived
  in memory only and was overwritten on the next reload, so the
  rename detection ran every request.
- Repository.anonymize: stop materialising a dummy {path:"",name:""}
  FileModel for empty repos. That row collided with the special case
  in AnonymizedFile.getFileInfo and surfaced in unfiltered listings.
- streamer/route POST /: reject filePath segments containing ".." or
  empty parts. Defence in depth — the parent server validates against
  FileModel before calling, but the streamer joins filePath straight
  into the storage path, so any future caller forwarding an
  unvalidated path could traverse out of the repo root.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 09:19:05 +03:00
tdurieux f0bc53f093 feat: gist & co-authors 2026-05-04 13:10:44 +02:00
tdurieux f91db91cee wip 2026-05-04 11:30:42 +02:00
tdurieux 117406f2ce refactor: route anonymize preview through the backend
The form's live README/PR preview was running its own copy of
ContentAnonimizer in the browser. The two implementations had been
drifting — recent fixes for word boundaries (#175/#249), accent
matching (#280), custom replacements (#285), and the diacritic-stripped
variants only landed on the server. Reviewers saw one anonymization;
authors composing the form saw another.

Add POST /api/anonymize-preview that takes a snippet (or a batch) plus
the user's options and runs them through the same ContentAnonimizer
the file route uses. Replace the client-side anonymizeReadme() body
with a debounced call to that endpoint. The PR view's
anonymizePrContent() runs as a synchronous template expression, so it
now reads from a {original -> anonymized} cache that's refreshed in
the background whenever the PR details, terms, or options change.

Single-flight + debounce keep the form responsive; an in-flight
request is dropped on the next change.
2026-05-04 11:05:50 +02:00
tdurieux 3f095f0734 always use githubStream 2026-05-04 10:47:58 +02:00
tdurieux 88fe8570fd fix: include file path in cache ETag
Without the path, two different files in the same repo (same sha, same
anonymization options) shared an ETag. If a browser ever sent the cached
ETag for one file while requesting another, the server would have
returned 304 against the wrong cache entry. Fold the path into the
ETag so each file has its own fingerprint.

Follow-up to b3c1030 (#439).
2026-05-03 21:19:39 +02:00
tdurieux b3c1030e5c fix: revalidate cached files when anonymization options change
Files were being served with Cache-Control: max-age=18144000 (210 days)
keyed only on the upstream ?v=<sha>. Editing the term list left the
same URL serving stale anonymized bytes — visible to users in regular
tabs but not in incognito. The previous fix-by-incognito recipe in #439
is exactly this.

Switch to ETag-based revalidation that fingerprints both the upstream
sha and the saved anonymization options, with Cache-Control:
no-cache, must-revalidate. Browsers now revalidate on every request and
get a 304 when nothing has changed, or fresh content as soon as terms,
image/link/etc. options are updated.

Fixes #439.
2026-05-03 20:31:32 +02:00
tdurieux a5f66d6844 multiple fixes 2026-05-03 15:30:54 +02:00
tdurieux 88f826aab4 update design 2026-04-24 14:55:18 +02:00
Thomas Durieux 188066e91d Fix 9 bugs and add 103 tests for core anonymization, config, and routing (#669) 2026-04-15 09:41:00 +02:00
Thomas Durieux 8198a4b44a Standardize error responses with consistent format and human-readable messages (#667) 2026-04-15 09:27:08 +02:00
Thomas Durieux f4209110c7 Fix all 93 ESLint issues (3 errors, 90 warnings) (#666) 2026-04-15 09:04:22 +02:00
Thomas Durieux 95324fd09f Replace isomorphic-dompurify with sanitize-html for Node 21 compat (#663) 2026-04-15 04:57:21 +02:00
Thomas Durieux c6d6806d7a fix: prevent admin status reset on login (#659) 2026-04-15 04:34:36 +02:00
Thomas Durieux f3641c8ce3 Set up CI with ESLint linter and Mocha test runner (#661) 2026-04-15 04:34:03 +02:00
Thomas Durieux 812f8b6314 Sanitize markdown HTML output with DOMPurify to prevent XSS (#658) 2026-04-15 04:22:38 +02:00
Thomas Durieux b2d77faa6c try to fix repo access 2025-04-01 22:27:41 +02:00
tdurieux 532c094388 fix: improve token management 2024-06-18 12:00:53 +02:00
tdurieux dcf483ea03 feat: improve download anonymized repository 2024-05-06 11:52:32 +02:00
tdurieux ca04339529 feat: list files in folder in webview 2024-05-02 11:49:00 +01:00
tdurieux ed11e9db36 fix: undefined in path.join 2024-05-02 11:09:20 +01:00
tdurieux 3536f78a99 fix(#283): Fix webview for web in folder 2024-04-30 11:28:32 +01:00
tdurieux 72c8f80bce fix: fix file path in webview 2024-04-28 09:38:49 +01:00
tdurieux 17abc47d08 fix: fix webview on root repo 2024-04-28 08:08:39 +01:00
tdurieux 17cb1f294f chore: remove all reference to originalFiles 2024-04-27 18:04:51 +01:00
tdurieux 378942a28e fix: fix file list collection 2024-04-27 17:05:39 +01:00
tdurieux 2a145730b7 Improve log and GH token validation 2024-04-27 16:19:33 +01:00
tdurieux 710f7328e7 feat: flatten file tree for better performance 2024-04-26 10:32:09 +01:00
tdurieux daf3276f7f fix: fix queue admin 2024-04-12 09:56:39 +01:00
tdurieux 03835e86ab fix: handle error in queue admin 2024-04-11 17:48:41 +01:00
tdurieux 79c6b603b4 fix: handle error in queue admin 2024-04-11 17:22:08 +01:00
tdurieux 6b9574add3 fix: improve repository rename 2024-04-11 17:13:01 +01:00