244 Commits

Author SHA1 Message Date
tdurieux 9b6c8dbe62 improve admin overview 2026-05-11 12:23:24 +03:00
tdurieux afd9f36cfb improve admin overview 2026-05-11 12:10:17 +03:00
tdurieux 03e18fd572 repo change + daily stat improvements 2026-05-11 12:10:17 +03:00
tdurieux 369fd8edb2 redis cache 2026-05-07 15:55:28 +03:00
tdurieux b37a814f3a improve queue 2026-05-07 14:58:36 +03:00
tdurieux f817a29a4b loading improvements 2026-05-07 08:30:31 +03:00
tdurieux 2de08c3df3 Add missing error handlers on stream pipelines
- AnonymizedFile.anonymizedContent(): propagate content errors to the
  anonymizer so callers see the failure instead of hanging.
- AnonymizedFile.send() local path: add error handler on the anonymizer
  transform between content and response pipes.
- S3.send(): handle errors on the S3 body stream to avoid unhandled
  emits crashing the process.
- S3.archive() / FileSystem.archive(): propagate read-stream errors
  to the file transformer so archiver sees the failure.
- Add frontend translations for new error codes.
2026-05-07 07:47:29 +03:00
tdurieux fbbc694747 improve styling 2026-05-07 07:34:30 +03:00
tdurieux 9403f15ac3 Align error logging with admin dashboard field conventions
All warn/error log calls now use field names the dashboard's decorate()
function recognizes: `code` for the error code pill, `httpStatus` for the
status badge and severity bucket, `url` for the sidebar link, and
`repoId` for the repository link.

Key changes:
- Streamer errors surface code, httpStatus, url, and nested err in Raw tab
- Nested `{ err: serializeError(e) }` replaced with spread pattern so
  error fields (name, message, status) appear at the top level
- Raw Error objects in catch blocks now go through serializeError()
- Rate limit, token, and PR 404 warnings include code + httpStatus
- Dashboard stack walker traverses both `cause` and `err` chains
- Dashboard Raw tab renders repoId, filePath, upstream*, err, and cause
- trimRawArg recursively trims stacks in nested err/cause chains
- clampPayload strips heavy nested fields before falling back to
  truncated placeholder, preserving flat diagnostic fields
2026-05-07 05:54:18 +03:00
tdurieux 8fc7ac5175 Add user ban/activate feature
Add admin endpoints to ban and activate users, block banned users
from all auth flows (OAuth, token login, bearer auth), and invalidate
existing sessions on next request. Includes frontend translation and
user detail page ban/activate buttons.
2026-05-07 05:41:12 +03:00
tdurieux 48256e743c Style Ko-fi widgets to match light/dark theme
Floating button now initializes with theme-aware colors and updates
on toggle. Status page iframe uses a tuned CSS filter in dark mode
to blend with the warm palette.
2026-05-06 21:38:52 +03:00
tdurieux dfa5a2e2fd Fix repo link on admin errors page to point to repo view 2026-05-06 21:38:31 +03:00
tdurieux 1204eaffa9 Fix admin repository links and remove buttons
- Use $location.search() instead of window.location.search for URL
  params so cross-page links (owner, conference, search filters) work
  with AngularJS client-side navigation
- Add missing removeRepository() in both repos and user detail controllers
- Fix removeCache() spurious $scope.$apply() that caused digest errors
- Add confirmation prompts and list refresh after remove/cache operations
2026-05-06 21:27:57 +03:00
tdurieux d9104c2ec2 Update commit on branch refresh and validate commit exists on save
Refresh button now always updates the commit to the latest SHA instead
of preserving the stale one in edit mode. Both create and update routes
verify the commit still exists on GitHub before persisting.
2026-05-06 21:14:53 +03:00
tdurieux d1d6257512 fix audio url 2026-05-06 20:37:50 +03:00
tdurieux 2f6ec41a2c block indexing webpage as well 2026-05-06 20:36:04 +03:00
tdurieux bd8656206a fix persistance bugs 2026-05-06 20:00:59 +03:00
tdurieux 67cb2538b1 fix old github download repos 2026-05-06 19:37:16 +03:00
tdurieux aae6eae6eb handle rate limit 2026-05-06 17:50:01 +03:00
tdurieux c1e18f82a9 Improve error handling 2026-05-06 17:39:43 +03:00
tdurieux dcb524c8c1 Improve error handling 2026-05-06 16:45:22 +03:00
tdurieux 3613c895c8 improve logging 2026-05-06 16:31:10 +03:00
tdurieux 873c910dd3 Improve error dashboard 2026-05-06 16:12:37 +03:00
tdurieux 1254c56223 fix: dedupe folder contents on re-fetch (#701)
getFiles blindly appended fetched entries to $scope.files, so
re-opening a folder duplicated its children in the tree. Drop any
existing entries at the requested path before appending.
2026-05-06 11:16:12 +03:00
tdurieux c2d43164d0 error logging improvement, regex fix 2026-05-06 11:16:12 +03:00
tdurieux 18ce39e019 make repo error messages actionable (#550, #379, #519)
Rewrite repo_access_limited, repo_not_found, repo_empty, and
repo_not_accessible to point users at concrete next steps instead
of stating only that something failed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-06 07:50:41 +03:00
tdurieux 27d6b56da7 fix mulitple bugs 2026-05-05 12:34:03 +03:00
tdurieux 7384638993 dark theme support 2026-05-05 10:40:12 +03:00
tdurieux f8c91ca0af multiple fixes 2026-05-05 10:32:31 +03:00
tdurieux 5b72b630c4 fix: silent-truncation, token-refresh, and content-type bugs across hot paths
Follow-up review pass after the cache fixes turned up several bugs in
the same family — silent failures that look like success to the client,
plus content-correctness issues in the ZIP and per-file delivery paths.

- zipStream: stop calling archive.finalize() on upstream/parser errors.
  That produced a valid-looking ZIP (200 OK, archive opens) silently
  missing entries — same class as #694, but worse because the user has
  no signal anything went wrong. Destroy the response on failure
  instead so the client sees a connection drop.
- zipStream: apply per-repo image/pdf gates inside the entry handler.
  The single-file /file/... endpoint refuses to serve those types
  via AnonymizedFile.isFileSupported when image=false / pdf=false, but
  the ZIP shipped them anyway — privacy-relevant for maintainers who
  toggle image=false to suppress identifying screenshots. Threaded
  contentOptions through both ZIP entry points (direct and streamer).
- GitHubUtils.getToken: validate the OAuth token-refresh response
  before persisting. On a non-2xx response or a body without a string
  token, we used to overwrite the stored token with `undefined`, which
  then propagated as `Authorization: token undefined` to every API
  call — 401 even on public repos, with the config.GITHUB_TOKEN
  fallback unreachable because the field was no longer falsy.
- AnonymizedFile.send (streamer branch): forward Content-Type from the
  upstream streamer response. got.stream(...).pipe(res) carries body
  bytes only, so the parent response had no Content-Type and browsers
  guessed (text rendered as download, etc.). Also resolve on
  res.on("finish") in addition to "close" — keep-alive sockets stay
  open long after the response is delivered, delaying countView().
- Repository.updateIfNeeded: persist a renamed source.repositoryName
  even when the commit hasn't changed. Previously the new value lived
  in memory only and was overwritten on the next reload, so the
  rename detection ran every request.
- Repository.anonymize: stop materialising a dummy {path:"",name:""}
  FileModel for empty repos. That row collided with the special case
  in AnonymizedFile.getFileInfo and surfaced in unfiltered listings.
- streamer/route POST /: reject filePath segments containing ".." or
  empty parts. Defence in depth — the parent server validates against
  FileModel before calling, but the streamer joins filePath straight
  into the storage path, so any future caller forwarding an
  unvalidated path could traverse out of the repo root.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 09:19:05 +03:00
tdurieux ab266f2188 fix(i18n): add missing error translations for gist, auth, and co-author flows
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 00:50:40 +03:00
tdurieux dee406e2ea update design 2026-05-05 00:36:42 +03:00
tdurieux f0bc53f093 feat: gist & co-authors 2026-05-04 13:10:44 +02:00
tdurieux 7ace730960 fix: fall back to default branch when resolving relative image URLs
The Anonymize form's preview built the readme baseUrl as
"https://github.com/<owner>/<repo>/raw/<source.branch>/". When the
form rendered before the branch field had populated (initial load,
or while waiting on getBranches), the URL became ".../raw//" and
the browser collapsed the empty segment, fetching ".../raw/<file>"
instead of ".../raw/<branch>/<file>". Relative <img src="./X">
references then 404'd against a path with no branch — exactly the
"branch missing" pattern in #407.

Fall back to details.defaultBranch (then "main") so the base URL is
always well-formed.

Fixes #407.
2026-05-04 12:01:56 +02:00
tdurieux 62a2c1cd5c fix: force-fetch repo details so newly enabled features show up
The Anonymize form used the cached RepositoryModel for hasPage,
defaultBranch, etc. — so enabling GitHub Pages (or changing the
default branch) on the source after first cache wouldn't reflect in
the UI, leaving the GitHub Pages checkbox grayed out.

Pass force=1 when loading the form's repo details so the backend
re-queries the GitHub API once. The cost is a single GET /repos/...
call per form load.

Fixes #364.
2026-05-04 11:58:44 +02:00
tdurieux ef78e8ff3c feat: preserve raw bytes when anonymization is a no-op
When the anonymizer doesn't change a slice's text, the streamer used
to push Buffer.from(out, "utf8") — which loses any invalid-UTF-8 bytes
in the input (replaced by U+FFFD via StringDecoder). Files
mistakenly classified as text (binary blobs without a known extension,
text with stray non-UTF-8 bytes, BOMs) came out corrupted even though
nothing in the term list matched.

Track the raw chunk bytes alongside the decoded `pending`. On flush —
where we have every byte buffered — emit the original buffer directly
when the output equals the input, so a pure passthrough is bit-exact.
In the streaming OVERLAP path, do the same when the decode for that
slice round-trips losslessly; fall back to encoded output otherwise
(unchanged from before for that case).

Also add the "missing_content" locale entry for the
/api/anonymize-preview route.
2026-05-04 11:52:03 +02:00
tdurieux f91db91cee wip 2026-05-04 11:30:42 +02:00
tdurieux 96d9f3cee6 fix: show explorer sidebar 'Updated' timestamp on desktop
The .leftCol-foot was hidden by default and only revealed inside the
mobile/tablet media query, so the last-update line never appeared on
wider viewports.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-04 11:28:10 +02:00
tdurieux 03826fe58b fix: don't break folder lazy-load and file navigation
Two regressions stacked from the recent tree work:

1. expandAllFolders (#496) was marking every folder open, including
   folders whose children weren't fetched yet. The directive then
   rendered an empty <ul> after each <a>, and the openFolder handler's
   "no sibling means we need to load" check silently treated the empty
   <ul> as already-loaded — so clicking the folder toggled the class
   but the children never appeared.

   Skip folders with empty children when pre-expanding, and harden the
   click handler so an empty <ul> still triggers a fetch.

2. The $routeUpdate handler (#510 follow-up) became async and called
   $scope.$apply(updateContent) at the end. Inside an already-running
   digest cycle this no-ops or throws, leaving file navigation stuck.

   Run updateContent() synchronously like before, and kick off any
   missing parent-directory fetches in the background — getContent()
   already falls back to sha "0" when the metadata isn't loaded yet.
2026-05-04 11:25:32 +02:00
tdurieux 5c7eb23336 fix: don't crash and stay stuck on a subdir markdown link
Clicking a markdown link into a subdirectory's README threw
"Cannot read properties of undefined (reading 'sha')" and left the
viewer on Loading…. The route handler called updateContent() without
loading the new directory's file listing, so getSelectedFile() returned
undefined and getContent() then dereferenced fileInfo.sha.

Two fixes:
- getContent() falls back to sha "0" when fileInfo is undefined.
- The $routeUpdate handler walks the new path and loads any directory
  listings that aren't yet in $scope.files before rendering, so the
  selected file actually has its sha by the time we fetch.

Fixes #510.
2026-05-04 11:10:12 +02:00
tdurieux 117406f2ce refactor: route anonymize preview through the backend
The form's live README/PR preview was running its own copy of
ContentAnonimizer in the browser. The two implementations had been
drifting — recent fixes for word boundaries (#175/#249), accent
matching (#280), custom replacements (#285), and the diacritic-stripped
variants only landed on the server. Reviewers saw one anonymization;
authors composing the form saw another.

Add POST /api/anonymize-preview that takes a snippet (or a batch) plus
the user's options and runs them through the same ContentAnonimizer
the file route uses. Replace the client-side anonymizeReadme() body
with a debounced call to that endpoint. The PR view's
anonymizePrContent() runs as a synchronous template expression, so it
now reads from a {original -> anonymized} cache that's refreshed in
the background whenever the PR details, terms, or options change.

Single-flight + debounce keep the form responsive; an in-flight
request is dropped on the next change.
2026-05-04 11:05:50 +02:00
tdurieux c8fc561dac fix: regex characters in terms shouldn't block submission
Entering an IP address (e.g. 192.168.1.1) or any term with regex
metacharacters made the form invalid because the "regex characters
detected" hint was wired up via $setValidity('terms', 'regex', false).
The text in the UI labels it as a warning, but the form treated it as
an error and refused to save.

Track the warning as a plain $scope flag and show it via ng-show on
that flag, so the form stays valid (#430).
2026-05-04 10:58:17 +02:00
tdurieux 59d9805276 update titles 2026-05-04 09:33:44 +02:00
tdurieux 4bc83db416 feat: per-term anonymization output via term=>replacement
A term entered as "Anonymous=>ABC" now scrubs "Anonymous" to "ABC"
instead of "XXXX-N". Lets users keep anonymized identifiers valid in
source code (no hyphen) and align tokens between paper text and repo.
Indexing for default-mask terms is unchanged: a list of
"Alpha=>AAA", "Beta" still produces XXXX-2 for Beta.

Fixes #285.
2026-05-04 09:31:31 +02:00
tdurieux db2ac5307d fix admin 2026-05-03 22:29:01 +02:00
tdurieux 6096cb0744 fix: expand every folder by default in the explorer tree
The file tree opened collapsed, requiring the reviewer to click each
folder before they could see what was inside. Walk the tree on first
render and mark every folder open in $scope.opens. Folders the user
has explicitly toggled (a previous entry already exists in
$scope.opens) are left as-is, so collapsing still works.

Fixes #496.
2026-05-03 22:28:27 +02:00
tdurieux 1f966841ad docs: refresh home/dashboard/anonymize/explorer screenshots
Replace the outdated user-images.githubusercontent.com screenshot in
the README with a locally hosted image, and regenerate the in-app
screenshots to reflect the current UI.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 20:32:45 +02:00
tdurieux b316d18bd8 fix: clicking a gutter line updates the URL to #L<n>
The viewer already supported jumping to a line via #L42 in the URL but
never produced one — users had to type it manually. Wire guttermousedown
on the ACE editor to replaceState a #L<n> hash, with shift-click for a
range. Also reapply the highlight on hashchange so pasting a URL into
the address bar works without reload.

Fixes #392.
2026-05-03 20:25:41 +02:00
tdurieux d138d487f2 fix: don't drop a path segment when resolving "./" relative URLs
urlRel2abs() prepended an extra "." when it saw "./X", turning the
relative path into "../X" and silently moving up a directory. As a
result, raw HTML <img src="./imgs/run.png"> inside a README rendered
under /r/<repo>/<file> resolved to /r/<repo>/imgs/... instead of
/r/<repo>/<dir>/imgs/..., so the image 404'd. Markdown image syntax
went through marked-base-url and was unaffected.

Strip the leading "./" instead so the relative path concatenates
cleanly with baseUrl.

Fixes #346.
2026-05-03 20:01:33 +02:00
tdurieux 9feeab1055 fix: render heading IDs so anchor links resolve
marked v12 dropped the headerIds option, so headings rendered with no
id attributes and links like [Releases](#releases-and-contributing)
silently failed to scroll. Add a heading renderer that emits a
GitHub-style slug id, with a numeric suffix for duplicates within a
document.

Fixes #390.
2026-05-03 19:44:18 +02:00