* fix: anonymize Windows batch scripts (#735)
mime-types maps .bat to application/x-msdownload, the same MIME type as
.exe/.dll, so batch scripts were classified as binary and streamed
through without any anonymization. Special-case .bat/.cmd as text before
the MIME lookup, keeping .exe/.dll binary.
* fix: recover files missing from truncated tree listings (#738)
GitHub truncates tree listings of very large repositories. Folders whose
listing was truncated are recorded in truncatedFolders, but files that
fell outside the listing never reached the database, so requesting them
returned 404 file_not_found even though they exist on GitHub — and a
force refresh could not help.
When a file lookup misses and its directory is under a truncated folder,
fetch the file metadata directly from GitHub's contents API (object
media type, so it works past the 1MB inline limit), cache it in the
database, and serve it normally.
* feat: warn when a repository uses git submodules (#737)
GitHub archives and tree listings never include submodule contents, so
submodules end up as empty folders in the anonymized repository, which
surprises users. Detect a root .gitmodules file and show a warning
banner in the explorer explaining that submodule contents are not
included.
* feat: allow users to delete their account (#741)
Add DELETE /api/user: removes all anonymized repositories, gists, and
pull requests owned by the user, best-effort revokes the GitHub OAuth
grant, and scrubs personal data (username, emails, tokens, GitHub id,
photo) from the user record. The record itself is kept with a
placeholder username so removed repoIds stay reserved and owner
references remain resolvable.
The settings page gains an Account section with a confirmed delete
button.
* fix: add missing error translations for token_expired and job_is_active
The error-code coverage test failed because both backend codes had no
frontend translation.
- AnonymizedFile.anonymizedContent(): propagate content errors to the
anonymizer so callers see the failure instead of hanging.
- AnonymizedFile.send() local path: add error handler on the anonymizer
transform between content and response pipes.
- S3.send(): handle errors on the S3 body stream to avoid unhandled
emits crashing the process.
- S3.archive() / FileSystem.archive(): propagate read-stream errors
to the file transformer so archiver sees the failure.
- Add frontend translations for new error codes.
All warn/error log calls now use field names the dashboard's decorate()
function recognizes: `code` for the error code pill, `httpStatus` for the
status badge and severity bucket, `url` for the sidebar link, and
`repoId` for the repository link.
Key changes:
- Streamer errors surface code, httpStatus, url, and nested err in Raw tab
- Nested `{ err: serializeError(e) }` replaced with spread pattern so
error fields (name, message, status) appear at the top level
- Raw Error objects in catch blocks now go through serializeError()
- Rate limit, token, and PR 404 warnings include code + httpStatus
- Dashboard stack walker traverses both `cause` and `err` chains
- Dashboard Raw tab renders repoId, filePath, upstream*, err, and cause
- trimRawArg recursively trims stacks in nested err/cause chains
- clampPayload strips heavy nested fields before falling back to
truncated placeholder, preserving flat diagnostic fields
Follow-up review pass after the cache fixes turned up several bugs in
the same family — silent failures that look like success to the client,
plus content-correctness issues in the ZIP and per-file delivery paths.
- zipStream: stop calling archive.finalize() on upstream/parser errors.
That produced a valid-looking ZIP (200 OK, archive opens) silently
missing entries — same class as #694, but worse because the user has
no signal anything went wrong. Destroy the response on failure
instead so the client sees a connection drop.
- zipStream: apply per-repo image/pdf gates inside the entry handler.
The single-file /file/... endpoint refuses to serve those types
via AnonymizedFile.isFileSupported when image=false / pdf=false, but
the ZIP shipped them anyway — privacy-relevant for maintainers who
toggle image=false to suppress identifying screenshots. Threaded
contentOptions through both ZIP entry points (direct and streamer).
- GitHubUtils.getToken: validate the OAuth token-refresh response
before persisting. On a non-2xx response or a body without a string
token, we used to overwrite the stored token with `undefined`, which
then propagated as `Authorization: token undefined` to every API
call — 401 even on public repos, with the config.GITHUB_TOKEN
fallback unreachable because the field was no longer falsy.
- AnonymizedFile.send (streamer branch): forward Content-Type from the
upstream streamer response. got.stream(...).pipe(res) carries body
bytes only, so the parent response had no Content-Type and browsers
guessed (text rendered as download, etc.). Also resolve on
res.on("finish") in addition to "close" — keep-alive sockets stay
open long after the response is delivered, delaying countView().
- Repository.updateIfNeeded: persist a renamed source.repositoryName
even when the commit hasn't changed. Previously the new value lived
in memory only and was overwritten on the next reload, so the
rename detection ran every request.
- Repository.anonymize: stop materialising a dummy {path:"",name:""}
FileModel for empty repos. That row collided with the special case
in AnonymizedFile.getFileInfo and surfaced in unfiltered listings.
- streamer/route POST /: reject filePath segments containing ".." or
empty parts. Defence in depth — the parent server validates against
FileModel before calling, but the streamer joins filePath straight
into the storage path, so any future caller forwarding an
unvalidated path could traverse out of the repo root.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A failed/interrupted GitHub fetch could leave a 0-byte or truncated
file in the local cache. Subsequent reads happily streamed the empty
content as the file's body — visible to users as an "Empty file" with
HTTP 200. Reproduced on artifact-70B6/Lethe/configs.py (#694).
- FileSystem.write: stream into a sibling .tmp and rename into place
only on finish. Stream errors discard the tmp and leave any prior
cached file untouched. Drop the utf-8 encoding that was silently
corrupting binary blobs.
- GitHubStream.getFileContentCache: accept an expected size and treat
cached.size < expected as a poisoned cache (truncated fetch) → rm
and re-fetch. cached.size >= expected is accepted, which keeps
Git LFS-resolved files (whose FileModel.size is the pointer size)
working.
- AnonymizedFile: expose size() and pass it through to the streamer
alongside sha so the cache check has the upstream size.
Existing poisoned entries self-heal on next access.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The server set Accept-Ranges: none on every file response. For text we
anonymize on the fly so byte ranges aren't meaningful, but binary
entries pass through unchanged — and the explicit "none" header makes
some browsers refuse to play <video>/<audio> elements that would
otherwise fall back to a full download. Newly uploaded MP4s under the
inline-preview threshold rendered as a blank progress bar (#538).
Only set Accept-Ranges: none for text entries; let binary entries omit
it so the standard fallback kicks in.
Fixes#538.
Thomas Durieux