chore(deps): bump js-yaml, knip and mocha

Bumps [js-yaml](https://github.com/nodeca/js-yaml) to 4.1.1 and updates ancestor dependencies [js-yaml](https://github.com/nodeca/js-yaml), [knip](https://github.com/webpro-nl/knip/tree/HEAD/packages/knip) and [mocha](https://github.com/mochajs/mocha). These dependencies need to be updated together.


Updates `js-yaml` from 4.1.0 to 4.1.1
- [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/4.1.0...4.1.1)

Updates `knip` from 5.6.1 to 5.69.1
- [Release notes](https://github.com/webpro-nl/knip/releases)
- [Changelog](https://github.com/webpro-nl/knip/blob/main/packages/knip/.release-it.json)
- [Commits](https://github.com/webpro-nl/knip/commits/5.69.1/packages/knip)

Updates `mocha` from 10.4.0 to 10.8.2
- [Release notes](https://github.com/mochajs/mocha/releases)
- [Changelog](https://github.com/mochajs/mocha/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mochajs/mocha/compare/v10.4.0...v10.8.2)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 4.1.1
  dependency-type: indirect
- dependency-name: knip
  dependency-version: 5.69.1
  dependency-type: direct:development
- dependency-name: mocha
  dependency-version: 10.8.2
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

.dockerignore

@@ -1,3 +1,5 @@
 /repositories
-repo/
-db_backups
+db_backups
+build
+node_modules
+.github

Dockerfile

@@ -1,22 +1,21 @@
-FROM node:15-slim
+FROM node:21-slim
 
 ENV PORT 5000
 EXPOSE $PORT
 
 WORKDIR /app
 
-RUN npm install pm2 -g
-RUN pm2 install typescript
-
 COPY package.json .
 COPY package-lock.json .
 
-RUN npm install
-
-COPY ecosystem.config.js .
+COPY gulpfile.js .
+COPY tsconfig.json .
 COPY healthcheck.js .
-COPY src .
-COPY index.ts .
-COPY public .
 
-CMD [ "pm2-runtime", "ecosystem.config.js"]
+COPY public ./public
+COPY src ./src
+
+RUN npm install && npm run build && npm cache clean --force
+COPY opentelemetry.js .
+
+CMD [ "node", "--require", "./opentelemetry.js", "./build/server/index.js"]

README.md

@@ -1,46 +1,23 @@
 # Anonymous Github
 
-Anonymous Github is a system to anonymize Github repositories before referring to them in a double-anonymous paper submission.
-To start using Anonymous Github right now: **[http://anonymous.4open.science/](http://anonymous.4open.science/)**
+Anonymous Github is a system that helps anonymize Github repositories for double-anonymous paper submissions. A public instance of Anonymous Github is hosted at https://anonymous.4open.science/.
 
-Indeed, in a double-anonymous review process, the open-science data or code that is in the online appendix must be anonymized, similarly to paper anonymization. The authors must
+![screenshot](https://user-images.githubusercontent.com/5577568/217193282-42f608d3-2b46-4ebc-90df-772f248605be.png)
 
-- anonymize URLs: the name of the institution/department/group/authors should not appear in the URLs of the open-science appendix
-- anonymize the appendix content itself
 
-Anonymizing an open-science appendix needs some work, but fortunately, this can be automated, this is what Anonymous Github is about.
+Anonymous Github anonymizes the following:
 
-Anonymous Github anonymizes:
+- Github repository owner, organization, and name
+- File and directory names
+- File contents of all extensions, including markdown, text, Java, etc.
 
-- the Github owner / organization / repository name
-- the content of the repository
-  - file contents (all extensions, md/txt/java/etc)
-  - file and directory names
+## Usage
 
-Question / Feedback / Bug report: please open an issue in this repository.
+### Public instance
 
-## Using Anonymous Github
+**https://anonymous.4open.science/**
 
-## How to create a new anonymized repository
-
-To use it, open the main page (e.g., [http://anonymous.4open.science/](http://anonymous.4open.science/)), login with GitHub, and click on "Anonymize".
-Simply fill 1. the Github repo URL and 2. the id of the anonymized repository, 3. the terms to anonymize (which can be updated afterward).
-The anonymization of the content is done by replacing all occurrences of words in a list by "XXXX" (can be changed in the configuration).
-The word list is provided by the authors, and typically contains the institution name, author names, logins, etc...
-The README is anonymized as well as all files of the repository. Even filenames are anonymized.
-
-In a paper under double-anonymous review, instead of putting a link to Github, one puts a link to the Anonymous Github instance (e.g.
-<http://anonymous.4open.science/r/840c8c57-3c32-451e-bf12-0e20be300389/> which is an anonymous version of this repo).
-
-To start using Anonymous Github right now, a public instance of anonymous_github is hosted at 4open.science:
-
-**[http://anonymous.4open.science/](http://anonymous.4open.science/)**
-
-## What is the scope of anonymization?
-
-In double-anonymous peer-review, the boundary of anonymization is the paper plus its online appendix, and only this, it's not the whole world. Googling any part of the paper or the online appendix can be considered as a deliberate attempt to break anonymity ([explanation](http://www.monperrus.net/martin/open-science-double-anonymous))
-
-## CLI
+### CLI
 
 This CLI tool allows you to anonymize your GitHub repositories locally, generating an anonymized zip file based on your configuration settings.
 
@@ -51,13 +28,10 @@ npm install -g @tdurieux/anonymous_github
 # Run the Anonymous GitHub CLI tool
 anonymous_github
 ```
-## How does it work?
 
-Anonymous Github either download the complete repository and anonymize the content of the file or proxy the request to GitHub. In both case, the original and anonymized versions of the file are cached on the server.
+### Own instance
 
-## Installing Anonymous Github
-
-1. Clone the repository
+#### 1. Clone the repository
 
 ```bash
 git clone https://github.com/tdurieux/anonymous_github/
@@ -65,9 +39,9 @@ cd anonymous_github
 npm i
 ```
 
-2. Configure the Github token
+#### 2. Configure the GitHub token
 
-Create a file `.env` that contains
+Create a `.env` file with the following contents:
 
 ```env
 GITHUB_TOKEN=<GITHUB_TOKEN>
@@ -79,19 +53,27 @@ DB_PASSWORD=
 AUTH_CALLBACK=http://localhost:5000/github/auth,
 ```
 
-`GITHUB_TOKEN` can be generated here: https://github.com/settings/tokens/new with `repo` scope.
-`CLIENT_ID` and `CLIENT_SECRET` are the tokens are generated when you create a new GitHub app https://github.com/settings/applications/new.
-The callback of the GitHub app needs to be defined as `https://<host>/github/auth` (the same as defined in AUTH_CALLBACK).
+- `GITHUB_TOKEN` can be generated here: https://github.com/settings/tokens/new with `repo` scope.
+- `CLIENT_ID` and `CLIENT_SECRET` are the tokens are generated when you create a new GitHub app https://github.com/settings/applications/new.
+- The callback of the GitHub app needs to be defined as `https://<host>/github/auth` (the same as defined in AUTH_CALLBACK).
 
-3. Run Anonymous Github
+#### 3. Start Anonymous Github server
 
 ```bash
 docker-compose up -d
 ```
 
-4. Go to Anonymous Github
+#### 4. Go to Anonymous Github
 
-By default, Anonymous Github uses port 5000. It can be changed in `docker-compose.yml`.
+Go to http://localhost:5000. By default, Anonymous Github uses port 5000. It can be changed in `docker-compose.yml`. I would recommand to put Anonymous GitHub behind ngnix to handle the https certificates.
+
+## What is the scope of anonymization?
+
+In double-anonymous peer-review, the boundary of anonymization is the paper plus its online appendix, and only this, it's not the whole world. Googling any part of the paper or the online appendix can be considered as a deliberate attempt to break anonymity ([explanation](https://www.monperrus.net/martin/open-science-double-blind))
+
+## How does it work?
+
+Anonymous Github either downloads the complete repository and anonymizes the content of the file or proxies the request to GitHub. In both cases, the original and anonymized versions of the file are cached on the server.
 
 ## Related tools
 
@@ -102,3 +84,4 @@ By default, Anonymous Github uses port 5000. It can be changed in `docker-compos
 ## See also
 
 - [Open-science and double-anonymous Peer-Review](https://www.monperrus.net/martin/open-science-double-blind)
+- [ACM Policy on Double-Blind Reviewing](https://dl.acm.org/journal/tods/DoubleBlindPolicy)

docker-compose.yml

@@ -1,19 +1,21 @@
-version: "3"
+version: "3.8"
 
 services:
   anonymous_github:
     build: .
     restart: always
     image: tdurieux/anonymous_github:v2
+    ports:
+      - $EXPOSED_PORT:5000
     env_file:
       - ./.env
-    environment: 
+    volumes:
+      - ./repositories:/app/repositories/
+    environment:
+      - PORT=5000
       - REDIS_HOSTNAME=redis
       - DB_HOSTNAME=mongodb
-    volumes:
-      - .:/app
-    ports:
-      - $PORT:$PORT
+      - STREAMER_ENTRYPOINT=http://streamer:5000/
     healthcheck:
       test:
         - CMD
@@ -22,45 +24,108 @@ services:
       interval: 10s
       timeout: 10s
       retries: 5
-    links:
+    depends_on:
       - mongodb
       - redis
+      - streamer
+
+  streamer:
+    build: .
+    restart: always
+    image: tdurieux/anonymous_github:v2
+    deploy:
+      mode: replicated
+      replicas: 4
+      endpoint_mode: dnsrr
+    entrypoint: ["node", "--require", "./opentelemetry.js", "./build/streamer/index.js"]
+    env_file:
+      - ./.env
+    volumes:
+      - ./repositories:/app/repositories/
+    environment:
+      - PORT=5000
+      - SERVICE_NAME=Streamer
+    healthcheck:
+      test:
+        - CMD
+        - node
+        - healthcheck.js
+      interval: 10s
+      timeout: 10s
+      retries: 5
 
   redis:
     image: "redis:alpine"
+    restart: always
+    ports:
+      - 127.0.0.1:6379:6379
+    healthcheck:
+      test:
+        - CMD
+        - redis-cli
+        - ping
+      interval: 10s
+      timeout: 10s
+      retries: 5
 
   mongodb:
     image: mongo:latest
     restart: on-failure
-    ports:
-      - "127.0.0.1:27017:27017"
     environment:
       MONGO_INITDB_ROOT_USERNAME: $DB_USERNAME
       MONGO_INITDB_ROOT_PASSWORD: $DB_PASSWORD
     volumes:
       - mongodb_data_container:/data/db
+    ports:
+      - 127.0.0.1:27017:27017
     command: --quiet
     healthcheck:
       test:
         - CMD
-        - mongo
+        - mongosh
         - --eval
         - "db.adminCommand('ping')"
       interval: 10s
       timeout: 10s
       retries: 5
 
+  opentelemetry:
+    image: otel/opentelemetry-collector
+    restart: always
+    command: ["--config=/etc/otel-collector-config.yaml"]
+    volumes:
+      - ./opentelemetry-collector.yml:/etc/otel-collector-config.yaml
+    depends_on:
+      - jaeger
+      - prometheus
+
+  jaeger:
+    image: jaegertracing/all-in-one:latest
+    restart: always
+    ports:
+      - 127.0.0.1:16686:16686
+
+  prometheus:
+    image: prom/prometheus:latest
+    restart: always
+    volumes:
+      - ./prometheus.yaml:/etc/prometheus/prometheus.yml
+    ports:
+      - 127.0.0.1:9090:9090
+
   mongodb-backup:
     image: tiredofit/db-backup
     links:
       - mongodb
+    env_file:
+      - ./.env
     volumes:
       - ./db_backups:/backup
     environment:
       - DB_TYPE=mongo
       - DB_HOST=mongodb
-      - DB_DUMP_FREQ=60
-      - DB_CLEANUP_TIME=240
+      - DB_DUMP_FREQ=120
+      - DB_CLEANUP_TIME=500
       - COMPRESSION=XZ
       - DB_USER=$DB_USERNAME
       - DB_PASS=$DB_PASSWORD

ecosystem.config.js

@@ -1,21 +0,0 @@
-module.exports = {
-  apps: [
-    {
-      name: "AnonymousGitHub",
-      script: "./index.ts",
-      exec_mode: "fork",
-      watch: true,
-      ignore_watch: [
-        "node_modules",
-        "repositories",
-        "repo",
-        "public",
-        ".git",
-        "db_backups",
-        "dist",
-      ],
-      interpreter: "node",
-      interpreter_args: "--require ts-node/register",
-    },
-  ],
-};

gulpfile.js

@@ -0,0 +1,59 @@
+const { src, dest } = require("gulp");
+const uglify = require("gulp-uglify");
+const concat = require("gulp-concat");
+var order = require("gulp-order");
+const cleanCss = require("gulp-clean-css");
+
+function defaultTask(cb) {
+  const jsFiles = [
+    "public/script/external/angular.min.js",
+    "public/script/external/angular-translate.min.js",
+    "public/script/external/angular-translate-loader-static-files.min.js",
+    "public/script/external/angular-sanitize.min.js",
+    "public/script/external/angular-route.min.js",
+    "public/script/external/pdf.compat.js",
+    "public/script/external/pdf.js",
+    "public/script/external/github-emojis.js",
+    "public/script/external/marked-emoji.js",
+    "public/script/external/marked.min.js",
+    "public/script/external/purify.min.js",
+    "public/script/external/ansi_up.min.js",
+    "public/script/external/prism.min.js",
+    "public/script/external/katex.min.js",
+    "public/script/external/katex-auto-render.min.js",
+    "public/script/external/marked-katex-extension.umd.min.js",
+    "public/script/external/notebook.min.js",
+    "public/script/external/org.js",
+    "public/script/external/jquery-3.4.1.min.js",
+    "public/script/external/popper.min.js",
+    "public/script/external/bootstrap.min.js",
+    "public/script/external/ace/ace.js",
+    "public/script/external/ui-ace.min.js",
+    "public/script/utils.js",
+    "public/script/ng-pdfviewer.min.js",
+    "public/script/app.js",
+    "public/script/admin.js",
+  ];
+  const cssFiles = [
+    "public/css/bootstrap.min.css",
+    "public/css/font-awesome.min.css",
+    "public/css/notebook.css",
+    "public/css/katex.min.css",
+    "public/css/github-markdown.min.css",
+    "public/css/style.css",
+  ];
+  src(jsFiles)
+    .pipe(order(jsFiles, { base: "./" }))
+    .pipe(concat("bundle.min.js"))
+    .pipe(uglify())
+    .pipe(dest("public/script"))
+    .on("end", cb);
+
+  src(cssFiles)
+    .pipe(order(cssFiles, { base: "./" }))
+    .pipe(concat("all.min.css"))
+    .pipe(cleanCss())
+    .pipe(dest("public/css"));
+}
+
+exports.default = defaultTask;

healthcheck.js

@@ -1,22 +1,26 @@
+require("dotenv").config();
 const http = require("http");
-const config = require("./config");
+const config = require("./build/config");
 
 const options = {
-  host: "localhost",
-  port: config.PORT,
+  path: "/healthcheck",
+  method: "GET",
+  host: "127.0.0.1",
+  port: config.default.PORT,
   timeout: 2000,
 };
-
 const request = http.request(options, (res) => {
-  if (res.statusCode == 200) {
+  if (res.statusCode == 200 || res.statusCode == 404) {
     process.exit(0);
   } else {
+    const reqURL = `${res.req.protocol}://${res.req.host}:${options.port}${res.req.path}`;
+    console.log(reqURL, res.statusCode);
     process.exit(1);
   }
 });
 
 request.on("error", (err) => {
-  console.log("ERROR");
+  console.log("ERROR", err);
   process.exit(1);
 });
 

import.js

@@ -1,153 +0,0 @@
-const fs = require("fs").promises;
-const ofs = require("fs");
-const path = require("path");
-const gh = require("parse-github-url");
-const { Octokit } = require("@octokit/rest");
-
-const config = require("./config");
-const db = require("./utils/database");
-const repoUtils = require("./utils/repository");
-const fileUtils = require("./utils/file");
-const githubUtils = require("./utils/github");
-
-// const ROOT = "./repositories";
-const ROOT = "./repo";
-(async () => {
-  await db.connect();
-  const repositories = await fs.readdir(ROOT);
-  let index = 0;
-  for (let repo of repositories) {
-    // for (let repo of ["14bfc5c6-b794-487e-a58a-c54103a93c7b"]) {
-    console.log("Import ", index++, "/", repositories.length, " ", repo);
-    try {
-      const conf = await repoUtils.getConfig(repo);
-      if (conf) {
-        continue;
-      }
-      // const repoPath = path.join("./repositories", repo);
-      const repoPath = path.join(ROOT, repo);
-      const configPath = path.join(repoPath, "config.json");
-      if (!ofs.existsSync(configPath)) {
-        continue;
-      }
-      const repoConfig = JSON.parse(await fs.readFile(configPath));
-      const r = gh(repoConfig.repository);
-      if (r == null) {
-        console.log(`${repoConfig.repository} is not a valid github url.`);
-        continue;
-      }
-      const fullName = `${r.owner}/${r.name}`;
-
-      // const octokit = new Octokit({ auth: config.GITHUB_TOKEN });
-      // try {
-      //   await octokit.apps.checkToken({
-      //     client_id: config.CLIENT_ID,
-      //     access_token: repoConfig.token,
-      //   });
-      // } catch (error) {
-      //   delete repoConfig.token;
-      //   continue
-      // }
-      let token = repoConfig.token;
-
-      if (!token) {
-        token = config.GITHUB_TOKEN;
-      }
-
-      const branches = await repoUtils.getRepoBranches({
-        fullName,
-        token,
-      });
-      const details = await repoUtils.getRepoDetails({
-        fullName,
-        token,
-      });
-      let branch = details.default_branch;
-      if (r.branch && branches[r.branch]) {
-        branch = r.branch;
-      }
-      if (!branches[branch]) {
-        console.log(branch, details.default_branch, branches);
-      }
-      let commit = branches[branch].commit.sha;
-      const anonymizeDate = new Date();
-
-      let mode = "stream";
-      // if (details.size < 1024) {
-      //   mode = "download";
-      // }
-
-      let expirationDate = null;
-      if (repoConfig.expiration_date) {
-        expirationDate = new Date(repoConfig.expiration_date["$date"]);
-      }
-      const expirationMode = repoConfig.expiration
-        ? repoConfig.expiration
-        : "never";
-
-      const repoConfiguration = {
-        repoId: repo,
-        fullName,
-        // owner: "tdurieux",
-        owner: r.owner,
-        terms: repoConfig.terms,
-        repository: repoConfig.repository,
-        token: repoConfig.token,
-        branch,
-        commit,
-        anonymizeDate,
-        options: {
-          image: false,
-          mode,
-          expirationMode,
-          expirationDate,
-          update: true,
-          page: details.has_pages,
-          pdf: false,
-          notebook: true,
-          loc: false,
-          link: true,
-        },
-      };
-      await db.get("anonymized_repositories").updateOne(
-        {
-          repoId: repo,
-        },
-        {
-          $set: repoConfiguration,
-        },
-        { upsert: true }
-      );
-      if (ofs.existsSync(repoUtils.getOriginalPath(repo))) {
-        await fs.rm(repoUtils.getOriginalPath(repo), {
-          recursive: true,
-          force: true,
-        });
-      }
-      if (ofs.existsSync(repoUtils.getAnonymizedPath(repo))) {
-        await fs.rm(repoUtils.getAnonymizedPath(repo), {
-          recursive: true,
-          force: true,
-        });
-      }
-      // await githubUtils.downloadRepoAndAnonymize(repoConfiguration);
-      // await fileUtils.getFileList({ repoConfig: repoConfiguration });
-      await repoUtils.updateStatus(repoConfiguration, "ready");
-      console.log(
-        expirationDate,
-        expirationDate != null && expirationDate < new Date(),
-        expirationDate < new Date()
-      );
-      if (
-        expirationMode != "never" &&
-        expirationDate != null &&
-        expirationDate < new Date()
-      ) {
-        await repoUtils.updateStatus(repoConfiguration, "expired");
-      }
-    } catch (error) {
-      console.error(error);
-    }
-  }
-  await db.close();
-})();

index.ts

@@ -1,7 +0,0 @@
-import { config } from "dotenv";
-config();
-
-import server from "./src/server";
-
-// start the server
-server();

migrateDB.ts

@@ -1,217 +0,0 @@
-require("dotenv").config();
-
-import * as mongoose from "mongoose";
-import config from "./config";
-import * as database from "./src/database/database";
-import RepositoryModel from "./src/database/repositories/repositories.model";
-import AnonymizedRepositoryModel from "./src/database/anonymizedRepositories/anonymizedRepositories.model";
-import UserModel from "./src/database/users/users.model";
-
-const MONGO_URL = `mongodb://${config.DB_USERNAME}:${config.DB_PASSWORD}@${config.DB_HOSTNAME}:27017/`;
-
-async function connect(db) {
-  const t = new mongoose.Mongoose();
-  t.set("useNewUrlParser", true);
-  t.set("useFindAndModify", true);
-  t.set("useUnifiedTopology", true);
-
-  const database = t.connection;
-
-  await t.connect(MONGO_URL + db, {
-    authSource: "admin",
-    useCreateIndex: true,
-    useFindAndModify: true,
-  });
-
-  return database;
-}
-
-(async () => {
-  await database.connect();
-  const oldDB = await connect("anonymous_github");
-
-  console.log("Import Users");
-  let index = 0;
-  const userQuery = oldDB.collection("users").find();
-  const totalUser = await userQuery.count();
-
-  while (await userQuery.hasNext()) {
-    const r = await userQuery.next();
-    index++;
-    console.log(`Import User [${index}/${totalUser}]: ${r.username}`);
-
-    const newRepos = [];
-    const allRepoIds = [];
-    if (r.repositories) {
-      const finds = await RepositoryModel.find({
-        externalId: {
-          $in: r.repositories.map((repo) => "gh_" + repo.id),
-        },
-      }).select("externalId");
-      finds.forEach((f) => allRepoIds.push(f.id));
-
-      const repoIds = new Set<string>();
-      const toInsert = r.repositories.filter((f) => {
-        if (repoIds.has(f.id)) return false;
-        repoIds.add(f.id);
-        const externalId = "gh_" + f.id;
-        return finds.filter((f) => f.externalId == externalId).length == 0;
-      });
-
-      for (const repo of toInsert) {
-        newRepos.push(
-          new RepositoryModel({
-            externalId: "gh_" + repo.id,
-            name: repo.full_name,
-            url: repo.html_url,
-            size: repo.size,
-            defaultBranch: repo.default_branch,
-          })
-        );
-      }
-      if (newRepos.length > 0) {
-        await RepositoryModel.insertMany(newRepos);
-      }
-      newRepos.forEach((f) => allRepoIds.push(f.id));
-    }
-    const user = new UserModel({
-      accessTokens: {
-        github: r.accessToken,
-      },
-      externalIDs: {
-        github: r.profile.id,
-      },
-      username: r.username,
-      emails: r.profile.emails?.map((email) => {
-        return { email: email.value, default: false };
-      }),
-      photo: r.profile.photos[0]?.value,
-      repositories: allRepoIds,
-      default: {
-        terms: r.default?.terms,
-        options: r.default?.options,
-      },
-    });
-    if (user.emails?.length) user.emails[0].default = true;
-
-    await user.save();
-  }
-
-  console.log("Import Repositories");
-  const repoQuery = oldDB.collection("repositories").find();
-  const totalRepository = await repoQuery.count();
-  index = 0;
-  while (await repoQuery.hasNext()) {
-    const r = await repoQuery.next();
-    if (!r.id) continue;
-    index++;
-    console.log(
-      `Import Repository [${index}/${totalRepository}]: ${r.fullName}`
-    );
-
-    let find = await RepositoryModel.findOne({
-      externalId: "gh_" + r.id,
-    });
-
-    if (find == null) {
-      find = new RepositoryModel({
-        externalId: "gh_" + r.id,
-        name: r.fullName,
-        url: r.html_url,
-        size: r.size,
-        defaultBranch: r.default_branch,
-      });
-    }
-    if (r.branches) {
-      const branches = [...Object.values(r.branches)].map((b: any) => {
-        const o: any = { name: b.name, commit: b.commit.sha };
-        if (b.name == find.defaultBranch) {
-          o.readme = r.readme;
-        }
-        return o;
-      });
-      find.branches = branches;
-    }
-    await find.save();
-  }
-
-  console.log("Import Anonymized Repositories");
-  const anoQuery = oldDB.collection("anonymized_repositories").find();
-  const totalAno = await anoQuery.count();
-  index = 0;
-  while (await anoQuery.hasNext()) {
-    const r = await anoQuery.next();
-
-    index++;
-    console.log(
-      `Import Anonymized Repository [${index}/${totalAno}]: ${r.repoId}`
-    );
-
-    let repo = await RepositoryModel.findOne({ name: r.fullName });
-    if (repo == null) {
-      const tmp = await oldDB
-        .collection("repositories")
-        .findOne({ fullName: r.fullName });
-      if (tmp) {
-        repo = await RepositoryModel.findOne({ externalId: "gh_" + tmp.id });
-      } else {
-        console.error(`Repository ${r.fullName} is not found (renamed)`);
-      }
-    }
-    let size = { storage: 0, file: 0 };
-    function recursiveCount(files) {
-      const out = { storage: 0, file: 0 };
-      for (const name in files) {
-        const file = files[name];
-        if (file.size && file.sha && parseInt(file.size) == file.size) {
-          out.storage += file.size as number;
-          out.file++;
-        } else if (typeof file == "object") {
-          const r = recursiveCount(file);
-          out.storage += r.storage;
-          out.file += r.file;
-        }
-      }
-      return out;
-    }
-
-    if (r.originalFiles) {
-      size = recursiveCount(r.originalFiles);
-    }
-    const owner = await UserModel.findOne({ username: r.owner }).select("_id");
-    await new AnonymizedRepositoryModel({
-      repoId: r.repoId,
-      status: r.status,
-      anonymizeDate: r.anonymizeDate,
-      lastView: r.lastView,
-      pageView: r.pageView,
-      owner: owner?.id,
-      size,
-      source: {
-        accessToken: r.token,
-        type: r.options.mode == "download" ? "GitHubDownload" : "GitHubStream",
-        branch: r.branch,
-        commit: r.commit,
-        repositoryId: repo?.id,
-        repositoryName: r.fullName,
-      },
-      options: {
-        terms: r.terms,
-        expirationMode: r.options.expirationMode,
-        expirationDate: r.options.expirationDate
-          ? new Date(r.options.expirationDate)
-          : null,
-        update: r.options.update,
-        image: r.options.image,
-        pdf: r.options.pdf,
-        notebook: r.options.notebook,
-        loc: r.options.loc,
-        link: r.options.link,
-        page: r.options.page,
-        pageSource: r.options.pageSource,
-      },
-    }).save();
-  }
-  console.log("Import finished!");
-  setTimeout(() => process.exit(), 5000);
-})();

opentelemetry-collector.yml

@@ -0,0 +1,40 @@
+receivers:
+  otlp:
+    protocols:
+      grpc:
+
+exporters:
+  prometheus:
+    endpoint: "0.0.0.0:8889"
+    const_labels:
+      label1: value1
+
+  debug:
+
+  otlp:
+    endpoint: jaeger:4317
+    tls:
+      insecure: true
+
+processors:
+  batch:
+
+extensions:
+  health_check:
+  pprof:
+    endpoint: :1888
+  zpages:
+    endpoint: :55679
+
+service:
+  extensions: [health_check, pprof, zpages]
+  pipelines:
+    traces:
+      receivers: [otlp]
+      exporters: [debug, otlp]
+    metrics:
+      receivers: [otlp]
+      exporters: [debug, prometheus]
+    logs:
+      receivers: [otlp]
+      exporters: [debug]

opentelemetry.js

@@ -0,0 +1,29 @@
+const opentelemetry = require("@opentelemetry/sdk-node");
+const {
+  getNodeAutoInstrumentations,
+} = require("@opentelemetry/auto-instrumentations-node");
+const {
+  OTLPTraceExporter,
+} = require("@opentelemetry/exporter-trace-otlp-grpc");
+const {
+  OTLPMetricExporter,
+} = require("@opentelemetry/exporter-metrics-otlp-grpc");
+const { PeriodicExportingMetricReader } = require("@opentelemetry/sdk-metrics");
+const { diag, DiagConsoleLogger, DiagLogLevel } = require("@opentelemetry/api");
+
+// diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.INFO);
+
+const sdk = new opentelemetry.NodeSDK({
+  serviceName: process.env.SERVICE_NAME || "Anonymous-GitHub",
+  logRecordProcessor: getNodeAutoInstrumentations().logRecordProcessor,
+  traceExporter: new OTLPTraceExporter({
+    url: "http://opentelemetry:4317/v1/traces",
+  }),
+  metricReader: new PeriodicExportingMetricReader({
+    exporter: new OTLPMetricExporter({
+      url: "http://opentelemetry:4317/v1/metrics",
+    }),
+  }),
+  instrumentations: [getNodeAutoInstrumentations()],
+});
+sdk.start();

package.json

@@ -1,16 +1,16 @@
 {
   "name": "@tdurieux/anonymous_github",
-  "version": "2.1.1",
+  "version": "2.2.0",
   "description": "Anonymise Github repositories for double-anonymous reviews",
   "bin": {
-    "anonymous_github": "build/cli.js"
+    "anonymous_github": "build/cli/index.js"
   },
   "scripts": {
     "test": "mocha --reporter spec",
-    "start": "node --inspect=5858 -r ts-node/register ./index.ts",
-    "dev": "nodemon --transpile-only index.ts",
-    "migrateDB": "ts-node --transpile-only migrateDB.ts",
-    "build": "tsc"
+    "start": "node --inspect=5858 -r ts-node/register ./src/server/index.ts",
+    "dev": "nodemon --transpile-only ./src/server/index.ts",
+    "build": "rm -rf build && tsc && gulp",
+    "knip": "knip"
   },
   "repository": {
     "type": "git",
@@ -30,60 +30,72 @@
     "build"
   ],
   "dependencies": {
-    "@octokit/oauth-app": "^4.1.0",
-    "@octokit/rest": "^19.0.5",
-    "@pm2/io": "^5.0.0",
-    "archiver": "^5.3.1",
-    "aws-sdk": "^2.1238.0",
-    "bullmq": "^2.3.2",
+    "@aws-sdk/client-s3": "^3.540.0",
+    "@aws-sdk/lib-storage": "^3.540.0",
+    "@mongodb-js/zstd": "^1.2.0",
+    "@octokit/rest": "^20.0.2",
+    "@opentelemetry/api": "^1.8.0",
+    "@opentelemetry/auto-instrumentations-node": "^0.43.0",
+    "@opentelemetry/exporter-metrics-otlp-grpc": "^0.49.1",
+    "@opentelemetry/exporter-metrics-otlp-proto": "^0.49.1",
+    "@opentelemetry/exporter-trace-otlp-grpc": "^0.49.1",
+    "@opentelemetry/exporter-trace-otlp-proto": "^0.49.1",
+    "@opentelemetry/sdk-metrics": "^1.22.0",
+    "@opentelemetry/sdk-node": "^0.49.1",
+    "@opentelemetry/sdk-trace-node": "^1.22.0",
+    "@smithy/node-http-handler": "^2.5.0",
+    "archiver": "^5.3.2",
+    "bullmq": "^2.4.0",
+    "cacheable-lookup": "^5.0.3",
     "compression": "^1.7.4",
-    "connect-redis": "^6.1.3",
-    "decompress-stream-to-s3": "^1.3.1",
-    "dotenv": "^16.0.3",
-    "express": "^4.18.2",
-    "express-rate-limit": "^6.6.0",
-    "express-session": "^1.17.3",
-    "express-slow-down": "^1.5.0",
-    "got": "^11.8.5",
-    "inquirer": "^8.2.5",
-    "istextorbinary": "^6.0.0",
-    "marked": "^4.1.1",
+    "connect-redis": "^7.0.1",
+    "crypto-js": "^4.2.0",
+    "decompress-stream-to-s3": "^2.1.1",
+    "dotenv": "^16.4.5",
+    "express": "^4.19.2",
+    "express-rate-limit": "^6.11.2",
+    "express-session": "^1.18.0",
+    "express-slow-down": "^2.0.1",
+    "got": "^11.8.6",
+    "inquirer": "^8.2.6",
+    "istextorbinary": "^9.5.0",
+    "marked": "^5.1.2",
     "mime-types": "^2.1.35",
-    "mongoose": "^6.6.7",
-    "node-schedule": "^2.1.0",
+    "mongoose": "^7.6.10",
+    "node-schedule": "^2.1.1",
     "parse-github-url": "^1.0.2",
     "passport": "^0.6.0",
     "passport-github2": "^0.1.12",
-    "rate-limit-redis": "^3.0.1",
-    "redis": "^4.3.1",
-    "textextensions": "^5.15.0",
-    "ts-custom-error": "^3.3.0",
-    "unzip-stream": "^0.3.1",
-    "xml-flow": "^1.0.4"
+    "rate-limit-redis": "^4.2.0",
+    "redis": "^4.6.13",
+    "ts-custom-error": "^3.3.1",
+    "unzip-stream": "^0.3.1"
   },
   "devDependencies": {
-    "@types/archiver": "^5.3.1",
-    "@types/compression": "^1.7.1",
-    "@types/connect-redis": "^0.0.18",
-    "@types/express": "^4.17.14",
-    "@types/express-rate-limit": "^6.0.0",
-    "@types/express-session": "^1.17.5",
-    "@types/express-slow-down": "^1.3.2",
+    "@types/archiver": "^5.3.4",
+    "@types/compression": "^1.7.5",
+    "@types/crypto-js": "^4.2.2",
+    "@types/express": "^4.17.21",
+    "@types/express-session": "^1.18.0",
     "@types/got": "^9.6.12",
-    "@types/inquirer": "^8.0.0",
-    "@types/marked": "^4.0.7",
-    "@types/mime-types": "^2.1.0",
-    "@types/parse-github-url": "^1.0.0",
-    "@types/passport": "^1.0.11",
-    "@types/passport-github2": "^1.2.5",
-    "@types/rate-limit-redis": "^1.7.4",
-    "@types/tar-fs": "^2.0.1",
-    "@types/unzip-stream": "^0.3.1",
-    "@types/xml-flow": "^1.0.1",
-    "chai": "^4.3.6",
-    "mocha": "^10.1.0",
-    "ts-node": "^10.9.1",
-    "typescript": "^4.8.4"
+    "@types/inquirer": "^8.2.10",
+    "@types/marked": "^5.0.2",
+    "@types/mime-types": "^2.1.4",
+    "@types/node-schedule": "^2.1.6",
+    "@types/parse-github-url": "^1.0.3",
+    "@types/passport": "^1.0.16",
+    "@types/passport-github2": "^1.2.9",
+    "@types/unzip-stream": "^0.3.4",
+    "gulp": "^5.0.0",
+    "gulp-clean-css": "^4.3.0",
+    "gulp-concat": "^2.6.1",
+    "gulp-order": "^1.2.0",
+    "gulp-uglify": "^3.0.2",
+    "knip": "^5.1.0",
+    "mocha": "^10.4.0",
+    "nodemon": "^3.1.0",
+    "ts-node": "^10.9.2",
+    "typescript": "^5.4.3"
   },
   "nodemonConfig": {
     "ignore": [

prometheus.yaml

@@ -0,0 +1,6 @@
+scrape_configs:
+  - job_name: 'otel-collector'
+    scrape_interval: 10s
+    static_configs:
+      - targets: ['opentelemetry:8889']
+      - targets: ['opentelemetry:8888']

public/css/prism-okaidia.css

@@ -0,0 +1,123 @@
+/**
+ * okaidia theme for JavaScript, CSS and HTML
+ * Loosely based on Monokai textmate theme by http://www.monokai.nl/
+ * @author ocodia
+ */
+
+code[class*="language-"],
+pre[class*="language-"] {
+	color: #f8f8f2;
+	background: none;
+	text-shadow: 0 1px rgba(0, 0, 0, 0.3);
+	font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace;
+	font-size: 1em;
+	text-align: left;
+	white-space: pre;
+	word-spacing: normal;
+	word-break: normal;
+	word-wrap: normal;
+	line-height: 1.5;
+
+	-moz-tab-size: 4;
+	-o-tab-size: 4;
+	tab-size: 4;
+
+	-webkit-hyphens: none;
+	-moz-hyphens: none;
+	-ms-hyphens: none;
+	hyphens: none;
+}
+
+/* Code blocks */
+pre[class*="language-"] {
+	padding: 1em;
+	margin: .5em 0;
+	overflow: auto;
+	border-radius: 0.3em;
+}
+
+:not(pre) > code[class*="language-"],
+pre[class*="language-"] {
+	background: #272822;
+}
+
+/* Inline code */
+:not(pre) > code[class*="language-"] {
+	padding: .1em;
+	border-radius: .3em;
+	white-space: normal;
+}
+
+.token.comment,
+.token.prolog,
+.token.doctype,
+.token.cdata {
+	color: #8292a2;
+}
+
+.token.punctuation {
+	color: #f8f8f2;
+}
+
+.token.namespace {
+	opacity: .7;
+}
+
+.token.property,
+.token.tag,
+.token.constant,
+.token.symbol,
+.token.deleted {
+	color: #f92672;
+}
+
+.token.boolean,
+.token.number {
+	color: #ae81ff;
+}
+
+.token.selector,
+.token.attr-name,
+.token.string,
+.token.char,
+.token.builtin,
+.token.inserted {
+	color: #a6e22e;
+}
+
+.token.operator,
+.token.entity,
+.token.url,
+.language-css .token.string,
+.style .token.string,
+.token.variable {
+	color: #f8f8f2;
+}
+
+.token.atrule,
+.token.attr-value,
+.token.function,
+.token.class-name {
+	color: #e6db74;
+}
+
+.token.keyword {
+	color: #66d9ef;
+}
+
+.token.regex,
+.token.important {
+	color: #fd971f;
+}
+
+.token.important,
+.token.bold {
+	font-weight: bold;
+}
+.token.italic {
+	font-style: italic;
+}
+
+.token.entity {
+	cursor: help;
+}

public/css/style.css

@@ -133,6 +133,12 @@ a:hover {
   color: var(--link-hover-color);
 }
 
+.markdown-body .emoji {
+  height: 1.3em;
+  margin: 0;
+  vertical-align: -0.1em;
+}
+
 .navbar {
   background: var(--header-bg-color) !important;
 }
@@ -428,6 +434,11 @@ notebook {
   padding-left: 100px;
 }
 
+.nb-output th,
+.nb-output td {
+  border: 1px solid var(--border-color) !important;
+}
+
 .floatingchat-container-wrap {
   left: inherit !important;
   bottom: inherit !important;
@@ -579,6 +590,7 @@ pre,
 code {
   font-family: "Fira Code", "Courier New", Courier, monospace;
   line-height: 1.1;
+  color: var(--color)
 }
 
 .diff-lines,

public/favicon/site.webmanifest

@@ -3,12 +3,12 @@
     "short_name": "Anonymous Github",
     "icons": [
         {
-            "src": "/android-chrome-192x192.png",
+            "src": "/favicon/android-chrome-192x192.png",
             "sizes": "192x192",
             "type": "image/png"
         },
         {
-            "src": "/android-chrome-256x256.png",
+            "src": "/favicon/android-chrome-256x256.png",
             "sizes": "256x256",
             "type": "image/png"
         }

public/i18n/locale-en.json

@@ -1,5 +1,9 @@
 {
   "ERRORS": {
+    "unknown_error": "Unknown error, contact the admin.",
+    "unreachable": "Anonymous GitHub is unreachable, contact the admin.",
+    "request_error": "Unable to download the file, check your connection or contact the admin.",
+    "repo_access_limited": "Access to repository limited by org.",
     "repo_not_found": "The repository is not found.",
     "repo_not_accessible": "Anonymous GitHub is unable to or is forbidden to access the repository.",
     "repository_expired": "The repository is expired",

public/index.html

@@ -33,51 +33,7 @@
     <meta name="msapplication-config" content="/favicon/browserconfig.xml" />
     <meta name="theme-color" content="#ffffff" />
 
-    <!-- CSS -->
-    <link rel="stylesheet" href="/css/bootstrap.min.css" />
-    <link rel="stylesheet" href="/css/font-awesome.min.css" />
-
-    <link rel="stylesheet" href="/css/color-schema.css" />
-    <link rel="stylesheet" href="/css/style.css" />
-    <link rel="stylesheet" href="/css/notebook.css" />
-    <link rel="stylesheet" href="/css/prism.css" />
-    <link rel="stylesheet" href="/css/katex.min.css" />
-    <link rel="stylesheet" href="/css/github-markdown.min.css" />
-
-    <!-- JS -->
-    <script src="/script/external/angular.min.js"></script>
-    <script src="/script/external/angular-translate.min.js"></script>
-    <script src="/script/external/angular-translate-loader-static-files.min.js"></script>
-    <script src="/script/external/angular-sanitize.min.js"></script>
-    <script src="/script/external/angular-route.min.js"></script>
-    <script src="/script/external/ana.min.js"></script>
-
-    <script src="/script/external/jquery-3.4.1.min.js"></script>
-    <script src="/script/external/popper.min.js"></script>
-    <script src="/script/external/bootstrap.min.js"></script>
-
-    <script src="/script/external/pdf.compat.js"></script>
-    <script src="/script/external/pdf.js"></script>
-    <script src="/script/external/ace/ace.js"></script>
-    <script src="/script/external/ui-ace.min.js"></script>
-    <script src="/script/langColors.js"></script>
-
-    <!-- Notebook -->
-    <script src="/script/external/marked.min.js"></script>
-    <script src="/script/external/purify.min.js"></script>
-    <script src="/script/external/ansi_up.min.js"></script>
-    <script src="/script/external/prism.min.js"></script>
-    <script src="/script/external/katex.min.js"></script>
-    <script src="/script/external/katex-auto-render.min.js"></script>
-    <script src="/script/external/notebook.min.js"></script>
-
-    <script src="/script/external/org.js"></script>
-
-    <!-- Anonymous GitHub scripts -->
-    <script src="/script/utils.js"></script>
-    <script src="/script/ng-pdfviewer.min.js"></script>
-    <script src="/script/admin.js"></script>
-    <script src="/script/app.js"></script>
+    <link rel="stylesheet" href="/css/all.min.css" />
   </head>
   <body keypress-events class="d-flex flex-column">
     <ng-include src="'partials/header.htm'"></ng-include>
@@ -112,6 +68,11 @@
       </div>
     </div>
 
+    <script src="/script/bundle.min.js"></script>
+    <script>
+      ace.config.set("basePath", "/script/external/ace/");
+      PDFJS.workerSrc = "/script/external/pdf.worker.js";
+    </script>
     <script src="https://storage.ko-fi.com/cdn/scripts/overlay-widget.js"></script>
     <script>
       kofiWidgetOverlay.draw("tdurieux", {

public/partials/admin/queues.htm

@@ -148,5 +148,79 @@
         There is no job to display.
       </li>
     </ul>
+    <h1>Remove Cache</h1>
+    <ul class="p-0 m-0 w-100">
+      <li
+        class="col-12 d-flex px-0 py-3 border-bottom color-border-secondary"
+        ng-repeat="job in removeCaches as filteredRemoveCache"
+      >
+        <div class="w-100">
+          <div class="">
+            <h3>
+              <a target="__blank" ng-href="/r/{{job.id}}" ng-bind="job.id"></a>
+              <span class="badge" ng-bind="job.progress.status | title"></span>
+            </h3>
+          </div>
+          <div class="color-text-secondary mb-1">
+            <span ng-if="job.timestamp">
+              Created on:
+              <span ng-bind="job.timestamp | humanTime"></span>
+            </span>
+            <span ng-if="job.finishedOn">
+              Finished on:
+              <span ng-bind="job.finishedOn | humanTime"></span>
+            </span>
+            <span ng-if="job.processedOn">
+              Processed on:
+              <span ng-bind="job.processedOn | humanTime"></span>
+            </span>
+          </div>
+          <div>
+            <pre
+              ng-repeat="stack in job.stacktrace track by $index"
+            ><code ng-bind="stack"></code></pre>
+          </div>
+        </div>
+        <div class="d-flex">
+          <div class="dropdown">
+            <button
+              class="btn black_border dropdown-toggle btn-sm"
+              type="button"
+              id="dropdownMenuButton"
+              data-toggle="dropdown"
+              aria-haspopup="true"
+              aria-expanded="false"
+            >
+              Actions
+            </button>
+            <div class="dropdown-menu" aria-labelledby="dropdownMenuButton">
+              <a
+                class="dropdown-item"
+                href="#"
+                ng-click="removeJob('cache', job)"
+              >
+                <i class="fas fa-trash-alt"></i> Remove
+              </a>
+              <a
+                class="dropdown-item"
+                href="#"
+                ng-click="retryJob('cache', job)"
+              >
+                <i class="fas fa-sync"></i> Retry
+              </a>
+              <a class="dropdown-item" href="/anonymize/{{job.id}}">
+                <i class="far fa-edit" aria-hidden="true"></i> Edit
+              </a>
+            </div>
+          </div>
+        </div>
+      </li>
+      <li
+        class="col-12 d-flex px-0 py-3 border-bottom color-border-secondary"
+        ng-if="filteredRemoveCache.length == 0"
+      >
+        There is no job to display.
+      </li>
+    </ul>
   </div>
 </div>

public/partials/admin/repositories.htm

@@ -313,6 +313,9 @@
               Actions
             </button>
             <div class="dropdown-menu" aria-labelledby="dropdownMenuButton">
+              <a class="dropdown-item" href="#" ng-click="removeCache(repo)">
+                <i class="fas fa-trash-alt"></i> Remove Cache
+              </a>
               <a class="dropdown-item" href="/anonymize/{{repo.repoId}}">
                 <i class="far fa-edit" aria-hidden="true"></i> Edit
               </a>

public/partials/admin/user.htm

@@ -1,9 +1,15 @@
 <div class="container page">
   <div class="row">
     <h1>
-      <img ng-src="{{userInfo.photo}}" ng-if="userInfo.photo" width="30" height="30" class="rounded-circle ng-scope">
+      <img
+        ng-src="{{userInfo.photo}}"
+        ng-if="userInfo.photo"
+        width="30"
+        height="30"
+        class="rounded-circle ng-scope"
+      />
       {{userInfo.username}}
-      <span class="badge"><span ng-bind="userInfo.status | title"></span>
+      <span class="badge"><span ng-bind="userInfo.status | title"></span></span>
     </h1>
     <div class="row mb-3 m-0 py-2 border">
       <div class="col-2 font-weight-bold">ID</div>
@@ -16,12 +22,47 @@
       <div class="col-10">{{userInfo.accessTokens.github}}</div>
 
       <div class="col-2 font-weight-bold">Github</div>
-      <div class="col-10"><a ng-href="https://github.com/{{userInfo.username}}">{{userInfo.username}}</a></div>
+      <div class="col-10">
+        <a ng-href="https://github.com/{{userInfo.username}}"
+          >{{userInfo.username}}</a
+        >
+      </div>
 
       <div class="col-2 font-weight-bold">Github Repositories</div>
-      <div class="col-10">{{userInfo.repositories.length}}</a></div>
+      <div class="col-10" ng-click="showRepos =!showRepos">
+        {{userInfo.repositories.length}}
+      </div>
+      <button
+        class="btn btn-primary m-1 mx-3"
+        ng-click="getGitHubRepositories()"
+      >
+        Regresh Repositories
+      </button>
+      <ul class="m-0 col-12" ng-if="showRepos">
+        <li
+          class="col-12 d-flex px-0 py-3 border-bottom color-border-secondary"
+          ng-repeat="repo in userInfo.repositories"
+        >
+          <div class="w-100">
+            <div class="">
+              {{repo.name}}
+            </div>
+            <div class="color-text-secondary mt-2">
+              <span
+                class="ml-0 mr-3"
+                title="Size: {{::repo.size | humanFileSize}}"
+                data-toggle="tooltip"
+                data-placement="bottom"
+              >
+                <i class="fas fa-database"></i> {{::repo.size |
+                humanFileSize}}</span
+              >
+            </div>
+          </div>
+        </li>
+      </ul>
     </div>
-    
+
     <h3>Repositories {{repositories.length}}</h3>
     <div class="border-bottom color-border-secondary py-3 w-100">
       <div class="d-flex flex-items-start w-100">
@@ -245,6 +286,64 @@
             >
           </div>
         </div>
+        <div class="d-flex">
+          <div class="dropdown">
+            <button
+              class="btn black_border dropdown-toggle btn-sm"
+              type="button"
+              id="dropdownMenuButton"
+              data-toggle="dropdown"
+              aria-haspopup="true"
+              aria-expanded="false"
+            >
+              Actions
+            </button>
+            <div class="dropdown-menu" aria-labelledby="dropdownMenuButton">
+              <a class="dropdown-item" href="#" ng-click="removeCache(repo)">
+                <i class="fas fa-trash-alt"></i> Remove Cache
+              </a>
+              <a class="dropdown-item" href="/anonymize/{{repo.repoId}}">
+                <i class="far fa-edit" aria-hidden="true"></i> Edit
+              </a>
+              <a
+                class="dropdown-item"
+                href="#"
+                ng-show="repo.status == 'ready' || repo.status == 'error'"
+                ng-click="updateRepository(repo)"
+              >
+                <i class="fas fa-sync"></i> Force update
+              </a>
+              <a
+                class="dropdown-item"
+                href="#"
+                ng-show="repo.status == 'removed'"
+                ng-click="updateRepository(repo)"
+              >
+                <i class="fas fa-check-circle"></i>
+                Enable
+              </a>
+              <a
+                class="dropdown-item"
+                href="#"
+                ng-show="repo.status == 'ready'"
+                ng-click="removeRepository(repo)"
+              >
+                <i class="fas fa-trash-alt"></i> Remove
+              </a>
+              <a class="dropdown-item" href="/r/{{repo.repoId}}/">
+                <i class="fa fa-eye" aria-hidden="true"></i> View Repo
+              </a>
+              <a
+                class="dropdown-item"
+                href="/w/{{repo.repoId}}/"
+                target="_self"
+                ng-if="repo.options.page && repo.status == 'ready'"
+              >
+                <i class="fas fa-globe"></i> View Page
+              </a>
+            </div>
+          </div>
+        </div>
       </li>
       <li
         class="col-12 d-flex px-0 py-3 border-bottom color-border-secondary"

public/partials/anonymize.htm

@@ -13,14 +13,9 @@
           name="anonymize"
           novalidate
         >
-          <h5 class="card-title">Anonymize a repository</h5>
-          <h6 class="card-subtitle mb-2 text-muted">
-            Fill the information to anonymize! It will only take 5min.
-          </h6>
-          <h2>Source</h2>
+          <h3 class="card-title mb-3">Anonymize your repository</h3>
           <!-- repoUrl -->
-          <div class="form-group">
-            <label for="repoUrl">Type the url of your repository</label>
+          <div class="form-group mb-0">
             <input
               type="text"
               class="form-control"
@@ -28,6 +23,7 @@
               id="repoUrl"
               ng-class="{'is-invalid': anonymize.repoUrl.$invalid}"
               ng-model="repoUrl"
+              placeholder="URL of your GitHub repository"
               ng-model-options="{ debounce: {default: 1000, blur: 0, click: 0}, updateOn: 'default blur click' }"
               ng-change="repoSelected()"
             />
@@ -58,37 +54,6 @@
               {{repoUrl}} is already anonymized
             </div>
           </div>
-          <!-- select repo -->
-          <div class="form-group" ng-hide="repoUrl">
-            <label for="repositories">Or select one of your repository</label>
-            <div class="input-group mb-3">
-              <select
-                class="form-control"
-                id="repositories"
-                name="repositories"
-                ng-model="repoUrl"
-                ng-change="repoSelected()"
-              >
-                <option selected value="">None</option>
-                <option
-                  ng-repeat="repo in repositories|orderBy:'fullName'"
-                  value="https://github.com/{{ repo.fullName }}"
-                  ng-bind="repo.fullName"
-                ></option>
-              </select>
-              <div class="input-group-append">
-                <button
-                  class="btn btn-outline-secondary"
-                  ng-click="getRepositories(true)"
-                  title="Refresh!"
-                  data-toggle="tooltip"
-                  data-placement="bottom"
-                >
-                  <i class="fa fa-undo"></i>
-                </button>
-              </div>
-            </div>
-          </div>
           <div ng-show="repoUrl">
             <!-- Branch -->
             <div class="form-group">
@@ -386,29 +351,6 @@
                           >Display Notebooks</label
                         >
                       </div>
-
-                      <div class="form-group">
-                        <label for="mode">Proxy mode</label>
-                        <select
-                          class="form-control"
-                          id="mode"
-                          name="mode"
-                          ng-model="source.type"
-                        >
-                          <option value="GitHubStream" selected>Stream</option>
-                          <option value="GitHubDownload">Download</option>
-                        </select>
-                        <small class="form-text text-muted"
-                          >How the repository will be anonymized. Stream mode
-                          will request the content on the flight. This is the
-                          only option for repositories bigger than
-                          {{site_options.MAX_REPO_SIZE * 1024| humanFileSize}}.
-                          This repository is {{details.size * 8 *1024 |
-                          humanFileSize}}. Download will download the repository
-                          the repository on the anonymous.4open.science server,
-                          it is faster and offer more features.</small
-                        >
-                      </div>
                     </div>
                     <div class="form-group">
                       <div class="form-check">

public/partials/dashboard.htm

@@ -242,7 +242,7 @@
       <div class="d-flex d-inline-flex mt-2">
         <div class="alert alert-info alert-dismissible my-0 ml-1" ng-show="!v" role="alert" ng-repeat="(f, v) in filters.status">
           <strong>{{f | title}}</strong>
-          <button type="button" class="close" data-dismiss="alert" aria-label="Close">
+          <button type="button" class="close" data-dismiss="alert" aria-label="Close" ng-click="filters.status[f] = true;">
             <span aria-hidden="true">&times;</span>
           </button>
         </div>

public/partials/explorer.htm

@@ -1,9 +1,14 @@
 <div class="container-fluid h-100">
   <div class="row h-100">
-    <div class="leftCol shadow p-1 overflow-auto" ng-show="files">
+    <div class="leftCol shadow p-1 overflow-auto" ng-show="files.length">
       <tree class="files" file="files"></tree>
       <div class="bottom column">
-        <div class="last-update">
+        <div
+          class="last-update"
+          data-toggle="tooltip"
+          data-placement="top"
+          title="{{options.lastUpdateDate}}"
+        >
           Last Update: {{options.lastUpdateDate|date}}
         </div>
       </div>
@@ -17,9 +22,23 @@
         </ol>
         <div class="">
           <a
+            ng-if="options.isAdmin || options.isOwner"
+            ng-href="/anonymize/{{repoId}}"
+            class="btn btn-outline-primary btn-sm"
+            >Edit</a
+          >
+          <a
+            ng-show="content != null"
             ng-href="{{url}}"
             target="__self"
             class="btn btn-outline-primary btn-sm"
+            >View raw</a
+          >
+          <a
+            ng-show="content != null"
+            ng-href="{{url}}&download=true"
+            target="__self"
+            class="btn btn-outline-primary btn-sm"
             >Download file</a
           >
           <a
@@ -29,6 +48,13 @@
             class="btn btn-outline-primary btn-sm"
             >Download Repository</a
           >
+          <a
+            ng-if="options.hasWebsite"
+            ng-href="/w/{{repoId}}/"
+            target="__self"
+            class="btn btn-outline-primary btn-sm"
+            >Website</a
+          >
         </div>
       </div>
       <div class="align-items-stretch h-100 w-100 overflow-auto">

public/partials/faq.htm

@@ -13,42 +13,6 @@
             role="tablist"
             aria-multiselectable="true"
           >
-            <div class="panel panel-default mb-4">
-              <div class="panel-heading p-3" role="tab" id="heading0">
-                <h3 class="panel-title">
-                  <a
-                    class="collapsed"
-                    role="button"
-                    title=""
-                    data-toggle="collapse"
-                    data-parent="#faq"
-                    href="#download"
-                    aria-expanded="true"
-                    aria-controls="download"
-                  >
-                    Can I download the repository?
-                  </a>
-                </h3>
-              </div>
-              <div
-                id="download"
-                class="panel-collapse collapse"
-                role="tabpanel"
-                aria-labelledby="heading0"
-              >
-                <div class="panel-body p-3">
-                  <p>
-                    It is currently not possible to download an anonymized
-                    repository neither to clone it.
-                    It is technically possible to implement however it 
-                    would require additional processing power and storage. 
-                    I am currently not able to cover the cost of this feature.
-                    If you want to see this feature on Anonymous GitHub, please consider doing a donation.
-                  </p>
-                </div>
-              </div>
-            </div>
-
             <div class="panel panel-default mb-4">
               <div class="panel-heading p-3" role="tab" id="heading6">
                 <h3 class="panel-title">
@@ -75,11 +39,9 @@
                 <div class="panel-body p-3">
                   <p>
                     <ul>
-                      <li>
-                        Anonymous GitHub does not allow to download the repository.
-                      </li>
                       <li>
                         Anonymous GitHub only anonymizes textual files.
+                        It does not support the use of a static site generator, such as Jekyll, with GitHub Pages (although Markdown files are converted to HTML without any special formatting).
                       </li>
                       <li>
                         Anonymous GitHub does not support files that are larger than 8Mo.
@@ -119,12 +81,7 @@
                 <div class="panel-body p-3">
                   <p>
                     Anonymous Github is able to display pure textual files, such as text or source code. It can also render images, PDFs, and notbooks.
-                    However, only textual based files are anonymized. Anonymous Github considers the following file format as textual:
-                  </p>
-                  <p>
-                    <ul>
-                      <li ng-repeat="format in supportedFileTypes" ng-bind="format"></li>
-                    </ul>
+                    However, only textual based files are anonymized. Anonymous Github analyzes the content of the file to detect if it is textual or not.
                   </p>
                 </div>
               </div>

public/partials/header.htm

@@ -137,7 +137,7 @@
           aria-expanded="false"
         >
           <img
-            src="{{user.photo}}"
+            ng-src="{{user.photo}}"
             ng-if="user.photo"
             width="30"
             height="30"

public/partials/home.htm

@@ -96,7 +96,8 @@
             The reviewers can explore your repository with ease, the source code
             is highlighted, PDFs, images, Notebook are rendered. The goal is to
             make is as easy as possible for the reviewer to explore and review
-            the repository. GitHub pages are also supported.
+            the repository. <a href="https://pages.github.com">GitHub Pages</a>
+            is also supported, but not static site generators, such as Jekyll.
           </p>
         </div>
         <div class="col-md-5">

public/partials/pageView.htm

@@ -2,10 +2,14 @@
 <div ng-if="type == 'html'" ng-bind-html="content" class="file-content markdown-body"></div>
 <div ng-if="type == 'code' && content != null" ui-ace="aceOption" ng-model="content"></div>
 <img ng-if="type == 'image'" class="image-content" ng-src="{{url}}"></img>
+<iframe class="h-100 overflow-auto w-100 b-0" ng-if="type == 'media'" ng-src="{{url}}"></iframe>
 <div class="h-100 overflow-auto" ng-if="type == 'pdf'">
   <pdfviewer class="h-100 overflow-auto" src="{{url}}" id="viewer"></pdfviewer>
 </div>
+<div ng-if="type == 'audio'"><audio controls="controls"><source src="{{url}}" /></audio></div>
 <div ng-if="type == 'IPython'"><notebook file="url"></notebook></div>
 <div ng-if="type == 'error'" class="file-error container d-flex h-100"><h1 class="display-1 m-auto" translate="ERRORS.{{content}}">Error</h1></div></div>
 <div ng-if="type == 'loading' && !error" class="file-error container d-flex h-100"><h1 class="display-1 m-auto">Loading...</h1></div></div>
-<div ng-if="content == null" class="file-error container d-flex h-100"><h1 class="display-1 m-auto">Empty file!</h1></div>
+<div ng-if="type == 'empty'" class="file-error container d-flex h-100"><h1 class="display-1 m-auto">Empty repository!</h1></div>
+<div ng-if="content == null && type != 'empty'" class="file-error container d-flex h-100"><h1 class="display-1 m-auto">Empty file!</h1></div>
+<div ng-if="type == 'binary'" class="file-error container d-flex h-100"><h1 class="display-1 m-auto">Unsupported binary file. You can download the file: <a target="_blank" ng-href="{{url}}&download=true">here</a>.</h1></div>

public/partials/status.htm

@@ -21,10 +21,9 @@
       >
         <span>
           {{repo.status | title}}
-          <span
-            ng-if="repo.status == 'download' && repo.statusMessage"
-            ng-bind="repo.statusMessage | humanFileSize"
-          ></span>
+          <span ng-if="repo.statusMessage"
+            >: {{repo.statusMessage | title}}</span
+          >
         </span>
       </div>
     </div>
@@ -56,10 +55,16 @@
     </p>
   </section>
 
-  <hr />
-
   <section class="py-4">
-    <h2>Support the project</h2>
+    <h2 class="md-1">Support Anonymous GitHub</h2>
+
+    <iframe
+      id="kofiframe"
+      src="https://ko-fi.com/tdurieux/?hidefeed=true&widget=true&embed=true&preview=true"
+      style="border: none; width: 100%"
+      height="650"
+      title="tdurieux"
+    ></iframe>
 
     <div class="row text-center">
       <div class="col-lg-4">

public/script/admin.js

@@ -20,22 +20,56 @@ angular
       $scope.query = {
         page: 1,
         limit: 25,
-        sort: "source.repositoryName",
+        sort: "lastView",
         search: "",
-        ready: true,
-        expired: true,
-        removed: true,
+        ready: false,
+        expired: false,
+        removed: false,
         error: true,
         preparing: true,
       };
 
+      $scope.removeCache = (repo) => {
+        $http.delete("/api/admin/repos/" + repo.repoId).then(
+          (res) => {
+            $scope.$apply();
+          },
+          (err) => {
+            console.error(err);
+          }
+        );
+      };
+
+      $scope.updateRepository = (repo) => {
+        const toast = {
+          title: `Refreshing ${repo.repoId}...`,
+          date: new Date(),
+          body: `The repository ${repo.repoId} is going to be refreshed.`,
+        };
+        $scope.toasts.push(toast);
+        repo.s;
+
+        $http.post(`/api/repo/${repo.repoId}/refresh`).then(
+          (res) => {
+            if (res.data.status == "ready") {
+              toast.title = `${repo.repoId} is refreshed.`;
+            } else {
+              toast.title = `Refreshing of ${repo.repoId}.`;
+            }
+          },
+          (error) => {
+            toast.title = `Error during the refresh of ${repo.repoId}.`;
+            toast.body = error.body;
+          }
+        );
+      };
+
       function getRepositories() {
         $http.get("/api/admin/repos", { params: $scope.query }).then(
           (res) => {
             $scope.total = res.data.total;
             $scope.totalPage = Math.ceil(res.data.total / $scope.query.limit);
             $scope.repositories = res.data.results;
-            $scope.$apply();
           },
           (err) => {
             console.error(err);
@@ -138,7 +172,7 @@ angular
 
         return false;
       };
-      
+
       function getUserRepositories(username) {
         $http.get("/api/admin/users/" + username + "/repos", {}).then(
           (res) => {
@@ -162,6 +196,51 @@ angular
       getUser($routeParams.username);
       getUserRepositories($routeParams.username);
 
+      $scope.removeCache = (repo) => {
+        $http.delete("/api/admin/repos/" + repo.repoId).then(
+          (res) => {
+            $scope.$apply();
+          },
+          (err) => {
+            console.error(err);
+          }
+        );
+      };
+
+      $scope.updateRepository = (repo) => {
+        const toast = {
+          title: `Refreshing ${repo.repoId}...`,
+          date: new Date(),
+          body: `The repository ${repo.repoId} is going to be refreshed.`,
+        };
+        $scope.toasts.push(toast);
+        repo.s;
+
+        $http.post(`/api/repo/${repo.repoId}/refresh`).then(
+          (res) => {
+            if (res.data.status == "ready") {
+              toast.title = `${repo.repoId} is refreshed.`;
+            } else {
+              toast.title = `Refreshing of ${repo.repoId}.`;
+            }
+          },
+          (error) => {
+            toast.title = `Error during the refresh of ${repo.repoId}.`;
+            toast.body = error.body;
+          }
+        );
+      };
+
+      $scope.getGitHubRepositories = (force) => {
+        $http
+          .get(`/api/user/${$scope.userInfo.username}/all_repositories`, {
+            params: { force: "1" },
+          })
+          .then((res) => {
+            $scope.userInfo.repositories = res.data;
+          });
+      };
+
       let timeClear = null;
       $scope.$watch(
         "query",
@@ -247,6 +326,7 @@ angular
           (res) => {
             $scope.downloadJobs = res.data.downloadQueue;
             $scope.removeJobs = res.data.removeQueue;
+            $scope.removeCaches = res.data.cacheQueue;
           },
           (err) => {
             console.error(err);

public/script/external/katex-auto-render.min.js

@@ -1 +1 @@
-!function(e,t){"object"==typeof exports&&"object"==typeof module?module.exports=t(require("katex")):"function"==typeof define&&define.amd?define(["katex"],t):"object"==typeof exports?exports.renderMathInElement=t(require("katex")):e.renderMathInElement=t(e.katex)}("undefined"!=typeof self?self:this,function(e){return function(e){var t={};function r(n){if(t[n])return t[n].exports;var o=t[n]={i:n,l:!1,exports:{}};return e[n].call(o.exports,o,o.exports,r),o.l=!0,o.exports}return r.m=e,r.c=t,r.d=function(e,t,n){r.o(e,t)||Object.defineProperty(e,t,{enumerable:!0,get:n})},r.r=function(e){"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},r.t=function(e,t){if(1&t&&(e=r(e)),8&t)return e;if(4&t&&"object"==typeof e&&e&&e.__esModule)return e;var n=Object.create(null);if(r.r(n),Object.defineProperty(n,"default",{enumerable:!0,value:e}),2&t&&"string"!=typeof e)for(var o in e)r.d(n,o,function(t){return e[t]}.bind(null,o));return n},r.n=function(e){var t=e&&e.__esModule?function(){return e.default}:function(){return e};return r.d(t,"a",t),t},r.o=function(e,t){return Object.prototype.hasOwnProperty.call(e,t)},r.p="",r(r.s=1)}([function(t,r){t.exports=e},function(e,t,r){"use strict";r.r(t);var n=r(0),o=r.n(n),a=function(e,t,r){for(var n=r,o=0,a=e.length;n<t.length;){var i=t[n];if(o<=0&&t.slice(n,n+a)===e)return n;"\\"===i?n++:"{"===i?o++:"}"===i&&o--,n++}return-1},i=function(e,t,r,n){for(var o=[],i=0;i<e.length;i++)if("text"===e[i].type){var l=e[i].data,d=!0,s=0,f=void 0;for(-1!==(f=l.indexOf(t))&&(s=f,o.push({type:"text",data:l.slice(0,s)}),d=!1);;){if(d){if(-1===(f=l.indexOf(t,s)))break;o.push({type:"text",data:l.slice(s,f)}),s=f}else{if(-1===(f=a(r,l,s+t.length)))break;o.push({type:"math",data:l.slice(s+t.length,f),rawData:l.slice(s,f+r.length),display:n}),s=f+r.length}d=!d}o.push({type:"text",data:l.slice(s)})}else o.push(e[i]);return o},l=function(e,t){var r=function(e,t){for(var r=[{type:"text",data:e}],n=0;n<t.length;n++){var o=t[n];r=i(r,o.left,o.right,o.display||!1)}return r}(e,t.delimiters);if(1===r.length&&"text"===r[0].type)return null;for(var n=document.createDocumentFragment(),a=0;a<r.length;a++)if("text"===r[a].type)n.appendChild(document.createTextNode(r[a].data));else{var l=document.createElement("span"),d=r[a].data;t.displayMode=r[a].display;try{t.preProcess&&(d=t.preProcess(d)),o.a.render(d,l,t)}catch(e){if(!(e instanceof o.a.ParseError))throw e;t.errorCallback("KaTeX auto-render: Failed to parse `"+r[a].data+"` with ",e),n.appendChild(document.createTextNode(r[a].rawData));continue}n.appendChild(l)}return n};t.default=function(e,t){if(!e)throw new Error("No element provided to render");var r={};for(var n in t)t.hasOwnProperty(n)&&(r[n]=t[n]);r.delimiters=r.delimiters||[{left:"$$",right:"$$",display:!0},{left:"\\(",right:"\\)",display:!1},{left:"\\[",right:"\\]",display:!0}],r.ignoredTags=r.ignoredTags||["script","noscript","style","textarea","pre","code","option"],r.ignoredClasses=r.ignoredClasses||[],r.errorCallback=r.errorCallback||console.error,r.macros=r.macros||{},function e(t,r){for(var n=0;n<t.childNodes.length;n++){var o=t.childNodes[n];if(3===o.nodeType){var a=l(o.textContent,r);a&&(n+=a.childNodes.length-1,t.replaceChild(a,o))}else 1===o.nodeType&&function(){var t=" "+o.className+" ";-1===r.ignoredTags.indexOf(o.nodeName.toLowerCase())&&r.ignoredClasses.every(function(e){return-1===t.indexOf(" "+e+" ")})&&e(o,r)}()}}(e,r)}}]).default});
+!function(e,t){"object"==typeof exports&&"object"==typeof module?module.exports=t(require("katex")):"function"==typeof define&&define.amd?define(["katex"],t):"object"==typeof exports?exports.renderMathInElement=t(require("katex")):e.renderMathInElement=t(e.katex)}("undefined"!=typeof self?self:this,(function(e){return function(){"use strict";var t={771:function(t){t.exports=e}},n={};function r(e){var o=n[e];if(void 0!==o)return o.exports;var i=n[e]={exports:{}};return t[e](i,i.exports,r),i.exports}r.n=function(e){var t=e&&e.__esModule?function(){return e.default}:function(){return e};return r.d(t,{a:t}),t},r.d=function(e,t){for(var n in t)r.o(t,n)&&!r.o(e,n)&&Object.defineProperty(e,n,{enumerable:!0,get:t[n]})},r.o=function(e,t){return Object.prototype.hasOwnProperty.call(e,t)};var o={};return function(){r.d(o,{default:function(){return d}});var e=r(771),t=r.n(e);const n=function(e,t,n){let r=n,o=0;const i=e.length;for(;r<t.length;){const n=t[r];if(o<=0&&t.slice(r,r+i)===e)return r;"\\"===n?r++:"{"===n?o++:"}"===n&&o--,r++}return-1},i=/^\\begin{/;var a=function(e,t){let r;const o=[],a=new RegExp("("+t.map((e=>e.left.replace(/[-/\\^$*+?.()|[\]{}]/g,"\\$&"))).join("|")+")");for(;r=e.search(a),-1!==r;){r>0&&(o.push({type:"text",data:e.slice(0,r)}),e=e.slice(r));const a=t.findIndex((t=>e.startsWith(t.left)));if(r=n(t[a].right,e,t[a].left.length),-1===r)break;const l=e.slice(0,r+t[a].right.length),s=i.test(l)?l:e.slice(t[a].left.length,r);o.push({type:"math",data:s,rawData:l,display:t[a].display}),e=e.slice(r+t[a].right.length)}return""!==e&&o.push({type:"text",data:e}),o};const l=function(e,n){const r=a(e,n.delimiters);if(1===r.length&&"text"===r[0].type)return null;const o=document.createDocumentFragment();for(let e=0;e<r.length;e++)if("text"===r[e].type)o.appendChild(document.createTextNode(r[e].data));else{const i=document.createElement("span");let a=r[e].data;n.displayMode=r[e].display;try{n.preProcess&&(a=n.preProcess(a)),t().render(a,i,n)}catch(i){if(!(i instanceof t().ParseError))throw i;n.errorCallback("KaTeX auto-render: Failed to parse `"+r[e].data+"` with ",i),o.appendChild(document.createTextNode(r[e].rawData));continue}o.appendChild(i)}return o},s=function(e,t){for(let n=0;n<e.childNodes.length;n++){const r=e.childNodes[n];if(3===r.nodeType){let o=r.textContent,i=r.nextSibling,a=0;for(;i&&i.nodeType===Node.TEXT_NODE;)o+=i.textContent,i=i.nextSibling,a++;const s=l(o,t);if(s){for(let e=0;e<a;e++)r.nextSibling.remove();n+=s.childNodes.length-1,e.replaceChild(s,r)}else n+=a}else if(1===r.nodeType){const e=" "+r.className+" ";-1===t.ignoredTags.indexOf(r.nodeName.toLowerCase())&&t.ignoredClasses.every((t=>-1===e.indexOf(" "+t+" ")))&&s(r,t)}}};var d=function(e,t){if(!e)throw new Error("No element provided to render");const n={};for(const e in t)t.hasOwnProperty(e)&&(n[e]=t[e]);n.delimiters=n.delimiters||[{left:"$$",right:"$$",display:!0},{left:"\\(",right:"\\)",display:!1},{left:"\\begin{equation}",right:"\\end{equation}",display:!0},{left:"\\begin{align}",right:"\\end{align}",display:!0},{left:"\\begin{alignat}",right:"\\end{alignat}",display:!0},{left:"\\begin{gather}",right:"\\end{gather}",display:!0},{left:"\\begin{CD}",right:"\\end{CD}",display:!0},{left:"\\[",right:"\\]",display:!0}],n.ignoredTags=n.ignoredTags||["script","noscript","style","textarea","pre","code","option"],n.ignoredClasses=n.ignoredClasses||[],n.errorCallback=n.errorCallback||console.error,n.macros=n.macros||{},s(e,n)}}(),o=o.default}()}));