Record the source of loaded modules for auditability

Now that installed module packages load automatically, record where every
module came from:

- --list-modules groups the available modules by source, one line per
  source with the modules comma-separated: MVT itself with its version,
  each installed package with its version and VCS commit when recorded
  (PEP 610 direct_url.json), and each --load-module/MVT_CUSTOM_MODULES
  file with its SHA-256 hash.
- Commands log one line per module source with its version or hash and
  the modules loaded from it, so command.log records exactly which
  modules ran and where they came from.
- Make init_logging() idempotent: a loaded module package importing an
  MVT CLI module would previously add a second console handler and
  duplicate every console log line.
This commit is contained in:
Donncha Ó Cearbhaill
2026-08-19 21:30:07 +02:00
parent cb25137423
commit e77008b49a
7 changed files with 271 additions and 4 deletions
+14
View File
@@ -160,6 +160,20 @@ For a `pipx` installation of MVT, inject the package into MVT's environment:
pipx inject mvt my-mvt-modules
```
## Auditing loaded modules
Because installed module packages load automatically, MVT records where every
module came from:
- `--list-modules` groups the available modules by source: MVT itself
(with its version), each installed package (with its version and, when
installed directly from a repository, the commit), and each file loaded
with `--load-module` or `MVT_CUSTOM_MODULES` (with the SHA-256 hash of the
file).
- When a command runs with an `--output` folder, the `command.log` file
records one line per module source with the source's version or hash and
the list of modules loaded from it.
## Profiling
Some MVT modules extract and process significant amounts of data during the analysis process or while checking results against known indicators. Care must be