mirror of
https://github.com/mvt-project/mvt.git
synced 2026-08-21 18:37:46 +02:00
Load installed module packages via entry points (#883)
* Load installed module packages via entry points Python packages can already register custom CLI commands which load automatically, but custom modules still require --load-module or the MVT_CUSTOM_MODULES environment variable on every invocation. Add an mvt.modules entry-point group so installed packages can register forensic modules which load automatically into every module-running check-* command. An entry point resolves to an iterable of MVTModule subclasses, or a callable returning one. Broken entry points are skipped with a warning so a faulty package cannot break MVT. * Record the source of loaded modules for auditability Now that installed module packages load automatically, record where every module came from: - --list-modules groups the available modules by source, one line per source with the modules comma-separated: MVT itself with its version, each installed package with its version and VCS commit when recorded (PEP 610 direct_url.json), and each --load-module/MVT_CUSTOM_MODULES file with its SHA-256 hash. - Commands log one line per module source with its version or hash and the modules loaded from it, so command.log records exactly which modules ran and where they came from. - Make init_logging() idempotent: a loaded module package importing an MVT CLI module would previously add a second console handler and duplicate every console log line. * Route loaded module logging under the mvt.ext namespace Modules loaded from installed packages or file paths live outside the mvt logger hierarchy, so their log records never reach MVT's console and file handlers and instead fall through to logging.lastResort: alerts print as bare unformatted lines and INFO messages are dropped entirely. Add get_module_logger() and use it everywhere module loggers are created. Built-in mvt.* modules keep their existing logger names, and everything external is parented under a dedicated mvt.ext namespace so records reach the handlers and external names can never collide with MVT's internal logger tree. File-path modules are named after their file (mvt.ext.<stem>) instead of the mangled internal import name. Document a naming convention for community module packages: distribute as mvt-plugin-<name> with import package mvt_plugin_<name>, including the publishing organization in the name. The prefix is advisory (loading is by entry point, and it is no mark of authenticity), but conforming packages get a cleaner logger namespace: the mvt_plugin_ prefix is stripped, so mvt_plugin_amnesty_custom logs as mvt.ext.amnesty_custom.
This commit is contained in:
@@ -117,6 +117,93 @@ class DependentCustomModule(MVTModule):
|
||||
self.results = [{"manifest_entries": len(manifest_results)}]
|
||||
```
|
||||
|
||||
## Installed module packages
|
||||
|
||||
Python packages can register modules so they load automatically in every
|
||||
module-running `check-*` command, without `--load-module` or
|
||||
`MVT_CUSTOM_MODULES`. Register an entry point in the `mvt.modules` group in
|
||||
the package's `pyproject.toml`:
|
||||
|
||||
```toml
|
||||
[project.entry-points."mvt.modules"]
|
||||
mvt-plugin-amnesty-custom = "mvt_plugin_amnesty_custom:get_modules"
|
||||
```
|
||||
|
||||
The entry point must resolve to an iterable of `MVTModule` subclasses, or to
|
||||
a callable returning one:
|
||||
|
||||
```python
|
||||
from mvt.common.module import MVTModule
|
||||
|
||||
|
||||
class PackagedModule(MVTModule):
|
||||
supported_commands = (("ios", "check-backup"),)
|
||||
|
||||
def run(self):
|
||||
self.results = [{"message": "packaged module ran"}]
|
||||
|
||||
|
||||
def get_modules() -> list[type[MVTModule]]:
|
||||
return [PackagedModule]
|
||||
```
|
||||
|
||||
Installed modules follow the same rules as other custom modules: each module
|
||||
must declare `supported_commands`, and dependencies are resolved with the
|
||||
standard ordering logic. A broken entry point is skipped with a warning and
|
||||
does not prevent MVT from running. As with custom commands, installed module
|
||||
packages run as trusted code inside the MVT process, so install only packages
|
||||
from sources you trust.
|
||||
|
||||
For a `pipx` installation of MVT, inject the package into MVT's environment:
|
||||
|
||||
```bash
|
||||
pipx inject mvt mvt-plugin-amnesty-custom
|
||||
```
|
||||
|
||||
### Naming module packages
|
||||
|
||||
Name module packages `mvt-plugin-<name>` (import package `mvt_plugin_<name>`),
|
||||
and include the name of the publishing organization or author so packages from
|
||||
different groups do not collide: for example, Amnesty International's custom
|
||||
modules would be distributed as `mvt-plugin-amnesty-custom` with the import
|
||||
package `mvt_plugin_amnesty_custom`.
|
||||
|
||||
The prefix makes module packages easy to find on PyPI and keeps their import
|
||||
names from clashing with unrelated Python packages. It is a convention, not a
|
||||
technical requirement: modules load through the `mvt.modules` entry point
|
||||
regardless of what the package is called, and existing packages with other
|
||||
names keep working. Note that the prefix is also not a mark of authenticity —
|
||||
anyone can publish a package with any available name, so vet a module package
|
||||
and its publisher before installing it, whatever it is called.
|
||||
|
||||
### Module logging
|
||||
|
||||
Modules log through `self.log`, and MVT names the logger for where the module
|
||||
came from. MVT's own modules log under their dotted path (for example
|
||||
`mvt.ios.modules.mixed.whatsapp`). Everything external is namespaced under
|
||||
`mvt.ext` to keep it visually distinct from built-in modules and isolated from
|
||||
MVT's internal logger tree:
|
||||
|
||||
- Installed packages log under `mvt.ext.<package>`, with the `mvt_plugin_`
|
||||
prefix stripped: modules in `mvt_plugin_amnesty_custom` log as
|
||||
`mvt.ext.amnesty_custom.*`.
|
||||
- Files loaded with `--load-module` or `MVT_CUSTOM_MODULES` log as
|
||||
`mvt.ext.<file name>`.
|
||||
|
||||
## Auditing loaded modules
|
||||
|
||||
Because installed module packages load automatically, MVT records where every
|
||||
module came from:
|
||||
|
||||
- `--list-modules` groups the available modules by source: MVT itself
|
||||
(with its version), each installed package (with its version and, when
|
||||
installed directly from a repository, the commit), and each file loaded
|
||||
with `--load-module` or `MVT_CUSTOM_MODULES` (with the SHA-256 hash of the
|
||||
file).
|
||||
- When a command runs with an `--output` folder, the `command.log` file
|
||||
records one line per module source with the source's version or hash and
|
||||
the list of modules loaded from it.
|
||||
|
||||
## Profiling
|
||||
|
||||
Some MVT modules extract and process significant amounts of data during the analysis process or while checking results against known indicators. Care must be
|
||||
|
||||
Reference in New Issue
Block a user