A module is part of check-iocs for a platform when it declares the
check-iocs pair, as before. It is now also part of check-iocs when it
overrides check_indicators() and supports at least one command of that
platform.
The rule lives in module_supports_command(). CmdCheckIOCS already uses
that function to pick its modules, so --list-modules and --module follow
it too. Built-in modules are unaffected. check-iocs takes them from
IOS_CHECK_IOCS_MODULES and ANDROID_CHECK_IOCS_MODULES.
A custom module which replaces a built-in module is covered by the same
rule. If it subclasses the module it replaces, it inherits its
check_indicators() and takes over the re-check of the results file. The
"Replacing a built-in module" section of the plugin documentation is
updated to say that.
A custom module which extends a built-in one ran alongside it, and both
wrote to the same results file when they shared a slug, with the run
order deciding the surviving content.
Custom modules can now name the module class they supersede in a
`replaces` attribute. When both are available to a command, the named
module is dropped from the run and the substitution is logged with the
origin of the replacement, so it is recorded in command.log. Only the
replacements which are applied are reported: a declaration from a
disabled module is ignored, so that replacing a module cannot silently
disable it, and modules which replace each other in a cycle all keep
running and replace nothing, with a warning naming every one of them.
Dependencies are remapped along with the modules themselves: a module
depending on a replaced class is ordered against, and receives the
results of, the module which took its place. Replacing a module which
others depend on therefore does not make that dependency unavailable.
The remapping applies wherever dependencies are read, so a replacement
which cannot run is skipped like any other module with an unavailable
dependency, and takes the modules depending on the module it replaced
with it. Those warnings name the dependency the author declared as well
as the module which replaces it.
A replacement does not have to keep the class name of the module it
replaces, so `--module` now falls back to the name of a replaced module
and runs its replacement. A name which matches no module at all stops
the run with a warning instead of silently analyzing nothing, as does a
selection left with nothing to run once skipped modules are dropped.
Sharing a slug outside a replacement stays possible and is now reported.
Two modules writing to the same results file is a forensic-integrity
problem rather than an error, so both still run and a warning names them,
where each came from, and the file the later one overwrites. Taking over
the slug of a replaced module is not reported, because that module is no
longer part of the run.
mvt.plugin re-exports the names a plugin needs from MVT under one import
path. It holds the module base classes and Command, the alert and result
types, the database errors a module raises, the timestamp converters, the
plugin settings API, MVT's settings, get_plugin_logger() and MVT_VERSION.
The names it exports are kept working on a best-effort basis. Changes to
them are announced in the release notes. Anything else in mvt can still be
imported, and may change between releases without notice.
get_plugin_logger(__name__) returns a logger under mvt.ext for plugin code
outside a module class. Its records then reach the console and the
command.log file of a run. A file loaded with --load-module or
--load-command is named after the file.
check-iocs takes its custom modules from load_custom_modules() like every
check-* command and matches result files to modules by slug, so a plugin
module's stored results are re-checked whenever it declares the
check-iocs pair of its platform; nothing asserted it.
check-iocs re-checks the results a previous run stored, so its module
list is every module of the platform that could have written one. Each
platform's CLI composed that list inline, concatenating the families by
hand, so the list existed only inside the click callback: anything else
needing to know what check-iocs runs had to build its own copy, and the
two could drift apart without a test noticing.
Give each platform a command_modules.py holding the one list, and have
its CLI assign it. The modules check-iocs runs are unchanged, and a
test pins each list to the families it is composed of.
set_verbose_logging() adjusted the first handler on the "mvt" logger,
whichever handler that happened to be. Anything else attaching a
handler to that logger - an embedding application, a plugin, a test
harness - had it mistaken for the console and raised or lowered behind
its back, and the same slot can hold the file handler a command
attaches to its output folder, whose level a --verbose flag should
never decide: command.log records the whole run either way.
Walk the handlers instead and adjust only MVT's own console handler,
found by its MVTLogHandler type. Every other handler on the logger is
left alone. Behaviour is otherwise unchanged.
Add a "plugins" command to the platform-neutral mvt command. "plugins
list" shows every installed plugin package with its version, where it was
installed from, how many forensic modules it contributes and which
commands it adds. "plugins check-updates" checks for updates immediately,
without waiting for the automatic check, and prints the command which
upgrades a plugin instead of installing anything.
It lives on mvt only. The packages it lists extend mvt-ios and mvt-android
too, but auditing them is not the job of a command which analyses one
platform, and the two platform CLIs should not carry commands which are
not about an acquisition.
The command is registered as a built-in, before any external command, so
that an installed package cannot replace this audit surface.
Report available updates to the installed MVT plugin packages in the
startup banner, for plugins installed from a package index and for
plugins installed directly from a repository. Repository installs pinned
to a commit or a tag are never reported as outdated.
MVT only prints the command which upgrades a plugin. Installing the
update stays a deliberate choice of the analyst. The check runs at most
once every twelve hours, and in between prints the findings of the
latest check which still apply to what is installed.
Nothing about the check can interrupt a running command: the parts of
the suggested command come from package metadata and are quoted for the
shell, the repository query refuses to prompt for credentials and never
passes metadata as a git option, and a corrupt or stale cache is
discarded rather than trusted.
Several MVT commands have nothing to do with the acquisition of one
platform, yet they were reachable only through mvt-ios and mvt-android.
Asking which version is installed or downloading the public indicators
meant picking one of the two platform commands arbitrarily, and each of
those tasks had to be written, documented and maintained twice.
Add a third console script, mvt, hosting the commands which belong to no
platform: version and download-iocs for now, with completion following
in a later commit.
Commands installed in the new mvt.cli_plugins entry-point group are
registered on mvt, and on mvt only, so that a command package chooses
the CLI each of its commands is added to: mvt.ios.cli_plugins for
mvt-ios, mvt.android.cli_plugins for mvt-android and mvt.cli_plugins for
mvt. A command wanted on both platform CLIs is registered in both
platform groups; no group adds a command to every CLI. The
MVT_CUSTOM_COMMANDS variable loads command files and folders into mvt
the way the platform variables already do for mvt-ios and mvt-android.
Run on its own, mvt prints the banner and its help instead of a usage
error. The help text reminds that the forensic analysis of an
acquisition runs through mvt-ios and mvt-android, so that the command
which knows nothing about acquisitions says where they are analysed.
version and download-iocs stay on mvt-ios and mvt-android for now, so
that no documented invocation stops working. They are to be dropped from
the platform CLIs in a later release, once mvt has been available long
enough for the change to be announced.
Unlike mvt-ios and mvt-android, which point at their subpackages, the
console script points at mvt.cli:main and the mvt package re-exports
nothing of it. Importing mvt has to stay cheap and free of side effects:
it is the package plugins import from, and pulling in Click, the CLI and
everything the commands import merely because something imported mvt
would work against that.
While here, give the version command of both platform CLIs the context
settings every other command already has, so that "mvt-ios version -h"
prints its help instead of failing on an unknown option.
A module declaring a dependency its command does not provide made
_ordered_modules() give up on the whole run, so a single wrong declaration
in a module scoped to several commands turned a forensic analysis into
zero executed modules with one warning to explain it.
Drop only the modules that cannot run: the one with the unavailable
dependency, and anything depending on it. Each gets its own warning naming
the module missing a dependency and the dependency it is missing, and the
remaining modules run in the same stable topological order as before. A
cycle in the dependency graph is still a programming error and still stops
the run.
A plugin with a settings class already names itself in `plugin_name`;
passing the name again to plugin_data_folder() repeats it and can drift.
Add a `data_folder()` class method on MVTPluginSettings which returns
plugin_data_folder() for the class's validated plugin name (works on the
class and on an instance); plugin_data_folder(name) stays as the function
underneath for plugins without a settings class.
Plugins had no sanctioned place to keep the data they persist, so the
plugin configuration documentation suggested a CACHE_FOLDER setting
defaulting to ~/.cache/example-plugin. That is a Linux convention which
is wrong on macOS, nothing expands or creates it, and it turns a path
into a setting a user can be asked to configure.
Add plugin_data_folder(), which returns the folder a plugin should use
for caches, downloaded artifacts, synchronization state or anything
else it writes to disk, and creates it if it is missing. The plugin
name is validated before anything is created, so a name holding a path
separator raises an error and leaves no folder behind, and calling the
function again returns the same folder with its contents untouched.
The folder sits under plugin-data rather than under the plugins folder
which holds the settings files. On macOS the configuration folder and
the data folder are the same directory, so reusing the plugins name
would leave each plugin's data folder in among the settings files.
Both the plugin-data folder and the folder of each plugin are created
with 0700 permissions. MVT is a forensic tool, and what a plugin keeps
there, such as API responses or sample metadata, is private by default.
The path is resolved on every call, as the configuration folder already
is, so it follows the current environment rather than whatever it was
when MVT was imported.
The documentation now points plugins at the helper, and the example
settings class carries a plain integer setting in place of its cache
folder.
Plugin packages need somewhere to keep their own settings, but MVT
rewrites its config.yaml with only the fields it knows about, so any
foreign section is dropped. Add MVTPluginSettings, a pydantic-settings
base class that gives each plugin its own file under the MVT config
folder and its own MVT_PLUGIN_<NAME>_ environment variable namespace.
Settings resolve from constructor arguments, then the environment, then
the plugin file, then the field defaults. Saving skips the values the
environment currently supplies, so credentials passed as environment
variables are not copied to disk, and writes through a private temporary
file so a settings file is never partially written or briefly readable
by other users.
* Load installed module packages via entry points
Python packages can already register custom CLI commands which load
automatically, but custom modules still require --load-module or the
MVT_CUSTOM_MODULES environment variable on every invocation.
Add an mvt.modules entry-point group so installed packages can register
forensic modules which load automatically into every module-running
check-* command. An entry point resolves to an iterable of MVTModule
subclasses, or a callable returning one. Broken entry points are
skipped with a warning so a faulty package cannot break MVT.
* Record the source of loaded modules for auditability
Now that installed module packages load automatically, record where every
module came from:
- --list-modules groups the available modules by source, one line per
source with the modules comma-separated: MVT itself with its version,
each installed package with its version and VCS commit when recorded
(PEP 610 direct_url.json), and each --load-module/MVT_CUSTOM_MODULES
file with its SHA-256 hash.
- Commands log one line per module source with its version or hash and
the modules loaded from it, so command.log records exactly which
modules ran and where they came from.
- Make init_logging() idempotent: a loaded module package importing an
MVT CLI module would previously add a second console handler and
duplicate every console log line.
* Route loaded module logging under the mvt.ext namespace
Modules loaded from installed packages or file paths live outside the
mvt logger hierarchy, so their log records never reach MVT's console
and file handlers and instead fall through to logging.lastResort:
alerts print as bare unformatted lines and INFO messages are dropped
entirely.
Add get_module_logger() and use it everywhere module loggers are
created. Built-in mvt.* modules keep their existing logger names, and
everything external is parented under a dedicated mvt.ext namespace so
records reach the handlers and external names can never collide with
MVT's internal logger tree. File-path modules are named after their
file (mvt.ext.<stem>) instead of the mangled internal import name.
Document a naming convention for community module packages:
distribute as mvt-plugin-<name> with import package mvt_plugin_<name>,
including the publishing organization in the name. The prefix is
advisory (loading is by entry point, and it is no mark of
authenticity), but conforming packages get a cleaner logger namespace:
the mvt_plugin_ prefix is stripped, so mvt_plugin_amnesty_custom logs
as mvt.ext.amnesty_custom.
* Run bugreport and backup modules during check-androidqf
Adding support to automatically run ADB backup and bugreport modules
automatically when running the check-androidqf command. This is a first
step to deduplicate the code for Android modules.
* Deduplicate modules which are run by the sub-commands.
* Raise the proper NoAndroidQFBackup exception when a back-up isn't found
* Remove check-adb command and update docs
* Remove check-apk code and old dependencies
* Major refactor to add structured alerting and typed indicators
This commit makes a structural change to MVT by changing binary
detected/not detected logic into a structured multi-level system
of alerts. This gives far more power to extend MVT and manage
alerts.
This commit also begins the process of adding proper typing for
key objects used in MVT including Indicators, IndicatorMatches,
and ModuleResults. This will also be keep to programmatically using
the output of MVT.
* Fix up, remove ADB module base
* Rework old detections tracking into stuctured alert levels
* Quote STIX path in log line
* Fix profile events log line
* Close open archive (zip/tar) file handles
* Fix root_binaries and mounts modules to use alertstore
* Update tests to use alertstore instead of detected attribute
* Fix alertstore method calls - use high() instead of warning()
* Fix remaining test errors
- Add log_latest() call in root_binaries to log each alert
- Fix UnboundLocalError in cmd_check_androidqf by initializing bugreport variable
- Remove incorrect backup.close() call since load_backup() returns bytes
- Remove duplicate from_ab method in cmd_check_backup that was using old attributes
* Log alerts on add
* Remove slug from alertstore calls
* update alerts.py
* update alerts.py
* move indicator_match to alert object
* .
* - Remove timeline_detected and route to alertstore
* fix typing for mypy
* Remove unused type imports
* Fix check_receiver_prefix and check_android_property_name
- check_receiver_prefix() used dict syntax (ioc["value"]) on Indicator
dataclass objects from get_iocs(). Changed to ioc.value/ioc.name.
- check_receiver_prefix() returned raw ioc instead of IndicatorMatch.
Now returns IndicatorMatch with descriptive message.
- Fixed return type annotations on both methods to Optional[IndicatorMatch].
- Removed unused Union import.
* Fix residual self.detected usage in packages and dumpsys_receivers
These modules still used self.detected.append() which no longer exists
after the alertstore migration. Converted to alertstore calls:
- packages.py: ROOT_PACKAGES detection → alertstore.high()
- dumpsys_receivers.py: receiver IOC match → alertstore.critical()
* Fix SMS module alertstore.high() call passing slug as message
The first argument was self.get_slug() (module slug) instead of a
human-readable message. The module is already auto-detected via
AlertStore._get_calling_module(). Also removed redundant log_latest().
* Apply suggestions from code review
Fix JSON serialization in `module.save_to_json` and fix argument order in iOS alertstore calls.
Co-authored-by: tes <tesitura@users.noreply.github.com>
* Remove unsupported ADB modules
* Fail removed check-adb command
* Fix alert serialization and logging
* Close sqlite connections in iOS modules
* Fix DEBUG messages not reaching handlers, save_to_json for dictionary results and TypeError on mixed event_time types in safary_history
* add matched_indicator via alertstore instead of directly modifying json objects
* Alert on battery daily uninstall and downgrade
* Lower alert severity to medium for suspicious items
* Switch version to 2026.4.28 CalVer
---------
Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
Co-authored-by: tes <tesitura@users.noreply.github.com>
Co-authored-by: Janik Besendorf <janik.besendorf@reporter-ohne-grenzen.de>
* Fix betterproto2 migration: update generated proto code and callers
The dependency switch from betterproto to betterproto2 was incomplete.
This updates all affected files to use the betterproto2 API:
- tombstone.py: rewrite generated code to use betterproto2.field() with
explicit TYPE_* constants, repeated/optional/group flags, and map_meta()
for map fields
- tombstone_crashes.py: update import and fix to_dict() call to use
keyword-only casing= argument required by betterproto2
- pyproject.toml: replace betterproto[compiler] dev dep with betterproto2-compiler
- Makefile: update protoc plugin flag to --python_betterproto2_out
* Fix STIX2 hash key parsing to accept spec-compliant algorithm names
The STIX2 specification requires single quotes around hash algorithm
names that contain hyphens (e.g. file:hashes.'SHA-256'). MVT only
accepted a non-standard lowercase form (file:hashes.sha256), silently
dropping any indicators using the spec-correct spelling.
Normalize hash algorithm keys in _process_indicator by stripping quotes
and hyphens from the algorithm portion before matching, so all of the
following are accepted for SHA-256, SHA-1 and MD5:
file:hashes.'SHA-256' (STIX2 spec)
file:hashes.SHA-256
file:hashes.SHA256
file:hashes.sha256 (previously the only accepted form)
The same normalization is applied to app:cert.* keys.
Update generate_stix.py to use the spec-compliant quoted forms, and add
test_parse_stix2_hash_key_variants to cover all spelling variants.
* Add initial parser for ADB dumpsys
* Add ADBState tests and support for AndroidQF and
check-adb
* Handle case where ADB is not available in device dumpsys
* Add prelimary ipv4-addr ioc matching support under collection domains
* Add IP addresses as a valid IOC type
This currently just supports IPv4 addresses which
are treated as domains internally in MVT.
---------
Co-authored-by: renini <renini@local>
* Improves STIX2 support and testing
* Adds documentation on STIX2 support in MVT
---------
Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
Adds a custom JSON encoder class to fix serialisation issues where modules included bytes types containing non-utf8 bytes, which can't be serialised to JSON.
---------
Co-authored-by: Rory Flynn <rory.flynn@amnesty.org>
Donncha Ó Cearbhaill