* Fix betterproto2 migration: update generated proto code and callers
The dependency switch from betterproto to betterproto2 was incomplete.
This updates all affected files to use the betterproto2 API:
- tombstone.py: rewrite generated code to use betterproto2.field() with
explicit TYPE_* constants, repeated/optional/group flags, and map_meta()
for map fields
- tombstone_crashes.py: update import and fix to_dict() call to use
keyword-only casing= argument required by betterproto2
- pyproject.toml: replace betterproto[compiler] dev dep with betterproto2-compiler
- Makefile: update protoc plugin flag to --python_betterproto2_out
* Fix STIX2 hash key parsing to accept spec-compliant algorithm names
The STIX2 specification requires single quotes around hash algorithm
names that contain hyphens (e.g. file:hashes.'SHA-256'). MVT only
accepted a non-standard lowercase form (file:hashes.sha256), silently
dropping any indicators using the spec-correct spelling.
Normalize hash algorithm keys in _process_indicator by stripping quotes
and hyphens from the algorithm portion before matching, so all of the
following are accepted for SHA-256, SHA-1 and MD5:
file:hashes.'SHA-256' (STIX2 spec)
file:hashes.SHA-256
file:hashes.SHA256
file:hashes.sha256 (previously the only accepted form)
The same normalization is applied to app:cert.* keys.
Update generate_stix.py to use the spec-compliant quoted forms, and add
test_parse_stix2_hash_key_variants to cover all spelling variants.
* Run bugreport and backup modules during check-androidqf
Adding support to automatically run ADB backup and bugreport modules
automatically when running the check-androidqf command. This is a first
step to deduplicate the code for Android modules.
* Deduplicate modules which are run by the sub-commands.
* Raise the proper NoAndroidQFBackup exception when a back-up isn't found
* add missing import
* Fix imports and remove duplicate hashes param
* Rename from_folder to from_dir in tests
---------
Co-authored-by: besendorf <janik@besendorf.org>
* Use local timestamp for Files module timeline.
Most other Android timestamps appear to be local time. The
results timeline is more useful if all the timestamps
are consistent. I would prefer to use UTC, but that would
mean converting all the other timestamps to UTC as well. We probably
do not have sufficient information to do that accurately,
especially if the device is moving between timezones..
* Add file timestamp modules to add logs into timeline
* Handle case were we cannot load device timezone
* Fix crash if prop file does not exist
* Move _get_file_modification_time to BugReportModule
* Add backport for timezone and fix Tombstone module to use local time.
* Fix import for backported Zoneinfo
* Fix ruff error
* Add initial parser for ADB dumpsys
* Add ADBState tests and support for AndroidQF and
check-adb
* Handle case where ADB is not available in device dumpsys
* Add prelimary ipv4-addr ioc matching support under collection domains
* Add IP addresses as a valid IOC type
This currently just supports IPv4 addresses which
are treated as domains internally in MVT.
---------
Co-authored-by: renini <renini@local>
* Improves STIX2 support and testing
* Adds documentation on STIX2 support in MVT
---------
Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
Adds a custom JSON encoder class to fix serialisation issues where modules included bytes types containing non-utf8 bytes, which can't be serialised to JSON.
---------
Co-authored-by: Rory Flynn <rory.flynn@amnesty.org>
