Compare commits

..
Author SHA1 Message Date
Donncha Ó Cearbhaill 377e2ee95a Merge branch 'main' into intrusion-logging 2026-05-12 17:22:19 +02:00
tes cc32370530 Merge commit from fork
file_path from main_entry.txt inside the bugreport zip is device controlled
and was used directly to open files on the host without validation.

Validate the resolved path stays within extract_path using
Path.resolve() + is_relative_to() before opening. Unsafe paths raise
ValueError and abort the operation.

Fixes GHSA-58fm-wv78-6929
2026-05-12 17:18:06 +02:00
tes 2a3423d826 Merge commit from fork
file_id values from backup manifests are device controlled and were used
directly to construct host filesystem paths without validation, allowing
a malicious backup to read or write files outside the designated directories.

Validate each file_id with Path.resolve() + is_relative_to() before use
as a source or destination path. Unsafe entries are skipped and logged.

Fixes GHSA-5h3g-px23-w6vw
2026-05-12 17:17:48 +02:00
tes 570749119d Rename intrusion logs folder from intrusion-logs to instrusion_logs to match AndroidQF output 2026-05-12 12:05:19 -03:00
Janik Besendorf 9b6cf82b34 Warn on unknown intrusion log event types 2026-05-12 11:47:19 +02:00
besendorf f405d38e68 Merge branch 'main' into intrusion-logging 2026-05-11 23:21:55 +02:00
besendorf 436a0dc7c0 Convert dependency management to uv (#785) 2026-05-11 23:21:40 +02:00
Janik Besendorf 758903811c Add Android intrusion log checks 2026-05-11 23:17:20 +02:00
besendorf c2b0b6db28 Revise breaking changes notice in README (#784)
Updated warning about breaking changes after merging the v3 branch.
2026-05-03 15:08:51 +02:00
c782d79974 V3 (#716)
* Run bugreport and backup modules during check-androidqf

Adding support to automatically run ADB backup and bugreport modules
automatically when running the check-androidqf command. This is a first
step to deduplicate the code for Android modules.

* Deduplicate modules which are run by the sub-commands.

* Raise the proper NoAndroidQFBackup exception when a back-up isn't found

* Remove check-adb command and update docs

* Remove check-apk code and old dependencies

* Major refactor to add structured alerting and typed indicators

This commit makes a structural change to MVT by changing binary
detected/not detected logic into a structured multi-level system
of alerts. This gives far more power to extend MVT and manage
alerts.

This commit also begins the process of adding proper typing for
key objects used in MVT including Indicators, IndicatorMatches,
and ModuleResults. This will also be keep to programmatically using
the output of MVT.

* Fix up, remove ADB module base

* Rework old detections tracking into stuctured alert levels

* Quote STIX path in log line

* Fix profile events log line

* Close open archive (zip/tar) file handles

* Fix root_binaries and mounts modules to use alertstore

* Update tests to use alertstore instead of detected attribute

* Fix alertstore method calls - use high() instead of warning()

* Fix remaining test errors

- Add log_latest() call in root_binaries to log each alert
- Fix UnboundLocalError in cmd_check_androidqf by initializing bugreport variable
- Remove incorrect backup.close() call since load_backup() returns bytes
- Remove duplicate from_ab method in cmd_check_backup that was using old attributes

* Log alerts on add

* Remove slug from alertstore calls

* update alerts.py

* update alerts.py

* move indicator_match to alert object

* .

* - Remove timeline_detected and route to alertstore

* fix typing for mypy

* Remove unused type imports

* Fix check_receiver_prefix and check_android_property_name

- check_receiver_prefix() used dict syntax (ioc["value"]) on Indicator
  dataclass objects from get_iocs(). Changed to ioc.value/ioc.name.
- check_receiver_prefix() returned raw ioc instead of IndicatorMatch.
  Now returns IndicatorMatch with descriptive message.
- Fixed return type annotations on both methods to Optional[IndicatorMatch].
- Removed unused Union import.

* Fix residual self.detected usage in packages and dumpsys_receivers

These modules still used self.detected.append() which no longer exists
after the alertstore migration. Converted to alertstore calls:
- packages.py: ROOT_PACKAGES detection → alertstore.high()
- dumpsys_receivers.py: receiver IOC match → alertstore.critical()

* Fix SMS module alertstore.high() call passing slug as message

The first argument was self.get_slug() (module slug) instead of a
human-readable message. The module is already auto-detected via
AlertStore._get_calling_module(). Also removed redundant log_latest().

* Apply suggestions from code review

Fix JSON serialization in `module.save_to_json` and fix argument order in iOS alertstore calls.

Co-authored-by: tes <tesitura@users.noreply.github.com>

* Remove unsupported ADB modules

* Fail removed check-adb command

* Fix alert serialization and logging

* Close sqlite connections in iOS modules

* Fix DEBUG messages not reaching handlers, save_to_json for dictionary results and TypeError on mixed event_time types in safary_history

* add matched_indicator via alertstore instead of directly modifying json objects

* Alert on battery daily uninstall and downgrade

* Lower alert severity to medium for suspicious items

* Switch version to 2026.4.28 CalVer

---------

Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
Co-authored-by: tes <tesitura@users.noreply.github.com>
Co-authored-by: Janik Besendorf <janik.besendorf@reporter-ohne-grenzen.de>
2026-04-29 14:32:29 +02:00
dependabot[bot] dad2a7a928 Bump mkdocs-material from 9.6.20 to 9.7.6 (#758)
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.20 to 9.7.6.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.6.20...9.7.6)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-version: 9.7.6
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-27 15:20:17 +02:00
dependabot[bot]andbesendorf 93df1eb1ea Bump mkdocstrings from 1.0.0 to 1.0.4 (#759)
Bumps [mkdocstrings](https://github.com/mkdocstrings/mkdocstrings) from 1.0.0 to 1.0.4.
- [Release notes](https://github.com/mkdocstrings/mkdocstrings/releases)
- [Changelog](https://github.com/mkdocstrings/mkdocstrings/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mkdocstrings/mkdocstrings/compare/1.0.0...1.0.4)

---
updated-dependencies:
- dependency-name: mkdocstrings
  dependency-version: 1.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-04-27 15:17:25 +02:00
dependabot[bot]andbesendorf 01157efbfa Bump mkdocs-autorefs from 1.4.3 to 1.4.4 (#776)
Bumps [mkdocs-autorefs](https://github.com/mkdocstrings/autorefs) from 1.4.3 to 1.4.4.
- [Release notes](https://github.com/mkdocstrings/autorefs/releases)
- [Changelog](https://github.com/mkdocstrings/autorefs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mkdocstrings/autorefs/compare/1.4.3...1.4.4)

---
updated-dependencies:
- dependency-name: mkdocs-autorefs
  dependency-version: 1.4.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-04-27 15:16:05 +02:00
0b612a9e39 Add new iOS versions and build numbers (#756)
Co-authored-by: DonnchaC <3081375+DonnchaC@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-04-27 15:07:30 +02:00
dependabot[bot] 28a32de72f Bump pydantic from 2.12.5 to 2.13.3 (#777)
Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.12.5 to 2.13.3.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](https://github.com/pydantic/pydantic/compare/v2.12.5...v2.13.3)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-version: 2.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-27 15:05:34 +02:00
dependabot[bot] 241a181de0 Bump cryptography from 46.0.6 to 47.0.0 (#775)
Bumps [cryptography](https://github.com/pyca/cryptography) from 46.0.6 to 47.0.0.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/46.0.6...47.0.0)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-version: 46.0.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-27 15:02:56 +02:00
besendorfandJanik Besendorf b6be706778 Abort analysis and warn user when backup is encrypted (#772)
* Abort analysis and warn user when backup is encrypted

When `check-backup` is run against an encrypted backup, Manifest.db
cannot be opened as a plain SQLite database. Previously this caused
a flood of confusing "file is not a database" errors across all modules.

Now the Manifest module detects the sqlite3.DatabaseError on its first
query and raises a new EncryptedBackupError. This exception propagates
out of run_module() and is caught in Command.run(), which logs a clear
critical message instructing the user to decrypt the backup first with
`mvt-ios decrypt-backup`, then stops the analysis immediately.

Fixes #769

* Fix ruff F821: use self.log instead of undefined log

---------

Co-authored-by: Janik Besendorf <janik.besendorf@reporter-ohne-grenzen.de>
2026-04-21 15:14:10 +02:00
besendorf f26303c930 Update README with warning about v3 breaking changes (#771)
Added important note about upcoming breaking changes in v3.
2026-04-12 09:54:29 +02:00
besendorfandJanik Besendorf 4edab3c4f8 handle empty sms databases (#770)
Co-authored-by: Janik Besendorf <janik.besendorf@reporter-ohne-grenzen.de>
2026-04-08 18:40:56 +02:00
besendorf d754f58c1a Update add-to-project action version (#768) 2026-04-07 20:52:01 +02:00
besendorf 6c537c624e Fix STIX2 hash key parsing to accept spec-compliant algorithm names (#767)
* Fix betterproto2 migration: update generated proto code and callers

The dependency switch from betterproto to betterproto2 was incomplete.
This updates all affected files to use the betterproto2 API:

- tombstone.py: rewrite generated code to use betterproto2.field() with
  explicit TYPE_* constants, repeated/optional/group flags, and map_meta()
  for map fields
- tombstone_crashes.py: update import and fix to_dict() call to use
  keyword-only casing= argument required by betterproto2
- pyproject.toml: replace betterproto[compiler] dev dep with betterproto2-compiler
- Makefile: update protoc plugin flag to --python_betterproto2_out

* Fix STIX2 hash key parsing to accept spec-compliant algorithm names

The STIX2 specification requires single quotes around hash algorithm
names that contain hyphens (e.g. file:hashes.'SHA-256'). MVT only
accepted a non-standard lowercase form (file:hashes.sha256), silently
dropping any indicators using the spec-correct spelling.

Normalize hash algorithm keys in _process_indicator by stripping quotes
and hyphens from the algorithm portion before matching, so all of the
following are accepted for SHA-256, SHA-1 and MD5:

  file:hashes.'SHA-256'   (STIX2 spec)
  file:hashes.SHA-256
  file:hashes.SHA256
  file:hashes.sha256      (previously the only accepted form)

The same normalization is applied to app:cert.* keys.

Update generate_stix.py to use the spec-compliant quoted forms, and add
test_parse_stix2_hash_key_variants to cover all spelling variants.
2026-04-07 20:41:40 +02:00
besendorf fd31f31aae Fix betterproto2 migration: update generated proto code and callers (#765)
The dependency switch from betterproto to betterproto2 was incomplete.
This updates all affected files to use the betterproto2 API:

- tombstone.py: rewrite generated code to use betterproto2.field() with
  explicit TYPE_* constants, repeated/optional/group flags, and map_meta()
  for map fields
- tombstone_crashes.py: update import and fix to_dict() call to use
  keyword-only casing= argument required by betterproto2
- pyproject.toml: replace betterproto[compiler] dev dep with betterproto2-compiler
- Makefile: update protoc plugin flag to --python_betterproto2_out
2026-04-07 14:09:38 +02:00
dependabot[bot] 9305c655bb Bump rich from 14.2.0 to 14.3.3 (#760)
Bumps [rich](https://github.com/Textualize/rich) from 14.2.0 to 14.3.3.
- [Release notes](https://github.com/Textualize/rich/releases)
- [Changelog](https://github.com/Textualize/rich/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Textualize/rich/compare/v14.2.0...v14.3.3)

---
updated-dependencies:
- dependency-name: rich
  dependency-version: 14.3.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-06 18:31:44 +02:00
dependabot[bot] ec43f93eb9 Bump click from 8.3.1 to 8.3.2 (#762)
Bumps [click](https://github.com/pallets/click) from 8.3.1 to 8.3.2.
- [Release notes](https://github.com/pallets/click/releases)
- [Changelog](https://github.com/pallets/click/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/click/compare/8.3.1...8.3.2)

---
updated-dependencies:
- dependency-name: click
  dependency-version: 8.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-06 18:29:22 +02:00
dependabot[bot]andbesendorf 7e398310b6 Bump tzdata from 2025.3 to 2026.1 (#761)
Bumps [tzdata](https://github.com/python/tzdata) from 2025.3 to 2026.1.
- [Release notes](https://github.com/python/tzdata/releases)
- [Changelog](https://github.com/python/tzdata/blob/master/NEWS.md)
- [Commits](https://github.com/python/tzdata/compare/2025.3...2026.1)

---
updated-dependencies:
- dependency-name: tzdata
  dependency-version: '2026.1'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-04-06 18:25:44 +02:00
besendorf 600e6dcf8f Replace betterproto with betterproto2 in dependencies (#763) 2026-04-06 18:23:33 +02:00
dependabot[bot]andbesendorf 3d1407b78c Bump rich from 14.1.0 to 14.2.0 (#728)
Bumps [rich](https://github.com/Textualize/rich) from 14.1.0 to 14.2.0.
- [Release notes](https://github.com/Textualize/rich/releases)
- [Changelog](https://github.com/Textualize/rich/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Textualize/rich/compare/v14.1.0...v14.2.0)

---
updated-dependencies:
- dependency-name: rich
  dependency-version: 14.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-04-06 17:27:15 +02:00
dependabot[bot] d41ff6d604 Bump cryptography from 46.0.5 to 46.0.6 (#755)
Bumps [cryptography](https://github.com/pyca/cryptography) from 46.0.5 to 46.0.6.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/46.0.5...46.0.6)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-version: 46.0.6
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-06 17:21:00 +02:00
Donncha Ó Cearbhaillandscribblemaniac ccd563f2ba Improve docker images tags based on PR #740 (#754)
* Modify docker image deployment behavior

* Use build-push-action to create image provenance & sbom attestations

* Upgrade github actions

* Fix inconsistent capitialization in dockerfiles

---------

Co-authored-by: scribblemaniac <scribblemaniac@users.noreply.github.com>
2026-04-01 18:42:30 +02:00
dependabot[bot]andbesendorf c681d264b3 Bump tzdata from 2025.2 to 2025.3 (#729)
Bumps [tzdata](https://github.com/python/tzdata) from 2025.2 to 2025.3.
- [Release notes](https://github.com/python/tzdata/releases)
- [Changelog](https://github.com/python/tzdata/blob/master/NEWS.md)
- [Commits](https://github.com/python/tzdata/compare/2025.2...2025.3)

---
updated-dependencies:
- dependency-name: tzdata
  dependency-version: '2025.3'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-04-01 10:57:30 +02:00
dependabot[bot]andbesendorf 261b0ae000 Bump pydantic-settings from 2.10.1 to 2.13.1 (#750)
Bumps [pydantic-settings](https://github.com/pydantic/pydantic-settings) from 2.10.1 to 2.13.1.
- [Release notes](https://github.com/pydantic/pydantic-settings/releases)
- [Commits](https://github.com/pydantic/pydantic-settings/compare/2.10.1...v2.13.1)

---
updated-dependencies:
- dependency-name: pydantic-settings
  dependency-version: 2.13.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-04-01 10:54:19 +02:00
dependabot[bot]andbesendorf 89d30e84f4 Bump packaging from 25.0 to 26.0 (#751)
Bumps [packaging](https://github.com/pypa/packaging) from 25.0 to 26.0.
- [Release notes](https://github.com/pypa/packaging/releases)
- [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pypa/packaging/compare/25.0...26.0)

---
updated-dependencies:
- dependency-name: packaging
  dependency-version: '26.0'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-04-01 10:51:50 +02:00
dependabot[bot] 557d0a0cd6 Bump requests from 2.32.5 to 2.33.1 (#752)
Bumps [requests](https://github.com/psf/requests) from 2.32.5 to 2.33.1.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.5...v2.33.1)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.33.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-01 10:48:45 +02:00
github-actions[bot]andDonnchaC 134bfce90f Add new iOS versions and build numbers (#743)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2026-03-26 12:16:59 -04:00
Tek 0141da4293 Fixes bug in IOC import (#749) 2026-03-25 23:23:08 +01:00
dependabot[bot]andbesendorf 5cba61b180 Bump mkdocstrings from 0.30.1 to 1.0.0 (#730)
Bumps [mkdocstrings](https://github.com/mkdocstrings/mkdocstrings) from 0.30.1 to 1.0.0.
- [Release notes](https://github.com/mkdocstrings/mkdocstrings/releases)
- [Changelog](https://github.com/mkdocstrings/mkdocstrings/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mkdocstrings/mkdocstrings/compare/0.30.1...1.0.0)

---
updated-dependencies:
- dependency-name: mkdocstrings
  dependency-version: 1.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-03-25 15:24:07 +01:00
dependabot[bot]andbesendorf 29475acb47 Bump click from 8.3.0 to 8.3.1 (#731)
Bumps [click](https://github.com/pallets/click) from 8.3.0 to 8.3.1.
- [Release notes](https://github.com/pallets/click/releases)
- [Changelog](https://github.com/pallets/click/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/click/compare/8.3.0...8.3.1)

---
updated-dependencies:
- dependency-name: click
  dependency-version: 8.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-03-25 14:37:59 +01:00
dependabot[bot]andbesendorf 1d5c83582c Bump pydantic from 2.12.3 to 2.12.5 (#732)
Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.12.3 to 2.12.5.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](https://github.com/pydantic/pydantic/compare/v2.12.3...v2.12.5)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-version: 2.12.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-03-25 14:26:09 +01:00
dependabot[bot] 2dd1428787 Bump cryptography from 46.0.3 to 46.0.5 (#747)
Bumps [cryptography](https://github.com/pyca/cryptography) from 46.0.3 to 46.0.5.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/46.0.3...46.0.5)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-version: 46.0.5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-25 08:49:46 +01:00
besendorf f2d9f420f2 Detect uninstall and downgrade in battery daily (#736) 2026-03-16 12:32:54 +01:00
github-actions[bot]andDonnchaC e2f8437831 Add new iOS versions and build numbers (#742)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2026-03-05 05:48:15 +01:00
github-actions[bot]andDonnchaC 0134bf80d1 Add new iOS versions and build numbers (#739)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2026-02-19 08:47:07 -05:00
Max-RSF c8f82f796b Add AQF support for bugreport modules (#741) 2026-02-16 17:11:16 +01:00
github-actions[bot] 61947d17af Add new iOS versions and build numbers (#738) 2026-02-04 20:51:11 +01:00
7173e02a6f Check receiver names for IoCs (#721)
* receiver names are checked if a known malicious app id is a substring

* ruff syntax fixes

---------

Co-authored-by: Viktor <vik@tor.me>
Co-authored-by: besendorf <janik@besendorf.org>
2026-01-10 15:24:20 +01:00
Donncha Ó Cearbhaill 8f34902bed Bump version for release v2.7.0 (#727) 2025-12-19 13:48:15 +01:00
Donncha Ó Cearbhaill 939bec82ff Fix Makefile and PyProtject config for current Ruff (#726) 2025-12-19 13:43:20 +01:00
dependabot[bot]andDonncha Ó Cearbhaill b183ca33b5 Bump click from 8.2.1 to 8.3.0 (#696)
Bumps [click](https://github.com/pallets/click) from 8.2.1 to 8.3.0.
- [Release notes](https://github.com/pallets/click/releases)
- [Changelog](https://github.com/pallets/click/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/click/compare/8.2.1...8.3.0)

---
updated-dependencies:
- dependency-name: click
  dependency-version: 8.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
2025-12-19 13:17:12 +01:00
dependabot[bot]andDonncha Ó Cearbhaill a2c9e0c6cf Bump simplejson from 3.20.1 to 3.20.2 (#699)
Bumps [simplejson](https://github.com/simplejson/simplejson) from 3.20.1 to 3.20.2.
- [Release notes](https://github.com/simplejson/simplejson/releases)
- [Changelog](https://github.com/simplejson/simplejson/blob/master/CHANGES.txt)
- [Commits](https://github.com/simplejson/simplejson/compare/v3.20.1...v3.20.2)

---
updated-dependencies:
- dependency-name: simplejson
  dependency-version: 3.20.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
2025-12-19 13:14:39 +01:00
Donncha Ó Cearbhaill 4bfad1f87d Fix outdated security contact point (#725) 2025-12-19 13:12:23 +01:00
dependabot[bot]andDonncha Ó Cearbhaill c3dc3d96d5 Bump cryptography from 45.0.6 to 46.0.3 (#709)
Bumps [cryptography](https://github.com/pyca/cryptography) from 45.0.6 to 46.0.3.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/45.0.6...46.0.3)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-version: 46.0.3
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
2025-12-19 13:09:59 +01:00
Donncha Ó Cearbhaill afab222f93 Run CI tests against Python3.14 too (#724)
Resolves #707
2025-12-19 12:54:29 +01:00
besendorfandDonncha Ó Cearbhaill 5a1166c416 Deprecate check-adb and recommend AndroidQF (#723)
Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
2025-12-19 12:44:43 +01:00
dependabot[bot]andDonncha Ó Cearbhaill dd3d665bea Bump requests from 2.32.4 to 2.32.5 (#684)
Bumps [requests](https://github.com/psf/requests) from 2.32.4 to 2.32.5.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.4...v2.32.5)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.32.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
2025-12-19 12:42:20 +01:00
dependabot[bot] 5c3b92aeee Bump pydantic from 2.11.7 to 2.12.3 (#708)
Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.11.7 to 2.12.3.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](https://github.com/pydantic/pydantic/compare/v2.11.7...v2.12.3)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-version: 2.12.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-17 19:28:36 +01:00
r-txandr-tx d7e058af43 add missing iPhone 16 and 17 models (#717)
Co-authored-by: r-tx <r-tx@users.noreply.github.com>
2025-12-15 09:48:11 +01:00
github-actions[bot]andDonnchaC cdbaad94cc Add new iOS versions and build numbers (#722)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-12-15 09:43:23 +01:00
github-actions[bot]andDonnchaC 981371bd8b Add new iOS versions and build numbers (#714)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-11-06 19:18:07 +01:00
github-actions[bot]andDonnchaC c7d00978c6 Add new iOS versions and build numbers (#712)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-11-04 19:28:19 +01:00
Donncha Ó Cearbhaillandbesendorf 339a1d0712 Deduplicate ADB AndroidQF and other modules (#606)
* Run bugreport and backup modules during check-androidqf

Adding support to automatically run ADB backup and bugreport modules
automatically when running the check-androidqf command. This is a first
step to deduplicate the code for Android modules.

* Deduplicate modules which are run by the sub-commands.

* Raise the proper NoAndroidQFBackup exception when a back-up isn't found

* add missing import

* Fix imports and remove duplicate hashes param

* Rename from_folder to from_dir in tests

---------

Co-authored-by: besendorf <janik@besendorf.org>
2025-10-31 13:46:33 +01:00
besendorf 7009cddc8c webkit session resource: fail gracefully when date conversion fails (#664)
* webkit session resource: fail gracefully when date conversion fails

* fix syntax
2025-10-23 15:19:08 +02:00
besendorf 9b4d10139c Add Options to disable update checks (#674)
* reduce update check timeouts to 5s

* add error hadnling for Update checks

* Add CLI flags to disable version and indicator checks

* ruff syntax fix

* fix tests
2025-10-23 15:13:36 +02:00
besendorfandUser b795ea3129 Add root_binaries androidqf module (#676)
* Add root_binaries androidqf module

* Fix AndroidQF file count test

* fix ruff

---------

Co-authored-by: User <user@DESKTOP-3T8T346.localdomain>
2025-10-23 15:12:01 +02:00
besendorf 5be5ffbf49 add mounts module for androidqf (#710)
* add mounts module for androidqf

* adds test for mounts module
2025-10-23 15:09:37 +02:00
besendorf 2701490501 fix tombstone unpack parsing bug (#711) 2025-10-23 15:08:01 +02:00
besendorf 779842567d Make revision field a string in TombstoneCrash model to fix error where (#702)
there were characters in the revision field
2025-10-09 11:28:47 +02:00
besendorf d3cc8cf590 Add tzdata dependency (#700)
* Add tzdata dependency

* fix tzdata name
2025-10-05 13:29:54 +02:00
github-actions[bot]andDonnchaC b8a42eaf8f Add new iOS versions and build numbers (#698)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-09-29 20:42:12 +02:00
dependabot[bot] 62b880fbff Bump mkdocstrings from 0.30.0 to 0.30.1 (#697)
Bumps [mkdocstrings](https://github.com/mkdocstrings/mkdocstrings) from 0.30.0 to 0.30.1.
- [Release notes](https://github.com/mkdocstrings/mkdocstrings/releases)
- [Changelog](https://github.com/mkdocstrings/mkdocstrings/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mkdocstrings/mkdocstrings/compare/0.30.0...0.30.1)

---
updated-dependencies:
- dependency-name: mkdocstrings
  dependency-version: 0.30.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-22 20:03:04 +02:00
besendorf 0778d448df make virustotal check also work with androidqf extractions (#685) 2025-09-19 07:31:17 +02:00
github-actions[bot]andDonnchaC f020655a1a Add new iOS versions and build numbers (#693)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-09-16 15:52:32 +02:00
github-actions[bot]andDonnchaC 91c34e6664 Add new iOS versions and build numbers (#692)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-09-15 20:13:40 +02:00
dependabot[bot] b4a8dd226a Bump mkdocs-material from 9.6.18 to 9.6.20 (#691)
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.18 to 9.6.20.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.6.18...9.6.20)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-version: 9.6.20
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-15 19:40:29 +02:00
dependabot[bot] 88213e12c9 Bump mkdocs-autorefs from 1.4.2 to 1.4.3 (#686)
Bumps [mkdocs-autorefs](https://github.com/mkdocstrings/autorefs) from 1.4.2 to 1.4.3.
- [Release notes](https://github.com/mkdocstrings/autorefs/releases)
- [Changelog](https://github.com/mkdocstrings/autorefs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mkdocstrings/autorefs/compare/1.4.2...1.4.3)

---
updated-dependencies:
- dependency-name: mkdocs-autorefs
  dependency-version: 1.4.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-05 18:30:36 +02:00
f75b8e186a add iOS 18.6.2 (#682)
* iOS 18.6.2

* iOS 18.6.2

---------

Co-authored-by: r-tx <r-tx@users.noreply.github.com>
Co-authored-by: Tek <tek@randhome.io>
2025-08-26 13:52:55 +02:00
dependabot[bot] 5babc1fcf3 Bump mkdocs-material from 9.6.17 to 9.6.18 (#683)
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.17 to 9.6.18.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.6.17...9.6.18)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-version: 9.6.18
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-26 11:25:22 +02:00
besendorf b723ebf28e move test dependencies to dev dependency group (#679) 2025-08-21 16:10:03 +02:00
dependabot[bot]andTek 616e870212 Bump mkdocs-material from 9.6.16 to 9.6.17 (#678)
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.16 to 9.6.17.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.6.16...9.6.17)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-version: 9.6.17
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tek <tek@randhome.io>
2025-08-20 11:13:59 +02:00
Tek 847b0e087b Adds iOS 18.6.1 (#681) 2025-08-20 11:10:20 +02:00
dependabot[bot] 86a0772eb2 Bump cryptography from 45.0.5 to 45.0.6 (#675)
Bumps [cryptography](https://github.com/pyca/cryptography) from 45.0.5 to 45.0.6.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/45.0.5...45.0.6)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-version: 45.0.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 10:38:19 +02:00
github-actions[bot]andDonnchaC 7d0be9db4f Add new iOS versions and build numbers (#673)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-07-31 13:20:34 +02:00
dependabot[bot] 4e120b2640 Bump pydantic-settings from 2.9.1 to 2.10.1 (#655)
Bumps [pydantic-settings](https://github.com/pydantic/pydantic-settings) from 2.9.1 to 2.10.1.
- [Release notes](https://github.com/pydantic/pydantic-settings/releases)
- [Commits](https://github.com/pydantic/pydantic-settings/compare/v2.9.1...2.10.1)

---
updated-dependencies:
- dependency-name: pydantic-settings
  dependency-version: 2.10.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-28 22:58:12 +02:00
dependabot[bot]andTek dbe9e5db9b Bump mkdocstrings from 0.29.1 to 0.30.0 (#671)
Bumps [mkdocstrings](https://github.com/mkdocstrings/mkdocstrings) from 0.29.1 to 0.30.0.
- [Release notes](https://github.com/mkdocstrings/mkdocstrings/releases)
- [Changelog](https://github.com/mkdocstrings/mkdocstrings/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mkdocstrings/mkdocstrings/compare/0.29.1...0.30.0)

---
updated-dependencies:
- dependency-name: mkdocstrings
  dependency-version: 0.30.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tek <tek@randhome.io>
2025-07-28 22:42:37 +02:00
dependabot[bot] 0b00398729 Bump rich from 14.0.0 to 14.1.0 (#670)
Bumps [rich](https://github.com/Textualize/rich) from 14.0.0 to 14.1.0.
- [Release notes](https://github.com/Textualize/rich/releases)
- [Changelog](https://github.com/Textualize/rich/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Textualize/rich/compare/v14.0.0...v14.1.0)

---
updated-dependencies:
- dependency-name: rich
  dependency-version: 14.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-28 22:37:42 +02:00
dependabot[bot] 87034d2c7a Bump mkdocs-material from 9.6.14 to 9.6.16 (#672)
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.14 to 9.6.16.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.6.14...9.6.16)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-version: 9.6.16
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-28 22:29:04 +02:00
besendorf 595a2f6536 Merge pull request #656 from mvt-project/fix/install_non_market_apps
remove deprecated install_non_market_apps permission check
2025-07-22 19:32:05 +02:00
besendorf 8ead44a31e Merge branch 'main' into fix/install_non_market_apps 2025-07-22 19:12:44 +02:00
besendorf 5c19d02a73 Merge pull request #659 from mvt-project/fix/tcc
fix #579 TCC: no such table: access
2025-07-22 19:02:32 +02:00
besendorf 14ebc9ee4e Merge branch 'main' into fix/tcc 2025-07-22 18:56:10 +02:00
besendorf de53cc07f8 Merge pull request #660 from mvt-project/fix/safari_browserstate
catch sqlite exception in safari_browserstate.py
2025-07-22 18:33:39 +02:00
besendorf 22e066fc4a Merge branch 'main' into fix/safari_browserstate 2025-07-22 18:20:07 +02:00
besendorf 242052b8ec Merge branch 'main' into fix/install_non_market_apps 2025-07-17 11:45:34 +02:00
dependabot[bot] 1df61b5bbf Bump cryptography from 45.0.4 to 45.0.5 (#661)
Bumps [cryptography](https://github.com/pyca/cryptography) from 45.0.4 to 45.0.5.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/45.0.4...45.0.5)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-version: 45.0.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-07 21:03:08 +02:00
besendorf b691de2cc0 catch sqlite exception in safari_browserstate.py 2025-07-04 17:52:05 +02:00
besendorf 10915f250c catch tcc error 2025-07-04 17:46:50 +02:00
besendorf c60cef4009 Merge branch 'main' into fix/install_non_market_apps 2025-07-04 17:04:13 +02:00
besendorf dda798df8e Merge pull request #658 from mvt-project/fix-mms
initialise message_links in backup parser to fix sms module bug
2025-07-04 15:32:47 +02:00
besendorf ffe6ad2014 initialise message_links in backup parser to fix sms module bug 2025-07-04 15:29:36 +02:00
dependabot[bot] a125b20fc5 Bump pydantic from 2.11.5 to 2.11.7 (#651)
Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.11.5 to 2.11.7.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](https://github.com/pydantic/pydantic/compare/v2.11.5...v2.11.7)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-version: 2.11.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-02 20:59:41 +02:00
besendorf 49108e67e2 remove deprecated install_non_market_apps permission check 2025-07-02 10:11:35 +02:00
dependabot[bot]andTek 883b450601 Bump mkdocstrings from 0.23.0 to 0.29.1 (#649)
Bumps [mkdocstrings](https://github.com/mkdocstrings/mkdocstrings) from 0.23.0 to 0.29.1.
- [Release notes](https://github.com/mkdocstrings/mkdocstrings/releases)
- [Changelog](https://github.com/mkdocstrings/mkdocstrings/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mkdocstrings/mkdocstrings/compare/0.23.0...0.29.1)

---
updated-dependencies:
- dependency-name: mkdocstrings
  dependency-version: 0.29.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tek <tek@randhome.io>
2025-06-20 11:29:34 +02:00
dependabot[bot] ce813568ff Bump mkdocs-autorefs from 1.2.0 to 1.4.2 (#648)
Bumps [mkdocs-autorefs](https://github.com/mkdocstrings/autorefs) from 1.2.0 to 1.4.2.
- [Release notes](https://github.com/mkdocstrings/autorefs/releases)
- [Changelog](https://github.com/mkdocstrings/autorefs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mkdocstrings/autorefs/compare/1.2.0...1.4.2)

---
updated-dependencies:
- dependency-name: mkdocs-autorefs
  dependency-version: 1.4.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-20 11:28:24 +02:00
dependabot[bot]andTek 93303f181a Bump mkdocs-material from 9.5.42 to 9.6.14 (#647)
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.42 to 9.6.14.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.5.42...9.6.14)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-version: 9.6.14
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tek <tek@randhome.io>
2025-06-20 11:24:12 +02:00
dependabot[bot] bee453a090 Bump cryptography from 45.0.3 to 45.0.4 (#645)
Bumps [cryptography](https://github.com/pyca/cryptography) from 45.0.3 to 45.0.4.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/45.0.3...45.0.4)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-version: 45.0.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-20 11:22:26 +02:00
dependabot[bot] 42106aa4d6 Bump pyahocorasick from 2.1.0 to 2.2.0 (#646)
Bumps [pyahocorasick](https://github.com/WojciechMula/pyahocorasick) from 2.1.0 to 2.2.0.
- [Release notes](https://github.com/WojciechMula/pyahocorasick/releases)
- [Changelog](https://github.com/WojciechMula/pyahocorasick/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/WojciechMula/pyahocorasick/compare/2.1.0...v2.2.0)

---
updated-dependencies:
- dependency-name: pyahocorasick
  dependency-version: 2.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-20 11:19:57 +02:00
Tek 95076c8f71 Create dependabot.yml (#644) 2025-06-20 11:17:40 +02:00
dependabot[bot] c9ac12f336 Bump requests from 2.32.2 to 2.32.4 (#642)
Bumps [requests](https://github.com/psf/requests) from 2.32.2 to 2.32.4.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.2...v2.32.4)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.32.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-19 23:55:42 +02:00
Tekandmakitos666 486e3e7e9b Update global_preferences.py (#641)
Added a warning for lockdown mode when the property is not present
---------

Co-authored-by: makitos666 <makitos666@gmail.com>
2025-06-19 23:51:59 +02:00
besendorf be1fc3bd8b Update NSKeyedUnarchiver (#636) 2025-06-12 22:42:02 +02:00
Tek 4757cff262 Fixes date parsing issue in tombstones (#635) 2025-06-12 20:49:31 +02:00
Tek 61f51caf31 Freeze versions and bump version (#632)
* Freeze versions and bump version
* Drops support for python below 3.10
2025-06-12 16:33:15 +02:00
besendorf 511063fd0e Update pyproject.toml (#630) 2025-06-04 13:00:04 +02:00
scribblemaniacandTek 88bc5672cb Upgrade main dockerfile runtime to ubuntu:24.04 (#619)
Co-authored-by: Tek <tek@randhome.io>
2025-05-14 11:34:40 +02:00
github-actions[bot]andDonnchaC 0fce0acf7a Add new iOS versions and build numbers (#626)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-05-14 11:12:13 +02:00
github-actions[bot]andDonnchaC 61f95d07d3 Add new iOS versions and build numbers (#625)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-05-12 22:37:46 +02:00
ping2A 3dedd169c4 Fix issue #574 for a module without IOCs output (#620)
* Fix issue #574 for a module without IOCs output
2025-04-30 10:30:39 +02:00
Tek e34e03d3a3 Fixes Android Dumpsys ADB parsing issue 2025-04-18 17:43:08 +02:00
github-actions[bot]andDonnchaC 34374699ce Add new iOS versions and build numbers (#622)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-04-17 09:46:17 +02:00
github-actions[bot]andDonnchaC cf5aa7c89f Add new iOS versions and build numbers (#618)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-04-01 16:04:06 +02:00
Donncha Ó Cearbhaill 2766739512 Fix bug where default values were dropped when parsing protobuf tombstones (#617) 2025-03-11 14:10:34 +01:00
cacu 9c84afb4b0 Update logo.py (#615)
add instructions to update mvt via pipx
2025-03-11 13:46:59 +01:00
Donncha Ó Cearbhaill 80fc8bd879 Fix YAML format (#611) 2025-02-21 15:48:00 +01:00
Donncha Ó Cearbhaill ca41f7f106 Always open automatic PRs as drafts (#609) 2025-02-21 15:35:06 +01:00
github-actions[bot]andDonnchaC 55ddd86ad5 Add new iOS versions and build numbers (#607)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-02-21 15:24:27 +01:00
156 changed files with 6622 additions and 2445 deletions

No files matched your search

+13
View File
@@ -0,0 +1,13 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
updates:
- package-ecosystem: "uv" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"
cooldown:
default-days: 7
+10 -8
View File
@@ -7,17 +7,19 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: 3.9
cache: 'pip'
- name: Checkout
uses: actions/checkout@master
uses: actions/checkout@v6
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: "3.10"
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
- name: Install Dependencies
run: |
pip install mypy
uv sync --locked --group dev
- name: mypy
run: |
make mypy
+29 -11
View File
@@ -4,6 +4,8 @@ name: Create and publish a Docker image
# Configures this workflow to run every time a release is published.
on:
workflow_dispatch:
push:
branches: [main]
release:
types: [published]
@@ -23,9 +25,18 @@ jobs:
attestations: write
id-token: write
#
strategy:
matrix:
platform:
- dockerfile: "Dockerfile"
tag-suffix: ""
- dockerfile: "Dockerfile.ios"
tag-suffix: "-ios"
- dockerfile: "Dockerfile.android"
tag-suffix: "-android"
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
# Uses the `docker/login-action` action to log in to the Container registry registry using the account and password that will publish the packages. Once published, the packages are scoped to the account defined here.
- name: Log in to the Container registry
uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
@@ -36,26 +47,33 @@ jobs:
# This step uses [docker/metadata-action](https://github.com/docker/metadata-action#about) to extract tags and labels that will be applied to the specified image. The `id` "meta" allows the output of this step to be referenced in a subsequent step. The `images` value provides the base name for the tags and labels.
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
flavor: |
latest=false
tags: |
type=raw,value=latest,enable={{ is_default_branch }},suffix=${{ matrix.platform.tag-suffix }}
type=raw,enable=${{ github.event_name == 'release' || github.ref_type == 'tag' }},value=stable,suffix=${{ matrix.platform.tag-suffix }}
type=raw,enable=${{ github.event_name == 'release' }},value=${{ github.event.release.tag_name }},suffix=${{ matrix.platform.tag-suffix }}
type=raw,enable=${{ github.ref_type == 'tag' }},value=${{ github.ref_name }},suffix=${{ matrix.platform.tag-suffix }}
type=sha,suffix=${{ matrix.platform.tag-suffix }}
type=sha,format=long,suffix=${{ matrix.platform.tag-suffix }}
# This step sets up some additional capabilities to generate the provenance and sbom attestations
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# This step uses the `docker/build-push-action` action to build the image, based on your repository's `Dockerfile`. If the build succeeds, it pushes the image to GitHub Packages.
# It uses the `context` parameter to define the build's context as the set of files located in the specified path. For more information, see "[Usage](https://github.com/docker/build-push-action#usage)" in the README of the `docker/build-push-action` repository.
# It uses the `tags` and `labels` parameters to tag and label the image with the output from the "meta" step.
- name: Build and push Docker image
id: push
uses: docker/build-push-action@f2a1d5e99d037542a71f64918e516c093c6f3fc4
uses: docker/build-push-action@v6
with:
file: ${{ matrix.platform.dockerfile }}
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
# This step generates an artifact attestation for the image, which is an unforgeable statement about where and how it was built. It increases supply chain security for people who consume the image. For more information, see "[AUTOTITLE](/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds)."
- name: Generate artifact attestation
uses: actions/attest-build-provenance@v1
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true
provenance: mode=max
sbom: true
+10 -8
View File
@@ -11,17 +11,19 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: 3.9
cache: 'pip'
- name: Checkout
uses: actions/checkout@master
uses: actions/checkout@v6
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: "3.10"
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
- name: Install Dependencies
run: |
pip install ruff
uv sync --locked --group dev
- name: ruff
run: |
make ruff
+9 -6
View File
@@ -12,18 +12,21 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ['3.8', '3.9', '3.10'] # , '3.11']
python-version: ['3.10', '3.11', '3.12', '3.13', '3.14']
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
- name: Install Python dependencies
run: |
make install
make test-requirements
uv sync --locked --group dev --python ${{ matrix.python-version }}
- name: Test with pytest
run: |
set -o pipefail
@@ -35,4 +38,4 @@ jobs:
if: github.event_name == 'pull_request'
with:
pytest-coverage-path: ./pytest-coverage.txt
junitxml-path: ./pytest.xml
junitxml-path: ./pytest.xml
+3 -2
View File
@@ -16,14 +16,15 @@ jobs:
- name: Run script to fetch latest iOS releases from Apple RSS feed.
run: python3 .github/workflows/scripts/update-ios-releases.py
- name: Create Pull Request
uses: peter-evans/create-pull-request@v5
uses: peter-evans/create-pull-request@v8
with:
title: '[auto] Update iOS releases and versions'
commit-message: Add new iOS versions and build numbers
branch: auto/add-new-ios-releases
draft: true
body: |
This is an automated pull request to update the iOS releases and version numbers.
add-paths: |
*.json
labels: |
automated pr
automated pr
+4 -1
View File
@@ -86,6 +86,9 @@ ipython_config.py
# pyenv
.python-version
# uv project Python version
!.python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
@@ -138,4 +141,4 @@ dmypy.json
.idea
# Sublime Text project files
*.sublime*
*.sublime*
+1
View File
@@ -0,0 +1 @@
3.10
+6 -4
View File
@@ -6,14 +6,16 @@
version: 2
build:
os: "ubuntu-22.04"
os: "ubuntu-24.04"
tools:
python: "3.11"
mkdocs:
configuration: mkdocs.yml
# Optionally set the version of Python and requirements required to build your docs
python:
install:
- requirements: docs/requirements.txt
install:
- method: uv
command: sync
groups:
- docs
+11 -15
View File
@@ -1,6 +1,6 @@
# Base image for building libraries
# ---------------------------------
FROM ubuntu:22.04 as build-base
FROM ubuntu:22.04 AS build-base
ARG DEBIAN_FRONTEND=noninteractive
@@ -22,7 +22,7 @@ RUN apt-get update \
# libplist
# --------
FROM build-base as build-libplist
FROM build-base AS build-libplist
# Build
RUN git clone https://github.com/libimobiledevice/libplist && cd libplist \
@@ -32,7 +32,7 @@ RUN git clone https://github.com/libimobiledevice/libplist && cd libplist \
# libimobiledevice-glue
# ---------------------
FROM build-base as build-libimobiledevice-glue
FROM build-base AS build-libimobiledevice-glue
# Install dependencies
COPY --from=build-libplist /build /
@@ -45,7 +45,7 @@ RUN git clone https://github.com/libimobiledevice/libimobiledevice-glue && cd li
# libtatsu
# --------
FROM build-base as build-libtatsu
FROM build-base AS build-libtatsu
# Install dependencies
COPY --from=build-libplist /build /
@@ -58,7 +58,7 @@ RUN git clone https://github.com/libimobiledevice/libtatsu && cd libtatsu \
# libusbmuxd
# ----------
FROM build-base as build-libusbmuxd
FROM build-base AS build-libusbmuxd
# Install dependencies
COPY --from=build-libplist /build /
@@ -72,7 +72,7 @@ RUN git clone https://github.com/libimobiledevice/libusbmuxd && cd libusbmuxd \
# libimobiledevice
# ----------------
FROM build-base as build-libimobiledevice
FROM build-base AS build-libimobiledevice
# Install dependencies
COPY --from=build-libplist /build /
@@ -88,7 +88,7 @@ RUN git clone https://github.com/libimobiledevice/libimobiledevice && cd libimob
# usbmuxd
# -------
FROM build-base as build-usbmuxd
FROM build-base AS build-usbmuxd
# Install dependencies
COPY --from=build-libplist /build /
@@ -103,7 +103,8 @@ RUN git clone https://github.com/libimobiledevice/usbmuxd && cd usbmuxd \
# Create main image
FROM ubuntu:22.04 as main
FROM ubuntu:24.04 AS main
COPY --from=ghcr.io/astral-sh/uv:0.11.8 /uv /uvx /usr/local/bin/
LABEL org.opencontainers.image.url="https://mvt.re"
LABEL org.opencontainers.image.documentation="https://docs.mvt.re"
@@ -133,13 +134,8 @@ COPY --from=build-usbmuxd /build /
# Install mvt using the locally checked out source
COPY . mvt/
RUN apt-get update \
&& apt-get install -y git python3-pip \
&& PIP_NO_CACHE_DIR=1 pip3 install --upgrade pip \
&& PIP_NO_CACHE_DIR=1 pip3 install ./mvt \
&& apt-get remove -y python3-pip git && apt-get autoremove -y \
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf mvt
RUN uv pip install --system --break-system-packages --no-cache ./mvt \
&& rm -rf mvt
# Installing ABE
ADD --checksum=sha256:a20e07f8b2ea47620aff0267f230c3f1f495f097081fd709eec51cf2a2e11632 \
+3 -2
View File
@@ -1,5 +1,6 @@
# Create main image
FROM python:3.10.14-alpine3.20 as main
FROM python:3.10.14-alpine3.20 AS main
COPY --from=ghcr.io/astral-sh/uv:0.11.8 /uv /uvx /usr/local/bin/
LABEL org.opencontainers.image.url="https://mvt.re"
LABEL org.opencontainers.image.documentation="https://docs.mvt.re"
@@ -20,7 +21,7 @@ RUN apk add --no-cache \
# Install mvt
COPY ./ mvt
RUN apk add --no-cache --virtual .build-deps gcc musl-dev \
&& PIP_NO_CACHE_DIR=1 pip3 install ./mvt \
&& uv pip install --system --no-cache ./mvt \
&& apk del .build-deps gcc musl-dev && rm -rf ./mvt
# Installing ABE
+10 -9
View File
@@ -1,6 +1,6 @@
# Base image for building libraries
# ---------------------------------
FROM ubuntu:22.04 as build-base
FROM ubuntu:22.04 AS build-base
ARG DEBIAN_FRONTEND=noninteractive
@@ -22,7 +22,7 @@ RUN apt-get update \
# libplist
# --------
FROM build-base as build-libplist
FROM build-base AS build-libplist
# Build
RUN git clone https://github.com/libimobiledevice/libplist && cd libplist \
@@ -32,7 +32,7 @@ RUN git clone https://github.com/libimobiledevice/libplist && cd libplist \
# libimobiledevice-glue
# ---------------------
FROM build-base as build-libimobiledevice-glue
FROM build-base AS build-libimobiledevice-glue
# Install dependencies
COPY --from=build-libplist /build /
@@ -45,7 +45,7 @@ RUN git clone https://github.com/libimobiledevice/libimobiledevice-glue && cd li
# libtatsu
# --------
FROM build-base as build-libtatsu
FROM build-base AS build-libtatsu
# Install dependencies
COPY --from=build-libplist /build /
@@ -58,7 +58,7 @@ RUN git clone https://github.com/libimobiledevice/libtatsu && cd libtatsu \
# libusbmuxd
# ----------
FROM build-base as build-libusbmuxd
FROM build-base AS build-libusbmuxd
# Install dependencies
COPY --from=build-libplist /build /
@@ -72,7 +72,7 @@ RUN git clone https://github.com/libimobiledevice/libusbmuxd && cd libusbmuxd \
# libimobiledevice
# ----------------
FROM build-base as build-libimobiledevice
FROM build-base AS build-libimobiledevice
# Install dependencies
COPY --from=build-libplist /build /
@@ -88,7 +88,7 @@ RUN git clone https://github.com/libimobiledevice/libimobiledevice && cd libimob
# usbmuxd
# -------
FROM build-base as build-usbmuxd
FROM build-base AS build-usbmuxd
# Install dependencies
COPY --from=build-libplist /build /
@@ -104,7 +104,8 @@ RUN git clone https://github.com/libimobiledevice/usbmuxd && cd usbmuxd \
# Main image
# ----------
FROM python:3.10.14-alpine3.20 as main
FROM python:3.10.14-alpine3.20 AS main
COPY --from=ghcr.io/astral-sh/uv:0.11.8 /uv /uvx /usr/local/bin/
LABEL org.opencontainers.image.url="https://mvt.re"
LABEL org.opencontainers.image.documentation="https://docs.mvt.re"
@@ -131,7 +132,7 @@ COPY --from=build-usbmuxd /build /
# Install mvt using the locally checked out source
COPY ./ mvt
RUN apk add --no-cache --virtual .build-deps git gcc musl-dev \
&& PIP_NO_CACHE_DIR=1 pip3 install ./mvt \
&& uv pip install --system --no-cache ./mvt \
&& apk del .build-deps git gcc musl-dev && rm -rf ./mvt
ENTRYPOINT [ "/usr/local/bin/mvt-ios" ]
+11 -16
View File
@@ -1,44 +1,39 @@
PWD = $(shell pwd)
autofix:
ruff format .
ruff check --fix .
UV ?= uv
check: ruff mypy
ruff:
ruff format --check .
ruff check -q .
$(UV) run ruff check .
mypy:
mypy
$(UV) run mypy
test:
python3 -m pytest
$(UV) run pytest
test-ci:
python3 -m pytest -v
$(UV) run pytest -v
install:
python3 -m pip install --upgrade -e .
$(UV) sync
test-requirements:
python3 -m pip install --upgrade -r test-requirements.txt
$(UV) sync --group dev
generate-proto-parsers:
# Generate python parsers for protobuf files
PROTO_FILES=$$(find src/mvt/android/parsers/proto/ -iname "*.proto"); \
protoc -Isrc/mvt/android/parsers/proto/ --python_betterproto_out=src/mvt/android/parsers/proto/ $$PROTO_FILES
$(UV) run protoc -Isrc/mvt/android/parsers/proto/ --python_betterproto2_out=src/mvt/android/parsers/proto/ $$PROTO_FILES
clean:
rm -rf $(PWD)/build $(PWD)/dist $(PWD)/src/mvt.egg-info
dist:
python3 -m pip install --upgrade build
python3 -m build
$(UV) build
upload:
python3 -m twine upload dist/*
$(UV) tool run twine upload dist/*
test-upload:
python3 -m twine upload --repository testpypi dist/*
$(UV) tool run twine upload --repository testpypi dist/*
+15
View File
@@ -4,6 +4,9 @@
# Mobile Verification Toolkit
> [!IMPORTANT]
> We recently merged the "v3" branch. This introduced breaking changes. If you relied on mvt output in other scripts They might have broken. More details: https://github.com/mvt-project/mvt/issues/757
[![](https://img.shields.io/pypi/v/mvt)](https://pypi.org/project/mvt/)
[![Documentation Status](https://readthedocs.org/projects/mvt/badge/?version=latest)](https://docs.mvt.re/en/latest/?badge=latest)
[![CI](https://github.com/mvt-project/mvt/actions/workflows/tests.yml/badge.svg)](https://github.com/mvt-project/mvt/actions/workflows/tests.yml)
@@ -38,6 +41,18 @@ MVT can be installed from sources or from [PyPI](https://pypi.org/project/mvt/)
pip3 install mvt
```
You can also install MVT from PyPI with [uv](https://docs.astral.sh/uv/). First, install uv:
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
Then install MVT as a command-line tool:
```bash
uv tool install mvt
```
For alternative installation options and known issues, please refer to the [documentation](https://docs.mvt.re/en/latest/install/) as well as [GitHub Issues](https://github.com/mvt-project/mvt/issues).
+58 -1
View File
@@ -2,4 +2,61 @@
Thank you for your interest in reporting security issues and vulnerabilities! Security research is of utmost importance and we take all reports seriously. If you discover an issue please report it to us right away!
Please DO NOT file a public issue, instead send your report privately to *nex [at] nex [dot] sx*. You can also write PGP-encrypted emails to [this key](https://keybase.io/nex/pgp_keys.asc?fingerprint=05216f3b86848a303c2fe37dd166f1667359d880).
Please DO NOT file a public issue, instead send your report privately to the MVT maintainers at Amnesty International via `security [at] amnesty [dot] tech`.
You can also write PGP-encrypted emails to key `CFBF9698DCA8EB2A80F48ADEA035A030FA04ED13`. The corresponding PGP public key is lited below.
```
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBGlFPwsBEADQ+d7SeHrFPYv3wPOjWs2oMpp0DPdfIyGbg+iYWOC36FegZhKY
+WeK96GqJWt8wD6kwFUVwQI795WZrjSd1q4a7wR+kj/h7xlRB6ZfVICA6O5DOOm6
GNMvqy7ESm8g1XZDpb2u1BXmSS9X8f6rjB0e86kYsF1mB5/2USTM63jgDs0GGTkZ
Q1z4Mq4gYyqH32b3gvXkbb68LeQmONUIM3cgmec9q8/pNc1l7fcoLWhOVADRj17Q
plisa/EUf/SYqdtk9w7EHGggNenKNwVM235mkPcMqmE72bTpjT6XCxvZY3ByG5yi
7L+tHJU45ZuXtt62EvX03azxThVfSmH/WbRk8lH8+CW8XMmiWZphG4ydPWqgVKCB
2UOXm+6CQnKA+7Dt1AeK2t5ciATrv9LvwgSxk5WKc3288XFLA6eGMrTdQygYlLjJ
+42RSdK/7fCt/qk4q13oUw8ZTVcCia98uZFi704XuuYTH6NrntIB7j/0oucIS4Y9
cTWNO5LBerez4v8VI4YHcYESPeIWGFkXhvJzo0VMg1zidBLtiPoGF2JKZGwaK7/p
yY1xALskLp4H+5OY4eB1kf8kl4vGsEK8xA/NNzOiapVmwBXpvVvmXIQJE2k+olNf
sAuyB8+aO1Ws7tFYt3D+olC7iaprOdK7uA4GCgmYYhq6QQPg+cxfczgHfwARAQAB
tD1TZWN1cml0eSBMYWIgYXQgQW1uZXN0eSBJbnRlcm5hdGlvbmFsIDxzZWN1cml0
eUBhbW5lc3R5LnRlY2g+iQJRBBMBCAA7FiEEz7+WmNyo6yqA9IreoDWgMPoE7RMF
AmlFPwsCGwMFCwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQoDWgMPoE7RNr
2w//a88uP90uSN6lgeIwKsHr1ri27QIBbzCV6hLN/gZBFR2uaiOn/xfFDbnR0Cjo
5nMCJCT1k4nrPbMTlfmWLCD+YKELBzVqWlw4J2SOg3nznPl2JrL8QBKjwts0sF+h
QbRWDsT54wBZnl6ZJJ79eLShNTokBbKnQ7071dMrENr5e2P2sClQXyiIc51ga4FM
fHyhsx+GsrdiZNd2AH8912ljW1GuEi3epTO7KMZprmr37mjpZSUToiV59Yhl1Gbo
2pixkYJqi62DG02/gTpCjq9NH3cEMxcxjh4E7yCA8ggLG6+IN6woIvPIdOsnQ+Yj
d3H4rMNBjPSKoL+bdHILkCnp5HokcbVjNY3QAyOAF4qWhk4GtgpTshwxUmb4Tbay
tWLJC2bzjuUBxLkGzMVFfU3B96sVS4Fi0sBaEMBtHskl2f45X8LJhSq//Lw/2L/8
34uP/RxDSn+DPvj/yqMpekdCcmeFSTX1A19xkPcc0rVhMRde4VL338R86vzh0gMI
1LySDAhXZyVWzrQ5s3n6N3EvCaHCn3qu7ieyFJifCSR7gZqevCEznMQRVpkMTzUt
rk13Z6NOOb4IlTW7HFoY3omJG8Z5jV4kMIE7n6nb0qpNYQiG+YvjenQ3VrMoISyh
lpS2De8+oOtwrxBVX3+qKWvQqzufeE3416kw2Z+5mxH7bx25Ag0EaUU/CwEQALyZ
b+kwLN1yHObTm2yDBEn5HbCT3H1GremvPNmbAaTnfrjUngoKa8MuWWzbX5ptgmZR
UpYY/ylOYcgGydz58vUNrPlhIZT9UhmiifPgZLEXyd0uFpr/NsbRajHMkK10iEZf
h5bHNobiB7pGCu4Uj9e1cMiIZ4yEaYeyXYUoNHf6ISP39mJhHy6ov5yIpm9q0wzm
tGUQPupxGXmEZlOPr3lxqXQ3Ekdv6cWDY5r/oOq71QJ/HUQ13QUuGFIbhnMbT8zd
zaS6f/v772YKsWPc4NNUhtlf25VnQ4FuUtjCe3p6iYP4OVD8gJm0GvXyvyTuiQbL
CSk/378JiNT7nZzYXxrWchMwvEoMIU55+/UaBc50HI5xvDQ858CX7PYGiimcdsO1
EkQzhVxRfjlILfWrC2lgt+H5qhTn4Fah250Xe1PnLjXGHVUQnY/f3MFeiWQgf92b
02+MfvOeC5OKttP1z5lcx6RFWCIa1E/u8Nj7YrH9hk0ZBRAnBaeAncDFY8dfX2zX
VMoc0dV16gM7RrZ6i7D3CG3eLLkQlX0jbW9dzTuG/3f098EWB1p8vOfS/RbNCBRX
jqGiqacL/aFF3Ci3nQ4O5tSv1XipbgrUhvXnwm9pxrLPS/45iaO59WN4RRGWLLQ7
LHmeBxoa9avv0SdBYUL+eBxY46GXb/j5VLzHYhSnABEBAAGJAjYEGAEIACAWIQTP
v5aY3KjrKoD0it6gNaAw+gTtEwUCaUU/CwIbDAAKCRCgNaAw+gTtEyvsEACnyFFD
alOZTrrJTXNnUejuiExLh+qTO3T91p5bte597jpwCZnYGwkxEfffsqqhlY6ftEOf
d5tNWE5isai4v8XCbplWomz4KBpepxcn2b+9o5dSyr1vohEFuCJziZDsta1J2DX5
IE9U48kTgLDfdIBhuOyHNRkvXRHP2OVLCaiw4d9q+hlrraR8pehHt2BJSxh+QZoe
n0iHvIZCBIUA45zLEGmXFpNTGeEf2dKPp3xOkAXOhAMPptE0V1itkF3R7kEW4aFO
SZo8L3C1aWSz/gQ4/vvW5t1IJxirNMUgTMQFvqEkAwX3fm6GCxlgRSvTTRXdcrS8
6qyFdH1nkCNsavPahN3N2RGGIlWtODEMTO1Hjy0kZtTYdW+JH9sendliCoJES+yN
DjM125SgdAgrqlSYm/g8n9knWpxZv1QM6jU/sVz1J+l6/ixugL2i+CAL2d6uv4tT
QmXnu7Ei4/2kHBUu3Lf59MNgmLHm6F7AhOWErszSeoJKsp+3yA1oTT/npz67sRzY
VVyxz4NBIollna59a1lz0RhlWzNKqNB27jhylyM4ltdzHB7r4VMAVJyttozmIIOC
35ucYxl5BHLuapaRSaYHdUId1LOccYyaOOFF/PSyCu9dKzXk7zEz2HNcIboWSkAE
8ZDExMYM4WVpVCOj+frdsaBvzItHacRWuijtkw==
=JAXX
-----END PGP PUBLIC KEY BLOCK-----
```
+19 -16
View File
@@ -1,26 +1,29 @@
# Deprecation of ADB command in MVT
# ADB Analysis Removed
!!! warning
The ability to analyze Android devices directly over the [Android Debug Bridge (adb)](https://developer.android.com/studio/command-line/adb) has been removed from MVT.
The `mvt-android check-adb` command has been deprecated and removed from MVT.
Use [AndroidQF](https://github.com/mvt-project/androidqf) to collect forensic artifacts from an Android device, then analyze the collected output with:
The ability to analyze Android devices over ADB (`mvt-android check-adb`) has been removed from MVT due to several technical and forensic limitations.
```bash
mvt-android check-androidqf /path/to/androidqf-output
```
## Reasons for Deprecation
For a standalone Android bug report, use:
```bash
mvt-android check-bugreport /path/to/bugreport.zip
```
## Reasons for Removal
1. **Inconsistent Data Collection Across Devices**
Android devices vary significantly in their system architecture, security policies, and available diagnostic logs. This inconsistency makes it difficult to ensure that MVT can reliably collect necessary forensic data across all devices.
Android devices vary significantly in system architecture, security policy, and diagnostic log availability. This made it difficult to collect reliable forensic data across devices.
2. **Incomplete Forensic Data Acquisition**
The `check-adb` command did not retrieve a full forensic snapshot of all available data on the device. For example, critical logs such as the **full bugreport** were not systematically collected, leading to potential gaps in forensic analysis. This can be a serious problem in scenarios where the analyst only had one time access to the Android device.
Direct ADB analysis did not retrieve a complete forensic snapshot. Critical artifacts such as full bug reports could be missing.
4. **Code Duplication and Difficulty Ensuring Consistent Behavior Across Sources**
Similar forensic data such as "dumpsys" logs were being loaded and parsed by MVT's ADB, AndroidQF and Bugreport commands. Multiple modules were needed to handle each source format which created duplication leading to inconsistent
behavior and difficulties in maintaining the code base.
3. **Duplicated Analysis Paths**
Similar artifacts were parsed through separate ADB, AndroidQF, and bugreport modules, which made behavior harder to keep consistent.
5. **Alignment with iOS Workflow**
MVT’s forensic workflow for iOS relies on pre-extracted artifacts, such as iTunes backups or filesystem dumps, rather than preforming commands or interactions directly on a live device. Removing the ADB functionality ensures a more consistent methodology across both Android and iOS mobile forensic.
## Alternative: Using AndroidQF for Forensic Data Collection
To replace the deprecated ADB-based approach, forensic analysts should use [AndroidQF](https://github.com/mvt-project/androidqf) for comprehensive data collection, followed by MVT for forensic analysis. The workflow is outlined in the MVT [Android methodology](./methodology.md)
4. **Workflow Consistency**
MVT now focuses on analyzing acquired artifacts rather than interacting with live devices directly.
-28
View File
@@ -1,28 +0,0 @@
# Downloading APKs from an Android phone
MVT allows you to attempt to download all available installed packages (APKs) from a device in order to further inspect them and potentially identify any which might be malicious in nature.
You can do so by launching the following command:
```bash
mvt-android download-apks --output /path/to/folder
```
It might take several minutes to complete.
!!! info
MVT will likely warn you it was unable to download certain installed packages. There is no reason to be alarmed: this is typically expected behavior when MVT attempts to download a system package it has no privileges to access.
Optionally, you can decide to enable lookups of the SHA256 hash of all the extracted APKs on [VirusTotal](https://www.virustotal.com). While these lookups do not provide any conclusive assessment on all of the extracted APKs, they might highlight any known malicious ones:
```bash
MVT_VT_API_KEY=<key> mvt-android download-apks --output /path/to/folder --virustotal
```
Please note that in order to use VirusTotal lookups you are required to provide your own API key through the `MVT_VT_API_KEY` environment variable. You should also note that VirusTotal enforces strict API usage. Be mindful that MVT might consume your hourly search quota.
In case you have a previous extraction of APKs you want to later check against VirusTotal, you can do so with the following arguments:
```bash
MVT_VT_API_KEY=<key> mvt-android download-apks --from-file /path/to/folder/apks.json --virustotal
```
+2 -4
View File
@@ -38,11 +38,9 @@ By separating artifact collection from forensic analysis, this approach ensures
For more information, refer to the [AndroidQF project documentation](https://github.com/mvt-project/androidqf).
## Check the device over Android Debug Bridge
## Android Debug Bridge analysis removed
The ability to analyze Android devices over ADB (`mvt-android check-adb`) has been removed from MVT.
See the [Android ADB documentation](./adb.md) for more information.
The ability to analyze Android devices directly over ADB has been removed from MVT. Use AndroidQF for device acquisition and `mvt-android check-androidqf` for analysis.
## Check an Android Backup (SMS messages)
+10 -2
View File
@@ -2,9 +2,17 @@
The Mobile Verification Toolkit team welcomes contributions of new forensic modules or other contributions which help improve the software.
## Local environment
MVT uses `uv` for dependency management. To install the project and development dependencies from the locked environment, run:
```bash
make install
```
## Testing
MVT uses `pytest` for unit and integration tests. Code style consistency is maintained with `flake8`, `ruff` and `black`. All can
MVT uses `pytest` for unit and integration tests. Code style consistency is maintained with `ruff` and `mypy`. All can
be run automatically with:
```bash
@@ -24,4 +32,4 @@ MVT modules can be profiled with Python built-in `cProfile` by setting the `MVT_
MVT_PROFILE=1 dev/mvt-ios check-backup test_backup
```
Open an issue or PR if you are encountering significant performance issues when analyzing a device with MVT.
Open an issue or PR if you are encountering significant performance issues when analyzing a device with MVT.
-5
View File
@@ -1,5 +0,0 @@
mkdocs==1.6.1
mkdocs-autorefs==1.2.0
mkdocs-material==9.5.42
mkdocs-material-extensions==1.3.1
mkdocstrings==0.23.0
-2
View File
@@ -42,9 +42,7 @@ nav:
- Records extracted by mvt-ios: "ios/records.md"
- MVT for Android:
- Android Forensic Methodology: "android/methodology.md"
- Check over ADB: "android/adb.md"
- Check an Android Backup (SMS messages): "android/backup.md"
- Download APKs: "android/download_apks.md"
- Indicators of Compromise: "iocs.md"
- Development: "development.md"
- License: "license.md"
+62 -42
View File
@@ -1,13 +1,11 @@
[project]
name = "mvt"
dynamic = ["version"]
authors = [
{name = "Claudio Guarnieri", email = "nex@nex.sx"}
]
authors = [{ name = "Claudio Guarnieri", email = "nex@nex.sx" }]
maintainers = [
{name = "Etienne Maynier", email = "tek@randhome.io"},
{name = "Donncha Ó Cearbhaill", email = "donncha.ocearbhaill@amnesty.org"},
{name = "Rory Flynn", email = "rory.flynn@amnesty.org"}
{ name = "Etienne Maynier", email = "tek@randhome.io" },
{ name = "Donncha Ó Cearbhaill", email = "donncha.ocearbhaill@amnesty.org" },
{ name = "Rory Flynn", email = "rory.flynn@amnesty.org" },
]
description = "Mobile Verification Toolkit"
readme = "README.md"
@@ -16,64 +14,86 @@ classifiers = [
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Information Technology",
"Operating System :: OS Independent",
"Programming Language :: Python"
"Programming Language :: Python",
]
dependencies = [
"click >=8.1.3",
"rich >=12.6.0",
"tld >=0.12.6",
"requests >=2.28.1",
"simplejson >=3.17.6",
"packaging >=21.3",
"appdirs >=1.4.4",
"iOSbackup >=0.9.923",
"cryptography >=42.0.5",
"pyyaml >=6.0",
"pyahocorasick >= 2.0.0",
"betterproto >=1.2.0",
"pydantic >= 2.10.0",
"pydantic-settings >= 2.7.0",
'backports.zoneinfo; python_version < "3.9"',
"click==8.3.2",
"rich==14.3.3",
"tld==0.13.1",
"requests==2.33.1",
"simplejson==3.20.2",
"packaging==26.0",
"appdirs==1.4.4",
"iOSbackup==0.9.925",
"adb-shell[usb]==0.4.4",
"libusb1==3.3.1",
"cryptography==47.0.0",
"PyYAML>=6.0.2",
"pyahocorasick==2.2.0",
"betterproto2==0.9.1",
"pydantic==2.13.3",
"pydantic-settings==2.13.1",
"NSKeyedUnArchiver==1.5.2",
"python-dateutil==2.9.0.post0",
"tzdata==2026.1",
]
requires-python = ">= 3.8"
requires-python = ">= 3.10"
[project.urls]
homepage = "https://docs.mvt.re/en/latest/"
repository = "https://github.com/mvt-project/mvt"
[project.scripts]
mvt-ios = "mvt.ios:cli"
mvt-android = "mvt.android:cli"
mvt-ios = "mvt.ios:cli"
mvt-android = "mvt.android:cli"
[dependency-groups]
dev = [
"requests>=2.31.0",
"pytest>=7.4.3",
"pytest-cov>=4.1.0",
"pytest-github-actions-annotate-failures>=0.2.0",
"pytest-mock>=3.14.0",
"stix2>=3.0.1",
"ruff>=0.1.6",
"mypy>=1.7.1",
"betterproto2-compiler",
"types-PyYAML>=6.0.12.20250516",
"types-python-dateutil>=2.9.0.20250822",
"types-requests>=2.32.4.20250913",
]
docs = [
"mkdocs==1.6.1",
"mkdocs-autorefs==1.4.4",
"mkdocs-material==9.7.6",
"mkdocs-material-extensions==1.3.1",
"mkdocstrings==1.0.4",
]
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[tool.coverage.run]
omit = [
"tests/*",
]
omit = ["tests/*"]
[tool.coverage.html]
directory= "htmlcov"
directory = "htmlcov"
[tool.mypy]
install_types = true
non_interactive = true
install_types = false
ignore_missing_imports = true
packages = "src"
[tool.pytest.ini_options]
addopts = "-ra -q --cov=mvt --cov-report html --junitxml=pytest.xml --cov-report=term-missing:skip-covered"
testpaths = [
"tests"
]
testpaths = ["tests"]
[tool.ruff.lint]
select = ["C90", "E", "F", "W"] # flake8 default set
ignore = [
"E501", # don't enforce line length violations
"C901", # complex-structure
[tool.ruff]
lint.select = ["C90", "E", "F", "W"] # flake8 default set
lint.ignore = [
"E501", # don't enforce line length violations
"C901", # complex-structure
# These were previously ignored but don't seem to be required:
# "E265", # no-space-after-block-comment
@@ -85,14 +105,14 @@ ignore = [
]
[tool.ruff.lint.per-file-ignores]
"__init__.py" = ["F401"] # unused-import
"__init__.py" = ["F401"] # unused-import
[tool.ruff.lint.mccabe]
max-complexity = 10
[tool.setuptools]
include-package-data = true
package-dir = {"" = "src"}
package-dir = { "" = "src" }
[tool.setuptools.packages.find]
where = ["src"]
@@ -101,4 +121,4 @@ where = ["src"]
mvt = ["ios/data/*.json"]
[tool.setuptools.dynamic]
version = {attr = "mvt.common.version.MVT_VERSION"}
version = { attr = "mvt.common.version.MVT_VERSION" }
+29 -13
View File
@@ -20,23 +20,39 @@ class AndroidArtifact(Artifact):
:param binary: whether the dumpsys should be pared as binary or not (bool)
:return: section extracted (string or bytes)
"""
lines = []
in_section = False
delimiter = "------------------------------------------------------------------------------"
delimiter_str = "------------------------------------------------------------------------------"
delimiter_bytes = b"------------------------------------------------------------------------------"
if binary:
delimiter = delimiter.encode("utf-8")
lines_bytes = []
for line in dumpsys.splitlines(): # type: ignore[union-attr]
if line.strip() == separator: # type: ignore[arg-type]
in_section = True
continue
for line in dumpsys.splitlines():
if line.strip() == separator:
in_section = True
continue
if not in_section:
continue
if not in_section:
continue
if line.strip().startswith(delimiter_bytes): # type: ignore[arg-type]
break
if line.strip().startswith(delimiter):
break
lines_bytes.append(line) # type: ignore[arg-type]
lines.append(line)
return b"\n".join(lines_bytes) # type: ignore[return-value,arg-type]
else:
lines_str = []
for line in dumpsys.splitlines(): # type: ignore[union-attr]
if line.strip() == separator: # type: ignore[arg-type]
in_section = True
continue
return b"\n".join(lines) if binary else "\n".join(lines)
if not in_section:
continue
if line.strip().startswith(delimiter_str): # type: ignore[arg-type]
break
lines_str.append(line) # type: ignore[arg-type]
return "\n".join(lines_str) # type: ignore[return-value,arg-type]
@@ -16,8 +16,9 @@ class DumpsysAccessibilityArtifact(AndroidArtifact):
for result in self.results:
ioc_match = self.indicators.check_app_id(result["package_name"])
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
continue
def parse(self, content: str) -> None:
+12 -6
View File
@@ -4,6 +4,7 @@
# https://license.mvt.re/1.1/
import base64
import binascii
import hashlib
from .artifact import AndroidArtifact
@@ -83,17 +84,22 @@ class DumpsysADBArtifact(AndroidArtifact):
return keystore
@staticmethod
def calculate_key_info(user_key: bytes) -> str:
def calculate_key_info(user_key: bytes) -> dict:
if b" " in user_key:
key_base64, user = user_key.split(b" ", 1)
else:
key_base64, user = user_key, b""
key_raw = base64.b64decode(key_base64)
key_fingerprint = hashlib.md5(key_raw).hexdigest().upper()
key_fingerprint_colon = ":".join(
[key_fingerprint[i : i + 2] for i in range(0, len(key_fingerprint), 2)]
)
try:
key_raw = base64.b64decode(key_base64)
key_fingerprint = hashlib.md5(key_raw).hexdigest().upper()
key_fingerprint_colon = ":".join(
[key_fingerprint[i : i + 2] for i in range(0, len(key_fingerprint), 2)]
)
except binascii.Error:
# Impossible to parse base64
key_fingerprint_colon = ""
return {
"user": user.decode("utf-8"),
"fingerprint": key_fingerprint_colon,
+6 -9
View File
@@ -4,13 +4,13 @@
# https://license.mvt.re/1.1/
from datetime import datetime
from typing import Any
from mvt.common.utils import convert_datetime_to_iso
from mvt.common.module_types import ModuleAtomicResult, ModuleSerializedResult
from mvt.common.utils import convert_datetime_to_iso
from .artifact import AndroidArtifact
RISKY_PERMISSIONS = ["REQUEST_INSTALL_PACKAGES"]
RISKY_PACKAGES = ["com.android.shell"]
@@ -45,9 +45,8 @@ class DumpsysAppopsArtifact(AndroidArtifact):
if self.indicators:
ioc_match = self.indicators.check_app_id(result.get("package_name"))
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(
self.get_slug(), ioc_match.message, "", result
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
continue
@@ -66,7 +65,6 @@ class DumpsysAppopsArtifact(AndroidArtifact):
cleaned_result = result.copy()
cleaned_result["permissions"] = [perm]
self.alertstore.medium(
self.get_slug(),
f"Package '{result['package_name']}' had risky permission '{perm['name']}' set to '{entry['access']}' at {entry['timestamp']}",
entry["timestamp"],
cleaned_result,
@@ -80,7 +78,6 @@ class DumpsysAppopsArtifact(AndroidArtifact):
cleaned_result = result.copy()
cleaned_result["permissions"] = [perm]
self.alertstore.medium(
self.get_slug(),
f"Risky package '{result['package_name']}' had '{perm['name']}' permission set to '{entry['access']}' at {entry['timestamp']}",
entry["timestamp"],
cleaned_result,
@@ -88,9 +85,9 @@ class DumpsysAppopsArtifact(AndroidArtifact):
def parse(self, output: str) -> None:
# self.results: List[Dict[str, Any]] = []
perm = {}
package = {}
entry = {}
perm: dict[str, Any] = {}
package: dict[str, Any] = {}
entry: dict[str, Any] = {}
uid = None
in_packages = False
@@ -3,9 +3,11 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from typing import Any
from mvt.common.module_types import ModuleAtomicResult, ModuleSerializedResult
from .artifact import AndroidArtifact
from mvt.common.module_types import ModuleSerializedResult, ModuleAtomicResult
class DumpsysBatteryDailyArtifact(AndroidArtifact):
@@ -14,12 +16,23 @@ class DumpsysBatteryDailyArtifact(AndroidArtifact):
"""
def serialize(self, record: ModuleAtomicResult) -> ModuleSerializedResult:
action = record.get("action", "update")
package_name = record["package_name"]
vers = record["vers"]
if vers == "0":
data = f"Recorded uninstall of package {package_name} (vers 0)"
elif action == "downgrade":
prev_vers = record.get("previous_vers", "unknown")
data = f"Recorded downgrade of package {package_name} from vers {prev_vers} to vers {vers}"
else:
data = f"Recorded update of package {package_name} with vers {vers}"
return {
"timestamp": record["from"],
"module": self.__class__.__name__,
"event": "battery_daily",
"data": f"Recorded update of package {record['package_name']} "
f"with vers {record['vers']}",
"data": data,
}
def check_indicators(self) -> None:
@@ -29,13 +42,17 @@ class DumpsysBatteryDailyArtifact(AndroidArtifact):
for result in self.results:
ioc_match = self.indicators.check_app_id(result["package_name"])
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
continue
def parse(self, output: str) -> None:
daily = None
daily_updates = []
daily_updates: list[dict[str, Any]] = []
package_versions: dict[
str, str
] = {} # Track package versions to detect downgrades
for line in output.splitlines():
if line.startswith(" Daily from "):
if len(daily_updates) > 0:
@@ -64,15 +81,43 @@ class DumpsysBatteryDailyArtifact(AndroidArtifact):
break
if not already_seen:
daily_updates.append(
{
"action": "update",
"from": daily["from"],
"to": daily["to"],
"package_name": package_name,
"vers": vers_nr,
}
)
update_record: dict[str, Any] = {
"action": "update",
"from": daily["from"],
"to": daily["to"],
"package_name": package_name,
"vers": vers_nr,
}
# Check for uninstall (version 0)
if vers_nr == "0":
self.alertstore.medium(
f"Detected uninstall of package {package_name} (vers 0)",
daily["from"],
update_record,
)
# Check for downgrade
elif package_name in package_versions:
try:
current_vers = int(vers_nr)
previous_vers = int(package_versions[package_name])
if current_vers < previous_vers:
update_record["action"] = "downgrade"
update_record["previous_vers"] = str(previous_vers)
self.alertstore.medium(
f"Detected downgrade of package {package_name} "
f"from vers {previous_vers} to vers {current_vers}",
daily["from"],
update_record,
)
except ValueError:
# If version numbers aren't integers, skip comparison
pass
# Update tracking dictionary
package_versions[package_name] = vers_nr
daily_updates.append(update_record)
if len(daily_updates) > 0:
self.results.extend(daily_updates)
@@ -18,8 +18,9 @@ class DumpsysBatteryHistoryArtifact(AndroidArtifact):
for result in self.results:
ioc_match = self.indicators.check_app_id(result["package_name"])
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
continue
def parse(self, data: str) -> None:
+1 -2
View File
@@ -22,9 +22,8 @@ class DumpsysDBInfoArtifact(AndroidArtifact):
for part in path.split("/"):
ioc_match = self.indicators.check_app_id(part)
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(
self.get_slug(), ioc_match.message, "", result
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
continue
@@ -14,9 +14,8 @@ class DumpsysPackageActivitiesArtifact(AndroidArtifact):
for activity in self.results:
ioc_match = self.indicators.check_app_id(activity["package_name"])
if ioc_match:
activity["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(
self.get_slug(), ioc_match.message, "", activity
ioc_match.message, "", activity, matched_indicator=ioc_match.ioc
)
continue
+8 -10
View File
@@ -7,9 +7,9 @@ import re
from typing import Any, Dict, List
from mvt.android.utils import ROOT_PACKAGES
from mvt.common.module_types import ModuleAtomicResult, ModuleSerializedResult
from .artifact import AndroidArtifact
from mvt.common.module_types import ModuleAtomicResult, ModuleSerializedResult
class DumpsysPackagesArtifact(AndroidArtifact):
@@ -18,12 +18,10 @@ class DumpsysPackagesArtifact(AndroidArtifact):
# XXX: De-duplication Package detections
if result["package_name"] in ROOT_PACKAGES:
self.alertstore.medium(
self.get_slug(),
f'Found an installed package related to rooting/jailbreaking: "{result["package_name"]}"',
"",
result,
)
self.alertstore.log_latest()
continue
if not self.indicators:
@@ -31,9 +29,9 @@ class DumpsysPackagesArtifact(AndroidArtifact):
ioc_match = self.indicators.check_app_id(result.get("package_name", ""))
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.log_latest()
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
def serialize(self, record: ModuleAtomicResult) -> ModuleSerializedResult:
records = []
@@ -63,15 +61,15 @@ class DumpsysPackagesArtifact(AndroidArtifact):
"""
Parse one entry of a dumpsys package information
"""
details = {
details: Dict[str, Any] = {
"uid": "",
"version_name": "",
"version_code": "",
"timestamp": "",
"first_install_time": "",
"last_update_time": "",
"permissions": [],
"requested_permissions": [],
"permissions": list(),
"requested_permissions": list(),
}
in_install_permissions = False
in_runtime_permissions = False
@@ -149,7 +147,7 @@ class DumpsysPackagesArtifact(AndroidArtifact):
results = []
package_name = None
package = {}
lines = []
lines: list[str] = []
for line in output.splitlines():
if line.startswith(" Package ["):
if len(lines) > 0:
@@ -18,8 +18,9 @@ class DumpsysPlatformCompatArtifact(AndroidArtifact):
for result in self.results:
ioc_match = self.indicators.check_app_id(result["package_name"])
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
continue
def parse(self, data: str) -> None:
@@ -52,14 +52,16 @@ class DumpsysReceiversArtifact(AndroidArtifact):
ioc_match = self.indicators.check_app_id(receiver["package_name"])
if ioc_match:
receiver["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(
self.get_slug(), ioc_match.message, "", {intent: receiver}
ioc_match.message,
"",
{intent: receiver},
matched_indicator=ioc_match.ioc,
)
continue
def parse(self, output: str) -> None:
self.results = {}
self.results: dict[str, list[dict[str, str]]] = {}
in_receiver_resolver_table = False
in_non_data_actions = False
+8 -6
View File
@@ -39,10 +39,10 @@ class GetProp(AndroidArtifact):
if not matches or len(matches[0]) != 2:
continue
entry = {"name": matches[0][0], "value": matches[0][1]}
self.results.append(entry)
prop_entry = {"name": matches[0][0], "value": matches[0][1]}
self.results.append(prop_entry)
def get_device_timezone(self) -> str:
def get_device_timezone(self) -> str | None:
"""
Get the device timezone from the getprop results
@@ -60,7 +60,8 @@ class GetProp(AndroidArtifact):
if entry["name"] == "ro.build.version.security_patch":
warning_message = warn_android_patch_level(entry["value"], self.log)
self.alertstore.medium(self.get_slug(), warning_message, "", entry)
if isinstance(warning_message, str):
self.alertstore.medium(warning_message, "", entry)
if not self.indicators:
return
@@ -70,5 +71,6 @@ class GetProp(AndroidArtifact):
result.get("name", "")
)
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
+197
View File
@@ -0,0 +1,197 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from typing import Any
from .artifact import AndroidArtifact
SUSPICIOUS_MOUNT_POINTS = [
"/system",
"/vendor",
"/product",
"/system_ext",
]
SUSPICIOUS_OPTIONS = [
"rw",
"remount",
"noatime",
"nodiratime",
]
ALLOWLIST_NOATIME = [
"/system_dlkm",
"/system_ext",
"/product",
"/vendor",
"/vendor_dlkm",
]
class Mounts(AndroidArtifact):
"""
This artifact parses mount information from /proc/mounts or similar mount data.
It can detect potentially suspicious mount configurations that may indicate
a rooted or compromised device.
"""
def parse(self, entry: str) -> None:
"""
Parse mount information from the provided entry.
Examples:
/dev/block/bootdevice/by-name/system /system ext4 ro,seclabel,relatime 0 0
/dev/block/dm-12 on / type ext4 (ro,seclabel,noatime)
"""
self.results: list[dict[str, Any]] = []
for line in entry.splitlines():
line = line.strip()
if not line:
continue
device = None
mount_point = None
filesystem_type = None
mount_options = ""
if " on " in line and " type " in line:
try:
# Format: device on mount_point type filesystem_type (options)
device_part, rest = line.split(" on ", 1)
device = device_part.strip()
# Split by 'type' to get mount_point and filesystem info
mount_part, fs_part = rest.split(" type ", 1)
mount_point = mount_part.strip()
# Parse filesystem and options
if "(" in fs_part and fs_part.endswith(")"):
# Format: filesystem_type (options)
fs_and_opts = fs_part.strip()
paren_idx = fs_and_opts.find("(")
filesystem_type = fs_and_opts[:paren_idx].strip()
mount_options = fs_and_opts[paren_idx + 1 : -1].strip()
else:
# No options in parentheses, just filesystem type
filesystem_type = fs_part.strip()
mount_options = ""
# Skip if we don't have essential info
if not device or not mount_point or not filesystem_type:
continue
# Parse options into list
options_list = (
[opt.strip() for opt in mount_options.split(",") if opt.strip()]
if mount_options
else []
)
# Check if it's a system partition
is_system_partition = mount_point in SUSPICIOUS_MOUNT_POINTS or any(
mount_point.startswith(sp) for sp in SUSPICIOUS_MOUNT_POINTS
)
# Check if it's mounted read-write
is_read_write = "rw" in options_list
mount_entry = {
"device": device,
"mount_point": mount_point,
"filesystem_type": filesystem_type,
"mount_options": mount_options,
"options_list": options_list,
"is_system_partition": is_system_partition,
"is_read_write": is_read_write,
}
self.results.append(mount_entry)
except ValueError:
# If parsing fails, skip this line
continue
else:
# Skip lines that don't match expected format
continue
def check_indicators(self) -> None:
"""
Check for suspicious mount configurations that may indicate root access
or other security concerns.
"""
system_rw_mounts = []
suspicious_mounts = []
for mount in self.results:
mount_point = mount["mount_point"]
options = mount["options_list"]
# Check for system partitions mounted as read-write
if mount["is_system_partition"] and mount["is_read_write"]:
system_rw_mounts.append(mount)
if mount_point == "/system":
self.alertstore.high(
"Root detected /system partition is mounted as read-write (rw)",
"",
mount,
)
else:
self.alertstore.high(
f"System partition {mount_point} is mounted as read-write (rw). This may indicate system modifications.",
"",
mount,
)
# Check for other suspicious mount options
suspicious_opts = [opt for opt in options if opt in SUSPICIOUS_OPTIONS]
if suspicious_opts and mount["is_system_partition"]:
if (
"noatime" in mount["mount_options"]
and mount["mount_point"] in ALLOWLIST_NOATIME
):
continue
suspicious_mounts.append(mount)
self.alertstore.medium(
f"Suspicious mount options found for {mount_point}: {', '.join(suspicious_opts)}",
"",
mount,
)
# Log interesting mount information
if mount_point == "/data" or mount_point.startswith("/sdcard"):
self.log.info(
"Data partition: %s mounted as %s with options: %s",
mount_point,
mount["filesystem_type"],
mount["mount_options"],
)
self.log.info("Parsed %d mount entries", len(self.results))
# Check indicators if available
if not self.indicators:
return
for mount in self.results:
# Check if any mount points match indicators
ioc = self.indicators.check_file_path(mount.get("mount_point", ""))
if ioc:
self.alertstore.critical(
ioc.message,
"",
mount,
matched_indicator=ioc.ioc,
)
# Check device paths for indicators
ioc = self.indicators.check_file_path(mount.get("device", ""))
if ioc:
self.alertstore.critical(
ioc.message,
"",
mount,
matched_indicator=ioc.ioc,
)
+6 -4
View File
@@ -60,11 +60,13 @@ class Processes(AndroidArtifact):
ioc_match = self.indicators.check_app_id(proc_name)
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
continue
ioc_match = self.indicators.check_process(proc_name)
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
-5
View File
@@ -51,11 +51,6 @@ ANDROID_DANGEROUS_SETTINGS = [
"key": "send_action_app_error",
"safe_value": "1",
},
{
"description": "enabled installation of non Google Play apps",
"key": "install_non_market_apps",
"safe_value": "0",
},
{
"description": "enabled accessibility services",
"key": "accessibility_enabled",
+62 -66
View File
@@ -7,13 +7,14 @@ import datetime
from typing import List, Optional
import pydantic
import betterproto
import betterproto2
from dateutil import parser
from mvt.common.utils import convert_datetime_to_iso
from mvt.common.module_types import ModuleAtomicResult, ModuleSerializedResult
from mvt.android.parsers.proto.tombstone import Tombstone
from .artifact import AndroidArtifact
from mvt.common.module_types import ModuleAtomicResult, ModuleSerializedResult
from mvt.common.utils import convert_datetime_to_iso
from .artifact import AndroidArtifact
TOMBSTONE_DELIMITER = "*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***"
@@ -53,7 +54,7 @@ class TombstoneCrashResult(pydantic.BaseModel):
file_name: str
file_timestamp: str # We store the timestamp as a string to avoid timezone issues
build_fingerprint: str
revision: int
revision: str
arch: Optional[str] = None
timestamp: str # We store the timestamp as a string to avoid timezone issues
process_uptime: Optional[int] = None
@@ -63,14 +64,14 @@ class TombstoneCrashResult(pydantic.BaseModel):
process_name: Optional[str] = None
binary_path: Optional[str] = None
selinux_label: Optional[str] = None
uid: Optional[int] = None
uid: int
signal_info: SignalInfo
cause: Optional[str] = None
extra: Optional[str] = None
class TombstoneCrashArtifact(AndroidArtifact):
""" "
"""
Parser for Android tombstone crash files.
This parser can parse both text and protobuf tombstone crash files.
@@ -94,17 +95,18 @@ class TombstoneCrashArtifact(AndroidArtifact):
for result in self.results:
ioc_match = self.indicators.check_process(result["process_name"])
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
continue
if result.get("command_line", []):
command_name = result.get("command_line")[0].split("/")[-1]
command_name = result["command_line"][0]
ioc_match = self.indicators.check_process(command_name)
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(
self.get_slug(), ioc_match.message, "", result
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
continue
@@ -115,7 +117,6 @@ class TombstoneCrashArtifact(AndroidArtifact):
]
if result["uid"] in SUSPICIOUS_UIDS:
self.alertstore.medium(
self.get_slug(),
(
f"Potentially suspicious crash in process '{result['process_name']}' "
f"running as UID '{result['uid']}' in tombstone '{result['file_name']}' at {result['timestamp']}"
@@ -127,11 +128,11 @@ class TombstoneCrashArtifact(AndroidArtifact):
def parse_protobuf(
self, file_name: str, file_timestamp: datetime.datetime, data: bytes
) -> None:
"""
Parse Android tombstone crash files from a protobuf object.
"""
"""Parse Android tombstone crash files from a protobuf object."""
tombstone_pb = Tombstone().parse(data)
tombstone_dict = tombstone_pb.to_dict(betterproto.Casing.SNAKE)
tombstone_dict = tombstone_pb.to_dict(
casing=betterproto2.Casing.SNAKE, include_default_values=True
)
# Add some extra metadata
tombstone_dict["timestamp"] = self._parse_timestamp_string(
@@ -148,21 +149,23 @@ class TombstoneCrashArtifact(AndroidArtifact):
def parse(
self, file_name: str, file_timestamp: datetime.datetime, content: bytes
) -> None:
"""
Parse text Android tombstone crash files.
"""
# Split the tombstone file into a dictonary
"""Parse text Android tombstone crash files."""
tombstone_dict = {
"file_name": file_name,
"file_timestamp": convert_datetime_to_iso(file_timestamp),
}
lines = content.decode("utf-8").splitlines()
for line in lines:
for line_num, line in enumerate(lines, 1):
if not line.strip() or TOMBSTONE_DELIMITER in line:
continue
for key, destination_key in TOMBSTONE_TEXT_KEY_MAPPINGS.items():
self._parse_tombstone_line(line, key, destination_key, tombstone_dict)
try:
for key, destination_key in TOMBSTONE_TEXT_KEY_MAPPINGS.items():
if self._parse_tombstone_line(
line, key, destination_key, tombstone_dict
):
break
except Exception as e:
raise ValueError(f"Error parsing line {line_num}: {str(e)}")
# Validate the tombstone and add it to the results
tombstone = TombstoneCrashResult.model_validate(tombstone_dict)
@@ -172,7 +175,7 @@ class TombstoneCrashArtifact(AndroidArtifact):
self, line: str, key: str, destination_key: str, tombstone: dict
) -> bool:
if not line.startswith(f"{key}"):
return None
return False
if key == "pid":
return self._load_pid_line(line, tombstone)
@@ -191,7 +194,7 @@ class TombstoneCrashArtifact(AndroidArtifact):
raise ValueError(f"Expected key {key}, got {line_key}")
value_clean = value.strip().strip("'")
if destination_key in ["uid", "revision"]:
if destination_key == "uid":
tombstone[destination_key] = int(value_clean)
elif destination_key == "process_uptime":
# eg. "Process uptime: 40s"
@@ -204,51 +207,50 @@ class TombstoneCrashArtifact(AndroidArtifact):
return True
def _load_pid_line(self, line: str, tombstone: dict) -> bool:
pid_part, tid_part, name_part = [part.strip() for part in line.split(",")]
try:
parts = line.split(" >>> ") if " >>> " in line else line.split(">>>")
process_info = parts[0]
pid_key, pid_value = pid_part.split(":", 1)
if pid_key != "pid":
raise ValueError(f"Expected key pid, got {pid_key}")
pid_value = int(pid_value.strip())
# Parse pid, tid, name from process info
info_parts = [p.strip() for p in process_info.split(",")]
for info in info_parts:
key, value = info.split(":", 1)
key = key.strip()
value = value.strip()
tid_key, tid_value = tid_part.split(":", 1)
if tid_key != "tid":
raise ValueError(f"Expected key tid, got {tid_key}")
tid_value = int(tid_value.strip())
if key == "pid":
tombstone["pid"] = int(value)
elif key == "tid":
tombstone["tid"] = int(value)
elif key == "name":
tombstone["process_name"] = value
name_key, name_value = name_part.split(":", 1)
if name_key != "name":
raise ValueError(f"Expected key name, got {name_key}")
name_value = name_value.strip()
process_name, binary_path = self._parse_process_name(name_value, tombstone)
# Extract binary path if it exists
if len(parts) > 1:
tombstone["binary_path"] = parts[1].strip().rstrip(" <")
tombstone["pid"] = pid_value
tombstone["tid"] = tid_value
tombstone["process_name"] = process_name
tombstone["binary_path"] = binary_path
return True
return True
def _parse_process_name(self, process_name_part, tombstone: dict) -> bool:
process_name, process_path = process_name_part.split(">>>")
process_name = process_name.strip()
binary_path = process_path.strip().split(" ")[0]
return process_name, binary_path
except Exception as e:
raise ValueError(f"Failed to parse PID line: {str(e)}")
def _load_signal_line(self, line: str, tombstone: dict) -> bool:
signal, code, _ = [part.strip() for part in line.split(",", 2)]
signal = signal.split("signal ")[1]
signal_code, signal_name = signal.split(" ")
signal_name = signal_name.strip("()")
signal_part, code_part = map(str.strip, line.split(",")[:2])
code_part = code.split("code ")[1]
code_number, code_name = code_part.split(" ")
code_name = code_name.strip("()")
def parse_part(part: str, prefix: str) -> tuple[int, str]:
match = part.split(prefix)[1]
number = int(match.split()[0])
name = match.split("(")[1].split(")")[0] if "(" in match else "UNKNOWN"
return number, name
signal_number, signal_name = parse_part(signal_part, "signal ")
code_number, code_name = parse_part(code_part, "code ")
tombstone["signal_info"] = {
"code": int(code_number),
"code": code_number,
"code_name": code_name,
"name": signal_name,
"number": int(signal_code),
"number": signal_number,
}
return True
@@ -259,13 +261,7 @@ class TombstoneCrashArtifact(AndroidArtifact):
@staticmethod
def _parse_timestamp_string(timestamp: str) -> str:
timestamp_date, timezone = timestamp.split("+")
# Truncate microseconds before parsing
timestamp_without_micro = timestamp_date.split(".")[0] + "+" + timezone
timestamp_parsed = datetime.datetime.strptime(
timestamp_without_micro, "%Y-%m-%d %H:%M:%S%z"
)
timestamp_parsed = parser.parse(timestamp)
# HACK: Swap the local timestamp to UTC, so keep the original time and avoid timezone conversion.
local_timestamp = timestamp_parsed.replace(tzinfo=datetime.timezone.utc)
return convert_datetime_to_iso(local_timestamp)
+123 -16
View File
@@ -9,35 +9,39 @@ import click
from mvt.common.cmd_check_iocs import CmdCheckIOCS
from mvt.common.help import (
HELP_MSG_VERSION,
HELP_MSG_OUTPUT,
HELP_MSG_VERBOSE,
HELP_MSG_ANDROID_BACKUP_PASSWORD,
HELP_MSG_CHECK_ADB_REMOVED,
HELP_MSG_CHECK_ADB_REMOVED_DESCRIPTION,
HELP_MSG_CHECK_ANDROID_BACKUP,
HELP_MSG_CHECK_ANDROIDQF,
HELP_MSG_CHECK_BUGREPORT,
HELP_MSG_CHECK_IOCS,
HELP_MSG_CHECK_INTRUSION_LOGS,
HELP_MSG_DISABLE_INDICATOR_UPDATE_CHECK,
HELP_MSG_DISABLE_UPDATE_CHECK,
HELP_MSG_HASHES,
HELP_MSG_IOC,
HELP_MSG_LIST_MODULES,
HELP_MSG_MODULE,
HELP_MSG_NONINTERACTIVE,
HELP_MSG_ANDROID_BACKUP_PASSWORD,
HELP_MSG_CHECK_ADB_REMOVED,
HELP_MSG_CHECK_ADB_REMOVED_DESCRIPTION,
HELP_MSG_CHECK_BUGREPORT,
HELP_MSG_CHECK_ANDROID_BACKUP,
HELP_MSG_CHECK_ANDROIDQF,
HELP_MSG_HASHES,
HELP_MSG_CHECK_IOCS,
HELP_MSG_OUTPUT,
HELP_MSG_STIX2,
HELP_MSG_VERBOSE,
HELP_MSG_VERSION,
)
from mvt.common.logo import logo
from mvt.common.updates import IndicatorsUpdates
from mvt.common.utils import init_logging, set_verbose_logging
from .cmd_check_androidqf import CmdAndroidCheckAndroidQF
from .cmd_check_backup import CmdAndroidCheckBackup
from .cmd_check_bugreport import CmdAndroidCheckBugreport
from .cmd_check_intrusion_logs import CmdAndroidCheckIntrusionLogs
from .modules.intrusion_logs import INTRUSION_LOGS_MODULES
from .modules.androidqf import ANDROIDQF_MODULES
from .modules.backup import BACKUP_MODULES
from .modules.backup.helpers import cli_load_android_backup_password
from .modules.bugreport import BUGREPORT_MODULES
from .modules.androidqf import ANDROIDQF_MODULES
init_logging()
log = logging.getLogger("mvt")
@@ -45,12 +49,37 @@ log = logging.getLogger("mvt")
CONTEXT_SETTINGS = dict(help_option_names=["-h", "--help"])
def _get_disable_flags(ctx):
"""Helper function to safely get disable flags from context."""
if ctx.obj is None:
return False, False
return (
ctx.obj.get("disable_version_check", False),
ctx.obj.get("disable_indicator_check", False),
)
# ==============================================================================
# Main
# ==============================================================================
@click.group(invoke_without_command=False)
def cli():
logo()
@click.option(
"--disable-update-check", is_flag=True, help=HELP_MSG_DISABLE_UPDATE_CHECK
)
@click.option(
"--disable-indicator-update-check",
is_flag=True,
help=HELP_MSG_DISABLE_INDICATOR_UPDATE_CHECK,
)
@click.pass_context
def cli(ctx, disable_update_check, disable_indicator_update_check):
ctx.ensure_object(dict)
ctx.obj["disable_version_check"] = disable_update_check
ctx.obj["disable_indicator_check"] = disable_indicator_update_check
logo(
disable_version_check=disable_update_check,
disable_indicator_check=disable_indicator_update_check,
)
# ==============================================================================
@@ -70,6 +99,7 @@ def version():
@click.pass_context
def check_adb(ctx):
log.error(HELP_MSG_CHECK_ADB_REMOVED_DESCRIPTION)
ctx.exit(1)
# ==============================================================================
@@ -101,6 +131,8 @@ def check_bugreport(ctx, iocs, output, list_modules, module, verbose, bugreport_
ioc_files=iocs,
module_name=module,
hashes=True,
disable_version_check=_get_disable_flags(ctx)[0],
disable_indicator_check=_get_disable_flags(ctx)[1],
)
if list_modules:
@@ -159,6 +191,8 @@ def check_backup(
"interactive": not non_interactive,
"backup_password": cli_load_android_backup_password(log, backup_password),
},
disable_version_check=_get_disable_flags(ctx)[0],
disable_indicator_check=_get_disable_flags(ctx)[1],
)
if list_modules:
@@ -219,6 +253,8 @@ def check_androidqf(
"interactive": not non_interactive,
"backup_password": cli_load_android_backup_password(log, backup_password),
},
disable_version_check=_get_disable_flags(ctx)[0],
disable_indicator_check=_get_disable_flags(ctx)[1],
)
if list_modules:
@@ -233,6 +269,75 @@ def check_androidqf(
cmd.show_support_message()
# ==============================================================================
# Command: check-intrusion-logs
# ==============================================================================
@cli.command(
"check-intrusion-logs",
context_settings=CONTEXT_SETTINGS,
help=HELP_MSG_CHECK_INTRUSION_LOGS,
)
@click.option(
"--iocs",
"-i",
type=click.Path(exists=True),
multiple=True,
default=[],
help=HELP_MSG_IOC,
)
@click.option("--output", "-o", type=click.Path(exists=False), help=HELP_MSG_OUTPUT)
@click.option("--list-modules", "-l", is_flag=True, help=HELP_MSG_LIST_MODULES)
@click.option("--module", "-m", help=HELP_MSG_MODULE)
@click.option(
"--timezone",
"-t",
default=None,
help=(
"IANA timezone name for the device, for example 'Europe/Paris'. "
"When provided, event timestamps are expressed in the device's local "
"time instead of UTC."
),
)
@click.option("--verbose", "-v", is_flag=True, help=HELP_MSG_VERBOSE)
@click.argument("LOGS_PATH", type=click.Path(exists=True))
@click.pass_context
def check_intrusion_logs(
ctx,
iocs,
output,
list_modules,
module,
timezone,
verbose,
logs_path,
):
set_verbose_logging(verbose)
module_options = {}
if timezone:
module_options["device_timezone"] = timezone
cmd = CmdAndroidCheckIntrusionLogs(
target_path=logs_path,
results_path=output,
ioc_files=iocs,
module_name=module,
module_options=module_options if module_options else None,
disable_version_check=_get_disable_flags(ctx)[0],
disable_indicator_check=_get_disable_flags(ctx)[1],
)
if list_modules:
cmd.list_modules()
return
log.info("Checking intrusion logs at path: %s", logs_path)
cmd.run()
cmd.show_alerts_brief()
cmd.show_support_message()
# ==============================================================================
# Command: check-iocs
# ==============================================================================
@@ -251,7 +356,9 @@ def check_androidqf(
@click.pass_context
def check_iocs(ctx, iocs, list_modules, module, folder):
cmd = CmdCheckIOCS(target_path=folder, ioc_files=iocs, module_name=module)
cmd.modules = BACKUP_MODULES + BUGREPORT_MODULES + ANDROIDQF_MODULES
cmd.modules = (
BACKUP_MODULES + BUGREPORT_MODULES + ANDROIDQF_MODULES + INTRUSION_LOGS_MODULES
)
if list_modules:
cmd.list_modules()
+144 -12
View File
@@ -5,16 +5,19 @@
import logging
import os
import shutil
import tempfile
import zipfile
from pathlib import Path
from typing import List, Optional
from mvt.android.artifacts.getprop import GetProp
from mvt.android.cmd_check_intrusion_logs import CmdAndroidCheckIntrusionLogs
from mvt.android.cmd_check_backup import CmdAndroidCheckBackup
from mvt.android.cmd_check_bugreport import CmdAndroidCheckBugreport
from mvt.common.command import Command
from mvt.common.indicators import Indicators
from mvt.android.cmd_check_bugreport import CmdAndroidCheckBugreport
from mvt.android.cmd_check_backup import CmdAndroidCheckBackup
from .modules.androidqf import ANDROIDQF_MODULES
from .modules.androidqf.base import AndroidQFModule
@@ -45,6 +48,8 @@ class CmdAndroidCheckAndroidQF(Command):
module_options: Optional[dict] = None,
hashes: Optional[bool] = False,
sub_command: Optional[bool] = False,
disable_version_check: bool = False,
disable_indicator_check: bool = False,
) -> None:
super().__init__(
target_path=target_path,
@@ -57,6 +62,8 @@ class CmdAndroidCheckAndroidQF(Command):
hashes=hashes,
sub_command=sub_command,
log=log,
disable_version_check=disable_version_check,
disable_indicator_check=disable_indicator_check,
)
self.name = "check-androidqf"
@@ -136,7 +143,57 @@ class CmdAndroidCheckAndroidQF(Command):
raise NoAndroidQFBackup
def _read_device_timezone(self) -> Optional[str]:
getprop_files = [
f for f in self.__files if f.replace("\\", "/").endswith("getprop.txt")
]
if not getprop_files:
self.log.warning(
"Could not find getprop.txt; intrusion log timestamps will use UTC."
)
return None
try:
content = self._get_file_content(getprop_files[0]).decode(
"utf-8", errors="ignore"
)
except Exception as exc:
self.log.warning("Could not read getprop.txt: %s", exc)
return None
props = GetProp()
props.parse(content)
timezone = props.get_device_timezone()
if timezone:
self.log.info(
"Device timezone identified from getprop.txt: %s",
timezone,
)
else:
self.log.warning(
"persist.sys.timezone not found in getprop.txt; "
"intrusion log timestamps will use UTC."
)
return timezone
def _get_file_content(self, file_path: str) -> bytes:
if self.__format == "zip" and self.__zip:
handle = self.__zip.open(file_path)
try:
return handle.read()
finally:
handle.close()
if self.__format == "dir" and self.target_path:
parent_path = Path(self.target_path).absolute().parent.as_posix()
with open(os.path.join(parent_path, file_path), "rb") as handle:
return handle.read()
raise FileNotFoundError(file_path)
def run_bugreport_cmd(self) -> bool:
bugreport = None
try:
bugreport = self.load_bugreport()
except NoAndroidQFBugReport:
@@ -163,6 +220,8 @@ class CmdAndroidCheckAndroidQF(Command):
if bugreport:
bugreport.close()
return True
def run_backup_cmd(self) -> bool:
try:
backup = self.load_backup()
@@ -171,29 +230,102 @@ class CmdAndroidCheckAndroidQF(Command):
"Skipping backup modules as no backup.ab found in AndroidQF data."
)
return False
else:
cmd = CmdAndroidCheckBackup(
target_path=None,
cmd = CmdAndroidCheckBackup(
target_path=None,
results_path=self.results_path,
ioc_files=self.ioc_files,
iocs=self.iocs,
module_options=self.module_options,
hashes=self.hashes,
sub_command=True,
)
cmd.from_ab(backup)
cmd.run()
self.timeline.extend(cmd.timeline)
self.alertstore.extend(cmd.alertstore.alerts)
return True
def run_intrusion_logs_cmd(self) -> bool:
intrusion_log_files = [
f
for f in self.__files
if "/intrusion_logs/" in f.replace("\\", "/")
or f.replace("\\", "/").startswith("intrusion_logs/")
]
if not intrusion_log_files:
self.log.info(
"No intrusion_logs folder found in AndroidQF data, "
"skipping intrusion logs analysis."
)
return False
self.log.info(
"Found intrusion_logs folder in AndroidQF data, running intrusion logs analysis."
)
intrusion_logs_path = None
temp_dir = None
try:
if self.__format == "dir" and self.target_path:
intrusion_logs_path = os.path.join(
os.path.abspath(self.target_path), "intrusion_logs"
)
if not os.path.isdir(intrusion_logs_path):
self.log.warning(
"intrusion_logs directory not found at %s",
intrusion_logs_path,
)
return False
elif self.__format == "zip" and self.__zip:
temp_dir = tempfile.mkdtemp(prefix="mvt_intrusion_logs_")
for entry in intrusion_log_files:
normalized = entry.replace("\\", "/")
idx = normalized.find("intrusion_logs/")
relative = normalized[idx + len("intrusion_logs/") :]
if not relative or relative.endswith("/"):
continue
target = os.path.join(temp_dir, relative)
os.makedirs(os.path.dirname(target), exist_ok=True)
with self.__zip.open(entry) as src, open(target, "wb") as dst:
dst.write(src.read())
intrusion_logs_path = temp_dir
else:
return False
adv_module_options = dict(self.module_options or {})
if device_timezone := self._read_device_timezone():
adv_module_options["device_timezone"] = device_timezone
cmd = CmdAndroidCheckIntrusionLogs(
target_path=intrusion_logs_path,
results_path=self.results_path,
ioc_files=self.ioc_files,
iocs=self.iocs,
module_options=self.module_options,
module_options=adv_module_options,
hashes=self.hashes,
sub_command=True,
)
cmd.from_ab(backup)
cmd.run()
self.timeline.extend(cmd.timeline)
self.alertstore.extend(cmd.alertstore.alerts)
finally:
if backup:
backup.close()
return True
finally:
if temp_dir:
shutil.rmtree(temp_dir, ignore_errors=True)
def finish(self) -> None:
"""
Run the bugreport and backup modules if the respective files are found in the AndroidQF data.
Run nested modules if their respective files are found in AndroidQF data.
"""
self.run_bugreport_cmd()
self.run_backup_cmd()
self.run_intrusion_logs_cmd()
+17 -7
View File
@@ -39,6 +39,8 @@ class CmdAndroidCheckBackup(Command):
module_options: Optional[dict] = None,
hashes: Optional[bool] = False,
sub_command: Optional[bool] = False,
disable_version_check: bool = False,
disable_indicator_check: bool = False,
) -> None:
super().__init__(
target_path=target_path,
@@ -51,6 +53,8 @@ class CmdAndroidCheckBackup(Command):
hashes=hashes,
sub_command=sub_command,
log=log,
disable_version_check=disable_version_check,
disable_indicator_check=disable_indicator_check,
)
self.name = "check-backup"
@@ -89,22 +93,28 @@ class CmdAndroidCheckBackup(Command):
self.__files.append(member.name)
def init(self) -> None:
if not self.target_path:
if not self.target_path: # type: ignore[has-type]
return
if os.path.isfile(self.target_path):
# Type guard: we know it's not None here after the check above
assert self.target_path is not None # type: ignore[has-type]
# Use a different local variable name to avoid any scoping issues
backup_path: str = self.target_path # type: ignore[has-type]
if os.path.isfile(backup_path):
self.__type = "ab"
with open(self.target_path, "rb") as handle:
with open(backup_path, "rb") as handle:
ab_file_bytes = handle.read()
self.from_ab(ab_file_bytes)
elif os.path.isdir(self.target_path):
elif os.path.isdir(backup_path):
self.__type = "folder"
self.target_path = Path(self.target_path).absolute().as_posix()
for root, subdirs, subfiles in os.walk(os.path.abspath(self.target_path)):
backup_path = Path(backup_path).absolute().as_posix()
self.target_path = backup_path
for root, subdirs, subfiles in os.walk(os.path.abspath(backup_path)):
for fname in subfiles:
self.__files.append(
os.path.relpath(os.path.join(root, fname), self.target_path)
os.path.relpath(os.path.join(root, fname), backup_path)
)
else:
log.critical(
+6
View File
@@ -30,6 +30,8 @@ class CmdAndroidCheckBugreport(Command):
module_options: Optional[dict] = None,
hashes: Optional[bool] = False,
sub_command: Optional[bool] = False,
disable_version_check: bool = False,
disable_indicator_check: bool = False,
) -> None:
super().__init__(
target_path=target_path,
@@ -42,6 +44,8 @@ class CmdAndroidCheckBugreport(Command):
hashes=hashes,
sub_command=sub_command,
log=log,
disable_version_check=disable_version_check,
disable_indicator_check=disable_indicator_check,
)
self.name = "check-bugreport"
@@ -92,6 +96,8 @@ class CmdAndroidCheckBugreport(Command):
if self.__format == "zip":
module.from_zip(self.__zip, self.__files)
else:
if not self.target_path:
raise ValueError("target_path is not set")
module.from_dir(self.target_path, self.__files)
def finish(self) -> None:
+113
View File
@@ -0,0 +1,113 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
import os
from typing import Optional
from mvt.common.command import Command
from mvt.common.indicators import Indicators
from .modules.intrusion_logs import (
INTRUSION_LOGS_MODULES,
KNOWN_INTRUSION_LOG_EVENT_TYPES,
)
from .modules.intrusion_logs.base import IntrusionLogsModule
log = logging.getLogger(__name__)
class CmdAndroidCheckIntrusionLogs(Command):
"""Command to check Android Intrusion Logging files."""
def __init__(
self,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
ioc_files: Optional[list] = None,
iocs: Optional[Indicators] = None,
module_name: Optional[str] = None,
serial: Optional[str] = None,
module_options: Optional[dict] = None,
hashes: Optional[bool] = False,
sub_command: Optional[bool] = False,
disable_version_check: bool = False,
disable_indicator_check: bool = False,
) -> None:
super().__init__(
target_path=target_path,
results_path=results_path,
ioc_files=ioc_files,
iocs=iocs,
module_name=module_name,
serial=serial,
module_options=module_options,
hashes=hashes,
sub_command=sub_command,
log=log,
disable_version_check=disable_version_check,
disable_indicator_check=disable_indicator_check,
)
self.name = "check-intrusion-logs"
self.modules = INTRUSION_LOGS_MODULES
self._all_events: dict[str, list[dict]] = {}
def init(self) -> None:
if not self.target_path:
raise ValueError("No target path specified")
if not os.path.isdir(self.target_path) and not (
os.path.isfile(self.target_path)
and self.target_path.lower().endswith(".zip")
):
raise ValueError(
f"Target path must be a directory or a .zip file: {self.target_path}"
)
self.log.info("Checking intrusion logs at path: %s", self.target_path)
self._all_events = self._pre_load_events()
def module_init(self, module: IntrusionLogsModule) -> None: # type: ignore[override]
module.il_events_by_type = self._all_events
def finish(self) -> None:
return
def _pre_load_events(self) -> dict[str, list[dict]]:
"""Load and parse all advanced-log files once for reuse by all modules."""
self.log.info("Pre-loading intrusion log files from: %s", self.target_path)
loader = IntrusionLogsModule(
target_path=self.target_path,
log=self.log,
)
try:
all_events = loader.load_all_events(self.target_path)
except Exception as exc:
self.log.error("Failed to pre-load events: %s", exc)
return {}
total_events = sum(len(events) for events in all_events.values())
self.log.info(
"Pre-loaded %d events across %d type(s); modules will reuse this data",
total_events,
len(all_events),
)
unknown_event_types = sorted(
event_type
for event_type in all_events
if event_type not in KNOWN_INTRUSION_LOG_EVENT_TYPES
)
if unknown_event_types:
self.log.warning(
"Found unknown intrusion logging event type(s): %s. "
"Please open an issue on GitHub so MVT can add support for them.",
", ".join(unknown_event_types),
)
return all_events
-32
View File
@@ -1,32 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from .chrome_history import ChromeHistory
from .dumpsys_full import DumpsysFull
from .files import Files
from .getprop import Getprop
from .logcat import Logcat
from .packages import Packages
from .processes import Processes
from .root_binaries import RootBinaries
from .selinux_status import SELinuxStatus
from .settings import Settings
from .sms import SMS
from .whatsapp import Whatsapp
ADB_MODULES = [
ChromeHistory,
SMS,
Whatsapp,
Processes,
Getprop,
Settings,
SELinuxStatus,
DumpsysFull,
Packages,
Logcat,
RootBinaries,
Files,
]
@@ -1,116 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
import os
import sqlite3
from typing import Optional
from mvt.common.utils import convert_chrometime_to_datetime, convert_datetime_to_iso
from mvt.common.module_types import (
ModuleAtomicResult,
ModuleSerializedResult,
ModuleResults,
)
from .base import AndroidExtraction
CHROME_HISTORY_PATH = "data/data/com.android.chrome/app_chrome/Default/History"
class ChromeHistory(AndroidExtraction):
"""This module extracts records from Android's Chrome browsing history."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
self.results = []
def serialize(self, record: ModuleAtomicResult) -> ModuleSerializedResult:
return {
"timestamp": record["isodate"],
"module": self.__class__.__name__,
"event": "visit",
"data": f"{record['id']} - {record['url']} (visit ID: {record['visit_id']}, "
f"redirect source: {record['redirect_source']})",
}
def check_indicators(self) -> None:
if not self.indicators:
return
for result in self.results:
ioc_match = self.indicators.check_url(result["url"])
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
def _parse_db(self, db_path: str) -> None:
"""Parse a Chrome History database file.
:param db_path: Path to the History database to process.
"""
assert isinstance(self.results, list) # assert results type for mypy
conn = sqlite3.connect(db_path)
cur = conn.cursor()
cur.execute(
"""
SELECT
urls.id,
urls.url,
visits.id,
visits.visit_time,
visits.from_visit
FROM urls
JOIN visits ON visits.url = urls.id
ORDER BY visits.visit_time;
"""
)
for item in cur:
self.results.append(
{
"id": item[0],
"url": item[1],
"visit_id": item[2],
"timestamp": item[3],
"isodate": convert_datetime_to_iso(
convert_chrometime_to_datetime(item[3])
),
"redirect_source": item[4],
}
)
cur.close()
conn.close()
self.log.info("Extracted a total of %d history items", len(self.results))
def run(self) -> None:
self._adb_connect()
try:
self._adb_process_file(
os.path.join("/", CHROME_HISTORY_PATH), self._parse_db
)
except Exception as exc:
self.log.error(exc)
self._adb_disconnect()
@@ -1,46 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
import os
from typing import Optional
from .base import AndroidExtraction
from mvt.common.module_types import ModuleResults
class DumpsysFull(AndroidExtraction):
"""This module extracts stats on battery consumption by processes."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def run(self) -> None:
self._adb_connect()
output = self._adb_command("dumpsys")
if self.results_path:
output_path = os.path.join(self.results_path, "dumpsys.txt")
with open(output_path, "w", encoding="utf-8") as handle:
handle.write(output)
self.log.info("Full dumpsys output stored at %s", output_path)
self._adb_disconnect()
-156
View File
@@ -1,156 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
import os
import stat
from typing import Optional, Union
from mvt.common.utils import convert_unix_to_iso
from mvt.common.module_types import ModuleResults
from .base import AndroidExtraction
ANDROID_TMP_FOLDERS = [
"/tmp/",
"/data/local/tmp/",
]
ANDROID_MEDIA_FOLDERS = [
"/data/media/0",
"/sdcard/",
]
class Files(AndroidExtraction):
"""This module extracts the list of files on the device."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
self.full_find = False
def serialize(self, record: dict) -> Union[dict, list, None]:
if "modified_time" in record:
return {
"timestamp": record["modified_time"],
"module": self.__class__.__name__,
"event": "file_modified",
"data": record["path"],
}
return None
def check_indicators(self) -> None:
for result in self.results:
if result.get("is_suid"):
self.log.warning(
'Found an SUID file in a non-standard directory "%s".',
result["path"],
)
if self.indicators and self.indicators.check_file_path(result["path"]):
self.log.warning(
'Found a known suspicous file at path: "%s"', result["path"]
)
self.detected.append(result)
def backup_file(self, file_path: str) -> None:
if not self.results_path:
return
local_file_name = file_path.replace("/", "_").replace(" ", "-")
local_files_folder = os.path.join(self.results_path, "files")
if not os.path.exists(local_files_folder):
os.mkdir(local_files_folder)
local_file_path = os.path.join(local_files_folder, local_file_name)
try:
self._adb_download(remote_path=file_path, local_path=local_file_path)
except Exception:
pass
else:
self.log.info(
"Downloaded file %s to local copy at %s", file_path, local_file_path
)
def find_files(self, folder: str) -> None:
assert isinstance(self.results, list)
if self.full_find:
cmd = f"find '{folder}' -type f -printf '%T@ %m %s %u %g %p\n' 2> /dev/null"
output = self._adb_command(cmd)
for file_line in output.splitlines():
file_info = file_line.rstrip().split(" ", 5)
if len(file_line) < 6:
self.log.info("Skipping invalid file info - %s", file_line.rstrip())
continue
[unix_timestamp, mode, size, owner, group, full_path] = file_info
mod_time = convert_unix_to_iso(unix_timestamp)
self.results.append(
{
"path": full_path,
"modified_time": mod_time,
"mode": mode,
"is_suid": (int(mode, 8) & stat.S_ISUID) == 2048,
"is_sgid": (int(mode, 8) & stat.S_ISGID) == 1024,
"size": size,
"owner": owner,
"group": group,
}
)
else:
output = self._adb_command(f"find '{folder}' -type f 2> /dev/null")
for file_line in output.splitlines():
self.results.append({"path": file_line.rstrip()})
def run(self) -> None:
self._adb_connect()
cmd = "find '/' -maxdepth 1 -printf '%T@ %m %s %u %g %p\n' 2> /dev/null"
output = self._adb_command(cmd)
if output or output.strip().splitlines():
self.full_find = True
for tmp_folder in ANDROID_TMP_FOLDERS:
self.find_files(tmp_folder)
for entry in self.results:
self.log.info("Found file in tmp folder at path %s", entry.get("path"))
self.backup_file(entry.get("path"))
for media_folder in ANDROID_MEDIA_FOLDERS:
self.find_files(media_folder)
self.log.info(
"Found %s files in primary Android tmp and media folders", len(self.results)
)
if self.module_options.get("fast_mode", None):
self.log.info(
"The `fast_mode` option was enabled: skipping full file listing"
)
else:
self.log.info("Processing full file listing. This may take a while...")
self.find_files("/")
self.log.info("Found %s total files", len(self.results))
self._adb_disconnect()
-44
View File
@@ -1,44 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from mvt.android.artifacts.getprop import GetProp as GetPropArtifact
from .base import AndroidExtraction
from mvt.common.module_types import ModuleResults
class Getprop(GetPropArtifact, AndroidExtraction):
"""This module extracts device properties from getprop command."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
self.results = {} if not results else results
def run(self) -> None:
self._adb_connect()
output = self._adb_command("getprop")
self._adb_disconnect()
self.parse(output)
self.log.info("Extracted %d Android system properties", len(self.results))
-58
View File
@@ -1,58 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
import os
from typing import Optional
from .base import AndroidExtraction
from mvt.common.module_types import ModuleResults
class Logcat(AndroidExtraction):
"""This module extracts details on installed packages."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def run(self) -> None:
self._adb_connect()
# Get the current logcat.
output = self._adb_command('logcat -d -b all "*:V"')
# Get the locat prior to last reboot.
last_output = self._adb_command('logcat -L -b all "*:V"')
if self.results_path:
logcat_path = os.path.join(self.results_path, "logcat.txt")
with open(logcat_path, "w", encoding="utf-8") as handle:
handle.write(output)
self.log.info("Current logcat logs stored at %s", logcat_path)
logcat_last_path = os.path.join(self.results_path, "logcat_last.txt")
with open(logcat_last_path, "w", encoding="utf-8") as handle:
handle.write(last_output)
self.log.info(
"Logcat logs prior to last reboot stored at %s", logcat_last_path
)
self._adb_disconnect()
-311
View File
@@ -1,311 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from mvt.android.artifacts.dumpsys_packages import DumpsysPackagesArtifact
from mvt.android.utils import (
DANGEROUS_PERMISSIONS,
DANGEROUS_PERMISSIONS_THRESHOLD,
ROOT_PACKAGES,
SECURITY_PACKAGES,
SYSTEM_UPDATE_PACKAGES,
)
from mvt.common.module_types import (
ModuleAtomicResult,
ModuleResults,
ModuleSerializedResult,
)
from .base import AndroidExtraction
class Packages(AndroidExtraction):
"""This module extracts the list of installed packages."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
self._user_needed = False
def serialize(self, record: ModuleAtomicResult) -> ModuleSerializedResult:
records = []
timestamps = [
{"event": "package_install", "timestamp": record["timestamp"]},
{
"event": "package_first_install",
"timestamp": record["first_install_time"],
},
{"event": "package_last_update", "timestamp": record["last_update_time"]},
]
for timestamp in timestamps:
records.append(
{
"timestamp": timestamp["timestamp"],
"module": self.__class__.__name__,
"event": timestamp["event"],
"data": f"{record['package_name']} (system: {record['system']},"
f" third party: {record['third_party']})",
}
)
return records
def check_indicators(self) -> None:
for result in self.results:
if result["package_name"] in ROOT_PACKAGES:
self.log.warning(
'Found an installed package related to rooting/jailbreaking: "%s"',
result["package_name"],
)
self.detected.append(result)
continue
if result["package_name"] in SECURITY_PACKAGES and result["disabled"]:
self.log.warning(
'Found a security package disabled: "%s"', result["package_name"]
)
if result["package_name"] in SYSTEM_UPDATE_PACKAGES and result["disabled"]:
self.log.warning(
'System OTA update package "%s" disabled on the phone',
result["package_name"],
)
if not self.indicators:
continue
ioc_match = self.indicators.check_app_id(result.get("package_name"))
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
for package_file in result.get("files", []):
ioc_match = self.indicators.check_file_hash(package_file["sha256"])
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(
self.get_slug(), ioc_match.message, "", result
)
# @staticmethod
# def check_virustotal(packages: list) -> None:
# hashes = []
# for package in packages:
# for file in package.get("files", []):
# if file["sha256"] not in hashes:
# hashes.append(file["sha256"])
# total_hashes = len(hashes)
# detections = {}
# progress_desc = f"Looking up {total_hashes} files..."
# for i in track(range(total_hashes), description=progress_desc):
# try:
# results = virustotal_lookup(hashes[i])
# except VTNoKey:
# return
# except VTQuotaExceeded as exc:
# print("Unable to continue: %s", exc)
# break
# if not results:
# continue
# positives = results["attributes"]["last_analysis_stats"]["malicious"]
# total = len(results["attributes"]["last_analysis_results"])
# detections[hashes[i]] = f"{positives}/{total}"
# table = Table(title="VirusTotal Packages Detections")
# table.add_column("Package name")
# table.add_column("File path")
# table.add_column("Detections")
# for package in packages:
# for file in package.get("files", []):
# row = [package["package_name"], file["path"]]
# if file["sha256"] in detections:
# detection = detections[file["sha256"]]
# positives = detection.split("/")[0]
# if int(positives) > 0:
# row.append(Text(detection, "red bold"))
# else:
# row.append(detection)
# else:
# row.append("not found")
# table.add_row(*row)
# console = Console()
# console.print(table)
@staticmethod
def parse_package_for_details(output: str) -> dict:
lines = []
in_packages = False
for line in output.splitlines():
if in_packages:
if line.strip() == "":
break
lines.append(line)
if line.strip() == "Packages:":
in_packages = True
return DumpsysPackagesArtifact.parse_dumpsys_package_for_details(
"\n".join(lines)
)
def _get_files_for_package(self, package_name: str) -> list:
command = f"pm path {package_name}"
if self._user_needed:
command += " --user 0"
output = self._adb_command(command)
output = output.strip().replace("package:", "")
if not output:
return []
package_files = []
for file_path in output.splitlines():
file_path = file_path.strip()
md5 = self._adb_command(f"md5sum {file_path}").split(" ", maxsplit=1)[0]
sha1 = self._adb_command(f"sha1sum {file_path}").split(" ", maxsplit=1)[0]
sha256 = self._adb_command(f"sha256sum {file_path}").split(" ", maxsplit=1)[
0
]
sha512 = self._adb_command(f"sha512sum {file_path}").split(" ", maxsplit=1)[
0
]
package_files.append(
{
"path": file_path,
"md5": md5,
"sha1": sha1,
"sha256": sha256,
"sha512": sha512,
}
)
return package_files
def run(self) -> None:
self._adb_connect()
packages = self._adb_command("pm list packages -u -i -f")
if "java.lang.SecurityException" in packages or packages.strip() == "":
self._user_needed = True
packages = self._adb_command("pm list packages -u -i -f --user 0")
for line in packages.splitlines():
line = line.strip()
if not line.startswith("package:"):
continue
fields = line.split()
file_name, package_name = fields[0].split(":")[1].rsplit("=", 1)
try:
installer = fields[1].split("=")[1].strip()
except IndexError:
installer = None
else:
if installer == "null":
installer = None
package_files = self._get_files_for_package(package_name)
new_package = {
"package_name": package_name,
"file_name": file_name,
"installer": installer,
"disabled": False,
"system": False,
"third_party": False,
"files": package_files,
}
dumpsys_package = self._adb_command(f"dumpsys package {package_name}")
package_details = self.parse_package_for_details(dumpsys_package)
new_package.update(package_details)
self.results.append(new_package)
cmds = [
{"field": "disabled", "arg": "-d"},
{"field": "system", "arg": "-s"},
{"field": "third_party", "arg": "-3"},
]
for cmd in cmds:
command = f"pm list packages {cmd['arg']}"
if self._user_needed:
command += " --user 0"
output = self._adb_command(command)
for line in output.splitlines():
line = line.strip()
if not line.startswith("package:"):
continue
package_name = line.split(":", 1)[1]
for i, result in enumerate(self.results):
if result["package_name"] == package_name:
self.results[i][cmd["field"]] = True
for result in self.results:
if not result["third_party"]:
continue
dangerous_permissions_count = 0
for perm in result["requested_permissions"]:
if perm in DANGEROUS_PERMISSIONS:
dangerous_permissions_count += 1
if dangerous_permissions_count >= DANGEROUS_PERMISSIONS_THRESHOLD:
self.log.info(
'Third-party package "%s" requested %d '
"potentially dangerous permissions",
result["package_name"],
dangerous_permissions_count,
)
packages_to_lookup = []
for result in self.results:
if result["system"]:
continue
packages_to_lookup.append(result)
self.log.info(
'Found non-system package with name "%s" installed by "%s" on %s',
result["package_name"],
result["installer"],
result["timestamp"],
)
if not self.module_options.get("fast_mode", None):
self.check_virustotal(packages_to_lookup)
self.log.info(
"Extracted at total of %d installed package names", len(self.results)
)
self._adb_disconnect()
-43
View File
@@ -1,43 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from mvt.android.artifacts.processes import Processes as ProcessesArtifact
from .base import AndroidExtraction
from mvt.common.module_types import ModuleResults
class Processes(ProcessesArtifact, AndroidExtraction):
"""This module extracts details on running processes."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def run(self) -> None:
self._adb_connect()
output = self._adb_command("ps -A")
self.parse(output)
self._adb_disconnect()
self.log.info("Extracted records on a total of %d processes", len(self.results))
@@ -1,71 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from .base import AndroidExtraction
from mvt.common.module_types import ModuleResults
class RootBinaries(AndroidExtraction):
"""This module extracts the list of installed packages."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def check_indicators(self) -> None:
for root_binary in self.results:
self.detected.append(root_binary)
self.log.warning('Found root binary "%s"', root_binary)
def run(self) -> None:
root_binaries = [
"su",
"busybox",
"supersu",
"Superuser.apk",
"KingoUser.apk",
"SuperSu.apk",
"magisk",
"magiskhide",
"magiskinit",
"magiskpolicy",
]
self._adb_connect()
for root_binary in root_binaries:
root_binary = root_binary.strip()
if not root_binary:
continue
output = self._adb_command(f"which -a {root_binary}")
output = output.strip()
if not output:
continue
if "which: not found" in output:
continue
self.results.append(root_binary)
self._adb_disconnect()
@@ -1,49 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from .base import AndroidExtraction
from mvt.common.module_types import ModuleResults
class SELinuxStatus(AndroidExtraction):
"""This module checks if SELinux is being enforced."""
slug = "selinux_status"
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
self.results = {} if not results else results
def run(self) -> None:
self._adb_connect()
output = self._adb_command("getenforce")
self._adb_disconnect()
status = output.lower().strip()
self.results["status"] = status
if status == "enforcing":
self.log.info("SELinux is being regularly enforced")
else:
self.log.warning('SELinux status is "%s"!', status)
-59
View File
@@ -1,59 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from mvt.android.artifacts.settings import Settings as SettingsArtifact
from mvt.common.module_types import ModuleResults
from .base import AndroidExtraction
class Settings(SettingsArtifact, AndroidExtraction):
"""This module extracts Android system settings."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
self.results = {} if not results else results
def run(self) -> None:
self._adb_connect()
for namespace in ["system", "secure", "global"]:
out = self._adb_command(f"cmd settings list {namespace}")
if not out:
continue
self.results[namespace] = {}
for line in out.splitlines():
line = line.strip()
if line == "":
continue
fields = line.split("=", 1)
try:
self.results[namespace][fields[0]] = fields[1]
except IndexError:
continue
self._adb_disconnect()
-187
View File
@@ -1,187 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
import os
import sqlite3
from typing import Optional
from mvt.android.parsers.backup import AndroidBackupParsingError, parse_tar_for_sms
from mvt.common.module import InsufficientPrivileges
from mvt.common.utils import check_for_links, convert_unix_to_iso
from mvt.common.module_types import (
ModuleAtomicResult,
ModuleResults,
ModuleSerializedResult,
)
from .base import AndroidExtraction
SMS_BUGLE_PATH = "data/data/com.google.android.apps.messaging/databases/bugle_db"
SMS_BUGLE_QUERY = """
SELECT
ppl.normalized_destination AS address,
p.timestamp AS timestamp,
CASE WHEN m.sender_id IN
(SELECT _id FROM participants WHERE contact_id=-1)
THEN 2 ELSE 1 END incoming, p.text AS body
FROM messages m, conversations c, parts p,
participants ppl, conversation_participants cp
WHERE (m.conversation_id = c._id)
AND (m._id = p.message_id)
AND (cp.conversation_id = c._id)
AND (cp.participant_id = ppl._id);
"""
SMS_MMSSMS_PATH = "data/data/com.android.providers.telephony/databases/mmssms.db"
SMS_MMSMS_QUERY = """
SELECT
address AS address,
date_sent AS timestamp,
type as incoming,
body AS body
FROM sms;
"""
class SMS(AndroidExtraction):
"""This module extracts all SMS messages."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
self.sms_db_type = 0
def serialize(self, record: ModuleAtomicResult) -> ModuleSerializedResult:
body = record["body"].replace("\n", "\\n")
return {
"timestamp": record["isodate"],
"module": self.__class__.__name__,
"event": f"sms_{record['direction']}",
"data": f'{record.get("address", "unknown source")}: "{body}"',
}
def check_indicators(self) -> None:
if not self.indicators:
return
for message in self.results:
if "body" not in message:
continue
message_links = message.get("links", [])
if message_links == []:
message_links = check_for_links(message["body"])
ioc_match = self.indicators.check_urls(message_links)
if ioc_match:
message["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(
self.get_slug(), ioc_match.message, "", message
)
def _parse_db(self, db_path: str) -> None:
"""Parse an Android bugle_db SMS database file.
:param db_path: Path to the Android SMS database file to process
"""
conn = sqlite3.connect(db_path)
cur = conn.cursor()
if self.sms_db_type == 1:
cur.execute(SMS_BUGLE_QUERY)
elif self.sms_db_type == 2:
cur.execute(SMS_MMSMS_QUERY)
names = [description[0] for description in cur.description]
for item in cur:
message = {}
for index, value in enumerate(item):
message[names[index]] = value
message["direction"] = "received" if message["incoming"] == 1 else "sent"
message["isodate"] = convert_unix_to_iso(message["timestamp"])
# Extract links in the message body
body = message.get("body", None)
if body:
links = check_for_links(message["body"])
message["links"] = links
self.results.append(message)
cur.close()
conn.close()
self.log.info("Extracted a total of %d SMS messages", len(self.results))
def _extract_sms_adb(self) -> None:
"""Use the Android backup command to extract SMS data from the native
SMS app.
It is crucial to use the under-documented "-nocompress" flag to disable
the non-standard Java compression algorithm. This module only supports
an unencrypted ADB backup.
"""
backup_tar = self._generate_backup("com.android.providers.telephony")
if not backup_tar:
return
try:
self.results = parse_tar_for_sms(backup_tar)
except AndroidBackupParsingError:
self.log.info(
"Impossible to read SMS from the Android Backup, "
"please extract the SMS and try extracting it with "
"Android Backup Extractor"
)
return
self.log.info("Extracted a total of %d SMS messages", len(self.results))
def run(self) -> None:
self._adb_connect()
try:
if self._adb_check_file_exists(os.path.join("/", SMS_BUGLE_PATH)):
self.sms_db_type = 1
self._adb_process_file(
os.path.join("/", SMS_BUGLE_PATH), self._parse_db
)
elif self._adb_check_file_exists(os.path.join("/", SMS_MMSSMS_PATH)):
self.sms_db_type = 2
self._adb_process_file(
os.path.join("/", SMS_MMSSMS_PATH), self._parse_db
)
self._adb_disconnect()
return
except InsufficientPrivileges:
pass
self.log.info(
"No SMS database found. Trying extraction of SMS data "
"using Android backup feature."
)
self._extract_sms_adb()
self._adb_disconnect()
-118
View File
@@ -1,118 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import base64
import logging
import os
import sqlite3
from typing import Optional
from mvt.common.utils import check_for_links, convert_unix_to_iso
from .base import AndroidExtraction
from mvt.common.module_types import (
ModuleAtomicResult,
ModuleSerializedResult,
ModuleResults,
)
WHATSAPP_PATH = "data/data/com.whatsapp/databases/msgstore.db"
class Whatsapp(AndroidExtraction):
"""This module extracts all WhatsApp messages containing links."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: ModuleResults = [],
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def serialize(self, record: ModuleAtomicResult) -> ModuleSerializedResult:
text = record["data"].replace("\n", "\\n")
return {
"timestamp": record["isodate"],
"module": self.__class__.__name__,
"event": f"whatsapp_msg_{record['direction']}",
"data": f'"{text}"',
}
def check_indicators(self) -> None:
if not self.indicators:
return
for message in self.results:
if "data" not in message:
continue
message_links = check_for_links(message["data"])
if self.indicators.check_urls(message_links):
self.detected.append(message)
continue
def _parse_db(self, db_path: str) -> None:
"""Parse an Android msgstore.db WhatsApp database file.
:param db_path: Path to the Android WhatsApp database file to process
"""
conn = sqlite3.connect(db_path)
cur = conn.cursor()
cur.execute(
"""
SELECT * FROM messages;
"""
)
names = [description[0] for description in cur.description]
messages = []
for item in cur:
message = {}
for index, value in enumerate(item):
message[names[index]] = value
if not message["data"]:
continue
message["direction"] = "send" if message["key_from_me"] == 1 else "received"
message["isodate"] = convert_unix_to_iso(message["timestamp"])
# If we find links in the messages or if they are empty we add them
# to the list.
if check_for_links(message["data"]) or message["data"].strip() == "":
if message.get("thumb_image"):
message["thumb_image"] = base64.b64encode(message["thumb_image"])
messages.append(message)
cur.close()
conn.close()
self.log.info(
"Extracted a total of %d WhatsApp messages containing links", len(messages)
)
self.results = messages
def run(self) -> None:
self._adb_connect()
try:
self._adb_process_file(os.path.join("/", WHATSAPP_PATH), self._parse_db)
except Exception as exc:
self.log.error(exc)
self._adb_disconnect()
@@ -3,11 +3,13 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from .aqf_files import AQFFiles
from .aqf_getprop import AQFGetProp
from .aqf_packages import AQFPackages
from .aqf_processes import AQFProcesses
from .aqf_settings import AQFSettings
from .aqf_files import AQFFiles
from .mounts import Mounts
from .root_binaries import RootBinaries
from .sms import SMS
ANDROIDQF_MODULES = [
@@ -17,4 +19,6 @@ ANDROIDQF_MODULES = [
AQFSettings,
AQFFiles,
SMS,
RootBinaries,
Mounts,
]
@@ -15,8 +15,8 @@ from typing import Optional
from mvt.android.modules.androidqf.base import AndroidQFModule
from mvt.common.module_types import (
ModuleResults,
ModuleAtomicResult,
ModuleResults,
ModuleSerializedResult,
)
from mvt.common.utils import convert_datetime_to_iso
@@ -89,9 +89,9 @@ class AQFFiles(AndroidQFModule):
for result in self.results:
ioc_match = self.indicators.check_file_path(result["path"])
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.log_latest()
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
continue
# NOTE: Update with final path used for Android collector.
@@ -105,16 +105,16 @@ class AQFFiles(AndroidQFModule):
file_type = "executable "
msg = f'Found {file_type}file at suspicious path "{result["path"]}"'
self.alertstore.high(self.get_slug(), msg, "", result)
self.alertstore.log_latest()
self.alertstore.high(msg, "", result)
if result.get("sha256", "") == "":
continue
ioc_match = self.indicators.check_file_hash(result["sha256"])
ioc_match = self.indicators.check_file_hash(result.get("sha256") or "")
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
# TODO: adds SHA1 and MD5 when available in MVT
@@ -7,9 +7,9 @@ import logging
from typing import Optional
from mvt.android.artifacts.getprop import GetProp as GetPropArtifact
from mvt.common.module_types import ModuleResults
from .base import AndroidQFModule
from mvt.common.module_types import ModuleResults
class AQFGetProp(GetPropArtifact, AndroidQFModule):
@@ -32,7 +32,7 @@ class AQFGetProp(GetPropArtifact, AndroidQFModule):
log=log,
results=results,
)
self.results = []
self.results: list = []
def run(self) -> None:
getprop_files = self._get_files_by_pattern("*/getprop.txt")
@@ -3,15 +3,16 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import os
import datetime
import logging
import os
from typing import Optional
from mvt.common.utils import convert_datetime_to_iso
from mvt.common.module_types import ModuleResults
from .base import AndroidQFModule
from mvt.android.artifacts.file_timestamps import FileTimestampsArtifact
from mvt.common.module_types import ModuleResults
from mvt.common.utils import convert_datetime_to_iso
from .base import AndroidQFModule
class AQFLogTimestamps(FileTimestampsArtifact, AndroidQFModule):
@@ -37,11 +38,13 @@ class AQFLogTimestamps(FileTimestampsArtifact, AndroidQFModule):
results=results,
)
def _get_file_modification_time(self, file_path: str) -> dict:
def _get_file_modification_time(self, file_path: str) -> datetime.datetime:
if self.archive:
file_timetuple = self.archive.getinfo(file_path).date_time
return datetime.datetime(*file_timetuple)
else:
if not self.parent_path:
raise ValueError("parent_path is not set")
file_stat = os.stat(os.path.join(self.parent_path, file_path))
return datetime.datetime.fromtimestamp(file_stat.st_mtime)
@@ -11,13 +11,13 @@ from mvt.android.utils import (
BROWSER_INSTALLERS,
PLAY_STORE_INSTALLERS,
ROOT_PACKAGES,
THIRD_PARTY_STORE_INSTALLERS,
SECURITY_PACKAGES,
SYSTEM_UPDATE_PACKAGES,
THIRD_PARTY_STORE_INSTALLERS,
)
from mvt.common.module_types import ModuleResults
from .base import AndroidQFModule
from mvt.common.module_types import ModuleResults
class AQFPackages(AndroidQFModule):
@@ -45,79 +45,67 @@ class AQFPackages(AndroidQFModule):
for result in self.results:
if result["name"] in ROOT_PACKAGES:
self.alertstore.medium(
self.get_slug(),
f'Found an installed package related to rooting/jailbreaking: "{result["name"]}"',
"",
result,
)
self.alertstore.log_latest()
continue
# Detections for apps installed via unusual methods.
if result["installer"] in THIRD_PARTY_STORE_INSTALLERS:
self.alertstore.info(
self.get_slug(),
f'Found a package installed via a third party store (installer="{result["installer"]}"): "{result["name"]}"',
"",
result,
)
self.alertstore.log_latest()
elif result["installer"] in BROWSER_INSTALLERS:
self.alertstore.medium(
self.get_slug(),
f'Found a package installed via a browser (installer="{result["installer"]}"): "{result["name"]}"',
"",
result,
)
self.alertstore.log_latest()
elif result["installer"] == "null" and result["system"] is False:
self.alertstore.high(
self.get_slug(),
self.alertstore.medium(
f'Found a non-system package installed via adb or another method: "{result["name"]}"',
"",
result,
)
self.alertstore.log_latest()
elif result["installer"] in PLAY_STORE_INSTALLERS:
pass
# Check for disabled security or software update packages.
package_disabled = result.get("disabled", None)
if result["name"] in SECURITY_PACKAGES and package_disabled:
self.alertstore.high(
self.get_slug(),
self.alertstore.medium(
f'Security package "{result["name"]}" disabled on the phone',
"",
result,
)
self.alertstore.log_latest()
if result["name"] in SYSTEM_UPDATE_PACKAGES and package_disabled:
self.alertstore.high(
self.get_slug(),
self.alertstore.medium(
f'System OTA update package "{result["name"]}" disabled on the phone',
"",
result,
)
self.alertstore.log_latest()
if not self.indicators:
continue
ioc_match = self.indicators.check_app_id(result.get("name"))
ioc_match = self.indicators.check_app_id(result.get("name") or "")
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(self.get_slug(), ioc_match.message, "", result)
self.alertstore.log_latest()
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
for package_file in result.get("files", []):
ioc_match = self.indicators.check_file_hash(package_file["sha256"])
ioc_match = self.indicators.check_file_hash(
package_file.get("sha256") or ""
)
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(
self.get_slug(), ioc_match.message, "", result
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
)
self.alertstore.log_latest()
if "certificate" not in package_file:
continue
@@ -129,11 +117,12 @@ class AQFPackages(AndroidQFModule):
certificate_hash
)
if ioc_match:
result["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(
self.get_slug(), ioc_match.message, "", result
ioc_match.message,
"",
result,
matched_indicator=ioc_match.ioc,
)
self.alertstore.log_latest()
break
def run(self) -> None:
@@ -7,9 +7,9 @@ import logging
from typing import Optional
from mvt.android.artifacts.settings import Settings as SettingsArtifact
from mvt.common.module_types import ModuleResults
from .base import AndroidQFModule
from mvt.common.module_types import ModuleResults
class AQFSettings(SettingsArtifact, AndroidQFModule):
@@ -32,7 +32,7 @@ class AQFSettings(SettingsArtifact, AndroidQFModule):
log=log,
results=results,
)
self.results = {}
self.results: dict = {}
def run(self) -> None:
for setting_file in self._get_files_by_pattern("*/settings_*.txt"):
+2 -2
View File
@@ -33,8 +33,8 @@ class AndroidQFModule(MVTModule):
log=log,
results=results,
)
self.parent_path = None
self._path: str = target_path
self.parent_path: Optional[str] = None
self._path: Optional[str] = target_path
self.files: List[str] = []
self.archive: Optional[zipfile.ZipFile] = None
@@ -0,0 +1,74 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import json
import logging
from typing import Optional
from mvt.android.artifacts.mounts import Mounts as MountsArtifact
from .base import AndroidQFModule
class Mounts(MountsArtifact, AndroidQFModule):
"""This module extracts and analyzes mount information from AndroidQF acquisitions."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
self.results: list = []
def run(self) -> None:
"""
Run the mounts analysis module.
This module looks for mount information files collected by androidqf
and analyzes them for suspicious configurations, particularly focusing
on detecting root access indicators like /system mounted as read-write.
"""
mount_files = self._get_files_by_pattern("*/mounts.json")
if not mount_files:
self.log.info("No mount information file found")
return
self.log.info("Found mount information file: %s", mount_files[0])
try:
data = self._get_file_content(mount_files[0]).decode(
"utf-8", errors="replace"
)
except Exception as exc:
self.log.error("Failed to read mount information file: %s", exc)
return
# Parse the mount data
try:
json_data = json.loads(data)
if isinstance(json_data, list):
# AndroidQF format: array of strings like
# "/dev/block/dm-12 on / type ext4 (ro,seclabel,noatime)"
mount_content = "\n".join(json_data)
self.parse(mount_content)
except Exception as exc:
self.log.error("Failed to parse mount information: %s", exc)
return
self.log.info("Extracted a total of %d mount entries", len(self.results))
@@ -0,0 +1,120 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import json
import logging
from typing import Optional
from .base import AndroidQFModule
class RootBinaries(AndroidQFModule):
"""This module analyzes root_binaries.json for root binaries found by androidqf."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def serialize(self, record: dict) -> dict:
return {
"timestamp": record.get("timestamp"),
"module": self.__class__.__name__,
"event": "root_binary_found",
"data": f"Root binary found: {record['path']} (binary: {record['binary_name']})",
}
def check_indicators(self) -> None:
"""Check for indicators of device rooting."""
if not self.results:
return
# All found root binaries are considered indicators of rooting
for result in self.results:
self.alertstore.high(
f'Found root binary "{result["binary_name"]}" at path "{result["path"]}"',
"",
result,
)
if self.results:
self.log.warning(
"Device shows signs of rooting with %d root binaries found",
len(self.results),
)
def run(self) -> None:
"""Run the root binaries analysis."""
root_binaries_files = self._get_files_by_pattern("*/root_binaries.json")
if not root_binaries_files:
self.log.info("No root_binaries.json file found")
return
rawdata = self._get_file_content(root_binaries_files[0]).decode(
"utf-8", errors="ignore"
)
try:
root_binary_paths = json.loads(rawdata)
except json.JSONDecodeError as e:
self.log.error("Failed to parse root_binaries.json: %s", e)
return
if not isinstance(root_binary_paths, list):
self.log.error("Expected root_binaries.json to contain a list of paths")
return
# Known root binary names that might be found and their descriptions
# This maps the binary name to a human-readable description
known_root_binaries = {
"su": "SuperUser binary",
"busybox": "BusyBox utilities",
"supersu": "SuperSU root management",
"Superuser.apk": "Superuser app",
"KingoUser.apk": "KingRoot app",
"SuperSu.apk": "SuperSU app",
"magisk": "Magisk root framework",
"magiskhide": "Magisk hide utility",
"magiskinit": "Magisk init binary",
"magiskpolicy": "Magisk policy binary",
}
for path in root_binary_paths:
if not path or not isinstance(path, str):
continue
# Extract binary name from path
binary_name = path.split("/")[-1].lower()
# Check if this matches a known root binary by exact name match
description = "Unknown root binary"
for known_binary in known_root_binaries:
if binary_name == known_binary.lower():
description = known_root_binaries[known_binary]
break
result = {
"path": path.strip(),
"binary_name": binary_name,
"description": description,
}
self.results.append(result)
self.log.info("Found %d root binaries", len(self.results))
+1 -2
View File
@@ -55,9 +55,8 @@ class SMS(AndroidQFModule):
ioc_match = self.indicators.check_domains(message.get("links", []))
if ioc_match:
message["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(
self.get_slug(), ioc_match.message, "", message
ioc_match.message, "", message, matched_indicator=ioc_match.ioc
)
def parse_backup(self, data):
+10 -7
View File
@@ -9,7 +9,7 @@ import os
from tarfile import TarFile
from typing import List, Optional
from mvt.common.module import MVTModule, ModuleResults
from mvt.common.module import ModuleResults, MVTModule
class BackupModule(MVTModule):
@@ -32,10 +32,10 @@ class BackupModule(MVTModule):
log=log,
results=results,
)
self.ab = None
self.backup_path = None
self.tar = None
self.files = []
self.ab: Optional[str] = None
self.backup_path: Optional[str] = None
self.tar: Optional[TarFile] = None
self.files: list = []
def from_dir(self, backup_path: Optional[str], files: List[str]) -> None:
self.backup_path = backup_path
@@ -55,12 +55,15 @@ class BackupModule(MVTModule):
return fnmatch.filter(self.files, pattern)
def _get_file_content(self, file_path: str) -> bytes:
handle = None
if self.tar:
try:
member = self.tar.getmember(file_path)
handle = self.tar.extractfile(member)
if not handle:
raise ValueError(f"Could not extract file: {file_path}")
except KeyError:
return None
handle = self.tar.extractfile(member)
raise FileNotFoundError(f"File not found in tar: {file_path}")
elif self.backup_path:
handle = open(os.path.join(self.backup_path, file_path), "rb")
else:
+4 -5
View File
@@ -4,12 +4,12 @@
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from typing import Any, Optional
from mvt.android.modules.backup.base import BackupModule
from mvt.android.parsers.backup import parse_sms_file
from mvt.common.utils import check_for_links
from mvt.common.module_types import ModuleResults
from mvt.common.utils import check_for_links
class SMS(BackupModule):
@@ -30,7 +30,7 @@ class SMS(BackupModule):
log=log,
results=results,
)
self.results = []
self.results: list[dict[str, Any]] = []
def check_indicators(self) -> None:
if not self.indicators:
@@ -46,9 +46,8 @@ class SMS(BackupModule):
ioc_match = self.indicators.check_urls(message_links)
if ioc_match:
message["matched_indicator"] = ioc_match.ioc
self.alertstore.critical(
self.get_slug(), ioc_match.message, "", message
ioc_match.message, "", message, matched_indicator=ioc_match.ioc
)
continue
+20 -9
View File
@@ -6,11 +6,11 @@ import datetime
import fnmatch
import logging
import os
from pathlib import Path
from typing import List, Optional
from zipfile import ZipFile
from mvt.common.module import MVTModule, ModuleResults
from mvt.common.module import ModuleResults, MVTModule
class BugReportModule(MVTModule):
@@ -69,14 +69,19 @@ class BugReportModule(MVTModule):
if self.zip_archive:
handle = self.zip_archive.open(file_path)
else:
handle = open(os.path.join(self.extract_path, file_path), "rb")
if not self.extract_path:
raise ValueError("extract_path is not set")
joined = os.path.join(self.extract_path, file_path)
if not Path(joined).resolve().is_relative_to(Path(self.extract_path).resolve()):
raise ValueError("unsafe file_path")
handle = open(joined, "rb")
data = handle.read()
handle.close()
return data
def _get_dumpstate_file(self) -> bytes:
def _get_dumpstate_file(self) -> Optional[bytes]:
main = self._get_files_by_pattern("main_entry.txt")
if main:
main_content = self._get_file_content(main[0])
@@ -84,17 +89,23 @@ class BugReportModule(MVTModule):
return self._get_file_content(main_content.decode().strip())
except KeyError:
return None
else:
dumpstate_logs = self._get_files_by_pattern("dumpState_*.log")
if not dumpstate_logs:
return None
dumpstate_logs = self._get_files_by_pattern("dumpState_*.log")
if dumpstate_logs:
return self._get_file_content(dumpstate_logs[0])
def _get_file_modification_time(self, file_path: str) -> dict:
dumpsys_files = self._get_files_by_pattern("*/dumpsys.txt")
if dumpsys_files:
return self._get_file_content(dumpsys_files[0])
return None
def _get_file_modification_time(self, file_path: str) -> datetime.datetime:
if self.zip_archive:
file_timetuple = self.zip_archive.getinfo(file_path).date_time
return datetime.datetime(*file_timetuple)
else:
if not self.extract_path:
raise ValueError("extract_path is not set")
file_stat = os.stat(os.path.join(self.extract_path, file_path))
return datetime.datetime.fromtimestamp(file_stat.st_mtime)
@@ -6,9 +6,9 @@
import logging
from typing import Optional
from mvt.common.module_types import ModuleResults
from mvt.android.artifacts.dumpsys_packages import DumpsysPackagesArtifact
from mvt.android.utils import DANGEROUS_PERMISSIONS, DANGEROUS_PERMISSIONS_THRESHOLD
from mvt.common.module_types import ModuleResults
from .base import BugReportModule
@@ -43,8 +43,9 @@ class DumpsysPackages(DumpsysPackagesArtifact, BugReportModule):
)
return
data = data.decode("utf-8", errors="replace")
content = self.extract_dumpsys_section(data, "DUMP OF SERVICE package:")
content = self.extract_dumpsys_section(
data.decode("utf-8", errors="replace"), "DUMP OF SERVICE package:"
)
self.parse(content)
for result in self.results:
@@ -35,6 +35,24 @@ class DumpsysReceivers(DumpsysReceiversArtifact, BugReportModule):
self.results = results if results else {}
def check_indicators(self) -> None:
for result in self.results:
if self.indicators:
receiver_name = self.results[result][0]["receiver"]
# return IoC if the stix2 process name a substring of the receiver name
ioc_match = self.indicators.check_receiver_prefix(receiver_name)
if ioc_match:
self.alertstore.critical(
ioc_match.message,
"",
self.results[result][0],
matched_indicator=ioc_match.ioc,
)
continue
def run(self) -> None:
content = self._get_dumpstate_file()
if not content:
@@ -0,0 +1,20 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from .connect_event import ConnectEvent
from .dns_event import DnsEvent
from .security_event import SecurityEvent
INTRUSION_LOGS_MODULES = [
DnsEvent,
ConnectEvent,
SecurityEvent,
]
KNOWN_INTRUSION_LOG_EVENT_TYPES = {
"connect_event",
"dns_event",
"security_event",
}
@@ -0,0 +1,395 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import datetime
import io
import json
import logging
import zipfile
from pathlib import Path
from typing import Optional, Union
try:
import zoneinfo
except ImportError:
from backports import zoneinfo # type: ignore[no-redef]
from mvt.common.module import MVTModule
from mvt.common.utils import convert_datetime_to_iso, convert_unix_to_iso
class IntrusionLogsModule(MVTModule):
"""Base class for modules analyzing intrusion logs (newline-delimited JSON).
Performance note
----------------
Log files can be large and are shared by every module in this package.
To avoid re-reading and re-parsing the same files N times (once per
module), the command layer should call :meth:`load_all_events` exactly
once and then assign the returned dict to the ``il_events_by_type``
attribute of every module instance **before** calling ``run_module``.
When ``il_events_by_type`` is populated:
* :meth:`collect_txt` becomes a no-op (no disk I/O).
* :meth:`parse_collected_txt` iterates the in-memory list for the
requested event type instead of re-parsing raw text.
Modules that are used standalone (e.g. in tests) still work as before
because ``il_events_by_type`` defaults to ``None``, which preserves the
original file-loading code path.
"""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
# Raw file content collected by collect_txt (fallback path only).
self.il_files: list[tuple[str, str]] = []
# Pre-parsed events injected by the command layer.
# Keys are event-type strings (e.g. "dns_event"), values are lists of
# raw event-data dicts exactly as they appear in the JSON lines.
# When this is not None, collect_txt and parse_collected_txt use it
# instead of touching the file system.
self.il_events_by_type: Optional[dict[str, list[dict]]] = None
# ------------------------------------------------------------------
# Serialization helper
# ------------------------------------------------------------------
def serialize(self, record: dict) -> Union[dict, list]:
"""Serialize a record for timeline output."""
return {
"timestamp": record.get("timestamp", record.get("isodate")),
"module": self.__class__.__name__,
"event": record.get("event_type", ""),
"data": str(record),
}
# ------------------------------------------------------------------
# File collection
# ------------------------------------------------------------------
def collect_txt(self, source) -> None:
"""Collect text log files from *source* into ``self.il_files``.
Entry points:
* directory → walk recursively
* zip file → walk zip entries
* anything else → silently skip
If ``self.il_events_by_type`` has already been populated (i.e. the
command layer pre-loaded the events), this method returns immediately
without any disk I/O.
"""
if self.il_events_by_type is not None:
self.log.debug(
"Pre-loaded events available — skipping file collection for %s",
self.__class__.__name__,
)
return
path = Path(source)
if path.is_dir():
self._walk_directory(path)
return
if path.is_file() and path.suffix.lower() == ".zip":
try:
with zipfile.ZipFile(path) as z:
self._walk_zip(z)
except zipfile.BadZipFile:
self.log.debug("Skipping invalid zip: %s", path)
return
self.log.debug("Skipping unsupported source: %s", source)
def _walk_directory(self, root: Path, prefix: str = "") -> None:
for item in root.iterdir():
if item.is_dir():
self._walk_directory(item, prefix=f"{prefix}{item.name}/")
continue
if item.suffix.lower() == ".txt":
self.il_files.append(
(f"{prefix}{item.name}", item.read_text(errors="ignore"))
)
elif item.suffix.lower() == ".zip":
try:
with zipfile.ZipFile(item) as z:
self._walk_zip(z, prefix=f"{prefix}{item.name}::")
except zipfile.BadZipFile:
self.log.warning("Skipping invalid zip: %s", item)
def _walk_zip(self, zf: zipfile.ZipFile, prefix: str = "") -> None:
for info in zf.infolist():
if info.is_dir():
continue
name = info.filename
with zf.open(info) as f:
data = f.read()
if name.lower().endswith(".txt"):
self.il_files.append((f"{prefix}{name}", data.decode(errors="ignore")))
elif name.lower().endswith(".zip"):
with zipfile.ZipFile(io.BytesIO(data)) as inner:
self._walk_zip(inner, prefix=f"{prefix}{name}::")
# ------------------------------------------------------------------
# Single-pass loader (used by the command layer)
# ------------------------------------------------------------------
def load_all_events(self, source) -> dict[str, list[dict]]:
"""Read every log file under *source* **once** and parse all JSON
lines in a single pass, routing events into per-type buckets.
Returns a ``dict`` mapping *event_type* strings to lists of raw
event-data dicts. The result is also stored in
``self.il_events_by_type`` so that subsequent calls to
:meth:`collect_txt` and :meth:`parse_collected_txt` on *this*
instance are no-ops.
Intended usage in the command layer::
loader = IntrusionLogsModule(target_path=target, log=log)
all_events = loader.load_all_events(target)
for module_cls in INTRUSION_LOGS_MODULES:
m = module_cls(target_path=target, ...)
m.il_events_by_type = all_events # inject — no re-reading
run_module(m)
"""
# Reset so that _collect_txt actually runs (il_events_by_type is None).
self.il_events_by_type = None
self.il_files = []
self.collect_txt(source)
events_by_type: dict[str, list[dict]] = {}
# JSON fingerprints used to drop events that appear in more than one
# log file (overlapping daily files are the most common source of
# cross-file duplicates).
seen_fingerprints: set[str] = set()
total_lines = 0
skipped_lines = 0
duplicate_lines = 0
for file_name, text in self.il_files:
for line_num, line in enumerate(text.splitlines(), start=1):
line = line.strip()
if not line:
continue
total_lines += 1
try:
entry = json.loads(line)
for event_type, event_data in entry.items():
if isinstance(event_data, dict):
fingerprint = json.dumps(event_data, sort_keys=True)
if fingerprint in seen_fingerprints:
duplicate_lines += 1
continue
seen_fingerprints.add(fingerprint)
events_by_type.setdefault(event_type, []).append(event_data)
except json.JSONDecodeError as e:
skipped_lines += 1
self.log.warning(
"Failed to parse JSON on line %d in %s: %s",
line_num,
file_name,
e,
)
except Exception as e:
skipped_lines += 1
self.log.warning(
"Error processing line %d in %s: %s",
line_num,
file_name,
e,
)
if duplicate_lines:
self.log.info(
"Removed %d duplicate event(s) seen across multiple log files",
duplicate_lines,
)
self.log.info(
"Loaded %d log files, parsed %d lines (%d skipped), found event types: %s",
len(self.il_files),
total_lines,
skipped_lines,
{k: len(v) for k, v in events_by_type.items()},
)
# Cache so this instance also benefits from the fast path.
self.il_events_by_type = events_by_type
return events_by_type
# ------------------------------------------------------------------
# Parsing
# ------------------------------------------------------------------
def parse_collected_txt(self, event_type: str) -> None:
"""Parse collected log text and dispatch events of *event_type*.
Fast path
~~~~~~~~~
When ``self.il_events_by_type`` is populated (injected by the command
layer after a single shared :meth:`load_all_events` call), the method
iterates the already-parsed in-memory list for *event_type* — no
re-reading, no re-parsing of JSON.
Fallback path
~~~~~~~~~~~~~
When ``self.il_events_by_type`` is ``None``, the method falls back to
iterating ``self.il_files`` and parsing each JSON line, which is the
original behaviour.
"""
if self.il_events_by_type is not None:
events = self.il_events_by_type.get(event_type, [])
self.log.debug(
"Using pre-loaded events: dispatching %d '%s' events",
len(events),
event_type,
)
for event_data in events:
try:
# Work on a shallow copy so that mutations in one module
# (e.g. adding "timestamp") do not affect other modules
# that share the same dict reference.
self.process_event(dict(event_data))
except Exception as e:
self.log.warning(
"Error processing pre-parsed '%s' event: %s",
event_type,
e,
)
return
# Fallback: parse raw text collected by collect_txt().
# Use the same JSON-fingerprint approach as MVTModule._deduplicate_timeline
# to drop events that appear verbatim in more than one log file.
seen_fingerprints: set[str] = set()
duplicate_count = 0
for file_name, text in self.il_files:
for line_num, line in enumerate(text.splitlines(), start=1):
line = line.strip()
if not line:
continue
try:
entry = json.loads(line)
if event_type in entry:
event_data = entry[event_type]
fingerprint = json.dumps(event_data, sort_keys=True)
if fingerprint in seen_fingerprints:
duplicate_count += 1
continue
seen_fingerprints.add(fingerprint)
event_data["event_type"] = event_type
self.process_event(event_data)
except json.JSONDecodeError as e:
self.log.warning(
"Failed to parse JSON on line %d in %s: %s",
line_num,
file_name,
str(e),
)
except Exception as e:
self.log.warning(
"Error processing line %d in %s: %s",
line_num,
file_name,
str(e),
)
if duplicate_count:
self.log.info(
"Removed %d duplicate '%s' event(s) seen across multiple log files",
duplicate_count,
event_type,
)
# ------------------------------------------------------------------
# Event processing
# ------------------------------------------------------------------
def process_event(self, event_data: dict) -> None:
"""Process an individual event. Override this in subclasses.
Args:
event_data: Dictionary containing the event data.
"""
self.results.append(event_data)
# ------------------------------------------------------------------
# Timestamp localisation
# ------------------------------------------------------------------
def _localize_timestamp(self, event_time_seconds: float) -> str:
"""Convert a Unix timestamp (in seconds) to an ISO string.
When the device timezone is available via ``module_options["device_timezone"]``
(a IANA timezone name such as ``"Europe/Paris"`` read from
``persist.sys.timezone`` in ``getprop.txt``), the UTC instant is
converted to the device's local time before formatting — mirroring the
approach used by ``AQFFiles``.
When no timezone is configured the method falls back to UTC, which is
consistent with all other MVT modules that call ``convert_unix_to_iso``.
Args:
event_time_seconds: Unix epoch timestamp expressed in **seconds**
(callers are responsible for dividing ms/ns values first).
Returns:
ISO-formatted datetime string (``YYYY-mm-dd HH:MM:SS.ffffff``).
The string always represents the device-local time (or UTC when no
timezone is known); no UTC offset suffix is appended, matching the
format produced by :func:`mvt.common.utils.convert_unix_to_iso`.
"""
tz_name: Optional[str] = self.module_options.get("device_timezone")
if tz_name:
try:
device_tz = zoneinfo.ZoneInfo(tz_name)
utc_dt = datetime.datetime.fromtimestamp(
event_time_seconds, tz=datetime.timezone.utc
)
local_dt = utc_dt.astimezone(device_tz)
# Strip tzinfo so that convert_datetime_to_iso outputs the
# local wall-clock time without a timezone suffix. This is
# the same pattern used by AQFFiles.
return convert_datetime_to_iso(local_dt.replace(tzinfo=None))
except Exception as e:
self.log.warning(
"Could not apply device timezone '%s', falling back to UTC: %s",
tz_name,
e,
)
return convert_unix_to_iso(event_time_seconds)
# ------------------------------------------------------------------
# Abstract interface
# ------------------------------------------------------------------
def run(self) -> None:
"""Main execution method. Must be implemented by subclasses."""
raise NotImplementedError("Subclasses must implement the run() method")
@@ -0,0 +1,121 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional, Union
from .base import IntrusionLogsModule
class ConnectEvent(IntrusionLogsModule):
"""This module analyzes network connection events from intrusion logs."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def check_indicators(self) -> None:
"""Check connection events against indicators of compromise."""
if not self.indicators:
return
for result in self.results:
# Check IP address against indicators
ip_address = result.get("ip_address", "")
if ip_address:
# Clean IP address (remove leading slash and extract IP from format like "ip6-localhost/::1")
if "/" in ip_address:
parts = ip_address.split("/")
clean_ip = parts[-1] if len(parts) > 1 else parts[0]
else:
clean_ip = ip_address.lstrip("/")
# Skip localhost addresses
if clean_ip and clean_ip not in ["::1", "127.0.0.1", "0.0.0.0"]:
ioc = self.indicators.check_domain(clean_ip)
if ioc:
result["matched_ip"] = clean_ip
self.alertstore.critical(
ioc.message,
result.get("timestamp") or "",
result,
matched_indicator=ioc.ioc,
)
# Check package name against app identifiers
package_name = result.get("package_name", "")
if package_name:
ioc = self.indicators.check_app_id(package_name)
if ioc:
self.alertstore.critical(
ioc.message,
result.get("timestamp") or "",
result,
matched_indicator=ioc.ioc,
)
def serialize(self, record: dict) -> Union[dict, list]:
"""Serialize a connection event record for timeline output."""
ip_address = record.get("ip_address", "")
port = record.get("port", 0)
package_name = record.get("package_name", "")
matched_ip = record.get("matched_ip", "")
# Clean IP address for display
if "/" in ip_address:
parts = ip_address.split("/")
clean_ip = parts[-1] if len(parts) > 1 else parts[0]
else:
clean_ip = ip_address.lstrip("/")
# Indicate when IP matched an IoC
if matched_ip:
data = f"Connection to {clean_ip}:{port} by {package_name} [Matched IP: {matched_ip}]"
else:
data = f"Connection to {clean_ip}:{port} by {package_name}"
return {
"timestamp": record.get("timestamp"),
"module": self.__class__.__name__,
"event": "network_connection",
"data": data,
}
def process_event(self, event_data: dict) -> None:
"""Process a connection event and add it to results."""
# Convert event_time from milliseconds to ISO format
event_time = event_data.get("event_time")
if event_time:
# Android event times are in milliseconds since epoch
event_data["timestamp"] = self._localize_timestamp(event_time / 1000.0)
else:
event_data["timestamp"] = None
self.results.append(event_data)
def run(self) -> None:
"""Extract and analyze connection events from intrusion logs."""
if not self.target_path:
self.log.error("No target path specified")
return
self.collect_txt(self.target_path)
self.parse_collected_txt("connect_event")
self.log.info("Identified %d connection events", len(self.results))
@@ -0,0 +1,141 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional, Union
from .base import IntrusionLogsModule
class DnsEvent(IntrusionLogsModule):
"""This module analyzes DNS events from intrusion logs."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def check_indicators(self) -> None:
"""Check DNS events against indicators of compromise."""
if not self.indicators:
return
for result in self.results:
# Check hostname against domain indicators
hostname = result.get("hostname", "")
if hostname:
ioc = self.indicators.check_domain(hostname)
if ioc:
self.alertstore.critical(
ioc.message,
result.get("timestamp") or "",
result,
matched_indicator=ioc.ioc,
)
# Check IP addresses against indicators
ip_addresses = result.get("ip_addresses", [])
matched_ips = []
for ip_addr in ip_addresses:
# Remove leading slash if present
clean_ip = (
ip_addr.lstrip("/") if isinstance(ip_addr, str) else str(ip_addr)
)
if clean_ip and clean_ip != "0.0.0.0":
ioc = self.indicators.check_domain(clean_ip)
if ioc:
matched_ips.append(clean_ip)
self.alertstore.critical(
ioc.message,
result.get("timestamp") or "",
result,
matched_indicator=ioc.ioc,
)
# Store matched IPs for timeline display
if matched_ips:
result["matched_ips"] = matched_ips
# Check package name against app identifiers
package_name = result.get("package_name", "")
if package_name:
ioc = self.indicators.check_app_id(package_name)
if ioc:
self.alertstore.critical(
ioc.message,
result.get("timestamp") or "",
result,
matched_indicator=ioc.ioc,
)
def serialize(self, record: dict) -> Union[dict, list]:
"""Serialize a DNS event record for timeline output."""
hostname = record.get("hostname", "")
package_name = record.get("package_name", "")
# Get IP addresses for display
ip_addresses = record.get("ip_addresses", [])
matched_ips = record.get("matched_ips", [])
# Clean up IP addresses (remove leading slashes)
clean_ips = []
for ip_addr in ip_addresses:
clean_ip = ip_addr.lstrip("/") if isinstance(ip_addr, str) else str(ip_addr)
if clean_ip and clean_ip != "0.0.0.0":
clean_ips.append(clean_ip)
# Build the data string with actual IPs
if matched_ips:
# Highlight matched IPs in the output
ip_display = ", ".join(matched_ips)
data = f"DNS query for {hostname} by {package_name} [Matched IPs: {ip_display}]"
elif clean_ips:
ip_display = ", ".join(clean_ips)
data = f"DNS query for {hostname} by {package_name} [IPs: {ip_display}]"
else:
data = f"DNS query for {hostname} by {package_name}"
return {
"timestamp": record.get("timestamp"),
"module": self.__class__.__name__,
"event": "dns_query",
"data": data,
}
def process_event(self, event_data: dict) -> None:
"""Process a DNS event and add it to results."""
# Convert event_time from milliseconds to ISO format
event_time = event_data.get("event_time")
if event_time:
# Android event times are in milliseconds since epoch
event_data["timestamp"] = self._localize_timestamp(event_time / 1000.0)
else:
event_data["timestamp"] = None
self.results.append(event_data)
def run(self) -> None:
"""Extract and analyze DNS events from intrusion logs."""
if not self.target_path:
self.log.error("No target path specified")
return
self.collect_txt(self.target_path)
self.parse_collected_txt("dns_event")
self.log.info("Identified %d DNS events", len(self.results))
@@ -0,0 +1,758 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional, Union
from .base import IntrusionLogsModule
# Security event tags based on Android SecurityLog API
# Reference: https://developer.android.com/reference/android/app/admin/SecurityLog
SECURITY_EVENT_TAGS = {
# ADB events (API level 24)
"adb_shell_interactive": {
"tag_id": 210001,
"name": "ADB Shell Interactive",
"description": "An ADB interactive shell was opened via 'adb shell'",
},
"adb_shell_cmd": {
"tag_id": 210002,
"name": "ADB Shell Command",
"description": "A shell command was issued over ADB via 'adb shell <command>'",
},
"adb_sync_recv_file": {
"tag_id": 210003,
"name": "ADB Sync Recv File",
"description": "A file was pulled from the device via adb daemon (adb pull)",
},
"adb_sync_send_file": {
"tag_id": 210004,
"name": "ADB Sync Send File",
"description": "A file was pushed to the device via adb daemon (adb push)",
},
# App process events (API level 24)
"app_process_start": {
"tag_id": 210005,
"name": "App Process Start",
"description": "An app process was started",
},
# Keyguard events (API level 24)
"keyguard_dismissed": {
"tag_id": 210006,
"name": "Keyguard Dismissed",
"description": "Keyguard has been dismissed",
},
"keyguard_dismiss_auth_attempt": {
"tag_id": 210007,
"name": "Keyguard Dismiss Auth Attempt",
"description": "Authentication attempt to dismiss keyguard",
},
"keyguard_secured": {
"tag_id": 210008,
"name": "Keyguard Secured",
"description": "Device has been locked",
},
# OS events (API level 28)
"os_startup": {
"tag_id": 210009,
"name": "OS Startup",
"description": "Android OS has started",
},
"os_shutdown": {
"tag_id": 210010,
"name": "OS Shutdown",
"description": "Android OS has shutdown",
},
# Logging events (API level 28)
"logging_started": {
"tag_id": 210011,
"name": "Logging Started",
"description": "Audit logging has started",
},
"logging_stopped": {
"tag_id": 210012,
"name": "Logging Stopped",
"description": "Audit logging has stopped",
},
# Media events (API level 28)
"media_mount": {
"tag_id": 210013,
"name": "Media Mount",
"description": "Removable media has been mounted",
},
"media_unmount": {
"tag_id": 210014,
"name": "Media Unmount",
"description": "Removable media was unmounted",
},
# Log buffer event (API level 28)
"log_buffer_size_critical": {
"tag_id": 210015,
"name": "Log Buffer Size Critical",
"description": "Audit log buffer has reached 90% capacity",
},
# Password policy events (API level 28)
"password_expiration_set": {
"tag_id": 210016,
"name": "Password Expiration Set",
"description": "Admin set password expiration timeout",
},
"password_complexity_set": {
"tag_id": 210017,
"name": "Password Complexity Set",
"description": "Admin set password complexity requirement",
},
"password_history_length_set": {
"tag_id": 210018,
"name": "Password History Length Set",
"description": "Admin set password history length",
},
"max_screen_lock_timeout_set": {
"tag_id": 210019,
"name": "Max Screen Lock Timeout Set",
"description": "Admin set maximum screen lock timeout",
},
"max_password_attempts_set": {
"tag_id": 210020,
"name": "Max Password Attempts Set",
"description": "Admin set maximum failed password attempts before wipe",
},
"keyguard_disabled_features_set": {
"tag_id": 210021,
"name": "Keyguard Disabled Features Set",
"description": "Admin set disabled keyguard features",
},
# Remote lock event (API level 28)
"remote_lock": {
"tag_id": 210022,
"name": "Remote Lock",
"description": "Admin remotely locked the device or profile",
},
# Wipe failure event (API level 28)
"wipe_failure": {
"tag_id": 210023,
"name": "Wipe Failure",
"description": "Failed to wipe device or user data",
},
# Cryptographic key events (API level 28)
"key_generated": {
"tag_id": 210024,
"name": "Key Generated",
"description": "Cryptographic key was generated",
},
"key_import": {
"tag_id": 210025,
"name": "Key Import",
"description": "Cryptographic key was imported",
},
"key_destruction": {
"tag_id": 210026,
"name": "Key Destruction",
"description": "Cryptographic key was destroyed",
},
# User restriction events (API level 28)
"user_restriction_added": {
"tag_id": 210027,
"name": "User Restriction Added",
"description": "Admin added a user restriction",
},
"user_restriction_removed": {
"tag_id": 210028,
"name": "User Restriction Removed",
"description": "Admin removed a user restriction",
},
# Certificate events (API level 28)
"cert_authority_installed": {
"tag_id": 210029,
"name": "Certificate Authority Installed",
"description": "Root certificate installed to trusted storage",
},
"cert_authority_removed": {
"tag_id": 210030,
"name": "Certificate Authority Removed",
"description": "Root certificate removed from trusted storage",
},
"crypto_self_test_completed": {
"tag_id": 210031,
"name": "Crypto Self Test Completed",
"description": "Cryptographic functionality self test completed",
},
"key_integrity_violation": {
"tag_id": 210032,
"name": "Key Integrity Violation",
"description": "Key integrity violation detected",
},
"cert_validation_failure": {
"tag_id": 210033,
"name": "Certificate Validation Failure",
"description": "X.509v3 certificate validation failed",
},
# Camera policy event (API level 30)
"camera_policy_set": {
"tag_id": 210034,
"name": "Camera Policy Set",
"description": "Admin set policy to disable camera",
},
# Password complexity events (API level 31/33)
"password_complexity_required": {
"tag_id": 210035,
"name": "Password Complexity Required",
"description": "Admin set password complexity requirement using predefined levels",
},
"password_changed": {
"tag_id": 210036,
"name": "Password Changed",
"description": "User changed their lockscreen password",
},
# WiFi events (API level 33)
"wifi_connection": {
"tag_id": 210037,
"name": "WiFi Connection",
"description": "Device attempted to connect to a managed WiFi network",
},
"wifi_disconnection": {
"tag_id": 210038,
"name": "WiFi Disconnection",
"description": "Device disconnected from a managed WiFi network",
},
# Bluetooth events (API level 33)
"bluetooth_connection": {
"tag_id": 210039,
"name": "Bluetooth Connection",
"description": "Device attempted to connect to a Bluetooth device",
},
"bluetooth_disconnection": {
"tag_id": 210040,
"name": "Bluetooth Disconnection",
"description": "Device disconnected from a Bluetooth device",
},
# Package events (API level 34)
"package_installed": {
"tag_id": 210041,
"name": "Package Installed",
"description": "Application package was installed",
},
"package_updated": {
"tag_id": 210042,
"name": "Package Updated",
"description": "Application package was updated",
},
"package_uninstalled": {
"tag_id": 210043,
"name": "Package Uninstalled",
"description": "Application package was uninstalled",
},
# Backup service event (API level 35)
"backup_service_toggled": {
"tag_id": 210044,
"name": "Backup Service Toggled",
"description": "Admin enabled or disabled backup service",
},
# NFC events (API level 36)
"nfc_enabled": {
"tag_id": 210045,
"name": "NFC Enabled",
"description": "NFC service is enabled",
},
"nfc_disabled": {
"tag_id": 210046,
"name": "NFC Disabled",
"description": "NFC service is disabled",
},
}
SECURITY_EVENT_METADATA_KEYS = {
"event_time",
"event_type",
"timestamp",
}
class SecurityEvent(IntrusionLogsModule):
"""This module analyzes security events from intrusion logs."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
self.event_type_counts: dict[str, int] = {}
def _get_event_tag(self, event_data: dict) -> Optional[str]:
"""Return the security-event tag key, including tags unknown to MVT."""
for key in event_data:
if key not in SECURITY_EVENT_METADATA_KEYS:
return key
return None
def check_indicators(self) -> None:
"""Check security events against indicators of compromise."""
if not self.indicators:
return
for result in self.results:
# Check app process start events for suspicious package names
if "app_process_start" in result:
process_info = result["app_process_start"]
process_name = process_info.get("process", "")
if process_name:
# Check the full process name
ioc = self.indicators.check_app_id(process_name)
if ioc:
self.alertstore.critical(
ioc.message,
result.get("timestamp") or "",
result,
matched_indicator=ioc.ioc,
)
# Also check process components after the first colon
# Example: "com.google.android.webview:sandboxed_process0:org.chromium.content.app.SandboxedProcessService0:0"
# We want to check "sandboxed_process0" and subsequent components
if ":" in process_name:
components = process_name.split(":")
for component in components[
1:
]: # Skip the first component (main package name)
if component:
ioc = self.indicators.check_app_id(component)
if ioc:
self.alertstore.critical(
ioc.message,
result.get("timestamp") or "",
result,
matched_indicator=ioc.ioc,
)
break
# Check package operations for suspicious packages
for pkg_event in [
"package_installed",
"package_updated",
"package_uninstalled",
]:
if pkg_event in result:
pkg_info = result[pkg_event]
pkg_name = pkg_info.get("package_name", "")
if pkg_name:
ioc = self.indicators.check_app_id(pkg_name)
if ioc:
self.alertstore.critical(
ioc.message,
result.get("timestamp") or "",
result,
matched_indicator=ioc.ioc,
)
# Check ADB shell commands for suspicious patterns
if "adb_shell_cmd" in result:
cmd_info = result["adb_shell_cmd"]
command = cmd_info.get("command", "")
if command:
# Check if command contains any suspicious app IDs
ioc = self.indicators.check_app_id(command)
if ioc:
self.alertstore.critical(
ioc.message,
result.get("timestamp") or "",
result,
matched_indicator=ioc.ioc,
)
# Check ADB file sync events for suspicious paths
for adb_event in ["adb_sync_recv_file", "adb_sync_send_file"]:
if adb_event in result:
file_info = result[adb_event]
file_path = file_info.get("path", "")
if file_path:
ioc = self.indicators.check_file_path(file_path)
if ioc:
self.alertstore.critical(
ioc.message,
result.get("timestamp") or "",
result,
matched_indicator=ioc.ioc,
)
# Flag failed cryptographic operations as potentially suspicious
if "key_generated" in result:
if not result["key_generated"].get("success", True):
self.log.warning(
"Failed key generation detected for key_id: %s",
result["key_generated"].get("key_id", "unknown"),
)
# Flag certificate validation failures
if "cert_validation_failure" in result:
self.log.warning(
"Certificate validation failure detected: %s",
result.get("cert_validation_failure"),
)
# Flag key integrity violations
if "key_integrity_violation" in result:
self.alertstore.medium(
f"Key integrity violation detected: {result.get('key_integrity_violation')}",
result.get("timestamp") or "",
result,
)
# Flag certificate authority installations (potential MITM)
if "cert_authority_installed" in result:
cert_info = result["cert_authority_installed"]
self.log.warning(
"Certificate authority installed: %s (success: %s)",
cert_info.get("subject", "unknown"),
cert_info.get("success", "unknown"),
)
# Flag wipe failures
if "wipe_failure" in result:
self.alertstore.medium(
"Device wipe failure detected",
result.get("timestamp") or "",
result,
)
# Flag crypto self test failures
if "crypto_self_test_completed" in result:
test_result = result["crypto_self_test_completed"]
if isinstance(test_result, dict):
success = test_result.get("success", True)
else:
success = test_result == 1
if not success:
self.alertstore.medium(
"Cryptographic self test failed",
result.get("timestamp") or "",
result,
)
def serialize(self, record: dict) -> Union[dict, list]:
"""Serialize a security event record for timeline output."""
# Determine the event sub-type
event_subtype = None
event_data_str = ""
event_subtype = self._get_event_tag(record)
if event_subtype:
event_info = record[event_subtype]
if event_subtype in SECURITY_EVENT_TAGS:
# ADB events
if event_subtype == "adb_shell_interactive":
event_data_str = "ADB interactive shell opened"
elif event_subtype == "adb_shell_cmd":
command = event_info.get("command", "")
event_data_str = f"ADB shell command: {command}"
elif event_subtype == "adb_sync_recv_file":
path = event_info.get("path", "")
event_data_str = f"File pulled via ADB: {path}"
elif event_subtype == "adb_sync_send_file":
path = event_info.get("path", "")
event_data_str = f"File pushed via ADB: {path}"
# App process events
elif event_subtype == "app_process_start":
process_name = event_info.get("process", "")
uid = event_info.get("uid", "")
pid = event_info.get("pid", "")
event_data_str = (
f"Process started: {process_name} (UID: {uid}, PID: {pid})"
)
# Keyguard events
elif event_subtype == "keyguard_dismiss_auth_attempt":
success = event_info.get("success", False)
method = event_info.get("method_strength", 0)
event_data_str = f"Auth attempt: {'Success' if success else 'Failed'} (method strength: {method})"
elif event_subtype == "keyguard_dismissed":
event_data_str = "Keyguard dismissed"
elif event_subtype == "keyguard_secured":
event_data_str = "Device locked"
elif event_subtype == "keyguard_disabled_features_set":
admin = event_info.get("admin_package", "")
features = event_info.get("disabled_features", "")
event_data_str = (
f"Keyguard features disabled by {admin}: {features}"
)
# Key events
elif event_subtype == "key_generated":
success = event_info.get("success", False)
key_id = event_info.get("key_id", "unknown")
uid = event_info.get("uid", "")
event_data_str = f"Key {'generated' if success else 'generation failed'}: {key_id} (UID: {uid})"
elif event_subtype == "key_destruction":
success = event_info.get("success", False)
key_id = event_info.get("key_id", "unknown")
uid = event_info.get("uid", "")
event_data_str = f"Key {'destroyed' if success else 'destruction failed'}: {key_id} (UID: {uid})"
elif event_subtype == "key_import":
success = event_info.get("success", False)
key_id = event_info.get("key_id", "unknown")
event_data_str = (
f"Key {'imported' if success else 'import failed'}: {key_id}"
)
elif event_subtype == "key_integrity_violation":
key_id = event_info.get("key_id", "unknown")
event_data_str = f"Key integrity violation: {key_id}"
# Certificate events
elif event_subtype == "cert_authority_installed":
success = event_info.get("success", False)
subject = event_info.get("subject", "unknown")
event_data_str = f"Cert {'installed' if success else 'install failed'}: {subject}"
elif event_subtype == "cert_authority_removed":
success = event_info.get("success", False)
subject = event_info.get("subject", "unknown")
event_data_str = (
f"Cert {'removed' if success else 'removal failed'}: {subject}"
)
elif event_subtype == "cert_validation_failure":
reason = (
event_info if isinstance(event_info, str) else str(event_info)
)
event_data_str = f"Certificate validation failure: {reason}"
elif event_subtype == "crypto_self_test_completed":
if isinstance(event_info, dict):
success = event_info.get("success", False)
else:
success = event_info == 1
event_data_str = (
f"Crypto self test: {'passed' if success else 'FAILED'}"
)
# Package events
elif event_subtype in [
"package_installed",
"package_updated",
"package_uninstalled",
]:
pkg_name = event_info.get("package_name", "")
version = event_info.get("version_code", "")
user_id = event_info.get("user_id", "")
action = event_subtype.replace("package_", "").title()
event_data_str = (
f"Package {action}: {pkg_name} (v{version}, user: {user_id})"
)
# OS events
elif event_subtype == "os_startup":
verified_boot = event_info.get("verified_boot_state", "")
dm_verity = event_info.get("dm_verity_mode", "")
event_data_str = f"OS startup (verified boot: {verified_boot}, dm-verity: {dm_verity})"
elif event_subtype == "os_shutdown":
event_data_str = "OS shutdown"
# Logging events
elif event_subtype == "logging_started":
event_data_str = "Audit logging started"
elif event_subtype == "logging_stopped":
event_data_str = "Audit logging stopped"
elif event_subtype == "log_buffer_size_critical":
event_data_str = "Log buffer at 90% capacity"
# Media events
elif event_subtype == "media_mount":
mount_point = event_info.get("mount_point", "")
label = event_info.get("volume_label", "")
event_data_str = f"Media mounted: {mount_point} ({label})"
elif event_subtype == "media_unmount":
mount_point = event_info.get("mount_point", "")
label = event_info.get("volume_label", "")
event_data_str = f"Media unmounted: {mount_point} ({label})"
# Password policy events
elif event_subtype == "password_expiration_set":
admin = event_info.get("admin_package", "")
timeout = event_info.get("timeout_ms", "")
event_data_str = f"Password expiration set by {admin}: {timeout}ms"
elif event_subtype == "password_complexity_set":
admin = event_info.get("admin_package", "")
event_data_str = f"Password complexity set by {admin}"
elif event_subtype == "password_complexity_required":
admin = event_info.get("admin_package", "")
complexity = event_info.get("complexity", "")
event_data_str = (
f"Password complexity required by {admin}: {complexity}"
)
elif event_subtype == "password_history_length_set":
admin = event_info.get("admin_package", "")
length = event_info.get("length", "")
event_data_str = f"Password history length set by {admin}: {length}"
elif event_subtype == "password_changed":
complexity = event_info.get("complexity", "")
user_id = event_info.get("user_id", "")
event_data_str = (
f"Password changed (complexity: {complexity}, user: {user_id})"
)
elif event_subtype == "max_screen_lock_timeout_set":
admin = event_info.get("admin_package", "")
timeout = event_info.get("timeout_ms", "")
event_data_str = (
f"Max screen lock timeout set by {admin}: {timeout}ms"
)
elif event_subtype == "max_password_attempts_set":
admin = event_info.get("admin_package", "")
attempts = event_info.get("max_attempts", "")
event_data_str = f"Max password attempts set by {admin}: {attempts}"
# Remote lock and wipe events
elif event_subtype == "remote_lock":
admin = event_info.get("admin_package", "")
event_data_str = f"Device remotely locked by {admin}"
elif event_subtype == "wipe_failure":
event_data_str = "Device wipe failed"
# User restriction events
elif event_subtype == "user_restriction_added":
admin = event_info.get("admin_package", "")
restriction = event_info.get("restriction", "")
event_data_str = f"User restriction added by {admin}: {restriction}"
elif event_subtype == "user_restriction_removed":
admin = event_info.get("admin_package", "")
restriction = event_info.get("restriction", "")
event_data_str = (
f"User restriction removed by {admin}: {restriction}"
)
# WiFi events
elif event_subtype == "wifi_connection":
bssid = event_info.get("bssid", "")
event_type = event_info.get("event_type", "")
reason = event_info.get("reason", "")
event_data_str = f"WiFi connection: {event_type} (BSSID: {bssid})"
if reason:
event_data_str += f" - {reason}"
elif event_subtype == "wifi_disconnection":
bssid = event_info.get("bssid", "")
reason = event_info.get("reason", "")
event_data_str = f"WiFi disconnection (BSSID: {bssid})"
if reason:
event_data_str += f" - {reason}"
# Bluetooth events
elif event_subtype == "bluetooth_connection":
mac = event_info.get("mac_address", "")
success = event_info.get("success", False)
reason = event_info.get("reason", "")
event_data_str = f"Bluetooth {'connected' if success else 'connection failed'}: {mac}"
if reason:
event_data_str += f" - {reason}"
elif event_subtype == "bluetooth_disconnection":
mac = event_info.get("mac_address", "")
reason = event_info.get("reason", "")
event_data_str = f"Bluetooth disconnected: {mac}"
if reason:
event_data_str += f" - {reason}"
# Camera policy event
elif event_subtype == "camera_policy_set":
admin = event_info.get("admin_package", "")
disabled = event_info.get("disabled", False)
event_data_str = (
f"Camera {'disabled' if disabled else 'enabled'} by {admin}"
)
# Backup service event
elif event_subtype == "backup_service_toggled":
admin = event_info.get("admin_package", "")
enabled = event_info.get("enabled", False)
event_data_str = f"Backup service {'enabled' if enabled else 'disabled'} by {admin}"
# NFC events
elif event_subtype == "nfc_enabled":
event_data_str = "NFC enabled"
elif event_subtype == "nfc_disabled":
event_data_str = "NFC disabled"
else:
event_data_str = (
f"{SECURITY_EVENT_TAGS.get(event_subtype, {}).get('name', event_subtype)}: "
f"{event_info}"
)
else:
event_data_str = f"{event_subtype}: {event_info}"
if not event_subtype:
event_subtype = "unknown"
event_data_str = str(record)
return {
"timestamp": record.get("timestamp"),
"module": self.__class__.__name__,
"event": event_subtype,
"data": event_data_str,
}
def process_event(self, event_data: dict) -> None:
"""Process a security event and add it to results."""
# Convert event_time to ISO format
# Security events use nanoseconds since epoch
event_time = event_data.get("event_time")
if event_time:
# Convert nanoseconds to seconds
event_data["timestamp"] = self._localize_timestamp(
event_time / 1_000_000_000.0
)
else:
event_data["timestamp"] = None
# Track event type statistics, including future tags unknown to MVT.
event_tag = self._get_event_tag(event_data)
if event_tag:
self.event_type_counts[event_tag] = (
self.event_type_counts.get(event_tag, 0) + 1
)
self.results.append(event_data)
def run(self) -> None:
"""Extract and analyze security events from intrusion logs."""
if not self.target_path:
self.log.error("No target path specified")
return
self.collect_txt(self.target_path)
self.parse_collected_txt("security_event")
self.log.info("Identified %d security events", len(self.results))
# Log event type breakdown
if self.event_type_counts:
self.log.info("Security event breakdown:")
for event_type, count in sorted(
self.event_type_counts.items(), key=lambda x: x[1], reverse=True
):
event_name = SECURITY_EVENT_TAGS.get(event_type, {}).get(
"name", event_type
)
self.log.info(" - %s: %d", event_name, count)
unknown_event_types = sorted(
event_type
for event_type in self.event_type_counts
if event_type not in SECURITY_EVENT_TAGS
)
if unknown_event_types:
self.log.warning(
"Found unknown intrusion logging security event type(s): %s. "
"Please open an issue on GitHub so MVT can add support for them.",
", ".join(unknown_event_types),
)
+1
View File
@@ -231,6 +231,7 @@ def parse_sms_file(data):
entry.pop("mms_body")
body = entry.get("body", None)
message_links = None
if body:
message_links = check_for_links(entry["body"])
+138 -139
View File
@@ -1,13 +1,12 @@
# Generated by the protocol buffer compiler. DO NOT EDIT!
# sources: tombstone.proto
# plugin: python-betterproto
# plugin: python-betterproto2
from dataclasses import dataclass
from typing import Dict, List
import betterproto
import betterproto2
class Architecture(betterproto.Enum):
class Architecture(betterproto2.Enum):
ARM32 = 0
ARM64 = 1
X86 = 2
@@ -16,12 +15,12 @@ class Architecture(betterproto.Enum):
NONE = 5
class MemoryErrorTool(betterproto.Enum):
class MemoryErrorTool(betterproto2.Enum):
GWP_ASAN = 0
SCUDO = 1
class MemoryErrorType(betterproto.Enum):
class MemoryErrorType(betterproto2.Enum):
UNKNOWN = 0
USE_AFTER_FREE = 1
DOUBLE_FREE = 2
@@ -30,179 +29,179 @@ class MemoryErrorType(betterproto.Enum):
BUFFER_UNDERFLOW = 5
@dataclass
class CrashDetail(betterproto.Message):
@dataclass(eq=False, repr=False)
class CrashDetail(betterproto2.Message):
"""
NOTE TO OEMS: If you add custom fields to this proto, do not use numbers in
the reserved range.
"""
name: bytes = betterproto.bytes_field(1)
data: bytes = betterproto.bytes_field(2)
name: "bytes" = betterproto2.field(1, betterproto2.TYPE_BYTES)
data: "bytes" = betterproto2.field(2, betterproto2.TYPE_BYTES)
@dataclass
class StackHistoryBufferEntry(betterproto.Message):
addr: "BacktraceFrame" = betterproto.message_field(1)
fp: int = betterproto.uint64_field(2)
tag: int = betterproto.uint64_field(3)
@dataclass(eq=False, repr=False)
class StackHistoryBufferEntry(betterproto2.Message):
addr: "BacktraceFrame | None" = betterproto2.field(1, betterproto2.TYPE_MESSAGE, optional=True)
fp: "int" = betterproto2.field(2, betterproto2.TYPE_UINT64)
tag: "int" = betterproto2.field(3, betterproto2.TYPE_UINT64)
@dataclass
class StackHistoryBuffer(betterproto.Message):
tid: int = betterproto.uint64_field(1)
entries: List["StackHistoryBufferEntry"] = betterproto.message_field(2)
@dataclass(eq=False, repr=False)
class StackHistoryBuffer(betterproto2.Message):
tid: "int" = betterproto2.field(1, betterproto2.TYPE_UINT64)
entries: "list[StackHistoryBufferEntry]" = betterproto2.field(2, betterproto2.TYPE_MESSAGE, repeated=True)
@dataclass
class Tombstone(betterproto.Message):
arch: "Architecture" = betterproto.enum_field(1)
guest_arch: "Architecture" = betterproto.enum_field(24)
build_fingerprint: str = betterproto.string_field(2)
revision: str = betterproto.string_field(3)
timestamp: str = betterproto.string_field(4)
pid: int = betterproto.uint32_field(5)
tid: int = betterproto.uint32_field(6)
uid: int = betterproto.uint32_field(7)
selinux_label: str = betterproto.string_field(8)
command_line: List[str] = betterproto.string_field(9)
@dataclass(eq=False, repr=False)
class Tombstone(betterproto2.Message):
arch: "Architecture" = betterproto2.field(1, betterproto2.TYPE_ENUM, default_factory=lambda: Architecture(0))
guest_arch: "Architecture" = betterproto2.field(24, betterproto2.TYPE_ENUM, default_factory=lambda: Architecture(0))
build_fingerprint: "str" = betterproto2.field(2, betterproto2.TYPE_STRING)
revision: "str" = betterproto2.field(3, betterproto2.TYPE_STRING)
timestamp: "str" = betterproto2.field(4, betterproto2.TYPE_STRING)
pid: "int" = betterproto2.field(5, betterproto2.TYPE_UINT32)
tid: "int" = betterproto2.field(6, betterproto2.TYPE_UINT32)
uid: "int" = betterproto2.field(7, betterproto2.TYPE_UINT32)
selinux_label: "str" = betterproto2.field(8, betterproto2.TYPE_STRING)
command_line: "list[str]" = betterproto2.field(9, betterproto2.TYPE_STRING, repeated=True)
# Process uptime in seconds.
process_uptime: int = betterproto.uint32_field(20)
signal_info: "Signal" = betterproto.message_field(10)
abort_message: str = betterproto.string_field(14)
crash_details: List["CrashDetail"] = betterproto.message_field(21)
causes: List["Cause"] = betterproto.message_field(15)
threads: Dict[int, "Thread"] = betterproto.map_field(
16, betterproto.TYPE_UINT32, betterproto.TYPE_MESSAGE
process_uptime: "int" = betterproto2.field(20, betterproto2.TYPE_UINT32)
signal_info: "Signal | None" = betterproto2.field(10, betterproto2.TYPE_MESSAGE, optional=True)
abort_message: "str" = betterproto2.field(14, betterproto2.TYPE_STRING)
crash_details: "list[CrashDetail]" = betterproto2.field(21, betterproto2.TYPE_MESSAGE, repeated=True)
causes: "list[Cause]" = betterproto2.field(15, betterproto2.TYPE_MESSAGE, repeated=True)
threads: "dict[int, Thread]" = betterproto2.field(
16, betterproto2.TYPE_MAP, map_meta=betterproto2.map_meta(betterproto2.TYPE_UINT32, betterproto2.TYPE_MESSAGE)
)
guest_threads: Dict[int, "Thread"] = betterproto.map_field(
25, betterproto.TYPE_UINT32, betterproto.TYPE_MESSAGE
guest_threads: "dict[int, Thread]" = betterproto2.field(
25, betterproto2.TYPE_MAP, map_meta=betterproto2.map_meta(betterproto2.TYPE_UINT32, betterproto2.TYPE_MESSAGE)
)
memory_mappings: List["MemoryMapping"] = betterproto.message_field(17)
log_buffers: List["LogBuffer"] = betterproto.message_field(18)
open_fds: List["FD"] = betterproto.message_field(19)
page_size: int = betterproto.uint32_field(22)
has_been_16kb_mode: bool = betterproto.bool_field(23)
stack_history_buffer: "StackHistoryBuffer" = betterproto.message_field(26)
memory_mappings: "list[MemoryMapping]" = betterproto2.field(17, betterproto2.TYPE_MESSAGE, repeated=True)
log_buffers: "list[LogBuffer]" = betterproto2.field(18, betterproto2.TYPE_MESSAGE, repeated=True)
open_fds: "list[FD]" = betterproto2.field(19, betterproto2.TYPE_MESSAGE, repeated=True)
page_size: "int" = betterproto2.field(22, betterproto2.TYPE_UINT32)
has_been_16kb_mode: "bool" = betterproto2.field(23, betterproto2.TYPE_BOOL)
stack_history_buffer: "StackHistoryBuffer | None" = betterproto2.field(26, betterproto2.TYPE_MESSAGE, optional=True)
@dataclass
class Signal(betterproto.Message):
number: int = betterproto.int32_field(1)
name: str = betterproto.string_field(2)
code: int = betterproto.int32_field(3)
code_name: str = betterproto.string_field(4)
has_sender: bool = betterproto.bool_field(5)
sender_uid: int = betterproto.int32_field(6)
sender_pid: int = betterproto.int32_field(7)
has_fault_address: bool = betterproto.bool_field(8)
fault_address: int = betterproto.uint64_field(9)
@dataclass(eq=False, repr=False)
class Signal(betterproto2.Message):
number: "int" = betterproto2.field(1, betterproto2.TYPE_INT32)
name: "str" = betterproto2.field(2, betterproto2.TYPE_STRING)
code: "int" = betterproto2.field(3, betterproto2.TYPE_INT32)
code_name: "str" = betterproto2.field(4, betterproto2.TYPE_STRING)
has_sender: "bool" = betterproto2.field(5, betterproto2.TYPE_BOOL)
sender_uid: "int" = betterproto2.field(6, betterproto2.TYPE_INT32)
sender_pid: "int" = betterproto2.field(7, betterproto2.TYPE_INT32)
has_fault_address: "bool" = betterproto2.field(8, betterproto2.TYPE_BOOL)
fault_address: "int" = betterproto2.field(9, betterproto2.TYPE_UINT64)
# Note, may or may not contain the dump of the actual memory contents.
# Currently, on arm64, we only include metadata, and not the contents.
fault_adjacent_metadata: "MemoryDump" = betterproto.message_field(10)
fault_adjacent_metadata: "MemoryDump | None" = betterproto2.field(10, betterproto2.TYPE_MESSAGE, optional=True)
@dataclass
class HeapObject(betterproto.Message):
address: int = betterproto.uint64_field(1)
size: int = betterproto.uint64_field(2)
allocation_tid: int = betterproto.uint64_field(3)
allocation_backtrace: List["BacktraceFrame"] = betterproto.message_field(4)
deallocation_tid: int = betterproto.uint64_field(5)
deallocation_backtrace: List["BacktraceFrame"] = betterproto.message_field(6)
@dataclass(eq=False, repr=False)
class HeapObject(betterproto2.Message):
address: "int" = betterproto2.field(1, betterproto2.TYPE_UINT64)
size: "int" = betterproto2.field(2, betterproto2.TYPE_UINT64)
allocation_tid: "int" = betterproto2.field(3, betterproto2.TYPE_UINT64)
allocation_backtrace: "list[BacktraceFrame]" = betterproto2.field(4, betterproto2.TYPE_MESSAGE, repeated=True)
deallocation_tid: "int" = betterproto2.field(5, betterproto2.TYPE_UINT64)
deallocation_backtrace: "list[BacktraceFrame]" = betterproto2.field(6, betterproto2.TYPE_MESSAGE, repeated=True)
@dataclass
class MemoryError(betterproto.Message):
tool: "MemoryErrorTool" = betterproto.enum_field(1)
type: "MemoryErrorType" = betterproto.enum_field(2)
heap: "HeapObject" = betterproto.message_field(3, group="location")
@dataclass(eq=False, repr=False)
class MemoryError(betterproto2.Message):
tool: "MemoryErrorTool" = betterproto2.field(1, betterproto2.TYPE_ENUM, default_factory=lambda: MemoryErrorTool(0))
type: "MemoryErrorType" = betterproto2.field(2, betterproto2.TYPE_ENUM, default_factory=lambda: MemoryErrorType(0))
heap: "HeapObject | None" = betterproto2.field(3, betterproto2.TYPE_MESSAGE, optional=True, group="location")
@dataclass
class Cause(betterproto.Message):
human_readable: str = betterproto.string_field(1)
memory_error: "MemoryError" = betterproto.message_field(2, group="details")
@dataclass(eq=False, repr=False)
class Cause(betterproto2.Message):
human_readable: "str" = betterproto2.field(1, betterproto2.TYPE_STRING)
memory_error: "MemoryError | None" = betterproto2.field(2, betterproto2.TYPE_MESSAGE, optional=True, group="details")
@dataclass
class Register(betterproto.Message):
name: str = betterproto.string_field(1)
u64: int = betterproto.uint64_field(2)
@dataclass(eq=False, repr=False)
class Register(betterproto2.Message):
name: "str" = betterproto2.field(1, betterproto2.TYPE_STRING)
u64: "int" = betterproto2.field(2, betterproto2.TYPE_UINT64)
@dataclass
class Thread(betterproto.Message):
id: int = betterproto.int32_field(1)
name: str = betterproto.string_field(2)
registers: List["Register"] = betterproto.message_field(3)
backtrace_note: List[str] = betterproto.string_field(7)
unreadable_elf_files: List[str] = betterproto.string_field(9)
current_backtrace: List["BacktraceFrame"] = betterproto.message_field(4)
memory_dump: List["MemoryDump"] = betterproto.message_field(5)
tagged_addr_ctrl: int = betterproto.int64_field(6)
pac_enabled_keys: int = betterproto.int64_field(8)
@dataclass(eq=False, repr=False)
class Thread(betterproto2.Message):
id: "int" = betterproto2.field(1, betterproto2.TYPE_INT32)
name: "str" = betterproto2.field(2, betterproto2.TYPE_STRING)
registers: "list[Register]" = betterproto2.field(3, betterproto2.TYPE_MESSAGE, repeated=True)
backtrace_note: "list[str]" = betterproto2.field(7, betterproto2.TYPE_STRING, repeated=True)
unreadable_elf_files: "list[str]" = betterproto2.field(9, betterproto2.TYPE_STRING, repeated=True)
current_backtrace: "list[BacktraceFrame]" = betterproto2.field(4, betterproto2.TYPE_MESSAGE, repeated=True)
memory_dump: "list[MemoryDump]" = betterproto2.field(5, betterproto2.TYPE_MESSAGE, repeated=True)
tagged_addr_ctrl: "int" = betterproto2.field(6, betterproto2.TYPE_INT64)
pac_enabled_keys: "int" = betterproto2.field(8, betterproto2.TYPE_INT64)
@dataclass
class BacktraceFrame(betterproto.Message):
rel_pc: int = betterproto.uint64_field(1)
pc: int = betterproto.uint64_field(2)
sp: int = betterproto.uint64_field(3)
function_name: str = betterproto.string_field(4)
function_offset: int = betterproto.uint64_field(5)
file_name: str = betterproto.string_field(6)
file_map_offset: int = betterproto.uint64_field(7)
build_id: str = betterproto.string_field(8)
@dataclass(eq=False, repr=False)
class BacktraceFrame(betterproto2.Message):
rel_pc: "int" = betterproto2.field(1, betterproto2.TYPE_UINT64)
pc: "int" = betterproto2.field(2, betterproto2.TYPE_UINT64)
sp: "int" = betterproto2.field(3, betterproto2.TYPE_UINT64)
function_name: "str" = betterproto2.field(4, betterproto2.TYPE_STRING)
function_offset: "int" = betterproto2.field(5, betterproto2.TYPE_UINT64)
file_name: "str" = betterproto2.field(6, betterproto2.TYPE_STRING)
file_map_offset: "int" = betterproto2.field(7, betterproto2.TYPE_UINT64)
build_id: "str" = betterproto2.field(8, betterproto2.TYPE_STRING)
@dataclass
class ArmMTEMetadata(betterproto.Message):
@dataclass(eq=False, repr=False)
class ArmMTEMetadata(betterproto2.Message):
# One memory tag per granule (e.g. every 16 bytes) of regular memory.
memory_tags: bytes = betterproto.bytes_field(1)
memory_tags: "bytes" = betterproto2.field(1, betterproto2.TYPE_BYTES)
@dataclass
class MemoryDump(betterproto.Message):
register_name: str = betterproto.string_field(1)
mapping_name: str = betterproto.string_field(2)
begin_address: int = betterproto.uint64_field(3)
memory: bytes = betterproto.bytes_field(4)
arm_mte_metadata: "ArmMTEMetadata" = betterproto.message_field(6, group="metadata")
@dataclass(eq=False, repr=False)
class MemoryDump(betterproto2.Message):
register_name: "str" = betterproto2.field(1, betterproto2.TYPE_STRING)
mapping_name: "str" = betterproto2.field(2, betterproto2.TYPE_STRING)
begin_address: "int" = betterproto2.field(3, betterproto2.TYPE_UINT64)
memory: "bytes" = betterproto2.field(4, betterproto2.TYPE_BYTES)
arm_mte_metadata: "ArmMTEMetadata | None" = betterproto2.field(6, betterproto2.TYPE_MESSAGE, optional=True, group="metadata")
@dataclass
class MemoryMapping(betterproto.Message):
begin_address: int = betterproto.uint64_field(1)
end_address: int = betterproto.uint64_field(2)
offset: int = betterproto.uint64_field(3)
read: bool = betterproto.bool_field(4)
write: bool = betterproto.bool_field(5)
execute: bool = betterproto.bool_field(6)
mapping_name: str = betterproto.string_field(7)
build_id: str = betterproto.string_field(8)
load_bias: int = betterproto.uint64_field(9)
@dataclass(eq=False, repr=False)
class MemoryMapping(betterproto2.Message):
begin_address: "int" = betterproto2.field(1, betterproto2.TYPE_UINT64)
end_address: "int" = betterproto2.field(2, betterproto2.TYPE_UINT64)
offset: "int" = betterproto2.field(3, betterproto2.TYPE_UINT64)
read: "bool" = betterproto2.field(4, betterproto2.TYPE_BOOL)
write: "bool" = betterproto2.field(5, betterproto2.TYPE_BOOL)
execute: "bool" = betterproto2.field(6, betterproto2.TYPE_BOOL)
mapping_name: "str" = betterproto2.field(7, betterproto2.TYPE_STRING)
build_id: "str" = betterproto2.field(8, betterproto2.TYPE_STRING)
load_bias: "int" = betterproto2.field(9, betterproto2.TYPE_UINT64)
@dataclass
class FD(betterproto.Message):
fd: int = betterproto.int32_field(1)
path: str = betterproto.string_field(2)
owner: str = betterproto.string_field(3)
tag: int = betterproto.uint64_field(4)
@dataclass(eq=False, repr=False)
class FD(betterproto2.Message):
fd: "int" = betterproto2.field(1, betterproto2.TYPE_INT32)
path: "str" = betterproto2.field(2, betterproto2.TYPE_STRING)
owner: "str" = betterproto2.field(3, betterproto2.TYPE_STRING)
tag: "int" = betterproto2.field(4, betterproto2.TYPE_UINT64)
@dataclass
class LogBuffer(betterproto.Message):
name: str = betterproto.string_field(1)
logs: List["LogMessage"] = betterproto.message_field(2)
@dataclass(eq=False, repr=False)
class LogBuffer(betterproto2.Message):
name: "str" = betterproto2.field(1, betterproto2.TYPE_STRING)
logs: "list[LogMessage]" = betterproto2.field(2, betterproto2.TYPE_MESSAGE, repeated=True)
@dataclass
class LogMessage(betterproto.Message):
timestamp: str = betterproto.string_field(1)
pid: int = betterproto.uint32_field(2)
tid: int = betterproto.uint32_field(3)
priority: int = betterproto.uint32_field(4)
tag: str = betterproto.string_field(5)
message: str = betterproto.string_field(6)
@dataclass(eq=False, repr=False)
class LogMessage(betterproto2.Message):
timestamp: "str" = betterproto2.field(1, betterproto2.TYPE_STRING)
pid: "int" = betterproto2.field(2, betterproto2.TYPE_UINT32)
tid: "int" = betterproto2.field(3, betterproto2.TYPE_UINT32)
priority: "int" = betterproto2.field(4, betterproto2.TYPE_UINT32)
tag: "str" = betterproto2.field(5, betterproto2.TYPE_STRING)
message: "str" = betterproto2.field(6, betterproto2.TYPE_STRING)
+2 -3
View File
@@ -6,14 +6,13 @@ from datetime import datetime, timedelta
from typing import List
def warn_android_patch_level(patch_level: str, log) -> str:
def warn_android_patch_level(patch_level: str, log) -> str | bool:
"""Alert if Android patch level out-of-date"""
patch_date = datetime.strptime(patch_level, "%Y-%m-%d")
if (datetime.now() - patch_date) > timedelta(days=6 * 31):
warning_message = (
f"This phone has not received security updates "
f"for more than six months (last update: {patch_level}).",
patch_level,
f"for more than six months (last update: {patch_level})."
)
return warning_message
+83 -14
View File
@@ -4,12 +4,13 @@
# https://license.mvt.re/1.1/
import csv
import inspect
import logging
from dataclasses import asdict, dataclass
from enum import Enum
from dataclasses import dataclass, asdict
from typing import List, Dict, Any, Optional
from typing import Any, Dict, List, Optional
from .log import INFO_ALERT, LOW_ALERT, HIGH_ALERT, CRITICAL_ALERT, MEDIUM_ALERT
from .log import CRITICAL_ALERT, HIGH_ALERT, INFO_ALERT, LOW_ALERT, MEDIUM_ALERT
from .module_types import ModuleAtomicResult
@@ -28,6 +29,7 @@ class Alert:
message: str
event_time: str
event: ModuleAtomicResult
matched_indicator: Optional[Any] = None
class AlertStore:
@@ -35,78 +37,138 @@ class AlertStore:
self.__alerts: List[Alert] = []
self.__log = log
def _get_calling_module(self) -> str:
"""
Automatically detect the calling MVT module and return its slug.
Walks up the call stack to find the first frame that belongs to an MVT module
(artifact or extraction module) and extracts its slug.
:return: Module slug string
"""
frame = inspect.currentframe()
try:
# Walk up the call stack
while frame is not None:
frame = frame.f_back
if frame is None:
break
# Get the 'self' object from the frame's local variables
frame_locals = frame.f_locals
if "self" in frame_locals:
obj = frame_locals["self"]
# Check if it has a get_slug method (MVT modules have this)
if hasattr(obj, "get_slug") and callable(obj.get_slug):
try:
return str(obj.get_slug())
except Exception:
pass
# Fallback: return "unknown" if we can't find the module
return "unknown"
finally:
del frame
@property
def alerts(self) -> List[Alert]:
return self.__alerts
def add(self, alert: Alert) -> None:
self.__alerts.append(alert)
self.log(alert)
def extend(self, alerts: List[Alert]) -> None:
self.__alerts.extend(alerts)
for alert in alerts:
self.add(alert)
def info(
self, module: str, message: str, event_time: str, event: ModuleAtomicResult
self,
message: str,
event_time: str,
event: ModuleAtomicResult,
matched_indicator: Optional[Any] = None,
):
self.add(
Alert(
level=AlertLevel.INFORMATIONAL,
module=module,
module=self._get_calling_module(),
message=message,
event_time=event_time,
event=event,
matched_indicator=matched_indicator,
)
)
def low(
self, module: str, message: str, event_time: str, event: ModuleAtomicResult
self,
message: str,
event_time: str,
event: ModuleAtomicResult,
matched_indicator: Optional[Any] = None,
):
self.add(
Alert(
level=AlertLevel.LOW,
module=module,
module=self._get_calling_module(),
message=message,
event_time=event_time,
event=event,
matched_indicator=matched_indicator,
)
)
def medium(
self, module: str, message: str, event_time: str, event: ModuleAtomicResult
self,
message: str,
event_time: str,
event: ModuleAtomicResult,
matched_indicator: Optional[Any] = None,
):
self.add(
Alert(
level=AlertLevel.MEDIUM,
module=module,
module=self._get_calling_module(),
message=message,
event_time=event_time,
event=event,
matched_indicator=matched_indicator,
)
)
def high(
self, module: str, message: str, event_time: str, event: ModuleAtomicResult
self,
message: str,
event_time: str,
event: ModuleAtomicResult,
matched_indicator: Optional[Any] = None,
):
self.add(
Alert(
level=AlertLevel.HIGH,
module=module,
module=self._get_calling_module(),
message=message,
event_time=event_time,
event=event,
matched_indicator=matched_indicator,
)
)
def critical(
self, module: str, message: str, event_time: str, event: ModuleAtomicResult
self,
message: str,
event_time: str,
event: ModuleAtomicResult,
matched_indicator: Optional[Any] = None,
):
self.add(
Alert(
level=AlertLevel.CRITICAL,
module=module,
module=self._get_calling_module(),
message=message,
event_time=event_time,
event=event,
matched_indicator=matched_indicator,
)
)
@@ -145,6 +207,13 @@ class AlertStore:
alert_dict = asdict(alert)
# This is required because an Enum is not JSON serializable.
alert_dict["level"] = alert.level.name
if isinstance(alert_dict.get("event"), dict):
event_matched_indicator = alert_dict["event"].pop(
"matched_indicator", None
)
if alert_dict["matched_indicator"] is None:
alert_dict["matched_indicator"] = event_matched_indicator
alerts.append(alert_dict)
return alerts
+8
View File
@@ -24,6 +24,8 @@ class CmdCheckIOCS(Command):
module_options: Optional[dict] = None,
hashes: Optional[bool] = False,
sub_command: Optional[bool] = False,
disable_version_check: bool = False,
disable_indicator_check: bool = False,
) -> None:
super().__init__(
target_path=target_path,
@@ -35,6 +37,8 @@ class CmdCheckIOCS(Command):
hashes=hashes,
sub_command=sub_command,
log=log,
disable_version_check=disable_version_check,
disable_indicator_check=disable_indicator_check,
)
self.name = "check-iocs"
@@ -69,6 +73,10 @@ class CmdCheckIOCS(Command):
m = iocs_module.from_json(
file_path, log=logging.getLogger(iocs_module.__module__)
)
if not m:
log.warning("No result from this module, skipping it")
continue
if self.iocs.total_ioc_count > 0:
m.indicators = self.iocs
m.indicators.log = m.log
+25 -12
View File
@@ -8,20 +8,22 @@ import logging
import os
import sys
from datetime import datetime
from typing import Optional
from typing import Any, Optional
from rich.console import Console
from rich.panel import Panel
from rich.text import Text
from .alerts import AlertLevel, AlertStore
from .config import settings
from .indicators import Indicators
from .module import MVTModule, run_module, save_timeline
from .module import EncryptedBackupError, MVTModule, run_module, save_timeline
from .utils import (
CustomJSONEncoder,
convert_datetime_to_iso,
generate_hashes_from_path,
get_sha256_from_file_path,
)
from .config import settings
from .alerts import AlertStore, AlertLevel
from .version import MVT_VERSION
@@ -38,9 +40,11 @@ class Command:
hashes: Optional[bool] = False,
sub_command: Optional[bool] = False,
log: logging.Logger = logging.getLogger(__name__),
disable_version_check: bool = False,
disable_indicator_check: bool = False,
) -> None:
self.name = ""
self.modules = []
self.modules: list[Any] = []
self.target_path = target_path
self.results_path = results_path
@@ -49,6 +53,8 @@ class Command:
self.serial = serial
self.log = log
self.sub_command = sub_command
self.disable_version_check = disable_version_check
self.disable_indicator_check = disable_indicator_check
# This dictionary can contain options that will be passed down from
# the Command to all modules. This can for example be used to pass
@@ -57,10 +63,10 @@ class Command:
# This list will contain all executed modules.
# We can use this to reference e.g. self.executed[0].results.
self.executed = []
self.executed: list[Any] = []
self.hashes = hashes
self.hash_values = []
self.timeline = []
self.hash_values: list[dict[str, Any]] = []
self.timeline: list[dict[str, Any]] = []
# Load IOCs
self._create_storage()
@@ -136,7 +142,7 @@ class Command:
alerts_path = os.path.join(self.results_path, "alerts.json")
with open(alerts_path, "w+", encoding="utf-8") as handle:
json.dump(alerts, handle, indent=4)
json.dump(alerts, handle, indent=4, cls=CustomJSONEncoder)
def _store_alerts_timeline(self) -> None:
if not self.results_path:
@@ -149,11 +155,11 @@ class Command:
if not self.results_path:
return
target_path = None
target_path: Optional[str] = None
if self.target_path:
target_path = os.path.abspath(self.target_path)
info = {
info: dict[str, Any] = {
"target_path": target_path,
"mvt_version": MVT_VERSION,
"date": convert_datetime_to_iso(datetime.now()),
@@ -289,7 +295,14 @@ class Command:
except NotImplementedError:
pass
run_module(m)
try:
run_module(m)
except EncryptedBackupError:
self.log.critical(
"The backup appears to be encrypted. "
"Please decrypt it first using `mvt-ios decrypt-backup`."
)
return
self.executed.append(m)
self.timeline.extend(m.timeline)
+22 -23
View File
@@ -1,13 +1,12 @@
import os
import yaml
import json
import os
from typing import Optional, Tuple, Type
from typing import Tuple, Type, Optional
import yaml
from appdirs import user_config_dir
from pydantic import AnyHttpUrl, Field
from pydantic import Field
from pydantic_settings import (
BaseSettings,
InitSettingsSource,
PydanticBaseSettingsSource,
SettingsConfigDict,
YamlConfigSettingsSource,
@@ -22,51 +21,51 @@ class MVTSettings(BaseSettings):
env_prefix="MVT_",
env_nested_delimiter="_",
extra="ignore",
nested_model_default_partial_updates=True,
)
# Allow to decided if want to load environment variables
load_env: bool = Field(True, exclude=True)
# General settings
PYPI_UPDATE_URL: AnyHttpUrl = Field(
"https://pypi.org/pypi/mvt/json",
validate_default=False,
PYPI_UPDATE_URL: str = Field(
default="https://pypi.org/pypi/mvt/json",
)
NETWORK_ACCESS_ALLOWED: bool = True
NETWORK_TIMEOUT: int = 15
# Command default settings, all can be specified by MVT_ prefixed environment variables too.
IOS_BACKUP_PASSWORD: Optional[str] = Field(
None, description="Default password to use to decrypt iOS backups"
default=None, description="Default password to use to decrypt iOS backups"
)
ANDROID_BACKUP_PASSWORD: Optional[str] = Field(
None, description="Default password to use to decrypt Android backups"
default=None, description="Default password to use to decrypt Android backups"
)
STIX2: Optional[str] = Field(
None, description="List of directories where STIX2 files are stored"
default=None, description="List of directories where STIX2 files are stored"
)
VT_API_KEY: Optional[str] = Field(
None, description="API key to use for VirusTotal lookups"
default=None, description="API key to use for VirusTotal lookups"
)
PROFILE: bool = Field(False, description="Profile the execution of MVT modules")
HASH_FILES: bool = Field(False, description="Should MVT hash output files")
PROFILE: bool = Field(
default=False, description="Profile the execution of MVT modules"
)
HASH_FILES: bool = Field(default=False, description="Should MVT hash output files")
@classmethod
def settings_customise_sources(
cls,
settings_cls: Type[BaseSettings],
init_settings: InitSettingsSource,
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> Tuple[PydanticBaseSettingsSource, ...]:
sources = (
YamlConfigSettingsSource(settings_cls, MVT_CONFIG_PATH),
yaml_source = YamlConfigSettingsSource(settings_cls, MVT_CONFIG_PATH)
sources: Tuple[PydanticBaseSettingsSource, ...] = (
yaml_source,
init_settings,
)
# Load env variables if enabled
if init_settings.init_kwargs.get("load_env", True):
sources = (env_settings,) + sources
# Always load env variables by default
sources = (env_settings,) + sources
return sources
def save_settings(
@@ -94,11 +93,11 @@ class MVTSettings(BaseSettings):
Afterwards we load the settings again, this time including the env variables.
"""
# Set invalid env prefix to avoid loading env variables.
settings = MVTSettings(load_env=False)
settings = cls(load_env=False)
settings.save_settings()
# Load the settings again with any ENV variables.
settings = MVTSettings(load_env=True)
settings = cls(load_env=True)
return settings
+3
View File
@@ -15,6 +15,8 @@ HELP_MSG_HASHES = "Generate hashes of all the files analyzed"
HELP_MSG_VERBOSE = "Verbose mode"
HELP_MSG_CHECK_IOCS = "Compare stored JSON results to provided indicators"
HELP_MSG_STIX2 = "Download public STIX2 indicators"
HELP_MSG_DISABLE_UPDATE_CHECK = "Disable MVT version update check"
HELP_MSG_DISABLE_INDICATOR_UPDATE_CHECK = "Disable indicators update check"
# IOS Specific
HELP_MSG_DECRYPT_BACKUP = "Decrypt an encrypted iTunes backup"
@@ -45,3 +47,4 @@ HELP_MSG_CHECK_ADB_REMOVED_DESCRIPTION = (
HELP_MSG_CHECK_BUGREPORT = "Check an Android Bug Report"
HELP_MSG_CHECK_ANDROID_BACKUP = "Check an Android Backup"
HELP_MSG_CHECK_ANDROIDQF = "Check data collected with AndroidQF"
HELP_MSG_CHECK_INTRUSION_LOGS = "Check Android Intrusion Logging files"
+40 -8
View File
@@ -7,15 +7,15 @@ import glob
import json
import logging
import os
from dataclasses import dataclass
from functools import lru_cache
from typing import Any, Dict, Iterator, List, Optional
from dataclasses import dataclass
import ahocorasick
from appdirs import user_data_dir
from .url import URL
from .config import settings
from .url import URL
MVT_DATA_FOLDER = user_data_dir("mvt")
MVT_INDICATORS_FOLDER = os.path.join(MVT_DATA_FOLDER, "indicators")
@@ -67,9 +67,7 @@ class Indicators:
if os.path.isfile(path) and path.lower().endswith(".stix2"):
self.parse_stix2(path)
elif os.path.isdir(path):
for file in glob.glob(
os.path.join(path, "**", "*.stix2", recursive=True)
):
for file in glob.glob(os.path.join(path, "**", "*.stix2"), recursive=True):
self.parse_stix2(file)
else:
self.log.error(
@@ -117,6 +115,17 @@ class Indicators:
key, value = indicator.get("pattern", "").strip("[]").split("=")
key = key.strip()
# Normalize hash algorithm keys so that both the STIX2-spec-compliant
# form (e.g. file:hashes.'SHA-256', which requires quotes around
# algorithm names that contain hyphens) and the non-standard lowercase
# form (e.g. file:hashes.sha256) are accepted. Strip single quotes and
# hyphens from the algorithm name only, then lowercase it.
for sep in ("hashes.", "cert."):
if sep in key:
prefix, _, algo = key.partition(sep)
key = prefix + sep + algo.replace("'", "").replace("-", "").lower()
break
if key == "domain-name:value":
# We force domain names to lower case.
self._add_indicator(
@@ -350,7 +359,7 @@ class Indicators:
@lru_cache()
def get_ioc_matcher(
self, ioc_type: Optional[str] = None, ioc_list: Optional[list] = None
self, ioc_type: Optional[str] = None, ioc_list: Optional[List[Indicator]] = None
) -> ahocorasick.Automaton:
"""
Build an Aho-Corasick automaton from a list of iocs (i.e indicators)
@@ -370,9 +379,9 @@ class Indicators:
"""
automaton = ahocorasick.Automaton()
if ioc_type:
iocs = self.get_iocs(ioc_type)
iocs: Iterator[Indicator] = self.get_iocs(ioc_type)
elif ioc_list:
iocs = ioc_list
iocs = iter(ioc_list)
else:
raise ValueError("Must provide either ioc_type or ioc_list")
@@ -718,6 +727,29 @@ class Indicators:
return None
def check_receiver_prefix(
self, receiver_name: str
) -> Optional[IndicatorMatch]:
"""Check the provided receiver name against the list of indicators.
An IoC match is detected when a substring of the receiver matches the indicator.
:param receiver_name: Receiver name to check against app ID indicators
:type receiver_name: str
:returns: IndicatorMatch if matched, otherwise None
"""
if not receiver_name:
return None
for ioc in self.get_iocs("app_ids"):
if ioc.value.lower() in receiver_name.lower():
return IndicatorMatch(
ioc=ioc,
message=f'Found a known suspicious receiver with name "{receiver_name}" matching indicators from "{ioc.name}"',
)
return None
def check_android_property_name(
self, property_name: str
) -> Optional[IndicatorMatch]:
+67 -56
View File
@@ -12,74 +12,85 @@ from .updates import IndicatorsUpdates, MVTUpdates
from .version import MVT_VERSION
def check_updates() -> None:
def check_updates(
disable_version_check: bool = False, disable_indicator_check: bool = False
) -> None:
log = logging.getLogger("mvt")
# First we check for MVT version updates.
try:
mvt_updates = MVTUpdates()
latest_version = mvt_updates.check()
except (requests.exceptions.ConnectionError, requests.exceptions.Timeout):
rich_print(
"\t\t[bold]Note: Could not check for MVT updates.[/bold] "
"You may be working offline. Please update MVT regularly."
)
except Exception as e:
log.error("Error encountered when trying to check latest MVT version: %s", e)
else:
if latest_version:
if not disable_version_check:
try:
mvt_updates = MVTUpdates()
latest_version = mvt_updates.check()
except (requests.exceptions.ConnectionError, requests.exceptions.Timeout):
rich_print(
f"\t\t[bold]Version {latest_version} is available! "
"Upgrade mvt with `pip3 install -U mvt`[/bold]"
"\t\t[bold]Note: Could not check for MVT updates.[/bold] "
"You may be working offline. Please update MVT regularly."
)
# Then we check for indicators files updates.
ioc_updates = IndicatorsUpdates()
# Before proceeding, we check if we have downloaded an indicators index.
# If not, there's no point in proceeding with the updates check.
if ioc_updates.get_latest_update() == 0:
rich_print(
"\t\t[bold]You have not yet downloaded any indicators, check "
"the `download-iocs` command![/bold]"
)
return
# We only perform this check at a fixed frequency, in order to not
# overburden the user with too many lookups if the command is being run
# multiple times.
should_check, hours = ioc_updates.should_check()
if not should_check:
rich_print(
f"\t\tIndicators updates checked recently, next automatic check "
f"in {int(hours)} hours"
)
return
try:
ioc_to_update = ioc_updates.check()
except (requests.exceptions.ConnectionError, requests.exceptions.Timeout):
rich_print(
"\t\t[bold]Note: Could not check for indicator updates.[/bold] "
"You may be working offline. Please update MVT indicators regularly."
)
except Exception as e:
log.error("Error encountered when trying to check latest MVT indicators: %s", e)
else:
if ioc_to_update:
rich_print(
"\t\t[bold]There are updates to your indicators files! "
"Run the `download-iocs` command to update![/bold]"
except Exception as e:
log.error(
"Error encountered when trying to check latest MVT version: %s", e
)
else:
rich_print("\t\tYour indicators files seem to be up to date.")
if latest_version:
rich_print(
f"\t\t[bold]Version {latest_version} is available! "
"Upgrade mvt with `pip3 install -U mvt` or with `pipx upgrade mvt`[/bold]"
)
# Then we check for indicators files updates.
if not disable_indicator_check:
ioc_updates = IndicatorsUpdates()
# Before proceeding, we check if we have downloaded an indicators index.
# If not, there's no point in proceeding with the updates check.
if ioc_updates.get_latest_update() == 0:
rich_print(
"\t\t[bold]You have not yet downloaded any indicators, check "
"the `download-iocs` command![/bold]"
)
return
# We only perform this check at a fixed frequency, in order to not
# overburden the user with too many lookups if the command is being run
# multiple times.
should_check, hours = ioc_updates.should_check()
if not should_check:
rich_print(
f"\t\tIndicators updates checked recently, next automatic check "
f"in {int(hours)} hours"
)
return
try:
ioc_to_update = ioc_updates.check()
except (requests.exceptions.ConnectionError, requests.exceptions.Timeout):
rich_print(
"\t\t[bold]Note: Could not check for indicator updates.[/bold] "
"You may be working offline. Please update MVT indicators regularly."
)
except Exception as e:
log.error(
"Error encountered when trying to check latest MVT indicators: %s", e
)
else:
if ioc_to_update:
rich_print(
"\t\t[bold]There are updates to your indicators files! "
"Run the `download-iocs` command to update![/bold]"
)
else:
rich_print("\t\tYour indicators files seem to be up to date.")
def logo() -> None:
def logo(
disable_version_check: bool = False, disable_indicator_check: bool = False
) -> None:
rich_print("\n")
rich_print("\t[bold]MVT[/bold] - Mobile Verification Toolkit")
rich_print("\t\thttps://mvt.re")
rich_print(f"\t\tVersion: {MVT_VERSION}")
check_updates()
check_updates(disable_version_check, disable_indicator_check)
rich_print("\n")
+25 -23
View File
@@ -11,15 +11,15 @@ import re
from dataclasses import asdict, is_dataclass
from typing import Any, Dict, Optional
from .utils import CustomJSONEncoder, exec_or_profile
from .indicators import Indicators
from .alerts import AlertStore
from .indicators import Indicators
from .module_types import (
ModuleResults,
ModuleTimeline,
ModuleSerializedResult,
ModuleAtomicResult,
ModuleResults,
ModuleSerializedResult,
ModuleTimeline,
)
from .utils import CustomJSONEncoder, exec_or_profile
class DatabaseNotFoundError(Exception):
@@ -30,6 +30,10 @@ class DatabaseCorruptedError(Exception):
pass
class EncryptedBackupError(Exception):
pass
class InsufficientPrivileges(Exception):
pass
@@ -77,7 +81,6 @@ class MVTModule:
self.results: ModuleResults = results if results else []
self.timeline: ModuleTimeline = []
self.timeline_detected: ModuleTimeline = []
@classmethod
def from_json(cls, json_path: str, log: logging.Logger):
@@ -106,10 +109,13 @@ class MVTModule:
name = self.get_slug()
if self.results:
converted_results = [
asdict(result) if is_dataclass(result) else result
for result in self.results
]
if isinstance(self.results, dict):
converted_results = self.results
else:
converted_results = [
asdict(result) if is_dataclass(result) else result
for result in self.results
]
results_file_name = f"{name}.json"
results_json_path = os.path.join(self.results_path, results_file_name)
with open(results_json_path, "w", encoding="utf-8") as handle:
@@ -130,7 +136,7 @@ class MVTModule:
detected_json_path = os.path.join(self.results_path, detected_file_name)
with open(detected_json_path, "w", encoding="utf-8") as handle:
json.dump(
self.alertstore.alerts, handle, indent=4, cls=CustomJSONEncoder
self.alertstore.as_json(), handle, indent=4, cls=CustomJSONEncoder
)
def serialize(self, result: ModuleAtomicResult) -> ModuleSerializedResult:
@@ -147,7 +153,10 @@ class MVTModule:
for record in timeline:
timeline_set.add(
json.dumps(
asdict(record) if is_dataclass(record) else record, sort_keys=True
asdict(record)
if is_dataclass(record) and not isinstance(record, type)
else record,
sort_keys=True,
)
)
@@ -162,21 +171,12 @@ class MVTModule:
record: ModuleSerializedResult = self.serialize(result)
if record:
if isinstance(record, list):
self.timeline.extend(record)
self.timeline.extend(record) # type: ignore[arg-type]
else:
self.timeline.append(record)
# for detected in self.alertstore.alerts:
# record = self.serialize(detected)
# if record:
# if isinstance(record, list):
# self.timeline_detected.extend(record)
# else:
# self.timeline_detected.append(record)
self.timeline.append(record) # type: ignore[arg-type]
# De-duplicate timeline entries.
self.timeline = self._deduplicate_timeline(self.timeline)
# self.timeline_detected = self._deduplicate_timeline(self.timeline_detected)
def run(self) -> None:
"""Run the main module procedure."""
@@ -188,6 +188,8 @@ def run_module(module: MVTModule) -> None:
try:
exec_or_profile("module.run()", globals(), locals())
except EncryptedBackupError:
raise
except NotImplementedError:
module.log.exception(
"The run() procedure of module %s was not implemented yet!",
+12 -7
View File
@@ -3,15 +3,17 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from .indicators import Indicator
from dataclasses import dataclass
from typing import List, Union, Optional
from typing import Any, Dict, List, Union
@dataclass
class ModuleAtomicResult:
timestamp: Optional[str]
matched_indicator: Optional[Indicator]
# ModuleAtomicResult is a flexible dictionary that can contain any data.
# Common fields include:
# - timestamp: Optional[str] - timestamp string
# - isodate: Optional[str] - ISO formatted date string
# - matched_indicator: Optional[Indicator] - indicator that matched this result
# - Any other module-specific fields
ModuleAtomicResult = Dict[str, Any]
ModuleResults = List[ModuleAtomicResult]
@@ -26,4 +28,7 @@ class ModuleAtomicTimeline:
ModuleTimeline = List[ModuleAtomicTimeline]
ModuleSerializedResult = Union[ModuleAtomicTimeline, ModuleTimeline]
# ModuleSerializedResult can be a proper timeline object or a plain dict for compatibility
ModuleSerializedResult = Union[
ModuleAtomicTimeline, ModuleTimeline, Dict[str, Any], List[Dict[str, Any]]
]
+24 -5
View File
@@ -12,9 +12,9 @@ import requests
import yaml
from packaging import version
from .config import settings
from .indicators import MVT_DATA_FOLDER, MVT_INDICATORS_FOLDER
from .version import MVT_VERSION
from .config import settings
log = logging.getLogger(__name__)
@@ -24,7 +24,11 @@ INDICATORS_CHECK_FREQUENCY = 12
class MVTUpdates:
def check(self) -> str:
res = requests.get(settings.PYPI_UPDATE_URL, timeout=15)
try:
res = requests.get(str(settings.PYPI_UPDATE_URL), timeout=5)
except requests.exceptions.RequestException as e:
log.error("Failed to check for updates, skipping updates: %s", e)
return ""
data = res.json()
latest_version = data.get("info", {}).get("version", "")
@@ -93,7 +97,12 @@ class IndicatorsUpdates:
url = self.github_raw_url.format(
self.index_owner, self.index_repo, self.index_branch, self.index_path
)
res = requests.get(url, timeout=15)
try:
res = requests.get(url, timeout=5)
except requests.exceptions.RequestException as e:
log.error("Failed to retrieve indicators index from %s: %s", url, e)
return None
if res.status_code != 200:
log.error(
"Failed to retrieve indicators index located at %s (error %d)",
@@ -105,7 +114,12 @@ class IndicatorsUpdates:
return yaml.safe_load(res.content)
def download_remote_ioc(self, ioc_url: str) -> Optional[str]:
res = requests.get(ioc_url, timeout=15)
try:
res = requests.get(ioc_url, timeout=15)
except requests.exceptions.RequestException as e:
log.error("Failed to download indicators file from %s: %s", ioc_url, e)
return None
if res.status_code != 200:
log.error(
"Failed to download indicators file from %s (error %d)",
@@ -171,7 +185,12 @@ class IndicatorsUpdates:
file_commit_url = (
f"https://api.github.com/repos/{owner}/{repo}/commits?path={path}"
)
res = requests.get(file_commit_url, timeout=15)
try:
res = requests.get(file_commit_url, timeout=5)
except requests.exceptions.RequestException as e:
log.error("Failed to get details about file %s: %s", file_commit_url, e)
return -1
if res.status_code != 200:
log.error(
"Failed to get details about file %s (error %d)",
+12 -6
View File
@@ -338,11 +338,12 @@ class URL:
:rtype: str
"""
return (
get_tld(self.url, as_object=True, fix_protocol=True)
.parsed_url.netloc.lower()
.lstrip("www.")
)
tld_obj = get_tld(self.url, as_object=True, fix_protocol=True)
if isinstance(tld_obj, str):
return tld_obj
if tld_obj is None:
return ""
return tld_obj.parsed_url.netloc.lower().lstrip("www.")
def get_top_level(self) -> str:
"""Get only the top-level domain from a URL.
@@ -351,7 +352,12 @@ class URL:
:rtype: str
"""
return get_tld(self.url, as_object=True, fix_protocol=True).fld.lower()
tld_obj = get_tld(self.url, as_object=True, fix_protocol=True)
if isinstance(tld_obj, str):
return tld_obj
if tld_obj is None:
return ""
return tld_obj.fld.lower()
def check_if_shortened(self) -> bool:
"""Check if the URL is among list of shortener services.
+5 -1
View File
@@ -10,6 +10,7 @@ import json
import logging
import os
import re
from dataclasses import asdict, is_dataclass
from typing import Any, Iterator, Union
from .log import MVTLogHandler
@@ -30,6 +31,9 @@ class CustomJSONEncoder(json.JSONEncoder):
"""
def default(self, o):
# Unwrap dataclass instances (such as Indicator) to dict. Skip class itself.
if is_dataclass(o) and not isinstance(o, type):
return asdict(o)
if isinstance(o, bytes):
# Decode as utf-8, replace any invalid UTF-8 bytes with escaped hex
return o.decode("utf-8", errors="backslashreplace")
@@ -235,7 +239,7 @@ def init_logging(verbose: bool = False):
Initialise logging for the MVT module
"""
log = logging.getLogger("mvt")
log.setLevel(logging.INFO)
log.setLevel(logging.DEBUG)
consoleHandler = MVTLogHandler()
consoleHandler.setFormatter(logging.Formatter("%(message)s"))
if verbose:
+1 -1
View File
@@ -3,4 +3,4 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
MVT_VERSION = "3.0.0"
MVT_VERSION = "2026.4.28"
+40 -3
View File
@@ -37,6 +37,8 @@ from mvt.common.help import (
HELP_MSG_CHECK_IOCS,
HELP_MSG_STIX2,
HELP_MSG_CHECK_IOS_BACKUP,
HELP_MSG_DISABLE_UPDATE_CHECK,
HELP_MSG_DISABLE_INDICATOR_UPDATE_CHECK,
)
from .cmd_check_backup import CmdIOSCheckBackup
from .cmd_check_fs import CmdIOSCheckFS
@@ -53,12 +55,37 @@ MVT_IOS_BACKUP_PASSWORD = "MVT_IOS_BACKUP_PASSWORD"
CONTEXT_SETTINGS = dict(help_option_names=["-h", "--help"])
def _get_disable_flags(ctx):
"""Helper function to safely get disable flags from context."""
if ctx.obj is None:
return False, False
return (
ctx.obj.get("disable_version_check", False),
ctx.obj.get("disable_indicator_check", False),
)
# ==============================================================================
# Main
# ==============================================================================
@click.group(invoke_without_command=False)
def cli():
logo()
@click.option(
"--disable-update-check", is_flag=True, help=HELP_MSG_DISABLE_UPDATE_CHECK
)
@click.option(
"--disable-indicator-update-check",
is_flag=True,
help=HELP_MSG_DISABLE_INDICATOR_UPDATE_CHECK,
)
@click.pass_context
def cli(ctx, disable_update_check, disable_indicator_update_check):
ctx.ensure_object(dict)
ctx.obj["disable_version_check"] = disable_update_check
ctx.obj["disable_indicator_check"] = disable_indicator_update_check
logo(
disable_version_check=disable_update_check,
disable_indicator_check=disable_indicator_update_check,
)
# ==============================================================================
@@ -219,6 +246,8 @@ def check_backup(
module_name=module,
module_options=module_options,
hashes=hashes,
disable_version_check=_get_disable_flags(ctx)[0],
disable_indicator_check=_get_disable_flags(ctx)[1],
)
if list_modules:
@@ -263,6 +292,8 @@ def check_fs(ctx, iocs, output, fast, list_modules, module, hashes, verbose, dum
module_name=module,
module_options=module_options,
hashes=hashes,
disable_version_check=_get_disable_flags(ctx)[0],
disable_indicator_check=_get_disable_flags(ctx)[1],
)
if list_modules:
@@ -293,7 +324,13 @@ def check_fs(ctx, iocs, output, fast, list_modules, module, hashes, verbose, dum
@click.argument("FOLDER", type=click.Path(exists=True))
@click.pass_context
def check_iocs(ctx, iocs, list_modules, module, folder):
cmd = CmdCheckIOCS(target_path=folder, ioc_files=iocs, module_name=module)
cmd = CmdCheckIOCS(
target_path=folder,
ioc_files=iocs,
module_name=module,
disable_version_check=_get_disable_flags(ctx)[0],
disable_indicator_check=_get_disable_flags(ctx)[1],
)
cmd.modules = BACKUP_MODULES + FS_MODULES + MIXED_MODULES
if list_modules:
+4
View File
@@ -27,6 +27,8 @@ class CmdIOSCheckBackup(Command):
module_options: Optional[dict] = None,
hashes: bool = False,
sub_command: bool = False,
disable_version_check: bool = False,
disable_indicator_check: bool = False,
) -> None:
super().__init__(
target_path=target_path,
@@ -39,6 +41,8 @@ class CmdIOSCheckBackup(Command):
hashes=hashes,
sub_command=sub_command,
log=log,
disable_version_check=disable_version_check,
disable_indicator_check=disable_indicator_check,
)
self.name = "check-backup"
+4 -1
View File
@@ -27,6 +27,8 @@ class CmdIOSCheckFS(Command):
module_options: Optional[dict] = None,
hashes: bool = False,
sub_command: bool = False,
disable_version_check: bool = False,
disable_indicator_check: bool = False,
) -> None:
super().__init__(
target_path=target_path,
@@ -34,11 +36,12 @@ class CmdIOSCheckFS(Command):
ioc_files=ioc_files,
iocs=iocs,
module_name=module_name,
serial=serial,
module_options=module_options,
hashes=hashes,
sub_command=sub_command,
log=log,
disable_version_check=disable_version_check,
disable_indicator_check=disable_indicator_check,
)
self.name = "check-fs"
+36
View File
@@ -194,5 +194,41 @@
{
"identifier": "iPhone16,2",
"description": "iPhone 15 Pro Max"
},
{
"identifier": "iPhone17,1",
"description": "iPhone 16 Pro"
},
{
"identifier": "iPhone17,2",
"description": "iPhone 16 Pro Max"
},
{
"identifier": "iPhone17,3",
"description": "iPhone 16"
},
{
"identifier": "iPhone17,4",
"description": "iPhone 16 Plus"
},
{
"identifier": "iPhone17,5",
"description": "iPhone 16e"
},
{
"identifier": "iPhone18,1",
"description": "iPhone 17 Pro"
},
{
"identifier": "iPhone18,2",
"description": "iPhone 17 Pro Max"
},
{
"identifier": "iPhone18,3",
"description": "iPhone 17"
},
{
"identifier": "iPhone18,4",
"description": "iPhone Air"
}
]
+136
View File
@@ -631,6 +631,10 @@
"build": "16H81",
"version": "12.5.7"
},
{
"version": "12.5.8",
"build": "16H88"
},
{
"build": "17A577",
"version": "13.0"
@@ -891,6 +895,22 @@
"version": "15.8.2",
"build": "19H384"
},
{
"version": "15.8.4",
"build": "19H390"
},
{
"version": "15.8.5",
"build": "19H394"
},
{
"version": "15.8.6",
"build": "19H402"
},
{
"version": "15.8.7",
"build": "19H411"
},
{
"build": "20A362",
"version": "16.0"
@@ -992,6 +1012,22 @@
"version": "16.7.8",
"build": "20H343"
},
{
"version": "16.7.11",
"build": "20H360"
},
{
"version": "16.7.12",
"build": "20H364"
},
{
"version": "16.7.14",
"build": "20H370"
},
{
"version": "16.7.15",
"build": "20H380"
},
{
"version": "17.0",
"build": "21A327"
@@ -1076,6 +1112,10 @@
"version": "17.6.1",
"build": "21G101"
},
{
"version": "17.7.7",
"build": "21H433"
},
{
"version": "18",
"build": "22A3354"
@@ -1103,5 +1143,101 @@
{
"version": "18.3",
"build": "22D63"
},
{
"version": "18.3.1",
"build": "22D72"
},
{
"version": "18.4",
"build": "22E240"
},
{
"version": "18.4.1",
"build": "22E252"
},
{
"version": "18.5",
"build": "22F76"
},
{
"version": "18.6",
"build": "22G86"
},
{
"version": "18.6.1",
"build": "22G90"
},
{
"version": "18.6.2",
"build": "22G100"
},
{
"version": "18.7",
"build": "22H20"
},
{
"version": "18.7.2",
"build": "22H124"
},
{
"version": "18.7.3",
"build": "22H217"
},
{
"version": "18.7.4",
"build": "22H218"
},
{
"version": "18.7.5",
"build": "22H311"
},
{
"version": "18.7.6",
"build": "22H320"
},
{
"version": "18.7.7",
"build": "22H333"
},
{
"version": "18.7.8",
"build": "22H352"
},
{
"version": "26",
"build": "23A341"
},
{
"version": "26.0.1",
"build": "23A355"
},
{
"version": "26.1",
"build": "23B85"
},
{
"version": "26.2",
"build": "23C55"
},
{
"version": "26.2.1",
"build": "23C71"
},
{
"version": "26.3",
"build": "23D127"
},
{
"version": "26.3.1",
"build": "23D8133"
},
{
"version": "26.4",
"build": "23E246"
},
{
"version": "26.4.2",
"build": "23E261"
}
]
+15 -2
View File
@@ -11,6 +11,7 @@ import os
import os.path
import shutil
import sqlite3
from pathlib import Path
from typing import Optional
from iOSbackup import iOSbackup
@@ -46,18 +47,21 @@ class DecryptBackup:
"""
conn = sqlite3.connect(os.path.join(backup_path, "Manifest.db"))
cur = conn.cursor()
try:
cur = conn.cursor()
cur.execute("SELECT fileID FROM Files LIMIT 1;")
except sqlite3.DatabaseError:
return True
else:
log.critical("The backup does not seem encrypted!")
return False
finally:
conn.close()
def _process_file(
self, relative_path: str, domain: str, item, file_id: str, item_folder: str
) -> None:
assert self._backup is not None
self._backup.getFileDecryptedCopy(
manifestEntry=item, targetName=file_id, targetFolder=item_folder
)
@@ -70,6 +74,9 @@ class DecryptBackup:
)
def process_backup(self) -> None:
assert self._backup is not None
assert self.dest_path is not None
if not os.path.exists(self.dest_path):
os.makedirs(self.dest_path)
@@ -90,6 +97,9 @@ class DecryptBackup:
# This may be a partial backup. Skip files from the manifest
# which do not exist locally.
source_file_path = os.path.join(self.backup_path, file_id[0:2], file_id)
if not Path(source_file_path).resolve().is_relative_to(Path(self.backup_path).resolve()):
log.warning("Skipping unsafe file_id: %r", file_id)
continue
if not os.path.exists(source_file_path):
log.debug(
"Skipping file %s. File not found in encrypted backup directory.",
@@ -97,7 +107,10 @@ class DecryptBackup:
)
continue
item_folder = os.path.join(self.dest_path, file_id[0:2])
item_folder = os.path.join(self.dest_path, file_id[0:2]) # type: ignore[arg-type]
if not Path(os.path.join(item_folder, file_id)).resolve().is_relative_to(Path(self.dest_path).resolve()):
log.warning("Skipping unsafe file_id: %r", file_id)
continue
if not os.path.exists(item_folder):
os.makedirs(item_folder)
+3 -1
View File
@@ -36,9 +36,11 @@ class BackupInfo(IOSExtraction):
results=results,
)
self.results = {}
self.results: dict = {}
def run(self) -> None:
if not self.target_path:
raise DatabaseNotFoundError("target_path is not set")
info_path = os.path.join(self.target_path, "Info.plist")
if not os.path.exists(info_path):
raise DatabaseNotFoundError(
@@ -9,12 +9,12 @@ import plistlib
from base64 import b64encode
from typing import Optional
from mvt.common.utils import convert_datetime_to_iso
from mvt.common.module_types import (
ModuleAtomicResult,
ModuleSerializedResult,
ModuleResults,
ModuleSerializedResult,
)
from mvt.common.utils import convert_datetime_to_iso
from ..base import IOSExtraction
@@ -72,24 +72,17 @@ class ConfigurationProfiles(IOSExtraction):
result["plist"]["PayloadUUID"]
)
if ioc_match:
warning_message = (
f'Found a known malicious configuration profile "{result["plist"]["PayloadDisplayName"]}" with UUID "{result["plist"]["PayloadUUID"]}"',
)
result["matched_indicator"] = ioc_match.ioc
warning_message = f'Found a known malicious configuration profile "{result["plist"]["PayloadDisplayName"]}" with UUID "{result["plist"]["PayloadUUID"]}"'
self.alertstore.critical(
self.get_slug(), warning_message, "", result
warning_message, "", result, matched_indicator=ioc_match.ioc
)
self.alertstore.log_latest()
continue
# Highlight suspicious configuration profiles which may be used
# to hide notifications.
if payload_content["PayloadType"] in ["com.apple.notificationsettings"]:
warning_message = (
f'Found a potentially suspicious configuration profile "{result["plist"]["PayloadDisplayName"]}" with payload type {payload_content["PayloadType"]}',
)
self.alertstore.medum(self.get_slug(), warning_message, "", result)
self.alertstore.log_latest()
warning_message = f'Found a potentially suspicious configuration profile "{result["plist"]["PayloadDisplayName"]}" with payload type {payload_content["PayloadType"]}'
self.alertstore.medium(warning_message, "", result)
continue
def run(self) -> None:
Loaded 100 of 156 files, more files were not shown because too many files have changed in this diff. Show more