Compare commits

..
Author SHA1 Message Date
Janik Besendorf 895307e242 Merge main and resolve upstream integration conflicts 2026-10-11 14:27:41 +02:00
Janik Besendorf e5a9a1f919 Normalize browser history ZIP member paths 2026-10-11 09:49:42 +02:00
besendorf b3aa443819 Merge pull request #970 from WingManOO7/fix/settings-leap-day
Keep settings history timestamps on 29 February
2026-10-09 12:52:44 +02:00
besendorf aac41ec8c2 Test settings leap-day resolution across a skipped century
Ensure a February 29 history entry in a January 2104 dump resolves to 2096, since 2100 is not a leap year.
2026-10-09 03:49:54 -07:00
besendorf 1e05f438d7 Include the eighth prior year when resolving settings leap days
Adjust the year range for leap year calculation to account for century years.
2026-10-09 03:48:37 -07:00
Muhammad Mohid Shafiq 10f739d669 Keep settings history timestamps on 29 February
dumpsys prints the settings change history without a year, so
_resolve_timestamp() parsed the value with strptime("%m-%d ...") and only
then replaced the year. strptime() parses a value without a year in 1900,
which has no 29 February, so every change made on a leap day raised
ValueError and lost its timestamp, dropping it from the timeline and from
the alert of a dangerous setting.

Parse the value together with a year instead, starting from the year the
section was dumped and going back until the date exists and is not later
than the dump. Eight years always reach a leap year.

This also removes the DeprecationWarning Python 3.13 raises for parsing a
day of month without a year (python/cpython#70647).
2026-10-08 19:35:29 +05:00
besendorf 94e9cd3712 Merge pull request #968 from va-resident/fix/adb-state-closing-brace
Fix spurious ADB state parse error on devices without stored keys
2026-10-07 17:27:15 +02:00
besendorf 300cc26112 Merge branch 'main' into fix/adb-state-closing-brace 2026-10-07 17:25:29 +02:00
besendorf 3c12e1a019 Merge pull request #925 from mvt-project/docs/recommend-pymobiledevice3
Recommend pymobiledevice3 for iOS acquisition
2026-10-07 16:12:55 +02:00
besendorf c86bc93852 Merge branch 'main' into docs/recommend-pymobiledevice3 2026-10-07 16:11:14 +02:00
va@resident 3a462e1e2c Fix spurious ADB state parse error on devices without stored keys
Since #957 the parsed body keeps the closing brace of debugging_manager
and the blank lines before it. The brace popped two levels, and a blank
line after system_keys raised IndexError. Don't pop when the
indentation drops on a closing brace, and skip blank lines outside
multiline values.

Fixes #967.
2026-10-06 15:29:41 +05:00
besendorf c95eeed825 Merge pull request #964 from forest-savage1234/docs/python-minimum-q093-20260929
Correct the Python minimum in installation documentation
2026-09-30 10:53:41 +02:00
Forest Savage 60d1b414fe Correct the Python minimum in installation documentation 2026-09-29 21:13:11 -08:00
besendorf 563fe44bf3 Merge pull request #959 from mvt-project/auto/add-new-ios-releases
[auto] Update iOS releases and versions
2026-09-29 20:44:00 +02:00
besendorf 85fca1c825 Merge branch 'main' into auto/add-new-ios-releases 2026-09-29 20:42:15 +02:00
besendorf 3704bf26c0 Merge pull request #960 from Yi-111-a/fix/manifest-flag-missing-backup-files
fix(manifest): flag backup files listed in the manifest but not stored
2026-09-29 20:41:50 +02:00
Janik Besendorf a80b0fff2e fix(manifest): validate backup inventory before marking files missing 2026-09-29 18:40:02 +02:00
Yi-111-a f98f4f6cd2 fix(manifest): flag backup files listed in the manifest but not stored
An incomplete iTunes backup still lists in its Manifest.db the files it
failed to acquire, so a module which found nothing for one of them looks
exactly like a module which found nothing on a device that never had the
artifact. The Manifest module now records a "missing" flag on those
records and reports the total, so a gap in the acquisition is visible in
manifest.json and in the command output.

The stored file IDs are collected by walking the backup folder once, which
keeps the check off the per-entry filesystem lookup path: on a 20k entry
manifest the module runs in the same time as before the change.
2026-09-29 23:59:15 +08:00
DonnchaC 3e99412c32 Add new iOS versions and build numbers 2026-09-28 18:17:44 +00:00
besendorf fd37c4d8d9 Merge pull request #957 from itzzdev09/fix/adb-state-bounds-main
Bound the ADB manager state by matching braces
2026-09-28 16:16:03 +02:00
Janik Besendorf c615d5e304 Merge main into PR #957 and preserve ADB state bounds 2026-09-28 16:12:15 +02:00
besendorf 9bdc25f57d Merge pull request #949 from NeuroSaki987/codex/fix-windows-dumpsys-crlf
fix: parse dumpsys ADB output with CRLF line endings
2026-09-28 16:06:05 +02:00
besendorf 8238d80035 Merge branch 'main' into codex/fix-windows-dumpsys-crlf 2026-09-28 16:04:49 +02:00
besendorf 22d97f0a88 Merge pull request #954 from SomeoneUnlicensed/fix/windows-tests
Make the test suite pass on Windows
2026-09-28 16:01:08 +02:00
besendorf d224db5a78 Merge branch 'main' into fix/windows-tests 2026-09-28 15:54:55 +02:00
Janik Besendorf 658c7aa327 Test Windows path normalization and preserve backup safety coverage 2026-09-28 15:47:15 +02:00
itzzdev09 9316ac7fb0 Bound the ADB manager state by matching braces 2026-09-28 02:07:03 +05:30
besendorf 6ddb2c9671 Merge branch 'main' into codex/fix-windows-dumpsys-crlf 2026-09-27 22:25:21 +02:00
besendorf 165d87b97f Merge pull request #956 from itzzdev09/fix/battery-daily-malformed-update
Skip a malformed battery daily Update line instead of aborting
2026-09-27 20:44:35 +02:00
itzzdev09 d2e992e40e Skip a malformed battery daily Update line instead of aborting 2026-09-28 00:07:17 +05:30
besendorf aa3ae511ad Merge branch 'main' into codex/fix-windows-dumpsys-crlf 2026-09-27 19:24:54 +02:00
besendorf 7700749cfa Merge pull request #950 from itzzdev09/fix/url-domain-www-prefix
Strip only a whole www. prefix from parsed domains
2026-09-27 19:24:23 +02:00
besendorf 6eb1a53873 Merge branch 'main' into fix/url-domain-www-prefix 2026-09-27 19:20:51 +02:00
Neruo Saki 02aa23cb81 Merge branch 'main' into codex/fix-windows-dumpsys-crlf 2026-09-27 17:54:12 +08:00
besendorf 9c3db579ee Merge pull request #946 from va-resident/fix/accessibility-installed-service-count
Carry the accessibility service count the dump states
2026-09-25 19:38:10 +02:00
besendorf 983e970443 Merge branch 'main' into fix/accessibility-installed-service-count 2026-09-25 19:36:53 +02:00
besendorf cfaeb99bf7 Merge pull request #944 from va-resident/fix/bugreport-oem-wrapper-archive
Descend into an OEM wrapper archive in check-bugreport
2026-09-25 19:35:42 +02:00
besendorf 00a5f11124 Merge branch 'main' into fix/accessibility-installed-service-count 2026-09-25 19:30:09 +02:00
va-resident b1b9958f4d Descend into an OEM wrapper archive in check-bugreport
MIUI / HyperOS hands out a zip of app logs, ANR traces and tcpdump captures
with the real bugreport-<device>-<timestamp>.zip nested inside. The outer
archive carries none of the entry points the bug report modules read, so every
module reported it found no files and check-bugreport still exited 0 with an
empty result — an empty analysis that looks like a finished one.

Name the entry points once in modules/bugreport/base.py, next to the
_get_dumpstate_file() that tries them, and when an archive has none of them,
open its zip members in memory and use the first one that does.

Measured over 44 bug report collections: the 14 wrapper collections go from 0
artifacts to 260 artifacts and 586638 records; the 30 normal collections are
unchanged, the descent being unreachable for them.

Fixes #935
2026-09-25 21:21:40 +04:00
va-resident ad6caa155f Track accessibility state sections and unnamed services per user
Two issues from review:

* The set of printed state sections was global, so a section printed for
  one user decided the flags of another. A user with an installed list and
  no enabled section was marked enabled=False and got a LOW "installed, not
  enabled" alert. Sections are now tracked per user.

* A count-only record was added only when a user had no named component at
  all. A dump stating installedServiceCount=2 and naming one service read
  as complete. The count-only record is now added whenever the stated count
  exceeds the distinct named components for that user, and carries the
  difference in a new unnamed_service_count field. The module summary adds
  up that remainder.
2026-09-25 20:55:01 +04:00
besendorf c2030e6e84 Merge pull request #945 from va-resident/fix/bugreport-section-duration-line
Do not treat a section's timing line as a section boundary
2026-09-25 18:36:28 +02:00
SomeoneUnlicensed 99b3b5f014 Make the test suite pass on Windows
The Filesystem module stored the paths of an iOS dump with the separator
of the system checking it, so on Windows the process and file path
indicators, which split paths on "/", never matched. It now stores them
as POSIX paths.

The rest are test fixes:
- the completion install tests also redirect USERPROFILE, which
  Path.home() reads on Windows; they wrote to the real home folder;
- the plugin table helpers accept the light header Rich draws on
  consoles that cannot show the heavy one;
- the completion test quotes the command path, whose backslashes were
  dropped when COMP_WORDS was split;
- tests that need symbolic links or the sqlite3 binary are skipped when
  those are not available;
- two assertions no longer depend on the path separator or on the line
  ending text mode writes.
2026-09-24 21:29:45 +03:00
besendorf 9ca4254649 Fix formatting in test_artifact_dumpsys_adb.py 2026-09-24 09:38:18 -07:00
besendorf f1e52b297a test: cover mixed ADB state line endings
Regression for CRLF after the manager brace and LF after the outer brace.
2026-09-24 04:40:48 -07:00
besendorf 8c5278f540 Refactor JSON extraction logic in dumpsys_adb.py 2026-09-24 04:37:25 -07:00
besendorf df3b130ca8 Test w-prefixed subdomain indicator matching
Add test for URL matching with prefixed subdomain.
2026-09-24 04:34:52 -07:00
besendorf 3bb652f2d2 Test w-prefixed shortener detection 2026-09-24 04:29:33 -07:00
besendorf c399608345 Merge branch 'main' into fix/url-domain-www-prefix 2026-09-24 13:24:47 +02:00
itzzdev09 7f19d96297 Strip only a whole www. prefix from parsed domains 2026-09-23 19:27:08 +05:30
StarRailHub ff5ebf73cc fix: parse dumpsys ADB output with CRLF line endings 2026-09-23 12:11:58 +08:00
va-resident bdbc07a3b3 Carry the accessibility service count the dump states
Most builds never print the `installed services: {...}` block. They state
installedServiceCount=N in the user's attributes line and list nothing, so the
parser recorded no service at all and MVT logged "Identified a total of 0
accessibility services" — an artifact that says "no accessibility services"
about a dump that said there are five. The dump pasted in #744 is itself an
example: it states installedServiceCount=6 and names none.

Parse the count per user and, for a user whose services the dump did not list,
keep one record carrying it, raised as LOW: a count is a coverage statement,
not a running service.

Name the service state in the alert and split its severity, as asked on #744:
a service the dump states is switched off is LOW, enabled or bound is MEDIUM,
and a dump that does not state the enabled state stays MEDIUM, because "not
stated" is not "not enabled". A state section the dump never printed now reads
None rather than False. IOC matching is unaffected by the state: a disabled
service still returns CRITICAL on a match.

Measured over 163 bug reports carrying an accessibility section: MEDIUM alerts
305 -> 4, LOW 0 -> 407. The four that stay MEDIUM are the only records in the
set with enabled=True.

Fixes #744
2026-09-22 18:13:57 +03:00
va-resident 6f65eee85a Do not treat a section's timing line as a section boundary
extract_command_section() ends a section at any line starting with "------".
dumpstate prints a section's timing line when THAT section finishes, and it can
land in the middle of the section currently being written, so the section is
truncated at an arbitrary point and the rest is silently dropped.

Skip the timing line instead of returning it: it never reaches a parser as
content, and the real "------ <NAME> ------" boundary still ends the section.

Measured over 40 bug reports carrying a dumpstate: SYSTEM PROPERTIES was
truncated on 6 of them, losing 6559 properties, and 3 parsed no property at
all. On one Samsung archive getprop goes from 418 to 1259 properties and from
0 to 473 ro.* ones, bringing back ro.product.model,
ro.build.version.security_patch and ro.boot.verifiedbootstate. The other 34
archives are byte-identical.

Fixes #938
2026-09-22 18:11:42 +03:00
besendorf dbbcdd52df Merge branch 'main' into docs/recommend-pymobiledevice3 2026-09-22 16:11:02 +02:00
besendorf 777db67291 Merge pull request #926 from mvt-project/besendorf-patch-5
Remove adb-shell dependency from pyproject.toml
2026-09-22 15:36:20 +02:00
besendorf a0bc233eec Merge branch 'main' into besendorf-patch-5 2026-09-22 15:34:36 +02:00
Janik Besendorf 1db5d239d0 Merge main into PR 926 2026-09-22 15:33:17 +02:00
Donncha Ó Cearbhaill 2426178baa Enable automated PyPI deployments for tagged releases (#943)
* Limit PyPi deploys to tag relaeses

* Remove unnecessary tests and docs from sdist build
2026-09-22 13:02:49 +02:00
besendorf e0697a58e9 Merge pull request #937 from va-resident/fix/mountinfo-rw-both-layers
Fix mountinfo read-write detection across both option layers
2026-09-18 17:48:33 +02:00
va-residentandClaude Opus 5 d0fc0379b9 Fix mountinfo read-write detection across both option layers
/proc/PID/mountinfo carries two option sets with different meaning: fields[5]
are the per-mount (VFS) flags, the field after the "-" separator belongs to the
superblock. A mount is writable only if both allow it.

parse_mountinfo() merged both into one list and set
is_read_write = "rw" in options, so a "rw" VFS mount over a read-only
superblock was reported as writable. On stock Xiaomi-family builds that made
every read-only mi_ext customisation overlay a HIGH "system partition is
mounted as read-write".

Require "rw" in both layers, and let "rw" count as a suspicious mount option
only when the mount is actually writable; remount, noatime and nodiratime keep
their current meaning in either layer. mount_options and options_list still
carry both layers, so nothing downstream loses data.

Measured on 50 bug reports carrying mountinfo - the 28 where the rule changes
the output plus 22 controls, 14 brands, Android 10-16: HIGH 94 -> 0,
MEDIUM 119 -> 25, the 22 controls identical, and 36233 mount entries parsed
either way. Every removed alert is a read-only superblock under a "rw" VFS
mount; no report gains an alert. A partition that really is writable still
raises the HIGH, which the new test asserts explicitly.

Fixes #936

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 18:43:06 +03:00
besendorf 7973e90589 Merge pull request #934 from mvt-project/chore/update-iphone-backup-decrypt-0.10.0
Bump iphone_backup_decrypt to 0.10.0
2026-09-18 10:36:59 +02:00
Janik Besendorf 935747519f Bump iphone_backup_decrypt to 0.10.0 2026-09-18 10:22:21 +02:00
besendorf 178ac52aab Merge pull request #927 from mvt-project/auto/add-new-ios-releases
[auto] Update iOS releases and versions
2026-09-15 03:37:45 +02:00
DonnchaC f20fd14075 Add new iOS versions and build numbers 2026-09-15 00:37:07 +00:00
Janik Besendorf a9d787c446 Remove unused libusb1 and refresh dependency lockfile 2026-09-14 00:30:48 +02:00
besendorf 518becb825 Remove adb-shell dependency from pyproject.toml
not used anymore
2026-09-13 20:06:15 +02:00
Janik Besendorf d0cf303aa2 Replace libimobiledevice with pymobiledevice3 in Docker images 2026-09-11 11:21:57 +02:00
Janik Besendorf 86b217a926 Recommend pymobiledevice3 for iOS acquisition 2026-09-11 11:14:53 +02:00
besendorf a3d289c709 Merge pull request #924 from mvt-project/auto/add-new-ios-releases
[auto] Update iOS releases and versions
2026-09-08 19:52:51 +01:00
DonnchaC b4b0a06470 Add new iOS versions and build numbers 2026-09-08 18:16:19 +00:00
Donncha Ó Cearbhaill eceafdc785 Keep the test suite out of the user's MVT folders (#923)
The tests read the settings file of whoever runs them, and importing
mvt.common.config writes it back. On a machine whose config.yaml sets
NETWORK_ACCESS_ALLOWED to false, the plugin update and URL batch tests
fail before their mocked requests are reached, and every Command run in
the suite parses the indicators downloaded on that machine, which made
the suite take minutes instead of seconds.

MVT_CONFIG_FOLDER and MVT_DATA_FOLDER in the environment now relocate
the settings file and the downloaded indicators with their update-check
state. The test conftest points both at a throwaway folder before any
mvt module is imported, and removes it at exit. Subprocesses started by
the tests inherit the variables; the isolated interpreter helper already
gives them a temporary home.

On the machine that prompted this the suite goes from 6 failures in six
minutes to none in seven seconds, and config.yaml is left alone.
2026-09-08 01:04:43 +01:00
Donncha Ó Cearbhaill ffae240355 Skip AppleDouble sidecars when listing a sysdiagnose (#922)
Device-generated sysdiagnose archives carry a ._name entry beside every
file that has extended attributes, an ACL or Finder info; one iOS 26
archive held 1234 of them among 3648 members, and the count grows with
each release. bsdtar folds them back into the file on extraction and
hides them from listings, but tarfile returns them as regular members,
so check-sysdiagnose extracted them and handed them to every module.
A module that globs for plists or logs then tries to parse AppleDouble
headers and logs one warning per sidecar.

Leave them out of the file list, both for archives and for folders
extracted on a system that keeps them as files.
2026-09-08 01:03:05 +01:00
3623e430f4 Defer indicator loading until first use (#919)
* Defer indicator loading until first use

* Load indicators once at the start of a run

The lazy property loads indicators wherever they are first read, which
in Command.run() is inside the module loop, after init(). For
check-androidqf that means the whole acquisition is walked before a
missing --iocs file is reported, and the STIX parsing lines land between
the module list and the first module.

Read the indicators once after the module list is settled, so that a bad
--module name still loads nothing, and hand the same object to every
module. check-iocs reads them once too, so that a wrong --iocs path is
reported even when no stored result matches a module.

* Drop two test assertions the change does not need

Retrying after a failed indicator load is incidental to the property
rather than a requirement, so nothing should pin it. The exact wording
of the check-backup rejection belongs to that command's own tests; exit
code 1 already shows the path was rejected before indicators were
loaded.

* Create the output folder and command.log when a run starts

Command.__init__ created the --output folder and attached the
command.log handler, so `--list-modules -o out` or a rejected target
path left an empty folder behind for a run that never happened.

Do both at the top of run(), before the module list is resolved so that
its warnings still reach the log. The nested commands run by
check-androidqf re-attach the handler at the start of their runs, as
they did at construction. Lines the CLI logs between construction and
run(), such as the target path being checked, no longer reach
command.log; info.json records the target path.

This is the remaining part of #888.

* Cache the indicators with functools.cached_property

A property with a setter and a backing attribute does by hand what
cached_property does: compute on first read, store the result on the
instance, and accept assignment, which is how nested commands receive
their parent's indicators. A failed load is still not cached.

* Announce the target from the command so that it reaches command.log

The CLI logged which backup, filesystem or acquisition it was about to
check just before run(), which is now before command.log exists, so the
line only reached the console. Each command logs it from init() instead,
which runs once the log is attached, and run() logs the module list
after init() so that the target still comes first. Nested commands
without a target path log nothing, as before.

* Log the Android backup path from the typed local

mypy cannot determine the type of target_path in this command, which the
surrounding lines already silence, so read the announced path from the
local that carries the type instead of adding another ignore.

---------

Co-authored-by: bitmeta69 <206962233+bitmeta69@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
2026-09-08 01:01:02 +01:00
Donncha Ó Cearbhaill 60e652bafc Read bugreport file timestamps in the device's timezone (#921)
A bugreport zip stores each entry's time as the device's wall clock, and
an unpacked bugreport's mtimes are whatever the extraction left. The
bugreport modules read both as naive local datetimes, so a tombstone's
file_timestamp moved with the analysing machine's timezone and sat three
hours from the crash time the tombstone itself records for a device in
Nairobi.

check-bugreport now resolves the device's timezone once, from --timezone
or from persist.sys.timezone in the dumpstate's SYSTEM PROPERTIES, and
hands it to the modules as device_timezone the way check-androidqf does;
a zone already known, as when androidqf drives the bugreport inside its
own archive, is kept. Zip entry times are read in that zone and the
mtimes of an unpacked bugreport as UTC instants, so convert_datetime_to_iso
writes both in UTC. Without a zone the wall clock is kept naive and a
warning says so, and an unpacked bugreport gets a warning that its file
timestamps are the extraction's, not the device's. BugReportTimestamps
uses the same reading instead of parsing the properties itself.
2026-09-08 00:08:30 +01:00
Donncha Ó Cearbhaill 92f780df5a Report an unreadable sysdiagnose archive instead of aborting silently (#920)
A sysdiagnose tarball whose download stopped halfway ends in an EOFError
from gzip while check-sysdiagnose extracts it. Click turns EOFError into
click.Abort, so the command printed nothing but "Aborted!", even with
-v. The extraction now catches the read errors an archive can raise,
names the file and the reason at critical level, says the file may be
truncated or not a gzip tarball, and exits 1. A plain .tar given to the
gzip reader gets the same message rather than a traceback.
2026-09-07 23:58:43 +01:00
Donncha Ó Cearbhaill 2eb40b85cf Add a SysdiagnoseInfo module to check-sysdiagnose (#917)
* Add a SysdiagnoseInfo module to check-sysdiagnose

check-sysdiagnose had no module of its own: it prepared the archive for
plugin modules and refused to run without one. SysdiagnoseInfo is the
first built-in module. It writes sysdiagnose_info.json with details
about the device and the archive: product type and model, iOS version
and build, serial number, IMEI, MEID and UDID from remotectl_dumpstate.txt
and the mobile activation request, the Apple account name and email from
the App Store daemon database, and the archive's original file name and
creation time from sysdiagnose.log. The build is checked against the
known iOS versions the way BackupInfo does.

The App Store database is copied out of the archive together with its
-wal and -shm sidecars before it is opened, so rows still in the
write-ahead log are read.

With a built-in module the command's list is never empty, so the "no
custom modules" error and its test go. The module joins
IOS_CHECK_IOCS_MODULES like every other module that writes a results
file.

* Note that newer sysdiagnoses lack the App Store daemon database

* Keep refusing check-sysdiagnose runs without a custom module

* Warn instead of refusing when no forensic sysdiagnose module is loaded
2026-09-05 23:45:14 +02:00
Donncha Ó Cearbhaill b9055e365c Merge pull request #916 from mvt-project/fix/settings-env-persist
Stop persisting environment variables to config.yaml
2026-09-05 16:44:40 +02:00
Donncha Ó Cearbhaill fc217e9608 Merge branch 'main' into fix/settings-env-persist 2026-09-05 16:43:43 +02:00
Donncha Ó Cearbhaill d7148c03b4 Stop persisting environment variables to config.yaml
MVTSettings.initialise() constructs the settings once with load_env=False
and writes the result to config.yaml, so that values taken from MVT_*
environment variables are never persisted, then constructs them again
with the environment applied.

Since #716 settings_customise_sources() has added env_settings
unconditionally, making load_env dead code. Any MVT_* variable set for a
single run, including MVT_IOS_BACKUP_PASSWORD, MVT_ANDROID_BACKUP_PASSWORD
and MVT_VT_API_KEY, was written in plaintext to config.yaml and read back
on every later run.

Only add env_settings when load_env is true, and add a regression test.

Fixes #915
2026-09-05 16:41:50 +02:00
Donncha Ó Cearbhaill 0ddcab904d Merge pull request #914 from DonnchaC/feature/bugreport-settings-records
Parse dumpsys settings as per-record results
2026-09-05 16:32:02 +02:00
Donncha Ó Cearbhaill 858496e60b End a settings record at a blank line
dumpsys prints a blank line after every namespace block and after a
change history, and the generation registry and any vendor dumps follow
the last block before the section trailer. A record ran until the next
`_id:` line, heading or trailer, so the last row of the section
absorbed those dumps into whichever field came last.

A blank line now closes the record being read; lines that follow it
without an `_id:` are skipped until the next row or heading. The fixture
carries a generation registry after its last block, modelled on the
AOSP dump, and the last row is pinned to exactly its own fields.
2026-09-05 16:17:43 +02:00
Donncha Ó Cearbhaill e5fced2a08 Peel tag and restore flags off settings rows
A dumpsys settings row can end in `tag:` and, on some vendor builds, in
`isValuePreservedInRestore:` or a bare `notPreservedInRestore` token.
The parser only peeled `default:` and `defaultSystemSet:` off the end
of a record, so on a row without a default those tokens stayed inside
the value, and on a row with one they landed in `defaultSystemSet`. A
value of `0 tag:null` is not the safe value `0`, so a setting at its
safe value was reported as dangerous; these are the false positives
measured in #912.

The metadata keys are ordered fields like the rest of the record, so
`_split_fields` reads them now, which also replaces the separate
handling of the default. The bare token has no `key:` shape and is
printed last, so it is stripped first and recorded as
`isValuePreservedInRestore: false`.

The bugreport fixture gains the three row shapes from #912: a restore
flag after `defaultSystemSet:`, and a `tag:` or a bare token directly
after the value. Two of them sit on dangerous settings at their safe
value, so the unchanged alert count of one is the false-positive check.
2026-09-05 15:56:55 +02:00
Donncha Ó Cearbhaill fe505c63a6 Merge branch 'main' into feature/bugreport-settings-records 2026-09-05 00:59:11 +02:00
Donncha Ó Cearbhaill 25a571c462 Merge pull request #913 from mvt-project/fix/coverage-logging
Fix CI coverage reports and fix matrix testing each Python version
2026-09-04 20:34:18 +02:00
Donncha Ó Cearbhaill 03012d2c2d Fix CI coverage comment and test each matrix Python version
The coverage comment never posted: the per-file table with a link on
every file and missing line range exceeds GitHub's 65536-character
comment limit, so PRs got only a badge. Every matrix job also raced to
post the same comment, and the action was unpinned at @main.

Post from one job only, limit the table to files changed in the PR,
drop per-line links, and pin the action. Write the full coverage table
to the job summary as well, which also works for PRs from forks where
the token is read-only.

Set UV_PYTHON from the matrix. Without it, .python-version pins 3.10
and `uv run` rebuilt the venv with 3.10 after `uv sync --python X`, so
all five matrix jobs were testing on Python 3.10.
2026-09-04 20:25:07 +02:00
Donncha Ó Cearbhaill 0e231eefaf Parse dumpsys settings as per-record results
The `dumpsys settings` parser matched a single regex per line, which
truncated every value that spans more than one line and, when
`defaultSystemSet:` did not fall on the first line, left the trailing
`default:` metadata inside the value. It also keyed results by setting
name within a namespace, so a name recorded twice kept only the last row
and a row without a `pkg:` field was dropped entirely.

Replace it with a line loop that accumulates one record at a time and
splits the `key:value` fields once the whole record has been read.
Results become a list of records carrying the fields dumpsys prints:
namespace, user, _id, name, value, pkg, default and defaultSystemSet,
plus the per-setting change history. History timestamps are printed
without a year, so they are resolved against the "ending at:" time of
the section and serialized into the timeline. This shows which package
changed a security-relevant setting, and when.

The androidqf settings module shares this artifact, so it now emits the
same record shape.
2026-09-04 17:44:51 +02:00
besendorf a463e8509c Merge pull request #774 from mvt-project/backup_extraction
Replace iOSbackup with iphone_backup_decrypt
2026-09-02 19:29:12 +02:00
Janik Besendorf ca063a2ddc Merge current main into backup decryption 2026-09-02 19:27:52 +02:00
Donncha Ó Cearbhaill ece44a7e7f Merge pull request #910 from mvt-project/feature/monthly-releases
Release monthly from a scheduled workflow
2026-09-01 19:41:01 +02:00
Donncha Ó Cearbhaill 27cfe7f1e7 Release weekly instead of monthly 2026-09-01 19:31:23 +02:00
Donncha Ó Cearbhaill 531d63ab07 Release monthly from a scheduled workflow
The package version now comes from the latest v* git tag, so a release
is a tag and nothing else. A scheduled workflow tags main on the first
of the month when something was merged since the last release, creates
the GitHub release with generated notes, and publishes to PyPI and the
container registry. Pushing a v* tag by hand goes through the same path.
2026-09-01 15:49:25 +02:00
Donncha Ó Cearbhaill b18afb4a48 Merge pull request #871 from mvt-project/fix/bugreport-parser-coverage
Fix Android bugreport parser coverage
2026-08-30 14:50:39 +02:00
Janik Besendorf c5d7f5a1d0 Parse Android browser history databases 2026-08-17 14:12:50 +02:00
Janik Besendorf 545ac19158 Restore concurrent backup decryption 2026-08-17 13:24:30 +02:00
Codex 2de8312349 Merge main and fix encrypted backup extraction 2026-08-16 22:39:08 +05:30
besendorf 8b195d2696 Merge branch 'main' into backup_extraction 2026-04-27 15:03:42 +02:00
Janik Besendorf 39a26d0f0b Replace iOSbackup with iphone_backup_decrypt
Replace the unmaintained iOSbackup dependency with iphone_backup_decrypt
(MIT licensed, actively maintained). This fixes file corruption caused by
iOSbackup truncating files to inaccurate sizes from backup metadata.

The extract-key command and --key-file option are preserved via an
MVTEncryptedBackup subclass that patches the keybag unlock to
capture/reuse the derived PBKDF2 key.

Closes #669
2026-04-12 10:51:56 +02:00
92 changed files with 4250 additions and 698 deletions

No files matched your search

@@ -37,6 +37,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v7
with:
fetch-depth: 0 # the package version comes from the git tags
# Uses the `docker/login-action` action to log in to the Container registry registry using the account and password that will publish the packages. Once published, the packages are scoped to the account defined here.
- name: Log in to the Container registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
+45
View File
@@ -0,0 +1,45 @@
name: Release
on:
schedule:
- cron: "0 9 * * 1" # Mondays, 09:00 UTC
workflow_dispatch:
push:
tags: ["v*"]
jobs:
tag:
if: github.ref_type != 'tag'
runs-on: ubuntu-latest
permissions:
contents: write
actions: write
env:
GH_TOKEN: ${{ github.token }}
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- run: |
last=$(git describe --tags --abbrev=0 --match 'v*')
if [ "$(git rev-list "$last..HEAD" --count)" = 0 ]; then
echo "Nothing merged since $last"; exit 0
fi
tag=v$(date -u +%Y.%-m.%-d)
gh release create "$tag" --target "$GITHUB_SHA" --generate-notes
# Events made with GITHUB_TOKEN don't start other workflows; run them by hand.
gh workflow run release.yml --ref "$tag"
gh workflow run publish-release-docker.yml --ref "$tag"
publish:
if: github.ref_type == 'tag'
runs-on: ubuntu-latest
environment: pypi # only tag refs may deploy; the PyPI trusted publisher requires it
permissions:
id-token: write # PyPI trusted publishing
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- uses: astral-sh/setup-uv@v10.0.0
- run: uv build && uv publish
+19 -4
View File
@@ -5,6 +5,10 @@ on:
pull_request:
branches: [ main ]
permissions:
contents: read
pull-requests: write # coverage comment
jobs:
build:
name: Run Python Tests
@@ -13,6 +17,10 @@ jobs:
fail-fast: false
matrix:
python-version: ['3.10', '3.11', '3.12', '3.13', '3.14']
env:
# Takes precedence over .python-version, which otherwise makes `uv run`
# rebuild the venv with 3.10 and test every matrix entry on 3.10.
UV_PYTHON: ${{ matrix.python-version }}
steps:
- uses: actions/checkout@v7
@@ -26,16 +34,23 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Install Python dependencies
run: |
uv sync --locked --group dev --python ${{ matrix.python-version }}
uv sync --locked --group dev
- name: Test with pytest
run: |
set -o pipefail
make test-ci | tee pytest-coverage.txt
- name: Coverage job summary
run: uv run coverage report --format=markdown --show-missing --skip-covered >> "$GITHUB_STEP_SUMMARY"
- name: Pytest coverage comment
continue-on-error: true # Workflows running on a fork can't post comments
uses: MishaKav/pytest-coverage-comment@main
if: github.event_name == 'pull_request'
# One comment per PR, not one per matrix entry. PRs from forks get a
# read-only token and can't post; the job summary above still works.
if: github.event_name == 'pull_request' && matrix.python-version == '3.13'
continue-on-error: true
uses: MishaKav/pytest-coverage-comment@v1.12.2
with:
pytest-coverage-path: ./pytest-coverage.txt
junitxml-path: ./pytest.xml
# The full table with per-line links exceeds GitHub's 65536-char comment limit.
report-only-changed-files: true
remove-links-to-lines: true
+14 -116
View File
@@ -1,107 +1,3 @@
# Base image for building libraries
# ---------------------------------
FROM ubuntu:22.04 AS build-base
ARG DEBIAN_FRONTEND=noninteractive
# Install build tools and dependencies
RUN apt-get update \
&& apt-get install -y \
build-essential \
git \
autoconf \
automake \
libtool-bin \
pkg-config \
libcurl4-openssl-dev \
libusb-1.0-0-dev \
libssl-dev \
udev \
&& rm -rf /var/lib/apt/lists/*
# libplist
# --------
FROM build-base AS build-libplist
# Build
RUN git clone https://github.com/libimobiledevice/libplist && cd libplist \
&& ./autogen.sh && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf libplist
# libimobiledevice-glue
# ---------------------
FROM build-base AS build-libimobiledevice-glue
# Install dependencies
COPY --from=build-libplist /build /
# Build
RUN git clone https://github.com/libimobiledevice/libimobiledevice-glue && cd libimobiledevice-glue \
&& ./autogen.sh && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf libimobiledevice-glue
# libtatsu
# --------
FROM build-base AS build-libtatsu
# Install dependencies
COPY --from=build-libplist /build /
# Build
RUN git clone https://github.com/libimobiledevice/libtatsu && cd libtatsu \
&& ./autogen.sh && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf libtatsu
# libusbmuxd
# ----------
FROM build-base AS build-libusbmuxd
# Install dependencies
COPY --from=build-libplist /build /
COPY --from=build-libimobiledevice-glue /build /
# Build
RUN git clone https://github.com/libimobiledevice/libusbmuxd && cd libusbmuxd \
&& ./autogen.sh && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf libusbmuxd
# libimobiledevice
# ----------------
FROM build-base AS build-libimobiledevice
# Install dependencies
COPY --from=build-libplist /build /
COPY --from=build-libtatsu /build /
COPY --from=build-libimobiledevice-glue /build /
COPY --from=build-libusbmuxd /build /
# Build
RUN git clone https://github.com/libimobiledevice/libimobiledevice && cd libimobiledevice \
&& ./autogen.sh --enable-debug && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf libimobiledevice
# usbmuxd
# -------
FROM build-base AS build-usbmuxd
# Install dependencies
COPY --from=build-libplist /build /
COPY --from=build-libimobiledevice-glue /build /
COPY --from=build-libusbmuxd /build /
COPY --from=build-libimobiledevice /build /
# Build
RUN git clone https://github.com/libimobiledevice/usbmuxd && cd usbmuxd \
&& ./autogen.sh --sysconfdir=/etc --localstatedir=/var --runstatedir=/run && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf usbmuxd && mv /build/lib /build/usr/lib
# Create main image
FROM ubuntu:24.04 AS main
COPY --from=ghcr.io/astral-sh/uv:0.11.8 /uv /uvx /usr/local/bin/
@@ -112,7 +8,7 @@ LABEL org.opencontainers.image.source="https://github.com/mvt-project/mvt"
LABEL org.opencontainers.image.title="Mobile Verification Toolkit"
LABEL org.opencontainers.image.description="MVT is a forensic tool to look for signs of infection in smartphone devices."
LABEL org.opencontainers.image.licenses="MVT License 1.1"
LABEL org.opencontainers.image.base.name=docker.io/library/ubuntu:22.04
LABEL org.opencontainers.image.base.name=docker.io/library/ubuntu:24.04
# Install runtime dependencies
ARG DEBIAN_FRONTEND=noninteractive
@@ -122,9 +18,9 @@ RUN apt-get update \
binutils \
default-jre-headless \
file \
git \
jq \
less \
libcurl4 \
libimage-exiftool-perl \
libssl3 \
libusb-1.0-0 \
@@ -135,18 +31,20 @@ RUN apt-get update \
sqlite3 \
tree \
unzip \
xxd
COPY --from=build-libplist /build /
COPY --from=build-libimobiledevice-glue /build /
COPY --from=build-libtatsu /build /
COPY --from=build-libusbmuxd /build /
COPY --from=build-libimobiledevice /build /
COPY --from=build-usbmuxd /build /
xxd \
&& rm -rf /var/lib/apt/lists/*
# Install mvt using the locally checked out source
# Install MVT and pymobiledevice3. Native build dependencies are needed on ARM64.
ARG PYMOBILEDEVICE3_VERSION=11.12.4
COPY . mvt/
RUN uv pip install --system --break-system-packages --no-cache ./mvt \
&& rm -rf mvt
RUN apt-get update \
&& apt-get install -y --no-install-recommends build-essential python3-dev libssl-dev \
&& uv pip install --system --break-system-packages --no-cache ./mvt "pymobiledevice3==${PYMOBILEDEVICE3_VERSION}" \
&& apt-get purge -y --auto-remove build-essential python3-dev libssl-dev \
&& rm -rf mvt /var/lib/apt/lists/* \
&& pymobiledevice3 --help > /dev/null \
&& pymobiledevice3 backup2 --help > /dev/null \
&& mvt-ios --help > /dev/null
# Installing ABE
ADD --checksum=sha256:a20e07f8b2ea47620aff0267f230c3f1f495f097081fd709eec51cf2a2e11632 \
+22 -122
View File
@@ -1,110 +1,6 @@
# Base image for building libraries
# ---------------------------------
FROM ubuntu:22.04 AS build-base
ARG DEBIAN_FRONTEND=noninteractive
# Install build tools and dependencies
RUN apt-get update \
&& apt-get install -y \
build-essential \
git \
autoconf \
automake \
libtool-bin \
pkg-config \
libcurl4-openssl-dev \
libusb-1.0-0-dev \
libssl-dev \
udev \
&& rm -rf /var/lib/apt/lists/*
# libplist
# --------
FROM build-base AS build-libplist
# Build
RUN git clone https://github.com/libimobiledevice/libplist && cd libplist \
&& ./autogen.sh && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf libplist
# libimobiledevice-glue
# ---------------------
FROM build-base AS build-libimobiledevice-glue
# Install dependencies
COPY --from=build-libplist /build /
# Build
RUN git clone https://github.com/libimobiledevice/libimobiledevice-glue && cd libimobiledevice-glue \
&& ./autogen.sh && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf libimobiledevice-glue
# libtatsu
# --------
FROM build-base AS build-libtatsu
# Install dependencies
COPY --from=build-libplist /build /
# Build
RUN git clone https://github.com/libimobiledevice/libtatsu && cd libtatsu \
&& ./autogen.sh && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf libtatsu
# libusbmuxd
# ----------
FROM build-base AS build-libusbmuxd
# Install dependencies
COPY --from=build-libplist /build /
COPY --from=build-libimobiledevice-glue /build /
# Build
RUN git clone https://github.com/libimobiledevice/libusbmuxd && cd libusbmuxd \
&& ./autogen.sh && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf libusbmuxd
# libimobiledevice
# ----------------
FROM build-base AS build-libimobiledevice
# Install dependencies
COPY --from=build-libplist /build /
COPY --from=build-libtatsu /build /
COPY --from=build-libimobiledevice-glue /build /
COPY --from=build-libusbmuxd /build /
# Build
RUN git clone https://github.com/libimobiledevice/libimobiledevice && cd libimobiledevice \
&& ./autogen.sh --enable-debug && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf libimobiledevice
# usbmuxd
# -------
FROM build-base AS build-usbmuxd
# Install dependencies
COPY --from=build-libplist /build /
COPY --from=build-libimobiledevice-glue /build /
COPY --from=build-libusbmuxd /build /
COPY --from=build-libimobiledevice /build /
# Build
RUN git clone https://github.com/libimobiledevice/usbmuxd && cd usbmuxd \
&& ./autogen.sh --sysconfdir=/etc --localstatedir=/var --runstatedir=/run && make -j "$(nproc)" && make install DESTDIR=/build \
&& cd .. && rm -rf usbmuxd && mv /build/lib /build/usr/lib
# Main image
# ----------
FROM python:3.10.14-alpine3.20 AS main
FROM ubuntu:24.04 AS main
COPY --from=ghcr.io/astral-sh/uv:0.11.8 /uv /uvx /usr/local/bin/
LABEL org.opencontainers.image.url="https://mvt.re"
@@ -113,26 +9,30 @@ LABEL org.opencontainers.image.source="https://github.com/mvt-project/mvt"
LABEL org.opencontainers.image.title="Mobile Verification Toolkit (iOS)"
LABEL org.opencontainers.image.description="MVT is a forensic tool to look for signs of infection in smartphone devices."
LABEL org.opencontainers.image.licenses="MVT License 1.1"
LABEL org.opencontainers.image.base.name=docker.io/library/python:3.10.14-alpine3.20
LABEL org.opencontainers.image.base.name=docker.io/library/ubuntu:24.04
# Install runtime dependencies
RUN apk add --no-cache \
gcompat \
libcurl \
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates \
git \
libssl3 \
libusb \
sqlite
COPY --from=build-libplist /build /
COPY --from=build-libimobiledevice-glue /build /
COPY --from=build-libtatsu /build /
COPY --from=build-libusbmuxd /build /
COPY --from=build-libimobiledevice /build /
COPY --from=build-usbmuxd /build /
libusb-1.0-0 \
python3 \
sqlite3 \
&& rm -rf /var/lib/apt/lists/*
# Install mvt using the locally checked out source
COPY ./ mvt
RUN apk add --no-cache --virtual .build-deps git gcc musl-dev \
&& uv pip install --system --no-cache ./mvt \
&& apk del .build-deps git gcc musl-dev && rm -rf ./mvt
# Install MVT and pymobiledevice3. Native build dependencies are needed on ARM64.
ARG PYMOBILEDEVICE3_VERSION=11.12.4
COPY . mvt/
RUN apt-get update \
&& apt-get install -y --no-install-recommends build-essential python3-dev libssl-dev \
&& uv pip install --system --break-system-packages --no-cache ./mvt "pymobiledevice3==${PYMOBILEDEVICE3_VERSION}" \
&& apt-get purge -y --auto-remove build-essential python3-dev libssl-dev \
&& rm -rf mvt /var/lib/apt/lists/* \
&& pymobiledevice3 --help > /dev/null \
&& pymobiledevice3 backup2 --help > /dev/null \
&& mvt-ios --help > /dev/null
ENTRYPOINT [ "/usr/local/bin/mvt-ios" ]
+4
View File
@@ -0,0 +1,4 @@
prune tests
prune docs
prune .github
exclude uv.lock Dockerfile* mkdocs.yml .readthedocs.yaml .safety-policy.yml .python-version .gitignore Makefile
-6
View File
@@ -31,9 +31,3 @@ clean:
dist:
$(UV) build
upload:
$(UV) tool run twine upload dist/*
test-upload:
$(UV) tool run twine upload --repository testpypi dist/*
+21
View File
@@ -62,6 +62,27 @@ AndroidQF will prompt the user to download, decrypt and collect device intrusion
For cases where intrusion logs were collected outside of an AndroidQF acquisition, the standalone `mvt-android check-intrusion-logs` command can analyse them directly. See [Check Android Intrusion Logs](intrusion_logs.md) for details, and the [feature announcment from Amnesty International's Security Lab](https://securitylab.amnesty.org/latest/2026/05/android-intrusion-logging-as-a-new-source-of-data-for-consensual-forensic-analysis/) for background on the data source.
## Browser history
AndroidQF can optionally collect browser `History` databases when a device
already has working root access. When its acquisition contains the
`browser_history/manifest.json` file, `mvt-android check-androidqf` parses every
listed database, including any collected SQLite WAL and SHM sidecars. Browser
visits are added to the module results, timeline, URL output, and indicator
checks.
MVT can also inspect the supported database locations in a full Android
filesystem dump, or inspect one explicitly supplied Chromium `History` file:
```bash
mvt-android check-fs --output /path/to/results /path/to/filesystem-dump
mvt-android check-fs --output /path/to/results /path/to/History
```
The built-in locations cover Chrome, Brave, Microsoft Edge, and Samsung
Internet. Other Chromium-compatible databases can be supplied explicitly as a
file without assigning an unverified browser identity.
## Android Debug Bridge analysis removed
The ability to analyze Android devices directly over ADB has been removed from MVT. Direct extraction of data from ADB was error-prone and frequently resulted in inconsistent data collection between ADB and AndroidQF acquisitions. Use AndroidQF for device acquisition and `mvt-android check-androidqf` for analysis.
+24 -3
View File
@@ -1,4 +1,6 @@
Using Docker simplifies having all the required dependencies and tools (including most recent versions of [libimobiledevice](https://libimobiledevice.org)) readily installed. Note that this requires a Linux host, as Docker for Windows and Mac [doesn't support passing through USB devices](https://docs.docker.com/desktop/faqs/#can-i-pass-through-a-usb-device-to-a-container).
Using Docker simplifies running MVT with its dependencies readily installed. Note that this requires a Linux host, as Docker for Windows and Mac [doesn't support passing through USB devices](https://docs.docker.com/desktop/faqs/#can-i-pass-through-a-usb-device-to-a-container).
The main and iOS Docker images include pymobiledevice3 for [creating iOS backups](ios/backup/pymobiledevice3.md). You do not need to install it separately on the host. Older image tags may still include libimobiledevice instead; build from the updated source if pymobiledevice3 is unavailable.
Install Docker following the [official documentation](https://docs.docker.com/get-docker/).
@@ -37,12 +39,31 @@ If a prompt is spawned successfully, you can close it with `exit`.
On the Linux host, install and start [usbmuxd](https://github.com/libimobiledevice/usbmuxd), then connect and unlock the iOS device. The daemon exposes the device through the `/var/run/usbmuxd` socket.
Bind that socket into the container to let MVT communicate with the device:
Bind that socket into the container to let pymobiledevice3 communicate with the device. Also mount a local directory so acquired backups persist after the container exits:
```bash
docker run -it \
mkdir -p "$PWD/cases"
docker run --rm -it \
--mount type=bind,source=/var/run/usbmuxd,target=/var/run/usbmuxd \
--mount type=bind,source="$PWD/cases",target=/home/cases \
ghcr.io/mvt-project/mvt
```
Inside the container, verify connectivity:
```bash
pymobiledevice3 lockdown info
```
Accept the trust prompt on the unlocked device if requested. Then follow the [backup instructions](ios/backup/pymobiledevice3.md), using a destination under `/home/cases`, such as `/home/cases/backup`.
The iOS-only image defaults to running `mvt-ios`. To use pymobiledevice3 instead, override its entrypoint:
```bash
docker run --rm -it \
--mount type=bind,source=/var/run/usbmuxd,target=/var/run/usbmuxd \
--entrypoint pymobiledevice3 \
ghcr.io/mvt-project/mvt:latest-ios lockdown info
```
If you built the image from source, replace `ghcr.io/mvt-project/mvt` with `mvt`.
+1 -1
View File
@@ -1,6 +1,6 @@
# Installation
Before proceeding, please note that MVT requires Python 3.6+ to run. While it should be available on most operating systems, please make sure of that before proceeding.
Before proceeding, please note that MVT requires Python 3.10 or newer to run. While it should be available on most operating systems, please make sure of that before proceeding.
## Dependencies on Linux
+1 -17
View File
@@ -1,19 +1,3 @@
# Backup with libimobiledevice
If you have correctly [installed libimobiledevice](../install.md) you can easily generate an iTunes backup using the `idevicebackup2` tool included in the suite. First, you might want to ensure that backup encryption is enabled (**note: encrypted backup contain more data than unencrypted backups**):
```bash
idevicebackup2 -i encryption on
```
Note that if a backup password was previously set on this device, you might need to use the same or change it. You can try changing password using `idevicebackup2 -i changepw`, or by turning off encryption (`idevicebackup2 -i encryption off`) and turning it back on again.
If you are not able to recover or change the password, you should try to disable encryption and obtain an unencrypted backup.
If all else fails, as a *last resort* you can try resetting the password by [resetting all the settings through the iPhone's Settings app](https://support.apple.com/en-us/HT205220), via `Settings » General » Reset » Reset All Settings`. Note that resetting the settings through the iPhone's Settings app will wipe some of the files that contain useful forensic traces, so try the options explained above first.
Once ready, you can proceed performing the backup:
```bash
idevicebackup2 backup --full /path/to/backup/
```
We now recommend pymobiledevice3 for iOS backups. See [Backup with pymobiledevice3](pymobiledevice3.md) for the current instructions.
+37
View File
@@ -0,0 +1,37 @@
# Backup with pymobiledevice3
After [installing pymobiledevice3](../install.md) and pairing your device, you can generate an iTunes-compatible backup using its `backup2` commands.
## Enable backup encryption
We recommend encrypted backups because they contain more data than unencrypted backups. If encryption is not already enabled, enable it with:
```bash
pymobiledevice3 backup2 encryption on 'YOUR_BACKUP_PASSWORD'
```
Replace `YOUR_BACKUP_PASSWORD` with a strong password and keep it safe: you will need it to decrypt the backup for analysis. Passwords supplied on the command line may be saved in shell history or visible to other processes.
If a backup password was previously set, use that password. To change a known password:
```bash
pymobiledevice3 backup2 change-password 'CURRENT_PASSWORD' 'NEW_PASSWORD'
```
You can also disable encryption with `pymobiledevice3 backup2 encryption off 'CURRENT_PASSWORD'` and then enable it again. Disabling encryption requires the current password; it is not a way to bypass an unknown password.
!!! warning
If you cannot recover the password, resetting it through [Reset All Settings in the iPhone's Settings app](https://support.apple.com/en-us/HT205220) should be a last resort. Resetting settings can remove files containing useful forensic traces.
## Create a backup
Choose a new destination directory to avoid overwriting an earlier acquisition:
```bash
mkdir -p /path/to/backup/
pymobiledevice3 backup2 backup --full /path/to/backup/
```
The backup is saved in a subdirectory named after the device's UDID, such as `/path/to/backup/udid/`. Follow [Check a Backup with mvt-ios](check.md) to decrypt and analyze that directory.
For additional options, see the [pymobiledevice3 backup2 reference](https://doronz88.github.io/pymobiledevice3/cli/backup2/).
+20 -31
View File
@@ -1,55 +1,44 @@
# Install libimobiledevice
# Install pymobiledevice3
Before proceeding with doing any acquisition of iOS devices we recommend installing [libimobiledevice](https://libimobiledevice.org/) utilities. These utilities will become useful when extracting crash logs and generating iTunes backups. Because the utilities and its libraries are subject to frequent changes in response to new versions of iOS, you might want to consider compiling libimobiledevice utilities from sources. Otherwise, if available, you can try installing packages available in your distribution:
Before acquiring data from an iOS device, we recommend installing [pymobiledevice3](https://github.com/doronz88/pymobiledevice3). It provides command-line tools for generating iTunes-compatible backups and extracting crash logs.
## Installation
Install `pipx` following the [MVT installation instructions](../install.md), then run:
```bash
sudo apt install libimobiledevice-utils
pipx install pymobiledevice3
```
On Mac, you can try installing it from brew:
To update an existing installation:
```bash
brew install --HEAD libimobiledevice
pipx upgrade pymobiledevice3
```
If you have a reasonably recent version of libimobiledevice in your package manager, it might work straight out of the box. Try connecting your iOS device to your computer via USB and run:
On Linux, you also need `usbmuxd` to communicate with devices over USB. On Debian and Ubuntu, install it with:
```bash
ideviceinfo
sudo apt install usbmuxd
```
If you encounter unexpected issues, uninstall the packages and try compiling libimobiledevcice from sources.
macOS includes the required USB device service. For other platforms and additional requirements, see the [pymobiledevice3 installation documentation](https://doronz88.github.io/pymobiledevice3/installation/).
## Compile libimobiledevice from sources
## Verify connectivity
!!! warning
The following instructions are a best effort. The installation from source requires several steps, and it is likely some have been forgotten here and that won't work for you. You will likely need to fiddle around a bit before getting this right.
Make sure you have uninstalled all the libimobiledevice tools from your package manage:
Connect the iOS device to your computer with a USB cable and unlock it. Accept the **Trust This Computer** prompt and enter the device passcode if requested. Then run:
```bash
sudo apt remove --purge libimobiledevice-utils libimobiledevice-dev libimobiledevice6 libplist-dev libplist3 libusbmuxd-dev libusbmuxd-tools libusbmuxd4 libusbmuxd6 usbmuxd
pymobiledevice3 usbmux list
pymobiledevice3 lockdown info
```
Firstly you need to install [libplist](https://github.com/libimobiledevice/libplist). Then you can install [libusbmuxd](https://github.com/libimobiledevice/libusbmuxd).
Now you should be able to to download and install the actual suite of tools at [https://github.com/libimobiledevice/libimobiledevice](https://github.com/libimobiledevice/libimobiledevice).
You can now also build and install [usbmuxd](https://github.com/libimobiledevice/usbmuxd).
## Making sure everything works fine.
Once the idevice tools are available you can check if everything works fine by connecting your iOS device and running:
These commands should list the connected device and display its details. If pairing is needed, run:
```bash
ideviceinfo
pymobiledevice3 lockdown pair
```
This should show many details on the connected iOS device. If you are connecting the device to your laptop for the first time, it will require to unlock and enter the PIN code on the mobile device. If it complains that no device is connected and the mobile device is indeed plugged in through the USB cable, you might need to do this first, although typically the pairing is automatically done when connecting the device:
If no device is found, check the USB connection, make sure the device is unlocked, and, on Linux, check that `usbmuxd` is running. See the [upstream troubleshooting guide](https://doronz88.github.io/pymobiledevice3/guides/troubleshooting/) for further help.
```bash
sudo usbmuxd -f -v
idevicepair pair
```
Again, it will ask to unlock the phone and enter the PIN code.
Once connected, follow the [backup instructions](backup/pymobiledevice3.md).
+12
View File
@@ -204,6 +204,10 @@ This JSON file is created by mvt-ios' `Manifest` module. The module extracts rec
If indicators are provided through the command-line, they are checked against the original relative path in case. In some cases, there might be records of files created containing a domain name in their name, for example in the case of browser cache folders. Any matches are stored in *manifest_detected.json*.
An incomplete backup still lists in its manifest the files it failed to acquire. Those records carry a `"missing": true` field, and the module reports how many of them it found. Use it to tell a module which returned nothing because the artifact was not acquired apart from one which found nothing on a device that never had it.
Files must be stored at their expected paths inside the backup folder. If the module cannot finish listing the backup files, it logs a warning and skips the missing-file check; the manifest metadata is still extracted.
---
### `os_analytics_ad_daily.json`
@@ -435,3 +439,11 @@ This JSON file is created by mvt-ios' `WhatsappContacts` module. The module extr
This database is often missing from incremental backups. When it cannot be found, the module logs a warning and produces no results, in which case the disappearing messages state of chats cannot be determined from the backup.
---
## Records extracted by `check-sysdiagnose`
### `sysdiagnose_info.json`
This JSON file is created by mvt-ios' `SysdiagnoseInfo` module. The module extracts details about the device and the sysdiagnose itself: the product type and model, iOS version and build, serial number, IMEI, MEID and UDID from the remotectl dump state and the mobile activation request, the Apple account name and email from the App Store daemon database (no longer part of a sysdiagnose on newer iOS versions, still read from older archives), and the original file name and creation time of the archive from *sysdiagnose.log*.
+7 -4
View File
@@ -1,10 +1,13 @@
# Check an iOS Sysdiagnose
`mvt-ios check-sysdiagnose` prepares an iOS sysdiagnose archive for analysis by
custom MVT modules. MVT does not include built-in sysdiagnose modules. The
command runs the modules of the installed
`mvt-ios check-sysdiagnose` analyzes an iOS sysdiagnose archive. MVT's own
`SysdiagnoseInfo` module extracts details about the device and the archive
(see [`sysdiagnose_info.json`](records.md#sysdiagnose_infojson)); the checks
come from the modules of the installed
[plugin packages](../development/index.md#installed-module-packages) which
declare support for it. Install at least one such package first.
declare support for the command. Without any such module the command still
records the device details, and warns that no forensic sysdiagnose modules
have been loaded so that the run cannot pass for a clean analysis.
The command accepts either an extracted sysdiagnose directory or the original
gzip-compressed tar archive.
+3 -2
View File
@@ -5,6 +5,7 @@ copyright: Copyright &copy; 2021-2023 MVT Project Developers
site_description: Mobile Verification Toolkit Documentation
not_in_nav: |
android/adb.md
ios/backup/libimobiledevice.md
markdown_extensions:
- attr_list
- admonition
@@ -34,10 +35,10 @@ nav:
- Using Docker: "docker.md"
- MVT for iOS:
- iOS Forensic Methodology: "ios/methodology.md"
- Install libimobiledevice: "ios/install.md"
- Install pymobiledevice3: "ios/install.md"
- Check an iTunes Backup:
- Backup with iTunes app: "ios/backup/itunes.md"
- Backup with libimobiledevice: "ios/backup/libimobiledevice.md"
- Backup with pymobiledevice3: "ios/backup/pymobiledevice3.md"
- Check a Backup with mvt-ios: "ios/backup/check.md"
- Check a Filesystem Dump:
- Dumping the filesystem: "ios/filesystem/dump.md"
+6 -6
View File
@@ -24,9 +24,8 @@ dependencies = [
"simplejson==4.1.1",
"packaging==26.3",
"appdirs==1.4.4",
"iOSbackup==0.9.925",
"adb-shell[usb]==0.4.4",
"libusb1==3.4.0",
"iphone_backup_decrypt==0.10.0",
"pycryptodome>=3.20.0",
"cryptography==50.0.0",
"PyYAML>=6.0.2",
"pyahocorasick==2.3.1",
@@ -72,7 +71,7 @@ docs = [
]
[build-system]
requires = ["setuptools>=61.0"]
requires = ["setuptools>=61.0", "setuptools-scm>=8"]
build-backend = "setuptools.build_meta"
[tool.coverage.run]
@@ -121,5 +120,6 @@ where = ["src"]
[tool.setuptools.package-data]
mvt = ["ios/data/*.json"]
[tool.setuptools.dynamic]
version = { attr = "mvt.common.version.MVT_VERSION" }
[tool.setuptools_scm]
# The version is the latest v* tag; ignore the archive/* tags.
git_describe_command = "git describe --dirty --tags --long --match 'v*'"
@@ -0,0 +1,162 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import sqlite3
import shutil
import tempfile
from pathlib import Path
from typing import Any
from mvt.common.module_types import ModuleAtomicResult, ModuleSerializedResult
from mvt.common.utils import convert_chrometime_to_datetime, convert_datetime_to_iso
from .artifact import AndroidArtifact
class BrowserHistoryArtifact(AndroidArtifact):
"""Shared Chromium History database parsing and result handling."""
def _parse_browser_history(
self,
connection: sqlite3.Connection,
*,
browser: str,
package: str,
profile: str,
source_path: str,
) -> None:
cursor = connection.cursor()
try:
cursor.execute(
"""
SELECT
urls.id,
urls.url,
urls.title,
urls.visit_count,
urls.typed_count,
visits.id,
visits.visit_time,
visits.from_visit,
visits.transition
FROM urls
JOIN visits ON visits.url = urls.id
ORDER BY visits.visit_time;
"""
)
for row in cursor:
timestamp = int(row[6])
self.results.append(
{
"id": row[0],
"url": row[1],
"title": row[2],
"visit_count": row[3],
"typed_count": row[4],
"visit_id": row[5],
"timestamp": timestamp,
"isodate": convert_datetime_to_iso(
convert_chrometime_to_datetime(timestamp)
),
"redirect_source": row[7],
"transition": row[8],
"browser": browser,
"package": package,
"profile": profile,
"source_path": source_path,
}
)
finally:
cursor.close()
def serialize(self, record: ModuleAtomicResult) -> ModuleSerializedResult:
return {
"timestamp": record["isodate"],
"module": self.__class__.__name__,
"event": "browser_history",
"data": (
f"{record['browser']} visit to {record['url']} "
f"(visit ID: {record['visit_id']}, profile: {record['profile']})"
),
}
def check_indicators(self) -> None:
if not self.indicators:
return
for result, match in zip(
self.results,
self.indicators.check_url_batches(
[[result["url"]] for result in self.results]
),
):
if match:
self.alertstore.critical(
match.message, "", result, matched_indicator=match.ioc
)
def collect_url_results(self) -> None:
for result in self.results:
self.add_url_result(result["url"], result.get("isodate"), "browser_history")
class TemporarySQLiteConnection(sqlite3.Connection):
temporary_directory: tempfile.TemporaryDirectory | None = None
def close(self) -> None:
try:
super().close()
finally:
if self.temporary_directory:
self.temporary_directory.cleanup()
self.temporary_directory = None
def open_browser_history_database(database_path: Path) -> sqlite3.Connection:
"""Open a staged History database without modifying forensic evidence."""
database_uri = database_path.resolve().as_uri()
if not Path(f"{database_path}-wal").is_file():
return sqlite3.connect(f"{database_uri}?mode=ro&immutable=1", uri=True)
temporary_directory = tempfile.TemporaryDirectory(prefix="mvt_sqlite_")
temporary_path = Path(temporary_directory.name) / database_path.name
shutil.copy2(database_path, temporary_path)
for suffix in ("-wal", "-shm"):
sidecar = Path(f"{database_path}{suffix}")
if sidecar.is_file():
shutil.copy2(sidecar, Path(f"{temporary_path}{suffix}"))
try:
connection = sqlite3.connect(
f"{temporary_path.resolve().as_uri()}?mode=ro",
uri=True,
factory=TemporarySQLiteConnection,
)
except Exception:
temporary_directory.cleanup()
raise
connection.temporary_directory = temporary_directory
return connection
def validate_manifest_database(database: Any) -> dict[str, Any]:
if not isinstance(database, dict):
raise ValueError("database entry is not an object")
required = ("browser", "package", "profile", "device_path", "archive_path")
for field in required:
if not isinstance(database.get(field), str) or not database[field]:
raise ValueError(f"database entry has invalid {field}")
archive_path = database["archive_path"]
path = Path(archive_path)
if (
"\\" in archive_path
or path.is_absolute()
or ".." in path.parts
or path.parts[:1] != ("browser_history",)
):
raise ValueError(f"unsafe browser history archive path: {archive_path}")
return database
@@ -10,8 +10,45 @@ from .artifact import AndroidArtifact
class DumpsysAccessibilityArtifact(AndroidArtifact):
# One list for both record shapes — a service record and a count-only
# record must stay the same shape.
_FIELDS = (
"user_id",
"component",
"package_name",
"service_name",
"installed",
"enabled",
"binding",
"bound",
"crashed",
"accessibility_tool",
"installed_service_count",
"unnamed_service_count",
)
def check_indicators(self) -> None:
for result in self.results:
# A stated count the dump does not back with component names is a
# coverage statement, not a service: low, but not silent.
if not result.get("component"):
stated = result["installed_service_count"]
unnamed = result["unnamed_service_count"]
if unnamed == stated:
detail = "does not list their component names"
else:
detail = (
f"lists the component names of only {stated - unnamed} "
f"of them ({unnamed} unnamed)"
)
self.alertstore.low(
f"The accessibility dump states {stated} installed "
f"service(s) for user {result['user_id']} but {detail}",
"",
result,
)
continue
if self.indicators:
ioc_match = self.indicators.check_app_id(result["package_name"])
if ioc_match:
@@ -20,11 +57,22 @@ class DumpsysAccessibilityArtifact(AndroidArtifact):
)
continue
self.alertstore.medium(
f'Found accessibility service: "{result["component"]}"',
"",
result,
# Installed is not enabled. A service can sit installed for years
# without ever being switched on, and that is the difference this
# channel is read for — an alert that says only "found" makes every
# device look equally exposed. A service the dump says is switched
# OFF is reported LOW, so it still reaches the analyst without
# competing with one that is actually running; a dump that does not
# state the enabled state stays MEDIUM, because "not stated" is not
# "not enabled".
message = (
f'Found accessibility service: "{result["component"]}" '
f"({self._describe_state(result)})"
)
if result.get("enabled") is False and not result.get("bound"):
self.alertstore.low(message, "", result)
else:
self.alertstore.medium(message, "", result)
def parse(self, content: str) -> None:
"""
@@ -36,6 +84,13 @@ class DumpsysAccessibilityArtifact(AndroidArtifact):
self.results: list[dict[str, Any]] = []
services: dict[tuple[int | None, str], dict] = {}
# Which state sections the dump printed, per user: one user's printed
# `enabled services` says nothing about another user's.
seen_states: dict[int | None, set[str]] = {}
# `installedServiceCount=N` from the user's `attributes:{…}` line is on
# most builds the only statement about installed services in the dump:
# few print the `installed services: {…}` block.
installed_counts: dict[int | None, int] = {}
user_id: int | None = None
state: str | None = None
@@ -44,6 +99,10 @@ class DumpsysAccessibilityArtifact(AndroidArtifact):
if user_match:
user_id = int(user_match.group(1))
count_match = re.search(r"installedServiceCount=(\d+)", line)
if count_match:
installed_counts[user_id] = int(count_match.group(1))
stripped = line.strip()
state_match = re.match(
r"(?i)(installed|enabled|binding|bound|crashed) services\s*:\s*\{(.*)",
@@ -51,6 +110,7 @@ class DumpsysAccessibilityArtifact(AndroidArtifact):
)
if state_match:
state = state_match.group(1).lower()
seen_states.setdefault(user_id, set()).add(self._state_field(state))
inline = state_match.group(2)
for component in re.findall(
r"\{?([\w.$-]+/[\w.$-]+)(?:\s+\(A11yTool\))?\}?", inline
@@ -79,24 +139,65 @@ class DumpsysAccessibilityArtifact(AndroidArtifact):
service[self._state_field(state)] = True
service["accessibility_tool"] = "(A11yTool)" in stripped
# A section that was never printed is NOT the same as one printed
# empty: the first says nothing, the second says nothing is enabled.
# Defaulting every flag to False would turn "not stated" into "not
# enabled". Flags for sections this dump never printed stay None.
named: dict[int | None, int] = {}
for (service_user, _component), service in services.items():
printed = seen_states.get(service_user, set())
for state in ("installed", "enabled", "binding", "bound", "crashed"):
if self._state_field(state) not in printed:
service[self._state_field(state)] = None
service["installed_service_count"] = installed_counts.get(service_user)
named[service_user] = named.get(service_user, 0) + 1
self.results.extend(services.values())
# A stated count the named services do not add up to would leave no
# trace of the rest: the module would log "a total of 0" about a dump
# that said five, or "a total of 1" about one that said two.
for count_user, count in installed_counts.items():
unnamed = count - named.get(count_user, 0)
if unnamed > 0:
self.results.append(self._new_unlisted(count_user, count, unnamed))
@staticmethod
def _describe_state(result: dict) -> str:
if result.get("bound"):
return "enabled and bound"
if result.get("enabled"):
return "enabled"
if result.get("enabled") is None:
return "installed, enabled state not stated"
return "installed, not enabled"
@staticmethod
def _state_field(state: str) -> str:
return {"binding": "binding", "bound": "bound"}.get(state, state)
@staticmethod
def _new_unlisted(user_id: int | None, count: int, unnamed: int) -> dict:
"""The dump's own count for a user, and how many of those services it
did not name.
Every other field stays unknown: the dump named no service to carry it.
"""
record: dict[str, Any] = dict.fromkeys(DumpsysAccessibilityArtifact._FIELDS)
record["user_id"] = user_id
record["installed_service_count"] = count
record["unnamed_service_count"] = unnamed
return record
@staticmethod
def _new_service(component: str, user_id: int | None) -> dict:
package_name, service_name = component.split("/", 1)
return {
"user_id": user_id,
"component": component,
"package_name": package_name,
"service_name": service_name,
"installed": False,
"enabled": False,
"binding": False,
"bound": False,
"crashed": False,
"accessibility_tool": False,
}
record: dict[str, Any] = dict.fromkeys(
DumpsysAccessibilityArtifact._FIELDS, False
)
record["user_id"] = user_id
record["component"] = component
record["package_name"], record["service_name"] = component.split("/", 1)
# Filled in after parsing: the count is per user.
record["installed_service_count"] = None
record["unnamed_service_count"] = None
return record
+54 -6
View File
@@ -29,11 +29,26 @@ class DumpsysADBArtifact(AndroidArtifact):
stack = [res]
cur_indent = 0
in_multiline = False
for line in dump_data.strip(b"\n").split(b"\n"):
for line in dump_data.strip(b"\r\n").split(b"\n"):
line = line.removesuffix(b"\r")
# Track the level of indentation
indent = len(line) - len(line.lstrip())
if indent < cur_indent:
# A closing brace is dedented from the block it closes, and the
# brace handler below pops that block. Popping on the dedent as
# well would close two levels for one brace. A blank line has no
# indentation to compare: it ends a multiline value (handled
# below) and is otherwise skipped.
if indent < cur_indent and line.strip() not in (b"", b"}"):
# If the current line is less indented than the previous one, back out
if len(stack) <= 1:
# Dedenting below the outermost level means this is not the
# well-formed block the parser expects. Stop here rather than
# raise IndexError out of the module on the next line.
self.log.error(
"Unexpected indentation in ADB manager state, "
"stopping the parse of this section"
)
break
stack.pop()
cur_indent = indent
else:
@@ -62,7 +77,16 @@ class DumpsysADBArtifact(AndroidArtifact):
stack.pop()
current_dict = stack[-1]
if not line.strip():
continue
if key == "}":
if len(stack) <= 1:
self.log.error(
"Unbalanced closing brace in ADB manager state, "
"stopping the parse of this section"
)
break
stack.pop()
continue
@@ -161,6 +185,26 @@ class DumpsysADBArtifact(AndroidArtifact):
f"'{user_key['fingerprint']}'"
)
@staticmethod
def _find_state_end(content: bytes, open_brace: int) -> int:
"""Find the unindented line closing the ADB manager state, or -1.
Braces inside key comments or embedded keystore data are values, while
nested structural closing braces are indented.
"""
line_start = open_brace
while line_start < len(content):
line_end = content.find(b"\n", line_start)
if line_end == -1:
line_end = len(content)
line = content[line_start:line_end].removesuffix(b"\r")
if line_start > open_brace and line == b"}":
return line_start
if line.startswith((b"---------", b"DUMP OF SERVICE ")):
break
line_start = line_end + 1
return -1
def parse(self, content: bytes) -> None:
"""
Parse the Dumpsys ADB section
@@ -180,12 +224,16 @@ class DumpsysADBArtifact(AndroidArtifact):
self.log.error("Unable to find ADB manager state in dumpsys output")
return
end_of_json = content.rfind(b"}\n")
if end_of_json == -1 or end_of_json <= start_of_json:
self.log.error("Unable to find complete ADB manager state in dumpsys output")
end_of_json = self._find_state_end(content, start_of_json + 1)
if end_of_json == -1:
self.log.error(
"Unable to find complete ADB manager state in dumpsys output"
)
return
json_content = content[start_of_json + 2 : end_of_json - 2].rstrip()
# The brace that opens the state and the one that closes it are not part
# of the indented body.
json_content = content[start_of_json + 2 : end_of_json].rstrip()
parsed = self.indented_dump_parser(json_content)
if parsed.get("debugging_manager") is None:
@@ -73,8 +73,12 @@ class DumpsysBatteryDailyArtifact(AndroidArtifact):
continue
line = line.strip().replace("Update ", "")
package_name, vers = line.split(" ", 1)
vers_raw = vers.split("=", 1)[1]
# A truncated or vendor-specific line must not abort the parse and
# lose every record after it.
package_name, _, vers = line.partition(" ")
vers_raw = vers.partition("=")[2]
if not package_name or not vers_raw:
continue
try:
version_code: int | str = int(vers_raw)
except ValueError:
+13 -2
View File
@@ -136,6 +136,10 @@ class Mounts(AndroidArtifact):
mount_options = fields[5].split(",")
super_options = fields[separator + 3].split(",")
options = list(dict.fromkeys(mount_options + super_options))
# fields[5] are the per-mount (VFS) flags, the field after the
# separator the superblock's; a mount is writable only if both
# allow it. The merged list made every read-only overlay a HIGH.
is_read_write = "rw" in mount_options and "rw" in super_options
mount_point = fields[4].replace("\\040", " ")
device = fields[separator + 2].replace("\\040", " ")
filesystem_type = fields[separator + 1]
@@ -156,7 +160,7 @@ class Mounts(AndroidArtifact):
"options_list": options,
"optional_fields": fields[6:separator],
"is_system_partition": is_system,
"is_read_write": "rw" in options,
"is_read_write": is_read_write,
"process_ids": [process_id],
}
)
@@ -191,7 +195,14 @@ class Mounts(AndroidArtifact):
)
# Check for other suspicious mount options
suspicious_opts = [opt for opt in options if opt in SUSPICIOUS_OPTIONS]
# `rw` sitting in the merged option list is not evidence of
# writability (see is_read_write in parse_mountinfo); every other
# suspicious option is meaningful in either list.
suspicious_opts = [
opt
for opt in options
if opt in SUSPICIOUS_OPTIONS and (opt != "rw" or mount["is_read_write"])
]
if suspicious_opts and mount["is_system_partition"]:
if (
"noatime" in mount["mount_options"]
+242 -36
View File
@@ -4,6 +4,11 @@
# https://license.mvt.re/1.1/
import re
from datetime import datetime
from typing import Optional, Sequence
from mvt.common.module_types import ModuleAtomicResult, ModuleSerializedResult
from mvt.common.utils import convert_datetime_to_iso
from .artifact import AndroidArtifact
@@ -60,45 +65,246 @@ ANDROID_DANGEROUS_SETTINGS = [
},
]
# dumpsys prints the fields of a setting record, and of a change history entry,
# always in this order and separated by a single space. After the value come
# `default:` and `defaultSystemSet:` when a default is recorded, then `tag:`;
# some vendor builds add whether the value survives a restore, either as
# `isValuePreservedInRestore:` or as a bare `notPreservedInRestore` token.
SETTING_FIELDS = (
"_id",
"name",
"pkg",
"value",
"default",
"defaultSystemSet",
"tag",
"isValuePreservedInRestore",
)
HISTORY_FIELDS = ("time", "mode", "oldValue", "newValue", "package")
NAMESPACE_PATTERN = re.compile(
r"^(CONFIG|GLOBAL|SECURE|SYSTEM) SETTINGS \(user (\d+)\)$"
)
SECTION_END_PATTERN = re.compile(r"ending at: (\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})")
class Settings(AndroidArtifact):
def parse(self, content: str) -> None:
self.results: dict[str, dict[str, str]] = {}
namespace: str | None = None
for line in content.splitlines():
heading = re.match(
r"^(CONFIG|GLOBAL|SECURE|SYSTEM) SETTINGS \(user (\d+)\)$",
line.strip(),
)
if heading:
namespace = f"{heading.group(1).lower()}:user_{heading.group(2)}"
self.results[namespace] = {}
"""Parser for the `dumpsys settings` output.
Every row of the settings provider becomes one result, keeping the fields
dumpsys prints alongside the value: the row id, the package which recorded
the setting, the default, the tag, and the change history. A setting name
can appear more than once within a namespace, so results are a list rather
than a mapping.
"""
def serialize(self, result: ModuleAtomicResult) -> ModuleSerializedResult:
records = []
for entry in result.get("history", []):
if not entry.get("timestamp"):
continue
if namespace is None or not line.startswith("_id:"):
continue
setting = re.match(
r"^_id:\S+\s+name:(.*?)\s+pkg:.*?\s+value:(.*?)"
r"(?:\s+default:.*\s+defaultSystemSet:(?:true|false))?$",
line,
records.append(
{
"timestamp": entry["timestamp"],
"module": self.__class__.__name__,
"event": "settings_change",
"data": (
f"{result.get('namespace')} setting "
f'"{result.get("name")}" changed from '
f'"{entry.get("oldValue")}" to "{entry.get("newValue")}" '
f"by {entry.get('pkg')}"
),
}
)
if setting:
self.results[namespace][setting.group(1)] = setting.group(2)
return records
def check_indicators(self) -> None:
for namespace, settings in self.results.items():
for key, value in settings.items():
for danger in ANDROID_DANGEROUS_SETTINGS:
# Check if one of the dangerous settings is using an unsafe
# value (different than the one specified).
if danger["key"] == key and danger["safe_value"] != value:
self.alertstore.medium(
f'Found suspicious "{namespace}" setting "{key} = {value}" ({danger["description"]})',
"",
{
"namespace": namespace,
"key": key,
"value": value,
"description": danger["description"],
},
)
break
for result in self.results:
name = result.get("name")
value = result.get("value")
for danger in ANDROID_DANGEROUS_SETTINGS:
# Check if one of the dangerous settings is using an unsafe
# value (different than the one specified).
if danger["key"] != name or danger["safe_value"] == value:
continue
history = result.get("history") or []
self.alertstore.medium(
f'Found suspicious "{result.get("namespace")}" setting '
f'"{name} = {value}" ({danger["description"]})',
history[-1]["timestamp"] if history else "",
result,
)
break
def parse(self, content: str) -> None:
self.results: list[ModuleAtomicResult] = []
section_end = self._parse_section_end(content)
namespace: Optional[str] = None
user: Optional[str] = None
record_lines: list[str] = []
history_lines: list[str] = []
in_history = False
def flush() -> None:
nonlocal record_lines, history_lines, in_history
if record_lines:
self.results.append(
self._build_record(
namespace, user, record_lines, history_lines, section_end
)
)
record_lines = []
history_lines = []
in_history = False
for line in content.splitlines():
heading = NAMESPACE_PATTERN.match(line.strip())
if heading:
flush()
namespace = heading.group(1).lower()
user = heading.group(2)
continue
if line.startswith("--------- "):
# dumpsys closes every section with a duration trailer.
flush()
namespace = None
continue
if namespace is None:
continue
if not line.strip():
# dumpsys prints a blank line after every namespace block and
# after a change history, and other dumps such as the
# generation registry follow the last block, so a blank line
# closes the record being read.
flush()
continue
if line.startswith("_id:"):
flush()
record_lines = [line]
continue
if not record_lines:
continue
stripped = line.strip()
if stripped.startswith("History ("):
in_history = True
continue
if in_history:
if stripped.startswith("time:"):
history_lines.append(stripped)
elif stripped and history_lines:
# A history entry can be wrapped over several lines.
history_lines[-1] += " " + stripped
continue
# Anything else continues the value of the record being read.
record_lines.append(line)
flush()
@staticmethod
def _split_fields(text: str, keys: Sequence[str]) -> dict[str, str]:
"""Split the `key:value` fields of one record.
Values are free-form and may contain spaces and newlines, so a field
runs up to the start of the next key which is actually present. Keys
dumpsys did not print are skipped.
"""
fields: dict[str, str] = {}
key = keys[0]
if not text.startswith(f"{key}:"):
return fields
remainder = text[len(key) + 1 :]
for next_key in keys[1:]:
value, separator, rest = remainder.partition(f" {next_key}:")
if separator:
fields[key] = value
key, remainder = next_key, rest
fields[key] = remainder
return fields
@staticmethod
def _parse_section_end(content: str) -> Optional[datetime]:
"""Return the time the settings section was dumped, if reported."""
match = SECTION_END_PATTERN.search(content)
if not match:
return None
try:
return datetime.strptime(match.group(1), "%Y-%m-%d %H:%M:%S")
except ValueError:
return None
@staticmethod
def _resolve_timestamp(
value: str, section_end: Optional[datetime]
) -> Optional[str]:
"""Add the missing year to a `MM-DD HH:MM:SS.mmm` history timestamp.
dumpsys prints the change history without a year, so it is resolved
against the time the section was dumped: the most recent matching date
at or before that time.
The year is parsed together with the value, because strptime() would
otherwise parse it in 1900, which has no 29 February. The most recent
matching leap day can be eight years earlier when a century year is
not a leap year and the current year's leap day is still in the future.
"""
if section_end is None:
return None
for year in range(section_end.year, section_end.year - 9, -1):
try:
timestamp = datetime.strptime(f"{year}-{value}", "%Y-%m-%d %H:%M:%S.%f")
except ValueError:
continue
if timestamp <= section_end:
return convert_datetime_to_iso(timestamp)
return None
def _parse_history(
self, line: str, section_end: Optional[datetime]
) -> ModuleAtomicResult:
fields = self._split_fields(line, HISTORY_FIELDS)
return {
"timestamp": self._resolve_timestamp(fields.get("time", ""), section_end),
"oldValue": fields.get("oldValue"),
"newValue": fields.get("newValue"),
"pkg": fields.get("package"),
}
def _build_record(
self,
namespace: Optional[str],
user: Optional[str],
record_lines: list[str],
history_lines: list[str],
section_end: Optional[datetime],
) -> ModuleAtomicResult:
text = "\n".join(record_lines).rstrip()
# The bare `notPreservedInRestore` token has no `key:` shape and is
# printed last, so peel it off before splitting the fields.
head = text.removesuffix(" notPreservedInRestore")
record: ModuleAtomicResult = {"namespace": namespace, "user": user}
record.update(self._split_fields(head, SETTING_FIELDS))
if head != text:
record["isValuePreservedInRestore"] = "false"
record["history"] = [
self._parse_history(entry, section_end) for entry in history_lines
]
return record
+71 -8
View File
@@ -21,6 +21,7 @@ from mvt.common.help import (
HELP_MSG_CHECK_ANDROID_BACKUP,
HELP_MSG_CHECK_ANDROIDQF,
HELP_MSG_CHECK_BUGREPORT,
HELP_MSG_CHECK_FS,
HELP_MSG_CHECK_IOCS,
HELP_MSG_CHECK_INTRUSION_LOGS,
HELP_MSG_DELAY_CHECKS,
@@ -150,6 +151,16 @@ def check_adb(ctx):
default=[],
help=HELP_MSG_LOAD_MODULE,
)
@click.option(
"--timezone",
"-t",
default=None,
help=(
"IANA timezone name for the device, for example 'Europe/Paris'. "
"Bugreport file timestamps are the device's wall clock; by default the "
"zone is read from persist.sys.timezone in the bugreport itself."
),
)
@click.option("--verbose", "-v", is_flag=True, help=HELP_MSG_VERBOSE_COMMAND)
@click.argument("BUGREPORT_PATH", type=click.Path(exists=True))
@click.pass_context
@@ -160,6 +171,7 @@ def check_bugreport(
list_modules,
module,
load_module,
timezone,
verbose,
bugreport_path,
):
@@ -167,12 +179,18 @@ def check_bugreport(
set_verbose_logging(verbose or _get_verbose(ctx))
custom_modules = _load_custom_modules(load_module)
module_options = {}
if timezone:
module_options["device_timezone"] = timezone
# Always generate hashes as bug reports are small.
cmd = CmdAndroidCheckBugreport(
target_path=bugreport_path,
results_path=output,
ioc_files=iocs,
module_name=module,
module_options=module_options if module_options else None,
hashes=True,
disable_version_check=_get_disable_flags(ctx)[0],
disable_indicator_check=_get_disable_flags(ctx)[1],
@@ -183,8 +201,6 @@ def check_bugreport(
cmd.list_modules()
return
log.info("Checking Android bug report at path: %s", bugreport_path)
try:
cmd.run()
except BadZipFile as exc:
@@ -259,8 +275,6 @@ def check_backup(
cmd.list_modules()
return
log.info("Checking Android backup at path: %s", backup_path)
cmd.run()
cmd.show_alerts_brief()
cmd.show_support_message()
@@ -342,14 +356,65 @@ def check_androidqf(
cmd.list_modules()
return
log.info("Checking AndroidQF acquisition at path: %s", androidqf_path)
cmd.run()
cmd.show_alerts_brief()
cmd.show_disable_adb_warning()
cmd.show_support_message()
# ==============================================================================
# Command: check-fs
# ==============================================================================
@cli.command("check-fs", context_settings=CONTEXT_SETTINGS, help=HELP_MSG_CHECK_FS)
@click.option(
"--iocs",
"-i",
type=click.Path(exists=True),
multiple=True,
default=[],
help=HELP_MSG_IOC,
)
@click.option("--output", "-o", type=click.Path(exists=False), help=HELP_MSG_OUTPUT)
@click.option("--list-modules", "-l", is_flag=True, help=HELP_MSG_LIST_MODULES)
@click.option("--module", "-m", help=HELP_MSG_MODULE)
@click.option(
"--load-module",
type=click.Path(exists=True),
multiple=True,
default=[],
help=HELP_MSG_LOAD_MODULE,
)
@click.option("--hashes", "-H", is_flag=True, help=HELP_MSG_HASHES)
@click.option("--verbose", "-v", is_flag=True, help=HELP_MSG_VERBOSE)
@click.argument("DUMP_PATH", type=click.Path(exists=True))
@click.pass_context
def check_fs(
ctx, iocs, output, list_modules, module, load_module, hashes, verbose, dump_path
):
set_verbose_logging(verbose)
custom_modules = _load_custom_modules(load_module)
from .cmd_check_fs import CmdAndroidCheckFS
cmd = CmdAndroidCheckFS(
target_path=dump_path,
results_path=output,
ioc_files=iocs,
module_name=module,
hashes=hashes,
disable_version_check=_get_disable_flags(ctx)[0],
disable_indicator_check=_get_disable_flags(ctx)[1],
custom_modules=custom_modules,
)
if list_modules:
cmd.list_modules()
return
log.info("Checking Android filesystem located at: %s", dump_path)
cmd.run()
cmd.show_alerts_brief()
cmd.show_support_message()
# ==============================================================================
# Command: check-intrusion-logs
# ==============================================================================
@@ -424,8 +489,6 @@ def check_intrusion_logs(
cmd.list_modules()
return
log.info("Checking intrusion logs at path: %s", logs_path)
cmd.run()
cmd.show_alerts_brief()
cmd.show_support_message()
+2
View File
@@ -82,6 +82,8 @@ class CmdAndroidCheckAndroidQF(Command):
if not self.target_path:
raise NoAndroidQFTargetPath
self.log.info("Checking AndroidQF acquisition at path: %s", self.target_path)
if os.path.isdir(self.target_path):
self.__format = "dir"
parent_path = Path(self.target_path).absolute().parent.as_posix()
+1
View File
@@ -129,6 +129,7 @@ class CmdAndroidCheckBackup(Command):
assert self.target_path is not None # type: ignore[has-type]
# Use a different local variable name to avoid any scoping issues
backup_path: str = self.target_path # type: ignore[has-type]
self.log.info("Checking Android backup at path: %s", backup_path)
if os.path.isfile(backup_path):
self.__type = "ab"
+99 -7
View File
@@ -3,13 +3,19 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import fnmatch
import io
import logging
import os
from pathlib import Path
from typing import List, Optional
from zipfile import ZipFile
from zipfile import BadZipFile, ZipFile
from mvt.android.modules.bugreport.base import BugReportModule
from mvt.android.artifacts.getprop import GetProp
from mvt.android.modules.bugreport.base import (
DUMPSTATE_ENTRY_POINTS,
BugReportModule,
)
from mvt.common.command import Command
from mvt.common.indicators import Indicators
from mvt.common.module import MVTModule
@@ -87,14 +93,100 @@ class CmdAndroidCheckBugreport(Command):
for file_name in self.__zip.namelist():
self.__files.append(file_name)
if not self._has_dumpstate(self.__files):
nested = self._nested_bugreport(bugreport_zip)
if nested:
self.__zip = nested
self.__files = list(nested.namelist())
@staticmethod
def _has_dumpstate(file_names: List[str]) -> bool:
"""Whether these members carry any of the entry points the bug report
modules read (see `BugReportModule._get_dumpstate_file`)."""
return any(
fnmatch.filter(file_names, pattern) for pattern in DUMPSTATE_ENTRY_POINTS
)
def _nested_bugreport(self, outer: ZipFile) -> Optional[ZipFile]:
"""Descend one level into an OEM wrapper archive.
MIUI / HyperOS hands out a zip of app logs, ANR traces and tcpdump
captures with the real `bugreport-<device>-<timestamp>.zip` nested
inside. Without this descent the outer archive has no entry point,
every module reports it found no files, and the command still exits 0
with an empty result.
"""
candidates = [
name for name in outer.namelist() if name.lower().endswith(".zip")
]
candidates.sort(
key=lambda name: (
"bugreport" not in name.lower() and "dumpstate" not in name.lower()
)
)
for name in candidates:
try:
inner = ZipFile(io.BytesIO(outer.read(name)))
except (BadZipFile, OSError):
continue
if self._has_dumpstate(inner.namelist()):
log.info("Found the bug report nested inside the archive: %s", name)
return inner
inner.close()
return None
def init(self) -> None:
if not self.target_path:
if self.target_path:
self.log.info("Checking Android bug report at path: %s", self.target_path)
if os.path.isfile(self.target_path):
self.from_zip(ZipFile(self.target_path))
elif os.path.isdir(self.target_path):
self.from_dir(self.target_path)
self.log.warning(
"Analysing an unpacked bugreport: file timestamps come from "
"the extraction, not from the device. Analyse the original "
"zip to keep the device's file timestamps."
)
if self.__format:
self._resolve_device_timezone()
def _resolve_device_timezone(self) -> None:
"""Name the device's timezone in module_options unless it is known already.
A bugreport's SYSTEM PROPERTIES section carries persist.sys.timezone.
Zip entry times are the device's wall clock, and modules read them in
this zone; --timezone or check-androidqf's own reading takes precedence.
"""
if self.module_options.get("device_timezone"):
self.log.info("Device timezone: %s", self.module_options["device_timezone"])
return
if os.path.isfile(self.target_path):
self.from_zip(ZipFile(self.target_path))
elif os.path.isdir(self.target_path):
self.from_dir(self.target_path)
probe = BugReportModule(log=self.log)
self.module_init(probe)
timezone = None
try:
dumpstate = probe._get_dumpstate_file()
except Exception as exc:
self.log.warning("Could not read the bugreport's dumpstate: %s", exc)
dumpstate = None
if dumpstate:
properties = GetProp()
properties.parse(
BugReportModule.extract_command_section(
dumpstate.decode("utf-8", errors="replace"),
"------ SYSTEM PROPERTIES",
)
)
timezone = properties.get_device_timezone()
if timezone:
self.log.info("Device timezone identified from the bugreport: %s", timezone)
self.module_options["device_timezone"] = timezone
else:
self.log.warning(
"persist.sys.timezone not found in the bugreport; file timestamps "
"are the device's wall clock without a timezone. Pass --timezone "
"to name it."
)
def module_init(self, module: BugReportModule) -> None: # type: ignore[override]
if self.__format == "zip":
+47
View File
@@ -0,0 +1,47 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from mvt.common.command import Command
from mvt.common.indicators import Indicators
from mvt.common.module import MVTModule
from .modules.fs import FS_MODULES
log = logging.getLogger(__name__)
class CmdAndroidCheckFS(Command):
def __init__(
self,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
ioc_files: Optional[list] = None,
iocs: Optional[Indicators] = None,
module_name: Optional[str] = None,
module_options: Optional[dict] = None,
hashes: bool = False,
disable_version_check: bool = False,
disable_indicator_check: bool = False,
custom_modules: Optional[list[type[MVTModule]]] = None,
) -> None:
super().__init__(
target_path=target_path,
results_path=results_path,
ioc_files=ioc_files,
iocs=iocs,
module_name=module_name,
module_options=module_options,
hashes=hashes,
log=log,
disable_version_check=disable_version_check,
disable_indicator_check=disable_indicator_check,
custom_modules=custom_modules,
)
self.platform = "android"
self.name = "check-fs"
self.modules = FS_MODULES
@@ -63,6 +63,8 @@ class CmdAndroidCheckIntrusionLogs(Command):
if not self.target_path:
raise ValueError("No target path specified")
self.log.info("Checking intrusion logs at path: %s", self.target_path)
if not os.path.isdir(self.target_path) and not (
os.path.isfile(self.target_path)
and self.target_path.lower().endswith(".zip")
@@ -11,6 +11,7 @@ from .aqf_log_timestamps import AQFLogTimestamps
from .aqf_packages import AQFPackages
from .aqf_processes import AQFProcesses
from .aqf_settings import AQFSettings
from .browser_history import BrowserHistory
from .mounts import Mounts
from .root_binaries import RootBinaries
@@ -23,4 +24,5 @@ ANDROIDQF_MODULES: list[type[MVTModule]] = [
AQFLogTimestamps,
RootBinaries,
Mounts,
BrowserHistory,
]
@@ -3,11 +3,7 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from mvt.android.artifacts.settings import Settings as SettingsArtifact
from mvt.common.module_types import ModuleResults
from .base import AndroidQFModule
@@ -15,43 +11,23 @@ from .base import AndroidQFModule
class AQFSettings(SettingsArtifact, AndroidQFModule):
"""This module analyse setting files"""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[ModuleResults] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
self.results: dict = results if results is not None else {}
def run(self) -> None:
for setting_file in self._get_files_by_pattern("*/settings_*.txt"):
namespace = setting_file[setting_file.rfind("_") + 1 : -4]
self.results[namespace] = {}
data = self._get_file_content(setting_file)
for line in data.decode("utf-8").splitlines():
line = line.strip()
try:
key, value = line.split("=", 1)
except ValueError:
name, separator, value = line.strip().partition("=")
if not separator:
continue
try:
self.results[namespace][key] = value
except IndexError:
continue
self.results.append(
{
"namespace": namespace,
"user": None,
"name": name,
"value": value,
}
)
self.log.info(
"Identified %d settings", sum([len(val) for val in self.results.values()])
)
self.log.info("Identified %d settings", len(self.results))
@@ -0,0 +1,115 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import json
import sqlite3
import tempfile
from pathlib import Path, PurePosixPath
from mvt.android.artifacts.browser_history import (
BrowserHistoryArtifact,
open_browser_history_database,
validate_manifest_database,
)
from .base import AndroidQFModule
class BrowserHistory(BrowserHistoryArtifact, AndroidQFModule):
"""Extract browser visits collected by AndroidQF."""
supported_commands = (("android", "check-androidqf"),)
def _find_manifest(self) -> str | None:
manifests = [
file_path
for file_path in self.files
if file_path.replace("\\", "/").endswith("browser_history/manifest.json")
]
if not manifests:
return None
if len(manifests) > 1:
self.log.warning(
"Found multiple browser history manifests; using %s", manifests[0]
)
return manifests[0]
def _stage_database(
self, archive_path: str, prefix: str, temporary_path: Path
) -> Path:
available_files = {
str(PurePosixPath(file_path.replace("\\", "/"))): file_path
for file_path in self.files
}
normalized_path = str(PurePosixPath(prefix, archive_path))
source_path = available_files.get(normalized_path)
if not source_path:
raise FileNotFoundError(archive_path)
staged_path = temporary_path / "History"
staged_path.write_bytes(self._get_file_content(source_path))
for suffix in ("-wal", "-shm"):
sidecar = available_files.get(normalized_path + suffix)
if sidecar:
Path(f"{staged_path}{suffix}").write_bytes(
self._get_file_content(sidecar)
)
return staged_path
def run(self) -> None:
manifest_path = self._find_manifest()
if not manifest_path:
self.log.info("No AndroidQF browser history manifest found")
return
try:
manifest = json.loads(self._get_file_content(manifest_path))
except (json.JSONDecodeError, OSError, TypeError, UnicodeDecodeError) as exc:
self.log.error("Unable to read browser history manifest: %s", exc)
return
if not isinstance(manifest, dict) or manifest.get("schema_version") != 1:
self.log.error("Unsupported AndroidQF browser history manifest")
return
databases = manifest.get("databases", [])
if not isinstance(databases, list):
self.log.error("Invalid AndroidQF browser history database list")
return
normalized_manifest = manifest_path.replace("\\", "/")
marker = "browser_history/manifest.json"
prefix = normalized_manifest[: -len(marker)].rstrip("/")
for raw_database in databases:
try:
database = validate_manifest_database(raw_database)
with tempfile.TemporaryDirectory(prefix="mvt_browser_history_") as temp:
staged_path = self._stage_database(
database["archive_path"], prefix, Path(temp)
)
connection = open_browser_history_database(staged_path)
try:
self._parse_browser_history(
connection,
browser=database["browser"],
package=database["package"],
profile=database["profile"],
source_path=database["device_path"],
)
finally:
connection.close()
except (
FileNotFoundError,
OSError,
OverflowError,
sqlite3.Error,
TypeError,
ValueError,
) as exc:
self.log.error("Unable to parse browser history database: %s", exc)
self.log.info(
"Extracted a total of %d browser history items", len(self.results)
)
+47 -9
View File
@@ -6,12 +6,23 @@ import datetime
import fnmatch
import logging
import os
import re
from pathlib import Path
from typing import List, Optional
from zipfile import ZipFile
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
from mvt.common.module import ModuleResults, MVTModule
# `------ 0.101s was the duration of 'SOME SECTION' ------`, printed when that
# section finishes and not necessarily between two sections.
SECTION_DURATION = re.compile(r"^-{3,}\s*[0-9.]+s was the duration of", re.IGNORECASE)
# The members a bug report archive can be entered through, in the order
# _get_dumpstate_file() tries them. An archive carrying none of them is not a
# bug report at this level (see CmdAndroidCheckBugreport._has_dumpstate).
DUMPSTATE_ENTRY_POINTS = ("main_entry.txt", "dumpState_*.log", "*/dumpsys.txt")
class BugReportModule(MVTModule):
"""This class provides a base for all Android Bug Report modules."""
@@ -86,7 +97,9 @@ class BugReportModule(MVTModule):
return data
def _get_dumpstate_file(self) -> Optional[bytes]:
main = self._get_files_by_pattern("main_entry.txt")
main_entry, dumpstate_log, dumpsys_txt = DUMPSTATE_ENTRY_POINTS
main = self._get_files_by_pattern(main_entry)
if main:
main_content = self._get_file_content(main[0])
try:
@@ -94,11 +107,11 @@ class BugReportModule(MVTModule):
except KeyError:
return None
dumpstate_logs = self._get_files_by_pattern("dumpState_*.log")
dumpstate_logs = self._get_files_by_pattern(dumpstate_log)
if dumpstate_logs:
return self._get_file_content(dumpstate_logs[0])
dumpsys_files = self._get_files_by_pattern("*/dumpsys.txt")
dumpsys_files = self._get_files_by_pattern(dumpsys_txt)
if dumpsys_files:
return self._get_file_content(dumpsys_files[0])
@@ -121,16 +134,41 @@ class BugReportModule(MVTModule):
in_section = True
continue
if stripped.startswith("------"):
# dumpstate prints a section's timing line when that section
# finishes, which can land in the middle of the one being
# written. Treating it as a boundary truncates the section at
# an arbitrary point, silently.
if SECTION_DURATION.match(stripped):
continue
break
lines.append(line)
return "\n".join(lines)
def _device_timezone(self) -> Optional[datetime.tzinfo]:
"""The device's timezone named in module_options, or None when unknown."""
name = self.module_options.get("device_timezone")
if not name:
return None
try:
return ZoneInfo(name)
except ZoneInfoNotFoundError:
self.log.warning("Unknown device timezone %s", name)
return None
def _get_file_modification_time(self, file_path: str) -> datetime.datetime:
"""When the file was last modified.
A zip entry carries the device's wall clock, so it is returned in the
device's timezone when the bugreport names one and naive otherwise.
An unpacked bugreport's mtime is whatever the extraction left, an
instant returned in UTC.
"""
if self.zip_archive:
file_timetuple = self.zip_archive.getinfo(file_path).date_time
return datetime.datetime(*file_timetuple)
else:
if not self.extract_path:
raise ValueError("extract_path is not set")
file_stat = os.stat(os.path.join(self.extract_path, file_path))
return datetime.datetime.fromtimestamp(file_stat.st_mtime)
return datetime.datetime(*file_timetuple, tzinfo=self._device_timezone())
if not self.extract_path:
raise ValueError("extract_path is not set")
file_stat = os.stat(os.path.join(self.extract_path, file_path))
return datetime.datetime.fromtimestamp(
file_stat.st_mtime, tz=datetime.timezone.utc
)
@@ -48,9 +48,22 @@ class DumpsysAccessibility(DumpsysAccessibilityArtifact, BugReportModule):
)
self.parse(content)
listed = unnamed = 0
for result in self.results:
self.log.info('Found accessibility service "%s"', result.get("component"))
if result.get("component"):
listed += 1
self.log.info(
'Found accessibility service "%s"', result.get("component")
)
continue
# The operator gets this per user as a LOW alert from
# check_indicators(); here it only has to survive into the summary,
# so that a stated count never reads as "a total of 0".
unnamed += result.get("unnamed_service_count") or 0
self.log.info(
"Identified a total of %d accessibility services", len(self.results)
"Identified a total of %d accessibility services (%d more stated by "
"the dump without a component name)",
listed,
unnamed,
)
@@ -4,15 +4,12 @@
# https://license.mvt.re/1.1/
import logging
import datetime
from typing import Optional
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
from mvt.common.utils import convert_datetime_to_iso
from .base import BugReportModule
from mvt.common.module_types import ModuleResults
from mvt.android.artifacts.file_timestamps import FileTimestampsArtifact
from mvt.android.artifacts.getprop import GetProp
class BugReportTimestamps(FileTimestampsArtifact, BugReportModule):
@@ -41,41 +38,28 @@ class BugReportTimestamps(FileTimestampsArtifact, BugReportModule):
def run(self) -> None:
filesystem_files = self._get_files_by_pattern("FS/*")
timezone_name = None
dumpstate = self._get_dumpstate_file()
if dumpstate:
section = self.extract_command_section(
dumpstate.decode("utf-8", errors="replace"),
"------ SYSTEM PROPERTIES",
)
properties = GetProp()
properties.parse(section)
timezone_name = properties.get_device_timezone()
timezone = None
if timezone_name:
try:
timezone = ZoneInfo(timezone_name)
except ZoneInfoNotFoundError:
self.log.warning("Unknown device timezone %s", timezone_name)
self.results = []
for file in filesystem_files:
# Only the modification time is available in the zip file metadata.
# The timezone is the local timezone of the machine the phone.
# A zip entry keeps the device's wall clock, read in the device's
# timezone when the command found one (see CmdAndroidCheckBugreport);
# an unpacked bugreport's mtime is read as a UTC instant.
modification_time = self._get_file_modification_time(file)
utc_time = None
if timezone is not None:
utc_time = convert_datetime_to_iso(
modification_time.replace(tzinfo=timezone).astimezone(
datetime.timezone.utc
)
)
self.results.append(
{
"path": file,
"modified_time": convert_datetime_to_iso(modification_time),
"modified_time_utc": utc_time,
"timezone": timezone_name,
"modified_time": convert_datetime_to_iso(
modification_time.replace(tzinfo=None)
),
"modified_time_utc": (
convert_datetime_to_iso(modification_time)
if modification_time.tzinfo
else None
),
"timezone": (
self.module_options.get("device_timezone")
if self.zip_archive
else "UTC"
),
"timestamp_source": (
"zip_metadata" if self.zip_archive else "filesystem_metadata"
),
@@ -18,5 +18,4 @@ class Settings(SettingsArtifact, BugReportModule):
data.decode("utf-8", errors="replace"), "DUMP OF SERVICE settings:"
)
self.parse(section)
count = sum(len(settings) for settings in self.results.values())
self.log.info("Identified %d Android settings", count)
self.log.info("Identified %d Android settings", len(self.results))
+10
View File
@@ -0,0 +1,10 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from mvt.common.module import MVTModule
from .browser_history import BrowserHistory
FS_MODULES: list[type[MVTModule]] = [BrowserHistory]
@@ -0,0 +1,115 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
import sqlite3
from pathlib import Path
from typing import Optional
from mvt.android.artifacts.browser_history import (
BrowserHistoryArtifact,
open_browser_history_database,
)
from mvt.common.module import MVTModule
from mvt.common.module_types import ModuleResults
# These locations are deliberately limited to paths backed by public parser
# fixtures or the historical MVT implementation.
BROWSER_HISTORY_PATHS = {
"data/data/com.android.chrome/app_chrome/Default/History": (
"Chrome",
"com.android.chrome",
"Default",
),
"data/data/com.brave.browser/app_chrome/Default/History": (
"Brave",
"com.brave.browser",
"Default",
),
"data/data/com.microsoft.emmx/app_chrome/Default/History": (
"Microsoft Edge",
"com.microsoft.emmx",
"Default",
),
"data/data/com.sec.android.app.sbrowser/app_sbrowser/Default/History": (
"Samsung Internet",
"com.sec.android.app.sbrowser",
"Default",
),
}
class BrowserHistory(BrowserHistoryArtifact, MVTModule):
"""Extract supported Chromium History databases from a filesystem dump."""
supported_commands = (("android", "check-fs"),)
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[ModuleResults] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def _database_paths(self) -> list[tuple[Path, str, str, str]]:
if not self.target_path:
return []
target = Path(self.target_path)
if target.is_file():
normalized_target = target.resolve().as_posix()
for relative_path, identity in BROWSER_HISTORY_PATHS.items():
if normalized_target.endswith(f"/{relative_path}"):
return [(target, *identity)]
return [(target, "Chromium", "unknown", "unknown")]
databases = []
for relative_path, identity in BROWSER_HISTORY_PATHS.items():
database_path = target / relative_path
if database_path.is_file():
databases.append((database_path, *identity))
return databases
def run(self) -> None:
for database_path, browser, package, profile in self._database_paths():
try:
connection = open_browser_history_database(database_path)
try:
self._parse_browser_history(
connection,
browser=browser,
package=package,
profile=profile,
source_path=str(database_path),
)
finally:
connection.close()
except (
OSError,
OverflowError,
sqlite3.Error,
TypeError,
ValueError,
) as exc:
self.log.error(
"Unable to parse browser history database %s: %s",
database_path,
exc,
)
self.log.info(
"Extracted a total of %d browser history items", len(self.results)
)
+6 -2
View File
@@ -56,6 +56,10 @@ class CmdCheckIOCS(Command):
if entry not in all_modules:
all_modules.append(entry)
# Read the indicators once, so that a missing indicators file is
# reported even when no stored result matches a module.
iocs = self.iocs
log.info("Checking stored results against provided indicators...")
total_detections = 0
@@ -83,8 +87,8 @@ class CmdCheckIOCS(Command):
log.warning("No result from this module, skipping it")
continue
if self.iocs.total_ioc_count > 0:
m.indicators = self.iocs
if iocs.total_ioc_count > 0:
m.indicators = iocs
m.indicators.log = m.log
try:
+24 -10
View File
@@ -8,6 +8,7 @@ import logging
import os
import sys
from datetime import datetime
from functools import cached_property
from heapq import heappop, heappush
from typing import Any, Optional
@@ -84,18 +85,18 @@ class Command:
self.timeline: ModuleTimeline = []
self.url_results: list[URLResult] = []
# Load IOCs
self._create_storage()
self._setup_logging()
if iocs is not None:
self.iocs = iocs
else:
self.iocs = Indicators(self.log)
self.iocs.load_indicators_files(self.ioc_files)
self.alertstore = AlertStore()
@cached_property
def iocs(self) -> Indicators:
"""Load indicators on first use. Nested commands share their parent's."""
iocs = Indicators(self.log)
iocs.load_indicators_files(self.ioc_files)
return iocs
def _create_storage(self) -> None:
if self.results_path and not os.path.exists(self.results_path):
try:
@@ -710,17 +711,30 @@ class Command:
return ordered
def run(self) -> None:
# The output folder and its command.log exist for a run, so that
# listing modules or rejecting a target leaves nothing behind.
# Resolving the module list can warn, so the log comes first.
self._create_storage()
self._setup_logging()
ordered_modules = self._ordered_modules()
if ordered_modules is None:
return
self._log_loaded_modules(ordered_modules)
# Read the indicators once the run is certain to happen, before
# init() does any work on the target, so that a missing indicators
# file is reported first and every module gets the same object.
iocs = self.iocs
# Commands announce their target from init(), so it goes before the
# module list.
try:
self.init()
except NotImplementedError:
pass
self._log_loaded_modules(ordered_modules)
executed_by_type: dict[type[MVTModule], MVTModule] = {}
for module in ordered_modules:
@@ -740,8 +754,8 @@ class Command:
for dependency, resolved in self._module_dependencies(module)
}
if self.iocs.total_ioc_count:
m.indicators = self.iocs
if iocs.total_ioc_count:
m.indicators = iocs
m.indicators.log = m.log
if self.serial:
+11 -6
View File
@@ -12,7 +12,9 @@ from pydantic_settings import (
YamlConfigSettingsSource,
)
MVT_CONFIG_FOLDER = user_config_dir("mvt")
# MVT_CONFIG_FOLDER in the environment relocates the settings file, so that
# a test run or a scripted install never touches the user's own.
MVT_CONFIG_FOLDER = os.environ.get("MVT_CONFIG_FOLDER") or user_config_dir("mvt")
MVT_CONFIG_PATH = os.path.join(MVT_CONFIG_FOLDER, "config.yaml")
@@ -59,13 +61,16 @@ class MVTSettings(BaseSettings):
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> Tuple[PydanticBaseSettingsSource, ...]:
yaml_source = YamlConfigSettingsSource(settings_cls, MVT_CONFIG_PATH)
sources: Tuple[PydanticBaseSettingsSource, ...] = (
yaml_source,
YamlConfigSettingsSource(settings_cls, MVT_CONFIG_PATH),
init_settings,
)
# Always load env variables by default
sources = (env_settings,) + sources
# Load env variables only when asked to. initialise() constructs the
# settings once without them so that what gets written back to
# config.yaml never includes values taken from the environment.
# init_settings() returns the keyword arguments passed to the constructor.
if init_settings().get("load_env", True):
sources = (env_settings,) + sources
return sources
def save_settings(
@@ -92,7 +97,7 @@ class MVTSettings(BaseSettings):
Afterwards we load the settings again, this time including the env variables.
"""
# Set invalid env prefix to avoid loading env variables.
# Construct the settings without env variables so they are not persisted.
settings = cls(load_env=False)
settings.save_settings()
+1
View File
@@ -39,6 +39,7 @@ HELP_MSG_DECRYPT_BACKUP = "Decrypt an encrypted iTunes backup"
HELP_MSG_BACKUP_DESTINATION = (
"Path to the folder where the decrypted backup should be stored"
)
HELP_MSG_DECRYPT_JOBS = "Number of files to decrypt concurrently"
HELP_MSG_IOS_BACKUP_PASSWORD = (
"Password to use to decrypt the backup (or, set the {MVT_IOS_BACKUP_PASSWORD} "
"environment variable)"
+7 -5
View File
@@ -18,7 +18,9 @@ from appdirs import user_data_dir
from .config import settings
from .url import URL
MVT_DATA_FOLDER = user_data_dir("mvt")
# MVT_DATA_FOLDER in the environment relocates the downloaded indicators and
# the update-check state kept next to them.
MVT_DATA_FOLDER = os.environ.get("MVT_DATA_FOLDER") or user_data_dir("mvt")
MVT_INDICATORS_FOLDER = os.path.join(MVT_DATA_FOLDER, "indicators")
logger = logging.getLogger(__name__)
@@ -71,7 +73,9 @@ class Indicators:
if os.path.isfile(path) and path.lower().endswith(".stix2"):
self.parse_stix2(path)
elif os.path.isdir(path):
for file in glob.glob(os.path.join(path, "**", "*.stix2"), recursive=True):
for file in glob.glob(
os.path.join(path, "**", "*.stix2"), recursive=True
):
self.parse_stix2(file)
else:
self.log.error(
@@ -518,9 +522,7 @@ class Indicators:
the original URL order.
"""
batches = [list(urls) if urls else [] for urls in url_batches]
unique_urls = list(
dict.fromkeys(url for urls in batches for url in urls)
)
unique_urls = list(dict.fromkeys(url for urls in batches for url in urls))
if not unique_urls:
return [None] * len(batches)
+3 -1
View File
@@ -349,7 +349,9 @@ class URL:
return tld_obj
if tld_obj is None:
return ""
return tld_obj.parsed_url.netloc.lower().lstrip("www.")
# removeprefix, not lstrip: lstrip takes a set of characters, so it ate the
# leading "w"s and dots of any domain ("web.evil.com" -> "eb.evil.com").
return tld_obj.parsed_url.netloc.lower().removeprefix("www.")
def get_top_level(self) -> str:
"""Get only the top-level domain from a URL.
+3 -1
View File
@@ -3,4 +3,6 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
MVT_VERSION = "2026.7.29"
from importlib.metadata import version
MVT_VERSION = version("mvt")
+22 -12
View File
@@ -24,6 +24,7 @@ from mvt.common.help import (
HELP_MSG_VERSION,
HELP_MSG_DECRYPT_BACKUP,
HELP_MSG_BACKUP_DESTINATION,
HELP_MSG_DECRYPT_JOBS,
HELP_MSG_IOS_BACKUP_PASSWORD,
HELP_MSG_BACKUP_KEYFILE,
HELP_MSG_HASHES,
@@ -45,6 +46,10 @@ from mvt.common.help import (
HELP_MSG_DISABLE_INDICATOR_UPDATE_CHECK,
)
from mvt.common.password import prompt_password
from .decrypt_config import (
DEFAULT_DECRYPT_WORKERS,
MAX_DECRYPT_WORKERS,
)
# The commands import what they run only when they are invoked. This module is
# imported at every start of mvt-ios, including by shell completion on every
@@ -128,6 +133,13 @@ def version():
"decrypt-backup", context_settings=CONTEXT_SETTINGS, help=HELP_MSG_DECRYPT_BACKUP
)
@click.option("--destination", "-d", required=True, help=HELP_MSG_BACKUP_DESTINATION)
@click.option(
"--jobs",
type=click.IntRange(1, MAX_DECRYPT_WORKERS),
default=DEFAULT_DECRYPT_WORKERS,
show_default=True,
help=HELP_MSG_DECRYPT_JOBS,
)
@click.option(
"--password",
"-p",
@@ -146,10 +158,10 @@ def version():
@click.option("--hashes", "-H", is_flag=True, help=HELP_MSG_HASHES)
@click.argument("BACKUP_PATH", type=click.Path(exists=True))
@click.pass_context
def decrypt_backup(ctx, destination, password, key_file, hashes, backup_path):
def decrypt_backup(ctx, destination, jobs, password, key_file, hashes, backup_path):
from .decrypt import DecryptBackup
backup = DecryptBackup(backup_path, destination)
backup = DecryptBackup(backup_path, destination, max_workers=jobs)
if key_file:
if MVT_IOS_BACKUP_PASSWORD in os.environ:
@@ -306,8 +318,6 @@ def check_backup(
if not cmd.resolve_backup_path():
ctx.exit(1)
log.info("Checking iTunes backup located at: %s", cmd.target_path)
cmd.run()
cmd.show_alerts_brief()
cmd.show_support_message()
@@ -374,8 +384,6 @@ def check_fs(
cmd.list_modules()
return
log.info("Checking iOS filesystem located at: %s", dump_path)
cmd.run()
cmd.show_alerts_brief()
cmd.show_support_message()
@@ -437,18 +445,20 @@ def check_sysdiagnose(
custom_modules=custom_modules,
)
if not cmd._available_modules():
raise click.ClickException(
"No custom modules support mvt-ios check-sysdiagnose. "
"Load a module that declares supported_commands = "
"((\"ios\", \"check-sysdiagnose\"),)."
# MVT's own module only records the device details; the checks come from
# custom modules, so a run without any must not look like a clean analysis.
if all(module in cmd.modules for module in cmd._available_modules()):
log.warning(
"No forensic sysdiagnose modules have been loaded: MVT's own "
"SysdiagnoseInfo module only records the device details. Install a "
"module package or load a module that declares supported_commands = "
'(("ios", "check-sysdiagnose"),) to check the sysdiagnose.'
)
if list_modules:
cmd.list_modules()
return
log.info("Checking iOS sysdiagnose at path: %s", sysdiagnose_path)
cmd.run()
cmd.show_alerts_brief()
cmd.show_support_message()
+3
View File
@@ -59,6 +59,9 @@ class CmdIOSCheckBackup(Command):
self.name = "check-backup"
self.modules = BACKUP_MODULES + MIXED_MODULES
def init(self) -> None:
self.log.info("Checking iTunes backup located at: %s", self.target_path)
def resolve_backup_path(self) -> bool:
target_path = getattr(self, "target_path", None)
if not isinstance(target_path, str) or not target_path:
+3
View File
@@ -51,5 +51,8 @@ class CmdIOSCheckFS(Command):
self.name = "check-fs"
self.modules = FS_MODULES + MIXED_MODULES
def init(self) -> None:
self.log.info("Checking iOS filesystem located at: %s", self.target_path)
def module_init(self, module):
module.is_fs_dump = True
+27 -2
View File
@@ -7,7 +7,9 @@ import json
import logging
import os
import shutil
import sys
import tarfile
import zlib
from pathlib import Path, PurePosixPath
from tempfile import TemporaryDirectory
from typing import Any, Optional
@@ -16,6 +18,8 @@ from mvt.common.command import Command
from mvt.common.indicators import Indicators
from mvt.common.module import MVTModule
from .modules.sysdiagnose import SYSDIAGNOSE_MODULES
log = logging.getLogger(__name__)
@@ -52,6 +56,7 @@ class CmdIOSCheckSysdiagnose(Command):
)
self.platform = "ios"
self.name = "check-sysdiagnose"
self.modules = SYSDIAGNOSE_MODULES
self.sysdiagnose_format: Optional[str] = None
self.sysdiagnose_archive: Optional[tarfile.TarFile] = None
self.sysdiagnose_files: list[str] = []
@@ -76,11 +81,15 @@ class CmdIOSCheckSysdiagnose(Command):
if not self.target_path:
raise ValueError("A sysdiagnose path is required")
self.log.info("Checking iOS sysdiagnose at path: %s", self.target_path)
if os.path.isdir(self.target_path):
self.sysdiagnose_format = "dir"
parent_path = Path(self.target_path).absolute().parent
for root, _, filenames in os.walk(self.target_path):
for filename in filenames:
if filename.startswith("._"):
continue
absolute_path = os.path.join(root, filename)
file_path = os.path.relpath(absolute_path, parent_path)
self.sysdiagnose_files.append(file_path)
@@ -94,8 +103,19 @@ class CmdIOSCheckSysdiagnose(Command):
self.log.info("Parsing sysdiagnose archive. This might take a while...")
self.sysdiagnose_format = "tar"
self.sysdiagnose_archive = tarfile.open(self.target_path, "r:gz")
self._extract_sysdiagnose_archive()
try:
self.sysdiagnose_archive = tarfile.open(self.target_path, "r:gz")
self._extract_sysdiagnose_archive()
except (tarfile.ReadError, EOFError, zlib.error, OSError) as exc:
# A truncated archive ends in EOFError from gzip, which Click would
# otherwise report as a bare "Aborted!" with no reason.
self.log.critical(
"Unable to read the sysdiagnose archive %s: %s. "
"The file may be truncated or not a gzip-compressed tarball.",
self.target_path,
exc,
)
sys.exit(1)
def _extract_sysdiagnose_archive(self) -> None:
archive = self.sysdiagnose_archive
@@ -119,6 +139,11 @@ class CmdIOSCheckSysdiagnose(Command):
if not member_path.parts:
continue
# AppleDouble sidecars (._name) carry a file's extended attributes,
# not sysdiagnose content. Device archives hold hundreds of them;
# bsdtar hides them from listings, tarfile does not.
if member_path.name.startswith("._"):
continue
archive_roots.add(member_path.parts[0])
if member.isdir():
+2 -1
View File
@@ -16,7 +16,8 @@ from mvt.common.module import MVTModule
from .modules.backup import BACKUP_MODULES
from .modules.fs import FS_MODULES
from .modules.mixed import MIXED_MODULES
from .modules.sysdiagnose import SYSDIAGNOSE_MODULES
IOS_CHECK_IOCS_MODULES: list[type[MVTModule]] = (
BACKUP_MODULES + FS_MODULES + MIXED_MODULES
BACKUP_MODULES + FS_MODULES + MIXED_MODULES + SYSDIAGNOSE_MODULES
)
+12
View File
@@ -1271,5 +1271,17 @@
{
"version": "26.6.1",
"build": "23G83"
},
{
"version": "26.6.2",
"build": "23G90"
},
{
"version": "27.0",
"build": "24A437"
},
{
"version": "27.0.1",
"build": "24A446"
}
]
+294 -76
View File
@@ -6,18 +6,179 @@
import binascii
import glob
import logging
import multiprocessing
import os
import os.path
import plistlib
import shutil
import sqlite3
import tempfile
from concurrent.futures import (
ALL_COMPLETED,
FIRST_COMPLETED,
Future,
ThreadPoolExecutor,
wait,
)
from pathlib import Path
from typing import Optional
from iOSbackup import iOSbackup
from iphone_backup_decrypt import EncryptedBackup
from iphone_backup_decrypt import google_iphone_dataprotection
from iphone_backup_decrypt.utils import FilePlist
from .decrypt_config import DEFAULT_DECRYPT_WORKERS, MAX_DECRYPT_WORKERS
log = logging.getLogger(__name__)
# Import pbkdf2_hmac from the same source iphone_backup_decrypt uses internally,
# so our key derivation is consistent with theirs.
try:
from fastpbkdf2 import pbkdf2_hmac
except ImportError:
import Crypto.Hash.SHA1
import Crypto.Hash.SHA256
import Crypto.Protocol.KDF
_HASH_FNS = {"sha1": Crypto.Hash.SHA1, "sha256": Crypto.Hash.SHA256}
def pbkdf2_hmac(hash_name, password, salt, iterations, dklen=None):
return Crypto.Protocol.KDF.PBKDF2(
password, salt, dklen, iterations, hmac_hash_module=_HASH_FNS[hash_name]
)
class MVTEncryptedBackup(EncryptedBackup):
"""Extends EncryptedBackup with derived key export/import.
NOTE: This subclass relies on internal APIs of iphone_backup_decrypt
(specifically _read_and_unlock_keybag, _keybag, and the Keybag class
internals). Pinned to iphone_backup_decrypt==0.10.0.
"""
def __init__(self, *, backup_directory, passphrase=None, derived_key=None):
if passphrase:
super().__init__(backup_directory=backup_directory, passphrase=passphrase)
self._derived_key = None # Will be set after keybag unlock
elif derived_key:
self._init_without_passphrase(backup_directory, derived_key)
else:
raise ValueError("Either passphrase or derived_key must be provided")
def _init_without_passphrase(self, backup_directory, derived_key):
"""Replicate parent __init__ state without requiring a passphrase."""
self.decrypted = False
self._backup_directory = os.path.expandvars(backup_directory)
self._passphrase = None
self._manifest_plist_path = os.path.join(
self._backup_directory, "Manifest.plist"
)
self._manifest_plist = None
self._manifest_db_path = os.path.join(self._backup_directory, "Manifest.db")
self._keybag = None
self._unlocked = False
self._temporary_folder = tempfile.mkdtemp()
self._temp_decrypted_manifest_db_path = os.path.join(
self._temporary_folder, "Manifest.db"
)
self._temp_manifest_db_conn = None
self._derived_key = derived_key # 32 raw bytes
def _read_and_unlock_keybag(self):
"""Override to capture derived key on password unlock, or use
a pre-derived key to skip PBKDF2."""
if self._unlocked:
return self._unlocked
with open(self._manifest_plist_path, "rb") as infile:
self._manifest_plist = plistlib.load(infile)
self._keybag = google_iphone_dataprotection.Keybag(
self._manifest_plist["BackupKeyBag"]
)
if self._derived_key:
# Skip PBKDF2, unwrap class keys directly with pre-derived key
self._unlocked = _unlock_keybag_with_derived_key(
self._keybag, self._derived_key
)
else:
# Normal path: full PBKDF2 derivation, capturing the intermediate key
self._unlocked, self._derived_key = _unlock_keybag_and_capture_key(
self._keybag, self._passphrase
)
self._passphrase = None
if not self._unlocked:
raise ValueError("Failed to decrypt keys: incorrect passphrase?")
return True
def get_decryption_key(self):
"""Return derived key as hex string (64 chars / 32 bytes)."""
if self._derived_key is None:
raise ValueError("No derived key available")
return self._derived_key.hex()
def extract_file_by_id(self, *, file_id, file_bplist, output_filename):
"""Extract one manifest entry without loading the whole file into memory."""
self._read_and_unlock_keybag()
file_plist = FilePlist(file_bplist)
if file_plist.encryption_key is None:
source_filename = os.path.join(
self._backup_directory, file_id[:2], file_id
)
shutil.copy2(source_filename, output_filename)
return
inner_key = self._keybag.unwrapKeyForClass(
file_plist.protection_class, file_plist.encryption_key
)
self._decrypt_file_to_disk(
file_id=file_id,
key=inner_key,
file_plist=file_plist,
output_filepath=output_filename,
)
def _unlock_keybag_with_derived_key(keybag, passphrase_key):
"""Unlock keybag class keys using a pre-derived passphrase_key,
skipping the expensive PBKDF2 rounds."""
WRAP_PASSPHRASE = 2
for classkey in keybag.classKeys.values():
if b"WPKY" not in classkey:
continue
if classkey[b"WRAP"] & WRAP_PASSPHRASE:
k = google_iphone_dataprotection._AESUnwrap(
passphrase_key, classkey[b"WPKY"]
)
if not k:
return False
classkey[b"KEY"] = k
return True
def _unlock_keybag_and_capture_key(keybag, passphrase):
"""Run full PBKDF2 key derivation and AES unwrap, returning
(success, passphrase_key) so the derived key can be exported."""
passphrase_round1 = pbkdf2_hmac(
"sha256", passphrase, keybag.attrs[b"DPSL"], keybag.attrs[b"DPIC"], 32
)
passphrase_key = pbkdf2_hmac(
"sha1", passphrase_round1, keybag.attrs[b"SALT"], keybag.attrs[b"ITER"], 32
)
WRAP_PASSPHRASE = 2
for classkey in keybag.classKeys.values():
if b"WPKY" not in classkey:
continue
if classkey[b"WRAP"] & WRAP_PASSPHRASE:
k = google_iphone_dataprotection._AESUnwrap(
passphrase_key, classkey[b"WPKY"]
)
if not k:
return False, None
classkey[b"KEY"] = k
return True, passphrase_key
class DecryptBackup:
"""This class provides functions to decrypt an encrypted iTunes backup
@@ -26,19 +187,68 @@ class DecryptBackup:
"""
def __init__(self, backup_path: str, dest_path: Optional[str] = None) -> None:
def __init__(
self,
backup_path: str,
dest_path: Optional[str] = None,
max_workers: int = DEFAULT_DECRYPT_WORKERS,
) -> None:
"""Decrypts an encrypted iOS backup.
:param backup_path: Path to the encrypted backup folder
:param dest_path: Path to the folder where to store the decrypted backup
"""
self.backup_path = os.path.abspath(backup_path)
self.dest_path = dest_path
self._backup = None
self._decryption_key = None
if not 1 <= max_workers <= MAX_DECRYPT_WORKERS:
raise ValueError(f"max_workers must be between 1 and {MAX_DECRYPT_WORKERS}")
self.max_workers = max_workers
self._backup: Optional[MVTEncryptedBackup] = None
self._decryption_key: Optional[str] = None
def can_process(self) -> bool:
return self._backup is not None
def _process_file(
self,
*,
file_id: str,
file_bplist: bytes,
output_path: Path,
relative_path: str,
domain: str,
) -> None:
assert self._backup is not None
self._backup.extract_file_by_id(
file_id=file_id,
file_bplist=file_bplist,
output_filename=str(output_path),
)
log.info(
"Decrypted file %s [%s] to %s/%s",
relative_path,
domain,
output_path.parent,
file_id,
)
@staticmethod
def _wait_for_files(
pending: dict[Future[None], str], *, all_files: bool = False
) -> None:
if not pending:
return
done, _ = wait(
pending,
return_when=ALL_COMPLETED if all_files else FIRST_COMPLETED,
)
for future in done:
relative_path = pending.pop(future)
try:
future.result()
except Exception as exc:
log.error("Failed to decrypt file %s: %s", relative_path, exc)
@staticmethod
def is_encrypted(backup_path: str) -> bool:
"""Query Manifest.db file to see if it's encrypted or not.
@@ -58,21 +268,6 @@ class DecryptBackup:
finally:
conn.close()
def _process_file(
self, relative_path: str, domain: str, item, file_id: str, item_folder: str
) -> None:
assert self._backup is not None
self._backup.getFileDecryptedCopy(
manifestEntry=item, targetName=file_id, targetFolder=item_folder
)
log.info(
"Decrypted file %s [%s] to %s/%s",
relative_path,
domain,
item_folder,
file_id,
)
def process_backup(self) -> None:
assert self._backup is not None
assert self.dest_path is not None
@@ -81,53 +276,71 @@ class DecryptBackup:
os.makedirs(self.dest_path)
manifest_path = os.path.join(self.dest_path, "Manifest.db")
# We extract a decrypted Manifest.db.
self._backup.getManifestDB()
# We store it to the destination folder.
shutil.copy(self._backup.manifestDB, manifest_path)
# Extract a decrypted Manifest.db to the destination folder.
self._backup.save_manifest_file(output_filename=manifest_path)
pool = multiprocessing.Pool(multiprocessing.cpu_count())
for item in self._backup.getBackupFilesList():
try:
file_id = item["backupFile"]
relative_path = item["relativePath"]
domain = item["domain"]
# This may be a partial backup. Skip files from the manifest
# which do not exist locally.
source_file_path = os.path.join(self.backup_path, file_id[0:2], file_id)
if not Path(source_file_path).resolve().is_relative_to(Path(self.backup_path).resolve()):
log.warning("Skipping unsafe file_id: %r", file_id)
continue
if not os.path.exists(source_file_path):
log.debug(
"Skipping file %s. File not found in encrypted backup directory.",
source_file_path,
)
continue
item_folder = os.path.join(self.dest_path, file_id[0:2]) # type: ignore[arg-type]
if not Path(os.path.join(item_folder, file_id)).resolve().is_relative_to(Path(self.dest_path).resolve()):
log.warning("Skipping unsafe file_id: %r", file_id)
continue
if not os.path.exists(item_folder):
os.makedirs(item_folder)
# iOSBackup getFileDecryptedCopy() claims to read a "file"
# parameter but the code actually is reading the "manifest" key.
# Add manifest plist to both keys to handle this.
item["manifest"] = item["file"]
pool.apply_async(
self._process_file,
args=(relative_path, domain, item, file_id, item_folder),
# Iterate over all files in the backup and decrypt them,
# preserving the XX/file_id directory structure that downstream
# modules expect.
backup_root = Path(self.backup_path).resolve()
dest_root = Path(self.dest_path).resolve()
pending: dict[Future[None], str] = {}
with ThreadPoolExecutor(max_workers=self.max_workers) as executor:
with self._backup.manifest_db_cursor() as cur:
cur.execute(
"SELECT fileID, domain, relativePath, file FROM Files WHERE flags=1"
)
except Exception as exc:
log.error("Failed to decrypt file %s: %s", relative_path, exc)
for file_id, domain, relative_path, file_bplist in cur:
# This may be a partial backup. Skip files from the manifest
# which do not exist locally.
source_file_path = backup_root / file_id[:2] / file_id
if not source_file_path.resolve().is_relative_to(backup_root):
log.warning("Skipping unsafe file_id: %r", file_id)
continue
if not os.path.exists(source_file_path):
log.debug(
"Skipping file %s. File not found in encrypted "
"backup directory.",
source_file_path,
)
continue
pool.close()
pool.join()
output_path = dest_root / file_id[:2] / file_id
if not output_path.resolve().is_relative_to(dest_root):
log.warning("Skipping unsafe file_id: %r", file_id)
continue
output_path.parent.mkdir(parents=True, exist_ok=True)
if self.max_workers == 1:
try:
self._process_file(
file_id=file_id,
file_bplist=file_bplist,
output_path=output_path,
relative_path=relative_path,
domain=domain,
)
except Exception as exc:
log.error(
"Failed to decrypt file %s: %s",
relative_path,
exc,
)
continue
future = executor.submit(
self._process_file,
file_id=file_id,
file_bplist=file_bplist,
output_path=output_path,
relative_path=relative_path,
domain=domain,
)
pending[future] = relative_path
if len(pending) >= self.max_workers:
self._wait_for_files(pending)
self._wait_for_files(pending, all_files=True)
# Copying over the root plist files as well.
for file_name in os.listdir(self.backup_path):
@@ -168,20 +381,23 @@ class DecryptBackup:
return
try:
self._backup = iOSbackup(
udid=os.path.basename(self.backup_path),
cleartextpassword=password,
backuproot=os.path.dirname(self.backup_path),
self._backup = MVTEncryptedBackup(
backup_directory=self.backup_path,
passphrase=password,
)
# Eagerly trigger keybag unlock so wrong-password errors
# surface here rather than later during process_backup().
self._backup.test_decryption()
except Exception as exc:
self._backup = None
if (
isinstance(exc, KeyError)
and len(exc.args) > 0
and exc.args[0] == b"KEY"
isinstance(exc, ValueError)
and "passphrase" in str(exc).lower()
):
log.critical("Failed to decrypt backup. Password is probably wrong.")
elif (
isinstance(exc, FileNotFoundError)
and hasattr(exc, "filename")
and os.path.basename(exc.filename) == "Manifest.plist"
):
log.critical(
@@ -224,12 +440,14 @@ class DecryptBackup:
try:
key_bytes_raw = binascii.unhexlify(key_bytes)
self._backup = iOSbackup(
udid=os.path.basename(self.backup_path),
derivedkey=key_bytes_raw,
backuproot=os.path.dirname(self.backup_path),
self._backup = MVTEncryptedBackup(
backup_directory=self.backup_path,
derived_key=key_bytes_raw,
)
# Eagerly trigger keybag unlock so wrong-key errors surface here.
self._backup.test_decryption()
except Exception as exc:
self._backup = None
log.exception(exc)
log.critical(
"Failed to decrypt backup. Did you provide the correct key file?"
@@ -240,7 +458,7 @@ class DecryptBackup:
if not self._backup:
return
self._decryption_key = self._backup.getDecryptionKey()
self._decryption_key = self._backup.get_decryption_key()
log.info(
'Derived decryption key for backup at path %s is: "%s"',
self.backup_path,
+9
View File
@@ -0,0 +1,9 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
"""Configuration shared by the iOS CLI and backup decryption implementation."""
DEFAULT_DECRYPT_WORKERS = 4
MAX_DECRYPT_WORKERS = 32
+29
View File
@@ -147,6 +147,12 @@ class Manifest(IOSExtraction):
)
names = [description[0] for description in cur.description]
# An incomplete backup still lists the files it failed to acquire in
# its manifest. Only regular files are stored in the backup folder, so
# directories and symlinks (flags 2 and 4) are not looked up.
stored_file_ids = self._get_stored_backup_file_ids()
missing_files = 0
for file_entry in cur:
file_data = {}
for index, value in enumerate(file_entry):
@@ -160,6 +166,22 @@ class Manifest(IOSExtraction):
"created": "",
}
if (
stored_file_ids is not None
and file_data["flags"] == 1
and file_data["fileID"] not in stored_file_ids
):
# Without this, a module which found nothing for one of these
# files would look like a negative result rather than a gap in
# the acquisition.
cleaned_metadata["missing"] = True
missing_files += 1
self.log.debug(
"File %s is listed in the manifest but was not found in the "
"backup folder",
cleaned_metadata["relative_path"],
)
if file_data["file"]:
try:
file_plist = plistlib.load(io.BytesIO(file_data["file"]))
@@ -197,3 +219,10 @@ class Manifest(IOSExtraction):
conn.close()
self.log.info("Extracted a total of %d file metadata items", len(self.results))
if missing_files:
self.log.info(
"Found %d files listed in the manifest but missing from the backup "
"folder. The backup might be incomplete.",
missing_files,
)
+49
View File
@@ -216,6 +216,55 @@ class IOSExtraction(MVTModule):
return None
def _get_stored_backup_file_ids(self) -> Optional[set[str]]:
"""List the IDs of the files actually stored in the backup folder.
A backup folder stores each file under a two character subfolder named
after the first two characters of its file ID. Walking the folders once
is cheaper than a filesystem lookup per file ID, and it keeps callers
which compare a whole manifest against the folder from paying a
`resolve()` for every entry.
:returns: The file IDs found at their expected paths within the backup
folder, or None if the inventory could not be completed.
"""
if not self.target_path:
return None
file_ids: set[str] = set()
try:
backup_root = Path(self.target_path).resolve()
with os.scandir(self.target_path) as entries:
for entry in entries:
if len(entry.name) != 2 or any(
char not in "0123456789abcdef" for char in entry.name
):
continue
if not entry.is_dir():
continue
if not Path(entry.path).resolve().is_relative_to(backup_root):
continue
with os.scandir(entry.path) as sub_entries:
for sub_entry in sub_entries:
if sub_entry.name[:2] != entry.name:
continue
if sub_entry.is_symlink() and not Path(
sub_entry.path
).resolve().is_relative_to(backup_root):
continue
if sub_entry.is_file():
file_ids.add(sub_entry.name)
except OSError as exc:
self.log.warning(
"Unable to list the files stored in the backup folder %s: %s. "
"Skipping the missing-file check.",
self.target_path,
exc,
)
return None
return file_ids
def _get_fs_files_from_patterns(self, root_paths: list) -> Iterator[str]:
if not self.target_path:
return
+6 -2
View File
@@ -82,7 +82,9 @@ class Filesystem(IOSExtraction):
try:
dir_path = os.path.join(root, dir_name)
result = {
"path": os.path.relpath(dir_path, self.target_path),
"path": os.path.relpath(dir_path, self.target_path).replace(
os.sep, "/"
),
"modified": convert_unix_to_iso(os.stat(dir_path).st_mtime),
}
except Exception:
@@ -94,7 +96,9 @@ class Filesystem(IOSExtraction):
try:
file_path = os.path.join(root, file_name)
result = {
"path": os.path.relpath(file_path, self.target_path),
"path": os.path.relpath(file_path, self.target_path).replace(
os.sep, "/"
),
"modified": convert_unix_to_iso(os.stat(file_path).st_mtime),
}
except Exception:
@@ -3,4 +3,9 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from mvt.common.module import MVTModule
from .base import SysdiagnoseExtraction
from .sysdiagnose_info import SysdiagnoseInfo
SYSDIAGNOSE_MODULES: list[type[MVTModule]] = [SysdiagnoseInfo]
@@ -0,0 +1,228 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import json
import logging
import os
import plistlib
import re
import sqlite3
import tempfile
from datetime import datetime
from pathlib import Path
from typing import Optional
from mvt.common.module_types import ModuleResults
from mvt.common.utils import convert_datetime_to_iso
from mvt.ios.versions import (
find_version_by_build,
get_device_desc_from_id,
is_ios_version_outdated,
)
from .base import SysdiagnoseExtraction
# The fields dumpsys prints in the remotectl dump state and the mobile
# activation request which are worth a log line of their own.
LOGGED_FIELDS = (
"ProductName",
"ProductType",
"SerialNumber",
"OSVersion",
"RegionCode",
"IMEI",
"BuildVersion",
)
class SysdiagnoseInfo(SysdiagnoseExtraction):
"""Extract details about the device and the sysdiagnose itself.
The fields come from four files of the archive: the remotectl dump state
(product type, OS version, serial number, region and the rest of its
Properties block), the mobile activation request (UDID, IMEI, MEID and the
OS build), the App Store daemon database (the Apple account name and email)
and sysdiagnose.log (the archive's original file name and creation time).
Newer iOS versions no longer include the App Store daemon database in a
sysdiagnose; it is still read for the analysis of older archives.
"""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[ModuleResults] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
self.results: dict = results if results is not None else {}
def _copy_sqlite_db(self, file_path: str, directory: str) -> str:
"""Copy a database and its WAL sidecars out of the archive.
A database dumped mid-transaction keeps its latest rows in the -wal
file next to it, which SQLite only reads when both sit in the same
directory under the same name.
"""
available_files = self.tar_files if self.tar else self.files
for suffix in ("", "-wal", "-shm"):
if suffix and f"{file_path}{suffix}" not in available_files:
continue
copy_path = os.path.join(directory, f"{Path(file_path).name}{suffix}")
with open(copy_path, "wb") as handle:
handle.write(self._get_file_content(f"{file_path}{suffix}"))
return os.path.join(directory, Path(file_path).name)
def _process_appstored(self, file_path: str) -> None:
self.log.info("Found App Store daemon database at: %s", file_path)
with tempfile.TemporaryDirectory(prefix="mvt_sqlite_") as directory:
db_path = Path(self._copy_sqlite_db(file_path, directory)).resolve()
conn = sqlite3.connect(f"{db_path.as_uri()}?mode=ro", uri=True)
try:
self._read_appstored(conn)
finally:
conn.close()
def _read_appstored(self, conn: sqlite3.Connection) -> None:
cur = conn.cursor()
# The account name sits in an opaque structure of every asset row.
try:
rows = cur.execute("SELECT sinfs_data FROM asset;").fetchall()
except sqlite3.DatabaseError as exc:
self.log.debug("Unable to read the asset table: %s", exc)
rows = []
for (sinfs_data,) in rows:
try:
sinf = plistlib.loads(sinfs_data)[0]["sinf"]
except (plistlib.InvalidFileException, IndexError, KeyError, TypeError):
continue
match = re.search(rb"name(.*?)\x00", sinf)
if match:
self.results["Account Name"] = match.group(1).decode(
"utf-8", errors="replace"
)
break
try:
row = cur.execute(
"SELECT store_account_name FROM job_software "
"WHERE store_account_name IS NOT NULL LIMIT 1;"
).fetchone()
except sqlite3.DatabaseError as exc:
self.log.debug("Unable to read the job_software table: %s", exc)
return
if row:
self.results["Email Address"] = row[0]
def _process_activation_log(self, file_path: str) -> None:
self.log.info("Found mobile activation request at: %s", file_path)
content = self._get_file_content(file_path)
match = re.search(rb"BODY:\s+({.+?})\s", content, re.MULTILINE)
if not match:
return
try:
body = json.loads(match.group(1))
except json.JSONDecodeError as exc:
self.log.warning("Unable to parse the activation request body: %s", exc)
return
self.results.update(
{
"SerialNumber": body.get("serial-number"),
"ProductType": body.get("productType"),
"ProductName": body.get("productName"),
"IMEI": body.get("imei"),
"ProductVersion": body.get("os-version"),
"UniqueIdentifier": body.get("udid"),
"MEID": body.get("meid"),
"BuildVersion": body.get("os-build"),
}
)
def _process_dumpstate(self, file_path: str) -> None:
self.log.info("Found remotectl dump state at: %s", file_path)
content = self._get_file_content(file_path).decode("utf-8", errors="replace")
in_properties = False
for line in content.splitlines():
if not in_properties:
in_properties = line == "\tProperties: {"
continue
if line == "\t}":
break
key, separator, value = line.partition("=>")
if separator:
self.results[key.strip()] = value.strip()
def _process_sysdiagnose_log(self, file_path: str) -> None:
self.log.info("Found sysdiagnose.log at: %s", file_path)
content = self._get_file_content(file_path).decode("utf-8", errors="replace")
match = re.search(r"sysdiagnose_\S+?\.tar\.gz", content)
if not match:
self.log.info("Could not find the original output path in sysdiagnose.log")
return
file_name = os.path.basename(match.group(0))
try:
created = datetime.strptime(
"_".join(file_name.split("_")[1:3]), "%Y.%m.%d_%H-%M-%S%z"
)
except ValueError:
self.log.warning("Unexpected sysdiagnose file name: %s", file_name)
return
self.results["OriginalFilename"] = file_name
self.results["CreatedTimestamp"] = convert_datetime_to_iso(created)
def run(self) -> None:
for file_path in self._get_files_by_pattern(
"*/logs/appinstallation/appstored.sqlitedb"
):
self._process_appstored(file_path)
for file_path in self._get_files_by_pattern(
"*/logs/MobileActivation/collection_oob_request.txt"
):
self._process_activation_log(file_path)
for file_path in self._get_files_by_pattern("*/remotectl_dumpstate.txt"):
self._process_dumpstate(file_path)
for file_path in self._get_files_by_pattern("*/sysdiagnose.log"):
self._process_sysdiagnose_log(file_path)
# The activation request names the product "iPhone OS"; the model
# description is what an analyst wants to read.
product_name = get_device_desc_from_id(self.results.get("ProductType", ""))
if product_name:
self.results["ProductName"] = product_name
for field in LOGGED_FIELDS:
if field not in self.results:
continue
value = self.results[field]
if field == "BuildVersion" and value:
self.log.info("%s: %s - %s", field, value, find_version_by_build(value))
else:
self.log.info("%s: %s", field, value)
if self.results.get("BuildVersion"):
is_ios_version_outdated(self.results["BuildVersion"], self.log)
@@ -35,11 +35,16 @@ class TestDumpsysAccessibilityArtifact:
assert len(da.results) == 0
da.parse(data)
assert len(da.results) == 1
# One named service, plus one count-only record: the dump states
# `installedServiceCount=2` and names only one component.
assert len(da.results) == 2
assert da.results[0]["package_name"] == "com.malware.accessibility"
assert da.results[0]["service_name"] == "com.malware.service.malwareservice"
assert da.results[0]["enabled"] is True
assert da.results[0]["installed"] is False
# This fixture never prints an `installed services:` section, so the
# dump does not state the installed status. Reporting False would turn
# "not stated" into "not installed", so it reads None here.
assert da.results[0]["installed"] is None
def test_accessibility_service_alert(self):
da = DumpsysAccessibilityArtifact()
@@ -50,9 +55,11 @@ class TestDumpsysAccessibilityArtifact:
da.check_indicators()
assert len(da.alertstore.alerts) == 1
assert len(da.alertstore.alerts) == 2
assert da.alertstore.alerts[0].level == AlertLevel.MEDIUM
assert da.alertstore.alerts[0].event == da.results[0]
assert da.alertstore.alerts[1].level == AlertLevel.LOW
assert da.alertstore.alerts[1].event == da.results[1]
def test_same_component_is_kept_for_each_user(self):
da = DumpsysAccessibilityArtifact()
@@ -84,7 +91,11 @@ User state[attributes:{id=10
assert len(da.alertstore.alerts) == 0
da.check_indicators()
assert len(da.alertstore.alerts) == len(da.results)
assert da.alertstore.count(AlertLevel.MEDIUM) == 3
# Every service in this fixture is installed and switched off
# (`enabled services:{}` is printed and empty), so the three non-IOC
# findings are LOW, not MEDIUM. The IOC match is unaffected by the
# state.
assert da.alertstore.count(AlertLevel.LOW) == 3
assert da.alertstore.count(AlertLevel.CRITICAL) == 1
critical_alert = next(
alert
@@ -0,0 +1,149 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
"""The dump's own installed-service count must survive into the artifact.
Most builds never print the `installed services: {…}` block; they state `installedServiceCount=N` in the user's `attributes:{…}` line and
list nothing. Dropping that number makes an artifact that says "no
accessibility services" about a dump that said there are five.
"""
from mvt.android.artifacts.dumpsys_accessibility import DumpsysAccessibilityArtifact
from mvt.common.alerts import AlertLevel
from ..utils import get_artifact
AOSP_NO_LIST = """\
ACCESSIBILITY MANAGER (dumpsys accessibility)
User state[
attributes:{id=0, touchExplorationEnabled=false, installedServiceCount=5}
Bound services:{}
Enabled services:{}
Binding services:{}
Crashed services:{}
"""
ONE_UI_WITH_LIST = """\
ACCESSIBILITY MANAGER (dumpsys accessibility)
User state[attributes:{id=0, installedServiceCount=2}
installed services: {
0 : com.example.app/com.example.app.Service
1 : com.other.app/.Helper
}
enabled services: {
}
"""
TWO_USERS = """\
ACCESSIBILITY MANAGER (dumpsys accessibility)
User state[attributes:{id=0, installedServiceCount=1}
installed services: {
0 : com.example.app/com.example.app.Service
}
User state[attributes:{id=95, installedServiceCount=3}
Enabled services:{}
"""
PARTIAL_TWO_USERS = """\
ACCESSIBILITY MANAGER (dumpsys accessibility)
User state[attributes:{id=0, installedServiceCount=3}
Enabled services:{{com.example.app/com.example.app.Service}}
User state[attributes:{id=10, installedServiceCount=1}
Enabled services:{{com.other.app/.Helper}}
"""
ZERO_COUNT = """\
ACCESSIBILITY MANAGER (dumpsys accessibility)
User state[attributes:{id=0, installedServiceCount=0}
Enabled services:{}
"""
def _parse(content):
artifact = DumpsysAccessibilityArtifact()
artifact.results = []
artifact.parse(content)
return artifact
class TestAccessibilityInstalledServiceCount:
def test_stated_count_without_a_list_is_kept(self):
artifact = _parse(AOSP_NO_LIST)
assert len(artifact.results) == 1
record = artifact.results[0]
assert record["installed_service_count"] == 5
assert record["component"] is None
# Every state flag stays unknown: the dump named no service to which a
# state could belong.
assert record["installed"] is None
assert record["enabled"] is None
def test_a_stated_count_is_reported_as_low(self):
artifact = _parse(AOSP_NO_LIST)
artifact.check_indicators()
alerts = artifact.alertstore.alerts
assert len(alerts) == 1
# A count is a coverage statement, not a running service: it must not
# compete with a service the dump says is enabled.
assert alerts[0].level == AlertLevel.LOW
assert "does not list their component names" in alerts[0].message
assert "5 installed" in alerts[0].message
def test_a_listed_user_carries_the_count_on_each_service(self):
artifact = _parse(ONE_UI_WITH_LIST)
assert len(artifact.results) == 2
assert {record["installed_service_count"] for record in artifact.results} == {2}
assert all(record["component"] for record in artifact.results)
def test_only_the_unlisted_user_gets_a_count_record(self):
artifact = _parse(TWO_USERS)
listed = [record for record in artifact.results if record["component"]]
unlisted = [record for record in artifact.results if not record["component"]]
assert [record["user_id"] for record in listed] == [0]
assert [record["user_id"] for record in unlisted] == [95]
assert unlisted[0]["installed_service_count"] == 3
def test_a_zero_count_adds_nothing(self):
# "Zero installed" is a negative result the empty section already
# states; a record for it would be noise.
assert _parse(ZERO_COUNT).results == []
def test_a_partly_named_count_reports_the_unnamed_rest(self):
# The Android 14 fixture states `installedServiceCount=2` and names one
# enabled component. The listing is incomplete, and must not read as
# complete.
artifact = DumpsysAccessibilityArtifact()
artifact.results = []
with open(
get_artifact("android_data/dumpsys_accessibility_v14_or_later.txt")
) as handle:
artifact.parse(handle.read())
unlisted = [record for record in artifact.results if not record["component"]]
assert len(unlisted) == 1
assert unlisted[0]["installed_service_count"] == 2
assert unlisted[0]["unnamed_service_count"] == 1
artifact.check_indicators()
low = [
alert
for alert in artifact.alertstore.alerts
if alert.level == AlertLevel.LOW
]
assert len(low) == 1
assert "only 1 of them (1 unnamed)" in low[0].message
def test_the_unnamed_rest_is_counted_per_user(self):
# User 0 names one of three, user 10 names its only one: the gap
# belongs to user 0 alone.
artifact = _parse(PARTIAL_TWO_USERS)
unlisted = [record for record in artifact.results if not record["component"]]
assert [
(record["user_id"], record["unnamed_service_count"]) for record in unlisted
] == [(0, 2)]
def test_a_fully_named_count_adds_nothing(self):
artifact = _parse(ONE_UI_WITH_LIST)
assert all(
record["unnamed_service_count"] is None for record in artifact.results
)
@@ -0,0 +1,155 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
"""Installed is not the same as enabled, and the dump says which.
Two things this guards:
* a section the dump never printed must read as None ("not stated"), not as
False ("not enabled");
* the alert must name the state, instead of firing identically on a device
where nothing is switched on and one where something is bound.
"""
from types import SimpleNamespace
from mvt.android.artifacts.dumpsys_accessibility import DumpsysAccessibilityArtifact
from mvt.common.alerts import AlertLevel
from ..utils import get_artifact
NO_STATE_SECTIONS = """\
ACCESSIBILITY MANAGER (dumpsys accessibility)
User state[attributes:{id=0, currentUser=true}
installed services: {
0 : com.example.app/com.example.app.Service
}
"""
ENABLED_BLOCK = """\
ACCESSIBILITY MANAGER (dumpsys accessibility)
User state[attributes:{id=0, currentUser=true}
installed services: {
0 : com.example.app/com.example.app.Service
1 : com.other.app/.Helper
}
enabled services: {
0 : com.other.app/.Helper
}
bound services:{
0 : com.other.app/.Helper
}
"""
# User 0 prints only `installed services`, user 10 only `Enabled services`.
# Neither section speaks for the other user.
TWO_USERS_DIFFERENT_SECTIONS = """\
ACCESSIBILITY MANAGER (dumpsys accessibility)
User state[attributes:{id=0, currentUser=true}
installed services: {
0 : com.example.app/com.example.app.Service
}
User state[attributes:{id=10, currentUser=false}
Enabled services:{{com.other.app/.Helper}}
"""
class _IndicatorsMatching:
"""Minimal stand-in: matches one package id, like the STIX2 loader would."""
def __init__(self, package_name: str) -> None:
self.package_name = package_name
def check_app_id(self, app_id):
if app_id != self.package_name:
return None
return SimpleNamespace(
message=f"Found a known suspicious app: {app_id}", ioc={"value": app_id}
)
class TestAccessibilityServiceState:
def _parse(self, content):
artifact = DumpsysAccessibilityArtifact()
artifact.results = []
artifact.parse(content)
return {r["component"]: r for r in artifact.results}
def test_absent_sections_leave_the_state_unknown(self):
# None, not False: a build that does not print the sections says
# nothing about what is enabled, and that must not read as "nothing".
state = self._parse(NO_STATE_SECTIONS)[
"com.example.app/com.example.app.Service"
]
assert state["installed"] is True
assert state["enabled"] is None
assert state["bound"] is None
def test_enabled_and_bound_are_attributed_per_service(self):
results = self._parse(ENABLED_BLOCK)
installed_only = results["com.example.app/com.example.app.Service"]
active = results["com.other.app/.Helper"]
assert (installed_only["enabled"], installed_only["bound"]) == (False, False)
assert (active["enabled"], active["bound"]) == (True, True)
def test_alert_message_carries_the_state(self):
artifact = DumpsysAccessibilityArtifact()
artifact.results = []
with open(get_artifact("android_data/dumpsys_accessibility.txt")) as handle:
artifact.parse(handle.read())
artifact.check_indicators()
assert artifact.alertstore.alerts
assert all("installed" in alert.message for alert in artifact.alertstore.alerts)
def test_a_switched_off_service_is_low_and_a_running_one_medium(self):
# A service the dump says is OFF still reaches the analyst, but must not
# compete with one that is actually bound. "Not stated" is not "off".
artifact = DumpsysAccessibilityArtifact()
artifact.results = []
artifact.parse(ENABLED_BLOCK)
artifact.check_indicators()
by_level = {}
for alert in artifact.alertstore.alerts:
by_level.setdefault(alert.level, []).append(alert.message)
assert artifact.alertstore.count(AlertLevel.LOW) == 1
assert artifact.alertstore.count(AlertLevel.MEDIUM) == 1
assert "com.example.app" in by_level[AlertLevel.LOW][0]
assert "com.other.app" in by_level[AlertLevel.MEDIUM][0]
def test_an_unstated_enabled_state_stays_medium(self):
artifact = DumpsysAccessibilityArtifact()
artifact.results = []
artifact.parse(NO_STATE_SECTIONS)
artifact.check_indicators()
assert artifact.alertstore.count(AlertLevel.MEDIUM) == 1
assert artifact.alertstore.count(AlertLevel.LOW) == 0
def test_a_disabled_service_is_still_matched_against_indicators(self):
# The state decides the severity of an ordinary finding, never whether
# the package is compared with the IOC feeds.
artifact = DumpsysAccessibilityArtifact()
artifact.results = []
artifact.parse(ENABLED_BLOCK)
artifact.indicators = _IndicatorsMatching("com.example.app")
artifact.check_indicators()
assert artifact.alertstore.count(AlertLevel.CRITICAL) == 1
assert artifact.alertstore.count(AlertLevel.LOW) == 0
def test_printed_sections_are_tracked_per_user(self):
artifact = DumpsysAccessibilityArtifact()
artifact.results = []
artifact.parse(TWO_USERS_DIFFERENT_SECTIONS)
by_user = {r["user_id"]: r for r in artifact.results}
# User 0's enabled state is not stated, so it is unknown, not off.
assert (by_user[0]["installed"], by_user[0]["enabled"]) == (True, None)
# User 10's installed state is not stated either.
assert (by_user[10]["installed"], by_user[10]["enabled"]) == (None, True)
artifact.check_indicators()
assert artifact.alertstore.count(AlertLevel.LOW) == 0
assert artifact.alertstore.count(AlertLevel.MEDIUM) == 2
assert not any(
"installed, not enabled" in alert.message
for alert in artifact.alertstore.alerts
)
+174
View File
@@ -130,6 +130,180 @@ class TestDumpsysADBArtifact:
assert key_store_entry["fingerprint"] == expected_fingerprint
assert key_store_entry["last_connected"] == "1628501829898"
def test_parsing_adb_xml_with_crlf_line_endings(self):
da_adb = DumpsysADBArtifact()
file = get_artifact("android_data/dumpsys_adb_xml.txt")
with open(file, "rb") as f:
data = f.read().replace(b"\r\n", b"\n").replace(b"\n", b"\r\n")
da_adb.parse(data)
assert len(da_adb.results) == 1
assert da_adb.results[0]["user_keys"][0]["user"] == "user@laptop"
assert da_adb.results[0]["keystore"][0]["last_connected"] == "1628501829898"
def test_parsing_adb_wifi_with_mixed_line_endings(self):
da_adb = DumpsysADBArtifact()
data = (
b"ADB MANAGER STATE (dumpsys adb):\n"
b"{\n"
b" debugging_manager={\n"
b" connected_to_adb=true\n"
b" user_keys=QUJDRA== host@example\n"
b" adb_wifi={\n"
b" enabled=false\n"
b" }\n"
b" }\r\n"
b"}\n"
b"--------- duration\n"
)
da_adb.parse(data)
assert len(da_adb.results) == 1
assert da_adb.results[0]["user_keys"][0]["user"] == "host@example"
assert da_adb.results[0]["adb_wifi"]["enabled"] == b"false"
ADB_STATE = (
b"ADB MANAGER STATE (dumpsys adb):\n"
b"{\n"
b" debugging_manager={\n"
b" connected_to_adb=true\n"
b" user_keys=QUJDRA== host@example\n"
b" }\n"
b"}\n"
b"--------- 0.5s was the duration of 'dumpsys adb'\n"
)
def test_a_later_dumpsys_section_does_not_extend_the_adb_state(self):
# A bug report holds many sections. Looking for the last closing brace
# in the whole output pulled a later section into this one, which threw
# IndexError out of parse() and lost the ADB records entirely.
da_adb = DumpsysADBArtifact()
da_adb.parse(
self.ADB_STATE + b"DUMP OF SERVICE other:\n"
b" debugging_manager={\n"
b" connected_to_adb=false\n"
b" user_keys=RVZJTA== attacker@host\n"
b" }\n"
b"}\n"
)
assert len(da_adb.results) == 1
assert [key["user"] for key in da_adb.results[0]["user_keys"]] == [
"host@example"
]
assert da_adb.results[0]["connected_to_adb"] is True
def test_braces_in_key_comment_are_not_state_delimiters(self):
for user in (b"host{example", b"host}}example"):
da_adb = DumpsysADBArtifact()
da_adb.parse(
b"ADB MANAGER STATE (dumpsys adb):\n"
b"{\n"
b" debugging_manager={\n"
b" connected_to_adb=true\n"
b" user_keys=QUJDRA== " + user + b"\n }\n}\n"
)
assert len(da_adb.results) == 1
assert da_adb.results[0]["user_keys"][0]["user"] == user.decode()
def test_state_without_keys_is_parsed_without_error(self, caplog):
# Shape printed by every device with no stored ADB keys (Samsung
# SM-A525F, Android 12). The block's closing brace is dedented from
# the last entry and must close exactly one level.
da_adb = DumpsysADBArtifact()
da_adb.parse(
b"ADB MANAGER STATE (dumpsys adb):\n"
b"{\n"
b" debugging_manager={\n"
b" connected_to_adb=false\n"
b" }\n"
b"}\n"
b"--------- 0.015s was the duration of dumpsys adb\n"
)
assert len(da_adb.results) == 1
assert da_adb.results[0]["connected_to_adb"] is False
assert not [r for r in caplog.records if r.levelname == "ERROR"]
def test_nested_block_as_last_entry_is_parsed_without_error(self, caplog):
# Shape printed by Pixel devices with no stored ADB keys.
da_adb = DumpsysADBArtifact()
da_adb.parse(
b"ADB MANAGER STATE (dumpsys adb):\n"
b"{\n"
b" debugging_manager={\n"
b" connected_to_adb=false\n"
b" adb_wifi={\n"
b" enabled=false\n"
b" tls_port=0\n"
b" }\n"
b" }\n"
b"}\n"
)
assert len(da_adb.results) == 1
assert da_adb.results[0]["connected_to_adb"] is False
assert da_adb.results[0]["adb_wifi"]["tls_port"] == b"0"
assert not [r for r in caplog.records if r.levelname == "ERROR"]
def test_entry_after_nested_block_stays_in_debugging_manager(self):
da_adb = DumpsysADBArtifact()
da_adb.parse(
b"ADB MANAGER STATE (dumpsys adb):\n"
b"{\n"
b" debugging_manager={\n"
b" adb_wifi={\n"
b" enabled=false\n"
b" }\n"
b" connected_to_adb=true\n"
b" }\n"
b"}\n"
)
assert len(da_adb.results) == 1
assert da_adb.results[0]["connected_to_adb"] is True
def test_blank_line_after_single_line_value_does_not_raise(self):
# `system_keys` is the content of /adb_keys and ends with a newline,
# so a blank line follows it. It is not a multiline field, so the
# blank line is not a multiline terminator either.
da_adb = DumpsysADBArtifact()
da_adb.parse(
b"ADB MANAGER STATE (dumpsys adb):\n"
b"{\n"
b" debugging_manager={\n"
b" connected_to_adb=true\n"
b" user_keys=QUJDRA== host@example\n"
b"\n"
b" system_keys=RUZHSA== system@example\n"
b"\n"
b" }\n"
b"}\n"
)
assert len(da_adb.results) == 1
assert da_adb.results[0]["user_keys"][0]["user"] == "host@example"
assert da_adb.results[0]["system_keys"] == "RUZHSA== system@example"
def test_unbalanced_state_is_reported_rather_than_raising(self):
da_adb = DumpsysADBArtifact()
da_adb.parse(
b"ADB MANAGER STATE (dumpsys adb):\n"
b"{\n"
b" debugging_manager={\n"
b" connected_to_adb=true\n"
b" }\n"
b" }\n"
b"}\n"
)
# No exception, and whatever was read before the bad line is kept.
assert len(da_adb.results) == 1
assert da_adb.results[0]["connected_to_adb"] is True
class TestDumpsysADBStateAlerts:
def test_no_androidqf_context_preserves_existing_behavior(self):
@@ -144,3 +144,18 @@ class TestDumpsysBatteryDailyArtifact:
"Detected uninstall of package com.example.app (vers 0)"
)
assert uninstall_alert.event_time == "2026-01-10"
def test_malformed_update_line_does_not_lose_later_records(self):
# A truncated "Update" line, or one from a vendor that omits "vers=",
# used to raise out of parse() and lose every record after it.
dba = DumpsysBatteryDailyArtifact()
dba.parse(
" Daily from 2021-05-10-08-00-00 to 2021-05-11-08-00-00:\n"
" Update com.first vers=1\n"
" Update com.truncated\n"
" Update com.no.equals vers 2\n"
" Update com.last vers=3\n"
)
assert [r["package_name"] for r in dba.results] == ["com.first", "com.last"]
assert [r["version_code"] for r in dba.results] == [1, 3]
+43
View File
@@ -0,0 +1,43 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
"""A mount is writable only if both mountinfo option layers allow it."""
from mvt.android.artifacts.mounts import Mounts
from mvt.common.alerts import AlertLevel
# fields[5] is the per-mount (VFS) layer, the field after "-" the superblock.
RO_OVERLAY = (
"304 166 0:130 / /product/usr rw,relatime shared:63 - overlay overlay "
"ro,seclabel,lowerdir=/mnt/vendor/ext/product/usr:/product/usr"
)
RW_SYSTEM = (
"305 166 0:131 / /system rw,relatime shared:64 - ext4 /dev/block/dm-1 "
"rw,seclabel,errors=panic"
)
def _run(line):
mounts = Mounts()
mounts.results = Mounts.parse_mountinfo(line, 1)
mounts.check_indicators()
return mounts
class TestMountsReadWriteLayers:
def test_rw_vfs_over_ro_superblock_is_not_writable(self):
mounts = _run(RO_OVERLAY)
assert mounts.results[0]["is_read_write"] is False
assert mounts.alertstore.count(AlertLevel.HIGH) == 0
assert mounts.alertstore.count(AlertLevel.MEDIUM) == 0
def test_rw_in_both_layers_still_alerts(self):
mounts = _run(RW_SYSTEM)
assert mounts.results[0]["is_read_write"] is True
assert mounts.alertstore.count(AlertLevel.HIGH) == 1
def test_both_option_layers_are_still_reported(self):
mounts = _run(RO_OVERLAY)
options = mounts.results[0]["options_list"]
assert "rw" in options and "ro" in options
+213
View File
@@ -0,0 +1,213 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from typing import Optional
from mvt.android.artifacts.settings import Settings
from ..utils import get_artifact
def parse_bugreport_settings() -> Settings:
settings = Settings()
with open(get_artifact("android_data/bugreport/dumpstate.txt")) as handle:
data = handle.read()
settings.parse(settings.extract_dumpsys_section(data, "DUMP OF SERVICE settings:"))
return settings
def find(settings: Settings, name: str) -> list:
return [result for result in settings.results if result["name"] == name]
def resolve_history_time(time: str, section_end: str) -> Optional[str]:
settings = Settings()
settings.parse(
"SECURE SETTINGS (user 0)\n"
"_id:240 name:accessibility_enabled pkg:android value:1\n"
"\tHistory (accessibility_enabled)\n"
f"\t\ttime:{time} mode:update oldValue:0 newValue:1 "
"package:com.example.helper\n"
"\n"
"--------- 0.019s was the duration of dumpsys settings, "
f"ending at: {section_end}\n"
)
return settings.results[0]["history"][0]["timestamp"]
class TestSettingsArtifact:
def test_parsing(self):
settings = parse_bugreport_settings()
assert len(settings.results) == 12
assert {result["namespace"] for result in settings.results} == {
"config",
"global",
"secure",
}
assert settings.results[0] == {
"namespace": "config",
"user": "0",
"_id": "682",
"name": "namespace_one/blocked_components",
"pkg": "com.example.services",
"value": (
"com.android.settings,com.android.vending,\n"
"com.example.dialer,\n"
"com.example.camera"
),
"default": (
"com.android.settings,\n"
" com.android.vending,\n"
" com.example.dialer"
),
"defaultSystemSet": "false",
"history": [],
}
def test_multiline_values_are_kept_whole(self):
settings = parse_bugreport_settings()
assert find(settings, "namespace_one/allowed_packages")[0]["value"] == (
"com.example.messaging,\ncom.example.chat"
)
assert find(settings, "widget_instance_data")[0]["value"] == (
'{\n "version": 1,\n "data": [\n {\n "number": 10000,\n'
' "package_name": "com.example.widget"\n }\n ]\n}'
)
def test_trailing_default_is_not_part_of_the_value(self):
settings = parse_bugreport_settings()
record = find(settings, "namespace_one/streaming_blocked_components")[0]
assert record["value"] == "com.example.dialer,com.example.camera"
assert record["default"] == "com.android.settings,\n com.android.vending"
def test_trailing_metadata_is_not_part_of_the_value(self):
settings = parse_bugreport_settings()
record = find(settings, "lock_screen_show_notifications")[0]
assert record["value"] == "1"
assert record["defaultSystemSet"] == "true"
assert record["isValuePreservedInRestore"] == "true"
# Without a default, the tag or the restore token follows the value.
record = find(settings, "accessibility_enabled")[1]
assert record["value"] == "0"
assert record["tag"] == "null"
assert "default" not in record
record = find(settings, "send_action_app_error")[0]
assert record["value"] == "1"
assert record["isValuePreservedInRestore"] == "false"
def test_dumps_after_the_last_block_are_not_part_of_the_last_row(self):
settings = parse_bugreport_settings()
assert settings.results[-1] == {
"namespace": "secure",
"user": "10",
"_id": "311",
"name": "accessibility_enabled",
"pkg": "android",
"value": "0",
"tag": "null",
"history": [],
}
def test_repeated_names_are_kept_as_separate_records(self):
settings = parse_bugreport_settings()
widgets = find(settings, "widget_instance_data")
assert [record["_id"] for record in widgets] == ["771", "41654"]
accessibility = find(settings, "accessibility_enabled")
assert [(record["user"], record["value"]) for record in accessibility] == [
("0", "1"),
("10", "0"),
]
def test_setting_without_recording_package(self):
settings = parse_bugreport_settings()
record = find(settings, "hidden_api_blacklist_exemptions")[0]
assert "pkg" not in record
assert record["value"] == "{null}"
def test_history_timestamps_resolved_against_section_end(self):
settings = parse_bugreport_settings()
# The section was dumped on 2022-03-29, so an 11-02 entry belongs to
# the previous year and an 03-14 entry to the same year.
assert find(settings, "development_settings_enabled")[0]["history"] == [
{
"timestamp": "2021-11-02 11:21:22.212000",
"oldValue": "null",
"newValue": "1",
"pkg": "com.android.settings",
},
{
"timestamp": "2022-03-14 09:02:11.100000",
"oldValue": "1",
"newValue": "0",
"pkg": "com.example.updater",
},
]
def test_history_without_a_section_end_has_no_timestamp(self):
settings = Settings()
settings.parse(
"SECURE SETTINGS (user 0)\n"
"_id:240 name:accessibility_enabled pkg:android value:1\n"
"\tHistory (accessibility_enabled)\n"
"\t\ttime:03-28 22:41:07.980 mode:update oldValue:0 newValue:1 "
"package:com.example.helper\n"
)
assert settings.results[0]["history"] == [
{
"timestamp": None,
"oldValue": "0",
"newValue": "1",
"pkg": "com.example.helper",
}
]
def test_leap_day_history_keeps_its_timestamp(self):
assert (
resolve_history_time("02-29 22:41:07.980", "2024-03-05 23:14:28")
== "2024-02-29 22:41:07.980000"
)
def test_leap_day_history_resolved_to_the_last_leap_year(self):
# The most recent 29 February at or before the dump can be several
# years back when the dump was taken in a year without one.
assert (
resolve_history_time("02-29 22:41:07.980", "2025-03-01 10:00:00")
== "2024-02-29 22:41:07.980000"
)
assert (
resolve_history_time("02-29 22:41:07.980", "2024-01-10 10:00:00")
== "2020-02-29 22:41:07.980000"
)
def test_leap_day_history_before_a_leap_day_after_a_skipped_century(self):
# 2100 is not a leap year, so before 29 February 2104 the most
# recent matching date is eight years earlier, in 2096.
assert (
resolve_history_time("02-29 22:41:07.980", "2104-01-10 10:00:00")
== "2096-02-29 22:41:07.980000"
)
def test_dangerous_setting_is_detected_with_the_changing_package(self):
settings = parse_bugreport_settings()
settings.check_indicators()
assert len(settings.alertstore.alerts) == 1
alert = settings.alertstore.alerts[0]
assert "accessibility_enabled = 1" in alert.message
assert alert.event_time == "2022-03-28 22:41:07.980000"
assert alert.event["history"][0]["pkg"] == "com.example.helper"
+261
View File
@@ -0,0 +1,261 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import json
import sqlite3
import zipfile
from pathlib import Path
import pytest
from mvt.android.modules.androidqf.browser_history import (
BrowserHistory as AndroidQFBrowserHistory,
)
from mvt.android.modules.fs.browser_history import BrowserHistory as FSBrowserHistory
from mvt.android.cmd_check_fs import CmdAndroidCheckFS
CHROME_TIME = 13_348_540_800_000_000
URL = "https://example.org/path"
def create_history_database(path: Path, *, url: str = URL) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
with sqlite3.connect(path) as connection:
connection.executescript(
"""
CREATE TABLE urls (
id INTEGER PRIMARY KEY,
url TEXT,
title TEXT,
visit_count INTEGER,
typed_count INTEGER
);
CREATE TABLE visits (
id INTEGER PRIMARY KEY,
url INTEGER,
visit_time INTEGER,
from_visit INTEGER,
transition INTEGER
);
"""
)
connection.execute("INSERT INTO urls VALUES (1, ?, 'Example', 1, 0)", (url,))
connection.execute(
"INSERT INTO visits VALUES (7, 1, ?, 0, 805306368)",
(CHROME_TIME,),
)
def manifest(database_paths: list[tuple[str, str]]) -> dict:
return {
"schema_version": 1,
"status": "collected",
"databases": [
{
"browser": "Chrome",
"package": package,
"profile": "Default",
"device_path": f"/data/data/{package}/app_chrome/Default/History",
"archive_path": archive_path,
"sidecars": [],
}
for package, archive_path in database_paths
],
}
@pytest.mark.parametrize("use_zip", [False, True])
def test_androidqf_browser_history_directory_and_zip(tmp_path, use_zip):
source_database = tmp_path / "source" / "History"
create_history_database(source_database)
archive_path = "browser_history/com.android.chrome/Default/History"
manifest_data = manifest([("com.android.chrome", archive_path)])
module = AndroidQFBrowserHistory()
if use_zip:
acquisition_path = tmp_path / "acquisition.zip"
with zipfile.ZipFile(acquisition_path, "w") as archive:
archive.write(source_database, archive_path)
archive.writestr("browser_history/manifest.json", json.dumps(manifest_data))
with zipfile.ZipFile(acquisition_path) as archive:
module.from_zip(archive, archive.namelist())
module.run()
else:
acquisition = tmp_path / "acquisition"
database_path = acquisition / archive_path
database_path.parent.mkdir(parents=True)
database_path.write_bytes(source_database.read_bytes())
manifest_path = acquisition / "browser_history" / "manifest.json"
manifest_path.write_text(json.dumps(manifest_data))
files = [
path.relative_to(tmp_path).as_posix()
for path in acquisition.rglob("*")
if path.is_file()
]
module.from_dir(str(tmp_path), files)
module.run()
assert len(module.results) == 1
assert module.results[0]["url"] == URL
assert module.results[0]["browser"] == "Chrome"
assert module.results[0]["source_path"].endswith("/History")
module.collect_url_results()
module.to_timeline()
assert module.url_results[0]["url"] == URL
assert module.timeline[0]["event"] == "browser_history"
@pytest.mark.parametrize("prefix", ["", "./", "acquisition/", "./acquisition/"])
@pytest.mark.parametrize("separator", ["/", "\\"])
def test_androidqf_zip_paths_preserve_wal_visits(tmp_path, prefix, separator):
database = tmp_path / "History"
create_history_database(database)
connection = sqlite3.connect(database)
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA wal_checkpoint(TRUNCATE)")
connection.execute(
"INSERT INTO urls VALUES (2, ?, 'WAL', 1, 0)", (URL + "/wal",)
)
connection.execute(
"INSERT INTO visits VALUES (8, 2, ?, 0, 805306368)", (CHROME_TIME,)
)
connection.commit()
archive_path = "browser_history/com.android.chrome/Default/History"
acquisition = tmp_path / "acquisition.zip"
try:
with zipfile.ZipFile(acquisition, "w") as archive:
for suffix in ("", "-wal", "-shm"):
archive.writestr(
(prefix + archive_path + suffix).replace("/", separator),
Path(f"{database}{suffix}").read_bytes(),
)
archive.writestr(
(prefix + "browser_history/manifest.json").replace("/", separator),
json.dumps(manifest([("com.android.chrome", archive_path)])),
)
module = AndroidQFBrowserHistory()
with zipfile.ZipFile(acquisition) as archive:
module.from_zip(archive, archive.namelist())
module.run()
finally:
connection.close()
assert {result["url"] for result in module.results} == {URL, URL + "/wal"}
def test_androidqf_browser_history_isolates_malformed_database(tmp_path, caplog):
acquisition = tmp_path / "acquisition"
good_archive_path = "browser_history/com.android.chrome/Default/History"
bad_archive_path = "browser_history/com.brave.browser/Default/History"
create_history_database(acquisition / good_archive_path)
bad_path = acquisition / bad_archive_path
bad_path.parent.mkdir(parents=True)
bad_path.write_bytes(b"not sqlite")
manifest_path = acquisition / "browser_history" / "manifest.json"
manifest_path.write_text(
json.dumps(
manifest(
[
("com.brave.browser", bad_archive_path),
("com.android.chrome", good_archive_path),
]
)
)
)
module = AndroidQFBrowserHistory()
files = [
path.relative_to(tmp_path).as_posix()
for path in acquisition.rglob("*")
if path.is_file()
]
module.from_dir(str(tmp_path), files)
module.run()
assert [result["url"] for result in module.results] == [URL]
assert "Unable to parse browser history database" in caplog.text
def test_androidqf_browser_history_rejects_unsafe_manifest_path(tmp_path, caplog):
acquisition = tmp_path / "acquisition"
manifest_path = acquisition / "browser_history" / "manifest.json"
manifest_path.parent.mkdir(parents=True)
manifest_path.write_text(
json.dumps(manifest([("com.android.chrome", "browser_history/../secret")]))
)
module = AndroidQFBrowserHistory()
module.from_dir(str(tmp_path), [manifest_path.relative_to(tmp_path).as_posix()])
module.run()
assert module.results == []
assert "unsafe browser history archive path" in caplog.text
def test_filesystem_browser_history_reads_wal_only_visit(tmp_path):
database_path = tmp_path / "History"
connection = sqlite3.connect(database_path)
connection.execute("PRAGMA journal_mode=WAL")
connection.executescript(
"""
CREATE TABLE urls (
id INTEGER PRIMARY KEY,
url TEXT,
title TEXT,
visit_count INTEGER,
typed_count INTEGER
);
CREATE TABLE visits (
id INTEGER PRIMARY KEY,
url INTEGER,
visit_time INTEGER,
from_visit INTEGER,
transition INTEGER
);
"""
)
connection.commit()
connection.execute("PRAGMA wal_checkpoint(TRUNCATE)")
connection.execute("INSERT INTO urls VALUES (1, ?, 'WAL', 1, 0)", (URL,))
connection.execute(
"INSERT INTO visits VALUES (8, 1, ?, 0, 805306368)", (CHROME_TIME,)
)
connection.commit()
try:
assert Path(f"{database_path}-wal").stat().st_size > 0
module = FSBrowserHistory(target_path=str(database_path))
module.run()
finally:
connection.close()
assert [result["url"] for result in module.results] == [URL]
assert module.results[0]["browser"] == "Chromium"
def test_android_check_fs_finds_all_supported_browser_paths(tmp_path):
for index, relative_path in enumerate(
(
"data/data/com.android.chrome/app_chrome/Default/History",
"data/data/com.brave.browser/app_chrome/Default/History",
"data/data/com.microsoft.emmx/app_chrome/Default/History",
"data/data/com.sec.android.app.sbrowser/app_sbrowser/Default/History",
)
):
create_history_database(
tmp_path / relative_path, url=f"https://example.org/{index}"
)
command = CmdAndroidCheckFS(target_path=str(tmp_path), module_name="BrowserHistory")
command.run()
assert len(command.executed) == 1
assert {result["package"] for result in command.executed[0].results} == {
"com.android.chrome",
"com.brave.browser",
"com.microsoft.emmx",
"com.sec.android.app.sbrowser",
}
assert len(command.url_results) == 4
assert len(command.timeline) == 4
@@ -0,0 +1,42 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
"""A section ends at the next section, not at a timing line printed inside it."""
from mvt.android.modules.bugreport.base import BugReportModule
# dumpstate prints a section's duration when that section finishes, which can
# land in the middle of the section currently being written.
DUMPSTATE = """\
------ SYSTEM PROPERTIES (getprop) ------
[nfc.initialized]: [true]
------ 0.101s was the duration of 'DROPBOX SYSTEM SERVER CRASHES' ------
[ro.build.version.sdk]: [30]
[ro.product.model]: [SM-A305F]
------ 0.064s was the duration of 'SYSTEM PROPERTIES' ------
------ STORAGE INFO (df) ------
/dev/root 2.9G
"""
class TestExtractCommandSection:
def test_a_foreign_timing_line_does_not_end_the_section(self):
section = BugReportModule.extract_command_section(
DUMPSTATE, "------ SYSTEM PROPERTIES"
)
assert "[ro.product.model]: [SM-A305F]" in section
assert section.count("\n") == 2
def test_the_next_section_is_still_the_boundary(self):
section = BugReportModule.extract_command_section(
DUMPSTATE, "------ SYSTEM PROPERTIES"
)
assert "STORAGE INFO" not in section
assert "/dev/root" not in section
def test_timing_lines_are_not_returned_as_content(self):
section = BugReportModule.extract_command_section(
DUMPSTATE, "------ SYSTEM PROPERTIES"
)
assert "was the duration of" not in section
@@ -0,0 +1,56 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
"""An OEM wrapper archive must not read as an empty bug report.
MIUI / HyperOS hands out a zip of app logs with the real
`bugreport-<device>-<timestamp>.zip` nested inside. The outer archive has
none of the entry points the modules read, so every module
reported it found nothing and the command still exited 0 — an empty analysis
that looks like a finished one.
"""
import io
import zipfile
from mvt.android.cmd_check_bugreport import CmdAndroidCheckBugreport
DUMPSTATE = "== dumpstate: 2026-01-01 00:00:00\nDUMP OF SERVICE package:\n"
def _inner_zip() -> bytes:
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, "w") as inner:
inner.writestr("main_entry.txt", "bugreport-test-2026-01-01-00-00-00.txt")
inner.writestr("bugreport-test-2026-01-01-00-00-00.txt", DUMPSTATE)
return buffer.getvalue()
def _wrapper_zip() -> zipfile.ZipFile:
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, "w") as outer:
outer.writestr("app_logs/hilog.txt", "unrelated OEM log\n")
outer.writestr("bugreport-test-2026-01-01-00-00-00.zip", _inner_zip())
return zipfile.ZipFile(io.BytesIO(buffer.getvalue()))
class TestCheckBugreportWrapper:
def test_nested_bugreport_is_used(self, tmp_path):
cmd = CmdAndroidCheckBugreport(results_path=str(tmp_path))
cmd.from_zip(_wrapper_zip())
module = cmd.modules[0](results_path=str(tmp_path))
cmd.module_init(module)
assert "main_entry.txt" in module.zip_files
def test_plain_bugreport_is_left_alone(self, tmp_path):
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, "w") as archive:
archive.writestr("main_entry.txt", "bugreport.txt")
archive.writestr("bugreport.txt", DUMPSTATE)
archive.writestr("attachments/extra.zip", _inner_zip())
cmd = CmdAndroidCheckBugreport(results_path=str(tmp_path))
cmd.from_zip(zipfile.ZipFile(io.BytesIO(buffer.getvalue())))
module = cmd.modules[0](results_path=str(tmp_path))
cmd.module_init(module)
assert "attachments/extra.zip" in module.zip_files
+8 -17
View File
@@ -6,27 +6,12 @@
from pathlib import Path
from mvt.android.modules.androidqf.aqf_settings import AQFSettings
from mvt.android.artifacts.settings import Settings
from mvt.common.module import run_module
from ..utils import get_android_androidqf, list_files
class TestSettingsModule:
def test_bugreport_settings_format(self):
settings = Settings()
settings.parse(
"GLOBAL SETTINGS (user 0)\n"
"_id:1 name:adb_wifi_enabled pkg:android value:0 default:0 defaultSystemSet:true\n"
"SECURE SETTINGS (user 10)\n"
"_id:2 name:accessibility_enabled pkg:android value:1\n"
)
assert settings.results == {
"global:user_0": {"adb_wifi_enabled": "0"},
"secure:user_10": {"accessibility_enabled": "1"},
}
def test_parsing(self):
data_path = get_android_androidqf()
m = AQFSettings(target_path=data_path)
@@ -34,7 +19,13 @@ class TestSettingsModule:
parent_path = Path(data_path).absolute().parent.as_posix()
m.from_dir(parent_path, files)
run_module(m)
assert len(m.results) == 1
assert "random" in m.results.keys()
assert len(m.results) == 9
assert {result["namespace"] for result in m.results} == {"random"}
assert m.results[0] == {
"namespace": "random",
"user": None,
"name": "samsung_errorlog_agree",
"value": "0",
}
assert len(m.alertstore.alerts) == 1
assert "samsung_errorlog_agree" in m.alertstore.alerts[0].message
+20
View File
@@ -10,6 +10,7 @@ from mvt.android.modules.bugreport.dumpsys_appops import DumpsysAppops
from mvt.android.modules.bugreport.dumpsys_getprop import DumpsysGetProp
from mvt.android.modules.bugreport.dumpsys_packages import DumpsysPackages
from mvt.android.modules.bugreport.dumpsys_receivers import DumpsysReceivers
from mvt.android.modules.bugreport.settings import Settings
from mvt.android.modules.bugreport.tombstones import Tombstones
from mvt.common.module import run_module
@@ -93,6 +94,25 @@ class TestBugreportAnalysis:
assert alert.event == malicious_receiver
assert alert.matched_indicator.value == "com.android.services"
def test_settings_module(self):
m = self.launch_bug_report_module(Settings)
assert len(m.results) == 12
assert len(m.alertstore.alerts) == 1
assert "accessibility_enabled = 1" in m.alertstore.alerts[0].message
assert len(m.timeline) == 3
change = [
entry
for entry in m.timeline
if entry["timestamp"] == "2022-03-28 22:41:07.980000"
][0]
assert change["event"] == "settings_change"
assert change["data"] == (
'secure setting "accessibility_enabled" changed from "0" to "1" '
"by com.example.helper"
)
def test_tombstones_modules(self):
m = self.launch_bug_report_module(Tombstones)
assert len(m.results) == 2
@@ -264,5 +264,53 @@ ChangeId(143539591; name=SELINUX_LATEST_CHANGES; disabled)
ChangeId(247079863; name=DISALLOW_INVALID_GROUP_REFERENCE; enableSinceTargetSdk=34)
ChangeId(174227820; name=FORCE_DISABLE_HEVC_SUPPORT; disabled)
ChangeId(168419799; name=DOWNSCALED; disabled; packageOverrides={com.google.android.apps.tachyon=false, org.torproject.torbrowser=false}; rawOverrides={org.torproject.torbrowser=false, org.article19.circulo.next=false}; overridable)
-------------------------------------------------------------------------------
DUMP OF SERVICE settings:
CONFIG SETTINGS (user 0)
_id:682 name:namespace_one/blocked_components pkg:com.example.services value:com.android.settings,com.android.vending,
com.example.dialer,
com.example.camera default:com.android.settings,
com.android.vending,
com.example.dialer defaultSystemSet:false
_id:684 name:namespace_one/streaming_blocked_components pkg:com.example.services value:com.example.dialer,com.example.camera default:com.android.settings,
com.android.vending defaultSystemSet:false
_id:680 name:namespace_one/allowed_packages pkg:com.example.services value:com.example.messaging,
com.example.chat
GLOBAL SETTINGS (user 0)
_id:2070 name:adb_wifi_enabled pkg:android value:0 default:0 defaultSystemSet:true
_id:778 name:hidden_api_blacklist_exemptions value:{null}
_id:9640 name:send_action_app_error pkg:android value:1 notPreservedInRestore
_id:9631 name:development_settings_enabled pkg:com.android.settings value:1 default:1 defaultSystemSet:true
History (development_settings_enabled)
time:11-02 11:21:22.212 mode:update oldValue:null newValue:1 package:com.android.settings
time:03-14 09:02:11.100 mode:update oldValue:1 newValue:0 package:com.example.updater
_id:771 name:widget_instance_data pkg:com.android.systemui value:{
"version": 1,
"data": [
{
"number": 10000,
"package_name": "com.example.widget"
}
]
} defaultSystemSet:true
_id:41654 name:widget_instance_data pkg:com.android.systemui value:{
"version": 3,
"data": []
} defaultSystemSet:true
SECURE SETTINGS (user 0)
_id:907 name:lock_screen_show_notifications pkg:com.android.settings value:1 default:1 defaultSystemSet:true isValuePreservedInRestore:true
_id:240 name:accessibility_enabled pkg:android value:1 default:0 defaultSystemSet:true
History (accessibility_enabled)
time:03-28 22:41:07.980 mode:update oldValue:0 newValue:1 package:com.example.helper
SECURE SETTINGS (user 10)
_id:311 name:accessibility_enabled pkg:android value:0 tag:null
GENERATION REGISTRY
Maximum number of backing stores:8
Number of backing stores:1
_Backing store for type:SETTINGS_SECURE user:10 size:1024 cachedEntries:1
--------- 0.019s was the duration of dumpsys settings, ending at: 2022-03-29 23:14:28
+2 -1
View File
@@ -1,3 +1,4 @@
import shlex
from types import SimpleNamespace
import click
@@ -110,7 +111,7 @@ def test_load_command_option_supports_folders_and_repeated_paths(tmp_path):
def test_loaded_command_participates_in_shell_completion(tmp_path):
command_path = _write_command(tmp_path / "hello.py", "hello")
group = _make_group()
words = f"group --load-command {command_path} he"
words = f"group --load-command {shlex.quote(str(command_path))} he"
result = CliRunner().invoke(
group,
+12 -8
View File
@@ -1,4 +1,5 @@
import json
import re
from types import SimpleNamespace
import pytest
@@ -54,20 +55,23 @@ def _run(command, arguments):
return CliRunner().invoke(command, arguments, env={"COLUMNS": "200"})
def _table_rows(output):
"""Return the content of the table rows, without the header and the box."""
def _table_lines(output):
"""Return the cells of each line of the table, header first."""
return [
[cell.strip() for cell in line.strip().strip("│").split("│")]
[cell.strip() for cell in re.split("[│┃]", line.strip().strip("│┃"))]
for line in output.splitlines()
if "│" in line
if "│" in line or "┃" in line
]
def _table_rows(output):
"""Return the content of the table rows, without the header and the box."""
return _table_lines(output)[1:]
def _table_header(output):
for line in output.splitlines():
if "┃" in line:
return [cell.strip() for cell in line.strip().strip("┃").split("┃")]
return []
lines = _table_lines(output)
return lines[0] if lines else []
def _install(monkeypatch, distributions, entry_points):
+85
View File
@@ -5,8 +5,13 @@
import json
import logging
from unittest.mock import patch
import pytest
from click.testing import CliRunner
from mvt.common.command import Command
from mvt.common.indicators import Indicators
from mvt.common.module import MVTModule
@@ -197,6 +202,86 @@ class RecordingCommand(Command):
class TestCommand:
def test_listing_modules_does_not_load_indicators(self):
with patch("mvt.common.command.Indicators.load_indicators_files") as load:
cmd = RecordingCommand()
cmd.list_modules()
load.assert_not_called()
def test_output_folder_is_created_by_a_run_not_by_listing(self, tmp_path):
output_path = tmp_path / "out"
with patch("mvt.common.command.Indicators.load_indicators_files"):
cmd = RecordingCommand(results_path=str(output_path))
cmd.list_modules()
assert not output_path.exists()
cmd.run()
assert (output_path / "command.log").is_file()
def test_indicators_load_once_and_are_shared(self, indicator_file, monkeypatch):
from mvt.common.config import settings
monkeypatch.setattr(settings, "STIX2", "")
monkeypatch.setattr(Indicators, "_load_downloaded_indicators", lambda self: None)
original = Indicators.load_indicators_files
with patch.object(
Indicators, "load_indicators_files", autospec=True, side_effect=original
) as load:
cmd = RecordingCommand(ioc_files=[indicator_file])
load.assert_not_called()
indicators = cmd.iocs
assert indicators.total_ioc_count == 9
assert len(indicators.ioc_collections) == 1
assert cmd.iocs is indicators
child = RecordingCommand(iocs=indicators)
assert child.iocs is indicators
load.assert_called_once_with(indicators, [indicator_file])
@pytest.mark.parametrize("assign", [False, True])
def test_supplied_empty_indicators_are_not_loaded(self, assign):
indicators = Indicators(logging.getLogger(__name__))
with patch.object(Indicators, "load_indicators_files") as load:
cmd = RecordingCommand(iocs=None if assign else indicators)
if assign:
cmd.iocs = indicators
assert cmd.iocs is indicators
assert cmd.iocs.total_ioc_count == 0
load.assert_not_called()
@pytest.mark.parametrize("list_modules", [False, True])
def test_backup_cli_does_not_load_indicators_before_analysis(
self, tmp_path, list_modules
):
from mvt.ios.cli import check_backup
args = [str(tmp_path)]
if list_modules:
args.insert(0, "--list-modules")
with patch.object(Indicators, "load_indicators_files") as load:
result = CliRunner().invoke(check_backup, args)
assert result.exit_code == (0 if list_modules else 1)
load.assert_not_called()
def test_run_checks_synthetic_indicators(self, indicator_file, monkeypatch):
from mvt.common.config import settings
monkeypatch.setattr(settings, "STIX2", "")
monkeypatch.setattr(Indicators, "_load_downloaded_indicators", lambda self: None)
class MatchingModule(RecordingModule):
def run(self):
self.results = ["https://example.org/test"]
def check_indicators(self):
self.detected = [
url for url in self.results if self.indicators.check_domain(url)
]
cmd = RecordingCommand(ioc_files=[indicator_file])
cmd.modules = [MatchingModule]
cmd.run()
assert cmd.executed[0].detected == ["https://example.org/test"]
assert cmd.executed[0].indicators is cmd.iocs
def setup_method(self):
RecordingModule.run_order = []
+4 -1
View File
@@ -12,13 +12,16 @@ from mvt.ios.command_modules import IOS_CHECK_IOCS_MODULES
from mvt.ios.modules.backup import BACKUP_MODULES as IOS_BACKUP_MODULES
from mvt.ios.modules.fs import FS_MODULES
from mvt.ios.modules.mixed import MIXED_MODULES
from mvt.ios.modules.sysdiagnose import SYSDIAGNOSE_MODULES
def test_the_check_iocs_lists_are_the_families_of_their_platform():
# The CLI reads these same lists, so nothing composing one elsewhere can
# drift from what the command runs. This pins what the lists are composed
# of.
assert IOS_CHECK_IOCS_MODULES == IOS_BACKUP_MODULES + FS_MODULES + MIXED_MODULES
assert IOS_CHECK_IOCS_MODULES == (
IOS_BACKUP_MODULES + FS_MODULES + MIXED_MODULES + SYSDIAGNOSE_MODULES
)
assert ANDROID_CHECK_IOCS_MODULES == (
ANDROID_BACKUP_MODULES
+ BUGREPORT_MODULES
+30
View File
@@ -0,0 +1,30 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import os
import yaml
from mvt.common import config
from mvt.common.config import MVTSettings
def test_env_variables_are_not_persisted_to_config_file(tmp_path, monkeypatch):
config_path = tmp_path / "config.yaml"
monkeypatch.setattr(config, "MVT_CONFIG_FOLDER", str(tmp_path))
monkeypatch.setattr(config, "MVT_CONFIG_PATH", str(config_path))
monkeypatch.setenv("MVT_NETWORK_ACCESS_ALLOWED", "false")
monkeypatch.setenv("MVT_IOS_BACKUP_PASSWORD", "env-only-password")
settings = MVTSettings.initialise()
assert os.path.isfile(config_path)
saved = yaml.safe_load(config_path.read_text()) or {}
assert "NETWORK_ACCESS_ALLOWED" not in saved
assert "IOS_BACKUP_PASSWORD" not in saved
# The environment must still apply to the settings in use.
assert settings.NETWORK_ACCESS_ALLOWED is False
assert settings.IOS_BACKUP_PASSWORD == "env-only-password"
+28
View File
@@ -311,3 +311,31 @@ class TestIndicators:
ind = Indicators(log=logging)
ind.load_indicators_files([], load_default=False)
assert ind.total_ioc_count == 9
def test_check_url_matches_w_prefixed_subdomain(self, tmp_path):
import json
stix_file = tmp_path / "w-domain.stix2"
stix_file.write_text(
json.dumps(
{
"objects": [
{
"type": "indicator",
"pattern": "[domain-name:value = 'web.evil.com']",
}
]
}
),
encoding="utf-8",
)
ind = Indicators(log=logging)
ind.load_indicators_files([str(stix_file)], load_default=False)
for url in (
"https://web.evil.com/path",
"https://www.web.evil.com/path",
):
match = ind.check_url(url)
assert match is not None
assert match.ioc.value == "web.evil.com"
+20
View File
@@ -22,3 +22,23 @@ def test_google_maps_url_is_not_shortened(url):
def test_other_google_short_url_is_shortened():
assert URL("https://goo.gl/example").check_if_shortened() is True
@pytest.mark.parametrize(
"url, domain",
[
("https://www.example.com/path", "example.com"),
# Only the whole "www." prefix comes off, not any leading "w" or "." character.
("https://web.example.com", "web.example.com"),
("https://wow.com", "wow.com"),
("https://wired.com", "wired.com"),
("https://www.wow.com", "wow.com"),
],
)
def test_get_domain_strips_only_a_whole_www_prefix(url, domain):
assert URL(url).domain == domain
def test_shortener_starting_with_w_is_detected():
assert URL("https://w3t.org/example").check_if_shortened() is True
assert URL("https://www.w3t.org/example").check_if_shortened() is True
+20 -7
View File
@@ -3,20 +3,26 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import atexit
import logging
import os
import shutil
import tempfile
import pytest
from mvt.common.cli_plugins import (
MVT_ANDROID_CUSTOM_COMMANDS_ENV,
MVT_CUSTOM_COMMANDS_ENV,
MVT_IOS_CUSTOM_COMMANDS_ENV,
)
from mvt.common.indicators import Indicators
from .artifacts.generate_stix import generate_test_stix_file
# The suite must neither read nor write the developer's own MVT settings,
# downloaded indicators or update-check state, and mvt.common.config saves
# the settings file as soon as it is imported. Both folders are redirected
# before any mvt module is imported, which is why this file imports none at
# the top; the subprocesses the tests start inherit the variables.
MVT_TEST_HOME = tempfile.mkdtemp(prefix="mvt-tests-")
atexit.register(shutil.rmtree, MVT_TEST_HOME, ignore_errors=True)
os.environ["MVT_CONFIG_FOLDER"] = os.path.join(MVT_TEST_HOME, "config")
os.environ["MVT_DATA_FOLDER"] = os.path.join(MVT_TEST_HOME, "data")
@pytest.fixture(scope="session", autouse=True)
def indicator_file(request, tmp_path_factory):
@@ -47,6 +53,8 @@ def indicators_factory(indicator_file):
android_property_names=[],
files_sha256=[],
):
from mvt.common.indicators import Indicators
ind = Indicators(log=logging.getLogger())
ind.parse_stix2(indicator_file)
@@ -77,6 +85,11 @@ def restore_cli_commands(monkeypatch):
"""
from mvt.android.cli import cli as android_cli
from mvt.cli import cli as neutral_cli
from mvt.common.cli_plugins import (
MVT_ANDROID_CUSTOM_COMMANDS_ENV,
MVT_CUSTOM_COMMANDS_ENV,
MVT_IOS_CUSTOM_COMMANDS_ENV,
)
from mvt.ios.cli import cli as ios_cli
groups = (neutral_cli, ios_cli, android_cli)
+203
View File
@@ -0,0 +1,203 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
import threading
from pathlib import Path
import pytest
from Crypto.Cipher import AES
from mvt.ios.decrypt import DecryptBackup, MVTEncryptedBackup
def _encrypted_file(backup_path, file_id, key, plaintext):
padding_length = AES.block_size - (len(plaintext) % AES.block_size)
padded = plaintext + bytes([padding_length]) * padding_length
encrypted = AES.new(key, AES.MODE_CBC, iv=b"\x00" * AES.block_size).encrypt(
padded
)
source_path = backup_path / file_id[:2] / file_id
source_path.parent.mkdir(parents=True)
source_path.write_bytes(encrypted)
def test_extract_file_by_id_preserves_bytes_with_wrong_manifest_size(
mocker, tmp_path
):
file_id = "ab" + "1" * 38
plaintext = b"complete decrypted content"
inner_key = b"k" * 32
_encrypted_file(tmp_path, file_id, inner_key, plaintext)
file_plist = mocker.Mock(
encryption_key=b"wrapped-key",
protection_class=1,
filesize=1,
mtime=None,
)
mocker.patch("mvt.ios.decrypt.FilePlist", return_value=file_plist)
backup = MVTEncryptedBackup(
backup_directory=str(tmp_path), derived_key=b"d" * 32
)
mocker.patch.object(backup, "_read_and_unlock_keybag", return_value=True)
backup._keybag = mocker.Mock()
backup._keybag.unwrapKeyForClass.return_value = inner_key
streaming_decrypt = mocker.spy(backup, "_decrypt_file_to_disk")
output_path = tmp_path / "output"
backup.extract_file_by_id(
file_id=file_id,
file_bplist=b"plist",
output_filename=str(output_path),
)
assert output_path.read_bytes() == plaintext
streaming_decrypt.assert_called_once()
def test_extract_file_by_id_copies_unencrypted_files(mocker, tmp_path):
file_id = "cd" + "2" * 38
source_path = tmp_path / file_id[:2] / file_id
source_path.parent.mkdir(parents=True)
source_path.write_bytes(b"plain content")
file_plist = mocker.Mock(encryption_key=None)
mocker.patch("mvt.ios.decrypt.FilePlist", return_value=file_plist)
backup = MVTEncryptedBackup(
backup_directory=str(tmp_path), derived_key=b"d" * 32
)
mocker.patch.object(backup, "_read_and_unlock_keybag", return_value=True)
output_path = tmp_path / "output"
backup.extract_file_by_id(
file_id=file_id,
file_bplist=b"plist",
output_filename=str(output_path),
)
assert output_path.read_bytes() == b"plain content"
@pytest.mark.parametrize("with_symlink", [False, True], ids=["file-id", "symlink"])
def test_process_backup_rejects_unsafe_file_ids_and_destinations(
mocker, tmp_path, with_symlink
):
backup_path = tmp_path / "backup"
destination = tmp_path / "destination"
outside = tmp_path / "outside"
backup_path.mkdir()
destination.mkdir()
outside.mkdir()
safe_file_id = "ef" + "3" * 38
unsafe_file_id = "../../outside-file"
symlink_file_id = "ab" + "4" * 38
file_ids = [safe_file_id]
if with_symlink:
file_ids.append(symlink_file_id)
for file_id in file_ids:
source_path = backup_path / file_id[:2] / file_id
source_path.parent.mkdir(parents=True, exist_ok=True)
source_path.write_bytes(b"encrypted")
if with_symlink:
try:
(destination / "ab").symlink_to(outside, target_is_directory=True)
except OSError:
pytest.skip("creating symbolic links is not permitted on this system")
cursor = mocker.MagicMock()
records = [
(safe_file_id, "Domain", "safe", b"plist"),
(unsafe_file_id, "Domain", "unsafe", b"plist"),
]
if with_symlink:
records.append((symlink_file_id, "Domain", "symlink", b"plist"))
cursor.__iter__.return_value = iter(records)
cursor_context = mocker.MagicMock()
cursor_context.__enter__.return_value = cursor
backup = mocker.MagicMock()
backup.manifest_db_cursor.return_value = cursor_context
def extract_file_by_id(*, output_filename, **kwargs):
Path(output_filename).write_bytes(b"decrypted")
backup.extract_file_by_id.side_effect = extract_file_by_id
decryptor = DecryptBackup(
str(backup_path), str(destination), max_workers=1
)
decryptor._backup = backup
decryptor.process_backup()
assert (destination / safe_file_id[:2] / safe_file_id).read_bytes() == b"decrypted"
if with_symlink:
assert not (outside / symlink_file_id).exists()
backup.extract_file_by_id.assert_called_once()
assert backup.extract_file_by_id.call_args.kwargs["file_id"] == safe_file_id
def test_process_backup_decrypts_files_concurrently(mocker, tmp_path):
backup_path = tmp_path / "backup"
destination = tmp_path / "destination"
backup_path.mkdir()
file_ids = ["ab" + "1" * 38, "cd" + "2" * 38]
for file_id in file_ids:
source_path = backup_path / file_id[:2] / file_id
source_path.parent.mkdir()
source_path.write_bytes(b"encrypted")
cursor = mocker.MagicMock()
cursor.__iter__.return_value = iter(
(file_id, "Domain", file_id, b"plist") for file_id in file_ids
)
cursor_context = mocker.MagicMock()
cursor_context.__enter__.return_value = cursor
barrier = threading.Barrier(2)
backup = mocker.MagicMock()
backup.manifest_db_cursor.return_value = cursor_context
def extract_file_by_id(*, file_id, output_filename, **kwargs):
barrier.wait(timeout=5)
Path(output_filename).write_bytes(file_id.encode())
backup.extract_file_by_id.side_effect = extract_file_by_id
decryptor = DecryptBackup(str(backup_path), str(destination), max_workers=2)
decryptor._backup = backup
decryptor.process_backup()
for file_id in file_ids:
assert (destination / file_id[:2] / file_id).read_bytes() == file_id.encode()
def test_process_backup_logs_worker_errors(mocker, tmp_path, caplog):
backup_path = tmp_path / "backup"
destination = tmp_path / "destination"
backup_path.mkdir()
file_id = "ef" + "3" * 38
source_path = backup_path / file_id[:2] / file_id
source_path.parent.mkdir()
source_path.write_bytes(b"encrypted")
cursor = mocker.MagicMock()
cursor.__iter__.return_value = iter([(file_id, "Domain", "failing-file", b"plist")])
cursor_context = mocker.MagicMock()
cursor_context.__enter__.return_value = cursor
backup = mocker.MagicMock()
backup.manifest_db_cursor.return_value = cursor_context
backup.extract_file_by_id.side_effect = ValueError("broken file")
decryptor = DecryptBackup(str(backup_path), str(destination))
decryptor._backup = backup
with caplog.at_level(logging.ERROR, logger="mvt.ios.decrypt"):
decryptor.process_backup()
assert "Failed to decrypt file failing-file: broken file" in caplog.text
+152
View File
@@ -5,7 +5,12 @@
import gc
import logging
import os
import shutil
import warnings
from pathlib import Path
import pytest
from mvt.common.indicators import Indicators
from mvt.common.module import run_module
@@ -14,8 +19,58 @@ from mvt.ios.modules.backup.manifest import Manifest
from ..utils import get_ios_backup_folder
# fileID of HomeDomain::Library/SMS/sms.db in the test backup. It is one of the
# few files the test backup actually stores.
SMS_FILE_ID = "3d0d7e5fb2ce288813306e4d4636395e047a3d28"
@pytest.fixture
def backup_without_stored_files(tmp_path):
"""A copy of the test backup with every stored file removed.
The test backup only ships a handful of the files its manifest lists, so
dropping what it does store leaves a backup where every regular file the
manifest mentions is missing from the folder.
"""
backup_path = tmp_path / "backup"
shutil.copytree(get_ios_backup_folder(), backup_path)
for file_id_folder in backup_path.iterdir():
if not file_id_folder.is_dir():
continue
for backup_file in file_id_folder.iterdir():
backup_file.unlink()
return str(backup_path)
class TestIOSExtraction:
@pytest.mark.parametrize("link_directory", [False, True], ids=["file", "directory"])
@pytest.mark.parametrize("inside_backup", [False, True], ids=["outside", "inside"])
def test_stored_file_inventory_matches_symlink_resolution(
self, tmp_path, link_directory, inside_backup
):
backup_path = tmp_path / "backup"
backup_path.mkdir()
destination = (backup_path if inside_backup else tmp_path) / "contents"
destination.mkdir()
(destination / SMS_FILE_ID).write_bytes(b"backup file")
bucket = backup_path / SMS_FILE_ID[:2]
try:
if link_directory:
bucket.symlink_to(destination, target_is_directory=True)
else:
bucket.mkdir()
(bucket / SMS_FILE_ID).symlink_to(destination / SMS_FILE_ID)
except OSError:
pytest.skip("creating symbolic links is not permitted on this system")
m = IOSExtraction(target_path=str(backup_path))
assert bool(m._get_backup_file_from_id(SMS_FILE_ID)) is inside_backup
assert m._get_stored_backup_file_ids() == (
{SMS_FILE_ID} if inside_backup else set()
)
def test_get_backup_files_from_manifest_closes_connection(self):
m = IOSExtraction(target_path=get_ios_backup_folder())
@@ -41,6 +96,103 @@ class TestManifestModule:
assert len(m.timeline) == 5881
assert len(m.alertstore.alerts) == 0
def test_manifest_flags_the_files_missing_from_an_incomplete_backup(self):
m = Manifest(target_path=get_ios_backup_folder())
run_module(m)
missing = [result for result in m.results if result.get("missing")]
# The test backup only stores a handful of the files its manifest
# lists, and every one of the rest is a regular file (flags 1).
assert len(missing) == 1079
assert all(result["flags"] == 1 for result in missing)
stored = [result for result in m.results if result["file_id"] == SMS_FILE_ID]
assert len(stored) == 1
assert "missing" not in stored[0]
def test_manifest_flags_every_stored_file_removed_from_the_backup_folder(
self, backup_without_stored_files
):
m = Manifest(target_path=backup_without_stored_files)
run_module(m)
missing = [result for result in m.results if result.get("missing")]
assert len(missing) == 1089
def test_manifest_flags_a_file_removed_from_the_backup_folder(self, tmp_path):
backup_path = tmp_path / "backup"
shutil.copytree(get_ios_backup_folder(), backup_path)
(backup_path / SMS_FILE_ID[:2] / SMS_FILE_ID).unlink()
m = Manifest(target_path=str(backup_path))
run_module(m)
removed = [result for result in m.results if result["file_id"] == SMS_FILE_ID]
assert len(removed) == 1
assert removed[0]["missing"] is True
@pytest.mark.parametrize("failed_folder", ["", "3d"], ids=["root", "bucket"])
def test_manifest_skips_missing_check_when_inventory_fails(
self, monkeypatch, caplog, failed_folder
):
backup_path = Path(get_ios_backup_folder())
failed_path = backup_path / failed_folder
scandir = os.scandir
def fail_listing(path):
if Path(path) == failed_path:
raise PermissionError("cannot list backup folder")
return scandir(path)
monkeypatch.setattr(os, "scandir", fail_listing)
m = Manifest(target_path=str(backup_path))
with caplog.at_level(logging.INFO):
m.run()
assert len(m.results) == 3721
assert m._get_backup_file_from_id(SMS_FILE_ID) is not None
assert all("missing" not in result for result in m.results)
assert "Skipping the missing-file check" in caplog.text
assert "The backup might be incomplete" not in caplog.text
def test_manifest_ignores_unreadable_unrelated_folders(self, tmp_path, monkeypatch):
backup_path = tmp_path / "backup"
shutil.copytree(get_ios_backup_folder(), backup_path)
unrelated = backup_path / "notes"
unrelated.mkdir()
scandir = os.scandir
def fail_listing(path):
if Path(path) == unrelated:
raise PermissionError("cannot list unrelated folder")
return scandir(path)
monkeypatch.setattr(os, "scandir", fail_listing)
m = Manifest(target_path=str(backup_path))
m.run()
assert sum(bool(result.get("missing")) for result in m.results) == 1079
stored = next(
result for result in m.results if result["file_id"] == SMS_FILE_ID
)
assert "missing" not in stored
@pytest.mark.parametrize("wrong_folder", ["ab", "notes"])
def test_manifest_flags_files_in_the_wrong_folder(self, tmp_path, wrong_folder):
backup_path = tmp_path / "backup"
shutil.copytree(get_ios_backup_folder(), backup_path)
destination = backup_path / wrong_folder
destination.mkdir(exist_ok=True)
(backup_path / SMS_FILE_ID[:2] / SMS_FILE_ID).rename(destination / SMS_FILE_ID)
m = Manifest(target_path=str(backup_path))
m.run()
assert m._get_backup_file_from_id(SMS_FILE_ID) is None
moved = next(result for result in m.results if result["file_id"] == SMS_FILE_ID)
assert moved["missing"] is True
assert sum(bool(result.get("missing")) for result in m.results) == 1080
def test_detection(self, indicator_file):
m = Manifest(target_path=get_ios_backup_folder())
ind = Indicators(log=logging.getLogger())
+5
View File
@@ -9,6 +9,8 @@ import plistlib
import shutil
import sqlite3
import pytest
from mvt.ios.modules.base import IOSExtraction
from mvt.ios.modules.fs.analytics import Analytics
@@ -45,6 +47,9 @@ def test_open_sqlite_reads_wal_without_modifying_evidence(tmp_path):
assert not os.path.exists(str(evidence_path) + "-shm")
@pytest.mark.skipif(
shutil.which("sqlite3") is None, reason="the recovery needs the sqlite3 binary"
)
def test_recovery_preserves_source_database(tmp_path):
database_path = tmp_path / "source.db"
conn = sqlite3.connect(database_path)
+30
View File
@@ -3,15 +3,45 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
import ntpath
from types import SimpleNamespace
from mvt.common.indicators import Indicators
from mvt.common.module import run_module
from mvt.ios.modules.fs import filesystem
from mvt.ios.modules.fs.filesystem import Filesystem
from ..utils import get_ios_backup_folder
class TestFilesystem:
def test_windows_paths_are_normalized_for_indicators(
self, monkeypatch, indicators_factory
):
m = Filesystem(target_path=r"C:\dump")
m.indicators = indicators_factory(
file_paths=["matched/directory"], processes=["matched"]
)
monkeypatch.setattr(
filesystem,
"os",
SimpleNamespace(
sep="\\",
path=ntpath,
walk=lambda _: [(r"C:\dump\matched", ["directory"], ["file.txt"])],
stat=lambda _: SimpleNamespace(st_mtime=0),
),
)
m.run()
m.check_indicators()
assert {result["path"] for result in m.results} == {
"matched/directory",
"matched/file.txt",
}
assert len(m.alertstore.alerts) == 3
def test_filesystem(self):
m = Filesystem(target_path=get_ios_backup_folder())
run_module(m)
+4
View File
@@ -0,0 +1,4 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
@@ -0,0 +1,161 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import json
import plistlib
import sqlite3
import tarfile
from mvt.common.module import run_module
from mvt.ios.cmd_check_sysdiagnose import CmdIOSCheckSysdiagnose
from mvt.ios.modules.sysdiagnose.sysdiagnose_info import SysdiagnoseInfo
from mvt.ios.versions import get_device_desc_from_id
# The name sysdiagnose gives its archive: the time it ran, then the OS and build.
ARCHIVE_NAME = "sysdiagnose_2024.01.02_03-04-05+0200_iPhone-OS_iPhone_21C62"
DUMPSTATE = (
"Found device: ...\n"
"\tProperties: {\n"
"\t\tProductType => iPhone12,1\n"
"\t\tOSVersion => 17.2\n"
"\t\tSerialNumber => C0FFEE000000\n"
"\t\tRegionCode => LL\n"
"\t}\n"
"\tServices: {\n"
"\t\tcom.apple.example => ignored\n"
"\t}\n"
)
ACTIVATION_BODY = {
"serial-number": "C0FFEE000000",
"productType": "iPhone12,1",
"productName": "iPhone OS",
"imei": "000000000000000",
"os-version": "17.2",
"os-build": "21C62",
"udid": "00000000-0000000000000000",
"meid": "00000000000000",
}
def make_sysdiagnose(tmp_path, activation_body=None):
folder = tmp_path / ARCHIVE_NAME
folder.mkdir()
(folder / "sysdiagnose.log").write_text(
f"Output available at '/private/var/tmp/{ARCHIVE_NAME}.tar.gz'\n",
encoding="utf-8",
)
(folder / "remotectl_dumpstate.txt").write_text(DUMPSTATE, encoding="utf-8")
activation = folder / "logs" / "MobileActivation"
activation.mkdir(parents=True)
body = json.dumps(
activation_body if activation_body is not None else ACTIVATION_BODY
)
(activation / "collection_oob_request.txt").write_text(
f"HEADERS: {{}}\nBODY: {body}\nEND\n", encoding="utf-8"
)
appinstallation = folder / "logs" / "appinstallation"
appinstallation.mkdir(parents=True)
conn = sqlite3.connect(appinstallation / "appstored.sqlitedb")
conn.execute("CREATE TABLE asset (sinfs_data BLOB)")
conn.execute(
"INSERT INTO asset VALUES (?)",
(plistlib.dumps([{"sinf": b"\x00\x10nameExample Person\x00\x00rest"}]),),
)
conn.execute("CREATE TABLE job_software (store_account_name TEXT)")
conn.execute("INSERT INTO job_software VALUES (NULL)")
conn.execute("INSERT INTO job_software VALUES ('person@example.com')")
conn.commit()
conn.close()
return folder
def run_command(target, results_path=None):
command = CmdIOSCheckSysdiagnose(target_path=str(target), results_path=results_path)
command.run()
(module,) = [m for m in command.executed if isinstance(m, SysdiagnoseInfo)]
return module
def test_device_details_from_a_sysdiagnose_folder(tmp_path):
results_path = tmp_path / "results"
results_path.mkdir()
module = run_command(make_sysdiagnose(tmp_path), str(results_path))
assert module.results["SerialNumber"] == "C0FFEE000000"
assert module.results["ProductType"] == "iPhone12,1"
assert module.results["ProductName"] == get_device_desc_from_id("iPhone12,1")
assert module.results["ProductName"] != "iPhone OS"
assert module.results["OSVersion"] == "17.2"
assert module.results["BuildVersion"] == "21C62"
assert module.results["UniqueIdentifier"] == "00000000-0000000000000000"
assert module.results["RegionCode"] == "LL"
assert "com.apple.example" not in module.results
assert module.results["Account Name"] == "Example Person"
assert module.results["Email Address"] == "person@example.com"
assert module.results["OriginalFilename"] == f"{ARCHIVE_NAME}.tar.gz"
assert module.results["CreatedTimestamp"] == "2024-01-02 01:04:05.000000"
assert (results_path / "sysdiagnose_info.json").exists()
def test_device_details_from_a_sysdiagnose_archive(tmp_path):
folder = make_sysdiagnose(tmp_path)
archive_path = tmp_path / f"{ARCHIVE_NAME}.tar.gz"
with tarfile.open(archive_path, "w:gz") as archive:
archive.add(folder, arcname=ARCHIVE_NAME)
module = run_command(archive_path)
assert module.results["SerialNumber"] == "C0FFEE000000"
assert module.results["Account Name"] == "Example Person"
assert module.results["OriginalFilename"] == f"{ARCHIVE_NAME}.tar.gz"
def test_wal_sidecars_are_copied_beside_the_database(tmp_path):
folder = tmp_path / ARCHIVE_NAME
(folder / "logs").mkdir(parents=True)
(folder / "logs" / "db.sqlite").write_bytes(b"main")
(folder / "logs" / "db.sqlite-wal").write_bytes(b"wal")
module = SysdiagnoseInfo()
module.from_sysdiagnose_folder(
str(folder),
[f"{ARCHIVE_NAME}/logs/db.sqlite", f"{ARCHIVE_NAME}/logs/db.sqlite-wal"],
)
copies = tmp_path / "copies"
copies.mkdir()
db_path = module._copy_sqlite_db(f"{ARCHIVE_NAME}/logs/db.sqlite", str(copies))
assert db_path == str(copies / "db.sqlite")
assert (copies / "db.sqlite").read_bytes() == b"main"
assert (copies / "db.sqlite-wal").read_bytes() == b"wal"
assert not (copies / "db.sqlite-shm").exists()
def test_a_sysdiagnose_without_the_files_yields_nothing(tmp_path):
folder = tmp_path / ARCHIVE_NAME
folder.mkdir()
(folder / "other.txt").write_text("nothing here", encoding="utf-8")
module = SysdiagnoseInfo()
module.from_sysdiagnose_folder(str(folder), [f"{ARCHIVE_NAME}/other.txt"])
run_module(module)
assert module.results == {}
def test_a_malformed_activation_request_is_skipped(tmp_path):
folder = make_sysdiagnose(tmp_path)
(folder / "logs" / "MobileActivation" / "collection_oob_request.txt").write_text(
"BODY: {not json}\n", encoding="utf-8"
)
module = run_command(folder)
assert "IMEI" not in module.results
assert module.results["SerialNumber"] == "C0FFEE000000"
+1 -1
View File
@@ -85,7 +85,7 @@ class TestCheckAndroidqfCommand:
def test_acquisition_context_falls_back_to_public_key_file(self, tmp_path):
data_path = tmp_path / "androidqf"
data_path.mkdir()
(data_path / "adb_host_key.pub").write_text("QUJDRA== acquisition@host\n")
(data_path / "adb_host_key.pub").write_bytes(b"QUJDRA== acquisition@host\n")
command = CmdAndroidCheckAndroidQF(target_path=str(data_path))
command.init()
+92
View File
@@ -3,11 +3,16 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import datetime
import logging
import os
import shutil
import zipfile
from click.testing import CliRunner
from mvt.android.cli import check_bugreport
from mvt.android.cmd_check_bugreport import CmdAndroidCheckBugreport
from .utils import get_artifact_folder
@@ -28,3 +33,90 @@ class TestCheckBugreportCommand:
assert result.exit_code == 1
assert "Invalid bugreport archive" in result.output
assert "Traceback" not in result.output
PROPERTIES = (
"------ SYSTEM PROPERTIES (getprop) ------\n"
"[persist.sys.timezone]: [Africa/Nairobi]\n"
"------ 0.01s was the duration of 'SYSTEM PROPERTIES' ------\n"
)
TOMBSTONE = "android_data/bugreport/FS/data/tombstones/tombstone_00"
def _bugreport_zip(tmp_path, dumpstate=PROPERTIES):
"""A bugreport zip holding one tombstone written at 11:38:10 device time.
An even second: zip entry times have a two-second resolution.
"""
path = tmp_path / "bugreport.zip"
with open(os.path.join(get_artifact_folder(), TOMBSTONE), "rb") as handle:
tombstone = handle.read()
with zipfile.ZipFile(path, "w") as archive:
archive.writestr("main_entry.txt", "dumpstate.txt")
archive.writestr("dumpstate.txt", dumpstate)
entry = zipfile.ZipInfo(
"FS/data/tombstones/tombstone_00", date_time=(2023, 3, 10, 11, 38, 10)
)
archive.writestr(entry, tombstone)
return str(path)
def _tombstone_timestamp(target, **options):
cmd = CmdAndroidCheckBugreport(
target_path=target,
module_name="Tombstones",
disable_version_check=True,
disable_indicator_check=True,
**options,
)
cmd.run()
return cmd, cmd.executed[0].results[0]["file_timestamp"]
class TestCheckBugreportTimezone:
def test_zip_entry_times_are_read_in_the_device_timezone(self, tmp_path):
cmd, file_timestamp = _tombstone_timestamp(_bugreport_zip(tmp_path))
assert cmd.module_options["device_timezone"] == "Africa/Nairobi"
# 11:38:10 in Nairobi is 08:38:10 UTC.
assert file_timestamp == "2023-03-10 08:38:10.000000"
def test_timezone_option_wins_over_the_bugreport(self, tmp_path):
_, file_timestamp = _tombstone_timestamp(
_bugreport_zip(tmp_path), module_options={"device_timezone": "Europe/Paris"}
)
assert file_timestamp == "2023-03-10 10:38:10.000000"
result = CliRunner().invoke(
check_bugreport,
["-t", "Europe/Paris", "-m", "Tombstones", _bugreport_zip(tmp_path)],
)
assert result.exit_code == 0, result.output
def test_without_a_timezone_the_wall_clock_is_kept_and_a_warning_given(
self, tmp_path, caplog
):
with caplog.at_level(logging.WARNING, logger="mvt"):
_, file_timestamp = _tombstone_timestamp(_bugreport_zip(tmp_path, ""))
assert file_timestamp == "2023-03-10 11:38:10.000000"
assert "persist.sys.timezone not found" in caplog.text
def test_unpacked_bugreport_warns_and_reads_mtimes_as_utc(self, tmp_path, caplog):
unpacked = tmp_path / "bugreport"
shutil.copytree(
os.path.join(get_artifact_folder(), "android_data/bugreport"), unpacked
)
instant = datetime.datetime(
2023, 3, 10, 8, 38, 11, tzinfo=datetime.timezone.utc
).timestamp()
for name in ("tombstone_00", "tombstone_01"):
os.utime(unpacked / "FS" / "data" / "tombstones" / name, (instant, instant))
with caplog.at_level(logging.WARNING, logger="mvt"):
_, file_timestamp = _tombstone_timestamp(str(unpacked))
assert "unpacked bugreport" in caplog.text
# Whatever the zone of the machine running the analysis.
assert file_timestamp == "2023-03-10 08:38:11.000000"
+8
View File
@@ -19,6 +19,14 @@ class TestCheckBackupCommand:
result = runner.invoke(check_backup, [path])
assert result.exit_code == 0
def test_check_logs_the_backup_path_to_the_command_log(self, tmp_path):
path = get_ios_backup_folder()
output_path = tmp_path / "out"
result = CliRunner().invoke(check_backup, ["--output", str(output_path), path])
assert result.exit_code == 0
command_log = (output_path / "command.log").read_text(encoding="utf-8")
assert f"Checking iTunes backup located at: {path}" in command_log
def test_check_finds_backup_in_subfolder(self, tmp_path, caplog):
runner = CliRunner()
backup_path = tmp_path / "MobileSync" / "Backup" / "device-id"
+39 -4
View File
@@ -1,3 +1,7 @@
import logging
import os
import tarfile
from click.testing import CliRunner
from mvt.ios.cli import check_sysdiagnose
@@ -50,8 +54,39 @@ def test_check_sysdiagnose_runs_explicitly_scoped_custom_module(tmp_path):
assert (output_path / "custom_sysdiagnose_module.json").exists()
def test_check_sysdiagnose_requires_an_explicitly_scoped_module(tmp_path):
result = CliRunner().invoke(check_sysdiagnose, [str(_create_sysdiagnose_folder(tmp_path))])
def test_check_sysdiagnose_warns_without_a_custom_module(tmp_path, caplog):
# The built-in SysdiagnoseInfo alone performs no check, so the run goes
# ahead but says so.
with caplog.at_level(logging.WARNING, logger="mvt"):
result = CliRunner().invoke(
check_sysdiagnose, [str(_create_sysdiagnose_folder(tmp_path))]
)
assert result.exit_code != 0
assert "No custom modules support mvt-ios check-sysdiagnose" in result.output
assert result.exit_code == 0
assert "No forensic sysdiagnose modules have been loaded" in caplog.text
def _create_truncated_sysdiagnose_archive(tmp_path):
folder = tmp_path / "sysdiagnose_2026.01.01_00-00-00+0000_iPhone-OS_iPhone_23A000"
folder.mkdir()
(folder / "sysdiagnose.log").write_bytes(os.urandom(200_000))
archive = tmp_path / "sysdiagnose.tar.gz"
with tarfile.open(archive, "w:gz") as tar:
tar.add(folder, arcname=folder.name)
data = archive.read_bytes()
archive.write_bytes(data[: len(data) // 2])
return archive
def test_check_sysdiagnose_reports_a_truncated_archive(tmp_path, caplog):
# A download that stopped halfway ends in EOFError from gzip, which Click
# would otherwise turn into a bare "Aborted!" with no reason given.
archive = _create_truncated_sysdiagnose_archive(tmp_path)
with caplog.at_level(logging.CRITICAL, logger="mvt"):
result = CliRunner().invoke(check_sysdiagnose, [str(archive)])
assert result.exit_code == 1
assert "Unable to read the sysdiagnose archive" in caplog.text
assert "truncated" in caplog.text
assert "Aborted!" not in result.output
+14 -8
View File
@@ -34,6 +34,7 @@ def _create_sysdiagnose_folder(tmp_path):
"sysdiagnose_2024.01.02_03-04-05+0200.tar.gz", encoding="utf-8"
)
(folder / "report.ips").write_text('{"bug_type": 210}\nbody', encoding="utf-8")
(folder / "._artifact.txt").write_bytes(b"\x00\x05\x16\x07AppleDouble")
return folder
@@ -45,6 +46,11 @@ def _create_sysdiagnose_archive(tmp_path, folder):
return archive_path
def _test_module(command):
(module,) = [m for m in command.executed if isinstance(m, SysdiagnoseTestModule)]
return module
def _run_command(path):
command = CmdIOSCheckSysdiagnose(
target_path=str(path), custom_modules=[SysdiagnoseTestModule]
@@ -56,30 +62,30 @@ def _run_command(path):
def test_check_sysdiagnose_from_folder(tmp_path):
command = _run_command(_create_sysdiagnose_folder(tmp_path))
assert command.executed[0].results == [
assert _test_module(command).results == [
{"content": "artifact", "timezone_offset": timedelta(hours=2).seconds}
]
assert command.executed[0].ips_files == [
assert _test_module(command).ips_files == [
{"file_path": str(tmp_path / "sysdiagnose" / "report.ips"), "bug_type": 210}
]
assert "sysdiagnose/._artifact.txt" not in command.sysdiagnose_files
def test_check_sysdiagnose_from_archive_closes_archive(tmp_path):
folder = _create_sysdiagnose_folder(tmp_path)
command = _run_command(_create_sysdiagnose_archive(tmp_path, folder))
assert command.executed[0].results == [
assert _test_module(command).results == [
{"content": "artifact", "timezone_offset": timedelta(hours=2).seconds}
]
assert command.executed[0].ips_files == [
assert _test_module(command).ips_files == [
{
"file_path": str(
Path(command.extracted_sysdiagnose_path) / "report.ips"
),
"file_path": str(Path(command.extracted_sysdiagnose_path) / "report.ips"),
"bug_type": 210,
}
]
assert command.sysdiagnose_archive is None
assert "sysdiagnose/._artifact.txt" not in command.sysdiagnose_files
def test_archive_is_extracted_once_and_unsafe_members_are_skipped(tmp_path):
@@ -110,7 +116,7 @@ def test_archive_is_extracted_once_and_unsafe_members_are_skipped(tmp_path):
module = SysdiagnoseExtraction()
command.module_init(module)
assert module.tar is None
assert module.parent_path == str(extracted_path.parent)
assert Path(module.parent_path) == extracted_path.parent
finally:
command.finish()
+2
View File
@@ -43,6 +43,7 @@ class TestCompletionCommand:
def test_completion_install_updates_bashrc_once(self, tmp_path, monkeypatch):
monkeypatch.setenv("HOME", str(tmp_path))
monkeypatch.setenv("USERPROFILE", str(tmp_path))
runner = CliRunner()
result = runner.invoke(mvt_cli, ["completion", "bash", "--install"])
@@ -67,6 +68,7 @@ class TestCompletionCommand:
self, tmp_path, monkeypatch
):
monkeypatch.setenv("HOME", str(tmp_path))
monkeypatch.setenv("USERPROFILE", str(tmp_path))
runner = CliRunner()
result = runner.invoke(mvt_cli, ["completion", "fish", "--install"])
Generated
+8 -59
View File
@@ -6,22 +6,6 @@ resolution-markers = [
"python_full_version < '3.15'",
]
[[package]]
name = "adb-shell"
version = "0.4.4"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "cryptography" },
{ name = "pyasn1" },
{ name = "rsa" },
]
sdist = { url = "https://files.pythonhosted.org/packages/8f/73/d246034db6f3e374dad9a35ee3f61345a6b239d4febd2a41ab69df9936fe/adb_shell-0.4.4.tar.gz", hash = "sha256:04c305f30a2ca25d5c54b3cd6ce9bb64c36e5f07967b23b3fb6aaecc851b90b6", size = 61822, upload-time = "2023-09-01T03:48:40.348Z" }
[package.optional-dependencies]
usb = [
{ name = "libusb1" },
]
[[package]]
name = "annotated-types"
version = "0.8.0"
@@ -624,16 +608,15 @@ wheels = [
]
[[package]]
name = "iosbackup"
version = "0.9.925"
name = "iphone-backup-decrypt"
version = "0.10.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nskeyedunarchiver" },
{ name = "pycryptodome" },
]
sdist = { url = "https://files.pythonhosted.org/packages/db/b8/4cd52322deceb942b9e18b127d45d112c2f7a3ec7821ab528659d4f04275/iOSbackup-0.9.925.tar.gz", hash = "sha256:33545a9249e5b3faaadf1ee782fe6bdfcdb70fae0defba1acee336a65f93d1ca", size = 25228, upload-time = "2022-10-16T01:02:46.474Z" }
sdist = { url = "https://files.pythonhosted.org/packages/01/b8/a2ec2b10d792403d36bb6661fea6357179c31ece1ed3285c442e636d6335/iphone_backup_decrypt-0.10.0.tar.gz", hash = "sha256:5f2799e58f908cce28d7929dd82231abc8acd57c25b37bce6a2d6a82e018d343", size = 20000, upload-time = "2026-09-17T21:46:44Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e0/4e/8da5fdc2df642080c79f128b9412a6f33768a610a58d59fbdb7b4d019364/iOSbackup-0.9.925-py3-none-any.whl", hash = "sha256:348edab2b82499d55c17c4bfe02fc6ae7915a2cc8da2728893ed1e9ae61bcef3", size = 18326, upload-time = "2022-10-16T01:02:44.164Z" },
{ url = "https://files.pythonhosted.org/packages/89/10/349b2a720e012921d1457e1d1c2b921b16300a5f3570c4c64d0d3787539f/iphone_backup_decrypt-0.10.0-py3-none-any.whl", hash = "sha256:1ac4345a07b104a3eacab20932b8621cd1e0164e0f03d913e66eed6ea5ee173e", size = 17400, upload-time = "2026-09-17T21:46:42Z" },
]
[[package]]
@@ -778,17 +761,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/38/a6/800800bfed7b1fb10fc3f3d557785c3854e80d3f7a9800d784b176a1fc2d/librt-0.15.0-cp315-cp315t-win_arm64.whl", hash = "sha256:84d244b00604d17df3fc7736c327892d6bba66181254aa4087be807b6c342bdc", size = 110700, upload-time = "2026-08-07T10:49:15.499Z" },
]
[[package]]
name = "libusb1"
version = "3.4.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/89/35/f9d2a990d092d647b47540cd229e1d68432c0f51183484ca189612a4824c/libusb1-3.4.0.tar.gz", hash = "sha256:9cf5638506d54f21bf36550d97ea63189111a23c4d8078f630103a2052135f45", size = 91206, upload-time = "2026-05-16T20:59:19.315Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d4/64/d4b59444e4d3b6979aa5eb58840634465a24b41a9ab03dcf8434c9b89551/libusb1-3.4.0-py3-none-any.whl", hash = "sha256:e83d034e44c3efe1c4599c6281d34bca50a38c12cab3b7b6217d583161a01ffd", size = 67373, upload-time = "2026-05-16T20:59:13.142Z" },
{ url = "https://files.pythonhosted.org/packages/7e/55/a838a4278fac4ee49bf670ddc59d621999a1cb2a3188fb13a23609ea4f06/libusb1-3.4.0-py3-none-win32.whl", hash = "sha256:0a1aa1416034690eb9dc9a895eda0fef44d853bca1053eb1de50a5906684846d", size = 129704, upload-time = "2026-05-16T20:59:15.243Z" },
{ url = "https://files.pythonhosted.org/packages/5f/08/02aecf6dad627534a5835244ece14d7f187ef430354d9b8551199934d059/libusb1-3.4.0-py3-none-win_amd64.whl", hash = "sha256:b7dcc1f324a895af6aac708bc5513a17373f97349cc2ab8a277519c788bd18ca", size = 141212, upload-time = "2026-05-16T20:59:17.286Z" },
]
[[package]]
name = "markdown"
version = "3.10.3"
@@ -1017,16 +989,15 @@ wheels = [
name = "mvt"
source = { editable = "." }
dependencies = [
{ name = "adb-shell", extra = ["usb"] },
{ name = "appdirs" },
{ name = "betterproto2" },
{ name = "click" },
{ name = "cryptography" },
{ name = "iosbackup" },
{ name = "libusb1" },
{ name = "iphone-backup-decrypt" },
{ name = "nskeyedunarchiver" },
{ name = "packaging" },
{ name = "pyahocorasick" },
{ name = "pycryptodome" },
{ name = "pydantic" },
{ name = "pydantic-settings" },
{ name = "python-dateutil" },
@@ -1063,16 +1034,15 @@ docs = [
[package.metadata]
requires-dist = [
{ name = "adb-shell", extras = ["usb"], specifier = "==0.4.4" },
{ name = "appdirs", specifier = "==1.4.4" },
{ name = "betterproto2", specifier = "==0.10.0" },
{ name = "click", specifier = "==8.4.2" },
{ name = "cryptography", specifier = "==50.0.0" },
{ name = "iosbackup", specifier = "==0.9.925" },
{ name = "libusb1", specifier = "==3.4.0" },
{ name = "iphone-backup-decrypt", specifier = "==0.10.0" },
{ name = "nskeyedunarchiver", specifier = "==1.5.2" },
{ name = "packaging", specifier = "==26.3" },
{ name = "pyahocorasick", specifier = "==2.3.1" },
{ name = "pycryptodome", specifier = ">=3.20.0" },
{ name = "pydantic", specifier = "==2.13.4" },
{ name = "pydantic-settings", specifier = "==2.15.0" },
{ name = "python-dateutil", specifier = "==2.9.0.post0" },
@@ -1273,15 +1243,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/c5/96/37c50ac951bb0260ec38d8d12e5b51587ef1ef4035c279088f2771544b28/pyahocorasick-2.3.1-cp314-cp314-win_amd64.whl", hash = "sha256:4acb11a0a2ff10519465749d22ad70789e9fe7f81dc8fe9957a8868e499e18ab", size = 35987, upload-time = "2026-04-27T16:32:07.08Z" },
]
[[package]]
name = "pyasn1"
version = "0.6.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/a4/9a/23310166d960def5897e91fe20e5b724601b02a22e84ba1f94232c0b7f67/pyasn1-0.6.4.tar.gz", hash = "sha256:9c447d8431c947fe4c8febc4ed9e760bc29011a5b01e5c74b67025bd9fb8ce81", size = 151262, upload-time = "2026-07-09T01:12:33.988Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/9a/3b/6163796d69c3977d1e4287bea4a6979161cbbdd170ebb430511e8e1999ce/pyasn1-0.6.4-py3-none-any.whl", hash = "sha256:deda9277cfd454080ec40b207fb6df82206a3a2688735233cdcd8d3d565f088b", size = 84410, upload-time = "2026-07-09T01:12:32.92Z" },
]
[[package]]
name = "pycparser"
version = "3.0"
@@ -1683,18 +1644,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", hash = "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", size = 310654, upload-time = "2026-04-12T08:24:02.83Z" },
]
[[package]]
name = "rsa"
version = "4.9.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "pyasn1" },
]
sdist = { url = "https://files.pythonhosted.org/packages/da/8a/22b7beea3ee0d44b1916c0c1cb0ee3af23b700b6da9f04991899d0c555d4/rsa-4.9.1.tar.gz", hash = "sha256:e7bdbfdb5497da4c07dfd35530e1a902659db6ff241e39d9953cad06ebd0ae75", size = 29034, upload-time = "2025-04-16T09:51:18.218Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/64/8d/0133e4eb4beed9e425d9a98ed6e081a55d195481b7632472be1af08d2f6b/rsa-4.9.1-py3-none-any.whl", hash = "sha256:68635866661c6836b8d39430f97a996acbd61bfa49406748ea243539fe239762", size = 34696, upload-time = "2025-04-16T09:51:17.142Z" },
]
[[package]]
name = "ruff"
version = "0.16.2"