Compare commits

...
Author SHA1 Message Date
Janik Besendorf 3c11c71daa Handle mis-indented dumpsys receiver actions 2026-07-28 18:32:02 +02:00
besendorf 797411e1e5 Fix text tombstone crashing thread parsing (#848) 2026-07-27 17:58:34 +02:00
84df51c518 Scan Safari profile databases for history and browser state (#846)
* fix(ios): scan Safari profile databases for history and browser state

Safari profiles (iOS 17 and later) keep their own databases under
Library/Safari/Profiles/<UUID>/, but SafariHistory and SafariBrowserState
only ever looked at the default profile's Library/Safari/History.db and
Library/Safari/BrowserState.db.

On a device where browsing happens inside a profile, MVT silently skipped
that history and still reported no detections, so an indicator only ever
visited within a profile went unnoticed.

Both modules now also match Library/Safari/Profiles/*/ in backups and in
filesystem dumps. No helper changes were needed: the Manifest.db lookup
already translates "*" into a SQL LIKE wildcard, and the filesystem lookup
already globs.

Found while examining an encrypted iOS 26.5.2 backup that contained 14
per-profile History.db files under
AppDomain-com.apple.mobilesafari::Library/Safari/Profiles/<UUID>/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(ios): scope Safari redirects to history database

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Janik Besendorf <janik@besendorf.org>
2026-07-27 15:06:44 +02:00
besendorf a6c3a805d8 Parallelize URL indicator checks (#844) 2026-07-27 15:03:02 +02:00
besendorf 123c9081ed Skip resolving Google Maps short URLs (#843) 2026-07-19 17:13:57 +02:00
FelixandJanik Besendorf 5ed8b3c1a5 fix: terminate dumpsys adb multiline values at structural lines (#842)
* fix: terminate dumpsys adb multiline values at structural lines

* fix dumpsys ADB multiline boundaries

---------

Co-authored-by: Janik Besendorf <janik@besendorf.org>
2026-07-17 18:25:47 +02:00
besendorf afcfda4720 Deduplicate AndroidQF SMS analysis (#837) 2026-07-15 09:09:04 +02:00
besendorf 911115b0c8 Match receiver indicators by package ID (#836) 2026-07-15 08:32:13 +02:00
besendorf c806fd8d61 Support rotated iOS shutdown logs (#834) 2026-07-14 23:27:44 +02:00
besendorf 516ba06cf7 Suppress benign PinStorage key generation warning (#835) 2026-07-14 23:20:53 +02:00
besendorf 1532578b39 Validate iOS backup path before checks (#825)
* Validate iOS backup path before checks

* Fix iOS backup path mypy typing

* Fix custom module test backup fixture
2026-07-14 21:58:46 +02:00
Nimrod B.andJanik Besendorf f5b0a3cd91 WIP: Addition of a timer to virustotal checks (#593)
* Add delay option to virustotal checks (#408)

* Fix missing delay argument

* Fix VirusTotal delay handling

* Fix mypy type for VirusTotal package map

---------

Co-authored-by: Janik Besendorf <janik@besendorf.org>
2026-07-01 12:36:56 +02:00
besendorf a18e632ec8 Add shell completion command (#817) 2026-07-01 12:35:21 +02:00
52 changed files with 1649 additions and 352 deletions

No files matched your search

+17
View File
@@ -60,6 +60,23 @@ For alternative installation options and known issues, please refer to the [docu
MVT provides two commands `mvt-ios` and `mvt-android`. [Check out the documentation to learn how to use them!](https://docs.mvt.re/)
### Shell completion
MVT can generate shell completion scripts for Bash, Zsh, and Fish:
```bash
mvt-ios completion
mvt-android completion
```
The commands print setup instructions by default. To generate a completion script directly, pass the shell name:
```bash
mvt-ios completion bash
mvt-android completion zsh
```
MVT only writes completion files or shell configuration when `--install` is passed. See the [command completion documentation](https://docs.mvt.re/en/latest/command_completion/) for details.
Module-running `check-*` commands can load custom Python modules with
`--load-module PATH` or from a folder set in `MVT_CUSTOM_MODULES`. See the
[development documentation](https://docs.mvt.re/en/latest/development/) for
+16
View File
@@ -38,6 +38,22 @@ By separating artifact collection from forensic analysis, this approach ensures
For more information, refer to the [AndroidQF project documentation](https://github.com/mvt-project/androidqf).
### VirusTotal package lookups
AndroidQF records APK file hashes in `packages.json`. MVT can optionally look up non-system APK hashes on VirusTotal while checking an AndroidQF acquisition:
```bash
MVT_VT_API_KEY=<key> mvt-android check-androidqf --virustotal /path/to/androidqf-output
```
The `--virustotal` option is disabled by default because it sends APK hashes to VirusTotal and requires network access. It uses the `VT_API_KEY` MVT configuration value, which can also be provided through the `MVT_VT_API_KEY` environment variable.
To avoid exhausting free VirusTotal API quotas, MVT waits 16 seconds between package hash requests by default. Use `--delay` to change the delay, or `--delay 0` to disable throttling:
```bash
mvt-android check-androidqf --virustotal --delay 30 /path/to/androidqf-output
```
## Android Intrusion Logs
On devices where the user has opted into Android's [**Advanced Protection Mode**](https://support.google.com/android/answer/16339980) and turned on the optional Intrusion Logging featrue, Android can create and archive structured *Intrusion Logs* in an encrypted format. These logs record DNS queries, outbound network connections, process starts, ADB activity and other security-relevant events, and are a high-fidelity complement to the rest of an AndroidQF acquisition. The logs are generated on-device and encrypted before being stored in the Google account associated with the device. The encryption key is protected by the user device PIN. The intrusion log data is not accessible to Google.
+36 -13
View File
@@ -1,43 +1,66 @@
# Command Completion
# Command Completion
MVT utilizes the [Click](https://click.palletsprojects.com/en/stable/) library for creating its command line interface.
MVT utilizes the [Click](https://click.palletsprojects.com/en/stable/) library for creating its command line interface.
Click provides tab completion support for Bash (version 4.4 and up), Zsh, and Fish.
To enable it, you need to manually register a special function with your shell, which varies depending on the shell you are using.
To enable it, you need to register a completion script with your shell, which varies depending on the shell you are using.
The following describes how to generate the command completion scripts and add them to your shell configuration.
The following describes how to generate the command completion scripts and add them to your shell configuration.
> **Note: You will need to start a new shell for the changes to take effect.**
### For Bash
```bash
# Generates bash completion scripts
echo "$(_MVT_IOS_COMPLETE=bash_source mvt-ios)" > ~/.mvt-ios-complete.bash &&
echo "$(_MVT_ANDROID_COMPLETE=bash_source mvt-android)" > ~/.mvt-android-complete.bash
# Generate bash completion scripts
mvt-ios completion bash > ~/.mvt-ios-complete.bash
mvt-android completion bash > ~/.mvt-android-complete.bash
```
Add the following to `~/.bashrc`:
```bash
# source mvt completion scripts
. ~/.mvt-ios-complete.bash && . ~/.mvt-android-complete.bash
[ -f ~/.mvt-ios-complete.bash ] && . ~/.mvt-ios-complete.bash
[ -f ~/.mvt-android-complete.bash ] && . ~/.mvt-android-complete.bash
```
### For Zsh
```bash
# Generates zsh completion scripts
echo "$(_MVT_IOS_COMPLETE=zsh_source mvt-ios)" > ~/.mvt-ios-complete.zsh &&
echo "$(_MVT_ANDROID_COMPLETE=zsh_source mvt-android)" > ~/.mvt-android-complete.zsh
# Generate zsh completion scripts
mvt-ios completion zsh > ~/.mvt-ios-complete.zsh
mvt-android completion zsh > ~/.mvt-android-complete.zsh
```
Add the following to `~/.zshrc`:
```bash
# source mvt completion scripts
. ~/.mvt-ios-complete.zsh && . ~/.mvt-android-complete.zsh
[ -f ~/.mvt-ios-complete.zsh ] && . ~/.mvt-ios-complete.zsh
[ -f ~/.mvt-android-complete.zsh ] && . ~/.mvt-android-complete.zsh
```
### For Fish
```bash
# Generate fish completion scripts
mkdir -p ~/.config/fish/completions
mvt-ios completion fish > ~/.config/fish/completions/mvt-ios.fish
mvt-android completion fish > ~/.config/fish/completions/mvt-android.fish
```
Fish loads completion files from `~/.config/fish/completions` automatically.
### Automatic Installation
MVT can write the completion file and update the relevant shell configuration for Bash and Zsh when you pass `--install`:
```bash
mvt-ios completion bash --install
mvt-android completion bash --install
```
Replace `bash` with `zsh` or `fish` as needed. For Fish, `--install` writes the completion file into `~/.config/fish/completions`.
For more information, visit the official [Click Docs](https://click.palletsprojects.com/en/stable/shell-completion/#enabling-completion).
+4 -3
View File
@@ -37,8 +37,10 @@ export MVT_STIX2="/home/user/IOC1.stix2:/home/user/IOC2.stix2"
## Network Access
When checking URL indicators, MVT follows recognized shortened URLs with an
HTTP `HEAD` request. The following environment variables control these
requests:
HTTP `HEAD` request. URL checks are deduplicated and run concurrently, with at
most 20 requests in progress at a time. Redirects within an individual URL
chain are still followed sequentially. The following environment variables
control these requests:
- `MVT_NETWORK_ACCESS_ALLOWED` enables or disables network requests. It defaults
to `true`. Set it to `false` to prevent MVT from attempting to resolve
@@ -73,4 +75,3 @@ You can automaticallly download the latest public indicator files with the comma
Please [open an issue](https://github.com/mvt-project/mvt/issues/) to suggest new sources of STIX-formatted IOCs.
+18 -3
View File
@@ -13,6 +13,13 @@ from .artifact import AndroidArtifact
class DumpsysADBArtifact(AndroidArtifact):
multiline_fields = ["user_keys", "keystore"]
@staticmethod
def is_structural_line(key: str, vals: list) -> bool:
if key == "}":
return True
# XML keystore continuations also split on "=", but never into an identifier.
return len(vals) == 2 and key.isidentifier()
def indented_dump_parser(self, dump_data):
"""
Parse the indented dumpsys output, generated by DualDumpOutputStream in Android.
@@ -41,10 +48,18 @@ class DumpsysADBArtifact(AndroidArtifact):
if key == "":
# If the line is empty, it's the terminator for the multiline value
in_multiline = False
stack.pop()
else:
if isinstance(stack[-1], list):
stack.pop()
continue
if not self.is_structural_line(key, vals):
current_dict.append(line.lstrip())
continue
continue
in_multiline = False
if isinstance(stack[-1], list):
stack.pop()
current_dict = stack[-1]
if key == "}":
stack.pop()
+13 -1
View File
@@ -96,6 +96,18 @@ class DumpsysReceiversArtifact(AndroidArtifact):
self.results[intent] = []
continue
parts = line.strip().split(" ")
if len(parts) < 2:
# A single-token line here is not a receiver. Real dumpstate
# output can print an action header mis-indented (observed with
# 15 leading spaces instead of 6), which used to raise
# IndexError and abort the whole module. Treat a trailing-colon
# token as the next action, skip anything else.
if parts[0].endswith(":"):
intent = parts[0][:-1]
self.results.setdefault(intent, [])
continue
# If we are not in an intent block yet, skip.
if not intent:
continue
@@ -109,7 +121,7 @@ class DumpsysReceiversArtifact(AndroidArtifact):
# If we got this far, we are processing receivers for the
# activities we are interested in.
receiver = line.strip().split(" ")[1]
receiver = parts[1]
package_name = receiver.split("/")[0]
self.results[intent].append(
@@ -206,6 +206,11 @@ class TombstoneCrashArtifact(AndroidArtifact):
return True
def _load_pid_line(self, line: str, tombstone: dict) -> bool:
# The first pid line identifies the crashing thread. Full text tombstones
# contain additional pid lines for the other threads in the process.
if "pid" in tombstone:
return True
try:
parts = line.split(" >>> ") if " >>> " in line else line.split(">>>")
process_info = parts[0]
+56 -4
View File
@@ -8,6 +8,12 @@ import logging
import click
from mvt.common.cmd_check_iocs import CmdCheckIOCS
from mvt.common.completion import (
SUPPORTED_SHELLS,
completion_instructions,
generate_completion_script,
install_completion_script,
)
from mvt.common.help import (
HELP_MSG_ANDROID_BACKUP_PASSWORD,
HELP_MSG_CHECK_ADB_REMOVED,
@@ -17,6 +23,8 @@ from mvt.common.help import (
HELP_MSG_CHECK_BUGREPORT,
HELP_MSG_CHECK_IOCS,
HELP_MSG_CHECK_INTRUSION_LOGS,
HELP_MSG_DELAY_CHECKS,
HELP_MSG_COMPLETION,
HELP_MSG_DISABLE_INDICATOR_UPDATE_CHECK,
HELP_MSG_DISABLE_UPDATE_CHECK,
HELP_MSG_HASHES,
@@ -29,6 +37,7 @@ from mvt.common.help import (
HELP_MSG_STIX2,
HELP_MSG_VERBOSE,
HELP_MSG_VERSION,
HELP_MSG_VIRUS_TOTAL,
)
from mvt.common.logo import logo
from mvt.common.module_loader import CustomModuleLoadError, load_custom_modules
@@ -85,10 +94,11 @@ def cli(ctx, disable_update_check, disable_indicator_update_check):
ctx.ensure_object(dict)
ctx.obj["disable_version_check"] = disable_update_check
ctx.obj["disable_indicator_check"] = disable_indicator_update_check
logo(
disable_version_check=disable_update_check,
disable_indicator_check=disable_indicator_update_check,
)
if ctx.invoked_subcommand != "completion":
logo(
disable_version_check=disable_update_check,
disable_indicator_check=disable_indicator_update_check,
)
# ==============================================================================
@@ -99,6 +109,40 @@ def version():
return
# ==============================================================================
# Command: completion
# ==============================================================================
@cli.command("completion", context_settings=CONTEXT_SETTINGS, help=HELP_MSG_COMPLETION)
@click.argument("shell", required=False, type=click.Choice(SUPPORTED_SHELLS))
@click.option(
"--install",
is_flag=True,
help="Write completion files and update shell configuration.",
)
@click.pass_context
def completion(ctx, shell, install):
program_name = "mvt-android"
if shell is None:
if install:
raise click.UsageError("A shell is required when using --install.")
click.echo(completion_instructions(program_name))
return
root_cli = ctx.find_root().command
if install:
script_path = install_completion_script(root_cli, program_name, shell)
click.echo(f"Installed {shell} completion to {script_path}")
if shell in ("bash", "zsh"):
click.echo(f"Updated ~/.{shell}rc")
else:
click.echo("Fish loads completion files automatically.")
return
click.echo(generate_completion_script(root_cli, program_name, shell))
# ==============================================================================
# Command: check-adb (removed)
# ==============================================================================
@@ -268,6 +312,10 @@ def check_backup(
help=HELP_MSG_LOAD_MODULE,
)
@click.option("--hashes", "-H", is_flag=True, help=HELP_MSG_HASHES)
@click.option("--virustotal", "-V", is_flag=True, help=HELP_MSG_VIRUS_TOTAL)
@click.option(
"--delay", "-d", type=click.IntRange(min=0), default=16, help=HELP_MSG_DELAY_CHECKS
)
@click.option("--non-interactive", "-n", is_flag=True, help=HELP_MSG_NONINTERACTIVE)
@click.option("--backup-password", "-p", help=HELP_MSG_ANDROID_BACKUP_PASSWORD)
@click.option("--verbose", "-v", is_flag=True, help=HELP_MSG_VERBOSE)
@@ -281,6 +329,8 @@ def check_androidqf(
module,
load_module,
hashes,
virustotal,
delay,
non_interactive,
backup_password,
verbose,
@@ -298,6 +348,8 @@ def check_androidqf(
module_options={
"interactive": not non_interactive,
"backup_password": cli_load_android_backup_password(log, backup_password),
"virustotal": virustotal,
"virustotal_delay": delay,
},
disable_version_check=_get_disable_flags(ctx)[0],
disable_indicator_check=_get_disable_flags(ctx)[1],
@@ -10,7 +10,6 @@ from .aqf_processes import AQFProcesses
from .aqf_settings import AQFSettings
from .mounts import Mounts
from .root_binaries import RootBinaries
from .sms import SMS
ANDROIDQF_MODULES = [
AQFPackages,
@@ -18,7 +17,6 @@ ANDROIDQF_MODULES = [
AQFGetProp,
AQFSettings,
AQFFiles,
SMS,
RootBinaries,
Mounts,
]
@@ -5,8 +5,11 @@
import json
import logging
import time
from typing import Optional
from rich.progress import track
from mvt.android.utils import (
BROWSER_INSTALLERS,
PLAY_STORE_INSTALLERS,
@@ -15,7 +18,8 @@ from mvt.android.utils import (
SYSTEM_UPDATE_PACKAGES,
THIRD_PARTY_STORE_INSTALLERS,
)
from mvt.common.module_types import ModuleResults
from mvt.common.module_types import ModuleAtomicResult, ModuleResults
from mvt.common.virustotal import VTNoKey, VTQuotaExceeded, virustotal_lookup
from .base import AndroidQFModule
@@ -125,6 +129,65 @@ class AQFPackages(AndroidQFModule):
)
break
if self.module_options.get("virustotal", False):
self.check_virustotal(
delay=self.module_options.get("virustotal_delay", 0)
)
def check_virustotal(self, delay: int = 0) -> None:
files_by_hash: dict[
str, list[tuple[ModuleAtomicResult, ModuleAtomicResult]]
] = {}
for package in self.results:
if package.get("system", False):
continue
for package_file in package.get("files", []):
file_hash = package_file.get("sha256")
if not file_hash:
continue
files_by_hash.setdefault(file_hash, []).append((package, package_file))
total_hashes = len(files_by_hash)
if total_hashes == 0:
return
progress_desc = f"Looking up {total_hashes} package files on VirusTotal..."
for index, file_hash in enumerate(
track(files_by_hash, description=progress_desc)
):
try:
results = virustotal_lookup(file_hash)
except VTNoKey as exc:
self.log.warning("%s", exc)
return
except VTQuotaExceeded as exc:
self.log.warning("Unable to continue VirusTotal lookups: %s", exc)
break
if index < total_hashes - 1 and delay > 0:
time.sleep(delay)
if not results:
continue
attributes = results.get("attributes", {})
stats = attributes.get("last_analysis_stats", {})
positives = stats.get("malicious", 0)
total = len(attributes.get("last_analysis_results", {}))
detection = f"{positives}/{total}"
for package, package_file in files_by_hash[file_hash]:
package_file["virustotal"] = detection
if positives > 0:
self.alertstore.high(
f'VirusTotal flagged package "{package["name"]}" file '
f'"{package_file["path"]}" with {detection} detections',
"",
package,
)
def run(self) -> None:
packages = self._get_files_by_pattern("*/packages.json")
if not packages:
-105
View File
@@ -1,105 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1
import logging
from typing import Optional
from mvt.android.modules.backup.helpers import prompt_or_load_android_backup_password
from mvt.android.parsers.backup import (
AndroidBackupParsingError,
InvalidBackupPassword,
parse_ab_header,
parse_backup_file,
parse_tar_for_sms,
)
from .base import AndroidQFModule
class SMS(AndroidQFModule):
"""
This module analyse SMS file in backup
"""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def check_indicators(self) -> None:
if not self.indicators:
return
for message in self.results:
if "body" not in message:
continue
ioc_match = self.indicators.check_domains(message.get("links", []))
if ioc_match:
self.alertstore.critical(
ioc_match.message, "", message, matched_indicator=ioc_match.ioc
)
def parse_backup(self, data):
header = parse_ab_header(data)
if not header["backup"]:
self.log.warning("Invalid backup format, backup.ab was not analysed")
return
password = None
if header["encryption"] != "none":
password = prompt_or_load_android_backup_password(
self.log, self.module_options
)
if not password:
self.log.critical("No backup password provided.")
return
try:
tardata = parse_backup_file(data, password=password)
except InvalidBackupPassword:
self.log.critical("Invalid backup password")
return
except AndroidBackupParsingError:
self.log.warning(
"Impossible to parse this backup file, please use"
" Android Backup Extractor instead"
)
return
if not tardata:
return
try:
self.results = parse_tar_for_sms(tardata)
except AndroidBackupParsingError:
self.log.info(
"Impossible to read SMS from the Android Backup, "
"please extract the SMS and try extracting it with "
"Android Backup Extractor"
)
return
def run(self) -> None:
files = self._get_files_by_pattern("*/backup.ab")
if not files:
self.log.info("No backup data found")
return
self.parse_backup(self._get_file_content(files[0]))
self.log.info("Identified %d SMS in backup data", len(self.results))
+2 -3
View File
@@ -4,9 +4,8 @@
# https://license.mvt.re/1.1/
from rich.prompt import Prompt
from mvt.common.config import settings
from mvt.common.password import prompt_password
MVT_ANDROID_BACKUP_PASSWORD = "MVT_ANDROID_BACKUP_PASSWORD"
@@ -49,7 +48,7 @@ def prompt_or_load_android_backup_password(log, module_options):
# The default is to allow interactivity
elif module_options.get("interactive", True):
backup_password = Prompt.ask(prompt="Enter backup password", password=True)
backup_password = prompt_password("Enter backup password: ")
else:
log.critical(
"Cannot decrypt backup because interactivity"
+8 -2
View File
@@ -36,6 +36,8 @@ class SMS(BackupModule):
if not self.indicators:
return
messages = []
url_batches = []
for message in self.results:
if "body" not in message:
continue
@@ -44,12 +46,16 @@ class SMS(BackupModule):
if message_links == []:
message_links = check_for_links(message.get("text", ""))
ioc_match = self.indicators.check_urls(message_links)
messages.append(message)
url_batches.append(message_links)
for message, ioc_match in zip(
messages, self.indicators.check_url_batches(url_batches)
):
if ioc_match:
self.alertstore.critical(
ioc_match.message, "", message, matched_indicator=ioc_match.ioc
)
continue
def run(self) -> None:
sms_path = "apps/com.android.providers.telephony/d_f/*_sms_backup"
@@ -35,24 +35,6 @@ class DumpsysReceivers(DumpsysReceiversArtifact, BugReportModule):
self.results = results if results else {}
def check_indicators(self) -> None:
for result in self.results:
if self.indicators:
receiver_name = self.results[result][0]["receiver"]
# return IoC if the stix2 process name a substring of the receiver name
ioc_match = self.indicators.check_receiver_prefix(receiver_name)
if ioc_match:
self.alertstore.critical(
ioc_match.message,
"",
self.results[result][0],
matched_indicator=ioc_match.ioc,
)
continue
def run(self) -> None:
content = self._get_dumpstate_file()
if not content:
@@ -270,6 +270,11 @@ SECURITY_EVENT_METADATA_KEYS = {
"timestamp",
}
# Known platform failures that remain in the timeline but do not need a warning.
KNOWN_BENIGN_KEY_GENERATION_FAILURES = {
("PinStorage_crossReboot_key", 1001),
}
class SecurityEvent(IntrusionLogsModule):
"""This module analyzes security events from intrusion logs."""
@@ -400,10 +405,17 @@ class SecurityEvent(IntrusionLogsModule):
independent of any loaded indicators."""
# Flag failed cryptographic operations as potentially suspicious
if "key_generated" in result:
if not result["key_generated"].get("success", True):
key_info = result["key_generated"]
key_id = key_info.get("key_id", "unknown")
uid = key_info.get("uid")
is_known_benign_failure = (
key_id,
uid,
) in KNOWN_BENIGN_KEY_GENERATION_FAILURES
if not key_info.get("success", True) and not is_known_benign_failure:
self.log.warning(
"Failed key generation detected for key_id: %s",
result["key_generated"].get("key_id", "unknown"),
key_id,
)
# Flag certificate validation failures
+94
View File
@@ -0,0 +1,94 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from pathlib import Path
import shlex
import click
from click.shell_completion import get_completion_class
SUPPORTED_SHELLS = ("bash", "zsh", "fish")
def completion_instructions(program_name: str) -> str:
return f"""Shell completion for {program_name}
Print a completion script:
{program_name} completion bash > ~/.{program_name}-complete.bash
{program_name} completion zsh > ~/.{program_name}-complete.zsh
mkdir -p ~/.config/fish/completions
{program_name} completion fish > ~/.config/fish/completions/{program_name}.fish
Load the generated Bash script from ~/.bashrc:
[ -f ~/.{program_name}-complete.bash ] && . ~/.{program_name}-complete.bash
Load the generated Zsh script from ~/.zshrc:
[ -f ~/.{program_name}-complete.zsh ] && . ~/.{program_name}-complete.zsh
Fish loads completion files from ~/.config/fish/completions automatically.
To write these files and update Bash/Zsh shell configuration automatically:
{program_name} completion bash --install
{program_name} completion zsh --install
{program_name} completion fish --install
"""
def generate_completion_script(cli: click.Command, program_name: str, shell: str) -> str:
completion_class = get_completion_class(shell)
if completion_class is None:
raise click.ClickException(f"Unsupported shell: {shell}")
complete_var = f"_{program_name.upper().replace('-', '_')}_COMPLETE"
return completion_class(cli, {}, program_name, complete_var).source()
def install_completion_script(
cli: click.Command,
program_name: str,
shell: str,
) -> Path:
script = generate_completion_script(cli, program_name, shell)
script_path = _completion_script_path(program_name, shell)
script_path.parent.mkdir(parents=True, exist_ok=True)
script_path.write_text(script, encoding="utf-8")
if shell in ("bash", "zsh"):
_install_shell_source_line(program_name, shell, script_path)
return script_path
def _completion_script_path(program_name: str, shell: str) -> Path:
home = Path.home()
if shell == "fish":
return home / ".config" / "fish" / "completions" / f"{program_name}.fish"
return home / f".{program_name}-complete.{shell}"
def _install_shell_source_line(program_name: str, shell: str, script_path: Path) -> None:
shell_config_path = Path.home() / f".{shell}rc"
source_line = (
f"[ -f {shlex.quote(str(script_path))} ] && "
f". {shlex.quote(str(script_path))}"
)
block = (
f"# MVT shell completion for {program_name}\n"
f"{source_line}\n"
)
if shell_config_path.exists():
shell_config = shell_config_path.read_text(encoding="utf-8")
if source_line in shell_config:
return
else:
shell_config = ""
separator = "" if not shell_config or shell_config.endswith("\n") else "\n"
with shell_config_path.open("a", encoding="utf-8") as handle:
handle.write(f"{separator}{block}")
+3
View File
@@ -21,6 +21,7 @@ HELP_MSG_CHECK_IOCS = "Compare stored JSON results to provided indicators"
HELP_MSG_STIX2 = "Download public STIX2 indicators"
HELP_MSG_DISABLE_UPDATE_CHECK = "Disable MVT version update check"
HELP_MSG_DISABLE_INDICATOR_UPDATE_CHECK = "Disable indicators update check"
HELP_MSG_COMPLETION = "Generate or install shell completion"
# IOS Specific
HELP_MSG_DECRYPT_BACKUP = "Decrypt an encrypted iTunes backup"
@@ -52,3 +53,5 @@ HELP_MSG_CHECK_BUGREPORT = "Check an Android Bug Report"
HELP_MSG_CHECK_ANDROID_BACKUP = "Check an Android Backup"
HELP_MSG_CHECK_ANDROIDQF = "Check data collected with AndroidQF"
HELP_MSG_CHECK_INTRUSION_LOGS = "Check Android Intrusion Logging files"
HELP_MSG_VIRUS_TOTAL = "Check package APK hashes on VirusTotal"
HELP_MSG_DELAY_CHECKS = "Delay in seconds between VirusTotal requests"
+38 -29
View File
@@ -7,9 +7,10 @@ import glob
import json
import logging
import os
from concurrent.futures import ThreadPoolExecutor
from dataclasses import dataclass
from functools import lru_cache
from typing import Any, Dict, Iterator, List, Optional
from typing import Any, Dict, Iterator, List, Optional, Sequence
import ahocorasick
from appdirs import user_data_dir
@@ -22,6 +23,8 @@ MVT_INDICATORS_FOLDER = os.path.join(MVT_DATA_FOLDER, "indicators")
logger = logging.getLogger(__name__)
URL_CHECK_MAX_WORKERS = 20
@dataclass
class Indicator:
@@ -490,12 +493,41 @@ class Indicators:
if not urls:
return None
for url in urls:
check = self.check_url(url)
if check:
return check
return self.check_url_batches([urls])[0]
return None
def check_url_batches(
self, url_batches: Sequence[Optional[Sequence[str]]]
) -> List[Optional[IndicatorMatch]]:
"""Check batches of URLs concurrently while preserving batch order.
URLs are deduplicated across batches before checking. Each returned item
is the first indicator match from the corresponding input batch, using
the original URL order.
"""
batches = [list(urls) if urls else [] for urls in url_batches]
unique_urls = list(
dict.fromkeys(url for urls in batches for url in urls)
)
if not unique_urls:
return [None] * len(batches)
if settings.NETWORK_ACCESS_ALLOWED and len(unique_urls) > 1:
worker_count = min(URL_CHECK_MAX_WORKERS, len(unique_urls))
with ThreadPoolExecutor(max_workers=worker_count) as executor:
url_matches = dict(
zip(unique_urls, executor.map(self.check_url, unique_urls))
)
else:
url_matches = {url: self.check_url(url) for url in unique_urls}
return [
next(
(url_matches[url] for url in urls if url_matches[url] is not None),
None,
)
for urls in batches
]
def check_process(self, process: str) -> Optional[IndicatorMatch]:
"""Check the provided process name against the list of process
@@ -727,29 +759,6 @@ class Indicators:
return None
def check_receiver_prefix(
self, receiver_name: str
) -> Optional[IndicatorMatch]:
"""Check the provided receiver name against the list of indicators.
An IoC match is detected when a substring of the receiver matches the indicator.
:param receiver_name: Receiver name to check against app ID indicators
:type receiver_name: str
:returns: IndicatorMatch if matched, otherwise None
"""
if not receiver_name:
return None
for ioc in self.get_iocs("app_ids"):
if ioc.value.lower() in receiver_name.lower():
return IndicatorMatch(
ioc=ioc,
message=f'Found a known suspicious receiver with name "{receiver_name}" matching indicators from "{ioc.name}"',
)
return None
def check_android_property_name(
self, property_name: str
) -> Optional[IndicatorMatch]:
+143
View File
@@ -0,0 +1,143 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
"""Password prompts with masked keyboard feedback.
This is based on the ``echo_char`` support added to :mod:`getpass` in Python
3.14. MVT supports Python 3.10 and later, so it cannot use that API directly.
"""
import contextlib
import io
import os
import sys
import warnings
from typing import TextIO
def prompt_password(prompt: str) -> str:
"""Return a password while displaying an asterisk for each character."""
try:
import termios
return _unix_getpass(prompt, termios)
except ImportError:
try:
import msvcrt
return _windows_getpass(prompt, msvcrt)
except ImportError:
return _fallback_getpass(prompt)
def _unix_getpass(prompt: str, termios) -> str:
password = None
input_stream: TextIO
output_stream: TextIO
with contextlib.ExitStack() as stack:
try:
fd = os.open("/dev/tty", os.O_RDWR | os.O_NOCTTY)
tty = io.FileIO(fd, "w+")
stack.enter_context(tty)
input_stream = io.TextIOWrapper(tty)
stack.enter_context(input_stream)
output_stream = input_stream
except OSError:
stack.close()
try:
fd = sys.stdin.fileno()
except (AttributeError, ValueError):
return _fallback_getpass(prompt)
input_stream = sys.stdin
output_stream = sys.stderr
try:
old = termios.tcgetattr(fd)
new = old[:]
new[3] &= ~termios.ECHO
new[3] &= ~termios.ICANON
try:
termios.tcsetattr(fd, termios.TCSAFLUSH, new)
password = _readline_with_asterisks(output_stream, input_stream, prompt)
finally:
termios.tcsetattr(fd, termios.TCSAFLUSH, old)
output_stream.flush()
except termios.error:
if password is not None:
raise
password = _fallback_getpass(prompt)
output_stream.write("\n")
return password
def _windows_getpass(prompt: str, msvcrt) -> str:
if sys.stdin is not sys.__stdin__:
return _fallback_getpass(prompt)
for char in prompt:
msvcrt.putwch(char)
password = ""
while True:
char = msvcrt.getwch()
if char in ("\r", "\n"):
break
if char == "\x03":
raise KeyboardInterrupt
if char == "\b":
if password:
msvcrt.putwch("\b")
msvcrt.putwch(" ")
msvcrt.putwch("\b")
password = password[:-1]
else:
password += char
msvcrt.putwch("*")
msvcrt.putwch("\r")
msvcrt.putwch("\n")
return password
def _fallback_getpass(prompt: str) -> str:
warnings.warn(
"Can not control echo on the terminal.",
category=UserWarning,
stacklevel=2,
)
print("Warning: Password input may be echoed.", file=sys.stderr)
return input(prompt)
def _readline_with_asterisks(
output_stream: TextIO, input_stream: TextIO, prompt: str
) -> str:
output_stream.write(prompt)
output_stream.flush()
password = ""
eof_pressed = False
while True:
char = input_stream.read(1)
if char in ("\n", "\r"):
break
if char == "\x03":
raise KeyboardInterrupt
if char in ("\x7f", "\b"):
if password:
output_stream.write("\b \b")
output_stream.flush()
password = password[:-1]
elif char == "\x04":
if eof_pressed:
break
eof_pressed = True
elif char != "\x00":
password += char
output_stream.write("*")
output_stream.flush()
eof_pressed = False
return password
+5
View File
@@ -5,6 +5,7 @@
import logging
from typing import Optional
from urllib.parse import urlparse
import requests
from tld import get_tld
@@ -373,6 +374,10 @@ class URL:
:rtype: bool
"""
parsed_url = urlparse(self.url if "://" in self.url else f"//{self.url}")
if self.domain.lower() == "goo.gl" and parsed_url.path.startswith("/maps/"):
return False
if self.domain.lower() in SHORTENER_DOMAINS:
self.is_shortened = True
+53
View File
@@ -0,0 +1,53 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Any, Optional
import requests
from .config import settings
log = logging.getLogger(__name__)
class VTNoKey(Exception):
pass
class VTQuotaExceeded(Exception):
pass
def virustotal_lookup(file_hash: str) -> Optional[dict[str, Any]]:
if not settings.VT_API_KEY:
raise VTNoKey(
"No VirusTotal API key provided: to use VirusTotal lookups please set "
"MVT_VT_API_KEY or VT_API_KEY in the MVT configuration file"
)
headers = {
"User-Agent": "VirusTotal",
"Content-Type": "application/json",
"x-apikey": settings.VT_API_KEY,
}
res = requests.get(
f"https://www.virustotal.com/api/v3/files/{file_hash}",
headers=headers,
timeout=settings.NETWORK_TIMEOUT,
)
if res.status_code == 200:
report = res.json()
return report["data"]
if res.status_code == 404:
log.info("Could not find results for file with hash %s", file_hash)
elif res.status_code == 429:
raise VTQuotaExceeded("You have exceeded the quota for your VirusTotal API key")
else:
raise RuntimeError(f"Unexpected response from VirusTotal: {res.status_code}")
return None
+53 -9
View File
@@ -8,9 +8,13 @@ import logging
import os
import click
from rich.prompt import Prompt
from mvt.common.cmd_check_iocs import CmdCheckIOCS
from mvt.common.completion import (
SUPPORTED_SHELLS,
completion_instructions,
generate_completion_script,
install_completion_script,
)
from mvt.common.logo import logo
from mvt.common.options import MutuallyExclusiveOption
from mvt.common.updates import IndicatorsUpdates
@@ -40,8 +44,10 @@ from mvt.common.help import (
HELP_MSG_CHECK_IOS_BACKUP,
HELP_MSG_DISABLE_UPDATE_CHECK,
HELP_MSG_DISABLE_INDICATOR_UPDATE_CHECK,
HELP_MSG_COMPLETION,
)
from mvt.common.module_loader import CustomModuleLoadError, load_custom_modules
from mvt.common.password import prompt_password
from .cmd_check_backup import CmdIOSCheckBackup
from .cmd_check_fs import CmdIOSCheckFS
from .decrypt import DecryptBackup
@@ -91,10 +97,11 @@ def cli(ctx, disable_update_check, disable_indicator_update_check):
ctx.ensure_object(dict)
ctx.obj["disable_version_check"] = disable_update_check
ctx.obj["disable_indicator_check"] = disable_indicator_update_check
logo(
disable_version_check=disable_update_check,
disable_indicator_check=disable_indicator_update_check,
)
if ctx.invoked_subcommand != "completion":
logo(
disable_version_check=disable_update_check,
disable_indicator_check=disable_indicator_update_check,
)
# ==============================================================================
@@ -105,6 +112,40 @@ def version():
return
# ==============================================================================
# Command: completion
# ==============================================================================
@cli.command("completion", context_settings=CONTEXT_SETTINGS, help=HELP_MSG_COMPLETION)
@click.argument("shell", required=False, type=click.Choice(SUPPORTED_SHELLS))
@click.option(
"--install",
is_flag=True,
help="Write completion files and update shell configuration.",
)
@click.pass_context
def completion(ctx, shell, install):
program_name = "mvt-ios"
if shell is None:
if install:
raise click.UsageError("A shell is required when using --install.")
click.echo(completion_instructions(program_name))
return
root_cli = ctx.find_root().command
if install:
script_path = install_completion_script(root_cli, program_name, shell)
click.echo(f"Installed {shell} completion to {script_path}")
if shell in ("bash", "zsh"):
click.echo(f"Updated ~/.{shell}rc")
else:
click.echo("Fish loads completion files automatically.")
return
click.echo(generate_completion_script(root_cli, program_name, shell))
# ==============================================================================
# Command: decrypt-backup
# ==============================================================================
@@ -159,7 +200,7 @@ def decrypt_backup(ctx, destination, password, key_file, hashes, backup_path):
log.info("Using password from %s environment variable", MVT_IOS_BACKUP_PASSWORD)
backup.decrypt_with_password(os.environ[MVT_IOS_BACKUP_PASSWORD])
else:
sekrit = Prompt.ask("Enter backup password", password=True)
sekrit = prompt_password("Enter backup password: ")
backup.decrypt_with_password(sekrit)
if not backup.can_process():
@@ -211,7 +252,7 @@ def extract_key(password, key_file, backup_path):
log.info("Using password from %s environment variable", MVT_IOS_BACKUP_PASSWORD)
password = os.environ[MVT_IOS_BACKUP_PASSWORD]
else:
password = Prompt.ask("Enter backup password", password=True)
password = prompt_password("Enter backup password: ")
backup.decrypt_with_password(password)
backup.get_key()
@@ -281,7 +322,10 @@ def check_backup(
cmd.list_modules()
return
log.info("Checking iTunes backup located at: %s", backup_path)
if not cmd.resolve_backup_path():
ctx.exit(1)
log.info("Checking iTunes backup located at: %s", cmd.target_path)
cmd.run()
cmd.show_alerts_brief()
+48
View File
@@ -4,6 +4,7 @@
# https://license.mvt.re/1.1/
import logging
import os
from typing import Optional
from mvt.common.command import Command
@@ -16,6 +17,12 @@ from .modules.mixed import MIXED_MODULES
log = logging.getLogger(__name__)
def is_ios_backup_folder(path: str) -> bool:
return os.path.isfile(os.path.join(path, "Manifest.db")) and os.path.isfile(
os.path.join(path, "Info.plist")
)
class CmdIOSCheckBackup(Command):
def __init__(
self,
@@ -52,5 +59,46 @@ class CmdIOSCheckBackup(Command):
self.name = "check-backup"
self.modules = BACKUP_MODULES + MIXED_MODULES
def resolve_backup_path(self) -> bool:
target_path = getattr(self, "target_path", None)
if not isinstance(target_path, str) or not target_path:
return False
if is_ios_backup_folder(target_path):
return True
if not os.path.isdir(target_path):
self.log.critical(
"%s does not appear to be an iTunes backup folder. "
"Expected Manifest.db and Info.plist.",
target_path,
)
return False
candidates = []
for entry_name in sorted(os.listdir(target_path)):
entry_path = os.path.join(target_path, entry_name)
if os.path.isdir(entry_path) and is_ios_backup_folder(entry_path):
candidates.append(entry_path)
if len(candidates) == 1:
self.log.info("Found iTunes backup in subfolder: %s", candidates[0])
self.target_path = candidates[0]
return True
if candidates:
self.log.critical(
"Found multiple iTunes backups in %s. Please specify one backup folder.",
target_path,
)
return False
self.log.critical(
"%s does not appear to be an iTunes backup folder. "
"Expected Manifest.db and Info.plist.",
target_path,
)
return False
def module_init(self, module):
module.is_backup = True
+15 -6
View File
@@ -6,6 +6,7 @@
import logging
from typing import Optional
from mvt.common.module import DatabaseNotFoundError
from mvt.common.module_types import (
ModuleAtomicResult,
ModuleResults,
@@ -17,6 +18,7 @@ from ..base import IOSExtraction
SHUTDOWN_LOG_PATH = [
"private/var/db/diagnostics/shutdown.log",
"private/var/db/diagnostics/shutdown.*.log",
]
@@ -132,10 +134,17 @@ class ShutdownLog(IOSExtraction):
self.results = sorted(self.results, key=lambda entry: entry["isodate"])
def run(self) -> None:
self._find_ios_database(root_paths=SHUTDOWN_LOG_PATH)
self.log.info("Found shutdown log at path: %s", self.file_path)
if self.file_path:
shutdown_log_paths = [self.file_path]
else:
shutdown_log_paths = sorted(
self._get_fs_files_from_patterns(SHUTDOWN_LOG_PATH)
)
if not self.file_path:
return
with open(self.file_path, "r", encoding="utf-8") as handle:
self.process_shutdownlog(handle.read())
if not shutdown_log_paths:
raise DatabaseNotFoundError("unable to find any shutdown log files")
for shutdown_log_path in shutdown_log_paths:
self.log.info("Found shutdown log at path: %s", shutdown_log_path)
with open(shutdown_log_path, "r", encoding="utf-8") as handle:
self.process_shutdownlog(handle.read())
@@ -19,10 +19,17 @@ from mvt.common.utils import convert_mactime_to_iso, keys_bytes_to_string
from ..base import IOSExtraction
SAFARI_BROWSER_STATE_BACKUP_RELPATH = "Library/Safari/BrowserState.db"
# Safari profiles (iOS 17 and later) keep their per-profile databases under
# Library/Safari/Profiles/<UUID>/, separate from the default profile's.
SAFARI_BROWSER_STATE_BACKUP_RELPATHS = [
"Library/Safari/BrowserState.db",
"Library/Safari/Profiles/*/BrowserState.db",
]
SAFARI_BROWSER_STATE_ROOT_PATHS = [
"private/var/mobile/Library/Safari/BrowserState.db",
"private/var/mobile/Library/Safari/Profiles/*/BrowserState.db",
"private/var/mobile/Containers/Data/Application/*/Library/Safari/BrowserState.db",
"private/var/mobile/Containers/Data/Application/*/Library/Safari/Profiles/*/BrowserState.db",
]
@@ -174,20 +181,22 @@ class SafariBrowserState(IOSExtraction):
def run(self) -> None:
if self.is_backup:
for backup_file in self._get_backup_files_from_manifest(
relative_path=SAFARI_BROWSER_STATE_BACKUP_RELPATH
):
browserstate_path = self._get_backup_file_from_id(
backup_file["file_id"]
)
for relative_path in SAFARI_BROWSER_STATE_BACKUP_RELPATHS:
for backup_file in self._get_backup_files_from_manifest(
relative_path=relative_path
):
browserstate_path = self._get_backup_file_from_id(
backup_file["file_id"]
)
if not browserstate_path:
continue
if not browserstate_path:
continue
self.log.info(
"Found Safari browser state database at path: %s", browserstate_path
)
self._process_browser_state_db(browserstate_path)
self.log.info(
"Found Safari browser state database at path: %s",
browserstate_path,
)
self._process_browser_state_db(browserstate_path)
elif self.is_fs_dump:
for browserstate_path in self._get_fs_files_from_patterns(
SAFARI_BROWSER_STATE_ROOT_PATHS
+24 -9
View File
@@ -17,10 +17,17 @@ from mvt.common.utils import convert_mactime_to_datetime, convert_mactime_to_iso
from ..base import IOSExtraction
SAFARI_HISTORY_BACKUP_RELPATH = "Library/Safari/History.db"
# Safari profiles (iOS 17 and later) each keep their own History.db under
# Library/Safari/Profiles/<UUID>/, separate from the default profile's database.
SAFARI_HISTORY_BACKUP_RELPATHS = [
"Library/Safari/History.db",
"Library/Safari/Profiles/*/History.db",
]
SAFARI_HISTORY_ROOT_PATHS = [
"private/var/mobile/Library/Safari/History.db",
"private/var/mobile/Library/Safari/Profiles/*/History.db",
"private/var/mobile/Containers/Data/Application/*/Library/Safari/History.db",
"private/var/mobile/Containers/Data/Application/*/Library/Safari/Profiles/*/History.db",
]
@@ -75,6 +82,9 @@ class SafariHistory(IOSExtraction):
# We loop again through visits in order to find redirect record.
for redirect in self.results:
if redirect["safari_history_db"] != result["safari_history_db"]:
continue
if redirect["visit_id"] != result["redirect_destination"]:
continue
@@ -159,17 +169,22 @@ class SafariHistory(IOSExtraction):
def run(self) -> None:
if self.is_backup:
for history_file in self._get_backup_files_from_manifest(
relative_path=SAFARI_HISTORY_BACKUP_RELPATH
):
history_path = self._get_backup_file_from_id(history_file["file_id"])
for relative_path in SAFARI_HISTORY_BACKUP_RELPATHS:
for history_file in self._get_backup_files_from_manifest(
relative_path=relative_path
):
history_path = self._get_backup_file_from_id(
history_file["file_id"]
)
if not history_path:
continue
if not history_path:
continue
self.log.info("Found Safari history database at path: %s", history_path)
self.log.info(
"Found Safari history database at path: %s", history_path
)
self._process_history_db(history_path)
self._process_history_db(history_path)
elif self.is_fs_dump:
for history_path in self._get_fs_files_from_patterns(
SAFARI_HISTORY_ROOT_PATHS
+4 -2
View File
@@ -76,8 +76,10 @@ class Shortcuts(IOSExtraction):
if not self.indicators:
return
for result in self.results:
ioc_match = self.indicators.check_urls(result["action_urls"])
url_batches = [result["action_urls"] for result in self.results]
for result, ioc_match in zip(
self.results, self.indicators.check_url_batches(url_batches)
):
if ioc_match:
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
+6 -1
View File
@@ -85,12 +85,17 @@ class SMS(IOSExtraction):
if not self.indicators:
return
url_batches = []
for result in self.results:
message_links = result.get("links", [])
# Making sure not link was ignored
if message_links == []:
message_links = check_for_links(result.get("text", ""))
ioc_match = self.indicators.check_urls(message_links)
url_batches.append(message_links)
for result, ioc_match in zip(
self.results, self.indicators.check_url_batches(url_batches)
):
if ioc_match:
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
@@ -70,6 +70,8 @@ class WebkitSessionResourceLog(IOSExtraction):
if not self.indicators:
return
records = []
url_batches = []
for _, entries in self.results.items():
for entry in entries:
source_domains = self._extract_domains(entry["redirect_source"])
@@ -94,36 +96,42 @@ class WebkitSessionResourceLog(IOSExtraction):
)
)
ioc_match = self.indicators.check_urls(all_origins)
if ioc_match:
self.alertstore.critical(
ioc_match.message, "", entry, matched_indicator=ioc_match.ioc
)
records.append((entry, source_domains, destination_domains))
url_batches.append(all_origins)
redirect_path = ""
if len(source_domains) > 0:
redirect_path += "SOURCE: "
for idx, item in enumerate(source_domains):
source_domains[idx] = f'"{item}"'
for record, ioc_match in zip(
records, self.indicators.check_url_batches(url_batches)
):
if ioc_match:
entry, source_domains, destination_domains = record
self.alertstore.critical(
ioc_match.message, "", entry, matched_indicator=ioc_match.ioc
)
redirect_path += ", ".join(source_domains)
redirect_path += " -> "
redirect_path = ""
if len(source_domains) > 0:
redirect_path += "SOURCE: "
for idx, item in enumerate(source_domains):
source_domains[idx] = f'"{item}"'
redirect_path += f'ORIGIN: "{entry["origin"]}"'
redirect_path += ", ".join(source_domains)
redirect_path += " -> "
if len(destination_domains) > 0:
redirect_path += " -> "
redirect_path += "DESTINATION: "
for idx, item in enumerate(destination_domains):
destination_domains[idx] = f'"{item}"'
redirect_path += f'ORIGIN: "{entry["origin"]}"'
redirect_path += ", ".join(destination_domains)
if len(destination_domains) > 0:
redirect_path += " -> "
redirect_path += "DESTINATION: "
for idx, item in enumerate(destination_domains):
destination_domains[idx] = f'"{item}"'
self.alertstore.high(
f"Found HTTP redirect between suspicious domains: {redirect_path}",
"",
entry,
)
redirect_path += ", ".join(destination_domains)
self.alertstore.high(
f"Found HTTP redirect between suspicious domains: {redirect_path}",
"",
entry,
)
def _extract_browsing_stats(self, log_path):
items = []
+4 -2
View File
@@ -61,8 +61,10 @@ class Whatsapp(IOSExtraction):
if not self.indicators:
return
for result in self.results:
ioc_match = self.indicators.check_urls(result.get("links", []))
url_batches = [result.get("links", []) for result in self.results]
for result, ioc_match in zip(
self.results, self.indicators.check_url_batches(url_batches)
):
if ioc_match:
self.alertstore.critical(
ioc_match.message, "", result, matched_indicator=ioc_match.ioc
@@ -30,6 +30,66 @@ class TestDumpsysADBArtifact:
)
assert user_key["user"] == "user@linux"
def test_parsing_adb_wifi(self):
da_adb = DumpsysADBArtifact()
file = get_artifact("android_data/dumpsys_adb_wifi.txt")
with open(file, "rb") as f:
data = f.read()
da_adb.parse(data)
assert len(da_adb.results) == 1
adb_data = da_adb.results[0]
assert "user_keys" in adb_data
assert len(adb_data["user_keys"]) == 1
user_key = adb_data["user_keys"][0]
assert (
user_key["fingerprint"] == "F0:A1:3D:8C:B3:F4:7B:09:9F:EE:8B:D8:38:2E:BD:C6"
)
assert user_key["user"] == "user@linux"
# The adb_wifi block following the keystore is not part of the keystore.
assert b"adb_wifi" not in adb_data["keystore"]
def test_parsing_multiline_terminated_by_structural_line(self):
dump_data = (
b"debugging_manager={\n"
b" keystore=ABX\x00\x0bkeyStore\x00\x02\x11\n"
b" connected_to_adb=true\n"
b" adb_wifi={\n"
b" enabled=false\n"
b" tls_port=0\n"
b" }\n"
)
parsed = DumpsysADBArtifact().indented_dump_parser(dump_data)
debugging_manager = parsed["debugging_manager"]
assert debugging_manager["keystore"] == [b"ABX\x00\x0bkeyStore\x00\x02\x11"]
assert debugging_manager["connected_to_adb"] == b"true"
assert debugging_manager["adb_wifi"] == {
"enabled": b"false",
"tls_port": b"0",
}
def test_parsing_multiline_terminated_by_closing_brace(self):
dump_data = (
b"debugging_manager={\n"
b" keystore=ABX\x00\x0bkeyStore\x00\x02\x11\n"
b"}\n"
b"other={\n"
b" value=true\n"
b"}\n"
)
parsed = DumpsysADBArtifact().indented_dump_parser(dump_data)
assert parsed["debugging_manager"]["keystore"] == [
b"ABX\x00\x0bkeyStore\x00\x02\x11"
]
assert parsed["other"] == {"value": b"true"}
def test_parsing_adb_xml(self):
da_adb = DumpsysADBArtifact()
file = get_artifact("android_data/dumpsys_adb_xml.txt")
@@ -31,6 +31,44 @@ class TestDumpsysReceiversArtifact:
== "com.android.storagemanager"
)
def test_parsing_misindented_action(self):
dr = DumpsysReceiversArtifact()
data = """\
Receiver Resolver Table:
Non-Data Actions:
android.app.action.ENTER_CAR_MODE:
b5b40f6 com.google.android.projection.gearhead/.CarModeBroadcastReceiver
android.intent.action.MY_PACKAGE_REPLACED:
e7706c1 com.psycatgames.nhiegame/.ScheduledNotificationBootReceiver
"""
dr.parse(data)
assert (
dr.results["android.intent.action.MY_PACKAGE_REPLACED"][0][
"package_name"
]
== "com.psycatgames.nhiegame"
)
def test_parsing_misindented_first_action(self):
dr = DumpsysReceiversArtifact()
data = """\
Receiver Resolver Table:
Non-Data Actions:
android.intent.action.MY_PACKAGE_REPLACED:
e7706c1 com.psycatgames.nhiegame/.ScheduledNotificationBootReceiver
"""
dr.parse(data)
assert (
dr.results["android.intent.action.MY_PACKAGE_REPLACED"][0][
"package_name"
]
== "com.psycatgames.nhiegame"
)
def test_ioc_check(self, indicator_file):
dr = DumpsysReceiversArtifact()
file = get_artifact("android_data/dumpsys_packages.txt")
+26
View File
@@ -42,6 +42,32 @@ class TestTombstoneCrashArtifact:
assert len(tombstone_artifact.results) == 1
self.validate_tombstone_result(tombstone_artifact.results[0])
def test_text_tombstone_keeps_crashing_thread(self):
tombstone_artifact = TombstoneCrashArtifact()
artifact_path = "android_data/tombstone_process.txt"
file = get_artifact(artifact_path)
with open(file, "rb") as f:
data = f.read()
data += (
b"\npid: 25541, tid: 31896, name: worker-thread"
b" >>> /vendor/bin/other <<<\n"
)
tombstone_artifact.parse(
os.path.basename(artifact_path),
datetime.datetime(2023, 4, 12, 12, 32, 40, 518290),
data,
)
result = tombstone_artifact.results[0]
assert result["pid"] == 25541
assert result["tid"] == 21307
assert result["process_name"] == "mtk.ape.decoder"
assert (
result["binary_path"]
== "/vendor/bin/hw/android.hardware.media.c2@1.2-mediatek"
)
@pytest.mark.skip(reason="Not implemented yet")
def test_tombtone_kernel_parsing(self):
tombstone_artifact = TombstoneCrashArtifact()
+46
View File
@@ -215,6 +215,52 @@ def _run_security_heuristics(results):
return module.alertstore.alerts
@pytest.mark.parametrize("success", [False, 0])
def test_known_pinstorage_key_generation_failure_does_not_warn(success, caplog):
record = {
"timestamp": "2026-06-17 15:31:02.014",
"key_generated": {
"success": success,
"key_id": "PinStorage_crossReboot_key",
"uid": 1001,
},
}
with caplog.at_level(logging.WARNING):
_run_security_heuristics([record])
assert "Failed key generation detected" not in caplog.text
timeline_event = SecurityEvent().serialize(record)
assert timeline_event["event"] == "key_generated"
assert "Key generation failed: PinStorage_crossReboot_key" in timeline_event["data"]
@pytest.mark.parametrize(
("key_id", "uid"),
[
("PinStorage_crossReboot_key", 10_000),
("another_key", 1001),
],
)
def test_other_key_generation_failures_still_warn(key_id, uid, caplog):
with caplog.at_level(logging.WARNING):
_run_security_heuristics(
[
{
"timestamp": "2026-06-17 15:31:02.014",
"key_generated": {
"success": False,
"key_id": key_id,
"uid": uid,
},
}
]
)
assert f"Failed key generation detected for key_id: {key_id}" in caplog.text
def test_cert_authority_installed_raises_medium_alert_without_indicators():
alerts = _run_security_heuristics(
[
+55
View File
@@ -7,8 +7,11 @@ import logging
from pathlib import Path
import pytest
from click.testing import CliRunner
from mvt.android.cli import check_androidqf
from mvt.android.modules.androidqf.aqf_packages import AQFPackages
from mvt.android.modules.androidqf import aqf_packages as aqf_packages_module
from mvt.common.module import run_module
from ..utils import get_android_androidqf, list_files
@@ -132,3 +135,55 @@ class TestAndroidqfPackages:
possible_detected_app[0].matched_indicator.value
== "c7e56178748be1441370416d4c10e34817ea0c961eb636c8e9d98e0fd79bf730"
)
def test_virustotal_delays_after_missing_result(self, monkeypatch):
lookups = []
sleeps = []
def fake_virustotal_lookup(file_hash):
lookups.append(file_hash)
if file_hash == "missing_hash":
return None
return {
"attributes": {
"last_analysis_stats": {"malicious": 1},
"last_analysis_results": {"engine": {}},
}
}
monkeypatch.setattr(
aqf_packages_module, "virustotal_lookup", fake_virustotal_lookup
)
monkeypatch.setattr(aqf_packages_module.time, "sleep", sleeps.append)
module = AQFPackages(
module_options={"virustotal": True, "virustotal_delay": 16},
results=[
{
"name": "org.example",
"installer": "com.android.vending",
"disabled": False,
"system": False,
"files": [
{"path": "/data/app/missing.apk", "sha256": "missing_hash"},
{"path": "/data/app/found.apk", "sha256": "found_hash"},
],
}
],
)
module.check_indicators()
assert lookups == ["missing_hash", "found_hash"]
assert sleeps == [16]
assert module.results[0]["files"][1]["virustotal"] == "1/1"
assert len(module.alertstore.alerts) == 1
def test_check_androidqf_rejects_negative_virustotal_delay(data_path):
runner = CliRunner()
result = runner.invoke(check_androidqf, ["--delay", "-1", data_path])
assert result.exit_code == 2
assert "Invalid value for '--delay'" in result.output
-91
View File
@@ -1,91 +0,0 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
import os
from pathlib import Path
from mvt.android.modules.androidqf.sms import SMS
from mvt.common.module import run_module
from ..utils import get_android_androidqf, get_artifact_folder, list_files
TEST_BACKUP_PASSWORD = "123456"
class TestAndroidqfSMSAnalysis:
def test_androidqf_sms(self):
data_path = get_android_androidqf()
m = SMS(target_path=data_path, log=logging)
files = list_files(data_path)
parent_path = Path(data_path).absolute().parent.as_posix()
m.from_dir(parent_path, files)
run_module(m)
assert len(m.results) == 2
assert len(m.timeline) == 0
assert len(m.alertstore.alerts) == 0
def test_androidqf_sms_encrypted_password_valid(self):
data_path = os.path.join(get_artifact_folder(), "androidqf_encrypted")
m = SMS(
target_path=data_path,
log=logging,
module_options={"backup_password": TEST_BACKUP_PASSWORD},
)
files = list_files(data_path)
parent_path = Path(data_path).absolute().parent.as_posix()
m.from_dir(parent_path, files)
run_module(m)
assert len(m.results) == 1
def test_androidqf_sms_encrypted_password_prompt(self, mocker):
data_path = os.path.join(get_artifact_folder(), "androidqf_encrypted")
prompt_mock = mocker.patch(
"rich.prompt.Prompt.ask", return_value=TEST_BACKUP_PASSWORD
)
m = SMS(
target_path=data_path,
log=logging,
module_options={},
)
files = list_files(data_path)
parent_path = Path(data_path).absolute().parent.as_posix()
m.from_dir(parent_path, files)
run_module(m)
assert prompt_mock.call_count == 1
assert len(m.results) == 1
def test_androidqf_sms_encrypted_password_invalid(self, caplog):
data_path = os.path.join(get_artifact_folder(), "androidqf_encrypted")
with caplog.at_level(logging.CRITICAL):
m = SMS(
target_path=data_path,
log=logging,
module_options={"backup_password": "invalid_password"},
)
files = list_files(data_path)
parent_path = Path(data_path).absolute().parent.as_posix()
m.from_dir(parent_path, files)
run_module(m)
assert len(m.results) == 0
assert "Invalid backup password" in caplog.text
def test_androidqf_sms_encrypted_no_interactive(self, caplog):
data_path = os.path.join(get_artifact_folder(), "androidqf_encrypted")
with caplog.at_level(logging.CRITICAL):
m = SMS(
target_path=data_path,
log=logging,
module_options={"interactive": False},
)
files = list_files(data_path)
parent_path = Path(data_path).absolute().parent.as_posix()
m.from_dir(parent_path, files)
run_module(m)
assert len(m.results) == 0
assert (
"Cannot decrypt backup because interactivity was disabled and the password was not supplied"
in caplog.text
)
+26
View File
@@ -9,6 +9,7 @@ from pathlib import Path
from mvt.android.modules.bugreport.dumpsys_appops import DumpsysAppops
from mvt.android.modules.bugreport.dumpsys_getprop import DumpsysGetProp
from mvt.android.modules.bugreport.dumpsys_packages import DumpsysPackages
from mvt.android.modules.bugreport.dumpsys_receivers import DumpsysReceivers
from mvt.android.modules.bugreport.tombstones import Tombstones
from mvt.common.module import run_module
@@ -60,6 +61,31 @@ class TestBugreportAnalysis:
m = self.launch_bug_report_module(DumpsysGetProp)
assert len(m.results) == 0
def test_receivers_match_exact_package_name(self, indicators_factory):
intent = "android.intent.action.PHONE_STATE"
false_positive = {
"package_name": "com.android.phone",
"receiver": (
"com.android.phone/"
"com.android.services.telephony.sip.SipIncomingCallReceiver"
),
}
malicious_receiver = {
"package_name": "com.android.services",
"receiver": "com.android.services/com.example.SomeReceiver",
}
module = DumpsysReceivers(
results={intent: [false_positive, malicious_receiver]}
)
module.indicators = indicators_factory(app_ids=["com.android.services"])
module.check_indicators()
assert len(module.alertstore.alerts) == 1
alert = module.alertstore.alerts[0]
assert alert.event == {intent: malicious_receiver}
assert alert.matched_indicator.value == "com.android.services"
def test_tombstones_modules(self):
m = self.launch_bug_report_module(Tombstones)
assert len(m.results) == 2
Binary file not shown.
+123
View File
@@ -5,7 +5,9 @@
import logging
import os
import threading
import requests
from mvt.common.config import settings
from mvt.common.indicators import Indicators
@@ -80,6 +82,127 @@ class TestIndicators:
assert ind.check_url("https://198.51.100.1:8080/")
assert ind.check_url("https://1.1.1.1/") is None
def test_google_maps_short_url_is_not_resolved(self, indicator_file, mocker):
head_request = mocker.patch("mvt.common.url.requests.head")
ind = Indicators(log=logging)
ind.load_indicators_files([indicator_file], load_default=False)
assert ind.check_url("https://goo.gl/maps/example") is None
head_request.assert_not_called()
def test_check_url_batches_preserves_order(self, indicator_file):
ind = Indicators(log=logging)
ind.load_indicators_files([indicator_file], load_default=False)
matches = ind.check_url_batches(
[
[
"https://github.com",
"http://example.com/thisisbad",
"https://www.example.org/foobar",
],
["https://github.com", "https://www.example.org/foobar"],
[],
None,
]
)
assert matches[0]
assert matches[0].ioc.value == "http://example.com/thisisbad"
assert matches[1]
assert matches[1].ioc.value == "example.org"
assert matches[2] is None
assert matches[3] is None
def test_check_url_batches_deduplicates_and_limits_workers(
self, indicator_file, mocker
):
ind = Indicators(log=logging)
ind.load_indicators_files([indicator_file], load_default=False)
mocker.patch("mvt.common.indicators.URL_CHECK_MAX_WORKERS", 2)
barrier = threading.Barrier(2)
lock = threading.Lock()
calls = []
active = 0
max_active = 0
def head_request(url, timeout):
nonlocal active, max_active
with lock:
calls.append(url)
active += 1
max_active = max(max_active, active)
call_number = len(calls)
try:
if call_number <= 2:
barrier.wait(timeout=5)
return mocker.Mock(status_code=200, headers={})
finally:
with lock:
active -= 1
mocker.patch("mvt.common.url.requests.head", side_effect=head_request)
urls = [
"https://bit.ly/one",
"https://tinyurl.com/two",
"https://t.co/three",
]
assert ind.check_url_batches([urls, [urls[0]]]) == [None, None]
assert sorted(calls) == sorted(urls)
assert max_active == 2
def test_check_url_batches_respects_disabled_network(
self, indicator_file, mocker
):
ind = Indicators(log=logging)
ind.load_indicators_files([indicator_file], load_default=False)
mocker.patch("mvt.common.indicators.settings.NETWORK_ACCESS_ALLOWED", False)
head_request = mocker.patch("mvt.common.url.requests.head")
assert ind.check_url_batches([["https://bit.ly/example"]]) == [None]
head_request.assert_not_called()
def test_check_url_batches_handles_nested_redirects_and_request_failures(
self, indicator_file, mocker
):
ind = Indicators(log=logging)
ind.load_indicators_files([indicator_file], load_default=False)
def head_request(url, timeout):
if url == "https://bit.ly/failure":
raise requests.Timeout()
if url == "https://tinyurl.com/nested":
return mocker.Mock(
status_code=301,
headers={"Location": "https://t.co/nested"},
)
if url == "https://t.co/nested":
return mocker.Mock(
status_code=302,
headers={"Location": "https://www.example.org/landing"},
)
raise AssertionError(f"Unexpected URL: {url}")
head = mocker.patch(
"mvt.common.url.requests.head", side_effect=head_request
)
matches = ind.check_url_batches(
[["https://bit.ly/failure"], ["https://tinyurl.com/nested"]]
)
assert matches[0] is None
assert matches[1]
assert matches[1].ioc.value == "example.org"
assert {call.args[0] for call in head.call_args_list} == {
"https://bit.ly/failure",
"https://tinyurl.com/nested",
"https://t.co/nested",
}
def test_check_file_hash(self, indicator_file):
ind = Indicators(log=logging)
ind.load_indicators_files([indicator_file], load_default=False)
+28
View File
@@ -0,0 +1,28 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2026 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from io import StringIO
from mvt.common.password import _readline_with_asterisks
def test_readline_with_asterisks():
output = StringIO()
password = _readline_with_asterisks(
output, StringIO("pass\x7fword\n"), "Enter backup password: "
)
assert password == "pasword"
assert output.getvalue() == "Enter backup password: ****\b \b****"
def test_readline_with_asterisks_ignores_nul_and_handles_eof():
output = StringIO()
password = _readline_with_asterisks(output, StringIO("a\x00b\x04\x04"), "")
assert password == "ab"
assert output.getvalue() == "**"
+24
View File
@@ -0,0 +1,24 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import pytest
from mvt.common.url import URL
@pytest.mark.parametrize(
"url",
[
"https://goo.gl/maps/example",
"http://goo.gl/maps/example?entry=message",
"goo.gl/maps/example",
],
)
def test_google_maps_url_is_not_shortened(url):
assert URL(url).check_if_shortened() is False
def test_other_google_short_url_is_shortened():
assert URL("https://goo.gl/example").check_if_shortened() is True
+2
View File
@@ -35,6 +35,7 @@ def indicators_factory(indicator_file):
domains=[],
emails=[],
file_names=[],
file_paths=[],
processes=[],
app_ids=[],
app_cert_hashes=[],
@@ -47,6 +48,7 @@ def indicators_factory(indicator_file):
ind.ioc_collections[0]["domains"].extend(domains)
ind.ioc_collections[0]["emails"].extend(emails)
ind.ioc_collections[0]["file_names"].extend(file_names)
ind.ioc_collections[0]["file_paths"].extend(file_paths)
ind.ioc_collections[0]["processes"].extend(processes)
ind.ioc_collections[0]["app_ids"].extend(app_ids)
ind.ioc_collections[0]["android_property_names"].extend(android_property_names)
+42 -1
View File
@@ -4,12 +4,44 @@
# https://license.mvt.re/1.1/
import logging
import shutil
import pytest
from mvt.common.indicators import Indicators
from mvt.common.module import run_module
from mvt.ios.modules.mixed.safari_browserstate import SafariBrowserState
from ..utils import get_ios_backup_folder
from ..utils import add_backup_manifest_entry, get_ios_backup_folder
# fileID of HomeDomain::Library/Safari/BrowserState.db in the test backup.
DEFAULT_BROWSER_STATE_FILE_ID = "3a47b0981ed7c10f3e2800aa66bac96a3b5db28e"
PROFILE_UUID = "00000000-0000-4000-A000-000000000001"
PROFILE_BROWSER_STATE_FILE_ID = "bb00000000000000000000000000000000000001"
@pytest.fixture
def backup_with_safari_profile(tmp_path):
"""An iTunes backup where a Safari profile has its own browser state."""
backup_path = tmp_path / "backup"
shutil.copytree(get_ios_backup_folder(), backup_path)
profile_db = (
backup_path / PROFILE_BROWSER_STATE_FILE_ID[:2] / PROFILE_BROWSER_STATE_FILE_ID
)
profile_db.parent.mkdir(parents=True, exist_ok=True)
shutil.copyfile(
backup_path / DEFAULT_BROWSER_STATE_FILE_ID[:2] / DEFAULT_BROWSER_STATE_FILE_ID,
profile_db,
)
add_backup_manifest_entry(
backup_path,
PROFILE_BROWSER_STATE_FILE_ID,
"AppDomain-com.apple.mobilesafari",
f"Library/Safari/Profiles/{PROFILE_UUID}/BrowserState.db",
)
return str(backup_path)
class TestSafariBrowserStateModule:
@@ -21,6 +53,15 @@ class TestSafariBrowserStateModule:
assert len(m.timeline) == 1
assert len(m.alertstore.alerts) == 0
def test_parsing_backup_with_profile(self, backup_with_safari_profile):
m = SafariBrowserState(target_path=backup_with_safari_profile)
m.is_backup = True
run_module(m)
# Both the default profile and the named profile are extracted.
assert len(m.results) == 2
assert len({result["safari_browser_state_db"] for result in m.results}) == 2
def test_detection(self, indicator_file):
m = SafariBrowserState(target_path=get_ios_backup_folder())
m.is_backup = True
+165
View File
@@ -0,0 +1,165 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
import shutil
import sqlite3
from pathlib import Path
import pytest
from mvt.common.indicators import Indicators
from mvt.common.module import run_module
from mvt.ios.modules.mixed.safari_history import SafariHistory
from ..utils import add_backup_manifest_entry, get_ios_backup_folder
# fileID of HomeDomain::Library/Safari/History.db in the test backup.
DEFAULT_HISTORY_FILE_ID = "1a0e7afc19d307da602ccdcece51af33afe92c53"
PROFILE_UUID = "00000000-0000-4000-A000-000000000001"
PROFILE_HISTORY_FILE_ID = "aa00000000000000000000000000000000000001"
# example.org is already a test indicator, so use domains that do not match.
DEFAULT_URL = "https://default.example.net/visited-page"
PROFILE_URL = "https://profile.example.net/visited-page"
def create_history_db(path, url):
"""Create a minimal Safari History.db holding a single visit."""
path.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(path)
conn.executescript(
"""
CREATE TABLE history_items (id INTEGER PRIMARY KEY, url TEXT);
CREATE TABLE history_visits (
id INTEGER PRIMARY KEY,
history_item INTEGER,
visit_time REAL,
redirect_source INTEGER,
redirect_destination INTEGER
);
"""
)
conn.execute("INSERT INTO history_items VALUES (1, ?);", (url,))
conn.execute("INSERT INTO history_visits VALUES (1, 1, 726100000.0, NULL, NULL);")
conn.commit()
conn.close()
@pytest.fixture
def backup_with_safari_profile(tmp_path):
"""An iTunes backup where Safari has both a default and a named profile."""
backup_path = tmp_path / "backup"
shutil.copytree(get_ios_backup_folder(), backup_path)
# The default profile's database ships empty, so give it a visit to make
# sure the profile lookup does not replace the pre-existing one.
create_history_db(
backup_path / DEFAULT_HISTORY_FILE_ID[:2] / DEFAULT_HISTORY_FILE_ID,
DEFAULT_URL,
)
create_history_db(
backup_path / PROFILE_HISTORY_FILE_ID[:2] / PROFILE_HISTORY_FILE_ID,
PROFILE_URL,
)
add_backup_manifest_entry(
backup_path,
PROFILE_HISTORY_FILE_ID,
"AppDomain-com.apple.mobilesafari",
f"Library/Safari/Profiles/{PROFILE_UUID}/History.db",
)
return str(backup_path)
@pytest.fixture
def fs_dump_with_safari_profile(tmp_path):
"""A filesystem dump where Safari has both a default and a named profile."""
safari_path = tmp_path / "private" / "var" / "mobile" / "Library" / "Safari"
profile_path = safari_path / "Profiles" / PROFILE_UUID
profile_path.mkdir(parents=True)
create_history_db(safari_path / "History.db", DEFAULT_URL)
create_history_db(profile_path / "History.db", PROFILE_URL)
return str(tmp_path)
class TestSafariHistoryModule:
def test_parsing(self):
m = SafariHistory(target_path=get_ios_backup_folder())
m.is_backup = True
run_module(m)
assert len(m.results) == 0
assert len(m.alertstore.alerts) == 0
def test_parsing_backup_with_profile(self, backup_with_safari_profile):
m = SafariHistory(target_path=backup_with_safari_profile)
m.is_backup = True
run_module(m)
# Both the default profile and the named profile are extracted.
assert len(m.results) == 2
assert {result["url"] for result in m.results} == {DEFAULT_URL, PROFILE_URL}
assert len({result["safari_history_db"] for result in m.results}) == 2
def test_parsing_fs_dump_with_profile(self, fs_dump_with_safari_profile):
m = SafariHistory(target_path=fs_dump_with_safari_profile)
m.is_fs_dump = True
run_module(m)
assert len(m.results) == 2
assert {result["url"] for result in m.results} == {DEFAULT_URL, PROFILE_URL}
def test_redirect_ids_are_scoped_to_database(self, fs_dump_with_safari_profile):
safari_path = (
Path(fs_dump_with_safari_profile)
/ "private"
/ "var"
/ "mobile"
/ "Library"
/ "Safari"
)
with sqlite3.connect(safari_path / "History.db") as conn:
conn.execute(
"UPDATE history_items SET url = ? WHERE id = 1;",
("http://safe.example.com/start",),
)
conn.execute(
"INSERT INTO history_items VALUES (2, ?);",
("https://safe.example.com/end",),
)
conn.execute(
"UPDATE history_visits SET redirect_destination = 2 WHERE id = 1;"
)
conn.execute(
"INSERT INTO history_visits VALUES (2, 2, 726100000.1, 1, NULL);"
)
profile_db = safari_path / "Profiles" / PROFILE_UUID / "History.db"
with sqlite3.connect(profile_db) as conn:
# Visit IDs are local to each database and commonly overlap.
conn.execute("UPDATE history_visits SET id = 2 WHERE id = 1;")
m = SafariHistory(target_path=fs_dump_with_safari_profile)
m.is_fs_dump = True
run_module(m)
assert len(m.results) == 3
assert len(m.alertstore.alerts) == 0
def test_detection_in_profile(self, backup_with_safari_profile, indicator_file):
"""An indicator only visited inside a Safari profile still alerts."""
m = SafariHistory(target_path=backup_with_safari_profile)
m.is_backup = True
ind = Indicators(log=logging.getLogger())
ind.parse_stix2(indicator_file)
ind.ioc_collections[0]["domains"].append("profile.example.net")
m.indicators = ind
run_module(m)
assert len(m.alertstore.alerts) == 1
assert m.alertstore.alerts[0].event["url"] == PROFILE_URL
+28
View File
@@ -29,3 +29,31 @@ class TestSMSModule:
m.indicators = ind
run_module(m)
assert len(m.alertstore.alerts) == 1
def test_detection_batches_urls_and_preserves_event(self, indicator_file, mocker):
results = [
{
"text": "first",
"links": ["http://example.com/thisisbad"],
},
{
"text": "second",
"links": ["https://github.com"],
},
]
m = SMS(results=results)
ind = Indicators(log=logging.getLogger())
ind.parse_stix2(indicator_file)
batch_check = mocker.spy(ind, "check_url_batches")
m.indicators = ind
m.check_indicators()
batch_check.assert_called_once_with(
[
["http://example.com/thisisbad"],
["https://github.com"],
]
)
assert len(m.alertstore.alerts) == 1
assert m.alertstore.alerts[0].event is results[0]
+59
View File
@@ -0,0 +1,59 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from mvt.common.module import run_module
from mvt.ios.modules.fs.shutdownlog import ShutdownLog
def _shutdown_log_entry(pid: int, client: str, timestamp: int) -> str:
return (
f"remaining client pid: {pid} ({client})\n"
f"SIGTERM: [{timestamp}]\n"
)
class TestShutdownLog:
def test_discovers_rotated_shutdown_logs(self, tmp_path):
diagnostics_path = tmp_path / "private/var/db/diagnostics"
diagnostics_path.mkdir(parents=True)
(diagnostics_path / "shutdown.log").write_text(
_shutdown_log_entry(100, "/usr/libexec/first", 1_700_000_000),
encoding="utf-8",
)
(diagnostics_path / "shutdown.0.log").write_text(
_shutdown_log_entry(200, "/usr/libexec/second", 1_700_000_001),
encoding="utf-8",
)
module = ShutdownLog(target_path=str(tmp_path))
run_module(module)
assert {result["client"] for result in module.results} == {
"/usr/libexec/first",
"/usr/libexec/second",
}
def test_file_path_indicator_matches_client_with_trailing_uuid(
self, indicators_factory
):
executable_path = "/usr/sbin/filecoordinationd"
client = f"{executable_path}/123e4567-e89b-12d3-a456-426614174000"
module = ShutdownLog(
results=[
{
"isodate": "2023-11-14 22:13:20.000000",
"pid": "100",
"client": client,
"delay": 0.0,
"times_delayed": 0,
}
]
)
module.indicators = indicators_factory(file_paths=[executable_path])
module.check_indicators()
assert len(module.alertstore.alerts) == 1
assert module.alertstore.alerts[0].matched_indicator.value == executable_path
+9 -5
View File
@@ -27,21 +27,24 @@ class TestCheckAndroidqfCommand:
def test_check_encrypted_backup_prompt_valid(self, mocker):
"""Prompt for password on CLI"""
prompt_mock = mocker.patch(
"rich.prompt.Prompt.ask", return_value=TEST_BACKUP_PASSWORD
"mvt.android.modules.backup.helpers.prompt_password",
return_value=TEST_BACKUP_PASSWORD,
)
runner = CliRunner()
path = os.path.join(get_artifact_folder(), "androidqf_encrypted")
result = runner.invoke(check_androidqf, [path])
# Called twice, once in AnroidQF SMS module and once in Backup SMS module
assert prompt_mock.call_count == 2
# The password entered for the AndroidQF SMS module is reused by the
# nested backup command.
assert prompt_mock.call_count == 1
assert result.exit_code == 0
def test_check_encrypted_backup_cli(self, mocker):
"""Provide password as CLI argument"""
prompt_mock = mocker.patch(
"rich.prompt.Prompt.ask", return_value=TEST_BACKUP_PASSWORD
"mvt.android.modules.backup.helpers.prompt_password",
return_value=TEST_BACKUP_PASSWORD,
)
runner = CliRunner()
@@ -56,7 +59,8 @@ class TestCheckAndroidqfCommand:
def test_check_encrypted_backup_env(self, mocker):
"""Provide password as environment variable"""
prompt_mock = mocker.patch(
"rich.prompt.Prompt.ask", return_value=TEST_BACKUP_PASSWORD
"mvt.android.modules.backup.helpers.prompt_password",
return_value=TEST_BACKUP_PASSWORD,
)
os.environ["MVT_ANDROID_BACKUP_PASSWORD"] = TEST_BACKUP_PASSWORD
+6 -3
View File
@@ -20,7 +20,8 @@ class TestCheckAndroidBackupCommand:
def test_check_encrypted_backup_prompt_valid(self, mocker):
"""Prompt for password on CLI"""
prompt_mock = mocker.patch(
"rich.prompt.Prompt.ask", return_value=TEST_BACKUP_PASSWORD
"mvt.android.modules.backup.helpers.prompt_password",
return_value=TEST_BACKUP_PASSWORD,
)
runner = CliRunner()
path = os.path.join(get_artifact_folder(), "androidqf_encrypted/backup.ab")
@@ -32,7 +33,8 @@ class TestCheckAndroidBackupCommand:
def test_check_encrypted_backup_cli(self, mocker):
"""Provide password as CLI argument"""
prompt_mock = mocker.patch(
"rich.prompt.Prompt.ask", return_value=TEST_BACKUP_PASSWORD
"mvt.android.modules.backup.helpers.prompt_password",
return_value=TEST_BACKUP_PASSWORD,
)
runner = CliRunner()
@@ -60,7 +62,8 @@ class TestCheckAndroidBackupCommand:
def test_check_encrypted_backup_env(self, mocker):
"""Provide password as environment variable"""
prompt_mock = mocker.patch(
"rich.prompt.Prompt.ask", return_value=TEST_BACKUP_PASSWORD
"mvt.android.modules.backup.helpers.prompt_password",
return_value=TEST_BACKUP_PASSWORD,
)
os.environ["MVT_ANDROID_BACKUP_PASSWORD"] = TEST_BACKUP_PASSWORD
+17
View File
@@ -3,6 +3,8 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import shutil
from click.testing import CliRunner
from mvt.ios.cli import check_backup
@@ -16,3 +18,18 @@ class TestCheckBackupCommand:
path = get_ios_backup_folder()
result = runner.invoke(check_backup, [path])
assert result.exit_code == 0
def test_check_finds_backup_in_subfolder(self, tmp_path, caplog):
runner = CliRunner()
backup_path = tmp_path / "MobileSync" / "Backup" / "device-id"
shutil.copytree(get_ios_backup_folder(), backup_path)
result = runner.invoke(check_backup, [str(backup_path.parent)])
assert result.exit_code == 0
assert f"Found iTunes backup in subfolder: {backup_path}" in caplog.text
def test_check_rejects_non_backup_folder(self, tmp_path, caplog):
runner = CliRunner()
result = runner.invoke(check_backup, [str(tmp_path)])
assert result.exit_code == 1
assert "does not appear to be an iTunes backup folder" in caplog.text
+78
View File
@@ -0,0 +1,78 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from click.testing import CliRunner
from mvt.android.cli import cli as android_cli
from mvt.ios.cli import cli as ios_cli
class TestCompletionCommand:
def test_completion_prints_instructions_by_default(self):
runner = CliRunner()
result = runner.invoke(ios_cli, ["completion"])
assert result.exit_code == 0
assert "Shell completion for mvt-ios" in result.output
assert "mvt-ios completion bash > ~/.mvt-ios-complete.bash" in result.output
assert "Mobile Verification Toolkit" not in result.output
def test_completion_prints_bash_script(self):
runner = CliRunner()
result = runner.invoke(ios_cli, ["completion", "bash"])
assert result.exit_code == 0
assert "_MVT_IOS_COMPLETE=bash_complete" in result.output
assert "complete -o nosort" in result.output
assert "mvt-ios" in result.output
assert "Mobile Verification Toolkit" not in result.output
def test_completion_prints_fish_script(self):
runner = CliRunner()
result = runner.invoke(android_cli, ["completion", "fish"])
assert result.exit_code == 0
assert "_MVT_ANDROID_COMPLETE=fish_complete" in result.output
assert "complete --no-files --command mvt-android" in result.output
assert "Mobile Verification Toolkit" not in result.output
def test_completion_install_updates_bashrc_once(self, tmp_path, monkeypatch):
monkeypatch.setenv("HOME", str(tmp_path))
runner = CliRunner()
result = runner.invoke(ios_cli, ["completion", "bash", "--install"])
assert result.exit_code == 0
script_path = tmp_path / ".mvt-ios-complete.bash"
bashrc_path = tmp_path / ".bashrc"
assert script_path.exists()
assert "_MVT_IOS_COMPLETE=bash_complete" in script_path.read_text(
encoding="utf-8"
)
bashrc = bashrc_path.read_text(encoding="utf-8")
assert "[ -f" in bashrc
assert ".mvt-ios-complete.bash" in bashrc
result = runner.invoke(ios_cli, ["completion", "bash", "--install"])
assert result.exit_code == 0
assert bashrc_path.read_text(encoding="utf-8") == bashrc
def test_completion_install_fish_does_not_update_shell_rc(
self, tmp_path, monkeypatch
):
monkeypatch.setenv("HOME", str(tmp_path))
runner = CliRunner()
result = runner.invoke(android_cli, ["completion", "fish", "--install"])
assert result.exit_code == 0
script_path = (
tmp_path / ".config" / "fish" / "completions" / "mvt-android.fish"
)
assert script_path.exists()
assert "_MVT_ANDROID_COMPLETE=fish_complete" in script_path.read_text(
encoding="utf-8"
)
assert not (tmp_path / ".fishrc").exists()
+2
View File
@@ -63,6 +63,8 @@ def test_load_module_appears_only_for_supported_cli_command(tmp_path):
def test_module_option_runs_supported_custom_module(tmp_path):
(tmp_path / "Manifest.db").touch()
(tmp_path / "Info.plist").touch()
module_path = _write_custom_module(
tmp_path / "custom.py",
"CustomRunModule",
+18
View File
@@ -4,6 +4,7 @@
# https://license.mvt.re/1.1/
import os
import sqlite3
from pathlib import Path
@@ -37,6 +38,23 @@ def get_indicator_file():
print("PYTEST env", os.getenv("PYTEST_CURRENT_TEST"))
def add_backup_manifest_entry(backup_path, file_id, domain, relative_path):
"""
Register an extra file in a test backup's Manifest.db
"""
conn = sqlite3.connect(os.path.join(backup_path, "Manifest.db"))
conn.execute(
"INSERT INTO Files (fileID, domain, relativePath, flags, file) "
"VALUES (?, ?, ?, 1, ?);",
(file_id, domain, relative_path, b""),
)
conn.commit()
# MVT opens backup databases with immutable=1, which ignores any -wal file,
# so the new row has to be checkpointed into Manifest.db itself.
conn.execute("PRAGMA journal_mode=DELETE;")
conn.close()
def delete_tmp_db_files(file_path):
"""
Remove Sqlite temporary files that appear on some platforms