Commit Graph
1230 Commits
Author SHA1 Message Date
Donncha Ó Cearbhaill 30c11f68c7 Add WhatsApp contacts module and fix InteractionC contact resolution (#882)
* Add WhatsappContacts module to extract WhatsApp disappearing messages state

WhatsApp on iOS stores the disappearing messages timer for 1:1 chats on
the contact records in ContactsV2.sqlite, not in ChatStorage.sqlite. Add
a new WhatsappContacts module which extracts contact records from this
database, including phone numbers, WhatsApp and LID identifiers, and the
per-contact disappearing messages duration, and emits a timeline event
when a disappearing messages timer was set.

The database is often missing from incremental backups, so the module
logs a clear warning and returns no results instead of failing. Columns
are selected based on the actual table schema to tolerate changes across
WhatsApp versions, and if the disappearing messages column is absent the
state is reported as unknown rather than off.

The test fixture is a synthetic ContactsV2.sqlite with fictional
contacts, stored under the backup file ID derived from the WhatsApp
shared app group domain.

* Fix InteractionC contact resolution and resolve WhatsApp LIDs to contacts

The two primary InteractionC queries contained a SQL syntax error in
their direction CASE expression (a double column alias), so they always
failed and the module silently fell back to a reduced query without the
recipient join. As a result outgoing messages were serialized with no
counterpart at all ("from None (None)"). Fix the syntax so recipient
names and identifiers are extracted again, and normalize the raw 0/1
direction values from the fallback queries to INCOMING/OUTGOING.

WhatsApp identifies chat peers in interactionC.db by LID and stores the
peer LID in the domain identifier, which InteractionC could not map to a
person. Declare a dependency on the WhatsappContacts module and resolve
sender, recipient and domain identifiers (LID, JID or phone number)
against the WhatsApp contacts database, adding resolved phone number and
name fields to WhatsApp records.

Rewrite the timeline serialization to use the resolved values, fall back
to the chat peer from the domain identifier when no recipient was
recorded, label the local user instead of printing None, and include the
message direction and group name.

* Add timeline events for all WhatsApp contact timestamps

Extract ZABOUTEXPIRATIONTIMESTAMP and emit a timeline event for each
timestamp stored on a WhatsApp contact record: disappearing messages
timer changes, "about" text changes and scheduled expiry, and contact
record updates. ContactsV2.sqlite stores no other date attributes in
any released schema version.

* Add first and last interaction timeline events for WhatsApp chats

Extract one record per ZWACHATSESSION with the first and last stored
message dates, the session's own last-message date, the group creation
date and message counts. Each chat produces chat_first_message and
chat_last_message timeline events, and groups a group_created event.
The session last-message date is preferred over the newest stored
message because it survives message deletion.

* Resolve WhatsApp LID chat identifiers via the LID pair table

Recent WhatsApp versions key 1:1 chat sessions by an opaque LID rather
than the contact's phone number. Extract the ZWAPHONENUMBERLIDPAIR
table from the dedicated LID.sqlite database (or from ChatStorage
itself in versions that store it there) and use it to populate
partner_resolved_phone_number on chat session records and in timeline
events, without requiring the often-missing ContactsV2.sqlite. Each
pair is also extracted as a record and produces a lid_pair_recorded
timeline event marking when the association was learned.

* Reduce duplicate InteractionC timeline events

The interaction record's creation date normally trails its start date
by milliseconds, so serializing both nearly doubled the timeline with
duplicate entries. Only emit the creation date when it diverges from
the start date by more than an hour, with explicit wording, since a
record created long after its event indicates backfill by sync,
restore or tampering.

Per-contact aggregate dates from ZCONTACTS repeat on every interaction
row of the same contact and carried that row's message text. Serialize
them with contact-centric data strings instead, so timeline
de-duplication collapses them into one first/last-seen event per
contact.
2026-08-19 14:07:27 +02:00
besendorfandDonncha Ó Cearbhaill a92ec7f963 Build Docker images for ARM64 (#880)
Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
2026-08-19 11:46:48 +02:00
Donncha Ó Cearbhaill 10a5bccbe0 Fix usability nits in the Docker images (#881)
* Fix error in ADB key generation commmand

* Add useful forensic analysis tools
2026-08-19 11:43:40 +02:00
github-actions[bot]andDonnchaC 3b21c9347c Add new iOS versions and build numbers (#879)
Co-authored-by: DonnchaC <3081375+DonnchaC@users.noreply.github.com>
2026-08-18 07:04:35 +02:00
besendorf 0fdc2c34b0 Document iOS device access from Docker (#875)
* Document iOS device access from Docker

* Simplify docker.md instructions

Removed unnecessary note about mounting the entire '/var/run' directory.
2026-08-17 13:42:22 +02:00
besendorf 0ee25edf0a Fix dumpsys package system flag parsing (#874) 2026-08-14 15:58:05 +02:00
besendorf fa24b5465b Scope package fields to the primary user (#872) 2026-08-14 14:33:28 +02:00
besendorf 683b8ba133 Parse package installer from bugreports (#868) 2026-08-14 09:40:34 +02:00
besendorf 7b64463727 Update GitHub Actions (#866) 2026-08-12 15:42:15 +02:00
github-actions[bot]andDonnchaC 47fd771b2a Add new iOS versions and build numbers (#865)
Co-authored-by: DonnchaC <3081375+DonnchaC@users.noreply.github.com>
2026-08-11 10:07:52 +02:00
besendorf 8123db26d9 Fix iOS update workflow and refresh dependencies (#864) 2026-08-10 22:06:15 +02:00
besendorf d92a60c9be Preserve tombstone crash causes (#863) 2026-08-10 20:59:56 +02:00
besendorf 0b48d9fe1d Speed up compressed sysdiagnose analysis (#861)
* Speed up compressed sysdiagnose analysis

* ci: retrigger Ruff check
2026-08-07 09:07:33 +02:00
besendorf 067f053627 Add extensible CLI commands (#853)
* Add extensible CLI commands

* Handle plugin SystemExit failures
2026-08-05 23:30:28 +02:00
besendorf 93b7fb5232 Store message URLs in analysis output (#856) 2026-08-05 23:21:14 +02:00
besendorf 8617e0bf54 Alert on AndroidQF trusted ADB keys (#860) 2026-08-05 17:36:49 +02:00
Rory Flynn 53fb12aee8 Correct version.py (#855) v2026.7.29 2026-07-29 16:08:42 +02:00
Rory Flynn 8140e350f7 Update version.py (#854) 2026-07-29 13:57:34 +02:00
besendorf f483223e23 Add iOS sysdiagnose checking (#832)
* Add iOS sysdiagnose checking

* Clarify documentation navigation
2026-07-28 18:59:58 +02:00
besendorf 2dfe3cbcb1 Fix module audit findings (#850)
* Fix module audit findings

* Always parse paired tombstones
2026-07-28 18:58:46 +02:00
besendorf 3eff0c550d Handle mis-indented dumpsys receiver actions (#852) 2026-07-28 18:34:13 +02:00
besendorf 797411e1e5 Fix text tombstone crashing thread parsing (#848) 2026-07-27 17:58:34 +02:00
84df51c518 Scan Safari profile databases for history and browser state (#846)
* fix(ios): scan Safari profile databases for history and browser state

Safari profiles (iOS 17 and later) keep their own databases under
Library/Safari/Profiles/<UUID>/, but SafariHistory and SafariBrowserState
only ever looked at the default profile's Library/Safari/History.db and
Library/Safari/BrowserState.db.

On a device where browsing happens inside a profile, MVT silently skipped
that history and still reported no detections, so an indicator only ever
visited within a profile went unnoticed.

Both modules now also match Library/Safari/Profiles/*/ in backups and in
filesystem dumps. No helper changes were needed: the Manifest.db lookup
already translates "*" into a SQL LIKE wildcard, and the filesystem lookup
already globs.

Found while examining an encrypted iOS 26.5.2 backup that contained 14
per-profile History.db files under
AppDomain-com.apple.mobilesafari::Library/Safari/Profiles/<UUID>/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(ios): scope Safari redirects to history database

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Janik Besendorf <janik@besendorf.org>
2026-07-27 15:06:44 +02:00
besendorf a6c3a805d8 Parallelize URL indicator checks (#844) 2026-07-27 15:03:02 +02:00
besendorf 123c9081ed Skip resolving Google Maps short URLs (#843) 2026-07-19 17:13:57 +02:00
FelixandJanik Besendorf 5ed8b3c1a5 fix: terminate dumpsys adb multiline values at structural lines (#842)
* fix: terminate dumpsys adb multiline values at structural lines

* fix dumpsys ADB multiline boundaries

---------

Co-authored-by: Janik Besendorf <janik@besendorf.org>
2026-07-17 18:25:47 +02:00
besendorf afcfda4720 Deduplicate AndroidQF SMS analysis (#837) 2026-07-15 09:09:04 +02:00
besendorf 911115b0c8 Match receiver indicators by package ID (#836) 2026-07-15 08:32:13 +02:00
besendorf c806fd8d61 Support rotated iOS shutdown logs (#834) 2026-07-14 23:27:44 +02:00
besendorf 516ba06cf7 Suppress benign PinStorage key generation warning (#835) 2026-07-14 23:20:53 +02:00
besendorf 1532578b39 Validate iOS backup path before checks (#825)
* Validate iOS backup path before checks

* Fix iOS backup path mypy typing

* Fix custom module test backup fixture
2026-07-14 21:58:46 +02:00
Nimrod B.andJanik Besendorf f5b0a3cd91 WIP: Addition of a timer to virustotal checks (#593)
* Add delay option to virustotal checks (#408)

* Fix missing delay argument

* Fix VirusTotal delay handling

* Fix mypy type for VirusTotal package map

---------

Co-authored-by: Janik Besendorf <janik@besendorf.org>
2026-07-01 12:36:56 +02:00
besendorf a18e632ec8 Add shell completion command (#817) 2026-07-01 12:35:21 +02:00
besendorf 2689176c0e Add custom module loading (#816) 2026-07-01 12:33:38 +02:00
besendorf 638937e838 Handle malformed AndroidQF backups (#824) 2026-06-24 12:41:00 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> b9f13b8146 Bump pydantic-settings from 2.13.1 to 2.14.0
Bumps [pydantic-settings](https://github.com/pydantic/pydantic-settings) from 2.13.1 to 2.14.0.
- [Release notes](https://github.com/pydantic/pydantic-settings/releases)
- [Commits](https://github.com/pydantic/pydantic-settings/compare/v2.13.1...v2.14.0)

---
updated-dependencies:
- dependency-name: pydantic-settings
  dependency-version: 2.14.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-17 18:17:22 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 58558fdfb7 Bump click from 8.3.2 to 8.3.3
Bumps [click](https://github.com/pallets/click) from 8.3.2 to 8.3.3.
- [Release notes](https://github.com/pallets/click/releases)
- [Changelog](https://github.com/pallets/click/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/click/compare/8.3.2...8.3.3)

---
updated-dependencies:
- dependency-name: click
  dependency-version: 8.3.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-17 18:16:14 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 7c7e332e7b Bump ruff from 0.9.10 to 0.15.16
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.9.10 to 0.15.16.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.9.10...0.15.16)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.15.12
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-17 18:12:58 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> a232c988f9 Bump packaging from 26.0 to 26.2
Bumps [packaging](https://github.com/pypa/packaging) from 26.0 to 26.2.
- [Release notes](https://github.com/pypa/packaging/releases)
- [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pypa/packaging/compare/26.0...26.2)

---
updated-dependencies:
- dependency-name: packaging
  dependency-version: '26.2'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-17 18:11:51 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 2f4123222f Bump tld from 0.13.1 to 0.13.2
Bumps [tld](https://github.com/barseghyanartur/tld) from 0.13.1 to 0.13.2.
- [Release notes](https://github.com/barseghyanartur/tld/releases)
- [Changelog](https://github.com/barseghyanartur/tld/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/barseghyanartur/tld/compare/0.13.1...0.13.2)

---
updated-dependencies:
- dependency-name: tld
  dependency-version: 0.13.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-17 18:09:37 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 81a7bc977b Bump pyahocorasick from 2.2.0 to 2.3.1
Bumps [pyahocorasick](https://github.com/WojciechMula/pyahocorasick) from 2.2.0 to 2.3.1.
- [Release notes](https://github.com/WojciechMula/pyahocorasick/releases)
- [Changelog](https://github.com/WojciechMula/pyahocorasick/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/WojciechMula/pyahocorasick/compare/v2.2.0...v2.3.1)

---
updated-dependencies:
- dependency-name: pyahocorasick
  dependency-version: 2.3.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-17 18:08:29 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 3b20bd944b Bump types-requests from 2.33.0.20260503 to 2.33.0.20260508
Bumps [types-requests](https://github.com/python/typeshed) from 2.33.0.20260503 to 2.33.0.20260508.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-version: 2.33.0.20260508
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-17 18:06:44 +02:00
besendorf 6bbb5957af Fix dumpsys battery daily downgrade detection (#805) 2026-06-17 17:54:05 +02:00
6a6c1758c3 intrusion_logs: alert on certificate events and run heuristics without IOCs (#811)
* intrusion_logs: alert on certificate events and run heuristics without IOCs

SecurityEvent.check_indicators() returned early when no indicator set was
loaded, so none of its heuristic alerts (key integrity, wipe failure, crypto
self-test, certificate events) reached the alert store on a default run. On
top of that, cert_authority_installed and cert_validation_failure only emitted
log.warning and never alerted even when indicators were present.

Run the heuristic alerts independently of the loaded indicators (matching the
accessibility fix in #807) and surface the two certificate events through the
alert store at medium severity. A successfully installed root CA and a
certificate validation failure are interception/MITM-relevant signals that
belong in the alert report.

Adds regression tests for both certificate events and for heuristics firing
with no indicators loaded.

* intrusion_logs: gate certificate authority install alert on success

Failed install attempts log a warning instead of raising the
"Certificate authority installed" alert. Add a regression test
covering success encoded as bool and as int.

---------

Co-authored-by: John Kavanagh <668351+kavanista@users.noreply.github.com>
Co-authored-by: besendorf <janik@besendorf.org>
2026-06-17 17:24:06 +02:00
besendorf 174ce08812 Run mypy on pull requests (#810) 2026-06-17 17:15:11 +02:00
besendorf d590706e62 Add dependency-aware module execution (#806)
* Add dependency-aware module ordering

* Annotate module run order test state
2026-06-17 17:03:47 +02:00
besendorf 08e6a0eae2 Fix intrusion log event ID parsing (#815) 2026-06-11 19:27:26 +02:00
besendorf 1e67a343ca Document network access options (#808) 2026-06-05 20:27:42 +02:00
tesandbesendorf 0a8da50c2b fix: enforce NETWORK_ACCESS_ALLOWED and timeout on URL.unshorten (#800)
Co-authored-by: besendorf <janik@besendorf.org>
2026-06-05 20:20:34 +02:00
besendorf b8331ddac8 Replace split("\n") with splitlines() for platform compatibility and other todos and chores (#746)
* Replace split("\n") with splitlines() for platform compatibility

* Remove dead commented-out code in webkit_session_resource_log

* Remove stale FIXME comment in command.py

* Narrow bare except to specific exception types in convert_mactime_to_datetime

* Fix typo in aqf_files.py comment

* Refactor b64 encoding in configuration_profiles into helper methods

* Pass branch parameter to GitHub commits API in update checker

* Replace bare KeyError catch with explicit key check in net_base

* Remove confirmed Chrome database path TODOs

Backup IDs verified via SHA-1 of AppDomain-com.google.chrome.ios paths.

* Extract additional timestamps from WebKit ObservedDomains table

Query mostRecentUserInteractionTime and mostRecentWebPushInteractionTime
with fallback to the original 4-column query for older iOS versions.

* Clarify command_line list format matches protobuf schema in tombstone parser

* Support SHA1 and MD5 hash matching in AQF files module

* Remove resolved TODO about --output requirement in download-apks

* Clean up code TODOs and type checks

* Fix WebKit timestamp schema handling
2026-06-05 20:15:56 +02:00