13 Commits

Author SHA1 Message Date
Roger ecf8ba0585 refactor(browser): simplify credential storage config (#593) 2026-05-14 16:29:35 +08:00
Roger 0c6c781567 feat(yandex): password and credit card decryption (#585) 2026-04-23 17:00:09 +08:00
Roger 7a5db25b4f feat(safari): localstorage extraction (#582)
* feat(safari): localstorage extraction

Extracts Safari 17+ localStorage from WebKit's nested layout —
WebsiteDataStore/<uuid>/Origins/<top-hash>/<frame-hash>/LocalStorage/
localstorage.sqlite3 for named profiles, WebsiteData/Default for the
default profile. Parses the binary SecurityOrigin serialization
(length-prefixed scheme+host plus 0x00 default-port or 0x01 <uint16_le>
explicit-port section) and decodes UTF-16 LE ItemTable value BLOBs,
capping oversized values at 2048 bytes to match the Chromium extractor.
Reports the frame origin URL so partitioned third-party storage is
attributed to the iframe origin JavaScript actually sees.

Closes the remaining LocalStorage checkbox in #565.

* docs(safari): add RFC-011 data storage

Documents Safari's profile structure, per-category file layouts, and
storage formats including the Safari 17+ nested WebKit Origins
localStorage layout and binary SecurityOrigin serialization. Defers
Keychain credential extraction to RFC-006 §7 and notes the cross-browser
differences (plaintext cookies, plist bookmarks/downloads, Core Data
epoch timestamps, partitioned storage).

* fix(safari): latin-1 origin decoding, NULL key skip, count fast-path

- Decode originEncASCII via decodeLatin1 so high-byte records preserve
  their ISO-8859-1 meaning instead of being interpreted as UTF-8.
  Matches the pattern in chromium/extract_storage.go.
- Skip ItemTable rows where key is NULL — SQLite's UNIQUE constraint
  permits multiple NULLs, and silently lowering them to empty strings
  would collide with legitimate empty-string keys.
- countLocalStorage now walks origin dirs and runs SELECT COUNT(key)
  per localstorage.sqlite3 instead of fully decoding every value.
  COUNT(key) naturally excludes NULLs, keeping count and extract
  symmetric.

Addresses Copilot review feedback on #582.

* fix(safari): round-2 review — WAL replay, stable ordering, error context

- Drop immutable=1 on temp-copy SQLite opens in readLocalStorageFile /
  countLocalStorageFile. Session.Acquire copies the -wal / -shm sidecars,
  so mode=ro alone lets SQLite replay WAL on the ephemeral copy and
  surface entries Safari committed to WAL but hasn't checkpointed yet.
  Live-file reads in profiles.go keep immutable=1 as before.
- Order ItemTable query by (key, rowid) for deterministic exports across
  runs and SQLite versions.
- Wrap os.ReadFile / os.ReadDir errors with the offending path so
  multi-origin debug logs stay scannable.
- RFC-011 §7 rewritten to explain the live-vs-temp split.
- New regression test asserts ORDER BY surfaces rows in key order.

Addresses round-2 Copilot review on #582.
2026-04-21 20:47:11 +08:00
Roger 7b9a973c9c fix: per-tier master-key retrievers for mixed-cipher profiles (#579)
* fix: per-tier master-key retrievers for mixed-cipher profiles
2026-04-20 17:33:38 +08:00
Roger ae1ec66ccb refactor(windows): split Windows code into winapi (#575) 2026-04-19 18:12:37 +08:00
Roger 76e2615db2 refactor(windows): clean up Chrome ABE module (#574)
* refactor(abe): remove --abe-key flag and its global state
* refactor(abe): rework scratch protocol and Go/C structure
2026-04-19 15:20:51 +08:00
Roger eb58ebbbf4 fix: support Linux v11 cipher prefix for Chromium decryption (#571) 2026-04-13 22:12:40 +08:00
Roger 370c5882c4 feat: add Safari password extraction from macOS Keychain (#568) 2026-04-13 21:34:40 +08:00
Roger d8032ac824 docs: rewrite readme, rfcs, and contributing (#555)
* docs: rewrite README, RFCs, and CONTRIBUTING
* docs: fix Linux storage labels in RFC-006 (Opera/Vivaldi swapped)
2026-04-06 00:16:47 +08:00
Roger 00ad0e0bd4 feat: add output package with Formatter interface (#537)
* docs: add RFC-004 for CLI (cobra) and output design
* feat: add output package with Formatter interface and BrowserData.Each
* fix: golangci config array syntax + add output package tests
* refactor: encapsulated Output as Writer, collect-then-write pattern
* refactor: unified row type with reflection-based CSV/JSON output
* fix: ProfileName empty guard, writeFile close error check, sync RFC-004
2026-04-04 01:41:02 +08:00
Roger 1a3aea553e feat: add Firefox Browser with new v2 architecture (#536)
* feat: add Firefox Browser implementation with new v2 architecture

Add Firefox NewBrowsers + Extract pipeline following the Chromium v2
pattern. Firefox-specific differences handled:
- Profile discovery: random directory names (e.g. abc123.default-release)
- Master key: NSS/ASN1PBE from key4.db (platform-agnostic, no DPAPI/Keychain)
- Key validation: reuse logins.json from acquireFiles tempPaths
- Extract: only Password needs masterKey; Cookie is plaintext
- No CreditCard or SessionStorage support

Files:
- firefox_new.go: Browser struct, NewBrowsers, Extract, getMasterKey,
  extractCategory, deriveKeys, validateKeyWithLogins, profile discovery
- masterkey.go: extracted shared NSS logic (processMasterKey, queryMetaData,
  queryNssPrivateCandidates, parseLoginCipherPairs, canDecryptAnyLoginCipherPair)
- firefox_new_test.go: table-driven tests with shared fixtures
- source.go: remove dataSource wrapper, use []sourcePath directly
- firefox.go: remove functions moved to masterkey.go

* fix: address Copilot review feedback on Firefox v2

- Fix stale comment referencing removed readLoginCipherPairs
- Rename finallyKey to derivedKey for clarity in processMasterKey
- Add sqlite driver import to masterkey.go for self-containedness

* refactor: rewrite Firefox masterkey and improve naming

Masterkey rewrite:
- Replace raw SQL functions with structured key4DB type (globalSalt,
  passwordCheck, privateKeys) for clear data modeling
- Split processMasterKey into verifyPasswordCheck + decryptPrivateKey
- Add nssKeyTypeTag constant for the magic bytes
- Rename finallyKey to derivedKey
- Add sqlite driver import for self-containedness
- Return error (not fallback) when logins validation explicitly fails

Naming cleanup:
- loginPair → encryptedLogin (clarify these are encrypted blobs)
- parseLoginPairs → sampleEncryptedLogins (clarify sampling purpose)
- canDecryptLogin → tryDecryptLogins (accurate verb, plural alignment)
- Expand abbreviated variables: p→login, uPBE→userPBE, pPBE→pwdPBE

Password extraction:
- Keep entries when decryptPBE fails (URL preserved, user/pwd empty)
- Align with Chromium behavior where decrypt failure doesn't skip records

Old code cleanup:
- firefox.go GetMasterKey now delegates to retrieveMasterKey
- Remove functions moved to masterkey.go

* docs: add RFC-003 for crypto package naming cleanup

Track accumulated naming and structural issues in crypto/asn1pbe.go
and cross-browser shared code for a future dedicated refactoring pass.

* refactor: move masterkey tests to masterkey_test.go

- Rename firefox_test.go to masterkey_test.go since all tests in
  this file test masterkey.go functions (readKey4DB, sampleEncryptedLogins)
- Fix TestReadKey4DB to check nssPrivate rows as a set instead of
  assuming SQLite insertion order
- Future deletion of firefox.go won't accidentally remove masterkey tests
2026-04-04 01:41:02 +08:00
Roger e86e3e62d6 feat: add browserdata/datautil helpers (#513)
* feat: add browserdata/datautil helpers (QuerySQLite, QueryRows, DecryptChromiumValue)

Phase 2 of architecture refactoring (RFC-002 Section 3):

- datautil/sqlite.go: QuerySQLite() — shared SQLite open/query/scan helper
  with optional journal_mode=off for Firefox databases
- datautil/query.go: QueryRows[T]() — generic helper (Go 1.20) that wraps
  QuerySQLite and collects results into a typed slice
- datautil/decrypt.go: DecryptChromiumValue() — unified Chromium decryption
  (DPAPI first, then AES-GCM/CBC fallback)
- datautil/sqlite_test.go: tests for all helpers

* refactor: move DecryptChromiumValue from datautil to browser/chromium

- Remove browserdata/datautil/decrypt.go (Chromium-specific, not a generic util)
- Will be added as browser/chromium/decrypt.go (unexported decryptValue)
  in the chromium extract methods PR
- Update RFCs to reflect the change
- Remove decrypt test from datautil tests

* refactor: move datautil to utils/sqliteutil for consistency

- Rename browserdata/datautil/ → utils/sqliteutil/
- Aligns with existing utils/ convention (fileutil, typeutil, byteutil)
- QuerySQLite/QueryRows are generic SQLite helpers, not browserdata-specific
- Update package name from datautil to sqliteutil
- Update both RFCs to reflect new location

* fix: apply review suggestions for sqliteutil

- QuerySQLite: validate dbPath exists before sql.Open to prevent
  silently creating empty databases
- Tests: check db.Close() errors with require.NoError
2026-04-04 01:41:01 +08:00
Roger 9959c0839a docs: add architecture refactoring RFCs and switch gitignore to whitelist (#510)
* docs: add architecture refactoring RFCs and switch gitignore to whitelist

- Rename rfc/ to rfcs/
- RFC-001: overall architecture redesign (data models, crypto layer,
  browser registration, CLI separation, error handling)
- RFC-002: data extraction and file acquisition refactoring
- Replace .gitignore blacklist (212 lines) with precise whitelist (43 lines)
  to prevent accidental commit of sensitive browser data files

* feat: update architecture refactoring documentation

- Refactor the architecture to improve scalability and maintainability
- Streamline browser data and file acquisition processes for efficiency

* docs(rfcs): add extract_* naming convention and queryRows[T] helper

- RFC-001: add file naming convention section explaining extract_* prefix
  grouping, add datautil/query.go for queryRows[T] generic helper
- RFC-002: update all extract examples to use datautil.QueryRows[T],
  add Section 3.2 with queryRows[T] definition

* feat: update architecture refactoring documentation

- RFC-001: rename BrowserConfig→Config, BrowsingData→Extract,
  add public/private visibility table, add isValidBrowserDir
  in PickBrowsers, remove storage from Chromium struct,
  NewChain returns KeyRetriever interface, add error wrapping
  convention, unexport PBKDF2 params, flatten log/level
- RFC-002: replace outPutter with writeFile/writeJSON/writeCSV,
  remove golang.org/x/text dependency (3-byte BOM), add Windows
  locked file handling (copyLocked), fix discoverDataFiles to
  check file vs dir type, Firefox New() takes profileDir only,
  add decryptPBE helper, add error handling section, add
  profile discovery with tests, add platform config example
2026-03-23 01:07:56 +08:00