feat: add browserdata/datautil helpers (#513)

* feat: add browserdata/datautil helpers (QuerySQLite, QueryRows, DecryptChromiumValue)

Phase 2 of architecture refactoring (RFC-002 Section 3):

- datautil/sqlite.go: QuerySQLite() — shared SQLite open/query/scan helper
  with optional journal_mode=off for Firefox databases
- datautil/query.go: QueryRows[T]() — generic helper (Go 1.20) that wraps
  QuerySQLite and collects results into a typed slice
- datautil/decrypt.go: DecryptChromiumValue() — unified Chromium decryption
  (DPAPI first, then AES-GCM/CBC fallback)
- datautil/sqlite_test.go: tests for all helpers

* refactor: move DecryptChromiumValue from datautil to browser/chromium

- Remove browserdata/datautil/decrypt.go (Chromium-specific, not a generic util)
- Will be added as browser/chromium/decrypt.go (unexported decryptValue)
  in the chromium extract methods PR
- Update RFCs to reflect the change
- Remove decrypt test from datautil tests

* refactor: move datautil to utils/sqliteutil for consistency

- Rename browserdata/datautil/ → utils/sqliteutil/
- Aligns with existing utils/ convention (fileutil, typeutil, byteutil)
- QuerySQLite/QueryRows are generic SQLite helpers, not browserdata-specific
- Update package name from datautil to sqliteutil
- Update both RFCs to reflect new location

* fix: apply review suggestions for sqliteutil

- QuerySQLite: validate dbPath exists before sql.Open to prevent
  silently creating empty databases
- Tests: check db.Close() errors with require.NoError
This commit is contained in:
Roger
2026-03-24 23:27:31 +08:00
committed by moonD4rk
parent b680d43caa
commit e86e3e62d6
5 changed files with 231 additions and 36 deletions
+9 -10
View File
@@ -41,6 +41,7 @@ hackbrowserdata/
│ │ ├── chromium_windows.go # platform key retriever wiring
│ │ ├── chromium_linux.go # platform key retriever wiring
│ │ ├── source.go # chromiumSources, yandexSources maps
│ │ ├── decrypt.go # decryptValue() — Chromium-specific DPAPI/AES fallback
│ │ ├── extract_password.go # extractPasswords() + default SQL query
│ │ ├── extract_cookie.go # extractCookies() + default SQL query
│ │ ├── extract_history.go # extractHistories() + default SQL query
@@ -70,11 +71,6 @@ hackbrowserdata/
│ ├── browserdata.go # BrowserData struct (typed slices)
│ ├── output.go # BrowserData.Output() — CSV/JSON writer
│ ├── output_test.go
│ │
│ └── datautil/
│ ├── sqlite.go # QuerySQLite() helper
│ ├── query.go # queryRows[T]() generic helper (Go 1.20)
│ └── decrypt.go # DecryptChromiumValue() helper
├── crypto/
│ ├── crypto.go # AESCBCDecrypt, AESGCMDecrypt, DES3, PKCS5
@@ -114,6 +110,9 @@ hackbrowserdata/
├── fileutil/
│ ├── fileutil.go # renamed from filetutil.go
│ └── fileutil_test.go
├── sqliteutil/
│ ├── sqlite.go # QuerySQLite() helper
│ └── query.go # QueryRows[T]() generic helper (Go 1.20)
├── typeutil/
│ ├── typeutil.go
│ └── typeutil_test.go
@@ -126,7 +125,7 @@ hackbrowserdata/
| Change | Current | Target |
|--------|---------|--------|
| **New** `browserdata/datautil/` | — | SQLite + decrypt helpers |
| **New** `utils/sqliteutil/` | — | QuerySQLite + QueryRows[T] helpers |
| **New** `filemanager/` | — | Session-based temp file management |
| **New** `crypto/keyretriever/` | — | Master key retrieval abstraction |
| **New** `crypto/version.go` | — | Cipher version detection |
@@ -155,9 +154,9 @@ hackbrowserdata/
| Strategy chain | `keyretriever` | `ChainRetriever` | `keyretriever.go` |
| Cipher version | `crypto` | `CipherVersion` | `version.go` |
| Temp file session | `filemanager` | `Session` | `session.go` |
| SQLite helper | `datautil` | `QuerySQLite` (func) | `sqlite.go` |
| Generic query helper | `datautil` | `queryRows[T]` (func) | `query.go` |
| Decrypt helper | `datautil` | `DecryptChromiumValue` (func) | `decrypt.go` |
| SQLite helper | `sqliteutil` | `QuerySQLite` (func) | `sqlite.go` |
| Generic query helper | `sqliteutil` | `QueryRows[T]` (func) | `query.go` |
| Chromium decrypt | `chromium` | `decryptValue` (unexported func) | `decrypt.go` |
### Public vs private
@@ -751,7 +750,7 @@ data.Output(dir, b.Name(), format) // output whatever succeeded
| Phase | Scope | Risk |
|-------|-------|------|
| 1 | `types/category.go` + `types/models.go` + `browserdata/browserdata.go` | Zero — new files only |
| 2 | `browserdata/datautil/sqlite.go` + `decrypt.go` | Zero — new files only |
| 2 | `utils/sqliteutil/sqlite.go` + `query.go` | Zero — new files only |
| 3 | `crypto/version.go`, rename `AESCBCDecrypt` | Low — internal crypto changes |
| 4 | `crypto/keyretriever/` | Low — new package |
| 5 | `browser/chromium/source.go` + `extract_*.go` | Medium — new extract methods |
@@ -289,12 +289,12 @@ func platformBrowsers() []Config {
---
## 3. Shared Helpers: `browserdata/datautil/`
## 3. Shared Helpers: `utils/sqliteutil/`
### 3.1 SQLite query helper
```go
// browserdata/datautil/sqlite.go
// utils/sqliteutil/sqlite.go
func QuerySQLite(dbPath string, journalOff bool, query string, scanFn func(*sql.Rows) error) error {
db, err := sql.Open("sqlite", dbPath)
@@ -322,7 +322,7 @@ func QuerySQLite(dbPath string, journalOff bool, query string, scanFn func(*sql.
### 3.2 Generic query helper — `datautil/query.go`
```go
package datautil
package sqliteutil
// queryRows is a generic helper (Go 1.20) that wraps QuerySQLite
// and collects results into a typed slice. Each extract method
@@ -341,21 +341,7 @@ func QueryRows[T any](path string, journalOff bool, query string, scanRow func(*
### 3.3 Chromium decrypt helper
```go
// browserdata/datautil/decrypt.go
func DecryptChromiumValue(masterKey, encrypted []byte) ([]byte, error) {
if len(encrypted) == 0 { return nil, nil }
if len(masterKey) == 0 {
return crypto.DecryptWithDPAPI(encrypted)
}
value, err := crypto.DecryptWithDPAPI(encrypted)
if err != nil {
value, err = crypto.DecryptWithChromium(masterKey, encrypted)
}
return value, err
}
```
Moved to `browser/chromium/decrypt.go` as an unexported function `decryptValue()`. It is Chromium-specific (DPAPI → AES-GCM/CBC fallback) and only used by Chromium extract methods. See RFC-001 for details.
---
@@ -371,7 +357,7 @@ Each extract method lives in its own `extract_*.go` file inside the browser engi
const defaultLoginQuery = `SELECT origin_url, username_value, password_value, date_created FROM logins`
func (c *Chromium) extractPasswords(masterKey []byte, path string) ([]types.LoginEntry, error) {
logins, err := datautil.QueryRows(path, false, c.query(types.Password),
logins, err := sqliteutil.QueryRows(path, false, c.query(types.Password),
func(rows *sql.Rows) (types.LoginEntry, error) {
var url, username string
var pwd []byte
@@ -379,7 +365,7 @@ func (c *Chromium) extractPasswords(masterKey []byte, path string) ([]types.Logi
if err := rows.Scan(&url, &username, &pwd, &created); err != nil {
return types.LoginEntry{}, err
}
password, _ := datautil.DecryptChromiumValue(masterKey, pwd)
password, _ := decryptValue(masterKey, pwd)
return types.LoginEntry{
URL: url,
Username: username,
@@ -406,7 +392,7 @@ const defaultCookieQuery = `SELECT name, encrypted_value, host_key, path,
has_expires, is_persistent FROM cookies`
func (c *Chromium) extractCookies(masterKey []byte, path string) ([]types.CookieEntry, error) {
cookies, err := datautil.QueryRows(path, false, c.query(types.Cookie),
cookies, err := sqliteutil.QueryRows(path, false, c.query(types.Cookie),
func(rows *sql.Rows) (types.CookieEntry, error) {
var (
name, host, path string
@@ -420,7 +406,7 @@ func (c *Chromium) extractCookies(masterKey []byte, path string) ([]types.Cookie
return types.CookieEntry{}, err
}
value, _ := datautil.DecryptChromiumValue(masterKey, encryptedValue)
value, _ := decryptValue(masterKey, encryptedValue)
return types.CookieEntry{
Name: name,
Host: host,
@@ -503,7 +489,7 @@ const firefoxCookieQuery = `SELECT name, value, host, path,
creationTime, expiry, isSecure, isHttpOnly FROM moz_cookies`
func (f *Firefox) extractCookies(path string) ([]types.CookieEntry, error) {
cookies, err := datautil.QueryRows(path, true, firefoxCookieQuery,
cookies, err := sqliteutil.QueryRows(path, true, firefoxCookieQuery,
func(rows *sql.Rows) (types.CookieEntry, error) {
var (
name, value, host, path string
@@ -777,8 +763,8 @@ func formatFilename(browserName, dataName, format string) string {
1. `types/category.go` — Category enum
2. `types/models.go` — all *Entry structs
3. `browserdata/browserdata.go` — BrowserData struct
4. `browserdata/datautil/sqlite.go` — QuerySQLite()
5. `browserdata/datautil/decrypt.go`DecryptChromiumValue()
4. `utils/sqliteutil/sqlite.go` — QuerySQLite()
5. `browser/chromium/decrypt.go`decryptValue() (Chromium-specific, unexported)
6. `filemanager/session.go` — Session
### Phase 2: Extract methods (new files, coexist with old code)
@@ -839,5 +825,5 @@ GOOS=darwin GOARCH=amd64 go build ./cmd/hack-browser-data/
| File source mapping | — | covered |
| File acquisition | — | covered |
| Extract methods | — | covered |
| datautil helpers | — | covered |
| sqliteutil helpers | — | covered |
| Output | — | covered |