Compare commits

...

39 Commits

Author SHA1 Message Date
Ginder Singh 2e3e5a67e1 added missing func. 2026-07-02 15:21:08 -04:00
Ginder Singh 8704db2476 Add mobile sandbox optimizations for v1.5.3
- Skip systemd-resolved initialization on Android (ChromeOS crash fix)
- Skip system DNS discovery commands on iOS mobile (sandbox restrictions)
- Skip route-based DNS discovery on Android
- Skip systemd resolver on Android
- Add mobile platform checks to prevent sandbox access violations

These changes ensure ctrld works correctly in mobile sandboxed environments
where system commands and file access are restricted.
2026-07-02 15:10:57 -04:00
Cuong Manh Le 5bf26da585 Merge pull request #318 from Control-D-Inc/release-branch-v1.5.3
Release branch v1.5.3
2026-06-22 14:40:36 +07:00
Cuong Manh Le a5d536ab79 Upgrade quic-go to v0.59.1
For fixing CVE-2026-40898.
2026-06-16 15:17:11 +07:00
Codescribe 735590d244 fix: allow intercept fallback for default listener 2026-06-16 15:05:19 +07:00
Codescribe 18f01baa01 fix: flush pf states after forced DNS intercept reload 2026-06-16 15:05:08 +07:00
Cuong Manh Le 723c7827ba fix: stop self-upgrade tests from fork-bombing the windows test runner
The test:windows CI job intermittently failed to clean up .testbin with
"Access to the path '...cmd_cli.test.exe' is denied". This was previously
attributed to Windows Defender scanning the large unsigned test binaries,
and mitigated with Defender exclusions and cleanup retries. That was
treating a symptom.

Root cause: performUpgrade() self-upgrades by running
exec.Command(os.Executable(), "upgrade", "prod", "-vv") as a detached,
windowless child. In the real ctrld binary this re-execs ctrld and is
correct. Under `go test`, os.Executable() is the test binary itself, and
`go test` stops flag parsing at the first positional arg ("upgrade") and
ignores the rest -- so the child silently re-runs the entire test suite.
That child hits the upgrade tests again and spawns more detached children,
recursively: a fork bomb of hidden processes that pins the runner's
CPU/memory and keeps the test binary's image file locked. Windows refuses
to delete the image of a running process, hence the "Access is denied"
during after_script. Whether any children are still alive when cleanup
runs is a timing race, which is why the failure was flaky.

Two tests reached this path: Test_performUpgrade (directly) and
Test_selfUpgradeCheck (via selfUpgradeCheck -> performUpgrade on the
"upgrade allowed" case).

Fix:
- prog.go: extract the command construction into a package-level
  newUpgradeCmd var. Production behavior is unchanged.
- main_test.go: stub newUpgradeCmd once in TestMain so the whole test
  binary self-execs with `-test.run=^$` (matches no tests, exits
  immediately) instead of re-running the suite. This covers every test
  that reaches performUpgrade, present and future, while still exercising
  the cmd.Start() success path.
2026-06-16 14:52:45 +07:00
Dev Scribe 1e1c998c89 Refresh macOS VPN DNS after pf stabilization 2026-06-16 14:52:28 +07:00
Cuong Manh Le da454db8ef docker: update Dockerfile to use bookworm
Stick to go1.25 for now, since using go1.26 causing a runtime panic when
building arm platforms.
2026-06-16 14:49:33 +07:00
Cuong Manh Le 3fe9b27fb4 fix(doh,doq): reject oversized upstream DNS responses
DoH, DoH3, and DoQ response paths previously used io.ReadAll on
attacker-controlled upstream responses before enforcing any protocol-level
size limit. A malicious or compromised upstream could return an oversized
body or stream and force ctrld to buffer unbounded data before eventually
failing DNS parsing.

Cap DoH/DoH3 response bodies at dns.MaxMsgSize and cap DoQ streams at the
2-byte length prefix plus dns.MaxMsgSize. Also limit non-200 DoH error
bodies so error formatting cannot consume large upstream responses.
2026-06-16 14:49:01 +07:00
Cuong Manh Le 35455eb0b9 fix(doq): share QUIC transport, close send side before read (RFC 9250)
DoQ pools now keep a single quic.Transport and UDP socket for all dials,
so parallel dial and reconnect churn no longer allocate a new socket per
attempt or leak the winner's UDP conn when the caller owns the packet
conn.

quicParallelDialer accepts an optional transport: when set, dials use
Transport.DialEarly on that socket; when nil, behavior matches the old
per-dial ListenUDP path (losers close their sockets).

Per RFC 9250 §4.2, close the query stream's send side before reading the
response so strict upstreams see STREAM FIN before answering.

CloseIdleConnections closes the shared transport and underlying UDP
conn so checked-out connections and the OS socket are torn down.

Add a FIN-strict test server, coverage for bootstrap vs parallel-dial
paths, and a Linux-only FD churn regression test.
2026-06-16 14:48:43 +07:00
Cuong Manh Le f1309121ae doq: validate DNS-over-QUIC response framing
DoQ responses are length-prefixed per RFC 9250. The resolver previously
assumed the stream always contained at least two bytes and unpacked from
buf[2:], which could panic on truncated or malicious replies.

Validate the prefix against the bytes read, return a clear error, and
retire the connection from the pool on framing failure. Unpack only the
slice declared by the prefix so a short read cannot be misinterpreted as
a full message.

Add regression coverage with a small test server that returns malformed
raw payloads (empty, one byte, prefix-only, prefix larger than payload).
2026-06-16 14:48:33 +07:00
Cuong Manh Le 06668a2b6c cmd/cli: rate-limit PIN brute-force on control socket
Currently there is no limit on PIN attempts, allowing unlimited
brute force if an attacker gains socket access. While the socket is
root-only by default, rate limiting is cheap defense-in-depth.
2026-06-16 14:47:27 +07:00
Cuong Manh Le 97e5e99b8d cmd/cli: use os.CreateTemp for symlink-safe temp file creation
Current code writes to a predictable path, which on systems without
`fs.protected_symlinks` (e.g. embedded routers) could allow a local
attacker with API compromise to perform symlink attacks.
2026-06-16 14:47:19 +07:00
Cuong Manh Le c54ff701bd internal/router/dnsmasq: use text/template instead of html/template
Since this is a plain-text config, not html.
2026-06-16 14:47:08 +07:00
Cuong Manh Le 33682e2312 all: explicit TLS MinVersion in tls.Config
Go's default is already TLS 1.2+ (since Go 1.18), but making this
explicit satisfies RFC 7858/9250 recommendations and makes the security
intent clear for auditors.
2026-06-16 14:46:42 +07:00
Cuong Manh Le d629ecda33 Merge pull request #317 from Control-D-Inc/update-ci
Update ci
2026-06-02 03:24:30 -04:00
Cuong Manh Le 87ddf03b90 .github/workflows: bump go and staticcheck version 2026-06-02 14:20:05 +07:00
Cuong Manh Le d49a4c67c9 Bump golang.org/x/net to v0.55.0
For GO-2026-5026 security fix.
2026-06-02 14:18:23 +07:00
Cuong Manh Le 2c38ff74c3 Merge pull request #316 from Control-D-Inc/release-branch-v1.5.2
Release v1.5.2
2026-06-02 03:08:55 -04:00
Cuong Manh Le 75e8447c75 test: isolate VPN DNS settling tests from host adapters 2026-06-01 16:30:34 +07:00
Codescribe 4395efcb22 fix: stabilize Windows VPN DNS during adapter settling
Fixes Windows DNS-intercept behavior for AD/internal split-rule domains
during sleep/wake or VPN adapter settling without relying on a fixed
timeout.
2026-06-01 15:41:38 +07:00
Cuong Manh Le 7e6f88b4ed Merge pull request #301 from Control-D-Inc/release-branch-v1.5.1
Release branch v1.5.1
2026-05-25 07:08:15 -04:00
Cuong Manh Le 5dd5846cca cmd/cli: skip upstream.os healthcheck when WFP loopback protect enabled
Since the check will always be failed in this case, causing unnecessary
log spamming.
2026-05-05 22:15:54 +07:00
Codescribe 2b27c148be dns: recovery race condition fix
Three changes to reduce worst-case recovery from ~30s to <3s:

1. debounceRecovery() for network changes (500ms window) — coalesces
   rapid consecutive network changes into a single recovery pass,
   eliminating the cancel-and-restart race.

2. ForceReBootstrap() on recovery entry — closes dead connections and
   creates fresh transports synchronously before probing, replacing
   the lazy ReBootstrap() flag that left stale connections.

3. Combined effect: recovery probes never inherit dead connections
   from a canceled prior recovery attempt.
2026-04-30 19:09:21 +07:00
CodeScribe 8cb383d87e dns_intercept: add WFP loopback protect for VPN block-outside-dns
When third-party VPN software (e.g., OpenVPN) installs WFP block filters via
block-outside-dns, all DNS traffic to non-tunnel interfaces is blocked —
including DNS to 127.0.0.1 (ctrld's NRPT target). This breaks DNS mode
interception because the NRPT catch-all rule routes queries to loopback,
but WFP blocks the connection before it reaches ctrld's listener.

Fix: after exhausting all NRPT recovery attempts, activate a minimal WFP
session with "hard permit" filters (FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT)
for DNS to localhost in a max-priority sublayer (weight 0xFFFF). This
overrides the VPN's block for loopback DNS only, while preserving the
VPN's DNS leak protection for all other (non-loopback) DNS traffic.

The loopback protect is:
- Only activated when NRPT probes fail (not preemptively)
- Harmless when no conflicting WFP blocks exist (permit-only, no blocks)
- Persistent until ctrld shutdown (survives VPN reconnect cycles)
- Cleaned up by the existing cleanupWFPFilters path on shutdown
2026-04-29 15:21:38 +07:00
Codescribe afed925404 log: persist internal runtime logs to disk
Add file-backed persistence to the internal logWriter so runtime logs
survive service restarts. When internal logging is enabled (CD mode,
no explicit log_path), writes are teed to both the existing in-memory
ring buffer and a rotated file on disk (ctrld.log in the home directory).

File rotation: 5MB max with 1 backup (ctrld.log.1), so max ~10MB on disk.
Log view/send now reads from the persisted files (including backup) to
provide complete history across restarts. Live tail continues to use
the in-memory subscriber mechanism unchanged.

Activation: same conditions as existing internal logging — CD mode only,
no log_path configured. No new config options or dependencies.
2026-04-29 15:12:44 +07:00
Cuong Manh Le d1ea70d688 fix: prevent panic on network change during SetSelfIP
SetSelfIP unconditionally accessed t.dhcp, but t.dhcp is only
initialized when DHCP discovery is enabled. A network change event
can fire SetSelfIP regardless of the discovery configuration,
causing a nil pointer dereference.

Guard the t.dhcp access with a nil check so the self IP is still
updated on the Table even when DHCP discovery is disabled.
2026-04-22 15:30:59 +07:00
Cuong Manh Le ed98104384 doq: use OpenStreamSync and retry on StreamLimitReachedError
Replace conn.OpenStream (non-blocking) with conn.OpenStreamSync so that
the resolver waits for the server's MAX_STREAMS credit replenishment frame
instead of immediately failing when the stream limit is temporarily
exhausted. Also retry on StreamLimitReachedError as defense-in-depth for
servers that are slow or fail to send MAX_STREAMS updates.
2026-04-13 17:56:16 +07:00
Codescribe eaa171f66f doq: configure QUIC keep-alive and retry on idle timeout
Pass a quic.Config with KeepAlivePeriod (15s) to DoQ dial calls instead
of nil, so pooled connections send periodic QUIC PINGs to stay alive and
detect dead paths proactively.

Also add IdleTimeoutError to the DoQ retry conditions alongside io.EOF,
so stale pooled connections trigger a transparent retry instead of
propagating as a query failure.
2026-04-13 17:55:57 +07:00
Cuong Manh Le 839b8236e7 docs: add known issue for daemon crashing on Merlin 2026-04-07 11:34:07 +07:00
Codescribe 3f59cdad1a fix: block IPv6 DNS in intercept mode, remove raw socket approach
IPv6 DNS interception on macOS is not feasible with current pf capabilities.
The kernel rejects sendmsg from [::1] to global unicast (EINVAL), nat on lo0
doesn't fire for route-to'd packets, raw sockets bypass routing but pf doesn't
match them against rdr state, and DIOCNATLOOK can't be used because bind()
fails for non-local addresses.

Replace all IPv6 interception code with a simple pf block rule:
  block out quick on ! lo0 inet6 proto { udp, tcp } from any to any port 53

macOS automatically retries DNS over IPv4 when IPv6 is blocked.

Changes:
- Remove rawipv6_darwin.go and rawipv6_other.go
- Remove [::1] listener spawn on macOS (needLocalIPv6Listener returns false)
- Remove IPv6 rdr, route-to, pass, and reply-to pf rules
- Add block rule for all outbound IPv6 DNS
- Update docs/pf-dns-intercept.md with what was tried and why it failed
2026-04-01 17:35:08 +07:00
Codescribe c55e2a722c fix: declare ipv6Handler as dns.Handler to match wrapIPv6Handler return type
The handler variable is dns.HandlerFunc but wrapIPv6Handler returns
dns.Handler (interface). Go's type inference picked dns.HandlerFunc
for ipv6Handler, causing a compile error on assignment. Explicit
type declaration fixes the mismatch.
2026-04-01 17:24:36 +07:00
Codescribe 22a796f673 fix: use raw IPv6 socket for DNS responses in macOS intercept mode
macOS rejects sendmsg from [::1] to global unicast IPv6 (EINVAL), and
nat on lo0 doesn't fire for route-to'd packets (pf skips translation
on the second interface pass). ULA addresses on lo0 also fail (EHOSTUNREACH
- kernel segregates lo0 routing).

Solution: wrap the [::1] UDP listener's ResponseWriter with rawIPv6Writer
that sends responses via SOCK_RAW (IPPROTO_UDP) on lo0, bypassing the
kernel's routing validation. pf's rdr state reverses the address
translation on the response path.

Changes:
- Add rawipv6_darwin.go: rawIPv6Writer wraps dns.ResponseWriter, sends
  UDP responses via raw IPv6 socket with proper checksum calculation
- Add rawipv6_other.go: no-op wrapIPv6Handler for non-darwin platforms
- Remove nat rules from pf anchor (no longer needed)
- Block IPv6 TCP DNS (block return) - falls back to IPv4 (~1s, rare)
- Remove IPv6 TCP rdr/route-to/pass rules (only UDP intercepted)
2026-04-01 17:24:17 +07:00
Codescribe 95dd871e2d fix: bracket IPv6 addresses in VPN DNS upstream config
upstreamConfigFor() used strings.Contains(":") to detect whether to
append ":53", but IPv6 addresses contain colons, so IPv6 servers were
passed as bare addresses (e.g. "2a0d:6fc0:9b0:3600::1") to net.Dial
which rejects them with "too many colons in address".

Use net.JoinHostPort() which handles both IPv4 and IPv6 correctly,
producing "[2a0d:6fc0:9b0:3600::1]:53" for IPv6.
2026-04-01 17:23:53 +07:00
Codescribe 5c0585b2e8 Add log tail command for live log streaming
This commit adds a new `ctrld log tail` subcommand that streams
runtime debug logs to the terminal in real-time, similar to `tail -f`.

Changes:
- log_writer.go: Add Subscribe/tailLastLines for fan-out to tail clients
- control_server.go: Add /log/tail endpoint with streaming response
  - Internal logging: subscribes to logWriter for live data
  - File-based logging: polls log file for new data (200ms interval)
  - Sends last N lines as initial context on connect
- commands.go: Add `log tail` cobra subcommand with --lines/-n flag
- control_client.go: Add postStream() with no timeout for long-lived connections

Usage:
  sudo ctrld log tail          # shows last 10 lines then follows
  sudo ctrld log tail -n 50    # shows last 50 lines then follows
  Ctrl+C to stop
2026-03-25 13:58:44 +07:00
Codescribe 112d1cb5a9 fix: close handle leak in hasLocalDnsServerRunning()
Add defer windows.CloseHandle(h) after CreateToolhelp32Snapshot to ensure
the process snapshot handle is properly released on all code paths (match
found, enumeration exhausted, or error).
2026-03-25 13:58:24 +07:00
Codescribe bd9bb90dd4 Fix dnsFromResolvConf not filtering loopback IPs
The continue statement only broke out of the inner loop, so
loopback/local IPs (e.g. 127.0.0.1) were never filtered.
This caused ctrld to use itself as bootstrap DNS when already
installed as the system resolver — a self-referential loop.

Use the same isLocal flag pattern as getDNSFromScutil() and
getAllDHCPNameservers().
2026-03-25 13:57:46 +07:00
Codescribe 82fc628bf3 docs: add DNS Intercept Mode section to README 2026-03-25 13:57:35 +07:00
49 changed files with 3393 additions and 330 deletions
+2 -2
View File
@@ -9,7 +9,7 @@ jobs:
fail-fast: false
matrix:
os: ["windows-latest", "ubuntu-latest", "macOS-latest"]
go: ["1.24.x"]
go: ["1.25.x"]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
@@ -21,6 +21,6 @@ jobs:
- run: "go test -race ./..."
- uses: dominikh/staticcheck-action@v1.4.0
with:
version: "2025.1.1"
version: "2026.1"
install-go: false
cache-key: ${{ matrix.go }}
+63 -1
View File
@@ -100,7 +100,7 @@ docker build -t controldns/ctrld . -f docker/Dockerfile
# Usage
The cli is self documenting, so free free to run `--help` on any sub-command to get specific usages.
The cli is self documenting, so feel free to run `--help` on any sub-command to get specific usages.
## Arguments
```
@@ -266,5 +266,67 @@ The above will start a foreground process and:
- Excluding `*.company.int` and `very-secure.local` matching queries, that are forwarded to `10.0.10.1:53`
- Write a debug log to `/path/to/log.log`
## DNS Intercept Mode
When running `ctrld` alongside VPN software, DNS conflicts can cause intermittent failures, bypassed filtering, or configuration loops. DNS Intercept Mode prevents these issues by transparently capturing all DNS traffic on the system and routing it through `ctrld`, without modifying network adapter DNS settings.
### When to Use
Enable DNS Intercept Mode if you:
- Use corporate VPN software (F5, Cisco AnyConnect, Palo Alto GlobalProtect, Zscaler)
- Run overlay networks like Tailscale or WireGuard
- Experience random DNS failures when VPN connects/disconnects
- See gaps in your Control D analytics when VPN is active
- Have endpoint security software that also manages DNS
### Command
Windows (Admin Shell)
```shell
ctrld.exe start --intercept-mode dns --cd RESOLVER_ID_HERE
```
macOS
```shell
sudo ctrld start --intercept-mode dns --cd RESOLVER_ID_HERE
```
`--intercept-mode dns` automatically detects VPN internal domains and routes them to the VPN's DNS server, while Control D handles everything else.
To disable intercept mode on a service that already has it enabled:
Windows (Admin Shell)
```shell
ctrld.exe start --intercept-mode off
```
macOS
```shell
sudo ctrld start --intercept-mode off
```
This removes the intercept rules and reverts to standard interface-based DNS configuration.
### Platform Support
| Platform | Supported | Mechanism |
|----------|-----------|-----------|
| Windows | ✅ | NRPT (Name Resolution Policy Table) |
| macOS | ✅ | pf (packet filter) redirect |
| Linux | ❌ | Not currently supported |
### Features
- **VPN split routing** — VPN-specific domains are automatically detected and forwarded to the VPN's DNS server
- **Captive portal recovery** — Wi-Fi login pages (hotels, airports, coffee shops) work automatically
- **No network adapter changes** — DNS settings stay untouched, eliminating conflicts entirely
- **Automatic port 53 conflict resolution** — if another process (e.g., `mDNSResponder` on macOS) is already using port 53, `ctrld` automatically listens on a different port. OS-level packet interception redirects all DNS traffic to `ctrld` transparently, so no manual configuration is needed. This only applies to intercept mode.
### Tested VPN Software
- F5 BIG-IP APM
- Cisco AnyConnect
- Palo Alto GlobalProtect
- Tailscale (including Exit Nodes)
- Windscribe
- WireGuard
For more details, see the [DNS Intercept Mode documentation](https://docs.controld.com/docs/dns-intercept).
## Contributing
See [Contribution Guideline](./docs/contributing.md)
+46 -7
View File
@@ -638,6 +638,19 @@ const defaultDeactivationPin = -1
// cdDeactivationPin is used in cd mode to decide whether stop and uninstall commands can be run.
var cdDeactivationPin atomic.Int64
// Brute-force protection for the deactivation PIN endpoint on the control socket.
// After deactivationMaxFailedAttempts consecutive wrong PINs, further attempts are
// rejected for deactivationLockoutSeconds. Counter resets on a correct PIN.
const (
deactivationMaxFailedAttempts = 5
deactivationLockoutSeconds = 60
)
var (
deactivationFailedAttempts atomic.Int64
deactivationLockedUntil atomic.Int64
)
func init() {
cdDeactivationPin.Store(defaultDeactivationPin)
}
@@ -1245,7 +1258,7 @@ func tryUpdateListenerConfigIntercept(cfg *ctrld.Config, notifyFunc func(), fata
return false, true
}
hasExplicitConfig := lc.IP != "" && lc.IP != "0.0.0.0" && lc.Port != 0
hasExplicitConfig := isExplicitInterceptListener(lc.IP, lc.Port)
if !hasExplicitConfig {
// Set defaults for intercept mode
if lc.IP == "" || lc.IP == "0.0.0.0" {
@@ -1303,6 +1316,16 @@ func tryUpdateListenerConfigIntercept(cfg *ctrld.Config, notifyFunc func(), fata
return updated, false
}
func isExplicitInterceptListener(ip string, port int) bool {
if ip == "" || ip == "0.0.0.0" || port == 0 {
return false
}
// 127.0.0.1:53 is the default macOS DNS-intercept listener. It can appear
// in generated/custom Control D configs, but it should still be allowed to
// fall back to 127.0.0.1:5354 when mDNSResponder already owns port 53.
return !(ip == "127.0.0.1" && port == 53)
}
// tryUpdateListenerConfig tries updating listener config with a working one.
// If fatal is true, and there's listen address conflicted, the function do
// fatal error.
@@ -1809,6 +1832,9 @@ var errInvalidDeactivationPin = errors.New("deactivation pin is invalid")
// errRequiredDeactivationPin indicates that the deactivation pin is required but not provided by users.
var errRequiredDeactivationPin = errors.New("deactivation pin is required to stop or uninstall the service")
// errTooManyDeactivationPin represents an error indicating excessive deactivation PIN request attempts.
var errTooManyDeactivationPin = errors.New("too many request attempts")
// checkDeactivationPin validates if the deactivation pin matches one in ControlD config.
func checkDeactivationPin(s service.Service, stopCh chan struct{}) error {
mainLog.Load().Debug().Msg("Checking deactivation pin")
@@ -1837,6 +1863,9 @@ func checkDeactivationPin(s service.Service, stopCh chan struct{}) error {
case http.StatusBadRequest:
mainLog.Load().Error().Msg(errRequiredDeactivationPin.Error())
return errRequiredDeactivationPin // pin is required
case http.StatusTooManyRequests:
mainLog.Load().Error().Msg(errTooManyDeactivationPin.Error())
return errTooManyDeactivationPin
case http.StatusOK:
return nil // valid pin
case http.StatusNotFound:
@@ -1849,7 +1878,9 @@ func checkDeactivationPin(s service.Service, stopCh chan struct{}) error {
// isCheckDeactivationPinErr reports whether there is an error during check deactivation pin process.
func isCheckDeactivationPinErr(err error) bool {
return errors.Is(err, errInvalidDeactivationPin) || errors.Is(err, errRequiredDeactivationPin)
return errors.Is(err, errInvalidDeactivationPin) ||
errors.Is(err, errRequiredDeactivationPin) ||
errors.Is(err, errTooManyDeactivationPin)
}
// ensureUninstall ensures that s.Uninstall will remove ctrld service from system completely.
@@ -2002,17 +2033,25 @@ func doValidateCdRemoteConfig(cdUID string, fatal bool) error {
} else {
if errors.As(cfgErr, &viper.ConfigParseError{}) {
if configStr, _ := base64.StdEncoding.DecodeString(rc.Ctrld.CustomConfig); len(configStr) > 0 {
tmpDir := os.TempDir()
tmpConfFile := filepath.Join(tmpDir, "ctrld.toml")
errorLogged := false
// Write remote config to a temporary file to get details error.
if we := os.WriteFile(tmpConfFile, configStr, 0600); we == nil {
// Write remote config to a uniquely named temporary file to get detailed error.
if tmpFile, tmpErr := os.CreateTemp("", "ctrld-*.toml"); tmpErr == nil {
tmpConfFile := tmpFile.Name()
if _, err := tmpFile.Write(configStr); err != nil {
mainLog.Load().Error().Err(err).Msg("failed to write temporary config file")
}
if err := tmpFile.Close(); err != nil {
mainLog.Load().Error().Err(err).Msg("failed to save temporary config file")
}
if de := decoderErrorFromTomlFile(tmpConfFile); de != nil {
row, col := de.Position()
mainLog.Load().Error().Msgf("failed to parse custom config at line: %d, column: %d, error: %s", row, col, de.Error())
errorLogged = true
}
_ = os.Remove(tmpConfFile)
if err := os.Remove(tmpConfFile); err != nil {
mainLog.Load().Error().Err(err).Msg("failed to remove temporary config file")
}
}
// If we could not log details error, emit what we have already got.
if !errorLogged {
+28
View File
@@ -0,0 +1,28 @@
package cli
import "testing"
func TestIsExplicitInterceptListener(t *testing.T) {
tests := []struct {
name string
ip string
port int
want bool
}{
{name: "empty", ip: "", port: 0, want: false},
{name: "wildcard", ip: "0.0.0.0", port: 53, want: false},
{name: "zero port", ip: "127.0.0.1", port: 0, want: false},
{name: "default intercept listener", ip: "127.0.0.1", port: 53, want: false},
{name: "fallback port explicit", ip: "127.0.0.1", port: 5354, want: true},
{name: "custom loopback explicit", ip: "127.0.0.2", port: 53, want: true},
{name: "custom address explicit", ip: "192.0.2.10", port: 53, want: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := isExplicitInterceptListener(tt.ip, tt.port); got != tt.want {
t.Fatalf("isExplicitInterceptListener(%q, %d) = %v, want %v", tt.ip, tt.port, got, tt.want)
}
})
}
}
+85
View File
@@ -11,12 +11,14 @@ import (
"net/http"
"os"
"os/exec"
"os/signal"
"path/filepath"
"runtime"
"slices"
"sort"
"strconv"
"strings"
"syscall"
"time"
"github.com/docker/go-units"
@@ -146,6 +148,88 @@ func initLogCmd() *cobra.Command {
fmt.Println(logs.Data)
},
}
var tailLines int
logTailCmd := &cobra.Command{
Use: "tail",
Short: "Tail live runtime debug logs",
Long: "Stream live runtime debug logs to the terminal, similar to tail -f. Press Ctrl+C to stop.",
Args: cobra.NoArgs,
PreRun: func(cmd *cobra.Command, args []string) {
checkHasElevatedPrivilege()
},
Run: func(cmd *cobra.Command, args []string) {
p := &prog{router: router.New(&cfg, false)}
s, _ := newService(p, svcConfig)
status, err := s.Status()
if errors.Is(err, service.ErrNotInstalled) {
mainLog.Load().Warn().Msg("service not installed")
return
}
if status == service.StatusStopped {
mainLog.Load().Warn().Msg("service is not running")
return
}
dir, err := socketDir()
if err != nil {
mainLog.Load().Fatal().Err(err).Msg("failed to find ctrld home dir")
}
cc := newControlClient(filepath.Join(dir, ctrldControlUnixSock))
tailPath := fmt.Sprintf("%s?lines=%d", tailLogsPath, tailLines)
resp, err := cc.postStream(tailPath, nil)
if err != nil {
mainLog.Load().Fatal().Err(err).Msg("failed to connect for log tailing")
}
defer resp.Body.Close()
switch resp.StatusCode {
case http.StatusMovedPermanently:
warnRuntimeLoggingNotEnabled()
return
case http.StatusOK:
default:
mainLog.Load().Fatal().Msgf("unexpected response status: %d", resp.StatusCode)
return
}
// Set up signal handling for clean shutdown.
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
done := make(chan struct{})
go func() {
defer close(done)
// Stream output to stdout.
buf := make([]byte, 4096)
for {
n, readErr := resp.Body.Read(buf)
if n > 0 {
os.Stdout.Write(buf[:n])
}
if readErr != nil {
if readErr != io.EOF {
mainLog.Load().Error().Err(readErr).Msg("error reading log stream")
}
return
}
}
}()
select {
case <-ctx.Done():
if errors.Is(ctx.Err(), context.Canceled) {
msg := fmt.Sprintf("\nexiting: %s\n", context.Cause(ctx).Error())
os.Stdout.WriteString(msg)
}
case <-done:
}
},
}
logTailCmd.Flags().IntVarP(&tailLines, "lines", "n", 10, "Number of historical lines to show on connect")
logCmd := &cobra.Command{
Use: "log",
Short: "Manage runtime debug logs",
@@ -156,6 +240,7 @@ func initLogCmd() *cobra.Command {
}
logCmd.AddCommand(logSendCmd)
logCmd.AddCommand(logViewCmd)
logCmd.AddCommand(logTailCmd)
rootCmd.AddCommand(logCmd)
return logCmd
+6
View File
@@ -32,6 +32,12 @@ func (c *controlClient) post(path string, data io.Reader) (*http.Response, error
return c.c.Post("http://unix"+path, contentTypeJson, data)
}
// postStream sends a POST request with no timeout, suitable for long-lived streaming connections.
func (c *controlClient) postStream(path string, data io.Reader) (*http.Response, error) {
c.c.Timeout = 0
return c.c.Post("http://unix"+path, contentTypeJson, data)
}
// deactivationRequest represents request for validating deactivation pin.
type deactivationRequest struct {
Pin int64 `json:"pin"`
+187 -3
View File
@@ -10,6 +10,7 @@ import (
"os"
"reflect"
"sort"
"strconv"
"time"
"github.com/kardianos/service"
@@ -29,6 +30,7 @@ const (
ifacePath = "/iface"
viewLogsPath = "/log/view"
sendLogsPath = "/log/send"
tailLogsPath = "/log/tail"
)
type ifaceResponse struct {
@@ -57,12 +59,18 @@ func newControlServer(addr string) (*controlServer, error) {
func (s *controlServer) start() error {
_ = os.Remove(s.addr)
unixListener, err := net.Listen("unix", s.addr)
if l, ok := unixListener.(*net.UnixListener); ok {
l.SetUnlinkOnClose(true)
}
if err != nil {
return err
}
// Restrict socket permissions to owner-only (0600) so that only the
// process owner (typically root) can connect. Defense-in-depth since
// the control server endpoints carry no authentication of their own.
if err := os.Chmod(s.addr, 0600); err != nil {
return err
}
if l, ok := unixListener.(*net.UnixListener); ok {
l.SetUnlinkOnClose(true)
}
go s.server.Serve(unixListener)
return nil
}
@@ -217,6 +225,12 @@ func (p *prog) registerControlServerHandler() {
return
}
// Reject further attempts while locked out due to repeated wrong PINs.
if now := time.Now().Unix(); now < deactivationLockedUntil.Load() {
w.WriteHeader(http.StatusTooManyRequests)
return
}
// Re-fetch pin code from API.
rcReq := &controld.ResolverConfigRequest{
RawUID: cdUID,
@@ -250,6 +264,7 @@ func (p *prog) registerControlServerHandler() {
switch req.Pin {
case cdDeactivationPin.Load():
code = http.StatusOK
deactivationFailedAttempts.Store(0)
select {
case p.pinCodeValidCh <- struct{}{}:
default:
@@ -257,6 +272,11 @@ func (p *prog) registerControlServerHandler() {
case defaultDeactivationPin:
// If the pin code was set, but users do not provide --pin, return proper code to client.
code = http.StatusBadRequest
default:
if deactivationFailedAttempts.Add(1) >= deactivationMaxFailedAttempts {
deactivationLockedUntil.Store(time.Now().Unix() + deactivationLockoutSeconds)
deactivationFailedAttempts.Store(0)
}
}
w.WriteHeader(code)
}))
@@ -344,6 +364,170 @@ func (p *prog) registerControlServerHandler() {
}
p.internalLogSent = time.Now()
}))
p.cs.register(tailLogsPath, http.HandlerFunc(func(w http.ResponseWriter, request *http.Request) {
flusher, ok := w.(http.Flusher)
if !ok {
http.Error(w, "streaming unsupported", http.StatusInternalServerError)
return
}
// Determine logging mode and validate before starting the stream.
var lw *logWriter
useInternalLog := p.needInternalLogging()
if useInternalLog {
p.mu.Lock()
lw = p.internalLogWriter
p.mu.Unlock()
if lw == nil {
w.WriteHeader(http.StatusMovedPermanently)
return
}
} else if p.cfg.Service.LogPath == "" {
// No logging configured at all.
w.WriteHeader(http.StatusMovedPermanently)
return
}
// Parse optional "lines" query param for initial context.
numLines := 10
if v := request.URL.Query().Get("lines"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n >= 0 {
numLines = n
}
}
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.Header().Set("Transfer-Encoding", "chunked")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(http.StatusOK)
if useInternalLog {
// Internal logging mode: subscribe to the logWriter.
// Send last N lines as initial context.
if numLines > 0 {
if tail := lw.tailLastLines(numLines); len(tail) > 0 {
w.Write(tail)
flusher.Flush()
}
}
ch, unsub := lw.Subscribe()
defer unsub()
for {
select {
case data, ok := <-ch:
if !ok {
return
}
if _, err := w.Write(data); err != nil {
return
}
flusher.Flush()
case <-request.Context().Done():
return
}
}
} else {
// File-based logging mode: tail the log file.
logFile := normalizeLogFilePath(p.cfg.Service.LogPath)
f, err := os.Open(logFile)
if err != nil {
// Already committed 200, just return.
return
}
defer f.Close()
// Seek to show last N lines.
if numLines > 0 {
if tail := tailFileLastLines(f, numLines); len(tail) > 0 {
w.Write(tail)
flusher.Flush()
}
} else {
// Seek to end.
f.Seek(0, io.SeekEnd)
}
// Poll for new data.
buf := make([]byte, 4096)
ticker := time.NewTicker(200 * time.Millisecond)
defer ticker.Stop()
for {
select {
case <-ticker.C:
n, err := f.Read(buf)
if n > 0 {
if _, werr := w.Write(buf[:n]); werr != nil {
return
}
flusher.Flush()
}
if err != nil && err != io.EOF {
return
}
case <-request.Context().Done():
return
}
}
}
}))
}
// tailFileLastLines reads the last n lines from a file and returns them.
// The file position is left at the end of the file after this call.
func tailFileLastLines(f *os.File, n int) []byte {
stat, err := f.Stat()
if err != nil || stat.Size() == 0 {
return nil
}
// Read from the end in chunks to find the last n lines.
const chunkSize = 4096
fileSize := stat.Size()
var lines []byte
offset := fileSize
count := 0
for offset > 0 && count <= n {
readSize := int64(chunkSize)
if readSize > offset {
readSize = offset
}
offset -= readSize
buf := make([]byte, readSize)
nRead, err := f.ReadAt(buf, offset)
if err != nil && err != io.EOF {
break
}
buf = buf[:nRead]
lines = append(buf, lines...)
// Count newlines in this chunk.
for _, b := range buf {
if b == '\n' {
count++
}
}
}
// Trim to last n lines.
idx := 0
nlCount := 0
for i := len(lines) - 1; i >= 0; i-- {
if lines[i] == '\n' {
nlCount++
if nlCount == n+1 {
idx = i + 1
break
}
}
}
lines = lines[idx:]
// Seek to end of file for subsequent reads.
f.Seek(0, io.SeekEnd)
return lines
}
func jsonResponse(next http.Handler) http.Handler {
+46 -53
View File
@@ -321,14 +321,13 @@ func (p *prog) startDNSIntercept() error {
// options → normalization (scrub) → queueing → translation (nat/rdr) → filtering (pass/block/anchor)
//
// "pfctl -sr" returns BOTH scrub-anchor (normalization) AND anchor/pass/block (filter) rules.
// "pfctl -sn" returns nat-anchor AND rdr-anchor (translation) rules.
// "pfctl -sn" returns rdr-anchor (translation) rules.
// Both commands emit "No ALTQ support in kernel" warnings on stderr.
//
// We must reassemble in correct order: scrub → nat/rdr → filter.
//
// The anchor reference does not survive a reboot, but ctrld re-adds it on every start.
func (p *prog) ensurePFAnchorReference() error {
natAnchorRef := fmt.Sprintf("nat-anchor \"%s\"", pfAnchorName)
rdrAnchorRef := fmt.Sprintf("rdr-anchor \"%s\"", pfAnchorName)
anchorRef := fmt.Sprintf("anchor \"%s\"", pfAnchorName)
@@ -347,11 +346,10 @@ func (p *prog) ensurePFAnchorReference() error {
natLines := pfFilterRuleLines(string(natOut))
filterLines := pfFilterRuleLines(string(filterOut))
hasNatAnchor := pfContainsRule(natLines, natAnchorRef)
hasRdrAnchor := pfContainsRule(natLines, rdrAnchorRef)
hasAnchor := pfContainsRule(filterLines, anchorRef)
if hasNatAnchor && hasRdrAnchor && hasAnchor {
if hasRdrAnchor && hasAnchor {
// Verify anchor ordering: our anchor should appear before other anchors
// for reliable DNS interception priority. Log a warning if out of order,
// but don't force a reload (the interface-specific rules in our anchor
@@ -380,15 +378,8 @@ func (p *prog) ensurePFAnchorReference() error {
// rules in whichever anchor appears first win. By prepending, our DNS
// intercept rules match port 53 traffic before a VPN app's broader
// "pass out quick on <iface> all" rules in their anchor.
if !hasNatAnchor || !hasRdrAnchor {
var newRefs []string
if !hasNatAnchor {
newRefs = append(newRefs, natAnchorRef)
}
if !hasRdrAnchor {
newRefs = append(newRefs, rdrAnchorRef)
}
natLines = append(newRefs, natLines...)
if !hasRdrAnchor {
natLines = append([]string{rdrAnchorRef}, natLines...)
}
if !hasAnchor {
pureFilterLines = append([]string{anchorRef}, pureFilterLines...)
@@ -590,7 +581,6 @@ func (p *prog) stopDNSIntercept() error {
// The anchor itself is already flushed by stopDNSIntercept, so even if removal
// fails, the empty anchor is a no-op.
func (p *prog) removePFAnchorReference() error {
natAnchorRef := fmt.Sprintf("nat-anchor \"%s\"", pfAnchorName)
rdrAnchorRef := fmt.Sprintf("rdr-anchor \"%s\"", pfAnchorName)
anchorRef := fmt.Sprintf("anchor \"%s\"", pfAnchorName)
@@ -609,7 +599,7 @@ func (p *prog) removePFAnchorReference() error {
var cleanNat []string
for _, line := range natLines {
if !strings.Contains(line, rdrAnchorRef) && !strings.Contains(line, natAnchorRef) {
if !strings.Contains(line, rdrAnchorRef) {
cleanNat = append(cleanNat, line)
}
}
@@ -804,23 +794,13 @@ func (p *prog) buildPFAnchorRules(vpnExemptions []vpnDNSExemption) string {
// a stateful entry that handles response routing. Using "rdr pass" would skip filter
// evaluation, and its implicit state alone is insufficient for response delivery —
// proven by commit 51cf029 where responses were silently dropped.
rules.WriteString("# --- Translation rules (nat + rdr) ---\n")
// NAT source to ::1 for IPv6 DNS on loopback. macOS/BSD rejects sendmsg from
// [::1] to a global unicast IPv6 address (EINVAL), unlike IPv4 where sendmsg from
// 127.0.0.1 to local private IPs works fine. The rdr rewrites the destination but
// preserves the original source (machine's global IPv6). Without nat, ctrld cannot
// reply. pf reverses both translations on the response path.
// Note: nat must appear before rdr (pf evaluates nat first in translation phase).
listenerAddr6 := fmt.Sprintf("::1 port %d", listenerPort)
rules.WriteString("nat on lo0 inet6 proto udp from ! ::1 to ! ::1 port 53 -> ::1\n")
rules.WriteString("nat on lo0 inet6 proto tcp from ! ::1 to ! ::1 port 53 -> ::1\n")
rules.WriteString("# --- Translation rules (rdr) ---\n")
rules.WriteString("# Redirect DNS on loopback to ctrld's listener.\n")
rules.WriteString(fmt.Sprintf("rdr on lo0 inet proto udp from any to ! %s port 53 -> %s\n", listenerIP, listenerAddr))
rules.WriteString(fmt.Sprintf("rdr on lo0 inet proto tcp from any to ! %s port 53 -> %s\n", listenerIP, listenerAddr))
rules.WriteString(fmt.Sprintf("rdr on lo0 inet6 proto udp from any to ! ::1 port 53 -> %s\n", listenerAddr6))
rules.WriteString(fmt.Sprintf("rdr on lo0 inet6 proto tcp from any to ! ::1 port 53 -> %s\n\n", listenerAddr6))
// No IPv6 rdr — IPv6 DNS is blocked at the filter level (see below).
rules.WriteString("\n")
// --- Filtering rules ---
rules.WriteString("# --- Filtering rules (pass) ---\n\n")
@@ -982,8 +962,7 @@ func (p *prog) buildPFAnchorRules(vpnExemptions []vpnDNSExemption) string {
}
rules.WriteString(fmt.Sprintf("pass out quick on %s route-to lo0 inet proto udp from any to ! %s port 53\n", iface, listenerIP))
rules.WriteString(fmt.Sprintf("pass out quick on %s route-to lo0 inet proto tcp from any to ! %s port 53\n", iface, listenerIP))
rules.WriteString(fmt.Sprintf("pass out quick on %s route-to lo0 inet6 proto udp from any to ! ::1 port 53\n", iface))
rules.WriteString(fmt.Sprintf("pass out quick on %s route-to lo0 inet6 proto tcp from any to ! ::1 port 53\n", iface))
// No IPv6 route-to — IPv6 DNS is blocked, not intercepted.
}
rules.WriteString("\n")
}
@@ -1003,10 +982,14 @@ func (p *prog) buildPFAnchorRules(vpnExemptions []vpnDNSExemption) string {
rules.WriteString(fmt.Sprintf("pass out quick on ! lo0 route-to lo0 inet proto udp from any to ! %s port 53\n", listenerIP))
rules.WriteString(fmt.Sprintf("pass out quick on ! lo0 route-to lo0 inet proto tcp from any to ! %s port 53\n\n", listenerIP))
// Force remaining outbound IPv6 DNS through loopback for interception.
rules.WriteString("# Force remaining outbound IPv6 DNS through loopback for interception.\n")
rules.WriteString("pass out quick on ! lo0 route-to lo0 inet6 proto udp from any to ! ::1 port 53\n")
rules.WriteString("pass out quick on ! lo0 route-to lo0 inet6 proto tcp from any to ! ::1 port 53\n\n")
// Block all outbound IPv6 DNS. ctrld only intercepts IPv4 DNS via the loopback
// redirect. IPv6 DNS interception on macOS is not feasible because the kernel rejects
// sendmsg from [::1] to global unicast IPv6 (EINVAL), and pf's nat-on-lo0 doesn't
// fire for route-to'd packets. Blocking forces macOS to fall back to IPv4 DNS,
// which is fully intercepted. See docs/pf-dns-intercept.md for details.
rules.WriteString("# Block outbound IPv6 DNS — ctrld intercepts IPv4 only.\n")
rules.WriteString("# macOS falls back to IPv4 DNS automatically.\n")
rules.WriteString("block out quick on ! lo0 inet6 proto { udp, tcp } from any to any port 53\n\n")
// Allow route-to'd DNS packets to pass outbound on lo0.
// Without this, VPN firewalls with "block drop all" (e.g., Windscribe) drop the packet
@@ -1018,8 +1001,8 @@ func (p *prog) buildPFAnchorRules(vpnExemptions []vpnDNSExemption) string {
rules.WriteString("# Pass route-to'd DNS outbound on lo0 — no state to avoid bypassing rdr inbound.\n")
rules.WriteString(fmt.Sprintf("pass out quick on lo0 inet proto udp from any to ! %s port 53 no state\n", listenerIP))
rules.WriteString(fmt.Sprintf("pass out quick on lo0 inet proto tcp from any to ! %s port 53 no state\n", listenerIP))
rules.WriteString("pass out quick on lo0 inet6 proto udp from any to ! ::1 port 53 no state\n")
rules.WriteString("pass out quick on lo0 inet6 proto tcp from any to ! ::1 port 53 no state\n\n")
// No IPv6 lo0 pass — IPv6 DNS is blocked, not routed through lo0.
rules.WriteString("\n")
// Allow the redirected traffic through on loopback (inbound after rdr).
//
@@ -1034,7 +1017,7 @@ func (p *prog) buildPFAnchorRules(vpnExemptions []vpnDNSExemption) string {
// (source 127.0.0.1 → original DNS server IP, e.g., 10.255.255.3).
rules.WriteString("# Accept redirected DNS — reply-to lo0 forces response through loopback.\n")
rules.WriteString(fmt.Sprintf("pass in quick on lo0 reply-to lo0 inet proto { udp, tcp } from any to %s\n", listenerAddr))
rules.WriteString(fmt.Sprintf("pass in quick on lo0 reply-to lo0 inet6 proto { udp, tcp } from any to %s\n", listenerAddr6))
// No IPv6 pass-in — IPv6 DNS is blocked, not redirected to [::1].
return rules.String()
}
@@ -1043,12 +1026,11 @@ func (p *prog) buildPFAnchorRules(vpnExemptions []vpnDNSExemption) string {
// It verifies both the anchor references in the main ruleset and the rules within
// our anchor. Failures are logged at ERROR level to make them impossible to miss.
func (p *prog) verifyPFState() {
natAnchorRef := fmt.Sprintf("nat-anchor \"%s\"", pfAnchorName)
rdrAnchorRef := fmt.Sprintf("rdr-anchor \"%s\"", pfAnchorName)
anchorRef := fmt.Sprintf("anchor \"%s\"", pfAnchorName)
verified := true
// Check main ruleset for anchor references (nat-anchor + rdr-anchor in translation rules).
// Check main ruleset for anchor references (rdr-anchor in translation rules).
natOut, err := exec.Command("pfctl", "-sn").CombinedOutput()
if err != nil {
mainLog.Load().Error().Err(err).Msg("DNS intercept: VERIFICATION FAILED — could not dump NAT rules")
@@ -1059,10 +1041,6 @@ func (p *prog) verifyPFState() {
mainLog.Load().Error().Msg("DNS intercept: VERIFICATION FAILED — rdr-anchor reference missing from running NAT rules")
verified = false
}
if !strings.Contains(natStr, natAnchorRef) {
mainLog.Load().Error().Msg("DNS intercept: VERIFICATION FAILED — nat-anchor reference missing from running NAT rules")
verified = false
}
}
filterOut, err := exec.Command("pfctl", "-sr").CombinedOutput()
@@ -1305,6 +1283,10 @@ func (p *prog) pfStabilizationLoop(ctx context.Context, stableRequired time.Dura
p.pfStabilizing.Store(false)
mainLog.Load().Info().Msgf("DNS intercept: pf stable for %s — restoring anchor rules", stableRequired)
p.ensurePFAnchorActive()
routes, domainlessServers, exemptions := p.refreshDNSAfterVPNSettle("pf_stabilized")
if routes == 0 && domainlessServers == 0 && exemptions == 0 {
p.scheduleDNSAfterVPNSettleRefresh("pf_stabilized_followup", pfAnchorRecheckDelayLong)
}
p.pfLastRestoreTime.Store(time.Now().UnixMilli())
return
}
@@ -1347,7 +1329,6 @@ func (p *prog) ensurePFAnchorActive() bool {
}
}
natAnchorRef := fmt.Sprintf("nat-anchor \"%s\"", pfAnchorName)
rdrAnchorRef := fmt.Sprintf("rdr-anchor \"%s\"", pfAnchorName)
anchorRef := fmt.Sprintf("anchor \"%s\"", pfAnchorName)
needsRestore := false
@@ -1363,10 +1344,6 @@ func (p *prog) ensurePFAnchorActive() bool {
mainLog.Load().Warn().Msg("DNS intercept watchdog: rdr-anchor reference missing from running ruleset")
needsRestore = true
}
if !strings.Contains(natStr, natAnchorRef) {
mainLog.Load().Warn().Msg("DNS intercept watchdog: nat-anchor reference missing from running ruleset")
needsRestore = true
}
if !needsRestore {
filterOut, err := exec.Command("pfctl", "-sr").CombinedOutput()
@@ -1472,6 +1449,15 @@ func (p *prog) ensurePFAnchorActive() bool {
return true
}
func (p *prog) scheduleDNSAfterVPNSettleRefresh(reason string, delay time.Duration) {
time.AfterFunc(delay, func() {
if p.dnsInterceptState == nil {
return
}
p.refreshDNSAfterVPNSettle(reason)
})
}
// pfWatchdog periodically checks that our pf anchor is still active.
// Other programs (e.g., Windscribe desktop app, macOS configd) can replace
// scheduleDelayedRechecks schedules delayed re-checks after a network change event.
@@ -1762,7 +1748,6 @@ func (p *prog) pfInterceptMonitor() {
// The reload is safe for VPN interop because it reassembles from the current running
// ruleset (pfctl -sr/-sn), preserving all existing anchors and rules.
func (p *prog) forceReloadPFMainRuleset() {
natAnchorRef := fmt.Sprintf("nat-anchor \"%s\"", pfAnchorName)
rdrAnchorRef := fmt.Sprintf("rdr-anchor \"%s\"", pfAnchorName)
anchorRef := fmt.Sprintf("anchor \"%s\"", pfAnchorName)
@@ -1793,9 +1778,6 @@ func (p *prog) forceReloadPFMainRuleset() {
}
// Ensure our anchor references are present (they may have been wiped).
if !pfContainsRule(natLines, natAnchorRef) {
natLines = append([]string{natAnchorRef}, natLines...)
}
if !pfContainsRule(natLines, rdrAnchorRef) {
natLines = append([]string{rdrAnchorRef}, natLines...)
}
@@ -1841,8 +1823,19 @@ func (p *prog) forceReloadPFMainRuleset() {
mainLog.Load().Error().Err(err).Msgf("DNS intercept: force reload — failed to load anchor (output: %s)", strings.TrimSpace(string(out)))
}
// Reset upstream transports — pf reload flushes state table, killing DoH connections.
// Flush stale rdr/reply states after the forced ruleset + anchor reload.
// Without this, macOS can keep using pre-reload state and try to send
// redirected DNS replies directly from loopback to tunnel client addresses
// (for example, 127.0.0.1:<listener> -> 100.64.0.0/10), which fails with
// "sendmsg: can't assign requested address".
flushPFStates()
// Reset upstream transports — pf reload/state flush kills existing DoH connections.
p.resetUpstreamTransports()
mainLog.Load().Info().Msg("DNS intercept: force reload — pf ruleset and anchor reloaded successfully")
}
// osHealthcheckSuppressed always returns false on darwin — WFP loopback
// protect (the trigger for suppression) is Windows-only.
func (p *prog) osHealthcheckSuppressed() bool { return false }
+4
View File
@@ -37,3 +37,7 @@ func (p *prog) scheduleDelayedRechecks() {}
// pfInterceptMonitor is a no-op on unsupported platforms.
func (p *prog) pfInterceptMonitor() {}
// osHealthcheckSuppressed always returns false on non-Windows platforms —
// WFP loopback protect (the trigger for suppression) is Windows-only.
func (p *prog) osHealthcheckSuppressed() bool { return false }
+50
View File
@@ -0,0 +1,50 @@
package cli
import "github.com/Control-D-Inc/ctrld"
var initializeOsResolver = ctrld.InitializeOsResolver
func (p *prog) refreshDNSAfterVPNSettle(reason string) (routes, domainlessServers, exemptions int) {
mainLog.Load().Info().Msgf("DNS intercept: refreshing OS/VPN DNS route state after VPN settle (%s)", reason)
ns := initializeOsResolver(true)
mainLog.Load().Debug().Msgf("DNS intercept: post-settle OS resolver nameservers: %v", ns)
if p.vpnDNS == nil {
mainLog.Load().Debug().Msg("DNS intercept: post-settle VPN DNS route refresh skipped — manager unavailable")
return 0, 0, 0
}
beforeExemptions := p.vpnDNS.CurrentExemptions()
routes, domainlessServers, exemptions = p.vpnDNS.RefreshRoutesOnly()
afterExemptions := p.vpnDNS.CurrentExemptions()
if vpnDNSExemptionsEqual(beforeExemptions, afterExemptions) {
mainLog.Load().Info().Msgf("DNS intercept: post-settle VPN DNS route refresh completed — %d routes, %d domainless servers, %d exemptions (pf unchanged)",
routes, domainlessServers, exemptions)
return routes, domainlessServers, exemptions
}
if err := p.exemptVPNDNSServers(afterExemptions); err != nil {
mainLog.Load().Warn().Err(err).Msg("DNS intercept: post-settle VPN DNS exemption update failed")
} else {
mainLog.Load().Info().Msgf("DNS intercept: post-settle VPN DNS exemptions changed — updated pf/WFP with %d exemptions", len(afterExemptions))
}
return routes, domainlessServers, exemptions
}
func vpnDNSExemptionsEqual(a, b []vpnDNSExemption) bool {
if len(a) != len(b) {
return false
}
seen := make(map[vpnDNSExemption]int, len(a))
for _, ex := range a {
seen[ex]++
}
for _, ex := range b {
if seen[ex] == 0 {
return false
}
seen[ex]--
}
return true
}
+49
View File
@@ -0,0 +1,49 @@
package cli
import (
"context"
"testing"
"github.com/Control-D-Inc/ctrld"
)
func TestRefreshDNSAfterVPNSettleRefreshesOSResolverAndVPNRoutes(t *testing.T) {
oldInitialize := initializeOsResolver
defer func() { initializeOsResolver = oldInitialize }()
var initialized []bool
initializeOsResolver = func(force bool) []string {
initialized = append(initialized, force)
return []string{"10.102.26.10:53"}
}
var exemptionUpdates [][]vpnDNSExemption
p := &prog{}
p.vpnDNS = newVPNDNSManager(func(exemptions []vpnDNSExemption) error {
exemptionUpdates = append(exemptionUpdates, append([]vpnDNSExemption{}, exemptions...))
return nil
})
p.vpnDNS.discoverVPNDNS = func(context.Context) []ctrld.VPNDNSConfig {
return []ctrld.VPNDNSConfig{{
InterfaceName: "utun4",
Servers: []string{"10.102.26.10"},
Domains: []string{"bmwgroup.net"},
}}
}
routes, domainlessServers, exemptions := p.refreshDNSAfterVPNSettle("test")
if routes != 1 || domainlessServers != 0 || exemptions != 1 {
t.Fatalf("expected 1 route, 0 domainless servers, 1 exemption, got routes=%d domainless=%d exemptions=%d",
routes, domainlessServers, exemptions)
}
if len(initialized) != 1 || !initialized[0] {
t.Fatalf("expected forced OS resolver refresh once, got %v", initialized)
}
if got := p.vpnDNS.UpstreamForDomain("jira.cc.bmwgroup.net."); len(got) != 1 || got[0] != "10.102.26.10" {
t.Fatalf("expected refreshed VPN DNS route, got %v", got)
}
if len(exemptionUpdates) != 0 {
t.Fatalf("expected route-only refresh to avoid pf exemption updates, got %+v", exemptionUpdates)
}
}
+284 -21
View File
@@ -9,6 +9,7 @@ import (
"net"
"os/exec"
"runtime"
"sync"
"sync/atomic"
"time"
"unsafe"
@@ -118,6 +119,14 @@ const (
// DNS port.
dnsPort uint16 = 53
// FWPM_FILTER_FLAG constants from fwpmtypes.h.
// See: https://learn.microsoft.com/en-us/windows/win32/api/fwpmtypes/ns-fwpmtypes-fwpm_filter0
//
// FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT (0x08) prevents lower-weight sublayers
// from overriding this filter's PERMIT action ("hard permit"). Used in DNS
// mode to override third-party WFP blocks (e.g., OpenVPN's block-outside-dns).
fwpmFilterFlagClearActionRight uint32 = 0x00000008
)
// WFP API structures. These mirror the C structures from fwpmtypes.h and fwptypes.h.
@@ -258,6 +267,17 @@ type wfpState struct {
listenerIP string
// stopCh is used to shut down the NRPT health monitor goroutine.
stopCh chan struct{}
// mu protects loopbackProtectActive, loopbackPermitIDs, and engineHandle
// from concurrent access between nrptProbeAndHeal (goroutine) and
// stopDNSIntercept / cleanupWFPFilters (main goroutine).
mu sync.Mutex
// loopbackProtectActive is true when DNS mode has activated a minimal WFP
// session to permit loopback DNS. This counters third-party WFP block filters
// (e.g., OpenVPN's block-outside-dns) that prevent NRPT from routing queries
// to ctrld's listener on 127.0.0.1. See issue #526.
loopbackProtectActive bool
// loopbackPermitIDs stores the filter IDs for the loopback protect permits.
loopbackPermitIDs []uint64
}
// Lazy-loaded WFP DLL procedures.
@@ -607,6 +627,17 @@ func (p *prog) startDNSIntercept() error {
}
} else {
mainLog.Load().Info().Msg("DNS intercept: dns mode — NRPT only, no WFP filters (graceful)")
// Proactively add loopback WFP permit filters to protect the NRPT
// → 127.0.0.1 path from third-party DNS block filters (e.g., OpenVPN's
// block-outside-dns). These are narrowly scoped (port 53 to localhost
// only) and use CLEAR_ACTION_RIGHT to override any block from other
// sublayers. Adding them at startup eliminates the DNS outage window
// that would otherwise occur between VPN connect and reactive activation.
if err := p.activateLoopbackWFPProtect(state); err != nil {
// Non-fatal: loopback protect is a defense-in-depth measure.
// NRPT still works when no third-party WFP blocks are present.
mainLog.Load().Warn().Err(err).Msg("DNS intercept: failed to activate proactive loopback WFP protect — will retry on probe failure")
}
}
p.dnsInterceptState = state
@@ -878,14 +909,13 @@ func (p *prog) addWFPPermitLocalhostFilter(engineHandle uintptr, name string, la
return filterID, nil
}
// addWFPPermitSubnetFilter adds a WFP filter that permits outbound DNS to a given
// IPv4 subnet (addr/mask in host byte order). Used to exempt RFC1918 and CGNAT ranges
// so VPN DNS servers on private IPs are not blocked.
func (p *prog) addWFPPermitSubnetFilter(engineHandle uintptr, name string, proto uint8, addr, mask uint32) (uint64, error) {
// addWFPPermitDNSFilter is the unified helper for adding a WFP permit filter for
// outbound DNS (port 53) with caller-specified address condition, flags, and weight.
// Both subnet permits (RFC1918/CGNAT, flags=0, weight=10) and hard loopback permits
// (CLEAR_ACTION_RIGHT, weight=15) use this to avoid code drift.
func (p *prog) addWFPPermitDNSFilter(engineHandle uintptr, name string, layerKey windows.GUID, proto uint8, addrCond fwpmFilterCondition0, flags uint32, weight uint8) (uint64, error) {
filterName, _ := windows.UTF16PtrFromString("ctrld: " + name)
addrMask := fwpV4AddrAndMask{addr: addr, mask: mask}
conditions := make([]fwpmFilterCondition0, 3)
conditions[0] = fwpmFilterCondition0{
@@ -902,22 +932,18 @@ func (p *prog) addWFPPermitSubnetFilter(engineHandle uintptr, name string, proto
conditions[1].condValue.valueType = fwpUint16
conditions[1].condValue.value = uint64(dnsPort)
conditions[2] = fwpmFilterCondition0{
fieldKey: fwpmConditionIPRemoteAddress,
matchType: fwpMatchEqual,
}
conditions[2].condValue.valueType = fwpV4AddrMask
conditions[2].condValue.value = uint64(uintptr(unsafe.Pointer(&addrMask)))
conditions[2] = addrCond
filter := fwpmFilter0{
layerKey: fwpmLayerALEAuthConnectV4,
flags: flags,
layerKey: layerKey,
subLayerKey: ctrldSubLayerGUID,
numFilterConds: 3,
filterCondition: &conditions[0],
}
filter.displayData.name = filterName
filter.weight.valueType = fwpUint8
filter.weight.value = 10
filter.weight.value = uint64(weight)
filter.action.actionType = fwpActionPermit
var filterID uint64
@@ -927,7 +953,6 @@ func (p *prog) addWFPPermitSubnetFilter(engineHandle uintptr, name string, proto
0,
uintptr(unsafe.Pointer(&filterID)),
)
runtime.KeepAlive(&addrMask)
runtime.KeepAlive(conditions)
if r1 != 0 {
return 0, fmt.Errorf("FwpmFilterAdd0 failed: HRESULT 0x%x", r1)
@@ -935,6 +960,24 @@ func (p *prog) addWFPPermitSubnetFilter(engineHandle uintptr, name string, proto
return filterID, nil
}
// addWFPPermitSubnetFilter adds a WFP filter that permits outbound DNS to a given
// IPv4 subnet (addr/mask in host byte order). Used to exempt RFC1918 and CGNAT ranges
// so VPN DNS servers on private IPs are not blocked.
func (p *prog) addWFPPermitSubnetFilter(engineHandle uintptr, name string, proto uint8, addr, mask uint32) (uint64, error) {
addrMask := fwpV4AddrAndMask{addr: addr, mask: mask}
addrCond := fwpmFilterCondition0{
fieldKey: fwpmConditionIPRemoteAddress,
matchType: fwpMatchEqual,
}
addrCond.condValue.valueType = fwpV4AddrMask
addrCond.condValue.value = uint64(uintptr(unsafe.Pointer(&addrMask)))
filterID, err := p.addWFPPermitDNSFilter(engineHandle, name, fwpmLayerALEAuthConnectV4, proto, addrCond, 0, 10)
runtime.KeepAlive(&addrMask)
return filterID, err
}
// wfpSublayerExists checks whether our WFP sublayer still exists in the engine.
// Used by the watchdog to detect if another program removed our filters.
func wfpSublayerExists(engineHandle uintptr) bool {
@@ -962,6 +1005,21 @@ func (p *prog) cleanupWFPFilters(state *wfpState) {
return
}
// Clean up loopback protect filters (DNS mode VPN workaround).
state.mu.Lock()
loopbackIDs := state.loopbackPermitIDs
state.loopbackPermitIDs = nil
state.loopbackProtectActive = false
state.mu.Unlock()
for _, filterID := range loopbackIDs {
r1, _, _ := procFwpmFilterDeleteById0.Call(state.engineHandle, uintptr(filterID))
if r1 != 0 {
mainLog.Load().Warn().Msgf("DNS intercept: failed to remove loopback protect filter (ID: %d, code: 0x%x)", filterID, r1)
} else {
mainLog.Load().Debug().Msgf("DNS intercept: removed loopback protect filter (ID: %d)", filterID)
}
}
for _, filterID := range state.vpnPermitFilterIDs {
r1, _, _ := procFwpmFilterDeleteById0.Call(state.engineHandle, uintptr(filterID))
if r1 != 0 {
@@ -1024,6 +1082,174 @@ func (p *prog) cleanupWFPFilters(state *wfpState) {
}
}
// activateLoopbackWFPProtect opens a minimal WFP session and adds "hard permit"
// filters for DNS to localhost. This is used in DNS mode when NRPT probe failures
// are detected, typically caused by third-party VPN software (e.g., OpenVPN) that
// installs WFP block filters via block-outside-dns. The hard permit (with
// FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT) in a max-weight sublayer overrides the
// third-party blocks without affecting their protection for non-loopback DNS.
//
// See: https://gitlab.int.windscribe.com/controld/clients/ctrld/-/issues/526
func (p *prog) activateLoopbackWFPProtect(state *wfpState) error {
state.mu.Lock()
defer state.mu.Unlock()
if state.loopbackProtectActive {
mainLog.Load().Debug().Msg("DNS intercept: loopback WFP protect already active")
return nil
}
// Only activate in DNS mode. Hard mode manages its own full WFP state
// (block + permit filters in the same sublayer). Activating loopback
// protect would delete the hard mode sublayer and all its filters.
if hardIntercept {
mainLog.Load().Debug().Msg("DNS intercept: skipping loopback WFP protect in hard mode")
return nil
}
mainLog.Load().Info().Msg("DNS intercept: activating loopback WFP protect (countering third-party DNS block filters)")
// Open WFP engine if not already open (DNS mode doesn't open it normally).
if state.engineHandle == 0 {
var engineHandle uintptr
session := fwpmSession0{}
sessionName, _ := windows.UTF16PtrFromString("ctrld DNS Loopback Protect")
session.displayData.name = sessionName
const rpcCAuthnDefault = 0xFFFFFFFF
r1, _, _ := procFwpmEngineOpen0.Call(
0,
uintptr(rpcCAuthnDefault),
0,
uintptr(unsafe.Pointer(&session)),
uintptr(unsafe.Pointer(&engineHandle)),
)
if r1 != 0 {
return fmt.Errorf("FwpmEngineOpen0 failed: HRESULT 0x%x", r1)
}
mainLog.Load().Info().Msgf("DNS intercept: WFP engine opened for loopback protect (handle: 0x%x)", engineHandle)
state.engineHandle = engineHandle
}
// Clean up any stale sublayer from a previous session.
procFwpmSubLayerDeleteByKey0.Call(
state.engineHandle,
uintptr(unsafe.Pointer(&ctrldSubLayerGUID)),
)
// Create sublayer at maximum priority.
sublayer := fwpmSublayer0{
subLayerKey: ctrldSubLayerGUID,
weight: 0xFFFF,
}
sublayerName, _ := windows.UTF16PtrFromString("ctrld DNS Loopback Protect Sublayer")
sublayerDesc, _ := windows.UTF16PtrFromString("Permits DNS to localhost, overriding third-party VPN block filters")
sublayer.displayData.name = sublayerName
sublayer.displayData.description = sublayerDesc
r1, _, _ := procFwpmSubLayerAdd0.Call(
state.engineHandle,
uintptr(unsafe.Pointer(&sublayer)),
0,
)
if r1 != 0 {
return fmt.Errorf("FwpmSubLayerAdd0 failed: HRESULT 0x%x", r1)
}
// Add hard permit filters for loopback DNS (v4+v6, UDP+TCP).
permitFilters := []struct {
name string
layer windows.GUID
proto uint8
}{
{"Loopback Protect: Permit DNS to localhost (IPv4/UDP)", fwpmLayerALEAuthConnectV4, ipprotoUDP},
{"Loopback Protect: Permit DNS to localhost (IPv4/TCP)", fwpmLayerALEAuthConnectV4, ipprotoTCP},
{"Loopback Protect: Permit DNS to localhost (IPv6/UDP)", fwpmLayerALEAuthConnectV6, ipprotoUDP},
{"Loopback Protect: Permit DNS to localhost (IPv6/TCP)", fwpmLayerALEAuthConnectV6, ipprotoTCP},
}
for _, pf := range permitFilters {
filterID, err := p.addWFPHardPermitLocalhostFilter(state.engineHandle, pf.name, pf.layer, pf.proto, state.listenerIP)
if err != nil {
// Partial failure — clean up what we added (already holding mu).
p.deactivateLoopbackWFPProtectLocked(state)
return fmt.Errorf("failed to add loopback protect filter %q: %w", pf.name, err)
}
state.loopbackPermitIDs = append(state.loopbackPermitIDs, filterID)
mainLog.Load().Debug().Str("filter", pf.name).Uint64("id", filterID).Msg("DNS intercept: added loopback protect filter")
}
state.loopbackProtectActive = true
mainLog.Load().Info().Int("filters", len(state.loopbackPermitIDs)).
Msg("DNS intercept: loopback WFP protect activated — localhost DNS permitted with CLEAR_ACTION_RIGHT")
return nil
}
// osHealthcheckSuppressed reports whether the upstream.os healthcheck should
// be skipped because DNS intercept mode is active and the WFP loopback protect
// has been engaged. Loopback protect is only activated when an external WFP
// block filter (e.g. OpenVPN's block-outside-dns) is interfering with DNS,
// which is the same condition that makes the OS resolver healthcheck fail
// every 2s with i/o timeout — so suppressing the check avoids the log spam
// described in issue #526.
func (p *prog) osHealthcheckSuppressed() bool {
if !dnsIntercept || p.dnsInterceptState == nil {
return false
}
state, ok := p.dnsInterceptState.(*wfpState)
if !ok || state == nil {
return false
}
state.mu.Lock()
defer state.mu.Unlock()
return state.loopbackProtectActive
}
// deactivateLoopbackWFPProtectLocked is the lock-free inner implementation.
// Caller must hold state.mu.
func (p *prog) deactivateLoopbackWFPProtectLocked(state *wfpState) {
if !state.loopbackProtectActive && len(state.loopbackPermitIDs) == 0 {
return
}
for _, filterID := range state.loopbackPermitIDs {
if state.engineHandle != 0 {
r1, _, _ := procFwpmFilterDeleteById0.Call(state.engineHandle, uintptr(filterID))
if r1 != 0 {
mainLog.Load().Warn().Msgf("DNS intercept: failed to remove loopback protect filter (ID: %d, code: 0x%x)", filterID, r1)
}
}
}
state.loopbackPermitIDs = nil
state.loopbackProtectActive = false
mainLog.Load().Info().Msg("DNS intercept: loopback WFP protect deactivated")
}
// addWFPHardPermitLocalhostFilter adds a WFP permit filter for DNS to localhost with
// FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT. This "hard permit" prevents lower-priority
// sublayers (e.g., OpenVPN's block-outside-dns sublayer) from blocking DNS to
// ctrld's loopback listener. Weight is set to 15 (above hard mode's permit=10).
// For IPv4, the address is derived from listenerIP (e.g., 127.0.0.1 or 127.0.0.2).
func (p *prog) addWFPHardPermitLocalhostFilter(engineHandle uintptr, name string, layerKey windows.GUID, proto uint8, listenerIP string) (uint64, error) {
addrCond := fwpmFilterCondition0{
fieldKey: fwpmConditionIPRemoteAddress,
matchType: fwpMatchEqual,
}
ipv6Loopback := [16]byte{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}
if layerKey == fwpmLayerALEAuthConnectV4 {
addrCond.condValue.valueType = fwpUint32
addrCond.condValue.value = uint64(parseIPv4AsUint32(listenerIP))
} else {
addrCond.condValue.valueType = fwpByteArray16Type
addrCond.condValue.value = uint64(uintptr(unsafe.Pointer(&ipv6Loopback)))
}
filterID, err := p.addWFPPermitDNSFilter(engineHandle, name, layerKey, proto, addrCond, fwpmFilterFlagClearActionRight, 15)
runtime.KeepAlive(&ipv6Loopback)
return filterID, err
}
// stopDNSIntercept removes all WFP filters and shuts down the DNS interception.
func (p *prog) stopDNSIntercept() error {
if p.dnsInterceptState == nil {
@@ -1050,7 +1276,7 @@ func (p *prog) stopDNSIntercept() error {
state.nrptActive = false
}
// Only clean up WFP if we actually opened the engine (hard mode).
// Clean up WFP if the engine was opened (hard mode or loopback protect).
if state.engineHandle != 0 {
mainLog.Load().Info().Msg("DNS intercept: shutting down WFP filters")
p.cleanupWFPFilters(state)
@@ -1079,10 +1305,13 @@ func (p *prog) exemptVPNDNSServers(exemptions []vpnDNSExemption) error {
if !ok || state == nil {
return fmt.Errorf("DNS intercept state not available")
}
// In dns mode (no WFP), VPN DNS exemptions are not needed — there are no
// block filters to exempt from.
if state.engineHandle == 0 {
mainLog.Load().Debug().Msg("DNS intercept: dns mode — skipping VPN DNS exemptions (no WFP filters)")
// In dns mode (no WFP) or loopback-protect-only mode, VPN DNS exemptions
// are not needed — there are no ctrld block filters to exempt from.
// Loopback protect only adds hard-permit filters for localhost DNS;
// VPN DNS traffic uses the tunnel interface and is already permitted by
// the VPN's own WFP rules.
if state.engineHandle == 0 || state.loopbackProtectActive {
mainLog.Load().Debug().Msg("DNS intercept: dns mode — skipping VPN DNS exemptions (no WFP block filters)")
return nil
}
@@ -1634,6 +1863,40 @@ func (p *prog) nrptProbeAndHeal() {
}
logNRPTParentKeyState("probe-failed-final")
mainLog.Load().Error().Msg("DNS intercept: NRPT verification failed after all retries including two-phase recovery — " +
mainLog.Load().Warn().Msg("DNS intercept: NRPT verification failed after all retries including two-phase recovery")
// Last resort: activate WFP loopback protection.
// Third-party VPN software (e.g., OpenVPN with block-outside-dns) may have
// installed WFP filters that block DNS to non-tunnel interfaces, including
// loopback. A high-priority "hard permit" for localhost DNS overrides these
// blocks and restores NRPT routing to ctrld's listener.
// See: https://gitlab.int.windscribe.com/controld/clients/ctrld/-/issues/526
loopbackState, ok := p.dnsInterceptState.(*wfpState)
if !ok || loopbackState == nil {
mainLog.Load().Error().Msg("DNS intercept: no state available for loopback WFP protect")
return
}
// Bail out if shutdown is in progress — avoid racing with cleanupWFPFilters.
select {
case <-loopbackState.stopCh:
mainLog.Load().Info().Msg("DNS intercept: shutdown in progress, skipping loopback WFP protect activation")
return
default:
}
if err := p.activateLoopbackWFPProtect(loopbackState); err != nil {
mainLog.Load().Error().Err(err).Msg("DNS intercept: failed to activate loopback WFP protect — " +
"DNS queries may not be routed through ctrld. A network interface toggle may be needed.")
return
}
// Retry NRPT probe now that loopback DNS is explicitly permitted through WFP.
time.Sleep(500 * time.Millisecond)
if p.probeNRPT() {
mainLog.Load().Info().Msg("DNS intercept: NRPT verified working after loopback WFP protect activation")
return
}
mainLog.Load().Error().Msg("DNS intercept: NRPT probe still failing after loopback WFP protect — " +
"DNS queries may not be routed through ctrld. A network interface toggle may be needed.")
}
+105 -16
View File
@@ -548,6 +548,7 @@ func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
if vpnServers := p.vpnDNS.UpstreamForDomain(domain); len(vpnServers) > 0 {
ctrld.Log(ctx, mainLog.Load().Debug(), "VPN DNS route matched for domain %s, using servers: %v", domain, vpnServers)
var gotTransportFailure bool
for _, server := range vpnServers {
upstreamConfig := p.vpnDNS.upstreamConfigFor(server)
ctrld.Log(ctx, mainLog.Load().Debug(), "Querying VPN DNS server: %s", server)
@@ -561,6 +562,7 @@ func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
answer, err := dnsResolver.Resolve(resolveCtx, req.msg)
cancel()
if answer != nil {
p.vpnDNS.VPNDNSReachable()
ctrld.Log(ctx, mainLog.Load().Debug(), "VPN DNS query successful")
if p.cache != nil {
ttl := 60 * time.Second
@@ -573,9 +575,22 @@ func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
}
return &proxyResponse{answer: answer}
}
gotTransportFailure = true
ctrld.Log(ctx, mainLog.Load().Debug().Err(err), "VPN DNS server %s failed", server)
}
// Explicit VPN DNS routes are authoritative for their suffix. If all
// routed servers fail at the transport layer while Windows is serving
// retained VPN DNS state, fail closed instead of leaking VPN/internal
// names to normal upstreams.
if gotTransportFailure && p.vpnDNS.ShouldFailClosedAfterVPNDNSTransportFailure(domain, vpnServers) {
ctrld.Log(ctx, mainLog.Load().Debug(),
"All VPN DNS servers had transport failures for %s; returning SERVFAIL while retained VPN DNS state is active", domain)
answer := new(dns.Msg)
answer.SetRcode(req.msg, dns.RcodeServerFailure)
return &proxyResponse{answer: answer}
}
ctrld.Log(ctx, mainLog.Load().Debug(), "All VPN DNS servers failed, falling back to normal upstreams")
}
}
@@ -589,13 +604,15 @@ func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
// polluting captive portal / DHCP flows.
if dnsIntercept && p.vpnDNS != nil && req.ufr.matched &&
len(upstreams) > 0 && upstreams[0] == upstreamOS &&
len(req.msg.Question) > 0 && !p.isAdDomainQuery(req.msg) {
len(req.msg.Question) > 0 {
if dlServers := p.vpnDNS.DomainlessServers(); len(dlServers) > 0 {
domain := req.msg.Question[0].Name
ctrld.Log(ctx, mainLog.Load().Debug(),
"Split-rule query %s going to upstream.os, trying %d domain-less VPN DNS servers first: %v",
domain, len(dlServers), dlServers)
var gotDNSAnswer bool
var gotTransportFailure bool
for _, server := range dlServers {
upstreamCfg := p.vpnDNS.upstreamConfigFor(server)
ctrld.Log(ctx, mainLog.Load().Debug(), "Querying domain-less VPN DNS server: %s", server)
@@ -608,6 +625,10 @@ func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
resolveCtx, cancel := upstreamCfg.Context(ctx)
answer, err := dnsResolver.Resolve(resolveCtx, req.msg)
cancel()
if answer != nil {
gotDNSAnswer = true
p.vpnDNS.VPNDNSReachable()
}
if answer != nil && answer.Rcode == dns.RcodeSuccess {
ctrld.Log(ctx, mainLog.Load().Debug(),
"Domain-less VPN DNS server %s answered %s successfully", server, domain)
@@ -618,10 +639,25 @@ func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
"Domain-less VPN DNS server %s returned %s for %s, trying next",
server, dns.RcodeToString[answer.Rcode], domain)
} else {
gotTransportFailure = true
ctrld.Log(ctx, mainLog.Load().Debug().Err(err),
"Domain-less VPN DNS server %s failed for %s", server, domain)
}
}
// If every domainless VPN DNS attempt failed before receiving a DNS
// packet while Windows is serving retained VPN DNS state, fail closed
// instead of asking LAN/public DNS about internal split-rule names and
// caching false negatives. Reachable negative DNS responses still fall
// through to the old OS fallback behavior below.
if !gotDNSAnswer && gotTransportFailure && p.vpnDNS.ShouldFailClosedAfterVPNDNSTransportFailure(domain, dlServers) {
ctrld.Log(ctx, mainLog.Load().Debug(),
"All domain-less VPN DNS servers had transport failures for %s; returning SERVFAIL while retained VPN DNS state is active", domain)
answer := new(dns.Msg)
answer.SetRcode(req.msg, dns.RcodeServerFailure)
return &proxyResponse{answer: answer}
}
ctrld.Log(ctx, mainLog.Load().Debug(),
"All domain-less VPN DNS servers failed for %s, falling back to OS resolver", domain)
}
@@ -744,6 +780,7 @@ func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
var reason RecoveryReason
if upstreams[0] == upstreamOS {
reason = RecoveryReasonOSFailure
} else {
reason = RecoveryReasonRegularFailure
}
@@ -903,16 +940,13 @@ func needLocalIPv6Listener(interceptMode string) bool {
mainLog.Load().Debug().Msg("IPv6 listener: enabled (Windows)")
return true
}
// On macOS in intercept mode, pf can't redirect IPv6 DNS to an IPv4 listener (cross-AF rdr
// not supported), and blocking IPv6 DNS causes ~1s timeouts (BSD doesn't deliver ICMP errors
// to unconnected UDP sockets). Listening on [::1] lets us intercept IPv6 DNS directly.
//
// NOTE: We accept the intercept mode string as a parameter instead of reading the global
// dnsIntercept bool, because dnsIntercept is derived later in prog.run() — after the
// listener goroutines are already spawned. Same pattern as the port 5354 fallback fix (MR !860).
if (interceptMode == "dns" || interceptMode == "hard") && runtime.GOOS == "darwin" {
mainLog.Load().Debug().Msg("IPv6 listener: enabled (macOS intercept mode)")
return true
// macOS: IPv6 DNS is blocked at the pf level (not intercepted). The [::1] listener
// is not needed — macOS falls back to IPv4 DNS automatically. See #507 and
// docs/pf-dns-intercept.md for why IPv6 interception on macOS is not feasible
// (sendmsg EINVAL from ::1 to global unicast, nat-on-lo0 doesn't fire for route-to).
if runtime.GOOS == "darwin" {
mainLog.Load().Debug().Msg("IPv6 listener: not needed (macOS — IPv6 DNS blocked at pf, fallback to IPv4)")
return false
}
mainLog.Load().Debug().Str("os", runtime.GOOS).Str("interceptMode", interceptMode).Msg("IPv6 listener: not needed")
return false
@@ -1583,7 +1617,7 @@ func (p *prog) monitorNetworkChanges() error {
// we only trigger recovery flow for network changes on non router devices
if router.Name() == "" {
p.handleRecovery(RecoveryReasonNetworkChange)
p.debounceRecovery()
}
// After network changes, verify our pf anchor is still active and
@@ -1660,6 +1694,8 @@ func interfaceIPsEqual(a, b []netip.Prefix) bool {
return true
}
var errOsHealthcheckSuppressed = errors.New("upstream os health check suppressed")
// checkUpstreamOnce sends a test query to the specified upstream.
// Returns nil if the upstream responds successfully.
func (p *prog) checkUpstreamOnce(upstream string, uc *ctrld.UpstreamConfig) error {
@@ -1689,11 +1725,48 @@ func (p *prog) checkUpstreamOnce(upstream string, uc *ctrld.UpstreamConfig) erro
duration := time.Since(start)
if err != nil {
// Demote upstream.os check failures to debug while WFP loopback
// protect is active: an external WFP block filter is interfering
// with plain DNS so repeated failures here are expected. Other
// upstreams keep error level so real outages stay visible.
if upstream == upstreamOS && p.osHealthcheckSuppressed() {
mainLog.Load().Debug().Err(err).Msgf("Upstream %s check failed after %v (WFP loopback protect active)", upstream, duration)
return errOsHealthcheckSuppressed
}
mainLog.Load().Error().Err(err).Msgf("Upstream %s check failed after %v", upstream, duration)
} else {
mainLog.Load().Debug().Msgf("Upstream %s responded successfully in %v", upstream, duration)
return err
}
return err
mainLog.Load().Debug().Msgf("Upstream %s responded successfully in %v", upstream, duration)
return nil
}
// recoveryDebounceWindow is the time to wait after the last network change
// before triggering handleRecovery. This coalesces rapid consecutive network
// changes (e.g., hotspot→LAN causing en1 drop + en0 pickup + en1 re-pickup)
// into a single recovery pass, avoiding the cancel-and-restart race that
// leaves DoH transports in a stale state.
const recoveryDebounceWindow = 500 * time.Millisecond
// debounceRecovery schedules a handleRecovery(NetworkChange) call after a debounce
// window. If called again before the window expires, the timer is reset so that
// recovery runs once with the final network state. All other state updates (IP,
// pf anchor, VPN DNS, tunnel checks) run immediately — only the recovery flow
// with its upstream probing and DHCP bypass logic is debounced.
func (p *prog) debounceRecovery() {
p.recoveryDebounceMu.Lock()
defer p.recoveryDebounceMu.Unlock()
if p.recoveryDebounceTimer != nil {
p.recoveryDebounceTimer.Stop()
mainLog.Load().Debug().Msg("Recovery debounce: resetting timer (rapid network change)")
}
p.recoveryDebounceTimer = time.AfterFunc(recoveryDebounceWindow, func() {
p.recoveryDebounceMu.Lock()
p.recoveryDebounceTimer = nil
p.recoveryDebounceMu.Unlock()
p.handleRecovery(RecoveryReasonNetworkChange)
})
mainLog.Load().Debug().Msg("Recovery debounce: scheduled (500ms window)")
}
// handleRecovery performs a unified recovery by removing DNS settings,
@@ -1723,6 +1796,21 @@ func (p *prog) handleRecovery(reason RecoveryReason) {
p.recoveryCancelMu.Unlock()
}
// For network changes, force-reset all upstream transports synchronously.
// The lazy ReBootstrap() called earlier in the network change callback only
// sets a flag — the old transport's dead connections can still be used by
// recovery probes, causing context deadline timeouts. ForceReBootstrap()
// closes old connections and creates fresh transports so probes succeed on
// first attempt.
if reason == RecoveryReasonNetworkChange {
for _, uc := range p.cfg.Upstream {
if uc != nil {
uc.ForceReBootstrap()
}
}
mainLog.Load().Info().Msg("Force-reset upstream transports for network change recovery")
}
// Create a new recovery context without a fixed timeout.
p.recoveryCancelMu.Lock()
recoveryCtx, cancel := context.WithCancel(context.Background())
@@ -1868,7 +1956,8 @@ func (p *prog) waitForUpstreamRecovery(ctx context.Context, upstreams map[string
default:
attempts++
// checkUpstreamOnce will reset any failure counters on success.
if err := p.checkUpstreamOnce(name, uc); err == nil {
err := p.checkUpstreamOnce(name, uc)
if err == nil || errors.Is(err, errOsHealthcheckSuppressed) {
mainLog.Load().Debug().Msgf("Upstream %s recovered successfully", name)
select {
case recoveredCh <- name:
+339
View File
@@ -0,0 +1,339 @@
package cli
import (
"io"
"os"
"strings"
"sync"
"testing"
"time"
)
// =============================================================================
// logWriter.tailLastLines tests
// =============================================================================
func Test_logWriter_tailLastLines_Empty(t *testing.T) {
lw := newLogWriterWithSize(4096)
if got := lw.tailLastLines(10); got != nil {
t.Fatalf("expected nil for empty buffer, got %q", got)
}
}
func Test_logWriter_tailLastLines_ZeroLines(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\n"))
if got := lw.tailLastLines(0); got != nil {
t.Fatalf("expected nil for n=0, got %q", got)
}
}
func Test_logWriter_tailLastLines_NegativeLines(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\n"))
if got := lw.tailLastLines(-1); got != nil {
t.Fatalf("expected nil for n=-1, got %q", got)
}
}
func Test_logWriter_tailLastLines_FewerThanN(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\n"))
got := string(lw.tailLastLines(10))
want := "line1\nline2\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_logWriter_tailLastLines_ExactN(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\nline3\n"))
got := string(lw.tailLastLines(3))
want := "line1\nline2\nline3\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_logWriter_tailLastLines_MoreThanN(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\nline3\nline4\nline5\n"))
got := string(lw.tailLastLines(2))
want := "line4\nline5\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_logWriter_tailLastLines_NoTrailingNewline(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\nline3"))
// Without trailing newline, "line3" is a partial line.
// Asking for 1 line returns the last newline-terminated line plus the partial.
got := string(lw.tailLastLines(1))
want := "line2\nline3"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_logWriter_tailLastLines_SingleLineNoNewline(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("only line"))
got := string(lw.tailLastLines(5))
want := "only line"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_logWriter_tailLastLines_SingleLineWithNewline(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("only line\n"))
got := string(lw.tailLastLines(1))
want := "only line\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
// =============================================================================
// logWriter.Subscribe tests
// =============================================================================
func Test_logWriter_Subscribe_Basic(t *testing.T) {
lw := newLogWriterWithSize(4096)
ch, unsub := lw.Subscribe()
defer unsub()
msg := []byte("hello world\n")
lw.Write(msg)
select {
case got := <-ch:
if string(got) != string(msg) {
t.Fatalf("got %q, want %q", got, msg)
}
case <-time.After(time.Second):
t.Fatal("timed out waiting for subscriber data")
}
}
func Test_logWriter_Subscribe_MultipleSubscribers(t *testing.T) {
lw := newLogWriterWithSize(4096)
ch1, unsub1 := lw.Subscribe()
defer unsub1()
ch2, unsub2 := lw.Subscribe()
defer unsub2()
msg := []byte("broadcast\n")
lw.Write(msg)
for i, ch := range []<-chan []byte{ch1, ch2} {
select {
case got := <-ch:
if string(got) != string(msg) {
t.Fatalf("subscriber %d: got %q, want %q", i, got, msg)
}
case <-time.After(time.Second):
t.Fatalf("subscriber %d: timed out", i)
}
}
}
func Test_logWriter_Subscribe_Unsubscribe(t *testing.T) {
lw := newLogWriterWithSize(4096)
ch, unsub := lw.Subscribe()
// Verify subscribed.
lw.Write([]byte("before unsub\n"))
select {
case <-ch:
case <-time.After(time.Second):
t.Fatal("timed out before unsub")
}
unsub()
// Channel should be closed after unsub.
if _, ok := <-ch; ok {
t.Fatal("channel should be closed after unsubscribe")
}
// Verify subscriber list is empty.
lw.mu.Lock()
count := len(lw.subscribers)
lw.mu.Unlock()
if count != 0 {
t.Fatalf("expected 0 subscribers after unsub, got %d", count)
}
}
func Test_logWriter_Subscribe_UnsubscribeIdempotent(t *testing.T) {
lw := newLogWriterWithSize(4096)
_, unsub := lw.Subscribe()
unsub()
// Second unsub should not panic.
unsub()
}
func Test_logWriter_Subscribe_SlowSubscriberDropped(t *testing.T) {
lw := newLogWriterWithSize(4096)
ch, unsub := lw.Subscribe()
defer unsub()
// Fill the subscriber channel (buffer size is 256).
for i := 0; i < 300; i++ {
lw.Write([]byte("msg\n"))
}
// Should have 256 buffered messages, rest dropped.
count := 0
for {
select {
case <-ch:
count++
default:
goto done
}
}
done:
if count != 256 {
t.Fatalf("expected 256 buffered messages, got %d", count)
}
}
func Test_logWriter_Subscribe_ConcurrentWriteAndRead(t *testing.T) {
lw := newLogWriterWithSize(64 * 1024)
ch, unsub := lw.Subscribe()
defer unsub()
const numWrites = 100
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < numWrites; i++ {
lw.Write([]byte("concurrent write\n"))
}
}()
received := 0
timeout := time.After(5 * time.Second)
for received < numWrites {
select {
case <-ch:
received++
case <-timeout:
t.Fatalf("timed out after receiving %d/%d messages", received, numWrites)
}
}
wg.Wait()
}
// =============================================================================
// tailFileLastLines tests
// =============================================================================
func writeTempFile(t *testing.T, content string) *os.File {
t.Helper()
f, err := os.CreateTemp(t.TempDir(), "tail-test-*")
if err != nil {
t.Fatal(err)
}
if _, err := f.WriteString(content); err != nil {
t.Fatal(err)
}
return f
}
func Test_tailFileLastLines_Empty(t *testing.T) {
f := writeTempFile(t, "")
defer f.Close()
if got := tailFileLastLines(f, 10); got != nil {
t.Fatalf("expected nil for empty file, got %q", got)
}
}
func Test_tailFileLastLines_FewerThanN(t *testing.T) {
f := writeTempFile(t, "line1\nline2\n")
defer f.Close()
got := string(tailFileLastLines(f, 10))
want := "line1\nline2\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_tailFileLastLines_ExactN(t *testing.T) {
f := writeTempFile(t, "a\nb\nc\n")
defer f.Close()
got := string(tailFileLastLines(f, 3))
want := "a\nb\nc\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_tailFileLastLines_MoreThanN(t *testing.T) {
f := writeTempFile(t, "line1\nline2\nline3\nline4\nline5\n")
defer f.Close()
got := string(tailFileLastLines(f, 2))
want := "line4\nline5\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_tailFileLastLines_NoTrailingNewline(t *testing.T) {
f := writeTempFile(t, "line1\nline2\nline3")
defer f.Close()
// Without trailing newline, partial last line comes with the previous line.
got := string(tailFileLastLines(f, 1))
want := "line2\nline3"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_tailFileLastLines_LargerThanChunk(t *testing.T) {
// Build content larger than the 4096 chunk size to exercise multi-chunk reads.
var sb strings.Builder
for i := 0; i < 200; i++ {
sb.WriteString(strings.Repeat("x", 50))
sb.WriteByte('\n')
}
f := writeTempFile(t, sb.String())
defer f.Close()
got := string(tailFileLastLines(f, 3))
lines := strings.Split(strings.TrimRight(got, "\n"), "\n")
if len(lines) != 3 {
t.Fatalf("expected 3 lines, got %d: %q", len(lines), got)
}
expectedLine := strings.Repeat("x", 50)
for _, line := range lines {
if line != expectedLine {
t.Fatalf("unexpected line content: %q", line)
}
}
}
func Test_tailFileLastLines_SeeksToEnd(t *testing.T) {
f := writeTempFile(t, "line1\nline2\nline3\n")
defer f.Close()
tailFileLastLines(f, 1)
// After tailFileLastLines, file position should be at the end.
pos, err := f.Seek(0, io.SeekCurrent)
if err != nil {
t.Fatal(err)
}
stat, err := f.Stat()
if err != nil {
t.Fatal(err)
}
if pos != stat.Size() {
t.Fatalf("expected file position at end (%d), got %d", stat.Size(), pos)
}
}
+267 -4
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"io"
"os"
"path/filepath"
"strings"
"sync"
"time"
@@ -22,6 +23,9 @@ const (
logWriterSentInterval = time.Minute
logWriterInitEndMarker = "\n\n=== INIT_END ===\n\n"
logWriterLogEndMarker = "\n\n=== LOG_END ===\n\n"
logFileName = "ctrld.log"
logFileMaxSize = 1024 * 1024 * 5 // 5 MB
)
type logViewResponse struct {
@@ -38,11 +42,24 @@ type logReader struct {
size int64
}
// logSubscriber represents a subscriber to live log output.
type logSubscriber struct {
ch chan []byte
}
// logWriter is an internal buffer to keep track of runtime log when no logging is enabled.
// When a file path is configured via setLogFile, writes are also persisted to
// a rotated file on disk (max logFileMaxSize, 1 backup) so logs survive restarts.
type logWriter struct {
mu sync.Mutex
buf bytes.Buffer
size int
mu sync.Mutex
buf bytes.Buffer
size int
subscribers []*logSubscriber
// File persistence fields.
logFile *os.File
logFilePath string
logFileSize int64
}
// newLogWriter creates an internal log writer.
@@ -61,10 +78,154 @@ func newLogWriterWithSize(size int) *logWriter {
return lw
}
// setLogFile configures file-backed persistence for the log writer.
// The directory is created if it does not exist. An existing file is
// opened in append mode and its current size is tracked for rotation.
func (lw *logWriter) setLogFile(path string) error {
dir := filepath.Dir(path)
if err := os.MkdirAll(dir, 0750); err != nil {
return fmt.Errorf("creating log directory: %w", err)
}
f, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR|os.O_APPEND, 0600)
if err != nil {
return fmt.Errorf("opening log file: %w", err)
}
st, err := f.Stat()
if err != nil {
f.Close()
return fmt.Errorf("stat log file: %w", err)
}
lw.mu.Lock()
defer lw.mu.Unlock()
lw.logFile = f
lw.logFilePath = path
lw.logFileSize = st.Size()
return nil
}
// rotateLogFile rotates the current log file to a .1 backup.
// It returns true if lw.logFile is usable after the call, false otherwise.
// Must be called with lw.mu held.
func (lw *logWriter) rotateLogFile() bool {
if lw.logFile == nil {
return false
}
lw.logFile.Close()
backupPath := lw.logFilePath + ".1"
// Best effort: rename current to backup (overwrites old backup).
os.Rename(lw.logFilePath, backupPath)
f, err := os.OpenFile(lw.logFilePath, os.O_CREATE|os.O_RDWR|os.O_TRUNC, 0600)
if err != nil {
// If we can't reopen, disable file logging.
lw.logFile = nil
lw.logFileSize = 0
return false
}
lw.logFile = f
lw.logFileSize = 0
return true
}
// closeLogFile closes the backing file if open.
func (lw *logWriter) closeLogFile() {
lw.mu.Lock()
defer lw.mu.Unlock()
if lw.logFile != nil {
lw.logFile.Close()
lw.logFile = nil
}
}
// logFilePaths returns the paths to the current log file and its backup
// (if they exist) for inclusion in log send payloads.
func (lw *logWriter) logFilePaths() (current, backup string) {
lw.mu.Lock()
defer lw.mu.Unlock()
if lw.logFilePath == "" {
return "", ""
}
current = lw.logFilePath
bp := lw.logFilePath + ".1"
if _, err := os.Stat(bp); err == nil {
backup = bp
}
return current, backup
}
// Subscribe returns a channel that receives new log data as it's written,
// and an unsubscribe function to clean up when done.
func (lw *logWriter) Subscribe() (<-chan []byte, func()) {
lw.mu.Lock()
defer lw.mu.Unlock()
sub := &logSubscriber{ch: make(chan []byte, 256)}
lw.subscribers = append(lw.subscribers, sub)
unsub := func() {
lw.mu.Lock()
defer lw.mu.Unlock()
for i, s := range lw.subscribers {
if s == sub {
lw.subscribers = append(lw.subscribers[:i], lw.subscribers[i+1:]...)
close(sub.ch)
break
}
}
}
return sub.ch, unsub
}
// tailLastLines returns the last n lines from the current buffer.
func (lw *logWriter) tailLastLines(n int) []byte {
lw.mu.Lock()
defer lw.mu.Unlock()
data := lw.buf.Bytes()
if n <= 0 || len(data) == 0 {
return nil
}
// Find the last n newlines from the end.
count := 0
pos := len(data)
for pos > 0 {
pos--
if data[pos] == '\n' {
count++
if count == n+1 {
pos++ // move past this newline
break
}
}
}
result := make([]byte, len(data)-pos)
copy(result, data[pos:])
return result
}
func (lw *logWriter) Write(p []byte) (int, error) {
lw.mu.Lock()
defer lw.mu.Unlock()
// Fan-out to subscribers (non-blocking).
if len(lw.subscribers) > 0 {
cp := make([]byte, len(p))
copy(cp, p)
for _, sub := range lw.subscribers {
select {
case sub.ch <- cp:
default:
// Drop if subscriber is slow to avoid blocking the logger.
}
}
}
// Write to backing file if configured.
if lw.logFile != nil {
needsRotation := lw.logFileSize+int64(len(p)) > logFileMaxSize
if !needsRotation || lw.rotateLogFile() {
if n, err := lw.logFile.Write(p); err == nil {
lw.logFileSize += int64(n)
}
}
}
// If writing p causes overflows, discard old data.
if lw.buf.Len()+len(p) > lw.size {
buf := lw.buf.Bytes()
@@ -102,6 +263,12 @@ func (p *prog) initLogging(backup bool) {
p.initInternalLogging(logWriters)
}
// internalLogFilePath returns the path for persisted internal logs.
// The file lives in the ctrld home directory alongside other runtime state.
func internalLogFilePath() string {
return absHomeDir(logFileName)
}
// initInternalLogging performs internal logging if there's no log enabled.
func (p *prog) initInternalLogging(writers []io.Writer) {
if !p.needInternalLogging() {
@@ -112,6 +279,14 @@ func (p *prog) initInternalLogging(writers []io.Writer) {
p.internalLogWriter = newLogWriter()
p.internalLogSent = time.Now().Add(-logWriterSentInterval)
p.internalWarnLogWriter = newSmallLogWriter()
// Persist internal logs to disk so they survive restarts.
if path := internalLogFilePath(); path != "" {
if err := p.internalLogWriter.setLogFile(path); err != nil {
mainLog.Load().Warn().Err(err).Msg("could not enable persistent internal logging")
} else {
mainLog.Load().Notice().Msgf("internal log file: %s", path)
}
}
})
p.mu.Lock()
lw := p.internalLogWriter
@@ -166,7 +341,15 @@ func (p *prog) logReader() (*logReader, error) {
if wlw == nil {
return nil, errors.New("nil internal warn log writer")
}
// Normal log content.
// If we have a persisted log file, read from disk (includes data
// from previous runs that the in-memory buffer wouldn't have).
current, backup := lw.logFilePaths()
if current != "" {
return p.logReaderFromFiles(current, backup, wlw)
}
// Fall back to in-memory buffer.
lw.mu.Lock()
lwReader := bytes.NewReader(lw.buf.Bytes())
lwSize := lw.buf.Len()
@@ -202,3 +385,83 @@ func (p *prog) logReader() (*logReader, error) {
}
return lr, nil
}
// logReaderFromFiles builds a logReader that concatenates the backup file
// (if it exists), the current log file, and the in-memory warn log buffer.
func (p *prog) logReaderFromFiles(current, backup string, wlw *logWriter) (*logReader, error) {
var rcs []io.ReadCloser
var totalSize int64
closeAll := func() {
for _, rc := range rcs {
rc.Close()
}
}
// Read backup file first (older entries).
if backup != "" {
if bf, err := os.Open(backup); err == nil {
if st, err := bf.Stat(); err == nil {
totalSize += st.Size()
}
rcs = append(rcs, bf)
}
}
// Read current file.
cf, err := os.Open(current)
if err != nil {
closeAll()
return nil, fmt.Errorf("opening current log file: %w", err)
}
if st, err := cf.Stat(); err == nil {
totalSize += st.Size()
}
rcs = append(rcs, cf)
// Append warn log content from memory.
wlw.mu.Lock()
warnData := make([]byte, wlw.buf.Len())
copy(warnData, wlw.buf.Bytes())
wlw.mu.Unlock()
if len(warnData) > 0 {
rcs = append(rcs, io.NopCloser(bytes.NewReader([]byte(logWriterLogEndMarker))))
rcs = append(rcs, io.NopCloser(bytes.NewReader(warnData)))
totalSize += int64(len(logWriterLogEndMarker) + len(warnData))
}
if totalSize == 0 {
closeAll()
return nil, errors.New("internal log is empty")
}
readers := make([]io.Reader, len(rcs))
closers := make([]io.Closer, len(rcs))
for i, rc := range rcs {
readers[i] = rc
closers[i] = rc
}
combined := io.MultiReader(readers...)
lr := &logReader{
r: &multiCloser{Reader: combined, closers: closers},
size: totalSize,
}
return lr, nil
}
// multiCloser wraps an io.Reader and closes multiple underlying closers.
type multiCloser struct {
io.Reader
closers []io.Closer
}
func (mc *multiCloser) Close() error {
var firstErr error
for _, c := range mc.closers {
if err := c.Close(); err != nil && firstErr == nil {
firstErr = err
}
}
return firstErr
}
+125
View File
@@ -1,6 +1,8 @@
package cli
import (
"os"
"path/filepath"
"strings"
"sync"
"testing"
@@ -83,3 +85,126 @@ func Test_logWriter_MarkerInitEnd(t *testing.T) {
t.Fatalf("unexpected log content: %s", lw.buf.String())
}
}
func Test_logWriter_SetLogFile(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "test.log")
lw := newLogWriterWithSize(logWriterSize)
if err := lw.setLogFile(path); err != nil {
t.Fatalf("setLogFile: %v", err)
}
defer lw.closeLogFile()
msg := "hello file\n"
lw.Write([]byte(msg))
// Verify data in memory buffer.
if lw.buf.String() != msg {
t.Fatalf("buffer: got %q, want %q", lw.buf.String(), msg)
}
// Verify data on disk.
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
if string(data) != msg {
t.Fatalf("file: got %q, want %q", data, msg)
}
}
func Test_logWriter_FileRotation(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "test.log")
// Use a tiny max size to trigger rotation quickly.
lw := newLogWriterWithSize(logWriterSize)
if err := lw.setLogFile(path); err != nil {
t.Fatalf("setLogFile: %v", err)
}
defer lw.closeLogFile()
// Write enough to exceed logFileMaxSize.
chunk := strings.Repeat("X", 1024) + "\n"
written := 0
for written < logFileMaxSize+1024 {
lw.Write([]byte(chunk))
written += len(chunk)
}
// Backup file should exist.
backupPath := path + ".1"
if _, err := os.Stat(backupPath); os.IsNotExist(err) {
t.Fatal("expected backup file to exist after rotation")
}
// Current file should be smaller than max (it was rotated).
st, err := os.Stat(path)
if err != nil {
t.Fatalf("stat current: %v", err)
}
if st.Size() > logFileMaxSize {
t.Fatalf("current file too large after rotation: %d", st.Size())
}
}
func Test_logWriter_FilePaths(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "test.log")
lw := newLogWriterWithSize(logWriterSize)
// No file configured.
c, b := lw.logFilePaths()
if c != "" || b != "" {
t.Fatalf("expected empty paths, got %q %q", c, b)
}
if err := lw.setLogFile(path); err != nil {
t.Fatalf("setLogFile: %v", err)
}
defer lw.closeLogFile()
// Current exists, no backup yet.
c, b = lw.logFilePaths()
if c != path {
t.Fatalf("current: got %q, want %q", c, path)
}
if b != "" {
t.Fatalf("backup should be empty, got %q", b)
}
// Create a backup file manually.
os.WriteFile(path+".1", []byte("old"), 0600)
_, b = lw.logFilePaths()
if b != path+".1" {
t.Fatalf("backup: got %q, want %q", b, path+".1")
}
}
func Test_logWriter_FileAppendOnRestart(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "test.log")
// Simulate first run.
lw1 := newLogWriterWithSize(logWriterSize)
if err := lw1.setLogFile(path); err != nil {
t.Fatalf("setLogFile: %v", err)
}
lw1.Write([]byte("run1\n"))
lw1.closeLogFile()
// Simulate second run (restart) — file should be appended.
lw2 := newLogWriterWithSize(logWriterSize)
if err := lw2.setLogFile(path); err != nil {
t.Fatalf("setLogFile: %v", err)
}
lw2.Write([]byte("run2\n"))
lw2.closeLogFile()
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
want := "run1\nrun2\n"
if string(data) != want {
t.Fatalf("file: got %q, want %q", data, want)
}
}
+16
View File
@@ -2,6 +2,7 @@ package cli
import (
"os"
"os/exec"
"strings"
"testing"
@@ -13,5 +14,20 @@ var logOutput strings.Builder
func TestMain(m *testing.M) {
l := zerolog.New(&logOutput)
mainLog.Store(&l)
// Stub the self-upgrade command builder for the whole test binary. The real
// builder execs os.Executable() — which under `go test` IS this test binary
// — with positional args ("upgrade", ...). `go test` stops flag parsing at
// the first positional arg and ignores the rest, so the child just re-runs
// the entire suite, hits the upgrade tests again, and spawns more children:
// a fork bomb of detached processes that stalls the host and (on Windows)
// holds the test binary's image locked, breaking CI artifact cleanup.
// Point it at the test binary with a no-match -test.run so any test that
// reaches performUpgrade still exercises the cmd.Start() success path while
// the child exits immediately without recursing.
newUpgradeCmd = func(exe string) *exec.Cmd {
return exec.Command(exe, "-test.run=^$")
}
os.Exit(m.Run())
}
+20 -2
View File
@@ -146,6 +146,12 @@ type prog struct {
recoveryCancel context.CancelFunc
recoveryRunning atomic.Bool
// recoveryDebounceTimer coalesces rapid NetworkChange recovery triggers
// into a single handleRecovery call. Only handleRecovery is debounced —
// all other state updates (IP, pf anchor, VPN DNS) run immediately.
recoveryDebounceMu sync.Mutex
recoveryDebounceTimer *time.Timer
// recoveryBypass is set when dns-intercept mode enters recovery.
// When true, proxy() forwards all queries to OS/DHCP resolver
// instead of using the normal upstream flow.
@@ -1645,6 +1651,19 @@ func shouldUpgrade(vt string, cv *semver.Version, logger *zerolog.Logger) bool {
return true
}
// newUpgradeCmd builds the detached command used to self-upgrade. It is a
// package-level variable so tests can stub it. With the real implementation a
// *test* binary would re-exec itself — os.Executable() is the test binary, and
// because `go test` stops flag parsing at the first positional arg ("upgrade")
// it ignores the args and re-runs the entire suite. That child hits the same
// upgrade test and spawns another child, recursively: a fork bomb of detached
// processes that pins the host and locks the test binary's image file.
var newUpgradeCmd = func(exe string) *exec.Cmd {
cmd := exec.Command(exe, "upgrade", "prod", "-vv")
cmd.SysProcAttr = sysProcAttrForDetachedChildProcess()
return cmd
}
// performUpgrade executes the self-upgrade command.
// Returns true if upgrade was initiated successfully, false otherwise.
func performUpgrade(vt string) bool {
@@ -1653,8 +1672,7 @@ func performUpgrade(vt string) bool {
mainLog.Load().Error().Err(err).Msg("failed to get executable path, skipped self-upgrade")
return false
}
cmd := exec.Command(exe, "upgrade", "prod", "-vv")
cmd.SysProcAttr = sysProcAttrForDetachedChildProcess()
cmd := newUpgradeCmd(exe)
if err := cmd.Start(); err != nil {
mainLog.Load().Error().Err(err).Msg("failed to start self-upgrade")
return false
+3
View File
@@ -14,6 +14,9 @@ import (
)
func init() {
if isAndroid() {
return
}
if r, err := newLoopbackOSConfigurator(); err == nil {
useSystemdResolved = r.Mode() == "systemd-resolved"
}
+2
View File
@@ -262,6 +262,8 @@ func Test_performUpgrade(t *testing.T) {
},
}
// newUpgradeCmd is stubbed in TestMain so performUpgrade does not re-exec
// (and fork-bomb) the test binary; see the comment there.
for _, tc := range tests {
tc := tc
t.Run(tc.name, func(t *testing.T) {
+1
View File
@@ -160,6 +160,7 @@ func hasLocalDnsServerRunning() bool {
if e != nil {
return false
}
defer windows.CloseHandle(h)
p := windows.ProcessEntry32{Size: processEntrySize}
for {
e := windows.Process32Next(h, &p)
+185 -29
View File
@@ -2,14 +2,19 @@ package cli
import (
"context"
"net"
"runtime"
"strings"
"sync"
"github.com/rs/zerolog"
"tailscale.com/net/netmon"
"github.com/Control-D-Inc/ctrld"
)
var vpnDNSSettlingEnabled = runtime.GOOS == "windows"
// vpnDNSExemption represents a VPN DNS server that needs pf/WFP exemption,
// including the interface it was discovered on. The interface is used on macOS
// to create interface-scoped pf exemptions that allow the VPN's local DNS
@@ -37,6 +42,14 @@ type vpnDNSManager struct {
// as additional nameservers for queries that match split-DNS rules
// (from ctrld config, AD domain, or VPN suffix config).
domainlessServers []string
// retainedAfterEmptyDiscovery means Windows reported an empty VPN DNS
// snapshot once while previous VPN DNS state existed. We keep that last-known
// state for one guarded refresh cycle because Windows can briefly report an
// intermediate empty adapter/DNS state after sleep/wake or reconnect.
retainedAfterEmptyDiscovery bool
// discoverVPNDNS is injected for tests so Refresh does not depend on the
// runner host's real VPN/virtual adapter state.
discoverVPNDNS func(context.Context) []ctrld.VPNDNSConfig
// Called when VPN DNS server list changes, to update intercept exemptions.
onServersChanged vpnDNSExemptFunc
}
@@ -47,6 +60,7 @@ type vpnDNSManager struct {
func newVPNDNSManager(exemptFunc vpnDNSExemptFunc) *vpnDNSManager {
return &vpnDNSManager{
routes: make(map[string][]string),
discoverVPNDNS: ctrld.DiscoverVPNDNS,
onServersChanged: exemptFunc,
}
}
@@ -57,7 +71,11 @@ func (m *vpnDNSManager) Refresh(guardAgainstNoNameservers bool) {
logger := mainLog.Load()
logger.Debug().Msg("Refreshing VPN DNS configurations")
configs := ctrld.DiscoverVPNDNS(context.Background())
discoverVPNDNS := m.discoverVPNDNS
if discoverVPNDNS == nil {
discoverVPNDNS = ctrld.DiscoverVPNDNS
}
configs := discoverVPNDNS(context.Background())
// Detect exit mode: if the default route goes through a VPN DNS interface,
// the VPN is routing ALL traffic (exit node / full tunnel). This is more
@@ -77,6 +95,31 @@ func (m *vpnDNSManager) Refresh(guardAgainstNoNameservers bool) {
m.mu.Lock()
defer m.mu.Unlock()
previousExemptions := m.currentExemptionsLocked()
if vpnDNSSettlingEnabled && len(configs) == 0 && guardAgainstNoNameservers && m.hasVPNDNSStateLocked() {
if !m.retainedAfterEmptyDiscovery {
exemptions := m.currentExemptionsLocked()
m.retainedAfterEmptyDiscovery = true
logger.Debug().Msgf(
"VPN DNS discovery empty; retaining last-known VPN DNS state for one guarded refresh (%d domainless servers, %d exemptions)",
len(m.domainlessServers), len(exemptions))
if m.onServersChanged != nil {
if err := m.onServersChanged(exemptions); err != nil {
logger.Error().Err(err).Msg("Failed to re-apply retained VPN DNS exemptions")
}
}
return
}
logger.Debug().Msgf(
"VPN DNS discovery still empty on next guarded refresh; clearing retained VPN DNS state (%d domainless servers)",
len(m.domainlessServers))
}
// Any discovery path that does not return with retained state clears the
// settling marker: non-empty discovery replaces old servers immediately, and
// an unguarded/second empty discovery clears stale state below.
m.retainedAfterEmptyDiscovery = false
m.configs = configs
m.routes = make(map[string][]string)
@@ -140,14 +183,142 @@ func (m *vpnDNSManager) Refresh(guardAgainstNoNameservers bool) {
logger.Debug().Msgf("VPN DNS refresh completed: %d configs, %d routes, %d domainless servers, %d unique exemptions",
len(m.configs), len(m.routes), len(m.domainlessServers), len(exemptions))
// Update intercept rules to permit VPN DNS traffic.
// Always call onServersChanged — including when exemptions is empty — so that
// stale exemptions from a previous VPN session get cleared on disconnect.
if m.onServersChanged != nil {
if err := m.onServersChanged(exemptions); err != nil {
logger.Error().Err(err).Msg("Failed to update intercept exemptions for VPN DNS servers")
// Update intercept rules to permit VPN DNS traffic only when the exemption set
// actually changes. Network-change events can fire repeatedly while macOS/VPN
// state is otherwise identical; rewriting pf for identical exemptions can feed
// a self-triggering network-change loop. Empty exemptions are still applied
// when they differ from the previous set, so stale VPN exemptions are cleared
// on disconnect.
m.updateInterceptExemptionsIfChanged(logger, previousExemptions, exemptions, "VPN DNS")
}
func (m *vpnDNSManager) updateInterceptExemptionsIfChanged(logger *zerolog.Logger, before, after []vpnDNSExemption, reason string) {
if m.onServersChanged == nil {
return
}
if vpnDNSExemptionsEqual(before, after) {
logger.Debug().Msgf("VPN DNS exemptions unchanged after %s refresh; skipping intercept rule update", reason)
return
}
if err := m.onServersChanged(after); err != nil {
logger.Error().Err(err).Msg("Failed to update intercept exemptions for VPN DNS servers")
}
}
// RefreshRoutesOnly re-discovers VPN DNS configs and updates only ctrld's
// in-memory split-DNS routes. It intentionally does not call onServersChanged,
// so it does not rewrite/reload pf/WFP rules. Use this for post-settle discovery
// checks where we only need to learn late-published VPN search domains.
func (m *vpnDNSManager) RefreshRoutesOnly() (routes, domainlessServers, exemptions int) {
logger := mainLog.Load()
logger.Debug().Msg("Refreshing VPN DNS route state only")
discoverVPNDNS := m.discoverVPNDNS
if discoverVPNDNS == nil {
discoverVPNDNS = ctrld.DiscoverVPNDNS
}
configs := discoverVPNDNS(context.Background())
if dri, err := netmon.DefaultRouteInterface(); err == nil && dri != "" {
for i := range configs {
if configs[i].InterfaceName == dri {
configs[i].IsExitMode = true
}
}
}
m.mu.Lock()
defer m.mu.Unlock()
m.retainedAfterEmptyDiscovery = false
m.configs = configs
m.routes = make(map[string][]string)
for _, config := range configs {
for _, domain := range config.Domains {
domain = strings.TrimPrefix(domain, "~")
domain = strings.TrimPrefix(domain, ".")
domain = strings.ToLower(domain)
if domain != "" {
m.routes[domain] = append([]string{}, config.Servers...)
}
}
}
var domainless []string
seenDomainless := make(map[string]bool)
for _, config := range configs {
if len(config.Domains) == 0 && len(config.Servers) > 0 {
for _, server := range config.Servers {
if !seenDomainless[server] {
seenDomainless[server] = true
domainless = append(domainless, server)
}
}
}
}
m.domainlessServers = domainless
logger.Debug().Msgf("VPN DNS route-only refresh completed: %d configs, %d routes, %d domainless servers, %d exemptions",
len(m.configs), len(m.routes), len(m.domainlessServers), len(m.currentExemptionsLocked()))
return len(m.routes), len(m.domainlessServers), len(m.currentExemptionsLocked())
}
func (m *vpnDNSManager) hasVPNDNSStateLocked() bool {
return len(m.configs) > 0 || len(m.routes) > 0 || len(m.domainlessServers) > 0
}
func (m *vpnDNSManager) currentExemptionsLocked() []vpnDNSExemption {
type key struct{ server, iface string }
seen := make(map[key]bool)
var exemptions []vpnDNSExemption
for _, config := range m.configs {
for _, server := range config.Servers {
k := key{server, config.InterfaceName}
if seen[k] {
continue
}
seen[k] = true
exemptions = append(exemptions, vpnDNSExemption{
Server: server,
Interface: config.InterfaceName,
IsExitMode: config.IsExitMode,
})
}
}
return exemptions
}
// ShouldFailClosedAfterVPNDNSTransportFailure reports whether split-rule
// queries should fail closed instead of falling back to OS/public DNS after
// every candidate VPN DNS server failed before returning a DNS packet. This is
// Windows-only and only active while serving retained VPN DNS state from a
// guarded empty discovery, which is the short window where Windows can report
// VPN DNS before routes to those servers are usable after wake/reconnect.
func (m *vpnDNSManager) ShouldFailClosedAfterVPNDNSTransportFailure(domain string, servers []string) bool {
m.mu.RLock()
defer m.mu.RUnlock()
if !vpnDNSSettlingEnabled || len(servers) == 0 || !m.retainedAfterEmptyDiscovery || !m.hasVPNDNSStateLocked() {
return false
}
mainLog.Load().Debug().Msgf(
"VPN DNS transport failed for %s while retained VPN DNS state is active; suppressing OS fallback for this query (servers=%v)",
domain, servers)
return true
}
// VPNDNSReachable records that a VPN DNS server returned a DNS response. The
// response may be negative (NXDOMAIN/SERVFAIL); the important signal is that
// the VPN DNS transport is reachable again.
func (m *vpnDNSManager) VPNDNSReachable() {
m.mu.Lock()
defer m.mu.Unlock()
if m.retainedAfterEmptyDiscovery {
mainLog.Load().Debug().Msg("VPN DNS transport recovered; clearing retained-empty-discovery state")
}
m.retainedAfterEmptyDiscovery = false
}
// UpstreamForDomain checks if the domain matches any VPN search domain.
@@ -207,24 +378,7 @@ func (m *vpnDNSManager) CurrentServers() []string {
func (m *vpnDNSManager) CurrentExemptions() []vpnDNSExemption {
m.mu.RLock()
defer m.mu.RUnlock()
type key struct{ server, iface string }
seen := make(map[key]bool)
var exemptions []vpnDNSExemption
for _, config := range m.configs {
for _, server := range config.Servers {
k := key{server, config.InterfaceName}
if !seen[k] {
seen[k] = true
exemptions = append(exemptions, vpnDNSExemption{
Server: server,
Interface: config.InterfaceName,
IsExitMode: config.IsExitMode,
})
}
}
}
return exemptions
return m.currentExemptionsLocked()
}
// Routes returns a copy of the current VPN DNS routes for debugging.
@@ -241,10 +395,12 @@ func (m *vpnDNSManager) Routes() map[string][]string {
// upstreamConfigFor creates a legacy upstream configuration for the given VPN DNS server.
func (m *vpnDNSManager) upstreamConfigFor(server string) *ctrld.UpstreamConfig {
endpoint := server
if !strings.Contains(server, ":") {
endpoint = server + ":53"
}
// Use net.JoinHostPort to correctly handle both IPv4 and IPv6 addresses.
// Previously, the strings.Contains(":") check would skip appending ":53"
// for IPv6 addresses (they contain colons), leaving a bare address like
// "2a0d:6fc0:9b0:3600::1" which net.Dial rejects with "too many colons".
// net.JoinHostPort produces "[2a0d:6fc0:9b0:3600::1]:53" as required.
endpoint := net.JoinHostPort(server, "53")
return &ctrld.UpstreamConfig{
Name: "VPN DNS",
+115
View File
@@ -0,0 +1,115 @@
package cli
import (
"context"
"testing"
"github.com/Control-D-Inc/ctrld"
)
func withVPNDNSSettlingEnabled(t *testing.T) {
t.Helper()
old := vpnDNSSettlingEnabled
vpnDNSSettlingEnabled = true
t.Cleanup(func() { vpnDNSSettlingEnabled = old })
}
func TestVPNDNSRefreshRetainsStateForOneGuardedEmptyDiscovery(t *testing.T) {
withVPNDNSSettlingEnabled(t)
var gotExemptions []vpnDNSExemption
m := newVPNDNSManager(func(exemptions []vpnDNSExemption) error {
gotExemptions = exemptions
return nil
})
m.discoverVPNDNS = func(context.Context) []ctrld.VPNDNSConfig { return nil }
m.configs = []ctrld.VPNDNSConfig{{
InterfaceName: "Ethernet 6",
Servers: []string{"10.25.37.21", "10.25.37.22"},
}}
m.domainlessServers = []string{"10.25.37.21", "10.25.37.22"}
m.Refresh(true)
if got := m.DomainlessServers(); len(got) != 2 {
t.Fatalf("expected retained domainless servers, got %v", got)
}
if len(gotExemptions) != 2 {
t.Fatalf("expected retained exemptions to be re-applied, got %v", gotExemptions)
}
if !m.retainedAfterEmptyDiscovery {
t.Fatal("expected empty discovery retention to be marked")
}
}
func TestVPNDNSRefreshClearsOnSecondGuardedEmptyDiscovery(t *testing.T) {
withVPNDNSSettlingEnabled(t)
var gotExemptions []vpnDNSExemption
m := newVPNDNSManager(func(exemptions []vpnDNSExemption) error {
gotExemptions = exemptions
return nil
})
m.discoverVPNDNS = func(context.Context) []ctrld.VPNDNSConfig { return nil }
m.configs = []ctrld.VPNDNSConfig{{
InterfaceName: "Ethernet 6",
Servers: []string{"10.25.37.21"},
}}
m.domainlessServers = []string{"10.25.37.21"}
m.retainedAfterEmptyDiscovery = true
m.Refresh(true)
if got := m.DomainlessServers(); len(got) != 0 {
t.Fatalf("expected domainless servers to be cleared on second empty discovery, got %v", got)
}
if len(gotExemptions) != 0 {
t.Fatalf("expected empty exemptions after clearing stale state, got %v", gotExemptions)
}
if m.retainedAfterEmptyDiscovery {
t.Fatal("expected retained empty-discovery marker to be cleared with stale state")
}
}
func TestVPNDNSRefreshSkipsUnchangedInterceptExemptions(t *testing.T) {
var updates [][]vpnDNSExemption
m := newVPNDNSManager(func(exemptions []vpnDNSExemption) error {
updates = append(updates, append([]vpnDNSExemption{}, exemptions...))
return nil
})
m.discoverVPNDNS = func(context.Context) []ctrld.VPNDNSConfig {
return []ctrld.VPNDNSConfig{{
InterfaceName: "utun-test",
Servers: []string{"10.102.26.10"},
Domains: []string{"example.internal"},
}}
}
m.Refresh(true)
m.Refresh(true)
if len(updates) != 1 {
t.Fatalf("expected exactly one intercept exemption update for unchanged VPN DNS state, got %d", len(updates))
}
if len(updates[0]) != 1 || updates[0][0].Server != "10.102.26.10" || updates[0][0].Interface != "utun-test" {
t.Fatalf("unexpected exemption update: %+v", updates[0])
}
}
func TestVPNDNSTransportFailureSuppressesFallbackOnlyWhileRetainingState(t *testing.T) {
withVPNDNSSettlingEnabled(t)
m := newVPNDNSManager(nil)
m.domainlessServers = []string{"10.25.37.21"}
if m.ShouldFailClosedAfterVPNDNSTransportFailure("splunk.aws.arena.net.", []string{"10.25.37.21"}) {
t.Fatal("did not expect transport failure to suppress OS fallback outside retained empty-discovery state")
}
m.retainedAfterEmptyDiscovery = true
if !m.ShouldFailClosedAfterVPNDNSTransportFailure("splunk.aws.arena.net.", []string{"10.25.37.21"}) {
t.Fatal("expected transport failure to suppress OS fallback while retained state is active")
}
m.VPNDNSReachable()
if m.retainedAfterEmptyDiscovery {
t.Fatal("expected reachable DNS response to clear retained empty-discovery state")
}
}
+1
View File
@@ -640,6 +640,7 @@ func (uc *UpstreamConfig) newDOHTransport(addrs []string) *http.Transport {
transport.TLSClientConfig = &tls.Config{
RootCAs: uc.certPool,
ClientSessionCache: tls.NewLRUClientSessionCache(0),
MinVersion: tls.VersionTLS12,
}
// Prevent bad tcp connection hanging the requests for too long.
+29 -7
View File
@@ -18,7 +18,7 @@ func (uc *UpstreamConfig) newDOH3Transport(addrs []string) http.RoundTripper {
return nil
}
rt := &http3.Transport{}
rt.TLSClientConfig = &tls.Config{RootCAs: uc.certPool}
rt.TLSClientConfig = &tls.Config{RootCAs: uc.certPool, MinVersion: tls.VersionTLS12}
rt.Dial = func(ctx context.Context, addr string, tlsCfg *tls.Config, cfg *quic.Config) (*quic.Conn, error) {
_, port, _ := net.SplitHostPort(addr)
// if we have a bootstrap ip set, use it to avoid DNS lookup
@@ -77,7 +77,17 @@ type parallelDialerResult struct {
err error
}
type quicParallelDialer struct{}
// quicParallelDialer races DialEarly across a list of remote addresses and
// returns the first successful connection. When transport is non-nil, all
// dials share that transport's UDP socket, which removes both the per-dial
// socket allocation and the winner-path socket leak that an owner-of-the-conn
// receiver cannot clean up. When transport is nil, the dialer falls back to a
// fresh UDP socket per attempt (compat path used where no shared transport is
// available yet); the loser paths close their sockets, and the winner path's
// socket is owned by quic.DialEarly's internal transport.
type quicParallelDialer struct {
transport *quic.Transport
}
// Dial performs parallel dialing to the given address list.
func (d *quicParallelDialer) Dial(ctx context.Context, addrs []string, tlsCfg *tls.Config, cfg *quic.Config) (*quic.Conn, error) {
@@ -105,12 +115,24 @@ func (d *quicParallelDialer) Dial(ctx context.Context, addrs []string, tlsCfg *t
ch <- &parallelDialerResult{conn: nil, err: err}
return
}
udpConn, err := net.ListenUDP("udp", nil)
if err != nil {
ch <- &parallelDialerResult{conn: nil, err: err}
return
var (
conn *quic.Conn
udpConn *net.UDPConn
)
if d.transport != nil {
conn, err = d.transport.DialEarly(ctx, remoteAddr, tlsCfg, cfg)
} else {
udpConn, err = net.ListenUDP("udp", nil)
if err != nil {
ch <- &parallelDialerResult{conn: nil, err: err}
return
}
conn, err = quic.DialEarly(ctx, udpConn, remoteAddr, tlsCfg, cfg)
if err != nil {
udpConn.Close()
udpConn = nil
}
}
conn, err := quic.DialEarly(ctx, udpConn, remoteAddr, tlsCfg, cfg)
select {
case ch <- &parallelDialerResult{conn: conn, err: err}:
case <-done:
+4 -3
View File
@@ -1,4 +1,4 @@
# Using Debian bullseye for building regular image.
# Using Debian bookworm for building regular image.
# Using scratch image for minimal image size.
# The final image has:
#
@@ -8,11 +8,12 @@
# - Non-cgo ctrld binary.
#
# CI_COMMIT_TAG is used to set the version of ctrld binary.
FROM golang:1.20-bullseye as base
FROM golang:1.25-bookworm AS base
WORKDIR /app
RUN apt-get update && apt-get install -y upx-ucl
RUN echo "deb http://deb.debian.org/debian bookworm-backports main" | tee /etc/apt/sources.list.d/backports.list
RUN apt update && apt install -t bookworm-backports upx-ucl
COPY . .
+4 -3
View File
@@ -1,4 +1,4 @@
# Using Debian bullseye for building regular image.
# Using Debian bookworm for building regular image.
# Using scratch image for minimal image size.
# The final image has:
#
@@ -8,11 +8,12 @@
# - Non-cgo ctrld binary.
#
# CI_COMMIT_TAG is used to set the version of ctrld binary.
FROM golang:bullseye as base
FROM golang:1.25-bookworm AS base
WORKDIR /app
RUN apt-get update && apt-get install -y upx-ucl
RUN echo "deb http://deb.debian.org/debian bookworm-backports main" | tee /etc/apt/sources.list.d/backports.list
RUN apt update && apt install -t bookworm-backports upx-ucl
COPY . .
+30 -3
View File
@@ -56,7 +56,7 @@ ctrld run --intercept-mode hard --cd <resolver-uid>
Windows DNS intercept uses a two-tier architecture with mode-dependent enforcement:
- **`dns` mode**: NRPT only — graceful DNS routing through the Windows DNS Client service. At worst, a VPN overwrites NRPT and queries bypass ctrld temporarily. DNS never breaks.
- **`dns` mode**: NRPT + loopback WFP protect — graceful DNS routing through the Windows DNS Client service, with proactive WFP permit filters that protect the NRPT → localhost path from third-party DNS block filters (e.g., OpenVPN's `block-outside-dns`).
- **`hard` mode**: NRPT + WFP — same NRPT routing, plus WFP kernel-level block filters that prevent any outbound DNS bypass. Equivalent enforcement to macOS pf.
#### Why This Design?
@@ -70,8 +70,9 @@ Separating them into modes means most users get `dns` mode (safe, can never brea
1. Creates NRPT catch-all registry rule (`.``127.0.0.1`) under `HKLM\...\DnsPolicyConfig\CtrldCatchAll`
2. Triggers Group Policy refresh via `RefreshPolicyEx` (userenv.dll) so DNS Client loads NRPT immediately
3. Flushes DNS cache to clear stale entries
4. Starts NRPT health monitor (30s periodic check)
5. Launches async NRPT probe-and-heal to verify NRPT is actually routing queries
4. **Activates loopback WFP protect** — adds 4 permit filters (IPv4/IPv6 × UDP/TCP) for DNS to localhost with `FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT`. These prevent third-party WFP block filters from blocking the NRPT → `127.0.0.1` path (see [Loopback WFP Protect](#loopback-wfp-protect) below). Non-fatal if this fails.
5. Starts NRPT health monitor (30s periodic check)
6. Launches async NRPT probe-and-heal to verify NRPT is actually routing queries
#### Startup Sequence (hard mode)
@@ -112,6 +113,32 @@ The **Name Resolution Policy Table** is a Windows feature (originally for Direct
**VPN coexistence**: VPN software can set DNS to whatever it wants on the interface — for public IPs, the WFP block filter prevents those servers from being reached on port 53. For private IPs, the subnet permits allow it. ctrld handles all DNS routing through NRPT and can forward VPN-specific domains to VPN DNS servers through its own upstream mechanism.
#### Loopback WFP Protect (dns mode)
Third-party VPN software (e.g., OpenVPN, Securepoint SSL VPN) can install WFP block filters via `block-outside-dns` that block **all** DNS traffic to non-tunnel interfaces — including loopback. This breaks the NRPT → `127.0.0.1:53` path that ctrld depends on, causing DNS resolution to time out.
ctrld proactively adds 4 WFP "hard permit" filters at startup:
| Filter | Layer | Protocol |
|---|---|---|
| Permit DNS to localhost (IPv4/UDP) | ALE_AUTH_CONNECT_V4 | UDP |
| Permit DNS to localhost (IPv4/TCP) | ALE_AUTH_CONNECT_V4 | TCP |
| Permit DNS to localhost (IPv6/UDP) | ALE_AUTH_CONNECT_V6 | UDP |
| Permit DNS to localhost (IPv6/TCP) | ALE_AUTH_CONNECT_V6 | TCP |
**Key properties:**
- **Scope**: Port 53 to `127.0.0.1` (or configured listener IP) and `::1` only
- **Flag**: `FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT` (0x08) — "hard permit" that overrides BLOCK decisions from other sublayers regardless of weight or insertion order
- **Weight**: 15 (above hard mode's permit=10)
- **Sublayer**: ctrld's sublayer at maximum priority (0xFFFF)
- **Lifetime**: Process lifetime — added at startup, removed on shutdown/uninstall
Because `CLEAR_ACTION_RIGHT` is a cross-sublayer override, the order of filter installation doesn't matter — even if a VPN connects hours later and adds its own WFP block filters, ctrld's hard permit for loopback DNS is never overridden.
The reactive fallback in `nrptProbeAndHeal()` is preserved as defense-in-depth for edge cases where proactive activation fails at startup.
See: [Issue #526](https://gitlab.int.windscribe.com/controld/clients/ctrld/-/issues/526)
#### NRPT Probe and Auto-Heal
`RefreshPolicyEx` returns immediately — it does NOT wait for the DNS Client service to actually load the NRPT rule. On cold machines (first boot, fresh install), the DNS Client may take several seconds to process the policy refresh. During this window, the NRPT rule exists in the registry but isn't active.
+51
View File
@@ -22,6 +22,57 @@ This document outlines known issues with ctrld and their current status, workaro
---
## Merlin Issues
### Daemon Crashing on `Ctrl+C`
**Issue**: `ctrld` daemon terminates unexpectedly after stopping a log tailing command. This typically occurs when running the daemon and the log viewer within the same SSH session on ASUSWRT-Merlin routers.
**Description**
The issue is caused by `Signal Propagation` within a shared `Process Group (PGID)`.
Steps to reproduce:
1. You start the daemon manually: `ctrld start --cd=<uid>`.
2. You view internal logs in the same terminal: `ctrld log tail`.
3. You press `Ctrl+C` to stop viewing logs.
4. The `ctrld` daemon service stops immediately along with the log command.
When you execute commands sequentially in a single interactive SSH session on Merlin, the shell often assigns them to the same Process Group. In Linux, the `SIGINT` signal (triggered by `Ctrl+C`) is not just sent to the foreground application, but is frequently propagated to every process belonging to that specific process group.
Because the `ctrld` daemon remains "attached" to the terminal session's process group, it "hears" the interrupt signal intended for the `log tail` command and shuts down.
**Workarounds**:
To isolate the signals, avoid running the log viewer in the same window as the daemon:
* **Window A:** Start the daemon and leave it running.
* **Window B:** Open a new SSH connection to run `ctrld log tail`.
Because Window B has a different **Session ID** and **Process Group ID**, pressing `Ctrl+C` in Window B will not affect the process in Window A.
## Windows Issues
### VPN `block-outside-dns` Breaks DNS When Using ctrld in DNS Mode
**Issue**: VPN software that uses OpenVPN's `block-outside-dns` directive installs WFP (Windows Filtering Platform) block filters that prevent DNS queries from reaching ctrld's loopback listener.
**Status**: Fixed in v1.5.1
**Description**: When a VPN connects with `block-outside-dns` enabled, OpenVPN adds WFP filters that block all DNS traffic to non-tunnel interfaces — including loopback (`127.0.0.1`). Since ctrld's NRPT catch-all rule routes DNS through the Windows DNS Client to `127.0.0.1:53`, the WFP block filters prevent DNS Client from reaching ctrld, causing all DNS queries to time out.
This affects any VPN client that implements `block-outside-dns` via WFP, including:
- OpenVPN GUI (community)
- Securepoint SSL VPN
- Any OpenVPN-based client that honors the `block-outside-dns` push directive
**Fix**: ctrld now proactively adds WFP "hard permit" filters for DNS to localhost at startup. These use `FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT` to override block decisions from any other WFP sublayer, ensuring the NRPT → loopback path is always available regardless of VPN state. See `docs/dns-intercept-mode.md` for technical details.
**Affected Versions**: ctrld ≤ v1.5.0 in `dns` intercept mode on Windows
**Last Updated**: 04/28/2026
---
## Contributing to Known Issues
If you encounter an issue not listed here, please:
+23 -58
View File
@@ -17,7 +17,7 @@ options (set) → normalization (scrub) → queueing → translation (nat/rdr)
| Anchor Type | Section | Purpose |
|-------------|---------|---------|
| `scrub-anchor` | Normalization | Packet normalization |
| `nat-anchor` | Translation | NAT rules |
| `nat-anchor` | Translation | NAT rules (not used by ctrld) |
| `rdr-anchor` | Translation | Redirect rules |
| `anchor` | Filtering | Pass/block rules |
@@ -122,69 +122,31 @@ Three problems prevent a simple "mirror the IPv4 rules" approach:
3. **sendmsg from `[::1]` to global unicast fails**: Unlike IPv4 where the kernel allows `sendmsg` from `127.0.0.1` to local private IPs (e.g., `10.x.x.x`), macOS/BSD rejects `sendmsg` from `[::1]` to a global unicast IPv6 address with `EINVAL`. Since pf's `rdr` preserves the original source IP (the machine's global IPv6 address), ctrld's reply would fail.
### Solution: nat + rdr + [::1] Listener
### Solution: Block IPv6 DNS, Fallback to IPv4
After extensive testing (#507), IPv6 DNS interception on macOS is not feasible with current pf capabilities. The solution is to block all outbound IPv6 DNS:
```
# NAT: rewrite source to ::1 so ctrld can reply
nat on lo0 inet6 proto udp from ! ::1 to ! ::1 port 53 -> ::1
nat on lo0 inet6 proto tcp from ! ::1 to ! ::1 port 53 -> ::1
# RDR: redirect destination to ctrld's IPv6 listener
rdr on lo0 inet6 proto udp from any to ! ::1 port 53 -> ::1 port 53
rdr on lo0 inet6 proto tcp from any to ! ::1 port 53 -> ::1 port 53
# Filter: route-to forces IPv6 DNS to loopback (mirrors IPv4 rules)
pass out quick on ! lo0 route-to lo0 inet6 proto udp from any to ! ::1 port 53
pass out quick on ! lo0 route-to lo0 inet6 proto tcp from any to ! ::1 port 53
# Pass on lo0 without state (mirrors IPv4)
pass out quick on lo0 inet6 proto udp from any to ! ::1 port 53 no state
pass out quick on lo0 inet6 proto tcp from any to ! ::1 port 53 no state
# Accept redirected IPv6 DNS with reply-to (mirrors IPv4)
pass in quick on lo0 reply-to lo0 inet6 proto { udp, tcp } from any to ::1 port 53
block out quick on ! lo0 inet6 proto { udp, tcp } from any to any port 53
```
### IPv6 Packet Flow
macOS automatically retries DNS over IPv4 when the IPv6 path is blocked. The IPv4 path is fully intercepted via the normal route-to + rdr mechanism. Impact is minimal — at most ~1s latency on the very first DNS query while the IPv6 attempt is blocked.
```
Application queries [2607:f0c8:8000:8210::1]:53 (IPv6 DNS server)
pf filter: "pass out route-to lo0 inet6 ... port 53" → redirects to lo0
pf (outbound lo0): "pass out on lo0 inet6 ... no state" → passes
Loopback reflects packet inbound on lo0
pf nat: rewrites source 2607:f0c8:...:ec6e → ::1
pf rdr: rewrites dest [2607:f0c8:8000:8210::1]:53 → [::1]:53
ctrld receives query from [::1]:port → [::1]:53
ctrld resolves via DoH, replies to [::1]:port (kernel accepts ::1 → ::1)
pf reverses both translations:
- nat reverse: dest ::1 → 2607:f0c8:...:ec6e (original client)
- rdr reverse: src ::1 → 2607:f0c8:8000:8210::1 (original DNS server)
Application receives response from [2607:f0c8:8000:8210::1]:53 ✓
```
### What Was Tried and Why It Failed
### Client IP Recovery
The `nat` rewrites the source to `::1`, so ctrld sees the client as `::1` (loopback). The existing `spoofLoopbackIpInClientInfo()` logic detects this and replaces it with the machine's real RFC1918 IPv4 address (e.g., `10.0.10.211`). This is the same mechanism used when queries arrive from `127.0.0.1` — no client identity is lost.
| Approach | Result |
|----------|--------|
| `nat on lo0 inet6` to rewrite source to `::1` | pf skips translation on second interface pass — nat doesn't fire for route-to'd packets arriving on lo0 |
| ULA address on lo0 (`fd00:53::1`) | Kernel rejects: `EHOSTUNREACH` — lo0's routing table is segregated from global unicast |
| Raw IPv6 socket (`SOCK_RAW` + `IPPROTO_UDP`) | Bypasses sendmsg validation, but pf doesn't match raw socket packets against rdr state — response arrives from `::1` not the original server |
| `DIOCNATLOOK` to get original dest + raw socket from that addr | Can't `bind()` to a non-local address (`EADDRNOTAVAIL`) — macOS has no `IPV6_HDRINCL` for source spoofing |
| BPF packet injection on lo0 | Theoretically possible but extremely complex — not justified for the marginal benefit |
### IPv6 Listener
The `[::1]` listener reuses the existing infrastructure from Windows (where it was added for the same reason — can't suppress IPv6 DNS resolvers from the system config). The `needLocalIPv6Listener()` function gates it, returning `true` on:
- **Windows**: Always (if IPv6 is available)
- **macOS**: Only in intercept mode
If the `[::1]` listener fails to bind, it logs a warning and continues — the IPv4 listener is primary.
### nat-anchor Requirement
The `nat` rules in our anchor require a `nat-anchor "com.controld.ctrld"` reference in the main pf ruleset, in addition to the existing `rdr-anchor` and `anchor` references. All pf management functions (inject, remove, verify, watchdog, force-reload) handle all three anchor types.
The `[::1]` listener is used on:
- **Windows**: Always (if IPv6 is available) — Windows can't easily suppress IPv6 DNS resolvers
- **macOS**: **Not used** — IPv6 DNS is blocked at pf, no listener needed
## Rule Ordering Within the Anchor
@@ -236,7 +198,7 @@ The trickiest part. macOS only processes anchors declared in the active pf rules
1. Read `/etc/pf.conf`
2. If our anchor reference already exists, reload as-is
3. Otherwise, inject `nat-anchor "com.controld.ctrld"` and `rdr-anchor "com.controld.ctrld"` in the translation section and `anchor "com.controld.ctrld"` in the filter section
3. Otherwise, inject `rdr-anchor "com.controld.ctrld"` in the translation section and `anchor "com.controld.ctrld"` in the filter section
4. Write to a **temp file** and load with `pfctl -f <tmpfile>`
5. **We never modify `/etc/pf.conf` on disk** — changes are runtime-only and don't survive reboot (ctrld re-injects on every start)
@@ -376,5 +338,8 @@ We chose `route-to + rdr` as the best balance of effectiveness and deployability
9. **`pass out quick` exemptions work with route-to** — they fire in the same phase (filter), so `quick` + rule ordering means exempted packets never hit the route-to rule
10. **pf cannot cross-AF redirect**`rdr on lo0 inet6 ... -> 127.0.0.1` is invalid. IPv6 DNS must be handled by an `[::1]` listener.
11. **`block return` doesn't work for IPv6 DNS** — BSD doesn't deliver ICMPv6 unreachable to unconnected UDP sockets (`sendto`). Apps timeout waiting for a response that never comes.
12. **sendmsg from `::1` to global unicast fails on macOS** — unlike IPv4 where `127.0.0.1` can send to any local address, `::1` cannot send to the machine's own global IPv6 address. `nat` on lo0 is required to rewrite the source.
13. **`nat-anchor` is separate from `rdr-anchor`** — pf requires both in the main ruleset for nat and rdr rules in an anchor to be evaluated. `rdr-anchor` alone does not cover nat rules.
12. **sendmsg from `::1` to global unicast fails on macOS** — unlike IPv4 where `127.0.0.1` can send to any local address, `::1` cannot send to the machine's own global IPv6 address (`EINVAL`). This is the fundamental asymmetry that makes IPv6 DNS interception infeasible.
13. **`nat on lo0` doesn't fire for `route-to`'d packets** — pf runs translation on the original outbound interface (en0), then skips it on lo0's outbound pass. `rdr` works because lo0 inbound is a genuinely new direction. Any lo0 address (including ULAs) can't route to global unicast — the kernel segregates lo0's routing table.
14. **Raw IPv6 sockets bypass routing validation but pf doesn't match them**`SOCK_RAW` can send from `::1` to global unicast, but pf treats raw socket packets as new connections (not matching rdr state), so reverse-translation doesn't happen. The client sees `::1` as the source, not the original DNS server.
15. **`DIOCNATLOOK` can find the original dest but you can't use it** — The ioctl returns the pre-rdr destination, but `bind()` fails with `EADDRNOTAVAIL` because it's not a local address. macOS IPv6 raw sockets don't support `IPV6_HDRINCL` for source spoofing.
16. **Blocking IPv6 DNS is the pragmatic solution** — macOS automatically retries over IPv4. The ~1s penalty on the first blocked query is negligible compared to the complexity of working around the kernel's IPv6 loopback restrictions.
+13 -7
View File
@@ -15,11 +15,15 @@ the same enforcement guarantees as macOS pf.
```
┌─────────────────────────────────────────────────────────────────┐
dns mode (NRPT only)
│ dns mode (NRPT + loopback WFP protect)
│ │
│ App DNS query → DNS Client service → NRPT lookup │
│ → "." catch-all matches → forward to 127.0.0.1 (ctrld) │
│ │
│ Loopback WFP protect: 4 hard-permit filters (port 53 to │
│ localhost, CLEAR_ACTION_RIGHT) prevent third-party VPN WFP │
│ blocks (e.g., OpenVPN block-outside-dns) from breaking NRPT. │
│ │
│ If VPN clears NRPT: health monitor re-adds within 30s │
│ Worst case: queries go to VPN DNS until NRPT restored │
│ DNS never breaks — graceful degradation │
@@ -182,8 +186,10 @@ When `vpnDNSManager.Refresh()` discovers VPN DNS servers on public IPs:
- Both UDP and TCP for each IP
3. Store new filter IDs for next cleanup cycle
**In `dns` mode, VPN DNS exemptions are skipped** — there are no WFP block
filters to exempt from.
**In `dns` mode, VPN DNS exemptions are skipped** — there are no ctrld WFP block
filters to exempt from. The loopback WFP protect filters only permit localhost
DNS; VPN DNS traffic goes through the tunnel interface and is already permitted
by the VPN's own WFP rules.
### Session Lifecycle
@@ -202,8 +208,8 @@ filters to exempt from.
**Startup (dns mode):**
```
1. Add NRPT catch-all rule + GP refresh + DNS flush
2. Start NRPT health monitor goroutine
3. (No WFP — done)
2. Activate loopback WFP protect (4 hard-permit filters for localhost DNS)
3. Start NRPT health monitor goroutine
```
**Shutdown:**
@@ -338,9 +344,9 @@ breaking DNS.
| Aspect | macOS (pf) | Windows dns mode | Windows hard mode |
|--------|-----------|------------------|-------------------|
| **Routing** | `rdr` redirect | NRPT policy | NRPT policy |
| **Enforcement** | `route-to` + block rules | None (graceful) | WFP block filters |
| **Enforcement** | `route-to` + block rules | Loopback WFP protect | WFP block filters |
| **Can break DNS?** | Yes (pf corruption) | No | Yes (if NRPT lost) |
| **VPN coexistence** | Watchdog + stabilization | NRPT most-specific-match | Same + WFP permits |
| **VPN coexistence** | Watchdog + stabilization | NRPT + loopback hard-permit | Same + WFP permits |
| **Bypass protection** | pf catches all packets | None | WFP catches all connections |
| **Recovery** | Probe + auto-heal | Health monitor re-adds | Full restart on sublayer loss |
+18 -4
View File
@@ -25,6 +25,16 @@ const (
dohOsHeader = "x-cd-os"
dohClientIDPrefHeader = "x-cd-cpref"
headerApplicationDNS = "application/dns-message"
// dohMaxResponseSize caps the response body read from a DoH/DoH3
// upstream. A DNS message is bounded by the protocol's 16-bit length
// field; anything larger cannot be a valid response. The cap stops a
// malicious or compromised upstream from driving ctrld into unbounded
// memory growth via io.ReadAll on attacker-controlled bytes.
dohMaxResponseSize = dns.MaxMsgSize
// dohMaxErrorBodySize bounds how much of a non-200 response body is
// read for inclusion in the returned error.
dohMaxErrorBodySize = 1024
)
// EncodeOsNameMap provides mapping from OS name to a shorter string, used for encoding x-cd-os value.
@@ -130,13 +140,17 @@ func (r *dohResolver) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, erro
}
defer resp.Body.Close()
buf, err := io.ReadAll(resp.Body)
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(io.LimitReader(resp.Body, dohMaxErrorBodySize))
return nil, fmt.Errorf("wrong response from DOH server, got: %s, status: %d", string(body), resp.StatusCode)
}
buf, err := io.ReadAll(io.LimitReader(resp.Body, dohMaxResponseSize+1))
if err != nil {
return nil, fmt.Errorf("could not read message from response: %w", err)
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("wrong response from DOH server, got: %s, status: %d", string(buf), resp.StatusCode)
if len(buf) > dohMaxResponseSize {
return nil, fmt.Errorf("DoH response exceeds %d-byte maximum DNS message size", dohMaxResponseSize)
}
answer := new(dns.Msg)
+221
View File
@@ -12,6 +12,7 @@ import (
"net/url"
"runtime"
"strings"
"sync/atomic"
"testing"
"time"
@@ -196,6 +197,7 @@ func testTLSServer(t *testing.T, handler http.Handler) (*httptest.Server, *x509.
server := httptest.NewUnstartedServer(handler)
server.TLS = &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
MinVersion: tls.VersionTLS12,
}
server.StartTLS()
@@ -232,6 +234,7 @@ func newTestHTTP3Server(t *testing.T, handler http.Handler) *testHTTP3Server {
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"h3"}, // HTTP/3 protocol identifier
MinVersion: tls.VersionTLS12,
}
// Create HTTP/3 server
@@ -264,3 +267,221 @@ func newTestHTTP3Server(t *testing.T, handler http.Handler) *testHTTP3Server {
return h3Server
}
// oversizedDoHHandler streams `bodyBytes` bytes of zeros with the given
// HTTP status. The atomic counter records bytes the handler actually
// wrote, so tests can confirm the client tore down the stream before
// consuming the whole attacker-controlled body.
func oversizedDoHHandler(status int, bodyBytes int64, written *atomic.Int64) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", headerApplicationDNS)
w.WriteHeader(status)
chunk := make([]byte, 64*1024)
var sent int64
for sent < bodyBytes {
n := int64(len(chunk))
if remaining := bodyBytes - sent; remaining < n {
n = remaining
}
m, err := w.Write(chunk[:n])
if err != nil {
return
}
sent += int64(m)
if written != nil {
written.Add(int64(m))
}
if f, ok := w.(http.Flusher); ok {
f.Flush()
}
}
}
}
// dohUpstreamForTLSServer wires an UpstreamConfig at a local httptest TLS
// server, trusting its self-signed certificate. BootstrapIP is set so no
// real DNS lookup runs.
func dohUpstreamForTLSServer(t *testing.T, srv *httptest.Server) *UpstreamConfig {
t.Helper()
pool := x509.NewCertPool()
pool.AddCert(srv.Certificate())
u, err := url.Parse(srv.URL)
if err != nil {
t.Fatalf("parse server URL: %v", err)
}
uc := &UpstreamConfig{
Name: "doh-oversize",
Type: ResolverTypeDOH,
Endpoint: srv.URL + "/dns-query",
BootstrapIP: u.Hostname(),
Timeout: 2000,
}
uc.SetCertPool(pool)
uc.Init()
return uc
}
// doh3UpstreamForAddr wires an UpstreamConfig at a local HTTP/3 server,
// trusting its self-signed certificate.
func doh3UpstreamForAddr(t *testing.T, addr string, cert *x509.Certificate) *UpstreamConfig {
t.Helper()
pool := x509.NewCertPool()
pool.AddCert(cert)
host, _, err := net.SplitHostPort(addr)
if err != nil {
t.Fatalf("split host/port %q: %v", addr, err)
}
uc := &UpstreamConfig{
Name: "doh3-oversize",
Type: ResolverTypeDOH3,
Endpoint: "h3://" + addr + "/dns-query",
BootstrapIP: host,
Timeout: 5000,
}
uc.SetCertPool(pool)
uc.Init()
return uc
}
// TestDoHResolve_OversizedBody_Rejected locks in the fix for
// github.com/Control-D-Inc/ctrld/issues/312: a malicious DoH upstream
// returning a body larger than the DNS protocol allows must be rejected
// with an explicit size error rather than buffered into ctrld memory.
func TestDoHResolve_OversizedBody_Rejected(t *testing.T) {
const oversized = 2 * 1024 * 1024 // far past dohMaxResponseSize (~64 KiB)
var written atomic.Int64
srv := httptest.NewUnstartedServer(oversizedDoHHandler(http.StatusOK, oversized, &written))
testCert := generateTestCertificate(t)
srv.TLS = &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"h2", "http/1.1"},
MinVersion: tls.VersionTLS12,
}
srv.StartTLS()
t.Cleanup(srv.Close)
uc := dohUpstreamForTLSServer(t, srv)
r, err := NewResolver(uc)
if err != nil {
t.Fatalf("NewResolver: %v", err)
}
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
answer, err := r.Resolve(ctx, msg)
if err == nil {
t.Fatalf("Resolve unexpectedly succeeded; answer=%v", answer)
}
if !strings.Contains(err.Error(), "maximum DNS message size") {
t.Fatalf("error %q does not mention the size cap", err)
}
if answer != nil {
t.Fatalf("Resolve returned non-nil answer alongside error: %v", answer)
}
if got := written.Load(); got >= int64(oversized) {
t.Fatalf("server wrote the entire %d-byte body before client tore down (wrote=%d) — cap not effective", oversized, got)
}
}
// TestDoHResolve_NonOKStatus_BoundedErrorBody locks in that a non-200
// response with a huge body does not pull the body fully into ctrld
// memory just to format an error string.
func TestDoHResolve_NonOKStatus_BoundedErrorBody(t *testing.T) {
const huge = 8 * 1024 * 1024
var written atomic.Int64
srv := httptest.NewUnstartedServer(oversizedDoHHandler(http.StatusBadGateway, huge, &written))
testCert := generateTestCertificate(t)
srv.TLS = &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"h2", "http/1.1"},
MinVersion: tls.VersionTLS12,
}
srv.StartTLS()
t.Cleanup(srv.Close)
uc := dohUpstreamForTLSServer(t, srv)
r, err := NewResolver(uc)
if err != nil {
t.Fatalf("NewResolver: %v", err)
}
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
answer, err := r.Resolve(ctx, msg)
if err == nil {
t.Fatalf("Resolve unexpectedly succeeded; answer=%v", answer)
}
if !strings.Contains(err.Error(), "status: 502") {
t.Fatalf("error %q does not surface the upstream status", err)
}
if answer != nil {
t.Fatalf("Resolve returned non-nil answer alongside error: %v", answer)
}
if got := written.Load(); got > 1024*1024 {
t.Fatalf("server wrote %d bytes before client tore down — error path is reading too much body", got)
}
}
// TestDoHResolve_OversizedBody_DoH3 mirrors the DoH oversized-body check
// on the HTTP/3 transport, since github-312 specifically reproduced the
// OOM via DoH3.
func TestDoHResolve_OversizedBody_DoH3(t *testing.T) {
const oversized = 2 * 1024 * 1024
testCert := generateTestCertificate(t)
udpConn, err := net.ListenUDP("udp", &net.UDPAddr{IP: net.ParseIP("127.0.0.1"), Port: 0})
if err != nil {
t.Fatalf("udp listen: %v", err)
}
h3 := &http3.Server{
Handler: oversizedDoHHandler(http.StatusOK, oversized, nil),
TLSConfig: &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"h3"},
MinVersion: tls.VersionTLS12,
},
}
go func() {
if err := h3.Serve(udpConn); err != nil && !errors.Is(err, http.ErrServerClosed) {
t.Logf("h3 server: %v", err)
}
}()
t.Cleanup(func() {
_ = h3.Close()
_ = udpConn.Close()
})
time.Sleep(100 * time.Millisecond)
uc := doh3UpstreamForAddr(t, udpConn.LocalAddr().String(), testCert.cert)
r, err := NewResolver(uc)
if err != nil {
t.Fatalf("NewResolver: %v", err)
}
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
answer, err := r.Resolve(ctx, msg)
if err == nil {
t.Fatalf("Resolve unexpectedly succeeded; answer=%v", answer)
}
if !strings.Contains(err.Error(), "maximum DNS message size") {
t.Fatalf("error %q does not mention the size cap", err)
}
if answer != nil {
t.Fatalf("Resolve returned non-nil answer alongside error: %v", answer)
}
}
+161 -43
View File
@@ -6,15 +6,23 @@ import (
"context"
"crypto/tls"
"errors"
"fmt"
"io"
"net"
"runtime"
"sync"
"time"
"github.com/miekg/dns"
"github.com/quic-go/quic-go"
)
// doqMaxResponseSize caps the bytes read from a DoQ stream: a 2-byte
// length prefix plus a DNS message bounded by dns.MaxMsgSize. Anything
// larger cannot be a valid response and is rejected before buffering more
// data from the upstream.
const doqMaxResponseSize = 2 + dns.MaxMsgSize
type doqResolver struct {
uc *UpstreamConfig
}
@@ -41,12 +49,24 @@ func (r *doqResolver) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, erro
const doqPoolSize = 16
// doqConnPool manages a pool of QUIC connections for DoQ queries using a buffered channel.
// A single quic.Transport (and its UDP socket) is shared by every connection in the pool,
// so the OS socket lifecycle is tied to the pool rather than to each dial. Without this
// ownership model, a strict DoQ upstream that triggers reconnect churn would leak one
// caller-owned UDP socket per dial — see github.com/Control-D-Inc/ctrld/issues/309.
type doqConnPool struct {
uc *UpstreamConfig
addrs []string
port string
tlsConfig *tls.Config
conns chan *doqConn
uc *UpstreamConfig
addrs []string
port string
tlsConfig *tls.Config
quicConfig *quic.Config
conns chan *doqConn
transportMu sync.Mutex
transport *quic.Transport
transportConn *net.UDPConn
transportErr error
transportInit bool
closed bool
}
type doqConn struct {
@@ -63,14 +83,20 @@ func newDOQConnPool(uc *UpstreamConfig, addrs []string) *doqConnPool {
NextProtos: []string{"doq"},
RootCAs: uc.certPool,
ServerName: uc.Domain,
MinVersion: tls.VersionTLS12,
}
quicConfig := &quic.Config{
KeepAlivePeriod: 15 * time.Second,
}
pool := &doqConnPool{
uc: uc,
addrs: addrs,
port: port,
tlsConfig: tlsConfig,
conns: make(chan *doqConn, doqPoolSize),
uc: uc,
addrs: addrs,
port: port,
tlsConfig: tlsConfig,
quicConfig: quicConfig,
conns: make(chan *doqConn, doqPoolSize),
}
// Use SetFinalizer here because we need to call a method on the pool itself.
@@ -85,12 +111,22 @@ func newDOQConnPool(uc *UpstreamConfig, addrs []string) *doqConnPool {
// Resolve performs a DNS query using a pooled QUIC connection.
func (p *doqConnPool) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, error) {
// Retry logic for io.EOF errors (as per original implementation)
// Retry logic for transient errors: io.EOF (connection reset),
// IdleTimeoutError (stale pooled connection timed out), and
// StreamLimitReachedError (stream credit exhausted before server MAX_STREAMS arrived).
for range 5 {
answer, err := p.doResolve(ctx, msg)
if err == io.EOF {
continue
}
var idleErr *quic.IdleTimeoutError
if errors.As(err, &idleErr) {
continue
}
var streamLimitErr quic.StreamLimitReachedError
if errors.As(err, &streamLimitErr) {
continue
}
if err != nil {
return nil, wrapCertificateVerificationError(err)
}
@@ -115,18 +151,25 @@ func (p *doqConnPool) doResolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, er
return nil, err
}
// Open a new stream for this query
stream, err := conn.OpenStream()
// Ensure the context has a deadline before calling OpenStreamSync, which
// blocks until the server sends a MAX_STREAMS update. Without a deadline the
// call could block indefinitely when the server never sends the update.
deadline, ok := ctx.Deadline()
if !ok {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, 5*time.Second)
defer cancel()
deadline, _ = ctx.Deadline()
}
// OpenStreamSync blocks until the server's MAX_STREAMS credit arrives,
// avoiding the StreamLimitReachedError race that OpenStream (non-blocking)
// triggers when the credit replenishment frame is still in flight.
stream, err := conn.OpenStreamSync(ctx)
if err != nil {
p.putConn(conn, false)
return nil, err
}
// Set deadline
deadline, ok := ctx.Deadline()
if !ok {
deadline = time.Now().Add(5 * time.Second)
}
_ = stream.SetDeadline(deadline)
// Write message length (2 bytes) followed by message
@@ -144,26 +187,59 @@ func (p *doqConnPool) doResolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, er
return nil, err
}
// Read response
buf, err := io.ReadAll(stream)
stream.Close()
// Return connection to pool (mark as potentially bad if error occurred)
isGood := err == nil && len(buf) > 0
p.putConn(conn, isGood)
if err != nil {
// RFC 9250 section 4.2 requires the client to indicate end-of-request by
// closing the send side of the stream (STREAM FIN). Servers may defer
// processing until FIN arrives, so the close must happen before reading.
// Stream.Close closes only the send direction; the receive direction
// remains open for the response.
if err := stream.Close(); err != nil {
p.putConn(conn, false)
return nil, err
}
// io.ReadAll hides io.EOF error, so check for empty buffer
// A DoQ response is a 2-byte length prefix followed by a DNS message.
// The DNS message is bounded by the protocol at dns.MaxMsgSize, so a
// well-formed response is at most doqMaxResponseSize bytes. Read one
// byte past that cap to distinguish "at limit" from "over limit" and
// reject oversized responses before they can drive memory growth from
// a malicious or compromised upstream.
buf, err := io.ReadAll(io.LimitReader(stream, doqMaxResponseSize+1))
if err != nil {
p.putConn(conn, false)
return nil, err
}
// io.ReadAll hides io.EOF error, so check for empty buffer.
if len(buf) == 0 {
p.putConn(conn, false)
return nil, io.EOF
}
// Unpack DNS response (skip 2-byte length prefix)
if len(buf) > doqMaxResponseSize {
p.putConn(conn, false)
return nil, fmt.Errorf("DoQ response exceeds %d-byte maximum", doqMaxResponseSize)
}
// RFC 9250: each DoQ DNS message is encoded as a 2-octet length field
// followed by the DNS message. Reject responses that are shorter than
// the prefix or whose prefix declares more bytes than were received,
// and retire the misbehaving connection. Without this guard, buf[2:]
// would panic when len(buf) < 2.
if len(buf) < 2 {
p.putConn(conn, false)
return nil, fmt.Errorf("malformed DoQ response: %d byte(s), need >= 2 for length prefix", len(buf))
}
respLen := int(buf[0])<<8 | int(buf[1])
if 2+respLen > len(buf) {
p.putConn(conn, false)
return nil, fmt.Errorf("malformed DoQ response: length prefix %d exceeds payload %d", respLen, len(buf)-2)
}
p.putConn(conn, true)
// Unpack DNS response (skip 2-byte length prefix).
answer := new(dns.Msg)
if err := answer.Unpack(buf[2:]); err != nil {
if err := answer.Unpack(buf[2 : 2+respLen]); err != nil {
return nil, err
}
answer.SetReply(msg)
@@ -210,25 +286,26 @@ func (p *doqConnPool) putConn(conn *quic.Conn, isGood bool) {
}
// dialConn creates a new QUIC connection using parallel dialing like DoH3.
// All connections from the pool multiplex on a single pool-owned UDP socket,
// so reconnect churn cannot grow the host's FD count.
func (p *doqConnPool) dialConn(ctx context.Context) (string, *quic.Conn, error) {
logger := ProxyLogger.Load()
tr, err := p.getOrInitTransport()
if err != nil {
return "", nil, err
}
// If we have a bootstrap IP, use it directly
if p.uc.BootstrapIP != "" {
addr := net.JoinHostPort(p.uc.BootstrapIP, p.port)
Log(ctx, logger.Debug(), "Sending DoQ request to: %s", addr)
udpConn, err := net.ListenUDP("udp", nil)
if err != nil {
return "", nil, err
}
remoteAddr, err := net.ResolveUDPAddr("udp", addr)
if err != nil {
udpConn.Close()
return "", nil, err
}
conn, err := quic.DialEarly(ctx, udpConn, remoteAddr, p.tlsConfig, nil)
conn, err := tr.DialEarly(ctx, remoteAddr, p.tlsConfig, p.quicConfig)
if err != nil {
udpConn.Close()
return "", nil, err
}
return addr, conn, nil
@@ -240,8 +317,8 @@ func (p *doqConnPool) dialConn(ctx context.Context) (string, *quic.Conn, error)
dialAddrs[i] = net.JoinHostPort(p.addrs[i], p.port)
}
pd := &quicParallelDialer{}
conn, err := pd.Dial(ctx, dialAddrs, p.tlsConfig, nil)
pd := &quicParallelDialer{transport: tr}
conn, err := pd.Dial(ctx, dialAddrs, p.tlsConfig, p.quicConfig)
if err != nil {
return "", nil, err
}
@@ -251,9 +328,35 @@ func (p *doqConnPool) dialConn(ctx context.Context) (string, *quic.Conn, error)
return addr, conn, nil
}
// CloseIdleConnections closes all connections in the pool.
// Connections currently checked out (in use) are not closed.
// getOrInitTransport returns the pool's shared quic.Transport, initialising it
// on first call. Once the pool has been closed it permanently returns an error
// so that callers cannot resurrect a dead pool.
func (p *doqConnPool) getOrInitTransport() (*quic.Transport, error) {
p.transportMu.Lock()
defer p.transportMu.Unlock()
if p.closed {
return nil, errors.New("doq pool closed")
}
if p.transportInit {
return p.transport, p.transportErr
}
p.transportInit = true
udpConn, err := net.ListenUDP("udp", nil)
if err != nil {
p.transportErr = err
return nil, err
}
p.transportConn = udpConn
p.transport = &quic.Transport{Conn: udpConn}
return p.transport, nil
}
// CloseIdleConnections closes all idle connections, the shared quic.Transport,
// and the pool's UDP socket. Connections currently checked out (in use) get
// terminated by the transport close as well — without that, the OS socket
// would remain bound to a goroutine that the caller cannot reach to clean up.
func (p *doqConnPool) CloseIdleConnections() {
drain:
for {
select {
case dc := <-p.conns:
@@ -261,7 +364,22 @@ func (p *doqConnPool) CloseIdleConnections() {
dc.conn.CloseWithError(quic.ApplicationErrorCode(quic.NoError), "")
}
default:
return
break drain
}
}
p.transportMu.Lock()
if p.closed {
p.transportMu.Unlock()
return
}
p.closed = true
tr := p.transport
udpConn := p.transportConn
p.transportMu.Unlock()
if tr != nil {
_ = tr.Close()
}
if udpConn != nil {
_ = udpConn.Close()
}
}
+494 -1
View File
@@ -1,4 +1,3 @@
// test_helpers.go
package ctrld
import (
@@ -8,8 +7,11 @@ import (
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"io"
"math/big"
"net"
"os"
"runtime"
"strings"
"testing"
"time"
@@ -99,6 +101,7 @@ func newTestQUICServer(t *testing.T) *testQUICServer {
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"doq"},
MinVersion: tls.VersionTLS12,
}
// Create QUIC listener
@@ -221,3 +224,493 @@ func (s *testQUICServer) handleStream(t *testing.T, stream *quic.Stream) {
return
}
}
// malformedDoQServer is a test QUIC server that drains the client's DoQ
// request and writes caller-supplied raw bytes back. The bytes are not
// required to be a well-framed DoQ response, which is what lets the
// regression tests exercise malformed-response handling.
type malformedDoQServer struct {
listener *quic.Listener
cert *x509.Certificate
addr string
response []byte
}
func newMalformedDoQServer(t *testing.T, response []byte) *malformedDoQServer {
t.Helper()
testCert := generateTestCertificate(t)
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"doq"},
}
listener, err := quic.ListenAddr("127.0.0.1:0", tlsConfig, nil)
if err != nil {
t.Fatalf("failed to create QUIC listener: %v", err)
}
s := &malformedDoQServer{
listener: listener,
cert: testCert.cert,
addr: listener.Addr().String(),
response: response,
}
go s.serve()
t.Cleanup(func() { _ = listener.Close() })
return s
}
func (s *malformedDoQServer) serve() {
for {
conn, err := s.listener.Accept(context.Background())
if err != nil {
return
}
go s.handleConn(conn)
}
}
func (s *malformedDoQServer) handleConn(conn *quic.Conn) {
for {
stream, err := conn.AcceptStream(context.Background())
if err != nil {
return
}
go s.handleStream(stream)
}
}
func (s *malformedDoQServer) handleStream(stream *quic.Stream) {
defer stream.Close()
// Drain the client's DoQ-framed request so the client's writes complete
// cleanly before we reply with our attacker-controlled bytes. Using
// io.ReadFull because a single Read on a QUIC stream may return short.
lenBuf := make([]byte, 2)
if _, err := io.ReadFull(stream, lenBuf); err != nil {
return
}
msgLen := uint16(lenBuf[0])<<8 | uint16(lenBuf[1])
if msgLen > 0 {
discard := make([]byte, msgLen)
if _, err := io.ReadFull(stream, discard); err != nil {
return
}
}
if len(s.response) > 0 {
_, _ = stream.Write(s.response)
}
}
// newMalformedDoQUpstream builds an UpstreamConfig wired to a local
// malformed test server with the test certificate trusted via a custom
// cert pool. We bypass SetupBootstrapIP by setting BootstrapIP directly,
// so the pool dials 127.0.0.1 without any DNS lookup.
func newMalformedDoQUpstream(t *testing.T, cert *x509.Certificate, addr string) *UpstreamConfig {
t.Helper()
pool := x509.NewCertPool()
pool.AddCert(cert)
host, _, err := net.SplitHostPort(addr)
if err != nil {
t.Fatalf("split host/port %q: %v", addr, err)
}
uc := &UpstreamConfig{
Name: "doq-malformed",
Type: ResolverTypeDOQ,
Endpoint: addr,
Domain: host,
BootstrapIP: host,
Timeout: 2000,
}
uc.SetCertPool(pool)
return uc
}
// TestDoQResolve_MalformedResponse verifies that DoQ upstream
// responses violating RFC 9250 framing — fewer than 2 bytes, or a
// length prefix declaring more payload than was received — return a
// handled error instead of panicking on the length-prefix slice.
func TestDoQResolve_MalformedResponse(t *testing.T) {
tests := []struct {
name string
response []byte
}{
// Empty stream is already handled via io.EOF; locked in so a
// future change that drops that branch is caught.
{"empty response", nil},
// One byte: too short to hold the 2-octet length prefix.
{"single byte response", []byte{0x00}},
// Length prefix declares 16 bytes; payload is absent.
{"length prefix only", []byte{0x00, 0x10}},
// Length prefix declares 65535 bytes; only 1 byte of payload
// arrived.
{"length prefix larger than payload", []byte{0xFF, 0xFF, 0x00}},
}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
server := newMalformedDoQServer(t, tt.response)
uc := newMalformedDoQUpstream(t, server.cert, server.addr)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
pool := newDOQConnPool(uc, []string{"127.0.0.1"})
t.Cleanup(pool.CloseIdleConnections)
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
answer, err := pool.Resolve(ctx, msg)
if err == nil {
t.Fatalf("Resolve unexpectedly succeeded for malformed response %v; answer=%v", tt.response, answer)
}
if answer != nil {
t.Fatalf("Resolve returned non-nil answer alongside error: answer=%v err=%v", answer, err)
}
})
}
}
// strictDoQServer accepts DoQ queries but defers the response until the
// client signals end-of-request with STREAM FIN, as required by RFC 9250
// section 4.2. It exists to lock in the fix for
// github.com/Control-D-Inc/ctrld/issues/309 where a client
// that never closes its send side caused the server to wait forever and the
// client to churn through reconnects.
type strictDoQServer struct {
listener *quic.Listener
cert *x509.Certificate
addr string
}
func newStrictDoQServer(t *testing.T) *strictDoQServer {
t.Helper()
testCert := generateTestCertificate(t)
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"doq"},
MinVersion: tls.VersionTLS12,
}
listener, err := quic.ListenAddr("127.0.0.1:0", tlsConfig, nil)
if err != nil {
t.Fatalf("failed to create QUIC listener: %v", err)
}
s := &strictDoQServer{
listener: listener,
cert: testCert.cert,
addr: listener.Addr().String(),
}
go s.serve()
t.Cleanup(func() { _ = listener.Close() })
return s
}
func (s *strictDoQServer) serve() {
for {
conn, err := s.listener.Accept(context.Background())
if err != nil {
return
}
go s.handleConn(conn)
}
}
func (s *strictDoQServer) handleConn(conn *quic.Conn) {
for {
stream, err := conn.AcceptStream(context.Background())
if err != nil {
return
}
go s.handleStream(stream)
}
}
func (s *strictDoQServer) handleStream(stream *quic.Stream) {
defer stream.Close()
// Drain until the client closes the send side. This is the behaviour
// that triggered the bug: if the client never sends STREAM FIN, this
// read blocks until the stream's deadline fires.
body, err := io.ReadAll(stream)
if err != nil {
return
}
if len(body) < 2 {
return
}
msgLen := uint16(body[0])<<8 | uint16(body[1])
if int(msgLen) != len(body)-2 {
return
}
msg := new(dns.Msg)
if err := msg.Unpack(body[2:]); err != nil {
return
}
response := new(dns.Msg)
response.SetReply(msg)
response.Authoritative = true
if len(msg.Question) > 0 && msg.Question[0].Qtype == dns.TypeA {
response.Answer = append(response.Answer, &dns.A{
Hdr: dns.RR_Header{
Name: msg.Question[0].Name,
Rrtype: dns.TypeA,
Class: dns.ClassINET,
Ttl: 300,
},
A: net.ParseIP("192.0.2.1"),
})
}
respBytes, err := response.Pack()
if err != nil {
return
}
respLen := uint16(len(respBytes))
if _, err := stream.Write([]byte{byte(respLen >> 8), byte(respLen & 0xFF)}); err != nil {
return
}
if _, err := stream.Write(respBytes); err != nil {
return
}
}
func newStrictDoQUpstream(t *testing.T, cert *x509.Certificate, addr string, useBootstrap bool) *UpstreamConfig {
t.Helper()
pool := x509.NewCertPool()
pool.AddCert(cert)
host, _, err := net.SplitHostPort(addr)
if err != nil {
t.Fatalf("split host/port %q: %v", addr, err)
}
uc := &UpstreamConfig{
Name: "doq-strict",
Type: ResolverTypeDOQ,
Endpoint: addr,
Domain: host,
Timeout: 3000,
}
if useBootstrap {
uc.BootstrapIP = host
}
uc.SetCertPool(pool)
return uc
}
// TestDoQResolve_StrictServerWaitsForFIN exercises the RFC 9250 client-FIN
// requirement. With the bug present, the server's io.ReadAll blocks until
// the stream deadline expires and the client sees a timeout, so a successful
// resolve here proves that the client now sends STREAM FIN before reading.
func TestDoQResolve_StrictServerWaitsForFIN(t *testing.T) {
t.Parallel()
server := newStrictDoQServer(t)
uc := newStrictDoQUpstream(t, server.cert, server.addr, true)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
host, _, _ := net.SplitHostPort(server.addr)
pool := newDOQConnPool(uc, []string{host})
t.Cleanup(pool.CloseIdleConnections)
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
answer, err := pool.Resolve(ctx, msg)
if err != nil {
t.Fatalf("Resolve failed against strict DoQ server: %v", err)
}
if answer == nil || len(answer.Answer) == 0 {
t.Fatalf("Resolve returned no answer records: %+v", answer)
}
a, ok := answer.Answer[0].(*dns.A)
if !ok || !a.A.Equal(net.ParseIP("192.0.2.1")) {
t.Fatalf("unexpected answer: %+v", answer.Answer[0])
}
}
// TestDoQResolve_ParallelDialPathStrictFIN exercises the parallel-dial path
// (no BootstrapIP) against the same FIN-strict server, so that both the
// single-dial branch and the parallel-dial branch are covered.
func TestDoQResolve_ParallelDialPathStrictFIN(t *testing.T) {
t.Parallel()
server := newStrictDoQServer(t)
uc := newStrictDoQUpstream(t, server.cert, server.addr, false)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
host, _, _ := net.SplitHostPort(server.addr)
pool := newDOQConnPool(uc, []string{host})
t.Cleanup(pool.CloseIdleConnections)
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
answer, err := pool.Resolve(ctx, msg)
if err != nil {
t.Fatalf("Resolve (parallel-dial path) failed against strict DoQ server: %v", err)
}
if answer == nil || len(answer.Answer) == 0 {
t.Fatalf("Resolve (parallel-dial path) returned no answer records: %+v", answer)
}
}
// TestDoQPool_ChurnDoesNotGrowFDs exercises the reconnect-churn scenario
// described in github.com/Control-D-Inc/ctrld/issues/309: repeated dials
// against a server that closes existing connections must not grow the process
// FD count, because the pool now shares one UDP socket via quic.Transport instead
// of allocating one per dial. Linux-only because /proc/self/fd is the cheapest
// portable proxy for "what's still open."
func TestDoQPool_ChurnDoesNotGrowFDs(t *testing.T) {
if runtime.GOOS != "linux" {
t.Skip("FD accounting via /proc/self/fd is linux-only")
}
t.Parallel()
server := newStrictDoQServer(t)
uc := newStrictDoQUpstream(t, server.cert, server.addr, true)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
host, _, _ := net.SplitHostPort(server.addr)
pool := newDOQConnPool(uc, []string{host})
t.Cleanup(pool.CloseIdleConnections)
makeQuery := func(i int) *dns.Msg {
msg := new(dns.Msg)
// Vary the question so any caching layer cannot short-circuit.
msg.SetQuestion(dns.Fqdn(strings.Repeat("a", 1+i%8)+".example.com"), dns.TypeA)
msg.RecursionDesired = true
return msg
}
// Warm the pool so the steady-state transport and at least one
// connection are open. Without this, the first resolve in the measured
// loop would inflate the baseline.
if _, err := pool.Resolve(ctx, makeQuery(0)); err != nil {
t.Fatalf("warm-up Resolve failed: %v", err)
}
baseline := countOpenFDs(t)
// Force reconnect churn by closing the connection between each query.
// Without the fix this would leak one UDP socket per round; with the
// fix the pool's shared transport keeps a single socket open.
const rounds = 20
for i := 1; i <= rounds; i++ {
// Drain any pooled connection so the next Resolve has to redial.
drainPooledConns(pool)
if _, err := pool.Resolve(ctx, makeQuery(i)); err != nil {
t.Fatalf("Resolve in churn loop iteration %d failed: %v", i, err)
}
}
// Give quic-go a moment to drop any background goroutines that hold
// references to closed sockets.
time.Sleep(200 * time.Millisecond)
after := countOpenFDs(t)
// Allow a small slack for transient FDs (goroutine wake-ups, qlog,
// etc.) but reject anything that scales with the number of rounds.
const slack = 5
if after > baseline+slack {
t.Fatalf("FD count grew under DoQ churn: baseline=%d after=%d rounds=%d (slack=%d)", baseline, after, rounds, slack)
}
}
// drainPooledConns removes any idle pooled connections so the next Resolve
// is forced to dial a fresh one. It does not close the pool's transport.
func drainPooledConns(p *doqConnPool) {
for {
select {
case dc := <-p.conns:
if dc.conn != nil {
dc.conn.CloseWithError(quic.ApplicationErrorCode(quic.NoError), "")
}
default:
return
}
}
}
func countOpenFDs(t *testing.T) int {
t.Helper()
entries, err := os.ReadDir("/proc/self/fd")
if err != nil {
t.Fatalf("read /proc/self/fd: %v", err)
}
return len(entries)
}
// TestDoQResolve_OversizedResponse_Rejected locks in the fix for
// github.com/Control-D-Inc/ctrld/issues/312 on the DoQ transport: a
// malicious upstream that writes a response larger than the DNS protocol
// allows must be rejected with an explicit size error, not buffered
// without bound into ctrld memory.
func TestDoQResolve_OversizedResponse_Rejected(t *testing.T) {
t.Parallel()
// doqMaxResponseSize is 2 + dns.MaxMsgSize. Send something well past
// that. 256 KiB is enough to exceed the cap while keeping the test
// fast on loopback.
response := make([]byte, 256*1024)
// A well-formed length prefix isn't required: the size cap should
// fire before any framing check runs. Use a non-zero prefix so the
// test also documents that the order of validation is "size first,
// framing later."
response[0] = 0xFF
response[1] = 0xFF
server := newMalformedDoQServer(t, response)
uc := newMalformedDoQUpstream(t, server.cert, server.addr)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
pool := newDOQConnPool(uc, []string{"127.0.0.1"})
t.Cleanup(pool.CloseIdleConnections)
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
answer, err := pool.Resolve(ctx, msg)
if err == nil {
t.Fatalf("Resolve unexpectedly succeeded for oversized response; answer=%v", answer)
}
if !strings.Contains(err.Error(), "exceeds") {
t.Fatalf("error %q does not surface the size cap", err)
}
if answer != nil {
t.Fatalf("Resolve returned non-nil answer alongside error: %v", answer)
}
}
+2 -1
View File
@@ -64,7 +64,8 @@ func newDOTClientPool(uc *UpstreamConfig, addrs []string) *dotConnPool {
dialer := newDialer(net.JoinHostPort(controldPublicDns, "53"))
tlsConfig := &tls.Config{
RootCAs: uc.certPool,
RootCAs: uc.certPool,
MinVersion: tls.VersionTLS12,
}
if uc.BootstrapIP != "" {
+10 -10
View File
@@ -1,6 +1,6 @@
module github.com/Control-D-Inc/ctrld
go 1.24
go 1.25.0
require (
github.com/Masterminds/semver/v3 v3.2.1
@@ -29,16 +29,16 @@ require (
github.com/prometheus/client_golang v1.19.1
github.com/prometheus/client_model v0.5.0
github.com/prometheus/prom2json v1.3.3
github.com/quic-go/quic-go v0.57.1
github.com/quic-go/quic-go v0.59.1
github.com/rs/zerolog v1.28.0
github.com/spf13/cobra v1.9.1
github.com/spf13/pflag v1.0.6
github.com/spf13/viper v1.16.0
github.com/stretchr/testify v1.11.1
github.com/vishvananda/netlink v1.3.1
golang.org/x/net v0.43.0
golang.org/x/sync v0.16.0
golang.org/x/sys v0.35.0
golang.org/x/net v0.56.0
golang.org/x/sync v0.21.0
golang.org/x/sys v0.46.0
golang.zx2c4.com/wireguard/windows v0.5.3
tailscale.com v1.74.0
)
@@ -92,11 +92,11 @@ require (
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go4.org/mem v0.0.0-20220726221520-4f986261bf13 // indirect
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
golang.org/x/crypto v0.41.0 // indirect
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842 // indirect
golang.org/x/mod v0.27.0 // indirect
golang.org/x/text v0.28.0 // indirect
golang.org/x/tools v0.36.0 // indirect
golang.org/x/crypto v0.53.0 // indirect
golang.org/x/exp v0.0.0-20240119083558-1b970713d09a // indirect
golang.org/x/mod v0.36.0 // indirect
golang.org/x/text v0.38.0 // indirect
golang.org/x/tools v0.45.0 // indirect
google.golang.org/protobuf v1.33.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
+18 -20
View File
@@ -271,8 +271,8 @@ github.com/prometheus/prom2json v1.3.3 h1:IYfSMiZ7sSOfliBoo89PcufjWO4eAR0gznGcET
github.com/prometheus/prom2json v1.3.3/go.mod h1:Pv4yIPktEkK7btWsrUTWDDDrnpUrAELaOCj+oFwlgmc=
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/quic-go/quic-go v0.57.1 h1:25KAAR9QR8KZrCZRThWMKVAwGoiHIrNbT72ULHTuI10=
github.com/quic-go/quic-go v0.57.1/go.mod h1:ly4QBAjHA2VhdnxhojRsCUOeJwKYg+taDlos92xb1+s=
github.com/quic-go/quic-go v0.59.1 h1:0Gmua0HW1Tv7ANR7hUYwRyD0MG5OJfgvYSZasGZzBic=
github.com/quic-go/quic-go v0.59.1/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis=
github.com/rivo/uniseg v0.4.4/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
@@ -349,8 +349,8 @@ golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm
golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.41.0 h1:WKYxWedPGCTVVl5+WHSSrOBT0O8lx32+zxmHxijgXp4=
golang.org/x/crypto v0.41.0/go.mod h1:pO5AFd7FA68rFak7rOAGVuygIISepHftHnr8dr6+sUc=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -361,8 +361,8 @@ golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u0
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842 h1:vr/HnozRka3pE4EsMEg1lgkXJkTFJCVUX+S/ZT6wYzM=
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842/go.mod h1:XtvwrStGgqGPLc4cjQfWqZHG1YFdYs6swckp8vpsjnc=
golang.org/x/exp v0.0.0-20240119083558-1b970713d09a h1:Q8/wZp0KX97QFTc2ywcOE0YRjZPVIx+MXInMzdvQqcA=
golang.org/x/exp v0.0.0-20240119083558-1b970713d09a/go.mod h1:idGWGoKP1toJGkd5/ig9ZLuPcZBC3ewk7SzmH0uou08=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
@@ -386,8 +386,8 @@ golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.27.0 h1:kb+q2PyFnEADO2IEF935ehFUXlWiNjJWtRNgBLSfbxQ=
golang.org/x/mod v0.27.0/go.mod h1:rWI627Fq0DEoudcK+MBkNkCe0EetEaDSwJJkCcjpazc=
golang.org/x/mod v0.36.0 h1:JJjpVx6myfUsUdAzZuOSTTmRE0PfZeNWzzvKrP7amb4=
golang.org/x/mod v0.36.0/go.mod h1:moc6ELqsWcOw5Ef3xVprK5ul/MvtVvkIXLziUOICjUQ=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -420,8 +420,8 @@ golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v
golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE=
golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg=
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -441,8 +441,8 @@ golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -492,8 +492,8 @@ golang.org/x/sys v0.4.1-0.20230131160137-e7d7f63158de/go.mod h1:oPkhp1MJrh7nUepC
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI=
golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -504,13 +504,11 @@ golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng=
golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU=
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=
golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
@@ -558,8 +556,8 @@ golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4f
golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
golang.org/x/tools v0.36.0 h1:kWS0uv/zsvHEle1LbV5LE8QujrxB3wfQyxHfhOk0Qkg=
golang.org/x/tools v0.36.0/go.mod h1:WBDiHKJK8YgLHlcQPYQzNCkUxUypCaa5ZegCVutKm+s=
golang.org/x/tools v0.45.0 h1:18qN3FAooORvApf5XjCXgsuayZOEtXf6JK18I3+ONa8=
golang.org/x/tools v0.45.0/go.mod h1:LuUGqqaXcXMEFEruIVJVm5mgDD8vww/z/SR1gQ4uE/0=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+2 -1
View File
@@ -11,7 +11,8 @@ func TestCACertPool(t *testing.T) {
c := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
RootCAs: CACertPool(),
RootCAs: CACertPool(),
MinVersion: tls.VersionTLS12,
},
},
Timeout: 2 * time.Second,
+4 -2
View File
@@ -173,8 +173,10 @@ func (t *Table) SetSelfIP(ip string) {
t.selfIPLock.Lock()
defer t.selfIPLock.Unlock()
t.selfIP = ip
t.dhcp.selfIP = t.selfIP
t.dhcp.addSelf()
if t.dhcp != nil {
t.dhcp.selfIP = t.selfIP
t.dhcp.addSelf()
}
}
// initSelfDiscover initializes necessary client metadata for self query.
+55
View File
@@ -1,9 +1,64 @@
package clientinfo
import (
"sync"
"testing"
)
// TestTable_SetSelfIP_NilDHCP ensures SetSelfIP does not panic when t.dhcp is
// nil, which happens when DHCP discovery is disabled and the network-change
// callback fires before or without initialisation.
func TestTable_SetSelfIP_NilDHCP(t *testing.T) {
table := &Table{} // dhcp is nil
// Must not panic.
table.SetSelfIP("192.168.1.1")
if got := table.SelfIP(); got != "192.168.1.1" {
t.Fatalf("SelfIP() = %q, want %q", got, "192.168.1.1")
}
}
// TestTable_SetSelfIP_UpdatesDHCP ensures SetSelfIP propagates the new IP to
// the dhcp discover and calls addSelf when dhcp is initialised.
func TestTable_SetSelfIP_UpdatesDHCP(t *testing.T) {
table := &Table{
dhcp: &dhcp{selfIP: "10.0.0.1"},
}
table.SetSelfIP("10.0.0.2")
if got := table.SelfIP(); got != "10.0.0.2" {
t.Fatalf("SelfIP() = %q, want %q", got, "10.0.0.2")
}
if table.dhcp.selfIP != "10.0.0.2" {
t.Fatalf("dhcp.selfIP = %q, want %q", table.dhcp.selfIP, "10.0.0.2")
}
}
// TestTable_SetSelfIP_Concurrent ensures concurrent calls to SetSelfIP do not
// race, regardless of whether dhcp is nil or not.
func TestTable_SetSelfIP_Concurrent(t *testing.T) {
for _, tc := range []struct {
name string
table *Table
}{
{"nil dhcp", &Table{}},
{"with dhcp", &Table{dhcp: &dhcp{}}},
} {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
var wg sync.WaitGroup
for range 10 {
wg.Add(1)
go func() {
defer wg.Done()
tc.table.SetSelfIP("192.168.1.1")
_ = tc.table.SelfIP()
}()
}
wg.Wait()
})
}
}
func Test_normalizeIP(t *testing.T) {
tests := []struct {
name string
+1 -1
View File
@@ -293,7 +293,7 @@ func apiTransport(cdDev bool) *http.Transport {
return dial(ctx, "tcp6", addrsFromPort(apiIpsV6, port))
}
if router.Name() == ddwrt.Name || runtime.GOOS == "android" {
transport.TLSClientConfig = &tls.Config{RootCAs: certs.CACertPool()}
transport.TLSClientConfig = &tls.Config{RootCAs: certs.CACertPool(), MinVersion: tls.VersionTLS12}
}
return transport
}
+1 -1
View File
@@ -2,11 +2,11 @@ package dnsmasq
import (
"errors"
"html/template"
"net"
"os"
"path/filepath"
"strings"
"text/template"
"github.com/Control-D-Inc/ctrld"
)
+7
View File
@@ -1,7 +1,14 @@
package ctrld
import "runtime"
type dnsFn func() []string
// isMobile reports whether the current OS is a mobile platform.
func isMobile() bool {
return runtime.GOOS == "android" || runtime.GOOS == "ios"
}
// nameservers returns DNS nameservers from system settings.
func nameservers() []string {
var dns []string
+16
View File
@@ -25,6 +25,12 @@ func dnsFns() []dnsFn {
func getDNSFromScutil() []string {
logger := *ProxyLogger.Load()
// Skip scutil on mobile platforms - not available in sandbox
if isMobile() {
Log(context.Background(), logger.Debug(), "skipping scutil DNS discovery on mobile platform")
return nil
}
const (
maxRetries = 10
retryInterval = 100 * time.Millisecond
@@ -89,6 +95,11 @@ func getDNSFromScutil() []string {
}
func getDHCPNameservers(iface string) ([]string, error) {
// Skip ipconfig on mobile platforms - not available in sandbox
if isMobile() {
return nil, fmt.Errorf("ipconfig not available on mobile")
}
// Run the ipconfig command for the given interface.
cmd := exec.Command("ipconfig", "getpacket", iface)
output, err := cmd.Output()
@@ -201,6 +212,11 @@ func getAllDHCPNameservers() []string {
}
func patchNetIfaceName(iface *net.Interface) (bool, error) {
// Skip networksetup on mobile platforms - not available in sandbox
if isMobile() {
return false, nil
}
b, err := exec.Command("networksetup", "-listnetworkserviceorder").Output()
if err != nil {
return false, err
+16
View File
@@ -7,6 +7,7 @@ import (
"net"
"net/netip"
"os"
"runtime"
"strings"
"tailscale.com/net/netmon"
@@ -24,6 +25,11 @@ func dnsFns() []dnsFn {
}
func dns4() []string {
// Skip route-based DNS discovery on Android
if runtime.GOOS == "android" {
return nil
}
f, err := os.Open(v4RouteFile)
if err != nil {
return nil
@@ -64,6 +70,11 @@ func dns4() []string {
}
func dns6() []string {
// Skip route-based DNS discovery on Android
if runtime.GOOS == "android" {
return nil
}
f, err := os.Open(v6RouteFile)
if err != nil {
return nil
@@ -98,6 +109,11 @@ func dns6() []string {
}
func dnsFromSystemdResolver() []string {
// Skip systemd resolver on Android
if runtime.GOOS == "android" {
return nil
}
c, err := resolvconffile.ParseFile("/run/systemd/resolve/resolv.conf")
if err != nil {
return nil
+27 -18
View File
@@ -4,6 +4,7 @@ package ctrld
import (
"net"
"net/netip"
"slices"
"time"
@@ -17,6 +18,31 @@ func currentNameserversFromResolvconf() []string {
return resolvconffile.NameServers()
}
// localNameservers filters a list of nameserver strings, returning only those
// that are not loopback or local machine IP addresses.
func localNameservers(nss []string, regularIPs, loopbackIPs []netip.Addr) []string {
var result []string
seen := make(map[string]bool)
for _, ns := range nss {
if ip := net.ParseIP(ns); ip != nil {
// skip loopback and local IPs
isLocal := false
for _, v := range slices.Concat(regularIPs, loopbackIPs) {
if ip.String() == v.String() {
isLocal = true
break
}
}
if !isLocal && !seen[ip.String()] {
seen[ip.String()] = true
result = append(result, ip.String())
}
}
}
return result
}
// dnsFromResolvConf reads usable nameservers from /etc/resolv.conf file.
// A nameserver is usable if it's not one of current machine's IP addresses
// and loopback IP addresses.
@@ -35,24 +61,7 @@ func dnsFromResolvConf() []string {
}
nss := resolvconffile.NameServers()
var localDNS []string
seen := make(map[string]bool)
for _, ns := range nss {
if ip := net.ParseIP(ns); ip != nil {
// skip loopback IPs
for _, v := range slices.Concat(regularIPs, loopbackIPs) {
ipStr := v.String()
if ip.String() == ipStr {
continue
}
}
if !seen[ip.String()] {
seen[ip.String()] = true
localDNS = append(localDNS, ip.String())
}
}
}
localDNS := localNameservers(nss, regularIPs, loopbackIPs)
// If we successfully read the file and found nameservers, return them
if len(localDNS) > 0 {
+105
View File
@@ -0,0 +1,105 @@
//go:build unix
package ctrld
import (
"net/netip"
"testing"
)
func Test_localNameservers(t *testing.T) {
loopbackIPs := []netip.Addr{
netip.MustParseAddr("127.0.0.1"),
netip.MustParseAddr("::1"),
}
regularIPs := []netip.Addr{
netip.MustParseAddr("192.168.1.100"),
netip.MustParseAddr("10.0.0.5"),
}
tests := []struct {
name string
nss []string
regularIPs []netip.Addr
loopbackIPs []netip.Addr
want []string
}{
{
name: "filters loopback IPv4",
nss: []string{"127.0.0.1", "8.8.8.8"},
regularIPs: nil,
loopbackIPs: loopbackIPs,
want: []string{"8.8.8.8"},
},
{
name: "filters loopback IPv6",
nss: []string{"::1", "1.1.1.1"},
regularIPs: nil,
loopbackIPs: loopbackIPs,
want: []string{"1.1.1.1"},
},
{
name: "filters local machine IPs",
nss: []string{"192.168.1.100", "8.8.4.4"},
regularIPs: regularIPs,
loopbackIPs: nil,
want: []string{"8.8.4.4"},
},
{
name: "filters both loopback and local IPs",
nss: []string{"127.0.0.1", "192.168.1.100", "8.8.8.8"},
regularIPs: regularIPs,
loopbackIPs: loopbackIPs,
want: []string{"8.8.8.8"},
},
{
name: "deduplicates results",
nss: []string{"8.8.8.8", "8.8.8.8", "1.1.1.1"},
regularIPs: regularIPs,
loopbackIPs: loopbackIPs,
want: []string{"8.8.8.8", "1.1.1.1"},
},
{
name: "all filtered returns nil",
nss: []string{"127.0.0.1", "::1", "192.168.1.100"},
regularIPs: regularIPs,
loopbackIPs: loopbackIPs,
want: nil,
},
{
name: "empty input returns nil",
nss: nil,
regularIPs: regularIPs,
loopbackIPs: loopbackIPs,
want: nil,
},
{
name: "skips unparseable entries",
nss: []string{"not-an-ip", "8.8.8.8"},
regularIPs: regularIPs,
loopbackIPs: loopbackIPs,
want: []string{"8.8.8.8"},
},
{
name: "no local IPs filters nothing",
nss: []string{"8.8.8.8", "1.1.1.1"},
regularIPs: nil,
loopbackIPs: nil,
want: []string{"8.8.8.8", "1.1.1.1"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := localNameservers(tt.nss, tt.regularIPs, tt.loopbackIPs)
if len(got) != len(tt.want) {
t.Fatalf("localNameservers() = %v, want %v", got, tt.want)
}
for i := range got {
if got[i] != tt.want[i] {
t.Errorf("localNameservers()[%d] = %q, want %q", i, got[i], tt.want[i])
}
}
})
}
}
+52 -9
View File
@@ -56,6 +56,8 @@ const (
var (
dcRetryMu sync.Mutex
dcRetryCancel context.CancelFunc
dcRetryDomain string
dcRetryID uint64
// Lazy-loaded netapi32 for DsGetDcNameW calls.
netapi32DLL = windows.NewLazySystemDLL("netapi32.dll")
@@ -133,9 +135,6 @@ func dnsFromAdapter() []string {
func getDNSServers(ctx context.Context) ([]string, error) {
logger := *ProxyLogger.Load()
// Cancel any in-flight DC retry from a previous network state.
cancelDCRetry()
// Check context before making the call
if ctx.Err() != nil {
return nil, ctx.Err()
@@ -157,6 +156,9 @@ func getDNSServers(ctx context.Context) ([]string, error) {
var dcServers []string
var adDomain string
isDomain := checkDomainJoined()
if !isDomain {
cancelDCRetry()
}
if isDomain {
domainName, err := system.GetActiveDirectoryDomain()
if err != nil {
@@ -164,6 +166,7 @@ func getDNSServers(ctx context.Context) ([]string, error) {
"Failed to get local AD domain: %v", err)
} else {
adDomain = domainName
cancelDCRetryForOtherDomain(domainName)
// Load netapi32.dll
var info *DomainControllerInfo
flags := uint32(DS_RETURN_DNS_NAME | DS_IP_REQUIRED | DS_IS_DNS_NAME)
@@ -205,7 +208,7 @@ func getDNSServers(ctx context.Context) ([]string, error) {
// Start background retry for transient DC errors.
if isTransientDCError(ret) {
Log(ctx, logger.Info(),
"AD DC detection failed with transient error %d for %s, starting background retry", ret, domainName)
"AD DC detection failed with retryable error %d for %s, ensuring background retry", ret, domainName)
startDCRetry(domainName)
}
} else if info != nil {
@@ -219,6 +222,7 @@ func getDNSServers(ctx context.Context) ([]string, error) {
if ip := net.ParseIP(dcAddr); ip != nil {
dcServers = append(dcServers, ip.String())
cancelDCRetry()
Log(ctx, logger.Debug(),
"Added domain controller DNS servers: %v", dcServers)
}
@@ -387,10 +391,13 @@ func checkDomainJoined() bool {
}
// isTransientDCError returns true if the DsGetDcName error code indicates
// a transient failure that may succeed on retry.
// a transient failure that may succeed on retry. ERROR_NO_SUCH_DOMAIN is
// retryable here because we only call this path after Windows already reported
// the machine is domain joined and returned a local AD domain name; during VPN
// DNS churn, DC locator can temporarily fail to resolve that known domain.
func isTransientDCError(code uintptr) bool {
switch code {
case errConnReset, errRPCUnavailable, errNoLogonServers, errDCNotFound, errNetUnreachable:
case errNoSuchDomain, errConnReset, errRPCUnavailable, errNoLogonServers, errDCNotFound, errNetUnreachable:
return true
default:
return false
@@ -404,23 +411,49 @@ func cancelDCRetry() {
if dcRetryCancel != nil {
dcRetryCancel()
dcRetryCancel = nil
dcRetryDomain = ""
dcRetryID = 0
}
}
// cancelDCRetryForOtherDomain keeps an existing retry alive during noisy
// network-change refreshes, but stops it if Windows reports a different AD
// domain. This avoids the start/cancel storm seen when DsGetDcName briefly
// returns ERROR_NO_SUCH_DOMAIN while VPN DNS is still settling.
func cancelDCRetryForOtherDomain(domainName string) {
dcRetryMu.Lock()
defer dcRetryMu.Unlock()
if dcRetryCancel == nil || dcRetryDomain == "" || strings.EqualFold(dcRetryDomain, domainName) {
return
}
dcRetryCancel()
dcRetryCancel = nil
dcRetryDomain = ""
dcRetryID = 0
}
// startDCRetry spawns a background goroutine that retries DsGetDcName with
// exponential backoff. On success it appends the DC IP to the OS resolver.
func startDCRetry(domainName string) {
dcRetryMu.Lock()
// Cancel any previous retry.
if dcRetryCancel != nil && strings.EqualFold(dcRetryDomain, domainName) {
ProxyLogger.Load().Debug().Msgf("AD DC retry already running for domain %s", domainName)
dcRetryMu.Unlock()
return
}
if dcRetryCancel != nil {
dcRetryCancel()
}
ctx, cancel := context.WithCancel(context.Background())
dcRetryID++
retryID := dcRetryID
dcRetryCancel = cancel
dcRetryDomain = domainName
dcRetryMu.Unlock()
go func() {
go func(retryID uint64) {
logger := *ProxyLogger.Load()
defer clearDCRetryIfCurrent(domainName, retryID)
delay := dcRetryInitialDelay
for attempt := 1; attempt <= dcRetryMaxAttempts; attempt++ {
@@ -472,7 +505,17 @@ func startDCRetry(domainName string) {
Log(ctx, logger.Warn(),
"AD DC retry exhausted %d attempts for domain %s", dcRetryMaxAttempts, domainName)
}()
}(retryID)
}
func clearDCRetryIfCurrent(domainName string, retryID uint64) {
dcRetryMu.Lock()
defer dcRetryMu.Unlock()
if dcRetryCancel != nil && dcRetryID == retryID && strings.EqualFold(dcRetryDomain, domainName) {
dcRetryCancel = nil
dcRetryDomain = ""
dcRetryID = 0
}
}
// tryGetDCAddress attempts a single DsGetDcName call and returns the DC IP on success,