The test:windows CI job intermittently failed to clean up .testbin with
"Access to the path '...cmd_cli.test.exe' is denied". This was previously
attributed to Windows Defender scanning the large unsigned test binaries,
and mitigated with Defender exclusions and cleanup retries. That was
treating a symptom.
Root cause: performUpgrade() self-upgrades by running
exec.Command(os.Executable(), "upgrade", "prod", "-vv") as a detached,
windowless child. In the real ctrld binary this re-execs ctrld and is
correct. Under `go test`, os.Executable() is the test binary itself, and
`go test` stops flag parsing at the first positional arg ("upgrade") and
ignores the rest -- so the child silently re-runs the entire test suite.
That child hits the upgrade tests again and spawns more detached children,
recursively: a fork bomb of hidden processes that pins the runner's
CPU/memory and keeps the test binary's image file locked. Windows refuses
to delete the image of a running process, hence the "Access is denied"
during after_script. Whether any children are still alive when cleanup
runs is a timing race, which is why the failure was flaky.
Two tests reached this path: Test_performUpgrade (directly) and
Test_selfUpgradeCheck (via selfUpgradeCheck -> performUpgrade on the
"upgrade allowed" case).
Fix:
- prog.go: extract the command construction into a package-level
newUpgradeCmd var. Production behavior is unchanged.
- main_test.go: stub newUpgradeCmd once in TestMain so the whole test
binary self-execs with `-test.run=^$` (matches no tests, exits
immediately) instead of re-running the suite. This covers every test
that reaches performUpgrade, present and future, while still exercising
the cmd.Start() success path.
DoH, DoH3, and DoQ response paths previously used io.ReadAll on
attacker-controlled upstream responses before enforcing any protocol-level
size limit. A malicious or compromised upstream could return an oversized
body or stream and force ctrld to buffer unbounded data before eventually
failing DNS parsing.
Cap DoH/DoH3 response bodies at dns.MaxMsgSize and cap DoQ streams at the
2-byte length prefix plus dns.MaxMsgSize. Also limit non-200 DoH error
bodies so error formatting cannot consume large upstream responses.
DoQ pools now keep a single quic.Transport and UDP socket for all dials,
so parallel dial and reconnect churn no longer allocate a new socket per
attempt or leak the winner's UDP conn when the caller owns the packet
conn.
quicParallelDialer accepts an optional transport: when set, dials use
Transport.DialEarly on that socket; when nil, behavior matches the old
per-dial ListenUDP path (losers close their sockets).
Per RFC 9250 §4.2, close the query stream's send side before reading the
response so strict upstreams see STREAM FIN before answering.
CloseIdleConnections closes the shared transport and underlying UDP
conn so checked-out connections and the OS socket are torn down.
Add a FIN-strict test server, coverage for bootstrap vs parallel-dial
paths, and a Linux-only FD churn regression test.
DoQ responses are length-prefixed per RFC 9250. The resolver previously
assumed the stream always contained at least two bytes and unpacked from
buf[2:], which could panic on truncated or malicious replies.
Validate the prefix against the bytes read, return a clear error, and
retire the connection from the pool on framing failure. Unpack only the
slice declared by the prefix so a short read cannot be misinterpreted as
a full message.
Add regression coverage with a small test server that returns malformed
raw payloads (empty, one byte, prefix-only, prefix larger than payload).
Currently there is no limit on PIN attempts, allowing unlimited
brute force if an attacker gains socket access. While the socket is
root-only by default, rate limiting is cheap defense-in-depth.
Current code writes to a predictable path, which on systems without
`fs.protected_symlinks` (e.g. embedded routers) could allow a local
attacker with API compromise to perform symlink attacks.
Go's default is already TLS 1.2+ (since Go 1.18), but making this
explicit satisfies RFC 7858/9250 recommendations and makes the security
intent clear for auditors.
Three changes to reduce worst-case recovery from ~30s to <3s:
1. debounceRecovery() for network changes (500ms window) — coalesces
rapid consecutive network changes into a single recovery pass,
eliminating the cancel-and-restart race.
2. ForceReBootstrap() on recovery entry — closes dead connections and
creates fresh transports synchronously before probing, replacing
the lazy ReBootstrap() flag that left stale connections.
3. Combined effect: recovery probes never inherit dead connections
from a canceled prior recovery attempt.
When third-party VPN software (e.g., OpenVPN) installs WFP block filters via
block-outside-dns, all DNS traffic to non-tunnel interfaces is blocked —
including DNS to 127.0.0.1 (ctrld's NRPT target). This breaks DNS mode
interception because the NRPT catch-all rule routes queries to loopback,
but WFP blocks the connection before it reaches ctrld's listener.
Fix: after exhausting all NRPT recovery attempts, activate a minimal WFP
session with "hard permit" filters (FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT)
for DNS to localhost in a max-priority sublayer (weight 0xFFFF). This
overrides the VPN's block for loopback DNS only, while preserving the
VPN's DNS leak protection for all other (non-loopback) DNS traffic.
The loopback protect is:
- Only activated when NRPT probes fail (not preemptively)
- Harmless when no conflicting WFP blocks exist (permit-only, no blocks)
- Persistent until ctrld shutdown (survives VPN reconnect cycles)
- Cleaned up by the existing cleanupWFPFilters path on shutdown
Add file-backed persistence to the internal logWriter so runtime logs
survive service restarts. When internal logging is enabled (CD mode,
no explicit log_path), writes are teed to both the existing in-memory
ring buffer and a rotated file on disk (ctrld.log in the home directory).
File rotation: 5MB max with 1 backup (ctrld.log.1), so max ~10MB on disk.
Log view/send now reads from the persisted files (including backup) to
provide complete history across restarts. Live tail continues to use
the in-memory subscriber mechanism unchanged.
Activation: same conditions as existing internal logging — CD mode only,
no log_path configured. No new config options or dependencies.
SetSelfIP unconditionally accessed t.dhcp, but t.dhcp is only
initialized when DHCP discovery is enabled. A network change event
can fire SetSelfIP regardless of the discovery configuration,
causing a nil pointer dereference.
Guard the t.dhcp access with a nil check so the self IP is still
updated on the Table even when DHCP discovery is disabled.
Replace conn.OpenStream (non-blocking) with conn.OpenStreamSync so that
the resolver waits for the server's MAX_STREAMS credit replenishment frame
instead of immediately failing when the stream limit is temporarily
exhausted. Also retry on StreamLimitReachedError as defense-in-depth for
servers that are slow or fail to send MAX_STREAMS updates.
Pass a quic.Config with KeepAlivePeriod (15s) to DoQ dial calls instead
of nil, so pooled connections send periodic QUIC PINGs to stay alive and
detect dead paths proactively.
Also add IdleTimeoutError to the DoQ retry conditions alongside io.EOF,
so stale pooled connections trigger a transparent retry instead of
propagating as a query failure.
IPv6 DNS interception on macOS is not feasible with current pf capabilities.
The kernel rejects sendmsg from [::1] to global unicast (EINVAL), nat on lo0
doesn't fire for route-to'd packets, raw sockets bypass routing but pf doesn't
match them against rdr state, and DIOCNATLOOK can't be used because bind()
fails for non-local addresses.
Replace all IPv6 interception code with a simple pf block rule:
block out quick on ! lo0 inet6 proto { udp, tcp } from any to any port 53
macOS automatically retries DNS over IPv4 when IPv6 is blocked.
Changes:
- Remove rawipv6_darwin.go and rawipv6_other.go
- Remove [::1] listener spawn on macOS (needLocalIPv6Listener returns false)
- Remove IPv6 rdr, route-to, pass, and reply-to pf rules
- Add block rule for all outbound IPv6 DNS
- Update docs/pf-dns-intercept.md with what was tried and why it failed
The handler variable is dns.HandlerFunc but wrapIPv6Handler returns
dns.Handler (interface). Go's type inference picked dns.HandlerFunc
for ipv6Handler, causing a compile error on assignment. Explicit
type declaration fixes the mismatch.
macOS rejects sendmsg from [::1] to global unicast IPv6 (EINVAL), and
nat on lo0 doesn't fire for route-to'd packets (pf skips translation
on the second interface pass). ULA addresses on lo0 also fail (EHOSTUNREACH
- kernel segregates lo0 routing).
Solution: wrap the [::1] UDP listener's ResponseWriter with rawIPv6Writer
that sends responses via SOCK_RAW (IPPROTO_UDP) on lo0, bypassing the
kernel's routing validation. pf's rdr state reverses the address
translation on the response path.
Changes:
- Add rawipv6_darwin.go: rawIPv6Writer wraps dns.ResponseWriter, sends
UDP responses via raw IPv6 socket with proper checksum calculation
- Add rawipv6_other.go: no-op wrapIPv6Handler for non-darwin platforms
- Remove nat rules from pf anchor (no longer needed)
- Block IPv6 TCP DNS (block return) - falls back to IPv4 (~1s, rare)
- Remove IPv6 TCP rdr/route-to/pass rules (only UDP intercepted)
upstreamConfigFor() used strings.Contains(":") to detect whether to
append ":53", but IPv6 addresses contain colons, so IPv6 servers were
passed as bare addresses (e.g. "2a0d:6fc0:9b0:3600::1") to net.Dial
which rejects them with "too many colons in address".
Use net.JoinHostPort() which handles both IPv4 and IPv6 correctly,
producing "[2a0d:6fc0:9b0:3600::1]:53" for IPv6.
This commit adds a new `ctrld log tail` subcommand that streams
runtime debug logs to the terminal in real-time, similar to `tail -f`.
Changes:
- log_writer.go: Add Subscribe/tailLastLines for fan-out to tail clients
- control_server.go: Add /log/tail endpoint with streaming response
- Internal logging: subscribes to logWriter for live data
- File-based logging: polls log file for new data (200ms interval)
- Sends last N lines as initial context on connect
- commands.go: Add `log tail` cobra subcommand with --lines/-n flag
- control_client.go: Add postStream() with no timeout for long-lived connections
Usage:
sudo ctrld log tail # shows last 10 lines then follows
sudo ctrld log tail -n 50 # shows last 50 lines then follows
Ctrl+C to stop
Add defer windows.CloseHandle(h) after CreateToolhelp32Snapshot to ensure
the process snapshot handle is properly released on all code paths (match
found, enumeration exhausted, or error).
The continue statement only broke out of the inner loop, so
loopback/local IPs (e.g. 127.0.0.1) were never filtered.
This caused ctrld to use itself as bootstrap DNS when already
installed as the system resolver — a self-referential loop.
Use the same isLocal flag pattern as getDNSFromScutil() and
getAllDHCPNameservers().
Implement VPN DNS discovery and split routing for intercept mode:
- Discover VPN DNS servers from F5 BIG-IP, Tailscale, Network
Extension VPNs, and traditional VPN adapters
- Exit mode detection (split vs full tunnel) via routing table
- Interface-scoped pf exemptions for VPN DNS traffic (macOS)
- Windows VPN adapter filtering with routable address check
- AD domain controller detection with retry on transient failure
- Cleanup of stale exemptions on VPN disconnect
Squashed from intercept mode development on v1.0 branch (#497).
Implement DNS interception on Windows with dual-mode support:
- NRPT for --intercept-mode=dns: catch-all rule redirecting all DNS
to ctrld's listener, with GP vs local path detection
- WFP for --intercept-mode=hard: sublayer with callout filters
intercepting port 53 traffic
- NRPT probe-and-heal for async Group Policy refresh race
- Service registry verification for intercept mode persistence
- NRPT diagnostics script for troubleshooting
Includes WFP technical reference docs and Windows test scripts.
Squashed from intercept mode development on v1.0 branch (#497).
Implement DNS interception on macOS using pf (packet filter):
- Anchor injection into running ruleset (not /etc/pf.conf)
- route-to lo0 + rdr rules for locally-originated DNS capture
- _ctrld group exemption so ctrld's own queries bypass interception
- Watchdog to detect and restore wiped anchor rules
- Probe-based auto-heal for Parallels VM pf corruption
- IPv6 DNS blocking and block-return for clean timeouts
- Interface-specific tunnel detection for VPN coexistence
- Port 5354 fallback in intercept mode
Includes pf technical reference docs and test scripts.
Squashed from intercept mode development on v1.0 branch (#497).
Add --intercept-mode flag (dns/hard/off) with configuration support,
recovery bypass for captive portals, probe-based interception
verification, VPN DNS coexistence in the proxy layer, and IPv6
loopback listener guard.
Remove standalone mDNSResponder hack files — the port 53 binding
logic is now handled within the intercept mode infrastructure.
Squashed from intercept mode development on v1.0 branch (#497).
Send all available hostname sources (ComputerName, LocalHostName,
HostName, os.Hostname) in the metadata map when provisioning.
This allows the API to detect and repair generic hostnames like
'Mac' by picking the best available source server-side.
Belt and suspenders: preferredHostname() picks the right one
client-side, but metadata gives the API a second chance.
macOS Sequoia with Private Wi-Fi Address enabled causes os.Hostname()
to return generic names like "Mac.lan" from DHCP instead of the real
computer name. The /utility provisioning endpoint sends this raw,
resulting in devices named "Mac-lan" in the dashboard.
Fallback chain: ComputerName → LocalHostName → os.Hostname()
LocalHostName can also be affected by DHCP. ComputerName is the
user-set display name from System Settings, fully immune to network state.
When mDNSResponder is using port 53 on macOS, adjust listener config to
use 0.0.0.0:53, stop mDNSResponder before binding, and run cleanup on
install and uninstall so the DNS server can start reliably.
On Darwin 26.2+, sudo no longer preserves SUDO_USER, LOGNAME, and USER
(CVE-2025-43416), so env-based detection fails. Use the logname(1)
command on Unix first, then fall back to environment variables and
user.Current() so the real login user is still reported correctly.
Replace the map-based pool and refCount bookkeeping with a channel-based
pool. Drop the closed state, per-connection address tracking, and extra
mutexes so the pool relies on the channel for concurrency and lifecycle,
matching the approach used in the DoT pool.
Cuong Manh Le