Compare commits

..
Author SHA1 Message Date
Cuong Manh Le f8f66609da Merge pull request #330 from Control-D-Inc/release-branch-v1.5.6
Release v1.5.6
2026-08-24 23:26:42 +07:00
Anthony Wong d78e9bcf5b fix(cli): treat explicit intercept-mode off as final in listener setup
tryUpdateListenerConfig treated an explicit --intercept-mode off the same
as an empty flag and fell back to the persisted config value. run() selects
the listener strategy before it clears the persisted mode, so the first
start after a revert to standard mode selected the intercept strategy from
a stale dns/hard value while setDNS kept interception off.

Extract the resolution into listenerInterceptMode and make an explicit off
final, the same contract as setDNS. Add a regression test that fails
without the fix.
2026-08-22 01:34:48 +07:00
Cuong Manh Le 30acb846ca Bump staticcheck-action to v1.4.1
While at it, also removing the unmatched //lint line.
2026-08-21 15:38:35 +07:00
Dev Scribe 6615e431dc Apply managed DNS mode in the macOS package 2026-08-21 15:12:32 +07:00
Anthony Wong 1f001a559a feat(cli): add stable provisioning failure codes for manual and MDM installs 2026-08-21 14:52:02 +07:00
Cuong Manh Le 753d245029 Bump bump insomniacslk/dhcp to c76316d
For fixing nclient4 panic.

See: https://github.com/insomniacslk/dhcp/pull/583
2026-08-21 14:51:07 +07:00
Cuong Manh Le 8ce3b7ca6c cmd/cli: log the config error that rejected a custom config
The warning reported err, the resolver-config fetch error, which is nil on
every path that reaches it - so a rejected custom config was logged with no
reason attached. cfgErr holds the validation failure.
2026-08-21 14:50:43 +07:00
Cuong Manh Le 084c785ed5 cmd/cli: report whether ctrld finished starting up, not just what SCM thinks
"ctrld status" reported the service manager's view and nothing else, so it
printed "Service is running" and exited 0 for a process that was alive and
registered as started but had never got past startup: no control socket, no DNS
listener, no policy applied. The one command an operator reaches for first
confirmed the service was fine while the host had no working DNS.

Probe the control server's /started endpoint before reporting success. That
endpoint only answers once the onStarted hooks have completed, which is after the
listeners are up, so a successful probe means the process is serving rather than
merely alive. A service that is registered as running but cannot confirm startup
is now reported as such, with a pointer to the log, and exits 3 - distinct from
stopped (1) and unknown (2), because it needs a different response.

A probe blocked by permissions is not evidence of a broken service: an
unprivileged caller still gets "Service is running", with a note that startup was
not verified. The probe is bounded by a short timeout so status stays fast.

Document the exit codes in the command's help, and cover the probe (ready, not
finished starting, no socket, timed out) and the classification, including that
an unreadable socket is not reported as a failure.

The not-ready verdict is only reported when the probe could have found the
daemon's socket. socketDir() is caller-relative on unix - the system directory
when writable, the caller's home otherwise - so an unprivileged "ctrld status"
looks somewhere the root-owned daemon never listened and gets ENOENT, which is
"wrong path", not "not ready". Since only darwin has an elevation PreRun and the
root-level alias has none, that is the normal invocation; reporting exit 3 there
would have told a monitoring check to restart healthy daemons. Such a caller now
gets the service manager's view with startup reported as unverified. Windows and
mobile resolve the same directory for every caller, so the verdict stays fully
available on the platform the hung start was seen on. A successful probe is still
conclusive whoever ran it.
2026-08-21 14:50:27 +07:00
Cuong Manh Le 5c9d3dec4e cmd/cli: stop the replacement before rolling its binary back
Rollback ran os.Remove(bin) while the replacement service was still running from
that image. Windows locks a running executable, so the remove failed with
"Access is denied" - and it was fatal, so the os.Rename that restores the
previous binary never ran. The upgrade ended with the broken replacement still
installed and the working binary stranded at its _previous name.

Readiness failing is not evidence the process exited: the service manager can
report a started service whose process never became operational. So rollback now
stops the service and waits until the manager reports it stopped before touching
the executable, then cleans up DNS the way the restart path's Cleanup task does.

Restoring is now conditional on the previous binary reporting a version, since a
_previous file that exists but produces no version output would trade a service
that starts and hangs for one that cannot start at all. When it is unusable,
rollback keeps it for inspection, leaves the installed binary alone, and says so
instead of pressing on. The --version probe is bounded by a timeout so a binary
that hangs cannot hang the upgrade.

Remaining failures are reported rather than fatal, so each one says what state
the host was left in. os.Remove is retried while the path stays locked, since
Windows releases an image lock asynchronously after the process exits.

The helpers live in a new file rather than in commands.go, and the rollback is
extracted into rollbackToPreviousBinary() so it can be covered: the stop happens
while the executable is still present, an unusable previous binary is kept
without swapping or restarting, and a failed stop aborts before anything is
modified. Reversing the stop and the remove fails these tests.

The version probe is called through a variable so those tests do not have to
stage a runnable executable. Staging one is not portable: oldBin is
bin+"_previous", so a fixture named "ctrld" yields the extension-less
"ctrld_previous", which Windows refuses to execute, and a symlink to the test
binary needs a privilege Windows does not grant by default. The probe itself is
still covered against the real test binary. Production is unaffected: ctrld.exe
_previous does have an extension, and os/exec only appends PATHEXT entries when
a path has none at all - noted at binaryVersion so the suffix is not renamed
into something extension-less by accident.
2026-08-21 14:50:11 +07:00
Cuong Manh Le 2400f27962 all: keep the first-attempt error when the direct-ip fallback also fails
Both API requests and binary downloads retry against a hard-coded IP when the
attempt via hostname fails. Both then overwrote the first error with the
fallback's, so only the last failure was reported.

That discarded the diagnosis. When the hostname attempt is denied locally -
WSAEACCES on Windows, "An attempt was made to access a socket in a way forbidden
by its access permissions", which means the host is blocking ctrld - and the
direct-ip fallback fails with an unreachable IPv6 route, what surfaces to the
operator is "dial tcp6: no route to host": a routing problem that does not
exist, while the error naming the real cause is visible only in debug logs.

Report both failures instead, keeping the error chain intact so errors.Is still
matches either one. Also switch the final wrap in doWithRetry from %v to %w,
which had been flattening the chain even when a single error was reported.

This also changes retry classification, which is worth stating explicitly because
it is not obvious from "report both errors". processCDFlags decides whether to
keep backing off with errUrlNetworkError, which uses errors.As - and errors.As
returns the *first* match in the tree. Wrapping the hostname attempt first
therefore hands the predicate that attempt's failure, where previously only the
fallback's error survived to be classified.

The effect is intended. A locally denied socket (WSAEACCES) is not a transient
network error, so preflight now fails fast and reports instead of retrying
against a firewall that is not going to clear on its own. The case that justifies
retrying forever, a network unreachable on both attempts at boot, is unchanged.
Both classifications are pinned by tests, along with the wrap order they depend on
at each composition site, so reversing it fails loudly rather than silently
restoring the old behaviour.
2026-08-21 14:49:58 +07:00
Cuong Manh Le 4f730167d4 cmd/cli: bound API preflight by service lifetime
processCDFlags retries the resolver-config fetch indefinitely by design: a
device that has no working network at boot must eventually come up. The loop had
no cancellation, so a stop request arriving while the API is unreachable was
ignored - the process kept retrying long after the service reported itself
stopped, doing work on behalf of a service the OS considers stopped.

Thread a context through processCDFlags and derive it from p.stopCh, in both the
startup preflight and the config-reload path. The loop now returns as soon as the
context is cancelled, checked both before a retry and after backoff returns
(backoff can wake up on cancellation). A stop during preflight exits the way a
normal stop does, without Fatal, so the service manager does not treat it as a
failed start and apply its restart policy to a service the operator just asked
to stop.

Bind the two API requests themselves as well, so a stop does not have to wait out
an in-flight request. Without this the loop honours a stop only between attempts,
which leaves up to defaultTimeout (20s) of a request the service is no longer
interested in - the same "still working after Service stopped" the loop change
exists to end, one layer down.

Doing so means a context parameter on FetchResolverConfig, FetchResolverUID,
UpdateCustomLastFailed and SendLogs, since all four reach a request builder. The
callers that have no context pass context.Background(), which is what master
effectively does at those sites: its loggerCtx carries a logger, not
cancellation. doWithFallback needs no parameter, because it clones the request
with req.Context() and so inherits the binding. This also repairs
internal/controld/controld_test.go, which is behind //go:build controld and had
already been written against the context-taking signature, so it could not
compile.

Cover the cancellation paths; removing either check makes the tests hang until
timeout.

Sampling the stop state is the whole point of runAPIPreflight rather than doing
this inline. A stop and a failure need opposite handling - one exits quietly, the
other self-uninstalls a deleted device, surfaces the error to a mobile app, and
reports a failed start - so the two must not be confused. Reading it from the
context after cancelling would report "stopped" for every failure, since
CancelFunc sets ctx.Err() regardless of whether anyone asked to stop; the stop
channel is read directly instead, which also does not depend on the context
watcher goroutine having been scheduled.
2026-08-21 14:49:30 +07:00
Codescribe b74937fcf3 security: default metrics server to loopback
Fixes unauthenticated metrics exposure be defaulting to 127.0.0.1 when
no host is provided. Logs a warning when bound to non-loopback addresses.
2026-08-21 14:48:20 +07:00
Codescribe dfaad4a20d Redact provision token in logs 2026-08-21 14:48:10 +07:00
Cuong Manh Le d7c30b18ed fix(darwin): probe interception when stabilization finishes
The post-stabilization reconcile verifies rule text, which cannot tell a
live redirect from an anchor pf has stopped evaluating. Sleep/wake QA
caught exactly that split: references intact, anchor rules intact,
post-load verification passed, and every query through the system
resolver timing out while the direct listener answered.

Nothing else probed. The interception probe monitor stands down while
stabilization owns pf and is never re-armed afterwards, so functional
recovery waited for the periodic watchdog - 11 seconds in the captured
run, up to a full 30-second interval - on a host whose link and default
route were already back. The watchdog's probe then failed once, forced a
reload, and public and VPN split-DNS both recovered immediately.

Probe once at the end of stabilization and, if it fails, force exactly
one reload and confirm with one more probe. Not the probe monitor: that
keeps probing for ~7.5s and can force a reload per failed probe, where
this path needs a single bounded repair before handing back to the
watchdog. Skipped when a monitor already owns probing, when intercept
state is gone, or during exec backoff.

Hand ownership over deterministically rather than skipping on sight. A
probe monitor started by an ignored network change claimed
functional-probe ownership before checking whether it could work, then
stood down because stabilization still owned pf; the verifier read that
claimed flag as "somebody is probing" and skipped, so neither path
probed and recovery fell back to the watchdog anyway. The monitor now
checks eligibility before claiming, and the verifier waits out a holder
that releases, yielding only to one that keeps probing.

Extract the completion block into finishPFStabilization so the wiring is
testable, and cover the bounded repair, the healthy path that must not
reload, a prober that claims and stands down, a prober that keeps
working, and a monitor that must not claim ownership while stabilizing.
2026-08-21 14:47:58 +07:00
Dev Scribe a828c8853a fix: retry macOS OS resolver with route-selected source 2026-08-21 14:47:38 +07:00
Dev Scribe 4d026d836c windows: adopt GP-managed NRPT catch-all 2026-08-21 14:46:44 +07:00
Dev Scribe 779fe015f0 fix: validate pf state before stabilization 2026-08-21 14:45:56 +07:00
Cuong Manh Le 246c1b9691 Merge pull request #327 from Control-D-Inc/release-branch-v1.5.5
Release branch v1.5.5
2026-08-05 01:37:29 +07:00
Ginder Singh 959f49dae3 Add mobile sandbox optimizations for v1.5.3
- Skip systemd-resolved initialization on Android (ChromeOS crash fix)
- Skip system DNS discovery commands on iOS mobile (sandbox restrictions)
- Skip route-based DNS discovery on Android
- Skip systemd resolver on Android
- Add mobile platform checks to prevent sandbox access violations

These changes ensure ctrld works correctly in mobile sandboxed environments
where system commands and file access are restricted.
2026-08-05 00:39:42 +07:00
Cuong Manh Le 8ebe911b1a Bump golang.org/x/text to v0.40.0
For fixing GO-2026-5970.
2026-07-28 16:15:14 +07:00
Cuong Manh Le d38538f593 fix: partition DNS cache by EDNS Client Subnet
With cache_enable = true, one cache entry was shared by every client
asking the same name against the same upstream: the cache key
({Qtype, Qclass, Name, Upstream}) and the osResolver hot-cache/singleflight
key ("name:qtype:") both ignored the EDNS Client Subnet (ECS). A response
tailored for subnet A was therefore served to subnet B.

A cached answer's records are scoped to the network that generated them
(RFC 7871 §7.3), so sharing them across subnets returns the wrong
CDN/policy answer. Rewriting only the ECS option on the shared answer is
worse: forwarders that validate the echoed ECS (e.g. dnsmasq with
add-subnet) then accept the wrong-subnet answer instead of rejecting it as
a mismatch.

Partition both cache paths by a canonical ECS tuple (family, source-prefix,
masked address) via the new dnscache.CanonicalECS: the LRU key gains an ECS
field and the singleflight/hot-cache key appends the canonical ECS. Same
subnet still shares an entry; different subnets (or address families) never
do. Only a request with no ECS option collapses to the shared empty
partition; a carried /0 keeps its own family-scoped token, since it is
forwarded with an ECS option and must stay distinguishable from a no-ECS
query (RFC 7871 §7.3.1). SetCacheReply no longer touches ECS and only
reconciles the EDNS Cookie.

Adds real cache-path regression tests (LRU and osResolver hot cache) that
serve a different A record per subnet and verify the second subnet never
receives the first's record.

Fixes https://github.com/Control-D-Inc/ctrld/issues/324
2026-07-28 16:14:15 +07:00
Cuong Manh Le 737fc79b58 fix: harden DoH oversized-body tests against server write timing
TestDoHResolve_{OversizedBody_Rejected,NonOKStatus_BoundedErrorBody,
OversizedBody_DoH3} asserted how many bytes the test server managed to
write before the client tore down the connection. That count reflects
kernel socket send buffers and HTTP/2 flow-control windows, which vary
by OS and load, so the server could buffer the whole body before
teardown and fail the assertion. It flaked on the Windows CI runner, but
reproduces on Linux too.

Replace the server-side byte counter with a deterministic synchronization
point. The handler writes exactly the read cap (dohMaxResponseSize+1 for
the body, dohMaxErrorBodySize for the error path), flushes, then blocks
without ever returning, so the response stream never gets an EOF. The
test then requires Resolve to return the size/status error before the
handler is released: ctrld's bounded read (io.LimitReader) returns after
the capped prefix, while a read to EOF would block on the withheld stream
and trip the deadline.

This removes the socket-buffer timing dependence and, unlike asserting on
the returned error alone, still fails if the caps are removed -- verified
by reverting both reads in doh.go to io.ReadAll(resp.Body), which makes
all three tests time out.
2026-07-28 16:13:46 +07:00
Cuong Manh Le fa074f1f5e fix: count DoQ/UDP test server call before writing reply
countHandler incremented its call counter after w.WriteMsg, but the DNS
client returns as soon as it receives the reply. A test reading the
counter right after Resolve returned could therefore observe a stale
zero, e.g. Test_Edns0_CacheReply intermittently failing on CI with
"cache not hit, server was called: 0" while passing on retry.

Increment the counter before writing the reply so it is guaranteed
visible once the client has the response. Verified by widening the
post-write window to reproduce the failure deterministically, then
confirming the reordered handler passes 500x and under -race.
2026-07-28 16:13:36 +07:00
Cuong Manh Le c596ef586b fix: skip internal logging in silent mode
Running with --silent still created and grew log files in cd mode.
needInternalLogging() only checked cdUID and Service.LogPath, so a
--silent flag enabled internal logging, persisted it to disk, and
reset the global log level back to debug, overriding the NoLevel that
--silent had set.

Return false from needInternalLogging() when silent is set, so ctrld
neither creates the internal log file nor writes debug logs. Add
regression tests asserting needInternalLogging() is false in silent mode
and that initInternalLogging() creates no log file.

Refs https://github.com/Control-D-Inc/ctrld/issues/320
2026-07-28 16:13:19 +07:00
Cuong Manh Le a4cfd4e479 cmd/cli: discard upstream answers whose question mismatches the request
Defense in depth against cache poisoning: a compromised or misbehaving
upstream can return an answer for a different name than was asked (e.g.
records for attacker.example in response to a query for victim.example).
Such an answer would be cached under the legitimate request key and
served to subsequent queries.

Validate that the upstream answer echoes the request's question
(case-insensitive name plus Qtype/Qclass, per RFC 1035 section 4.1.2)
before serving or caching it. A mismatch is logged at debug level and
the upstream is skipped, failing safe to the next upstream or SERVFAIL.

Refs github.com/Control-D-Inc/ctrld/issues/322
2026-07-28 16:12:47 +07:00
Dev Scribe b79098658a fix: restore DNS during invalid-device uninstall 2026-07-28 16:12:36 +07:00
Cuong Manh Le d29e7d131e fix: wrong DoQ resolver rewriting upstream responses with SetReply
The DoQ resolver called SetReply on the already-unpacked upstream
response. SetReply is meant to build a reply from a request, so it
forces the RCODE to NOERROR and overwrites the Question with the
request's question. This masked upstream failures from the proxy's
failover logic (a SERVFAIL looked like a successful empty response) and
corrupted the Question section of the response served to clients.

Restore only the downstream transaction ID instead (RFC 9250 section
4.2.1 puts the DNS Message ID at 0 on the wire), preserving the upstream
RCODE, Question, and answer sections untouched. This matches how the DoH
and DoT resolvers return unpacked upstream responses.

Refs github.com/Control-D-Inc/ctrld/issues/322
2026-07-28 16:12:23 +07:00
Cuong Manh Le 836c9ccf12 fix: treat 464XLAT CLAT source as local, not WAN
On networks using 464XLAT (common on IPv6-only cellular carriers and iPhone
hotspots), the local machine's DNS queries can reach ctrld's listener with a
source address in the RFC 7335 IPv4 Service Continuity Prefix (192.0.0.0/29,
e.g. 192.0.0.2 on the CLAT/host side). isWanClient classified 192.0.0.x as a
WAN client, so with allow_wan_clients unset (the default) the query was refused,
breaking DNS resolution entirely on the affected connection even though it
originated from the local host.

Recognize the IPv4 Service Continuity Prefix as a local range, mirroring the
existing CGNAT special case:

- Add ipv4ServiceContinuityPrefix (192.0.0.0/29) and an isServiceContinuityAddr
  helper (single definition, reused by both call sites).
- isWanClient excludes the range, so 464XLAT/CLAT queries are served normally.
- isPrivatePtrLookup treats the range as private so reverse lookups are handled
  consistently.

Scoped to 192.0.0.0/29 (the exact 464XLAT range); this does not weaken
allow_wan_clients since no globally routable remote client can appear from it.
2026-07-28 16:11:51 +07:00
Cuong Manh Le 53d3d3d44a cmd/cli: preserve fallback listener port across reload in DNS intercept
On macOS DNS-intercept mode, when mDNSResponder owns *:53 ctrld falls back
to listening on 127.0.0.1:5354, and the pf rdr rules correctly redirect DNS
to the bound port at startup. However, DNS resolution later breaks with an
endless watchdog "anchor intact but probe FAILED -> force reload" loop and
`dig @127.0.0.1` timeouts.

Root cause is config reload. In CD mode, apiConfigReload refetches the
generated config (which always declares port 53) every hour and the reload
merge in runWait only inherits the running port when the new port is 0. The
generated config explicitly says 53, so `*p.cfg = *newCfg` reverts p.cfg to
port 53. The DNS listener goroutines are started only when !reload, so they
are never re-bound and stay on 5354. Every subsequent pf rebuild reads p.cfg
and targets the dead port 53.

Fix: after applying the reloaded config in DNS-intercept mode on darwin,
restore the actual bound listener IP/Port into the in-memory config via the
new preserveBoundListeners helper, logging the configured-vs-actual
divergence. A reload cannot move the running listener anyway, so this keeps
p.cfg consistent with reality; all pf rdr rules and the watchdog probe then
target the live port. The on-disk generated config is intentionally left
unchanged (still 53), so no generated-config change is required.
2026-07-28 16:11:37 +07:00
Cuong Manh Le d7f43ea4bf Merge pull request #323 from Control-D-Inc/release-branch-v1.5.4
Release v1.5.4
2026-07-14 21:06:20 +07:00
Dev Scribe 41ca69849a Add Windows NRPT recovery circuit breaker
Windows DNS intercept mode runs an NRPT health monitor that restores the
catch-all rule and re-signals DNS Client whenever Windows stops routing
queries to the local listener. When another agent (MDM, VPN, GPO) keeps
putting NRPT back into a broken state, that loop never converges: ctrld
repeatedly calls RefreshPolicyEx, Dnscache paramchange, and flushes the
DNS cache, producing continuous flash writes and SIEM noise while never
fixing anything.

Add a recovery limiter that trips after a configurable number of
consecutive recovery flows and enters a cooldown, during which recovery
is suppressed (logged at most once every 5 minutes). Clearing the
circuit requires two consecutive stable health successes rather than
one, because a probe can pass briefly right after delete/re-add even
when the underlying NRPT state is still broken.

New [service] options gate the behavior and default to the previous
unlimited behavior:
  - nrpt_recovery_max_attempts (default 0 = unlimited)
  - nrpt_recovery_cooldown     (default 30m)

Also collapse the repeated refresh + paramchange + flush sequence into a
single signalNRPTChange() helper, and make cleanGPPath /
cleanEmptyNRPTParent only mutate the registry and report whether cleanup
happened, so callers send exactly one DNS Client change signal instead
of several. When the GP DnsPolicyConfig parent exists but is empty,
nrptProbeAndHeal now cleans it and signals once before spending the
normal policy-refresh retry budget, since those retries cannot succeed
while DNS Client is stuck in GP mode.

Add unit tests for the limiter's cooldown, stable-reset, and unlimited
paths, and document the new options and the empty-GP repro.
2026-07-14 01:12:37 +07:00
Cuong Manh Le 0d8df38dc1 fix: back off unroutable IPv6 DoH upstream health-check spam
When IPv6 is available locally but the selected IPv6 DoH endpoint is
unroutable (e.g. dialing [2606:1a40::22]:443 returns "no route to host"
while IPv4 stays usable), ctrld re-bootstrapped and re-dialed the endpoint
every ~2s. A weekend soak produced ~46.7k "no route to host" lines, with
the dial/health-check loop dominating the log during bad windows.

Add bounded backoff/suppression for network-unreachable endpoints at two
levels:

- ParallelDialer (internal/net): track dial addresses that fail with
  ENETUNREACH/EHOSTUNREACH and skip them for an exponentially growing,
  bounded window (5s -> 60s). A successful dial clears the entry
  immediately, so recovery is preserved when the route returns. When every
  candidate is suppressed the dial fails fast and quietly instead of
  hammering known-unroutable addresses.

- Upstream recovery loop (cmd/cli): demote unreachable check failures to
  debug and back off the retry cadence (2s -> 60s) for an unreachable
  streak; any other failure resets to the base cadence.

The new IsUnreachable classifier lives in internal/net and is reused by
cmd/cli's errNetworkError, so the unreachable-errno matching has a single
definition. Note the explicit winsock constants (10051/10065) are required
on Windows: syscall.ENETUNREACH/EHOSTUNREACH are Go's portable "invented"
values and never equal the raw WSA codes a failing connect surfaces.

Suppression and backoff are always bounded, so IPv6 is never disabled until
restart and recovers on its own once the route is back. Split-stack
selection and the #549 macOS intercept recovery work are untouched.

Adds unit tests for the classifier, the dialer's suppression tracker, and
the recovery backoff schedule.
2026-07-14 01:12:34 +07:00
Dev Scribe 3ef17bc5b9 fix: back off macOS pf watchdog exec storms 2026-07-14 01:09:14 +07:00
Cuong Manh Le 5bf26da585 Merge pull request #318 from Control-D-Inc/release-branch-v1.5.3
Release branch v1.5.3
2026-06-22 14:40:36 +07:00
Cuong Manh Le a5d536ab79 Upgrade quic-go to v0.59.1
For fixing CVE-2026-40898.
2026-06-16 15:17:11 +07:00
Codescribe 735590d244 fix: allow intercept fallback for default listener 2026-06-16 15:05:19 +07:00
Codescribe 18f01baa01 fix: flush pf states after forced DNS intercept reload 2026-06-16 15:05:08 +07:00
Cuong Manh Le 723c7827ba fix: stop self-upgrade tests from fork-bombing the windows test runner
The test:windows CI job intermittently failed to clean up .testbin with
"Access to the path '...cmd_cli.test.exe' is denied". This was previously
attributed to Windows Defender scanning the large unsigned test binaries,
and mitigated with Defender exclusions and cleanup retries. That was
treating a symptom.

Root cause: performUpgrade() self-upgrades by running
exec.Command(os.Executable(), "upgrade", "prod", "-vv") as a detached,
windowless child. In the real ctrld binary this re-execs ctrld and is
correct. Under `go test`, os.Executable() is the test binary itself, and
`go test` stops flag parsing at the first positional arg ("upgrade") and
ignores the rest -- so the child silently re-runs the entire test suite.
That child hits the upgrade tests again and spawns more detached children,
recursively: a fork bomb of hidden processes that pins the runner's
CPU/memory and keeps the test binary's image file locked. Windows refuses
to delete the image of a running process, hence the "Access is denied"
during after_script. Whether any children are still alive when cleanup
runs is a timing race, which is why the failure was flaky.

Two tests reached this path: Test_performUpgrade (directly) and
Test_selfUpgradeCheck (via selfUpgradeCheck -> performUpgrade on the
"upgrade allowed" case).

Fix:
- prog.go: extract the command construction into a package-level
  newUpgradeCmd var. Production behavior is unchanged.
- main_test.go: stub newUpgradeCmd once in TestMain so the whole test
  binary self-execs with `-test.run=^$` (matches no tests, exits
  immediately) instead of re-running the suite. This covers every test
  that reaches performUpgrade, present and future, while still exercising
  the cmd.Start() success path.
2026-06-16 14:52:45 +07:00
Dev Scribe 1e1c998c89 Refresh macOS VPN DNS after pf stabilization 2026-06-16 14:52:28 +07:00
Cuong Manh Le da454db8ef docker: update Dockerfile to use bookworm
Stick to go1.25 for now, since using go1.26 causing a runtime panic when
building arm platforms.
2026-06-16 14:49:33 +07:00
Cuong Manh Le 3fe9b27fb4 fix(doh,doq): reject oversized upstream DNS responses
DoH, DoH3, and DoQ response paths previously used io.ReadAll on
attacker-controlled upstream responses before enforcing any protocol-level
size limit. A malicious or compromised upstream could return an oversized
body or stream and force ctrld to buffer unbounded data before eventually
failing DNS parsing.

Cap DoH/DoH3 response bodies at dns.MaxMsgSize and cap DoQ streams at the
2-byte length prefix plus dns.MaxMsgSize. Also limit non-200 DoH error
bodies so error formatting cannot consume large upstream responses.
2026-06-16 14:49:01 +07:00
Cuong Manh Le 35455eb0b9 fix(doq): share QUIC transport, close send side before read (RFC 9250)
DoQ pools now keep a single quic.Transport and UDP socket for all dials,
so parallel dial and reconnect churn no longer allocate a new socket per
attempt or leak the winner's UDP conn when the caller owns the packet
conn.

quicParallelDialer accepts an optional transport: when set, dials use
Transport.DialEarly on that socket; when nil, behavior matches the old
per-dial ListenUDP path (losers close their sockets).

Per RFC 9250 §4.2, close the query stream's send side before reading the
response so strict upstreams see STREAM FIN before answering.

CloseIdleConnections closes the shared transport and underlying UDP
conn so checked-out connections and the OS socket are torn down.

Add a FIN-strict test server, coverage for bootstrap vs parallel-dial
paths, and a Linux-only FD churn regression test.
2026-06-16 14:48:43 +07:00
Cuong Manh Le f1309121ae doq: validate DNS-over-QUIC response framing
DoQ responses are length-prefixed per RFC 9250. The resolver previously
assumed the stream always contained at least two bytes and unpacked from
buf[2:], which could panic on truncated or malicious replies.

Validate the prefix against the bytes read, return a clear error, and
retire the connection from the pool on framing failure. Unpack only the
slice declared by the prefix so a short read cannot be misinterpreted as
a full message.

Add regression coverage with a small test server that returns malformed
raw payloads (empty, one byte, prefix-only, prefix larger than payload).
2026-06-16 14:48:33 +07:00
Cuong Manh Le 06668a2b6c cmd/cli: rate-limit PIN brute-force on control socket
Currently there is no limit on PIN attempts, allowing unlimited
brute force if an attacker gains socket access. While the socket is
root-only by default, rate limiting is cheap defense-in-depth.
2026-06-16 14:47:27 +07:00
Cuong Manh Le 97e5e99b8d cmd/cli: use os.CreateTemp for symlink-safe temp file creation
Current code writes to a predictable path, which on systems without
`fs.protected_symlinks` (e.g. embedded routers) could allow a local
attacker with API compromise to perform symlink attacks.
2026-06-16 14:47:19 +07:00
Cuong Manh Le c54ff701bd internal/router/dnsmasq: use text/template instead of html/template
Since this is a plain-text config, not html.
2026-06-16 14:47:08 +07:00
Cuong Manh Le 33682e2312 all: explicit TLS MinVersion in tls.Config
Go's default is already TLS 1.2+ (since Go 1.18), but making this
explicit satisfies RFC 7858/9250 recommendations and makes the security
intent clear for auditors.
2026-06-16 14:46:42 +07:00
Cuong Manh Le d629ecda33 Merge pull request #317 from Control-D-Inc/update-ci
Update ci
2026-06-02 03:24:30 -04:00
Cuong Manh Le 87ddf03b90 .github/workflows: bump go and staticcheck version 2026-06-02 14:20:05 +07:00
Cuong Manh Le d49a4c67c9 Bump golang.org/x/net to v0.55.0
For GO-2026-5026 security fix.
2026-06-02 14:18:23 +07:00
Cuong Manh Le 2c38ff74c3 Merge pull request #316 from Control-D-Inc/release-branch-v1.5.2
Release v1.5.2
2026-06-02 03:08:55 -04:00
Cuong Manh Le 75e8447c75 test: isolate VPN DNS settling tests from host adapters 2026-06-01 16:30:34 +07:00
Codescribe 4395efcb22 fix: stabilize Windows VPN DNS during adapter settling
Fixes Windows DNS-intercept behavior for AD/internal split-rule domains
during sleep/wake or VPN adapter settling without relying on a fixed
timeout.
2026-06-01 15:41:38 +07:00
Cuong Manh Le 7e6f88b4ed Merge pull request #301 from Control-D-Inc/release-branch-v1.5.1
Release branch v1.5.1
2026-05-25 07:08:15 -04:00
Cuong Manh Le 5dd5846cca cmd/cli: skip upstream.os healthcheck when WFP loopback protect enabled
Since the check will always be failed in this case, causing unnecessary
log spamming.
2026-05-05 22:15:54 +07:00
Codescribe 2b27c148be dns: recovery race condition fix
Three changes to reduce worst-case recovery from ~30s to <3s:

1. debounceRecovery() for network changes (500ms window) — coalesces
   rapid consecutive network changes into a single recovery pass,
   eliminating the cancel-and-restart race.

2. ForceReBootstrap() on recovery entry — closes dead connections and
   creates fresh transports synchronously before probing, replacing
   the lazy ReBootstrap() flag that left stale connections.

3. Combined effect: recovery probes never inherit dead connections
   from a canceled prior recovery attempt.
2026-04-30 19:09:21 +07:00
CodeScribe 8cb383d87e dns_intercept: add WFP loopback protect for VPN block-outside-dns
When third-party VPN software (e.g., OpenVPN) installs WFP block filters via
block-outside-dns, all DNS traffic to non-tunnel interfaces is blocked —
including DNS to 127.0.0.1 (ctrld's NRPT target). This breaks DNS mode
interception because the NRPT catch-all rule routes queries to loopback,
but WFP blocks the connection before it reaches ctrld's listener.

Fix: after exhausting all NRPT recovery attempts, activate a minimal WFP
session with "hard permit" filters (FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT)
for DNS to localhost in a max-priority sublayer (weight 0xFFFF). This
overrides the VPN's block for loopback DNS only, while preserving the
VPN's DNS leak protection for all other (non-loopback) DNS traffic.

The loopback protect is:
- Only activated when NRPT probes fail (not preemptively)
- Harmless when no conflicting WFP blocks exist (permit-only, no blocks)
- Persistent until ctrld shutdown (survives VPN reconnect cycles)
- Cleaned up by the existing cleanupWFPFilters path on shutdown
2026-04-29 15:21:38 +07:00
Codescribe afed925404 log: persist internal runtime logs to disk
Add file-backed persistence to the internal logWriter so runtime logs
survive service restarts. When internal logging is enabled (CD mode,
no explicit log_path), writes are teed to both the existing in-memory
ring buffer and a rotated file on disk (ctrld.log in the home directory).

File rotation: 5MB max with 1 backup (ctrld.log.1), so max ~10MB on disk.
Log view/send now reads from the persisted files (including backup) to
provide complete history across restarts. Live tail continues to use
the in-memory subscriber mechanism unchanged.

Activation: same conditions as existing internal logging — CD mode only,
no log_path configured. No new config options or dependencies.
2026-04-29 15:12:44 +07:00
Cuong Manh Le d1ea70d688 fix: prevent panic on network change during SetSelfIP
SetSelfIP unconditionally accessed t.dhcp, but t.dhcp is only
initialized when DHCP discovery is enabled. A network change event
can fire SetSelfIP regardless of the discovery configuration,
causing a nil pointer dereference.

Guard the t.dhcp access with a nil check so the self IP is still
updated on the Table even when DHCP discovery is disabled.
2026-04-22 15:30:59 +07:00
Cuong Manh Le ed98104384 doq: use OpenStreamSync and retry on StreamLimitReachedError
Replace conn.OpenStream (non-blocking) with conn.OpenStreamSync so that
the resolver waits for the server's MAX_STREAMS credit replenishment frame
instead of immediately failing when the stream limit is temporarily
exhausted. Also retry on StreamLimitReachedError as defense-in-depth for
servers that are slow or fail to send MAX_STREAMS updates.
2026-04-13 17:56:16 +07:00
Codescribe eaa171f66f doq: configure QUIC keep-alive and retry on idle timeout
Pass a quic.Config with KeepAlivePeriod (15s) to DoQ dial calls instead
of nil, so pooled connections send periodic QUIC PINGs to stay alive and
detect dead paths proactively.

Also add IdleTimeoutError to the DoQ retry conditions alongside io.EOF,
so stale pooled connections trigger a transparent retry instead of
propagating as a query failure.
2026-04-13 17:55:57 +07:00
Cuong Manh Le 839b8236e7 docs: add known issue for daemon crashing on Merlin 2026-04-07 11:34:07 +07:00
Codescribe 3f59cdad1a fix: block IPv6 DNS in intercept mode, remove raw socket approach
IPv6 DNS interception on macOS is not feasible with current pf capabilities.
The kernel rejects sendmsg from [::1] to global unicast (EINVAL), nat on lo0
doesn't fire for route-to'd packets, raw sockets bypass routing but pf doesn't
match them against rdr state, and DIOCNATLOOK can't be used because bind()
fails for non-local addresses.

Replace all IPv6 interception code with a simple pf block rule:
  block out quick on ! lo0 inet6 proto { udp, tcp } from any to any port 53

macOS automatically retries DNS over IPv4 when IPv6 is blocked.

Changes:
- Remove rawipv6_darwin.go and rawipv6_other.go
- Remove [::1] listener spawn on macOS (needLocalIPv6Listener returns false)
- Remove IPv6 rdr, route-to, pass, and reply-to pf rules
- Add block rule for all outbound IPv6 DNS
- Update docs/pf-dns-intercept.md with what was tried and why it failed
2026-04-01 17:35:08 +07:00
Codescribe c55e2a722c fix: declare ipv6Handler as dns.Handler to match wrapIPv6Handler return type
The handler variable is dns.HandlerFunc but wrapIPv6Handler returns
dns.Handler (interface). Go's type inference picked dns.HandlerFunc
for ipv6Handler, causing a compile error on assignment. Explicit
type declaration fixes the mismatch.
2026-04-01 17:24:36 +07:00
Codescribe 22a796f673 fix: use raw IPv6 socket for DNS responses in macOS intercept mode
macOS rejects sendmsg from [::1] to global unicast IPv6 (EINVAL), and
nat on lo0 doesn't fire for route-to'd packets (pf skips translation
on the second interface pass). ULA addresses on lo0 also fail (EHOSTUNREACH
- kernel segregates lo0 routing).

Solution: wrap the [::1] UDP listener's ResponseWriter with rawIPv6Writer
that sends responses via SOCK_RAW (IPPROTO_UDP) on lo0, bypassing the
kernel's routing validation. pf's rdr state reverses the address
translation on the response path.

Changes:
- Add rawipv6_darwin.go: rawIPv6Writer wraps dns.ResponseWriter, sends
  UDP responses via raw IPv6 socket with proper checksum calculation
- Add rawipv6_other.go: no-op wrapIPv6Handler for non-darwin platforms
- Remove nat rules from pf anchor (no longer needed)
- Block IPv6 TCP DNS (block return) - falls back to IPv4 (~1s, rare)
- Remove IPv6 TCP rdr/route-to/pass rules (only UDP intercepted)
2026-04-01 17:24:17 +07:00
Codescribe 95dd871e2d fix: bracket IPv6 addresses in VPN DNS upstream config
upstreamConfigFor() used strings.Contains(":") to detect whether to
append ":53", but IPv6 addresses contain colons, so IPv6 servers were
passed as bare addresses (e.g. "2a0d:6fc0:9b0:3600::1") to net.Dial
which rejects them with "too many colons in address".

Use net.JoinHostPort() which handles both IPv4 and IPv6 correctly,
producing "[2a0d:6fc0:9b0:3600::1]:53" for IPv6.
2026-04-01 17:23:53 +07:00
Codescribe 5c0585b2e8 Add log tail command for live log streaming
This commit adds a new `ctrld log tail` subcommand that streams
runtime debug logs to the terminal in real-time, similar to `tail -f`.

Changes:
- log_writer.go: Add Subscribe/tailLastLines for fan-out to tail clients
- control_server.go: Add /log/tail endpoint with streaming response
  - Internal logging: subscribes to logWriter for live data
  - File-based logging: polls log file for new data (200ms interval)
  - Sends last N lines as initial context on connect
- commands.go: Add `log tail` cobra subcommand with --lines/-n flag
- control_client.go: Add postStream() with no timeout for long-lived connections

Usage:
  sudo ctrld log tail          # shows last 10 lines then follows
  sudo ctrld log tail -n 50    # shows last 50 lines then follows
  Ctrl+C to stop
2026-03-25 13:58:44 +07:00
Codescribe 112d1cb5a9 fix: close handle leak in hasLocalDnsServerRunning()
Add defer windows.CloseHandle(h) after CreateToolhelp32Snapshot to ensure
the process snapshot handle is properly released on all code paths (match
found, enumeration exhausted, or error).
2026-03-25 13:58:24 +07:00
Codescribe bd9bb90dd4 Fix dnsFromResolvConf not filtering loopback IPs
The continue statement only broke out of the inner loop, so
loopback/local IPs (e.g. 127.0.0.1) were never filtered.
This caused ctrld to use itself as bootstrap DNS when already
installed as the system resolver — a self-referential loop.

Use the same isLocal flag pattern as getDNSFromScutil() and
getAllDHCPNameservers().
2026-03-25 13:57:46 +07:00
Codescribe 82fc628bf3 docs: add DNS Intercept Mode section to README 2026-03-25 13:57:35 +07:00
Cuong Manh Le 2926c76b76 Merge pull request #295 from Control-D-Inc/release-branch-v1.5.0
Release branch v1.5.0
2026-03-04 20:56:15 +07:00
Cuong Manh Le fe08f00746 fix(darwin): correct pf rules tests 2026-03-03 15:36:46 +07:00
Cuong Manh Le 9be15aeec8 fix(windows): make staticcheck happy 2026-03-03 15:15:16 +07:00
Codescribe 9b2e51f53a feat: robust username detection and CI updates
Add platform-specific username detection for Control D metadata:
- macOS: directory services (dscl) with console user fallback
- Linux: systemd loginctl, utmp, /etc/passwd traversal
- Windows: WTS session enumeration, registry, token lookup
2026-03-03 14:29:58 +07:00
Codescribe e7040bd9f9 feat: add VPN DNS split routing
Implement VPN DNS discovery and split routing for intercept mode:
- Discover VPN DNS servers from F5 BIG-IP, Tailscale, Network
  Extension VPNs, and traditional VPN adapters
- Exit mode detection (split vs full tunnel) via routing table
- Interface-scoped pf exemptions for VPN DNS traffic (macOS)
- Windows VPN adapter filtering with routable address check
- AD domain controller detection with retry on transient failure
- Cleanup of stale exemptions on VPN disconnect

Squashed from intercept mode development on v1.0 branch (#497).
2026-03-03 14:29:31 +07:00
Codescribe 768cc81855 feat: add Windows NRPT and WFP DNS interception
Implement DNS interception on Windows with dual-mode support:
- NRPT for --intercept-mode=dns: catch-all rule redirecting all DNS
  to ctrld's listener, with GP vs local path detection
- WFP for --intercept-mode=hard: sublayer with callout filters
  intercepting port 53 traffic
- NRPT probe-and-heal for async Group Policy refresh race
- Service registry verification for intercept mode persistence
- NRPT diagnostics script for troubleshooting

Includes WFP technical reference docs and Windows test scripts.

Squashed from intercept mode development on v1.0 branch (#497).
2026-03-03 14:29:09 +07:00
Codescribe 289a46dc2c feat: add macOS pf DNS interception
Implement DNS interception on macOS using pf (packet filter):
- Anchor injection into running ruleset (not /etc/pf.conf)
- route-to lo0 + rdr rules for locally-originated DNS capture
- _ctrld group exemption so ctrld's own queries bypass interception
- Watchdog to detect and restore wiped anchor rules
- Probe-based auto-heal for Parallels VM pf corruption
- IPv6 DNS blocking and block-return for clean timeouts
- Interface-specific tunnel detection for VPN coexistence
- Port 5354 fallback in intercept mode

Includes pf technical reference docs and test scripts.

Squashed from intercept mode development on v1.0 branch (#497).
2026-03-03 14:27:43 +07:00
Codescribe 1e8240bd1c feat: introduce DNS intercept mode infrastructure
Add --intercept-mode flag (dns/hard/off) with configuration support,
recovery bypass for captive portals, probe-based interception
verification, VPN DNS coexistence in the proxy layer, and IPv6
loopback listener guard.

Remove standalone mDNSResponder hack files — the port 53 binding
logic is now handled within the intercept mode infrastructure.

Squashed from intercept mode development on v1.0 branch (#497).
2026-03-03 14:26:39 +07:00
Codescribe 12715e6f24 fix: include hostname hints in metadata for API-side fallback
Send all available hostname sources (ComputerName, LocalHostName,
HostName, os.Hostname) in the metadata map when provisioning.
This allows the API to detect and repair generic hostnames like
'Mac' by picking the best available source server-side.

Belt and suspenders: preferredHostname() picks the right one
client-side, but metadata gives the API a second chance.
2026-03-03 14:25:53 +07:00
Codescribe 147106f2b9 fix(darwin): use scutil for provisioning hostname (#485)
macOS Sequoia with Private Wi-Fi Address enabled causes os.Hostname()
to return generic names like "Mac.lan" from DHCP instead of the real
computer name. The /utility provisioning endpoint sends this raw,
resulting in devices named "Mac-lan" in the dashboard.

Fallback chain: ComputerName → LocalHostName → os.Hostname()

LocalHostName can also be affected by DHCP. ComputerName is the
user-set display name from System Settings, fully immune to network state.
2026-03-03 14:25:41 +07:00
Cuong Manh Le a4f0418811 fix(darwin): handle mDNSResponder on port 53 to avoid bind conflicts
When mDNSResponder is using port 53 on macOS, adjust listener config to
use 0.0.0.0:53, stop mDNSResponder before binding, and run cleanup on
install and uninstall so the DNS server can start reliably.
2026-03-03 14:25:25 +07:00
Cuong Manh Le 40c68a13a1 fix(metadata): detect login user via logname when running under sudo
On Darwin 26.2+, sudo no longer preserves SUDO_USER, LOGNAME, and USER
(CVE-2025-43416), so env-based detection fails. Use the logname(1)
command on Unix first, then fall back to environment variables and
user.Current() so the real login user is still reported correctly.
2026-03-03 14:25:11 +07:00
Cuong Manh Le 3f30ec30d8 refactor(doq): simplify DoQ connection pool implementation
Replace the map-based pool and refCount bookkeeping with a channel-based
pool. Drop the closed state, per-connection address tracking, and extra
mutexes so the pool relies on the channel for concurrency and lifecycle,
matching the approach used in the DoT pool.
2026-03-03 14:24:50 +07:00
Cuong Manh Le 4790eb2c88 refactor(dot): simplify DoT connection pool implementation
Replace the map-based pool and refCount bookkeeping with a channel-based
pool. Drop the closed state, per-connection address tracking, and
extra mutexes so the pool relies on the channel for concurrency and
lifecycle.
2026-03-03 14:24:39 +07:00
Cuong Manh Le da3ea05763 fix(dot): validate connections before reuse to prevent io.EOF errors
Add connection health check in getConn to validate TLS connections
before reusing them from the pool. This prevents io.EOF errors when
reusing connections that were closed by the server (e.g., due to idle
timeout).
2026-03-03 14:24:27 +07:00
Cuong Manh Le 209c9211b9 fix(dns): handle empty and invalid IP addresses gracefully
Add guard checks to prevent panics when processing client info with
empty IP addresses. Replace netip.MustParseAddr with ParseAddr to
handle invalid IP addresses gracefully instead of panicking.

Add test to verify queryFromSelf handles IP addresses safely.
2026-03-03 14:24:07 +07:00
Cuong Manh Le acbebcf7c2 perf(dot): implement connection pooling for improved performance
Implement TCP/TLS connection pooling for DoT resolver to match DoQ
performance. Previously, DoT created a new TCP/TLS connection for every
DNS query, incurring significant TLS handshake overhead. Now connections are
reused across queries, eliminating this overhead for subsequent requests.

The implementation follows the same pattern as DoQ, using parallel dialing
and connection pooling to achieve comparable performance characteristics.
2026-03-03 14:22:55 +07:00
Cuong Manh Le 2e8a0f00a0 fix(config): use three-state atomic for rebootstrap to prevent data race
Replace boolean rebootstrap flag with a three-state atomic integer to
prevent concurrent SetupTransport calls during rebootstrap. The atomic
state machine ensures only one goroutine can proceed from "started" to
"in progress", eliminating the need for a mutex while maintaining
thread safety.

States: NotStarted -> Started -> InProgress -> NotStarted

Note that the race condition is still acceptable because any additional
transports created during the race are functional. Once the connection
is established, the unused transports are safely handled by the garbage
collector.
2026-03-03 14:22:43 +07:00
Cuong Manh Le 1f4c47318e refactor(config): consolidate transport setup and eliminate duplication
Consolidate DoH/DoH3/DoQ transport initialization into a single
SetupTransport method and introduce generic helper functions to eliminate
duplicated IP stack selection logic across transport getters.

This reduces code duplication by ~77 lines while maintaining the same
functionality.
2026-03-03 14:22:32 +07:00
Cuong Manh Le e8d1a4604e perf(doq): implement connection pooling for improved performance
Implement QUIC connection pooling for DoQ resolver to match DoH3
performance. Previously, DoQ created a new QUIC connection for every
DNS query, incurring significant handshake overhead. Now connections are
reused across queries, eliminating this overhead for subsequent requests.

The implementation follows the same pattern as DoH3, using parallel dialing
and connection pooling to achieve comparable performance characteristics.
2026-03-03 14:22:16 +07:00
Cuong Manh Le 8d63a755ba Removing outdated netlink codes 2026-03-03 14:21:46 +07:00
Cuong Manh Le f05519d1c8 refactor(network): consolidate network change monitoring
Remove separate watchLinkState function and integrate link state change
handling directly into monitorNetworkChanges. This consolidates network
monitoring logic into a single place and simplifies the codebase.

Update netlink dependency from v1.2.1-beta.2 to v1.3.1 and netns from
v0.0.4 to v0.0.5 to use stable versions.
2026-03-03 14:21:27 +07:00
Cuong Manh Le 1804e6db67 fix(windows): improve DNS server discovery for domain-joined machines
Add DNS suffix matching for non-physical adapters when domain-joined.
This allows interfaces with matching DNS suffix to be considered valid
even if not in validInterfacesMap, improving DNS server discovery for
remote VPN scenarios.

While at it, also replacing context.Background() with proper ctx
parameter throughout the function for consistent context propagation.
2026-03-03 14:20:14 +07:00
Cuong Manh Le d0341497d1 Merge pull request #276 from Control-D-Inc/release-branch-v1.4.9
Release branch v1.4.9
2026-01-13 21:41:48 +07:00
Cuong Manh Le 27c5be43c2 fix(system): disable ghw warnings to reduce log noise
Disable warnings from ghw library when retrieving chassis information.
These warnings are undesirable but recoverable errors that emit unnecessary
log messages. Using WithDisableWarnings() suppresses them while maintaining
functionality.
2026-01-09 15:10:29 +07:00
Cuong Manh Le 3beffd0dc8 .github/workflows: temporary use actions/setup-go
Since WillAbides/setup-go-faster failed with macOS-latest.

See: https://github.com/WillAbides/setup-go-faster/issues/37
2025-12-18 17:10:43 +07:00
Cuong Manh Le 1f9c586444 docs: add documentation for runtime internal logging 2025-12-18 17:10:43 +07:00
Cuong Manh Le a92e1ca024 Upgrade quic-go to v0.57.1 2025-12-18 17:10:43 +07:00
Cuong Manh Le 705df72110 fix: remove incorrect transport close on DoH3 error
Remove the transport Close() call from DoH3 error handling path.
The transport is shared and reused across requests, and closing it
on error would break subsequent requests. The transport lifecycle
is already properly managed by the http.Client and the finalizer
set in newDOH3Transport().
2025-12-18 17:10:43 +07:00
Cuong Manh Le 22122c45b2 Including system metadata when posting to utility API 2025-12-18 17:10:39 +07:00
Cuong Manh Le 57a9bb9fab Merge pull request #268 from Control-D-Inc/release-branch-v1.4.8
Release branch v1.4.8
2025-12-02 21:39:38 +07:00
Cuong Manh Le 78ea2d6361 .github/workflows: upgrade staticcheck-action to v1.4.0
While at it, also bump go version to 1.24
2025-11-12 15:22:01 +07:00
Cuong Manh Le df3cf7ef62 Upgrade quic-go to v0.56.0 2025-11-12 15:15:16 +07:00
Cuong Manh Le 80e652b8d9 fix: ensure log and cache flags are processed during reload
During reload operations, log and cache flags were not being processed,
which prevented runtime internal logs from working correctly. To fix this,
processLogAndCacheFlags was refactored to accept explicit viper and config
parameters instead of relying on global state, enabling it to be called
during reload with the new configuration. This ensures that log and cache
settings are properly applied when the service reloads its configuration.
2025-11-12 15:15:05 +07:00
Cuong Manh Le 091c7edb19 Fix: Filter root domain from search domains on Linux
Remove empty and root domain (".") entries from search domains list
to prevent systemd-resolved errors. This addresses the issue where
systemd doesn't allow root domain in search domains configuration.

The filtering ensures only valid search domains are passed to
systemd-resolved, preventing DNS operation failures.
2025-11-12 15:14:40 +07:00
Cuong Manh Le 6c550b1d74 Upgrade quic-go to v0.55.0
While at it, also bump required go version to 1.24
2025-11-12 15:14:26 +07:00
Cuong Manh Le 3ca559e5a4 Merge pull request #264 from Control-D-Inc/release-branch-v1.4.7
Release branch v1.4.7
2025-10-07 01:02:39 +07:00
150 changed files with 27064 additions and 744 deletions
+4 -4
View File
@@ -9,18 +9,18 @@ jobs:
fail-fast: false
matrix:
os: ["windows-latest", "ubuntu-latest", "macOS-latest"]
go: ["1.24.x"]
go: ["1.26.x"]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 1
- uses: WillAbides/setup-go-faster@v1.8.0
- uses: actions/setup-go@v6
with:
go-version: ${{ matrix.go }}
- run: "go test -race ./..."
- uses: dominikh/staticcheck-action@v1.3.1
- uses: dominikh/staticcheck-action@v1.4.1
with:
version: "2025.1"
version: "2026.2"
install-go: false
cache-key: ${{ matrix.go }}
+2
View File
@@ -12,3 +12,5 @@ ctrld-*
# generated file
cmd/cli/rsrc_*.syso
ctrld
ctrld.exe
+63 -1
View File
@@ -100,7 +100,7 @@ docker build -t controldns/ctrld . -f docker/Dockerfile
# Usage
The cli is self documenting, so free free to run `--help` on any sub-command to get specific usages.
The cli is self documenting, so feel free to run `--help` on any sub-command to get specific usages.
## Arguments
```
@@ -266,5 +266,67 @@ The above will start a foreground process and:
- Excluding `*.company.int` and `very-secure.local` matching queries, that are forwarded to `10.0.10.1:53`
- Write a debug log to `/path/to/log.log`
## DNS Intercept Mode
When running `ctrld` alongside VPN software, DNS conflicts can cause intermittent failures, bypassed filtering, or configuration loops. DNS Intercept Mode prevents these issues by transparently capturing all DNS traffic on the system and routing it through `ctrld`, without modifying network adapter DNS settings.
### When to Use
Enable DNS Intercept Mode if you:
- Use corporate VPN software (F5, Cisco AnyConnect, Palo Alto GlobalProtect, Zscaler)
- Run overlay networks like Tailscale or WireGuard
- Experience random DNS failures when VPN connects/disconnects
- See gaps in your Control D analytics when VPN is active
- Have endpoint security software that also manages DNS
### Command
Windows (Admin Shell)
```shell
ctrld.exe start --intercept-mode dns --cd RESOLVER_ID_HERE
```
macOS
```shell
sudo ctrld start --intercept-mode dns --cd RESOLVER_ID_HERE
```
`--intercept-mode dns` automatically detects VPN internal domains and routes them to the VPN's DNS server, while Control D handles everything else.
To disable intercept mode on a service that already has it enabled:
Windows (Admin Shell)
```shell
ctrld.exe start --intercept-mode off
```
macOS
```shell
sudo ctrld start --intercept-mode off
```
This removes the intercept rules and reverts to standard interface-based DNS configuration.
### Platform Support
| Platform | Supported | Mechanism |
|----------|-----------|-----------|
| Windows | ✅ | NRPT (Name Resolution Policy Table) |
| macOS | ✅ | pf (packet filter) redirect |
| Linux | ❌ | Not currently supported |
### Features
- **VPN split routing** — VPN-specific domains are automatically detected and forwarded to the VPN's DNS server
- **Captive portal recovery** — Wi-Fi login pages (hotels, airports, coffee shops) work automatically
- **No network adapter changes** — DNS settings stay untouched, eliminating conflicts entirely
- **Automatic port 53 conflict resolution** — if another process (e.g., `mDNSResponder` on macOS) is already using port 53, `ctrld` automatically listens on a different port. OS-level packet interception redirects all DNS traffic to `ctrld` transparently, so no manual configuration is needed. This only applies to intercept mode.
### Tested VPN Software
- F5 BIG-IP APM
- Cisco AnyConnect
- Palo Alto GlobalProtect
- Tailscale (including Exit Nodes)
- Windscribe
- WireGuard
For more details, see the [DNS Intercept Mode documentation](https://docs.controld.com/docs/dns-intercept).
## Contributing
See [Contribution Guideline](./docs/contributing.md)
+206
View File
@@ -0,0 +1,206 @@
# SPEC: Stable customer-visible provisioning failure codes
Issue: [#586](https://gitlab.int.windscribe.com/controld/clients/ctrld/-/issues/586)
Requested by: Catt Garrod (@catt). Scope expanded by: Anthony Wong (@anthony).
## 1. Objective
Terminal provisioning failures in ctrld — bootstrap/API setup, listener
binding, and service installation/startup — must produce a stable,
support-facing failure identifier that survives process exit and reaches
both manual CLI users and MDM-driven installs. A customer or admin reports
one code; Support maps it to a scenario and a next action without asking
for reruns or verbose logs.
Motivating incident (v1.5.5, macOS): provisioning reached the Control D
API, then died with only `FTL listener.0 could not find available listen
ip and port`. The per-address UDP/TCP bind errors existed only at Info
level in an in-memory logger and vanished on exit. The macOS pkg
`postinstall` discards ctrld's stdout/stderr entirely and judges success
by plist existence, so nothing useful reached the MDM log.
**Users:** end customers and IT admins reporting failures; Support agents
triaging them; MDM/RMM operators reading installer logs.
### Failure contract (agreed design)
Three surfaces, all carrying the same identifier:
1. **Result file** — on terminal provisioning failure, ctrld writes a
small redacted JSON file (atomic write: temp + rename) in the ctrld
home directory (same base dir as the internal `ctrld.log`,
via `absHomeDir`). Removed/overwritten on later successful
provisioning so stale failures don't mislead. Schema:
```json
{
"version": 1,
"timestamp": "2026-08-18T12:00:00Z",
"stage": "listener",
"code": "LISTENER_BIND_FAILED",
"exit_code": 41,
"message": "could not find available listen ip and port",
"detail": {
"attempts": [
{"addr": "127.0.0.1:53", "proto": "udp", "os_error": "address already in use"}
]
}
}
```
`detail` is bounded (cap recorded bind attempts; cap string lengths)
and redacted by construction: no provisioning tokens, resolver IDs,
config contents, or unrelated host data.
2. **Exit code + final stderr line** — the installer-facing command
(`ctrld start`, and `ctrld run` when run manually in the foreground)
exits with a stage-scoped code and prints one final line containing
the string code and stage, e.g.
`provisioning failed: stage=listener code=LISTENER_BIND_FAILED (exit 41)`.
3. **Installer log (MDM path)** — `scripts/pkg/postinstall` stops
discarding the signal: it captures `ctrld start`'s output to a
private temp file, extracts only the fixed-charset identifier line
(`stage=[a-z]* code=[A-Z_]* (exit [0-9]*)` — structurally unable to
carry the token), and echoes it with the exit code into the
installer log. The result file's `message`/`detail` fields are
deliberately never surfaced there. The plist-existence check remains
the final success gate.
### Identifier format
- **Primary identifier: stable string codes.** Initial set —
bootstrap: `API_UNREACHABLE`, `API_REJECTED`, `API_DEVICE_INVALID`;
listener: `LISTENER_BIND_FAILED`, `LISTENER_CONFIGURED_ADDR_UNAVAILABLE`;
service: `SERVICE_INSTALL_FAILED`, `SERVICE_START_FAILED`,
`SERVICE_SELFCHECK_FAILED`. Codes are append-only; renames are new
codes plus a deprecation note in the mapping doc.
- **Secondary: stage-scoped process exit codes** as a coarse machine
signal: bootstrap 3039, listener 4049, service install/start 5059.
Each string code owns one exit code. Existing contracts are untouched:
`ctrld status` 03, deactivation-pin 126, success 0.
- One underlying failure maps to one code on every path (manual CLI and
MDM), on both branches.
### Propagation (daemon → installer)
The listener/bootstrap fatals fire inside the daemon process
(`ctrld run` under launchd/systemd/SCM), not in `ctrld start`. The
daemon writes the result file before exiting; the existing log-socket
exit notification (`notifyExitToLogServer`) already unblocks `ctrld
start`'s self-check. `ctrld start` then reads the result file, prints
the identifier, and exits with the mapped stage exit code. The daemon's
own exit-status semantics toward service managers are preserved —
in particular the deliberate exit-0 on permanent API rejection that
protects the restart-policy budget; the result file carries the failure
identity in that case.
### Support mapping
`docs/provisioning-failure-codes.md` in this repo: one row per code —
code, stage, exit code, failure scenario, next safe troubleshooting
action or evidence request. Updated in the same MR whenever a code is
added or changed.
### Branch scope
Full implementation on **both** `v1.0` (release line for v1.5.5) and
`master`. The branches diverge heavily (`v1.0`: zerolog fork,
`commands.go`, `service_status.go`, macOS pkg scripts; `master`: zap,
inline commands, no pkg scripts), so this is one shared contract
(codes, exit-code ranges, file schema, doc) implemented twice, as two
MRs referencing #586.
## 2. Commands
- Build: `go build ./...`
- Test: `go test ./cmd/cli/...` (full: `go test ./...`)
- Vet: `go vet ./...`
- Branch workflow: feature branch off `v1.0` for the v1.0 MR; separate
feature branch off `master` for the port MR. Rebase, never merge the
base branch in.
## 3. Project structure
New and touched files on `v1.0` (master port mirrors the same contract
at its equivalent emission points in its `cli.go`):
- `cmd/cli/provision_result.go` (new) — stage + code enums, exit-code
mapping, result-file schema, atomic write/read/clear helpers,
bounded/redacted detail builders. Pattern follows `service_status.go`
(small file: named constants + classifier + dedicated tests).
- `cmd/cli/provision_result_test.go` (new).
- `cmd/cli/cli.go` — emission points: `run()` bootstrap failure branches
(permanent rejection, invalid-device, fatal fetch), and
`tryUpdateListenerConfig` / `tryUpdateListenerConfigIntercept` fatals,
which now record per-attempt `{addr, proto, os_error}` bind detail.
- `cmd/cli/commands.go` — `initStartCmd`: doTasks install/start failures
and the self-check failure branch read the result file, print the
identifier, and exit with the stage code (replacing bare `os.Exit(1)`
on those paths).
- `scripts/pkg/postinstall` — propagate exit code + result-file contents
into the installer log (v1.0 only; master has no pkg scripts).
- `docs/provisioning-failure-codes.md` (new) — support mapping.
## 4. Code style
- Per repo conventions and global rules: guard clauses, small functions,
descriptive names, explicit error handling — never weaken existing
handling (e.g. keep the permanent-rejection exit-0 rationale intact).
- Comments only for non-obvious constraints (e.g. why the daemon must
still exit 0 on permanent rejection), simple-english, self-contained —
no issue/MR references in code.
- Match each branch's logging idiom: zerolog fork on `v1.0`, zap on
`master`. No new dependencies.
- Conventional Commits; MR titles in simple-english; both MRs reference
#586 (release-line MR carries `Closes #586`).
## 5. Testing strategy
Test-first where the harness allows. Coverage required by the issue:
- **Code/mapping unit tests** — every string code maps to exactly one
stage and one in-range exit code; ranges don't collide with existing
contracts (03 status, 126 pin).
- **Result file round-trip** — write/read/clear; atomic write; stale
file removed on success.
- **Redaction** — serialize a result built from inputs containing a
provision token, resolver ID, and config content; assert none appear.
- **Listener bind failure (regression test for the incident)** — occupy
a port, drive the listener-config path to exhaustion, assert the
result records `LISTENER_BIND_FAILED` with attempted address, UDP/TCP
operation, and OS error (`address already in use`-class).
- **Bootstrap failures** — mock API: permanent 4xx → `API_REJECTED`;
invalid-device 40402 → `API_DEVICE_INVALID`; unreachable →
`API_UNREACHABLE`.
- **Service install/start/self-check failures** — injected task
failures assert code selection and `ctrld start` exit code.
- **MDM surface** — shell-level check of `postinstall` failure branch
(result file present → correct log line and exit), aligned with the
existing `test-scripts/` approach; manual pkg verification steps
documented in the MR.
- Both branches: the shared contract tests exist on both; branch-specific
emission tests match each branch's structure.
## 6. Boundaries
**Always:**
- Redact tokens, resolver IDs, config contents, host data from every
customer-visible surface (result file, stderr line, installer log).
- Preserve existing exit-code contracts (`ctrld status` 03, pin 126)
and the daemon's service-manager-facing exit semantics.
- Bound all recorded detail (attempt counts, string lengths).
- Keep codes append-only once merged.
**Ask first:**
- Changing the daemon's (`ctrld run` under a service manager) exit codes
or restart-relevant behavior beyond writing the result file.
- Adding any persisted file outside the ctrld home directory.
- Expanding scope to runtime (post-provisioning) failures — this ticket
owns terminal provisioning failures only.
**Never:**
- Print or persist the provisioning token (the reason postinstall
discards output today — the replacement surface must stay token-free).
- Auto-detect or kill conflicting processes (explicitly out of scope).
- Break `ctrld status`'s documented exit-code contract.
+2 -1
View File
@@ -10,11 +10,12 @@ import (
hh "github.com/microsoft/wmi/pkg/hardware/host"
"github.com/Control-D-Inc/ctrld"
"github.com/Control-D-Inc/ctrld/internal/system"
)
// addExtraSplitDnsRule adds split DNS rule for domain if it's part of active directory.
func addExtraSplitDnsRule(cfg *ctrld.Config) bool {
domain, err := getActiveDirectoryDomain()
domain, err := system.GetActiveDirectoryDomain()
if err != nil {
mainLog.Load().Debug().Msgf("unable to get active directory domain: %v", err)
return false
+5 -3
View File
@@ -5,14 +5,16 @@ import (
"testing"
"time"
"github.com/Control-D-Inc/ctrld"
"github.com/Control-D-Inc/ctrld/testhelper"
"github.com/stretchr/testify/assert"
"github.com/Control-D-Inc/ctrld"
"github.com/Control-D-Inc/ctrld/internal/system"
"github.com/Control-D-Inc/ctrld/testhelper"
)
func Test_getActiveDirectoryDomain(t *testing.T) {
start := time.Now()
domain, err := getActiveDirectoryDomain()
domain, err := system.GetActiveDirectoryDomain()
if err != nil {
t.Fatal(err)
}
+535 -69
View File
@@ -147,6 +147,25 @@ func isMobile() bool {
return runtime.GOOS == "android" || runtime.GOOS == "ios"
}
func updateConfigInterceptMode(cfg *ctrld.Config, mode string) bool {
desired := ""
switch mode {
case "dns", "hard":
desired = mode
case "off":
desired = ""
case "":
return false
default:
return false
}
if cfg.Service.InterceptMode == desired {
return false
}
cfg.Service.InterceptMode = desired
return true
}
// isAndroid reports whether the current OS is Android.
func isAndroid() bool {
return runtime.GOOS == "android"
@@ -282,7 +301,7 @@ func run(appCallback *AppCallback, stopCh chan struct{}) {
}
p.mu.Unlock()
processLogAndCacheFlags()
processLogAndCacheFlags(v, &cfg)
// Log config do not have thing to validate, so it's safe to init log here,
// so it's able to log information in processCDFlags.
@@ -318,31 +337,55 @@ func run(appCallback *AppCallback, stopCh chan struct{}) {
}
if cdUID != "" {
validateCdUpstreamProtocol()
if rc, err := processCDFlags(&cfg); err != nil {
// Bound API preflight by the service lifetime. Without this, a stop request
// arriving while the API is unreachable leaves this retry/backoff loop running
// after "service stopped" was logged, so the process keeps working on behalf of
// a service the OS considers stopped.
pf := runAPIPreflight(p.stopCh, &cfg)
switch {
case pf.stopRequested:
// Stop requested during preflight, whether or not the fetch itself
// succeeded. A successful fetch does not entitle startup to continue: the
// operator asked for a stop, and carrying on would set up listeners and
// interception for a service the OS already considers stopping.
//
// Exit the way a normal stop does: no Fatal, so the OS service manager does
// not see a failed start and apply its restart policy to a service the
// operator just asked to stop.
mainLog.Load().Notice().Msg("stop requested while fetching resolver config, shutting down")
notifyExitToLogServer()
return
case pf.err != nil:
if isMobile() {
appCallback.Exit(err.Error())
appCallback.Exit(pf.err.Error())
return
}
cdLogger := mainLog.Load().With().Str("mode", "cd").Logger()
// Performs self-uninstallation if the ControlD device does not exist.
var uer *controld.ErrorResponse
if errors.As(err, &uer) && uer.ErrorField.Code == controld.InvalidConfigCode {
_ = uninstallInvalidCdUID(p, cdLogger, false)
}
notifyExitToLogServer()
cdLogger.Fatal().Err(err).Msg("failed to fetch resolver config")
} else {
handleAPIPreflightFailure(p, pf.err, notifyExitToLogServer)
return
default:
p.mu.Lock()
p.rc = rc
p.rc = pf.rc
p.mu.Unlock()
}
}
updated := updateListenerConfig(&cfg, notifyExitToLogServer)
// Bootstrap and listener binding both succeeded, so an earlier run's
// recorded failure no longer describes this install.
clearProvisionResult()
if cdUID != "" {
processLogAndCacheFlags()
processLogAndCacheFlags(v, &cfg)
}
// Keep config and the explicit CLI/service mode in sync. In particular, "off"
// must clear a previously persisted dns/hard value or the next service start
// would silently re-enable interception from config.
if updateConfigInterceptMode(&cfg, interceptMode) {
updated = true
mainLog.Load().Info().Msgf("writing intercept_mode = %q to config", cfg.Service.InterceptMode)
}
if updated {
@@ -441,18 +484,7 @@ func run(appCallback *AppCallback, stopCh chan struct{}) {
p.onStopped = append(p.onStopped, func() {
// restore static DNS settings or DHCP
p.resetDNS(false, true)
// Iterate over all physical interfaces and restore static DNS if a saved static config exists.
withEachPhysicalInterfaces("", "restore static DNS", func(i *net.Interface) error {
file := savedStaticDnsSettingsFilePath(i)
if _, err := os.Stat(file); err == nil {
if err := restoreDNS(i); err != nil {
mainLog.Load().Error().Err(err).Msgf("Could not restore static DNS on interface %s", i.Name)
} else {
mainLog.Load().Debug().Msgf("Restored static DNS on interface %s successfully", i.Name)
}
}
return nil
})
restoreSavedStaticDNS("", false)
})
close(waitCh)
@@ -628,6 +660,19 @@ const defaultDeactivationPin = -1
// cdDeactivationPin is used in cd mode to decide whether stop and uninstall commands can be run.
var cdDeactivationPin atomic.Int64
// Brute-force protection for the deactivation PIN endpoint on the control socket.
// After deactivationMaxFailedAttempts consecutive wrong PINs, further attempts are
// rejected for deactivationLockoutSeconds. Counter resets on a correct PIN.
const (
deactivationMaxFailedAttempts = 5
deactivationLockoutSeconds = 60
)
var (
deactivationFailedAttempts atomic.Int64
deactivationLockedUntil atomic.Int64
)
func init() {
cdDeactivationPin.Store(defaultDeactivationPin)
}
@@ -637,18 +682,218 @@ func deactivationPinSet() bool {
return cdDeactivationPin.Load() != defaultDeactivationPin
}
func processCDFlags(cfg *ctrld.Config) (*controld.ResolverConfig, error) {
// fetchResolverConfig is a test seam for the ControlD resolver-config API call.
var fetchResolverConfig = controld.FetchResolverConfig
// apiPreflight is the outcome of the API preflight fetch: the resolver config, the
// error if any, and whether the service was asked to stop while it ran.
type apiPreflight struct {
rc *controld.ResolverConfig
err error
stopRequested bool
}
// runAPIPreflight fetches the ControlD resolver config bounded by the service
// lifetime, and reports whether a stop was requested while it ran.
//
// The distinction matters because the caller does very different things with it: a stop
// exits quietly, while a failure self-uninstalls a deleted device, surfaces the error to
// a mobile app, and reports a failed start to the service manager.
//
// stopRequested must not be derived from the context once it has been cancelled.
// context.CancelFunc sets ctx.Err() unconditionally, so reading it after the cancel
// classifies *every* failure - a deleted device, an exhausted retry, a mobile caller
// with no stop channel - as an operator stop. Reading the stop channel directly is also
// independent of whether the context's watcher goroutine has been scheduled yet.
func runAPIPreflight(stopCh <-chan struct{}, cfg *ctrld.Config) apiPreflight {
rc, err := fetchCDConfigBoundedBy(stopCh, cfg)
return apiPreflight{rc: rc, err: err, stopRequested: stopRequested(stopCh)}
}
// permanentAPIRejection reports whether err is the API refusing this request in a way
// that a restart cannot change, and returns the rejection when it is.
//
// The type alone does not answer this. controld builds an *ErrorResponse for *any*
// non-200 whose body decodes, so a 502 from a load balancer and a 404 for a deleted
// device arrive as the same Go type. Treating both as permanent would let a few minutes
// of API trouble stop ctrld on every host with no service-manager retry behind it, which
// is strictly worse than the abnormal exit it replaced.
//
// So the HTTP status decides, and only a client-error status counts:
//
// - 4xx: the API examined this request and refused it - a deleted device, a revoked
// token, a malformed UID. The same request will be refused again.
// - 408 and 429 are the exceptions: they are the API asking for another attempt later.
// - 5xx, or no recorded status, says nothing about this configuration. Retry.
func permanentAPIRejection(err error) (*controld.ErrorResponse, bool) {
var uer *controld.ErrorResponse
if !errors.As(err, &uer) {
return nil, false
}
switch uer.StatusCode {
case http.StatusRequestTimeout, http.StatusTooManyRequests:
return nil, false
}
if uer.StatusCode < 400 || uer.StatusCode >= 500 {
return nil, false
}
return uer, true
}
// apiFailureCode maps a bootstrap preflight error to its provisioning code.
// A deleted device gets its own code because it triggers self-uninstall;
// other permanent rejections are generic; anything else counts as
// reachability trouble worth retrying.
func apiFailureCode(err error) (provisionFailureCode, bool) {
if err == nil {
return "", false
}
var uer *controld.ErrorResponse
if errors.As(err, &uer) && uer.ErrorField.Code == controld.InvalidConfigCode {
return provisionCodeAPIDeviceInvalid, true
}
if _, ok := permanentAPIRejection(err); ok {
return provisionCodeAPIRejected, true
}
return provisionCodeAPIUnreachable, true
}
// apiRejectionSummary reports the HTTP status only. The API's raw error body
// can echo back the value the caller sent, so it stays out of the artifact.
func apiRejectionSummary(statusCode int) string {
return fmt.Sprintf("ControlD API rejected this configuration (HTTP status %d)", statusCode)
}
// provisionSecrets lists every secret-bearing value to strip from provisioning
// artifacts, including both parts of a composite "<uid>/<clientID>" --cd
// value, which the API may echo back separately.
func provisionSecrets() []string {
uid, clientID := controld.ParseRawUID(cdUID)
return []string{cdUID, cdOrg, uid, clientID}
}
// uninstallInvalidCdUIDFn is a var so tests can observe the self-uninstall
// without driving the OS service manager.
var uninstallInvalidCdUIDFn = uninstallInvalidCdUID
// handleAPIPreflightFailure reports a failed resolver-config fetch. A deleted
// device self-uninstalls; it and any other permanent rejection return cleanly
// so a config problem cannot burn the service manager's restart budget (on
// Windows those restarts are what bring enforcement back after a real crash).
// Anything else exits nonzero through failProvision so the manager retries.
func handleAPIPreflightFailure(p *prog, err error, notify func()) {
cdLogger := mainLog.Load().With().Str("mode", "cd").Logger()
code, _ := apiFailureCode(err)
var uer *controld.ErrorResponse
if errors.As(err, &uer) && uer.ErrorField.Code == controld.InvalidConfigCode {
r := newProvisionResult(code, apiRejectionSummary(uer.StatusCode), nil, provisionSecrets()...)
if werr := writeProvisionResult(r); werr != nil {
cdLogger.Warn().Err(werr).Msg("could not persist provision result")
}
_ = uninstallInvalidCdUIDFn(p, cdLogger, false)
cdLogger.Error().Err(err).Int("status", uer.StatusCode).Msg("failed to fetch resolver config, the device no longer exists")
cdLogger.Error().Msg(r.failureLine())
notify()
return
}
if rejection, ok := permanentAPIRejection(err); ok {
r := newProvisionResult(code, apiRejectionSummary(rejection.StatusCode), nil, provisionSecrets()...)
if werr := writeProvisionResult(r); werr != nil {
cdLogger.Warn().Err(werr).Msg("could not persist provision result")
}
cdLogger.Error().Err(err).Int("status", rejection.StatusCode).Msg("failed to fetch resolver config, the API rejected this configuration")
cdLogger.Error().Msg(r.failureLine())
notify()
return
}
cdLogger.Error().Err(err).Msg("failed to fetch resolver config")
failProvision(newProvisionResult(code, fmt.Sprintf("failed to fetch resolver config: %v", err), nil, provisionSecrets()...), notify)
}
// processCDFlagsFn is the API fetch, indirected so the lifetime binding around it can be
// tested without reaching the network.
var processCDFlagsFn = processCDFlags
// fetchCDConfigBoundedBy runs the API fetch bounded by stopCh, so a fetch that cannot
// reach the API stops when the service is asked to stop instead of working on behalf of a
// service the OS already considers stopped. The derived context is always cancelled, which
// releases the goroutine watching stopCh.
func fetchCDConfigBoundedBy(stopCh <-chan struct{}, cfg *ctrld.Config) (*controld.ResolverConfig, error) {
ctx, cancel := contextFromStopCh(stopCh)
defer cancel()
return processCDFlagsFn(ctx, cfg)
}
// fetchCDConfigBoundedByLifetime is the reload path's fetch. Reload binds the same stop
// primitives as startup - it used to wire them up itself, where a dropped cancel or the
// wrong channel would have failed nothing.
func (p *prog) fetchCDConfigBoundedByLifetime(cfg *ctrld.Config) (*controld.ResolverConfig, error) {
return fetchCDConfigBoundedBy(p.stopCh, cfg)
}
// stopRequested reports whether stopCh has been closed. A nil channel - mobile passes
// none - blocks forever, so the default case is taken and it reads as "no stop".
func stopRequested(stopCh <-chan struct{}) bool {
select {
case <-stopCh:
return true
default:
return false
}
}
// contextFromStopCh returns a context that is cancelled when stopCh closes, so
// long-running startup work stops as soon as the service is asked to stop. The
// returned cancel func must be called to release the watcher goroutine.
func contextFromStopCh(stopCh <-chan struct{}) (context.Context, context.CancelFunc) {
ctx, cancel := context.WithCancel(context.Background())
if stopCh == nil {
return ctx, cancel
}
go func() {
select {
case <-stopCh:
cancel()
case <-ctx.Done():
}
}()
return ctx, cancel
}
// processCDFlags fetches the ControlD configuration for cdUID and applies it to cfg.
//
// ctx bounds the bootstrap-DNS retry loop below. That loop retries indefinitely by
// design (a device with no network yet must eventually come up), so it must be
// cancellable: otherwise a stop request during preflight is ignored and the process
// keeps retrying after the service reports itself stopped.
func processCDFlags(ctx context.Context, cfg *ctrld.Config) (*controld.ResolverConfig, error) {
logger := mainLog.Load().With().Str("mode", "cd").Logger()
logger.Info().Msgf("fetching Controld D configuration from API: %s", cdUID)
bo := backoff.NewBackoff("processCDFlags", logf, 30*time.Second)
bo.LogLongerThan = 30 * time.Second
ctx := context.Background()
resolverConfig, err := controld.FetchResolverConfig(cdUID, rootCmd.Version, cdDev)
if ctx == nil {
ctx = context.Background()
}
req := &controld.ResolverConfigRequest{
RawUID: cdUID,
Version: rootCmd.Version,
Metadata: ctrld.SystemMetadataRuntime(ctx),
}
resolverConfig, err := fetchResolverConfig(ctx, req, cdDev)
for {
if ctxErr := ctx.Err(); ctxErr != nil {
logger.Debug().Msg("resolver config fetch cancelled")
return nil, ctxErr
}
if errUrlNetworkError(err) {
bo.BackOff(ctx, err)
if ctxErr := ctx.Err(); ctxErr != nil {
logger.Debug().Msg("resolver config fetch cancelled during backoff")
return nil, ctxErr
}
logger.Warn().Msg("could not fetch resolver using bootstrap DNS, retrying...")
resolverConfig, err = controld.FetchResolverConfig(cdUID, rootCmd.Version, cdDev)
resolverConfig, err = fetchResolverConfig(ctx, req, cdDev)
continue
}
break
@@ -680,7 +925,10 @@ func processCDFlags(cfg *ctrld.Config) (*controld.ResolverConfig, error) {
return resolverConfig, nil
}
}
mainLog.Load().Warn().Err(err).Msg("disregarding invalid custom config")
// cfgErr, not err: err is the resolver-config fetch error from above, which is
// nil on every path that reaches here, so logging it said nothing about why the
// custom config was rejected.
mainLog.Load().Warn().Err(cfgErr).Msg("disregarding invalid custom config")
}
bootstrapIP := func(endpoint string) string {
@@ -780,7 +1028,8 @@ func processListenFlag() {
})
}
func processLogAndCacheFlags() {
// processLogAndCacheFlags processes log and cache related flags
func processLogAndCacheFlags(v *viper.Viper, cfg *ctrld.Config) {
if logPath != "" {
cfg.Service.LogPath = logPath
}
@@ -796,7 +1045,7 @@ func processLogAndCacheFlags() {
}
func netInterface(ifaceName string) (*net.Interface, error) {
if ifaceName == "auto" {
if ifaceName == autoIface {
ifaceName = defaultIfaceName()
}
var iface *net.Interface
@@ -1082,23 +1331,7 @@ func uninstall(p *prog, s service.Service) {
}
// restore static DNS settings or DHCP
p.resetDNS(false, true)
// Iterate over all physical interfaces and restore DNS if a saved static config exists.
withEachPhysicalInterfaces(p.runningIface, "restore static DNS", func(i *net.Interface) error {
file := savedStaticDnsSettingsFilePath(i)
if _, err := os.Stat(file); err == nil {
if err := restoreDNS(i); err != nil {
mainLog.Load().Error().Err(err).Msgf("Could not restore static DNS on interface %s", i.Name)
} else {
mainLog.Load().Debug().Msgf("Restored static DNS on interface %s successfully", i.Name)
err = os.Remove(file)
if err != nil {
mainLog.Load().Debug().Err(err).Msgf("Could not remove saved static DNS file for interface %s", i.Name)
}
}
}
return nil
})
restoreSavedStaticDNS(p.runningIface, true)
if router.Name() != "" {
mainLog.Load().Debug().Msg("Router cleanup")
@@ -1111,6 +1344,26 @@ func uninstall(p *prog, s service.Service) {
}
}
// restoreSavedStaticDNS restores DNS from saved static config files on physical interfaces.
func restoreSavedStaticDNS(excludeIfaceName string, removeSaved bool) {
withEachPhysicalInterfaces(excludeIfaceName, "restore static DNS", func(i *net.Interface) error {
file := savedStaticDnsSettingsFilePath(i)
if _, err := os.Stat(file); err == nil {
if err := restoreDNS(i); err != nil {
mainLog.Load().Error().Err(err).Msgf("Could not restore static DNS on interface %s", i.Name)
} else {
mainLog.Load().Debug().Msgf("Restored static DNS on interface %s successfully", i.Name)
if removeSaved {
if err := os.Remove(file); err != nil {
mainLog.Load().Debug().Err(err).Msgf("Could not remove saved static DNS file for interface %s", i.Name)
}
}
}
}
return nil
})
}
func validateConfig(cfg *ctrld.Config) error {
if err := ctrld.ValidateConfig(validator.New(), cfg); err != nil {
var ve validator.ValidationErrors
@@ -1213,10 +1466,131 @@ func updateListenerConfig(cfg *ctrld.Config, notifyToLogServerFunc func()) bool
return updated
}
// tryUpdateListenerConfigIntercept handles listener binding for dns-intercept mode on macOS.
// In intercept mode, pf redirects all outbound port-53 traffic to ctrld's listener,
// so ctrld can safely listen on a non-standard port if port 53 is unavailable
// (e.g., mDNSResponder holds *:53).
//
// Flow:
// 1. If config has explicit (non-default) IP:port → use exactly that, no fallback
// 2. Otherwise → try 127.0.0.1:53, then 127.0.0.1:5354, then fatal
func tryUpdateListenerConfigIntercept(cfg *ctrld.Config, notifyFunc func(), fatal bool) (updated, ok bool) {
ok = true
lc := cfg.FirstListener()
if lc == nil {
return false, true
}
hasExplicitConfig := isExplicitInterceptListener(lc.IP, lc.Port)
if !hasExplicitConfig {
// Set defaults for intercept mode
if lc.IP == "" || lc.IP == "0.0.0.0" {
lc.IP = "127.0.0.1"
updated = true
}
if lc.Port == 0 {
lc.Port = 53
updated = true
}
}
// bindAttempts feeds the provisioning result detail. newProvisionResult
// caps it, so it grows freely here.
var bindAttempts []provisionBindAttempt
recordBindAttempt := func(addr, proto string, err error) {
if err != nil {
bindAttempts = append(bindAttempts, provisionBindAttempt{Addr: addr, Proto: proto, OSError: err.Error()})
}
}
tryListen := func(ip string, port int) bool {
addr := net.JoinHostPort(ip, strconv.Itoa(port))
udpLn, udpErr := net.ListenPacket("udp", addr)
if udpLn != nil {
udpLn.Close()
}
recordBindAttempt(addr, "udp", udpErr)
tcpLn, tcpErr := net.Listen("tcp", addr)
if tcpLn != nil {
tcpLn.Close()
}
recordBindAttempt(addr, "tcp", tcpErr)
return udpErr == nil && tcpErr == nil
}
addr := net.JoinHostPort(lc.IP, strconv.Itoa(lc.Port))
if tryListen(lc.IP, lc.Port) {
mainLog.Load().Debug().Msgf("DNS intercept: listener available at %s", addr)
return updated, true
}
mainLog.Load().Info().Msgf("DNS intercept: cannot bind %s", addr)
if hasExplicitConfig {
// User specified explicit address — don't guess, just fail
if fatal {
msg := fmt.Sprintf("DNS intercept: cannot listen on configured address %s", addr)
mainLog.Load().Error().Msg(msg)
failProvision(newProvisionResult(provisionCodeListenerAddrUnavail, msg, bindAttempts, provisionSecrets()...), notifyFunc)
return updated, false
}
return updated, false
}
// Fallback: try port 5354 (mDNSResponder likely holds *:53)
if tryListen("127.0.0.1", 5354) {
mainLog.Load().Info().Msg("DNS intercept: port 53 unavailable (likely mDNSResponder), using 127.0.0.1:5354")
lc.IP = "127.0.0.1"
lc.Port = 5354
return true, true
}
if fatal {
const msg = "DNS intercept: cannot bind 127.0.0.1:53 or 127.0.0.1:5354"
mainLog.Load().Error().Msg(msg)
failProvision(newProvisionResult(provisionCodeListenerBindFailed, msg, bindAttempts, provisionSecrets()...), notifyFunc)
return updated, false
}
return updated, false
}
func isExplicitInterceptListener(ip string, port int) bool {
if ip == "" || ip == "0.0.0.0" || port == 0 {
return false
}
// 127.0.0.1:53 is the default macOS DNS-intercept listener. It can appear
// in generated/custom Control D configs, but it should still be allowed to
// fall back to 127.0.0.1:5354 when mDNSResponder already owns port 53.
return !(ip == "127.0.0.1" && port == 53)
}
// listenerInterceptMode resolves the mode that selects the listener binding
// strategy. An explicit "off" is final here, the same as in setDNS. A fallback
// to the config value would select the intercept strategy from a stale
// persisted mode on the first start after a revert to standard mode.
func listenerInterceptMode(cfg *ctrld.Config) string {
if interceptMode == "" {
return cfg.Service.InterceptMode
}
return interceptMode
}
// tryUpdateListenerConfig tries updating listener config with a working one.
// If fatal is true, and there's listen address conflicted, the function do
// fatal error.
func tryUpdateListenerConfig(cfg *ctrld.Config, infoLogger *zerolog.Logger, notifyFunc func(), fatal bool) (updated, ok bool) {
// In intercept mode (macOS), pf redirects all port-53 traffic to ctrld's listener,
// so ctrld can safely listen on a non-standard port. Use a simple two-attempt flow:
// 1. If config has explicit non-default IP:port, use exactly that
// 2. Otherwise: try 127.0.0.1:53, then 127.0.0.1:5354, then fatal
// This bypasses the full cd-mode listener probing loop entirely.
// dnsIntercept bool is derived later in prog.run(), but we need to know
// the intercept mode here to select the right listener probing strategy.
im := listenerInterceptMode(cfg)
if (im == "dns" || im == "hard") && runtime.GOOS == "darwin" {
return tryUpdateListenerConfigIntercept(cfg, notifyFunc, fatal)
}
ok = true
lcc := make(map[string]*listenerConfigCheck)
cdMode := cdUID != ""
@@ -1284,6 +1658,15 @@ func tryUpdateListenerConfig(cfg *ctrld.Config, infoLogger *zerolog.Logger, noti
_ = closer.Close()
}
}()
// bindAttempts feeds the provisioning result detail. newProvisionResult
// caps it, so it grows freely here.
var bindAttempts []provisionBindAttempt
recordBindAttempt := func(addr, proto string, err error) {
if err != nil {
bindAttempts = append(bindAttempts, provisionBindAttempt{Addr: addr, Proto: proto, OSError: err.Error()})
}
}
// tryListen attempts to listen on given udp and tcp address.
// Created listeners will be kept in listeners slice above, and close
// before function finished.
@@ -1292,16 +1675,21 @@ func tryUpdateListenerConfig(cfg *ctrld.Config, infoLogger *zerolog.Logger, noti
if udpLn != nil {
closers = append(closers, udpLn)
}
recordBindAttempt(addr, "udp", udpErr)
tcpLn, tcpErr := net.Listen("tcp", addr)
if tcpLn != nil {
closers = append(closers, tcpLn)
}
recordBindAttempt(addr, "tcp", tcpErr)
return errors.Join(udpErr, tcpErr)
}
listenerMsg := func(listenerNum int, format string, v ...any) string {
return fmt.Sprintf("listener.%d %s", listenerNum, fmt.Sprintf(format, v...))
}
logMsg := func(e *zerolog.Event, listenerNum int, format string, v ...any) {
e.MsgFunc(func() string {
return fmt.Sprintf("listener.%d %s", listenerNum, fmt.Sprintf(format, v...))
return listenerMsg(listenerNum, format, v...)
})
}
@@ -1353,8 +1741,10 @@ func tryUpdateListenerConfig(cfg *ctrld.Config, infoLogger *zerolog.Logger, noti
maxAttempts := 10
for {
if attempts == maxAttempts {
notifyFunc()
logMsg(mainLog.Load().Fatal(), n, "could not find available listen ip and port")
logMsg(mainLog.Load().Error(), n, "could not find available listen ip and port")
msg := listenerMsg(n, "could not find available listen ip and port")
failProvision(newProvisionResult(provisionCodeListenerBindFailed, msg, bindAttempts, provisionSecrets()...), notifyFunc)
return updated, false
}
addr := net.JoinHostPort(listener.IP, strconv.Itoa(listener.Port))
err := tryListen(addr)
@@ -1366,8 +1756,10 @@ func tryUpdateListenerConfig(cfg *ctrld.Config, infoLogger *zerolog.Logger, noti
if !check.IP && !check.Port {
if fatal {
notifyFunc()
logMsg(mainLog.Load().Fatal(), n, "failed to listen: %v", err)
logMsg(mainLog.Load().Error(), n, "failed to listen: %v", err)
msg := listenerMsg(n, "failed to listen: %v", err)
failProvision(newProvisionResult(provisionCodeListenerAddrUnavail, msg, bindAttempts, provisionSecrets()...), notifyFunc)
return updated, false
}
ok = false
break
@@ -1434,8 +1826,11 @@ func tryUpdateListenerConfig(cfg *ctrld.Config, infoLogger *zerolog.Logger, noti
}
if listener.IP == oldIP && listener.Port == oldPort {
if fatal {
notifyFunc()
logMsg(mainLog.Load().Fatal(), n, "could not listen on %s: %v", net.JoinHostPort(listener.IP, strconv.Itoa(listener.Port)), err)
triedAddr := net.JoinHostPort(listener.IP, strconv.Itoa(listener.Port))
logMsg(mainLog.Load().Error(), n, "could not listen on %s: %v", triedAddr, err)
msg := listenerMsg(n, "could not listen on %s: %v", triedAddr, err)
failProvision(newProvisionResult(provisionCodeListenerBindFailed, msg, bindAttempts, provisionSecrets()...), notifyFunc)
return updated, false
}
ok = false
break
@@ -1473,8 +1868,10 @@ func tryUpdateListenerConfig(cfg *ctrld.Config, infoLogger *zerolog.Logger, noti
}
}
if !found {
notifyFunc()
logMsg(mainLog.Load().Fatal(), n, "could not use %q as DNS nameserver with systemd resolved", listener.IP)
logMsg(mainLog.Load().Error(), n, "could not use %q as DNS nameserver with systemd resolved", listener.IP)
msg := listenerMsg(n, "could not use %q as DNS nameserver with systemd resolved", listener.IP)
failProvision(newProvisionResult(provisionCodeListenerAddrUnavail, msg, bindAttempts, provisionSecrets()...), notifyFunc)
return updated, false
}
}
}
@@ -1515,15 +1912,30 @@ func cdUIDFromProvToken() string {
if customHostname != "" && !validHostname(customHostname) {
mainLog.Load().Fatal().Msgf("invalid custom hostname: %q", customHostname)
}
req := &controld.UtilityOrgRequest{ProvToken: cdOrg, Hostname: customHostname}
req := &controld.UtilityOrgRequest{
ProvToken: cdOrg,
Hostname: customHostname,
Metadata: ctrld.SystemMetadata(context.Background()),
}
// Process provision token if provided.
resolverConfig, err := controld.FetchResolverUID(req, rootCmd.Version, cdDev)
resolverConfig, err := fetchResolverUIDFn(context.Background(), req, rootCmd.Version, cdDev)
if err != nil {
mainLog.Load().Fatal().Err(err).Msgf("failed to fetch resolver uid with provision token: %s", cdOrg)
// The token exchange is the first API call of an org/MDM install, so
// its failure must carry a code like every other bootstrap failure.
code, _ := apiFailureCode(err)
mainLog.Load().Error().Msgf("failed to fetch resolver uid with provision token: %s: %s",
redactToken(cdOrg), redactSecrets(err.Error(), provisionSecrets()...))
failProvision(newProvisionResult(code, fmt.Sprintf("provision token exchange failed: %v", err), nil, provisionSecrets()...), nil)
return ""
}
return resolverConfig.UID
}
// fetchResolverUIDFn is a var so tests can drive token-exchange failures
// without reaching the network.
var fetchResolverUIDFn = controld.FetchResolverUID
// removeOrgFlagsFromArgs removes organization flags from command line arguments.
// The flags are:
//
@@ -1698,6 +2110,9 @@ var errInvalidDeactivationPin = errors.New("deactivation pin is invalid")
// errRequiredDeactivationPin indicates that the deactivation pin is required but not provided by users.
var errRequiredDeactivationPin = errors.New("deactivation pin is required to stop or uninstall the service")
// errTooManyDeactivationPin represents an error indicating excessive deactivation PIN request attempts.
var errTooManyDeactivationPin = errors.New("too many request attempts")
// checkDeactivationPin validates if the deactivation pin matches one in ControlD config.
func checkDeactivationPin(s service.Service, stopCh chan struct{}) error {
mainLog.Load().Debug().Msg("Checking deactivation pin")
@@ -1726,6 +2141,9 @@ func checkDeactivationPin(s service.Service, stopCh chan struct{}) error {
case http.StatusBadRequest:
mainLog.Load().Error().Msg(errRequiredDeactivationPin.Error())
return errRequiredDeactivationPin // pin is required
case http.StatusTooManyRequests:
mainLog.Load().Error().Msg(errTooManyDeactivationPin.Error())
return errTooManyDeactivationPin
case http.StatusOK:
return nil // valid pin
case http.StatusNotFound:
@@ -1738,7 +2156,9 @@ func checkDeactivationPin(s service.Service, stopCh chan struct{}) error {
// isCheckDeactivationPinErr reports whether there is an error during check deactivation pin process.
func isCheckDeactivationPinErr(err error) bool {
return errors.Is(err, errInvalidDeactivationPin) || errors.Is(err, errRequiredDeactivationPin)
return errors.Is(err, errInvalidDeactivationPin) ||
errors.Is(err, errRequiredDeactivationPin) ||
errors.Is(err, errTooManyDeactivationPin)
}
// ensureUninstall ensures that s.Uninstall will remove ctrld service from system completely.
@@ -1856,7 +2276,14 @@ func runningIface(s service.Service) *ifaceResponse {
// doValidateCdRemoteConfig fetches and validates custom config for cdUID.
func doValidateCdRemoteConfig(cdUID string, fatal bool) error {
rc, err := controld.FetchResolverConfig(cdUID, rootCmd.Version, cdDev)
// Username is only sent during initial provisioning (cdUIDFromProvToken).
// All subsequent calls use lightweight metadata to avoid EDR triggers.
req := &controld.ResolverConfigRequest{
RawUID: cdUID,
Version: rootCmd.Version,
Metadata: ctrld.SystemMetadataRuntime(context.Background()),
}
rc, err := controld.FetchResolverConfig(context.Background(), req, cdDev)
if err != nil {
logger := mainLog.Load().Fatal()
if !fatal {
@@ -1884,17 +2311,25 @@ func doValidateCdRemoteConfig(cdUID string, fatal bool) error {
} else {
if errors.As(cfgErr, &viper.ConfigParseError{}) {
if configStr, _ := base64.StdEncoding.DecodeString(rc.Ctrld.CustomConfig); len(configStr) > 0 {
tmpDir := os.TempDir()
tmpConfFile := filepath.Join(tmpDir, "ctrld.toml")
errorLogged := false
// Write remote config to a temporary file to get details error.
if we := os.WriteFile(tmpConfFile, configStr, 0600); we == nil {
// Write remote config to a uniquely named temporary file to get detailed error.
if tmpFile, tmpErr := os.CreateTemp("", "ctrld-*.toml"); tmpErr == nil {
tmpConfFile := tmpFile.Name()
if _, err := tmpFile.Write(configStr); err != nil {
mainLog.Load().Error().Err(err).Msg("failed to write temporary config file")
}
if err := tmpFile.Close(); err != nil {
mainLog.Load().Error().Err(err).Msg("failed to save temporary config file")
}
if de := decoderErrorFromTomlFile(tmpConfFile); de != nil {
row, col := de.Position()
mainLog.Load().Error().Msgf("failed to parse custom config at line: %d, column: %d, error: %s", row, col, de.Error())
errorLogged = true
}
_ = os.Remove(tmpConfFile)
if err := os.Remove(tmpConfFile); err != nil {
mainLog.Load().Error().Err(err).Msg("failed to remove temporary config file")
}
}
// If we could not log details error, emit what we have already got.
if !errorLogged {
@@ -1912,6 +2347,23 @@ func doValidateCdRemoteConfig(cdUID string, fatal bool) error {
return nil
}
// ensureRunningIfaceForInvalidUninstall populates p.runningIface before the
// invalid-device self-uninstall resets DNS. This path can run early during
// service startup (e.g. right after a reboot) before the running interface is
// otherwise known. resetDNS, via resetDNSForRunningIface, silently skips DNS
// restoration when p.runningIface is empty, which would leave the OS pointed at
// ctrld's local listener after the service is removed. See issue-556.
func ensureRunningIfaceForInvalidUninstall(p *prog, s service.Service) {
if iface == "" {
iface = autoIface
}
p.preRun()
if ir := runningIface(s); ir != nil {
p.runningIface = ir.Name
p.requiredMultiNICsConfig = ir.All
}
}
// uninstallInvalidCdUID performs self-uninstallation because the ControlD device does not exist.
func uninstallInvalidCdUID(p *prog, logger zerolog.Logger, doStop bool) bool {
s, err := newService(p, svcConfig)
@@ -1919,8 +2371,13 @@ func uninstallInvalidCdUID(p *prog, logger zerolog.Logger, doStop bool) bool {
logger.Warn().Err(err).Msg("failed to create new service")
return false
}
ensureRunningIfaceForInvalidUninstall(p, s)
// restore static DNS settings or DHCP
p.resetDNS(false, true)
// The invalid-device path may run early during service startup before runningIface
// is known. Restore every saved static DNS file so uninstalling does not leave the
// OS pointed at ctrld's local listener after the service is removed.
restoreSavedStaticDNS("", true)
tasks := []task{{s.Uninstall, true, "Uninstall"}}
if doTasks(tasks) {
@@ -1932,3 +2389,12 @@ func uninstallInvalidCdUID(p *prog, logger zerolog.Logger, doStop bool) bool {
}
return false
}
// redactToken returns the first 4 characters of a token followed by ***,
// or just *** if the token is 4 characters or shorter.
func redactToken(s string) string {
if len(s) <= 4 {
return "***"
}
return s[:4] + "***"
}
+116
View File
@@ -0,0 +1,116 @@
package cli
import (
"testing"
"github.com/Control-D-Inc/ctrld"
)
func TestIsExplicitInterceptListener(t *testing.T) {
tests := []struct {
name string
ip string
port int
want bool
}{
{name: "empty", ip: "", port: 0, want: false},
{name: "wildcard", ip: "0.0.0.0", port: 53, want: false},
{name: "zero port", ip: "127.0.0.1", port: 0, want: false},
{name: "default intercept listener", ip: "127.0.0.1", port: 53, want: false},
{name: "fallback port explicit", ip: "127.0.0.1", port: 5354, want: true},
{name: "custom loopback explicit", ip: "127.0.0.2", port: 53, want: true},
{name: "custom address explicit", ip: "192.0.2.10", port: 53, want: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := isExplicitInterceptListener(tt.ip, tt.port); got != tt.want {
t.Fatalf("isExplicitInterceptListener(%q, %d) = %v, want %v", tt.ip, tt.port, got, tt.want)
}
})
}
}
// TestPreserveBoundListeners is a regression test for #551: on reload, the on-disk
// generated config still declares 127.0.0.1:53, but the running listener has fallen back
// to 127.0.0.1:5354. preserveBoundListeners must keep the in-memory config on the actual
// bound port so pf rdr rules and probes do not target the dead default port.
func TestPreserveBoundListeners(t *testing.T) {
// cur = actual running listener (fell back to 5354); newCfg = freshly read from disk (53).
cur := map[string]*ctrld.ListenerConfig{"0": {IP: "127.0.0.1", Port: 5354}}
newListeners := map[string]*ctrld.ListenerConfig{"0": {IP: "127.0.0.1", Port: 53}}
preserveBoundListeners(newListeners, cur)
if got := newListeners["0"].Port; got != 5354 {
t.Errorf("listener port after reload = %d, want 5354 (actual bound port)", got)
}
if got := newListeners["0"].IP; got != "127.0.0.1" {
t.Errorf("listener IP after reload = %q, want 127.0.0.1", got)
}
}
// TestPreserveBoundListeners_NoChange verifies that when the on-disk config matches the
// running listener, the config is left untouched (a legitimate reload with the same port).
func TestPreserveBoundListeners_NoChange(t *testing.T) {
cur := map[string]*ctrld.ListenerConfig{"0": {IP: "127.0.0.1", Port: 5354}}
newListeners := map[string]*ctrld.ListenerConfig{"0": {IP: "127.0.0.1", Port: 5354}}
preserveBoundListeners(newListeners, cur)
if got := newListeners["0"].Port; got != 5354 {
t.Errorf("listener port = %d, want 5354", got)
}
}
// TestPreserveBoundListeners_MissingCurrent verifies that a listener present on disk but not
// in the current running set (e.g. newly added) is left as configured.
func TestPreserveBoundListeners_MissingCurrent(t *testing.T) {
cur := map[string]*ctrld.ListenerConfig{"0": {IP: "127.0.0.1", Port: 5354}}
newListeners := map[string]*ctrld.ListenerConfig{
"0": {IP: "127.0.0.1", Port: 53},
"1": {IP: "127.0.0.1", Port: 5355},
}
preserveBoundListeners(newListeners, cur)
if got := newListeners["0"].Port; got != 5354 {
t.Errorf("listener 0 port = %d, want 5354 (preserved)", got)
}
if got := newListeners["1"].Port; got != 5355 {
t.Errorf("listener 1 port = %d, want 5355 (unchanged, no current binding)", got)
}
}
// TestPreserveBoundListeners_ExplicitChangeNotMasked verifies that an explicit, non-default
// listener in the reloaded config is applied rather than reverted to the old bound listener.
// Reverting an explicit change would make the control-server reload comparison return 200
// instead of 201, silently dropping the new listener. Regression guard for #551 review.
func TestPreserveBoundListeners_ExplicitChangeNotMasked(t *testing.T) {
// Running listener fell back to 5354; user reloads with an explicit new listener.
cur := map[string]*ctrld.ListenerConfig{"0": {IP: "127.0.0.1", Port: 5354}}
newListeners := map[string]*ctrld.ListenerConfig{"0": {IP: "127.0.0.2", Port: 5399}}
preserveBoundListeners(newListeners, cur)
if got := newListeners["0"].IP; got != "127.0.0.2" {
t.Errorf("explicit listener IP = %q, want 127.0.0.2 (not reverted)", got)
}
if got := newListeners["0"].Port; got != 5399 {
t.Errorf("explicit listener port = %d, want 5399 (not reverted)", got)
}
}
// TestPreserveBoundListeners_ExplicitDefaultPreserved verifies that the default
// 127.0.0.1:53 listener remains fallback-eligible: when it diverges from the running
// fallback port it is still preserved (isExplicitInterceptListener treats :53 as non-explicit).
func TestPreserveBoundListeners_ExplicitDefaultPreserved(t *testing.T) {
cur := map[string]*ctrld.ListenerConfig{"0": {IP: "127.0.0.1", Port: 5354}}
newListeners := map[string]*ctrld.ListenerConfig{"0": {IP: "127.0.0.1", Port: 53}}
preserveBoundListeners(newListeners, cur)
if got := newListeners["0"].Port; got != 5354 {
t.Errorf("default listener port = %d, want 5354 (preserved fallback)", got)
}
}
+403
View File
@@ -0,0 +1,403 @@
package cli
import (
"context"
"errors"
"fmt"
"net"
"net/http"
"net/url"
"sync/atomic"
"syscall"
"testing"
"time"
"github.com/Control-D-Inc/ctrld"
"github.com/Control-D-Inc/ctrld/internal/controld"
)
func TestContextFromStopCh(t *testing.T) {
t.Run("cancels when stopCh closes", func(t *testing.T) {
stopCh := make(chan struct{})
ctx, cancel := contextFromStopCh(stopCh)
defer cancel()
if ctx.Err() != nil {
t.Fatalf("context cancelled before the stop request: %v", ctx.Err())
}
close(stopCh)
select {
case <-ctx.Done():
case <-time.After(5 * time.Second):
t.Fatal("context was not cancelled after stopCh closed")
}
if !errors.Is(ctx.Err(), context.Canceled) {
t.Errorf("ctx.Err() = %v, want %v", ctx.Err(), context.Canceled)
}
})
t.Run("cancel releases the watcher", func(t *testing.T) {
// stopCh is never closed: cancel() must still end the goroutine watching it.
ctx, cancel := contextFromStopCh(make(chan struct{}))
cancel()
select {
case <-ctx.Done():
case <-time.After(5 * time.Second):
t.Fatal("context was not cancelled by cancel()")
}
})
t.Run("nil stopCh is usable", func(t *testing.T) {
// Mobile callers have no stop channel; preflight must still run.
ctx, cancel := contextFromStopCh(nil)
defer cancel()
if ctx.Err() != nil {
t.Fatalf("context cancelled immediately: %v", ctx.Err())
}
})
}
// retryableNetworkErr is the shape processCDFlags treats as "retry with bootstrap
// DNS": a url.Error wrapping a network failure.
func retryableNetworkErr() error {
return &url.Error{
Op: "Post",
URL: "https://api.controld.com/utility",
Err: &net.OpError{Op: "dial", Net: "tcp", Err: syscall.ECONNREFUSED},
}
}
func TestProcessCDFlagsStopsWhenCancelled(t *testing.T) {
oldFetch := fetchResolverConfig
oldUID := cdUID
t.Cleanup(func() {
fetchResolverConfig = oldFetch
cdUID = oldUID
})
cdUID = "testuid"
var calls atomic.Int64
fetchResolverConfig = func(ctx context.Context, req *controld.ResolverConfigRequest, dev bool) (*controld.ResolverConfig, error) {
calls.Add(1)
return nil, retryableNetworkErr()
}
// A stop request arriving while the API is unreachable. Before this was
// cancellable, the retry loop kept running after the service reported itself
// stopped, which is what kept the incident's process alive and enforcing.
stopCh := make(chan struct{})
ctx, cancel := contextFromStopCh(stopCh)
defer cancel()
done := make(chan error, 1)
go func() {
cfg := ctrld.Config{}
_, err := processCDFlags(ctx, &cfg)
done <- err
}()
// Let it fail at least once and settle into backoff before stopping.
deadline := time.After(10 * time.Second)
for calls.Load() == 0 {
select {
case <-deadline:
t.Fatal("resolver config was never fetched")
case err := <-done:
t.Fatalf("processCDFlags returned before any fetch: %v", err)
default:
time.Sleep(5 * time.Millisecond)
}
}
close(stopCh)
select {
case err := <-done:
if !errors.Is(err, context.Canceled) {
t.Errorf("processCDFlags err = %v, want it to report %v", err, context.Canceled)
}
case <-time.After(30 * time.Second):
t.Fatal("processCDFlags did not return after the stop request")
}
}
func TestProcessCDFlagsReturnsImmediatelyWhenAlreadyCancelled(t *testing.T) {
oldFetch := fetchResolverConfig
oldUID := cdUID
t.Cleanup(func() {
fetchResolverConfig = oldFetch
cdUID = oldUID
})
cdUID = "testuid"
var calls atomic.Int64
fetchResolverConfig = func(ctx context.Context, req *controld.ResolverConfigRequest, dev bool) (*controld.ResolverConfig, error) {
calls.Add(1)
return nil, retryableNetworkErr()
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
cfg := ctrld.Config{}
_, err := processCDFlags(ctx, &cfg)
if !errors.Is(err, context.Canceled) {
t.Errorf("processCDFlags err = %v, want %v", err, context.Canceled)
}
// One attempt is made before the loop notices; it must not retry past that.
if got := calls.Load(); got > 1 {
t.Errorf("fetched %d times with a cancelled context, want at most 1", got)
}
}
// TestRunAPIPreflightClassification is the regression guard for classifying a preflight
// failure as an operator stop.
//
// runAPIPreflight cancels the context it derived from stopCh. Sampling the stop state
// from that context afterwards reports "stopped" unconditionally, because
// context.CancelFunc sets ctx.Err() whether or not anyone asked to stop. run() then
// takes the stop branch for every failure, which skips self-uninstalling a deleted
// device, skips the mobile exit callback, and tells the service manager a failed start
// was a clean exit.
func TestRunAPIPreflightClassification(t *testing.T) {
oldFetch := fetchResolverConfig
oldUID := cdUID
t.Cleanup(func() {
fetchResolverConfig = oldFetch
cdUID = oldUID
})
cdUID = "testuid"
// A deleted ControlD device: non-retryable, so preflight returns promptly.
deletedDevice := func() error {
e := &controld.ErrorResponse{}
e.ErrorField.Code = controld.InvalidConfigCode
e.ErrorField.Message = "device does not exist"
return e
}
openCh := make(chan struct{})
closedCh := make(chan struct{})
close(closedCh)
tests := []struct {
name string
stopCh <-chan struct{}
fetchErr func() error
wantStop bool
}{
{
// The P1: no stop was requested, so this must reach the failure branch.
name: "api error with no stop request",
stopCh: openCh,
fetchErr: deletedDevice,
},
{
// Mobile passes no stop channel at all, so it could never have stopped.
name: "api error with a nil stop channel",
stopCh: nil,
fetchErr: deletedDevice,
},
{
name: "stop requested during preflight",
stopCh: closedCh,
fetchErr: func() error { return retryableNetworkErr() },
wantStop: true,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
fetchResolverConfig = func(context.Context, *controld.ResolverConfigRequest, bool) (*controld.ResolverConfig, error) {
return nil, tc.fetchErr()
}
cfg := ctrld.Config{}
pf := runAPIPreflight(tc.stopCh, &cfg)
if pf.err == nil {
t.Fatal("expected preflight to fail")
}
if pf.stopRequested != tc.wantStop {
t.Errorf("stopRequested = %v, want %v", pf.stopRequested, tc.wantStop)
}
})
}
}
// TestRunAPIPreflightPreservesAPIError verifies the error reaches the caller in a form
// the failure branch can still act on: self-uninstall keys off an *ErrorResponse with
// InvalidConfigCode, and it only runs if that error is both classified as a failure and
// still unwrappable.
func TestRunAPIPreflightPreservesAPIError(t *testing.T) {
oldFetch := fetchResolverConfig
oldUID := cdUID
t.Cleanup(func() {
fetchResolverConfig = oldFetch
cdUID = oldUID
})
cdUID = "testuid"
want := &controld.ErrorResponse{}
want.ErrorField.Code = controld.InvalidConfigCode
fetchResolverConfig = func(context.Context, *controld.ResolverConfigRequest, bool) (*controld.ResolverConfig, error) {
return nil, want
}
cfg := ctrld.Config{}
pf := runAPIPreflight(make(chan struct{}), &cfg)
if pf.stopRequested {
t.Error("a device-deleted failure must not be reported as an operator stop")
}
var got *controld.ErrorResponse
if !errors.As(pf.err, &got) {
t.Fatalf("error no longer unwraps to *controld.ErrorResponse: %v", pf.err)
}
if got.ErrorField.Code != controld.InvalidConfigCode {
t.Errorf("code = %d, want %d (self-uninstall would not trigger)", got.ErrorField.Code, controld.InvalidConfigCode)
}
}
// TestPermanentAPIRejectionNarrowsToClientErrors is the regression guard for the clean
// exit added above.
//
// controld builds an *ErrorResponse for any non-200 whose body decodes, so the Go type
// says nothing about whether the API's answer will change on a retry. Keying the clean
// exit off the type alone meant a 502 from a load balancer, or an API having a bad ten
// minutes, stopped ctrld on every affected host with no service-manager retry behind it -
// worse than the abnormal exit it replaced, because a Fatal at least gets restarted.
//
// Only a client-error status may take that path.
func TestPermanentAPIRejectionNarrowsToClientErrors(t *testing.T) {
rejection := func(status, code int) error {
e := &controld.ErrorResponse{StatusCode: status}
e.ErrorField.Code = code
e.ErrorField.Message = "api said no"
return e
}
tests := []struct {
name string
err error
wantPermanent bool
}{
{
// The case the clean exit exists for: the device is gone, and every restart
// will be told the same thing.
name: "deleted device",
err: rejection(http.StatusNotFound, controld.InvalidConfigCode),
wantPermanent: true,
},
{"revoked credentials", rejection(http.StatusUnauthorized, 0), true},
{"forbidden", rejection(http.StatusForbidden, 0), true},
{"malformed request", rejection(http.StatusBadRequest, 0), true},
// Server-side trouble. These must keep the abnormal exit so the service
// manager's recovery policy retries.
{"bad gateway", rejection(http.StatusBadGateway, 0), false},
{"internal error", rejection(http.StatusInternalServerError, 0), false},
{"service unavailable", rejection(http.StatusServiceUnavailable, 0), false},
// 4xx, but both are the API asking for a later attempt rather than refusing
// this configuration.
{"request timeout", rejection(http.StatusRequestTimeout, 0), false},
{"rate limited", rejection(http.StatusTooManyRequests, 0), false},
// An *ErrorResponse built without a recorded status carries no verdict. A
// hand-constructed one, or a decode path that forgets to record the status,
// must not silently gain the clean exit.
{"no recorded status", rejection(0, controld.InvalidConfigCode), false},
// Not an API answer at all: the incident's denied socket reaches Fatal.
{"network failure", retryableNetworkErr(), false},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got, ok := permanentAPIRejection(tc.err)
if ok != tc.wantPermanent {
t.Errorf("permanentAPIRejection() = %v, want %v", ok, tc.wantPermanent)
}
if ok && got == nil {
t.Error("a permanent rejection must return the rejection for reporting")
}
})
}
// The wrapped form matters too: preflight composes the fetch error, and errors.As has
// to reach through that for either branch to be chosen correctly.
wrapped := fmt.Errorf("processCDFlags: %w", rejection(http.StatusNotFound, controld.InvalidConfigCode))
if _, ok := permanentAPIRejection(wrapped); !ok {
t.Error("a wrapped API rejection must still be recognised")
}
wrappedTransient := fmt.Errorf("processCDFlags: %w", rejection(http.StatusBadGateway, 0))
if _, ok := permanentAPIRejection(wrappedTransient); ok {
t.Error("a wrapped 502 must not be treated as a permanent rejection")
}
}
func TestStopRequested(t *testing.T) {
closedCh := make(chan struct{})
close(closedCh)
if stopRequested(nil) {
t.Error("a nil stop channel must read as no stop (mobile passes none)")
}
if stopRequested(make(chan struct{})) {
t.Error("an open stop channel must read as no stop")
}
if !stopRequested(closedCh) {
t.Error("a closed stop channel must read as a stop")
}
}
// TestReloadFetchIsBoundedByServiceLifetime covers the reload path's stop wiring.
//
// Reload fetches the ControlD config too, and it used to build the bounded context
// itself. Nothing tested that: the wrong channel, or a dropped cancel, would have left a
// reload retrying against an unreachable API after "service stopped" was logged, and no
// test would have failed. Both paths now go through one bounded fetch, so this pins it.
func TestReloadFetchIsBoundedByServiceLifetime(t *testing.T) {
original := processCDFlagsFn
t.Cleanup(func() { processCDFlagsFn = original })
t.Run("a stop request cancels the reload fetch", func(t *testing.T) {
stopCh := make(chan struct{})
close(stopCh)
var sawCancelled bool
processCDFlagsFn = func(ctx context.Context, _ *ctrld.Config) (*controld.ResolverConfig, error) {
select {
case <-ctx.Done():
sawCancelled = true
case <-time.After(2 * time.Second):
}
return nil, ctx.Err()
}
p := &prog{stopCh: stopCh}
if _, err := p.fetchCDConfigBoundedByLifetime(&ctrld.Config{}); !errors.Is(err, context.Canceled) {
t.Errorf("reload fetch err = %v, want %v", err, context.Canceled)
}
if !sawCancelled {
t.Error("the reload fetch did not observe the stop request: it is not bound to the service lifetime")
}
})
t.Run("the derived context is always released", func(t *testing.T) {
// stopCh stays open: the fetch's own cancel is what must end the watcher, or
// every reload leaks a goroutine.
var captured context.Context
processCDFlagsFn = func(ctx context.Context, _ *ctrld.Config) (*controld.ResolverConfig, error) {
captured = ctx
return nil, nil
}
p := &prog{stopCh: make(chan struct{})}
if _, err := p.fetchCDConfigBoundedByLifetime(&ctrld.Config{}); err != nil {
t.Fatalf("unexpected error: %v", err)
}
select {
case <-captured.Done():
case <-time.After(time.Second):
t.Error("the reload fetch left its context uncancelled")
}
})
}
+329
View File
@@ -0,0 +1,329 @@
package cli
import (
"context"
"fmt"
"net"
"net/http"
"path/filepath"
"runtime"
"strconv"
"strings"
"testing"
"github.com/rs/zerolog"
"github.com/Control-D-Inc/ctrld"
"github.com/Control-D-Inc/ctrld/internal/controld"
)
// TestApiFailureCode covers the preflight-error mapping: a deleted device
// gets its own code (it drives self-uninstall), other permanent rejections
// are generic, anything else is retryable reachability trouble.
func TestApiFailureCode(t *testing.T) {
rejection := func(status, code int) error {
e := &controld.ErrorResponse{StatusCode: status}
e.ErrorField.Code = code
e.ErrorField.Message = "api said no"
return e
}
tests := []struct {
name string
err error
wantCode provisionFailureCode
wantOk bool
}{
{name: "nil error", err: nil, wantCode: "", wantOk: false},
{
name: "deleted device maps to device invalid",
err: rejection(http.StatusNotFound, controld.InvalidConfigCode),
wantCode: provisionCodeAPIDeviceInvalid,
wantOk: true,
},
{
name: "revoked credentials map to rejected",
err: rejection(http.StatusUnauthorized, 0),
wantCode: provisionCodeAPIRejected,
wantOk: true,
},
{
name: "server error maps to unreachable",
err: rejection(http.StatusBadGateway, 0),
wantCode: provisionCodeAPIUnreachable,
wantOk: true,
},
{
name: "network failure maps to unreachable",
err: retryableNetworkErr(),
wantCode: provisionCodeAPIUnreachable,
wantOk: true,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
code, ok := apiFailureCode(tc.err)
if ok != tc.wantOk {
t.Fatalf("apiFailureCode() ok = %v, want %v", ok, tc.wantOk)
}
if code != tc.wantCode {
t.Errorf("apiFailureCode() code = %s, want %s", code, tc.wantCode)
}
})
}
}
func stubProvisionGlobals(t *testing.T) (exitCode *int, notified *bool) {
t.Helper()
oldCdUID, oldCdOrg := cdUID, cdOrg
oldExit, oldUninstall := provisionExit, uninstallInvalidCdUIDFn
t.Cleanup(func() {
cdUID, cdOrg = oldCdUID, oldCdOrg
provisionExit, uninstallInvalidCdUIDFn = oldExit, oldUninstall
})
overrideProvisionResultPath(t)
code := -1
provisionExit = func(c int) { code = c }
n := false
return &code, &n
}
func TestHandleAPIPreflightFailure(t *testing.T) {
deviceInvalid := func() error {
e := &controld.ErrorResponse{StatusCode: http.StatusNotFound}
e.ErrorField.Code = controld.InvalidConfigCode
e.ErrorField.Message = "device does not exist"
return e
}
rejected := func() error {
e := &controld.ErrorResponse{StatusCode: http.StatusUnauthorized}
e.ErrorField.Message = "bad token"
return e
}
t.Run("permanent rejection returns cleanly", func(t *testing.T) {
exitCode, notified := stubProvisionGlobals(t)
handleAPIPreflightFailure(&prog{}, rejected(), func() { *notified = true })
if *exitCode != -1 {
t.Errorf("provisionExit called with %d, want a clean return", *exitCode)
}
if !*notified {
t.Error("notify not called")
}
r, err := readProvisionResult()
if err != nil {
t.Fatal(err)
}
if r.Code != string(provisionCodeAPIRejected) {
t.Errorf("code = %q, want API_REJECTED", r.Code)
}
})
t.Run("deleted device self-uninstalls and returns cleanly", func(t *testing.T) {
exitCode, notified := stubProvisionGlobals(t)
uninstalled := false
uninstallInvalidCdUIDFn = func(_ *prog, _ zerolog.Logger, _ bool) bool {
uninstalled = true
return true
}
handleAPIPreflightFailure(&prog{}, deviceInvalid(), func() { *notified = true })
if *exitCode != -1 {
t.Errorf("provisionExit called with %d, want a clean return", *exitCode)
}
if !uninstalled {
t.Error("self-uninstall not attempted")
}
if !*notified {
t.Error("notify not called")
}
r, err := readProvisionResult()
if err != nil {
t.Fatal(err)
}
if r.Code != string(provisionCodeAPIDeviceInvalid) {
t.Errorf("code = %q, want API_DEVICE_INVALID", r.Code)
}
})
t.Run("unreachable exits nonzero", func(t *testing.T) {
exitCode, notified := stubProvisionGlobals(t)
handleAPIPreflightFailure(&prog{}, retryableNetworkErr(), func() { *notified = true })
if *exitCode != provisionExitCodeForCode[provisionCodeAPIUnreachable] {
t.Errorf("exit = %d, want %d", *exitCode, provisionExitCodeForCode[provisionCodeAPIUnreachable])
}
if !*notified {
t.Error("notify not called")
}
r, err := readProvisionResult()
if err != nil {
t.Fatal(err)
}
if r.Code != string(provisionCodeAPIUnreachable) {
t.Errorf("code = %q, want API_UNREACHABLE", r.Code)
}
})
t.Run("bare uid from a composite --cd value is redacted", func(t *testing.T) {
_, _ = stubProvisionGlobals(t)
cdUID = "deviceabc/clientxyz"
cdOrg = ""
err := fmt.Errorf("failed: api says deviceabc is unknown")
handleAPIPreflightFailure(&prog{}, err, func() {})
r, rerr := readProvisionResult()
if rerr != nil {
t.Fatal(rerr)
}
if strings.Contains(r.Message, "deviceabc") {
t.Errorf("bare uid leaked into message: %q", r.Message)
}
})
}
func TestCdUIDFromProvTokenFailureEmitsCode(t *testing.T) {
exitCode, _ := stubProvisionGlobals(t)
oldFetch, oldHostname := fetchResolverUIDFn, customHostname
t.Cleanup(func() { fetchResolverUIDFn, customHostname = oldFetch, oldHostname })
cdUID = ""
cdOrg = "org-secret-token-123"
customHostname = ""
rejected := &controld.ErrorResponse{StatusCode: http.StatusUnauthorized}
rejected.ErrorField.Message = "bad provision token org-secret-token-123"
fetchResolverUIDFn = func(context.Context, *controld.UtilityOrgRequest, string, bool) (*controld.ResolverConfig, error) {
return nil, rejected
}
if got := cdUIDFromProvToken(); got != "" {
t.Errorf("cdUIDFromProvToken() = %q, want empty on failure", got)
}
if *exitCode != provisionExitCodeForCode[provisionCodeAPIRejected] {
t.Errorf("exit = %d, want API_REJECTED exit %d", *exitCode, provisionExitCodeForCode[provisionCodeAPIRejected])
}
r, err := readProvisionResult()
if err != nil {
t.Fatalf("no provision result written: %v", err)
}
if r.Code != string(provisionCodeAPIRejected) {
t.Errorf("code = %q, want API_REJECTED", r.Code)
}
if strings.Contains(r.Message, cdOrg) {
t.Errorf("token leaked into result message: %q", r.Message)
}
}
// Regression test: an explicit ip:port that fails to bind used to die with a
// bare fatal log automation could not tell apart from any other crash. It
// must report a stable code through the provisioning result instead.
func TestTryUpdateListenerConfigConfiguredAddrUnavailable(t *testing.T) {
// Occupy one localhost port on both udp and tcp, and hold both for the
// whole test so ctrld's own bind attempt is guaranteed to fail.
udpConn, err := net.ListenPacket("udp", "127.0.0.1:0")
if err != nil {
t.Fatalf("could not reserve a udp port: %v", err)
}
defer udpConn.Close()
host, portStr, err := net.SplitHostPort(udpConn.LocalAddr().String())
if err != nil {
t.Fatalf("could not parse reserved address: %v", err)
}
port, err := strconv.Atoi(portStr)
if err != nil {
t.Fatalf("could not parse reserved port: %v", err)
}
tcpLn, err := net.Listen("tcp", net.JoinHostPort(host, portStr))
if err != nil {
t.Fatalf("could not reserve the same port on tcp: %v", err)
}
defer tcpLn.Close()
oldCdUID, oldCdOrg, oldNextdns, oldIntercept := cdUID, cdOrg, nextdns, interceptMode
oldPath, oldExit := provisionResultPath, provisionExit
t.Cleanup(func() {
cdUID, cdOrg, nextdns, interceptMode = oldCdUID, oldCdOrg, oldNextdns, oldIntercept
provisionResultPath, provisionExit = oldPath, oldExit
})
// Non-cd, non-nextdns mode with an explicit ip:port: no fallback checks,
// the path that used to reach the fatal exit directly.
cdUID = ""
cdOrg = ""
nextdns = ""
interceptMode = ""
tmpDir := t.TempDir()
provisionResultPath = func() string { return filepath.Join(tmpDir, "provision_result.json") }
var exitCode int
var exited bool
provisionExit = func(code int) { exitCode = code; exited = true }
cfg := &ctrld.Config{
Listener: map[string]*ctrld.ListenerConfig{
"0": {IP: host, Port: port},
},
}
notified := false
_, ok := tryUpdateListenerConfig(cfg, nil, func() { notified = true }, true)
if ok {
t.Error("tryUpdateListenerConfig ok = true, want false")
}
if !notified {
t.Error("expected notifyFunc to run before the recorded exit")
}
if !exited {
t.Fatal("expected provisionExit to be called")
}
if exitCode != 42 {
t.Errorf("exit code = %d, want 42 (LISTENER_CONFIGURED_ADDR_UNAVAILABLE)", exitCode)
}
result, err := readProvisionResult()
if err != nil {
t.Fatalf("could not read provision result: %v", err)
}
if result.Code != string(provisionCodeListenerAddrUnavail) {
t.Errorf("result code = %s, want %s", result.Code, provisionCodeListenerAddrUnavail)
}
if result.Stage != string(provisionStageListener) {
t.Errorf("result stage = %s, want %s", result.Stage, provisionStageListener)
}
if result.ExitCode != 42 {
t.Errorf("result exit code = %d, want 42", result.ExitCode)
}
if result.Detail == nil || len(result.Detail.Attempts) == 0 {
t.Fatal("expected the occupied address to appear as a recorded bind attempt")
}
occupiedAddr := net.JoinHostPort(host, portStr)
// Windows words WSAEADDRINUSE differently, so only require the canonical
// message on platforms that produce it.
requireInUseText := runtime.GOOS != "windows"
var sawUDP, sawTCP bool
for _, a := range result.Detail.Attempts {
if a.Addr != occupiedAddr || a.OSError == "" {
continue
}
if requireInUseText && !strings.Contains(strings.ToLower(a.OSError), "address already in use") {
continue
}
switch a.Proto {
case "udp":
sawUDP = true
case "tcp":
sawTCP = true
}
}
if !sawUDP {
t.Error("expected a udp attempt on the occupied address with a bind error")
}
if !sawTCP {
t.Error("expected a tcp attempt on the occupied address with a bind error")
}
}
// The exhaustion path (exit 41) is not covered: forcing every fallback,
// including a freshly randomized ip/port, to fail has no deterministic seam,
// so a test would race whatever ports are free on the host.
+404 -91
View File
@@ -11,12 +11,14 @@ import (
"net/http"
"os"
"os/exec"
"os/signal"
"path/filepath"
"runtime"
"slices"
"sort"
"strconv"
"strings"
"syscall"
"time"
"github.com/docker/go-units"
@@ -146,6 +148,88 @@ func initLogCmd() *cobra.Command {
fmt.Println(logs.Data)
},
}
var tailLines int
logTailCmd := &cobra.Command{
Use: "tail",
Short: "Tail live runtime debug logs",
Long: "Stream live runtime debug logs to the terminal, similar to tail -f. Press Ctrl+C to stop.",
Args: cobra.NoArgs,
PreRun: func(cmd *cobra.Command, args []string) {
checkHasElevatedPrivilege()
},
Run: func(cmd *cobra.Command, args []string) {
p := &prog{router: router.New(&cfg, false)}
s, _ := newService(p, svcConfig)
status, err := s.Status()
if errors.Is(err, service.ErrNotInstalled) {
mainLog.Load().Warn().Msg("service not installed")
return
}
if status == service.StatusStopped {
mainLog.Load().Warn().Msg("service is not running")
return
}
dir, err := socketDir()
if err != nil {
mainLog.Load().Fatal().Err(err).Msg("failed to find ctrld home dir")
}
cc := newControlClient(filepath.Join(dir, ctrldControlUnixSock))
tailPath := fmt.Sprintf("%s?lines=%d", tailLogsPath, tailLines)
resp, err := cc.postStream(tailPath, nil)
if err != nil {
mainLog.Load().Fatal().Err(err).Msg("failed to connect for log tailing")
}
defer resp.Body.Close()
switch resp.StatusCode {
case http.StatusMovedPermanently:
warnRuntimeLoggingNotEnabled()
return
case http.StatusOK:
default:
mainLog.Load().Fatal().Msgf("unexpected response status: %d", resp.StatusCode)
return
}
// Set up signal handling for clean shutdown.
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
done := make(chan struct{})
go func() {
defer close(done)
// Stream output to stdout.
buf := make([]byte, 4096)
for {
n, readErr := resp.Body.Read(buf)
if n > 0 {
os.Stdout.Write(buf[:n])
}
if readErr != nil {
if readErr != io.EOF {
mainLog.Load().Error().Err(readErr).Msg("error reading log stream")
}
return
}
}
}()
select {
case <-ctx.Done():
if errors.Is(ctx.Err(), context.Canceled) {
msg := fmt.Sprintf("\nexiting: %s\n", context.Cause(ctx).Error())
os.Stdout.WriteString(msg)
}
case <-done:
}
},
}
logTailCmd.Flags().IntVarP(&tailLines, "lines", "n", 10, "Number of historical lines to show on connect")
logCmd := &cobra.Command{
Use: "log",
Short: "Manage runtime debug logs",
@@ -156,6 +240,7 @@ func initLogCmd() *cobra.Command {
}
logCmd.AddCommand(logSendCmd)
logCmd.AddCommand(logViewCmd)
logCmd.AddCommand(logTailCmd)
rootCmd.AddCommand(logCmd)
return logCmd
@@ -190,6 +275,7 @@ func initRunCmd() *cobra.Command {
_ = runCmd.Flags().MarkHidden("iface")
runCmd.Flags().StringVarP(&cdUpstreamProto, "proto", "", ctrld.ResolverTypeDOH, `Control D upstream type, either "doh" or "doh3"`)
runCmd.Flags().BoolVarP(&rfc1918, "rfc1918", "", false, "Listen on RFC1918 addresses when 127.0.0.1 is the only listener")
runCmd.Flags().StringVarP(&interceptMode, "intercept-mode", "", "", "OS-level DNS interception mode: 'dns' (with VPN split routing) or 'hard' (all DNS through ctrld, no VPN split routing)")
runCmd.FParseErrWhitelist = cobra.FParseErrWhitelist{UnknownFlags: true}
rootCmd.AddCommand(runCmd)
@@ -197,6 +283,46 @@ func initRunCmd() *cobra.Command {
return runCmd
}
// serviceStageFailureCode maps an aborted service-manager task to its
// provisioning code. Other abortOnError tasks (like config validation) keep
// their own error paths.
func serviceStageFailureCode(taskName string) (provisionFailureCode, bool) {
switch taskName {
case "Install":
return provisionCodeServiceInstall, true
case "Start":
return provisionCodeServiceStartFailed, true
default:
return "", false
}
}
// serviceTaskErrorSummary describes which service-manager task failed and why,
// for use as a provisioning result message.
func serviceTaskErrorSummary(taskName string, err error) string {
return fmt.Sprintf("%s failed: %v", taskName, err)
}
// resultStalenessTolerance absorbs clock granularity between "ctrld start"
// recording its start time and the daemon writing its result file.
const resultStalenessTolerance = 2 * time.Second
// reportStartFailure reports why "ctrld start" failed after install/start
// looked fine. A result file the daemon wrote during this attempt names the
// failure better than a generic self-check code, so it wins.
func reportStartFailure(startedAt time.Time, fallbackMsg string) {
if r, err := readProvisionResult(); err == nil && provisionResultTrusted(r) {
if ts, err := time.Parse(time.RFC3339, r.Timestamp); err == nil {
if !ts.Before(startedAt.Add(-resultStalenessTolerance)) {
mainLog.Load().Error().Msg(r.failureLine())
provisionExit(r.ExitCode)
return
}
}
}
failProvision(newProvisionResult(provisionCodeServiceSelfCheck, fallbackMsg, nil, provisionSecrets()...), nil)
}
func initStartCmd() *cobra.Command {
startCmd := &cobra.Command{
PreRun: func(cmd *cobra.Command, args []string) {
@@ -229,6 +355,14 @@ NOTE: running "ctrld start" without any arguments will start already installed c
setDependencies(sc)
sc.Arguments = append([]string{"run"}, osArgs...)
// Validate --intercept-mode early, before installing the service.
// Without this, a typo like "--intercept-mode fds" would install the service,
// the child process would Fatal() on the invalid value, and the parent would
// then uninstall — confusing and destructive.
if interceptMode != "" && !validInterceptMode(interceptMode) {
mainLog.Load().Fatal().Msgf("invalid --intercept-mode value %q: must be 'off', 'dns', or 'hard'", interceptMode)
}
p := &prog{
router: router.New(&cfg, cdUID != ""),
cfg: &cfg,
@@ -247,6 +381,51 @@ NOTE: running "ctrld start" without any arguments will start already installed c
// Get current running iface, if any.
var currentIface *ifaceResponse
// Handle "ctrld start --intercept-mode dns|hard" on an existing
// service BEFORE the pin check. Adding intercept mode is an enhancement, not
// deactivation, so it doesn't require the deactivation pin. We modify the
// plist/registry directly and restart the service via the OS service manager.
osArgsEarly := os.Args[2:]
if os.Args[1] == "service" {
osArgsEarly = os.Args[3:]
}
osArgsEarly = filterEmptyStrings(osArgsEarly)
interceptOnly := onlyInterceptFlags(osArgsEarly)
svcExists := serviceConfigFileExists()
mainLog.Load().Debug().Msgf("intercept upgrade check: args=%v interceptOnly=%v svcConfigExists=%v interceptMode=%q", osArgsEarly, interceptOnly, svcExists, interceptMode)
if interceptOnly && svcExists {
// Replace any existing split or --intercept-mode=<value> form. Keep an
// explicit "off" argument so it overrides a previously persisted config
// value while the service clears that value on startup.
if err := removeServiceFlag("--intercept-mode"); err != nil {
mainLog.Load().Fatal().Err(err).Msg("failed to remove existing intercept mode from service arguments")
}
if interceptMode == "off" {
mainLog.Load().Notice().Msg("Existing service detected — disabling intercept mode")
} else {
mainLog.Load().Notice().Msgf("Existing service detected — appending --intercept-mode %s to service arguments", interceptMode)
}
if err := appendServiceFlag("--intercept-mode"); err != nil {
mainLog.Load().Fatal().Err(err).Msg("failed to append intercept flag to service arguments")
}
if err := appendServiceFlag(interceptMode); err != nil {
mainLog.Load().Fatal().Err(err).Msg("failed to append intercept mode value to service arguments")
}
// Stop the service if running (bypasses ctrld pin — this is an
// enhancement, not deactivation). Then fall through to the normal
// startOnly path which handles start, self-check, and reporting.
if isCtrldRunning {
mainLog.Load().Notice().Msg("Stopping service for intercept mode upgrade")
_ = s.Stop()
isCtrldRunning = false
}
startOnly = true
isCtrldInstalled = true
// Fall through to startOnly path below.
}
// If pin code was set, do not allow running start command.
if isCtrldRunning {
if err := checkDeactivationPin(s, nil); isCheckDeactivationPinErr(err) {
@@ -262,7 +441,7 @@ NOTE: running "ctrld start" without any arguments will start already installed c
reportSetDnsOk := func(sockDir string) {
if cc := newSocketControlClient(ctx, s, sockDir); cc != nil {
if resp, _ := cc.post(ifacePath, nil); resp != nil && resp.StatusCode == http.StatusOK {
if iface == "auto" {
if iface == autoIface {
iface = defaultIfaceName()
}
res := &ifaceResponse{}
@@ -271,20 +450,31 @@ NOTE: running "ctrld start" without any arguments will start already installed c
return
}
if res.OK {
name := res.Name
if iff, err := net.InterfaceByName(name); err == nil {
_, _ = patchNetIfaceName(iff)
name = iff.Name
}
logger := mainLog.Load().With().Str("iface", name).Logger()
logger.Debug().Msg("setting DNS successfully")
if res.All {
// Log that DNS is set for other interfaces.
withEachPhysicalInterfaces(
name,
"set DNS",
func(i *net.Interface) error { return nil },
)
// In intercept mode, show intercept-specific status instead of
// per-interface DNS messages (which are irrelevant).
if res.InterceptMode != "" {
switch res.InterceptMode {
case "hard":
mainLog.Load().Notice().Msg("DNS hard intercept mode active — all DNS traffic intercepted, no VPN split routing")
default:
mainLog.Load().Notice().Msg("DNS intercept mode active — all DNS traffic intercepted via OS packet filter")
}
} else {
name := res.Name
if iff, err := net.InterfaceByName(name); err == nil {
_, _ = patchNetIfaceName(iff)
name = iff.Name
}
logger := mainLog.Load().With().Str("iface", name).Logger()
logger.Debug().Msg("setting DNS successfully")
if res.All {
// Log that DNS is set for other interfaces.
withEachPhysicalInterfaces(
name,
"set DNS",
func(i *net.Interface) error { return nil },
)
}
}
}
}
@@ -344,6 +534,7 @@ NOTE: running "ctrld start" without any arguments will start already installed c
if !startOnly {
startOnly = len(osArgs) == 0
}
// If user run "ctrld start" and ctrld is already installed, starting existing service.
if startOnly && isCtrldInstalled {
tryReadingConfigWithNotice(false, true)
@@ -375,24 +566,56 @@ NOTE: running "ctrld start" without any arguments will start already installed c
{s.Start, true, "Start"},
{noticeWritingControlDConfig, false, "Notice writing ControlD config"},
}
// Any result found later must come from this attempt, not a stale run.
clearProvisionResult()
startAttemptAt := time.Now()
mainLog.Load().Notice().Msg("Starting existing ctrld service")
if doTasks(tasks) {
mainLog.Load().Notice().Msg("Service started")
sockDir, err := socketDir()
if err != nil {
mainLog.Load().Warn().Err(err).Msg("Failed to get socket directory")
os.Exit(1)
failedTask, taskErr := doTasksE(tasks)
if taskErr != nil {
if code, ok := serviceStageFailureCode(failedTask); ok {
failProvision(newProvisionResult(code, serviceTaskErrorSummary(failedTask, taskErr), nil, provisionSecrets()...), nil)
return
}
reportSetDnsOk(sockDir)
} else {
mainLog.Load().Error().Err(err).Msg("Failed to start existing ctrld service")
os.Exit(1)
}
sockDir, err := socketDir()
if err != nil {
mainLog.Load().Warn().Err(err).Msg("Failed to get socket directory")
os.Exit(1)
}
// The daemon can start and still fail provisioning (for example a
// listener bind conflict). Self-check like a fresh install so this
// path reports the daemon's failure code instead of a false
// "Service started" — but never uninstall an existing service.
time.Sleep(1 * time.Second)
ok, status, err := selfCheckStatus(ctx, s, sockDir)
if !ok || status != service.StatusRunning {
fallbackMsg := "ctrld service did not pass its post-start self-check"
if err != nil {
fallbackMsg = fmt.Sprintf("An error occurred while performing test query: %s", err)
mainLog.Load().Error().Msg(fallbackMsg)
}
if status == service.StatusRunning && err == nil {
fallbackMsg = "ctrld service was running, but a DNS query could not be sent to its listener; check firewall rules blocking/intercepting/redirecting DNS queries"
mainLog.Load().Error().Msg(fallbackMsg)
}
reportStartFailure(startAttemptAt, fallbackMsg)
return
}
mainLog.Load().Notice().Msg("Service started")
clearProvisionResult()
reportSetDnsOk(sockDir)
// Verify service registration after successful start.
if err := verifyServiceRegistration(); err != nil {
mainLog.Load().Warn().Err(err).Msg("Service registry verification failed")
}
return
}
if cdUID != "" {
_ = doValidateCdRemoteConfig(cdUID, true)
// Skip doValidateCdRemoteConfig() here - run command will handle
// validation and config fetch via processCDFlags().
} else if uid := cdUIDFromProvToken(); uid != "" {
cdUID = uid
mainLog.Load().Debug().Msg("using uid from provision token")
@@ -451,7 +674,7 @@ NOTE: running "ctrld start" without any arguments will start already installed c
})
return nil
}, false, "Save current DNS"},
{s.Install, false, "Install"},
{s.Install, true, "Install"},
{func() error {
return ConfigureWindowsServiceFailureActions(ctrldServiceName)
}, false, "Configure Windows service failure actions"},
@@ -460,55 +683,77 @@ NOTE: running "ctrld start" without any arguments will start already installed c
// generated after s.Start, so we notice users here for consistent with nextdns mode.
{noticeWritingControlDConfig, false, "Notice writing ControlD config"},
}
// Any result found later must come from this attempt, not a stale run.
clearProvisionResult()
startAttemptAt := time.Now()
mainLog.Load().Notice().Msg("Starting service")
if doTasks(tasks) {
if err := p.router.Install(sc); err != nil {
mainLog.Load().Warn().Err(err).Msg("post installation failed, please check system/service log for details error")
failedTask, taskErr := doTasksE(tasks)
if taskErr != nil {
if code, ok := serviceStageFailureCode(failedTask); ok {
failProvision(newProvisionResult(code, serviceTaskErrorSummary(failedTask, taskErr), nil, provisionSecrets()...), nil)
return
}
// Not a service-stage task. doTasksE already logged the cause; exit
// non-zero instead of the old silent fall-through that exited 0.
os.Exit(1)
return
}
// add a small delay to ensure the service is started and did not crash
time.Sleep(1 * time.Second)
if err := p.router.Install(sc); err != nil {
mainLog.Load().Warn().Err(err).Msg("post installation failed, please check system/service log for details error")
return
}
ok, status, err := selfCheckStatus(ctx, s, sockDir)
switch {
case ok && status == service.StatusRunning:
mainLog.Load().Notice().Msg("Service started")
default:
marker := bytes.Repeat([]byte("="), 32)
// If ctrld service is not running, emitting log obtained from ctrld process.
if status != service.StatusRunning || ctx.Err() != nil {
mainLog.Load().Error().Msg("ctrld service may not have started due to an error or misconfiguration, service log:")
_, _ = mainLog.Load().Write(marker)
haveLog := false
for msg := range runCmdLogCh {
_, _ = mainLog.Load().Write([]byte(strings.ReplaceAll(msg, msgExit, "")))
haveLog = true
}
// If we're unable to get log from "ctrld run", notice users about it.
if !haveLog {
mainLog.Load().Write([]byte(`<no log output is obtained from ctrld process>"`))
}
}
// Report any error if occurred.
if err != nil {
_, _ = mainLog.Load().Write(marker)
msg := fmt.Sprintf("An error occurred while performing test query: %s", err)
mainLog.Load().Write([]byte(msg))
}
// If ctrld service is running but selfCheckStatus failed, it could be related
// to user's system firewall configuration, notice users about it.
if status == service.StatusRunning && err == nil {
_, _ = mainLog.Load().Write(marker)
mainLog.Load().Write([]byte(`ctrld service was running, but a DNS query could not be sent to its listener`))
mainLog.Load().Write([]byte(`Please check your system firewall if it is configured to block/intercept/redirect DNS queries`))
}
// add a small delay to ensure the service is started and did not crash
time.Sleep(1 * time.Second)
ok, status, err := selfCheckStatus(ctx, s, sockDir)
switch {
case ok && status == service.StatusRunning:
mainLog.Load().Notice().Msg("Service started")
clearProvisionResult()
default:
marker := bytes.Repeat([]byte("="), 32)
fallbackMsg := "ctrld service did not pass its post-start self-check"
// If ctrld service is not running, emitting log obtained from ctrld process.
if status != service.StatusRunning || ctx.Err() != nil {
mainLog.Load().Error().Msg("ctrld service may not have started due to an error or misconfiguration, service log:")
_, _ = mainLog.Load().Write(marker)
uninstall(p, s)
os.Exit(1)
haveLog := false
for msg := range runCmdLogCh {
_, _ = mainLog.Load().Write([]byte(strings.ReplaceAll(msg, msgExit, "")))
haveLog = true
}
// If we're unable to get log from "ctrld run", notice users about it.
if !haveLog {
mainLog.Load().Write([]byte(`<no log output is obtained from ctrld process>"`))
}
}
reportSetDnsOk(sockDir)
// Report any error if occurred.
if err != nil {
_, _ = mainLog.Load().Write(marker)
msg := fmt.Sprintf("An error occurred while performing test query: %s", err)
mainLog.Load().Write([]byte(msg))
fallbackMsg = msg
}
// If ctrld service is running but selfCheckStatus failed, it could be related
// to user's system firewall configuration, notice users about it.
if status == service.StatusRunning && err == nil {
_, _ = mainLog.Load().Write(marker)
mainLog.Load().Write([]byte(`ctrld service was running, but a DNS query could not be sent to its listener`))
mainLog.Load().Write([]byte(`Please check your system firewall if it is configured to block/intercept/redirect DNS queries`))
fallbackMsg = "ctrld service was running, but a DNS query could not be sent to its listener; check firewall rules blocking/intercepting/redirecting DNS queries"
}
_, _ = mainLog.Load().Write(marker)
uninstall(p, s)
reportStartFailure(startAttemptAt, fallbackMsg)
return
}
reportSetDnsOk(sockDir)
// Verify service registration after successful start.
if err := verifyServiceRegistration(); err != nil {
mainLog.Load().Warn().Err(err).Msg("Service registry verification failed")
}
},
}
@@ -533,6 +778,7 @@ NOTE: running "ctrld start" without any arguments will start already installed c
startCmd.Flags().BoolVarP(&startOnly, "start_only", "", false, "Do not install new service")
_ = startCmd.Flags().MarkHidden("start_only")
startCmd.Flags().BoolVarP(&rfc1918, "rfc1918", "", false, "Listen on RFC1918 addresses when 127.0.0.1 is the only listener")
startCmd.Flags().StringVarP(&interceptMode, "intercept-mode", "", "", "OS-level DNS interception mode: 'dns' (with VPN split routing) or 'hard' (all DNS through ctrld, no VPN split routing)")
routerCmd := &cobra.Command{
Use: "setup",
@@ -589,7 +835,7 @@ NOTE: running "ctrld start" without any arguments will start already installed c
startCmd.Run(cmd, args)
},
}
startCmdAlias.Flags().StringVarP(&ifaceStartStop, "iface", "", "auto", `Update DNS setting for iface, "auto" means the default interface gateway`)
startCmdAlias.Flags().StringVarP(&ifaceStartStop, "iface", "", autoIface, `Update DNS setting for iface, "auto" means the default interface gateway`)
startCmdAlias.Flags().AddFlagSet(startCmd.Flags())
rootCmd.AddCommand(startCmdAlias)
@@ -674,7 +920,7 @@ func initStopCmd() *cobra.Command {
stopCmd.Run(cmd, args)
},
}
stopCmdAlias.Flags().StringVarP(&ifaceStartStop, "iface", "", "auto", `Reset DNS setting for iface, "auto" means the default interface gateway`)
stopCmdAlias.Flags().StringVarP(&ifaceStartStop, "iface", "", autoIface, `Reset DNS setting for iface, "auto" means the default interface gateway`)
stopCmdAlias.Flags().AddFlagSet(stopCmd.Flags())
rootCmd.AddCommand(stopCmdAlias)
@@ -706,7 +952,7 @@ func initRestartCmd() *cobra.Command {
return
}
if iface == "" {
iface = "auto"
iface = autoIface
}
p.preRun()
if ir := runningIface(s); ir != nil {
@@ -888,6 +1134,7 @@ func initStatusCmd() *cobra.Command {
statusCmd := &cobra.Command{
Use: "status",
Short: "Show status of the ctrld service",
Long: statusCmdLong,
Args: cobra.NoArgs,
Run: func(cmd *cobra.Command, args []string) {
s, err := newService(&prog{}, svcConfig)
@@ -903,13 +1150,25 @@ func initStatusCmd() *cobra.Command {
switch status {
case service.StatusUnknown:
mainLog.Load().Notice().Msg("Unknown status")
os.Exit(2)
os.Exit(statusExitUnknown)
case service.StatusRunning:
mainLog.Load().Notice().Msg("Service is running")
os.Exit(0)
// The service manager only knows a process was created. It reports a
// service as running even when the process is still in startup, with
// no control socket, no DNS listener and no policy applied - so
// "Service is running" can describe a host with no working DNS.
// Probe readiness before claiming it.
ready, probeErr := serviceReady()
if probeErr != nil {
mainLog.Load().Debug().Err(probeErr).Msg("Readiness probe did not confirm startup")
}
r := classifyReadiness(ready, probeErr, readinessVerifiable())
for _, msg := range r.messages {
mainLog.Load().Notice().Msg(msg)
}
os.Exit(r.exitCode)
case service.StatusStopped:
mainLog.Load().Notice().Msg("Service is stopped")
os.Exit(1)
os.Exit(statusExitStopped)
}
},
}
@@ -923,6 +1182,7 @@ func initStatusCmd() *cobra.Command {
statusCmdAlias := &cobra.Command{
Use: "status",
Short: "Show status of the ctrld service",
Long: statusCmdLong,
Args: cobra.NoArgs,
Run: statusCmd.Run,
}
@@ -952,7 +1212,7 @@ NOTE: Uninstalling will set DNS to values provided by DHCP.`,
return
}
if iface == "" {
iface = "auto"
iface = autoIface
}
p.preRun()
if ir := runningIface(s); ir != nil {
@@ -1048,7 +1308,7 @@ NOTE: Uninstalling will set DNS to values provided by DHCP.`,
uninstallCmd.Run(cmd, args)
},
}
uninstallCmdAlias.Flags().StringVarP(&ifaceStartStop, "iface", "", "auto", `Reset DNS setting for iface, "auto" means the default interface gateway`)
uninstallCmdAlias.Flags().StringVarP(&ifaceStartStop, "iface", "", autoIface, `Reset DNS setting for iface, "auto" means the default interface gateway`)
uninstallCmdAlias.Flags().AddFlagSet(uninstallCmd.Flags())
rootCmd.AddCommand(uninstallCmdAlias)
@@ -1241,7 +1501,7 @@ func initUpgradeCmd() *cobra.Command {
return
}
if iface == "" {
iface = "auto"
iface = autoIface
}
p.preRun()
if ir := runningIface(s); ir != nil {
@@ -1342,28 +1602,31 @@ func initUpgradeCmd() *cobra.Command {
if doRestart() {
_ = os.Remove(oldBin)
_ = os.Chmod(bin, 0755)
ver := "unknown version"
out, err := exec.Command(bin, "--version").CombinedOutput()
ver, err := binaryVersion(bin)
if err != nil {
mainLog.Load().Warn().Err(err).Msg("Failed to get new binary version")
}
if after, found := strings.CutPrefix(string(out), "ctrld version "); found {
ver = after
ver = "unknown version"
}
mainLog.Load().Notice().Msgf("Upgrade successful - %s", ver)
return
}
mainLog.Load().Warn().Msgf("Upgrade failed, restoring previous binary: %s", oldBin)
if err := os.Remove(bin); err != nil {
mainLog.Load().Fatal().Err(err).Msg("failed to remove new binary")
mainLog.Load().Warn().Msg("Upgrade failed: the new binary did not become ready")
stop := func() error {
if !svcInstalled {
return nil
}
if err := stopServiceAndWait(s, upgradeStopTimeout); err != nil {
return err
}
// Mirror the Cleanup task in doRestart: leave DNS settings as the OS
// had them, not as a half-started ctrld left them.
p.router.Cleanup()
p.resetDNS(false, true)
return nil
}
if err := os.Rename(oldBin, bin); err != nil {
mainLog.Load().Fatal().Err(err).Msg("failed to restore old binary")
}
if doRestart() {
mainLog.Load().Notice().Msg("Restored previous binary successfully")
return
if err := rollbackToPreviousBinary(bin, oldBin, stop, doRestart); err != nil {
mainLog.Load().Error().Err(err).Msg("Rollback did not complete")
}
},
}
@@ -1395,3 +1658,53 @@ func filterEmptyStrings(slice []string) []string {
return s == ""
})
}
// validInterceptMode reports whether the given value is a recognized --intercept-mode.
// This is the single source of truth for mode validation — used by the early start
// command check, the runtime validation in prog.go, and onlyInterceptFlags below.
// Add new modes here to have them recognized everywhere.
func validInterceptMode(mode string) bool {
switch mode {
case "off", "dns", "hard":
return true
}
return false
}
// onlyInterceptFlags reports whether args contain only intercept mode
// flags (--intercept-mode <value>) and flags that are auto-added by the
// start command alias (--iface). This is used to detect "ctrld start --intercept-mode dns"
// (or "off" to disable) on an existing installation, where the intent is to modify the
// intercept flag on the existing service without replacing other arguments.
//
// Note: the startCmdAlias appends "--iface=auto" to os.Args when --iface isn't
// explicitly provided, so we must allow it here.
func onlyInterceptFlags(args []string) bool {
hasIntercept := false
for i := 0; i < len(args); i++ {
arg := args[i]
switch {
case arg == "--intercept-mode":
// Next arg must be a valid mode value.
if i+1 < len(args) && validInterceptMode(args[i+1]) {
hasIntercept = true
i++ // skip the value
} else {
return false
}
case strings.HasPrefix(arg, "--intercept-mode="):
val := strings.TrimPrefix(arg, "--intercept-mode=")
if validInterceptMode(val) {
hasIntercept = true
} else {
return false
}
case arg == "--iface="+autoIface || arg == "--iface" || arg == autoIface:
// Auto-added by startCmdAlias or its value; safe to ignore.
continue
default:
return false
}
}
return hasIntercept
}
+122
View File
@@ -0,0 +1,122 @@
package cli
import (
"testing"
"time"
)
func TestServiceStageFailureCode(t *testing.T) {
tests := []struct {
taskName string
wantCode provisionFailureCode
wantOK bool
}{
{"Install", provisionCodeServiceInstall, true},
{"Start", provisionCodeServiceStartFailed, true},
{"Checking config", "", false},
{"", "", false},
}
for _, tc := range tests {
code, ok := serviceStageFailureCode(tc.taskName)
if code != tc.wantCode || ok != tc.wantOK {
t.Errorf("serviceStageFailureCode(%q) = (%q, %v), want (%q, %v)", tc.taskName, code, ok, tc.wantCode, tc.wantOK)
}
}
}
func stubProvisionExit(t *testing.T) *int {
t.Helper()
exitCode := -1
old := provisionExit
provisionExit = func(code int) { exitCode = code }
t.Cleanup(func() { provisionExit = old })
return &exitCode
}
func TestReportStartFailureUsesFreshDaemonResult(t *testing.T) {
overrideProvisionResultPath(t)
exitCode := stubProvisionExit(t)
startedAt := time.Now()
daemonResult := newProvisionResult(provisionCodeAPIUnreachable, "daemon could not reach the API", nil)
if err := writeProvisionResult(daemonResult); err != nil {
t.Fatal(err)
}
reportStartFailure(startedAt, "generic self-check failure")
if *exitCode != provisionExitCodeForCode[provisionCodeAPIUnreachable] {
t.Errorf("exit code = %d, want the daemon's own exit code %d", *exitCode, provisionExitCodeForCode[provisionCodeAPIUnreachable])
}
out, err := readProvisionResult()
if err != nil {
t.Fatal(err)
}
if out.Code != string(provisionCodeAPIUnreachable) {
t.Errorf("persisted code = %q, want the daemon's own code untouched", out.Code)
}
}
func TestReportStartFailureFallsBackOnStaleDaemonResult(t *testing.T) {
overrideProvisionResultPath(t)
exitCode := stubProvisionExit(t)
stale := newProvisionResult(provisionCodeAPIUnreachable, "an old failure", nil)
stale.Timestamp = time.Now().Add(-1 * time.Hour).UTC().Format(time.RFC3339)
if err := writeProvisionResult(stale); err != nil {
t.Fatal(err)
}
startedAt := time.Now()
reportStartFailure(startedAt, "test query failed: timeout")
if *exitCode != provisionExitCodeForCode[provisionCodeServiceSelfCheck] {
t.Errorf("exit code = %d, want SERVICE_SELFCHECK_FAILED exit %d", *exitCode, provisionExitCodeForCode[provisionCodeServiceSelfCheck])
}
out, err := readProvisionResult()
if err != nil {
t.Fatal(err)
}
if out.Code != string(provisionCodeServiceSelfCheck) {
t.Errorf("persisted code = %q, want %q", out.Code, provisionCodeServiceSelfCheck)
}
if out.Message != "test query failed: timeout" {
t.Errorf("persisted message = %q, want the fallback message", out.Message)
}
}
func TestReportStartFailureRejectsUntrustedFile(t *testing.T) {
overrideProvisionResultPath(t)
exitCode := stubProvisionExit(t)
planted := newProvisionResult(provisionCodeAPIUnreachable, "planted", nil)
planted.Code = "FAKE_CODE"
planted.ExitCode = 99
if err := writeProvisionResult(planted); err != nil {
t.Fatal(err)
}
reportStartFailure(time.Now().Add(-time.Minute), "self-check failed")
if *exitCode != provisionExitCodeForCode[provisionCodeServiceSelfCheck] {
t.Errorf("exit = %d, want the fallback %d, never the planted 99", *exitCode, provisionExitCodeForCode[provisionCodeServiceSelfCheck])
}
}
func TestReportStartFailureFallsBackWhenResultFileMissing(t *testing.T) {
overrideProvisionResultPath(t)
exitCode := stubProvisionExit(t)
reportStartFailure(time.Now(), "firewall hint")
if *exitCode != provisionExitCodeForCode[provisionCodeServiceSelfCheck] {
t.Errorf("exit code = %d, want SERVICE_SELFCHECK_FAILED exit %d", *exitCode, provisionExitCodeForCode[provisionCodeServiceSelfCheck])
}
out, err := readProvisionResult()
if err != nil {
t.Fatal(err)
}
if out.Message != "firewall hint" {
t.Errorf("persisted message = %q, want the fallback message", out.Message)
}
}
+6
View File
@@ -32,6 +32,12 @@ func (c *controlClient) post(path string, data io.Reader) (*http.Response, error
return c.c.Post("http://unix"+path, contentTypeJson, data)
}
// postStream sends a POST request with no timeout, suitable for long-lived streaming connections.
func (c *controlClient) postStream(path string, data io.Reader) (*http.Response, error) {
c.c.Timeout = 0
return c.c.Post("http://unix"+path, contentTypeJson, data)
}
// deactivationRequest represents request for validating deactivation pin.
type deactivationRequest struct {
Pin int64 `json:"pin"`
+202 -8
View File
@@ -10,6 +10,7 @@ import (
"os"
"reflect"
"sort"
"strconv"
"time"
"github.com/kardianos/service"
@@ -29,12 +30,14 @@ const (
ifacePath = "/iface"
viewLogsPath = "/log/view"
sendLogsPath = "/log/send"
tailLogsPath = "/log/tail"
)
type ifaceResponse struct {
Name string `json:"name"`
All bool `json:"all"`
OK bool `json:"ok"`
Name string `json:"name"`
All bool `json:"all"`
OK bool `json:"ok"`
InterceptMode string `json:"intercept_mode,omitempty"` // "dns", "hard", or "" (not intercepting)
}
type controlServer struct {
@@ -56,12 +59,18 @@ func newControlServer(addr string) (*controlServer, error) {
func (s *controlServer) start() error {
_ = os.Remove(s.addr)
unixListener, err := net.Listen("unix", s.addr)
if l, ok := unixListener.(*net.UnixListener); ok {
l.SetUnlinkOnClose(true)
}
if err != nil {
return err
}
// Restrict socket permissions to owner-only (0600) so that only the
// process owner (typically root) can connect. Defense-in-depth since
// the control server endpoints carry no authentication of their own.
if err := os.Chmod(s.addr, 0600); err != nil {
return err
}
if l, ok := unixListener.(*net.UnixListener); ok {
l.SetUnlinkOnClose(true)
}
go s.server.Serve(unixListener)
return nil
}
@@ -216,8 +225,19 @@ func (p *prog) registerControlServerHandler() {
return
}
// Reject further attempts while locked out due to repeated wrong PINs.
if now := time.Now().Unix(); now < deactivationLockedUntil.Load() {
w.WriteHeader(http.StatusTooManyRequests)
return
}
// Re-fetch pin code from API.
if rc, err := controld.FetchResolverConfig(cdUID, rootCmd.Version, cdDev); rc != nil {
rcReq := &controld.ResolverConfigRequest{
RawUID: cdUID,
Version: rootCmd.Version,
Metadata: ctrld.SystemMetadataRuntime(context.Background()),
}
if rc, err := controld.FetchResolverConfig(context.Background(), rcReq, cdDev); rc != nil {
if rc.DeactivationPin != nil {
cdDeactivationPin.Store(*rc.DeactivationPin)
} else {
@@ -244,6 +264,7 @@ func (p *prog) registerControlServerHandler() {
switch req.Pin {
case cdDeactivationPin.Load():
code = http.StatusOK
deactivationFailedAttempts.Store(0)
select {
case p.pinCodeValidCh <- struct{}{}:
default:
@@ -251,6 +272,11 @@ func (p *prog) registerControlServerHandler() {
case defaultDeactivationPin:
// If the pin code was set, but users do not provide --pin, return proper code to client.
code = http.StatusBadRequest
default:
if deactivationFailedAttempts.Add(1) >= deactivationMaxFailedAttempts {
deactivationLockedUntil.Store(time.Now().Unix() + deactivationLockoutSeconds)
deactivationFailedAttempts.Store(0)
}
}
w.WriteHeader(code)
}))
@@ -271,6 +297,10 @@ func (p *prog) registerControlServerHandler() {
res.Name = p.runningIface
res.All = p.requiredMultiNICsConfig
res.OK = true
// Report intercept mode to the start command for proper log output.
if interceptMode == "dns" || interceptMode == "hard" {
res.InterceptMode = interceptMode
}
}
}
if err := json.NewEncoder(w).Encode(res); err != nil {
@@ -321,7 +351,7 @@ func (p *prog) registerControlServerHandler() {
}
mainLog.Load().Debug().Msg("sending log file to ControlD server")
resp := logSentResponse{Size: r.size}
if err := controld.SendLogs(req, cdDev); err != nil {
if err := controld.SendLogs(context.Background(), req, cdDev); err != nil {
mainLog.Load().Error().Msgf("could not send log file to ControlD server: %v", err)
resp.Error = err.Error()
w.WriteHeader(http.StatusInternalServerError)
@@ -334,6 +364,170 @@ func (p *prog) registerControlServerHandler() {
}
p.internalLogSent = time.Now()
}))
p.cs.register(tailLogsPath, http.HandlerFunc(func(w http.ResponseWriter, request *http.Request) {
flusher, ok := w.(http.Flusher)
if !ok {
http.Error(w, "streaming unsupported", http.StatusInternalServerError)
return
}
// Determine logging mode and validate before starting the stream.
var lw *logWriter
useInternalLog := p.needInternalLogging()
if useInternalLog {
p.mu.Lock()
lw = p.internalLogWriter
p.mu.Unlock()
if lw == nil {
w.WriteHeader(http.StatusMovedPermanently)
return
}
} else if p.cfg.Service.LogPath == "" {
// No logging configured at all.
w.WriteHeader(http.StatusMovedPermanently)
return
}
// Parse optional "lines" query param for initial context.
numLines := 10
if v := request.URL.Query().Get("lines"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n >= 0 {
numLines = n
}
}
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.Header().Set("Transfer-Encoding", "chunked")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(http.StatusOK)
if useInternalLog {
// Internal logging mode: subscribe to the logWriter.
// Send last N lines as initial context.
if numLines > 0 {
if tail := lw.tailLastLines(numLines); len(tail) > 0 {
w.Write(tail)
flusher.Flush()
}
}
ch, unsub := lw.Subscribe()
defer unsub()
for {
select {
case data, ok := <-ch:
if !ok {
return
}
if _, err := w.Write(data); err != nil {
return
}
flusher.Flush()
case <-request.Context().Done():
return
}
}
} else {
// File-based logging mode: tail the log file.
logFile := normalizeLogFilePath(p.cfg.Service.LogPath)
f, err := os.Open(logFile)
if err != nil {
// Already committed 200, just return.
return
}
defer f.Close()
// Seek to show last N lines.
if numLines > 0 {
if tail := tailFileLastLines(f, numLines); len(tail) > 0 {
w.Write(tail)
flusher.Flush()
}
} else {
// Seek to end.
f.Seek(0, io.SeekEnd)
}
// Poll for new data.
buf := make([]byte, 4096)
ticker := time.NewTicker(200 * time.Millisecond)
defer ticker.Stop()
for {
select {
case <-ticker.C:
n, err := f.Read(buf)
if n > 0 {
if _, werr := w.Write(buf[:n]); werr != nil {
return
}
flusher.Flush()
}
if err != nil && err != io.EOF {
return
}
case <-request.Context().Done():
return
}
}
}
}))
}
// tailFileLastLines reads the last n lines from a file and returns them.
// The file position is left at the end of the file after this call.
func tailFileLastLines(f *os.File, n int) []byte {
stat, err := f.Stat()
if err != nil || stat.Size() == 0 {
return nil
}
// Read from the end in chunks to find the last n lines.
const chunkSize = 4096
fileSize := stat.Size()
var lines []byte
offset := fileSize
count := 0
for offset > 0 && count <= n {
readSize := int64(chunkSize)
if readSize > offset {
readSize = offset
}
offset -= readSize
buf := make([]byte, readSize)
nRead, err := f.ReadAt(buf, offset)
if err != nil && err != io.EOF {
break
}
buf = buf[:nRead]
lines = append(buf, lines...)
// Count newlines in this chunk.
for _, b := range buf {
if b == '\n' {
count++
}
}
}
// Trim to last n lines.
idx := 0
nlCount := 0
for i := len(lines) - 1; i >= 0; i-- {
if lines[i] == '\n' {
nlCount++
if nlCount == n+1 {
idx = i + 1
break
}
}
}
lines = lines[idx:]
// Seek to end of file for subsequent reads.
f.Seek(0, io.SeekEnd)
return lines
}
func jsonResponse(next http.Handler) http.Handler {
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,20 @@
//go:build windows
package cli
import (
"testing"
"time"
)
func TestDNSInterceptIgnoredChangeReconcileDueWindowsPreservesImmediateBehavior(t *testing.T) {
p := &prog{}
now := time.Now()
if !p.dnsInterceptIgnoredChangeReconcileDue(now) {
t.Fatal("first ignored Windows change must reconcile immediately")
}
if !p.dnsInterceptIgnoredChangeReconcileDue(now) {
t.Fatal("Windows ignored changes must not inherit the macOS pf rate limit")
}
}
@@ -0,0 +1,319 @@
//go:build windows
package cli
import (
"runtime"
"testing"
"time"
)
// newInterceptTestProg returns a prog with a published intercept state, fake NRPT
// operations already installed, and no WFP engine (engineHandle 0).
//
// The fake is installed here, before anything can inspect registry state, and it is the
// safety boundary - not the empty wfpState. A zero-valued state has owner None, and
// shutdown's None branch sweeps orphaned ctrld rules, so an unfaked stopDNSIntercept would
// reach the production nrptCatchAllRuleExists / removeNRPTCatchAllRule / signalNRPTChange.
// On a host that has ctrld's deterministic key - a developer box, or a CI runner where
// ctrld is installed - that deletes live policy and forces a Group Policy refresh, a
// Dnscache paramchange and a cache flush. A green run on a clean runner proves nothing
// about that.
func newInterceptTestProg(t *testing.T) (*prog, *wfpState, *fakeNRPTOps) {
t.Helper()
f := fakeNRPTOpsForTest(t)
// Prove the fake is in effect before anything can inspect registry state. Asserting
// zero side effects afterwards cannot do that: an uninstalled fake reports zero
// whether it was consulted or bypassed.
requireFakeNRPTOpsInstalled(t, f)
state := &wfpState{stopCh: make(chan struct{}), listenerIP: "127.0.0.1"}
p := &prog{}
p.dnsInterceptState = state
return p, state, f
}
// assertNoNRPTSideEffects fails when a lifecycle path wrote NRPT policy or signalled the
// DNS Client. Every test in this file exercises a guard that is supposed to stand down, so
// any registry write or signal here means the guard did not hold - and, without the fake,
// would have hit the host's real policy.
func assertNoNRPTSideEffects(t *testing.T, f *fakeNRPTOps) {
t.Helper()
add, remove, signal, _ := f.counts()
if add != 0 || remove != 0 || signal != 0 {
t.Errorf("addRule = %d, removeRule = %d, signal = %d, want 0/0/0: this path must not write NRPT policy",
add, remove, signal)
}
if flush := f.flushCount(); flush != 0 {
t.Errorf("flush calls = %d, want 0: this path must not flush the resolver cache", flush)
}
}
// TestStopDNSInterceptRevokesBeforeTeardown pins the ordering the shutdown/monitor race
// depends on. Teardown deletes our WFP sublayer, and a missing sublayer is precisely what
// the health monitor treats as "our filters were wiped, rebuild everything". Were the
// state revoked only after teardown, a monitor tick inside that window would rebuild the
// intercept during shutdown.
func TestStopDNSInterceptRevokesBeforeTeardown(t *testing.T) {
p, state, f := newInterceptTestProg(t)
defer assertNoNRPTSideEffects(t, f)
if p.interceptStateRevoked(state) {
t.Fatal("a freshly published intercept state must not read as retired")
}
if err := p.stopDNSIntercept(); err != nil {
t.Fatalf("stopDNSIntercept() = %v", err)
}
if !p.interceptStateRevoked(state) {
t.Error("state still reads live after shutdown: the monitor and heal flows would keep writing host DNS state")
}
if p.dnsInterceptState != nil {
t.Error("dnsInterceptState survived shutdown")
}
if p.dnsInterceptStopRequested.Load() {
t.Error("stop-requested flag was left set; a later start would see a phantom shutdown")
}
}
// TestRebuildDNSInterceptRefusedAfterShutdown is the regression test for the reported
// race: SCM stop runs resetDNS -> stopDNSIntercept while the health monitor is mid-tick,
// and the monitor then reaches the rebuild path before the process exits. The rebuild
// must refuse - completing it would re-add the NRPT catch-all and the WFP filters moments
// before ctrld disappears, leaving Windows resolving through a listener that is gone.
//
// That refusal is also what keeps this test safe on a real Windows host: a rebuild that
// did not refuse would run startDNSIntercept and write NRPT policy to the machine
// running the tests.
func TestRebuildDNSInterceptRefusedAfterShutdown(t *testing.T) {
p, state, f := newInterceptTestProg(t)
defer assertNoNRPTSideEffects(t, f)
if err := p.stopDNSIntercept(); err != nil {
t.Fatalf("stopDNSIntercept() = %v", err)
}
if got := p.rebuildDNSIntercept(state, "WFP sublayer missing during health check"); got != interceptRebuildRetired {
t.Fatalf("rebuildDNSIntercept() = %v, want interceptRebuildRetired - a post-shutdown rebuild resurrects DNS interception", got)
}
if p.dnsInterceptState != nil {
t.Error("rebuild published new intercept state after shutdown")
}
}
// TestRebuildDNSInterceptRefusedForReplacedState covers the other stale-owner case: an
// earlier rebuild already replaced the state, so a goroutine still holding the old one
// must not tear down its successor.
func TestRebuildDNSInterceptRefusedForReplacedState(t *testing.T) {
p, old, f := newInterceptTestProg(t)
defer assertNoNRPTSideEffects(t, f)
current := &wfpState{stopCh: make(chan struct{}), listenerIP: "127.0.0.1"}
p.dnsInterceptState = current
if got := p.rebuildDNSIntercept(old, "WFP sublayer missing during health check"); got != interceptRebuildRetired {
t.Fatalf("rebuildDNSIntercept() = %v, want interceptRebuildRetired for a superseded state", got)
}
if p.dnsInterceptState != any(current) {
t.Error("a superseded state's rebuild replaced the live intercept")
}
if p.interceptStateRevoked(current) {
t.Error("the live state was revoked by a superseded rebuild")
}
}
// TestRepairMissingWFPStandsDownAfterShutdown checks the monitor's entry point. It must
// not even query WFP for a retired state - the sublayer it looks for is what teardown
// just deleted - and it must tell the monitor goroutine to exit.
func TestRepairMissingWFPStandsDownAfterShutdown(t *testing.T) {
p, state, f := newInterceptTestProg(t)
defer assertNoNRPTSideEffects(t, f)
if err := p.stopDNSIntercept(); err != nil {
t.Fatalf("stopDNSIntercept() = %v", err)
}
// Set the handle only after teardown. A fake handle proves the revocation check
// comes first, but must never reach the real WFP calls in cleanupWFPFilters.
state.engineHandle = 1
if !p.repairMissingWFP(state) {
t.Error("repairMissingWFP() = false after shutdown; the health monitor would keep running for a dead intercept")
}
if p.dnsInterceptState != nil {
t.Error("repairMissingWFP rebuilt the intercept after shutdown")
}
}
// TestPendingStopSignalsRevocation covers how a stop avoids waiting: while it is blocked
// on the lifecycle lock it must already read as revoked, so an in-flight NRPT heal
// abandons its probe backoff instead of making the service stop wait it out. A stop that
// waits too long is killed by the Service Control Manager, which cleans up nothing.
func TestPendingStopSignalsRevocation(t *testing.T) {
p, state, f := newInterceptTestProg(t)
defer assertNoNRPTSideEffects(t, f)
p.dnsInterceptMu.Lock()
stopped := make(chan struct{})
go func() {
defer close(stopped)
_ = p.stopDNSIntercept()
}()
// Wait for the stop to announce itself while it is blocked on the lock.
deadline := time.Now().Add(5 * time.Second)
for !p.dnsInterceptStopRequested.Load() {
if time.Now().After(deadline) {
p.dnsInterceptMu.Unlock()
<-stopped
t.Fatal("stop never announced itself before waiting for the lifecycle lock")
}
runtime.Gosched()
}
if !p.interceptStateRevoked(state) {
t.Error("a pending stop does not read as revoked; the heal flows would keep it waiting")
}
p.dnsInterceptMu.Unlock()
<-stopped
if p.dnsInterceptState != nil {
t.Error("the pending stop did not tear down the intercept once it acquired the lock")
}
}
// TestInterceptWaitAbandonsPromptlyOnPendingStop is the bound on how long a stop can be
// delayed by a recovery flow: the heal sequence's waits add up to tens of seconds, and
// each one must end as soon as a stop is pending.
func TestInterceptWaitAbandonsPromptlyOnPendingStop(t *testing.T) {
p, state, f := newInterceptTestProg(t)
defer assertNoNRPTSideEffects(t, f)
p.dnsInterceptStopRequested.Store(true)
start := time.Now()
if p.interceptWait(state, 30*time.Second) {
t.Fatal("interceptWait() = true with a stop pending; the caller would carry on writing host DNS state")
}
if elapsed := time.Since(start); elapsed > 2*time.Second {
t.Errorf("interceptWait took %v to notice a pending stop; shutdown would inherit that delay", elapsed)
}
}
// TestInterceptWaitRunsToCompletionWhileLive guards the other direction: the cancellable
// wait must still actually wait, or the recovery flows lose their backoff.
func TestInterceptWaitRunsToCompletionWhileLive(t *testing.T) {
p, state, f := newInterceptTestProg(t)
defer assertNoNRPTSideEffects(t, f)
start := time.Now()
if !p.interceptWait(state, 250*time.Millisecond) {
t.Fatal("interceptWait() = false for a live intercept")
}
if elapsed := time.Since(start); elapsed < 250*time.Millisecond {
t.Errorf("interceptWait returned after %v, want at least 250ms", elapsed)
}
}
// TestNRPTNeedsCtrldActivation covers the recovery gap that left a machine unfiltered
// until restart: a failed NRPT write clears ownership, and an owner-None tick used to do
// nothing at all, so nothing ever retried the write.
func TestNRPTNeedsCtrldActivation(t *testing.T) {
tests := []struct {
name string
owner nrptRuleOwner
ruleExists bool
want bool
}{
{
// The reported hole: activation failed, ownership was cleared, and no
// other path re-arms it. In hard mode WFP keeps blocking DNS meanwhile.
name: "no owner retries the failed write",
owner: nrptRuleOwnerNone,
want: true,
},
{
name: "no owner retries even if a rule is somehow present",
owner: nrptRuleOwnerNone,
ruleExists: true,
want: true,
},
{
name: "ctrld-owned rule removed externally is re-added",
owner: nrptRuleOwnerCtrld,
want: true,
},
{
name: "healthy ctrld-owned rule is left alone",
owner: nrptRuleOwnerCtrld,
ruleExists: true,
want: false,
},
{
// Writing beside external policy would be ambiguous policy, not recovery.
name: "external policy is never overwritten",
owner: nrptRuleOwnerGroupPolicy,
want: false,
},
{
name: "external policy is never overwritten even with a ctrld rule present",
owner: nrptRuleOwnerGroupPolicy,
ruleExists: true,
want: false,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := nrptNeedsCtrldActivation(tc.owner, tc.ruleExists); got != tc.want {
t.Errorf("nrptNeedsCtrldActivation(%v, %v) = %v, want %v", tc.owner, tc.ruleExists, got, tc.want)
}
})
}
}
// TestActivateCtrldNRPTFallbackRefusedAfterShutdown guards the worst leftover. A
// catch-all re-added after shutdown points every DNS query on the machine at a listener
// that no longer exists, so nothing resolves at all. Refusing early also keeps this test
// from writing NRPT policy on the machine running it.
func TestActivateCtrldNRPTFallbackRefusedAfterShutdown(t *testing.T) {
p, state, f := newInterceptTestProg(t)
defer assertNoNRPTSideEffects(t, f)
if err := p.stopDNSIntercept(); err != nil {
t.Fatalf("stopDNSIntercept() = %v", err)
}
if p.activateCtrldNRPTFallback(state, "ctrld-owned rule missing during health check") {
t.Error("activateCtrldNRPTFallback() = true after shutdown: the catch-all would outlive ctrld")
}
if owner, _ := state.nrptPolicyOwner(); owner != nrptRuleOwnerNone {
t.Errorf("NRPT owner = %v after a refused fallback, want nrptRuleOwnerNone", owner)
}
}
// TestAllowHandbackAttemptRateLimits covers the throttle on testing an external
// catch-all. Each attempt takes ctrld's rule out of the way for a probe, so a rule that
// never routes would cost a brief DNS outage on every 30s health tick without this - in
// hard mode a window where WFP blocks DNS and nothing redirects it.
func TestHandbackThrottleIsPerRule(t *testing.T) {
state := &wfpState{stopCh: make(chan struct{})}
now := time.Now()
if !state.handbackAllowed(now, "{GP-RULE}", nrptHandbackRetryInterval) {
t.Fatal("first handback attempt must be allowed")
}
// Checking alone must not spend the budget: a pre-probe can still abort the attempt
// without disturbing NRPT, and that must not cost the rule its next window.
if !state.handbackAllowed(now, "{GP-RULE}", nrptHandbackRetryInterval) {
t.Error("handbackAllowed must not consume the budget by itself")
}
state.recordHandbackAttempt(now, "{GP-RULE}", nrptHandbackRetryInterval)
if state.handbackAllowed(now.Add(nrptHandbackRetryInterval-time.Second), "{GP-RULE}", nrptHandbackRetryInterval) {
t.Error("re-testing the same rule inside the interval must be suppressed")
}
// Group Policy alternating between two names must not erase either one's memory:
// with a single slot every swap costs another removal of the live rule.
if !state.handbackAllowed(now.Add(time.Second), "{OTHER-RULE}", nrptHandbackRetryInterval) {
t.Error("a different rule name means the administrator changed policy: test it now")
}
state.recordHandbackAttempt(now.Add(time.Second), "{OTHER-RULE}", nrptHandbackRetryInterval)
if state.handbackAllowed(now.Add(2*time.Second), "{GP-RULE}", nrptHandbackRetryInterval) {
t.Error("testing another rule must not clear the first rule's throttle")
}
if !state.handbackAllowed(now.Add(2*nrptHandbackRetryInterval), "{GP-RULE}", nrptHandbackRetryInterval) {
t.Error("the same rule must be testable again after the interval")
}
}
+51
View File
@@ -0,0 +1,51 @@
//go:build !windows && !darwin
package cli
import (
"fmt"
"time"
)
// startDNSIntercept is not supported on this platform.
// DNS intercept mode is only available on Windows (via WFP) and macOS (via pf).
func (p *prog) startDNSIntercept() error {
return fmt.Errorf("dns intercept: not supported on this platform (only Windows and macOS)")
}
// stopDNSIntercept is a no-op on unsupported platforms.
func (p *prog) stopDNSIntercept() error {
return nil
}
// skipInitialDNSReset is Windows-only; other platforms keep the normal reset.
func (p *prog) skipInitialDNSReset() bool { return false }
// exemptVPNDNSServers is a no-op on unsupported platforms.
func (p *prog) exemptVPNDNSServers(exemptions []vpnDNSExemption) error {
return nil
}
// ensurePFAnchorActive is a no-op on unsupported platforms.
func (p *prog) ensurePFAnchorActive() pfAnchorCheckResult {
return pfAnchorCheckSkipped
}
// checkTunnelInterfaceChanges is a no-op on unsupported platforms.
func (p *prog) checkTunnelInterfaceChanges() bool {
return false
}
func (p *prog) dnsInterceptIgnoredChangeReconcileDue(time.Time) bool {
return false
}
// scheduleDelayedRechecks is a no-op on unsupported platforms.
func (p *prog) scheduleDelayedRechecks() {}
// pfInterceptMonitor is a no-op on unsupported platforms.
func (p *prog) pfInterceptMonitor() {}
// osHealthcheckSuppressed always returns false on non-Windows platforms —
// WFP loopback protect (the trigger for suppression) is Windows-only.
func (p *prog) osHealthcheckSuppressed() bool { return false }
+38
View File
@@ -0,0 +1,38 @@
package cli
import "github.com/Control-D-Inc/ctrld"
var initializeOsResolver = ctrld.InitializeOsResolver
func (p *prog) refreshDNSAfterVPNSettle(reason string) (routes, domainlessServers, exemptions int) {
mainLog.Load().Info().Msgf("DNS intercept: refreshing OS/VPN DNS route state after VPN settle (%s)", reason)
ns := initializeOsResolver(true)
mainLog.Load().Debug().Msgf("DNS intercept: post-settle OS resolver nameservers: %v", ns)
if p.vpnDNS == nil {
mainLog.Load().Debug().Msg("DNS intercept: post-settle VPN DNS route refresh skipped — manager unavailable")
return 0, 0, 0
}
routes, domainlessServers, exemptions = p.vpnDNS.RefreshRoutesOnly()
mainLog.Load().Info().Msgf("DNS intercept: post-settle VPN DNS route refresh completed — %d routes, %d domainless servers, %d exemptions",
routes, domainlessServers, exemptions)
return routes, domainlessServers, exemptions
}
func vpnDNSExemptionsEqual(a, b []vpnDNSExemption) bool {
if len(a) != len(b) {
return false
}
seen := make(map[vpnDNSExemption]int, len(a))
for _, ex := range a {
seen[ex]++
}
for _, ex := range b {
if seen[ex] == 0 {
return false
}
seen[ex]--
}
return true
}
+54
View File
@@ -0,0 +1,54 @@
package cli
import (
"context"
"testing"
"github.com/Control-D-Inc/ctrld"
)
func TestRefreshDNSAfterVPNSettleRefreshesOSResolverAndVPNRoutes(t *testing.T) {
oldInitialize := initializeOsResolver
defer func() { initializeOsResolver = oldInitialize }()
var initialized []bool
initializeOsResolver = func(force bool) []string {
initialized = append(initialized, force)
return []string{"10.102.26.10:53"}
}
var exemptionUpdates [][]vpnDNSExemption
p := &prog{}
p.vpnDNS = newVPNDNSManager(func(exemptions []vpnDNSExemption) error {
exemptionUpdates = append(exemptionUpdates, append([]vpnDNSExemption{}, exemptions...))
return nil
})
p.vpnDNS.discoverVPNDNS = func(context.Context) []ctrld.VPNDNSConfig {
return []ctrld.VPNDNSConfig{{
InterfaceName: "utun4",
Servers: []string{"10.102.26.10"},
Domains: []string{"bmwgroup.net"},
}}
}
routes, domainlessServers, exemptions := p.refreshDNSAfterVPNSettle("test")
if routes != 1 || domainlessServers != 0 || exemptions != 1 {
t.Fatalf("expected 1 route, 0 domainless servers, 1 exemption, got routes=%d domainless=%d exemptions=%d",
routes, domainlessServers, exemptions)
}
if len(initialized) != 1 || !initialized[0] {
t.Fatalf("expected forced OS resolver refresh once, got %v", initialized)
}
if got := p.vpnDNS.UpstreamForDomain("jira.cc.bmwgroup.net."); len(got) != 1 || got[0] != "10.102.26.10" {
t.Fatalf("expected refreshed VPN DNS route, got %v", got)
}
if len(exemptionUpdates) != 1 || len(exemptionUpdates[0]) != 1 || exemptionUpdates[0][0].Server != "10.102.26.10" {
t.Fatalf("expected one serialized pf exemption update for the late VPN DNS server, got %+v", exemptionUpdates)
}
p.refreshDNSAfterVPNSettle("test-repeat")
if len(exemptionUpdates) != 1 {
t.Fatalf("unchanged post-settle VPN DNS state rewrote pf: %+v", exemptionUpdates)
}
}
File diff suppressed because it is too large Load Diff
+571 -45
View File
@@ -104,7 +104,7 @@ func (p *prog) serveDNS(listenerNum string) error {
listenerConfig := p.cfg.Listener[listenerNum]
reqId := requestID()
ctx := context.WithValue(context.Background(), ctrld.ReqIdCtxKey{}, reqId)
if !listenerConfig.AllowWanClients && isWanClient(w.RemoteAddr()) {
if !listenerConfig.AllowWanClients && isWanClient(w.RemoteAddr()) && !isIPv6LoopbackListener(w.LocalAddr()) {
ctrld.Log(ctx, mainLog.Load().Debug(), "query refused, listener does not allow WAN clients: %s", w.RemoteAddr().String())
answer := new(dns.Msg)
answer.SetRcode(m, dns.RcodeRefused)
@@ -126,6 +126,17 @@ func (p *prog) serveDNS(listenerNum string) error {
return
}
// Interception probe: if we're expecting a probe query and this matches,
// signal the prober and respond NXDOMAIN. Used by both macOS pf probes
// (_pf-probe-*) and Windows NRPT probes (_nrpt-probe-*) to verify that
// DNS interception is actually routing queries to ctrld's listener.
if p.signalInterceptProbe(domain) {
answer := new(dns.Msg)
answer.SetRcode(m, dns.RcodeNameError) // NXDOMAIN
_ = w.WriteMsg(answer)
return
}
if _, ok := p.cacheFlushDomainsMap[domain]; ok && p.cache != nil {
p.cache.Purge()
ctrld.Log(ctx, mainLog.Load().Debug(), "received query %q, local cache is purged", domain)
@@ -192,7 +203,7 @@ func (p *prog) serveDNS(listenerNum string) error {
g, ctx := errgroup.WithContext(context.Background())
for _, proto := range []string{"udp", "tcp"} {
proto := proto
if needLocalIPv6Listener() {
if needLocalIPv6Listener(p.cfg.Service.InterceptMode) {
g.Go(func() error {
s, errCh := runDNSServer(net.JoinHostPort("::1", strconv.Itoa(listenerConfig.Port)), proto, handler)
defer s.Shutdown()
@@ -421,6 +432,24 @@ func (p *prog) proxyLanHostnameQuery(ctx context.Context, msg *dns.Msg) *dns.Msg
}
func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
// DNS intercept recovery bypass: forward all queries to OS/DHCP resolver.
// This runs when upstreams are unreachable (e.g., captive portal network)
// and allows the network's DNS to handle authentication pages.
if dnsIntercept && p.recoveryBypass.Load() {
ctrld.Log(ctx, mainLog.Load().Debug(), "Recovery bypass active: forwarding to OS resolver")
resolver, err := ctrld.NewResolver(osUpstreamConfig)
if err == nil {
resolveCtx, cancel := osUpstreamConfig.Context(ctx)
defer cancel()
answer, _ := resolver.Resolve(resolveCtx, req.msg)
if answer != nil {
return &proxyResponse{answer: answer}
}
}
ctrld.Log(ctx, mainLog.Load().Debug(), "OS resolver failed during recovery bypass")
// Fall through to normal flow as last resort
}
var staleAnswer *dns.Msg
upstreams := req.ufr.upstreams
serveStaleCache := p.cache != nil && p.cfg.Service.CacheServeStale
@@ -433,9 +462,9 @@ func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
// However, on Active Directory Domain Controller, where it has local DNS server
// running and listening on local addresses, these local addresses must be used
// as nameservers, so queries for ADDC could be resolved as expected.
if p.isAdDomainQuery(req.msg) {
if p.isAdDomainQuery(req.msg) && p.hasLocalDNS {
ctrld.Log(ctx, mainLog.Load().Debug(),
"AD domain query detected for %s in domain %s",
"AD domain query detected for %s in domain %s, using local DNS server",
req.msg.Question[0].Name, p.adDomain)
upstreamConfigs = []*ctrld.UpstreamConfig{localUpstreamConfig}
upstreams = []string{upstreamOSLocal}
@@ -506,6 +535,128 @@ func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
staleAnswer = answer
}
}
// VPN DNS split routing (only in dns-intercept mode)
if dnsIntercept && p.vpnDNS != nil && len(req.msg.Question) > 0 {
domain := req.msg.Question[0].Name
if vpnServers := p.vpnDNS.UpstreamForDomain(domain); len(vpnServers) > 0 {
ctrld.Log(ctx, mainLog.Load().Debug(), "VPN DNS route matched for domain %s, using servers: %v", domain, vpnServers)
var gotTransportFailure bool
for _, server := range vpnServers {
upstreamConfig := p.vpnDNS.upstreamConfigFor(server)
ctrld.Log(ctx, mainLog.Load().Debug(), "Querying VPN DNS server: %s", server)
dnsResolver, err := ctrld.NewResolver(upstreamConfig)
if err != nil {
ctrld.Log(ctx, mainLog.Load().Error().Err(err), "failed to create VPN DNS resolver")
continue
}
resolveCtx, cancel := upstreamConfig.Context(ctx)
answer, err := dnsResolver.Resolve(resolveCtx, req.msg)
cancel()
if answer != nil {
p.vpnDNS.VPNDNSReachable()
ctrld.Log(ctx, mainLog.Load().Debug(), "VPN DNS query successful")
if p.cache != nil {
ttl := 60 * time.Second
if len(answer.Answer) > 0 {
ttl = time.Duration(answer.Answer[0].Header().Ttl) * time.Second
}
for _, upstream := range upstreams {
p.cache.Add(dnscache.NewKey(req.msg, upstream), dnscache.NewValue(answer, time.Now().Add(ttl)))
}
}
return &proxyResponse{answer: answer}
}
gotTransportFailure = true
ctrld.Log(ctx, mainLog.Load().Debug().Err(err), "VPN DNS server %s failed", server)
}
// Explicit VPN DNS routes are authoritative for their suffix. If all
// routed servers fail at the transport layer while Windows is serving
// retained VPN DNS state, fail closed instead of leaking VPN/internal
// names to normal upstreams.
if gotTransportFailure && p.vpnDNS.ShouldFailClosedAfterVPNDNSTransportFailure(domain, vpnServers) {
ctrld.Log(ctx, mainLog.Load().Debug(),
"All VPN DNS servers had transport failures for %s; returning SERVFAIL while retained VPN DNS state is active", domain)
answer := new(dns.Msg)
answer.SetRcode(req.msg, dns.RcodeServerFailure)
return &proxyResponse{answer: answer}
}
ctrld.Log(ctx, mainLog.Load().Debug(), "All VPN DNS servers failed, falling back to normal upstreams")
}
}
// Domain-less VPN DNS fallback: when a query is going to upstream.os via a
// split-rule (matched policy) and we have VPN DNS servers with no associated
// domains, try those servers for this query. This handles cases like F5 VPN
// where the VPN doesn't advertise DNS search domains but its DNS servers
// know the internal zones referenced by split-rules (e.g., *.provisur.local).
// These servers are NOT used for general OS resolver queries to avoid
// polluting captive portal / DHCP flows.
if dnsIntercept && p.vpnDNS != nil && req.ufr.matched &&
len(upstreams) > 0 && upstreams[0] == upstreamOS &&
len(req.msg.Question) > 0 {
if dlServers := p.vpnDNS.DomainlessServers(); len(dlServers) > 0 {
domain := req.msg.Question[0].Name
ctrld.Log(ctx, mainLog.Load().Debug(),
"Split-rule query %s going to upstream.os, trying %d domain-less VPN DNS servers first: %v",
domain, len(dlServers), dlServers)
var gotDNSAnswer bool
var gotTransportFailure bool
for _, server := range dlServers {
upstreamCfg := p.vpnDNS.upstreamConfigFor(server)
ctrld.Log(ctx, mainLog.Load().Debug(), "Querying domain-less VPN DNS server: %s", server)
dnsResolver, err := ctrld.NewResolver(upstreamCfg)
if err != nil {
ctrld.Log(ctx, mainLog.Load().Error().Err(err), "failed to create domain-less VPN DNS resolver")
continue
}
resolveCtx, cancel := upstreamCfg.Context(ctx)
answer, err := dnsResolver.Resolve(resolveCtx, req.msg)
cancel()
if answer != nil {
gotDNSAnswer = true
p.vpnDNS.VPNDNSReachable()
}
if answer != nil && answer.Rcode == dns.RcodeSuccess {
ctrld.Log(ctx, mainLog.Load().Debug(),
"Domain-less VPN DNS server %s answered %s successfully", server, domain)
return &proxyResponse{answer: answer}
}
if answer != nil {
ctrld.Log(ctx, mainLog.Load().Debug(),
"Domain-less VPN DNS server %s returned %s for %s, trying next",
server, dns.RcodeToString[answer.Rcode], domain)
} else {
gotTransportFailure = true
ctrld.Log(ctx, mainLog.Load().Debug().Err(err),
"Domain-less VPN DNS server %s failed for %s", server, domain)
}
}
// If every domainless VPN DNS attempt failed before receiving a DNS
// packet while Windows is serving retained VPN DNS state, fail closed
// instead of asking LAN/public DNS about internal split-rule names and
// caching false negatives. Reachable negative DNS responses still fall
// through to the old OS fallback behavior below.
if !gotDNSAnswer && gotTransportFailure && p.vpnDNS.ShouldFailClosedAfterVPNDNSTransportFailure(domain, dlServers) {
ctrld.Log(ctx, mainLog.Load().Debug(),
"All domain-less VPN DNS servers had transport failures for %s; returning SERVFAIL while retained VPN DNS state is active", domain)
answer := new(dns.Msg)
answer.SetRcode(req.msg, dns.RcodeServerFailure)
return &proxyResponse{answer: answer}
}
ctrld.Log(ctx, mainLog.Load().Debug(),
"All domain-less VPN DNS servers failed for %s, falling back to OS resolver", domain)
}
}
resolve1 := func(upstream string, upstreamConfig *ctrld.UpstreamConfig, msg *dns.Msg) (*dns.Msg, error) {
ctrld.Log(ctx, mainLog.Load().Debug(), "sending query to %s: %s", upstream, upstreamConfig.Name)
dnsResolver, err := ctrld.NewResolver(upstreamConfig)
@@ -579,6 +730,17 @@ func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
}
continue
}
// Reject an answer whose question does not match the request before it
// can be served or cached. A mismatched question means the upstream
// answered a different name/type than asked; caching it would poison
// the shared cache with wrong-domain records for the requested name.
// See github.com/Control-D-Inc/ctrld/issues/322.
if !sameQuestion(req.msg, answer) {
ctrld.Log(ctx, mainLog.Load().Debug(),
"discarding answer from %s: question mismatch (asked %q, got %q)",
upstreams[n], questionString(req.msg), questionString(answer))
continue
}
// We are doing LAN/PTR lookup using private resolver, so always process next one.
// Except for the last, we want to send response instead of saying all upstream failed.
if answer.Rcode != dns.RcodeSuccess && isLanOrPtrQuery && n != len(upstreamConfigs)-1 {
@@ -623,6 +785,7 @@ func (p *prog) proxy(ctx context.Context, req *proxyRequest) *proxyResponse {
var reason RecoveryReason
if upstreams[0] == upstreamOS {
reason = RecoveryReasonOSFailure
} else {
reason = RecoveryReasonRegularFailure
}
@@ -741,6 +904,33 @@ func containRcode(rcodes []int, rcode int) bool {
return false
}
// sameQuestion reports whether the upstream answer echoes the request's
// question. A well-behaved resolver always copies the question section from
// the query (RFC 1035 section 4.1.2); names are compared case-insensitively
// because DNS names are case-insensitive. A mismatch means the upstream
// answered a different name/type than asked - malformed or malicious - and the
// answer must not be served or cached, or it would poison the shared cache with
// wrong-domain records. See github.com/Control-D-Inc/ctrld/issues/322.
func sameQuestion(req, answer *dns.Msg) bool {
if req == nil || answer == nil {
return false
}
if len(req.Question) == 0 || len(answer.Question) == 0 {
return false
}
rq, aq := req.Question[0], answer.Question[0]
return rq.Qtype == aq.Qtype && rq.Qclass == aq.Qclass && strings.EqualFold(rq.Name, aq.Name)
}
// questionString renders a message's first question as "name/type" for logging.
func questionString(msg *dns.Msg) string {
if msg == nil || len(msg.Question) == 0 {
return "<none>"
}
q := msg.Question[0]
return q.Name + "/" + dns.TypeToString[q.Qtype]
}
func setCachedAnswerTTL(answer *dns.Msg, now, expiredTime time.Time) {
ttlSecs := expiredTime.Sub(now).Seconds()
if ttlSecs < 0 {
@@ -771,10 +961,27 @@ func ttlFromMsg(msg *dns.Msg) uint32 {
return 0
}
func needLocalIPv6Listener() bool {
func needLocalIPv6Listener(interceptMode string) bool {
if !ctrldnet.SupportsIPv6ListenLocal() {
mainLog.Load().Debug().Msg("IPv6 listener: not needed — SupportsIPv6ListenLocal() is false")
return false
}
// On Windows, there's no easy way for disabling/removing IPv6 DNS resolver, so we check whether we can
// listen on ::1, then spawn a listener for receiving DNS requests.
return ctrldnet.SupportsIPv6ListenLocal() && runtime.GOOS == "windows"
if runtime.GOOS == "windows" {
mainLog.Load().Debug().Msg("IPv6 listener: enabled (Windows)")
return true
}
// macOS: IPv6 DNS is blocked at the pf level (not intercepted). The [::1] listener
// is not needed — macOS falls back to IPv4 DNS automatically. See #507 and
// docs/pf-dns-intercept.md for why IPv6 interception on macOS is not feasible
// (sendmsg EINVAL from ::1 to global unicast, nat-on-lo0 doesn't fire for route-to).
if runtime.GOOS == "darwin" {
mainLog.Load().Debug().Msg("IPv6 listener: not needed (macOS — IPv6 DNS blocked at pf, fallback to IPv4)")
return false
}
mainLog.Load().Debug().Str("os", runtime.GOOS).Str("interceptMode", interceptMode).Msg("IPv6 listener: not needed")
return false
}
// ipAndMacFromMsg extracts IP and MAC information included in a DNS message, if any.
@@ -910,13 +1117,36 @@ func (p *prog) getClientInfo(remoteIP string, msg *dns.Msg) *ctrld.ClientInfo {
} else {
ci.Hostname = p.ciTable.LookupHostname(ci.IP, ci.Mac)
}
ci.Self = p.queryFromSelf(ci.IP)
if ci.IP == "" {
mainLog.Load().Debug().Msgf("client info entry with empty IP address: %v", ci)
} else {
ci.Self = p.queryFromSelf(ci.IP)
}
// In DNS intercept mode, ALL queries are from the local machine — pf/WFP
// intercepts outbound DNS and redirects to ctrld. The source IP may be a
// virtual interface (Tailscale, VPN) that has no ARP/MAC entry, causing
// missing x-cd-mac, x-cd-host, and x-cd-os headers. Force Self=true and
// populate from the primary physical interface info.
if dnsIntercept && !ci.Self {
ci.Self = true
}
// If this is a query from self, but ci.IP is not loopback IP,
// try using hostname mapping for lookback IP if presents.
if ci.Self {
if name := p.ciTable.LocalHostname(); name != "" {
ci.Hostname = name
}
// If MAC is still empty (e.g., query arrived via virtual interface IP
// like Tailscale), fall back to the loopback MAC mapping which addSelf()
// populates from the primary physical interface.
if ci.Mac == "" {
if mac := p.ciTable.LookupMac("127.0.0.1"); mac != "" {
ci.Mac = mac
}
}
}
p.spoofLoopbackIpInClientInfo(ci)
return ci
@@ -954,7 +1184,13 @@ func (p *prog) doSelfUninstall(answer *dns.Msg) {
logger := mainLog.Load().With().Str("mode", "self-uninstall").Logger()
if p.refusedQueryCount > selfUninstallMaxQueries {
p.checkingSelfUninstall = true
_, err := controld.FetchResolverConfig(cdUID, rootCmd.Version, cdDev)
req := &controld.ResolverConfigRequest{
RawUID: cdUID,
Version: rootCmd.Version,
Metadata: ctrld.SystemMetadataRuntime(context.Background()),
}
_, err := controld.FetchResolverConfig(context.Background(), req, cdDev)
logger.Debug().Msg("maximum number of refused queries reached, checking device status")
selfUninstallCheck(err, p, logger)
@@ -1020,7 +1256,12 @@ func (p *prog) queryFromSelf(ip string) bool {
if val, ok := p.queryFromSelfMap.Load(ip); ok {
return val.(bool)
}
netIP := netip.MustParseAddr(ip)
netIP, err := netip.ParseAddr(ip)
if err != nil {
mainLog.Load().Debug().Err(err).Msgf("could not parse IP: %q", ip)
return false
}
regularIPs, loopbackIPs, err := netmon.LocalAddresses()
if err != nil {
mainLog.Load().Warn().Err(err).Msg("could not get local addresses")
@@ -1090,7 +1331,8 @@ func isPrivatePtrLookup(m *dns.Msg) bool {
return addr.IsPrivate() ||
addr.IsLoopback() ||
addr.IsLinkLocalUnicast() ||
tsaddr.CGNATRange().Contains(addr)
tsaddr.CGNATRange().Contains(addr) ||
isServiceContinuityAddr(addr)
}
}
return false
@@ -1128,6 +1370,20 @@ func isLanHostname(name string) bool {
strings.HasSuffix(name, ".local")
}
// ipv4ServiceContinuityPrefix is the RFC 7335 IPv4 Service Continuity Prefix
// (192.0.0.0/29), used by the CLAT in 464XLAT/DS-Lite transition setups. On such
// networks (common on IPv6-only cellular carriers and iPhone hotspots) the local
// machine's DNS queries reach ctrld with a source in this range (e.g. 192.0.0.2),
// so they must be treated as local, not WAN. Go's netip.IsPrivate does not cover
// this range — the same reason the CGNAT range is special-cased below. See #552.
var ipv4ServiceContinuityPrefix = netip.MustParsePrefix("192.0.0.0/29")
// isServiceContinuityAddr reports whether ip is in the RFC 7335 IPv4 Service
// Continuity Prefix (464XLAT/DS-Lite CLAT).
func isServiceContinuityAddr(ip netip.Addr) bool {
return ipv4ServiceContinuityPrefix.Contains(ip)
}
// isWanClient reports whether the input is a WAN address.
func isWanClient(na net.Addr) bool {
var ip netip.Addr
@@ -1138,7 +1394,20 @@ func isWanClient(na net.Addr) bool {
!ip.IsPrivate() &&
!ip.IsLinkLocalUnicast() &&
!ip.IsLinkLocalMulticast() &&
!tsaddr.CGNATRange().Contains(ip)
!tsaddr.CGNATRange().Contains(ip) &&
!isServiceContinuityAddr(ip)
}
// isIPv6LoopbackListener reports whether the listener address is [::1].
// The [::1] listener only serves locally-redirected traffic (via pf on macOS
// or system DNS on Windows), so queries arriving on it are always from this
// machine — even when the source IP is a global IPv6 address (pf preserves the
// original source IP during rdr).
func isIPv6LoopbackListener(na net.Addr) bool {
if ap, err := netip.ParseAddrPort(na.String()); err == nil {
return ap.Addr() == netip.IPv6Loopback()
}
return false
}
// resolveInternalDomainTestQuery resolves internal test domain query, returning the answer to the caller.
@@ -1266,6 +1535,14 @@ func (p *prog) monitorNetworkChanges() error {
mainLog.Load().Debug().Msg("Ignoring interface change - no valid interfaces affected")
// check if the default IPs are still on an interface that is up
ValidateDefaultLocalIPsFromDelta(delta.New)
// Minor interface changes can still accompany pf/WFP or VPN DNS changes.
// On macOS, bound the immediate full reconciliation so link-local-only
// notification storms do not run pfctl/scutil work for every event.
// Windows keeps the existing immediate behavior. Tunnel changes always
// bypass the macOS limit, and delayed checks provide a trailing refresh.
if dnsIntercept && p.dnsInterceptState != nil {
p.handleDNSInterceptIgnoredNetworkChange(delta, time.Now())
}
return
}
@@ -1274,6 +1551,11 @@ func (p *prog) monitorNetworkChanges() error {
return
}
mainLog.Load().Debug().Msg("Link state changed, re-bootstrapping")
for _, uc := range p.cfg.Upstream {
uc.ReBootstrap()
}
// Get IPs from default route interface in new state
selfIP := defaultRouteIP()
@@ -1332,7 +1614,27 @@ func (p *prog) monitorNetworkChanges() error {
// we only trigger recovery flow for network changes on non router devices
if router.Name() == "" {
p.handleRecovery(RecoveryReasonNetworkChange)
p.debounceRecovery()
}
// After network changes, verify our pf anchor is still active and
// refresh VPN DNS state. Order matters: tunnel checks first (may rebuild
// anchor), then VPN DNS refresh (updates exemptions in anchor), then
// delayed re-checks for async VPN teardown.
if dnsIntercept && p.dnsInterceptState != nil {
if !p.pfStabilizing.Load() {
p.ensurePFAnchorActive()
}
// Check tunnel interfaces unconditionally — it decides internally
// whether to enter stabilization or rebuild immediately.
p.checkTunnelInterfaceChanges()
// Refresh VPN DNS routes — runs after tunnel checks so the anchor
// rebuild includes current VPN DNS exemptions.
if p.vpnDNS != nil {
p.vpnDNS.Refresh(true)
}
// Schedule delayed re-checks to catch async VPN teardown changes.
p.scheduleDelayedRechecks()
}
})
@@ -1341,6 +1643,76 @@ func (p *prog) monitorNetworkChanges() error {
return nil
}
// handleDNSInterceptIgnoredNetworkChange runs the DNS-intercept work for a
// network delta that did not affect a usable interface. Keeping this path in a
// method lets tests exercise the callback wiring with synthetic deltas.
func (p *prog) handleDNSInterceptIgnoredNetworkChange(delta *netmon.ChangeDelta, now time.Time) {
reconcileNow := false
// Stabilization owns PF repair. Do not consume the next leading-edge slot
// until an ignored delta can actually perform the corresponding PF check.
if !p.pfStabilizing.Load() {
reconcileNow = p.dnsInterceptIgnoredChangeReconcileDue(now)
if reconcileNow {
p.ensurePFAnchorActive()
}
}
// Check tunnel interfaces unconditionally — it decides internally whether
// to enter stabilization or rebuild immediately.
tunnelChanged := p.checkTunnelInterfaceChanges()
// Schedule delayed re-checks to catch async VPN teardown changes. These also
// refresh the OS resolver and VPN DNS routes.
p.scheduleDelayedRechecks()
// Detect interface appearance/disappearance — hypervisors (Parallels,
// VMware, VirtualBox) reload pf when creating/destroying virtual network
// interfaces, which can corrupt pf's internal translation state. The rdr
// rules survive in text form (watchdog says "intact") but stop evaluating.
// Spawn an async monitor that probes pf interception with backoff and forces
// a full pf reload if broken.
if delta.Old != nil {
interfaceChanged := false
var changedIface string
for ifaceName := range delta.Old.Interface {
if ifaceName == "lo0" {
continue
}
if _, exists := delta.New.Interface[ifaceName]; !exists {
interfaceChanged = true
changedIface = ifaceName
break
}
}
if !interfaceChanged {
for ifaceName := range delta.New.Interface {
if ifaceName == "lo0" {
continue
}
if _, exists := delta.Old.Interface[ifaceName]; !exists {
interfaceChanged = true
changedIface = ifaceName
break
}
}
}
if interfaceChanged {
mainLog.Load().Info().Str("interface", changedIface).
Msg("DNS intercept: interface appeared/disappeared — starting interception probe monitor")
go p.pfInterceptMonitor()
}
}
// Refresh VPN DNS immediately for real tunnel changes even when the periodic
// ignored-change reconciliation is currently rate-limited - but not while
// stabilization owns pf. A refresh rebuilds the anchor, and these deltas arrive
// exactly when a VPN is bringing its own ruleset up, which is the collision
// stabilization is there to prevent. checkTunnelInterfaceChanges keeps the
// observation pending, so the transition is retried rather than dropped.
if p.vpnDNS != nil && (reconcileNow || tunnelChanged) && !p.pfStabilizing.Load() {
p.vpnDNS.Refresh(true)
}
}
// interfaceStatesEqual compares two interface states
func interfaceStatesEqual(a, b *netmon.Interface) bool {
if a == nil || b == nil {
@@ -1389,6 +1761,8 @@ func interfaceIPsEqual(a, b []netip.Prefix) bool {
return true
}
var errOsHealthcheckSuppressed = errors.New("upstream os health check suppressed")
// checkUpstreamOnce sends a test query to the specified upstream.
// Returns nil if the upstream responds successfully.
func (p *prog) checkUpstreamOnce(upstream string, uc *ctrld.UpstreamConfig) error {
@@ -1418,11 +1792,57 @@ func (p *prog) checkUpstreamOnce(upstream string, uc *ctrld.UpstreamConfig) erro
duration := time.Since(start)
if err != nil {
// Demote upstream.os check failures to debug while WFP loopback
// protect is active: an external WFP block filter is interfering
// with plain DNS so repeated failures here are expected. Other
// upstreams keep error level so real outages stay visible.
if upstream == upstreamOS && p.osHealthcheckSuppressed() {
mainLog.Load().Debug().Err(err).Msgf("Upstream %s check failed after %v (WFP loopback protect active)", upstream, duration)
return errOsHealthcheckSuppressed
}
// A no-route/network-unreachable failure means the endpoint's address
// family is available locally but unroutable (e.g. an IPv6 DoH endpoint
// while IPv6 is up but has no route). These repeat until the route
// returns and are handled by bounded backoff in the recovery loop, so
// keep them at debug to avoid sustained error-log spam.
if ctrldnet.IsUnreachable(err) {
mainLog.Load().Debug().Err(err).Msgf("Upstream %s check failed after %v (network unreachable)", upstream, duration)
return err
}
mainLog.Load().Error().Err(err).Msgf("Upstream %s check failed after %v", upstream, duration)
} else {
mainLog.Load().Debug().Msgf("Upstream %s responded successfully in %v", upstream, duration)
return err
}
return err
mainLog.Load().Debug().Msgf("Upstream %s responded successfully in %v", upstream, duration)
return nil
}
// recoveryDebounceWindow is the time to wait after the last network change
// before triggering handleRecovery. This coalesces rapid consecutive network
// changes (e.g., hotspot→LAN causing en1 drop + en0 pickup + en1 re-pickup)
// into a single recovery pass, avoiding the cancel-and-restart race that
// leaves DoH transports in a stale state.
const recoveryDebounceWindow = 500 * time.Millisecond
// debounceRecovery schedules a handleRecovery(NetworkChange) call after a debounce
// window. If called again before the window expires, the timer is reset so that
// recovery runs once with the final network state. All other state updates (IP,
// pf anchor, VPN DNS, tunnel checks) run immediately — only the recovery flow
// with its upstream probing and DHCP bypass logic is debounced.
func (p *prog) debounceRecovery() {
p.recoveryDebounceMu.Lock()
defer p.recoveryDebounceMu.Unlock()
if p.recoveryDebounceTimer != nil {
p.recoveryDebounceTimer.Stop()
mainLog.Load().Debug().Msg("Recovery debounce: resetting timer (rapid network change)")
}
p.recoveryDebounceTimer = time.AfterFunc(recoveryDebounceWindow, func() {
p.recoveryDebounceMu.Lock()
p.recoveryDebounceTimer = nil
p.recoveryDebounceMu.Unlock()
p.handleRecovery(RecoveryReasonNetworkChange)
})
mainLog.Load().Debug().Msg("Recovery debounce: scheduled (500ms window)")
}
// handleRecovery performs a unified recovery by removing DNS settings,
@@ -1452,28 +1872,78 @@ func (p *prog) handleRecovery(reason RecoveryReason) {
p.recoveryCancelMu.Unlock()
}
// For network changes, force-reset all upstream transports synchronously.
// The lazy ReBootstrap() called earlier in the network change callback only
// sets a flag — the old transport's dead connections can still be used by
// recovery probes, causing context deadline timeouts. ForceReBootstrap()
// closes old connections and creates fresh transports so probes succeed on
// first attempt.
if reason == RecoveryReasonNetworkChange {
for _, uc := range p.cfg.Upstream {
if uc != nil {
uc.ForceReBootstrap()
}
}
mainLog.Load().Info().Msg("Force-reset upstream transports for network change recovery")
}
// Create a new recovery context without a fixed timeout.
p.recoveryCancelMu.Lock()
recoveryCtx, cancel := context.WithCancel(context.Background())
p.recoveryCancel = cancel
p.recoveryCancelMu.Unlock()
// Immediately remove our DNS settings from the interface.
// set recoveryRunning to true to prevent watchdogs from putting the listener back on the interface
p.recoveryRunning.Store(true)
// we do not want to restore any static DNS settings
// we must try to get the DHCP values, any static DNS settings
// will be appended to nameservers from the saved interface values
p.resetDNS(false, false)
// For an OS failure, reinitialize OS resolver nameservers immediately.
if reason == RecoveryReasonOSFailure {
mainLog.Load().Debug().Msg("OS resolver failure detected; reinitializing OS resolver nameservers")
ns := ctrld.InitializeOsResolver(true)
if len(ns) == 0 {
mainLog.Load().Warn().Msg("No nameservers found for OS resolver; using existing values")
// In DNS intercept mode, don't tear down WFP/pf filters.
// Instead, enable recovery bypass so proxy() forwards queries to
// the OS/DHCP resolver. This handles captive portal authentication
// without the overhead of filter teardown/rebuild.
if dnsIntercept && p.dnsInterceptState != nil {
p.recoveryBypass.Store(true)
mainLog.Load().Info().Msg("DNS intercept recovery: enabling DHCP bypass (filters stay active)")
// Reinitialize OS resolver to discover DHCP servers on the new network.
mainLog.Load().Debug().Msg("DNS intercept recovery: discovering DHCP nameservers")
dhcpServers := ctrld.InitializeOsResolver(true)
if len(dhcpServers) == 0 {
mainLog.Load().Warn().Msg("DNS intercept recovery: no DHCP nameservers found")
} else {
mainLog.Load().Info().Msgf("Reinitialized OS resolver with nameservers: %v", ns)
mainLog.Load().Info().Msgf("DNS intercept recovery: found DHCP nameservers: %v", dhcpServers)
}
// Exempt DHCP nameservers from intercept filters so the OS resolver
// can actually reach them on port 53.
if len(dhcpServers) > 0 {
// Build exemptions without an Interface — DHCP servers are not VPN-specific,
// so they only generate group-scoped pf rules (ctrld process only).
exemptions := make([]vpnDNSExemption, 0, len(dhcpServers))
for _, s := range dhcpServers {
host := s
if h, _, err := net.SplitHostPort(s); err == nil {
host = h
}
exemptions = append(exemptions, vpnDNSExemption{Server: host})
}
mainLog.Load().Info().Msgf("DNS intercept recovery: exempting DHCP nameservers from filters: %v", exemptions)
if err := p.exemptVPNDNSServers(exemptions); err != nil {
mainLog.Load().Warn().Err(err).Msg("DNS intercept recovery: failed to exempt DHCP nameservers — recovery queries may fail")
}
}
} else {
// Traditional flow: remove DNS settings to expose DHCP nameservers
p.resetDNS(false, false)
// For an OS failure, reinitialize OS resolver nameservers immediately.
if reason == RecoveryReasonOSFailure {
mainLog.Load().Debug().Msg("OS resolver failure detected; reinitializing OS resolver nameservers")
ns := ctrld.InitializeOsResolver(true)
if len(ns) == 0 {
mainLog.Load().Warn().Msg("No nameservers found for OS resolver; using existing values")
} else {
mainLog.Load().Info().Msgf("Reinitialized OS resolver with nameservers: %v", ns)
}
}
}
@@ -1494,23 +1964,46 @@ func (p *prog) handleRecovery(reason RecoveryReason) {
// reset the upstream failure count and down state
p.um.reset(recovered)
// For network changes we also reinitialize the OS resolver.
if reason == RecoveryReasonNetworkChange {
ns := ctrld.InitializeOsResolver(true)
if len(ns) == 0 {
mainLog.Load().Warn().Msg("No nameservers found for OS resolver during network-change recovery; using existing values")
} else {
mainLog.Load().Info().Msgf("Reinitialized OS resolver with nameservers: %v", ns)
// In DNS intercept mode, just disable the bypass — filters are still active.
if dnsIntercept && p.dnsInterceptState != nil {
p.recoveryBypass.Store(false)
mainLog.Load().Info().Msg("DNS intercept recovery complete: disabling DHCP bypass, resuming normal flow")
// Refresh VPN DNS routes in case VPN state changed during recovery.
if p.vpnDNS != nil {
p.vpnDNS.Refresh(true)
}
// Reinitialize OS resolver for the recovered state.
if reason == RecoveryReasonNetworkChange {
ns := ctrld.InitializeOsResolver(true)
if len(ns) == 0 {
mainLog.Load().Warn().Msg("No nameservers found for OS resolver during network-change recovery; using existing values")
} else {
mainLog.Load().Info().Msgf("Reinitialized OS resolver with nameservers: %v", ns)
}
}
p.recoveryRunning.Store(false)
} else {
// For network changes we also reinitialize the OS resolver.
if reason == RecoveryReasonNetworkChange {
ns := ctrld.InitializeOsResolver(true)
if len(ns) == 0 {
mainLog.Load().Warn().Msg("No nameservers found for OS resolver during network-change recovery; using existing values")
} else {
mainLog.Load().Info().Msgf("Reinitialized OS resolver with nameservers: %v", ns)
}
}
// Apply our DNS settings back and log the interface state.
p.setDNS()
p.logInterfacesState()
// allow watchdogs to put the listener back on the interface if its changed for any reason
p.recoveryRunning.Store(false)
}
// Apply our DNS settings back and log the interface state.
p.setDNS()
p.logInterfacesState()
// allow watchdogs to put the listener back on the interface if its changed for any reason
p.recoveryRunning.Store(false)
// Clear the recovery cancellation for a clean slate.
p.recoveryCancelMu.Lock()
p.recoveryCancel = nil
@@ -1520,6 +2013,9 @@ func (p *prog) handleRecovery(reason RecoveryReason) {
// waitForUpstreamRecovery checks the provided upstreams concurrently until one recovers.
// It returns the name of the recovered upstream or an error if the check times out.
func (p *prog) waitForUpstreamRecovery(ctx context.Context, upstreams map[string]*ctrld.UpstreamConfig) (string, error) {
recoveryCtx, cancel := context.WithCancel(ctx)
defer cancel()
recoveredCh := make(chan string, 1)
var wg sync.WaitGroup
@@ -1531,26 +2027,45 @@ func (p *prog) waitForUpstreamRecovery(ctx context.Context, upstreams map[string
defer wg.Done()
mainLog.Load().Debug().Msgf("Starting recovery check loop for upstream: %s", name)
attempts := 0
unreachableStreak := 0
for {
select {
case <-ctx.Done():
case <-recoveryCtx.Done():
mainLog.Load().Debug().Msgf("Context canceled for upstream %s", name)
return
default:
attempts++
// checkUpstreamOnce will reset any failure counters on success.
if err := p.checkUpstreamOnce(name, uc); err == nil {
err := p.checkUpstreamOnce(name, uc)
if err == nil || errors.Is(err, errOsHealthcheckSuppressed) {
mainLog.Load().Debug().Msgf("Upstream %s recovered successfully", name)
select {
case recoveredCh <- name:
mainLog.Load().Debug().Msgf("Sent recovery notification for upstream %s", name)
cancel()
default:
mainLog.Load().Debug().Msg("Recovery channel full, another upstream already recovered")
}
return
}
mainLog.Load().Debug().Msgf("Upstream %s check failed, sleeping before retry", name)
time.Sleep(checkUpstreamBackoffSleep)
// Back off the retry cadence for an unroutable endpoint so a
// host with IPv6 up but no route to the IPv6 DoH endpoint does
// not re-bootstrap/re-check every checkUpstreamBackoffSleep and
// spam the log. The backoff is bounded (checkUpstreamUnreachableBackoffMax)
// so the endpoint is still re-probed and recovers when the route
// returns; any other failure resets to the base cadence.
sleep := checkUpstreamBackoffSleep
if ctrldnet.IsUnreachable(err) {
unreachableStreak++
sleep = unreachableRecoveryBackoff(unreachableStreak)
mainLog.Load().Debug().Msgf("Upstream %s unreachable (streak %d), backing off %s before retry", name, unreachableStreak, sleep)
} else {
unreachableStreak = 0
mainLog.Load().Debug().Msgf("Upstream %s check failed, sleeping before retry", name)
}
if !sleepWithContext(recoveryCtx, sleep) {
return
}
// if this is the upstreamOS and it's the 3rd attempt (or multiple of 3),
// we should try to reinit the OS resolver to ensure we can recover
@@ -1578,6 +2093,17 @@ func (p *prog) waitForUpstreamRecovery(ctx context.Context, upstreams map[string
return recovered, nil
}
func sleepWithContext(ctx context.Context, d time.Duration) bool {
timer := time.NewTimer(d)
defer timer.Stop()
select {
case <-timer.C:
return true
case <-ctx.Done():
return false
}
}
// buildRecoveryUpstreams constructs the map of upstream configurations to test.
// For OS failures we supply the manual OS resolver upstream configuration.
// For network change or regular failure we use the upstreams defined in p.cfg (ignoring OS).
+59 -12
View File
@@ -22,15 +22,15 @@ func Test_wildcardMatches(t *testing.T) {
domain string
match bool
}{
{"domain - prefix parent should not match", "*.windscribe.com", "windscribe.com", false},
{"domain - prefix", "*.windscribe.com", "anything.windscribe.com", true},
{"domain - prefix not match other s", "*.windscribe.com", "example.com", false},
{"domain - prefix not match s in name", "*.windscribe.com", "wwindscribe.com", false},
{"domain - suffix", "suffix.*", "suffix.windscribe.com", true},
{"domain - suffix not match other", "suffix.*", "suffix1.windscribe.com", false},
{"domain - both", "suffix.*.windscribe.com", "suffix.anything.windscribe.com", true},
{"domain - both not match", "suffix.*.windscribe.com", "suffix1.suffix.windscribe.com", false},
{"domain - case-insensitive", "*.WINDSCRIBE.com", "anything.windscribe.com", true},
{"domain - prefix parent should not match", "*.example.com", "example.com", false},
{"domain - prefix", "*.example.com", "anything.example.com", true},
{"domain - prefix not match other s", "*.example.com", "other.org", false},
{"domain - prefix not match s in name", "*.example.com", "eexample.com", false},
{"domain - suffix", "suffix.*", "suffix.example.com", true},
{"domain - suffix not match other", "suffix.*", "suffix1.example.com", false},
{"domain - both", "suffix.*.example.com", "suffix.anything.example.com", true},
{"domain - both not match", "suffix.*.example.com", "suffix1.suffix.example.com", false},
{"domain - case-insensitive", "*.EXAMPLE.com", "anything.example.com", true},
{"mac - prefix", "*:98:05:b4:2b", "d4:67:98:05:b4:2b", true},
{"mac - prefix not match other s", "*:98:05:b4:2b", "0d:ba:54:09:94:2c", false},
{"mac - prefix not match s in name", "*:98:05:b4:2b", "e4:67:97:05:b4:2b", false},
@@ -57,9 +57,9 @@ func Test_canonicalName(t *testing.T) {
domain string
canonical string
}{
{"fqdn to canonical", "windscribe.com.", "windscribe.com"},
{"already canonical", "windscribe.com", "windscribe.com"},
{"case insensitive", "Windscribe.Com.", "windscribe.com"},
{"fqdn to canonical", "example.com.", "example.com"},
{"already canonical", "example.com", "example.com"},
{"case insensitive", "Example.Com.", "example.com"},
}
for _, tc := range tests {
@@ -405,6 +405,8 @@ func Test_isPrivatePtrLookup(t *testing.T) {
{"CGNAT", newDnsMsgPtr("100.66.27.28", t), true},
{"Loopback", newDnsMsgPtr("127.0.0.1", t), true},
{"Link Local Unicast", newDnsMsgPtr("fe80::69f6:e16e:8bdb:433f", t), true},
// RFC 7335 IPv4 Service Continuity Prefix (464XLAT/DS-Lite CLAT), see #552.
{"464XLAT CLAT host", newDnsMsgPtr("192.0.0.2", t), true},
{"Public IP", newDnsMsgPtr("8.8.8.8", t), false},
}
for _, tc := range tests {
@@ -452,6 +454,11 @@ func Test_isWanClient(t *testing.T) {
{"CGNAT", &net.UDPAddr{IP: net.ParseIP("100.66.27.28")}, false},
{"Loopback", &net.UDPAddr{IP: net.ParseIP("127.0.0.1")}, false},
{"Link Local Unicast", &net.UDPAddr{IP: net.ParseIP("fe80::69f6:e16e:8bdb:433f")}, false},
// RFC 7335 IPv4 Service Continuity Prefix (464XLAT/DS-Lite CLAT), see #552.
{"464XLAT PLAT side", &net.UDPAddr{IP: net.ParseIP("192.0.0.1")}, false},
{"464XLAT CLAT host", &net.UDPAddr{IP: net.ParseIP("192.0.0.2")}, false},
// Outside the /29 but inside 192.0.0.0/24: still WAN (fix is scoped to /29).
{"192.0.0.0/24 outside /29", &net.UDPAddr{IP: net.ParseIP("192.0.0.100")}, true},
{"Public", &net.UDPAddr{IP: net.ParseIP("8.8.8.8")}, true},
}
for _, tc := range tests {
@@ -464,3 +471,43 @@ func Test_isWanClient(t *testing.T) {
})
}
}
func Test_prog_queryFromSelf(t *testing.T) {
p := &prog{}
require.NotPanics(t, func() {
p.queryFromSelf("")
})
require.NotPanics(t, func() {
p.queryFromSelf("foo")
})
}
func Test_sameQuestion(t *testing.T) {
mk := func(name string, qtype uint16) *dns.Msg {
m := new(dns.Msg)
m.SetQuestion(name, qtype)
return m
}
tests := []struct {
name string
req *dns.Msg
answer *dns.Msg
want bool
}{
{"identical", mk("example.com.", dns.TypeA), mk("example.com.", dns.TypeA), true},
{"case insensitive", mk("Example.COM.", dns.TypeA), mk("example.com.", dns.TypeA), true},
{"different name", mk("victim.example.", dns.TypeA), mk("attacker.example.", dns.TypeA), false},
{"different type", mk("example.com.", dns.TypeA), mk("example.com.", dns.TypeAAAA), false},
{"nil req", nil, mk("example.com.", dns.TypeA), false},
{"nil answer", mk("example.com.", dns.TypeA), nil, false},
{"empty answer question", mk("example.com.", dns.TypeA), new(dns.Msg), false},
}
for _, tc := range tests {
tc := tc
t.Run(tc.name, func(t *testing.T) {
if got := sameQuestion(tc.req, tc.answer); got != tc.want {
t.Errorf("sameQuestion() = %v, want %v", got, tc.want)
}
})
}
}
+38
View File
@@ -0,0 +1,38 @@
package cli
import (
"testing"
"github.com/Control-D-Inc/ctrld"
)
func TestUpdateConfigInterceptMode(t *testing.T) {
tests := []struct {
name string
current string
mode string
want string
wantUpdated bool
}{
{name: "empty flag preserves config", current: "dns", mode: "", want: "dns"},
{name: "dns is persisted", mode: "dns", want: "dns", wantUpdated: true},
{name: "hard is persisted", current: "dns", mode: "hard", want: "hard", wantUpdated: true},
{name: "off clears persisted mode", current: "dns", mode: "off", want: "", wantUpdated: true},
{name: "off is idempotent", mode: "off", want: ""},
{name: "invalid flag preserves config", current: "hard", mode: "invalid", want: "hard"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
cfg := &ctrld.Config{}
cfg.Service.InterceptMode = tc.current
updated := updateConfigInterceptMode(cfg, tc.mode)
if updated != tc.wantUpdated {
t.Fatalf("updateConfigInterceptMode() updated = %v, want %v", updated, tc.wantUpdated)
}
if cfg.Service.InterceptMode != tc.want {
t.Fatalf("service.intercept_mode = %q, want %q", cfg.Service.InterceptMode, tc.want)
}
})
}
}
+68
View File
@@ -0,0 +1,68 @@
package cli
// Interception probe registry.
//
// A probe sends a DNS query for a unique synthetic domain through the OS resolver and
// waits for ctrld's own handler to receive it. That is the only way to tell "the rules are
// present" from "the rules are actually redirecting packets", and both the macOS pf path
// and the Windows NRPT path use it.
//
// Each attempt registers its own domain, so overlapping probes cannot cancel each other,
// and deregistration only removes the entry it owns.
// registerInterceptProbe registers domain and returns the channel it will be signalled on
// plus the function that removes the registration.
//
//lint:ignore U1000 used on darwin (pf probes) and windows (NRPT probes)
func (p *prog) registerInterceptProbe(domain string) (<-chan struct{}, func()) {
ch := make(chan struct{}, 1)
p.interceptProbeMu.Lock()
current, _ := p.interceptProbes.Load().(map[string]chan struct{})
next := make(map[string]chan struct{}, len(current)+1)
for k, v := range current {
next[k] = v
}
next[domain] = ch
p.interceptProbes.Store(next)
p.interceptProbeMu.Unlock()
return ch, func() {
p.interceptProbeMu.Lock()
defer p.interceptProbeMu.Unlock()
current, _ := p.interceptProbes.Load().(map[string]chan struct{})
// Only drop the entry while it is still this attempt's channel. A later probe
// that reused the domain owns the slot now, and clearing it would make that one
// wait out its timeout for a query it already received.
if existing, ok := current[domain]; !ok || existing != ch {
return
}
next := make(map[string]chan struct{}, len(current))
for k, v := range current {
if k != domain {
next[k] = v
}
}
p.interceptProbes.Store(next)
}
}
// signalInterceptProbe reports whether domain is a pending probe, signalling its waiter
// when it is. Called from the DNS handler for every query, so the common case is a nil or
// empty map and no allocation.
func (p *prog) signalInterceptProbe(domain string) bool {
probes, _ := p.interceptProbes.Load().(map[string]chan struct{})
if len(probes) == 0 {
return false
}
ch, ok := probes[domain]
if !ok {
return false
}
select {
case ch <- struct{}{}:
default:
// Buffered channel already holds a signal: the waiter has what it needs.
}
return true
}
+17 -6
View File
@@ -50,8 +50,13 @@ func httpClientWithFallback(timeout time.Duration) *http.Client {
// doWithRetry performs an HTTP request with retries
func doWithRetry(req *http.Request, maxRetries int, ip string) (*http.Response, error) {
return doWithRetryClient(httpClientWithFallback(defaultHTTPTimeout), req, maxRetries, ip)
}
// doWithRetryClient is doWithRetry with an injectable client, so the retry and
// error-composition behaviour can be tested without real network access.
func doWithRetryClient(client *http.Client, req *http.Request, maxRetries int, ip string) (*http.Response, error) {
var lastErr error
client := httpClientWithFallback(defaultHTTPTimeout)
var ipReq *http.Request
if ip != "" {
ipReq = req.Clone(req.Context())
@@ -67,22 +72,28 @@ func doWithRetry(req *http.Request, maxRetries int, ip string) (*http.Response,
if err == nil {
return resp, nil
}
// Keep the hostname attempt's error: it carries the diagnosis (on Windows,
// a local firewall denying the socket shows up here as WSAEACCES), while the
// direct-IP fallback often fails for an unrelated reason such as an
// unreachable IPv6 route.
attemptErr := err
if ipReq != nil {
mainLog.Load().Warn().Err(err).Msgf("dial to %q failed", req.Host)
mainLog.Load().Warn().Msgf("fallback to direct IP to download prod version: %q", ip)
resp, err = client.Do(ipReq)
if err == nil {
resp, fallbackErr := client.Do(ipReq)
if fallbackErr == nil {
return resp, nil
}
attemptErr = fmt.Errorf("%w; fallback to direct ip %s failed: %w", attemptErr, ip, fallbackErr)
}
lastErr = err
mainLog.Load().Debug().Err(err).
lastErr = attemptErr
mainLog.Load().Debug().Err(attemptErr).
Str("method", req.Method).
Str("url", req.URL.String()).
Msgf("HTTP request attempt %d/%d failed", attempt+1, maxRetries)
}
return nil, fmt.Errorf("failed after %d attempts to %s %s: %v", maxRetries, req.Method, req.URL, lastErr)
return nil, fmt.Errorf("failed after %d attempts to %s %s: %w", maxRetries, req.Method, req.URL, lastErr)
}
// Helper for making GET requests with retries
+241
View File
@@ -0,0 +1,241 @@
package cli
import (
"errors"
"fmt"
"net"
"net/http"
"net/url"
"syscall"
"testing"
"github.com/Control-D-Inc/ctrld/internal/controld"
)
// wsaEACCES is WSAEACCES (10013): "An attempt was made to access a socket in a way
// forbidden by its access permissions." This is what Windows reports when a WFP
// filter denies the connect. Used as a plain errno so the test runs everywhere.
const wsaEACCES = syscall.Errno(10013)
// denyingRoundTripper denies the hostname attempt with firstErr and the direct-ip
// attempt with fbErr, the shape seen during the Firewall Mode incident: the
// hostname attempt was denied by ctrld's own stale block-all filters, while the
// direct-ip fallback failed on an unreachable IPv6 route.
type denyingRoundTripper struct {
hostname string
firstErr error
fbErr error
}
func (rt *denyingRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
if req.URL.Host == rt.hostname {
return nil, &net.OpError{Op: "dial", Net: "tcp4", Err: rt.firstErr}
}
return nil, &net.OpError{Op: "dial", Net: "tcp6", Err: rt.fbErr}
}
func TestDoWithRetryPreservesHostnameError(t *testing.T) {
const hostname = "dl.controld.dev"
req, err := http.NewRequest(http.MethodGet, "https://"+hostname+"/v2/windows-amd64/ctrld.exe", nil)
if err != nil {
t.Fatal(err)
}
rt := &denyingRoundTripper{
hostname: hostname,
firstErr: wsaEACCES,
fbErr: syscall.EHOSTUNREACH,
}
_, err = doWithRetryClient(&http.Client{Transport: rt}, req, 1, "23.171.240.151")
if err == nil {
t.Fatal("expected doWithRetry to fail when both attempts are denied")
}
if !errors.Is(err, wsaEACCES) {
t.Errorf("hostname-attempt error (WSAEACCES) was lost, got: %v", err)
}
if !errors.Is(err, syscall.EHOSTUNREACH) {
t.Errorf("fallback error was lost, got: %v", err)
}
}
// composedAttemptErrors builds the error shape the two-attempt paths return: each
// attempt's *url.Error (as produced by http.Client.Do) wrapped by a single fmt.Errorf
// with two %w verbs, hostname attempt first. Mirrors doWithFallback in
// internal/controld and doWithRetryClient above.
func composedAttemptErrors(first, fallback error) error {
attempt := func(network string, cause error) error {
return &url.Error{
Op: "Post",
URL: "https://api.controld.com/utility",
Err: &net.OpError{Op: "dial", Net: network, Err: cause},
}
}
return fmt.Errorf("request failed: %w; fallback to direct ip %s failed: %w",
attempt("tcp4", first), "147.185.34.1", attempt("tcp6", fallback))
}
// TestComposedFallbackErrorRetryClassification pins which attempt decides whether
// preflight keeps retrying.
//
// Reporting both attempt errors is not purely diagnostic: processCDFlags decides
// retryability with errUrlNetworkError, which uses errors.As, and errors.As is
// order-sensitive - it returns the *first* matching error in the tree. Composing the
// hostname attempt first therefore hands the retry predicate the hostname failure,
// where previously only the fallback's error survived to be classified.
//
// The consequence is deliberate: a locally denied socket (WSAEACCES, a firewall
// blocking ctrld) is no longer treated as a transient network error, so preflight fails
// fast and reports instead of backing off - the incident logged 256 retry cycles
// against filters that were never going to clear on their own. The boot case that
// justifies the indefinite retry, a network unreachable on both attempts, is preserved.
//
// If the wrap order is ever reversed, this test fails rather than silently restoring
// indefinite retries against a host that is actively refusing.
func TestComposedFallbackErrorRetryClassification(t *testing.T) {
tests := []struct {
name string
hostname error
fallback error
wantRetryable bool
}{
{
// The incident's pair: denied locally, IPv6 route unusable.
name: "denied socket then unreachable fallback fails fast",
hostname: wsaEACCES,
fallback: syscall.EHOSTUNREACH,
wantRetryable: false,
},
{
// Boot with no network yet: must still retry indefinitely.
name: "network unreachable on both attempts still retries",
hostname: syscall.ENETUNREACH,
fallback: syscall.ENETUNREACH,
wantRetryable: true,
},
{
name: "connection refused still retries",
hostname: syscall.ECONNREFUSED,
fallback: syscall.EHOSTUNREACH,
wantRetryable: true,
},
{
name: "permission denied on both attempts fails fast",
hostname: syscall.EACCES,
fallback: syscall.EACCES,
wantRetryable: false,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
err := composedAttemptErrors(tc.hostname, tc.fallback)
if got := errUrlNetworkError(err); got != tc.wantRetryable {
t.Errorf("errUrlNetworkError() = %v, want %v", got, tc.wantRetryable)
}
// Both attempts remain reportable regardless of classification.
if !errors.Is(err, tc.hostname) {
t.Error("hostname attempt error was lost")
}
if !errors.Is(err, tc.fallback) {
t.Error("fallback attempt error was lost")
}
})
}
}
// TestUnresolvedHostnameDefersToFallbackAttempt covers the asymmetric pair.
//
// Only the hostname attempt resolves DNS, and Go marks a *net.DNSError as temporary only
// for socket failures that reached the server - so a SERVFAIL or "no such host" answer is
// not temporary. At boot behind a captive portal, or before a router's forwarder is up,
// that is exactly how the hostname attempt fails while the network is merely not ready.
// Before the composed error existed only the fallback decided, so this pair retried;
// classifying the hostname attempt alone would fail it fast and reach Fatal.
//
// A name-resolution failure therefore carries no verdict: the fallback attempt decides.
// The locally-denied case above still fails fast, because a denied socket is definitive.
func TestUnresolvedHostnameDefersToFallbackAttempt(t *testing.T) {
dnsFailure := &url.Error{
Op: "Post",
URL: "https://api.controld.com/utility",
Err: &net.DNSError{Err: "server misbehaving", Name: "api.controld.com", IsTemporary: false},
}
attempt := func(cause error) error {
return &url.Error{
Op: "Post",
URL: "https://api.controld.com/utility",
Err: &net.OpError{Op: "dial", Net: "tcp6", Err: cause},
}
}
retryable := fmt.Errorf("request failed: %w; fallback to direct ip %s failed: %w",
dnsFailure, "147.185.34.1", attempt(syscall.ECONNREFUSED))
if !errUrlNetworkError(retryable) {
t.Error("an unresolved hostname with a retryable fallback must keep retrying: at boot the network is simply not up yet")
}
denied := fmt.Errorf("request failed: %w; fallback to direct ip %s failed: %w",
dnsFailure, "147.185.34.1", attempt(wsaEACCES))
if errUrlNetworkError(denied) {
t.Error("an unresolved hostname with a denied fallback must fail fast: nothing here clears on its own")
}
// A resolution failure alone still says nothing, so it must not be read as retryable.
if errUrlNetworkError(dnsFailure) {
t.Error("a bare name-resolution failure must not be classified as retryable")
}
}
// TestDoWithFallbackClassificationEndToEnd drives the real composition in
// internal/controld through the real predicate, instead of asserting a hand-written copy
// of its error shape against another hand-written copy. A change to either side's format
// string or wrap order is caught here.
func TestDoWithFallbackClassificationEndToEnd(t *testing.T) {
const hostname = "api.controld.com"
req, err := http.NewRequest(http.MethodPost, "https://"+hostname+"/utility", nil)
if err != nil {
t.Fatal(err)
}
rt := &denyingRoundTripper{
hostname: hostname,
firstErr: wsaEACCES,
fbErr: syscall.EHOSTUNREACH,
}
_, gotErr := controld.DoWithFallbackForTest(&http.Client{Transport: rt}, req, "147.185.34.1")
if gotErr == nil {
t.Fatal("expected both attempts to fail")
}
if errUrlNetworkError(gotErr) {
t.Errorf("the real composed error was classified as retryable: %v", gotErr)
}
if !errors.Is(gotErr, wsaEACCES) || !errors.Is(gotErr, syscall.EHOSTUNREACH) {
t.Errorf("the real composed error lost an attempt: %v", gotErr)
}
}
// TestDoWithRetryComposesHostnameAttemptFirst anchors the ordering assumption above to
// the real composition, so a reordering of the wrap in doWithRetryClient is caught here
// and not only in the hand-built shape.
func TestDoWithRetryComposesHostnameAttemptFirst(t *testing.T) {
const hostname = "dl.controld.dev"
req, err := http.NewRequest(http.MethodGet, "https://"+hostname+"/v2/windows-amd64/ctrld.exe", nil)
if err != nil {
t.Fatal(err)
}
rt := &denyingRoundTripper{hostname: hostname, firstErr: wsaEACCES, fbErr: syscall.EHOSTUNREACH}
_, gotErr := doWithRetryClient(&http.Client{Transport: rt}, req, 1, "23.171.240.151")
if gotErr == nil {
t.Fatal("expected both attempts to fail")
}
// errors.As must reach the hostname attempt first: that is what the retry
// predicate classifies.
var opErr *net.OpError
if !errors.As(gotErr, &opErr) {
t.Fatalf("no net.OpError in the chain: %v", gotErr)
}
if !errors.Is(opErr.Err, wsaEACCES) {
t.Errorf("first OpError in the chain is %v, want the hostname attempt (%v)", opErr.Err, wsaEACCES)
}
}
+34
View File
@@ -0,0 +1,34 @@
package cli
import (
"testing"
"github.com/Control-D-Inc/ctrld"
)
func TestListenerInterceptModeExplicitOff(t *testing.T) {
oldIntercept := interceptMode
t.Cleanup(func() { interceptMode = oldIntercept })
cfg := &ctrld.Config{}
cfg.Service.InterceptMode = "dns"
tests := []struct {
name string
flag string
want string
}{
{name: "explicit off is final", flag: "off", want: "off"},
{name: "empty flag falls back to config", flag: "", want: "dns"},
{name: "explicit dns wins over config", flag: "dns", want: "dns"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
interceptMode = tc.flag
if got := listenerInterceptMode(cfg); got != tc.want {
t.Fatalf("listenerInterceptMode() = %q, want %q", got, tc.want)
}
})
}
}
+339
View File
@@ -0,0 +1,339 @@
package cli
import (
"io"
"os"
"strings"
"sync"
"testing"
"time"
)
// =============================================================================
// logWriter.tailLastLines tests
// =============================================================================
func Test_logWriter_tailLastLines_Empty(t *testing.T) {
lw := newLogWriterWithSize(4096)
if got := lw.tailLastLines(10); got != nil {
t.Fatalf("expected nil for empty buffer, got %q", got)
}
}
func Test_logWriter_tailLastLines_ZeroLines(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\n"))
if got := lw.tailLastLines(0); got != nil {
t.Fatalf("expected nil for n=0, got %q", got)
}
}
func Test_logWriter_tailLastLines_NegativeLines(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\n"))
if got := lw.tailLastLines(-1); got != nil {
t.Fatalf("expected nil for n=-1, got %q", got)
}
}
func Test_logWriter_tailLastLines_FewerThanN(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\n"))
got := string(lw.tailLastLines(10))
want := "line1\nline2\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_logWriter_tailLastLines_ExactN(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\nline3\n"))
got := string(lw.tailLastLines(3))
want := "line1\nline2\nline3\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_logWriter_tailLastLines_MoreThanN(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\nline3\nline4\nline5\n"))
got := string(lw.tailLastLines(2))
want := "line4\nline5\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_logWriter_tailLastLines_NoTrailingNewline(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("line1\nline2\nline3"))
// Without trailing newline, "line3" is a partial line.
// Asking for 1 line returns the last newline-terminated line plus the partial.
got := string(lw.tailLastLines(1))
want := "line2\nline3"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_logWriter_tailLastLines_SingleLineNoNewline(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("only line"))
got := string(lw.tailLastLines(5))
want := "only line"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_logWriter_tailLastLines_SingleLineWithNewline(t *testing.T) {
lw := newLogWriterWithSize(4096)
lw.Write([]byte("only line\n"))
got := string(lw.tailLastLines(1))
want := "only line\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
// =============================================================================
// logWriter.Subscribe tests
// =============================================================================
func Test_logWriter_Subscribe_Basic(t *testing.T) {
lw := newLogWriterWithSize(4096)
ch, unsub := lw.Subscribe()
defer unsub()
msg := []byte("hello world\n")
lw.Write(msg)
select {
case got := <-ch:
if string(got) != string(msg) {
t.Fatalf("got %q, want %q", got, msg)
}
case <-time.After(time.Second):
t.Fatal("timed out waiting for subscriber data")
}
}
func Test_logWriter_Subscribe_MultipleSubscribers(t *testing.T) {
lw := newLogWriterWithSize(4096)
ch1, unsub1 := lw.Subscribe()
defer unsub1()
ch2, unsub2 := lw.Subscribe()
defer unsub2()
msg := []byte("broadcast\n")
lw.Write(msg)
for i, ch := range []<-chan []byte{ch1, ch2} {
select {
case got := <-ch:
if string(got) != string(msg) {
t.Fatalf("subscriber %d: got %q, want %q", i, got, msg)
}
case <-time.After(time.Second):
t.Fatalf("subscriber %d: timed out", i)
}
}
}
func Test_logWriter_Subscribe_Unsubscribe(t *testing.T) {
lw := newLogWriterWithSize(4096)
ch, unsub := lw.Subscribe()
// Verify subscribed.
lw.Write([]byte("before unsub\n"))
select {
case <-ch:
case <-time.After(time.Second):
t.Fatal("timed out before unsub")
}
unsub()
// Channel should be closed after unsub.
if _, ok := <-ch; ok {
t.Fatal("channel should be closed after unsubscribe")
}
// Verify subscriber list is empty.
lw.mu.Lock()
count := len(lw.subscribers)
lw.mu.Unlock()
if count != 0 {
t.Fatalf("expected 0 subscribers after unsub, got %d", count)
}
}
func Test_logWriter_Subscribe_UnsubscribeIdempotent(t *testing.T) {
lw := newLogWriterWithSize(4096)
_, unsub := lw.Subscribe()
unsub()
// Second unsub should not panic.
unsub()
}
func Test_logWriter_Subscribe_SlowSubscriberDropped(t *testing.T) {
lw := newLogWriterWithSize(4096)
ch, unsub := lw.Subscribe()
defer unsub()
// Fill the subscriber channel (buffer size is 256).
for i := 0; i < 300; i++ {
lw.Write([]byte("msg\n"))
}
// Should have 256 buffered messages, rest dropped.
count := 0
for {
select {
case <-ch:
count++
default:
goto done
}
}
done:
if count != 256 {
t.Fatalf("expected 256 buffered messages, got %d", count)
}
}
func Test_logWriter_Subscribe_ConcurrentWriteAndRead(t *testing.T) {
lw := newLogWriterWithSize(64 * 1024)
ch, unsub := lw.Subscribe()
defer unsub()
const numWrites = 100
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < numWrites; i++ {
lw.Write([]byte("concurrent write\n"))
}
}()
received := 0
timeout := time.After(5 * time.Second)
for received < numWrites {
select {
case <-ch:
received++
case <-timeout:
t.Fatalf("timed out after receiving %d/%d messages", received, numWrites)
}
}
wg.Wait()
}
// =============================================================================
// tailFileLastLines tests
// =============================================================================
func writeTempFile(t *testing.T, content string) *os.File {
t.Helper()
f, err := os.CreateTemp(t.TempDir(), "tail-test-*")
if err != nil {
t.Fatal(err)
}
if _, err := f.WriteString(content); err != nil {
t.Fatal(err)
}
return f
}
func Test_tailFileLastLines_Empty(t *testing.T) {
f := writeTempFile(t, "")
defer f.Close()
if got := tailFileLastLines(f, 10); got != nil {
t.Fatalf("expected nil for empty file, got %q", got)
}
}
func Test_tailFileLastLines_FewerThanN(t *testing.T) {
f := writeTempFile(t, "line1\nline2\n")
defer f.Close()
got := string(tailFileLastLines(f, 10))
want := "line1\nline2\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_tailFileLastLines_ExactN(t *testing.T) {
f := writeTempFile(t, "a\nb\nc\n")
defer f.Close()
got := string(tailFileLastLines(f, 3))
want := "a\nb\nc\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_tailFileLastLines_MoreThanN(t *testing.T) {
f := writeTempFile(t, "line1\nline2\nline3\nline4\nline5\n")
defer f.Close()
got := string(tailFileLastLines(f, 2))
want := "line4\nline5\n"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_tailFileLastLines_NoTrailingNewline(t *testing.T) {
f := writeTempFile(t, "line1\nline2\nline3")
defer f.Close()
// Without trailing newline, partial last line comes with the previous line.
got := string(tailFileLastLines(f, 1))
want := "line2\nline3"
if got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func Test_tailFileLastLines_LargerThanChunk(t *testing.T) {
// Build content larger than the 4096 chunk size to exercise multi-chunk reads.
var sb strings.Builder
for i := 0; i < 200; i++ {
sb.WriteString(strings.Repeat("x", 50))
sb.WriteByte('\n')
}
f := writeTempFile(t, sb.String())
defer f.Close()
got := string(tailFileLastLines(f, 3))
lines := strings.Split(strings.TrimRight(got, "\n"), "\n")
if len(lines) != 3 {
t.Fatalf("expected 3 lines, got %d: %q", len(lines), got)
}
expectedLine := strings.Repeat("x", 50)
for _, line := range lines {
if line != expectedLine {
t.Fatalf("unexpected line content: %q", line)
}
}
}
func Test_tailFileLastLines_SeeksToEnd(t *testing.T) {
f := writeTempFile(t, "line1\nline2\nline3\n")
defer f.Close()
tailFileLastLines(f, 1)
// After tailFileLastLines, file position should be at the end.
pos, err := f.Seek(0, io.SeekCurrent)
if err != nil {
t.Fatal(err)
}
stat, err := f.Stat()
if err != nil {
t.Fatal(err)
}
if pos != stat.Size() {
t.Fatalf("expected file position at end (%d), got %d", stat.Size(), pos)
}
}
+273 -4
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"io"
"os"
"path/filepath"
"strings"
"sync"
"time"
@@ -22,6 +23,9 @@ const (
logWriterSentInterval = time.Minute
logWriterInitEndMarker = "\n\n=== INIT_END ===\n\n"
logWriterLogEndMarker = "\n\n=== LOG_END ===\n\n"
logFileName = "ctrld.log"
logFileMaxSize = 1024 * 1024 * 5 // 5 MB
)
type logViewResponse struct {
@@ -38,11 +42,24 @@ type logReader struct {
size int64
}
// logSubscriber represents a subscriber to live log output.
type logSubscriber struct {
ch chan []byte
}
// logWriter is an internal buffer to keep track of runtime log when no logging is enabled.
// When a file path is configured via setLogFile, writes are also persisted to
// a rotated file on disk (max logFileMaxSize, 1 backup) so logs survive restarts.
type logWriter struct {
mu sync.Mutex
buf bytes.Buffer
size int
mu sync.Mutex
buf bytes.Buffer
size int
subscribers []*logSubscriber
// File persistence fields.
logFile *os.File
logFilePath string
logFileSize int64
}
// newLogWriter creates an internal log writer.
@@ -61,10 +78,154 @@ func newLogWriterWithSize(size int) *logWriter {
return lw
}
// setLogFile configures file-backed persistence for the log writer.
// The directory is created if it does not exist. An existing file is
// opened in append mode and its current size is tracked for rotation.
func (lw *logWriter) setLogFile(path string) error {
dir := filepath.Dir(path)
if err := os.MkdirAll(dir, 0750); err != nil {
return fmt.Errorf("creating log directory: %w", err)
}
f, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR|os.O_APPEND, 0600)
if err != nil {
return fmt.Errorf("opening log file: %w", err)
}
st, err := f.Stat()
if err != nil {
f.Close()
return fmt.Errorf("stat log file: %w", err)
}
lw.mu.Lock()
defer lw.mu.Unlock()
lw.logFile = f
lw.logFilePath = path
lw.logFileSize = st.Size()
return nil
}
// rotateLogFile rotates the current log file to a .1 backup.
// It returns true if lw.logFile is usable after the call, false otherwise.
// Must be called with lw.mu held.
func (lw *logWriter) rotateLogFile() bool {
if lw.logFile == nil {
return false
}
lw.logFile.Close()
backupPath := lw.logFilePath + ".1"
// Best effort: rename current to backup (overwrites old backup).
os.Rename(lw.logFilePath, backupPath)
f, err := os.OpenFile(lw.logFilePath, os.O_CREATE|os.O_RDWR|os.O_TRUNC, 0600)
if err != nil {
// If we can't reopen, disable file logging.
lw.logFile = nil
lw.logFileSize = 0
return false
}
lw.logFile = f
lw.logFileSize = 0
return true
}
// closeLogFile closes the backing file if open.
func (lw *logWriter) closeLogFile() {
lw.mu.Lock()
defer lw.mu.Unlock()
if lw.logFile != nil {
lw.logFile.Close()
lw.logFile = nil
}
}
// logFilePaths returns the paths to the current log file and its backup
// (if they exist) for inclusion in log send payloads.
func (lw *logWriter) logFilePaths() (current, backup string) {
lw.mu.Lock()
defer lw.mu.Unlock()
if lw.logFilePath == "" {
return "", ""
}
current = lw.logFilePath
bp := lw.logFilePath + ".1"
if _, err := os.Stat(bp); err == nil {
backup = bp
}
return current, backup
}
// Subscribe returns a channel that receives new log data as it's written,
// and an unsubscribe function to clean up when done.
func (lw *logWriter) Subscribe() (<-chan []byte, func()) {
lw.mu.Lock()
defer lw.mu.Unlock()
sub := &logSubscriber{ch: make(chan []byte, 256)}
lw.subscribers = append(lw.subscribers, sub)
unsub := func() {
lw.mu.Lock()
defer lw.mu.Unlock()
for i, s := range lw.subscribers {
if s == sub {
lw.subscribers = append(lw.subscribers[:i], lw.subscribers[i+1:]...)
close(sub.ch)
break
}
}
}
return sub.ch, unsub
}
// tailLastLines returns the last n lines from the current buffer.
func (lw *logWriter) tailLastLines(n int) []byte {
lw.mu.Lock()
defer lw.mu.Unlock()
data := lw.buf.Bytes()
if n <= 0 || len(data) == 0 {
return nil
}
// Find the last n newlines from the end.
count := 0
pos := len(data)
for pos > 0 {
pos--
if data[pos] == '\n' {
count++
if count == n+1 {
pos++ // move past this newline
break
}
}
}
result := make([]byte, len(data)-pos)
copy(result, data[pos:])
return result
}
func (lw *logWriter) Write(p []byte) (int, error) {
lw.mu.Lock()
defer lw.mu.Unlock()
// Fan-out to subscribers (non-blocking).
if len(lw.subscribers) > 0 {
cp := make([]byte, len(p))
copy(cp, p)
for _, sub := range lw.subscribers {
select {
case sub.ch <- cp:
default:
// Drop if subscriber is slow to avoid blocking the logger.
}
}
}
// Write to backing file if configured.
if lw.logFile != nil {
needsRotation := lw.logFileSize+int64(len(p)) > logFileMaxSize
if !needsRotation || lw.rotateLogFile() {
if n, err := lw.logFile.Write(p); err == nil {
lw.logFileSize += int64(n)
}
}
}
// If writing p causes overflows, discard old data.
if lw.buf.Len()+len(p) > lw.size {
buf := lw.buf.Bytes()
@@ -102,6 +263,12 @@ func (p *prog) initLogging(backup bool) {
p.initInternalLogging(logWriters)
}
// internalLogFilePath returns the path for persisted internal logs.
// The file lives in the ctrld home directory alongside other runtime state.
func internalLogFilePath() string {
return absHomeDir(logFileName)
}
// initInternalLogging performs internal logging if there's no log enabled.
func (p *prog) initInternalLogging(writers []io.Writer) {
if !p.needInternalLogging() {
@@ -112,6 +279,14 @@ func (p *prog) initInternalLogging(writers []io.Writer) {
p.internalLogWriter = newLogWriter()
p.internalLogSent = time.Now().Add(-logWriterSentInterval)
p.internalWarnLogWriter = newSmallLogWriter()
// Persist internal logs to disk so they survive restarts.
if path := internalLogFilePath(); path != "" {
if err := p.internalLogWriter.setLogFile(path); err != nil {
mainLog.Load().Warn().Err(err).Msg("could not enable persistent internal logging")
} else {
mainLog.Load().Notice().Msgf("internal log file: %s", path)
}
}
})
p.mu.Lock()
lw := p.internalLogWriter
@@ -143,6 +318,12 @@ func (p *prog) initInternalLogging(writers []io.Writer) {
// needInternalLogging reports whether prog needs to run internal logging.
func (p *prog) needInternalLogging() bool {
// Do not run in silent mode: the user explicitly asked for no logging, so
// ctrld must not create or write the persisted internal log file (nor reset
// the global level back to debug). See https://github.com/Control-D-Inc/ctrld/issues/320.
if silent {
return false
}
// Do not run in non-cd mode.
if cdUID == "" {
return false
@@ -166,7 +347,15 @@ func (p *prog) logReader() (*logReader, error) {
if wlw == nil {
return nil, errors.New("nil internal warn log writer")
}
// Normal log content.
// If we have a persisted log file, read from disk (includes data
// from previous runs that the in-memory buffer wouldn't have).
current, backup := lw.logFilePaths()
if current != "" {
return p.logReaderFromFiles(current, backup, wlw)
}
// Fall back to in-memory buffer.
lw.mu.Lock()
lwReader := bytes.NewReader(lw.buf.Bytes())
lwSize := lw.buf.Len()
@@ -202,3 +391,83 @@ func (p *prog) logReader() (*logReader, error) {
}
return lr, nil
}
// logReaderFromFiles builds a logReader that concatenates the backup file
// (if it exists), the current log file, and the in-memory warn log buffer.
func (p *prog) logReaderFromFiles(current, backup string, wlw *logWriter) (*logReader, error) {
var rcs []io.ReadCloser
var totalSize int64
closeAll := func() {
for _, rc := range rcs {
rc.Close()
}
}
// Read backup file first (older entries).
if backup != "" {
if bf, err := os.Open(backup); err == nil {
if st, err := bf.Stat(); err == nil {
totalSize += st.Size()
}
rcs = append(rcs, bf)
}
}
// Read current file.
cf, err := os.Open(current)
if err != nil {
closeAll()
return nil, fmt.Errorf("opening current log file: %w", err)
}
if st, err := cf.Stat(); err == nil {
totalSize += st.Size()
}
rcs = append(rcs, cf)
// Append warn log content from memory.
wlw.mu.Lock()
warnData := make([]byte, wlw.buf.Len())
copy(warnData, wlw.buf.Bytes())
wlw.mu.Unlock()
if len(warnData) > 0 {
rcs = append(rcs, io.NopCloser(bytes.NewReader([]byte(logWriterLogEndMarker))))
rcs = append(rcs, io.NopCloser(bytes.NewReader(warnData)))
totalSize += int64(len(logWriterLogEndMarker) + len(warnData))
}
if totalSize == 0 {
closeAll()
return nil, errors.New("internal log is empty")
}
readers := make([]io.Reader, len(rcs))
closers := make([]io.Closer, len(rcs))
for i, rc := range rcs {
readers[i] = rc
closers[i] = rc
}
combined := io.MultiReader(readers...)
lr := &logReader{
r: &multiCloser{Reader: combined, closers: closers},
size: totalSize,
}
return lr, nil
}
// multiCloser wraps an io.Reader and closes multiple underlying closers.
type multiCloser struct {
io.Reader
closers []io.Closer
}
func (mc *multiCloser) Close() error {
var firstErr error
for _, c := range mc.closers {
if err := c.Close(); err != nil && firstErr == nil {
firstErr = err
}
}
return firstErr
}
+66
View File
@@ -0,0 +1,66 @@
package cli
import (
"os"
"path/filepath"
"testing"
"github.com/Control-D-Inc/ctrld"
)
// Test_needInternalLogging_silent is a regression test for
// https://github.com/Control-D-Inc/ctrld/issues/320: running with --silent must
// not enable internal logging, otherwise ctrld creates and writes
// <homedir>/ctrld.log (and, when verbose==0, resets the global level back to
// debug) despite the user asking for silence.
func Test_needInternalLogging_silent(t *testing.T) {
origSilent, origCdUID := silent, cdUID
t.Cleanup(func() { silent, cdUID = origSilent, origCdUID })
tests := []struct {
name string
silent bool
cdUID string
logPath string
want bool
}{
{"silent suppresses internal logging in cd mode", true, "test-uid", "", false},
{"cd mode enables internal logging", false, "test-uid", "", true},
{"non-cd mode disabled", false, "", "", false},
{"explicit log path disables internal logging", false, "test-uid", "/var/log/ctrld.log", false},
}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
silent = tt.silent
cdUID = tt.cdUID
p := &prog{cfg: &ctrld.Config{}}
p.cfg.Service.LogPath = tt.logPath
if got := p.needInternalLogging(); got != tt.want {
t.Fatalf("needInternalLogging() = %v, want %v", got, tt.want)
}
})
}
}
// Test_initInternalLogging_silentCreatesNoFile drives the real initInternalLogging
// path and asserts that a --silent --cd run does not create <homedir>/ctrld.log,
// which is the observable failure reported in
// https://github.com/Control-D-Inc/ctrld/issues/320.
func Test_initInternalLogging_silentCreatesNoFile(t *testing.T) {
origSilent, origCdUID, origHomedir := silent, cdUID, homedir
t.Cleanup(func() { silent, cdUID, homedir = origSilent, origCdUID, origHomedir })
dir := t.TempDir()
homedir = dir
cdUID = "test-uid" // cd mode, which would otherwise enable internal logging
silent = true
p := &prog{cfg: &ctrld.Config{}}
p.initInternalLogging(nil)
logPath := filepath.Join(dir, logFileName)
if _, err := os.Stat(logPath); !os.IsNotExist(err) {
t.Fatalf("silent mode must not create %s (stat err = %v)", logPath, err)
}
}
+125
View File
@@ -1,6 +1,8 @@
package cli
import (
"os"
"path/filepath"
"strings"
"sync"
"testing"
@@ -83,3 +85,126 @@ func Test_logWriter_MarkerInitEnd(t *testing.T) {
t.Fatalf("unexpected log content: %s", lw.buf.String())
}
}
func Test_logWriter_SetLogFile(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "test.log")
lw := newLogWriterWithSize(logWriterSize)
if err := lw.setLogFile(path); err != nil {
t.Fatalf("setLogFile: %v", err)
}
defer lw.closeLogFile()
msg := "hello file\n"
lw.Write([]byte(msg))
// Verify data in memory buffer.
if lw.buf.String() != msg {
t.Fatalf("buffer: got %q, want %q", lw.buf.String(), msg)
}
// Verify data on disk.
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
if string(data) != msg {
t.Fatalf("file: got %q, want %q", data, msg)
}
}
func Test_logWriter_FileRotation(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "test.log")
// Use a tiny max size to trigger rotation quickly.
lw := newLogWriterWithSize(logWriterSize)
if err := lw.setLogFile(path); err != nil {
t.Fatalf("setLogFile: %v", err)
}
defer lw.closeLogFile()
// Write enough to exceed logFileMaxSize.
chunk := strings.Repeat("X", 1024) + "\n"
written := 0
for written < logFileMaxSize+1024 {
lw.Write([]byte(chunk))
written += len(chunk)
}
// Backup file should exist.
backupPath := path + ".1"
if _, err := os.Stat(backupPath); os.IsNotExist(err) {
t.Fatal("expected backup file to exist after rotation")
}
// Current file should be smaller than max (it was rotated).
st, err := os.Stat(path)
if err != nil {
t.Fatalf("stat current: %v", err)
}
if st.Size() > logFileMaxSize {
t.Fatalf("current file too large after rotation: %d", st.Size())
}
}
func Test_logWriter_FilePaths(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "test.log")
lw := newLogWriterWithSize(logWriterSize)
// No file configured.
c, b := lw.logFilePaths()
if c != "" || b != "" {
t.Fatalf("expected empty paths, got %q %q", c, b)
}
if err := lw.setLogFile(path); err != nil {
t.Fatalf("setLogFile: %v", err)
}
defer lw.closeLogFile()
// Current exists, no backup yet.
c, b = lw.logFilePaths()
if c != path {
t.Fatalf("current: got %q, want %q", c, path)
}
if b != "" {
t.Fatalf("backup should be empty, got %q", b)
}
// Create a backup file manually.
os.WriteFile(path+".1", []byte("old"), 0600)
_, b = lw.logFilePaths()
if b != path+".1" {
t.Fatalf("backup: got %q, want %q", b, path+".1")
}
}
func Test_logWriter_FileAppendOnRestart(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "test.log")
// Simulate first run.
lw1 := newLogWriterWithSize(logWriterSize)
if err := lw1.setLogFile(path); err != nil {
t.Fatalf("setLogFile: %v", err)
}
lw1.Write([]byte("run1\n"))
lw1.closeLogFile()
// Simulate second run (restart) — file should be appended.
lw2 := newLogWriterWithSize(logWriterSize)
if err := lw2.setLogFile(path); err != nil {
t.Fatalf("setLogFile: %v", err)
}
lw2.Write([]byte("run2\n"))
lw2.closeLogFile()
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
want := "run1\nrun2\n"
if string(data) != want {
t.Fatalf("file: got %q, want %q", data, want)
}
}
+40
View File
@@ -1,7 +1,9 @@
package cli
import (
"encoding/hex"
"io"
"net"
"os"
"path/filepath"
"sync/atomic"
@@ -40,6 +42,9 @@ var (
cleanup bool
startOnly bool
rfc1918 bool
interceptMode string // "", "dns", or "hard" — set via --intercept-mode flag or config
dnsIntercept bool // derived: interceptMode == "dns" || interceptMode == "hard"
hardIntercept bool // derived: interceptMode == "hard"
mainLog atomic.Pointer[zerolog.Logger]
consoleWriter zerolog.ConsoleWriter
@@ -51,6 +56,9 @@ const (
cdOrgFlagName = "cd-org"
customHostnameFlagName = "custom-hostname"
nextdnsFlagName = "nextdns"
// autoIface is the sentinel --iface value meaning "use the default gateway interface".
autoIface = "auto"
)
func init() {
@@ -59,6 +67,16 @@ func init() {
}
func Main() {
// Fast path for pf interception probe subprocess. This runs before cobra
// initialization to minimize startup time. The parent process spawns us with
// "pf-probe-send <host> <hex-dns-packet>" and a non-_ctrld GID so pf
// intercepts the DNS query. If pf rdr is working, the query reaches ctrld's
// listener; if not, it goes to the real DNS server and ctrld detects the miss.
if len(os.Args) >= 4 && os.Args[1] == "pf-probe-send" {
pfProbeSend(os.Args[2], os.Args[3])
return
}
ctrld.InitConfig(v, "ctrld")
initCLI()
if err := rootCmd.Execute(); err != nil {
@@ -189,3 +207,25 @@ func initCache() {
cfg.Service.CacheSize = 4096
}
}
// pfProbeSend is a minimal subprocess that sends a pre-built DNS query packet
// to the specified host on port 53. It's invoked by probePFIntercept() with a
// non-_ctrld GID so pf interception applies to the query.
//
// Usage: ctrld pf-probe-send <host> <hex-encoded-dns-packet>
func pfProbeSend(host, hexPacket string) {
packet, err := hex.DecodeString(hexPacket)
if err != nil {
os.Exit(1)
}
conn, err := net.DialTimeout("udp", net.JoinHostPort(host, "53"), time.Second)
if err != nil {
os.Exit(1)
}
defer conn.Close()
conn.SetDeadline(time.Now().Add(time.Second))
_, _ = conn.Write(packet)
// Read response (don't care about result, just need the send to happen)
buf := make([]byte, 512)
_, _ = conn.Read(buf)
}
+60 -1
View File
@@ -1,17 +1,76 @@
package cli
import (
"fmt"
"os"
"os/exec"
"strings"
"sync"
"testing"
"github.com/rs/zerolog"
)
var logOutput strings.Builder
// logOutput is the log sink for the whole test binary. Tests share it with any
// background goroutine the code under test starts (watchdogs, timers), so it
// must tolerate concurrent writes.
var logOutput syncBuffer
// syncBuffer is a strings.Builder guarded by a mutex.
type syncBuffer struct {
mu sync.Mutex
sb strings.Builder
}
func (b *syncBuffer) Write(p []byte) (int, error) {
b.mu.Lock()
defer b.mu.Unlock()
return b.sb.Write(p)
}
func (b *syncBuffer) String() string {
b.mu.Lock()
defer b.mu.Unlock()
return b.sb.String()
}
// envFakeVersionOutput makes this test binary impersonate a ctrld executable: when
// set, the process writes the value to stdout and exits without running any test, so
// binaryVersion() can be exercised on every platform without building or shipping a
// fixture binary. The value envFakeVersionSilent produces no output at all, which
// reproduces a ctrld.exe_previous that exists but reports no version.
//
// This must be handled before m.Run(), which is what parses the test flags: the child
// is invoked as "<binary> --version" and would otherwise die on an unknown flag.
const (
envFakeVersionOutput = "CTRLD_TEST_FAKE_VERSION_OUTPUT"
envFakeVersionSilent = "<silent>"
)
func TestMain(m *testing.M) {
if out := os.Getenv(envFakeVersionOutput); out != "" {
if out != envFakeVersionSilent {
fmt.Println(out)
}
os.Exit(0)
}
l := zerolog.New(&logOutput)
mainLog.Store(&l)
// Stub the self-upgrade command builder for the whole test binary. The real
// builder execs os.Executable() — which under `go test` IS this test binary
// — with positional args ("upgrade", ...). `go test` stops flag parsing at
// the first positional arg and ignores the rest, so the child just re-runs
// the entire suite, hits the upgrade tests again, and spawns more children:
// a fork bomb of detached processes that stalls the host and (on Windows)
// holds the test binary's image locked, breaking CI artifact cleanup.
// Point it at the test binary with a no-match -test.run so any test that
// reaches performUpgrade still exercises the cmd.Start() success path while
// the child exits immediately without recursing.
newUpgradeCmd = func(exe string) *exec.Cmd {
return exec.Command(exe, "-test.run=^$")
}
os.Exit(m.Run())
}
+16
View File
@@ -113,6 +113,22 @@ func (p *prog) runMetricsServer(ctx context.Context, reloadCh chan struct{}) {
}
addr := p.cfg.Service.MetricsListener
if addr != "" {
host, port, err := net.SplitHostPort(addr)
if err != nil {
mainLog.Load().Warn().Err(err).Msgf("Invalid metrics listener address (%s); expected host:port", addr)
} else {
if host == "" {
host = "127.0.0.1"
addr = net.JoinHostPort(host, port)
}
ip := net.ParseIP(host)
if (ip != nil && !ip.IsLoopback()) || (ip == nil && host != "localhost") {
mainLog.Load().Warn().Msgf("Metrics server is bound to a non-loopback address (%s). This exposes sensitive data without authentication.", addr)
}
}
}
ms, err := newMetricsServer(addr, reg)
if err != nil {
mainLog.Load().Warn().Err(err).Msg("could not create new metrics server")
-34
View File
@@ -1,34 +0,0 @@
package cli
import (
"context"
"github.com/vishvananda/netlink"
"golang.org/x/sys/unix"
)
func (p *prog) watchLinkState(ctx context.Context) {
ch := make(chan netlink.LinkUpdate)
done := make(chan struct{})
defer close(done)
if err := netlink.LinkSubscribe(ch, done); err != nil {
mainLog.Load().Warn().Err(err).Msg("could not subscribe link")
return
}
for {
select {
case <-ctx.Done():
return
case lu := <-ch:
if lu.Change == 0xFFFFFFFF {
continue
}
if lu.Change&unix.IFF_UP != 0 {
mainLog.Load().Debug().Msgf("link state changed, re-bootstrapping")
for _, uc := range p.cfg.Upstream {
uc.ReBootstrap()
}
}
}
}
}
-7
View File
@@ -1,7 +0,0 @@
//go:build !linux
package cli
import "context"
func (p *prog) watchLinkState(ctx context.Context) {}
+63
View File
@@ -0,0 +1,63 @@
package cli
import (
"errors"
"net/netip"
"strings"
)
const nrptRuleName = `CtrldCatchAll`
// errGPNRPTVerified marks an intercept startup failure that happened while an externally
// managed (Group Policy) NRPT catch-all was proved - by probe, not by registry shape
// alone - to be routing DNS to this listener. It is the difference between "intercept
// failed but DNS still reaches ctrld" and "intercept failed and nothing is filtering",
// which is what decides whether the interface-DNS fallback must run.
//
// Only the Windows path produces it, but setDNS is shared, so the sentinel and its
// predicate live here with the other platform-neutral NRPT helpers.
var errGPNRPTVerified = errors.New("GP-managed NRPT verified routing to ctrld")
// errGPNRPTIneffective marks a startup that ends with externally managed NRPT owning the
// namespace while no probe has proved it routes to ctrld. DNS is not reaching ctrld, but
// adapter DNS was deliberately preserved and no ctrld rule may be written beside an
// administrator's catch-all - so this is a failed start that must not take the
// interface-DNS fallback either.
var errGPNRPTIneffective = errors.New("GP-managed NRPT owns the namespace but no probe reached ctrld")
// interceptFailedWithVerifiedExternalDNS reports whether an intercept startup failure
// happened while externally managed DNS policy was verified to be routing to ctrld.
func interceptFailedWithVerifiedExternalDNS(err error) bool {
return errors.Is(err, errGPNRPTVerified)
}
// interceptFailedUnderExternalDNSPolicy reports whether an intercept startup failure
// happened while externally managed DNS policy owned the namespace, whether or not it was
// proved to route. Either way the interface-DNS fallback must not run: adapter DNS was
// preserved on purpose, and rewriting it would violate the policy ctrld just deferred to.
// Only the verified case is a successful start.
func interceptFailedUnderExternalDNSPolicy(err error) bool {
return errors.Is(err, errGPNRPTVerified) || errors.Is(err, errGPNRPTIneffective)
}
// isExternalGPCatchAll recognizes only a single catch-all namespace that is not
// ctrld's deterministic GP key. Registry access stays in the Windows file; this
// pure classifier is shared with host-runnable tests.
func isExternalGPCatchAll(ruleName string, namespaces []string) bool {
return ruleName != "" && !strings.EqualFold(ruleName, nrptRuleName) && len(namespaces) == 1 && strings.TrimSpace(namespaces[0]) == "."
}
func isMatchingGPNRPTRule(ruleName string, namespaces []string, dnsServers, listenerIP string) bool {
if !isExternalGPCatchAll(ruleName, namespaces) {
return false
}
server, err := netip.ParseAddr(strings.TrimSpace(dnsServers))
if err != nil {
return false
}
listener, err := netip.ParseAddr(strings.TrimSpace(listenerIP))
if err != nil {
return false
}
return server.Unmap() == listener.Unmap()
}
+129
View File
@@ -0,0 +1,129 @@
package cli
import (
"errors"
"fmt"
"testing"
)
func TestIsMatchingGPNRPTRule(t *testing.T) {
tests := []struct {
name string
ruleName string
namespaces []string
servers string
listener string
want bool
}{
{
name: "exact IPv4 catch-all",
ruleName: "{A1B2C3D4}",
namespaces: []string{"."},
servers: "127.0.0.1",
listener: "127.0.0.1",
want: true,
},
{
name: "normalized IPv4-mapped listener",
ruleName: "{A1B2C3D4}",
namespaces: []string{"."},
servers: "::ffff:127.0.0.1",
listener: "127.0.0.1",
want: true,
},
{
name: "ctrld GP key is not external",
ruleName: "ctrldcatchall",
namespaces: []string{"."},
servers: "127.0.0.1",
listener: "127.0.0.1",
},
{
name: "partial namespace",
ruleName: "{A1B2C3D4}",
namespaces: []string{"corp.example"},
servers: "127.0.0.1",
listener: "127.0.0.1",
},
{
name: "multiple namespaces",
ruleName: "{A1B2C3D4}",
namespaces: []string{".", "corp.example"},
servers: "127.0.0.1",
listener: "127.0.0.1",
},
{
name: "wrong listener",
ruleName: "{A1B2C3D4}",
namespaces: []string{"."},
servers: "127.0.0.2",
listener: "127.0.0.1",
},
{
name: "multiple nameservers",
ruleName: "{A1B2C3D4}",
namespaces: []string{"."},
servers: "127.0.0.1;127.0.0.2",
listener: "127.0.0.1",
},
{
name: "malformed nameserver",
ruleName: "{A1B2C3D4}",
namespaces: []string{"."},
servers: "localhost",
listener: "127.0.0.1",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := isMatchingGPNRPTRule(tt.ruleName, tt.namespaces, tt.servers, tt.listener); got != tt.want {
t.Fatalf("isMatchingGPNRPTRule() = %t, want %t", got, tt.want)
}
})
}
}
func TestIsExternalGPCatchAll(t *testing.T) {
tests := []struct {
name string
ruleName string
namespaces []string
want bool
}{
{name: "external catch-all", ruleName: "{GP-RULE}", namespaces: []string{"."}, want: true},
{name: "ctrld key", ruleName: nrptRuleName, namespaces: []string{"."}},
{name: "partial namespace", ruleName: "{GP-RULE}", namespaces: []string{"corp.example"}},
{name: "multiple namespaces", ruleName: "{GP-RULE}", namespaces: []string{".", "corp.example"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := isExternalGPCatchAll(tt.ruleName, tt.namespaces); got != tt.want {
t.Fatalf("isExternalGPCatchAll() = %t, want %t", got, tt.want)
}
})
}
}
// TestInterceptFailedWithVerifiedExternalDNS covers the distinction the interface-DNS
// fallback turns on. "A GP rule exists" is not enough: if it is not actually routing and
// intercept failed too, skipping the fallback leaves the machine with no NRPT, no WFP and
// no adapter DNS - that is, unfiltered. Only a probe-verified route earns the skip.
func TestInterceptFailedWithVerifiedExternalDNS(t *testing.T) {
wfpErr := errors.New("FwpmEngineOpen0 failed: HRESULT 0x5")
verified := fmt.Errorf("dns intercept: WFP setup failed: %w: %w", wfpErr, errGPNRPTVerified)
if !interceptFailedWithVerifiedExternalDNS(verified) {
t.Error("a failure carrying errGPNRPTVerified must skip the interface-DNS fallback")
}
if !errors.Is(verified, wfpErr) {
t.Error("the underlying cause must stay inspectable for logs and callers")
}
if interceptFailedWithVerifiedExternalDNS(fmt.Errorf("dns intercept: WFP setup failed: %w", wfpErr)) {
t.Error("an unverified failure must take the interface-DNS fallback rather than leave the machine unfiltered")
}
if interceptFailedWithVerifiedExternalDNS(nil) {
t.Error("no error must not read as a verified external route")
}
}
+20
View File
@@ -0,0 +1,20 @@
//go:build windows
package cli
import "testing"
func TestWFPStateNRPTPolicyOwner(t *testing.T) {
state := &wfpState{}
state.setNRPTPolicyOwner(nrptRuleOwnerGroupPolicy, "{GP-RULE}")
owner, ruleName := state.nrptPolicyOwner()
if owner != nrptRuleOwnerGroupPolicy || ruleName != "{GP-RULE}" {
t.Fatalf("owner = %v, rule = %q", owner, ruleName)
}
state.setNRPTPolicyOwner(nrptRuleOwnerCtrld, "")
owner, ruleName = state.nrptPolicyOwner()
if owner != nrptRuleOwnerCtrld || ruleName != "" {
t.Fatalf("owner = %v, rule = %q", owner, ruleName)
}
}
File diff suppressed because it is too large Load Diff
+101
View File
@@ -0,0 +1,101 @@
//go:build windows
package cli
import (
"sync"
"time"
"github.com/Control-D-Inc/ctrld"
)
const (
// Default to current behavior: keep recovering indefinitely unless configured.
defaultNRPTRecoveryMaxAttempts = 0
defaultNRPTRecoveryCooldown = 30 * time.Minute
// Require more than one good health tick before clearing the circuit. A probe can
// pass briefly after delete/re-add even when another agent recreates broken NRPT state.
nrptRecoveryStableSuccessesToReset = 2
)
type nrptRecoveryLimiter struct {
mu sync.Mutex
attempts int
stableSuccesses int
cooldownUntil time.Time
lastSkipLog time.Time
}
func nrptRecoveryMaxAttempts(cfg *ctrld.Config) int {
if cfg != nil && cfg.Service.NRPTRecoveryMaxAttempts != nil {
return *cfg.Service.NRPTRecoveryMaxAttempts
}
return defaultNRPTRecoveryMaxAttempts
}
func nrptRecoveryCooldown(cfg *ctrld.Config) time.Duration {
if cfg != nil && cfg.Service.NRPTRecoveryCooldown != nil {
return *cfg.Service.NRPTRecoveryCooldown
}
return defaultNRPTRecoveryCooldown
}
func (l *nrptRecoveryLimiter) allow(now time.Time, cfg *ctrld.Config) (bool, time.Duration) {
maxAttempts := nrptRecoveryMaxAttempts(cfg)
if maxAttempts <= 0 {
return true, 0
}
l.mu.Lock()
defer l.mu.Unlock()
if now.Before(l.cooldownUntil) {
return false, l.cooldownUntil.Sub(now)
}
return true, 0
}
func (l *nrptRecoveryLimiter) recordRecoveryFlow(now time.Time, cfg *ctrld.Config) {
maxAttempts := nrptRecoveryMaxAttempts(cfg)
if maxAttempts <= 0 {
return
}
cooldown := nrptRecoveryCooldown(cfg)
if cooldown <= 0 {
cooldown = defaultNRPTRecoveryCooldown
}
l.mu.Lock()
defer l.mu.Unlock()
l.stableSuccesses = 0
l.attempts++
if l.attempts >= maxAttempts {
l.cooldownUntil = now.Add(cooldown)
}
}
func (l *nrptRecoveryLimiter) recordStableSuccess() {
l.mu.Lock()
defer l.mu.Unlock()
l.stableSuccesses++
if l.stableSuccesses >= nrptRecoveryStableSuccessesToReset {
l.attempts = 0
l.cooldownUntil = time.Time{}
l.lastSkipLog = time.Time{}
}
}
func (l *nrptRecoveryLimiter) shouldLogSkip(now time.Time) bool {
l.mu.Lock()
defer l.mu.Unlock()
if l.lastSkipLog.IsZero() || now.Sub(l.lastSkipLog) >= 5*time.Minute {
l.lastSkipLog = now
return true
}
return false
}
@@ -0,0 +1,74 @@
//go:build windows
package cli
import (
"testing"
"time"
"github.com/Control-D-Inc/ctrld"
)
func TestNRPTRecoveryLimiterCooldownAndStableReset(t *testing.T) {
maxAttempts := 2
cooldown := 10 * time.Minute
cfg := &ctrld.Config{}
cfg.Service.NRPTRecoveryMaxAttempts = &maxAttempts
cfg.Service.NRPTRecoveryCooldown = &cooldown
limiter := &nrptRecoveryLimiter{}
now := time.Unix(100, 0)
if ok, wait := limiter.allow(now, cfg); !ok || wait != 0 {
t.Fatalf("initial allow = %v, %v; want true, 0", ok, wait)
}
limiter.recordRecoveryFlow(now, cfg)
if ok, wait := limiter.allow(now.Add(time.Second), cfg); !ok || wait != 0 {
t.Fatalf("allow after first flow = %v, %v; want true, 0", ok, wait)
}
limiter.recordRecoveryFlow(now.Add(2*time.Second), cfg)
if ok, wait := limiter.allow(now.Add(3*time.Second), cfg); ok || wait <= 0 {
t.Fatalf("allow after max flows = %v, %v; want false, positive wait", ok, wait)
}
limiter.recordStableSuccess()
if ok, _ := limiter.allow(now.Add(4*time.Second), cfg); ok {
t.Fatal("one stable success cleared cooldown; want cooldown to remain")
}
limiter.recordStableSuccess()
if ok, wait := limiter.allow(now.Add(5*time.Second), cfg); !ok || wait != 0 {
t.Fatalf("allow after stable reset = %v, %v; want true, 0", ok, wait)
}
}
func TestNRPTRecoveryLimiterDefaultIsUnlimited(t *testing.T) {
cfg := &ctrld.Config{}
limiter := &nrptRecoveryLimiter{}
now := time.Unix(100, 0)
for i := 0; i < 10; i++ {
limiter.recordRecoveryFlow(now.Add(time.Duration(i)*time.Second), cfg)
}
if ok, wait := limiter.allow(now.Add(time.Hour), cfg); !ok || wait != 0 {
t.Fatalf("default allow after recovery flows = %v, %v; want true, 0", ok, wait)
}
}
func TestNRPTRecoveryLimiterUnlimited(t *testing.T) {
maxAttempts := 0
cfg := &ctrld.Config{}
cfg.Service.NRPTRecoveryMaxAttempts = &maxAttempts
limiter := &nrptRecoveryLimiter{}
now := time.Unix(100, 0)
for i := 0; i < 10; i++ {
limiter.recordRecoveryFlow(now.Add(time.Duration(i)*time.Second), cfg)
}
if ok, wait := limiter.allow(now.Add(time.Hour), cfg); !ok || wait != 0 {
t.Fatalf("unlimited allow = %v, %v; want true, 0", ok, wait)
}
}
+9 -1
View File
@@ -72,7 +72,15 @@ func setDNS(iface *net.Interface, nameservers []string) error {
SearchDomains: []dnsname.FQDN{},
}
if sds, err := searchDomains(); err == nil {
osConfig.SearchDomains = sds
// Filter the root domain, since it's not allowed by systemd.
// See https://github.com/systemd/systemd/issues/9515
filteredSds := slices.DeleteFunc(sds, func(s dnsname.FQDN) bool {
return s == "" || s == "."
})
if len(filteredSds) != len(sds) {
mainLog.Load().Debug().Msg(`Removed root domain "." from search domains list`)
}
osConfig.SearchDomains = filteredSds
} else {
mainLog.Load().Debug().Err(err).Msg("failed to get search domains list")
}
+1 -1
View File
@@ -55,7 +55,7 @@ func setDNS(iface *net.Interface, nameservers []string) error {
mainLog.Load().Debug().Msgf("Existing forwarders content: %s", string(oldForwardersContent))
}
hasLocalIPv6Listener := needLocalIPv6Listener()
hasLocalIPv6Listener := needLocalIPv6Listener(interceptMode)
mainLog.Load().Debug().Bool("has_ipv6_listener", hasLocalIPv6Listener).Msg("IPv6 listener status")
forwarders := slices.DeleteFunc(slices.Clone(nameservers), func(s string) bool {
+79
View File
@@ -0,0 +1,79 @@
package cli
import (
"fmt"
"strings"
)
// pfNoRulesMarker is what pfctl prints for a ruleset that contains nothing.
const pfNoRulesMarker = "(no rules)"
// pfFilterRuleLines reduces pfctl output to the lines that are actually pf rules.
//
// It exists because every pfctl reader here uses CombinedOutput, and pfctl on macOS
// writes "No ALTQ support in kernel" and "ALTQ related functions disabled" to stderr on
// essentially every show command, so raw output is never a clean rule list. An empty
// ruleset can also report "(no rules)", which is a status line rather than a rule.
//
// Two consequences follow from getting this wrong, and both have bitten this file:
// callers that test the output for emptiness can never see empty, and callers that feed
// the lines back into "pfctl -f -" would splice non-rule text into a ruleset and have
// the reload rejected.
//
// Registry access and platform specifics stay elsewhere; this is pure string handling
// so it can be tested on any host.
func pfFilterRuleLines(output string) []string {
var rules []string
for _, line := range strings.Split(output, "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
// pfctl stderr warnings, merged in by CombinedOutput.
if strings.Contains(line, "ALTQ") {
continue
}
// Status line for an empty ruleset, not a rule.
if line == pfNoRulesMarker {
continue
}
rules = append(rules, line)
}
return rules
}
// pfRulesetEmpty reports whether pfctl output describes a ruleset with no rules.
//
// Use this rather than testing the raw output for emptiness: the merged stderr warnings
// described above mean a raw test is always false, so the condition it guards - an
// anchor whose contents were flushed - would never be detected.
func pfRulesetEmpty(output string) bool {
return len(pfFilterRuleLines(output)) == 0
}
// pfContainsRule checks if any line in the slice contains the given rule string.
// Uses substring matching because pfctl may append extra tokens like " all" to rules
// (e.g., `rdr-anchor "com.controld.ctrld" all`), which would fail exact matching.
func pfContainsRule(lines []string, rule string) bool {
for _, line := range lines {
if strings.Contains(line, rule) {
return true
}
}
return false
}
// pfAnchorReferencesPresent reports whether ctrld's anchor references appear in the
// running ruleset, given the output of "pfctl -sn" and "pfctl -sr".
//
// Removing the references means reloading the entire main ruleset, and that reload
// carries no options section - so it resets system-wide pf options, including any
// third-party "set skip" directives. Doing that when there is nothing of ours to
// remove is pure collateral damage, which is what a startup rollback would otherwise
// cause after failing before the references were ever added.
func pfAnchorReferencesPresent(natOutput, filterOutput, anchorName string) bool {
rdrAnchorRef := fmt.Sprintf("rdr-anchor %q", anchorName)
anchorRef := fmt.Sprintf("anchor %q", anchorName)
return pfContainsRule(pfFilterRuleLines(natOutput), rdrAnchorRef) ||
pfContainsRule(pfFilterRuleLines(filterOutput), anchorRef)
}
+157
View File
@@ -0,0 +1,157 @@
package cli
import "testing"
// altqNoise is what macOS pfctl writes to stderr on show commands. Because every
// pfctl reader here uses CombinedOutput, it lands in the middle of the data being
// parsed — which is why these helpers exist.
const altqNoise = "No ALTQ support in kernel\nALTQ related functions disabled\n"
// TestPFRulesetEmpty is the regression guard for a flushed anchor being undetectable.
//
// The anchor-content checks in verifyPFState and ensurePFAnchorActive decide whether pf
// still has ctrld's rules. Testing the raw pfctl output for emptiness can never be true
// on macOS, because the merged ALTQ warnings are always present — so a genuinely flushed
// anchor reads as healthy and neither the startup gate nor the watchdog restore fires.
func TestPFRulesetEmpty(t *testing.T) {
tests := []struct {
name string
output string
want bool
}{
{
// The case that was broken: nothing but merged stderr.
name: "only ALTQ warnings",
output: altqNoise,
want: true,
},
{
// As captured on macOS 26.6 from "pfctl -sn -a com.controld.ctrld".
name: "ALTQ warnings plus the empty-ruleset marker",
output: altqNoise + "(no rules)\n",
want: true,
},
{
name: "empty output",
output: "",
want: true,
},
{
name: "whitespace only",
output: "\n \n\t\n",
want: true,
},
{
name: "a real rdr rule behind the warnings",
output: altqNoise + "rdr on lo0 inet proto udp from any to ! 127.0.0.1 port = 53 -> 127.0.0.1 port 5354\n",
want: false,
},
{
name: "a real filter rule behind the warnings",
output: altqNoise + "pass in quick on lo0 reply-to lo0 inet proto udp from any to 127.0.0.1 port = 5354\n",
want: false,
},
{
name: "rule with no warnings at all",
output: "anchor \"com.controld.ctrld\" all\n",
want: false,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := pfRulesetEmpty(tc.output); got != tc.want {
t.Errorf("pfRulesetEmpty() = %v, want %v\noutput:\n%s", got, tc.want, tc.output)
}
})
}
}
// TestPFFilterRuleLines checks what survives filtering, since these lines are fed back
// into "pfctl -f -" by the ruleset-rebuild paths. Splicing a warning or the
// empty-ruleset marker into a ruleset would have the reload rejected outright.
func TestPFFilterRuleLines(t *testing.T) {
got := pfFilterRuleLines(altqNoise + "(no rules)\nrdr-anchor \"com.controld.ctrld\" all\n\nanchor \"com.controld.ctrld\" all\n")
want := []string{
`rdr-anchor "com.controld.ctrld" all`,
`anchor "com.controld.ctrld" all`,
}
if len(got) != len(want) {
t.Fatalf("got %d lines %q, want %d %q", len(got), got, len(want), want)
}
for i := range want {
if got[i] != want[i] {
t.Errorf("line %d = %q, want %q", i, got[i], want[i])
}
}
if lines := pfFilterRuleLines(altqNoise); lines != nil {
t.Errorf("warnings alone must yield no rule lines, got %q", lines)
}
}
// TestPFAnchorReferencesPresent guards when the main ruleset may be rewritten.
//
// Removing our anchor references means reloading the whole main ruleset, and that
// reload carries no options section — so it resets system-wide pf options, including
// third-party "set skip" directives. Startup rollback runs after failures that happen
// before the references were ever added, so without this check it would reset another
// application's pf options while removing nothing of ours.
func TestPFAnchorReferencesPresent(t *testing.T) {
const anchor = "com.controld.ctrld"
const otherAppRules = "scrub-anchor \"com.apple/*\" all fragment reassemble\nanchor \"com.vendor.vpn\" all\n"
tests := []struct {
name string
nat string
filter string
want bool
}{
{
name: "both references present",
nat: altqNoise + "rdr-anchor \"com.controld.ctrld\" all\n",
filter: altqNoise + "anchor \"com.controld.ctrld\" all\n",
want: true,
},
{
// pfctl appends tokens like " all", so matching is substring-based.
name: "rdr reference only",
nat: altqNoise + "rdr-anchor \"com.controld.ctrld\" all\n",
filter: altqNoise + otherAppRules,
want: true,
},
{
name: "filter reference only",
nat: altqNoise,
filter: altqNoise + "anchor \"com.controld.ctrld\"\n",
want: true,
},
{
// The rollback case: we failed before adding anything, and another
// application owns the ruleset. Rewriting it would be pure collateral.
name: "someone else's ruleset, none of ours",
nat: altqNoise,
filter: altqNoise + otherAppRules,
want: false,
},
{
name: "empty ruleset",
nat: altqNoise + "(no rules)\n",
filter: altqNoise + "(no rules)\n",
want: false,
},
{
// A different anchor whose name merely contains ours must not count.
name: "another anchor with a similar name",
nat: altqNoise,
filter: altqNoise + "anchor \"com.vendor.controld-shim\" all\n",
want: false,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := pfAnchorReferencesPresent(tc.nat, tc.filter, anchor); got != tc.want {
t.Errorf("pfAnchorReferencesPresent() = %v, want %v", got, tc.want)
}
})
}
}
+407 -21
View File
@@ -34,6 +34,7 @@ import (
"github.com/Control-D-Inc/ctrld/internal/clientinfo"
"github.com/Control-D-Inc/ctrld/internal/controld"
"github.com/Control-D-Inc/ctrld/internal/dnscache"
ctrldnet "github.com/Control-D-Inc/ctrld/internal/net"
"github.com/Control-D-Inc/ctrld/internal/router"
"github.com/Control-D-Inc/ctrld/internal/router/dnsmasq"
)
@@ -91,6 +92,16 @@ var svcConfig = &service.Config{
var useSystemdResolved = false
type pfAnchorCheckResult uint8
const (
pfAnchorCheckSkipped pfAnchorCheckResult = iota
pfAnchorCheckIntact
pfAnchorCheckRestored
pfAnchorCheckDeferred
pfAnchorCheckFailed
)
type prog struct {
mu sync.Mutex
waitCh chan struct{}
@@ -131,6 +142,7 @@ type prog struct {
runningIface string
requiredMultiNICsConfig bool
adDomain string
hasLocalDNS bool
runningOnDomainController bool
selfUninstallMu sync.Mutex
@@ -145,6 +157,105 @@ type prog struct {
recoveryCancel context.CancelFunc
recoveryRunning atomic.Bool
// recoveryDebounceTimer coalesces rapid NetworkChange recovery triggers
// into a single handleRecovery call. Only handleRecovery is debounced —
// all other state updates (IP, pf anchor, VPN DNS) run immediately.
recoveryDebounceMu sync.Mutex
recoveryDebounceTimer *time.Timer
// recoveryBypass is set when dns-intercept mode enters recovery.
// When true, proxy() forwards all queries to OS/DHCP resolver
// instead of using the normal upstream flow.
recoveryBypass atomic.Bool
// DNS intercept mode state (platform-specific).
// On Windows: *wfpState, on macOS: *pfState, nil on other platforms.
dnsInterceptState any
// dnsInterceptMu serializes DNS intercept lifecycle transitions - start, stop and
// the health monitor's rebuild - and guards every write to dnsInterceptState, so a
// service stop can never interleave with a monitor-driven rebuild.
dnsInterceptMu sync.Mutex //lint:ignore U1000 used on windows
// dnsInterceptStopRequested is set while a stop waits for dnsInterceptMu. The
// health and recovery flows read it as a shutdown signal and abandon their work,
// rather than making the stop wait out their probe backoffs.
dnsInterceptStopRequested atomic.Bool //lint:ignore U1000 used on windows
// nrptTransitionMu makes one NRPT ownership transition - observe, mutate, signal,
// record owner - atomic against shutdown and against another transition. It is
// deliberately finer-grained than dnsInterceptMu: it is taken for the duration of a
// single transition, never across the recovery flows' probe backoffs.
nrptTransitionMu sync.Mutex //lint:ignore U1000 used on windows
// lastTunnelIfaces tracks the tunnel set included in the last successfully loaded
// pf anchor. Pending tunnel state is kept separately so failed PF work is retried
// instead of being mistaken for an applied update. Protected by mu.
lastTunnelIfaces []string //lint:ignore U1000 used on darwin
pendingTunnelIfaces []string //lint:ignore U1000 used on darwin
hasPendingTunnelIfaces bool //lint:ignore U1000 used on darwin
// pfStabilizing is true while we're waiting for a VPN's pf ruleset to settle.
// While true, the watchdog and network change callbacks do NOT restore our rules.
pfStabilizing atomic.Bool
// pfStabilizeCancel cancels the active stabilization goroutine, if any.
// Protected by mu.
pfStabilizeCancel context.CancelFunc //lint:ignore U1000 used on darwin
// pfLastRestoreTime records when we last restored our anchor (unix millis).
// Used to detect immediate re-wipes (VPN reconnect cycle).
pfLastRestoreTime atomic.Int64 //lint:ignore U1000 used on darwin
// pfBackoffMultiplier tracks exponential backoff for stabilization.
// Resets to 0 when rules survive for >60s.
pfBackoffMultiplier atomic.Int32 //lint:ignore U1000 used on darwin
// pfMonitorRunning ensures only one pfInterceptMonitor goroutine runs at a time.
// When an interface appears/disappears, we spawn a monitor that probes pf
// interception with exponential backoff and auto-heals if broken.
pfMonitorRunning atomic.Bool //lint:ignore U1000 used on darwin
// pfEnsureRunning ensures only one pf validation or mutation runs at a time.
// Network callbacks, VPN exemption updates, delayed rechecks, probes, and the
// watchdog can converge during macOS churn; concurrent pfctl/scutil work can
// exhaust process/file limits or interleave anchor snapshots.
pfEnsureRunning atomic.Bool //lint:ignore U1000 used on darwin
// pfExecBackoffUntil suppresses pf anchor validation after pfctl/scutil execs
// fail due host resource exhaustion (fork unavailable, too many open files).
pfExecBackoffUntil atomic.Int64 //lint:ignore U1000 used on darwin
// pfDelayedRecheckTimers coalesces delayed DNS-intercept rechecks after noisy
// network changes. Protected by pfDelayedRecheckMu.
pfDelayedRecheckMu sync.Mutex //lint:ignore U1000 used on darwin
pfDelayedRecheckTimers []*time.Timer //lint:ignore U1000 used on darwin
// pfIgnoredChangeLastReconcile bounds immediate pf/VPN-DNS work for noisy
// ignored macOS network deltas. Tunnel changes bypass this limit, and the
// existing delayed checks provide a trailing reconciliation after churn.
pfIgnoredChangeLastReconcile atomic.Int64 //lint:ignore U1000 used on darwin
// interceptProbes maps the domain of each pending interception probe to the channel
// that probe waits on. A probe verifies that interception is actually translating or
// redirecting packets, not merely present in rule text: the DNS handler looks up
// incoming queries here and signals the matching waiter.
//
// It holds one entry per in-flight probe rather than a single slot, because probes do
// overlap - the health monitor, a handback and a heal cycle can each have one out at
// the same time - and a single slot means the last registration wins and the loser
// waits out its timeout for a query that was answered. A false failure then triggers
// recovery work that was not needed.
//
// Registrations are rare and lookups happen on every query, so the map is stored as
// an immutable snapshot behind an atomic: readers never take a lock, writers copy
// under interceptProbeMu.
interceptProbes atomic.Value // map[string]chan struct{}
interceptProbeMu sync.Mutex //lint:ignore U1000 written only by registerInterceptProbe, used on darwin/windows
// VPN DNS manager for split DNS routing when intercept mode is active.
vpnDNS *vpnDNSManager
started chan struct{}
onStartedDone chan struct{}
onStarted []func()
@@ -213,7 +324,8 @@ func (p *prog) runWait() {
continue
}
if cdUID != "" {
if rc, err := processCDFlags(newCfg); err != nil {
rc, err := p.fetchCDConfigBoundedByLifetime(newCfg)
if err != nil {
logger.Err(err).Msg("could not fetch ControlD config")
waitOldRunDone()
continue
@@ -225,6 +337,10 @@ func (p *prog) runWait() {
}
}
// Though the log configuration could not be changed during reloading, we still need to
// process the current flags here, so runtime internal logs can be used correctly.
processLogAndCacheFlags(v, newCfg)
waitOldRunDone()
p.mu.Lock()
@@ -260,6 +376,18 @@ func (p *prog) runWait() {
p.mu.Lock()
*p.cfg = *newCfg
// In DNS-intercept mode on macOS, the DNS listener is bound once at startup and is
// NOT re-bound on reload (see prog.run: serveDNS is started only when !reload). When
// the configured/generated port (e.g. 127.0.0.1:53) is unavailable at startup because
// mDNSResponder owns *:53, ctrld falls back to an alternate local port (e.g. 5354).
// The on-disk config still declares 53, so adopting it here would revert p.cfg to a
// port nothing is listening on, and the pf rdr rules/probes rebuilt from p.cfg would
// target a dead port. Since a reload cannot move the running listener anyway, keep
// p.cfg pointing at the actual bound listener. The on-disk config (written above) is
// left unchanged. See #551.
if dnsIntercept && runtime.GOOS == "darwin" {
preserveBoundListeners(p.cfg.Listener, curListener)
}
p.mu.Unlock()
logger.Notice().Msg("reloading config successfully")
@@ -271,8 +399,41 @@ func (p *prog) runWait() {
}
}
// preserveBoundListeners overrides the IP/Port of each listener in newListeners with the
// actual bound address from curListeners when they differ, logging the divergence. It is used
// on config reload in DNS-intercept mode where the running listener is never re-bound, so a
// port change on disk (e.g. reverting a fallback 5354 back to the generated 53) must not be
// applied to the in-memory config that drives pf rdr rules and probes.
//
// Preservation is limited to fallback-eligible (default/unset, i.e. 127.0.0.1:53) listeners.
// An explicit, non-default listener in the reloaded config is an intentional change that must
// be applied: tryUpdateListenerConfigIntercept binds explicit listeners exactly (no fallback),
// and the control-server reload handler detects the IP/port diff to trigger a restart that
// re-binds. Reverting an explicit change here would make that comparison return 200 instead of
// 201, silently dropping the new listener. See #551.
func preserveBoundListeners(newListeners, curListeners map[string]*ctrld.ListenerConfig) {
for n, curLc := range curListeners {
newLc := newListeners[n]
if newLc == nil || curLc == nil {
continue
}
if newLc.IP == curLc.IP && newLc.Port == curLc.Port {
continue
}
if isExplicitInterceptListener(newLc.IP, newLc.Port) {
continue
}
mainLog.Load().Info().
Str("configured", net.JoinHostPort(newLc.IP, strconv.Itoa(newLc.Port))).
Str("actual", net.JoinHostPort(curLc.IP, strconv.Itoa(curLc.Port))).
Msg("DNS intercept: preserving actual bound listener across reload; on-disk config port not applied to running listener")
newLc.IP = curLc.IP
newLc.Port = curLc.Port
}
}
func (p *prog) preRun() {
if iface == "auto" {
if iface == autoIface {
iface = defaultIfaceName()
p.requiredMultiNICsConfig = requiredMultiNICsConfig()
}
@@ -286,7 +447,12 @@ func (p *prog) postRun() {
p.runningOnDomainController = isDC
mainLog.Load().Debug().Msgf("running on domain controller: %t, role: %d", p.runningOnDomainController, roleInt)
}
p.resetDNS(false, false)
// A Windows organization can install a GP-owned NRPT catch-all before
// starting ctrld. Detect that policy before resetDNS touches adapter DNS;
// startDNSIntercept will then prove the rule functionally before adopting it.
if !p.skipInitialDNSReset() {
p.resetDNS(false, false)
}
ns := ctrld.InitializeOsResolver(false)
mainLog.Load().Debug().Msgf("initialized OS resolver with nameservers: %v", ns)
p.setDNS()
@@ -320,7 +486,12 @@ func (p *prog) apiConfigReload() {
}
doReloadApiConfig := func(forced bool, logger zerolog.Logger) {
resolverConfig, err := controld.FetchResolverConfig(cdUID, rootCmd.Version, cdDev)
req := &controld.ResolverConfigRequest{
RawUID: cdUID,
Version: rootCmd.Version,
Metadata: ctrld.SystemMetadataRuntime(context.Background()),
}
resolverConfig, err := controld.FetchResolverConfig(context.Background(), req, cdDev)
selfUninstallCheck(err, p, logger)
if err != nil {
logger.Warn().Err(err).Msg("could not fetch resolver config")
@@ -378,7 +549,7 @@ func (p *prog) apiConfigReload() {
}
if cfgErr != nil {
logger.Warn().Err(err).Msg("skipping invalid custom config")
if _, err := controld.UpdateCustomLastFailed(cdUID, rootCmd.Version, cdDev, true); err != nil {
if _, err := controld.UpdateCustomLastFailed(context.Background(), cdUID, rootCmd.Version, cdDev, true); err != nil {
logger.Error().Err(err).Msg("could not mark custom last update failed")
}
return
@@ -481,9 +652,13 @@ func (p *prog) run(reload bool, reloadCh chan struct{}) {
}
}
}
if domain, err := getActiveDirectoryDomain(); err == nil && domain != "" && hasLocalDnsServerRunning() {
if domain, err := getActiveDirectoryDomain(); err == nil && domain != "" {
mainLog.Load().Debug().Msgf("active directory domain: %s", domain)
p.adDomain = domain
if hasLocalDnsServerRunning() {
mainLog.Load().Debug().Msg("local DNS server detected (Domain Controller)")
p.hasLocalDNS = true
}
}
var wg sync.WaitGroup
@@ -527,7 +702,6 @@ func (p *prog) run(reload bool, reloadCh chan struct{}) {
defer wg.Done()
p.runClientInfoDiscover(ctx)
}()
go p.watchLinkState(ctx)
}
if !reload {
@@ -709,12 +883,147 @@ func (p *prog) deAllocateIP() error {
return nil
}
// Seams for the intercept-start failure lifecycle. Choosing between the interface-DNS
// fallback and refusing it has side effects - restoring the host's DNS, then
// terminating - which a test has to observe without reconfiguring the host or exiting
// the test binary. The intercept start itself is indirected for the same reason: it is
// the real platform interceptor, which on macOS mutates pf and on Windows installs an
// NRPT rule, so a test of what happens *after* it fails must not be the thing that
// runs it.
var (
localResolverIPFn = router.LocalResolverIP
startDNSInterceptFn = (*prog).startDNSIntercept
setDnsForRunningIfaceFn = (*prog).setDnsForRunningIface
resetDNSFn = (*prog).resetDNS
refuseFallbackFatal = func(format string, v ...any) {
mainLog.Load().Fatal().Msgf(format, v...)
}
)
// interfaceDNSFallbackViable reports whether the interface-DNS fallback can actually
// direct queries to ctrld's listener.
//
// Interface DNS names a resolver by IP and has no port field - true of macOS interface
// settings and of Windows NRPT rules - so pointing the system straight at a listener
// that did not bind :53 sends queries to whatever owns :53 instead, and that resolver's
// upstream is ctrld's address: a loop, not a fallback.
//
// A nil or portless listener is treated as viable: the port is resolved elsewhere and
// defaults to 53, so there is nothing to refuse yet.
//
// A non-53 listener is still viable where a local resolver owns :53 and forwards to
// ctrld's port. That is the arrangement on the router platforms with a dnsmasq of their
// own: ctrld writes "server=<listener ip>#<listener port>", so the forward follows
// whatever port ctrld actually bound. setDNS then points the interface at that resolver
// rather than at the listener - see the lc.Port != 53 case there, which this mirrors.
// Refusing on port alone would turn a working configuration into a startup failure on
// those routers.
func interfaceDNSFallbackViable(lc *ctrld.ListenerConfig, localResolverIP string) bool {
return lc == nil || lc.Port == 0 || lc.Port == 53 || localResolverIP != ""
}
func (p *prog) setDNS() {
setDnsOK := false
defer func() {
p.csSetDnsOk = setDnsOK
}()
// Validate and resolve intercept mode.
// CLI flag (--intercept-mode) takes priority over config file.
// Valid values: "" (use config), "off" (explicitly disable), "dns" (with VPN
// split routing), and "hard" (all DNS through ctrld).
if interceptMode != "" && !validInterceptMode(interceptMode) {
mainLog.Load().Fatal().Msgf("invalid --intercept-mode value %q: must be 'off', 'dns', or 'hard'", interceptMode)
}
if interceptMode == "" {
interceptMode = p.configuredInterceptMode()
if interceptMode != "" && interceptMode != "off" {
mainLog.Load().Info().Msgf("Intercept mode enabled via config (intercept_mode = %q)", interceptMode)
}
}
// Derive convenience bools from interceptMode.
switch interceptMode {
case "dns":
dnsIntercept = true
case "hard":
dnsIntercept = true
hardIntercept = true
}
// DNS intercept mode: use OS-level packet interception (WFP/pf) instead of
// modifying interface DNS settings. This eliminates race conditions with VPN
// software that also manages DNS. See issue #489.
if dnsIntercept {
if err := startDNSInterceptFn(p); err != nil {
// This check comes first: it is the one failure where DNS already works
// without ctrld touching anything else, so neither the refusal below nor the
// fallback applies.
//
// An externally managed rule was proved - by probe, not by registry shape -
// to be routing DNS to this listener. Falling through would rewrite adapter
// DNS after explicitly preserving it, and DNS still works, so stop here.
//
// Only a verified route earns this. A rule that merely exists does not: if it
// is not actually routing and intercept failed too, the machine would be left
// with no NRPT, no WFP and no adapter fallback - that is, unfiltered - so
// every other failure takes the paths below.
if interceptFailedUnderExternalDNSPolicy(err) {
if interceptFailedWithVerifiedExternalDNS(err) {
mainLog.Load().Error().Err(err).Msg("DNS intercept mode failed but externally managed DNS policy is verified routing to ctrld — not falling back to interface DNS settings")
} else {
// Owned by external policy but not proved to route: DNS is not
// reaching ctrld. Adapter DNS still stays as the organization set it,
// and setDnsOK stays false, so this start reports as failed until a
// probe succeeds.
mainLog.Load().Error().Err(err).Msg("DNS intercept mode failed and externally managed DNS policy is not routing to ctrld — leaving interface DNS settings untouched; the service is not ready")
}
return
}
// Interface DNS cannot express a port: macOS interface settings and Windows
// NRPT rules both name a resolver by IP alone. So it is only a usable
// fallback when the listener actually bound :53. When something else owns
// :53 - mDNSResponder on macOS, which is the whole reason the :5354 fallback
// exists - pointing the system at 127.0.0.1 hands queries to that other
// resolver, whose own upstream is now ctrld's address. That is a resolution
// loop, not degraded operation: a healthy ctrld listener nothing on the host
// can reach, no working DNS, and no recovery short of stopping the service.
//
// Refuse instead, after putting the host's own DNS back. A visible startup
// failure beats DNS that is broken by design, and it stops a fallback that
// cannot work from quietly undoing the fail-closed verification above.
if lc := cfg.FirstListener(); !interfaceDNSFallbackViable(lc, localResolverIPFn()) {
mainLog.Load().Error().Err(err).Msgf("DNS intercept mode failed with the listener on port %d", lc.Port)
// Leave the host resolvable: restore static settings or DHCP rather than
// exiting with an interface still pointed at a ctrld that is not serving.
resetDNSFn(p, false, true)
refuseFallbackFatal("Refusing to fall back to interface DNS: it cannot direct queries to %s:%d, which would leave this host with no working resolver. Free port 53 for ctrld, or resolve the intercept failure, then start again.", lc.IP, lc.Port)
// Unreachable in production - the line above exits - but returning
// explicitly keeps the refusal from depending on that, so nothing can
// fall through to installing the fallback this just rejected.
return
}
mainLog.Load().Error().Err(err).Msg("DNS intercept mode failed — falling back to interface DNS settings")
// Fall through to traditional setDNS behavior.
} else {
if hardIntercept {
mainLog.Load().Info().Msg("Hard intercept mode active — all DNS through ctrld, no VPN split routing")
} else {
mainLog.Load().Info().Msg("DNS intercept mode active — skipping interface DNS configuration and watchdog")
// Initialize VPN DNS manager for split DNS routing.
// Discovers search domains from virtual/VPN interfaces and forwards
// matching queries to the DNS server on that interface.
// Skipped in --intercept-mode hard where all DNS goes through ctrld.
p.vpnDNS = newVPNDNSManager(p.exemptVPNDNSServers)
p.vpnDNS.Refresh(true)
}
setDnsOK = true
return
}
}
if cfg.Listener == nil {
return
}
@@ -729,7 +1038,7 @@ func (p *prog) setDNS() {
ns = "127.0.0.1"
case lc.Port != 53:
ns = "127.0.0.1"
if resolver := router.LocalResolverIP(); resolver != "" {
if resolver := localResolverIPFn(); resolver != "" {
ns = resolver
}
default:
@@ -741,14 +1050,14 @@ func (p *prog) setDNS() {
if needRFC1918Listeners(lc) {
nameservers = append(nameservers, ctrld.Rfc1918Addresses()...)
}
if needLocalIPv6Listener() {
if needLocalIPv6Listener(p.cfg.Service.InterceptMode) {
nameservers = append(nameservers, "::1")
}
slices.Sort(nameservers)
netIfaceName := ""
netIface := p.setDnsForRunningIface(nameservers)
netIface := setDnsForRunningIfaceFn(p, nameservers)
if netIface != nil {
netIfaceName = netIface.Name
}
@@ -781,6 +1090,17 @@ func (p *prog) setDNS() {
}
}
// configuredInterceptMode resolves the service's effective intercept mode without
// mutating package state. Platform startup preflights use the same precedence as
// setDNS so they do not make adapter-DNS decisions from a different mode value.
func (p *prog) configuredInterceptMode() string {
im := interceptMode
if im == "" || im == "off" {
im = p.cfg.Service.InterceptMode
}
return im
}
func (p *prog) setDnsForRunningIface(nameservers []string) (runningIface *net.Interface) {
if p.runningIface == "" {
return
@@ -936,7 +1256,18 @@ func (p *prog) dnsWatchdog(iface *net.Interface, nameservers []string) {
}
// resetDNS performs a DNS reset for all interfaces.
// In DNS intercept mode, this tears down the WFP/pf filters instead.
func (p *prog) resetDNS(isStart bool, restoreStatic bool) {
if dnsIntercept && p.dnsInterceptState != nil {
if err := p.stopDNSIntercept(); err != nil {
mainLog.Load().Error().Err(err).Msg("Failed to stop DNS intercept mode during reset")
}
// Clean up VPN DNS manager
p.vpnDNS = nil
return
}
netIfaceName := ""
if netIface := p.resetDNSForRunningIface(isStart, restoreStatic); netIface != nil {
netIfaceName = netIface.Name
@@ -1200,37 +1531,80 @@ func errAddrInUse(err error) bool {
var _ = errAddrInUse
// The unreachable winsock errnos (ENETUNREACH/EHOSTUNREACH) are matched via
// ctrldnet.IsUnreachable, which owns their definitions.
//
// https://learn.microsoft.com/en-us/windows/win32/winsock/windows-sockets-error-codes-2
var (
windowsECONNREFUSED = syscall.Errno(10061)
windowsENETUNREACH = syscall.Errno(10051)
windowsEINVAL = syscall.Errno(10022)
windowsEADDRINUSE = syscall.Errno(10048)
windowsEHOSTUNREACH = syscall.Errno(10065)
)
// errUrlNetworkError reports whether a failed HTTP attempt is worth retrying.
//
// The two-attempt paths compose one *url.Error per attempt - hostname first, then the
// direct-IP fallback - so this walks them in order rather than classifying only the first
// one errors.As happens to find. Each attempt can say one of three things:
//
// - retryable (unreachable, refused, temporary): retry, whichever attempt said it;
// - a name-resolution failure: no verdict. Only the hostname attempt resolves DNS, and
// at boot behind a captive portal or before the router's forwarder is up it fails
// this way while the network is merely not ready yet. Consult the next attempt;
// - anything else, notably a locally denied socket (WSAEACCES from a firewall blocking
// ctrld): definitive. Stop, because retrying cannot clear it - the Firewall Mode
// incident spent 256 retry cycles against filters that were never going to clear.
func errUrlNetworkError(err error) bool {
var urlErr *url.Error
if errors.As(err, &urlErr) {
return errNetworkError(urlErr.Err)
for _, attempt := range attemptErrors(err) {
var urlErr *url.Error
if !errors.As(attempt, &urlErr) {
continue
}
switch {
case errNetworkError(urlErr.Err):
return true
case errDNSResolutionFailure(urlErr.Err):
// Neutral; let a later attempt decide.
default:
return false
}
}
return false
}
// attemptErrors returns the per-attempt errors recorded in err, in the order they were
// tried. A composed fallback error wraps one per attempt; anything else is a single
// attempt.
func attemptErrors(err error) []error {
if multi, ok := err.(interface{ Unwrap() []error }); ok {
return multi.Unwrap()
}
return []error{err}
}
// errDNSResolutionFailure reports whether err is a name-resolution failure. Go marks a
// *net.DNSError as temporary only for socket failures that reached the server, so a
// SERVFAIL or "no such host" answer is not temporary - but it is also not evidence that
// retrying is pointless, which is why callers treat it as no verdict.
func errDNSResolutionFailure(err error) bool {
var dnsErr *net.DNSError
return errors.As(err, &dnsErr)
}
func errNetworkError(err error) bool {
var opErr *net.OpError
if errors.As(err, &opErr) {
if opErr.Temporary() {
return true
}
if ctrldnet.IsUnreachable(err) {
return true
}
switch {
case errors.Is(opErr.Err, syscall.ECONNREFUSED),
errors.Is(opErr.Err, syscall.EINVAL),
errors.Is(opErr.Err, syscall.ENETUNREACH),
errors.Is(opErr.Err, windowsENETUNREACH),
errors.Is(opErr.Err, windowsEINVAL),
errors.Is(opErr.Err, windowsECONNREFUSED),
errors.Is(opErr.Err, windowsEHOSTUNREACH):
errors.Is(opErr.Err, windowsECONNREFUSED):
return true
}
}
@@ -1523,6 +1897,19 @@ func shouldUpgrade(vt string, cv *semver.Version, logger *zerolog.Logger) bool {
return true
}
// newUpgradeCmd builds the detached command used to self-upgrade. It is a
// package-level variable so tests can stub it. With the real implementation a
// *test* binary would re-exec itself — os.Executable() is the test binary, and
// because `go test` stops flag parsing at the first positional arg ("upgrade")
// it ignores the args and re-runs the entire suite. That child hits the same
// upgrade test and spawns another child, recursively: a fork bomb of detached
// processes that pins the host and locks the test binary's image file.
var newUpgradeCmd = func(exe string) *exec.Cmd {
cmd := exec.Command(exe, "upgrade", "prod", "-vv")
cmd.SysProcAttr = sysProcAttrForDetachedChildProcess()
return cmd
}
// performUpgrade executes the self-upgrade command.
// Returns true if upgrade was initiated successfully, false otherwise.
func performUpgrade(vt string) bool {
@@ -1531,8 +1918,7 @@ func performUpgrade(vt string) bool {
mainLog.Load().Error().Err(err).Msg("failed to get executable path, skipped self-upgrade")
return false
}
cmd := exec.Command(exe, "upgrade", "prod", "-vv")
cmd.SysProcAttr = sysProcAttrForDetachedChildProcess()
cmd := newUpgradeCmd(exe)
if err := cmd.Start(); err != nil {
mainLog.Load().Error().Err(err).Msg("failed to start self-upgrade")
return false
+237
View File
@@ -0,0 +1,237 @@
package cli
import (
"errors"
"fmt"
"net"
"slices"
"strings"
"testing"
"github.com/Control-D-Inc/ctrld"
)
// TestInterfaceDNSFallbackViable covers when the interface-DNS fallback may be used
// after DNS intercept fails to start.
//
// The fallback names a resolver by IP with no port, so it can only reach a listener on
// :53. Taking it with the listener on a redirect-dependent port produced a total DNS
// outage on macOS: the interface points at 127.0.0.1, mDNSResponder answers there, and
// its upstream is ctrld's own address - a resolution loop with a healthy ctrld listener
// nothing can reach. Intercept startup refuses the fallback in that case rather than
// creating it.
func TestInterfaceDNSFallbackViable(t *testing.T) {
tests := []struct {
name string
lc *ctrld.ListenerConfig
localResolver string
want bool
}{
{
name: "listener on 53 can be reached by interface DNS",
lc: &ctrld.ListenerConfig{IP: "127.0.0.1", Port: 53},
want: true,
},
{
// The reported outage: no local resolver, so the :5354 fallback port
// cannot be expressed by interface DNS.
name: "listener on the fallback port cannot",
lc: &ctrld.ListenerConfig{IP: "127.0.0.1", Port: 5354},
want: false,
},
{
name: "any other non-53 port cannot",
lc: &ctrld.ListenerConfig{IP: "127.0.0.1", Port: 5300},
want: false,
},
{
// Router platforms with their own dnsmasq: it owns :53 and forwards to
// ctrld's port, so interface DNS reaches the listener through it.
// Refusing here would break a working EdgeOS/Firewalla setup.
name: "non-53 listener behind a forwarding local resolver",
lc: &ctrld.ListenerConfig{IP: "127.0.0.1", Port: 5354},
localResolver: "192.168.1.1",
want: true,
},
{
// Port is resolved elsewhere and defaults to 53; nothing to refuse yet.
name: "unset port is not refused",
lc: &ctrld.ListenerConfig{IP: "127.0.0.1"},
want: true,
},
{
name: "no listener is not refused",
lc: nil,
want: true,
},
{
// A non-loopback listener on 53 is still reachable by IP.
name: "non-loopback listener on 53",
lc: &ctrld.ListenerConfig{IP: "192.168.1.10", Port: 53},
want: true,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := interfaceDNSFallbackViable(tc.lc, tc.localResolver); got != tc.want {
t.Errorf("interfaceDNSFallbackViable() = %v, want %v", got, tc.want)
}
})
}
}
// interceptFallbackHarness drives setDNS() through the intercept-start failure path and
// records the side effects that decide whether the host ends up with a working
// resolver.
//
// Every host-touching step is stubbed, including the intercept start itself: this test
// runs untagged on Linux, macOS and Windows runners, where the real startDNSIntercept
// would set up pf or install an NRPT rule on the machine running the tests. Stubbing it
// also makes the precondition deterministic - the failure under test is injected rather
// than depending on the runner denying a privileged operation.
type interceptFallbackHarness struct {
interceptCalls int
installedNameservers []string
installCalls int
resetCalls int
refusals []string
}
func newInterceptFallbackHarness(t *testing.T, lc *ctrld.ListenerConfig) *interceptFallbackHarness {
t.Helper()
h := &interceptFallbackHarness{}
origStart, origInstall := startDNSInterceptFn, setDnsForRunningIfaceFn
origReset, origFatal := resetDNSFn, refuseFallbackFatal
origResolver := localResolverIPFn
origCfg, origMode, origIntercept, origHard := cfg, interceptMode, dnsIntercept, hardIntercept
t.Cleanup(func() {
startDNSInterceptFn, setDnsForRunningIfaceFn = origStart, origInstall
resetDNSFn, refuseFallbackFatal = origReset, origFatal
localResolverIPFn = origResolver
cfg, interceptMode, dnsIntercept, hardIntercept = origCfg, origMode, origIntercept, origHard
})
// Default to no local resolver: the desktop case. Router cases set it per test.
localResolverIPFn = func() string { return "" }
// Never reach the real interceptor: it would configure pf on macOS and NRPT on
// Windows, on the machine running the tests.
startDNSInterceptFn = func(_ *prog) error {
h.interceptCalls++
return errors.New("dns intercept: injected start failure")
}
setDnsForRunningIfaceFn = func(_ *prog, nameservers []string) *net.Interface {
h.installCalls++
h.installedNameservers = nameservers
return nil
}
resetDNSFn = func(_ *prog, _ bool, _ bool) { h.resetCalls++ }
refuseFallbackFatal = func(format string, v ...any) {
h.refusals = append(h.refusals, fmt.Sprintf(format, v...))
}
cfg = ctrld.Config{}
cfg.Service.InterceptMode = "dns"
cfg.Listener = map[string]*ctrld.ListenerConfig{"0": lc}
watchdogOff := false
cfg.Service.DnsWatchdogEnabled = &watchdogOff
interceptMode, dnsIntercept, hardIntercept = "dns", false, false
return h
}
func (h *interceptFallbackHarness) run(t *testing.T) {
t.Helper()
p := &prog{cfg: &cfg}
p.setDNS()
}
func TestSetDNSExplicitOffOverridesConfig(t *testing.T) {
h := newInterceptFallbackHarness(t, &ctrld.ListenerConfig{IP: "127.0.0.1", Port: 53})
interceptMode = "off"
dnsIntercept = false
hardIntercept = false
h.run(t)
if h.interceptCalls != 0 {
t.Fatalf("intercept start called %d time(s), want 0: explicit off must override service.intercept_mode", h.interceptCalls)
}
if h.installCalls != 1 {
t.Fatalf("interface DNS installed %d time(s), want 1", h.installCalls)
}
}
// TestSetDNSRefusesUnreachableFallback is the behaviour test for the reported outage: it
// drives the real setDNS() lifecycle rather than the classification helper alone.
//
// Deleting or bypassing the guard in setDNS makes the first case fail, because interface
// DNS then gets installed pointing at a listener that cannot answer on :53 - which is
// the resolution loop this refuses to create.
func TestSetDNSRefusesUnreachableFallback(t *testing.T) {
t.Run("non-53 listener refuses the fallback and restores DNS", func(t *testing.T) {
h := newInterceptFallbackHarness(t, &ctrld.ListenerConfig{IP: "127.0.0.1", Port: 5354})
h.run(t)
if h.interceptCalls != 1 {
t.Fatalf("intercept start called %d time(s) through the seam, want 1 — the real platform interceptor must never run here", h.interceptCalls)
}
if h.installCalls != 0 {
t.Errorf("interface DNS was installed %d time(s) for a listener on :5354 — that is the resolver loop", h.installCalls)
}
if h.resetCalls == 0 {
t.Error("host DNS was not restored before refusing, leaving the interface pointed at a ctrld that is not serving")
}
if len(h.refusals) == 0 {
t.Fatal("refusal was not surfaced: startup must fail loudly rather than silently skip the fallback")
}
if !strings.Contains(h.refusals[0], "5354") {
t.Errorf("refusal does not name the unreachable port: %q", h.refusals[0])
}
})
t.Run("non-53 listener behind a local resolver still falls back", func(t *testing.T) {
// EdgeOS/Firewalla: dnsmasq owns :53 and forwards to ctrld's port, so the
// fallback works and must not be refused. setDNS points the interface at the
// resolver rather than at the listener.
h := newInterceptFallbackHarness(t, &ctrld.ListenerConfig{IP: "127.0.0.1", Port: 5354})
localResolverIPFn = func() string { return "192.168.1.1" }
h.run(t)
if h.installCalls != 1 {
t.Errorf("interface DNS installed %d time(s), want 1: a forwarding local resolver makes the fallback usable", h.installCalls)
}
if len(h.refusals) != 0 {
t.Errorf("refused a fallback that a local resolver can serve: %v", h.refusals)
}
// Assert on membership, not on the exact set: setDNS appends platform-dependent
// entries beside the chosen nameserver - "::1" on Windows for the local IPv6
// listener, the RFC1918 addresses where those listeners are needed. What matters
// is that the interface points at the resolver and not at the listener IP, whose
// port the interface cannot express.
if !slices.Contains(h.installedNameservers, "192.168.1.1") {
t.Errorf("nameservers = %v, want the local resolver among them so queries reach ctrld through it", h.installedNameservers)
}
if slices.Contains(h.installedNameservers, "127.0.0.1") {
t.Errorf("nameservers = %v, must not name the listener IP: interface DNS cannot reach it on :5354", h.installedNameservers)
}
})
t.Run("listener on 53 still reaches the interface-DNS fallback", func(t *testing.T) {
h := newInterceptFallbackHarness(t, &ctrld.ListenerConfig{IP: "127.0.0.1", Port: 53})
h.run(t)
if h.interceptCalls != 1 {
t.Fatalf("intercept start called %d time(s) through the seam, want 1", h.interceptCalls)
}
if h.installCalls != 1 {
t.Errorf("interface DNS installed %d time(s), want 1: a listener on :53 is reachable, so the fallback must still apply", h.installCalls)
}
if len(h.refusals) != 0 {
t.Errorf("unexpected refusal for a reachable listener: %v", h.refusals)
}
if len(h.installedNameservers) == 0 {
t.Error("fallback installed no nameservers")
}
})
}
+3
View File
@@ -14,6 +14,9 @@ import (
)
func init() {
if isAndroid() {
return
}
if r, err := newLoopbackOSConfigurator(); err == nil {
useSystemdResolved = r.Mode() == "systemd-resolved"
}
+32
View File
@@ -1,7 +1,11 @@
package cli
import (
"context"
"net"
"net/url"
"runtime"
"syscall"
"testing"
"time"
@@ -12,6 +16,32 @@ import (
"github.com/Control-D-Inc/ctrld"
)
func TestErrNetworkErrorTreatsNoRouteAsNetworkError(t *testing.T) {
err := &net.OpError{Op: "dial", Net: "tcp", Err: syscall.EHOSTUNREACH}
assert.True(t, errNetworkError(err))
assert.True(t, errUrlNetworkError(&url.Error{Op: "Get", URL: "https://dns.controld.com", Err: err}))
}
func TestSleepWithContext(t *testing.T) {
assert.True(t, sleepWithContext(context.Background(), time.Millisecond))
ctx, cancel := context.WithCancel(context.Background())
cancel()
start := time.Now()
assert.False(t, sleepWithContext(ctx, time.Minute))
assert.Less(t, time.Since(start), 100*time.Millisecond)
}
func TestUnreachableRecoveryBackoff(t *testing.T) {
// Streak starts at the base cadence and doubles each attempt, capped at the max.
assert.Equal(t, checkUpstreamBackoffSleep, unreachableRecoveryBackoff(0))
assert.Equal(t, checkUpstreamBackoffSleep, unreachableRecoveryBackoff(1))
assert.Equal(t, 2*checkUpstreamBackoffSleep, unreachableRecoveryBackoff(2))
assert.Equal(t, 4*checkUpstreamBackoffSleep, unreachableRecoveryBackoff(3))
assert.Equal(t, checkUpstreamUnreachableBackoffMax, unreachableRecoveryBackoff(100))
}
func Test_prog_dnsWatchdogEnabled(t *testing.T) {
p := &prog{cfg: &ctrld.Config{}}
@@ -262,6 +292,8 @@ func Test_performUpgrade(t *testing.T) {
},
}
// newUpgradeCmd is stubbed in TestMain so performUpgrade does not re-exec
// (and fork-bomb) the test binary; see the comment there.
for _, tc := range tests {
tc := tc
t.Run(tc.name, func(t *testing.T) {
+247
View File
@@ -0,0 +1,247 @@
package cli
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"time"
"unicode/utf8"
)
// A terminal provisioning failure reports the same stable code on three
// surfaces: a persisted result file, one fixed-format output line, and a
// stage-scoped process exit code. docs/provisioning-failure-codes.md maps
// each code to its scenario and must stay in sync with the constants below.
// Codes are append-only once released; renaming or reusing one breaks the
// support contract.
type provisionStage string
const (
provisionStageBootstrap provisionStage = "bootstrap"
provisionStageListener provisionStage = "listener"
provisionStageService provisionStage = "service"
)
type provisionFailureCode string
const (
provisionCodeAPIUnreachable provisionFailureCode = "API_UNREACHABLE"
provisionCodeAPIRejected provisionFailureCode = "API_REJECTED"
provisionCodeAPIDeviceInvalid provisionFailureCode = "API_DEVICE_INVALID"
provisionCodeListenerBindFailed provisionFailureCode = "LISTENER_BIND_FAILED"
provisionCodeListenerAddrUnavail provisionFailureCode = "LISTENER_CONFIGURED_ADDR_UNAVAILABLE"
provisionCodeServiceInstall provisionFailureCode = "SERVICE_INSTALL_FAILED"
provisionCodeServiceStartFailed provisionFailureCode = "SERVICE_START_FAILED"
provisionCodeServiceSelfCheck provisionFailureCode = "SERVICE_SELFCHECK_FAILED"
)
var allProvisionFailureCodes = []provisionFailureCode{
provisionCodeAPIUnreachable,
provisionCodeAPIRejected,
provisionCodeAPIDeviceInvalid,
provisionCodeListenerBindFailed,
provisionCodeListenerAddrUnavail,
provisionCodeServiceInstall,
provisionCodeServiceStartFailed,
provisionCodeServiceSelfCheck,
}
var provisionStageForCode = map[provisionFailureCode]provisionStage{
provisionCodeAPIUnreachable: provisionStageBootstrap,
provisionCodeAPIRejected: provisionStageBootstrap,
provisionCodeAPIDeviceInvalid: provisionStageBootstrap,
provisionCodeListenerBindFailed: provisionStageListener,
provisionCodeListenerAddrUnavail: provisionStageListener,
provisionCodeServiceInstall: provisionStageService,
provisionCodeServiceStartFailed: provisionStageService,
provisionCodeServiceSelfCheck: provisionStageService,
}
// Exit codes are grouped by stage (bootstrap 30-39, listener 40-49, service
// 50-59) so the exit code alone names the failed stage. 0-3 belong to
// "ctrld status" and 126 to the deactivation pin check; never reuse those.
var provisionExitCodeForCode = map[provisionFailureCode]int{
provisionCodeAPIUnreachable: 30,
provisionCodeAPIRejected: 31,
provisionCodeAPIDeviceInvalid: 32,
provisionCodeListenerBindFailed: 41,
provisionCodeListenerAddrUnavail: 42,
provisionCodeServiceInstall: 51,
provisionCodeServiceStartFailed: 52,
provisionCodeServiceSelfCheck: 53,
}
const (
provisionResultFileName = "provision_result.json"
// Detail identifies a failure, it is not a log. Caps keep the artifact
// small and predictable.
maxProvisionBindAttempts = 12
maxProvisionStringLen = 256
)
type provisionBindAttempt struct {
Addr string `json:"addr"`
Proto string `json:"proto"`
OSError string `json:"os_error"`
}
type provisionDetail struct {
Attempts []provisionBindAttempt `json:"attempts,omitempty"`
}
type provisionResult struct {
Version int `json:"version"`
Timestamp string `json:"timestamp"`
Stage string `json:"stage"`
Code string `json:"code"`
ExitCode int `json:"exit_code"`
Message string `json:"message"`
Detail *provisionDetail `json:"detail,omitempty"`
}
// provisionResultPath is a var so tests can point it at a temp dir.
var provisionResultPath = func() string {
return absHomeDir(provisionResultFileName)
}
// provisionExit is a var so tests can observe the exit code instead of dying.
var provisionExit = os.Exit
// newProvisionResult builds a result with every field bounded and the given
// secrets stripped. The artifact reaches installer logs and support tickets,
// so callers pass every secret in scope (provision token, cd UID).
func newProvisionResult(code provisionFailureCode, message string, attempts []provisionBindAttempt, secrets ...string) *provisionResult {
sanitize := func(s string) string {
s = redactSecrets(s, secrets...)
if len(s) > maxProvisionStringLen {
// Cut on a rune boundary so a localized OS error does not end in
// a broken multi-byte sequence.
cut := maxProvisionStringLen
for cut > 0 && !utf8.RuneStart(s[cut]) {
cut--
}
s = s[:cut]
}
return s
}
r := &provisionResult{
Version: 1,
Timestamp: time.Now().UTC().Format(time.RFC3339),
Stage: string(provisionStageForCode[code]),
Code: string(code),
ExitCode: provisionExitCodeForCode[code],
Message: sanitize(message),
}
if len(attempts) > 0 {
if len(attempts) > maxProvisionBindAttempts {
attempts = attempts[:maxProvisionBindAttempts]
}
detail := &provisionDetail{Attempts: make([]provisionBindAttempt, 0, len(attempts))}
for _, a := range attempts {
detail.Attempts = append(detail.Attempts, provisionBindAttempt{
Addr: sanitize(a.Addr),
Proto: sanitize(a.Proto),
OSError: sanitize(a.OSError),
})
}
r.Detail = detail
}
return r
}
// redactSecrets removes every non-empty secret from s.
func redactSecrets(s string, secrets ...string) string {
for _, secret := range secrets {
if secret == "" {
continue
}
s = strings.ReplaceAll(s, secret, "[redacted]")
}
return s
}
// provisionResultTrusted rejects a result whose code, stage, or exit code is
// not part of the known contract, so a corrupt or planted file cannot drive
// what "ctrld start" logs and exits with.
func provisionResultTrusted(r *provisionResult) bool {
code := provisionFailureCode(r.Code)
stage, ok := provisionStageForCode[code]
if !ok {
return false
}
return r.Stage == string(stage) && r.ExitCode == provisionExitCodeForCode[code]
}
func (r *provisionResult) failureLine() string {
return fmt.Sprintf("provisioning failed: stage=%s code=%s (exit %d)", r.Stage, r.Code, r.ExitCode)
}
// writeProvisionResult persists the result atomically (temp file + rename in
// the same directory) so a reader never sees a partial file.
func writeProvisionResult(r *provisionResult) error {
path := provisionResultPath()
buf, err := json.MarshalIndent(r, "", " ")
if err != nil {
return err
}
tmp, err := os.CreateTemp(filepath.Dir(path), provisionResultFileName+".tmp*")
if err != nil {
return err
}
tmpName := tmp.Name()
if _, err := tmp.Write(buf); err != nil {
_ = tmp.Close()
_ = os.Remove(tmpName)
return err
}
if err := tmp.Close(); err != nil {
_ = os.Remove(tmpName)
return err
}
if err := os.Chmod(tmpName, 0o600); err != nil {
_ = os.Remove(tmpName)
return err
}
if err := os.Rename(tmpName, path); err != nil {
_ = os.Remove(tmpName)
return err
}
return nil
}
func readProvisionResult() (*provisionResult, error) {
buf, err := os.ReadFile(provisionResultPath())
if err != nil {
return nil, err
}
r := &provisionResult{}
if err := json.Unmarshal(buf, r); err != nil {
return nil, err
}
return r, nil
}
// clearProvisionResult removes a stale result once provisioning succeeds, so
// support never diagnoses a healthy install from an old failure.
func clearProvisionResult() {
if err := os.Remove(provisionResultPath()); err != nil && !os.IsNotExist(err) {
mainLog.Load().Debug().Err(err).Msg("could not remove provision result file")
}
}
// failProvision persists the result, prints the identifier line, unblocks a
// waiting "ctrld start" via notify, then exits with the stage code. The write
// comes first so the file survives even if logging or notify misbehaves.
func failProvision(r *provisionResult, notify func()) {
if err := writeProvisionResult(r); err != nil {
mainLog.Load().Warn().Err(err).Msg("could not persist provision result")
}
mainLog.Load().Error().Msg(r.failureLine())
if notify != nil {
notify()
}
provisionExit(r.ExitCode)
}
+278
View File
@@ -0,0 +1,278 @@
package cli
import (
"encoding/json"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
"unicode/utf8"
)
func overrideProvisionResultPath(t *testing.T) string {
t.Helper()
path := filepath.Join(t.TempDir(), provisionResultFileName)
old := provisionResultPath
provisionResultPath = func() string { return path }
t.Cleanup(func() { provisionResultPath = old })
return path
}
func TestProvisionCodesMapToOneStageAndInRangeExit(t *testing.T) {
stageRanges := map[provisionStage][2]int{
provisionStageBootstrap: {30, 39},
provisionStageListener: {40, 49},
provisionStageService: {50, 59},
}
reservedExits := map[int]string{
statusExitRunning: "ctrld status running",
statusExitStopped: "ctrld status stopped",
statusExitUnknown: "ctrld status unknown",
statusExitNotReady: "ctrld status not ready",
deactivationPinInvalidExitCode: "deactivation pin invalid",
}
seenExits := make(map[int]provisionFailureCode)
for _, code := range allProvisionFailureCodes {
stage, ok := provisionStageForCode[code]
if !ok {
t.Fatalf("code %s has no stage", code)
}
exit, ok := provisionExitCodeForCode[code]
if !ok {
t.Fatalf("code %s has no exit code", code)
}
r := stageRanges[stage]
if exit < r[0] || exit > r[1] {
t.Errorf("code %s exit %d outside stage %s range %v", code, exit, stage, r)
}
if owner, ok := reservedExits[exit]; ok {
t.Errorf("code %s exit %d collides with %s", code, exit, owner)
}
if prev, dup := seenExits[exit]; dup {
t.Errorf("codes %s and %s share exit %d", prev, code, exit)
}
seenExits[exit] = code
}
if len(allProvisionFailureCodes) != 8 {
t.Errorf("expected 8 codes, got %d", len(allProvisionFailureCodes))
}
}
func TestNewProvisionResultRedactsSecrets(t *testing.T) {
token := "org-secret-token-12345"
cdUIDValue := "abcdef123456"
attempts := []provisionBindAttempt{
{Addr: "127.0.0.1:53", Proto: "udp", OSError: "bind failed for " + token},
}
r := newProvisionResult(
provisionCodeListenerBindFailed,
"could not bind, token="+token+" uid="+cdUIDValue,
attempts,
token, cdUIDValue,
)
raw, err := json.Marshal(r)
if err != nil {
t.Fatal(err)
}
for _, secret := range []string{token, cdUIDValue} {
if strings.Contains(string(raw), secret) {
t.Errorf("serialized result contains secret %q: %s", secret, raw)
}
}
}
func TestNewProvisionResultBoundsDetail(t *testing.T) {
long := strings.Repeat("x", 1000)
var attempts []provisionBindAttempt
for i := 0; i < 50; i++ {
attempts = append(attempts, provisionBindAttempt{Addr: long, Proto: "udp", OSError: long})
}
r := newProvisionResult(provisionCodeListenerBindFailed, long, attempts)
if got := len(r.Detail.Attempts); got > maxProvisionBindAttempts {
t.Errorf("attempts not capped: %d > %d", got, maxProvisionBindAttempts)
}
if len(r.Message) > maxProvisionStringLen {
t.Errorf("message not capped: %d", len(r.Message))
}
for _, a := range r.Detail.Attempts {
if len(a.Addr) > maxProvisionStringLen || len(a.OSError) > maxProvisionStringLen {
t.Error("attempt fields not capped")
}
}
}
func TestProvisionResultFields(t *testing.T) {
r := newProvisionResult(provisionCodeAPIRejected, "the API rejected this configuration", nil)
if r.Version != 1 {
t.Errorf("version = %d, want 1", r.Version)
}
if r.Stage != string(provisionStageBootstrap) {
t.Errorf("stage = %q, want bootstrap", r.Stage)
}
if r.ExitCode != provisionExitCodeForCode[provisionCodeAPIRejected] {
t.Errorf("exit = %d", r.ExitCode)
}
if _, err := time.Parse(time.RFC3339, r.Timestamp); err != nil {
t.Errorf("timestamp %q not RFC3339: %v", r.Timestamp, err)
}
if r.Detail != nil {
t.Error("nil attempts should give nil detail")
}
}
func TestProvisionResultTrusted(t *testing.T) {
good := newProvisionResult(provisionCodeListenerBindFailed, "x", nil)
if !provisionResultTrusted(good) {
t.Error("constructor-built result must be trusted")
}
bogusCode := newProvisionResult(provisionCodeListenerBindFailed, "x", nil)
bogusCode.Code = "TOTALLY_MADE_UP"
if provisionResultTrusted(bogusCode) {
t.Error("unknown code must not be trusted")
}
wrongExit := newProvisionResult(provisionCodeListenerBindFailed, "x", nil)
wrongExit.ExitCode = 126
if provisionResultTrusted(wrongExit) {
t.Error("exit code not matching the contract must not be trusted")
}
wrongStage := newProvisionResult(provisionCodeListenerBindFailed, "x", nil)
wrongStage.Stage = string(provisionStageService)
if provisionResultTrusted(wrongStage) {
t.Error("stage not matching the code must not be trusted")
}
}
func TestNewProvisionResultTruncatesOnRuneBoundary(t *testing.T) {
msg := strings.Repeat("é", maxProvisionStringLen) // 2 bytes per rune
r := newProvisionResult(provisionCodeListenerBindFailed, msg, nil)
if len(r.Message) > maxProvisionStringLen {
t.Errorf("message not capped: %d bytes", len(r.Message))
}
if !utf8.ValidString(r.Message) {
t.Error("truncation split a multi-byte rune")
}
}
func TestFailureCodeDocTableMatchesConstants(t *testing.T) {
buf, err := os.ReadFile(filepath.Join("..", "..", "docs", "provisioning-failure-codes.md"))
if os.IsNotExist(err) {
// The Windows CI runner executes prebuilt test binaries outside the
// repo; the sync guarantee is still enforced on runners with a checkout.
t.Skip("failure-code doc not available in this test environment")
}
if err != nil {
t.Fatalf("could not read the failure-code doc: %v", err)
}
doc := string(buf)
rows := 0
for _, line := range strings.Split(doc, "\n") {
if strings.HasPrefix(line, "| `") {
rows++
}
}
if rows != len(allProvisionFailureCodes) {
t.Errorf("doc table has %d code rows, want %d", rows, len(allProvisionFailureCodes))
}
for _, code := range allProvisionFailureCodes {
row := "| `" + string(code) + "` | " + string(provisionStageForCode[code]) + " | " + strconv.Itoa(provisionExitCodeForCode[code]) + " |"
if !strings.Contains(doc, row) {
t.Errorf("doc table missing row for %s (want prefix %q)", code, row)
}
}
}
func TestProvisionFailureLineFormat(t *testing.T) {
r := newProvisionResult(provisionCodeListenerBindFailed, "could not find available listen ip and port", nil)
want := "provisioning failed: stage=listener code=LISTENER_BIND_FAILED (exit 41)"
if got := r.failureLine(); got != want {
t.Errorf("failureLine() = %q, want %q", got, want)
}
}
func TestProvisionResultRoundTrip(t *testing.T) {
overrideProvisionResultPath(t)
in := newProvisionResult(provisionCodeServiceStartFailed, "service failed to start", nil)
if err := writeProvisionResult(in); err != nil {
t.Fatal(err)
}
out, err := readProvisionResult()
if err != nil {
t.Fatal(err)
}
if out.Code != in.Code || out.Stage != in.Stage || out.ExitCode != in.ExitCode || out.Message != in.Message {
t.Errorf("round trip mismatch: in=%+v out=%+v", in, out)
}
}
func TestWriteProvisionResultOverwritesAtomically(t *testing.T) {
path := overrideProvisionResultPath(t)
first := newProvisionResult(provisionCodeAPIUnreachable, "first", nil)
if err := writeProvisionResult(first); err != nil {
t.Fatal(err)
}
second := newProvisionResult(provisionCodeListenerBindFailed, "second", nil)
if err := writeProvisionResult(second); err != nil {
t.Fatal(err)
}
out, err := readProvisionResult()
if err != nil {
t.Fatal(err)
}
if out.Code != string(provisionCodeListenerBindFailed) || out.Message != "second" {
t.Errorf("overwrite failed: %+v", out)
}
entries, err := os.ReadDir(filepath.Dir(path))
if err != nil {
t.Fatal(err)
}
if len(entries) != 1 {
t.Errorf("temp files left behind: %v", entries)
}
}
func TestClearProvisionResult(t *testing.T) {
path := overrideProvisionResultPath(t)
clearProvisionResult() // missing file must not panic or error loudly
if err := writeProvisionResult(newProvisionResult(provisionCodeAPIUnreachable, "x", nil)); err != nil {
t.Fatal(err)
}
clearProvisionResult()
if _, err := os.Stat(path); !os.IsNotExist(err) {
t.Errorf("result file still present after clear: %v", err)
}
}
func TestReadProvisionResultMissing(t *testing.T) {
overrideProvisionResultPath(t)
if _, err := readProvisionResult(); err == nil {
t.Error("expected error reading missing result file")
}
}
func TestFailProvisionWritesLogsNotifiesAndExits(t *testing.T) {
overrideProvisionResultPath(t)
exitCode := -1
oldExit := provisionExit
provisionExit = func(code int) { exitCode = code }
t.Cleanup(func() { provisionExit = oldExit })
notified := false
r := newProvisionResult(provisionCodeListenerBindFailed, "no listen addr", nil)
failProvision(r, func() { notified = true })
if !notified {
t.Error("notify func not called")
}
if exitCode != provisionExitCodeForCode[provisionCodeListenerBindFailed] {
t.Errorf("exit code = %d", exitCode)
}
out, err := readProvisionResult()
if err != nil {
t.Fatalf("result not persisted: %v", err)
}
if out.Code != string(provisionCodeListenerBindFailed) {
t.Errorf("persisted code = %q", out.Code)
}
}
+22 -10
View File
@@ -162,6 +162,10 @@ func (s *systemd) Start() error {
// This is necessary for running self-upgrade flow.
func ensureSystemdKillMode(r io.Reader) (opts []*unit.UnitOption, change bool) {
opts, err := unit.DeserializeOptions(r)
// staticcheck sees only the explicit non-nil sends on the lexer's error
// channel, so it reports this comparison as always true. On success the
// lexer sends nothing and closes the channel, so the receive yields a nil
// error and this branch is not taken.
if err != nil {
mainLog.Load().Error().Err(err).Msg("failed to deserialize options")
return
@@ -216,22 +220,30 @@ type task struct {
Name string
}
func doTasks(tasks []task) bool {
for _, task := range tasks {
mainLog.Load().Debug().Msgf("Running task %s", task.Name)
if err := task.f(); err != nil {
if task.abortOnError {
mainLog.Load().Error().Msgf("error running task %s: %v", task.Name, err)
return false
// doTasksE runs tasks in order and reports which abortOnError task, if any,
// stopped the run. Use it over doTasks when the failure must be attributed
// to a specific task.
func doTasksE(tasks []task) (failedTaskName string, err error) {
for _, t := range tasks {
mainLog.Load().Debug().Msgf("Running task %s", t.Name)
if taskErr := t.f(); taskErr != nil {
if t.abortOnError {
mainLog.Load().Error().Msgf("error running task %s: %v", t.Name, taskErr)
return t.Name, taskErr
}
// if this is darwin stop command, dont print debug
// since launchctl complains on every start
if runtime.GOOS != "darwin" || task.Name != "Stop" {
mainLog.Load().Debug().Msgf("error running task %s: %v", task.Name, err)
if runtime.GOOS != "darwin" || t.Name != "Stop" {
mainLog.Load().Debug().Msgf("error running task %s: %v", t.Name, taskErr)
}
}
}
return true
return "", nil
}
func doTasks(tasks []task) bool {
_, err := doTasksE(tasks)
return err == nil
}
func checkHasElevatedPrivilege() {
+146
View File
@@ -0,0 +1,146 @@
//go:build darwin
package cli
import (
"fmt"
"os"
"os/exec"
"strings"
)
const launchdPlistPath = "/Library/LaunchDaemons/ctrld.plist"
// serviceConfigFileExists returns true if the launchd plist for ctrld exists on disk.
// This is more reliable than checking launchctl status, which may report "not found"
// if the service was unloaded but the plist file still exists.
func serviceConfigFileExists() bool {
_, err := os.Stat(launchdPlistPath)
return err == nil
}
// appendServiceFlag appends a CLI flag (e.g., "--intercept-mode") to the installed
// service's launch arguments. This is used when upgrading an existing installation
// to intercept mode without losing the existing --cd flag and other arguments.
//
// On macOS, this modifies the launchd plist at /Library/LaunchDaemons/ctrld.plist
// using PlistBuddy for exact array reads and writes.
//
// The function is idempotent: if the flag already exists, it's a no-op.
func appendServiceFlag(flag string) error {
// Read current ProgramArguments from plist.
out, err := exec.Command("/usr/libexec/PlistBuddy", "-c", "Print :ProgramArguments", launchdPlistPath).CombinedOutput()
if err != nil {
return fmt.Errorf("failed to read plist ProgramArguments: %w (output: %s)", err, strings.TrimSpace(string(out)))
}
// Check exact array entries. A substring match can confuse a mode such as "off"
// with an unrelated path or argument and leave the flag without its value.
if serviceArgumentPresent(out, flag) {
mainLog.Load().Debug().Msgf("Service flag %q already present in plist, skipping", flag)
return nil
}
// Use PlistBuddy to append the flag to ProgramArguments array.
// PlistBuddy is more reliable than "defaults" for array manipulation.
addCmd := exec.Command(
"/usr/libexec/PlistBuddy",
"-c", fmt.Sprintf("Add :ProgramArguments: string %s", flag),
launchdPlistPath,
)
if out, err := addCmd.CombinedOutput(); err != nil {
return fmt.Errorf("failed to append %q to plist ProgramArguments: %w (output: %s)", flag, err, strings.TrimSpace(string(out)))
}
mainLog.Load().Info().Msgf("Appended %q to service launch arguments", flag)
return nil
}
// verifyServiceRegistration is a no-op on macOS (launchd plist verification not needed).
func verifyServiceRegistration() error {
return nil
}
// removeServiceFlag removes both "--flag value" and "--flag=value" forms from the
// installed service's launch arguments.
//
// The function is idempotent: if the flag doesn't exist, it's a no-op.
func removeServiceFlag(flag string) error {
// Read current ProgramArguments to find the index.
out, err := exec.Command("/usr/libexec/PlistBuddy", "-c", "Print :ProgramArguments", launchdPlistPath).CombinedOutput()
if err != nil {
return fmt.Errorf("failed to read plist ProgramArguments: %w (output: %s)", err, strings.TrimSpace(string(out)))
}
// Parse the PlistBuddy output to find the flag's index.
// PlistBuddy prints arrays as:
// Array {
// /path/to/ctrld
// run
// --cd=xxx
// --intercept-mode
// dns
// }
lines := strings.Split(string(out), "\n")
var entries []string
for _, line := range lines {
trimmed := strings.TrimSpace(line)
if trimmed == "Array {" || trimmed == "}" || trimmed == "" {
continue
}
entries = append(entries, trimmed)
}
index, hasValue := serviceFlagPosition(entries, flag)
if index < 0 {
mainLog.Load().Debug().Msgf("Service flag %q not present in plist, skipping removal", flag)
return nil
}
// Delete a separate value first. An inline --flag=value entry is one array item.
if hasValue {
delVal := exec.Command(
"/usr/libexec/PlistBuddy",
"-c", fmt.Sprintf("Delete :ProgramArguments:%d", index+1),
launchdPlistPath,
)
if out, err := delVal.CombinedOutput(); err != nil {
return fmt.Errorf("failed to remove value for %q from plist: %w (output: %s)", flag, err, strings.TrimSpace(string(out)))
}
}
// Delete the flag itself.
delCmd := exec.Command(
"/usr/libexec/PlistBuddy",
"-c", fmt.Sprintf("Delete :ProgramArguments:%d", index),
launchdPlistPath,
)
if out, err := delCmd.CombinedOutput(); err != nil {
return fmt.Errorf("failed to remove %q from plist ProgramArguments: %w (output: %s)", flag, err, strings.TrimSpace(string(out)))
}
mainLog.Load().Info().Msgf("Removed %q from service launch arguments", flag)
return nil
}
func serviceArgumentPresent(out []byte, argument string) bool {
for _, line := range strings.Split(string(out), "\n") {
if strings.TrimSpace(line) == argument {
return true
}
}
return false
}
func serviceFlagPosition(entries []string, flag string) (index int, hasValue bool) {
for i, entry := range entries {
switch {
case entry == flag:
return i, i+1 < len(entries) && !strings.HasPrefix(entries[i+1], "-")
case strings.HasPrefix(entry, flag+"="):
return i, false
}
}
return -1, false
}
+58
View File
@@ -0,0 +1,58 @@
//go:build darwin
package cli
import "testing"
func TestServiceArgumentPresent(t *testing.T) {
out := []byte("Array {\n /usr/local/bin/ctrld\n run\n --config=/Users/officer/ctrld.toml\n --intercept-mode=dns\n}\n")
if !serviceArgumentPresent(out, "--intercept-mode=dns") {
t.Fatal("exact inline argument was not found")
}
if serviceArgumentPresent(out, "--intercept-mode") {
t.Fatal("inline flag was mistaken for a separate flag argument")
}
if serviceArgumentPresent(out, "off") {
t.Fatal("substring in an unrelated path was mistaken for the off argument")
}
}
func TestServiceFlagPosition(t *testing.T) {
tests := []struct {
name string
entries []string
wantIndex int
wantHasValue bool
}{
{
name: "split form",
entries: []string{"run", "--cd=uid", "--intercept-mode", "dns"},
wantIndex: 2,
wantHasValue: true,
},
{
name: "inline form",
entries: []string{"run", "--cd=uid", "--intercept-mode=dns"},
wantIndex: 2,
},
{
name: "flag followed by another flag",
entries: []string{"run", "--intercept-mode", "--config=/etc/ctrld.toml"},
wantIndex: 1,
},
{
name: "absent",
entries: []string{"run", "--cd=uid"},
wantIndex: -1,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
index, hasValue := serviceFlagPosition(tc.entries, "--intercept-mode")
if index != tc.wantIndex || hasValue != tc.wantHasValue {
t.Fatalf("serviceFlagPosition() = (%d, %v), want (%d, %v)", index, hasValue, tc.wantIndex, tc.wantHasValue)
}
})
}
}
+42
View File
@@ -0,0 +1,42 @@
//go:build !darwin && !windows
package cli
import (
"errors"
"os"
)
// errServiceFlagsUnsupported is returned by the service-argument helpers on
// platforms that do not store service arguments in a file ctrld can rewrite.
var errServiceFlagsUnsupported = errors.New("modifying service flags is not supported on this platform; use intercept_mode in config instead")
// serviceConfigFileExists checks common service config file locations on Linux.
func serviceConfigFileExists() bool {
// systemd unit file
if _, err := os.Stat("/etc/systemd/system/ctrld.service"); err == nil {
return true
}
// SysV init script
if _, err := os.Stat("/etc/init.d/ctrld"); err == nil {
return true
}
return false
}
// appendServiceFlag is not yet implemented on this platform.
// Linux services (systemd) store args in unit files; intercept mode
// should be set via the config file (intercept_mode) on these platforms.
func appendServiceFlag(flag string) error {
return errServiceFlagsUnsupported
}
// verifyServiceRegistration is a no-op on this platform.
func verifyServiceRegistration() error {
return nil
}
// removeServiceFlag is not yet implemented on this platform.
func removeServiceFlag(flag string) error {
return errServiceFlagsUnsupported
}
+169
View File
@@ -0,0 +1,169 @@
//go:build windows
package cli
import (
"fmt"
"strings"
"golang.org/x/sys/windows/svc/mgr"
)
// serviceConfigFileExists returns true if the ctrld Windows service is registered.
func serviceConfigFileExists() bool {
m, err := mgr.Connect()
if err != nil {
return false
}
defer m.Disconnect()
s, err := m.OpenService(ctrldServiceName)
if err != nil {
return false
}
s.Close()
return true
}
// appendServiceFlag appends a CLI flag (e.g., "--intercept-mode") to the installed
// Windows service's BinPath arguments. This is used when upgrading an existing
// installation to intercept mode without losing the existing --cd flag.
//
// The function is idempotent: if the flag already exists, it's a no-op.
func appendServiceFlag(flag string) error {
m, err := mgr.Connect()
if err != nil {
return fmt.Errorf("failed to connect to Windows SCM: %w", err)
}
defer m.Disconnect()
s, err := m.OpenService(ctrldServiceName)
if err != nil {
return fmt.Errorf("failed to open service %q: %w", ctrldServiceName, err)
}
defer s.Close()
config, err := s.Config()
if err != nil {
return fmt.Errorf("failed to read service config: %w", err)
}
// Check exact arguments so a short mode such as "off" is not confused with
// an unrelated path or value.
if binaryPathArgumentPresent(config.BinaryPathName, flag) {
mainLog.Load().Debug().Msgf("Service flag %q already present in BinPath, skipping", flag)
return nil
}
// Append the flag to BinPath.
config.BinaryPathName = strings.TrimSpace(config.BinaryPathName) + " " + flag
if err := s.UpdateConfig(config); err != nil {
return fmt.Errorf("failed to update service config with %q: %w", flag, err)
}
mainLog.Load().Info().Msgf("Appended %q to service BinPath", flag)
return nil
}
// verifyServiceRegistration opens the Windows Service Control Manager and verifies
// that the ctrld service is correctly registered: logs the BinaryPathName, checks
// that --intercept-mode is present if expected, and verifies SERVICE_AUTO_START.
func verifyServiceRegistration() error {
m, err := mgr.Connect()
if err != nil {
return fmt.Errorf("failed to connect to Windows SCM: %w", err)
}
defer m.Disconnect()
s, err := m.OpenService(ctrldServiceName)
if err != nil {
return fmt.Errorf("failed to open service %q: %w", ctrldServiceName, err)
}
defer s.Close()
config, err := s.Config()
if err != nil {
return fmt.Errorf("failed to read service config: %w", err)
}
mainLog.Load().Debug().Msgf("Service registry: BinaryPathName = %q", config.BinaryPathName)
// If intercept mode is set, verify the flag is present in BinPath.
if interceptMode == "dns" || interceptMode == "hard" {
if !strings.Contains(config.BinaryPathName, "--intercept-mode") {
return fmt.Errorf("service registry: --intercept-mode flag missing from BinaryPathName (expected mode %q)", interceptMode)
}
mainLog.Load().Debug().Msgf("Service registry: --intercept-mode flag present in BinaryPathName")
}
// Verify auto-start. mgr.StartAutomatic == 2 == SERVICE_AUTO_START.
if config.StartType != mgr.StartAutomatic {
return fmt.Errorf("service registry: StartType is %d, expected SERVICE_AUTO_START (%d)", config.StartType, mgr.StartAutomatic)
}
return nil
}
// removeServiceFlag removes both "--flag value" and "--flag=value" forms from the
// installed Windows service's BinPath. The function is idempotent.
func removeServiceFlag(flag string) error {
m, err := mgr.Connect()
if err != nil {
return fmt.Errorf("failed to connect to Windows SCM: %w", err)
}
defer m.Disconnect()
s, err := m.OpenService(ctrldServiceName)
if err != nil {
return fmt.Errorf("failed to open service %q: %w", ctrldServiceName, err)
}
defer s.Close()
config, err := s.Config()
if err != nil {
return fmt.Errorf("failed to read service config: %w", err)
}
updatedPath, removed := removeBinaryPathFlag(config.BinaryPathName, flag)
if !removed {
mainLog.Load().Debug().Msgf("Service flag %q not present in BinPath, skipping removal", flag)
return nil
}
config.BinaryPathName = updatedPath
if err := s.UpdateConfig(config); err != nil {
return fmt.Errorf("failed to update service config: %w", err)
}
mainLog.Load().Info().Msgf("Removed %q from service BinPath", flag)
return nil
}
func binaryPathArgumentPresent(binaryPath, argument string) bool {
for _, part := range strings.Fields(binaryPath) {
if part == argument {
return true
}
}
return false
}
func removeBinaryPathFlag(binaryPath, flag string) (string, bool) {
parts := strings.Fields(binaryPath)
newParts := make([]string, 0, len(parts))
removed := false
for i := 0; i < len(parts); i++ {
switch {
case parts[i] == flag:
removed = true
if i+1 < len(parts) && !strings.HasPrefix(parts[i+1], "-") {
i++
}
case strings.HasPrefix(parts[i], flag+"="):
removed = true
default:
newParts = append(newParts, parts[i])
}
}
return strings.Join(newParts, " "), removed
}
+54
View File
@@ -0,0 +1,54 @@
//go:build windows
package cli
import "testing"
func TestBinaryPathArgumentPresent(t *testing.T) {
path := `C:\ControlD\ctrld.exe run --config=C:\Users\officer\ctrld.toml --intercept-mode=dns`
if !binaryPathArgumentPresent(path, "--intercept-mode=dns") {
t.Fatal("exact inline argument was not found")
}
if binaryPathArgumentPresent(path, "--intercept-mode") {
t.Fatal("inline flag was mistaken for a separate flag argument")
}
if binaryPathArgumentPresent(path, "off") {
t.Fatal("substring in an unrelated path was mistaken for the off argument")
}
}
func TestRemoveBinaryPathFlag(t *testing.T) {
tests := []struct {
name string
binaryPath string
wantPath string
wantRemoved bool
}{
{
name: "split form",
binaryPath: `ctrld.exe run --cd=uid --intercept-mode dns --config=ctrld.toml`,
wantPath: `ctrld.exe run --cd=uid --config=ctrld.toml`,
wantRemoved: true,
},
{
name: "inline form",
binaryPath: `ctrld.exe run --cd=uid --intercept-mode=dns --config=ctrld.toml`,
wantPath: `ctrld.exe run --cd=uid --config=ctrld.toml`,
wantRemoved: true,
},
{
name: "absent",
binaryPath: `ctrld.exe run --cd=uid`,
wantPath: `ctrld.exe run --cd=uid`,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
path, removed := removeBinaryPathFlag(tc.binaryPath, "--intercept-mode")
if path != tc.wantPath || removed != tc.wantRemoved {
t.Fatalf("removeBinaryPathFlag() = (%q, %v), want (%q, %v)", path, removed, tc.wantPath, tc.wantRemoved)
}
})
}
}
+59
View File
@@ -0,0 +1,59 @@
package cli
import "strings"
// serviceBinaryFromImagePath extracts the executable path from a Windows service
// ImagePath value, which carries the command line rather than a bare path: it may be
// quoted and is usually followed by arguments, e.g.
//
// "C:\Program Files\Control D\ctrld.exe" run --config C:\...\ctrld.toml
//
// It returns "" when no path can be read, which callers must treat as "cannot tell"
// rather than "does not match".
func serviceBinaryFromImagePath(imagePath string) string {
imagePath = strings.TrimSpace(imagePath)
if imagePath == "" {
return ""
}
if imagePath[0] == '"' {
// Quoted form: everything up to the closing quote is the path, so a directory
// containing spaces stays intact.
if end := strings.IndexByte(imagePath[1:], '"'); end >= 0 {
return strings.TrimSpace(imagePath[1 : 1+end])
}
return strings.TrimSpace(imagePath[1:])
}
// Unquoted form: the path cannot contain spaces, so the first field is it.
if idx := strings.IndexByte(imagePath, ' '); idx >= 0 {
return strings.TrimSpace(imagePath[:idx])
}
return imagePath
}
// sameExecutableDir reports whether two Windows executable paths live in the same
// directory, compared case-insensitively because Windows paths are.
//
// The separator handling is explicit rather than filepath's, because filepath follows the
// *host* rules: off Windows it does not treat "\\" as a separator, so every backslash path
// would reduce to the same directory and any two paths would compare equal. Doing it here
// keeps the comparison correct and testable on any host.
//
// A path with no directory part answers false, which callers read as "cannot tell".
func sameExecutableDir(a, b string) bool {
dirA, dirB := windowsExecutableDir(a), windowsExecutableDir(b)
if dirA == "" || dirB == "" {
return false
}
return strings.EqualFold(dirA, dirB)
}
// windowsExecutableDir returns the directory part of a Windows path, accepting either
// separator and normalising to a backslash. It returns "" when there is no directory part.
func windowsExecutableDir(path string) string {
path = strings.TrimSpace(path)
idx := strings.LastIndexAny(path, `\/`)
if idx <= 0 {
return ""
}
return strings.ReplaceAll(path[:idx], "/", `\`)
}
+8
View File
@@ -0,0 +1,8 @@
//go:build !windows
package cli
// installedServiceDirMatches is Windows-only: it exists because socketDir() there is
// relative to the running executable. Other platforms answer this question through
// hasElevatedPrivilege in readinessVerifiable.
func installedServiceDirMatches() bool { return true }
+103
View File
@@ -0,0 +1,103 @@
package cli
import "testing"
// TestServiceBinaryFromImagePath covers the ImagePath shapes Windows stores. Getting this
// wrong makes readinessVerifiable compare the wrong directories, and "ctrld status" would
// then report a healthy service as not-ready - the false positive the readiness exit code
// exists to avoid.
func TestServiceBinaryFromImagePath(t *testing.T) {
tests := []struct {
name string
imagePath string
want string
}{
{
// The installed form: quoted because the directory contains a space, with the
// service arguments following it.
name: "quoted path with arguments",
imagePath: `"C:\Program Files\Control D\ctrld.exe" run --config "C:\ProgramData\Control D\ctrld.toml"`,
want: `C:\Program Files\Control D\ctrld.exe`,
},
{
name: "quoted path without arguments",
imagePath: `"C:\Program Files\Control D\ctrld.exe"`,
want: `C:\Program Files\Control D\ctrld.exe`,
},
{
name: "unquoted path with arguments",
imagePath: `C:\ctrld\ctrld.exe run --cd abc123`,
want: `C:\ctrld\ctrld.exe`,
},
{
name: "unquoted path alone",
imagePath: `C:\ctrld\ctrld.exe`,
want: `C:\ctrld\ctrld.exe`,
},
{
name: "surrounding whitespace",
imagePath: ` "C:\ctrld\ctrld.exe" run `,
want: `C:\ctrld\ctrld.exe`,
},
{
// Unterminated quote: take what is there rather than returning nothing, since
// "" means "cannot tell" and would silently disable the check.
name: "unterminated quote",
imagePath: `"C:\ctrld\ctrld.exe run`,
want: `C:\ctrld\ctrld.exe run`,
},
{
name: "empty",
imagePath: "",
want: "",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := serviceBinaryFromImagePath(tc.imagePath); got != tc.want {
t.Errorf("serviceBinaryFromImagePath(%q) = %q, want %q", tc.imagePath, got, tc.want)
}
})
}
}
// TestSameExecutableDir pins the comparison itself: Windows paths are case-insensitive, and
// an empty side means "cannot tell", which must never read as a match.
func TestSameExecutableDir(t *testing.T) {
tests := []struct {
name string
a string
b string
want bool
}{
{
name: "same directory",
a: `C:\Program Files\Control D\ctrld.exe`,
b: `C:\Program Files\Control D\ctrld.exe`,
want: true,
},
{
name: "same directory different case",
a: `C:\Program Files\Control D\ctrld.exe`,
b: `c:\program files\control d\ctrld.exe`,
want: true,
},
{
// The case the check exists for: a copy run from a download directory
// resolves a different control socket than the installed service.
name: "different directory",
a: `C:\Program Files\Control D\ctrld.exe`,
b: `C:\Users\admin\Downloads\ctrld.exe`,
want: false,
},
{name: "unknown installed path", a: "", b: `C:\ctrld\ctrld.exe`, want: false},
{name: "unknown self path", a: `C:\ctrld\ctrld.exe`, b: "", want: false},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := sameExecutableDir(tc.a, tc.b); got != tc.want {
t.Errorf("sameExecutableDir(%q, %q) = %v, want %v", tc.a, tc.b, got, tc.want)
}
})
}
}
+41
View File
@@ -0,0 +1,41 @@
//go:build windows
package cli
import (
"os"
"golang.org/x/sys/windows/registry"
)
// installedServiceDirMatches reports whether this executable is the installed service
// binary, by comparing its directory with the one in the service's registered ImagePath.
//
// socketDir() on Windows is relative to the running executable, so a ctrld.exe run from
// somewhere else - a download directory, a build tree - looks for the control socket in
// its own directory and never finds the installed daemon's. A failed probe from there
// says nothing about the service's health, and reporting "not ready" for it would tell
// monitoring to restart a healthy service.
//
// Anything unreadable answers true, keeping the previous behaviour: readiness stays
// verifiable unless there is positive evidence of a different install.
func installedServiceDirMatches() bool {
self, err := os.Executable()
if err != nil {
return true
}
key, err := registry.OpenKey(registry.LOCAL_MACHINE, `SYSTEM\CurrentControlSet\Services\`+ctrldServiceName, registry.QUERY_VALUE)
if err != nil {
return true
}
defer key.Close()
imagePath, _, err := key.GetStringValue("ImagePath")
if err != nil {
return true
}
installed := serviceBinaryFromImagePath(imagePath)
if installed == "" {
return true
}
return sameExecutableDir(installed, self)
}
+177
View File
@@ -0,0 +1,177 @@
package cli
import (
"errors"
"fmt"
"io/fs"
"net/http"
"path/filepath"
"runtime"
"time"
)
// Exit codes reported by "ctrld status".
const (
statusExitRunning = 0
statusExitStopped = 1
statusExitUnknown = 2
// statusExitNotReady means the service manager considers the service running,
// but the process has not finished starting up, so it is not serving DNS or
// applying policy. This is a distinct code because it needs a distinct response:
// the process exists, so restarting the service is what recovers it, while a
// stopped service needs starting and an unknown state needs investigation.
statusExitNotReady = 3
)
// serviceReadinessTimeout bounds the control-socket probe. Status must answer
// quickly, and a service that cannot respond within this window is not usefully
// "running" from a caller's point of view either way.
const serviceReadinessTimeout = 3 * time.Second
// statusCmdLong documents what the reported states mean, including that a service the
// OS calls running is not necessarily serving.
const statusCmdLong = `Show status of the ctrld service.
Reports both what the OS service manager thinks and whether ctrld has finished
starting up, since a service can be registered as running while its process is
still in startup and serving nothing.
Exit codes:
0 running and serving, or running with startup not verified
1 stopped
2 status unknown
3 registered as running, but startup has not completed
Verifying startup requires reaching ctrld's control socket. On Linux, BSD and macOS
that socket lives in a directory only the privileged user resolves, so an
unprivileged "ctrld status" reports the service manager's view and says startup was
not verified rather than claiming the service is unhealthy. Exit 3 is only reported
when the check could actually be made.`
// readiness is what "ctrld status" reports for a service the service manager
// considers running.
type readiness struct {
messages []string
exitCode int
}
// readinessVerifiable reports whether a failed control-socket probe can be trusted to
// mean "the service has not finished starting up".
//
// It can only mean that if this process resolves the same socket path the daemon
// created, and socketDir() is caller-relative on unix: it returns the system directory
// only when that is writable, and the caller's home directory otherwise. So a
// root-owned daemon listens on /var/run/ctrld_control.sock while an unprivileged
// "ctrld status" looks under $HOME, finds nothing, and gets ENOENT - which means "wrong
// path", not "not ready". Reporting exit 3 there would tell a monitoring check to
// restart a perfectly healthy daemon.
//
// On Windows and mobile socketDir() is the install/home directory for every caller, so
// the probe is comparable - which matters because Windows is where the hung-start this
// exit code exists for was seen. On Windows that only holds while this binary is the
// installed one: a copy run from elsewhere resolves a different socket directory, so its
// failed probe would say nothing about the service. installedServiceDirMatches() checks
// that, and answers true when it cannot tell, preserving the previous behaviour.
func readinessVerifiable() bool {
if isMobile() {
return true
}
if runtime.GOOS == "windows" {
return installedServiceDirMatches()
}
elevated, err := hasElevatedPrivilege()
return err == nil && elevated
}
// classifyReadiness turns a control-socket probe result into the report for a service
// the service manager calls running.
//
// verifiable comes from readinessVerifiable: when it is false a failed probe says
// nothing about the service, so the report falls back to the service manager's view.
// A *successful* probe is still conclusive either way - reaching the socket at all is
// positive evidence, whoever the caller is.
func classifyReadiness(ready bool, err error, verifiable bool) readiness {
switch {
case ready:
return readiness{
messages: []string{"Service is running"},
exitCode: statusExitRunning,
}
case !verifiable:
return readiness{
messages: []string{"Service is running (startup not verified: re-run with elevated privileges to check readiness)"},
exitCode: statusExitRunning,
}
case errors.Is(err, errReadinessNotReported):
// The service answered, just not with a verdict - an older daemon without the
// /started route. It is alive and reachable, so the service manager's view is
// the best available answer.
return readiness{
messages: []string{"Service is running (startup not verified: this ctrld build does not report readiness)"},
exitCode: statusExitRunning,
}
case errors.Is(err, fs.ErrPermission):
// Without access to the control socket there is nothing to report beyond the
// service manager's view. Do not call a service unhealthy because the caller
// lacks privilege.
return readiness{
messages: []string{"Service is running (startup not verified: control socket requires elevated privileges)"},
exitCode: statusExitRunning,
}
default:
return readiness{
messages: []string{
"Service is registered as running, but has not completed startup: it is not serving DNS",
"Check the ctrld log for why startup did not finish, then restart the service",
},
exitCode: statusExitNotReady,
}
}
}
// serviceReady reports whether a running ctrld has finished starting up, by asking
// its control server. The control server answers /started only once the onStarted
// hooks have completed, which is after the DNS listeners are up, so a successful
// probe means the process is actually serving rather than merely alive.
//
// An error means "could not confirm readiness" and is returned for the caller to
// classify: a refused connection or missing socket is a process that never got that
// far, while a permission error says nothing about the service's health.
func serviceReady() (bool, error) {
dir, err := socketDir()
if err != nil {
return false, err
}
return serviceReadyAt(filepath.Join(dir, ControlSocketName()), serviceReadinessTimeout)
}
// errReadinessNotReported marks a control server that answered without a readiness
// verdict.
//
// http.Client.Post returns (resp, nil) for any status, so a daemon with no /started
// route answers 404 and an internal failure answers 5xx - neither says the service has
// not started. Reporting "not ready" there tells a monitoring check to restart a healthy
// service, and it happens in normal operation: after an upgrade replaces the binary on
// disk but before the service restarts, and throughout a mixed-version rollout.
var errReadinessNotReported = errors.New("control server did not report readiness")
// serviceReadyAt is serviceReady against an explicit socket path and timeout.
func serviceReadyAt(sockPath string, timeout time.Duration) (bool, error) {
cc := newControlClient(sockPath)
cc.c.Timeout = timeout
resp, err := cc.post(startedPath, nil)
if err != nil {
return false, err
}
defer resp.Body.Close()
switch resp.StatusCode {
case http.StatusOK:
return true, nil
case http.StatusRequestTimeout:
// The daemon's own verdict: its onStarted hooks have not completed. This is the
// hung start statusExitNotReady exists for.
return false, nil
default:
return false, fmt.Errorf("%w: HTTP %d", errReadinessNotReported, resp.StatusCode)
}
}
+281
View File
@@ -0,0 +1,281 @@
package cli
import (
"errors"
"io/fs"
"net"
"net/http"
"os"
"path/filepath"
"runtime"
"testing"
"time"
)
// startControlSocket serves handler on a unix socket and returns its path.
func startControlSocket(t *testing.T, handler http.HandlerFunc) string {
t.Helper()
// Keep the path short: unix socket paths have a low length limit.
dir, err := os.MkdirTemp("", "ctrldsock")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = os.RemoveAll(dir) })
sockPath := filepath.Join(dir, "s.sock")
ln, err := net.Listen("unix", sockPath)
if err != nil {
t.Skipf("cannot listen on a unix socket: %v", err)
}
mux := http.NewServeMux()
mux.Handle(startedPath, handler)
srv := &http.Server{Handler: mux}
go func() { _ = srv.Serve(ln) }()
t.Cleanup(func() { _ = srv.Close() })
return sockPath
}
func TestServiceReadyAt(t *testing.T) {
t.Run("ready when the control server reports started", func(t *testing.T) {
sock := startControlSocket(t, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
})
ready, err := serviceReadyAt(sock, time.Second)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !ready {
t.Error("ready = false, want true")
}
})
t.Run("not ready when startup has not finished", func(t *testing.T) {
// What /started returns when the onStarted hooks have not completed.
sock := startControlSocket(t, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusRequestTimeout)
})
ready, err := serviceReadyAt(sock, time.Second)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if ready {
t.Error("ready = true for a control server that has not finished startup")
}
})
t.Run("not ready when there is no control socket", func(t *testing.T) {
// The incident: the process was alive but had never created the socket, so
// every control request was refused.
ready, err := serviceReadyAt(filepath.Join(t.TempDir(), "absent.sock"), time.Second)
if ready {
t.Error("ready = true with no control socket")
}
if err == nil {
t.Error("expected an error when the control socket does not exist")
}
})
t.Run("not ready when the probe times out", func(t *testing.T) {
sock := startControlSocket(t, func(w http.ResponseWriter, r *http.Request) {
time.Sleep(2 * time.Second)
w.WriteHeader(http.StatusOK)
})
ready, err := serviceReadyAt(sock, 50*time.Millisecond)
if ready {
t.Error("ready = true for a probe that timed out")
}
if err == nil {
t.Error("expected an error when the probe times out")
}
})
}
func TestClassifyReadiness(t *testing.T) {
tests := []struct {
name string
ready bool
err error
verifiable bool
wantCode int
}{
{
name: "ready",
ready: true,
verifiable: true,
wantCode: statusExitRunning,
},
{
// The service manager says running, the process is not serving. This
// must not report success.
name: "running but never finished startup",
err: errors.New("connect: connection refused"),
verifiable: true,
wantCode: statusExitNotReady,
},
{
// A caller without privilege cannot probe; that is not evidence of a
// broken service, so it must not be reported as one.
name: "probe not permitted",
err: fs.ErrPermission,
verifiable: true,
wantCode: statusExitRunning,
},
{
name: "wrapped permission error",
err: &net.OpError{Op: "dial", Err: fs.ErrPermission},
verifiable: true,
wantCode: statusExitRunning,
},
{
// The P2: an unprivileged caller on unix resolves a socket path the
// daemon never used, so the probe fails with ENOENT rather than a
// permission error. That says nothing about the service and must not be
// reported as unhealthy - a monitoring check acting on exit 3 would
// restart a healthy daemon.
name: "missing socket at an unverifiable path",
err: &net.OpError{Op: "dial", Err: os.ErrNotExist},
verifiable: false,
wantCode: statusExitRunning,
},
{
name: "connection refused at an unverifiable path",
err: errors.New("connect: connection refused"),
verifiable: false,
wantCode: statusExitRunning,
},
{
// A probe that actually reached the socket is conclusive whoever ran it.
name: "successful probe is trusted even when unverifiable",
ready: true,
verifiable: false,
wantCode: statusExitRunning,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got := classifyReadiness(tc.ready, tc.err, tc.verifiable)
if got.exitCode != tc.wantCode {
t.Errorf("exitCode = %d, want %d", got.exitCode, tc.wantCode)
}
if len(got.messages) == 0 {
t.Error("no message to report")
}
})
}
}
// TestReadinessVerifiableMatchesSocketVisibility is the closure test for the P2: the
// not-ready verdict must only be reachable when this process resolves the same socket
// directory the daemon uses.
//
// On unix that is the privileged user's path, so an unprivileged run - which is how
// "ctrld status" is normally invoked, since only darwin has an elevation PreRun and the
// root-level alias has none - must not be able to reach exit 3.
func TestReadinessVerifiableMatchesSocketVisibility(t *testing.T) {
verifiable := readinessVerifiable()
if runtime.GOOS == "windows" {
if !verifiable {
t.Error("on Windows every caller resolves the install directory, so the probe is always verifiable")
}
return
}
elevated, err := hasElevatedPrivilege()
if err != nil {
t.Skipf("cannot determine privilege: %v", err)
}
if verifiable != elevated {
t.Errorf("readinessVerifiable() = %v, want %v (elevated)", verifiable, elevated)
}
if !elevated {
// The shape the review asked to assert: unprivileged, healthy daemon, and a
// probe that cannot see its socket must still report running.
dir, err := socketDir()
if err != nil {
t.Fatalf("socketDir(): %v", err)
}
if dir == "/var/run" {
t.Skip("unprivileged but /var/run is writable, so the probe path does match")
}
r := classifyReadiness(false, &net.OpError{Op: "dial", Err: os.ErrNotExist}, verifiable)
if r.exitCode == statusExitNotReady {
t.Errorf("unprivileged status probing %q reported not-ready (exit %d) for a healthy service", dir, r.exitCode)
}
}
}
// Every status must map to its own exit code: a caller that cannot tell a hung
// service from a healthy or a stopped one is back to the incident's diagnostics.
//
// The literal values are the contract. statusCmdLong documents them and monitoring
// scripts key off them, so asserting the constants against each other would let a
// renumbering keep the suite green while silently breaking every caller.
func TestStatusExitCodesAreDistinct(t *testing.T) {
for _, tc := range []struct {
name string
got int
want int
}{
{"running", statusExitRunning, 0},
{"stopped", statusExitStopped, 1},
{"unknown", statusExitUnknown, 2},
{"not ready", statusExitNotReady, 3},
} {
if tc.got != tc.want {
t.Errorf("%s exit code = %d, want %d: statusCmdLong and monitoring scripts document this value", tc.name, tc.got, tc.want)
}
}
codes := map[int]string{
statusExitRunning: "running",
statusExitStopped: "stopped",
statusExitUnknown: "unknown",
statusExitNotReady: "not ready",
}
if len(codes) != 4 {
t.Errorf("status exit codes collide, only %d distinct: %v", len(codes), codes)
}
}
// TestReadinessProbeStatusHandling covers what each control-server answer means.
//
// http.Client.Post returns (resp, nil) for any status code, so a daemon without the
// /started route answers 404 and the probe must report "cannot confirm" rather than "not
// started". That state is reached in normal operation - after an upgrade replaces the
// binary but before the service restarts, and throughout a mixed-version rollout - and
// reporting exit 3 there tells monitoring to restart a healthy service.
func TestReadinessProbeStatusHandling(t *testing.T) {
tests := []struct {
name string
status int
wantReady bool
wantReported bool // whether the answer carries a readiness verdict
wantExitCode int
}{
{"started", http.StatusOK, true, true, statusExitRunning},
{"still starting", http.StatusRequestTimeout, false, true, statusExitNotReady},
{"no readiness route", http.StatusNotFound, false, false, statusExitRunning},
{"control server error", http.StatusInternalServerError, false, false, statusExitRunning},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
status := tc.status
sock := startControlSocket(t, func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(status)
})
ready, err := serviceReadyAt(sock, time.Second)
if ready != tc.wantReady {
t.Errorf("ready = %v, want %v", ready, tc.wantReady)
}
if reported := !errors.Is(err, errReadinessNotReported); reported != tc.wantReported {
t.Errorf("readiness reported = %v, want %v (err: %v)", reported, tc.wantReported, err)
}
if got := classifyReadiness(ready, err, true).exitCode; got != tc.wantExitCode {
t.Errorf("exit code = %d, want %d", got, tc.wantExitCode)
}
})
}
}
+57
View File
@@ -1,6 +1,7 @@
package cli
import (
"errors"
"strings"
"testing"
)
@@ -26,3 +27,59 @@ func Test_ensureSystemdKillMode(t *testing.T) {
})
}
}
func TestDoTasksESuccess(t *testing.T) {
var ran []string
tasks := []task{
{func() error { ran = append(ran, "a"); return nil }, false, "a"},
{func() error { ran = append(ran, "b"); return nil }, true, "b"},
}
failedTask, err := doTasksE(tasks)
if failedTask != "" || err != nil {
t.Errorf("doTasksE() = (%q, %v), want (\"\", nil)", failedTask, err)
}
if got := strings.Join(ran, ","); got != "a,b" {
t.Errorf("ran tasks %q, want all tasks run in order", got)
}
}
func TestDoTasksEAbortsOnAbortOnErrorTask(t *testing.T) {
wantErr := errors.New("install failed")
var ran []string
tasks := []task{
{func() error { ran = append(ran, "Stop"); return nil }, false, "Stop"},
{func() error { ran = append(ran, "Install"); return wantErr }, true, "Install"},
{func() error { ran = append(ran, "Start"); return nil }, true, "Start"},
}
failedTask, err := doTasksE(tasks)
if failedTask != "Install" || !errors.Is(err, wantErr) {
t.Errorf("doTasksE() = (%q, %v), want (\"Install\", %v)", failedTask, err, wantErr)
}
if got := strings.Join(ran, ","); got != "Stop,Install" {
t.Errorf("ran tasks %q, want the run to stop right after the abort", got)
}
}
func TestDoTasksENonAbortFailureContinues(t *testing.T) {
var ran []string
tasks := []task{
{func() error { ran = append(ran, "a"); return errors.New("a failed") }, false, "a"},
{func() error { ran = append(ran, "b"); return nil }, true, "b"},
}
failedTask, err := doTasksE(tasks)
if failedTask != "" || err != nil {
t.Errorf("doTasksE() = (%q, %v), want (\"\", nil) since the failing task did not abort", failedTask, err)
}
if got := strings.Join(ran, ","); got != "a,b" {
t.Errorf("ran tasks %q, want the run to continue past the non-abort failure", got)
}
}
func TestDoTasksDelegatesToDoTasksE(t *testing.T) {
if !doTasks([]task{{func() error { return nil }, true, "ok"}}) {
t.Error("doTasks() = false, want true on success")
}
if doTasks([]task{{func() error { return errors.New("boom") }, true, "boom"}}) {
t.Error("doTasks() = true, want false when an abortOnError task fails")
}
}
+1
View File
@@ -160,6 +160,7 @@ func hasLocalDnsServerRunning() bool {
if e != nil {
return false
}
defer windows.CloseHandle(h)
p := windows.ProcessEntry32{Size: processEntrySize}
for {
e := windows.Process32Next(h, &p)
+61
View File
@@ -0,0 +1,61 @@
package cli
import "testing"
// Test_ensureRunningIfaceForInvalidUninstall is a regression test for issue-556:
// after a reboot, the invalid-device self-uninstall path could run before the
// running interface was known. Because resetDNS (via resetDNSForRunningIface)
// silently skips DNS restoration when p.runningIface is empty, the OS was left
// pointed at ctrld's local listener with no internet after the service was
// removed. ensureRunningIfaceForInvalidUninstall must populate p.runningIface
// before resetDNS runs.
func Test_ensureRunningIfaceForInvalidUninstall(t *testing.T) {
// preRun mutates the package-level iface global; restore it after the test.
origIface := iface
t.Cleanup(func() { iface = origIface })
// newService needs the package service config; it is safe to build here
// because ensureRunningIfaceForInvalidUninstall only queries the (absent)
// control socket via runningIface, which returns nil when ctrld is not
// running, and performs no DNS or service mutation.
s, err := newService(&prog{}, svcConfig)
if err != nil {
t.Fatalf("newService: %v", err)
}
t.Run("iface flag already resolved but not yet copied", func(t *testing.T) {
iface = "eth-test"
p := &prog{}
// Precondition mirrors the buggy post-reboot state: an empty running
// interface would make resetDNS skip restoration entirely.
if p.runningIface != "" {
t.Fatalf("precondition: runningIface = %q, want empty", p.runningIface)
}
ensureRunningIfaceForInvalidUninstall(p, s)
if p.runningIface == "" {
t.Fatal("runningIface still empty after prepare: resetDNS would skip DNS " +
"restoration and leave the OS pointed at ctrld's local listener")
}
if p.runningIface != "eth-test" {
t.Fatalf("runningIface = %q, want the resolved iface %q", p.runningIface, "eth-test")
}
})
t.Run("iface unset falls back to auto-detected interface", func(t *testing.T) {
iface = ""
p := &prog{}
ensureRunningIfaceForInvalidUninstall(p, s)
// With iface unset the prep resolves "auto" to the default interface
// (defaultIfaceName never returns empty on the supported platforms), so
// resetDNS has a concrete interface to restore.
if p.runningIface == "" {
t.Fatal("runningIface still empty after prepare with iface unset: " +
"resetDNS would skip DNS restoration")
}
})
}
+195
View File
@@ -0,0 +1,195 @@
package cli
import (
"context"
"errors"
"fmt"
"os"
"os/exec"
"strings"
"time"
"github.com/kardianos/service"
)
const (
// upgradeStopTimeout bounds how long rollback waits for the replacement process to
// exit, and for Windows to release the lock on its image afterwards.
upgradeStopTimeout = 30 * time.Second
// upgradeStopPollInterval is how often the service status is re-checked while
// waiting for the process to exit.
upgradeStopPollInterval = 500 * time.Millisecond
// binaryVersionTimeout bounds the "--version" probe, so a binary that hangs on
// startup cannot hang the upgrade.
binaryVersionTimeout = 10 * time.Second
)
// rollbackToPreviousBinary restores oldBin over bin after the replacement failed to
// become ready, and restarts the service on the restored binary.
//
// stop must leave the replacement's process gone, because every step here modifies
// the executable that process is running from. It is called first for that reason:
// readiness failing does not mean the process exited - the service manager can report
// a started service whose process never became operational. Windows holds an
// exclusive lock on a running executable's image, so the previous code's
// os.Remove(bin) failed there with "Access is denied", and because that was fatal the
// restore never ran: the broken binary stayed installed with the previous one
// stranded at its _previous name.
//
// Stopping first also puts the host back in a known state, since a stopped ctrld
// holds no DNS or intercept enforcement.
func rollbackToPreviousBinary(bin, oldBin string, stop func() error, restart func() bool) error {
if err := stop(); err != nil {
mainLog.Load().Error().Err(err).Msg("Could not confirm the service stopped; not modifying its binary")
return err
}
// Only restore a previous binary that actually runs: a _previous file that exists
// but reports no version would replace a service that starts and hangs with one
// that cannot start at all.
//
// The probe is retried for the same reason removeBinaryWithRetry is: on Windows a
// single exec can fail transiently while antivirus scans the file or the disk is
// busy, and treating that as "no usable previous binary" leaves the host stopped
// with the broken binary installed - an end state worse than restoring a binary
// that turns out to be bad, which the restart check below catches.
//
// Running "--version" proves the file executes. It is not an authenticity check:
// nothing here compares a signature or checksum before a file becomes the installed
// service binary. That is acceptable only because the install directory is writable
// by administrators alone, which is this command's standing assumption.
prevVer, err := binaryVersionWithRetry(oldBin, upgradeStopTimeout)
if err != nil {
mainLog.Load().Error().Err(err).Msgf("Previous binary at %s is not usable, keeping it for inspection", oldBin)
mainLog.Load().Notice().Msgf("Service is stopped and %s is still the installed binary", bin)
return fmt.Errorf("upgrade failed and no usable previous binary to restore: %w", err)
}
mainLog.Load().Warn().Msgf("Restoring previous binary: %s (%s)", oldBin, prevVer)
if err := removeBinaryWithRetry(bin, upgradeStopTimeout); err != nil {
mainLog.Load().Error().Err(err).Msg("Failed to remove new binary")
mainLog.Load().Notice().Msg("Service is stopped")
return err
}
if err := os.Rename(oldBin, bin); err != nil {
mainLog.Load().Error().Err(err).Msg("Failed to restore old binary")
mainLog.Load().Notice().Msgf("Service is stopped and %s is missing; reinstall ctrld to recover", bin)
return err
}
if restart() {
mainLog.Load().Notice().Msgf("Restored previous binary successfully - %s", prevVer)
return nil
}
mainLog.Load().Error().Msg("Restored the previous binary but it did not become ready either")
return errors.New("upgrade failed and the restored binary did not become ready")
}
// stopServiceAndWait stops the service and waits until the service manager reports
// it stopped. Rollback needs the process gone, not merely asked to stop: a stop
// request returns before the process exits, and on Windows the executable stays
// locked until it does.
func stopServiceAndWait(s service.Service, timeout time.Duration) error {
if err := s.Stop(); err != nil {
// Not fatal: the service may already be stopped, or stopping may fail while
// the process is exiting anyway. The status poll below decides.
mainLog.Load().Debug().Err(err).Msg("Stop request failed, waiting for the process to exit anyway")
}
deadline := time.Now().Add(timeout)
statusReadable := false
var lastErr error
for {
status, err := s.Status()
switch {
case errors.Is(err, service.ErrNotInstalled):
return nil
case err == nil:
statusReadable = true
if status == service.StatusStopped {
return nil
}
default:
lastErr = err
}
if !time.Now().Before(deadline) {
if !statusReadable {
// The status was never readable, so "did not stop" was never observed -
// only "could not be observed". Refusing to continue here would leave the
// broken binary installed with the service stopped, which is the outcome
// rollback exists to avoid. Let the caller proceed: the remove is retried
// while the image is locked, and the restart check still has to pass
// before this reports success.
mainLog.Load().Warn().Err(lastErr).Msgf("Could not read service status within %s; continuing with rollback", timeout)
return nil
}
return fmt.Errorf("service did not stop within %s", timeout)
}
time.Sleep(upgradeStopPollInterval)
}
}
// binaryVersionWithRetry probes a binary's version, retrying transient exec failures
// until timeout. Only the last error is reported: the earlier attempts are noise once a
// retry has been made.
func binaryVersionWithRetry(path string, timeout time.Duration) (string, error) {
deadline := time.Now().Add(timeout)
for {
version, err := binaryVersionFn(path)
if err == nil {
return version, nil
}
if !time.Now().Before(deadline) {
return "", err
}
mainLog.Load().Debug().Err(err).Msgf("Version probe of %s failed, retrying", path)
time.Sleep(upgradeStopPollInterval)
}
}
// removeBinaryWithRetry removes path, retrying while it is still locked. Windows
// releases the lock on an executable's image asynchronously after its process exits,
// so a remove issued immediately after the service reports stopped can still fail
// with "Access is denied".
func removeBinaryWithRetry(path string, timeout time.Duration) error {
deadline := time.Now().Add(timeout)
for {
err := os.Remove(path)
if err == nil || errors.Is(err, os.ErrNotExist) {
return nil
}
if !time.Now().Before(deadline) {
return fmt.Errorf("could not remove %s within %s: %w", path, timeout, err)
}
time.Sleep(upgradeStopPollInterval)
}
}
// binaryVersionFn is indirected so rollback can be tested without staging a runnable
// executable per platform. The probe itself is covered directly against the test
// binary; see TestBinaryVersion.
var binaryVersionFn = binaryVersion
// binaryVersion runs path with "--version" and returns the version it reports. It
// answers "can this binary actually run on this host", which is what rollback needs
// to know before making a file the installed ctrld.
//
// On Windows path is ctrld.exe_previous, whose extension is not in PATHEXT. That
// resolves because os/exec only falls back to appending PATHEXT entries when the path
// has no extension at all (lp_windows.go findExecutable): with one present and the
// file on disk, it is used as-is. A suffix that left no extension - renaming
// oldBinSuffix such that the result is "ctrld_previous" - would break this probe with
// "executable file not found in %PATH%", and rollback would then refuse to restore a
// perfectly good binary.
func binaryVersion(path string) (string, error) {
ctx, cancel := context.WithTimeout(context.Background(), binaryVersionTimeout)
defer cancel()
out, err := exec.CommandContext(ctx, path, "--version").CombinedOutput()
if err != nil {
return "", fmt.Errorf("running %s --version: %w", path, err)
}
ver, found := strings.CutPrefix(strings.TrimSpace(string(out)), "ctrld version ")
if !found {
return "", fmt.Errorf("unexpected --version output from %s: %q", path, strings.TrimSpace(string(out)))
}
return ver, nil
}
+288
View File
@@ -0,0 +1,288 @@
package cli
import (
"errors"
"os"
"path/filepath"
"testing"
"time"
"github.com/kardianos/service"
)
// fakeService implements the parts of service.Service that rollback uses. Any other
// method panics, which keeps accidental dependencies visible.
type fakeService struct {
service.Service
stopErr error
stopCalls int
statuses []service.Status // consumed one per Status() call; the last repeats
statusErr error
onStopCall func()
}
func (f *fakeService) Stop() error {
f.stopCalls++
if f.onStopCall != nil {
f.onStopCall()
}
return f.stopErr
}
func (f *fakeService) Status() (service.Status, error) {
if f.statusErr != nil {
return service.StatusUnknown, f.statusErr
}
if len(f.statuses) == 0 {
return service.StatusStopped, nil
}
st := f.statuses[0]
if len(f.statuses) > 1 {
f.statuses = f.statuses[1:]
}
return st, nil
}
func TestStopServiceAndWait(t *testing.T) {
tests := []struct {
name string
svc *fakeService
timeout time.Duration
wantErr bool
}{
{
name: "stops after a few polls",
svc: &fakeService{statuses: []service.Status{service.StatusRunning, service.StatusRunning, service.StatusStopped}},
timeout: 5 * time.Second,
},
{
name: "already stopped",
svc: &fakeService{statuses: []service.Status{service.StatusStopped}},
timeout: 5 * time.Second,
},
{
// A stop request that errors is not fatal on its own: the process may be
// exiting anyway, so the status poll decides.
name: "stop errors but service is stopped",
svc: &fakeService{stopErr: errors.New("already stopped"), statuses: []service.Status{service.StatusStopped}},
timeout: 5 * time.Second,
},
{
name: "not installed",
svc: &fakeService{statusErr: service.ErrNotInstalled},
timeout: 5 * time.Second,
},
{
// The process never exits. Rollback must be told so, because modifying a
// running executable is what produced "Access is denied".
name: "never stops",
svc: &fakeService{statuses: []service.Status{service.StatusRunning}},
timeout: time.Millisecond,
wantErr: true,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
err := stopServiceAndWait(tc.svc, tc.timeout)
if tc.wantErr && err == nil {
t.Fatal("expected an error, got nil")
}
if !tc.wantErr && err != nil {
t.Fatalf("unexpected error: %v", err)
}
if tc.svc.stopCalls != 1 {
t.Errorf("Stop() called %d times, want 1", tc.svc.stopCalls)
}
})
}
}
func TestRemoveBinaryWithRetry(t *testing.T) {
t.Run("removes an existing file", func(t *testing.T) {
path := filepath.Join(t.TempDir(), "ctrld")
if err := os.WriteFile(path, []byte("binary"), 0o755); err != nil {
t.Fatal(err)
}
if err := removeBinaryWithRetry(path, time.Second); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if _, err := os.Stat(path); !errors.Is(err, os.ErrNotExist) {
t.Errorf("file still exists after removal: %v", err)
}
})
t.Run("missing file is not an error", func(t *testing.T) {
path := filepath.Join(t.TempDir(), "absent")
if err := removeBinaryWithRetry(path, time.Second); err != nil {
t.Fatalf("unexpected error: %v", err)
}
})
t.Run("gives up and reports when the path cannot be removed", func(t *testing.T) {
// A non-empty directory stands in for a locked executable: os.Remove keeps
// failing, so the retry loop must surface the error rather than hang.
dir := filepath.Join(t.TempDir(), "locked")
if err := os.Mkdir(dir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "child"), nil, 0o644); err != nil {
t.Fatal(err)
}
if err := removeBinaryWithRetry(dir, time.Millisecond); err == nil {
t.Fatal("expected an error for a path that cannot be removed")
}
})
}
func TestBinaryVersion(t *testing.T) {
t.Run("reports the version", func(t *testing.T) {
t.Setenv(envFakeVersionOutput, "ctrld version dev-94fbd3f")
got, err := binaryVersion(os.Args[0])
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != "dev-94fbd3f" {
t.Errorf("binaryVersion() = %q, want %q", got, "dev-94fbd3f")
}
})
t.Run("rejects a binary that prints no version", func(t *testing.T) {
// A ctrld.exe_previous that exists and runs, but produces no version output.
// Restoring it would replace a hung service with one that cannot start at all.
t.Setenv(envFakeVersionOutput, envFakeVersionSilent)
if _, err := binaryVersion(os.Args[0]); err == nil {
t.Fatal("expected an error for a binary with no version output")
}
})
t.Run("rejects a missing binary", func(t *testing.T) {
if _, err := binaryVersion(filepath.Join(t.TempDir(), "absent")); err == nil {
t.Fatal("expected an error for a missing binary")
}
})
}
// stubBinaryVersion makes the version probe report ver for any path, so a rollback
// test does not have to stage a runnable executable.
//
// Staging one is not portable: oldBin is bin+"_previous", so a fixture named "ctrld"
// yields the extension-less "ctrld_previous", which Windows refuses to execute
// ("executable file not found in %PATH%"), and a symlink to the test binary needs a
// privilege Windows does not grant by default. The probe itself is covered against the
// real test binary in TestBinaryVersion; these tests are about rollback's ordering.
func stubBinaryVersion(t *testing.T, ver string, err error) {
t.Helper()
prev := binaryVersionFn
binaryVersionFn = func(string) (string, error) { return ver, err }
t.Cleanup(func() { binaryVersionFn = prev })
}
func TestRollbackToPreviousBinaryStopsBeforeTouchingTheBinary(t *testing.T) {
dir := t.TempDir()
bin := filepath.Join(dir, "ctrld")
oldBin := bin + oldBinSuffix
if err := os.WriteFile(bin, []byte("replacement"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(oldBin, []byte("previous"), 0o755); err != nil {
t.Fatal(err)
}
stubBinaryVersion(t, "dev-a75d669", nil)
// The invariant: when stop runs, the replacement's executable is still untouched.
// Reversing these two is exactly the "Access is denied" defect.
var stopped bool
var binExistedAtStop bool
stop := func() error {
stopped = true
_, err := os.Stat(bin)
binExistedAtStop = err == nil
return nil
}
restarted := false
restart := func() bool { restarted = true; return true }
if err := rollbackToPreviousBinary(bin, oldBin, stop, restart); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !stopped {
t.Error("rollback did not stop the service")
}
if !binExistedAtStop {
t.Error("the binary was modified before the service was stopped")
}
if !restarted {
t.Error("rollback did not restart the service")
}
if _, err := os.Stat(oldBin); !errors.Is(err, os.ErrNotExist) {
t.Errorf("previous binary was not moved into place: %v", err)
}
if _, err := os.Stat(bin); err != nil {
t.Errorf("restored binary is missing: %v", err)
}
}
func TestRollbackToPreviousBinaryKeepsUnusablePrevious(t *testing.T) {
dir := t.TempDir()
bin := filepath.Join(dir, "ctrld")
oldBin := bin + oldBinSuffix
if err := os.WriteFile(bin, []byte("replacement"), 0o755); err != nil {
t.Fatal(err)
}
// A previous binary that exists but does not report a version.
if err := os.WriteFile(oldBin, []byte("not a working binary"), 0o755); err != nil {
t.Fatal(err)
}
// Stubbed rather than left to the real probe: that would fail here for the right
// reason on unix (not an executable) but the wrong one on Windows (the fixture's
// name has no extension), so the assertion would not be about usability at all.
stubBinaryVersion(t, "", errors.New("unexpected --version output"))
stopped := false
restarted := false
err := rollbackToPreviousBinary(bin, oldBin,
func() error { stopped = true; return nil },
func() bool { restarted = true; return true },
)
if err == nil {
t.Fatal("expected an error when the previous binary is unusable")
}
if !stopped {
t.Error("the service must still be stopped: a broken replacement holds enforcement")
}
if restarted {
t.Error("must not restart the service with an unusable binary")
}
// Nothing was swapped, and the previous file is kept for inspection.
if _, err := os.Stat(oldBin); err != nil {
t.Errorf("unusable previous binary was not preserved: %v", err)
}
if _, err := os.Stat(bin); err != nil {
t.Errorf("installed binary was removed despite having nothing to restore: %v", err)
}
}
func TestRollbackToPreviousBinaryAbortsWhenStopFails(t *testing.T) {
dir := t.TempDir()
bin := filepath.Join(dir, "ctrld")
oldBin := bin + oldBinSuffix
for _, p := range []string{bin, oldBin} {
if err := os.WriteFile(p, []byte("binary"), 0o755); err != nil {
t.Fatal(err)
}
}
stopErr := errors.New("service did not stop within 30s")
err := rollbackToPreviousBinary(bin, oldBin,
func() error { return stopErr },
func() bool { t.Error("must not restart after a failed stop"); return false },
)
if !errors.Is(err, stopErr) {
t.Fatalf("error = %v, want %v", err, stopErr)
}
// The executable of a process that may still be running must be left alone.
if _, err := os.Stat(bin); err != nil {
t.Errorf("binary was modified even though the stop failed: %v", err)
}
}
+19
View File
@@ -12,8 +12,27 @@ const (
maxFailureRequest = 50
// checkUpstreamBackoffSleep is the time interval between each upstream checks.
checkUpstreamBackoffSleep = 2 * time.Second
// checkUpstreamUnreachableBackoffMax caps the recovery retry interval for an
// endpoint that keeps failing with a network-unreachable error. It bounds
// the backoff so an unroutable endpoint is still re-probed periodically and
// recovers once the route returns.
checkUpstreamUnreachableBackoffMax = 60 * time.Second
)
// unreachableRecoveryBackoff returns the retry interval for the given streak of
// consecutive network-unreachable failures. It starts at checkUpstreamBackoffSleep
// and doubles each attempt, capped at checkUpstreamUnreachableBackoffMax.
func unreachableRecoveryBackoff(streak int) time.Duration {
d := checkUpstreamBackoffSleep
for i := 1; i < streak; i++ {
d *= 2
if d >= checkUpstreamUnreachableBackoffMax {
return checkUpstreamUnreachableBackoffMax
}
}
return d
}
// upstreamMonitor performs monitoring upstreams health.
type upstreamMonitor struct {
cfg *ctrld.Config
+469
View File
@@ -0,0 +1,469 @@
package cli
import (
"context"
"net"
"runtime"
"strings"
"sync"
"github.com/rs/zerolog"
"tailscale.com/net/netmon"
"github.com/Control-D-Inc/ctrld"
)
var vpnDNSSettlingEnabled = runtime.GOOS == "windows"
// vpnDNSExemption represents a VPN DNS server that needs pf/WFP exemption,
// including the interface it was discovered on. The interface is used on macOS
// to create interface-scoped pf exemptions that allow the VPN's local DNS
// handler (e.g., Tailscale's MagicDNS Network Extension) to receive queries
// from all processes — not just ctrld.
type vpnDNSExemption struct {
Server string // DNS server IP (e.g., "100.100.100.100")
Interface string // Interface name from scutil (e.g., "utun11"), may be empty
IsExitMode bool // True if this VPN is in exit/full-tunnel mode (all traffic routed through VPN)
}
// vpnDNSExemptFunc is called when VPN DNS servers change, to update
// the intercept layer (WFP/pf) to permit VPN DNS traffic.
type vpnDNSExemptFunc func(exemptions []vpnDNSExemption) error
// vpnDNSManager tracks active VPN DNS configurations and provides
// domain-to-upstream routing for VPN split DNS.
type vpnDNSManager struct {
mu sync.RWMutex
configs []ctrld.VPNDNSConfig
// Map of domain suffix → DNS servers for fast lookup
routes map[string][]string
// DNS servers from VPN interfaces that have no domain/suffix config.
// These are NOT added to the global OS resolver. They're only used
// as additional nameservers for queries that match split-DNS rules
// (from ctrld config, AD domain, or VPN suffix config).
domainlessServers []string
// appliedExemptions advances only after the platform PF/WFP callback succeeds.
// Keeping it separate from discovered configs makes failed rule updates retryable.
appliedExemptions []vpnDNSExemption
// retainedAfterEmptyDiscovery means Windows reported an empty VPN DNS
// snapshot once while previous VPN DNS state existed. We keep that last-known
// state for one guarded refresh cycle because Windows can briefly report an
// intermediate empty adapter/DNS state after sleep/wake or reconnect.
retainedAfterEmptyDiscovery bool
// discoverVPNDNS is injected for tests so Refresh does not depend on the
// runner host's real VPN/virtual adapter state.
discoverVPNDNS func(context.Context) []ctrld.VPNDNSConfig
// refreshStateMu keeps noisy network-change storms from running overlapping
// full VPN DNS refreshes and retains one trailing refresh when an event arrives
// during discovery so the newest OS state is not lost.
refreshStateMu sync.Mutex
refreshRunning bool
refreshPending bool
discoveryMu sync.Mutex
// Called when VPN DNS server list changes, to update intercept exemptions.
onServersChanged vpnDNSExemptFunc
}
// newVPNDNSManager creates a new manager. Only call when dnsIntercept is active.
// exemptFunc is called whenever VPN DNS servers are discovered/changed, to update
// the OS-level intercept rules to permit ctrld's outbound queries to those IPs.
func newVPNDNSManager(exemptFunc vpnDNSExemptFunc) *vpnDNSManager {
return &vpnDNSManager{
routes: make(map[string][]string),
discoverVPNDNS: ctrld.DiscoverVPNDNS,
onServersChanged: exemptFunc,
}
}
// Refresh re-discovers VPN DNS configs from the OS.
// Called on network change events. Overlapping calls are coalesced into one
// trailing refresh so a newer OS snapshot is never silently discarded.
func (m *vpnDNSManager) Refresh(guardAgainstNoNameservers bool) {
m.refreshStateMu.Lock()
if m.refreshRunning {
m.refreshPending = true
m.refreshStateMu.Unlock()
mainLog.Load().Debug().Msg("VPN DNS refresh already running, coalescing trailing refresh")
return
}
m.refreshRunning = true
m.refreshStateMu.Unlock()
for {
m.refreshOnce(guardAgainstNoNameservers)
m.refreshStateMu.Lock()
if m.refreshPending {
m.refreshPending = false
m.refreshStateMu.Unlock()
guardAgainstNoNameservers = true
continue
}
m.refreshRunning = false
m.refreshStateMu.Unlock()
return
}
}
func (m *vpnDNSManager) refreshOnce(guardAgainstNoNameservers bool) {
logger := mainLog.Load()
m.discoveryMu.Lock()
defer m.discoveryMu.Unlock()
logger.Debug().Msg("Refreshing VPN DNS configurations")
discoverVPNDNS := m.discoverVPNDNS
if discoverVPNDNS == nil {
discoverVPNDNS = ctrld.DiscoverVPNDNS
}
configs := discoverVPNDNS(context.Background())
// Detect exit mode: if the default route goes through a VPN DNS interface,
// the VPN is routing ALL traffic (exit node / full tunnel). This is more
// reliable than scutil flag parsing because the routing table is the ground
// truth for traffic flow, regardless of how the VPN presents itself in scutil.
if dri, err := netmon.DefaultRouteInterface(); err == nil && dri != "" {
for i := range configs {
if configs[i].InterfaceName == dri {
if !configs[i].IsExitMode {
logger.Info().Msgf("VPN DNS on %s: default route interface match — EXIT MODE (route-based detection)", dri)
}
configs[i].IsExitMode = true
}
}
}
m.mu.Lock()
defer m.mu.Unlock()
if vpnDNSSettlingEnabled && len(configs) == 0 && guardAgainstNoNameservers && m.hasVPNDNSStateLocked() {
if !m.retainedAfterEmptyDiscovery {
exemptions := m.currentExemptionsLocked()
m.retainedAfterEmptyDiscovery = true
logger.Debug().Msgf(
"VPN DNS discovery empty; retaining last-known VPN DNS state for one guarded refresh (%d domainless servers, %d exemptions)",
len(m.domainlessServers), len(exemptions))
if m.onServersChanged != nil {
if err := m.onServersChanged(exemptions); err != nil {
logger.Error().Err(err).Msg("Failed to re-apply retained VPN DNS exemptions")
} else {
m.appliedExemptions = append([]vpnDNSExemption(nil), exemptions...)
}
}
return
}
logger.Debug().Msgf(
"VPN DNS discovery still empty on next guarded refresh; clearing retained VPN DNS state (%d domainless servers)",
len(m.domainlessServers))
}
// Any discovery path that does not return with retained state clears the
// settling marker: non-empty discovery replaces old servers immediately, and
// an unguarded/second empty discovery clears stale state below.
m.retainedAfterEmptyDiscovery = false
m.configs = configs
m.routes = make(map[string][]string)
// Build domain -> DNS servers mapping
for _, config := range configs {
logger.Debug().Msgf("Processing VPN interface %s with %d domains and %d servers",
config.InterfaceName, len(config.Domains), len(config.Servers))
for _, domain := range config.Domains {
// Normalize domain: remove leading dot, Linux routing domain prefix (~),
// and convert to lowercase.
domain = strings.TrimPrefix(domain, "~")
domain = strings.TrimPrefix(domain, ".")
domain = strings.ToLower(domain)
if domain != "" {
m.routes[domain] = append([]string{}, config.Servers...)
logger.Debug().Msgf("Added VPN DNS route: %s -> %v", domain, config.Servers)
}
}
}
// Collect unique VPN DNS exemptions (server + interface) for pf/WFP rules.
type exemptionKey struct{ server, iface string }
seen := make(map[exemptionKey]bool)
var exemptions []vpnDNSExemption
for _, config := range configs {
for _, server := range config.Servers {
key := exemptionKey{server, config.InterfaceName}
if !seen[key] {
seen[key] = true
exemptions = append(exemptions, vpnDNSExemption{
Server: server,
Interface: config.InterfaceName,
IsExitMode: config.IsExitMode,
})
}
}
}
// Collect domain-less VPN DNS servers. These are NOT added to the global
// OS resolver (that would pollute captive portal / DHCP flows). Instead,
// they're stored separately and only used for queries that match existing
// split-DNS rules (from ctrld config, AD domain, or VPN suffix config).
var domainlessServers []string
seen2 := make(map[string]bool)
for _, config := range configs {
if len(config.Domains) == 0 && len(config.Servers) > 0 {
logger.Debug().Msgf("VPN interface %s has DNS servers but no domains, storing as split-rule fallback: %v",
config.InterfaceName, config.Servers)
for _, s := range config.Servers {
if !seen2[s] {
seen2[s] = true
domainlessServers = append(domainlessServers, s)
}
}
}
}
m.domainlessServers = domainlessServers
logger.Debug().Msgf("VPN DNS refresh completed: %d configs, %d routes, %d domainless servers, %d unique exemptions",
len(m.configs), len(m.routes), len(m.domainlessServers), len(exemptions))
// Update intercept rules only when desired exemptions differ from the last
// successfully applied set. Failed PF/WFP callbacks remain retryable on the
// next refresh even when discovery returns the same VPN DNS state.
m.updateInterceptExemptionsIfChanged(logger, exemptions, "VPN DNS")
}
func (m *vpnDNSManager) updateInterceptExemptionsIfChanged(logger *zerolog.Logger, desired []vpnDNSExemption, reason string) {
if m.onServersChanged == nil {
return
}
if vpnDNSExemptionsEqual(m.appliedExemptions, desired) {
logger.Debug().Msgf("VPN DNS exemptions unchanged after %s refresh; skipping intercept rule update", reason)
return
}
if err := m.onServersChanged(desired); err != nil {
logger.Error().Err(err).Msg("Failed to update intercept exemptions for VPN DNS servers")
return
}
m.appliedExemptions = append([]vpnDNSExemption(nil), desired...)
}
// RefreshRoutesOnly re-discovers VPN DNS configs and updates ctrld's
// in-memory split-DNS routes. It applies intercept exemptions only when that set
// changes, while holding the shared discovery lane so a concurrent full refresh
// cannot commit a newer snapshot and then be overwritten by this one.
func (m *vpnDNSManager) RefreshRoutesOnly() (routes, domainlessServers, exemptions int) {
logger := mainLog.Load()
m.discoveryMu.Lock()
defer m.discoveryMu.Unlock()
logger.Debug().Msg("Refreshing VPN DNS route state only")
discoverVPNDNS := m.discoverVPNDNS
if discoverVPNDNS == nil {
discoverVPNDNS = ctrld.DiscoverVPNDNS
}
configs := discoverVPNDNS(context.Background())
if dri, err := netmon.DefaultRouteInterface(); err == nil && dri != "" {
for i := range configs {
if configs[i].InterfaceName == dri {
configs[i].IsExitMode = true
}
}
}
m.mu.Lock()
defer m.mu.Unlock()
m.retainedAfterEmptyDiscovery = false
m.configs = configs
m.routes = make(map[string][]string)
for _, config := range configs {
for _, domain := range config.Domains {
domain = strings.TrimPrefix(domain, "~")
domain = strings.TrimPrefix(domain, ".")
domain = strings.ToLower(domain)
if domain != "" {
m.routes[domain] = append([]string{}, config.Servers...)
}
}
}
var domainless []string
seenDomainless := make(map[string]bool)
for _, config := range configs {
if len(config.Domains) == 0 && len(config.Servers) > 0 {
for _, server := range config.Servers {
if !seenDomainless[server] {
seenDomainless[server] = true
domainless = append(domainless, server)
}
}
}
}
m.domainlessServers = domainless
currentExemptions := m.currentExemptionsLocked()
logger.Debug().Msgf("VPN DNS route-only refresh completed: %d configs, %d routes, %d domainless servers, %d exemptions",
len(m.configs), len(m.routes), len(m.domainlessServers), len(currentExemptions))
m.updateInterceptExemptionsIfChanged(logger, currentExemptions, "route-only VPN DNS")
return len(m.routes), len(m.domainlessServers), len(currentExemptions)
}
func (m *vpnDNSManager) markInterceptExemptionsApplied(applied []vpnDNSExemption) {
m.mu.Lock()
defer m.mu.Unlock()
if vpnDNSExemptionsEqual(m.currentExemptionsLocked(), applied) {
m.appliedExemptions = append([]vpnDNSExemption(nil), applied...)
}
}
func (m *vpnDNSManager) interceptExemptionsPending() bool {
m.mu.RLock()
defer m.mu.RUnlock()
return !vpnDNSExemptionsEqual(m.appliedExemptions, m.currentExemptionsLocked())
}
func (m *vpnDNSManager) hasVPNDNSStateLocked() bool {
return len(m.configs) > 0 || len(m.routes) > 0 || len(m.domainlessServers) > 0
}
func (m *vpnDNSManager) currentExemptionsLocked() []vpnDNSExemption {
type key struct{ server, iface string }
seen := make(map[key]bool)
var exemptions []vpnDNSExemption
for _, config := range m.configs {
for _, server := range config.Servers {
k := key{server, config.InterfaceName}
if seen[k] {
continue
}
seen[k] = true
exemptions = append(exemptions, vpnDNSExemption{
Server: server,
Interface: config.InterfaceName,
IsExitMode: config.IsExitMode,
})
}
}
return exemptions
}
// ShouldFailClosedAfterVPNDNSTransportFailure reports whether split-rule
// queries should fail closed instead of falling back to OS/public DNS after
// every candidate VPN DNS server failed before returning a DNS packet. This is
// Windows-only and only active while serving retained VPN DNS state from a
// guarded empty discovery, which is the short window where Windows can report
// VPN DNS before routes to those servers are usable after wake/reconnect.
func (m *vpnDNSManager) ShouldFailClosedAfterVPNDNSTransportFailure(domain string, servers []string) bool {
m.mu.RLock()
defer m.mu.RUnlock()
if !vpnDNSSettlingEnabled || len(servers) == 0 || !m.retainedAfterEmptyDiscovery || !m.hasVPNDNSStateLocked() {
return false
}
mainLog.Load().Debug().Msgf(
"VPN DNS transport failed for %s while retained VPN DNS state is active; suppressing OS fallback for this query (servers=%v)",
domain, servers)
return true
}
// VPNDNSReachable records that a VPN DNS server returned a DNS response. The
// response may be negative (NXDOMAIN/SERVFAIL); the important signal is that
// the VPN DNS transport is reachable again.
func (m *vpnDNSManager) VPNDNSReachable() {
m.mu.Lock()
defer m.mu.Unlock()
if m.retainedAfterEmptyDiscovery {
mainLog.Load().Debug().Msg("VPN DNS transport recovered; clearing retained-empty-discovery state")
}
m.retainedAfterEmptyDiscovery = false
}
// UpstreamForDomain checks if the domain matches any VPN search domain.
// Returns VPN DNS servers if matched, nil otherwise.
func (m *vpnDNSManager) UpstreamForDomain(domain string) []string {
if domain == "" {
return nil
}
m.mu.RLock()
defer m.mu.RUnlock()
domain = strings.TrimSuffix(domain, ".")
domain = strings.ToLower(domain)
if servers, ok := m.routes[domain]; ok {
return append([]string{}, servers...)
}
for vpnDomain, servers := range m.routes {
if strings.HasSuffix(domain, "."+vpnDomain) {
return append([]string{}, servers...)
}
}
return nil
}
// DomainlessServers returns VPN DNS servers that have no associated domains.
// These should only be used for queries matching split-DNS rules, not for
// general OS resolver queries (to avoid polluting captive portal / DHCP flows).
func (m *vpnDNSManager) DomainlessServers() []string {
m.mu.RLock()
defer m.mu.RUnlock()
return append([]string{}, m.domainlessServers...)
}
// CurrentServers returns the current set of unique VPN DNS server IPs.
func (m *vpnDNSManager) CurrentServers() []string {
m.mu.RLock()
defer m.mu.RUnlock()
seen := make(map[string]bool)
var servers []string
for _, ss := range m.routes {
for _, s := range ss {
if !seen[s] {
seen[s] = true
servers = append(servers, s)
}
}
}
return servers
}
// CurrentExemptions returns VPN DNS server + interface pairs for pf exemption rules.
func (m *vpnDNSManager) CurrentExemptions() []vpnDNSExemption {
m.mu.RLock()
defer m.mu.RUnlock()
return m.currentExemptionsLocked()
}
// Routes returns a copy of the current VPN DNS routes for debugging.
func (m *vpnDNSManager) Routes() map[string][]string {
m.mu.RLock()
defer m.mu.RUnlock()
routes := make(map[string][]string)
for domain, servers := range m.routes {
routes[domain] = append([]string{}, servers...)
}
return routes
}
// upstreamConfigFor creates a legacy upstream configuration for the given VPN DNS server.
func (m *vpnDNSManager) upstreamConfigFor(server string) *ctrld.UpstreamConfig {
// Use net.JoinHostPort to correctly handle both IPv4 and IPv6 addresses.
// Previously, the strings.Contains(":") check would skip appending ":53"
// for IPv6 addresses (they contain colons), leaving a bare address like
// "2a0d:6fc0:9b0:3600::1" which net.Dial rejects with "too many colons".
// net.JoinHostPort produces "[2a0d:6fc0:9b0:3600::1]:53" as required.
endpoint := net.JoinHostPort(server, "53")
return &ctrld.UpstreamConfig{
Name: "VPN DNS",
Type: ctrld.ResolverTypeLegacy,
Endpoint: endpoint,
Timeout: 2000,
}
}
+298
View File
@@ -0,0 +1,298 @@
package cli
import (
"context"
"errors"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/Control-D-Inc/ctrld"
)
func withVPNDNSSettlingEnabled(t *testing.T) {
t.Helper()
old := vpnDNSSettlingEnabled
vpnDNSSettlingEnabled = true
t.Cleanup(func() { vpnDNSSettlingEnabled = old })
}
func TestVPNDNSRefreshCoalescesConcurrentTrailingRefresh(t *testing.T) {
m := newVPNDNSManager(nil)
started := make(chan struct{})
release := make(chan struct{})
done := make(chan struct{})
var once sync.Once
var calls atomic.Int32
m.discoverVPNDNS = func(context.Context) []ctrld.VPNDNSConfig {
call := calls.Add(1)
once.Do(func() { close(started) })
<-release
if call == 2 {
return []ctrld.VPNDNSConfig{{
InterfaceName: "utun-latest",
Servers: []string{"10.0.0.2"},
Domains: []string{"latest.internal"},
}}
}
return nil
}
go func() {
defer close(done)
m.Refresh(true)
}()
<-started
m.Refresh(true)
close(release)
<-done
if calls.Load() != 2 {
t.Fatalf("expected one active and one trailing discovery call, got %d", calls.Load())
}
if got := m.Routes()["latest.internal"]; len(got) != 1 || got[0] != "10.0.0.2" {
t.Fatalf("trailing refresh did not publish latest OS snapshot: %v", got)
}
}
func TestVPNDNSRefreshRetainsStateForOneGuardedEmptyDiscovery(t *testing.T) {
withVPNDNSSettlingEnabled(t)
var gotExemptions []vpnDNSExemption
m := newVPNDNSManager(func(exemptions []vpnDNSExemption) error {
gotExemptions = exemptions
return nil
})
m.discoverVPNDNS = func(context.Context) []ctrld.VPNDNSConfig { return nil }
m.configs = []ctrld.VPNDNSConfig{{
InterfaceName: "Ethernet 6",
Servers: []string{"10.25.37.21", "10.25.37.22"},
}}
m.domainlessServers = []string{"10.25.37.21", "10.25.37.22"}
m.Refresh(true)
if got := m.DomainlessServers(); len(got) != 2 {
t.Fatalf("expected retained domainless servers, got %v", got)
}
if len(gotExemptions) != 2 {
t.Fatalf("expected retained exemptions to be re-applied, got %v", gotExemptions)
}
if !m.retainedAfterEmptyDiscovery {
t.Fatal("expected empty discovery retention to be marked")
}
}
func TestVPNDNSRefreshClearsOnSecondGuardedEmptyDiscovery(t *testing.T) {
withVPNDNSSettlingEnabled(t)
var gotExemptions []vpnDNSExemption
updates := 0
m := newVPNDNSManager(func(exemptions []vpnDNSExemption) error {
updates++
gotExemptions = exemptions
return nil
})
m.discoverVPNDNS = func(context.Context) []ctrld.VPNDNSConfig { return nil }
m.configs = []ctrld.VPNDNSConfig{{
InterfaceName: "Ethernet 6",
Servers: []string{"10.25.37.21"},
}}
m.domainlessServers = []string{"10.25.37.21"}
m.appliedExemptions = []vpnDNSExemption{{Server: "10.25.37.21", Interface: "Ethernet 6"}}
m.retainedAfterEmptyDiscovery = true
m.Refresh(true)
if got := m.DomainlessServers(); len(got) != 0 {
t.Fatalf("expected domainless servers to be cleared on second empty discovery, got %v", got)
}
if updates != 1 || len(gotExemptions) != 0 {
t.Fatalf("expected one empty exemption update after clearing stale state, calls=%d exemptions=%v", updates, gotExemptions)
}
if m.retainedAfterEmptyDiscovery {
t.Fatal("expected retained empty-discovery marker to be cleared with stale state")
}
}
func TestVPNDNSRefreshSkipsUnchangedInterceptExemptions(t *testing.T) {
var updates [][]vpnDNSExemption
m := newVPNDNSManager(func(exemptions []vpnDNSExemption) error {
updates = append(updates, append([]vpnDNSExemption{}, exemptions...))
return nil
})
m.discoverVPNDNS = func(context.Context) []ctrld.VPNDNSConfig {
return []ctrld.VPNDNSConfig{{
InterfaceName: "utun-test",
Servers: []string{"10.102.26.10"},
Domains: []string{"example.internal"},
}}
}
m.Refresh(true)
m.Refresh(true)
if len(updates) != 1 {
t.Fatalf("expected exactly one intercept exemption update for unchanged VPN DNS state, got %d", len(updates))
}
if len(updates[0]) != 1 || updates[0][0].Server != "10.102.26.10" || updates[0][0].Interface != "utun-test" {
t.Fatalf("unexpected exemption update: %+v", updates[0])
}
}
func TestVPNDNSRefreshRetriesFailedInterceptExemptionUpdate(t *testing.T) {
attempts := 0
m := newVPNDNSManager(func([]vpnDNSExemption) error {
attempts++
if attempts == 1 {
return errors.New("pf update failed")
}
return nil
})
m.discoverVPNDNS = func(context.Context) []ctrld.VPNDNSConfig {
return []ctrld.VPNDNSConfig{{
InterfaceName: "utun-test",
Servers: []string{"10.102.26.10"},
Domains: []string{"internal.test"},
}}
}
m.Refresh(true)
if !m.interceptExemptionsPending() {
t.Fatal("failed intercept exemption update was not retained for retry")
}
m.Refresh(true)
if m.interceptExemptionsPending() {
t.Fatal("successful intercept exemption retry did not advance applied state")
}
m.Refresh(true)
if attempts != 2 {
t.Fatalf("intercept exemption update attempts = %d, want failed attempt plus one retry", attempts)
}
if len(m.appliedExemptions) != 1 || m.appliedExemptions[0].Server != "10.102.26.10" {
t.Fatalf("applied exemptions = %+v, want successful retry state", m.appliedExemptions)
}
}
func TestVPNDNSMarkAppliedExemptionsRejectsStaleSnapshot(t *testing.T) {
m := newVPNDNSManager(nil)
m.configs = []ctrld.VPNDNSConfig{{InterfaceName: "utun-new", Servers: []string{"10.0.0.2"}}}
m.markInterceptExemptionsApplied([]vpnDNSExemption{{Server: "10.0.0.1", Interface: "utun-old"}})
if !m.interceptExemptionsPending() {
t.Fatal("stale PF snapshot incorrectly advanced applied exemptions")
}
m.markInterceptExemptionsApplied([]vpnDNSExemption{{Server: "10.0.0.2", Interface: "utun-new"}})
if m.interceptExemptionsPending() {
t.Fatal("current PF snapshot did not advance applied exemptions")
}
}
func TestVPNDNSTransportFailureSuppressesFallbackOnlyWhileRetainingState(t *testing.T) {
withVPNDNSSettlingEnabled(t)
m := newVPNDNSManager(nil)
m.domainlessServers = []string{"10.25.37.21"}
if m.ShouldFailClosedAfterVPNDNSTransportFailure("splunk.aws.arena.net.", []string{"10.25.37.21"}) {
t.Fatal("did not expect transport failure to suppress OS fallback outside retained empty-discovery state")
}
m.retainedAfterEmptyDiscovery = true
if !m.ShouldFailClosedAfterVPNDNSTransportFailure("splunk.aws.arena.net.", []string{"10.25.37.21"}) {
t.Fatal("expected transport failure to suppress OS fallback while retained state is active")
}
m.VPNDNSReachable()
if m.retainedAfterEmptyDiscovery {
t.Fatal("expected reachable DNS response to clear retained empty-discovery state")
}
}
func TestVPNDNSFullAndRouteOnlyDiscoveryAreSerialized(t *testing.T) {
var updateMu sync.Mutex
var exemptionUpdates []string
m := newVPNDNSManager(func(exemptions []vpnDNSExemption) error {
updateMu.Lock()
defer updateMu.Unlock()
if len(exemptions) == 0 {
exemptionUpdates = append(exemptionUpdates, "")
} else {
exemptionUpdates = append(exemptionUpdates, exemptions[0].Server)
}
return nil
})
firstStarted := make(chan struct{})
releaseFirst := make(chan struct{})
secondStarted := make(chan struct{})
var calls atomic.Int32
m.discoverVPNDNS = func(context.Context) []ctrld.VPNDNSConfig {
switch calls.Add(1) {
case 1:
close(firstStarted)
<-releaseFirst
return []ctrld.VPNDNSConfig{{
InterfaceName: "utun-old",
Servers: []string{"10.0.0.1"},
Domains: []string{"old.internal"},
}}
case 2:
close(secondStarted)
return []ctrld.VPNDNSConfig{{
InterfaceName: "utun-new",
Servers: []string{"10.0.0.2"},
Domains: []string{"new.internal"},
}}
default:
t.Fatalf("unexpected discovery call %d", calls.Load())
return nil
}
}
routesDone := make(chan struct{})
go func() {
defer close(routesDone)
m.RefreshRoutesOnly()
}()
<-firstStarted
fullDone := make(chan struct{})
go func() {
defer close(fullDone)
m.Refresh(false)
}()
select {
case <-secondStarted:
t.Fatal("full and route-only VPN DNS discovery overlapped")
case <-time.After(50 * time.Millisecond):
}
close(releaseFirst)
select {
case <-routesDone:
case <-time.After(time.Second):
t.Fatal("route-only refresh did not finish")
}
select {
case <-fullDone:
case <-time.After(time.Second):
t.Fatal("full refresh did not finish")
}
routes := m.Routes()
if _, ok := routes["old.internal"]; ok {
t.Fatalf("older route-only snapshot overwrote newer full refresh: %v", routes)
}
if got := routes["new.internal"]; len(got) != 1 || got[0] != "10.0.0.2" {
t.Fatalf("final VPN DNS routes = %v, want new.internal -> 10.0.0.2", routes)
}
updateMu.Lock()
defer updateMu.Unlock()
if len(exemptionUpdates) != 2 || exemptionUpdates[0] != "10.0.0.1" || exemptionUpdates[1] != "10.0.0.2" {
t.Fatalf("serialized exemption updates = %v, want old then new", exemptionUpdates)
}
}
+149 -63
View File
@@ -9,7 +9,6 @@ import (
"errors"
"fmt"
"io"
"math/rand"
"net"
"net/http"
"net/netip"
@@ -83,6 +82,10 @@ const (
endpointPrefixQUIC = "quic://"
endpointPrefixH3 = "h3://"
endpointPrefixSdns = "sdns://"
rebootstrapNotStarted = 0
rebootstrapStarted = 1
rebootstrapInProgress = 2
)
var (
@@ -237,6 +240,9 @@ type ServiceConfig struct {
RefetchTime *int `mapstructure:"refetch_time" toml:"refetch_time,omitempty"`
ForceRefetchWaitTime *int `mapstructure:"force_refetch_wait_time" toml:"force_refetch_wait_time,omitempty"`
LeakOnUpstreamFailure *bool `mapstructure:"leak_on_upstream_failure" toml:"leak_on_upstream_failure,omitempty"`
InterceptMode string `mapstructure:"intercept_mode" toml:"intercept_mode,omitempty" validate:"omitempty,oneof=off dns hard"`
NRPTRecoveryMaxAttempts *int `mapstructure:"nrpt_recovery_max_attempts" toml:"nrpt_recovery_max_attempts,omitempty" validate:"omitempty,gte=0"`
NRPTRecoveryCooldown *time.Duration `mapstructure:"nrpt_recovery_cooldown" toml:"nrpt_recovery_cooldown,omitempty"`
Daemon bool `mapstructure:"-" toml:"-"`
AllocateIP bool `mapstructure:"-" toml:"-"`
}
@@ -265,7 +271,7 @@ type UpstreamConfig struct {
Discoverable *bool `mapstructure:"discoverable" toml:"discoverable"`
g singleflight.Group
rebootstrap atomic.Bool
rebootstrap atomic.Int64
bootstrapIPs []string
bootstrapIPs4 []string
bootstrapIPs6 []string
@@ -276,6 +282,12 @@ type UpstreamConfig struct {
http3RoundTripper http.RoundTripper
http3RoundTripper4 http.RoundTripper
http3RoundTripper6 http.RoundTripper
doqConnPool *doqConnPool
doqConnPool4 *doqConnPool
doqConnPool6 *doqConnPool
dotClientPool *dotConnPool
dotClientPool4 *dotConnPool
dotClientPool6 *dotConnPool
certPool *x509.CertPool
u *url.URL
fallbackOnce sync.Once
@@ -490,54 +502,147 @@ func (uc *UpstreamConfig) SetupBootstrapIP() {
// ReBootstrap re-setup the bootstrap IP and the transport.
func (uc *UpstreamConfig) ReBootstrap() {
switch uc.Type {
case ResolverTypeDOH, ResolverTypeDOH3:
case ResolverTypeDOH, ResolverTypeDOH3, ResolverTypeDOQ, ResolverTypeDOT:
default:
return
}
_, _, _ = uc.g.Do("ReBootstrap", func() (any, error) {
if uc.rebootstrap.CompareAndSwap(false, true) {
if uc.rebootstrap.CompareAndSwap(rebootstrapNotStarted, rebootstrapStarted) {
ProxyLogger.Load().Debug().Msgf("re-bootstrapping upstream ip for %v", uc)
}
return true, nil
})
}
// SetupTransport initializes the network transport used to connect to upstream server.
// For now, only DoH upstream is supported.
func (uc *UpstreamConfig) SetupTransport() {
// ForceReBootstrap immediately replaces the upstream transport, closing old
// connections and creating new ones synchronously. Unlike ReBootstrap() which
// sets a lazy flag (new transport created on next query), this ensures the
// transport is ready before any queries arrive. Use when external events
// (e.g. firewall state flush) are known to have killed existing connections.
func (uc *UpstreamConfig) ForceReBootstrap() {
switch uc.Type {
case ResolverTypeDOH:
uc.setupDOHTransport()
case ResolverTypeDOH3:
uc.setupDOH3Transport()
case ResolverTypeDOH, ResolverTypeDOH3, ResolverTypeDOQ, ResolverTypeDOT:
default:
return
}
ProxyLogger.Load().Debug().Msgf("force re-bootstrapping upstream transport for %v", uc)
uc.SetupTransport()
// Clear any pending lazy re-bootstrap flag so ensureSetupTransport()
// doesn't redundantly recreate the transport we just built.
uc.rebootstrap.Store(rebootstrapNotStarted)
}
// closeTransports closes idle connections on all existing transports.
// This is called before creating new transports during re-bootstrap to
// force in-flight requests on stale connections to fail quickly, rather
// than waiting for the full context deadline (e.g. 5s) after a firewall
// state table flush kills the underlying TCP/QUIC connections.
func (uc *UpstreamConfig) closeTransports() {
if t := uc.transport; t != nil {
t.CloseIdleConnections()
}
if t := uc.transport4; t != nil {
t.CloseIdleConnections()
}
if t := uc.transport6; t != nil {
t.CloseIdleConnections()
}
if p := uc.doqConnPool; p != nil {
p.CloseIdleConnections()
}
if p := uc.doqConnPool4; p != nil {
p.CloseIdleConnections()
}
if p := uc.doqConnPool6; p != nil {
p.CloseIdleConnections()
}
if p := uc.dotClientPool; p != nil {
p.CloseIdleConnections()
}
if p := uc.dotClientPool4; p != nil {
p.CloseIdleConnections()
}
if p := uc.dotClientPool6; p != nil {
p.CloseIdleConnections()
}
// http3RoundTripper is stored as http.RoundTripper but the concrete type
// (*http3.Transport) exposes CloseIdleConnections via this interface.
type idleCloser interface {
CloseIdleConnections()
}
for _, rt := range []http.RoundTripper{uc.http3RoundTripper, uc.http3RoundTripper4, uc.http3RoundTripper6} {
if c, ok := rt.(idleCloser); ok {
c.CloseIdleConnections()
}
}
}
func (uc *UpstreamConfig) setupDOHTransport() {
// SetupTransport initializes the network transport used to connect to upstream servers.
// For now, DoH/DoH3/DoQ/DoT upstreams are supported.
func (uc *UpstreamConfig) SetupTransport() {
switch uc.Type {
case ResolverTypeDOH, ResolverTypeDOH3, ResolverTypeDOQ, ResolverTypeDOT:
default:
return
}
// Close existing transport connections before creating new ones.
// This forces in-flight requests on stale connections (e.g. after a
// firewall state table flush) to fail fast instead of waiting for
// the full context deadline timeout.
uc.closeTransports()
ips := uc.bootstrapIPs
switch uc.IPStack {
case IpStackBoth, "":
uc.transport = uc.newDOHTransport(uc.bootstrapIPs)
case IpStackV4:
uc.transport = uc.newDOHTransport(uc.bootstrapIPs4)
ips = uc.bootstrapIPs4
case IpStackV6:
uc.transport = uc.newDOHTransport(uc.bootstrapIPs6)
case IpStackSplit:
ips = uc.bootstrapIPs6
}
uc.transport = uc.newDOHTransport(ips)
uc.http3RoundTripper = uc.newDOH3Transport(ips)
uc.doqConnPool = uc.newDOQConnPool(ips)
uc.dotClientPool = uc.newDOTClientPool(ips)
if uc.IPStack == IpStackSplit {
uc.transport4 = uc.newDOHTransport(uc.bootstrapIPs4)
uc.http3RoundTripper4 = uc.newDOH3Transport(uc.bootstrapIPs4)
uc.doqConnPool4 = uc.newDOQConnPool(uc.bootstrapIPs4)
uc.dotClientPool4 = uc.newDOTClientPool(uc.bootstrapIPs4)
if HasIPv6() {
uc.transport6 = uc.newDOHTransport(uc.bootstrapIPs6)
uc.http3RoundTripper6 = uc.newDOH3Transport(uc.bootstrapIPs6)
uc.doqConnPool6 = uc.newDOQConnPool(uc.bootstrapIPs6)
uc.dotClientPool6 = uc.newDOTClientPool(uc.bootstrapIPs6)
} else {
uc.transport6 = uc.transport4
uc.http3RoundTripper6 = uc.http3RoundTripper4
uc.doqConnPool6 = uc.doqConnPool4
uc.dotClientPool6 = uc.dotClientPool4
}
uc.transport = uc.newDOHTransport(uc.bootstrapIPs)
}
}
func (uc *UpstreamConfig) ensureSetupTransport() {
uc.transportOnce.Do(func() {
uc.SetupTransport()
})
if uc.rebootstrap.CompareAndSwap(rebootstrapStarted, rebootstrapInProgress) {
uc.SetupTransport()
uc.rebootstrap.Store(rebootstrapNotStarted)
}
}
func (uc *UpstreamConfig) newDOHTransport(addrs []string) *http.Transport {
if uc.Type != ResolverTypeDOH {
return nil
}
transport := http.DefaultTransport.(*http.Transport).Clone()
transport.MaxIdleConnsPerHost = 100
transport.TLSClientConfig = &tls.Config{
RootCAs: uc.certPool,
ClientSessionCache: tls.NewLRUClientSessionCache(0),
MinVersion: tls.VersionTLS12,
}
// Prevent bad tcp connection hanging the requests for too long.
@@ -595,7 +700,7 @@ func (uc *UpstreamConfig) ErrorPing() error {
func (uc *UpstreamConfig) ping() error {
switch uc.Type {
case ResolverTypeDOH, ResolverTypeDOH3:
case ResolverTypeDOH, ResolverTypeDOH3, ResolverTypeDOQ:
default:
return nil
}
@@ -629,6 +734,14 @@ func (uc *UpstreamConfig) ping() error {
if err := ping(uc.doh3Transport(typ)); err != nil {
return err
}
case ResolverTypeDOQ:
// For DoQ, we just ensure transport is set up by calling doqTransport
// DoQ doesn't use HTTP, so we can't ping it the same way
_ = uc.doqTransport(typ)
case ResolverTypeDOT:
// For DoT, we just ensure transport is set up by calling dotTransport
// DoT doesn't use HTTP, so we can't ping it the same way
_ = uc.dotTransport(typ)
}
}
@@ -662,46 +775,8 @@ func (uc *UpstreamConfig) isNextDNS() bool {
}
func (uc *UpstreamConfig) dohTransport(dnsType uint16) http.RoundTripper {
uc.transportOnce.Do(func() {
uc.SetupTransport()
})
if uc.rebootstrap.CompareAndSwap(true, false) {
uc.SetupTransport()
}
switch uc.IPStack {
case IpStackBoth, IpStackV4, IpStackV6:
return uc.transport
case IpStackSplit:
switch dnsType {
case dns.TypeA:
return uc.transport4
default:
return uc.transport6
}
}
return uc.transport
}
func (uc *UpstreamConfig) bootstrapIPForDNSType(dnsType uint16) string {
switch uc.IPStack {
case IpStackBoth:
return pick(uc.bootstrapIPs)
case IpStackV4:
return pick(uc.bootstrapIPs4)
case IpStackV6:
return pick(uc.bootstrapIPs6)
case IpStackSplit:
switch dnsType {
case dns.TypeA:
return pick(uc.bootstrapIPs4)
default:
if HasIPv6() {
return pick(uc.bootstrapIPs6)
}
return pick(uc.bootstrapIPs4)
}
}
return pick(uc.bootstrapIPs)
uc.ensureSetupTransport()
return transportByIpStack(uc.IPStack, dnsType, uc.transport, uc.transport4, uc.transport6)
}
func (uc *UpstreamConfig) netForDNSType(dnsType uint16) (string, string) {
@@ -946,10 +1021,6 @@ func ResolverTypeFromEndpoint(endpoint string) string {
return ResolverTypeDOT
}
func pick(s []string) string {
return s[rand.Intn(len(s))]
}
// upstreamUID generates an unique identifier for an upstream.
func upstreamUID() string {
b := make([]byte, 4)
@@ -985,3 +1056,18 @@ func bootstrapIPsFromControlDDomain(domain string) []string {
}
return nil
}
func transportByIpStack[T any](ipStack string, dnsType uint16, transport, transport4, transport6 T) T {
switch ipStack {
case IpStackBoth, IpStackV4, IpStackV6:
return transport
case IpStackSplit:
switch dnsType {
case dns.TypeA:
return transport4
default:
return transport6
}
}
return transport
}
+47
View File
@@ -2,6 +2,7 @@ package ctrld
import (
"net/url"
"sync"
"testing"
"github.com/stretchr/testify/assert"
@@ -505,6 +506,52 @@ func TestUpstreamConfig_IsDiscoverable(t *testing.T) {
}
}
func TestRebootstrapRace(t *testing.T) {
uc := &UpstreamConfig{
Name: "test-doh",
Type: ResolverTypeDOH,
Endpoint: "https://example.com/dns-query",
Domain: "example.com",
bootstrapIPs: []string{"1.1.1.1", "1.0.0.1"},
}
uc.SetupTransport()
if uc.transport == nil {
t.Fatal("initial transport should be set")
}
const goroutines = 100
uc.ReBootstrap()
started := make(chan struct{})
go func() {
close(started)
for {
switch uc.rebootstrap.Load() {
case rebootstrapStarted, rebootstrapInProgress:
uc.ReBootstrap()
default:
return
}
}
}()
<-started
var wg sync.WaitGroup
wg.Add(goroutines)
for range goroutines {
go func() {
defer wg.Done()
uc.ensureSetupTransport()
}()
}
wg.Wait()
}
func ptrBool(b bool) *bool {
return &b
}
+58 -45
View File
@@ -9,33 +9,16 @@ import (
"runtime"
"sync"
"github.com/miekg/dns"
"github.com/quic-go/quic-go"
"github.com/quic-go/quic-go/http3"
)
func (uc *UpstreamConfig) setupDOH3Transport() {
switch uc.IPStack {
case IpStackBoth, "":
uc.http3RoundTripper = uc.newDOH3Transport(uc.bootstrapIPs)
case IpStackV4:
uc.http3RoundTripper = uc.newDOH3Transport(uc.bootstrapIPs4)
case IpStackV6:
uc.http3RoundTripper = uc.newDOH3Transport(uc.bootstrapIPs6)
case IpStackSplit:
uc.http3RoundTripper4 = uc.newDOH3Transport(uc.bootstrapIPs4)
if HasIPv6() {
uc.http3RoundTripper6 = uc.newDOH3Transport(uc.bootstrapIPs6)
} else {
uc.http3RoundTripper6 = uc.http3RoundTripper4
}
uc.http3RoundTripper = uc.newDOH3Transport(uc.bootstrapIPs)
}
}
func (uc *UpstreamConfig) newDOH3Transport(addrs []string) http.RoundTripper {
if uc.Type != ResolverTypeDOH3 {
return nil
}
rt := &http3.Transport{}
rt.TLSClientConfig = &tls.Config{RootCAs: uc.certPool}
rt.TLSClientConfig = &tls.Config{RootCAs: uc.certPool, MinVersion: tls.VersionTLS12}
rt.Dial = func(ctx context.Context, addr string, tlsCfg *tls.Config, cfg *quic.Config) (*quic.Conn, error) {
_, port, _ := net.SplitHostPort(addr)
// if we have a bootstrap ip set, use it to avoid DNS lookup
@@ -71,24 +54,18 @@ func (uc *UpstreamConfig) newDOH3Transport(addrs []string) http.RoundTripper {
}
func (uc *UpstreamConfig) doh3Transport(dnsType uint16) http.RoundTripper {
uc.transportOnce.Do(func() {
uc.SetupTransport()
})
if uc.rebootstrap.CompareAndSwap(true, false) {
uc.SetupTransport()
}
switch uc.IPStack {
case IpStackBoth, IpStackV4, IpStackV6:
return uc.http3RoundTripper
case IpStackSplit:
switch dnsType {
case dns.TypeA:
return uc.http3RoundTripper4
default:
return uc.http3RoundTripper6
}
}
return uc.http3RoundTripper
uc.ensureSetupTransport()
return transportByIpStack(uc.IPStack, dnsType, uc.http3RoundTripper, uc.http3RoundTripper4, uc.http3RoundTripper6)
}
func (uc *UpstreamConfig) doqTransport(dnsType uint16) *doqConnPool {
uc.ensureSetupTransport()
return transportByIpStack(uc.IPStack, dnsType, uc.doqConnPool, uc.doqConnPool4, uc.doqConnPool6)
}
func (uc *UpstreamConfig) dotTransport(dnsType uint16) *dotConnPool {
uc.ensureSetupTransport()
return transportByIpStack(uc.IPStack, dnsType, uc.dotClientPool, uc.dotClientPool4, uc.dotClientPool6)
}
// Putting the code for quic parallel dialer here:
@@ -100,7 +77,17 @@ type parallelDialerResult struct {
err error
}
type quicParallelDialer struct{}
// quicParallelDialer races DialEarly across a list of remote addresses and
// returns the first successful connection. When transport is non-nil, all
// dials share that transport's UDP socket, which removes both the per-dial
// socket allocation and the winner-path socket leak that an owner-of-the-conn
// receiver cannot clean up. When transport is nil, the dialer falls back to a
// fresh UDP socket per attempt (compat path used where no shared transport is
// available yet); the loser paths close their sockets, and the winner path's
// socket is owned by quic.DialEarly's internal transport.
type quicParallelDialer struct {
transport *quic.Transport
}
// Dial performs parallel dialing to the given address list.
func (d *quicParallelDialer) Dial(ctx context.Context, addrs []string, tlsCfg *tls.Config, cfg *quic.Config) (*quic.Conn, error) {
@@ -128,12 +115,24 @@ func (d *quicParallelDialer) Dial(ctx context.Context, addrs []string, tlsCfg *t
ch <- &parallelDialerResult{conn: nil, err: err}
return
}
udpConn, err := net.ListenUDP("udp", nil)
if err != nil {
ch <- &parallelDialerResult{conn: nil, err: err}
return
var (
conn *quic.Conn
udpConn *net.UDPConn
)
if d.transport != nil {
conn, err = d.transport.DialEarly(ctx, remoteAddr, tlsCfg, cfg)
} else {
udpConn, err = net.ListenUDP("udp", nil)
if err != nil {
ch <- &parallelDialerResult{conn: nil, err: err}
return
}
conn, err = quic.DialEarly(ctx, udpConn, remoteAddr, tlsCfg, cfg)
if err != nil {
udpConn.Close()
udpConn = nil
}
}
conn, err := quic.DialEarly(ctx, udpConn, remoteAddr, tlsCfg, cfg)
select {
case ch <- &parallelDialerResult{conn: conn, err: err}:
case <-done:
@@ -158,3 +157,17 @@ func (d *quicParallelDialer) Dial(ctx context.Context, addrs []string, tlsCfg *t
return nil, errors.Join(errs...)
}
func (uc *UpstreamConfig) newDOQConnPool(addrs []string) *doqConnPool {
if uc.Type != ResolverTypeDOQ {
return nil
}
return newDOQConnPool(uc, addrs)
}
func (uc *UpstreamConfig) newDOTClientPool(addrs []string) *dotConnPool {
if uc.Type != ResolverTypeDOT {
return nil
}
return newDOTClientPool(uc, addrs)
}
+135
View File
@@ -0,0 +1,135 @@
//go:build darwin
package ctrld
import (
"context"
"os/exec"
"strconv"
"strings"
)
// DiscoverMainUser attempts to find the primary user on macOS systems.
// This is designed to work reliably under RMM deployments where traditional
// environment variables and session detection may not be available.
//
// Priority chain (deterministic, lowest UID wins among candidates):
// 1. Console user from stat -f %Su /dev/console
// 2. Active console session user via scutil
// 3. First user with UID >= 501 from dscl (standard macOS user range)
func DiscoverMainUser(ctx context.Context) string {
logger := ProxyLogger.Load().Debug()
// Method 1: Check console owner via stat
logger.Msg("attempting to discover user via console stat")
if user := getConsoleUser(ctx); user != "" && user != "root" {
logger.Str("method", "stat").Str("user", user).Msg("found user via console stat")
return user
}
// Method 2: Check active console session via scutil
logger.Msg("attempting to discover user via scutil ConsoleUser")
if user := getScutilConsoleUser(ctx); user != "" && user != "root" {
logger.Str("method", "scutil").Str("user", user).Msg("found user via scutil ConsoleUser")
return user
}
// Method 3: Find lowest UID >= 501 from directory services
logger.Msg("attempting to discover user via dscl directory scan")
if user := getLowestRegularUser(ctx); user != "" {
logger.Str("method", "dscl").Str("user", user).Msg("found user via dscl scan")
return user
}
logger.Msg("all user discovery methods failed")
return "unknown"
}
// getConsoleUser uses stat to find the owner of /dev/console
func getConsoleUser(ctx context.Context) string {
cmd := exec.CommandContext(ctx, "stat", "-f", "%Su", "/dev/console")
out, err := cmd.Output()
if err != nil {
ProxyLogger.Load().Debug().Err(err).Msg("failed to stat /dev/console")
return ""
}
return strings.TrimSpace(string(out))
}
// getScutilConsoleUser uses scutil to get the current console user
func getScutilConsoleUser(ctx context.Context) string {
cmd := exec.CommandContext(ctx, "scutil", "-r", "ConsoleUser")
out, err := cmd.Output()
if err != nil {
ProxyLogger.Load().Debug().Err(err).Msg("failed to get ConsoleUser via scutil")
return ""
}
lines := strings.Split(string(out), "\n")
for _, line := range lines {
if strings.Contains(line, "Name :") {
parts := strings.Fields(line)
if len(parts) >= 3 {
return strings.TrimSpace(parts[2])
}
}
}
return ""
}
// getLowestRegularUser finds the user with the lowest UID >= 501
func getLowestRegularUser(ctx context.Context) string {
// Get list of all users with UID >= 501
cmd := exec.CommandContext(ctx, "dscl", ".", "list", "/Users", "UniqueID")
out, err := cmd.Output()
if err != nil {
ProxyLogger.Load().Debug().Err(err).Msg("failed to list users via dscl")
return ""
}
var candidates []struct {
name string
uid int
}
lines := strings.Split(string(out), "\n")
for _, line := range lines {
fields := strings.Fields(line)
if len(fields) != 2 {
continue
}
username := fields[0]
uidStr := fields[1]
uid, err := strconv.Atoi(uidStr)
if err != nil {
continue
}
// Only consider regular users (UID >= 501 on macOS)
if uid >= 501 {
candidates = append(candidates, struct {
name string
uid int
}{username, uid})
}
}
if len(candidates) == 0 {
return ""
}
// Find the candidate with the lowest UID (deterministic choice)
lowestUID := candidates[0].uid
result := candidates[0].name
for _, candidate := range candidates[1:] {
if candidate.uid < lowestUID {
lowestUID = candidate.uid
result = candidate.name
}
}
return result
}
+238
View File
@@ -0,0 +1,238 @@
//go:build linux
package ctrld
import (
"bufio"
"context"
"os"
"os/exec"
"strconv"
"strings"
)
// DiscoverMainUser attempts to find the primary user on Linux systems.
// This is designed to work reliably under RMM deployments where traditional
// environment variables and session detection may not be available.
//
// Priority chain (deterministic, lowest UID wins among candidates):
// 1. Active users from loginctl list-users
// 2. Parse /etc/passwd for users with UID >= 1000, prefer admin group members
// 3. Fallback to lowest UID >= 1000 from /etc/passwd
func DiscoverMainUser(ctx context.Context) string {
logger := ProxyLogger.Load().Debug()
// Method 1: Check active users via loginctl
logger.Msg("attempting to discover user via loginctl")
if user := getLoginctlUser(ctx); user != "" {
logger.Str("method", "loginctl").Str("user", user).Msg("found user via loginctl")
return user
}
// Method 2: Parse /etc/passwd and find admin users first
logger.Msg("attempting to discover user via /etc/passwd with admin preference")
if user := getPasswdUserWithAdminPreference(ctx); user != "" {
logger.Str("method", "passwd+admin").Str("user", user).Msg("found admin user via /etc/passwd")
return user
}
// Method 3: Fallback to lowest UID >= 1000 from /etc/passwd
logger.Msg("attempting to discover user via /etc/passwd lowest UID")
if user := getLowestPasswdUser(ctx); user != "" {
logger.Str("method", "passwd").Str("user", user).Msg("found user via /etc/passwd")
return user
}
logger.Msg("all user discovery methods failed")
return "unknown"
}
// getLoginctlUser uses loginctl to find active users
func getLoginctlUser(ctx context.Context) string {
cmd := exec.CommandContext(ctx, "loginctl", "list-users", "--no-legend")
out, err := cmd.Output()
if err != nil {
ProxyLogger.Load().Debug().Err(err).Msg("failed to run loginctl list-users")
return ""
}
var candidates []struct {
name string
uid int
}
lines := strings.Split(string(out), "\n")
for _, line := range lines {
fields := strings.Fields(line)
if len(fields) < 2 {
continue
}
uidStr := fields[0]
username := fields[1]
uid, err := strconv.Atoi(uidStr)
if err != nil {
continue
}
// Only consider regular users (UID >= 1000 on Linux)
if uid >= 1000 {
candidates = append(candidates, struct {
name string
uid int
}{username, uid})
}
}
if len(candidates) == 0 {
return ""
}
// Return user with lowest UID (deterministic choice)
lowestUID := candidates[0].uid
result := candidates[0].name
for _, candidate := range candidates[1:] {
if candidate.uid < lowestUID {
lowestUID = candidate.uid
result = candidate.name
}
}
return result
}
// getPasswdUserWithAdminPreference parses /etc/passwd and prefers admin group members
func getPasswdUserWithAdminPreference(ctx context.Context) string {
users := parsePasswdFile()
if len(users) == 0 {
return ""
}
var adminUsers []struct {
name string
uid int
}
var regularUsers []struct {
name string
uid int
}
// Separate admin and regular users
for _, user := range users {
if isUserInAdminGroups(ctx, user.name) {
adminUsers = append(adminUsers, user)
} else {
regularUsers = append(regularUsers, user)
}
}
// Prefer admin users, then regular users
candidates := adminUsers
if len(candidates) == 0 {
candidates = regularUsers
}
if len(candidates) == 0 {
return ""
}
// Return user with lowest UID (deterministic choice)
lowestUID := candidates[0].uid
result := candidates[0].name
for _, candidate := range candidates[1:] {
if candidate.uid < lowestUID {
lowestUID = candidate.uid
result = candidate.name
}
}
return result
}
// getLowestPasswdUser returns the user with lowest UID >= 1000 from /etc/passwd
func getLowestPasswdUser(ctx context.Context) string {
users := parsePasswdFile()
if len(users) == 0 {
return ""
}
// Return user with lowest UID (deterministic choice)
lowestUID := users[0].uid
result := users[0].name
for _, user := range users[1:] {
if user.uid < lowestUID {
lowestUID = user.uid
result = user.name
}
}
return result
}
// parsePasswdFile parses /etc/passwd and returns users with UID >= 1000
func parsePasswdFile() []struct {
name string
uid int
} {
file, err := os.Open("/etc/passwd")
if err != nil {
ProxyLogger.Load().Debug().Err(err).Msg("failed to open /etc/passwd")
return nil
}
defer file.Close()
var users []struct {
name string
uid int
}
scanner := bufio.NewScanner(file)
for scanner.Scan() {
line := scanner.Text()
fields := strings.Split(line, ":")
if len(fields) < 3 {
continue
}
username := fields[0]
uidStr := fields[2]
uid, err := strconv.Atoi(uidStr)
if err != nil {
continue
}
// Only consider regular users (UID >= 1000 on Linux)
if uid >= 1000 {
users = append(users, struct {
name string
uid int
}{username, uid})
}
}
return users
}
// isUserInAdminGroups checks if a user is in common admin groups
func isUserInAdminGroups(ctx context.Context, username string) bool {
adminGroups := []string{"sudo", "wheel", "admin"}
for _, group := range adminGroups {
cmd := exec.CommandContext(ctx, "groups", username)
out, err := cmd.Output()
if err != nil {
continue
}
if strings.Contains(string(out), group) {
return true
}
}
return false
}
+13
View File
@@ -0,0 +1,13 @@
//go:build !windows && !linux && !darwin
package ctrld
import "context"
// DiscoverMainUser returns "unknown" for unsupported platforms.
// This is a stub implementation for platforms where username detection
// is not yet implemented.
func DiscoverMainUser(ctx context.Context) string {
ProxyLogger.Load().Debug().Msg("username discovery not implemented for this platform")
return "unknown"
}
+292
View File
@@ -0,0 +1,292 @@
//go:build windows
package ctrld
import (
"context"
"strconv"
"strings"
"syscall"
"unsafe"
"golang.org/x/sys/windows"
"golang.org/x/sys/windows/registry"
)
var (
wtsapi32 = windows.NewLazySystemDLL("wtsapi32.dll")
procWTSGetActiveConsoleSessionId = wtsapi32.NewProc("WTSGetActiveConsoleSessionId")
procWTSQuerySessionInformation = wtsapi32.NewProc("WTSQuerySessionInformationW")
procWTSFreeMemory = wtsapi32.NewProc("WTSFreeMemory")
)
const (
WTSUserName = 5
)
// DiscoverMainUser attempts to find the primary user on Windows systems.
// This is designed to work reliably under RMM deployments where traditional
// environment variables and session detection may not be available.
//
// Priority chain (deterministic, lowest RID wins among candidates):
// 1. Active console session user via WTSGetActiveConsoleSessionId
// 2. Registry ProfileList scan for Administrators group members
// 3. Fallback to lowest RID from ProfileList
func DiscoverMainUser(ctx context.Context) string {
logger := ProxyLogger.Load().Debug()
// Method 1: Check active console session
logger.Msg("attempting to discover user via active console session")
if user := getActiveConsoleUser(ctx); user != "" {
logger.Str("method", "console").Str("user", user).Msg("found user via active console session")
return user
}
// Method 2: Scan registry for admin users
logger.Msg("attempting to discover user via registry with admin preference")
if user := getRegistryUserWithAdminPreference(ctx); user != "" {
logger.Str("method", "registry+admin").Str("user", user).Msg("found admin user via registry")
return user
}
// Method 3: Fallback to lowest RID from registry
logger.Msg("attempting to discover user via registry lowest RID")
if user := getLowestRegistryUser(ctx); user != "" {
logger.Str("method", "registry").Str("user", user).Msg("found user via registry")
return user
}
logger.Msg("all user discovery methods failed")
return "unknown"
}
// getActiveConsoleUser gets the username of the active console session
func getActiveConsoleUser(ctx context.Context) string {
// Guard against missing WTS procedures (e.g., Windows Server Core).
if err := procWTSGetActiveConsoleSessionId.Find(); err != nil {
ProxyLogger.Load().Debug().Err(err).Msg("WTSGetActiveConsoleSessionId not available, skipping console session check")
return ""
}
sessionId, _, _ := procWTSGetActiveConsoleSessionId.Call()
if sessionId == 0xFFFFFFFF { // Invalid session
ProxyLogger.Load().Debug().Msg("no active console session found")
return ""
}
var buffer uintptr
var bytesReturned uint32
if err := procWTSQuerySessionInformation.Find(); err != nil {
ProxyLogger.Load().Debug().Err(err).Msg("WTSQuerySessionInformationW not available")
return ""
}
ret, _, _ := procWTSQuerySessionInformation.Call(
0, // WTS_CURRENT_SERVER_HANDLE
sessionId,
uintptr(WTSUserName),
uintptr(unsafe.Pointer(&buffer)),
uintptr(unsafe.Pointer(&bytesReturned)),
)
if ret == 0 {
ProxyLogger.Load().Debug().Msg("failed to query session information")
return ""
}
defer procWTSFreeMemory.Call(buffer)
// Convert buffer to string
username := windows.UTF16PtrToString((*uint16)(unsafe.Pointer(buffer)))
if username == "" {
return ""
}
return username
}
// getRegistryUserWithAdminPreference scans registry profiles and prefers admin users
func getRegistryUserWithAdminPreference(ctx context.Context) string {
profiles := getRegistryProfiles()
if len(profiles) == 0 {
return ""
}
var adminProfiles []registryProfile
var regularProfiles []registryProfile
// Separate admin and regular users
for _, profile := range profiles {
if isUserInAdministratorsGroup(profile.username) {
adminProfiles = append(adminProfiles, profile)
} else {
regularProfiles = append(regularProfiles, profile)
}
}
// Prefer admin users, then regular users
candidates := adminProfiles
if len(candidates) == 0 {
candidates = regularProfiles
}
if len(candidates) == 0 {
return ""
}
// Return user with lowest RID (deterministic choice)
lowestRID := candidates[0].rid
result := candidates[0].username
for _, candidate := range candidates[1:] {
if candidate.rid < lowestRID {
lowestRID = candidate.rid
result = candidate.username
}
}
return result
}
// getLowestRegistryUser returns the user with lowest RID from registry
func getLowestRegistryUser(ctx context.Context) string {
profiles := getRegistryProfiles()
if len(profiles) == 0 {
return ""
}
// Return user with lowest RID (deterministic choice)
lowestRID := profiles[0].rid
result := profiles[0].username
for _, profile := range profiles[1:] {
if profile.rid < lowestRID {
lowestRID = profile.rid
result = profile.username
}
}
return result
}
type registryProfile struct {
username string
rid uint32
sid string
}
// getRegistryProfiles scans the registry ProfileList for user profiles
func getRegistryProfiles() []registryProfile {
key, err := registry.OpenKey(registry.LOCAL_MACHINE, `SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList`, registry.ENUMERATE_SUB_KEYS)
if err != nil {
ProxyLogger.Load().Debug().Err(err).Msg("failed to open ProfileList registry key")
return nil
}
defer key.Close()
subkeys, err := key.ReadSubKeyNames(-1)
if err != nil {
ProxyLogger.Load().Debug().Err(err).Msg("failed to read ProfileList subkeys")
return nil
}
var profiles []registryProfile
for _, subkey := range subkeys {
// Only process SIDs that start with S-1-5-21 (domain/local user accounts)
if !strings.HasPrefix(subkey, "S-1-5-21-") {
continue
}
profileKey, err := registry.OpenKey(key, subkey, registry.QUERY_VALUE)
if err != nil {
continue
}
profileImagePath, _, err := profileKey.GetStringValue("ProfileImagePath")
profileKey.Close()
if err != nil {
continue
}
// Extract username from profile path (e.g., C:\Users\username)
pathParts := strings.Split(profileImagePath, `\`)
if len(pathParts) == 0 {
continue
}
username := pathParts[len(pathParts)-1]
// Extract RID from SID (last component after final hyphen)
sidParts := strings.Split(subkey, "-")
if len(sidParts) == 0 {
continue
}
ridStr := sidParts[len(sidParts)-1]
rid, err := strconv.ParseUint(ridStr, 10, 32)
if err != nil {
continue
}
// Only consider regular users (RID >= 1000, excludes built-in accounts).
// rid == 500 is the default Administrator account (DOMAIN_USER_RID_ADMIN).
// See: https://learn.microsoft.com/en-us/windows/win32/secauthz/well-known-sids
if rid == 500 || rid >= 1000 {
profiles = append(profiles, registryProfile{
username: username,
rid: uint32(rid),
sid: subkey,
})
}
}
return profiles
}
// isUserInAdministratorsGroup checks if a user is in the Administrators group
func isUserInAdministratorsGroup(username string) bool {
// Open the user account
usernamePtr, err := syscall.UTF16PtrFromString(username)
if err != nil {
return false
}
var userSID *windows.SID
var domain *uint16
var userSIDSize, domainSize uint32
var use uint32
// First call to get buffer sizes
err = windows.LookupAccountName(nil, usernamePtr, userSID, &userSIDSize, domain, &domainSize, &use)
if err != nil && err != windows.ERROR_INSUFFICIENT_BUFFER {
return false
}
// Allocate buffers and make actual call
userSID = (*windows.SID)(unsafe.Pointer(&make([]byte, userSIDSize)[0]))
domain = (*uint16)(unsafe.Pointer(&make([]uint16, domainSize)[0]))
err = windows.LookupAccountName(nil, usernamePtr, userSID, &userSIDSize, domain, &domainSize, &use)
if err != nil {
return false
}
// Check if user is member of Administrators group (S-1-5-32-544)
adminSID, err := windows.CreateWellKnownSid(windows.WinBuiltinAdministratorsSid)
if err != nil {
return false
}
// Open user token (this is a simplified check)
var token windows.Token
err = windows.OpenProcessToken(windows.CurrentProcess(), windows.TOKEN_QUERY, &token)
if err != nil {
return false
}
defer token.Close()
// Check group membership
member, err := token.IsMember(adminSID)
if err != nil {
return false
}
return member
}
+6
View File
@@ -14,6 +14,12 @@ func SetCacheReply(answer, msg *dns.Msg, code int) {
// See https://datatracker.ietf.org/doc/html/rfc7873#section-4
sCookie.Cookie = cCookie.Cookie[:16] + sCookie.Cookie[16:]
}
// NOTE: the answer's EDNS Client Subnet (ECS) is intentionally left as the
// upstream returned it. Correctness across clients is guaranteed by
// partitioning the cache and singleflight keys by ECS (see
// dnscache.CanonicalECS), so a cache hit only ever serves an answer that was
// resolved for the requester's own subnet. Rewriting the ECS option here
// without re-scoping the Answer records would violate RFC 7871 §7.3.
}
// getEdns0Cookie returns Edns0 cookie from *dns.OPT if present.
+57
View File
@@ -0,0 +1,57 @@
package ctrld
import (
"net"
"testing"
"github.com/miekg/dns"
)
// Test_SetCacheReply_DoesNotRewriteECS documents the post-#564 contract: cross-client
// correctness is guaranteed by partitioning the cache/singleflight keys by ECS
// (dnscache.CanonicalECS), NOT by rewriting the cached answer's ECS option. Rewriting the
// ECS metadata while leaving the Answer records scoped to another subnet would violate
// RFC 7871 §7.3 and make forwarders accept a wrong-subnet answer. SetCacheReply must
// therefore leave the answer's ECS untouched.
func Test_SetCacheReply_DoesNotRewriteECS(t *testing.T) {
answer := new(dns.Msg)
answer.SetQuestion(dns.Fqdn("controld.com"), dns.TypeA)
answer.SetEdns0(4096, false)
cachedSubnet := &dns.EDNS0_SUBNET{
Code: dns.EDNS0SUBNET,
Family: 2,
SourceNetmask: 64,
SourceScope: 64,
Address: net.ParseIP("2001:db8:1::"),
}
answer.IsEdns0().Option = append(answer.IsEdns0().Option, cachedSubnet)
req := new(dns.Msg)
req.SetQuestion(dns.Fqdn("controld.com"), dns.TypeA)
req.SetEdns0(4096, true)
req.IsEdns0().Option = append(req.IsEdns0().Option, &dns.EDNS0_SUBNET{
Code: dns.EDNS0SUBNET,
Family: 2,
SourceNetmask: 64,
Address: net.ParseIP("2001:db8:2::"),
})
SetCacheReply(answer, req, dns.RcodeSuccess)
var got *dns.EDNS0_SUBNET
for _, o := range answer.IsEdns0().Option {
if e, ok := o.(*dns.EDNS0_SUBNET); ok {
got = e
break
}
}
if got == nil {
t.Fatal("SetCacheReply dropped the answer's ECS option")
}
if want := net.ParseIP("2001:db8:1::"); !got.Address.Equal(want) {
t.Fatalf("SetCacheReply rewrote the answer ECS to the requester's subnet: got %v, want %v (unchanged)", got.Address, want)
}
if got.SourceScope != 64 {
t.Fatalf("SetCacheReply altered the answer ECS scope: got %d, want 64 (unchanged)", got.SourceScope)
}
}
+4 -3
View File
@@ -1,4 +1,4 @@
# Using Debian bullseye for building regular image.
# Using Debian bookworm for building regular image.
# Using scratch image for minimal image size.
# The final image has:
#
@@ -8,11 +8,12 @@
# - Non-cgo ctrld binary.
#
# CI_COMMIT_TAG is used to set the version of ctrld binary.
FROM golang:1.20-bullseye as base
FROM golang:1.25-bookworm AS base
WORKDIR /app
RUN apt-get update && apt-get install -y upx-ucl
RUN echo "deb http://deb.debian.org/debian bookworm-backports main" | tee /etc/apt/sources.list.d/backports.list
RUN apt update && apt install -t bookworm-backports upx-ucl
COPY . .
+4 -3
View File
@@ -1,4 +1,4 @@
# Using Debian bullseye for building regular image.
# Using Debian bookworm for building regular image.
# Using scratch image for minimal image size.
# The final image has:
#
@@ -8,11 +8,12 @@
# - Non-cgo ctrld binary.
#
# CI_COMMIT_TAG is used to set the version of ctrld binary.
FROM golang:bullseye as base
FROM golang:1.25-bookworm AS base
WORKDIR /app
RUN apt-get update && apt-get install -y upx-ucl
RUN echo "deb http://deb.debian.org/debian bookworm-backports main" | tee /etc/apt/sources.list.d/backports.list
RUN apt update && apt install -t bookworm-backports upx-ucl
COPY . .
+16
View File
@@ -295,6 +295,22 @@ If a remote upstream fails to resolve a query or is unreachable, `ctrld` will fo
- Required: no
- Default: true on Windows, MacOS and non-router Linux.
### nrpt_recovery_max_attempts
Windows DNS intercept mode uses NRPT health probes and recovery when Windows stops routing queries to the local `ctrld` listener. This limits how many consecutive recovery flows can run before `ctrld` enters a cooldown and stops making policy/Dnscache changes.
Set to `0` to disable this circuit breaker and keep retrying indefinitely.
- Type: integer
- Required: no
- Default: 0 (unlimited, current behavior)
### nrpt_recovery_cooldown
Cooldown duration after `nrpt_recovery_max_attempts` consecutive Windows NRPT recovery flows. During cooldown, `ctrld` logs the suppressed recovery and avoids additional `RefreshPolicyEx`, Dnscache `paramchange`, and DNS cache flush calls.
- Type: time duration string
- Required: no
- Default: 30m
## Upstream
The `[upstream]` section specifies the DNS upstream servers that `ctrld` will forward DNS requests to.
+599
View File
@@ -0,0 +1,599 @@
# DNS Intercept Mode
## Overview
DNS intercept mode is an alternative approach to DNS management that uses OS-level packet interception instead of modifying network interface DNS settings. This eliminates race conditions with VPN software, endpoint security tools, and other programs that also manage DNS.
## The Problem
By default, ctrld sets DNS to `127.0.0.1` on network interfaces so all queries go through ctrld's local listener. However, VPN software (F5 BIG-IP, Cisco AnyConnect, Palo Alto GlobalProtect, etc.) also overwrites interface DNS settings, creating conflicts:
1. **DNS Setting War**: ctrld sets DNS to `127.0.0.1`, VPN overwrites to its DNS servers, ctrld's watchdog detects the change and restores `127.0.0.1`, VPN overwrites again — infinitely.
2. **Bypass Window**: During the watchdog polling interval (up to 20 seconds), DNS queries may go to the VPN's DNS servers, bypassing ctrld's filtering profiles (malware blocking, content filtering, etc.).
3. **Resolution Failures**: During the brief moments when DNS is being rewritten, queries may fail entirely, causing intermittent connectivity loss.
## The Solution
DNS intercept mode works at a lower level than interface settings:
- **Windows**: Uses NRPT (Name Resolution Policy Table) to route all DNS queries to `127.0.0.1` (ctrld's listener) via the Windows DNS Client service. In `hard` mode, additionally uses WFP (Windows Filtering Platform) to block all outbound DNS (port 53) except to localhost and private ranges, preventing any bypass. VPN software can set interface DNS freely — NRPT's most-specific-match ensures VPN-specific domains still resolve correctly while ctrld handles everything else.
- **macOS**: Uses pf (packet filter) to redirect all outbound DNS (port 53) traffic to ctrld's listener at `127.0.0.1:53`. Any DNS query, regardless of which DNS server the OS thinks it's using, gets transparently redirected to ctrld.
## Usage
```bash
# Start ctrld with DNS intercept mode (auto-detects VPN search domains)
ctrld start --intercept-mode dns --cd <resolver-uid>
# Hard intercept: all DNS through ctrld, no VPN split routing
ctrld start --intercept-mode hard --cd <resolver-uid>
# Or with a config file
ctrld start --intercept-mode dns -c /path/to/ctrld.toml
# Run in foreground (debug)
ctrld run --intercept-mode dns --cd <resolver-uid>
ctrld run --intercept-mode hard --cd <resolver-uid>
```
### Intercept Modes
| Flag | DNS Interception | VPN Split Routing | Captive Portal Recovery |
|------|-----------------|-------------------|------------------------|
| `--intercept-mode dns` | ✅ WFP/pf | ✅ Auto-detect & forward | ✅ Active |
| `--intercept-mode hard` | ✅ WFP/pf | ❌ All through ctrld | ✅ Active |
**`--intercept-mode dns`** (recommended): Intercepts all DNS via WFP/pf, but automatically discovers search domains from VPN and virtual network adapters (Tailscale, F5, Cisco AnyConnect, etc.) and forwards matching queries to the DNS server on that interface. This allows VPN internal resources (e.g., `*.corp.local`) to resolve correctly while ctrld handles everything else.
**`--intercept-mode hard`**: Same OS-level interception, but does NOT forward any queries to VPN DNS servers. Every DNS query goes through ctrld's configured upstreams. Use this when you want total DNS control and don't need VPN internal domain resolution. Captive portal recovery still works — network authentication pages are handled automatically.
## How It Works
### Windows (NRPT + WFP)
Windows DNS intercept uses a two-tier architecture with mode-dependent enforcement:
- **`dns` mode**: NRPT + loopback WFP protect — graceful DNS routing through the Windows DNS Client service, with proactive WFP permit filters that protect the NRPT → localhost path from third-party DNS block filters (e.g., OpenVPN's `block-outside-dns`).
- **`hard` mode**: NRPT + WFP — same NRPT routing, plus WFP kernel-level block filters that prevent any outbound DNS bypass. Equivalent enforcement to macOS pf.
#### Why This Design?
WFP can only **block** or **permit** connections — it **cannot redirect** them (redirection requires kernel-mode callout drivers). Without NRPT, WFP blocks outbound DNS but doesn't tell applications where to send queries instead — they see DNS failures. NRPT provides the "positive routing" while WFP provides enforcement.
Separating them into modes means most users get `dns` mode (safe, can never break DNS) while high-security deployments use `hard` mode (full enforcement, same guarantees as macOS pf).
#### Startup Sequence (dns mode)
1. Checks for a non-ctrld GP child whose only namespace is `.` and whose only nameserver is ctrld's actual listener IP.
2. When that candidate exists, preserves adapter DNS, sends a DNS Client probe before any NRPT mutation, and re-reads the same GP child. A matching before/after rule plus a received probe enters **GP-managed mode**; ctrld does not write NRPT, call `RefreshPolicyEx`/`paramchange`, or flush DNS for policy activation.
3. Without a still-matching GP candidate, creates the normal ctrld-owned catch-all, signals DNS Client, and flushes stale cache entries.
4. **Activates loopback WFP protect** — adds 4 permit filters (IPv4/IPv6 × UDP/TCP) for DNS to localhost with `FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT`. These prevent third-party WFP block filters from blocking the NRPT → listener path (see [Loopback WFP Protect](#loopback-wfp-protect) below). Non-fatal if this fails.
5. Starts the 30-second ownership-aware NRPT health monitor.
6. Re-verifies an initially ineffective GP candidate synchronously after WFP setup; ctrld-owned NRPT uses the asynchronous probe-and-heal sequence.
#### Startup Sequence (hard mode)
1. Establishes NRPT routing using the same GP-managed adoption or ctrld-owned fallback sequence as `dns` mode.
2. Opens WFP engine with `RPC_C_AUTHN_DEFAULT` (0xFFFFFFFF).
3. Cleans up any stale sublayer from a previous unclean shutdown.
4. Creates sublayer with maximum weight (0xFFFF).
5. Adds **permit** filters (weight 10) for DNS to localhost (`127.0.0.1`/`::1` port 53).
6. Adds **permit** filters (weight 10) for DNS to RFC1918 + CGNAT subnets (10/8, 172.16/12, 192.168/16, 100.64/10).
7. Adds **block** filters (weight 1) for all other outbound DNS (port 53 UDP+TCP).
8. Starts the NRPT/WFP health monitor.
**Atomic guarantee:** NRPT routing must exist before WFP starts. If WFP setup fails, ctrld rolls back only a rule it owns. A GP-managed child is never deleted, rewritten, or replaced with interface DNS merely because ctrld's WFP setup failed.
On shutdown, ctrld stops its monitor and WFP session. It removes and signals only ctrld-owned NRPT state; a GP-managed catch-all remains untouched.
#### NRPT Details
The **Name Resolution Policy Table** is a Windows feature (originally for DirectAccess) that tells the DNS Client service to route queries matching specific namespace patterns to specific DNS servers. ctrld adds a catch-all rule:
| Registry Value | Type | Value | Purpose |
|---|---|---|---|
| `Name` | REG_MULTI_SZ | `.` | Namespace pattern (`.` = catch-all, matches everything) |
| `GenericDNSServers` | REG_SZ | `127.0.0.1` | DNS server to use for matching queries |
| `ConfigOptions` | REG_DWORD | `0x8` | Standard DNS resolution (no DirectAccess) |
| `Version` | REG_DWORD | `0x2` | NRPT rule version 2 |
**Registry path**: `HKLM\SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig\CtrldCatchAll`
**Group Policy refresh**: The DNS Client service only reads NRPT from registry during Group Policy processing cycles (default: every 90 minutes). ctrld calls `RefreshPolicyEx(bMachine=TRUE, dwOptions=RP_FORCE)` when activating or repairing rules it owns. While Group Policy remains the owner, ctrld does not run NRPT activation/heal signaling; the one transition that removes a ctrld fallback is signaled after the external rule has been proven.
#### GP-managed NRPT ownership
Enterprise deployments may install a computer-scoped GP child before starting ctrld with:
- exactly one namespace: `.`;
- exactly one `GenericDNSServers` value; and
- that nameserver equal to ctrld's actual loopback listener (`127.0.0.1` or the alternate loopback selected on an AD DNS server).
At service startup ctrld reads that candidate before the normal adapter reset, probes through Windows DNS Client while its listener is already bound, and re-reads the same child. When the rule remains present and the probe arrives, ctrld records **Group Policy** as the NRPT owner. Adapter DNS stays on the organization's resolvers, and ctrld does not create, delete, refresh, or flush NRPT policy.
The health monitor keeps using functional probes:
- matching GP rule + successful probe: observe only;
- matching GP rule + failed probe: retry loopback WFP protection, then report the external policy as ineffective without running NRPT heal signals;
- matching GP rule disappears: create the normal ctrld-owned fallback and verify it, unless another GP catch-all targets a different resolver;
- GP catch-all targets another resolver: report the conflict and do not create a second ambiguous catch-all;
- matching GP rule returns: prove it with a probe, remove only ctrld's deterministic fallback keys, and return ownership to Group Policy.
Deploy the GPO **before** starting or restarting ctrld if adapter DNS must remain completely untouched. Remove or unlink the GP rule before intentionally removing the ctrld service. A GP catch-all that remains pointed at loopback while no listener is running causes DNS failure by design; ctrld cannot safely delete an administrator-owned policy during uninstall.
#### WFP Filter Architecture
**Filter priority**: Permit filters have weight 10, block filters have weight 1. WFP evaluates higher-weight filters first, so localhost and private-range DNS is always permitted.
**RFC1918 + CGNAT permits**: Static subnet permit filters allow DNS to private IP ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10). This means VPN DNS servers on private IPs (Tailscale MagicDNS on 100.100.100.100, corporate VPN DNS on 10.x.x.x, etc.) work without needing dynamic per-server exemptions.
**VPN coexistence**: VPN software can set DNS to whatever it wants on the interface — for public IPs, the WFP block filter prevents those servers from being reached on port 53. For private IPs, the subnet permits allow it. ctrld handles all DNS routing through NRPT and can forward VPN-specific domains to VPN DNS servers through its own upstream mechanism.
#### Loopback WFP Protect (dns mode)
Third-party VPN software (e.g., OpenVPN, Securepoint SSL VPN) can install WFP block filters via `block-outside-dns` that block **all** DNS traffic to non-tunnel interfaces — including loopback. This breaks the NRPT → `127.0.0.1:53` path that ctrld depends on, causing DNS resolution to time out.
ctrld proactively adds 4 WFP "hard permit" filters at startup:
| Filter | Layer | Protocol |
|---|---|---|
| Permit DNS to localhost (IPv4/UDP) | ALE_AUTH_CONNECT_V4 | UDP |
| Permit DNS to localhost (IPv4/TCP) | ALE_AUTH_CONNECT_V4 | TCP |
| Permit DNS to localhost (IPv6/UDP) | ALE_AUTH_CONNECT_V6 | UDP |
| Permit DNS to localhost (IPv6/TCP) | ALE_AUTH_CONNECT_V6 | TCP |
**Key properties:**
- **Scope**: Port 53 to `127.0.0.1` (or configured listener IP) and `::1` only
- **Flag**: `FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT` (0x08) — "hard permit" that overrides BLOCK decisions from other sublayers regardless of weight or insertion order
- **Weight**: 15 (above hard mode's permit=10)
- **Sublayer**: ctrld's sublayer at maximum priority (0xFFFF)
- **Lifetime**: Process lifetime — added at startup, removed on shutdown/uninstall
Because `CLEAR_ACTION_RIGHT` is a cross-sublayer override, the order of filter installation doesn't matter — even if a VPN connects hours later and adds its own WFP block filters, ctrld's hard permit for loopback DNS is never overridden.
The reactive fallback in `nrptProbeAndHeal()` is preserved as defense-in-depth for edge cases where proactive activation fails at startup.
See: [Issue #526](https://gitlab.int.windscribe.com/controld/clients/ctrld/-/issues/526)
#### NRPT Probe and Auto-Heal
`RefreshPolicyEx` returns immediately — it does NOT wait for the DNS Client service to actually load the NRPT rule. On cold machines (first boot, fresh install), the DNS Client may take several seconds to process the policy refresh. During this window, the NRPT rule exists in the registry but isn't active.
ctrld verifies NRPT is actually working by sending a probe DNS query (`_nrpt-probe-<hex>.nrpt-probe.ctrld.test`) through Go's `net.Resolver` (which calls `GetAddrInfoW` → DNS Client → NRPT path). If ctrld receives the probe on its listener, NRPT is active.
**Startup probes:** A matching GP candidate is probed synchronously before any NRPT mutation and re-read afterward. ctrld-owned rules keep the asynchronous activation/heal sequence: immediate probe, bounded policy signaling retries, then two-phase delete/re-add recovery. Only one probe sequence runs at a time.
**Ownership boundary:** When the active owner is Group Policy, a failed probe never enters ctrld's NRPT refresh/delete/re-add sequence. ctrld may repair its narrowly scoped loopback WFP permits, but leaves the external registry child and DNS Client policy signaling to the administrator.
#### NRPT Health Monitor
A dedicated background goroutine (`nrptHealthMonitor`) runs every 30 seconds and now performs active probing:
1. **Ownership check:** Distinguish a matching external GP child from ctrld's deterministic local/GP keys.
2. **Active probe:** Verify Windows DNS Client still routes to the listener.
3. **Transition:** If the external child disappears, activate ctrld's normal fallback. If it returns while the fallback is active, prove it before removing only ctrld's keys.
4. **Owned recovery:** Restore/heal only when ctrld owns the NRPT rule.
5. **(hard mode)** Verify the WFP sublayer exists and fully restart intercept state on loss.
This is periodic (not just network-event-driven) because VPN software can clear NRPT at any time. Additionally, `scheduleDelayedRechecks()` (called on network change events) performs immediate NRPT verification at 2s and 4s after changes.
#### Known Caveats
- **`nslookup` bypasses NRPT**: `nslookup.exe` uses its own DNS resolver implementation and does NOT go through the Windows DNS Client service, so it ignores NRPT rules entirely. Use `Resolve-DnsName` (PowerShell) or `ping` to verify DNS resolution through NRPT. This is a well-known Windows behavior, not a ctrld bug.
- **`RPC_C_AUTHN_DEFAULT`**: `FwpmEngineOpen0` requires `RPC_C_AUTHN_DEFAULT` (0xFFFFFFFF) for the authentication service parameter. Using `RPC_C_AUTHN_NONE` (0) returns `ERROR_NOT_SUPPORTED` on some configurations (e.g., Parallels VMs).
- **FWP_DATA_TYPE enum**: The `FWP_DATA_TYPE` enum starts at `FWP_EMPTY=0`, making `FWP_UINT8=1`, `FWP_UINT16=2`, etc. Some documentation examples incorrectly start at 0.
### macOS (pf)
1. ctrld writes a pf anchor file at `/etc/pf.anchors/com.controld.ctrld`
2. Adds the anchor reference to `/etc/pf.conf` (if not present)
3. Loads the anchor with `pfctl -a com.controld.ctrld -f <file>`
4. Enables pf with `pfctl -e` (if not already enabled)
5. The anchor redirects all outbound DNS (port 53) on non-loopback interfaces to `127.0.0.1:53`
6. On shutdown, the anchor is flushed, the file removed, and references cleaned from `pf.conf`
**ctrld's own traffic**: ctrld's upstream queries use DoH (HTTPS on port 443), not plain DNS on port 53, so the pf redirect does not create a loop for DoH upstreams. **Warning:** If an "os" upstream is configured (which uses plain DNS on port 53 to external servers), the pf redirect will capture ctrld's own outbound queries and create a loop. ctrld will log a warning at startup if this is detected. Use DoH upstreams when DNS intercept mode is active.
## What Changes vs Default Mode
| Behavior | Default Mode | DNS Intercept Mode |
|----------|-------------|-------------------|
| Interface DNS settings | Set to `127.0.0.1` | **Not modified** |
| DNS watchdog | Active (polls every 20s) | **Disabled** |
| VPN DNS conflict | Race condition possible | **Eliminated** |
| Profile bypass window | Up to 20 seconds | **Zero** |
| Requires admin/root | Yes | Yes |
| Additional OS requirements | None | WFP (Windows), pf (macOS) |
## Logging
DNS intercept mode produces detailed logs for troubleshooting:
```
DNS intercept: initializing Windows Filtering Platform (WFP)
DNS intercept: WFP engine opened (handle: 0x1a2b3c)
DNS intercept: WFP sublayer created (weight: 0xFFFF — maximum priority)
DNS intercept: added permit filter "Permit DNS to localhost (IPv4/UDP)" (ID: 12345)
DNS intercept: added block filter "Block outbound DNS (IPv4/UDP)" (ID: 12349)
DNS intercept: WFP filters active — all outbound DNS (port 53) blocked except to localhost
```
On macOS:
```
DNS intercept: initializing macOS packet filter (pf) redirect
DNS intercept: wrote pf anchor file: /etc/pf.anchors/com.controld.ctrld
DNS intercept: loaded pf anchor "com.controld.ctrld"
DNS intercept: pf anchor "com.controld.ctrld" active with 3 rules
DNS intercept: pf redirect active — all outbound DNS (port 53) redirected to 127.0.0.1:53
```
## Troubleshooting
### Windows
```powershell
# Check NRPT rules (should show CtrldCatchAll with . → 127.0.0.1)
Get-DnsClientNrptRule
# Check NRPT registry directly
Get-ChildItem "HKLM:\SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig"
# Force Group Policy refresh (if NRPT not taking effect)
gpupdate /target:computer /force
# Check if WFP filters are active
netsh wfp show filters
# Check ctrld's specific filters (look for "ctrld" in output)
netsh wfp show filters | Select-String "ctrld"
# Test DNS resolution (use Resolve-DnsName, NOT nslookup!)
# nslookup bypasses DNS Client / NRPT — it will NOT reflect NRPT routing
Resolve-DnsName example.com
ping example.com
# If you must use nslookup, specify localhost explicitly:
nslookup example.com 127.0.0.1
```
### macOS
```bash
# Check if pf is enabled
sudo pfctl -si
# Check ctrld's anchor rules
sudo pfctl -a com.controld.ctrld -sr
sudo pfctl -a com.controld.ctrld -sn
# Check pf.conf for anchor reference
cat /etc/pf.conf | grep ctrld
# Test DNS is going through ctrld
dig @127.0.0.1 example.com
```
## Limitations
- **Linux**: Not supported. Linux uses `systemd-resolved` or `/etc/resolv.conf` which don't have the same VPN conflict issues. If needed in the future, `iptables`/`nftables` REDIRECT could be used.
- **Split DNS for VPN internal domains**: In `--intercept-mode dns` mode, VPN search domains are auto-detected from virtual network adapters and forwarded to the VPN's DNS servers automatically. In `--intercept-mode hard` mode, VPN internal domains (e.g., `*.corp.local`) will NOT resolve unless configured as explicit upstream rules in ctrld's configuration.
- **macOS mDNSResponder interaction**: On macOS, ctrld uses a workaround ("mDNSResponder hack") that binds to `0.0.0.0:53` instead of `127.0.0.1:53` and refuses queries from non-localhost sources. In dns-intercept mode, pf's `rdr` rewrites the destination IP to `127.0.0.1:53` but preserves the original source IP (e.g., `192.168.2.73`). The mDNSResponder source-IP check is automatically bypassed in dns-intercept mode because the pf/WFP rules already ensure only legitimate intercepted DNS traffic reaches ctrld's listener.
- **Other WFP/pf users**: If other software (VPN, firewall, endpoint security) also uses WFP or pf for DNS interception, there may be priority conflicts. ctrld uses maximum sublayer weight on Windows and a named anchor on macOS to minimize this risk. See "VPN App Coexistence" below for macOS-specific defenses.
## VPN App Coexistence (macOS)
VPN apps (Windscribe, Cisco AnyConnect, F5 BIG-IP, etc.) often manage pf rules themselves, which can interfere with ctrld's DNS intercept. ctrld uses a multi-layered defense strategy:
### 1. Anchor Priority Enforcement
When injecting our anchor reference into the running pf ruleset, ctrld **prepends** both the `rdr-anchor` and `anchor` references before all other anchors. pf evaluates rules top-to-bottom, so our DNS intercept `quick` rules match port 53 traffic before a VPN app's broader rules in their own anchor.
### 2. Interface-Specific Tunnel Rules
VPN apps commonly add rules like `pass out quick on ipsec0 inet all` that match ALL traffic on the VPN interface. If their anchor is evaluated before ours (e.g., after a ruleset reload), these broad rules capture DNS. ctrld counters this by adding explicit DNS intercept rules for each active tunnel interface (ipsec*, utun*, ppp*, tap*, tun*). These interface-specific rules match port 53 only, so they take priority over the VPN app's broader "all" match even within the same anchor evaluation pass.
### 3. Dynamic Tunnel Interface Detection
The network change monitor (`validInterfacesMap()`) only tracks physical hardware ports (en0, bridge0, etc.) — it doesn't see tunnel interfaces (utun*, ipsec*, etc.) created by VPN software. When a VPN connects and creates a new interface (e.g., utun420 for WireGuard), ctrld detects this through a separate tunnel interface change check and rebuilds the pf anchor to include explicit intercept rules for the new interface. This runs on every network change event, even if no physical interface changed.
### 4. pf Watchdog + Network Change Hooks
A background watchdog (30s interval) plus immediate checks on network change events detect when another program replaces the entire pf ruleset (e.g., Windscribe's `pfctl -f /etc/pf.conf`). When detected, ctrld rebuilds its anchor with up-to-date tunnel interface rules and re-injects the anchor reference at the top of the ruleset. A 2-second delayed re-check catches race conditions where the other program clears rules slightly after the network event.
### 4a. Active Interception Probe (pf Translation State Corruption)
Programs like Parallels Desktop reload `/etc/pf.conf` when creating/destroying virtual network interfaces (bridge100, vmenet0). This can corrupt pf's internal translation engine — rdr rules survive in text form but stop evaluating, causing DNS interception to silently fail while the watchdog reports "intact."
ctrld detects interface appearance/disappearance and spawns an async probe monitor:
1. **Probe mechanism:** A subprocess runs with GID=0 (wheel, not `_ctrld`) and sends a DNS query to the OS resolver. If pf interception is working, the query gets redirected to ctrld (127.0.0.1:53) and is detected in the DNS handler. If broken, it times out after 1s.
2. **Backoff schedule:** Probes at 0, 0.5, 1, 2, 4 seconds (~8s window) to win the race against async pf reloads by the hypervisor. Only one monitor runs at a time (atomic singleton).
3. **Auto-heal:** On probe failure, `forceReloadPFMainRuleset()` dumps the running ruleset and pipes it back through `pfctl -f -`, resetting pf's translation engine. VPN-safe because it reassembles from the current running state.
4. **Watchdog integration:** The 30s watchdog also runs the probe when rule text checks pass, as a safety net for unknown corruption causes.
This approach detects **actual broken DNS** rather than guessing from trigger events, making it robust against future unknown corruption scenarios.
### 5. Proactive DoH Connection Pool Reset
When the watchdog detects a pf ruleset replacement, it force-rebootstraps all upstream transports via `ForceReBootstrap()`. This is necessary because `pfctl -f` flushes the entire pf state table, which kills existing TCP connections (including ctrld's DoH connections to upstream DNS servers like 76.76.2.22:443).
The force-rebootstrap does two things that the lazy `ReBootstrap()` cannot:
1. **Closes idle connections on the old transport** (`CloseIdleConnections()`), causing in-flight HTTP/2 requests on dead connections to fail immediately instead of waiting for the 5s context deadline
2. **Creates the new transport synchronously**, so it's ready before any DNS queries arrive post-wipe
Without this, Go's `http.Transport` keeps trying dead connections until each request's context deadline expires (~5s), then the lazy rebootstrap creates a new transport for the *next* request. With force-rebootstrap, the blackout is reduced from ~5s to ~100ms (one fresh TLS handshake).
### 6. Blanket Process Exemption (group _ctrld)
ctrld creates a macOS system group (`_ctrld`) and sets its effective GID at startup via `syscall.Setegid()`. The pf anchor includes a blanket rule:
```
pass out quick group _ctrld
```
This exempts **all** outbound traffic from the ctrld process — not just DNS (port 53), but also DoH (TCP 443), DoT (TCP 853), health checks, and any other connections. This is essential because VPN firewalls like Windscribe load `block drop all` rulesets that would otherwise block ctrld's upstream connections even after the pf anchor is restored.
Because ctrld's anchor is prepended before all other anchors, and this rule uses `quick`, it evaluates before any VPN firewall rules. The result: ctrld's traffic is never blocked regardless of what other pf rulesets are loaded.
The per-IP exemptions (OS resolver, VPN DNS) remain as defense-in-depth for the DNS redirect loop prevention — the blanket rule handles everything else.
### 7. Loopback Outbound Pass Rule
When `route-to lo0` redirects a DNS packet to loopback, pf re-evaluates the packet **outbound on lo0**. None of the existing route-to rules match on lo0 (they're all `on ! lo0` or `on utunX`), so without an explicit pass rule, the packet falls through to the main ruleset where VPN firewalls' `block drop all` drops it — before it ever reaches the inbound rdr rule.
```
pass out quick on lo0 inet proto udp from any to ! 127.0.0.1 port 53
pass out quick on lo0 inet proto tcp from any to ! 127.0.0.1 port 53
```
This bridges the route-to → rdr gap: route-to sends outbound on lo0 → this rule passes it → loopback reflects it inbound → rdr rewrites destination to 127.0.0.1:53 → ctrld receives the query. Without this rule, DNS intercept fails whenever a `block drop all` firewall (Windscribe, etc.) is active.
### 8. Response Routing via `reply-to lo0`
After rdr redirects DNS to 127.0.0.1:53, ctrld responds to the original client source IP (e.g., 100.94.163.168 — a VPN tunnel IP). Without intervention, the kernel routes this response through the VPN tunnel interface (utun420) based on its routing table, and the response is lost.
```
pass in quick on lo0 reply-to lo0 inet proto { udp, tcp } from any to 127.0.0.1 port 53
```
`reply-to lo0` tells pf to force response packets for this connection back through lo0, overriding the kernel routing table. The response stays local, rdr reverse NAT rewrites the source from 127.0.0.1 back to the original DNS server IP (e.g., 10.255.255.3), and the client process receives a correctly-addressed response.
### 9. VPN DNS Split Routing and Exit Mode Detection
When a VPN like Tailscale MagicDNS is active, two distinct modes require different pf handling:
#### The Problem: DNS Proxy Loop
VPN DNS handlers like Tailscale's MagicDNS run as macOS Network Extensions. MagicDNS
listens on 100.100.100.100 and forwards queries to internal upstream nameservers
(e.g., 10.0.0.11, 10.0.0.12) via the VPN tunnel interface (utun13).
Without special handling, pf's generic `pass out quick on ! lo0 route-to lo0` rule
intercepts MagicDNS's upstream queries on the tunnel interface, routing them back
to ctrld → which matches VPN DNS split routing → forwards to MagicDNS → loop:
```
┌──────────────────────────────────────────────────────────────────────┐
│ THE LOOP (without passthrough rules) │
│ │
│ 1. dig vpn-internal.example.com │
│ → pf intercepts → route-to lo0 → rdr → ctrld (127.0.0.1:53) │
│ │
│ 2. ctrld: VPN DNS match → forward to 100.100.100.100:53 │
│ → group _ctrld exempts → reaches MagicDNS │
│ │
│ 3. MagicDNS: forward to upstream 10.0.0.11:53 via utun13 │
│ → pf generic rule matches (utun13 ≠ lo0, 10.0.0.11 ≠ skip) │
│ → route-to lo0 → rdr → back to ctrld ← LOOP! │
└──────────────────────────────────────────────────────────────────────┘
```
#### The Fix: Interface Passthrough + Exit Mode Detection
**Split DNS mode** (VPN handles only specific domains):
ctrld adds passthrough rules for VPN DNS interfaces that let MagicDNS's upstream
queries flow without interception. A `<vpn_dns>` table contains the VPN DNS server
IPs (e.g., 100.100.100.100) — traffic TO those IPs is NOT passed through (still
intercepted by pf → ctrld enforces profile):
```
table <vpn_dns> { 100.100.100.100 }
# MagicDNS upstream queries (to 10.0.0.11 etc.) — pass through
pass out quick on utun13 inet proto udp from any to ! <vpn_dns> port 53
pass out quick on utun13 inet proto tcp from any to ! <vpn_dns> port 53
# Queries TO MagicDNS (100.100.100.100) — not matched above,
# falls through to generic rule → intercepted → ctrld → profile enforced
```
```
┌──────────────────────────────────────────────────────────────────────┐
│ SPLIT DNS MODE (with passthrough rules) │
│ │
│ Non-VPN domain (popads.net): │
│ dig popads.net → system routes to 100.100.100.100 on utun13 │
│ → passthrough rule: dest IS in <vpn_dns> → NOT matched │
│ → generic rule: route-to lo0 → rdr → ctrld → profile blocks it ✅ │
│ │
│ VPN domain (vpn-internal.example.com): │
│ dig vpn-internal.example.com → pf intercepts → ctrld │
│ → VPN DNS match → forward to 100.100.100.100 (group exempt) │
│ → MagicDNS → upstream 10.0.0.11 on utun13 │
│ → passthrough rule: dest NOT in <vpn_dns> → MATCHED → passes ✅ │
│ → 10.0.0.11 returns correct internal answer (10.0.0.113) │
└──────────────────────────────────────────────────────────────────────┘
```
**Exit mode** (all traffic through VPN):
When Tailscale exit node is enabled, MagicDNS becomes the system's **default**
resolver (not just supplemental). If we added passthrough rules, ALL DNS would
bypass ctrld — losing profile enforcement.
Exit mode is detected using two independent signals (either triggers exit mode):
**1. Default route detection (primary, most reliable):**
Uses `netmon.DefaultRouteInterface()` to check if the system's default route
(0.0.0.0/0) goes through a VPN DNS interface. If `DefaultRouteInterface` matches
a VPN DNS interface name (e.g., utun13), the VPN owns the default route — it's
exit mode. This is the ground truth: the routing table directly reflects whether
all traffic flows through the VPN, regardless of how the VPN presents itself in
scutil.
**2. scutil flag detection (secondary, fallback):**
If the VPN DNS server IP appears in a `scutil --dns` resolver entry that has
**no search domains** and **no Supplemental flag**, it's acting as the system's
default resolver (exit mode). This catches edge cases where the default route
hasn't changed yet but scutil already shows the VPN as the default DNS.
```
# Non-exit mode — default route on en0, 100.100.100.100 is Supplemental:
$ route -n get 0.0.0.0 | grep interface
interface: en0 ← physical NIC, not VPN
resolver #1
search domain[0] : vpn.example.com
nameserver[0] : 100.100.100.100
flags : Supplemental, Request A records
# Exit mode — default route on utun13, 100.100.100.100 is default resolver:
$ route -n get 0.0.0.0 | grep interface
interface: utun13 ← VPN interface!
resolver #2
nameserver[0] : 100.100.100.100 ← MagicDNS is default
flags : Request A records ← no Supplemental!
```
In exit mode, NO passthrough rules are generated. pf intercepts all DNS → ctrld
enforces its profile on everything. VPN search domains still resolve correctly
via ctrld's VPN DNS split routing (forwarded to MagicDNS through the group
exemption).
#### Summary Table
| Scenario | Passthrough | Profile Enforced | VPN Domains |
|----------|-------------|-----------------|-------------|
| No VPN | None | ✅ All traffic | N/A |
| Split DNS (Tailscale non-exit) | ✅ VPN interface | ✅ Non-VPN domains | ✅ Via MagicDNS |
| Exit mode (Tailscale exit node) | ❌ None | ✅ All traffic | ✅ Via ctrld split routing |
| Windscribe | None (different flow) | ✅ All traffic | N/A |
| Hard intercept | None | ✅ All traffic | ❌ Not forwarded |
### Nuclear Option (Future)
If anchor ordering + interface rules prove insufficient, an alternative approach is available: inject DNS intercept rules directly into the **main pf ruleset** (not inside an anchor). Main ruleset rules are evaluated before ALL anchors, making them impossible for another app to override without explicitly removing them. This is more invasive and not currently implemented, but documented here as a known escalation path.
## Known VPN Conflicts
### F5 BIG-IP APM
F5 BIG-IP APM VPN is a known source of DNS conflicts with ctrld (a known support scenario). The conflict occurs because F5's VPN client aggressively manages DNS:
**How the conflict manifests:**
1. ctrld sets system DNS to `127.0.0.1` / `::1` for local forwarding
2. F5 VPN connects and **overwrites DNS on all interfaces** by prepending its own servers (e.g., `10.20.30.1`, `10.20.30.2`)
3. F5 enforces split DNS patterns (e.g., `*.corp.example.com`) and activates its DNS Relay Proxy (`F5FltSrv.exe` / `F5FltSrv.sys`)
4. ctrld's watchdog detects the change and restores `127.0.0.1` — F5 overwrites again
5. This loop causes intermittent resolution failures, slow responses, and VPN disconnects
**Why `--intercept-mode dns` solves this:**
- ctrld no longer modifies interface DNS settings — there is nothing for F5 to overwrite
- WFP (Windows) blocks all outbound DNS except to localhost, so F5's prepended DNS servers are unreachable on port 53
- F5's DNS Relay Proxy (`F5FltSrv`) becomes irrelevant since no queries reach it
- In `--intercept-mode dns` mode, F5's split DNS domains (e.g., `*.corp.example.com`) are auto-detected from the VPN adapter and forwarded to F5's DNS servers through ctrld's upstream mechanism
**F5-side mitigations (if `--intercept-mode dns` is not available):**
- In APM Network Access DNS settings, enable **"Allow Local DNS Servers"** (`AllowLocalDNSServersAccess = 1`)
- Disable **"Enforce DNS Name Resolution Order"**
- Switch to IP-based split tunneling instead of DNS-pattern-based to avoid activating F5's relay proxy
- Update F5 to version 17.x+ which includes DNS handling fixes (see F5 KB K80231353)
**Additional considerations:**
- CrowdStrike Falcon and similar endpoint security with network inspection can compound the conflict (three-way DNS stomping)
- F5's relay proxy (`F5FltSrv`) performs similar functions to ctrld — they are in direct conflict when both active
- The seemingly random failure pattern is caused by timing-dependent race conditions between ctrld's watchdog, F5's DNS enforcement, and (optionally) endpoint security inspection
### Cisco AnyConnect
Cisco AnyConnect exhibits similar DNS override behavior. `--intercept-mode dns` mode prevents the conflict by operating at the packet filter level rather than competing for interface DNS settings.
### Windscribe Desktop App
Windscribe's macOS firewall implementation (`FirewallController_mac`) replaces the entire pf ruleset when connecting/disconnecting via `pfctl -f`, which wipes ctrld's anchor references and flushes the pf state table (killing active DoH connections). ctrld handles this with multiple defenses:
1. **pf watchdog** detects the wipe and restores anchor rules immediately on network change events (or within 30s via periodic check)
2. **DoH transport force-reset** immediately replaces upstream transports when a pf wipe is detected (closing old connections + creating new ones synchronously), reducing the DNS blackout from ~5s to ~100ms
3. **Tunnel interface detection** adds explicit intercept rules for Windscribe's WireGuard interface (e.g., utun420) when it appears
4. **Dual delayed re-checks** (2s + 4s after network event) catch race conditions where VPN apps modify pf rules and DNS settings asynchronously after the initial network change
5. **Deferred pf restore** waits for VPN to finish its pf modifications before restoring ctrld's rules, preventing the reconnect death spiral
6. **Blanket group exemption** (`pass out quick group _ctrld`) ensures all ctrld traffic (including DoH on port 443) passes through VPN firewalls like Windscribe's `block drop all`
## 7. VPN DNS Lifecycle
When VPN software connects or disconnects, ctrld must track DNS state changes to ensure correct routing and avoid stale state.
### Network Change Event Flow (macOS)
```
Network change detected (netmon callback)
├─ Immediate actions:
│ ├─ ensurePFAnchorActive() — verify/restore pf anchor references
│ ├─ checkTunnelInterfaceChanges() — detect new/removed VPN interfaces
│ │ ├─ New tunnel → pfStartStabilization() (wait for VPN to finish pf changes)
│ │ └─ Removed tunnel → rebuild anchor immediately (with VPN DNS exemptions)
│ └─ vpnDNS.Refresh() — re-discover VPN DNS from scutil --dns
├─ Delayed re-check at 2s:
│ ├─ ensurePFAnchorActive() — catch async pf wipes
│ ├─ checkTunnelInterfaceChanges()
│ ├─ InitializeOsResolver() — clear stale DNS from scutil
│ └─ vpnDNS.Refresh() — clear stale VPN DNS routes
└─ Delayed re-check at 4s:
└─ (same as 2s — catches slower VPN teardowns)
```
### VPN Connect Sequence
1. VPN creates tunnel interface (e.g., utun420)
2. Network change fires → `checkTunnelInterfaceChanges()` detects new tunnel
3. **Stabilization mode** activates — suppresses pf restores while VPN modifies rules
4. Stabilization loop polls `pfctl -sr` hash every 1.5s
5. When hash stable for 6s → VPN finished → restore ctrld's pf anchor
6. `vpnDNS.Refresh()` discovers VPN's search domains and DNS servers from `scutil --dns`
7. Anchor rebuild includes VPN DNS exemptions (so ctrld can reach VPN DNS on port 53)
### VPN Disconnect Sequence
1. VPN removes tunnel interface
2. Network change fires → `checkTunnelInterfaceChanges()` detects removal
3. Anchor rebuilt immediately (no stabilization needed for removals)
4. VPN app may asynchronously wipe pf rules (`pfctl -f /etc/pf.conf`)
5. VPN app may asynchronously clean up DNS settings from `scutil --dns`
6. **2s delayed re-check**: restores pf anchor if wiped, refreshes OS resolver
7. **4s delayed re-check**: catches slower VPN teardowns
8. `vpnDNS.Refresh()` returns empty → `onServersChanged(nil)` clears stale exemptions
9. `InitializeOsResolver()` re-reads `scutil --dns` → clears stale LAN nameservers
### Key Design Decisions
- **`buildPFAnchorRules()` receives VPN DNS servers**: All call sites (tunnel rebuild, watchdog restore, stabilization exit) pass `vpnDNS.CurrentServers()` so exemptions are preserved for still-active VPNs.
- **`onServersChanged` called even when server list is empty**: Ensures stale pf exemptions from a previous VPN session are cleaned up on disconnect.
- **OS resolver refresh in delayed re-checks**: VPN apps often finish DNS cleanup 1-3s after the network change event. The delayed `InitializeOsResolver()` call ensures stale LAN nameservers (e.g., a VPN's DNS IP (e.g., 10.255.255.3)) don't cause 2s query timeouts.
- **Ordering: tunnel checks → VPN DNS refresh → delayed re-checks**: Ensures anchor rebuilds from tunnel changes include current VPN DNS exemptions.
## Related
- F5 BIG-IP APM VPN DNS conflict (a known support scenario)
+51
View File
@@ -22,6 +22,57 @@ This document outlines known issues with ctrld and their current status, workaro
---
## Merlin Issues
### Daemon Crashing on `Ctrl+C`
**Issue**: `ctrld` daemon terminates unexpectedly after stopping a log tailing command. This typically occurs when running the daemon and the log viewer within the same SSH session on ASUSWRT-Merlin routers.
**Description**
The issue is caused by `Signal Propagation` within a shared `Process Group (PGID)`.
Steps to reproduce:
1. You start the daemon manually: `ctrld start --cd=<uid>`.
2. You view internal logs in the same terminal: `ctrld log tail`.
3. You press `Ctrl+C` to stop viewing logs.
4. The `ctrld` daemon service stops immediately along with the log command.
When you execute commands sequentially in a single interactive SSH session on Merlin, the shell often assigns them to the same Process Group. In Linux, the `SIGINT` signal (triggered by `Ctrl+C`) is not just sent to the foreground application, but is frequently propagated to every process belonging to that specific process group.
Because the `ctrld` daemon remains "attached" to the terminal session's process group, it "hears" the interrupt signal intended for the `log tail` command and shuts down.
**Workarounds**:
To isolate the signals, avoid running the log viewer in the same window as the daemon:
* **Window A:** Start the daemon and leave it running.
* **Window B:** Open a new SSH connection to run `ctrld log tail`.
Because Window B has a different **Session ID** and **Process Group ID**, pressing `Ctrl+C` in Window B will not affect the process in Window A.
## Windows Issues
### VPN `block-outside-dns` Breaks DNS When Using ctrld in DNS Mode
**Issue**: VPN software that uses OpenVPN's `block-outside-dns` directive installs WFP (Windows Filtering Platform) block filters that prevent DNS queries from reaching ctrld's loopback listener.
**Status**: Fixed in v1.5.1
**Description**: When a VPN connects with `block-outside-dns` enabled, OpenVPN adds WFP filters that block all DNS traffic to non-tunnel interfaces — including loopback (`127.0.0.1`). Since ctrld's NRPT catch-all rule routes DNS through the Windows DNS Client to `127.0.0.1:53`, the WFP block filters prevent DNS Client from reaching ctrld, causing all DNS queries to time out.
This affects any VPN client that implements `block-outside-dns` via WFP, including:
- OpenVPN GUI (community)
- Securepoint SSL VPN
- Any OpenVPN-based client that honors the `block-outside-dns` push directive
**Fix**: ctrld now proactively adds WFP "hard permit" filters for DNS to localhost at startup. These use `FWPM_FILTER_FLAG_CLEAR_ACTION_RIGHT` to override block decisions from any other WFP sublayer, ensuring the NRPT → loopback path is always available regardless of VPN state. See `docs/dns-intercept-mode.md` for technical details.
**Affected Versions**: ctrld ≤ v1.5.0 in `dns` intercept mode on Windows
**Last Updated**: 04/28/2026
---
## Contributing to Known Issues
If you encounter an issue not listed here, please:
+356
View File
@@ -0,0 +1,356 @@
# macOS pf DNS Interception — Technical Reference
## Overview
ctrld uses macOS's built-in packet filter (pf) to intercept all DNS traffic at the kernel level, redirecting it to ctrld's local listeners at `127.0.0.1:53` (IPv4) and `[::1]:53` (IPv6). This operates below interface DNS settings, making it immune to VPN software (F5, Cisco, GlobalProtect, etc.) that overwrites DNS on network interfaces.
## How pf Works (Relevant Basics)
pf is a stateful packet filter built into macOS (and BSD). It processes packets through a pipeline with **strict rule ordering**:
```
options (set) → normalization (scrub) → queueing → translation (nat/rdr) → filtering (pass/block)
```
**Anchors** are named rule containers that allow programs to manage their own rules without modifying the global ruleset. Each anchor type must appear in the correct section:
| Anchor Type | Section | Purpose |
|-------------|---------|---------|
| `scrub-anchor` | Normalization | Packet normalization |
| `nat-anchor` | Translation | NAT rules (not used by ctrld) |
| `rdr-anchor` | Translation | Redirect rules |
| `anchor` | Filtering | Pass/block rules |
**Critical constraint:** If you place a `rdr-anchor` line after an `anchor` line, pf rejects the entire config with "Rules must be in order."
## Why We Can't Just Use `rdr on ! lo0`
The obvious approach:
```
rdr pass on ! lo0 proto udp from any to any port 53 -> 127.0.0.1 port 53
```
**This doesn't work.** macOS pf `rdr` rules only apply to *forwarded/routed* traffic — packets passing through the machine to another destination. DNS queries originating from the machine itself (locally-originated) are never matched by `rdr` on non-loopback interfaces.
This is a well-known pf limitation on macOS/BSD. It means the VPN client's DNS queries would be redirected (if routed through the machine), but the user's own applications querying DNS directly would not.
## Our Approach: route-to + rdr (Two-Step)
We use a two-step technique to intercept locally-originated DNS:
```
Step 1: Force outbound DNS through loopback
pass out quick on ! lo0 route-to lo0 inet proto udp from any to ! 127.0.0.1 port 53
Step 2: Pass the packet outbound on lo0 (needed when VPN firewalls have "block drop all")
pass out quick on lo0 inet proto udp from any to ! 127.0.0.1 port 53 no state
Step 3: Redirect it on loopback to ctrld's listener
rdr on lo0 inet proto udp from any to ! 127.0.0.1 port 53 -> 127.0.0.1 port 53
Step 4: Accept and create state for response routing
pass in quick on lo0 reply-to lo0 inet proto { udp, tcp } from any to 127.0.0.1 port 53
```
> **State handling is critical for VPN firewall coexistence:**
> - **route-to**: `keep state` (default). State is interface-bound on macOS — doesn't match on lo0.
> - **pass out lo0**: `no state`. If this created state, it would match inbound on lo0 and bypass rdr.
> - **rdr**: no `pass` keyword. Packet must go through filter so `pass in` can create response state.
> - **pass in lo0**: `keep state` (default). Creates the ONLY state on lo0 — handles response routing.
### Packet Flow
```
Application queries 10.255.255.3:53 (e.g., VPN DNS server)
Kernel: outbound on en0 (or utun420 for VPN)
pf filter: "pass out route-to lo0 ... port 53" → redirects to lo0, creates state on en0
pf filter (outbound lo0): "pass out on lo0 ... no state" → passes, NO state created
Loopback reflects packet inbound on lo0
pf rdr (inbound lo0): "rdr on lo0 ... port 53 -> 127.0.0.1:53" → rewrites destination
pf filter (inbound lo0): "pass in reply-to lo0 ... to 127.0.0.1:53" → creates state + reply route
ctrld receives query on 127.0.0.1:53
ctrld resolves via DoH (port 443, exempted by group _ctrld)
Response from ctrld: 127.0.0.1:53 → 100.94.163.168:54851
reply-to lo0: forces response through lo0 (without this, kernel routes via utun420 → lost in VPN tunnel)
pf applies rdr reverse NAT: src 127.0.0.1 → 10.255.255.3
Application receives response from 10.255.255.3:53 ✓
```
### Why This Works
1. `route-to lo0` forces the packet onto loopback at the filter stage
2. `pass out on lo0 no state` gets past VPN "block drop all" without creating state
3. No state on lo0 means rdr gets fresh evaluation on the inbound pass
4. `reply-to lo0` on `pass in` forces the response through lo0 — without it, the kernel routes the response to VPN tunnel IPs via the VPN interface and it's lost
4. `rdr` (without `pass`) redirects then hands off to filter rules
5. `pass in keep state` creates the response state — the only state on the lo0 path
6. Traffic already destined for `127.0.0.1` is excluded (`to ! 127.0.0.1`) to prevent loops
7. ctrld's own upstream queries use DoH (port 443), bypassing port 53 rules entirely
### Why Each State Decision Matters
| Rule | State | Why |
|------|-------|-----|
| route-to on en0/utun | keep state | Needed for return routing. Interface-bound, won't match on lo0. |
| pass out on lo0 | **no state** | If stateful, it would match inbound lo0 → bypass rdr → DNS broken |
| rdr on lo0 | N/A (no pass) | Must go through filter so pass-in creates response state |
| pass in on lo0 | keep state + reply-to lo0 | Creates lo0 state. `reply-to` forces response through lo0 (not VPN tunnel). |
## IPv6 DNS Interception
macOS systems with IPv6 nameservers (common — `scutil --dns` often shows an IPv6 nameserver at index 0) send DNS queries over IPv6. Without IPv6 interception, these queries bypass ctrld, causing ~1s delays (the IPv6 query times out, then the app falls back to IPv4).
### Why IPv6 Needs Special Handling
Three problems prevent a simple "mirror the IPv4 rules" approach:
1. **Cross-AF redirect is impossible**: pf cannot `rdr on lo0 inet6 ... -> 127.0.0.1` (redirecting IPv6 to IPv4). ctrld must listen on `[::1]` to handle IPv6 DNS.
2. **`block return` is ineffective for IPv6 DNS**: BSD doesn't deliver ICMPv6 unreachable errors to unconnected UDP sockets (which `dig` and most resolvers use). So `block return out inet6 ... port 53` generates the ICMP error, but the application never receives it — it waits for the full timeout (~1s).
3. **sendmsg from `[::1]` to global unicast fails**: Unlike IPv4 where the kernel allows `sendmsg` from `127.0.0.1` to local private IPs (e.g., `10.x.x.x`), macOS/BSD rejects `sendmsg` from `[::1]` to a global unicast IPv6 address with `EINVAL`. Since pf's `rdr` preserves the original source IP (the machine's global IPv6 address), ctrld's reply would fail.
### Solution: Block IPv6 DNS, Fallback to IPv4
After extensive testing (#507), IPv6 DNS interception on macOS is not feasible with current pf capabilities. The solution is to block all outbound IPv6 DNS:
```
block out quick on ! lo0 inet6 proto { udp, tcp } from any to any port 53
```
macOS automatically retries DNS over IPv4 when the IPv6 path is blocked. The IPv4 path is fully intercepted via the normal route-to + rdr mechanism. Impact is minimal — at most ~1s latency on the very first DNS query while the IPv6 attempt is blocked.
### What Was Tried and Why It Failed
| Approach | Result |
|----------|--------|
| `nat on lo0 inet6` to rewrite source to `::1` | pf skips translation on second interface pass — nat doesn't fire for route-to'd packets arriving on lo0 |
| ULA address on lo0 (`fd00:53::1`) | Kernel rejects: `EHOSTUNREACH` — lo0's routing table is segregated from global unicast |
| Raw IPv6 socket (`SOCK_RAW` + `IPPROTO_UDP`) | Bypasses sendmsg validation, but pf doesn't match raw socket packets against rdr state — response arrives from `::1` not the original server |
| `DIOCNATLOOK` to get original dest + raw socket from that addr | Can't `bind()` to a non-local address (`EADDRNOTAVAIL`) — macOS has no `IPV6_HDRINCL` for source spoofing |
| BPF packet injection on lo0 | Theoretically possible but extremely complex — not justified for the marginal benefit |
### IPv6 Listener
The `[::1]` listener is used on:
- **Windows**: Always (if IPv6 is available) — Windows can't easily suppress IPv6 DNS resolvers
- **macOS**: **Not used** — IPv6 DNS is blocked at pf, no listener needed
## Rule Ordering Within the Anchor
pf requires translation rules before filter rules, even within an anchor:
```pf
# === Translation rules (MUST come first) ===
rdr on lo0 inet proto udp from any to ! 127.0.0.1 port 53 -> 127.0.0.1 port 53
rdr on lo0 inet proto tcp from any to ! 127.0.0.1 port 53 -> 127.0.0.1 port 53
# === Exemptions (filter phase, scoped to _ctrld group) ===
pass out quick on ! lo0 inet proto { udp, tcp } from any to <OS_RESOLVER_IP> port 53 group _ctrld
pass out quick on ! lo0 inet proto { udp, tcp } from any to <VPN_DNS_IP> port 53 group _ctrld
# === Main intercept (filter phase) ===
pass out quick on ! lo0 route-to lo0 inet proto udp from any to ! 127.0.0.1 port 53
pass out quick on ! lo0 route-to lo0 inet proto tcp from any to ! 127.0.0.1 port 53
# === Allow redirected traffic on loopback ===
pass in quick on lo0 reply-to lo0 inet proto { udp, tcp } from any to 127.0.0.1 port 53
```
### Exemption Mechanism (Group-Scoped)
Some IPs must bypass the redirect:
- **OS resolver nameservers** (e.g., DHCP-assigned DNS): ctrld's recovery/bootstrap path may query these on port 53. Without exemption, these queries loop back to ctrld.
- **VPN DNS servers**: When ctrld forwards VPN-specific domains (split DNS) to the VPN's internal DNS, those queries must reach the VPN DNS server directly.
Exemptions use `pass out quick` with `group _ctrld` **before** the `route-to` rule. The `group _ctrld` constraint ensures that **only ctrld's own process** can bypass the redirect — other applications cannot circumvent DNS interception by querying the exempted IPs directly. Because pf evaluates filter rules in order and `quick` terminates evaluation, the exempted packet goes directly out the real interface and never hits the `route-to` or `rdr`.
### The `_ctrld` Group
To scope pf exemptions to ctrld's process only, we use a dedicated macOS system group:
1. **Creation**: On startup, `ensureCtrldGroup()` creates a `_ctrld` system group via `dscl` (macOS Directory Services) if it doesn't already exist. The GID is chosen from the 350-450 range to avoid conflicts with Apple's reserved ranges. The function is idempotent.
2. **Process GID**: Before loading pf rules, ctrld sets its effective GID to `_ctrld` via `syscall.Setegid()`. All sockets created by ctrld after this point are tagged with this GID.
3. **pf matching**: Exemption rules include `group _ctrld`, so pf only allows bypass for packets from processes with this effective GID. Other processes querying the same exempt IPs are still redirected to ctrld.
4. **Lifecycle**: The group is **never removed** on shutdown or uninstall. It's a harmless system group, and leaving it avoids race conditions during rapid restart cycles. It is recreated (no-op if exists) on every start.
## Anchor Injection into pf.conf
The trickiest part. macOS only processes anchors declared in the active pf ruleset. We must inject our anchor references into the running config.
### What We Do
1. Read `/etc/pf.conf`
2. If our anchor reference already exists, reload as-is
3. Otherwise, inject `rdr-anchor "com.controld.ctrld"` in the translation section and `anchor "com.controld.ctrld"` in the filter section
4. Write to a **temp file** and load with `pfctl -f <tmpfile>`
5. **We never modify `/etc/pf.conf` on disk** — changes are runtime-only and don't survive reboot (ctrld re-injects on every start)
### Injection Logic
Finding the right insertion point requires understanding the existing pf.conf structure. The algorithm:
1. **Scan** for existing `rdr-anchor`/`nat-anchor`/`binat-anchor` lines (translation section) and `anchor` lines (filter section)
2. **Insert `rdr-anchor`**:
- Before the first existing `rdr-anchor` line (if any exist)
- Else before the first `anchor` line (translation must come before filtering)
- Else before the first `pass`/`block` line
- Last resort: append (but this should never happen with a valid pf.conf)
3. **Insert `anchor`**:
- Before the first existing `anchor` line (if any)
- Else before the first `pass`/`block` line
- Last resort: append
### Real-World pf.conf Scenarios
We test against these configurations:
#### Default macOS (Sequoia/Sonoma)
```
scrub-anchor "com.apple/*"
nat-anchor "com.apple/*"
rdr-anchor "com.apple/*"
anchor "com.apple/*"
load anchor "com.apple" from "/etc/pf.anchors/com.apple"
```
Our `rdr-anchor` goes before `rdr-anchor "com.apple/*"`, our `anchor` goes before `anchor "com.apple/*"`.
#### Little Snitch
Adds `rdr-anchor "com.obdev.littlesnitch"` and `anchor "com.obdev.littlesnitch"` in the appropriate sections. Our anchors coexist — pf processes multiple anchors in order.
#### Lulu Firewall (Objective-See)
Adds `anchor "com.objective-see.lulu"`. We insert `rdr-anchor` before it (translation before filtering) and `anchor` before it.
#### Cisco AnyConnect
Adds `nat-anchor "com.cisco.anyconnect"`, `rdr-anchor "com.cisco.anyconnect"`, `anchor "com.cisco.anyconnect"`. Our anchors insert alongside Cisco's in their respective sections.
#### Minimal pf.conf (no anchors)
Just `set skip on lo0` and `pass all`. We insert `rdr-anchor` and `anchor` before the `pass` line.
#### Empty pf.conf
Both anchors appended. This is a degenerate case that shouldn't occur in practice.
## Failure Modes and Safety
### What happens if our injection fails?
- `ensurePFAnchorReference` returns an error, logged as a warning
- ctrld continues running but DNS interception may not work
- The anchor file and rules are cleaned up on shutdown
- **No damage to existing pf config** — we never modify files on disk
### What happens if ctrld crashes (SIGKILL)?
- pf anchor rules persist in kernel memory
- DNS is redirected to 127.0.0.1:53 but nothing is listening → DNS breaks
- On next `ctrld start`, we detect the stale anchor file, flush the anchor, and start fresh
- Without ctrld restart: `sudo pfctl -a com.controld.ctrld -F all` manually clears it
### What if another program flushes all pf rules?
- Our anchor references are removed from the running config
- DNS interception stops (traffic goes direct again — fails open, not closed)
- The periodic watchdog (30s) detects missing rules and restores them
- ctrld continues working for queries sent to 127.0.0.1 directly
### What if another program reloads pf.conf (corrupting translation state)?
Programs like Parallels Desktop reload `/etc/pf.conf` when creating or destroying
virtual network interfaces (bridge100, vmenet0). This can corrupt pf's internal
translation engine — **rdr rules survive in text form but stop evaluating**.
The watchdog's rule-text checks say "intact" while DNS is silently broken.
**Detection:** ctrld detects interface appearance/disappearance in the network
change handler and spawns an asynchronous interception probe monitor:
1. A subprocess sends a DNS query WITHOUT the `_ctrld` group GID, so pf
intercept rules apply to it
2. If ctrld receives the query → pf interception is working
3. If the query times out (1s) → pf translation is broken
4. On failure: `forceReloadPFMainRuleset()` does `pfctl -f -` with the current
running ruleset, resetting pf's translation engine
The monitor probes with exponential backoff (0, 0.5, 1, 2, 4s) to win the race
against async pf reloads. Only one monitor runs at a time (singleton). The
watchdog also runs the probe every 30s as a safety net.
The full pf reload is VPN-safe: it reassembles from `pfctl -sr` + `pfctl -sn`
(the current running state), preserving all existing anchors and rules.
### What if another program adds conflicting rdr rules?
- pf processes anchors in declaration order
- If another program redirects port 53 before our anchor, their redirect wins
- If after, ours wins (first match with `quick` or `rdr pass`)
- Our maximum-weight sublayer approach on Windows (WFP) doesn't apply to pf — pf uses rule ordering, not weights
### What about `set skip on lo0`?
Some pf.conf files include `set skip on lo0` which tells pf to skip ALL processing on loopback. **This would break our approach** since both the `rdr on lo0` and `pass in on lo0` rules would be skipped.
**Mitigation:** the interception probe. `probePFIntercept()` sends a real query from
outside the `_ctrld` group and confirms the listener received the redirect, which cannot
succeed while pf is bypassing loopback — so a skip on `lo0` shows up as a probe failure
and triggers a full reload.
**Not implemented, contrary to earlier versions of this document:** ctrld does *not*
strip `lo0` from `set skip on` directives, and the watchdog does *not* inspect skip
state. Apple's `pfctl` offers no way to read it — `pfctl(8)` accepts `-s` nat, queue,
rules, Anchors, states, Sources, info, References, labels, timeouts, memory, Tables,
osfp, Interfaces, all, with no options or skip modifier — so text-based detection is not
available on macOS.
Adding an explicit check is tracked as follow-up: `pfctl(8)` documents
`-s Interfaces -v` as additionally listing which interfaces have skip rules activated,
which is the query to build on once its output shape is confirmed on a host that has a
skip configured.
## Cleanup
On shutdown (`stopDNSIntercept`):
1. `pfctl -a com.controld.ctrld -F all` — flush all rules from our anchor
2. Remove `/etc/pf.anchors/com.controld.ctrld` anchor file
3. `pfctl -f /etc/pf.conf` — reload original pf.conf, removing our injected anchor references from the running config
This is clean: no files modified on disk, no residual rules.
## Comparison with Other Approaches
| Approach | Intercepts local DNS? | Survives VPN DNS override? | Risk of loops? | Complexity |
|----------|----------------------|---------------------------|----------------|------------|
| `rdr on ! lo0` | ❌ No | Yes | Low | Low |
| `route-to lo0` + `rdr on lo0` | ✅ Yes | Yes | Medium (need exemptions) | Medium |
| `/etc/resolver/` | Partial (per-domain only) | No (VPN can overwrite) | Low | Low |
| `NEDNSProxyProvider` | ✅ Yes | Yes | Low | High (needs app bundle) |
| NRPT (Windows only) | N/A | Partial | Low | Medium |
We chose `route-to + rdr` as the best balance of effectiveness and deployability (no app bundle needed, no kernel extension, works with existing ctrld binary).
## Key pf Nuances Learned
1. **`rdr` doesn't match locally-originated traffic** — this is the biggest gotcha
2. **Rule ordering is enforced** — translation before filtering, always
3. **Anchors must be declared in the main ruleset** — just loading an anchor file isn't enough
4. **`rdr` without `pass`** — redirected packets must go through filter rules so `pass in keep state` can create response state. `rdr pass` alone is insufficient for response delivery.
5. **State handling is nuanced** — route-to uses `keep state` (state is floating). `pass out on lo0` must use `no state` (prevents rdr bypass). `pass in on lo0` uses `keep state` + `reply-to lo0` (creates response state AND forces response through loopback instead of VPN tunnel). Getting any of these wrong breaks either the forward or return path.
6. **`quick` terminates evaluation** — exemption rules must use `quick` and appear before the route-to rule
7. **Piping to `pfctl -f -` can fail** — special characters in pf.conf content cause issues; use temp files
8. **`set skip on lo0` would break us** — but it's not in default macOS pf.conf
9. **`pass out quick` exemptions work with route-to** — they fire in the same phase (filter), so `quick` + rule ordering means exempted packets never hit the route-to rule
10. **pf cannot cross-AF redirect**`rdr on lo0 inet6 ... -> 127.0.0.1` is invalid. IPv6 DNS must be handled by an `[::1]` listener.
11. **`block return` doesn't work for IPv6 DNS** — BSD doesn't deliver ICMPv6 unreachable to unconnected UDP sockets (`sendto`). Apps timeout waiting for a response that never comes.
12. **sendmsg from `::1` to global unicast fails on macOS** — unlike IPv4 where `127.0.0.1` can send to any local address, `::1` cannot send to the machine's own global IPv6 address (`EINVAL`). This is the fundamental asymmetry that makes IPv6 DNS interception infeasible.
13. **`nat on lo0` doesn't fire for `route-to`'d packets** — pf runs translation on the original outbound interface (en0), then skips it on lo0's outbound pass. `rdr` works because lo0 inbound is a genuinely new direction. Any lo0 address (including ULAs) can't route to global unicast — the kernel segregates lo0's routing table.
14. **Raw IPv6 sockets bypass routing validation but pf doesn't match them**`SOCK_RAW` can send from `::1` to global unicast, but pf treats raw socket packets as new connections (not matching rdr state), so reverse-translation doesn't happen. The client sees `::1` as the source, not the original DNS server.
15. **`DIOCNATLOOK` can find the original dest but you can't use it** — The ioctl returns the pre-rdr destination, but `bind()` fails with `EADDRNOTAVAIL` because it's not a local address. macOS IPv6 raw sockets don't support `IPV6_HDRINCL` for source spoofing.
16. **Blocking IPv6 DNS is the pragmatic solution** — macOS automatically retries over IPv4. The ~1s penalty on the first blocked query is negligible compared to the complexity of working around the kernel's IPv6 loopback restrictions.
+61
View File
@@ -0,0 +1,61 @@
# Provisioning failure codes
When ctrld hits a terminal failure during provisioning, it reports the same
stable code on three surfaces:
- **Result file**`provision_result.json` in the ctrld home directory
(next to the persisted internal `ctrld.log`). JSON with `stage`, `code`,
`exit_code`, `message`, and for listener failures a bounded
`detail.attempts` list of `{addr, proto, os_error}`. Written atomically,
removed on the next successful provisioning. Never contains provision
tokens, resolver/device IDs, or configuration contents.
- **Output line** — one fixed-format line on the CLI output:
`provisioning failed: stage=<stage> code=<CODE> (exit <N>)`.
The macOS pkg `postinstall` extracts exactly this line into the installer
log, so MDM consoles see it without any ctrld log configuration.
- **Exit code** — stage-scoped: bootstrap 3039, listener 4049,
service 5059. Unrelated existing contracts are unchanged
(`ctrld status` exits 03; invalid deactivation pin exits 126).
A customer or administrator only needs to report the code (or the whole
output line). The table below is the maintained support mapping; it must
stay in sync with `cmd/cli/provision_result.go` and changes in the same MR.
## Codes
| Code | Stage | Exit | Failure scenario | Next action / evidence |
|---|---|---|---|---|
| `API_UNREACHABLE` | bootstrap | 30 | The Control D API could not be reached or answered with a retryable error (network failure, proxy interference, 5xx, timeout) and retries ran out. The service manager may retry the service later. | Check the device's network path to `api.controld.com` (DNS, proxy, firewall, captive portal). Ask for the result file's `message` and whether other TLS traffic works. |
| `API_REJECTED` | bootstrap | 31 | The API answered and permanently rejected the configuration (4xx other than 408/429): bad or revoked token, malformed request. ctrld exits without burning service-manager restarts because retrying cannot change the answer. | Verify the provision token / org configuration in the Control D dashboard. Re-push after fixing credentials. Evidence: HTTP status in the result file `message`. |
| `API_DEVICE_INVALID` | bootstrap | 32 | The API reports the device/resolver no longer exists (error code 40402). ctrld self-uninstalls its service because the identity is gone server-side. | Confirm the device was deleted or re-provisioned in the dashboard; re-provision with a current token. No local evidence needed beyond the code. |
| `LISTENER_BIND_FAILED` | listener | 41 | No listen address could be bound after all fallbacks (configured address, 0.0.0.0:53, localhost:53, port 5354, random) were exhausted. `detail.attempts` records each tried address with the UDP/TCP OS error, e.g. `address already in use` (another DNS service owns the port) or `can't assign requested address` (address not on any interface). | Read `detail.attempts`: `address already in use` → find the process owning the port (`sudo lsof -i :53 -nP`); `can't assign requested address` → the configured IP is not present on the device. Then fix the conflict or the listener config. |
| `LISTENER_CONFIGURED_ADDR_UNAVAILABLE` | listener | 42 | An explicitly configured listener address could not be bound and configuration checks forbid falling back to another address, or (macOS intercept mode) the required explicit address is unavailable. | The configured `ip:port` in the listener config is wrong for this device or occupied. Verify the address exists on an interface and nothing else binds it; correct the config rather than expecting fallback. |
| `SERVICE_INSTALL_FAILED` | service | 51 | The OS service manager refused to install the service (launchd/systemd/SCM registration failed). | Check OS-level constraints: permissions/elevation, MDM policy blocking daemon installation, corrupted previous install. Evidence: result file `message` (service manager error), plus `launchctl print system/ctrld` / `systemctl status ctrld` / SCM state. |
| `SERVICE_START_FAILED` | service | 52 | The service installed but the service manager could not start it. | Check the service manager's own log for the start error, then the ctrld home dir `ctrld.log`. Often permissions or a binary quarantined by security tooling. |
| `SERVICE_SELFCHECK_FAILED` | service | 53 | The service started but never became healthy: no fresher failure was reported by the daemon, and the post-install DNS self-check failed. The just-installed service is rolled back (uninstalled). If the daemon itself recorded a more specific failure (e.g. a listener code), that code is reported instead of this one. | Ask for the drained service log printed by `ctrld start` and the result file. If the service was running but unreachable, check host firewall rules intercepting DNS to the listener. |
## Reading the result file
macOS and Linux (default service home is `/etc/controld`):
```sh
sudo cat /etc/controld/provision_result.json
```
On Windows the file sits next to `ctrld.exe` in the install directory. A
custom `homedir` config moves it accordingly; routers and mobile use their
platform home directory.
The file sits in the same directory as the persisted internal log
(`ctrld.log`) for the user the service runs as. On a healthy install the
file is absent.
## Rules for maintainers
- Codes are append-only once released. Never rename, renumber, or reuse a
code or exit number; add a new one and note the deprecation here.
- Every code added in `cmd/cli/provision_result.go` needs a row here in the
same MR. Tests enforce the code/stage/exit maps and that this table has
exactly one row per code.
- Detail must stay bounded and free of secrets: the constructor strips the
provision token and cd UID and caps sizes; do not bypass it.
+46
View File
@@ -0,0 +1,46 @@
# Runtime Internal Logging
When no logging is configured (i.e., `log_path` is not set), ctrld automatically enables an internal logging system. This system stores logs in memory to provide troubleshooting information when problems occur.
## Purpose
The runtime internal logging system is designed primarily for **ctrld developers**, not end users. It captures detailed diagnostic information that can be useful for troubleshooting issues when they arise, especially in production environments where explicit logging may not be configured.
## When It's Enabled
Internal logging is automatically enabled when:
- ctrld is running in Control D mode (i.e., `--cd` flag is provided)
- No log file is configured (i.e., `log_path` is empty or not set)
If a log file is explicitly configured via `log_path`, internal logging will **not** be enabled, as the configured log file serves the logging purpose.
## How It Works
The internal logging system:
- Stores logs in **in-memory buffers** (not written to disk)
- Captures logs at **debug level** for normal operations and **warn level** for warnings
- Maintains separate buffers for normal logs and warning logs
- Automatically manages buffer size to prevent unbounded memory growth
- Preserves initialization logs even when buffers overflow
## Configuration
**Important**: The `log_level` configuration option does **not** affect the internal logging system. Internal logging always operates at debug level for normal logs and warn level for warnings, regardless of the `log_level` setting in the configuration file.
The `log_level` setting only affects:
- Console output (when running interactively)
- File-based logging (when `log_path` is configured)
## Accessing Internal Logs
Internal logs can be accessed through the control server API endpoints. This functionality is intended for developers and support personnel who need to diagnose issues.
## Notes
- Internal logging is **not** a replacement for proper log file configuration in production environments
- For production deployments, it is recommended to configure `log_path` to enable persistent file-based logging
- Internal logs are stored in memory and will be lost if the process terminates unexpectedly
- The internal logging system is automatically disabled when explicit logging is configured
+126
View File
@@ -0,0 +1,126 @@
# Username Detection in ctrld
## Overview
The ctrld client needs to detect the primary user of a system for telemetry and configuration purposes. This is particularly challenging in RMM (Remote Monitoring and Management) deployments where traditional session-based detection methods fail.
## The Problem
In traditional desktop environments, username detection is straightforward using environment variables like `$USER`, `$LOGNAME`, or `$SUDO_USER`. However, RMM deployments present unique challenges:
- **No active login session**: RMM agents often run as system services without an associated user session
- **Missing environment variables**: Common user environment variables are not available in service contexts
- **Root/SYSTEM execution**: The ctrld process may run with elevated privileges, masking the actual user
## Solution Approach
ctrld implements a multi-tier, deterministic username detection system through the `DiscoverMainUser()` function with platform-specific implementations:
### Key Principles
1. **Deterministic selection**: No randomness - always returns the same result for the same system state
2. **Priority chain**: Multiple detection methods with clear fallback order
3. **Lowest UID/RID wins**: Among multiple candidates, select the user with the lowest identifier (typically the first user created)
4. **Fast execution**: All operations complete in <100ms using local system resources
5. **Debug logging**: Each decision point logs its rationale for troubleshooting
## Platform-Specific Implementation
### macOS (`discover_user_darwin.go`)
**Detection chain:**
1. **Console owner** (`stat -f %Su /dev/console`) - Most reliable for active GUI sessions
2. **scutil ConsoleUser** - Alternative session detection via System Configuration framework
3. **Directory Services scan** (`dscl . list /Users UniqueID`) - Scan all users with UID ≥ 501, select lowest
**Rationale**: macOS systems typically have a primary user who owns the console. Service contexts can still access device ownership information.
### Linux (`discover_user_linux.go`)
**Detection chain:**
1. **loginctl active users** (`loginctl list-users`) - systemd's session management
2. **Admin user preference** - Parse `/etc/passwd` for UID ≥ 1000, prefer sudo/wheel/admin group members
3. **Lowest UID fallback** - From `/etc/passwd`, select user with UID ≥ 1000 and lowest UID
**Rationale**: Linux systems may have multiple regular users. Prioritize users in administrative groups as they're more likely to be primary system users.
### Windows (`discover_user_windows.go`)
**Detection chain:**
1. **Active console session** (`WTSGetActiveConsoleSessionId` + `WTSQuerySessionInformation`) - Direct Windows API for active user
2. **Registry admin preference** - Scan `HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList`, prefer Administrators group members
3. **Lowest RID fallback** - From ProfileList, select user with RID ≥ 1000 and lowest RID
**Rationale**: Windows has well-defined APIs for session management. Registry ProfileList provides a complete view of all user accounts when no active session exists.
### Other Platforms (`discover_user_others.go`)
Returns `"unknown"` - placeholder for unsupported platforms.
## Implementation Details
### Error Handling
- Individual detection methods log failures at Debug level and continue to next method
- Only final failure (all methods failed) is noteworthy
- Graceful degradation ensures the system continues operating with `"unknown"` user
### Performance Considerations
- Registry/file parsing uses native Go where possible
- External command execution limited to necessary cases
- No network calls or blocking operations
- Timeout context honored for all operations
### Security
- No privilege escalation required
- Read-only operations on system resources
- No user data collected beyond username
- Respects system access controls
## Testing Scenarios
This implementation addresses these common RMM scenarios:
1. **Windows Service context**: No interactive user session, service running as SYSTEM
2. **Linux systemd service**: No login session, running as root daemon
3. **macOS LaunchDaemon**: No GUI user context, running as root
4. **Multi-user systems**: Multiple valid candidates, deterministic selection
5. **Minimalist systems**: Limited user accounts, fallback to available options
## Metadata Submission Strategy
System metadata (OS, chassis, username, domain) is sent to the Control D API via POST `/utility`. To avoid duplicate submissions and minimize EDR-triggering user discovery, ctrld uses a tiered approach:
### When metadata is sent
| Scenario | Metadata sent? | Username included? |
|---|---|---|
| `ctrld start` with `--cd-org` (provisioning via `cdUIDFromProvToken`) | ✅ Full | ✅ Yes |
| `ctrld run` startup (config validation / processCDFlags) | ✅ Lightweight | ❌ No |
| Runtime config reload (`doReloadApiConfig`) | ✅ Lightweight | ❌ No |
| Runtime self-uninstall check | ✅ Lightweight | ❌ No |
| Runtime deactivation pin refresh | ✅ Lightweight | ❌ No |
Username is only collected and sent once — during initial provisioning via `cdUIDFromProvToken()`. All other API calls use `SystemMetadataRuntime()` which omits username discovery entirely.
### Runtime metadata (`SystemMetadataRuntime`)
Runtime API calls (config reload, self-uninstall check, deactivation pin refresh) use `SystemMetadataRuntime()` which includes OS and chassis info but **skips username discovery**. This avoids:
- **EDR false positives**: Repeated user enumeration (registry scans, WTS queries, loginctl calls) can trigger endpoint detection and response alerts
- **Unnecessary work**: Username is unlikely to change while the service is running
## Migration Notes
The previous `currentLoginUser()` function has been replaced by `DiscoverMainUser()` with these changes:
- **Removed dependencies**: No longer uses `logname(1)`, environment variables as primary detection
- **Added platform specificity**: Separate files for each OS with optimized detection logic
- **Improved RMM compatibility**: Designed specifically for service/daemon contexts
- **Maintained compatibility**: Returns same format (string username or "unknown")
## Future Extensions
This architecture allows easy addition of new platforms by creating additional `discover_user_<os>.go` files following the same interface pattern.
+465
View File
@@ -0,0 +1,465 @@
# Windows DNS Intercept — Technical Reference
## Overview
On Windows, DNS intercept mode uses a two-layer architecture:
- **`dns` mode (default)**: NRPT only — graceful DNS routing via the Windows DNS Client service
- **`hard` mode**: NRPT + WFP — full enforcement with kernel-level block filters
`dns` mode avoids ctrld's outbound block filters and therefore degrades more gracefully when owned NRPT is removed. Resolution still depends on the active NRPT target being reachable; an administrator-owned GP catch-all intentionally remains fail-closed if its loopback listener is stopped.
## Architecture: dns vs hard Mode
```
┌─────────────────────────────────────────────────────────────────┐
│ dns mode (NRPT + loopback WFP protect) │
│ │
│ App DNS query → DNS Client service → NRPT lookup │
│ → "." catch-all matches → forward to 127.0.0.1 (ctrld) │
│ │
│ Loopback WFP protect: 4 hard-permit filters (port 53 to │
│ localhost, CLEAR_ACTION_RIGHT) prevent third-party VPN WFP │
│ blocks (e.g., OpenVPN block-outside-dns) from breaking NRPT. │
│ │
│ Owned rule missing → restore; GP missing → owned fallback │
│ GP rule + dead listener remains intentionally fail-closed │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ hard mode (NRPT + WFP) │
│ │
│ App DNS query → DNS Client service → NRPT → 127.0.0.1 (ctrld)│
│ │
│ Bypass attempt (raw 8.8.8.8:53) → WFP BLOCK filter │
│ VPN DNS on private IP → WFP subnet PERMIT filter → allowed │
│ │
│ NRPT route is established before WFP starts │
│ WFP failure rolls back only ctrld-owned NRPT; GP is untouched │
└─────────────────────────────────────────────────────────────────┘
```
## NRPT (Name Resolution Policy Table)
### What It Does
NRPT is a Windows feature (originally for DirectAccess) that tells the DNS Client
service to route queries matching specific namespace patterns to specific DNS servers.
ctrld adds a catch-all rule that routes ALL DNS to `127.0.0.1`:
| Registry Value | Type | Value | Purpose |
|---|---|---|---|
| `Name` | REG_MULTI_SZ | `.` | Namespace (`.` = catch-all) |
| `GenericDNSServers` | REG_SZ | `127.0.0.1` | Target DNS server |
| `ConfigOptions` | REG_DWORD | `0x8` | Standard DNS resolution |
| `Version` | REG_DWORD | `0x2` | NRPT rule version 2 |
| `Comment` | REG_SZ | `` | Empty (matches PowerShell behavior) |
| `DisplayName` | REG_SZ | `` | Empty (matches PowerShell behavior) |
| `IPSECCARestriction` | REG_SZ | `` | Empty (matches PowerShell behavior) |
### Registry Paths — GP vs Local (Critical)
Windows NRPT has two registry paths with **all-or-nothing** precedence:
| Path | Name | Mode |
|---|---|---|
| `HKLM\SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig` | **GP path** | Group Policy mode |
| `HKLM\SYSTEM\CurrentControlSet\Services\Dnscache\Parameters\DnsPolicyConfig` | **Local path** | Local/service store mode |
**Precedence rule**: If ANY rules exist in the GP path (from IT policy, VPN, MDM,
or our own earlier builds), DNS Client enters "GP mode" and **ignores ALL local-path
rules entirely**. This is not per-rule — it's a binary switch.
**Consequence**: On non-domain-joined (WORKGROUP) machines, `RefreshPolicyEx` is
unreliable. If we write to the GP path, DNS Client enters GP mode but the rules
never activate — resulting in `Get-DnsClientNrptPolicy` returning empty even though
`Get-DnsClientNrptRule` shows the rule in registry.
ctrld uses an adaptive strategy (matching [Tailscale's approach](https://github.com/tailscale/tailscale/blob/main/net/dns/nrpt_windows.go)) when it owns NRPT:
1. **Always write to the local path** using a deterministic GUID key name
(`{B2E9A3C1-7F4D-4A8E-9D6B-5C1E0F3A2B8D}`). This is the baseline that works
on all non-domain machines.
2. **Check if other software has GP NRPT rules** (`otherGPRulesExist()`). If
foreign GP rules are present (IT policy, VPN), DNS Client is already in GP mode
and our local rule would be invisible — so we also write to the GP path.
3. **If no foreign GP rules exist**, clean any stale ctrld GP rules and delete
the empty GP parent key. This ensures DNS Client stays in "local mode" where
the local-path rule activates immediately via `paramchange`.
### Adopting an Organization-Owned GP Catch-All
Before applying that ctrld-owned strategy, service startup looks for a non-ctrld GP child with exactly `Name=["."]` and one `GenericDNSServers` value equal to the actual listener IP. The registry match is only an ownership candidate: ctrld sends its unique DNS Client probe before any NRPT write and re-reads the same child afterward.
Intercept state stays unpublished while that happens. Startup publishes nothing until it has fully succeeded, which is why the probe and heal flows take the state as an argument instead of reading the published field — publishing early would expose a half-built `wfpState`, with no engine handle and filter IDs still being assigned, to callers such as the VPN DNS exemption path. The one deliberate exception is hard mode when WFP setup fails while GP-managed NRPT is verified routing: that state is published so the health monitor can keep retrying WFP, and the service start is still reported as failed.
When the rule remains unchanged and the probe arrives, Group Policy owns NRPT:
- the startup adapter reset is skipped;
- no ctrld NRPT key is created;
- `RefreshPolicyEx`, Dnscache `paramchange`, and cache flush are not used for that policy;
- shutdown/uninstall leave the GP child untouched.
A matching GP child that remains present but fails its probe is reported as ineffective and is not rewritten. If the child disappears, ctrld activates its normal owned fallback. A GP catch-all that instead changes to another resolver is reported as a conflict; ctrld does not create a second ambiguous catch-all. If the matching rule later returns, ctrld probes first, removes only its deterministic fallback keys, signals the ownership transition once, and resumes observing Group Policy.
**Deployment ordering:** apply the GPO before starting ctrld to guarantee adapter DNS is never reset. Remove/unlink it before intentionally stopping or uninstalling ctrld. A GP catch-all still targeting loopback with no listener running is a deliberate fail-closed state and will break DNS.
### Reproducing the Empty GP Parent Case
This is a production code reference, so the temporary repro script is not kept in
the repository. For MR !942 review, the test script and exact before/after steps
are posted in the MR discussion. The scenario to compare is:
1. Run the same approved PowerShell repro script against a pre-fix build and this
branch with the same ctrld config.
2. Create an empty GP NRPT parent key while ctrld is running in DNS intercept mode.
3. Confirm pre-fix logs can spend policy refresh/paramchange retries while the GP
parent remains empty.
4. Confirm post-fix logs clean the empty GP parent, send one NRPT-change signal,
and re-probe before normal retries.
### VPN Coexistence
NRPT uses most-specific-match. VPN NRPT rules for specific domains (e.g.,
`*.corp.local``10.20.30.1`) take priority over ctrld's `.` catch-all.
This means VPN split DNS works naturally — VPN-specific domains go to VPN DNS,
everything else goes to ctrld. No exemptions or special handling needed.
### DNS Client Notification
After writing NRPT rules, DNS Client must be notified to reload:
1. **`paramchange`**: `sc control dnscache paramchange` — signals DNS Client to
re-read configuration. Works for local-path rules on most machines.
2. **`RefreshPolicyEx`**: `RefreshPolicyEx(bMachine=TRUE, dwOptions=RP_FORCE)` from
`userenv.dll` — triggers GP refresh for GP-path rules. Unreliable on non-domain
machines (WORKGROUP). Fallback: `gpupdate /target:computer /force`.
3. **DNS cache flush**: `DnsFlushResolverCache` from `dnsapi.dll` or `ipconfig /flushdns`
— clears stale cached results from before NRPT was active.
### DNS Cache Flush
After NRPT changes, stale DNS cache entries could bypass the new routing. ctrld flushes:
1. **Primary**: `DnsFlushResolverCache` from `dnsapi.dll`
2. **Fallback**: `ipconfig /flushdns` (subprocess)
### Known Limitation: nslookup
`nslookup.exe` implements its own DNS resolver and does NOT use the Windows DNS Client
service. It ignores NRPT entirely. Use `Resolve-DnsName` (PowerShell) or `ping` to
verify DNS resolution through NRPT. This is a well-known Windows behavior.
## WFP (Windows Filtering Platform) — hard Mode Only
### Filter Stack
```
┌─────────────────────────────────────────────────────────────────┐
│ Sublayer: "ctrld DNS Intercept" (weight 0xFFFF — max priority) │
│ │
│ ┌─ Permit Filters (weight 10) ─────────────────────────────┐ │
│ │ • IPv4/UDP to 127.0.0.1:53 → PERMIT │ │
│ │ • IPv4/TCP to 127.0.0.1:53 → PERMIT │ │
│ │ • IPv6/UDP to ::1:53 → PERMIT │ │
│ │ • IPv6/TCP to ::1:53 → PERMIT │ │
│ │ • RFC1918 + CGNAT subnets:53 → PERMIT (VPN DNS) │ │
│ │ • VPN DNS exemptions (dynamic) → PERMIT │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
│ ┌─ Block Filters (weight 1) ───────────────────────────────┐ │
│ │ • All IPv4/UDP to *:53 → BLOCK │ │
│ │ • All IPv4/TCP to *:53 → BLOCK │ │
│ │ • All IPv6/UDP to *:53 → BLOCK │ │
│ │ • All IPv6/TCP to *:53 → BLOCK │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
│ Filter evaluation: higher weight wins → permits checked first │
└─────────────────────────────────────────────────────────────────┘
```
### Why WFP Can't Work Alone
WFP operates at the connection authorization layer (`FWPM_LAYER_ALE_AUTH_CONNECT`).
It can only **block** or **permit** connections — it **cannot redirect** them.
Redirection requires kernel-mode callout drivers (`FwpsCalloutRegister` in
`fwpkclnt.lib`) using `FWPM_LAYER_ALE_CONNECT_REDIRECT_V4/V6`, which are not
accessible from userspace.
Without NRPT, WFP blocks outbound DNS but doesn't tell applications where to send
queries instead — they just see DNS failures. This is why `hard` mode requires NRPT
to be active first, and why WFP is rolled back if NRPT setup fails.
### Sublayer Priority
Weight `0xFFFF` (maximum) ensures ctrld's filters take priority over any other WFP
sublayers from VPN software, endpoint security, or Windows Defender Firewall.
### RFC1918 + CGNAT Subnet Permits
Static permit filters for private IP ranges (10.0.0.0/8, 172.16.0.0/12,
192.168.0.0/16, 100.64.0.0/10) allow VPN DNS servers on private IPs to work
without dynamic per-server exemptions. This covers Tailscale MagicDNS
(100.100.100.100), corporate VPN DNS (10.x.x.x), and similar.
### VPN DNS Exemption Updates
When `vpnDNSManager.Refresh()` discovers VPN DNS servers on public IPs:
1. Delete all existing VPN permit filters (by stored IDs)
2. For each VPN DNS server IP:
- IPv4: `addWFPPermitIPFilter()` on `ALE_AUTH_CONNECT_V4`
- IPv6: `addWFPPermitIPv6Filter()` on `ALE_AUTH_CONNECT_V6`
- Both UDP and TCP for each IP
3. Store new filter IDs for next cleanup cycle
**In `dns` mode, VPN DNS exemptions are skipped** — there are no ctrld WFP block
filters to exempt from. The loopback WFP protect filters only permit localhost
DNS; VPN DNS traffic goes through the tunnel interface and is already permitted
by the VPN's own WFP rules.
### Session Lifecycle
**Startup (hard mode):**
```
1. Adopt a proven matching GP catch-all, or install ctrld-owned NRPT
2. FwpmEngineOpen0() with RPC_C_AUTHN_DEFAULT (0xFFFFFFFF)
3. Delete stale sublayer (crash recovery)
4. FwpmSubLayerAdd0() — weight 0xFFFF
5. Add 4 localhost permit filters
6. Add 4 block filters
7. Add RFC1918 + CGNAT subnet permits
8. Start ownership-aware NRPT/WFP health monitor
```
**Startup (dns mode):**
```
1. Adopt a proven matching GP catch-all, or install ctrld-owned NRPT
2. Activate loopback WFP protect (4 hard-permit filters for localhost DNS)
3. Start ownership-aware NRPT health monitor
```
**Shutdown:**
```
1. Stop NRPT health monitor
2. Remove + signal only ctrld-owned NRPT; leave GP-managed policy untouched
3. Clean up ctrld WFP filters, sublayer, and engine session
```
**Crash Recovery:**
On startup, `FwpmSubLayerDeleteByKey0` removes any stale sublayer from a previous
unclean shutdown, including all its child filters (deterministic GUID ensures we
only clean up our own).
## NRPT Probe and Auto-Heal
### The Problem: Async GP Refresh Race
`RefreshPolicyEx` triggers a Group Policy refresh but returns immediately — it does
NOT wait for the DNS Client service to actually reload NRPT from the registry. On
cold machines (first boot, fresh install, long sleep), the DNS Client may take
several seconds to process the policy refresh. During this window, NRPT rules exist
in the registry but the DNS Client hasn't loaded them — queries bypass ctrld.
### The Solution: Active Probing
After writing NRPT to the registry, ctrld sends a probe DNS query through the
Windows DNS Client path to verify NRPT is actually working:
1. Generate a unique probe domain: `_nrpt-probe-<hex>.nrpt-probe.ctrld.test`
2. Send it via Go's `net.Resolver` (calls `GetAddrInfoW` → DNS Client → NRPT)
3. If NRPT is active, DNS Client routes it to 127.0.0.1 → ctrld receives it
4. ctrld's DNS handler recognizes the probe prefix and signals success
5. If the probe times out (2s), NRPT isn't loaded yet → retry with remediation
### Startup Probes
For a matching GP candidate, startup blocks for one 2-second probe before any NRPT mutation and then re-reads the same child. A received query plus the unchanged rule proves both routing and external ownership. If that first probe fails, ctrld installs its normal WFP protection and performs one more ownership-safe probe before advertising startup readiness.
ctrld-owned NRPT retains the asynchronous sequence:
```
Immediate probe
├─ Success → "NRPT verified working", done
└─ Timeout
├─ Empty GP parent → clean once, signal once, re-probe
└─ Otherwise → bounded 1s/2s/4s signal + probe retries
└─ Still failing → two-phase remove/signal/re-add/final probe
```
### GP-Managed Probe Failure
A matching GP child still owns Windows' effective NRPT store even when its probe fails. Writing a local rule cannot override that precedence, and rewriting the GP child would violate administrator ownership. ctrld therefore retries only its loopback WFP permit protection, reports the ineffective external policy, and leaves NRPT registry values and policy signals untouched.
### Health Monitor Integration
The 30s periodic health monitor now does actual probing, not just registry checks:
```
Every 30s:
├─ GP-managed owner
│ ├─ Matching child + probe success → observe only
│ ├─ Matching child + probe failure → WFP-only retry; no NRPT mutation
│ └─ Matching child gone → activate ctrld-owned fallback + verify
├─ ctrld-owned owner
│ ├─ Working matching GP child returns → remove only ctrld keys; adopt GP
│ ├─ ctrld key missing → restore + signal + verify
│ └─ ctrld key present → probe; run owned heal sequence on failure
└─ (hard mode) Check WFP sublayer; full intercept restart if missing
```
**Singleton guard:** Only one asynchronous probe-and-heal sequence runs at a time (atomic bool). Startup's GP-candidate probe completes before the health monitor starts; direct periodic probes finish before they schedule a heal sequence.
**Why periodic, not just network-event?** VPN software or Group Policy updates can
clear NRPT at any time, not just during network changes. A 30s periodic check ensures
recovery within a bounded window.
**Hard mode safety:** The health monitor verifies NRPT before checking WFP. If NRPT
is gone, it's restored first. WFP is never running without NRPT — this prevents
DNS blackholes where WFP blocks everything but NRPT isn't routing to ctrld.
## DNS Flow Diagrams
### Normal Resolution (both modes)
```
App → DNS Client → NRPT lookup → "." matches → 127.0.0.1 → ctrld
→ Control D DoH (port 443, not affected by WFP port-53 rules)
→ response flows back
```
### VPN Split DNS (both modes)
```
App → DNS Client → NRPT lookup:
VPN domain (*.corp.local) → VPN's NRPT rule wins → VPN DNS server
Everything else → ctrld's "." catch-all → 127.0.0.1 → ctrld
→ VPN domain match → forward to VPN DNS (port 53)
→ (hard mode: WFP subnet permit allows private IP DNS)
```
### Bypass Attempt (hard mode only)
```
App → raw socket to 8.8.8.8:53 → WFP ALE_AUTH_CONNECT → BLOCK
```
In `dns` mode, this query would succeed (no WFP) — the tradeoff for never
breaking DNS.
## Key Differences from macOS (pf)
| Aspect | macOS (pf) | Windows dns mode | Windows hard mode |
|--------|-----------|------------------|-------------------|
| **Routing** | `rdr` redirect | NRPT policy | NRPT policy |
| **Enforcement** | `route-to` + block rules | Loopback WFP protect | WFP block filters |
| **Can break DNS?** | Yes (pf corruption) | No | Yes (if NRPT lost) |
| **VPN coexistence** | Watchdog + stabilization | NRPT + loopback hard-permit | Same + WFP permits |
| **Bypass protection** | pf catches all packets | None | WFP catches all connections |
| **Recovery** | Probe + auto-heal | Health monitor re-adds | Full restart on sublayer loss |
## WFP API Notes
### Struct Layouts
WFP C API structures are manually defined in Go (`golang.org/x/sys/windows` doesn't
include WFP types). Field alignment must match the C ABI exactly — any mismatch
causes access violations or silent corruption.
### FWP_DATA_TYPE Enum
```
FWP_EMPTY = 0
FWP_UINT8 = 1
FWP_UINT16 = 2
FWP_UINT32 = 3
FWP_UINT64 = 4
...
```
**⚠️** Some documentation examples incorrectly start at 1. The enum starts at 0
(`FWP_EMPTY`), making all subsequent values offset by 1 from what you might expect.
### GC Safety
When passing Go heap objects to WFP syscalls via `unsafe.Pointer`, use
`runtime.KeepAlive()` to prevent garbage collection during the call:
```go
conditions := make([]fwpmFilterCondition0, 3)
filter.filterCondition = &conditions[0]
r1, _, _ := procFwpmFilterAdd0.Call(...)
runtime.KeepAlive(conditions)
```
### Authentication
`FwpmEngineOpen0` requires `RPC_C_AUTHN_DEFAULT` (0xFFFFFFFF) for the authentication
service parameter. `RPC_C_AUTHN_NONE` (0) returns `ERROR_NOT_SUPPORTED` on some
configurations (e.g., Parallels VMs).
### Elevation
WFP requires admin/SYSTEM privileges. `FwpmEngineOpen0` fails with HRESULT 0x32
when run non-elevated. Services running as SYSTEM have this automatically.
## Debugging
### Check NRPT Rules
```powershell
# PowerShell — show active NRPT rules
Get-DnsClientNrptRule
# Check registry directly
Get-ChildItem "HKLM:\SOFTWARE\Policies\Microsoft\Windows NT\DNSClient\DnsPolicyConfig"
```
### Check WFP Filters (hard mode)
```powershell
# Show all WFP filters (requires admin) — output is XML
netsh wfp show filters
# Search for ctrld's filters
Select-String "ctrld" filters.xml
```
### Verify DNS Resolution
```powershell
# Use Resolve-DnsName, NOT nslookup (nslookup bypasses NRPT)
Resolve-DnsName example.com
ping example.com
# If you must use nslookup, specify localhost:
nslookup example.com 127.0.0.1
# Force GP refresh (if NRPT not loading)
gpupdate /target:computer /force
# Verify service registration
sc qc ctrld
```
### Service Verification
After install, verify the Windows service is correctly registered:
```powershell
# Check binary path and start type
sc qc ctrld
# Should show:
# BINARY_PATH_NAME: "C:\...\ctrld.exe" run --cd xxxxx --intercept-mode dns
# START_TYPE: AUTO_START
```
## Related
- [DNS Intercept Mode Overview](dns-intercept-mode.md) — cross-platform documentation
- [pf DNS Intercept](pf-dns-intercept.md) — macOS technical reference
- [Microsoft WFP Documentation](https://docs.microsoft.com/en-us/windows/win32/fwp/windows-filtering-platform-start-page)
- [Microsoft NRPT Documentation](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn593632(v=ws.11))
+22 -9
View File
@@ -25,6 +25,16 @@ const (
dohOsHeader = "x-cd-os"
dohClientIDPrefHeader = "x-cd-cpref"
headerApplicationDNS = "application/dns-message"
// dohMaxResponseSize caps the response body read from a DoH/DoH3
// upstream. A DNS message is bounded by the protocol's 16-bit length
// field; anything larger cannot be a valid response. The cap stops a
// malicious or compromised upstream from driving ctrld into unbounded
// memory growth via io.ReadAll on attacker-controlled bytes.
dohMaxResponseSize = dns.MaxMsgSize
// dohMaxErrorBodySize bounds how much of a non-200 response body is
// read for inclusion in the returned error.
dohMaxErrorBodySize = 1024
)
// EncodeOsNameMap provides mapping from OS name to a shorter string, used for encoding x-cd-os value.
@@ -85,6 +95,10 @@ type dohResolver struct {
// Resolve performs DNS query with given DNS message using DOH protocol.
func (r *dohResolver) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, error) {
if err := validateMsg(msg); err != nil {
return nil, err
}
data, err := msg.Pack()
if err != nil {
return nil, err
@@ -122,22 +136,21 @@ func (r *dohResolver) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, erro
}
if err != nil {
err = wrapUrlError(err)
if r.isDoH3 {
if closer, ok := c.Transport.(io.Closer); ok {
closer.Close()
}
}
return nil, fmt.Errorf("could not perform request: %w", err)
}
defer resp.Body.Close()
buf, err := io.ReadAll(resp.Body)
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(io.LimitReader(resp.Body, dohMaxErrorBodySize))
return nil, fmt.Errorf("wrong response from DOH server, got: %s, status: %d", string(body), resp.StatusCode)
}
buf, err := io.ReadAll(io.LimitReader(resp.Body, dohMaxResponseSize+1))
if err != nil {
return nil, fmt.Errorf("could not read message from response: %w", err)
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("wrong response from DOH server, got: %s, status: %d", string(buf), resp.StatusCode)
if len(buf) > dohMaxResponseSize {
return nil, fmt.Errorf("DoH response exceeds %d-byte maximum DNS message size", dohMaxResponseSize)
}
answer := new(dns.Msg)
+214
View File
@@ -196,6 +196,7 @@ func testTLSServer(t *testing.T, handler http.Handler) (*httptest.Server, *x509.
server := httptest.NewUnstartedServer(handler)
server.TLS = &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
MinVersion: tls.VersionTLS12,
}
server.StartTLS()
@@ -232,6 +233,7 @@ func newTestHTTP3Server(t *testing.T, handler http.Handler) *testHTTP3Server {
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"h3"}, // HTTP/3 protocol identifier
MinVersion: tls.VersionTLS12,
}
// Create HTTP/3 server
@@ -264,3 +266,215 @@ func newTestHTTP3Server(t *testing.T, handler http.Handler) *testHTTP3Server {
return h3Server
}
// blockingBodyHandler writes exactly nbytes of body with the given status,
// flushes them, then blocks until release is closed WITHOUT ever returning.
// Because the handler does not return, the response stream is never terminated
// (no EOF/FIN). A client that stops after a bounded prefix therefore completes,
// while a client that reads to EOF blocks. Tests set nbytes to the exact read
// cap so the client consumes the whole written body (no half-written frame is
// left blocking on flow control) yet still never sees EOF.
func blockingBodyHandler(status, nbytes int, release <-chan struct{}) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", headerApplicationDNS)
w.WriteHeader(status)
if _, err := w.Write(make([]byte, nbytes)); err != nil {
return
}
if f, ok := w.(http.Flusher); ok {
f.Flush()
}
<-release
}
}
// requireBoundedResolve asserts that r.Resolve returns the expected size/status
// error while the server is still withholding EOF (the handler is blocked in
// blockingBodyHandler). Returning under those conditions proves ctrld read only
// a bounded prefix of the body: a resolver that instead read to EOF would block
// on the withheld stream and trip the deadline. This is the deterministic
// regression guard for the issue-312 OOM protections, replacing the earlier
// flaky server-side byte counter (issue-561).
func requireBoundedResolve(t *testing.T, r Resolver, msg *dns.Msg, wantErrSubstr string) {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
type result struct {
answer *dns.Msg
err error
}
done := make(chan result, 1)
go func() {
answer, err := r.Resolve(ctx, msg)
done <- result{answer, err}
}()
select {
case res := <-done:
if res.err == nil {
t.Fatalf("Resolve unexpectedly succeeded; answer=%v", res.answer)
}
if !strings.Contains(res.err.Error(), wantErrSubstr) {
t.Fatalf("error %q does not contain %q", res.err, wantErrSubstr)
}
if res.answer != nil {
t.Fatalf("Resolve returned non-nil answer alongside error: %v", res.answer)
}
case <-time.After(5 * time.Second):
t.Fatal("Resolve did not return while the server withheld EOF: the body is being read to EOF instead of a bounded prefix (issue-312 OOM protection missing)")
}
}
// dohUpstreamForTLSServer wires an UpstreamConfig at a local httptest TLS
// server, trusting its self-signed certificate. BootstrapIP is set so no
// real DNS lookup runs.
func dohUpstreamForTLSServer(t *testing.T, srv *httptest.Server) *UpstreamConfig {
t.Helper()
pool := x509.NewCertPool()
pool.AddCert(srv.Certificate())
u, err := url.Parse(srv.URL)
if err != nil {
t.Fatalf("parse server URL: %v", err)
}
uc := &UpstreamConfig{
Name: "doh-oversize",
Type: ResolverTypeDOH,
Endpoint: srv.URL + "/dns-query",
BootstrapIP: u.Hostname(),
Timeout: 2000,
}
uc.SetCertPool(pool)
uc.Init()
return uc
}
// doh3UpstreamForAddr wires an UpstreamConfig at a local HTTP/3 server,
// trusting its self-signed certificate.
func doh3UpstreamForAddr(t *testing.T, addr string, cert *x509.Certificate) *UpstreamConfig {
t.Helper()
pool := x509.NewCertPool()
pool.AddCert(cert)
host, _, err := net.SplitHostPort(addr)
if err != nil {
t.Fatalf("split host/port %q: %v", addr, err)
}
uc := &UpstreamConfig{
Name: "doh3-oversize",
Type: ResolverTypeDOH3,
Endpoint: "h3://" + addr + "/dns-query",
BootstrapIP: host,
Timeout: 5000,
}
uc.SetCertPool(pool)
uc.Init()
return uc
}
// TestDoHResolve_OversizedBody_Rejected locks in the fix for
// github.com/Control-D-Inc/ctrld/issues/312: a malicious DoH upstream
// returning a body larger than the DNS protocol allows must be rejected
// with an explicit size error rather than buffered into ctrld memory.
func TestDoHResolve_OversizedBody_Rejected(t *testing.T) {
// Write exactly the LimitReader cap, then withhold EOF. ctrld's bounded
// read (io.LimitReader of dohMaxResponseSize+1) returns after this prefix;
// an unbounded read would block on the missing EOF and trip the deadline.
release := make(chan struct{})
defer close(release)
srv := httptest.NewUnstartedServer(blockingBodyHandler(http.StatusOK, dohMaxResponseSize+1, release))
testCert := generateTestCertificate(t)
srv.TLS = &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"h2", "http/1.1"},
MinVersion: tls.VersionTLS12,
}
srv.StartTLS()
t.Cleanup(srv.Close)
uc := dohUpstreamForTLSServer(t, srv)
r, err := NewResolver(uc)
if err != nil {
t.Fatalf("NewResolver: %v", err)
}
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
requireBoundedResolve(t, r, msg, "maximum DNS message size")
}
// TestDoHResolve_NonOKStatus_BoundedErrorBody locks in that a non-200
// response with a huge body does not pull the body fully into ctrld
// memory just to format an error string.
func TestDoHResolve_NonOKStatus_BoundedErrorBody(t *testing.T) {
// Same synchronization as the oversized-body test, but at the error-body
// cap: the non-200 path reads through an io.LimitReader of
// dohMaxErrorBodySize, so it must return after this prefix without EOF.
release := make(chan struct{})
defer close(release)
srv := httptest.NewUnstartedServer(blockingBodyHandler(http.StatusBadGateway, dohMaxErrorBodySize, release))
testCert := generateTestCertificate(t)
srv.TLS = &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"h2", "http/1.1"},
MinVersion: tls.VersionTLS12,
}
srv.StartTLS()
t.Cleanup(srv.Close)
uc := dohUpstreamForTLSServer(t, srv)
r, err := NewResolver(uc)
if err != nil {
t.Fatalf("NewResolver: %v", err)
}
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
requireBoundedResolve(t, r, msg, "status: 502")
}
// TestDoHResolve_OversizedBody_DoH3 mirrors the DoH oversized-body check
// on the HTTP/3 transport, since github-312 specifically reproduced the
// OOM via DoH3.
func TestDoHResolve_OversizedBody_DoH3(t *testing.T) {
release := make(chan struct{})
defer close(release)
testCert := generateTestCertificate(t)
udpConn, err := net.ListenUDP("udp", &net.UDPAddr{IP: net.ParseIP("127.0.0.1"), Port: 0})
if err != nil {
t.Fatalf("udp listen: %v", err)
}
h3 := &http3.Server{
Handler: blockingBodyHandler(http.StatusOK, dohMaxResponseSize+1, release),
TLSConfig: &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"h3"},
MinVersion: tls.VersionTLS12,
},
}
go func() {
if err := h3.Serve(udpConn); err != nil && !errors.Is(err, http.ErrServerClosed) {
t.Logf("h3 server: %v", err)
}
}()
t.Cleanup(func() {
_ = h3.Close()
_ = udpConn.Close()
})
time.Sleep(100 * time.Millisecond)
uc := doh3UpstreamForAddr(t, udpConn.LocalAddr().String(), testCert.cert)
r, err := NewResolver(uc)
if err != nil {
t.Fatalf("NewResolver: %v", err)
}
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
requireBoundedResolve(t, r, msg, "maximum DNS message size")
}
+323 -37
View File
@@ -5,103 +5,389 @@ package ctrld
import (
"context"
"crypto/tls"
"errors"
"fmt"
"io"
"net"
"runtime"
"sync"
"time"
"github.com/miekg/dns"
"github.com/quic-go/quic-go"
)
// doqMaxResponseSize caps the bytes read from a DoQ stream: a 2-byte
// length prefix plus a DNS message bounded by dns.MaxMsgSize. Anything
// larger cannot be a valid response and is rejected before buffering more
// data from the upstream.
const doqMaxResponseSize = 2 + dns.MaxMsgSize
type doqResolver struct {
uc *UpstreamConfig
}
func (r *doqResolver) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, error) {
endpoint := r.uc.Endpoint
tlsConfig := &tls.Config{NextProtos: []string{"doq"}}
ip := r.uc.BootstrapIP
if ip == "" {
dnsTyp := uint16(0)
if msg != nil && len(msg.Question) > 0 {
dnsTyp = msg.Question[0].Qtype
}
ip = r.uc.bootstrapIPForDNSType(dnsTyp)
if err := validateMsg(msg); err != nil {
return nil, err
}
tlsConfig.ServerName = r.uc.Domain
_, port, _ := net.SplitHostPort(endpoint)
endpoint = net.JoinHostPort(ip, port)
return resolve(ctx, msg, endpoint, tlsConfig)
// Get the appropriate connection pool based on DNS type and IP stack
dnsTyp := uint16(0)
if msg != nil && len(msg.Question) > 0 {
dnsTyp = msg.Question[0].Qtype
}
pool := r.uc.doqTransport(dnsTyp)
if pool == nil {
return nil, errors.New("DoQ connection pool is not available")
}
return pool.Resolve(ctx, msg)
}
func resolve(ctx context.Context, msg *dns.Msg, endpoint string, tlsConfig *tls.Config) (*dns.Msg, error) {
// DoQ quic-go server returns io.EOF error after running for a long time,
// even for a good stream. So retrying the query for 5 times before giving up.
for i := 0; i < 5; i++ {
answer, err := doResolve(ctx, msg, endpoint, tlsConfig)
const doqPoolSize = 16
// doqConnPool manages a pool of QUIC connections for DoQ queries using a buffered channel.
// A single quic.Transport (and its UDP socket) is shared by every connection in the pool,
// so the OS socket lifecycle is tied to the pool rather than to each dial. Without this
// ownership model, a strict DoQ upstream that triggers reconnect churn would leak one
// caller-owned UDP socket per dial — see github.com/Control-D-Inc/ctrld/issues/309.
type doqConnPool struct {
uc *UpstreamConfig
addrs []string
port string
tlsConfig *tls.Config
quicConfig *quic.Config
conns chan *doqConn
transportMu sync.Mutex
transport *quic.Transport
transportConn *net.UDPConn
transportErr error
transportInit bool
closed bool
}
type doqConn struct {
conn *quic.Conn
}
func newDOQConnPool(uc *UpstreamConfig, addrs []string) *doqConnPool {
_, port, _ := net.SplitHostPort(uc.Endpoint)
if port == "" {
port = "853"
}
tlsConfig := &tls.Config{
NextProtos: []string{"doq"},
RootCAs: uc.certPool,
ServerName: uc.Domain,
MinVersion: tls.VersionTLS12,
}
quicConfig := &quic.Config{
KeepAlivePeriod: 15 * time.Second,
}
pool := &doqConnPool{
uc: uc,
addrs: addrs,
port: port,
tlsConfig: tlsConfig,
quicConfig: quicConfig,
conns: make(chan *doqConn, doqPoolSize),
}
// Use SetFinalizer here because we need to call a method on the pool itself.
// AddCleanup would require passing the pool as arg (which panics) or capturing
// it in a closure (which prevents GC). SetFinalizer is appropriate for this case.
runtime.SetFinalizer(pool, func(p *doqConnPool) {
p.CloseIdleConnections()
})
return pool
}
// Resolve performs a DNS query using a pooled QUIC connection.
func (p *doqConnPool) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, error) {
// Retry logic for transient errors: io.EOF (connection reset),
// IdleTimeoutError (stale pooled connection timed out), and
// StreamLimitReachedError (stream credit exhausted before server MAX_STREAMS arrived).
for range 5 {
answer, err := p.doResolve(ctx, msg)
if err == io.EOF {
continue
}
var idleErr *quic.IdleTimeoutError
if errors.As(err, &idleErr) {
continue
}
var streamLimitErr quic.StreamLimitReachedError
if errors.As(err, &streamLimitErr) {
continue
}
if err != nil {
return nil, wrapCertificateVerificationError(err)
}
return answer, nil
}
return nil, &quic.ApplicationError{ErrorCode: quic.ApplicationErrorCode(quic.InternalError), ErrorMessage: quic.InternalError.Message()}
return nil, &quic.ApplicationError{
ErrorCode: quic.ApplicationErrorCode(quic.InternalError),
ErrorMessage: quic.InternalError.Message(),
}
}
func doResolve(ctx context.Context, msg *dns.Msg, endpoint string, tlsConfig *tls.Config) (*dns.Msg, error) {
session, err := quic.DialAddr(ctx, endpoint, tlsConfig, nil)
func (p *doqConnPool) doResolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, error) {
conn, err := p.getConn(ctx)
if err != nil {
return nil, err
}
defer session.CloseWithError(quic.ApplicationErrorCode(quic.NoError), "")
// Pack the DNS message
msgBytes, err := msg.Pack()
if err != nil {
p.putConn(conn, false)
return nil, err
}
stream, err := session.OpenStream()
if err != nil {
return nil, err
}
// Ensure the context has a deadline before calling OpenStreamSync, which
// blocks until the server sends a MAX_STREAMS update. Without a deadline the
// call could block indefinitely when the server never sends the update.
deadline, ok := ctx.Deadline()
if !ok {
deadline = time.Now().Add(5 * time.Second)
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, 5*time.Second)
defer cancel()
deadline, _ = ctx.Deadline()
}
// OpenStreamSync blocks until the server's MAX_STREAMS credit arrives,
// avoiding the StreamLimitReachedError race that OpenStream (non-blocking)
// triggers when the credit replenishment frame is still in flight.
stream, err := conn.OpenStreamSync(ctx)
if err != nil {
p.putConn(conn, false)
return nil, err
}
_ = stream.SetDeadline(deadline)
// Write message length (2 bytes) followed by message
var msgLen = uint16(len(msgBytes))
var msgLenBytes = []byte{byte(msgLen >> 8), byte(msgLen & 0xFF)}
if _, err := stream.Write(msgLenBytes); err != nil {
stream.Close()
p.putConn(conn, false)
return nil, err
}
if _, err := stream.Write(msgBytes); err != nil {
stream.Close()
p.putConn(conn, false)
return nil, err
}
buf, err := io.ReadAll(stream)
// RFC 9250 section 4.2 requires the client to indicate end-of-request by
// closing the send side of the stream (STREAM FIN). Servers may defer
// processing until FIN arrives, so the close must happen before reading.
// Stream.Close closes only the send direction; the receive direction
// remains open for the response.
if err := stream.Close(); err != nil {
p.putConn(conn, false)
return nil, err
}
// A DoQ response is a 2-byte length prefix followed by a DNS message.
// The DNS message is bounded by the protocol at dns.MaxMsgSize, so a
// well-formed response is at most doqMaxResponseSize bytes. Read one
// byte past that cap to distinguish "at limit" from "over limit" and
// reject oversized responses before they can drive memory growth from
// a malicious or compromised upstream.
buf, err := io.ReadAll(io.LimitReader(stream, doqMaxResponseSize+1))
if err != nil {
p.putConn(conn, false)
return nil, err
}
_ = stream.Close()
// io.ReadAll hide the io.EOF error returned by quic-go server.
// Once we figure out why quic-go server sends io.EOF after running
// for a long time, we can have a better way to handle this. For now,
// make sure io.EOF error returned, so the caller can handle it cleanly.
// io.ReadAll hides io.EOF error, so check for empty buffer.
if len(buf) == 0 {
p.putConn(conn, false)
return nil, io.EOF
}
if len(buf) > doqMaxResponseSize {
p.putConn(conn, false)
return nil, fmt.Errorf("DoQ response exceeds %d-byte maximum", doqMaxResponseSize)
}
// RFC 9250: each DoQ DNS message is encoded as a 2-octet length field
// followed by the DNS message. Reject responses that are shorter than
// the prefix or whose prefix declares more bytes than were received,
// and retire the misbehaving connection. Without this guard, buf[2:]
// would panic when len(buf) < 2.
if len(buf) < 2 {
p.putConn(conn, false)
return nil, fmt.Errorf("malformed DoQ response: %d byte(s), need >= 2 for length prefix", len(buf))
}
respLen := int(buf[0])<<8 | int(buf[1])
if 2+respLen > len(buf) {
p.putConn(conn, false)
return nil, fmt.Errorf("malformed DoQ response: length prefix %d exceeds payload %d", respLen, len(buf)-2)
}
p.putConn(conn, true)
// Unpack DNS response (skip 2-byte length prefix).
answer := new(dns.Msg)
if err := answer.Unpack(buf[2:]); err != nil {
if err := answer.Unpack(buf[2 : 2+respLen]); err != nil {
return nil, err
}
answer.SetReply(msg)
// RFC 9250 section 4.2.1 requires the DNS Message ID to be 0 on the wire,
// so restore the downstream transaction ID for the client. Do NOT use
// SetReply here: it rewrites the RCODE to NOERROR and overwrites the
// Question with the request's, which would mask upstream failures from the
// failover logic (a SERVFAIL would look like success) and let a
// wrong-question answer pass validation and poison the cache. Preserve the
// upstream RCODE, Question, and answer sections untouched so the proxy can
// evaluate them. See github.com/Control-D-Inc/ctrld/issues/322.
answer.Id = msg.Id
return answer, nil
}
// getConn gets a QUIC connection from the pool or creates a new one.
// A connection is taken from the channel while in use; putConn returns it.
func (p *doqConnPool) getConn(ctx context.Context) (*quic.Conn, error) {
for {
select {
case dc := <-p.conns:
if dc.conn != nil && dc.conn.Context().Err() == nil {
return dc.conn, nil
}
if dc.conn != nil {
dc.conn.CloseWithError(quic.ApplicationErrorCode(quic.NoError), "")
}
default:
_, conn, err := p.dialConn(ctx)
if err != nil {
return nil, err
}
return conn, nil
}
}
}
// putConn returns a connection to the pool for reuse by other goroutines.
func (p *doqConnPool) putConn(conn *quic.Conn, isGood bool) {
if !isGood || conn == nil || conn.Context().Err() != nil {
if conn != nil {
conn.CloseWithError(quic.ApplicationErrorCode(quic.NoError), "")
}
return
}
dc := &doqConn{conn: conn}
select {
case p.conns <- dc:
default:
// Channel full, close the connection
dc.conn.CloseWithError(quic.ApplicationErrorCode(quic.NoError), "")
}
}
// dialConn creates a new QUIC connection using parallel dialing like DoH3.
// All connections from the pool multiplex on a single pool-owned UDP socket,
// so reconnect churn cannot grow the host's FD count.
func (p *doqConnPool) dialConn(ctx context.Context) (string, *quic.Conn, error) {
logger := ProxyLogger.Load()
tr, err := p.getOrInitTransport()
if err != nil {
return "", nil, err
}
// If we have a bootstrap IP, use it directly
if p.uc.BootstrapIP != "" {
addr := net.JoinHostPort(p.uc.BootstrapIP, p.port)
Log(ctx, logger.Debug(), "Sending DoQ request to: %s", addr)
remoteAddr, err := net.ResolveUDPAddr("udp", addr)
if err != nil {
return "", nil, err
}
conn, err := tr.DialEarly(ctx, remoteAddr, p.tlsConfig, p.quicConfig)
if err != nil {
return "", nil, err
}
return addr, conn, nil
}
// Use parallel dialing like DoH3
dialAddrs := make([]string, len(p.addrs))
for i := range p.addrs {
dialAddrs[i] = net.JoinHostPort(p.addrs[i], p.port)
}
pd := &quicParallelDialer{transport: tr}
conn, err := pd.Dial(ctx, dialAddrs, p.tlsConfig, p.quicConfig)
if err != nil {
return "", nil, err
}
addr := conn.RemoteAddr().String()
Log(ctx, logger.Debug(), "Sending DoQ request to: %s", addr)
return addr, conn, nil
}
// getOrInitTransport returns the pool's shared quic.Transport, initialising it
// on first call. Once the pool has been closed it permanently returns an error
// so that callers cannot resurrect a dead pool.
func (p *doqConnPool) getOrInitTransport() (*quic.Transport, error) {
p.transportMu.Lock()
defer p.transportMu.Unlock()
if p.closed {
return nil, errors.New("doq pool closed")
}
if p.transportInit {
return p.transport, p.transportErr
}
p.transportInit = true
udpConn, err := net.ListenUDP("udp", nil)
if err != nil {
p.transportErr = err
return nil, err
}
p.transportConn = udpConn
p.transport = &quic.Transport{Conn: udpConn}
return p.transport, nil
}
// CloseIdleConnections closes all idle connections, the shared quic.Transport,
// and the pool's UDP socket. Connections currently checked out (in use) get
// terminated by the transport close as well — without that, the OS socket
// would remain bound to a goroutine that the caller cannot reach to clean up.
func (p *doqConnPool) CloseIdleConnections() {
drain:
for {
select {
case dc := <-p.conns:
if dc.conn != nil {
dc.conn.CloseWithError(quic.ApplicationErrorCode(quic.NoError), "")
}
default:
break drain
}
}
p.transportMu.Lock()
if p.closed {
p.transportMu.Unlock()
return
}
p.closed = true
tr := p.transport
udpConn := p.transportConn
p.transportMu.Unlock()
if tr != nil {
_ = tr.Close()
}
if udpConn != nil {
_ = udpConn.Close()
}
}
+596 -1
View File
@@ -1,4 +1,3 @@
// test_helpers.go
package ctrld
import (
@@ -8,8 +7,11 @@ import (
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"io"
"math/big"
"net"
"os"
"runtime"
"strings"
"testing"
"time"
@@ -99,6 +101,7 @@ func newTestQUICServer(t *testing.T) *testQUICServer {
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"doq"},
MinVersion: tls.VersionTLS12,
}
// Create QUIC listener
@@ -221,3 +224,595 @@ func (s *testQUICServer) handleStream(t *testing.T, stream *quic.Stream) {
return
}
}
// malformedDoQServer is a test QUIC server that drains the client's DoQ
// request and writes caller-supplied raw bytes back. The bytes are not
// required to be a well-framed DoQ response, which is what lets the
// regression tests exercise malformed-response handling.
type malformedDoQServer struct {
listener *quic.Listener
cert *x509.Certificate
addr string
response []byte
}
func newMalformedDoQServer(t *testing.T, response []byte) *malformedDoQServer {
t.Helper()
testCert := generateTestCertificate(t)
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"doq"},
}
listener, err := quic.ListenAddr("127.0.0.1:0", tlsConfig, nil)
if err != nil {
t.Fatalf("failed to create QUIC listener: %v", err)
}
s := &malformedDoQServer{
listener: listener,
cert: testCert.cert,
addr: listener.Addr().String(),
response: response,
}
go s.serve()
t.Cleanup(func() { _ = listener.Close() })
return s
}
func (s *malformedDoQServer) serve() {
for {
conn, err := s.listener.Accept(context.Background())
if err != nil {
return
}
go s.handleConn(conn)
}
}
func (s *malformedDoQServer) handleConn(conn *quic.Conn) {
for {
stream, err := conn.AcceptStream(context.Background())
if err != nil {
return
}
go s.handleStream(stream)
}
}
func (s *malformedDoQServer) handleStream(stream *quic.Stream) {
defer stream.Close()
// Drain the client's DoQ-framed request so the client's writes complete
// cleanly before we reply with our attacker-controlled bytes. Using
// io.ReadFull because a single Read on a QUIC stream may return short.
lenBuf := make([]byte, 2)
if _, err := io.ReadFull(stream, lenBuf); err != nil {
return
}
msgLen := uint16(lenBuf[0])<<8 | uint16(lenBuf[1])
if msgLen > 0 {
discard := make([]byte, msgLen)
if _, err := io.ReadFull(stream, discard); err != nil {
return
}
}
if len(s.response) > 0 {
_, _ = stream.Write(s.response)
}
}
// newMalformedDoQUpstream builds an UpstreamConfig wired to a local
// malformed test server with the test certificate trusted via a custom
// cert pool. We bypass SetupBootstrapIP by setting BootstrapIP directly,
// so the pool dials 127.0.0.1 without any DNS lookup.
func newMalformedDoQUpstream(t *testing.T, cert *x509.Certificate, addr string) *UpstreamConfig {
t.Helper()
pool := x509.NewCertPool()
pool.AddCert(cert)
host, _, err := net.SplitHostPort(addr)
if err != nil {
t.Fatalf("split host/port %q: %v", addr, err)
}
uc := &UpstreamConfig{
Name: "doq-malformed",
Type: ResolverTypeDOQ,
Endpoint: addr,
Domain: host,
BootstrapIP: host,
Timeout: 2000,
}
uc.SetCertPool(pool)
return uc
}
// TestDoQResolve_MalformedResponse verifies that DoQ upstream
// responses violating RFC 9250 framing — fewer than 2 bytes, or a
// length prefix declaring more payload than was received — return a
// handled error instead of panicking on the length-prefix slice.
func TestDoQResolve_MalformedResponse(t *testing.T) {
tests := []struct {
name string
response []byte
}{
// Empty stream is already handled via io.EOF; locked in so a
// future change that drops that branch is caught.
{"empty response", nil},
// One byte: too short to hold the 2-octet length prefix.
{"single byte response", []byte{0x00}},
// Length prefix declares 16 bytes; payload is absent.
{"length prefix only", []byte{0x00, 0x10}},
// Length prefix declares 65535 bytes; only 1 byte of payload
// arrived.
{"length prefix larger than payload", []byte{0xFF, 0xFF, 0x00}},
}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
server := newMalformedDoQServer(t, tt.response)
uc := newMalformedDoQUpstream(t, server.cert, server.addr)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
pool := newDOQConnPool(uc, []string{"127.0.0.1"})
t.Cleanup(pool.CloseIdleConnections)
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
answer, err := pool.Resolve(ctx, msg)
if err == nil {
t.Fatalf("Resolve unexpectedly succeeded for malformed response %v; answer=%v", tt.response, answer)
}
if answer != nil {
t.Fatalf("Resolve returned non-nil answer alongside error: answer=%v err=%v", answer, err)
}
})
}
}
// strictDoQServer accepts DoQ queries but defers the response until the
// client signals end-of-request with STREAM FIN, as required by RFC 9250
// section 4.2. It exists to lock in the fix for
// github.com/Control-D-Inc/ctrld/issues/309 where a client
// that never closes its send side caused the server to wait forever and the
// client to churn through reconnects.
type strictDoQServer struct {
listener *quic.Listener
cert *x509.Certificate
addr string
}
func newStrictDoQServer(t *testing.T) *strictDoQServer {
t.Helper()
testCert := generateTestCertificate(t)
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{testCert.tlsCert},
NextProtos: []string{"doq"},
MinVersion: tls.VersionTLS12,
}
listener, err := quic.ListenAddr("127.0.0.1:0", tlsConfig, nil)
if err != nil {
t.Fatalf("failed to create QUIC listener: %v", err)
}
s := &strictDoQServer{
listener: listener,
cert: testCert.cert,
addr: listener.Addr().String(),
}
go s.serve()
t.Cleanup(func() { _ = listener.Close() })
return s
}
func (s *strictDoQServer) serve() {
for {
conn, err := s.listener.Accept(context.Background())
if err != nil {
return
}
go s.handleConn(conn)
}
}
func (s *strictDoQServer) handleConn(conn *quic.Conn) {
for {
stream, err := conn.AcceptStream(context.Background())
if err != nil {
return
}
go s.handleStream(stream)
}
}
func (s *strictDoQServer) handleStream(stream *quic.Stream) {
defer stream.Close()
// Drain until the client closes the send side. This is the behaviour
// that triggered the bug: if the client never sends STREAM FIN, this
// read blocks until the stream's deadline fires.
body, err := io.ReadAll(stream)
if err != nil {
return
}
if len(body) < 2 {
return
}
msgLen := uint16(body[0])<<8 | uint16(body[1])
if int(msgLen) != len(body)-2 {
return
}
msg := new(dns.Msg)
if err := msg.Unpack(body[2:]); err != nil {
return
}
response := new(dns.Msg)
response.SetReply(msg)
response.Authoritative = true
if len(msg.Question) > 0 && msg.Question[0].Qtype == dns.TypeA {
response.Answer = append(response.Answer, &dns.A{
Hdr: dns.RR_Header{
Name: msg.Question[0].Name,
Rrtype: dns.TypeA,
Class: dns.ClassINET,
Ttl: 300,
},
A: net.ParseIP("192.0.2.1"),
})
}
respBytes, err := response.Pack()
if err != nil {
return
}
respLen := uint16(len(respBytes))
if _, err := stream.Write([]byte{byte(respLen >> 8), byte(respLen & 0xFF)}); err != nil {
return
}
if _, err := stream.Write(respBytes); err != nil {
return
}
}
func newStrictDoQUpstream(t *testing.T, cert *x509.Certificate, addr string, useBootstrap bool) *UpstreamConfig {
t.Helper()
pool := x509.NewCertPool()
pool.AddCert(cert)
host, _, err := net.SplitHostPort(addr)
if err != nil {
t.Fatalf("split host/port %q: %v", addr, err)
}
uc := &UpstreamConfig{
Name: "doq-strict",
Type: ResolverTypeDOQ,
Endpoint: addr,
Domain: host,
Timeout: 3000,
}
if useBootstrap {
uc.BootstrapIP = host
}
uc.SetCertPool(pool)
return uc
}
// TestDoQResolve_StrictServerWaitsForFIN exercises the RFC 9250 client-FIN
// requirement. With the bug present, the server's io.ReadAll blocks until
// the stream deadline expires and the client sees a timeout, so a successful
// resolve here proves that the client now sends STREAM FIN before reading.
func TestDoQResolve_StrictServerWaitsForFIN(t *testing.T) {
t.Parallel()
server := newStrictDoQServer(t)
uc := newStrictDoQUpstream(t, server.cert, server.addr, true)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
host, _, _ := net.SplitHostPort(server.addr)
pool := newDOQConnPool(uc, []string{host})
t.Cleanup(pool.CloseIdleConnections)
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
answer, err := pool.Resolve(ctx, msg)
if err != nil {
t.Fatalf("Resolve failed against strict DoQ server: %v", err)
}
if answer == nil || len(answer.Answer) == 0 {
t.Fatalf("Resolve returned no answer records: %+v", answer)
}
a, ok := answer.Answer[0].(*dns.A)
if !ok || !a.A.Equal(net.ParseIP("192.0.2.1")) {
t.Fatalf("unexpected answer: %+v", answer.Answer[0])
}
}
// TestDoQResolve_ParallelDialPathStrictFIN exercises the parallel-dial path
// (no BootstrapIP) against the same FIN-strict server, so that both the
// single-dial branch and the parallel-dial branch are covered.
func TestDoQResolve_ParallelDialPathStrictFIN(t *testing.T) {
t.Parallel()
server := newStrictDoQServer(t)
uc := newStrictDoQUpstream(t, server.cert, server.addr, false)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
host, _, _ := net.SplitHostPort(server.addr)
pool := newDOQConnPool(uc, []string{host})
t.Cleanup(pool.CloseIdleConnections)
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
answer, err := pool.Resolve(ctx, msg)
if err != nil {
t.Fatalf("Resolve (parallel-dial path) failed against strict DoQ server: %v", err)
}
if answer == nil || len(answer.Answer) == 0 {
t.Fatalf("Resolve (parallel-dial path) returned no answer records: %+v", answer)
}
}
// TestDoQPool_ChurnDoesNotGrowFDs exercises the reconnect-churn scenario
// described in github.com/Control-D-Inc/ctrld/issues/309: repeated dials
// against a server that closes existing connections must not grow the process
// FD count, because the pool now shares one UDP socket via quic.Transport instead
// of allocating one per dial. Linux-only because /proc/self/fd is the cheapest
// portable proxy for "what's still open."
func TestDoQPool_ChurnDoesNotGrowFDs(t *testing.T) {
if runtime.GOOS != "linux" {
t.Skip("FD accounting via /proc/self/fd is linux-only")
}
t.Parallel()
server := newStrictDoQServer(t)
uc := newStrictDoQUpstream(t, server.cert, server.addr, true)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
host, _, _ := net.SplitHostPort(server.addr)
pool := newDOQConnPool(uc, []string{host})
t.Cleanup(pool.CloseIdleConnections)
makeQuery := func(i int) *dns.Msg {
msg := new(dns.Msg)
// Vary the question so any caching layer cannot short-circuit.
msg.SetQuestion(dns.Fqdn(strings.Repeat("a", 1+i%8)+".example.com"), dns.TypeA)
msg.RecursionDesired = true
return msg
}
// Warm the pool so the steady-state transport and at least one
// connection are open. Without this, the first resolve in the measured
// loop would inflate the baseline.
if _, err := pool.Resolve(ctx, makeQuery(0)); err != nil {
t.Fatalf("warm-up Resolve failed: %v", err)
}
baseline := countOpenFDs(t)
// Force reconnect churn by closing the connection between each query.
// Without the fix this would leak one UDP socket per round; with the
// fix the pool's shared transport keeps a single socket open.
const rounds = 20
for i := 1; i <= rounds; i++ {
// Drain any pooled connection so the next Resolve has to redial.
drainPooledConns(pool)
if _, err := pool.Resolve(ctx, makeQuery(i)); err != nil {
t.Fatalf("Resolve in churn loop iteration %d failed: %v", i, err)
}
}
// Give quic-go a moment to drop any background goroutines that hold
// references to closed sockets.
time.Sleep(200 * time.Millisecond)
after := countOpenFDs(t)
// Allow a small slack for transient FDs (goroutine wake-ups, qlog,
// etc.) but reject anything that scales with the number of rounds.
const slack = 5
if after > baseline+slack {
t.Fatalf("FD count grew under DoQ churn: baseline=%d after=%d rounds=%d (slack=%d)", baseline, after, rounds, slack)
}
}
// drainPooledConns removes any idle pooled connections so the next Resolve
// is forced to dial a fresh one. It does not close the pool's transport.
func drainPooledConns(p *doqConnPool) {
for {
select {
case dc := <-p.conns:
if dc.conn != nil {
dc.conn.CloseWithError(quic.ApplicationErrorCode(quic.NoError), "")
}
default:
return
}
}
}
func countOpenFDs(t *testing.T) int {
t.Helper()
entries, err := os.ReadDir("/proc/self/fd")
if err != nil {
t.Fatalf("read /proc/self/fd: %v", err)
}
return len(entries)
}
// TestDoQResolve_OversizedResponse_Rejected locks in the fix for
// github.com/Control-D-Inc/ctrld/issues/312 on the DoQ transport: a
// malicious upstream that writes a response larger than the DNS protocol
// allows must be rejected with an explicit size error, not buffered
// without bound into ctrld memory.
func TestDoQResolve_OversizedResponse_Rejected(t *testing.T) {
t.Parallel()
// doqMaxResponseSize is 2 + dns.MaxMsgSize. Send something well past
// that. 256 KiB is enough to exceed the cap while keeping the test
// fast on loopback.
response := make([]byte, 256*1024)
// A well-formed length prefix isn't required: the size cap should
// fire before any framing check runs. Use a non-zero prefix so the
// test also documents that the order of validation is "size first,
// framing later."
response[0] = 0xFF
response[1] = 0xFF
server := newMalformedDoQServer(t, response)
uc := newMalformedDoQUpstream(t, server.cert, server.addr)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
pool := newDOQConnPool(uc, []string{"127.0.0.1"})
t.Cleanup(pool.CloseIdleConnections)
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
answer, err := pool.Resolve(ctx, msg)
if err == nil {
t.Fatalf("Resolve unexpectedly succeeded for oversized response; answer=%v", answer)
}
if !strings.Contains(err.Error(), "exceeds") {
t.Fatalf("error %q does not surface the size cap", err)
}
if answer != nil {
t.Fatalf("Resolve returned non-nil answer alongside error: %v", answer)
}
}
// frameDoQResponse packs msg and prepends the RFC 9250 2-octet length prefix,
// producing the exact bytes a DoQ server writes on the wire.
func frameDoQResponse(t *testing.T, msg *dns.Msg) []byte {
t.Helper()
b, err := msg.Pack()
if err != nil {
t.Fatalf("pack response: %v", err)
}
n := uint16(len(b))
return append([]byte{byte(n >> 8), byte(n & 0xFF)}, b...)
}
// TestDoQResolve_PreservesRcode locks in the fix for
// github.com/Control-D-Inc/ctrld/issues/322: the DoQ resolver must not rewrite
// an upstream response with SetReply, which would clobber a SERVFAIL into
// NOERROR and hide the failure from the proxy's failover logic. The upstream
// RCODE must survive; only the transaction ID is restored for the client.
func TestDoQResolve_PreservesRcode(t *testing.T) {
t.Parallel()
// RFC 9250 puts the DNS Message ID at 0 on the wire.
resp := new(dns.Msg)
resp.SetQuestion("example.com.", dns.TypeA)
resp.Response = true
resp.Id = 0
resp.Rcode = dns.RcodeServerFailure
server := newMalformedDoQServer(t, frameDoQResponse(t, resp))
uc := newMalformedDoQUpstream(t, server.cert, server.addr)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
pool := newDOQConnPool(uc, []string{"127.0.0.1"})
t.Cleanup(pool.CloseIdleConnections)
msg := new(dns.Msg)
msg.SetQuestion("example.com.", dns.TypeA)
msg.RecursionDesired = true
answer, err := pool.Resolve(ctx, msg)
if err != nil {
t.Fatalf("Resolve failed: %v", err)
}
if answer.Rcode != dns.RcodeServerFailure {
t.Fatalf("upstream SERVFAIL was rewritten to %s; failover would be bypassed",
dns.RcodeToString[answer.Rcode])
}
if answer.Id != msg.Id {
t.Fatalf("transaction ID not restored: got %d, want %d", answer.Id, msg.Id)
}
}
// TestDoQResolve_PreservesWrongQuestion locks in the fix for
// github.com/Control-D-Inc/ctrld/issues/322: when an upstream answers a
// different name than asked, the resolver must preserve the upstream's
// question rather than rewriting it to the request's question (as SetReply
// did). Rewriting would hide the mismatch and let wrong-domain records poison
// the shared cache.
func TestDoQResolve_PreservesWrongQuestion(t *testing.T) {
t.Parallel()
resp := new(dns.Msg)
resp.SetQuestion("attacker.example.", dns.TypeA)
resp.Response = true
resp.Id = 0
resp.Answer = append(resp.Answer, &dns.A{
Hdr: dns.RR_Header{
Name: "attacker.example.",
Rrtype: dns.TypeA,
Class: dns.ClassINET,
Ttl: 300,
},
A: net.ParseIP("192.0.2.1"),
})
server := newMalformedDoQServer(t, frameDoQResponse(t, resp))
uc := newMalformedDoQUpstream(t, server.cert, server.addr)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
pool := newDOQConnPool(uc, []string{"127.0.0.1"})
t.Cleanup(pool.CloseIdleConnections)
msg := new(dns.Msg)
msg.SetQuestion("victim.example.", dns.TypeA)
msg.RecursionDesired = true
answer, err := pool.Resolve(ctx, msg)
if err != nil {
t.Fatalf("Resolve failed: %v", err)
}
if len(answer.Question) == 0 || !strings.EqualFold(answer.Question[0].Name, "attacker.example.") {
t.Fatalf("upstream question was rewritten; got %v, want the upstream's attacker.example.",
answer.Question)
}
if answer.Id != msg.Id {
t.Fatalf("transaction ID not restored: got %d, want %d", answer.Id, msg.Id)
}
}
+270 -20
View File
@@ -3,7 +3,11 @@ package ctrld
import (
"context"
"crypto/tls"
"errors"
"io"
"net"
"runtime"
"time"
"github.com/miekg/dns"
)
@@ -13,30 +17,276 @@ type dotResolver struct {
}
func (r *dotResolver) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, error) {
// The dialer is used to prevent bootstrapping cycle.
// If r.endpoint is set to dns.controld.dev, we need to resolve
// dns.controld.dev first. By using a dialer with custom resolver,
// we ensure that we can always resolve the bootstrap domain
// regardless of the machine DNS status.
dialer := newDialer(net.JoinHostPort(controldPublicDns, "53"))
if err := validateMsg(msg); err != nil {
return nil, err
}
dnsTyp := uint16(0)
if msg != nil && len(msg.Question) > 0 {
dnsTyp = msg.Question[0].Qtype
}
tcpNet, _ := r.uc.netForDNSType(dnsTyp)
dnsClient := &dns.Client{
Net: tcpNet,
Dialer: dialer,
TLSConfig: &tls.Config{RootCAs: r.uc.certPool},
}
endpoint := r.uc.Endpoint
if r.uc.BootstrapIP != "" {
dnsClient.TLSConfig.ServerName = r.uc.Domain
dnsClient.Net = "tcp-tls"
_, port, _ := net.SplitHostPort(endpoint)
endpoint = net.JoinHostPort(r.uc.BootstrapIP, port)
pool := r.uc.dotTransport(dnsTyp)
if pool == nil {
return nil, errors.New("DoT client pool is not available")
}
answer, _, err := dnsClient.ExchangeContext(ctx, msg, endpoint)
return answer, wrapCertificateVerificationError(err)
return pool.Resolve(ctx, msg)
}
const dotPoolSize = 16
// dotConnPool manages a pool of TCP/TLS connections for DoT queries using a buffered channel.
type dotConnPool struct {
uc *UpstreamConfig
addrs []string
port string
tlsConfig *tls.Config
dialer *net.Dialer
conns chan *dotConn
}
type dotConn struct {
conn *tls.Conn
}
func newDOTClientPool(uc *UpstreamConfig, addrs []string) *dotConnPool {
_, port, _ := net.SplitHostPort(uc.Endpoint)
if port == "" {
port = "853"
}
// The dialer is used to prevent bootstrapping cycle.
// If endpoint is set to dns.controld.dev, we need to resolve
// dns.controld.dev first. By using a dialer with custom resolver,
// we ensure that we can always resolve the bootstrap domain
// regardless of the machine DNS status.
dialer := newDialer(net.JoinHostPort(controldPublicDns, "53"))
tlsConfig := &tls.Config{
RootCAs: uc.certPool,
MinVersion: tls.VersionTLS12,
}
if uc.BootstrapIP != "" {
tlsConfig.ServerName = uc.Domain
}
pool := &dotConnPool{
uc: uc,
addrs: addrs,
port: port,
tlsConfig: tlsConfig,
dialer: dialer,
conns: make(chan *dotConn, dotPoolSize),
}
// Use SetFinalizer here because we need to call a method on the pool itself.
// AddCleanup would require passing the pool as arg (which panics) or capturing
// it in a closure (which prevents GC). SetFinalizer is appropriate for this case.
runtime.SetFinalizer(pool, func(p *dotConnPool) {
p.CloseIdleConnections()
})
return pool
}
// Resolve performs a DNS query using a pooled TCP/TLS connection.
func (p *dotConnPool) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, error) {
if msg == nil {
return nil, errors.New("nil DNS message")
}
conn, err := p.getConn(ctx)
if err != nil {
return nil, wrapCertificateVerificationError(err)
}
client := dns.Client{Net: "tcp-tls"}
answer, _, err := client.ExchangeWithConnContext(ctx, msg, &dns.Conn{Conn: conn})
isGood := err == nil
p.putConn(conn, isGood)
if err != nil {
return nil, wrapCertificateVerificationError(err)
}
return answer, nil
}
// getConn gets a TCP/TLS connection from the pool or creates a new one.
// A connection is taken from the channel while in use; putConn returns it.
func (p *dotConnPool) getConn(ctx context.Context) (net.Conn, error) {
for {
select {
case dc := <-p.conns:
if dc.conn != nil && isAlive(dc.conn) {
return dc.conn, nil
}
if dc.conn != nil {
dc.conn.Close()
}
default:
_, conn, err := p.dialConn(ctx)
if err != nil {
return nil, err
}
return conn, nil
}
}
}
// putConn returns a connection to the pool for reuse by other goroutines.
func (p *dotConnPool) putConn(conn net.Conn, isGood bool) {
if !isGood || conn == nil {
if conn != nil {
conn.Close()
}
return
}
dc := &dotConn{conn: conn.(*tls.Conn)}
select {
case p.conns <- dc:
default:
// Channel full, close the connection
dc.conn.Close()
}
}
// dialConn creates a new TCP/TLS connection.
func (p *dotConnPool) dialConn(ctx context.Context) (string, *tls.Conn, error) {
logger := ProxyLogger.Load()
var endpoint string
if p.uc.BootstrapIP != "" {
endpoint = net.JoinHostPort(p.uc.BootstrapIP, p.port)
Log(ctx, logger.Debug(), "Sending DoT request to: %s", endpoint)
conn, err := p.dialer.DialContext(ctx, "tcp", endpoint)
if err != nil {
return "", nil, err
}
tlsConn := tls.Client(conn, p.tlsConfig)
if err := tlsConn.HandshakeContext(ctx); err != nil {
conn.Close()
return "", nil, err
}
return endpoint, tlsConn, nil
}
// Try bootstrap IPs in parallel
if len(p.addrs) > 0 {
type result struct {
conn *tls.Conn
addr string
err error
}
ch := make(chan result, len(p.addrs))
done := make(chan struct{})
defer close(done)
for _, addr := range p.addrs {
go func(addr string) {
endpoint := net.JoinHostPort(addr, p.port)
conn, err := p.dialer.DialContext(ctx, "tcp", endpoint)
if err != nil {
select {
case ch <- result{conn: nil, addr: endpoint, err: err}:
case <-done:
}
return
}
tlsConfig := p.tlsConfig.Clone()
tlsConfig.ServerName = p.uc.Domain
tlsConn := tls.Client(conn, tlsConfig)
if err := tlsConn.HandshakeContext(ctx); err != nil {
conn.Close()
select {
case ch <- result{conn: nil, addr: endpoint, err: err}:
case <-done:
}
return
}
select {
case ch <- result{conn: tlsConn, addr: endpoint, err: nil}:
case <-done:
if conn != nil {
conn.Close()
}
}
}(addr)
}
errs := make([]error, 0, len(p.addrs))
for range len(p.addrs) {
select {
case res := <-ch:
if res.err == nil && res.conn != nil {
Log(ctx, logger.Debug(), "Sending DoT request to: %s", res.addr)
return res.addr, res.conn, nil
}
if res.err != nil {
errs = append(errs, res.err)
}
case <-ctx.Done():
return "", nil, ctx.Err()
}
}
return "", nil, errors.Join(errs...)
}
// Fallback to endpoint resolution
endpoint = p.uc.Endpoint
Log(ctx, logger.Debug(), "Sending DoT request to: %s", endpoint)
conn, err := p.dialer.DialContext(ctx, "tcp", endpoint)
if err != nil {
return "", nil, err
}
tlsConn := tls.Client(conn, p.tlsConfig)
if err := tlsConn.HandshakeContext(ctx); err != nil {
conn.Close()
return "", nil, err
}
return endpoint, tlsConn, nil
}
// CloseIdleConnections closes all connections in the pool.
// Connections currently checked out (in use) are not closed.
func (p *dotConnPool) CloseIdleConnections() {
for {
select {
case dc := <-p.conns:
if dc.conn != nil {
dc.conn.Close()
}
default:
return
}
}
}
func isAlive(c *tls.Conn) bool {
// Set a very short deadline for the read
c.SetReadDeadline(time.Now().Add(1 * time.Millisecond))
// Try to read 1 byte without consuming it (using a small buffer)
one := make([]byte, 1)
_, err := c.Read(one)
// Reset the deadline for future operations
c.SetReadDeadline(time.Time{})
if err == io.EOF {
return false // Connection is definitely closed
}
// If we get a timeout, it means no data is waiting,
// but the connection is likely still "up."
var netErr net.Error
if errors.As(err, &netErr) && netErr.Timeout() {
return true
}
return err == nil
}
+24 -20
View File
@@ -1,12 +1,11 @@
module github.com/Control-D-Inc/ctrld
go 1.23.0
toolchain go1.23.7
go 1.25.0
require (
github.com/Masterminds/semver/v3 v3.2.1
github.com/ameshkov/dnsstamps v1.0.3
github.com/brunogui0812/sysprofiler v0.5.0
github.com/coreos/go-systemd/v22 v22.5.0
github.com/cuonglm/osinfo v0.0.0-20230921071424-e0e1b1e0bbbf
github.com/docker/go-units v0.5.0
@@ -16,7 +15,8 @@ require (
github.com/godbus/dbus/v5 v5.1.1-0.20230522191255-76236955d466
github.com/hashicorp/golang-lru/v2 v2.0.1
github.com/illarion/gonotify/v2 v2.0.3
github.com/insomniacslk/dhcp v0.0.0-20231206064809-8c70d406f6d2
github.com/insomniacslk/dhcp v0.0.0-20260719225207-c76316d4aa82
github.com/jaypipes/ghw v0.21.0
github.com/jaytaylor/go-hostsfile v0.0.0-20220426042432-61485ac1fa6c
github.com/josharian/native v1.1.1-0.20230202152459-5c7d0dd6ab86
github.com/kardianos/service v1.2.1
@@ -29,16 +29,16 @@ require (
github.com/prometheus/client_golang v1.19.1
github.com/prometheus/client_model v0.5.0
github.com/prometheus/prom2json v1.3.3
github.com/quic-go/quic-go v0.54.0
github.com/quic-go/quic-go v0.59.1
github.com/rs/zerolog v1.28.0
github.com/spf13/cobra v1.8.1
github.com/spf13/pflag v1.0.5
github.com/spf13/cobra v1.9.1
github.com/spf13/pflag v1.0.6
github.com/spf13/viper v1.16.0
github.com/stretchr/testify v1.9.0
github.com/vishvananda/netlink v1.2.1-beta.2
golang.org/x/net v0.38.0
golang.org/x/sync v0.12.0
golang.org/x/sys v0.31.0
github.com/stretchr/testify v1.11.1
github.com/vishvananda/netlink v1.3.1
golang.org/x/net v0.56.0
golang.org/x/sync v0.22.0
golang.org/x/sys v0.46.0
golang.zx2c4.com/wireguard/windows v0.5.3
tailscale.com v1.74.0
)
@@ -57,8 +57,10 @@ require (
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/groob/plist v0.0.0-20200425180238-0f631f258c01 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jaypipes/pcidb v1.1.1 // indirect
github.com/jsimonetti/rtnetlink v1.4.0 // indirect
github.com/kr/pretty v0.3.1 // indirect
github.com/kr/text v0.2.0 // indirect
@@ -77,26 +79,28 @@ require (
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/common v0.48.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
github.com/quic-go/qpack v0.5.1 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/rivo/uniseg v0.4.4 // indirect
github.com/rogpeppe/go-internal v1.11.0 // indirect
github.com/spakin/awk v1.0.0 // indirect
github.com/spf13/afero v1.9.5 // indirect
github.com/spf13/cast v1.6.0 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/subosito/gotenv v1.4.2 // indirect
github.com/u-root/uio v0.0.0-20240118234441-a3c409a6018e // indirect
github.com/vishvananda/netns v0.0.4 // indirect
go.uber.org/mock v0.5.0 // indirect
github.com/vishvananda/netns v0.0.5 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go4.org/mem v0.0.0-20220726221520-4f986261bf13 // indirect
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
golang.org/x/crypto v0.36.0 // indirect
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842 // indirect
golang.org/x/mod v0.19.0 // indirect
golang.org/x/text v0.23.0 // indirect
golang.org/x/tools v0.23.0 // indirect
golang.org/x/crypto v0.53.0 // indirect
golang.org/x/exp v0.0.0-20240119083558-1b970713d09a // indirect
golang.org/x/mod v0.37.0 // indirect
golang.org/x/text v0.40.0 // indirect
golang.org/x/tools v0.47.0 // indirect
google.golang.org/protobuf v1.33.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
howett.net/plist v1.0.2-0.20250314012144-ee69052608d9 // indirect
)
replace github.com/mr-karan/doggo => github.com/Windscribe/doggo v0.0.0-20220919152748-2c118fc391f8
+54 -38
View File
@@ -50,6 +50,8 @@ github.com/ameshkov/dnsstamps v1.0.3 h1:Srzik+J9mivH1alRACTbys2xOxs0lRH9qnTA7Y1O
github.com/ameshkov/dnsstamps v1.0.3/go.mod h1:Ii3eUu73dx4Vw5O4wjzmT5+lkCwovjzaEZZ4gKyIH5A=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/brunogui0812/sysprofiler v0.5.0 h1:AUekplOKG/VKH6sPSBRxsKOA9Uv5OsI8qolXM73dXPU=
github.com/brunogui0812/sysprofiler v0.5.0/go.mod h1:lLd7gvylgd4nsTSC8exq1YY6qhLWXkgnalxjVzdlbEM=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
@@ -64,7 +66,7 @@ github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnht
github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/coreos/go-systemd/v22 v22.5.0 h1:RrqgGjYQKalulkV8NGVIfkXQf6YYmOyiJKk8iXXhfZs=
github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/cuonglm/osinfo v0.0.0-20230921071424-e0e1b1e0bbbf h1:40DHYsri+d1bnroFDU2FQAeq68f3kAlOzlQ93kCf26Q=
github.com/cuonglm/osinfo v0.0.0-20230921071424-e0e1b1e0bbbf/go.mod h1:G45410zMgmnSjLVKCq4f6GpbYAzoP2plX9rPwgx6C24=
@@ -91,6 +93,7 @@ github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-json-experiment/json v0.0.0-20231102232822-2e55bd4e08b0 h1:ymLjT4f35nQbASLnvxEde4XOBL+Sn7rFuV+FOJqkljg=
github.com/go-json-experiment/json v0.0.0-20231102232822-2e55bd4e08b0/go.mod h1:6daplAwHHGbUGib4990V3Il26O0OC4aRyvewaaAihaA=
github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=
github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=
github.com/go-playground/assert/v2 v2.0.1 h1:MsBgLAaY856+nPRTKrp3/OZK38U/wa0CcBYNjji3q3A=
@@ -165,6 +168,8 @@ github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/googleapis/google-cloud-go-testing v0.0.0-20200911160855-bcd43fbb19e8/go.mod h1:dvDLG8qkwmyD9a/MJJN3XJcT3xFxOKAvTZGvuZmac9g=
github.com/groob/plist v0.0.0-20200425180238-0f631f258c01 h1:0T3XGXebqLj7zSVLng9wX9axQzTEnvj/h6eT7iLfUas=
github.com/groob/plist v0.0.0-20200425180238-0f631f258c01/go.mod h1:itkABA+w2cw7x5nYUS/pLRef6ludkZKOigbROmCTaFw=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru/v2 v2.0.1 h1:5pv5N1lT1fjLg2VQ5KWc7kmucp2x/kvFOnxuVTqZ6x4=
@@ -179,10 +184,15 @@ github.com/illarion/gonotify/v2 v2.0.3 h1:B6+SKPo/0Sw8cRJh1aLzNEeNVFfzE3c6N+o+vy
github.com/illarion/gonotify/v2 v2.0.3/go.mod h1:38oIJTgFqupkEydkkClkbL6i5lXV/bxdH9do5TALPEE=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/insomniacslk/dhcp v0.0.0-20231206064809-8c70d406f6d2 h1:9K06NfxkBh25x56yVhWWlKFE8YpicaSfHwoV8SFbueA=
github.com/insomniacslk/dhcp v0.0.0-20231206064809-8c70d406f6d2/go.mod h1:3A9PQ1cunSDF/1rbTq99Ts4pVnycWg+vlPkfeD2NLFI=
github.com/insomniacslk/dhcp v0.0.0-20260719225207-c76316d4aa82 h1:y5aU8Uvl7eyM5WNgdQvRxbMJb+zo7pD+S72/Yo4pvnQ=
github.com/insomniacslk/dhcp v0.0.0-20260719225207-c76316d4aa82/go.mod h1:qfvBmyDNp+/liLEYWRvqny/PEz9hGe2Dz833eXILSmo=
github.com/jaypipes/ghw v0.21.0 h1:ClG2xWtYY0c1ud9jZYwVGdSgfCI7AbmZmZyw3S5HHz8=
github.com/jaypipes/ghw v0.21.0/go.mod h1:GPrvwbtPoxYUenr74+nAnWbardIZq600vJDD5HnPsPE=
github.com/jaypipes/pcidb v1.1.1 h1:QmPhpsbmmnCwZmHeYAATxEaoRuiMAJusKYkUncMC0ro=
github.com/jaypipes/pcidb v1.1.1/go.mod h1:x27LT2krrUgjf875KxQXKB0Ha/YXLdZRVmw6hH0G7g8=
github.com/jaytaylor/go-hostsfile v0.0.0-20220426042432-61485ac1fa6c h1:kbTQ8oGf+BVFvt/fM+ECI+NbZDCqoi0vtZTfB2p2hrI=
github.com/jaytaylor/go-hostsfile v0.0.0-20220426042432-61485ac1fa6c/go.mod h1:k6+89xKz7BSMJ+DzIerBdtpEUeTlBMugO/hcVSzahog=
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
github.com/josharian/native v1.0.1-0.20221213033349-c1e37c09b531/go.mod h1:7X/raswPFr05uY3HiLlYeyQntB6OO7E/d2Cu7qoaN2w=
github.com/josharian/native v1.1.1-0.20230202152459-5c7d0dd6ab86 h1:elKwZS1OcdQ0WwEDBeqxKwb7WB62QX8bvZ/FJnVXIfk=
github.com/josharian/native v1.1.1-0.20230202152459-5c7d0dd6ab86/go.mod h1:aFAMtuldEgx/4q7iSGazk22+IcgvtiC+HIimFO9XlS8=
@@ -259,10 +269,10 @@ github.com/prometheus/procfs v0.12.0 h1:jluTpSng7V9hY0O2R9DzzJHYb2xULk9VTR1V1R/k
github.com/prometheus/procfs v0.12.0/go.mod h1:pcuDEFsWDnvcgNzo4EEweacyhjeA9Zk3cnaOZAZEfOo=
github.com/prometheus/prom2json v1.3.3 h1:IYfSMiZ7sSOfliBoo89PcufjWO4eAR0gznGcETyaUgo=
github.com/prometheus/prom2json v1.3.3/go.mod h1:Pv4yIPktEkK7btWsrUTWDDDrnpUrAELaOCj+oFwlgmc=
github.com/quic-go/qpack v0.5.1 h1:giqksBPnT/HDtZ6VhtFKgoLOWmlyo9Ei6u9PqzIMbhI=
github.com/quic-go/qpack v0.5.1/go.mod h1:+PC4XFrEskIVkcLzpEkbLqq1uCoxPhQuvK5rH1ZgaEg=
github.com/quic-go/quic-go v0.54.0 h1:6s1YB9QotYI6Ospeiguknbp2Znb/jZYjZLRXn9kMQBg=
github.com/quic-go/quic-go v0.54.0/go.mod h1:e68ZEaCdyviluZmy44P6Iey98v/Wfz6HCjQEm+l8zTY=
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/quic-go/quic-go v0.59.1 h1:0Gmua0HW1Tv7ANR7hUYwRyD0MG5OJfgvYSZasGZzBic=
github.com/quic-go/quic-go v0.59.1/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis=
github.com/rivo/uniseg v0.4.4/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
@@ -274,16 +284,18 @@ github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDN
github.com/rogpeppe/go-internal v1.11.0/go.mod h1:ddIwULY96R17DhadqLgMfk9H9tvdUzkipdSkR5nkCZA=
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spakin/awk v1.0.0 h1:5ulBVgJhdN3XoFGNVv/MOHOIUfPVPvMCIlLH6O6ZqU4=
github.com/spakin/awk v1.0.0/go.mod h1:e7FnxcIEcRqdKwStPYWonox4n9DpharWk+3nnn1IqJs=
github.com/spf13/afero v1.9.5 h1:stMpOSZFs//0Lv29HduCmli3GUfpFoF3Y1Q/aXj/wVM=
github.com/spf13/afero v1.9.5/go.mod h1:UBogFpq8E9Hx+xc5CNTTEpTnuHVmXDwZcZcE1eb/UhQ=
github.com/spf13/cast v1.6.0 h1:GEiTHELF+vaR5dhz3VqZfFSzZjYbgeKDpBxQVS4GYJ0=
github.com/spf13/cast v1.6.0/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM=
github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y=
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
github.com/spf13/jwalterweatherman v1.1.0 h1:ue6voC5bR5F8YxI5S67j9i582FU4Qvo2bmqnqMYADFk=
github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.16.0 h1:rGGH0XDZhdUOryiDWjmIvUSWpbNqisK8Wk0Vyefw8hc=
github.com/spf13/viper v1.16.0/go.mod h1:yg78JgCJcbrQOvV9YLXgkLaZqUidkY9K+Dd1FofRzQg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -299,29 +311,30 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/subosito/gotenv v1.4.2 h1:X1TuBLAMDFbaTAChgCBLu3DU3UPyELpnF2jjJ2cz/S8=
github.com/subosito/gotenv v1.4.2/go.mod h1:ayKnFf/c6rvx/2iiLrJUk1e6plDbT3edrFNGqEflhK0=
github.com/u-root/uio v0.0.0-20240118234441-a3c409a6018e h1:BA9O3BmlTmpjbvajAwzWx4Wo2TRVdpPXZEeemGQcajw=
github.com/u-root/uio v0.0.0-20240118234441-a3c409a6018e/go.mod h1:eLL9Nub3yfAho7qB0MzZizFhTU2QkLeoVsWdHtDW264=
github.com/vishvananda/netlink v1.2.1-beta.2 h1:Llsql0lnQEbHj0I1OuKyp8otXp0r3q0mPkuhwHfStVs=
github.com/vishvananda/netlink v1.2.1-beta.2/go.mod h1:twkDnbuQxJYemMlGd4JFIcuhgX83tXhKS2B/PRMpOho=
github.com/vishvananda/netns v0.0.0-20200728191858-db3c7e526aae/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0=
github.com/vishvananda/netns v0.0.4 h1:Oeaw1EM2JMxD51g9uhtC0D7erkIjgmj8+JZc26m1YX8=
github.com/vishvananda/netns v0.0.4/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
github.com/vishvananda/netlink v1.3.1 h1:3AEMt62VKqz90r0tmNhog0r/PpWKmrEShJU0wJW6bV0=
github.com/vishvananda/netlink v1.3.1/go.mod h1:ARtKouGSTGchR8aMwmkzC0qiNPrrWO5JS/XMVl45+b4=
github.com/vishvananda/netns v0.0.5 h1:DfiHV+j8bA32MFM7bfEunvT8IAqQ/NzSJHtcmW5zdEY=
github.com/vishvananda/netns v0.0.5/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
go.uber.org/mock v0.5.0 h1:KAMbZvZPyBPWgD14IrIQ38QCyjwpvVVV6K/bHl1IwQU=
go.uber.org/mock v0.5.0/go.mod h1:ge71pBPLYDk7QIi1LupWxdAykm7KIEFchiOqd6z7qMM=
go.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko=
go.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o=
go4.org/mem v0.0.0-20220726221520-4f986261bf13 h1:CbZeCBZ0aZj8EfVgnqQcYZgf0lpZ3H9rmp5nkDTAst8=
go4.org/mem v0.0.0-20220726221520-4f986261bf13/go.mod h1:reUoABIJ9ikfM5sgtSF3Wushcza7+WeD01VB9Lirh3g=
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba h1:0b9z3AuHCjxk0x/opv64kcgZLBseWJUpBw5I82+2U4M=
@@ -336,8 +349,8 @@ golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm
golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34=
golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -348,8 +361,8 @@ golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u0
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842 h1:vr/HnozRka3pE4EsMEg1lgkXJkTFJCVUX+S/ZT6wYzM=
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842/go.mod h1:XtvwrStGgqGPLc4cjQfWqZHG1YFdYs6swckp8vpsjnc=
golang.org/x/exp v0.0.0-20240119083558-1b970713d09a h1:Q8/wZp0KX97QFTc2ywcOE0YRjZPVIx+MXInMzdvQqcA=
golang.org/x/exp v0.0.0-20240119083558-1b970713d09a/go.mod h1:idGWGoKP1toJGkd5/ig9ZLuPcZBC3ewk7SzmH0uou08=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
@@ -373,8 +386,8 @@ golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.19.0 h1:fEdghXQSo20giMthA7cd28ZC+jts4amQ3YMXiP5oMQ8=
golang.org/x/mod v0.19.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -407,8 +420,8 @@ golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v
golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8=
golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8=
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -428,8 +441,8 @@ golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.12.0 h1:MHc5BpPuC30uJk597Ri8TV3CNZcTLu6B6z4lJy+g6Jw=
golang.org/x/sync v0.12.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -439,6 +452,7 @@ golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -447,7 +461,6 @@ golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200217220822-9197077df867/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -456,7 +469,6 @@ golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200728102440-3e129f6d46b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -475,11 +487,13 @@ golang.org/x/sys v0.0.0-20220622161953-175b2fd9d664/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220817070843-5a390386f1f2/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.4.1-0.20230131160137-e7d7f63158de/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik=
golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -490,8 +504,8 @@ golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.23.0 h1:D71I7dUrlY+VX0gQShAThNGHFxZ13dGLBHQLVl1mJlY=
golang.org/x/text v0.23.0/go.mod h1:/BLNzu4aZCJ1+kcD0DNRotWKage4q2rGVAg4o22unh4=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -542,8 +556,8 @@ golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4f
golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
golang.org/x/tools v0.23.0 h1:SGsXPZ+2l4JsgaCKkx+FQ9YZ5XEtA1GZYuoDjenLjvg=
golang.org/x/tools v0.23.0/go.mod h1:pnu6ufv6vQkll6szChhK3C3L/ruaIv5eBeztNG8wtsI=
golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -659,6 +673,8 @@ honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWh
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
howett.net/plist v1.0.2-0.20250314012144-ee69052608d9 h1:eeH1AIcPvSc0Z25ThsYF+Xoqbn0CI/YnXVYoTLFdGQw=
howett.net/plist v1.0.2-0.20250314012144-ee69052608d9/go.mod h1:fyFX5Hj5tP1Mpk8obqA9MZgXT416Q5711SDT7dQLTLk=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=

Some files were not shown because too many files have changed in this diff Show More