cmd/cli: lowercase AD domain to be consistent with network rules

While at it, also add a note that the domain comparison are done in
case-insensitive manner.

.dockerignore

@@ -0,0 +1,2 @@
+Dockerfile
+.git/

.github/workflows/ci.yml

@@ -9,18 +9,18 @@ jobs:
       fail-fast: false
       matrix:
         os: ["windows-latest", "ubuntu-latest", "macOS-latest"]
-        go: ["1.19.x"]
+        go: ["1.23.x"]
     runs-on: ${{ matrix.os }}
     steps:
-      - uses: actions/checkout@v1
+      - uses: actions/checkout@v3
         with:
           fetch-depth: 1
-      - uses: WillAbides/setup-go-faster@v1.7.0
+      - uses: WillAbides/setup-go-faster@v1.8.0
         with:
           go-version: ${{ matrix.go }}
       - run: "go test -race ./..."
       - uses: dominikh/staticcheck-action@v1.2.0
         with:
-          version: "2022.1.1"
+          version: "2024.1.1"
           install-go: false
           cache-key: ${{ matrix.go }}

.gitignore

@@ -1,3 +1,14 @@
-
 dist/
 gon.hcl
+
+/Build
+.DS_Store
+
+# Release folder
+dist/
+
+# Binaries
+ctrld-*
+
+# generated file
+cmd/cli/rsrc_*.syso

.goreleaser-darwin.yaml

@@ -9,6 +9,8 @@ builds:
       - -trimpath
     ldflags:
       - -s -w
+      - -X main.version={{.Version}}
+      - -X main.commit={{.Commit}}
     goos:
       - darwin
     goarch:

.goreleaser-qf.yaml

@@ -0,0 +1,44 @@
+before:
+  hooks:
+    - go mod tidy
+builds:
+  - id: ctrld
+    env:
+      - CGO_ENABLED=0
+    flags:
+      - -trimpath
+    ldflags:
+      - -s -w
+      - -X main.version={{.Version}}
+      - -X main.commit={{.Commit}}
+    goos:
+      - darwin
+      - linux
+      - windows
+    goarch:
+      - 386
+      - arm
+      - amd64
+      - arm64
+    tags:
+      - qf
+    main: ./cmd/ctrld
+    hooks:
+      post: /bin/sh ./scripts/upx.sh {{ .Path }}
+archives:
+  - format_overrides:
+      - goos: windows
+        format: zip
+    strip_parent_binary_folder: true
+    wrap_in_directory: true
+    files:
+      - README.md
+checksum:
+  name_template: 'checksums.txt'
+snapshot:
+  name_template: "{{ incpatch .Version }}-next"
+changelog:
+  sort: asc
+  filters:
+    exclude:
+      - '^docs:'

.goreleaser.yaml

@@ -9,13 +9,17 @@ builds:
       - -trimpath
     ldflags:
       - -s -w
+      - -X main.version={{.Version}}
+      - -X main.commit={{.Commit}}
     goos:
       - linux
+      - freebsd
       - windows
     goarch:
       - 386
       - arm
       - mips
+      - mipsle
       - amd64
       - arm64
     goarm:
@@ -25,6 +29,12 @@ builds:
     gomips:
       - softfloat
     main: ./cmd/ctrld
+    hooks:
+      post: /bin/sh ./scripts/upx.sh {{ .Path }}
+    ignore:
+      - goos: freebsd
+        goarch: arm
+        goarm: 5
 archives:
   - format_overrides:
       - goos: windows

README.md

@@ -1,9 +1,22 @@
 # ctrld
+
+![Test](https://github.com/Control-D-Inc/ctrld/actions/workflows/ci.yml/badge.svg)
+[![Go Reference](https://pkg.go.dev/badge/github.com/Control-D-Inc/ctrld.svg)](https://pkg.go.dev/github.com/Control-D-Inc/ctrld)
+[![Go Report Card](https://goreportcard.com/badge/github.com/Control-D-Inc/ctrld)](https://goreportcard.com/report/github.com/Control-D-Inc/ctrld)
+
+![ctrld spash image](/docs/ctrldsplash.png)
+
 A highly configurable DNS forwarding proxy with support for:
 - Multiple listeners for incoming queries
 - Multiple upstreams with fallbacks
 - Multiple network policy driven DNS query steering
 - Policy driven domain based "split horizon" DNS with wildcard support
+- Integrations with common router vendors and firmware
+- LAN client discovery via DHCP, mDNS, ARP, NDP, hosts file parsing
+- Prometheus metrics exporter 
+
+## TLDR
+Proxy legacy DNS traffic to secure DNS upstreams in highly configurable ways. 
 
 All DNS protocols are supported, including:
 - `UDP 53`
@@ -12,23 +25,46 @@ All DNS protocols are supported, including:
 - `DNS-over-HTTP/3` (DOH3)
 - `DNS-over-QUIC`
 
-## Use Cases
+# Use Cases
 1. Use secure DNS protocols on networks and devices that don't natively support them (legacy routers, legacy OSes, TVs, smart toasters).
 2. Create source IP based DNS routing policies with variable secure DNS upstreams. Subnet 1 (admin) uses upstream resolver A, while Subnet 2 (employee) uses upstream resolver B.
 3. Create destination IP based DNS routing policies with variable secure DNS upstreams. Listener 1 uses upstream resolver C, while Listener 2 uses upstream resolver D.
 4. Create domain level "split horizon" DNS routing policies to send internal domains (*.company.int) to a local DNS server, while everything else goes to another upstream.
+5. Deploy on a router and create LAN client specific DNS routing policies from a web GUI (When using ControlD.com).
 
 
 ## OS Support
 - Windows (386, amd64, arm)
 - Mac (amd64, arm64)
 - Linux (386, amd64, arm, mips)
+- FreeBSD
+- Common routers (See Router Mode below)
 
-## Download
-Download pre-compiled binaries from the [Releases](https://github.com/Control-D-Inc/ctrld/releases) section.
+# Install
+There are several ways to download and install `ctrld.
+
+## Quick Install
+The simplest way to download and install `ctrld` is to use the following installer command on any UNIX-like platform:
+
+```shell
+sh -c 'sh -c "$(curl -sL https://api.controld.com/dl)"'
+```
+
+Windows user and prefer Powershell (who doesn't)? No problem, execute this command instead in administrative cmd:
+```shell
+powershell -Command "(Invoke-WebRequest -Uri 'https://api.controld.com/dl' -UseBasicParsing).Content | Set-Content 'ctrld_install.bat'" && ctrld_install.bat
+```
+
+Or you can pull and run a Docker container from [Docker Hub](https://hub.docker.com/r/controldns/ctrld)
+```
+$ docker pull controldns/ctrld
+```
+
+## Download Manually
+Alternatively, if you know what you're doing you can download pre-compiled binaries from the [Releases](https://github.com/Control-D-Inc/ctrld/releases) section for the appropriate platform. 
 
 ## Build
-`ctrld` requires `go1.19+`:
+Lastly, you can build `ctrld` from source which requires `go1.21+`:
 
 ```shell
 $ go build ./cmd/ctrld
@@ -40,6 +76,17 @@ or
 $ go install github.com/Control-D-Inc/ctrld/cmd/ctrld@latest
 ```
 
+or 
+
+```
+$ docker build -t controldns/ctrld . -f docker/Dockerfile
+$ docker run -d --name=ctrld -p 53:53/tcp -p 53:53/udp controldns/ctrld --cd=RESOLVER_ID_GOES_HERE -vv
+```
+
+
+# Usage
+The cli is self documenting, so free free to run `--help` on any sub-command to get specific usages. 
+
 ## Arguments
 ```
         __         .__       .___
@@ -55,18 +102,25 @@ Usage:
 Available Commands:
   run         Run the DNS proxy server
   service     Manage ctrld service
-  start       Quick start service and configure DNS on default interface
-  stop        Quick stop service and remove DNS from default interface
+  start       Quick start service and configure DNS on interface
+  stop        Quick stop service and remove DNS from interface
+  restart     Restart the ctrld service
+  reload      Reload the ctrld service
+  status      Show status of the ctrld service
+  uninstall   Stop and uninstall the ctrld service
+  clients     Manage clients
+  upgrade     Upgrading ctrld to latest version
 
 Flags:
   -h, --help            help for ctrld
+  -s, --silent          do not write any log output
   -v, --verbose count   verbose log output, "-v" basic logging, "-vv" debug level logging
       --version         version for ctrld
 
 Use "ctrld [command] --help" for more information about a command.
 ```
 
-## Usage
+## Basic Run Mode
 To start the server with default configuration, simply run: `./ctrld run`. This will create a generic `ctrld.toml` file in the **working directory** and start the application in foreground. 
 1. Start the server
   ```
@@ -80,40 +134,33 @@ To start the server with default configuration, simply run: `./ctrld run`. This
   147.185.34.1
   ```
 
-If `verify.controld.com` resolves, you're successfully using the default Control D upstream.
+If `verify.controld.com` resolves, you're successfully using the default Control D upstream. From here, you can start editing the config file and go nuts with it. To enforce a new config, restart the server. 
 
-### Service Mode
-To run the application in service mode, simply run: `./ctrld start` as system/root user. This will create a generic `ctrld.toml` file in the **user home** directory, start the system service, and configure the listener on the default interface. Service will start on OS boot.
+## Service Mode
+To run the application in service mode on any Windows, MacOS, Linux distibution or supported router, simply run: `./ctrld start` as system/root user. This will create a generic `ctrld.toml` file in the **user home** directory (on Windows) or `/etc/controld/` (almost everywhere else), start the system service, and configure the listener on the default network interface. Service will start on OS boot.
 
-In order to stop the service, and restore your DNS to original state, simply run `./ctrld stop`.
+When Control D upstreams are used, `ctrld` willl [relay your network topology](https://docs.controld.com/docs/device-clients) to Control D (LAN IPs, MAC addresses, and hostnames), and you will be able to see your LAN devices in the web panel, view analytics and apply unique profiles to them. 
 
-For granular control of the service, run the `service` command. Each sub-command has its own help section so you can see what arguments you can supply.
+In order to stop the service, and restore your DNS to original state, simply run `./ctrld stop`. If you wish to stop and uninstall the service permanently, run `./ctrld uninstall`. 
 
-```
-  Manage ctrld service
 
-  Usage:
-    ctrld service [command]
+### Supported Routers
+You can run `ctrld` on any supported router, which will function similarly to the Service Mode mentioned above. The list of supported routers and firmware includes:
+- Asus Merlin
+- DD-WRT
+- Firewalla
+- FreshTomato
+- GL.iNet
+- OpenWRT
+- pfSense / OPNsense
+- Synology 
+- Ubiquiti (UniFi, EdgeOS)
 
-  Available Commands:
-    interfaces  Manage network interfaces
-    restart     Restart the ctrld service
-    start       Start the ctrld service
-    status      Show status of the ctrld service
-    stop        Stop the ctrld service
-    uninstall   Uninstall the ctrld service
+`ctrld` will attempt to interface with dnsmasq whenever possible and set itself as the upstream, while running on port 5354. On FreeBSD based OSes, `ctrld` will terminate dnsmasq and unbound in order to be able to listen on port 53 directly.  
 
-  Flags:
-    -h, --help   help for service
-
-  Global Flags:
-    -v, --verbose count   verbose log output, "-v" basic logging, "-vv" debug level logging
-
-  Use "ctrld service [command] --help" for more information about a command.
-```
 
 ### Control D Auto Configuration
-Application can be started with a specific resolver config, instead of the default one. Simply supply your resolver ID with a `--cd` flag, when using the `run` (foreground) or `start` (service) modes. 
+Application can be started with a specific resolver config, instead of the default one. Simply supply your Resolver ID with a `--cd` flag, when using the `run` (foreground) or `start` (service) modes. 
 
 The following command will start the application in foreground mode, using the free "p2" resolver, which blocks Ads & Trackers. 
 
@@ -121,22 +168,22 @@ The following command will start the application in foreground mode, using the f
 ./ctrld run --cd p2
 ```
 
-Alternatively, you can use your own personal Control D Device resolver, and start the application in service mode. Your resolver ID is the part after the slash of your DNS-over-HTTPS resolver. ie. https://dns.controld.com/abcd1234
+Alternatively, you can use your own personal Control D Device resolver, and start the application in service mode. Your resolver ID is displayed on the "Show Resolvers" screen for the relevant Control D Device. 
 
 ```shell
 ./ctrld start --cd abcd1234
 ```
 
-Once you run the above command, the following things will happen:
+Once you run the above commands (in service mode only), the following things will happen:
 - You resolver configuration will be fetched from the API, and config file templated with the resolver data
-- Application will start as a service, and keep running (even after reboot) until you run the `stop` or `service uninstall` sub-commands
+- Application will start as a service, and keep running (even after reboot) until you run the `stop` or `uninstall` sub-commands
 - Your default network interface will be updated to use the listener started by the service
 - All OS DNS queries will be sent to the listener
 
-## Configuration
+# Configuration
 See [Configuration Docs](docs/config.md).
 
-### Example 
+## Example 
 - Start `listener.0` on 127.0.0.1:53
 - Accept queries from any source address
 - Send all queries to `upstream.0` via DoH protocol
@@ -146,8 +193,8 @@ See [Configuration Docs](docs/config.md).
 [listener]
 
   [listener.0]
-    ip = "127.0.0.1"
-    port = 53
+    ip = ""
+    port = 0
     restricted = false
 
 [network]
@@ -178,17 +225,18 @@ See [Configuration Docs](docs/config.md).
 
 ```
 
-### Advanced
+`ctrld` will pick a working config for `listener.0` then writing the default config to disk for the first run.
+
+## Advanced Configuration
 The above is the most basic example, which will work out of the box. If you're looking to do advanced configurations using policies, see [Configuration Docs](docs/config.md) for complete documentation of the config file.
 
 You can also supply configuration via launch argeuments, in [Ephemeral Mode](docs/ephemeral_mode.md).
 
 ## Contributing
-
 See [Contribution Guideline](./docs/contributing.md)
 
 ## Roadmap
 The following functionality is on the roadmap and will be available in future releases. 
-- Router self-installation
-- Client hostname/MAC passthrough
-- Prometheus metrics exporter 
+- DNS intercept mode
+- Direct listener mode
+- Support for more routers (let us know which ones)

client_info.go

@@ -0,0 +1,22 @@
+package ctrld
+
+// ClientInfoCtxKey is the context key to store client info.
+type ClientInfoCtxKey struct{}
+
+// ClientInfo represents ctrld's clients information.
+type ClientInfo struct {
+	Mac          string
+	IP           string
+	Hostname     string
+	Self         bool
+	ClientIDPref string
+}
+
+// LeaseFileFormat specifies the format of DHCP lease file.
+type LeaseFileFormat string
+
+const (
+	Dnsmasq  LeaseFileFormat = "dnsmasq"
+	IscDhcpd LeaseFileFormat = "isc-dhcpd"
+	KeaDHCP4 LeaseFileFormat = "kea-dhcp4"
+)

cmd/cli/ad_others.go

@@ -0,0 +1,10 @@
+//go:build !windows
+
+package cli
+
+import (
+	"github.com/Control-D-Inc/ctrld"
+)
+
+// addExtraSplitDnsRule adds split DNS rule if present.
+func addExtraSplitDnsRule(_ *ctrld.Config) bool { return false }

cmd/cli/ad_windows.go

@@ -0,0 +1,49 @@
+package cli
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/Control-D-Inc/ctrld"
+)
+
+// addExtraSplitDnsRule adds split DNS rule for domain if it's part of active directory.
+func addExtraSplitDnsRule(cfg *ctrld.Config) bool {
+	domain, err := getActiveDirectoryDomain()
+	if err != nil {
+		mainLog.Load().Debug().Msgf("unable to get active directory domain: %v", err)
+		return false
+	}
+	if domain == "" {
+		mainLog.Load().Debug().Msg("no active directory domain found")
+		return false
+	}
+	// Network rules are lowercase during toml config marshaling,
+	// lowercase the domain here too for consistency.
+	domain = strings.ToLower(domain)
+	for n, lc := range cfg.Listener {
+		if lc.Policy == nil {
+			lc.Policy = &ctrld.ListenerPolicyConfig{}
+		}
+		domainRule := "*." + strings.TrimPrefix(domain, ".")
+		for _, rule := range lc.Policy.Rules {
+			if _, ok := rule[domainRule]; ok {
+				mainLog.Load().Debug().Msgf("domain rule already exist for listener.%s", n)
+				return false
+			}
+		}
+		mainLog.Load().Debug().Msgf("adding active directory domain for listener.%s", n)
+		lc.Policy.Rules = append(lc.Policy.Rules, ctrld.Rule{domainRule: []string{}})
+	}
+	return true
+}
+
+// getActiveDirectoryDomain returns AD domain name of this computer.
+func getActiveDirectoryDomain() (string, error) {
+	cmd := "$obj = Get-WmiObject Win32_ComputerSystem; if ($obj.PartOfDomain) { $obj.Domain }"
+	output, err := powershell(cmd)
+	if err != nil {
+		return "", fmt.Errorf("failed to get domain name: %w, output:\n\n%s", err, string(output))
+	}
+	return string(output), nil
+}

cmd/cli/cli_test.go

@@ -0,0 +1,46 @@
+package cli
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func Test_writeConfigFile(t *testing.T) {
+	tmpdir := t.TempDir()
+	// simulate --config CLI flag by setting configPath manually.
+	configPath = filepath.Join(tmpdir, "ctrld.toml")
+	_, err := os.Stat(configPath)
+	assert.True(t, os.IsNotExist(err))
+
+	assert.NoError(t, writeConfigFile(&cfg))
+
+	_, err = os.Stat(configPath)
+	require.NoError(t, err)
+}
+
+func Test_isStableVersion(t *testing.T) {
+	tests := []struct {
+		name     string
+		ver      string
+		isStable bool
+	}{
+		{"stable", "v1.3.5", true},
+		{"pre", "v1.3.5-next", false},
+		{"pre with commit hash", "v1.3.5-next-asdf", false},
+		{"dev", "dev", false},
+		{"empty", "dev", false},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			if got := isStableVersion(tc.ver); got != tc.isStable {
+				t.Errorf("unexpected result for %s, want: %v, got: %v", tc.ver, tc.isStable, got)
+			}
+		})
+	}
+}

cmd/cli/conn.go

@@ -0,0 +1,51 @@
+package cli
+
+import (
+	"net"
+	"time"
+)
+
+// logConn wraps a net.Conn, override the Write behavior.
+// runCmd uses this wrapper, so as long as startCmd finished,
+// ctrld log won't be flushed with un-necessary write errors.
+type logConn struct {
+	conn net.Conn
+}
+
+func (lc *logConn) Read(b []byte) (n int, err error) {
+	return lc.conn.Read(b)
+}
+
+func (lc *logConn) Close() error {
+	return lc.conn.Close()
+}
+
+func (lc *logConn) LocalAddr() net.Addr {
+	return lc.conn.LocalAddr()
+}
+
+func (lc *logConn) RemoteAddr() net.Addr {
+	return lc.conn.RemoteAddr()
+}
+
+func (lc *logConn) SetDeadline(t time.Time) error {
+	return lc.conn.SetDeadline(t)
+}
+
+func (lc *logConn) SetReadDeadline(t time.Time) error {
+	return lc.conn.SetReadDeadline(t)
+}
+
+func (lc *logConn) SetWriteDeadline(t time.Time) error {
+	return lc.conn.SetWriteDeadline(t)
+}
+
+func (lc *logConn) Write(b []byte) (int, error) {
+	// Write performs writes with underlying net.Conn, ignore any errors happen.
+	// "ctrld run" command use this wrapper to report errors to "ctrld start".
+	// If no error occurred, "ctrld start" may finish before "ctrld run" attempt
+	// to close the connection, so ignore errors conservatively here, prevent
+	// un-necessary error "write to closed connection" flushed to ctrld log.
+	_, _ = lc.conn.Write(b)
+	return len(b), nil
+}

cmd/cli/control_client.go

@@ -0,0 +1,34 @@
+package cli
+
+import (
+	"context"
+	"io"
+	"net"
+	"net/http"
+	"time"
+)
+
+type controlClient struct {
+	c *http.Client
+}
+
+func newControlClient(addr string) *controlClient {
+	return &controlClient{c: &http.Client{
+		Transport: &http.Transport{
+			DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
+				d := net.Dialer{}
+				return d.DialContext(ctx, "unix", addr)
+			},
+		},
+		Timeout: time.Second * 30,
+	}}
+}
+
+func (c *controlClient) post(path string, data io.Reader) (*http.Response, error) {
+	return c.c.Post("http://unix"+path, contentTypeJson, data)
+}
+
+// deactivationRequest represents request for validating deactivation pin.
+type deactivationRequest struct {
+	Pin int64 `json:"pin"`
+}

cmd/cli/control_server.go

@@ -0,0 +1,221 @@
+package cli
+
+import (
+	"context"
+	"encoding/json"
+	"net"
+	"net/http"
+	"os"
+	"reflect"
+	"sort"
+	"time"
+
+	"github.com/kardianos/service"
+	dto "github.com/prometheus/client_model/go"
+
+	"github.com/Control-D-Inc/ctrld"
+	"github.com/Control-D-Inc/ctrld/internal/controld"
+)
+
+const (
+	contentTypeJson  = "application/json"
+	listClientsPath  = "/clients"
+	startedPath      = "/started"
+	reloadPath       = "/reload"
+	deactivationPath = "/deactivation"
+	cdPath           = "/cd"
+	ifacePath        = "/iface"
+)
+
+type controlServer struct {
+	server *http.Server
+	mux    *http.ServeMux
+	addr   string
+}
+
+func newControlServer(addr string) (*controlServer, error) {
+	mux := http.NewServeMux()
+	s := &controlServer{
+		server: &http.Server{Handler: mux},
+		mux:    mux,
+	}
+	s.addr = addr
+	return s, nil
+}
+
+func (s *controlServer) start() error {
+	_ = os.Remove(s.addr)
+	unixListener, err := net.Listen("unix", s.addr)
+	if l, ok := unixListener.(*net.UnixListener); ok {
+		l.SetUnlinkOnClose(true)
+	}
+	if err != nil {
+		return err
+	}
+	go s.server.Serve(unixListener)
+	return nil
+}
+
+func (s *controlServer) stop() error {
+	_ = os.Remove(s.addr)
+	ctx, cancel := context.WithTimeout(context.Background(), time.Second*2)
+	defer cancel()
+	return s.server.Shutdown(ctx)
+}
+
+func (s *controlServer) register(pattern string, handler http.Handler) {
+	s.mux.Handle(pattern, jsonResponse(handler))
+}
+
+func (p *prog) registerControlServerHandler() {
+	p.cs.register(listClientsPath, http.HandlerFunc(func(w http.ResponseWriter, request *http.Request) {
+		clients := p.ciTable.ListClients()
+		sort.Slice(clients, func(i, j int) bool {
+			return clients[i].IP.Less(clients[j].IP)
+		})
+		if p.metricsQueryStats.Load() {
+			for _, client := range clients {
+				client.IncludeQueryCount = true
+				dm := &dto.Metric{}
+				m, err := statsClientQueriesCount.MetricVec.GetMetricWithLabelValues(
+					client.IP.String(),
+					client.Mac,
+					client.Hostname,
+				)
+				if err != nil {
+					mainLog.Load().Debug().Err(err).Msgf("could not get metrics for client: %v", client)
+					continue
+				}
+				if err := m.Write(dm); err == nil {
+					client.QueryCount = int64(dm.Counter.GetValue())
+				}
+			}
+		}
+
+		if err := json.NewEncoder(w).Encode(&clients); err != nil {
+			http.Error(w, err.Error(), http.StatusInternalServerError)
+			return
+		}
+	}))
+	p.cs.register(startedPath, http.HandlerFunc(func(w http.ResponseWriter, request *http.Request) {
+		select {
+		case <-p.onStartedDone:
+			w.WriteHeader(http.StatusOK)
+		case <-time.After(10 * time.Second):
+			w.WriteHeader(http.StatusRequestTimeout)
+		}
+	}))
+	p.cs.register(reloadPath, http.HandlerFunc(func(w http.ResponseWriter, request *http.Request) {
+		listeners := make(map[string]*ctrld.ListenerConfig)
+		p.mu.Lock()
+		for k, v := range p.cfg.Listener {
+			listeners[k] = &ctrld.ListenerConfig{
+				IP:   v.IP,
+				Port: v.Port,
+			}
+		}
+		oldSvc := p.cfg.Service
+		p.mu.Unlock()
+		if err := p.sendReloadSignal(); err != nil {
+			mainLog.Load().Err(err).Msg("could not send reload signal")
+			http.Error(w, err.Error(), http.StatusInternalServerError)
+			return
+		}
+		select {
+		case <-p.reloadDoneCh:
+		case <-time.After(5 * time.Second):
+			http.Error(w, "timeout waiting for ctrld reload", http.StatusInternalServerError)
+			return
+		}
+
+		p.mu.Lock()
+		defer p.mu.Unlock()
+
+		// Checking for cases that we could not do a reload.
+
+		// 1. Listener config ip or port changes.
+		for k, v := range p.cfg.Listener {
+			l := listeners[k]
+			if l == nil || l.IP != v.IP || l.Port != v.Port {
+				w.WriteHeader(http.StatusCreated)
+				return
+			}
+		}
+
+		// 2. Service config changes.
+		if !reflect.DeepEqual(oldSvc, p.cfg.Service) {
+			w.WriteHeader(http.StatusCreated)
+			return
+		}
+
+		// Otherwise, reload is done.
+		w.WriteHeader(http.StatusOK)
+	}))
+	p.cs.register(deactivationPath, http.HandlerFunc(func(w http.ResponseWriter, request *http.Request) {
+		// Non-cd mode always allowing deactivation.
+		if cdUID == "" {
+			w.WriteHeader(http.StatusOK)
+			return
+		}
+
+		// Re-fetch pin code from API.
+		if rc, err := controld.FetchResolverConfig(cdUID, rootCmd.Version, cdDev); rc != nil {
+			if rc.DeactivationPin != nil {
+				cdDeactivationPin.Store(*rc.DeactivationPin)
+			} else {
+				cdDeactivationPin.Store(defaultDeactivationPin)
+			}
+		} else {
+			mainLog.Load().Warn().Err(err).Msg("could not re-fetch deactivation pin code")
+		}
+
+		// If pin code not set, allowing deactivation.
+		if deactivationPinNotSet() {
+			w.WriteHeader(http.StatusOK)
+			return
+		}
+
+		var req deactivationRequest
+		if err := json.NewDecoder(request.Body).Decode(&req); err != nil {
+			w.WriteHeader(http.StatusPreconditionFailed)
+			mainLog.Load().Err(err).Msg("invalid deactivation request")
+			return
+		}
+
+		code := http.StatusForbidden
+		switch req.Pin {
+		case cdDeactivationPin.Load():
+			code = http.StatusOK
+		case defaultDeactivationPin:
+			// If the pin code was set, but users do not provide --pin, return proper code to client.
+			code = http.StatusBadRequest
+		}
+		w.WriteHeader(code)
+	}))
+	p.cs.register(cdPath, http.HandlerFunc(func(w http.ResponseWriter, request *http.Request) {
+		if cdUID != "" {
+			w.WriteHeader(http.StatusOK)
+			w.Write([]byte(cdUID))
+			return
+		}
+		w.WriteHeader(http.StatusBadRequest)
+	}))
+	p.cs.register(ifacePath, http.HandlerFunc(func(w http.ResponseWriter, request *http.Request) {
+		// p.setDNS is only called when running as a service
+		if !service.Interactive() {
+			<-p.csSetDnsDone
+			if p.csSetDnsOk {
+				w.Write([]byte(iface))
+				return
+			}
+		}
+		w.WriteHeader(http.StatusBadRequest)
+	}))
+}
+
+func jsonResponse(next http.Handler) http.Handler {
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Content-Type", "application/json")
+		next.ServeHTTP(w, r)
+	})
+}

cmd/cli/control_server_test.go

@@ -0,0 +1,54 @@
+package cli
+
+import (
+	"bytes"
+	"io"
+	"net/http"
+	"os"
+	"testing"
+)
+
+func TestControlServer(t *testing.T) {
+	f, err := os.CreateTemp("", "")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.Remove(f.Name())
+	f.Close()
+
+	s, err := newControlServer(f.Name())
+	if err != nil {
+		t.Fatal(err)
+	}
+	pattern := "/ping"
+	respBody := []byte("pong")
+	s.register(pattern, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		_, _ = w.Write(respBody)
+	}))
+	if err := s.start(); err != nil {
+		t.Fatal(err)
+	}
+
+	c := newControlClient(f.Name())
+	resp, err := c.post(pattern, nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode != 200 {
+		t.Fatalf("unepxected response code: %d", resp.StatusCode)
+	}
+	if ct := resp.Header.Get("content-type"); ct != contentTypeJson {
+		t.Fatalf("unexpected content type: %s", ct)
+	}
+	buf, err := io.ReadAll(resp.Body)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !bytes.Equal(buf, respBody) {
+		t.Errorf("unexpected response body, want: %q, got: %q", string(respBody), string(buf))
+	}
+	if err := s.stop(); err != nil {
+		t.Fatal(err)
+	}
+}

cmd/cli/dns.go

@@ -1,4 +1,4 @@
-package main
+package cli
 
 //lint:ignore U1000 use in os_linux.go
 type getDNS func(iface string) []string

cmd/cli/dns_proxy_test.go

@@ -0,0 +1,441 @@
+package cli
+
+import (
+	"context"
+	"net"
+	"testing"
+	"time"
+
+	"github.com/miekg/dns"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+
+	"github.com/Control-D-Inc/ctrld"
+	"github.com/Control-D-Inc/ctrld/internal/dnscache"
+	"github.com/Control-D-Inc/ctrld/testhelper"
+)
+
+func Test_wildcardMatches(t *testing.T) {
+	tests := []struct {
+		name     string
+		wildcard string
+		domain   string
+		match    bool
+	}{
+		{"domain - prefix parent should not match", "*.windscribe.com", "windscribe.com", false},
+		{"domain - prefix", "*.windscribe.com", "anything.windscribe.com", true},
+		{"domain - prefix not match other s", "*.windscribe.com", "example.com", false},
+		{"domain - prefix not match s in name", "*.windscribe.com", "wwindscribe.com", false},
+		{"domain - suffix", "suffix.*", "suffix.windscribe.com", true},
+		{"domain - suffix not match other", "suffix.*", "suffix1.windscribe.com", false},
+		{"domain - both", "suffix.*.windscribe.com", "suffix.anything.windscribe.com", true},
+		{"domain - both not match", "suffix.*.windscribe.com", "suffix1.suffix.windscribe.com", false},
+		{"domain - case-insensitive", "*.WINDSCRIBE.com", "anything.windscribe.com", true},
+		{"mac - prefix", "*:98:05:b4:2b", "d4:67:98:05:b4:2b", true},
+		{"mac - prefix not match other s", "*:98:05:b4:2b", "0d:ba:54:09:94:2c", false},
+		{"mac - prefix not match s in name", "*:98:05:b4:2b", "e4:67:97:05:b4:2b", false},
+		{"mac - suffix", "d4:67:98:*", "d4:67:98:05:b4:2b", true},
+		{"mac - suffix not match other", "d4:67:98:*", "d4:67:97:15:b4:2b", false},
+		{"mac - both", "d4:67:98:*:b4:2b", "d4:67:98:05:b4:2b", true},
+		{"mac - both not match", "d4:67:98:*:b4:2b", "d4:67:97:05:c4:2b", false},
+	}
+
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			if got := wildcardMatches(tc.wildcard, tc.domain); got != tc.match {
+				t.Errorf("unexpected result, wildcard: %s, domain: %s, want: %v, got: %v", tc.wildcard, tc.domain, tc.match, got)
+			}
+		})
+	}
+}
+
+func Test_canonicalName(t *testing.T) {
+	tests := []struct {
+		name      string
+		domain    string
+		canonical string
+	}{
+		{"fqdn to canonical", "windscribe.com.", "windscribe.com"},
+		{"already canonical", "windscribe.com", "windscribe.com"},
+		{"case insensitive", "Windscribe.Com.", "windscribe.com"},
+	}
+
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			if got := canonicalName(tc.domain); got != tc.canonical {
+				t.Errorf("unexpected result, want: %s, got: %s", tc.canonical, got)
+			}
+		})
+	}
+}
+
+func Test_prog_upstreamFor(t *testing.T) {
+	cfg := testhelper.SampleConfig(t)
+	p := &prog{cfg: cfg}
+	p.um = newUpstreamMonitor(p.cfg)
+	p.lanLoopGuard = newLoopGuard()
+	p.ptrLoopGuard = newLoopGuard()
+	for _, nc := range p.cfg.Network {
+		for _, cidr := range nc.Cidrs {
+			_, ipNet, err := net.ParseCIDR(cidr)
+			if err != nil {
+				t.Fatal(err)
+			}
+			nc.IPNets = append(nc.IPNets, ipNet)
+		}
+	}
+
+	tests := []struct {
+		name               string
+		ip                 string
+		mac                string
+		defaultUpstreamNum string
+		lc                 *ctrld.ListenerConfig
+		domain             string
+		upstreams          []string
+		matched            bool
+		testLogMsg         string
+	}{
+		{"Policy map matches", "192.168.0.1:0", "", "0", p.cfg.Listener["0"], "abc.xyz", []string{"upstream.1", "upstream.0"}, true, ""},
+		{"Policy split matches", "192.168.0.1:0", "", "0", p.cfg.Listener["0"], "abc.ru", []string{"upstream.1"}, true, ""},
+		{"Policy map for other network matches", "192.168.1.2:0", "", "0", p.cfg.Listener["0"], "abc.xyz", []string{"upstream.0"}, true, ""},
+		{"No policy map for listener", "192.168.1.2:0", "", "1", p.cfg.Listener["1"], "abc.ru", []string{"upstream.1"}, false, ""},
+		{"unenforced loging", "192.168.1.2:0", "", "0", p.cfg.Listener["0"], "abc.ru", []string{"upstream.1"}, true, "My Policy, network.1 (unenforced), *.ru -> [upstream.1]"},
+		{"Policy Macs matches upper", "192.168.0.1:0", "14:45:A0:67:83:0A", "0", p.cfg.Listener["0"], "abc.xyz", []string{"upstream.2"}, true, "14:45:a0:67:83:0a"},
+		{"Policy Macs matches lower", "192.168.0.1:0", "14:54:4a:8e:08:2d", "0", p.cfg.Listener["0"], "abc.xyz", []string{"upstream.2"}, true, "14:54:4a:8e:08:2d"},
+		{"Policy Macs matches case-insensitive", "192.168.0.1:0", "14:54:4A:8E:08:2D", "0", p.cfg.Listener["0"], "abc.xyz", []string{"upstream.2"}, true, "14:54:4a:8e:08:2d"},
+	}
+
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			for _, network := range []string{"udp", "tcp"} {
+				var (
+					addr net.Addr
+					err  error
+				)
+				switch network {
+				case "udp":
+					addr, err = net.ResolveUDPAddr(network, tc.ip)
+				case "tcp":
+					addr, err = net.ResolveTCPAddr(network, tc.ip)
+				}
+				require.NoError(t, err)
+				require.NotNil(t, addr)
+				ctx := context.WithValue(context.Background(), ctrld.ReqIdCtxKey{}, requestID())
+				ufr := p.upstreamFor(ctx, tc.defaultUpstreamNum, tc.lc, addr, tc.mac, tc.domain)
+				p.proxy(ctx, &proxyRequest{
+					msg: newDnsMsgWithHostname("foo", dns.TypeA),
+					ufr: ufr,
+				})
+				assert.Equal(t, tc.matched, ufr.matched)
+				assert.Equal(t, tc.upstreams, ufr.upstreams)
+				if tc.testLogMsg != "" {
+					assert.Contains(t, logOutput.String(), tc.testLogMsg)
+				}
+			}
+		})
+	}
+}
+
+func TestCache(t *testing.T) {
+	cfg := testhelper.SampleConfig(t)
+	prog := &prog{cfg: cfg}
+	for _, nc := range prog.cfg.Network {
+		for _, cidr := range nc.Cidrs {
+			_, ipNet, err := net.ParseCIDR(cidr)
+			if err != nil {
+				t.Fatal(err)
+			}
+			nc.IPNets = append(nc.IPNets, ipNet)
+		}
+	}
+	cacher, err := dnscache.NewLRUCache(4096)
+	require.NoError(t, err)
+	prog.cache = cacher
+
+	msg := new(dns.Msg)
+	msg.SetQuestion("example.com", dns.TypeA)
+	msg.MsgHdr.RecursionDesired = true
+	answer1 := new(dns.Msg)
+	answer1.SetRcode(msg, dns.RcodeSuccess)
+
+	prog.cache.Add(dnscache.NewKey(msg, "upstream.1"), dnscache.NewValue(answer1, time.Now().Add(time.Minute)))
+	answer2 := new(dns.Msg)
+	answer2.SetRcode(msg, dns.RcodeRefused)
+	prog.cache.Add(dnscache.NewKey(msg, "upstream.0"), dnscache.NewValue(answer2, time.Now().Add(time.Minute)))
+
+	req1 := &proxyRequest{
+		msg:            msg,
+		ci:             nil,
+		failoverRcodes: nil,
+		ufr: &upstreamForResult{
+			upstreams:      []string{"upstream.1"},
+			matchedPolicy:  "",
+			matchedNetwork: "",
+			matchedRule:    "",
+			matched:        false,
+		},
+	}
+	req2 := &proxyRequest{
+		msg:            msg,
+		ci:             nil,
+		failoverRcodes: nil,
+		ufr: &upstreamForResult{
+			upstreams:      []string{"upstream.0"},
+			matchedPolicy:  "",
+			matchedNetwork: "",
+			matchedRule:    "",
+			matched:        false,
+		},
+	}
+	got1 := prog.proxy(context.Background(), req1)
+	got2 := prog.proxy(context.Background(), req2)
+	assert.NotSame(t, got1, got2)
+	assert.Equal(t, answer1.Rcode, got1.answer.Rcode)
+	assert.Equal(t, answer2.Rcode, got2.answer.Rcode)
+}
+
+func Test_ipAndMacFromMsg(t *testing.T) {
+	tests := []struct {
+		name    string
+		ip      string
+		wantIp  bool
+		mac     string
+		wantMac bool
+	}{
+		{"has ip v4 and mac", "1.2.3.4", true, "4c:20:b8:ab:87:1b", true},
+		{"has ip v6 and mac", "2606:1a40:3::1", true, "4c:20:b8:ab:87:1b", true},
+		{"no ip", "1.2.3.4", false, "4c:20:b8:ab:87:1b", false},
+		{"no mac", "1.2.3.4", false, "4c:20:b8:ab:87:1b", false},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			ip := net.ParseIP(tc.ip)
+			if ip == nil {
+				t.Fatal("missing IP")
+			}
+			hw, err := net.ParseMAC(tc.mac)
+			if err != nil {
+				t.Fatal(err)
+			}
+			m := new(dns.Msg)
+			m.SetQuestion("example.com.", dns.TypeA)
+			o := &dns.OPT{Hdr: dns.RR_Header{Name: ".", Rrtype: dns.TypeOPT}}
+			if tc.wantMac {
+				ec1 := &dns.EDNS0_LOCAL{Code: EDNS0_OPTION_MAC, Data: hw}
+				o.Option = append(o.Option, ec1)
+			}
+			if tc.wantIp {
+				ec2 := &dns.EDNS0_SUBNET{Address: ip}
+				o.Option = append(o.Option, ec2)
+			}
+			m.Extra = append(m.Extra, o)
+			gotIP, gotMac := ipAndMacFromMsg(m)
+			if tc.wantMac && gotMac != tc.mac {
+				t.Errorf("mismatch, want: %q, got: %q", tc.mac, gotMac)
+			}
+			if !tc.wantMac && gotMac != "" {
+				t.Errorf("unexpected mac: %q", gotMac)
+			}
+			if tc.wantIp && gotIP != tc.ip {
+				t.Errorf("mismatch, want: %q, got: %q", tc.ip, gotIP)
+			}
+			if !tc.wantIp && gotIP != "" {
+				t.Errorf("unexpected ip: %q", gotIP)
+			}
+		})
+	}
+}
+
+func Test_remoteAddrFromMsg(t *testing.T) {
+	loopbackIP := net.ParseIP("127.0.0.1")
+	tests := []struct {
+		name string
+		addr net.Addr
+		ci   *ctrld.ClientInfo
+		want string
+	}{
+		{"tcp", &net.TCPAddr{IP: loopbackIP, Port: 12345}, &ctrld.ClientInfo{IP: "192.168.1.10"}, "192.168.1.10:12345"},
+		{"udp", &net.UDPAddr{IP: loopbackIP, Port: 12345}, &ctrld.ClientInfo{IP: "192.168.1.11"}, "192.168.1.11:12345"},
+		{"nil client info", &net.UDPAddr{IP: loopbackIP, Port: 12345}, nil, "127.0.0.1:12345"},
+		{"empty ip", &net.UDPAddr{IP: loopbackIP, Port: 12345}, &ctrld.ClientInfo{}, "127.0.0.1:12345"},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			addr := spoofRemoteAddr(tc.addr, tc.ci)
+			if addr.String() != tc.want {
+				t.Errorf("unexpected result, want: %q, got: %q", tc.want, addr.String())
+			}
+		})
+	}
+}
+
+func Test_ipFromARPA(t *testing.T) {
+	tests := []struct {
+		IP   string
+		ARPA string
+	}{
+		{"1.2.3.4", "4.3.2.1.in-addr.arpa."},
+		{"245.110.36.114", "114.36.110.245.in-addr.arpa."},
+		{"::ffff:12.34.56.78", "78.56.34.12.in-addr.arpa."},
+		{"::1", "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa."},
+		{"1::", "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.0.0.0.ip6.arpa."},
+		{"1234:567::89a:bcde", "e.d.c.b.a.9.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.7.6.5.0.4.3.2.1.ip6.arpa."},
+		{"1234:567:fefe:bcbc:adad:9e4a:89a:bcde", "e.d.c.b.a.9.8.0.a.4.e.9.d.a.d.a.c.b.c.b.e.f.e.f.7.6.5.0.4.3.2.1.ip6.arpa."},
+		{"", "asd.in-addr.arpa."},
+		{"", "asd.ip6.arpa."},
+	}
+
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.IP, func(t *testing.T) {
+			t.Parallel()
+			if got := ipFromARPA(tc.ARPA); !got.Equal(net.ParseIP(tc.IP)) {
+				t.Errorf("unexpected ip, want: %s, got: %s", tc.IP, got)
+			}
+		})
+	}
+}
+
+func newDnsMsgWithClientIP(ip string) *dns.Msg {
+	m := new(dns.Msg)
+	m.SetQuestion("example.com.", dns.TypeA)
+	o := &dns.OPT{Hdr: dns.RR_Header{Name: ".", Rrtype: dns.TypeOPT}}
+	o.Option = append(o.Option, &dns.EDNS0_SUBNET{Address: net.ParseIP(ip)})
+	m.Extra = append(m.Extra, o)
+	return m
+}
+
+func Test_stripClientSubnet(t *testing.T) {
+	tests := []struct {
+		name       string
+		msg        *dns.Msg
+		wantSubnet bool
+	}{
+		{"no edns0", new(dns.Msg), false},
+		{"loopback IP v4", newDnsMsgWithClientIP("127.0.0.1"), false},
+		{"loopback IP v6", newDnsMsgWithClientIP("::1"), false},
+		{"private IP v4", newDnsMsgWithClientIP("192.168.1.123"), false},
+		{"private IP v6", newDnsMsgWithClientIP("fd12:3456:789a:1::1"), false},
+		{"public IP", newDnsMsgWithClientIP("1.1.1.1"), true},
+		{"invalid IP", newDnsMsgWithClientIP(""), true},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			stripClientSubnet(tc.msg)
+			hasSubnet := false
+			if opt := tc.msg.IsEdns0(); opt != nil {
+				for _, s := range opt.Option {
+					if _, ok := s.(*dns.EDNS0_SUBNET); ok {
+						hasSubnet = true
+					}
+				}
+			}
+			if tc.wantSubnet != hasSubnet {
+				t.Errorf("unexpected result, want: %v, got: %v", tc.wantSubnet, hasSubnet)
+			}
+		})
+	}
+}
+
+func newDnsMsgWithHostname(hostname string, typ uint16) *dns.Msg {
+	m := new(dns.Msg)
+	m.SetQuestion(hostname, typ)
+	return m
+}
+
+func Test_isLanHostnameQuery(t *testing.T) {
+	tests := []struct {
+		name               string
+		msg                *dns.Msg
+		isLanHostnameQuery bool
+	}{
+		{"A", newDnsMsgWithHostname("foo", dns.TypeA), true},
+		{"AAAA", newDnsMsgWithHostname("foo", dns.TypeAAAA), true},
+		{"A not LAN", newDnsMsgWithHostname("example.com", dns.TypeA), false},
+		{"AAAA not LAN", newDnsMsgWithHostname("example.com", dns.TypeAAAA), false},
+		{"Not A or AAAA", newDnsMsgWithHostname("foo", dns.TypeTXT), false},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			if got := isLanHostnameQuery(tc.msg); tc.isLanHostnameQuery != got {
+				t.Errorf("unexpected result, want: %v, got: %v", tc.isLanHostnameQuery, got)
+			}
+		})
+	}
+}
+
+func newDnsMsgPtr(ip string, t *testing.T) *dns.Msg {
+	t.Helper()
+	m := new(dns.Msg)
+	ptr, err := dns.ReverseAddr(ip)
+	if err != nil {
+		t.Fatal(err)
+	}
+	m.SetQuestion(ptr, dns.TypePTR)
+	return m
+}
+
+func Test_isPrivatePtrLookup(t *testing.T) {
+	tests := []struct {
+		name               string
+		msg                *dns.Msg
+		isPrivatePtrLookup bool
+	}{
+		// RFC 1918 allocates 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16 as
+		{"10.0.0.0/8", newDnsMsgPtr("10.0.0.123", t), true},
+		{"172.16.0.0/12", newDnsMsgPtr("172.16.0.123", t), true},
+		{"192.168.0.0/16", newDnsMsgPtr("192.168.1.123", t), true},
+		{"CGNAT", newDnsMsgPtr("100.66.27.28", t), true},
+		{"Loopback", newDnsMsgPtr("127.0.0.1", t), true},
+		{"Link Local Unicast", newDnsMsgPtr("fe80::69f6:e16e:8bdb:433f", t), true},
+		{"Public IP", newDnsMsgPtr("8.8.8.8", t), false},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			if got := isPrivatePtrLookup(tc.msg); tc.isPrivatePtrLookup != got {
+				t.Errorf("unexpected result, want: %v, got: %v", tc.isPrivatePtrLookup, got)
+			}
+		})
+	}
+}
+
+func Test_isWanClient(t *testing.T) {
+	tests := []struct {
+		name        string
+		addr        net.Addr
+		isWanClient bool
+	}{
+		// RFC 1918 allocates 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16 as
+		{"10.0.0.0/8", &net.UDPAddr{IP: net.ParseIP("10.0.0.123")}, false},
+		{"172.16.0.0/12", &net.UDPAddr{IP: net.ParseIP("172.16.0.123")}, false},
+		{"192.168.0.0/16", &net.UDPAddr{IP: net.ParseIP("192.168.1.123")}, false},
+		{"CGNAT", &net.UDPAddr{IP: net.ParseIP("100.66.27.28")}, false},
+		{"Loopback", &net.UDPAddr{IP: net.ParseIP("127.0.0.1")}, false},
+		{"Link Local Unicast", &net.UDPAddr{IP: net.ParseIP("fe80::69f6:e16e:8bdb:433f")}, false},
+		{"Public", &net.UDPAddr{IP: net.ParseIP("8.8.8.8")}, true},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			if got := isWanClient(tc.addr); tc.isWanClient != got {
+				t.Errorf("unexpected result, want: %v, got: %v", tc.isWanClient, got)
+			}
+		})
+	}
+}

cmd/cli/hostname.go

@@ -0,0 +1,14 @@
+package cli
+
+import "regexp"
+
+// validHostname reports whether hostname is a valid hostname.
+// A valid hostname contains 3 -> 64 characters and conform to RFC1123.
+func validHostname(hostname string) bool {
+	hostnameLen := len(hostname)
+	if hostnameLen < 3 || hostnameLen > 64 {
+		return false
+	}
+	validHostnameRfc1123 := regexp.MustCompile(`^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$`)
+	return validHostnameRfc1123.MatchString(hostname)
+}

cmd/cli/hostname_test.go

@@ -0,0 +1,35 @@
+package cli
+
+import (
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func Test_validHostname(t *testing.T) {
+	tests := []struct {
+		name     string
+		hostname string
+		valid    bool
+	}{
+		{"localhost", "localhost", true},
+		{"localdomain", "localhost.localdomain", true},
+		{"localhost6", "localhost6.localdomain6", true},
+		{"ip6", "ip6-localhost", true},
+		{"non-domain", "controld", true},
+		{"domain", "controld.com", true},
+		{"empty", "", false},
+		{"min length", "fo", false},
+		{"max length", strings.Repeat("a", 65), false},
+		{"special char", "foo!", false},
+		{"non-ascii", "fooΩ", false},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.hostname, func(t *testing.T) {
+			t.Parallel()
+			assert.True(t, validHostname(tc.hostname) == tc.valid)
+		})
+	}
+}

cmd/cli/library.go

@@ -0,0 +1,19 @@
+package cli
+
+// AppCallback provides hooks for injecting certain functionalities
+// from mobile platforms to main ctrld cli.
+type AppCallback struct {
+	HostName   func() string
+	LanIp      func() string
+	MacAddress func() string
+	Exit       func(error string)
+}
+
+// AppConfig allows overwriting ctrld cli flags from mobile platforms.
+type AppConfig struct {
+	CdUID         string
+	HomeDir       string
+	UpstreamProto string
+	Verbose       int
+	LogPath       string
+}

cmd/cli/loop.go

@@ -0,0 +1,145 @@
+package cli
+
+import (
+	"context"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/miekg/dns"
+
+	"github.com/Control-D-Inc/ctrld"
+)
+
+const (
+	loopTestDomain = ".test"
+	loopTestQtype  = dns.TypeTXT
+)
+
+// newLoopGuard returns new loopGuard.
+func newLoopGuard() *loopGuard {
+	return &loopGuard{inflight: make(map[string]struct{})}
+}
+
+// loopGuard guards against DNS loop, ensuring only one query
+// for a given domain is processed at a time.
+type loopGuard struct {
+	mu       sync.Mutex
+	inflight map[string]struct{}
+}
+
+// TryLock marks the domain as being processed.
+func (lg *loopGuard) TryLock(domain string) bool {
+	lg.mu.Lock()
+	defer lg.mu.Unlock()
+	if _, inflight := lg.inflight[domain]; !inflight {
+		lg.inflight[domain] = struct{}{}
+		return true
+	}
+	return false
+}
+
+// Unlock marks the domain as being done.
+func (lg *loopGuard) Unlock(domain string) {
+	lg.mu.Lock()
+	defer lg.mu.Unlock()
+	delete(lg.inflight, domain)
+}
+
+// isLoop reports whether the given upstream config is detected as having DNS loop.
+func (p *prog) isLoop(uc *ctrld.UpstreamConfig) bool {
+	p.loopMu.Lock()
+	defer p.loopMu.Unlock()
+	return p.loop[uc.UID()]
+}
+
+// detectLoop checks if the given DNS message is initialized sent by ctrld.
+// If yes, marking the corresponding upstream as loop, prevent infinite DNS
+// forwarding loop.
+//
+// See p.checkDnsLoop for more details how it works.
+func (p *prog) detectLoop(msg *dns.Msg) {
+	if len(msg.Question) != 1 {
+		return
+	}
+	q := msg.Question[0]
+	if q.Qtype != loopTestQtype {
+		return
+	}
+	unFQDNname := strings.TrimSuffix(q.Name, ".")
+	uid := strings.TrimSuffix(unFQDNname, loopTestDomain)
+	p.loopMu.Lock()
+	if _, loop := p.loop[uid]; loop {
+		p.loop[uid] = loop
+	}
+	p.loopMu.Unlock()
+}
+
+// checkDnsLoop sends a message to check if there's any DNS forwarding loop
+// with all the upstreams. The way it works based on dnsmasq --dns-loop-detect.
+//
+// - Generating a TXT test query and sending it to all upstream.
+// - The test query is formed by upstream UID and test domain: <uid>.test
+// - If the test query returns to ctrld, mark the corresponding upstream as loop (see p.detectLoop).
+//
+// See: https://thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html
+func (p *prog) checkDnsLoop() {
+	mainLog.Load().Debug().Msg("start checking DNS loop")
+	upstream := make(map[string]*ctrld.UpstreamConfig)
+	p.loopMu.Lock()
+	for n, uc := range p.cfg.Upstream {
+		if p.um.isDown("upstream." + n) {
+			continue
+		}
+		// Do not send test query to external upstream.
+		if !canBeLocalUpstream(uc.Domain) {
+			mainLog.Load().Debug().Msgf("skipping external: upstream.%s", n)
+			continue
+		}
+		uid := uc.UID()
+		p.loop[uid] = false
+		upstream[uid] = uc
+	}
+	p.loopMu.Unlock()
+
+	for uid := range p.loop {
+		msg := loopTestMsg(uid)
+		uc := upstream[uid]
+		// Skipping upstream which is being marked as down.
+		if uc == nil {
+			continue
+		}
+		resolver, err := ctrld.NewResolver(uc)
+		if err != nil {
+			mainLog.Load().Warn().Err(err).Msgf("could not perform loop check for upstream: %q, endpoint: %q", uc.Name, uc.Endpoint)
+			continue
+		}
+		if _, err := resolver.Resolve(context.Background(), msg); err != nil {
+			mainLog.Load().Warn().Err(err).Msgf("could not send DNS loop check query for upstream: %q, endpoint: %q", uc.Name, uc.Endpoint)
+		}
+	}
+	mainLog.Load().Debug().Msg("end checking DNS loop")
+}
+
+// checkDnsLoopTicker performs p.checkDnsLoop every minute.
+func (p *prog) checkDnsLoopTicker(ctx context.Context) {
+	timer := time.NewTicker(time.Minute)
+	defer timer.Stop()
+	for {
+		select {
+		case <-p.stopCh:
+			return
+		case <-ctx.Done():
+			return
+		case <-timer.C:
+			p.checkDnsLoop()
+		}
+	}
+}
+
+// loopTestMsg generates DNS message for checking loop.
+func loopTestMsg(uid string) *dns.Msg {
+	msg := new(dns.Msg)
+	msg.SetQuestion(dns.Fqdn(uid+loopTestDomain), loopTestQtype)
+	return msg
+}

cmd/cli/loop_test.go

@@ -0,0 +1,42 @@
+package cli
+
+import (
+	"sync"
+	"sync/atomic"
+	"testing"
+)
+
+func Test_loopGuard(t *testing.T) {
+	lg := newLoopGuard()
+	key := "foo"
+
+	var i atomic.Int64
+	var started atomic.Int64
+	n := 1000
+	do := func() {
+		locked := lg.TryLock(key)
+		defer lg.Unlock(key)
+		started.Add(1)
+		for started.Load() < 2 {
+			// Wait until at least 2 goroutines started, otherwise, on system with heavy load,
+			// or having only 1 CPU, all goroutines can be scheduled to run consequently.
+		}
+		if locked {
+			i.Add(1)
+		}
+	}
+
+	var wg sync.WaitGroup
+	wg.Add(n)
+	for i := 0; i < n; i++ {
+		go func() {
+			defer wg.Done()
+			do()
+		}()
+	}
+	wg.Wait()
+
+	if i.Load() == int64(n) {
+		t.Fatalf("i must not be increased %d times", n)
+	}
+}

cmd/cli/main.go

@@ -0,0 +1,178 @@
+package cli
+
+import (
+	"io"
+	"os"
+	"path/filepath"
+	"sync/atomic"
+	"time"
+
+	"github.com/kardianos/service"
+	"github.com/rs/zerolog"
+
+	"github.com/Control-D-Inc/ctrld"
+)
+
+var (
+	configPath        string
+	configBase64      string
+	daemon            bool
+	listenAddress     string
+	primaryUpstream   string
+	secondaryUpstream string
+	domains           []string
+	logPath           string
+	homedir           string
+	cacheSize         int
+	cfg               ctrld.Config
+	verbose           int
+	silent            bool
+	cdUID             string
+	cdOrg             string
+	customHostname    string
+	cdDev             bool
+	iface             string
+	ifaceStartStop    string
+	nextdns           string
+	cdUpstreamProto   string
+	deactivationPin   int64
+	skipSelfChecks    bool
+	cleanup           bool
+	startOnly         bool
+
+	mainLog       atomic.Pointer[zerolog.Logger]
+	consoleWriter zerolog.ConsoleWriter
+	noConfigStart bool
+)
+
+const (
+	cdUidFlagName          = "cd"
+	cdOrgFlagName          = "cd-org"
+	customHostnameFlagName = "custom-hostname"
+	nextdnsFlagName        = "nextdns"
+)
+
+func init() {
+	l := zerolog.New(io.Discard)
+	mainLog.Store(&l)
+}
+
+func Main() {
+	ctrld.InitConfig(v, "ctrld")
+	initCLI()
+	if err := rootCmd.Execute(); err != nil {
+		mainLog.Load().Error().Msg(err.Error())
+		os.Exit(1)
+	}
+}
+
+func normalizeLogFilePath(logFilePath string) string {
+	if logFilePath == "" || filepath.IsAbs(logFilePath) || service.Interactive() {
+		return logFilePath
+	}
+	if homedir != "" {
+		return filepath.Join(homedir, logFilePath)
+	}
+	dir, _ := userHomeDir()
+	if dir == "" {
+		return logFilePath
+	}
+	return filepath.Join(dir, logFilePath)
+}
+
+// initConsoleLogging initializes console logging, then storing to mainLog.
+func initConsoleLogging() {
+	consoleWriter = zerolog.NewConsoleWriter(func(w *zerolog.ConsoleWriter) {
+		w.TimeFormat = time.StampMilli
+	})
+	multi := zerolog.MultiLevelWriter(consoleWriter)
+	l := mainLog.Load().Output(multi).With().Timestamp().Logger()
+	mainLog.Store(&l)
+	switch {
+	case silent:
+		zerolog.SetGlobalLevel(zerolog.NoLevel)
+	case verbose == 1:
+		zerolog.SetGlobalLevel(zerolog.InfoLevel)
+	case verbose > 1:
+		zerolog.SetGlobalLevel(zerolog.DebugLevel)
+	default:
+		zerolog.SetGlobalLevel(zerolog.NoticeLevel)
+	}
+}
+
+// initLogging initializes global logging setup.
+func initLogging() {
+	zerolog.TimeFieldFormat = time.RFC3339 + ".000"
+	initLoggingWithBackup(true)
+}
+
+// initLoggingWithBackup initializes log setup base on current config.
+// If doBackup is true, backup old log file with ".1" suffix.
+//
+// This is only used in runCmd for special handling in case of logging config
+// change in cd mode. Without special reason, the caller should use initLogging
+// wrapper instead of calling this function directly.
+func initLoggingWithBackup(doBackup bool) {
+	writers := []io.Writer{io.Discard}
+	if logFilePath := normalizeLogFilePath(cfg.Service.LogPath); logFilePath != "" {
+		// Create parent directory if necessary.
+		if err := os.MkdirAll(filepath.Dir(logFilePath), 0750); err != nil {
+			mainLog.Load().Error().Msgf("failed to create log path: %v", err)
+			os.Exit(1)
+		}
+
+		// Default open log file in append mode.
+		flags := os.O_CREATE | os.O_RDWR | os.O_APPEND
+		if doBackup {
+			// Backup old log file with .1 suffix.
+			if err := os.Rename(logFilePath, logFilePath+oldLogSuffix); err != nil && !os.IsNotExist(err) {
+				mainLog.Load().Error().Msgf("could not backup old log file: %v", err)
+			} else {
+				// Backup was created, set flags for truncating old log file.
+				flags = os.O_CREATE | os.O_RDWR
+			}
+		}
+		logFile, err := openLogFile(logFilePath, flags)
+		if err != nil {
+			mainLog.Load().Error().Msgf("failed to create log file: %v", err)
+			os.Exit(1)
+		}
+		writers = append(writers, logFile)
+	}
+	writers = append(writers, consoleWriter)
+	multi := zerolog.MultiLevelWriter(writers...)
+	l := mainLog.Load().Output(multi).With().Logger()
+	mainLog.Store(&l)
+	// TODO: find a better way.
+	ctrld.ProxyLogger.Store(&l)
+
+	zerolog.SetGlobalLevel(zerolog.NoticeLevel)
+	logLevel := cfg.Service.LogLevel
+	switch {
+	case silent:
+		zerolog.SetGlobalLevel(zerolog.NoLevel)
+		return
+	case verbose == 1:
+		logLevel = "info"
+	case verbose > 1:
+		logLevel = "debug"
+	}
+	if logLevel == "" {
+		return
+	}
+	level, err := zerolog.ParseLevel(logLevel)
+	if err != nil {
+		mainLog.Load().Warn().Err(err).Msg("could not set log level")
+		return
+	}
+	zerolog.SetGlobalLevel(level)
+}
+
+func initCache() {
+	if !cfg.Service.CacheEnable {
+		return
+	}
+	if cfg.Service.CacheSize == 0 {
+		cfg.Service.CacheSize = 4096
+	}
+}

cmd/cli/main_test.go

@@ -0,0 +1,17 @@
+package cli
+
+import (
+	"os"
+	"strings"
+	"testing"
+
+	"github.com/rs/zerolog"
+)
+
+var logOutput strings.Builder
+
+func TestMain(m *testing.M) {
+	l := zerolog.New(&logOutput)
+	mainLog.Store(&l)
+	os.Exit(m.Run())
+}

cmd/cli/metrics.go

@@ -0,0 +1,150 @@
+package cli
+
+import (
+	"context"
+	"encoding/json"
+	"net"
+	"net/http"
+	"runtime"
+	"time"
+
+	"github.com/prometheus/client_golang/prometheus"
+	"github.com/prometheus/client_golang/prometheus/collectors"
+	"github.com/prometheus/client_golang/prometheus/promhttp"
+	"github.com/prometheus/prom2json"
+)
+
+// metricsServer represents a server to expose Prometheus metrics via HTTP.
+type metricsServer struct {
+	server  *http.Server
+	mux     *http.ServeMux
+	reg     *prometheus.Registry
+	addr    string
+	started bool
+}
+
+// newMetricsServer returns new metrics server.
+func newMetricsServer(addr string, reg *prometheus.Registry) (*metricsServer, error) {
+	mux := http.NewServeMux()
+	ms := &metricsServer{
+		server: &http.Server{Handler: mux},
+		mux:    mux,
+		reg:    reg,
+	}
+	ms.addr = addr
+	ms.registerMetricsServerHandler()
+	return ms, nil
+}
+
+// register adds handlers for given pattern.
+func (ms *metricsServer) register(pattern string, handler http.Handler) {
+	ms.mux.Handle(pattern, handler)
+}
+
+// registerMetricsServerHandler adds handlers for metrics server.
+func (ms *metricsServer) registerMetricsServerHandler() {
+	ms.register("/metrics", promhttp.HandlerFor(
+		ms.reg,
+		promhttp.HandlerOpts{
+			EnableOpenMetrics: true,
+			Timeout:           10 * time.Second,
+		},
+	))
+	ms.register("/metrics/json", jsonResponse(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		g := prometheus.ToTransactionalGatherer(ms.reg)
+		mfs, done, err := g.Gather()
+		defer done()
+		if err != nil {
+			msg := "could not gather metrics"
+			mainLog.Load().Warn().Err(err).Msg(msg)
+			http.Error(w, msg, http.StatusInternalServerError)
+			return
+		}
+		result := make([]*prom2json.Family, 0, len(mfs))
+		for _, mf := range mfs {
+			result = append(result, prom2json.NewFamily(mf))
+		}
+		if err := json.NewEncoder(w).Encode(result); err != nil {
+			msg := "could not marshal metrics result"
+			mainLog.Load().Warn().Err(err).Msg(msg)
+			http.Error(w, msg, http.StatusInternalServerError)
+			return
+		}
+	})))
+}
+
+// start runs the metricsServer.
+func (ms *metricsServer) start() error {
+	listener, err := net.Listen("tcp", ms.addr)
+	if err != nil {
+		return err
+	}
+	go ms.server.Serve(listener)
+	ms.started = true
+	return nil
+}
+
+// stop shutdowns the metricsServer within 2 seconds timeout.
+func (ms *metricsServer) stop() error {
+	if !ms.started {
+		return nil
+	}
+	ctx, cancel := context.WithTimeout(context.Background(), time.Second*1)
+	defer cancel()
+	return ms.server.Shutdown(ctx)
+}
+
+// runMetricsServer initializes metrics stats and runs the metrics server if enabled.
+func (p *prog) runMetricsServer(ctx context.Context, reloadCh chan struct{}) {
+	if !p.metricsEnabled() {
+		return
+	}
+
+	// Reset all stats.
+	statsVersion.Reset()
+	statsQueriesCount.Reset()
+	statsClientQueriesCount.Reset()
+
+	reg := prometheus.NewRegistry()
+	// Register queries count stats if enabled.
+	if p.metricsQueryStats.Load() {
+		reg.MustRegister(statsQueriesCount)
+		reg.MustRegister(statsClientQueriesCount)
+	}
+
+	addr := p.cfg.Service.MetricsListener
+	ms, err := newMetricsServer(addr, reg)
+	if err != nil {
+		mainLog.Load().Warn().Err(err).Msg("could not create new metrics server")
+		return
+	}
+	// Only start listener address if defined.
+	if addr != "" {
+		// Go runtime stats.
+		reg.MustRegister(collectors.NewBuildInfoCollector())
+		reg.MustRegister(collectors.NewGoCollector(
+			collectors.WithGoCollectorRuntimeMetrics(collectors.MetricsAll),
+		))
+		// ctrld stats.
+		reg.MustRegister(statsVersion)
+		statsVersion.WithLabelValues(commit, runtime.Version(), curVersion()).Inc()
+		reg.MustRegister(statsTimeStart)
+		statsTimeStart.Set(float64(time.Now().Unix()))
+		mainLog.Load().Debug().Msgf("starting metrics server on: %s", addr)
+		if err := ms.start(); err != nil {
+			mainLog.Load().Warn().Err(err).Msg("could not start metrics server")
+			return
+		}
+	}
+
+	select {
+	case <-p.stopCh:
+	case <-ctx.Done():
+	case <-reloadCh:
+	}
+
+	if err := ms.stop(); err != nil {
+		mainLog.Load().Warn().Err(err).Msg("could not stop metrics server")
+		return
+	}
+}

cmd/cli/net_darwin.go

@@ -0,0 +1,75 @@
+package cli
+
+import (
+	"bufio"
+	"bytes"
+	"io"
+	"net"
+	"os/exec"
+	"strings"
+)
+
+func patchNetIfaceName(iface *net.Interface) error {
+	b, err := exec.Command("networksetup", "-listnetworkserviceorder").Output()
+	if err != nil {
+		return err
+	}
+
+	if name := networkServiceName(iface.Name, bytes.NewReader(b)); name != "" {
+		iface.Name = name
+		mainLog.Load().Debug().Str("network_service", name).Msg("found network service name for interface")
+	}
+	return nil
+}
+
+func networkServiceName(ifaceName string, r io.Reader) string {
+	scanner := bufio.NewScanner(r)
+	prevLine := ""
+	for scanner.Scan() {
+		line := scanner.Text()
+		if strings.Contains(line, "*") {
+			// Network services is disabled.
+			continue
+		}
+		if !strings.Contains(line, "Device: "+ifaceName) {
+			prevLine = line
+			continue
+		}
+		parts := strings.SplitN(prevLine, " ", 2)
+		if len(parts) == 2 {
+			return strings.TrimSpace(parts[1])
+		}
+	}
+	return ""
+}
+
+// validInterface reports whether the *net.Interface is a valid one.
+func validInterface(iface *net.Interface, validIfacesMap map[string]struct{}) bool {
+	_, ok := validIfacesMap[iface.Name]
+	return ok
+}
+
+// validInterfacesMap returns a set of all valid hardware ports.
+func validInterfacesMap() map[string]struct{} {
+	b, err := exec.Command("networksetup", "-listallhardwareports").Output()
+	if err != nil {
+		return nil
+	}
+	return parseListAllHardwarePorts(bytes.NewReader(b))
+}
+
+// parseListAllHardwarePorts parses output of "networksetup -listallhardwareports"
+// and returns map presents all hardware ports.
+func parseListAllHardwarePorts(r io.Reader) map[string]struct{} {
+	m := make(map[string]struct{})
+	scanner := bufio.NewScanner(r)
+	for scanner.Scan() {
+		line := scanner.Text()
+		after, ok := strings.CutPrefix(line, "Device: ")
+		if !ok {
+			continue
+		}
+		m[after] = struct{}{}
+	}
+	return m
+}

cmd/cli/net_darwin_test.go

@@ -0,0 +1,104 @@
+package cli
+
+import (
+	"maps"
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+const listnetworkserviceorderOutput = `
+(1) USB 10/100/1000 LAN 2
+(Hardware Port: USB 10/100/1000 LAN, Device: en7)
+
+(2) Ethernet
+(Hardware Port: Ethernet, Device: en0)
+
+(3) Wi-Fi
+(Hardware Port: Wi-Fi, Device: en1)
+
+(4) Bluetooth PAN
+(Hardware Port: Bluetooth PAN, Device: en4)
+
+(5) Thunderbolt Bridge
+(Hardware Port: Thunderbolt Bridge, Device: bridge0)
+
+(6) kernal
+(Hardware Port: com.wireguard.macos, Device: )
+
+(7) WS BT
+(Hardware Port: com.wireguard.macos, Device: )
+
+(8) ca-001-stg
+(Hardware Port: com.wireguard.macos, Device: )
+
+(9) ca-001-stg-2
+(Hardware Port: com.wireguard.macos, Device: )
+
+`
+
+func Test_networkServiceName(t *testing.T) {
+	tests := []struct {
+		ifaceName          string
+		networkServiceName string
+	}{
+		{"en7", "USB 10/100/1000 LAN 2"},
+		{"en0", "Ethernet"},
+		{"en1", "Wi-Fi"},
+		{"en4", "Bluetooth PAN"},
+		{"bridge0", "Thunderbolt Bridge"},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.ifaceName, func(t *testing.T) {
+			t.Parallel()
+			name := networkServiceName(tc.ifaceName, strings.NewReader(listnetworkserviceorderOutput))
+			assert.Equal(t, tc.networkServiceName, name)
+		})
+	}
+}
+
+const listallhardwareportsOutput = `
+Hardware Port: Ethernet Adapter (en6)
+Device: en6
+Ethernet Address: 3a:3e:fc:1e:ab:41
+
+Hardware Port: Ethernet Adapter (en7)
+Device: en7
+Ethernet Address: 3a:3e:fc:1e:ab:42
+
+Hardware Port: Thunderbolt Bridge
+Device: bridge0
+Ethernet Address: 36:21:bb:3a:7a:40
+
+Hardware Port: Wi-Fi
+Device: en0
+Ethernet Address: a0:78:17:68:56:3f
+
+Hardware Port: Thunderbolt 1
+Device: en1
+Ethernet Address: 36:21:bb:3a:7a:40
+
+Hardware Port: Thunderbolt 2
+Device: en2
+Ethernet Address: 36:21:bb:3a:7a:44
+
+VLAN Configurations
+===================
+`
+
+func Test_parseListAllHardwarePorts(t *testing.T) {
+	expected := map[string]struct{}{
+		"en0":     {},
+		"en1":     {},
+		"en2":     {},
+		"en6":     {},
+		"en7":     {},
+		"bridge0": {},
+	}
+	m := parseListAllHardwarePorts(strings.NewReader(listallhardwareportsOutput))
+	if !maps.Equal(m, expected) {
+		t.Errorf("unexpected output, want: %v, got: %v", expected, m)
+	}
+}

cmd/cli/net_others.go

@@ -0,0 +1,11 @@
+//go:build !darwin && !windows
+
+package cli
+
+import "net"
+
+func patchNetIfaceName(iface *net.Interface) error { return nil }
+
+func validInterface(iface *net.Interface, validIfacesMap map[string]struct{}) bool { return true }
+
+func validInterfacesMap() map[string]struct{} { return nil }

cmd/cli/net_windows.go

@@ -0,0 +1,34 @@
+package cli
+
+import (
+	"bufio"
+	"bytes"
+	"net"
+	"strings"
+)
+
+func patchNetIfaceName(iface *net.Interface) error {
+	return nil
+}
+
+// validInterface reports whether the *net.Interface is a valid one.
+// On Windows, only physical interfaces are considered valid.
+func validInterface(iface *net.Interface, validIfacesMap map[string]struct{}) bool {
+	_, ok := validIfacesMap[iface.Name]
+	return ok
+}
+
+// validInterfacesMap returns a set of all physical interfaces.
+func validInterfacesMap() map[string]struct{} {
+	out, err := powershell("Get-NetAdapter -Physical | Select-Object -ExpandProperty Name")
+	if err != nil {
+		return nil
+	}
+	m := make(map[string]struct{})
+	scanner := bufio.NewScanner(bytes.NewReader(out))
+	for scanner.Scan() {
+		ifaceName := strings.TrimSpace(scanner.Text())
+		m[ifaceName] = struct{}{}
+	}
+	return m
+}

cmd/cli/netlink_linux.go

@@ -0,0 +1,34 @@
+package cli
+
+import (
+	"context"
+
+	"github.com/vishvananda/netlink"
+	"golang.org/x/sys/unix"
+)
+
+func (p *prog) watchLinkState(ctx context.Context) {
+	ch := make(chan netlink.LinkUpdate)
+	done := make(chan struct{})
+	defer close(done)
+	if err := netlink.LinkSubscribe(ch, done); err != nil {
+		mainLog.Load().Warn().Err(err).Msg("could not subscribe link")
+		return
+	}
+	for {
+		select {
+		case <-ctx.Done():
+			return
+		case lu := <-ch:
+			if lu.Change == 0xFFFFFFFF {
+				continue
+			}
+			if lu.Change&unix.IFF_UP != 0 {
+				mainLog.Load().Debug().Msgf("link state changed, re-bootstrapping")
+				for _, uc := range p.cfg.Upstream {
+					uc.ReBootstrap()
+				}
+			}
+		}
+	}
+}

cmd/cli/netlink_others.go

@@ -0,0 +1,7 @@
+//go:build !linux
+
+package cli
+
+import "context"
+
+func (p *prog) watchLinkState(ctx context.Context) {}

cmd/cli/network_manager_linux.go

@@ -1,10 +1,10 @@
-package main
+package cli
 
 import (
 	"context"
 	"os"
+	"os/exec"
 	"path/filepath"
-	"runtime"
 	"time"
 
 	"github.com/coreos/go-systemd/v22/dbus"
@@ -17,53 +17,56 @@ const (
 dns=none
 systemd-resolved=false
 `
-	nmSystemdUnitName   = "NetworkManager.service"
-	systemdEnabledState = "enabled"
+	nmSystemdUnitName = "NetworkManager.service"
 )
 
 var networkManagerCtrldConfFile = filepath.Join(nmConfDir, nmCtrldConfFilename)
 
+// hasNetworkManager reports whether NetworkManager executable found.
+func hasNetworkManager() bool {
+	exe, _ := exec.LookPath("NetworkManager")
+	return exe != ""
+}
+
 func setupNetworkManager() error {
-	if runtime.GOOS != "linux" {
-		mainLog.Debug().Msg("skipping NetworkManager setup, not on Linux")
+	if !hasNetworkManager() {
 		return nil
 	}
 	if content, _ := os.ReadFile(nmCtrldConfContent); string(content) == nmCtrldConfContent {
-		mainLog.Debug().Msg("NetworkManager already setup, nothing to do")
+		mainLog.Load().Debug().Msg("NetworkManager already setup, nothing to do")
 		return nil
 	}
 	err := os.WriteFile(networkManagerCtrldConfFile, []byte(nmCtrldConfContent), os.FileMode(0644))
 	if os.IsNotExist(err) {
-		mainLog.Debug().Msg("NetworkManager is not available")
+		mainLog.Load().Debug().Msg("NetworkManager is not available")
 		return nil
 	}
 	if err != nil {
-		mainLog.Debug().Err(err).Msg("could not write NetworkManager ctrld config file")
+		mainLog.Load().Debug().Err(err).Msg("could not write NetworkManager ctrld config file")
 		return err
 	}
 
 	reloadNetworkManager()
-	mainLog.Debug().Msg("setup NetworkManager done")
+	mainLog.Load().Debug().Msg("setup NetworkManager done")
 	return nil
 }
 
 func restoreNetworkManager() error {
-	if runtime.GOOS != "linux" {
-		mainLog.Debug().Msg("skipping NetworkManager restoring, not on Linux")
+	if !hasNetworkManager() {
 		return nil
 	}
 	err := os.Remove(networkManagerCtrldConfFile)
 	if os.IsNotExist(err) {
-		mainLog.Debug().Msg("NetworkManager is not available")
+		mainLog.Load().Debug().Msg("NetworkManager is not available")
 		return nil
 	}
 	if err != nil {
-		mainLog.Debug().Err(err).Msg("could not remove NetworkManager ctrld config file")
+		mainLog.Load().Debug().Err(err).Msg("could not remove NetworkManager ctrld config file")
 		return err
 	}
 
 	reloadNetworkManager()
-	mainLog.Debug().Msg("restore NetworkManager done")
+	mainLog.Load().Debug().Msg("restore NetworkManager done")
 	return nil
 }
 
@@ -72,14 +75,15 @@ func reloadNetworkManager() {
 	defer cancel()
 	conn, err := dbus.NewSystemConnectionContext(ctx)
 	if err != nil {
-		mainLog.Error().Err(err).Msg("could not create new system connection")
+		mainLog.Load().Error().Err(err).Msg("could not create new system connection")
 		return
 	}
 	defer conn.Close()
 
 	waitCh := make(chan string)
 	if _, err := conn.ReloadUnitContext(ctx, nmSystemdUnitName, "ignore-dependencies", waitCh); err != nil {
-		mainLog.Debug().Err(err).Msg("could not reload NetworkManager")
+		mainLog.Load().Debug().Err(err).Msg("could not reload NetworkManager")
+		return
 	}
 	<-waitCh
 }

cmd/cli/network_manager_others.go

@@ -0,0 +1,15 @@
+//go:build !linux
+
+package cli
+
+func setupNetworkManager() error {
+	reloadNetworkManager()
+	return nil
+}
+
+func restoreNetworkManager() error {
+	reloadNetworkManager()
+	return nil
+}
+
+func reloadNetworkManager() {}

cmd/cli/nextdns.go

@@ -0,0 +1,31 @@
+package cli
+
+import (
+	"fmt"
+
+	"github.com/Control-D-Inc/ctrld"
+)
+
+const nextdnsURL = "https://dns.nextdns.io"
+
+func generateNextDNSConfig(uid string) {
+	if uid == "" {
+		return
+	}
+	mainLog.Load().Info().Msg("generating ctrld config for NextDNS resolver")
+	cfg = ctrld.Config{
+		Listener: map[string]*ctrld.ListenerConfig{
+			"0": {
+				IP:   "0.0.0.0",
+				Port: 53,
+			},
+		},
+		Upstream: map[string]*ctrld.UpstreamConfig{
+			"0": {
+				Type:     ctrld.ResolverTypeDOH3,
+				Endpoint: fmt.Sprintf("%s/%s", nextdnsURL, uid),
+				Timeout:  5000,
+			},
+		},
+	}
+}

cmd/cli/os_darwin.go

@@ -0,0 +1,107 @@
+package cli
+
+import (
+	"bufio"
+	"bytes"
+	"fmt"
+	"net"
+	"os/exec"
+	"strings"
+
+	"github.com/Control-D-Inc/ctrld/internal/resolvconffile"
+)
+
+// allocate loopback ip
+// sudo ifconfig lo0 alias 127.0.0.2 up
+func allocateIP(ip string) error {
+	cmd := exec.Command("ifconfig", "lo0", "alias", ip, "up")
+	if err := cmd.Run(); err != nil {
+		mainLog.Load().Error().Err(err).Msg("allocateIP failed")
+		return err
+	}
+	return nil
+}
+
+func deAllocateIP(ip string) error {
+	cmd := exec.Command("ifconfig", "lo0", "-alias", ip)
+	if err := cmd.Run(); err != nil {
+		mainLog.Load().Error().Err(err).Msg("deAllocateIP failed")
+		return err
+	}
+	return nil
+}
+
+// setDnsIgnoreUnusableInterface likes setDNS, but return a nil error if the interface is not usable.
+func setDnsIgnoreUnusableInterface(iface *net.Interface, nameservers []string) error {
+	if err := setDNS(iface, nameservers); err != nil {
+		// TODO: investiate whether we can detect this without relying on error message.
+		if strings.Contains(err.Error(), " is not a recognized network service") {
+			return nil
+		}
+		return err
+	}
+	return nil
+}
+
+// set the dns server for the provided network interface
+// networksetup -setdnsservers Wi-Fi 8.8.8.8 1.1.1.1
+// TODO(cuonglm): use system API
+func setDNS(iface *net.Interface, nameservers []string) error {
+	cmd := "networksetup"
+	args := []string{"-setdnsservers", iface.Name}
+	args = append(args, nameservers...)
+	if out, err := exec.Command(cmd, args...).CombinedOutput(); err != nil {
+		return fmt.Errorf("%v: %w", string(out), err)
+	}
+	return nil
+}
+
+// resetDnsIgnoreUnusableInterface likes resetDNS, but return a nil error if the interface is not usable.
+func resetDnsIgnoreUnusableInterface(iface *net.Interface) error {
+	if err := resetDNS(iface); err != nil {
+		// TODO: investiate whether we can detect this without relying on error message.
+		if strings.Contains(err.Error(), " is not a recognized network service") {
+			return nil
+		}
+		return err
+	}
+	return nil
+}
+
+// TODO(cuonglm): use system API
+func resetDNS(iface *net.Interface) error {
+	if ns := savedStaticNameservers(iface); len(ns) > 0 {
+		if err := setDNS(iface, ns); err == nil {
+			return nil
+		}
+	}
+	cmd := "networksetup"
+	args := []string{"-setdnsservers", iface.Name, "empty"}
+	if out, err := exec.Command(cmd, args...).CombinedOutput(); err != nil {
+		return fmt.Errorf("%v: %w", string(out), err)
+	}
+	return nil
+}
+
+func currentDNS(_ *net.Interface) []string {
+	return resolvconffile.NameServers("")
+}
+
+// currentStaticDNS returns the current static DNS settings of given interface.
+func currentStaticDNS(iface *net.Interface) ([]string, error) {
+	cmd := "networksetup"
+	args := []string{"-getdnsservers", iface.Name}
+	out, err := exec.Command(cmd, args...).Output()
+	if err != nil {
+		return nil, err
+	}
+	scanner := bufio.NewScanner(bytes.NewReader(out))
+	var ns []string
+	for scanner.Scan() {
+		line := scanner.Text()
+		if ip := net.ParseIP(line); ip != nil {
+			ns = append(ns, ip.String())
+		}
+	}
+	return ns, nil
+}

cmd/cli/os_freebsd.go

@@ -0,0 +1,86 @@
+package cli
+
+import (
+	"net"
+	"net/netip"
+	"os/exec"
+
+	"tailscale.com/control/controlknobs"
+	"tailscale.com/health"
+
+	"github.com/Control-D-Inc/ctrld/internal/dns"
+	"github.com/Control-D-Inc/ctrld/internal/resolvconffile"
+)
+
+// allocate loopback ip
+// sudo ifconfig lo0 127.0.0.53 alias
+func allocateIP(ip string) error {
+	cmd := exec.Command("ifconfig", "lo0", ip, "alias")
+	if err := cmd.Run(); err != nil {
+		mainLog.Load().Error().Err(err).Msg("allocateIP failed")
+		return err
+	}
+	return nil
+}
+
+func deAllocateIP(ip string) error {
+	cmd := exec.Command("ifconfig", "lo0", ip, "-alias")
+	if err := cmd.Run(); err != nil {
+		mainLog.Load().Error().Err(err).Msg("deAllocateIP failed")
+		return err
+	}
+	return nil
+}
+
+// setDnsIgnoreUnusableInterface likes setDNS, but return a nil error if the interface is not usable.
+func setDnsIgnoreUnusableInterface(iface *net.Interface, nameservers []string) error {
+	return setDNS(iface, nameservers)
+}
+
+// set the dns server for the provided network interface
+func setDNS(iface *net.Interface, nameservers []string) error {
+	r, err := dns.NewOSConfigurator(logf, &health.Tracker{}, &controlknobs.Knobs{}, iface.Name)
+	if err != nil {
+		mainLog.Load().Error().Err(err).Msg("failed to create DNS OS configurator")
+		return err
+	}
+
+	ns := make([]netip.Addr, 0, len(nameservers))
+	for _, nameserver := range nameservers {
+		ns = append(ns, netip.MustParseAddr(nameserver))
+	}
+
+	if err := r.SetDNS(dns.OSConfig{Nameservers: ns}); err != nil {
+		mainLog.Load().Error().Err(err).Msg("failed to set DNS")
+		return err
+	}
+	return nil
+}
+
+// resetDnsIgnoreUnusableInterface likes resetDNS, but return a nil error if the interface is not usable.
+func resetDnsIgnoreUnusableInterface(iface *net.Interface) error {
+	return resetDNS(iface)
+}
+
+func resetDNS(iface *net.Interface) error {
+	r, err := dns.NewOSConfigurator(logf, &health.Tracker{}, &controlknobs.Knobs{}, iface.Name)
+	if err != nil {
+		mainLog.Load().Error().Err(err).Msg("failed to create DNS OS configurator")
+		return err
+	}
+
+	if err := r.Close(); err != nil {
+		mainLog.Load().Error().Err(err).Msg("failed to rollback DNS setting")
+		return err
+	}
+	return nil
+}
+
+func currentDNS(_ *net.Interface) []string {
+	return resolvconffile.NameServers("")
+}
+
+// currentStaticDNS returns the current static DNS settings of given interface.
+func currentStaticDNS(iface *net.Interface) ([]string, error) {
+	return currentDNS(iface), nil
+}

cmd/cli/os_linux.go

@@ -0,0 +1,300 @@
+package cli
+
+import (
+	"bufio"
+	"bytes"
+	"context"
+	"fmt"
+	"io"
+	"net"
+	"net/netip"
+	"os/exec"
+	"slices"
+	"strings"
+	"syscall"
+	"time"
+
+	"github.com/insomniacslk/dhcp/dhcpv4/nclient4"
+	"github.com/insomniacslk/dhcp/dhcpv6"
+	"github.com/insomniacslk/dhcp/dhcpv6/client6"
+	"tailscale.com/control/controlknobs"
+	"tailscale.com/health"
+	"tailscale.com/util/dnsname"
+
+	"github.com/Control-D-Inc/ctrld/internal/dns"
+	ctrldnet "github.com/Control-D-Inc/ctrld/internal/net"
+	"github.com/Control-D-Inc/ctrld/internal/resolvconffile"
+)
+
+const resolvConfBackupFailedMsg = "open /etc/resolv.pre-ctrld-backup.conf: read-only file system"
+
+// allocate loopback ip
+// sudo ip a add 127.0.0.2/24 dev lo
+func allocateIP(ip string) error {
+	cmd := exec.Command("ip", "a", "add", ip+"/24", "dev", "lo")
+	if out, err := cmd.CombinedOutput(); err != nil {
+		mainLog.Load().Error().Err(err).Msgf("allocateIP failed: %s", string(out))
+		return err
+	}
+	return nil
+}
+
+func deAllocateIP(ip string) error {
+	cmd := exec.Command("ip", "a", "del", ip+"/24", "dev", "lo")
+	if err := cmd.Run(); err != nil {
+		mainLog.Load().Error().Err(err).Msg("deAllocateIP failed")
+		return err
+	}
+	return nil
+}
+
+const maxSetDNSAttempts = 5
+
+// setDnsIgnoreUnusableInterface likes setDNS, but return a nil error if the interface is not usable.
+func setDnsIgnoreUnusableInterface(iface *net.Interface, nameservers []string) error {
+	return setDNS(iface, nameservers)
+}
+
+func setDNS(iface *net.Interface, nameservers []string) error {
+	r, err := dns.NewOSConfigurator(logf, &health.Tracker{}, &controlknobs.Knobs{}, iface.Name)
+	if err != nil {
+		mainLog.Load().Error().Err(err).Msg("failed to create DNS OS configurator")
+		return err
+	}
+
+	ns := make([]netip.Addr, 0, len(nameservers))
+	for _, nameserver := range nameservers {
+		ns = append(ns, netip.MustParseAddr(nameserver))
+	}
+
+	osConfig := dns.OSConfig{
+		Nameservers:   ns,
+		SearchDomains: []dnsname.FQDN{},
+	}
+	trySystemdResolve := false
+	for i := 0; i < maxSetDNSAttempts; i++ {
+		if err := r.SetDNS(osConfig); err != nil {
+			if strings.Contains(err.Error(), "Rejected send message") &&
+				strings.Contains(err.Error(), "org.freedesktop.network1.Manager") {
+				mainLog.Load().Warn().Msg("Interfaces are managed by systemd-networkd, switch to systemd-resolve for setting DNS")
+				trySystemdResolve = true
+				break
+			}
+			// This error happens on read-only file system, which causes ctrld failed to create backup
+			// for /etc/resolv.conf file. It is ok, because the DNS is still set anyway, and restore
+			// DNS will fallback to use DHCP if there's no backup /etc/resolv.conf file.
+			// The error format is controlled by us, so checking for error string is fine.
+			// See: ../../internal/dns/direct.go:L278
+			if r.Mode() == "direct" && strings.Contains(err.Error(), resolvConfBackupFailedMsg) {
+				return nil
+			}
+			return err
+		}
+		if useSystemdResolved {
+			if out, err := exec.Command("systemctl", "restart", "systemd-resolved").CombinedOutput(); err != nil {
+				mainLog.Load().Warn().Err(err).Msgf("could not restart systemd-resolved: %s", string(out))
+			}
+		}
+		currentNS := currentDNS(iface)
+		if isSubSet(nameservers, currentNS) {
+			return nil
+		}
+	}
+	if trySystemdResolve {
+		// Stop systemd-networkd and retry setting DNS.
+		if out, err := exec.Command("systemctl", "stop", "systemd-networkd").CombinedOutput(); err != nil {
+			return fmt.Errorf("%s: %w", string(out), err)
+		}
+		args := []string{"--interface=" + iface.Name, "--set-domain=~"}
+		for _, nameserver := range nameservers {
+			args = append(args, "--set-dns="+nameserver)
+		}
+		for i := 0; i < maxSetDNSAttempts; i++ {
+			if out, err := exec.Command("systemd-resolve", args...).CombinedOutput(); err != nil {
+				return fmt.Errorf("%s: %w", string(out), err)
+			}
+			currentNS := currentDNS(iface)
+			if isSubSet(nameservers, currentNS) {
+				return nil
+			}
+			time.Sleep(time.Second)
+		}
+	}
+	mainLog.Load().Debug().Msg("DNS was not set for some reason")
+	return nil
+}
+
+// resetDnsIgnoreUnusableInterface likes resetDNS, but return a nil error if the interface is not usable.
+func resetDnsIgnoreUnusableInterface(iface *net.Interface) error {
+	return resetDNS(iface)
+}
+
+func resetDNS(iface *net.Interface) (err error) {
+	defer func() {
+		if err == nil {
+			return
+		}
+		// Start systemd-networkd if present.
+		if exe, _ := exec.LookPath("/lib/systemd/systemd-networkd"); exe != "" {
+			_ = exec.Command("systemctl", "start", "systemd-networkd").Run()
+		}
+		if r, oerr := dns.NewOSConfigurator(logf, &health.Tracker{}, &controlknobs.Knobs{}, iface.Name); oerr == nil {
+			_ = r.SetDNS(dns.OSConfig{})
+			if err := r.Close(); err != nil {
+				mainLog.Load().Error().Err(err).Msg("failed to rollback DNS setting")
+				return
+			}
+			err = nil
+		}
+	}()
+
+	var ns []string
+	c, err := nclient4.New(iface.Name)
+	if err != nil {
+		return fmt.Errorf("nclient4.New: %w", err)
+	}
+	defer c.Close()
+
+	ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)
+	defer cancel()
+	lease, err := c.Request(ctx)
+	if err != nil {
+		return fmt.Errorf("nclient4.Request: %w", err)
+	}
+	for _, nameserver := range lease.ACK.DNS() {
+		if nameserver.Equal(net.IPv4zero) {
+			continue
+		}
+		ns = append(ns, nameserver.String())
+	}
+
+	// TODO(cuonglm): handle DHCPv6 properly.
+	if ctrldnet.IPv6Available(ctx) {
+		c := client6.NewClient()
+		conversation, err := c.Exchange(iface.Name)
+		if err != nil && !errAddrInUse(err) {
+			mainLog.Load().Debug().Err(err).Msg("could not exchange DHCPv6")
+		}
+		for _, packet := range conversation {
+			if packet.Type() == dhcpv6.MessageTypeReply {
+				msg, err := packet.GetInnerMessage()
+				if err != nil {
+					mainLog.Load().Debug().Err(err).Msg("could not get inner DHCPv6 message")
+					return nil
+				}
+				nameservers := msg.Options.DNS()
+				for _, nameserver := range nameservers {
+					ns = append(ns, nameserver.String())
+				}
+			}
+		}
+	}
+
+	return ignoringEINTR(func() error {
+		return setDNS(iface, ns)
+	})
+}
+
+func currentDNS(iface *net.Interface) []string {
+	for _, fn := range []getDNS{getDNSByResolvectl, getDNSBySystemdResolved, getDNSByNmcli, resolvconffile.NameServers} {
+		if ns := fn(iface.Name); len(ns) > 0 {
+			return ns
+		}
+	}
+	return nil
+}
+
+// currentStaticDNS returns the current static DNS settings of given interface.
+func currentStaticDNS(iface *net.Interface) ([]string, error) {
+	return currentDNS(iface), nil
+}
+
+func getDNSByResolvectl(iface string) []string {
+	b, err := exec.Command("resolvectl", "dns", "-i", iface).Output()
+	if err != nil {
+		return nil
+	}
+
+	parts := strings.Fields(strings.SplitN(string(b), "%", 2)[0])
+	if len(parts) > 2 {
+		return parts[3:]
+	}
+	return nil
+}
+
+func getDNSBySystemdResolved(iface string) []string {
+	b, err := exec.Command("systemd-resolve", "--status", iface).Output()
+	if err != nil {
+		return nil
+	}
+	return getDNSBySystemdResolvedFromReader(bytes.NewReader(b))
+}
+
+func getDNSBySystemdResolvedFromReader(r io.Reader) []string {
+	scanner := bufio.NewScanner(r)
+	var ret []string
+	for scanner.Scan() {
+		line := strings.TrimSpace(scanner.Text())
+		if len(ret) > 0 {
+			if net.ParseIP(line) != nil {
+				ret = append(ret, line)
+			}
+			continue
+		}
+		after, found := strings.CutPrefix(line, "DNS Servers: ")
+		if !found {
+			continue
+		}
+		if net.ParseIP(after) != nil {
+			ret = append(ret, after)
+		}
+	}
+	return ret
+}
+
+func getDNSByNmcli(iface string) []string {
+	b, err := exec.Command("nmcli", "dev", "show", iface).Output()
+	if err != nil {
+		return nil
+	}
+	s := bufio.NewScanner(bytes.NewReader(b))
+	var dns []string
+	do := func(line string) {
+		parts := strings.SplitN(line, ":", 2)
+		if len(parts) > 1 {
+			dns = append(dns, strings.TrimSpace(parts[1]))
+		}
+	}
+	for s.Scan() {
+		line := s.Text()
+		switch {
+		case strings.HasPrefix(line, "IP4.DNS"):
+			fallthrough
+		case strings.HasPrefix(line, "IP6.DNS"):
+			do(line)
+		}
+	}
+	return dns
+}
+
+func ignoringEINTR(fn func() error) error {
+	for {
+		err := fn()
+		if err != syscall.EINTR {
+			return err
+		}
+	}
+}
+
+// isSubSet reports whether s2 contains all elements of s1.
+func isSubSet(s1, s2 []string) bool {
+	ok := true
+	for _, ns := range s1 {
+		if slices.Contains(s2, ns) {
+			continue
+		}
+		ok = false
+		break
+	}
+	return ok
+}

cmd/cli/os_linux_test.go

@@ -0,0 +1,23 @@
+package cli
+
+import (
+	"reflect"
+	"strings"
+	"testing"
+)
+
+func Test_getDNSBySystemdResolvedFromReader(t *testing.T) {
+	r := strings.NewReader(`Link 2 (eth0)
+      Current Scopes: DNS
+       LLMNR setting: yes
+MulticastDNS setting: no
+      DNSSEC setting: no
+    DNSSEC supported: no
+         DNS Servers: 8.8.8.8
+                      8.8.4.4`)
+	want := []string{"8.8.8.8", "8.8.4.4"}
+	ns := getDNSBySystemdResolvedFromReader(r)
+	if !reflect.DeepEqual(ns, want) {
+		t.Logf("unexpected result, want: %v, got: %v", want, ns)
+	}
+}

cmd/cli/os_others.go

@@ -0,0 +1,13 @@
+//go:build !linux && !darwin && !freebsd
+
+package cli
+
+// TODO(cuonglm): implement.
+func allocateIP(ip string) error {
+	return nil
+}
+
+// TODO(cuonglm): implement.
+func deAllocateIP(ip string) error {
+	return nil
+}

cmd/cli/os_windows.go

@@ -0,0 +1,218 @@
+package cli
+
+import (
+	"errors"
+	"fmt"
+	"net"
+	"os"
+	"slices"
+	"strconv"
+	"strings"
+	"sync"
+
+	"golang.zx2c4.com/wireguard/windows/tunnel/winipcfg"
+
+	ctrldnet "github.com/Control-D-Inc/ctrld/internal/net"
+)
+
+const (
+	v4InterfaceKeyPathFormat = `HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\`
+	v6InterfaceKeyPathFormat = `HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\Interfaces\`
+)
+
+var (
+	setDNSOnce   sync.Once
+	resetDNSOnce sync.Once
+)
+
+// setDnsIgnoreUnusableInterface likes setDNS, but return a nil error if the interface is not usable.
+func setDnsIgnoreUnusableInterface(iface *net.Interface, nameservers []string) error {
+	return setDNS(iface, nameservers)
+}
+
+func setDnsPowershellCmd(iface *net.Interface, nameservers []string) string {
+	nss := make([]string, 0, len(nameservers))
+	for _, ns := range nameservers {
+		nss = append(nss, strconv.Quote(ns))
+	}
+	return fmt.Sprintf("Set-DnsClientServerAddress -InterfaceIndex %d -ServerAddresses (%s)", iface.Index, strings.Join(nss, ","))
+}
+
+// setDNS sets the dns server for the provided network interface
+func setDNS(iface *net.Interface, nameservers []string) error {
+	if len(nameservers) == 0 {
+		return errors.New("empty DNS nameservers")
+	}
+	setDNSOnce.Do(func() {
+		// If there's a Dns server running, that means we are on AD with Dns feature enabled.
+		// Configuring the Dns server to forward queries to ctrld instead.
+		if windowsHasLocalDnsServerRunning() {
+			file := absHomeDir(windowsForwardersFilename)
+			oldForwardersContent, _ := os.ReadFile(file)
+			hasLocalIPv6Listener := needLocalIPv6Listener()
+			forwarders := slices.DeleteFunc(slices.Clone(nameservers), func(s string) bool {
+				if !hasLocalIPv6Listener {
+					return false
+				}
+				return s == "::1"
+			})
+			if err := os.WriteFile(file, []byte(strings.Join(forwarders, ",")), 0600); err != nil {
+				mainLog.Load().Warn().Err(err).Msg("could not save forwarders settings")
+			}
+			oldForwarders := strings.Split(string(oldForwardersContent), ",")
+			if err := addDnsServerForwarders(forwarders, oldForwarders); err != nil {
+				mainLog.Load().Warn().Err(err).Msg("could not set forwarders settings")
+			}
+		}
+	})
+	out, err := powershell(setDnsPowershellCmd(iface, nameservers))
+	if err != nil {
+		return fmt.Errorf("%w: %s", err, string(out))
+	}
+	return nil
+}
+
+// resetDnsIgnoreUnusableInterface likes resetDNS, but return a nil error if the interface is not usable.
+func resetDnsIgnoreUnusableInterface(iface *net.Interface) error {
+	return resetDNS(iface)
+}
+
+// TODO(cuonglm): should we use system API?
+func resetDNS(iface *net.Interface) error {
+	resetDNSOnce.Do(func() {
+		// See corresponding comment in setDNS.
+		if windowsHasLocalDnsServerRunning() {
+			file := absHomeDir(windowsForwardersFilename)
+			content, err := os.ReadFile(file)
+			if err != nil {
+				mainLog.Load().Error().Err(err).Msg("could not read forwarders settings")
+				return
+			}
+			nameservers := strings.Split(string(content), ",")
+			if err := removeDnsServerForwarders(nameservers); err != nil {
+				mainLog.Load().Error().Err(err).Msg("could not remove forwarders settings")
+				return
+			}
+		}
+	})
+
+	// Restoring DHCP settings.
+	cmd := fmt.Sprintf("Set-DnsClientServerAddress -InterfaceIndex %d -ResetServerAddresses", iface.Index)
+	out, err := powershell(cmd)
+	if err != nil {
+		return fmt.Errorf("%w: %s", err, string(out))
+	}
+
+	// If there's static DNS saved, restoring it.
+	if nss := savedStaticNameservers(iface); len(nss) > 0 {
+		v4ns := make([]string, 0, 2)
+		v6ns := make([]string, 0, 2)
+		for _, ns := range nss {
+			if ctrldnet.IsIPv6(ns) {
+				v6ns = append(v6ns, ns)
+			} else {
+				v4ns = append(v4ns, ns)
+			}
+		}
+
+		for _, ns := range [][]string{v4ns, v6ns} {
+			if len(ns) == 0 {
+				continue
+			}
+			mainLog.Load().Debug().Msgf("setting static DNS for interface %q", iface.Name)
+			if err := setDNS(iface, ns); err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}
+
+func currentDNS(iface *net.Interface) []string {
+	luid, err := winipcfg.LUIDFromIndex(uint32(iface.Index))
+	if err != nil {
+		mainLog.Load().Error().Err(err).Msg("failed to get interface LUID")
+		return nil
+	}
+	nameservers, err := luid.DNS()
+	if err != nil {
+		mainLog.Load().Error().Err(err).Msg("failed to get interface DNS")
+		return nil
+	}
+	ns := make([]string, 0, len(nameservers))
+	for _, nameserver := range nameservers {
+		ns = append(ns, nameserver.String())
+	}
+	return ns
+}
+
+// currentStaticDNS returns the current static DNS settings of given interface.
+func currentStaticDNS(iface *net.Interface) ([]string, error) {
+	luid, err := winipcfg.LUIDFromIndex(uint32(iface.Index))
+	if err != nil {
+		return nil, err
+	}
+	guid, err := luid.GUID()
+	if err != nil {
+		return nil, err
+	}
+	var ns []string
+	for _, path := range []string{v4InterfaceKeyPathFormat, v6InterfaceKeyPathFormat} {
+		interfaceKeyPath := path + guid.String()
+		found := false
+		for _, key := range []string{"NameServer", "ProfileNameServer"} {
+			if found {
+				continue
+			}
+			cmd := fmt.Sprintf(`Get-ItemPropertyValue -Path "%s" -Name "%s"`, interfaceKeyPath, key)
+			out, err := powershell(cmd)
+			if err == nil && len(out) > 0 {
+				found = true
+				for _, e := range strings.Split(string(out), ",") {
+					ns = append(ns, strings.TrimRight(e, "\x00"))
+				}
+			}
+		}
+	}
+	return ns, nil
+}
+
+// addDnsServerForwarders adds given nameservers to DNS server forwarders list,
+// and also removing old forwarders if provided.
+func addDnsServerForwarders(nameservers, old []string) error {
+	newForwardersMap := make(map[string]struct{})
+	newForwarders := make([]string, len(nameservers))
+	for i := range nameservers {
+		newForwardersMap[nameservers[i]] = struct{}{}
+		newForwarders[i] = fmt.Sprintf("%q", nameservers[i])
+	}
+	oldForwarders := old[:0]
+	for _, fwd := range old {
+		if _, ok := newForwardersMap[fwd]; !ok {
+			oldForwarders = append(oldForwarders, fwd)
+		}
+	}
+	// NOTE: It is important to add new forwarder before removing old one.
+	//       Testing on Windows Server 2022 shows that removing forwarder1
+	//       then adding forwarder2 sometimes ends up adding both of them
+	//       to the forwarders list.
+	cmd := fmt.Sprintf("Add-DnsServerForwarder -IPAddress %s", strings.Join(newForwarders, ","))
+	if len(oldForwarders) > 0 {
+		cmd = fmt.Sprintf("%s ; Remove-DnsServerForwarder -IPAddress %s -Force", cmd, strings.Join(oldForwarders, ","))
+	}
+	if out, err := powershell(cmd); err != nil {
+		return fmt.Errorf("%w: %s", err, string(out))
+	}
+	return nil
+}
+
+// removeDnsServerForwarders removes given nameservers from DNS server forwarders list.
+func removeDnsServerForwarders(nameservers []string) error {
+	for _, ns := range nameservers {
+		cmd := fmt.Sprintf("Remove-DnsServerForwarder -IPAddress %s -Force", ns)
+		if out, err := powershell(cmd); err != nil {
+			return fmt.Errorf("%w: %s", err, string(out))
+		}
+	}
+	return nil
+}

cmd/cli/prog_darwin.go

@@ -0,0 +1,11 @@
+package cli
+
+import (
+	"github.com/kardianos/service"
+)
+
+func setDependencies(svc *service.Config) {}
+
+func setWorkingDirectory(svc *service.Config, dir string) {
+	svc.WorkingDirectory = dir
+}

cmd/cli/prog_freebsd.go

@@ -0,0 +1,14 @@
+package cli
+
+import (
+	"os"
+
+	"github.com/kardianos/service"
+)
+
+func setDependencies(svc *service.Config) {
+	// TODO(cuonglm): remove once https://github.com/kardianos/service/issues/359 fixed.
+	_ = os.MkdirAll("/usr/local/etc/rc.d", 0755)
+}
+
+func setWorkingDirectory(svc *service.Config, dir string) {}

cmd/cli/prog_linux.go

@@ -0,0 +1,64 @@
+package cli
+
+import (
+	"bufio"
+	"bytes"
+	"io"
+	"os"
+	"os/exec"
+	"strings"
+
+	"github.com/kardianos/service"
+	"tailscale.com/control/controlknobs"
+	"tailscale.com/health"
+
+	"github.com/Control-D-Inc/ctrld/internal/dns"
+)
+
+func init() {
+	if r, err := dns.NewOSConfigurator(func(format string, args ...any) {}, &health.Tracker{}, &controlknobs.Knobs{}, "lo"); err == nil {
+		useSystemdResolved = r.Mode() == "systemd-resolved"
+	}
+	// Disable quic-go's ECN support by default, see https://github.com/quic-go/quic-go/issues/3911
+	if os.Getenv("QUIC_GO_DISABLE_ECN") == "" {
+		os.Setenv("QUIC_GO_DISABLE_ECN", "true")
+	}
+}
+
+func setDependencies(svc *service.Config) {
+	svc.Dependencies = []string{
+		"Wants=network-online.target",
+		"After=network-online.target",
+		"Wants=NetworkManager-wait-online.service",
+		"After=NetworkManager-wait-online.service",
+		"Wants=nss-lookup.target",
+		"After=nss-lookup.target",
+	}
+	if out, _ := exec.Command("networkctl", "--no-pager").CombinedOutput(); len(out) > 0 {
+		if wantsSystemDNetworkdWaitOnline(bytes.NewReader(out)) {
+			svc.Dependencies = append(svc.Dependencies, "Wants=systemd-networkd-wait-online.service")
+		}
+	}
+}
+
+func setWorkingDirectory(svc *service.Config, dir string) {
+	svc.WorkingDirectory = dir
+}
+
+// wantsSystemDNetworkdWaitOnline reports whether "systemd-networkd-wait-online" service
+// is required to be added to ctrld dependencies services.
+// The input reader r is the output of "networkctl --no-pager" command.
+func wantsSystemDNetworkdWaitOnline(r io.Reader) bool {
+	scanner := bufio.NewScanner(r)
+	// Skip header
+	scanner.Scan()
+	configured := false
+	for scanner.Scan() {
+		fields := strings.Fields(scanner.Text())
+		if len(fields) > 0 && fields[len(fields)-1] == "configured" {
+			configured = true
+			break
+		}
+	}
+	return configured
+}

cmd/cli/prog_linux_test.go

@@ -0,0 +1,48 @@
+package cli
+
+import (
+	"io"
+	"strings"
+	"testing"
+)
+
+const (
+	networkctlUnmanagedOutput = `IDX LINK            TYPE     OPERATIONAL SETUP    
+  1 lo              loopback carrier     unmanaged
+  2 wlp0s20f3       wlan     routable    unmanaged
+  3 tailscale0      none     routable    unmanaged
+  4 br-9ac33145e060 bridge   no-carrier  unmanaged
+  5 docker0         bridge   no-carrier  unmanaged
+
+5 links listed.
+`
+	networkctlManagedOutput = `IDX LINK            TYPE     OPERATIONAL SETUP    
+  1 lo              loopback carrier     unmanaged
+  2 wlp0s20f3       wlan     routable    configured
+  3 tailscale0      none     routable    unmanaged
+  4 br-9ac33145e060 bridge   no-carrier  unmanaged
+  5 docker0         bridge   no-carrier  unmanaged
+
+5 links listed.
+`
+)
+
+func Test_wantsSystemDNetworkdWaitOnline(t *testing.T) {
+	tests := []struct {
+		name     string
+		r        io.Reader
+		required bool
+	}{
+		{"unmanaged", strings.NewReader(networkctlUnmanagedOutput), false},
+		{"managed", strings.NewReader(networkctlManagedOutput), true},
+		{"empty", strings.NewReader(""), false},
+	}
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			if required := wantsSystemDNetworkdWaitOnline(tc.r); required != tc.required {
+				t.Errorf("wants %v got %v", tc.required, required)
+			}
+		})
+	}
+}

cmd/cli/prog_others.go

@@ -0,0 +1,12 @@
+//go:build !linux && !freebsd && !darwin
+
+package cli
+
+import "github.com/kardianos/service"
+
+func setDependencies(svc *service.Config) {}
+
+func setWorkingDirectory(svc *service.Config, dir string) {
+	// WorkingDirectory is not supported on Windows.
+	svc.WorkingDirectory = dir
+}

cmd/cli/prog_test.go

@@ -0,0 +1,57 @@
+package cli
+
+import (
+	"testing"
+	"time"
+
+	"github.com/Control-D-Inc/ctrld"
+	"github.com/stretchr/testify/assert"
+)
+
+func Test_prog_dnsWatchdogEnabled(t *testing.T) {
+	p := &prog{cfg: &ctrld.Config{}}
+
+	// Default value is true.
+	assert.True(t, p.dnsWatchdogEnabled())
+
+	tests := []struct {
+		name    string
+		enabled bool
+	}{
+		{"enabled", true},
+		{"disabled", false},
+	}
+
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			p.cfg.Service.DnsWatchdogEnabled = &tc.enabled
+			assert.Equal(t, tc.enabled, p.dnsWatchdogEnabled())
+		})
+	}
+}
+
+func Test_prog_dnsWatchdogInterval(t *testing.T) {
+	p := &prog{cfg: &ctrld.Config{}}
+
+	// Default value is 20s.
+	assert.Equal(t, dnsWatchdogDefaultInterval, p.dnsWatchdogDuration())
+
+	tests := []struct {
+		name     string
+		duration time.Duration
+		expected time.Duration
+	}{
+		{"valid", time.Minute, time.Minute},
+		{"zero", 0, dnsWatchdogDefaultInterval},
+		{"nagative", time.Duration(-1 * time.Minute), dnsWatchdogDefaultInterval},
+	}
+
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			p.cfg.Service.DnsWatchdogInvterval = &tc.duration
+			assert.Equal(t, tc.expected, p.dnsWatchdogDuration())
+		})
+	}
+}

cmd/cli/prometheus.go

@@ -0,0 +1,57 @@
+package cli
+
+import "github.com/prometheus/client_golang/prometheus"
+
+const (
+	metricsLabelListener       = "listener"
+	metricsLabelClientSourceIP = "client_source_ip"
+	metricsLabelClientMac      = "client_mac"
+	metricsLabelClientHostname = "client_hostname"
+	metricsLabelUpstream       = "upstream"
+	metricsLabelRRType         = "rr_type"
+	metricsLabelRCode          = "rcode"
+)
+
+// statsVersion represent ctrld version.
+var statsVersion = prometheus.NewCounterVec(prometheus.CounterOpts{
+	Name: "ctrld_build_info",
+	Help: "Version of ctrld process.",
+}, []string{"gitref", "goversion", "version"})
+
+// statsTimeStart represents start time of ctrld service.
+var statsTimeStart = prometheus.NewGauge(prometheus.GaugeOpts{
+	Name: "ctrld_time_seconds",
+	Help: "Start time of the ctrld process since unix epoch in seconds.",
+})
+
+var statsQueriesCountLabels = []string{
+	metricsLabelListener,
+	metricsLabelClientSourceIP,
+	metricsLabelClientMac,
+	metricsLabelClientHostname,
+	metricsLabelUpstream,
+	metricsLabelRRType,
+	metricsLabelRCode,
+}
+
+// statsQueriesCount counts total number of queries.
+var statsQueriesCount = prometheus.NewCounterVec(prometheus.CounterOpts{
+	Name: "ctrld_queries_count",
+	Help: "Total number of queries.",
+}, statsQueriesCountLabels)
+
+// statsClientQueriesCount counts total number of queries of a client.
+//
+// The labels "client_source_ip", "client_mac", "client_hostname" are unbounded,
+// thus this stat is highly inefficient if there are many devices.
+var statsClientQueriesCount = prometheus.NewCounterVec(prometheus.CounterOpts{
+	Name: "ctrld_client_queries_count",
+	Help: "Total number queries of a client.",
+}, []string{metricsLabelClientSourceIP, metricsLabelClientMac, metricsLabelClientHostname})
+
+// WithLabelValuesInc increases prometheus counter by 1 if query stats is enabled.
+func (p *prog) WithLabelValuesInc(c *prometheus.CounterVec, lvs ...string) {
+	if p.metricsQueryStats.Load() {
+		c.WithLabelValues(lvs...).Inc()
+	}
+}

cmd/cli/reload_others.go

@@ -0,0 +1,17 @@
+//go:build !windows
+
+package cli
+
+import (
+	"os"
+	"os/signal"
+	"syscall"
+)
+
+func notifyReloadSigCh(ch chan os.Signal) {
+	signal.Notify(ch, syscall.SIGUSR1)
+}
+
+func (p *prog) sendReloadSignal() error {
+	return syscall.Kill(syscall.Getpid(), syscall.SIGUSR1)
+}

cmd/cli/reload_windows.go

@@ -0,0 +1,18 @@
+package cli
+
+import (
+	"errors"
+	"os"
+	"time"
+)
+
+func notifyReloadSigCh(ch chan os.Signal) {}
+
+func (p *prog) sendReloadSignal() error {
+	select {
+	case p.reloadCh <- struct{}{}:
+		return nil
+	case <-time.After(5 * time.Second):
+	}
+	return errors.New("timeout while sending reload signal")
+}

cmd/cli/resolvconf.go

@@ -0,0 +1,73 @@
+package cli
+
+import (
+	"net"
+	"net/netip"
+	"path/filepath"
+
+	"github.com/fsnotify/fsnotify"
+)
+
+// watchResolvConf watches any changes to /etc/resolv.conf file,
+// and reverting to the original config set by ctrld.
+func (p *prog) watchResolvConf(iface *net.Interface, ns []netip.Addr, setDnsFn func(iface *net.Interface, ns []netip.Addr) error) {
+	resolvConfPath := "/etc/resolv.conf"
+	// Evaluating symbolics link to watch the target file that /etc/resolv.conf point to.
+	if rp, _ := filepath.EvalSymlinks(resolvConfPath); rp != "" {
+		resolvConfPath = rp
+	}
+	mainLog.Load().Debug().Msgf("start watching %s file", resolvConfPath)
+	watcher, err := fsnotify.NewWatcher()
+	if err != nil {
+		mainLog.Load().Warn().Err(err).Msg("could not create watcher for /etc/resolv.conf")
+		return
+	}
+	defer watcher.Close()
+
+	// We watch /etc instead of /etc/resolv.conf directly,
+	// see: https://github.com/fsnotify/fsnotify#watching-a-file-doesnt-work-well
+	watchDir := filepath.Dir(resolvConfPath)
+	if err := watcher.Add(watchDir); err != nil {
+		mainLog.Load().Warn().Err(err).Msgf("could not add %s to watcher list", watchDir)
+		return
+	}
+
+	for {
+		select {
+		case <-p.dnsWatcherStopCh:
+			return
+		case <-p.stopCh:
+			mainLog.Load().Debug().Msgf("stopping watcher for %s", resolvConfPath)
+			return
+		case event, ok := <-watcher.Events:
+			if p.leakingQuery.Load() {
+				return
+			}
+			if !ok {
+				return
+			}
+			if event.Name != resolvConfPath { // skip if not /etc/resolv.conf changes.
+				continue
+			}
+			if event.Has(fsnotify.Write) || event.Has(fsnotify.Create) {
+				mainLog.Load().Debug().Msg("/etc/resolv.conf changes detected, reverting to ctrld setting")
+				if err := watcher.Remove(watchDir); err != nil {
+					mainLog.Load().Error().Err(err).Msg("failed to pause watcher")
+					continue
+				}
+				if err := setDnsFn(iface, ns); err != nil {
+					mainLog.Load().Error().Err(err).Msg("failed to revert /etc/resolv.conf changes")
+				}
+				if err := watcher.Add(watchDir); err != nil {
+					mainLog.Load().Error().Err(err).Msg("failed to continue running watcher")
+					return
+				}
+			}
+		case err, ok := <-watcher.Errors:
+			if !ok {
+				return
+			}
+			mainLog.Load().Err(err).Msg("could not get event for /etc/resolv.conf")
+		}
+	}
+}

cmd/cli/resolvconf_darwin.go

@@ -0,0 +1,49 @@
+package cli
+
+import (
+	"net"
+	"net/netip"
+	"os"
+	"slices"
+
+	"github.com/Control-D-Inc/ctrld/internal/dns/resolvconffile"
+)
+
+const resolvConfPath = "/etc/resolv.conf"
+
+// setResolvConf sets the content of resolv.conf file using the given nameservers list.
+func setResolvConf(iface *net.Interface, ns []netip.Addr) error {
+	servers := make([]string, len(ns))
+	for i := range ns {
+		servers[i] = ns[i].String()
+	}
+	if err := setDNS(iface, servers); err != nil {
+		return err
+	}
+	slices.Sort(servers)
+	curNs := currentDNS(iface)
+	slices.Sort(curNs)
+	if !slices.Equal(curNs, servers) {
+		c, err := resolvconffile.ParseFile(resolvConfPath)
+		if err != nil {
+			return err
+		}
+		c.Nameservers = ns
+		f, err := os.Create(resolvConfPath)
+		if err != nil {
+			return err
+		}
+		defer f.Close()
+
+		if err := c.Write(f); err != nil {
+			return err
+		}
+		return f.Close()
+	}
+	return nil
+}
+
+// shouldWatchResolvconf reports whether ctrld should watch changes to resolv.conf file with given OS configurator.
+func shouldWatchResolvconf() bool {
+	return true
+}

cmd/cli/resolvconf_not_darwin_unix.go

@@ -0,0 +1,42 @@
+//go:build unix && !darwin
+
+package cli
+
+import (
+	"net"
+	"net/netip"
+
+	"tailscale.com/control/controlknobs"
+	"tailscale.com/health"
+	"tailscale.com/util/dnsname"
+
+	"github.com/Control-D-Inc/ctrld/internal/dns"
+)
+
+// setResolvConf sets the content of resolv.conf file using the given nameservers list.
+func setResolvConf(iface *net.Interface, ns []netip.Addr) error {
+	r, err := dns.NewOSConfigurator(func(format string, args ...any) {}, &health.Tracker{}, &controlknobs.Knobs{}, "lo") // interface name does not matter.
+	if err != nil {
+		return err
+	}
+
+	oc := dns.OSConfig{
+		Nameservers:   ns,
+		SearchDomains: []dnsname.FQDN{},
+	}
+	return r.SetDNS(oc)
+}
+
+// shouldWatchResolvconf reports whether ctrld should watch changes to resolv.conf file with given OS configurator.
+func shouldWatchResolvconf() bool {
+	r, err := dns.NewOSConfigurator(func(format string, args ...any) {}, &health.Tracker{}, &controlknobs.Knobs{}, "lo") // interface name does not matter.
+	if err != nil {
+		return false
+	}
+	switch r.Mode() {
+	case "direct", "resolvconf":
+		return true
+	default:
+		return false
+	}
+}

cmd/cli/resolvconf_windows.go

@@ -0,0 +1,16 @@
+package cli
+
+import (
+	"net"
+	"net/netip"
+)
+
+// setResolvConf sets the content of resolv.conf file using the given nameservers list.
+func setResolvConf(_ *net.Interface, _ []netip.Addr) error {
+	return nil
+}
+
+// shouldWatchResolvconf reports whether ctrld should watch changes to resolv.conf file with given OS configurator.
+func shouldWatchResolvconf() bool {
+	return false
+}

cmd/cli/self_delete_others.go

@@ -0,0 +1,7 @@
+//go:build !windows
+
+package cli
+
+var supportedSelfDelete = true
+
+func selfDeleteExe() error { return nil }

cmd/cli/self_delete_windows.go

@@ -0,0 +1,134 @@
+// Copied from https://github.com/secur30nly/go-self-delete
+// with modification to suitable for ctrld usage.
+
+/*
+   License: MIT Licence
+
+   References:
+       - https://github.com/LloydLabs/delete-self-poc
+       - https://twitter.com/jonasLyk/status/1350401461985955840
+*/
+
+package cli
+
+import (
+	"unsafe"
+
+	"golang.org/x/sys/windows"
+)
+
+var supportedSelfDelete = false
+
+type FILE_RENAME_INFO struct {
+	Union struct {
+		ReplaceIfExists bool
+		Flags           uint32
+	}
+	RootDirectory  windows.Handle
+	FileNameLength uint32
+	FileName       [1]uint16
+}
+
+type FILE_DISPOSITION_INFO struct {
+	DeleteFile bool
+}
+
+func dsOpenHandle(pwPath *uint16) (windows.Handle, error) {
+	handle, err := windows.CreateFile(
+		pwPath,
+		windows.DELETE,
+		0,
+		nil,
+		windows.OPEN_EXISTING,
+		windows.FILE_ATTRIBUTE_NORMAL,
+		0,
+	)
+
+	if err != nil {
+		return 0, err
+	}
+
+	return handle, nil
+}
+
+func dsRenameHandle(hHandle windows.Handle) error {
+	var fRename FILE_RENAME_INFO
+	DS_STREAM_RENAME, err := windows.UTF16FromString(":deadbeef")
+
+	if err != nil {
+		return err
+	}
+
+	lpwStream := &DS_STREAM_RENAME[0]
+	fRename.FileNameLength = uint32(unsafe.Sizeof(lpwStream))
+
+	windows.NewLazyDLL("kernel32.dll").NewProc("RtlCopyMemory").Call(
+		uintptr(unsafe.Pointer(&fRename.FileName[0])),
+		uintptr(unsafe.Pointer(lpwStream)),
+		unsafe.Sizeof(lpwStream),
+	)
+
+	err = windows.SetFileInformationByHandle(
+		hHandle,
+		windows.FileRenameInfo,
+		(*byte)(unsafe.Pointer(&fRename)),
+		uint32(unsafe.Sizeof(fRename)+unsafe.Sizeof(lpwStream)),
+	)
+
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func dsDepositeHandle(hHandle windows.Handle) error {
+	var fDelete FILE_DISPOSITION_INFO
+	fDelete.DeleteFile = true
+
+	err := windows.SetFileInformationByHandle(
+		hHandle,
+		windows.FileDispositionInfo,
+		(*byte)(unsafe.Pointer(&fDelete)),
+		uint32(unsafe.Sizeof(fDelete)),
+	)
+
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func selfDeleteExe() error {
+	var wcPath [windows.MAX_PATH + 1]uint16
+	var hCurrent windows.Handle
+
+	_, err := windows.GetModuleFileName(0, &wcPath[0], windows.MAX_PATH)
+	if err != nil {
+		return err
+	}
+
+	hCurrent, err = dsOpenHandle(&wcPath[0])
+	if err != nil || hCurrent == windows.InvalidHandle {
+		return err
+	}
+
+	if err := dsRenameHandle(hCurrent); err != nil {
+		_ = windows.CloseHandle(hCurrent)
+		return err
+	}
+	_ = windows.CloseHandle(hCurrent)
+
+	hCurrent, err = dsOpenHandle(&wcPath[0])
+	if err != nil || hCurrent == windows.InvalidHandle {
+		return err
+	}
+
+	if err := dsDepositeHandle(hCurrent); err != nil {
+		_ = windows.CloseHandle(hCurrent)
+		return err
+	}
+
+	return windows.CloseHandle(hCurrent)
+}

cmd/cli/self_kill_others.go

@@ -0,0 +1,16 @@
+//go:build !unix
+
+package cli
+
+import (
+	"os"
+
+	"github.com/rs/zerolog"
+)
+
+func selfUninstall(p *prog, logger zerolog.Logger) {
+	if uninstallInvalidCdUID(p, logger, false) {
+		logger.Warn().Msgf("service was uninstalled because device %q does not exist", cdUID)
+		os.Exit(0)
+	}
+}

cmd/cli/self_kill_unix.go

@@ -0,0 +1,45 @@
+//go:build unix
+
+package cli
+
+import (
+	"fmt"
+	"os"
+	"os/exec"
+	"runtime"
+	"syscall"
+
+	"github.com/rs/zerolog"
+)
+
+func selfUninstall(p *prog, logger zerolog.Logger) {
+	if runtime.GOOS == "linux" {
+		selfUninstallLinux(p, logger)
+	}
+
+	bin, err := os.Executable()
+	if err != nil {
+		logger.Fatal().Err(err).Msg("could not determine executable")
+	}
+	args := []string{"uninstall"}
+	if !deactivationPinNotSet() {
+		args = append(args, fmt.Sprintf("--pin=%d", cdDeactivationPin.Load()))
+	}
+	cmd := exec.Command(bin, args...)
+	cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
+	if err := cmd.Start(); err != nil {
+		logger.Fatal().Err(err).Msg("could not start self uninstall command")
+	}
+	cmd.Stdout = os.Stdout
+	cmd.Stderr = os.Stderr
+	logger.Warn().Msgf("service was uninstalled because device %q does not exist", cdUID)
+	_ = cmd.Wait()
+	os.Exit(0)
+}
+
+func selfUninstallLinux(p *prog, logger zerolog.Logger) {
+	if uninstallInvalidCdUID(p, logger, true) {
+		logger.Warn().Msgf("service was uninstalled because device %q does not exist", cdUID)
+		os.Exit(0)
+	}
+}

cmd/cli/sema.go

@@ -0,0 +1,24 @@
+package cli
+
+type semaphore interface {
+	acquire()
+	release()
+}
+
+type noopSemaphore struct{}
+
+func (n noopSemaphore) acquire() {}
+
+func (n noopSemaphore) release() {}
+
+type chanSemaphore struct {
+	ready chan struct{}
+}
+
+func (c *chanSemaphore) acquire() {
+	c.ready <- struct{}{}
+}
+
+func (c *chanSemaphore) release() {
+	<-c.ready
+}

cmd/cli/service.go

@@ -0,0 +1,199 @@
+package cli
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"os"
+	"os/exec"
+
+	"github.com/kardianos/service"
+
+	"github.com/Control-D-Inc/ctrld/internal/router"
+)
+
+// newService wraps service.New call to return service.Service
+// wrapper which is suitable for the current platform.
+func newService(i service.Interface, c *service.Config) (service.Service, error) {
+	s, err := service.New(i, c)
+	if err != nil {
+		return nil, err
+	}
+	switch {
+	case router.IsOldOpenwrt(), router.IsNetGearOrbi():
+		return &procd{sysV: &sysV{s}, svcConfig: c}, nil
+	case router.IsGLiNet():
+		return &sysV{s}, nil
+	case s.Platform() == "unix-systemv":
+		return &sysV{s}, nil
+	case s.Platform() == "linux-systemd":
+		return &systemd{s}, nil
+	case s.Platform() == "darwin-launchd":
+		return newLaunchd(s), nil
+
+	}
+	return s, nil
+}
+
+// sysV wraps a service.Service, and provide start/stop/status command
+// base on "/etc/init.d/<service_name>".
+//
+// Use this on system where "service" command is not available, like GL.iNET router.
+type sysV struct {
+	service.Service
+}
+
+func (s *sysV) installed() bool {
+	fi, err := os.Stat("/etc/init.d/ctrld")
+	if err != nil {
+		return false
+	}
+	mode := fi.Mode()
+	return mode.IsRegular() && (mode&0111) != 0
+}
+
+func (s *sysV) Start() error {
+	if !s.installed() {
+		return service.ErrNotInstalled
+	}
+	_, err := exec.Command("/etc/init.d/ctrld", "start").CombinedOutput()
+	return err
+}
+
+func (s *sysV) Stop() error {
+	if !s.installed() {
+		return service.ErrNotInstalled
+	}
+	_, err := exec.Command("/etc/init.d/ctrld", "stop").CombinedOutput()
+	return err
+}
+
+func (s *sysV) Restart() error {
+	if !s.installed() {
+		return service.ErrNotInstalled
+	}
+	// We don't care about error returned by s.Stop,
+	// because the service may already be stopped.
+	_ = s.Stop()
+	return s.Start()
+}
+
+func (s *sysV) Status() (service.Status, error) {
+	if !s.installed() {
+		return service.StatusUnknown, service.ErrNotInstalled
+	}
+	return unixSystemVServiceStatus()
+}
+
+// procd wraps a service.Service, and provide start/stop command
+// base on "/etc/init.d/<service_name>", status command base on parsing "ps" command output.
+//
+// Use this on system where "/etc/init.d/<service_name> status" command is not available,
+// like old GL.iNET Opal router.
+type procd struct {
+	*sysV
+	svcConfig *service.Config
+}
+
+func (s *procd) Status() (service.Status, error) {
+	if !s.installed() {
+		return service.StatusUnknown, service.ErrNotInstalled
+	}
+	bin := s.svcConfig.Executable
+	if bin == "" {
+		exe, err := os.Executable()
+		if err != nil {
+			return service.StatusUnknown, nil
+		}
+		bin = exe
+	}
+
+	// Looking for something like "/sbin/ctrld run ".
+	shellCmd := fmt.Sprintf("ps | grep -q %q", bin+" [r]un ")
+	if err := exec.Command("sh", "-c", shellCmd).Run(); err != nil {
+		return service.StatusStopped, nil
+	}
+	return service.StatusRunning, nil
+}
+
+// systemd wraps a service.Service, and provide status command to
+// report the status correctly.
+type systemd struct {
+	service.Service
+}
+
+func (s *systemd) Status() (service.Status, error) {
+	out, _ := exec.Command("systemctl", "status", "ctrld").CombinedOutput()
+	if bytes.Contains(out, []byte("/FAILURE)")) {
+		return service.StatusStopped, nil
+	}
+	return s.Service.Status()
+}
+
+func newLaunchd(s service.Service) *launchd {
+	return &launchd{
+		Service:      s,
+		statusErrMsg: "Permission denied",
+	}
+}
+
+// launchd wraps a service.Service, and provide status command to
+// report the status correctly when not running as root on Darwin.
+//
+// TODO: remove this wrapper once https://github.com/kardianos/service/issues/400 fixed.
+type launchd struct {
+	service.Service
+	statusErrMsg string
+}
+
+func (l *launchd) Status() (service.Status, error) {
+	if os.Geteuid() != 0 {
+		return service.StatusUnknown, errors.New(l.statusErrMsg)
+	}
+	return l.Service.Status()
+}
+
+type task struct {
+	f            func() error
+	abortOnError bool
+}
+
+func doTasks(tasks []task) bool {
+	var prevErr error
+	for _, task := range tasks {
+		if err := task.f(); err != nil {
+			if task.abortOnError {
+				mainLog.Load().Error().Msg(errors.Join(prevErr, err).Error())
+				return false
+			}
+			prevErr = err
+		}
+	}
+	return true
+}
+
+func checkHasElevatedPrivilege() {
+	ok, err := hasElevatedPrivilege()
+	if err != nil {
+		mainLog.Load().Error().Msgf("could not detect user privilege: %v", err)
+		return
+	}
+	if !ok {
+		mainLog.Load().Error().Msg("Please relaunch process with admin/root privilege.")
+		os.Exit(1)
+	}
+}
+
+func unixSystemVServiceStatus() (service.Status, error) {
+	out, err := exec.Command("/etc/init.d/ctrld", "status").CombinedOutput()
+	if err != nil {
+		return service.StatusUnknown, nil
+	}
+
+	switch string(bytes.ToLower(bytes.TrimSpace(out))) {
+	case "running":
+		return service.StatusRunning, nil
+	default:
+		return service.StatusStopped, nil
+	}
+}

cmd/cli/service_others.go

@@ -0,0 +1,15 @@
+//go:build !windows
+
+package cli
+
+import (
+	"os"
+)
+
+func hasElevatedPrivilege() (bool, error) {
+	return os.Geteuid() == 0, nil
+}
+
+func openLogFile(path string, flags int) (*os.File, error) {
+	return os.OpenFile(path, flags, os.FileMode(0o600))
+}

cmd/cli/service_windows.go

@@ -0,0 +1,81 @@
+package cli
+
+import (
+	"os"
+	"syscall"
+
+	"golang.org/x/sys/windows"
+)
+
+func hasElevatedPrivilege() (bool, error) {
+	var sid *windows.SID
+	if err := windows.AllocateAndInitializeSid(
+		&windows.SECURITY_NT_AUTHORITY,
+		2,
+		windows.SECURITY_BUILTIN_DOMAIN_RID,
+		windows.DOMAIN_ALIAS_RID_ADMINS,
+		0,
+		0,
+		0,
+		0,
+		0,
+		0,
+		&sid,
+	); err != nil {
+		return false, err
+	}
+	token := windows.Token(0)
+	return token.IsMember(sid)
+}
+
+func openLogFile(path string, mode int) (*os.File, error) {
+	if len(path) == 0 {
+		return nil, &os.PathError{Path: path, Op: "open", Err: syscall.ERROR_FILE_NOT_FOUND}
+	}
+
+	pathP, err := syscall.UTF16PtrFromString(path)
+	if err != nil {
+		return nil, err
+	}
+	var access uint32
+	switch mode & (os.O_RDONLY | os.O_WRONLY | os.O_RDWR) {
+	case os.O_RDONLY:
+		access = windows.GENERIC_READ
+	case os.O_WRONLY:
+		access = windows.GENERIC_WRITE
+	case os.O_RDWR:
+		access = windows.GENERIC_READ | windows.GENERIC_WRITE
+	}
+	if mode&os.O_CREATE != 0 {
+		access |= windows.GENERIC_WRITE
+	}
+	if mode&os.O_APPEND != 0 {
+		access &^= windows.GENERIC_WRITE
+		access |= windows.FILE_APPEND_DATA
+	}
+
+	shareMode := uint32(syscall.FILE_SHARE_READ | syscall.FILE_SHARE_WRITE | syscall.FILE_SHARE_DELETE)
+
+	var sa *syscall.SecurityAttributes
+
+	var createMode uint32
+	switch {
+	case mode&(os.O_CREATE|os.O_EXCL) == (os.O_CREATE | os.O_EXCL):
+		createMode = windows.CREATE_NEW
+	case mode&(os.O_CREATE|os.O_TRUNC) == (os.O_CREATE | os.O_TRUNC):
+		createMode = windows.CREATE_ALWAYS
+	case mode&os.O_CREATE == os.O_CREATE:
+		createMode = windows.OPEN_ALWAYS
+	case mode&os.O_TRUNC == os.O_TRUNC:
+		createMode = windows.TRUNCATE_EXISTING
+	default:
+		createMode = windows.OPEN_EXISTING
+	}
+
+	handle, err := syscall.CreateFile(pathP, access, shareMode, sa, createMode, syscall.FILE_ATTRIBUTE_NORMAL, 0)
+	if err != nil {
+		return nil, &os.PathError{Path: path, Op: "open", Err: err}
+	}
+
+	return os.NewFile(uintptr(handle), path), nil
+}

cmd/cli/upstream_monitor.go

@@ -0,0 +1,118 @@
+package cli
+
+import (
+	"context"
+	"sync"
+	"time"
+
+	"github.com/miekg/dns"
+
+	"github.com/Control-D-Inc/ctrld"
+)
+
+const (
+	// maxFailureRequest is the maximum failed queries allowed before an upstream is marked as down.
+	maxFailureRequest = 100
+	// checkUpstreamBackoffSleep is the time interval between each upstream checks.
+	checkUpstreamBackoffSleep = 2 * time.Second
+)
+
+// upstreamMonitor performs monitoring upstreams health.
+type upstreamMonitor struct {
+	cfg *ctrld.Config
+
+	mu         sync.Mutex
+	checking   map[string]bool
+	down       map[string]bool
+	failureReq map[string]uint64
+}
+
+func newUpstreamMonitor(cfg *ctrld.Config) *upstreamMonitor {
+	um := &upstreamMonitor{
+		cfg:        cfg,
+		checking:   make(map[string]bool),
+		down:       make(map[string]bool),
+		failureReq: make(map[string]uint64),
+	}
+	for n := range cfg.Upstream {
+		upstream := upstreamPrefix + n
+		um.reset(upstream)
+	}
+	um.reset(upstreamOS)
+	return um
+}
+
+// increaseFailureCount increase failed queries count for an upstream by 1.
+func (um *upstreamMonitor) increaseFailureCount(upstream string) {
+	um.mu.Lock()
+	defer um.mu.Unlock()
+
+	um.failureReq[upstream] += 1
+	failedCount := um.failureReq[upstream]
+	um.down[upstream] = failedCount >= maxFailureRequest
+}
+
+// isDown reports whether the given upstream is being marked as down.
+func (um *upstreamMonitor) isDown(upstream string) bool {
+	um.mu.Lock()
+	defer um.mu.Unlock()
+
+	return um.down[upstream]
+}
+
+// reset marks an upstream as up and set failed queries counter to zero.
+func (um *upstreamMonitor) reset(upstream string) {
+	um.mu.Lock()
+	defer um.mu.Unlock()
+
+	um.failureReq[upstream] = 0
+	um.down[upstream] = false
+}
+
+// checkUpstream checks the given upstream status, periodically sending query to upstream
+// until successfully. An upstream status/counter will be reset once it becomes reachable.
+func (p *prog) checkUpstream(upstream string, uc *ctrld.UpstreamConfig) {
+	p.um.mu.Lock()
+	isChecking := p.um.checking[upstream]
+	if isChecking {
+		p.um.mu.Unlock()
+		return
+	}
+	p.um.checking[upstream] = true
+	p.um.mu.Unlock()
+	defer func() {
+		p.um.mu.Lock()
+		p.um.checking[upstream] = false
+		p.um.mu.Unlock()
+	}()
+
+	resolver, err := ctrld.NewResolver(uc)
+	if err != nil {
+		mainLog.Load().Warn().Err(err).Msg("could not check upstream")
+		return
+	}
+	msg := new(dns.Msg)
+	msg.SetQuestion(".", dns.TypeNS)
+
+	check := func() error {
+		ctx, cancel := context.WithTimeout(context.Background(), time.Second)
+		defer cancel()
+		uc.ReBootstrap()
+		_, err := resolver.Resolve(ctx, msg)
+		return err
+	}
+	for {
+		if err := check(); err == nil {
+			mainLog.Load().Debug().Msgf("upstream %q is online", uc.Endpoint)
+			p.um.reset(upstream)
+			if p.leakingQuery.CompareAndSwap(true, false) {
+				p.leakingQueryMu.Lock()
+				p.leakingQueryWasRun = false
+				p.leakingQueryMu.Unlock()
+				mainLog.Load().Warn().Msg("stop leaking query")
+			}
+			return
+		}
+		time.Sleep(checkUpstreamBackoffSleep)
+	}
+}

cmd/cli/winres/winres.json

@@ -0,0 +1,20 @@
+{
+  "RT_VERSION": {
+    "#1": {
+      "0000": {
+        "fixed": {
+          "file_version": "0.0.0.1"
+        },
+        "info": {
+          "0409": {
+            "CompanyName": "ControlD Inc",
+            "FileDescription": "Control D DNS daemon",
+            "ProductName": "ctrld",
+            "InternalName": "ctrld",
+            "LegalCopyright": "ControlD Inc 2024"
+          }
+        }
+      }
+    }
+  }
+}

cmd/cli/winres_windows.go

@@ -0,0 +1,4 @@
+//go:generate go-winres make --product-version=git-tag --file-version=git-tag
+package cli
+
+// Placeholder file for windows builds.

cmd/ctrld/cli.go

@@ -1,668 +0,0 @@
-package main
-
-import (
-	"bytes"
-	"encoding/base64"
-	"errors"
-	"fmt"
-	"log"
-	"net"
-	"net/netip"
-	"os"
-	"os/exec"
-	"path/filepath"
-	"runtime"
-	"strconv"
-	"strings"
-
-	"github.com/go-playground/validator/v10"
-	"github.com/kardianos/service"
-	"github.com/pelletier/go-toml/v2"
-	"github.com/spf13/cobra"
-	"github.com/spf13/viper"
-	"tailscale.com/net/interfaces"
-
-	"github.com/Control-D-Inc/ctrld"
-	"github.com/Control-D-Inc/ctrld/internal/controld"
-)
-
-var (
-	v                    = viper.NewWithOptions(viper.KeyDelimiter("::"))
-	defaultConfigWritten = false
-	defaultConfigFile    = "ctrld.toml"
-)
-
-var basicModeFlags = []string{"listen", "primary_upstream", "secondary_upstream", "domains"}
-
-func isNoConfigStart(cmd *cobra.Command) bool {
-	for _, flagName := range basicModeFlags {
-		if cmd.Flags().Lookup(flagName).Changed {
-			return true
-		}
-	}
-	return false
-}
-
-const rootShortDesc = `
-        __         .__       .___
-  _____/  |________|  |    __| _/
-_/ ___\   __\_  __ \  |   / __ |
-\  \___|  |  |  | \/  |__/ /_/ |
- \___  >__|  |__|  |____/\____ |
-     \/ dns forwarding proxy  \/
-`
-
-func initCLI() {
-	// Enable opening via explorer.exe on Windows.
-	// See: https://github.com/spf13/cobra/issues/844.
-	cobra.MousetrapHelpText = ""
-
-	rootCmd := &cobra.Command{
-		Use:     "ctrld",
-		Short:   strings.TrimLeft(rootShortDesc, "\n"),
-		Version: "1.1.0",
-	}
-	rootCmd.PersistentFlags().CountVarP(
-		&verbose,
-		"verbose",
-		"v",
-		`verbose log output, "-v" basic logging, "-vv" debug level logging`,
-	)
-	rootCmd.SetHelpCommand(&cobra.Command{Hidden: true})
-	rootCmd.CompletionOptions.HiddenDefaultCmd = true
-
-	runCmd := &cobra.Command{
-		Use:   "run",
-		Short: "Run the DNS proxy server",
-		Args:  cobra.NoArgs,
-		Run: func(cmd *cobra.Command, args []string) {
-			if daemon && runtime.GOOS == "windows" {
-				log.Fatal("Cannot run in daemon mode. Please install a Windows service.")
-			}
-
-			noConfigStart := isNoConfigStart(cmd)
-			writeDefaultConfig := !noConfigStart && configBase64 == ""
-			configs := []struct {
-				name    string
-				written bool
-			}{
-				// For compatibility, we check for config.toml first, but only read it if exists.
-				{"config", false},
-				{"ctrld", writeDefaultConfig},
-			}
-			for _, config := range configs {
-				ctrld.SetConfigName(v, config.name)
-				v.SetConfigFile(configPath)
-				if readConfigFile(config.written) {
-					break
-				}
-			}
-
-			readBase64Config()
-			processNoConfigFlags(noConfigStart)
-			if err := v.Unmarshal(&cfg); err != nil {
-				log.Fatalf("failed to unmarshal config: %v", err)
-			}
-			// Wait for network up.
-			if !netUp() {
-				log.Fatal("network is not up yet")
-			}
-			processLogAndCacheFlags()
-			// Log config do not have thing to validate, so it's safe to init log here,
-			// so it's able to log information in processCDFlags.
-			initLogging()
-			processCDFlags()
-			if err := ctrld.ValidateConfig(validator.New(), &cfg); err != nil {
-				log.Fatalf("invalid config: %v", err)
-			}
-			initCache()
-
-			if daemon {
-				exe, err := os.Executable()
-				if err != nil {
-					mainLog.Error().Err(err).Msg("failed to find the binary")
-					os.Exit(1)
-				}
-				curDir, err := os.Getwd()
-				if err != nil {
-					mainLog.Error().Err(err).Msg("failed to get current working directory")
-					os.Exit(1)
-				}
-				// If running as daemon, re-run the command in background, with daemon off.
-				cmd := exec.Command(exe, append(os.Args[1:], "-d=false")...)
-				cmd.Dir = curDir
-				if err := cmd.Start(); err != nil {
-					mainLog.Error().Err(err).Msg("failed to start process as daemon")
-					os.Exit(1)
-				}
-				mainLog.Info().Int("pid", cmd.Process.Pid).Msg("DNS proxy started")
-				os.Exit(0)
-			}
-
-			s, err := service.New(&prog{}, svcConfig)
-			if err != nil {
-				mainLog.Fatal().Err(err).Msg("failed create new service")
-			}
-			serviceLogger, err := s.Logger(nil)
-			if err != nil {
-				mainLog.Error().Err(err).Msg("failed to get service logger")
-				return
-			}
-
-			if err := s.Run(); err != nil {
-				if sErr := serviceLogger.Error(err); sErr != nil {
-					mainLog.Error().Err(sErr).Msg("failed to write service log")
-				}
-				mainLog.Error().Err(err).Msg("failed to start service")
-			}
-		},
-	}
-	runCmd.Flags().BoolVarP(&daemon, "daemon", "d", false, "Run as daemon")
-	runCmd.Flags().StringVarP(&configPath, "config", "c", "", "Path to config file")
-	runCmd.Flags().StringVarP(&configBase64, "base64_config", "", "", "Base64 encoded config")
-	runCmd.Flags().StringVarP(&listenAddress, "listen", "", "", "Listener address and port, in format: address:port")
-	runCmd.Flags().StringVarP(&primaryUpstream, "primary_upstream", "", "", "Primary upstream endpoint")
-	runCmd.Flags().StringVarP(&secondaryUpstream, "secondary_upstream", "", "", "Secondary upstream endpoint")
-	runCmd.Flags().StringSliceVarP(&domains, "domains", "", nil, "List of domain to apply in a split DNS policy")
-	runCmd.Flags().StringVarP(&logPath, "log", "", "", "Path to log file")
-	runCmd.Flags().IntVarP(&cacheSize, "cache_size", "", 0, "Enable cache with size items")
-	runCmd.Flags().StringVarP(&cdUID, "cd", "", "", "Control D resolver uid")
-	runCmd.Flags().StringVarP(&homedir, "homedir", "", "", "")
-	_ = runCmd.Flags().MarkHidden("homedir")
-	runCmd.Flags().StringVarP(&iface, "iface", "", "", `Update DNS setting for iface, "auto" means the default interface gateway`)
-	_ = runCmd.Flags().MarkHidden("iface")
-
-	rootCmd.AddCommand(runCmd)
-
-	startCmd := &cobra.Command{
-		PreRun: checkHasElevatedPrivilege,
-		Use:    "start",
-		Short:  "Start the ctrld service",
-		Args:   cobra.NoArgs,
-		Run: func(cmd *cobra.Command, args []string) {
-			sc := &service.Config{}
-			*sc = *svcConfig
-			osArgs := os.Args[2:]
-			if os.Args[1] == "service" {
-				osArgs = os.Args[3:]
-			}
-			setDependencies(sc)
-			sc.Arguments = append([]string{"run"}, osArgs...)
-			if dir, err := os.UserHomeDir(); err == nil {
-				// WorkingDirectory is not supported on Windows.
-				sc.WorkingDirectory = dir
-				// No config path, generating config in HOME directory.
-				noConfigStart := isNoConfigStart(cmd)
-				writeDefaultConfig := !noConfigStart && configBase64 == ""
-				if configPath == "" && writeDefaultConfig {
-					defaultConfigFile = filepath.Join(dir, defaultConfigFile)
-					readConfigFile(writeDefaultConfig && cdUID == "")
-				}
-				sc.Arguments = append(sc.Arguments, "--homedir="+dir)
-			}
-
-			initLogging()
-			processCDFlags()
-			// On Windows, the service will be run as SYSTEM, so if ctrld start as Admin,
-			// the user home dir is different, so pass specific arguments that relevant here.
-			if runtime.GOOS == "windows" {
-				if configPath == "" {
-					sc.Arguments = append(sc.Arguments, "--config="+defaultConfigFile)
-				}
-			}
-
-			prog := &prog{}
-			s, err := service.New(prog, sc)
-			if err != nil {
-				stderrMsg(err.Error())
-				return
-			}
-			tasks := []task{
-				{s.Stop, false},
-				{s.Uninstall, false},
-				{s.Install, false},
-				{s.Start, true},
-			}
-			if doTasks(tasks) {
-				prog.setDNS()
-				mainLog.Info().Msg("Service started")
-			}
-		},
-	}
-	// Keep these flags in sync with runCmd above, except for "-d".
-	startCmd.Flags().StringVarP(&configPath, "config", "c", "", "Path to config file")
-	startCmd.Flags().StringVarP(&configBase64, "base64_config", "", "", "Base64 encoded config")
-	startCmd.Flags().StringVarP(&listenAddress, "listen", "", "", "Listener address and port, in format: address:port")
-	startCmd.Flags().StringVarP(&primaryUpstream, "primary_upstream", "", "", "Primary upstream endpoint")
-	startCmd.Flags().StringVarP(&secondaryUpstream, "secondary_upstream", "", "", "Secondary upstream endpoint")
-	startCmd.Flags().StringSliceVarP(&domains, "domains", "", nil, "List of domain to apply in a split DNS policy")
-	startCmd.Flags().StringVarP(&logPath, "log", "", "", "Path to log file")
-	startCmd.Flags().IntVarP(&cacheSize, "cache_size", "", 0, "Enable cache with size items")
-	startCmd.Flags().StringVarP(&cdUID, "cd", "", "", "Control D resolver uid")
-	startCmd.Flags().StringVarP(&iface, "iface", "", "", `Update DNS setting for iface, "auto" means the default interface gateway`)
-
-	stopCmd := &cobra.Command{
-		PreRun: checkHasElevatedPrivilege,
-		Use:    "stop",
-		Short:  "Stop the ctrld service",
-		Args:   cobra.NoArgs,
-		Run: func(cmd *cobra.Command, args []string) {
-			prog := &prog{}
-			s, err := service.New(prog, svcConfig)
-			if err != nil {
-				stderrMsg(err.Error())
-				return
-			}
-			initLogging()
-			if doTasks([]task{{s.Stop, true}}) {
-				prog.resetDNS()
-				mainLog.Info().Msg("Service stopped")
-			}
-		},
-	}
-	stopCmd.Flags().StringVarP(&iface, "iface", "", "", `Reset DNS setting for iface, "auto" means the default interface gateway`)
-
-	restartCmd := &cobra.Command{
-		PreRun: checkHasElevatedPrivilege,
-		Use:    "restart",
-		Short:  "Restart the ctrld service",
-		Args:   cobra.NoArgs,
-		Run: func(cmd *cobra.Command, args []string) {
-			s, err := service.New(&prog{}, svcConfig)
-			if err != nil {
-				stderrMsg(err.Error())
-				return
-			}
-			initLogging()
-			if doTasks([]task{{s.Restart, true}}) {
-				stdoutMsg("Service restarted")
-			}
-		},
-	}
-
-	statusCmd := &cobra.Command{
-		Use:   "status",
-		Short: "Show status of the ctrld service",
-		Args:  cobra.NoArgs,
-		Run: func(cmd *cobra.Command, args []string) {
-			s, err := service.New(&prog{}, svcConfig)
-			if err != nil {
-				stderrMsg(err.Error())
-				return
-			}
-			status, err := s.Status()
-			if err != nil {
-				stderrMsg(err.Error())
-				return
-			}
-			switch status {
-			case service.StatusUnknown:
-				stdoutMsg("Unknown status")
-			case service.StatusRunning:
-				stdoutMsg("Service is running")
-			case service.StatusStopped:
-				stdoutMsg("Service is stopped")
-			}
-		},
-	}
-
-	uninstallCmd := &cobra.Command{
-		PreRun: checkHasElevatedPrivilege,
-		Use:    "uninstall",
-		Short:  "Uninstall the ctrld service",
-		Args:   cobra.NoArgs,
-		Run: func(cmd *cobra.Command, args []string) {
-			prog := &prog{}
-			s, err := service.New(prog, svcConfig)
-			if err != nil {
-				stderrMsg(err.Error())
-				return
-			}
-			tasks := []task{
-				{s.Stop, false},
-				{s.Uninstall, true},
-			}
-			initLogging()
-			if doTasks(tasks) {
-				prog.resetDNS()
-				mainLog.Info().Msg("Service uninstalled")
-				return
-			}
-		},
-	}
-	uninstallCmd.Flags().StringVarP(&iface, "iface", "", "auto", `Reset DNS setting for iface, "auto" means the default interface gateway`)
-
-	listIfacesCmd := &cobra.Command{
-		Use:   "list",
-		Short: "List network interfaces of the host",
-		Args:  cobra.NoArgs,
-		Run: func(cmd *cobra.Command, args []string) {
-			err := interfaces.ForeachInterface(func(i interfaces.Interface, prefixes []netip.Prefix) {
-				fmt.Printf("Index : %d\n", i.Index)
-				fmt.Printf("Name  : %s\n", i.Name)
-				addrs, _ := i.Addrs()
-				for i, ipaddr := range addrs {
-					if i == 0 {
-						fmt.Printf("Addrs : %v\n", ipaddr)
-						continue
-					}
-					fmt.Printf("        %v\n", ipaddr)
-				}
-				for i, dns := range currentDNS(i.Interface) {
-					if i == 0 {
-						fmt.Printf("DNS   : %s\n", dns)
-						continue
-					}
-					fmt.Printf("      : %s\n", dns)
-				}
-				println()
-			})
-			if err != nil {
-				stderrMsg(err.Error())
-			}
-		},
-	}
-	interfacesCmd := &cobra.Command{
-		Use:   "interfaces",
-		Short: "Manage network interfaces",
-		Args:  cobra.OnlyValidArgs,
-		ValidArgs: []string{
-			listIfacesCmd.Use,
-		},
-	}
-	interfacesCmd.AddCommand(listIfacesCmd)
-
-	serviceCmd := &cobra.Command{
-		Use:   "service",
-		Short: "Manage ctrld service",
-		Args:  cobra.OnlyValidArgs,
-		ValidArgs: []string{
-			statusCmd.Use,
-			stopCmd.Use,
-			restartCmd.Use,
-			statusCmd.Use,
-			uninstallCmd.Use,
-			interfacesCmd.Use,
-		},
-	}
-	serviceCmd.AddCommand(startCmd)
-	serviceCmd.AddCommand(stopCmd)
-	serviceCmd.AddCommand(restartCmd)
-	serviceCmd.AddCommand(statusCmd)
-	serviceCmd.AddCommand(uninstallCmd)
-	serviceCmd.AddCommand(interfacesCmd)
-	rootCmd.AddCommand(serviceCmd)
-	startCmdAlias := &cobra.Command{
-		PreRun: checkHasElevatedPrivilege,
-		Use:    "start",
-		Short:  "Quick start service and configure DNS on interface",
-		Run: func(cmd *cobra.Command, args []string) {
-			if !cmd.Flags().Changed("iface") {
-				os.Args = append(os.Args, "--iface="+ifaceStartStop)
-			}
-			iface = ifaceStartStop
-			startCmd.Run(cmd, args)
-		},
-	}
-	startCmdAlias.Flags().StringVarP(&ifaceStartStop, "iface", "", "auto", `Update DNS setting for iface, "auto" means the default interface gateway`)
-	startCmdAlias.Flags().AddFlagSet(startCmd.Flags())
-	rootCmd.AddCommand(startCmdAlias)
-	stopCmdAlias := &cobra.Command{
-		PreRun: checkHasElevatedPrivilege,
-		Use:    "stop",
-		Short:  "Quick stop service and remove DNS from interface",
-		Run: func(cmd *cobra.Command, args []string) {
-			if !cmd.Flags().Changed("iface") {
-				os.Args = append(os.Args, "--iface="+ifaceStartStop)
-			}
-			iface = ifaceStartStop
-			stopCmd.Run(cmd, args)
-		},
-	}
-	stopCmdAlias.Flags().StringVarP(&ifaceStartStop, "iface", "", "auto", `Reset DNS setting for iface, "auto" means the default interface gateway`)
-	stopCmdAlias.Flags().AddFlagSet(stopCmd.Flags())
-	rootCmd.AddCommand(stopCmdAlias)
-
-	if err := rootCmd.Execute(); err != nil {
-		stderrMsg(err.Error())
-		os.Exit(1)
-	}
-}
-
-func writeConfigFile() error {
-	if cfu := v.ConfigFileUsed(); cfu != "" {
-		defaultConfigFile = cfu
-	}
-	f, err := os.OpenFile(defaultConfigFile, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, os.FileMode(0o644))
-	if err != nil {
-		return err
-	}
-	defer f.Close()
-	if cdUID != "" {
-		if _, err := f.WriteString("# AUTO-GENERATED VIA CD FLAG - DO NOT MODIFY\n\n"); err != nil {
-			return err
-		}
-	}
-	enc := toml.NewEncoder(f).SetIndentTables(true)
-	if err := enc.Encode(v.AllSettings()); err != nil {
-		return err
-	}
-	if err := f.Close(); err != nil {
-		return err
-	}
-	return nil
-}
-
-func readConfigFile(writeDefaultConfig bool) bool {
-	// If err == nil, there's a config supplied via `--config`, no default config written.
-	err := v.ReadInConfig()
-	if err == nil {
-		fmt.Println("loading config file from:", v.ConfigFileUsed())
-		defaultConfigFile = v.ConfigFileUsed()
-		return true
-	}
-
-	if !writeDefaultConfig {
-		return false
-	}
-
-	// If error is viper.ConfigFileNotFoundError, write default config.
-	if _, ok := err.(viper.ConfigFileNotFoundError); ok {
-		if err := writeConfigFile(); err != nil {
-			log.Fatalf("failed to write default config file: %v", err)
-		} else {
-			fmt.Println("writing default config file to: " + defaultConfigFile)
-		}
-		defaultConfigWritten = true
-		return false
-	}
-	// Otherwise, report fatal error and exit.
-	log.Fatalf("failed to decode config file: %v", err)
-	return false
-}
-
-func readBase64Config() {
-	if configBase64 == "" {
-		return
-	}
-	configStr, err := base64.StdEncoding.DecodeString(configBase64)
-	if err != nil {
-		log.Fatalf("invalid base64 config: %v", err)
-	}
-	if err := v.ReadConfig(bytes.NewReader(configStr)); err != nil {
-		log.Fatalf("failed to read base64 config: %v", err)
-	}
-}
-
-func processNoConfigFlags(noConfigStart bool) {
-	if !noConfigStart {
-		return
-	}
-	if listenAddress == "" || primaryUpstream == "" {
-		log.Fatal(`"listen" and "primary_upstream" flags must be set in no config mode`)
-	}
-	processListenFlag()
-
-	upstream := map[string]*ctrld.UpstreamConfig{
-		"0": {
-			Name:     primaryUpstream,
-			Endpoint: primaryUpstream,
-			Type:     ctrld.ResolverTypeDOH,
-		},
-	}
-	if secondaryUpstream != "" {
-		upstream["1"] = &ctrld.UpstreamConfig{
-			Name:     secondaryUpstream,
-			Endpoint: secondaryUpstream,
-			Type:     ctrld.ResolverTypeLegacy,
-		}
-		rules := make([]ctrld.Rule, 0, len(domains))
-		for _, domain := range domains {
-			rules = append(rules, ctrld.Rule{domain: []string{"upstream.1"}})
-		}
-		lc := v.Get("listener").(map[string]*ctrld.ListenerConfig)["0"]
-		lc.Policy = &ctrld.ListenerPolicyConfig{Name: "My Policy", Rules: rules}
-	}
-	v.Set("upstream", upstream)
-}
-
-func processCDFlags() {
-	if cdUID == "" {
-		return
-	}
-	if iface == "" {
-		iface = "auto"
-	}
-	logger := mainLog.With().Str("mode", "cd").Logger()
-	logger.Info().Msg("fetching Controld-D configuration")
-	resolverConfig, err := controld.FetchResolverConfig(cdUID)
-	if uer, ok := err.(*controld.UtilityErrorResponse); ok && uer.ErrorField.Code == controld.InvalidConfigCode {
-		s, err := service.New(&prog{}, svcConfig)
-		if err != nil {
-			logger.Warn().Err(err).Msg("failed to create new service")
-			return
-		}
-
-		if netIface, _ := netInterface(iface); netIface != nil {
-			if err := restoreNetworkManager(); err != nil {
-				logger.Error().Err(err).Msg("could not restore NetworkManager")
-				return
-			}
-			logger.Debug().Str("iface", netIface.Name).Msg("Restoring DNS for interface")
-			if err := resetDNS(netIface); err != nil {
-				logger.Warn().Err(err).Msg("something went wrong while restoring DNS")
-			} else {
-				logger.Debug().Str("iface", netIface.Name).Msg("Restoring DNS successfully")
-			}
-		}
-
-		tasks := []task{{s.Uninstall, true}}
-		if doTasks(tasks) {
-			logger.Info().Msg("uninstalled service")
-		}
-		logger.Fatal().Err(uer).Msg("failed to fetch resolver config")
-	}
-	if err != nil {
-		logger.Warn().Err(err).Msg("could not fetch resolver config")
-		return
-	}
-
-	logger.Info().Msg("generating ctrld config from Controld-D configuration")
-	cfg = ctrld.Config{}
-	cfg.Network = make(map[string]*ctrld.NetworkConfig)
-	cfg.Network["0"] = &ctrld.NetworkConfig{
-		Name:  "Network 0",
-		Cidrs: []string{"0.0.0.0/0"},
-	}
-	cfg.Upstream = make(map[string]*ctrld.UpstreamConfig)
-	cfg.Upstream["0"] = &ctrld.UpstreamConfig{
-		Endpoint: resolverConfig.DOH,
-		Type:     ctrld.ResolverTypeDOH,
-		Timeout:  5000,
-	}
-	rules := make([]ctrld.Rule, 0, len(resolverConfig.Exclude))
-	for _, domain := range resolverConfig.Exclude {
-		rules = append(rules, ctrld.Rule{domain: []string{}})
-	}
-	cfg.Listener = make(map[string]*ctrld.ListenerConfig)
-	cfg.Listener["0"] = &ctrld.ListenerConfig{
-		IP:   "127.0.0.1",
-		Port: 53,
-		Policy: &ctrld.ListenerPolicyConfig{
-			Name:  "My Policy",
-			Rules: rules,
-		},
-	}
-
-	v = viper.NewWithOptions(viper.KeyDelimiter("::"))
-	v.Set("network", cfg.Network)
-	v.Set("upstream", cfg.Upstream)
-	v.Set("listener", cfg.Listener)
-	processLogAndCacheFlags()
-	if err := writeConfigFile(); err != nil {
-		logger.Fatal().Err(err).Msg("failed to write config file")
-	} else {
-		logger.Info().Msg("writing config file to: " + defaultConfigFile)
-	}
-}
-
-func processListenFlag() {
-	if listenAddress == "" {
-		return
-	}
-	host, portStr, err := net.SplitHostPort(listenAddress)
-	if err != nil {
-		log.Fatalf("invalid listener address: %v", err)
-	}
-	port, err := strconv.Atoi(portStr)
-	if err != nil {
-		log.Fatalf("invalid port number: %v", err)
-	}
-	lc := &ctrld.ListenerConfig{
-		IP:   host,
-		Port: port,
-	}
-	v.Set("listener", map[string]*ctrld.ListenerConfig{
-		"0": lc,
-	})
-}
-
-func processLogAndCacheFlags() {
-	if logPath != "" {
-		cfg.Service.LogLevel = "debug"
-		cfg.Service.LogPath = logPath
-	}
-
-	if cacheSize != 0 {
-		cfg.Service.CacheEnable = true
-		cfg.Service.CacheSize = cacheSize
-	}
-	v.Set("service", cfg.Service)
-}
-
-func netInterface(ifaceName string) (*net.Interface, error) {
-	if ifaceName == "auto" {
-		ifaceName = defaultIfaceName()
-	}
-	var iface *net.Interface
-	err := interfaces.ForeachInterface(func(i interfaces.Interface, prefixes []netip.Prefix) {
-		if i.Name == ifaceName {
-			iface = i.Interface
-		}
-	})
-	if iface == nil {
-		return nil, errors.New("interface not found")
-	}
-	if err := patchNetIfaceName(iface); err != nil {
-		return nil, err
-	}
-	return iface, err
-}
-
-func defaultIfaceName() string {
-	dri, err := interfaces.DefaultRouteInterface()
-	if err != nil {
-		mainLog.Fatal().Err(err).Msg("failed to get default route interface")
-	}
-	return dri
-}

cmd/ctrld/dns_proxy.go

@@ -1,321 +0,0 @@
-package main
-
-import (
-	"context"
-	"crypto/rand"
-	"encoding/hex"
-	"fmt"
-	"net"
-	"runtime"
-	"strconv"
-	"strings"
-	"time"
-
-	"github.com/miekg/dns"
-
-	"github.com/Control-D-Inc/ctrld"
-	"github.com/Control-D-Inc/ctrld/internal/dnscache"
-)
-
-const staleTTL = 60 * time.Second
-
-func (p *prog) serveUDP(listenerNum string) error {
-	listenerConfig := p.cfg.Listener[listenerNum]
-	// make sure ip is allocated
-	if allocErr := p.allocateIP(listenerConfig.IP); allocErr != nil {
-		mainLog.Error().Err(allocErr).Str("ip", listenerConfig.IP).Msg("serveUDP: failed to allocate listen ip")
-		return allocErr
-	}
-	var failoverRcodes []int
-	if listenerConfig.Policy != nil {
-		failoverRcodes = listenerConfig.Policy.FailoverRcodeNumbers
-	}
-	handler := dns.HandlerFunc(func(w dns.ResponseWriter, m *dns.Msg) {
-		domain := canonicalName(m.Question[0].Name)
-		reqId := requestID()
-		fmtSrcToDest := fmtRemoteToLocal(listenerNum, w.RemoteAddr().String(), w.LocalAddr().String())
-		t := time.Now()
-		ctx := context.WithValue(context.Background(), ctrld.ReqIdCtxKey{}, reqId)
-		ctrld.Log(ctx, proxyLog.Debug(), "%s received query: %s", fmtSrcToDest, domain)
-		upstreams, matched := p.upstreamFor(ctx, listenerNum, listenerConfig, w.RemoteAddr(), domain)
-		var answer *dns.Msg
-		if !matched && listenerConfig.Restricted {
-			answer = new(dns.Msg)
-			answer.SetRcode(m, dns.RcodeRefused)
-
-		} else {
-			answer = p.proxy(ctx, upstreams, failoverRcodes, m)
-			rtt := time.Since(t)
-			ctrld.Log(ctx, proxyLog.Debug(), "received response of %d bytes in %s", answer.Len(), rtt)
-		}
-		if err := w.WriteMsg(answer); err != nil {
-			ctrld.Log(ctx, mainLog.Error().Err(err), "serveUDP: failed to send DNS response to client")
-		}
-	})
-
-	// On Windows, there's no easy way for disabling/removing IPv6 DNS resolver, so we check whether we can
-	// listen on ::1, then spawn a listener for receiving DNS requests.
-	if runtime.GOOS == "windows" && supportsIPv6ListenLocal() {
-		go func() {
-			s := &dns.Server{
-				Addr:    net.JoinHostPort("::1", strconv.Itoa(listenerConfig.Port)),
-				Net:     "udp",
-				Handler: handler,
-			}
-			_ = s.ListenAndServe()
-		}()
-	}
-
-	s := &dns.Server{
-		Addr:    net.JoinHostPort(listenerConfig.IP, strconv.Itoa(listenerConfig.Port)),
-		Net:     "udp",
-		Handler: handler,
-	}
-	return s.ListenAndServe()
-}
-
-func (p *prog) upstreamFor(ctx context.Context, defaultUpstreamNum string, lc *ctrld.ListenerConfig, addr net.Addr, domain string) ([]string, bool) {
-	upstreams := []string{"upstream." + defaultUpstreamNum}
-	matchedPolicy := "no policy"
-	matchedNetwork := "no network"
-	matchedRule := "no rule"
-	matched := false
-
-	defer func() {
-		if !matched && lc.Restricted {
-			ctrld.Log(ctx, proxyLog.Info(), "query refused, %s does not match any network policy", addr.String())
-			return
-		}
-		ctrld.Log(ctx, proxyLog.Info(), "%s, %s, %s -> %v", matchedPolicy, matchedNetwork, matchedRule, upstreams)
-	}()
-
-	if lc.Policy == nil {
-		return upstreams, false
-	}
-
-	do := func(policyUpstreams []string) {
-		upstreams = append([]string(nil), policyUpstreams...)
-	}
-
-	for _, rule := range lc.Policy.Rules {
-		// There's only one entry per rule, config validation ensures this.
-		for source, targets := range rule {
-			if source == domain || wildcardMatches(source, domain) {
-				matchedPolicy = lc.Policy.Name
-				matchedRule = source
-				do(targets)
-				matched = true
-				return upstreams, matched
-			}
-		}
-	}
-
-	var sourceIP net.IP
-	switch addr := addr.(type) {
-	case *net.UDPAddr:
-		sourceIP = addr.IP
-	case *net.TCPAddr:
-		sourceIP = addr.IP
-	}
-	for _, rule := range lc.Policy.Networks {
-		for source, targets := range rule {
-			networkNum := strings.TrimPrefix(source, "network.")
-			nc := p.cfg.Network[networkNum]
-			if nc == nil {
-				continue
-			}
-
-			for _, ipNet := range nc.IPNets {
-				if ipNet.Contains(sourceIP) {
-					matchedPolicy = lc.Policy.Name
-					matchedNetwork = source
-					do(targets)
-					matched = true
-					return upstreams, matched
-				}
-			}
-		}
-	}
-
-	return upstreams, matched
-}
-
-func (p *prog) proxy(ctx context.Context, upstreams []string, failoverRcodes []int, msg *dns.Msg) *dns.Msg {
-	var staleAnswer *dns.Msg
-	serveStaleCache := p.cache != nil && p.cfg.Service.CacheServeStale
-	upstreamConfigs := p.upstreamConfigsFromUpstreamNumbers(upstreams)
-	if len(upstreamConfigs) == 0 {
-		upstreamConfigs = []*ctrld.UpstreamConfig{osUpstreamConfig}
-		upstreams = []string{"upstream.os"}
-	}
-	// Inverse query should not be cached: https://www.rfc-editor.org/rfc/rfc1035#section-7.4
-	if p.cache != nil && msg.Question[0].Qtype != dns.TypePTR {
-		for _, upstream := range upstreams {
-			cachedValue := p.cache.Get(dnscache.NewKey(msg, upstream))
-			if cachedValue == nil {
-				continue
-			}
-			answer := cachedValue.Msg.Copy()
-			answer.SetRcode(msg, answer.Rcode)
-			now := time.Now()
-			if cachedValue.Expire.After(now) {
-				ctrld.Log(ctx, proxyLog.Debug(), "hit cached response")
-				setCachedAnswerTTL(answer, now, cachedValue.Expire)
-				return answer
-			}
-			staleAnswer = answer
-		}
-	}
-	resolve := func(n int, upstreamConfig *ctrld.UpstreamConfig, msg *dns.Msg) *dns.Msg {
-		ctrld.Log(ctx, proxyLog.Debug(), "sending query to %s: %s", upstreams[n], upstreamConfig.Name)
-		dnsResolver, err := ctrld.NewResolver(upstreamConfig)
-		if err != nil {
-			ctrld.Log(ctx, proxyLog.Error().Err(err), "failed to create resolver")
-			return nil
-		}
-		resolveCtx, cancel := context.WithCancel(ctx)
-		defer cancel()
-		if upstreamConfig.Timeout > 0 {
-			timeoutCtx, cancel := context.WithTimeout(resolveCtx, time.Millisecond*time.Duration(upstreamConfig.Timeout))
-			defer cancel()
-			resolveCtx = timeoutCtx
-		}
-		answer, err := dnsResolver.Resolve(resolveCtx, msg)
-		if err != nil {
-			ctrld.Log(ctx, proxyLog.Error().Err(err), "failed to resolve query")
-			return nil
-		}
-		return answer
-	}
-	for n, upstreamConfig := range upstreamConfigs {
-		answer := resolve(n, upstreamConfig, msg)
-		if answer == nil {
-			if serveStaleCache && staleAnswer != nil {
-				ctrld.Log(ctx, proxyLog.Debug(), "serving stale cached response")
-				now := time.Now()
-				setCachedAnswerTTL(staleAnswer, now, now.Add(staleTTL))
-				return staleAnswer
-			}
-			continue
-		}
-		if answer.Rcode != dns.RcodeSuccess && len(upstreamConfigs) > 1 && containRcode(failoverRcodes, answer.Rcode) {
-			ctrld.Log(ctx, proxyLog.Debug(), "failover rcode matched, process to next upstream")
-			continue
-		}
-		if p.cache != nil {
-			ttl := ttlFromMsg(answer)
-			now := time.Now()
-			expired := now.Add(time.Duration(ttl) * time.Second)
-			if cachedTTL := p.cfg.Service.CacheTTLOverride; cachedTTL > 0 {
-				expired = now.Add(time.Duration(cachedTTL) * time.Second)
-			}
-			setCachedAnswerTTL(answer, now, expired)
-			p.cache.Add(dnscache.NewKey(msg, upstreams[n]), dnscache.NewValue(answer, expired))
-			ctrld.Log(ctx, proxyLog.Debug(), "add cached response")
-		}
-		return answer
-	}
-	ctrld.Log(ctx, proxyLog.Error(), "all upstreams failed")
-	answer := new(dns.Msg)
-	answer.SetRcode(msg, dns.RcodeServerFailure)
-	return answer
-}
-
-func (p *prog) upstreamConfigsFromUpstreamNumbers(upstreams []string) []*ctrld.UpstreamConfig {
-	upstreamConfigs := make([]*ctrld.UpstreamConfig, 0, len(upstreams))
-	for _, upstream := range upstreams {
-		upstreamNum := strings.TrimPrefix(upstream, "upstream.")
-		upstreamConfigs = append(upstreamConfigs, p.cfg.Upstream[upstreamNum])
-	}
-	return upstreamConfigs
-}
-
-// canonicalName returns canonical name from FQDN with "." trimmed.
-func canonicalName(fqdn string) string {
-	q := strings.TrimSpace(fqdn)
-	q = strings.TrimSuffix(q, ".")
-	// https://datatracker.ietf.org/doc/html/rfc4343
-	q = strings.ToLower(q)
-
-	return q
-}
-
-func wildcardMatches(wildcard, domain string) bool {
-	// Wildcard match.
-	wildCardParts := strings.Split(wildcard, "*")
-	if len(wildCardParts) != 2 {
-		return false
-	}
-
-	switch {
-	case len(wildCardParts[0]) > 0 && len(wildCardParts[1]) > 0:
-		// Domain must match both prefix and suffix.
-		return strings.HasPrefix(domain, wildCardParts[0]) && strings.HasSuffix(domain, wildCardParts[1])
-
-	case len(wildCardParts[1]) > 0:
-		// Only suffix must match.
-		return strings.HasSuffix(domain, wildCardParts[1])
-
-	case len(wildCardParts[0]) > 0:
-		// Only prefix must match.
-		return strings.HasPrefix(domain, wildCardParts[0])
-	}
-
-	return false
-}
-
-func fmtRemoteToLocal(listenerNum, remote, local string) string {
-	return fmt.Sprintf("%s -> listener.%s: %s:", remote, listenerNum, local)
-}
-
-func requestID() string {
-	b := make([]byte, 3) // 6 chars
-	if _, err := rand.Read(b); err != nil {
-		panic(err)
-	}
-	return hex.EncodeToString(b)
-}
-
-func containRcode(rcodes []int, rcode int) bool {
-	for i := range rcodes {
-		if rcodes[i] == rcode {
-			return true
-		}
-	}
-	return false
-}
-
-func setCachedAnswerTTL(answer *dns.Msg, now, expiredTime time.Time) {
-	ttlSecs := expiredTime.Sub(now).Seconds()
-	if ttlSecs < 0 {
-		return
-	}
-
-	ttl := uint32(ttlSecs)
-	for _, rr := range answer.Answer {
-		rr.Header().Ttl = ttl
-	}
-	for _, rr := range answer.Ns {
-		rr.Header().Ttl = ttl
-	}
-	for _, rr := range answer.Extra {
-		if rr.Header().Rrtype != dns.TypeOPT {
-			rr.Header().Ttl = ttl
-		}
-	}
-}
-
-func ttlFromMsg(msg *dns.Msg) uint32 {
-	for _, rr := range msg.Answer {
-		return rr.Header().Ttl
-	}
-	for _, rr := range msg.Ns {
-		return rr.Header().Ttl
-	}
-	return 0
-}
-
-var osUpstreamConfig = &ctrld.UpstreamConfig{
-	Name: "OS resolver",
-	Type: ctrld.ResolverTypeOS,
-}

cmd/ctrld/dns_proxy_test.go

@@ -1,154 +0,0 @@
-package main
-
-import (
-	"context"
-	"net"
-	"testing"
-	"time"
-
-	"github.com/miekg/dns"
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
-
-	"github.com/Control-D-Inc/ctrld"
-	"github.com/Control-D-Inc/ctrld/internal/dnscache"
-	"github.com/Control-D-Inc/ctrld/testhelper"
-)
-
-func Test_wildcardMatches(t *testing.T) {
-	tests := []struct {
-		name     string
-		wildcard string
-		domain   string
-		match    bool
-	}{
-		{"prefix parent should not match", "*.windscribe.com", "windscribe.com", false},
-		{"prefix", "*.windscribe.com", "anything.windscribe.com", true},
-		{"prefix not match other domain", "*.windscribe.com", "example.com", false},
-		{"prefix not match domain in name", "*.windscribe.com", "wwindscribe.com", false},
-		{"suffix", "suffix.*", "suffix.windscribe.com", true},
-		{"suffix not match other", "suffix.*", "suffix1.windscribe.com", false},
-		{"both", "suffix.*.windscribe.com", "suffix.anything.windscribe.com", true},
-		{"both not match", "suffix.*.windscribe.com", "suffix1.suffix.windscribe.com", false},
-	}
-
-	for _, tc := range tests {
-		tc := tc
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-			if got := wildcardMatches(tc.wildcard, tc.domain); got != tc.match {
-				t.Errorf("unexpected result, wildcard: %s, domain: %s, want: %v, got: %v", tc.wildcard, tc.domain, tc.match, got)
-			}
-		})
-	}
-}
-
-func Test_canonicalName(t *testing.T) {
-	tests := []struct {
-		name      string
-		domain    string
-		canonical string
-	}{
-		{"fqdn to canonical", "windscribe.com.", "windscribe.com"},
-		{"already canonical", "windscribe.com", "windscribe.com"},
-		{"case insensitive", "Windscribe.Com.", "windscribe.com"},
-	}
-
-	for _, tc := range tests {
-		tc := tc
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-			if got := canonicalName(tc.domain); got != tc.canonical {
-				t.Errorf("unexpected result, want: %s, got: %s", tc.canonical, got)
-			}
-		})
-	}
-}
-
-func Test_prog_upstreamFor(t *testing.T) {
-	cfg := testhelper.SampleConfig(t)
-	prog := &prog{cfg: cfg}
-	for _, nc := range prog.cfg.Network {
-		for _, cidr := range nc.Cidrs {
-			_, ipNet, err := net.ParseCIDR(cidr)
-			if err != nil {
-				t.Fatal(err)
-			}
-			nc.IPNets = append(nc.IPNets, ipNet)
-		}
-	}
-
-	tests := []struct {
-		name               string
-		ip                 string
-		defaultUpstreamNum string
-		lc                 *ctrld.ListenerConfig
-		domain             string
-		upstreams          []string
-		matched            bool
-	}{
-		{"Policy map matches", "192.168.0.1:0", "0", prog.cfg.Listener["0"], "abc.xyz", []string{"upstream.1", "upstream.0"}, true},
-		{"Policy split matches", "192.168.0.1:0", "0", prog.cfg.Listener["0"], "abc.ru", []string{"upstream.1"}, true},
-		{"Policy map for other network matches", "192.168.1.2:0", "0", prog.cfg.Listener["0"], "abc.xyz", []string{"upstream.0"}, true},
-		{"No policy map for listener", "192.168.1.2:0", "1", prog.cfg.Listener["1"], "abc.ru", []string{"upstream.1"}, false},
-	}
-
-	for _, tc := range tests {
-		tc := tc
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-			for _, network := range []string{"udp", "tcp"} {
-				var (
-					addr net.Addr
-					err  error
-				)
-				switch network {
-				case "udp":
-					addr, err = net.ResolveUDPAddr(network, tc.ip)
-				case "tcp":
-					addr, err = net.ResolveTCPAddr(network, tc.ip)
-				}
-				require.NoError(t, err)
-				require.NotNil(t, addr)
-				ctx := context.WithValue(context.Background(), ctrld.ReqIdCtxKey{}, requestID())
-				upstreams, matched := prog.upstreamFor(ctx, tc.defaultUpstreamNum, tc.lc, addr, tc.domain)
-				assert.Equal(t, tc.matched, matched)
-				assert.Equal(t, tc.upstreams, upstreams)
-			}
-		})
-	}
-}
-
-func TestCache(t *testing.T) {
-	cfg := testhelper.SampleConfig(t)
-	prog := &prog{cfg: cfg}
-	for _, nc := range prog.cfg.Network {
-		for _, cidr := range nc.Cidrs {
-			_, ipNet, err := net.ParseCIDR(cidr)
-			if err != nil {
-				t.Fatal(err)
-			}
-			nc.IPNets = append(nc.IPNets, ipNet)
-		}
-	}
-	cacher, err := dnscache.NewLRUCache(4096)
-	require.NoError(t, err)
-	prog.cache = cacher
-
-	msg := new(dns.Msg)
-	msg.SetQuestion("example.com", dns.TypeA)
-	msg.MsgHdr.RecursionDesired = true
-	answer1 := new(dns.Msg)
-	answer1.SetRcode(msg, dns.RcodeSuccess)
-
-	prog.cache.Add(dnscache.NewKey(msg, "upstream.1"), dnscache.NewValue(answer1, time.Now().Add(time.Minute)))
-	answer2 := new(dns.Msg)
-	answer2.SetRcode(msg, dns.RcodeRefused)
-	prog.cache.Add(dnscache.NewKey(msg, "upstream.0"), dnscache.NewValue(answer2, time.Now().Add(time.Minute)))
-
-	got1 := prog.proxy(context.Background(), []string{"upstream.1"}, nil, msg)
-	got2 := prog.proxy(context.Background(), []string{"upstream.0"}, nil, msg)
-	assert.NotSame(t, got1, got2)
-	assert.Equal(t, answer1.Rcode, got1.Rcode)
-	assert.Equal(t, answer2.Rcode, got2.Rcode)
-}

cmd/ctrld/main.go

@@ -1,113 +1,7 @@
 package main
 
-import (
-	"fmt"
-	"io"
-	"os"
-	"path/filepath"
-	"time"
-
-	"github.com/kardianos/service"
-	"github.com/rs/zerolog"
-
-	"github.com/Control-D-Inc/ctrld"
-)
-
-var (
-	configPath        string
-	configBase64      string
-	daemon            bool
-	listenAddress     string
-	primaryUpstream   string
-	secondaryUpstream string
-	domains           []string
-	logPath           string
-	homedir           string
-	cacheSize         int
-	cfg               ctrld.Config
-	verbose           int
-
-	bootstrapDNS = "76.76.2.0"
-
-	rootLogger = zerolog.New(io.Discard)
-	mainLog    = rootLogger
-	proxyLog   = rootLogger
-
-	cdUID          string
-	iface          string
-	ifaceStartStop string
-)
+import "github.com/Control-D-Inc/ctrld/cmd/cli"
 
 func main() {
-	ctrld.InitConfig(v, "ctrld")
-	initCLI()
-}
-
-func normalizeLogFilePath(logFilePath string) string {
-	if logFilePath == "" || filepath.IsAbs(logFilePath) || service.Interactive() {
-		return logFilePath
-	}
-	if homedir != "" {
-		return filepath.Join(homedir, logFilePath)
-	}
-	dir, _ := os.UserHomeDir()
-	if dir == "" {
-		return logFilePath
-	}
-	return filepath.Join(dir, logFilePath)
-}
-
-func initLogging() {
-	writers := []io.Writer{io.Discard}
-	isLog := cfg.Service.LogLevel != ""
-	if logFilePath := normalizeLogFilePath(cfg.Service.LogPath); logFilePath != "" {
-		// Create parent directory if necessary.
-		if err := os.MkdirAll(filepath.Dir(logFilePath), 0750); err != nil {
-			fmt.Fprintf(os.Stderr, "failed to create log path: %v", err)
-			os.Exit(1)
-		}
-		logFile, err := os.OpenFile(logFilePath, os.O_APPEND|os.O_CREATE|os.O_RDWR, os.FileMode(0o600))
-		if err != nil {
-			fmt.Fprintf(os.Stderr, "failed to create log file: %v", err)
-			os.Exit(1)
-		}
-		isLog = true
-		writers = append(writers, logFile)
-	}
-	zerolog.TimeFieldFormat = zerolog.TimeFormatUnixMs
-	consoleWriter := zerolog.NewConsoleWriter(func(w *zerolog.ConsoleWriter) {
-		w.TimeFormat = time.StampMilli
-	})
-	writers = append(writers, consoleWriter)
-	multi := zerolog.MultiLevelWriter(writers...)
-	mainLog = mainLog.Output(multi).With().Timestamp().Str("prefix", "main").Logger()
-	if verbose > 0 || isLog {
-		proxyLog = proxyLog.Output(multi).With().Timestamp().Logger()
-		// TODO: find a better way.
-		ctrld.ProxyLog = proxyLog
-	}
-
-	zerolog.SetGlobalLevel(zerolog.InfoLevel)
-	logLevel := cfg.Service.LogLevel
-	if verbose > 1 {
-		logLevel = "debug"
-	}
-	if logLevel == "" {
-		return
-	}
-	level, err := zerolog.ParseLevel(logLevel)
-	if err != nil {
-		mainLog.Warn().Err(err).Msg("could not set log level")
-		return
-	}
-	zerolog.SetGlobalLevel(level)
-}
-
-func initCache() {
-	if !cfg.Service.CacheEnable {
-		return
-	}
-	if cfg.Service.CacheSize == 0 {
-		cfg.Service.CacheSize = 4096
-	}
+	cli.Main()
 }

cmd/ctrld/net.go

@@ -1,65 +0,0 @@
-package main
-
-import (
-	"context"
-	"net"
-	"sync"
-	"time"
-
-	"tailscale.com/logtail/backoff"
-
-	"github.com/Control-D-Inc/ctrld/internal/controld"
-)
-
-const (
-	controldIPv6Test = "ipv6.controld.io"
-)
-
-var (
-	stackOnce          sync.Once
-	ipv6Enabled        bool
-	canListenIPv6Local bool
-	hasNetworkUp       bool
-)
-
-func probeStack() {
-	b := backoff.NewBackoff("probeStack", func(format string, args ...any) {}, time.Minute)
-	for {
-		if _, err := controld.Dialer.Dial("udp", net.JoinHostPort(bootstrapDNS, "53")); err == nil {
-			hasNetworkUp = true
-			break
-		} else {
-			b.BackOff(context.Background(), err)
-		}
-	}
-	if _, err := controld.Dialer.Dial("tcp6", net.JoinHostPort(controldIPv6Test, "80")); err == nil {
-		ipv6Enabled = true
-	}
-	if ln, err := net.Listen("tcp6", "[::1]:53"); err == nil {
-		ln.Close()
-		canListenIPv6Local = true
-	}
-}
-
-func netUp() bool {
-	stackOnce.Do(probeStack)
-	return hasNetworkUp
-}
-
-func supportsIPv6() bool {
-	stackOnce.Do(probeStack)
-	return ipv6Enabled
-}
-
-func supportsIPv6ListenLocal() bool {
-	stackOnce.Do(probeStack)
-	return canListenIPv6Local
-}
-
-// isIPv6 checks if the provided IP is v6.
-//
-//lint:ignore U1000 use in os_windows.go
-func isIPv6(ip string) bool {
-	parsedIP := net.ParseIP(ip)
-	return parsedIP != nil && parsedIP.To4() == nil && parsedIP.To16() != nil
-}

cmd/ctrld/net_darwin.go

@@ -1,34 +0,0 @@
-package main
-
-import (
-	"bufio"
-	"bytes"
-	"net"
-	"os/exec"
-	"strings"
-)
-
-func patchNetIfaceName(iface *net.Interface) error {
-	b, err := exec.Command("networksetup", "-listnetworkserviceorder").Output()
-	if err != nil {
-		return err
-	}
-
-	scanner := bufio.NewScanner(bytes.NewReader(b))
-	for scanner.Scan() {
-		line := scanner.Text()
-		if strings.Contains(line, "*") {
-			// Network services is disabled.
-			continue
-		}
-		if !strings.Contains(line, "Device: "+iface.Name) {
-			continue
-		}
-		parts := strings.Split(line, ",")
-		if _, networkServiceName, ok := strings.Cut(parts[0], "(Hardware Port: "); ok {
-			mainLog.Debug().Str("network_service", networkServiceName).Msg("found network service name for interface")
-			iface.Name = networkServiceName
-		}
-	}
-	return nil
-}

cmd/ctrld/net_others.go

@@ -1,7 +0,0 @@
-//go:build !darwin
-
-package main
-
-import "net"
-
-func patchNetIfaceName(iface *net.Interface) error { return nil }

cmd/ctrld/os_linux.go

@@ -1,194 +0,0 @@
-package main
-
-import (
-	"bufio"
-	"bytes"
-	"context"
-	"fmt"
-	"net"
-	"net/netip"
-	"os/exec"
-	"reflect"
-	"strings"
-	"syscall"
-	"time"
-
-	"github.com/insomniacslk/dhcp/dhcpv4/nclient4"
-	"github.com/insomniacslk/dhcp/dhcpv6"
-	"github.com/insomniacslk/dhcp/dhcpv6/client6"
-	"tailscale.com/util/dnsname"
-
-	"github.com/Control-D-Inc/ctrld/internal/dns"
-	"github.com/Control-D-Inc/ctrld/internal/resolvconffile"
-)
-
-var logf = func(format string, args ...any) {
-	mainLog.Debug().Msgf(format, args...)
-}
-
-// allocate loopback ip
-// sudo ip a add 127.0.0.2/24 dev lo
-func allocateIP(ip string) error {
-	cmd := exec.Command("ip", "a", "add", ip+"/24", "dev", "lo")
-	if err := cmd.Run(); err != nil {
-		mainLog.Error().Err(err).Msg("allocateIP failed")
-		return err
-	}
-	return nil
-}
-
-func deAllocateIP(ip string) error {
-	cmd := exec.Command("ip", "a", "del", ip+"/24", "dev", "lo")
-	if err := cmd.Run(); err != nil {
-		mainLog.Error().Err(err).Msg("deAllocateIP failed")
-		return err
-	}
-	return nil
-}
-
-const maxSetDNSAttempts = 5
-
-// set the dns server for the provided network interface
-func setDNS(iface *net.Interface, nameservers []string) error {
-	r, err := dns.NewOSConfigurator(logf, iface.Name)
-	if err != nil {
-		mainLog.Error().Err(err).Msg("failed to create DNS OS configurator")
-		return err
-	}
-
-	ns := make([]netip.Addr, 0, len(nameservers))
-	for _, nameserver := range nameservers {
-		ns = append(ns, netip.MustParseAddr(nameserver))
-	}
-
-	osConfig := dns.OSConfig{
-		Nameservers:   ns,
-		SearchDomains: []dnsname.FQDN{},
-	}
-
-	for i := 0; i < maxSetDNSAttempts; i++ {
-		if err := r.SetDNS(osConfig); err != nil {
-			return err
-		}
-		currentNS := currentDNS(iface)
-		if reflect.DeepEqual(currentNS, nameservers) {
-			return nil
-		}
-	}
-	mainLog.Debug().Msg("DNS was not set for some reason")
-	return nil
-}
-
-func resetDNS(iface *net.Interface) error {
-	if r, err := dns.NewOSConfigurator(logf, iface.Name); err == nil {
-		if err := r.Close(); err != nil {
-			mainLog.Error().Err(err).Msg("failed to rollback DNS setting")
-			return err
-		}
-		if r.Mode() == "direct" {
-			return nil
-		}
-	}
-
-	var ns []string
-	c, err := nclient4.New(iface.Name)
-	if err != nil {
-		return fmt.Errorf("nclient4.New: %w", err)
-	}
-	defer c.Close()
-
-	ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)
-	defer cancel()
-	lease, err := c.Request(ctx)
-	if err != nil {
-		return fmt.Errorf("nclient4.Request: %w", err)
-	}
-	for _, nameserver := range lease.ACK.DNS() {
-		if nameserver.Equal(net.IPv4zero) {
-			continue
-		}
-		ns = append(ns, nameserver.String())
-	}
-
-	// TODO(cuonglm): handle DHCPv6 properly.
-	if supportsIPv6() {
-		c := client6.NewClient()
-		conversation, err := c.Exchange(iface.Name)
-		if err != nil {
-			mainLog.Debug().Err(err).Msg("could not exchange DHCPv6")
-		}
-		for _, packet := range conversation {
-			if packet.Type() == dhcpv6.MessageTypeReply {
-				msg, err := packet.GetInnerMessage()
-				if err != nil {
-					mainLog.Debug().Err(err).Msg("could not get inner DHCPv6 message")
-					return nil
-				}
-				nameservers := msg.Options.DNS()
-				for _, nameserver := range nameservers {
-					ns = append(ns, nameserver.String())
-				}
-			}
-		}
-	}
-
-	return ignoringEINTR(func() error {
-		return setDNS(iface, ns)
-	})
-}
-
-func currentDNS(iface *net.Interface) []string {
-	for _, fn := range []getDNS{getDNSByResolvectl, getDNSByNmcli, resolvconffile.NameServers} {
-		if ns := fn(iface.Name); len(ns) > 0 {
-			return ns
-		}
-	}
-	return nil
-}
-
-func getDNSByResolvectl(iface string) []string {
-	b, err := exec.Command("resolvectl", "dns", "-i", iface).Output()
-	if err != nil {
-		return nil
-	}
-
-	parts := strings.Fields(strings.SplitN(string(b), "%", 2)[0])
-	if len(parts) > 2 {
-		return parts[3:]
-	}
-	return nil
-}
-
-func getDNSByNmcli(iface string) []string {
-	b, err := exec.Command("nmcli", "dev", "show", iface).Output()
-	if err != nil {
-		return nil
-	}
-	s := bufio.NewScanner(bytes.NewReader(b))
-	var dns []string
-	do := func(line string) {
-		parts := strings.SplitN(line, ":", 2)
-		if len(parts) > 1 {
-			dns = append(dns, strings.TrimSpace(parts[1]))
-		}
-	}
-	for s.Scan() {
-		line := s.Text()
-		switch {
-		case strings.HasPrefix(line, "IP4.DNS"):
-			fallthrough
-		case strings.HasPrefix(line, "IP6.DNS"):
-			do(line)
-		}
-	}
-	return dns
-}
-
-func ignoringEINTR(fn func() error) error {
-	for {
-		err := fn()
-		if err != syscall.EINTR {
-			return err
-		}
-	}
-}

cmd/ctrld/os_mac.go

@@ -1,62 +0,0 @@
-//go:build darwin
-// +build darwin
-
-package main
-
-import (
-	"net"
-	"os/exec"
-
-	"github.com/Control-D-Inc/ctrld/internal/resolvconffile"
-)
-
-// allocate loopback ip
-// sudo ifconfig lo0 alias 127.0.0.2 up
-func allocateIP(ip string) error {
-	cmd := exec.Command("ifconfig", "lo0", "alias", ip, "up")
-	if err := cmd.Run(); err != nil {
-		mainLog.Error().Err(err).Msg("allocateIP failed")
-		return err
-	}
-	return nil
-}
-
-func deAllocateIP(ip string) error {
-	cmd := exec.Command("ifconfig", "lo0", "-alias", ip)
-	if err := cmd.Run(); err != nil {
-		mainLog.Error().Err(err).Msg("deAllocateIP failed")
-		return err
-	}
-	return nil
-}
-
-// set the dns server for the provided network interface
-// networksetup -setdnsservers Wi-Fi 8.8.8.8 1.1.1.1
-// TODO(cuonglm): use system API
-func setDNS(iface *net.Interface, nameservers []string) error {
-	cmd := "networksetup"
-	args := []string{"-setdnsservers", iface.Name}
-	args = append(args, nameservers...)
-
-	if err := exec.Command(cmd, args...).Run(); err != nil {
-		mainLog.Error().Err(err).Msgf("setDNS failed, ips = %q", nameservers)
-		return err
-	}
-	return nil
-}
-
-// TODO(cuonglm): use system API
-func resetDNS(iface *net.Interface) error {
-	cmd := "networksetup"
-	args := []string{"-setdnsservers", iface.Name, "empty"}
-
-	if err := exec.Command(cmd, args...).Run(); err != nil {
-		mainLog.Error().Err(err).Msgf("resetDNS failed")
-		return err
-	}
-	return nil
-}
-
-func currentDNS(_ *net.Interface) []string {
-	return resolvconffile.NameServers("")
-}

cmd/ctrld/os_windows.go

@@ -1,106 +0,0 @@
-//go:build windows
-// +build windows
-
-package main
-
-import (
-	"errors"
-	"net"
-	"os/exec"
-	"strconv"
-
-	"golang.zx2c4.com/wireguard/windows/tunnel/winipcfg"
-)
-
-// TODO(cuonglm): implement.
-func allocateIP(ip string) error {
-	return nil
-}
-
-// TODO(cuonglm): implement.
-func deAllocateIP(ip string) error {
-	return nil
-}
-
-func setDNS(iface *net.Interface, nameservers []string) error {
-	if len(nameservers) == 0 {
-		return errors.New("empty DNS nameservers")
-	}
-	primaryDNS := nameservers[0]
-	if err := setPrimaryDNS(iface, primaryDNS); err != nil {
-		return err
-	}
-	if len(nameservers) > 1 {
-		secondaryDNS := nameservers[1]
-		_ = addSecondaryDNS(iface, secondaryDNS)
-	}
-	return nil
-}
-
-// TODO(cuonglm): should we use system API?
-func resetDNS(iface *net.Interface) error {
-	if supportsIPv6ListenLocal() {
-		if output, err := netsh("interface", "ipv6", "set", "dnsserver", strconv.Itoa(iface.Index), "dhcp"); err != nil {
-			mainLog.Warn().Err(err).Msgf("failed to reset ipv6 DNS: %s", string(output))
-		}
-	}
-	output, err := netsh("interface", "ipv4", "set", "dnsserver", strconv.Itoa(iface.Index), "dhcp")
-	if err != nil {
-		mainLog.Error().Err(err).Msgf("failed to reset ipv4 DNS: %s", string(output))
-		return err
-	}
-	return nil
-}
-
-func setPrimaryDNS(iface *net.Interface, dns string) error {
-	ipVer := "ipv4"
-	if isIPv6(dns) {
-		ipVer = "ipv6"
-	}
-	idx := strconv.Itoa(iface.Index)
-	output, err := netsh("interface", ipVer, "set", "dnsserver", idx, "static", dns)
-	if err != nil {
-		mainLog.Error().Err(err).Msgf("failed to set primary DNS: %s", string(output))
-		return err
-	}
-	if ipVer == "ipv4" {
-		// Disable IPv6 DNS, so the query will be fallback to IPv4.
-		_, _ = netsh("interface", "ipv6", "set", "dnsserver", idx, "static", "::1", "primary")
-	}
-
-	return nil
-}
-
-func addSecondaryDNS(iface *net.Interface, dns string) error {
-	ipVer := "ipv4"
-	if isIPv6(dns) {
-		ipVer = "ipv6"
-	}
-	output, err := netsh("interface", ipVer, "add", "dns", strconv.Itoa(iface.Index), dns, "index=2")
-	if err != nil {
-		mainLog.Warn().Err(err).Msgf("failed to add secondary DNS: %s", string(output))
-	}
-	return nil
-}
-
-func netsh(args ...string) ([]byte, error) {
-	return exec.Command("netsh", args...).Output()
-}
-
-func currentDNS(iface *net.Interface) []string {
-	luid, err := winipcfg.LUIDFromIndex(uint32(iface.Index))
-	if err != nil {
-		mainLog.Error().Err(err).Msg("failed to get interface LUID")
-		return nil
-	}
-	nameservers, err := luid.DNS()
-	if err != nil {
-		mainLog.Error().Err(err).Msg("failed to get interface DNS")
-		return nil
-	}
-	ns := make([]string, 0, len(nameservers))
-	for _, nameserver := range nameservers {
-		ns = append(ns, nameserver.String())
-	}
-	return ns
-}

cmd/ctrld/prog.go

@@ -1,251 +0,0 @@
-package main
-
-import (
-	"errors"
-	"net"
-	"os"
-	"strconv"
-	"sync"
-	"syscall"
-
-	"github.com/kardianos/service"
-	"github.com/miekg/dns"
-
-	"github.com/Control-D-Inc/ctrld"
-	"github.com/Control-D-Inc/ctrld/internal/dnscache"
-)
-
-var errWindowsAddrInUse = syscall.Errno(0x2740)
-
-var svcConfig = &service.Config{
-	Name:        "ctrld",
-	DisplayName: "Control-D Helper Service",
-}
-
-type prog struct {
-	cfg   *ctrld.Config
-	cache dnscache.Cacher
-}
-
-func (p *prog) Start(s service.Service) error {
-	p.cfg = &cfg
-	go p.run()
-	mainLog.Info().Msg("Service started")
-	return nil
-}
-
-func (p *prog) run() {
-	p.preRun()
-	if p.cfg.Service.CacheEnable {
-		cacher, err := dnscache.NewLRUCache(p.cfg.Service.CacheSize)
-		if err != nil {
-			mainLog.Error().Err(err).Msg("failed to create cacher, caching is disabled")
-		} else {
-			p.cache = cacher
-		}
-	}
-	var wg sync.WaitGroup
-	wg.Add(len(p.cfg.Listener))
-
-	for _, nc := range p.cfg.Network {
-		for _, cidr := range nc.Cidrs {
-			_, ipNet, err := net.ParseCIDR(cidr)
-			if err != nil {
-				proxyLog.Error().Err(err).Str("network", nc.Name).Str("cidr", cidr).Msg("invalid cidr")
-				continue
-			}
-			nc.IPNets = append(nc.IPNets, ipNet)
-		}
-	}
-	for n := range p.cfg.Upstream {
-		uc := p.cfg.Upstream[n]
-		uc.Init()
-		if uc.BootstrapIP == "" {
-			// resolve it manually and set the bootstrap ip
-			c := new(dns.Client)
-			for _, dnsType := range []uint16{dns.TypeAAAA, dns.TypeA} {
-				if !supportsIPv6() && dnsType == dns.TypeAAAA {
-					continue
-				}
-				m := new(dns.Msg)
-				m.SetQuestion(uc.Domain+".", dnsType)
-				m.RecursionDesired = true
-				r, _, err := c.Exchange(m, net.JoinHostPort(bootstrapDNS, "53"))
-				if err != nil {
-					proxyLog.Error().Err(err).Msgf("could not resolve domain %s for upstream.%s", uc.Domain, n)
-					continue
-				}
-				if r.Rcode != dns.RcodeSuccess {
-					proxyLog.Error().Msgf("could not resolve domain return code: %d, upstream.%s", r.Rcode, n)
-					continue
-				}
-				if len(r.Answer) == 0 {
-					continue
-				}
-				for _, a := range r.Answer {
-					switch ar := a.(type) {
-					case *dns.A:
-						uc.BootstrapIP = ar.A.String()
-					case *dns.AAAA:
-						uc.BootstrapIP = ar.AAAA.String()
-					default:
-						continue
-					}
-					mainLog.Info().Str("bootstrap_ip", uc.BootstrapIP).Msgf("Setting bootstrap IP for upstream.%s", n)
-					// Stop if we reached here, because we got the bootstrap IP from r.Answer.
-					break
-				}
-				// If we reached here, uc.BootstrapIP was set, nothing to do anymore.
-				break
-			}
-		}
-		uc.SetupTransport()
-	}
-
-	for listenerNum := range p.cfg.Listener {
-		p.cfg.Listener[listenerNum].Init()
-		go func(listenerNum string) {
-			defer wg.Done()
-			listenerConfig := p.cfg.Listener[listenerNum]
-			upstreamConfig := p.cfg.Upstream[listenerNum]
-			if upstreamConfig == nil {
-				proxyLog.Error().Msgf("missing upstream config for: [listener.%s]", listenerNum)
-				return
-			}
-			addr := net.JoinHostPort(listenerConfig.IP, strconv.Itoa(listenerConfig.Port))
-			mainLog.Info().Msgf("Starting DNS server on listener.%s: %s", listenerNum, addr)
-			err := p.serveUDP(listenerNum)
-			if err != nil && !defaultConfigWritten {
-				proxyLog.Fatal().Err(err).Msgf("Unable to start dns proxy on listener.%s", listenerNum)
-				return
-			}
-			if err == nil {
-				return
-			}
-
-			if opErr, ok := err.(*net.OpError); ok {
-				if sErr, ok := opErr.Err.(*os.SyscallError); ok && errors.Is(opErr.Err, syscall.EADDRINUSE) || errors.Is(sErr.Err, errWindowsAddrInUse) {
-					proxyLog.Warn().Msgf("Address %s already in used, pick a random one", addr)
-					pc, err := net.ListenPacket("udp", net.JoinHostPort(listenerConfig.IP, "0"))
-					if err != nil {
-						proxyLog.Fatal().Err(err).Msg("failed to listen packet")
-						return
-					}
-					_, portStr, _ := net.SplitHostPort(pc.LocalAddr().String())
-					port, err := strconv.Atoi(portStr)
-					if err != nil {
-						proxyLog.Fatal().Err(err).Msg("malformed port")
-						return
-					}
-					listenerConfig.Port = port
-					v.Set("listener", map[string]*ctrld.ListenerConfig{
-						"0": {
-							IP:   "127.0.0.1",
-							Port: port,
-						},
-					})
-					if err := writeConfigFile(); err != nil {
-						proxyLog.Fatal().Err(err).Msg("failed to write config file")
-					} else {
-						mainLog.Info().Msg("writing config file to: " + defaultConfigFile)
-					}
-					mainLog.Info().Msgf("Starting DNS server on listener.%s: %s", listenerNum, pc.LocalAddr())
-					// There can be a race between closing the listener and start our own UDP server, but it's
-					// rare, and we only do this once, so let conservative here.
-					if err := pc.Close(); err != nil {
-						proxyLog.Fatal().Err(err).Msg("failed to close packet conn")
-						return
-					}
-					if err := p.serveUDP(listenerNum); err != nil {
-						proxyLog.Fatal().Err(err).Msgf("Unable to start dns proxy on listener.%s", listenerNum)
-						return
-					}
-				}
-			}
-			proxyLog.Fatal().Err(err).Msgf("Unable to start dns proxy on listener.%s", listenerNum)
-		}(listenerNum)
-	}
-
-	wg.Wait()
-}
-
-func (p *prog) Stop(s service.Service) error {
-	if err := p.deAllocateIP(); err != nil {
-		mainLog.Error().Err(err).Msg("de-allocate ip failed")
-		return err
-	}
-	mainLog.Info().Msg("Service stopped")
-	return nil
-}
-
-func (p *prog) allocateIP(ip string) error {
-	if !p.cfg.Service.AllocateIP {
-		return nil
-	}
-	return allocateIP(ip)
-}
-
-func (p *prog) deAllocateIP() error {
-	if !p.cfg.Service.AllocateIP {
-		return nil
-	}
-	for _, lc := range p.cfg.Listener {
-		if err := deAllocateIP(lc.IP); err != nil {
-			return err
-		}
-	}
-	return nil
-}
-
-func (p *prog) setDNS() {
-	if cfg.Listener == nil || cfg.Listener["0"] == nil {
-		return
-	}
-	if iface == "" {
-		return
-	}
-	if iface == "auto" {
-		iface = defaultIfaceName()
-	}
-	logger := mainLog.With().Str("iface", iface).Logger()
-	netIface, err := netInterface(iface)
-	if err != nil {
-		logger.Error().Err(err).Msg("could not get interface")
-		return
-	}
-	if err := setupNetworkManager(); err != nil {
-		logger.Error().Err(err).Msg("could not patch NetworkManager")
-		return
-	}
-	logger.Debug().Msg("setting DNS for interface")
-	if err := setDNS(netIface, []string{cfg.Listener["0"].IP}); err != nil {
-		logger.Error().Err(err).Msgf("could not set DNS for interface")
-		return
-	}
-	logger.Debug().Msg("setting DNS successfully")
-}
-
-func (p *prog) resetDNS() {
-	if iface == "" {
-		return
-	}
-	if iface == "auto" {
-		iface = defaultIfaceName()
-	}
-	logger := mainLog.With().Str("iface", iface).Logger()
-	netIface, err := netInterface(iface)
-	if err != nil {
-		logger.Error().Err(err).Msg("could not get interface")
-		return
-	}
-	if err := restoreNetworkManager(); err != nil {
-		logger.Error().Err(err).Msg("could not restore NetworkManager")
-		return
-	}
-	logger.Debug().Msg("Restoring DNS for interface")
-	if err := resetDNS(netIface); err != nil {
-		logger.Error().Err(err).Msgf("could not reset DNS")
-		return
-	}
-	logger.Debug().Msg("Restoring DNS successfully")
-}

cmd/ctrld/prog_linux.go

@@ -1,20 +0,0 @@
-package main
-
-import (
-	"github.com/kardianos/service"
-)
-
-func (p *prog) preRun() {
-	if !service.Interactive() {
-		p.setDNS()
-	}
-}
-
-func setDependencies(svc *service.Config) {
-	svc.Dependencies = []string{
-		"Wants=network-online.target",
-		"After=network-online.target",
-		"Wants=NetworkManager-wait-online.service",
-		"After=NetworkManager-wait-online.service",
-	}
-}

cmd/ctrld/prog_others.go

@@ -1,10 +0,0 @@
-//go:build !linux
-// +build !linux
-
-package main
-
-import "github.com/kardianos/service"
-
-func (p *prog) preRun() {}
-
-func setDependencies(svc *service.Config) {}

cmd/ctrld/service.go

@@ -1,45 +0,0 @@
-package main
-
-import (
-	"fmt"
-	"os"
-
-	"github.com/spf13/cobra"
-)
-
-func stderrMsg(msg string) {
-	_, _ = fmt.Fprintln(os.Stderr, msg)
-}
-
-func stdoutMsg(msg string) {
-	_, _ = fmt.Fprintln(os.Stdout, msg)
-}
-
-type task struct {
-	f            func() error
-	abortOnError bool
-}
-
-func doTasks(tasks []task) bool {
-	for _, task := range tasks {
-		if err := task.f(); err != nil {
-			if task.abortOnError {
-				stderrMsg(err.Error())
-				return false
-			}
-		}
-	}
-	return true
-}
-
-func checkHasElevatedPrivilege(cmd *cobra.Command, args []string) {
-	ok, err := hasElevatedPrivilege()
-	if err != nil {
-		fmt.Printf("could not detect user privilege: %v", err)
-		return
-	}
-	if !ok {
-		fmt.Println("Please relaunch process with admin/root privilege.")
-		os.Exit(1)
-	}
-}

cmd/ctrld/service_others.go

@@ -1,11 +0,0 @@
-//go:build !windows
-
-package main
-
-import (
-	"os"
-)
-
-func hasElevatedPrivilege() (bool, error) {
-	return os.Geteuid() == 0, nil
-}

cmd/ctrld/service_windows.go

@@ -1,24 +0,0 @@
-package main
-
-import "golang.org/x/sys/windows"
-
-func hasElevatedPrivilege() (bool, error) {
-	var sid *windows.SID
-	if err := windows.AllocateAndInitializeSid(
-		&windows.SECURITY_NT_AUTHORITY,
-		2,
-		windows.SECURITY_BUILTIN_DOMAIN_RID,
-		windows.DOMAIN_ALIAS_RID_ADMINS,
-		0,
-		0,
-		0,
-		0,
-		0,
-		0,
-		&sid,
-	); err != nil {
-		return false, err
-	}
-	token := windows.Token(0)
-	return token.IsMember(sid)
-}

cmd/ctrld_library/main.go

@@ -0,0 +1,80 @@
+package ctrld_library
+
+import (
+	"github.com/Control-D-Inc/ctrld/cmd/cli"
+)
+
+// Controller holds global state
+type Controller struct {
+	stopCh      chan struct{}
+	AppCallback AppCallback
+	Config      cli.AppConfig
+}
+
+// NewController provides reference to global state to be managed by android vpn service and iOS network extension.
+// reference is not safe for concurrent use.
+func NewController(appCallback AppCallback) *Controller {
+	return &Controller{AppCallback: appCallback}
+}
+
+// AppCallback provides access to app instance.
+type AppCallback interface {
+	Hostname() string
+	LanIp() string
+	MacAddress() string
+	Exit(error string)
+}
+
+// Start configures utility with config.toml from provided directory.
+// This function will block until Stop is called
+// Check port availability prior to calling it.
+func (c *Controller) Start(CdUID string, HomeDir string, UpstreamProto string, logLevel int, logPath string) {
+	if c.stopCh == nil {
+		c.stopCh = make(chan struct{})
+		c.Config = cli.AppConfig{
+			CdUID:         CdUID,
+			HomeDir:       HomeDir,
+			UpstreamProto: UpstreamProto,
+			Verbose:       logLevel,
+			LogPath:       logPath,
+		}
+		appCallback := mapCallback(c.AppCallback)
+		cli.RunMobile(&c.Config, &appCallback, c.stopCh)
+	}
+}
+
+// As workaround to avoid circular dependency between cli and ctrld_library module
+func mapCallback(callback AppCallback) cli.AppCallback {
+	return cli.AppCallback{
+		HostName: func() string {
+			return callback.Hostname()
+		},
+		LanIp: func() string {
+			return callback.LanIp()
+		},
+		MacAddress: func() string {
+			return callback.MacAddress()
+		},
+		Exit: func(err string) {
+			callback.Exit(err)
+		},
+	}
+}
+
+func (c *Controller) Stop(restart bool, pin int64) int {
+	var errorCode = 0
+	// Force disconnect without checking pin.
+	// In iOS restart is required if vpn detects no connectivity after network change.
+	if !restart {
+		errorCode = cli.CheckDeactivationPin(pin, c.stopCh)
+	}
+	if errorCode == 0 && c.stopCh != nil {
+		close(c.stopCh)
+		c.stopCh = nil
+	}
+	return errorCode
+}
+
+func (c *Controller) IsRunning() bool {
+	return c.stopCh != nil
+}

config.go

@@ -2,44 +2,115 @@ package ctrld
 
 import (
 	"context"
+	crand "crypto/rand"
 	"crypto/tls"
+	"crypto/x509"
+	"encoding/hex"
+	"errors"
+	"fmt"
+	"io"
+	"math/rand"
 	"net"
 	"net/http"
+	"net/netip"
 	"net/url"
 	"os"
+	"runtime"
+	"sort"
+	"strconv"
 	"strings"
+	"sync"
+	"sync/atomic"
 	"time"
 
-	"github.com/Control-D-Inc/ctrld/internal/dnsrcode"
+	"github.com/ameshkov/dnsstamps"
 	"github.com/go-playground/validator/v10"
-	"github.com/lucas-clemente/quic-go"
-	"github.com/lucas-clemente/quic-go/http3"
 	"github.com/miekg/dns"
 	"github.com/spf13/viper"
+	"golang.org/x/net/http2"
+	"golang.org/x/sync/singleflight"
+	"tailscale.com/logtail/backoff"
+	"tailscale.com/net/tsaddr"
+
+	"github.com/Control-D-Inc/ctrld/internal/dnsrcode"
+	ctrldnet "github.com/Control-D-Inc/ctrld/internal/net"
+)
+
+// IpStackBoth ...
+const (
+	// IpStackBoth indicates that ctrld will use either ipv4 or ipv6 for connecting to upstream,
+	// depending on which stack is available when receiving the DNS query.
+	IpStackBoth = "both"
+	// IpStackV4 indicates that ctrld will use only ipv4 for connecting to upstream.
+	IpStackV4 = "v4"
+	// IpStackV6 indicates that ctrld will use only ipv6 for connecting to upstream.
+	IpStackV6 = "v6"
+	// IpStackSplit indicates that ctrld will use either ipv4 or ipv6 for connecting to upstream,
+	// depending on the record type of the DNS query.
+	IpStackSplit = "split"
+
+	// FreeDnsDomain is the domain name of free ControlD service.
+	FreeDnsDomain = "freedns.controld.com"
+	// FreeDNSBoostrapIP is the IP address of freedns.controld.com.
+	FreeDNSBoostrapIP = "76.76.2.11"
+	// PremiumDnsDomain is the domain name of premium ControlD service.
+	PremiumDnsDomain = "dns.controld.com"
+	// PremiumDNSBoostrapIP is the IP address of dns.controld.com.
+	PremiumDNSBoostrapIP = "76.76.2.22"
+
+	controlDComDomain = "controld.com"
+	controlDNetDomain = "controld.net"
+	controlDDevDomain = "controld.dev"
+
+	endpointPrefixHTTPS = "https://"
+	endpointPrefixQUIC  = "quic://"
+	endpointPrefixH3    = "h3://"
+	endpointPrefixSdns  = "sdns://"
+)
+
+var (
+	controldParentDomains  = []string{controlDComDomain, controlDNetDomain, controlDDevDomain}
+	controldVerifiedDomain = map[string]string{
+		controlDComDomain: "verify.controld.com",
+		controlDDevDomain: "verify.controld.dev",
+	}
 )
 
 // SetConfigName set the config name that ctrld will look for.
+// DEPRECATED: use SetConfigNameWithPath instead.
 func SetConfigName(v *viper.Viper, name string) {
-	v.SetConfigName(name)
-
 	configPath := "$HOME"
 	// viper has its own way to get user home directory:  https://github.com/spf13/viper/blob/v1.14.0/util.go#L134
 	// To be consistent, we prefer os.UserHomeDir instead.
 	if homeDir, err := os.UserHomeDir(); err == nil {
 		configPath = homeDir
 	}
+	SetConfigNameWithPath(v, name, configPath)
+}
+
+// SetConfigNameWithPath set the config path and name that ctrld will look for.
+func SetConfigNameWithPath(v *viper.Viper, name, configPath string) {
+	v.SetConfigName(name)
 	v.AddConfigPath(configPath)
 	v.AddConfigPath(".")
 }
 
 // InitConfig initializes default config values for given *viper.Viper instance.
 func InitConfig(v *viper.Viper, name string) {
-	SetConfigName(v, name)
-
 	v.SetDefault("listener", map[string]*ListenerConfig{
 		"0": {
-			IP:   "127.0.0.1",
-			Port: 53,
+			IP:   "",
+			Port: 0,
+			Policy: &ListenerPolicyConfig{
+				Name: "Main Policy",
+				Networks: []Rule{
+					{"network.0": []string{"upstream.0"}},
+				},
+				Rules: []Rule{
+					{"example.com": []string{"upstream.0"}},
+					{"*.ads.com": []string{"upstream.1"}},
+				},
+			},
 		},
 	})
 	v.SetDefault("network", map[string]*NetworkConfig{
@@ -50,14 +121,14 @@ func InitConfig(v *viper.Viper, name string) {
 	})
 	v.SetDefault("upstream", map[string]*UpstreamConfig{
 		"0": {
-			BootstrapIP: "76.76.2.11",
+			BootstrapIP: FreeDNSBoostrapIP,
 			Name:        "Control D - Anti-Malware",
 			Type:        ResolverTypeDOH,
 			Endpoint:    "https://freedns.controld.com/p1",
 			Timeout:     5000,
 		},
 		"1": {
-			BootstrapIP: "76.76.2.11",
+			BootstrapIP: FreeDNSBoostrapIP,
 			Name:        "Control D - No Ads",
 			Type:        ResolverTypeDOQ,
 			Endpoint:    "p2.freedns.controld.com",
@@ -69,21 +140,88 @@ func InitConfig(v *viper.Viper, name string) {
 // Config represents ctrld supported configuration.
 type Config struct {
 	Service  ServiceConfig              `mapstructure:"service" toml:"service,omitempty"`
+	Listener map[string]*ListenerConfig `mapstructure:"listener" toml:"listener" validate:"min=1,dive"`
 	Network  map[string]*NetworkConfig  `mapstructure:"network" toml:"network" validate:"min=1,dive"`
 	Upstream map[string]*UpstreamConfig `mapstructure:"upstream" toml:"upstream" validate:"min=1,dive"`
-	Listener map[string]*ListenerConfig `mapstructure:"listener" toml:"listener" validate:"min=1,dive"`
+}
+
+// HasUpstreamSendClientInfo reports whether the config has any upstream
+// is configured to send client info to Control D DNS server.
+func (c *Config) HasUpstreamSendClientInfo() bool {
+	for _, uc := range c.Upstream {
+		if uc.UpstreamSendClientInfo() {
+			return true
+		}
+	}
+	return false
+}
+
+// FirstListener returns the first listener config of current config. Listeners are sorted numerically.
+//
+// It panics if Config has no listeners configured.
+func (c *Config) FirstListener() *ListenerConfig {
+	listeners := make([]int, 0, len(c.Listener))
+	for k := range c.Listener {
+		n, err := strconv.Atoi(k)
+		if err != nil {
+			continue
+		}
+		listeners = append(listeners, n)
+	}
+	if len(listeners) == 0 {
+		panic("missing listener config")
+	}
+	sort.Ints(listeners)
+	return c.Listener[strconv.Itoa(listeners[0])]
+}
+
+// FirstUpstream returns the first upstream of current config. Upstreams are sorted numerically.
+//
+// It panics if Config has no upstreams configured.
+func (c *Config) FirstUpstream() *UpstreamConfig {
+	upstreams := make([]int, 0, len(c.Upstream))
+	for k := range c.Upstream {
+		n, err := strconv.Atoi(k)
+		if err != nil {
+			continue
+		}
+		upstreams = append(upstreams, n)
+	}
+	if len(upstreams) == 0 {
+		panic("missing listener config")
+	}
+	sort.Ints(upstreams)
+	return c.Upstream[strconv.Itoa(upstreams[0])]
 }
 
 // ServiceConfig specifies the general ctrld config.
 type ServiceConfig struct {
-	LogLevel         string `mapstructure:"log_level" toml:"log_level,omitempty"`
-	LogPath          string `mapstructure:"log_path" toml:"log_path,omitempty"`
-	CacheEnable      bool   `mapstructure:"cache_enable" toml:"cache_enable,omitempty"`
-	CacheSize        int    `mapstructure:"cache_size" toml:"cache_size,omitempty"`
-	CacheTTLOverride int    `mapstructure:"cache_ttl_override" toml:"cache_ttl_override,omitempty"`
-	CacheServeStale  bool   `mapstructure:"cache_serve_stale" toml:"cache_serve_stale,omitempty"`
-	Daemon           bool   `mapstructure:"-" toml:"-"`
-	AllocateIP       bool   `mapstructure:"-" toml:"-"`
+	LogLevel                string         `mapstructure:"log_level" toml:"log_level,omitempty"`
+	LogPath                 string         `mapstructure:"log_path" toml:"log_path,omitempty"`
+	CacheEnable             bool           `mapstructure:"cache_enable" toml:"cache_enable,omitempty"`
+	CacheSize               int            `mapstructure:"cache_size" toml:"cache_size,omitempty"`
+	CacheTTLOverride        int            `mapstructure:"cache_ttl_override" toml:"cache_ttl_override,omitempty"`
+	CacheServeStale         bool           `mapstructure:"cache_serve_stale" toml:"cache_serve_stale,omitempty"`
+	CacheFlushDomains       []string       `mapstructure:"cache_flush_domains" toml:"cache_flush_domains" validate:"max=256"`
+	MaxConcurrentRequests   *int           `mapstructure:"max_concurrent_requests" toml:"max_concurrent_requests,omitempty" validate:"omitempty,gte=0"`
+	DHCPLeaseFile           string         `mapstructure:"dhcp_lease_file_path" toml:"dhcp_lease_file_path" validate:"omitempty,file"`
+	DHCPLeaseFileFormat     string         `mapstructure:"dhcp_lease_file_format" toml:"dhcp_lease_file_format" validate:"required_unless=DHCPLeaseFile '',omitempty,oneof=dnsmasq isc-dhcp"`
+	DiscoverMDNS            *bool          `mapstructure:"discover_mdns" toml:"discover_mdns,omitempty"`
+	DiscoverARP             *bool          `mapstructure:"discover_arp" toml:"discover_arp,omitempty"`
+	DiscoverDHCP            *bool          `mapstructure:"discover_dhcp" toml:"discover_dhcp,omitempty"`
+	DiscoverPtr             *bool          `mapstructure:"discover_ptr" toml:"discover_ptr,omitempty"`
+	DiscoverHosts           *bool          `mapstructure:"discover_hosts" toml:"discover_hosts,omitempty"`
+	DiscoverRefreshInterval int            `mapstructure:"discover_refresh_interval" toml:"discover_refresh_interval,omitempty"`
+	ClientIDPref            string         `mapstructure:"client_id_preference" toml:"client_id_preference,omitempty" validate:"omitempty,oneof=host mac"`
+	MetricsQueryStats       bool           `mapstructure:"metrics_query_stats" toml:"metrics_query_stats,omitempty"`
+	MetricsListener         string         `mapstructure:"metrics_listener" toml:"metrics_listener,omitempty"`
+	DnsWatchdogEnabled      *bool          `mapstructure:"dns_watchdog_enabled" toml:"dns_watchdog_enabled,omitempty"`
+	DnsWatchdogInvterval    *time.Duration `mapstructure:"dns_watchdog_interval" toml:"dns_watchdog_interval,omitempty"`
+	RefetchTime             *int           `mapstructure:"refetch_time" toml:"refetch_time,omitempty"`
+	ForceRefetchWaitTime    *int           `mapstructure:"force_refetch_wait_time" toml:"force_refetch_wait_time,omitempty"`
+	LeakOnUpstreamFailure   *bool          `mapstructure:"leak_on_upstream_failure" toml:"leak_on_upstream_failure,omitempty"`
+	Daemon                  bool           `mapstructure:"-" toml:"-"`
+	AllocateIP              bool           `mapstructure:"-" toml:"-"`
 }
 
 // NetworkConfig specifies configuration for networks where ctrld will handle requests.
@@ -95,22 +233,62 @@ type NetworkConfig struct {
 
 // UpstreamConfig specifies configuration for upstreams that ctrld will forward requests to.
 type UpstreamConfig struct {
-	Name              string              `mapstructure:"name" toml:"name,omitempty"`
-	Type              string              `mapstructure:"type" toml:"type,omitempty" validate:"oneof=doh doh3 dot doq os legacy"`
-	Endpoint          string              `mapstructure:"endpoint" toml:"endpoint,omitempty" validate:"required_unless=Type os"`
-	BootstrapIP       string              `mapstructure:"bootstrap_ip" toml:"bootstrap_ip,omitempty"`
-	Domain            string              `mapstructure:"-" toml:"-"`
-	Timeout           int                 `mapstructure:"timeout" toml:"timeout,omitempty" validate:"gte=0"`
-	transport         *http.Transport     `mapstructure:"-" toml:"-"`
-	http3RoundTripper *http3.RoundTripper `mapstructure:"-" toml:"-"`
+	Name        string `mapstructure:"name" toml:"name,omitempty"`
+	Type        string `mapstructure:"type" toml:"type,omitempty" validate:"oneof=doh doh3 dot doq os legacy sdns ''"`
+	Endpoint    string `mapstructure:"endpoint" toml:"endpoint,omitempty"`
+	BootstrapIP string `mapstructure:"bootstrap_ip" toml:"bootstrap_ip,omitempty"`
+	Domain      string `mapstructure:"-" toml:"-"`
+	IPStack     string `mapstructure:"ip_stack" toml:"ip_stack,omitempty" validate:"ipstack"`
+	Timeout     int    `mapstructure:"timeout" toml:"timeout,omitempty" validate:"gte=0"`
+	// The caller should not access this field directly.
+	// Use UpstreamSendClientInfo instead.
+	SendClientInfo *bool `mapstructure:"send_client_info" toml:"send_client_info,omitempty"`
+	// The caller should not access this field directly.
+	// Use IsDiscoverable instead.
+	Discoverable *bool `mapstructure:"discoverable" toml:"discoverable"`
+
+	g                  singleflight.Group
+	rebootstrap        atomic.Bool
+	bootstrapIPs       []string
+	bootstrapIPs4      []string
+	bootstrapIPs6      []string
+	transport          *http.Transport
+	transportOnce      sync.Once
+	transport4         *http.Transport
+	transport6         *http.Transport
+	http3RoundTripper  http.RoundTripper
+	http3RoundTripper4 http.RoundTripper
+	http3RoundTripper6 http.RoundTripper
+	certPool           *x509.CertPool
+	u                  *url.URL
+	uid                string
 }
 
 // ListenerConfig specifies the networks configuration that ctrld will run on.
 type ListenerConfig struct {
-	IP         string                `mapstructure:"ip" toml:"ip,omitempty" validate:"ip"`
-	Port       int                   `mapstructure:"port" toml:"port,omitempty" validate:"gt=0"`
-	Restricted bool                  `mapstructure:"restricted" toml:"restricted,omitempty"`
-	Policy     *ListenerPolicyConfig `mapstructure:"policy" toml:"policy,omitempty"`
+	IP              string                `mapstructure:"ip" toml:"ip,omitempty" validate:"iporempty"`
+	Port            int                   `mapstructure:"port" toml:"port,omitempty" validate:"gte=0"`
+	Restricted      bool                  `mapstructure:"restricted" toml:"restricted,omitempty"`
+	AllowWanClients bool                  `mapstructure:"allow_wan_clients" toml:"allow_wan_clients,omitempty"`
+	Policy          *ListenerPolicyConfig `mapstructure:"policy" toml:"policy,omitempty"`
+}
+
+// IsDirectDnsListener reports whether ctrld can be a direct listener on port 53.
+// It returns true only if ctrld can listen on port 53 for all interfaces. That means
+// there's no other software listening on port 53.
+//
+// If someone listening on port 53, or ctrld could only listen on port 53 for a specific
+// interface, ctrld could only be configured as a DNS forwarder.
+func (lc *ListenerConfig) IsDirectDnsListener() bool {
+	if lc == nil || lc.Port != 53 {
+		return false
+	}
+	switch lc.IP {
+	case "", "::", "0.0.0.0":
+		return true
+	default:
+		return false
+	}
 }
 
 // ListenerPolicyConfig specifies the policy rules for ctrld to filter incoming requests.
@@ -118,6 +296,7 @@ type ListenerPolicyConfig struct {
 	Name                 string   `mapstructure:"name" toml:"name,omitempty"`
 	Networks             []Rule   `mapstructure:"networks" toml:"networks,omitempty,inline,multiline" validate:"dive,len=1"`
 	Rules                []Rule   `mapstructure:"rules" toml:"rules,omitempty,inline,multiline" validate:"dive,len=1"`
+	Macs                 []Rule   `mapstructure:"macs" toml:"macs,omitempty,inline,multiline" validate:"dive,len=1"`
 	FailoverRcodes       []string `mapstructure:"failover_rcodes" toml:"failover_rcodes,omitempty" validate:"dive,dnsrcode"`
 	FailoverRcodeNumbers []int    `mapstructure:"-" toml:"-"`
 }
@@ -129,22 +308,160 @@ type Rule map[string][]string
 
 // Init initialized necessary values for an UpstreamConfig.
 func (uc *UpstreamConfig) Init() {
-	if u, err := url.Parse(uc.Endpoint); err == nil {
-		uc.Domain = u.Host
+	if err := uc.initDnsStamps(); err != nil {
+		ProxyLogger.Load().Fatal().Err(err).Msg("invalid DNS Stamps")
 	}
-	if uc.Domain != "" {
+	uc.initDoHScheme()
+	uc.uid = upstreamUID()
+	if u, err := url.Parse(uc.Endpoint); err == nil {
+		uc.Domain = u.Hostname()
+		switch uc.Type {
+		case ResolverTypeDOH, ResolverTypeDOH3:
+			uc.u = u
+		}
+	}
+	if uc.Domain == "" {
+		if !strings.Contains(uc.Endpoint, ":") {
+			uc.Domain = uc.Endpoint
+			uc.Endpoint = net.JoinHostPort(uc.Endpoint, defaultPortFor(uc.Type))
+		}
+		host, _, _ := net.SplitHostPort(uc.Endpoint)
+		uc.Domain = host
+		if net.ParseIP(uc.Domain) != nil {
+			uc.BootstrapIP = uc.Domain
+		}
+	}
+	if uc.IPStack == "" {
+		if uc.IsControlD() {
+			uc.IPStack = IpStackSplit
+		} else {
+			uc.IPStack = IpStackBoth
+		}
+	}
+}
+
+// VerifyDomain returns the domain name that could be resolved by the upstream endpoint.
+// It returns empty for non-ControlD upstream endpoint.
+func (uc *UpstreamConfig) VerifyDomain() string {
+	domain := uc.Domain
+	if domain == "" {
+		if u, err := url.Parse(uc.Endpoint); err == nil {
+			domain = u.Hostname()
+		}
+	}
+	for _, parent := range controldParentDomains {
+		if dns.IsSubDomain(parent, domain) {
+			return controldVerifiedDomain[parent]
+		}
+	}
+	return ""
+}
+
+// UpstreamSendClientInfo reports whether the upstream is
+// configured to send client info to Control D DNS server.
+//
+// Client info includes:
+//   - MAC
+//   - Lan IP
+//   - Hostname
+func (uc *UpstreamConfig) UpstreamSendClientInfo() bool {
+	if uc.SendClientInfo != nil {
+		return *uc.SendClientInfo
+	}
+	switch uc.Type {
+	case ResolverTypeDOH, ResolverTypeDOH3:
+		if uc.IsControlD() || uc.isNextDNS() {
+			return true
+		}
+	}
+	return false
+}
+
+// IsDiscoverable reports whether the upstream can be used for PTR discovery.
+// The caller must ensure uc.Init() was called before calling this.
+func (uc *UpstreamConfig) IsDiscoverable() bool {
+	if uc.Discoverable != nil {
+		return *uc.Discoverable
+	}
+	switch uc.Type {
+	case ResolverTypeOS, ResolverTypeLegacy, ResolverTypePrivate:
+		if ip, err := netip.ParseAddr(uc.Domain); err == nil {
+			return ip.IsLoopback() || ip.IsPrivate() || ip.IsLinkLocalUnicast() || tsaddr.CGNATRange().Contains(ip)
+		}
+	}
+	return false
+}
+
+// BootstrapIPs returns the bootstrap IPs list of upstreams.
+func (uc *UpstreamConfig) BootstrapIPs() []string {
+	return uc.bootstrapIPs
+}
+
+// SetCertPool sets the system cert pool used for TLS connections.
+func (uc *UpstreamConfig) SetCertPool(cp *x509.CertPool) {
+	uc.certPool = cp
+}
+
+// SetupBootstrapIP manually find all available IPs of the upstream.
+// The first usable IP will be used as bootstrap IP of the upstream.
+func (uc *UpstreamConfig) SetupBootstrapIP() {
+	uc.setupBootstrapIP(true)
+}
+
+// UID returns the unique identifier of the upstream.
+func (uc *UpstreamConfig) UID() string {
+	return uc.uid
+}
+
+// SetupBootstrapIP manually find all available IPs of the upstream.
+// The first usable IP will be used as bootstrap IP of the upstream.
+func (uc *UpstreamConfig) setupBootstrapIP(withBootstrapDNS bool) {
+	b := backoff.NewBackoff("setupBootstrapIP", func(format string, args ...any) {}, 10*time.Second)
+	isControlD := uc.IsControlD()
+	for {
+		uc.bootstrapIPs = lookupIP(uc.Domain, uc.Timeout, withBootstrapDNS)
+		// For ControlD upstream, the bootstrap IPs could not be RFC 1918 addresses,
+		// filtering them out here to prevent weird behavior.
+		if isControlD {
+			n := 0
+			for _, ip := range uc.bootstrapIPs {
+				netIP := net.ParseIP(ip)
+				if netIP != nil && !netIP.IsPrivate() {
+					uc.bootstrapIPs[n] = ip
+					n++
+				}
+			}
+			uc.bootstrapIPs = uc.bootstrapIPs[:n]
+		}
+		if len(uc.bootstrapIPs) > 0 {
+			break
+		}
+		ProxyLogger.Load().Warn().Msg("could not resolve bootstrap IPs, retrying...")
+		b.BackOff(context.Background(), errors.New("no bootstrap IPs"))
+	}
+	for _, ip := range uc.bootstrapIPs {
+		if ctrldnet.IsIPv6(ip) {
+			uc.bootstrapIPs6 = append(uc.bootstrapIPs6, ip)
+		} else {
+			uc.bootstrapIPs4 = append(uc.bootstrapIPs4, ip)
+		}
+	}
+	ProxyLogger.Load().Debug().Msgf("bootstrap IPs: %v", uc.bootstrapIPs)
+}
+
+// ReBootstrap re-setup the bootstrap IP and the transport.
+func (uc *UpstreamConfig) ReBootstrap() {
+	switch uc.Type {
+	case ResolverTypeDOH, ResolverTypeDOH3:
+	default:
 		return
 	}
-
-	if !strings.Contains(uc.Endpoint, ":") {
-		uc.Domain = uc.Endpoint
-		uc.Endpoint = net.JoinHostPort(uc.Endpoint, defaultPortFor(uc.Type))
-	}
-	host, _, _ := net.SplitHostPort(uc.Endpoint)
-	uc.Domain = host
-	if net.ParseIP(uc.Domain) != nil {
-		uc.BootstrapIP = uc.Domain
-	}
+	_, _, _ = uc.g.Do("ReBootstrap", func() (any, error) {
+		if uc.rebootstrap.CompareAndSwap(false, true) {
+			ProxyLogger.Load().Debug().Msg("re-bootstrapping upstream ip")
+		}
+		return true, nil
+	})
 }
 
 // SetupTransport initializes the network transport used to connect to upstream server.
@@ -159,66 +476,277 @@ func (uc *UpstreamConfig) SetupTransport() {
 }
 
 func (uc *UpstreamConfig) setupDOHTransport() {
-	uc.transport = http.DefaultTransport.(*http.Transport).Clone()
-	uc.transport.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {
-		dialer := &net.Dialer{
-			Timeout:   10 * time.Second,
-			KeepAlive: 10 * time.Second,
+	switch uc.IPStack {
+	case IpStackBoth, "":
+		uc.transport = uc.newDOHTransport(uc.bootstrapIPs)
+	case IpStackV4:
+		uc.transport = uc.newDOHTransport(uc.bootstrapIPs4)
+	case IpStackV6:
+		uc.transport = uc.newDOHTransport(uc.bootstrapIPs6)
+	case IpStackSplit:
+		uc.transport4 = uc.newDOHTransport(uc.bootstrapIPs4)
+		if hasIPv6() {
+			uc.transport6 = uc.newDOHTransport(uc.bootstrapIPs6)
+		} else {
+			uc.transport6 = uc.transport4
 		}
-		Log(ctx, ProxyLog.Debug(), "debug dial context %s - %s - %s", addr, network, bootstrapDNS)
-		// if we have a bootstrap ip set, use it to avoid DNS lookup
-		if uc.BootstrapIP != "" {
-			if _, port, _ := net.SplitHostPort(addr); port != "" {
-				addr = net.JoinHostPort(uc.BootstrapIP, port)
-			}
-			Log(ctx, ProxyLog.Debug(), "sending doh request to: %s", addr)
-		}
-		return dialer.DialContext(ctx, network, addr)
+		uc.transport = uc.newDOHTransport(uc.bootstrapIPs)
 	}
-
-	uc.pingUpstream()
 }
 
-func (uc *UpstreamConfig) setupDOH3Transport() {
-	uc.http3RoundTripper = &http3.RoundTripper{}
-	uc.http3RoundTripper.Dial = func(ctx context.Context, addr string, tlsCfg *tls.Config, cfg *quic.Config) (quic.EarlyConnection, error) {
-		host := addr
-		ProxyLog.Debug().Msgf("debug dial context D0H3 %s - %s", addr, bootstrapDNS)
-		// if we have a bootstrap ip set, use it to avoid DNS lookup
+func (uc *UpstreamConfig) newDOHTransport(addrs []string) *http.Transport {
+	transport := http.DefaultTransport.(*http.Transport).Clone()
+	transport.MaxIdleConnsPerHost = 100
+	transport.TLSClientConfig = &tls.Config{
+		RootCAs:            uc.certPool,
+		ClientSessionCache: tls.NewLRUClientSessionCache(0),
+	}
+
+	// Prevent bad tcp connection hanging the requests for too long.
+	// See: https://github.com/golang/go/issues/36026
+	if t2, err := http2.ConfigureTransports(transport); err == nil {
+		t2.ReadIdleTimeout = 10 * time.Second
+		t2.PingTimeout = 5 * time.Second
+	}
+
+	dialerTimeoutMs := 2000
+	if uc.Timeout > 0 && uc.Timeout < dialerTimeoutMs {
+		dialerTimeoutMs = uc.Timeout
+	}
+	dialerTimeout := time.Duration(dialerTimeoutMs) * time.Millisecond
+	transport.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {
+		_, port, _ := net.SplitHostPort(addr)
 		if uc.BootstrapIP != "" {
-			if _, port, _ := net.SplitHostPort(addr); port != "" {
-				addr = net.JoinHostPort(uc.BootstrapIP, port)
-			}
-			ProxyLog.Debug().Msgf("sending doh3 request to: %s", addr)
+			dialer := net.Dialer{Timeout: dialerTimeout, KeepAlive: dialerTimeout}
+			addr := net.JoinHostPort(uc.BootstrapIP, port)
+			Log(ctx, ProxyLogger.Load().Debug(), "sending doh request to: %s", addr)
+			return dialer.DialContext(ctx, network, addr)
 		}
-		remoteAddr, err := net.ResolveUDPAddr("udp", addr)
+		pd := &ctrldnet.ParallelDialer{}
+		pd.Timeout = dialerTimeout
+		pd.KeepAlive = dialerTimeout
+		dialAddrs := make([]string, len(addrs))
+		for i := range addrs {
+			dialAddrs[i] = net.JoinHostPort(addrs[i], port)
+		}
+		conn, err := pd.DialContext(ctx, network, dialAddrs)
 		if err != nil {
 			return nil, err
 		}
-
-		udpConn, err := net.ListenUDP("udp", nil)
-		if err != nil {
-			return nil, err
-		}
-		return quic.DialEarlyContext(ctx, udpConn, remoteAddr, host, tlsCfg, cfg)
+		Log(ctx, ProxyLogger.Load().Debug(), "sending doh request to: %s", conn.RemoteAddr())
+		return conn, nil
 	}
-
-	uc.pingUpstream()
+	runtime.SetFinalizer(transport, func(transport *http.Transport) {
+		transport.CloseIdleConnections()
+	})
+	return transport
 }
 
-func (uc *UpstreamConfig) pingUpstream() {
-	// Warming up the transport by querying a test packet.
-	dnsResolver, err := NewResolver(uc)
-	if err != nil {
-		ProxyLog.Error().Err(err).Msgf("failed to create resolver for upstream: %s", uc.Name)
+// Ping warms up the connection to DoH/DoH3 upstream.
+func (uc *UpstreamConfig) Ping() {
+	_ = uc.ping()
+}
+
+// ErrorPing is like Ping, but return an error if any.
+func (uc *UpstreamConfig) ErrorPing() error {
+	return uc.ping()
+}
+
+func (uc *UpstreamConfig) ping() error {
+	switch uc.Type {
+	case ResolverTypeDOH, ResolverTypeDOH3:
+	default:
+		return nil
+	}
+
+	ping := func(t http.RoundTripper) error {
+		if t == nil {
+			return nil
+		}
+		ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
+		defer cancel()
+		req, err := http.NewRequestWithContext(ctx, "HEAD", uc.Endpoint, nil)
+		if err != nil {
+			return err
+		}
+		resp, err := t.RoundTrip(req)
+		if err != nil {
+			return err
+		}
+		defer resp.Body.Close()
+		_, _ = io.Copy(io.Discard, resp.Body)
+		return nil
+	}
+
+	for _, typ := range []uint16{dns.TypeA, dns.TypeAAAA} {
+		switch uc.Type {
+		case ResolverTypeDOH:
+
+			if err := ping(uc.dohTransport(typ)); err != nil {
+				return err
+			}
+		case ResolverTypeDOH3:
+			if err := ping(uc.doh3Transport(typ)); err != nil {
+				return err
+			}
+		}
+	}
+
+	return nil
+}
+
+// IsControlD reports whether this is a ControlD upstream.
+func (uc *UpstreamConfig) IsControlD() bool {
+	domain := uc.Domain
+	if domain == "" {
+		if u, err := url.Parse(uc.Endpoint); err == nil {
+			domain = u.Hostname()
+		}
+	}
+	for _, parent := range controldParentDomains {
+		if dns.IsSubDomain(parent, domain) {
+			return true
+		}
+	}
+	return false
+}
+
+func (uc *UpstreamConfig) isNextDNS() bool {
+	domain := uc.Domain
+	if domain == "" {
+		if u, err := url.Parse(uc.Endpoint); err == nil {
+			domain = u.Hostname()
+		}
+	}
+	return domain == "dns.nextdns.io"
+}
+
+func (uc *UpstreamConfig) dohTransport(dnsType uint16) http.RoundTripper {
+	uc.transportOnce.Do(func() {
+		uc.SetupTransport()
+	})
+	if uc.rebootstrap.CompareAndSwap(true, false) {
+		uc.SetupTransport()
+	}
+	switch uc.IPStack {
+	case IpStackBoth, IpStackV4, IpStackV6:
+		return uc.transport
+	case IpStackSplit:
+		switch dnsType {
+		case dns.TypeA:
+			return uc.transport4
+		default:
+			return uc.transport6
+		}
+	}
+	return uc.transport
+}
+
+func (uc *UpstreamConfig) bootstrapIPForDNSType(dnsType uint16) string {
+	switch uc.IPStack {
+	case IpStackBoth:
+		return pick(uc.bootstrapIPs)
+	case IpStackV4:
+		return pick(uc.bootstrapIPs4)
+	case IpStackV6:
+		return pick(uc.bootstrapIPs6)
+	case IpStackSplit:
+		switch dnsType {
+		case dns.TypeA:
+			return pick(uc.bootstrapIPs4)
+		default:
+			if hasIPv6() {
+				return pick(uc.bootstrapIPs6)
+			}
+			return pick(uc.bootstrapIPs4)
+		}
+	}
+	return pick(uc.bootstrapIPs)
+}
+
+func (uc *UpstreamConfig) netForDNSType(dnsType uint16) (string, string) {
+	switch uc.IPStack {
+	case IpStackBoth:
+		return "tcp-tls", "udp"
+	case IpStackV4:
+		return "tcp4-tls", "udp4"
+	case IpStackV6:
+		return "tcp6-tls", "udp6"
+	case IpStackSplit:
+		switch dnsType {
+		case dns.TypeA:
+			return "tcp4-tls", "udp4"
+		default:
+			if hasIPv6() {
+				return "tcp6-tls", "udp6"
+			}
+			return "tcp4-tls", "udp4"
+		}
+	}
+	return "tcp-tls", "udp"
+}
+
+// initDoHScheme initializes the endpoint scheme for DoH/DoH3 upstream if not present.
+func (uc *UpstreamConfig) initDoHScheme() {
+	if strings.HasPrefix(uc.Endpoint, endpointPrefixH3) && uc.Type == "" {
+		uc.Type = ResolverTypeDOH3
+	}
+	switch uc.Type {
+	case ResolverTypeDOH:
+	case ResolverTypeDOH3:
+		if after, found := strings.CutPrefix(uc.Endpoint, endpointPrefixH3); found {
+			uc.Endpoint = endpointPrefixHTTPS + after
+		}
+	default:
 		return
 	}
-	msg := new(dns.Msg)
-	msg.SetQuestion(".", dns.TypeNS)
-	msg.MsgHdr.RecursionDesired = true
-	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
-	defer cancel()
-	_, _ = dnsResolver.Resolve(ctx, msg)
+	if !strings.HasPrefix(uc.Endpoint, endpointPrefixHTTPS) {
+		uc.Endpoint = endpointPrefixHTTPS + uc.Endpoint
+	}
+}
+
+// initDnsStamps initializes upstream config based on encoded DNS Stamps Endpoint.
+func (uc *UpstreamConfig) initDnsStamps() error {
+	if strings.HasPrefix(uc.Endpoint, endpointPrefixSdns) && uc.Type == "" {
+		uc.Type = ResolverTypeSDNS
+	}
+	if uc.Type != ResolverTypeSDNS {
+		return nil
+	}
+	sdns, err := dnsstamps.NewServerStampFromString(uc.Endpoint)
+	if err != nil {
+		return err
+	}
+	ip, port, _ := net.SplitHostPort(sdns.ServerAddrStr)
+	providerName, port2, _ := net.SplitHostPort(sdns.ProviderName)
+	if port2 != "" {
+		port = port2
+	}
+	if providerName == "" {
+		providerName = sdns.ProviderName
+	}
+	switch sdns.Proto {
+	case dnsstamps.StampProtoTypeDoH:
+		uc.Type = ResolverTypeDOH
+		host := sdns.ProviderName
+		if port != "" && port != defaultPortFor(uc.Type) {
+			host = net.JoinHostPort(providerName, port)
+		}
+		uc.Endpoint = "https://" + host + sdns.Path
+	case dnsstamps.StampProtoTypeTLS:
+		uc.Type = ResolverTypeDOT
+		uc.Endpoint = net.JoinHostPort(providerName, port)
+	case dnsstamps.StampProtoTypeDoQ:
+		uc.Type = ResolverTypeDOQ
+		uc.Endpoint = net.JoinHostPort(providerName, port)
+	case dnsstamps.StampProtoTypePlain:
+		uc.Type = ResolverTypeLegacy
+		uc.Endpoint = sdns.ServerAddrStr
+	default:
+		return fmt.Errorf("unsupported stamp protocol %q", sdns.Proto)
+	}
+	uc.BootstrapIP = ip
+	return nil
 }
 
 // Init initialized necessary values for an ListenerConfig.
@@ -234,6 +762,9 @@ func (lc *ListenerConfig) Init() {
 // ValidateConfig validates the given config.
 func ValidateConfig(validate *validator.Validate, cfg *Config) error {
 	_ = validate.RegisterValidation("dnsrcode", validateDnsRcode)
+	_ = validate.RegisterValidation("ipstack", validateIpStack)
+	_ = validate.RegisterValidation("iporempty", validateIpOrEmpty)
+	validate.RegisterStructValidation(upstreamConfigStructLevelValidation, UpstreamConfig{})
 	return validate.Struct(cfg)
 }
 
@@ -241,6 +772,63 @@ func validateDnsRcode(fl validator.FieldLevel) bool {
 	return dnsrcode.FromString(fl.Field().String()) != -1
 }
 
+func validateIpStack(fl validator.FieldLevel) bool {
+	switch fl.Field().String() {
+	case IpStackBoth, IpStackV4, IpStackV6, IpStackSplit, "":
+		return true
+	default:
+		return false
+	}
+}
+
+func validateIpOrEmpty(fl validator.FieldLevel) bool {
+	val := fl.Field().String()
+	if val == "" {
+		return true
+	}
+	return net.ParseIP(val) != nil
+}
+
+func upstreamConfigStructLevelValidation(sl validator.StructLevel) {
+	uc := sl.Current().Addr().Interface().(*UpstreamConfig)
+	if uc.Type == ResolverTypeOS {
+		return
+	}
+
+	// Endpoint is required for non os resolver.
+	if uc.Endpoint == "" {
+		sl.ReportError(uc.Endpoint, "endpoint", "Endpoint", "required_unless", "")
+		return
+	}
+
+	// Empty type is ok only for endpoints starts with "h3://" and "sdns://".
+	if uc.Type == "" && !strings.HasPrefix(uc.Endpoint, endpointPrefixH3) && !strings.HasPrefix(uc.Endpoint, endpointPrefixSdns) {
+		sl.ReportError(uc.Endpoint, "type", "type", "oneof", "doh doh3 dot doq os legacy sdns")
+		return
+	}
+
+	// initDoHScheme/initDnsStamps may change upstreams information,
+	// so restoring changed values after validation to keep original one.
+	defer func(ep, typ string) {
+		uc.Endpoint = ep
+		uc.Type = typ
+	}(uc.Endpoint, uc.Type)
+
+	if err := uc.initDnsStamps(); err != nil {
+		sl.ReportError(uc.Endpoint, "endpoint", "Endpoint", "http_url", "")
+		return
+	}
+	uc.initDoHScheme()
+	// DoH/DoH3 requires endpoint is an HTTP url.
+	if uc.Type == ResolverTypeDOH || uc.Type == ResolverTypeDOH3 {
+		u, err := url.Parse(uc.Endpoint)
+		if err != nil || u.Host == "" {
+			sl.ReportError(uc.Endpoint, "endpoint", "Endpoint", "http_url", "")
+			return
+		}
+	}
+}
+
 func defaultPortFor(typ string) string {
 	switch typ {
 	case ResolverTypeDOH, ResolverTypeDOH3:
@@ -252,3 +840,49 @@ func defaultPortFor(typ string) string {
 	}
 	return "53"
 }
+
+// ResolverTypeFromEndpoint tries guessing the resolver type with a given endpoint
+// using following rules:
+//
+// - If endpoint is an IP address ->  ResolverTypeLegacy
+// - If endpoint starts with "https://" -> ResolverTypeDOH
+// - If endpoint starts with "quic://" -> ResolverTypeDOQ
+// - If endpoint starts with "h3://" -> ResolverTypeDOH3
+// - If endpoint starts with "sdns://" -> ResolverTypeSDNS
+// - For anything else -> ResolverTypeDOT
+func ResolverTypeFromEndpoint(endpoint string) string {
+	switch {
+	case strings.HasPrefix(endpoint, endpointPrefixHTTPS):
+		return ResolverTypeDOH
+	case strings.HasPrefix(endpoint, endpointPrefixQUIC):
+		return ResolverTypeDOQ
+	case strings.HasPrefix(endpoint, endpointPrefixH3):
+		return ResolverTypeDOH3
+	case strings.HasPrefix(endpoint, endpointPrefixSdns):
+		return ResolverTypeSDNS
+	}
+	host := endpoint
+	if strings.Contains(endpoint, ":") {
+		host, _, _ = net.SplitHostPort(host)
+	}
+	if ip := net.ParseIP(host); ip != nil {
+		return ResolverTypeLegacy
+	}
+	return ResolverTypeDOT
+}
+
+func pick(s []string) string {
+	return s[rand.Intn(len(s))]
+}
+
+// upstreamUID generates an unique identifier for an upstream.
+func upstreamUID() string {
+	b := make([]byte, 4)
+	for {
+		if _, err := crand.Read(b); err != nil {
+			ProxyLogger.Load().Warn().Err(err).Msg("could not generate uid for upstream, retrying...")
+			continue
+		}
+		return hex.EncodeToString(b)
+	}
+}

config_internal_test.go

@@ -0,0 +1,486 @@
+package ctrld
+
+import (
+	"net/url"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestUpstreamConfig_SetupBootstrapIP(t *testing.T) {
+	uc := &UpstreamConfig{
+		Name:     "test",
+		Type:     ResolverTypeDOH,
+		Endpoint: "https://freedns.controld.com/p2",
+		Timeout:  5000,
+	}
+	uc.Init()
+	uc.setupBootstrapIP(false)
+	if len(uc.bootstrapIPs) == 0 {
+		t.Log(defaultNameservers())
+		t.Fatal("could not bootstrap ip without bootstrap DNS")
+	}
+	t.Log(uc)
+}
+
+func TestUpstreamConfig_Init(t *testing.T) {
+	u1, _ := url.Parse("https://example.com")
+	u2, _ := url.Parse("https://example.com?k=v")
+	u3, _ := url.Parse("https://freedns.controld.com/p1")
+	tests := []struct {
+		name     string
+		uc       *UpstreamConfig
+		expected *UpstreamConfig
+	}{
+		{
+			"doh+doh3",
+			&UpstreamConfig{
+				Name:        "doh",
+				Type:        "doh",
+				Endpoint:    "https://example.com",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+			},
+			&UpstreamConfig{
+				Name:        "doh",
+				Type:        "doh",
+				Endpoint:    "https://example.com",
+				BootstrapIP: "",
+				Domain:      "example.com",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+				u:           u1,
+			},
+		},
+		{
+			"doh+doh3 with query param",
+			&UpstreamConfig{
+				Name:        "doh",
+				Type:        "doh",
+				Endpoint:    "https://example.com?k=v",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+			},
+			&UpstreamConfig{
+				Name:        "doh",
+				Type:        "doh",
+				Endpoint:    "https://example.com?k=v",
+				BootstrapIP: "",
+				Domain:      "example.com",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+				u:           u2,
+			},
+		},
+		{
+			"dot+doq",
+			&UpstreamConfig{
+				Name:        "dot",
+				Type:        "dot",
+				Endpoint:    "freedns.controld.com:8853",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+			},
+			&UpstreamConfig{
+				Name:        "dot",
+				Type:        "dot",
+				Endpoint:    "freedns.controld.com:8853",
+				BootstrapIP: "",
+				Domain:      "freedns.controld.com",
+				Timeout:     0,
+				IPStack:     IpStackSplit,
+			},
+		},
+		{
+			"dot+doq without port",
+			&UpstreamConfig{
+				Name:        "dot",
+				Type:        "dot",
+				Endpoint:    "freedns.controld.com",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+				IPStack:     IpStackSplit,
+			},
+			&UpstreamConfig{
+				Name:        "dot",
+				Type:        "dot",
+				Endpoint:    "freedns.controld.com:853",
+				BootstrapIP: "",
+				Domain:      "freedns.controld.com",
+				Timeout:     0,
+				IPStack:     IpStackSplit,
+			},
+		},
+		{
+			"legacy",
+			&UpstreamConfig{
+				Name:        "legacy",
+				Type:        "legacy",
+				Endpoint:    "1.2.3.4:53",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+			},
+			&UpstreamConfig{
+				Name:        "legacy",
+				Type:        "legacy",
+				Endpoint:    "1.2.3.4:53",
+				BootstrapIP: "1.2.3.4",
+				Domain:      "1.2.3.4",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+			},
+		},
+		{
+			"legacy without port",
+			&UpstreamConfig{
+				Name:        "legacy",
+				Type:        "legacy",
+				Endpoint:    "1.2.3.4",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+			},
+			&UpstreamConfig{
+				Name:        "legacy",
+				Type:        "legacy",
+				Endpoint:    "1.2.3.4:53",
+				BootstrapIP: "1.2.3.4",
+				Domain:      "1.2.3.4",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+			},
+		},
+		{
+			"doh+doh3 with send client info set",
+			&UpstreamConfig{
+				Name:           "doh",
+				Type:           "doh",
+				Endpoint:       "https://example.com?k=v",
+				BootstrapIP:    "",
+				Domain:         "",
+				Timeout:        0,
+				SendClientInfo: ptrBool(false),
+				IPStack:        IpStackBoth,
+			},
+			&UpstreamConfig{
+				Name:           "doh",
+				Type:           "doh",
+				Endpoint:       "https://example.com?k=v",
+				BootstrapIP:    "",
+				Domain:         "example.com",
+				Timeout:        0,
+				SendClientInfo: ptrBool(false),
+				IPStack:        IpStackBoth,
+				u:              u2,
+			},
+		},
+		{
+			"h3",
+			&UpstreamConfig{
+				Name:        "doh3",
+				Type:        "doh3",
+				Endpoint:    "h3://example.com",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+			},
+			&UpstreamConfig{
+				Name:        "doh3",
+				Type:        "doh3",
+				Endpoint:    "https://example.com",
+				BootstrapIP: "",
+				Domain:      "example.com",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+				u:           u1,
+			},
+		},
+		{
+			"h3 without type",
+			&UpstreamConfig{
+				Name:        "doh3",
+				Endpoint:    "h3://example.com",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+			},
+			&UpstreamConfig{
+				Name:        "doh3",
+				Type:        "doh3",
+				Endpoint:    "https://example.com",
+				BootstrapIP: "",
+				Domain:      "example.com",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+				u:           u1,
+			},
+		},
+		{
+			"sdns -> doh",
+			&UpstreamConfig{
+				Name:        "sdns",
+				Type:        "sdns",
+				Endpoint:    "sdns://AgMAAAAAAAAACjc2Ljc2LjIuMTEAFGZyZWVkbnMuY29udHJvbGQuY29tAy9wMQ",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+			},
+			&UpstreamConfig{
+				Name:        "sdns",
+				Type:        "doh",
+				Endpoint:    "https://freedns.controld.com/p1",
+				BootstrapIP: "76.76.2.11",
+				Domain:      "freedns.controld.com",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+				u:           u3,
+			},
+		},
+		{
+			"sdns -> dot",
+			&UpstreamConfig{
+				Name:        "sdns",
+				Type:        "sdns",
+				Endpoint:    "sdns://AwcAAAAAAAAACjc2Ljc2LjIuMTEAFGZyZWVkbnMuY29udHJvbGQuY29t",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+			},
+			&UpstreamConfig{
+				Name:        "sdns",
+				Type:        "dot",
+				Endpoint:    "freedns.controld.com:843",
+				BootstrapIP: "76.76.2.11",
+				Domain:      "freedns.controld.com",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+			},
+		},
+		{
+			"sdns -> doq",
+			&UpstreamConfig{
+				Name:        "sdns",
+				Type:        "sdns",
+				Endpoint:    "sdns://BAcAAAAAAAAACjc2Ljc2LjIuMTEAFGZyZWVkbnMuY29udHJvbGQuY29t",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+			},
+			&UpstreamConfig{
+				Name:        "sdns",
+				Type:        "doq",
+				Endpoint:    "freedns.controld.com:784",
+				BootstrapIP: "76.76.2.11",
+				Domain:      "freedns.controld.com",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+			},
+		},
+		{
+			"sdns -> legacy",
+			&UpstreamConfig{
+				Name:        "sdns",
+				Type:        "sdns",
+				Endpoint:    "sdns://AAcAAAAAAAAACjc2Ljc2LjIuMTE",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+			},
+			&UpstreamConfig{
+				Name:        "sdns",
+				Type:        "legacy",
+				Endpoint:    "76.76.2.11:53",
+				BootstrapIP: "76.76.2.11",
+				Domain:      "76.76.2.11",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+			},
+		},
+		{
+			"sdns without type",
+			&UpstreamConfig{
+				Name:        "sdns",
+				Endpoint:    "sdns://AAcAAAAAAAAACjc2Ljc2LjIuMTE",
+				BootstrapIP: "",
+				Domain:      "",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+			},
+			&UpstreamConfig{
+				Name:        "sdns",
+				Type:        "legacy",
+				Endpoint:    "76.76.2.11:53",
+				BootstrapIP: "76.76.2.11",
+				Domain:      "76.76.2.11",
+				Timeout:     0,
+				IPStack:     IpStackBoth,
+			},
+		},
+	}
+
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			tc.uc.Init()
+			tc.uc.uid = "" // we don't care about the uid.
+			assert.Equal(t, tc.expected, tc.uc)
+		})
+	}
+}
+
+func TestUpstreamConfig_VerifyDomain(t *testing.T) {
+	tests := []struct {
+		name         string
+		uc           *UpstreamConfig
+		verifyDomain string
+	}{
+		{
+			controlDComDomain,
+			&UpstreamConfig{Endpoint: "https://freedns.controld.com/p2"},
+			controldVerifiedDomain[controlDComDomain],
+		},
+		{
+			controlDDevDomain,
+			&UpstreamConfig{Endpoint: "https://freedns.controld.dev/p2"},
+			controldVerifiedDomain[controlDDevDomain],
+		},
+		{
+			"non-ControlD upstream",
+			&UpstreamConfig{Endpoint: "https://dns.google/dns-query"},
+			"",
+		},
+	}
+
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			if got := tc.uc.VerifyDomain(); got != tc.verifyDomain {
+				t.Errorf("unexpected verify domain, want: %q, got: %q", tc.verifyDomain, got)
+			}
+		})
+	}
+}
+
+func TestUpstreamConfig_UpstreamSendClientInfo(t *testing.T) {
+	tests := []struct {
+		name           string
+		uc             *UpstreamConfig
+		sendClientInfo bool
+	}{
+		{
+			"default with controld upstream DoH",
+			&UpstreamConfig{Endpoint: "https://freedns.controld.com/p2", Type: ResolverTypeDOH},
+			true,
+		},
+		{
+			"default with controld upstream DoH3",
+			&UpstreamConfig{Endpoint: "https://freedns.controld.com/p2", Type: ResolverTypeDOH3},
+			true,
+		},
+		{
+			"default with non-ControlD upstream",
+			&UpstreamConfig{Endpoint: "https://dns.google/dns-query", Type: ResolverTypeDOH},
+			false,
+		},
+		{
+			"set false with controld upstream",
+			&UpstreamConfig{Endpoint: "https://freedns.controld.com/p2", Type: ResolverTypeDOH, SendClientInfo: ptrBool(false)},
+			false,
+		},
+		{
+			"set true with controld upstream",
+			&UpstreamConfig{Endpoint: "https://freedns.controld.com/p2", SendClientInfo: ptrBool(true)},
+			true,
+		},
+		{
+			"set false with non-ControlD upstream",
+			&UpstreamConfig{Endpoint: "https://dns.google/dns-query", SendClientInfo: ptrBool(false)},
+			false,
+		},
+		{
+			"set true with non-ControlD upstream",
+			&UpstreamConfig{Endpoint: "https://dns.google/dns-query", Type: ResolverTypeDOH, SendClientInfo: ptrBool(true)},
+			true,
+		},
+	}
+
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			if got := tc.uc.UpstreamSendClientInfo(); got != tc.sendClientInfo {
+				t.Errorf("unexpected result, want: %v, got: %v", tc.sendClientInfo, got)
+			}
+		})
+	}
+}
+
+func TestUpstreamConfig_IsDiscoverable(t *testing.T) {
+	tests := []struct {
+		name         string
+		uc           *UpstreamConfig
+		discoverable bool
+	}{
+		{
+			"loopback",
+			&UpstreamConfig{Endpoint: "127.0.0.1", Type: ResolverTypeLegacy},
+			true,
+		},
+		{
+			"rfc1918",
+			&UpstreamConfig{Endpoint: "192.168.1.1", Type: ResolverTypeLegacy},
+			true,
+		},
+		{
+			"CGNAT",
+			&UpstreamConfig{Endpoint: "100.66.67.68", Type: ResolverTypeLegacy},
+			true,
+		},
+		{
+			"Public IP",
+			&UpstreamConfig{Endpoint: "8.8.8.8", Type: ResolverTypeLegacy},
+			false,
+		},
+		{
+			"override discoverable",
+			&UpstreamConfig{Endpoint: "127.0.0.1", Type: ResolverTypeLegacy, Discoverable: ptrBool(false)},
+			false,
+		},
+		{
+			"override non-public",
+			&UpstreamConfig{Endpoint: "1.1.1.1", Type: ResolverTypeLegacy, Discoverable: ptrBool(true)},
+			true,
+		},
+		{
+			"non-legacy upstream",
+			&UpstreamConfig{Endpoint: "https://192.168.1.1/custom-doh", Type: ResolverTypeDOH},
+			false,
+		},
+	}
+
+	for _, tc := range tests {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			tc.uc.Init()
+			if got := tc.uc.IsDiscoverable(); got != tc.discoverable {
+				t.Errorf("unexpected result, want: %v, got: %v", tc.discoverable, got)
+			}
+		})
+	}
+}
+
+func ptrBool(b bool) *bool {
+	return &b
+}

config_quic.go

@@ -0,0 +1,160 @@
+package ctrld
+
+import (
+	"context"
+	"crypto/tls"
+	"errors"
+	"net"
+	"net/http"
+	"runtime"
+	"sync"
+
+	"github.com/miekg/dns"
+	"github.com/quic-go/quic-go"
+	"github.com/quic-go/quic-go/http3"
+)
+
+func (uc *UpstreamConfig) setupDOH3Transport() {
+	switch uc.IPStack {
+	case IpStackBoth, "":
+		uc.http3RoundTripper = uc.newDOH3Transport(uc.bootstrapIPs)
+	case IpStackV4:
+		uc.http3RoundTripper = uc.newDOH3Transport(uc.bootstrapIPs4)
+	case IpStackV6:
+		uc.http3RoundTripper = uc.newDOH3Transport(uc.bootstrapIPs6)
+	case IpStackSplit:
+		uc.http3RoundTripper4 = uc.newDOH3Transport(uc.bootstrapIPs4)
+		if hasIPv6() {
+			uc.http3RoundTripper6 = uc.newDOH3Transport(uc.bootstrapIPs6)
+		} else {
+			uc.http3RoundTripper6 = uc.http3RoundTripper4
+		}
+		uc.http3RoundTripper = uc.newDOH3Transport(uc.bootstrapIPs)
+	}
+}
+
+func (uc *UpstreamConfig) newDOH3Transport(addrs []string) http.RoundTripper {
+	rt := &http3.RoundTripper{}
+	rt.TLSClientConfig = &tls.Config{RootCAs: uc.certPool}
+	rt.Dial = func(ctx context.Context, addr string, tlsCfg *tls.Config, cfg *quic.Config) (quic.EarlyConnection, error) {
+		_, port, _ := net.SplitHostPort(addr)
+		// if we have a bootstrap ip set, use it to avoid DNS lookup
+		if uc.BootstrapIP != "" {
+			addr = net.JoinHostPort(uc.BootstrapIP, port)
+			ProxyLogger.Load().Debug().Msgf("sending doh3 request to: %s", addr)
+			udpConn, err := net.ListenUDP("udp", nil)
+			if err != nil {
+				return nil, err
+			}
+			remoteAddr, err := net.ResolveUDPAddr("udp", addr)
+			if err != nil {
+				return nil, err
+			}
+			return quic.DialEarly(ctx, udpConn, remoteAddr, tlsCfg, cfg)
+		}
+		dialAddrs := make([]string, len(addrs))
+		for i := range addrs {
+			dialAddrs[i] = net.JoinHostPort(addrs[i], port)
+		}
+		pd := &quicParallelDialer{}
+		conn, err := pd.Dial(ctx, dialAddrs, tlsCfg, cfg)
+		if err != nil {
+			return nil, err
+		}
+		ProxyLogger.Load().Debug().Msgf("sending doh3 request to: %s", conn.RemoteAddr())
+		return conn, err
+	}
+	runtime.SetFinalizer(rt, func(rt *http3.RoundTripper) {
+		rt.CloseIdleConnections()
+	})
+	return rt
+}
+
+func (uc *UpstreamConfig) doh3Transport(dnsType uint16) http.RoundTripper {
+	uc.transportOnce.Do(func() {
+		uc.SetupTransport()
+	})
+	if uc.rebootstrap.CompareAndSwap(true, false) {
+		uc.SetupTransport()
+	}
+	switch uc.IPStack {
+	case IpStackBoth, IpStackV4, IpStackV6:
+		return uc.http3RoundTripper
+	case IpStackSplit:
+		switch dnsType {
+		case dns.TypeA:
+			return uc.http3RoundTripper4
+		default:
+			return uc.http3RoundTripper6
+		}
+	}
+	return uc.http3RoundTripper
+}
+
+// Putting the code for quic parallel dialer here:
+//
+//   - quic dialer is different with net.Dialer
+//   - simplification for quic free version
+type parallelDialerResult struct {
+	conn quic.EarlyConnection
+	err  error
+}
+
+type quicParallelDialer struct{}
+
+// Dial performs parallel dialing to the given address list.
+func (d *quicParallelDialer) Dial(ctx context.Context, addrs []string, tlsCfg *tls.Config, cfg *quic.Config) (quic.EarlyConnection, error) {
+	if len(addrs) == 0 {
+		return nil, errors.New("empty addresses")
+	}
+	ctx, cancel := context.WithCancel(ctx)
+	defer cancel()
+
+	done := make(chan struct{})
+	defer close(done)
+	ch := make(chan *parallelDialerResult, len(addrs))
+	var wg sync.WaitGroup
+	wg.Add(len(addrs))
+	go func() {
+		wg.Wait()
+		close(ch)
+	}()
+
+	for _, addr := range addrs {
+		go func(addr string) {
+			defer wg.Done()
+			remoteAddr, err := net.ResolveUDPAddr("udp", addr)
+			if err != nil {
+				ch <- &parallelDialerResult{conn: nil, err: err}
+				return
+			}
+			udpConn, err := net.ListenUDP("udp", nil)
+			if err != nil {
+				ch <- &parallelDialerResult{conn: nil, err: err}
+				return
+			}
+			conn, err := quic.DialEarly(ctx, udpConn, remoteAddr, tlsCfg, cfg)
+			select {
+			case ch <- &parallelDialerResult{conn: conn, err: err}:
+			case <-done:
+				if conn != nil {
+					conn.CloseWithError(quic.ApplicationErrorCode(http3.ErrCodeNoError), "")
+				}
+				if udpConn != nil {
+					udpConn.Close()
+				}
+			}
+		}(addr)
+	}
+
+	errs := make([]error, 0, len(addrs))
+	for res := range ch {
+		if res.err == nil {
+			cancel()
+			return res.conn, res.err
+		}
+		errs = append(errs, res.err)
+	}
+
+	return nil, errors.Join(errs...)
+}

config_test.go

@@ -1,7 +1,11 @@
 package ctrld_test
 
 import (
+	"fmt"
+	"os"
+	"strings"
 	"testing"
+	"time"
 
 	"github.com/go-playground/validator/v10"
 	"github.com/spf13/viper"
@@ -19,15 +23,19 @@ func TestLoadConfig(t *testing.T) {
 
 	assert.Equal(t, "info", cfg.Service.LogLevel)
 	assert.Equal(t, "/path/to/log.log", cfg.Service.LogPath)
+	assert.Equal(t, false, *cfg.Service.DnsWatchdogEnabled)
+	assert.Equal(t, time.Duration(20*time.Second), *cfg.Service.DnsWatchdogInvterval)
 
 	assert.Len(t, cfg.Network, 2)
 	assert.Contains(t, cfg.Network, "0")
 	assert.Contains(t, cfg.Network, "1")
 
-	assert.Len(t, cfg.Upstream, 3)
+	assert.Len(t, cfg.Upstream, 4)
 	assert.Contains(t, cfg.Upstream, "0")
 	assert.Contains(t, cfg.Upstream, "1")
 	assert.Contains(t, cfg.Upstream, "2")
+	assert.Contains(t, cfg.Upstream, "3")
+	assert.NotNil(t, cfg.Upstream["3"].SendClientInfo)
 
 	assert.Len(t, cfg.Listener, 2)
 	assert.Contains(t, cfg.Listener, "0")
@@ -42,16 +50,37 @@ func TestLoadConfig(t *testing.T) {
 	assert.Len(t, cfg.Listener["0"].Policy.Rules, 2)
 	assert.Contains(t, cfg.Listener["0"].Policy.Rules[0], "*.ru")
 	assert.Contains(t, cfg.Listener["0"].Policy.Rules[1], "*.local.host")
+
+	assert.True(t, cfg.HasUpstreamSendClientInfo())
 }
 
 func TestLoadDefaultConfig(t *testing.T) {
 	cfg := defaultConfig(t)
 	validate := validator.New()
 	require.NoError(t, ctrld.ValidateConfig(validate, cfg))
-	assert.Len(t, cfg.Listener, 1)
+	if assert.Len(t, cfg.Listener, 1) {
+		l0 := cfg.Listener["0"]
+		require.NotNil(t, l0.Policy)
+		assert.Len(t, l0.Policy.Networks, 1)
+		assert.Len(t, l0.Policy.Rules, 2)
+	}
 	assert.Len(t, cfg.Upstream, 2)
 }
 
+func TestConfigOverride(t *testing.T) {
+	v := viper.NewWithOptions(viper.KeyDelimiter("::"))
+	ctrld.InitConfig(v, "test_load_config")
+	v.SetConfigType("toml")
+	require.NoError(t, v.ReadConfig(strings.NewReader(testhelper.SampleConfigStr(t))))
+	cfg := ctrld.Config{Listener: map[string]*ctrld.ListenerConfig{
+		"0": {IP: "127.0.0.1", Port: 53},
+	}}
+	require.NoError(t, v.Unmarshal(&cfg))
+
+	assert.Equal(t, "10.10.42.69", cfg.Listener["1"].IP)
+	assert.Equal(t, 1337, cfg.Listener["1"].Port)
+}
+
 func TestConfigValidation(t *testing.T) {
 	tests := []struct {
 		name    string
@@ -61,6 +90,7 @@ func TestConfigValidation(t *testing.T) {
 		{"invalid Config", &ctrld.Config{}, true},
 		{"default Config", defaultConfig(t), false},
 		{"sample Config", testhelper.SampleConfig(t), false},
+		{"empty listener IP", emptyListenerIP(t), false},
 		{"invalid cidr", invalidNetworkConfig(t), true},
 		{"invalid upstream type", invalidUpstreamType(t), true},
 		{"invalid upstream timeout", invalidUpstreamTimeout(t), true},
@@ -70,6 +100,17 @@ func TestConfigValidation(t *testing.T) {
 		{"os upstream", configWithOsUpstream(t), false},
 		{"invalid rules", configWithInvalidRules(t), true},
 		{"invalid dns rcodes", configWithInvalidRcodes(t), true},
+		{"invalid max concurrent requests", configWithInvalidMaxConcurrentRequests(t), true},
+		{"non-existed lease file", configWithNonExistedLeaseFile(t), true},
+		{"lease file format required if lease file exist", configWithExistedLeaseFile(t), true},
+		{"invalid lease file format", configWithInvalidLeaseFileFormat(t), true},
+		{"invalid doh/doh3 endpoint", configWithInvalidDoHEndpoint(t), true},
+		{"invalid client id pref", configWithInvalidClientIDPref(t), true},
+		{"doh endpoint without scheme", dohUpstreamEndpointWithoutScheme(t), false},
+		{"doh endpoint without type", dohUpstreamEndpointWithoutType(t), true},
+		{"doh3 endpoint without type", doh3UpstreamEndpointWithoutType(t), false},
+		{"sdns endpoint without type", sdnsUpstreamEndpointWithoutType(t), false},
+		{"maximum number of flush cache domains", configWithInvalidFlushCacheDomain(t), true},
 	}
 
 	for _, tc := range tests {
@@ -89,6 +130,44 @@ func TestConfigValidation(t *testing.T) {
 	}
 }
 
+func TestConfigValidationDoNotChangeEndpoint(t *testing.T) {
+	cfg := configWithInvalidDoHEndpoint(t)
+	endpointMap := map[string]struct{}{}
+	for _, uc := range cfg.Upstream {
+		endpointMap[uc.Endpoint] = struct{}{}
+	}
+	validate := validator.New()
+	_ = ctrld.ValidateConfig(validate, cfg)
+	for _, uc := range cfg.Upstream {
+		if _, ok := endpointMap[uc.Endpoint]; !ok {
+			t.Fatalf("expected endpoint '%s' to exist", uc.Endpoint)
+		}
+	}
+}
+
+func TestConfigDiscoverOverride(t *testing.T) {
+	v := viper.NewWithOptions(viper.KeyDelimiter("::"))
+	ctrld.InitConfig(v, "test_config_discover_override")
+	v.SetConfigType("toml")
+	configStr := `
+[service]
+discover_arp = false
+discover_dhcp = false
+discover_hosts = false
+discover_mdns = false
+discover_ptr = false
+`
+	require.NoError(t, v.ReadConfig(strings.NewReader(configStr)))
+	cfg := ctrld.Config{}
+	require.NoError(t, v.Unmarshal(&cfg))
+
+	require.False(t, *cfg.Service.DiscoverARP)
+	require.False(t, *cfg.Service.DiscoverDHCP)
+	require.False(t, *cfg.Service.DiscoverHosts)
+	require.False(t, *cfg.Service.DiscoverMDNS)
+	require.False(t, *cfg.Service.DiscoverPtr)
+}
+
 func defaultConfig(t *testing.T) *ctrld.Config {
 	v := viper.New()
 	ctrld.InitConfig(v, "test_load_default_config")
@@ -112,6 +191,33 @@ func invalidUpstreamType(t *testing.T) *ctrld.Config {
 	return cfg
 }
 
+func dohUpstreamEndpointWithoutScheme(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	cfg.Upstream["0"].Endpoint = "freedns.controld.com/p1"
+	return cfg
+}
+
+func dohUpstreamEndpointWithoutType(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	cfg.Upstream["0"].Endpoint = "https://freedns.controld.com/p1"
+	cfg.Upstream["0"].Type = ""
+	return cfg
+}
+
+func doh3UpstreamEndpointWithoutType(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	cfg.Upstream["0"].Endpoint = "h3://freedns.controld.com/p1"
+	cfg.Upstream["0"].Type = ""
+	return cfg
+}
+
+func sdnsUpstreamEndpointWithoutType(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	cfg.Upstream["0"].Endpoint = "sdns://AgMAAAAAAAAACjc2Ljc2LjIuMTEAFGZyZWVkbnMuY29udHJvbGQuY29tAy9wMQ"
+	cfg.Upstream["0"].Type = ""
+	return cfg
+}
+
 func invalidUpstreamTimeout(t *testing.T) *ctrld.Config {
 	cfg := defaultConfig(t)
 	cfg.Upstream["0"].Timeout = -1
@@ -130,9 +236,15 @@ func invalidListenerIP(t *testing.T) *ctrld.Config {
 	return cfg
 }
 
+func emptyListenerIP(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	cfg.Listener["0"].IP = ""
+	return cfg
+}
+
 func invalidListenerPort(t *testing.T) *ctrld.Config {
 	cfg := defaultConfig(t)
-	cfg.Listener["0"].Port = 0
+	cfg.Listener["0"].Port = -1
 	return cfg
 }
 
@@ -166,115 +278,53 @@ func configWithInvalidRcodes(t *testing.T) *ctrld.Config {
 	return cfg
 }
 
-func TestUpstreamConfig_Init(t *testing.T) {
-	tests := []struct {
-		name     string
-		uc       *ctrld.UpstreamConfig
-		expected *ctrld.UpstreamConfig
-	}{
-		{
-			"doh+doh3",
-			&ctrld.UpstreamConfig{
-				Name:        "doh",
-				Type:        "doh",
-				Endpoint:    "https://example.com",
-				BootstrapIP: "",
-				Domain:      "",
-				Timeout:     0,
-			},
-			&ctrld.UpstreamConfig{
-				Name:        "doh",
-				Type:        "doh",
-				Endpoint:    "https://example.com",
-				BootstrapIP: "",
-				Domain:      "example.com",
-				Timeout:     0,
-			},
-		},
-		{
-			"dot+doq",
-			&ctrld.UpstreamConfig{
-				Name:        "dot",
-				Type:        "dot",
-				Endpoint:    "freedns.controld.com:8853",
-				BootstrapIP: "",
-				Domain:      "",
-				Timeout:     0,
-			},
-			&ctrld.UpstreamConfig{
-				Name:        "dot",
-				Type:        "dot",
-				Endpoint:    "freedns.controld.com:8853",
-				BootstrapIP: "",
-				Domain:      "freedns.controld.com",
-				Timeout:     0,
-			},
-		},
-		{
-			"dot+doq without port",
-			&ctrld.UpstreamConfig{
-				Name:        "dot",
-				Type:        "dot",
-				Endpoint:    "freedns.controld.com",
-				BootstrapIP: "",
-				Domain:      "",
-				Timeout:     0,
-			},
-			&ctrld.UpstreamConfig{
-				Name:        "dot",
-				Type:        "dot",
-				Endpoint:    "freedns.controld.com:853",
-				BootstrapIP: "",
-				Domain:      "freedns.controld.com",
-				Timeout:     0,
-			},
-		},
-		{
-			"legacy",
-			&ctrld.UpstreamConfig{
-				Name:        "legacy",
-				Type:        "legacy",
-				Endpoint:    "1.2.3.4:53",
-				BootstrapIP: "",
-				Domain:      "",
-				Timeout:     0,
-			},
-			&ctrld.UpstreamConfig{
-				Name:        "legacy",
-				Type:        "legacy",
-				Endpoint:    "1.2.3.4:53",
-				BootstrapIP: "1.2.3.4",
-				Domain:      "1.2.3.4",
-				Timeout:     0,
-			},
-		},
-		{
-			"legacy without port",
-			&ctrld.UpstreamConfig{
-				Name:        "legacy",
-				Type:        "legacy",
-				Endpoint:    "1.2.3.4",
-				BootstrapIP: "",
-				Domain:      "",
-				Timeout:     0,
-			},
-			&ctrld.UpstreamConfig{
-				Name:        "legacy",
-				Type:        "legacy",
-				Endpoint:    "1.2.3.4:53",
-				BootstrapIP: "1.2.3.4",
-				Domain:      "1.2.3.4",
-				Timeout:     0,
-			},
-		},
-	}
-
-	for _, tc := range tests {
-		tc := tc
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-			tc.uc.Init()
-			assert.Equal(t, tc.expected, tc.uc)
-		})
-	}
+func configWithInvalidMaxConcurrentRequests(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	n := -1
+	cfg.Service.MaxConcurrentRequests = &n
+	return cfg
+}
+
+func configWithNonExistedLeaseFile(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	cfg.Service.DHCPLeaseFile = "non-existed"
+	return cfg
+}
+
+func configWithExistedLeaseFile(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	exe, err := os.Executable()
+	if err != nil {
+		t.Fatal(err)
+	}
+	cfg.Service.DHCPLeaseFile = exe
+	return cfg
+}
+
+func configWithInvalidLeaseFileFormat(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	cfg.Service.DHCPLeaseFileFormat = "invalid"
+	return cfg
+}
+
+func configWithInvalidDoHEndpoint(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	cfg.Upstream["0"].Endpoint = "/1.1.1.1"
+	cfg.Upstream["0"].Type = ctrld.ResolverTypeDOH
+	return cfg
+}
+
+func configWithInvalidClientIDPref(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	cfg.Service.ClientIDPref = "foo"
+	return cfg
+}
+
+func configWithInvalidFlushCacheDomain(t *testing.T) *ctrld.Config {
+	cfg := defaultConfig(t)
+	cfg.Service.CacheFlushDomains = make([]string, 257)
+	for i := range cfg.Service.CacheFlushDomains {
+		cfg.Service.CacheFlushDomains[i] = fmt.Sprintf("%d.com", i)
+	}
+	return cfg
 }

docker/Dockerfile

@@ -0,0 +1,32 @@
+# Using Debian bullseye for building regular image.
+# Using scratch image for minimal image size.
+# The final image has:
+#
+# - Timezone info file.
+# - CA certs file.
+# - /etc/{passwd,group} file.
+# - Non-cgo ctrld binary.
+#
+# CI_COMMIT_TAG is used to set the version of ctrld binary.
+FROM golang:1.20-bullseye as base
+
+WORKDIR /app
+
+RUN apt-get update && apt-get install -y upx-ucl
+
+COPY . .
+
+ARG tag=master
+ENV CI_COMMIT_TAG=$tag
+RUN CTRLD_NO_QF=yes CGO_ENABLED=0 ./scripts/build.sh
+
+FROM scratch
+
+COPY --from=base /usr/share/zoneinfo /usr/share/zoneinfo
+COPY --from=base /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
+COPY --from=base /etc/passwd /etc/passwd
+COPY --from=base /etc/group /etc/group
+
+COPY --from=base /app/ctrld-linux-*-nocgo ctrld
+
+ENTRYPOINT ["./ctrld", "run"]

docker/Dockerfile.debug

@@ -0,0 +1,32 @@
+# Using Debian bullseye for building regular image.
+# Using scratch image for minimal image size.
+# The final image has:
+#
+# - Timezone info file.
+# - CA certs file.
+# - /etc/{passwd,group} file.
+# - Non-cgo ctrld binary.
+#
+# CI_COMMIT_TAG is used to set the version of ctrld binary.
+FROM golang:bullseye as base
+
+WORKDIR /app
+
+RUN apt-get update && apt-get install -y upx-ucl
+
+COPY . .
+
+ARG tag=master
+ENV CI_COMMIT_TAG=$tag
+RUN CTRLD_NO_QF=yes CGO_ENABLED=0 ./scripts/build.sh
+
+FROM alpine
+
+COPY --from=base /usr/share/zoneinfo /usr/share/zoneinfo
+COPY --from=base /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
+COPY --from=base /etc/passwd /etc/passwd
+COPY --from=base /etc/group /etc/group
+
+COPY --from=base /app/ctrld-linux-*-nocgo ctrld
+
+ENTRYPOINT ["./ctrld", "run"]

docs/config.md

@@ -14,10 +14,15 @@ The config file allows for advanced configuration of the `ctrld` utility to cove
 
 
 ## Config Location
-`ctrld` uses [TOML](toml_link) format for its configuration file. Default configuration file is `config.toml` found in following order:
+`ctrld` uses [TOML][toml_link] format for its configuration file. Default configuration file is `ctrld.toml` found in following order:
 
- - `$HOME/.ctrld`
- - Current directory
+ - `/etc/controld` on *nix.
+ - User's home directory on Windows.
+ - Same directory with `ctrld` binary on these routers:
+   - `ddwrt`
+   - `merlin`
+   - `freshtomato`
+ - Current directory.
 
 The user can choose to override default value using command line `--config` or `-c`:
 
@@ -38,6 +43,8 @@ if it's existed.
     log_path = ""
     cache_enable = true
     cache_size = 4096
+    cache_ttl_override = 60
+    cache_serve_stale = true
 
 [network.0]
     cidrs = ["0.0.0.0/0"]
@@ -53,6 +60,7 @@ if it's existed.
     name = "Control D - Anti-Malware"
     timeout = 5000
     type = "doh"
+    ip_stack = "both"
 
 [upstream.1]
     bootstrap_ip = "76.76.2.11"
@@ -60,6 +68,7 @@ if it's existed.
     name = "Control D - No Ads"
     timeout = 5000
     type = "doq"
+    ip_stack = "split"
 
 [upstream.2]
     bootstrap_ip = "76.76.2.22"
@@ -67,6 +76,7 @@ if it's existed.
     name = "Control D - Private"
     timeout = 5000
     type = "dot"
+    ip_stack = "v4"
 
 [listener.0]
     ip = "127.0.0.1"
@@ -104,8 +114,8 @@ Logging level you wish to enable.
 
  - Type: string
  - Required: no
- - Valid values: `debug`, `info`, `warn`, `error`, `fatal`, `panic`
- - Default: `info`
+ - Valid values: `debug`, `info`, `warn`, `notice`, `error`, `fatal`, `panic`
+ - Default: `notice`
 
 
 ### log_path
@@ -113,12 +123,14 @@ Relative or absolute path of the log file.
 
 - Type: string
 - Required: no
+- Default: ""
 
 ### cache_enable
 When `cache_enable = true`, all resolved DNS query responses will be cached for duration of the upstream record TTLs.
 
 - Type: boolean
 - Required: no
+- Default: false
 
 ### cache_size
 The number of cached records, must be a positive integer. Tweaking this value with care depends on your available RAM. 
@@ -128,29 +140,153 @@ An invalid `cache_size` value will disable the cache, regardless of `cache_enabl
 
 - Type: int
 - Required: no
+- Default: 4096
 
 ### cache_ttl_override
 When `cache_ttl_override` is set to a positive value (in seconds), TTLs are overridden to this value and cached for this long.
 
 - Type: int
 - Required: no
+- Default: 0
 
 ### cache_serve_stale
 When `cache_serve_stale = true`, in cases of upstream failures (upstreams not reachable), `ctrld` will keep serving
 stale cached records (regardless of their TTLs) until upstream comes online.
 
-The above config will look like this at query time.
+- Type: boolean
+- Required: no
+- Default: false
 
-```
-2022-11-14T22:18:53.808 INF Setting bootstrap IP for upstream.0 bootstrap_ip=76.76.2.11
-2022-11-14T22:18:53.808 INF Starting DNS server on listener.0: 127.0.0.1:53
-2022-11-14T22:18:56.381 DBG [9fd5d3] 127.0.0.1:53978 -> listener.0: 127.0.0.1:53: received query: verify.controld.com
-2022-11-14T22:18:56.381 INF [9fd5d3] no policy, no network, no rule -> [upstream.0]
-2022-11-14T22:18:56.381 DBG [9fd5d3] sending query to upstream.0: Control D - DOH Free
-2022-11-14T22:18:56.381 DBG [9fd5d3] debug dial context freedns.controld.com:443 - tcp - 76.76.2.0
-2022-11-14T22:18:56.381 DBG [9fd5d3] sending doh request to: 76.76.2.11:443
-2022-11-14T22:18:56.420 DBG [9fd5d3] received response of 118 bytes in 39.662597ms
-```
+### cache_flush_domains
+When `ctrld` receives query with domain name in `cache_flush_domains`, the local cache will be discarded
+before serving the query.
+
+- Type: array of strings
+- Required: no
+
+### max_concurrent_requests
+The number of concurrent requests that will be handled, must be a non-negative integer. 
+Tweaking this value depends on the capacity of your system.
+
+- Type: number
+- Required: no
+- Default: 256
+
+### discover_mdns
+Perform LAN client discovery using mDNS. This will spawn a listener on port 5353. 
+
+- Type: boolean
+- Required: no
+- Default: true
+
+### discover_arp
+Perform LAN client discovery using ARP.  
+
+- Type: boolean
+- Required: no
+- Default: true
+
+### discover_dhcp
+Perform LAN client discovery using DHCP leases files. Common file locations are auto-discovered.  
+
+- Type: boolean
+- Required: no
+- Default: true
+
+### discover_ptr
+Perform LAN client discovery using PTR queries.  
+
+- Type: boolean
+- Required: no
+- Default: true
+
+### discover_hosts
+Perform LAN client discovery using hosts file.
+
+- Type: boolean
+- Required: no
+- Default: true
+
+### discover_refresh_interval
+Time in seconds between each discovery refresh loop to update new client information data. 
+The default value is 120 seconds, lower this value to make the discovery process run more aggressively.
+
+- Type: integer
+- Required: no
+- Default: 120
+
+### dhcp_lease_file_path
+Relative or absolute path to a custom DHCP leases file location. 
+
+- Type: string
+- Required: no
+- Default: ""
+
+### dhcp_lease_file_format
+DHCP leases file format. 
+
+- Type: string
+- Required: no
+- Valid values: `dnsmasq`, `isc-dhcp`, `kea-dhcp4`
+- Default: ""
+
+### client_id_preference
+Decide how the client ID is generated. By default client ID will use both MAC address and Hostname i.e. `hash(mac + host)`. To override this behavior, select one of the 2 allowed values to scope client ID to just MAC address OR Hostname.  
+
+- Type: string
+- Required: no
+- Valid values: `mac`, `host`
+- Default: ""
+
+### metrics_query_stats
+If set to `true`, collect and export the query counters, and show them in `clients list` command.
+
+- Type: boolean
+- Required: no
+- Default: false
+
+### metrics_listener
+Specifying the `ip` and `port` of the Prometheus metrics server. The Prometheus metrics will be available on: `http://ip:port/metrics`. You can also append `/metrics/json` to get the same data in json format. 
+
+- Type: string
+- Required: no
+- Default: ""
+
+### dns_watchdog_enabled
+Checking DNS changes to network interfaces and reverting to ctrld's own settings.
+
+The DNS watchdog process only runs on Windows and MacOS.
+
+- Type: boolean
+- Required: no
+- Default: true
+
+### dns_watchdog_interval
+Time duration between each DNS watchdog iteration.
+
+A duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix,
+such as "300ms", "-1.5h" or "2h45m". Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h".
+
+If the time duration is non-positive, default value will be used.
+
+- Type: time duration string
+- Required: no
+- Default: 20s
+
+### refetch_time
+Time in seconds between each iteration that reloads custom config if changed.
+
+The value must be a positive number, any invalid value will be ignored and default value will be used.
+- Type: number
+- Required: no
+- Default: 3600
+
+### leak_on_upstream_failure
+Once ctrld is "offline", mean ctrld could not connect to any upstream, next queries will be leaked to OS resolver.
+
+- Type: boolean
+- Required: no
+- Default: true on Windows, MacOS and non-router Linux.
 
 ## Upstream
 The `[upstream]` section specifies the DNS upstream servers that `ctrld` will forward DNS requests to.
@@ -162,6 +298,7 @@ The `[upstream]` section specifies the DNS upstream servers that `ctrld` will fo
   name = "Control D - DOH"
   timeout = 5000
   type = "doh"
+  ip_stack = "split"
   
 [upstream.1]
   bootstrap_ip = ""
@@ -169,6 +306,7 @@ The `[upstream]` section specifies the DNS upstream servers that `ctrld` will fo
   name = "Control D - DOH3"
   timeout = 5000
   type = "doh3"
+  ip_stack = "both"
   
 [upstream.2]
   bootstrap_ip = ""
@@ -176,6 +314,7 @@ The `[upstream]` section specifies the DNS upstream servers that `ctrld` will fo
   name = "Controld D - DOT"
   timeout = 5000
   type = "dot"
+  ip_stack = "v4"
   
 [upstream.3]
   bootstrap_ip = ""
@@ -183,6 +322,7 @@ The `[upstream]` section specifies the DNS upstream servers that `ctrld` will fo
   name = "Controld D - DOT"
   timeout = 5000
   type = "doq"
+  ip_stack = "v6"
   
 [upstream.4]
   bootstrap_ip = ""
@@ -190,6 +330,7 @@ The `[upstream]` section specifies the DNS upstream servers that `ctrld` will fo
   name = "Control D - Ad Blocking"
   timeout = 5000
   type = "legacy"
+  ip_stack = "both"
 ```
 
 ### bootstrap_ip
@@ -200,6 +341,7 @@ If `bootstrap_ip` is empty, `ctrld` will resolve this itself using its own boots
 
  - type: ip address string
  - required: no
+ - Default: ""
 
 ### endpoint
 IP address, hostname or URL of upstream DNS. Used together with `Type` of the endpoint.
@@ -214,6 +356,7 @@ Human-readable name of the upstream.
 
 - Type: string
 - Required: no
+- Default: ""
 
 ### timeout
 Timeout in milliseconds before request failsover to the next upstream (if defined). 
@@ -221,14 +364,51 @@ Timeout in milliseconds before request failsover to the next upstream (if define
 Value `0` means no timeout.
 
  - Type: number
- - required: no
+ - Required: no
+ - Default: 0
 
 ### type
 The protocol that `ctrld` will use to send DNS requests to upstream.
 
  - Type: string
- - required: yes
- - Valid values: `doh`, `doh3`, `dot`, `doq`, `legacy`, `os`
+ - Required: yes
+ - Valid values: `doh`, `doh3`, `dot`, `doq`, `legacy`
+
+### ip_stack
+Specifying what kind of ip stack that `ctrld` will use to connect to upstream.
+
+ - Type: string
+ - Required: no
+ - Valid values:
+   - `both`: using either ipv4 or ipv6.
+   - `v4`:   only dial upstream via IPv4, never dial IPv6.
+   - `v6`:   only dial upstream via IPv6, never dial IPv4.
+   - `split`:
+     - If `A` record is requested -> dial via ipv4.
+     - If `AAAA` or any other record is requested -> dial ipv6 (if available, otherwise ipv4)
+
+If `ip_stack` is empty, or undefined:
+
+ - Default value is `both` for non-Control D resolvers.
+ - Default value is `split` for Control D resolvers.
+
+### send_client_info
+Specifying whether to include client info when sending query to upstream. **This will only work with `doh` or `doh3` type upstreams.** 
+
+- Type: boolean
+- Required: no
+- Default:
+  - `true` for ControlD upstreams.
+  - `false` for other upstreams.
+
+### discoverable
+Specifying whether the upstream can be used for PTR discovery.
+
+- Type: boolean
+- Required: no
+- Default:
+    - `true` for loopback/RFC1918/CGNAT IP address.
+    - `false` for public IP address.
 
 ## Network
 The `[network]` section defines networks from which DNS queries can originate from. These are used in policies. You can define multiple networks, and each one can have multiple cidrs.
@@ -248,12 +428,14 @@ Name of the network.
 
  - Type: string
  - Required: no
+ - Default: ""
 
 ### cidrs
 Specifies the network addresses that the `listener` will accept requests from. You will see more details in the listener policy section.
 
  - Type: array of network CIDR string
  - Required: no
+ - Default: []
 
 
 ## listener
@@ -271,28 +453,46 @@ The `[listener]` section specifies the ip and port of the local DNS server. You
 ```
 
 ### ip
-IP address that serves the incoming requests.
+IP address that serves the incoming requests. If `ip` is empty, ctrld will listen on all available addresses.
 
-- Type: string
-- Required: yes
+- Type: ip address string
+- Required: no
+- Default: "0.0.0.0" or RFC1918 addess or "127.0.0.1" (depending on platform)
 
 ### port
-Port number that the listener will listen on for incoming requests.
+Port number that the listener will listen on for incoming requests. If `port` is `0`, a random available port will be chosen.
 
 - Type: number
-- Required: yes
+- Required: no
+- Default: 0 or 53 or 5354 (depending on platform)
 
 ### restricted
-If set to `true` makes the listener `REFUSE` DNS queries from all source IP addresses that are not explicitly defined in the policy using a `network`. 
+If set to `true`, makes the listener `REFUSED` DNS queries from all source IP addresses that are not explicitly defined in the policy using a `network`. 
 
 - Type: bool
 - Required: no
+- Default: false
+
+### allow_wan_clients
+The listener will refuse DNS queries from WAN IPs using `REFUSED` RCODE by default. Set to `true` to disable this behavior, but this is not recommended. 
+
+- Type: bool
+- Required: no
+- Default: false
 
 ### policy
 Allows `ctrld` to set policy rules to determine which upstreams the requests will be forwarded to.
 If no `policy` is defined or the requests do not match any policy rules, it will be forwarded to corresponding upstream of the listener. For example, the request to `listener.0` will be forwarded to `upstream.0`.
 
-The policy `rule` syntax is a simple `toml` inline table with exactly one key/value pair per rule. `key` is either the `network` or a domain. Value is the list of the upstreams. For example:
+The policy `rule` syntax is a simple `toml` inline table with exactly one key/value pair per rule. `key` is either:
+
+ - Network.
+ - Domain.
+ - Mac Address.
+
+Value is the list of the upstreams.
+
+For example:
 
 ```toml
 [listener.0.policy]
@@ -306,12 +506,18 @@ rules = [
     {"*.local" = ["upstream.1"]},
     {"test.com" = ["upstream.2", "upstream.1"]},
 ]
+
+macs = [
+    {"14:54:4a:8e:08:2d" = ["upstream.3"]},
+]
 ```
 
 Above policy will:
-- Forward requests on `listener.0` from `network.0` to `upstream.1`.
+
 - Forward requests on `listener.0` for `.local` suffixed domains to `upstream.1`.
 - Forward requests on `listener.0` for `test.com` to `upstream.2`. If timeout is reached, retry on `upstream.1`.
+- Forward requests on `listener.0` from client with Mac `14:54:4a:8e:08:2d` to `upstream.3`.
+- Forward requests on `listener.0` from `network.0` to `upstream.1`.
 - All other requests on `listener.0` that do not match above conditions will be forwarded to `upstream.0`.
 
 An empty upstream would not route the request to any defined upstreams, and use the OS default resolver.
@@ -325,24 +531,60 @@ rules = [
 ]
 ```
 
+---
+
+Note that the order of matching preference:
+
+```
+rules => macs => networks
+```
+
+And within each policy, the rules are processed from top to bottom.
+
+---
+
 #### name
 `name` is the name for the policy.
 
 - Type: string
 - Required: no
+- Default: ""
 
 ### networks:
 `networks` is the list of network rules of the policy.
 
-- type: array of networks
+- Type: array of networks
+- Required: no
+- Default: []
 
 ### rules:
 `rules` is the list of domain rules within the policy. Domain can be either FQDN or wildcard domain.
 
-- type: array of rule
+- Type: array of rule
+- Required: no
+- Default: []
+
+---
+
+Note that the domain comparisons are done in case in-sensitive manner following [RFC 1034](https://datatracker.ietf.org/doc/html/rfc1034#section-3.1)
+
+---
+
+### macs:
+`macs` is the list of mac rules within the policy. Mac address value is case-insensitive.
+
+- Type: array of macs
+- Required: no
+- Default: []
 
 ### failover_rcodes
-For non success response, `failover_rcodes` allows the request to be forwarded to next upstream, if the response `RCODE` matches any value defined in `failover_rcodes`. For example:
+For non success response, `failover_rcodes` allows the request to be forwarded to next upstream, if the response `RCODE` matches any value defined in `failover_rcodes`.
+
+- Type: array of strings
+- Required: no
+- Default: []
+- 
+For example:
 
 ```toml
 [listener.0.policy]
@@ -355,7 +597,7 @@ networks = [
 
 If `upstream.0` returns a NXDOMAIN response, the request will be forwarded to `upstream.1` instead of returning immediately to the client.
 
-See all available DNS Rcodes value [here](rcode_link).
+See all available DNS Rcodes value [here][rcode_link].
 
 [toml_link]: https://toml.io/en
 [rcode_link]: https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-parameters-6

doh.go

@@ -3,53 +3,121 @@ package ctrld
 import (
 	"context"
 	"encoding/base64"
+	"errors"
 	"fmt"
 	"io"
 	"net/http"
+	"net/url"
+	"runtime"
+	"strings"
+	"sync"
+
+	"github.com/cuonglm/osinfo"
 
-	"github.com/lucas-clemente/quic-go/http3"
 	"github.com/miekg/dns"
 )
 
+const (
+	dohMacHeader          = "x-cd-mac"
+	dohIPHeader           = "x-cd-ip"
+	dohHostHeader         = "x-cd-host"
+	dohOsHeader           = "x-cd-os"
+	dohClientIDPrefHeader = "x-cd-cpref"
+	headerApplicationDNS  = "application/dns-message"
+)
+
+// EncodeOsNameMap provides mapping from OS name to a shorter string, used for encoding x-cd-os value.
+var EncodeOsNameMap = map[string]string{
+	"windows": "1",
+	"darwin":  "2",
+	"linux":   "3",
+	"freebsd": "4",
+}
+
+// DecodeOsNameMap provides mapping from encoded OS name to real value, used for decoding x-cd-os value.
+var DecodeOsNameMap = map[string]string{}
+
+// EncodeArchNameMap provides mapping from OS arch to a shorter string, used for encoding x-cd-os value.
+var EncodeArchNameMap = map[string]string{
+	"amd64":  "1",
+	"arm64":  "2",
+	"arm":    "3",
+	"386":    "4",
+	"mips":   "5",
+	"mipsle": "6",
+	"mips64": "7",
+}
+
+// DecodeArchNameMap provides mapping from encoded OS arch to real value, used for decoding x-cd-os value.
+var DecodeArchNameMap = map[string]string{}
+
+func init() {
+	for k, v := range EncodeOsNameMap {
+		DecodeOsNameMap[v] = k
+	}
+	for k, v := range EncodeArchNameMap {
+		DecodeArchNameMap[v] = k
+	}
+}
+
+var dohOsHeaderValue = sync.OnceValue(func() string {
+	oi := osinfo.New()
+	return strings.Join([]string{EncodeOsNameMap[runtime.GOOS], EncodeArchNameMap[runtime.GOARCH], oi.Dist}, "-")
+})()
+
 func newDohResolver(uc *UpstreamConfig) *dohResolver {
 	r := &dohResolver{
-		endpoint:          uc.Endpoint,
+		endpoint:          uc.u,
 		isDoH3:            uc.Type == ResolverTypeDOH3,
-		transport:         uc.transport,
 		http3RoundTripper: uc.http3RoundTripper,
+		uc:                uc,
 	}
 	return r
 }
 
 type dohResolver struct {
-	endpoint          string
+	uc                *UpstreamConfig
+	endpoint          *url.URL
 	isDoH3            bool
-	transport         *http.Transport
-	http3RoundTripper *http3.RoundTripper
+	http3RoundTripper http.RoundTripper
 }
 
+// Resolve performs DNS query with given DNS message using DOH protocol.
 func (r *dohResolver) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, error) {
 	data, err := msg.Pack()
 	if err != nil {
 		return nil, err
 	}
+
 	enc := base64.RawURLEncoding.EncodeToString(data)
-	url := fmt.Sprintf("%s?dns=%s", r.endpoint, enc)
-	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
+	query := r.endpoint.Query()
+	query.Add("dns", enc)
+
+	endpoint := *r.endpoint
+	endpoint.RawQuery = query.Encode()
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint.String(), nil)
 	if err != nil {
 		return nil, fmt.Errorf("could not create request: %w", err)
 	}
-	req.Header.Set("Content-Type", "application/dns-message")
-	req.Header.Set("Accept", "application/dns-message")
-
-	c := http.Client{Transport: r.transport}
+	addHeader(ctx, req, r.uc)
+	dnsTyp := uint16(0)
+	if len(msg.Question) > 0 {
+		dnsTyp = msg.Question[0].Qtype
+	}
+	c := http.Client{Transport: r.uc.dohTransport(dnsTyp)}
 	if r.isDoH3 {
-		c.Transport = r.http3RoundTripper
+		transport := r.uc.doh3Transport(dnsTyp)
+		if transport == nil {
+			return nil, errors.New("DoH3 is not supported")
+		}
+		c.Transport = transport
 	}
 	resp, err := c.Do(req)
 	if err != nil {
 		if r.isDoH3 {
-			r.http3RoundTripper.Close()
+			if closer, ok := c.Transport.(io.Closer); ok {
+				closer.Close()
+			}
 		}
 		return nil, fmt.Errorf("could not perform request: %w", err)
 	}
@@ -65,5 +133,72 @@ func (r *dohResolver) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, erro
 	}
 
 	answer := new(dns.Msg)
-	return answer, answer.Unpack(buf)
+	if err := answer.Unpack(buf); err != nil {
+		return nil, fmt.Errorf("answer.Unpack: %w", err)
+	}
+	return answer, nil
+}
+
+// addHeader adds necessary HTTP header to request based on upstream config.
+func addHeader(ctx context.Context, req *http.Request, uc *UpstreamConfig) {
+	printed := false
+	dohHeader := make(http.Header)
+	if uc.UpstreamSendClientInfo() {
+		if ci, ok := ctx.Value(ClientInfoCtxKey{}).(*ClientInfo); ok && ci != nil {
+			printed = ci.Mac != "" || ci.IP != "" || ci.Hostname != ""
+			switch {
+			case uc.IsControlD():
+				dohHeader = newControlDHeaders(ci)
+			case uc.isNextDNS():
+				dohHeader = newNextDNSHeaders(ci)
+			}
+		}
+	}
+	if printed {
+		Log(ctx, ProxyLogger.Load().Debug(), "sending request header: %v", dohHeader)
+	}
+	dohHeader.Set("Content-Type", headerApplicationDNS)
+	dohHeader.Set("Accept", headerApplicationDNS)
+	req.Header = dohHeader
+}
+
+// newControlDHeaders returns DoH/Doh3 HTTP request headers for ControlD upstream.
+func newControlDHeaders(ci *ClientInfo) http.Header {
+	header := make(http.Header)
+	if ci.Mac != "" {
+		header.Set(dohMacHeader, ci.Mac)
+	}
+	if ci.IP != "" {
+		header.Set(dohIPHeader, ci.IP)
+	}
+	if ci.Hostname != "" {
+		header.Set(dohHostHeader, ci.Hostname)
+	}
+	if ci.Self {
+		header.Set(dohOsHeader, dohOsHeaderValue)
+	}
+	switch ci.ClientIDPref {
+	case "mac":
+		header.Set(dohClientIDPrefHeader, "1")
+	case "host":
+		header.Set(dohClientIDPrefHeader, "2")
+	}
+	return header
+}
+
+// newNextDNSHeaders returns DoH/Doh3 HTTP request headers for nextdns upstream.
+// https://github.com/nextdns/nextdns/blob/v1.41.0/resolver/doh.go#L100
+func newNextDNSHeaders(ci *ClientInfo) http.Header {
+	header := make(http.Header)
+	if ci.Mac != "" {
+		// https: //github.com/nextdns/nextdns/blob/v1.41.0/run.go#L543
+		header.Set("X-Device-Model", "mac:"+ci.Mac[:8])
+	}
+	if ci.IP != "" {
+		header.Set("X-Device-Ip", ci.IP)
+	}
+	if ci.Hostname != "" {
+		header.Set("X-Device-Name", ci.Hostname)
+	}
+	return header
 }

doh_test.go

@@ -0,0 +1,23 @@
+package ctrld
+
+import (
+	"runtime"
+	"testing"
+)
+
+func Test_dohOsHeaderValue(t *testing.T) {
+	val := dohOsHeaderValue
+	if val == "" {
+		t.Fatalf("empty %s", dohOsHeader)
+	}
+	t.Log(val)
+
+	encodedOs := EncodeOsNameMap[runtime.GOOS]
+	if encodedOs == "" {
+		t.Fatalf("missing encoding value for: %q", runtime.GOOS)
+	}
+	decodedOs := DecodeOsNameMap[encodedOs]
+	if decodedOs == "" {
+		t.Fatalf("missing decoding value for: %q", runtime.GOOS)
+	}
+}

doq.go

@@ -1,3 +1,5 @@
+//go:build !qf
+
 package ctrld
 
 import (
@@ -7,8 +9,8 @@ import (
 	"net"
 	"time"
 
-	"github.com/lucas-clemente/quic-go"
 	"github.com/miekg/dns"
+	"github.com/quic-go/quic-go"
 )
 
 type doqResolver struct {
@@ -18,11 +20,17 @@ type doqResolver struct {
 func (r *doqResolver) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, error) {
 	endpoint := r.uc.Endpoint
 	tlsConfig := &tls.Config{NextProtos: []string{"doq"}}
-	if r.uc.BootstrapIP != "" {
-		tlsConfig.ServerName = r.uc.Domain
-		_, port, _ := net.SplitHostPort(endpoint)
-		endpoint = net.JoinHostPort(r.uc.BootstrapIP, port)
+	ip := r.uc.BootstrapIP
+	if ip == "" {
+		dnsTyp := uint16(0)
+		if msg != nil && len(msg.Question) > 0 {
+			dnsTyp = msg.Question[0].Qtype
+		}
+		ip = r.uc.bootstrapIPForDNSType(dnsTyp)
 	}
+	tlsConfig.ServerName = r.uc.Domain
+	_, port, _ := net.SplitHostPort(endpoint)
+	endpoint = net.JoinHostPort(ip, port)
 	return resolve(ctx, msg, endpoint, tlsConfig)
 }
 
@@ -43,7 +51,7 @@ func resolve(ctx context.Context, msg *dns.Msg, endpoint string, tlsConfig *tls.
 }
 
 func doResolve(ctx context.Context, msg *dns.Msg, endpoint string, tlsConfig *tls.Config) (*dns.Msg, error) {
-	session, err := quic.DialAddr(endpoint, tlsConfig, nil)
+	session, err := quic.DialAddr(ctx, endpoint, tlsConfig, nil)
 	if err != nil {
 		return nil, err
 	}

dot.go

@@ -14,18 +14,26 @@ type dotResolver struct {
 
 func (r *dotResolver) Resolve(ctx context.Context, msg *dns.Msg) (*dns.Msg, error) {
 	// The dialer is used to prevent bootstrapping cycle.
-	// If r.endpoing is set to dns.controld.dev, we need to resolve
+	// If r.endpoint is set to dns.controld.dev, we need to resolve
 	// dns.controld.dev first. By using a dialer with custom resolver,
 	// we ensure that we can always resolve the bootstrap domain
 	// regardless of the machine DNS status.
-	dialer := newDialer(net.JoinHostPort(bootstrapDNS, "53"))
+	dialer := newDialer(net.JoinHostPort(controldBootstrapDns, "53"))
+	dnsTyp := uint16(0)
+	if msg != nil && len(msg.Question) > 0 {
+		dnsTyp = msg.Question[0].Qtype
+	}
+
+	tcpNet, _ := r.uc.netForDNSType(dnsTyp)
 	dnsClient := &dns.Client{
-		Net:    "tcp-tls",
-		Dialer: dialer,
+		Net:       tcpNet,
+		Dialer:    dialer,
+		TLSConfig: &tls.Config{RootCAs: r.uc.certPool},
 	}
 	endpoint := r.uc.Endpoint
 	if r.uc.BootstrapIP != "" {
-		dnsClient.TLSConfig = &tls.Config{ServerName: r.uc.Domain}
+		dnsClient.TLSConfig.ServerName = r.uc.Domain
+		dnsClient.Net = "tcp-tls"
 		_, port, _ := net.SplitHostPort(endpoint)
 		endpoint = net.JoinHostPort(r.uc.BootstrapIP, port)
 	}

go.mod

@@ -1,76 +1,102 @@
 module github.com/Control-D-Inc/ctrld
 
-go 1.19
+go 1.23
+
+toolchain go1.23.1
 
 require (
-	github.com/coreos/go-systemd/v22 v22.3.3-0.20220203105225-a9a7ef127534
-	github.com/frankban/quicktest v1.14.3
+	github.com/Masterminds/semver v1.5.0
+	github.com/ameshkov/dnsstamps v1.0.3
+	github.com/coreos/go-systemd/v22 v22.5.0
+	github.com/cuonglm/osinfo v0.0.0-20230921071424-e0e1b1e0bbbf
+	github.com/frankban/quicktest v1.14.6
+	github.com/fsnotify/fsnotify v1.7.0
 	github.com/go-playground/validator/v10 v10.11.1
-	github.com/godbus/dbus/v5 v5.0.6
+	github.com/godbus/dbus/v5 v5.1.1-0.20230522191255-76236955d466
 	github.com/hashicorp/golang-lru/v2 v2.0.1
-	github.com/illarion/gonotify v1.0.1
-	github.com/insomniacslk/dhcp v0.0.0-20211209223715-7d93572ebe8e
+	github.com/illarion/gonotify/v2 v2.0.3
+	github.com/insomniacslk/dhcp v0.0.0-20231206064809-8c70d406f6d2
+	github.com/jaytaylor/go-hostsfile v0.0.0-20220426042432-61485ac1fa6c
+	github.com/josharian/native v1.1.1-0.20230202152459-5c7d0dd6ab86
 	github.com/kardianos/service v1.2.1
-	github.com/lucas-clemente/quic-go v0.29.1
-	github.com/miekg/dns v1.1.50
-	github.com/pelletier/go-toml/v2 v2.0.6
+	github.com/mdlayher/ndp v1.0.1
+	github.com/miekg/dns v1.1.58
+	github.com/minio/selfupdate v0.6.0
+	github.com/olekukonko/tablewriter v0.0.5
+	github.com/pelletier/go-toml/v2 v2.0.8
+	github.com/prometheus/client_golang v1.19.1
+	github.com/prometheus/client_model v0.5.0
+	github.com/prometheus/prom2json v1.3.3
+	github.com/quic-go/quic-go v0.42.0
 	github.com/rs/zerolog v1.28.0
-	github.com/spf13/cobra v1.4.0
-	github.com/spf13/viper v1.14.0
-	github.com/stretchr/testify v1.8.1
-	golang.org/x/sys v0.4.0
+	github.com/spf13/cobra v1.8.1
+	github.com/spf13/pflag v1.0.5
+	github.com/spf13/viper v1.16.0
+	github.com/stretchr/testify v1.9.0
+	github.com/vishvananda/netlink v1.2.1-beta.2
+	golang.org/x/net v0.27.0
+	golang.org/x/sync v0.7.0
+	golang.org/x/sys v0.22.0
 	golang.zx2c4.com/wireguard/windows v0.5.3
-	tailscale.com v1.34.1
+	tailscale.com v1.74.0
 )
 
 require (
-	github.com/alexbrainman/sspi v0.0.0-20210105120005-909beea2cc74 // indirect
-	github.com/davecgh/go-spew v1.1.1 // indirect
-	github.com/fsnotify/fsnotify v1.6.0 // indirect
+	aead.dev/minisign v0.2.0 // indirect
+	github.com/alexbrainman/sspi v0.0.0-20231016080023-1a75b4708caa // indirect
+	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/cespare/xxhash/v2 v2.2.0 // indirect
+	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
+	github.com/dblohm7/wingoes v0.0.0-20240119213807-a09d6be7affa // indirect
+	github.com/go-json-experiment/json v0.0.0-20231102232822-2e55bd4e08b0 // indirect
 	github.com/go-playground/locales v0.14.0 // indirect
 	github.com/go-playground/universal-translator v0.18.0 // indirect
-	github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0 // indirect
-	github.com/golang/mock v1.6.0 // indirect
-	github.com/google/go-cmp v0.5.9 // indirect
+	github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect
+	github.com/golang/protobuf v1.5.4 // indirect
+	github.com/google/go-cmp v0.6.0 // indirect
+	github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd // indirect
 	github.com/hashicorp/hcl v1.0.0 // indirect
-	github.com/inconshreveable/mousetrap v1.0.0 // indirect
-	github.com/josharian/native v1.0.0 // indirect
-	github.com/jsimonetti/rtnetlink v1.1.2-0.20220408201609-d380b505068b // indirect
-	github.com/kr/pretty v0.3.0 // indirect
+	github.com/inconshreveable/mousetrap v1.1.0 // indirect
+	github.com/jsimonetti/rtnetlink v1.4.0 // indirect
+	github.com/kr/pretty v0.3.1 // indirect
 	github.com/kr/text v0.2.0 // indirect
 	github.com/leodido/go-urn v1.2.1 // indirect
 	github.com/magiconair/properties v1.8.7 // indirect
-	github.com/marten-seemann/qpack v0.2.1 // indirect
-	github.com/marten-seemann/qtls-go1-18 v0.1.2 // indirect
-	github.com/marten-seemann/qtls-go1-19 v0.1.0 // indirect
-	github.com/mattn/go-colorable v0.1.12 // indirect
-	github.com/mattn/go-isatty v0.0.14 // indirect
-	github.com/mdlayher/ethernet v0.0.0-20190606142754-0394541c37b7 // indirect
-	github.com/mdlayher/netlink v1.6.0 // indirect
-	github.com/mdlayher/raw v0.0.0-20191009151244-50f2db8cc065 // indirect
-	github.com/mdlayher/socket v0.2.3 // indirect
+	github.com/mattn/go-colorable v0.1.13 // indirect
+	github.com/mattn/go-isatty v0.0.20 // indirect
+	github.com/mattn/go-runewidth v0.0.14 // indirect
+	github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
+	github.com/mdlayher/netlink v1.7.2 // indirect
+	github.com/mdlayher/packet v1.1.2 // indirect
+	github.com/mdlayher/socket v0.5.0 // indirect
 	github.com/mitchellh/mapstructure v1.5.0 // indirect
-	github.com/nxadm/tail v1.4.8 // indirect
-	github.com/onsi/ginkgo v1.16.5 // indirect
-	github.com/pelletier/go-toml v1.9.5 // indirect
-	github.com/pmezard/go-difflib v1.0.0 // indirect
-	github.com/rogpeppe/go-internal v1.8.1-0.20211023094830-115ce09fd6b4 // indirect
-	github.com/spf13/afero v1.9.3 // indirect
-	github.com/spf13/cast v1.5.0 // indirect
+	github.com/onsi/ginkgo/v2 v2.9.5 // indirect
+	github.com/pierrec/lz4/v4 v4.1.21 // indirect
+	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
+	github.com/prometheus/common v0.48.0 // indirect
+	github.com/prometheus/procfs v0.12.0 // indirect
+	github.com/quic-go/qpack v0.4.0 // indirect
+	github.com/rivo/uniseg v0.4.4 // indirect
+	github.com/rogpeppe/go-internal v1.11.0 // indirect
+	github.com/spf13/afero v1.9.5 // indirect
+	github.com/spf13/cast v1.6.0 // indirect
 	github.com/spf13/jwalterweatherman v1.1.0 // indirect
-	github.com/spf13/pflag v1.0.5 // indirect
-	github.com/subosito/gotenv v1.4.1 // indirect
-	github.com/u-root/uio v0.0.0-20220204230159-dac05f7d2cb4 // indirect
-	go4.org/mem v0.0.0-20210711025021-927187094b94 // indirect
-	golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90 // indirect
-	golang.org/x/exp v0.0.0-20220827204233-334a2380cb91 // indirect
-	golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 // indirect
-	golang.org/x/net v0.5.1-0.20230105164244-f8411da775a6 // indirect
-	golang.org/x/sync v0.1.0 // indirect
-	golang.org/x/text v0.6.0 // indirect
-	golang.org/x/tools v0.1.12 // indirect
+	github.com/subosito/gotenv v1.4.2 // indirect
+	github.com/u-root/uio v0.0.0-20240118234441-a3c409a6018e // indirect
+	github.com/vishvananda/netns v0.0.4 // indirect
+	go.uber.org/mock v0.4.0 // indirect
+	go4.org/mem v0.0.0-20220726221520-4f986261bf13 // indirect
+	go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
+	golang.org/x/crypto v0.25.0 // indirect
+	golang.org/x/exp v0.0.0-20240119083558-1b970713d09a // indirect
+	golang.org/x/mod v0.19.0 // indirect
+	golang.org/x/text v0.16.0 // indirect
+	golang.org/x/tools v0.23.0 // indirect
+	google.golang.org/protobuf v1.33.0 // indirect
 	gopkg.in/ini.v1 v1.67.0 // indirect
-	gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
-	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )
+
+replace github.com/mr-karan/doggo => github.com/Windscribe/doggo v0.0.0-20220919152748-2c118fc391f8
+
+replace github.com/rs/zerolog => github.com/Windscribe/zerolog v0.0.0-20230503170159-e6aa153233be

go.sum

@@ -1,3 +1,5 @@
+aead.dev/minisign v0.2.0 h1:kAWrq/hBRu4AARY6AlciO83xhNnW9UaC8YipS2uhLPk=
+aead.dev/minisign v0.2.0/go.mod h1:zdq6LdSd9TbuSxchxwhpA9zEb9YXcVGoE8JakuiGaIQ=
 cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
 cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
 cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
@@ -38,42 +40,57 @@ cloud.google.com/go/storage v1.14.0/go.mod h1:GrKmX003DSIwi9o29oFT7YDnHYwZoctc3f
 dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
 github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
 github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
-github.com/alexbrainman/sspi v0.0.0-20210105120005-909beea2cc74 h1:Kk6a4nehpJ3UuJRqlA3JxYxBZEqCeOmATOvrbT4p9RA=
-github.com/alexbrainman/sspi v0.0.0-20210105120005-909beea2cc74/go.mod h1:cEWa1LVoE5KvSD9ONXsZrj0z6KqySlCCNKHlLzbqAt4=
+github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
+github.com/Masterminds/semver v1.5.0/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF078ddwwvV3Y=
+github.com/Windscribe/zerolog v0.0.0-20230503170159-e6aa153233be h1:qBKVRi7Mom5heOkyZ+NCIu9HZBiNCsRqrRe5t9pooik=
+github.com/Windscribe/zerolog v0.0.0-20230503170159-e6aa153233be/go.mod h1:/tk+P47gFdPXq4QYjvCmT5/Gsug2nagsFWBWhAiSi1w=
+github.com/alexbrainman/sspi v0.0.0-20231016080023-1a75b4708caa h1:LHTHcTQiSGT7VVbI0o4wBRNQIgn917usHWOd6VAffYI=
+github.com/alexbrainman/sspi v0.0.0-20231016080023-1a75b4708caa/go.mod h1:cEWa1LVoE5KvSD9ONXsZrj0z6KqySlCCNKHlLzbqAt4=
+github.com/ameshkov/dnsstamps v1.0.3 h1:Srzik+J9mivH1alRACTbys2xOxs0lRH9qnTA7Y1OYVo=
+github.com/ameshkov/dnsstamps v1.0.3/go.mod h1:Ii3eUu73dx4Vw5O4wjzmT5+lkCwovjzaEZZ4gKyIH5A=
+github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
+github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
 github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
+github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
+github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
 github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
 github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
 github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
-github.com/cilium/ebpf v0.8.1 h1:bLSSEbBLqGPXxls55pGr5qWZaTqcmfDJHhou7t254ao=
-github.com/cilium/ebpf v0.8.1/go.mod h1:f5zLIM0FSNuAkSyLAN7X+Hy6yznlF1mNiWUMfxMtrgk=
+github.com/cilium/ebpf v0.15.0 h1:7NxJhNiBT3NG8pZJ3c+yfrVdHY8ScgKD27sScgjLMMk=
+github.com/cilium/ebpf v0.15.0/go.mod h1:DHp1WyrLeiBh19Cf/tfiSMhqheEiK8fXFZ4No0P1Hso=
 github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
 github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
 github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
 github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
-github.com/coreos/go-systemd/v22 v22.3.3-0.20220203105225-a9a7ef127534 h1:rtAn27wIbmOGUs7RIbVgPEjb31ehTVniDwPGXyMxm5U=
-github.com/coreos/go-systemd/v22 v22.3.3-0.20220203105225-a9a7ef127534/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
-github.com/cpuguy83/go-md2man/v2 v2.0.1/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
+github.com/coreos/go-systemd/v22 v22.5.0 h1:RrqgGjYQKalulkV8NGVIfkXQf6YYmOyiJKk8iXXhfZs=
+github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
+github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
+github.com/cuonglm/osinfo v0.0.0-20230921071424-e0e1b1e0bbbf h1:40DHYsri+d1bnroFDU2FQAeq68f3kAlOzlQ93kCf26Q=
+github.com/cuonglm/osinfo v0.0.0-20230921071424-e0e1b1e0bbbf/go.mod h1:G45410zMgmnSjLVKCq4f6GpbYAzoP2plX9rPwgx6C24=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
+github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/dblohm7/wingoes v0.0.0-20240119213807-a09d6be7affa h1:h8TfIT1xc8FWbwwpmHn1J5i43Y0uZP97GqasGCzSRJk=
+github.com/dblohm7/wingoes v0.0.0-20240119213807-a09d6be7affa/go.mod h1:Nx87SkVqTKd8UtT+xu7sM/l+LgXs6c0aHrlKusR+2EQ=
 github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
 github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
 github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
 github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=
 github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=
 github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
-github.com/fanliao/go-promise v0.0.0-20141029170127-1890db352a72/go.mod h1:PjfxuH4FZdUyfMdtBio2lsRr1AKEaVPwelzuHuh8Lqc=
-github.com/frankban/quicktest v1.14.0/go.mod h1:NeW+ay9A/U67EYXNFA1nPE8e/tnQv/09mUdL/ijj8og=
-github.com/frankban/quicktest v1.14.3 h1:FJKSZTDHjyhriyC81FLQ0LY93eSai0ZyR/ZIkd3ZUKE=
-github.com/frankban/quicktest v1.14.3/go.mod h1:mgiwOwqx65TmIk1wJ6Q7wvnVMocbUorkibMOrVTHZps=
-github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
-github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
-github.com/fsnotify/fsnotify v1.6.0 h1:n+5WquG0fcWoWp6xPWfHdbskMCQaFnG6PfBrh1Ky4HY=
-github.com/fsnotify/fsnotify v1.6.0/go.mod h1:sl3t1tCWJFWoRz9R8WJCbQihKKwmorjAbSClcnxKAGw=
+github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
+github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
+github.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=
+github.com/fsnotify/fsnotify v1.7.0/go.mod h1:40Bi/Hjc2AVfZrqy+aj+yEI+/bRxZnMJyTJwOpGvigM=
 github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
 github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
 github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
+github.com/go-json-experiment/json v0.0.0-20231102232822-2e55bd4e08b0 h1:ymLjT4f35nQbASLnvxEde4XOBL+Sn7rFuV+FOJqkljg=
+github.com/go-json-experiment/json v0.0.0-20231102232822-2e55bd4e08b0/go.mod h1:6daplAwHHGbUGib4990V3Il26O0OC4aRyvewaaAihaA=
+github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
+github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
 github.com/go-playground/assert/v2 v2.0.1 h1:MsBgLAaY856+nPRTKrp3/OZK38U/wa0CcBYNjji3q3A=
 github.com/go-playground/assert/v2 v2.0.1/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
 github.com/go-playground/locales v0.14.0 h1:u50s323jtVGugKlcYeyzC0etD1HifMjqmJqb8WugfUU=
@@ -82,11 +99,11 @@ github.com/go-playground/universal-translator v0.18.0 h1:82dyy6p4OuJq4/CByFNOn/j
 github.com/go-playground/universal-translator v0.18.0/go.mod h1:UvRDBj+xPUEGrFYl+lu/H90nyDXpg0fqeB/AQUGNTVA=
 github.com/go-playground/validator/v10 v10.11.1 h1:prmOlTVv+YjZjmRmNSF3VmspqJIxJWXmqUsHwfTRRkQ=
 github.com/go-playground/validator/v10 v10.11.1/go.mod h1:i+3WkQ1FvaUjjxh1kSvIA4dMGDBiPU55YFDl0WbKdWU=
-github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0 h1:p104kn46Q8WdvHunIJ9dAyjPVtrBPhSr3KT2yUst43I=
-github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE=
+github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI=
+github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls=
 github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
-github.com/godbus/dbus/v5 v5.0.6 h1:mkgN1ofwASrYnJ5W6U/BxG15eXXXjirgZc7CLqkcaro=
-github.com/godbus/dbus/v5 v5.0.6/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
+github.com/godbus/dbus/v5 v5.1.1-0.20230522191255-76236955d466 h1:sQspH8M4niEijh3PFscJRLDnkL547IeP7kpPe3uUhEg=
+github.com/godbus/dbus/v5 v5.1.1-0.20230522191255-76236955d466/go.mod h1:ZiQxhyQ+bbbfxUKVvjfO498oPYvtYhZzycal3G/NHmU=
 github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
 github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
 github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
@@ -98,8 +115,6 @@ github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt
 github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
 github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
 github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
-github.com/golang/mock v1.6.0 h1:ErTB+efbowRARo13NNdxyJji2egdxLGQhRaY+DUumQc=
-github.com/golang/mock v1.6.0/go.mod h1:p6yTPP+5HYm5mzsMV8JkE6ZKdX+/wYM6Hr+LicevLPs=
 github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
 github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
 github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
@@ -114,7 +129,8 @@ github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvq
 github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
 github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
 github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
-github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=
+github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
+github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
 github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
 github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
 github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
@@ -126,10 +142,9 @@ github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
 github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE=
-github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
 github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
+github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
 github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
 github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
 github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
@@ -143,134 +158,141 @@ github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hf
 github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/pprof v0.0.0-20201218002935-b9804c9f04c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
+github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo=
+github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw=
 github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
 github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
 github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
 github.com/googleapis/google-cloud-go-testing v0.0.0-20200911160855-bcd43fbb19e8/go.mod h1:dvDLG8qkwmyD9a/MJJN3XJcT3xFxOKAvTZGvuZmac9g=
-github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
 github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
 github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
 github.com/hashicorp/golang-lru/v2 v2.0.1 h1:5pv5N1lT1fjLg2VQ5KWc7kmucp2x/kvFOnxuVTqZ6x4=
 github.com/hashicorp/golang-lru/v2 v2.0.1/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
 github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
 github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
-github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
 github.com/hugelgupf/socketpair v0.0.0-20190730060125-05d35a94e714 h1:/jC7qQFrv8CrSJVmaolDVOxTfS9kc36uB6H40kdbQq8=
 github.com/hugelgupf/socketpair v0.0.0-20190730060125-05d35a94e714/go.mod h1:2Goc3h8EklBH5mspfHFxBnEoURQCGzQQH1ga9Myjvis=
 github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
 github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
-github.com/illarion/gonotify v1.0.1 h1:F1d+0Fgbq/sDWjj/r66ekjDG+IDeecQKUFH4wNwsoio=
-github.com/illarion/gonotify v1.0.1/go.mod h1:zt5pmDofZpU1f8aqlK0+95eQhoEAn/d4G4B/FjVW4jE=
-github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
-github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
-github.com/insomniacslk/dhcp v0.0.0-20211209223715-7d93572ebe8e h1:IQpunlq7T+NiJJMO7ODYV2YWBiv/KnObR3gofX0mWOo=
-github.com/insomniacslk/dhcp v0.0.0-20211209223715-7d93572ebe8e/go.mod h1:h+MxyHxRg9NH3terB1nfRIUaQEcI0XOVkdR9LNBlp8E=
-github.com/josharian/native v1.0.0 h1:Ts/E8zCSEsG17dUqv7joXJFybuMLjQfWE04tsBODTxk=
-github.com/josharian/native v1.0.0/go.mod h1:7X/raswPFr05uY3HiLlYeyQntB6OO7E/d2Cu7qoaN2w=
-github.com/jsimonetti/rtnetlink v0.0.0-20190606172950-9527aa82566a/go.mod h1:Oz+70psSo5OFh8DBl0Zv2ACw7Esh6pPUphlvZG9x7uw=
-github.com/jsimonetti/rtnetlink v0.0.0-20200117123717-f846d4f6c1f4/go.mod h1:WGuG/smIU4J/54PblvSbh+xvCZmpJnFgr3ds6Z55XMQ=
-github.com/jsimonetti/rtnetlink v0.0.0-20201009170750-9c6f07d100c1/go.mod h1:hqoO/u39cqLeBLebZ8fWdE96O7FxrAsRYhnVOdgHxok=
-github.com/jsimonetti/rtnetlink v0.0.0-20201110080708-d2c240429e6c/go.mod h1:huN4d1phzjhlOsNIjFsw2SVRbwIHj3fJDMEU2SDPTmg=
-github.com/jsimonetti/rtnetlink v1.1.2-0.20220408201609-d380b505068b h1:Yws7RV6kZr2O7PPdT+RkbSmmOponA8i/1DuGHe8BRsM=
-github.com/jsimonetti/rtnetlink v1.1.2-0.20220408201609-d380b505068b/go.mod h1:TzDCVOZKUa79z6iXbbXqhtAflVgUKaFkZ21M5tK5tzY=
+github.com/illarion/gonotify/v2 v2.0.3 h1:B6+SKPo/0Sw8cRJh1aLzNEeNVFfzE3c6N+o+vyxM+9A=
+github.com/illarion/gonotify/v2 v2.0.3/go.mod h1:38oIJTgFqupkEydkkClkbL6i5lXV/bxdH9do5TALPEE=
+github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
+github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
+github.com/insomniacslk/dhcp v0.0.0-20231206064809-8c70d406f6d2 h1:9K06NfxkBh25x56yVhWWlKFE8YpicaSfHwoV8SFbueA=
+github.com/insomniacslk/dhcp v0.0.0-20231206064809-8c70d406f6d2/go.mod h1:3A9PQ1cunSDF/1rbTq99Ts4pVnycWg+vlPkfeD2NLFI=
+github.com/jaytaylor/go-hostsfile v0.0.0-20220426042432-61485ac1fa6c h1:kbTQ8oGf+BVFvt/fM+ECI+NbZDCqoi0vtZTfB2p2hrI=
+github.com/jaytaylor/go-hostsfile v0.0.0-20220426042432-61485ac1fa6c/go.mod h1:k6+89xKz7BSMJ+DzIerBdtpEUeTlBMugO/hcVSzahog=
+github.com/josharian/native v1.0.1-0.20221213033349-c1e37c09b531/go.mod h1:7X/raswPFr05uY3HiLlYeyQntB6OO7E/d2Cu7qoaN2w=
+github.com/josharian/native v1.1.1-0.20230202152459-5c7d0dd6ab86 h1:elKwZS1OcdQ0WwEDBeqxKwb7WB62QX8bvZ/FJnVXIfk=
+github.com/josharian/native v1.1.1-0.20230202152459-5c7d0dd6ab86/go.mod h1:aFAMtuldEgx/4q7iSGazk22+IcgvtiC+HIimFO9XlS8=
+github.com/jsimonetti/rtnetlink v1.4.0 h1:Z1BF0fRgcETPEa0Kt0MRk3yV5+kF1FWTni6KUFKrq2I=
+github.com/jsimonetti/rtnetlink v1.4.0/go.mod h1:5W1jDvWdnthFJ7fxYX1GMK07BUpI4oskfOqvPteYS6E=
 github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
 github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
-github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
 github.com/kardianos/service v1.2.1 h1:AYndMsehS+ywIS6RB9KOlcXzteWUzxgMgBymJD7+BYk=
 github.com/kardianos/service v1.2.1/go.mod h1:CIMRFEJVL+0DS1a3Nx06NaMn4Dz63Ng6O7dl0qH0zVM=
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
 github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
 github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
 github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
-github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
 github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
+github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
+github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
 github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
 github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
 github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/leodido/go-urn v1.2.1 h1:BqpAaACuzVSgi/VLzGZIobT2z4v53pjosyNd9Yv6n/w=
 github.com/leodido/go-urn v1.2.1/go.mod h1:zt4jvISO2HfUBqxjfIshjdMTYS56ZS/qv49ictyFfxY=
-github.com/lucas-clemente/quic-go v0.29.1 h1:Z+WMJ++qMLhvpFkRZA+jl3BTxUjm415YBmWanXB8zP0=
-github.com/lucas-clemente/quic-go v0.29.1/go.mod h1:CTcNfLYJS2UuRNB+zcNlgvkjBhxX6Hm3WUxxAQx2mgE=
 github.com/magiconair/properties v1.8.7 h1:IeQXZAiQcpL9mgcAe1Nu6cX9LLw6ExEHKjN0VQdvPDY=
 github.com/magiconair/properties v1.8.7/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
-github.com/marten-seemann/qpack v0.2.1 h1:jvTsT/HpCn2UZJdP+UUB53FfUUgeOyG5K1ns0OJOGVs=
-github.com/marten-seemann/qpack v0.2.1/go.mod h1:F7Gl5L1jIgN1D11ucXefiuJS9UMVP2opoCp2jDKb7wc=
-github.com/marten-seemann/qtls-go1-18 v0.1.2 h1:JH6jmzbduz0ITVQ7ShevK10Av5+jBEKAHMntXmIV7kM=
-github.com/marten-seemann/qtls-go1-18 v0.1.2/go.mod h1:mJttiymBAByA49mhlNZZGrH5u1uXYZJ+RW28Py7f4m4=
-github.com/marten-seemann/qtls-go1-19 v0.1.0 h1:rLFKD/9mp/uq1SYGYuVZhm83wkmU95pK5df3GufyYYU=
-github.com/marten-seemann/qtls-go1-19 v0.1.0/go.mod h1:5HTDWtVudo/WFsHKRNuOhWlbdjrfs5JHrYb0wIJqGpI=
-github.com/mattn/go-colorable v0.1.12 h1:jF+Du6AlPIjs2BiUiQlKOX0rt3SujHxPnksPKZbaA40=
 github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
-github.com/mattn/go-isatty v0.0.14 h1:yVuAays6BHfxijgZPzw+3Zlu5yQgKGP2/hcQbHb7S9Y=
+github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
+github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
 github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
-github.com/mdlayher/ethernet v0.0.0-20190606142754-0394541c37b7 h1:lez6TS6aAau+8wXUP3G9I3TGlmPFEq2CTxBaRqY6AGE=
-github.com/mdlayher/ethernet v0.0.0-20190606142754-0394541c37b7/go.mod h1:U6ZQobyTjI/tJyq2HG+i/dfSoFUt8/aZCM+GKtmFk/Y=
-github.com/mdlayher/netlink v0.0.0-20190409211403-11939a169225/go.mod h1:eQB3mZE4aiYnlUsyGGCOpPETfdQq4Jhsgf1fk3cwQaA=
-github.com/mdlayher/netlink v1.0.0/go.mod h1:KxeJAFOFLG6AjpyDkQ/iIhxygIUKD+vcwqcnu43w/+M=
-github.com/mdlayher/netlink v1.1.0/go.mod h1:H4WCitaheIsdF9yOYu8CFmCgQthAPIWZmcKp9uZHgmY=
-github.com/mdlayher/netlink v1.1.1/go.mod h1:WTYpFb/WTvlRJAyKhZL5/uy69TDDpHHu2VZmb2XgV7o=
-github.com/mdlayher/netlink v1.6.0 h1:rOHX5yl7qnlpiVkFWoqccueppMtXzeziFjWAjLg6sz0=
-github.com/mdlayher/netlink v1.6.0/go.mod h1:0o3PlBmGst1xve7wQ7j/hwpNaFaH4qCRyWCdcZk8/vA=
-github.com/mdlayher/raw v0.0.0-20190606142536-fef19f00fc18/go.mod h1:7EpbotpCmVZcu+KCX4g9WaRNuu11uyhiW7+Le1dKawg=
-github.com/mdlayher/raw v0.0.0-20191009151244-50f2db8cc065 h1:aFkJ6lx4FPip+S+Uw4aTegFMct9shDvP+79PsSxpm3w=
-github.com/mdlayher/raw v0.0.0-20191009151244-50f2db8cc065/go.mod h1:7EpbotpCmVZcu+KCX4g9WaRNuu11uyhiW7+Le1dKawg=
-github.com/mdlayher/socket v0.1.1/go.mod h1:mYV5YIZAfHh4dzDVzI8x8tWLWCliuX8Mon5Awbj+qDs=
-github.com/mdlayher/socket v0.2.3 h1:XZA2X2TjdOwNoNPVPclRCURoX/hokBY8nkTmRZFEheM=
-github.com/mdlayher/socket v0.2.3/go.mod h1:bz12/FozYNH/VbvC3q7TRIK/Y6dH1kCKsXaUeXi/FmY=
-github.com/miekg/dns v1.1.50 h1:DQUfb9uc6smULcREF09Uc+/Gd46YWqJd5DbpPE9xkcA=
-github.com/miekg/dns v1.1.50/go.mod h1:e3IlAVfNqAllflbibAZEWOXOQ+Ynzk/dDozDxY7XnME=
+github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
+github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
+github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
+github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
+github.com/mattn/go-runewidth v0.0.14 h1:+xnbZSEeDbOIg5/mE6JF0w6n9duR1l3/WmbinWVwUuU=
+github.com/mattn/go-runewidth v0.0.14/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
+github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo=
+github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
+github.com/mdlayher/ndp v1.0.1 h1:+yAD79/BWyFlvAoeG5ncPS0ItlHP/eVbH7bQ6/+LVA4=
+github.com/mdlayher/ndp v1.0.1/go.mod h1:rf3wKaWhAYJEXFKpgF8kQ2AxypxVbfNcZbqoAo6fVzk=
+github.com/mdlayher/netlink v1.7.2 h1:/UtM3ofJap7Vl4QWCPDGXY8d3GIY2UGSDbK+QWmY8/g=
+github.com/mdlayher/netlink v1.7.2/go.mod h1:xraEF7uJbxLhc5fpHL4cPe221LI2bdttWlU+ZGLfQSw=
+github.com/mdlayher/packet v1.1.2 h1:3Up1NG6LZrsgDVn6X4L9Ge/iyRyxFEFD9o6Pr3Q1nQY=
+github.com/mdlayher/packet v1.1.2/go.mod h1:GEu1+n9sG5VtiRE4SydOmX5GTwyyYlteZiFU+x0kew4=
+github.com/mdlayher/socket v0.5.0 h1:ilICZmJcQz70vrWVes1MFera4jGiWNocSkykwwoy3XI=
+github.com/mdlayher/socket v0.5.0/go.mod h1:WkcBFfvyG8QENs5+hfQPl1X6Jpd2yeLIYgrGFmJiJxI=
+github.com/miekg/dns v1.1.58 h1:ca2Hdkz+cDg/7eNF6V56jjzuZ4aCAE+DbVkILdQWG/4=
+github.com/miekg/dns v1.1.58/go.mod h1:Ypv+3b/KadlvW9vJfXOTf300O4UqaHFzFCuHz+rPkBY=
+github.com/minio/selfupdate v0.6.0 h1:i76PgT0K5xO9+hjzKcacQtO7+MjJ4JKA8Ak8XQ9DDwU=
+github.com/minio/selfupdate v0.6.0/go.mod h1:bO02GTIPCMQFTEvE5h4DjYB58bCoZ35XLeBf0buTDdM=
 github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
 github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
-github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
-github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE=
-github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=
-github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
-github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=
-github.com/onsi/ginkgo v1.14.0/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY=
-github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=
-github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU=
-github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=
-github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
-github.com/onsi/gomega v1.20.1 h1:PA/3qinGoukvymdIDV8pii6tiZgC8kbmJO6Z5+b002Q=
-github.com/pelletier/go-toml v1.9.5 h1:4yBQzkHv+7BHq2PQUZF3Mx0IYxG7LsP222s7Agd3ve8=
-github.com/pelletier/go-toml v1.9.5/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=
-github.com/pelletier/go-toml/v2 v2.0.6 h1:nrzqCb7j9cDFj2coyLNLaZuJTLjWjlaz6nvTvIwycIU=
-github.com/pelletier/go-toml/v2 v2.0.6/go.mod h1:eumQOmlWiOPt5WriQQqoM5y18pDHwha2N+QD+EUNTek=
+github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec=
+github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY=
+github.com/onsi/ginkgo/v2 v2.9.5 h1:+6Hr4uxzP4XIUyAkg61dWBw8lb/gc4/X5luuxN/EC+Q=
+github.com/onsi/ginkgo/v2 v2.9.5/go.mod h1:tvAoo1QUJwNEU2ITftXTpR7R1RbCzoZUOs3RonqW57k=
+github.com/onsi/gomega v1.27.6 h1:ENqfyGeS5AX/rlXDd/ETokDz93u0YufY1Pgxuy/PvWE=
+github.com/onsi/gomega v1.27.6/go.mod h1:PIQNjfQwkP3aQAH7lf7j87O/5FiNr+ZR8+ipb+qQlhg=
+github.com/pelletier/go-toml/v2 v2.0.8 h1:0ctb6s9mE31h0/lhu+J6OPmVeDxJn+kYnJc2jZR9tGQ=
+github.com/pelletier/go-toml/v2 v2.0.8/go.mod h1:vuYfssBdrU2XDZ9bYydBu6t+6a6PYNcZljzZR9VXg+4=
+github.com/pierrec/lz4/v4 v4.1.14/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
+github.com/pierrec/lz4/v4 v4.1.21 h1:yOVMLb6qSIDP67pl/5F7RepeKYu/VmTyEXvuMI5d9mQ=
+github.com/pierrec/lz4/v4 v4.1.21/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
 github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pkg/sftp v1.13.1/go.mod h1:3HaPG6Dq1ILlpPZRO0HVMrsydcdLt6HRDccSgb87qRg=
-github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
+github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/prometheus/client_golang v1.19.1 h1:wZWJDwK+NameRJuPGDhlnFgx8e8HN3XHQeLaYJFJBOE=
+github.com/prometheus/client_golang v1.19.1/go.mod h1:mP78NwGzrVks5S2H6ab8+ZZGJLZUq1hoULYBAYBw1Ho=
 github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/prometheus/client_model v0.5.0 h1:VQw1hfvPvk3Uv6Qf29VrPF32JB6rtbgI6cYPYQjL0Qw=
+github.com/prometheus/client_model v0.5.0/go.mod h1:dTiFglRmd66nLR9Pv9f0mZi7B7fk5Pm3gvsjB5tr+kI=
+github.com/prometheus/common v0.48.0 h1:QO8U2CdOzSn1BBsmXJXduaaW+dY/5QLjfB8svtSzKKE=
+github.com/prometheus/common v0.48.0/go.mod h1:0/KsvlIEfPQCQ5I2iNSAWKPZziNCvRs5EC6ILDTlAPc=
+github.com/prometheus/procfs v0.12.0 h1:jluTpSng7V9hY0O2R9DzzJHYb2xULk9VTR1V1R/k6Bo=
+github.com/prometheus/procfs v0.12.0/go.mod h1:pcuDEFsWDnvcgNzo4EEweacyhjeA9Zk3cnaOZAZEfOo=
+github.com/prometheus/prom2json v1.3.3 h1:IYfSMiZ7sSOfliBoo89PcufjWO4eAR0gznGcETyaUgo=
+github.com/prometheus/prom2json v1.3.3/go.mod h1:Pv4yIPktEkK7btWsrUTWDDDrnpUrAELaOCj+oFwlgmc=
+github.com/quic-go/qpack v0.4.0 h1:Cr9BXA1sQS2SmDUWjSofMPNKmvF6IiIfDRmgU0w1ZCo=
+github.com/quic-go/qpack v0.4.0/go.mod h1:UZVnYIfi5GRk+zI9UMaCPsmZ2xKJP7XBUvVyT1Knj9A=
+github.com/quic-go/quic-go v0.42.0 h1:uSfdap0eveIl8KXnipv9K7nlwZ5IqLlYOpJ58u5utpM=
+github.com/quic-go/quic-go v0.42.0/go.mod h1:132kz4kL3F9vxhW3CtQJLDVwcFe5wdWeJXXijhsO57M=
+github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
+github.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis=
+github.com/rivo/uniseg v0.4.4/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
 github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
 github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
 github.com/rogpeppe/go-internal v1.8.0/go.mod h1:WmiCO8CzOY8rg0OYDC4/i/2WRWAB6poM+XZ2dLUbcbE=
-github.com/rogpeppe/go-internal v1.8.1-0.20211023094830-115ce09fd6b4 h1:Ha8xCaq6ln1a+R91Km45Oq6lPXj2Mla6CRJYcuV2h1w=
-github.com/rogpeppe/go-internal v1.8.1-0.20211023094830-115ce09fd6b4/go.mod h1:JeRgkft04UBgHMgCIwADu4Pn6Mtm5d4nPKWu0nJ5d+o=
-github.com/rs/xid v1.4.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
-github.com/rs/zerolog v1.28.0 h1:MirSo27VyNi7RJYP3078AA1+Cyzd2GB66qy3aUHvsWY=
-github.com/rs/zerolog v1.28.0/go.mod h1:NILgTygv/Uej1ra5XxGf82ZFSLk58MFGAUS2o6usyD0=
+github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
+github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDNXVBh4M=
+github.com/rogpeppe/go-internal v1.11.0/go.mod h1:ddIwULY96R17DhadqLgMfk9H9tvdUzkipdSkR5nkCZA=
+github.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
 github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
-github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
-github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
-github.com/spf13/afero v1.9.3 h1:41FoI0fD7OR7mGcKE/aOiLkGreyf8ifIOQmJANWogMk=
-github.com/spf13/afero v1.9.3/go.mod h1:iUV7ddyEEZPO5gA3zD4fJt6iStLlL+Lg4m2cihcDf8Y=
-github.com/spf13/cast v1.5.0 h1:rj3WzYc11XZaIZMPKmwP96zkFEnnAmV8s6XbB2aY32w=
-github.com/spf13/cast v1.5.0/go.mod h1:SpXXQ5YoyJw6s3/6cMTQuxvgRl3PCJiyaX9p6b155UU=
-github.com/spf13/cobra v1.4.0 h1:y+wJpx64xcgO1V+RcnwW0LEHxTKRi2ZDPSBjWnrg88Q=
-github.com/spf13/cobra v1.4.0/go.mod h1:Wo4iy3BUC+X2Fybo0PDqwJIv3dNRiZLHQymsfxlB84g=
+github.com/spf13/afero v1.9.5 h1:stMpOSZFs//0Lv29HduCmli3GUfpFoF3Y1Q/aXj/wVM=
+github.com/spf13/afero v1.9.5/go.mod h1:UBogFpq8E9Hx+xc5CNTTEpTnuHVmXDwZcZcE1eb/UhQ=
+github.com/spf13/cast v1.6.0 h1:GEiTHELF+vaR5dhz3VqZfFSzZjYbgeKDpBxQVS4GYJ0=
+github.com/spf13/cast v1.6.0/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
+github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM=
+github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y=
 github.com/spf13/jwalterweatherman v1.1.0 h1:ue6voC5bR5F8YxI5S67j9i582FU4Qvo2bmqnqMYADFk=
 github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=
 github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
 github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
-github.com/spf13/viper v1.14.0 h1:Rg7d3Lo706X9tHsJMUjdiwMpHB7W8WnSVOssIY+JElU=
-github.com/spf13/viper v1.14.0/go.mod h1:WT//axPky3FdvXHzGw33dNdXXXfFQqmEalje+egj8As=
+github.com/spf13/viper v1.16.0 h1:rGGH0XDZhdUOryiDWjmIvUSWpbNqisK8Wk0Vyefw8hc=
+github.com/spf13/viper v1.16.0/go.mod h1:yg78JgCJcbrQOvV9YLXgkLaZqUidkY9K+Dd1FofRzQg=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
-github.com/stretchr/objx v0.5.0 h1:1zr/of2m5FGMsad5YfcqgdqdWrIhu+EBEJRhR1U7z/c=
 github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
+github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
+github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
 github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
 github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
 github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
@@ -278,36 +300,46 @@ github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
 github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
-github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
-github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
-github.com/subosito/gotenv v1.4.1 h1:jyEFiXpy21Wm81FBN71l9VoMMV8H8jG+qIK3GCpY6Qs=
-github.com/subosito/gotenv v1.4.1/go.mod h1:ayKnFf/c6rvx/2iiLrJUk1e6plDbT3edrFNGqEflhK0=
-github.com/u-root/uio v0.0.0-20210528114334-82958018845c/go.mod h1:LpEX5FO/cB+WF4TYGY1V5qktpaZLkKkSegbr0V4eYXA=
-github.com/u-root/uio v0.0.0-20220204230159-dac05f7d2cb4 h1:hl6sK6aFgTLISijk6xIzeqnPzQcsLqqvL6vEfTPinME=
-github.com/u-root/uio v0.0.0-20220204230159-dac05f7d2cb4/go.mod h1:LpEX5FO/cB+WF4TYGY1V5qktpaZLkKkSegbr0V4eYXA=
+github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
+github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
+github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/subosito/gotenv v1.4.2 h1:X1TuBLAMDFbaTAChgCBLu3DU3UPyELpnF2jjJ2cz/S8=
+github.com/subosito/gotenv v1.4.2/go.mod h1:ayKnFf/c6rvx/2iiLrJUk1e6plDbT3edrFNGqEflhK0=
+github.com/u-root/uio v0.0.0-20240118234441-a3c409a6018e h1:BA9O3BmlTmpjbvajAwzWx4Wo2TRVdpPXZEeemGQcajw=
+github.com/u-root/uio v0.0.0-20240118234441-a3c409a6018e/go.mod h1:eLL9Nub3yfAho7qB0MzZizFhTU2QkLeoVsWdHtDW264=
+github.com/vishvananda/netlink v1.2.1-beta.2 h1:Llsql0lnQEbHj0I1OuKyp8otXp0r3q0mPkuhwHfStVs=
+github.com/vishvananda/netlink v1.2.1-beta.2/go.mod h1:twkDnbuQxJYemMlGd4JFIcuhgX83tXhKS2B/PRMpOho=
+github.com/vishvananda/netns v0.0.0-20200728191858-db3c7e526aae/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0=
+github.com/vishvananda/netns v0.0.4 h1:Oeaw1EM2JMxD51g9uhtC0D7erkIjgmj8+JZc26m1YX8=
+github.com/vishvananda/netns v0.0.4/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
 github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
-github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
 go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
 go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
 go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
 go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
 go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
 go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
-go4.org/mem v0.0.0-20210711025021-927187094b94 h1:OAAkygi2Js191AJP1Ds42MhJRgeofeKGjuoUqNp1QC4=
-go4.org/mem v0.0.0-20210711025021-927187094b94/go.mod h1:reUoABIJ9ikfM5sgtSF3Wushcza7+WeD01VB9Lirh3g=
+go.uber.org/mock v0.4.0 h1:VcM4ZOtdbR4f6VXfiOpwpVJDL6lCReaZ6mw31wqh7KU=
+go.uber.org/mock v0.4.0/go.mod h1:a6FSlNadKUHUa9IP5Vyt1zh4fC7uAwxMutEAscFbkZc=
+go4.org/mem v0.0.0-20220726221520-4f986261bf13 h1:CbZeCBZ0aZj8EfVgnqQcYZgf0lpZ3H9rmp5nkDTAst8=
+go4.org/mem v0.0.0-20220726221520-4f986261bf13/go.mod h1:reUoABIJ9ikfM5sgtSF3Wushcza7+WeD01VB9Lirh3g=
+go4.org/netipx v0.0.0-20231129151722-fdeea329fbba h1:0b9z3AuHCjxk0x/opv64kcgZLBseWJUpBw5I82+2U4M=
+go4.org/netipx v0.0.0-20231129151722-fdeea329fbba/go.mod h1:PLyyIXexvUFg3Owu6p/WfdlivPbZJsZdgWZlrGope/Y=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
 golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
-golang.org/x/crypto v0.0.0-20211108221036-ceb1ce70b4fa/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
+golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
 golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
-golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90 h1:Y/gsMcFOcR+6S6f3YeMKl5g+dZMEWqcz5Czj/GWYbkM=
-golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
+golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
+golang.org/x/crypto v0.25.0 h1:ypSNr+bnYL2YhwoMt2zPxHFmbAN1KZs/njMG3hxUp30=
+golang.org/x/crypto v0.25.0/go.mod h1:T+wALwcMOSE0kXgUAnPAHqTLW+XHgcELELW8VaDgm/M=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -318,8 +350,8 @@ golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u0
 golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
 golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
 golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
-golang.org/x/exp v0.0.0-20220827204233-334a2380cb91 h1:tnebWN09GYg9OLPss1KXj8txwZc6X6uMr6VFdcGNbHw=
-golang.org/x/exp v0.0.0-20220827204233-334a2380cb91/go.mod h1:cyybsKvd6eL0RnXn6p/Grxp8F5bW7iYuBgsNCOHpMYE=
+golang.org/x/exp v0.0.0-20240119083558-1b970713d09a h1:Q8/wZp0KX97QFTc2ywcOE0YRjZPVIx+MXInMzdvQqcA=
+golang.org/x/exp v0.0.0-20240119083558-1b970713d09a/go.mod h1:idGWGoKP1toJGkd5/ig9ZLuPcZBC3ewk7SzmH0uou08=
 golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
 golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
 golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
@@ -343,25 +375,20 @@ golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
-golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
+golang.org/x/mod v0.19.0 h1:fEdghXQSo20giMthA7cd28ZC+jts4amQ3YMXiP5oMQ8=
+golang.org/x/mod v0.19.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
 golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.0.0-20190419010253-1f3472d942ba/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20191007182048-72f939374954/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
@@ -372,25 +399,18 @@ golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/
 golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
 golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
 golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
-golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
 golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
 golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
 golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
 golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
-golang.org/x/net v0.0.0-20201010224723-4f7140c49acb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
 golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
 golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
-golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
 golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
-golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
-golang.org/x/net v0.0.0-20210726213435-c6fcb2dbf985/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
-golang.org/x/net v0.0.0-20210928044308-7d9f5e0b762b/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
 golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
-golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
-golang.org/x/net v0.5.1-0.20230105164244-f8411da775a6 h1:pKt/LWZC6+FwNujj5E7DdVyWcbtQvKqPuN0GPKWMyB8=
-golang.org/x/net v0.5.1-0.20230105164244-f8411da775a6/go.mod h1:DivGGAXEgPSlEBzxGzZI+ZLohi+xUj054jfeKui00ws=
+golang.org/x/net v0.27.0 h1:5K3Njcw06/l2y9vpGCSdcxWOYHOUk3dVNGDXN+FvAys=
+golang.org/x/net v0.27.0/go.mod h1:dDi0PyhWNoiUOrAS8uXv/vnScO4wnHQO4mj9fn/RytE=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -410,34 +430,26 @@ golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJ
 golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o=
-golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
+golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
 golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190411185658-b44545bcd369/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190418153312-f0ce4c0180be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190606122018-79a91cf218c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200217220822-9197077df867/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -445,37 +457,33 @@ golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200519105757-fe76b779f299/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200728102440-3e129f6d46b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20201009025420-dfb3f7c4e634/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201015000850-e3ed0017c211/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20201101102859-da207088b7d1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210225134936-a50acf3fe073/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210228012217-479acdf4ea46/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210525143221-35b2ab0089ea/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210906170528-6f6e22806c34/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220128215802-99c3d69c2c27/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.4.0 h1:Zr2JFtRQNX3BCZ8YtxRE9hNJYC8J6I1MVbMg6owUp18=
-golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220622161953-175b2fd9d664/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220817070843-5a390386f1f2/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.4.1-0.20230131160137-e7d7f63158de/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI=
+golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
-golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -484,18 +492,19 @@ golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
-golang.org/x/text v0.6.0 h1:3XmdazWV+ubf7QgHSTWeykHOci5oeekaGJBLkrkaw4k=
-golang.org/x/text v0.6.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
+golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4=
+golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
 golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
+golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
 golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
 golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
 golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
 golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
 golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
 golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
 golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
 golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
@@ -534,14 +543,11 @@ golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82u
 golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
-golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
-golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
-golang.org/x/tools v0.1.6-0.20210726203631-07bc1bf47fb2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
-golang.org/x/tools v0.1.12 h1:VveCTK38A2rkS8ZqFY25HIDFscX5X9OoEhJd3quQmXU=
-golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
+golang.org/x/tools v0.23.0 h1:SGsXPZ+2l4JsgaCKkx+FQ9YZ5XEtA1GZYuoDjenLjvg=
+golang.org/x/tools v0.23.0/go.mod h1:pnu6ufv6vQkll6szChhK3C3L/ruaIv5eBeztNG8wtsI=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -636,22 +642,16 @@ google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2
 google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
 google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
 google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
-google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w=
+google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
+google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
 gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
-gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
 gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
 gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
-gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
-gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
 gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
-gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
@@ -666,5 +666,5 @@ honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9
 rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
 rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
 rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
-tailscale.com v1.34.1 h1:tqm9Ww4ltyYp3IPe7vCGch6tT6j5G/WXPQ6BrVZ6pdI=
-tailscale.com v1.34.1/go.mod h1:ZsBP7rjzzB2rp+UCOumr9DAe0EQ6OPivwSXcz/BrekQ=
+tailscale.com v1.74.0 h1:J+vRN9o3D4wCqZBiwvDg9kZpQag2mG4Xz5RXNpmV3KE=
+tailscale.com v1.74.0/go.mod h1:3iACpCONQ4lauDXvwfoGlwNCpfbVxjdc2j6G9EuFOW8=

internal/certs/root_ca.go

@@ -0,0 +1,22 @@
+package certs
+
+import (
+	"crypto/x509"
+	_ "embed"
+	"sync"
+)
+
+var (
+	//go:embed cacert.pem
+	caRoots        []byte
+	caCertPoolOnce sync.Once
+	caCertPool     *x509.CertPool
+)
+
+func CACertPool() *x509.CertPool {
+	caCertPoolOnce.Do(func() {
+		caCertPool = x509.NewCertPool()
+		caCertPool.AppendCertsFromPEM(caRoots)
+	})
+	return caCertPool
+}