A backoff with small max time will flood requests to Control D server,
causing false positive for abuse mitiation system. While a big max time
will cause ctrld not realize network change as fast as possible.
While at it, also sync DoH3 code with DoH code, ensuring no others place
can trigger requests flooding for ipv6 probing.
Otherwise, network changes may not be seen on some platforms, causing
ctrld failed to recover and failing all requests.
While at it, also doing the check DNS in separate goroutine, prevent it
from blocking ctrld from notifying others that it "started". The issue
was seen when ctrld is configured as direct listener, requests are
flooded before ctrld started, causing the healtch process failed.
The provision token is only used once, then do not have any effect after
Control D uid is fetched. So making it appears in "ctrld run" command is
useless.
VPN clients often have empty MAC address, because they come from virtual
network interface. However, there's other setup/devices also create
virtual interface, but is not VPN.
Changing source of those clients to empty to prevent confustion in
clients list command output.
Some users mentioned that when there is an Internet outage, ctrld fails
to recover, crashing or locks up the router. When requests start
failing, this results in the clients emitting more queries, creating a
resource spiral of death that can brick the device entirely.
To guard against this case, this commit implement an upstream monitor
approach:
- Marking upstream as down after 100 consecutive failed queries.
- Start a goroutine to check when the upstream is back again.
- When upstream is down, answer all queries with SERVFAIL.
- The checking process uses backoff retry to reduce high requests rate.
- As long as the query succeeded, marking the upstream as alive then
start operate normally.
Currently, ctrld assumes that NetworkManager is not available if writing
to /etc/NetworkManager/conf.d return directory not exist error. That
would work on most Linux distros. However, cloud provider may do some
hacks, causing ctrld confusion and think that NetworkManager is
available.
Fixing this by checking whether NetworkManager binary presents first.
While at it, also fixing a bug when restarting NetworkManager failed
causing ctrld hangs. The go-systemd library is not clear about this, but
the waitCh channel won't never be closed if error occurred, so we must
return immediately instead of receiving from it blindly.
In case the resolver could not reach nameserver, ptr discover should
only print error message once, then stop doing the query until the
nameserver is reachable. This would prevent ptr discover from flooding
ctrld log with a lot of duplicated messages.
So ctrld can record the raw/original client IP instead of looking up
from MAC to IP, which may not the right choice in some network setup
like using wireguard/vpn on Merlin router.
The current approach to get default route IP is finding the LAN
interface with the same MAC address. However, there could be multiple
interfaces like that, making ctrld confused.
This commit fixes this issue, by listing all possible private IPs, then
sorting them and use the smallest one for router self queries.
For reporting router queries, ctrld uses private IP of the default route
interface. However, when the default route is conntected directly to
ISP, the interface will have a public IP, and another interface with the
same MAC address will be created for LAN ip. So when no private IP found
for default route interface, ctrld must look at the other interface to
find the corret LAN ip.
The only reason that forces ctrld to depend on vyatta-dhcpd service on
EdgeOS is allowing ctrld to watch lease files properly, because those
files may not be created at the time client info table initialized.
However, on some EdgeOS version, vyatta-dhcpd could not start with an
empty config file, causing restart loop itself, flooding systemd log,
making the router run out of memory.
To fix this, instead of depending on vyatta-dhcpd, we should just watch
for lease files creation, then adding them to watch list.
While at it, also making ctrld starts after nss-lookup, ensuring we have
a working DNS before starting ctrld.
Using the same approach as in cd mode, but do it only once when running
ctrld the first time, then the config will be re-used then.
While at it, also adding Dockerfile.debug for better troubleshooting
with alpine base image.
- Watch more events for lease file changes
- Improving network up detection by using bootstrap IPv6 along side
IPv4 one.
- Emitting log to notice user that ctrld is starting.
- Using systemd wrapper to provide correct status.
- Restoring DNS on stop on Windows.
On routers where we want to wait for NTP by checking nvram key. Before
waiting, we clean up the router to ensure it's restored to original
state. However, router.Cleanup is not idempotent, causing dnsmasq
restarted. On tomato/ddwrt, restarting have no delay, and spawning new
dnsmasq process immediately. On merlin, somehow it takes time to spawn
new dnsmasq process, causing ctrld wrongly think there's no one
listening on port 53.
Fixing this by ensuring router.Cleanup is idempotent. While at it, also
adding "ntp_done" to nvram key, which is now using on latest ddwrt.
For local config, we don't want to alter what user explicitly set, and
only try filling in missing value.
While at it, also remove the dnsmasq port delete on openwrt, we don't
need that hack anymore.
Config fetching/generating in cd mode is currently weird, error prone,
and easy for user to break ctrld when using custom config.
This commit reworks the flow:
- Fetching config from Control D API.
- No custom config, use the current default config.
- If custom config presents, but there's no listener, use 0.0.0.0:53.
- Try listening on current ip+port config, if ok, ctrld could be a
direct listener with current setup, moving on.
- If failed, trying 127.0.0.1:53.
- If failed, trying current ip + port 5354
- If still failed, pick a random ip:port pair, retry until listening ok.
With this flow, thing is more predictable/stable, and help removing the
Config interface for router.
dohTransport returns a http.RoundTripper. When pinging upstream, we do
it both for doh and doh3, and checking whether the transport is nil
before performing the check.
However, dohTransport returns a concrete *http.Transport. Thus
dohTransport will always return a non-nil http.Roundtripper, causing
invalid memory dereference when upstream is configured to use doh3.
Performing ping upstream separately will fix the issue.
Depending on system, the output of `/etc/init.d/ctrld status` can be
either "Running" or "running", we must do in-sensitive comparison to get
the right status of ctrld.
The current error message is not much helpful, not all users are able to
investigate system log file to find the reason.
Instead, gathering the log output of "ctrld run" command, and if error
happens or self-check failed, print the log to users.
Cuong Manh Le