diff --git a/Troubleshooting-Guide.md b/Troubleshooting-Guide.md new file mode 100644 index 0000000..1d1fc44 --- /dev/null +++ b/Troubleshooting-Guide.md @@ -0,0 +1,156 @@ +# Potential Problems +Under normal circumstances and if there are no conflicts with other software, you should not experience any issues. That being said, nothing is perfect so if you encounter any issues with the ctrld utility (it stops resolving DNS queries), here are some useful things to watch out for, and provide to support when opening an issue. + +Below assumes you're using a Linux-like operating system. + +## Check if ctrld is running +First, let's check to see if `ctrld` is still running. + +``` +ps aux | grep ctrld +``` + +Alternatively, if you get an error from the above command, especially on routers you may need to run: +``` +ps | grep ctrld +``` + +If you don't see a running ctrld process, that would be an issue. Try restarting using the `ctrld start` command. + +## Check listeners +if the process is running, let's dig deeper. `ctrld` will spawn one or more DNS listeners in order to receive DNS queries. Let's check what they are: + +``` +netstat -tupln | grep ctrld +``` + +You may need to prefix the above command with `sudo` if you're not using a root user. The output may look something like this + +``` +root@UDMPro:/tmp# netstat -tupln | grep ctrld +tcp6 0 0 :::5354 :::* LISTEN 287225/ctrld +udp 0 0 0.0.0.0:5353 0.0.0.0:* 287225/ctrld +udp6 0 0 :::5354 :::* 287225/ctrld +``` + +You can ignore the listeners running on UDP 5353, as that's used for mDNS client discovery. The other instances on port 5354 (yours may be different) are the DNS listeners we're interested in. + +If you don't see any listeners, or nothing except 5353, that suggests you may have a configuration problem as no DNS listener was spawned. This is very unlikely, as the service wouldn't have started in the first place. You should check your `ctrld.toml` config file, it's typically located in `/etc/controld/` directory but can be different on some routers (Asus Merlin, DD-WRT, Tomato) and see what listener port is defined. + +## Check service +If `ctrld` is not listening on port 53, that means it's an upstream for another DNS server, usually dnsmasq. To rule out `ctrld` as the cause of the problem, let's make a DNS query directly against it, bypassing dnsmasq. + +If you have `dig` installed, run: + +``` +dig verify.controld.com @127.0.0.1 -p5354 +``` + +If all is well, you should get the following response + +``` +root@UDMPro:/tmp# dig verify.controld.com @127.0.0.1 -p5354 + +; <<>> DiG 9.16.42-Debian <<>> verify.controld.com @127.0.0.1 -p5354 +;; global options: +cmd +;; Got answer: +;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40286 +;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 + +;; QUESTION SECTION: +;verify.controld.com. IN A + +;; ANSWER SECTION: +verify.controld.com. 20 IN CNAME api.controld.com. +api.controld.com. 24 IN A 147.185.34.1 + +;; Query time: 0 msec +;; SERVER: 127.0.0.1#5354(127.0.0.1) +;; WHEN: Fri Sep 08 19:36:41 EDT 2023 +;; MSG SIZE rcvd: 71 +``` + +If you don't have `dig` installed, you can try using `nslookup` + +``` +root@UDMPro:~# nslookup -port=5354 verify.controld.com 127.0.0.1 +Server: 127.0.0.1 +Address: 127.0.0.1#5354 + +Non-authoritative answer: +verify.controld.com canonical name = api.controld.com. +Name: api.controld.com +Address: 147.185.34.1 +Name: api.controld.com +Address: 2606:1a40:3::1 +``` + +If you're getting the above responses, then `ctrld` is working correctly, and the issue lies elsewhere (read on!). If you do not get the above response while using a Control D DNS resolver, then something is wrong here, which can include: + +1. Lack of Internet connectivity +2. Service encountered some unexpected state + +If you have Internet access (can ping external IPs), then you should restart the `ctrld` service using the `ctrld restart` command and see if it comes back to life and DNS starts working. + +## Check resolv.conf +If the above step was successful, yet DNS is still broken, you should check your `/etc/resolv.conf` file. Under normal circumstances, it will look like this: + +``` +root@UDMPro:~# cat /etc/resolv.conf +# resolv.conf(5) file generated by ctrld +# DO NOT EDIT THIS FILE BY HAND -- CHANGES WILL BE OVERWRITTEN + +nameserver 127.0.0.1 +root@UDMPro:~# +``` + +If your resolv.conf file has no mentions of `ctrld`, that means another process rendered this file. This can be other DNS related software, so you should uninstall whatever you have running that may be trying to do the same things as `ctrld`. If it looks like the above, let's continue. + +The above assumes 127.0.0.1:53, which means your DNS queries are being sent to whichever process is listening on this IP + port. In most cases it will be dnsmasq, unless your `ctrld` instances runs on port 53 directly, but since we already tested it in the earlier step this is likely not the case for you. Let's confirm this. + +``` +root@UDMPro:~# netstat -tupln | grep ":53\b" +tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN 1315695/dnsmasq +tcp 0 0 192.168.1.1:53 0.0.0.0:* LISTEN 1315695/dnsmasq +tcp 0 0 127.0.2.1:53 0.0.0.0:* LISTEN 1/init +tcp6 0 0 ::1:53 :::* LISTEN 1315695/dnsmasq +tcp6 0 0 fe80::6022:32ff:fea1:53 :::* LISTEN 1315695/dnsmasq +udp 0 0 127.0.0.1:53 0.0.0.0:* 1315695/dnsmasq +udp 0 0 192.168.1.1:53 0.0.0.0:* 1315695/dnsmasq +udp 0 0 127.0.2.1:53 0.0.0.0:* 1/init +udp6 0 0 ::1:53 :::* 1315695/dnsmasq +udp6 0 0 fe80::6022:32ff:fea1:53 :::* 1315695/dnsmasq +root@UDMPro:~# +``` + +The above indeed suggests dnsmasq is running on port 53, on all IPs. If your earlier dig/nslookup commands worked against the IP + port that `ctrld` listens on, but do NOT work against port 53, the issues lies in dnsmasq, and it not sending DNS queries to `ctrld`. + +The cause for this can be other DNS related software manipulating dnsmasq config file, misconfiguration on your part, or a `ctrld` bug. Yeah, this is not very helpful, but you should find your dnsmasq config file and see if the `server` param is set correctly, pointing to your `ctrld` listener IP + port. If all is well, it will look something like this, but your path may be different. + +``` +root@UDMPro:~# cat /run/dnsmasq.conf.d/zzzctrld.conf +# GENERATED BY ctrld - DO NOT MODIFY +no-resolv +server=127.0.0.1#5354 +add-mac +root@UDMPro:~# +``` + +Other paths to look in, depending on your hardware. + +``` +/etc/dhcpd/dhcpd-zzz-ctrld.conf +/etc/dnsmasq.d/dnsmasq-zzz-ctrld.conf +/tmp/dnsmasq.d/ctrld.conf +``` + +If this config doesn't exist, or has a different IP + port, that's a problem. Try restarting the `ctrld` service and see if the issue goes away. + +## Debug launch +If the service doesn't start, or hangs, try running it in debug mode and collect a log. This can be done using the following command: + +``` +ctrld start --log /tmp/ctrld.log -vv +``` + +The log file should contain some useful data. Open an issue with this data, and everything you collected above. \ No newline at end of file