linuxexpert-practice-labs/labs/lfcs/2-3-network-troubleshooting/README.md

LFCS 2.3: Network troubleshooting

Finding a wrong gateway, a wrong DNS server and a service listening on the wrong address, capturing DNS traffic, and finding which process owns a port.

Before you start

The peer, a second machine at 10.99.0.2, is the one to fix in tasks 1 and 2; this VM, 10.99.0.1, is its gateway. Get a shell there with sudo ip netns exec lepeer bash. The peer has no route to the internet: from it, 198.51.100.9 stands in for an internet address such as 9.9.9.9, and 198.51.100.53 is the network's DNS server, both beyond its gateway. The peer reads its network settings at boot from /etc/netns/lepeer/network (ip -batch commands) and /etc/netns/lepeer/resolv.conf, so fix those as well as the running system. The lab has also set up a few things on this VM.

Tasks

1. The peer can't reach 198.51.100.9. Find out why from the symptoms alone, and fix it.
2. On the peer, show that addresses work but names don't, find out why, and fix it, so that getent hosts example.org works there.
3. The VM's web server (unit le-lab-web) can't be reached from the peer at http://10.99.0.1/. Show from the peer why it can't connect, and fix it.
4. Capture the DNS queries the VM sends while you run getent hosts example.org, save them to /root/lab-answers/4, and read the file back. On Ubuntu, run sudo resolvectl flush-caches first, or the answer may come from the cache.
5. Find which process is listening on UDP port 8125, and on which addresses. Write its name and the addresses to /root/lab-answers/5.

Hints

• Work up the layers: ip -br addr, ip route, ip neigh, ping, ip route get.
• getent hosts versus dig @198.51.100.53 (or ping by address).
• ss -tlnp on the VM, curl -v from the peer; systemctl cat and systemctl edit for the unit.
• tcpdump -n -i any port 53 -w FILE, then tcpdump -n -r FILE. tcpdump may switch to its own user before it writes the file: add -Z root, or write to /tmp and copy it.
• ss -ulnp.