Sign in Get started

Networking Tutorial · Routing and Network Services · Lesson 21 of 28

Network Troubleshooting

A step-by-step method and the commands (dig, ping, traceroute, nc, curl, ss, tcpdump) to find why a connection fails.

Create a free account to save this page, take notes and track your progress. Sign in

Work up the layers

When "the API is unreachable", check one layer at a time, from the client's side:

Step Question Command
1. DNS Does the name resolve, to the address you expect? dig +short api.example.com
2. Route Is there a path to the host? traceroute / mtr, ip route get <ip>
3. Port Does a TCP connection open? nc -vz api.example.com 443
4. TLS Is the certificate valid for this name? openssl s_client -connect host:443 -servername host
5. HTTP What does the server answer? curl -v https://api.example.com/healthz
6. Server Is the app listening, on which address? ss -tlnp

Many cloud networks block ICMP, so a failed ping alone proves nothing. Test the real port with nc or curl.

Reading the symptoms

Symptom Usually means
Could not resolve host DNS: wrong name, missing record, wrong resolver
Connection refused (fast) Host reached, but nothing listens on that port (or a firewall actively rejects)
Connection times out (slow) Packets silently dropped: security group, NACL, route table or NAT
Connection reset A proxy, firewall or the app closed the connection: idle timeouts, TLS mismatch
TLS error Certificate expired, wrong name, missing intermediate
502 / 504 The proxy reached you but your app failed or was slow
Check one layer at a timeTry it yourself
def diagnose(resolves, route, port_open, tls_ok, http_status):
    checks = [
        ("DNS", resolves, "fix the record or resolver"),
        ("route", route, "check route tables, peering and NAT"),
        ("TCP port", port_open, "timeout -> security group/NACL; refused -> app not listening"),
        ("TLS", tls_ok, "check certificate name, expiry and chain"),
        ("HTTP", http_status < 500, "read the app and load balancer logs"),
    ]
    for layer, ok, hint in checks:
        print(f"{layer:8} {'ok' if ok else 'FAIL'}")
        if not ok:
            return f"stop here: {hint}"
    return "network path is fine"

print(diagnose(resolves=True, route=True, port_open=False, tls_ok=True, http_status=200))

Output

DNS      ok
route    ok
TCP port FAIL
stop here: timeout -> security group/NACL; refused -> app not listening

Seeing packets

tcpdump -n -i any port 443 shows packets as they arrive and leave. In the cloud, VPC Flow Logs (AWS and Google Cloud) and NSG / VNet flow logs (Azure) record accepted and rejected connections, and AWS Reachability Analyzer, Azure Network Watcher and Google Cloud Connectivity Tests explain why a path is blocked without sending traffic.

Test yourself

curl to a private API hangs for 30 seconds and then times out. Which is the most likely cause?

  1. A security group, network ACL or route silently drops the packets
  2. Nothing is listening on the port
  3. The DNS record is missing
  4. The TLS certificate has expired
Show answer

Answer: A security group, network ACL or route silently drops the packets

Silent drops cause timeouts. A host with nothing listening answers immediately with 'connection refused'.

Was this page helpful?