Networking interview question · Most asked
Why can keep-alive timeouts cause intermittent 502 errors behind a load balancer?
Summary
If the backend closes idle connections before the load balancer does, the load balancer can send a request on a connection the backend is closing at that moment and gets no response, returning 502. The backend's keep-alive timeout must be longer than the load balancer's idle timeout (60 s default on an AWS ALB; Node.js defaults to 5 s, Gunicorn to 2 s).
Longer Answer
The 502s don't appear in application logs because the request never reached the handler. ALB access logs show the target didn't respond. The same rule applies to any proxy chain (nginx to app server, Envoy to service).
Key Points
- Backend timeout must exceed the proxy's.
- Defaults are mismatched in many stacks.
- 502s without app errors are the signal.
- Applies to every proxy hop.
Deep dive: Cloud Networking in Production: Random 502s Behind the Load Balancer
Explanation
What interviewers look for
The race explanation and the concrete configuration fix.
More most-asked Networking questions
- How does NAT work, and what are its limits?
- What's the difference between TCP and UDP, and when would you choose UDP?
- What do 502, 503 and 504 errors from a load balancer usually mean?
- What is CIDR notation? How many usable addresses are in a /26, and in an AWS /26 subnet?
- How does DNS resolution work, and what does a record's TTL control?
- What does TLS provide, and how does certificate validation work?
- What happens, at the network level, when you type a URL into a browser and press Enter?
- What's the difference between stateful and stateless firewalls? Give cloud examples.