Why do restrictive network ACLs break return traffic?

They are stateless, so replies are evaluated like any other packet. Replies go to the initiator's ephemeral port, which must be explicitly allowed in the opposite direction.

Which ephemeral port range should an AWS NACL allow for return traffic, and why?

1024-65535, because the initiator might be Linux (32768-60999), Windows (49152-65535) or an AWS NAT gateway or load balancer (1024-65535).

How do you see that a NACL is dropping traffic?

VPC Flow Logs record REJECT for the dropped packets; inbound rejects to high ports usually mean return traffic is blocked. Reachability Analyzer explains which rule blocks a given path.

The Network ACL That Broke Some Clients — Cloud Networking in Production

Scenario

Example team: to satisfy an audit finding, a team replaces the default "allow all" network ACL on the application subnets with explicit rules. Inbound: HTTPS from anywhere, and "return traffic" on ports 32768-60999, copied from a Linux ip_local_port_range.

What you see in production

Most traffic still works. But calls from the application to some external APIs fail intermittently, Windows-based batch servers in another subnet can't reach the app at all, and traffic through the NAT gateway fails for a fraction of connections. Flow logs show REJECT on inbound packets to ports like 1050 and 61500.

The naive code

NACL app-subnets (inbound)
  100  allow  tcp  443           0.0.0.0/0
  110  allow  tcp  32768-60999   0.0.0.0/0     # "ephemeral ports" (Linux default)
  *    deny   all  all           0.0.0.0/0

Reproduce it

Return traffic goes to whichever ephemeral port the connecting side chose, and that range differs by operating system and device:

Which return packets does this NACL allow?Try it yourself

import random

random.seed(42)
RANGES = {"Linux client": (32768, 60999), "Windows client": (49152, 65535), "NAT gateway / ELB": (1024, 65535)}
RULE = (32768, 60999)                       # the NACL's inbound "return traffic" rule

def allowed(port, rule):
    return rule[0] <= port <= rule[1]

for who, (lo, hi) in RANGES.items():
    ports = [random.randint(lo, hi) for _ in range(1000)]
    ok = sum(allowed(p, RULE) for p in ports)
    print(f"{who:18} {ok / 10:5.1f}% of return packets allowed")

print("with 1024-65535:", all(allowed(p, (1024, 65535)) for p in range(1024, 65536)))

Output

Linux client       100.0% of return packets allowed
Windows client      69.9% of return packets allowed
NAT gateway / ELB   44.0% of return packets allowed
with 1024-65535: True

Root cause

Network ACLs are stateless: they don't remember that a connection was started from inside, so every reply needs its own allow rule. Replies are addressed to the initiator's ephemeral port, and the range depends on the initiator: Linux uses 32768-60999, Windows 49152-65535, and AWS NAT gateways and load balancers use 1024-65535. A rule tuned to one operating system silently drops a share of everyone else's traffic, which looks intermittent.

The fix

Keep NACLs simple: the default allow-all, plus a few deny rules for subnet-wide guardrails. Do fine-grained filtering with stateful security groups, which handle return traffic automatically.
If NACLs must be restrictive, allow inbound and outbound 1024-65535 for return traffic, as AWS recommends, and remember both directions (outbound replies for inbound connections too).
Change NACLs through code review with a connectivity test (Reachability Analyzer) for the critical paths.

NACL app-subnets (inbound)
  100  allow  tcp  443         0.0.0.0/0
  110  allow  tcp  1024-65535  0.0.0.0/0    # return traffic for connections started from this subnet
NACL app-subnets (outbound)
  100  allow  tcp  443         0.0.0.0/0
  110  allow  tcp  1024-65535  0.0.0.0/0    # replies to clients connecting in

Azure NSGs and Google Cloud firewall rules are stateful, so this class of problem is mostly an AWS NACL (and on-premises ACL) issue. AWS documents the recommended ranges in Ephemeral ports.

Trade-offs

Wide ephemeral ranges look permissive in audits; explain that they only permit packets of established flows in practice, and that security groups do the real filtering.
Two layers of rules (NACL + security group) double the places to debug; keep one of them trivial.

Detect it in production

VPC Flow Logs: REJECT records on high destination ports in the inbound direction are return traffic being dropped.
Reachability Analyzer or synthetic checks from each client type (Linux, Windows, NAT, load balancer) after every NACL change.

Key points

NACLs are stateless; replies need explicit rules to the initiator's ephemeral port range.
Ranges differ: Linux 32768-60999, Windows 49152-65535, NAT gateways and load balancers 1024-65535.
Prefer stateful security groups; if NACLs must be strict, allow 1024-65535 for return traffic.