The Service Ignores Stop Signals

Scenario

Every deploy takes 30 seconds longer than it should, and in-flight requests fail. The orchestrator sends SIGTERM, waits for the grace period, then sends SIGKILL. The app has a graceful shutdown handler, but it never runs.

What you see in production

• Stops take exactly the timeout (10 s in Docker, 30 s in Kubernetes, 90 s in systemd by default).
• Logs never show the app's "shutting down" message.
• The process list shows a shell script as the parent (or PID 1) and the app as its child.

The naive code

#!/usr/bin/env bash
# start.sh, the container's entrypoint
export APP_ENV=production
python3 server.py           # bash waits here; SIGTERM goes to bash, not to python

Reproduce it

cat > server.py <<'EOF'
import signal, sys, time
signal.signal(signal.SIGTERM, lambda *_: (print("server: graceful shutdown", flush=True), sys.exit(0)))
print("server: running", flush=True)
time.sleep(30)
EOF
cat > start.sh <<'EOF'
#!/usr/bin/env bash
python3 "$PWD/server.py"
EOF
chmod +x start.sh
./start.sh & wrapper=$!
sleep 0.5
kill -TERM $wrapper
sleep 0.5
server=$(pgrep -f "$PWD/server.py")          # this directory's server only
if [[ -n $server ]]; then echo "wrapper is gone, but server.py is still running (orphaned)"; fi
kill $server

server: running
wrapper is gone, but server.py is still running (orphaned)
server: graceful shutdown

Root cause

The signal is delivered to the process the supervisor started: the shell. A non-interactive Bash doesn't forward signals to its children, and while it waits for a foreground command it doesn't even run traps until that command finishes. The app never sees SIGTERM, so the supervisor eventually sends SIGKILL, or, as here, the app is left orphaned.

The fix

Make the app replace the shell with exec, so it receives signals directly (and becomes PID 1 in a container):

cat > server.py <<'EOF'
import signal, sys, time
signal.signal(signal.SIGTERM, lambda *_: (print("server: graceful shutdown", flush=True), sys.exit(0)))
print("server: running", flush=True)
time.sleep(30)
EOF
cat > start.sh <<'EOF'
#!/usr/bin/env bash
export APP_ENV=production
exec python3 server.py
EOF
chmod +x start.sh
./start.sh & pid=$!
sleep 0.5
ps -o args= -p $pid
kill -TERM $pid
wait $pid; echo "exit status: $?"

server: running
python3 server.py
server: graceful shutdown
exit status: 0

• End wrapper scripts with exec "$@" or exec app ....
• Use the exec form in Dockerfiles: CMD ["python3", "server.py"], not CMD python3 server.py.
• When a script must run several processes, trap SIGTERM and forward it, or use a tiny init (tini, docker run --init).

Trade-offs

• exec means nothing in the script runs after the app; do setup before it.
• An init process also reaps zombie processes, which matters when the app spawns children.
• Graceful shutdown still needs the app to stop accepting work and finish in-flight requests within the grace period.

Detect it in production

• Measure stop duration; stops that always hit the timeout indicate swallowed signals.
• ps -o pid,ppid,args inside the container: the app should be PID 1 or a direct child of an init.
• Check for "graceful shutdown" log lines during deploys.

Key points

• Shell wrappers don't forward signals to their children.
• exec replaces the shell so the app receives SIGTERM.
• Use exec-form CMD, or an init like tini when you need several processes.

Further reading: signal(7) · bash(1): exec · Docker: ENTRYPOINT and exec form