At-Most-Once, At-Least-Once and Effectively-Once — Idempotency & Retry-Safe Systems

At-most-once: send once, never retry — no duplicates but possible loss. At-least-once: retry until acknowledged — no loss but possible duplicates; this is what most brokers and robust clients provide. Exactly-once delivery is impossible over unreliable networks in general, but effectively-once processing is achievable: at-least-once delivery combined with idempotent or deduplicated processing (or transactional systems like Kafka's exactly-once semantics within Kafka).

Delivery semantics

Criterion	At-most-once	At-least-once	Effectively-once
Duplicates	Never	Possible	Absorbed by dedupe
Loss	Possible	Never (until retries exhausted)	Never
Mechanism	Fire and forget / ack before processing	Retry until ack / ack after processing	At-least-once + idempotent consumer
Good for	Metrics, best-effort notifications	Most business events	Payments, orders, ledgers

Common pitfalls

Believing a broker setting gives end-to-end exactly-once across external side effects.
Acking before processing (silently becomes at-most-once).
Dedupe windows shorter than retry windows.

Key points

Exactly-once delivery is a myth across systems
At-least-once + idempotency = effectively-once
Ack after processing

Further reading: Confluent: Exactly-once semantics in Kafka

Practice questions

Why doesn't Kafka's exactly-once semantics cover sending an email from a consumer?

Answer

Kafka transactions make reads, writes and offset commits within Kafka atomic; an external side effect like an email or HTTP call isn't part of that transaction and may happen twice.

Where does 'ack after processing' put you in delivery semantics?

Answer

At-least-once: if processing succeeds but the ack fails, the message is redelivered, so processing must be idempotent.