Retry operations with exponential backoff and jitter

Approach

Retry only retryable errors, wait with exponential backoff plus random jitter, cap the number of attempts, and stop when the context is cancelled.

Solution

package retry

import (
	"context"
	"errors"
	"math/rand/v2"
	"time"
)

var ErrPermanent = errors.New("permanent failure")

type Policy struct {
	Attempts  int
	BaseDelay time.Duration
	MaxDelay  time.Duration
}

func Do(ctx context.Context, p Policy, op func(context.Context) error) error {
	var err error
	for attempt := range p.Attempts {
		if err = op(ctx); err == nil || errors.Is(err, ErrPermanent) {
			return err
		}
		if attempt == p.Attempts-1 {
			break
		}
		delay := min(p.BaseDelay<<attempt, p.MaxDelay)
		delay = delay/2 + rand.N(delay/2+1) // "equal jitter"
		select {
		case <-time.After(delay):
		case <-ctx.Done():
			return errors.Join(err, ctx.Err())
		}
	}
	return err
}

// Usage:
// err := retry.Do(ctx, retry.Policy{Attempts: 4, BaseDelay: 100 * time.Millisecond, MaxDelay: 2 * time.Second},
//     func(ctx context.Context) error { return client.Send(ctx, msg) })

Pitfalls

• Retrying non-idempotent operations (charging a card) without an idempotency key.
• Retrying everywhere in a call chain, multiplying attempts (3 layers x 3 retries = 27 calls).
• Fixed delays without jitter, so clients retry in synchronised waves.

Variations

• Respect Retry-After headers from HTTP 429/503 responses.
• Combine with a circuit breaker to stop calling a dependency that's clearly down.

Further reading: Timeouts, retries and backoff with jitter