Rate limit requests per client

Approach

Keep a token-bucket limiter per client key (API key or IP) using golang.org/x/time/rate, evict idle entries, and return 429 with Retry-After when a request is over the limit.

Solution

package ratelimit

import (
	"net"
	"net/http"
	"sync"
	"time"

	"golang.org/x/time/rate"
)

type entry struct {
	limiter  *rate.Limiter
	lastSeen time.Time
}

type PerClient struct {
	mu      sync.Mutex
	clients map[string]*entry
	rate    rate.Limit
	burst   int
}

func New(perSecond float64, burst int) *PerClient {
	l := &PerClient{clients: map[string]*entry{}, rate: rate.Limit(perSecond), burst: burst}
	go l.evict()
	return l
}

func (l *PerClient) allow(key string) bool {
	l.mu.Lock()
	defer l.mu.Unlock()
	e, ok := l.clients[key]
	if !ok {
		e = &entry{limiter: rate.NewLimiter(l.rate, l.burst)}
		l.clients[key] = e
	}
	e.lastSeen = time.Now()
	return e.limiter.Allow()
}

func (l *PerClient) evict() {
	for range time.Tick(time.Minute) {
		l.mu.Lock()
		for key, e := range l.clients {
			if time.Since(e.lastSeen) > 3*time.Minute {
				delete(l.clients, key)
			}
		}
		l.mu.Unlock()
	}
}

func (l *PerClient) Middleware(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		key := r.Header.Get("X-Api-Key")
		if key == "" {
			key, _, _ = net.SplitHostPort(r.RemoteAddr)
		}
		if !l.allow(key) {
			w.Header().Set("Retry-After", "1")
			http.Error(w, "rate limit exceeded", http.StatusTooManyRequests)
			return
		}
		next.ServeHTTP(w, r)
	})
}

Pitfalls

• Keying by RemoteAddr behind a proxy (everyone shares the proxy's IP); use the trusted forwarded header.
• Never evicting limiters, growing memory with every new IP.
• In-memory limits with several replicas; each replica allows the full rate. Use Redis or the gateway for global limits.

Variations

• Use limiter.Wait(ctx) for outbound calls, where waiting is better than rejecting.
• Different limits per plan or endpoint by choosing the limiter from the API key's metadata.

Further reading: x/time/rate