Prometheus and PromQL 3 cheatsheet

Prometheus and PromQL 3: Rates and aggregation

11 common Prometheus and PromQL 3 tasks.

Per-second rate of a counter

Average per-second increase over five minutes. Always use rate (or increase) on counters, never the raw value.

rate(http_requests_total[5m])

How much a counter grew in the last hour.

increase(http_requests_total[1h])

Sum the rate across all instances, keeping one series per job and status.

sum by (job, status) (rate(http_requests_total[5m]))

The share of requests that fail, from two aggregated rates.

sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))

Estimate a percentile from a histogram's buckets. Always sum by le first.

histogram_quantile(0.95, sum by (le) (rate(http_request_duration_seconds_bucket[5m])))

Return the five busiest series.

topk(5, sum by (instance) (rate(http_requests_total[5m])))

Smooth a gauge by averaging over a window.

avg_over_time(node_load1[10m])

Estimate free disk space four hours from now by extrapolating the last six hours.

predict_linear(node_filesystem_avail_bytes[6h], 4 * 3600)

Returns 1 when no series match, which is useful for alerting on vanished targets.

absent(up{job="api"})

A common node_exporter query for host memory use.

100 * (1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)

A common node_exporter query for host CPU use.

100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)