Turn an SLO into alerts that page for the right reasons.
Enter the SLO and the PromQL for its SLI. You get the error budget, how fast each alert fires, and rules for Prometheus, Grafana, and Terraform. The method is the multiwindow, multi-burn-rate alert from chapter 5 of the Google SRE Workbook.
Objective
Step 1Each selector is a plain PromQL series selector. The tool wraps it in sum(rate(...[window])). Group-by labels go into every sum by and every and on, so each cluster or namespace burns its own budget. On Prometheus 3, whole-number bucket boundaries are stored as le="1.0", not le="1".
Alert policy
Step 2Each row alerts when the long and the short window both burn faster than the row's rate. Burn rate = budget share × SLO window ÷ long window. The defaults are the workbook's starting point.
| Budget burned (%) | Long window | Short window | Severity | Remove |
|---|
Error budget
Downtime means a full outage: every request failing. Partial failure stretches the same budget further.
When the alerts fire
Step 3| Policy | Burn rate | Fires above | Full outage fires in | Budget spent by then | At the what-if rate |
|---|
Times assume the window was clean before the errors started, and ignore scrape and rule evaluation intervals. The short window keeps an alert from firing late on old errors and lets it resolve soon after recovery.
Generated config
Prometheus rule file
Recording rules for every window, then one alert per policy row
Grafana file provisioning
Grafana-managed rules, no recording rules needed
Terraform
grafana_rule_group for the grafana/grafana provider
Grafana rules carry a for of one minute, so they fire about a minute after the Prometheus rules. Replace the runbook_url placeholders before you ship.