Grafana 10: Alerting and Dashboards as Code

Grafana 10.0 (12 June 2023) continues the unified alerting model introduced in version 8: alert rules are written against any data source and routed with contact points and notification policies.

An alert rule in words: CPU above 85 percent for 10 minutes, labelled severity=warning, send to the on-call contact point. Build one under Alerting, Alert rules, New alert rule.

Keep configuration in Git using provisioning files. Data source, /etc/grafana/provisioning/datasources/prom.yaml:

apiVersion: 1
datasources:
  - name: Prometheus
    type: prometheus
    url: http://prometheus:9090
    isDefault: true

Dashboards, /etc/grafana/provisioning/dashboards/default.yaml:

apiVersion: 1
providers:
  - name: default
    folder: Ops
    type: file
    options: { path: /var/lib/grafana/dashboards }

Export a dashboard’s JSON from the UI, commit it, and Grafana loads it at start-up. Review changes through pull requests. Use variables (such as $instance) so one dashboard serves many hosts.