The kube-prometheus-stack chart installs the Prometheus Operator, Prometheus, Alertmanager, Grafana, node exporters and a set of dashboards and alert rules for Kubernetes.
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install monitoring prometheus-community/kube-prometheus-stack \
-n monitoring --create-namespace -f values.yaml
kubectl -n monitoring port-forward svc/monitoring-grafana 3000:80
Scrape your own application with a ServiceMonitor:
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: api
namespace: monitoring
labels: { release: monitoring }
spec:
namespaceSelector: { matchNames: [shop] }
selector: { matchLabels: { app: api } }
endpoints: [{ port: metrics, interval: 30s }]
Set a persistent volume and retention for Prometheus in values.yaml, and route Alertmanager to Slack or email. Alerts worth having first: pod crash looping, node not ready, persistent volume nearly full, and deployment replicas unavailable.