No results
3
Guide Alerting
Sofiane Gerhardt edited this page 2025-10-15 17:13:16 +02:00
Prometheus Alerting
Table of Contents
Overview
This guide explains how to configure the AlertmanagerConfig CRD to route Prometheus alerts to appropriate receivers.
AlertManagerConfig
The example below demonstrates configuring an email receiver for alerts.
First, create a Kubernetes Secret containing the SMTP credentials used by the email receiver.
apiVersion: v1
kind: Secret
metadata:
name: smtp-secret
type: Opaque
stringData:
password: $ùpèr3Pa$sword
Next, deploy the AlertmanagerConfig resource into the namespace you wish to monitor.
apiVersion: monitoring.coreos.com/v1beta1
kind: AlertmanagerConfig
metadata:
name: alerts
spec:
# Receivers define destinations for alerts (email, pagerduty, webhook, etc.)
receivers:
- name: email-receiver
emailConfigs:
- to: "operations@phoenix-technologies.ch"
from: "openshift@phoenix-technologies.ch"
smarthost: "smtp.office365.com:587"
authUsername: "openshift@phoenix-technologies.ch"
authPassword:
name: smtp-secret
key: password
sendResolved: true
headers:
- key: Subject
# Template used to format email subject. Includes namespace, status and group labels.
value: '[{{ .CommonLabels.namespace }}][{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .GroupLabels.SortedPairs.Values | join " " }} {{ if gt (len .CommonLabels) (len .GroupLabels) }}({{ with .CommonLabels.Remove .GroupLabels.Names }}{{ .Values | join " " }}{{ end }}){{ end }}'
# InhibitRules suppress notifications for lower-severity alerts when a matching higher-severity alert is firing
inhibitRules:
- sourceMatch:
- name: severity
value: critical
targetMatch:
- name: severity
value: warning
equal: ["alertname"] # Fields that must match between source and target alerts to apply inhibition
# Route defines how alerts are grouped and which receiver is used by default
route:
groupBy:
- alertname # Alerts with the same values for these labels are grouped together
- severity
groupWait: 30s # Wait time to collect alerts before sending first notification for a group
groupInterval: 5m # Minimum interval between notifications for an alert group
repeatInterval: 12h # How often to resend notifications for ongoing alerts
receiver: email-receiver # Default receiver for alerts that don't match child routes
# Child routes allow overriding routing for specific alert matchers (e.g., severity)
routes:
- matchers:
- name: severity
value: critical
matchType: "="
receiver: email-receiver
groupWait: 2m
groupInterval: 30m
repeatInterval: 3h
- matchers:
- name: severity
value: warning
matchType: "="
receiver: email-receiver
groupWait: 5m
groupInterval: 1h
repeatInterval: 12h