first draft of alerting usage
parent
398e53bb38
commit
4c34aac394
1 changed files with 96 additions and 1 deletions
|
|
@ -1,2 +1,97 @@
|
|||
# WIP
|
||||
# Prometheus Alerting
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Overview](#overview)
|
||||
- [AlertManagerConfig](#AlertManagerConfig)
|
||||
- [References](#references)
|
||||
|
||||
## Overview
|
||||
|
||||
This guide explains how to configure the AlertmanagerConfig CRD to route Prometheus alerts to appropriate receivers.
|
||||
|
||||
## AlertManagerConfig
|
||||
|
||||
The example below demonstrates configuring an email receiver for alerts.
|
||||
|
||||
First, create a Kubernetes Secret containing the SMTP credentials used by the email receiver.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: smtp-secret
|
||||
type: Opaque
|
||||
stringData:
|
||||
password: $ùpèr3Pa$sword
|
||||
```
|
||||
|
||||
Next, deploy the AlertmanagerConfig resource into the namespace you wish to monitor.
|
||||
|
||||
```yaml
|
||||
apiVersion: monitoring.coreos.com/v1beta1
|
||||
kind: AlertmanagerConfig
|
||||
metadata:
|
||||
name: alerts
|
||||
spec:
|
||||
receivers:
|
||||
- name: email-receiver
|
||||
emailConfigs:
|
||||
- to: "operations@phoenix-technologies.ch"
|
||||
from: "openshift@phoenix-technologies.ch"
|
||||
smarthost: "smtp.office365.com:587"
|
||||
authUsername: "openshift@phoenix-technologies.ch"
|
||||
authPassword:
|
||||
name: smtp-secret
|
||||
key: password
|
||||
sendResolved: true
|
||||
headers:
|
||||
- key: Subject
|
||||
value: '[{{ .CommonLabels.namespace }}][{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .GroupLabels.SortedPairs.Values | join " " }} {{ if gt (len .CommonLabels) (len .GroupLabels) }}({{ with .CommonLabels.Remove .GroupLabels.Names }}{{ .Values | join " " }}{{ end }}){{ end }}'
|
||||
|
||||
inhibitRules:
|
||||
- sourceMatch:
|
||||
- name: severity
|
||||
value: critical
|
||||
targetMatch:
|
||||
- name: severity
|
||||
value: warning
|
||||
equal: ["alertname"]
|
||||
|
||||
route:
|
||||
groupBy:
|
||||
- alertname
|
||||
- severity
|
||||
groupWait: 30s
|
||||
groupInterval: 5m
|
||||
repeatInterval: 12h
|
||||
receiver: email-receiver
|
||||
routes:
|
||||
- matchers:
|
||||
- name: severity
|
||||
value: critical
|
||||
matchType: "="
|
||||
receiver: email-receiver
|
||||
groupWait: 2m
|
||||
groupInterval: 30m
|
||||
repeatInterval: 3h
|
||||
- matchers:
|
||||
- name: severity
|
||||
value: warning
|
||||
matchType: "="
|
||||
receiver: email-receiver
|
||||
groupWait: 5m
|
||||
groupInterval: 1h
|
||||
repeatInterval: 12h
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
### Official Documentation
|
||||
- [AlertManager Official Documentation](https://prometheus.io/docs/alerting/0.26/overview)
|
||||
- [AlertManager GitHub Repository](https://github.com/prometheus/alertmanager/tree/v0.26.0)
|
||||
|
||||
### Configuration Sections
|
||||
- [Inhibition](https://prometheus.io/docs/alerting/0.26/configuration/#inhibition-related-settings)
|
||||
- [Receiver](https://prometheus.io/docs/alerting/0.26/configuration/#receiver-integration-settings)
|
||||
- [Route](https://prometheus.io/docs/alerting/0.26/configuration/#route-related-settings)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue