first draft of alerting usage
parent
398e53bb38
commit
4c34aac394
1 changed files with 96 additions and 1 deletions
|
|
@ -1,2 +1,97 @@
|
||||||
# WIP
|
# Prometheus Alerting
|
||||||
|
|
||||||
|
## Table of Contents
|
||||||
|
|
||||||
|
- [Overview](#overview)
|
||||||
|
- [AlertManagerConfig](#AlertManagerConfig)
|
||||||
|
- [References](#references)
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This guide explains how to configure the AlertmanagerConfig CRD to route Prometheus alerts to appropriate receivers.
|
||||||
|
|
||||||
|
## AlertManagerConfig
|
||||||
|
|
||||||
|
The example below demonstrates configuring an email receiver for alerts.
|
||||||
|
|
||||||
|
First, create a Kubernetes Secret containing the SMTP credentials used by the email receiver.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Secret
|
||||||
|
metadata:
|
||||||
|
name: smtp-secret
|
||||||
|
type: Opaque
|
||||||
|
stringData:
|
||||||
|
password: $ùpèr3Pa$sword
|
||||||
|
```
|
||||||
|
|
||||||
|
Next, deploy the AlertmanagerConfig resource into the namespace you wish to monitor.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: monitoring.coreos.com/v1beta1
|
||||||
|
kind: AlertmanagerConfig
|
||||||
|
metadata:
|
||||||
|
name: alerts
|
||||||
|
spec:
|
||||||
|
receivers:
|
||||||
|
- name: email-receiver
|
||||||
|
emailConfigs:
|
||||||
|
- to: "operations@phoenix-technologies.ch"
|
||||||
|
from: "openshift@phoenix-technologies.ch"
|
||||||
|
smarthost: "smtp.office365.com:587"
|
||||||
|
authUsername: "openshift@phoenix-technologies.ch"
|
||||||
|
authPassword:
|
||||||
|
name: smtp-secret
|
||||||
|
key: password
|
||||||
|
sendResolved: true
|
||||||
|
headers:
|
||||||
|
- key: Subject
|
||||||
|
value: '[{{ .CommonLabels.namespace }}][{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .GroupLabels.SortedPairs.Values | join " " }} {{ if gt (len .CommonLabels) (len .GroupLabels) }}({{ with .CommonLabels.Remove .GroupLabels.Names }}{{ .Values | join " " }}{{ end }}){{ end }}'
|
||||||
|
|
||||||
|
inhibitRules:
|
||||||
|
- sourceMatch:
|
||||||
|
- name: severity
|
||||||
|
value: critical
|
||||||
|
targetMatch:
|
||||||
|
- name: severity
|
||||||
|
value: warning
|
||||||
|
equal: ["alertname"]
|
||||||
|
|
||||||
|
route:
|
||||||
|
groupBy:
|
||||||
|
- alertname
|
||||||
|
- severity
|
||||||
|
groupWait: 30s
|
||||||
|
groupInterval: 5m
|
||||||
|
repeatInterval: 12h
|
||||||
|
receiver: email-receiver
|
||||||
|
routes:
|
||||||
|
- matchers:
|
||||||
|
- name: severity
|
||||||
|
value: critical
|
||||||
|
matchType: "="
|
||||||
|
receiver: email-receiver
|
||||||
|
groupWait: 2m
|
||||||
|
groupInterval: 30m
|
||||||
|
repeatInterval: 3h
|
||||||
|
- matchers:
|
||||||
|
- name: severity
|
||||||
|
value: warning
|
||||||
|
matchType: "="
|
||||||
|
receiver: email-receiver
|
||||||
|
groupWait: 5m
|
||||||
|
groupInterval: 1h
|
||||||
|
repeatInterval: 12h
|
||||||
|
```
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
### Official Documentation
|
||||||
|
- [AlertManager Official Documentation](https://prometheus.io/docs/alerting/0.26/overview)
|
||||||
|
- [AlertManager GitHub Repository](https://github.com/prometheus/alertmanager/tree/v0.26.0)
|
||||||
|
|
||||||
|
### Configuration Sections
|
||||||
|
- [Inhibition](https://prometheus.io/docs/alerting/0.26/configuration/#inhibition-related-settings)
|
||||||
|
- [Receiver](https://prometheus.io/docs/alerting/0.26/configuration/#receiver-integration-settings)
|
||||||
|
- [Route](https://prometheus.io/docs/alerting/0.26/configuration/#route-related-settings)
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue