phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

(proxy prometheus) track api key and team in latency metrics (#5966)

Browse source 
* track api key and team in prom latency metric

* add test for latency metric

* test prometheus success metrics for latency
This commit is contained in:
Ishaan Jaff 2024-09-28 19:04:42 -07:00 • committed by GitHub
parent 49ec40b1cb
commit 7e69cdd1b4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 6 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

6
litellm/integrations/prometheus.py
View file

@ -264,6 +264,12 @@ class PrometheusLogger(CustomLogger):
)
# Deployment Latency tracking
team_and_key_labels = [
"hashed_api_key",
"api_key_alias",
"team",
"team_alias",
]
self.litellm_deployment_latency_per_output_token = Histogram(
name="litellm_deployment_latency_per_output_token",
documentation="LLM Deployment Analytics - Latency per output token",