phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

doc new prometheus metrics

Browse source
This commit is contained in:
Ishaan Jaff 2024-08-10 17:10:04 -07:00
parent a0a1feb7da
commit cc3316104f
2 changed files with 8 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/my-website/docs/proxy/prometheus.md
View file

@ -75,6 +75,12 @@ http://localhost:4000/metrics
| `deployment_state` | The state of the deployment: 0 = healthy, 1 = partial outage, 2 = complete outage. |
| `litellm_remaining_requests_metric` | Track `x-ratelimit-remaining-requests` returned from LLM API Deployment |
| `litellm_remaining_tokens` | Track `x-ratelimit-remaining-tokens` return from LLM API Deployment |
`llm_deployment_success_responses` | Total number of successful LLM API calls for deployment |
| `llm_deployment_failure_responses` | Total number of failed LLM API calls for deployment |
| `llm_deployment_total_requests` | Total number of LLM API calls for deployment - success + failure |
| `llm_deployment_latency_per_output_token` | Latency per output token for deployment |
| `llm_deployment_successful_fallbacks` | Number of successful fallback requests from primary model -> fallback model |
| `llm_deployment_failed_fallbacks` | Number of failed fallback requests from primary model -> fallback model |

4
litellm/integrations/prometheus.py
View file

@ -172,12 +172,12 @@ class PrometheusLogger(CustomLogger):
self.llm_deployment_successful_fallbacks = Counter(
"llm_deployment_successful_fallbacks",
"LLM Deployment Analytics - Number of successful fallback workloads",
"LLM Deployment Analytics - Number of successful fallback requests from primary model -> fallback model",
["primary_model", "fallback_model"],
)
self.llm_deployment_failed_fallbacks = Counter(
"llm_deployment_failed_fallbacks",
"LLM Deployment Analytics - Number of failed fallback workloads",
"LLM Deployment Analytics - Number of failed fallback requests from primary model -> fallback model",
["primary_model", "fallback_model"],
)