Commit graph

31 commits

Author SHA1 Message Date
Ishaan Jaff
4ebb7c8a7f
(docs + fix) Add docs on Moderations endpoint, Text Completion (#6947)
* fix _pass_through_moderation_endpoint_factory

* fix route_llm_request

* doc moderations api

* docs on /moderations

* add e2e tests for moderations api

* docs moderations api

* test_pass_through_moderation_endpoint_factory

* docs text completion
2024-11-27 16:30:48 -08:00
Ishaan Jaff
603299e3c8
(feat) prometheus have well defined latency buckets (#6211)
* fix prometheus have well defined latency buckets

* use a well define latency bucket

* use types file for prometheus logging

* add test for LATENCY_BUCKETS
2024-10-14 17:16:01 +05:30
Ishaan Jaff
c84cfe977e
(feat) add /key/health endpoint to test key based logging (#6073)
* add /key/health endpoint

* add /key/health endpoint

* fix return from /key/health

* update doc string

* fix doc string for /key/health

* add test for /key/health

* fix linting

* docs /key/health
2024-10-05 11:56:55 +05:30
Ishaan Jaff
49ec40b1cb
(feat proxy prometheus) track virtual key, key alias, error code, error code class on prometheus (#5968)
* track api key and team in prom latency metric

* add test for latency metric

* test prometheus success metrics for latency

* track team and key labels for deployment failures

* add test for litellm_deployment_failure_responses_total

* fix checks for premium user on prometheus

* log_success_fallback_event and log_failure_fallback_event

* log original_exception in log_success_fallback_event

* track key, team and exception status and class on fallback metrics

* use get_standard_logging_metadata

* fix import error

* track litellm_deployment_successful_fallbacks

* add test test_proxy_fallback_metrics

* add log log_success_fallback_event

* fix test prometheus
2024-09-28 19:00:21 -07:00
Krrish Dholakia
2a8eb492a1 test(test_otel.py): fix test 2024-09-23 08:10:06 -07:00
Krish Dholakia
8039b95aaf
LiteLLM Minor Fixes & Improvements (09/21/2024) (#5819)
* fix(router.py): fix error message

* Litellm disable keys (#5814)

* build(schema.prisma): allow blocking/unblocking keys

Fixes https://github.com/BerriAI/litellm/issues/5328

* fix(key_management_endpoints.py): fix pop

* feat(auth_checks.py): allow admin to enable/disable virtual keys

Closes https://github.com/BerriAI/litellm/issues/5328

* docs(vertex.md): add auth section for vertex ai

Addresses - https://github.com/BerriAI/litellm/issues/5768#issuecomment-2365284223

* build(model_prices_and_context_window.json): show which models support prompt_caching

Closes https://github.com/BerriAI/litellm/issues/5776

* fix(router.py): allow setting default priority for requests

* fix(router.py): add 'retry-after' header for concurrent request limit errors

Fixes https://github.com/BerriAI/litellm/issues/5783

* fix(router.py): correctly raise and use retry-after header from azure+openai

Fixes https://github.com/BerriAI/litellm/issues/5783

* fix(user_api_key_auth.py): fix valid token being none

* fix(auth_checks.py): fix model dump for cache management object

* fix(user_api_key_auth.py): pass prisma_client to obj

* test(test_otel.py): update test for new key check

* test: fix test
2024-09-21 18:51:53 -07:00
Ishaan Jaff
711932294c
[Feat] Add testing for prometheus failure metrics (#5823)
* prom - show status code and class type on prom

* log exception_class name on prometheus metrics

* prometheus track error code and status

* add bad model

* add prometheus failure metric test

* remove outdated file

* fix litellm_proxy_total_requests_metric

* add prometheus metrics testing
2024-09-21 11:36:29 -07:00
Ishaan Jaff
1973ae8fb8
[Feat] Allow setting supports_vision for Custom OpenAI endpoints + Added testing (#5821)
* add test for using images with custom openai endpoints

* run all otel tests

* update name of test

* add custom openai model to test config

* add test for setting supports_vision=True for model

* fix test guardrails aporia

* docs supports vison

* fix yaml

* fix yaml

* docs supports vision

* fix bedrock guardrail test

* fix cohere rerank test

* update model_group doc string

* add better prints on test
2024-09-21 11:35:55 -07:00
Ishaan Jaff
da2cefc45a fix team based tag routing 2024-08-29 14:37:44 -07:00
Ishaan Jaff
f592aeaa38 add test_chat_completion_with_no_tags 2024-08-29 13:54:11 -07:00
Ishaan Jaff
34f1d32799 add test for tag based routing 2024-08-29 13:45:24 -07:00
Ishaan Jaff
c27640e6e4 add /rerank test 2024-08-27 17:50:37 -07:00
Ishaan Jaff
918e4fcfe5 feat add test for custom guardrails 2024-08-23 12:01:43 -07:00
Ishaan Jaff
1f0cc72531 test bedrock guardrails 2024-08-22 17:24:42 -07:00
Krish Dholakia
409306b266
Merge branch 'main' into litellm_fix_azure_api_version 2024-08-20 11:40:53 -07:00
Ishaan Jaff
aceab2669f test guardrails with API Key 2024-08-20 08:40:00 -07:00
Ishaan Jaff
04efdd66c0 fix test guardrails 2024-08-19 21:03:37 -07:00
Ishaan Jaff
b2d63f752c add testing for guardrails 2024-08-19 21:03:37 -07:00
Ishaan Jaff
6fa421ed8e test_llm_guard_triggered 2024-08-19 21:03:37 -07:00
Ishaan Jaff
9f927e0e4e add testing for aporia guardrails 2024-08-19 21:03:37 -07:00
Ishaan Jaff
57bbcba079 fix test guardrails 2024-08-19 18:58:31 -07:00
Ishaan Jaff
5d262a6ee7 add testing for guardrails 2024-08-19 18:57:02 -07:00
Ishaan Jaff
37e6d8a3aa test_llm_guard_triggered 2024-08-19 18:53:12 -07:00
Ishaan Jaff
0431600f7b add testing for aporia guardrails 2024-08-19 18:50:14 -07:00
Ishaan Jaff
35ecf3c06b fix otel test 2024-07-27 15:30:58 -07:00
Ishaan Jaff
7dac0e0001 fix otel logging 2024-07-27 12:05:24 -07:00
Ishaan Jaff
7c0ea16e3d test otel for batch_write_to_db 2024-07-27 11:47:17 -07:00
Ishaan Jaff
dec5c7a2ba fix otel trace test 2024-07-16 21:08:19 -07:00
Ishaan Jaff
b835165742 fix otel trace 2024-07-16 20:58:34 -07:00
Ishaan Jaff
15d35cd62f test otel 2024-07-11 09:54:23 -07:00
Ishaan Jaff
02ab3cb73d test- otel span recording 2024-07-11 08:47:16 -07:00