Commit graph

46 commits

Author SHA1 Message Date
Krish Dholakia
2cab33b061 LiteLLM Minor Fixes and Improvements (08/06/2024) (#5567)
* fix(utils.py): return citations for perplexity streaming

Fixes https://github.com/BerriAI/litellm/issues/5535

* fix(anthropic/chat.py): support fallbacks for anthropic streaming (#5542)

* fix(anthropic/chat.py): support fallbacks for anthropic streaming

Fixes https://github.com/BerriAI/litellm/issues/5512

* fix(anthropic/chat.py): use module level http client if none given (prevents early client closure)

* fix: fix linting errors

* fix(http_handler.py): fix raise_for_status error handling

* test: retry flaky test

* fix otel type

* fix(bedrock/embed): fix error raising

* test(test_openai_batches_and_files.py): skip azure batches test (for now) quota exceeded

* fix(test_router.py): skip azure batch route test (for now) - hit batch quota limits

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

* All `model_group_alias` should show up in `/models`, `/model/info` , `/model_group/info` (#5539)

* fix(router.py): support returning model_alias model names in `/v1/models`

* fix(proxy_server.py): support returning model alias'es on `/model/info`

* feat(router.py): support returning model group alias for `/model_group/info`

* fix(proxy_server.py): fix linting errors

* fix(proxy_server.py): fix linting errors

* build(model_prices_and_context_window.json): add amazon titan text premier pricing information

Closes https://github.com/BerriAI/litellm/issues/5560

* feat(litellm_logging.py): log standard logging response object for pass through endpoints. Allows bedrock /invoke agent calls to be correctly logged to langfuse + s3

* fix(success_handler.py): fix linting error

* fix(success_handler.py): fix linting errors

* fix(team_endpoints.py): Allows admin to update team member budgets

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2024-09-06 17:16:24 -07:00
Krrish Dholakia
b9b07cd932 test: fix assert string on test 2024-08-28 12:07:05 -07:00
Ishaan Jaff
528bb3f7ac test test_using_default_working_fallback 2024-08-20 13:32:55 -07:00
Ishaan Jaff
a3ccba6d6b test router fallbacks test 2024-08-20 13:00:00 -07:00
Ishaan Jaff
0847ebc82d run test on sync function too 2024-08-20 12:57:22 -07:00
Ishaan Jaff
078fe97053 fix fallbacks dont recurse on the same fallback 2024-08-20 12:50:20 -07:00
Krrish Dholakia
715153b27b fix(router.py): improve error message returned for fallbacks 2024-06-25 11:27:20 -07:00
Krrish Dholakia
f9ce6472d7 fix(router.py): check if azure returns 'content_filter' response + fallback available -> fallback
Exception maps azure content filter response exceptions
2024-06-22 19:10:15 -07:00
Krrish Dholakia
734bd5ef85 feat(router.py): support content policy fallbacks
Closes https://github.com/BerriAI/litellm/issues/2632
2024-06-14 17:15:44 -07:00
Ishaan Jaff
94210a86b4 test - client side fallbacks 2024-06-10 15:00:36 -07:00
Krrish Dholakia
b054f39bab fix(init.py): set 'default_fallbacks' as a litellm_setting 2024-05-14 11:15:53 -07:00
Krrish Dholakia
6f20389bd5 feat(router.py): enable default fallbacks
allow user to define a generic list of fallbacks, in case a new deployment is bad

Closes https://github.com/BerriAI/litellm/issues/3623
2024-05-13 17:49:56 -07:00
Krrish Dholakia
56b6efae50 fix(slack_alerting.py): don't fire spam alerts when backend api call fails 2024-05-13 10:04:43 -07:00
Ishaan Jaff
89b73e90f1 test fix - test_async_fallbacks_embeddings 2024-05-11 19:20:24 -07:00
Krrish Dholakia
62ba6f20f1 test(test_router_fallbacks.py): fix test 2024-05-10 09:58:40 -07:00
Krrish Dholakia
57ce36adf8 test(test_router_fallbacks.py): fix test - making it easier to debug 2024-05-06 18:49:36 -07:00
Krrish Dholakia
4e95463dbf test(test_router_fallbacks.py): bump test limits 2024-05-03 20:42:29 -07:00
Krrish Dholakia
92fbf19bdc test(test_router_fallbacks.py): rename test to run earlier 2024-04-30 22:04:20 -07:00
Krrish Dholakia
e5db3abe76 fix(test_router_fallbacks.py): reduce test rpm 2024-04-30 22:00:48 -07:00
Krrish Dholakia
b4ec16fd5f fix(test_router_fallbacks.py): fix tests 2024-04-30 18:48:39 -07:00
Krrish Dholakia
a12878b0f8 fix(router.py): cooldown deployments, for 401 errors 2024-04-30 17:54:00 -07:00
Krrish Dholakia
146e5d04ef test: fix test - set num_retries=0 2024-04-27 21:02:19 -07:00
Krrish Dholakia
ee518b7728 test(test_router_fallbacks.py): fix testing 2024-04-25 17:43:40 -07:00
Krrish Dholakia
03e8ce938b fix(router.py): support fallbacks / retries with sync embedding calls 2024-03-11 14:51:22 -07:00
ishaan-jaff
c20d9299cc (test) usage based routing with fallbacks 2024-01-22 14:21:30 -08:00
ishaan-jaff
33c3c438d7 (test) custom cooldown times - router 2024-01-19 19:41:52 -08:00
ishaan-jaff
81397836a4 (test) router-fallbacks 2024-01-19 14:12:09 -08:00
ishaan-jaff
ee5cbb7836 (test) usage based routing 2024-01-19 13:58:08 -08:00
ishaan-jaff
a7a0365f66 (test) usage based router + fallbacks 2024-01-19 12:34:08 -08:00
Krrish Dholakia
01c042fdc6 feat(router.py): add support for retry/fallbacks for async embedding calls 2024-01-02 11:54:28 +05:30
Krrish Dholakia
606de01ac0 fix(utils.py): allow text completion input to be either model or engine 2023-12-27 17:24:16 +05:30
Krrish Dholakia
2af635e9fd test(test_router_fallbacks.py): fix test to check previous models in pre_api_call not on success 2023-12-27 16:34:40 +05:30
Krrish Dholakia
517b22604b test(test_router_fallbacks.py): add testing for sync streaming fallbacks 2023-12-27 15:10:43 +05:30
Krrish Dholakia
fd5e6efb1d fix(azure.py,-openai.py): correctly raise errors if streaming calls fail 2023-12-27 15:08:37 +05:30
Krrish Dholakia
79978c44ba refactor: add black formatting 2023-12-25 14:11:20 +05:30
Krrish Dholakia
0cf0c2d6dd fix(router.py): deepcopy initial model list, don't mutate it 2023-12-12 09:54:06 -08:00
ishaan-jaff
ac91543848 (fix) test router 2023-12-12 09:50:44 -08:00
Krrish Dholakia
92cc39f00e test(test_router_fallbacks.py): update to use async logging 2023-12-11 20:12:29 -08:00
Krrish Dholakia
4ecd05df3e fix(router.py): log when a call is retried or fallback happens 2023-12-05 21:29:58 -08:00
Krrish Dholakia
67fe8824b3 fix(router.py): check for fallbacks in completion params for router 2023-11-25 18:46:45 -08:00
Krrish Dholakia
6f40fd8ee2 fix(proxy_server.py): fix linting issues 2023-11-24 11:39:01 -08:00
Krrish Dholakia
5a9a3aa89c fix(main.py): fix streaming_chunk_builder to return usage 2023-11-24 11:27:04 -08:00
Krrish Dholakia
8030a9b8d1 feat(proxy_server.py): /key/delete endpoint 2023-11-23 21:37:53 -08:00
Krrish Dholakia
e4deb09eb6 fix(router.py): add support for context window fallbacks on router 2023-11-23 16:43:02 -08:00
Krrish Dholakia
7f632e6e2f fix(router.py): enable async completions with model fallbacks 2023-11-23 16:15:57 -08:00
Krrish Dholakia
59d084342d fix(router.py): enable fallbacks for sync completions 2023-11-23 16:06:46 -08:00