Commit graph

47 commits

Author SHA1 Message Date
Krrish Dholakia
3560f0ef2c refactor: move all testing to top-level of repo
Closes https://github.com/BerriAI/litellm/issues/486
2024-09-28 21:08:14 -07:00
Krish Dholakia
72e961af3c
LiteLLM Minor Fixes and Improvements (08/06/2024) (#5567)
* fix(utils.py): return citations for perplexity streaming

Fixes https://github.com/BerriAI/litellm/issues/5535

* fix(anthropic/chat.py): support fallbacks for anthropic streaming (#5542)

* fix(anthropic/chat.py): support fallbacks for anthropic streaming

Fixes https://github.com/BerriAI/litellm/issues/5512

* fix(anthropic/chat.py): use module level http client if none given (prevents early client closure)

* fix: fix linting errors

* fix(http_handler.py): fix raise_for_status error handling

* test: retry flaky test

* fix otel type

* fix(bedrock/embed): fix error raising

* test(test_openai_batches_and_files.py): skip azure batches test (for now) quota exceeded

* fix(test_router.py): skip azure batch route test (for now) - hit batch quota limits

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

* All `model_group_alias` should show up in `/models`, `/model/info` , `/model_group/info` (#5539)

* fix(router.py): support returning model_alias model names in `/v1/models`

* fix(proxy_server.py): support returning model alias'es on `/model/info`

* feat(router.py): support returning model group alias for `/model_group/info`

* fix(proxy_server.py): fix linting errors

* fix(proxy_server.py): fix linting errors

* build(model_prices_and_context_window.json): add amazon titan text premier pricing information

Closes https://github.com/BerriAI/litellm/issues/5560

* feat(litellm_logging.py): log standard logging response object for pass through endpoints. Allows bedrock /invoke agent calls to be correctly logged to langfuse + s3

* fix(success_handler.py): fix linting error

* fix(success_handler.py): fix linting errors

* fix(team_endpoints.py): Allows admin to update team member budgets

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2024-09-06 17:16:24 -07:00
Krrish Dholakia
60381ffcdd test: fix assert string on test 2024-08-28 12:07:05 -07:00
Ishaan Jaff
c25a69fa78 test test_using_default_working_fallback 2024-08-20 13:32:55 -07:00
Ishaan Jaff
c4153f9340 test router fallbacks test 2024-08-20 13:00:00 -07:00
Ishaan Jaff
518be32a98 run test on sync function too 2024-08-20 12:57:22 -07:00
Ishaan Jaff
e4b5e88a57 fix fallbacks dont recurse on the same fallback 2024-08-20 12:50:20 -07:00
Krrish Dholakia
cccc55213b fix(router.py): improve error message returned for fallbacks 2024-06-25 11:27:20 -07:00
Krrish Dholakia
2c7a80d08d fix(router.py): check if azure returns 'content_filter' response + fallback available -> fallback
Exception maps azure content filter response exceptions
2024-06-22 19:10:15 -07:00
Krrish Dholakia
6f715b4782 feat(router.py): support content policy fallbacks
Closes https://github.com/BerriAI/litellm/issues/2632
2024-06-14 17:15:44 -07:00
Ishaan Jaff
dfe874c9e5 test - client side fallbacks 2024-06-10 15:00:36 -07:00
Krrish Dholakia
7557b3e2ff fix(init.py): set 'default_fallbacks' as a litellm_setting 2024-05-14 11:15:53 -07:00
Krrish Dholakia
5488bf4921 feat(router.py): enable default fallbacks
allow user to define a generic list of fallbacks, in case a new deployment is bad

Closes https://github.com/BerriAI/litellm/issues/3623
2024-05-13 17:49:56 -07:00
Krrish Dholakia
13e1577753 fix(slack_alerting.py): don't fire spam alerts when backend api call fails 2024-05-13 10:04:43 -07:00
Ishaan Jaff
04ac352407 test fix - test_async_fallbacks_embeddings 2024-05-11 19:20:24 -07:00
Krrish Dholakia
cdec7a414f test(test_router_fallbacks.py): fix test 2024-05-10 09:58:40 -07:00
Krrish Dholakia
69bcde6f6b test(test_router_fallbacks.py): fix test - making it easier to debug 2024-05-06 18:49:36 -07:00
Krrish Dholakia
a35ac050d2 test(test_router_fallbacks.py): bump test limits 2024-05-03 20:42:29 -07:00
Krrish Dholakia
cfc1eeb3c3 test(test_router_fallbacks.py): rename test to run earlier 2024-04-30 22:04:20 -07:00
Krrish Dholakia
e506e71cb9 fix(test_router_fallbacks.py): reduce test rpm 2024-04-30 22:00:48 -07:00
Krrish Dholakia
bc5c9d7da9 fix(test_router_fallbacks.py): fix tests 2024-04-30 18:48:39 -07:00
Krrish Dholakia
1baad80c7d fix(router.py): cooldown deployments, for 401 errors 2024-04-30 17:54:00 -07:00
Krrish Dholakia
b9c0b55e7c test: fix test - set num_retries=0 2024-04-27 21:02:19 -07:00
Krrish Dholakia
54241f2551 test(test_router_fallbacks.py): fix testing 2024-04-25 17:43:40 -07:00
Krrish Dholakia
9735250db7 fix(router.py): support fallbacks / retries with sync embedding calls 2024-03-11 14:51:22 -07:00
ishaan-jaff
5807718a69 (test) usage based routing with fallbacks 2024-01-22 14:21:30 -08:00
ishaan-jaff
0fa9d8af82 (test) custom cooldown times - router 2024-01-19 19:41:52 -08:00
ishaan-jaff
0196ac6376 (test) router-fallbacks 2024-01-19 14:12:09 -08:00
ishaan-jaff
57a20100e4 (test) usage based routing 2024-01-19 13:58:08 -08:00
ishaan-jaff
912e2de364 (test) usage based router + fallbacks 2024-01-19 12:34:08 -08:00
Krrish Dholakia
a37a18ca80 feat(router.py): add support for retry/fallbacks for async embedding calls 2024-01-02 11:54:28 +05:30
Krrish Dholakia
e516cfe9f5 fix(utils.py): allow text completion input to be either model or engine 2023-12-27 17:24:16 +05:30
Krrish Dholakia
10a74d02c1 test(test_router_fallbacks.py): fix test to check previous models in pre_api_call not on success 2023-12-27 16:34:40 +05:30
Krrish Dholakia
6d63c0015b test(test_router_fallbacks.py): add testing for sync streaming fallbacks 2023-12-27 15:10:43 +05:30
Krrish Dholakia
c9fdbaf898 fix(azure.py,-openai.py): correctly raise errors if streaming calls fail 2023-12-27 15:08:37 +05:30
Krrish Dholakia
4905929de3 refactor: add black formatting 2023-12-25 14:11:20 +05:30
Krrish Dholakia
9cf5ab468f fix(router.py): deepcopy initial model list, don't mutate it 2023-12-12 09:54:06 -08:00
ishaan-jaff
f5d64a4992 (fix) test router 2023-12-12 09:50:44 -08:00
Krrish Dholakia
b3600e9e8a test(test_router_fallbacks.py): update to use async logging 2023-12-11 20:12:29 -08:00
Krrish Dholakia
7b83238cb5 fix(router.py): log when a call is retried or fallback happens 2023-12-05 21:29:58 -08:00
Krrish Dholakia
fa713abfc3 fix(router.py): check for fallbacks in completion params for router 2023-11-25 18:46:45 -08:00
Krrish Dholakia
2e8d582a34 fix(proxy_server.py): fix linting issues 2023-11-24 11:39:01 -08:00
Krrish Dholakia
4a5dae3941 fix(main.py): fix streaming_chunk_builder to return usage 2023-11-24 11:27:04 -08:00
Krrish Dholakia
9a44433844 feat(proxy_server.py): /key/delete endpoint 2023-11-23 21:37:53 -08:00
Krrish Dholakia
c273d6f0d6 fix(router.py): add support for context window fallbacks on router 2023-11-23 16:43:02 -08:00
Krrish Dholakia
afac42e93a fix(router.py): enable async completions with model fallbacks 2023-11-23 16:15:57 -08:00
Krrish Dholakia
8ac03e492f fix(router.py): enable fallbacks for sync completions 2023-11-23 16:06:46 -08:00