Krish Dholakia
dd602753c0
Litellm fix router testing ( #5748 )
...
* test: fix testing - azure changed content policy error logic
* test: fix tests to use mock responses
* test(test_image_generation.py): handle api instability
* test(test_image_generation.py): handle azure api instability
* fix(utils.py): fix unbounded variable error
* fix(utils.py): fix unbounded variable error
* test: refactor test to use mock response
* test: mark flaky azure tests
2024-09-17 18:02:23 -07:00
Krish Dholakia
234185ec13
LiteLLM Minor Fixes & Improvements (09/16/2024) ( #5723 ) ( #5731 )
...
* LiteLLM Minor Fixes & Improvements (09/16/2024) (#5723 )
* coverage (#5713 )
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* Move (#5714 )
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix(litellm_logging.py): fix logging client re-init (#5710 )
Fixes https://github.com/BerriAI/litellm/issues/5695
* fix(presidio.py): Fix logging_hook response and add support for additional presidio variables in guardrails config
Fixes https://github.com/BerriAI/litellm/issues/5682
* feat(o1_handler.py): fake streaming for openai o1 models
Fixes https://github.com/BerriAI/litellm/issues/5694
* docs: deprecated traceloop integration in favor of native otel (#5249 )
* fix: fix linting errors
* fix: fix linting errors
* fix(main.py): fix o1 import
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view (#5730 )
* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view
Supports having `MonthlyGlobalSpend` view be a material view, and exposes an endpoint to refresh it
* fix(custom_logger.py): reset calltype
* fix: fix linting errors
* fix: fix linting error
* fix: fix import
* test(test_databricks.py): fix databricks tests
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
2024-09-17 08:05:52 -07:00
Krish Dholakia
da77706c26
Litellm stable dev ( #5711 )
...
* feat(aws_base_llm.py): prevents recreating boto3 credentials during high traffic
Leads to 100ms perf boost in local testing
* fix(base_aws_llm.py): fix credential caching check to see if token is set
* refactor(bedrock/chat): separate converse api and invoke api + isolate converse api transformation logic
Make it easier to see how requests are transformed for /converse
* fix: fix imports
* fix(bedrock/embed): fix reordering of headers
* fix(base_aws_llm.py): fix get credential logic
* fix(converse_handler.py): fix ai21 streaming response
2024-09-14 23:22:59 -07:00
Ishaan Jaff
085e1751ad
mark test as flaky
2024-09-12 09:29:37 -07:00
Krish Dholakia
72e961af3c
LiteLLM Minor Fixes and Improvements (08/06/2024) ( #5567 )
...
* fix(utils.py): return citations for perplexity streaming
Fixes https://github.com/BerriAI/litellm/issues/5535
* fix(anthropic/chat.py): support fallbacks for anthropic streaming (#5542 )
* fix(anthropic/chat.py): support fallbacks for anthropic streaming
Fixes https://github.com/BerriAI/litellm/issues/5512
* fix(anthropic/chat.py): use module level http client if none given (prevents early client closure)
* fix: fix linting errors
* fix(http_handler.py): fix raise_for_status error handling
* test: retry flaky test
* fix otel type
* fix(bedrock/embed): fix error raising
* test(test_openai_batches_and_files.py): skip azure batches test (for now) quota exceeded
* fix(test_router.py): skip azure batch route test (for now) - hit batch quota limits
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* All `model_group_alias` should show up in `/models`, `/model/info` , `/model_group/info` (#5539 )
* fix(router.py): support returning model_alias model names in `/v1/models`
* fix(proxy_server.py): support returning model alias'es on `/model/info`
* feat(router.py): support returning model group alias for `/model_group/info`
* fix(proxy_server.py): fix linting errors
* fix(proxy_server.py): fix linting errors
* build(model_prices_and_context_window.json): add amazon titan text premier pricing information
Closes https://github.com/BerriAI/litellm/issues/5560
* feat(litellm_logging.py): log standard logging response object for pass through endpoints. Allows bedrock /invoke agent calls to be correctly logged to langfuse + s3
* fix(success_handler.py): fix linting error
* fix(success_handler.py): fix linting errors
* fix(team_endpoints.py): Allows admin to update team member budgets
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2024-09-06 17:16:24 -07:00
Ishaan Jaff
b5349e97c7
run ci/cd again
2024-09-05 19:07:47 -07:00
Ishaan Jaff
0426aa5642
run test again
2024-09-05 15:26:48 -07:00
Krish Dholakia
be3c7b401e
LiteLLM Minor fixes + improvements (08/03/2024) ( #5488 )
...
* fix(internal_user_endpoints.py): set budget_reset_at for /user/update
* fix(vertex_and_google_ai_studio_gemini.py): handle accumulated json
Fixes https://github.com/BerriAI/litellm/issues/5479
* fix(vertex_ai_and_gemini.py): fix assistant message function call when content is not None
Fixes https://github.com/BerriAI/litellm/issues/5490
* fix(proxy_server.py): generic state uuid for okta sso
* fix(lago.py): improve debug logs
Debugging for https://github.com/BerriAI/litellm/issues/5477
* docs(bedrock.md): add bedrock cross-region inferencing to docs
* fix(azure.py): return azure response headers on aembedding call
* feat(azure.py): return azure response headers for `/audio/transcription`
* fix(types/utils.py): standardize deepseek / anthropic prompt caching usage information
Closes https://github.com/BerriAI/litellm/issues/5285
* docs(usage.md): add docs on litellm usage object
* test(test_completion.py): mark flaky test
2024-09-03 21:21:34 -07:00
Ishaan Jaff
e33d634e5d
mark test as flaky
2024-09-03 11:56:03 -07:00
Ishaan Jaff
ae25c5695f
add streaming test for ai21
2024-09-02 11:58:15 -07:00
Krrish Dholakia
f450b99368
test: skip test on end of life model
2024-08-31 14:34:16 -07:00
Ishaan Jaff
3966c545b4
fix flaky tests
2024-08-27 22:47:11 -07:00
Ishaan Jaff
f9c93c3adf
skip litellm.Timeout error
2024-08-27 21:32:27 -07:00
Ishaan Jaff
1b04380540
fix palm api is deactivated by google
2024-08-27 21:08:49 -07:00
Krrish Dholakia
3ec8a0d25d
fix: fix imports
2024-08-26 22:24:30 -07:00
Krrish Dholakia
b9d1296319
feat(utils.py): support gemini/vertex ai streaming function param usage
2024-08-26 11:23:45 -07:00
Krrish Dholakia
1b6db8359a
fix(utils.py): support openrouter streaming
...
Fixes https://github.com/BerriAI/litellm/issues/5080
2024-08-21 08:48:58 -07:00
Ishaan Jaff
a614c9f525
fix sagemaker old used test
2024-08-15 21:55:27 -07:00
Krrish Dholakia
3a1b3227d8
fix(utils.py): ignore none chunk in stream infinite loop check
...
Fixes https://github.com/BerriAI/litellm/issues/5158#issuecomment-2287156946
2024-08-13 15:06:44 -07:00
Krrish Dholakia
fdd9a07051
fix(utils.py): Break out of infinite streaming loop
...
Fixes https://github.com/BerriAI/litellm/issues/5158
2024-08-12 14:00:43 -07:00
Krrish Dholakia
3cf9148a4a
test: add vertex claude to streaming valid json str test
2024-08-07 10:18:17 -07:00
Krrish Dholakia
2ccb5a48b7
fix(bedrock_httpx.py): handle empty arguments returned during tool calling streaming
2024-08-07 09:54:50 -07:00
Krrish Dholakia
4919cc4d25
fix(anthropic.py): handle scenario where anthropic returns invalid json string for tool call while streaming
...
Fixes https://github.com/BerriAI/litellm/issues/5063
2024-08-07 09:24:11 -07:00
Krrish Dholakia
ca0a0bed46
fix(utils.py): fix anthropic streaming usage calculation
...
Fixes https://github.com/BerriAI/litellm/issues/4965
2024-08-01 14:45:54 -07:00
Ishaan Jaff
dce8e3efb8
testing fix - skip rate limit errors from anthropic api
2024-08-01 12:12:50 -07:00
Ishaan Jaff
54e1f18832
use timeouts for predibase - never use them in prod !
2024-07-30 22:54:01 -07:00
Ishaan Jaff
19ab0614c4
fix predibase tests
2024-07-30 22:49:33 -07:00
Krrish Dholakia
24395492aa
test: cleanup duplicate tests + add error handling for backend api errors
2024-07-30 21:47:52 -07:00
Krrish Dholakia
eecd93c81d
test(test_streaming.py): fix streaming test
2024-07-30 20:36:19 -07:00
Ishaan Jaff
6c0506a144
handle predibase failing streaming tests
2024-07-30 17:53:27 -07:00
Krrish Dholakia
84513c0254
test(test_streaming.py): move to mock implementation for sagemaker streaming tests
2024-07-30 15:34:27 -07:00
Krrish Dholakia
e1cbb397fe
test(test_streaming.py): handle predibase instability
2024-07-30 15:32:36 -07:00
Krrish Dholakia
a2fd8459fc
fix(utils.py): don't raise error on openai content filter during streaming - return as is
...
Fixes issue where we would raise an error vs. openai who return the chunk with finish reason as 'content_filter'
2024-07-25 19:50:52 -07:00
Krrish Dholakia
f64a3309d1
fix(utils.py): support raw response headers for streaming requests
2024-07-23 11:58:58 -07:00
Krrish Dholakia
a27454b8e3
fix(openai.py): support completion, streaming, async_streaming
2024-07-20 15:23:42 -07:00
Krrish Dholakia
96471c145e
fix(bedrock_httpx.py): support jamba streaming
2024-07-18 19:36:50 -07:00
Ishaan Jaff
3110abc832
shorten predibase streaming test
2024-07-17 17:50:42 -07:00
Krrish Dholakia
b2e46086dd
fix(utils.py): fix recreating model response object when stream usage is true
2024-07-11 21:01:12 -07:00
Ishaan Jaff
7efe9beac5
fix test_bedrock_httpx_streaming
2024-07-10 17:14:53 -07:00
Krish Dholakia
65bee737c5
Merge branch 'main' into litellm_gemini_stream_tool_calling
2024-07-06 19:07:31 -07:00
Ishaan Jaff
482913c2e9
test had "assert False" for some reason ?
2024-07-06 15:46:28 -07:00
Krrish Dholakia
faa88a1ab1
fix(vertex_httpx.py): support tool calling w/ streaming for vertex ai + gemini
2024-07-06 14:02:25 -07:00
Krrish Dholakia
f2dabc65be
fix(anthropic.py): fix anthropic tool calling + streaming
...
Fixes https://github.com/BerriAI/litellm/issues/4537
2024-07-04 16:30:24 -07:00
Krrish Dholakia
2e5a81f280
fix(utils.py): stream_options working across all providers
2024-07-03 20:40:46 -07:00
Krrish Dholakia
344268e053
fix(anthropic.py): support *real* anthropic tool calling + streaming
...
Parses each chunk and translates to openai format
2024-07-03 19:48:35 -07:00
Krrish Dholakia
d18dc2648c
fix(replicate.py): correctly handly invalid replicate response
2024-07-03 10:09:46 -07:00
Ishaan Jaff
03dbc29c85
fix test_completion_replicate_llama3_streaming
2024-06-29 19:51:07 -07:00
Krrish Dholakia
831745e710
test(test_streaming.py): try-except replicate api instability
2024-06-28 22:19:44 -07:00
Krrish Dholakia
0001683036
fix(cost_calculator.py): handle unexpected error in cost_calculator.py
2024-06-28 14:53:00 -07:00
Krrish Dholakia
a7122f91a1
fix(support-'alt=sse'-param): Fixes https://github.com/BerriAI/litellm/issues/4459
2024-06-28 10:40:47 -07:00
