phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-27 03:34:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
17541 commits 2497 branches 897 tags 550 MiB
Commit graph

60 commits

Author SHA1 Message Date
Krish Dholakia
6fdee99632 LiteLLM Minor fixes + improvements (08/04/2024) (#5505) 
* Minor IAM AWS OIDC Improvements (#5246)

* AWS IAM: Temporary tokens are valid across all regions after being issued, so it is wasteful to request one for each region.

* AWS IAM: Include an inline policy, to help reduce misuse of overly permissive IAM roles.

* (test_bedrock_completion.py): Ensure we are testing cross AWS region OIDC flow.

* fix(router.py): log rejected requests

Fixes https://github.com/BerriAI/litellm/issues/5498

* refactor: don't use verbose_logger.exception, if exception is raised

User might already have handling for this. But alerting systems in prod will raise this as an unhandled error.

* fix(datadog.py): support setting datadog source as an env var

Fixes https://github.com/BerriAI/litellm/issues/5508

* docs(logging.md): add dd_source to datadog docs

* fix(proxy_server.py): expose `/customer/list` endpoint for showing all customers

* (bedrock): Fix usage with Cloudflare AI Gateway, and proxies in general. (#5509)

* feat(anthropic.py): support 'cache_control' param for content when it is a string

* Revert "(bedrock): Fix usage with Cloudflare AI Gateway, and proxies in gener…" (#5519)

This reverts commit 3fac0349c2.

* refactor: ci/cd run again

---------

Co-authored-by: David Manouchehri <david.manouchehri@ai.moda>
 2024-09-04 22:16:55 -07:00
Krish Dholakia
8eb7cb5300 LiteLLM Minor fixes + improvements (08/03/2024) (#5488) 
* fix(internal_user_endpoints.py): set budget_reset_at for /user/update

* fix(vertex_and_google_ai_studio_gemini.py): handle accumulated json

Fixes https://github.com/BerriAI/litellm/issues/5479

* fix(vertex_ai_and_gemini.py): fix assistant message function call when content is not None

Fixes https://github.com/BerriAI/litellm/issues/5490

* fix(proxy_server.py): generic state uuid for okta sso

* fix(lago.py): improve debug logs

Debugging for https://github.com/BerriAI/litellm/issues/5477

* docs(bedrock.md): add bedrock cross-region inferencing to docs

* fix(azure.py): return azure response headers on aembedding call

* feat(azure.py): return azure response headers for `/audio/transcription`

* fix(types/utils.py): standardize deepseek / anthropic prompt caching usage information

Closes https://github.com/BerriAI/litellm/issues/5285

* docs(usage.md): add docs on litellm usage object

* test(test_completion.py): mark flaky test
 2024-09-03 21:21:34 -07:00
Krish Dholakia
e474c3665a Bedrock Embeddings refactor + model support (#5462) 
* refactor(bedrock): initial commit to refactor bedrock to a folder

Improve code readability + maintainability

* refactor: more refactor work

* fix: fix imports

* feat(bedrock/embeddings.py): support translating embedding into amazon embedding formats

* fix: fix linting errors

* test: skip test on end of life model

* fix(cohere/embed.py): fix linting error

* fix(cohere/embed.py): fix typing

* fix(cohere/embed.py): fix post-call logging for cohere embedding call

* test(test_embeddings.py): fix error message assertion in test
 2024-09-01 13:29:58 -07:00
Krish Dholakia
321b0961b5 fix: Minor LiteLLM Fixes + Improvements (29/08/2024) (#5436) 
* fix(model_checks.py): support returning wildcard models on `/v1/models`

Fixes https://github.com/BerriAI/litellm/issues/4903

* fix(bedrock_httpx.py): support calling bedrock via api_base

Closes https://github.com/BerriAI/litellm/pull/4587

* fix(litellm_logging.py): only leave last 4 char of gemini key unmasked

Fixes https://github.com/BerriAI/litellm/issues/5433

* feat(router.py): support setting 'weight' param for models on router

Closes https://github.com/BerriAI/litellm/issues/5410

* test(test_bedrock_completion.py): add unit test for custom api base

* fix(model_checks.py): handle no "/" in model
 2024-08-29 22:40:25 -07:00
Krrish Dholakia
722ccba323 fix(bedrock_httpx.py): support 'Auth' header as extra_header 
Fixes https://github.com/BerriAI/litellm/issues/5389#issuecomment-2313677977
 2024-08-27 16:08:54 -07:00
Krrish Dholakia
900d8ecbf0 feat(factory.py): enable 'user_continue_message' for interweaving user/assistant messages when provider requires it 
allows bedrock to be used with autogen
 2024-08-22 11:03:33 -07:00
Ishaan Jaff
98c9191f84 pass trace through for bedrock guardrails 2024-08-16 09:10:56 -07:00
Krrish Dholakia
76c48bf5d9 fix(bedrock_httpx.py): fix error code for not found provider/model combo to be 404 2024-08-13 20:36:12 -07:00
Krrish Dholakia
4a21554b4f fix(bedrock_httpx.py): raise bad request error if invalid bedrock model given 2024-08-13 19:27:06 -07:00
Krrish Dholakia
08a584035c fix(bedrock_httpx.py): handle empty stop string 2024-08-13 07:30:30 -07:00
Krrish Dholakia
8454cfb329 fix(bedrock_httpx.py): handle bedrock empty system message 2024-08-13 07:17:17 -07:00
Ishaan Jaff
cc7b9fa43e test bedrock tool call names 2024-08-09 17:14:56 -07:00
Ishaan Jaff
631ea02840 test invalid tool namehandling 2024-08-09 13:26:21 -07:00
David Manouchehri
afeba87a2f (test_bedrock_completion.py) - Use FIPS endpoints for testing. 2024-07-31 16:51:58 +00:00
Ishaan Jaff
8c0c727e21 test - bedrock guardrailConfig 2024-07-29 14:13:08 -07:00
Krrish Dholakia
b838ff22d5 fix(utils.py): add exception mapping for bedrock image internal server error 2024-07-19 19:30:41 -07:00
Krrish Dholakia
fd16974996 fix(factory.py): handle message content being a list instead of string 
Fixes https://github.com/BerriAI/litellm/issues/4679
 2024-07-12 19:00:39 -07:00
Krrish Dholakia
4895cc7c01 fix(bedrock_httpx.py): handle user error - malformed system prompt 
if user passes in system prompt as a list of content blocks, handle that
 2024-07-12 08:28:50 -07:00
Krrish Dholakia
39348317d2 fix(main.py): get the region name from boto3 client if dynamic var not set 2024-07-02 09:24:07 -07:00
Ishaan Jaff
fee09a6b7f fix test test_provisioned_throughput 2024-06-29 19:41:05 -07:00
Brian Schultheiss
195cab8e07 Resolve merge conflicts 2024-06-29 15:53:02 -07:00
Krrish Dholakia
7fe4b3d8f4 fix(bedrock_httpx.py): Fix https://github.com/BerriAI/litellm/issues/4415 2024-06-26 16:19:46 -07:00
Brian Schultheiss
728e23f7c6 Update tests to verify streaming works 2024-06-25 14:33:40 -07:00
Brian Schultheiss
e405114401 added test for change 2024-06-23 15:19:54 -07:00
David Manouchehri
b0c3bd4748 Merge remote-tracking branch 'upstream/main' into oidc-bedrock-httpx-caching-part-1 2024-06-11 15:42:31 +00:00
Krrish Dholakia
672dcf0c6f fix(factory.py): handle bedrock claude image url's 2024-06-07 10:04:03 -07:00
David Manouchehri
516c9026d5 test(test_bedrock_completion.py): Add tests to ensure caching isn't breaking anything. 2024-06-01 15:22:08 +00:00
David Manouchehri
3ba7f2d931 Fix: Use David's AWS account to pass unit tests. 2024-05-31 13:50:49 +00:00
David Manouchehri
83cc226369 Add unit test for bedrock httpx oidc auth. 2024-05-31 12:44:53 +00:00
Krrish Dholakia
4aa7e0b17c fix(main.py): pass extra headers through for async calls 2024-05-27 19:11:40 -07:00
Krrish Dholakia
0d3867ea5c test(test_bedrock_completion.py): refactor test bedrock headers test 2024-05-27 19:01:07 -07:00
Krrish Dholakia
68a8b23b59 fix(bedrock_httpx.py): fix bedrock ptu model id str encoding 
Fixes https://github.com/BerriAI/litellm/issues/3805
 2024-05-25 10:54:01 -07:00
Krrish Dholakia
1a9e4d99ac fix(bedrock_httpx.py): support bedrock ptu's 
Fixes https://github.com/BerriAI/litellm/issues/3805
 2024-05-24 23:02:04 -07:00
Ishaan Jaff
bac23e43a6 fix test - retry claude-3 image error 3 times 2024-05-20 16:17:09 -07:00
Ishaan Jaff
e79a7c50d1 Update test_bedrock_completion.py 
cc @Manouchehri - can u lmk what needs to be in our env to pass this test ? 

attaching the test log here: cda0de1d-3851-469c-8851-ef12dc27fab2/jobs/20819/tests#failed-test-0
 2024-05-11 16:30:29 -07:00
David Manouchehri
47449d19e8 feat(bedrock.py): Support using OIDC tokens. 2024-05-07 15:46:54 +00:00
Lucca Zenóbio
146a49103f Merge branch 'main' into main 2024-05-06 09:40:23 -03:00
Krrish Dholakia
8f22092c65 fix(bedrock.py): map finish reason for bedrock 2024-05-04 12:45:40 -07:00
Lucca Zenobio
5d20b9d7d6 test 2024-04-29 10:05:30 -03:00
Nilanjan De
8163d9d0a8 add test 2024-04-19 00:42:48 +04:00
Ishaan Jaff
b00ad976af fix function calling prompt - ask llm to respond in fahrenheit 2024-04-16 21:09:53 -07:00
Krrish Dholakia
47b2b0df17 fix(bedrock.py): fix supported openai params for bedrock claude 3 2024-03-23 16:02:15 -07:00
Krrish Dholakia
f0a5e0ffe9 fix(bedrock.py): working image calls to claude 3 2024-03-04 18:12:47 -08:00
Krrish Dholakia
dc1f347a63 fix(bedrock.py): support bedrock anthropic claude 3 tool calling 2024-03-04 17:47:28 -08:00
Krrish Dholakia
dad65ca602 fix(bedrock.py): support anthropic messages api on bedrock (claude-3) 2024-03-04 17:15:47 -08:00
Tim Xia
5b63ba6aa5 comment out tests 2024-03-01 23:28:25 -05:00
Tim Xia
291aa4f6d0 add assertion for test 2024-03-01 23:27:02 -05:00
Tim Xia
8a0385a51b skip test but it did work locally 2024-03-01 22:49:31 -05:00
Tim Xia
6f389097d2 add support for bedrock mistral models 2024-03-01 22:45:54 -05:00
Krrish Dholakia
267d365067 test: skip aws test - aws account suspended 2024-02-28 14:27:01 -08:00