Krish Dholakia
1e7e538261
LiteLLM Minor fixes + improvements (08/04/2024) ( #5505 )
...
* Minor IAM AWS OIDC Improvements (#5246 )
* AWS IAM: Temporary tokens are valid across all regions after being issued, so it is wasteful to request one for each region.
* AWS IAM: Include an inline policy, to help reduce misuse of overly permissive IAM roles.
* (test_bedrock_completion.py): Ensure we are testing cross AWS region OIDC flow.
* fix(router.py): log rejected requests
Fixes https://github.com/BerriAI/litellm/issues/5498
* refactor: don't use verbose_logger.exception, if exception is raised
User might already have handling for this. But alerting systems in prod will raise this as an unhandled error.
* fix(datadog.py): support setting datadog source as an env var
Fixes https://github.com/BerriAI/litellm/issues/5508
* docs(logging.md): add dd_source to datadog docs
* fix(proxy_server.py): expose `/customer/list` endpoint for showing all customers
* (bedrock): Fix usage with Cloudflare AI Gateway, and proxies in general. (#5509 )
* feat(anthropic.py): support 'cache_control' param for content when it is a string
* Revert "(bedrock): Fix usage with Cloudflare AI Gateway, and proxies in gener…" (#5519 )
This reverts commit 3fac0349c2
2024-09-04 22:16:55 -07:00
Krish Dholakia
be3c7b401e
LiteLLM Minor fixes + improvements (08/03/2024) ( #5488 )
...
* fix(internal_user_endpoints.py): set budget_reset_at for /user/update
* fix(vertex_and_google_ai_studio_gemini.py): handle accumulated json
Fixes https://github.com/BerriAI/litellm/issues/5479
* fix(vertex_ai_and_gemini.py): fix assistant message function call when content is not None
Fixes https://github.com/BerriAI/litellm/issues/5490
* fix(proxy_server.py): generic state uuid for okta sso
* fix(lago.py): improve debug logs
Debugging for https://github.com/BerriAI/litellm/issues/5477
* docs(bedrock.md): add bedrock cross-region inferencing to docs
* fix(azure.py): return azure response headers on aembedding call
* feat(azure.py): return azure response headers for `/audio/transcription`
* fix(types/utils.py): standardize deepseek / anthropic prompt caching usage information
Closes https://github.com/BerriAI/litellm/issues/5285
* docs(usage.md): add docs on litellm usage object
* test(test_completion.py): mark flaky test
2024-09-03 21:21:34 -07:00
Krish Dholakia
37f9705d6e
Bedrock Embeddings refactor + model support ( #5462 )
...
* refactor(bedrock): initial commit to refactor bedrock to a folder
Improve code readability + maintainability
* refactor: more refactor work
* fix: fix imports
* feat(bedrock/embeddings.py): support translating embedding into amazon embedding formats
* fix: fix linting errors
* test: skip test on end of life model
* fix(cohere/embed.py): fix linting error
* fix(cohere/embed.py): fix typing
* fix(cohere/embed.py): fix post-call logging for cohere embedding call
* test(test_embeddings.py): fix error message assertion in test
2024-09-01 13:29:58 -07:00
Krish Dholakia
dd7b008161
fix: Minor LiteLLM Fixes + Improvements (29/08/2024) ( #5436 )
...
* fix(model_checks.py): support returning wildcard models on `/v1/models`
Fixes https://github.com/BerriAI/litellm/issues/4903
* fix(bedrock_httpx.py): support calling bedrock via api_base
Closes https://github.com/BerriAI/litellm/pull/4587
* fix(litellm_logging.py): only leave last 4 char of gemini key unmasked
Fixes https://github.com/BerriAI/litellm/issues/5433
* feat(router.py): support setting 'weight' param for models on router
Closes https://github.com/BerriAI/litellm/issues/5410
* test(test_bedrock_completion.py): add unit test for custom api base
* fix(model_checks.py): handle no "/" in model
2024-08-29 22:40:25 -07:00
Krrish Dholakia
6431af0678
fix(bedrock_httpx.py): support 'Auth' header as extra_header
...
Fixes https://github.com/BerriAI/litellm/issues/5389#issuecomment-2313677977
2024-08-27 16:08:54 -07:00
Krrish Dholakia
70bf8bd4f4
feat(factory.py): enable 'user_continue_message' for interweaving user/assistant messages when provider requires it
...
allows bedrock to be used with autogen
2024-08-22 11:03:33 -07:00
Ishaan Jaff
89ba7b3e11
pass trace through for bedrock guardrails
2024-08-16 09:10:56 -07:00
Krrish Dholakia
c1279ed809
fix(bedrock_httpx.py): fix error code for not found provider/model combo to be 404
2024-08-13 20:36:12 -07:00
Krrish Dholakia
66d77f177f
fix(bedrock_httpx.py): raise bad request error if invalid bedrock model given
2024-08-13 19:27:06 -07:00
Krrish Dholakia
526b196f83
fix(bedrock_httpx.py): handle empty stop string
2024-08-13 07:30:30 -07:00
Krrish Dholakia
6e8d2856b0
fix(bedrock_httpx.py): handle bedrock empty system message
2024-08-13 07:17:17 -07:00
Ishaan Jaff
42617c207a
test bedrock tool call names
2024-08-09 17:14:56 -07:00
Ishaan Jaff
6dc9b39095
test invalid tool namehandling
2024-08-09 13:26:21 -07:00
David Manouchehri
507529e8df
(test_bedrock_completion.py) - Use FIPS endpoints for testing.
2024-07-31 16:51:58 +00:00
Ishaan Jaff
46555ab78b
test - bedrock guardrailConfig
2024-07-29 14:13:08 -07:00
Krrish Dholakia
e2d275f1b7
fix(utils.py): add exception mapping for bedrock image internal server error
2024-07-19 19:30:41 -07:00
Krrish Dholakia
0decc36bed
fix(factory.py): handle message content being a list instead of string
...
Fixes https://github.com/BerriAI/litellm/issues/4679
2024-07-12 19:00:39 -07:00
Krrish Dholakia
88eb25da5c
fix(bedrock_httpx.py): handle user error - malformed system prompt
...
if user passes in system prompt as a list of content blocks, handle that
2024-07-12 08:28:50 -07:00
Krrish Dholakia
79670ab82e
fix(main.py): get the region name from boto3 client if dynamic var not set
2024-07-02 09:24:07 -07:00
Ishaan Jaff
bad49a270d
fix test test_provisioned_throughput
2024-06-29 19:41:05 -07:00
Brian Schultheiss
632b7ce17d
Resolve merge conflicts
2024-06-29 15:53:02 -07:00
Krrish Dholakia
151d19960e
fix(bedrock_httpx.py): Fix https://github.com/BerriAI/litellm/issues/4415
2024-06-26 16:19:46 -07:00
Brian Schultheiss
09492cceba
Update tests to verify streaming works
2024-06-25 14:33:40 -07:00
Brian Schultheiss
5a6588342c
added test for change
2024-06-23 15:19:54 -07:00
David Manouchehri
02aaaf5976
Merge remote-tracking branch 'upstream/main' into oidc-bedrock-httpx-caching-part-1
2024-06-11 15:42:31 +00:00
Krrish Dholakia
e66b3d264f
fix(factory.py): handle bedrock claude image url's
2024-06-07 10:04:03 -07:00
David Manouchehri
3410367610
test(test_bedrock_completion.py): Add tests to ensure caching isn't breaking anything.
2024-06-01 15:22:08 +00:00
David Manouchehri
d70d484e10
Fix: Use David's AWS account to pass unit tests.
2024-05-31 13:50:49 +00:00
David Manouchehri
08ee4519b6
Add unit test for bedrock httpx oidc auth.
2024-05-31 12:44:53 +00:00
Krrish Dholakia
6b50e656b8
fix(main.py): pass extra headers through for async calls
2024-05-27 19:11:40 -07:00
Krrish Dholakia
24eb79da91
test(test_bedrock_completion.py): refactor test bedrock headers test
2024-05-27 19:01:07 -07:00
Krrish Dholakia
d2e14ca833
fix(bedrock_httpx.py): fix bedrock ptu model id str encoding
...
Fixes https://github.com/BerriAI/litellm/issues/3805
2024-05-25 10:54:01 -07:00
Krrish Dholakia
00af8e350f
fix(bedrock_httpx.py): support bedrock ptu's
...
Fixes https://github.com/BerriAI/litellm/issues/3805
2024-05-24 23:02:04 -07:00
Ishaan Jaff
0ddaf320ef
fix test - retry claude-3 image error 3 times
2024-05-20 16:17:09 -07:00
Ishaan Jaff
d77aea7253
Update test_bedrock_completion.py
...
cc @Manouchehri - can u lmk what needs to be in our env to pass this test ?
attaching the test log here: cda0de1d
2024-05-11 16:30:29 -07:00
David Manouchehri
3ee0328b04
feat(bedrock.py): Support using OIDC tokens.
2024-05-07 15:46:54 +00:00
Lucca Zenóbio
b22517845e
Merge branch 'main' into main
2024-05-06 09:40:23 -03:00
Krrish Dholakia
09d7121af2
fix(bedrock.py): map finish reason for bedrock
2024-05-04 12:45:40 -07:00
Lucca Zenobio
a9e2ef6212
test
2024-04-29 10:05:30 -03:00
Nilanjan De
5113d47023
add test
2024-04-19 00:42:48 +04:00
Ishaan Jaff
5393930701
fix function calling prompt - ask llm to respond in fahrenheit
2024-04-16 21:09:53 -07:00
Krrish Dholakia
2fabff06c0
fix(bedrock.py): fix supported openai params for bedrock claude 3
2024-03-23 16:02:15 -07:00
Krrish Dholakia
caa17d484a
fix(bedrock.py): working image calls to claude 3
2024-03-04 18:12:47 -08:00
Krrish Dholakia
818c29516d
fix(bedrock.py): support bedrock anthropic claude 3 tool calling
2024-03-04 17:47:28 -08:00
Krrish Dholakia
478307d4cf
fix(bedrock.py): support anthropic messages api on bedrock (claude-3)
2024-03-04 17:15:47 -08:00
Tim Xia
2321f19fe7
comment out tests
2024-03-01 23:28:25 -05:00
Tim Xia
58ed6e77de
add assertion for test
2024-03-01 23:27:02 -05:00
Tim Xia
8907b2733a
skip test but it did work locally
2024-03-01 22:49:31 -05:00
Tim Xia
739f4f05f6
add support for bedrock mistral models
2024-03-01 22:45:54 -05:00
Krrish Dholakia
9ec8e33a5a
test: skip aws test - aws account suspended
2024-02-28 14:27:01 -08:00
