Krish Dholakia
e0d81434ed
LiteLLM minor fixes + improvements (31/08/2024) ( #5464 )
...
* fix(vertex_endpoints.py): fix vertex ai pass through endpoints
* test(test_streaming.py): skip model due to end of life
* feat(custom_logger.py): add special callback for model hitting tpm/rpm limits
Closes https://github.com/BerriAI/litellm/issues/4096
2024-09-01 13:31:42 -07:00
Krish Dholakia
37f9705d6e
Bedrock Embeddings refactor + model support ( #5462 )
...
* refactor(bedrock): initial commit to refactor bedrock to a folder
Improve code readability + maintainability
* refactor: more refactor work
* fix: fix imports
* feat(bedrock/embeddings.py): support translating embedding into amazon embedding formats
* fix: fix linting errors
* test: skip test on end of life model
* fix(cohere/embed.py): fix linting error
* fix(cohere/embed.py): fix typing
* fix(cohere/embed.py): fix post-call logging for cohere embedding call
* test(test_embeddings.py): fix error message assertion in test
2024-09-01 13:29:58 -07:00
Ishaan Jaff
d9f3ac5159
Merge pull request #5463 from BerriAI/litellm_track_error_per_model
...
[Feat - Prometheus] - Track error_code, model metric
2024-08-31 16:36:04 -07:00
Ishaan Jaff
56f10224df
Merge pull request #5457 from BerriAI/litellm_track_spend_logs_for_vertex_pass_through_endpoints
...
[Feat-Proxy] track spend logs for vertex pass through endpoints
2024-08-31 16:30:15 -07:00
Krish Dholakia
65a9c933ad
anthropic prompt caching cost tracking ( #5453 )
...
* fix(utils.py): support 'drop_params' for embedding requests
Fixes https://github.com/BerriAI/litellm/issues/5444
* feat(anthropic/cost_calculation.py): Support calculating cost for prompt caching on anthropic
* feat(types/utils.py): allows us to migrate to openai's equivalent, once that comes out
* fix: fix linting errors
* test: mark flaky test
2024-08-31 14:50:52 -07:00
Krish Dholakia
9c8f1d7815
anthropic prompt caching cost tracking ( #5453 )
...
* fix(utils.py): support 'drop_params' for embedding requests
Fixes https://github.com/BerriAI/litellm/issues/5444
* feat(anthropic/cost_calculation.py): Support calculating cost for prompt caching on anthropic
* feat(types/utils.py): allows us to migrate to openai's equivalent, once that comes out
* fix: fix linting errors
* test: mark flaky test
2024-08-31 14:09:35 -07:00
Ishaan Jaff
6ab601432b
feat prometheus add metric for failure / model
2024-08-31 10:05:23 -07:00
Ishaan Jaff
b35bfb0302
fix cost tracking for vertex ai native
2024-08-31 08:22:27 -07:00
Ishaan Jaff
414d2dcb52
call spend logs endpoint
2024-08-30 16:35:07 -07:00
Ishaan Jaff
f3f85f6141
add test for vertex basic pass throgh
2024-08-30 16:26:00 -07:00
Ishaan Jaff
8ed0ffea54
fix use existing custom_auth.py
2024-08-30 16:22:28 -07:00
Ishaan Jaff
36fda54970
allow pass through routes as LLM API routes
2024-08-30 16:08:44 -07:00
Ishaan Jaff
f50374e81d
use helper class for pass through success handler
2024-08-30 15:52:47 -07:00
Ishaan Jaff
e1e1e2e566
add example custom
2024-08-30 15:46:45 -07:00
Ishaan Jaff
7f303db955
Merge pull request #5450 from BerriAI/litellm_load_config_from_gcs
...
[Feat-Proxy] Load config.yaml from GCS Bucket
2024-08-30 12:08:54 -07:00
Ishaan Jaff
50014aa78e
vertex forward all headers from vertex
2024-08-30 11:05:23 -07:00
Ishaan Jaff
7d746064ab
add gcs bucket base
2024-08-30 10:41:39 -07:00
Ishaan Jaff
051ac50fca
use helper to get_config_file_contents_from_gcs
2024-08-30 10:26:42 -07:00
Ishaan Jaff
a9ca183021
Merge pull request #5438 from BerriAI/litellm_show_error_types_swagger
...
[Feat-Proxy] Show all exceptioons types on swagger for LiteLLM Proxy
2024-08-30 07:21:23 -07:00
Krish Dholakia
dd7b008161
fix: Minor LiteLLM Fixes + Improvements (29/08/2024) ( #5436 )
...
* fix(model_checks.py): support returning wildcard models on `/v1/models`
Fixes https://github.com/BerriAI/litellm/issues/4903
* fix(bedrock_httpx.py): support calling bedrock via api_base
Closes https://github.com/BerriAI/litellm/pull/4587
* fix(litellm_logging.py): only leave last 4 char of gemini key unmasked
Fixes https://github.com/BerriAI/litellm/issues/5433
* feat(router.py): support setting 'weight' param for models on router
Closes https://github.com/BerriAI/litellm/issues/5410
* test(test_bedrock_completion.py): add unit test for custom api base
* fix(model_checks.py): handle no "/" in model
2024-08-29 22:40:25 -07:00
Ishaan Jaff
ad88c7d0a8
show all error types on swagger
2024-08-29 18:50:41 -07:00
Ishaan Jaff
e329c4509a
Merge branch 'main' into litellm_add_tag_control_team
2024-08-29 17:34:40 -07:00
Ishaan Jaff
010f526226
Merge branch 'main' into litellm_fwd_vtx_sdk_headers
2024-08-29 17:24:31 -07:00
Ishaan Jaff
a4b88c16dc
fix indentation
2024-08-29 17:01:23 -07:00
Ishaan Jaff
748cc80783
fix auth checks for provider routes
2024-08-29 16:40:46 -07:00
Ishaan Jaff
41a5daf20c
add test for vertex sdk foward headers
2024-08-29 15:50:54 -07:00
Ishaan Jaff
9fb188a3b6
vertex add vertex endpoints
2024-08-29 15:49:25 -07:00
Krrish Dholakia
601945d114
docs(docker_quick_start.md): add new quick start doc for litellm proxy
2024-08-29 15:35:39 -07:00
Ishaan Jaff
da2cefc45a
fix team based tag routing
2024-08-29 14:37:44 -07:00
Ishaan Jaff
242f66054d
enable_tag_filtering
2024-08-29 14:14:46 -07:00
Ishaan Jaff
f592aeaa38
add test_chat_completion_with_no_tags
2024-08-29 13:54:11 -07:00
Ishaan Jaff
2cb4882e70
define tags on model list
2024-08-29 13:06:39 -07:00
Ishaan Jaff
ff42962750
add_team_based_tags_to_metadata
2024-08-29 13:06:03 -07:00
Ishaan Jaff
ffeb5ce22a
add set / update tags for a team
2024-08-29 13:05:00 -07:00
Ishaan Jaff
5e590e7326
allow settings tags per team
2024-08-29 13:03:49 -07:00
Krish Dholakia
8ae0fc693f
Merge pull request #5414 from BerriAI/litellm_main_staging
...
fixes: minor litellm fixes
2024-08-28 22:01:01 -07:00
Krrish Dholakia
2119030738
fix(team_endpoints.py): update to include the budget in the response
2024-08-28 21:58:41 -07:00
Krrish Dholakia
18689c25e9
feat(team_endpoints.py): return team member budgets in /team/info call
...
Fixes https://github.com/BerriAI/litellm/issues/5390
2024-08-28 19:14:01 -07:00
Krish Dholakia
a857f4a8ee
Merge branch 'main' into litellm_main_staging
2024-08-28 18:05:27 -07:00
Krrish Dholakia
f0fb8bdf45
fix(router.py): fix cooldown check
2024-08-28 16:38:42 -07:00
Ishaan Jaff
a982cb7ae5
Merge pull request #5420 from BerriAI/litellm_add_oauth2_mapping
...
[Feat-Proxy] Add hook for oauth2 proxy headers
2024-08-28 16:34:26 -07:00
Ishaan Jaff
8948583ddc
Merge branch 'main' into litellm_add_request_body_safety_checks
2024-08-28 16:14:46 -07:00
Ishaan Jaff
946e3b63e1
add hook for oauth2 proxy
2024-08-28 15:51:20 -07:00
Ishaan Jaff
6c49083573
Merge pull request #5409 from BerriAI/litellm_ui_budgets
...
[UI] Allow editing budget duration
2024-08-28 14:43:57 -07:00
Ishaan Jaff
5ad6d4cf77
new ui build
2024-08-28 14:43:33 -07:00
Ishaan Jaff
9c2d974f5e
fix checking request body
2024-08-28 14:10:09 -07:00
Ishaan Jaff
d3a5162103
add checks for safe request body
2024-08-28 14:07:53 -07:00
Ishaan Jaff
ba1912afd1
add check for is_request_body_safe
2024-08-28 13:40:14 -07:00
Krrish Dholakia
325538f8a4
fix(key_management_endpoints.py): expose 'key' param, for setting your own key value
2024-08-28 12:27:32 -07:00
Krrish Dholakia
0861180bfe
fix(rds_iam_token.py): support common aws env var's - AWS_ROLE_ARN, AWS_WEB_IDENTITY_TOKEN_FILE
2024-08-28 12:18:45 -07:00
