Commit graph

11307 commits

Author SHA1 Message Date
Krish Dholakia
d121fc4775 fix(proxy/utils.py): auto-update if required view missing from db. raise warning for optional views. (#5675)
Prevents missing optional views from blocking proxy startup.
2024-09-12 22:15:44 -07:00
Ishaan Jaff
e7c22f63e7 [Fix-Router] Don't cooldown when only 1 deployment exists (#5673)
* fix get model list

* fix test custom callback router

* fix embedding fallback test

* fix router retry policy on AuthErrors

* fix router test

* add test for single deployments no cooldown test prod

* add test test_single_deployment_no_cooldowns_test_prod_mock_completion_calls
2024-09-12 19:14:58 -07:00
Ishaan Jaff
91dd3e11c4 [Feat-Perf] Use Batching + Squashing (#5645)
* use folder for slack alerting

* clean up slack alerting

* fix test alerting
2024-09-12 18:37:53 -07:00
Ishaan Jaff
b9036da070 Merge branch 'main' into litellm_add_openai_o1 2024-09-12 16:15:43 -07:00
Ishaan Jaff
e67f27e094 fix gcs logging 2024-09-12 15:24:04 -07:00
Ishaan Jaff
02d375e492 fix type errors 2024-09-12 14:49:43 -07:00
Ishaan Jaff
852132c0c7 fix handle user message 2024-09-12 14:34:32 -07:00
Ishaan Jaff
4e365443f2 fix linting 2024-09-12 14:15:18 -07:00
Ishaan Jaff
5fe95f7232 fix handle o1 not supporting system message 2024-09-12 14:09:13 -07:00
Ishaan Jaff
769753f7b5 fix pricing 2024-09-12 13:41:01 -07:00
Ishaan Jaff
d845842071 add o1 reasoning tests 2024-09-12 13:40:15 -07:00
Krish Dholakia
0b249278bb Refactor 'check_view_exists' logic (#5659)
* fix(proxy/utils.py): comment out auto-upsert logic in check_view_exists

Prevents proxy from failing on startup due to faulty logic

* fix(db/migration_scripts/create_views.py): fix 'DailyTagSpend' quotation on check

* fix(create_views.py): mongly global spend time period should be 30d not 20d

* fix(schema.prisma): index on startTime and endUser for efficient UI querying
2024-09-12 13:39:50 -07:00
Ishaan Jaff
f71083bdad add OpenAI o1 config 2024-09-12 13:22:59 -07:00
Ishaan Jaff
c72a47d287 mark test as flaky 2024-09-12 09:29:37 -07:00
Ishaan Jaff
5d660f239e fix config.yml 2024-09-12 09:28:45 -07:00
Ishaan Jaff
1985b4a508 ci/cd run again 2024-09-12 08:42:34 -07:00
Ishaan Jaff
5469ebe53a ci/cd run again 2024-09-12 08:31:17 -07:00
Krish Dholakia
dec53961f7 LiteLLM Minor Fixes and Improvements (11/09/2024) (#5634)
* fix(caching.py): set ttl for async_increment cache

fixes issue where ttl for redis client was not being set on increment_cache

Fixes https://github.com/BerriAI/litellm/issues/5609

* fix(caching.py): fix increment cache w/ ttl for sync increment cache on redis

Fixes https://github.com/BerriAI/litellm/issues/5609

* fix(router.py): support adding retry policy + allowed fails policy via config.yaml

* fix(router.py): don't cooldown single deployments

No point, as there's no other deployment to loadbalance with.

* fix(user_api_key_auth.py): support setting allowed email domains on jwt tokens

Closes https://github.com/BerriAI/litellm/issues/5605

* docs(token_auth.md): add user upsert + allowed email domain to jwt auth docs

* fix(litellm_pre_call_utils.py): fix dynamic key logging when team id is set

Fixes issue where key logging would not be set if team metadata was not none

* fix(secret_managers/main.py): load environment variables correctly

Fixes issue where os.environ/ was not being loaded correctly

* test(test_router.py): fix test

* feat(spend_tracking_utils.py): support logging additional usage params - e.g. prompt caching values for deepseek

* test: fix tests

* test: fix test

* test: fix test

* test: fix test

* test: fix test
2024-09-11 22:36:06 -07:00
Ishaan Jaff
f7fc14ac34 Merge branch 'main' into litellm_otel_fixes 2024-09-11 18:06:29 -07:00
Ishaan Jaff
6f9d7a7df8 Merge pull request #5638 from BerriAI/litellm_langsmith_perf
[Langsmith Perf Improvement] Use /batch for Langsmith Logging
2024-09-11 17:43:26 -07:00
Ishaan Jaff
73d838e7c8 fix move logic to custom_batch_logger 2024-09-11 16:19:24 -07:00
steffen-sbt
357dd3cad5 Add the option to specify a schema in the postgres DB, also modify docs (#5640) 2024-09-11 14:53:52 -07:00
Ishaan Jaff
3e0c4448cd use vars for batch size and flush interval seconds 2024-09-11 14:40:58 -07:00
Ishaan Jaff
c72c8c0383 fix otel use sensible defaults 2024-09-11 14:24:04 -07:00
Ishaan Jaff
36fe499d53 fix vtx test 2024-09-11 14:17:03 -07:00
Ishaan Jaff
53734dbcfc fix langsmith tenacity 2024-09-11 13:48:44 -07:00
Ishaan Jaff
d0ae85a7bb use lock to flush events to langsmith 2024-09-11 13:27:16 -07:00
Ishaan Jaff
c5f64ef99e add better debugging for flush interval 2024-09-11 13:02:34 -07:00
Ishaan Jaff
385286c089 use tenacity for langsmith 2024-09-11 12:41:22 -07:00
Ishaan Jaff
67b3ce8740 fix langsmith clear logged queue on success 2024-09-11 11:56:24 -07:00
Krish Dholakia
7f47c48b35 LiteLLM Minor Fixes and Improvements (09/10/2024) (#5618)
* fix(cost_calculator.py): move to debug for noisy warning message on cost calculation error

Fixes https://github.com/BerriAI/litellm/issues/5610

* fix(databricks/cost_calculator.py): Handles model name issues for databricks models

* fix(main.py): fix stream chunk builder for multiple tool calls

Fixes https://github.com/BerriAI/litellm/issues/5591

* fix: correctly set user_alias when passed in

Fixes https://github.com/BerriAI/litellm/issues/5612

* fix(types/utils.py): allow passing role for message object

https://github.com/BerriAI/litellm/issues/5621

* fix(litellm_logging.py): Fix langfuse logging across multiple projects

Fixes issue where langfuse logger was re-using the old logging object

* feat(proxy/_types.py): support adding key-based tags for tag-based routing

Enable tag based routing at key-level

* fix(proxy/_types.py): fix inheritance

* test(test_key_generate_prisma.py): fix test

* test: fix test

* fix(litellm_logging.py): return used callback object
2024-09-11 11:30:29 -07:00
Ishaan Jaff
a053464fc5 langsmith use batching for logging 2024-09-11 11:28:27 -07:00
Ishaan Jaff
283370aeef fix langsmith_batch_size 2024-09-11 11:25:33 -07:00
Ishaan Jaff
530cc34866 stash - langsmith use batching for logging 2024-09-11 08:06:56 -07:00
Ishaan Jaff
11099ff057 ci/cd run again 2024-09-10 19:21:13 -07:00
Ishaan Jaff
4515f43976 Merge pull request #5623 from BerriAI/litellm_vertex_use_async_for_getting_token
[Feat-Vertex Perf] Use async func to get auth credentials
2024-09-10 18:53:48 -07:00
Ishaan Jaff
cfe084c4f5 add doc string to vertex llm base 2024-09-10 18:52:43 -07:00
Ishaan Jaff
dbc85c3dfe fix gemini streaming test 2024-09-10 17:50:24 -07:00
Ishaan Jaff
64fab8a56b fix test get token url 2024-09-10 17:38:03 -07:00
Ishaan Jaff
2762bee138 Merge pull request #5571 from jalammar/cohere-updated-models
Add Cohere refresh models and update pricing
2024-09-10 17:22:51 -07:00
Ishaan Jaff
14a7b9d7c1 fix gemini test 2024-09-10 17:20:01 -07:00
Ishaan Jaff
c8fe600dbf fix case when gemini is used 2024-09-10 17:06:45 -07:00
Ishaan Jaff
7891b3742c fix vertex use async func to set auth creds 2024-09-10 16:12:18 -07:00
Ishaan Jaff
b7e11bb22e Merge pull request #5622 from BerriAI/litellm_fix_auth_refresh_vertex
[Feat-Perf Improvement Vertex] Only Refresh credentials when token is expired
2024-09-10 15:03:35 -07:00
Ishaan Jaff
536ca7d516 Merge branch 'main' into litellm_use_helper_to_get_httpx_clients 2024-09-10 15:02:54 -07:00
Ishaan Jaff
d35e35bab6 fix bedrock get async client 2024-09-10 14:17:18 -07:00
Ishaan Jaff
3ac9711524 fix types for vertex project id 2024-09-10 14:02:15 -07:00
Ishaan Jaff
852a2baa39 Merge pull request #5619 from BerriAI/litellm_vertex_use_get_httpx_client
[Fix-Perf] Vertex AI cache httpx clients
2024-09-10 13:59:39 -07:00
Ishaan Jaff
571490d0f2 fix getting params 2024-09-10 13:54:42 -07:00
Ishaan Jaff
0008c6fe6c fix vertex only refresh auth when required 2024-09-10 13:49:28 -07:00