Commit graph

17692 commits

Author SHA1 Message Date
Ishaan Jaff
85acdb9193
[Feat] Add max_completion_tokens param (#5691)
* add max_completion_tokens

* add max_completion_tokens

* add max_completion_tokens support for OpenAI models

* add max_completion_tokens param

* add max_completion_tokens for bedrock converse models

* add test for converse maxTokens

* fix openai o1 param mapping test

* move test optional params

* add max_completion_tokens for anthropic api

* fix conftest

* add max_completion tokens for vertex ai partner models

* add max_completion_tokens for fireworks ai

* add max_completion_tokens for hf rest api

* add test for param mapping

* add param mapping for vertex, gemini + testing

* predibase is the most unstable and unusable llm api in prod, can't handle our ci/cd

* add max_completion_tokens to openai supported params

* fix fireworks ai param mapping
2024-09-14 14:57:01 -07:00
Ahmet
415a3ede9e
Update model_prices_and_context_window.json (#5700)
added audio_speech mode on the sample_spec for clarity.
2024-09-14 11:22:08 -07:00
Krish Dholakia
dad1ad2077
LiteLLM Minor Fixes and Improvements (09/14/2024) (#5697)
* fix(health_check.py): hide sensitive keys from health check debug information k

* fix(route_llm_request.py): fix proxy model not found error message to indicate how to resolve issue

* fix(vertex_llm_base.py): fix exception message to not log credentials
2024-09-14 10:32:39 -07:00
Krish Dholakia
60709a0753
LiteLLM Minor Fixes and Improvements (09/13/2024) (#5689)
* refactor: cleanup unused variables + fix pyright errors

* feat(health_check.py): Closes https://github.com/BerriAI/litellm/issues/5686

* fix(o1_reasoning.py): add stricter check for o-1 reasoning model

* refactor(mistral/): make it easier to see mistral transformation logic

* fix(openai.py): fix openai o-1 model param mapping

Fixes https://github.com/BerriAI/litellm/issues/5685

* feat(main.py): infer finetuned gemini model from base model

Fixes https://github.com/BerriAI/litellm/issues/5678

* docs(vertex.md): update docs to call finetuned gemini models

* feat(proxy_server.py): allow admin to hide proxy model aliases

Closes https://github.com/BerriAI/litellm/issues/5692

* docs(load_balancing.md): add docs on hiding alias models from proxy config

* fix(base.py): don't raise notimplemented error

* fix(user_api_key_auth.py): fix model max budget check

* fix(router.py): fix elif

* fix(user_api_key_auth.py): don't set team_id to empty str

* fix(team_endpoints.py): fix response type

* test(test_completion.py): handle predibase error

* test(test_proxy_server.py): fix test

* fix(o1_transformation.py): fix max_completion_token mapping

* test(test_image_generation.py): mark flaky test
2024-09-14 10:02:55 -07:00
F1bos
db3af20d84
(models): Added missing gemini experimental models + fixed pricing for gemini-1.5-pro-exp-0827 (#5693)
* Fixed gemini-1.5-flash pricing

* (models): Added missing gemini experimental models + fixed pricing for gemini-1.5-pro-exp-0827
2024-09-14 08:41:48 -07:00
Ishaan Jaff
741c8e8a45
[Feat - Perf Improvement] DataDog Logger 91% lower latency (#5687)
* fix refactor dd to be an instance of custom logger

* migrate dd logger to be async

* clean up dd logging

* add datadog sync and async code

* use batching for datadog logger

* add doc string for dd logging

* add clear doc string

* fix doc string

* allow debugging intake url

* clean up requirements.txt

* allow setting custom batch size on logger

* fix dd logging to use compression

* fix linting

* add dd load test

* fix dd load test

* fix dd url

* add test_datadog_logging_http_request

* fix test_datadog_logging_http_request
2024-09-13 17:39:17 -07:00
Ishaan Jaff
cd8d7ca915
[Fix] Performance - use in memory cache when downloading images from a url (#5657)
* fix use in memory cache when getting images

* fix linting

* fix load testing

* fix load test size

* fix load test size

* trigger ci/cd again
2024-09-13 07:23:42 -07:00
Krrish Dholakia
cdd7cd4d69 build: bump from 1.44.28 -> 1.45.0 2024-09-12 23:10:29 -07:00
Krish Dholakia
4657a40ef1
LiteLLM Minor Fixes and Improvements (09/12/2024) (#5658)
* fix(factory.py): handle tool call content as list

Fixes https://github.com/BerriAI/litellm/issues/5652

* fix(factory.py): enforce stronger typing

* fix(router.py): return model alias in /v1/model/info and /v1/model_group/info

* fix(user_api_key_auth.py): move noisy warning message to debug

cleanup logs

* fix(types.py): cleanup pydantic v2 deprecated param

Fixes https://github.com/BerriAI/litellm/issues/5649

* docs(gemini.md): show how to pass inline data to gemini api

Fixes https://github.com/BerriAI/litellm/issues/5674
2024-09-12 23:04:06 -07:00
David Manouchehri
795047c37f
Add o1 models on OpenRouter. (#5676) 2024-09-12 22:16:10 -07:00
Krish Dholakia
00047de1c6
fix(user_dashboard.tsx): don't call /global/spend on startup (#5668)
at 1m+ rows, query timeouts cause ui errors
2024-09-12 22:15:52 -07:00
Krish Dholakia
d94d47424f
fix(proxy/utils.py): auto-update if required view missing from db. raise warning for optional views. (#5675)
Prevents missing optional views from blocking proxy startup.
2024-09-12 22:15:44 -07:00
Ishaan Jaff
fa01b5c7d9 bump: version 1.44.27 → 1.44.28 2024-09-12 19:17:34 -07:00
Ishaan Jaff
19a06d7842
[Fix-Router] Don't cooldown when only 1 deployment exists (#5673)
* fix get model list

* fix test custom callback router

* fix embedding fallback test

* fix router retry policy on AuthErrors

* fix router test

* add test for single deployments no cooldown test prod

* add test test_single_deployment_no_cooldowns_test_prod_mock_completion_calls
2024-09-12 19:14:58 -07:00
Ishaan Jaff
13ba22d6fd docs add o1 to docs 2024-09-12 19:06:13 -07:00
Ishaan Jaff
e7c9716841
[Feat-Perf] Use Batching + Squashing (#5645)
* use folder for slack alerting

* clean up slack alerting

* fix test alerting
2024-09-12 18:37:53 -07:00
Ishaan Jaff
fe5e0bcd15
Merge pull request #5666 from BerriAI/litellm_add_openai_o1
[Feat] Add OpenAI O1 Family Param mapping / config
2024-09-12 16:15:53 -07:00
Ishaan Jaff
a1fe2701f2 Merge branch 'main' into litellm_add_openai_o1 2024-09-12 16:15:43 -07:00
Ishaan Jaff
bb38e9cbf8 fix gcs logging 2024-09-12 15:24:04 -07:00
Ishaan Jaff
46ce4995b8 fix type errors 2024-09-12 14:49:43 -07:00
Ishaan Jaff
0f24f339f3 fix handle user message 2024-09-12 14:34:32 -07:00
Ishaan Jaff
ded40e4d41 bump openai to 1.45.0 2024-09-12 14:18:15 -07:00
Ishaan Jaff
14dc7b3b54 fix linting 2024-09-12 14:15:18 -07:00
Ishaan Jaff
a5a0773b19 fix handle o1 not supporting system message 2024-09-12 14:09:13 -07:00
Ishaan Jaff
3490862795 bump: version 1.44.26 → 1.44.27 2024-09-12 13:41:05 -07:00
Ishaan Jaff
d2510a04a2 fix pricing 2024-09-12 13:41:01 -07:00
Ishaan Jaff
f5e9e9fc9a add o1 reasoning tests 2024-09-12 13:40:15 -07:00
Krish Dholakia
c76d2c6ade
Refactor 'check_view_exists' logic (#5659)
* fix(proxy/utils.py): comment out auto-upsert logic in check_view_exists

Prevents proxy from failing on startup due to faulty logic

* fix(db/migration_scripts/create_views.py): fix 'DailyTagSpend' quotation on check

* fix(create_views.py): mongly global spend time period should be 30d not 20d

* fix(schema.prisma): index on startTime and endUser for efficient UI querying
2024-09-12 13:39:50 -07:00
David Manouchehri
5c1a70be21
Fix token and remove dups. (#5662) 2024-09-12 13:33:35 -07:00
Ishaan Jaff
fed9c89cc7 add OpenAI o1 config 2024-09-12 13:22:59 -07:00
David Manouchehri
b4f97763f0
(models): Add o1 pricing. (#5661) 2024-09-12 11:47:04 -07:00
Ishaan Jaff
fab176fc20
Merge pull request #5660 from lowjiansheng/js-openai-o1
Add gpt o1 and o1 mini models
2024-09-12 11:35:06 -07:00
lowjiansheng
3afe70c1f2 gpt o1 and o1 mini 2024-09-13 02:27:57 +08:00
Ishaan Jaff
ead1e0c708
Merge pull request #5655 from BerriAI/litellm_testing_clean_up
[Fix Ci/cd] Separate testing pipeline for litellm router
2024-09-12 11:05:26 -07:00
Ishaan Jaff
085e1751ad mark test as flaky 2024-09-12 09:29:37 -07:00
Ishaan Jaff
bea34c9231 fix config.yml 2024-09-12 09:28:45 -07:00
Ishaan Jaff
90d096b639 ci/cd run again 2024-09-12 08:42:34 -07:00
Ishaan Jaff
9ca7de58d6 fix testing 2024-09-12 08:42:00 -07:00
Ishaan Jaff
d038568be4 ci/cd run again 2024-09-12 08:31:17 -07:00
Ishaan Jaff
e5a776dc07 make separate assistants testing pipeline 2024-09-12 08:30:21 -07:00
Ishaan Jaff
f880e2b958 fix respx 2024-09-12 08:26:31 -07:00
Ishaan Jaff
fbe92df87e fix router tests 2024-09-12 08:24:37 -07:00
Ishaan Jaff
9c79c1c7b2 fix ci/cd tests 2024-09-12 08:23:56 -07:00
Ishaan Jaff
d944bd98b9 fix config.yml 2024-09-12 08:21:05 -07:00
Ishaan Jaff
d65ba87014 add litellm router testing 2024-09-12 08:19:34 -07:00
Krrish Dholakia
69df1f5660 bump: version 1.44.25 → 1.44.26 2024-09-12 08:08:48 -07:00
Krish Dholakia
98c34a7e27
LiteLLM Minor Fixes and Improvements (11/09/2024) (#5634)
* fix(caching.py): set ttl for async_increment cache

fixes issue where ttl for redis client was not being set on increment_cache

Fixes https://github.com/BerriAI/litellm/issues/5609

* fix(caching.py): fix increment cache w/ ttl for sync increment cache on redis

Fixes https://github.com/BerriAI/litellm/issues/5609

* fix(router.py): support adding retry policy + allowed fails policy via config.yaml

* fix(router.py): don't cooldown single deployments

No point, as there's no other deployment to loadbalance with.

* fix(user_api_key_auth.py): support setting allowed email domains on jwt tokens

Closes https://github.com/BerriAI/litellm/issues/5605

* docs(token_auth.md): add user upsert + allowed email domain to jwt auth docs

* fix(litellm_pre_call_utils.py): fix dynamic key logging when team id is set

Fixes issue where key logging would not be set if team metadata was not none

* fix(secret_managers/main.py): load environment variables correctly

Fixes issue where os.environ/ was not being loaded correctly

* test(test_router.py): fix test

* feat(spend_tracking_utils.py): support logging additional usage params - e.g. prompt caching values for deepseek

* test: fix tests

* test: fix test

* test: fix test

* test: fix test

* test: fix test
2024-09-11 22:36:06 -07:00
Ishaan Jaff
70100d716b bump: version 1.44.24 → 1.44.25 2024-09-11 21:31:05 -07:00
Ishaan Jaff
9d2b09099f
Merge pull request #5646 from BerriAI/litellm_add_load_testing_logging
[Feat] Add Load Testing for Langsmith, and OTEL logging
2024-09-11 21:30:37 -07:00
Ishaan Jaff
88706488f9 fix otel load test 2024-09-11 21:27:31 -07:00