Commit graph

18422 commits

Author SHA1 Message Date
Ishaan Jaff
f6f5529621
Merge branch 'main' into litellm_fix_async_http_handler 2024-11-21 19:02:54 -08:00
Ishaan Jaff
71ebf47cef
fix latency issues on google ai studio (#6852) 2024-11-21 19:02:08 -08:00
Krrish Dholakia
2903fd4164 docs: update json mode docs 2024-11-22 03:00:45 +05:30
Ishaan Jaff
d03455a72c fix import 2024-11-21 13:11:06 -08:00
Krrish Dholakia
b8edef389c bump: version 1.52.12 → 1.52.13 2024-11-22 02:29:16 +05:30
Ishaan Jaff
9067a5031b fix langfuse import 2024-11-21 12:48:17 -08:00
Ishaan Jaff
45130c2d4c fix tests using in_memory_llm_clients_cache 2024-11-21 12:41:09 -08:00
Krish Dholakia
7e5085dc7b
Litellm dev 11 21 2024 (#6837)
* Fix Vertex AI function calling invoke: use JSON format instead of protobuf text format. (#6702)

* test: test tool_call conversion when arguments is empty dict

Fixes https://github.com/BerriAI/litellm/issues/6833

* fix(openai_like/handler.py): return more descriptive error message

Fixes https://github.com/BerriAI/litellm/issues/6812

* test: skip overloaded model

* docs(anthropic.md): update anthropic docs to show how to route to any new model

* feat(groq/): fake stream when 'response_format' param is passed

Groq doesn't support streaming when response_format is set

* feat(groq/): add response_format support for groq

Closes https://github.com/BerriAI/litellm/issues/6845

* fix(o1_handler.py): remove fake streaming for o1

Closes https://github.com/BerriAI/litellm/issues/6801

* build(model_prices_and_context_window.json): add groq llama3.2b model pricing

Closes https://github.com/BerriAI/litellm/issues/6807

* fix(utils.py): fix handling ollama response format param

Fixes https://github.com/BerriAI/litellm/issues/6848#issuecomment-2491215485

* docs(sidebars.js): refactor chat endpoint placement

* fix: fix linting errors

* test: fix test

* test: fix test

* fix(openai_like/handler): handle max retries

* fix(streaming_handler.py): fix streaming check for openai-compatible providers

* test: update test

* test: correctly handle model is overloaded error

* test: update test

* test: fix test

* test: mark flaky test

---------

Co-authored-by: Guowang Li <Guowang@users.noreply.github.com>
2024-11-22 01:53:52 +05:30
Ishaan Jaff
a7d5536872
(fix) passthrough - allow internal users to access /anthropic (#6843)
* fix /anthropic/

* test llm_passthrough_router

* fix test_gemini_pass_through_endpoint
2024-11-21 11:46:50 -08:00
Ishaan Jaff
e63ea48894 fix get_async_httpx_client 2024-11-21 11:18:07 -08:00
Ishaan Jaff
81c0125737 fix check_for_async_http_handler 2024-11-21 10:45:57 -08:00
Ishaan Jaff
ce0061d136 add check for AsyncClient 2024-11-21 10:39:34 -08:00
Krrish Dholakia
e8f47e96c3 test: cleanup mistral model 2024-11-21 10:32:08 -08:00
Ishaan Jaff
bb75af618f fix check_for_async_http_handler 2024-11-21 10:30:16 -08:00
Ishaan Jaff
d4dc8e60b6 fix make_async_azure_httpx_request 2024-11-21 10:27:08 -08:00
Ishaan Jaff
89d76d1eb7 fix get_async_httpx_client 2024-11-21 10:26:18 -08:00
Ishaan Jaff
398e6d0ac6 fix get_async_httpx_client 2024-11-21 10:24:18 -08:00
Ishaan Jaff
0a10b1ef1c fix get_async_httpx_client vertex 2024-11-21 10:22:30 -08:00
Ishaan Jaff
f7f9e8c41f fix dbricks get_async_httpx_client 2024-11-21 10:21:06 -08:00
Ishaan Jaff
0ee9f0fa44 fix vertex fine tuning 2024-11-21 10:20:16 -08:00
Ishaan Jaff
6af0494483 fix anthropic use get_async_httpx_client 2024-11-21 10:18:26 -08:00
Ishaan Jaff
fb5cc97387 fix PREDIBASE 2024-11-21 10:17:18 -08:00
Ishaan Jaff
4d56249eb9 add test_no_async_http_handler_usage 2024-11-21 10:16:07 -08:00
Krrish Dholakia
50d2510b60 test: cleanup mistral model 2024-11-21 23:44:50 +05:30
Ishaan Jaff
77232f9bc4 fix HUGGINGFACE 2024-11-21 09:46:04 -08:00
Ishaan Jaff
2719f7fcbf fix CLARIFAI 2024-11-21 09:43:04 -08:00
Ishaan Jaff
3d3d651b89 fix REPLICATE 2024-11-21 09:42:01 -08:00
Ishaan Jaff
fdaee84b82 fix TEXT_COMPLETION_CODESTRAL 2024-11-21 09:40:26 -08:00
Ishaan Jaff
0420b07c13 fix triton 2024-11-21 09:39:48 -08:00
Ishaan Jaff
ddfe687b13
(fix) don't block proxy startup if license check fails & using prometheus (#6839)
* fix - don't block proxy startup if not a premium user

* test_litellm_proxy_server_config_with_prometheus

* add test for proxy startup

* fix remove unused test

* fix startup test

* add comment on bad-license
2024-11-20 17:55:39 -08:00
Ishaan Jaff
cc1f8ff0ba
(testing) - add e2e tests for anthropic pass through endpoints (#6840)
* tests - add e2e tests for anthropic pass through

* fix swagger

* fix pass through tests
2024-11-20 17:55:13 -08:00
Ishaan Jaff
c107bae7ae
(feat) add usage / cost tracking for Anthropic passthrough routes (#6835)
* move _process_response in transformation

* fix AnthropicConfig test

* add AnthropicConfig

* fix anthropic_passthrough_handler

* fix get_response_body

* fix check for streaming response

* use 1 helper to return stream_response on passthrough
2024-11-20 17:25:12 -08:00
Ishaan Jaff
434b1d3d86
(refactor) anthropic - move _process_response in transformation.py (#6834)
* move _process_response in transformation

* fix AnthropicConfig test
2024-11-20 17:24:19 -08:00
Krish Dholakia
b11bc0374e
Litellm dev 11 20 2024 (#6838)
* feat(customer_endpoints.py): support passing budget duration via `/customer/new` endpoint

Closes https://github.com/BerriAI/litellm/issues/5651

* docs: add missing params to swagger + api documentation test

* docs: add documentation for all key endpoints

documents all params on swagger

* docs(internal_user_endpoints.py): document all /user/new params

Ensures all params are documented

* docs(team_endpoints.py): add missing documentation for team endpoints

Ensures 100% param documentation on swagger

* docs(organization_endpoints.py): document all org params

Adds documentation for all params in org endpoint

* docs(customer_endpoints.py): add coverage for all params on /customer endpoints

ensures all /customer/* params are documented

* ci(config.yml): add endpoint doc testing to ci/cd

* fix: fix internal_user_endpoints.py

* fix(internal_user_endpoints.py): support 'duration' param

* fix(partner_models/main.py): fix anthropic re-raise exception on vertex

* fix: fix pydantic obj

* build(model_prices_and_context_window.json): add new vertex claude model names

vertex claude changed model names - causes cost tracking errors
2024-11-21 05:20:37 +05:30
Krrish Dholakia
0b0253f7ad build: update ui build 2024-11-21 05:16:58 +05:30
Krrish Dholakia
746881485f bump: version 1.52.11 → 1.52.12 2024-11-21 04:38:04 +05:30
Krish Dholakia
689cd677c6
Litellm dev 11 20 2024 (#6831)
* feat(customer_endpoints.py): support passing budget duration via `/customer/new` endpoint

Closes https://github.com/BerriAI/litellm/issues/5651

* docs: add missing params to swagger + api documentation test

* docs: add documentation for all key endpoints

documents all params on swagger

* docs(internal_user_endpoints.py): document all /user/new params

Ensures all params are documented

* docs(team_endpoints.py): add missing documentation for team endpoints

Ensures 100% param documentation on swagger

* docs(organization_endpoints.py): document all org params

Adds documentation for all params in org endpoint

* docs(customer_endpoints.py): add coverage for all params on /customer endpoints

ensures all /customer/* params are documented

* ci(config.yml): add endpoint doc testing to ci/cd

* fix: fix internal_user_endpoints.py

* fix(internal_user_endpoints.py): support 'duration' param

* fix(partner_models/main.py): fix anthropic re-raise exception on vertex

* fix: fix pydantic obj
2024-11-21 04:06:06 +05:30
David Manouchehri
a1f06de53d
Add gpt-4o-2024-11-20. (#6832) 2024-11-21 03:48:29 +05:30
Krish Dholakia
b0be5bf3a1
LiteLLM Minor Fixes & Improvements (11/19/2024) (#6820)
* fix(anthropic/chat/transformation.py): add json schema as values: json_schema

fixes passing pydantic obj to anthropic

Fixes https://github.com/BerriAI/litellm/issues/6766

* (feat): Add timestamp_granularities parameter to transcription API (#6457)

* Add timestamp_granularities parameter to transcription API

* add param to the local test

* fix(databricks/chat.py): handle max_retries optional param handling for openai-like calls

Fixes issue with calling finetuned vertex ai models via databricks route

* build(ui/): add team admins via proxy ui

* fix: fix linting error

* test: fix test

* docs(vertex.md): refactor docs

* test: handle overloaded anthropic model error

* test: remove duplicate test

* test: fix test

* test: update test to handle model overloaded error

---------

Co-authored-by: Show <35062952+BrunooShow@users.noreply.github.com>
2024-11-21 00:57:58 +05:30
Krrish Dholakia
7d0e1f05ac build: run new build 2024-11-20 19:48:57 +05:30
Krrish Dholakia
6a816bceee test: fix test 2024-11-20 14:13:14 +05:30
Ishaan Jaff
132569dafc ci/cd run again 2024-11-19 22:38:45 -08:00
Ishaan Jaff
8631f3bb60 use correct name for test file 2024-11-19 22:11:52 -08:00
Ishaan Jaff
8b92e4f77a fix test_prometheus_metric_tracking 2024-11-19 22:11:30 -08:00
Ishaan Jaff
7463dab9c6
(feat) provider budget routing improvements (#6827)
* minor fix for provider budget

* fix raise good error message when budget crossed for provider budget

* fix test provider budgets

* test provider budgets

* feat - emit llm provider spend on prometheus

* test_prometheus_metric_tracking

* doc provider budgets
2024-11-19 21:25:08 -08:00
Ishaan Jaff
3c6fe21935
(Feat) Add provider specific budget routing (#6817)
* add ProviderBudgetConfig

* working test_provider_budgets_e2e_test

* test_provider_budgets_e2e_test_expect_to_fail

* use 1 cache read for getting provider spend

* test_provider_budgets_e2e_test

* add doc on provider budgets

* clean up provider budgets

* unit testing for provider budget routing

* use as flag, not routing strat

* fix init provider budget routing

* use async_filter_deployments

* fix test provider budgets

* doc provider budget routing

* doc provider budget routing

* fix docs changes

* fix comment
2024-11-19 20:25:27 -08:00
Krrish Dholakia
59a9b71d21 build: fix test 2024-11-20 05:50:08 +05:30
Krish Dholakia
cf579fe644
Litellm stable pr 10 30 2024 (#6821)
* Update organization_endpoints.py to be able to list organizations (#6473)

* Update organization_endpoints.py to be able to list organizations

* Update test_organizations.py

* Update test_organizations.py

add test for list

* Update test_organizations.py

correct indentation

* Add unreleased Claude 3.5 Haiku models. (#6476)

---------

Co-authored-by: superpoussin22 <vincent.nadal@orange.fr>
Co-authored-by: David Manouchehri <david.manouchehri@ai.moda>
2024-11-20 05:03:42 +05:30
Ishaan Jaff
98c7889013
feat - add qwen2p5-coder-32b-instruct (#6818) 2024-11-19 14:50:51 -08:00
Ishaan Jaff
1890fde3f3
(Proxy) add support for DOCS_URL and REDOC_URL (#6806)
* add support for DOCS_URL and REDOC_URL

* document env vars

* add unit tests for docs url and redocs url
2024-11-19 07:02:12 -08:00