Krish Dholakia
c03e5da41f
LiteLLM Minor Fixes & Improvements (10/24/2024) ( #6421 )
...
* fix(utils.py): support passing dynamic api base to validate_environment
Returns True if just api base is required and api base is passed
* fix(litellm_pre_call_utils.py): feature flag sending client headers to llm api
Fixes https://github.com/BerriAI/litellm/issues/6410
* fix(anthropic/chat/transformation.py): return correct error message
* fix(http_handler.py): add error response text in places where we expect it
* fix(factory.py): handle base case of no non-system messages to bedrock
Fixes https://github.com/BerriAI/litellm/issues/6411
* feat(cohere/embed): Support cohere image embeddings
Closes https://github.com/BerriAI/litellm/issues/6413
* fix(__init__.py): fix linting error
* docs(supported_embedding.md): add image embedding example to docs
* feat(cohere/embed): use cohere embedding returned usage for cost calc
* build(model_prices_and_context_window.json): add embed-english-v3.0 details (image cost + 'supports_image_input' flag)
* fix(cohere_transformation.py): fix linting error
* test(test_proxy_server.py): cleanup test
* test: cleanup test
* fix: fix linting errors
2024-10-25 15:55:56 -07:00
Krish Dholakia
cc8dd80209
allow configuring httpx hooks for AsyncHTTPHandler ( #6290 ) ( #6415 )
...
* allow configuring httpx hooks for AsyncHTTPHandler (#6290 )
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* Fixes and minor improvements for Helm Chart (#6402 )
* reckoner hack
* fix default
* add extracontainers option
* revert chart
* fix extracontainers
* fix deployment
* remove init container
* update docs
* add helm lint to deploy step
* change name
* (refactor) prometheus async_log_success_event to be under 100 LOC (#6416 )
* unit testig for prometheus
* unit testing for success metrics
* use 1 helper for _increment_token_metrics
* use helper for _increment_remaining_budget_metrics
* use _increment_remaining_budget_metrics
* use _increment_top_level_request_and_spend_metrics
* use helper for _set_latency_metrics
* remove noqa violation
* fix test prometheus
* test prometheus
* unit testing for all prometheus helper functions
* fix prom unit tests
* fix unit tests prometheus
* fix unit test prom
* (refactor) router - use static methods for client init utils (#6420 )
* use InitalizeOpenAISDKClient
* use InitalizeOpenAISDKClient static method
* fix # noqa: PLR0915
* (code cleanup) remove unused and undocumented logging integrations - litedebugger, berrispend (#6406 )
* code cleanup remove unused and undocumented code files
* fix unused logging integrations cleanup
* update chart version
* add circleci tests
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
* fix: fix linting error
* fix(http_handler.py): fix linting error
---------
Co-authored-by: Alejandro Rodríguez <alejorro70@gmail.com>
Co-authored-by: Robert Brennan <accounts@rbren.io>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
2024-10-24 22:00:24 -07:00
Krish Dholakia
f252350881
LiteLLM Minor Fixes & Improvements (10/17/2024) ( #6293 )
...
* fix(ui_sso.py): fix faulty admin only check
Fixes https://github.com/BerriAI/litellm/issues/6286
* refactor(sso_helper_utils.py): refactor /sso/callback to use helper utils, covered by unit testing
Prevent future regressions
* feat(prompt_factory): support 'ensure_alternating_roles' param
Closes https://github.com/BerriAI/litellm/issues/6257
* fix(proxy/utils.py): add dailytagspend to expected views
* feat(auth_utils.py): support setting regex for clientside auth credentials
Fixes https://github.com/BerriAI/litellm/issues/6203
* build(cookbook): add tutorial for mlflow + langchain + litellm proxy tracing
* feat(argilla.py): add argilla logging integration
Closes https://github.com/BerriAI/litellm/issues/6201
* fix: fix linting errors
* fix: fix ruff error
* test: fix test
* fix: update vertex ai assumption - parts not always guaranteed (#6296 )
* docs(configs.md): add argila env var to docs
2024-10-17 22:09:11 -07:00
Krish Dholakia
d57be47b0f
Litellm ruff linting enforcement ( #5992 )
...
* ci(config.yml): add a 'check_code_quality' step
Addresses https://github.com/BerriAI/litellm/issues/5991
* ci(config.yml): check why circle ci doesn't pick up this test
* ci(config.yml): fix to run 'check_code_quality' tests
* fix(__init__.py): fix unprotected import
* fix(__init__.py): don't remove unused imports
* build(ruff.toml): update ruff.toml to ignore unused imports
* fix: fix: ruff + pyright - fix linting + type-checking errors
* fix: fix linting errors
* fix(lago.py): fix module init error
* fix: fix linting errors
* ci(config.yml): cd into correct dir for checks
* fix(proxy_server.py): fix linting error
* fix(utils.py): fix bare except
causes ruff linting errors
* fix: ruff - fix remaining linting errors
* fix(clickhouse.py): use standard logging object
* fix(__init__.py): fix unprotected import
* fix: ruff - fix linting errors
* fix: fix linting errors
* ci(config.yml): cleanup code qa step (formatting handled in local_testing)
* fix(_health_endpoints.py): fix ruff linting errors
* ci(config.yml): just use ruff in check_code_quality pipeline for now
* build(custom_guardrail.py): include missing file
* style(embedding_handler.py): fix ruff check
2024-10-01 19:44:20 -04:00
Krish Dholakia
d46660ea0f
LiteLLM Minor Fixes & Improvements (09/18/2024) ( #5772 )
...
* fix(proxy_server.py): fix azure key vault logic to not require client id/secret
* feat(cost_calculator.py): support fireworks ai cost tracking
* build(docker-compose.yml): add lines for mounting config.yaml to docker compose
Closes https://github.com/BerriAI/litellm/issues/5739
* fix(input.md): update docs to clarify litellm supports content as a list of dictionaries
Fixes https://github.com/BerriAI/litellm/issues/5755
* fix(input.md): update input.md to include all message values
* fix(image_handling.py): follow image url redirects
Fixes https://github.com/BerriAI/litellm/issues/5763
* fix(router.py): Fix model key/base leak in error message
Fixes https://github.com/BerriAI/litellm/issues/5762
* fix(http_handler.py): fix linting error
* fix(azure.py): fix logging to show azure_ad_token being used
Fixes https://github.com/BerriAI/litellm/issues/5767
* fix(_redis.py): add redis sentinel support
Closes https://github.com/BerriAI/litellm/issues/4381
* feat(_redis.py): add redis sentinel support
Closes https://github.com/BerriAI/litellm/issues/4381
* test(test_completion_cost.py): fix test
* Databricks Integration: Integrate Databricks SDK as optional mechanism for fetching API base and token, if unspecified (#5746 )
* LiteLLM Minor Fixes & Improvements (09/16/2024) (#5723 )
* coverage (#5713 )
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* Move (#5714 )
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix(litellm_logging.py): fix logging client re-init (#5710 )
Fixes https://github.com/BerriAI/litellm/issues/5695
* fix(presidio.py): Fix logging_hook response and add support for additional presidio variables in guardrails config
Fixes https://github.com/BerriAI/litellm/issues/5682
* feat(o1_handler.py): fake streaming for openai o1 models
Fixes https://github.com/BerriAI/litellm/issues/5694
* docs: deprecated traceloop integration in favor of native otel (#5249 )
* fix: fix linting errors
* fix: fix linting errors
* fix(main.py): fix o1 import
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view (#5730 )
* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view
Supports having `MonthlyGlobalSpend` view be a material view, and exposes an endpoint to refresh it
* fix(custom_logger.py): reset calltype
* fix: fix linting errors
* fix: fix linting error
* fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix: fix import
* Fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* DB test
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* Coverage
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* progress
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix test name
Signed-off-by: dbczumar <corey.zumar@databricks.com>
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
* test: fix test
* test(test_databricks.py): fix test
* fix(databricks/chat.py): handle custom endpoint (e.g. sagemaker)
* Apply code scanning fix for clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* fix(__init__.py): fix known fireworks ai models
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2024-09-19 13:25:29 -07:00
Ishaan Jaff
72dd21dc47
fix linting error
2024-09-10 13:29:02 -07:00
Ishaan Jaff
421b857714
pass llm provider when creating async httpx clients
2024-09-10 11:51:42 -07:00
Ishaan Jaff
d4b9a1307d
rename get_async_httpx_client
2024-09-10 10:38:01 -07:00
Krish Dholakia
72e961af3c
LiteLLM Minor Fixes and Improvements (08/06/2024) ( #5567 )
...
* fix(utils.py): return citations for perplexity streaming
Fixes https://github.com/BerriAI/litellm/issues/5535
* fix(anthropic/chat.py): support fallbacks for anthropic streaming (#5542 )
* fix(anthropic/chat.py): support fallbacks for anthropic streaming
Fixes https://github.com/BerriAI/litellm/issues/5512
* fix(anthropic/chat.py): use module level http client if none given (prevents early client closure)
* fix: fix linting errors
* fix(http_handler.py): fix raise_for_status error handling
* test: retry flaky test
* fix otel type
* fix(bedrock/embed): fix error raising
* test(test_openai_batches_and_files.py): skip azure batches test (for now) quota exceeded
* fix(test_router.py): skip azure batch route test (for now) - hit batch quota limits
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* All `model_group_alias` should show up in `/models`, `/model/info` , `/model_group/info` (#5539 )
* fix(router.py): support returning model_alias model names in `/v1/models`
* fix(proxy_server.py): support returning model alias'es on `/model/info`
* feat(router.py): support returning model group alias for `/model_group/info`
* fix(proxy_server.py): fix linting errors
* fix(proxy_server.py): fix linting errors
* build(model_prices_and_context_window.json): add amazon titan text premier pricing information
Closes https://github.com/BerriAI/litellm/issues/5560
* feat(litellm_logging.py): log standard logging response object for pass through endpoints. Allows bedrock /invoke agent calls to be correctly logged to langfuse + s3
* fix(success_handler.py): fix linting error
* fix(success_handler.py): fix linting errors
* fix(team_endpoints.py): Allows admin to update team member budgets
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2024-09-06 17:16:24 -07:00
Krish Dholakia
f458f565af
Merge pull request #5292 from OgnjenFrancuski/main
...
Update SSL verification
2024-08-23 20:42:35 -07:00
Ognjen Francuski
765e020e75
Update handling of 'litellm.ssl_verify' in HTTP handlers to allow for custom, self-signed certificates.
2024-08-20 10:16:03 +02:00
Haadi Rakhangi
c6716673c1
added put method in http_handler
2024-08-19 21:10:18 +05:30
Haadi Rakhangi
7f1c3f5edf
implemented RestAPI and added support for cloud and local Qdrant clusters
2024-08-19 20:46:30 +05:30
Ishaan Jaff
5962267bf5
delete object from gcs
2024-08-01 14:25:25 -07:00
Krrish Dholakia
6202f9bbb0
fix(http_handler.py): correctly re-raise timeout exception
2024-07-31 14:51:28 -07:00
Krrish Dholakia
c583cbdbcd
fix: fix linting errors
2024-07-31 07:19:45 -07:00
Ishaan Jaff
82e5ea059b
fix predibase timeout exceptions
2024-07-30 22:52:08 -07:00
Ishaan Jaff
da494da12c
support timeouts on http handler
2024-07-30 22:49:00 -07:00
David Manouchehri
54e0b29079
feat(httpx): Send litellm version upstream.
2024-07-08 05:40:32 +00:00
Krrish Dholakia
7210fe6926
fix(http_handler.py): cleanup custom mounting logic
2024-07-04 15:32:50 -07:00
Krrish Dholakia
8571cb45e8
fix(http_handler.py): add retry logic for httpx.ConnectError
2024-06-28 22:35:26 -07:00
Krrish Dholakia
c151a1d244
fix(http_handler.py): raise more detailed http status errors
2024-06-28 15:12:38 -07:00
Krrish Dholakia
a7122f91a1
fix(support-'alt=sse'-param): Fixes https://github.com/BerriAI/litellm/issues/4459
2024-06-28 10:40:47 -07:00
Ishaan Jaff
38995def54
refactor to use _get_async_httpx_client
2024-06-14 21:30:42 -07:00
Krrish Dholakia
46d57526c4
fix(http_handler.py): add retry logic on httpx.remoteprotocolerror
2024-06-13 14:05:29 -07:00
Krrish Dholakia
d45db9a5a2
fix(http_handler.py): retry on httpx connection errors
...
Fixes https://github.com/BerriAI/litellm/issues/4097
2024-06-13 12:54:58 -07:00
Krrish Dholakia
35e4323095
refactor(main.py): only route anthropic calls through converse api
...
v0 scope let's move function calling to converse api
2024-06-07 08:47:51 -07:00
Krrish Dholakia
51ba5652a0
feat(bedrock_httpx.py): working bedrock converse api streaming
2024-06-06 22:13:21 -07:00
Krrish Dholakia
a995a0b172
fix(bedrock_httpx.py): working claude 3 function calling
2024-06-06 20:12:41 -07:00
Krrish Dholakia
5d3674b63d
fix(main.py): fix typing for image gen response
2024-06-04 08:29:30 -07:00
Krrish Dholakia
a16a1c407a
fix(http_handler.py): allow setting ca bundle path
2024-06-01 14:48:53 -07:00
Krrish Dholakia
f1ce7bb300
fix(http_handler.py): add bool check for os.getenv verif ssl value
2024-06-01 14:07:41 -07:00
Krrish Dholakia
22a0c400dd
fix(http_handler.py): support verify_ssl=False when using httpx client
2024-06-01 13:56:45 -07:00
Krrish Dholakia
93c3635b64
fix: fix streaming with httpx client
...
prevent overwriting streams in parallel streaming calls
2024-05-31 10:55:18 -07:00
Krrish Dholakia
324bf027f5
fix(anthropic.py): fix parallel streaming on anthropic.py
...
prevent parallel requests from cancelling each other
Fixes https://github.com/BerriAI/litellm/issues/3881
2024-05-28 16:29:09 -07:00
Krrish Dholakia
d2229dcd21
feat(databricks.py): adds databricks support - completion, async, streaming
...
Closes https://github.com/BerriAI/litellm/issues/2160
2024-05-23 16:29:46 -07:00
Krrish Dholakia
64650c0279
feat(bedrock_httpx.py): working bedrock command-r sync+async streaming
2024-05-11 19:39:51 -07:00
Krrish Dholakia
59c8c0adff
feat(bedrock_httpx.py): working cohere command r async calls
2024-05-11 15:04:38 -07:00
Krrish Dholakia
62a8bf134f
fix(http_handler.py): fix linting error
2024-04-19 15:45:24 -07:00
Josh Mandel
f411443e58
fix: Stream completion responses from anthropic. (Fix 3129)
2024-04-19 16:13:19 -05:00
Krrish Dholakia
26286a54b8
fix(anthropic_text.py): add support for async text completion calls
2024-04-15 08:15:00 -07:00
Ishaan Jaff
9be250c0f0
add exit and aenter
2024-04-06 19:27:26 -07:00
Ishaan Jaff
5c796b4365
async streaming anthropic
2024-04-06 17:53:06 -07:00
Ishaan Jaff
58c4b02447
feat - make anthropic async
2024-04-06 15:50:13 -07:00
Krrish Dholakia
1d341970ba
feat(vertex_ai_anthropic.py): add claude 3 on vertex ai support - working .completions call
...
.completions() call works
2024-04-02 22:07:39 -07:00
Krrish Dholakia
05029fdcc7
feat(vertex_ai_anthropic.py): Add support for claude 3 on vertex ai
2024-03-23 15:53:04 -07:00
