Ravi N
97cf32630d
remove aws_sagemaker_allow_zero_temp from the parameters passed to inference
2024-08-12 21:09:50 -04:00
Ravi N
b321f2988b
Allow zero temperature for Sagemaker models based on config
...
Since Sagemaker can host any kind of model, some models allow
zero temperature. However, this is not enabled by default and
only allowed based on config
2024-07-28 21:55:53 -04:00
Krrish Dholakia
2f773d9cb6
fix(litellm_cost_calc/google.py): support meta llama vertex ai cost tracking
2024-07-25 22:12:07 -07:00
Ishaan Jaff
079a41fbe1
Merge branch 'main' into litellm_proxy_support_all_providers
2024-07-25 20:15:37 -07:00
Krrish Dholakia
a2fd8459fc
fix(utils.py): don't raise error on openai content filter during streaming - return as is
...
Fixes issue where we would raise an error vs. openai who return the chunk with finish reason as 'content_filter'
2024-07-25 19:50:52 -07:00
Krish Dholakia
b6ca4406b6
Merge branch 'main' into bedrock-llama3.1-405b
2024-07-25 19:29:10 -07:00
Ishaan Jaff
9863520376
support using */*
2024-07-25 18:48:56 -07:00
Krrish Dholakia
060249c7e0
feat(utils.py): support async streaming for custom llm provider
2024-07-25 17:11:57 -07:00
Krrish Dholakia
b4e3a77ad0
feat(utils.py): support sync streaming for custom llm provider
2024-07-25 16:47:32 -07:00
Krrish Dholakia
6bf1b9353b
feat(custom_llm.py): initial working commit for writing your own custom LLM handler
...
Fixes https://github.com/BerriAI/litellm/issues/4675
Also Addresses https://github.com/BerriAI/litellm/discussions/4677
2024-07-25 15:33:05 -07:00
David Manouchehri
64adae6e7f
Check for converse support first.
2024-07-25 21:16:23 +00:00
Krrish Dholakia
4e51f712f3
fix(main.py): fix calling openai gpt-3.5-turbo-instruct via /completions
...
Fixes https://github.com/BerriAI/litellm/issues/749
2024-07-25 09:57:19 -07:00
wslee
40bb165108
support dynamic api base
2024-07-25 11:14:38 +09:00
wslee
dd10da4d46
add support for friendli dedicated endpoint
2024-07-25 11:14:35 +09:00
Ishaan Jaff
8ea4b73c27
add UnsupportedParamsError to litellm exceptions
2024-07-24 12:20:14 -07:00
Krrish Dholakia
7df94100e8
build(model_prices_and_context_window.json): add model pricing for vertex ai llama 3.1 api
2024-07-23 17:36:07 -07:00
Krrish Dholakia
83ef52e180
feat(vertex_ai_llama.py): vertex ai llama3.1 api support
...
Initial working commit for vertex ai llama 3.1 api support
2024-07-23 17:07:30 -07:00
Krrish Dholakia
f64a3309d1
fix(utils.py): support raw response headers for streaming requests
2024-07-23 11:58:58 -07:00
Krrish Dholakia
dcb974dd1e
feat(utils.py): support passing openai response headers to client, if enabled
...
Allows openai/openai-compatible provider response headers to be sent to client, if 'return_response_headers' is enabled
2024-07-23 11:30:52 -07:00
Ishaan Jaff
1355932bf4
Merge pull request #3905 from giritatavarty-8451/litellm_triton_chatcompletion_support
...
Litellm triton chatcompletion support - Resubmit of #3895
2024-07-23 10:30:26 -07:00
Ishaan Jaff
8f9638f2c1
fix raise correct provider on content policy violation
2024-07-22 16:03:15 -07:00
Ishaan Jaff
15c109f023
fix checking if _known_custom_logger_compatible_callbacks
2024-07-22 15:43:43 -07:00
Krrish Dholakia
f10af7596c
fix(utils.py): allow dropping extra_body in additional_drop_params
...
Fixes https://github.com/BerriAI/litellm/issues/4769
2024-07-20 19:12:58 -07:00
Ishaan Jaff
5e4d291244
rename to _response_headers
2024-07-20 17:31:16 -07:00
Ishaan Jaff
46cf4f69ae
return response headers in response
2024-07-20 14:59:08 -07:00
Krish Dholakia
3053f52c43
Merge pull request #4801 from BerriAI/litellm_dynamic_params_oai_compatible_endpoints
...
fix(utils.py): support dynamic params for openai-compatible providers
2024-07-19 21:07:06 -07:00
Krrish Dholakia
95a0f6839f
fix(utils.py): fix token_counter to handle empty tool calls in messages
...
Fixes https://github.com/BerriAI/litellm/pull/4749
2024-07-19 19:39:00 -07:00
Krrish Dholakia
e45956d77e
fix(utils.py): fix get_llm_provider to support dynamic params for openai-compatible providers
2024-07-19 19:36:31 -07:00
Krrish Dholakia
e2d275f1b7
fix(utils.py): add exception mapping for bedrock image internal server error
2024-07-19 19:30:41 -07:00
Sophia Loris
d779253949
resolve merge conflicts
2024-07-19 09:45:53 -05:00
Sophia Loris
d5c65c6be2
Add support for Triton streaming & triton async completions
2024-07-19 09:35:27 -05:00
Krrish Dholakia
b23a633cf1
fix(utils.py): fix status code in exception mapping
2024-07-18 18:04:59 -07:00
Krish Dholakia
57f6923ab6
Merge pull request #4729 from vingiarrusso/vgiarrusso/guardrails
...
Add enabled_roles to Guardrails configuration, Update Lakera guardrail moderation hook
2024-07-17 22:24:35 -07:00
Ishaan Jaff
ee53b9093b
Merge pull request #4758 from BerriAI/litellm_langsmith_async_support
...
[Feat] Use Async Httpx client for langsmith logging
2024-07-17 16:54:40 -07:00
Ishaan Jaff
5f04f7b7c1
fix langsmith logging for streaming
2024-07-17 16:04:45 -07:00
Ishaan Jaff
9c00fb64c4
use langsmith as a custom callback class
2024-07-17 15:35:13 -07:00
Krrish Dholakia
a176feeacc
fix(utils.py): return optional params from groq
2024-07-17 12:09:08 -07:00
Krrish Dholakia
4cf293395b
fix(utils.py): fix linting error
2024-07-16 20:57:34 -07:00
Krrish Dholakia
155ba055ee
fix(utils.py): fix get_api_base to use vertexai_anthropic
2024-07-16 19:17:45 -07:00
Ishaan Jaff
3981be6a99
fix install on python 3.8
2024-07-16 17:00:32 -07:00
Ishaan Jaff
95af5c260e
fix installing on python3.8
2024-07-16 16:56:15 -07:00
Vinnie Giarrusso
b83f47e941
refactor a bit
2024-07-16 12:19:31 -07:00
Krish Dholakia
0fb88e527c
Merge pull request #4716 from pamelafox/countfuncs
...
Add token counting for OpenAI tools/tool_choice
2024-07-16 07:21:31 -07:00
Vinnie Giarrusso
6ff863ee00
Add enabled_roles to Guardrails configuration, Update Lakera guardrail moderation hook
2024-07-16 01:52:08 -07:00
Krrish Dholakia
a15ba2592a
fix(utils.py): allow passing dynamic api base for openai-compatible endpoints
2024-07-15 20:00:44 -07:00
Krrish Dholakia
959c627dd3
fix(litellm_logging.py): log response_cost=0 for failed calls
...
Fixes https://github.com/BerriAI/litellm/issues/4604
2024-07-15 19:25:56 -07:00
Krrish Dholakia
9cc2daeec9
fix(utils.py): update get_model_info docstring
...
Fixes https://github.com/BerriAI/litellm/issues/4711
2024-07-15 18:18:50 -07:00
Pamela Fox
d0fe1a8906
Docstring
2024-07-15 11:12:42 -07:00
Pamela Fox
8d01f91056
Less changes
2024-07-15 11:11:21 -07:00
Pamela Fox
a2188a869e
Less changes
2024-07-15 11:09:45 -07:00
