phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

LiteLLM Minor Fixes & Improvements (09/24/2024) (#5880)

Browse source 
* LiteLLM Minor Fixes & Improvements (09/23/2024)  (#5842)

* feat(auth_utils.py): enable admin to allow client-side credentials to be passed

Makes it easier for devs to experiment with finetuned fireworks ai models

* feat(router.py): allow setting configurable_clientside_auth_params for a model

Closes https://github.com/BerriAI/litellm/issues/5843

* build(model_prices_and_context_window.json): fix anthropic claude-3-5-sonnet max output token limit

Fixes https://github.com/BerriAI/litellm/issues/5850

* fix(azure_ai/): support content list for azure ai

Fixes https://github.com/BerriAI/litellm/issues/4237

* fix(litellm_logging.py): always set saved_cache_cost

Set to 0 by default

* fix(fireworks_ai/cost_calculator.py): add fireworks ai default pricing

handles calling 405b+ size models

* fix(slack_alerting.py): fix error alerting for failed spend tracking

Fixes regression with slack alerting error monitoring

* fix(vertex_and_google_ai_studio_gemini.py): handle gemini no candidates in streaming chunk error

* docs(bedrock.md): add llama3-1 models

* test: fix tests

* fix(azure_ai/chat): fix transformation for azure ai calls

* feat(azure_ai/embed): Add azure ai embeddings support

Closes https://github.com/BerriAI/litellm/issues/5861

* fix(azure_ai/embed): enable async embedding

* feat(azure_ai/embed): support azure ai multimodal embeddings

* fix(azure_ai/embed): support async multi modal embeddings

* feat(together_ai/embed): support together ai embedding calls

* feat(rerank/main.py): log source documents for rerank endpoints to langfuse

improves rerank endpoint logging

* fix(langfuse.py): support logging `/audio/speech` input to langfuse

* test(test_embedding.py): fix test

* test(test_completion_cost.py): fix helper util
This commit is contained in:
Krish Dholakia 2024-09-25 22:11:57 -07:00 • committed by GitHub
parent 5bc5eaff8a
commit 16c0307eab
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
25 changed files with 1675 additions and 340 deletions
Download patch file Download diff file Expand all files Collapse all files

14
litellm/llms/OpenAI/openai.py
View file

@ -1215,7 +1215,6 @@ class OpenAIChatCompletion(BaseLLM):
client: Optional[AsyncOpenAI] = None,
max_retries=None,
):
response = None
try:
openai_aclient: AsyncOpenAI = self._get_openai_client( # type: ignore
is_async=True,
@ -1237,12 +1236,15 @@ class OpenAIChatCompletion(BaseLLM):
additional_args={"complete_input_dict": data},
original_response=stringified_response,
)
return convert_to_model_response_object(
returned_response: (
litellm.EmbeddingResponse
) = convert_to_model_response_object(
response_object=stringified_response,
model_response_object=model_response,
response_type="embedding",
_response_headers=headers,
) # type: ignore
return returned_response
except OpenAIError as e:
## LOGGING
logging_obj.post_call(
@ -1284,7 +1286,6 @@ class OpenAIChatCompletion(BaseLLM):
aembedding=None,
):
super().embedding()
exception_mapping_worked = False
try:
model = model
data = {"model": model, "input": input, **optional_params}
@ -1299,7 +1300,7 @@ class OpenAIChatCompletion(BaseLLM):
)
if aembedding is True:
response = self.aembedding(
async_response = self.aembedding(
data=data,
input=input,
logging_obj=logging_obj,
@ -1310,7 +1311,7 @@ class OpenAIChatCompletion(BaseLLM):
client=client,
max_retries=max_retries,
)
return response
return async_response
openai_client: OpenAI = self._get_openai_client( # type: ignore
is_async=False,
@ -1335,12 +1336,13 @@ class OpenAIChatCompletion(BaseLLM):
additional_args={"complete_input_dict": data},
original_response=sync_embedding_response,
)
return convert_to_model_response_object(
response: litellm.EmbeddingResponse = convert_to_model_response_object(
response_object=sync_embedding_response.model_dump(),
model_response_object=model_response,
_response_headers=headers,
response_type="embedding",
) # type: ignore
return response
except OpenAIError as e:
raise e
except Exception as e: