phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 18:54:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix(hosted_vllm/transformation.py): return fake api key, if none give… (#7301)

Browse source 
* fix(hosted_vllm/transformation.py): return fake api key, if none give. Prevents httpx error

Fixes https://github.com/BerriAI/litellm/issues/7291

* test: fix test

* fix(main.py): add hosted_vllm/ support for embeddings endpoint

Closes https://github.com/BerriAI/litellm/issues/7290

* docs(vllm.md): add docs on vllm embeddings usage

* fix(__init__.py): fix sambanova model test

* fix(base_llm_unit_tests.py): skip pydantic obj test if model takes >5s to respond
This commit is contained in:
Krish Dholakia 2024-12-18 18:41:53 -08:00 • committed by GitHub
parent 70883bc1b8
commit e95820367f
9 changed files with 189 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

3
tests/llm_translation/base_llm_unit_tests.py
View file

@ -219,6 +219,7 @@ class BaseLLMChatTest(ABC):
},
],
response_format=TestModel,
timeout=5,
)
assert res is not None
@ -226,6 +227,8 @@ class BaseLLMChatTest(ABC):
assert res.choices[0].message.content is not None
assert res.choices[0].message.tool_calls is None
except litellm.Timeout:
pytest.skip("Model took too long to respond")
except litellm.InternalServerError:
pytest.skip("Model is overloaded")