phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 11:14:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

(Bug Fix) Add health check support for realtime models (#7453)

Browse source 
* add mode: realtime

* add _realtime_health_check

* test_realtime_health_check

* azure _realtime_health_check

* _realtime_health_check

* Realtime Models

* fix code quality
This commit is contained in:
Ishaan Jaff 2024-12-28 18:15:00 -08:00 • committed by GitHub
parent 6eabc7268a
commit 4825e3eee5
6 changed files with 110 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

12
litellm/llms/azure/azure.py
View file

@ -1585,6 +1585,18 @@ class AzureChatCompletion(BaseLLM):
)
elif mode == "batch":
completion = await client.batches.with_raw_response.list(limit=1) # type: ignore
elif mode == "realtime":
from litellm.realtime_api.main import _realtime_health_check
# create a websocket connection
await _realtime_health_check(
model=model or "",
api_key=api_key,
api_base=api_base,
api_version=api_version,
custom_llm_provider="azure",
)
return {}
else:
raise Exception("mode not set")
response = {}