2 commits

Author	SHA1	Message	Date
Ishaan Jaff	d81ae45827	(Perf / latency improvement) improve pass through endpoint latency to ~50ms (before PR was 400ms) (#6874) ... * use correct location for types * fix types location * perf improvement for pass through endpoints * update lint check * fix import * fix ensure async clients test * fix azure.py health check * fix ollama	2024-11-22 18:47:26 -08:00
Ishaan Jaff	920f4c9f82	(fix) add linting check to ban creating AsyncHTTPHandler during LLM calling (#6855) ... * fix triton * fix TEXT_COMPLETION_CODESTRAL * fix REPLICATE * fix CLARIFAI * fix HUGGINGFACE * add test_no_async_http_handler_usage * fix PREDIBASE * fix anthropic use get_async_httpx_client * fix vertex fine tuning * fix dbricks get_async_httpx_client * fix get_async_httpx_client vertex * fix get_async_httpx_client * fix get_async_httpx_client * fix make_async_azure_httpx_request * fix check_for_async_http_handler * test: cleanup mistral model * add check for AsyncClient * fix check_for_async_http_handler * fix get_async_httpx_client * fix tests using in_memory_llm_clients_cache * fix langfuse import * fix import --------- Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>	2024-11-21 19:03:02 -08:00