Ishaan Jaff
|
a85de46ef7
|
(proxy - RPS) - Get 2K RPS at 4 instances, minor fix aiohttp_openai/ (#7659)
* speed up transform_response
* use 2 workers
* undo changes to uvicorn
* ci/cd run again
|
2025-01-09 17:24:18 -08:00 |
|
Ishaan Jaff
|
2ca0977921
|
aiohttp_openai/ fixes - allow using aiohttp_openai/gpt-4o (#7598)
* fixes for get_complete_url
* update aiohttp tests
* fix event loop for aiohtto
* ci/cd run again
* test_aiohttp_openai
|
2025-01-06 21:39:11 -08:00 |
|
Ishaan Jaff
|
23104d9a14
|
test_aiohttp_openai
|
2025-01-03 15:12:56 -08:00 |
|
Ishaan Jaff
|
d861aa8ff3
|
(perf) use aiohttp for custom_openai (#7514)
* use aiohttp handler
* BaseLLMAIOHTTPHandler
* use CustomOpenAIChatConfig
* CustomOpenAIChatConfig
* CustomOpenAIChatConfig
* fix linting
* AiohttpOpenAIChatConfig
* fix order
* aiohttp_openai
|
2025-01-02 22:15:17 -08:00 |
|