Ishaan Jaff
|
7d680d411d
|
(proxy - RPS) - Get 2K RPS at 4 instances, minor fix aiohttp_openai/ (#7659)
* speed up transform_response
* use 2 workers
* undo changes to uvicorn
* ci/cd run again
|
2025-01-09 17:24:18 -08:00 |
|
Ishaan Jaff
|
5d3ae6da62
|
aiohttp_openai/ fixes - allow using aiohttp_openai/gpt-4o (#7598)
* fixes for get_complete_url
* update aiohttp tests
* fix event loop for aiohtto
* ci/cd run again
* test_aiohttp_openai
|
2025-01-06 21:39:11 -08:00 |
|
Ishaan Jaff
|
3a454ee2ce
|
(perf) use aiohttp for custom_openai (#7514)
* use aiohttp handler
* BaseLLMAIOHTTPHandler
* use CustomOpenAIChatConfig
* CustomOpenAIChatConfig
* CustomOpenAIChatConfig
* fix linting
* AiohttpOpenAIChatConfig
* fix order
* aiohttp_openai
|
2025-01-02 22:15:17 -08:00 |
|