Krrish Dholakia
a8f997eceb
feat(main.py): add async image generation support
2023-12-20 16:58:40 +05:30
Krrish Dholakia
23d0278739
feat(azure.py): add support for azure image generations endpoint
2023-12-20 16:37:21 +05:30
Krrish Dholakia
636ac9b605
feat(ollama.py): add support for ollama function calling
2023-12-20 14:59:55 +05:30
Krish Dholakia
7e3f9d344c
Merge branch 'main' into main
2023-12-18 17:54:34 -08:00
Krrish Dholakia
e03713ef74
fix(main.py): return async completion calls
2023-12-18 17:41:54 -08:00
ishaan-jaff
121149fed4
(feat) OR default transforms=[]
2023-12-18 10:59:09 +05:30
ishaan-jaff
37554f7e3e
(fix) default args batch completion
2023-12-18 10:05:44 +05:30
ishaan-jaff
49f57ddd54
(feat) completion set function, function_call default None
2023-12-18 09:54:43 +05:30
ishaan-jaff
759a4702d7
(feat) openrouter set transforms=[] default
2023-12-18 09:16:33 +05:30
Joel Eriksson
76e6b40337
Fix for issue that occured when proxying to ollama
...
In the text_completion() function, it previously threw an exception at:
raw_response = response._hidden_params.get("original_response", None)
Due to response being an coroutine object to an ollama_acompletion call,
so I added an asyncio.iscoroutine() check for the response and handle it
by calling response = asyncio.run(response)
I also had to fix atext_completion(), where init_response was an instance
of TextCompletionResponse.
Since this case was not handled by the if-elif that checks if init_response
is a coroutine, a dict or a ModelResponse instance, response was unbound
which threw an exception on the "return response" line.
Note that a regular pyright based linter detects that response is possibly
unbound, and that the same code pattern is used in multiple other places
in main.py.
I would suggest that you either change these cases:
init_response = await loop.run_in_executor(...
if isinstance(init_response, ...
response = init_response
elif asyncio.iscoroutine(init_response):
response = await init_response
To either just:
response = await loop.run_in_executor(
if asyncio.iscoroutine(response):
response = await response
Or at the very least, include an else statement and set response = init_response,
so that response is never unbound when the code proceeds.
2023-12-17 17:27:47 +02:00
Krrish Dholakia
5f4310f592
fix(ollama.py): fix sync ollama streaming
2023-12-16 21:23:21 -08:00
Krrish Dholakia
51cb16a015
feat(main.py): add support for image generation endpoint
2023-12-16 21:07:29 -08:00
Krrish Dholakia
662324afd4
docs(routing.md): add docs on using caching groups across deployments
2023-12-15 21:51:59 -08:00
Krrish Dholakia
e76ed6be7d
feat(router.py): support caching groups
2023-12-15 21:45:51 -08:00
ishaan-jaff
90bb6d4caf
(feat) add ollama/llava
2023-12-16 10:35:27 +05:30
Krrish Dholakia
726289cb49
fix(huggingface_restapi.py): add support for additional hf embedding formats
2023-12-15 21:02:41 -08:00
Krrish Dholakia
3d6ade8f26
fix(ollama.py): fix ollama async streaming for /completions calls
2023-12-15 09:28:32 -08:00
Krrish Dholakia
096f3baea6
fix(main.py): log user for embedding calls
2023-12-14 15:14:37 -08:00
Krrish Dholakia
bb5b883316
fix(main.py): support async streaming for text completions endpoint
2023-12-14 13:56:32 -08:00
ishaan-jaff
303d9aa286
(feat) add mistral api
2023-12-14 18:17:48 +05:30
ishaan-jaff
773b1adf5e
(fix) aembedding - don't pop aembedding out
2023-12-14 17:13:35 +05:30
Ishaan Jaff
9bb69d10d0
Merge pull request #1112 from Undertone0809/add-cs-return-type-for-completion
...
refactor: add CustomStreamWrapper return type for completion
2023-12-14 16:52:11 +05:30
Krrish Dholakia
2231601d5a
fix(ollama.py): fix async completion calls for ollama
2023-12-13 13:10:25 -08:00
Krrish Dholakia
13a7f2dd58
fix(vertex_ai.py): add support for real async streaming + completion calls
2023-12-13 11:53:55 -08:00
Krrish Dholakia
f51d4ab26d
fix(vertex_ai.py): support optional params + enable async calls for gemini
2023-12-13 11:01:23 -08:00
Krrish Dholakia
43b160d70d
feat(vertex_ai.py): adds support for gemini-pro on vertex ai
2023-12-13 10:26:30 -08:00
ishaan-jaff
4bcddec2ce
(feat) pass vertex_ai/ as custom_llm_provider
2023-12-13 19:02:24 +03:00
zeeland
ac30309909
refactor: add CustomStreamWrapper return type for completion
2023-12-13 22:57:19 +08:00
Krrish Dholakia
e396fcb55c
fix(main.py): pass user_id + encoding_format for logging + to openai/azure
2023-12-12 15:46:44 -08:00
Krrish Dholakia
93c7393ae8
fix(utils.py): add more logging
2023-12-12 15:46:12 -08:00
Krrish Dholakia
0a3320ed6b
fix(utils.py): add more logging
2023-12-12 15:46:00 -08:00
ishaan-jaff
d540bdf824
(fix) from re import T - junk import
2023-12-12 12:26:15 -08:00
Krrish Dholakia
0cf0c2d6dd
fix(router.py): deepcopy initial model list, don't mutate it
2023-12-12 09:54:06 -08:00
Krrish Dholakia
b80a81b419
fix(ollama.py): enable parallel ollama completion calls
2023-12-11 23:18:37 -08:00
Krrish Dholakia
4fd400015f
test(test_custom_callback_router.py): add async azure testing for router
2023-12-11 16:40:35 -08:00
Krrish Dholakia
02cfefa257
test(test_custom_callback_input.py): embedding callback tests for azure, openai, bedrock
2023-12-11 15:32:46 -08:00
Krrish Dholakia
72591a2165
test(test_custom_callback_input.py): add bedrock testing
...
n
n
2023-12-11 13:00:01 -08:00
Krrish Dholakia
47d0884c0c
test(test_custom_callback_unit.py): adding unit tests for custom callbacks + fixing related bugs
2023-12-11 11:44:09 -08:00
ishaan-jaff
0e3f7ea28f
(feat) access metadata in embedding kwargs()
2023-12-11 09:39:25 -08:00
ishaan-jaff
33d6b5206d
(feat) caching + stream - bedrock
2023-12-11 08:43:50 -08:00
Krish Dholakia
3ebb7c7dc7
Merge pull request #985 from estill01/patch-1
...
Enable setting default `model` value for `LiteLLM`, `Chat`, `Completions`
2023-12-09 13:59:00 -08:00
ishaan-jaff
e056696831
(feat) custom logger: async stream,assemble chunks
2023-12-09 10:10:48 -08:00
ishaan-jaff
ee8b23cf91
(feat) proxy: log model_info + proxy_server request
2023-12-08 14:26:18 -08:00
ishaan-jaff
d7af5f1d85
(feat) embedding - pass model_info, proxy_server request
2023-12-08 14:26:18 -08:00
ishaan-jaff
de6880dc09
(feat) pass model_info, proxy_server_request to callback
2023-12-08 14:26:18 -08:00
ishaan-jaff
f744445db4
(fix) make print_verbose non blocking
2023-12-07 17:31:32 -08:00
Krrish Dholakia
583de6ab92
fix(bedrock.py): fix output format for cohere embeddings
2023-12-06 22:47:01 -08:00
ishaan-jaff
40faeb6cd7
(feat) aembedding - add custom logging support
2023-12-06 19:09:06 -08:00
Krrish Dholakia
45e9c3eb31
feat(sagemaker.py): support huggingface embedding models
2023-12-06 11:41:38 -08:00
Krrish Dholakia
f9b74e54a3
fix(sagemaker.py): enable passing hf model name for prompt template
2023-12-05 16:31:59 -08:00
