Krish Dholakia
5edc987209
Merge pull request #1199 from neubig/add_safety_settings_default
...
Add a default for safety settings in vertex AI
2023-12-21 09:07:23 +05:30
ishaan-jaff
3e5cfee1f4
(ci/cd) run again
2023-12-21 07:25:22 +05:30
ishaan-jaff
b701a356cc
(fix) vertex ai auth file
2023-12-21 07:22:25 +05:30
Krrish Dholakia
6795f0447a
fix(utils.py): fix non_default_param pop error for ollama
2023-12-21 06:59:13 +05:30
David Manouchehri
93c4556eb0
Add aws_bedrock_runtime_endpoint support.
2023-12-20 19:31:43 -05:00
Graham Neubig
6e9267ca66
Make vertex_chat work with generate_content
2023-12-20 15:32:44 -05:00
Mateo Cámara
b72d372aa7
feat: added explicit args to acomplete
2023-12-20 19:49:12 +01:00
Graham Neubig
482b3b5bc3
Add a default for safety settings in vertex AI
2023-12-20 13:12:50 -05:00
Krrish Dholakia
04bbd0649f
fix(router.py): only do sync image gen fallbacks for now
...
The customhttptransport we use for dall-e-2 only works for sync httpx calls, not async. Will need to spend some time writing the async version
n
2023-12-20 19:10:59 +05:30
Krrish Dholakia
350389f501
fix(utils.py): add support for anyscale function calling
2023-12-20 17:48:33 +05:30
Krrish Dholakia
4040f60feb
feat(router.py): support async image generation on router
2023-12-20 17:24:20 +05:30
Krrish Dholakia
f355e03515
feat(main.py): add async image generation support
2023-12-20 16:58:40 +05:30
Krrish Dholakia
f59b9436be
feat(main.py): add async image generation support
2023-12-20 16:58:15 +05:30
Krrish Dholakia
b3962e483f
feat(azure.py): add support for azure image generations endpoint
2023-12-20 16:37:21 +05:30
AllentDan
6b19db0327
fix least_busy router by updating min_traffic
2023-12-20 18:16:00 +08:00
Krrish Dholakia
f0df28362a
feat(ollama.py): add support for ollama function calling
2023-12-20 14:59:55 +05:30
ishaan-jaff
683a1ee979
(feat) proxy key/generate pass metadata in requests
2023-12-20 13:42:49 +05:30
ishaan-jaff
7ad21de441
(feat) proxy /key/generate add metadata to _types
2023-12-20 13:42:49 +05:30
ishaan-jaff
c4b7ab6579
(feat) proxy - add metadata for keys
2023-12-20 13:42:49 +05:30
Krish Dholakia
93c4efd715
Merge pull request #1190 from neubig/add_vertexai_safety_settings
...
Add partial support of VertexAI safety settings
2023-12-20 07:31:10 +00:00
ishaan-jaff
229b56fc35
(docs) swagger - add embedding tag
2023-12-20 09:04:56 +05:30
Graham Neubig
2d15e5384b
Add partial support of vertexai safety settings
2023-12-19 22:26:55 -05:00
ishaan-jaff
aa78415894
(docs) swager - add embeddings tag
2023-12-20 06:29:36 +05:30
ishaan-jaff
9548334e2f
(docs) swagger docs add description
2023-12-20 06:27:26 +05:30
ishaan-jaff
8b26e64b5d
(fix) proxy: add link t swagger docs on startup
2023-12-20 06:02:05 +05:30
Krrish Dholakia
c5340b8709
fix(utils.py): vertex ai exception mapping
2023-12-19 15:25:29 +00:00
Krrish Dholakia
5936664a16
fix(ollama.py): raise async errors
2023-12-19 15:01:12 +00:00
ishaan-jaff
9995229b97
(fix) proxy + ollama - raise exception correctly
2023-12-19 18:48:34 +05:30
Max Deichmann
b93fe99b74
debug langfuse on env variable
2023-12-19 12:20:58 +01:00
Max Deichmann
a05a955653
add log
2023-12-19 10:06:03 +01:00
Max Deichmann
dc8ae16d84
add users
2023-12-19 09:54:51 +01:00
Max Deichmann
c397da7c59
changes
2023-12-19 09:41:49 +01:00
sumanth
96e23daf4d
fix
2023-12-19 13:15:48 +05:30
ishaan-jaff
ce1b0b89ba
(fix) proxy - health checks support cli model
2023-12-19 12:55:20 +05:30
Krish Dholakia
408f232bd7
Merge branch 'main' into main
2023-12-18 17:54:34 -08:00
Krrish Dholakia
071283c102
fix(router.py): init deployment_latency_map even if model_list is empty
2023-12-18 17:50:26 -08:00
Krrish Dholakia
34509d8dda
fix(main.py): return async completion calls
2023-12-18 17:41:54 -08:00
Max Deichmann
99034709dc
improve integration
2023-12-18 23:32:02 +01:00
ishaan-jaff
bd15c61a65
(feat) OR default transforms=[]
2023-12-18 10:59:09 +05:30
ishaan-jaff
3a97a2817f
(fix) default args batch completion
2023-12-18 10:05:44 +05:30
ishaan-jaff
6a0c853ae4
(feat) add open router transforms, models, route
2023-12-18 09:55:35 +05:30
ishaan-jaff
1e57c0c152
(feat) completion set function, function_call default None
2023-12-18 09:54:43 +05:30
ishaan-jaff
6b272076d7
(feat) openrouter set transforms=[] default
2023-12-18 09:16:33 +05:30
ishaan-jaff
b15682bc1f
(feat) set default openrouter configs
2023-12-18 08:55:51 +05:30
Joel Eriksson
e214e6ab47
Fix bug when iterating over lines in ollama response
...
async for line in resp.content.iter_any() will return
incomplete lines when the lines are long, and that
results in an exception being thrown by json.loads()
when it tries to parse the incomplete JSON
The default behavior of the stream reader for aiohttp
response objects is to iterate over lines, so just
removing .iter_any() fixes the bug
2023-12-17 20:23:26 +02:00
Joel Eriksson
a419d59542
Fix for issue that occured when proxying to ollama
...
In the text_completion() function, it previously threw an exception at:
raw_response = response._hidden_params.get("original_response", None)
Due to response being an coroutine object to an ollama_acompletion call,
so I added an asyncio.iscoroutine() check for the response and handle it
by calling response = asyncio.run(response)
I also had to fix atext_completion(), where init_response was an instance
of TextCompletionResponse.
Since this case was not handled by the if-elif that checks if init_response
is a coroutine, a dict or a ModelResponse instance, response was unbound
which threw an exception on the "return response" line.
Note that a regular pyright based linter detects that response is possibly
unbound, and that the same code pattern is used in multiple other places
in main.py.
I would suggest that you either change these cases:
init_response = await loop.run_in_executor(...
if isinstance(init_response, ...
response = init_response
elif asyncio.iscoroutine(init_response):
response = await init_response
To either just:
response = await loop.run_in_executor(
if asyncio.iscoroutine(response):
response = await response
Or at the very least, include an else statement and set response = init_response,
so that response is never unbound when the code proceeds.
2023-12-17 17:27:47 +02:00
Krrish Dholakia
a3c7a340a5
fix(ollama.py): fix sync ollama streaming
2023-12-16 21:23:21 -08:00
Krrish Dholakia
13d088b72e
feat(main.py): add support for image generation endpoint
2023-12-16 21:07:29 -08:00
Krrish Dholakia
7847ae1e23
fix(traceloop.py): add additional openllmetry traces
2023-12-16 19:21:39 -08:00
Krrish Dholakia
7c2fad2d57
fix(azure.py): fix azure streaming logging
2023-12-16 18:06:08 -08:00
