Krrish Dholakia
7880dcbdf7
fix(openai.py): support passing prompt as list instead of concat string
2024-04-03 15:23:20 -07:00
Krrish Dholakia
f3fcfe3007
fix(proxy_server.py): return original model response via response headers - /v1/completions
...
to help devs with debugging
2024-04-03 13:05:43 -07:00
yishiyiyuan
8619d29741
🐞 fix: djl vllm support
...
support vllm response format on sagemaker, which only return one choice.
2024-04-03 11:00:51 +08:00
Krrish Dholakia
de3b314620
fix(openai.py): switch to using openai sdk for text completion calls
2024-04-02 15:08:12 -07:00
Krrish Dholakia
ed46af19ec
fix(openai.py): return logprobs for text completion calls
2024-04-02 14:05:56 -07:00
jdhuang
b5dd970a76
Add sync iterator
2024-04-02 20:14:37 +08:00
Krrish Dholakia
5546f9f10a
fix(main.py): support max retries for transcription calls
2024-04-01 18:37:53 -07:00
DaxServer
947ba9d15b
docs: Update references to Ollama repository url
...
Updated references to the Ollama repository URL from https://github.com/jmorganca/ollama to https://github.com/ollama/ollama .
2024-03-31 19:35:37 +02:00
Krrish Dholakia
69f27aa25c
fix(factory.py): parse list in xml tool calling response (anthropic)
...
improves tool calling outparsing to check if list in response. Also returns the raw response back to the user via `response._hidden_params["original_response"]`, so user can see exactly what anthropic returned
2024-03-29 11:51:26 -07:00
Krish Dholakia
de31af0dc2
Merge pull request #2640 from mnicstruwig/fix/fix-xml-function-args-parsing
...
Fix XML function calling args parsing.
2024-03-29 10:11:52 -07:00
Krrish Dholakia
271fe184eb
fix(sagemaker.py): support model_id consistently. support dynamic args for async calls
2024-03-29 09:05:00 -07:00
Krrish Dholakia
62ac3e1de4
fix(sagemaker.py): support 'model_id' param for sagemaker
...
allow passing inference component param to sagemaker in the same format as we handle this for bedrock
2024-03-29 08:43:17 -07:00
Krrish Dholakia
f7dd1758bb
fix(ollama.py): fix type issue
2024-03-28 15:01:56 -07:00
Krish Dholakia
8f0b4457fe
Merge pull request #2720 from onukura/ollama-batch-embedding
...
Batch embedding for Ollama
2024-03-28 14:58:55 -07:00
Krrish Dholakia
46937935d1
feat(proxy/utils.py): enable updating db in a separate server
2024-03-27 16:02:36 -07:00
onukura
1bd60287ba
Add a feature to ollama aembedding to accept batch input
2024-03-27 21:39:19 +00:00
Krish Dholakia
e885c0faa7
Merge pull request #2701 from rmann-nflx/main
...
Updating the default Claude3 max tokens
2024-03-27 10:14:20 -07:00
Rob Mann
4b7c5e4c2e
Updating the default Claude3 max tokens
2024-03-26 11:46:59 -04:00
onukura
6ee8f26746
Fix ollama embedding response
2024-03-25 16:26:49 +00:00
Krrish Dholakia
47b2b0df17
fix(bedrock.py): fix supported openai params for bedrock claude 3
2024-03-23 16:02:15 -07:00
Krrish Dholakia
44c0a7f63e
fix(factory.py): fix anthropic check
2024-03-23 00:27:24 -07:00
Krrish Dholakia
16368fec98
fix(anthropic.py): support async claude 3 tool calling + streaming
...
https://github.com/BerriAI/litellm/issues/2644
2024-03-22 19:57:01 -07:00
Krrish Dholakia
4dad400b57
fix(anthropic.py): handle multiple system prompts
2024-03-22 18:14:15 -07:00
Krrish Dholakia
4d539295de
fix(ollama_chat.py): don't pop from dictionary while iterating through it
2024-03-22 08:18:22 -07:00
Michael Struwig
671c27065d
Fix XML function calling args parsing.
2024-03-22 15:05:29 +02:00
Krrish Dholakia
e8fbe9a9a5
fix(bedrock.py): support claude 3 function calling when stream=true
...
https://github.com/BerriAI/litellm/issues/2615
2024-03-21 18:39:03 -07:00
Krish Dholakia
2009133d87
Merge branch 'main' into litellm_llm_api_prompt_injection_check
2024-03-21 09:57:10 -07:00
Krrish Dholakia
859a2af3ed
build: fix mypy build issues
2024-03-21 08:27:23 -07:00
Krrish Dholakia
e9cc6b4cc9
feat(proxy_server.py): enable llm api based prompt injection checks
...
run user calls through an llm api to check for prompt injection attacks. This happens in parallel to th
e actual llm call using `async_moderation_hook`
2024-03-20 22:43:42 -07:00
Krrish Dholakia
9a221851d5
fix(handle_jwt.py): track spend for user using jwt auth
2024-03-20 10:55:52 -07:00
Krrish Dholakia
416cccdc6a
fix(utils.py): support response_format param for ollama
...
https://github.com/BerriAI/litellm/issues/2580
2024-03-19 21:07:20 -07:00
Krish Dholakia
b7fc51e2b8
Merge pull request #2558 from lucasmrdt/main
...
fix(anthropic): tool calling detection
2024-03-19 11:48:05 -07:00
Krish Dholakia
8be0db6bb8
Merge branch 'main' into litellm_non_openai_tool_call_prompt
2024-03-18 18:29:36 -07:00
garfeildma
0c72f0a145
support multiple system message tranlation for bedrock claude-3
2024-03-18 19:41:15 +08:00
lucasmrdt
a50b4fb485
fix(anthropic): tool calling detection
2024-03-16 16:06:38 -07:00
Krrish Dholakia
8ebf4a62dc
fix(proxy_server.py): bug fix on getting user obj from cache
2024-03-16 11:07:38 -07:00
Krish Dholakia
06cb2652ca
Merge branch 'main' into support_anthropic_function_result
2024-03-16 09:58:08 -07:00
Krrish Dholakia
235787ff67
fix(cohere.py): return usage as a pydantic object not dict
2024-03-15 10:00:22 -07:00
Krrish Dholakia
da5dc402c1
fix(bedrock.py): add all supported bedrock / anthropic messages api params
2024-03-15 09:41:40 -07:00
Krrish Dholakia
a0d2a786ea
fix(factory.py): raise exception on invalid message being received
2024-03-15 07:55:29 -07:00
Zihao Li
d3260204ac
Refactor to allow tool invoke conversion and tool results submission; Allow alternating consecutuve message roles and fix a bug
2024-03-15 22:47:11 +08:00
Zihao Li
31644c9503
Add function call result submission support for Claude 3 models
2024-03-15 11:34:54 +08:00
Krrish Dholakia
f40ec91578
fix(factory.py): fix mistral api prompt formatting
2024-03-13 11:34:45 -07:00
Krish Dholakia
ce3c865adb
Merge pull request #2472 from BerriAI/litellm_anthropic_streaming_tool_calling
...
fix(anthropic.py): support claude-3 streaming with function calling
2024-03-12 21:36:01 -07:00
Ishaan Jaff
2c4407bb04
Merge pull request #2479 from BerriAI/litellm_cohere_tool_call
...
[FEAT Cohere/command-r tool calling
2024-03-12 21:20:59 -07:00
Krrish Dholakia
cc36028a76
fix(anthropic.py): add more logging
2024-03-12 21:18:38 -07:00
Krrish Dholakia
c871d61218
fix(anthropic.py): bug fix
2024-03-12 19:32:42 -07:00
Krrish Dholakia
2877bb7e0a
fix(anthropic.py): concurrent request fix
2024-03-12 19:25:05 -07:00
Krrish Dholakia
709f6d6fd2
test(anthropic.py): more logging for test
2024-03-12 19:07:52 -07:00
ishaan-jaff
836029b5ab
(feat) support tool_calling on cohere command-r
2024-03-12 14:24:48 -07:00
