phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

docs link to litellm batch completions

Browse source
This commit is contained in:
Ishaan Jaff 2024-05-11 13:45:32 -07:00
parent 5918ee543b
commit 62276fc221
1 changed files with 6 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/my-website/docs/completion/batching.md
View file

@ -4,6 +4,12 @@ LiteLLM allows you to:
* Send 1 completion call to many models: Return Fastest Response
* Send 1 completion call to many models: Return All Responses
:::info
Trying to do batch completion on LiteLLM Proxy ? Go here: https://docs.litellm.ai/docs/proxy/user_keys#beta-batch-completions---pass-model-as-list
:::
## Send multiple completion calls to 1 model
In the batch_completion method, you provide a list of `messages` where each sub-list of messages is passed to `litellm.completion()`, allowing you to process multiple prompts efficiently in a single API call.