forked from phoenix/litellm-mirror
update docs
This commit is contained in:
parent
6e6ec8c65f
commit
e8f48c8a72
1 changed files with 34 additions and 0 deletions
34
docs/stream.md
Normal file
34
docs/stream.md
Normal file
|
@ -0,0 +1,34 @@
|
||||||
|
# Streaming Responses & Async Completion
|
||||||
|
|
||||||
|
- [Streaming Responses](#streaming-responses)
|
||||||
|
- [Async Completion](#async-completion)
|
||||||
|
|
||||||
|
LiteLLM supports streaming the model response back by passing `stream=True` as an argument to the completion function
|
||||||
|
|
||||||
|
## Streaming Responses
|
||||||
|
### Usage
|
||||||
|
```python
|
||||||
|
response = completion(model="gpt-3.5-turbo", messages=messages, stream=True)
|
||||||
|
for chunk in response:
|
||||||
|
print(chunk['choices'][0]['delta'])
|
||||||
|
|
||||||
|
```
|
||||||
|
Asynchronous Completion with LiteLLM
|
||||||
|
LiteLLM provides an asynchronous version of the completion function called `acompletion`
|
||||||
|
|
||||||
|
## Async Completion
|
||||||
|
### Usage
|
||||||
|
```
|
||||||
|
from litellm import acompletion
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
async def test_get_response():
|
||||||
|
user_message = "Hello, how are you?"
|
||||||
|
messages = [{"content": user_message, "role": "user"}]
|
||||||
|
response = await acompletion(model="gpt-3.5-turbo", messages=messages)
|
||||||
|
return response
|
||||||
|
|
||||||
|
response = asyncio.run(test_get_response())
|
||||||
|
print(response)
|
||||||
|
|
||||||
|
```
|
Loading…
Add table
Add a link
Reference in a new issue