phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

Update README.md

Browse source
This commit is contained in:
Ishaan Jaff 2023-10-28 12:25:58 -07:00 • committed by GitHub
parent 347a5c46e2
commit 489d45cdfd
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 11 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

11
README.md
View file

@ -64,6 +64,7 @@ print(response)
liteLLM supports streaming the model response back, pass `stream=True` to get a streaming iterator in response. liteLLM supports streaming the model response back, pass `stream=True` to get a streaming iterator in response.
Streaming is supported for all models (Bedrock, Huggingface, TogetherAI, Azure, OpenAI, etc.) Streaming is supported for all models (Bedrock, Huggingface, TogetherAI, Azure, OpenAI, etc.)
```python ```python
from litellm import completion
response = completion(model="gpt-3.5-turbo", messages=messages, stream=True) response = completion(model="gpt-3.5-turbo", messages=messages, stream=True)
for chunk in response: for chunk in response:
print(chunk['choices'][0]['delta']) print(chunk['choices'][0]['delta'])
@ -74,6 +75,16 @@ for chunk in result:
print(chunk['choices'][0]['delta']) print(chunk['choices'][0]['delta'])
``` ```
## Reliability - Fallback LLMs
Never fail a request using LiteLLM
```python
from litellm import completion
# if gpt-4 fails, retry the request with gpt-3.5-turbo->command-nightly->claude-instant-1
response = completion(model="gpt-4",messages=messages, fallbacks=["gpt-3.5-turbo" "command-nightly", "claude-instant-1"])
```
## Supported Provider ([Docs](https://docs.litellm.ai/docs/providers)) ## Supported Provider ([Docs](https://docs.litellm.ai/docs/providers))
| Provider | [Completion](https://docs.litellm.ai/docs/#basic-usage) | [Streaming](https://docs.litellm.ai/docs/completion/stream#streaming-responses) | [Async Completion](https://docs.litellm.ai/docs/completion/stream#async-completion) | [Async Streaming](https://docs.litellm.ai/docs/completion/stream#async-streaming) | | Provider | [Completion](https://docs.litellm.ai/docs/#basic-usage) | [Streaming](https://docs.litellm.ai/docs/completion/stream#streaming-responses) | [Async Completion](https://docs.litellm.ai/docs/completion/stream#async-completion) | [Async Streaming](https://docs.litellm.ai/docs/completion/stream#async-streaming) |
| ------------- | ------------- | ------------- | ------------- | ------------- | | ------------- | ------------- | ------------- | ------------- | ------------- |