forked from phoenix/litellm-mirror
docs(routing.md): add retry_after to docs
This commit is contained in:
parent
4882325c35
commit
1f76b0e721
1 changed files with 19 additions and 0 deletions
|
@ -251,6 +251,25 @@ response = router.completion(model="gpt-3.5-turbo", messages=messages)
|
||||||
print(f"response: {response}")
|
print(f"response: {response}")
|
||||||
```
|
```
|
||||||
|
|
||||||
|
We also support setting minimum time to wait before retrying a failed request. This is via the `retry_after` param.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from litellm import Router
|
||||||
|
|
||||||
|
model_list = [{...}]
|
||||||
|
|
||||||
|
router = Router(model_list=model_list,
|
||||||
|
num_retries=3, retry_after=5) # waits min 5s before retrying request
|
||||||
|
|
||||||
|
user_message = "Hello, whats the weather in San Francisco??"
|
||||||
|
messages = [{"content": user_message, "role": "user"}]
|
||||||
|
|
||||||
|
# normal call
|
||||||
|
response = router.completion(model="gpt-3.5-turbo", messages=messages)
|
||||||
|
|
||||||
|
print(f"response: {response}")
|
||||||
|
```
|
||||||
|
|
||||||
### Fallbacks
|
### Fallbacks
|
||||||
|
|
||||||
If a call fails after num_retries, fall back to another model group.
|
If a call fails after num_retries, fall back to another model group.
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue