forked from phoenix/litellm-mirror
(docs) caching use context manager
This commit is contained in:
parent
e35f17ca3c
commit
1c93642951
1 changed files with 32 additions and 1 deletions
|
@ -28,8 +28,39 @@ response2 = completion(
|
|||
# response1 == response2, response 1 is cached
|
||||
```
|
||||
|
||||
### Custom Cache Keys:
|
||||
## Cache Context Manager - Enable, Disable, Update Cache
|
||||
Use the context manager for easily enabling, disabling & updating the litellm cache
|
||||
|
||||
### Enabling Cache
|
||||
|
||||
Quick Start Enable
|
||||
```python
|
||||
litellm.enable_cache()
|
||||
```
|
||||
|
||||
Advanced Params
|
||||
|
||||
```python
|
||||
litellm.enable_cache(
|
||||
type: Optional[Literal["local", "redis"]] = "local",
|
||||
host: Optional[str] = None,
|
||||
port: Optional[str] = None,
|
||||
password: Optional[str] = None,
|
||||
supported_call_types: Optional[
|
||||
List[Literal["completion", "acompletion", "embedding", "aembedding"]]
|
||||
] = ["completion", "acompletion", "embedding", "aembedding"],
|
||||
**kwargs,
|
||||
)
|
||||
```
|
||||
|
||||
### Disabling Cache
|
||||
|
||||
Switch caching off
|
||||
```python
|
||||
litellm.disable_cache()
|
||||
```
|
||||
|
||||
## Custom Cache Keys:
|
||||
Define function to return cache key
|
||||
```python
|
||||
# this function takes in *args, **kwargs and returns the key you want to use for caching
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue