forked from phoenix/litellm-mirror
(docs) caching use context manager
This commit is contained in:
parent
e35f17ca3c
commit
1c93642951
1 changed files with 32 additions and 1 deletions
|
@ -28,8 +28,39 @@ response2 = completion(
|
||||||
# response1 == response2, response 1 is cached
|
# response1 == response2, response 1 is cached
|
||||||
```
|
```
|
||||||
|
|
||||||
### Custom Cache Keys:
|
## Cache Context Manager - Enable, Disable, Update Cache
|
||||||
|
Use the context manager for easily enabling, disabling & updating the litellm cache
|
||||||
|
|
||||||
|
### Enabling Cache
|
||||||
|
|
||||||
|
Quick Start Enable
|
||||||
|
```python
|
||||||
|
litellm.enable_cache()
|
||||||
|
```
|
||||||
|
|
||||||
|
Advanced Params
|
||||||
|
|
||||||
|
```python
|
||||||
|
litellm.enable_cache(
|
||||||
|
type: Optional[Literal["local", "redis"]] = "local",
|
||||||
|
host: Optional[str] = None,
|
||||||
|
port: Optional[str] = None,
|
||||||
|
password: Optional[str] = None,
|
||||||
|
supported_call_types: Optional[
|
||||||
|
List[Literal["completion", "acompletion", "embedding", "aembedding"]]
|
||||||
|
] = ["completion", "acompletion", "embedding", "aembedding"],
|
||||||
|
**kwargs,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Disabling Cache
|
||||||
|
|
||||||
|
Switch caching off
|
||||||
|
```python
|
||||||
|
litellm.disable_cache()
|
||||||
|
```
|
||||||
|
|
||||||
|
## Custom Cache Keys:
|
||||||
Define function to return cache key
|
Define function to return cache key
|
||||||
```python
|
```python
|
||||||
# this function takes in *args, **kwargs and returns the key you want to use for caching
|
# this function takes in *args, **kwargs and returns the key you want to use for caching
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue