From 901b0e690e7d0095205af0ffe4930f9b19ec5117 Mon Sep 17 00:00:00 2001 From: ishaan-jaff Date: Wed, 8 Nov 2023 17:55:11 -0800 Subject: [PATCH] (docs) simple proxy --- docs/my-website/docs/simple_proxy.md | 28 ---------------------------- 1 file changed, 28 deletions(-) diff --git a/docs/my-website/docs/simple_proxy.md b/docs/my-website/docs/simple_proxy.md index 567c57e61b..aacb3292fb 100644 --- a/docs/my-website/docs/simple_proxy.md +++ b/docs/my-website/docs/simple_proxy.md @@ -344,33 +344,6 @@ print(result) ## Advanced - -### Caching -#### Control caching per completion request -Caching can be switched on/off per /chat/completions request -- Caching on for completion - pass `caching=True`: - ```shell - curl http://0.0.0.0:8000/v1/chat/completions \ - -H "Content-Type: application/json" \ - -d '{ - "model": "gpt-3.5-turbo", - "messages": [{"role": "user", "content": "write a poem about litellm!"}], - "temperature": 0.7, - "caching": true - }' - ``` -- Caching off for completion - pass `caching=False`: - ```shell - curl http://0.0.0.0:8000/v1/chat/completions \ - -H "Content-Type: application/json" \ - -d '{ - "model": "gpt-3.5-turbo", - "messages": [{"role": "user", "content": "write a poem about litellm!"}], - "temperature": 0.7, - "caching": false - }' - ``` - ### Set Custom Prompt Templates LiteLLM by default checks if a model has a [prompt template and applies it](./completion/prompt_formatting.md) (e.g. if a huggingface model has a saved chat template in it's tokenizer_config.json). However, you can also set a custom prompt template on your proxy in the `config.yaml`: @@ -480,7 +453,6 @@ model_list: api_base: your_api_base # url where model is deployed ``` - ## Proxy CLI Arguments #### --host