phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

docs(proxy_server.md): add logs, save keys, model fallbacks, config file template to proxy server docs

Browse source
This commit is contained in:
Krrish Dholakia 2023-10-14 10:51:54 -07:00
parent 342925814b
commit fc757dc1b4
3 changed files with 194 additions and 51 deletions
Download patch file Download diff file Expand all files Collapse all files

1
docs/my-website/docs/tutorials/model_config_proxy.md
View file

@ -3,6 +3,7 @@ import Image from '@theme/IdealImage';
# Customize Prompt Templates on OpenAI-Compatible server
**You will learn:** How to set a custom prompt template on our OpenAI compatible server.
**How?** We will modify the prompt template for CodeLlama
## Step 1: Start OpenAI Compatible server
Let's spin up a local OpenAI-compatible server, to call a deployed `codellama/CodeLlama-34b-Instruct-hf` model using Huggingface's [Text-Generation-Inference (TGI)](https://github.com/huggingface/text-generation-inference) format.