phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

docs(pii_masking.md): add output parsing to docs

Browse source
This commit is contained in:
Krrish Dholakia 2024-02-14 12:09:05 -08:00
parent f93091419e
commit a61b0bf9e5
1 changed files with 29 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

33
docs/my-website/docs/proxy/pii_masking.md
View file

@ -4,21 +4,22 @@ import Image from '@theme/IdealImage';
LiteLLM supports [Microsoft Presidio](https://github.com/microsoft/presidio/) for PII masking. LiteLLM supports [Microsoft Presidio](https://github.com/microsoft/presidio/) for PII masking.
## Step 1. Add env ## Quick Start
### Step 1. Add env
```bash ```bash
export PRESIDIO_ANALYZER_API_BASE="http://localhost:5002" export PRESIDIO_ANALYZER_API_BASE="http://localhost:5002"
export PRESIDIO_ANONYMIZER_API_BASE="http://localhost:5001" export PRESIDIO_ANONYMIZER_API_BASE="http://localhost:5001"
``` ```
## Step 2. Set it as a callback in config.yaml ### Step 2. Set it as a callback in config.yaml
```yaml ```yaml
litellm_settings: litellm_settings:
callbacks = ["presidio", ...] # e.g. ["presidio", custom_callbacks.proxy_handler_instance] callbacks = ["presidio", ...] # e.g. ["presidio", custom_callbacks.proxy_handler_instance]
``` ```
## Start proxy ### Step 3. Start proxy
``` ```
litellm --config /path/to/config.yaml litellm --config /path/to/config.yaml
@ -27,4 +28,28 @@ litellm --config /path/to/config.yaml
This will mask the input going to the llm provider This will mask the input going to the llm provider
<Image img={require('../../img/presidio_screenshot.png')} /> <Image img={require('../../img/presidio_screenshot.png')} />
## Output parsing
LLM responses can sometimes contain the masked tokens.
For presidio 'replace' operations, LiteLLM can check the LLM response and replace the masked token with the user-submitted values.
Just set `litellm.output_parse_pii = True`, to enable this.
```yaml
litellm_settings:
output_parse_pii: true
```
**Expected Flow: **
1. User Input: "hello world, my name is Jane Doe. My number is: 034453334"
2. LLM Input: "hello world, my name is [PERSON]. My number is: [PHONE_NUMBER]"
3. LLM Response: "Hey [PERSON], nice to meet you!"
4. User Response: "Hey Jane Doe, nice to meet you!"