phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

docs update Aporia Example

Browse source
This commit is contained in:
Ishaan Jaff 2024-08-20 07:56:40 -07:00
parent 192c8cb21d
commit 8b23a2334f
1 changed files with 7 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

13
docs/my-website/docs/tutorials/litellm_proxy_aporia.md
View file

@ -61,7 +61,7 @@ guardrails:
- `pre_call` Run **before** LLM call, on **input** - `pre_call` Run **before** LLM call, on **input**
- `post_call` Run **after** LLM call, on **input & output** - `post_call` Run **after** LLM call, on **input & output**
- `during_call` Run **during** LLM call, on **input** - `during_call` Run **during** LLM call, on **input** Same as `pre_call` but runs in parallel as LLM call. Response not returned until guardrail check completes
## 3. Start LiteLLM Gateway ## 3. Start LiteLLM Gateway
@ -72,6 +72,8 @@ litellm --config config.yaml --detailed_debug
## 4. Test request ## 4. Test request
**[Langchain, OpenAI SDK Usage Examples](../proxy/user_keys##request-format)**
<Tabs> <Tabs>
<TabItem label="Unsuccessful call" value = "not-allowed"> <TabItem label="Unsuccessful call" value = "not-allowed">
@ -134,11 +136,10 @@ curl -i http://localhost:4000/v1/chat/completions \
</Tabs> </Tabs>
## Advanced ## 5. Control Guardrails per Project (API Key)
### Control Guardrails per Project (API Key)
Use this to control what guardrails run per project. In this tutorial we only want the following guardrails to run for 1 project (API Key) Use this to control what guardrails run per project. In this tutorial we only want the following guardrails to run for 1 project (API Key)
- `guardrails`: ["aporia-pre-guard", "aporia"] - `guardrails`: ["aporia-pre-guard", "aporia-post-guard"]
**Step 1** Create Key with guardrail settings **Step 1** Create Key with guardrail settings
@ -150,7 +151,7 @@ curl -X POST 'http://0.0.0.0:4000/key/generate' \
-H 'Authorization: Bearer sk-1234' \ -H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \ -H 'Content-Type: application/json' \
-D '{ -D '{
"guardrails": ["aporia-pre-guard", "aporia"] "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
} }
}' }'
``` ```
@ -164,7 +165,7 @@ curl --location 'http://0.0.0.0:4000/key/update' \
--header 'Content-Type: application/json' \ --header 'Content-Type: application/json' \
--data '{ --data '{
"key": "sk-jNm1Zar7XfNdZXp49Z1kSQ", "key": "sk-jNm1Zar7XfNdZXp49Z1kSQ",
"guardrails": ["aporia-pre-guard", "aporia"] "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
} }
}' }'
``` ```