phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-25 10:44:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs responses routing
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 19s
Details
Helm unit test / unit-test (push) Successful in 52s
Details

Browse source
This commit is contained in:
Ishaan Jaff 2025-04-21 23:05:53 -07:00
parent a7db0df043
commit ebfff975d4
1 changed files with 0 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

7
docs/my-website/docs/response_api.md
View file

@ -631,10 +631,3 @@ follow_up = client.responses.create(
</TabItem> </TabItem>
</Tabs> </Tabs>
#### How It Works
1. When a user makes an initial request to the Responses API, LiteLLM caches which model deployment that returned the specific response. (Stored in Redis if you connected LiteLLM to Redis)
2. When a subsequent request includes `previous_response_id`, LiteLLM automatically routes it to the same deployment
3. If the original deployment is unavailable, or if the `previous_response_id` isn't found in the cache, LiteLLM falls back to normal routing