phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

(docs) input params

Browse source
This commit is contained in:
ishaan-jaff 2023-11-17 16:07:23 -08:00
parent 985583023a
commit fdbaceab8e
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/my-website/docs/completion/input.md
View file

@ -135,7 +135,7 @@ def completion(
- `response_format`: *object (optional)* - An object specifying the format that the model must output. - `response_format`: *object (optional)* - An object specifying the format that the model must output.
- Setting to { "type": "json_object" } enables JSON mode, which guarantees the message the model generates is valid JSON. - Setting to `{ "type": "json_object" }` enables JSON mode, which guarantees the message the model generates is valid JSON.
- Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if finish_reason="length", which indicates the generation exceeded max_tokens or the conversation exceeded the max context length. - Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if finish_reason="length", which indicates the generation exceeded max_tokens or the conversation exceeded the max context length.