featu: support passing "extra body" throught to providers

# What does this PR do?
Allows passing through extra_body parameters to inference providers.


closes #2720

## Test Plan
CI and added new test
This commit is contained in:
Eric Huang 2025-10-10 15:53:56 -07:00
parent 80d58ab519
commit 28dff0b89e
37 changed files with 2964 additions and 200 deletions

View file

@ -1098,7 +1098,7 @@ paths:
content:
application/json:
schema:
$ref: '#/components/schemas/OpenAIChatCompletionRequest'
$ref: '#/components/schemas/OpenAIChatCompletionRequestWithExtraBody'
required: true
deprecated: true
/v1/openai/v1/chat/completions/{completion_id}:
@ -1167,7 +1167,7 @@ paths:
content:
application/json:
schema:
$ref: '#/components/schemas/OpenAICompletionRequest'
$ref: '#/components/schemas/OpenAICompletionRequestWithExtraBody'
required: true
deprecated: true
/v1/openai/v1/embeddings:
@ -5575,7 +5575,7 @@ components:
title: OpenAIResponseFormatText
description: >-
Text response format for OpenAI-compatible chat completion requests.
OpenAIChatCompletionRequest:
OpenAIChatCompletionRequestWithExtraBody:
type: object
properties:
model:
@ -5717,7 +5717,7 @@ components:
required:
- model
- messages
title: OpenAIChatCompletionRequest
title: OpenAIChatCompletionRequestWithExtraBody
description: >-
Request parameters for OpenAI-compatible chat completion endpoint.
OpenAIChatCompletion:
@ -5885,7 +5885,7 @@ components:
- model
- input_messages
title: OpenAICompletionWithInputMessages
OpenAICompletionRequest:
OpenAICompletionRequestWithExtraBody:
type: object
properties:
model:
@ -5973,18 +5973,6 @@ components:
user:
type: string
description: (Optional) The user to use.
guided_choice:
type: array
items:
type: string
description: >-
(Optional) vLLM-specific parameter for guided generation with a list of
choices.
prompt_logprobs:
type: integer
description: >-
(Optional) vLLM-specific parameter for number of log probabilities to
return for prompt tokens.
suffix:
type: string
description: >-
@ -5993,7 +5981,7 @@ components:
required:
- model
- prompt
title: OpenAICompletionRequest
title: OpenAICompletionRequestWithExtraBody
description: >-
Request parameters for OpenAI-compatible completion endpoint.
OpenAICompletion: