featu: support passing "extra body" throught to providers

# What does this PR do?
Allows passing through extra_body parameters to inference providers.


closes #2720

## Test Plan
CI and added new test
This commit is contained in:
Eric Huang 2025-10-10 15:46:56 -07:00
parent 80d58ab519
commit 10c7e67fca
35 changed files with 1893 additions and 200 deletions

View file

@ -101,7 +101,7 @@ paths:
content:
application/json:
schema:
$ref: '#/components/schemas/OpenAIChatCompletionRequest'
$ref: '#/components/schemas/OpenAIChatCompletionRequestWithExtraBody'
required: true
deprecated: false
/v1/chat/completions/{completion_id}:
@ -170,7 +170,7 @@ paths:
content:
application/json:
schema:
$ref: '#/components/schemas/OpenAICompletionRequest'
$ref: '#/components/schemas/OpenAICompletionRequestWithExtraBody'
required: true
deprecated: false
/v1/conversations:
@ -5269,7 +5269,7 @@ components:
title: OpenAIResponseFormatText
description: >-
Text response format for OpenAI-compatible chat completion requests.
OpenAIChatCompletionRequest:
OpenAIChatCompletionRequestWithExtraBody:
type: object
properties:
model:
@ -5411,7 +5411,7 @@ components:
required:
- model
- messages
title: OpenAIChatCompletionRequest
title: OpenAIChatCompletionRequestWithExtraBody
description: >-
Request parameters for OpenAI-compatible chat completion endpoint.
OpenAIChatCompletion:
@ -5579,7 +5579,7 @@ components:
- model
- input_messages
title: OpenAICompletionWithInputMessages
OpenAICompletionRequest:
OpenAICompletionRequestWithExtraBody:
type: object
properties:
model:
@ -5667,18 +5667,6 @@ components:
user:
type: string
description: (Optional) The user to use.
guided_choice:
type: array
items:
type: string
description: >-
(Optional) vLLM-specific parameter for guided generation with a list of
choices.
prompt_logprobs:
type: integer
description: >-
(Optional) vLLM-specific parameter for number of log probabilities to
return for prompt tokens.
suffix:
type: string
description: >-
@ -5687,7 +5675,7 @@ components:
required:
- model
- prompt
title: OpenAICompletionRequest
title: OpenAICompletionRequestWithExtraBody
description: >-
Request parameters for OpenAI-compatible completion endpoint.
OpenAICompletion: