featu: support passing "extra body" throught to providers

# What does this PR do?
Allows passing through extra_body parameters to inference providers.


closes #2720

## Test Plan
CI and added new test
This commit is contained in:
Eric Huang 2025-10-10 16:00:30 -07:00
parent 80d58ab519
commit c4dbaa9d4c
41 changed files with 3145 additions and 200 deletions

View file

@ -98,7 +98,7 @@ paths:
content:
application/json:
schema:
$ref: '#/components/schemas/OpenAIChatCompletionRequest'
$ref: '#/components/schemas/OpenAIChatCompletionRequestWithExtraBody'
required: true
deprecated: false
/v1/chat/completions/{completion_id}:
@ -167,7 +167,7 @@ paths:
content:
application/json:
schema:
$ref: '#/components/schemas/OpenAICompletionRequest'
$ref: '#/components/schemas/OpenAICompletionRequestWithExtraBody'
required: true
deprecated: false
/v1/conversations:
@ -3824,7 +3824,7 @@ components:
title: OpenAIResponseFormatText
description: >-
Text response format for OpenAI-compatible chat completion requests.
OpenAIChatCompletionRequest:
OpenAIChatCompletionRequestWithExtraBody:
type: object
properties:
model:
@ -3966,7 +3966,7 @@ components:
required:
- model
- messages
title: OpenAIChatCompletionRequest
title: OpenAIChatCompletionRequestWithExtraBody
description: >-
Request parameters for OpenAI-compatible chat completion endpoint.
OpenAIChatCompletion:
@ -4134,7 +4134,7 @@ components:
- model
- input_messages
title: OpenAICompletionWithInputMessages
OpenAICompletionRequest:
OpenAICompletionRequestWithExtraBody:
type: object
properties:
model:
@ -4222,18 +4222,6 @@ components:
user:
type: string
description: (Optional) The user to use.
guided_choice:
type: array
items:
type: string
description: >-
(Optional) vLLM-specific parameter for guided generation with a list of
choices.
prompt_logprobs:
type: integer
description: >-
(Optional) vLLM-specific parameter for number of log probabilities to
return for prompt tokens.
suffix:
type: string
description: >-
@ -4242,7 +4230,7 @@ components:
required:
- model
- prompt
title: OpenAICompletionRequest
title: OpenAICompletionRequestWithExtraBody
description: >-
Request parameters for OpenAI-compatible completion endpoint.
OpenAICompletion: