(feat) Add cost tracking for /batches requests OpenAI (#7384)

* add basic logging for create`batch`

* add create_batch as a call type

* add basic dd logging for batches

* basic batch creation logging on DD

* batch endpoints add cost calc

* fix batches_async_logging

* separate folder for batches testing

* new job for batches tests

* test batches logging

* fix validation logic

* add vertex_batch_completions.jsonl

* test test_async_create_batch

* test_async_create_batch

* update tests

* test_completion_with_no_model

* remove dead code

* update load_vertex_ai_credentials

* test_avertex_batch_prediction

* update get async httpx client

* fix get_async_httpx_client

* update test_avertex_batch_prediction

* fix batches testing config.yaml

* add google deps

* fix vertex files handler
This commit is contained in:
Ishaan Jaff 2024-12-23 17:47:26 -08:00 committed by GitHub
parent 9d66976162
commit 00544b97c8
13 changed files with 649 additions and 78 deletions

View file

@ -826,6 +826,11 @@ def completion( # type: ignore # noqa: PLR0915
- It supports various optional parameters for customizing the completion behavior.
- If 'mock_response' is provided, a mock completion response is returned for testing or debugging.
"""
### VALIDATE Request ###
if model is None:
raise ValueError("model param not passed in.")
# validate messages
messages = validate_chat_completion_messages(messages=messages)
######### unpacking kwargs #####################
args = locals()
api_base = kwargs.get("api_base", None)
@ -997,9 +1002,6 @@ def completion( # type: ignore # noqa: PLR0915
"aws_region_name", None
) # support region-based pricing for bedrock
### VALIDATE USER MESSAGES ###
messages = validate_chat_completion_messages(messages=messages)
### TIMEOUT LOGIC ###
timeout = timeout or kwargs.get("request_timeout", 600) or 600
# set timeout for 10 minutes by default