phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-26 03:04:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

(feat) Add cost tracking for /batches requests OpenAI (#7384)

Browse source 
* add basic logging for create`batch`

* add create_batch as a call type

* add basic dd logging for batches

* basic batch creation logging on DD

* batch endpoints add cost calc

* fix batches_async_logging

* separate folder for batches testing

* new job for batches tests

* test batches logging

* fix validation logic

* add vertex_batch_completions.jsonl

* test test_async_create_batch

* test_async_create_batch

* update tests

* test_completion_with_no_model

* remove dead code

* update load_vertex_ai_credentials

* test_avertex_batch_prediction

* update get async httpx client

* fix get_async_httpx_client

* update test_avertex_batch_prediction

* fix batches testing config.yaml

* add google deps

* fix vertex files handler
This commit is contained in:
Ishaan Jaff 2024-12-23 17:47:26 -08:00 • committed by GitHub
parent 87f19d6f13
commit 05b0d2026f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 649 additions and 78 deletions
Download patch file Download diff file Expand all files Collapse all files

3
litellm/constants.py
View file

@ -92,3 +92,6 @@ BEDROCK_AGENT_RUNTIME_PASS_THROUGH_ROUTES = [
"generateQuery/",
"optimize-prompt/",
]
BATCH_STATUS_POLL_INTERVAL_SECONDS = 10
BATCH_STATUS_POLL_MAX_ATTEMPTS = 10