Commit graph

6033 commits

Author SHA1 Message Date
ishaan-jaff
8a4f2c198b (test) test_usage_based_routing 2024-01-19 14:59:56 -08:00
Krrish Dholakia
f5ced089d6 test(tests/): add unit testing for proxy server endpoints 2024-01-19 14:54:29 -08:00
ishaan-jaff
f408162449 (test) debug logs test - router 2024-01-19 14:22:18 -08:00
ishaan-jaff
0196ac6376 (test) router-fallbacks 2024-01-19 14:12:09 -08:00
Ishaan Jaff
b2b41727ce
Merge pull request #1524 from timothyasp/langsmith-project-env-bug-fix
Langsmith: Add envs for project/run names; fix bug with None metadata
2024-01-19 14:04:16 -08:00
ishaan-jaff
d0442ae0f2 (feat) router - usage based routing - consider input_tokens 2024-01-19 13:59:49 -08:00
ishaan-jaff
57a20100e4 (test) usage based routing 2024-01-19 13:58:08 -08:00
ishaan-jaff
8c0b7b1015 (feat) - improve router logging/debugging messages 2024-01-19 13:57:33 -08:00
ishaan-jaff
912e2de364 (test) usage based router + fallbacks 2024-01-19 12:34:08 -08:00
ishaan-jaff
7b2c15aa51 (feat) improve litellm.Router logging 2024-01-19 12:28:51 -08:00
Tim Asp
9d6a8b5de3 Cleaner fallback 2024-01-19 11:56:45 -08:00
Ishaan Jaff
650036071f
Merge pull request #1519 from BerriAI/litellm_proxy_make_success_handler_non_blocking
[Feat] litellm.acompletion() make Langfuse success handler non blocking
2024-01-19 11:41:00 -08:00
Tim Asp
315fbcd09f update docs 2024-01-19 11:39:39 -08:00
ishaan-jaff
8cf8da1378 (test) langfuse_latency_test_user 2024-01-19 11:38:44 -08:00
ishaan-jaff
cb99cd18e7 (feat) log cache_hit as langfuse tags 2024-01-19 11:38:18 -08:00
Tim Asp
27087e5662 fix error when metadata is none, prevents logging silently 2024-01-19 11:37:45 -08:00
Tim Asp
141d2219e2 Add env for project wide langsmith default project settings 2024-01-19 11:37:15 -08:00
ishaan-jaff
2f429f37b7 (test) test latency added with langfuse call 2024-01-19 11:28:10 -08:00
ishaan-jaff
cb40f58cd3 (fix) return usage in mock_completion 2024-01-19 11:25:47 -08:00
Krrish Dholakia
b03552790f build(config.yml): add aiohttp to circle ci 2024-01-19 10:52:53 -08:00
Krrish Dholakia
7a6b2888c2 build(config.yml): fix path for test folder 2024-01-19 10:49:31 -08:00
Krrish Dholakia
77f8db7050 build(config.yml): fix config.yml 2024-01-19 10:46:24 -08:00
ishaan-jaff
6a695477ba (fix) async langfuse logger 2024-01-19 10:44:51 -08:00
Krrish Dholakia
a0ac815859 build(config.yml): upgrade python version in docker build test 2024-01-19 10:44:39 -08:00
Krrish Dholakia
165bdd2fdf build(config.yml): modify config.yml to use python test script against docker image 2024-01-19 10:39:25 -08:00
Krrish Dholakia
326d7856e2 bump: version 1.18.4 → 1.18.5 2024-01-19 10:23:18 -08:00
Krrish Dholakia
1a29272b47 fix(parallel_request_limiter.py): handle tpm/rpm limits being null 2024-01-19 10:22:27 -08:00
ishaan-jaff
f2cfb76920 (fix) use asyncio run_in_executor 2024-01-19 09:52:51 -08:00
Krrish Dholakia
62e35f6a14 docs(users.md): add tpm rpm rate limits to docs 2024-01-19 08:56:47 -08:00
ishaan-jaff
e6b5152e63 (chore) update load test 2024-01-19 08:52:17 -08:00
ishaan-jaff
cd08a02764 (test) add blocking callback test 2024-01-19 08:51:52 -08:00
ishaan-jaff
a9c5b02303 (v0) fix 2024-01-19 08:51:14 -08:00
Krrish Dholakia
c5e144af23 docs(health.md): add /health/readiness and /health/liveliness to docs 2024-01-19 08:45:23 -08:00
Ishaan Jaff
9aaa2c417d
Merge pull request #1515 from ErikBjare/patch-1
nit: switch to valid SPDX license identifier `MIT` in pyproject.toml
2024-01-19 08:03:39 -08:00
Erik Bjäreholt
3eab194014
nit: switch to valid SPDX license identifier MIT in pyproject.toml
See: https://python-poetry.org/docs/pyproject/#license
2024-01-19 10:47:29 +01:00
Ishaan Jaff
8fef1a1b69
Merge pull request #1499 from duarteocarmo/patch-2
Adds s3_path prefix so that we can save objects in predifined location in s3 bucket
2024-01-18 21:22:15 -08:00
Ishaan Jaff
8338d04d1a
Merge pull request #1506 from HaruHunab1320/main
add headers to budget manager
2024-01-18 21:20:45 -08:00
Ishaan Jaff
79e261f533
Merge pull request #1509 from BerriAI/litellm_track_cost_user_id_chat_completions
[Feat] Proxy - Track Cost Per User (Using `user` passed to requests)
2024-01-18 20:44:02 -08:00
Keegan McCallum
3b719b2afd
Allow overriding headers for anthropic 2024-01-18 20:12:59 -08:00
Krrish Dholakia
b6b584fdba bump: version 1.18.3 → 1.18.4 2024-01-18 19:37:40 -08:00
Krrish Dholakia
f7694bc193 Merge branch 'main' into litellm_tpm_rpm_rate_limits 2024-01-18 19:10:07 -08:00
ishaan-jaff
5698be0df1 (fix) safe access litellm_params, proxy_server_request 2024-01-18 18:05:51 -08:00
Krrish Dholakia
94ce524c63 test(test_completion.py): handle together ai timeout 2024-01-18 17:54:16 -08:00
ishaan-jaff
16f3d7e0ed (feat) use user_id passed to request - cost track 2024-01-18 17:51:48 -08:00
ishaan-jaff
697c511e76 (feat) support user param for all providers 2024-01-18 17:45:59 -08:00
ishaan-jaff
ddd9ca86a7 (feat) proxy - track cost for user_ids that do not exist 2024-01-18 17:44:39 -08:00
Krrish Dholakia
b21a357ced bump: version 1.18.2 → 1.18.3 2024-01-18 17:14:22 -08:00
Krrish Dholakia
e0aaa94f28 fix(main.py): read azure ad token from optional params extra body 2024-01-18 17:14:03 -08:00
Krrish Dholakia
1e5efdfa37 fix(proxy_server.py): support setting tpm/rpm limits per user / per key 2024-01-18 17:03:18 -08:00
Ishaan Jaff
a8ba5df90e
Merge pull request #1500 from BerriAI/litellm_create_keys_with_team_id
[Feat] /key/generate - create keys with`team_id`
2024-01-18 16:35:14 -08:00