Ishaan Jaff
ba247d0104
[Fix] Tag Based Routing not work with wildcard routing ( #5805 )
...
* allow using tag routing for free
* only enforce tags for teams / keys
2024-09-20 14:05:56 -07:00
Ishaan Jaff
0927bd789d
[Fix] o1-mini causes pydantic warnings on reasoning_tokens ( #5754 )
...
* add requester_metadata in standard logging payload
* log requester_metadata in metadata
* use StandardLoggingPayload for logging
* docs StandardLoggingPayload
* fix import
* include standard logging object in failure
* add test for requester metadata
* handle completion_tokens_details
* add test for completion_tokens_details
2024-09-17 20:23:14 -07:00
Krish Dholakia
dec53961f7
LiteLLM Minor Fixes and Improvements (11/09/2024) ( #5634 )
...
* fix(caching.py): set ttl for async_increment cache
fixes issue where ttl for redis client was not being set on increment_cache
Fixes https://github.com/BerriAI/litellm/issues/5609
* fix(caching.py): fix increment cache w/ ttl for sync increment cache on redis
Fixes https://github.com/BerriAI/litellm/issues/5609
* fix(router.py): support adding retry policy + allowed fails policy via config.yaml
* fix(router.py): don't cooldown single deployments
No point, as there's no other deployment to loadbalance with.
* fix(user_api_key_auth.py): support setting allowed email domains on jwt tokens
Closes https://github.com/BerriAI/litellm/issues/5605
* docs(token_auth.md): add user upsert + allowed email domain to jwt auth docs
* fix(litellm_pre_call_utils.py): fix dynamic key logging when team id is set
Fixes issue where key logging would not be set if team metadata was not none
* fix(secret_managers/main.py): load environment variables correctly
Fixes issue where os.environ/ was not being loaded correctly
* test(test_router.py): fix test
* feat(spend_tracking_utils.py): support logging additional usage params - e.g. prompt caching values for deepseek
* test: fix tests
* test: fix test
* test: fix test
* test: fix test
* test: fix test
2024-09-11 22:36:06 -07:00
Ishaan Jaff
d303a3d03c
fix log failures for key based logging
2024-09-09 16:33:06 -07:00
Ishaan Jaff
7b1e0beda5
fix allow setting language per call to presidio
2024-09-04 12:46:59 -07:00
Krrish Dholakia
fa609c6d54
fix(litellm_pre_call_utils.py): don't override k-v pair sent in spend_logs_metadata by user
2024-08-23 07:10:18 -07:00
Krrish Dholakia
ac5c6c8751
fix(litellm_pre_call_utils.py): handle dynamic keys via api correctly
2024-08-21 13:37:21 -07:00
Krrish Dholakia
7eb474b87c
feat(litellm_pre_call_utils.py): support passing tags/spend logs metadata from keys/team metadata to request
2024-08-21 08:13:36 -07:00
Ishaan Jaff
83fe4909f2
enforece guardrails per API Key as enterprise
2024-08-20 17:34:28 -07:00
Krrish Dholakia
0af1d602ca
fix(litellm_pre_call_utils.py): only pass api_version if set
2024-08-20 16:00:46 -07:00
Krish Dholakia
85c5d4f8c2
Merge branch 'main' into litellm_fix_azure_api_version
2024-08-20 11:40:53 -07:00
Ishaan Jaff
09d67eb7a9
test guardrails with API Key
2024-08-20 08:40:00 -07:00
Krrish Dholakia
2b4d692353
fix(litellm_pre_call_utils.py): handle no query params in request
2024-08-19 21:09:03 -07:00
Ishaan Jaff
bcd58be51b
feat - guardrails v2
2024-08-19 21:03:37 -07:00
Krrish Dholakia
cc341ddf35
feat(langfuse_endpoints.py): support team based logging for langfuse pass-through endpoints
2024-08-19 21:03:37 -07:00
Ishaan Jaff
c82714757a
Merge pull request #5288 from BerriAI/litellm_aporia_refactor
...
[Feat] V2 aporia guardrails litellm
2024-08-19 20:41:45 -07:00
Ishaan Jaff
319690ab5e
feat - guardrails v2
2024-08-19 18:24:20 -07:00
Krrish Dholakia
a9025280bd
feat(langfuse_endpoints.py): support team based logging for langfuse pass-through endpoints
2024-08-19 17:58:39 -07:00
Krrish Dholakia
d08479b52c
feat(azure.py): support dynamic api versions
...
Closes https://github.com/BerriAI/litellm/issues/5228
2024-08-19 12:17:43 -07:00
Ishaan Jaff
62314c7f8e
fix using prompt caching on proxy
2024-08-15 20:12:11 -07:00
Krish Dholakia
3445307297
Merge pull request #5176 from BerriAI/litellm_key_logging
...
Allow specifying langfuse project for logging in key metadata
2024-08-14 12:55:07 -07:00
Ishaan Jaff
d872a7e32e
feat log use_x_forwarded_for
2024-08-13 15:22:54 -07:00
Krrish Dholakia
9fcb6f8f57
fix(litellm_pre_call_utils.py): support routing to logging project by api key
2024-08-12 21:21:40 -07:00
Ishaan Jaff
866519b659
use itellm.forward_traceparent_to_llm_provider
2024-08-01 09:05:13 -07:00
Ishaan Jaff
46a441cfd1
fix batches inserting metadata
2024-07-26 18:08:54 -07:00
Ishaan Jaff
344010e127
Pass litellm proxy specific metadata
2024-07-23 15:31:30 -07:00
Ishaan Jaff
e9f26d4e36
add debug logging for team callback settings
2024-07-23 08:41:05 -07:00
Ishaan Jaff
ac8d008942
control team callbacks using API
2024-07-22 18:29:21 -07:00
Ishaan Jaff
5aa7430d80
add tags to metadata
2024-07-18 21:55:53 -07:00
Ishaan Jaff
c2856c6e8f
check if using tag based routing
2024-07-18 20:10:45 -07:00
Ishaan Jaff
52682ea0cc
fix remove previous code on free/paid tier
2024-07-18 19:24:13 -07:00
Ishaan Jaff
c9c9d9239f
feat - enterprise
2024-07-18 17:15:47 -07:00
Ishaan Jaff
f43fa3cfec
fix pre call utils on embedding
2024-07-17 18:29:34 -07:00
Ishaan Jaff
bce7b5f8c8
feat - support /create assistants endpoint
2024-07-09 10:03:47 -07:00
Ishaan Jaff
7bbf4047e7
track user_ip address per request
2024-07-08 09:00:08 -07:00
Krrish Dholakia
1dae0a5b6a
fix(utils.py): cleanup 'additionalProperties=False' for tool calling with zod
...
Fixes issue with zod passing in additionalProperties=False, causing vertex ai / gemini calls to fail
2024-07-06 17:27:37 -07:00
Ishaan Jaff
bdb812ff97
Merge branch 'main' into litellm_fix_in_mem_usage
2024-06-27 21:12:06 -07:00
Ishaan Jaff
196d15967b
fix pre call utils adding extra headers
2024-06-27 21:03:36 -07:00
Ishaan Jaff
0ac77551be
forward otel traceparent in request headers
2024-06-27 20:20:46 -07:00
Ishaan Jaff
0368c89dd8
forward otel traceparent in request headers
2024-06-26 12:31:28 -07:00
Ishaan Jaff
2037545c32
fix - thread create endpoints
2024-06-18 07:54:47 -07:00
Ishaan Jaff
ee9e2ef980
feat - add remaining budget for key on prometheus
2024-06-13 14:37:02 -07:00
Ishaan Jaff
8d3c9aeea3
feat - add remaining team budget gauge
2024-06-13 14:28:25 -07:00
Ishaan Jaff
dbdf102a01
feat - add mistral embedding config
2024-06-12 15:00:00 -07:00
Krrish Dholakia
22b51c5af4
fix(litellm_pre_call_utils.py): add support for key level caching params
2024-06-07 22:09:14 -07:00
Ishaan Jaff
8106a6dc9b
fix simplify - pass litellm_parent_otel_span
2024-06-07 13:48:21 -07:00
Ishaan Jaff
22e653d922
fix importing litellm
2024-06-07 12:50:50 -07:00
Ishaan Jaff
5df327aca2
fix - refactor proxy server to use common func
2024-06-07 12:38:38 -07:00
Ishaan Jaff
522da0c4f8
feat - refactor /chat/completions to have a common helper
2024-06-07 12:18:53 -07:00
