Krish Dholakia
234185ec13
LiteLLM Minor Fixes & Improvements (09/16/2024) ( #5723 ) ( #5731 )
...
* LiteLLM Minor Fixes & Improvements (09/16/2024) (#5723 )
* coverage (#5713 )
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* Move (#5714 )
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix(litellm_logging.py): fix logging client re-init (#5710 )
Fixes https://github.com/BerriAI/litellm/issues/5695
* fix(presidio.py): Fix logging_hook response and add support for additional presidio variables in guardrails config
Fixes https://github.com/BerriAI/litellm/issues/5682
* feat(o1_handler.py): fake streaming for openai o1 models
Fixes https://github.com/BerriAI/litellm/issues/5694
* docs: deprecated traceloop integration in favor of native otel (#5249 )
* fix: fix linting errors
* fix: fix linting errors
* fix(main.py): fix o1 import
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view (#5730 )
* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view
Supports having `MonthlyGlobalSpend` view be a material view, and exposes an endpoint to refresh it
* fix(custom_logger.py): reset calltype
* fix: fix linting errors
* fix: fix linting error
* fix: fix import
* test(test_databricks.py): fix databricks tests
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
2024-09-17 08:05:52 -07:00
Ishaan Jaff
9f5a33015f
fix linting
2024-09-16 18:07:48 -07:00
Ishaan Jaff
b6ae2204a8
[Feat-Proxy] Slack Alerting - allow using os.environ/ vars for alert to webhook url ( #5726 )
...
* allow using os.environ for slack urls
* use env vars for webhook urls
* fix types for get_secret
* fix linting
* fix linting
* fix linting
* linting fixes
* linting fix
* docs alerting slack
* fix get data
2024-09-16 18:03:37 -07:00
Krish Dholakia
d94d47424f
fix(proxy/utils.py): auto-update if required view missing from db. raise warning for optional views. ( #5675 )
...
Prevents missing optional views from blocking proxy startup.
2024-09-12 22:15:44 -07:00
Ishaan Jaff
e7c9716841
[Feat-Perf] Use Batching + Squashing ( #5645 )
...
* use folder for slack alerting
* clean up slack alerting
* fix test alerting
2024-09-12 18:37:53 -07:00
Krish Dholakia
c76d2c6ade
Refactor 'check_view_exists' logic ( #5659 )
...
* fix(proxy/utils.py): comment out auto-upsert logic in check_view_exists
Prevents proxy from failing on startup due to faulty logic
* fix(db/migration_scripts/create_views.py): fix 'DailyTagSpend' quotation on check
* fix(create_views.py): mongly global spend time period should be 30d not 20d
* fix(schema.prisma): index on startTime and endUser for efficient UI querying
2024-09-12 13:39:50 -07:00
Ishaan Jaff
67751d0ecc
fix use view for getting tag usage
2024-09-06 16:28:24 -07:00
Ishaan Jaff
e0400accca
fix create view - MonthlyGlobalSpendPerUserPerKey
2024-09-05 12:11:59 -07:00
Ishaan Jaff
fd122cb759
fix always read redis
2024-09-02 21:08:32 -07:00
Krrish Dholakia
65097d595c
fix(proxy/utils.py): fix model dump to exclude none values
2024-08-28 12:02:44 -07:00
Ishaan Jaff
fb5be57bb8
v0 add rerank on litellm proxy
2024-08-27 17:28:39 -07:00
Ishaan Jaff
a99258440c
fix use guardrail for pre call hook
2024-08-23 09:34:08 -07:00
Ishaan Jaff
af92cff44d
add custom guardrail reference
2024-08-23 08:32:07 -07:00
Krrish Dholakia
fa6c9bf42e
feat(user_api_key_auth.py): allow team admin to add new members to team
2024-08-20 14:01:12 -07:00
Ishaan Jaff
a0361b0e76
run during_call_hook
2024-08-19 12:07:46 -07:00
Ishaan Jaff
4685b9909a
feat - allow accessing data post success call
2024-08-19 11:35:33 -07:00
Ishaan Jaff
ffb7f9f280
add fallback_reports as slack alert
2024-08-10 15:26:32 -07:00
Ishaan Jaff
764b1ed869
Merge pull request #5058 from BerriAI/litellm_add_debug_statements_connecting_prisma
...
Fix - add debug statements when connecting to prisma DB
2024-08-05 20:53:55 -07:00
Ishaan Jaff
da848696e3
Merge pull request #5059 from BerriAI/litelm_log_otel_args
...
OTEL - Log DB queries / functions on OTEL
2024-08-05 20:51:11 -07:00
Ishaan Jaff
bec0350127
otel fix async_service_failure_hook
2024-08-05 20:18:40 -07:00
Ishaan Jaff
8dcea009b8
log event_metadata on otel service loggers
2024-08-05 19:58:55 -07:00
Krrish Dholakia
5c6dcd6029
test: improve debugging for test
2024-08-05 19:41:08 -07:00
Ishaan Jaff
a5cd041ccf
add debug statements when connected to prisma db
2024-08-05 19:13:46 -07:00
Ishaan Jaff
19fb5cc11c
use common helpers for writing to otel
2024-07-27 11:40:39 -07:00
Ishaan Jaff
d5d9ed73af
use _get_parent_otel_span_from_kwargs
2024-07-27 11:14:06 -07:00
Krrish Dholakia
6ab2527fdc
feat(auth_check.py): support using redis cache for team objects
...
Allows team update / check logic to work across instances instantly
2024-07-24 18:14:49 -07:00
Marc Abramowitz
642f1a7bcc
Check existence of multiple views in 1 query
...
instead of multiple queries. This is more efficient because it lets us check for
all views in one query instead of multiple queries.
2024-07-23 21:55:02 -07:00
Ishaan Jaff
b137207ae6
doc alert_to_webhook_url
2024-07-23 11:04:15 -07:00
Ishaan Jaff
c1593c0cd1
update alert_to_webhook_url
2024-07-23 10:07:08 -07:00
Krish Dholakia
c4db6aa15e
Merge pull request #4810 from BerriAI/litellm_team_modify_guardrails
...
feat(auth_checks.py): Allow admin to disable team from turning on/off guardrails
2024-07-22 22:32:24 -07:00
Krrish Dholakia
1a83935aa4
fix(proxy/utils.py): add stronger typing for litellm params in failure call logging
2024-07-22 21:31:39 -07:00
Ishaan Jaff
c9e2f977dd
feat - return team_metadata in user_api_key_auth
2024-07-22 18:21:50 -07:00
Krrish Dholakia
8b3c8102a7
feat(auth_checks.py): Allow admin to disable team from turning on/off guardrails.
2024-07-20 18:39:05 -07:00
Krrish Dholakia
99aa311083
fix(user_api_key_auth.py): update team values in token cache if refreshed more recently
2024-07-19 17:35:59 -07:00
Ishaan Jaff
df1ac92222
fix add fix to update spend logs
2024-07-19 12:49:23 -07:00
Ishaan Jaff
3736152e7d
fix calculate correct alerting threshold
2024-07-16 21:43:17 -07:00
Ishaan Jaff
6c918f2373
fix tracking hanging requests
2024-07-16 21:43:16 -07:00
Ishaan Jaff
36be9967d1
fix storing request status in mem
2024-07-16 21:43:16 -07:00
Krrish Dholakia
ec03e675c9
fix(proxy/utils.py): fix failure logging for rejected requests. + unit tests
2024-07-16 17:15:20 -07:00
Ishaan Jaff
69f74c1e6c
fix only use per request tracking if slack alerting is being used
2024-07-13 18:01:53 -07:00
Krrish Dholakia
0cc273d77b
feat(pass_through_endpoint.py): support enforcing key rpm limits on pass through endpoints
...
Closes https://github.com/BerriAI/litellm/issues/4698
2024-07-13 13:29:44 -07:00
Ishaan Jaff
92bf98b30f
Merge pull request #4684 from BerriAI/litellm_safe_memory_mode
...
[Feat] Allow safe memory mode
2024-07-12 18:32:16 -07:00
Ishaan Jaff
08efef5316
feat add safe_memory_mode
2024-07-12 18:18:39 -07:00
Ishaan Jaff
8c8dcdbdb1
reduce ttil for update_request_status
2024-07-12 15:14:54 -07:00
Ishaan Jaff
3a06e2e425
fix show exact prisma exception when starting proxy
2024-07-09 18:20:09 -07:00
Krrish Dholakia
20e39d6acc
fix(utils.py): cleanup 'additionalProperties=False' for tool calling with zod
...
Fixes issue with zod passing in additionalProperties=False, causing vertex ai / gemini calls to fail
2024-07-06 17:27:37 -07:00
Ishaan Jaff
d61cc598b0
Merge pull request #4576 from BerriAI/litellm_encrypt_decrypt_using_salt
...
[Refactor] Use helper function to encrypt/decrypt model credentials
2024-07-06 15:11:09 -07:00
Ishaan Jaff
7f6f85ec3c
improve sign up flow - show missing env vars
2024-07-06 13:57:19 -07:00
Krrish Dholakia
71edb28d6a
fix(proxy_server.py): fix embedding model exception mapping
2024-07-06 11:14:41 -07:00
Ishaan Jaff
8f0b55879f
move encrypt / decrypt to helper
2024-07-06 11:09:47 -07:00
