Ishaan Jaff
da503eab18
Merge branch 'main' into litellm_remove_litellm_telemetry
2024-03-26 11:35:02 -07:00
Ishaan Jaff
6b4b05b58f
(fix) remove litellm.telemetry
2024-03-26 11:21:09 -07:00
Ishaan Jaff
2ecdd92619
Merge pull request #2702 from BerriAI/litellm_cache_flush
...
[Feat] Proxy - /cache/flushall - delete all elements from cache
2024-03-26 09:34:39 -07:00
Ishaan Jaff
7409dcd222
(fix) doc string
2024-03-26 09:25:44 -07:00
Ishaan Jaff
b8af946fb9
(feat) /cache/flushall
2024-03-26 09:18:58 -07:00
Ishaan Jaff
151b717ae2
(feat) support cache flush on redis
2024-03-26 09:12:30 -07:00
Antonio Molner Domenech
c713648db1
Update print statements to use verbose logger and DEBUG level
2024-03-26 22:41:28 +07:00
Krrish Dholakia
49e8cdbff9
fix(router.py): check for context window error when handling 400 status code errors
...
was causing proxy context window fallbacks to not work as expected
2024-03-26 08:08:15 -07:00
Ishaan Jaff
7134d66fae
Merge branch 'main' into litellm_batch_write_redis_cache
2024-03-25 16:41:29 -07:00
Ishaan Jaff
4ae51e9fd5
(fix) locust load test use uuid
2024-03-25 15:36:30 -07:00
Krish Dholakia
f15ba10170
Merge pull request #2687 from BerriAI/litellm_jwt_auth_fixes_2
...
Litellm jwt auth fixes
2024-03-25 13:27:19 -07:00
Krrish Dholakia
2e4e97a48f
test(test_jwt.py): add unit tests for jwt auth integration
2024-03-25 13:24:39 -07:00
Krrish Dholakia
c2ffb83c71
fix(handle_jwt.py): cache public keys
...
caches jwt public keys - reducing need for making http calls on every request
2024-03-25 12:36:32 -07:00
Krrish Dholakia
f1b1640a31
fix(handle_jwt.py): support scopes being a list
...
allow scopes in jwt to be a list, not just a space-separated string
2024-03-25 12:30:40 -07:00
Krrish Dholakia
5aa1c10929
fix(handle_jwt.py): don't require kid to be set
2024-03-25 12:28:16 -07:00
Krrish Dholakia
93959ab5aa
fix(handle_jwt.py): allow setting proxy admin role string for jwt auth
2024-03-25 12:20:14 -07:00
Krrish Dholakia
edd00af6f2
fix(handle_jwt.py): enable user to set jwt admin scope string in config
2024-03-25 11:34:49 -07:00
Ishaan Jaff
734a51c049
(fix) stop using f strings in verbose logger
2024-03-25 10:55:30 -07:00
Ishaan Jaff
5d121a9f3c
(fix) stop using f strings with logger
2024-03-25 10:47:18 -07:00
Ishaan Jaff
dad4bd58bc
(feat) stop eagerly evaluating fstring
2024-03-25 09:01:42 -07:00
Krish Dholakia
8f966cd78f
Merge pull request #2666 from BerriAI/litellm_model_fix_v_3
...
fix(proxy_server.py): fix model info check
2024-03-23 19:39:03 -07:00
Krish Dholakia
c92fa1af7c
Merge pull request #2669 from BerriAI/litellm_router_pre_call_checks
...
feat(router.py): enable pre-call checks
2024-03-23 19:38:09 -07:00
Krrish Dholakia
eb3ca85d7e
feat(router.py): enable pre-call checks
...
filter models outside of context window limits of a given message for a model group
https://github.com/BerriAI/litellm/issues/872
2024-03-23 18:03:30 -07:00
Ishaan Jaff
925e785a6a
(fix) update locustfile
2024-03-23 17:59:54 -07:00
Ishaan Jaff
0c17fe9a32
(feat) update config.yaml
2024-03-23 17:59:16 -07:00
Krrish Dholakia
c81c9c2583
fix(proxy_server.py): fix model info check
2024-03-23 15:59:17 -07:00
Ishaan Jaff
aca6ec85e2
(fix) add some better load testing
2024-03-22 19:48:54 -07:00
Ishaan Jaff
28e62af4e1
(fix) update load test used
2024-03-22 19:48:54 -07:00
Krrish Dholakia
d06b9a5a47
fix(proxy_server.py): enable jwt-auth for users
...
allow a user to auth into the proxy via jwt's and call allowed routes
2024-03-22 17:08:10 -07:00
Krrish Dholakia
9bf086386e
fix(handle_jwt.py): add more logging for jwt header
2024-03-22 16:33:32 -07:00
Krish Dholakia
8c45986e72
Merge pull request #2619 from BerriAI/litellm_install_tenacity
...
(fix) include tenacity in req.txt
2024-03-21 20:55:56 -07:00
Krrish Dholakia
d57995a73d
build(schema.prisma): use jsonProtocol to fix db connection issues
...
https://github.com/prisma/prisma/discussions/19978
2024-03-21 18:01:45 -07:00
Krrish Dholakia
4ac14a4e85
build(networking.tsx): fix trailing slash
2024-03-21 17:50:59 -07:00
Krrish Dholakia
d9577c3e2b
build(networking.tsx): modify url to prevent redirects
2024-03-21 17:43:41 -07:00
Krrish Dholakia
33964233a5
fix(proxy_server.py): allow user to disable swagger ui docs via env
...
user can disable swagger ui docs by setting 'NO_DOCS="True"' in their env
2024-03-21 17:15:18 -07:00
Krish Dholakia
33a433eb0a
Merge branch 'main' into litellm_llm_api_prompt_injection_check
2024-03-21 09:57:10 -07:00
Krrish Dholakia
0521e8a1d9
fix(prompt_injection_detection.py): fix type check
2024-03-21 08:56:13 -07:00
Krrish Dholakia
8e8c4e214e
fix: fix linting issue
2024-03-21 08:19:09 -07:00
Ishaan Jaff
14459b89e3
(fix) include tenacity in req.txt
2024-03-21 08:10:12 -07:00
Ishaan Jaff
bcd62034ed
Merge pull request #2563 from eltociear/patch-2
...
Update proxy_server.py
2024-03-21 07:29:33 -07:00
Krrish Dholakia
d91f9a9f50
feat(proxy_server.py): enable llm api based prompt injection checks
...
run user calls through an llm api to check for prompt injection attacks. This happens in parallel to th
e actual llm call using `async_moderation_hook`
2024-03-20 22:43:42 -07:00
Krish Dholakia
007d439017
Merge pull request #2606 from BerriAI/litellm_jwt_auth_updates
...
fix(handle_jwt.py): track spend for user using jwt auth
2024-03-20 19:40:17 -07:00
Krrish Dholakia
f24d3ffdb6
fix(proxy_server.py): fix import
2024-03-20 19:15:06 -07:00
Krrish Dholakia
3bb0e24cb7
fix(prompt_injection_detection.py): ensure combinations are actual phrases, not just 1-2 words
...
reduces misflagging
https://github.com/BerriAI/litellm/issues/2601
2024-03-20 19:09:38 -07:00
Krrish Dholakia
8bb00c4ae8
fix(caching.py): enable async setting of cache for dual cache
2024-03-20 18:42:34 -07:00
Krrish Dholakia
90e17b5422
fix(handle_jwt.py): track spend for user using jwt auth
2024-03-20 10:55:52 -07:00
Ishaan Jaff
3ad6e5ffc1
(feat) start proxy with default num_workers=1
2024-03-20 10:46:32 -07:00
Krrish Dholakia
ca970a90c4
fix(handle_jwt.py): remove issuer check
2024-03-20 08:35:23 -07:00
Ishaan Jaff
4ed551dc52
(feat) better debugging for /cache/ping
2024-03-20 08:30:11 -07:00
Ishaan Jaff
2256ece5a9
(feat) litellm cache ping
2024-03-20 08:24:13 -07:00
