455 commits

Author	SHA1	Message	Date
Krrish Dholakia	d7601a4844	perf(proxy_server.py): batch write spend logs ... reduces prisma client errors, by batch writing spend logs - max 1k logs at a time	2024-04-02 18:46:55 -07:00
Krrish Dholakia	6467dd4e11	fix(tpm_rpm_limiter.py): fix cache init logic	2024-04-01 18:01:38 -07:00
Krrish Dholakia	9c0aecf9b8	fix(proxy/utils.py): support redis caching for alerting	2024-04-01 16:13:59 -07:00
Krrish Dholakia	3b8e7241b4	fix(proxy/utils.py): uncomment max parallel request limit check	2024-03-30 20:51:59 -07:00
Krrish Dholakia	d9ff13b624	fix(utils.py): set redis_usage_cache to none by default	2024-03-30 20:10:56 -07:00
Krrish Dholakia	f58fefd589	fix(tpm_rpm_limiter.py): enable redis caching for tpm/rpm checks on keys/user/teams ... allows tpm/rpm checks to work across instances https://github.com/BerriAI/litellm/issues/2730	2024-03-30 20:01:36 -07:00
Krrish Dholakia	5280fc809f	fix(proxy_server.py): enforce end user budgets with 'litellm.max_end_user_budget' param	2024-03-29 17:14:40 -07:00
Krrish Dholakia	c15ba368e7	fix(proxy_server.py): enable spend tracking for team-based jwt auth	2024-03-28 20:16:22 -07:00
Krrish Dholakia	7c44b32cc2	refactor(proxy/utils.py): add more debug logs	2024-03-28 18:44:35 -07:00
Krish Dholakia	934a9ac2b4	Merge pull request #2722 from BerriAI/litellm_db_perf_improvement ... feat(proxy/utils.py): enable updating db in a separate server	2024-03-28 14:56:14 -07:00
Krrish Dholakia	e8d80509b1	test(test_update_spend.py): allow db_client to be none	2024-03-28 13:44:40 -07:00
Krrish Dholakia	082f1e4085	fix(proxy_server.py): allow user to pass in spend logs collector url	2024-03-28 09:14:30 -07:00
Ishaan Jaff	c3f78af2c6	Merge pull request #2728 from BerriAI/litellm_reduce_deep_copies ... [FEAT] Proxy - reduce deep copies	2024-03-27 21:26:09 -07:00
Ishaan Jaff	f2e1d938f3	(fix) remove deep copy from all responses	2024-03-27 20:36:53 -07:00
Krrish Dholakia	2926d5a8eb	fix(proxy/utils.py): check cache before alerting user	2024-03-27 20:09:15 -07:00
Krrish Dholakia	4eb93832e4	feat(auth_checks.py): enable admin to enforce 'user' param for all openai endpoints	2024-03-27 17:36:27 -07:00
Krrish Dholakia	1e856443e1	feat(proxy/utils.py): enable updating db in a separate server	2024-03-27 16:02:36 -07:00
Krrish Dholakia	e10eb8f6fe	feat(llm_guard.py): enable key-specific llm guard check	2024-03-26 17:21:51 -07:00
Ishaan Jaff	5d121a9f3c	(fix) stop using f strings with logger	2024-03-25 10:47:18 -07:00
Ishaan Jaff	dad4bd58bc	(feat) stop eagerly evaluating fstring	2024-03-25 09:01:42 -07:00
Krrish Dholakia	d91f9a9f50	feat(proxy_server.py): enable llm api based prompt injection checks ... run user calls through an llm api to check for prompt injection attacks. This happens in parallel to th e actual llm call using `async_moderation_hook`	2024-03-20 22:43:42 -07:00
Krrish Dholakia	2dfdc8dd69	Revert "Merge pull request #2593 from BerriAI/litellm_reset_budget_fix" ... This reverts commit afd363129f, reversing changes made to c94bc94ad5.	2024-03-19 20:25:41 -07:00
Krrish Dholakia	3a866dd349	bump: version 1.32.5 → 1.32.6	2024-03-19 20:17:50 -07:00
Krrish Dholakia	f6de3a0359	fix: better debug logs	2024-03-19 19:28:26 -07:00
Krrish Dholakia	e0d3b18835	fix(proxy/utils.py): fix reset budget logic ... uses fewer clients - prevents read timeouts	2024-03-19 18:02:09 -07:00
Krrish Dholakia	8c91156842	build: build fixes	2024-03-19 16:59:59 -07:00
Krrish Dholakia	c03694ae1e	fix(proxy/utils.py): cleanup logging	2024-03-19 09:29:29 -07:00
Krrish Dholakia	8fefe625d9	fix(proxy/utils.py): batch writing updates to db	2024-03-18 16:47:02 -07:00
Krrish Dholakia	077b9c6234	fix(proxy/utils.py): move to batch writing db updates	2024-03-16 22:32:00 -07:00
Krrish Dholakia	1b807fa3f5	fix(proxy_server.py): fix key caching logic	2024-03-13 19:10:24 -07:00
Krrish Dholakia	fd4086d5da	fix(proxy_server.py): fix /user/info for non-existent user id	2024-03-12 11:05:41 -07:00
Krrish Dholakia	2addd66393	fix(proxy_server.py): bug fix	2024-03-11 13:54:58 -07:00
Krrish Dholakia	4eb244c3ca	fix(proxy_server.py): prevent user from deleting non-user owned keys when they use ui	2024-03-11 12:13:30 -07:00
Krrish Dholakia	fa45c569fd	feat: add cost tracking + caching for transcription calls	2024-03-09 15:43:38 -08:00
Krrish Dholakia	4b60bea975	fix(proxy/utils.py): add more logging for prisma client get_data error	2024-03-09 13:06:30 -08:00
ishaan-jaff	8c3ec15856	(fix) improve improve prisma alerting/debug	2024-03-07 15:44:03 -08:00
ishaan-jaff	5217e800c4	(fix) better prisma debug logs	2024-03-07 14:04:02 -08:00
ishaan-jaff	b64e4c3e6f	(feat) better debugging when reading prisma	2024-03-07 13:42:54 -08:00
Krrish Dholakia	d1d8adfb11	fix(proxy_server.py): fix sql query	2024-03-06 19:41:12 -08:00
Krrish Dholakia	ca97ea8acd	feat(proxy_server.py): team based model aliases ... allow setting model aliases at a team level (e.g. route all 'gpt-3.5-turbo' requests from team-1 to model-deployment-group-2)	2024-03-06 17:42:08 -08:00
ishaan-jaff	ee468a4e05	(feat) circular ref error on prisa	2024-03-06 12:08:22 -08:00
ishaan-jaff	c17789f170	(fix) circular ref error h	2024-03-06 12:02:44 -08:00
ishaan-jaff	2313f5f90a	(feat) handle litellm circular ref error	2024-03-06 10:21:25 -08:00
ishaan-jaff	52e59e050c	(fix) use str(e)	2024-03-05 18:57:26 -08:00
ishaan-jaff	00666566c2	(fix) chore	2024-03-05 18:54:04 -08:00
Ishaan Jaff	03fa322b38	Merge branch 'main' into litellm_fixes_proxy_db	2024-03-05 18:51:55 -08:00
ishaan-jaff	735728c4dd	(feat) add better debugging when DB read/write fails	2024-03-05 18:16:59 -08:00
Krrish Dholakia	387864662e	fix(main.py): trigger new build	2024-03-05 15:50:40 -08:00
ishaan-jaff	b706598415	(v0) fixes	2024-03-05 15:27:06 -08:00
Krrish Dholakia	db0489d603	fix(proxy/utils.py): add linting fix	2024-03-02 21:11:25 -08:00