3256 commits

Author	SHA1	Message	Date
Krrish Dholakia	0f7d03f761	fix(proxy/rules.md): add docs on setting post-call rules on the proxy	2024-01-04 11:16:50 +05:30
ishaan-jaff	54653f9a4a	(test) proxy + s3 caching	2024-01-04 11:11:08 +05:30
ishaan-jaff	aa757d19f5	(test) router - init clients - azure cloudflare, openai etc	2024-01-04 10:55:18 +05:30
ishaan-jaff	0864713b62	(test) cf azure	2024-01-04 10:26:41 +05:30
ishaan-jaff	8e10a1eb81	(docs) config with cloudflare exampel	2024-01-04 10:25:35 +05:30
ishaan-jaff	6d21ee3a2f	(fix) proxy - cloudflare + Azure bug [non-streaming]	2024-01-04 10:24:51 +05:30
Dustin Miller	b10f64face	Adds "pre-fill" support for Claude	2024-01-03 18:45:36 -06:00
David Manouchehri	c54e0813b4	(caching) improve s3 backend by specifying cache-control and content-type	2024-01-03 13:44:28 -05:00
ishaan-jaff	30c6d164d2	(docs) use s3 Cache on litellm proxy	2024-01-03 17:56:44 +05:30
Krrish Dholakia	469ae0a378	fix(proxy/utils.py): don't keep connecting to db if connection already established	2024-01-03 17:43:44 +05:30
Krrish Dholakia	f2da345173	fix(caching.py): handle cached_response being a dict not json string	2024-01-03 17:29:27 +05:30
Krrish Dholakia	f2210787cd	feat(proxy_server.py): allow admins to update config via /config/update endpoint	2024-01-03 17:18:33 +05:30
ishaan-jaff	4680a26e2e	(test) proxy - load test	2024-01-03 16:16:18 +05:30
Krrish Dholakia	d45101b652	fix(proxy_server.py): fix master key reset, to preserve key from env	2024-01-03 16:10:10 +05:30
Krrish Dholakia	40c974999e	fix(proxy_server.py): reject bad /model/new POST requests	2024-01-03 15:54:58 +05:30
ishaan-jaff	d14a41863f	(test) s3 cache with setting s3_bucket_name	2024-01-03 15:42:23 +05:30
ishaan-jaff	58ce5d44ae	(feat) s3 cache support all boto3 params	2024-01-03 15:42:23 +05:30
Krrish Dholakia	b51371952b	fix(proxy_server.py): handle base case for /model/info	2024-01-03 15:33:29 +05:30
ishaan-jaff	fea0a933ae	(test) use s3 buckets cache	2024-01-03 15:13:43 +05:30
ishaan-jaff	00364da993	(feat) add s3 Bucket as Cache	2024-01-03 15:13:43 +05:30
Krrish Dholakia	14e501845f	fix(proxy_server.py): add support for setting master key via .env	2024-01-03 15:10:25 +05:30
Krrish Dholakia	ef8f1acfa4	refactor(proxy_server.py): more debug statements	2024-01-03 13:59:41 +05:30
Krrish Dholakia	8cee267a5b	fix(caching.py): support ttl, s-max-age, and no-cache cache controls ... https://github.com/BerriAI/litellm/issues/1306	2024-01-03 12:42:43 +05:30
ishaan-jaff	2bea0c742e	(test) completion tokens counting + azure stream	2024-01-03 12:06:39 +05:30
ishaan-jaff	96cb6f3b10	(fix) azure+stream: count completion tokens	2024-01-03 12:06:39 +05:30
ishaan-jaff	f3b8d9c3ef	(fix) counting response tokens+streaming	2024-01-03 12:06:39 +05:30
Krrish Dholakia	cd98d256b5	fix(proxy_server.py): add alerting for responses taking too long ... https://github.com/BerriAI/litellm/issues/1298	2024-01-03 11:18:21 +05:30
Krrish Dholakia	0d13c51615	fix(proxy/utils.py): fix self.alerting null case ... https://github.com/BerriAI/litellm/issues/1298#issuecomment-1874798056	2024-01-03 10:12:21 +05:30
Krrish Dholakia	a778f8a00e	bump: version 1.16.10 → 1.16.11	2024-01-02 22:26:47 +05:30
Krrish Dholakia	070520d237	fix(proxy_server.py): support smtp email auth ... previously was a hard resend package dependency. removed in favor of allowing any smtp server connection (including resend)	2024-01-02 22:22:19 +05:30
Ased Mammad	c39c8f70eb	fix(proxy_server.py) Check when '_hidden_params' is None	2024-01-02 19:04:51 +03:30
Krrish Dholakia	940569703e	feat(proxy_server.py): add slack alerting to proxy server ... add alerting for calls hanging, failing and db read/writes failing https://github.com/BerriAI/litellm/issues/1298	2024-01-02 17:45:18 +05:30
ishaan-jaff	c27b1fc5f8	(feat) proxy swagger - make admin link clickable	2024-01-02 17:04:32 +05:30
ishaan-jaff	14738ec89d	(test) xinference on litellm router	2024-01-02 16:51:08 +05:30
ishaan-jaff	bfbed2d93d	(test) xinference embeddings	2024-01-02 15:41:51 +05:30
ishaan-jaff	790dcff5e0	(feat) add xinference as an embedding provider	2024-01-02 15:32:26 +05:30
Krrish Dholakia	0fffcc1579	fix(utils.py): support token counting for gpt-4-vision models	2024-01-02 14:41:42 +05:30
ishaan-jaff	bfae0fe935	(test) proxy - pass user_config	2024-01-02 14:15:03 +05:30
ishaan-jaff	075eb1a516	(types) routerConfig	2024-01-02 14:14:29 +05:30
ishaan-jaff	9afdc8b4ee	(feat) add Router init Pydantic Type	2024-01-02 13:30:24 +05:30
ishaan-jaff	1f8fc6d2a7	(feat) litellm add types for completion, embedding request	2024-01-02 12:27:08 +05:30
ishaan-jaff	6d2b9fd470	(feat) use - user router for aembedding	2024-01-02 12:27:08 +05:30
Krrish Dholakia	2ab31bcaf8	fix(lowest_tpm_rpm.py): handle null case for text/message input	2024-01-02 12:24:29 +05:30
ishaan-jaff	0acaaf8f8f	(test) sustained load test proxy	2024-01-02 12:10:34 +05:30
ishaan-jaff	31a896908b	(test) proxy - use, user provided model_list	2024-01-02 12:10:34 +05:30
ishaan-jaff	ddc31c4810	(feat) proxy - use user_config for /chat/compeltions	2024-01-02 12:10:34 +05:30
Krrish Dholakia	a37a18ca80	feat(router.py): add support for retry/fallbacks for async embedding calls	2024-01-02 11:54:28 +05:30
Krrish Dholakia	c12e3bd565	fix(router.py): fix model name passed through	2024-01-02 11:15:30 +05:30
Krrish Dholakia	dff4c172d0	refactor(test_router_caching.py): move tpm/rpm routing tests to separate file	2024-01-02 11:10:11 +05:30
ishaan-jaff	18ef244230	(test) bedrock-test passing boto3 client	2024-01-02 10:23:28 +05:30