598 commits

Author	SHA1	Message	Date
Krrish Dholakia	7610b1f0af	feat(proxy_server.py): add in-memory caching for user api keys	2023-11-23 13:21:45 -08:00
Krrish Dholakia	0e3064ac8c	fix(router.py): fix caching for tracking cooldowns + usage	2023-11-23 11:13:32 -08:00
Krrish Dholakia	2df4791ae9	fix: fix linting errors	2023-11-22 19:59:25 -08:00
Krrish Dholakia	497419a766	fix(router.py): add support for cooldowns with redis	2023-11-22 19:54:22 -08:00
Krrish Dholakia	a2207d462e	feat(router.py): add server cooldown logic	2023-11-22 15:59:48 -08:00
Krrish Dholakia	73d70ef01c	feat(router.py): adding latency-based routing strategy	2023-11-21 21:19:27 -08:00
ishaan-jaff	2b1fc64f36	(fix) using callbacks with router	2023-11-20 19:08:53 -08:00
Krrish Dholakia	2ac804a42f	feat(proxy_server.py): enable model aliases	2023-11-20 16:51:04 -08:00
Krrish Dholakia	7472be1529	fix(routing.py): update token usage on streaming	2023-11-20 14:19:25 -08:00
Krrish Dholakia	5f4dca301b	refactor(router.py): adding user support message	2023-11-18 19:05:45 -08:00
Krrish Dholakia	43c26f3382	docs(routing.md): updating docs for managing multiple deployments	2023-11-18 19:02:50 -08:00
Krrish Dholakia	cf0a9f591c	fix(router.py): introducing usage-based-routing	2023-11-17 17:56:26 -08:00
Krrish Dholakia	452946b2f8	refactor(router.py): code cleanup	2023-11-17 17:05:46 -08:00
Krrish Dholakia	7d70bf84a7	test(test_langfuse.py): handle timeouts	2023-11-17 17:05:46 -08:00
Krrish Dholakia	02ed97d0b2	fix(acompletion): support client side timeouts + raise exceptions correctly for async calls	2023-11-17 15:39:47 -08:00
Krrish Dholakia	a753487d79	fix(router.py): check if async response is coroutine	2023-11-16 21:53:35 -08:00
Krrish Dholakia	d9123ea2e8	docs(routing.md): update tutorial on deploying router	2023-11-16 21:46:43 -08:00
Krrish Dholakia	c47ca6cc50	refactor(router.py): renaming variable	2023-11-15 12:31:29 -08:00
Krrish Dholakia	4676b3dabd	feat(router.py): enable passing chat completion params for Router.chat.completion.create	2023-11-15 12:28:16 -08:00
Krrish Dholakia	0f6713993d	fix(router.py): enabling retrying with expo backoff (without tenacity) for router	2023-11-14 20:57:51 -08:00
Krrish Dholakia	5d58bb9cd0	fix(main.py): misrouting ollama models to nlp cloud	2023-11-14 18:55:08 -08:00
Krrish Dholakia	1032b49f23	bump: version 1.0.0.dev1 → 1.0.0	2023-11-13 16:48:53 -08:00
Krrish Dholakia	4e233c915b	fix(router.py): enable calling router with instructor	2023-11-13 15:16:57 -08:00
Krrish Dholakia	67e8b12a09	fix(utils.py): fix cached responses - translate dict to objects	2023-11-10 10:38:20 -08:00
Nathan Kim	ada2d40a43	first commit	2023-11-09 22:12:39 -08:00
Nathan Kim	672cd09e3f	first commit	2023-11-09 22:07:19 -08:00
Krrish Dholakia	78fb8cf941	fix(router.py): fix linting issues	2023-11-06 18:50:09 -08:00
Krrish Dholakia	a7a0605b4f	fix(router.py): adding health checks	2023-11-06 18:26:41 -08:00
Krrish Dholakia	632533f2e2	bump: version 0.13.6.dev3 → 0.13.6	2023-11-06 18:19:20 -08:00
Krish Dholakia	a157a3da8c	Merge pull request #722 from karvetskiy/fix-router-caching ... Fix caching for Router	2023-10-31 16:39:18 -07:00
mc-marcocheng	c7b6911c7b	Handle empty input edge case	2023-10-31 14:38:04 +08:00
seva	f0a9f8c61e	Router & Caching fixes: ... - Add optional TTL to Cache parameters - Fix tpm and rpm caching in Router	2023-10-30 13:29:35 +01:00
mc-marcocheng	f74c0f101f	avoid overwriting litellm_params	2023-10-27 15:30:34 +08:00
mc-marcocheng	bd43b96d8d	Feature: Router aembedding	2023-10-25 11:19:45 +08:00
Krrish Dholakia	3477486886	fix(router.py): adding support for async completion calls ... https://github.com/BerriAI/litellm/issues/676	2023-10-24 17:20:31 -07:00
Krrish Dholakia	e6e9a595f8	refactor(router.py): spacing	2023-10-24 12:30:59 -07:00
Krrish Dholakia	98c25b08cd	fix(vertex_ai.py): fix output parsing	2023-10-24 12:08:22 -07:00
Krrish Dholakia	c7e8bf3e80	test(test_router.py): fixing router testing	2023-10-24 10:21:10 -07:00
mc-marcocheng	fc3c32dde5	Fix data being overwritten	2023-10-24 16:02:15 +08:00
Krrish Dholakia	0aad58cf48	fix(router.py): enable additional params to be passe din	2023-10-23 20:41:18 -07:00
Krrish Dholakia	eb40ba8f2e	docs(router.py): fix linting issue	2023-10-23 15:15:10 -07:00
Krrish Dholakia	cc760d783e	fix(router.py): fix null model names error ... https://github.com/BerriAI/litellm/issues/669#issuecomment-1775456245	2023-10-23 09:30:04 -07:00
ishaan-jaff	9347b25e37	(fix) litellm proxy with router	2023-10-20 09:38:22 -07:00
Krrish Dholakia	494b9bbc62	test: fix linting testing	2023-10-19 18:40:33 -07:00
Krrish Dholakia	0c083e7a5c	feat(proxy_server.py): add model router to proxy	2023-10-18 17:40:08 -07:00
Krrish Dholakia	204218508d	fix(router.py): completing redis support work for router	2023-10-18 12:13:00 -07:00
Krrish Dholakia	168b5a181c	fix(test_proxy.py): fix tests	2023-10-17 22:34:12 -07:00
Krrish Dholakia	f23431fdc4	feat(router.py): add routing capabilities for multiple deployments	2023-10-17 22:28:31 -07:00