65 commits

Author	SHA1	Message	Date
Krrish Dholakia	67fe8824b3	fix(router.py): check for fallbacks in completion params for router	2023-11-25 18:46:45 -08:00
Krrish Dholakia	8884ceb606	fix(proxy_server.py): expose a /health endpoint	2023-11-25 18:28:47 -08:00
Krrish Dholakia	2b9c5bf706	fix(router.py): check if fallbacks is none	2023-11-25 14:58:07 -08:00
Krrish Dholakia	b1b2d0c2b7	fix(utils.py): fix bedrock + cohere calls	2023-11-25 14:45:42 -08:00
Krrish Dholakia	e421642ba8	fix: fix linting issues	2023-11-24 15:46:25 -08:00
Krrish Dholakia	68168cc743	fix(router.py): fix retry logic	2023-11-24 13:27:44 -08:00
Krrish Dholakia	9618718080	test: refactor testing order	2023-11-24 12:47:28 -08:00
Krrish Dholakia	27fd144950	docs(simple_proxy.md): add tutorial for doing fallbacks + retries + timeouts on the proxy	2023-11-24 12:20:38 -08:00
Krrish Dholakia	9185d57f6d	fix(router.py): fixing embedding call	2023-11-23 21:07:02 -08:00
Krrish Dholakia	f3bef86848	fix(router.py): use an older version of async for compatibility	2023-11-23 21:00:53 -08:00
Krrish Dholakia	3a8d7ec835	fix(router.py): add modelgroup to call metadata	2023-11-23 20:55:49 -08:00
Krrish Dholakia	1b26a0931e	fix(utils.py): make failure logging sync	2023-11-23 20:19:27 -08:00
Krrish Dholakia	ae5d674c15	fix(router.py): fix linting errors	2023-11-23 16:50:19 -08:00
Krrish Dholakia	e4deb09eb6	fix(router.py): add support for context window fallbacks on router	2023-11-23 16:43:02 -08:00
ishaan-jaff	616b876f23	(fix) router	2023-11-23 16:28:19 -08:00
Krrish Dholakia	7f632e6e2f	fix(router.py): enable async completions with model fallbacks	2023-11-23 16:15:57 -08:00
Krrish Dholakia	59d084342d	fix(router.py): enable fallbacks for sync completions	2023-11-23 16:06:46 -08:00
Krrish Dholakia	7610b1f0af	feat(proxy_server.py): add in-memory caching for user api keys	2023-11-23 13:21:45 -08:00
Krrish Dholakia	0e3064ac8c	fix(router.py): fix caching for tracking cooldowns + usage	2023-11-23 11:13:32 -08:00
Krrish Dholakia	2df4791ae9	fix: fix linting errors	2023-11-22 19:59:25 -08:00
Krrish Dholakia	497419a766	fix(router.py): add support for cooldowns with redis	2023-11-22 19:54:22 -08:00
Krrish Dholakia	a2207d462e	feat(router.py): add server cooldown logic	2023-11-22 15:59:48 -08:00
Krrish Dholakia	73d70ef01c	feat(router.py): adding latency-based routing strategy	2023-11-21 21:19:27 -08:00
ishaan-jaff	2b1fc64f36	(fix) using callbacks with router	2023-11-20 19:08:53 -08:00
Krrish Dholakia	2ac804a42f	feat(proxy_server.py): enable model aliases	2023-11-20 16:51:04 -08:00
Krrish Dholakia	7472be1529	fix(routing.py): update token usage on streaming	2023-11-20 14:19:25 -08:00
Krrish Dholakia	5f4dca301b	refactor(router.py): adding user support message	2023-11-18 19:05:45 -08:00
Krrish Dholakia	43c26f3382	docs(routing.md): updating docs for managing multiple deployments	2023-11-18 19:02:50 -08:00
Krrish Dholakia	cf0a9f591c	fix(router.py): introducing usage-based-routing	2023-11-17 17:56:26 -08:00
Krrish Dholakia	452946b2f8	refactor(router.py): code cleanup	2023-11-17 17:05:46 -08:00
Krrish Dholakia	7d70bf84a7	test(test_langfuse.py): handle timeouts	2023-11-17 17:05:46 -08:00
Krrish Dholakia	02ed97d0b2	fix(acompletion): support client side timeouts + raise exceptions correctly for async calls	2023-11-17 15:39:47 -08:00
Krrish Dholakia	a753487d79	fix(router.py): check if async response is coroutine	2023-11-16 21:53:35 -08:00
Krrish Dholakia	d9123ea2e8	docs(routing.md): update tutorial on deploying router	2023-11-16 21:46:43 -08:00
Krrish Dholakia	c47ca6cc50	refactor(router.py): renaming variable	2023-11-15 12:31:29 -08:00
Krrish Dholakia	4676b3dabd	feat(router.py): enable passing chat completion params for Router.chat.completion.create	2023-11-15 12:28:16 -08:00
Krrish Dholakia	0f6713993d	fix(router.py): enabling retrying with expo backoff (without tenacity) for router	2023-11-14 20:57:51 -08:00
Krrish Dholakia	5d58bb9cd0	fix(main.py): misrouting ollama models to nlp cloud	2023-11-14 18:55:08 -08:00
Krrish Dholakia	1032b49f23	bump: version 1.0.0.dev1 → 1.0.0	2023-11-13 16:48:53 -08:00
Krrish Dholakia	4e233c915b	fix(router.py): enable calling router with instructor	2023-11-13 15:16:57 -08:00
Krrish Dholakia	67e8b12a09	fix(utils.py): fix cached responses - translate dict to objects	2023-11-10 10:38:20 -08:00
Nathan Kim	ada2d40a43	first commit	2023-11-09 22:12:39 -08:00
Nathan Kim	672cd09e3f	first commit	2023-11-09 22:07:19 -08:00
Krrish Dholakia	78fb8cf941	fix(router.py): fix linting issues	2023-11-06 18:50:09 -08:00
Krrish Dholakia	a7a0605b4f	fix(router.py): adding health checks	2023-11-06 18:26:41 -08:00
Krrish Dholakia	632533f2e2	bump: version 0.13.6.dev3 → 0.13.6	2023-11-06 18:19:20 -08:00
Krish Dholakia	a157a3da8c	Merge pull request #722 from karvetskiy/fix-router-caching ... Fix caching for Router	2023-10-31 16:39:18 -07:00
mc-marcocheng	c7b6911c7b	Handle empty input edge case	2023-10-31 14:38:04 +08:00
seva	f0a9f8c61e	Router & Caching fixes: ... - Add optional TTL to Cache parameters - Fix tpm and rpm caching in Router	2023-10-30 13:29:35 +01:00
mc-marcocheng	f74c0f101f	avoid overwriting litellm_params	2023-10-27 15:30:34 +08:00