66 commits

Author	SHA1	Message	Date
Krrish Dholakia	59ba1560e5	fix(router.py): fix fallbacks	2023-11-25 19:34:20 -08:00
Krrish Dholakia	fa713abfc3	fix(router.py): check for fallbacks in completion params for router	2023-11-25 18:46:45 -08:00
Krrish Dholakia	e4f302a8e2	fix(proxy_server.py): expose a /health endpoint	2023-11-25 18:28:47 -08:00
Krrish Dholakia	ab0bc87427	fix(router.py): check if fallbacks is none	2023-11-25 14:58:07 -08:00
Krrish Dholakia	95579fda7d	fix(utils.py): fix bedrock + cohere calls	2023-11-25 14:45:42 -08:00
Krrish Dholakia	d62da29cbe	fix: fix linting issues	2023-11-24 15:46:25 -08:00
Krrish Dholakia	2686894823	fix(router.py): fix retry logic	2023-11-24 13:27:44 -08:00
Krrish Dholakia	16e1070dbe	test: refactor testing order	2023-11-24 12:47:28 -08:00
Krrish Dholakia	12dbdc4c15	docs(simple_proxy.md): add tutorial for doing fallbacks + retries + timeouts on the proxy	2023-11-24 12:20:38 -08:00
Krrish Dholakia	5c18771f9d	fix(router.py): fixing embedding call	2023-11-23 21:07:02 -08:00
Krrish Dholakia	02464f6661	fix(router.py): use an older version of async for compatibility	2023-11-23 21:00:53 -08:00
Krrish Dholakia	187403c5cc	fix(router.py): add modelgroup to call metadata	2023-11-23 20:55:49 -08:00
Krrish Dholakia	7d221fe863	fix(utils.py): make failure logging sync	2023-11-23 20:19:27 -08:00
Krrish Dholakia	dc17f63d0b	fix(router.py): fix linting errors	2023-11-23 16:50:19 -08:00
Krrish Dholakia	c273d6f0d6	fix(router.py): add support for context window fallbacks on router	2023-11-23 16:43:02 -08:00
ishaan-jaff	f01865e960	(fix) router	2023-11-23 16:28:19 -08:00
Krrish Dholakia	afac42e93a	fix(router.py): enable async completions with model fallbacks	2023-11-23 16:15:57 -08:00
Krrish Dholakia	8ac03e492f	fix(router.py): enable fallbacks for sync completions	2023-11-23 16:06:46 -08:00
Krrish Dholakia	8c4e8d6c62	feat(proxy_server.py): add in-memory caching for user api keys	2023-11-23 13:21:45 -08:00
Krrish Dholakia	61fc76a8c4	fix(router.py): fix caching for tracking cooldowns + usage	2023-11-23 11:13:32 -08:00
Krrish Dholakia	2f93c0155a	fix: fix linting errors	2023-11-22 19:59:25 -08:00
Krrish Dholakia	5d5ca9f7ef	fix(router.py): add support for cooldowns with redis	2023-11-22 19:54:22 -08:00
Krrish Dholakia	3e76d4b422	feat(router.py): add server cooldown logic	2023-11-22 15:59:48 -08:00
Krrish Dholakia	76f46902ed	feat(router.py): adding latency-based routing strategy	2023-11-21 21:19:27 -08:00
ishaan-jaff	80884e9cb3	(fix) using callbacks with router	2023-11-20 19:08:53 -08:00
Krrish Dholakia	33e47dae8e	feat(proxy_server.py): enable model aliases	2023-11-20 16:51:04 -08:00
Krrish Dholakia	1976d0f7d6	fix(routing.py): update token usage on streaming	2023-11-20 14:19:25 -08:00
Krrish Dholakia	33422112d0	refactor(router.py): adding user support message	2023-11-18 19:05:45 -08:00
Krrish Dholakia	9e88014ff4	docs(routing.md): updating docs for managing multiple deployments	2023-11-18 19:02:50 -08:00
Krrish Dholakia	478bd7def6	fix(router.py): introducing usage-based-routing	2023-11-17 17:56:26 -08:00
Krrish Dholakia	7376e57e9c	refactor(router.py): code cleanup	2023-11-17 17:05:46 -08:00
Krrish Dholakia	5cddab9e54	test(test_langfuse.py): handle timeouts	2023-11-17 17:05:46 -08:00
Krrish Dholakia	0ab6b2451d	fix(acompletion): support client side timeouts + raise exceptions correctly for async calls	2023-11-17 15:39:47 -08:00
Krrish Dholakia	75ef1d7eb4	fix(router.py): check if async response is coroutine	2023-11-16 21:53:35 -08:00
Krrish Dholakia	7456c26940	docs(routing.md): update tutorial on deploying router	2023-11-16 21:46:43 -08:00
Krrish Dholakia	563a3d3016	refactor(router.py): renaming variable	2023-11-15 12:31:29 -08:00
Krrish Dholakia	03303033e5	feat(router.py): enable passing chat completion params for Router.chat.completion.create	2023-11-15 12:28:16 -08:00
Krrish Dholakia	59eaeba92a	fix(router.py): enabling retrying with expo backoff (without tenacity) for router	2023-11-14 20:57:51 -08:00
Krrish Dholakia	1738341dcb	fix(main.py): misrouting ollama models to nlp cloud	2023-11-14 18:55:08 -08:00
Krrish Dholakia	4475f2bbdd	bump: version 1.0.0.dev1 → 1.0.0	2023-11-13 16:48:53 -08:00
Krrish Dholakia	1d635aff08	fix(router.py): enable calling router with instructor	2023-11-13 15:16:57 -08:00
Krrish Dholakia	a4c9e6bd46	fix(utils.py): fix cached responses - translate dict to objects	2023-11-10 10:38:20 -08:00
Nathan Kim	f7dc2bf545	first commit	2023-11-09 22:12:39 -08:00
Nathan Kim	ee92c9c8b6	first commit	2023-11-09 22:07:19 -08:00
Krrish Dholakia	ba66294a3d	fix(router.py): fix linting issues	2023-11-06 18:50:09 -08:00
Krrish Dholakia	2b48a1fc3e	fix(router.py): adding health checks	2023-11-06 18:26:41 -08:00
Krrish Dholakia	6a45879932	bump: version 0.13.6.dev3 → 0.13.6	2023-11-06 18:19:20 -08:00
Krish Dholakia	9bef396d04	Merge pull request #722 from karvetskiy/fix-router-caching ... Fix caching for Router	2023-10-31 16:39:18 -07:00
mc-marcocheng	0a2cb9555c	Handle empty input edge case	2023-10-31 14:38:04 +08:00
seva	5e1e8820b4	Router & Caching fixes: ... - Add optional TTL to Cache parameters - Fix tpm and rpm caching in Router	2023-10-30 13:29:35 +01:00