2042 commits

Author	SHA1	Message	Date
chabala98	c5ce11541b	adfix: ded **optional_params in input data when system prompt is available (allows to papass other optional params apart from sys prompt)	2023-12-11 14:42:05 +01:00
chabala98	c2e2e927fb	fix system prompts for replicate	2023-12-01 13:16:35 +01:00
ishaan-jaff	1e9aa69268	(fix) proxy: use orjson.loads()	2023-11-30 20:24:31 -08:00
ishaan-jaff	06805c7f47	(fix) formatting	2023-11-30 20:03:56 -08:00
ishaan-jaff	1081d4c766	(feat) aembedding: return raw openai response	2023-11-30 20:02:47 -08:00
ishaan-jaff	10d5ab8643	(feat) proxy: /embedding -> use ORJSON responses	2023-11-30 20:00:35 -08:00
ishaan-jaff	2d55cc753e	(feat) proxy: use orjson	2023-11-30 19:50:47 -08:00
ishaan-jaff	853af29a25	(test) load test embedding	2023-11-30 19:04:51 -08:00
Krrish Dholakia	aa36bd2784	fix(utils.py): expand openai_token_counter selection	2023-11-30 18:51:51 -08:00
Krrish Dholakia	ff4457e2d2	fix(router.py): back-off if no models available	2023-11-30 18:42:29 -08:00
Krrish Dholakia	7b99cfe6b9	test(test_token_counter.py): include testing for function calls	2023-11-30 18:24:21 -08:00
Krrish Dholakia	7f04758bcb	(fix) support counting tokens for tool calls	2023-11-30 18:24:21 -08:00
Frank Colson	7ddfeb75bc	Add backwards compatability	2023-11-30 16:35:19 -07:00
Frank Colson	5e6913dff2	Use poetry extras for proxy	2023-11-30 16:23:34 -07:00
ishaan-jaff	bc2299184b	(fix) proxy - don't overwrite metadata passed	2023-11-30 15:15:47 -08:00
Krrish Dholakia	c473abde49	fix(azure.py): logging fix	2023-11-30 14:13:40 -08:00
Krrish Dholakia	032f71adb2	fix(router.py): support cloudflare ai gateway for azure models on router	2023-11-30 14:09:06 -08:00
ishaan-jaff	936c27c9ee	(fix +test) langfuse log metadata	2023-11-30 13:53:43 -08:00
Krrish Dholakia	a5c99d8594	fix(langfuse.py): fix metadata logging for langfuse	2023-11-30 13:45:34 -08:00
Krrish Dholakia	82553e8aac	fix(azure.py): fix linting errors	2023-11-30 13:32:29 -08:00
ishaan-jaff	9a1accfe2a	(chore) proxy: remove junk load test	2023-11-30 13:31:23 -08:00
Krrish Dholakia	4f07c8565a	feat(main.py): add support for azure-openai via cloudflare ai gateway	2023-11-30 13:19:49 -08:00
ishaan-jaff	be8bdb580a	(test) proxy + router: add bursty load test	2023-11-30 13:17:11 -08:00
ishaan-jaff	a8a6838867	(docs) example: azure config.yaml	2023-11-30 13:16:41 -08:00
ishaan-jaff	97ff0caf70	(feat) proxy: config - azure allow users to pass in base_url	2023-11-30 10:56:55 -08:00
Krrish Dholakia	062ede96e3	refactor(proxy_server.py): fix linting issues	2023-11-30 09:24:59 -08:00
Krrish Dholakia	b4b7acdb72	fix(utils.py): fix azure completion cost calculation	2023-11-30 09:19:35 -08:00
Krrish Dholakia	7ee089b5ca	fix(proxy_server.py): provide an endpoint that gives model-specific info from proxy	2023-11-30 09:08:19 -08:00
ishaan-jaff	a56d4a1e83	(fix) proxy: print cwd()	2023-11-30 08:52:06 -08:00
Krrish Dholakia	01c7e18f31	fix(utils.py): include system fingerprint in streaming response object	2023-11-30 08:45:52 -08:00
ishaan-jaff	ecdc5bdad6	(dos) config.yaml	2023-11-30 08:34:36 -08:00
Krrish Dholakia	e98fac66da	fix(utils.py): fix register model cost map	2023-11-29 21:12:29 -08:00
ishaan-jaff	e5ce45dc2c	(cleanup) proxy/health	2023-11-29 20:15:52 -08:00
Krrish Dholakia	0d200cd8dc	feat(main.py): allow updating model cost via completion()	2023-11-29 20:14:39 -08:00
ishaan-jaff	4c1ef4e270	(chore) fix testing	2023-11-29 20:05:13 -08:00
Krrish Dholakia	50cc4a8595	fix(proxy_server.py): have /health and /routes be router endpoints	2023-11-29 19:59:56 -08:00
ishaan-jaff	4ed5b3b46d	(chore) linting fix	2023-11-29 19:58:12 -08:00
Krrish Dholakia	a9fdae0d23	test(test_streaming.py): refactor testing	2023-11-29 19:58:04 -08:00
Krrish Dholakia	c312ac4ca8	fix(main.py): don't pass stream to petals	2023-11-29 19:58:04 -08:00
ishaan-jaff	9780efca4b	(feat) router: async client Azure, OpenAI	2023-11-29 19:45:08 -08:00
Krrish Dholakia	1f5a1122fc	fix(replicate.py): fix custom prompt formatting	2023-11-29 19:44:09 -08:00
ishaan-jaff	c05da0797b	(feat) Embedding: Async Azure	2023-11-29 19:43:47 -08:00
ishaan-jaff	53554bae85	(test) aembedding	2023-11-29 19:36:42 -08:00
ishaan-jaff	10e21ae978	(test) aembedding	2023-11-29 19:35:32 -08:00
ishaan-jaff	09caab549a	(feat) async embeddings: OpenAI	2023-11-29 19:35:08 -08:00
ishaan-jaff	3891462b29	(fix) router: azure/embedding support	2023-11-29 19:06:36 -08:00
ishaan-jaff	7bcc23e8e9	(fix) router: set default rpm/tpm when not set	2023-11-29 18:13:27 -08:00
ishaan-jaff	305faab542	(test) router:get_available_deployment	2023-11-29 17:54:41 -08:00
ishaan-jaff	23af756531	(feat) router: random pick based on tpm/rpm	2023-11-29 17:54:06 -08:00
ishaan-jaff	2c74dbed17	(chore) util: remove_model_id	2023-11-29 17:30:33 -08:00