Krish Dholakia
14165d3648
LiteLLM Minor Fixes & Improvements (10/02/2024) ( #6023 )
...
* feat(together_ai/completion): handle together ai completion calls
* fix: handle list of int / list of list of int for text completion calls
* fix(utils.py): check if base model in bedrock converse model list
Fixes https://github.com/BerriAI/litellm/issues/6003
* test(test_optional_params.py): add unit tests for bedrock optional param mapping
Fixes https://github.com/BerriAI/litellm/issues/6003
* feat(utils.py): enable passing dummy tool call for anthropic/bedrock calls if tool_use blocks exist
Fixes https://github.com/BerriAI/litellm/issues/5388
* fixed an issue with tool use of claude models with anthropic and bedrock (#6013 )
* fix(utils.py): handle empty schema for anthropic/bedrock
Fixes https://github.com/BerriAI/litellm/issues/6012
* fix: fix linting errors
* fix: fix linting errors
* fix: fix linting errors
* fix(proxy_cli.py): fix import route for app + health checks path (#6026 )
* (testing): Enable testing us.anthropic.claude-3-haiku-20240307-v1:0. (#6018 )
* fix(proxy_cli.py): fix import route for app + health checks gettsburg.wav
Fixes https://github.com/BerriAI/litellm/issues/5999
---------
Co-authored-by: David Manouchehri <david.manouchehri@ai.moda>
---------
Co-authored-by: Ved Patwardhan <54766411+vedpatwardhan@users.noreply.github.com>
Co-authored-by: David Manouchehri <david.manouchehri@ai.moda>
2024-10-02 22:00:28 -04:00
Krish Dholakia
d57be47b0f
Litellm ruff linting enforcement ( #5992 )
...
* ci(config.yml): add a 'check_code_quality' step
Addresses https://github.com/BerriAI/litellm/issues/5991
* ci(config.yml): check why circle ci doesn't pick up this test
* ci(config.yml): fix to run 'check_code_quality' tests
* fix(__init__.py): fix unprotected import
* fix(__init__.py): don't remove unused imports
* build(ruff.toml): update ruff.toml to ignore unused imports
* fix: fix: ruff + pyright - fix linting + type-checking errors
* fix: fix linting errors
* fix(lago.py): fix module init error
* fix: fix linting errors
* ci(config.yml): cd into correct dir for checks
* fix(proxy_server.py): fix linting error
* fix(utils.py): fix bare except
causes ruff linting errors
* fix: ruff - fix remaining linting errors
* fix(clickhouse.py): use standard logging object
* fix(__init__.py): fix unprotected import
* fix: ruff - fix linting errors
* fix: fix linting errors
* ci(config.yml): cleanup code qa step (formatting handled in local_testing)
* fix(_health_endpoints.py): fix ruff linting errors
* ci(config.yml): just use ruff in check_code_quality pipeline for now
* build(custom_guardrail.py): include missing file
* style(embedding_handler.py): fix ruff check
2024-10-01 19:44:20 -04:00
Ishaan Jaff
0d0f46a826
[Feat Proxy] Allow using hypercorn for http v2 ( #5950 )
...
* use run_hypercorn
* add docs on using hypercorn
2024-09-28 15:03:50 -07:00
Krish Dholakia
a1d9e96b31
LiteLLM Minor Fixes & Improvements (09/25/2024) ( #5893 )
...
* fix(langfuse.py): support new langfuse prompt_chat class init params
* fix(langfuse.py): handle new init values on prompt chat + prompt text templates
fixes error caused during langfuse logging
* docs(openai_compatible.md): clarify `openai/` handles correct routing for `/v1/completions` route
Fixes https://github.com/BerriAI/litellm/issues/5876
* fix(utils.py): handle unmapped gemini model optional param translation
Fixes https://github.com/BerriAI/litellm/issues/5888
* fix(o1_transformation.py): fix o-1 validation, to not raise error if temperature=1
Fixes https://github.com/BerriAI/litellm/issues/5884
* fix(prisma_client.py): refresh iam token
Fixes https://github.com/BerriAI/litellm/issues/5896
* fix: pass drop params where required
* fix(utils.py): pass drop_params correctly
* fix(types/vertex_ai.py): fix generation config
* test(test_max_completion_tokens.py): fix test
* fix(vertex_and_google_ai_studio_gemini.py): fix map openai params
2024-09-26 16:41:44 -07:00
steffen-sbt
de9a39e7c6
Add the option to specify a schema in the postgres DB, also modify docs ( #5640 )
2024-09-11 14:53:52 -07:00
Ishaan Jaff
3c898e23ea
refactor secret managers
2024-09-03 10:58:02 -07:00
Ishaan Jaff
b0178a85cf
refactor get_secret
2024-09-03 10:42:12 -07:00
Krrish Dholakia
8625663458
feat(proxy_server.py): support azure batch api endpoints
2024-08-22 15:21:43 -07:00
Krrish Dholakia
8ce8680a9a
fix(proxy_cli.py): support database_host, database_username, database_password, database_name
2024-08-19 16:17:45 -07:00
Krish Dholakia
036a6821d5
Merge pull request #5057 from BerriAI/litellm_rds_iam_auth
...
feat(proxy_cli.py): support iam-based auth to rds
2024-08-06 10:44:33 -07:00
Krrish Dholakia
1cc7c7fc59
feat(proxy_cli.py): support iam-based auth to rds
...
Initial pr for iam-based auth support for rds
2024-08-05 17:35:48 -07:00
Krrish Dholakia
936640948d
fix: bump default allowed_fails + reduce default db pool limit
...
Fixes issues with running proxy server in production
2024-08-05 15:07:46 -07:00
Krrish Dholakia
fe62e4e1c4
fix(proxy_cli.py): bump default azure api version
2024-07-08 16:28:22 -07:00
Krrish Dholakia
b84d335624
fix(proxy_cli.py): run aws kms decrypt before starting proxy server
2024-06-28 16:03:56 -07:00
Ishaan Jaff
aa3c14fa46
make sure linting runs proxy_cli.py
2024-06-20 20:20:08 -07:00
Chris Van Pelt
306c2b425d
Update proxy_cli.py
...
Fixed indentation to so we don't get an `UnboundLocalError`. Fixes #4324
2024-06-20 17:48:16 -07:00
Krrish Dholakia
248ee488f0
fix(proxy_cli.py): fix double counting json logs
2024-06-20 15:15:23 -07:00
Krrish Dholakia
e4dbb9b2db
fix(proxy_cli.py): support passing the database url as an encrypted kms key
2024-06-10 15:48:27 -07:00
Krrish Dholakia
0d3e52373c
fix(proxy/_logging.py): fix default logging level
2024-06-05 17:42:49 -07:00
Krrish Dholakia
3167bee25a
fix(proxy_cli.py): enable json logging via litellm_settings param on config
...
allows user to enable json logs without needing to figure out env variables
2024-05-29 21:41:20 -07:00
Krrish Dholakia
058bfb101d
feat(proxy_cli.py): support json logs on proxy
...
allow user to enable 'json logs' for proxy server
2024-05-20 09:18:12 -07:00
Krrish Dholakia
9eee2f3889
docs(prod.md): add 'disable load_dotenv' tutorial to docs
2024-05-14 19:13:22 -07:00
Krrish Dholakia
1ab4974773
fix: disable 'load_dotenv' for prod environments
2024-05-14 19:09:36 -07:00
Krrish Dholakia
6575143460
feat(proxy_server.py): return litellm version in response headers
2024-05-08 16:00:08 -07:00
Krrish Dholakia
b2741933dc
fix(proxy_cli.py): don't double load the router config
...
was causing callbacks to be instantiated twice - double couting usage in cache
2024-04-10 13:23:56 -07:00
Krrish Dholakia
6d32323e3d
fix(proxy_cli.py): revert db timeout change - user-controllable param
...
db timeout is a user controllable param, not necessary to change defaults
2024-04-03 09:37:57 -07:00
Krrish Dholakia
f07500c5ea
fix(proxy_server.py): bump default db timeouts
2024-04-03 09:35:08 -07:00
Krrish Dholakia
1e856443e1
feat(proxy/utils.py): enable updating db in a separate server
2024-03-27 16:02:36 -07:00
Krrish Dholakia
b204f0c01c
fix(proxy_cli.py): fix circular import issue
2024-03-26 21:16:41 -07:00
Krrish Dholakia
6d418a2920
fix(llm_guard.py): working llm-guard 'key-specific' mode
2024-03-26 17:47:20 -07:00
Krish Dholakia
9d7aceb06e
Merge pull request #2697 from antoniomdk/fix-database-credentials-leakage
...
(fix) Remove print statements from append_query_params
2024-03-26 16:07:33 -07:00
Ishaan Jaff
6b4b05b58f
(fix) remove litellm.telemetry
2024-03-26 11:21:09 -07:00
Antonio Molner Domenech
c713648db1
Update print statements to use verbose logger and DEBUG level
2024-03-26 22:41:28 +07:00
Ishaan Jaff
3ad6e5ffc1
(feat) start proxy with default num_workers=1
2024-03-20 10:46:32 -07:00
ishaan-jaff
ea6f42216c
(docs) use port 4000
2024-03-08 21:59:00 -08:00
ishaan-jaff
9ed51e791b
(fix) default num workers
2024-03-08 15:24:08 -08:00
Krrish Dholakia
3303236305
test(test_proxy_server.py): add back bedrock embedding tests
2024-03-04 18:28:05 -08:00
Krrish Dholakia
045d84e167
fix(proxy_cli.py): allow user to control db connection pool + timeouts from config
2024-02-17 19:03:43 -08:00
ishaan-jaff
c7e4432597
(feat) inc default gunicorn timeout
2024-02-12 21:49:32 -08:00
ishaan-jaff
18e73ec03d
(feat) litellm set ssl_certificates
2024-02-10 17:52:47 -08:00
Krrish Dholakia
ff93609453
build(schema.prisma): support direct url on prisma schema
2024-02-08 17:37:37 -08:00
Krrish Dholakia
0d803e1379
fix(proxy_cli.py): fix max connection limit issue on db
2024-02-07 22:57:44 -08:00
Krrish Dholakia
4a0df3cb4f
fix(proxy_cli.py-&&-proxy_server.py): bump reset budget intervals and fix pool limits for prisma connections
2024-02-06 19:39:49 -08:00
Krish Dholakia
9ab59045a3
Merge branch 'main' into litellm_embedding_caching_updates
2024-02-03 18:08:47 -08:00
Krrish Dholakia
8e9197b5b4
build(proxy_cli.py): make running gunicorn an optional cli arg
...
when running proxy locally, running with uvicorn is much better for debugging
2024-01-29 15:32:34 -08:00
Krrish Dholakia
dda115fcb7
build(ui/litellm_dashboard_v_2): allow app owner to create keys and view their keys
2024-01-27 13:48:35 -08:00
Krrish Dholakia
1e5efdfa37
fix(proxy_server.py): support setting tpm/rpm limits per user / per key
2024-01-18 17:03:18 -08:00
Krrish Dholakia
73daee7e07
fix(proxy_cli.py): ensure proxy always retries if db push fails to connect to db
2024-01-17 17:37:59 -08:00
ishaan-jaff
893627cab4
(feat) use env variables to set host, port, num_workers
2024-01-16 08:36:29 -08:00
ishaan-jaff
6a09fcfa06
(feat) print num_workers when starting prox
2024-01-16 08:21:37 -08:00
