Ishaan Jaff
51ffe93e77
(docs) add docstrings for all /key, /user, /team, /customer endpoints ( #6804 )
...
* use helper to handle_exception_on_proxy
* add doc string for /key/regenerate
* use 1 helper for handle_exception_on_proxy
* add doc string for /key/block
* add doc string for /key/unblock
* remove deprecated function
* remove deprecated endpoints
* remove incorrect tag for endpoint
* fix linting
* fix /key/regenerate
* fix regen key
* fix use port 4000 for user endpoints
* fix clean up - use separate file for customer endpoints
* add docstring for user/update
* fix imports
* doc string /user/list
* doc string for /team/delete
* fix team block endpoint
* fix import block user
* add doc string for /team/unblock
* add doc string for /team/list
* add doc string for /team/info
* add doc string for key endpoints
* fix customer_endpoints
* add doc string for customer endpoints
* fix import new_end_user
* fix testing
* fix import new_end_user
* fix add check for allow_user_auth
2024-11-18 19:44:06 -08:00
Ishaan Jaff
610974b4fc
(code quality) add ruff check PLR0915 for too-many-statements ( #6309 )
...
* ruff add PLR0915
* add noqa for PLR0915
* fix noqa
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* add # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
* # noqa: PLR0915
2024-10-18 15:36:49 +05:30
Krish Dholakia
2acb0c0675
Litellm Minor Fixes & Improvements (10/12/2024) ( #6179 )
...
* build(model_prices_and_context_window.json): add bedrock llama3.2 pricing
* build(model_prices_and_context_window.json): add bedrock cross region inference pricing
* Revert "(perf) move s3 logging to Batch logging + async [94% faster perf under 100 RPS on 1 litellm instance] (#6165 )"
This reverts commit 2a5624af47#6174 )
* LiteLLM Minor Fixes & Improvements (10/10/2024) (#6158 )
* refactor(vertex_ai_partner_models/anthropic): refactor anthropic to use partner model logic
* fix(vertex_ai/): support passing custom api base to partner models
Fixes https://github.com/BerriAI/litellm/issues/4317
* fix(proxy_server.py): Fix prometheus premium user check logic
* docs(prometheus.md): update quick start docs
* fix(custom_llm.py): support passing dynamic api key + api base
* fix(realtime_api/main.py): Add request/response logging for realtime api endpoints
Closes https://github.com/BerriAI/litellm/issues/6081
* feat(openai/realtime): add openai realtime api logging
Closes https://github.com/BerriAI/litellm/issues/6081
* fix(realtime_streaming.py): fix linting errors
* fix(realtime_streaming.py): fix linting errors
* fix: fix linting errors
* fix pattern match router
* Add literalai in the sidebar observability category (#6163 )
* fix: add literalai in the sidebar
* fix: typo
* update (#6160 )
* Feat: Add Langtrace integration (#5341 )
* Feat: Add Langtrace integration
* add langtrace service name
* fix timestamps for traces
* add tests
* Discard Callback + use existing otel logger
* cleanup
* remove print statments
* remove callback
* add docs
* docs
* add logging docs
* format logging
* remove emoji and add litellm proxy example
* format logging
* format `logging.md`
* add langtrace docs to logging.md
* sync conflict
* docs fix
* (perf) move s3 logging to Batch logging + async [94% faster perf under 100 RPS on 1 litellm instance] (#6165 )
* fix move s3 to use customLogger
* add basic s3 logging test
* add s3 to custom logger compatible
* use batch logger for s3
* s3 set flush interval and batch size
* fix s3 logging
* add notes on s3 logging
* fix s3 logging
* add basic s3 logging test
* fix s3 type errors
* add test for sync logging on s3
* fix: fix to debug log
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Willy Douhard <willy.douhard@gmail.com>
Co-authored-by: yujonglee <yujonglee.dev@gmail.com>
Co-authored-by: Ali Waleed <ali@scale3labs.com>
* docs(custom_llm_server.md): update doc on passing custom params
* fix(pass_through_endpoints.py): don't require headers
Fixes https://github.com/BerriAI/litellm/issues/6128
* feat(utils.py): add support for caching rerank endpoints
Closes https://github.com/BerriAI/litellm/issues/6144
* feat(litellm_logging.py'): add response headers for failed requests
Closes https://github.com/BerriAI/litellm/issues/6159
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Willy Douhard <willy.douhard@gmail.com>
Co-authored-by: yujonglee <yujonglee.dev@gmail.com>
Co-authored-by: Ali Waleed <ali@scale3labs.com>
2024-10-12 11:48:34 -07:00
Krish Dholakia
3933fba41f
LiteLLM Minor Fixes & Improvements (09/19/2024) ( #5793 )
...
* fix(model_prices_and_context_window.json): add cost tracking for more vertex llama3.1 model
8b and 70b models
* fix(proxy/utils.py): handle data being none on pre-call hooks
* fix(proxy/): create views on initial proxy startup
fixes base case, where user starts proxy for first time
Fixes https://github.com/BerriAI/litellm/issues/5756
* build(config.yml): fix vertex version for test
* feat(ui/): support enabling/disabling slack alerting
Allows admin to turn on/off slack alerting through ui
* feat(rerank/main.py): support langfuse logging
* fix(proxy/utils.py): fix linting errors
* fix(langfuse.py): log clean metadata
* test(tests): replace deprecated openai model
2024-09-20 08:19:52 -07:00
Krish Dholakia
72e961af3c
LiteLLM Minor Fixes and Improvements (08/06/2024) ( #5567 )
...
* fix(utils.py): return citations for perplexity streaming
Fixes https://github.com/BerriAI/litellm/issues/5535
* fix(anthropic/chat.py): support fallbacks for anthropic streaming (#5542 )
* fix(anthropic/chat.py): support fallbacks for anthropic streaming
Fixes https://github.com/BerriAI/litellm/issues/5512
* fix(anthropic/chat.py): use module level http client if none given (prevents early client closure)
* fix: fix linting errors
* fix(http_handler.py): fix raise_for_status error handling
* test: retry flaky test
* fix otel type
* fix(bedrock/embed): fix error raising
* test(test_openai_batches_and_files.py): skip azure batches test (for now) quota exceeded
* fix(test_router.py): skip azure batch route test (for now) - hit batch quota limits
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* All `model_group_alias` should show up in `/models`, `/model/info` , `/model_group/info` (#5539 )
* fix(router.py): support returning model_alias model names in `/v1/models`
* fix(proxy_server.py): support returning model alias'es on `/model/info`
* feat(router.py): support returning model group alias for `/model_group/info`
* fix(proxy_server.py): fix linting errors
* fix(proxy_server.py): fix linting errors
* build(model_prices_and_context_window.json): add amazon titan text premier pricing information
Closes https://github.com/BerriAI/litellm/issues/5560
* feat(litellm_logging.py): log standard logging response object for pass through endpoints. Allows bedrock /invoke agent calls to be correctly logged to langfuse + s3
* fix(success_handler.py): fix linting error
* fix(success_handler.py): fix linting errors
* fix(team_endpoints.py): Allows admin to update team member budgets
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2024-09-06 17:16:24 -07:00
Ishaan Jaff
c364d311b9
rename type
2024-09-04 16:33:36 -07:00
Ishaan Jaff
8eda374d79
feat log request / response on pass through endpoints
2024-09-04 16:26:32 -07:00
Ishaan Jaff
42b95c5979
code cleanup
2024-09-02 16:36:19 -07:00
Ishaan Jaff
a6d4a27207
pass through track usage for streaming endpoints
2024-09-02 16:11:20 -07:00
Ishaan Jaff
73d0a78444
use chunk_processort
2024-09-02 15:51:52 -07:00
Ishaan Jaff
f50374e81d
use helper class for pass through success handler
2024-08-30 15:52:47 -07:00
Ishaan Jaff
bcc0f99476
fix pass through endpoints
2024-08-21 17:21:22 -07:00
Krrish Dholakia
e747127e3b
fix(pass_through_endpoints.py): fix query param pass through
2024-08-19 21:38:30 -07:00
Krrish Dholakia
663a0c1b83
feat(Support-pass-through-for-bedrock-endpoints): Allows pass-through support for bedrock endpoints
2024-08-17 17:57:43 -07:00
Krrish Dholakia
f7a2e04426
feat(pass_through_endpoints.py): add pass-through support for all cohere endpoints
2024-08-17 16:57:55 -07:00
Krrish Dholakia
db54b66457
style(vertex_httpx.py): make vertex error string more helpful
2024-08-17 15:09:55 -07:00
Krrish Dholakia
fd44cf8d26
feat(pass_through_endpoints.py): support streaming requests
2024-08-17 12:46:57 -07:00
Krrish Dholakia
bc0023a409
feat(google_ai_studio_endpoints.py): support pass-through endpoint for all google ai studio requests
...
New Feature
2024-08-17 10:46:59 -07:00
Krrish Dholakia
b56ecd7e02
fix(pass_through_endpoints.py): fix returned response headers for pass-through endpoitns
2024-08-17 09:00:00 -07:00
Krrish Dholakia
61f4b71ef7
refactor: replace .error() with .exception() logging for better debugging on sentry
2024-08-16 09:22:47 -07:00
Krrish Dholakia
589da45c24
feat(pass_through_endpoints.py): initial working CRUD endpoints for /pass_through_endoints
2024-08-15 21:23:26 -07:00
Krrish Dholakia
28faafadb1
feat(pass_through_endpoints.py): initial commit of crud endpoints for pass through endpoints
2024-08-14 21:36:07 -07:00
Krrish Dholakia
742e3cbccf
feat(user_api_key_auth.py): support calling langfuse with litellm user_api_key_auth
2024-08-13 20:26:24 -07:00
Ishaan Jaff
ab920ab461
fix pass through endpoint tests
2024-08-06 21:16:07 -07:00
Ishaan Jaff
c277a71c1e
init pass through endpoints
2024-08-06 12:04:04 -07:00
Ishaan Jaff
bd1f323297
use helper to forward headers from request
2024-08-06 11:34:10 -07:00
Ishaan Jaff
d1b8c4e08d
forward headers from request
2024-08-05 21:45:44 -07:00
Krrish Dholakia
77325358b4
fix(pass_through_endpoints.py): fix client init
2024-07-13 14:46:56 -07:00
Krrish Dholakia
0cc273d77b
feat(pass_through_endpoint.py): support enforcing key rpm limits on pass through endpoints
...
Closes https://github.com/BerriAI/litellm/issues/4698
2024-07-13 13:29:44 -07:00
Krrish Dholakia
667fd2b376
docs(pass_through.md): add doc on creating custom chat endpoints on proxy
...
Allows developers to call proxy with anthropic sdk/boto3/etc.
2024-07-12 18:48:40 -07:00
Ishaan Jaff
c578816434
feat - setting up auth on pass through endpoint
2024-06-29 08:38:44 -07:00
Ishaan Jaff
6af1293384
feat - pass through langfuse requests
2024-06-28 17:27:13 -07:00
Ishaan Jaff
8f2931937a
fix use os.environ/ vars for pass through endpoints
2024-06-28 15:30:31 -07:00
Ishaan Jaff
f52cc18adb
feat - support pass through endpoints
2024-06-28 15:03:21 -07:00
