Commit graph

151 commits

Author SHA1 Message Date
Krish Dholakia
3933fba41f
LiteLLM Minor Fixes & Improvements (09/19/2024) (#5793)
* fix(model_prices_and_context_window.json): add cost tracking for more vertex llama3.1 model

8b and 70b models

* fix(proxy/utils.py): handle data being none on pre-call hooks

* fix(proxy/): create views on initial proxy startup

fixes base case, where user starts proxy for first time

 Fixes https://github.com/BerriAI/litellm/issues/5756

* build(config.yml): fix vertex version for test

* feat(ui/): support enabling/disabling slack alerting

Allows admin to turn on/off slack alerting through ui

* feat(rerank/main.py): support langfuse logging

* fix(proxy/utils.py): fix linting errors

* fix(langfuse.py): log clean metadata

* test(tests): replace deprecated openai model
2024-09-20 08:19:52 -07:00
Krish Dholakia
72e961af3c
LiteLLM Minor Fixes and Improvements (08/06/2024) (#5567)
* fix(utils.py): return citations for perplexity streaming

Fixes https://github.com/BerriAI/litellm/issues/5535

* fix(anthropic/chat.py): support fallbacks for anthropic streaming (#5542)

* fix(anthropic/chat.py): support fallbacks for anthropic streaming

Fixes https://github.com/BerriAI/litellm/issues/5512

* fix(anthropic/chat.py): use module level http client if none given (prevents early client closure)

* fix: fix linting errors

* fix(http_handler.py): fix raise_for_status error handling

* test: retry flaky test

* fix otel type

* fix(bedrock/embed): fix error raising

* test(test_openai_batches_and_files.py): skip azure batches test (for now) quota exceeded

* fix(test_router.py): skip azure batch route test (for now) - hit batch quota limits

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

* All `model_group_alias` should show up in `/models`, `/model/info` , `/model_group/info` (#5539)

* fix(router.py): support returning model_alias model names in `/v1/models`

* fix(proxy_server.py): support returning model alias'es on `/model/info`

* feat(router.py): support returning model group alias for `/model_group/info`

* fix(proxy_server.py): fix linting errors

* fix(proxy_server.py): fix linting errors

* build(model_prices_and_context_window.json): add amazon titan text premier pricing information

Closes https://github.com/BerriAI/litellm/issues/5560

* feat(litellm_logging.py): log standard logging response object for pass through endpoints. Allows bedrock /invoke agent calls to be correctly logged to langfuse + s3

* fix(success_handler.py): fix linting error

* fix(success_handler.py): fix linting errors

* fix(team_endpoints.py): Allows admin to update team member budgets

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2024-09-06 17:16:24 -07:00
Krish Dholakia
8d6a0bdc81
- merge - fix TypeError: 'CompletionUsage' object is not subscriptable #5441 (#5448)
* fix TypeError: 'CompletionUsage' object is not subscriptable (#5441)

* test(test_team_logging.py): mark flaky test

---------

Co-authored-by: yafei lee <yafei@dao42.com>
2024-08-30 08:54:42 -07:00
Krrish Dholakia
61f4b71ef7 refactor: replace .error() with .exception() logging for better debugging on sentry 2024-08-16 09:22:47 -07:00
Ishaan Jaff
a59ed00fd3 litellm always log cache_key on hits/misses 2024-08-15 09:59:58 -07:00
Ishaan Jaff
d8ef882905 fix langfuse log_provider_specific_information_as_span 2024-08-14 17:54:18 -07:00
Ishaan Jaff
42bd5de7c0 feat allow controlling logged tags on langfuse 2024-08-13 12:24:01 -07:00
Ishaan Jaff
4c4ccaff66 fix _hidden_params is None case 2024-08-09 19:17:11 -07:00
Ishaan Jaff
3e2a1fe0aa log provider specific metadata as a span 2024-08-09 14:32:02 -07:00
Ishaan Jaff
75fba18c9f fix langfuse hardcoded public key 2024-08-02 07:21:02 -07:00
Krrish Dholakia
f506eb341b feat(litellm_logging.py): log exception response headers to langfuse 2024-08-01 18:07:47 -07:00
Ishaan Jaff
285925e10a log output from /audio on langfuse 2024-07-29 08:21:22 -07:00
Ishaan Jaff
95f063f978 fix default input/output values for /audio/trancription logging 2024-07-29 08:03:08 -07:00
Krrish Dholakia
548e4f53f8 feat(redact_messages.py): allow remove sensitive key information before passing to logging integration 2024-07-22 20:58:02 -07:00
Andrea Ponti
496445481d Rollback to metadata deepcopy 2024-07-12 11:25:23 +02:00
Ishaan Jaff
d0a7983a41 fix try / except langfuse deep copy 2024-07-10 17:22:14 -07:00
Krrish Dholakia
1193ee8803 fix(presidio_pii_masking.py): fix presidio unset url check + add same check for langfuse 2024-07-06 17:50:55 -07:00
Krrish Dholakia
b4c8af771d fix(langfuse.py): use clean metadata instead of deepcopy 2024-06-25 18:20:39 -07:00
Krrish Dholakia
f8b390d421 fix(langfuse.py): cleanup 2024-06-24 21:43:40 -07:00
Krrish Dholakia
a4bea47a2d fix(router.py): log rejected router requests to langfuse
Fixes issue where rejected requests weren't being logged
2024-06-24 17:52:01 -07:00
Krrish Dholakia
682ec33aa0 fix(litellm_logging.py): initialize global variables
Fixes https://github.com/BerriAI/litellm/issues/4281
2024-06-19 18:39:45 -07:00
Ishaan Jaff
04038a0bef feat - _add_prompt_to_generation_params for langfuse 2024-06-18 19:55:16 -07:00
Hannes Burrichter
d338a94a57 Set Langfuse output to null for embedding responses 2024-06-16 15:14:34 +02:00
Krish Dholakia
056913fd70
Merge pull request #3559 from Intellegam/main
Langfuse integration support for `parent_observation_id` parameter
2024-06-14 06:55:45 -07:00
Krish Dholakia
677e0255c8
Merge branch 'main' into litellm_cleanup_traceback 2024-06-06 16:32:08 -07:00
Krrish Dholakia
6cca5612d2 refactor: replace 'traceback.print_exc()' with logging library
allows error logs to be in json format for otel logging
2024-06-06 13:47:43 -07:00
Ishaan Jaff
059c59f206 fix add_metadata_from_header 2024-06-06 09:53:12 -07:00
afel
aad0ea80f6 address review comments 2024-06-06 08:01:42 +02:00
afel
2b7d48f7b4 add metadata from header changes 2024-06-03 22:11:57 +02:00
Krrish Dholakia
872cd2d8a0 fix(langfuse.py): log litellm response cost as part of langfuse metadata 2024-06-03 12:58:30 -07:00
Ishaan Jaff
8c6a19d3ab fix put litellm prefix in generation name 2024-05-29 18:40:53 -07:00
Ishaan Jaff
67f1f374ec fix comment 2024-05-29 18:10:45 -07:00
Ishaan Jaff
1744176e63 feat - langfuse show _user_api_key_alias as generation nam 2024-05-29 18:03:13 -07:00
Ishaan Jaff
33a6647fac fix don't log langfuse cache_hit in tags 2024-05-21 14:18:53 -07:00
Hannes Burrichter
8ed41dee09 Revert set Langfuse output to null for embedding responses 2024-05-21 18:25:24 +02:00
Hannes Burrichter
82391d270c Add null check to parent_observation_id assignment 2024-05-21 18:24:18 +02:00
Hannes Burrichter
b89b3d8c44
Merge branch 'BerriAI:main' into main 2024-05-21 13:51:55 +02:00
Krrish Dholakia
4b3551abfc fix(slack_alerting.py): show langfuse traces on error messages 2024-05-17 18:42:30 -07:00
Hannes Burrichter
1bd6a1ba05
Merge branch 'BerriAI:main' into main 2024-05-14 13:31:07 +02:00
Alex Epstein
3bf2ccc856 feat(langfuse.py): Allow for individual call message/response redaction 2024-05-12 22:38:29 -04:00
Krish Dholakia
1d651c6049
Merge branch 'main' into litellm_bedrock_command_r_support 2024-05-11 21:24:42 -07:00
Krrish Dholakia
d142478b75 fix(langfuse.py): fix handling of dict object for langfuse prompt management 2024-05-11 20:42:55 -07:00
Ishaan Jaff
a41bef5297 debug langfuse 2024-05-11 14:12:26 -07:00
Ishaan Jaff
1bf8e7ac75 fix langfuse debug mode 2024-05-11 14:08:39 -07:00
Ishaan Jaff
038522ab24 fix - support debugging litellm params 2024-05-11 14:02:16 -07:00
Krrish Dholakia
4a3b084961 feat(bedrock_httpx.py): moves to using httpx client for bedrock cohere calls 2024-05-11 13:43:08 -07:00
Ishaan Jaff
ebb5c76e37 fix langfuse log clean metadata 2024-05-11 10:19:02 -07:00
Ishaan Jaff
e83743f8e1 fix langfuse - log metadata on traces 2024-05-11 09:59:05 -07:00
Hannes Burrichter
d0493248f4
Merge branch 'BerriAI:main' into main 2024-05-11 18:28:16 +02:00
Hannes Burrichter
84141917cf Set Langfuse output to null for embedding responses 2024-05-11 18:25:53 +02:00