Krish Dholakia
11f85d883f
LiteLLM Minor Fixes + Improvements ( #5474 )
...
* feat(proxy/_types.py): add lago billing to callbacks ui
Closes https://github.com/BerriAI/litellm/issues/5472
* fix(anthropic.py): return anthropic prompt caching information
Fixes https://github.com/BerriAI/litellm/issues/5364
* feat(bedrock/chat.py): support 'json_schema' for bedrock models
Closes https://github.com/BerriAI/litellm/issues/5434
* fix(bedrock/embed/embeddings.py): support async embeddings for amazon titan models
* fix: linting fixes
* fix: handle key errors
* fix(bedrock/chat.py): fix bedrock ai21 streaming object
* feat(bedrock/embed): support bedrock embedding optional params
* fix(databricks.py): fix usage chunk
* fix(internal_user_endpoints.py): apply internal user defaults, if user role updated
Fixes issue where user update wouldn't apply defaults
* feat(slack_alerting.py): provide multiple slack channels for a given alert type
multiple channels might be interested in receiving an alert for a given type
* docs(alerting.md): add multiple channel alerting to docs
2024-09-02 14:29:57 -07:00
Krish Dholakia
ca4e746545
LiteLLM minor fixes + improvements (31/08/2024) ( #5464 )
...
* fix(vertex_endpoints.py): fix vertex ai pass through endpoints
* test(test_streaming.py): skip model due to end of life
* feat(custom_logger.py): add special callback for model hitting tpm/rpm limits
Closes https://github.com/BerriAI/litellm/issues/4096
2024-09-01 13:31:42 -07:00
Ishaan Jaff
3fae5eb94e
feat prometheus add metric for failure / model
2024-08-31 10:05:23 -07:00
Ishaan Jaff
c60125d7be
add gcs bucket base
2024-08-30 10:41:39 -07:00
Krish Dholakia
fe2a3c02e5
- merge - fix TypeError: 'CompletionUsage' object is not subscriptable #5441 ( #5448 )
...
* fix TypeError: 'CompletionUsage' object is not subscriptable (#5441 )
* test(test_team_logging.py): mark flaky test
---------
Co-authored-by: yafei lee <yafei@dao42.com>
2024-08-30 08:54:42 -07:00
Ishaan Jaff
fddf10eeb8
prometheus - safe update start / end time
2024-08-28 16:13:56 -07:00
Ishaan Jaff
359a003ac8
v0 add rerank on litellm proxy
2024-08-27 17:28:39 -07:00
Ishaan Jaff
a8e192a868
fix use guardrail for pre call hook
2024-08-23 09:34:08 -07:00
Ishaan Jaff
be853d93da
fix prom latency metrics
2024-08-23 06:59:19 -07:00
Ishaan Jaff
9476582fb7
update promtheus metric names
2024-08-22 14:03:00 -07:00
Ishaan Jaff
c719c375f7
track litellm_request_latency_metric
2024-08-22 13:58:10 -07:00
Ishaan Jaff
0ccb1c17f7
fix init correct prometheus metrics
2024-08-22 13:29:35 -07:00
Krish Dholakia
41835d9397
Merge pull request #5323 from MarkRx/feature/langsmith-ids
...
Support LangSmith parent_run_id, trace_id, session_id
2024-08-21 15:38:50 -07:00
MarkRx
58529e2c9c
Support LangSmith parent_run_id, trace_id, session_id
2024-08-21 16:09:30 -04:00
Ishaan Jaff
cdbd245c3d
working lakera ai during call hook
2024-08-20 14:39:04 -07:00
Ishaan Jaff
2f618d08be
fix _get_spend_report_for_time_range
2024-08-19 20:53:39 -07:00
Ishaan Jaff
319690ab5e
feat - guardrails v2
2024-08-19 18:24:20 -07:00
Ishaan Jaff
b4bca8db82
feat - allow accessing data post success call
2024-08-19 11:35:33 -07:00
Ishaan Jaff
db8f789318
Merge pull request #5259 from BerriAI/litellm_return_remaining_tokens_in_header
...
[Feat] return `x-litellm-key-remaining-requests-{model}`: 1, `x-litellm-key-remaining-tokens-{model}: None` in response headers
2024-08-17 12:41:16 -07:00
Ishaan Jaff
a62277a6aa
feat - use commong helper for getting model group
2024-08-17 10:46:04 -07:00
Ishaan Jaff
2dd098f384
show correct metric
2024-08-17 10:12:23 -07:00
Ishaan Jaff
03196742d2
add litellm-key-remaining-tokens on prometheus
2024-08-17 10:02:20 -07:00
Krrish Dholakia
2874b94fb1
refactor: replace .error() with .exception() logging for better debugging on sentry
2024-08-16 09:22:47 -07:00
Krish Dholakia
ca07898fbb
Merge pull request #5235 from BerriAI/litellm_fix_s3_logs
...
fix(s3.py): fix s3 logging payload to have valid json values
2024-08-15 23:00:18 -07:00
Krrish Dholakia
b08492bc29
fix(s3.py): fix s3 logging payload to have valid json values
...
Previously pydantic objects were being stringified, making them unparsable
2024-08-15 17:09:02 -07:00
Ishaan Jaff
17857ecc2b
litellm always log cache_key on hits/misses
2024-08-15 09:59:58 -07:00
Ishaan Jaff
539fcad7e7
fix langfuse log_provider_specific_information_as_span
2024-08-14 17:54:18 -07:00
Ishaan Jaff
794055660d
Merge pull request #5202 from BerriAI/litellm_prom_prefix_litellm
...
[Fix] Prometheus use 'litellm_' prefix for new deployment metrics
2024-08-14 09:50:36 -07:00
Ishaan Jaff
38868a0a45
use litellm_ prefix for new deployment metrics
2024-08-14 09:08:14 -07:00
Ishaan Jaff
e1c70a6954
log failure calls on gcs + testing
2024-08-14 08:55:51 -07:00
Ishaan Jaff
da61511a8e
feat log fail events on gcs
2024-08-14 08:39:16 -07:00
Krrish Dholakia
a181af3d2e
fix(langsmith.py): support langsmith 'extra' field object
...
Closes https://github.com/BerriAI/litellm/issues/5179
2024-08-13 15:20:50 -07:00
Ishaan Jaff
e5ccd3bdaf
allow using langfuse_default_tags
2024-08-13 12:26:37 -07:00
Ishaan Jaff
c5515513a9
feat allow controlling logged tags on langfuse
2024-08-13 12:24:01 -07:00
Ishaan Jaff
f043de98e3
feat log responses in folders
2024-08-12 16:28:12 -07:00
Ishaan Jaff
eaf338aa5f
feat gcs log user api key metadata
2024-08-12 16:06:10 -07:00
Ishaan Jaff
b0629e5947
Merge pull request #5166 from BerriAI/litellm_log_key_created_slack
...
[Feat-Security] Send Slack Alert when CRUD ops done on Virtual Keys, Teams, Internal Users
2024-08-12 12:18:04 -07:00
Ishaan Jaff
0f7d575992
send alert on all key events
2024-08-12 11:39:24 -07:00
Krrish Dholakia
3fa00408f1
fix(proxy_server.py): add info log when spend logs is skipped because disable_spend_logs=True.
2024-08-12 11:20:30 -07:00
Ishaan Jaff
2e2da04024
v0 log KeyCreatedEvent
2024-08-12 10:56:11 -07:00
Ishaan Jaff
7059615fb1
allow setting PROMETHEUS_SELECTED_INSTANCE
2024-08-10 17:31:05 -07:00
Ishaan Jaff
00443aa0f9
Merge pull request #5154 from BerriAI/litellm_send_prometheus_fallbacks_from_slack
...
[Feat-Proxy] send prometheus fallbacks stats to slack
2024-08-10 17:14:01 -07:00
Ishaan Jaff
ecec37e220
doc new prometheus metrics
2024-08-10 17:13:36 -07:00
Ishaan Jaff
fe26412f03
feat - use api to get prometheus api metrics
2024-08-10 16:36:06 -07:00
Ishaan Jaff
f324ca7b86
add fallback_reports in slack alert types
2024-08-10 16:08:36 -07:00
Ishaan Jaff
328a78a0f8
feat add prometheus api to get data from endpoint
2024-08-10 16:07:08 -07:00
Ishaan Jaff
3ecf4db741
prometheus log_success_fallback_event
2024-08-10 14:05:18 -07:00
Ishaan Jaff
3ca503cc8c
v0 add helper for loging success/fail fallback events
2024-08-10 13:26:39 -07:00
Ishaan Jaff
5765baa5b2
feat - track latency per llm deployment
2024-08-10 12:53:56 -07:00
Ishaan Jaff
e086479fd7
track llm_deployment_success_responses
2024-08-10 10:05:33 -07:00
