Commit graph

110 commits

Author SHA1 Message Date
Ishaan Jaff
6220f3e7b8
[Feat SSO] Add LiteLLM SCIM Integration for Team and User management (#10072)
* fix NewUser response type

* add scim router

* add v0 scim v2 endpoints

* working scim transformation

* use 1 file for types

* fix scim firstname and givenName storage

* working SCIMErrorResponse

* working team / group provisioning on SCIM

* add SCIMPatchOp

* move scim folder

* fix import scim_router

* fix dont auto create scim keys

* add auth on all scim endpoints

* add is_virtual_key_allowed_to_call_route

* fix allowed routes

* fix for key management

* fix allowed routes check

* clean up error message

* fix code check

* fix for route checks

* ui SCIM support

* add UI tab for SCIM

* fixes SCIM

* fixes for SCIM settings on ui

* scim settings

* clean up scim view

* add migration for allowed_routes in keys table

* refactor scim transform

* fix SCIM linting error

* fix code quality check

* fix ui linting

* test_scim_transformations.py
2025-04-16 19:21:47 -07:00
Krish Dholakia
c0d7e9f16d
Add new /tag/daily/activity endpoint + Add tag dashboard to UI (#10073)
Some checks failed
Read Version from pyproject.toml / read-version (push) Successful in 15s
Helm unit test / unit-test (push) Successful in 24s
Publish Prisma Migrations / publish-migrations (push) Failing after 1m47s
* feat: initial commit adding daily tag spend table to db

* feat(db_spend_update_writer.py): correctly log tag spend transactions

* build(schema.prisma): add new tag table to root

* build: add new migration file

* feat(common_daily_activity.py): add `/tag/daily/activity` API endpoint

allows viewing daily spend by tag

* feat(tag_management_endpoints.py): support comma separated list of tags + tag breakdown metric

allows querying multiple tags + knowing what tags are driving spend

* feat(entity_usage.tsx): initial commit adding tag based usage to litellm dashboard

brings back tag based usage tracking to UI at 1m+ spend logs

* feat(entity_usage.tsx): add top api key view to ui

* feat(entity_usage.tsx): add tag table to ui

* feat(entity_usage.tsx): allow filtering by tag

* refactor(entity_usage.tsx): reorder components

* build(ui/): fix linting error

* fix: fix ruff checks

* fix(schema.prisma): drop uniqueness requirement on tag

allows dailytagspend to have multiple rows with the same tag

* build(schema.prisma): drop uniqueness requirement on tag in dailytagspend

allows tag agg. view to work on multiple rows with same tag

* build(schema.prisma): drop tag uniqueness requirement
2025-04-16 15:24:44 -07:00
Krish Dholakia
d8a1071bc4
Add aggregate spend by tag (#10071)
* feat: initial commit adding daily tag spend table to db

* feat(db_spend_update_writer.py): correctly log tag spend transactions

* build(schema.prisma): add new tag table to root

* build: add new migration file
2025-04-16 12:26:21 -07:00
Ishaan Jaff
bd88263b29
[Feat - Cost Tracking improvement] Track prompt caching metrics in DailyUserSpendTransactions (#10029)
* stash changes

* emit cache read/write tokens to daily spend update

* emit cache read/write tokens on daily activity

* update types.ts

* docs prompt caching

* undo ui change

* fix activity metrics

* fix prompt caching metrics

* fix typed dict fields

* fix get_aggregated_daily_spend_update_transactions

* fix aggregating cache tokens

* test_cache_token_fields_aggregation

* daily_transaction

* add cache_creation_input_tokens and cache_read_input_tokens to LiteLLM_DailyUserSpend

* test_daily_spend_update_queue.py
2025-04-15 21:40:57 -07:00
Krish Dholakia
9b77559ccf
Add aggregate team based usage logging (#10039)
* feat(schema.prisma): initial commit adding aggregate table for team spend

allows team spend to be visible at 1m+ logs

* feat(db_spend_update_writer.py): support logging aggregate team spend

allows usage dashboard to work at 1m+ logs

* feat(litellm-proxy-extras/): add new migration file

* fix(db_spend_update_writer.py): fix return type

* build: bump requirements

* fix: fix ruff error
2025-04-15 20:58:48 -07:00
Ishaan Jaff
fb0c3d9e18
[DB / Infra] Add new column team_member_permissions (#9941)
* add team_member_permissions to team table

* add migration.sql file

* fix poetry lock

* fix prisma migrations

* fix poetry lock

* fix migration
2025-04-12 09:06:04 -07:00
Krish Dholakia
421e0a3004
Litellm add managed files db (#9930)
* fix(openai.py): ensure openai file object shows up on logs

* fix(managed_files.py): return unified file id as b64 str

allows retrieve file id to work as expected

* fix(managed_files.py): apply decoded file id transformation

* fix: add unit test for file id + decode logic

* fix: initial commit for litellm_proxy support with CRUD Endpoints

* fix(managed_files.py): support retrieve file operation

* fix(managed_files.py): support for DELETE endpoint for files

* fix(managed_files.py): retrieve file content support

supports retrieve file content api from openai

* fix: fix linting error

* test: update tests

* fix: fix linting error

* feat(managed_files.py): support reading / writing files in DB

* feat(managed_files.py): support deleting file from DB on delete

* test: update testing

* fix(spend_tracking_utils.py): ensure each file create request is logged correctly

* fix(managed_files.py): fix storing / returning managed file object from cache

* fix(files/main.py): pass litellm params to azure route

* test: fix test

* build: add new prisma migration

* build: bump requirements

* test: add more testing

* refactor: cleanup post merge w/ main

* fix: fix code qa errors
2025-04-12 08:24:46 -07:00
Krish Dholakia
62ad84fb64
UI (new_usage.tsx): Report 'total_tokens' + report success/failure calls (#9675)
* feat(internal_user_endpoints.py): return 'total_tokens' in `/user/daily/analytics`

* test(test_internal_user_endpoints.py): add unit test to assert spend metrics and dailyspend metadata always report the same fields

* build(schema.prisma): record success + failure calls to daily user table

allows understanding why model requests might exceed provider requests (e.g. user hit rate limit error)

* fix(internal_user_endpoints.py): report success / failure requests in API

* fix(proxy/utils.py): default to success

status can be missing or none at times for successful requests

* feat(new_usage.tsx): show success/failure calls on UI

* style(new_usage.tsx): ui cleanup

* fix: fix linting error

* fix: fix linting error

* feat(litellm-proxy-extras/): add new migration files
2025-03-31 22:48:43 -07:00
Ishaan Jaff
7e8a02099c Merge branch 'main' into litellm_use_redis_for_updates 2025-03-28 20:12:29 -07:00
Krish Dholakia
cdcc8ea9b7
Connect UI to "LiteLLM_DailyUserSpend" spend table - enables usage tab to work at 1m+ spend logs (#9603)
* feat(spend_management_endpoints.py): expose new endpoint for querying user's usage at 1m+ spend logs

Allows user to view their spend at 1m+ spend logs

* build(schema.prisma): add api_requests to dailyuserspend table

* build(migration.sql): add migration file for new column to daily user spend table

* build(prisma_client.py): add logic for copying over migration folder, if deploy/migrations present in expected location

enables easier testing of prisma migration flow

* build(ui/): initial commit successfully using the dailyuserspend table on the UI

* refactor(internal_user_endpoints.py): refactor `/user/daily/activity` to give breakdowns by provider/model/key

* feat: feature parity (cost page) with existing 'usage' page

* build(ui/): add activity tab to new_usage.tsx

gets to feature parity on 'All Up' page of 'usage.tsx'

* fix(proxy/utils.py): count number of api requests in daily user spend table

allows us to see activity by model on new usage tab

* style(new_usage.tsx): fix y-axis to be in ascending order of date

* fix: fix linting errors

* fix: fix ruff check errors
2025-03-27 23:29:15 -07:00
Ishaan Jaff
1ba73f491a working cron job updates 2025-03-27 23:13:01 -07:00
Ishaan Jaff
0cb6cb6c66 fix schema to work with lock acquisition 2025-03-27 22:54:46 -07:00
Ishaan Jaff
ea93a09b7a add model CronJob 2025-03-27 21:10:26 -07:00
Krish Dholakia
37265c8b1c
Add Daily User Spend Aggregate view - allows UI Usage tab to work > 1m rows (#9538)
* ci: update github action

* build(schema.prisma): enable daily user spend table

allows storing aggregate view of user's daily spend

* build(schema.prisma): add new daily user spend table

* feat: working daily user spend tracking

maintains an aggregate view for easier querying in high traffic

* setup_google_dns

* ci: update ci yaml

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-03-26 16:36:36 -07:00
Krrish Dholakia
a962a97fcb feat(endpoints.py): support writing credentials to db 2025-03-10 18:27:43 -07:00
Krish Dholakia
2d2d1b9df5
Add created_by and updated_by fields to Keys table (#8885)
* fix(proxy/_types.py): return created_by and updated_by on /key/list

enables better trail of who made a key

* fix(all_keys_table.tsx): add created by to key table

allows easier tracking of who generated the key

* fix(key_management_endpoints.py): track 'created_by' and 'updated_by' fields

enable easier tracking of who created proxy keys
2025-02-27 18:12:58 -08:00
Krish Dholakia
13a3e8630e
Org UI Improvements (#8436)
* feat(team_endpoints.py): support returning teams filtered by organization_id

allows user to just get teams they belong to, within the org

Enables org admin to see filtered list of teams on UI

* fix(teams.tsx): simple filter for team on ui - just filter team based on selected org id

* feat(ui/organizations): show 'default org' in switcher, filter teams based on selected org

* feat(user_dashboard.tsx): update team in switcher when org changes

* feat(schema.prisma): add new 'organization_id' value to key table

allow org admin to directly issue keys to a user within their org

* fix(view_key_table.tsx): fix regression where admin couldn't see keys

caused by bad console log statement

* fix(team_endpoints.py): handle default org value in /team/list

* fix(key_management_endpoints.py): allow proxy admin to create keys for team they're not in

* fix(team_endpoints.py): fix team endpoint to handle org id not being passed in

* build(config.yml): investigate what pkg is installing posthog in ci/cd

* ci(config.yml): uninstall posthog

prevent it from being added in ci/cd

* ci: auto-install ci
2025-02-10 19:13:32 -08:00
Krrish Dholakia
d0c5639912 build: bump root schema.prisma
All checks were successful
Read Version from pyproject.toml / read-version (push) Successful in 41s
2025-01-31 23:04:51 -08:00
Ishaan Jaff
d3c2f4331a
(UI - View SpendLogs Table) (#7842)
* litellm log messages / responses

* add messages/response to schema.prisma

* add support for logging messages / responses in DB

* test_spend_logs_payload_with_prompts_enabled

* _get_messages_for_spend_logs_payload

* ui_view_spend_logs endpoint

* add tanstack and moment

* add uiSpendLogsCall

* ui view logs table

* ui view spendLogs table

* ui_view_spend_logs

* fix code quality

* test_spend_logs_payload_with_prompts_enabled

* _get_messages_for_spend_logs_payload

* test_spend_logs_payload_with_prompts_enabled

* test_spend_logs_payload_with_prompts_enabled

* ui view spend logs

* minor ui fix

* ui - update leftnav

* ui - clean up ui

* fix leftnav

* ui fix navbar

* ui fix moving chat ui tab
2025-01-17 18:53:45 -08:00
Krish Dholakia
f1540ceeab
Create and view organizations + assign org admins on the Proxy UI (#7557)
* feat: initial commit for new 'organizations' tab on ui

* build(ui/): create generic card for rendering complete org data table

can be reused in teams as well

simplifies things

* build(ui/): display created orgs on ui

* build(ui/): support adding orgs via UI

* build(ui/): add org in selection dropdown

* build(organizations.tsx): allow assigning org admins

* build(ui/): show org members on ui

* build(ui/): cleanup + show actual models on org dropdown

* build(ui/): explain user roles within organization
2025-01-04 17:31:24 -08:00
Ishaan Jaff
19f1425232
litellm db fixes LiteLLM_UserTable (#7089) 2024-12-07 19:08:37 -08:00
Ishaan Jaff
d4c52e2921
(feat) Track custom_llm_provider in LiteLLMSpendLogs (#7081)
* add custom_llm_provider to SpendLogsPayload

* add custom_llm_provider to SpendLogs

* add custom llm provider to SpendLogs payload

* test_spend_logs_payload
2024-12-07 13:40:22 -08:00
Ishaan Jaff
cce118e091 track created, updated at virtual keys 2024-10-25 07:19:29 +04:00
Ishaan Jaff
8a9bb51f4e fix schema.prisma change 2024-10-09 15:25:27 +05:30
Ishaan Jaff
1fd437e263
(feat proxy) [beta] add support for organization role based access controls (#6112)
* track LiteLLM_OrganizationMembership

* add add_internal_user_to_organization

* add org membership to schema

* read organization membership when reading user info in auth checks

* add check for valid organization_id

* add test for test_create_new_user_in_organization

* test test_create_new_user_in_organization

* add new ADMIN role

* add test for org admins creating teams

* add test for test_org_admin_create_user_permissions

* test_org_admin_create_user_team_wrong_org_permissions

* test_org_admin_create_user_team_wrong_org_permissions

* fix organization_role_based_access_check

* fix getting user members

* fix TeamBase

* fix types used for use role

* fix type checks

* sync prisma schema

* docs - organization admins

* fix use organization_endpoints for /organization management

* add types for org member endpoints

* fix role name for org admin

* add type for member add response

* add organization/member_add

* add error handling for adding members to an org

* add nice doc string for oranization/member_add

* fix test_create_new_user_in_organization

* linting fix

* use simple route changes

* fix types

* add organization member roles

* add org admin auth checks

* add auth checks for orgs

* test for creating teams as org admin

* simplify org id usage

* fix typo

* test test_org_admin_create_user_team_wrong_org_permissions

* fix type check issue

* code quality fix

* fix schema.prisma
2024-10-09 15:18:18 +05:30
Ishaan Jaff
391b107909
[Feat UI sso] store 'provider' in user metadata (#5856)
* store sso provider in user metadata

* store user metadata

* store user auth_provider in user metadata

* add "metadata" for LiteLLM_UserTable

* fix sso test
2024-09-23 17:49:36 -07:00
Krrish Dholakia
dad3964207 build(schema.prisma): add column 'blocked' for litellm keys
enables blocking/unblocking litellm keys
2024-09-20 19:40:45 -07:00
Krish Dholakia
c76d2c6ade
Refactor 'check_view_exists' logic (#5659)
* fix(proxy/utils.py): comment out auto-upsert logic in check_view_exists

Prevents proxy from failing on startup due to faulty logic

* fix(db/migration_scripts/create_views.py): fix 'DailyTagSpend' quotation on check

* fix(create_views.py): mongly global spend time period should be 30d not 20d

* fix(schema.prisma): index on startTime and endUser for efficient UI querying
2024-09-12 13:39:50 -07:00
Ishaan Jaff
a44596a067 fix created_at and updated_at not existing error 2024-08-26 21:04:39 -07:00
Ishaan Jaff
5745f3d6cc fix schema 2024-08-26 17:27:06 -07:00
Ishaan Jaff
fb150f7ce5 update schema 2024-08-26 16:52:19 -07:00
Ishaan Jaff
78eb5164df fix DB accept null values for api_base, user, etc 2024-07-23 16:33:04 -07:00
Ishaan Jaff
b99ffcf8ae SpendLogsPayload- track user ip 2024-07-08 10:16:58 -07:00
Krish Dholakia
55a20c7cce
Merge pull request #4084 from BerriAI/litellm_batch_add_team_members
fix(proxy_server.py): allow passing in a list of team members
2024-06-10 20:26:35 -07:00
Krrish Dholakia
def648ed3f fix(proxy_server.py): allow passing in a list of team members
allows batch adding members to a team by passing in a list. fixes concurrency issue caused by calling team/member_add in parallel
2024-06-08 19:03:45 -07:00
Ishaan Jaff
9c3c42fa4b feat - support litellm_changed_by 2024-06-08 16:02:18 -07:00
Ishaan Jaff
5bd658493f feat - working audit logs for create, update delete team 2024-06-05 17:50:27 -07:00
Ishaan Jaff
970c3dfdc7 v0 audit logs 2024-06-05 16:19:38 -07:00
Krrish Dholakia
86b66c13a4 feat(proxy_server.py): allow admin to invite users via invite link
Closes https://github.com/BerriAI/litellm/issues/3863
2024-05-27 20:32:25 -07:00
Krish Dholakia
5a3aca10ce
Merge pull request #3789 from BerriAI/litellm_ttft_ui
feat(schema.prisma): store model id + model group as part of spend logs allows precise model metrics
2024-05-22 18:22:39 -07:00
Krrish Dholakia
3c0e9cb48e feat(schema.prisma): store model id + model group as part of spend logs
allows precise model metrics
2024-05-22 17:29:44 -07:00
Ishaan Jaff
50461eb22c feat - create budgets when team/member_add 2024-05-22 17:16:19 -07:00
Krrish Dholakia
9f518030b9 build(schema.prisma): add completion_start_time to spend logs
enables time to first token tracking for models
2024-05-22 16:43:08 -07:00
Krrish Dholakia
db666b01e5 feat(proxy_server.py): add CRUD endpoints for 'end_user' management
allow admin to specify region + default models for end users
2024-05-08 18:50:36 -07:00
Ishaan Jaff
f2849d0641 fix - track litellm_model_name in LiteLLM_ErrorLogs 2024-04-30 17:31:40 -07:00
Ishaan Jaff
4b8fda4ac4 log startTime and EndTime for exceptions 2024-04-30 13:34:14 -07:00
Ishaan Jaff
ad5fddef15 fix log model_group 2024-04-30 13:11:09 -07:00
Ishaan Jaff
ee2a2ce559 fix - log api_base in errors 2024-04-30 13:02:42 -07:00
Ishaan Jaff
c7f979e0fe fix schema error logs 2024-04-30 12:31:19 -07:00
Ishaan Jaff
d6f7fa7f4e v0 prisma schema 2024-04-30 11:42:17 -07:00