phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

Merge pull request #3897 from BerriAI/litellm_fix_latency_cal

Browse source 
ui  - fix latency analytics on `completion_tokens`
This commit is contained in:
Ishaan Jaff 2024-05-29 17:32:17 -07:00 • committed by GitHub
commit ea30769190
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
litellm/proxy/proxy_server.py
View file

@ -10209,7 +10209,7 @@ async def model_metrics(
model_group,
model,
DATE_TRUNC('day', "startTime")::DATE AS day,
AVG(EXTRACT(epoch FROM ("endTime" - "startTime"))) / SUM(total_tokens) AS avg_latency_per_token
AVG(EXTRACT(epoch FROM ("endTime" - "startTime")) / "completion_tokens") AS avg_latency_per_token
FROM
"LiteLLM_SpendLogs"
WHERE
@ -10221,7 +10221,7 @@ async def model_metrics(
model,
day
HAVING
SUM(total_tokens) > 0
SUM(completion_tokens) > 0
ORDER BY
avg_latency_per_token DESC;
"""