Emilio Garcia ba50790a28 feat(tests): metrics tests (#3966) ... # What does this PR do? 1. Make telemetry tests as easy as possible for users by expanding the `SpanStub` data class and creating the `MetricStub` dataclass as a way to consistently marshal telemetry data in test fixtures and unmarshal and handle it in tests. 2. Structure server and client tests to always follow the same standards for consistent testing experience by using the `SpanStub` and `MetricStub` data class objects. 3. Enable Metrics Testing for completions endpoint 4. Correct token metrics to use histograms instead of counts to capture tokens per request rather than a cumulative count of tokens over the lifecycle of the server. ## Test Plan These are tests		2025-11-05 10:26:15 -08:00
..
02c93bb3c314427bae2b7a7a6f054792b9f22d2cb4522eab802810be8672d3dc.json	fix(models)!: always prefix models with provider_id when registering (#3822)	2025-10-16 06:47:39 -07:00
4df315784095a200b2d275c6f8dda8be845e250000208127d20cf8c4f0bc666c.json	fix(tests): ensure test isolation in server mode (#3737)	2025-10-08 12:03:36 -07:00
5b03940f8f14616ba20bf3b695138b785ffc26aed814ef01db492f4a5674d6c5.json	fix(tests): ensure test isolation in server mode (#3737)	2025-10-08 12:03:36 -07:00
41c28019c2c89e5962ae3043dc7015ee45aa5ee235645768a230a5fa5cd45ad9.json	fix(tests): ensure test isolation in server mode (#3737)	2025-10-08 12:03:36 -07:00
224f7e7bd332f0ada326039866d13b7f167d5bfa91ce752022010f1e885d869f.json	feat(tests): make inference_recorder into api_recorder (include tool_invoke) (#3403)	2025-10-09 14:27:51 -07:00
395c30078677826058a0cbe136dfd07c816854cfb7015ee4ece0e414d16e7e52.json	fix(tests): ensure test isolation in server mode (#3737)	2025-10-08 12:03:36 -07:00
1098240ef53bbd378adf8dafbd5838b16eef7d6a7d6e75d24e3c120e25e73750.json	fix(tests): ensure test isolation in server mode (#3737)	2025-10-08 12:03:36 -07:00
a6ad8748dce1ebe53352c6ac4ccd9b209d614ce5c6ff86992b4aed3dc344eafc.json	feat(tests): make inference_recorder into api_recorder (include tool_invoke) (#3403)	2025-10-09 14:27:51 -07:00
a4416482053457914b5834398c2664ceb843d8c7deaec80a59d5e20dbb1ca090.json	feat(tests): make inference_recorder into api_recorder (include tool_invoke) (#3403)	2025-10-09 14:27:51 -07:00
ab1a32474062bbad640ce43d02d6b61ed9f174c225597f0241cf120c47c7d2fa.json	test(telemetry): Telemetry Tests (#3805)	2025-10-17 10:43:33 -07:00
b2c646582d0a4d9d8986789261c0d630d5b604ee6291cf8aa3d44ab761f2c676.json	fix(tests): ensure test isolation in server mode (#3737)	2025-10-08 12:03:36 -07:00
bfc8818f4ad237ba6c9649d47eaff8946e334ea6a2bcb564d74f4f14dbc3497b.json	fix(tests): ensure test isolation in server mode (#3737)	2025-10-08 12:03:36 -07:00
c4f314b202711805808eb75f1947cb6cca0bf8dbffb0dfabb814f9da0083b3c3.json	fix(tests): ensure test isolation in server mode (#3737)	2025-10-08 12:03:36 -07:00
c34cccb2af2fb9f02f7136b0dd350e75e7d2a77d222ef26a9bc419e10fa33c56.json	fix(tests): ensure test isolation in server mode (#3737)	2025-10-08 12:03:36 -07:00
models-64a2277c90f0f42576f60c1030e3a020403d34a95f56931b792d5939f4cebc57-6a62c598.json	fix(responses): fix subtle bugs in non-function tool calling (#3817)	2025-10-15 13:57:37 -07:00
models-64a2277c90f0f42576f60c1030e3a020403d34a95f56931b792d5939f4cebc57-826d44c3.json	feat(tests): metrics tests (#3966)	2025-11-05 10:26:15 -08:00
models-64a2277c90f0f42576f60c1030e3a020403d34a95f56931b792d5939f4cebc57-e660ee4a.json	feat: Add /v1/embeddings endpoint to batches API (#3384)	2025-10-10 13:25:58 -07:00