Rashmi Pawar c169c164b3 fix: NVIDIA embedding results in InternalServerError (#1851) ... Closes #1819 ## Test Plan ```bash pytest -v tests/integration/inference/test_embedding.py --stack-config=http://localhost:5002 --embedding-model=nvidia/llama-3.2-nv-embedqa-1b-v2 =============================================================================== test session starts ================================================================================ platform linux -- Python 3.10.0, pytest-8.3.5, pluggy-1.5.0 -- /home/ubuntu/miniconda/envs/nvidia-1/bin/python cachedir: .pytest_cache rootdir: /home/ubuntu/llama-stack configfile: pyproject.toml plugins: anyio-4.9.0 collected 23 items tests/integration/inference/test_embedding.py::test_embedding_text[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-list[string]] PASSED [ 4%] tests/integration/inference/test_embedding.py::test_embedding_text[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-list[text]] PASSED [ 8%] tests/integration/inference/test_embedding.py::test_embedding_image[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-list[url,base64]] XFAIL (nvidia/llama-3.2-nv-embedqa-1b-v2 doe...) [ 13%] tests/integration/inference/test_embedding.py::test_embedding_image[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-list[url,string,base64,text]] XFAIL (nvidia/llama-3.2-nv-embed...) [ 17%] tests/integration/inference/test_embedding.py::test_embedding_truncation[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-long-end] PASSED [ 21%] tests/integration/inference/test_embedding.py::test_embedding_truncation[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-long-start] PASSED [ 26%] tests/integration/inference/test_embedding.py::test_embedding_truncation[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-short-end] PASSED [ 30%] tests/integration/inference/test_embedding.py::test_embedding_truncation[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-short-start] PASSED [ 34%] tests/integration/inference/test_embedding.py::test_embedding_truncation_error[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-long-text-None] PASSED [ 39%] tests/integration/inference/test_embedding.py::test_embedding_truncation_error[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-long-text-none] PASSED [ 43%] tests/integration/inference/test_embedding.py::test_embedding_truncation_error[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-long-str-None] PASSED [ 47%] tests/integration/inference/test_embedding.py::test_embedding_truncation_error[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-long-str-none] PASSED [ 52%] tests/integration/inference/test_embedding.py::test_embedding_output_dimension[emb=nvidia/llama-3.2-nv-embedqa-1b-v2] PASSED [ 56%] tests/integration/inference/test_embedding.py::test_embedding_task_type[emb=nvidia/llama-3.2-nv-embedqa-1b-v2] PASSED [ 60%] tests/integration/inference/test_embedding.py::test_embedding_text_truncation[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-None] PASSED [ 65%] tests/integration/inference/test_embedding.py::test_embedding_text_truncation[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-none] PASSED [ 69%] tests/integration/inference/test_embedding.py::test_embedding_text_truncation[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-end] PASSED [ 73%] tests/integration/inference/test_embedding.py::test_embedding_text_truncation[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-start] PASSED [ 78%] tests/integration/inference/test_embedding.py::test_embedding_text_truncation_error[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-NONE] PASSED [ 82%] tests/integration/inference/test_embedding.py::test_embedding_text_truncation_error[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-END] PASSED [ 86%] tests/integration/inference/test_embedding.py::test_embedding_text_truncation_error[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-START] PASSED [ 91%] tests/integration/inference/test_embedding.py::test_embedding_text_truncation_error[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-left] PASSED [ 95%] tests/integration/inference/test_embedding.py::test_embedding_text_truncation_error[emb=nvidia/llama-3.2-nv-embedqa-1b-v2-right] PASSED [100%] ===================================================================== 21 passed, 2 xfailed, 1 warning in 7.18s ===================================================================== ``` [//]: # (## Documentation) cc: @dglogo @mattf @sumitb		2025-04-01 13:31:29 +02:00
..
anthropic	feat(providers): Groq now uses LiteLLM openai-compat (#1303)	2025-02-27 13:16:50 -08:00
bedrock	fix: solve ruff B008 warnings (#1444)	2025-03-06 16:48:35 -08:00
cerebras	fix: solve ruff B008 warnings (#1444)	2025-03-06 16:48:35 -08:00
databricks	test: add unit test to ensure all config types are instantiable (#1601)	2025-03-12 22:29:58 -07:00
fireworks	fix: remove Llama-3.2-1B-Instruct for fireworks (#1558)	2025-03-11 11:19:29 -07:00
gemini	feat(providers): Groq now uses LiteLLM openai-compat (#1303)	2025-02-27 13:16:50 -08:00
groq	fix: register provider model name and HF alias in run.yaml (#1304)	2025-02-27 16:39:23 -08:00
nvidia	fix: NVIDIA embedding results in InternalServerError (#1851)	2025-04-01 13:31:29 +02:00
ollama	feat(logging): implement category-based logging (#1362)	2025-03-07 11:34:30 -08:00
openai	feat(providers): Groq now uses LiteLLM openai-compat (#1303)	2025-02-27 13:16:50 -08:00
passthrough	fix: passthrough impl response.content.text (#1665)	2025-03-17 13:42:08 -07:00
runpod	test: add unit test to ensure all config types are instantiable (#1601)	2025-03-12 22:29:58 -07:00
sambanova	fix: Updating ToolCall.arguments to allow for json strings that can be decoded on client side (#1685)	2025-03-19 10:36:19 -07:00
tgi	fix: solve ruff B008 warnings (#1444)	2025-03-06 16:48:35 -08:00
together	feat: Add open benchmark template codegen (#1579)	2025-03-12 11:12:08 -07:00
vllm	fix: Updating ToolCall.arguments to allow for json strings that can be decoded on client side (#1685)	2025-03-19 10:36:19 -07:00
__init__.py	impls -> inline, adapters -> remote (#381)	2024-11-06 14:54:05 -08:00