phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-08-06 02:32:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

add cerebras and ollama

Browse source
This commit is contained in:
Sixian Yi 2025-01-21 22:59:49 -08:00
parent 35a00d004a
commit 7ec2d955ee
6 changed files with 159 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

45
llama_stack/templates/cerebras/report.md Normal file
View file

@ -0,0 +1,45 @@
# Report for cerebras distribution
## Supported Models:
| Model Descriptor | cerebras |
|:---|:---|
| Llama-3-8B-Instruct | ❌ |
| Llama-3-70B-Instruct | ❌ |
| Llama3.1-8B-Instruct | ✅ |
| Llama3.1-70B-Instruct | ❌ |
| Llama3.1-405B-Instruct | ❌ |
| Llama3.2-1B-Instruct | ❌ |
| Llama3.2-3B-Instruct | ❌ |
| Llama3.2-11B-Vision-Instruct | ❌ |
| Llama3.2-90B-Vision-Instruct | ❌ |
| Llama3.3-70B-Instruct | ✅ |
| Llama-Guard-3-11B-Vision | ❌ |
| Llama-Guard-3-1B | ❌ |
| Llama-Guard-3-8B | ❌ |
| Llama-Guard-2-8B | ❌ |
## Inference:
| Model | API | Capability | Test | Status |
|:----- |:-----|:-----|:-----|:-----|
| Text | /chat_completion | streaming | test_text_chat_completion_streaming | ✅ |
| Vision | /chat_completion | streaming | test_image_chat_completion_streaming | ⏭️ |
| Vision | /chat_completion | non_streaming | test_image_chat_completion_non_streaming | ⏭️ |
| Text | /chat_completion | non_streaming | test_text_chat_completion_non_streaming | ✅ |
| Text | /chat_completion | tool_calling | test_text_chat_completion_with_tool_calling_and_streaming | ✅ |
| Text | /chat_completion | tool_calling | test_text_chat_completion_with_tool_calling_and_non_streaming | ✅ |
| Text | /completion | streaming | test_text_completion_streaming | ✅ |
| Text | /completion | non_streaming | test_text_completion_non_streaming | ✅ |
| Text | /completion | structured_output | test_text_completion_structured_output | ❌ |
## Memory:
| API | Capability | Test | Status |
|:-----|:-----|:-----|:-----|
| /insert, /query | inline | test_memory_bank_insert_inline_and_query | ✅ |
| /insert, /query | url | test_memory_bank_insert_from_url_and_query | ✅ |
## Agents:
| API | Capability | Test | Status |
|:-----|:-----|:-----|:-----|
| create_agent_turn | rag | test_rag_agent | ✅ |
| create_agent_turn | custom_tool | test_custom_tool | ✅ |
| create_agent_turn | code_execution | test_code_execution | ✅ |

6
llama_stack/templates/fireworks/report.md
View file

@ -34,12 +34,12 @@
## Memory:
| API | Capability | Test | Status |
|:-----|:-----|:-----|:-----|
| /insert, /query | inline | test_memory_bank_insert_inline_and_query | ❌ |
| /insert, /query | url | test_memory_bank_insert_from_url_and_query | ❌ |
| /insert, /query | inline | test_memory_bank_insert_inline_and_query | ✅ |
| /insert, /query | url | test_memory_bank_insert_from_url_and_query | ✅ |
## Agents:
| API | Capability | Test | Status |
|:-----|:-----|:-----|:-----|
| create_agent_turn | rag | test_rag_agent | ❌ |
| create_agent_turn | custom_tool | test_custom_tool | ✅ |
| create_agent_turn | custom_tool | test_custom_tool | ❌ |
| create_agent_turn | code_execution | test_code_execution | ❌ |

45
llama_stack/templates/ollama/report.md Normal file
View file

@ -0,0 +1,45 @@
# Report for ollama distribution
## Supported Models:
| Model Descriptor | ollama |
|:---|:---|
| Llama-3-8B-Instruct | ❌ |
| Llama-3-70B-Instruct | ❌ |
| Llama3.1-8B-Instruct | ✅ |
| Llama3.1-70B-Instruct | ✅ |
| Llama3.1-405B-Instruct | ✅ |
| Llama3.2-1B-Instruct | ✅ |
| Llama3.2-3B-Instruct | ✅ |
| Llama3.2-11B-Vision-Instruct | ✅ |
| Llama3.2-90B-Vision-Instruct | ✅ |
| Llama3.3-70B-Instruct | ✅ |
| Llama-Guard-3-11B-Vision | ❌ |
| Llama-Guard-3-1B | ✅ |
| Llama-Guard-3-8B | ✅ |
| Llama-Guard-2-8B | ❌ |
## Inference:
| Model | API | Capability | Test | Status |
|:----- |:-----|:-----|:-----|:-----|
| Text | /chat_completion | streaming | test_text_chat_completion_streaming | ✅ |
| Vision | /chat_completion | streaming | test_image_chat_completion_streaming | ⏭️ |
| Vision | /chat_completion | non_streaming | test_image_chat_completion_non_streaming | ⏭️ |
| Text | /chat_completion | non_streaming | test_text_chat_completion_non_streaming | ✅ |
| Text | /chat_completion | tool_calling | test_text_chat_completion_with_tool_calling_and_streaming | ✅ |
| Text | /chat_completion | tool_calling | test_text_chat_completion_with_tool_calling_and_non_streaming | ✅ |
| Text | /completion | streaming | test_text_completion_streaming | ✅ |
| Text | /completion | non_streaming | test_text_completion_non_streaming | ✅ |
| Text | /completion | structured_output | test_text_completion_structured_output | ✅ |
## Memory:
| API | Capability | Test | Status |
|:-----|:-----|:-----|:-----|
| /insert, /query | inline | test_memory_bank_insert_inline_and_query | ✅ |
| /insert, /query | url | test_memory_bank_insert_from_url_and_query | ✅ |
## Agents:
| API | Capability | Test | Status |
|:-----|:-----|:-----|:-----|
| create_agent_turn | rag | test_rag_agent | ✅ |
| create_agent_turn | custom_tool | test_custom_tool | ❌ |
| create_agent_turn | code_execution | test_code_execution | ✅ |

45
llama_stack/templates/tgi/report.md Normal file
View file

@ -0,0 +1,45 @@
# Report for tgi distribution
## Supported Models:
| Model Descriptor | tgi |
|:---|:---|
| Llama-3-8B-Instruct | ✅ |
| Llama-3-70B-Instruct | ✅ |
| Llama3.1-8B-Instruct | ✅ |
| Llama3.1-70B-Instruct | ✅ |
| Llama3.1-405B-Instruct | ✅ |
| Llama3.2-1B-Instruct | ✅ |
| Llama3.2-3B-Instruct | ✅ |
| Llama3.2-11B-Vision-Instruct | ✅ |
| Llama3.2-90B-Vision-Instruct | ✅ |
| Llama3.3-70B-Instruct | ✅ |
| Llama-Guard-3-11B-Vision | ✅ |
| Llama-Guard-3-1B | ✅ |
| Llama-Guard-3-8B | ✅ |
| Llama-Guard-2-8B | ✅ |
## Inference:
| Model | API | Capability | Test | Status |
|:----- |:-----|:-----|:-----|:-----|
| Text | /chat_completion | streaming | test_text_chat_completion_streaming | ✅ |
| Vision | /chat_completion | streaming | test_image_chat_completion_streaming | ⏭️ |
| Vision | /chat_completion | non_streaming | test_image_chat_completion_non_streaming | ⏭️ |
| Text | /chat_completion | non_streaming | test_text_chat_completion_non_streaming | ✅ |
| Text | /chat_completion | tool_calling | test_text_chat_completion_with_tool_calling_and_streaming | ✅ |
| Text | /chat_completion | tool_calling | test_text_chat_completion_with_tool_calling_and_non_streaming | ✅ |
| Text | /completion | streaming | test_text_completion_streaming | ✅ |
| Text | /completion | non_streaming | test_text_completion_non_streaming | ✅ |
| Text | /completion | structured_output | test_text_completion_structured_output | ✅ |
## Memory:
| API | Capability | Test | Status |
|:-----|:-----|:-----|:-----|
| /insert, /query | inline | test_memory_bank_insert_inline_and_query | ✅ |
| /insert, /query | url | test_memory_bank_insert_from_url_and_query | ✅ |
## Agents:
| API | Capability | Test | Status |
|:-----|:-----|:-----|:-----|
| create_agent_turn | rag | test_rag_agent | ✅ |
| create_agent_turn | custom_tool | test_custom_tool | ❌ |
| create_agent_turn | code_execution | test_code_execution | ✅ |

1
tests/client-sdk/conftest.py
View file

@ -32,6 +32,7 @@ def pytest_addoption(parser):
TEXT_MODEL = "meta-llama/Llama-3.1-8B-Instruct"
INFERENCE_MODEL = "meta-llama/Llama-3.2-11B-Vision-Instruct"
@pytest.fixture(scope="session")
def provider_data():
# check env for tavily secret, brave secret and inject all into provider data

20
tests/client-sdk/report.py
View file

@ -68,6 +68,26 @@ SUPPORTED_MODELS = {
CoreModelId.llama_guard_3_11b_vision.value,
]
),
"tgi": set(
[
model.core_model_id.value
for model in all_registered_models()
if model.huggingface_repo
]
),
"vllm": set(
[
model.core_model_id.value
for model in all_registered_models()
if model.huggingface_repo
]
),
"cerebras": set(
[
CoreModelId.llama3_1_8b_instruct.value,
CoreModelId.llama3_3_70b_instruct.value,
]
),
}