phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-20 04:58:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

pushed docker image, updated documentation

Browse source
This commit is contained in:
Jinan Zhou 2024-11-22 00:22:46 +00:00
parent cb82b1ee9e
commit e1c6a2c61c
4 changed files with 55 additions and 29 deletions
Download patch file Download diff file Expand all files Collapse all files

81
llama_stack/templates/nutanix/doc_template.md
View file

@ -1,40 +1,63 @@
# Nutanix Distribution
The `llamastack/distribution-nutanix` distribution consists of the following provider configurations.
```{toctree}
:maxdepth: 2
:hidden:
| **API** | **Inference** | **Agents** | **Memory** | **Safety** | **Telemetry** |
|----------------- |--------------- |---------------- |-------------------------------------------------- |---------------- |---------------- |
| **Provider(s)** | remote::nutanix | meta-reference | meta-reference | meta-reference | meta-reference |
### Start the Distribution (Hosted remote)
> [!NOTE]
> This assumes you have an hosted Nutanix AI endpoint and an API Key.
1. Clone the repo
```
git clone git@github.com:meta-llama/llama-stack.git
cd llama-stack
self
```
2. Config the model name
The `llamastack/distribution-{{ name }}` distribution consists of the following provider configurations.
Please adjust the `NUTANIX_SUPPORTED_MODELS` variable at line 29 in `llama_stack/providers/adapters/inference/nutanix/nutanix.py` according to your deployment.
{{ providers_table }}
3. Build the distrbution
{% if run_config_env_vars %}
### Environment Variables
The following environment variables can be configured:
{% for var, (default_value, description) in run_config_env_vars.items() %}
- `{{ var }}`: {{ description }} (default: `{{ default_value }}`)
{% endfor %}
{% endif %}
{% if default_models %}
### Models
The following models are available by default:
{% for model in default_models %}
- `{{ model.model_id }} ({{ model.provider_model_id }})`
{% endfor %}
{% endif %}
### Prerequisite: API Keys
Make sure you have a Nutanix AI Endpoint deployed and a API key.
## Running Llama Stack with Nutanix
You can do this via Conda (build code) or Docker.
### Via Docker
```bash
llama stack build --template nutanix --image-type docker
LLAMA_STACK_PORT=1740
llama stack run nutanix \
--port $LLAMA_STACK_PORT \
--env NUTANIX_API_KEY=$NUTANIX_API_KEY
```
pip install -e .
### Via Conda
```bash
llama stack build --template nutanix --image-type conda
```
4. Edit the yaml file
```
vim
```
5. Serve and enjoy!
```
llama stack run ntnx --port 174
LLAMA_STACK_PORT=1740
llama stack run ./run.yaml \
--port $LLAMA_STACK_PORT \
--env NUTANIX_API_KEY=$NUTANIX_API_KEY
```