phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions

docs

Browse source
This commit is contained in:
Xi Yan 2024-11-09 08:57:51 -08:00
parent 4986e46188
commit 0c14761453
1 changed files with 36 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

36
docs/source/getting_started/index.md
View file

@ -255,6 +255,18 @@ $ llama stack run ~/.llama/distributions/llamastack-meta-reference-gpu/meta-refe
llama stack build --template tgi --image-type conda
```
Note: If you wish to use pgvector or chromadb as memory provider. You may need to update generated `run.yaml` file to point to the desired memory provider. See [Memory Providers](https://llama-stack.readthedocs.io/en/latest/api_providers/memory_api.html) for more details. Or comment out the pgvector or chromadb memory provider in `run.yaml` file to use the default inline memory provider, keeping only the following section:
```
memory:
- provider_id: faiss-0
provider_type: faiss
config:
kvstore:
namespace: null
type: sqlite
db_path: ~/.llama/runtime/faiss_store.db
```
3. Start a TGI server endpoint
4. Make sure in your `run.yaml` file, your `conda_env` is pointing to the conda environment and inference provider is pointing to the correct TGI server endpoint. E.g.
@ -272,6 +284,18 @@ inference:
```bash
$ llama stack run ~/.llama/distributions/llamastack-tgi/tgi-run.yaml
```
Note: If you wish to use pgvector or chromadb as memory provider. You may need to update generated `run.yaml` file to point to the desired memory provider. See [Memory Providers](https://llama-stack.readthedocs.io/en/latest/api_providers/memory_api.html) for more details. Or comment out the pgvector or chromadb memory provider in `run.yaml` file to use the default inline memory provider, keeping only the following section:
```
memory:
- provider_id: faiss-0
provider_type: faiss
config:
kvstore:
namespace: null
type: sqlite
db_path: ~/.llama/runtime/faiss_store.db
```
:::
:::{tab-item} ollama
@ -309,6 +333,18 @@ llama stack build --template ollama --image-type conda
llama stack run ~/.llama/distributions/llamastack-ollama/ollama-run.yaml
```
Note: If you wish to use pgvector or chromadb as memory provider. You may need to update generated `run.yaml` file to point to the desired memory provider. See [Memory Providers](https://llama-stack.readthedocs.io/en/latest/api_providers/memory_api.html) for more details. Or comment out the pgvector or chromadb memory provider in `run.yaml` file to use the default inline memory provider, keeping only the following section:
```
memory:
- provider_id: faiss-0
provider_type: faiss
config:
kvstore:
namespace: null
type: sqlite
db_path: ~/.llama/runtime/faiss_store.db
```
:::
:::{tab-item} fireworks