From dfd646149843385a2aa4adc53ac5e386b5405f66 Mon Sep 17 00:00:00 2001 From: Ashwin Bharambe Date: Sun, 2 Feb 2025 06:49:01 -0800 Subject: [PATCH] kill old readme --- distributions/together/README.md | 65 -------------------------------- 1 file changed, 65 deletions(-) delete mode 100644 distributions/together/README.md diff --git a/distributions/together/README.md b/distributions/together/README.md deleted file mode 100644 index 72d02437a..000000000 --- a/distributions/together/README.md +++ /dev/null @@ -1,65 +0,0 @@ -# Together Distribution - -### Connect to a Llama Stack Together Endpoint -- You may connect to a hosted endpoint `https://llama-stack.together.ai`, serving a Llama Stack distribution - -The `llamastack/distribution-together` distribution consists of the following provider configurations. - - -| **API** | **Inference** | **Agents** | **Memory** | **Safety** | **Telemetry** | -|----------------- |--------------- |---------------- |-------------------------------------------------- |---------------- |---------------- | -| **Provider(s)** | remote::together | meta-reference | meta-reference, remote::weaviate | meta-reference | meta-reference | - - -### Docker: Start the Distribution (Single Node CPU) - -> [!NOTE] -> This assumes you have an hosted endpoint at Together with API Key. - -``` -$ cd distributions/together -$ ls -compose.yaml run.yaml -$ docker compose up -``` - -Make sure in you `run.yaml` file, you inference provider is pointing to the correct Together URL server endpoint. E.g. -``` -inference: - - provider_id: together - provider_type: remote::together - config: - url: https://api.together.xyz/v1 - api_key: -``` - -### Conda llama stack run (Single Node CPU) - -```bash -llama stack build --template together --image-type conda -# -- modify run.yaml to a valid Together server endpoint -llama stack run ./run.yaml -``` - -### (Optional) Update Model Serving Configuration - -Use `llama-stack-client models list` to check the available models served by together. - -``` -$ llama-stack-client models list -+------------------------------+------------------------------+---------------+------------+ -| identifier | llama_model | provider_id | metadata | -+==============================+==============================+===============+============+ -| Llama3.1-8B-Instruct | Llama3.1-8B-Instruct | together0 | {} | -+------------------------------+------------------------------+---------------+------------+ -| Llama3.1-70B-Instruct | Llama3.1-70B-Instruct | together0 | {} | -+------------------------------+------------------------------+---------------+------------+ -| Llama3.1-405B-Instruct | Llama3.1-405B-Instruct | together0 | {} | -+------------------------------+------------------------------+---------------+------------+ -| Llama3.2-3B-Instruct | Llama3.2-3B-Instruct | together0 | {} | -+------------------------------+------------------------------+---------------+------------+ -| Llama3.2-11B-Vision-Instruct | Llama3.2-11B-Vision-Instruct | together0 | {} | -+------------------------------+------------------------------+---------------+------------+ -| Llama3.2-90B-Vision-Instruct | Llama3.2-90B-Vision-Instruct | together0 | {} | -+------------------------------+------------------------------+---------------+------------+ -```