From 9ddc28eca7cf6e093357a0912ef931f58844b82d Mon Sep 17 00:00:00 2001 From: Ashwin Bharambe Date: Tue, 29 Oct 2024 12:39:03 -0700 Subject: [PATCH] updates --- docs/source/getting_started.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/source/getting_started.md b/docs/source/getting_started.md index 7bcdf71e7..3725c540f 100644 --- a/docs/source/getting_started.md +++ b/docs/source/getting_started.md @@ -41,6 +41,17 @@ Both of these provide options to run model inference using our reference impleme ### Docker +Running inference of the underlying Llama model is one of the most critical requirements. Depending on what hardware you have available, you have various options: + +**Do you have access to a machine with powerful GPUs?** +If so, we suggest... + +**Are you running on a "regular" desktop machine?** +In that case, we suggest ollama + +**Do you have access to a remote inference provider like Fireworks, Togther, etc.?** +... + We provide pre-built Docker image of Llama Stack distribution, which can be found in the following links in the [distributions](../distributions/) folder. > [!NOTE]