mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-08-03 09:21:45 +00:00
added tabs for the tutorial output and rephrased thing based on feedback
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
This commit is contained in:
Francisco Javier Arceo
parent
0961987962
commit
5bdd767e8d
1 changed files with 166 additions and 148 deletions
|
|
@ -45,32 +45,33 @@ Setup your virtual environment.
|
|||
uv venv --python 3.10
|
||||
source .venv/bin/activate
|
||||
```
|
||||
## Step 2: Install Llama Stack
|
||||
## Step 2: Run Llama Stack
|
||||
Llama Stack is a server that exposes multiple APIs, you connect with it using the Llama Stack client SDK.
|
||||
|
||||
```bash
|
||||
uv pip install llama-stack
|
||||
```
|
||||
Note the Llama Stack Server includes the client SDK as well.
|
||||
|
||||
## Step 3: Build and Run the Llama Stack Server
|
||||
Llama Stack uses a [configuration file](../distributions/configuration.md) to define the stack.
|
||||
The config file is a YAML file that specifies the providers and their configurations. Now let's build and run the
|
||||
Llama Stack config for Ollama.
|
||||
::::{tab-set}
|
||||
|
||||
:::{tab-item} Using Python
|
||||
You can use Python to build and run the Llama Stack server. This is useful for testing and development purposes.
|
||||
:::{tab-item} Using `venv`
|
||||
You can use Python to build and run the Llama Stack server, which is useful for testing and development.
|
||||
|
||||
Llama Stack uses a [YAML configuration file](../distributions/configuration.md) to specify the stack setup,
|
||||
which defines the providers and their settings.
|
||||
Now let's build and run the Llama Stack config for Ollama.
|
||||
|
||||
```bash
|
||||
INFERENCE_MODEL=llama3.2:3b llama stack build --template ollama --image-type venv --run
|
||||
```
|
||||
You will see output like below:
|
||||
```
|
||||
INFO: Application startup complete.
|
||||
INFO: Uvicorn running on http://['::', '0.0.0.0']:8321 (Press CTRL+C to quit)
|
||||
:::
|
||||
:::{tab-item} Using `conda`
|
||||
You can use Python to build and run the Llama Stack server, which is useful for testing and development.
|
||||
|
||||
Llama Stack uses a [YAML configuration file](../distributions/configuration.md) to specify the stack setup,
|
||||
which defines the providers and their settings.
|
||||
Now let's build and run the Llama Stack config for Ollama.
|
||||
|
||||
```bash
|
||||
INFERENCE_MODEL=llama3.2:3b llama stack build --template ollama --image-type conda --run
|
||||
```
|
||||
:::
|
||||
|
||||
:::{tab-item} Using a Container
|
||||
You can use a container image to run the Llama Stack server. We provide several container images for the server
|
||||
component that works with different inference providers out of the box. For this guide, we will use
|
||||
|
|
@ -101,8 +102,7 @@ with `host.containers.internal`.
|
|||
|
||||
The configuration YAML for the Ollama distribution is available at `distributions/ollama/run.yaml`.
|
||||
|
||||
```{admonition} Note
|
||||
:class: note
|
||||
```{tip}
|
||||
|
||||
Docker containers run in their own isolated network namespaces on Linux. To allow the container to communicate with services running on the host via `localhost`, you need `--network=host`. This makes the container use the host’s network directly so it can connect to Ollama running on `localhost:11434`.
|
||||
|
||||
|
|
@ -120,27 +120,32 @@ docker run -it \
|
|||
```
|
||||
:::
|
||||
::::
|
||||
You will see output like below:
|
||||
```
|
||||
INFO: Application startup complete.
|
||||
INFO: Uvicorn running on http://['::', '0.0.0.0']:8321 (Press CTRL+C to quit)
|
||||
```
|
||||
|
||||
Now you can use the Llama Stack client to run inference and build agents!
|
||||
|
||||
You can reuse the server setup or use the [Llama Stack Client](https://github.com/meta-llama/llama-stack-client-python/).
|
||||
Note that the client package is already included in the `llama-stack` package.
|
||||
|
||||
### ii. Using the Llama Stack Client
|
||||
## Step 3: Run Client CLI
|
||||
|
||||
Open a new terminal and navigate to the same directory you started the server from. Then set up a new or activate your
|
||||
existing server virtual environment.
|
||||
|
||||
::::{tab-set}
|
||||
|
||||
:::{tab-item} Reuse the Server Setup
|
||||
:::{tab-item} Reuse Server `venv`
|
||||
```bash
|
||||
# As mentioned, the client is included in the llama-stack package so we can just activate the server virtual environment
|
||||
# The client is included in the llama-stack package so we just activate the server venv
|
||||
source .venv/bin/activate
|
||||
```
|
||||
:::
|
||||
|
||||
:::{tab-item} Install the Llama Stack Client (venv)
|
||||
:::{tab-item} Install with `venv`
|
||||
```bash
|
||||
uv venv client --python 3.10
|
||||
source client/bin/activate
|
||||
|
|
@ -148,7 +153,7 @@ pip install llama-stack-client
|
|||
```
|
||||
:::
|
||||
|
||||
:::{tab-item} Install the Llama Stack Client (conda)
|
||||
:::{tab-item} Install with `conda`
|
||||
```bash
|
||||
yes | conda create -n stack-client python=3.10
|
||||
conda activate stack-client
|
||||
|
|
@ -157,7 +162,8 @@ pip install llama-stack-client
|
|||
:::
|
||||
::::
|
||||
|
||||
Now let's use the `llama-stack-client` CLI to check the connectivity to the server.
|
||||
Now let's use the `llama-stack-client` [CLI](../references/llama_stack_client_cli_reference.md) to check the
|
||||
connectivity to the server.
|
||||
|
||||
```bash
|
||||
llama-stack-client configure --endpoint http://localhost:8321 --api-key none
|
||||
|
|
@ -185,7 +191,14 @@ Total models: 2
|
|||
|
||||
```
|
||||
|
||||
## Step 4: Run Basic Inference
|
||||
## Step 4: Run the Demos
|
||||
|
||||
Note that these demos show the [Python Client SDK](../references/python_sdk_reference/index.md).
|
||||
Other SDKs are also available, please refer to the [Client SDK](../index.md#client-sdks) list for the complete options.
|
||||
|
||||
::::{tab-set}
|
||||
|
||||
:::{tab-item} Basic Inference with the CLI
|
||||
You can test basic Llama inference completion using the CLI.
|
||||
|
||||
```bash
|
||||
|
|
@ -208,6 +221,9 @@ ChatCompletionResponse(
|
|||
],
|
||||
)
|
||||
```
|
||||
:::
|
||||
|
||||
:::{tab-item} Basic Inference with a Script
|
||||
Alternatively, you can run inference using the Llama Stack client SDK.
|
||||
|
||||
### i. Create the Script
|
||||
|
|
@ -250,8 +266,9 @@ Lines of code unfold
|
|||
Logic flows through digital night
|
||||
Beauty in the bits
|
||||
```
|
||||
:::
|
||||
|
||||
## Step 5: Build a Simple Agent
|
||||
:::{tab-item} Build a Simple Agent
|
||||
Now we can move beyond simple inference and build an agent that can perform tasks using the Llama Stack server.
|
||||
### i. Create the Script
|
||||
Create a file `agent.py` and add the following code:
|
||||
|
|
@ -299,129 +316,129 @@ Let's run the script using `uv`
|
|||
```bash
|
||||
uv run python agent.py
|
||||
```
|
||||
:::{dropdown} `👋 Click here to see the sample output`
|
||||
```
|
||||
Non-streaming ...
|
||||
agent> I'm an artificial intelligence designed to assist and communicate with users like you. I don't have a personal identity, but I'm here to provide information, answer questions, and help with tasks to the best of my abilities.
|
||||
|
||||
I can be used for a wide range of purposes, such as:
|
||||
```{dropdown} 👋 Click here to see the sample output
|
||||
Non-streaming ...
|
||||
agent> I'm an artificial intelligence designed to assist and communicate with users like you. I don't have a personal identity, but I'm here to provide information, answer questions, and help with tasks to the best of my abilities.
|
||||
|
||||
* Providing definitions and explanations
|
||||
* Offering suggestions and ideas
|
||||
* Helping with language translation
|
||||
* Assisting with writing and proofreading
|
||||
* Generating text or responses to questions
|
||||
* Playing simple games or chatting about topics of interest
|
||||
I can be used for a wide range of purposes, such as:
|
||||
|
||||
I'm constantly learning and improving my abilities, so feel free to ask me anything, and I'll do my best to help!
|
||||
* Providing definitions and explanations
|
||||
* Offering suggestions and ideas
|
||||
* Helping with language translation
|
||||
* Assisting with writing and proofreading
|
||||
* Generating text or responses to questions
|
||||
* Playing simple games or chatting about topics of interest
|
||||
|
||||
Streaming ...
|
||||
AgentTurnResponseStreamChunk(
|
||||
│ event=TurnResponseEvent(
|
||||
│ │ payload=AgentTurnResponseStepStartPayload(
|
||||
│ │ │ event_type='step_start',
|
||||
│ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ step_type='inference',
|
||||
│ │ │ metadata={}
|
||||
│ │ )
|
||||
│ )
|
||||
)
|
||||
AgentTurnResponseStreamChunk(
|
||||
│ event=TurnResponseEvent(
|
||||
│ │ payload=AgentTurnResponseStepProgressPayload(
|
||||
│ │ │ delta=TextDelta(text='As', type='text'),
|
||||
│ │ │ event_type='step_progress',
|
||||
│ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ step_type='inference'
|
||||
│ │ )
|
||||
│ )
|
||||
)
|
||||
AgentTurnResponseStreamChunk(
|
||||
│ event=TurnResponseEvent(
|
||||
│ │ payload=AgentTurnResponseStepProgressPayload(
|
||||
│ │ │ delta=TextDelta(text=' a', type='text'),
|
||||
│ │ │ event_type='step_progress',
|
||||
│ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ step_type='inference'
|
||||
│ │ )
|
||||
│ )
|
||||
)
|
||||
...
|
||||
AgentTurnResponseStreamChunk(
|
||||
│ event=TurnResponseEvent(
|
||||
│ │ payload=AgentTurnResponseStepCompletePayload(
|
||||
│ │ │ event_type='step_complete',
|
||||
│ │ │ step_details=InferenceStep(
|
||||
│ │ │ │ api_model_response=CompletionMessage(
|
||||
│ │ │ │ │ content='As a conversational AI, I don\'t have a personal identity in the classical sense. I exist as a program running on computer servers, designed to process and respond to text-based inputs.\n\nI\'m an instance of a type of artificial intelligence called a "language model," which is trained on vast amounts of text data to generate human-like responses. My primary function is to understand and respond to natural language inputs, like our conversation right now.\n\nThink of me as a virtual assistant, a chatbot, or a conversational interface – I\'m here to provide information, answer questions, and engage in conversation to the best of my abilities. I don\'t have feelings, emotions, or consciousness like humans do, but I\'m designed to simulate human-like interactions to make our conversations feel more natural and helpful.\n\nSo, that\'s me in a nutshell! What can I help you with today?',
|
||||
│ │ │ │ │ role='assistant',
|
||||
│ │ │ │ │ stop_reason='end_of_turn',
|
||||
│ │ │ │ │ tool_calls=[]
|
||||
│ │ │ │ ),
|
||||
│ │ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ │ step_type='inference',
|
||||
│ │ │ │ turn_id='8b360202-f7cb-4786-baa9-166a1b46e2ca',
|
||||
│ │ │ │ completed_at=datetime.datetime(2025, 4, 3, 1, 15, 21, 716174, tzinfo=TzInfo(UTC)),
|
||||
│ │ │ │ started_at=datetime.datetime(2025, 4, 3, 1, 15, 14, 28823, tzinfo=TzInfo(UTC))
|
||||
│ │ │ ),
|
||||
│ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ step_type='inference'
|
||||
│ │ )
|
||||
│ )
|
||||
)
|
||||
AgentTurnResponseStreamChunk(
|
||||
│ event=TurnResponseEvent(
|
||||
│ │ payload=AgentTurnResponseTurnCompletePayload(
|
||||
│ │ │ event_type='turn_complete',
|
||||
│ │ │ turn=Turn(
|
||||
│ │ │ │ input_messages=[UserMessage(content='Who are you?', role='user', context=None)],
|
||||
│ │ │ │ output_message=CompletionMessage(
|
||||
│ │ │ │ │ content='As a conversational AI, I don\'t have a personal identity in the classical sense. I exist as a program running on computer servers, designed to process and respond to text-based inputs.\n\nI\'m an instance of a type of artificial intelligence called a "language model," which is trained on vast amounts of text data to generate human-like responses. My primary function is to understand and respond to natural language inputs, like our conversation right now.\n\nThink of me as a virtual assistant, a chatbot, or a conversational interface – I\'m here to provide information, answer questions, and engage in conversation to the best of my abilities. I don\'t have feelings, emotions, or consciousness like humans do, but I\'m designed to simulate human-like interactions to make our conversations feel more natural and helpful.\n\nSo, that\'s me in a nutshell! What can I help you with today?',
|
||||
│ │ │ │ │ role='assistant',
|
||||
│ │ │ │ │ stop_reason='end_of_turn',
|
||||
│ │ │ │ │ tool_calls=[]
|
||||
│ │ │ │ ),
|
||||
│ │ │ │ session_id='abd4afea-4324-43f4-9513-cfe3970d92e8',
|
||||
│ │ │ │ started_at=datetime.datetime(2025, 4, 3, 1, 15, 14, 28722, tzinfo=TzInfo(UTC)),
|
||||
│ │ │ │ steps=[
|
||||
│ │ │ │ │ InferenceStep(
|
||||
│ │ │ │ │ │ api_model_response=CompletionMessage(
|
||||
│ │ │ │ │ │ │ content='As a conversational AI, I don\'t have a personal identity in the classical sense. I exist as a program running on computer servers, designed to process and respond to text-based inputs.\n\nI\'m an instance of a type of artificial intelligence called a "language model," which is trained on vast amounts of text data to generate human-like responses. My primary function is to understand and respond to natural language inputs, like our conversation right now.\n\nThink of me as a virtual assistant, a chatbot, or a conversational interface – I\'m here to provide information, answer questions, and engage in conversation to the best of my abilities. I don\'t have feelings, emotions, or consciousness like humans do, but I\'m designed to simulate human-like interactions to make our conversations feel more natural and helpful.\n\nSo, that\'s me in a nutshell! What can I help you with today?',
|
||||
│ │ │ │ │ │ │ role='assistant',
|
||||
│ │ │ │ │ │ │ stop_reason='end_of_turn',
|
||||
│ │ │ │ │ │ │ tool_calls=[]
|
||||
│ │ │ │ │ │ ),
|
||||
│ │ │ │ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ │ │ │ step_type='inference',
|
||||
│ │ │ │ │ │ turn_id='8b360202-f7cb-4786-baa9-166a1b46e2ca',
|
||||
│ │ │ │ │ │ completed_at=datetime.datetime(2025, 4, 3, 1, 15, 21, 716174, tzinfo=TzInfo(UTC)),
|
||||
│ │ │ │ │ │ started_at=datetime.datetime(2025, 4, 3, 1, 15, 14, 28823, tzinfo=TzInfo(UTC))
|
||||
│ │ │ │ │ )
|
||||
│ │ │ │ ],
|
||||
│ │ │ │ turn_id='8b360202-f7cb-4786-baa9-166a1b46e2ca',
|
||||
│ │ │ │ completed_at=datetime.datetime(2025, 4, 3, 1, 15, 21, 727364, tzinfo=TzInfo(UTC)),
|
||||
│ │ │ │ output_attachments=[]
|
||||
│ │ │ )
|
||||
│ │ )
|
||||
│ )
|
||||
)
|
||||
I'm constantly learning and improving my abilities, so feel free to ask me anything, and I'll do my best to help!
|
||||
|
||||
Streaming ...
|
||||
AgentTurnResponseStreamChunk(
|
||||
│ event=TurnResponseEvent(
|
||||
│ │ payload=AgentTurnResponseStepStartPayload(
|
||||
│ │ │ event_type='step_start',
|
||||
│ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ step_type='inference',
|
||||
│ │ │ metadata={}
|
||||
│ │ )
|
||||
│ )
|
||||
)
|
||||
AgentTurnResponseStreamChunk(
|
||||
│ event=TurnResponseEvent(
|
||||
│ │ payload=AgentTurnResponseStepProgressPayload(
|
||||
│ │ │ delta=TextDelta(text='As', type='text'),
|
||||
│ │ │ event_type='step_progress',
|
||||
│ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ step_type='inference'
|
||||
│ │ )
|
||||
│ )
|
||||
)
|
||||
AgentTurnResponseStreamChunk(
|
||||
│ event=TurnResponseEvent(
|
||||
│ │ payload=AgentTurnResponseStepProgressPayload(
|
||||
│ │ │ delta=TextDelta(text=' a', type='text'),
|
||||
│ │ │ event_type='step_progress',
|
||||
│ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ step_type='inference'
|
||||
│ │ )
|
||||
│ )
|
||||
)
|
||||
...
|
||||
AgentTurnResponseStreamChunk(
|
||||
│ event=TurnResponseEvent(
|
||||
│ │ payload=AgentTurnResponseStepCompletePayload(
|
||||
│ │ │ event_type='step_complete',
|
||||
│ │ │ step_details=InferenceStep(
|
||||
│ │ │ │ api_model_response=CompletionMessage(
|
||||
│ │ │ │ │ content='As a conversational AI, I don\'t have a personal identity in the classical sense. I exist as a program running on computer servers, designed to process and respond to text-based inputs.\n\nI\'m an instance of a type of artificial intelligence called a "language model," which is trained on vast amounts of text data to generate human-like responses. My primary function is to understand and respond to natural language inputs, like our conversation right now.\n\nThink of me as a virtual assistant, a chatbot, or a conversational interface – I\'m here to provide information, answer questions, and engage in conversation to the best of my abilities. I don\'t have feelings, emotions, or consciousness like humans do, but I\'m designed to simulate human-like interactions to make our conversations feel more natural and helpful.\n\nSo, that\'s me in a nutshell! What can I help you with today?',
|
||||
│ │ │ │ │ role='assistant',
|
||||
│ │ │ │ │ stop_reason='end_of_turn',
|
||||
│ │ │ │ │ tool_calls=[]
|
||||
│ │ │ │ ),
|
||||
│ │ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ │ step_type='inference',
|
||||
│ │ │ │ turn_id='8b360202-f7cb-4786-baa9-166a1b46e2ca',
|
||||
│ │ │ │ completed_at=datetime.datetime(2025, 4, 3, 1, 15, 21, 716174, tzinfo=TzInfo(UTC)),
|
||||
│ │ │ │ started_at=datetime.datetime(2025, 4, 3, 1, 15, 14, 28823, tzinfo=TzInfo(UTC))
|
||||
│ │ │ ),
|
||||
│ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ step_type='inference'
|
||||
│ │ )
|
||||
│ )
|
||||
)
|
||||
AgentTurnResponseStreamChunk(
|
||||
│ event=TurnResponseEvent(
|
||||
│ │ payload=AgentTurnResponseTurnCompletePayload(
|
||||
│ │ │ event_type='turn_complete',
|
||||
│ │ │ turn=Turn(
|
||||
│ │ │ │ input_messages=[UserMessage(content='Who are you?', role='user', context=None)],
|
||||
│ │ │ │ output_message=CompletionMessage(
|
||||
│ │ │ │ │ content='As a conversational AI, I don\'t have a personal identity in the classical sense. I exist as a program running on computer servers, designed to process and respond to text-based inputs.\n\nI\'m an instance of a type of artificial intelligence called a "language model," which is trained on vast amounts of text data to generate human-like responses. My primary function is to understand and respond to natural language inputs, like our conversation right now.\n\nThink of me as a virtual assistant, a chatbot, or a conversational interface – I\'m here to provide information, answer questions, and engage in conversation to the best of my abilities. I don\'t have feelings, emotions, or consciousness like humans do, but I\'m designed to simulate human-like interactions to make our conversations feel more natural and helpful.\n\nSo, that\'s me in a nutshell! What can I help you with today?',
|
||||
│ │ │ │ │ role='assistant',
|
||||
│ │ │ │ │ stop_reason='end_of_turn',
|
||||
│ │ │ │ │ tool_calls=[]
|
||||
│ │ │ │ ),
|
||||
│ │ │ │ session_id='abd4afea-4324-43f4-9513-cfe3970d92e8',
|
||||
│ │ │ │ started_at=datetime.datetime(2025, 4, 3, 1, 15, 14, 28722, tzinfo=TzInfo(UTC)),
|
||||
│ │ │ │ steps=[
|
||||
│ │ │ │ │ InferenceStep(
|
||||
│ │ │ │ │ │ api_model_response=CompletionMessage(
|
||||
│ │ │ │ │ │ │ content='As a conversational AI, I don\'t have a personal identity in the classical sense. I exist as a program running on computer servers, designed to process and respond to text-based inputs.\n\nI\'m an instance of a type of artificial intelligence called a "language model," which is trained on vast amounts of text data to generate human-like responses. My primary function is to understand and respond to natural language inputs, like our conversation right now.\n\nThink of me as a virtual assistant, a chatbot, or a conversational interface – I\'m here to provide information, answer questions, and engage in conversation to the best of my abilities. I don\'t have feelings, emotions, or consciousness like humans do, but I\'m designed to simulate human-like interactions to make our conversations feel more natural and helpful.\n\nSo, that\'s me in a nutshell! What can I help you with today?',
|
||||
│ │ │ │ │ │ │ role='assistant',
|
||||
│ │ │ │ │ │ │ stop_reason='end_of_turn',
|
||||
│ │ │ │ │ │ │ tool_calls=[]
|
||||
│ │ │ │ │ │ ),
|
||||
│ │ │ │ │ │ step_id='69831607-fa75-424a-949b-e2049e3129d1',
|
||||
│ │ │ │ │ │ step_type='inference',
|
||||
│ │ │ │ │ │ turn_id='8b360202-f7cb-4786-baa9-166a1b46e2ca',
|
||||
│ │ │ │ │ │ completed_at=datetime.datetime(2025, 4, 3, 1, 15, 21, 716174, tzinfo=TzInfo(UTC)),
|
||||
│ │ │ │ │ │ started_at=datetime.datetime(2025, 4, 3, 1, 15, 14, 28823, tzinfo=TzInfo(UTC))
|
||||
│ │ │ │ │ )
|
||||
│ │ │ │ ],
|
||||
│ │ │ │ turn_id='8b360202-f7cb-4786-baa9-166a1b46e2ca',
|
||||
│ │ │ │ completed_at=datetime.datetime(2025, 4, 3, 1, 15, 21, 727364, tzinfo=TzInfo(UTC)),
|
||||
│ │ │ │ output_attachments=[]
|
||||
│ │ │ )
|
||||
│ │ )
|
||||
│ )
|
||||
)
|
||||
|
||||
|
||||
Streaming with print helper...
|
||||
inference> Déjà vu!
|
||||
Streaming with print helper...
|
||||
inference> Déjà vu!
|
||||
|
||||
As I mentioned earlier, I'm an artificial intelligence language model. I don't have a personal identity or consciousness like humans do. I exist solely to process and respond to text-based inputs, providing information and assistance on a wide range of topics.
|
||||
As I mentioned earlier, I'm an artificial intelligence language model. I don't have a personal identity or consciousness like humans do. I exist solely to process and respond to text-based inputs, providing information and assistance on a wide range of topics.
|
||||
|
||||
I'm a computer program designed to simulate human-like conversations, using natural language processing (NLP) and machine learning algorithms to understand and generate responses. My purpose is to help users like you with their questions, provide information, and engage in conversation.
|
||||
I'm a computer program designed to simulate human-like conversations, using natural language processing (NLP) and machine learning algorithms to understand and generate responses. My purpose is to help users like you with their questions, provide information, and engage in conversation.
|
||||
|
||||
Think of me as a virtual companion, a helpful tool designed to make your interactions more efficient and enjoyable. I don't have personal opinions, emotions, or biases, but I'm here to provide accurate and informative responses to the best of my abilities.
|
||||
|
||||
So, who am I? I'm just a computer program designed to help you!
|
||||
Think of me as a virtual companion, a helpful tool designed to make your interactions more efficient and enjoyable. I don't have personal opinions, emotions, or biases, but I'm here to provide accurate and informative responses to the best of my abilities.
|
||||
|
||||
So, who am I? I'm just a computer program designed to help you!
|
||||
```
|
||||
:::
|
||||
|
||||
## Step 6: Build a RAG Agent
|
||||
:::{tab-item} Build a RAG Agent
|
||||
|
||||
For our last demo, we can build a RAG agent that can answer questions about the Torchtune project using the documents
|
||||
in a vector database.
|
||||
### i. Create the Script
|
||||
|
|
@ -505,23 +522,24 @@ Let's run the script using `uv`
|
|||
```bash
|
||||
uv run python rag_agent.py
|
||||
```
|
||||
:::{dropdown} `👋 Click here to see the sample output`
|
||||
|
||||
```
|
||||
user> what is torchtune
|
||||
inference> [knowledge_search(query='TorchTune')]
|
||||
tool_execution> Tool:knowledge_search Args:{'query': 'TorchTune'}
|
||||
tool_execution> Tool:knowledge_search Response:[TextContentItem(text='knowledge_search tool found 5 chunks:\nBEGIN of knowledge_search tool results.\n', type='text'), TextContentItem(text='Result 1:\nDocument_id:num-1\nContent: conversational data, :func:`~torchtune.datasets.chat_dataset` seems to be a good fit. ..., type='text'), TextContentItem(text='END of knowledge_search tool results.\n', type='text')]
|
||||
inference> Here is a high-level overview of the text:
|
||||
```{dropdown} 👋 Click here to see the sample output
|
||||
user> what is torchtune
|
||||
inference> [knowledge_search(query='TorchTune')]
|
||||
tool_execution> Tool:knowledge_search Args:{'query': 'TorchTune'}
|
||||
tool_execution> Tool:knowledge_search Response:[TextContentItem(text='knowledge_search tool found 5 chunks:\nBEGIN of knowledge_search tool results.\n', type='text'), TextContentItem(text='Result 1:\nDocument_id:num-1\nContent: conversational data, :func:`~torchtune.datasets.chat_dataset` seems to be a good fit. ..., type='text'), TextContentItem(text='END of knowledge_search tool results.\n', type='text')]
|
||||
inference> Here is a high-level overview of the text:
|
||||
|
||||
**LoRA Finetuning with PyTorch Tune**
|
||||
**LoRA Finetuning with PyTorch Tune**
|
||||
|
||||
PyTorch Tune provides a recipe for LoRA (Low-Rank Adaptation) finetuning, which is a technique to adapt pre-trained models to new tasks. The recipe uses the `lora_finetune_distributed` command.
|
||||
...
|
||||
Overall, DORA is a powerful reinforcement learning algorithm that can learn complex tasks from human demonstrations. However, it requires careful consideration of the challenges and limitations to achieve optimal results.
|
||||
PyTorch Tune provides a recipe for LoRA (Low-Rank Adaptation) finetuning, which is a technique to adapt pre-trained models to new tasks. The recipe uses the `lora_finetune_distributed` command.
|
||||
...
|
||||
Overall, DORA is a powerful reinforcement learning algorithm that can learn complex tasks from human demonstrations. However, it requires careful consideration of the challenges and limitations to achieve optimal results.
|
||||
```
|
||||
:::
|
||||
|
||||
::::
|
||||
|
||||
## Build Your Own Apps!
|
||||
|
||||
Congrats! 🥳 Now you're ready to [build your own Llama Stack applications](../building_applications/index)! 🚀
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue