mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-03 18:00:36 +00:00
4a5ef65286
# What does this PR do? This API hasn't received any traction and close to zero interest from the community. Let's revisit in the future if things change. Signed-off-by: Sébastien Han <seb@redhat.com> Co-authored-by: Ashwin Bharambe <ashwin.bharambe@gmail.com>
26 lines
1.2 KiB
Text
26 lines
1.2 KiB
Text
---
|
|
title: APIs
|
|
description: Available REST APIs and planned capabilities in Llama Stack
|
|
sidebar_label: APIs
|
|
sidebar_position: 1
|
|
---
|
|
|
|
# APIs
|
|
|
|
A Llama Stack API is described as a collection of REST endpoints. We currently support the following APIs:
|
|
|
|
- **Inference**: run inference with a LLM
|
|
- **Safety**: apply safety policies to the output at a Systems (not only model) level
|
|
- **Agents**: run multi-step agentic workflows with LLMs with tool usage, memory (RAG), etc.
|
|
- **DatasetIO**: interface with datasets and data loaders
|
|
- **Scoring**: evaluate outputs of the system
|
|
- **Eval**: generate outputs (via Inference or Agents) and perform scoring
|
|
- **VectorIO**: perform operations on vector stores, such as adding documents, searching, and deleting documents
|
|
- **Post Training**: fine-tune a model
|
|
- **Tool Runtime**: interact with various tools and protocols
|
|
- **Responses**: generate responses from an LLM using this OpenAI compatible API.
|
|
|
|
We are working on adding a few more APIs to complete the application lifecycle. These will include:
|
|
- **Batch Inference**: run inference on a dataset of inputs
|
|
- **Batch Agents**: run agents on a dataset of inputs
|
|
- **Batches**: OpenAI-compatible batch management for inference
|