mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-06-28 19:04:19 +00:00
5b0d778871
Fixing the bullets # What does this PR do? The bullets were not there as intended so I helped fix them. - [x] Addresses issue (#issue) ## Test Plan Please describe: Ran the test, and the bullets are there now to be consistent with the page. ## Sources N/A ## Before submitting - [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case). - [ ] Ran pre-commit to handle lint / formatting issues. - [ ] Read the [contributor guideline](https://github.com/meta-llama/llama-stack/blob/main/CONTRIBUTING.md), Pull Request section? - [ ] Updated relevant documentation. - [ ] Wrote necessary unit or integration tests.
63 lines
2.6 KiB
Markdown
63 lines
2.6 KiB
Markdown
# Why Llama Stack?
|
|
|
|
Building production AI applications today requires solving multiple challenges:
|
|
|
|
**Infrastructure Complexity**
|
|
- Running large language models efficiently requires specialized infrastructure.
|
|
- Different deployment scenarios (local development, cloud, edge) need different solutions.
|
|
- Moving from development to production often requires significant rework.
|
|
|
|
**Essential Capabilities**
|
|
- Safety guardrails and content filtering are necessary in an enterprise setting.
|
|
- Just model inference is not enough - Knowledge retrieval and RAG capabilities are required.
|
|
- Nearly any application needs composable multi-step workflows.
|
|
- Finally, without monitoring, observability and evaluation, you end up operating in the dark.
|
|
|
|
**Lack of Flexibility and Choice**
|
|
- Directly integrating with multiple providers creates tight coupling.
|
|
- Different providers have different APIs and abstractions.
|
|
- Changing providers requires significant code changes.
|
|
|
|
|
|
### Our Solution: A Universal Stack
|
|
|
|
```{image} ../../_static/llama-stack.png
|
|
:alt: Llama Stack
|
|
:width: 400px
|
|
```
|
|
|
|
Llama Stack addresses these challenges through a service-oriented, API-first approach:
|
|
|
|
**Develop Anywhere, Deploy Everywhere**
|
|
- Start locally with CPU-only setups
|
|
- Move to GPU acceleration when needed
|
|
- Deploy to cloud or edge without code changes
|
|
- Same APIs and developer experience everywhere
|
|
|
|
**Production-Ready Building Blocks**
|
|
- Pre-built safety guardrails and content filtering
|
|
- Built-in RAG and agent capabilities
|
|
- Comprehensive evaluation toolkit
|
|
- Full observability and monitoring
|
|
|
|
**True Provider Independence**
|
|
- Swap providers without application changes
|
|
- Mix and match best-in-class implementations
|
|
- Federation and fallback support
|
|
- No vendor lock-in
|
|
|
|
**Robust Ecosystem**
|
|
- Llama Stack is already integrated with distribution partners (cloud providers, hardware vendors, and AI-focused companies).
|
|
- Ecosystem offers tailored infrastructure, software, and services for deploying Llama models.
|
|
|
|
|
|
### Our Philosophy
|
|
|
|
- **Service-Oriented**: REST APIs enforce clean interfaces and enable seamless transitions across different environments.
|
|
- **Composability**: Every component is independent but works together seamlessly
|
|
- **Production Ready**: Built for real-world applications, not just demos
|
|
- **Turnkey Solutions**: Easy to deploy built in solutions for popular deployment scenarios
|
|
- **Llama First**: Explicit focus on Meta's Llama models and partnering ecosystem
|
|
|
|
|
|
With Llama Stack, you can focus on building your application while we handle the infrastructure complexity, essential capabilities, and provider integrations.
|