mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-10-05 04:17:32 +00:00
d23865757f
# What does this PR do? <!-- Provide a short summary of what this PR does and why. Link to relevant issues if applicable. --> <!-- If resolving an issue, uncomment and update the line below --> <!-- Closes #[issue-number] --> - Updates provider and distro codegen to handle the new format - Migrates provider and distro files to the new format ## Test Plan - Manual testing <!-- Describe the tests you ran to verify your changes with result summaries. *Provide clear instructions so the plan can be easily re-executed.* -->
25 lines
825 B
Text
25 lines
825 B
Text
---
|
|
description: "HuggingFace Inference API serverless provider for on-demand model inference."
|
|
sidebar_label: Remote - Hf - Serverless
|
|
title: remote::hf::serverless
|
|
---
|
|
|
|
# remote::hf::serverless
|
|
|
|
## Description
|
|
|
|
HuggingFace Inference API serverless provider for on-demand model inference.
|
|
|
|
## Configuration
|
|
|
|
| Field | Type | Required | Default | Description |
|
|
|-------|------|----------|---------|-------------|
|
|
| `huggingface_repo` | `<class 'str'>` | No | | The model ID of the model on the Hugging Face Hub (e.g. 'meta-llama/Meta-Llama-3.1-70B-Instruct') |
|
|
| `api_token` | `pydantic.types.SecretStr \| None` | No | | Your Hugging Face user access token (will default to locally saved token if not provided) |
|
|
|
|
## Sample Configuration
|
|
|
|
```yaml
|
|
huggingface_repo: ${env.INFERENCE_MODEL}
|
|
api_token: ${env.HF_API_TOKEN}
|
|
```
|