phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-10-07 12:47:37 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Make TGI adapter compatible with HF Inference API

Browse source
This commit is contained in:
Wauplin 2024-09-25 15:33:58 +02:00
parent 45be9f3b85
commit 854ff85784
No known key found for this signature in database
GPG key ID: 9838FE02BECE1A02
7 changed files with 115 additions and 99 deletions
Download patch file Download diff file Expand all files Collapse all files

10
llama_stack/distribution/templates/local-hf-endpoint-build.yaml Normal file
View file

@ -0,0 +1,10 @@
name: local-hf-endpoint
distribution_spec:
description: "Like local, but use Hugging Face Inference Endpoints for running LLM inference.\nSee https://hf.co/docs/api-endpoints."
providers:
inference: remote::hf::endpoint
memory: meta-reference
safety: meta-reference
agents: meta-reference
telemetry: meta-reference
image_type: conda

10
llama_stack/distribution/templates/local-hf-serverless-build.yaml Normal file
View file

@ -0,0 +1,10 @@
name: local-hf-serverless
distribution_spec:
description: "Like local, but use Hugging Face Inference API (serverless) for running LLM inference.\nSee https://hf.co/docs/api-inference."
providers:
inference: remote::hf::serverless
memory: meta-reference
safety: meta-reference
agents: meta-reference
telemetry: meta-reference
image_type: conda

2
llama_stack/distribution/templates/local-tgi-build.yaml
View file

@ -1,6 +1,6 @@
name: local-tgi
distribution_spec:
description: Use TGI (local or with Hugging Face Inference Endpoints for running LLM inference. When using HF Inference Endpoints, you must provide the name of the endpoint).
description: Like local, but use a TGI server for running LLM inference.
providers:
inference: remote::tgi
memory: meta-reference