phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-10-04 12:07:34 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
llama-stack-mirror/llama_stack/providers/remote/inference
History
Matthew Farrellee c3fc859257 feat: add dynamic model registration support to TGI inference 
add new overwrite_completion_id feature to OpenAIMixin to deal with TGI always returning id=""

test with -

tgi: `docker run --gpus all --shm-size 1g -p 8080:80 -v /data:/data ghcr.io/huggingface/text-generation-inference --model-id Qwen/Qwen3-0.6B`

stack: `TGI_URL=http://localhost:8080 uv run llama stack build --image-type venv --distro ci-tests --run`

test: `./scripts/integration-tests.sh --stack-config http://localhost:8321 --setup tgi --subdirs inference --pattern openai`
2025-09-11 02:02:02 -04:00
..
anthropic chore: update the anthropic inference impl to use openai-python for openai-compat functions (#3366) 2025-09-07 14:00:42 -07:00
azure feat: add Azure OpenAI inference provider support (#3396) 2025-09-11 13:48:38 +02:00
bedrock fix: AWS Bedrock inference profile ID conversion for region-specific endpoints (#3386) 2025-09-11 11:41:53 +02:00
cerebras feat(starter)!: simplify starter distro; litellm model registry changes (#2916) 2025-07-25 15:02:04 -07:00
databricks feat(starter)!: simplify starter distro; litellm model registry changes (#2916) 2025-07-25 15:02:04 -07:00
fireworks refactor(logging): rename llama_stack logger categories (#3065) 2025-08-21 17:31:04 -07:00
gemini chore: update the gemini inference impl to use openai-python for openai-compat functions (#3351) 2025-09-06 12:22:20 -07:00
groq chore: update the groq inference impl to use openai-python for openai-compat functions (#3348) 2025-09-06 15:36:27 -07:00
llama_openai_compat chore: indicate to mypy that InferenceProvider.rerank is concrete (#3238) 2025-08-22 12:02:13 -07:00
nvidia docs: add VLM NIM example (#3277) 2025-08-29 16:23:52 -07:00
ollama feat(tests): auto-merge all model list responses and unify recordings (#3320) 2025-09-03 11:33:03 -07:00
openai refactor(logging): rename llama_stack logger categories (#3065) 2025-08-21 17:31:04 -07:00
passthrough chore(rename): move llama_stack.distribution to llama_stack.core (#2975) 2025-07-30 23:30:53 -07:00
runpod ci: test safety with starter (#2628) 2025-07-09 16:53:50 +02:00
sambanova chore: update the sambanova inference impl to use openai-python for openai-compat functions (#3345) 2025-09-06 12:25:13 -07:00
tgi feat: add dynamic model registration support to TGI inference 2025-09-11 02:02:02 -04:00
together refactor(logging): rename llama_stack logger categories (#3065) 2025-08-21 17:31:04 -07:00
vertexai ci: Re-enable pre-commit to fail (#3399) 2025-09-10 10:00:46 -04:00
vllm chore: indicate to mypy that InferenceProvider.batch_completion/batch_chat_completion is concrete (#3239) 2025-08-22 14:17:30 -07:00
watsonx chore(python-deps): replace ibm_watson_machine_learning with ibm_watsonx_ai (#3302) 2025-09-03 11:33:35 +02:00
__init__.py impls -> inline, adapters -> remote (#381) 2024-11-06 14:54:05 -08:00