phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions
llama-stack/docs/source/distributions
History
Rashmi Pawar ace82836c1
feat: NVIDIA allow non-llama model registration (#1859) 
# What does this PR do?
Adds custom model registration functionality to NVIDIAInferenceAdapter
which let's the inference happen on:
- post-training model
- non-llama models in API Catalogue(behind
https://integrate.api.nvidia.com and endpoints compatible with
AyncOpenAI)

## Example Usage:
```python
from llama_stack.apis.models import Model, ModelType
from llama_stack.distribution.library_client import LlamaStackAsLibraryClient
client = LlamaStackAsLibraryClient("nvidia")
_ = client.initialize()

client.models.register(
        model_id=model_name,
        model_type=ModelType.llm,
        provider_id="nvidia"
)

response = client.inference.chat_completion(
    model_id=model_name,
    messages=[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Write a limerick about the wonders of GPU computing."}],
)
```

## Test Plan
```bash
pytest tests/unit/providers/nvidia/test_supervised_fine_tuning.py 
========================================================== test session starts ===========================================================
platform linux -- Python 3.10.0, pytest-8.3.5, pluggy-1.5.0
rootdir: /home/ubuntu/llama-stack
configfile: pyproject.toml
plugins: anyio-4.9.0
collected 6 items                                                                                                                        

tests/unit/providers/nvidia/test_supervised_fine_tuning.py ......                                                                  [100%]

============================================================ warnings summary ============================================================
../miniconda/envs/nvidia-1/lib/python3.10/site-packages/pydantic/fields.py:1076
  /home/ubuntu/miniconda/envs/nvidia-1/lib/python3.10/site-packages/pydantic/fields.py:1076: PydanticDeprecatedSince20: Using extra keyword arguments on `Field` is deprecated and will be removed. Use `json_schema_extra` instead. (Extra keys: 'contentEncoding'). Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.11/migration/
    warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================================================== 6 passed, 1 warning in 1.51s ======================================================
```

[//]: # (## Documentation)
Updated Readme.md

cc: @dglogo, @sumitb, @mattf
2025-04-24 17:13:33 -07:00
..
ondevice_distro docs: 0.2.2 doc updates (#1961) 2025-04-15 13:26:17 -07:00
remote_hosted_distro docs: Add NVIDIA platform distro docs (#1971) 2025-04-17 05:54:30 -07:00
self_hosted_distro feat: NVIDIA allow non-llama model registration (#1859) 2025-04-24 17:13:33 -07:00
building_distro.md feat: allow building distro with external providers (#1967) 2025-04-18 17:18:28 +02:00
configuration.md docs: Update quickstart page to structure things a little more for the novices (#1873) 2025-04-10 14:09:00 -07:00
importing_as_library.md docs: update importing_as_library.md (#1863) 2025-04-07 12:31:04 +02:00
index.md docs: Updated documentation and Sphinx configuration (#1845) 2025-03-31 13:08:05 -07:00
kubernetes_deployment.md docs: fix errors in kubernetes deployment guide (#1914) 2025-04-11 13:04:13 +02:00
list_of_distributions.md docs: Updated documentation and Sphinx configuration (#1845) 2025-03-31 13:08:05 -07:00
starting_llama_stack_server.md docs: Update quickstart page to structure things a little more for the novices (#1873) 2025-04-10 14:09:00 -07:00