mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-29 07:12:22 +00:00
feat: remote ramalama provider implementation
Implement remote ramalama provider using AsyncOpenAI as the client since ramalama doesn't have its own Async library. Ramalama is similar to ollama, as it is a lightweight local inference server. However, it runs by default in a containerized mode. RAMALAMA_URL is http://localhost:8080 by default Signed-off-by: Charlie Doern <cdoern@redhat.com>
This commit is contained in:
parent
94f83382eb
commit
4de45560bf
8 changed files with 680 additions and 0 deletions
|
|
@ -306,6 +306,7 @@ async def instantiate_provider(
|
|||
additional_protocols = additional_protocols_map()
|
||||
|
||||
provider_spec = provider.spec
|
||||
|
||||
if not hasattr(provider_spec, "module"):
|
||||
raise AttributeError(f"ProviderSpec of type {type(provider_spec)} does not have a 'module' attribute")
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue