mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-10-05 12:21:52 +00:00
Introduce a "Router" layer for providers
Some providers need to be factorized and considered as thin routing layers on top of other providers. Consider two examples: - The inference API should be a routing layer over inference providers, routed using the "model" key - The memory banks API is another instance where various memory bank types will be provided by independent providers (e.g., a vector store is served by Chroma while a keyvalue memory can be served by Redis or PGVector) This commit introduces a generalized routing layer for this purpose.
This commit is contained in:
parent
5c1f2616b5
commit
b6a3ef51da
12 changed files with 384 additions and 118 deletions
|
@ -26,16 +26,16 @@ def available_providers() -> List[ProviderSpec]:
|
|||
config_class="llama_toolchain.memory.meta_reference.faiss.FaissImplConfig",
|
||||
),
|
||||
remote_provider_spec(
|
||||
api=Api.memory,
|
||||
adapter=AdapterSpec(
|
||||
Api.memory,
|
||||
AdapterSpec(
|
||||
adapter_id="chromadb",
|
||||
pip_packages=EMBEDDING_DEPS + ["chromadb-client"],
|
||||
module="llama_toolchain.memory.adapters.chroma",
|
||||
),
|
||||
),
|
||||
remote_provider_spec(
|
||||
api=Api.memory,
|
||||
adapter=AdapterSpec(
|
||||
Api.memory,
|
||||
AdapterSpec(
|
||||
adapter_id="pgvector",
|
||||
pip_packages=EMBEDDING_DEPS + ["psycopg2-binary"],
|
||||
module="llama_toolchain.memory.adapters.pgvector",
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue