removed assertion in ollama.py and fixed typo in the readme (#563)
# What does this PR do?
1. removed [incorrect
assertion](435f34b05e/llama_stack/providers/remote/inference/ollama/ollama.py (L183))
in ollama.py
2. fixed a typo in [this
line](435f34b05e/docs/source/distributions/importing_as_library.md (L24)),
as `model=` should be `model_id=` .
- [x] Addresses issue
([#issue562](https://github.com/meta-llama/llama-stack/issues/562))
## Test Plan
tested with code:
```python
import asyncio
import os
# pip install aiosqlite ollama faiss
from llama_stack_client.lib.direct.direct import LlamaStackDirectClient
from llama_stack_client.types import SystemMessage, UserMessage
async def main():
os.environ["INFERENCE_MODEL"] = "meta-llama/Llama-3.2-1B-Instruct"
client = await LlamaStackDirectClient.from_template("ollama")
await client.initialize()
response = await client.models.list()
print(response)
model_name = response[0].identifier
response = await client.inference.chat_completion(
messages=[
SystemMessage(content="You are a friendly assistant.", role="system"),
UserMessage(
content="hello world, write me a 2 sentence poem about the moon",
role="user",
),
],
model_id=model_name,
stream=False,
)
print("\nChat completion response:")
print(response, type(response))
asyncio.run(main())
```
OUTPUT:
```
python test.py
Using template ollama with config:
apis:
- agents
- inference
- memory
- safety
- telemetry
conda_env: ollama
datasets: []
docker_image: null
eval_tasks: []
image_name: ollama
memory_banks: []
metadata_store:
db_path: /Users/kaiwu/.llama/distributions/ollama/registry.db
namespace: null
type: sqlite
models:
- metadata: {}
model_id: meta-llama/Llama-3.2-1B-Instruct
provider_id: ollama
provider_model_id: null
providers:
agents:
- config:
persistence_store:
db_path:
/Users/kaiwu/.llama/distributions/ollama/agents_store.db
namespace: null
type: sqlite
provider_id: meta-reference
provider_type: inline::meta-reference
inference:
- config:
url: http://localhost:11434
provider_id: ollama
provider_type: remote::ollama
memory:
- config:
kvstore:
db_path:
/Users/kaiwu/.llama/distributions/ollama/faiss_store.db
namespace: null
type: sqlite
provider_id: faiss
provider_type: inline::faiss
safety:
- config: {}
provider_id: llama-guard
provider_type: inline::llama-guard
telemetry:
- config: {}
provider_id: meta-reference
provider_type: inline::meta-reference
scoring_fns: []
shields: []
version: '2'
[Model(identifier='meta-llama/Llama-3.2-1B-Instruct', provider_resource_id='llama3.2:1b-instruct-fp16', provider_id='ollama', type='model', metadata={})]
Chat completion response:
completion_message=CompletionMessage(role='assistant', content='Here is a short poem about the moon:\n\nThe moon glows bright in the midnight sky,\nA silver crescent shining, catching the eye.', stop_reason=<StopReason.end_of_turn: 'end_of_turn'>, tool_calls=[]) logprobs=None <class 'llama_stack.apis.inference.inference.ChatCompletionResponse'>
```
## Sources
Please link relevant resources if necessary.
## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the
other checks if that's the case).
- [ ] Ran pre-commit to handle lint / formatting issues.
- [ ] Read the [contributor
guideline](https://github.com/meta-llama/llama-stack/blob/main/CONTRIBUTING.md),
Pull Request section?
- [ ] Updated relevant documentation.
- [ ] Wrote necessary unit or integration tests.
b6500974ec