mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-10-03 19:57:35 +00:00
c3d3a0b833
Some checks failed
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 1s
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Has been skipped
Integration Tests (Replay) / Integration Tests (, , , client=, vision=) (push) Failing after 3s
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 4s
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 7s
Update ReadTheDocs / update-readthedocs (push) Failing after 3s
Test External API and Providers / test-external (venv) (push) Failing after 5s
Vector IO Integration Tests / test-matrix (push) Failing after 7s
Python Package Build Test / build (3.13) (push) Failing after 8s
Python Package Build Test / build (3.12) (push) Failing after 8s
Unit Tests / unit-tests (3.13) (push) Failing after 14s
Unit Tests / unit-tests (3.12) (push) Failing after 14s
UI Tests / ui-tests (22) (push) Successful in 1m7s
Pre-commit / pre-commit (push) Successful in 2m34s
One needed to specify record-replay related environment variables for running integration tests. We could not use defaults because integration tests could be run against Ollama instances which could be running different models. For example, text vs vision tests needed separate instances of Ollama because a single instance typically cannot serve both of these models if you assume the standard CI worker configuration on Github. As a result, `client.list()` as returned by the Ollama client would be different between these runs and we'd end up overwriting responses. This PR "solves" it by adding a small amount of complexity -- we store model list responses specially, keyed by the hashes of the models they return. At replay time, we merge all of them and pretend that we have the union of all models available. ## Test Plan Re-recorded all the tests using `scripts/integration-tests.sh --inference-mode record`, including the vision tests.
39 lines
1.8 KiB
JSON
39 lines
1.8 KiB
JSON
{
|
|
"request": {
|
|
"method": "POST",
|
|
"url": "http://localhost:11434/api/generate",
|
|
"headers": {},
|
|
"body": {
|
|
"model": "llama3.2:3b-instruct-fp16",
|
|
"raw": true,
|
|
"prompt": "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWhat is the smallest country in the world?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
|
|
"options": {
|
|
"temperature": 0.0
|
|
},
|
|
"stream": false
|
|
},
|
|
"endpoint": "/api/generate",
|
|
"model": "llama3.2:3b-instruct-fp16"
|
|
},
|
|
"response": {
|
|
"body": {
|
|
"__type__": "ollama._types.GenerateResponse",
|
|
"__data__": {
|
|
"model": "llama3.2:3b-instruct-fp16",
|
|
"created_at": "2025-09-03T17:39:54.374714Z",
|
|
"done": true,
|
|
"done_reason": "stop",
|
|
"total_duration": 6321793333,
|
|
"load_duration": 182255958,
|
|
"prompt_eval_count": 25,
|
|
"prompt_eval_duration": 67964459,
|
|
"eval_count": 150,
|
|
"eval_duration": 6070867875,
|
|
"response": "The smallest country in the world is the Vatican City, which has a total area of approximately 0.44 km\u00b2 (0.17 sq mi). It is an independent city-state located within Rome, Italy, and is home to the Pope and the central government of the Catholic Church.\n\nTo put that into perspective, the Vatican City is smaller than a golf course! Despite its tiny size, it has its own government, currency, postal system, and even its own police force. It's also home to numerous iconic landmarks like St. Peter's Basilica and the Sistine Chapel.\n\nInterestingly, the Vatican City is not only the smallest country in the world but also the most densely populated, with a population of just over 800 people!",
|
|
"thinking": null,
|
|
"context": null
|
|
}
|
|
},
|
|
"is_streaming": false
|
|
}
|
|
}
|