forked from phoenix/litellm-mirror
27e18358ab
* fix(pattern_match_deployments.py): default to user input if unable to map based on wildcards * test: fix test * test: reset test name * test: update conftest to reload proxy server module between tests * ci(config.yml): move langfuse out of local_testing reduce ci/cd time * ci(config.yml): cleanup langfuse ci/cd tests * fix: update test to not use global proxy_server app module * ci: move caching to a separate test pipeline speed up ci pipeline * test: update conftest to check if proxy_server attr exists before reloading * build(conftest.py): don't block on inability to reload proxy_server * ci(config.yml): update caching unit test filter to work on 'cache' keyword as well * fix(encrypt_decrypt_utils.py): use function to get salt key * test: mark flaky test * test: handle anthropic overloaded errors * refactor: create separate ci/cd pipeline for proxy unit tests make ci/cd faster * ci(config.yml): add litellm_proxy_unit_testing to build_and_test jobs * ci(config.yml): generate prisma binaries for proxy unit tests * test: readd vertex_key.json * ci(config.yml): remove `-s` from proxy_unit_test cmd speed up test * ci: remove any 'debug' logging flag speed up ci pipeline * test: fix test * test(test_braintrust.py): rerun * test: add delay for braintrust test
30 lines
No EOL
1.2 KiB
YAML
30 lines
No EOL
1.2 KiB
YAML
model_list:
|
|
- model_name: gpt-3.5-turbo-instruct
|
|
litellm_params:
|
|
model: ollama/zephyr
|
|
- model_name: gpt-4
|
|
litellm_params:
|
|
model: ollama/llama2
|
|
- model_name: gpt-3.5-turbo
|
|
litellm_params:
|
|
model: ollama/llama2
|
|
temperature: 0.1
|
|
max_tokens: 20
|
|
|
|
|
|
# request to gpt-4, response from ollama/llama2
|
|
# curl --location 'http://0.0.0.0:8000/chat/completions' \
|
|
# --header 'Content-Type: application/json' \
|
|
# --data ' {
|
|
# "model": "gpt-4",
|
|
# "messages": [
|
|
# {
|
|
# "role": "user",
|
|
# "content": "what llm are you"
|
|
# }
|
|
# ],
|
|
# }
|
|
# '
|
|
#
|
|
|
|
# {"id":"chatcmpl-27c85cf0-ab09-4bcf-8cb1-0ee950520743","choices":[{"finish_reason":"stop","index":0,"message":{"content":" Hello! I'm just an AI, I don't have personal experiences or emotions like humans do. However, I can help you with any questions or tasks you may have! Is there something specific you'd like to know or discuss?","role":"assistant","_logprobs":null}}],"created":1700094955.373751,"model":"ollama/llama2","object":"chat.completion","system_fingerprint":null,"usage":{"prompt_tokens":12,"completion_tokens":47,"total_tokens":59},"_response_ms":8028.017999999999}% |