phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

(docs) xinference embedding

Browse source
This commit is contained in:
ishaan-jaff 2024-01-02 15:39:25 +05:30
parent 790dcff5e0
commit fdd4e72503
2 changed files with 63 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

62
docs/my-website/docs/providers/xinference.md Normal file
View file

@ -0,0 +1,62 @@
# Xinference [Xorbits Inference]
https://inference.readthedocs.io/en/latest/index.html
## API Base, Key
```python
# env variable
os.environ['XINFERENCE_API_BASE'] = "http://127.0.0.1:9997/v1"
os.environ['XINFERENCE_API_KEY'] = "anything" #[optional] no api key required
```
## Sample Usage - Embedding
```python
from litellm import embedding
import os
os.environ['XINFERENCE_API_BASE'] = "http://127.0.0.1:9997/v1"
response = embedding(
model="xinference/bge-base-en",
input=["good morning from litellm"],
)
print(response)
```
## Sample Usage `api_base` param
```python
from litellm import embedding
import os
response = embedding(
model="xinference/bge-base-en",
api_base="http://127.0.0.1:9997/v1",
input=["good morning from litellm"],
)
print(response)
```
## Supported Models
All models listed here https://inference.readthedocs.io/en/latest/models/builtin/embedding/index.html are supported
| Model Name | Function Call |
|------------------------------|--------------------------------------------------------|
| bge-base-en | `embedding(model="xinference/bge-base-en", input)` |
| bge-base-en-v1.5 | `embedding(model="xinference/bge-base-en-v1.5", input)` |
| bge-base-zh | `embedding(model="xinference/bge-base-zh", input)` |
| bge-base-zh-v1.5 | `embedding(model="xinference/bge-base-zh-v1.5", input)` |
| bge-large-en | `embedding(model="xinference/bge-large-en", input)` |
| bge-large-en-v1.5 | `embedding(model="xinference/bge-large-en-v1.5", input)` |
| bge-large-zh | `embedding(model="xinference/bge-large-zh", input)` |
| bge-large-zh-noinstruct | `embedding(model="xinference/bge-large-zh-noinstruct", input)` |
| bge-large-zh-v1.5 | `embedding(model="xinference/bge-large-zh-v1.5", input)` |
| bge-small-en-v1.5 | `embedding(model="xinference/bge-small-en-v1.5", input)` |
| bge-small-zh | `embedding(model="xinference/bge-small-zh", input)` |
| bge-small-zh-v1.5 | `embedding(model="xinference/bge-small-zh-v1.5", input)` |
| e5-large-v2 | `embedding(model="xinference/e5-large-v2", input)` |
| gte-base | `embedding(model="xinference/gte-base", input)` |
| gte-large | `embedding(model="xinference/gte-large", input)` |
| jina-embeddings-v2-base-en | `embedding(model="xinference/jina-embeddings-v2-base-en", input)` |
| jina-embeddings-v2-small-en | `embedding(model="xinference/jina-embeddings-v2-small-en", input)` |
| multilingual-e5-large | `embedding(model="xinference/multilingual-e5-large", input)` |

1
docs/my-website/sidebars.js
View file

@ -75,6 +75,7 @@ const sidebars = {
"providers/anyscale", "providers/anyscale",
"providers/perplexity", "providers/perplexity",
"providers/vllm", "providers/vllm",
"providers/xinference",
"providers/cloudflare_workers", "providers/cloudflare_workers",
"providers/deepinfra", "providers/deepinfra",
"providers/ai21", "providers/ai21",