Perplexity AI (pplx-api)

https://www.perplexity.ai

API Key

# env variable
os.environ['PERPLEXITYAI_API_KEY']

Sample Usage

from litellm import completion
import os

os.environ['PERPLEXITYAI_API_KEY'] = ""
response = completion(
    model="perplexity/mistral-7b-instruct", 
    messages=messages
)
print(response)

Sample Usage - Streaming

from litellm import completion
import os

os.environ['PERPLEXITYAI_API_KEY'] = ""
response = completion(
    model="perplexity/mistral-7b-instruct", 
    messages=messages,
    stream=True
)

for chunk in response:
    print(chunk)

Supported Models

All models listed here https://docs.perplexity.ai/docs/model-cards are supported. Just do model=perplexity/<model-name>.

Model Name	Function Call
pplx-7b-chat	`completion(model="perplexity/pplx-7b-chat", messages)`
pplx-70b-chat	`completion(model="perplexity/pplx-70b-chat", messages)`
pplx-7b-online	`completion(model="perplexity/pplx-7b-online", messages)`
pplx-70b-online	`completion(model="perplexity/pplx-70b-online", messages)`
codellama-34b-instruct	`completion(model="perplexity/codellama-34b-instruct", messages)`
llama-2-13b-chat	`completion(model="perplexity/llama-2-13b-chat", messages)`
llama-2-70b-chat	`completion(model="perplexity/llama-2-70b-chat", messages)`
mistral-7b-instruct	`completion(model="perplexity/mistral-7b-instruct", messages)`
openhermes-2-mistral-7b	`completion(model="perplexity/openhermes-2-mistral-7b", messages)`
openhermes-2.5-mistral-7b	`completion(model="perplexity/openhermes-2.5-mistral-7b", messages)`
pplx-7b-chat-alpha	`completion(model="perplexity/pplx-7b-chat-alpha", messages)`
pplx-70b-chat-alpha	`completion(model="perplexity/pplx-70b-chat-alpha", messages)`

Return citations

Perplexity supports returning citations via return_citations=True. Perplexity Docs. Note: Perplexity has this feature in closed beta, so you need them to grant you access to get citations from their API.

If perplexity returns citations, LiteLLM will pass it straight through.

:::info

For passing more provider-specific, go here :::

from litellm import completion
import os

os.environ['PERPLEXITYAI_API_KEY'] = ""
response = completion(
    model="perplexity/mistral-7b-instruct", 
    messages=messages,
    return_citations=True
)
print(response)

Add perplexity to config.yaml

model_list:
  - model_name: "perplexity-model"
    litellm_params:
      model: "llama-3.1-sonar-small-128k-online"
      api_key: os.environ/PERPLEXITY_API_KEY

Start proxy

litellm --config /path/to/config.yaml

Test it!

curl -L -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
    "model": "perplexity-model",
    "messages": [
      {
        "role": "user",
        "content": "Who won the world cup in 2022?"
      }
    ],
    "return_citations": true
}'

Call w/ OpenAI SDK, Langchain, Instructor, etc.

3.7 KiB Raw Blame History