---
title: OpenAI API Compatibility
description: Use OpenAI clients and libraries with Llama Stack for seamless integration
sidebar_label: OpenAI Compatibility
sidebar_position: 11
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# OpenAI API Compatibility

Llama Stack provides OpenAI-compatible API endpoints, allowing you to use existing OpenAI clients and libraries with Llama Stack servers.

## Server Path

Llama Stack exposes an OpenAI-compatible API endpoint at `/v1/openai/v1`.

For a Llama Stack server running locally on port `8321`, the full URL to the OpenAI-compatible API endpoint is:

```
http://localhost:8321/v1/openai/v1
```

## Client Configuration

You can use any client that speaks OpenAI APIs with Llama Stack. We regularly test with the official Llama Stack clients as well as OpenAI's official Python client.

<Tabs>
<TabItem value="llamastack" label="Llama Stack Client">

When using the Llama Stack client, set the `base_url` to the root of your Llama Stack server. It will automatically route OpenAI-compatible requests to the right server endpoint for you.

```python
from llama_stack_client import LlamaStackClient

client = LlamaStackClient(base_url="http://localhost:8321")
```

</TabItem>
<TabItem value="openai" label="OpenAI Client">

When using an OpenAI client, set the `base_url` to the `/v1/openai/v1` path on your Llama Stack server.

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8321/v1/openai/v1",
    api_key="none"  # API key not required for local Llama Stack
)
```

</TabItem>
</Tabs>

Regardless of the client you choose, the following code examples should all work the same.

## Available APIs

### Models

Many of the APIs require you to pass in a model parameter. To see the list of models available in your Llama Stack server:

```python
models = client.models.list()
for model in models:
    print(f"Model ID: {model.id}")
```

### Responses

:::info[Development Status]
The Responses API implementation is still in active development. While it is quite usable, there are still unimplemented parts of the API. We'd love feedback on any use-cases you try that do not work to help prioritize the pieces left to implement. Please open issues in the [meta-llama/llama-stack](https://github.com/meta-llama/llama-stack) GitHub repository with details of anything that does not work.
:::

#### Simple Inference

Request:

```python
response = client.responses.create(
    model="meta-llama/Llama-3.2-3B-Instruct",
    input="Write a haiku about coding."
)

print(response.output_text)
```

**Example output:**
```text
Pixels dancing slow
Syntax whispers secrets sweet
Code's gentle silence
```

#### Structured Output

Request:

```python
response = client.responses.create(
    model="meta-llama/Llama-3.2-3B-Instruct",
    input=[
        {
            "role": "system",
            "content": "Extract the participants from the event information.",
        },
        {
            "role": "user",
            "content": "Alice and Bob are going to a science fair on Friday.",
        },
    ],
    text={
        "format": {
            "type": "json_schema",
            "name": "participants",
            "schema": {
                "type": "object",
                "properties": {
                    "participants": {"type": "array", "items": {"type": "string"}}
                },
                "required": ["participants"],
            },
        }
    },
)
print(response.output_text)
```

**Example output:**
```json
{ "participants": ["Alice", "Bob"] }
```

### Chat Completions

#### Simple Inference

Request:

```python
chat_completion = client.chat.completions.create(
    model="meta-llama/Llama-3.2-3B-Instruct",
    messages=[{"role": "user", "content": "Write a haiku about coding."}],
)

print(chat_completion.choices[0].message.content)
```

**Example output:**
```text
Lines of code unfold
Logic flows like a river
Code's gentle beauty
```

#### Structured Output

Request:

```python
chat_completion = client.chat.completions.create(
    model="meta-llama/Llama-3.2-3B-Instruct",
    messages=[
        {
            "role": "system",
            "content": "Extract the participants from the event information.",
        },
        {
            "role": "user",
            "content": "Alice and Bob are going to a science fair on Friday.",
        },
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "participants",
            "schema": {
                "type": "object",
                "properties": {
                    "participants": {"type": "array", "items": {"type": "string"}}
                },
                "required": ["participants"],
            },
        },
    },
)

print(chat_completion.choices[0].message.content)
```

**Example output:**
```json
{ "participants": ["Alice", "Bob"] }
```

#### Streaming Responses

Request:

```python
stream = client.chat.completions.create(
    model="meta-llama/Llama-3.2-3B-Instruct",
    messages=[{"role": "user", "content": "Count from 1 to 10."}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="")
```

### Completions

#### Simple Inference

Request:

```python
completion = client.completions.create(
    model="meta-llama/Llama-3.2-3B-Instruct",
    prompt="Write a haiku about coding."
)

print(completion.choices[0].text)
```

**Example output:**
```text
Lines of code unfurl
Logic whispers in the dark
Art in hidden form
```

## Migration from OpenAI

### Quick Migration Steps

1. **Update base URL**: Change from OpenAI's API endpoint to your Llama Stack server
2. **Remove API key requirement**: Local Llama Stack doesn't require authentication by default
3. **Update model names**: Use Llama Stack model identifiers
4. **Test functionality**: Verify that your existing code works as expected

### Migration Example

<Tabs>
<TabItem value="before" label="Before (OpenAI)">

```python
from openai import OpenAI

client = OpenAI(api_key="your-openai-api-key")

response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

</TabItem>
<TabItem value="after" label="After (Llama Stack)">

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8321/v1/openai/v1",
    api_key="none"  # Not required for local instance
)

response = client.chat.completions.create(
    model="meta-llama/Llama-3.2-3B-Instruct",  # Use Llama model
    messages=[{"role": "user", "content": "Hello!"}]
)
```

</TabItem>
</Tabs>

## Benefits

### 🔄 **Seamless Integration**
Use existing OpenAI-compatible code with minimal changes

### 🏠 **Local Control**
Run models locally while keeping familiar API patterns

### 📚 **Library Compatibility**
Works with existing tools and libraries built for OpenAI APIs

### 🔧 **Easy Migration**
Gradually migrate from OpenAI to Llama Stack without rewriting applications

## Limitations

While Llama Stack strives for full OpenAI compatibility, some features may have differences:

- **Model-specific capabilities**: Different models may support different features
- **Response format variations**: Some response fields may differ slightly
- **Rate limiting**: Different rate limiting behavior compared to OpenAI
- **Authentication**: Local instances may not require authentication

## Troubleshooting

### Common Issues

**Connection Errors:**
- Verify the Llama Stack server is running
- Check the base URL includes the correct path (`/v1/openai/v1`)
- Ensure the port number is correct

**Model Not Found:**
- Use `client.models.list()` to see available models
- Verify the model is loaded in your Llama Stack configuration

**API Differences:**
- Check the Llama Stack logs for detailed error information
- Refer to the [API documentation](/docs/api/) for Llama Stack-specific details

## Next Steps

1. **🚀 [Set up a Llama Stack server](/docs/distributions/)** if you haven't already
2. **📖 [Explore the full API reference](/docs/api/)** for advanced features
3. **🔧 [Configure your models](/docs/distributions/configuration)** for optimal performance
4. **💬 [Join the community](https://github.com/meta-llama/llama-stack/discussions)** for support and tips

## Related Resources

- **[Llama Stack Client](/docs/getting-started/libraries)** - Official Python client
- **[Provider Documentation](/docs/providers/)** - Understanding the provider system
- **[Configuration Guide](/docs/distributions/configuration)** - Server configuration options
