forked from phoenix/litellm-mirror
Litellm dev 11 11 2024 (#6693)
* fix(__init__.py): add 'watsonx_text' as mapped llm api route Fixes https://github.com/BerriAI/litellm/issues/6663 * fix(opentelemetry.py): fix passing parallel tool calls to otel Fixes https://github.com/BerriAI/litellm/issues/6677 * refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling reduces bugs in repo * fix(__init__.py): update provider-model mapping to include all known provider-model mappings Fixes https://github.com/BerriAI/litellm/issues/6669 * feat(anthropic): support passing document in llm api call * docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function * fix(factory.py): fix linting error
This commit is contained in:
parent
b8ae08b8eb
commit
f59cb46e71
21 changed files with 533 additions and 2264 deletions
|
@ -864,3 +864,96 @@ Human: How do I boil water?
|
|||
Assistant:
|
||||
```
|
||||
|
||||
|
||||
## Usage - PDF
|
||||
|
||||
Pass base64 encoded PDF files to Anthropic models using the `image_url` field.
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="sdk" label="SDK">
|
||||
|
||||
### **using base64**
|
||||
```python
|
||||
from litellm import completion, supports_pdf_input
|
||||
import base64
|
||||
import requests
|
||||
|
||||
# URL of the file
|
||||
url = "https://storage.googleapis.com/cloud-samples-data/generative-ai/pdf/2403.05530.pdf"
|
||||
|
||||
# Download the file
|
||||
response = requests.get(url)
|
||||
file_data = response.content
|
||||
|
||||
encoded_file = base64.b64encode(file_data).decode("utf-8")
|
||||
|
||||
## check if model supports pdf input - (2024/11/11) only claude-3-5-haiku-20241022 supports it
|
||||
supports_pdf_input("anthropic/claude-3-5-haiku-20241022") # True
|
||||
|
||||
response = completion(
|
||||
model="anthropic/claude-3-5-haiku-20241022",
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "You are a very professional document summarization specialist. Please summarize the given document."},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": f"data:application/pdf;base64,{encoded_file}", # 👈 PDF
|
||||
},
|
||||
],
|
||||
}
|
||||
],
|
||||
max_tokens=300,
|
||||
)
|
||||
|
||||
print(response.choices[0])
|
||||
```
|
||||
</TabItem>
|
||||
<TabItem value="proxy" lable="PROXY">
|
||||
|
||||
1. Add model to config
|
||||
|
||||
```yaml
|
||||
- model_name: claude-3-5-haiku-20241022
|
||||
litellm_params:
|
||||
model: anthropic/claude-3-5-haiku-20241022
|
||||
api_key: os.environ/ANTHROPIC_API_KEY
|
||||
```
|
||||
|
||||
2. Start Proxy
|
||||
|
||||
```
|
||||
litellm --config /path/to/config.yaml
|
||||
```
|
||||
|
||||
3. Test it!
|
||||
|
||||
```bash
|
||||
curl http://0.0.0.0:4000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer <YOUR-LITELLM-KEY>" \
|
||||
-d '{
|
||||
"model": "claude-3-5-haiku-20241022",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "You are a very professional document summarization specialist. Please summarize the given document"
|
||||
},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": "data:application/pdf;base64,{encoded_file}" # 👈 PDF
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"max_tokens": 300
|
||||
}'
|
||||
|
||||
```
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue