Support data: in URL for memory. Add ootb support for pdfs (#67)

* support data: in URL for memory. Add ootb support for pdfs

* moved utility to common and updated data_url parsing logic

---------

Co-authored-by: Hardik Shah <hjshah@fb.com>
This commit is contained in:
Hardik Shah 2024-09-12 13:00:21 -07:00 committed by GitHub
parent 736092f6bc
commit 1d0e91d802
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 112 additions and 12 deletions

View file

@ -10,6 +10,8 @@ from llama_toolchain.core.datatypes import * # noqa: F403
EMBEDDING_DEPS = [
"blobfile",
"chardet",
"PdfReader",
"sentence-transformers",
]