phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

Fix pkg_resources warning

Browse source 
by trying to use `importlib.resources` first and falling back to
`pkg_resources` if that fails.

With this and the changes in GH-3600 and GH-3601, the tests pass with **zero
warnings**!! 🎉 🎉

```shell
abramowi at marcs-mbp-3 in ~/Code/OpenSource/litellm (msabramo/fix-pydantic-warnings●●)
$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
====================================== test session starts ======================================
platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/abramowi/Code/OpenSource/litellm
plugins: anyio-4.3.0, mock-3.14.0
collected 12 items

litellm/tests/test_proxy_server.py s..........s                                           [100%]

================================= 10 passed, 2 skipped in 9.24s =================================
```
This commit is contained in:
Marc Abramowitz 2024-05-12 12:44:32 -07:00
parent 61143c8b45
commit bfaf8d033d
1 changed files with 9 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

21
litellm/utils.py
View file

@ -39,21 +39,18 @@ from litellm.caching import DualCache
oidc_cache = DualCache()
try:
# this works in python 3.8
import pkg_resources # type: ignore
filename = pkg_resources.resource_filename(__name__, "llms/tokenizers")
# try:
# filename = str(
# resources.files().joinpath("llms/tokenizers") # type: ignore
# ) # for python 3.8 and 3.12
except:
# this works in python 3.9+
# New and recommended way to access resources
from importlib import resources
filename = str(
resources.files(litellm).joinpath("llms/tokenizers") # for python 3.10
) # for python 3.10+
resources.files(litellm).joinpath("llms/tokenizers")
)
except ImportError:
# Old way to access resources, which setuptools deprecated some time ago
import pkg_resources # type: ignore
filename = pkg_resources.resource_filename(__name__, "llms/tokenizers")
os.environ["TIKTOKEN_CACHE_DIR"] = (
filename # use local copy of tiktoken b/c of - https://github.com/BerriAI/litellm/issues/1071
)