phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

adding new exception mapping details to documentation

Browse source
This commit is contained in:
Krrish Dholakia 2023-08-29 15:40:16 -07:00
parent 5bcf612724
commit d7fe4f6c90
2 changed files with 23 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

25
docs/my-website/docs/exception_mapping.md
View file

@ -1,13 +1,14 @@
# Exception Mapping
LiteLLM maps the 3 most common exceptions across all providers.
LiteLLM maps the 4 most common exceptions across all providers.
- Rate Limit Errors
- Context Window Errors
- InvalidAuth errors (key rotation stuff)
- Invalid Request Errors
- InvalidAuth Errors (incorrect key, etc.)
Base case - we return the original exception.
For all 3 cases, the exception returned inherits from the original OpenAI Exception but contains 3 additional attributes:
For all 4 cases, the exception returned inherits from the original OpenAI Exception but contains 3 additional attributes:
* status_code - the http status code of the exception
* message - the error message
* llm_provider - the provider raising the exception
@ -44,9 +45,25 @@ To see how it's implemented - [check out the code](https://github.com/BerriAI/li
| Replicate | Request was throttled | RateLimitError | 429 |
| Replicate | ReplicateError | ServiceUnavailableError | 500 |
| Cohere | invalid api token | AuthenticationError | 401 |
| Cohere | too many tokens | InvalidRequestError | 400 |
| Cohere | too many tokens | ContextWindowExceededError | 400 |
| Cohere | CohereConnectionError | RateLimitError | 429 |
| Huggingface | 401 | AuthenticationError | 401 |
| Huggingface | 400 | InvalidRequestError | 400 |
| Huggingface | 429 | RateLimitError | 429 |
| Openrouter | 413 | ContextWindowExceededError | 400 |
| Openrouter | 401 | AuthenticationError | 401 |
| Openrouter | 429 | RateLimitError | 429 |
| AI21 | Prompt has too many tokens | ContextWindowExceededError | 400 |
| AI21 | 422 | InvalidRequestError | 400 |
| AI21 | 401 | AuthenticationError | 401 |
| AI21 | 429 | RateLimitError | 429 |
| TogetherAI | inputs` tokens + `max_new_tokens` must be <= | ContextWindowExceededError | 400 |
| TogetherAI | INVALID_ARGUMENT | InvalidRequestError | 400 |
| TogetherAI | "error_type": "validation" | InvalidRequestError | 400 |
| TogetherAI | invalid private key | AuthenticationError | 401 |
| TogetherAI | 429 | RateLimitError | 429 |
The `ContextWindowExceededError` is a sub-class of `InvalidRequestError`. It was introduced to provide more granularity for exception-handling scenarios. Please refer to [this issue to learn more](https://github.com/BerriAI/litellm/issues/228).

8
litellm/utils.py
View file

@ -1332,8 +1332,6 @@ def exception_type(model, original_exception, custom_llm_provider):
# Handle the OpenAIError
exception_mapping_worked = True
if model in litellm.openrouter_models:
print(f"e: {original_exception}")
print(f"original_exception.http_status: {original_exception.http_status}")
if original_exception.http_status == 413:
raise ContextWindowExceededError(
message=str(original_exception),
@ -1422,7 +1420,7 @@ def exception_type(model, original_exception, custom_llm_provider):
)
elif "too many tokens" in error_str:
exception_mapping_worked = True
raise InvalidRequestError(
raise ContextWindowExceededError(
message=f"CohereException - {original_exception.message}",
model=model,
llm_provider="cohere",
@ -1474,7 +1472,7 @@ def exception_type(model, original_exception, custom_llm_provider):
message=f"AI21Exception - {original_exception.message}",
llm_provider="ai21",
)
if original_exception.status_code == 422 or "Prompt has too many tokens" in original_exception.message:
if original_exception.status_code == 422:
exception_mapping_worked = True
raise InvalidRequestError(
message=f"AI21Exception - {original_exception.message}",
@ -1522,8 +1520,6 @@ def exception_type(model, original_exception, custom_llm_provider):
message=f"TogetherAIException - {original_exception.message}",
llm_provider="together_ai",
)
print(f"error: {error_response}")
print(f"e: {original_exception}")
raise original_exception # base case - return the original exception
else:
raise original_exception